The overblown frequency vs cost efficiency trade-off

I've often read arguments that computing circuitry running at a high frequency is inefficient, power-wise or silicon area-wise or both. So roughly, 100 MHz is more efficient, in that you get more work done per unit of energy or area spent. And CPUs go for 1 GHz or 3 GHz because serial performance sells regardless of efficiency. But accelerators like GPUs or embedded DSPs or ISPs or codecs implemented in hardware etc. etc. – these don't need to run at a high frequency.

And I think this argument is less common now when say GPUs have caught up, and an embedded GPU might run at the same frequency as an embedded CPU. But still, I've just seen someone peddling a "neuromorphic chip" or some such, and there it was – "you need to run conventional machines at 1 GHz and it's terribly inefficient."

AFAIK the real story here is pretty simple, namely:

  1. As you increase frequency, you GAIN efficiency up to point;
  2. From that point on, you do start LOSING efficiency;
  3. That inflection point, for well-designed circuits, is much higher than people think (close to a CPU's frequency in the given manufacturing process, certainly not 10x less as people often claim);
  4. …and what fueled the myth is, accelerator makers used to be much worse at designing for high frequency than CPU makers. So marketeers together with "underdog sympathizers" have overblown the frequency vs efficiency trade-off completely out of proportions.

And below I'll detail these points; if you notice oversimplifications, please correct me (there are many conflicting goals in circuit implementation, and these goals are different across markets, so my experience might be too narrow.)

Frequency improves efficiency up to a point

What's the cost of a circuit, and how is it affected by frequency? (This section shows the happy part of the answer – the sad part is in the next section.)

  1. Silicon area. The higher the clock frequency, the more things the same circuit occupying this area does per unit of time – so you win!
  2. Leakage power – just powering up the circuit and doing nothing, not even toggling the clock signal, costs you a certain amount of energy per unit of time. Here again, the higher the frequency, the more work gets done in exchange for the same leakage power – again you win!
  3. Switching power – every time the clock signal changes its value from 0 to 1 and back, this triggers a bunch of changes to the values of other signals as dictated by the interconnection of the logic gates, flip-flops – everything making up the circuit. All this switching from 0 to 1 and back costs energy (and NOT switching does not; measure the power dissipated by a loop multiplying zeros vs a loop multiplying random data, and you'll see what I mean. This has implications for the role of software in conserving energy, but this is outside our scope here.) What's the impact of frequency on cost here? It turns out that frequency is neutral - the cost in energy is directly proportionate to the clock frequency, but so is the amount of work done.

Overall, higher frequency means spending less area and power per unit of work – the opposite of the peanut gallery's conventional wisdom.

Frequency degrades efficiency from some point

At some point, however, higher frequency does start to increase the cost of the circuit per unit of work. The reasons boil down to having to build your circuit out of physically larger elements that leak more power. Even further down the frequency-chasing path come other problems, such as having to break down your work to many more pipeline stages, spending area and power on storage for the intermediate results of these stages; and needing expensive cooling solutions for heat dissipation. So actually there are several points along the road, with the cost of extra MHz growing at each point – until you reach the physically impossible frequency for a given manufacturing process.

How do you find the point where an extra MHz isn't worth it? For synthesizable design (one created in a high-level language like Verilog and VHDL), you can synthesize it for different frequencies and you can measure the cost in area and power, and plot the results. My confidence of where I think the inflection point should be comes from looking at these plots. Of course the plot will depend on the design, bringing us to the next point.

Better-designed circuits' optimal frequency is higher

One hard part of circuit design is, you're basically making a hugely parallel system, where many parts do different things. Each part doing the same thing would be easy – they all take the same time, duh, so no bottleneck. Conversely, each part doing something else makes it really easy to create a bottleneck – and really hard to balance the parts (it's hard to tell exactly how much time a piece of work takes without trying, and there are a lot of options you could try, each breaking the work into different parts.)

You need to break the harder things into smaller pipeline stages (yes, a cost in itself as we've just said – but usually a small cost unless you target really high frequencies and so have to break everything into umpteen stages.) Pipelining is hard to get right when the pipeline stages are not truly independent, and people often recoil from it (a hardware bug is on average more likely to be catastrophically costly than somewhat crummier performance.) Simpler designs also shorten schedules, which may be better than reaching a higher frequency later.

So CPUs competing for a huge market on serial performance and (stupidly) advertised frequency, implementing a comparatively stable instruction set, justified the effort to overcome these obstacles. (Sometimes to the detriment of consumers, arguably, as say with Pentium 4 – namely, high frequency, low serial performance due to too much pipelining.)

Accelerators are different. You can to some extent compensate for poor serial performance by throwing money at the problem - add more cores. Sometimes you don't care about extra performance – if you can decode video at the peak required rate and resolution, extra performance might not win more business. Between frequency improvements and architecture improvements/implementing a huge new standard, the latter might be more worthwhile. And then the budgets are generally smaller, so you tend to design more conservatively.

So AFAIK this is why so many embedded accelerators had crummy frequencies when they started out (and they also had apologists explaining why it was a good thing). And that's why some of the accelerators caught up – basically it was never a technical limitation but an economic problem of where to spend effort, and changing circumstances caused effort to be invested into improving frequency. And that's why if you're making an accelerator core which is 3 times slower than the CPU in the same chip, my first guess is your design isn't stellar at this stage, though it might improve – if it ever has to.

P.S. I'll say it again – my perspective can be skewed; someone with different experience might point out some oversimplifications. Different process nodes and different implementation constraints mean that what's decisive in one's experience is of marginal importance in another's experience. So please do correct me if I'm wrong in your experience.

P.P.S. Theoretically, a design running at 1 GHz might be doing the exact same amount of work as a 2 GHz design – if the pipeline is 2x shorter and each stage in the 1 GHz thing does the work of 2 stages in the 2 GHz thing. In practice, the 1 GHz design will have stages doing less work, so they complete in less than 1 nanosecond (1/1GHz) and are idle during much of the cycle. And this is why you want to invest some effort to up the frequency in that design – to not have mostly-idle circuitry leaking power and using up area. But the theoretically possible perfectly balanced 1 GHz design is a valid counter-argument to all of the above, I just don't think that's what most crummy frequencies hide behind them.

Update: here's an interesting complication – Norman Yarvin's comment points to an article about near-threshold voltage research by Intel, from which it turns out that a Pentium implementation designed to operate at near-threshold voltage (at a near-2x cost in area) achieves its best energy efficiency at 100 MHz – 10x slower than its peak frequency but spending 47x less energy. The trouble is, if you want that 10x performance back, you'd need 10 such cores for an overall area increase of 20x, in return for overall energy savings of 4.7x. Other points on the graph will be less extreme (less area spent, less energy saved.)

So this makes sense when silicon area is tremendously cheaper than energy, or when there's a hard limit on how much energy you can spend but a much laxer limit on area. This is not the case most of the time, AFAIK (silicon costs a lot and then it simply takes physical space, which also costs), but it can be the case some of the time. NTV can also make sense if voltage is adjusted dynamically based on workload, and you don't need high performance most of the time, and you don't care that your peak performance is achieved at a 2x area cost as much as you're happy to be able to conserve energy tremendously when not needing the performance.

Anyway, it goes to show that it's more complicated than I stated, even if I'm right for the average design made under today's typical constraints.

146 comments ↓

#1 Norman Yarvin on 01.31.16 at 4:19 am

To get the circuit to work at a higher frequency, you often have to increase the voltage. That's where the increased switching losses come from; for those, power goes as voltage squared. Increasing the voltage also increases leakage losses, but I'm not sure how those scale.

Many CPUs these days do actually change their voltage as they change their frequency, and for exactly this reason. Transmeta, I believe, pioneered this; although they're defunct, others have picked it up.

#2 Yossi Kreinin on 01.31.16 at 8:15 am

I guess you mean that's where some of the super-linear (so cost-inefficient) increased switching losses come from (if you're increasing frequency and keeping the voltage, switching costs per unit of time also increase, but they increase proportionately to the amount of work done per unit of time so it's neutral efficiency-wise.)

And still – (1) at what frequency does it typically become necessary to increase the voltage, and (2) how much less cost-efficient is the circuit because of being able to reach a higher frequency at a higher voltage? AFAIK the answer to (1) is "pretty high" and the answer to (2) is "not much." Even when the answer to (1) is "pretty low", it means that you could beneficially make your circuit work at a higher frequency for those times where it's needed without losing much cost efficiency, and you chose not to do it because there wasn't much to gain by speeding up those rare/non-existent bursts of extraordinarily intensive, urgent work. So my main point would remain, namely, if your peak supported frequency is pretty low, it's not because supporting a higher peak frequency would result in a worse design, but because it was uneconomical given your schedule, development budget and use case. If design effort was free and everything else were kept constant, you'd probably do it.

But it is interesting how with all that said, essentially to the extent that you can lower power dissipation by lowering the frequency and voltage, you're trading silicon area for power and these are two pretty different variables (they're costs paid at different times and circumstances.) So I wonder how pronounced this effect is if you plot it – how low can you go frequency-wise and still gain something (I never experimented very much with it for various reasons – I probably would if I were in the cellphone processor market, for instance.)

#3 Yossi Kreinin on 01.31.16 at 8:44 am

One more thing is, if you're feeding off a battery and/or have trouble dissipating heat, it's beneficial to lower your frequency as much as you can lower it without the throughput falling below the threshold of acceptability – even if you can't also lower the voltage. That way, you get linear gains in switching power instead of super-linear, but in absolute terms, battery life is up and heat is down. This wouldn't be so if processors were powered down every time they finish the current bulk of work, but they aren't – in practice, waiting for the user involves a lot of non-productive switching activity and you save energy by doing this stuff slower.

The upshot is that we should see some processors in the field lowering their frequency to a much lower level than they would if all they pursued was a lower voltage.

#4 Dan Luu on 01.31.16 at 9:03 am

> So AFAIK this is why so many embedded accelerators had crummy frequencies when they started out (and they also had apologists explaining why it was a good thing). And that's why some of the accelerators caught up – basically it was never a technical limitation but an economic problem of where to spend effort, and changing circumstances caused effort to be invested into improving frequency.

This also matches my experience with non-embedded accelerators. If you're looking at (just for example) a 100x speedup, it's not so bad to target a less aggressive clock rate and take a 50x speedup with v1, which sharply reduces risk and eases schedule pressure. If that works out, then pull out all the stops for v2 or even v3.

#5 Yossi Kreinin on 01.31.16 at 11:40 am

Yeah – maybe I should have said plainly that accelerators accelerate, even if it's 50x instead of 100x; that's kinda what I meant by my vague "other architectural improvements." That's why it makes sense to leave that last, hard 2x for the next time.

#6 Norman Yarvin on 01.31.16 at 8:47 pm

Yes, "super-linear" was what I meant — or, well, I took it for granted that the question was switching losses per amount of work done, in which case it's a simple increase. As for hard numbers, I didn't have any in my head, but a search finds this report on some explorations that Intel did where they were able to make a Pentium that could run at as little as 2 milliwatts (though only at 3 MHz; the optimum was at more like 17 milliwatts and 100 MHz):

http://www.realworldtech.com/near-threshold-voltage/

#7 Yossi Kreinin on 02.01.16 at 9:35 am

Interesting! I updated the article. (I hope I got it right; I find it's really easy to be stupid about the simple things – forget a 2x here or a 10x there…)

#8 Johan Ouwerkerk on 02.06.16 at 5:09 am

There's also the fact that a lot of this hardware tend to start out as a simple 'slave' device to a master CPU. So the bottleneck is going to be I/O between the two "domains" and a 'naive' version of your faster accelerator mostly burns these extra cycles waiting for IO to complete.

Also, there's the fact that powering things down to lower clock speed/sleep mode and back up is not a free lunch either. So your higher clock speeds must be so much higher that this overhead in current draw and time is compensated for by the correspondingly greater time spent in low(er) power mode(s).

#9 Yossi Kreinin on 02.06.16 at 9:25 am

Both of these are true to some extent, though the SoCs of the last decade have far fewer communication overheads than say the CPU/GPU desktop setup which is always mentioned in these cases, and powering up/down probably doesn't take much more than ~1ms (but then of course some state might be destroyed by it that needs reinitialization, and there might be other costs.)

#10 Alex Orange on 10.27.17 at 7:47 pm

You seem to be confusing the best rate to run a given circuit at with the most efficient circuit. If you want to get from point A to point B with a car and your choices are a Honda Civic or a McLaren F1, the Civic is certainly going to get you there with less gas, but it can't go as fast as the F1. The Civic will have an optimal speed, and like your argument relative to circuits up to a certain point higher speed will give you higher efficiency. The F1's maximum efficiency speed will likely be higher than the Civic's maximum efficiency speed but it's efficiency will almost certainly be less due to it having a much larger engine then the Civic.

Similarly with circuits, a simple ripple carry adder is going to be excruciatingly slow, but also likely the lowest energy per add. A Kogge-Stone adder is going to be several times faster but will take up something like 5-6x the area and 5-6x the energy per operation. This is all talking about the architecture of the circuit (where to use an AND/NOR/NOT/XOR/etc gate). If you change the circuit type to something like dynamic gates you can speed up some more, but again at the cost of more energy. Almost universally, anything that you do in a given process to speed up an operation will burn more energy unless the original circuit was horribly designed (which they aren't).

By horribly designed I mean absolute mistakes like not using minimum length gates or building very area-inefficient gates. The differences between what's going on inside a CPU and a GPU other than process are going to be architecture and circuit type, not layout. Likely both are going to use custom layouts. The reason GPUs are "slower" is because their computations are MUCH more parallel then a CPU's. Therefore they measure their performance in GFLOPs total whereas a CPU measures its performance in GFLOPs or more often IOPs serial. CPU arithmetic circuits are therefore larger even taking speed into account whereas GPUs are tuned to fit as many operations/second into a given piece of area.

So, in conclusion, your statement of "I've often read arguments that computing circuitry running at a high frequency is inefficient, power-wise or silicon area-wise or both." would be better phrased as "…computing circuitry ***capable of*** running at a high frequency…" In which case the statement that such circuits are power and area inefficient is absolutely true.

#11 Alex Orange on 10.27.17 at 7:50 pm

P.S. By IOPs I meant integer operations/second. Just realized IOPs is I/O not integer ops/second.

#12 newz aimbot on 05.15.19 at 7:30 pm

Ha, here from yahoo, this is what i was browsing for.

#13 krunker hacks on 05.16.19 at 1:14 pm

Great article to check out, glad that duckduck led me here, Keep Up awsome job

#14 fortnite aimbot download on 05.16.19 at 5:08 pm

Enjoyed reading through this, very good stuff, thankyou .

#15 nonsensediamond on 05.17.19 at 7:23 am

I dugg some of you post as I thought they were very beneficial invaluable

#16 fallout 76 cheats on 05.17.19 at 10:48 am

I have interest in this, xexe.

#17 red dead redemption 2 digital key resale on 05.17.19 at 3:57 pm

Thank You for this.

#18 redline v3.0 on 05.17.19 at 7:02 pm

Hi, yahoo lead me here, keep up nice work.

#19 Gerald Madura on 05.18.19 at 4:31 am

I'm pleased with the way that yosefk.com deals with this sort of topic! Usually on point, sometimes polemic, consistently well-written and challenging.

#20 chaturbate hack cheat engine 2018 on 05.18.19 at 8:28 am

I conceive you have mentioned some very interesting details , appreciate it for the post.

#21 sniper fury cheats windows 10 on 05.18.19 at 3:19 pm

Hey, happy that i saw on this in bing. Thanks!

#22 mining simulator codes 2019 on 05.19.19 at 7:21 am

I’m impressed, I have to admit. Genuinely rarely should i encounter a weblog that’s both educative and entertaining, and let me tell you, you may have hit the nail about the head. Your idea is outstanding; the problem is an element that insufficient persons are speaking intelligently about. I am delighted we came across this during my look for something with this.

#23 smutstone on 05.20.19 at 12:00 pm

This is awesome!

#24 redline v3.0 on 05.21.19 at 7:31 am

Enjoyed reading through this, very good stuff, thankyou .

#25 free fire hack version unlimited diamond on 05.21.19 at 4:49 pm

Ha, here from bing, this is what i was looking for.

#26 nonsense diamond on 05.22.19 at 6:38 pm

I dugg some of you post as I thought they were very beneficial invaluable

#27 krunker hacks on 05.23.19 at 6:57 am

Thank You for this.

#28 bitcoin adder v.1.3.00 free download on 05.23.19 at 10:36 am

Great writing to see, glad that google took me here, Keep Up awsome Work

#29 vn hax on 05.23.19 at 7:19 pm

Enjoyed examining this, very good stuff, thanks .

#30 eternity.cc v9 on 05.24.19 at 8:08 am

Great article to check out, glad that google brought me here, Keep Up good job

#31 ispoofer pogo activate seriale on 05.24.19 at 6:41 pm

Appreciate it for this howling post, I am glad I observed this internet site on yahoo.

#32 cheats for hempire game on 05.26.19 at 6:47 am

Hey, google lead me here, keep up nice work.

#33 iobit uninstaller 7.5 key on 05.26.19 at 9:32 am

I like this site because so much useful stuff on here : D.

#34 smart defrag 6.2 serial key on 05.26.19 at 3:56 pm

Good, this is what I was searching for in google

#35 resetter epson l1110 on 05.26.19 at 6:42 pm

Deference to op , some superb selective information .

#36 sims 4 seasons code free on 05.27.19 at 8:00 am

This i like. Cheers!

#37 rust hacks on 05.27.19 at 8:28 pm

I like this website its a master peace ! Glad I found this on google .

#38 strucid hacks on 05.28.19 at 10:45 am

Enjoyed reading through this, very good stuff, thankyou .

#39 gamefly free trial on 05.28.19 at 1:43 pm

It is perfect time to make some plans for the long run and it is time to be happy.
I've read this submit and if I may just I wish to recommend you few
fascinating things or tips. Perhaps you can write subsequent articles relating to this article.

I want to learn even more things about it!

#40 expressvpn key on 05.28.19 at 7:48 pm

I conceive you have mentioned some very interesting details , appreciate it for the post.

#41 ispoofer key on 05.29.19 at 9:04 am

I like this site, some useful stuff on here : D.

#42 aimbot free download fortnite on 05.29.19 at 1:04 pm

Great writing to check out, glad that bing led me here, Keep Up great job

#43 redline v3.0 on 05.29.19 at 5:30 pm

Great article to see, glad that google led me here, Keep Up cool Work

#44 gamefly free trial on 05.30.19 at 1:48 am

Wow! In the end I got a blog from where I be able to actually take useful data concerning my study and knowledge.

#45 vn hax on 05.30.19 at 6:46 am

This is nice!

#46 gamefly free trial on 05.31.19 at 5:54 am

Everything is very open with a clear description of the issues.
It was really informative. Your site is extremely helpful.
Thank you for sharing!

#47 xbox one mods free download on 05.31.19 at 1:18 pm

Hello, here from yanex, me enjoyng this, i will come back soon.

#48 fortnite aimbot download on 05.31.19 at 4:01 pm

Intresting, will come back here again.

#49 gamefly free trial on 06.01.19 at 5:52 pm

bookmarked!!, I really like your web site!

#50 mpl pro on 06.01.19 at 6:48 pm

Hi, google lead me here, keep up good work.

#51 hacks counter blox script on 06.02.19 at 6:57 am

Yeah bookmaking this wasn’t a risky decision outstanding post! .

#52 gamefly free trial on 06.02.19 at 7:57 pm

Unquestionably believe that which you said. Your favorite reason appeared to be on the internet the easiest thing
to be aware of. I say to you, I definitely get
irked while people think about worries that
they just do not know about. You managed to hit the nail upon the top as well as defined out the whole thing without
having side-effects , people could take a signal.
Will probably be back to get more. Thanks

#53 roblox executor on 06.03.19 at 10:47 am

I conceive this web site holds some real superb information for everyone : D.

#54 gamefly free trial on 06.04.19 at 6:41 am

I visited various websites but the audio feature for audio songs current at this site is truly wonderful.

#55 gamefly free trial on 06.04.19 at 8:51 pm

Do you have a spam issue on this site; I also am a blogger, and I was wanting to know your situation; we have created some nice
practices and we are looking to swap solutions with others, be sure to shoot me an email
if interested.

#56 gamefly free trial on 06.07.19 at 6:41 am

For the reason that the admin of this website is working, no doubt very
rapidly it will be renowned, due to its quality contents.

#57 gamefly free trial on 06.07.19 at 12:49 pm

Nice post. I used to be checking continuously this blog and I'm inspired!

Very helpful info specifically the last section :) I handle such information a lot.
I was seeking this certain information for a long time.

Thanks and best of luck.

#58 Carlo Iacobucci on 06.07.19 at 7:06 pm

Love yosefk.com– extremely informative and lots to see!

#59 free ps4 games on 06.08.19 at 3:53 pm

Awesome post.

#60 gamefly free trial 2019 coupon on 06.10.19 at 5:20 pm

Ahaa, its good dialogue concerning this piece of writing
here at this weblog, I have read all that,
so at this time me also commenting at this place.

#61 ps4 best games ever made 2019 on 06.12.19 at 9:10 pm

I have been surfing online more than 3 hours today, yet I never found any interesting article like yours.
It is pretty worth enough for me. Personally, if all site owners and
bloggers made good content as you did, the net will be much more useful
than ever before.

#62 quest bars cheap on 06.15.19 at 4:27 am

This design is incredible! You obviously know how to keep a reader amused.
Between your wit and your videos, I was almost moved to start my
own blog (well, almost…HaHa!) Fantastic job. I really enjoyed what you had to say, and more than that, how you presented it.
Too cool!

#63 noob vs pro vs hacker vs god on 06.17.19 at 10:02 am

Ni hao, here from baidu, me enjoyng this, will come back soon.

#64 protosmasher cracked on 06.17.19 at 10:08 am

Very interesting points you have remarked, appreciate it for putting up.

#65 proxo key on 06.19.19 at 1:06 pm

I like this site because so much useful stuff on here : D.

#66 proxo key generator on 06.19.19 at 1:08 pm

I consider something really special in this site.

#67 vn hax pubg mobile on 06.20.19 at 9:42 pm

This does interest me

#68 vn hax pubg on 06.20.19 at 9:44 pm

I must say, as a lot as I enjoyed reading what you had to say, I couldnt help but lose interest after a while.

#69 nonsense diamond download on 06.21.19 at 10:46 am

Respect to website author , some wonderful entropy.

#70 nonsense diamond key generator on 06.21.19 at 10:48 am

I dugg some of you post as I thought they were very beneficial invaluable

#71 plenty of fish dating site on 06.22.19 at 6:32 am

Excellent post. I used to be checking continuously this weblog and I'm impressed!
Extremely helpful information specifically the remaining phase :) I care for such information a lot.
I was looking for this particular information for a very lengthy
time. Thanks and best of luck.

#72 quest bars cheap on 06.23.19 at 10:57 am

Hi there to all, how is all, I think every one is getting more
from this web page, and your views are good in support of
new visitors.

#73 badoo superpowers free on 06.23.19 at 8:08 pm

Enjoyed reading through this, very good stuff, thankyou .

#74 game of dice cheats on 06.23.19 at 8:10 pm

Intresting, will come back here once in a while.

#75 gx tool apk download on 06.24.19 at 6:07 pm

I conceive this web site holds some real superb information for everyone : D.

#76 gx tool pro apk download on 06.24.19 at 6:08 pm

Me like, will read more. Thanks!

#77 geometry dash 2.11 download pc on 06.25.19 at 10:48 pm

very Great post, i actually like this web site, carry on it

#78 fortnite mods on 06.25.19 at 10:49 pm

very Great post, i actually enjoyed this web site, carry on it

#79 skisploit on 06.26.19 at 9:20 am

Cheers, great stuff, I enjoying.

#80 krunker aimbot on 06.26.19 at 9:21 am

Some truly wow goodies on this web site , appreciate it for contribution.

#81 ispoofer license key on 06.27.19 at 8:29 am

Great, yahoo took me stright here. thanks btw for this. Cheers!

#82 ispoofer on 06.27.19 at 8:31 am

Good Morning, glad that i found on this in google. Thanks!

#83 synapse x serial key free on 06.27.19 at 11:24 pm

I really enjoy examining on this page , it has got cool stuff .

#84 synapse x serial key on 06.27.19 at 11:26 pm

This is good. Cheers!

#85 strucid hacks on 06.28.19 at 10:16 am

Yeah bookmaking this wasn’t a risky decision outstanding post! .

#86 advanced systemcare 11.5 serial on 06.28.19 at 3:19 pm

Awesome, this is what I was browsing for in bing

#87 advanced systemcare 11.5 key on 06.28.19 at 3:19 pm

Appreciate it for this howling post, I am glad I observed this internet site on yahoo.

#88 cryptotab hack script free download 2019 on 06.29.19 at 9:47 am

Appreciate it for this howling post, I am glad I observed this internet site on yahoo.

#89 zee 5 hack on 06.29.19 at 9:48 am

I am glad to be one of the visitors on this great website (:, appreciate it for posting .

#90 cryptotab hack script free download on 06.29.19 at 4:08 pm

Good Morning, bing lead me here, keep up good work.

#91 cryptotab script hack free on 06.29.19 at 4:08 pm

I’m impressed, I have to admit. Genuinely rarely should i encounter a weblog that’s both educative and entertaining, and let me tell you, you may have hit the nail about the head. Your idea is outstanding; the problem is an element that insufficient persons are speaking intelligently about. I am delighted we came across this during my look for something with this.

#92 asshurt injector on 07.01.19 at 11:04 am

Good Day, glad that i saw on this in google. Thanks!

#93 roblox robux hack no survey no download on 07.01.19 at 11:05 am

Ha, here from yahoo, this is what i was searching for.

#94 cheat fortnite download no virus on 07.01.19 at 9:48 pm

Ha, here from yahoo, this is what i was searching for.

#95 codes for mining simulator 2019 on 07.01.19 at 9:49 pm

Deference to op , some superb selective information .

#96 escape from tarkov cheats and hacks on 07.02.19 at 9:53 am

Hi, i really think i will be back to your site

#97 escape from tarkov cheats and hacks on 07.02.19 at 9:54 am

I consider something really special in this site.

#98 redline v3.0 on 07.02.19 at 3:00 pm

I am glad to be one of the visitors on this great website (:, appreciate it for posting .

#99 skin swapper on 07.02.19 at 3:00 pm

Your post has proven useful to me.

#100 download vnhax on 07.03.19 at 9:17 am

Hi, i really think i will be back to your site

#101 vnhax download on 07.03.19 at 9:18 am

I like, will read more. Cheers!

#102 cyberhackid on 07.03.19 at 9:15 pm

I truly enjoy looking through on this web site , it holds superb content .

#103 cyberhackid on 07.03.19 at 9:16 pm

Hey, bing lead me here, keep up nice work.

#104 prison life hacks on 07.04.19 at 9:14 am

I like this site because so much useful stuff on here : D.

#105 roblox prison life hack on 07.04.19 at 9:15 am

Very interesting points you have remarked, appreciate it for putting up.

#106 advanced seo tutorial pdf on 07.04.19 at 2:25 pm

Parasite backlink SEO works well :)

#107 subbot on 07.04.19 at 9:06 pm

Ha, here from bing, this is what i was browsing for.

#108 subbot on 07.04.19 at 9:07 pm

Thanks for this post. I definitely agree with what you are saying.

#109 open dego on 07.05.19 at 9:26 am

I’m impressed, I have to admit. Genuinely rarely should i encounter a weblog that’s both educative and entertaining, and let me tell you, you may have hit the nail about the head. Your idea is outstanding; the problem is an element that insufficient persons are speaking intelligently about. I am delighted we came across this during my look for something with this.

#110 erdas foundation 2015 on 07.05.19 at 9:42 pm

This does interest me

#111 erdas foundation 2015 on 07.05.19 at 9:43 pm

Found this on yahoo and I’m happy I did. Well written site.

#112 synapse x on 07.06.19 at 8:10 am

This is amazing!

#113 gx tool pubg uc hack on 07.06.19 at 12:17 pm

You got yourself a new follower.

#114 gx tool uc hack app download on 07.06.19 at 12:18 pm

Very interesting points you have remarked, appreciate it for putting up.

#115 rekordbox torrent on 07.07.19 at 2:35 am

I dugg some of you post as I thought they were very beneficial invaluable

#116 rekordbox torrent download on 07.07.19 at 2:37 am

Enjoyed reading through this, very good stuff, thankyou .

#117 call of duty black ops 4 licence key on 07.07.19 at 11:24 am

I consider something really special in this site.

#118 cod black ops 4 license key free on 07.07.19 at 11:26 am

Deference to op , some superb selective information .

#119 quest bars cheap 2019 coupon on 07.09.19 at 8:12 am

You really make it seem so easy with your presentation but I find this
topic to be actually something which I think I would never understand.

It seems too complex and very broad for me. I'm looking forward for your next post, I will
try to get the hang of it!

#120 roblox fps unlocker on 07.09.19 at 1:47 pm

Appreciate it for this howling post, I am glad I observed this internet site on yahoo.

#121 fps unlocker download on 07.09.19 at 1:48 pm

I am glad to be one of the visitors on this great website (:, appreciate it for posting .

#122 Franmuh on 07.13.19 at 9:08 am

Amoxicillin Shipping To China Viagra Generic Retailer [url=http://curerxfor.com]viagra online pharmacy[/url] Best Uk Online Pharnacy Order Propecia Online Consultation Cialis 45 Ans

#123 legalporno free on 07.16.19 at 12:19 am

great advice you give

#124 how to get help in windows 10 on 07.16.19 at 3:08 pm

I was more than happy to discover this great site.

I need to to thank you for ones time due to this wonderful read!!
I definitely liked every little bit of it
and I have you saved to fav to see new stuff in your blog.

#125 plenty of fish dating site on 07.18.19 at 11:49 am

Hmm is anyone else having problems with the pictures on this blog loading?
I'm trying to determine if its a problem on my end or if it's the blog.
Any suggestions would be greatly appreciated.

#126 liv_revamped on 07.19.19 at 2:02 am

you are amazing

#127 buy drug online on 07.19.19 at 3:00 am

This blog is amazing! Thank you.

#128 Tod Olejarski on 07.19.19 at 8:15 am

Skyking, Skyking, this message is your next bit of data. Feel free to message the agency at your convenience. No further information until next transmission. This is broadcast #3698. Do not delete.

#129 plenty of fish dating site on 07.19.19 at 7:06 pm

I think this is one of the most important info for me.
And i'm glad reading your article. But want to remark on few general
things, The website style is great, the articles
is really nice : D. Good job, cheers

#130 Franmuh on 07.21.19 at 2:48 pm

Que Es Cialis Levitra Doxycycline Secure Ordering Overseas Keflex And Immonium Ad [url=http://abtsam.com]viagra online pharmacy[/url] Buy Synthroid 150 Mcg

#131 how to get help in windows 10 on 07.21.19 at 3:09 pm

Hey! I know this is kind of off topic but I was wondering if
you knew where I could find a captcha plugin for my comment form?
I'm using the same blog platform as yours and I'm having trouble finding one?
Thanks a lot!

#132 [prodigy hack] on 07.21.19 at 5:37 pm

I truly enjoy looking through on this web site , it holds superb content .

#133 prodigy game files on 07.21.19 at 5:38 pm

Hi, i really think i will be back to your page

#134 natalielise on 07.22.19 at 3:41 pm

Hi, this weekend is nice in support of me, because this occasion i am reading this great educational piece of writing here at my residence.
pof natalielise

#135 plenty of fish dating site on 07.23.19 at 1:12 pm

Do you have a spam problem on this site; I also am a blogger, and I was curious about your situation; we have developed some nice procedures and we are looking to
trade techniques with others, please shoot me an email if
interested.

#136 evogame net wifi on 07.23.19 at 4:28 pm

I love reading through and I believe this website got some genuinely utilitarian stuff on it! .

#137 evogame.net/wifi on 07.23.19 at 4:29 pm

Morning, here from google, me enjoyng this, will come back again.

#138 date cougqar on 07.23.19 at 10:51 pm

I am 43 years old and a mother this helped me!

#139 date couygar on 07.23.19 at 11:33 pm

I am 43 years old and a mother this helped me!

#140 fortnite skin changer by darkshoz on 07.24.19 at 4:48 pm

I love reading through and I believe this website got some genuinely utilitarian stuff on it! .

#141 monster hunter world free key on 07.24.19 at 4:50 pm

Just wanna input on few general things, The website layout is perfect, the articles is very superb : D.

#142 ezfrags on 07.25.19 at 7:13 pm

I’m impressed, I have to admit. Genuinely rarely should i encounter a weblog that’s both educative and entertaining, and let me tell you, you may have hit the nail about the head. Your idea is outstanding; the problem is an element that insufficient persons are speaking intelligently about. I am delighted we came across this during my look for something with this.

#143 ezfrags on 07.25.19 at 7:15 pm

I dugg some of you post as I thought they were very beneficial invaluable

#144 Stepweero on 07.26.19 at 12:23 am

Viagra En Pharmacie Prix [url=http://nefoc.com]propecia comentarios[/url] Cialis Eintritt Wirkung Viagra Compresse Costo

#145 ezfrags on 07.26.19 at 8:24 pm

I am not rattling great with English but I get hold this really easygoing to read .

#146 skisploit on 07.26.19 at 8:26 pm

You got yourself a new rader.