Side effects or not, aliasing kills you

My programming personality is very depressed right now. For a while, I've been away from my usual long-term stuff and back in the trenches. I'm doing simple things, but there's lots of them, so it feels like I'm not moving anywhere even though I go as fast as I can. It's like a highway with monotonic view.

Well, this has completely drained my brain. So I won't tell you anything in this blog – instead, I'll ask you a question. This way, it's OK for me to be wrong, which is even more likely than usual in my current state. Clever trick, isn't it?

My question basically is, how is not having side effects in your programming language going to help you write correct, and optimal, and parallel programs? I've already mentioned it, but this time, I'll elaborate on the point that I fail to get.

The problem with parallelism is scheduling. We want scheduling to be:

  1. Correct: when A depends on B, B should happen before A.
  2. Optimal: A, B and everything else should complete as soon as possible.

So, in order to correctly schedule basic blocks of code called A and B, I need to know if they depend on each other. If I have side effects, this is going to be hard. A modifies an object called X, and B modifies Y. Can X and Y be the same thing? To figure it out, you need whole-program analysis, and if your language is Turing-complete, WPA won't necessarily succeed, either.

This is the aliasing problem. The aliasing problem rules. It's the problem that forces you to use restrict to optimize numeric code. It's the problem that forces you to use static typing if you want to optimize anything at all. And it's the problem that makes all safe languages completely and wildly unsafe when you have multiple threads. The rule of thumb is: if you have a good idea for compile-time or run-time anything, it won't work because of the aliasing problem.

Now, suppose we got rid of those pesky side effects. A doesn't modify X; instead, it creates a new object, Z. Similarly, B takes Y and makes W. Now we can prove that A doesn't depend on B. As soon as their inputs are ready, they can run. This takes care of the correctness problem, which was listed as the scheduling problem #1 at the beginning.

However, this didn't solve the aliasing problem – X and Y can still be the same. I suspect that the aliasing problem is going to raise its ugly head and kick our pretty butt. Specifically, while semantically we made Z from X, we'd really like to wipe out X and reuse its memory for hosting Z. Because that would be more efficient than copying X and modifying some of it and keeping two objects around. Now, we can't do this optimization if X and Y are the same thing, and we know nothing about the execution order of A and B, can we? If A replaced X with Z, B would get the wrong Y.

And if we copy things, we aren't going to get an optimal scheduling, because A and B now run slower – problem #2, right?

Seriously, what do I miss? I realize that there are parallel execution environments where you have to copy things anyway because you have a zillion processors and a zillion processors can't have shared memory. I also realize that you can have few objects shared between tasks that run in parallel, so the overhead of copying them isn't very important. I realize all kinds of things making the aliasing problem a non-problem for side-effect-free parallelization frameworks or languages or whatever.

What I fail to see is, if you run in a single-box, multi-core system, and you do have shared memory, and you do have lots of shared data, how does removing side effects help you? With side effects, you can get an efficient program, if you can handle the debugging. Without side effects, you get a correct program, and then you must somehow optimize object allocation. And I just don't see how you'd go about that without compromising correctness.

Maybe I'm ignorant. Maybe I should have read about this somewhere. But where do I find a levelheaded discussion of, say, common implementation techniques of pure functional language compilers? I just keep bumping into "sufficiently smart compiler" arguments, and then there are people who don't care much about performance (and shouldn't because of their application domain).

Compare this to imperative languages. Anybody can write a pretty decent C compiler. If you don't optimize at all, you get what, 4x slowdown compared to the state of the art optimizer? And on a scalar RISC machine, I bet you can bridge 50%-70% of that gap by your cheapest shot at register allocation. A no-brainer, you need about 30% of a comp sci degree and you're all set. Now, how am I supposed to optimize allocation if there are no side effects? I mean, your spec or your family of specs should come with implementation techniques. "Useful" is good by itself, but it must also be implementable.

By the way, I'd love to find out that I'm wrong on this one. I've done lots of parallelism-related debugging, and it sucks. And maybe I am wrong. Maybe there just isn't much talk about the implementation details of these things. I do think my reasoning makes sense though.

106 comments ↓

#1 Doug on 03.21.08 at 6:54 pm

What language are you talking about? C?

Using immutable (side-effect free) aliased data in a language with automatic garbage collection is quite straightforward. As long as any thread holds a reference (pointer) to the data, it stays allocated. When no threads hold references to the data, the memory can be reused.

Also, as far as making modified "copies", it's not necessary to actually copy. You can also use a "wrapper" object that dynamically applies the change(s) whenever the underlying data is accessed.

All of the "pure functional languages" that I know of use immutable (side-effect free) data with automatic garbage collection. It's also usually pretty easy to create those wrapper objects.

If automatic garbage collection is unacceptable for your application, you might want to look at the ADA language. It specifically restricts how aliasing can be used.

In the end, you can't have everything. Something will be compromised, whether it's efficiency, debugability, simplicity, reliability, or whatever. If there was a way that we could somehow "have it all", we would simply do things that way all of the time.

#2 Yossi Kreinin on 03.22.08 at 1:47 am

I was talking about a hypothetic language that was supposed to be efficient and guarantee correct parallelization, something I'd need to write a compiler for.

Suppose you can have automatic garbage collection. Now you enter A which makes Z from X. In order to create Z where X used to be, you have to know whether X is still used *right now*. Since you can't look at a ref count, which would break in presence of cyclic references, and garbage collection only runs periodically, you are likely to lack the "confidence" needed to dispose of X when you need to do that.

As to wrapper objects, that seems to add a level of memory indirection, which can be nasty performance-wise.

Intuitively, I think about converting programs with side effects into programs without them. Imperative programs frequently create large objects and then modify them; those modifications would look like copying most of the object if translated straightforwardly to a functional style. It could be that "native" functional programs look very different and so the real optimization problems are also different.

Regarding "have it all": I couldn't agree more :)

#3 cgibbard on 03.22.08 at 4:26 pm

Well, functional programs really do look rather different from typical imperative programs. There are special techniques for getting performance from immutable datastructures where you can be certain that once computed, the data will remain the same forever. These mostly have to do with sharing parts of datastructures which are really the same. For instance, you might have a set which is represented by some balanced binary tree structure. As opposed to making a modified wholesale copy of that tree, the function which, say, adds an element to that set will only generate log(n) new tree nodes — the path from the new node up to the root. All the other subtrees will simply be shared. In the case of mutable datastructures, this would be a bad policy, since modifications to the original set would then be strangely and perhaps incoherently reflected in the new one. It's the immutability which allows you to do it.

In general, in pure value-oriented languages, when you make a change to just some small part of a large datastructure, you generally only need to copy a small part of it (ideally O(log(n)) assuming that you've selected the right datastructure for your access pattern), and everything else just gets referred to.

There are also techniques for ensuring that datastructures are not constructed at all, by fusing the code which produces the datastructure with the code that consumes it, generally referred to as deforestation. If you have one producer and multiple consumers, there are of course limits on what you can do. Uniqueness typing is a way to get the compiler to enforce that a value is only ever used once, so that after that happens, the space used for it can be reclaimed and used for something else. Even without that though, the techniques for dealing with many short-lived objects are getting quite good for most purposes. This is typically to the extent that we hardly worry about it in most cases. You don't really think in terms of one thing getting mutated over and over as much as just what the value you're interested in is, and what other values might be useful in order to construct it. It's only once you have an explicit problem with space, and profiling turns up that you need to be careful with the lifetime of values that you start to worry about retention and making sure that things can be reclaimed earlier.

#4 Doug on 03.22.08 at 8:50 pm

You don't need to know whether X is still used *right now*. All you need to know is that the memory being used for your new object is no longer in use for anything else. How long ago it quit being used for anything else is unimportant. We don't need to "dispose of X" the instant that it is no longer in use – disposal can wait.

Most automatic garbage collection systems keep a pool of unused memory. New objects are created from that pool. The garbage collection system regularly sweeps through the data and finds unused objects and adds the associated memory to the pool. An unused object might sit around for a long time before the garbage collector realizes that the object is no longer being used. Then the reclaimed memory might sit around in the pool a long time before it is actually reused. No big deal.

It sounds to me like you're worrying about inefficiencies that are inconsequential for most programming tasks. "A level of memory indirection" is measured in the nanoseconds these days, and hardly qualifies as "nasty performance-wise" in the overwhelming majority of software domains.

For the few domains where this kind of nanosecond nitpicking is necessary, functional programming would never do. Today's multi-threaded language of choice for those domains is probably ADA.

In addition to ADA, you might want to look at Occam for ideas.

#5 cgibbard on 03.22.08 at 9:50 pm

Interestingly enough, when nanoseconds *do* matter, a very nice tactic is to use a functional language or some other high level language to construct assembly or other low-level code. Such programs can do lots of work to try different implementations of the task you're interested in, which someone coding directly in assembly would never be able to look through, and since you have domain specific knowledge of the problem, you can make use of algebraic properties which ordinary compilers don't know about.

A good example of this is FFTW (the Fastest Fourier Transform in the West), which is an O'Caml program that generates C programs to compute discrete Fourier transforms.

A more general approach to this sort of thing is being worked on in the Coconut project at McMaster University. (You can watch a Google video about that here: http://uk.youtube.com/watch?v=yHd0u6zuWdw)

They're designing a very high level Haskell DSL for automatically constructing efficient SIMD and SMP code for signal processing tasks. On some tasks they're doing 4x better than the best hand-optimised programs written in C with inline assembly, because their tool can search a broad space of possible implementations and make use of algebraic properties of the problem at hand while doing it.

#6 orib on 03.23.08 at 4:48 am

You can do away with pointers. APL, K and J are functional vector languages that do not allow pointers, thus making reference count authoritative with respect to safety of changing memory (just one reference means it's ok to change). I think Tcl does the same but I'm not sure.

Of course, this only happens at the language level — if you manufacture pointers yourself, e.g. by using array coordinates of a global variable indirectly, you're still in the same place. However, being (mostly) functional, 99% of the code written does not do any such thing, it is also parallelizable.

If you're not familiar with J, K or APL, you should — you'll probably like them. They do away with a lot of the unneeded cruft we've come to accept as necessary evil when programming. There's a steep learning curve, and the syntax is not newbie friendly, but the reward is programs that are 10-100 times shorter than their C equivalent (usually closer to the 100 part of the scale), while usually — at least K — comparable in speed.

APL was invented as an "algorithm notation language" for paper back in the fifties, and then implemented in the 60's when someone noticed that it's well defined enough to execute. J and K are two descendants, the former focusing on completeness and mathematical elegance (IMHO, failed), and the later focusing on speed, terseness and with a bias towards financial applications.

#7 Yossi Kreinin on 03.24.08 at 1:17 pm

To Doug: I am worried about the inefficiency of the delayed reclaiming and all the "right now" business because I'm anal-retentive about performance – not in general, but in the specific context of real time embedded machine vision apps, which is the context where this entry is supposed to belong. I realize that your typical functional programming application domain is different. Thanks for the pointer on aliasing restriction in Ada – I'll have a look.

To cgibbard: Thanks for the link; I love code generation. I do want to somehow eliminate side-effect/parallelism problems at the target machine though; it's a separate story.

To orib: I think that vector languages like APL are the way to go if you do data parallelism, but I'm not sure about task parallelism, which is what I have in mind. I'd like to look at K nonetheless; the 100x part sounds too good to be true…

#8 orib on 03.24.08 at 5:11 pm

I know it sounds too good to be true. And to get there, it does take dropping all the useless things we studied in the last 20 years or so of programming.

qn:{[n],/{:[n=#*x;,*x;,/_f'f x,:/:(!n)_dvl,/x]}'(f:0 -1 1+/:)@!n}
bd:{[p]`0:".Q"p=:!#p}

Yep, I know it looks unreadable, but it is not obfuscated, just very terse (did I mention the learning curve is steep?)

The first line "qn: …" defines a function of one variable, n, that solves the n-queen problem. The second line "bd:…" defines a function that prints a solution produced by the first line. http://nsl.com/papers/qn.htm for more.

A sudoku solver:
p,:3/:_(p:9:!81)%3
s:{*(,x)(,/{@[x;y;:;]'&21=x[&|/p[;y]=p]?!10}')/&~x}

That's actually idiomatic, NOT obfuscated. You'll have to trust me on this.

A function to (efficiently!) compute maximum running sum inside a list of numbers is:
|/0(0|+)

(that is: find the maximal sum between indices i and j for all possible indices j>i; This would be the whole vector if all numbers are nonnegative, but not if there are negative numbers).

A function to compute an average of a vector is {(+/x)%#x} read "sum over x divided by count of x". It is so straightforward that you see it inline even though it's in the standard library (as "avg" — the name "average" is as long as the function definition and lacks the semantics that the function itself carries)

A scalar product function is written {+/x*y} or just +/* and to turn it into a a matrix product, {+/*:/:[x;+y]}

A small database with a concept of order (more than relational), aggregation, etc implemented in 14 short lines can be found in http://nsl.com/k/t.k

K takes the APL approach to a radical place: Use ~60 basic functions, most of which aren't trivial, but which are mostly orthogonal and well chosen — apply them to very basic data types that include various atoms and vectors — and you find out that you don't need as much abstraction as all languages strive to present.

Now, having an APL heritage, those 60 functions are mapped onto ascii symbols, with each symbol getting one function as an unary op and another as a binary op (e.g., binary * is multiply, unary * is "first" — rhymes with pointer use in C). But there's a syntactic sugar layer on K (called "q") which makes everything into English, making "#&~~':" into the more readable "count where not eachprev match"

Being so terse, it's possible for the eyes to learn to pick patterns (like "average" above) instantaneously, and scan the whole database implementation on the screen at once.

Now, the other side is that APL and K are actually functional languages, thus they are also well suited for task parallelism. Specifically, K implements a no-pointer reference counting scheme, which ensures that all memory is reclaimed exactly when possible (although temporaries aren't efficiently handled in the current implementation). Running 4 functions simultaneously is as easy as writing .:': (f1;f2;f3;f4)

I am in no way affiliated with the people who make K. But my programming style has changed significantly since I was introduced to it — contributing to shorter programs, which are usually faster, and less prone to bugs, although I must admit less welcoming to a new maintainer.

#9 Yossi Kreinin on 03.26.08 at 12:26 pm

I ought to take a closer look at APL just to get a feeling how they do this… I can't use it as a basis for a DSL because I write DSLs for other people to use, my target audience normally being purposefully unsophisticated about their syntactic preference and just wishing for their usual style to compile (in other words, "not language geeks").

It's very easy to intrigue me with an opportunity to make code more compact though. In particular, I used to be intrigued by Forth, and to an extent, I still am. Examples of compact Forth programs: http://www.jwdt.com/~paysan/screenful.html

I currently believe that Forth cheats :) That is, the terseness comes at a price I wouldn't like to pay; a typical longer program in a more popular style doing the same thing would have features making it easier to deal with. For example, the Forth object system in the above-mentioned page isn't a substitute for a typical "object system" outside of Forth.

#10 kragen on 04.24.08 at 4:36 am

Simon Peyton Jones has put his book on implementing pure functional programming languages online. I found it very interesting to read, but I don't remember the title.

I think that if you want to get noticeable parallel speedups on multicore processors, you may have to give up on update-in-place of shared data structures. One way to do this is to take the Tcl approach of never having shared objects, ever, anywhere; this is called "linearity" or "linear logic". It would be a shame if you had to give up on update-in-place, because it pretty much rules out any kind of combinator-graph reduction machine. Somebody (Bawden? I forget) did a dissertation in the 80s on distributed linear graph reduction, though.

I think your title is a little misleading. The level at which aliasing causes problems is exactly the level at which side effects are involved.

#11 Yossi Kreinin on 04.24.08 at 11:26 am

"never having shared objects, ever, anywhere"

No can do. I ain't at no datacenter where you can have a 5x slowdown throughout the language/system and it's nothing as long as it allows to reasonably utilize the gazillion of processors you have. I'm talking about 10-20 core chips for real time embedded stuff, for example, where a 5x slowdown means you now have 2-4 cores, which is totally dumb. So I need the speed of C minus 10% overhead, tops. Can't use message passing to copy large objects, for example.

#12 kragen on 04.25.08 at 5:27 pm

Well, the whole linear-logic thing always sounded pretty crazy to me too, but apparently Henry Baker and the other guys who did this got pretty good benchmarks using it back in the 80s, and e.g. Wouter van Oortmerssen is still enthusiastic about the idea today. Sometimes they used reference-counting to make language-level "copies" actually happen lazily.

But I don't know if you can get within 10% of C speed with linear languages. Certainly they haven't hit the mainstream in the 25 years since they were invented.

How many 6502s can you fit in the transistor budget of a Xeon?

#13 jamii on 06.08.08 at 4:00 am

You might find Haskells nested data parallelism to be more interesting for image processing and similar style tasks. Its very much a work in progress but they've already shown some impressive benchmarks.

The basic idea is that functions are written as a combination of stream-like operators and array comprehensions on arrays of any type. The compiler then transforms this into operations on flat arrays with primitive types which can then be split across multiple processors. Stream fusion (http://lambda-the-ultimate.org/node/2192) is used to remove intermediate arrays.

One of the accepted summer of code projects for this year is a physics engine built using ndp which should be an interesting demonstration of the method.

#14 Yossi Kreinin on 06.09.08 at 4:09 am

Interesting. I somehow have the feeling that the SIMD instruction selection problem, the one that inflicts intrinsics upon us on all SIMD targets, isn't going to be attacked here. Distributing to multiple cores, and perhaps some forms of tiling for cache optimization, are more likely to work out – Intel are doing this with IPP in C.

#15 Alaric Snell-Pym on 09.14.11 at 4:36 am

You don't need to abandon sharing to use linear logic.

In Concurrent Clean, references may be linear (in which case the type system ensures they are the sole reference to whatever it refers to) or no.

If you mutate something you have a linear reference to, it's mutated in place and the same reference is returned.

If you mutate something you have a nonlinear reference to, it's copied afresh and the copy mutated, and a new (linear, as you just made it and haven't shared it) reference is returned, so subsequent mutations will be "fast".

(More or less. Some artistic licence was applied in that description.)

#16 Yossi Kreinin on 09.27.11 at 11:15 pm

Why is it called "linear", and what happens when two distinct copies of the same thing are created and modified as you described?

#17 calfre2020 on 07.26.18 at 1:17 pm

Thanks for sharing this Informative content. Well explained. Got to learn new things from your Blog

#18 premium proxies on 03.24.19 at 10:06 am

I’m not that much of a internet reader
to be honest but your sitds really nice,keep it up!

I'll go ahead and bookmark your site to come back down the road.
Many thanks

#19 Marissa Montrella on 05.14.19 at 4:11 pm

yosefk.com is an excellent read. I just forwarded this on 5/14/2019 to a fellow student who has been doing some work of her own on this topic. To say thanks, he just bought me a drink! So, I should probably say: Cheers for the drink!

#20 red piter on 05.15.19 at 12:59 pm

stays on topic and states valid points. Thank you.

#21 free fire hack version unlimited diamond on 05.21.19 at 3:30 pm

I must say, as a lot as I enjoyed reading what you had to say, I couldnt help but lose interest after a while.

#22 krunker hacks on 05.23.19 at 5:39 am

Enjoyed reading through this, very good stuff, thankyou .

#23 bitcoin adder v.1.3.00 free download on 05.23.19 at 9:16 am

yahoo bring me here. Cheers!

#24 eternity.cc v9 on 05.24.19 at 6:48 am

I have interest in this, xexe.

#25 ispoofer pogo activate seriale on 05.24.19 at 5:09 pm

Me enjoying, will read more. Cheers!

#26 how to get help in windows 10 on 05.29.19 at 8:08 am

Hey there! This is my first comment here so I just wanted
to give a quick shout out and say I really enjoy reading through your posts.

Can you recommend any other blogs/websites/forums that go over the same subjects?
Thank you!

#27 vn hax on 05.30.19 at 5:25 am

This is cool!

#28 how to get help in windows 10 on 05.31.19 at 6:15 am

Very nice post. I just stumbled upon your blog and wanted to
say that I have truly enjoyed surfing around
your blog posts. In any case I will be subscribing to your rss feed and I
hope you write again very soon!

#29 xbox one mods free download on 05.31.19 at 11:58 am

Your article has proven useful to me.

#30 gamefly free trial on 06.02.19 at 6:41 pm

I'm curious to find out what blog system you're working with?
I'm having some minor security problems with my latest blog and I would like
to find something more safeguarded. Do you have any solutions?

#31 gamefly free trial on 06.03.19 at 10:07 pm

Hello! I know this is kind of off topic but I was wondering if you knew where I could get a captcha plugin for my
comment form? I'm using the same blog platform as yours and I'm
having problems finding one? Thanks a lot!

#32 gamefly free trial on 06.05.19 at 5:25 pm

Howdy very cool web site!! Guy .. Excellent .. Wonderful ..
I will bookmark your blog and take the feeds also?
I'm glad to find numerous helpful information right here in the put up, we need
work out more strategies in this regard, thanks for sharing.
. . . . .

#33 gamefly free trial on 06.05.19 at 11:06 pm

Amazing blog! Do you have any tips and hints for aspiring writers?

I'm planning to start my own website soon but I'm a little lost on everything.
Would you recommend starting with a free platform like WordPress or go for
a paid option? There are so many choices out there that I'm completely overwhelmed ..
Any recommendations? Bless you!

#34 gamefly free trial on 06.06.19 at 7:27 pm

Good way of telling, and nice article to obtain facts on the topic of my presentation subject matter, which
i am going to deliver in academy.

#35 gamefly free trial on 06.07.19 at 1:38 am

hello there and thank you for your info – I've certainly picked up
something new from right here. I did however expertise several technical
points using this web site, since I experienced to reload the website many times previous to
I could get it to load properly. I had been wondering if your web host is OK?
Not that I'm complaining, but slow loading instances times will often affect your placement in google and could damage your quality score if ads and marketing
with Adwords. Anyway I'm adding this RSS to my email and could look out for much more of your
respective exciting content. Ensure that you update this again soon.

#36 gamefly free trial on 06.07.19 at 11:35 pm

Hi, Neat post. There's a problem along with your site
in internet explorer, might test this? IE still is the marketplace leader and
a huge component of other people will pass over your great writing
due to this problem.

#37 ps4 best games ever made 2019 on 06.12.19 at 1:09 am

Thanks to my father who informed me regarding this web site, this website is genuinely remarkable.

#38 playstation 4 best games ever made 2019 on 06.12.19 at 10:33 pm

Aw, this was an incredibly nice post. Taking the time and actual effort to produce a really good article… but what can I say… I put things off a
whole lot and never manage to get nearly anything done.

#39 quest bars cheap on 06.14.19 at 5:40 pm

If you desire to grow your know-how simply keep visiting this website and be updated with the most up-to-date information posted here.

#40 quest bars cheap on 06.15.19 at 2:57 am

I'm really enjoying the design and layout of your blog.
It's a very easy on the eyes which makes it much
more enjoyable for me to come here and visit more often. Did you hire out a developer to create your theme?
Great work!

#41 quest bars on 06.16.19 at 4:41 pm

Hello there, I found your website by the use
of Google whilst looking for a related subject, your website got here up,
it looks great. I have bookmarked it in my google bookmarks.

Hello there, just turned into aware of your blog via Google, and found that it's truly
informative. I'm going to watch out for brussels.
I'll appreciate should you proceed this in future. Many other
folks shall be benefited out of your writing. Cheers!

#42 chaturbate tokens hack generator 2018 pc on 06.16.19 at 5:06 pm

You got yourself a new follower.

#43 proxo key generator on 06.19.19 at 7:23 am

Thanks for this post. I definitely agree with what you are saying.

#44 vn hax on 06.20.19 at 4:30 pm

Ha, here from bing, this is what i was searching for.

#45 nonsense diamond key generator on 06.21.19 at 5:40 am

I must say got into this article. I found it to be interesting and loaded with unique points of interest.

#46 quest bars cheap on 06.23.19 at 2:51 pm

Hmm it seems like your website ate my first comment
(it was super long) so I guess I'll just sum it
up what I submitted and say, I'm thoroughly enjoying your blog.
I as well am an aspiring blog writer but I'm still new
to everything. Do you have any points for rookie blog writers?
I'd certainly appreciate it.

#47 star valor cheats on 06.23.19 at 3:16 pm

Just wanna input on few general things, The website layout is perfect, the articles is very superb : D.

#48 gmod hacks on 06.24.19 at 1:23 pm

Awesome, this is what I was looking for in bing

#49 how do we KNOW on 06.25.19 at 3:04 am

This does interest me

#50 geometry dash 2.11 download pc on 06.25.19 at 6:03 pm

Intresting, will come back here again.

#51 skisploit on 06.26.19 at 4:42 am

I am not rattling great with English but I get hold this really easygoing to read .

#52 ispoofer on 06.27.19 at 4:18 am

Thanks for this article. I definitely agree with what you are saying.

#53 synapse x serial key on 06.27.19 at 7:01 pm

I truly enjoy looking through on this web site , it holds superb content .

#54 strucid aimbot on 06.28.19 at 5:31 am

I have interest in this, danke.

#55 advanced systemcare 11.5 serial key on 06.28.19 at 12:11 pm

I love reading through and I believe this website got some genuinely utilitarian stuff on it! .

#56 how to get help in windows 10 on 06.29.19 at 1:02 am

Hmm is anyone else experiencing problems with the pictures on this blog loading?
I'm trying to find out if its a problem on my end or if it's the blog.
Any responses would be greatly appreciated.

#57 zee 5 hack on 06.29.19 at 7:40 am

I love reading through and I believe this website got some genuinely utilitarian stuff on it! .

#58 cryptotab balance hack script v1.4 cracked by cryptechy03 on 06.29.19 at 2:02 pm

Respect to website author , some wonderful entropy.

#59 qtx roblox download on 07.01.19 at 7:45 am

I simply must tell you that you have an excellent and unique site that I really enjoyed reading.

#60 tinyurl.com on 07.01.19 at 6:27 pm

It is truly a nice and helpful piece of info. I am glad that you
just shared this useful info with us. Please stay us up to date like
this. Thank you for sharing.

#61 cheat fortnite download no virus on 07.01.19 at 6:34 pm

I kinda got into this site. I found it to be interesting and loaded with unique points of interest.

#62 escape from tarkov cheats and hacks on 07.02.19 at 6:18 am

Good Morning, glad that i found on this in yahoo. Thanks!

#63 redline v3.0 on 07.02.19 at 12:00 pm

I like this site because so much useful stuff on here : D.

#64 vn hack on 07.03.19 at 6:05 am

Hello, google lead me here, keep up great work.

#65 cyberhackid on 07.03.19 at 6:02 pm

Good Morning, happy that i found on this in bing. Thanks!

#66 vehicle simulator script on 07.04.19 at 5:58 am

I truly enjoy looking through on this web site , it holds superb content .

#67 types of seo on 07.04.19 at 1:42 pm

Parasite backlink SEO works well :)

#68 subbot on 07.04.19 at 5:46 pm

This is awesome!

#69 dego pubg hack on 07.05.19 at 5:58 am

Your article has proven useful to me.

#70 Lesangent on 07.05.19 at 1:47 pm

Cheapeast Dutasteride Best Website In Germany Pharmacy Amoxicillin Vag Itching [url=http://cthosts.net]cialis 5mg[/url] Acquistare Il Cialis Doses 750 Mg Amoxicillin

#71 tom clancy's the division hacks on 07.05.19 at 6:11 pm

I must say, as a lot as I enjoyed reading what you had to say, I couldnt help but lose interest after a while.

#72 synapse x download on 07.06.19 at 5:51 am

I have interest in this, danke.

#73 gx tool apk download on 07.06.19 at 10:15 am

This does interest me

#74 Michael Messervy on 07.06.19 at 10:18 am

The next step of the puzzle is to interpret the order of the pyramid. This is your third confidential clue. 517232125

#75 rekordbox torrent on 07.06.19 at 7:19 pm

Found this on MSN and I’m happy I did. Well written site.

#76 call of duty black ops 4 license key free download on 07.07.19 at 6:41 am

I conceive this web site holds some real superb information for everyone : D.

#77 spyhunter 5.4.2.101 crack on 07.08.19 at 6:42 am

I like this website its a master peace ! Glad I found this on google .

#78 fps unlocker download on 07.09.19 at 8:18 am

I really enjoy examining on this internet site , it has got interesting article .

#79 alissa on 07.10.19 at 3:20 am

I need more info on this can you msg me!

#80 Candice45 on 07.10.19 at 11:51 pm

I am 45 year old women and this helped me out alot!

#81 quest bars cheap on 07.11.19 at 7:14 am

hello there and thank you for your information – I've definitely picked up something new from right here.
I did however expertise a few technical issues using this website, since I experienced to reload the site many times previous to
I could get it to load properly. I had been wondering if your web host is OK?
Not that I'm complaining, but sluggish loading instances times will often affect your placement in google and can damage your quality score
if ads and marketing with Adwords. Well
I am adding this RSS to my email and could look out for much more of your respective exciting content.
Ensure that you update this again very soon.

#82 milf fuck on 07.12.19 at 12:44 am

I am 45 and a mother and I need to read this thank you.

#83 sluts live chat on 07.15.19 at 2:10 am

some great ideas this gave me!

#84 plenty of fish dating site on 07.15.19 at 4:22 pm

Very soon this web site will be famous amid all blogging
visitors, due to it's nice articles or reviews

#85 how to get help in windows 10 on 07.16.19 at 6:40 pm

Hello, this weekend is nice in favor of me, as this point in time i am
reading this enormous educational article here at my house.

#86 how to get help in windows 10 on 07.17.19 at 9:52 am

Hello There. I discovered your blog using msn. That is a really well
written article. I'll be sure to bookmark it and come
back to learn extra of your helpful information. Thank you for
the post. I'll definitely return.

#87 Lesangent on 07.17.19 at 9:06 pm

Quimico De Propecia [url=http://gemeds.com]kamagra venta sin receta[/url] Prix Levitra Belgique Buy Z Pack Over The Internet What Amoxicillin Works For

#88 viktoria_ferrera on 07.19.19 at 1:43 am

amazing content thanks

#89 buydrugsonline on 07.19.19 at 2:49 am

This blog is amazing! Thank you.

#90 plenty of fish dating site on 07.19.19 at 5:57 am

I know this website provides quality dependent content and additional material, is there any other
website which presents these information in quality?

#91 Jeffery Reinartz on 07.19.19 at 6:21 am

Dreamwalker, this clue is your next bit of info. Please message the agency at your earliest convenience. No further information until next transmission. This is broadcast #4401. Do not delete.

#92 plenty of fish dating site on 07.20.19 at 12:19 am

Wonderful, what a blog it is! This website presents valuable data to us, keep it up.

#93 how to get help in windows 10 on 07.20.19 at 11:22 am

Very energetic blog, I loved that a lot. Will there be a part 2?

#94 prodigy hacks on 07.21.19 at 1:27 pm

I simply must tell you that you have an excellent and unique site that I kinda enjoyed reading.

#95 natalielise on 07.23.19 at 1:51 am

It's really a great and useful piece of info. I am happy that you
just shared this useful information with us.

Please keep us informed like this. Thank you
for sharing. natalielise plenty of fish

#96 acidswapper on 07.23.19 at 10:38 am

Yeah bookmaking this wasn’t a risky decision outstanding post! .

#97 plenty of fish dating site on 07.23.19 at 4:04 pm

I got this web site from my buddy who told me on the topic of this site and
now this time I am visiting this web page and reading very informative articles or reviews here.

#98 date coutgar on 07.23.19 at 10:39 pm

I am 43 years old and a mother this helped me!

#99 dathe cougar on 07.24.19 at 12:07 am

I am 43 years old and a mother this helped me!

#100 date coygar on 07.24.19 at 12:18 am

I am 43 years old and a mother this helped me!

#101 plenty of fish dating site on 07.24.19 at 7:02 am

Hi to all, how is the whole thing, I think every one is getting more
from this web site, and your views are good for new visitors.

#102 forza horizon 4 license key for pc on 07.24.19 at 11:11 am

I’m impressed, I have to admit. Genuinely rarely should i encounter a weblog that’s both educative and entertaining, and let me tell you, you may have hit the nail about the head. Your idea is outstanding; the problem is an element that insufficient persons are speaking intelligently about. I am delighted we came across this during my look for something with this.

#103 skisploit on 07.25.19 at 12:26 pm

Your web has proven useful to me.

#104 plenty of fish dating site on 07.25.19 at 3:47 pm

Good article! We are linking to this great article on our website.
Keep up the good writing.

#105 smore.com on 07.26.19 at 1:06 am

Hi, Neat post. There is an issue with your web site in web explorer,
may test this? IE nonetheless is the marketplace leader and a good element of other folks will leave
out your excellent writing due to this problem. plenty of fish natalielise

#106 ezfrags on 07.26.19 at 1:29 pm

This helps. Cheers!