I love globals, or Google Core Dump

The entire discussion only applies to unsafe languages, the ones that dump core. By which I mean, C. Or C++, if you're really out of luck.

If it can dump core, it will dump core, unless it decides to silently corrupt its data instead. Trust my experience of working in a multi-processor, multi-threaded, multi-programmer, multi-nightmare environment. Some C++ FQA Lite readers claimed that the fact that I deal with lots of crashes in C++ indicates that I'm a crappy programmer surrounded by similar people. Luckily, you don't really need to trust my experience, because you can trust Google's. Do this:

  1. Find a Google office near you.
  2. Visit a Google toilet.
  3. You'll find a page about software testing, with the subtitle "Debugging sucks. Testing rocks." Read it.
  4. Recover from the trauma.
  5. Realize that the chances of you being better at eliminating bugs than Google are low.
  6. Read about the AdWords multi-threaded billing server nightmare.
  7. The server was written in C++. The bug couldn't happen in a safe language. Meditate on it.
  8. Consider yourself enlightened.

This isn't the reason why this post has "Google core dump" in its title, but hopefully it's a reason for us to agree that your C/C++ program will crash, too.

I love globals

What happens when we face a core dump? Well, we need the same things you'd expect to look for in any investigation: names and addresses. Names of objects looking at which may explain what happened, their addresses to actually look at them, and type information to sensibly display them.

In C and C++, we have 3 kinds of addresses: stack, heap and global. Let's see who lives there.

Except the stack is overwritten, because it can be. Don't count on being able to see the function calls leading to the point of crash, nor the parameters and local variables of those functions. In fact, don't even count on being able to see the point of crash itself: the program counter, the link register, the frame pointer, all that stuff can contain garbage.

And the heap is overwritten, too, nearly as badly. The typical data structure used by C/C++ allocators (for example, dlmalloc) is a kind of linked list, where each memory block is prefixed with its size so you can jump to the next one. Overwrite one of these size values and you will have lost the boundaries of the chunks above that address. That's a loss of 50% of the heap objects on average, assuming uniform distribution of memory overwriting bugs across the address space of the heap.

So don't count on the stack or the heap. Your only hope is that someone has ignored the Best Practices and the finger-pointing by the more proficient colleagues, and allocated a global object. Possibly under the clever disguise of a "Singleton". Not a bad thing after all, that moronic "design pattern", because it ultimately allowed to counter cargo cult programmers' accusations of "globals are evil" with equally powerful cargo cult argument of "it's a design pattern". So people could allocate globals again.

Which is good, because a global always has an accurate name-to-address mapping, no matter what atrocity was committed by the bulk of unsafe code running on the loose. Can't overwrite a symbol table. And it has accurate type information, too. As opposed to objects you find through void*, or a base class pointer where the base class lacks virtual functions or the object vptr was overwritten, etc.

Which is why I frequently start debugging by firing an object view window on a global, or running debugger macros which read globals, etc. Of course you can fuck up a global variable to make debugging unpleasant. For example, if the variable is "static" in the C sense, you need to open the right file or function to display it, and you need the debugger front-end to understand the context, which will be especially challenging if it's a static variable in a template function (one of the best things in C++ is how neatly its new features interact with C's old ones).

Or you can stuff the global into a class or a namespace. I was never able to display globals by their qualified C++ name in, say, gdb 5. But no matter; nm <program> | grep <global> followed by p *(TypeOfGlobal*)addr always does the trick, and no attempts at obfuscating the symbol table will stop it. I still say make it a real, unashamed global to make debugging easier. If you're lucky, you'll get to piss off a couple of cargo cult followers as a nice side-effect.

Google Core Dump

A core dump is a web. Its sites are objects. It's hyperlinks are pointers. It's PageRank is a TypeRank: what's the type of this object according to the votes of the pointers stored in other objects? The spamdexing is done by pointer-like bit patterns stored in unused memory slots. The global variables are the major sites with high availability you can use as roots for the crawling.

What utilities would we like to have for this web? The usual stuff.

  • Browsers. Debugger object view window is the Firefox, and the memory view window is the Lynx. The core dump Lynx usually sucks in that it doesn't make it easy to follow pointers – can't click on a word and have the browser follow the pointer (by jumping to the memory pointed by it). No back button, either. Oh well.
  • DNS. The ability to translate variable names to raw addresses. Works very reliably for globals and passably otherwise. Works reliably for all objects in safe languages.
  • Reverse DNS. Given an address, tell me the object name. Problematic for dynamically allocated objects, although you could list the names of pointer variables leading to it (Google bombing). Works reliably for global functions and variables. For some reason, the standard addr2line program only supports functions though. Which is why I have an addr2sym program. It so happened that I have several of them, in fact. You can download one here. "Reverse DNS" is particularly useful when you find pointers somewhere in registers or memory and wonder what they could point to. In safe languages, you don't have that problem because everything is typed and so you can simply display the pointed object.
  • Google Core Dump, similar to Google Desktop or Google for the WWW. Crawl a core dump, figure out the object boundaries and types by parsing the heap linked list and the stack and looking at pointers' "votes", create an index, and allow me to query that index. Lots of work, that, some of it heuristical. And in order to get type information in C or C++, you'll have to either parse the source code (good luck doing it with C++), or parse the non-portable debug information format. But it's doable; in fact, we have it, for our particular target/debugger/allocator combo. Of course it has its glitches. Quirky and obscure enough to make open sourcing it not worth the trouble.

I really wish there was a reasonably portable and reliable Google Core Dump kind of thing. But it doesn't look like that many people care about debugging crashes at all. Most core dumps at customer sites seem to go to /dev/null, and those that can't be easily deciphered are apparently given up on until the bug manifests itself in some other way or its cause is guessed by someone.

Am I coming from a particularly weird niche where the code size is large enough and the development rapid enough to make crashes almost unavoidable, but crashes in the final product version are almost intolerable? Or do most good projects allocate everything on the stack and the heap, so with those smashed they're doomed no matter what? Or is the problem simply stinky enough to make it unattractive for a hobby project while lacking revenue potential to make a good commercial project?

Would you like this sort of thing? If you would, drop me a line. In the meanwhile, I satisfy my wish for a Google Core Dump with my perfect implementation for an embedded co-processor, the one I've poked at with Tcl commands. With 128K of memory, no dynamic allocation, and local variables effectively implemented as globals, perfect decoding is easy. I'm telling ya, globals rule.

As to my "reverse DNS" implementation:

  • I could make it more portable by parsing the output of nm --print-size. But just running nm on a 20M symbol table takes about 2 seconds. I want instantaneous output, 'cause I'm very impatient when I debug.
  • Alternatively, I could make it more portable by using a library such as bfd. But that would drag in a library such as bfd, and I had trouble with what looked like library/compiler version mismatches with bfd, whereas my ELF parsing code never had any trouble. Also, an implementation parsing ELF is more interesting as sample code because you get to see how easy to parse these formats are. So it's elfaddr2sym, not addr2sym. (It's really 32-bit-ELF-with-host-endianness-addr2sym, because I'm lazy and it covers all my targets.)
  • There's a ton of addr2sym code out there, and maybe a good addr2sym program. I just didn't find it. I have an acknowledged weakness in the wheel reinventing department.
  • Of course I don't demangle the ugly C++ names; piping to c++filt does.
  • The program is in D, because of the "instantaneous" bit, and because D is one of the best choices available today if you care about both speed and brevity. Look at this: lowerBound!("a.st_value <= b")(ssyms, addr) does a binary search for addr in the sorted ssyms array. As brief as it gets out of the box with any language and standard library, isn't it? The string is compiled statically into the instantiation of the lowerBound template; a & b are the arguments of the anonymous function represented by the string. Readable. Short. Fast. Easy to use – garbage-collected array outputs in functions like filter(), error messages to the point – that's why a decent grammar is a good thing even if you aren't the compiler writer. Looks a lot like C++, braces, static typing, everything. Thus easy to pimp in a 3GL environment, in particular, a C++ environment. You can download the Digital Mars D compiler for Linux, or wait for C++0x to solve 15% of the problems with <algorithm> by introducing worse problems.

By the way, the std.algorithm module, the one with the sort, filter, lowerBound and similar functions, is by Andrei Alexandrescu, of Modern C++ Design fame. How is it possible that his stuff in D is so yummy while his implementation of similar things in C++ is equally icky? Because C++ is to D what proper fixation is to anaesthesia. There, I bet you saw it coming.

What does "global" mean?

For the sake of completeness, I'd like to bore you with a discussion of the various aspects of globalhood, in the vanishingly small hope of this being useful in a battle against a cargo cult follower authoring a coding convention or such. In C++, "global" can mean at least 6 things:

  • Number of instances per process. A "global" is everything that's instantiated once.
  • Life cycle. A "global" is constructed before main and destroyed after main. A static variable inside a function is not "global" in this sense.
  • "Scope" in the "namespace" sense (as opposed to the life cycle sense). We have C-style file scope, class scope, function scope, and "the true global scope". And we have namespaces.
  • Storage. A "global" is assigned a link time address and stored there. In a singleton implementation calling new and assigning its output to a global pointer, the pointer is "global" in this sense but the object is not.
  • Access control. If it's in a class scope, it may be private or protected, which makes it less of a global in this fifth sense.
  • Responsibility. A global can be accessible from everywhere but only actually accessed from a couple of places. For example, you can allocate a large object instantiating lots of members near your main function and then call object methods which aren't aware that the stuff is allocated globally.

So when I share my love of globals with you, the question is which aspect of globality I mean. What I mean is this:

  1. I like global storage – link-time addresses – for everything which can be handled that way. A global pointer is better than nothing, but it can be overwritten and you will have lost the object; better allocate the entire thing globally.
  2. I like global scope, no classes, namespaces and access control keywords attached, to make symbol table look-up easier, thus making use of the global allocation.
  3. I like global life cycle – no Meyers' singletons and lazy initialization. In fact, I like trivial constructors/destructors, leaving the actual work to init/close functions called by main(). This way, you can actually control the order in which things are done and know what the dependencies are. With Meyers' singletons, the order of destruction is uncontrollable (it's the reverse order of initialization, which doesn't necessarily work). Solutions were proposed to this problem, so dreadful that I'm not going to discuss them. Just grow up, design the damned init/close sequence and be in control again. Why do people think that all major operations should be explicit except for initialization which should happen automagically when you least expect it?
  4. "Globals" in the sense of "touched by every piece of code" is the trademark style of a filthy swine. There are plenty of good reasons to use "globals"; none of them has anything to do with "globals" as in "variables nobody/everybody is responsible for".
  5. I think that everything that's instantiated once per process is a "global", and when you wrap it with scope, access control, and design patterns, you shouldn't stop calling it a global (and instead insist on "singleton", "static class member", etc.). It's still a global, and its wrapping should be evaluated by its practical virtues. Currently, I see no point in wrapping globals in anything – plain old global variables are the thing best supported by all software tools I know.

I think this can be used as "rationale" in a coding guideline, maybe in the part allowing the use of globals as an "exception". But I keep my hopes low.

127 comments ↓

#1 User links about "demangle" on iLinkShare on 01.25.09 at 9:03 am

[...] Linker>> saved by stefaanh 22 days ago3 votesProfile simuPOP>> saved by fadereu 28 days ago2 votesI love globals, or Google Core Dump>> saved by sanyaissues 41 days ago3 votesApple patches 25 flaws with latest update>> saved by fenec [...]

#2 bill on 07.03.09 at 2:52 am

The Google bug mentioned here is plain stupidity. To say that it can happen only in C++… is even worse. The same can happen in C (as you put it… any unsafe language). However, any decent C/C++ (multi-threaded… not necessarily) app developer must know… you don't use a pointer (or a boost::ref) to a stack allocated object in one thread from another. Nor can you use the same and put in a list and later de-reference it.

It is our stupidity if we do that… not the fault of C++. You learn lessons as you do serious development work. You make mistakes and correct them.

Every decent Java programmer knows that Java programs can have memory leaks and they learn it through experience. It is newbies that swallow the whole "there are no memory leaks in Java" nonsense without looking deeper.

BTW… multi-threading is hard. Not just in C/C++. You have to go the "no mutable data structures" to get it right (easily that is).

#3 Yossi Kreinin on 07.03.09 at 5:23 am

You know which part I liked best? The "…or a boost::ref" part.

#4 Nathan on 10.09.09 at 9:17 am

Want! I want that debugger! I once worked in a place where globals were used with abandon in the last sense. It was terrible. The boss man once actually said something like "Does anyone actually use those?" about a core dump.

#5 Yossi Kreinin on 10.09.09 at 9:24 am

@Nathan: I'm not sure I understood, but – are you saying that you want a program for figuring out the types and locations of live objects in a core dump, what I called Google Core Dump above?

#6 Chris on 06.30.10 at 3:08 pm

I hope my boss man reads this. My company <3 globals, "singletons" and static member variables. It seems like I am the only one at the company that gets that they are all the same thing. The only thing that changes is how it is accessed. But restricting the access still doesn't actually impose any real restrictions.

#7 Nathan on 12.02.10 at 7:26 am

Yes, I want a Google Core Dump.

#8 Yossi Kreinin on 12.02.10 at 7:37 am

Wow, that's a long pause in a conversation… well, you're the first to want it, I think; not that I have any – we had something basically working for one of our obscure platforms but it broke down due to size issues in the next revision of that platform.

#9 rarecactus on 01.28.11 at 2:09 pm

Read the Linux Kernel Style Guide. Then try writing a bit of C in that style yourself… a kernel module, perhaps.

Try this, and you will achieve enlightenment. And you'll never want to write a line of C++ again.

P.S. The best kind of globals are the ones that are static to file scope.

#10 Yossi Kreinin on 01.29.11 at 7:13 am

Um… What if I already don't want to write a line of C++ again? Looks like you're preaching to the choir, probably unintentionally.

As to static globals – theoretically great, but some software tools have trouble with those. So I prefer global visibility and a naming convention to prevent clashes.

#11 rarecactus on 01.29.11 at 3:28 pm

Personally, I read all the best C++ books — Scott Meyers, Joshuttis, etc. I learned all their recommendations by heart, and I put them into practice too.

However, it wasn't until I wrote code in the Linux kernel style that I really knew what good low-level code looked like. Just to take one feature, the 8-space tabs acted as a check on excessive nesting and overlong functions.

I realized then that C++ was just a framework that had gotten too big for its own good. The OO framework that the Linux kernel guys wrote (take a look at KObject, for example), is actually safer and saner than C++'s horrible vtable clusterfuck.

Over time, people have realized that composition of multiple objects is often a better way to go than inheritance. But C++ provides lots and lots of syntax for creating deep inheritance hierarchies, and almost none for composing objects. Consider how tedious it is to write a wrapper class or even to implement accessor functions in C++.

So to summarize: to a first approximation, C++ consists of an obsolete 1980s style OO framework, some perl-style features that make code slightly faster to write and much, much harder to read, and templates. Templates are good, but not good enough to justify the rest of the crap.

Anyway. I'm probably preaching to the choir again.

With regard to variables static to a file… I think gdb has pretty good support for those. I really like using them. Unlike private variables in C++, symbols static to a file really *are* hidden.

C.

#12 Lee Kannan on 05.14.19 at 8:44 pm

Appreciate this site – extremely user-friendly and lots to see!

#13 Shamika Kleve on 05.15.19 at 3:35 am

5/14/2019 @ 8:35:12 PM Like yosefk.com– extremely informative and a lot of stuff to see!

#14 resharper keygen on 05.15.19 at 3:48 pm

I like this website its a master peace ! Glad I found this on google .

#15 vn hax on 05.16.19 at 12:37 pm

I conceive you have mentioned some very interesting details , appreciate it for the post.

#16 fortnite aimbot download on 05.16.19 at 4:31 pm

I am not rattling great with English but I get hold this really easygoing to read .

#17 nonsense diamond key on 05.17.19 at 6:44 am

I simply must tell you that you have an excellent and unique website that I really enjoyed reading.

#18 fallout 76 hacks on 05.17.19 at 10:12 am

I am glad to be one of the visitors on this great website (:, appreciate it for posting .

#19 red dead redemption 2 digital key resale on 05.17.19 at 3:22 pm

I like this article, because so much useful stuff on here : D.

#20 redline v3.0 on 05.17.19 at 6:26 pm

I must say got into this site. I found it to be interesting and loaded with unique points of view.

#21 chaturbate hack cheat engine 2018 on 05.18.19 at 7:51 am

I must say, as a lot as I enjoyed reading what you had to say, I couldnt help but lose interest after a while.

#22 forza horizon 4 license key on 05.18.19 at 2:43 pm

Your article has proven useful to me.

#23 mining simulator 2019 on 05.19.19 at 6:43 am

I am glad to be one of the visitors on this great website (:, appreciate it for posting .

#24 smutstone on 05.20.19 at 11:22 am

I truly enjoy looking through on this web site , it holds superb content .

#25 redline v3.0 on 05.21.19 at 6:51 am

Awesome, this is what I was searching for in google

#26 free fire hack version unlimited diamond on 05.21.19 at 4:06 pm

Thank You for this.

#27 nonsense diamond on 05.22.19 at 5:56 pm

I was looking at some of your articles on this site and I believe this internet site is really instructive! Keep on posting .

#28 krunker aimbot on 05.23.19 at 6:14 am

Enjoyed reading through this, very good stuff, thankyou .

#29 bitcoin adder v.1.3.00 free download on 05.23.19 at 9:53 am

I like this website its a master peace ! Glad I found this on google .

#30 vn hax on 05.23.19 at 6:37 pm

very cool post, i actually love this web site, carry on it

#31 eternity.cc v9 on 05.24.19 at 7:25 am

I must say, as a lot as I enjoyed reading what you had to say, I couldnt help but lose interest after a while.

#32 ispoofer pogo activate seriale on 05.24.19 at 5:49 pm

Just wanna input on few general things, The website layout is perfect, the articles is very superb : D.

#33 cheats for hempire game on 05.26.19 at 6:17 am

Your website has proven useful to me.

#34 iobit uninstaller 7.5 key on 05.26.19 at 9:03 am

I have interest in this, xexe.

#35 smart defrag 6.2 serial key on 05.26.19 at 3:23 pm

Hi, glad that i saw on this in bing. Thanks!

#36 resetter epson l1110 on 05.26.19 at 5:59 pm

Awesome, this is what I was searching for in bing

#37 sims 4 seasons code free on 05.27.19 at 7:16 am

I was looking at some of your articles on this site and I believe this internet site is really instructive! Keep on posting .

#38 rust hacks on 05.27.19 at 7:50 pm

I truly enjoy looking through on this web site , it holds superb content .

#39 strucid hacks on 05.28.19 at 10:08 am

Intresting, will come back here once in a while.

#40 gamefly free trial on 05.28.19 at 4:29 pm

I was very pleased to discover this website.
I need to to thank you for ones time just for this wonderful read!!
I definitely enjoyed every little bit of it and i also have you saved as a favorite to
check out new stuff on your web site.

#41 expressvpn key on 05.28.19 at 7:11 pm

Good Morning, google lead me here, keep up great work.

#42 how to get help in windows 10 on 05.29.19 at 1:41 am

An impressive share! I have just forwarded this onto a friend who has been doing a little homework on this.
And he actually ordered me breakfast due to the fact that
I stumbled upon it for him… lol. So allow me
to reword this…. Thanks for the meal!! But yeah, thanx for spending the time to talk about this matter
here on your website.

#43 ispoofer key on 05.29.19 at 8:23 am

You got yourself a new follower.

#44 aimbot free download fortnite on 05.29.19 at 12:22 pm

Enjoyed reading through this, very good stuff, thankyou .

#45 redline v3.0 on 05.29.19 at 4:50 pm

I love reading through and I believe this website got some genuinely utilitarian stuff on it! .

#46 vn hax on 05.30.19 at 6:02 am

Very interesting points you have remarked, appreciate it for putting up.

#47 how to get help in windows 10 on 05.31.19 at 2:44 am

Good info. Lucky me I discovered your blog by accident (stumbleupon).

I have saved it for later!

#48 xbox one mods free download on 05.31.19 at 12:36 pm

Morning, here from bing, i enjoyng this, will come back soon.

#49 fortnite aimbot download on 05.31.19 at 3:20 pm

Enjoyed reading through this, very good stuff, thankyou .

#50 Reggie Dubrock on 05.31.19 at 7:22 pm

In my estimation, yosefk.com does a good job of covering subject matter like this! Even if sometimes deliberately contentious, the material posted is more often than not thoughtful and challenging.

#51 gamefly free trial on 05.31.19 at 8:28 pm

Wow, awesome weblog layout! How lengthy have you ever been running a blog for?
you make running a blog look easy. The total look of your site
is wonderful, let alone the content material!

#52 mpl pro on 06.01.19 at 6:19 pm

Some truly good stuff on this web site , appreciate it for contribution.

#53 gamefly free trial on 06.02.19 at 3:25 am

This is my first time visit at here and i am in fact pleassant to read
all at alone place.

#54 hacks counter blox script on 06.02.19 at 6:25 am

Found this on bing and I’m happy I did. Well written web.

#55 gamefly free trial on 06.03.19 at 4:33 am

Heya are using WordPress for your site platform? I'm new to the
blog world but I'm trying to get started and set up my own. Do you require
any html coding knowledge to make your own blog?
Any help would be really appreciated!

#56 roblox executor on 06.03.19 at 10:14 am

I like this page, useful stuff on here : D.

#57 gamefly free trial on 06.03.19 at 8:25 pm

Highly descriptive post, I liked that a lot.
Will there be a part 2?

#58 gamefly free trial on 06.03.19 at 9:21 pm

This information is invaluable. When can I find out more?

#59 gamefly free trial on 06.04.19 at 5:56 pm

Wonderful blog! I found it while browsing on Yahoo News. Do
you have any suggestions on how to get listed in Yahoo News?

I've been trying for a while but I never seem to get there!
Thank you

#60 gamefly free trial on 06.06.19 at 4:00 am

Hey! I just wanted to ask if you ever have any issues with hackers?
My last blog (wordpress) was hacked and I ended up losing several weeks of hard work due to no
back up. Do you have any solutions to stop hackers?

#61 Gerard Nishitani on 06.06.19 at 6:52 am

6/5/2019 @ 11:52:44 PM Love the site– very user-friendly and tons of stuff to consider!

#62 gamefly free trial on 06.06.19 at 11:27 am

Thank you for some other great post. Where else could anyone
get that kind of info in such a perfect method of writing?
I have a presentation subsequent week, and I'm at the search for such info.

#63 gamefly free trial on 06.06.19 at 11:47 am

It's hard to come by well-informed people about this topic, however, you seem like you know what you're talking about!
Thanks

#64 ps4 plus free games on 06.07.19 at 5:06 pm

Simply desire to say your article is as astounding.
The clarity on your submit is simply great
and i can assume you're a professional in this subject.

Well with your permission allow me to clutch your feed to stay updated with imminent post.
Thank you a million and please continue the enjoyable work.

#65 playstation 4 best games ever made 2019 on 06.12.19 at 7:02 am

Saved as a favorite, I love your site!

#66 quest bars cheap on 06.14.19 at 6:05 pm

That is very interesting, You're a very skilled blogger.
I have joined your rss feed and sit up for in the hunt for extra of your
great post. Additionally, I have shared your website in my social networks

#67 quest bars cheap on 06.15.19 at 12:55 am

It's very trouble-free to find out any matter on net as compared to textbooks, as I found this post at this website.

#68 krunker aimbot on 06.17.19 at 12:21 am

Some truly wonderful content on this web site , appreciate it for contribution.

#69 tinyurl.com on 06.17.19 at 3:32 pm

My brother recommended I would possibly like this blog.
He used to be entirely right. This publish actually made my day.
You can not imagine simply how a lot time I had spent for this info!

Thank you!

#70 proxo key generator on 06.19.19 at 9:45 am

Enjoyed reading through this, very good stuff, thankyou .

#71 vn hax on 06.20.19 at 6:38 pm

Awesome, this is what I was looking for in google

#72 nonsense diamond key generator on 06.21.19 at 7:51 am

stays on topic and states valid points. Thank you.

#73 what is a blog on 06.23.19 at 12:49 pm

You are so interesting! I don't believe I've truly read something like this before. So nice to find someone with a few original thoughts on this topic. Really.. many thanks for starting this up. This web site is one thing that is required on the web, someone with a little originality!

#74 badoo superpowers free on 06.23.19 at 5:17 pm

This does interest me

#75 gx tool apk on 06.24.19 at 3:21 pm

Yeah bookmaking this wasn’t a risky decision outstanding post! .

#76 how do we KNOW on 06.25.19 at 5:12 am

This is awesome!

#77 fortnite mods on 06.25.19 at 8:02 pm

Ha, here from yahoo, this is what i was looking for.

#78 krunker aimbot on 06.26.19 at 6:41 am

I’m impressed, I have to admit. Genuinely rarely should i encounter a weblog that’s both educative and entertaining, and let me tell you, you may have hit the nail about the head. Your idea is outstanding; the problem is an element that insufficient persons are speaking intelligently about. I am delighted we came across this during my look for something with this.

#79 ispoofer on 06.27.19 at 6:07 am

Enjoyed reading through this, very good stuff, thankyou .

#80 synapse x cracked on 06.27.19 at 8:55 pm

Cheers, great stuff, I enjoying.

#81 strucid hacks on 06.28.19 at 7:30 am

I really enjoy examining on this web , it has got fine goodies .

#82 advanced systemcare 11.5 pro key on 06.28.19 at 1:33 pm

This does interest me

#83 how to get help in windows 10 on 06.28.19 at 11:46 pm

Heya i'm for the first time here. I found this board and I find
It really useful & it helped me out much. I hope to give
something back and aid others like you helped me.

#84 cryptotab hack script free download 2019 on 06.29.19 at 8:34 am

I like this site because so much useful stuff on here : D.

#85 cryptotab hack script free download on 06.29.19 at 2:55 pm

Deference to op , some superb selective information .

#86 laxify on 07.01.19 at 9:10 am

Hey, happy that i found on this in bing. Thanks!

#87 cheat fortnite download no virus on 07.01.19 at 7:56 pm

I simply must tell you that you have an excellent and unique post that I kinda enjoyed reading.

#88 free cheats for rust on 07.02.19 at 7:50 am

Morning, i really think i will be back to your site

#89 nonsense diamond on 07.02.19 at 1:20 pm

I really enjoy examining on this internet site , it has got good posts .

#90 vn hax download pc on 07.03.19 at 7:29 am

I really enjoy examining on this website , it has got fine goodies .

#91 cyberhackid on 07.03.19 at 7:25 pm

Found this on google and I’m happy I did. Well written site.

#92 vehicle simulator script on 07.04.19 at 7:25 am

Great, this is what I was looking for in google

#93 seo free on 07.04.19 at 2:34 pm

Parasite backlink SEO works well :)

#94 subbot on 07.04.19 at 7:12 pm

I’m impressed, I have to admit. Genuinely rarely should i encounter a weblog that’s both educative and entertaining, and let me tell you, you may have hit the nail about the head. Your idea is outstanding; the problem is an element that insufficient persons are speaking intelligently about. I am delighted we came across this during my look for something with this.

#95 open dego file on 07.05.19 at 7:26 am

I conceive you have mentioned some very interesting details , appreciate it for the post.

#96 erdas foundation 2015 on 07.05.19 at 7:41 pm

I kinda got into this site. I found it to be interesting and loaded with unique points of view.

#97 synapse x download on 07.06.19 at 6:57 am

I was looking at some of your articles on this site and I believe this internet site is really instructive! Keep on posting .

#98 gx tool apk pubg uc on 07.06.19 at 11:12 am

Great article to check out, glad that duckduck brought me here, Keep Up cool job

#99 rekordbox torrent on 07.06.19 at 10:33 pm

Ha, here from yahoo, this is what i was looking for.

#100 call of duty black ops 4 activation key pc on 07.07.19 at 8:44 am

Good Morning, yahoo lead me here, keep up good work.

#101 spyhunter 5.4.2.101 serial on 07.08.19 at 8:48 am

Some truly cool goodies on this web site , appreciate it for contribution.

#102 fps unlocker download on 07.09.19 at 10:26 am

Great article to see, glad that Yahoo led me here, Keep Up great job

#103 Elljeks on 07.10.19 at 2:26 am

Zithromax Azithromycin Online [url=http://buyoxys.com]wheretobuylevitrapills[/url] Viagra 2 Day Delivery Cialis Forocoches Contact Customer Ed

#104 quest bars cheap on 07.11.19 at 6:11 am

We're a bunch of volunteers and opening a brand new scheme in our community.
Your site provided us with useful info to work on. You've done an impressive activity and our entire neighborhood might be grateful to you.

#105 how to get help in windows 10 on 07.16.19 at 7:03 pm

Thanks for sharing your thoughts. I really appreciate your efforts and I will be waiting for your further post thank you once again.

#106 Lesangent on 07.16.19 at 10:41 pm

Kamagra Impacto [url=http://leviprices.com]best prices for on line levitra[/url] Priligy Expedition Rapide Viagra Duree

#107 plenty of fish dating site on 07.18.19 at 5:43 pm

Hello everyone, it's my first pay a quick visit at this website, and
post is really fruitful for me, keep up posting these types of articles.

#108 prodigy game files on 07.21.19 at 3:04 pm

Very interesting points you have remarked, appreciate it for putting up.

#109 how to get help in windows 10 on 07.22.19 at 8:21 am

Hey There. I found your blog using msn. This is a really well written article.

I will be sure to bookmark it and return to
learn extra of your helpful info. Thanks for
the post. I'll certainly comeback.

#110 acid swapper on 07.23.19 at 1:09 pm

Appreciate it for this howling post, I am glad I observed this internet site on yahoo.

#111 natalielise on 07.24.19 at 3:51 am

WOW just what I was looking for. Came here by searching for natalielise pof natalielise

#112 pphud download on 07.24.19 at 1:30 pm

Your post has proven useful to me.

#113 roku.com/link on 07.24.19 at 9:05 pm

Thanks for sharing your thoughts

#114 ezfrags on 07.25.19 at 3:24 pm

I was looking at some of your articles on this site and I believe this internet site is really instructive! Keep on posting .

#115 plenty of fish dating site on 07.25.19 at 3:24 pm

Hello there, I discovered your blog via Google even as
looking for a similar subject, your web site got here
up, it appears great. I've bookmarked it in my google bookmarks.

Hi there, just turned into aware of your blog through Google,
and found that it is really informative. I'm going
to be careful for brussels. I will be grateful for those who continue this in future.
A lot of people will likely be benefited from your writing.
Cheers!

#116 smore.com on 07.26.19 at 1:49 pm

Simply desire to say your article is as amazing. The clarity in your post is just spectacular and i could assume you're
an expert on this subject. Fine with your permission allow me to grab your feed
to keep updated with forthcoming post. Thanks a million and
please carry on the rewarding work. pof natalielise

#117 ezfrags on 07.26.19 at 4:30 pm

I dugg some of you post as I thought they were very beneficial invaluable

#118 rokucoms on 08.28.19 at 11:16 am

Wonderful, what a website it is! This website gives useful
data to us, keep it up.

#119 Silver Dating on 09.20.19 at 9:21 am

Hello dear, I raed your whole content is very awesome, keep it up, Thanks for share this article such a nice informative post, keep up the good work.

#120 Hulu Support on 09.20.19 at 10:21 am

Really dear, I like your article it is easy to reading. You are a good blogger pls keep it up. Thank you so much for this type of informaton.

#121 Hulu Support on 09.20.19 at 12:17 pm

Your blog is very nice and amazing information for all of us. Be continue and do best always. thanks for sharing.

#122 Tinder Service on 09.20.19 at 12:18 pm

thanks you so much dear, I found bestest ever information on your blog , keep sharing and blogging.

#123 Seeking Arrangement on 09.20.19 at 12:56 pm

Great work !! Looking forward to more of such good work, Thanks for sharing this helpful information with us.

#124 Shein Service on 10.03.19 at 10:15 am

Wow it is extremely superb and great along these lines it is particularly helpful for me to comprehend numerous ideas and helped me a ton. it is extremely logical best and I got more information from your website

#125 Netflix com tvhelp on 10.07.19 at 9:37 am

I just want to say I am very new to blogs and truly savored you’re web site. More than likely I’m likely to bookmark your website . You amazingly come with superb articles and reviews. Regards for sharing your webpage.

#126 OurTime Dating Site on 10.07.19 at 1:27 pm

Usually I never comment on blogs but your article is so convincing that I never stop myself to say something about it. You’re doing a great job Man,Keep it up.

#127 Seeking Arrangement customer support on 10.07.19 at 3:17 pm

Great work !! Looking forward to more of such good work, Thanks for sharing this helpful information with us.

Leave a Comment