There's this very elegant way to shoot your leg off with a DVCS. Here's the recipe:
- I create a clone of the repository A and call it B. You create another clone of A and call it C.
- I add an if statement in B, fixing a bug. You fix the same bug in C in a very similar way.
- I pull your changes from C to B. There's a conflict around the bug since we both fixed it. I take my patch, throwing away your nearly identical patch.
- Meanwhile, your roommate pulled both B and C into his clone, D. And he had to resolve that conflict, too. And he took your patch, and threw mine away.
- Now, D and B are pushed back to A. DVCS quiz: what happens?
Answer: the system has accurately recorded the manual merges, so it has enough information to make this new merge automatically. It sees your patch, and it throws it away as it should according to my manual merge. It sees my patch, and it flushes it down the toilet since that's what your roommate said. Net result: both patches are gone, the bug is back in business. (Edit: it doesn't work that way – bk version 4 does a different thing, and other systems reportedly do still other things. Do you still want to read the rest of this?..)
Which of course doesn't matter, since it's immediately discovered by the massive Automated Test Suite. For example, if it's an OS kernel, each revision is automatically tested on all hardware configurations it should run on. And the whole process only takes 10 minutes, according to the Ten Minute Build XP Practice. No harm done, no reason to discuss it. I just thought it was a curious thing worth sharing.
Maybe it's a well-known thing, but I don't think it is, and if I'm right, it's definitely lovable. For example, here's what BitMover, maker of BitKeeper, the common ancestor of DVCSes, has to say about this:
"It's important to note that because BitKeeper has a star topology and its possible to share data with any repository, it's not necessarily recommended."
What this is trying to say is that the graph of pulls shouldn't be a generic graph and you're better off with a tree. That is, I shouldn't pull directly from you; we should both pull from and push to A. You and your roommate should also synchronize via A, or via A's "child" repository, but then you shouldn't push to A directly, only via that child, and so on. If we maintain this tree structure, the same conflict will never be resolved twice, and then we won't get screwed when the merges are merged.
I wonder if you could detect the situations when you "merge merges", that is, when the same conflict was resolved differently. You could then insist on human intervention and save those humans' bacon. I'm too lazy to think this out and too stupid to effortlessly see the answer, so I'll resort to a social heuristic like all of us uber-formal nerds do. The social heuristic is that Larry McVoy of BitMover has probably already thought about this, and found ways in which it could fail. So I'm not going to argue that BitKeeper merges are broken.
What I'm going to argue, at least for the next couple of paragraphs, is that it sucks when they tell you about their superstar topology and then explain that it's best to avoid actually using it. Not only that, but they fail to mention a fairly frightening and, trust me, not at all unlikely scenario which could actually persuade you to follow their advice.
Because when they tell me "we have this simple model of how things work – repositories with complete local history and changes flowing between them – but you should arbitrarily restrict yourself to a subset of this model, for reasons we aren't going to share with you, even though the general case works", when they tell me that, my reply is "I alias rm to rm -f". I understand how rm works, it's fairly simple, and I don't like to talk about it over and over again, "Are you sure?" – yes, I'm sure, thank you very much and good bye.
But the lovable part is, speaking of social heuristics, the lovable part is that BitMover said it right. Because if they mentioned that fairly frightening and not at all unlikely scenario, they'd scare people off rather than illustrate a point. On the other hand, when they say "It's good practice to think about how the data should flow", most people will nod and follow whatever advice they give them.
Just imagine a team of programmers engaging in the practice of thinking about how the data should flow, dammit, all on company time. Yeah, yeah, so BitKeeper earned a sarcastic comment on Proper Fixation. It's still a small price to pay for getting your message to the majority of your users.
You see, the majority of programmers are not just "irrational" as we all are, but their reliance on reasoning doesn't even exceed the mean of the population, which means they barely use reasoning at all, it's pure gut feeling.
For example, I was writing a bunch of macros in a proprietary debugger extension language. A guy who came to talk to me looked over my shoulder, and I explained, "Debugger macros. Very useful, a crappy language though." He said, "Yeah, looks like so."
HE COULDN'T POSSIBLY KNOW THAT. I knew he couldn't. How could he look at the code and realize that all variables were global? How could he know they were printed upon assignment, including loop counters ('cause it's a "macro", so it works just like assigning at the debugger console, which prints the variable)? He couldn't know any of that. So why did he agree with the "crappy" part? Oh, I knew why.
"You mean it has dollar signs?" Silence. "You mean it prefixes variable names with the dollar sign, don't you?" "Yeah, that." "Well, I like the dollar signs, helps you distinguish between your macro variables and the variables of the debugged C program. Anything else?" "Well, um, yeah, it looks kinda primitive." Low-end Ignorant Language Bigotry quiz: if "crappy" means "has dollar signs", what does "primitive" mean? Answer: no type declarations. I'm sure it was that, although I didn't go on and ask.
So that's "engineers" for you. If you want to write programs or tech docs for the average engineer, keep that picture in mind. Or don't. Aim for the minority, for people who don't work that way, under the assumption that they are the ones that actually matter the most. I don't know if this assumption is right, but at least it's lofty.
Why DVCS?
For the record, I had my share of both centralized and distributed version control systems, and today I like it distributed and I wouldn't ever want to go back, The Most Vexing Merge be damned. Why? I'll share my reasons with you through the story of My Last CVS To BitKeeper Exodus. I think I'll illustrate the engineers-and-reasoning-don't-mix point as a by-product, because that's how that story goes.
There recently was this argument about DVCS encouraging "code bombs", a.k.a "crawling into a cave". I haven't heard either of these terms, so I've been using a term of my own – "accumulating critical mass". The idea is to develop in your own corner, without integrating it with the main branch. You then show up with N KLOC of new stuff and kindly ask to merge it in.
Some people claimed this was particularly harmful for open source projects where there was no managerial authority to prevent this. Ha! Double ha. In an open source project, the key maintainers may say, "you see, we can't integrate it when it's done this way; we're sorry, but you should have talked to us." The changes will then be reimplemented or dropped on the floor.
Now, if you think that in a commercial environment a manager can easily decide to drop changes on the floor, together with the cost of implementing them and especially the cost of delaying the delivery of the features, if you think that, well, I wonder why you do. But perhaps a manager could insist on frequent integration? She could try, but she'd have to deal with real or imaginary cost of merges, increasing over time and getting in the way of deliveries. Of course there are perfect managers and perfect teams where it's all dealt with appropriately, you just have to find them.
So yeah, "code bombing" is a problem, especially in commercial projects. But the idea that DVCS encourages it? Hilarity! It's like saying that guns encourage murder. I prefer to think of guns as something that encourages fear of armed policemen, getting in the way of the natural instinct to club your neighbors to death. I mean, yeah, it's easier to code bomb with a DVCS, but with a centralized system, people use code bombing – or clubbing? – even more, because merging is harder, the cost of merges increases more quickly and the ability to force integration is thus lower. The criminals are poorly equipped, but so is the police.
And this is exactly what happened to the last team stuck with CVS that I helped migrate to BitKeeper. Everybody had their own version made up of file snapshots taken at different times and merged with the repository version to different extents. A centralized system doesn't record these local versions, so unless you immediately commit your merges, you are left with a version of a file which the system doesn't know. This means that the next merges are going to be really hard, because you've lost your GCA, the greatest common ancestor. So instead of a 3-way merge, you'll be doing a 2-way merge, which really sucks.
So I decided to not talk about the caves they were crawling into and the code bombs they were throwing at each other. Rather, I decided to show them how a 2-way merge couldn't do what a 3-way merge could. I still think it's the ultimate argument for DVCS, because DVCS is basically about accurate recording of all versions and not just the single time line of the main trunk. So the argument for it has to do with the benefits of such detailed recording.
So I gave this example where you start with a file having two lines:
aaa
bbb
And then I add a line, ccc, and you delete a line, aaa. If we have the GCA (a 3-way merge), then clearly the right result is deleting aaa and adding ccc, getting this:
bbb
ccc
But with a 2-way merge, we only have your file:
bbb
…and my file:
aaa
bbb
ccc
This can only be merged manually, because there's no way to automatically figure out that you deleted aaa and I added ccc; for all the tool knows, you could have done nothing and I've added two lines, so the right merge is:
aaa
bbb
ccc
…canceling your change. So it has to be manual merge. Manual merge means dozens of boring deltas you have to inspect in each file. That's what I call "costly".
Of course it doesn't matter in a right world, where people integrate frequently and always commit their merged files to the centralized repository. Except it wasn't so in the wrong world of the CVS developers I was "helping" to upgrade to new tools (for the last time in my life, people, for the last time in my life). And I thought we could avoid the discussion of the somewhat-technical-but-largely-social reasons of the constantly increasing cost of merges, and instead we could focus on the technical benefits of the 3-way merge and accurate GCA recording.
And of course I was wrong. The discussion immediately shifted to "we don't need merges" because everything is "modular" and there's a single owner to each file. Of course it wasn't, and there wasn't. Some things were used by everybody, like the awful build scripts and the DMA code. Some modules had two owners, or were in a transition state and had 1.5 owners, and so on. There were merges all over the place.
And if there weren't merges and merge-related problems, how come everybody worked on their own "pirate" version which was never recorded in the main trunk and was made from a colorful variety of files partially merged at different times? How come changes propagated with cp and emacs-diff and not cvs update? And why was the tech lead so passionate about moving to BitKeeper which doesn't let you partially update a repository so you have to merge everything? And why did everybody anxiously object that necessity if there were "no problems with merges"?
The final result: the tech lead simply forced the migration to bk. Everybody on the team hated me for my connection with the idea (I wasn't on their team but I used to be a likable satellite and now became a hateful satellite). Developers who I thought were their best eventually (and I mean eventually) told me it was actually a good thing. So it wasn't a bad closure. And still, I decided that I'm not going to "help" anybody "deploy" any kind of "tool" in this lifetime again, roughly speaking. Too much emotions for this programmer.
And this was supposed to show why I like DVCS, at least in the imperfect world where long-living branches occasionally happen, and the kind of reasoning I think is interesting in this context, and the kind of reasoning other people I came across found interesting. So there were are.
P.S. Why "most vexing"?
I thought I saw that "C++'s most vexing parse" from Scott Meyers' Effective STL has its own Wikipedia entry, but apparently it doesn't. It's basically a variation on the theme of C++'s declaration/definition ambiguity, and I liked the term, especially the "most" part where parses are unambiguously sorted along the vexing dimension. So I figured "X's most vexing Y" is a good template.
I'd like to use this opportunity to say that I skimmed though Effective C++, 3rd Edition, and… Where do I start? There's an advice to create date objects with "Date d(Day(31), Month::april(), Year(2000))" or something. That is, create special types for the constructor arguments. Well, it doesn't check that April comes without the 31th day, does it? The Date constructor could test for it though. Well, why not test for April 41st in the Date constructor, too, and, ya know, spare the users some keystrokes, if you see what I mean? The code is verbose. C++ compiler error messages are verbose. VERBOSITY EVERYWHERE! Help!
This raises the question to the author, whether he ever worked with a system where every piece of data comes covered with the toxic waste of overzealous static typing. But this borders on an ad hominem attack. And seriously, that sort of thing is to be avoided, at least until somebody proposes to have named constants for days or years and not just months.
So instead of the personal attack, I'll ask Software Development Consultants, all of them, to kindly change the phrasing "it's best to do so and so" to "I like so and so" or something. Because we have this huge crappy-dollar-sign crowd, and they copy style from each other like crazy, and their ultimate source of style is books by Software Development Consultants, and whenever they see a "best practice", their common sense is turned off and they add the technique to the bag of tricks. So Consultants have a great power in this world. It doesn't make the common sense shut-off feature their fault, but power they do have.
And with great power comes great responsibility, profanity deleted. I mean, you're obviously giving advice neither you nor others have tested for very long, out of generic principles, profanity deleted. Like "prefer algorithms such as for_each to loops", an advice issued before fully compliant implementations of STL were even widely available, profanity deleted. Quite a piece of advice, that, profanity deleted. Couldn't you at least phrase your advices in a little less self-assured way, fucking profanity deleted?
For example, Meyers has finally lowered the bridge and let the enemy of template metaprogramming occupy a notable share of pages in an Effective C++ edition. I still remember his promise to "never write about templates", in the preface to Modern C++ Design, I think. And now the promise is broken. Hordes of clueless weenies are rushing into the minefield of template metaprogramming as we speak, since it's now officially Mainstream C++. Can you imagine the consequences? I can't. It's too awful. I think I'll go to sleep now.
221 comments ↓
Eh? When I try to recreate this scenario in git, in the last step, no matter whether B or D has gone first, the second guy to try to push loses: s/he cannot push before pulling, and when s/he does, s/he gets a merge conflict that needs manual resolution. But then, git is known to refuse to merge some cases automatically where other DVCSs will try. Maybe this is one of them? (In which case that would resoundingly vindicate Linus’ insistence on this design, whence git got its “stupid content tracker” moniker.
Unfortunately in many software development organizations, once a project creates a branch most of the other developers stop paying any attention to it. So even if the changes are submitted to the branch in small, easily digestible chunks they will still be perceived as a Code Bomb by the rest of the team when the branch merges back to main. Thats human nature, not the technology of the version control system.
To AP: in bk, you'd have to pull before pushing, too, but the conflict resolution would be automatic. Apparently the merge heuristics differ; I wonder what makes git refuse to merge.
In general, I think automerges are always unsafe, so the whole question of deciding when you go ahead and use them and when you demand human intervention is completely heuristical. And I'm not very good at heuristics, that is, in deciding which of two provably incorrect methods is going to do less harm and more good across the (informally specified) distribution of use cases.
To DGentry: Exactly, people don't like the overhead of merging and try to delay it, which creates long-living branches, which makes merging even costlier, etc. A vicious cycle, made even more vicious by the fact that people won't admit its existence.
I really really hate STL, god the pain…. Rather C than STL.
Yes, well, Bram Cohen had a long argument going with Linus back when git was new, based on the fact that in the Codeville project, Bram and friends tried really hard to automerge as many things as possible. Linus was of the opinion that this is lunacy, and that beyond any but the most trivial conflicts you want a human to examine and decide. The git-merge manpage describes its strategies; the hint I get from it is that the design goal was to make the merge strategies less stupid, but not smarter, and that experiments with existing data informed the design process. Given how many other things git does in an unusual but unusually right way, even among DVCSs, I wouldn’t be surprised if this were yet another area in which it stands out from the pack.
I am unable to replicate your merge issues in Bazaar. When trying to push from D to A, I receive an error that the branches have diverged and I should merge before pushing. Executing `bzr merge` discovers the error and marks the conflict for manual resolution.
This is because when it comes time to push, the repository looks like this:
A: A[1]
B: A[1] -> B[1] -> B[2, B->C]
C: A[1] -> C[1]
D: A[1] -> D[1, B->D] -> D[2, C->D]
When you push from B to A, A is updated to:
A: A[1] -> B[1] -> B[2] (B->C)
And then trying to push from D to A obviously fails, because revisions exist in A that are not present in D.
BitKeeper's willingness to force the merge anyway and drop your revisions on the floor is just more indication that it is unsuitable for production. BitKeeper is only useful for people that do not mind silent data loss.
> This begs the question to the author, whether he ever worked with a system where
It raises the question. Begging the quesiton is a specific form of logical fallacy not present here.
To AP: I think working on a kernel raises the bar for merge safety. A large portion of bugs in kernelish code, interrupt handlers, context switching and stuff, a large portion of those bugs by which I got personally fucked came from automerges. As far as I recall, all those automerges happened in spots remote from each other and basically any merge algorithm would do them automatically and assume it was safe. However, while all kinds of automerges are a risk, you'd obviously want to at least restrict them to the safest subset once you realize you can't afford the cost of actually reviewing all conflicts.
To John: for starters, I can't reproduce the issue with bk, embarrassingly enough; what bk 4 apparently does is it takes the newer conflict resolution, so one of the patches is kept. Of course it still isn't "safe" in the sense that one manual resolution was dropped on the floor; it's just the right thing in my example assuming the semantics of the edits.
Regarding "unsuitable for production": just a little bit over the edge; I used bk a lot in production and it, um, worked. I mean, I never allowed myself to say that C++ was unsuitable for production, because the fact is that you can deliver with C++, and I did, and C++ is, really, an exceedingly hateful piece of excrement. "Unsuitable for production" is kinda under the belt for new systems and not very credible for old ones that have already been used in production.
Another thing is that all automerges are risky, and so are manual merges because you'll get tired of reviewing conflicts and err; the balance between manual and automatic merges is apparently a completely heuristical thing, because you can't prove anything about the relation of the semantics of a merged program to the semantics of the two original versions.
That said, I shouldn't have generalized my experience with bk (itself somewhat stale) to other distributed systems; I should have realized that merging is a can of warms, so quite likely everybody is doing it differently. That "merging merges" is a situation unique to DVCSs doesn't mean they use the same solution. Where's a brain when I need one?
And thanks for, um, begging the question… Fixed.
To Entity: while <algorithm> sucks better than your average vacuum cleaner, having <map>, <list> and <vector> comes handy. If I could add one thing to C, it would be a standard container library. The troubles with the STL containers are the trademark verbosity and debuggability problems, ranging from the awful type names to inability to make sense of them when they are partially overwritten. But it can still be better than C's nothing.
I do not mean "unusable", but "unsuitable". I consider the primary purpose of a VCS to be the storage and preservation of version data, and if a particular VCS will discard version data, I don't want it anywhere near my code. CVS, for example, is another system that I call unsuitable for production. Although it is obviously heavily used, much more than all DVCSes combined, it nevertheless is a poor choice in version control.
You could compare it to using a RAM disk for permanently storing vital data. I'm sure that somebody somewhere does so and has never had a problem with eg. power outages, but this fact still doesn't make it a good idea.
Well, bk won't throw away version data in the sense that it will preserve all revisions and you'll be able to go back to any point; that the new version it automatically generates isn't necessarily the right way to resolve the conflict it sees is another matter. So it's more like an opinionated hard disk than a RAM disk…
Not only can’t you afford to review every single conflict, but I would argue that there is an even more important reason you aren’t going to review every conflict. Essentially this reason is because the danger of semantic conflict introduced by a disparate changes increases with distance. Let me try to explain.
If the changes are close by, then trivial resolutions are more likely to be correct (because those that are incorrect are more likely to touch the same parts of the code and thus result in non-trivial conflicts), and in those few cases where they aren’t, the problem will manifest itself relatively quickly and probably in an obvious way. But detecting semantic conflicts introduced by merges between changes in widely separated areas of the code is likely to require holding more of the program in your head than most programmers can – or in projects the size of the kernel, than any programmer can. So even if you did review those 3-way diffs manually, it would be a comparatively useless exercise: the effort would be far disproportionate with the gain in confidence.
In fact, I would argue that merging is a red herring: this sort of semantic clash between disparate but interdependent parts of the code is inevitable the moment they can evolve separately. How this concurrent development process is organised – DVCS; CVCS; tarballs and patches; everyone edits the same files on a network share – is of no consequence at all to that fact. Merging doesn’t cause semantic clashes, programming causes semantic clashes.
And so you might as well at least relieve the programmers from the drudge work.
[...] an evil of distributed revision control systems (and a retraction, but still worth a [...]
Right now it sounds like Movable Type is the preferred blogging platform available right now.
(from what I've read) Is that what you are using on your blog?
I am in fact thankful to the holder of this website who has
shared this impressive article at at this place.
If you are going for best contents like myself,
simply go to see this website every day because it provides quality contents, thanks
Hey I know this is off topic but I was wondering if you knew of any widgets I could add to my blog that automatically tweet my newest
twitter updates. I've been looking for a plug-in like this
for quite some time and was hoping maybe you would have some experience
with something like this. Please let me know if you run into anything.
I truly enjoy reading your blog and I look forward
to your new updates.
I love your blog.. very nice colors & theme. Did you make this website yourself or did you hire someone to do it for you?
Plz reply as I'm looking to create my own blog and would like
to find out where u got this from. kudos
Greetings! This is my first visit to your blog! We are a team of volunteers and starting a new
project in a community in the same niche. Your blog provided us
beneficial information to work on. You have done a marvellous job!
I am glad to be one of the visitors on this great website (:, appreciate it for posting .
I consider something really special in this site.
I am not rattling great with English but I get hold this really easygoing to read .
Hi, i really think i will be back to your website
I am not rattling great with English but I get hold this really easygoing to read .
I am glad to be one of the visitors on this great website (:, appreciate it for posting .
I truly enjoy looking through on this web site , it holds superb content .
I simply must tell you that you have an excellent and unique article that I kinda enjoyed reading.
Helpful information. Lucky me I found your web site by accident,
and I am surprised why this accident did not took place in advance!
I bookmarked it.
I must say, as a lot as I enjoyed reading what you had to say, I couldnt help but lose interest after a while.
I consider something really special in this site.
I have interest in this, cheers.
very nice post, i actually enjoyed this web site, carry on it
Thanks for this site. I definitely agree with what you are saying.
Hi, here from bing, me enjoyng this, I come back again.
I consider something really special in this site.
Some truly good content on this web site , appreciate it for contribution.
I am glad to be one of the visitors on this great website (:, appreciate it for posting .
Me like, will read more. Thanks!
Hi, yahoo lead me here, keep up great work.
Deference to op , some superb selective information .
Hello, i really think i will be back to your website
bing took me here. Thanks!
Found this on bing and I’m happy I did. Well written website.
very Great post, i actually love this web site, carry on it
Enjoyed reading through this, very good stuff, thankyou .
I consider something really special in this site.
Great, this is what I was looking for in bing
Respect to website author , some wonderful entropy.
This is nice!
You actually make it seem so easy along with your presentation but I
to find this matter to be really one thing which I feel
I would never understand. It sort of feels too
complex and extremely extensive for me. I am looking forward on your next submit,
I'll try to get the cling of it!
Me enjoying, will read more. Thanks!
I really enjoy examining on this website , it has got good article .
It's a shame you don't have a donate button! I'd certainly donate to this fantastic blog!
I guess for now i'll settle for bookmarking and adding your
RSS feed to my Google account. I look forward to fresh updates and will talk about this blog with my
Facebook group. Talk soon!
This does interest me
You got yourself a new rader.
Hi to all, the contents existing at this site are truly amazing for people experience, well, keep up the nice work fellows.
Everyone loves it whenever people come together and share
opinions. Great site, stick with it!
Hello, just wanted to say, I loved this post.
It was helpful. Keep on posting!
Respect to website author , some wonderful entropy.
I have interest in this, xexe.
I believe that is among the most vital information for me.
And i'm glad studying your article. But should observation on few general things, The website taste is great, the articles is in point of fact excellent : D.
Good job, cheers
stays on topic and states valid points. Thank you.
Fantastic web site. Lots of useful information here.
I am sending it to a few buddies ans also sharing in delicious.
And obviously, thank you in your sweat!
Thanks for another excellent article. The place else could anyone get that type of info in such an ideal method of writing?
I've a presentation next week, and I'm on the search for such info.
Discount Elocon C.O.D. Can I Purchase With Free Shipping Propecia Cost In Usa Isotretinoin [url=http://sildenaf100mg.com]viagra[/url] Can I Get Cymbalta In Mexico Cipro Online No Prescription Overnight
My programmer is trying to convince me to move to .net from
PHP. I have always disliked the idea because of the expenses.
But he's tryiong none the less. I've been using Movable-type on several websites for about a year and
am anxious about switching to another platform. I have heard fantastic
things about blogengine.net. Is there a way I can transfer all my wordpress posts into it?
Any help would be greatly appreciated!
I love your blog.. very nice colors & theme.
Did you design this website yourself or did you hire someone to
do it for you? Plz reply as I'm looking to construct
my own blog and would like to know where u got this from.
many thanks
Awesome things here. I'm very satisfied to peer your post.
Thank you a lot and I'm looking forward to touch you. Will you kindly drop me a mail?
I enjoy what you guys are up too. This type of clever
work and coverage! Keep up the excellent works guys I've incorporated
you guys to my personal blogroll.
I feel this is one of the such a lot vital info for me.
And i'm happy reading your article. But should statement
on few normal issues, The website style is wonderful, the articles is
really great : D. Just right task, cheers
I believe this is one of the most important info for me.
And i am satisfied reading your article. However wanna
commentary on few basic things, The web site taste is wonderful, the articles is actually nice : D.
Just right job, cheers
Hi! Do you use Twitter? I'd like to follow you if that would be okay.
I'm undoubtedly enjoying your blog and look forward to new updates.
Incredible quest there. What occurred after? Thanks!
Propecia Ville [url=http://orderviapills.com]viagra[/url] Where To Buy Unisom 2 Viagra Advertised On Radio
I always used to read piece of writing in news papers but now as I am a user
of web so from now I am using net for articles, thanks
to web.
What i do not realize is if truth be told how you are now not actually much
more well-appreciated than you might be now. You are so intelligent.
You realize therefore considerably in relation to this subject,
made me for my part believe it from a lot of numerous angles.
Its like men and women are not interested unless it is
one thing to accomplish with Lady gaga! Your individual stuffs nice.
Always maintain it up!
You got yourself a new follower.
Enjoyed examining this, very good stuff, thanks .
I'd like to find out more? I'd care to find out more details.
I do not even know how I finished up right here, but I believed this post was good.
I do not recognise who you're however definitely you're going to
a famous blogger if you are not already.
Cheers!
Cephalexin Used For Dogs Cheap Generic Cialis Free Shipping [url=http://purchasecial.com]buy generic cialis online[/url] Indicaciones De Propecia Diflucan Compare Prices Viagra Kaufen Mit Uberweisung
Wow, wonderful weblog structure! How lengthy have you ever been blogging for?
you make blogging glance easy. The whole glance of your website is excellent,
let alone the content material!
Respect to website author , some wonderful entropy.
Ha, here from google, this is what i was browsing for.
I must say got into this web. I found it to be interesting and loaded with unique points of view.
I like this site because so much useful stuff on here : D.
This does interest me
Awesome, this is what I was looking for in google
Have you ever thought about including a little bit more than just your articles?
I mean, what you say is fundamental and all.
Nevertheless imagine if you added some great pictures or video clips to give your posts more,
"pop"! Your content is excellent but with pics and video clips,
this site could undeniably be one of the most beneficial in its field.
Awesome blog!
Hello! This post could not be written any better!
Reading through this post reminds me of my good old room mate!
He always kept chatting about this. I will forward this article to him.
Pretty sure he will have a good read. Thank you for sharing!
What a stuff of un-ambiguity and preserveness of
valuable knowledge concerning unexpected feelings.
Intresting, will come back here later too.
Thank You for this.
I constantly emailed this website post page to all my
contacts, since if like to read it next my contacts will too.
I dugg some of you post as I thought they were very beneficial invaluable
This does interest me
This does interest me
Great, google took me stright here. thanks btw for post. Cheers!
Good, this is what I was looking for in yahoo
Enjoyed reading through this, very good stuff, thankyou .
Great, this is what I was searching for in google
I like this site because so much useful stuff on here : D.
Hey I know this is off topic but I was wondering if you knew of any widgets I could add to my blog that automatically tweet my newest twitter
updates. I've been looking for a plug-in like this for quite some
time and was hoping maybe you would have some experience with something like this.
Please let me know if you run into anything.
I truly enjoy reading your blog and I look forward to your new updates.
Viagra Apotheke Nl Retin A Cream [url=http://sildenaf75.com]viagra[/url] Canada Pharmacy Online
"Major thanks for the blog post.Much thanks again. Really Great."
I dugg some of you post as I thought they were very beneficial invaluable
This is interesting!
Hello there, just became aware of your blog through Google, and found that it is truly informative.
I'm gonna watch out for brussels. I will appreciate if you continue
this in future. A lot of people will be benefited from your writing.
Cheers!
Appreciate it for this howling post, I am glad I observed this internet site on yahoo.
Thanks in favor of sharing such a good opinion, post is nice,
thats why i have read it fully
Very interesting points you have remarked, appreciate it for putting up.
I am glad to be one of the visitors on this great website (:, appreciate it for posting .
Enjoyed examining this, very good stuff, thanks .
I really got into this article. I found it to be interesting and loaded with unique points of interest.
Good, this is what I was looking for in bing
Hi, here from yanex, me enjoyng this, will come back soon.
I really like your blog.. very nice colors & theme. Did you
create this website yourself or did you hire someone to
do it for you? Plz reply as I'm looking to design my own blog
and would like to know where u got this from.
thanks a lot
I think what you said made a lot of sense. But, think about this, suppose you were
to create a awesome headline? I mean, I don't want
to tell you how to run your website, however what if you added a post title that makes
people desire more? I mean DVCS and its most vexing merge is kinda boring.
You might peek at Yahoo's front page and see how they create
news titles to grab viewers to click. You might add a related video or a pic or two to
get people interested about everything've written. In my opinion, it might bring your
posts a little bit more interesting.
Hello my friend! I want to say that this post is amazing,
nice written and come with approximately all important infos.
I would like to look extra posts like this .
I'm curious to find out what blog platform you are using?
I'm having some small security issues with my latest
website and I'd like to find something more safe.
Do you have any recommendations?
Heya i am for the first time here. I came across
this board and I find It really useful & it
helped me out a lot. I hope to give something back and aid others like you
aided me.
stays on topic and states valid points. Thank you.
I like this site because so much useful stuff on here : D.
My coder is trying to persuade me to move to .net from PHP.
I have always disliked the idea because of the costs. But he's tryiong none the less.
I've been using Movable-type on several websites for about a year and am anxious about switching to
another platform. I have heard good things about blogengine.net.
Is there a way I can transfer all my wordpress content into it?
Any help would be greatly appreciated!
I every time spent my half an hour to read this web
site's articles daily along with a cup of coffee.
I conceive this web site holds some real superb information for everyone : D.
Great, bing took me stright here. thanks btw for this. Cheers!
That is a very good tip particularly to those fresh to the
blogosphere. Brief but very precise info… Appreciate your sharing this one.
A must read article!
I am not rattling great with English but I get hold this really easygoing to read .
Good, this is what I was browsing for in bing
Deference to op , some superb selective information .
Respect to website author , some wonderful entropy.
Ha, here from bing, this is what i was looking for.
Very interesting points you have remarked, appreciate it for putting up.
I believe what you wrote was actually very reasonable.
However, what about this? what if you composed a catchier title?
I am not suggesting your content is not solid., but suppose you added something that
makes people desire more? I mean DVCS and its most vexing merge is a little plain. You might
look at Yahoo's home page and watch how they write news titles to
grab people to click. You might add a related
video or a pic or two to grab readers interested about everything've got to say.
In my opinion, it could bring your blog a little livelier.
I must say, as a lot as I enjoyed reading what you had to say, I couldnt help but lose interest after a while.
I am glad to be one of the visitors on this great website (:, appreciate it for posting .
Hey, happy that i saw on this in google. Thanks!
Enjoyed reading through this, very good stuff, thankyou .
Respect to website author , some wonderful entropy.
I like this website its a master peace ! Glad I found this on google .
Hi there, I enjoy reading through your article. I wanted to write a little comment to support you.
Parasite backlink SEO works well :)
Enjoyed examining this, very good stuff, thanks .
Great, this is what I was browsing for in google
Hello, google lead me here, keep up nice work.
I dugg some of you post as I thought they were very beneficial invaluable
Hello There. I discovered your blog the usage of msn. That is an extremely smartly written article.
I will make sure to bookmark it and come back to read extra of your helpful
info. Thanks for the post. I will certainly comeback.
I love reading through and I believe this website got some genuinely utilitarian stuff on it! .
Hi, yahoo lead me here, keep up nice work.
Farmaco Equivalente Priligy Hydrochlorothiazide 25mg Where Can I Buy In Usa Discount Amoxicilina Betalaktam For Sale Free Shipping Pharmacy [url=http://sildenaf100.com]viagra online pharmacy[/url] Levitra Avec Dapoxetine
Respect to website author , some wonderful entropy.
Very interesting points you have remarked, appreciate it for putting up.
The next stage of the labyrinth is to decode the order of the pyramid. This is your 3rd confidential lead!! 517232125
Appreciate it for this howling post, I am glad I observed this internet site on yahoo.
Ni hao, here from bing, me enjoyng this, will come back again.
Just wanna input on few general things, The website layout is perfect, the articles is very superb : D.
Me like, will read more. Thanks!
I was able to find good information from your blog posts.
Thank You for this.
Great stuff to Read, glad that yandex brought me here, Keep Up good job
I like this site because so much useful stuff on here : D.
I like this site because so much useful stuff on here : D.
Hi, I do believe this is an excellent web site. I stumbledupon it ;) I may come back yet again since I book marked it.
Money and freedom is the best way to change, may you
be rich and continue to guide others.
Great stuff to check out, glad that yandex took me here, Keep Up nice job
I’m impressed, I have to admit. Genuinely rarely should i encounter a weblog that’s both educative and entertaining, and let me tell you, you may have hit the nail about the head. Your idea is outstanding; the problem is an element that insufficient persons are speaking intelligently about. I am delighted we came across this during my look for something with this.
Trusted Tablets Pharmacy [url=http://sildenaf75mg.com]viagra[/url] Cephalexin Keflex 500 No Prescription Goldshield
Excellent post. I was checking constantly this blog and I am
impressed! Very useful information specially the last part :) I
care for such info much. I was looking for this particular information for
a long time. Thank you and best of luck.
Wow, wonderful blog layout! How long have you been blogging for?
you made blogging look easy. The overall look of your
site is excellent, as well as the content!
Good site you have got here.. It's hard to find good quality writing like yours these days.
I really appreciate individuals like you! Take care!!
It's very trouble-free to find out any topic on web as
compared to books, as I found this post at this website.
A person necessarily lend a hand to make critically articles I'd state.
That is the first time I frequented your web page and to this
point? I surprised with the research you made to
make this particular post incredible. Magnificent process!
I'm really loving the theme/design of your weblog. Do you ever run into any browser compatibility problems?
A handful of my blog visitors have complained about my blog not working correctly in Explorer but looks great
in Opera. Do you have any suggestions to help fix this issue?
I was recommended this blog by means of my cousin. I'm now not sure whether this post is written by him as no one else
recognize such specific approximately my trouble.
You're wonderful! Thank you!
Skyking, this drop is your next bit of info. Do transceive the agency at your convenience. No further information until next transmission. This is broadcast #4947. Do not delete.
I have read several just right stuff here. Certainly worth bookmarking for revisiting.
I surprise how a lot attempt you set to create the sort of wonderful informative
site.
Dreamwalker, this code is your next bit of info. Immediately contact the agency at your earliest convenience. No further information until next transmission. This is broadcast #3718. Do not delete.
Wow that was unusual. I just wrote an extremely long comment but after I clicked submit
my comment didn't appear. Grrrr… well I'm not writing all that over again. Anyways, just wanted to say excellent blog!
Right away I am ready to do my breakfast, when having
my breakfast coming again to read further news.
Today, I went to the beach front with my kids. I found a sea shell and gave
it to my 4 year old daughter and said "You can hear the ocean if you put this to your ear." She put the shell to her ear and screamed.
There was a hermit crab inside and it pinched her ear.
She never wants to go back! LoL I know this is totally
off topic but I had to tell someone!
I've been exploring for a little for any high-quality articles or
weblog posts on this kind of house . Exploring in Yahoo I eventually stumbled upon this web site.
Reading this info So i am happy to express that I've a
very good uncanny feeling I found out just what I needed.
I most surely will make sure to do not overlook this web site
and provides it a glance on a constant basis.
Hi, just wanted to tell you, I enjoyed this
blog post. It was funny. Keep on posting!
I'm not sure exactly why but this weblog is loading extremely
slow for me. Is anyone else having this issue or is it a problem on my end?
I'll check back later on and see if the problem still exists.
I like the helpful info you provide in your articles.
I will bookmark your blog and check again here frequently.
I am quite sure I will learn a lot of new stuff right here!
Good luck for the next!
I was extremely pleased to uncover this web site. I wanted to thank you for your time for this fantastic read!!
I definitely appreciated every part of it and I have
you saved as a favorite to see new things on your web site.
I don't even understand how I stopped up right here, however I thought this
post was good. I do not recognize who you are however definitely you're going to
a famous blogger for those who aren't already. Cheers!
Howdy! This post could not be written any better! Reading this post reminds me of my previous room mate!
He always kept chatting about this. I will forward this page to him.
Pretty sure he will have a good read. Thank you for sharing!
Very good post! We are linking to this particularly great post on our
site. Keep up the great writing.
Asking questions are actually fastidious thing if you are
not understanding something fully, but this post presents pleasant understanding even.
Le Meilleur Generique Du Cialis Online Pharmiacies [url=http://dan5325.com]online pharmacy[/url] Dapoxetina
very Great post, i actually love this web site, carry on it
I love reading through and I believe this website got some genuinely utilitarian stuff on it! .
This site truly has all the information I needed concerning this subject and didn't know who to
ask.
I’m not that much of a online reader to be honest but your blogs really nice, keep it up!
I'll go ahead and bookmark your website to come back later.
Cheers
Hello there! This is my 1st comment here so I
just wanted to give a quick shout out and tell you I truly enjoy reading your
articles. Can you recommend any other blogs/websites/forums that go
over the same topics? Thanks a lot!
Hello, i think that i saw you visited my site so i came to “return the favor”.I am trying to find things
to improve my website!I suppose its ok to use a few of your ideas!!
This article presents clear idea in support of the new
visitors of blogging, that genuinely how to do running a blog.
Ahaa, its good dialogue regarding this article at this place at this weblog, I
have read all that, so at this time me also commenting here.
Hello, I wish for to subscribe for this webpage to
get newest updates, therefore where can i do it please help.
natalielise pof
Enjoyed reading through this, very good stuff, thankyou .
I like this article, useful stuff on here : D.
I needed to thank you for this good read!! I absolutely enjoyed every little
bit of it. I have got you bookmarked to look at new stuff you post…
I'm really enjoying the design and layout of your
site. It's a very easy on the eyes which makes it much more pleasant
for me to come here and visit more often. Did you hire out a
designer to create your theme? Excellent work!
This does interest me
I really got into this web. I found it to be interesting and loaded with unique points of view.
Woah! I'm really digging the template/theme of this site.
It's simple, yet effective. A lot of times it's very difficult to get that "perfect balance" between usability
and appearance. I must say you've done a superb job with this.
Additionally, the blog loads extremely quick for me on Internet explorer.
Superb Blog!
Pretty nice post. I just stumbled upon your blog and wanted to say that I have really enjoyed
surfing around your blog posts. In any case I'll
be subscribing to your rss feed and I hope you write again soon!
I must say, as a lot as I enjoyed reading what you had to say, I couldnt help but lose interest after a while.
Thank You for this.
You really make it seem so easy with your presentation but
I find this matter to be really something which I think I would never understand.
It seems too complex and very broad for me. I'm looking forward for
your next post, I'll try to get the hang of it!
When some one searches for his vital thing, so he/she wants to be available that in detail,
thus that thing is maintained over here. plenty of fish natalielise
I like what you guys are usually up too. This sort of clever work
and exposure! Keep up the awesome works guys I've incorporated you guys to my blogroll.
Cheers, great stuff, Me like.
Respect to website author , some wonderful entropy.
My brother suggested I might like this blog. He was
totally right. This post actually made my day. You cann't imagine just how much time I had spent for this information! Thanks!
Amazing! Its truly awesome piece of writing, I have got much clear idea regarding from this piece of writing.
I think that is one of the most vital information for me.
And i am happy reading your article. But want to commentary on few basic
issues, The site style is wonderful, the articles is
in reality great : D. Excellent job, cheers
I don't even understand how I stopped up here, however I assumed this post was once good.
I don't realize who you are however certainly you're going to a famous blogger for those who aren't already.
Cheers!
I am curious to find out what blog platform you're using?
I'm having some small security problems with my latest blog and I'd like to find something more
safeguarded. Do you have any recommendations?
Way cool! Some very valid points! I appreciate you writing
this article and also the rest of the website is very good.
Wow that was odd. I just wrote an really long
comment but after I clicked submit my comment didn't appear.
Grrrr… well I'm not writing all that over again. Anyway, just wanted to
say great blog!