Things from Python I'd miss in Go

Let's assume Go has one killer feature – cheap concurrency – not present in any other language with similar serial performance (ruling out say Erlang). And let's agree that it gives Go its perfect niche – and we're only discussing uses of Go outside that niche from now on. So we won't come back to concurrency till the very end of the discussion. (Update – the "killer concurrency" assumption seems false… but anyway, we'll get to it later.)

***

Brian Kernighan once called Java "strongly-hyped". Certainly his ex-colleagues' Go language is one of the more strongly-hyped languages of today – and it happens to have a lot in common with Java in terms of what it does for you.

Rob Pike said he thought C++ programmers would switch to Go but it's Python programmers who end up switching. Perhaps some do; I, for one, am a Python programmer not planning to switch to Go – nor will I switch to Go from (the despicable) C++ or (the passable) C.

Why so?

What does Go have that C++ lacks? Mandatory garbage collection, memory safety, reflection, faster build times, modest runtime overhead. C++ programmers who wanted that switched to Java years ago. Those who still stick to C++, after all these years, either really can't live with the "overheads" (real time apps – those are waiting for Rust to mature), or think they can't (and nothing will convince them except a competitor eventually forcing their employer out of business).

What does Go have that Python lacks? Performance, static typing. But – again – if I needed that, I would have switched to Java from Python years before Go came along. Right?

Maybe not; maybe Java is more verbose than Go – especially before Java 8. Now, however, I'd certainly look at Java, though perhaps Go would still come out on top…

Except there are features I need that Go lacks (most of which Java lacks as well). Here's a list:

Dynamic code loading/eval

Many, many Python uses cases at work involve loading code from locations only known at run time – with imp.load_source, eval or similar. A couple of build systems we use do it, for instance – build.py program-definition.py, that kind of thing. Also compilers where the front-end evals (user's) Python code, building up the definitions that the back-end (compiler's code) then compiles down to something – especially nice for prototyping.

I don't think Go has a complete, working eval. So all those use cases seem basically impossible – though in theory you could try to build the eval'ing and eval'd code on the fly… Maybe in some cases it'd work. Certainly it's not something Go is designed for.

REPL

No REPL in Go – a consequence of not having eval. There are a few programs compiling Go on the fly, like go-repl, but you can't quite build up arbitrary state that way – you're not creating objects that "live" inside the session. Not to mention the following little behavior of Go – a production-friendly but prototyping-hostile language if there ever was one:

> + fmt
! fmt> fmt.Println("Hello, world!")
Hello, world!
! fmt> println("This won't work since fmt doesn't get used.")
Compile error: /tmp/gorepl.go:2: imported and not used: fmt

At home, I use DreamPie because I'm on Windows, where a scanner and a Wacom tablet work effortlessly, but there's no tcsh running under xterm. Perhaps go-repl could replace DreamPie – after all, you can't exactly build up state in live objects in tcsh which I'm quite used to. But I kinda got used to having "live objects" in DreamPie as a side effect of using it instead of tcsh.

And certainly Python REPLs already did and always will get more love than Go's because Python is designed for this kind of thing and Go is not.

numpy

So Python is slow and Go is fast, and just as readable, right? Try doing linear algebra with large matrices.

Python has numpy which can be configured as a thin wrapper around libraries like Intel's MKL – utilizing all of your cores and SIMD units, basically unbeatable even if you drop down to assembly. And the syntax is nice enough due to operator overloading.

Go doesn't have operator overloading, making scientific computing people or game programmers cringe. Go is ugly when it comes to linear algebra. Will it at least be as fast as Python with MKL? I doubt it – who'll continuously invest efforts to make something hopelessly ugly somewhat faster at these things? Even if someone does – even if someone did – who cares?

Why not have operator overloading? For the same reasons Java doesn't: "needless complexity". At least in Go there's a "real" reason – no exceptions, without which how would you handle errors in overloaded operators? Which brings us to…

No exceptions

Meaning that every time I open a file I need to clutter my code with an error handling path. If I want higher-level error context I need to propagate the error upwards, with an if at every function call along the way.

Maybe it makes sense for production servers. For code deployed internally it just makes it that much longer – and often error context will be simply omitted.

More code, less done. Yuck.

Update: a commenter pointed out, rather empathically I might add, that you can panic() instead of throwing an exception, and recover() instead of catching it, and defer() does what finally does elsewhere. Another commenter replied that the flow will be different in Go in some cases, because defer() works at a function level so you effectively need a function where you could use a try/finally block elsewhere.

So the tendency to return error descriptions instead of "panicking" is a library design style more than strictly a necessity. This doesn't change my conclusions, because I'd be using libraries a lot and most would use that style. (And libraries panicking a lot might get gnarly to use because of defer/recover not working quite like try/finally; "strict necessities" are not that different from this sort of "style choice".)

GUI bindings

We use Python's Qt bindings in several places at work. Go's Qt bindings are "not recommended for any real use" yet/"are at an alpha stage".

This might change perhaps – I don't know how hard making such bindings is. Certainly concurrent servers have no GUI and Go's designers and much of its community don't care about this aspect of programming.

Not unlike the numpy situation, not? Partly it's simply a question of who's been around for more time, partly of what everyone's priorities are.

All those things, combined

We have at work this largish program full of plugins that visualizes various things atop video clips, and we're in the process of making it support some kinds of increasingly interactive programming/its own REPL. Without eval, operator overloading for numeric stuff, the right GUI bindings or exceptions it doesn't seem easy to do this kind of thing. To take a generally recognizable example – Python can work as a Matlab replacement for some, Go can't.

Or, we do distributed machine learning, with numpy. What would we do in Go? Also – would its concurrency support help distribute the work? Of course not – we use multiple processes which Go doesn't directly support, and we have no use for multiple threads/goroutines/whatever.

Conclusion

Just looking at "requirements" – ignoring issues of "taste/convenience" like list comprehensions, keyword arguments/function forwarding etc. – shows that Go is no replacement for Python (at least to the extent that I got its feature set right).

What is Go really good at? Concurrent servers.

Which mature language Go competes with most directly? Java: rather fast (modulo gc/bounds checking), rather static (modulo reflection/polymorphic calls), rather safe (modulo – funnily enough – shared state concurrency bugs). Similar philosophies as well, resulting in feature set similarities such as lack of operator overloading – and a similar kind of hype.

(Updated thanks to an HN commenter) Does Go have a clear edge over Java when it comes to concurrency? With Quasar/Pulsar, maybe not anymore, though I haven't looked deep enough into either. Certainly if you're thinking about using Go, Java seems to be a worthy option to consider as well.

Would I rather be programming in Go than C or C++? Yes, but I can't. Would I rather be programming in Go than Python? Typically no, and I won't.

P.S.

Is there a large class of Python programmers who might switch to Go? Perhaps – people working on any kind of web server back-end code (blogs, wikis, shops, whatever). Here the question is what it takes to optimize a typical web site for performance.

If most of the load, in most sites, can be reduced drastically by say caching pages, maybe learning a relatively new language with a small community, less libraries and narrower applicability isn't worth it for most people. If on the other hand most code in most websites matters a lot for performance, then maybe you want Go (assuming other languages with similar runtime performance lack the cheap concurrency support).

I'm not that knowledgeable about server code so I might be off. But judging by the amount of stuff done in slow languages on the web over the years, and working just fine (Wikipedia looks like a good example of a very big but "static enough" site), maybe Go is a language for a small share of the server code out there. There are however Facebook/Twitter-like services which apparently proved "too dynamic" for slow languages.

Which kind of service is more typical might determine how much momentum Go ultimately gains. Outside servers – IMO if it gains any popularity, it will be a side-effect of its uses in servers. (And the result – such as doing linear algebra in Go – might be as ugly as using Node.js for server-side concurrency, "leveraging" JavaScript's popularity…)

And – why Python programmers would then be switching to Go but not C++ programmers? Because nobody is crazy enough to write web sites in C++ in the first place…

44 comments ↓

#1 David R. MacIver on 06.09.14 at 10:43 pm

"Because nobody is crazy enough to write web sites in C++ in the first place…"

Except (infamously) okcupid

#2 Yossi Kreinin on 06.09.14 at 10:53 pm

Seriously? All of it? Any interesting outcomes (like, I dunno, frisky messages to one user reaching another because of memory overruns or such?)

#3 Sundar on 06.09.14 at 11:25 pm

"Any interesting outcomes (like, I dunno, frisky messages to one user reaching another because of memory overruns or such?)"

Well, I hadn't seen any until today, but came across this on reddit only a minute ago: http://i.imgur.com/QDNSu0a.jpg Apparently, that's the maximum value for an unsigned 64 bit integer, so there's your integer underflow (I wonder why -1 people like him though, poor guy).

#4 Norman Yarvin on 06.09.14 at 11:37 pm

Go does sort of have exceptions:

http://blog.golang.org/defer-panic-and-recover

They're perhaps not quite the sort you'd want, but they are enough to make it so that you don't need to stick an if statement after every function call.

But as for "try doing linear algebra with large matrices", hell, try _declaring_ large matrices. Go doesn't make it easy. It has arrays, and you can make arrays of arrays (so far so good), but arrays are constant size, so you can't write a function that takes an N by N array of arrays as an input. For variable length arrays, Go has slices… but they're only one-dimensional. If you want two dimensions, you've got to use slices of slices, or use a single slice and do the address arithmetic by hand.

They actually could add multidimensional slices quite reasonably and cleanly, and I've told them they should, but they're not particularly interested:

https://code.google.com/p/go/issues/detail?id=6282

https://docs.google.com/document/d/1xHyOK3hSxwMtyMH-pbXSXxo7Td9A_Teoj6cbr9ECeLM

#5 Yossi Kreinin on 06.10.14 at 12:35 am

-1 likes – probably another bug… A joke about C++, coupling and decoupling seems fitting but I can't think of any.

Regarding defer, panic and recover – don't I need to stick a recover at every call if I want to log the call stack? And, I'd still need an if checking the error returned by opening files, right?

Regarding slices… yeah. C++ "mitigates" this problem with operator overloading as it does the problem of computing things given the matrices (and operator overloading is in itself not a great feature for a language like C++…)

#6 David R. MacIver on 06.10.14 at 3:14 am

Yossi: Not many that I know of. It's surprisingly reliable.

They've got their own webserver written in C++ (https://github.com/okws/okws) and do everything embedded in that. It's the sort of thing that should have gone horribly wrong and somehow seems not to have.

#7 Hraban on 06.10.14 at 3:51 am

Hi,

Don't forget generics / operator overloading / macros / anything that solves this:

http://godoc.org/flag#pkg-index

or:

http://godoc.org/sync/atomic#pkg-index

So much copy/pasting. And when you create your own library with type-agnostic functionality, you must follow the same pattern.

In practice, a lot of Go projects reimplement basic datastructures for their own datatypes. Result: bugs in code that shouldn't have been written in the first place.

Go semantics are so simplistic, it feels like a compilation target for a language with AST macros.

#8 Hraban on 06.10.14 at 3:52 am

Hi,

Don't forget generics / method overloading / macros / anything that solves this:

http://godoc.org/flag#pkg-index

or:

http://godoc.org/sync/atomic#pkg-index

So much copy/pasting. And when you create your own library with type-agnostic functionality, you must follow the same pattern.

In practice, a lot of Go projects reimplement basic datastructures for their own datatypes. Result: bugs in code that shouldn't have been written in the first place.

Go semantics are so simplistic, it feels like a compilation target for a language with AST macros.

#9 Caleb Cushing on 06.10.14 at 5:00 am

REPL, you know it surprises me that this is a requirement… I've never used one beyond learning a language, this only seems like it'd be useful when setting a breakpoint in a debugger and having it drop me to the REPL. I'd like to know what people who really make use of these really do with them.

#10 Bill Mill on 06.10.14 at 5:23 am

Caleb: Sometimes (depending on the task), I tend to build up my programs interactively at the REPL, testing little pieces of them there and moving them over into the code when they work. IPython is a superb tool for doing so, I recommend trying that style at least once.

#11 lmm on 06.10.14 at 5:39 am

I went from primarily-python to primarily-scala, so I'm curious how much you'd miss doing that. Scala probably doesn't have the linear algebra case down yet (though Spire is working at it), but we've got type-safety and performance without the verbosity of Java, we've got some limited form of eval (as a Java scripting engine) and a fully-functional REPL, we've got operator overloading and exceptions, and the JVM Qt bindings (smoke) are reasonably mature, though I can't speak to how easy it is to use them in idiomatic scala.

#12 smoe on 06.10.14 at 5:53 am

Caleb: REPL is definitely the feature I miss most in Go so far. For me its extremely handy in the following situations
- Debugging errors.
- Evaluating new libs
- Playing around with json apis
- Developing an idea without the distraction of the application it is surrounded
- Quickly modify objects with an orm
- Convert/restructure things
- Aa a calculator :)

#13 Jonathan Eunice on 06.10.14 at 6:27 am

I hear a lot about Python programmers embracing and flocking toward Go, but the cases I've seen detailed (e.g. Space Monkey) seem to be very much "well, duh!" situations: places concurrency & efficiency rule, and where the things that are great about Python are less critical.

I certainly wish Python had better out-of-the-box concurrency and efficiency. But I wouldn't give up its clarity, exceptions, reflection, or ecosystem unless I really, really had to.

For Pythonistas keener on the new-language shuffle, consider Julia as an option to Go. It's progressively typed, compiled with C/Java-like performance, and has rich metaprogramming. If I had to "leave Python for performance" (i.e. if PyPy and similar still not fast enough), Julia'd be worth a serious look.

#14 George on 06.10.14 at 6:36 am

Hmm I think python devs are changing from python to go cause of the differences between 2 and 3… And you can see how 2 is viable but python 3… who really wants to use it at the moment?

#15 junk science on 06.10.14 at 6:51 am

play.golang.org is not a full substitute for a REPL, but it is often enough for simple testing of ideas or syntax

Go is correct on "exceptions" despite your misgivings!

eval()…in twenty years of perl I never used it, so I don't miss it in Go

#16 Bob on 06.10.14 at 7:04 am

Exceptions are one of the features I hope NEVER gets put into Go. When I look at systemically buggy production code in Java the inability to debug the problem can, almost always, be traced back to exception handling. Seriously, how did we get the point were error checking is getting handled by the 21 century version of a goto statement?

#17 Wayne on 06.10.14 at 7:36 am

I'll stick with C# where I get every convenient language feature, the best tooling and good enough performance (better than Python at least) on the most stable and backwards compatible platform – Windows.

#18 Yossi Kreinin on 06.10.14 at 8:41 am

Generics – a problem for some, not for me, hence not mentioned. Macros – might be a problem because there's no eval, either; I don't see it'd be a problem in my use cases at first glance, hence not mentioned.

REPL as a requirement – think Matlab. A Matlab user will kill you if you take his REPL. Python replaces Matlab for some, Go won't.

Scala is much too complicated for my taste (and it's "taste" and no other argument). Go vs Python is a common argument in part because they're both kinda simplistic – "similar philosophies" – so for me Go is a more likely alternative than Scala in that sense (if it weren't for the missing features). Maybe some day I'll get a feeling that I don't care about Scala's complexity as it won't "leak" out of the implementation into something I have to deal with…

Julia – a guy actually advocated to use it; I didn't jump on the idea, maybe wrongly… I think numpy for dump linear algebra problems is unbeatable, but for hairy stuff Julia might shine; Julia gaining momentum might be a very good thing for people needing this type of stuff – certainly "if it works" in the many relevant ways it has a good chance to beat Python convincingly as the generally better alternative.

Going to Go primarily because of the Python 2/3 clusterfuck is kinda crazy, much as I think 2/3 is a dumb fork.

#19 srathbun on 06.10.14 at 9:59 am

@CalebCushing

I wrote my code such that the sqlalchemy backend was independently importable to my ipython repl. That gave me access to all of my db stuff right there for testing out sql queries or seeing the results of function calls. Totally worth it for seeing what my program would be doing.

#20 smoe on 06.10.14 at 10:05 am

I'm not thinking of switching from Python to Go completely but experimenting with it for certain use cases.

For example in my current site project almost all of the logic is within background workers who scrape data, analyze and categorize it. The Public API is basically just doing fixed sql queries on that prepared data.
So I don't have any real benefits using a dynamic language like python.

I planned to use Scala first because it seems to be a better match with Python in machine learning projects, but the learning curve was too high for me and I didn't want to lose too much time.

#21 Mohamed on 06.10.14 at 12:13 pm

Funny how the article and most of the comments leaves the impression that adopting a language requires exclusivity. Projects define the language you need, not the opposite. Hell you can even combine many of them if you plan wisely and carefully to leverage the strength of each one of them. You don't have to give up on Python if you think Go fits best in a situation.

sorry for stating the evidence.

#22 Norman Yarvin on 06.10.14 at 9:26 pm

If you want to log the call stack in Go, there's a function in the "runtime" package that returns it.

But yes, you still need an if statement to check for errors when opening files; Go's designers want it that way. They think that using exceptions to handle such errors makes for confusing code, what with its action-at-a-distance behavior: you have an open call throwing an exception in one place, and it being handled in another, whereas with if statements the check is right after the open. Of course if you don't want to catch the exception, but just bomb out on failure, that criticism doesn't apply; in that case the if statement is just more boilerplate code that you wouldn't have to enter in a language where errors were delivered via exceptions.

#23 Dobrosław Żybort on 06.10.14 at 11:52 pm

"Because nobody is crazy enough to write web sites in C++ in the first place…"
There are even web frameworks in C++: CppCMS, Duda.io, Wt Web Toolkit, TreeFrog Framework

And you should check:
http://www.techempower.com/benchmarks/

#24 RickyS on 06.11.14 at 1:35 am

1. "Machine Learning" is a lot of things. Many of them process very large data sets, where interpreted languages don't shine.

2. What percentage of apps do Linear Algebra calculations?

3. Languages live and die by their libraries and library-like functionality. The Go system of packages makes building and using libraries very nice. But that's not enough of a reason to switch from Python.

4. Lots of business apps, online stores, multi-terminal customer support, automating chain stores, day-trading, calling taxis, discussion forums, and so on are server-like. They have a large number of people online and a series of servers at headquarters or in the cloud. Go does very well at that.

5. Go is easy to learn and to use, and for a lot of shops may be a very good 2nd language. Depending…

6. If you want numpy and repl, Go is not your language.

#25 Christian on 06.11.14 at 1:38 am

I've experience in Python, Go, and Java. I liked Go because of its simplicity, but then I learned that it is a bit too simple: no generics, no good mocking support, missing libraries, ugly builds with C dependencies, primitive doc format and so on. I felt a lot less productive than in Java.

I'd suggest to try Java 8 or Scala. You get a huge ecosystem, the best tools and good performance.

#26 RickyS on 06.11.14 at 4:18 am

If the only effect of Go was to replace and displace PHP, then Go is making a great service to humanity.

#27 Ccx on 06.12.14 at 2:47 am

>> Performance, static typing. But – again – if I needed that, I would have switched to Java from Python years before Go came along. Right? <<

These are the reasons I'm investigating OCaml right now. Similar internal design to python (GIL and all that), but funcional with data types inference so you won't get java-crazy. And compiles down to efficient native binaries. Julia is also tempting though, as is Mercury.

#28 Larry Clapp on 06.12.14 at 7:14 pm

I'm a Go aficionado who writes Perl during the day. And I would just like to say: Fair enough.

Not all languages are for everyone. I read Programming Python years ago and basically felt kind of "meh" at the end. If Go isn't your style, I'd be the first to say, Don't do that then.

I hear you on the REPL. I use zsh the same way. Some of my "one liners" get pre-tty long! And the Lisp REPL was nice when I was playing with Lisp.

That being said … fwiw I've found that TDD can help ameliorate the pain of not having a REPL. And there's "gore" (https://github.com/sriram-srinivasan/gore), which gives you the Read-Eval part (and print if you want to), just no loop, and thus no persistent state.

Thanks for the article. I enjoyed it.

#29 Alexandre Zani on 06.13.14 at 5:22 am

I can't believe that nobody mentioned go's compiler's confusion of lint errors with syntax errors. Try to write an expression, assign its result to a variable and compile to see if you made a mistake. Well, you did because you're not using that variable and god forbid that an unused variable ever make it into production. I don't know about others, but I'm constantly compiling half-baked code to help me catch stupid mistakes early and go makes that very difficult.

#30 Yossi Kreinin on 06.13.14 at 6:47 am

Actually I sorta did mention it in my REPL example, though this one was unused imports. But I'm a bit ambivalent on this one; I tend to compile with -Werror, sometimes with -Wno-unused and sometimes without. Unused variables can be bugs, on the other hand they very often crop up during development. The trouble is, it's not clear when to turn off -Wno-unused.

#31 Ivan Tikhonov on 06.13.14 at 7:03 am

There is at least G-WAN and Wt. So i assume at least some people do write websites in C/C++

#32 Alexandre Zani on 06.13.14 at 10:03 am

How you want to deal with that will depend a lot upon the system you are developing and your development infrastructure. But I think it's fairly easy to setup a pre-commit hook that will try to compile your binary with strict flags. (And run the fast part of your test suite while you're at it) Similarly, if you have a deployment "process", you can also build your binary with stricter flags there.

To be honest, that complaint is a big reason why I gave up on go. It slowed down my code-compile-run cycle in a very frustrating way. The fact that failed assertions in unit tests print no useful information is another reason.

#33 Jason on 06.15.14 at 2:02 pm

Alexandre: re tests giving little info: agreed. GoConvey fixes this for me.

#34 Jason on 06.15.14 at 2:09 pm

Yossi, I miss the repl too. I think it will happen for go by 1.4, or 1.5 at the latest. The only missing pieces to support a repl have already been done, they just aren't merged into the mainline release yet.

#35 Sebastien on 06.16.14 at 1:03 pm

wrt the repl…

There is github.com/sbinet/igo
which I salvaged from the go-1 clean up.
It's by no means complete but still useful at times.

When go will support shared libraries, imports would probably be easier to implement and provide.

#36 Yossi Kreinin on 06.28.14 at 10:18 am

test

#37 Johan Ouwerkerk on 07.03.14 at 4:05 am

As it happens your local Java implementations might well have the necessary hooks which let you do REPL stuff with it, based on the 'javax.script' framework. It allows you to run, for example a hosted Python interpreter in your Java application and/or call code across language boundaries.

So if you want the dynamic stuff back you can do that relatively straightforwardly: port the code for which performance matters to, say, Java and keep most everything else as a script.

#38 cmccabe on 07.05.14 at 10:57 am

It really comes down to using the right tool for the job. Engineering is all about tradeoffs, and there is no magic programming language which is optimal for every use-case.

It's nice to have eval(), because then you can have an interactive shell or a REPL. Creating certain forms of self-modifying code become easier. On the other hand, if you have an eval(), then you are vulnerable to various forms of malicious code injection, and it becomes harder to reason about what any given program will do. You also have a lot of deployment complexity when loading code dynamically. Just mention the word "classpath" to a Java devops engineer and watch their reaction. (Note that C/C++ style loadable modules are on the Golang roadmap, so it will at least have that soon.)

You can write shorter and more elegant code when using exceptions instead of return codes. On the other hand, it becomes more difficult to reason about what happens when there are failures. Suddenly every function has almost an infinite number of exit points. "finally" blocks and "catch" clauses can help, but only at the cost of introducing even more ugliness and verbosity than return codes.

Operator overloading makes matrix math look a lot nicer. And probably a lot of other mathematical notations. But suddenly, to quote your own essays about C++, you don't even know what code is executed by "a + b" any more. If you crash deep inside some operator code, the call stack will look like a dog's breakfast. And of course, operator overloading requires exceptions.

These are fundamental tradeoffs in programming language design. Golang chose to make tradeoffs that prioritize robustness, simplicity, and high performance for servers, at the cost of expressiveness, interactivity, and flexibility. It turns out that for a fairly large class of applications, Golang's choices are excellent. And that (combined with Google's clout) is why the language is seeing so much adoption.

Python chose to make tradeoffs that are good for scripting and perhaps good for doing some types of scientific computing (where you don't need low-level optimizations, or can rely on pre-constructed optimized libraries). The mass migration from Python to Go is not about scientific computing or scripting, but about people who are just finding that servers are a lot easier in Go. And that's fine. Python is going to keep on keeping on, and nobody will blame you for sticking around (but keep an eye on Julia and R, of course…)

I think you're absolutely right that Java is the closest competitor to Golang. But unfortunately, your post made almost no useful comparisons between Java and Golang. If you think that the threading model is the only thing different between Java and Golang, you're seriously mistaken. The whole type system is different in fundamental ways, the compilation model is different, error handling is different, the runtime is completely different, etc. etc. Goroutines and slices is just the tip of the iceberg. Don't take this the wrong way, but your offhand comment that with Java 8 plus a library or two, Java and Go will basically be the same really reveals that you need to do more research here.

Anyway, as always, a very thought-provoking post. I hope you get a chance to tinker with Go, Rust, and some of the other new languages at some point. It's definitely fun to try something new. And you just might learn about something that takes your mind off Java :)

#39 Yossi Kreinin on 07.05.14 at 4:47 pm

I didn't say they were the same; I think they're fairly different despite profound similarities in their marketing, many core features and even the bondage and discipline approach ("languages for someone else" – "they're not researchers, they're Googlers, people straight out of college" to quote Rob Pike; Java's and Go's compiler-error-based error handling approaches are actually not quite different…).

But looking at those differences, is Go better?

* Java is way more portable today

* Java's IDE support is almost certainly better

* Java's CPU performance is almost certainly better (http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=all&lang=go&lang2=java&data=u32)

* Perhaps most importantly for power users – hackability of Java/JVM almost certainly beats Go's; for instance what you called "a couple of libraries" involves pretty heavy metaprogramming where coroutines are added to Java:

"Of the three pieces, coroutines are the most challenging to implement on the JVM, as they require bytecode instrumentation. Essentially, every possibly-pausing method (the Quasar/Pulsar nomenclature is “suspendable”) must be inspected to find all invocations of other suspendable methods. Before the call to a suspendable method, code must be injected to push the caller’s local variables onto a stack object. Also, code must be injected to the beginning of every suspendable method, that upon resuming would jump to the instruction following the pause-point. This has to be done to all suspendable methods on the (OS thread) stack.

Fortunately, I found Matthias Mann’s continuations library that not only does exactly that and only that, but is also small, fast and easy to understand."

So Java doesn't have coroutines but JVM metaprogramming lets you add them. Could you add coroutines to Go if they didn't come baked in with any degree of portability and without hacking your compiler? My bet is you couldn't. Similarly, there's a ton of languages atop JVM which interoperate nicely with Java. I don't think it's nearly as easy with Go. (And I'm sure Go's non-OO type system, introduced with much fanfare and arrogance, doesn't make bindings to popular libraries any easier.)

As to things like $CLASSPATH – "it's complicated"; certainly static linking makes deployment easier – and it also makes fixing bugs harder (more binaries to recompile), it makes some sorts of debugging harder (think ltrace), etc. Certainly once Go adds support to dynamic loading it'll have all the problems – and advantages – of $CLASSPATH, $LD_LIBRARY_PATH, $PYTHONPATH or whatever it is your devops friends hate the most…

What is the awful thing Java has that Go doesn't? A culture of verbosity. This is the one thing that I find off-putting about Java and given Go's momentum if I were to write a highly concurrent server I might choose Go over Java.

That said, it would be better for everyone if "Go" was a set of Java libraries adding coroutines and maybe less verbose ways to do things. Go adds a huge amount of duplication (think all the library functionality implemented in Java as well as Go and umpteen other languages) and for this cost "society", this amorphous being that has no representative body but does exist, basically got coroutines and a select statement that could be done elsewhere perfectly well.

Of course knowing Go's authors noone would expect anything "less" than starting a new language… And it could be worse…

Did I do "enough research" on Go and Java? Perhaps not, and I'd be interested to hear more details about those things that Go does better.

As to Rust – Rust is nice, and Rust is not redundant. Rust might actually replace C++ for embedded type of work some day and I'm certainly following it with much anticipation. Incidentally, the vibe coming off Rust developers doesn't rub me the wrong way the way Go's echo chamber does. That despite Go being more of a "get shit done" culture which I tend to like, and the Rust culture being perhaps overly "thoughtful" to my taste.

#40 cmccabe on 07.16.14 at 12:24 am

Golang is portable to MacOS, the BSDs, Linux, and Windows. That covers pretty much all the platforms that matter for server code. There is ARM support for embedded systems as well. The Go guys have done a great job of being responsive to bug reports and feature requests on alternate platforms, and there is an active culture of go developers on Windows… probably to a larger extent than many of the other languages we're discussing here.

Just about the only important thing you can't run Go on is a microcontroller, but you can't run Java or Python there either, so this is a silly discussion. Go isn't available on iOS and Android either, but neither are most other languages, for completely non-technical reasons.

Your portability comment particularly rankles me because I've spent a lot of time re-implementing platform support that Sun left out of Java because of their "less is more" portability approach. For example, I implemented this code to fix the fact that they left out UNIX domain sockets:
https://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/unix/DomainSocket.java

The issue is that if there was a platform-specific feature that could not be supported on all platforms, Java just left it out. Sometimes those platform-specific features are crucial, though. For example, without UNIX domain sockets, you can't implement file descriptor passing, and a whole class of optimizations become impossible. Golang takes the more sensible approach of making all platform features available if you want them, but also making it possible to write portable code if you want that too. Java is starting to fix some of these bad portability decisions… slowly. But a lot of them still bite, if you use the language for real.

The IDE support comment seems very unfair. Golang is a statically typed language with an extremely easy-to-parse grammar. IDE support is simple to add, and a lot of people are doing so. IntelliJ / IDEA is probably the best option if you want something familiar. It is trivial to create a "smart" IDE for Go because all the type information is known at compile time and the syntax is regular and consistent. I can't say the same thing for C++ or Python, languages which will always have poor IDE support until we invent an IDE that can pass the Turing test.

Let's talk about coroutines for a moment. Java had coroutines in the past. In fact, coroutines were the first thing that Java had… in version 1.1, using OS-level threads was not even an option. http://en.wikipedia.org/wiki/Green_threads#Green_threads_in_the_Java_virtual_machine. People have added coroutines to all sorts of low-level languages. Here's an example of adding coroutines to C: http://dunkels.com/adam/pt/ .

What makes Go better is not that it has coroutines, but that they're so thoroughly integrated into the language. When you're choosing libraries, you don't need to worry about coroutine support… you know all the libraries will support it. You will never be able to say the same about C, C++, or Java. I've written a substantial amount of non-blocking code, even a non-blocking RPC system in C++. I know what it's like. Basically you are constantly hoping, pleading with the compiler that you won't call a function that will block the OS thread for a long period. And it sucks. And the language chooses intelligently how many OS-level threads to use, rather than making it your problem.

It's frustrating that you attribute Java's problems to "a culture of verbosity." There are fundamental reasons why Java is verbose, and they flow from the type system, not from "a culture" or individual bad programmers.
I wrote about this here: http://www.club.cc.cmu.edu/~cmccabe/blog_golang_type_system.html

> "As to things like $CLASSPATH – "it's complicated"; certainly
> static linking makes deployment easier – and it also makes fixing
> bugs harder (more binaries to recompile), it makes some sorts of
> debugging harder (think ltrace), etc"

If you try out Go, you'll quickly find that it has lightning-fast compile times, even for very large projects. This makes the need to recompile to fix a bug not very important. In general, I don't buy the argument that libraries allow quicker deployment to nodes than static binaries. Maybe libraries would allow you to push out 1 MB to all your nodes rather than 10 MB. Are you using networking hardware made in the last 10 years? If so, you shouldn't care about 1 MB versus 10. This is why Google links their C++ binaries statically. The company I work for does the same with our C++ binaries, to the extent that it's practical.

Libraries make debugging harder most of the time. Even getting a stack trace becomes a challenge, because you can't easily map addresses to function names (different loading point each time). And if you don't have the same library versions as the customer did when they experienced the problem, most of the time you can just forget about debugging library issues.

Loadable modules are absolutely not the same thing as CLASSPATH. CLASSPATH is mandatory, even for programs that never want to load anything dynamically. That's why every Java program has to come with a loathsome shell script to chaperone it around. This would all have been a little bit better if Java had established a C-style convention about where to find those JAR libraries for each application, but, well… they didn't. And every program chose a different convention, and a different shell script to set up that convention.

> "As to Rust – Rust is nice, and Rust is not redundant. Rust
> might actually replace C++ for embedded type of work some
> day and I'm certainly following it with much anticipation."

I think you are being really unfair to Go. I've written a lot of C++ and Java code, and a little bit of Go code, and I like what I found. There are tons of features that you don't even know about. For example, Go's lightning-fast build system that doesn't even rely on Makefiles / pom files / any kind of crusty build script. These guys learned the lessons of large-scale software development well.

I agree that Rust is promising, because of the ability to avoid garbage collection. But we need to reserve judgement on Rust until they actually declare the language stable and start building an ecosystem. Then we will see what falls out.

#41 cmccabe on 07.16.14 at 12:25 am

"And it sucks. And the language chooses intelligently how many OS-level threads to use, rather than making it your problem."

Er, that should read, "And it sucks. Whereas in Golang, the language chooses…"

#42 Yossi Kreinin on 07.16.14 at 8:36 pm

You know, it's sad how what I wrote reads to you. I'm thinking to maybe publish a followup post instead of a reply…

I do know about Go's fast builds, I didn't praise them because Python isn't lacking in that department; just like I didn't mention Go's lack of generics because Python doesn't have them, either (and I don't think it's such a crippling thing for the kinds of things I want to do, regardless of what others are doing.)

Maybe a followup post…

I never said Go wasn't a smart thing for Google to do, or that it's not a smart thing to do in a server. I only said that Go does nothing for me that Java didn't, that it's a lot like Java and implicitly I assumed in all that that a new language is a big cost for society and how wonderful it would be if Go "leveraged" (love the word) an existing language/VM instead of being its own language reimplementing endless library functionality and fragmenting the library base for all of humanity for what to me seems like insufficient reasons to do so. Maybe it's too much like wishing for world peace though… Java, C#, Go, Swift…

Maybe a followup post. I do have answers to your points but at a larger level it's sad that I come across to you the way I apparently do and maybe I could reexplain myself…

#43 cmccabe on 07.17.14 at 2:43 am

I think you made some valid points about Golang not replacing Python in all use cases. numpy is pretty cool. I actually open up an interactive Python shell whenever I need to do some moderately complex calculation, and I write scripts to generate graphs and so forth. numpy has some graph generation stuff which is awesome for putting together a presentation.

It would be nice to see a reasonably thorough comparison between Java and Go. Maybe someday I'll try to write one. I can't claim to be unbiased, but maybe I can claim to be "properly biased" by having used both languages for a while.

Re-using the JVM is a double-edged sword. You can get your language off the ground faster, and the JVM is pretty highly optimized. And you get the ability to link with a lot of existing software. In return you give up: the ability to allocate objects on the stack (all objects are allocated on the heap in the JVM, except primitives), the ability to efficiently implement a non-inheritance-based type model like Go's, and fast startup times. Fast startup times seem like a trivial thing, but when sysadmins start writing shell scripts to call a bunch of tools for your project, it doesn't seem so trivial. (We have a lot of people convinced that HDFS is super slow because they wrote a shell script to upload files in a loop, spawning a new JVM each time to upload each file. Yeah, these people are dumb but… can you blame them for wanting convenience?) Rob Pike has said that you might need to add new bytecodes to the JVM to efficiently support Go. The other big issue is that once you start linking to Java code directly, you can no longer assume that all the libraries you're calling support coroutines well– since you have to support calling any old crazy code that someone wrote in 1999 for JDK 1.0.

Looking forward to your followup. Your posts are always interesting and well thought-out, whether or not I agree.

#44 Yossi Kreinin on 07.17.14 at 8:55 am

Rob Pike discussed the option of "Go on the JVM"? Interesting!

Leave a Comment