Go: the Good, the Bad and the Ugly

Posted on Tue 10 April 2018 Updated on Mon 22 June 2020

This is an additional post in the “Go is not good” series. Go does have some nice features, hence the “The Good” part in this post, but overall I find it cumbersome and painful to use when we go beyond API or network servers (which is what it was designed for) and use it for business domain logic. But even for network programming, it has a lot of gotchas both in its design and implementation that make it dangerous under an apparent simplicity.

What motivated this post is that I recently came back to using Go for a side project. I used Go extensively in my previous job to write a network proxy (both http and raw tcp) for a SaaS service. The network part was rather pleasant (I was also discovering the language), but the accounting and billing part that came with it was painful. As my side project was a simple API I thought using Go would be the right tool to get the job done quickly, but as we know many projects grow beyond their initial scope, so I had to write some data processing to compute statistics and the pains of Go came back. So here's my take on Go woes.

Some background: I love statically typed languages. My first significant programs were written in Pascal. I then used Ada and C/C++ when I started working in the early 90's. I later moved to Java and finally Scala (with some Go in between) and recently started learning Rust. I've also written a substantial amount of JavaScript, because up to recently it was the only language available in web browsers. I feel insecure with dynamically typed languages and try to limit their use to simple scripting. I'm comfortable with imperative, functional and object oriented approaches.

This is a long post, so here's the menu to whet your appetite:

The Good

Go is easy to learn

That's a fact: if you know any kind of programming language, you can learn most of Go's syntax in a couple of hours with the "Tour of Go", and write your first real program in a couple of days. Read and digest Effective Go, wander around in the standard library, play with a web toolkit like Gorilla or Go kit and you'll be a pretty decent Go developer.

This is because Go's overarching goal is simplicity. When I started learning Go it reminded me when I first discovered Java: a simple language and a rich but not bloated standard library. Learning Go was a refreshing experience coming from today's Java heavy environment. Because of Go's simplicity, Go programs are very readable, even if error handling adds quite some noise (more on this below).

But this may be false simplicity though. Quoting Rob Pike, simplicity is complicated, and we will see below that behind it there are a lot of gotchas waiting to bite us, and that simplicity and minimalism prevent writing DRY code.

Easy concurrent programming with goroutines and channels

Goroutines are probably the best feature of Go. They're lightweight computation threads, distinct from operating system threads.

When a Go program executes what looks like a blocking I/O operation, the Go runtime actually suspends the goroutine and resumes it when an event indicates that some result is available. In the meantime other goroutines have been scheduled for execution. We therefore have the scalability benefits of asynchronous programming with a synchronous programming model.

Goroutines are also lightweight: their stack grows and shrinks on demand, which means having 100s or even 1000s of goroutines is not a problem.

I once had a goroutine leak in an application: these goroutines were waiting for a channel to be closed before ending, and that channel was never closed (a common issue). The process was eating 90% of the CPU for no reason, and inspecting expvars showed 600k idle goroutines! I guess the CPU was used by the goroutine scheduler.

Sure, an actor system like Akka can handle millions of actors without a sweat, in part because actors don't have a stack, but they're far from being as easy to use as goroutines to write heavily concurrent request/response applications (i.e. http APIs).

Channels are how goroutines should communicate: they provide a convenient programming model to send and receive data between goroutines without having to rely on fragile low level synchronization primitives. Channels come with their own set of usage patterns.

Channels have to be thought out carefully though, as incorrectly sized channels (they're unbuffered by default) can lead to deadlocks. They also have a large number of gotchas and inconsistencies. We will also see below that using channels doesn't prevent race conditions because Go lacks immutability.

Great standard library

The Go standard library is really great, particularly for everything related to network protocols or API development: http client and server, crypto, archive formats, compressions, sending email, etc. There's even an html parser and a rather powerful templating engine to produce text & html with automatic escaping to avoid XSS (used for example by Hugo).

The various APIs are generally simple and easy to understand. They can sometimes look simplistic though: this is in part because the goroutine programming model means we just have to care about "seemingly synchronous" operations. This is also because a few versatile functions can also replace a lot of specialized ones as I found out recently for time calculations.

Go is performant

Go compiles to a native executable. Many users of Go come from Python, Ruby or Node.js. For them, this is a mind-blowing experience as they see a huge increase in the number concurrent requests a server can handle. This is actually pretty normal when you come from interpreted languages with either no concurrency (Node.js) or a global interpreter lock. Combined to the language simplicity, this explains part of the excitement for Go.

Compared to Java however, things are not so clear in raw performance benchmarks. Where Go beats Java though is on memory usage. Unless you're using Graal native-image which puts them in the same ballpark.

Go's garbage collector is designed to prioritize latency and avoid stop-the-world pauses, which is particularly important in servers. This may come with a higher CPU cost, but in a horizontally scalable architecture this is easily solved by adding more machines. Remember that Go was designed at Google, who are all but short on resources!

Compared to Java, the Go GC also has less work to do: a slice of structs is a contiguous array of structures, and not an array of pointers like in Java. Similarly Go maps use small arrays as buckets for the same purpose. This means less work for the GC, and also better CPU cache locality.

Go also beats Java for command-line utilities: being a native executable, a Go program has no startup cost contrarily to Java that first has to load and compile bytecode.

Language defined source code format

Some of the most heated debates in my career happened around the definition of a code format for the team. Go solves this by defining a canonical format for Go code. The gofmt tool reformats your code and has no options.

Like it or not, gofmt defines how Go code should be formatted and that problem is therefore solved once for all!

Standardized test framework

Go comes with a great test framework in its standard library. It has support for parallel testing, benchmarks, and contains a lot of utilities to easily test network clients and servers.

Go programs are great for operations

Compared to Python, Ruby or Node.js, having to install a single executable file is a dream for operations engineers. This is less and less an issue with the growing use of Docker, but standalone executables also means tiny Docker images.

Go also has some built-in observability features with the expvar package to publish internal statuses and metrics, and makes it easy to add new ones. Be careful though, as they are automatically exposed, unprotected, on the default http request handler. Java has JMX for a similar purposes, but it's much more complex.

Defer statement, to avoid forgetting to clean up

The defer statement serves a purpose similar to finally in Java: execute some clean up code at the end of the current function, no matter how this function is exited. The interesting thing with defer is that it's not linked to a block of code, and can appear at any time. This allows the clean up code to be written as close as possible to the code that creates what needs to be cleaned up:

file, err := os.Open(fileName)
if err != nil {
    return
}
defer file.Close()

// use file, we don't have to think about closing it anymore

Sure, Java's try-with-resource is less verbose and Rust automatically claims resources when their owner is dropped, but since Go requires you to be explicit about resource clean up, having it close to resource allocation is nice.

New types

I love types, and something that irritates/scares me is when for example we pass around persisted object identifiers as string or long everywhere. We usually encode the id's type in the parameter name, but this is a cause of subtle bugs when a function has several identifiers as parameters and some call mismatches parameter order.

Go has first-class support for new types, i.e. types that take an existing type and give it a separate identity, distinct from the original one. Contrarily to wrapping, new types have no runtime overhead. This allows the compiler to catch this kind of mistake:

type UserId string // <-- new type
type ProductId string

func AddProduct(userId UserId, productId ProductId) {}

func main() {
    userId := UserId("some-user-id")
    productId := ProductId("some-product-id")

    // Right order: all fine
    AddProduct(userId, productId)

    // Wrong order: would compile with raw strings
    AddProduct(productId, userId)
    // Compilation errors:
    // cannot use productId (type ProductId) as type UserId in argument to AddProduct
    // cannot use userId (type UserId) as type ProductId in argument to AddProduct
}

Unfortunately the lack of generics makes the use of new types cumbersome as writing reusable code for them requires to cast values to/from the original type.

The Bad

Go ignored advances in modern language design

In Less is exponentially more, Rob Pike explains that Go was meant to replace C and C++ at Google, and that its precursor was Newsqueak, a language he wrote in the 80's. Go also has a lot of references to Plan9, a distributed operating system the authors of Go developed in the 80's at Bell Labs.

There's even a Go assembly directly inspired from Plan9. Why not using LLVM that would have provided a wide range of target architectures out of the box? I may also be missing something here, but why is that needed? If you need to write assembly to get the most out of the CPU, shouldn't you use directly the target CPU assembly language?

Go creators deserve a lot of respect, but it looks like Go's design happened in a parallel universe (or their Plan9 lab?) where most of what happened in compilers and programming language design in the 90's and 2000's never happened. Or that Go was designed by system programmers who were also able to write a compiler.

Functional programming? No mention of it. Generics? You don't need them, look at the mess they produced in C++! Even if slice, map and channels are generic types as we'll see below.

Go's goal was to replace C and C++, and it's apparent that its creators didn't look much elsewhere. They missed their target though, as C and C++ developers at Google didn't adopt it. My guess is that the primary reason is the garbage collector. Low level C developers fiercely reject managed memory as they have no control on what happens and when. They like this control, even if it comes with additional complexity and opens the door to memory leaks and buffer overflows. Interestingly, Rust has taken a completely different approach with automatic memory management without a GC.

Go instead attracted users of scripting languages like Python and Ruby in the area of operation tools. They found in Go a way to have great performance and reduced memory/cpu/disk footprint. And more static typing too, which was new to them. The killer app for Go was Docker, that triggered its wide adoption in the devops world. The rise of Kubernetes strengthens this trend.

Interfaces are structural types

Go interfaces are like Java interfaces or Scala & Rust traits: they define behaviour that is later implemented by a type (I won't call it "class" here).

Unlike Java interfaces and Scala & Rust traits though, a type doesn't need to explicitly specify that it implements an interface: it just has to implement all functions defined in the interface. So Go interfaces are actually structural typing.

We may think that this is to allow interface implementations in other packages than the type they apply to, like class extensions that exist in Scala or Kotlin, or Rust traits, but this isn't the case: all methods related to a type must be defined in the type's package.

Go isn't the only language to use structural typing, but I find it has several drawbacks:

  • finding what types implement a given interface is hard as it relies on function definition matching. I often discover interesting implementations in Java or Scala by searching for classes that implement an interface.

  • when adding a method to an interface, you will find what types need to be updated only when they are used as values of this interface type. This can go unnoticed for quite some time. Go recommends to have tiny interfaces with very few methods, which is a way to prevent this.

  • a type may unknowingly implement an interface because it has the corresponding methods. But being accidental, the semantics of the implementation may be different from what is expected from the interface contract.

Update: for some ugliness with interfaces, see nil interface values below.

Interface methods don't support default implementations

Added after the Go 1.13 release. That may not seem like a big deal, but read on.

Go 1.13 introduced method chaining, adding a new Unwrap method to errors. Since Go interfaces don't support default implementations for their methods, adding a method to an existing interface would break a lot of existing code. So this new method is a "convention" rather than being part of the error interface. And because of that we can't just call err.Unwrap() to get the wrapped error. We have to use the separate function errors.Unwrap(err) which uses dynamic typing tests to check if Unwrap exists on its parameter.

Bye-bye compile-time checks, hello cumbersome syntax for what could have been a simple method call! Java faced a similar issue in JDK8 with the introduction of lambdas, and added default method implementation support to allow interfaces to evolve in a backwards-compatible way.

No enumerations

Go doesn't have enums, and in my opinion it's a missed opportunity.

There is iota to quickly generate auto-incrementing values, but it looks more like a hack than a feature. And a dangerous one, actually, since inserting a line in a series of iota-generated constants will change the value of the following ones. Since the generated value is the one that is used throughout the code, this can lead to interesting (not!) surprises.

This also means there is no way in Go to have the compiler check that a switch statement is exhaustive, and no way to describe the allowed values in a type.

The := / var dilemma

Go provides two ways to declare a variable and assign it a value: var x = "foo" and x := "foo". Why is that?

The main differences are that var allows declaration without initialization (and you then have to declare the type), like in var x string, whereas := requires assignment and allows a mix of existing and new variables. My guess is that := was invented to make error handling a bit less painful:

With var:

var x, err1 = SomeFunction()
if (err1 != nil) {
  return nil
}

var y, err2 = SomeOtherFunction()
if (err2 != nil) {
  return nil
}

With :=:

x, err := SomeFunction()
if (err != nil) {
  return nil
}

y, err := SomeOtherFunction()
if (err != nil) {
  return nil
}

The := syntax also easily allows to accidentally shadow a variable. I was caught more than once by this, as := (declare and assign) is too close too = (assign), as shown below:

foo := "bar"
if someCondition {
  foo := "baz"
  doSomething(foo)
}
// foo == "bar" even if "someCondition" is true

Zero values that panic

Go doesn't have constructors. Because of that, it insists on the fact that the "zero value" should be readily usable. This is an interesting approach, but in my opinion the simplification it brings is mostly for the language implementors.

In practice, many types can't do useful things without proper initialization. Let's look a the io.File object that is taken as an example in Effective Go:

type File struct {
    *file // os specific
}

func (f *File) Name() string {
    return f.name
}

func (f *File) Read(b []byte) (n int, err error) {
    if err := f.checkValid("read"); err != nil {
        return 0, err
    }
    n, e := f.read(b)
    return n, f.wrapErr("read", e)
}

func (f *File) checkValid(op string) error {
    if f == nil {
        return ErrInvalid
    }
    return nil
}

What can we see here?

  • Calling Name() on a zero-value File will panic, because its file field is nil.

  • The Read function, and pretty much every other File method, starts by checking if the file was initialized.

So basically a zero-value File is not only useless, but can lead to panics. You have to use one of the constructor functions like Open or Create. And checking proper initialization is an overhead you have to pay at every function call.

There are countless types like this one in the standard library, and some that don't even try to do something useful with their zero value. Call any method on a zero-value html.Template: they all panic.

And there is also a serious gotcha with map's zero value: you can query it, but storing something in it will panic:

var m1 = map[string]string{} // empty map
var m0 map[string]string     // zero map (nil)

println(len(m1))   // outputs '0'
println(len(m0))   // outputs '0'
println(m1["foo"]) // outputs ''
println(m0["foo"]) // outputs ''
m1["foo"] = "bar"  // ok
m0["foo"] = "bar"  // panics!

This requires you to be careful when a structure has a map field, since it has to be initialized before adding entries to it.

So, as a developer, you have to constantly check if a structure you want to use requires to call a constructor function or if the zero value is useful. This is a lot of burden put on code writers for some simplifications in the language.

Go doesn't have exceptions. Oh wait... it does!

The blog post "Why Go gets exceptions right" explains in detail why exceptions are bad, and why the Go approach to require returning error is better. I can agree with that, and exceptions are hard to deal with when using asynchronous programming or a functional style like Java streams (let's put aside that the former isn't necessary in Go thanks to goroutines and the latter is simply not possible). The blog post mentions panic as "always fatal to your program, game over", which is fine.

Now "Defer, panic and recover" that predates it, explains how to recover from panics (by actually catching them), and says "For a real-world example of panic and recover, see the json package from the Go standard library".

And indeed, the json decoder has a common error handling function that just panics, the panic being recovered in the top-level unmarshal function that checks the panic type and returns it as an error if it's a "local panic" or re-panics the error otherwise (losing the original panic's stacktrace on the way).

To any Java developer this definitely looks like a try / catch (DecodingException ex). So Go does have exceptions, uses them internally but tells you not to.

Fun fact: a non-googler fixed the json decoder a couple of weeks ago to using regular errors bubbling up.

The Ugly

The dependency management nightmare

Update: since this post was written, Go 1.11 has introduced module support with resolution based on semantic versioning. This solves most of the issues explained below, although the choice of the minimum version for dependency resolution is debatable (see also this post on why Rust's Cargo chooses the maximum version).

Let's start by quoting Jaana Dogan (aka JBD), a well known gopher at Google, who recently vented her frustration on Twitter:

Let's put it simply: there is no dependency management in Go. All current solutions are just hacks and workarounds.

This goes back its origins at Google, which famously uses a giant monolithic repository for all their source code. No need for module versioning, no need for 3rd party modules repositories, you build everything from your current branch. Unfortunately this doesn't work in the open Internet.

Adding a dependency in Go means cloning that dependency's source code repo in your GOPATH. What version? The current master branch at the time of cloning, whatever it contains. What if different projects need different versions of a dependency? They can't. The notion of "version" doesn't even exist.

Also, your own project has to live in GOPATH or the compiler won't find it. Want to have your projects cleanly organized in a separate directory? You have to hack per-project GOPATH or fiddle with symbolic links.

The community has developed workarounds with a large number of tools. Package management tools introduced vendoring and lock files holding the Git sha1 of whatever you cloned, to provide reproducible builds.

Finally in Go 1.6 the vendor directory was officially supported. But it's about vendoring what you cloned, and still not proper version management. No answer to conflicting imports from transitive dependencies that are usually solved with semantic versioning.

Things are getting better though: dep, the official dependency management tool was recently introduced to support vendoring. It supports versions (git tags) and has a version solver that follows semantic versioning conventions. It's not stable yet, but goes in the right direction. It still requires your project to live in GOPATH though.

But dep may not live long though as vgo, also from Google, wants to bring versioning in the language itself and has been making some waves lately.

So dependency management in Go is nightmarish. It's painful to setup, and you don't think about it while developing until it blows up when you add a new import or simply want to pull a branch of one of your team members in your GOPATH...

Let's go back to the code now.

Mutability is hardcoded in the language

There is no way to define immutable structures in Go: struct fields are mutable and the const keyword doesn't apply to them. Go makes it easy however to copy an entire struct with a simple assignment, so we may think that passing arguments by value is all that is needed to have immutability at the cost of copying.

However, and unsurprisingly, this does not copy values referenced by pointers. And the since built-in collections (map, slice and array) are references and are mutable, copying a struct that contains one of these just copies the pointer to the same underlying memory.

The example below illustrates this:

type S struct {
    A string
    B []string
}

func main() {
    x := S{"x-A", []string{"x-B"}}
    y := x // copy the struct
    y.A = "y-A"
    y.B[0] = "y-B"

    fmt.Println(x, y)
    // Outputs "{x-A [y-B]} {y-A [y-B]}" -- x was modified!
}

So you have to be extremely careful about this, and not assume immutability if you pass a parameter by value.

There are some deepcopy libraries that attempt to solve this using (slow) reflection, but they fall short since private fields can't be accessed with reflection. So defensive copying to avoid race conditions will be difficult, requiring lots of boilerplate code. Go doesn't even have a Clone interface that would standardize this.

Slice gotchas

Slices come with many gotchas: as explained in "Go slices: usage and internals", re-slicing a slice doesn't copy the underlying array for performance reasons. This is a laudable goal but means that sub-slices of a slice are just views that follow the mutations of the original slice. So don't forget to copy() a slice if you want to separate it from its origin.

Forgetting to copy() becomes more dangerous with the append function: appending values to a slice resizes the underlying array if it doesn't have enough capacity to hold the new values. This means that the result of append may or may not point to the original array depending on its initial capacity. This can cause hard to find non deterministic bugs.

In the code below we see that the effects of a function appending values to a sub-slice vary depending on the capacity of the original slice:

func doStuff(value []string) {
    fmt.Printf("value=%v\n", value)

    value2 := value[:]
    value2 = append(value2, "b")
    fmt.Printf("value=%v, value2=%v\n", value, value2)

    value2[0] = "z"
    fmt.Printf("value=%v, value2=%v\n", value, value2)
}

func main() {
    slice1 := []string{"a"} // length 1, capacity 1

    doStuff(slice1)
    // Output:
    // value=[a] -- ok
    // value=[a], value2=[a b] -- ok: value unchanged, value2 updated
    // value=[a], value2=[z b] -- ok: value unchanged, value2 updated

    slice10 := make([]string, 1, 10) // length 1, capacity 10
    slice10[0] = "a"

    doStuff(slice10)
    // Output:
    // value=[a] -- ok
    // value=[a], value2=[a b] -- ok: value unchanged, value2 updated
    // value=[z], value2=[z b] -- WTF?!? value changed???
}

Mutability and channels: race conditions made easy

Go concurrency is built on CSP using channels which makes coordinating goroutines much simpler and safer than synchronizing on shared data. The mantra here is "Do not communicate by sharing memory; instead, share memory by communicating". This is wishful thinking however and cannot be achieved safely in practice.

As we saw above there is no way in Go to have immutable data structures. This means that once we send a pointer on a channel, it's game over: we share mutable data between concurrent processes. Of course a channel of structures (and not pointers) copies the values sent on the channel, but as we saw above, this doesn't deep-copy references, including slices and maps, which are intrinsically mutable. Same goes with struct fields of an interface type: they are pointers, and any mutation method defined by the interface is an open door to race conditions.

So although channels apparently make concurrent programming easy, they don't prevent race conditions on shared data. And the intrinsic mutability of slices and maps makes them even more likely to happen.

Talking about race conditions, Go includes a race condition detection mode, which instruments the code to find unsynchronized shared access. It can only detect race problems when they happen though, so mostly during integration or load tests, hoping those will exercise the race condition. It cannot realistically be enabled in production because of its high runtime cost, except for temporary debug sessions.

Noisy error management

Something you will learn quickly in Go is the error handling pattern, repeated ad nauseam:

someData, err := SomeFunction()
if err != nil {
    return err;
}

Because Go claims to not support exceptions (although it does), every function that can end up with an error must have an error as its last result. This applies in particular to every function that performs some I/O, so this verbose pattern is extremely prevalent in network applications, which is Go's primary area.

Your eye will quickly develop a visual filter for this pattern and identify it as "yeah, error handling", but still it's a lot of noise and it's sometimes hard to find the actual code in the middle of error handling.

First of all, your functions should really return the error interface type and not a meaningful concrete type or the callers will encouter the dreaded "Why is my nil error value not equal to nil?" issue that has a dedicated FAQ entry (see also "nil interface values" below).

There are also a few gotchas, since an error result can actually be a nominal case, as for example when reading from the ubiquitous io.Reader: 

len, err := reader.Read(bytes)
if err != nil {
    if err == io.EOF {
        // All good, end of file
    } else {
        return err
    }
}

In "Error has values" Rob Pike suggests some strategies to reduce error handling verbosity. I find them to be actually a dangerous band-aid:

type errWriter struct {
    w   io.Writer
    err error
}

func (ew *errWriter) write(buf []byte) {
    if ew.err != nil {
        return // Write nothing if we already errored-out
    }
    _, ew.err = ew.w.Write(buf)
}

func doIt(fd io.Writer) {
    ew := &errWriter{w: fd}
    ew.write(p0[a:b])
    ew.write(p1[c:d])
    ew.write(p2[e:f])
    // and so on
    if ew.err != nil {
        return ew.err
    }
}

Basically this recognizes that checking errors all the time is painful, and provides a pattern to just ignore errors in a write sequence until its end. So any operation performed to feed the writer once it has errored-out is executed even if we know it shouldn't. What if these are more expensive than just getting a slice? We've just wasted resources because Go's error handling is a pain. What if evaluating parameters has side effects? We've just introduced a serious bugs to simplify error handling...

Rust had a similar issue: by not having exceptions (really not, contrarily to Go), functions that can fail return Result<T, Error> and require some pattern matching on the result. So Rust 1.0 came with the try! macro and, recognizing the pervasiveness of this pattern, made it a first-class language feature. So you have the terseness of the above code while keeping a correct error-handling.

Transposing Rust's approach to Go is unfortunately not possible because Go doesn't have generics nor macros.

Nil interface values

This is an update after redditor jmickeyd shows a weird behaviour of nil and interfaces, that definitely qualifies as ugly. I expanded it a bit:

type Explodes interface {
    Bang()
    Boom()
}

// Type Bomb implements Explodes
type Bomb struct {}
func (*Bomb) Bang() {}
func (Bomb) Boom() {}

func main() {
    var bomb *Bomb = nil
    var explodes Explodes = bomb
    println(bomb, explodes) // '0x0 (0x10a7060,0x0)'
    if explodes != nil {
        println("Not nil!") // 'Not nil!' What are we doing here?!?!
        explodes.Bang()     // works fine
        explodes.Boom()     // panic: value method main.Bomb.Boom called using nil *Bomb pointer
    } else {
        println("nil!")     // why don't we end up here?
    }
}

The above code verifies that explodes is not nil and yet the code panics in Boom but not in Bang. Why is that? The explanation is in the println line: the bomb pointer is 0x0 which is effectively nil, but explodes is the non-nil (0x10a7060,0x0).

This is because interface values are fat pointers. The first element of this pair is the pointer to the method dispatch table for the implementation of the Bomb interface by the Explodes type, and the second element is the address of the actual Explodes object, which is nil.

The call to Bang succeeds because it applies to pointers to a Bomb: there is no need to dereference the pointer to call the method. The Boom method acts on a value and so a call causes pointers to be dereferenced, which causes a panic.

Note that if we had written var explodes Explodes = nil, then != nil would have not succeeded.

So how should we write the test in a safe way? We have to nil-check both the interface value and if non-nil, check the value pointed to by the interface object... using reflection!

if explodes != nil && !reflect.ValueOf(explodes).IsNil() {
    println("Not nil!") // we no more end up here
    explodes.Bang()
    explodes.Boom()
} else {
    println("nil!")     // 'nil' -- all good!
}

Bug or feature? The Tour of Go has a dedicated page to explain this behaviour and clearly says "Note that an interface value that holds a nil concrete value is itself non-nil".

Still, this is ugly and can cause very subtle bugs. It looks to me a like a big flaw in the language design to make its implementation easier.

Added in Dec '19: Other subtle bugs (or hard crashes, actually) can occur when a field of an interface type is updated concurrenly. Since assignment of a fat pointer isn't atomic, we may end up with a fat pointer pointing to a type and to a value of a different type. This breaks Go's memory safety and was demonstrated in an exploit during a security challenge. This problem can probably be found with Go's race detector if your tests exercise it.

Struct field tags: runtime DSL in a string

If you've used JSON in Go, you've certainly encountered something similar:

type User struct {
    Id string    `json:"id"`
    Email string `json:"email"`
    Name string  `json:"name,omitempty"`
}

These are struct tags which the language spec says are a string "made visible through a reflection interface and take part in type identity for structs but are otherwise ignored". So basically, put whatever you want in this string, and parse it at runtime using reflection. And panic at runtime if the syntax isn't right.

This string is actually field metadata, something that has existed for decades in many languages as "annotations" or "attributes". With language support, their syntax is formally defined and checked at compile time, while still being extensible.

Why did Go decide to use a raw string that any library can decide to use with whatever DSL it wants, parsed at run time?

Things can get awkward when you use multiple libraries: here's an example taken out of Protocol Buffer's Go documentation: 

type Test struct {
    Label         *string             `protobuf:"bytes,1,req,name=label" json:"label,omitempty"`
    Type          *int32              `protobuf:"varint,2,opt,name=type,def=77" json:"type,omitempty"`
    Reps          []int64             `protobuf:"varint,3,rep,name=reps" json:"reps,omitempty"`
    Optionalgroup *Test_OptionalGroup `protobuf:"group,4,opt,name=OptionalGroup" json:"optionalgroup,omitempty"`
}

Side note: why are these tags so common when using JSON? Because in Go public fields must use UpperCamelCase, or at least start with an uppercase letter, whereas the common convention for naming fields in JSON is either lowerCamelCase or snake_case. Hence the need for tedious tagging.

The standard JSON encoder/decoder doesn't allow providing a naming strategy to automate the conversion, like Jackson does in Java. This probably explains why all fields in the Docker APIs are UpperCamelCase: that avoided the need for its developers to write these unwieldy tags for their large API.

Note however that the JSON parser is case insensitive, probably to make it easier to read camelCase JSON that is so common without requiring annotations. But this also means that writing the same data structure back will use the default UpperCamelCase naming and therefore produce a different JSON...

No generics... at least not for you

Update (2020-06): the Go team has come up with a rather nice proposal to add type parameters to Go, which will make this paragraph and the next one obsolete once it finally lands in the language. My only gripe is the builtin comparable constraint that could be defined as regular interface that any type (and not only builtin ones) can implement, and similarly the lack of widely used concepts like Ordered or Hashable. But that may come in due time.

It's hard to conceive a modern statically typed language without generics, but this is what you get with Go: it has no generics... or more precisely almost no generics, which as we'll see makes it worse than no generics at all.

The built-in slice, map, array and channel are generic. Declaring a map[string]MyStruct clearly shows the use of a generic type that has two parameters. Which is nice, as it allows type safe programming that catches all sorts of errors.

There are however no user-definable generic data structures. This means that you cannot define reusable abstractions that may work with any type, in a type-safe way. You have to use untyped interface{} and cast values to the proper type. Any mistake will only be caught at run time and will result in a panic. For a Java developer, it's like going back in the pre-Java 5 times, in 2004.

In "Less is exponentially more", Rob Pike surprisingly puts generics and inheritance in the same "typed programming" bag and says he favors composition over inheritance. Not liking inheritance is fine (I actually write a lot of Scala with little inheritance) but generics answer another concern: reusability while preserving type safety.

As we'll see below, the segregation between built-ins with generics and user-defined without generics has consequences on more than developer "comfort" and compile-time type safety: it impacts the whole Go ecosystem.

Go has few data structures beyond slice and map

The Go ecosystem doesn't have many data structures that provide added or different functionality from the built-in slice and map. Recent versions of Go added the containers package that provides a few of them. They all have the same caveat: they deal with interface{} values, meaning you lose all type safety.

Let's see an example with sync.Map which is a concurrent map with lower thread contention than guarding a regular map with a mutex:

type MetricValue struct {
    Value float64
    Time time.Time
}

func main() {
    metric := MetricValue{
        Value: 1.0,
        Time: time.Now(),
    }

    // Store a value

    m0 := map[string]MetricValue{}
    m0["foo"] = metric

    m1 := sync.Map{}
    m1.Store("foo", metric) // not type-checked

    // Load a value and print its square

    foo0 := m0["foo"].Value // rely on zero-value hack if not present
    fmt.Printf("Foo square = %f\n", math.Pow(foo0, 2))

    foo1 := 0.0
    if x, ok := m1.Load("foo"); ok { // have to make sure it's present (not bad, actually)
        foo1 = x.(MetricValue).Value // cast interface{} value
    }
    fmt.Printf("Foo square = %f\n", math.Pow(foo1, 2))

    // Sum all elements

    sum0 := 0.0
    for _, v := range m0 { // built-in range iteration on map
        sum0 += v.Value
    }
    fmt.Printf("Sum = %f\n", sum0)

    sum1 := 0.0
    m1.Range(func(key, value interface{}) bool { // no 'range' for you! Provide a function
        sum1 += value.(MetricValue).Value        // with untyped interface{} parameters
        return true // continue iteration
    })
    fmt.Printf("Sum = %f\n", sum1)
}

This is a great illustration of why there aren't many data structures in the Go ecosystem: they are a pain to use compared to the built-in slice and map. And for a simple reason: there are two categories of data structures in Go:

  • aristocracy, the built-in slice, map, array and channel: type safe and generic, convenient to use with range,
  • rest of the world written in Go code: can't provide type safety, clumsy to use because of required casts.

So library-defined data structures really have to provide solid benefits for us developers to be willing to pay the price of loosing type safety and the additional code verbosity.

The duality between built-in structures and Go code is painful in more subtle ways when we want to write reusable algorithms. This is an example from the standard library's sort package to sort a slice:

import "sort"

type Person struct {
    Name string
    Age  int
}

// ByAge implements sort.Interface for []Person based on the Age field.
type ByAge []Person

func (a ByAge) Len() int           { return len(a) }
func (a ByAge) Swap(i, j int)      { a[i], a[j] = a[j], a[i] }
func (a ByAge) Less(i, j int) bool { return a[i].Age < a[j].Age }

func SortPeople(people []Person) {
    sort.Sort(ByAge(people))
}

Wait... Seriously? We have to define a new type ByAge that has to implement 3 methods to bridge a generic (in the sense of "reusable") sort algorithm and the typed slice.

The only thing that should matter to us, developers, is the Less function that compares two objects and is domain-dependent. Everything else is noise and boilerplate required by the simple fact that Go has no generics. And we have to repeat it for each and every type that we want to sort. And every comparator too.

Update: Michael Stapelberg points me to sort.Slice that I missed. Looks better, although it uses reflection under the hood (eek!) and requires the comparator function to be a closure on the slice to sort, which is still ugly.

Every text explaining that Go doesn't need generics shows this as "the Go way" that allows having reusable algorithms while avoiding downcasting to interface{}...

Ok. Now to ease the pain, it would be nice if Go had macros that could generate this nonsensical boilerplate, right? Well, read on...

go generate: ok-ish, but...

Go 1.4 introduced the go generate command to trigger code generation from annotations in the source code. Well, "annotation" here actually means a magic //go:generate comment with strict rules: "the comment must start at the beginning of the line and have no spaces between the // and the go:generate". Get it wrong, add a space and no tool will warn you about it.

This covers several kinds of use cases:

  • Generating Go code from other sources: ProtoBuf / Thrift / Swagger schemas, language grammars, etc.

  • Generating Go code that complements existing code, such as stringer given as an example, that generates a String() method for a series of typed constants.

  • Add poor man's generics support. Since Go doesn't have generics, some clever band-aid solutions have emerged to generate them from template code.

First use case is ok, and the added value is that you don't have to fiddle with Makefiles and that generation instructions can be close to the generated code's usage.

For the second use case, many languages, such as Scala & Rust, have macros (which are mentioned in the design document) that have access to the source code's AST during compilation. Stringer actually imports the Go compiler's parser to traverse the AST. Java doesn't have macros but annotation processors play the same role.

Many languages also don't support macros so nothing fundamentally wrong here, except this fragile comment-driven syntax, which looks again like a quick hack that somehow does the job, and not something that was carefully thought out as coherent language design.

Oh, and did you know that the Go compiler actually has a number of annotations/pragmas and conditional compilation using this fragile comment syntax?

Conclusion

As you probably guessed, I have a love/hate relation with Go. Go is a bit like this friend that you like to hang out with because he's fun and great for small talk around beers, but that you find boring or painful when you want to have deeper conversations, and that you don't want to go on vacation with.

I like Go to quickly develop simple APIs (although JSON support is pretty basic compared to Java's Jackson or Rust's serde) or network stuff that goroutines make easy to reason about, I hate its limited expressiveness and half-baked type system when I have to implement business logic, and I hate all its quirks and gotchas waiting to hit you hard. Go requires a lot of attention and discipline to avoid them.

Up to recently there wasn't really an alternative in the space that Go occupies, which is developing efficient native executables without incurring the pain of C or C++. Rust is progressing quickly, and the more I play with it, the more I find it extremely interesting and superbly designed. I have the feeling that Rust is one of those friends that take some time to get along with, but that you'll finally want to engage with for a long term relationship.

Going back on more technical aspects, you'll find articles saying that Rust and Go don't play in the same park, that Rust is a systems language because it doesn't have a GC, etc. I think this is becoming less and less true. Rust is climbing higher in the stack with great web frameworks and nice ORMs. It also gives you that warm feeling of "if it compiles, errors will come from the logic I wrote, not language quirks I forgot to pay attention to".

We also see some interesting movements in the container/service mesh area with Buoyant (developers of Linkerd) developing their new Kubernetes service mesh Conduit as a combination of Go for the control plane (I guess because of the available Kubernetes libraries) and Rust for the data plane for its efficiency and robustness, and also Sozu proxy.

Swift is also part of this family or recent alternatives to C and C++. Its ecosystem is still too Apple-centric though, even if it's now available on Linux and has emerging server-side APIs and the Netty framework.

There is of course no silver bullet and no one-size-fits-all. But knowing the gotchas of the tools you use is important. I hope this blog post has taught you some things about Go that you weren't aware of, so that you avoid the traps rather than getting caught!

A few days later: #3 on Hacker News!

Update, 3 days after publishing: The reaction to this article has been amazing. It has made the front page of Hacker News (best rank I saw was #3) and /r/programming (best rank I saw was #5), and got some traction on Twitter.

The comments are generally positive (even on /r/golang/) or at least recognize that the article is balanced and tries to be fair. People on /r/rust of course liked my interest in Rust. Someone I never heard of even emailed me saying "I just wanted to let you know that I thought your write-up was the best ever. Thanks for all the work you put into it".

That was the hard part when writing it: try to be as factual and impartial as possible. This is of course not entirely possible as everyone has their own preferences, and why I focused on unexpected surprises and language ergonomics: how much the language helps you rather than getting in the way, or at least my way.

I also searched for code samples in the standard library or on golang.org and quoted people from the Go team, to base my analysis on authoritative material and avoid "meh, you quoted someone who got it wrong" reactions.

Writing this article used most of my evenings for two weeks, but it was lots of fun. And this is what you get when you do serious and honest work: lots of good vibes from teh Internets (if you ignore the few trolls and always grumpy people). Very motivating to write more in depth content!

← Local date time calculations in Go

Getting meaningful stack traces from Rust tests returning a Result →