Why is compiling with CGO_ENABLED=0 slower?

Question

When writing programs which utilize network, you can see quite noticeable slowdown of compilation with CGO_ENABLED=0.

For example, the simplest HTTP server:

package main

import (
    "flag"
    "fmt"
    "log"
    "net/http"
)

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hi! glad you requested %s.\n", r.URL.Path[1:])
}

func main() {
    port := flag.Int("port", 9000, "")
    flag.Parse()

    http.HandleFunc("/", handler)
    err := http.ListenAndServe(fmt.Sprintf(":%d", *port), nil)
    if err != nil {
        log.Fatal(err)
    }
}

the timings are:

% time go build
go build  0.46s user 0.06s system 131% cpu 0.396 total
% time CGO_ENABLED=0 go build
CGO_ENABLED=0 go build  3.93s user 0.15s system 143% cpu 2.849 total

So far I'm not using bindings to C so CGo handling or not seems irrelevant, what I'd like to do is to compile 100% static binary, but not if there's such slowdown.

What is the cause of such behavior?

Solution 1

The problem is that the standard library packages are built without flags. CGO_ENABLED changes build flags and therefore it can't use the pre-built packages so most of the standard library needs to be rebuilt.

As the other answer mentioned, go build -i will install the packages built with the new flags, but that won't really solve much because if you install packages built with CGO_ENABLED=0, it will speed up all future builds with CGO_ENABLED=0, but it will slow down all the builds without it.

Unfortunately the way pre-built packages are installed by default today is pretty inefficient because everything goes into the same directory under the same names regardless of how it's built. If you want to be able to have fast builds of go programs with different flags, besides doing go build -i you also need to use the the -installsuffix and/or the -pkgdir flags. In the system I work in, we have a handful of different compilation modes, each mode has different flags (because of a lot of old C code we interface with) and each mode also has its own -pkgdir.

Solution 2

This is the time spent re-building dependencies. Go build by default does not save rebuilt dependencies. See the -i flag: The -i flag installs the packages that are dependencies of the target.

Let's try your program with -i instead:

$ time go build -i . 
real    0m0.337s
user    0m0.343s
sys 0m0.121s

$ time CGO_ENABLED=0 go build -i .    
real    0m2.135s
user    0m3.098s
sys 0m0.196s

$ time CGO_ENABLED=0 go build .
real    0m0.329s
user    0m0.367s
sys 0m0.085s

$ time go build .    
real    0m2.588s
user    0m3.393s
sys 0m0.300s

The first time you switch cgo mode, it needs to rebuild dependencies. If you specify -i, it will save them and the second build call will be much faster.

Comments

[Wojciech Kaczmarek]

Hmmm, is this normal? after time CGO_ENABLED=0 go build -i I got go install net: open /usr/local/go/pkg/linux_amd64/net.a: permission denied

[Marc]

go install wants to put stdlib packages back in their original place: GOROOT. I believe work is being done to change that, maybe for 1.10. Until then, you can avoid that by not using the system go location and having your own (eg: I work with GOROOT=${HOME}/go).

[Wojciech Kaczmarek]

Yes there's certainly work on this in 1.10 or above. rsc was recently blogging about that. -i equivalent is going to be used by default, also in the future pkg dir will be gone.

[Wojciech Kaczmarek]

Back to your suggestion, I have already GOPATH set to $HOME/go. So the practical question is now, can GOROOT and GOPATH point to the same directory?

[Marc]

That I'm not sure. I would keep them separate though. Maybe have GOPATH=${HOME}/go and GOROOT=${HOME}/golang. Updating go version becomes just extracting the tarball and switching your golang symlink to the new version.

[Volker]

@WojciechKaczmarek: "can GOROOT and GOPATH point to the same directory?" No! Never do this!

[Wojciech Kaczmarek]

Btw looks like cache in 1.10 is going to help a lot... (also with a future plan to get rid of pkg/ completely)