Ramblings of an aging IT geek
← Ramblings of an aging IT geek
golang

building go binaries for a box i can't compile on

How I cross-compile Go for a low-memory ARM router from my laptop, and the handful of CGO and toolchain gotchas that catch me out every time.

A terminal showing Go source code

The box in question is a little ARM router with 256MB of RAM and the build patience of a teenager. Compiling Go on it directly is technically possible and practically miserable: it swaps itself into a coma somewhere around the third dependency. So I don't. I build on the laptop and copy the binary over. Go makes this almost embarrassingly easy, right up until it doesn't.

The happy path is one line:

GOOS=linux GOARCH=arm GOARM=7 go build -o myapp ./cmd/myapp

That's it. No cross-compiler to install, no sysroot to assemble, no afternoon lost to autotools. The Go toolchain ships every target it supports, and go tool dist list will show you the lot. For this router it's arm with GOARM=7, because it's an ARMv7 part. Get GOARM wrong and you either leave performance on the table or, worse, produce a binary that dies with an illegal instruction the moment it touches hardware floating point.

Close-up of code on a screen

the moment it stops being easy

Pure Go cross-compiles for free. The trouble starts when something pulls in cgo. A SQLite driver, a libpcap binding, anything that wraps a C library, and now CGO_ENABLED=1 means you need an actual ARM C cross-compiler, not just the Go toolchain.

You can do it. You install gcc-arm-linux-gnueabihf, point Go at it, and hope the target's glibc is new enough:

CGO_ENABLED=1 \
  CC=arm-linux-gnueabihf-gcc \
  GOOS=linux GOARCH=arm GOARM=7 \
  go build -o myapp ./cmd/myapp

But honestly, my first move is to delete the cgo dependency instead. For SQLite there's a pure-Go translation now; for most other things there's a Go-native library that's good enough. Every cgo dependency you remove is a cross-compile that goes back to being one boring line, and a binary that's statically linked and just runs.

the bits that actually bite

Two things catch me out every single time, so I'll write them down for future me.

First, file is your friend. After a build I run file myapp and check it actually says ARM, not x86-64. It's depressingly easy to forget the env vars, build a perfectly good amd64 binary, scp it over, and stare at "cannot execute binary file" wondering what's broken.

Second, glibc versioning. If you do go down the cgo route, the binary is linked against whatever glibc your build host has, and if the router is running something older it'll refuse to start with a GLIBC_2.34 not found style error. Static linking or musl sidesteps it. The pure-Go static binary sidesteps it harder, which is yet another reason I keep cgo out where I can.

The whole thing takes about four seconds and produces a single file I can drop onto the box with no runtime, no interpreter and no dependency hell. That's the part I never stop appreciating, even years in.