Ramblings of an aging IT geek
← Ramblings of an aging IT geek
golang

building go binaries for an arm box from my laptop

Cross-compiling a small Go service from an amd64 laptop to a 32-bit ARM single-board computer, the GOARM gotcha, the cgo wall, and how to keep the whole thing reproducible.

Go source code on a developer's screen

I have a small Go service that needs to run on an ARM single-board computer on the home network. The board is perfectly capable of running it, it is just a miserable place to compile anything: not much RAM, storage on an SD card, and a build that takes minutes on my laptop takes an age on it. The whole point of Go's toolchain is that I should not have to. So I cross-compile on the laptop and copy a finished binary across.

The headline is genuinely this simple. Go ships every target's standard library, so there is no toolchain to install, no sysroot to assemble, none of the misery that cross-compiling C usually involves:

GOOS=linux GOARCH=arm GOARM=7 go build -o thing-arm ./cmd/thing
scp thing-arm board:/usr/local/bin/thing

That is the entire trick on a good day. But there are a couple of places it goes wrong, and they go wrong quietly, so they are worth spelling out.

GOARM is not optional

GOARCH=arm is 32-bit ARM, and within that there are sub-variants. GOARM picks the floating-point and instruction expectations: 5 for ancient soft-float boards, 6 for the very oldest Raspberry Pi, 7 for basically anything modern with hardware floating point. Get it too high for the chip and the binary builds happily on the laptop and then dies on the board with an illegal instruction, which is a deeply unhelpful error if you do not know to look for it.

If your board is 64-bit, ignore all of this and use GOARCH=arm64, which has no GOARM knob at all. Check what you actually have before guessing:

ssh board 'uname -m'   # armv7l -> GOARCH=arm GOARM=7
                       # aarch64 -> GOARCH=arm64

I have wasted an evening before by assuming a board was 64-bit because it was new, when the OS image on it was a 32-bit one.

A laptop building code destined for a small ARM board

cgo is where the simplicity ends

Everything above holds because pure Go cross-compiles with zero external toolchain. The moment you import something that pulls in cgo, that promise evaporates. cgo invokes a C compiler, and a C compiler that targets ARM from an amd64 host is exactly the cross-toolchain headache Go usually saves you from.

So the first thing I do is make sure cgo is actually off, rather than assuming:

CGO_ENABLED=0 GOOS=linux GOARCH=arm GOARM=7 go build -o thing-arm ./cmd/thing

With CGO_ENABLED=0 the build will fail loudly if anything genuinely needs cgo, which is what I want. The usual offenders are the standard net and os/user packages, which have cgo-backed resolvers, and any SQLite driver. For net, the pure-Go resolver is fine for my uses and gets selected automatically when cgo is off. For SQLite there is no free lunch: either install an ARM cross-compiler and accept the complexity, or switch to a pure-Go database driver. I switched.

Keeping it honest

Two small habits make this reliable rather than lucky.

First, strip the build metadata so the binary is reproducible and a little smaller:

CGO_ENABLED=0 GOOS=linux GOARCH=arm GOARM=7 \
  go build -trimpath -ldflags="-s -w" -o thing-arm ./cmd/thing

-trimpath removes my laptop's filesystem paths from the binary, -s -w drop the symbol and DWARF tables. None of it is required, all of it is tidy.

Second, I put the exact invocation in a Makefile target rather than trusting myself to remember GOARM=7 at 11pm. The bug where you ship a GOARM=6 binary and it runs slightly slower forever, or a GOARM=8 one that crashes, is precisely the bug you introduce by typing the command from memory.

After all that ceremony, the actual workflow is lovely: edit on the laptop, one make arm, one scp, restart the service, done. The board never sees a compiler. That is the whole appeal, and once the GOARM and cgo traps are out of the way it just works.