I had a small problem that didn't deserve a big solution. A directory on a box fills up with files dropped by another process, and something needs to notice each new file, do a little validation, and POST it to an internal endpoint. The kind of job a shell script grows into, then resents.
I wrote it in Go instead, and it shipped this week. The thing I keep coming back to is not the language so much as the deployment: go build, copy one binary to the box, drop a systemd unit next to it, done. No runtime to install, no virtualenv to recreate, no "but it works on my machine" because the machine is irrelevant once it's a static binary.
the shape of it
The core is a fsnotify watcher and a worker that drains a channel. Nothing clever.
watcher, err := fsnotify.NewWatcher()
if err != nil {
log.Fatalf("creating watcher: %v", err)
}
defer watcher.Close()
if err := watcher.Add(*dir); err != nil {
log.Fatalf("watching %s: %v", *dir, err)
}
for {
select {
case ev := <-watcher.Events:
if ev.Op&fsnotify.Create == fsnotify.Create {
jobs <- ev.Name
}
case err := <-watcher.Errors:
log.Printf("watch error: %v", err)
}
}
The worker side reads a path off jobs, opens the file, and does its bit. The one thing I got wrong first time was acting on the Create event immediately. The file exists at that point, but the process writing it may not have finished. You get a half-written file and a confused POST. The fix was unglamorous: a short settle delay and a check that the file size has stopped changing before touching it. Inotify tells you a file appeared, not that it's done.
the small print that matters
A daemon that runs unattended needs to behave when things go wrong, and most of the effort went there rather than the happy path.
- Retries with a backoff when the endpoint is down, capped so a long outage doesn't pin the CPU.
- A dead-letter directory for files that fail validation, so nothing silently vanishes.
- Logging to stderr and letting systemd and the journal own it, rather than inventing my own log files that then fill up
/var. I have learned that lesson recently and at some cost.
It compiles to about six megabytes, uses a few MB of RAM at rest, and has done its job without complaint since Wednesday. There is a particular satisfaction in a program that is allowed to be small. Not every problem wants a framework. Sometimes it wants one binary, one config flag, and a unit file that restarts it if it ever falls over.
If I were to do it again I'd reach for Go just as quickly. The standard library covers HTTP, JSON and signals out of the box, the concurrency model fits a watch-and-dispatch job exactly, and the operational story is the best part. Write it, build it, copy it, forget about it. That's the whole pitch, and for a tool this size it's enough.