Ramblings of an aging IT geek
← Ramblings of an aging IT geek
news

the breach was in the tool that watches the build

Thoughts on the Codecov bash-uploader compromise and why a poisoned CI helper script is a worse blast radius than most people first assume.

A wall of news headlines about a security disclosure

The story doing the rounds this fortnight is the Codecov compromise: their Bash uploader script was quietly altered, for months, so that it would exfiltrate the environment of any CI job that ran it. If you have ever pasted bash <(curl -s https://codecov.io/bash) into a pipeline, and a great many of us have, that is the line that got you.

What makes this one land harder than the average disclosure is where it sat. It was not in a library you import and pin. It was in the bit of glue right at the end of your build, the line you copied from a README in 2018 and never looked at again. And the thing a CI job has lying around in its environment is precisely the set of secrets you would least like to lose: deploy keys, registry tokens, cloud credentials, the odd database password someone swore they would move to a secret store next sprint.

A city skyline standing in for the wider blast radius

I went and looked, because of course I did. We were not running the uploader, as it happens, more by luck than judgement. But the exercise was useful anyway. I grepped our pipelines for every curl ... | bash and bash <(curl ...) I could find, and the list was longer than I would have liked to admit. Each one is a standing promise that some third party's server will only ever serve the script you think it serves. That promise held for years and then, for Codecov, it didn't.

The honest lesson here is not "never pipe curl to bash", satisfying as that is to say. People will keep doing it because it is genuinely convenient and the alternative is more friction at exactly the moment nobody has patience for friction. The lesson is about what the script can reach once it runs. A CI job with the keys to production is a juicy target sitting behind a single unaudited download. Scope the credentials down. Use short-lived tokens. Pin the thing if you can pin it, checksum it if you can't, and assume that anything your build can see, an attacker who owns one line of your build can also see.

The rotation afterwards is the unglamorous part. Anyone affected has to treat every secret that ever passed through an exposed job as burned, which means rotating credentials you had honestly forgotten existed. That is the real cost of a supply-chain hit: not the patch, which is usually trivial, but the long tail of "what did this actually have access to, and for how long". For Codecov users the answer is "months", and that is a deeply uncomfortable number to sit with.

I have no clever closing trick. Go and grep your pipelines. You will find at least one line you forgot was there.