Ramblings of an aging IT geek
← Ramblings of an aging IT geek
rust

the borrow checker was right and i was tired

A long evening spent arguing with Rust's borrow checker over a self-referential cache, and the cheaper design it eventually nudged me towards.

Code on a screen

I lost an evening to the borrow checker this week, and the annoying part is that it was right the whole time. I knew it was right somewhere around the second hour. I kept going anyway, because admitting the design was wrong felt more expensive than fighting the compiler. It was not.

The shape of it: I had a struct holding a Vec<String> of parsed log lines, and I wanted a second field holding &str slices into those strings, a sort of index into the owned data. A self-referential struct. If you have written any Rust at all you already know the sound the compiler makes at this point.

error[E0515]: cannot return value referencing local variable `lines`
  --> src/parser.rs:42:5
   |
42 |     Index { lines, refs }
   |     ^^^^^^^^^^^^^^^^^^^^^^ returns a value referencing data owned by the current function

You cannot hold an owned buffer and borrows into that same buffer in one struct, because the moment the struct moves, those borrows point at the old location. This is not the borrow checker being fussy. It is the borrow checker stopping you from writing a use-after-move that C would happily hand you and your users would discover in production.

A diagram on a screen

I tried the things you try. I reached for Rc<str> so the strings would be reference-counted and I could clone cheap handles instead of borrowing. That compiled, but it meant an allocation and an atomic-ish bump per line, which for a parser doing millions of lines is exactly the cost I was trying to avoid. I looked at Pin, briefly, the way you look at a manual for a tool you are fairly sure will hurt you. I considered ouroboros, the crate that exists precisely to make self-referential structs safe, and that is a perfectly good answer, but pulling in a macro-heavy dependency to paper over a design I had not thought through felt like the wrong reflex.

The honest fix was to stop storing borrows at all. Instead of &str slices, I stored (usize, usize) ranges: a start offset and a length into the buffer. No lifetimes, no borrows, just integers. When I need the actual string I index into lines at lookup time. It is a touch more verbose at the call site, but the struct is now trivially movable, trivially Send, and the whole lifetime argument evaporated.

That is the pattern I keep relearning. When the borrow checker digs in this hard, it is usually telling me my ownership model is muddled, not that Rust is being awkward. The languages I came up on would have let me ship the muddle. Rust makes me resolve it at compile time, in a quiet room, instead of at 3am with a pager going off.

So I lost. Gracefully, in the end, after losing ungracefully for a couple of hours first. The code is faster than the Rc version, simpler than the ouroboros version, and I no longer have any lifetimes in that module at all. Index-into-the-buffer is not a clever trick, it is just the thing the compiler was pointing at the whole time while I looked the other way.