I thought networked multiplayer in Unreal would take a weekend. Two players, a shared box to push around, that's all. It is now the following weekend and I have a great deal more respect for everyone who ships this for a living.
The trap is that it works immediately. You tick a couple of replication boxes, you SetReplicates(true), you run two instances in PIE, and there they are, two characters who can see each other. It feels done. It is absolutely not done, because what you've built only holds together when nobody disagrees, and the entire point of networking is that everybody disagrees, constantly, about where things are.
My mistake was treating the client's idea of position as true. I moved the character locally, replicated the result, and on a LAN with no latency it looked perfect. The moment I added simulated lag, 150ms, nothing extravagant, it fell apart. The pushed box would jump backwards as the server's authoritative position overrode the client's prediction. Two players pushing the same box produced a sort of nervous twitch as each client and the server fought over who was right.
The thing I hadn't internalised is that the server has to be the authority, and the client has to predict, and then the two have to be reconciled gracefully when they differ. That's the whole game. Movement prediction, server reconciliation, interpolating remote actors so they don't teleport: this is the actual work, and it's the work that's invisible in every tutorial because the tutorial runs at zero latency on one machine.
Unreal does give you a lot for free here, to be fair. The CharacterMovementComponent already does client prediction and server correction for the player's own pawn, which is genuinely brilliant engineering and the reason my character moved smoothly while everything I touched myself stuttered. The bit it can't do for me is the gameplay-specific state, the box, the interactions, the things that aren't standard movement. That's mine to get right, and getting it right means thinking about authority for every single piece of state that more than one machine cares about.
The humbling part wasn't that it was hard. It's that I'd been quietly smug about distributed systems from the backend side, consensus, clocks, the lot, and here was the same problem wearing a different hat, and I walked straight into it. Authoritative state, eventual agreement, reconciling conflicting views of reality. I've debugged this exact class of bug in databases. I just didn't recognise it with a character mesh standing on top of it.
So the prototype is shelved for now, and that's fine. I got what I came for, which was a proper appreciation of how much careful thought sits underneath "two players push a box". Next time I'll start from the assumption that the server is right and everyone else is guessing, which, now I write it down, is how I should have started.