Ramblings of an aging IT geek
← Ramblings of an aging IT geek
gamedev

Custom NetSerialize: Packing Your Fields by Hand

Implementing a custom NetSerialize on a USTRUCT via TStructOpsTypeTraits, bit-packing fields, quantising floats, and serialising fields conditionally to save bandwidth.

A bit-packed binary layout diagram next to C++ serialisation code

Where we are in the series: part 3 of six. Part 2 left us with a struct that replicates atomically and member by member. Now we take control of the bytes.

When the default serialiser is too generous, you implement NetSerialize on the struct yourself. This is the single biggest bandwidth lever for value types, and it is less scary than it looks. You get an FArchive that works in both directions, and your one function handles both writing on the server and reading on the client. Same code, two modes, distinguished by Ar.IsSaving().

opting in

A struct's special operations are advertised through TStructOpsTypeTraits. To tell the engine "this struct has a custom net serialiser", you set WithNetSerializer:

USTRUCT()
struct FMovementSnapshot
{
    GENERATED_BODY()

    UPROPERTY()
    FVector_NetQuantize Location = FVector::ZeroVector;

    UPROPERTY()
    float Yaw = 0.f;

    UPROPERTY()
    uint8 Stance = 0;        // 0..3, two bits is plenty

    UPROPERTY()
    bool bIsSprinting = false;

    bool NetSerialize(FArchive& Ar, class UPackageMap* Map, bool& bOutSuccess);
};

template<>
struct TStructOpsTypeTraits<FMovementSnapshot>
    : public TStructOpsTypeTraitsBase2<FMovementSnapshot>
{
    enum
    {
        WithNetSerializer = true,
    };
};

The signature is fixed and worth memorising: bool NetSerialize(FArchive& Ar, UPackageMap* Map, bool& bOutSuccess). The boolean return is "did serialisation complete", and bOutSuccess is the more important "did it complete fully and correctly" flag. They are not the same thing. Return true to say the archive operation ran; set bOutSuccess = false when something could not be fully serialised this pass, most commonly an object reference whose network GUID is not yet known to the client. The engine uses bOutSuccess to decide whether the property needs retrying. Get this wrong and you either drop updates or spin retrying forever.

packing the bits

The whole point of doing this by hand is to stop spending a byte (or four) on something that needs a nibble. FArchive exposes bit-level operations for exactly this:

bool FMovementSnapshot::NetSerialize(FArchive& Ar, UPackageMap* Map, bool& bOutSuccess)
{
    // Location quantises itself.
    Location.NetSerialize(Ar, Map, bOutSuccess);

    // Yaw: full 360 degrees, one byte of precision is fine for a stance pose.
    // Map [0,360) onto a uint8 on save, expand on load.
    if (Ar.IsSaving())
    {
        uint8 PackedYaw = (uint8)FMath::RoundToInt(
            (FMath::Fmod(Yaw, 360.f) / 360.f) * 255.f);
        Ar << PackedYaw;
    }
    else
    {
        uint8 PackedYaw = 0;
        Ar << PackedYaw;
        Yaw = ((float)PackedYaw / 255.f) * 360.f;
    }

    // Stance is 0..3, so two bits. SerializeInt takes a value-max (exclusive).
    uint32 StanceWord = Stance;
    Ar.SerializeInt(StanceWord, 4);   // serialises just enough bits for [0,4)
    Stance = (uint8)StanceWord;

    // A single flag is a single bit.
    Ar.SerializeBits(&bIsSprinting, 1);

    bOutSuccess = true;
    return true;
}

Three techniques in there worth naming.

SerializeInt(Value, ValueMax) writes only as many bits as are needed to represent values in [0, ValueMax). Pass 4 and you spend two bits. This is the right tool for small bounded integers and enums; do not hand-roll bit counting when the engine will do it.

SerializeBits(Ptr, NumBits) reads or writes a precise number of bits from a buffer. For a single bool it is one bit, where the naive Ar << bIsSprinting would burn a full byte. Across many actors at tick rate, those bytes add up to real bandwidth.

Float quantisation is just a mapping you do yourself: pick the range and the resolution the gameplay actually needs, fold the float into an integer of that size on save, and expand it on load. A character's yaw to one degree is invisible to players and costs a byte instead of four. The FVector_NetQuantize types from part 2 are the same idea, prepackaged.

conditional fields

The other big saving is not serialising fields that the receiver does not need this pass. Put a header bit in front of an optional field and only write the field when the bit is set:

    // Only send the target actor when we actually have one.
    bool bHasTarget = (Ar.IsSaving() && Target != nullptr);
    Ar.SerializeBits(&bHasTarget, 1);

    if (bHasTarget)
    {
        // Object references go through the package map, which may fail to
        // resolve on the client if the target isn't yet relevant.
        bool bMappedOk = Map->SerializeObject(Ar, AActor::StaticClass(),
                                               (UObject*&)Target);
        if (!bMappedOk)
        {
            bOutSuccess = false;   // ask the engine to retry next time
        }
    }
    else if (!Ar.IsSaving())
    {
        Target = nullptr;
    }

This is where bOutSuccess earns its keep. UPackageMap::SerializeObject maps a UObject* to a network GUID and back. If the referenced object is not yet known to this client (it has not replicated yet, or is not relevant), the mapping fails, the pointer comes through null, and you must report bOutSuccess = false so the engine knows the state is incomplete and tries again later. Reporting success here is the classic bug behind "the reference is null sometimes and I cannot work out why".

what you give up

Doing this by hand means you own correctness. The default member-wise compare is gone, so the engine no longer knows how to tell whether your struct changed unless you also provide an identical operation (via WithIdenticalViaEquality plus an operator==, or WithIdentical and your own function). If you skip that, the engine falls back to comparing the serialised bytes, which works but costs a serialise on every comparison. For hot structs, provide an explicit equality.

You also own versioning. There is no schema; the writer and reader are the same function, so as long as you ship them together you are fine, but a struct that crosses a build boundary (saved games, replays) needs you to think about format changes deliberately.

next

We can now make a single value as small as it deserves to be. But the next problem is lists. A replicated TArray of these structs resends the whole array when any element changes, which is exactly the kind of waste we have spent this part avoiding. Part 4 introduces FFastArraySerializer, which sends only the items that actually changed.