Foundation

Journal and framing

The local journal's ownership, sequence and integrity rules, with clear format boundaries.

Status: implemented byte-framing foundation in frame.zig. This is not an implemented durable log or database. The codec performs no filesystem I/O, replay, repair, transaction validation, or acknowledgement.

API and ownership#

Frame contains kind: RecordKind, sequence: u64, and payload: []const u8.

encode(allocator, frame, max_payload: usize) EncodeError![]u8 creates one owned byte slice. Free it with the supplied allocator. The payload limit and length arithmetic are checked before allocation.
decode(input: []const u8, max_payload: usize) DecodeError!Decoded validates the first complete frame without allocating. Decoded contains frame: Frame and consumed: usize.
A decoded payload borrows input; it remains valid only while that input storage remains alive and unchanged. Unaligned input is supported.
For concatenated frames, advance by consumed and decode again. The codec does not examine bytes after the first frame. The caller must inspect any remaining bytes rather than silently ignoring them.
For fragmented input, retain the prefix after Incomplete and supply the combined bytes when available. This module does not maintain an accumulator. The caller must bound its own buffering and total stream size.

Both APIs require an explicit maximum payload size. There is no permissive default. Empty payloads and all u64 sequence values are representable; sequence ordering, uniqueness, and payload semantics belong to a higher layer.

Version 1 encoding#

All integer fields use unsigned little-endian encoding. The fixed header occupies 64 bytes, with no native struct padding or alignment requirement.

Offset	Bytes	Field
0	8	ASCII magic `RURALWAL`
8	2	Format version, currently `1`
10	2	Record kind: transaction `1`, checkpoint `2`, catalog `3`
12	4	Reserved; must be zero
16	8	Sequence number
24	8	Payload length in bytes
32	32	SHA-256 digest
64	Payload length	Opaque payload

The digest is SHA256(bytes[0..32] || payload), calculated with Zig's standard-library SHA-256 implementation. It covers magic, version, kind, reserved bytes, sequence, payload length, and payload; the digest field itself is excluded. Encoding the same frame always produces identical bytes.

SHA-256 detects accidental changes to the protected bytes; it is not keyed authentication, encryption, or proof of an authorized writer. Someone able to rewrite the file can also recompute the digest. Checksums do not prove commit, acknowledgement, sequence continuity, or payload validity.

Error contract#

Error	Meaning
`Incomplete`	More bytes are needed to determine whether the first frame is valid.
`InvalidMagic`	The available magic prefix does not match.
`UnsupportedVersion`	A complete version field is unsupported.
`InvalidKind`	A complete kind field is unknown.
`NonZeroReserved`	Reserved header bits are set.
`PayloadTooLarge`	The representable payload length exceeds the caller's limit.
`LengthOverflow`	The payload length cannot fit the host's `usize`, or adding the header would overflow; encoding also rejects lengths outside the format's `u64`.
`ChecksumMismatch`	A complete, structurally valid frame fails integrity verification.
`OutOfMemory`	Encoding could not allocate its output. Decoding never allocates.

Known invalid header fields are rejected as soon as their bytes are present. Once the first 32 bytes are available, decoding checks length representability, the configured limit, and header-plus-payload overflow before waiting for payload bytes. On a host narrower than 64 bits, an unrepresentable length reports LengthOverflow before a payload-limit check. SHA-256 verification happens only when the entire declared frame is present; its work is linear in that bounded payload length.

A damaged length that announces additional bytes may return Incomplete rather than ChecksumMismatch. This is unavoidable without the missing bytes or additional trusted state. An incomplete suffix alone cannot distinguish a partial unacknowledged write, corruption, a truncated previously acknowledged frame, or an input fragment still in transit.

The codec never scans forward for another magic value, repairs data, truncates a tail, or skips an invalid frame. The storage layer must establish which history was committed or acknowledged and define recovery policy separately. Unknown versions and kinds fail explicitly; changing the format requires a compatibility decision.

Verification#

Inline tests cover an independent exact-byte fixture, every prefix truncation of representative frames, all record kinds, empty/binary payloads, extreme sequences, unaligned and concatenated input, incomplete and invalid successors, malformed headers, malicious lengths, all single-bit mutations of the fixture, checksum coverage, and allocation failure with leak checking.

The fixture is a transaction with sequence 0x0102030405060708 and payload hello. Its SHA-256 digest is ac68a7ceb81dec1e9c49e4dd5bf4314173e1681f9eaba34cf848ecbf77fe4290.

These tests establish framing behavior only. Filesystem durability, durable publication, exclusive ownership, acknowledged-prefix recovery, checkpoint replacement, and disk-failure testing remain separate implementation work.