Local development protocol

Authenticated local HTTP admission and the shared document service contract.

P07 supplies the loopback-only HTTP admission layer for the five prototype routes: command, query, event batch, status, and changes. It accepts only bounded JSON POST requests with an allowed host/origin and a credential from the bounded registry. The owner token and scoped read-only tokens resolve to different principals; command and event routes deny read-only credentials.

src/service/authority.zig keeps the current credential epoch separate from engine freshness. src/service/runtime.zig validates the endpoint descriptor's store and session identities before a client uses it. Domain handlers are injected route-by-route and missing handlers fail closed.

Shared local document adapter#

LocalDocumentService borrows one database and a concurrency-capable allocator. Keep the service instance immovable from the first concurrent call until all callers have stopped and joined; its allocator, database and I/O must outlive those calls. All mutations for that database must use the same service gate; separate adapter instances or direct command/database writers bypass this serialization and are not covered by its guarantee.

Create, replace, delete and relation creation hold the service mutation mutex across receipt lookup, snapshot-dependent lowering and durable commit. Lock order is service admission, then the database's existing writer/publication locks. Durable I/O under admission is intentional: releasing admission before commit would reopen the retry/lowering race. Waiting acquisition is cancelable; only a successful acquisition installs its matching deferred unlock. A failed validation or commit releases admission without granting success or rolling back an uncertain durable effect. This service mutation mutex is not the Application command gate.

The typed query(plan, limits) acquires a retained database snapshot briefly, then evaluates outside the mutation gate. It forwards the caller's complete query limits and budget callback. Concurrent mutations may advance the database while the query retains its original view. Every query exit releases that snapshot; successful result rows belong to the caller and must be released with query_execute.deinitResults using the service allocator. Budget callbacks run outside service admission, so they may report cancellation or allow unrelated writes to progress. They must obey their own lifetime and synchronization rules.

The focused service tests use real temporary databases for same-ID concurrent logical retries, exact original receipts/sequence counts, failed/canceled admission, and a replacement during query evaluation followed by a fresh query. These local adapter guarantees do not establish C05 dynamic durable authority, HTTP/loopback bootstrap, revocation-at-delivery admission or a complete concurrent native/service application.

Application command sequencing#

The owner-only loopback Application now submits command requests to one caller-led FIFO queue. Each request handler keeps its parsed request, result arena and queue job alive until completion; no background commit worker owns them. The queue accepts at most 32 waiting requests and 4 MiB of charged request bodies, with a separate active group of the same limits. It combines only adjacent document commands already waiting when the oldest caller takes its turn. Schema/control writes, branch changes, merge and backup are ordered barriers. A canceled waiting request leaves the queue; cancellation after admission waits for its outcome before releasing request memory. A canceled response can still have a durable effect, so retry with the same request ID.

Document group members validate against each accepted staged predecessor, retain independent rejection/replay outcomes and share one durable sync for accepted frames. A group falls back to FIFO runSerial on the same turn only for GroupIntentLimit, GroupCowBudget, GroupMutationLimit, GroupPayloadLimit, or OutOfMemory. Those five are a pre-write retry, before any journal write. appendGroupExpected calls checkSize, which stats the file before allocating frames, so OutOfMemory can follow that stat and still precede any write. All frames are allocated before the first write. A write or sync failure is AppendOutcomeUnknown and is not retried. The fallback is not LogLimitExceeded, unknown sync, or cancel after a write may have started. It clears member proof, response, failure, receipt, and replayed state, clears intake, unlocks once, then runs serial; each serial command takes its own gate. A fired hook is not a fallback grant. The database lazily owns one bounded 8 MiB preparation buffer for grouped attempts and reuses it under its writer gate. A closed or recovery-fenced owner does not allocate it. It may remain resident after a failed attempt until the database closes, after all handlers have joined. First-allocation failure is still a preappend error eligible for the serial fallback; journal frames, published state and returned receipts do not borrow the retained buffer. The source change passed the frozen full correctness profile; the proposed grouped-latency regression guard remains open. The FIFO turn keeps the queued work. One stack GateTurn per handler uses held as its only gate flag. The hook only unlocks that mutex and sets held false. It takes no lock, calls no Database method, and does not take ownership of the turn. The gate stays held through preparation and drops only for a fresh, fully prepared append. A null hook, replay-only work, rejected work, and a group with no fresh frame leave the gate held. serialDatabase, groupDatabase, and documentsBatch reacquire with lockUncancelable only if held is false, after the database call returns and after the writer is released, and before cache, proof, and result bookkeeping. A fired hook is not proof that journal I/O began. Merge planning keeps the gate through captureMerge and mergePlan. readCompare does not take the Application gate. readIntake and Application.status are unchanged; status still uses database.status() plus two branchInfo calls and does not call statusWithBranches. readIntent captures control and main together using snapshotPair(2, 1) plus one control catalog load; a deleted main is BranchDeleted from that pair and precedes IntentDisabled. Reads may use retained snapshots outside the gate. Shutdown drain is required. Handlers must be stopped and joined, and the queue closed and drained, before Application, queue, database, or backing allocators are freed. After drain, finishPeriodicCheckpoints publishes once if the D1 trigger is still met. Releasing the Application gate around append does not make an in-flight handler freeable. Grouping is opportunistic; one request still uses the serial path, and no fixed batching delay or throughput/latency guarantee follows from this change.

Periodic checkpoints#

After each durable commit (serial or grouped), the Application evaluates the ADR-004 D1 trigger without holding the writer. Defaults: publish when at least 8,192 commits or 16 MiB of journal have accumulated since the last successful generation; at most one publish in flight; at least 60 s between attempts; skip while a profile backup holds the recovery gate. CheckpointOptions may override those values (synchronous forces the publish onto the caller for tests). A background publish reuses Database.publishCheckpoint. Status exposes checkpoint.last_generation, last_sequence, last_duration_ms, publish_failures and last_error. See checkpoint.md.

Snapshot-pinned reads#

pin_snapshot (query) retains the current (branch, head) root and returns a process-local lease_id with a 30-second TTL. At most 64 leases may be pinned per TCP connection and 1,024 per service. release drops a lease early. Document and analytics queries may pass lease_id to evaluate against the pinned root from any connection in the same process; after later commits the leased view stays fixed until expiry or release. Expired leases return lease_expired; full caps return lease_limit. Leases are not durable and do not cross process restarts. The P26 service driver (schema v2.1) requires a successful pin_snapshot, records lease provenance on the branches phase, and calls release when that phase ends. Expiry and root reclamation follow the query cursor store pattern.

python3 scripts/build_guard.py -- zsh scripts/zig.sh build test-prototype -Dcase=service -j1 --summary all
python3 scripts/build_guard.py -- zsh scripts/zig.sh build test-prototype -Dcase=pinned-reads -Doptimize=ReleaseSafe -j1 --summary all