SQL, typed queries and graph views

Typed query evaluation, graph relations, retained snapshots, cursors and bounded COUNT.

The plan represents one collection, scalar predicates, projection names, sort and a row limit. Evaluation runs against a retained immutable snapshot. Default ordering is bytewise record ID. A snapshot whose branch differs from the plan returns InvalidBranch before iteration or result allocation. Field sorting uses ascending categories missing, null, boolean, signed integer, string; descending reverses the primary key only, with record-ID ties always ascending. Strings compare bytewise. Object and array sort values return UnsupportedSortValue.

Numeric predicates and sorting use exact signed 64-bit integers. Integer JSON text and integer operands compare numerically without float conversion. Fractional, exponent, overflowing and floating representations return UnsupportedNumber when used in numeric comparisons or sorting. Numeric predicate operands are validated before scanning, even for empty collections. Different scalar kinds compare unequal (ne true; ordering predicates false), without coercing or parsing unsupported numeric storage against strings. Missing comparison fields never match, including ne; exists distinguishes missing from present null. Ordinary retrieval preserves other JSON numbers.

A nonempty projection replaces only the body with requested existing fields; missing requested fields are omitted. Original envelope metadata, including unknown fields, remains present. Empty projection preserves original bytes. Returned bytes are owned copies independent of snapshot storage.

The evaluator counts every snapshot iterator entry against the traversal budget, including indexes and other non-document namespaces. It bounds candidate count, candidate bytes, output rows and final response bytes. Defaults permit 10,000 candidates and 16 MiB of candidate accounting, which includes IDs, encoded rows, owned string sort keys and candidate structures. Checks precede owned candidate allocations. Array capacity and bounded per-record parsing/projection scratch are additional allocator overhead, not a claim of an exact process-memory cap. Per-record source parsing is bounded by the 64 KiB document envelope; candidate count bounds container capacity. Candidate limits apply before final sorting/row limiting; exceeding them returns LimitExceeded, never a silently incomplete result. Final response overflow returns ResponseTooLarge.

Limits.budget optionally supplies a borrowed context and fallible checkpoint callback for this synchronous call. The caller owns monotonic deadline and cancellation policy; the executor never reads a clock. Checkpoints run before work, between all snapshot entries, during projection, within in-place heap sorting, while assembling output and before publication. Callback errors pass through unchanged and release partial results. Enforcement is cooperative: a bounded record parse/serialization, comparison or allocation can finish before the next check; this is not hard real-time preemption. A null budget preserves existing standalone callers and supplies no timeout. The service must inject a real timeout policy when this executor is integrated; neither the default nor these tests establish a two-second service deadline.

Allocation failure returns OutOfMemory instead of silently omitting a valid row; malformed stored records retain the existing skip behavior. Projection, sorting and partial construction have exhaustive allocation-failure coverage.

The cursor-store module holds retained snapshots; service integration remains separate. Each binds its plan hash, principal, scope, authority epoch and expiry. Reusing one after revocation, expiry, changed scope or a different plan returns stale_cursor; it never moves to a newer head. take reclaims expired entries using the caller's supplied time; expire(now_ms) lets the owning service reclaim idle entries before admission. An entry remains valid at its exact deadline and expires strictly after it. The owner must schedule reclamation; there is no hidden clock or background worker. Failed retention keeps snapshot ownership with the caller and does not consume a token. Tokens never repeat: the maximum is issued once, then retention reports CursorLimit until a new store is created.

Relations use separate authoritative relation/<id> records with from, to, kind and JSON properties. Creation requires live same-branch endpoints and lowers the edge plus relation-forward/ and relation-reverse/ entries into one engine batch. Delete-with-edges lowers the endpoint, every incident edge, and every paired index entry into that same batch. Traversal is bounded breadth-first and records visited nodes, so cycles end. Endpoint deletion must reject incident edges unless the caller first constructs the explicit atomic delete-with-edges batch.

Partial query, graph and cursor construction releases owned allocations on failure. Graph traversal propagates allocation errors instead of reporting incomplete reachability as a successful result. Returned graph bytes outlive temporary JSON parsing storage.

Internal snapshot document pages#

query_execute.documentPage accepts a caller-retained immutable snapshot and query_plan.DocumentPage containing branch, collection and optional raw after_key. The canonical key is strictly exclusive and must belong to that collection. Branch zero, control branch, branch mismatch and protected collections refuse. Returned rows include tombstones, retain unchanged envelope bytes and follow canonical stored-key byte order. Every visited member is validated; malformed keys/envelopes and identity disagreement fail the call. This operation does not inherit ordinary query malformed-record skipping.

Each page owns its key/envelope buffers and optional continuation, released by DocumentPage.deinit; these buffers outlive the snapshot. Limits can lower but cannot raise 200 rows, 64 KiB combined keys/envelopes/continuation or 100,000 visited entries. Zero row/visit limits refuse. Parsing uses reusable 256 KiB scratch. Response bytes exclude container and parsing overhead. Visit exhaustion refuses. A row that cannot fit the first page refuses; a later boundary returns complete=false with the last emitted key, preserving the next row for retry. Only observed exhaustion returns complete=true with no continuation. Exact terminal byte boundaries need no continuation reservation. Budget checkpoints run before traversal, between entries and before publication; callback failures release partial buffers.

Raw positions carry no authority. Public cursor binding, expiry, aggregate budgets and service/native/HTTP integration remain separate. Internal collection and authoritative relation pages provide the next bounded membership primitives. These internal prerequisites do not complete D21 or D22.

Internal query_execute.run accepts an optional borrowed qualified_generation for the exact root and collection. A top-level equality predicate on a qualified unique field selects at most one document, then uses the existing residual, ordering and projection logic. Unsupported predicates and absent qualification retain the collection scan. A stale or mismatched qualification refuses even when the plan would otherwise scan. Exact numeric tokens use the existing i64 parser; NULL and missing remain distinct. Query counters exclude the earlier bounded full-generation qualification. The owner-only Application subset described below now supplies maintained generations; this internal path alone does not qualify wider product workflows.

Selective lookup refuses escaped prefix/hit keys beyond max_index_key_bytes (default 64 KiB) before their allocations. Work counters count admitted derived hits and decoded candidate documents, not physical map-node visits or terminal lookahead. Qualification scans remain separately accounted.

Configured owner-only Application queries audit their already-authorized pinned root and then borrow its qualification for synchronous execution. The audit limits default to 16 visited document keys (including tombstones), 128 derived entries and 64 KiB key accounting; selected writes use the same admission limits. The full-generation audit is a bounded scan with cancellation checks before and after it, not mid-scan interruption. Its cost is separate from selective-reader counters, so this first subset claims no sublinear total query cost or performance advantage. There is no qualification cache, user-controlled generation or new wire credential. Current delivery checks remain in place; malformed/stale data cannot produce a stale indexed success. Current Application consumer evidence covers unique-text maintenance; internal scalar tests separately cover exact numeric tokens, escaped text, NULL/missing and booleans. Scoped grants, general backfill/migration, efficient proof reuse and full F03/F05 acceptance remain open.

Filtered COUNT over the same snapshot#

The owner-authorized query operation accepts aggregate:{"operator":"count"} with a collection and existing predicates. It returns {"aggregate":"count","count":"3"} in the existing correlated response and snapshot/view metadata. The count is unsigned decimal text, including zero, and appends no receipt or journal entry. This counts matching live rows; it is not COUNT(field), SQL parsing, GROUP BY, joins or vendor compatibility.

Missing and NULL both contribute to an unfiltered count. Existing predicates keep their meanings: exists includes NULL, missing fails comparisons, and explicit NULL equality is supported. Tombstones do not count. Aggregate requests refuse explicit limit, projection, sort, cursor/after or continuation syntax. Results issue no cursor. Unsupported operators and malformed aggregate objects refuse. Ordinary row queries retain their prior behavior and hash identity.

Typed Plan.aggregate = .count requires limit = 0, no projection and no sort. query_execute.count(allocator, snapshot, plan, limits) shares the existing scan/qualified-candidate and predicate evaluator, returns checked u64, and materializes no result rows. run refuses aggregate plans. Row limits never truncate the count. Physical visited entries, including tombstones, decoded candidates, parser scratch and index-key allocations remain bounded; exhaustion, corruption or cancellation refuses instead of returning a partial count. Canonical document address is checked before COUNT skips a predicate or tombstone.

A qualified unique-equality candidate keeps exact-root/generation fences and residual semantics. Candidate counters do not include the separate full generation audit and must not be presented as total query complexity. Application uses the same authorized pinned root, deadline, qualification and delivery checks as ordinary queries. No new cache, permission or durable profile is introduced.

Two focused local tests passed against the maintained Application and pure evaluator: literal membership and counts, missing/NULL, tombstones, old roots, exact retry, refused syntax/authority, no row-limit truncation, physical/candidate counters, stale proof/collection refusal, and live/deleted address corruption refusal. Source and logs are retained in the execution packet evidence. This is a bounded COUNT subset; wider relational/document operators, migration parity and whole-phase or release acceptance remain open.