snapback2 0.0.1

package/dist/guide/quarry.md

@@ -0,0 +1,297 @@
+# Quarry v0.3
+
+Quarry is Snapback 2's only backend authoring language. It is a bounded
+front end to the compiler, checker, evaluator, and server. `snapback2 check`
+compiles `.q` in memory; it
+does not place generated files in `snapback/`. `snapback2 check` writes an inspection copy under `.snapback/build/quarry/`.
+
+## Project
+
+A project has one schema, `snapback/schema.q`. Declarations (`use identity`,
+tables, maintains, channels) belong only there; every other `.q` file contains
+at least one query or mutation and its stem is the API namespace. Thus
+`snapback/messages.q` exports `api.messages.thread`.
+
+TypeScript under `snapback/` is limited to `seed.ts`, `tests/*.test.ts`, and
+effect modules that export effects only. Effect bodies remain JavaScript I/O;
+tables, rules, channels, maintains, queries, and mutations are Quarry. An
+effect module's stem must differ from every `.q` stem.
+
+## Types and tables
+
+Columns and arguments use `principal`, `id`, table references, `int`, `time`,
+`bool`, `decimal <scale>`, `money <CUR> [scale]`, `text <=N`, `text M..N`,
+`json <=N`, `bytes <=N`, `enum(...)`, and bounded lists `[type] <=N`. Decimal
+and money scales are integers from 0 through 12; money defaults to scale 2 and
+its currency is exactly three uppercase letters. Add `?` for optional. Text
+can add `format handle` or `format url` after its bound. A minimum and a format
+become enforced text constraints; maximums remain part of the column type.
+Text values may not contain C0 control characters other than tab, newline,
+and carriage return.
+
+`image <=N` is a byte-bounded closed record `{ id, width, height }`; `video <=N`
+adds integer-millisecond `duration`. Both compose with `?` and bounded lists,
+and their fields work in expressions. Asset columns cannot be indexed or used
+in `maintain.by`.
+
+```q
+mutation inspect(photo: image <=5000000, clip: video <=50000000):
+  require clip.duration <= 60000 else TOO_LONG
+  return { photo, width: photo.width, clip }
+```
+
+Decimal/money literals are whole units (`2.50` → `250`); excess precision
+refuses. Matching nominal brands admit `+ - < <= > >= = != min max`; unary
+`-`/`* int` preserve them. Money × decimal → money either order; decimal ×
+decimal → left brand; money × money refuses.
+`/` and `%` rewrite to `divmod(a, b).quotient` and `.remainder`.
+
+`divmod(a,int)` returns `a`-branded fields and is Euclidean: remainder is
+nonnegative; quotient equals floor for positive divisors. `share` keeps the
+amount brand, requires matching bounded recipients, and assigns remainder units
+by ascending value. Share/branded multiplication floor because divisors are
+positive; i53 products can overflow. Wire arguments use integer minor units.
+Decimal/money seeds use whole-unit JSON numbers or decimal strings and are
+exactly normalized to minor units; exponent notation and excess precision refuse.
+Shaped rows and pages are typed and keep their brands.
+
+`principal` is the nominal type of `viewer`. It is neither a table reference
+nor `id`, even though all three have ID-shaped wire values. A column compared
+with `viewer`, or assigned `viewer`, must be `principal`.
+A text literal in an Id position (key, cursor, equality, or `??` fallback) is
+an Id value on the wire.
+
+Tables get `id` and `byId` implicitly. `by name: a, b` declares a named
+ordering; `by a, b` infers `byAB`. `unique name: a, b` declares a named unique
+ordering and its uniqueness constraint; `unique a, b` infers `byAB`. `id` may
+appear in an index. `immutable a, b` forbids changes to those columns. `public`
+accepts an optional reason string. An insert
+postcondition is `after insert <- exists table[...] [by index]`.
+
+`retain delivered-history` keeps rows that went hidden; the default evicts them.
+
+Rules are `allow`, `deny`, `public`, `system`, `exists`, `created`, `and`,
+`or`, `not`, and `=`/`!=` over `viewer`, literals, `.field`, and channel
+arguments. `created` is insert-only. Table rules omit a binder (`read <- …`);
+`.` is the row, or the candidate image for update. Only `update (old, next)`
+names binders.
+Rule `exists`/`created` accepts any index whose leading columns match the key;
+when several match, add `by index`. Parentheses allow multiline rules.
+Reciprocal references and rule dependencies are supported.
+
+`use identity` enables passwords and guests. `use identity password` and
+`use identity guests` select only that method.
+
+## Maintains
+
+Count/sum maintains copy group types into a target: count value is int; sum
+value matches its int/decimal/money source. The group
+index is unique; the block gives its read rule; user writes are denied.
+
+The optional `readable <- rule` is a deliberate aggregate disclosure. It
+allows certification to rely on reading a maintained summary when scanning
+the raw source would not have a uniform read rule; the certificate lists that
+choice as `E_DISCLOSURE`. It does not replace the target table's ordinary
+`read` rule.
+
+`total[key]` reads the target row through the unique grouping index.
+`#total[key]` yields its typed value or typed zero when absent/unreadable. A
+later `#` read in a mutation reflects its earlier writes.
+
+```q
+maintain messageCounts = count messages by conversationId:
+  read <- exists memberships[.conversationId, viewer]
+```
+
+## Programs
+
+Keys are positional (`table[value, …]`) or named
+(`table[column = value, …]`), as in `users[principal = viewer]`. Point reads
+need a complete unique key. Range order is `[after cursor] first|last N
+[by index]`; repeats/permutations refuse. Point reads may add
+`by index`; `exists table[keys]` accepts an index prefix. Shapes use `{ * }`,
+bare fields, or `name: expression`; nested work stays bounded. Byte ceilings
+bound canonical JSON: a shape is priced by the fields it selects and `*` by
+every column at its declared maximum (text at `4·max + 16` bytes), so focused
+shapes certify larger pages. An operations module may declare
+`shape card = { fields }`, use `row card`, `page card`, or a named shape as a
+field value, and splice it with `...card`; expansion keeps definition sites,
+refuses cycles, and is capped at depth 8.
+
+A pageable query declares `c: cursor ?` and reads `after c`; absent means page
+one. `complete` and `capped` results carry `next`, either a server-minted opaque
+cursor or `null`. Pass it back unchanged. It encodes the table, index, order,
+typed post-prefix suffix, and last returned row—never a hidden examined
+position. A mismatched or noncanonical token is `E_INPUT` before any scan.
+Tokens are not operation-site-bound. The index must be total (end in `id` or be
+unique); `after` follows scan order.
+
+`merge table [after cursor] first|last N by index:` fan-ins one or more
+indented lanes. Each lane is `[prefix keys]`, optionally `for name in` a bounded
+range read. Prefixes must have equal widths and types, leave an index suffix,
+and not be structurally identical. Quarry scans every lane at N, concatenates,
+sorts by the shared suffix, deduplicates intrinsic `id` keeping the first row,
+then returns the first N as a page. Shapes, `for`, and `count` work as for a
+range page; an unshaped API result uses the table's row type. A pageable merge
+index ends in `id`; its shared cursor is applied unchanged to every lane.
+
+```q
+query followingFeed(c: cursor ?):
+  page = merge posts after c last 20 by byAuthorTime:
+    [viewer]
+    [f.followed] for f in follows[viewer] first 50 by byFollower
+  return page { post: ., author: profiles[.author] authorCard }
+```
+
+This recipe admits at most 50 follow rows. The query reports `capped` when that
+domain hits its limit; it does not meet a product that must cover larger follow
+graphs. A merge is `complete` only when every member lane scan exhausted and
+the deduplicated union had at most N rows; otherwise it is `capped`.
+
+Statements are expression bindings, tuple bindings from `sorted`, `return`,
+`require condition else CODE`, `if/else`, bounded `for`, and
+`match read: some name: … none: …`. `return` is valid at top level and inside
+all arms. The last expression of any body is an implicit return. Match is a
+statement, not a binding expression.
+
+Object literals work anywhere expressions do; `{ id: conversationId,
+creatorMembershipId }` uses shorthand for the second field. `count(x)` is
+`x.length` for bounded lists and pages. Other built-ins are `tokens`, `range`,
+`min`, `max`, `divmod`, and `share`. `sorted` is only the two-name form
+`(lo, hi) = sorted(a, b)`. `walk table[seed] up .parent depth N` is bounded.
+`!` or `! else CODE` requires a present value; `??` supplies an absence
+fallback. Program `=`/`!=` are total: absent equals absent and differs from
+present; `!=` is `not (=)`. Ordered comparison refuses an optional operand;
+use `!` or `??` first. A direct `if x != null:` narrows a bound name or one-hop
+row field in that then branch; it does not narrow an `and` operand or the else.
+Conditions require Booleans.
+
+`now` is the read clock in queries and the commit clock in mutations; `commit`
+is the commit clock. In a mutation they are the same instant. `now` is pinned
+once per request rather than redrawn; `new` allocates a fresh ID on each
+evaluation.
+
+Writes are `insert table { fields }`, `update table[key] { patch }`,
+`upsert table[key] { row }`, and `delete table[key]`; a non-insert key is any
+complete unique key, composite alternate keys included. Alternate-key update
+and delete read the unique index once, require the row (`NOT_FOUND`), and write
+the base table by its intrinsic `id`; alternate-key upsert passes the matched
+`id`, or a fresh `new` id, to the platform's scalar-key `db.upsert`.
+
+Key expressions are evaluated once. An upsert injects omitted key columns from
+them; a row or patch may spell a key column only with the same expression as
+the key, otherwise `E_QUARRY_TYPE` at that field with a drop-field rewrite. An
+upsert row may not spell `id`: by-id derives it from the key; alternate-key
+keeps a present row's id or mints one on the absent insert branch. Insert yields the inserted row;
+update, upsert, and delete yield no value. Each inserted row expression is
+evaluated once, including inline `new`.
+
+## Effects
+
+A TypeScript module exports the effect body and imports its `then` mutation
+from the virtual module emitted for a Quarry file. The chat template splits
+the two declarations this way.
+
+`snapback/notifications.q`:
+
+```q
+mutation recordNotified(outboxId: id, effect: text <=100, result: json <=1000):
+  insert notifications {
+    outboxId,
+    effect,
+    providerId: result.providerId,
+  }
+  return true
+```
+
+`snapback/delivery.ts`:
+
+```ts
+import { effect } from "snapback2";
+import { recordNotified } from "./notifications";
+import { notifications } from "./schema";
+
+const baseHeaders = { accept: "application/json" };
+const requestDefaults = {
+  method: "POST",
+  headers: { ...baseHeaders, "content-type": "application/json" },
+};
+
+export const notify = effect({
+  grants: ["https://api.example.com"],
+  run: async (payload: { to: string; body: string }, { fetch, meta }) => {
+    function identity(value: string) { return value; }
+    const formatter = { format(value: string) { return identity(value); } };
+    const body = [payload.body].map(function keep(value) { return formatter.format(value); })[0];
+    try {
+      const response = await fetch("https://api.example.com/send", {
+        method: requestDefaults.method,
+        headers: { ...requestDefaults.headers, "idempotency-key": meta.id },
+        body: JSON.stringify({ ...payload, body }),
+      });
+      if (!response.ok) throw new Error(`provider ${response.status}`);
+      return { providerId: (await response.json()).id as string };
+    } catch (error) {
+      throw error;
+    }
+  },
+  then: recordNotified,
+});
+```
+
+`snapback/messages.q`:
+
+```q
+effect notify from delivery
+
+query thread(conversationId: conversations, c: cursor ?):
+  require exists memberships[conversationId, viewer] else MEMBERSHIP_REQUIRED
+  return messages[conversationId] after c last 50
+
+query summary(conversationId: conversations):
+  require exists memberships[conversationId, viewer] else MEMBERSHIP_REQUIRED
+  return {
+    conversation: conversations[conversationId],
+    messageCount: messageCounts[conversationId],
+  }
+
+mutation send(conversationId: conversations, recipientId: profiles, body: text <=4000):
+  require exists memberships[conversationId, viewer] else NOT_MEMBER
+  recipient = profiles[recipientId] ! else RECIPIENT_UNKNOWN
+  require exists memberships[conversationId, recipient.principal] else RECIPIENT_NOT_MEMBER
+  row = insert messages { conversationId, authorId: viewer, body, photo: null, clip: null, at: now }
+  for m in memberships[conversationId] first 50 by byConversationPrincipal:
+    insert deliveries { recipient: m.principal, conversationId, messageId: row.id, kind: 'message', at: now }
+  emit notify { to: recipient.principal, body }
+  return { id: row.id }
+```
+
+Edit/unsend target current members. Departed recipients evict unsent bodies by
+point absence on held-page revalidation, not tombstones (LLP 1012 §5).
+
+## Absence and bounds
+
+Point reads return a row or absence; absent shapes and optional members stay
+absent. `!` names a refusal and `??` handles absence. Every missing update or
+delete is `NOT_FOUND`; upsert keeps its absent insert branch. The residual
+existence-oracle case is by-id unreadable → `E_RULE`, alternate-key unreadable
+→ `NOT_FOUND`. A
+missing maintained group is zero only through `#`; the ordinary target-table
+read remains absent.
+
+Every scan, list, text, JSON, and bytes value is bounded. `for` iterates only
+a bounded list/page/range/tokens result. Lowered queries and mutations retain
+the same certificate, trace, and runtime budgets.
+
+## Refused platform gaps
+
+The following are refused with `E_QUARRY_UNSUPPORTED`: `schedule`, `where`,
+`device`, `oblivious`, `while`, user functions or recursion. Use a caller-invoked
+mutation instead of a schedule; explicit bounded arguments instead of
+device/oblivious state; bounded `for`/`walk` instead of while/recursion. An
+equality-conjunction `where` over indexable columns returns two repairs: the
+schema index line and the indexed prefix read; other predicates name the
+index, invariant (`after insert <- exists …`), or `merge` alternatives.
+
+Writes through a non-unique or incomplete key are index-selection refusals.
+

package/dist/guide/testing.md

@@ -0,0 +1,57 @@
+# Testing
+
+The seed is static data, applied as one system commit before derived checks and
+authored journeys:
+
+```ts
+    profiles: [
+      { id: "profile-alice", principal: "dev:alice", displayName: "Alice", handle: "alice", avatarUrl: "https://example.com/alice.png", bio: "", location: "", link: null },
+      { id: "profile-bob", principal: "dev:bob", displayName: "Bob", handle: "bob", avatarUrl: "https://example.com/bob.png", bio: "", location: "", link: null },
+    ],
+    conversations: [
+      { id: "conversation-1", title: "General", creatorId: "dev:alice" },
+      { id: "conversation-2", title: "Side room", creatorId: "dev:alice" },
+    ],
+    memberships: [
+      { id: "membership-1", conversationId: "conversation-1", principal: "dev:alice", role: "admin", joinedAt: 0 },
+      { id: "membership-2", conversationId: "conversation-1", principal: "dev:bob", role: "member", joinedAt: 0 },
+      { id: "membership-3", conversationId: "conversation-2", principal: "dev:alice", role: "admin", joinedAt: 0 },
+      { id: "membership-4", conversationId: "conversation-2", principal: "dev:charlie", role: "member", joinedAt: 0 },
+    ],
+    messages: [
+      { id: "message-1", conversationId: "conversation-1", authorId: "dev:alice", body: "Welcome", at: 1 },
+      { id: "message-2", conversationId: "conversation-2", authorId: "dev:alice", body: "Side note", at: 1 },
+    ],
+```
+
+Here `profile-alice` is an application row id while `dev:alice` is the persona
+Principal used by creator, membership, and author columns. Identity email and
+password hashes are deliberately absent from application seed rows. These
+`avatarUrl` strings are external URLs, not uploads; asset values are deliberately
+absent because seeds cannot mint asset ledger rows (LLP 1006 §2a). The chat
+fixture also keeps `charlie` as an account/persona without a profile so the
+idempotent post-auth profile-creation path has a real absent-row proof.
+
+Write journeys with `journey(name, async ({ as, tick, url }) => …)`. The test
+verb starts a fresh isolated owner, runs all six derived check families, runs
+every journey through the headless wire client, removes the test store, and
+writes `.snapback/receipts/<timestamp>.json`. Zero checks cannot pass. Use
+`snapback2 test --blocked "reason"` for an honest blocked receipt.
+
+Projects with scoped memberships also run the canonical `live-revocation`,
+`successor-policy-fence`, and `grant-incarnation-aba` authorization proofs.
+They bind revocation delivery, active-generation policy, and immutable grant
+incarnation to the served loop rather than treating lower-level unit coverage
+as the acceptance result.
+
+Derived checks use a small representative seed, not a load-test dataset.
+Rules compare every stored row for every seed persona, within a 16,384-probe
+budget; oversized tables block with the counts and a smaller-seed rewrite.
+Invalidation witnesses each query's distinct certificate-intersection groups
+and concrete-disjoint silence, rather than testing every query/mutation pair.
+The receipt names the candidate groups and actual physical attempts. Mutation
+search uses the first two scopes and six personas; if no committing witness is
+found within the limit, the check blocks. Replay still covers every generic
+mutation independently. Put operation-specific behavior and larger scenarios
+in authored journeys, and keep useful admitted and excluded personas and scopes
+near the start of the seed.