@gnolith/taproot 0.1.0-rc.0 → 0.3.0

package/docs/product-scope.md

@@ -1,20 +1,27 @@
 # Product scope
 
-Taproot exists so a Codex Site can own a Wikibase-style knowledge graph in D1
+Taproot exists so a D1-backed application can own a Wikibase-style knowledge graph
 and query it through Diamond without operating MediaWiki or another external
 knowledge service.
 
-Version 0.1 is complete for the package's declared entity domain: Wikibase
+Version 0.3 covers the package's declared entity domain: Wikibase
 core Items and Properties, canonical JSON interchange, statements and all
 their components, supported datatypes, revision/lifecycle history,
 attribution and audit, search, RDF/SPARQL projection, bulk workflows,
-migration, integrity, repair, policy hooks, and operational documentation.
+migration, integrity, repair, canonical entity authorization policy, and
+operational documentation.
 
 The following are separate products or future compatibility domains rather
 than missing Taproot implementation: MCP transport, agent orchestration,
-authentication/authorization, UI, wiki pages, media bytes, MediaWiki APIs,
+authentication, principal/membership/session management, UI, wiki pages,
+media bytes, MediaWiki APIs,
 Lexeme/Form/Sense, MediaInfo, and EntitySchema entity storage.
 
+Complete-Site assembly, resource provisioning, remote migration execution,
+deployment, and production acceptance are also outside this package. The Codex
+agent creating a Gnolith Site owns those responsibilities. Taproot qualifies
+only its published package and supported local runtime contracts.
+
 Adding an extension entity type changes IDs, JSON, commands, RDF, migration,
 and compatibility fixtures. It must be designed and versioned as an explicit
 new domain rather than accepted as unvalidated JSON.

package/docs/release-policy.md

@@ -11,6 +11,10 @@ inspects the packed artifact, publishes to npm through OIDC trusted publishing,
 and attaches the tarball, SBOM, checksums, and build attestation to a GitHub
 Release.
 
+A Taproot release qualifies the published package and its supported local
+runtime contracts. It does not assemble, provision, deploy, or accept a complete
+Gnolith Site; the Codex agent creating a Site owns those responsibilities.
+
 The current minor line receives fixes. Security support follows `SECURITY.md`.
 Deprecations are announced in the changelog before removal whenever a safe
 migration path exists.

package/docs/testing.md

@@ -11,11 +11,37 @@ projection repair, shared RDF values, and injected RDF failures. The injected
 failure proves that canonical JSON, revisions, audit events, search terms,
 ownership, and Diamond quads roll back as one D1 batch.
 
-`test/example.test.ts` runs the documented Codex Site example and queries its
-Taproot-generated graph through Diamond SPARQL. `consumer:check` packs the
+`test/interop.test.ts` runs the documented local D1 and Diamond interoperability
+example and queries its Taproot-generated graph through Diamond SPARQL.
+
+`test/persistence.test.ts` exercises Diamond's Node SQLite adapter in memory
+and on disk, including read-only planning, immutable identity, close/reopen,
+same-adapter enforcement, late-batch rollback, and concurrent writers.
+`consumer:check` packs the
 package, installs the tarball into a fresh temporary project with registry
 Diamond and Miniflare dependencies, initializes a new D1 database, writes an
-entity, and verifies the RDF through SPARQL using only public package exports.
+entity, and verifies the RDF through privileged package-test SPARQL using only
+public package exports. This proves RDF interoperability, not authorization of
+a user-facing SPARQL endpoint.
+
+`npm run check` is the complete local package release-quality gate. Coverage
+thresholds are enforced in `vitest.config.ts`; lowering them requires an
+explained review. Miniflare/Workerd and packed-consumer success verifies the
+package's supported D1 runtime contract, not assembly, provisioning, deployment,
+or acceptance of a complete Gnolith Site. The Codex agent creating a Site owns
+those checks.
+
+## Miniflare dependency security
+
+Miniflare is a development-only dependency used to qualify Taproot against its
+supported Workerd D1 runtime. Taproot applies an exact root override of
+`sharp@0.35.3` because Miniflare currently declares `sharp@0.34.5`, which is
+affected by GHSA-f88m-g3jw-g9cj. This is a temporary upstream-compatibility
+exception tracked in [Cloudflare workers-sdk PR #14493](https://github.com/cloudflare/workers-sdk/pull/14493).
+Remove the override once a Miniflare release in Taproot's supported range
+declares a non-vulnerable Sharp version and the full D1, interop,
+packed-consumer, Node 22/24, and audit gates pass without it.
 
-`npm run check` is the complete local release-quality gate. Coverage thresholds
-are enforced in `vitest.config.ts`; lowering them requires an explained review.
+The override is qualified for Taproot's Miniflare D1 test use. It is not a claim
+of general Cloudflare Images compatibility, nor a compatibility claim for
+macOS, ARM, or musl environments.

package/docs/threat-model.md

@@ -8,9 +8,19 @@ association between an edit and its claimed attribution metadata.
 
 ## Trust boundaries
 
-The host application is trusted to authenticate and authorize callers, select
-the D1 binding, supply an absolute site-owned base IRI, and protect mutation
-routes. Attribution is a stored claim, not proof of identity. Diamond executes
+Taproot assumes its consumer supplies a SQLite/D1 binding and absolute base IRI.
+Hosts create `AuthorizationContext` values only from authenticated state;
+Taproot loads persisted canonical policies and checks them before and after
+hydration. The normal package export has no raw canonical repository. Every
+public mutation requires a runtime-branded installation guard, a current
+context with exact `knowledge:write`, current target visibility, and canonical
+policy. Policy changes require exact `knowledge:policy`. Invalid guards or
+contexts fail before database work or observers. Mutation receipts do not
+disclose canonical bodies. Host assembly withholds the database binding,
+host capability, guard, and keys from request, user, agent, and MCP code. The
+host capability is bootstrap/maintenance authority, while ordinary mutations
+remain per-entity policy authorized.
+Attribution is a stored claim, not proof of identity. Diamond executes
 the RDF patch prepared by Taproot and exposes read-only SPARQL querying; hosts
 must not give untrusted callers an independent write path to Taproot or Diamond
 tables.
@@ -19,22 +29,44 @@ tables.
 
 - Lost updates are blocked by expected-revision guards inside the same D1 batch
   as all canonical and projected writes.
+- Authorization ABA races are blocked by a shared counter CAS plus a unique
+  durable advance ID asserted inside the same ordered batch.
 - Partial writes are blocked by D1 batch atomicity; injected Workerd failures
   verify rollback across Taproot and Diamond.
 - History tampering is deterred and detected by immutable-table triggers,
   SHA-256 content/parent chains, schema inspection, and integrity APIs.
 - Resource exhaustion is bounded by entity, command, page, bulk, redirect, and
-  RDF patch limits. Hosts still need request-size, authentication, and rate
-  limits at the network boundary.
+  RDF patch limits. Request-size, authentication, and network rate limits are
+  outside the package boundary.
 - RDF projection confusion is constrained by strict entity validation,
   deterministic mapping, site-owned namespaces, and per-entity quad ownership.
+- Application/search canonical reads fail closed on absent policy,
+  cross-installation identity, stale authorization revision, policy changes
+  during hydration, and malformed CNF visibility. Denials have a generic body.
+- Search administration requires the exact `search:admin` capability; personas
+  and generic administrator labels do not imply it.
+- Page cursors require a non-extractable host AES-GCM key, use fixed-size
+  plaintext padding, and bind caller, grants, operation, query/filter, auth
+  revision, and search generation.
 - Dependency and release compromise is reduced through locked installs, pinned
   GitHub Actions, dependency review, CodeQL, secret scanning, license checks,
   OIDC npm publishing, provenance, SBOMs, checksums, and attestations.
 
 ## Out of scope
 
-Taproot does not provide authentication, authorization, agent/MCP isolation,
+Taproot does not provide authentication, principal/workspace persistence, agent/MCP isolation,
 network rate limiting, D1 backup custody, UI sanitization, or arbitrary SPARQL
 Update. A compromised host with direct D1 access can bypass library controls;
 restore from a trusted backup and verify audit chains after such an incident.
+
+Taproot's local Workerd tests validate its package controls under the supported
+D1 contract. They do not qualify a deployed host. The Codex agent creating a
+Site owns assembly, provisioning, route and security wiring, deployment, backup
+policy, and production acceptance.
+
+Diamond's database-level SPARQL handler sees Taproot's complete RDF projection
+and is therefore privileged host maintenance/debug infrastructure, not a
+normal user, agent, MCP, or search endpoint. Workshop must construct an
+authorization-scoped dataset (with a final canonical recheck) before exposing
+SPARQL results. Possession of the SQLite/D1 binding itself remains privileged:
+a compromised host can query tables outside any package API.

package/examples/d1-diamond-interop/README.md

@@ -0,0 +1,12 @@
+# Local D1 and Diamond interoperability example
+
+Pass a D1-compatible database to `runTaprootInteropDemo(db)`. The demo
+initializes Taproot and Diamond in the same database, creates Properties and an
+Item, applies revision-guarded edits, and queries Taproot's RDF projection
+through Diamond SPARQL. The SPARQL call is a package interoperability probe
+over a privileged database binding, not an authorization-safe endpoint.
+
+The automated test runs this example locally with Miniflare's Workerd D1
+implementation. It verifies the published package's D1 and Diamond contracts;
+it does not assemble, provision, deploy, or accept a complete Gnolith Site.
+Those responsibilities belong to the Codex agent creating that Site.

package/examples/d1-diamond-interop/demo.ts

@@ -0,0 +1,281 @@
+import {
+  RevisionConflictError,
+  SEARCH_ADMIN_CAPABILITY,
+  KNOWLEDGE_WRITE_CAPABILITY,
+  KNOWLEDGE_POLICY_CAPABILITY,
+  addStatement,
+  bootstrapTaprootAuthorization,
+  createAuthorizationCursorCodec,
+  createAuthorizedTaproot,
+  createItem,
+  createProperty,
+  createTaprootHostWriteCapability,
+  createInstallationAuthorizationGuard,
+  exportEntityJson,
+  initializeTaproot,
+  setLabel,
+  setStatementRank,
+  type CanonicalAuthorizationPolicyInput,
+  type D1DatabaseLike,
+  type Statement,
+  type AuthorizationContext,
+} from '@gnolith/taproot';
+import { createSparqlHandler } from '@gnolith/diamond';
+
+export async function runTaprootInteropDemo(db: D1DatabaseLike) {
+  await initializeTaproot(db, { baseIri: 'https://knowledge.example' });
+  const options = { baseIri: 'https://knowledge.example' };
+  const writeCapability = createTaprootHostWriteCapability(
+    db,
+    options,
+    await crypto.subtle.generateKey({ name: 'HMAC', hash: 'SHA-256' }, false, [
+      'sign',
+    ]),
+  );
+  await bootstrapTaprootAuthorization(
+    db,
+    options,
+    writeCapability,
+    'demo-installation',
+  );
+  const authorizationGuard = await createInstallationAuthorizationGuard(
+    db,
+    options,
+    writeCapability,
+  );
+
+  await createProperty(db, options, authorizationGuard, writer(1), {
+    datatype: 'string',
+    labels: { en: { language: 'en', value: 'occupation' } },
+    authorization: policy(1),
+  });
+  await createProperty(db, options, authorizationGuard, writer(2), {
+    datatype: 'time',
+    labels: { en: { language: 'en', value: 'point in time' } },
+    authorization: policy(2),
+  });
+  await createProperty(db, options, authorizationGuard, writer(3), {
+    datatype: 'url',
+    labels: { en: { language: 'en', value: 'reference URL' } },
+    authorization: policy(3),
+  });
+
+  const created = await createItem(db, options, authorizationGuard, writer(4), {
+    labels: { en: { language: 'en', value: 'Ada Lovelace' } },
+    descriptions: {
+      en: { language: 'en', value: 'English mathematician' },
+    },
+    attribution: {
+      id: 'agent:demo-curator',
+      kind: 'agent',
+      tool: 'gnolith-mcp',
+    },
+    requestId: 'demo-request',
+    authorization: policy(4),
+  });
+  const occupation: Statement = {
+    id: `${created.entityId}$occupation`,
+    type: 'statement',
+    text: 'Ada Lovelace worked as a computer programmer in 1843.',
+    rank: 'normal',
+    mainsnak: {
+      snaktype: 'value',
+      property: 'P1',
+      datatype: 'string',
+      datavalue: { type: 'string', value: 'computer programmer' },
+    },
+    qualifiers: {
+      P2: [
+        {
+          snaktype: 'value',
+          property: 'P2',
+          datatype: 'time',
+          datavalue: {
+            type: 'time',
+            value: {
+              time: '+1843-01-01T00:00:00Z',
+              timezone: 0,
+              before: 0,
+              after: 0,
+              precision: 9,
+              calendarmodel: 'http://www.wikidata.org/entity/Q1985727',
+            },
+          },
+        },
+      ],
+    },
+    'qualifiers-order': ['P2'],
+    references: [
+      {
+        hash: 'demo-source',
+        snaks: {
+          P3: [
+            {
+              snaktype: 'value',
+              property: 'P3',
+              datatype: 'url',
+              datavalue: {
+                type: 'string',
+                value: 'https://example.test/source',
+              },
+            },
+          ],
+        },
+        'snaks-order': ['P3'],
+      },
+    ],
+  };
+  const added = await addStatement(
+    db,
+    options,
+    authorizationGuard,
+    writer(5, true),
+    created.entityId,
+    occupation,
+    {
+      expectedRevision: created.newRevision,
+      authorization: policy(5, { [occupation.id]: [] }),
+    },
+  );
+  const preferred = await setStatementRank(
+    db,
+    options,
+    authorizationGuard,
+    writer(6),
+    created.entityId,
+    occupation.id,
+    'preferred',
+    'Ada Lovelace worked as a computer programmer in 1843 (preferred).',
+    {
+      expectedRevision: added.newRevision,
+      authorization: policy(6, { [occupation.id]: [] }),
+    },
+  );
+  const unknownOccupation: Statement = {
+    id: `${created.entityId}$unknown-occupation`,
+    type: 'statement',
+    text: 'Ada Lovelace had another occupation whose value is unknown.',
+    rank: 'normal',
+    mainsnak: {
+      snaktype: 'somevalue',
+      property: 'P1',
+      datatype: 'string',
+    },
+    qualifiers: {},
+    'qualifiers-order': [],
+    references: [],
+  };
+  await addStatement(
+    db,
+    options,
+    authorizationGuard,
+    writer(7, true),
+    created.entityId,
+    unknownOccupation,
+    {
+      expectedRevision: preferred.newRevision,
+      authorization: policy(7, {
+        [occupation.id]: [],
+        [unknownOccupation.id]: [],
+      }),
+    },
+  );
+
+  const sparql = createSparqlHandler({ db });
+  const query = `SELECT ?occupation WHERE {
+    <https://knowledge.example/entity/${created.entityId}>
+      <https://knowledge.example/prop/direct/P1> ?occupation
+  }`;
+  const response = await sparql(
+    new Request(
+      `https://interop.example/sparql?query=${encodeURIComponent(query)}`,
+      { headers: { accept: 'application/sparql-results+json' } },
+    ),
+  );
+
+  let staleRevisionRejected = false;
+  try {
+    await setLabel(
+      db,
+      options,
+      authorizationGuard,
+      writer(8),
+      created.entityId,
+      'en',
+      'stale edit',
+      {
+        expectedRevision: created.newRevision,
+        authorization: policy(8, {
+          [occupation.id]: [],
+          [unknownOccupation.id]: [],
+        }),
+      },
+    );
+  } catch (cause) {
+    staleRevisionRejected = cause instanceof RevisionConflictError;
+  }
+
+  const sparqlResults: unknown = await response.json();
+  const reader = createAuthorizedTaproot(
+    db,
+    options,
+    {
+      installationId: 'demo-installation',
+      principalId: 'demo-principal',
+      activeWorkspaceId: null,
+      workspaceIds: [],
+      capabilities: [SEARCH_ADMIN_CAPABILITY],
+      authorizationRevision: 8,
+    },
+    {
+      cursorCodec: createAuthorizationCursorCodec(
+        await crypto.subtle.generateKey(
+          { name: 'AES-GCM', length: 256 },
+          false,
+          ['encrypt', 'decrypt'],
+        ),
+      ),
+    },
+  );
+  const audit = await reader.listAuditEvents({ entityId: created.entityId });
+  const integrity = await reader.inspectEntityIntegrity(created.entityId);
+  const canonical = await reader.getEntity(created.entityId);
+  return {
+    entityJson: exportEntityJson(canonical.entity),
+    sparqlResults,
+    staleRevisionRejected,
+    audit: audit.items,
+    integrity,
+  };
+}
+
+function policy(
+  expectedAuthorizationRevision: number,
+  statementRestrictions: CanonicalAuthorizationPolicyInput['statementRestrictions'] = {},
+): CanonicalAuthorizationPolicyInput {
+  return {
+    installationId: 'demo-installation',
+    workspaceId: null,
+    ownerPrincipalId: 'demo-principal',
+    visibility: { version: 1, clauses: [] },
+    statementRestrictions,
+    expectedAuthorizationRevision,
+  };
+}
+
+function writer(
+  authorizationRevision: number,
+  policyAuthority = false,
+): AuthorizationContext {
+  return {
+    installationId: 'demo-installation',
+    principalId: 'demo-principal',
+    activeWorkspaceId: null,
+    workspaceIds: [],
+    capabilities: [
+      KNOWLEDGE_WRITE_CAPABILITY,
+      ...(policyAuthority ? [KNOWLEDGE_POLICY_CAPABILITY] : []),
+    ],
+    authorizationRevision,
+  };
+}

package/migrations/0002_audit_operations.sql

@@ -1,6 +1,7 @@
--- Structural half of the v1 -> v2 migration. initializeTaproot() must run
--- immediately afterward to SHA-256 backfill legacy revisions, create their
--- audit events, install immutability triggers, and verify version metadata.
+-- Historical 0.1 structural migration retained as package documentation.
+-- Taproot 0.2 hosts must use the exported migration/initializer APIs rather
+-- than applying this file directly; checksummed adoption and JavaScript
+-- backfills cannot be represented by this SQL file alone.
 ALTER TABLE taproot_entity_revisions ADD COLUMN attribution_json TEXT CHECK (attribution_json IS NULL OR json_valid(attribution_json));
 ALTER TABLE taproot_entity_revisions ADD COLUMN tags_json TEXT NOT NULL DEFAULT '[]' CHECK (json_valid(tags_json));
 ALTER TABLE taproot_entity_revisions ADD COLUMN event_id TEXT;

package/migrations/0003_canonical_statement_text.sql

@@ -0,0 +1,31 @@
+-- Documentation copy of canonical JSON migration 3.
+-- Hosts must use applyTaprootMigrations() so the package migration ledger is
+-- updated atomically. No fallback statement prose is generated.
+INSERT INTO taproot_assertions(assertion_key)
+WITH canonical_json(entity_json) AS (
+  SELECT entity_json FROM taproot_entities
+  UNION ALL
+  SELECT entity_json FROM taproot_entity_revisions
+)
+SELECT NULL
+FROM canonical_json AS source,
+  json_each(source.entity_json, '$.claims') AS claim,
+  json_each(claim.value) AS statement
+WHERE json_type(statement.value, '$.text') IS NOT 'text'
+   OR trim(json_extract(statement.value, '$.text')) = ''
+LIMIT 1;
+
+UPDATE taproot_metadata SET metadata_value = '2'
+WHERE metadata_key = 'canonical_json_version'
+  AND metadata_value = '1';
+
+INSERT INTO taproot_assertions(assertion_key)
+SELECT NULL WHERE NOT EXISTS (
+  SELECT 1 FROM taproot_metadata
+  WHERE metadata_key = 'canonical_json_version'
+    AND metadata_value = '2'
+);
+
+INSERT INTO taproot_migrations(version, name)
+VALUES (3, 'canonical-statement-text')
+ON CONFLICT(version) DO NOTHING;