hikoutei 0.2.0

package/docs/mikro-orm-adapter-spike.md

@@ -0,0 +1,248 @@
+# MikroORM SQLite Adapter and Entity Facade Spike
+
+## Purpose
+
+This spike tests the proposed Sheets-side persistence shape without making
+MikroORM the typed-sheets public API:
+
+```text
+application server
+├─ primary entities → application MikroORM → primary database (for example MySQL)
+└─ Sheets entities  → typed-sheets EntityManager → MikroORM SQLite executor → SQLite → Google Sheets outbox
+```
+
+The two MikroORM instances are intentionally separate. A Sheets entity uses a
+dedicated SQLite database and cannot accidentally write through the
+application's MySQL entity manager. `TypedSheetsOrm` and
+`TypedSheetsEntityManager` are ours; MikroORM is only the first replaceable
+SQLite execution engine behind them.
+
+## Public entity lifecycle
+
+The application-facing lifecycle follows the familiar entity-manager shape,
+without returning a raw MikroORM manager:
+
+```ts
+import {
+  defineTypedSheetsEntityMapping,
+} from "typed-sheets/orm";
+import {
+  initializeMappedTypedSheetsOrm,
+} from "typed-sheets/mikro-orm";
+import { provisionRegisteredSyncSheets } from "typed-sheets";
+
+const orderMapping = defineTypedSheetsEntityMapping({
+  entity: Order,
+  logicalSheetId: "orders",
+  primaryKey: "id",
+  businessKey: "orderNumber",
+  schemaVersion: 1,
+  fields: [
+    {
+      property: "orderNumber",
+      cellKind: "string",
+      ownership: "user",
+      required: true,
+      unique: true,
+    },
+    {
+      property: "status",
+      cellKind: "string",
+      ownership: "user",
+      required: true,
+    },
+    {
+      property: "approvedAt",
+      cellKind: "date",
+      ownership: "system",
+    },
+  ],
+  projections: [
+    {
+      physicalSheetId: "orders-input",
+      spreadsheetId: process.env.GOOGLE_SHEET_ID!,
+      tabName: "User_Input",
+      registeredRange: "A:B",
+      projection: "user_input",
+    },
+    {
+      physicalSheetId: "orders-system",
+      spreadsheetId: process.env.GOOGLE_SHEET_ID!,
+      tabName: "System_State",
+      registeredRange: "A:D",
+      projection: "system_state",
+    },
+  ],
+});
+
+const typedSheetsOrm = await initializeMappedTypedSheetsOrm({
+  dbName: "./typed-sheets.sqlite",
+  entities: [Order],
+  mappings: [orderMapping],
+  writer: { writerId: "orders-service" },
+  onRegisteredProjections: (definitions) =>
+    provisionRegisteredSyncSheets(appsScriptGateway, definitions),
+});
+
+const em = typedSheetsOrm.em.fork();
+
+const order = em.create(Order, { id: "o-1", status: "pending" });
+em.persist(order);
+await em.flush();
+
+const loaded = await em.findOne(Order, { id: "o-1" });
+if (loaded !== null) {
+  loaded.status = "paid";
+  await em.flush();
+}
+```
+
+The initial facade exposes `fork`, `create`, `find`, `findOne`, `persist`,
+`remove`, `flush`, `transactional`, and `clear`. Reads use the local SQLite
+state only; they do not wait for a Google Sheets API call. The initial filter
+contract is an equality filter with optional `limit` and `offset`, rather than
+the entire MikroORM query language.
+
+`flush()` collects create, update, and delete changes, then the built-in mapping
+planner writes the entity row, canonical state, row binding, business-key
+index, and ordered Sheets outbox effects in one SQLite transaction. It does
+not synchronously call Google Sheets. If planning throws, all of those local
+writes roll back together.
+
+The mapping requires exactly one `System_State` route and may additionally
+declare one `User_Input` route. `System_State` receives the complete canonical
+view plus a system tombstone marker. A direct create or a direct change to a
+user-owned field also emits a candidate-preserving `User_Input` reconcile
+effect; it does not overwrite an unresolved Sheet candidate. A direct remove
+also creates a guarded `user_input_delete` effect when a `User_Input` route
+exists. Its payload contains the complete expected input row, so the gateway
+deletes only an anchored row whose visible revision and full row hash still
+match. The worker blocks that effect rather than deleting when an unresolved
+candidate owns any user-owned field.
+
+The production-shaped sync gateway source at
+`apps-script/gateway/Code.gs` is the thin operation data plane. Deploy the
+current source before connecting a service; older deployments may not
+understand the current signed operation protocol.
+
+The low-level `flushCoordinator` seam remains available for an adapter author
+or an advanced custom planner, but ordinary entity configuration no longer
+needs it.
+
+`onRegisteredProjections` is optional. When supplied, it receives the exact
+registry routes and generated headers after the local SQLite registration has
+succeeded; the example uses it to provision the reviewed Apps Script
+allowlist. Omitting it keeps remote spreadsheet provisioning separate from
+process startup.
+
+## Package boundary
+
+The typed-sheets entity lifecycle contracts are available from
+`typed-sheets/orm`. The MikroORM-specific initializer and SQLite adapter are
+available only from `typed-sheets/mikro-orm`. The root `typed-sheets` import
+does not load MikroORM.
+
+MikroORM is an optional peer dependency. This keeps the core/storage contract
+replaceable by another persistence integration later.
+
+```ts
+import {
+  initializeMikroOrmSqliteAdapter,
+  migrateMikroOrmSqliteStorageSchema,
+} from "typed-sheets/mikro-orm";
+
+const storage = await initializeMikroOrmSqliteAdapter({
+  dbName: "./typed-sheets.sqlite",
+  entities: [Order],
+});
+
+// Explicit on purpose: startup does not mutate a production schema unless the
+// application opts in.
+await migrateMikroOrmSqliteStorageSchema(storage);
+```
+
+`migrateMikroOrmSqliteStorageSchema()` first applies MikroORM's safe,
+non-destructive entity-schema update, then applies typed-sheets' versioned sync
+schema. The sync schema keeps its existing `PRAGMA user_version` migration
+contract while both schemas share the same SQLite connection.
+
+## Proven transaction boundary
+
+`MikroOrmSqliteAdapter.transactional()` provides the active MikroORM entity
+manager and an adapter-neutral SQL executor together:
+
+```ts
+import { persistObservedRowWithSql } from "typed-sheets";
+
+await storage.transactional(async ({ entityManager, sql }) => {
+  const order = entityManager.create(Order, { id: "o-1", status: "pending" });
+  entityManager.persist(order);
+  await entityManager.flush();
+
+  // The observation receipt, event ledger, canonical state, and Sheets outbox
+  // all use this same transaction-bound SQL executor. A later failure rolls
+  // the Order row back too.
+  await persistObservedRowWithSql(sql, fence, observedRow);
+});
+```
+
+Focused tests prove these operations commit or roll back together:
+
+- MikroORM entity row
+- typed-sheets canonical state
+- pending Google Sheets effect
+- writer lease and fencing (stale writer rejection)
+- registered Sheet projection/allowlist
+
+## Current boundary
+
+The durable SQLite paths now have adapter-backed counterparts:
+
+- writer lease, registry, canonical commit, and effect-outbox operations;
+- `persistObservedRowWithSql()` / `persistObservedRowWithAdapter()` for
+  observation receipt, event ledger, canonical state, conflict, quarantine,
+  and effects;
+- `persistResolutionCommandWithSql()` /
+  `persistResolutionCommandWithAdapter()` for trusted conflict commands;
+- `runSyncEffectWorkerWithAdapter()` from `typed-sheets/mikro-orm` for Sheets
+  effect claiming, application, and recovery.
+
+The legacy synchronous `node:sqlite` APIs remain available for existing
+runtimes. The `WithSql` variants require a caller-owned transaction, while the
+`WithAdapter` variants open one through the adapter. This distinction makes it
+explicit at the call site whether a user-entity mutation must share the same
+commit boundary.
+
+The public mapping layer now derives canonical commits and projection effects
+from entity lifecycle changes. It also supplies
+`persistMappedObservedRowWithMikroOrm()` for the reverse half of the durable
+boundary: an already-normalized, already-evaluated accepted observation is
+stored and applied to the mapped MikroORM entity in one SQLite transaction.
+It uses native MikroORM writes internally so it does not trigger a second
+outbound `flush()` loop.
+
+The raw Sheet snapshot → normalized observation → pure evaluator bridge is
+still explicit. In particular, physical row-anchor discovery and projection
+row-location persistence must be supplied by the existing gateway/ingestion
+flow before an observation reaches `persistMappedObservedRowWithMikroOrm()`.
+This prevents the mapping layer from guessing row identity from a row number or
+business key.
+
+`em.remove()` creates a canonical tombstone, a `System_State` tombstone
+projection, and a guarded physical `User_Input` deletion when that route is
+configured. A gateway response loss is resolved only through the deletion
+effect's own receipt; an absent row by itself is never treated as proof that a
+manual or unrelated deletion was ours. If the gateway or local conflict ledger
+detects an active candidate, the input row is retained and the deletion effect
+is marked `blocked_candidate` for explicit conflict handling.
+
+The mapping/planner depends only on the adapter-neutral storage and entity
+engine contracts. A future SQLite execution engine can replace MikroORM
+without changing the `typedSheetsOrm.em.fork()` workflow or mapping
+declarations.
+
+## Runtime requirement
+
+The current adapter uses MikroORM 7's Node SQLite dialect, which requires Node
+22.17 or later. Node's built-in SQLite API is still marked experimental; that
+warning is expected in the focused adapter tests.

package/docs/quick-start.md

@@ -0,0 +1,102 @@
+# Hikoutei Quick Start
+
+The project is called Hikoutei and is currently published on npm as
+`hikoutei`.
+
+## Installation
+
+```sh
+npm install hikoutei @mikro-orm/core @mikro-orm/sql
+```
+
+The MikroORM packages are optional peer dependencies of the root package. They
+are required only when the MikroORM persistence adapter is used.
+
+## Entity lifecycle
+
+The public API is entity-oriented: load an entity, mutate it, and flush the
+unit of work.
+
+```ts
+import { defineEntity, p } from "@mikro-orm/sql";
+import { defineTypedSheetsEntityMapping } from "hikoutei/orm";
+import { initializeMappedTypedSheetsOrm } from "hikoutei/mikro-orm";
+
+const OrderSchema = defineEntity({
+  name: "Order",
+  tableName: "orders",
+  properties: {
+    id: p.string().primary(),
+    status: p.string(),
+  },
+});
+
+class Order extends OrderSchema.class {
+  declare id: string;
+  declare status: string;
+}
+
+OrderSchema.setClass(Order);
+
+const orderMapping = defineTypedSheetsEntityMapping({
+  entity: Order,
+  logicalSheetId: "orders",
+  primaryKey: "id",
+  businessKey: "id",
+  schemaVersion: 1,
+  fields: [
+    {
+      property: "id",
+      cellKind: "string",
+      ownership: "user",
+      required: true,
+      unique: true,
+    },
+    {
+      property: "status",
+      cellKind: "string",
+      ownership: "user",
+      required: true,
+    },
+  ],
+  projections: [
+    {
+      physicalSheetId: "orders-system",
+      spreadsheetId: process.env.GOOGLE_SHEET_ID!,
+      tabName: "Orders_System",
+      registeredRange: "A:C",
+      projection: "system_state",
+    },
+  ],
+});
+
+const typedSheetsOrm = await initializeMappedTypedSheetsOrm({
+  dbName: "./hikoutei.sqlite",
+  entities: [Order],
+  mappings: [orderMapping],
+  writer: { writerId: "orders-service" },
+});
+
+const em = typedSheetsOrm.em.fork();
+const order = em.create(Order, { id: "o-1", status: "pending" });
+em.persist(order);
+await em.flush();
+
+const loaded = await em.findOne(Order, { id: "o-1" });
+if (loaded !== null) {
+  loaded.status = "paid";
+  await em.flush();
+}
+```
+
+The current manager supports `fork()`, `create()`, `find()`, `findOne()`,
+`persist()`, `remove()`, `flush()`, `transactional()`, and `clear()`.
+Reads are served from SQLite and do not wait for a Google Sheets request.
+
+## Gateway setup
+
+Deploy [`apps-script/gateway/Code.gs`](../apps-script/gateway/Code.gs) as a
+Google Apps Script Web App. Initialize the local SQLite registry first, then
+use `provisionRegisteredSyncSheets()` through the operation-backed gateway.
+
+Keep the shared gateway secret on the server and out of browser code and Git.

package/docs/sql-layer-plan.md

@@ -0,0 +1,59 @@
+# SQL Layer Plan
+
+## Positioning
+
+The SQL layer should sit above the existing repository/core layer.
+
+It should not turn Google Sheets into a general database. The goal is a small, predictable SQL subset for MVPs, internal tools, and low-traffic admin workflows.
+
+## Initial SQL Subset
+
+Candidate first subset:
+
+```sql
+SELECT * FROM Users;
+SELECT * FROM Users WHERE id = ?;
+INSERT INTO Users (id, email, active, _version) VALUES (?, ?, ?, ?);
+UPDATE Users SET active = ? WHERE id = ?;
+```
+
+## Required Constraints
+
+- One sheet tab maps to one table.
+- Table schema still comes from typed column definitions.
+- Header drift still fails.
+- `_version` optimistic locking remains the write-safety mechanism.
+- SQL parsing should call repository operations instead of bypassing them.
+
+## Out of Scope for the First SQL Version
+
+- Joins.
+- Transactions.
+- Multi-row atomic writes.
+- Aggregations.
+- Nested queries.
+- Arbitrary expressions.
+- Cross-spreadsheet queries.
+
+## Implementation Direction
+
+The SQL layer should be a separate package or module above core:
+
+```txt
+sql query
+  -> parser
+  -> typed table registry
+  -> repository operation
+  -> adapter
+  -> Google Sheets API
+```
+
+This keeps schema drift detection, parsing, duplicate key checks, and optimistic locking in one place.
+
+## 한국어
+
+SQL layer는 현재 repository/core layer 위에 올라가는 구조가 맞습니다.
+
+목표는 Google Sheets를 범용 DB로 만드는 것이 아니라, MVP/internal tool/low-traffic admin을 위한 작고 예측 가능한 SQL subset을 제공하는 것입니다.
+
+초기 SQL은 `SELECT`, key 기반 `WHERE`, `INSERT`, 단순 `UPDATE` 정도로 제한하는 것이 좋습니다. Join, transaction, multi-row atomic write는 후순위입니다.