@nac3/forge-cli 1.9.2 → 1.11.0

package/docs/doctrine/sizing.md

@@ -0,0 +1,182 @@
+# Sizing doctrine -- fixed options, measured envelopes
+
+This doctrine exists because the previous one was not enough.
+
+`concurrency-scale.md` opens with *"Every application Forge designs MUST
+be sized to its expected concurrency from day one. Scale is a
+RELEVAMIENTO input, not an afterthought bolted on when the app falls
+over."* That was already written when ADIP-KIKAN was built with its
+entire working set resident in the Node heap, at a volumetry
+(5.000 affiliates x 40 campaigns/year -> ~3,4M rows/year -> ~8 GB of
+live heap per year against a 2-4 GB V8 ceiling) that hits a wall in
+months.
+
+So the lesson is not "write more doctrine". The lesson is that **prose is
+advisory**. This doctrine describes a mechanism; the mechanism is the
+code in `src/sizing/`.
+
+## The four holes the Kikan post mortem found
+
+1. **The wrong axis.** The only volumetry Forge asked for was PEAK
+   CONCURRENCY (`src/commands/triage.ts`), and only to pick a complexity
+   tier. Kikan has 100 concurrent users -- a low number -- and died of
+   ROW COUNT. Data volume x retention was never a question.
+2. **No hardware envelope.** None. No RAM, no runtime heap ceiling, no
+   disk. The architecture was chosen without knowing the machine -- and
+   the ceiling that actually kills a Node process is V8's
+   `heap_size_limit`, a property of the machine and runtime, not of the
+   business.
+3. **A free-form choice.** Forge invented an approach and wrote a
+   rationale. With no fixed option set carrying validity ranges, there
+   was nothing to violate.
+4. **A soft gate.** The only gate touching volumetry was `GS4`, and soft
+   gates do not block.
+
+And the deepest one, which is really a fifth: **in Kikan the decision was
+never taken.** There was no moment called "choose the persistence
+architecture". The engine was written against a synchronous in-memory
+store as a prototype, durability was bolted on later, and the choice was
+made by omission in the commit that decided not to rewrite the engine. A
+gate that only fires in the design phase would have missed it.
+
+## What you must do
+
+### 1. Ask for five things, never four
+
+During relevamiento -- and before any change that makes a load-bearing
+decision about state:
+
+- **Entities and their cardinality.** "5.000 affiliates", "20.000
+  customers".
+- **Event rate per entity per year.** "40 campaigns per affiliate per
+  year".
+- **Rows per event, across EVERY table the event writes.** This is the
+  number people get wrong. In Kikan one campaign-affiliate event leaves
+  about 17 rows: a pending, three or four dispatch items, five messages,
+  as many traces, plus run events and a conversation. Not one row.
+- **Retention per data CLASS.** Business data and trace data have
+  different retention rights, and this is the distinction Kikan never
+  made. `0 days` means forever, and forever means unbounded.
+- **The hardware envelope**: `ram_mb`, `heap_cap_mb`, `disk_gb`,
+  `vcpus`. `heap_cap_mb` is the RUNTIME heap ceiling
+  (`node -e "console.log(require('v8').getHeapStatistics().heap_size_limit)"`),
+  **not** the RAM of the machine. Using RAM here is exactly what makes an
+  in-memory design look survivable.
+
+`yf triage` asks the first three (and uses the row rate to pick the tier);
+`yf clarify quantitative` asks the rest, derives, and records. In a chat
+session, ask them yourself and call `yujin.sizing.record`.
+
+Ask the owner. Do not invent the numbers. If the owner does not know,
+propose a default, say out loud that it is a default, and record it --
+the record keeps every revision, so a wrong assumption is findable
+later instead of invisible.
+
+### 2. Let the numbers be derived, not argued
+
+`yujin.sizing.show` derives, from those inputs: rows/year, rows at the
+horizon, live rows after retention, the heap a resident set would take
+and what share of the ceiling that is, and disk at the horizon. You do
+not estimate these and you do not debate them.
+
+### 3. Pick one option per axis, from the fixed catalogue
+
+Four axes. The option ids and their envelopes live in
+`src/sizing/model.ts`; `yujin.sizing.show` prints the catalogue and
+which options the current volumetry admits.
+
+**`state_residency` -- where the working set lives**
+
+| Option | Admissible when |
+|---|---|
+| `memory-only` | Zero users. A prototype nobody depends on |
+| `memory-write-through` | Every data class bounded by retention AND the live set under 25% of the heap ceiling |
+| `sql-async` | Always. The default |
+| `sql-plus-narrow-cache` | Only with a `measured_hotspot` naming the measurement that justifies it |
+
+25% and not 80%: the heap also holds every request in flight, the
+declarative catalog and V8's own slack, and mark-compact pauses grow with
+the live set long before the ceiling is reached. The system becomes
+unusable before it crashes.
+
+**`pagination` -- how a page is produced**
+
+| Option | Admissible when |
+|---|---|
+| `in-memory-slice` | Nothing unbounded AND under 100.000 rows at the horizon |
+| `sql-limit-offset` | Always |
+| `sql-keyset` | Always; the right default past a few hundred thousand rows |
+
+A slice taken after materialising the collection protects the wire, not
+the server. Kikan's own `paginacion.js` says so in a comment and does it
+anyway.
+
+**`retention` -- what bounds each class**
+
+| Option | Admissible when |
+|---|---|
+| `retention-none` | Not with a resident working set, and only while projected disk stays under 25% |
+| `retention-by-age` | At least one class declares a window |
+| `retention-by-age-and-count` | Same as above (it is a superset); the row cap is what survives a spike |
+
+A retention mechanism that ships inert (`max_days: 0`) is not retention.
+Kikan shipped one and it deleted nothing for a year.
+
+Consequence, and it is deliberate: with retention `0/0` the retention axis
+has **no admissible option at all**, so a project cannot be sized until it
+declares what bounds each data class. That is the Kikan lesson in one line.
+
+**`write_concurrency` -- what makes a multi-step mutation atomic**
+
+| Option | Admissible when |
+|---|---|
+| `implicit-single-thread` | Only with a synchronous resident store AND at most one concurrent user |
+| `explicit-transactions` | Always |
+| `unique-keys-and-upsert` | Always; the only correct fix for a check-then-act race |
+
+The moment reads become asynchronous, every `findOne`-then-`insert`
+becomes a race, and the guarantee that used to be free disappears
+without anything in the code changing shape. Kikan has 27 of those
+patterns and zero transactions.
+
+### 4. Record it, and accept the refusal
+
+`yujin.sizing.record` persists the sizing **only if every choice is
+inside its envelope**. A choice outside is refused and nothing is
+written. The refusal carries the derived numbers, which axes failed, and
+what would be admissible -- so fixing it is one more call, not a
+negotiation.
+
+The record is not a permit. `sizingGate()` re-validates it on every read,
+so a sizing that was fine at 500 affiliates stops passing when the
+declared number becomes 50.000.
+
+### 5. Re-check against reality
+
+`yujin.sizing.check` takes real counts (a `COUNT(*)` per table, an infra
+report) and re-runs the envelopes with the observed rate substituted in.
+The answer is not "the estimate was 3x off"; it is "at the rate you
+actually have, the architecture you chose is no longer admissible". Run
+it whenever real numbers exist. This is what catches a bad sizing in
+month 2 instead of month 12.
+
+### 6. The prototype trigger
+
+When a surface stops being a prototype -- a test store becomes the
+production store, an in-memory thing gets durability bolted on, a demo
+starts holding real data -- set `load_bearing_since`. From then on the
+gate applies outside any workflow.
+
+This is the Kikan case, and it is the one a design-phase gate alone
+cannot catch.
+
+## Proportionality
+
+None of this is a licence to gold-plate. A twenty-user internal tool with
+two tables sizes in one call and picks `sql-async` +
+`sql-limit-offset` + `retention-by-age` + `explicit-transactions`, which
+is the recommendation anyway. The cost of the gate for a small system is
+one verb call; the cost of not having it was ADIP-KIKAN.
+
+What the gate forbids is not ambition. It is choosing an architecture
+without knowing the numbers, and then discovering them in production.

package/package.json

@@ -1,117 +1,117 @@
-{
-  "name": "@nac3/forge-cli",
-  "version": "1.9.2",
-  "description": "Yujin Forge -- voice-first NAC-3 React development framework. CLI + chat panel + spec ingest + 10-format document reader + voice loop.",
-  "license": "SEE LICENSE IN LICENSE",
-  "author": "Pablo Kuschnirof <pablo@rpaforce.com>",
-  "homepage": "https://github.com/yujinapp/yujin-forge#readme",
-  "bugs": {
-    "url": "https://github.com/yujinapp/yujin-forge/issues"
-  },
-  "bin": {
-    "yf": "./dist/bin/yf.js"
-  },
-  "type": "module",
-  "exports": {
-    ".": "./dist/index.js"
-  },
-  "scripts": {
-    "version": "node scripts/sync_version.mjs && git add src/version.ts",
-    "build": "tsc && node scripts/copy_qa_assets.mjs",
-    "dev": "tsx watch src/bin/yf.ts",
-    "typecheck": "tsc --noEmit",
-    "typecheck:all": "pnpm typecheck && pnpm --filter @yujin/hito4-server typecheck && pnpm --filter @yujin/forge-mobile typecheck",
-    "test": "vitest run",
-    "test:watch": "vitest",
-    "lint": "tsc --noEmit",
-    "yf": "tsx src/bin/yf.ts",
-    "catalog:lint": "node scripts/catalog_lint.mjs",
-    "pairing:check": "node scripts/check_pairing_sync.mjs",
-    "brain-matrix:gate": "node scripts/brain_matrix_gate.mjs",
-    "hito4:lint": "node scripts/hito4_lint_bundle.mjs",
-    "publish-alpha-emergency": "node scripts/publish_alpha_emergency.mjs",
-    "prepublishOnly": "pnpm run build && pnpm run typecheck && pnpm run catalog:lint && pnpm run test && pnpm run brain-matrix:gate && pnpm run hito4:lint"
-  },
-  "dependencies": {
-    "@anthropic-ai/sdk": "^0.32.1",
-    "@aws-sdk/client-ecs": "^3.1053.0",
-    "@noble/ciphers": "^2.2.0",
-    "@noble/hashes": "^2.2.0",
-    "@scure/base": "^2.2.0",
-    "better-sqlite3": "^12.10.0",
-    "commander": "12.1.0",
-    "cross-spawn": "^7.0.6",
-    "dotenv": "^17.4.2",
-    "kleur": "4.1.5",
-    "mammoth": "^1.8.0",
-    "node-html-parser": "6.1.13",
-    "pdfjs-dist": "^4.10.38",
-    "prompts": "2.4.2",
-    "qrcode": "^1.5.4",
-    "typescript": "5.7.2",
-    "xlsx": "npm:@e965/xlsx@0.20.3"
-  },
-  "devDependencies": {
-    "@playwright/test": "^1.60.0",
-    "@types/better-sqlite3": "^7.6.13",
-    "@types/cross-spawn": "^6.0.6",
-    "@types/node": "22.10.2",
-    "@types/prompts": "2.4.9",
-    "@types/qrcode": "^1.5.6",
-    "acorn": "^8.16.0",
-    "pixelmatch": "^7.2.0",
-    "pngjs": "^7.0.0",
-    "tsx": "4.19.2",
-    "vitest": "2.1.8"
-  },
-  "engines": {
-    "node": ">=18.17",
-    "pnpm": ">=9"
-  },
-  "packageManager": "pnpm@11.3.0",
-  "files": [
-    "dist",
-    "templates",
-    "docs/manuals",
-    "docs/SQ.md",
-    "docs/SQ_BY_STEP.yaml",
-    "docs/doctrine",
-    "src/i18n/catalogs",
-    "yujin-design-system",
-    "yujin-design-system-artdeco",
-    "yujin-design-system-moderna",
-    "yujin-design-system-marina",
-    "README.md",
-    "LICENSE"
-  ],
-  "keywords": [
-    "nac",
-    "nac-3",
-    "react",
-    "framework",
-    "claude-code",
-    "yujin",
-    "voice",
-    "whisper",
-    "elevenlabs",
-    "tts",
-    "stt",
-    "document-reader",
-    "pdf",
-    "docx",
-    "epub",
-    "spec-ingest",
-    "scaffolding",
-    "accessibility",
-    "voice-first",
-    "adip"
-  ],
-  "repository": {
-    "type": "git",
-    "url": "git+https://github.com/yujinapp/yujin-forge.git"
-  },
-  "publishConfig": {
-    "access": "public",
-    "tag": "latest"
-  }
-}
+{
+  "name": "@nac3/forge-cli",
+  "version": "1.11.0",
+  "description": "Yujin Forge -- voice-first NAC-3 React development framework. CLI + chat panel + spec ingest + 10-format document reader + voice loop.",
+  "license": "SEE LICENSE IN LICENSE",
+  "author": "Pablo Kuschnirof <pablo@rpaforce.com>",
+  "homepage": "https://github.com/yujinapp/yujin-forge#readme",
+  "bugs": {
+    "url": "https://github.com/yujinapp/yujin-forge/issues"
+  },
+  "bin": {
+    "yf": "./dist/bin/yf.js"
+  },
+  "type": "module",
+  "exports": {
+    ".": "./dist/index.js"
+  },
+  "scripts": {
+    "version": "node scripts/sync_version.mjs && git add src/version.ts",
+    "build": "tsc && node scripts/copy_qa_assets.mjs",
+    "dev": "tsx watch src/bin/yf.ts",
+    "typecheck": "tsc --noEmit",
+    "typecheck:all": "pnpm typecheck && pnpm --filter @yujin/hito4-server typecheck && pnpm --filter @yujin/forge-mobile typecheck",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "lint": "tsc --noEmit",
+    "yf": "tsx src/bin/yf.ts",
+    "catalog:lint": "node scripts/catalog_lint.mjs",
+    "pairing:check": "node scripts/check_pairing_sync.mjs",
+    "brain-matrix:gate": "node scripts/brain_matrix_gate.mjs",
+    "hito4:lint": "node scripts/hito4_lint_bundle.mjs",
+    "publish-alpha-emergency": "node scripts/publish_alpha_emergency.mjs",
+    "prepublishOnly": "pnpm run build && pnpm run typecheck && pnpm run catalog:lint && pnpm run test && pnpm run brain-matrix:gate && pnpm run hito4:lint"
+  },
+  "dependencies": {
+    "@anthropic-ai/sdk": "^0.32.1",
+    "@aws-sdk/client-ecs": "^3.1053.0",
+    "@noble/ciphers": "^2.2.0",
+    "@noble/hashes": "^2.2.0",
+    "@scure/base": "^2.2.0",
+    "better-sqlite3": "^12.10.0",
+    "commander": "12.1.0",
+    "cross-spawn": "^7.0.6",
+    "dotenv": "^17.4.2",
+    "kleur": "4.1.5",
+    "mammoth": "^1.8.0",
+    "node-html-parser": "6.1.13",
+    "pdfjs-dist": "^4.10.38",
+    "prompts": "2.4.2",
+    "qrcode": "^1.5.4",
+    "typescript": "5.7.2",
+    "xlsx": "npm:@e965/xlsx@0.20.3"
+  },
+  "devDependencies": {
+    "@playwright/test": "^1.60.0",
+    "@types/better-sqlite3": "^7.6.13",
+    "@types/cross-spawn": "^6.0.6",
+    "@types/node": "22.10.2",
+    "@types/prompts": "2.4.9",
+    "@types/qrcode": "^1.5.6",
+    "acorn": "^8.16.0",
+    "pixelmatch": "^7.2.0",
+    "pngjs": "^7.0.0",
+    "tsx": "4.19.2",
+    "vitest": "2.1.8"
+  },
+  "engines": {
+    "node": ">=18.17",
+    "pnpm": ">=9"
+  },
+  "packageManager": "pnpm@11.3.0",
+  "files": [
+    "dist",
+    "templates",
+    "docs/manuals",
+    "docs/SQ.md",
+    "docs/SQ_BY_STEP.yaml",
+    "docs/doctrine",
+    "src/i18n/catalogs",
+    "yujin-design-system",
+    "yujin-design-system-artdeco",
+    "yujin-design-system-moderna",
+    "yujin-design-system-marina",
+    "README.md",
+    "LICENSE"
+  ],
+  "keywords": [
+    "nac",
+    "nac-3",
+    "react",
+    "framework",
+    "claude-code",
+    "yujin",
+    "voice",
+    "whisper",
+    "elevenlabs",
+    "tts",
+    "stt",
+    "document-reader",
+    "pdf",
+    "docx",
+    "epub",
+    "spec-ingest",
+    "scaffolding",
+    "accessibility",
+    "voice-first",
+    "adip"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/yujinapp/yujin-forge.git"
+  },
+  "publishConfig": {
+    "access": "public",
+    "tag": "latest"
+  }
+}

package/src/i18n/catalogs/ar.json

@@ -185,6 +185,7 @@
   "startup.tab.graph": "رسم بياني مرئي",
   "startup.tab.graphSem": "الرسم الدلالي",
   "startup.tab.pendings": "معلق",
+  "startup.tab.done": "منجز",
   "startup.tab.workflow": "سير العمل",
   "settings.startupTabs": "علامات التبويب عند الفتح",
   "graph.term.nodes": "عقد",

package/src/i18n/catalogs/de.json

@@ -185,6 +185,7 @@
   "startup.tab.graph": "Visuelles Diagramm",
   "startup.tab.graphSem": "Semantischer Graph",
   "startup.tab.pendings": "Ausstehend",
+  "startup.tab.done": "Erledigt",
   "startup.tab.workflow": "Workflow",
   "settings.startupTabs": "Tabs beim Start (Graph, ausstehend, Workflow)",
   "graph.term.nodes": "Knoten",

package/src/i18n/catalogs/en.json

@@ -185,6 +185,7 @@
   "startup.tab.graph": "Visual graph",
   "startup.tab.graphSem": "Semantic graph",
   "startup.tab.pendings": "Pendings",
+  "startup.tab.done": "Done",
   "startup.tab.workflow": "Workflow",
   "settings.startupTabs": "Tabs on open (graph, pendings, workflow)",
   "graph.term.nodes": "nodes",

package/src/i18n/catalogs/es.json

@@ -185,6 +185,7 @@
   "startup.tab.graph": "Grafo grafico",
   "startup.tab.graphSem": "Grafo semantico",
   "startup.tab.pendings": "Pendientes",
+  "startup.tab.done": "Hecho",
   "startup.tab.workflow": "Workflow",
   "settings.startupTabs": "Pestanas al abrir (grafo, pendings, workflow)",
   "graph.term.nodes": "nodos",

package/src/i18n/catalogs/fr.json

@@ -185,6 +185,7 @@
   "startup.tab.graph": "Graphe visuel",
   "startup.tab.graphSem": "Graphe semantique",
   "startup.tab.pendings": "En attente",
+  "startup.tab.done": "Termine",
   "startup.tab.workflow": "Workflow",
   "settings.startupTabs": "Onglets a l'ouverture (graphe, en attente, workflow)",
   "graph.term.nodes": "noeuds",

package/src/i18n/catalogs/hi.json

@@ -185,6 +185,7 @@
   "startup.tab.graph": "विज़ुअल ग्राफ़",
   "startup.tab.graphSem": "सिमेंटिक ग्राफ़",
   "startup.tab.pendings": "लंबित",
+  "startup.tab.done": "पूर्ण",
   "startup.tab.workflow": "वर्कफ़्लो",
   "settings.startupTabs": "खुलने पर टैब",
   "graph.term.nodes": "नोड",

package/src/i18n/catalogs/it.json

@@ -185,6 +185,7 @@
   "startup.tab.graph": "Grafo visuale",
   "startup.tab.graphSem": "Grafo semantico",
   "startup.tab.pendings": "In sospeso",
+  "startup.tab.done": "Fatto",
   "startup.tab.workflow": "Workflow",
   "settings.startupTabs": "Schede all'avvio (grafo, in sospeso, workflow)",
   "graph.term.nodes": "nodi",

package/src/i18n/catalogs/ja.json

@@ -185,6 +185,7 @@
   "startup.tab.graph": "グラフ（図）",
   "startup.tab.graphSem": "セマンティックグラフ",
   "startup.tab.pendings": "保留中",
+  "startup.tab.done": "完了",
   "startup.tab.workflow": "Workflow",
   "settings.startupTabs": "起動時のタブ（グラフ、保留、ワークフロー）",
   "graph.term.nodes": "ノード",

package/src/i18n/catalogs/pt.json

@@ -185,6 +185,7 @@
   "startup.tab.graph": "Grafo visual",
   "startup.tab.graphSem": "Grafo semantico",
   "startup.tab.pendings": "Pendencias",
+  "startup.tab.done": "Concluido",
   "startup.tab.workflow": "Workflow",
   "settings.startupTabs": "Abas ao abrir (grafo, pendencias, workflow)",
   "graph.term.nodes": "nos",

package/src/i18n/catalogs/zh.json

@@ -185,6 +185,7 @@
   "startup.tab.graph": "可视化图",
   "startup.tab.graphSem": "语义图",
   "startup.tab.pendings": "待办",
+  "startup.tab.done": "已完成",
   "startup.tab.workflow": "工作流",
   "settings.startupTabs": "启动时的标签页（图、待办、工作流）",
   "graph.term.nodes": "节点",