@pikku/cli 0.12.82 → 0.12.85

package/skills/pikku-scenario/SKILL.md

@@ -0,0 +1,215 @@
+---
+name: pikku-scenario
+description: >-
+  Use when writing or running Pikku scenarios, or when asked to test Pikku functions or improve
+  test coverage. A scenario (pikkuScenario) drives the app the way users do — steps run as actors
+  over the real transport against a running server — so a flow doubles as an e2e test and a
+  staged/production health check. Covers scenario.do / expectEventually / expectError /
+  expectService, actors and environments in pikku.config.json, SCENARIO_ACTOR_SECRET, the
+  `pikku scenario list|run` commands, live function coverage via `pikku dev --coverage`, and
+  plain unit tests for pure function logic. TRIGGER when: user asks about scenarios, testing a
+  Pikku function, test coverage, end-to-end flows, or health checks. DO NOT TRIGGER when: user
+  asks about browser/UI e2e (that is @pikku/cucumber, out of scope), running an existing test
+  suite (use Bash), or CI configuration.
+installGroups: [core]
+---
+
+# Pikku Scenarios
+
+## Agent Operating Procedure
+
+Use this skill as an execution checklist, not reference material.
+
+1. Discover before editing: `pikku scenario list` for what exists, `pikku info functions --verbose` for what a scenario can call.
+2. Identify the source files that own the behavior. Do not start by reading generated output, `.pikku`, `node_modules`, or build artifacts.
+3. Make the smallest source change that satisfies the task. Keep generated files generated.
+4. Validate with the narrowest relevant command first, then `pikku all --tsc` when functions, wirings or schemas may have changed.
+5. If validation fails, fix the source cause and rerun. Do not paper over generated errors by editing generated files.
+
+**`pikku tests` does not exist.** It was removed in #865 — scenarios own coverage now. Any reference you find to it is stale.
+
+## What a scenario is
+
+A scenario is a `pikkuScenario` export that drives the app **as real actors over the real transport**, against a running server. That is what lets one artifact serve as both an e2e test and a staged/production health check.
+
+Consequences that matter, and bite if ignored:
+
+- **There is no state reset.** A scenario runs against a live server. Scope what you create (unique ids, your own rows) and never assume a clean database.
+- **Every step needs an actor.** `scenario.do(...)` without `{ actor }` throws `Scenario tried to run '<rpc>' as an internal step…`. There is no bare internal-RPC step.
+- **Actors must be configured and signed in**, or the scenario cannot run.
+
+Scenarios live in `srcDirectories` like any other function — by convention `*.scenario.ts`.
+
+## Writing one
+
+`pikkuScenario` comes from the **generated** workflow types, not `@pikku/core`:
+
+```typescript
+import { pikkuScenario } from '#pikku/workflow/pikku-workflow-types.gen.js'
+
+export const orderSupportScenario = pikkuScenario<
+  { value?: number },
+  { doubled: number; message: string }
+>({
+  title: 'Order support (scenario)',
+  tags: ['scenario'],
+  func: async ({ logger }, data, { scenario, actors }) => {
+    if (!actors?.shopper || !actors?.support) {
+      throw new Error(
+        'orderSupportScenario needs run actors (shopper + support) — run via `pikku scenario run <environment>`'
+      )
+    }
+
+    const doubled = await scenario.do(
+      'shopper doubles their order',
+      'doubleValue',
+      { value: data?.value ?? 21 },
+      { actor: actors.shopper }
+    )
+
+    const settled = await scenario.expectEventually(
+      'support sees the greeting settle',
+      'formatMessage',
+      { greeting: 'Hello', name: 'Support' },
+      (out: { message: string }) => out.message.length > 0,
+      { actor: actors.support, within: '5s', interval: 50 }
+    )
+
+    return { doubled: doubled.result, message: settled.message }
+  },
+})
+```
+
+A scenario takes the same config fields as a workflow (`title`, `description`, `tags`, `input`/`output`, `auth`, `permissions`, `middleware`, `version`, …). The third argument is the scenario context: `{ scenario, actors }`.
+
+### The scenario API
+
+| Call | Purpose |
+| --- | --- |
+| `scenario.do(step, rpc, data, { actor })` | Run an RPC as that actor. The step name is what appears in the run output. |
+| `scenario.expectEventually(step, rpc, data, predicate, { actor, within, interval })` | Poll until `predicate(out)` passes or `within` elapses. For anything asynchronous — queues, workers, eventual state. |
+| `scenario.expectError(step, rpc, data, { actor, matches })` | Assert the call **fails**. For fault injection and negative paths. |
+| `scenario.expectService(step, 'service.method', { actor, calledWith })` | Assert a stubbed service was called. Requires the server to run with `--test`. |
+
+`expectEventually` is **scenario-only**. Calling it from a `pikkuWorkflowFunc` is a critical inspector error (`PKU675`) pointing you at `pikkuScenario`.
+
+Prefer `expectEventually` over sleeping. There is no `beforeEach`/`afterEach` — a scenario is a plain async function.
+
+## Configuration
+
+Actors and environments live in `pikku.config.json`:
+
+```json
+{
+  "scenarios": {
+    "actors": {
+      "shopper": {
+        "email": "shopper@actors.local",
+        "name": "Shopper",
+        "jobTitle": "First-time buyer",
+        "personality": "Impatient shopper who abandons slow checkouts"
+      },
+      "support": { "email": "support@actors.local", "name": "Support" }
+    },
+    "environments": {
+      "local": {
+        "apiUrl": "http://localhost:4077",
+        "signInPath": "/api/auth/sign-in/actor"
+      }
+    }
+  }
+}
+```
+
+- `environments.<name>.apiUrl` is required. `signInPath` defaults to `/auth/sign-in/actor`, `rpcPath` to `/rpc`.
+- **`SCENARIO_ACTOR_SECRET` is an environment variable and never goes in `pikku.config.json`.** It signs actors in. `pikku scenario run` throws without it; a server auto-building actors warns and runs without them.
+
+## Running
+
+```bash
+pikku scenario list                       # name [tags] + description; takes no options
+SCENARIO_ACTOR_SECRET=… pikku scenario run local
+SCENARIO_ACTOR_SECRET=… pikku scenario run local --flows orderSupportScenario
+SCENARIO_ACTOR_SECRET=… pikku scenario run local --tags smoke,scenario
+```
+
+`run` takes the environment as a **required positional** — the key from `scenarios.environments`. `--flows`/`-f` filters by scenario name, `--tags`/`-t` by tag (match-any).
+
+Output is `PASS <name> (<ms>) → <output>` / `FAIL <name> (<ms>): <error>`, then `N/M scenarios passed against '<env>'`.
+
+**Exit code is 1** if any scenario fails *or* if no scenario matched the filter — a typo'd `--flows` is a hard error, not a silent zero-run pass. It throws outright on an unknown environment, an unknown flow name, or a missing `SCENARIO_ACTOR_SECRET`.
+
+## Coverage
+
+Coverage is attributed by running scenarios against a server that is collecting it. It is **not** derived from unit tests.
+
+Prerequisites in `pikku.config.json`:
+
+```json
+{ "scaffold": { "scenarios": "auth" }, "verboseMeta": true }
+```
+
+`scaffold.scenarios` generates the coverage and stub RPCs into your project (`pikkuScenarioTakeLiveCoverage`, `pikkuScenarioResetLiveCoverage`, `pikkuScenarioResetStubs`, `pikkuScenarioGetStubCalls`), so scenario runs work against any server. `verboseMeta` is required — the coverage RPC reads the verbose functions meta and returns `null` without it.
+
+```bash
+pikku dev --coverage                                   # V8 precise coverage, in-process
+pikku dev --coverage --test                            # also enable stubs (needed for expectService)
+SCENARIO_ACTOR_SECRET=… pikku scenario run local --coverage
+```
+
+The run resets coverage before each scenario and snapshots after, writing **`<outDir>/coverage/scenario-coverage.json`**:
+
+```jsonc
+{ "generatedAt": "…", "environment": "local", "scenarios": { "<name>": { /* FunctionCoverageReport */ } } }
+```
+
+Coverage is best-effort: it disables itself with a warning if the server is not collecting or the first actor cannot invoke, and it needs at least one configured actor. If you get no coverage, check those first.
+
+**There is no AI-prompt output.** The old `--ai-out` flag died with `pikku tests`; nothing replaced it. To find what needs work, read `scenario-coverage.json` yourself and cross-reference `pikku meta functions list` for input/output schemas.
+
+### Filling coverage
+
+1. `pikku scenario run <env> --coverage`, then read `<outDir>/coverage/scenario-coverage.json` to see what is unexercised.
+2. `pikku meta functions list` for those functions' schemas.
+3. Write a `pikkuScenario` that reaches them **through a real user flow** with an actor — not a scenario per function. Scenarios are flows; coverage is a consequence.
+4. Re-run to confirm.
+
+## Unit tests for pure logic
+
+Scenarios are the repo-idiomatic way to test functions, and the only thing that contributes to live coverage. For pure logic with heavy branching, a plain unit test calling `func` directly is still valid and cheap:
+
+```typescript
+import { describe, test } from 'node:test'
+import assert from 'node:assert'
+
+describe('createTodo', () => {
+  test('creates a todo', async () => {
+    const services = { todoStore: { add: async (title: string) => ({ id: '1', title }) } }
+    const result = await createTodo.func(services as any, { title: 'Buy milk' })
+    assert.equal(result.title, 'Buy milk')
+  })
+})
+```
+
+```bash
+node --import tsx --test src/**/*.test.ts
+```
+
+Services are plain objects — a Pikku function is pure business logic, so a mock is just the shape the function destructures. Build real services via the `pikkuServices` / `pikkuWireServices` factories when a test needs them.
+
+## Red flags
+
+| Smell | Why it's wrong |
+| --- | --- |
+| `pikku tests …` | Removed in #865. Use `pikku scenario`. |
+| `.feature` files / Gherkin for function tests | Scenarios are TypeScript, not Gherkin. The in-process cucumber function world was deleted. |
+| `scenario.do(...)` with no `{ actor }` | Throws. Every step runs as somebody. |
+| A scenario per function | Scenarios are user flows. One flow covers many functions; that is the point. |
+| Assuming a clean database | There is no state reset — it may be a staging server. Scope what you create. |
+| `sleep()` before asserting | Use `expectEventually`. |
+| `expectEventually` in a `pikkuWorkflowFunc` | `PKU675` — scenario-only. |
+| Coverage silently 0 | Server not run with `--coverage`, `verboseMeta` off, `scaffold.scenarios` unset, or no actors configured. |
+
+`@pikku/cucumber` is a **browser/e2e** harness (`Actor`, `BrowserWorld`, `PersonaData`, `DbUtils`) — out of scope here.
+
+See `pikku-concepts` for the core mental model.

package/skills/pikku-schedule/SKILL.md

@@ -1,8 +1,11 @@
 ---
 name: pikku-schedule
-description: 'Use when setting up in-memory cron scheduling in a Pikku app. Covers InMemorySchedulerService for running scheduled tasks.
-TRIGGER when: code uses InMemorySchedulerService, PikkuTaskScheduler, or user asks about in-memory scheduling, cron jobs without external dependencies, or @pikku/schedule.
-DO NOT TRIGGER when: user asks about cron wiring (use pikku-cron) or queue-based scheduling with BullMQ/PgBoss (use pikku-queue).'
+description: >-
+  Use when setting up in-memory cron scheduling in a Pikku app. Covers InMemorySchedulerService
+  for running scheduled tasks. TRIGGER when: code uses InMemorySchedulerService,
+  PikkuTaskScheduler, or user asks about in-memory scheduling, cron jobs without external
+  dependencies, or @pikku/schedule. DO NOT TRIGGER when: user asks about cron wiring (use
+  pikku-cron) or queue-based scheduling with BullMQ/PgBoss (use pikku-queue).
 installGroups: [core]
 ---
 

package/skills/pikku-schema-ajv/SKILL.md

@@ -1,8 +1,10 @@
 ---
 name: pikku-schema-ajv
-description: 'Use when setting up JSON schema validation with AJV in a Pikku app. Covers AjvSchemaService for request/response validation.
-TRIGGER when: code uses AjvSchemaService, user asks about AJV, JSON schema validation, or @pikku/schema-ajv.
-DO NOT TRIGGER when: user asks about Cloudflare Workers schema validation (use pikku-schema-cfworker).'
+description: >-
+  Use when setting up JSON schema validation with AJV in a Pikku app. Covers AjvSchemaService for
+  request/response validation. TRIGGER when: code uses AjvSchemaService, user asks about AJV, JSON
+  schema validation, or @pikku/schema-ajv. DO NOT TRIGGER when: user asks about Cloudflare Workers
+  schema validation (use pikku-schema-cfworker).
 installGroups: [core]
 ---
 

package/skills/pikku-schema-cfworker/SKILL.md

@@ -1,8 +1,11 @@
 ---
 name: pikku-schema-cfworker
-description: 'Use when setting up JSON schema validation for Cloudflare Workers in a Pikku app. Covers CFWorkerSchemaService as a lightweight alternative to AJV.
-TRIGGER when: code uses CFWorkerSchemaService, user asks about schema validation on Cloudflare Workers, or @pikku/schema-cfworker.
-DO NOT TRIGGER when: user asks about AJV schema validation (use pikku-schema-ajv).'
+description: >-
+  Use when setting up JSON schema validation for Cloudflare Workers in a Pikku app. Covers
+  CFWorkerSchemaService as a lightweight alternative to AJV. TRIGGER when: code uses
+  CFWorkerSchemaService, user asks about schema validation on Cloudflare Workers, or
+  @pikku/schema-cfworker. DO NOT TRIGGER when: user asks about AJV schema validation (use
+  pikku-schema-ajv).
 installGroups: [core, fabric]
 ---
 

package/skills/pikku-security/SKILL.md

@@ -1,8 +1,12 @@
 ---
 name: pikku-security
-description: 'Use when adding authentication or session management to a Pikku app — pikkuAuth, session lifecycle (setSession/clearSession), built-in auth strategies (authBearer, authCookie, authAPIKey), or JWT setup.
-TRIGGER when: user asks about login, logout, session, bearer tokens, cookie auth, API keys, or JWT.
-DO NOT TRIGGER when: user asks about middleware (use pikku-middleware), permissions/authorization checks (use pikku-permissions), or secrets/env vars (use pikku-config).'
+description: >-
+  Use when adding authentication or session management to a Pikku app — pikkuAuth, session
+  lifecycle (setSession/clearSession), built-in auth strategies (authBearer, authCookie,
+  authAPIKey), or JWT setup. TRIGGER when: user asks about login, logout, session, bearer tokens,
+  cookie auth, API keys, or JWT. DO NOT TRIGGER when: user asks about middleware (use
+  pikku-middleware), permissions/authorization checks (use pikku-permissions), or secrets/env vars
+  (use pikku-config).
 installGroups: [core]
 ---
 
@@ -12,7 +16,7 @@ installGroups: [core]
 
 1. Discover before editing. Run `pikku info middleware --verbose` and `pikku info functions --verbose` to understand existing auth setup.
 2. Auth strategies live in wirings files — do not put `addHTTPMiddleware` calls inside function bodies.
-3. Validate with `pikku tsc` after changes; run `pikku all` if wirings changed.
+3. Validate with `pikku all --tsc` after changes — it regenerates and then type-checks in one pass, and fails on type errors. Use `--tsc-summary` for a compact one-line-per-error report.
 
 For **middleware** (including tag middleware and service-to-service bearer auth) see `pikku-middleware`.
 For **permissions** (pikkuPermission, pikkuAuth, per-function authorization) see `pikku-permissions`.

package/skills/pikku-services/SKILL.md

@@ -1,8 +1,12 @@
 ---
 name: pikku-services
-description: 'Use when setting up dependency injection, creating custom services, or configuring the service layer in a Pikku app. Covers pikkuServices (singleton), pikkuWireServices (per-request), service typing, built-in services, and tree-shaking.
-TRIGGER when: code uses pikkuServices/pikkuWireServices, user asks about services.ts, dependency injection, service factories, or built-in services (ConsoleLogger, JoseJWTService).
-DO NOT TRIGGER when: user asks about auth middleware (use pikku-security) or secrets/variables (use pikku-config).'
+description: >-
+  Use when setting up dependency injection, creating custom services, or configuring the service
+  layer in a Pikku app. Covers pikkuServices (singleton), pikkuWireServices (per-request), service
+  typing, built-in services, and tree-shaking. TRIGGER when: code uses
+  pikkuServices/pikkuWireServices, user asks about services.ts, dependency injection, service
+  factories, or built-in services (ConsoleLogger, JoseJWTService). DO NOT TRIGGER when: user asks
+  about auth middleware (use pikku-security) or secrets/variables (use pikku-config).
 installGroups: [core]
 ---
 
@@ -72,7 +76,7 @@ export const createWireServices = pikkuWireServices(
 
 ### Auto-Generated Service Manifest
 
-After `npx pikku prebuild`, Pikku generates `.pikku/pikku-services.gen.ts`, a manifest of which services are actually used by wired functions:
+After `npx pikku all`, Pikku generates `.pikku/pikku-services.gen.ts`, a manifest of which services are actually used by wired functions:
 
 ```typescript
 export const requiredSingletonServices = {
@@ -114,6 +118,40 @@ const getUser = pikkuFunc({
 })
 ```
 
+### Services Are Never Optional Inside a Function
+
+**Never write a `if (!service) throw ...` existence guard in a function body.** It is dead code, and it defeats the platform.
+
+Optionality lives in exactly one place — `services.ts` / the `SingletonServices` declaration — and it means *"this may not be created"*, not *"this may be missing at call time"*. A service is optional precisely because **nothing destructures it**, and `requireSingletonServices` therefore never creates it. The moment any wired function destructures it, Pikku creates it and guarantees it is there.
+
+The types enforce this rather than merely documenting it. The inspector records the services destructured by every wired `func`, `permissions` **and** `middleware`, and emits them as `RequiredSingletonServices`. The generated function types then default their service parameter to:
+
+```typescript
+export type WiredSingletonServices = RequiredSingletonServices & SingletonServices
+export type WiredServices = RequiredSingletonServices & Services
+```
+
+so a service that is `foo?: Foo` in `SingletonServices` arrives as a non-optional `Foo` in every function, permission and middleware that uses it. There is nothing to guard against.
+
+```typescript
+// ✅ Correct — destructure and use; creation is guaranteed by the manifest
+const listThreads = pikkuFunc({
+  func: async ({ agentRunService }, { threadId }) => {
+    return await agentRunService.getThreadMessages(threadId)
+  },
+})
+
+// ❌ Wrong — unreachable guard; signals a misunderstanding of service wiring
+const listThreads = pikkuFunc({
+  func: async ({ agentRunService }, { threadId }) => {
+    if (!agentRunService) throw new MissingServiceError('agentRunService')
+    return await agentRunService.getThreadMessages(threadId)
+  },
+})
+```
+
+If a service really is conditional at runtime (e.g. an optional integration a deployment may not configure), that is a **configuration** concern: branch on config, or fail fast at startup in `services.ts` — not per-request in every function.
+
 ### Dynamic Import Optimization
 
 Use the generated manifest to conditionally import heavy dependencies — only the services actually wired get instantiated:

package/skills/pikku-software-archaeology/README.md

@@ -0,0 +1,70 @@
+# pikku-software-archaeology
+
+Reverse-engineers an existing repository into a **Product Blueprint**: the product intelligence hidden inside an implementation (domains, entities, commands, queries, events, policies, workflows, invariants, integrations, gaps), extracted as schema-validated JSON that a generator — in our case Pikku — can rebuild from.
+
+```
+Existing Repository → pikku-software-archaeology → .knowledge/ blueprint → new Pikku application
+```
+
+This is **not** a code indexer or doc generator. It extracts *intent over implementation*: `POST /api/users/:id/status` becomes the command `ActivateUser`; three scattered `if (inv.user_id !== req.user.id)` checks become one `InvoiceOwnerOnly` policy with three `enforcedAt` citations.
+
+## Design decision: the AI is the parser
+
+There is deliberately **no scanner/AST tooling** in this skill. Static extraction is brittle and per-language (the first prototype's regex scanner broke before it ran once); the analyzing model already reads every language — JS, TS, Ruby, Python, PHP, Go — follows indirection, and understands intent. Determinism lives in the **output contract** instead: fixed file names, schema-validated shapes, sorted unordered collections (sequence-bearing arrays keep their observed order), and stable concept names, all enforced by a dumb JSON validator (`scripts/validate.mjs`). The audit is expensive; that's the trade we chose.
+
+## How to run it
+
+In Claude Code, from (or pointing at) the target repo:
+
+> Use the pikku-software-archaeology skill to extract a product blueprint from /path/to/repo
+
+The agent then:
+1. **Surveys** the repo (manifests, entry points, routes, jobs, webhooks, schema, config, TODO/HACK markers) — facts only.
+2. **Excavates the test suite** — `describe`/`it` names become workflow scenarios; assertions confirm policies and upgrade confidence; rules that exist *only* in tests are captured.
+3. **Extracts** through twelve lenses (domains, entities, commands, …) per the pipeline in `SKILL.md`. Large repos fan out subagents per lens and merge.
+4. **Cross-checks and validates**:
+   ```bash
+   node .claude/skills/pikku-software-archaeology/scripts/validate.mjs <repo>/.knowledge
+   ```
+   The validator checks every file against `references/blueprint.schema.json` plus referential integrity across files (commands reference defined domains, api surfaces map to defined commands/queries, events have producers, …).
+5. Writes `blueprint.md`, the human synthesis.
+
+Output lands in `<repo>/.knowledge/` — 14 core JSON files + `blueprint.md` (see `SKILL.md` for the full listing). Repos with a frontend and/or non-HTTP consumer channels also get an **optional consumer-surface layer**: `interfaces.json` (every way the product is used — web UI, CLI, MCP server for AI agents, OpenAPI/REST, SDK, realtime, webhooks) plus `frontend.json` / `frontend-routes.json` / `frontend-components.json`. The frontend component inventory's `rebuild` field is the key output: it separates trivially-rebuildable standard components from the **custom-logic** pieces (charts, complex tables, editors) that must be carefully ported. Backend-only repos omit these and the validator does not complain.
+
+### Incremental re-analysis
+
+Concept names are the stable IDs. On re-run after code changes, re-extract only the affected lens/domain, diff against the existing `.knowledge/`, and leave unrelated entries verbatim. Unordered collections are sorted so diffs stay reviewable; sequence-bearing arrays stay in observed order.
+
+## How Pikku consumes the blueprint
+
+Full mapping table in `references/pikku-mapping.md`. Summary: entities → Kysely migrations + Zod schemas; commands/queries → `pikkuFunc`s; api surfaces → `wireHTTP`; policies → shared permission functions (collapsing duplicated legacy checks); system workflows → `wireScheduler`/`wireQueueWorker`/`pikkuWorkflowFunc`; integrations → injected services with `wireSecret`/`wireCredential`; test-derived scenarios → `pikkuUserFlow` stories / e2e tests. Humans resolve `migration.json.decisionsNeeded` before any generation starts.
+
+## How uncertainty is represented
+
+Every extracted concept carries:
+
+```json
+{ "evidence": [{ "file": "controllers/invoices.js", "lines": "52", "note": "guard: only drafts editable" }],
+  "confidence": "high" }
+```
+
+- **high** — the behavior itself is in the cited code/schema/test. Generates directly.
+- **medium** — inferred from converging signals. Generates with a review marker.
+- **low** — plausible reconstruction. Never auto-generated; surfaced for human review.
+
+Two further distinctions keep facts and guesses separate:
+- `events[].explicit: false` — the event was *reconstructed* from side-effect clusters (email + status flip), not emitted by the code.
+- Comments/docs vs code: comments describe intent, code describes behavior. Disagreements are recorded as the code's behavior plus a `gaps.json` entry.
+
+## Repo layout
+
+```
+pikku-software-archaeology/
+├── SKILL.md                          # skill definition + extraction pipeline
+├── README.md                         # this file
+├── references/
+│   ├── blueprint.schema.json         # the output contract (JSON Schema)
+│   └── pikku-mapping.md              # blueprint → Pikku primitives
+└── scripts/
+    └── validate.mjs                  # schema + cross-file validation (node, no deps)
+```

package/skills/pikku-software-archaeology/SKILL.md

@@ -0,0 +1,186 @@
+---
+name: pikku-software-archaeology
+description: 'Use when reverse-engineering an existing repository into a Product Blueprint — recovering what product an undocumented or organically-grown codebase implements so it can be rebuilt cleanly (e.g. as a Pikku app). TRIGGER when: user says "extract a blueprint", "reverse engineer this app", "what does this codebase actually do as a product", "prepare this repo for a rewrite/migration", or points at a legacy repo (any language — JS, TS, Ruby, Python, PHP, Go) and asks for its domains, workflows, business rules, or a rebuild plan. DO NOT TRIGGER for: documenting code structure, generating API docs from an already-clean codebase, or code review.'
+installGroups: [fabric]
+---
+
+# Software Archaeology
+
+## Overview
+
+Extract **intent over implementation**. A repository is a fossil record of product decisions; your job is to recover the product — domains, entities, commands, queries, events, policies, workflows, invariants — not to describe the code. The output is a `.knowledge/` directory of schema-validated JSON plus a human-readable `blueprint.md`, consumable by a generator (Pikku) to rebuild the application cleanly.
+
+**You are the parser.** Do not build or rely on regex/AST scanners — read the code with your own tools (Grep, Read, subagents). This is what makes the skill language-agnostic: an Express app, a Rails app, and a Django app all yield the same blueprint shape.
+
+**Two layers, never merged silently:**
+- **Facts** — behavior directly observed in code, schema, or tests. Cite them.
+- **Inferred intent** — the product reasoning you reconstruct. Mark it with `confidence` and say what evidence it rests on.
+
+Never present a guess as a fact. `confidence: "high"` requires file:line evidence of the behavior itself.
+
+## Output Contract
+
+Everything goes in `<repo>/.knowledge/` (or a caller-specified directory):
+
+```
+.knowledge/
+├── product.json        # purpose, actors, capabilities, terminology
+├── domains.json        # business domains (NEVER folder names)
+├── entities.json       # domain entities: attributes, relationships, states, transitions
+├── commands.json       # state-changing actions (SendInvoice, not POST /invoices/:id/send)
+├── queries.json        # read operations and views
+├── events.json         # business facts, past tense (InvoicePaid) — no technical events
+├── policies.json       # authorization, validation, state, business-constraint rules
+├── workflows.json      # user + admin + system workflows, with test-derived scenarios
+├── api.json            # every surface, each mapped to a command/query/event-ingress
+├── integrations.json   # external services: purpose, direction, replaceability
+├── architecture.json   # components, datastores, deployment constraints worth keeping
+├── invariants.json     # what must ALWAYS be true, and what enforces it today
+├── gaps.json           # TODOs, hacks, duplication, dead code, open product decisions
+├── migration.json      # current files -> future concepts, drops, decisions needed
+├── blueprint.md        # human synthesis of all of the above
+│
+│   # OPTIONAL — the consumer-surface layer. Emit these when the repo has a
+│   # frontend and/or non-HTTP consumer channels. A backend-only repo omits them
+│   # and the validator does not complain.
+├── interfaces.json         # every way the product is consumed: web-ui, cli, mcp, openapi-rest, sdk, realtime, webhooks
+├── frontend.json           # web-UI app shape: framework, router, styling, data layer, auth (e.g. TanStack Start + better-auth + Mantine)
+├── frontend-routes.json    # the page/route tree — what a user navigates to, its data + components
+└── frontend-components.json# component inventory; `rebuild` splits trivial-Mantine from custom-logic-to-port
+```
+
+The exact field shapes live in `references/blueprint.schema.json` (in this skill's directory — read it before writing any output file). After writing, ALWAYS run:
+
+```bash
+node <skill-dir>/scripts/validate.mjs <repo>/.knowledge
+```
+
+and fix every ERROR (WARNs are prompts to double-check, not necessarily wrong). Do not declare the extraction done with validation errors outstanding.
+
+## The Pipeline
+
+Work in phases. For small repos (< ~50 source files) do them inline; for larger repos, fan out subagents per phase-3 lens and merge (see "Scaling up").
+
+### Phase 1 — Survey (facts only)
+
+Build an inventory before interpreting anything:
+- Manifests (`package.json`, `Gemfile`, `pyproject.toml`, `go.mod`, `composer.json`): dependencies are integration hints; scripts are entry points.
+- Entry points: servers, route registrations, cron/scheduler setup, queue workers, CLI binaries.
+- Data layer: migrations, schema files, model classes, raw DDL (check comments too — schemas hide in comments in migration-less repos).
+- Every HTTP/GraphQL/RPC surface, webhook, scheduled job, queue consumer.
+- **All consumer channels, not just HTTP**: a frontend app (`apps/`, `frontend/`, `web/`, `client/`), a CLI (`bin/`, `wireCLI`, a `cli/` dir, an `openapi`-generated command tool), an MCP server (`wireMCP`, `@modelcontextprotocol`, a `mcp`/`tools` dir), an OpenAPI/Swagger spec (`openapi.json`, `swagger`), a published/generated SDK, realtime channels (websocket/SSE). Each is an `interfaces.json` entry.
+- Env vars and config files.
+- TODO / FIXME / HACK / XXX / deprecated markers — each is a gaps.json candidate.
+- **The test suite** — locate it now, excavate it in Phase 2.
+
+Where intent hides, per ecosystem (read these first):
+
+| Ecosystem | Highest-yield locations |
+|---|---|
+| Rails | `config/routes.rb`, model validations + callbacks + `aasm`/state machines, `app/policies` (Pundit) / `ability.rb` (CanCan), Sidekiq/ActiveJob workers, `db/schema.rb`, specs (esp. request + model specs) |
+| Express/Node | route registration files, middleware chains (auth!), inline `if` guards in handlers, SQL/ORM models, `jobs/`+crontab refs, webhook handlers |
+| Django | `urls.py`, model `Meta`/constraints/`clean()`, DRF serializers + permissions classes, celery tasks, admin.py (reveals internal workflows) |
+| Laravel | `routes/`, FormRequests (validation), Policies/Gates, Jobs + scheduler in `Kernel.php`, migrations |
+| Go | mux/router setup, middleware, struct tags, `cmd/` binaries (each is a component) |
+| Frontend (React/Vue/etc.) | router config / file-based routes (pages a user reaches), the component tree, the design-system import (`@mantine/*`, `@mui/*`, Tailwind config) to judge consistency, the data layer (react-query/tRPC/fetch wrappers) to tie UI back to backend queries, charts/tables/editors (the `custom-logic` port risk), auth wiring |
+
+### Phase 2 — Test excavation (do not skip)
+
+Tests are the closest thing to an executable product spec. For every test file:
+- `describe`/`context`/`it` names → **scenarios** (attach to the matching workflow in `workflows.json` under `scenarios[]`, with `fromTest` set).
+- User-flow/journey harnesses (`pikkuUserFlow` stories, cucumber `.feature` files, Playwright journeys) are the highest-grade scenario source — they already ARE given/when/outcome sequences; extract them verbatim.
+- Assertions → confirmations of policies and invariants (upgrade their `confidence` to `high`, add the test as evidence).
+- Fixtures/factories → entity attribute shapes and realistic example data.
+- Edge-case tests → business rules that exist **nowhere else in the code** (e.g. "replayed webhook events are idempotent" may only be stated in a test).
+- Untested-but-critical paths, or a test suite that can't run (missing helpers, broken setup) → `gaps.json`.
+
+A rule attested by both an implementation guard AND a test is your strongest possible evidence — cite both.
+
+### Phase 3 — Extraction lenses
+
+Run each lens over the surveyed material. Rules that counter the classic failure modes:
+
+**Domains** — infer from data ownership, workflows, and vocabulary; NEVER from folder names. `controllers/` is not a domain; "Billing" is. A domain owns entities and the commands that mutate them.
+
+**Entities** — domain concepts, not tables. Include: attributes (from schema + serializers + fixtures), relationships, **lifecycle states and transitions** (grep status/state columns, then find every write to them — each write site is a transition with a trigger), ownership (which actor's rows), constraints.
+
+**Commands** — every way state changes: routes, jobs, webhooks, CLI, admin consoles, DB triggers. Name them imperative `VerbNoun` in domain language: `POST /api/users/:id/status` → `ActivateUser`. For each: actor, preconditions (every `if (...) return 4xx` guard is a precondition or policy), effects, events produced. Convention: authentication/token issuance is a command (`LogInUser`, effect: "issues a session/JWT") even though it writes no rows — it changes the caller's security state.
+
+**Queries** — reads and views, `GetX`/`ListX`/`SearchX`. Record the tenancy scoping each applies (a missing `WHERE user_id=` that exists elsewhere is a gaps.json security entry).
+
+**Events** — meaningful business facts, past tense. Most legacy apps have **implicit** events: an email send, a status flip, and a counter bump inside one handler are the event's consumers — reconstruct `InvoicePaid` from them and set `explicit: false`. Exclude technical noise (ButtonClicked, FunctionCalled). Inclusion threshold for implicit events: at least one observed consumer beyond the row write itself (an email, a downstream job, a webhook out, a derived-state flip). Plain CRUD facts with no reaction (`ClientCreated` that nothing listens to) do not become events.
+
+**Policies** — authorization, validation, state rules, business constraints. Record **every** location enforcing each rule in `enforcedAt`; the same rule enforced in 3 places (or worse, 2 slightly different versions) is a gaps.json duplication entry.
+
+**Workflows** — ALL of them: user journeys, admin/support operations, and **system workflows** (cron jobs, queue consumers, webhook reactions, syncs, notification sweeps). A crontab line in a comment is a workflow. Attach Phase-2 scenarios.
+
+**API** — list every surface but map each to its concept (`mapsTo: {type: command, name: SendInvoice}`). The route is evidence; the command is the deliverable. Auth per surface (including "none" and capability-URLs like tokened public links). A webhook whose handler changes state maps to a **command** (`RecordInvoicePayment`); reserve `event-ingress` for pure relay surfaces, where `name` must be an events.json event.
+
+**Integrations** — from deps + config + calls: purpose, data exchanged, direction, importance, replacement difficulty, env vars.
+
+**Architecture** — components as they actually run (API process, worker, cron job, SPA), datastores, and deployment constraints that must survive the rewrite (hardcoded ports with upstream expectations, raw-body middleware ordering, webhook retry semantics).
+
+**Invariants** — what must always hold, and `enforcedBy`: db-constraint, code-guard, convention, or `nothing` (an unenforced invariant is a gap). Include `atRiskBecause` when enforcement is fragile (e.g. read-then-insert sequence numbering races).
+
+**Gaps** — incomplete features, TODOs, hacks (hardcoded admin emails), duplicated logic that drifted, dead code, unclear ownership, and **open product decisions** the code never resolved (a FIXME asking "should deleting a client void invoices?" is a product decision, record it in both gaps.json and migration.json `decisionsNeeded`).
+
+**Migration** — map current file clusters → future domain + concepts; list files to drop with reasons; list decisions a human must make before rebuild.
+
+**Interfaces** (`interfaces.json`, optional) — every way the product is CONSUMED, one entry per channel, not per route. A product is usually driven through several: a **web UI** (humans), a **CLI** (developers/operators), an **MCP server** (AI agents — in a Pikku app each MCP tool IS a `pikkuFunc`), an **OpenAPI/REST** surface (developers/external systems, often *generated* from the routes), a **generated SDK**, **realtime** (websocket/SSE), and **webhooks** (in/out). For each: `kind`, `audience`, `purpose`, roughly how many ops it exposes, whether it's `generated` vs hand-written, which domains it serves, and `status` (complete/partial/stub — an MCP server with two tools is `stub`). This layer answers "who can drive this, and how" — it is the map the second-opinion skill needs to explain that the app is usable by people, developers, and agents.
+
+**Frontend** (`frontend.json` + `frontend-routes.json` + `frontend-components.json`, optional) — the web UI, which needs its own treatment because frontends vary wildly (framework, router, styling, state, data, auth) and the rebuild target is opinionated: **everything in one component system (Mantine), one data layer, one auth**.
+- `frontend.json` records the stack as FACTS: framework (e.g. TanStack Start), rendering (SSR/streaming/SPA), router, styling/design system, `designSystemConsistency`, state management, data layer (e.g. pikku-react-query vs REST helpers), auth (e.g. better-auth), i18n. Name the real technologies — the second-opinion skill weighs their tradeoffs, so record them precisely (do NOT editorialize here; this file is facts).
+- `frontend-routes.json` is the page tree: each route's `purpose` in product terms, `auth`, the `dataFrom` (query/command names it reads — reuse the backend concept names so the UI ties back to the domain), the `usesComponents`, and the `userFlows` it belongs to.
+- `frontend.json.designFindings` captures **broken/inconsistent design patterns** as concrete, cited observations (not taste). Actively hunt for: *interaction inconsistency* (the same job done as a modal in one place and a drawer in another; inconsistent confirm dialogs); *theming not tokenized* (hardcoded hex colors, magic spacing/font sizes, inline styles instead of theme tokens/variables — grep for `#[0-9a-f]{3,6}`, `style={{`, raw `px` values); *cross-page inconsistency* (the same element — button, page header, card — styled differently across routes); *component duplication* (three near-identical cards/tables for one purpose); *design-system bypass* (raw HTML/CSS where a library component exists). Each finding gets an example, its impact (feels unpolished / a color change means hunting every file), and a fix (standardize on one pattern / move to tokens / extract one shared component). These are almost always cheap cleanups, and they are exactly what a non-technical owner perceives as "the app looks off" without being able to say why.
+- `frontend-components.json` is where the frontend's real migration cost lives, in the **`rebuild`** field: `mantine-standard` (maps 1:1 to a Mantine component — trivial), `mantine-composition` (built from Mantine primitives — straightforward), `custom-style` (diverges only visually — normalize to Mantine), or **`custom-logic`** (bespoke behavior — a custom chart, a virtualized/complex table, a canvas, drag-and-drop, a rich editor — that must be **ported**, not re-skinned). A `custom-logic` component MUST fill `customLogic` explaining the behavior, and should list the `dependencies` (charting/table/editor libs) that make it a real port. This split — "trivially re-Mantine-able" vs "carries logic that must survive the port" — is the single most useful thing the frontend extraction produces.
+- **Server-rendered / non-React frontends still get all three files — do NOT skip them.** The `rebuild` enum is named for the target stack, but the distinction it draws is target-agnostic: *trivial stock element* vs *composed from primitives* vs *visual divergence only* vs **bespoke behavior that must be ported**. A Rails app (Slim/ERB + ViewComponents + Hotwire/Stimulus), a Django app (templates + HTMX), or a Laravel app (Blade + Livewire) all have that same split, and it is just as load-bearing there. Use the enum values verbatim (the validator enforces them), classify by what the thing actually *does*, and record the vocabulary mismatch in `frontend.json.notes`. Concretely: treat a template partial + its behavior controller (Stimulus/Alpine/Livewire) as ONE component and classify the pair; a server-rendered app's `custom-logic` is the same list as a SPA's (maps, charts, video players, payment elements, drag-to-reorder, rich text, QR, live-updating regions), plus anything whose behavior rides on a streaming/partial-update contract (Turbo Streams, HTMX swaps) — get that mapping wrong on rebuild and pages show stale data. Omitting these files because "it isn't a React app" hides the frontend's entire migration cost, which is the one thing this file exists to expose.
+- The `designFindings` grep hints above are React-flavored; the server-rendered equivalents are inline `style=` attributes in templates, hardcoded hex in the stylesheet tree, per-locale forked templates instead of i18n, and design-system bypass = raw markup where a component/partial already exists. Same findings, different needles.
+
+### Phase 4 — Cross-check, validate, synthesize
+
+1. Cross-checks before writing: every command has an actor and at least one precondition or policy (or you explain why it is genuinely unguarded); every state transition appears in some command/workflow — if a state is only reachable by manual/DB intervention, record the transition with trigger `"none (manual/DB-only)"` AND add a gaps.json entry; every event has a producer; every api surface maps to a defined concept; every integration is used by some command/workflow.
+2. Write all 14 JSON files, sorting **unordered identity collections** by `name` (or `path`) — on first extraction too, not just re-runs. Arrays whose order carries meaning (`workflows[].steps`, and any other observed sequence) stay in their observed order: sorting them would rewrite the behavior you extracted. Run the validator: done means `0 error(s)` and exit 0, and every WARN explicitly reviewed and either fixed or justified in your summary.
+3. Write `blueprint.md`: product summary → domain map → per-domain narrative (entities/commands/events with the interesting rules) → workflows → integrations/architecture → invariants → gaps and open decisions → rebuild recommendation. Write it for the engineer who will rebuild the product and has never seen the legacy code.
+
+## Evidence & Confidence Discipline
+
+- Every concept object carries `evidence: [{file, lines, note}]` and `confidence`. (In `api.json` and `architecture.json`, `confidence` is optional — include it whenever a surface/component is inferred rather than directly observed, e.g. an SPA known only from comments.)
+- `high` — the behavior itself is in the cited code/schema/test.
+- `medium` — inferred from multiple converging signals (naming + partial code + a test name).
+- `low` — plausible reconstruction; MUST also be phrased tentatively in blueprint.md and usually deserves a `decisionsNeeded` entry.
+- Comments and docs describe *intended* behavior; code describes *actual* behavior. When they disagree, record the code's behavior as the fact and the disagreement as a gap.
+
+## Scaling up (large repos)
+
+- Fan out one subagent per lens (or per candidate domain) with: the survey notes, the schema file path, and instructions to return JSON fragments with evidence. Merge, dedupe by concept name, then run Phase 4 yourself.
+- Fix the domain cut YOURSELF after the survey and hand every lens agent the same canonical domain list — domains are the shared IDs everything cross-references.
+- Give each lens agent ownership of whole files (never two agents writing one file), and pair coupled files under one agent (commands+queries+api must share names).
+- **Budget a reconciliation pass — parallel agents WILL drift on names** (observed at real scale: 472 validator warnings from one 7-agent run). The pattern: rebuild `domains.json`'s entity/command/query/event/policy roll-up lists LAST, generated by grouping the authoritative files by their `domain` field — never hand-written before those files exist; reconcile `eventsProduced` against curated `events.json` by rename (spelling variant) / drop (CRUD noise, no consumer) / add (only with verified consumer evidence); fill empty command policies by joining `api.json`'s per-surface `auth` against `policies.json` names. The validator's warning list is the reconciliation worklist.
+- **Incremental re-runs:** keep concept names stable (they are the IDs). Re-run the affected lens only, diff against the existing `.knowledge/` files, and preserve unrelated entries verbatim. Sort every array by `name` (or `path`) so diffs are meaningful.
+
+## Red Flags — you are about to produce a worthless blueprint
+
+| Thought | Reality |
+|---|---|
+| "I'll write it up as markdown docs" | Only `blueprint.md` is prose. The 14 JSON files ARE the deliverable; a generator consumes them. |
+| "The routes are the API contract" | Routes are evidence. Lift each to a command/query or you've documented plumbing, not product. |
+| "This is obvious, no citation needed" | Uncited claims are indistinguishable from hallucinations. Evidence on everything. |
+| "The folder structure tells me the domains" | Folders are how it grew, not what it is. Derive domains from ownership + vocabulary. |
+| "Tests are just tests, skip them" | Tests are the spec. Some rules exist ONLY in tests. Phase 2 is mandatory. |
+| "No events are emitted, so events: []" | Reconstruct implicit events from side-effect clusters; mark `explicit: false`. |
+| "Cron jobs aren't workflows" | System workflows are workflows. Include schedules, queue consumers, webhook reactions. |
+| "The frontend is just the web routes" | The web UI is ONE interface. Inventory the CLI, MCP server, OpenAPI/SDK, realtime, webhooks in `interfaces.json` too — the product is driven by people, developers, AND agents. |
+| "A component list is enough" | Without the `rebuild` split, you've hidden the frontend's real cost. Flag every `custom-logic` component (chart/table/canvas/editor) and say what the logic is — that's the port work. |
+| "I'll skip the validator, the JSON looks right" | Run it. Missing domains refs, dangling event names, and undescribed custom-logic components are exactly what it catches. |
+
+## Quick Reference
+
+```bash
+# 1. survey + excavate + extract (you, with Read/Grep/subagents)
+# 2. write <repo>/.knowledge/*.json + blueprint.md per references/blueprint.schema.json
+# 3. validate:
+node <skill-dir>/scripts/validate.mjs <repo>/.knowledge
+```
+
+- Schema/contract: `references/blueprint.schema.json`
+- How Pikku consumes the blueprint: `references/pikku-mapping.md`