@tangle-network/starter-foundry 0.5.3 → 0.7.0

package/README.md

@@ -22,10 +22,10 @@ The hot path is fully deterministic — no LLM calls, no network. A keyword scor
 
 | | Count | Examples |
 |---|---|---|
-| **Families** | 40 | nextjs-ts, react-vite-ts, agent-service-ts, forge-contracts, solana-program, python-api, go-api, sveltekit-ts |
-| **Capability layers** | 91 | See categories below |
-| **Slot layers** | 18 | database (sqlite/postgres/mongodb/convex), auth (clerk/better-auth/supabase), payments, sdk, queue |
-| **Partners** | 6 | Coinbase, Tangle, EigenLayer, Arbitrum, X Layer, Solana |
+| **Families** | 94 | nextjs-ts, react-vite-ts, agent-service-ts, forge-contracts, solana-native-rust, python-http, go-net-http, sveltekit-ts, wasm-rust, bun-http, tangle-blueprint, kotlin-multiplatform |
+| **Capability layers** | 104 | See categories below |
+| **Slot layers** | 28 | database (sqlite/postgres/mongodb/convex), auth (clerk/better-auth/supabase), payments (stripe/coinbase-commerce), sdk (evm-wallet/solana-web3/coinbase-cdp), queue (bullmq/trigger-dev), industry (10 verticals) |
+| **Partners** | 19 | Arbitrum, Avalanche, Chainlink, Coinbase, EigenLayer, Farcaster, Hyperliquid, Lens, Linea, Monad, Polygon, Sei-EVM, Solana, Sui, Tangle, Tempo, Tether, USDC-Circle, XLayer |
 | **Product archetypes** | 115+ | "Twitter clone" → fullstack-ts + realtime-ws + saas-teams |
 
 ### Capability layers by category
@@ -87,6 +87,22 @@ npm run build && node dist/cli.js <command>
 | `prove --corpus <path> --out <dir>` | Run proof suite over a prompt corpus |
 | `catalog` | List all families and layers |
 
+## Toolchain (for `pnpm test`)
+
+A handful of integration tests validate composed scaffolds end-to-end and
+need real binaries on `PATH`. Without them the tests fail with
+`spawn <bin> ENOENT`. `pnpm pretest` probes the env and prints install
+hints; the actual tests are strict (no auto-skip).
+
+| Tool | Needed for | Install |
+|---|---|---|
+| `forge` | `forge-contracts` family validation, `xlayer-foundry-deploy` / `layerzero-oft` / `account-abstraction` specs, `runPromptCorpus` multichain scenario, workspace benchmark | `curl -L https://foundry.paradigm.xyz \| bash && ~/.foundry/bin/foundryup` |
+| `cargo` | `solana-program` toolchain validation in workspace tests | `curl --proto "=https" --tlsv1.2 -sSf https://sh.rustup.rs \| sh` |
+
+After installing forge, ensure `~/.foundry/bin` is on your `PATH`
+(`foundryup` adds it to `~/.zshenv`/`~/.bashrc`; restart shell or
+`source` the rc file).
+
 ## Routing
 
 Three layers, evaluated in order:
@@ -104,9 +120,16 @@ After family selection, **capability detection** scans the prompt against capabi
 | Route accuracy (training corpus, 60 scenarios) | 100% |
 | Route accuracy (held-out corpus, 43 scenarios) | 100% |
 | Route accuracy (IdeasAI corpus, 60 scenarios) | 100% |
-| Unit + integration tests | 343/343 |
+| Unit + integration tests | 615/615 |
+| Scaffold audit pass rate | 85/89 (4 toolchain-blocked) |
 | Compose latency (warm) | ~5ms |
 
+### Template regeneration pipeline
+
+The `src/training/template_v1/` pipeline rewrites scaffold template files from the convergent-patterns miner, guarded by a four-stage audit: **compose → install → typecheck → correctness**. The correctness stage catches classes of bug the TypeScript compiler can't: unused imports in the candidate file (since no family tsconfig enables `noUnusedLocals`), missing `<script src=...>` targets, and `document.getElementById('X')` calls with no matching HTML id. Any stage failing rejects the candidate before it can land.
+
+Run: `pnpm tsx src/training/template_v1/run.ts --template-key <file> --family <id> --template-target <path> --source-path <repo path> [--apply --yes]`
+
 ## Install
 
 ```bash
@@ -138,3 +161,52 @@ if (plan.kind === 'starter') {
   const result = await composeStarter({ spec: plan.spec, outDir })
 }
 ```
+
+**Integrating into a downstream agent runtime?** See **[docs/INTEGRATION.md](./docs/INTEGRATION.md)** — programmatic API, multi-family workspaces, the 4 telemetry events to emit, scrubbing rules, and how to consume the pipeline's rankings.
+
+## Buildout pipeline (agent-behavior → registry signal)
+
+The `scripts/*-buildout*.mjs` pipeline mines Claude Code session transcripts of real agent buildouts on top of starter-foundry scaffolds and produces two ranked reports: **missing capabilities** (packages agents install because our router didn't attach the right layer) and **bad templates** (files agents rewrite within the first few turns).
+
+### Run it
+
+```bash
+pnpm build
+node scripts/run-buildout-pipeline.mjs   # mine → join → analyze
+```
+
+Outputs:
+- `.evolve/buildout-analysis.json` — per-scenario pass rate, top-added packages, top-rewritten files (committed evidence)
+- `.evolve/capability-gaps.json` — ranked (scenario, capability) router misses
+- `.evolve/traces/buildouts.jsonl` — append-only corpus (gitignored, regenerable)
+
+### Stages
+
+| Script | Input | Output |
+|---|---|---|
+| `mine-buildout-sessions.mjs` | `~/.claude/projects/**/factory-local-phase2-*/*.jsonl` | `.evolve/traces/buildouts.jsonl` |
+| `join-buildout-outcomes.mjs` | VB execution traces + buildouts | buildouts.jsonl annotated with outcomes |
+| `analyze-buildouts.mjs` | joined buildouts | `.evolve/buildout-analysis.json` |
+| `infer-capability-gaps.mjs` | joined buildouts + `registry/package-to-capability.json` | `.evolve/capability-gaps.json` |
+
+### Fault tolerance
+
+- **Resumable**: per-session mtime stored in `.evolve/traces/.buildouts-miner-state.json`. Unchanged sessions skip; next run is ~0ms.
+- **Concurrent-safe**: O_EXCL lock file prevents simultaneous miner entry. Stale locks (PID not alive) auto-steal.
+- **Corruption-tolerant**: a malformed state file is logged and reseeded instead of crashing.
+- **Schema-versioned**: bumping `BUILDOUT_SCHEMA_VERSION` auto-rebuilds from source.
+- **Append-only output**: JSONL with per-row schema version; malformed lines are skipped by every consumer.
+
+### Adding a new source (GLM, GPT, etc.)
+
+Write a new miner script (e.g., `mine-glm-sessions.mjs`) that emits rows conforming to `BuildoutEvent` in `src/lib/buildout-traces.ts` — same append-only JSONL, same schema version, different `sourceModel` value. The join + analyze stages consume it unchanged.
+
+### Extending `registry/package-to-capability.json`
+
+When an agent installs package X, we want our router to pre-attach the capability X maps to. Add an entry:
+
+```json
+"package-name": { "capability": "capability:foo", "confidence": 0.9 }
+```
+
+`tests/package-to-capability.test.ts` validates every capability ID against the live registry at build time.

package/corpus/held-out-validation.json

@@ -223,7 +223,7 @@
       "complexity": "simple",
       "partner": null,
       "prompt": "Write an Aptos Move module for a token vesting contract.",
-      "expected": { "kind": "starter", "family": "move-contracts" }
+      "expected": { "kind": "starter", "family": "aptos-move" }
     },
     {
       "id": "ho-workspace-saas-agent",

package/corpus/ideasai-prompts.json

@@ -7,6 +7,7 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-chat",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "ai-saas",
@@ -20,6 +21,7 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-chat",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "ai-saas",
@@ -35,6 +37,7 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-dashboard",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "ai-saas",
@@ -47,7 +50,9 @@
     "expectedFamily": "nextjs-ts",
     "expectedCapabilities": [
       "capability:ai-chat-ui",
-      "capability:layout-dashboard"
+      "capability:layout-dashboard",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "ai-saas",
     "actualLayers": []
@@ -73,6 +78,7 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-chat",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "ai-saas",
@@ -86,6 +92,7 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-dashboard",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "ai-saas",
@@ -97,9 +104,10 @@
     "votes": 142,
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
-      "capability:layout-dashboard",
       "capability:chart-widget",
-      "capability:shadcn"
+      "capability:layout-dashboard",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "ai-saas",
     "actualLayers": []
@@ -126,6 +134,7 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-chat",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "ai-saas",
@@ -141,6 +150,7 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-dashboard",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "ai-saas",
@@ -154,7 +164,8 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-dashboard",
-      "capability:shadcn"
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "ai-saas",
     "actualLayers": []
@@ -166,8 +177,10 @@
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
       "capability:ai-chat-ui",
+      "capability:chart-widget",
       "capability:layout-dashboard",
-      "capability:chart-widget"
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "ai-saas",
     "actualLayers": []
@@ -179,8 +192,10 @@
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
       "capability:ai-chat-ui",
+      "capability:chart-widget",
       "capability:layout-dashboard",
-      "capability:chart-widget"
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "ai-saas",
     "actualLayers": []
@@ -192,9 +207,10 @@
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
       "capability:ai-chat-ui",
-      "capability:layout-dashboard",
       "capability:chart-widget",
-      "capability:shadcn"
+      "capability:layout-dashboard",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "ai-saas",
     "actualLayers": []
@@ -207,7 +223,8 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-dashboard",
-      "capability:shadcn"
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "ai-saas",
     "actualLayers": []
@@ -220,6 +237,7 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-dashboard",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "ai-saas",
@@ -233,6 +251,7 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-chat",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "ai-saas",
@@ -246,6 +265,7 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-chat",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "ai-saas",
@@ -259,6 +279,7 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-dashboard",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "ai-saas",
@@ -271,7 +292,9 @@
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
       "capability:ai-chat-ui",
-      "capability:layout-dashboard"
+      "capability:layout-dashboard",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "ai-saas",
     "actualLayers": []
@@ -284,7 +307,8 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-dashboard",
-      "capability:shadcn"
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "ai-saas",
     "actualLayers": []
@@ -297,7 +321,8 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-dashboard",
-      "capability:shadcn"
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "ai-saas",
     "actualLayers": []
@@ -323,6 +348,7 @@
     "expectedCapabilities": [
       "capability:ai-chat-ui",
       "capability:layout-chat",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "ai-saas",
@@ -334,9 +360,10 @@
     "votes": 178,
     "expectedFamily": "nextjs-ts",
     "expectedCapabilities": [
-      "capability:marketplace",
       "capability:layout-dashboard",
-      "capability:shadcn"
+      "capability:marketplace",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "marketplace",
     "actualLayers": [
@@ -349,8 +376,9 @@
     "votes": 234,
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
-      "capability:marketplace",
       "capability:layout-dashboard",
+      "capability:marketplace",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "marketplace",
@@ -362,9 +390,10 @@
     "votes": 312,
     "expectedFamily": "nextjs-ts",
     "expectedCapabilities": [
-      "capability:marketplace",
       "capability:layout-dashboard",
-      "capability:shadcn"
+      "capability:marketplace",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "marketplace",
     "actualLayers": [
@@ -377,9 +406,10 @@
     "votes": 189,
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
-      "capability:marketplace",
       "capability:layout-dashboard",
-      "capability:shadcn"
+      "capability:marketplace",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "marketplace",
     "actualLayers": [
@@ -392,8 +422,9 @@
     "votes": 267,
     "expectedFamily": "nextjs-ts",
     "expectedCapabilities": [
-      "capability:marketplace",
       "capability:layout-dashboard",
+      "capability:marketplace",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "marketplace",
@@ -407,9 +438,10 @@
     "votes": 345,
     "expectedFamily": "nextjs-ts",
     "expectedCapabilities": [
-      "capability:marketplace",
       "capability:layout-dashboard",
-      "capability:shadcn"
+      "capability:marketplace",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "marketplace",
     "actualLayers": [
@@ -422,8 +454,9 @@
     "votes": 198,
     "expectedFamily": "nextjs-ts",
     "expectedCapabilities": [
-      "capability:marketplace",
       "capability:layout-dashboard",
+      "capability:marketplace",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "marketplace",
@@ -437,8 +470,9 @@
     "votes": 223,
     "expectedFamily": "nextjs-ts",
     "expectedCapabilities": [
-      "capability:marketplace",
       "capability:layout-dashboard",
+      "capability:marketplace",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "marketplace",
@@ -452,9 +486,10 @@
     "votes": 289,
     "expectedFamily": "nextjs-ts",
     "expectedCapabilities": [
-      "capability:marketplace",
       "capability:layout-dashboard",
-      "capability:shadcn"
+      "capability:marketplace",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "marketplace",
     "actualLayers": [
@@ -467,8 +502,9 @@
     "votes": 356,
     "expectedFamily": "nextjs-ts",
     "expectedCapabilities": [
-      "capability:marketplace",
       "capability:layout-dashboard",
+      "capability:marketplace",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "marketplace",
@@ -482,9 +518,10 @@
     "votes": 156,
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
-      "capability:layout-dashboard",
       "capability:layout-admin",
-      "capability:shadcn"
+      "capability:layout-dashboard",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "saas-tools",
     "actualLayers": [
@@ -497,9 +534,10 @@
     "votes": 234,
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
-      "capability:layout-dashboard",
       "capability:chart-widget",
-      "capability:shadcn"
+      "capability:layout-dashboard",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "saas-tools",
     "actualLayers": [
@@ -525,9 +563,10 @@
     "votes": 167,
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
-      "capability:layout-dashboard",
       "capability:chart-widget",
-      "capability:shadcn"
+      "capability:layout-dashboard",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "saas-tools",
     "actualLayers": [
@@ -540,9 +579,10 @@
     "votes": 145,
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
-      "capability:layout-dashboard",
       "capability:layout-auth",
-      "capability:shadcn"
+      "capability:layout-dashboard",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "saas-tools",
     "actualLayers": []
@@ -553,9 +593,10 @@
     "votes": 178,
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
-      "capability:layout-dashboard",
       "capability:chart-widget",
-      "capability:shadcn"
+      "capability:layout-dashboard",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "saas-tools",
     "actualLayers": [
@@ -569,9 +610,10 @@
     "votes": 201,
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
-      "capability:layout-dashboard",
       "capability:layout-admin",
-      "capability:shadcn"
+      "capability:layout-dashboard",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "saas-tools",
     "actualLayers": []
@@ -610,9 +652,10 @@
     "votes": 334,
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
-      "capability:layout-dashboard",
       "capability:chart-widget",
-      "capability:shadcn"
+      "capability:layout-dashboard",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "saas-tools",
     "actualLayers": []
@@ -623,9 +666,10 @@
     "votes": 245,
     "expectedFamily": "react-vite-ts",
     "expectedCapabilities": [
-      "capability:webrtc",
       "capability:layout-dashboard",
-      "capability:tailwind"
+      "capability:shadcn",
+      "capability:tailwind",
+      "capability:webrtc"
     ],
     "category": "webrtc",
     "actualLayers": [
@@ -638,9 +682,11 @@
     "votes": 389,
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
-      "capability:webrtc",
+      "capability:layout-auth",
       "capability:layout-dashboard",
-      "capability:layout-auth"
+      "capability:shadcn",
+      "capability:tailwind",
+      "capability:webrtc"
     ],
     "category": "webrtc",
     "actualLayers": []
@@ -651,9 +697,10 @@
     "votes": 312,
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
-      "capability:webrtc",
       "capability:layout-dashboard",
-      "capability:tailwind"
+      "capability:shadcn",
+      "capability:tailwind",
+      "capability:webrtc"
     ],
     "category": "webrtc",
     "actualLayers": [
@@ -666,9 +713,10 @@
     "votes": 198,
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
-      "capability:webrtc",
       "capability:layout-dashboard",
-      "capability:tailwind"
+      "capability:shadcn",
+      "capability:tailwind",
+      "capability:webrtc"
     ],
     "category": "webrtc",
     "actualLayers": []
@@ -679,9 +727,10 @@
     "votes": 223,
     "expectedFamily": "nextjs-ts",
     "expectedCapabilities": [
-      "capability:webrtc",
       "capability:layout-dashboard",
-      "capability:shadcn"
+      "capability:shadcn",
+      "capability:tailwind",
+      "capability:webrtc"
     ],
     "category": "webrtc",
     "actualLayers": []
@@ -693,8 +742,10 @@
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
       "capability:agent-intel",
+      "capability:chart-widget",
       "capability:layout-dashboard",
-      "capability:chart-widget"
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "data-intel",
     "actualLayers": [
@@ -710,8 +761,10 @@
     "expectedFamily": "nextjs-ts",
     "expectedCapabilities": [
       "capability:agent-intel",
+      "capability:chart-widget",
       "capability:layout-dashboard",
-      "capability:chart-widget"
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "data-intel",
     "actualLayers": [
@@ -738,8 +791,10 @@
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
       "capability:agent-intel",
+      "capability:chart-widget",
       "capability:layout-dashboard",
-      "capability:chart-widget"
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "data-intel",
     "actualLayers": []
@@ -751,7 +806,9 @@
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
       "capability:agent-intel",
-      "capability:layout-dashboard"
+      "capability:layout-dashboard",
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "data-intel",
     "actualLayers": [
@@ -764,8 +821,9 @@
     "votes": 189,
     "expectedFamily": "react-vite-ts",
     "expectedCapabilities": [
-      "capability:layout-dashboard",
       "capability:chart-widget",
+      "capability:layout-dashboard",
+      "capability:shadcn",
       "capability:tailwind"
     ],
     "category": "crypto",
@@ -789,7 +847,8 @@
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
       "capability:layout-dashboard",
-      "capability:shadcn"
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "crypto",
     "actualLayers": []
@@ -800,8 +859,10 @@
     "votes": 278,
     "expectedFamily": "fullstack-ts",
     "expectedCapabilities": [
+      "capability:chart-widget",
       "capability:layout-dashboard",
-      "capability:chart-widget"
+      "capability:shadcn",
+      "capability:tailwind"
     ],
     "category": "crypto",
     "actualLayers": []