@warmdrift/kgauto-compiler 2.0.0-alpha.10

package/README.md

@@ -0,0 +1,240 @@
+# @warmdrift/kgauto-compiler — v2.0.0-alpha.6
+
+> Prompt compiler + central learning brain for multi-model AI apps.
+> **Swap models without rewriting prompts.**
+
+Greenfield rewrite of `@warmdrift/kgauto` v1. v1 was a behavioral patcher
+with telemetry; v2 is a real prompt compiler with a self-improving learning
+layer designed for cross-app pollination.
+
+The "compiler" name is deliberate — every optimization is a pass on a
+structured Intermediate Representation (IR), not string surgery on a
+rendered prompt. This unlocks slicing, dedupe, intent-aware tool relevance,
+target-correct lowering with cache markers, and (in v2.1) outcome-driven
+mutations.
+
+## Status
+
+- **Package:** alpha — coexists with v1 (`@warmdrift/kgauto@1.2.0`) under
+  the temporary name `@warmdrift/kgauto-compiler`. Renames to v2 final once
+  v1 is fully retired from production.
+- **Tests:** 201/201 passing
+- **Build:** clean (47KB ESM, 68KB CJS)
+- **Brain:** schema ready (see `brain/migrations/001_initial_schema.sql`);
+  awaiting dedicated Supabase provisioning.
+- **Mutation engine:** v2.1 (after enough outcome data accumulates).
+
+## Quickstart
+
+Two entry points. Use whichever fits your call path.
+
+### `call()` — kgauto owns the network round-trip (alpha.3)
+
+For plain-fetch consumers who don't already drive the wire themselves. One async call → compiled, executed, normalized, recorded.
+
+```ts
+import { call, configureBrain } from '@warmdrift/kgauto-compiler';
+
+configureBrain({ endpoint: 'https://your-app.com/api/kgauto/v2', apiKey: '...' });
+
+const result = await call({
+  appId: 'my-app',
+  intent: { name: 'search', archetype: 'ask' },
+  sections: [
+    { id: 'role', text: 'You are an assistant.', cacheable: true },
+    { id: 'task', text: userQuestion },
+  ],
+  models: ['claude-sonnet-4-6', 'gemini-2.5-flash'],   // first = primary; rest = fallback chain
+});
+
+// result.actualModel       → what served (post-fallback)
+// result.requestedModel    → what kgauto initially picked
+// result.response.text     → normalized across providers
+// result.response.tokens   → { input, output, total, cached?, cacheCreated? }
+// result.response.toolCalls → ToolCall[] in normalized shape
+// result.attempts          → retry observability
+```
+
+API keys default to `process.env.{ANTHROPIC,GOOGLE,OPENAI,DEEPSEEK}_API_KEY`. Override per-call via `apiKeys: { anthropic, google, ... }`. Reach provider-specific fields (Gemini `safetySettings`, Anthropic `tool_choice`, OpenAI `seed`) via `providerOverrides: { google: {...}, anthropic: {...} }` shallow-merged into the lowered request.
+
+### `compile()` — drive the wire yourself (existing path)
+
+For consumers who already own provider plumbing (AI SDK adapters, custom retry logic, streaming).
+
+```ts
+import { compile, configureBrain, record } from '@warmdrift/kgauto-compiler';
+
+configureBrain({ endpoint: 'https://your-app.com/api/kgauto/v2', apiKey: '...' });
+
+const result = compile({
+  appId: 'my-app',
+  intent: { name: 'search', archetype: 'ask' },
+  sections: [
+    { id: 'role',  text: 'You are an assistant.', cacheable: true },
+    { id: 'task',  text: userQuestion },
+  ],
+  models: ['claude-sonnet-4-6', 'gemini-2.5-flash'],
+});
+
+const start = Date.now();
+const response = await callProvider(result.target, result.request);
+
+await record({
+  handle:    result.handle,
+  tokensIn:  response.usage.input,
+  tokensOut: response.usage.output,
+  latencyMs: Date.now() - start,
+  success:   true,
+  oracleScore: { score: 0.85 },
+});
+```
+
+## Architecture
+
+```
+APP (any consumer)
+  ├── kg.compile(IR) ── runs LOCALLY, no network
+  │     ├─ pass: slice (drop sections not for this intent)
+  │     ├─ pass: dedupe (collapse identical sections by hash)
+  │     ├─ pass: tool_relevance (drop tools below intent threshold)
+  │     ├─ pass: compress_history (summarize old turns)
+  │     ├─ pass: score_targets (rank allowed models)
+  │     ├─ pass: apply_cliffs (executable known_failures from profile)
+  │     ├─ pass: lower (target-specific wire format + cache markers)
+  │     └─ pass: validate (fits hard constraints)
+  │
+  ├── app calls provider with the wire request
+  │
+  └── kg.record(handle, outcome) ── async POST to brain
+
+BRAIN (centralized Supabase)
+  ├── compile_outcomes (multi-tenant from day 1)
+  ├── mutations (active rules — empty in v2.0; engine in v2.1)
+  ├── apps (consumer registry)
+  └── digest_runs (weekly summary audit trail)
+```
+
+## Dialect-v1 (cross-app vocabulary)
+
+Apps tag every call with an **intent archetype** (`ask`, `hunt`, `classify`,
+`summarize`, `generate`, `extract`, `plan`, `critique`, `transform`) and the
+compiler computes a **shape signature** (context bucket × tool count × history
+depth × output mode × examples flag).
+
+The `(archetype, model, shape)` tuple is the **learning key**. Apps that
+declare the same tuple inherit each other's mutations — even apps that have
+never seen each other's data.
+
+That's how *"what works for the dashboard, should be insights for the next
+dashboard"* becomes mechanical instead of aspirational.
+
+## Profiles — executable model knowledge
+
+`profiles.ts` carries every model's capabilities, cliffs, lowering rules, and
+recovery handlers as **executable code** — not prose:
+
+```ts
+gemini-2.5-flash:
+  cliffs: [
+    { metric: 'input_tokens', threshold: 8_000, action: 'downgrade_quality_warning' },
+    { metric: 'tool_count',   threshold: 20,    action: 'drop_to_top_relevant' },
+    { metric: 'thinking_with_short_output', threshold: 1, action: 'force_thinking_budget_zero' },
+  ],
+  recovery: [
+    { signal: 'empty_response_after_tool', action: 'retry_with_params',
+      retryParams: { 'generationConfig.thinkingConfig.thinkingBudget': 0 } },
+  ],
+  lowering: {
+    cache: { strategy: 'cachedContent', minTokens: 4096, discount: 0.25 },
+    thinking: { field: 'generationConfig.thinkingConfig.thinkingBudget', default: 'auto' },
+  },
+```
+
+The 5 prod empty-responses in tt-intelligence's `gemini-2.5-flash` dashboard
+calls? v2 catches those automatically — `expectedShortOutput` constraint plus
+the `force_thinking_budget_zero` cliff guard.
+
+## Tools
+
+Tools are first-class IR fields. The compiler's tool-relevance pass drops
+tools that don't apply to the current intent before lowering — saves
+context budget on every call.
+
+```ts
+const tools: ToolDefinition[] = [
+  {
+    name: 'web_search',
+    description: 'Search the public web',
+    parameters: { type: 'object', properties: { q: { type: 'string' } } },
+    relevanceByIntent: {
+      ask: 0.9,        // primary tool for ask
+      hunt: 0.9,
+      classify: 0.0,   // never useful for classification
+      summarize: 0.0,
+      extract: 0.1,
+    },
+  },
+  // ...
+];
+```
+
+Each tool declares per-intent relevance scores 0..1. The pass keeps tools
+where `relevanceByIntent[currentIntent] >= toolRelevanceThreshold` (default
+`0.2`). Missing entries default to neutral (`0.5`) — kept by default. Set
+explicit `0.0` to hard-exclude.
+
+Tool definitions eat ~350 tokens of context per tool (L-051), so trimming
+matters: 12 declared tools, only 3 relevant → 9 × 350 = 3150 tokens
+recovered per call.
+
+The `tool-bloat` advisory (alpha.6) fires when more than 10 tools survive
+the relevance pass on a short-output archetype (`classify`, `extract`,
+`summarize`, `transform`, `critique`) — those archetypes typically use
+≤3 tools, so a kept-count >10 indicates either missing `relevanceByIntent`
+or scores set too generously.
+
+DeepSeek profiles cap tools to 1 (sequential-only). Other providers
+inherit the count from the IR after the relevance pass.
+
+## Brain provisioning
+
+1. Create a NEW Supabase project (suggested name: `kgauto-brain`)
+2. Apply `brain/migrations/001_initial_schema.sql`
+3. Insert your apps:
+   ```sql
+   insert into apps (id, display_name, api_key_hash)
+     values ('my-app', 'My App', crypt('<bearer>', gen_salt('bf')));
+   ```
+4. Configure each consumer with `configureBrain({ endpoint, apiKey })`
+
+For staging without a dedicated brain, point consumers at the same Supabase
+they already use — the schema is identical and migration to a dedicated brain
+is a `pg_dump` away.
+
+## What's next
+
+- **v2.0.x:** real-app integrations (tt-intelligence, inspire-central,
+  playbacksam, inspirato/incantato). Brain accumulates outcome data.
+- **v2.1:** mutation engine. Shadow-test → statistical gate → promote → auto-rollback.
+- **v2.2:** weekly digest reporting back to the operator.
+- **v2.x:** dialect-v2 expanded with archetypes that emerge from real usage.
+
+## Why this exists
+
+The previous version (v1) treated prompts as opaque strings and could only
+*append* behavioral patches. It also tried to learn quality from structural
+signals (token counts) — but quality is semantic, not structural.
+
+v2 treats prompts as structured IR, makes every model-specific quirk
+*executable* (cliffs, lowering, recovery), and makes oracle scoring a
+first-class contract — so the brain learns from quality data, not its proxies.
+
+The whole point: every multi-model AI app needs a compiler. Building it
+inline ships one app's value. Building it portable with a shared brain
+ships every app's value to every other app.
+
+Communicating vessels — finally accurate to the name.
+
+## License
+
+MIT. © Warmdrift.