create-quorum-router 0.1.13 → 0.1.15

package/README.md

@@ -13,11 +13,12 @@ cd my-quorum-router-demo
 deno --version
 deno task smoke
 deno task calibration:demo
+deno task calibration:hierarchy-demo
 deno task intake
 deno task supabase:status
 ```
 
-Current package version: `create-quorum-router@0.1.13`. Releases are published
+Current package version: `create-quorum-router@0.1.15`. Releases are published
 from an immutable Git tag through GitHub Actions OIDC Trusted Publishing.
 
 ## What the generated project supports
@@ -26,13 +27,17 @@ from an immutable Git tag through GitHub Actions OIDC Trusted Publishing.
 API.
 
 `deno task calibration:demo` runs the bundled flat calibration-by-task API
-against deterministic local observations. For narrower diagnostics, generated
-projects also export `aggregateHierarchicalTaskCalibration()` and
+against deterministic local observations. `deno task calibration:hierarchy-demo`
+runs three deterministic scenarios that select at the prompt-pattern level, then
+fall back to task subtype, then task type as narrower sample buckets become
+insufficient. Generated projects also export
+`aggregateHierarchicalTaskCalibration()` and
 `resolveHierarchicalTaskCalibration()` with
 `prompt_pattern → task_subtype → task_type` sample-count fallback. Reports are
 advisory-only and are not connected to routing weights, provider eligibility, or
-execution. The command does not call provider APIs; on a new Deno installation,
-its first run resolves the pinned Zod dependency before execution.
+execution. Both demos are local-only; the hierarchy demo does not call provider
+APIs. On a new Deno installation, the first run resolves the pinned Zod
+dependency before execution.
 
 `deno task intake` is the first real setup command. It detects local provider
 wrappers, checks OAuth/session status, runs safe list-only model inventory where
@@ -43,6 +48,7 @@ the next command.
 deno task check
 deno task smoke
 deno task calibration:demo
+deno task calibration:hierarchy-demo
 deno task intake
 deno task auth:status
 deno task auth:login

package/bin/create-quorum-router.js

@@ -3,7 +3,7 @@ const fs = require("fs");
 const path = require("path");
 const { spawnSync } = require("child_process");
 
-const VERSION = "0.1.13";
+const VERSION = "0.1.15";
 const SUPPORTED_TEMPLATES = new Set(["basic"]);
 
 function usage() {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "create-quorum-router",
-  "version": "0.1.13",
+  "version": "0.1.15",
   "description": "Create a local QuorumRouter project.",
   "license": "MIT",
   "bin": {

package/templates/basic/README.md

@@ -1,7 +1,7 @@
 # QuorumRouter generated workspace
 
 This generated workspace contains the MIT-licensed QuorumRouter current release.
-npm latest targets v0.1.13.
+npm latest targets v0.1.15.
 
 QuorumRouter is **MIT**. It is **open source**. Commercial and production use
 are permitted under the MIT License.
@@ -38,6 +38,7 @@ deno --version
 ```bash
 deno task smoke
 deno task calibration:demo
+deno task calibration:hierarchy-demo
 deno task intake
 deno task auth:status
 deno task models:list
@@ -48,15 +49,18 @@ deno task supabase:status
 `smoke` proves the local scaffold runs with deterministic fixtures only. It does
 **not** call a real provider API.
 
-`calibration:demo` exercises the bundled calibration-by-task API with local
-fixture observations. Calibration reports are advisory-only: the scaffold does
-not use them to change routing weights, ranks, provider eligibility, quorum, or
-execution. The command does not call provider APIs; on a new Deno installation,
-its first run resolves the pinned Zod dependency before execution.
-
-For narrower diagnostics, import `aggregateHierarchicalTaskCalibration()` and
-`resolveHierarchicalTaskCalibration()` from `src/calibration.ts`. For example,
-classify an observation with `task_type: "code_review"`,
+`calibration:demo` exercises the bundled flat calibration-by-task API with local
+fixture observations. `calibration:hierarchy-demo` runs three deterministic
+queries that select a sufficient prompt-pattern group, fall back to task
+subtype, and fall back again to task type. Calibration reports are
+advisory-only: the scaffold does not use them to change routing weights, ranks,
+provider eligibility, quorum, or execution. Both demos are local-only; the
+hierarchy demo does not call provider APIs. On a new Deno installation, the
+first run resolves the pinned Zod dependency before execution.
+
+The hierarchical demo uses `aggregateHierarchicalTaskCalibration()` and
+`resolveHierarchicalTaskCalibration()` from `src/calibration.ts`. Its
+observations use caller-defined categories such as `task_type: "code-review"`,
 `task_subtype: "typescript"`, and `prompt_pattern: "schema-boundary-review"`.
 Resolution checks `prompt_pattern → task_subtype → task_type`, using the first
 group that reaches the configured sample threshold. Groups never cross exact

package/templates/basic/deno.json

@@ -7,6 +7,7 @@
     "check": "deno check main.ts src/*.ts",
     "smoke": "deno run main.ts",
     "calibration:demo": "deno run src/calibration_demo.ts",
+    "calibration:hierarchy-demo": "deno run src/hierarchical_calibration_demo.ts",
     "intake": "deno run --allow-read --allow-write --allow-env --allow-net --allow-run=grok,codex,claude,gemini,devin src/cli.ts intake",
     "auth:status": "deno run --allow-read --allow-env --allow-run=grok,codex,claude,gemini,devin src/cli.ts auth:status",
     "auth:login": "deno run src/cli.ts auth:login",

package/templates/basic/src/hierarchical_calibration_demo.ts

@@ -0,0 +1,132 @@
+import {
+  aggregateHierarchicalTaskCalibration,
+  resolveHierarchicalTaskCalibration,
+} from "./calibration.ts";
+
+const evaluated_at = "2026-07-14T00:00:00Z";
+const source = { provider: "OpenAI", model: "example-model" } as const;
+const observations = [
+  {
+    observation_id: "hierarchy-demo-pattern-1",
+    task_type: "code-review",
+    task_subtype: "typescript",
+    prompt_pattern: "schema-boundary-review",
+    source,
+    evaluation_basis: "caller_attested_external_ground_truth",
+    correct: true,
+    confidence: 0.9,
+    evaluated_at,
+  },
+  {
+    observation_id: "hierarchy-demo-pattern-2",
+    task_type: "code-review",
+    task_subtype: "typescript",
+    prompt_pattern: "schema-boundary-review",
+    source,
+    evaluation_basis: "caller_attested_external_ground_truth",
+    correct: true,
+    confidence: 0.8,
+    evaluated_at,
+  },
+  {
+    observation_id: "hierarchy-demo-pattern-3",
+    task_type: "code-review",
+    task_subtype: "typescript",
+    prompt_pattern: "schema-boundary-review",
+    source,
+    evaluation_basis: "caller_attested_external_ground_truth",
+    correct: false,
+    confidence: 0.7,
+    evaluated_at,
+  },
+  {
+    observation_id: "hierarchy-demo-subtype-1",
+    task_type: "code-review",
+    task_subtype: "typescript",
+    prompt_pattern: "concurrency-review",
+    source,
+    evaluation_basis: "caller_attested_external_ground_truth",
+    correct: true,
+    confidence: 0.8,
+    evaluated_at,
+  },
+  {
+    observation_id: "hierarchy-demo-parent-1",
+    task_type: "code-review",
+    task_subtype: "rust",
+    prompt_pattern: "unsafe-boundary-review",
+    source,
+    evaluation_basis: "caller_attested_external_ground_truth",
+    correct: true,
+    confidence: 0.7,
+    evaluated_at,
+  },
+  {
+    observation_id: "hierarchy-demo-foreign-1",
+    task_type: "code-review",
+    task_subtype: "typescript",
+    prompt_pattern: "schema-boundary-review",
+    source: { provider: "Anthropic", model: "example-model" },
+    evaluation_basis: "caller_attested_external_ground_truth",
+    correct: true,
+    confidence: 0.75,
+    evaluated_at,
+  },
+];
+
+const report = aggregateHierarchicalTaskCalibration(observations, {
+  minimum_sample_count: 3,
+});
+const scenarios = [
+  {
+    name: "pattern-sufficient",
+    query: {
+      task_type: "code-review",
+      task_subtype: "typescript",
+      prompt_pattern: "schema-boundary-review",
+      source,
+    },
+  },
+  {
+    name: "fallback-to-subtype",
+    query: {
+      task_type: "code-review",
+      task_subtype: "typescript",
+      prompt_pattern: "concurrency-review",
+      source,
+    },
+  },
+  {
+    name: "fallback-to-task",
+    query: {
+      task_type: "code-review",
+      task_subtype: "rust",
+      prompt_pattern: "unsafe-boundary-review",
+      source,
+    },
+  },
+].map(({ name, query }) => {
+  const selection = resolveHierarchicalTaskCalibration(report, query);
+  return {
+    name,
+    query,
+    candidates: selection.candidates,
+    resolution_status: selection.resolution_status,
+    selected_scope: selection.selected_scope,
+    selected_sample_count: selection.selected_group?.sample_count ?? null,
+  };
+});
+
+console.log(JSON.stringify(
+  {
+    schema_version: "quorum-router.hierarchical-demo.v1",
+    advisory_only: true,
+    provider_request_sent: false,
+    fallback_order: ["prompt_pattern", "task_subtype", "task_type"],
+    minimum_sample_count: report.minimum_sample_count,
+    observation_count: observations.length,
+    scenarios,
+  },
+  null,
+  2,
+));