@evoclock/pi-agentic-driver 0.9.2 → 0.9.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -53,6 +53,8 @@ proofs for agentic workflows.
53
53
  | `agentic_kanban_board_update` | Moves, closes, flags, edits, or removes cards on the board, always recording who authorized the change. | shipped |
54
54
  | `agentic_kanban_board_dispatch` | Claims an eligible board card for automated contained work and creates its assignment envelope. | shipped |
55
55
  | `agentic_kanban_pulse` | Checks for ready work and available capacity, then starts assignments under the board policy. | shipped |
56
+ | Router | Capacity/policy routing for Board Pulse: seats with scope statements, tunable policy values, and a binding dispatch gate. | shipped |
57
+ | Janus (bundled service) | Local Jev evaluation service on loopback: semantic ranking, the dispatch-gate judgment, redaction before every external call. See `janus/README.md`. | shipped |
56
58
 
57
59
  **Status: active development and testing.** Each extension ships only after
58
60
  it passes fixture-based acceptance, native tests, live-session checks, and
@@ -102,6 +104,82 @@ only, not a runtime dependency).
102
104
  - **inventory refresh.** Git-aware codebase inventory regeneration with
103
105
  verification receipts, so prior-art matching stays honest.
104
106
 
107
+ <details>
108
+ <summary><strong>router, capacity/policy routing with a binding dispatch gate</strong> <em>(released, 0.9.2)</em></summary>
109
+
110
+ Board Pulse selects a route seat through the capacity/policy router. The
111
+ router reads a closed configuration file and a local-first SQLite file for
112
+ operational state (reservations, observations, route-decision audit).
113
+
114
+ Two router features call **Janus** — a separate local service, bundled under
115
+ `janus/`, that is the only component allowed to talk to the TypeSafe Jev
116
+ models:
117
+
118
+ - **Semantic ranking** (optional): Janus ranks up to eight eligible seats;
119
+ on any Janus failure the router falls back to preference order.
120
+ - **The binding dispatch gate**: before a task is claimed, Janus judges
121
+ whether the task fits the selected seat. No verdict, no dispatch — the
122
+ gate fails closed when Janus is unreachable.
123
+
124
+ Janus binds loopback only (`127.0.0.1:8787`), reads its API key from the
125
+ macOS Keychain at runtime, redacts state before every external call, and
126
+ coalesces identical requests. See `janus/README.md` for the endpoint
127
+ contract, Keychain provisioning, and configuration.
128
+
129
+ - **Repository defaults** ship at `.agentic-driver/router.defaults.json`. The
130
+ template contains no accounts and one disabled placeholder seat, so it loads
131
+ out of the box and schedules nothing until you enable a real seat.
132
+ - **User profile (precedence over defaults):**
133
+ `~/.config/agentic-driver/router/profile.json`, or the path in the
134
+ `AGENTIC_DRIVER_ROUTER_PROFILE` environment variable. A profile section
135
+ replaces the matching defaults section wholesale; the merged config must
136
+ validate or the router refuses to produce decisions (fails closed).
137
+ - **Operational state:** `~/.local/share/agentic-driver/router/state.db`, or
138
+ `AGENTIC_DRIVER_ROUTER_DB`. This store is operational state only. The
139
+ authenticated claims file and the board writer remain the only dispatch
140
+ authority; the system repairs SQLite from them, never the reverse.
141
+ - **Runtime requirement:** the operational store uses the built-in
142
+ `node:sqlite` module. You need Node.js 22.5 or newer.
143
+
144
+ <details>
145
+ <summary><strong>Tunable configuration values</strong> <em>(all keys required, defaults are conservative starting values)</em></summary>
146
+
147
+ Every numeric policy in the router is a configuration key — never a
148
+ constant in code. All keys are **required**. The repository defaults below
149
+ are conservative starting values. Your profile overrides them. A missing key
150
+ is a configuration validation failure: the router fails closed rather than
151
+ guess. Changing a value participates in the configuration digest, so cached
152
+ routing decisions computed under the old values no longer apply.
153
+
154
+ | Section | Key | Default | What it controls |
155
+ |---|---|---|---|
156
+ | `eligibility.reserve` | `floorPercent` | 40 | Share of every quota window kept in reserve (per account, per window). Headroom for an unexpected burst and your next interactive session, without idling the account. |
157
+ | `eligibility.reserve` | `scope` | `account-window` | How widely the reserve floor applies (per account per window at launch). |
158
+ | `eligibility.reserve` | `coldStartFraction` | 0.25 | Consumption assumed for a seat with too little history: deliberately generous, so a mis-estimate fails eligibility instead of draining the account. |
159
+ | `eligibility.reserve` | `estimateSamples` | 20 | How many recent dispatches the consumption estimate averages: enough to smooth outliers, quick enough to track a model change. |
160
+ | `eligibility.reserve` | `estimateMinSamples` | 5 | Below this many observations, the estimator treats the average as noise and uses the cold-start value instead. |
161
+ | `eligibility.reserve` | `estimateOutlierSigma` | 3 | Outlier cut for the estimate: one unusually slow dispatch cannot inflate the average, real tail latency is kept. |
162
+ | `eligibility.reserve` | `ownerInteractiveOverride` | false | Whether your own interactive dispatches bypass the reserve rule (they do not, at launch). |
163
+ | `ranking` | `janusUrl` | `http://127.0.0.1:8787` | The local Janus evaluation service endpoint used for semantic ranking. |
164
+ | `ranking` | `maxCandidates` | 8 | The router ranks at most eight eligible seats per request; preference order trims the list first. |
165
+ | `ranking` | `fallback` | `preference-order` | What happens when ranking is unavailable: dispatch falls back to your preference order and never blocks. |
166
+ | `gate` | `threshold` | 0.9 | The dispatch gate's confidence threshold. A task is only dispatched when the evaluation clears it. Lower it only with evidence from your own audit log. |
167
+ | `gate` | `timeoutMs` | 8000 | How long the gate waits for an evaluation. On timeout there is no verdict, and the gate blocks dispatch (fail closed). |
168
+ | `gate` | `requestBudgetTokens` | 30000 | Maximum size of a gate evaluation request. Normal evaluations use under a thousand tokens. |
169
+ | seat record | `maxConcurrency` | 4 | How many workers one seat may run at once. The default suits a typical single machine or hosted seat. Tune it per seat: a cluster takes more, a constrained subscription endpoint may take less. |
170
+ | automation policy | `maxConcurrent` | 4 | How many workers may run at once across all seats. The default is a conservative start. Tune it to the capacity you actually have, and raise it when you add seats or hardware. |
171
+
172
+ Seats are named for infrastructure, not models: `dgx-spark`,
173
+ `dgx-spark-cluster`, `mac-studio`, `mac-studio-cluster`, `strix-halo`,
174
+ `strix-halo-cluster`, `merge-gateway-worker`. The model is a mutable field on
175
+ the seat, so changing models never renumbers your configuration. Each seat
176
+ also carries `scopeStatements`: plain-language descriptions of what the seat
177
+ is for. The dispatch gate reads them when judging whether a task fits.
178
+
179
+ </details>
180
+
181
+ </details>
182
+
105
183
  ## Multi-agent communication
106
184
 
107
185
  *Extensions for controlled coordination between agents.*
@@ -580,60 +658,6 @@ The package includes the `herdr-communication.ts`, `herdr-lifecycle.ts`, and
580
658
  disable individual resources from an installed package. You are not required
581
659
  to use every extension.
582
660
 
583
- ## Router configuration
584
-
585
- Board Pulse selects a route seat through the capacity/policy router. The
586
- router reads a closed configuration file and a local-first SQLite file for
587
- operational state (reservations, observations, route-decision audit).
588
-
589
- - **Repository defaults** ship at `.agentic-driver/router.defaults.json`. The
590
- template contains no accounts and one disabled placeholder seat, so it loads
591
- out of the box and schedules nothing until you enable a real seat.
592
- - **User profile (precedence over defaults):**
593
- `~/.config/agentic-driver/router/profile.json`, or the path in the
594
- `AGENTIC_DRIVER_ROUTER_PROFILE` environment variable. A profile section
595
- replaces the matching defaults section wholesale; the merged config must
596
- validate or the router refuses to produce decisions (fails closed).
597
- - **Operational state:** `~/.local/share/agentic-driver/router/state.db`, or
598
- `AGENTIC_DRIVER_ROUTER_DB`. This store is operational state only. The
599
- authenticated claims file and the board writer remain the only dispatch
600
- authority, and SQLite is repaired from them, never the reverse.
601
- - **Runtime requirement:** the operational store uses the built-in
602
- `node:sqlite` module. Node.js 22.5 or newer is required.
603
-
604
- ### Tunable configuration values
605
-
606
- Every numeric policy in the router is a configuration key — never a
607
- constant in code. All keys are **required**. The repository defaults below
608
- are conservative starting values. Your profile overrides them. A missing key
609
- is a configuration validation failure: the router fails closed rather than
610
- guess. Changing a value participates in the configuration digest, so cached
611
- routing decisions computed under the old values no longer apply.
612
-
613
- | Section | Key | Default | What it controls |
614
- |---|---|---|---|
615
- | `eligibility.reserve` | `floorPercent` | 40 | Share of every quota window kept in reserve (per account, per window). Headroom for an unexpected burst and your next interactive session, without idling the account. |
616
- | `eligibility.reserve` | `scope` | `account-window` | How widely the reserve floor applies (per account per window at launch). |
617
- | `eligibility.reserve` | `coldStartFraction` | 0.25 | Consumption assumed for a seat with too little history: deliberately generous, so a mis-estimate fails eligibility instead of draining the account. |
618
- | `eligibility.reserve` | `estimateSamples` | 20 | How many recent dispatches the consumption estimate averages: enough to smooth outliers, quick enough to track a model change. |
619
- | `eligibility.reserve` | `estimateMinSamples` | 5 | Below this many observations the average is treated as noise and the cold-start value is used instead. |
620
- | `eligibility.reserve` | `estimateOutlierSigma` | 3 | Outlier cut for the estimate: one unusually slow dispatch cannot inflate the average, real tail latency is kept. |
621
- | `eligibility.reserve` | `ownerInteractiveOverride` | false | Whether your own interactive dispatches bypass the reserve rule (they do not, at launch). |
622
- | `ranking` | `janusUrl` | `http://127.0.0.1:8787` | The local janus evaluation service endpoint used for semantic ranking. |
623
- | `ranking` | `maxCandidates` | 8 | At most eight eligible seats are ranked per request; preference order trims first. |
624
- | `ranking` | `fallback` | `preference-order` | What happens when ranking is unavailable: dispatch falls back to your preference order and never blocks. |
625
- | `gate` | `threshold` | 0.9 | The dispatch gate's confidence threshold. A task is only dispatched when the evaluation clears it. Lower it only with evidence from your own audit log. |
626
- | `gate` | `timeoutMs` | 8000 | How long the gate waits for an evaluation. On timeout there is no verdict and dispatch is blocked (fail closed). |
627
- | `gate` | `requestBudgetTokens` | 30000 | Maximum size of a gate evaluation request. Normal evaluations use under a thousand tokens. |
628
- | seat record | `maxConcurrency` | 4 | How many workers one seat may run at once. The default suits a typical single machine or hosted seat. Tune it per seat: a cluster takes more, a constrained subscription endpoint may take less. |
629
- | automation policy | `maxConcurrent` | 4 | How many workers may run at once across all seats. The default is a conservative start. Tune it to the capacity you actually have, and raise it when you add seats or hardware. |
630
-
631
- Seats are named for infrastructure, not models: `dgx-spark-cluster`,
632
- `mac-studio`, `merge-gateway-worker`. The model is a mutable field on the
633
- seat, so changing models never renumbers your configuration. Each seat also
634
- carries `scopeStatements`: plain-language descriptions of what the seat is
635
- for. The dispatch gate reads them when judging whether a task fits.
636
-
637
661
  ## License
638
662
 
639
663
  AGPL-3.0-only with author-attribution additional terms (Section 7(b));
@@ -0,0 +1,210 @@
1
+ # janus — Jev evaluation wrapper service
2
+
3
+ janus is the local wrapper service for the Jev decision layer (design of
4
+ record: `.private-planning/JEV_DECISION_LAYER_DESIGN.md` §5–§7, Revision 2.1).
5
+ It wraps the official `@typesafe-ai/sdk` (TypeSafe Direct — the sole
6
+ provider; the Vercel AI Gateway and merge gateway were dropped by owner
7
+ decision) and exposes one evaluation endpoint plus health/readiness/version
8
+ probes on loopback.
9
+
10
+ ## What janus does
11
+
12
+ - `POST /evaluate` — evaluates typed questions (boolean / choice / score)
13
+ against one shared state via TypeSafe Direct (`typesafe-ai`, model `jev-latest`).
14
+ - `GET /healthz` — liveness.
15
+ - `GET /readyz` — readiness (gateway key resolvable from the Keychain).
16
+ - `GET /version` — janus version, AI SDK version, schema versions, allowlist.
17
+
18
+ Guarantees:
19
+
20
+ - **Model allowlist** (server-side config; request-level model overrides are
21
+ rejected with `model_unauthorized`).
22
+ - **Loopback only** — binds `127.0.0.1` and nothing else. No auth beyond
23
+ loopback; it is a local tool. There is deliberately no bind-address option.
24
+ - **Single-flight** — identical requests (same evalInputDigest) coalesce;
25
+ distinct in-flight evaluations are capped (default 4) and rejected with
26
+ `busy`.
27
+ - **30k-token request budget** — oversized requests are rejected with
28
+ `state_too_large`, never truncated silently.
29
+ - **Redaction** — a general secrets/PII pass runs over the state before any
30
+ external call.
31
+ - **Audit log** — append-only JSONL of request id, timestamps, question
32
+ names, verdict probabilities, and token usage. **State contents never
33
+ reach the audit log.** 90-day default retention, size-capped with
34
+ oldest-first rotation.
35
+ - **Daily cost breaker** — token/cost ceilings in config; on exceed,
36
+ `/evaluate` returns `busy` with `budget_exhausted` detail until the UTC
37
+ day window resets.
38
+ - **Timeouts** — 30s service-side cap; callers may pass a shorter
39
+ per-request `timeoutMs` (capped at 30s).
40
+
41
+ ## Provisioning the Keychain item
42
+
43
+ The TypeSafe Direct key is read from the macOS Keychain at runtime — janus
44
+ does not rely on inherited environment variables. The `typesafe-ai` /
45
+ `Typesafe AI` generic-password item does not exist by default; create it
46
+ once (interactive provisioning):
47
+
48
+ ```sh
49
+ security add-generic-password -U -a typesafe-ai -s "Typesafe AI" -w
50
+ # prompts for the secret; nothing is echoed
51
+ ```
52
+
53
+ janus reads it with `security find-generic-password -w` under guarded
54
+ capture: stdout is piped directly into memory, never echoed, never logged,
55
+ never passed to spawned subprocesses. If the item is missing, janus returns
56
+ `model_unauthorized` with `detail.key_unavailable: true` and `GET /readyz`
57
+ reports 503 with the provisioning hint.
58
+
59
+ Verify the item is present (no secret material is printed):
60
+
61
+ ```sh
62
+ /usr/bin/security find-generic-password -s "Typesafe AI" -a typesafe-ai >/dev/null 2>&1 && echo present
63
+ ```
64
+
65
+ To rotate the secret, re-run the `security add-generic-password -U` command
66
+ above and enter the new value at the prompt (`-U` updates the existing item).
67
+ To remove the item entirely:
68
+
69
+ ```sh
70
+ security delete-generic-password -a typesafe-ai -s "Typesafe AI"
71
+ ```
72
+
73
+ ## Running
74
+
75
+ Dev (from `janus/`):
76
+
77
+ ```sh
78
+ npx tsx janus/server.ts # listens on 127.0.0.1:8787
79
+ ```
80
+
81
+ Config is read from `~/.config/typesafe-ai/jev/janus-config.json`
82
+ (override with `JANUS_CONFIG_PATH`). Defaults work with no file:
83
+
84
+ ```json
85
+ {
86
+ "port": 8787,
87
+ "modelAllowlist": ["typesafe-ai/jev"],
88
+ "defaultModel": "typesafe-ai/jev",
89
+ "maxConcurrent": 4,
90
+ "dailyTokenCeiling": 2000000,
91
+ "dailyCostCeilingUsd": 10,
92
+ "audit": { "path": "~/.local/share/janus/audit", "maxBytes": 10485760, "retentionDays": 90 }
93
+ }
94
+ ```
95
+
96
+ launchd — template at `janus/user.janus.plist`. The template is a starting
97
+ point: every user-specific value (the label, the checkout path, log
98
+ locations) is marked ADJUST in the file. Copy it out of the repository,
99
+ adjust it, then load it:
100
+
101
+ ```sh
102
+ cp janus/user.janus.plist ~/Library/LaunchAgents/
103
+ ```
104
+
105
+ ```sh
106
+ launchctl load ~/Library/LaunchAgents/user.janus.plist
107
+ ```
108
+
109
+ Tests (no live gateway calls — the SDK is mocked):
110
+
111
+ ```sh
112
+ cd janus && npm test
113
+ ```
114
+
115
+ ## Endpoint contract
116
+
117
+ ### POST /evaluate
118
+
119
+ Request:
120
+
121
+ ```json
122
+ {
123
+ "state": "string | object | array (message histories pass directly)",
124
+ "questions": [
125
+ { "type": "boolean", "question": "Was a refund issued?", "threshold": 0.9 },
126
+ { "type": "choice", "question": "Route this ticket", "choices": ["billing", "tech", "other"] },
127
+ { "type": "score", "question": "Rate this PR" }
128
+ ],
129
+ "model": "optional; must be on the server allowlist, overrides rejected",
130
+ "timeoutMs": 10000
131
+ }
132
+ ```
133
+
134
+ Question schema v1 (§6), enforced atomically — any malformed, absent, or
135
+ contradictory element fails the whole request with `validation_error`:
136
+
137
+ - **boolean** — `threshold` optional, default `0.9`, comparison
138
+ `probability >= threshold`. Optional `criteria: {true?, false?}` labels.
139
+ - **choice** — `choices` array REQUIRED; the verdict's `choice` is guaranteed
140
+ to be a member.
141
+ - **score** — numeric 0..5, one decimal. Ordinal vocabularies must be
142
+ **choice** questions, not scores.
143
+
144
+ §6 → SDK mapping (verified against ai@7.0.106 at launch step 1): the SDK's
145
+ `experimental_evaluate` has **no top-level `criteria` argument**; the
146
+ candidate set lives inside each question as a REQUIRED `criteria` field
147
+ (choice: `Record<option, description | null>`; score: ordered level array;
148
+ boolean: optional `{true?, false?}`). janus accepts §6's `choices` array at
149
+ the transport layer and maps it to `criteria: {a: null, b: null, ...}`. The
150
+ score vocabulary is passed as the ordered level array `["0".."5"]`.
151
+
152
+ Success response (200):
153
+
154
+ ```json
155
+ {
156
+ "request_id": "uuid",
157
+ "verdict": {
158
+ "Was a refund issued?": { "type": "boolean", "probability": 0.97, "passed": true },
159
+ "Route this ticket": { "type": "choice", "choice": "billing", "probabilities": {} },
160
+ "Rate this PR": { "type": "score", "score": 3.0 }
161
+ },
162
+ "usage": { "inputTokens": 0, "outputTokens": 0, "totalTokens": 0 },
163
+ "model": "typesafe-ai/jev",
164
+ "digest": "sha256 evalInputDigest"
165
+ }
166
+ ```
167
+
168
+ "Model returned false" is a **200** with `passed: false` / low probability —
169
+ not an error.
170
+
171
+ ### Error taxonomy
172
+
173
+ All errors: `{ "error": code, "message": ..., "request_id": ..., "detail"?: {...} }`
174
+
175
+ | Code | HTTP | Meaning |
176
+ |---|---|---|
177
+ | `validation_error` | 400 | bad request (schema, JSON, unknown route) |
178
+ | `state_too_large` | 413 | over the 30k-token budget; never truncated |
179
+ | `unknown_question` | 400 | reserved for router/schema-version mismatches |
180
+ | `model_unauthorized` | 403/502/503 | not on allowlist, credential rejected, or key unavailable (`detail.key_unavailable`) |
181
+ | `gateway_unavailable` | 503 | gateway unreachable / model missing / rate limit |
182
+ | `timeout` | 504 | evaluation exceeded the budget |
183
+ | `busy` | 429 | concurrency cap or `detail.budget_exhausted` |
184
+ | `degraded` | 503 | internal error |
185
+
186
+ Callers distinguish **"model returned false"** (200, low probability) from
187
+ **"no verdict"** (`timeout` / `gateway_unavailable`) from **"bad request"**
188
+ (`validation_error`).
189
+
190
+ ## Layout
191
+
192
+ ```
193
+ janus/
194
+ janus/
195
+ config.ts config load/validate (allowlist, ceilings, retention)
196
+ schema.ts §6 question schema v1 enforcement (atomic fail-closed)
197
+ verdict.ts verdict enforcement (membership, ranges, thresholds)
198
+ budget.ts token estimation, 30k budget, evalInputDigest
199
+ redaction pass: shared library typesafe-secure
200
+ lib/redaction (MIT), wired via a file: dependency
201
+ keychain.ts guarded `security find-generic-password -w` capture
202
+ single_flight.ts digest coalescing + concurrency cap
203
+ budget_breaker.ts daily token/cost breaker
204
+ audit.ts append-only JSONL audit log, rotation + retention
205
+ sdk.ts THE ONLY AI SDK import site (pinned versions)
206
+ evaluate.ts /evaluate pipeline + SDK error classification
207
+ server.ts HTTP surface, loopback bind, entrypoint
208
+ tests/ node:test suites (mocked SDK, no live calls)
209
+ user.janus.plist launchd template
210
+ ```
@@ -0,0 +1,196 @@
1
+ # janus — Jev evaluation wrapper service
2
+
3
+ janus is the local wrapper service for the Jev decision layer (design of
4
+ record: `.private-planning/JEV_DECISION_LAYER_DESIGN.md` §5–§7, Revision 2.1).
5
+ It wraps the official `@typesafe-ai/sdk` (TypeSafe Direct — the sole
6
+ provider; the Vercel AI Gateway and merge gateway were dropped by owner
7
+ decision) and exposes one evaluation endpoint plus health/readiness/version
8
+ probes on loopback.
9
+
10
+ ## What janus does
11
+
12
+ - `POST /evaluate` — evaluates typed questions (boolean / choice / score)
13
+ against one shared state via TypeSafe Direct (`typesafe-ai`, model `jev-latest`).
14
+ - `GET /healthz` — liveness.
15
+ - `GET /readyz` — readiness (gateway key resolvable from the Keychain).
16
+ - `GET /version` — janus version, AI SDK version, schema versions, allowlist.
17
+
18
+ Guarantees:
19
+
20
+ - **Model allowlist** (server-side config; request-level model overrides are
21
+ rejected with `model_unauthorized`).
22
+ - **Loopback only** — binds `127.0.0.1` and nothing else. No auth beyond
23
+ loopback; it is a local tool. There is deliberately no bind-address option.
24
+ - **Single-flight** — identical requests (same evalInputDigest) coalesce;
25
+ distinct in-flight evaluations are capped (default 4) and rejected with
26
+ `busy`.
27
+ - **30k-token request budget** — oversized requests are rejected with
28
+ `state_too_large`, never truncated silently.
29
+ - **Redaction** — a general secrets/PII pass runs over the state before any
30
+ external call.
31
+ - **Audit log** — append-only JSONL of request id, timestamps, question
32
+ names, verdict probabilities, and token usage. **State contents never
33
+ reach the audit log.** 90-day default retention, size-capped with
34
+ oldest-first rotation.
35
+ - **Daily cost breaker** — token/cost ceilings in config; on exceed,
36
+ `/evaluate` returns `busy` with `budget_exhausted` detail until the UTC
37
+ day window resets.
38
+ - **Timeouts** — 30s service-side cap; callers may pass a shorter
39
+ per-request `timeoutMs` (capped at 30s).
40
+
41
+ ## Provisioning the Keychain item
42
+
43
+ The TypeSafe Direct key is read from the macOS Keychain at runtime — janus
44
+ does not rely on inherited environment variables. The `typesafe-ai` /
45
+ `Typesafe AI` generic-password item does not exist by default; create it
46
+ once (interactive provisioning):
47
+
48
+ ```sh
49
+ security add-generic-password -U -a typesafe-ai -s "Typesafe AI" -w
50
+ # prompts for the secret; nothing is echoed
51
+ ```
52
+
53
+ janus reads it with `security find-generic-password -w` under guarded
54
+ capture: stdout is piped directly into memory, never echoed, never logged,
55
+ never passed to spawned subprocesses. If the item is missing, janus returns
56
+ `model_unauthorized` with `detail.key_unavailable: true` and `GET /readyz`
57
+ reports 503 with the provisioning hint.
58
+
59
+ ## Running
60
+
61
+ Dev (from `janus/`):
62
+
63
+ ```sh
64
+ npx tsx janus/server.ts # listens on 127.0.0.1:8787
65
+ ```
66
+
67
+ Config is read from `~/.config/typesafe-ai/jev/janus-config.json`
68
+ (override with `JANUS_CONFIG_PATH`). Defaults work with no file:
69
+
70
+ ```json
71
+ {
72
+ "port": 8787,
73
+ "modelAllowlist": ["typesafe-ai/jev"],
74
+ "defaultModel": "typesafe-ai/jev",
75
+ "maxConcurrent": 4,
76
+ "dailyTokenCeiling": 2000000,
77
+ "dailyCostCeilingUsd": 10,
78
+ "audit": { "path": "~/.local/share/janus/audit", "maxBytes": 10485760, "retentionDays": 90 }
79
+ }
80
+ ```
81
+
82
+ launchd — template at `janus/user.janus.plist`. The template is a starting
83
+ point: every user-specific value (the label, the checkout path, log
84
+ locations) is marked ADJUST in the file. Copy it out of the repository,
85
+ adjust it, then load it:
86
+
87
+ ```sh
88
+ cp janus/user.janus.plist ~/Library/LaunchAgents/
89
+ ```
90
+
91
+ ```sh
92
+ launchctl load ~/Library/LaunchAgents/user.janus.plist
93
+ ```
94
+
95
+ Tests (no live gateway calls — the SDK is mocked):
96
+
97
+ ```sh
98
+ cd janus && npm test
99
+ ```
100
+
101
+ ## Endpoint contract
102
+
103
+ ### POST /evaluate
104
+
105
+ Request:
106
+
107
+ ```json
108
+ {
109
+ "state": "string | object | array (message histories pass directly)",
110
+ "questions": [
111
+ { "type": "boolean", "question": "Was a refund issued?", "threshold": 0.9 },
112
+ { "type": "choice", "question": "Route this ticket", "choices": ["billing", "tech", "other"] },
113
+ { "type": "score", "question": "Rate this PR" }
114
+ ],
115
+ "model": "optional; must be on the server allowlist, overrides rejected",
116
+ "timeoutMs": 10000
117
+ }
118
+ ```
119
+
120
+ Question schema v1 (§6), enforced atomically — any malformed, absent, or
121
+ contradictory element fails the whole request with `validation_error`:
122
+
123
+ - **boolean** — `threshold` optional, default `0.9`, comparison
124
+ `probability >= threshold`. Optional `criteria: {true?, false?}` labels.
125
+ - **choice** — `choices` array REQUIRED; the verdict's `choice` is guaranteed
126
+ to be a member.
127
+ - **score** — numeric 0..5, one decimal. Ordinal vocabularies must be
128
+ **choice** questions, not scores.
129
+
130
+ §6 → SDK mapping (verified against ai@7.0.106 at launch step 1): the SDK's
131
+ `experimental_evaluate` has **no top-level `criteria` argument**; the
132
+ candidate set lives inside each question as a REQUIRED `criteria` field
133
+ (choice: `Record<option, description | null>`; score: ordered level array;
134
+ boolean: optional `{true?, false?}`). janus accepts §6's `choices` array at
135
+ the transport layer and maps it to `criteria: {a: null, b: null, ...}`. The
136
+ score vocabulary is passed as the ordered level array `["0".."5"]`.
137
+
138
+ Success response (200):
139
+
140
+ ```json
141
+ {
142
+ "request_id": "uuid",
143
+ "verdict": {
144
+ "Was a refund issued?": { "type": "boolean", "probability": 0.97, "passed": true },
145
+ "Route this ticket": { "type": "choice", "choice": "billing", "probabilities": {} },
146
+ "Rate this PR": { "type": "score", "score": 3.0 }
147
+ },
148
+ "usage": { "inputTokens": 0, "outputTokens": 0, "totalTokens": 0 },
149
+ "model": "typesafe-ai/jev",
150
+ "digest": "sha256 evalInputDigest"
151
+ }
152
+ ```
153
+
154
+ "Model returned false" is a **200** with `passed: false` / low probability —
155
+ not an error.
156
+
157
+ ### Error taxonomy
158
+
159
+ All errors: `{ "error": code, "message": ..., "request_id": ..., "detail"?: {...} }`
160
+
161
+ | Code | HTTP | Meaning |
162
+ |---|---|---|
163
+ | `validation_error` | 400 | bad request (schema, JSON, unknown route) |
164
+ | `state_too_large` | 413 | over the 30k-token budget; never truncated |
165
+ | `unknown_question` | 400 | reserved for router/schema-version mismatches |
166
+ | `model_unauthorized` | 403/502/503 | not on allowlist, credential rejected, or key unavailable (`detail.key_unavailable`) |
167
+ | `gateway_unavailable` | 503 | gateway unreachable / model missing / rate limit |
168
+ | `timeout` | 504 | evaluation exceeded the budget |
169
+ | `busy` | 429 | concurrency cap or `detail.budget_exhausted` |
170
+ | `degraded` | 503 | internal error |
171
+
172
+ Callers distinguish **"model returned false"** (200, low probability) from
173
+ **"no verdict"** (`timeout` / `gateway_unavailable`) from **"bad request"**
174
+ (`validation_error`).
175
+
176
+ ## Layout
177
+
178
+ ```
179
+ janus/
180
+ janus/
181
+ config.ts config load/validate (allowlist, ceilings, retention)
182
+ schema.ts §6 question schema v1 enforcement (atomic fail-closed)
183
+ verdict.ts verdict enforcement (membership, ranges, thresholds)
184
+ budget.ts token estimation, 30k budget, evalInputDigest
185
+ redaction pass: shared library typesafe-secure
186
+ lib/redaction (MIT), wired via a file: dependency
187
+ keychain.ts guarded `security find-generic-password -w` capture
188
+ single_flight.ts digest coalescing + concurrency cap
189
+ budget_breaker.ts daily token/cost breaker
190
+ audit.ts append-only JSONL audit log, rotation + retention
191
+ sdk.ts THE ONLY AI SDK import site (pinned versions)
192
+ evaluate.ts /evaluate pipeline + SDK error classification
193
+ server.ts HTTP surface, loopback bind, entrypoint
194
+ tests/ node:test suites (mocked SDK, no live calls)
195
+ user.janus.plist launchd template
196
+ ```