instar 1.3.423 → 1.3.425
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dashboard/index.html +121 -0
- package/dist/config/ConfigDefaults.d.ts.map +1 -1
- package/dist/config/ConfigDefaults.js +8 -0
- package/dist/config/ConfigDefaults.js.map +1 -1
- package/dist/core/CircuitBreakingIntelligenceProvider.d.ts +10 -0
- package/dist/core/CircuitBreakingIntelligenceProvider.d.ts.map +1 -1
- package/dist/core/CircuitBreakingIntelligenceProvider.js +45 -8
- package/dist/core/CircuitBreakingIntelligenceProvider.js.map +1 -1
- package/dist/core/ClaudeCliIntelligenceProvider.d.ts.map +1 -1
- package/dist/core/ClaudeCliIntelligenceProvider.js +6 -0
- package/dist/core/ClaudeCliIntelligenceProvider.js.map +1 -1
- package/dist/core/CodexCliIntelligenceProvider.d.ts.map +1 -1
- package/dist/core/CodexCliIntelligenceProvider.js +9 -0
- package/dist/core/CodexCliIntelligenceProvider.js.map +1 -1
- package/dist/core/ExternalOperationGate.js +1 -1
- package/dist/core/ExternalOperationGate.js.map +1 -1
- package/dist/core/GeminiCliIntelligenceProvider.d.ts.map +1 -1
- package/dist/core/GeminiCliIntelligenceProvider.js +7 -0
- package/dist/core/GeminiCliIntelligenceProvider.js.map +1 -1
- package/dist/core/InputGuard.js +1 -1
- package/dist/core/InputGuard.js.map +1 -1
- package/dist/core/IntelligenceRouter.d.ts +14 -0
- package/dist/core/IntelligenceRouter.d.ts.map +1 -1
- package/dist/core/IntelligenceRouter.js +55 -19
- package/dist/core/IntelligenceRouter.js.map +1 -1
- package/dist/core/IntentTestHarness.js +1 -1
- package/dist/core/IntentTestHarness.js.map +1 -1
- package/dist/core/InteractivePoolIntelligenceProvider.d.ts.map +1 -1
- package/dist/core/InteractivePoolIntelligenceProvider.js +8 -0
- package/dist/core/InteractivePoolIntelligenceProvider.js.map +1 -1
- package/dist/core/MessageSentinel.d.ts.map +1 -1
- package/dist/core/MessageSentinel.js +8 -1
- package/dist/core/MessageSentinel.js.map +1 -1
- package/dist/core/MessagingToneGate.js +1 -1
- package/dist/core/MessagingToneGate.js.map +1 -1
- package/dist/core/PiCliIntelligenceProvider.d.ts.map +1 -1
- package/dist/core/PiCliIntelligenceProvider.js +20 -0
- package/dist/core/PiCliIntelligenceProvider.js.map +1 -1
- package/dist/core/PostUpdateMigrator.js +6 -6
- package/dist/core/PostUpdateMigrator.js.map +1 -1
- package/dist/core/types.d.ts +41 -0
- package/dist/core/types.d.ts.map +1 -1
- package/dist/core/types.js.map +1 -1
- package/dist/monitoring/CommitmentSentinel.d.ts.map +1 -1
- package/dist/monitoring/CommitmentSentinel.js +16 -0
- package/dist/monitoring/CommitmentSentinel.js.map +1 -1
- package/dist/monitoring/FeatureMetricsLedger.d.ts +16 -0
- package/dist/monitoring/FeatureMetricsLedger.d.ts.map +1 -1
- package/dist/monitoring/FeatureMetricsLedger.js +71 -2
- package/dist/monitoring/FeatureMetricsLedger.js.map +1 -1
- package/dist/scaffold/templates.d.ts.map +1 -1
- package/dist/scaffold/templates.js +4 -3
- package/dist/scaffold/templates.js.map +1 -1
- package/dist/server/AgentServer.d.ts +1 -0
- package/dist/server/AgentServer.d.ts.map +1 -1
- package/dist/server/AgentServer.js +27 -0
- package/dist/server/AgentServer.js.map +1 -1
- package/package.json +1 -1
- package/src/data/builtin-manifest.json +20 -20
- package/src/scaffold/templates.ts +4 -3
- package/upgrades/1.3.424.md +28 -0
- package/upgrades/1.3.425.md +23 -0
- package/upgrades/eli16/observable-intelligence-llm-activity.md +26 -0
- package/upgrades/eli16/provider-swap.md +33 -0
- package/upgrades/side-effects/observable-intelligence-llm-activity.md +25 -0
- package/upgrades/side-effects/provider-swap.md +29 -0
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Upgrade Guide — vNEXT
|
|
2
|
+
|
|
3
|
+
<!-- assembled-by: assemble-next-md -->
|
|
4
|
+
<!-- bump: minor -->
|
|
5
|
+
|
|
6
|
+
## What Changed
|
|
7
|
+
|
|
8
|
+
The per-feature LLM metrics ledger now records, for every call the system makes on its own behalf (sentinels, gates, reflectors, background checks), **which provider + model actually ran it** and **whether the check acted (fired) or found nothing (noop)** — closing two long-standing blind spots: the `model` column was never populated, and the `fired` verdict was deferred to a "Phase 2" that never shipped, so codex/gemini calls were unattributable and every completed call read as a no-op. Recording is enforced at the single funnel (`CircuitBreakingIntelligenceProvider`) via two additive, optional seams on `IntelligenceOptions` — `onModel` (every provider surfaces its resolved model/framework, independent of token usage) and `classifyVerdict` (the caller classifies act-vs-no-act). A new **LLM Activity** dashboard tab renders it in plain language over a 24h/7d/30d window. Bounded retention (default 30 days, `monitoring.featureMetrics.retentionDays`) ages the audit trail out so it is never hoarded forever.
|
|
9
|
+
|
|
10
|
+
## What to Tell Your User
|
|
11
|
+
|
|
12
|
+
You can now audit what your agent's "autopilot" is actually doing: open the **LLM Activity** dashboard tab to see, per check, which AI provider ran it, how often it acted on something vs. found nothing, how often it was skipped to save rate limits, and what it cost — over time. This is the new *Observable Intelligence* standard: no autonomous AI action the system takes is allowed to be invisible. Nothing for you to do; it takes effect on the next restart.
|
|
13
|
+
|
|
14
|
+
## Summary of New Capabilities
|
|
15
|
+
|
|
16
|
+
- `/metrics/features` rows now include `frameworks[]`, `models[]`, real `fired`/`fireRate`, and `shed` — answering "which provider ran this sentinel?" and "is it doing real work or just being skipped?".
|
|
17
|
+
- New **LLM Activity** dashboard tab (read-only) over a selectable window.
|
|
18
|
+
- `IntelligenceOptions.onModel` + `classifyVerdict` seams; every provider (Claude/Codex/Gemini/Pi/InteractivePool) surfaces its model/framework; `MessageSentinel` + `CommitmentSentinel` classify their verdicts.
|
|
19
|
+
- Bounded retention via `FeatureMetricsLedger.pruneOlderThan` + `monitoring.featureMetrics.retentionDays` (default 30).
|
|
20
|
+
- New constitutional standard *Observable Intelligence — No Autonomous LLM Action Is Unauditable* (proposed, pending ratification) + spec `docs/specs/observable-intelligence.md`.
|
|
21
|
+
|
|
22
|
+
## Evidence
|
|
23
|
+
|
|
24
|
+
`tests/unit/FeatureMetricsLedger.test.ts` (12, +4: framework rollup, prune, idempotent column add), `tests/unit/CircuitBreaking-feature-metrics-tap.test.ts` (19, +7: onModel/classifyVerdict/error-path attribution + real-ledger rollup), `tests/integration/metrics-features-routes.test.ts` (+1: provider/model/fired through the route), `tests/e2e/metrics-features-lifecycle.test.ts` green; no-silent-fallbacks ratchet + feature-delivery-completeness parity green; `tsc` clean.
|
|
25
|
+
|
|
26
|
+
## Scope (honest)
|
|
27
|
+
|
|
28
|
+
Provider/model attribution + the fired/noop verdict are recorded for all providers. Per-call **token cost** is captured where the provider surfaces it (Claude, Pi); codex/gemini exec output carries no usage block, so their per-call tokens stay null (model/framework + outcome are still recorded; account-level codex cost is visible via `/codex/usage`) — per-call token parsing for those via their `--json` modes is a bounded follow-up. Read-only observability throughout: it never gates, blocks, or mutates the path it observes.
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# Upgrade Guide — vNEXT
|
|
2
|
+
|
|
3
|
+
<!-- assembled-by: assemble-next-md -->
|
|
4
|
+
<!-- bump: patch -->
|
|
5
|
+
|
|
6
|
+
## What Changed
|
|
7
|
+
|
|
8
|
+
Herd-aware provider-swap at the shared `IntelligenceRouter` (step 2 of the "No Silent Degradation to Brittle Fallback" initiative). On a RUNTIME provider failure, a SAFETY-GATING call (`attribution.gating: true`) walks the configured `componentFrameworks.failureSwap` framework list, skips any whose circuit is open (no herd), serves from the first healthy one, and re-throws if every target is down so the caller fails CLOSED. Only gating calls swap; advisory calls keep today's propagate-to-heuristic. New `failureSwap` config + `gating` attribution flag. 5 safety gates marked gating (ExternalOperationGate, MessagingToneGate, MessageSentinel, IntentLlmJudge, InputGuard). Generalizes across all accessible providers (Claude/Codex/Pi/Copilot-via-Pi); composes with the subscription-pool.
|
|
9
|
+
|
|
10
|
+
## What to Tell Your User
|
|
11
|
+
|
|
12
|
+
- "When a safety check's AI is rate-limited, I now try your other AI providers (Codex, Pi, Copilot) before giving up — only failing closed if every one is down. It skips any provider that's also overloaded, so I never make a rate-limit worse. It's off until you set a provider order, and it only kicks in for the real safety gates, not background chatter."
|
|
13
|
+
|
|
14
|
+
## Summary of New Capabilities
|
|
15
|
+
|
|
16
|
+
| Capability | How to Use |
|
|
17
|
+
|-----------|-----------|
|
|
18
|
+
| Provider-swap for safety gates | Set `componentFrameworks.failureSwap: ["codex-cli","pi-cli"]` in config |
|
|
19
|
+
| Per-call gating marker | Internal: safety-gating callers pass `attribution.gating: true` |
|
|
20
|
+
|
|
21
|
+
## Evidence
|
|
22
|
+
|
|
23
|
+
6 new router unit tests: swap-to-healthy-on-failure; fail-closed when all-down; no-swap for non-gating; no-swap when unconfigured; skip-circuit-open-target-and-use-next (herd-aware); unconfigured-unchanged. 17 router tests + 210 caller tests green; tsc clean. Default (no failureSwap) keeps byte-identical behavior.
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
# Observable Intelligence — in plain language
|
|
2
|
+
|
|
3
|
+
## The problem
|
|
4
|
+
|
|
5
|
+
Your agent has a bunch of little "autopilot" helpers running in the background — the things we call sentinels and gates. They watch your messages, decide whether something needs action, and quietly handle housekeeping. Each one can ask an AI model a quick question to make its call.
|
|
6
|
+
|
|
7
|
+
Someone asked a simple, fair question: *"Which AI provider are these helpers actually using, and are they even doing any real work?"* And the honest answer was: **we couldn't fully tell.** The log that's supposed to track this had two holes in it:
|
|
8
|
+
|
|
9
|
+
1. It never wrote down **which AI** ran each check. So if you'd switched your sentinels over to a different provider (say, Codex instead of Claude), the log couldn't prove it.
|
|
10
|
+
2. It never wrote down whether a check **actually did something** versus just looked and found nothing. So every check looked identical in the data — you couldn't tell a hard-working guard from a useless one.
|
|
11
|
+
|
|
12
|
+
A helper that acts on your behalf but can't show what it chose to do is a helper you can't hold accountable. That's the gap this closes.
|
|
13
|
+
|
|
14
|
+
## What changed
|
|
15
|
+
|
|
16
|
+
Now, every time one of these helpers asks an AI a question, the system writes down: **which provider and model answered, whether the helper acted or found nothing, whether it got skipped to save on rate limits, how much it cost, and how long it took.** It records this at one shared "chokepoint" that every helper already passes through — so it covers all of them automatically, today and in the future, with nothing to remember.
|
|
17
|
+
|
|
18
|
+
There's a new **"LLM Activity" tab** in your dashboard that shows all of this in plain English: one row per helper, over a window you pick (last day, week, or month). You can finally see, at a glance, which helpers earn their keep and which are mostly idle or skipped.
|
|
19
|
+
|
|
20
|
+
## The balance
|
|
21
|
+
|
|
22
|
+
We don't keep this log forever — that would just be hoarding. It's kept about a month (you can change that), then old entries age out. Long enough to spot trends, not so long it piles up.
|
|
23
|
+
|
|
24
|
+
## Is it risky?
|
|
25
|
+
|
|
26
|
+
No. This only **watches** — it never blocks, changes, or slows down anything the helpers do. If the new tracking ever hiccupped, the helper would carry on exactly as before. It's all extra, additive recording, fully covered by tests.
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Herd-Aware Provider-Swap — Plain-English Overview
|
|
2
|
+
|
|
3
|
+
> The one-line version: when a safety gate's AI is rate-limited, instead of giving up (and falling back to weak code), it tries your OTHER AI providers first — and only fails closed if every one is down.
|
|
4
|
+
|
|
5
|
+
## The problem in one breath
|
|
6
|
+
|
|
7
|
+
We just made the safety gates fail closed when their AI is down. That's safe, but it means more things pause for approval during a rate-limit. The better answer: don't fail at all if another AI provider is available — swap to it.
|
|
8
|
+
|
|
9
|
+
## What already exists
|
|
10
|
+
|
|
11
|
+
- **One shared AI router** that every gate and sentinel calls. It already routes different components to different AI frameworks (Claude, Codex, Pi) and gives each its own circuit breaker.
|
|
12
|
+
- **The fail-closed gates** (from the previous change) — on an AI failure they require approval / hold, rather than silently proceeding.
|
|
13
|
+
|
|
14
|
+
## What this adds
|
|
15
|
+
|
|
16
|
+
A **failure-swap** at that one router. When a SAFETY-GATING call's AI fails, the router walks a configured ordered list of fallback frameworks, **skips any whose circuit is already open** (so it never piles load onto a provider that's also struggling), and serves the answer from the first healthy one. Only if every provider is down does it fail closed.
|
|
17
|
+
|
|
18
|
+
## The new pieces
|
|
19
|
+
|
|
20
|
+
- **`failureSwap` config** — an ordered list of frameworks to try on a gating call's failure (e.g. Codex, then Pi). Default: empty = today's behavior, nothing changes unless you turn it on.
|
|
21
|
+
- **A `gating` flag** on the call — only safety-gating calls swap. This keeps the "herd" tiny: a rate-limited framework can't dump its whole load onto another, because only the few real gates swap (advisory calls keep degrading as before).
|
|
22
|
+
|
|
23
|
+
## The safeguards
|
|
24
|
+
|
|
25
|
+
**Prevents herding.** The original router deliberately didn't swap, to avoid a rate-limited framework dumping all its traffic onto the fallback at once. This keeps that protection two ways: only gating calls swap (small set), and any fallback whose circuit is already open is skipped instantly.
|
|
26
|
+
|
|
27
|
+
**Fail-closed is still the floor.** If every provider is down, the error re-throws and the gate fails closed — never a silent drop to weak code.
|
|
28
|
+
|
|
29
|
+
**Generalizes to all your providers.** Claude, Codex, Pi, and Copilot-via-Pi are each a separate account/quota, so the same model reachable through multiple paths is redundancy. Composes with the subscription-pool (which manages accounts within a provider).
|
|
30
|
+
|
|
31
|
+
## What ships when
|
|
32
|
+
|
|
33
|
+
This PR is the router swap + the gating flags on the five safety gates. Follow-ups: a model-family-diverse default order, the lint, the iterative re-audit to convergence, and the throwaway-agent test harness.
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# Side-effects review — Observable Intelligence / LLM Activity
|
|
2
|
+
|
|
3
|
+
**Tier:** 1 (large but purely additive, read-only observability; no existing behavior changes; reversible; full 3-tier tests). Risk floor flags "new capability" — declared below floor with this rationale, recorded in the decision audit.
|
|
4
|
+
|
|
5
|
+
## Surface touched
|
|
6
|
+
|
|
7
|
+
- `IntelligenceOptions` gains two **optional** callbacks (`onModel`, `classifyVerdict`). Additive: every existing caller is byte-identical (evaluate() still returns `Promise<string>`).
|
|
8
|
+
- `CircuitBreakingIntelligenceProvider` (the single funnel) captures model/framework + classifies the verdict and records them. Pure side-channel — wrapped in try/catch; a throw in either callback cannot change what `evaluate()` returns or break the observed path.
|
|
9
|
+
- Five providers (Claude/Codex/Gemini/Pi/InteractivePool) each call `onModel` once per call. Each call is try/caught (`@silent-fallback-ok`) so it can never break the LLM path.
|
|
10
|
+
- Two sentinels (MessageSentinel, CommitmentSentinel) pass a `classifyVerdict`; the classification reuses their existing parse and defaults to noop on throw.
|
|
11
|
+
- `FeatureMetricsLedger`: new `framework` column via idempotent pragma-guarded `ALTER TABLE ADD COLUMN` (existing DBs migrate at open, no data loss); new `pruneOlderThan`; rollup gains `frameworks[]`/`models[]`.
|
|
12
|
+
- `AgentServer`: a retention prune timer (boot + every 6h, `unref`'d), cleared at shutdown.
|
|
13
|
+
|
|
14
|
+
## Risks considered
|
|
15
|
+
|
|
16
|
+
- **Schema migration**: covered by a unit test that opens an old-schema DB and confirms the column is added without losing the legacy row. The ALTER is pragma-guarded and idempotent.
|
|
17
|
+
- **Performance**: recording is one extra SQLite insert per call (already happening) plus two cheap callback invocations. No new network calls.
|
|
18
|
+
- **Retention deletion**: `pruneOlderThan` only deletes rows older than the cutoff; fail-open (a failed prune leaves rows for the next tick). Default 30d, configurable, `0` disables.
|
|
19
|
+
- **Observability never gates**: same guarantee as TokenLedger. A failed metric write, failed prune, or thrown callback degrades silently; the LLM path is unaffected.
|
|
20
|
+
- **No new HTTP route** (reuses `/metrics/features`), so no new auth/route surface. Dashboard tab is read-only.
|
|
21
|
+
- **Migration parity**: existing agents get the enhanced CLAUDE.md section via the existing `/metrics/features` migrateClaudeMd block (enhanced in place, idempotent — verified by `PostUpdateMigrator-metricsFeatures.test.ts`); new agents via the template.
|
|
22
|
+
|
|
23
|
+
## Constitutional fit
|
|
24
|
+
|
|
25
|
+
Implements the proposed **Observable Intelligence** standard (pending operator ratification); balanced by Responsible-Resource bounded retention. Sharpens the existing **Observability** standard.
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# Side-Effects Review — Herd-aware provider-swap
|
|
2
|
+
|
|
3
|
+
**Version / slug:** `provider-swap` · **Date:** `2026-06-07` · **Author:** `Echo` · **Second-pass:** `not required (Tier-1)`
|
|
4
|
+
|
|
5
|
+
## Summary of the change
|
|
6
|
+
`IntelligenceRouter.evaluate` now, on a RUNTIME provider failure for a SAFETY-GATING call (`attribution.gating: true`), walks `componentFrameworks.failureSwap` (ordered frameworks), skips circuit-open targets, serves from the first healthy one, and re-throws if all are down (caller fails closed). New `failureSwap` config field + `gating` attribution flag. 5 safety-gating callers marked gating (ExternalOperationGate, MessagingToneGate, MessageSentinel, IntentLlmJudge, InputGuard).
|
|
7
|
+
|
|
8
|
+
## Decision-point inventory
|
|
9
|
+
- `IntelligenceRouter.evaluate` failure path — add (swap-on-failure for gating calls)
|
|
10
|
+
- `ComponentFrameworksConfig.failureSwap` — add (opt-in)
|
|
11
|
+
- `IntelligenceOptions.attribution.gating` — add (opt-in marker)
|
|
12
|
+
|
|
13
|
+
## 1. Over-block / Under-block
|
|
14
|
+
No allow/deny surface. The swap only changes WHICH provider answers a gating call on failure; the gate's own verdict logic is unchanged. Non-gating calls and the unconfigured default are byte-identical to before.
|
|
15
|
+
|
|
16
|
+
## 2. Data / state
|
|
17
|
+
None. No files, no schema, no persistence. Pure in-memory routing.
|
|
18
|
+
|
|
19
|
+
## 3. Performance
|
|
20
|
+
On the FAILURE path only, a gating call may make up to N extra provider attempts (N = failureSwap length), each short-circuited by its own breaker if open. The happy path (provider healthy) is unchanged — one call. No hot-path cost added.
|
|
21
|
+
|
|
22
|
+
## 4. Failure modes / herding
|
|
23
|
+
This IS failure-mode handling. Herding is bounded two ways: only gating calls swap (small set), and circuit-open targets are skipped (no load onto a stressed provider). All-down → re-throw → caller fails closed. No infinite loop (finite list, `target===framework` skipped).
|
|
24
|
+
|
|
25
|
+
## 5. Security / auth
|
|
26
|
+
Hardens availability of the safety gates (they get a working provider before failing closed). No new endpoints/capabilities/credentials. The swap reuses already-wired per-framework providers.
|
|
27
|
+
|
|
28
|
+
## 6. Migration / compatibility
|
|
29
|
+
No migration. Default (no `failureSwap`) = unchanged behavior. Opt-in per agent via `componentFrameworks.failureSwap`. `gating` is additive on attribution (ignored by old code paths).
|