@miller-tech/uap 2.11.0 → 2.16.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +7 -6
- package/config/capacity-policy.json +15 -3
- package/config/model-profiles/qwen38.json +42 -34
- package/dist/.tsbuildinfo +1 -1
- package/dist/bin/cli.js +39 -0
- package/dist/bin/cli.js.map +1 -1
- package/dist/capacity/policy.d.ts +15 -0
- package/dist/capacity/policy.d.ts.map +1 -1
- package/dist/capacity/policy.js +23 -0
- package/dist/capacity/policy.js.map +1 -1
- package/dist/capacity/probe.d.ts.map +1 -1
- package/dist/capacity/probe.js +6 -1
- package/dist/capacity/probe.js.map +1 -1
- package/dist/cli/deliver.d.ts.map +1 -1
- package/dist/cli/deliver.js +148 -2
- package/dist/cli/deliver.js.map +1 -1
- package/dist/cli/inference.d.ts +13 -0
- package/dist/cli/inference.d.ts.map +1 -0
- package/dist/cli/inference.js +132 -0
- package/dist/cli/inference.js.map +1 -0
- package/dist/cli/loops.d.ts +39 -0
- package/dist/cli/loops.d.ts.map +1 -0
- package/dist/cli/loops.js +154 -0
- package/dist/cli/loops.js.map +1 -0
- package/dist/cli/wizard-config.d.ts.map +1 -1
- package/dist/cli/wizard-config.js +5 -1
- package/dist/cli/wizard-config.js.map +1 -1
- package/dist/cli/worktree.d.ts.map +1 -1
- package/dist/cli/worktree.js +8 -4
- package/dist/cli/worktree.js.map +1 -1
- package/dist/delivery/adversarial-attack.d.ts +99 -0
- package/dist/delivery/adversarial-attack.d.ts.map +1 -0
- package/dist/delivery/adversarial-attack.js +326 -0
- package/dist/delivery/adversarial-attack.js.map +1 -0
- package/dist/delivery/adversarial-gate.d.ts +120 -0
- package/dist/delivery/adversarial-gate.d.ts.map +1 -0
- package/dist/delivery/adversarial-gate.js +259 -0
- package/dist/delivery/adversarial-gate.js.map +1 -0
- package/dist/delivery/adversarial-sanction.d.ts +52 -0
- package/dist/delivery/adversarial-sanction.d.ts.map +1 -0
- package/dist/delivery/adversarial-sanction.js +134 -0
- package/dist/delivery/adversarial-sanction.js.map +1 -0
- package/dist/delivery/agentic-executor.d.ts +69 -22
- package/dist/delivery/agentic-executor.d.ts.map +1 -1
- package/dist/delivery/agentic-executor.js +152 -8
- package/dist/delivery/agentic-executor.js.map +1 -1
- package/dist/delivery/applier.d.ts +8 -0
- package/dist/delivery/applier.d.ts.map +1 -1
- package/dist/delivery/applier.js +3 -1
- package/dist/delivery/applier.js.map +1 -1
- package/dist/delivery/convergence-loop.d.ts +15 -0
- package/dist/delivery/convergence-loop.d.ts.map +1 -1
- package/dist/delivery/convergence-loop.js.map +1 -1
- package/dist/delivery/criteria-lint.d.ts +74 -0
- package/dist/delivery/criteria-lint.d.ts.map +1 -0
- package/dist/delivery/criteria-lint.js +134 -0
- package/dist/delivery/criteria-lint.js.map +1 -0
- package/dist/delivery/deliver-toggle.d.ts +21 -0
- package/dist/delivery/deliver-toggle.d.ts.map +1 -0
- package/dist/delivery/deliver-toggle.js +40 -0
- package/dist/delivery/deliver-toggle.js.map +1 -0
- package/dist/delivery/delivery-evidence.d.ts +109 -0
- package/dist/delivery/delivery-evidence.d.ts.map +1 -0
- package/dist/delivery/delivery-evidence.js +139 -0
- package/dist/delivery/delivery-evidence.js.map +1 -0
- package/dist/delivery/epic-mission.d.ts +5 -1
- package/dist/delivery/epic-mission.d.ts.map +1 -1
- package/dist/delivery/epic-mission.js +34 -6
- package/dist/delivery/epic-mission.js.map +1 -1
- package/dist/delivery/execution-gate.d.ts.map +1 -1
- package/dist/delivery/execution-gate.js +90 -7
- package/dist/delivery/execution-gate.js.map +1 -1
- package/dist/delivery/index.d.ts +9 -1
- package/dist/delivery/index.d.ts.map +1 -1
- package/dist/delivery/index.js +9 -1
- package/dist/delivery/index.js.map +1 -1
- package/dist/delivery/journey-depth.d.ts +61 -0
- package/dist/delivery/journey-depth.d.ts.map +1 -0
- package/dist/delivery/journey-depth.js +91 -0
- package/dist/delivery/journey-depth.js.map +1 -0
- package/dist/delivery/master-pipeline.d.ts +50 -0
- package/dist/delivery/master-pipeline.d.ts.map +1 -0
- package/dist/delivery/master-pipeline.js +52 -0
- package/dist/delivery/master-pipeline.js.map +1 -0
- package/dist/delivery/mission-acceptance.d.ts.map +1 -1
- package/dist/delivery/mission-acceptance.js +54 -2
- package/dist/delivery/mission-acceptance.js.map +1 -1
- package/dist/delivery/orchestrated-mission.d.ts +4 -1
- package/dist/delivery/orchestrated-mission.d.ts.map +1 -1
- package/dist/delivery/orchestrated-mission.js +18 -9
- package/dist/delivery/orchestrated-mission.js.map +1 -1
- package/dist/delivery/plan-check.d.ts +1 -0
- package/dist/delivery/plan-check.d.ts.map +1 -1
- package/dist/delivery/plan-check.js +48 -0
- package/dist/delivery/plan-check.js.map +1 -1
- package/dist/delivery/task-orchestrator.d.ts +16 -9
- package/dist/delivery/task-orchestrator.d.ts.map +1 -1
- package/dist/delivery/task-orchestrator.js +4 -2
- package/dist/delivery/task-orchestrator.js.map +1 -1
- package/dist/delivery/task-workspace.d.ts +35 -5
- package/dist/delivery/task-workspace.d.ts.map +1 -1
- package/dist/delivery/task-workspace.js +53 -7
- package/dist/delivery/task-workspace.js.map +1 -1
- package/dist/delivery/test-oracle-additive.d.ts +26 -6
- package/dist/delivery/test-oracle-additive.d.ts.map +1 -1
- package/dist/delivery/test-oracle-additive.js +40 -15
- package/dist/delivery/test-oracle-additive.js.map +1 -1
- package/dist/delivery/user-validation.d.ts +14 -0
- package/dist/delivery/user-validation.d.ts.map +1 -1
- package/dist/delivery/user-validation.js +103 -9
- package/dist/delivery/user-validation.js.map +1 -1
- package/dist/delivery/verifier-ladder.d.ts.map +1 -1
- package/dist/delivery/verifier-ladder.js +24 -6
- package/dist/delivery/verifier-ladder.js.map +1 -1
- package/dist/inference/analysis.d.ts +138 -0
- package/dist/inference/analysis.d.ts.map +1 -0
- package/dist/inference/analysis.js +243 -0
- package/dist/inference/analysis.js.map +1 -0
- package/dist/inference/probe.d.ts +85 -0
- package/dist/inference/probe.d.ts.map +1 -0
- package/dist/inference/probe.js +403 -0
- package/dist/inference/probe.js.map +1 -0
- package/dist/models/types.d.ts.map +1 -1
- package/dist/models/types.js +45 -18
- package/dist/models/types.js.map +1 -1
- package/dist/self-tuning/profiles/qwen38.d.ts +17 -8
- package/dist/self-tuning/profiles/qwen38.d.ts.map +1 -1
- package/dist/self-tuning/profiles/qwen38.js +21 -11
- package/dist/self-tuning/profiles/qwen38.js.map +1 -1
- package/dist/utils/model-slots.d.ts +11 -5
- package/dist/utils/model-slots.d.ts.map +1 -1
- package/dist/utils/model-slots.js +11 -5
- package/dist/utils/model-slots.js.map +1 -1
- package/docs/INDEX.md +7 -1
- package/docs/guides/CAPACITY_POLICY.md +29 -4
- package/docs/guides/INFERENCE_HEALTH.md +184 -0
- package/docs/guides/PROXY.md +1 -0
- package/docs/guides/QWEN38_EXL3.md +268 -0
- package/docs/plans/hlg-master-architecture-uplift-2026-09-24.md +67 -0
- package/docs/plans/system1-uplift-2026-09-18.md +1 -2
- package/docs/reference/CONFIGURATION.md +10 -1
- package/docs/reference/CONFIGURATION_REFERENCE.md +14 -0
- package/docs/reference/automode-risk.md +1 -1
- package/docs/reference/evidence-bound-ship.md +3 -2
- package/docs/reference/semantic-supervisor.md +3 -3
- package/docs/reference/system1-classifier.md +4 -4
- package/package.json +2 -2
- package/scripts/lib/llama-upstream.sh +7 -1
- package/src/policies/enforcers/__pycache__/_common.cpython-312.pyc +0 -0
- package/src/policies/enforcers/__pycache__/expert_review_required.cpython-312.pyc +0 -0
- package/src/policies/enforcers/__pycache__/quality_metrics_gate.cpython-312.pyc +0 -0
- package/src/policies/enforcers/expert_review_required.py +1 -4
- package/templates/hooks/__pycache__/deliver_autoroute.cpython-312.pyc +0 -0
- package/tools/agents/scripts/__pycache__/toolcall_path_normalizer.cpython-312.pyc +0 -0
- package/tools/agents/scripts/anthropic_proxy.py +374 -16
- package/tools/agents/scripts/confidence_escalation.py +4 -2
- package/tools/agents/scripts/project_telemetry.py +1 -0
- package/tools/agents/scripts/tool_risk.py +1 -0
- package/tools/agents/scripts/tool_risk_calibrate.py +1 -0
- package/tools/agents/scripts/toolcall_path_normalizer.py +1 -0
- package/tools/agents/tests/test_anthropic_proxy_streaming.py +52 -1
- package/tools/agents/tests/test_confidence_escalation.py +3 -2
- package/tools/agents/tests/test_count_tokens_scale_per_session.py +126 -0
- package/tools/agents/tests/test_expert_review_pr_scope.py +10 -11
- package/tools/agents/tests/test_lockstep_escalation.py +324 -0
- package/tools/agents/tests/test_thinking_template_kwargs.py +10 -4
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"qwen38.d.ts","sourceRoot":"","sources":["../../../src/self-tuning/profiles/qwen38.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"qwen38.d.ts","sourceRoot":"","sources":["../../../src/self-tuning/profiles/qwen38.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA4BG;AAEH,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAE9C,eAAO,MAAM,cAAc,EAAE,UAyB5B,CAAC"}
|
|
@@ -3,14 +3,23 @@
|
|
|
3
3
|
* raises toward Opus (successor to the qwen3.6-a3b seed in qwen36.ts).
|
|
4
4
|
*
|
|
5
5
|
* Carried over from QWEN36_PROFILE unchanged EXCEPT for concurrency, which is
|
|
6
|
-
* not a preference here but a property of the server
|
|
7
|
-
*
|
|
8
|
-
*
|
|
9
|
-
*
|
|
10
|
-
*
|
|
11
|
-
*
|
|
12
|
-
*
|
|
13
|
-
*
|
|
6
|
+
* not a preference here but a property of the server.
|
|
7
|
+
*
|
|
8
|
+
* CORRECTED 2026-09-21. This header said the model was served by
|
|
9
|
+
* `ninfer-serve --max-concurrency 1` and that there was ONE rail. That engine
|
|
10
|
+
* is not what runs: the backend is buun-llama-cpp behind
|
|
11
|
+
* uap-gsq-rco-server.service with `-np 2`, and the proxy was raised to match
|
|
12
|
+
* (PROXY_CONCURRENCY_LIMIT / UAP_MODEL_SLOTS / PROXY_SESSION_ADMISSION_LIMIT
|
|
13
|
+
* all 2). The reasoning below still holds, only the number changed: slots must
|
|
14
|
+
* track what the server and proxy will actually run in parallel. Declaring
|
|
15
|
+
* MORE than that does not get more in flight — it gets one running and the
|
|
16
|
+
* rest queued, the extra slots buy latency and a longer wedge window rather
|
|
17
|
+
* than throughput, and the adaptive controller then reads the queueing delay
|
|
18
|
+
* as backpressure and throttles a server that was never saturated.
|
|
19
|
+
*
|
|
20
|
+
* Whether the second rail EARNS anything is a separate question from whether
|
|
21
|
+
* it exists: check llamacpp:n_busy_slots_per_decode in /metrics (>1 means
|
|
22
|
+
* requests genuinely overlapped), or `uap inference health`.
|
|
14
23
|
*
|
|
15
24
|
* Everything else is deliberately identical to the qwen36 seed: nothing has
|
|
16
25
|
* been re-measured on 3.8 yet, and the tuning loop's job is to beat this seed,
|
|
@@ -29,9 +38,10 @@ export const QWEN38_PROFILE = {
|
|
|
29
38
|
'handsfree.enabled': true,
|
|
30
39
|
'handsfree.intensity': 'aggressive',
|
|
31
40
|
UAP_HANDSFREE_STAGNATION_LIMIT: 6,
|
|
32
|
-
// Concurrency:
|
|
33
|
-
//
|
|
34
|
-
|
|
41
|
+
// Concurrency: TWO rails (see the header) — matches `-np 2` on the server
|
|
42
|
+
// and the proxy's concurrency/admission limits. Adaptive stays on so a
|
|
43
|
+
// genuinely overloaded server still backs off.
|
|
44
|
+
'modelConcurrency.slots': 2,
|
|
35
45
|
'modelConcurrency.adaptive': true,
|
|
36
46
|
// Memory: bigger short-term window + pattern RAG compensate for weak planning.
|
|
37
47
|
'memory.shortTerm.maxEntries': 80,
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"qwen38.js","sourceRoot":"","sources":["../../../src/self-tuning/profiles/qwen38.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"qwen38.js","sourceRoot":"","sources":["../../../src/self-tuning/profiles/qwen38.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA4BG;AAIH,MAAM,CAAC,MAAM,cAAc,GAAe;IACxC,yEAAyE;IACzE,iBAAiB,EAAE,IAAI;IACvB,gBAAgB,EAAE,QAAQ;IAC1B,6BAA6B,EAAE,GAAG;IAClC,iBAAiB,EAAE,CAAC;IACpB,wBAAwB,EAAE,KAAK;IAC/B,yEAAyE;IACzE,mBAAmB,EAAE,IAAI;IACzB,qBAAqB,EAAE,YAAY;IACnC,8BAA8B,EAAE,CAAC;IACjC,0EAA0E;IAC1E,uEAAuE;IACvE,+CAA+C;IAC/C,wBAAwB,EAAE,CAAC;IAC3B,2BAA2B,EAAE,IAAI;IACjC,+EAA+E;IAC/E,6BAA6B,EAAE,EAAE;IACjC,2BAA2B,EAAE,IAAI;IACjC,uEAAuE;IACvE,wBAAwB,EAAE,IAAI;IAC9B,kEAAkE;IAClE,iCAAiC,EAAE,EAAE;IACrC,kBAAkB,EAAE,IAAI;IACxB,iBAAiB,EAAE,IAAI;CACxB,CAAC"}
|
|
@@ -14,13 +14,19 @@ export declare function configuredSlots(cwd?: string): number | undefined;
|
|
|
14
14
|
* - the endpoint is unreachable -> we know nothing; 2 is a fair guess
|
|
15
15
|
* - the endpoint answered 404/405 -> this is NOT llama.cpp
|
|
16
16
|
*
|
|
17
|
-
*
|
|
18
|
-
*
|
|
19
|
-
*
|
|
20
|
-
*
|
|
21
|
-
*
|
|
17
|
+
* A server that ANSWERS 404/405 is telling us something: it is not llama.cpp,
|
|
18
|
+
* and engines without a `--parallel` equivalent typically run one request and
|
|
19
|
+
* queue the rest. Guessing 2 there does not double throughput — it puts a
|
|
20
|
+
* second request in a queue and then lets the adaptive controller read the
|
|
21
|
+
* queueing delay as backpressure on a server that was never saturated.
|
|
22
22
|
*
|
|
23
23
|
* So a live server that denies knowing about `/slots` reports 1, not "unknown".
|
|
24
|
+
*
|
|
25
|
+
* NOTE (2026-09-21): this used to cite the local backend as the motivating
|
|
26
|
+
* example — "ninfer-serve, which serves no /slots at all". That is no longer
|
|
27
|
+
* the case: the local backend is buun-llama-cpp and /slots answers with two
|
|
28
|
+
* rails, so the 404 branch is now the FOREIGN-engine path, not the normal one.
|
|
29
|
+
* The logic is unchanged and still correct; only the example was stale.
|
|
24
30
|
*/
|
|
25
31
|
export declare function probeSlots(base: string, timeoutMs?: number): Promise<number | null>;
|
|
26
32
|
/**
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"model-slots.d.ts","sourceRoot":"","sources":["../../src/utils/model-slots.ts"],"names":[],"mappings":"AAgBA,eAAO,MAAM,aAAa,IAAI,CAAC;AAY/B,MAAM,MAAM,YAAY,GAAG,KAAK,GAAG,QAAQ,GAAG,OAAO,GAAG,SAAS,CAAC;AAoBlE,4EAA4E;AAC5E,wBAAgB,aAAa,CAAC,GAAG,CAAC,EAAE,MAAM,GAAG,MAAM,CAclD;AAED,wBAAgB,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,GAAG,MAAM,CAE7C;AAED,wDAAwD;AACxD,wBAAgB,eAAe,CAAC,GAAG,CAAC,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAEhE;AAED
|
|
1
|
+
{"version":3,"file":"model-slots.d.ts","sourceRoot":"","sources":["../../src/utils/model-slots.ts"],"names":[],"mappings":"AAgBA,eAAO,MAAM,aAAa,IAAI,CAAC;AAY/B,MAAM,MAAM,YAAY,GAAG,KAAK,GAAG,QAAQ,GAAG,OAAO,GAAG,SAAS,CAAC;AAoBlE,4EAA4E;AAC5E,wBAAgB,aAAa,CAAC,GAAG,CAAC,EAAE,MAAM,GAAG,MAAM,CAclD;AAED,wBAAgB,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,GAAG,MAAM,CAE7C;AAED,wDAAwD;AACxD,wBAAgB,eAAe,CAAC,GAAG,CAAC,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAEhE;AAED;;;;;;;;;;;;;;;;;;;;;;GAsBG;AACH,wBAAsB,UAAU,CAAC,IAAI,EAAE,MAAM,EAAE,SAAS,SAAO,GAAG,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC,CAuBvF;AAMD;;;;GAIG;AACH,wBAAsB,kBAAkB,CACtC,GAAG,CAAC,EAAE,MAAM,EACZ,IAAI,GAAE;IAAE,KAAK,CAAC,EAAE,OAAO,CAAC;IAAC,KAAK,CAAC,EAAE,OAAO,CAAA;CAAO,GAC9C,OAAO,CAAC;IAAE,MAAM,EAAE,MAAM,CAAC;IAAC,KAAK,EAAE,MAAM,CAAC;IAAC,MAAM,EAAE,YAAY,CAAA;CAAE,CAAC,CA6BlE;AAED,gEAAgE;AAChE,wBAAsB,mBAAmB,CAAC,GAAG,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CAEvE;AAED;;;;GAIG;AACH,wBAAgB,sBAAsB,CAAC,GAAG,CAAC,EAAE,MAAM,GAAG,MAAM,CAK3D;AAED,wBAAgB,mBAAmB,IAAI,IAAI,CAE1C"}
|
|
@@ -63,13 +63,19 @@ export function configuredSlots(cwd) {
|
|
|
63
63
|
* - the endpoint is unreachable -> we know nothing; 2 is a fair guess
|
|
64
64
|
* - the endpoint answered 404/405 -> this is NOT llama.cpp
|
|
65
65
|
*
|
|
66
|
-
*
|
|
67
|
-
*
|
|
68
|
-
*
|
|
69
|
-
*
|
|
70
|
-
*
|
|
66
|
+
* A server that ANSWERS 404/405 is telling us something: it is not llama.cpp,
|
|
67
|
+
* and engines without a `--parallel` equivalent typically run one request and
|
|
68
|
+
* queue the rest. Guessing 2 there does not double throughput — it puts a
|
|
69
|
+
* second request in a queue and then lets the adaptive controller read the
|
|
70
|
+
* queueing delay as backpressure on a server that was never saturated.
|
|
71
71
|
*
|
|
72
72
|
* So a live server that denies knowing about `/slots` reports 1, not "unknown".
|
|
73
|
+
*
|
|
74
|
+
* NOTE (2026-09-21): this used to cite the local backend as the motivating
|
|
75
|
+
* example — "ninfer-serve, which serves no /slots at all". That is no longer
|
|
76
|
+
* the case: the local backend is buun-llama-cpp and /slots answers with two
|
|
77
|
+
* rails, so the 404 branch is now the FOREIGN-engine path, not the normal one.
|
|
78
|
+
* The logic is unchanged and still correct; only the example was stale.
|
|
73
79
|
*/
|
|
74
80
|
export async function probeSlots(base, timeoutMs = 1500) {
|
|
75
81
|
try {
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"model-slots.js","sourceRoot":"","sources":["../../src/utils/model-slots.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AACH,OAAO,EAAE,gBAAgB,EAAE,MAAM,oBAAoB,CAAC;AACtD,OAAO,EAAE,uBAAuB,EAAE,MAAM,sBAAsB,CAAC;AAE/D,MAAM,CAAC,MAAM,aAAa,GAAG,CAAC,CAAC,CAAC,iDAAiD;AACjF,MAAM,YAAY,GAAG,MAAM,CAAC;AAS5B,IAAI,MAAM,GAA+E,IAAI,CAAC;AAI9F,SAAS,GAAG,CAAC,GAAY;IACvB,IAAI,CAAC;QACH,MAAM,GAAG,GAAG,gBAAgB,CAAC,GAAG,IAAI,OAAO,CAAC,GAAG,EAAE,CAEzC,CAAC;QACT,OAAO,GAAG,EAAE,gBAAgB,IAAI,EAAE,CAAC;IACrC,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,EAAE,CAAC;IACZ,CAAC;AACH,CAAC;AAED,SAAS,MAAM,CAAC,IAAY;IAC1B,MAAM,CAAC,GAAG,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;IAC5B,IAAI,CAAC,CAAC;QAAE,OAAO,SAAS,CAAC;IACzB,MAAM,CAAC,GAAG,QAAQ,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;IAC1B,OAAO,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;AACrD,CAAC;AAED,4EAA4E;AAC5E,MAAM,UAAU,aAAa,CAAC,GAAY;IACxC,0EAA0E;IAC1E,uEAAuE;IACvE,8EAA8E;IAC9E,+EAA+E;IAC/E,oDAAoD;IACpD,MAAM,GAAG,GACP,OAAO,CAAC,GAAG,CAAC,kBAAkB;QAC9B,OAAO,CAAC,GAAG,CAAC,sBAAsB;QAClC,GAAG,CAAC,GAAG,CAAC,CAAC,QAAQ;QACjB,OAAO,CAAC,GAAG,CAAC,cAAc;QAC1B,uBAAuB,EAAE,CAAC,CAAC,CAAC;QAC5B,uBAAuB,CAAC;IAC1B,OAAO,GAAG,CAAC,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;AACzD,CAAC;AAED,MAAM,UAAU,QAAQ,CAAC,GAAY;IACnC,OAAO,MAAM,CAAC,yBAAyB,CAAC,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC,QAAQ,IAAI,CAAC,CAAC;AACrE,CAAC;AAED,wDAAwD;AACxD,MAAM,UAAU,eAAe,CAAC,GAAY;IAC1C,OAAO,MAAM,CAAC,iBAAiB,CAAC,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC;AACrD,CAAC;AAED
|
|
1
|
+
{"version":3,"file":"model-slots.js","sourceRoot":"","sources":["../../src/utils/model-slots.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AACH,OAAO,EAAE,gBAAgB,EAAE,MAAM,oBAAoB,CAAC;AACtD,OAAO,EAAE,uBAAuB,EAAE,MAAM,sBAAsB,CAAC;AAE/D,MAAM,CAAC,MAAM,aAAa,GAAG,CAAC,CAAC,CAAC,iDAAiD;AACjF,MAAM,YAAY,GAAG,MAAM,CAAC;AAS5B,IAAI,MAAM,GAA+E,IAAI,CAAC;AAI9F,SAAS,GAAG,CAAC,GAAY;IACvB,IAAI,CAAC;QACH,MAAM,GAAG,GAAG,gBAAgB,CAAC,GAAG,IAAI,OAAO,CAAC,GAAG,EAAE,CAEzC,CAAC;QACT,OAAO,GAAG,EAAE,gBAAgB,IAAI,EAAE,CAAC;IACrC,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,EAAE,CAAC;IACZ,CAAC;AACH,CAAC;AAED,SAAS,MAAM,CAAC,IAAY;IAC1B,MAAM,CAAC,GAAG,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;IAC5B,IAAI,CAAC,CAAC;QAAE,OAAO,SAAS,CAAC;IACzB,MAAM,CAAC,GAAG,QAAQ,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;IAC1B,OAAO,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;AACrD,CAAC;AAED,4EAA4E;AAC5E,MAAM,UAAU,aAAa,CAAC,GAAY;IACxC,0EAA0E;IAC1E,uEAAuE;IACvE,8EAA8E;IAC9E,+EAA+E;IAC/E,oDAAoD;IACpD,MAAM,GAAG,GACP,OAAO,CAAC,GAAG,CAAC,kBAAkB;QAC9B,OAAO,CAAC,GAAG,CAAC,sBAAsB;QAClC,GAAG,CAAC,GAAG,CAAC,CAAC,QAAQ;QACjB,OAAO,CAAC,GAAG,CAAC,cAAc;QAC1B,uBAAuB,EAAE,CAAC,CAAC,CAAC;QAC5B,uBAAuB,CAAC;IAC1B,OAAO,GAAG,CAAC,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;AACzD,CAAC;AAED,MAAM,UAAU,QAAQ,CAAC,GAAY;IACnC,OAAO,MAAM,CAAC,yBAAyB,CAAC,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC,QAAQ,IAAI,CAAC,CAAC;AACrE,CAAC;AAED,wDAAwD;AACxD,MAAM,UAAU,eAAe,CAAC,GAAY;IAC1C,OAAO,MAAM,CAAC,iBAAiB,CAAC,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC;AACrD,CAAC;AAED;;;;;;;;;;;;;;;;;;;;;;GAsBG;AACH,MAAM,CAAC,KAAK,UAAU,UAAU,CAAC,IAAY,EAAE,SAAS,GAAG,IAAI;IAC7D,IAAI,CAAC;QACH,MAAM,IAAI,GAAG,IAAI,eAAe,EAAE,CAAC;QACnC,MAAM,CAAC,GAAG,UAAU,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,KAAK,EAAE,EAAE,SAAS,CAAC,CAAC;QACpD,IAAI,CAAC;YACH,MAAM,GAAG,GAAG,MAAM,KAAK,CAAC,GAAG,IAAI,QAAQ,EAAE,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,CAAC,CAAC;YAClE,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,CAAC;gBACZ,+DAA+D;gBAC/D,sEAAsE;gBACtE,sEAAsE;gBACtE,oEAAoE;gBACpE,wBAAwB;gBACxB,OAAO,GAAG,CAAC,MAAM,KAAK,GAAG,IAAI,GAAG,CAAC,MAAM,KAAK,GAAG,IAAI,GAAG,CAAC,MAAM,KAAK,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;YACnF,CAAC;YACD,MAAM,IAAI,GAAG,CAAC,MAAM,GAAG,CAAC,IAAI,EAAE,CAAY,CAAC;YAC3C,IAAI,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,IAAI,CAAC,MAAM,GAAG,CAAC;gBAAE,OAAO,IAAI,CAAC,MAAM,CAAC;YAC/D,OAAO,IAAI,CAAC;QACd,CAAC;gBAAS,CAAC;YACT,YAAY,CAAC,CAAC,CAAC,CAAC;QAClB,CAAC;IACH,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,IAAI,CAAC;IACd,CAAC;AACH,CAAC;AAED,SAAS,aAAa,CAAC,KAAa,EAAE,GAAY;IAChD,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,KAAK,GAAG,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC;AAC5C,CAAC;AAED;;;;GAIG;AACH,MAAM,CAAC,KAAK,UAAU,kBAAkB,CACtC,GAAY,EACZ,OAA6C,EAAE;IAE/C,IAAI,CAAC,IAAI,CAAC,KAAK,IAAI,MAAM,IAAI,IAAI,CAAC,GAAG,EAAE,GAAG,MAAM,CAAC,EAAE,GAAG,YAAY,EAAE,CAAC;QACnE,OAAO,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,KAAK,EAAE,MAAM,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC;IAC/E,CAAC;IACD,IAAI,KAAa,CAAC;IAClB,IAAI,MAAoB,CAAC;IACzB,MAAM,QAAQ,GAAG,eAAe,CAAC,GAAG,CAAC,CAAC;IACtC,IAAI,MAAM,CAAC,iBAAiB,CAAC,KAAK,SAAS,EAAE,CAAC;QAC5C,KAAK,GAAG,QAAS,CAAC;QAClB,MAAM,GAAG,KAAK,CAAC;IACjB,CAAC;SAAM,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;QAClC,KAAK,GAAG,QAAQ,CAAC;QACjB,MAAM,GAAG,QAAQ,CAAC;IACpB,CAAC;SAAM,IAAI,IAAI,CAAC,KAAK,KAAK,KAAK,EAAE,CAAC;QAChC,MAAM,MAAM,GAAG,MAAM,UAAU,CAAC,aAAa,CAAC,GAAG,CAAC,CAAC,CAAC;QACpD,IAAI,MAAM,KAAK,IAAI,EAAE,CAAC;YACpB,KAAK,GAAG,MAAM,CAAC;YACf,MAAM,GAAG,OAAO,CAAC;QACnB,CAAC;aAAM,CAAC;YACN,KAAK,GAAG,aAAa,CAAC;YACtB,MAAM,GAAG,SAAS,CAAC;QACrB,CAAC;IACH,CAAC;SAAM,CAAC;QACN,KAAK,GAAG,aAAa,CAAC;QACtB,MAAM,GAAG,SAAS,CAAC;IACrB,CAAC;IACD,MAAM,MAAM,GAAG,aAAa,CAAC,KAAK,EAAE,GAAG,CAAC,CAAC;IACzC,MAAM,GAAG,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,EAAE,EAAE,IAAI,CAAC,GAAG,EAAE,EAAE,CAAC;IACnD,OAAO,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,CAAC;AACnC,CAAC;AAED,gEAAgE;AAChE,MAAM,CAAC,KAAK,UAAU,mBAAmB,CAAC,GAAY;IACpD,OAAO,CAAC,MAAM,kBAAkB,CAAC,GAAG,EAAE,EAAE,KAAK,EAAE,IAAI,EAAE,KAAK,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC;AAC9E,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,sBAAsB,CAAC,GAAY;IACjD,MAAM,QAAQ,GAAG,eAAe,CAAC,GAAG,CAAC,CAAC;IACtC,IAAI,QAAQ,KAAK,SAAS;QAAE,OAAO,aAAa,CAAC,QAAQ,EAAE,GAAG,CAAC,CAAC;IAChE,IAAI,MAAM,IAAI,IAAI,CAAC,GAAG,EAAE,GAAG,MAAM,CAAC,EAAE,GAAG,YAAY;QAAE,OAAO,MAAM,CAAC,MAAM,CAAC;IAC1E,OAAO,aAAa,CAAC,aAAa,EAAE,GAAG,CAAC,CAAC;AAC3C,CAAC;AAED,MAAM,UAAU,mBAAmB;IACjC,MAAM,GAAG,IAAI,CAAC;AAChB,CAAC"}
|
package/docs/INDEX.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# UAP Documentation
|
|
2
2
|
|
|
3
|
-
The complete documentation for the **Universal Agent Protocol** (`@miller-tech/uap`
|
|
3
|
+
The complete documentation for the **Universal Agent Protocol** (`@miller-tech/uap` v2.11.0) — the discipline layer that turns a talented-but-unreliable AI coding agent into a dependable member of your software delivery line. 367 TypeScript modules across 26 subsystems, 459 vitest suites plus a ~1,200-test Python enforcer/proxy suite, 60+ CLI command registrations, 32 executable policy enforcers, 9 supported agent harnesses.
|
|
4
4
|
|
|
5
5
|
> **Reviewing the system?** The reverse-engineered, code-verified reference set lives in
|
|
6
6
|
> [`documentation/`](../documentation/architecture.md): architecture, flows, permissions
|
|
@@ -25,6 +25,7 @@ UAP is organized like a delivery line. If you know which part of the pipeline yo
|
|
|
25
25
|
| **Coordination** — many workers, one floor | Parallel agents colliding, stale branches overwriting landed work | [Parallel Agents](guides/PARALLEL_AGENTS.md) · [Coordination](guides/COORDINATION.md) · [Deploy Batching](guides/DEPLOY_BATCHING.md) · [Semantic Supervisor](reference/semantic-supervisor.md) (`uap supervise`) |
|
|
26
26
|
| **Shipping** — out the door safely | Regressions, red CI, skipped bumps | [Worktree Workflow](guides/WORKTREE_WORKFLOW.md) · [Policies](guides/POLICIES.md) |
|
|
27
27
|
| **Capacity** — services inside their budgets | Silent OOMs, crash loops, headroom loss | [Capacity Policy](guides/CAPACITY_POLICY.md) (`uap doctor`) |
|
|
28
|
+
| **Inference** — the model still doing useful work | Throughput decaying with uptime, starved checkpoints, KV at its quality floor | [Inference Health](guides/INFERENCE_HEALTH.md) (`uap inference health`) |
|
|
28
29
|
| **Feedback** — the floor learns | The same mistake every session | [Memory](guides/MEMORY.md) · [Self-Harness](design/SELF_HARNESS.md) · [Self-Tuning](guides/SELF_TUNING.md) |
|
|
29
30
|
| *Cross-cutting* — the whole line | Ignored rules, bloated context | [Policies](guides/POLICIES.md) · [MCP Router](guides/MCP_ROUTER.md) |
|
|
30
31
|
|
|
@@ -78,6 +79,7 @@ Full map: **[The UAP Delivery Pipeline](guides/DELIVERY_PIPELINE.md)**.
|
|
|
78
79
|
| [Self-Harness](design/SELF_HARNESS.md) | Self-improving harness (shipped as `uap self-harness`): autonomous mine→propose→validate loop over a bounded Mod DSL; cross-model transfer, online mining |
|
|
79
80
|
| [LLM Self-Tuning (analysis)](design/LLM_SELF_TUNING_ANALYSIS.md) | The design behind `uap tune`: quality signal, LLM-guided + Gaussian-process flag search, closed-loop validation, model profiles, real-time adaptation |
|
|
80
81
|
| [Product Naming (analysis)](design/PRODUCT_NAMING.md) | UAP rename candidates: availability sweeps (npm/PyPI/domains), AI-space collision checks, railway-theme shortlist, recommendation. No decision committed; availability data is point-in-time |
|
|
82
|
+
| [ADRs](architecture/adr/) | Architecture decision records: [0001 AutoMode risk floor](architecture/adr/0001-automode-risk-floor.md) (Python heuristic scorer ahead of the TS SYS1 head), [0002 Evidence-bound ship](architecture/adr/0002-evidence-bound-ship.md) (gate evidence bound to the candidate SHA) |
|
|
81
83
|
|
|
82
84
|
## Reference
|
|
83
85
|
|
|
@@ -92,6 +94,10 @@ Full map: **[The UAP Delivery Pipeline](guides/DELIVERY_PIPELINE.md)**.
|
|
|
92
94
|
| [Platforms](reference/PLATFORMS.md) | The 9 supported harnesses + support matrix |
|
|
93
95
|
| [Review Pre-Pass](reference/review-prepass.md) | Deterministic ruleset scanner (`uap review prepass`) that feeds line-anchored findings to the parallel review protocol |
|
|
94
96
|
| [Visual Captures](reference/visual-captures.md) | Before/after capture binding for UI diffs (`uap review captures`) — ship-time gate 7 enforcement |
|
|
97
|
+
| [System-1 Classifier](reference/system1-classifier.md) | `uap classify` — the local, airgap-pure assessment backend (noul/score/choice questions) behind the Wave 1 consumers; thresholds ship as policy config |
|
|
98
|
+
| [Semantic Supervisor](reference/semantic-supervisor.md) | `uap supervise` — watchdog loop for `uap deliver` missions; nine assessment dimensions, deterministic safety-first policy, cooperative STOP only |
|
|
99
|
+
| [AutoMode Risk](reference/automode-risk.md) | Pre-execution tool-call risk classification (`tools/agents/scripts/tool_risk.py`) — advisory only, ahead of the deterministic enforcer floor |
|
|
100
|
+
| [Evidence-Bound Ship](reference/evidence-bound-ship.md) | Ship actions blocked unless `.uap/evidence/<head-sha>.json` proves the gates passed for the exact commit being shipped |
|
|
95
101
|
| [Configuration](reference/CONFIGURATION.md) | All config files and env vars |
|
|
96
102
|
| [**Configuration Reference**](reference/CONFIGURATION_REFERENCE.md) | Every setting `uap config` exposes — what it does, default, and a recommendation (generated from the registry); `uap config list/explain/set/doctor/wizard` ⭐ |
|
|
97
103
|
| [Database Schema](reference/DATABASE_SCHEMA.md) | SQLite databases + Qdrant collections |
|
|
@@ -9,7 +9,8 @@ This generalizes the 2026-09-18 llama.cpp OOM fix: four SIGABRT crashes
|
|
|
9
9
|
happened because a KV-cache budget was resolved against a GPU that also runs
|
|
10
10
|
a desktop, and nothing declared that relationship. The fix held, but the
|
|
11
11
|
*discipline* (declare budget + headroom + restart baseline, watch for
|
|
12
|
-
violations) lived in a commit message
|
|
12
|
+
violations) lived only in a commit message; the capacity policy moves it
|
|
13
|
+
into data.
|
|
13
14
|
|
|
14
15
|
## Policy file
|
|
15
16
|
|
|
@@ -41,12 +42,36 @@ the project, then `~/.config/uap/capacity-policy.json`.
|
|
|
41
42
|
`MemoryCurrent`. Declaring it when systemd cannot report memory
|
|
42
43
|
(cgroup accounting off) is RED — a declared budget that cannot be
|
|
43
44
|
verified is not healthy.
|
|
45
|
+
- `budget.execStartMustContain` **is** enforced: literal strings that must
|
|
46
|
+
all appear in the unit's probed `ExecStart`, matched as plain substrings
|
|
47
|
+
so ordering and surrounding flags do not matter.
|
|
48
|
+
|
|
49
|
+
```json
|
|
50
|
+
"budget": {
|
|
51
|
+
"vramMiB": 5120,
|
|
52
|
+
"note": "--vbr-vram 5120M, -c 229376, -np 2",
|
|
53
|
+
"execStartMustContain": ["-c 229376", "-np 2", "--vbr-vram 5120M"]
|
|
54
|
+
}
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
It exists because `note` is prose and drifts silently. On 2026-09-20 a
|
|
58
|
+
note still read `-np 1` while the unit had been running `-np 2` for hours,
|
|
59
|
+
and the doctor stayed GREEN throughout — it probed only liveness
|
|
60
|
+
properties and never looked at `ExecStart`. Anything load-bearing enough
|
|
61
|
+
to write in the note belongs here too, where a mismatch is RED. As with
|
|
62
|
+
the other budgets, *unverifiable* is also RED: if `systemctl` reports no
|
|
63
|
+
`ExecStart`, the declaration is not quietly assumed satisfied.
|
|
44
64
|
- **headroom.gpuMinFreeMiB** — free VRAM the *host* must keep; below it the
|
|
45
65
|
service is in OOM territory even while running.
|
|
46
66
|
- **restartBudget** — `knownRestarts` is the accepted historical baseline
|
|
47
67
|
(e.g. crashes already fixed); `allowedNew` new crashes beyond that turn the
|
|
48
68
|
service RED. Zero is the right default after a fix lands.
|
|
49
69
|
|
|
70
|
+
`uap doctor` answers whether a service is up and inside its declared budget.
|
|
71
|
+
Whether the local model is still doing useful *work* — throughput can decay
|
|
72
|
+
~8x over a long-lived process while every one of these checks stays GREEN — is
|
|
73
|
+
[`uap inference health`](INFERENCE_HEALTH.md).
|
|
74
|
+
|
|
50
75
|
## Health semantics
|
|
51
76
|
|
|
52
77
|
| Health | Meaning |
|
|
@@ -73,9 +98,9 @@ elsewhere should ship their own `config/capacity-policy.json` (project) or
|
|
|
73
98
|
`~/.config/uap/capacity-policy.json` (operator); the resolution order above
|
|
74
99
|
makes the committed file the fallback only inside this repo.
|
|
75
100
|
|
|
76
|
-
`uap doctor --strict` is the
|
|
77
|
-
|
|
78
|
-
degraded host.
|
|
101
|
+
`uap doctor --strict` is the health source the monitor service should
|
|
102
|
+
consult before restarting anything, and CI can use it to refuse a deploy
|
|
103
|
+
into a degraded host.
|
|
79
104
|
|
|
80
105
|
## Decisions
|
|
81
106
|
|
|
@@ -0,0 +1,184 @@
|
|
|
1
|
+
# Inference Health (`uap inference health`)
|
|
2
|
+
|
|
3
|
+
`uap doctor` answers *"is the service up and inside its declared budget?"*
|
|
4
|
+
This answers a different question: **is it still doing useful work?**
|
|
5
|
+
|
|
6
|
+
The distinction is not academic. On 2026-09-21 a local llama.cpp server
|
|
7
|
+
reported `active (running)` and GREEN from `uap doctor` for eighteen hours
|
|
8
|
+
while its prefill throughput fell roughly eight-fold and five client requests
|
|
9
|
+
blew a 1800-second deadline overnight. Nothing was down. Liveness checks had
|
|
10
|
+
nothing to report, and the only visible symptom — slow turns — looked like the
|
|
11
|
+
model being the model.
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
uap inference health # the running process
|
|
15
|
+
uap inference health --json # for monitors and dashboards
|
|
16
|
+
uap inference health --strict # exit 1 on WARN or RED
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
## What it looks at
|
|
20
|
+
|
|
21
|
+
| Check | Code | Why it exists |
|
|
22
|
+
| --- | --- | --- |
|
|
23
|
+
| Prefill throughput decay | `prefill-decay` | A long-lived process can get slower without failing. Only visible when samples are bucketed by prompt size. |
|
|
24
|
+
| Checkpoint starvation | `checkpoint-starved` | `--ctx-checkpoints` is **per slot**. Too few, and the checkpoint cannot track a growing conversation, so the same prefix is re-prefilled every turn. |
|
|
25
|
+
| KV pinned at the VBR floor | `kv-at-floor` | The pool is saturated and the cache has silently dropped to its lowest quality tier. |
|
|
26
|
+
| Client-visible timeouts | `generation-timeouts` | The symptom users actually feel, counted from the proxy journal. |
|
|
27
|
+
| Window spans a restart | `spans-restart` | A trend across a restart compares two process lifetimes, not decay within one. |
|
|
28
|
+
| Rail/pool geometry | `shared-pool` | Informational (never a fault). Under `--kv-unified` the pool is *shared*, so N rails each believing they own it is an overcommit waiting to happen. |
|
|
29
|
+
|
|
30
|
+
Health is `GREEN` / `WARN` / `RED`, or **`UNKNOWN` when there is no evidence**.
|
|
31
|
+
A fabricated GREEN is precisely what let the original incident run all day, so
|
|
32
|
+
every probe fails open and any that could not run is listed explicitly as
|
|
33
|
+
`unverified (probe unavailable)`.
|
|
34
|
+
|
|
35
|
+
`UNKNOWN` requires *all three* sampled signals to be missing — no usable
|
|
36
|
+
throughput trend, no checkpoint samples, and no KV reading. Losing one of them
|
|
37
|
+
does not suppress the rest. An **actionable finding always wins**: timeouts are
|
|
38
|
+
counted from the proxy journal and can fire with none of the sampled signals
|
|
39
|
+
present, and a RED finding must never be masked by an `UNKNOWN` rollup — which
|
|
40
|
+
would let it pass `--strict` with exit 0.
|
|
41
|
+
|
|
42
|
+
## Why prompt-size bucketing matters
|
|
43
|
+
|
|
44
|
+
Prefill throughput depends strongly on prompt length, and agent conversations
|
|
45
|
+
grow over a session. So an hourly average moves for reasons that have nothing
|
|
46
|
+
to do with the server, and a naive before/after comparison manufactures
|
|
47
|
+
trends that are not there.
|
|
48
|
+
|
|
49
|
+
The trend is therefore computed **within a size bucket** (`<5k`, `5-15k`,
|
|
50
|
+
`15-30k`, `>30k`), for every bucket with enough samples in both halves of the
|
|
51
|
+
window. If none qualifies, it reports that it cannot compare like with like
|
|
52
|
+
rather than guessing.
|
|
53
|
+
|
|
54
|
+
Each bucket is split at **its own** median time, not the global one. That
|
|
55
|
+
matters because agent conversations grow, so large prompts arrive late: on a
|
|
56
|
+
live 6h journal the `15-30k` bucket had 2 samples before the global midpoint
|
|
57
|
+
and 21 after, so the minimum-samples guard dropped it — while it had gone
|
|
58
|
+
616 → 73 tok/s. An 8× collapse, invisible, with the report showing a benign
|
|
59
|
+
`<5k` at 0.78 and emitting no finding at all. A bucket's own chronology is what
|
|
60
|
+
"did this get slower" means for that prompt size.
|
|
61
|
+
|
|
62
|
+
Findings are based on the **worst** bucket, and the others are named alongside
|
|
63
|
+
it. Ranking by sample count instead would hide the problem: on the real
|
|
64
|
+
incident window the `>30k` bucket had the most samples and sat at 0.52 (WARN),
|
|
65
|
+
while `5-15k` had collapsed to 0.29 (RED). Reporting the fattest bucket
|
|
66
|
+
understated the severity and flipped the remedy from *restart the server* to
|
|
67
|
+
*watch it* — and a thin bucket collapsing tenfold produced no finding at all.
|
|
68
|
+
|
|
69
|
+
## Replaying a past incident
|
|
70
|
+
|
|
71
|
+
`--until` bounds the journal window, which turns the command into a
|
|
72
|
+
post-incident tool:
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
uap inference health --since "2026-09-20 15:26:52" --until "2026-09-21 09:50:00"
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
```
|
|
79
|
+
REPLAY journal window 2026-09-20 15:26:52 .. 2026-09-21 09:50:00 — live readings below are omitted
|
|
80
|
+
RED findings for that window
|
|
81
|
+
prefill 628 → 205 tok/s ↓ in the 5-15k bucket (n=14/29)
|
|
82
|
+
reuse 20% of the reusable prefix restored (~17,373 tok/turn re-prefilled)
|
|
83
|
+
|
|
84
|
+
RED prefill in the 5-15k bucket fell 67% over this process's life (628 -> 205 tok/s); also down in <5k (54%), 15-30k (54%), >30k (49%)
|
|
85
|
+
→ restart the inference server; if it returns within days, investigate the prompt cache and idle-slot caching
|
|
86
|
+
WARN checkpoints recover only 20% of the reusable prefix (~17,373 tokens re-prefilled per turn)
|
|
87
|
+
RED 5 generation timeout(s) in the window — clients lost work
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
Pass **both** bounds. With `--until` alone the start would otherwise default to
|
|
91
|
+
the *currently running* process — which for any past incident is later than the
|
|
92
|
+
window's end, so `journalctl` returns nothing and the report is a clean
|
|
93
|
+
`UNKNOWN` with no hint that the window was backwards. The default is a relative
|
|
94
|
+
`-24h` in that case, but an explicit `--since` is what you want.
|
|
95
|
+
|
|
96
|
+
In this mode the live readings — uptime, `kv_bpv`, the unit's current flags —
|
|
97
|
+
are **dropped, not shown**. They describe the process running *now*, not the
|
|
98
|
+
one in the window. An early version did attribute the current
|
|
99
|
+
`--ctx-checkpoints 4` to a window in which it had been `1`, and recommended
|
|
100
|
+
the wrong remedy as a result.
|
|
101
|
+
|
|
102
|
+
## Reading the output
|
|
103
|
+
|
|
104
|
+
```
|
|
105
|
+
inference health: uap-gsq-rco-server.service
|
|
106
|
+
GREEN active, up 6m, 2 rail(s)
|
|
107
|
+
pool 229,376 cells shared (114,688/rail if split)
|
|
108
|
+
checkpoints 4 per slot
|
|
109
|
+
kv 8.125 bpv (floor 4.125)
|
|
110
|
+
prefill 682 → 766 tok/s ↑ in the 15-30k bucket (n=12/14)
|
|
111
|
+
reuse 94% of the reusable prefix restored (~2,600 tok/turn re-prefilled)
|
|
112
|
+
concurrency 1.058 busy slots/decode (>1 means the extra rail is actually being used)
|
|
113
|
+
|
|
114
|
+
· 2 rails share one 229,376-cell pool (114,688 each if evenly divided)
|
|
115
|
+
no problems detected
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
`pool ... shared` is deliberate wording. With a unified KV cache — which this
|
|
119
|
+
fork **forces** whenever VBR runs above one rail, rather than it being a flag
|
|
120
|
+
you set — `-c` is one shared pool and **`-np` does not divide it**. `/slots`
|
|
121
|
+
reports the full `n_ctx` for every slot and any rail may address all of it. Two rails each
|
|
122
|
+
believing they own the pool is an overcommit waiting to happen; the per-rail
|
|
123
|
+
figure is what it *would* be if split evenly, not an allowance the server
|
|
124
|
+
enforces.
|
|
125
|
+
|
|
126
|
+
`concurrency` is the honest test of whether an extra rail is earning anything.
|
|
127
|
+
A reading of `1.000` means requests never actually overlapped, so the second
|
|
128
|
+
rail bought nothing — which is a workload fact, not a server fault.
|
|
129
|
+
|
|
130
|
+
## Options
|
|
131
|
+
|
|
132
|
+
| Flag | Meaning |
|
|
133
|
+
| --- | --- |
|
|
134
|
+
| `--server-unit <unit>` | systemd user unit for the inference server |
|
|
135
|
+
| `--proxy-unit <unit>` | systemd user unit for the proxy (timeout counting) |
|
|
136
|
+
| `--url <url>` | server base URL (default `http://127.0.0.1:8080`) |
|
|
137
|
+
| `--since <expr>` | `journalctl --since` window (default: the server process start) |
|
|
138
|
+
| `--until <expr>` | `journalctl --until` bound — replay a past incident |
|
|
139
|
+
| `--json` | machine-readable, with a pinned `reportVersion` |
|
|
140
|
+
| `--strict` | exit 1 on WARN or RED, for CI and monitor loops |
|
|
141
|
+
|
|
142
|
+
The default window is **the server process's own start**, because throughput
|
|
143
|
+
decay is a property of one process — spanning a restart would manufacture a
|
|
144
|
+
cliff at the boundary. When `systemctl` is unavailable and that start time
|
|
145
|
+
cannot be read, it falls back to `-24h`. If the window does end up spanning
|
|
146
|
+
more than one server process, the report says so (`spans-restart`) rather than
|
|
147
|
+
presenting the boundary as decay.
|
|
148
|
+
|
|
149
|
+
In a replay (`--until`), the live readings are dropped from the analysis too,
|
|
150
|
+
not merely hidden — including the unit's current `-np`/`-c`, which describe the
|
|
151
|
+
process running now rather than the one in the window. The `shared-pool` line
|
|
152
|
+
therefore does not appear in a replay.
|
|
153
|
+
|
|
154
|
+
## Tuning checkpoints when KV sits at the floor
|
|
155
|
+
|
|
156
|
+
`kv-at-floor` usually means the pool is saturated — and on this stack a common
|
|
157
|
+
cause is the checkpoint allowance, because checkpoints are charged to the SAME
|
|
158
|
+
`--vbr-vram` budget as live context, at ~190 MiB each **per slot**:
|
|
159
|
+
|
|
160
|
+
| `--ctx-checkpoints` | cost across 2 rails | share of a 5120 MiB budget |
|
|
161
|
+
| --- | --- | --- |
|
|
162
|
+
| 4 | 1520 MiB | 30% |
|
|
163
|
+
| 3 | 1140 MiB | 22% |
|
|
164
|
+
| 2 | 760 MiB | 15% |
|
|
165
|
+
|
|
166
|
+
`scripts/set-ctx-checkpoints.sh N` retunes it, backing up the unit and leaving
|
|
167
|
+
the comment history intact:
|
|
168
|
+
|
|
169
|
+
```bash
|
|
170
|
+
bash scripts/set-ctx-checkpoints.sh 3 # stage only
|
|
171
|
+
bash scripts/set-ctx-checkpoints.sh 3 --now # stage and restart
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
There is no free direction. Raising it bought checkpoint recovery from 20% to
|
|
175
|
+
69% of the reusable prefix on 2026-09-21 — roughly 32k tokens of prefill saved
|
|
176
|
+
per turn — and cost KV quality, which settled at the 4.125 floor within six
|
|
177
|
+
hours. `uap inference health` reports both sides: `reuse NN%` is the checkpoint
|
|
178
|
+
value, `kv N.NNN bpv (floor …)` is the headroom it consumed.
|
|
179
|
+
|
|
180
|
+
## Related
|
|
181
|
+
|
|
182
|
+
- [Capacity Policy](CAPACITY_POLICY.md) (`uap doctor`) — liveness and declared
|
|
183
|
+
budgets, including the `budget.execStartMustContain` flag-drift check.
|
|
184
|
+
- [Local Models](LOCAL_MODELS.md) — the serving stack itself.
|
package/docs/guides/PROXY.md
CHANGED
|
@@ -181,6 +181,7 @@ set is below.
|
|
|
181
181
|
| **Recon-convergence** | `PROXY_RECON_CONVERGENCE_THRESHOLD` (40) | After N tool-using turns with **no** write/deliverable, inject "stop exploring, produce the deliverable"; escalates to stripping tools if it persists (0 disables) |
|
|
182
182
|
| **Hard-finalize** | `PROXY_HARD_FINALIZE_TURNS` (40) | Absolute turn cap before a forced finalize |
|
|
183
183
|
| **MANDATE-DELIVER** | `PROXY_MANDATE_DELIVER` (on) | On the delivery-enforcer block marker, pin `tool_choice` to `deliver` so the next turn *must* route through the gated path — makes routing binding for weak models |
|
|
184
|
+
| **Lockstep escalation** | `PROXY_LOCKSTEP_BREAK` (on), `PROXY_LOCKSTEP_PIVOT_AT` (2), `PROXY_LOCKSTEP_HARD_AT` (4) | The identical tool call failing with the identical error: inject a pivot directive at streak 2, a final warning one fire later, then hard-fail the session (400) at streak 4 instead of drifting (both directives always precede the stop, so it lands at `max(HARD_AT, PIVOT_AT+2)`). Each tier appends to the loop-incident ledger (`UAP_LOOP_INCIDENTS`, default `~/.config/uap/loop-incidents.jsonl`, ~3 records per episode, no rotation) for `uap loops` to inspect and hand to `uap ideate`/`uap deliver`. A fresh human user turn re-arms the guard. Disarmed if `PROXY_ERROR_LOOP=off` or `PROXY_DOUBLING_BREAK=off` (their trackers feed it) |
|
|
184
185
|
|
|
185
186
|
### Context & compaction
|
|
186
187
|
|