@miller-tech/uap 1.13.13 → 1.13.15

package/docs/benchmarks/SPECULATIVE_DECODING_JOURNEY_2026-03.md

@@ -0,0 +1,221 @@
+# Speculative Decoding Journey (2026-03)
+
+This document records the end-to-end speculative decoding stabilization journey across `llama.cpp` runtime tuning and `uap-anthropic-proxy` guardrails, including fixes, benchmark results, and the production profile now in use.
+
+## Scope
+
+- Runtime: `llama.cpp` with Qwen3.5 models, CUDA, `ctx-size=262144`.
+- Gateway: Anthropic-compatible proxy (`tools/agents/scripts/anthropic_proxy.py`).
+- Client behavior: agentic coding loops with tool calls (Claude Code style).
+
+## Goals
+
+1. Preserve high speculative decoding throughput.
+2. Eliminate pathological loops and malformed visible output.
+3. Keep tool-call behavior reliable under long sessions.
+4. Keep production context window at `262144`.
+
+## Phase 1 - Llama.cpp Speculative Stability
+
+### Problems Observed
+
+- Rollback loops and instability under aggressive speculative settings.
+- `find_slot` and related server warnings during long agentic sessions.
+- Throughput regressions compared to known fast baseline.
+
+### Work Performed
+
+- Implemented and tested multiple rollback strategies in `llama.cpp` worktree branches.
+- Compared baseline fast commit vs newer speculative logic.
+- Restored proven fast runtime path for production service while preserving learned guardrails.
+
+### Key Runtime Decisions
+
+- Keep production on fast validated binary lineage (`029edcafc` baseline family).
+- Use strict balanced speculative profile for 35B operations:
+  - `speculative.n_max=12`
+  - `speculative.n_min=2`
+  - `speculative.p_min=0.80`
+
+### Representative Throughput Findings
+
+- Qwen3.5-27B, `ctx=262144`, q4 KV cache:
+  - No spec: ~43 tok/s coding, ~41 tok/s pattern.
+  - Spec (balanced): ~43 tok/s coding, ~102 tok/s pattern.
+  - Main uplift appears in pattern-heavy turns, not all coding turns.
+
+## Phase 2 - Proxy Reasoning Fallback Leak Fix
+
+### Problems Observed
+
+- Empty visible output (`output_tokens=0`) with large hidden reasoning payloads.
+- Proxy emitted malformed chain-of-thought text as fallback, causing user-visible garbage:
+  - repeated fragments like `</parameter>`, tool schema echoes, policy text loops.
+
+### Fixes Implemented
+
+- Added explicit streaming fallback policy:
+  - `PROXY_STREAM_REASONING_FALLBACK=off|sanitized|visible`
+  - `PROXY_STREAM_REASONING_MAX_CHARS`
+- Set production default to `off`.
+
+### Result
+
+- Malformed reasoning fallback leakage is suppressed by default.
+- Debugging remains possible with `sanitized`/`visible` modes when intentionally enabled.
+
+## Phase 3 - Token Floor and Prune Controls
+
+### Problems Observed
+
+- Hardcoded `max_tokens` floor (`16384`) forced very long failure turns.
+- Pruning threshold flag alone could trigger pruning path without meaningful message reduction.
+
+### Fixes Implemented
+
+- Added configurable max token floor:
+  - `PROXY_MAX_TOKENS_FLOOR` (`0` disables floor)
+- Added configurable prune target:
+  - `PROXY_CONTEXT_PRUNE_TARGET_FRACTION`
+
+### Live A/B Result (Production-Like)
+
+`PROXY_MAX_TOKENS_FLOOR=16384` vs `4096`:
+
+- Silent reasoning-heavy turn:
+  - `16384`: avg `78.749s`
+  - `4096`: avg `19.777s`
+  - Latency reduction: ~`74.9%`
+  - Predicted throughput unchanged (~`208 tok/s` class)
+- Normal tool turns remained stable and slightly faster with `4096`.
+
+## Phase 4 - Malformed Tool-Loop Hardening
+
+### Problem Pattern
+
+Under adversarial or degraded prompt states, the model can emit pseudo-tool text instead of valid tool calls, e.g.:
+
+- `</parameter>` fragments
+- echoed policy snippets (`you MUST call a tool...`)
+- long no-progress text with no `tool_calls`
+
+### Feature Set Added (Flag Controlled)
+
+1. **Malformed tool guardrail + retry**
+   - `PROXY_MALFORMED_TOOL_GUARDRAIL`
+   - `PROXY_MALFORMED_TOOL_RETRY_MAX`
+   - `PROXY_MALFORMED_TOOL_RETRY_MAX_TOKENS`
+   - `PROXY_MALFORMED_TOOL_RETRY_TEMPERATURE`
+
+2. **Strict stream guardrail path**
+   - `PROXY_MALFORMED_TOOL_STREAM_STRICT`
+   - For stream+tools requests, proxy runs guarded non-stream upstream call, then replays SSE.
+
+3. **Tool narrowing (optional)**
+   - `PROXY_TOOL_NARROWING`
+   - `PROXY_TOOL_NARROWING_KEEP`
+   - `PROXY_TOOL_NARROWING_MIN_TOOLS`
+
+4. **Disable thinking on tool turns (optional)**
+   - `PROXY_DISABLE_THINKING_ON_TOOL_TURNS`
+
+5. **Session contamination breaker (optional safety net)**
+   - `PROXY_SESSION_CONTAMINATION_BREAKER`
+   - `PROXY_SESSION_CONTAMINATION_THRESHOLD`
+   - `PROXY_SESSION_CONTAMINATION_KEEP_LAST`
+
+6. **Agentic supplement mode**
+   - `PROXY_AGENTIC_SUPPLEMENT_MODE=clean|legacy`
+
+### Test Coverage
+
+- Unit tests in `tools/agents/tests/test_anthropic_proxy_streaming.py`
+- Current targeted suite count in this workstream: `16` passing tests.
+
+## Benchmark Highlights (Per-Option Toggles)
+
+### Artifact Stress Benchmark (v3)
+
+Source: `/tmp/proxy_visibility_benchmark_v3.json`
+
+| Mode | Key Flags | Outcome Summary |
+| --- | --- | --- |
+| Baseline | none | no tool call, policy-echo text surfaced |
+| Option 1 | malformed guardrail + strict stream | malformed detected and retried; returned `tool_use` with empty visible text |
+| Option 2 | tool narrowing only | not sufficient alone in stress case |
+| Option 3 | disable thinking only | not sufficient alone in stress case |
+| Option 4 | contamination breaker only | not sufficient alone in this synthetic workload |
+| Option 5 | clean supplement only | not sufficient alone in stress case |
+
+### Practical Conclusion
+
+- Strongest primary mitigation: **Option 1** (malformed guardrail + strict stream + bounded retry).
+- Other options are secondary tuning aids and should not replace Option 1 for this failure class.
+
+## 10-Turn Live Stability Soak
+
+Source: `/tmp/proxy_10turn_soak_results.json`
+
+- 10 turns, alternating malformed-stress and normal tool-call turns, single live session id.
+- Results:
+  - Error rate: `0.0%`
+  - Malformed visible output rate (stress turns): `0.0%`
+  - Normal tool-call success rate: `100.0%`
+  - Duration p50/p95: `10.2s` / `21.366s`
+  - Stop reasons: `tool_use=6`, `max_tokens=3`, `end_turn=1`
+
+## Production Profile (Current)
+
+File: `/home/cogtek/.config/uap/anthropic-proxy.env`
+
+```bash
+PROXY_MAX_TOKENS_FLOOR=4096
+PROXY_STREAM_REASONING_FALLBACK=off
+
+PROXY_MALFORMED_TOOL_GUARDRAIL=on
+PROXY_MALFORMED_TOOL_STREAM_STRICT=on
+PROXY_MALFORMED_TOOL_RETRY_MAX=1
+PROXY_MALFORMED_TOOL_RETRY_MAX_TOKENS=512
+PROXY_MALFORMED_TOOL_RETRY_TEMPERATURE=0
+
+PROXY_TOOL_NARROWING=off
+PROXY_DISABLE_THINKING_ON_TOOL_TURNS=off
+PROXY_SESSION_CONTAMINATION_BREAKER=off
+PROXY_AGENTIC_SUPPLEMENT_MODE=legacy
+```
+
+Rationale:
+
+- Keep the strongest practical fix enabled (malformed guardrail + strict stream path).
+- Keep latency-optimized floor (`4096`).
+- Keep optional secondary heuristics off unless new evidence warrants enablement.
+
+## Reproduction Checklist
+
+1. Restart services:
+
+```bash
+systemctl --user restart uap-llama-server.service
+systemctl --user restart uap-anthropic-proxy.service
+```
+
+2. Run targeted unit tests:
+
+```bash
+python3 -m pytest tools/agents/tests/test_anthropic_proxy_streaming.py -q
+```
+
+3. Run soak script (or equivalent alternating malformed/normal stream sequence).
+
+4. Validate logs:
+
+- `MALFORMED TOOL PAYLOAD`
+- `MALFORMED RETRY ...`
+- `STRICT STREAM GUARDRAIL`
+- Absence of user-visible malformed fragments.
+
+## Open Follow-Ups
+
+- Add a dedicated persistent benchmark harness under `scripts/` for this exact soak profile.
+- Add branch/commit links from `llama.cpp` worktrees for cross-repo traceability.
+- Optionally evaluate enabling `PROXY_TOOL_NARROWING` in production only after longer mixed-workload soak data.

package/docs/deployment/QWEN35_LLAMA_CPP.md

@@ -116,6 +116,50 @@ llama-server \
 | `--draft-max`   | `16`                             | Max tokens to draft per iteration. Higher = more throughput, more VRAM. |
 | `--draft-p-min` | `0.75`                           | Minimum acceptance probability. Lower = more aggressive drafting.       |
 
+## Extension Options for Speculative Decoding
+
+### Option 1: Adaptive Runtime Tuning (implemented)
+
+Use acceptance and rollback rates to auto-adjust `draft-max`, `draft-min`, and `draft-p-min` over time.
+
+- Best for immediate gains without kernel changes
+- Reduces bad bursts when acceptance drops
+- Increases burst length automatically during high-acceptance windows
+
+Commands:
+
+```bash
+# Tune once from observed metrics
+llama-optimize spec-autotune --acceptance 0.71 --rollback 0.14 --profile throughput
+
+# Compare static defaults vs adaptive tuning using deterministic simulation
+llama-optimize spec-benchmark --profile throughput --trace mixed --steps 180
+
+# Live benchmark active server and get tuned flag recommendation
+llama-optimize spec-benchmark-live \
+  --endpoint http://127.0.0.1:8080/v1 \
+  --model qwen3.5-a3b-iq4xs \
+  --runs 5 --max-tokens 256 --profile throughput
+```
+
+Recommended workflow:
+
+1. Run `spec-benchmark-live` with your current startup flags and note `Throughput`.
+2. Restart `llama-server` with the `Suggested params` flags.
+3. Re-run `spec-benchmark-live` with the same settings to measure actual gain.
+
+### Option 2: GPU Residency + Overlap
+
+- Keep draft model and draft KV fully on GPU
+- Preallocate buffers and overlap draft + verify passes with CUDA streams
+- Improves p95 latency consistency on long runs
+
+### Option 3: GPU Checkpoint/Rollback
+
+- Move speculative checkpoint snapshots from CPU RAM to GPU buffers
+- Remove host-device copy overhead from rollback paths
+- Highest upside, but requires deeper runtime changes
+
 ### Sampling
 
 | Flag               | Value  | Purpose                                           |
@@ -177,6 +221,37 @@ All settings are via environment variables:
 | `PROXY_LOG_LEVEL`       | `INFO`                               | Logging level (DEBUG/INFO/WARNING/ERROR) |
 | `PROXY_READ_TIMEOUT`    | `600`                                | Read timeout (seconds) for LLM streaming |
 | `PROXY_MAX_CONNECTIONS` | `20`                                 | Max concurrent upstream connections      |
+| `PROXY_MAX_TOKENS_FLOOR` | `16384`                             | Minimum floor applied to incoming `max_tokens` (`0` disables floor) |
+| `PROXY_CONTEXT_PRUNE_TARGET_FRACTION` | `0.65`            | Target context utilization after pruning (`0.0 < value < 1.0`) |
+| `PROXY_STREAM_REASONING_FALLBACK` | `off`                      | Streaming behavior for reasoning-only empty turns (`off`, `sanitized`, `visible`) |
+| `PROXY_STREAM_REASONING_MAX_CHARS` | `240`                      | Max fallback length when `PROXY_STREAM_REASONING_FALLBACK=sanitized` |
+| `PROXY_TOOL_NARROWING` | `off`                                | Narrow large tool lists to top relevant tools per turn |
+| `PROXY_TOOL_NARROWING_KEEP` | `8`                             | Number of tools to keep when narrowing is enabled |
+| `PROXY_TOOL_NARROWING_MIN_TOOLS` | `12`                       | Minimum tool count before narrowing activates |
+| `PROXY_DISABLE_THINKING_ON_TOOL_TURNS` | `off`                 | Sends `enable_thinking=false` when tools are present |
+| `PROXY_MALFORMED_TOOL_GUARDRAIL` | `on`                      | Detects malformed pseudo tool payloads and retries with strict settings |
+| `PROXY_MALFORMED_TOOL_RETRY_MAX` | `1`                        | Number of malformed-tool retries |
+| `PROXY_MALFORMED_TOOL_RETRY_MAX_TOKENS` | `2048`             | Retry cap for `max_tokens` during malformed-tool recovery |
+| `PROXY_MALFORMED_TOOL_RETRY_TEMPERATURE` | `0`                | Retry temperature for malformed-tool recovery |
+| `PROXY_MALFORMED_TOOL_STREAM_STRICT` | `off`                 | For stream+tools requests, use guarded non-stream upstream path then replay SSE |
+| `PROXY_SESSION_CONTAMINATION_BREAKER` | `on`                 | Resets long-running malformed sessions to recent context |
+| `PROXY_SESSION_CONTAMINATION_THRESHOLD` | `3`                | Consecutive malformed turns before reset |
+| `PROXY_SESSION_CONTAMINATION_KEEP_LAST` | `8`                 | Number of latest messages to preserve during contamination reset |
+| `PROXY_AGENTIC_SUPPLEMENT_MODE` | `clean`                   | Agentic system supplement variant (`clean`, `legacy`) |
+
+For agentic coding workloads, keep `PROXY_STREAM_REASONING_FALLBACK=off` (default) to avoid leaking malformed internal reasoning as user-visible output. Use `sanitized` only for debugging.
+
+For Claude Code + Qwen malformed-tool loops, recommended starting profile:
+
+```bash
+PROXY_STREAM_REASONING_FALLBACK=off
+PROXY_MAX_TOKENS_FLOOR=4096
+PROXY_MALFORMED_TOOL_GUARDRAIL=on
+PROXY_TOOL_NARROWING=on
+PROXY_DISABLE_THINKING_ON_TOOL_TURNS=on
+PROXY_SESSION_CONTAMINATION_BREAKER=on
+PROXY_AGENTIC_SUPPLEMENT_MODE=clean
+```
 
 ### Example: Custom upstream
 
@@ -339,3 +414,4 @@ Two possible causes:
 - `tools/agents/scripts/qwen_tool_call_test.py` - Test suite using OpenAI-compatible API
 - `src/cli/tool-calls.ts` - CLI command for template management
 - `src/bin/llama-server-optimize.ts` - llama-server startup optimizer
+- `docs/deployment/UAP_LLAMA_ANTHROPIC_PROXY_BOOTSTRAP.md` - service bootstrap + ngram-cache A/B benchmarking

package/docs/deployment/UAP_LLAMA_ANTHROPIC_PROXY_BOOTSTRAP.md

@@ -0,0 +1,279 @@
+# UAP + llama.cpp + Anthropic Proxy Bootstrap
+
+This guide captures the local continuity stack as a repeatable bootstrap:
+
+- `uap-llama-server.service` (llama.cpp)
+- `uap-anthropic-proxy.service` (Anthropic API compatibility)
+- A/B benchmark workflow for speculative decoding with `ngram-cache`
+
+It also documents the UAP-side support changes needed to keep llama.cpp speculative decoding stable in agentic workflows.
+
+## 1) Bootstrap services
+
+Run:
+
+```bash
+bash scripts/bootstrap/bootstrap-uap-llama-proxy-stack.sh
+```
+
+This writes:
+
+- `~/.config/uap/llama-server.env`
+- `~/.config/uap/anthropic-proxy.env`
+- `~/.config/systemd/user/uap-llama-server.service`
+- `~/.config/systemd/user/uap-anthropic-proxy.service`
+
+Then it enables and starts both user services.
+
+## 2) Key llama env knobs
+
+Edit `~/.config/uap/llama-server.env` and restart service:
+
+```bash
+systemctl --user restart uap-llama-server.service
+```
+
+Important variables:
+
+- `LLAMA_SPEC_TYPE` (`none`, `ngram-cache`, etc.)
+- `LLAMA_DRAFT_MAX`
+- `LLAMA_DRAFT_MIN`
+- `LLAMA_DRAFT_P_MIN`
+- `LLAMA_EXTRA_ARGS` (optional additional startup flags)
+
+## 3) Key proxy env knobs
+
+Edit `~/.config/uap/anthropic-proxy.env` and restart proxy:
+
+```bash
+systemctl --user restart uap-anthropic-proxy.service
+```
+
+Important variables:
+
+- `PROXY_PORT`
+- `LLAMA_CPP_BASE`
+- `PROXY_CONTEXT_WINDOW` (set to `262144` to match llama context)
+- Loop/guardrail options (`PROXY_LOOP_BREAKER`, `PROXY_FORCED_THRESHOLD`, etc.)
+
+## 4) Run ngram-cache signal benchmark
+
+Use the service-oriented A/B script:
+
+```bash
+bash scripts/benchmarks/run-spec-ngram-service-ab.sh
+```
+
+What it does:
+
+1. Stops managed `uap-llama-server.service` temporarily
+2. Runs transient systemd service benchmarks for:
+   - `spec-type=none`
+   - `spec-type=ngram-cache` (default draft params)
+   - `spec-type=ngram-cache` (tuned: `21/6/0.72`)
+3. Restores managed `uap-llama-server.service`
+4. Writes report artifacts under `benchmark-results/spec-ngram-ab-<timestamp>/`
+
+Outputs:
+
+- `report.json` machine-readable deltas
+- `report.md` human-readable summary
+
+## 5) Run automatic draft-parameter sweep (Option 2)
+
+Use this to search for the best local `ngram-cache` settings:
+
+```bash
+bash scripts/benchmarks/run-spec-ngram-sweep.sh
+```
+
+Useful overrides:
+
+```bash
+RUNS=5 MAX_TOKENS=256 \
+DRAFT_MAXS="16 18 20 22" \
+DRAFT_MINS="3 4 5 6" \
+DRAFT_P_MINS="0.70 0.72 0.75 0.78" \
+bash scripts/benchmarks/run-spec-ngram-sweep.sh
+```
+
+Outputs are written under `benchmark-results/spec-ngram-sweep-<timestamp>/`:
+
+- `results.jsonl` one entry per candidate
+- `summary.json` best candidate + stats
+- `summary.md` top 5 table
+
+## 6) Profiles for agentic coding vs max speed
+
+Use two explicit profiles depending on your goal.
+
+### A) Agentic coding continuity profile (recommended daily use)
+
+This profile prioritizes long, coherent coding sessions and minimizes `find_slot` warnings.
+
+`~/.config/uap/llama-server.env`:
+
+```env
+LLAMA_CTX_SIZE=262144
+LLAMA_SPEC_TYPE=ngram-cache
+LLAMA_DRAFT_MAX=12
+LLAMA_DRAFT_MIN=2
+LLAMA_DRAFT_P_MIN=0.80
+LLAMA_HYBRID_ROLLBACK_MODE=strict
+```
+
+Apply it:
+
+```bash
+systemctl --user restart uap-llama-server.service
+```
+
+`~/.config/uap/anthropic-proxy.env`:
+
+```env
+PROXY_CONTEXT_WINDOW=262144
+PROXY_LOOP_BREAKER=on
+PROXY_LOOP_WINDOW=6
+PROXY_LOOP_REPEAT_THRESHOLD=10
+PROXY_FORCED_THRESHOLD=18
+PROXY_NO_PROGRESS_THRESHOLD=5
+PROXY_CONTEXT_RELEASE_THRESHOLD=0.95
+PROXY_GUARDRAIL_RETRY=on
+```
+
+Apply it:
+
+```bash
+systemctl --user restart uap-anthropic-proxy.service
+```
+
+### B) Max-throughput benchmark profile (where 220+ tok/s was observed)
+
+The 220+ decode throughput observed in this session was achieved with:
+
+- CUDA build: `/home/cogtek/llama.cpp/.worktrees/001-llama-spec-rollback-fix/build-cuda/bin/llama-server`
+- GPU flags: `--device CUDA0 --n-gpu-layers all --flash-attn on`
+- Speculative mode: `--spec-type ngram-cache`
+- Rollback mode: `LLAMA_HYBRID_ROLLBACK_MODE=hybrid`
+- Workload: repetitive pattern prompt, `n_predict=512`
+
+Run command used for that profile:
+
+```bash
+LLAMA_HYBRID_ROLLBACK_MODE=hybrid \
+/home/cogtek/llama.cpp/.worktrees/001-llama-spec-rollback-fix/build-cuda/bin/llama-server \
+  -m "/home/cogtek/Downloads/Qwen3.5-35B-A3B-UD-IQ4_XS.gguf" \
+  --host 127.0.0.1 --port 18121 \
+  --ctx-size 16384 --parallel 1 --no-warmup \
+  --device CUDA0 --n-gpu-layers all --flash-attn on \
+  --spec-type ngram-cache
+```
+
+Important: this max-speed profile is workload-sensitive and was measured on a pattern-heavy prompt. For real agentic coding, use Profile A.
+
+## 7) Validated A/B findings (2026-03-23)
+
+Direct old-vs-new A/B was run against:
+
+- old fast commit: `029edcafc` (first pushed fast state around 21:35)
+- newer commit: `1f8225f8f`
+- model: `Qwen3.5-35B-A3B-UD-IQ4_XS.gguf`
+- speculative: `ngram-cache`, `draft-max=16`, `draft-min=3`, `draft-p-min=0.72`
+
+Notes:
+
+- Standalone launches at `ctx-size=262144` can fail GPU allocation on some runs for the old commit (`failed to allocate compute pp buffers`).
+- For controlled apples-to-apples throughput comparison, A/B was run at `ctx-size=16384`.
+
+Observed results (`/tmp/ab_matrix_ctx16_v2.json`):
+
+| Path            | Old `029edcafc` | New `1f8225f8f` | Delta (new vs old) |
+| --------------- | --------------- | --------------- | ------------------- |
+| Raw coding      | 107.97 tok/s    | 99.23 tok/s     | -8.1%               |
+| Raw pattern     | 158.71 tok/s    | 105.75 tok/s    | -33.4%              |
+| Proxy plain     | 113.74 tok/s    | 109.39 tok/s    | -3.8%               |
+| Agentic tool 2nd turn | `tool_use` (stable) | `tool_use` (stable) | parity on control flow |
+
+Behavioral observations:
+
+- Newer commit emitted many `find_slot: non-consecutive token position` warnings in raw/proxy runs under the same speculative settings.
+- Old commit produced materially cleaner logs and higher throughput in the same benchmark profile.
+- Proxy continuity fixes improved agentic tool-loop stability and no longer force premature stop in the tested loop.
+
+Decision for throughput-sensitive testing:
+
+- Prefer old fast commit `029edcafc` profile for max-throughput benchmarking.
+- Keep a separate continuity profile for long-context agentic coding if warning volume grows.
+
+Additional 27B impact snapshot (`Qwen3.5-27B-IQ4_XS`, `ctx=262144`, q4 KV cache):
+
+- no speculative: ~43 tok/s coding, ~41 tok/s pattern
+- aggressive speculative (`16/3/0.72`): ~44 tok/s coding, ~102 tok/s pattern
+- balanced speculative (`12/2/0.80`): ~43 tok/s coding, ~102 tok/s pattern
+
+Interpretation:
+
+- balanced profile is functionally safer for agentic sessions,
+- aggressive profile can edge higher on some coding runs,
+- both speculative profiles massively outperform no-spec on repetition-heavy drafts.
+
+## 8) Throughput interpretation and loop prevention
+
+When reading llama logs, treat these as different metrics:
+
+- `prompt eval time ... tokens per second` = prefill throughput
+- `eval time ... tokens per second` = decode/completion throughput
+
+In local continuity runs with large context, prompt throughput may exceed 2k tok/s while decode remains near 80-125 tok/s.
+
+For default stability, use the guardrails from Profile A. If you hit active loop incidents, temporarily tighten to:
+
+```env
+PROXY_LOOP_WINDOW=6
+PROXY_LOOP_REPEAT_THRESHOLD=8
+PROXY_FORCED_THRESHOLD=14
+PROXY_NO_PROGRESS_THRESHOLD=4
+PROXY_CONTEXT_RELEASE_THRESHOLD=0.90
+```
+
+Then restart proxy:
+
+```bash
+systemctl --user restart uap-anthropic-proxy.service
+```
+
+## 9) UAP support changes required for reliable operation
+
+The following UAP-side changes are part of the working stack and should be present:
+
+1. Session-scoped loop protection in Anthropic proxy (no cross-session contamination).
+2. Guardrail retry for unexpected text-only end-turn in active tool loops.
+3. Optional systemd scaffolding from CLI:
+   - `uap init --systemd-services`
+   - `uap setup --systemd-services`
+4. Dedicated launch scripts:
+   - `scripts/run-llama-server-continuity.sh`
+   - `scripts/run-anthropic-proxy-continuity.sh`
+
+These changes ensure llama speculative behavior is evaluated in a stable proxy/control-plane environment.
+
+## 10) Check service health
+
+```bash
+systemctl --user status uap-llama-server.service --no-pager
+systemctl --user status uap-anthropic-proxy.service --no-pager
+curl -sf http://127.0.0.1:8080/v1/models
+curl -sf http://127.0.0.1:4000/health
+```
+
+## 11) References and credits
+
+This implementation and tuning flow builds on prior llama.cpp and proxy work:
+
+- llama.cpp speculative docs: `docs/speculative.md`
+- llama.cpp hybrid rollout notes: `docs/development/speculative-hybrid-rollout.md`
+- llama.cpp speculative lineage: #5479, #6828, #6848, #19164
+- checkpoint/SWA context note:
+  - https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055
+
+Thanks to ggml-org/llama.cpp maintainers and contributors for speculative, cache, and memory-path groundwork.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@miller-tech/uap",
-  "version": "1.13.13",
+  "version": "1.13.15",
   "description": "Autonomous AI agent memory system with CLAUDE.md protocol enforcement",
   "type": "module",
   "main": "dist/index.js",