coding-guardrails 0.26.0__tar.gz → 0.28.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/CLAUDE.md +60 -22
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/HISTORY.md +24 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/PKG-INFO +12 -6
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/README.md +11 -5
- coding_guardrails-0.28.0/configs/ple-bench.yaml +98 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/docs/agents.md +14 -8
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/docs/architecture.md +11 -6
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/docs/models.md +151 -5
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/pyproject.toml +1 -1
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/cli.py +5 -5
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/models/profiles.py +150 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/proxy/handler.py +1 -1
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/proxy/server.py +1 -1
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/commands.py +12 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/server/cli.py +1 -1
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/server/download.py +29 -1
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/server/launcher.py +58 -3
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/server/manager.py +1 -1
- coding_guardrails-0.28.0/src/coding_guardrails/server/sources.py +224 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/server/version.py +16 -5
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_commands.py +17 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_plain_thinking.py +20 -2
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_server.py +105 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_sources.py +60 -0
- coding_guardrails-0.26.0/src/coding_guardrails/server/sources.py +0 -130
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/.github/workflows/ci.yaml +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/.gitignore +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/Dockerfile +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/Dockerfile.reusable +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/LICENSE +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/configs/aider.yaml +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/configs/claude-code.yaml +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/configs/guardrail-config.yaml +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/configs/opencode.json +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/configs/pi.yaml +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/docker-compose.yaml +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/docs/rules.md +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/docs/semantic-judge.md +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/docs/server.md +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/patches/pi-subagents/01-lenient-acceptance-parser.patch +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/patches/pi-subagents/02-stronger-acceptance-prompt.patch +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/patches/pi-subagents/03-stronger-finalization-prompt.patch +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/patches/pi-subagents/04-prefer-acceptance-message.patch +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/patches/pi-subagents/05-acceptance-schema-tolerance.patch +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/patches/pi-subagents/README.md +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/patches/pi-subagents/apply-patches.sh +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/patches/pi-subagents/test-acceptance-parser.mjs +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/patches/pi-subagents/test-get-final-output.mjs +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/patches/pi-subagents/test-real-world.mjs +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/reports/2026-06-08_qwen-vs-gemma-ab.md +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/reports/2026-06-08_qwen-vs-gemma-revalidation.md +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/reports/2026-06-27_ornith-assessment.md +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/scripts/smoke_proxy.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/__init__.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/__main__.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/config.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/eval.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/middleware.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/models/__init__.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/models/registry.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/proxy/__init__.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/proxy/acceptance.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/proxy/client.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/proxy/layer1.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/__init__.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/base.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/canary.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/dup_write.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/injection.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/judge.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/lint.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/loop_detection.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/network.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/path_safety.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/prerequisites.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/secrets.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/sensitive_files.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/sequencing.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/session_budget.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/thoroughness.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/rules/tool_resolution.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/server/__init__.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/server/build.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/server/manager_vram.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/server/paths.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/src/coding_guardrails/server/proxy_process.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/__init__.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/integration/__init__.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/integration/test_proxy.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/integration/test_proxy_pipeline.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/integration-subagent-smoke.sh +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/test_backend_manager.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/__init__.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/conftest.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_acceptance_prefill.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_acceptance_wrap.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_canary.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_client_wrappers.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_config.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_convergence_nudge.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_dup_write.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_fuzz.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_handler_respond.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_handler_security_wiring.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_injection.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_judge.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_layer1_tool_arg_validation.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_lint.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_loop_detection.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_loop_session_reset.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_manager.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_middleware.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_network.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_path_safety.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_paths.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_prerequisites.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_profiles_budget.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_proxy_process.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_reasoning_replay.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_role_normalization.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_secrets.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_sensitive_files.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_sequencing.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_session_budget.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_text_passthrough.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_thoroughness.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_tool_resolution.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/tests/unit/test_vision_captioning.py +0 -0
- {coding_guardrails-0.26.0 → coding_guardrails-0.28.0}/uv.lock +0 -0
|
@@ -6,7 +6,7 @@ An LLM proxy with safety guardrails, built on [Forge](https://github.com/antoine
|
|
|
6
6
|
|
|
7
7
|
```bash
|
|
8
8
|
source .venv/bin/activate
|
|
9
|
-
pytest tests/unit/ -q #
|
|
9
|
+
pytest tests/unit/ -q # 810 tests (~25s)
|
|
10
10
|
uv pip install -e ".[dev]" # refresh editable install
|
|
11
11
|
```
|
|
12
12
|
|
|
@@ -185,7 +185,7 @@ LLAMA=~/llama.cpp/llama-server
|
|
|
185
185
|
$LLAMA \
|
|
186
186
|
-m ~/.cache/lm-studio/models/unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q3_K_XL.gguf \
|
|
187
187
|
-md ~/models/dspine/Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf \
|
|
188
|
-
-c 120000 -ngl 99 --host
|
|
188
|
+
-c 120000 -ngl 99 --host 127.0.0.1 --port 8080 \
|
|
189
189
|
--jinja --flash-attn auto -ctk q4_0 -ctv q4_0 \
|
|
190
190
|
--spec-type draft-dflash --spec-draft-n-max 4 --spec-draft-ngl 99 \
|
|
191
191
|
--spec-draft-type-k q8_0 --spec-draft-type-v q8_0 -np 1 -v
|
|
@@ -208,7 +208,7 @@ $LLAMA \
|
|
|
208
208
|
# Manager: cg server start -m flint-UD-Q3_K_XL
|
|
209
209
|
$LLAMA \
|
|
210
210
|
-m ~/.local/share/coding-guardrails/models/flint-GGUF/flint-UD-Q3_K_XL.gguf \
|
|
211
|
-
-c 131072 -ngl 99 --host
|
|
211
|
+
-c 131072 -ngl 99 --host 127.0.0.1 --port 8080 \
|
|
212
212
|
--jinja --flash-attn auto -ctk q8_0 -ctv q8_0 -np 1 -v
|
|
213
213
|
|
|
214
214
|
# Qwen3.5-9B-MTP (previous default — best tool-calling reliability at 9B)
|
|
@@ -220,7 +220,7 @@ $LLAMA \
|
|
|
220
220
|
# PORT: backend MUST be :8080 — :8081 is the guardrails proxy and will collide.
|
|
221
221
|
$LLAMA \
|
|
222
222
|
-m ~/.cache/lm-studio/models/unsloth/Qwen3.5-9B-MTP-GGUF/Qwen3.5-9B-UD-Q4_K_XL.gguf \
|
|
223
|
-
-c 32768 -ngl 99 --host
|
|
223
|
+
-c 32768 -ngl 99 --host 127.0.0.1 --port 8080 \
|
|
224
224
|
--jinja --flash-attn auto --spec-type draft-mtp -np 1 -v
|
|
225
225
|
# Tune up: raise -c toward 200000 (max) for long context, --spec-draft-n-max 5 for more speed.
|
|
226
226
|
|
|
@@ -230,19 +230,19 @@ $LLAMA \
|
|
|
230
230
|
# Raw equivalent:
|
|
231
231
|
$LLAMA \
|
|
232
232
|
-m ~/.cache/lm-studio/models/unsloth/Qwen3.6-27B-MTP-GGUF/Qwen3.6-27B-UD-Q4_K_XL.gguf \
|
|
233
|
-
-c 49152 -ngl 99 --host
|
|
233
|
+
-c 49152 -ngl 99 --host 127.0.0.1 --port 8080 \
|
|
234
234
|
--jinja --flash-attn auto -ub 256 -ctk q8_0 -ctv q8_0 -np 1 -v
|
|
235
235
|
|
|
236
236
|
# Qwen3.6-27B UD-Q3_K_XL (49K ctx max, needs 24+ GB free VRAM — not profile-registered)
|
|
237
237
|
$LLAMA \
|
|
238
238
|
-m ~/.cache/lm-studio/models/unsloth/Qwen3.6-27B-MTP-GGUF/Qwen3.6-27B-UD-Q3_K_XL.gguf \
|
|
239
|
-
-c 81920 -ngl 99 --host
|
|
239
|
+
-c 81920 -ngl 99 --host 127.0.0.1 --port 8080 \
|
|
240
240
|
--jinja --flash-attn auto -v
|
|
241
241
|
|
|
242
242
|
# Gemma 4 12B Unified UD-Q4_K_XL (256K ctx, ~8 GB VRAM)
|
|
243
243
|
$LLAMA \
|
|
244
244
|
-m ~/.cache/lm-studio/models/unsloth/gemma-4-12b-it-GGUF/gemma-4-12b-it-UD-Q4_K_XL.gguf \
|
|
245
|
-
-c 256000 -ngl 99 --host
|
|
245
|
+
-c 256000 -ngl 99 --host 127.0.0.1 --port 8080 \
|
|
246
246
|
--jinja --flash-attn auto -np 1 -v
|
|
247
247
|
|
|
248
248
|
# Gemma 4 26B A4B QAT UD-Q4_K_XL (200K ctx, ~20 GB VRAM)
|
|
@@ -251,7 +251,7 @@ $LLAMA \
|
|
|
251
251
|
# Highest raw capability, but Qwen3.5-9B is the default (better tool-calling).
|
|
252
252
|
$LLAMA \
|
|
253
253
|
-m ~/.cache/lm-studio/models/unsloth/gemma-4-26B-A4B-it-qat-GGUF/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf \
|
|
254
|
-
-c 200000 -ngl 99 --host
|
|
254
|
+
-c 200000 -ngl 99 --host 127.0.0.1 --port 8080 \
|
|
255
255
|
--jinja --flash-attn auto -ctk q8_0 -ctv q8_0 \
|
|
256
256
|
--temp 1.0 --top-p 0.95 --top-k 64 -np 1 -v
|
|
257
257
|
|
|
@@ -319,6 +319,26 @@ coding-guardrails serve \
|
|
|
319
319
|
|
|
320
320
|
### Notes
|
|
321
321
|
|
|
322
|
+
- **Qwen3.8-27B-pi-adapted (default since 2026-11-19)**: a 319 MB F16 LoRA
|
|
323
|
+
(`harness-hub/Qwen3.8-27B-pi-adapted-GGUF`) over the UD-Q3_K_XL base, RL-trained (AsyncGRPO,
|
|
324
|
+
LoRA r=32 all-linear) *inside* unmodified pi. Card: SWE-bench Pro hard-51 39.2% -> 64.7%,
|
|
325
|
+
Terminal-Bench 2.1 39.5% -> 55.8%, turns per solved task 23 vs 59 (the earlier
|
|
326
|
+
"29.4% -> 62.7%" was retracted by the authors — nine rollouts died on a dead tunnel).
|
|
327
|
+
`coding-guardrails server start -m Qwen3.8-27B-pi-adapted`. It composes with the DSpine
|
|
328
|
+
drafter (verified 2026-11-19: adapter scale 1.0 AND draft-dflash n_max=4 in one process;
|
|
329
|
+
acceptance 0.58-0.80) at ~12% decode cost (60-63 vs 70.6 tok/s). ctx **160000** with
|
|
330
|
+
q4_0 KV = 21,068 MiB used / ~1.4-1.5 GB free (192K boots but leaves 704 MiB — too tight
|
|
331
|
+
on a shared desktop; ~26 KB per context token). `context_budget=120000` because the base
|
|
332
|
+
model's tool-call cliff does not reproduce on this adapter (24/24 to 98K, see below).
|
|
333
|
+
Thinking is ON (`--reasoning auto`); the `-nothink` sibling pins `--reasoning off`, the
|
|
334
|
+
mode it was trained and benchmarked in — that buys ~4.5x fewer tokens but costs ~10 points
|
|
335
|
+
on SWE-bench Pro hard-51. Caveat: trained on BF16, we run Q3_K_XL, so card numbers are
|
|
336
|
+
directional only. Logprobs are unavailable while the drafter is on.
|
|
337
|
+
- **LoRA plumbing**: `ModelProfile.lora` -> `LORAS` in `server/sources.py` -> `--lora` in
|
|
338
|
+
`server/launcher.py` (`_lora_argv`) -> `download_lora` in `server/download.py`. A missing
|
|
339
|
+
adapter WARNS loudly ("serving the BASE model, WITHOUT the adapter") and boots anyway,
|
|
340
|
+
same policy as a missing drafter. An adapted profile reuses its base profile's GGUF — its
|
|
341
|
+
`SOURCES` entry points at the same file, so only the adapter is downloaded.
|
|
322
342
|
- **llama-server**: use the local `~/llama.cpp/llama-server` (a symlink to `build/bin/llama-server`) for scratch experiments. cg's managed binary lives at `~/.local/share/coding-guardrails/llama.cpp` and is built from the pin in `server/version.py` — **bumped 2026-11-09 to `d493f84` (llama.cpp 0.9.0-dev) from the DSpine fork** (`PINNED_REPO = github.com/Ajay9o9/llama.cpp`, branch `dspine`; was `39febd2ad` upstream). The fork commit is `dflash : add DSpine draft model support` on top of upstream master, so everything the old pin carried (Gemma 4 tool-call fix, Flint `LLM_ARCH_FLINT`, LFM3 tri-hybrid, Qwen3.5/3.6/3.8, Ornith, LFM2.5, MiniCPM5) is still in. `build.py` re-points the checkout's `origin` every run, so a checkout cloned from ggml-org follows the pin to the fork with no manual git work. Rebuild: `coding-guardrails server build` (CUDA auto-detected; the CLI's shallow `git fetch <sha>` can fail with `early EOF` — retry, and clear any stale `.git/shallow.lock` left by a killed run). The 2026-11-09 rebuild was verified end-to-end: `cg server start -m Qwen3.8-27B-UD-Q3_K_XL` boots with `-md` + `--spec-type draft-dflash --spec-draft-n-max 4`, and a 150-run Forge eval through :8081 scored 150/150 completion / 149/150 correctness in 29.7 min. Switch `PINNED_REPO` back to ggml-org once DSpine lands upstream.
|
|
323
343
|
- **PORTS**: llama-server backend = `:8080`, guardrails proxy = `:8081`. Never run llama-server on `:8081` (collides with the proxy).
|
|
324
344
|
- **Qwen3.5-MTP `missing tensor 'blk.32.ssm_conv1d.weight'` error** = an OLD llama.cpp build (build 8276-era). The GGUF is correct (33 layers; blk.32 is an attention layer, not SSM). Build ≥ `5a6a0dd` reads the real layout and loads fine. If this recurs, the culprit is a stale binary on PATH / a vendored checkout, not the file.
|
|
@@ -334,12 +354,17 @@ coding-guardrails serve \
|
|
|
334
354
|
- **Gemma 4 12B Unified**: Dense 12B, 256K ctx, encoder-free multimodal (text+image+audio). Only ~8 GB VRAM at Q4 — massive headroom on 24 GB cards. **No MTP yet** (llama.cpp issue #22747). Sampling: temp=1.0, top_k=64, top_p=0.95.
|
|
335
355
|
- **Gemma 4 26B A4B QAT**: MoE (25.2B total / 3.8B active), native 256K ctx (run at 200K). ~14.25 GB weights, **~20 GB VRAM at 200K** with q8_0 KV cache — needs `-ctk q8_0 -ctv q8_0`. Sliding-window attention (5 global of 30 layers) keeps the KV cache tiny. Highest capability (88.3% AIME, 77.1% LiveCodeBench). **No MTP.** Use the **Unsloth UD-Q4_K_XL** QAT GGUF only — naive Q4_0 loses 15.4pp top-1.
|
|
336
356
|
- `SafeLlamafileClient` needs `gguf_path` (stem = model name): use `/tmp/<model-name>.gguf`
|
|
337
|
-
- **Tool-call cliff
|
|
338
|
-
|
|
339
|
-
flaky
|
|
340
|
-
min(profile_ctx, profile.context_budget or `--context-budget`), default 12000
|
|
341
|
-
TieredCompact fires at ~75% (~9K)
|
|
342
|
-
|
|
357
|
+
- **Tool-call cliff is MODEL-SPECIFIC — do not generalise it.** Measured 2026-08-31 on the
|
|
358
|
+
*base* Qwen3.8-27B: tool-calling collapses to prose at ~20-27K prompt tokens through the
|
|
359
|
+
proxy (solid <~11K, flaky 13-19K regardless of temperature). The serving ContextManager
|
|
360
|
+
budgets min(profile_ctx, profile.context_budget or `--context-budget`), default 12000, so
|
|
361
|
+
the base profile's `context_budget` stays 12000 and TieredCompact fires at ~75% (~9K).
|
|
362
|
+
**RE-PROBED 2026-11-19 on the pi-adapted LoRA (`Qwen3.8-27B-pi-adapted`): the cliff does
|
|
363
|
+
NOT reproduce** — 24/24 tool calls at 16K/24K/32K/49K/65K/98K prompt tokens backend-direct,
|
|
364
|
+
19/20 through the proxy at `--context-budget 120000` (the single miss uncorrelated with
|
|
365
|
+
length; 98K clean). That profile therefore ships `context_budget=120000`
|
|
366
|
+
(TieredCompact at ~90K). Long-horizon *quality* at 90K is still unmeasured — the probe
|
|
367
|
+
only proves the model does not fall back to prose. Probe: eval/scripts/probe_longctx.py.
|
|
343
368
|
- `recommended_sampling=False` — model not in Forge's registry
|
|
344
369
|
- any md file writing should be in gitignored plans/ folder.
|
|
345
370
|
- use one subagent at a time , no parallel agents.
|
|
@@ -347,11 +372,11 @@ coding-guardrails serve \
|
|
|
347
372
|
## Testing
|
|
348
373
|
|
|
349
374
|
```bash
|
|
350
|
-
pytest tests/unit/ -q # All
|
|
375
|
+
pytest tests/unit/ -q # All 810 tests
|
|
351
376
|
pytest tests/unit/ -q -k "loop" # Specific rule
|
|
352
377
|
```
|
|
353
378
|
|
|
354
|
-
All
|
|
379
|
+
All 810 tests must pass before committing.
|
|
355
380
|
|
|
356
381
|
## Eval
|
|
357
382
|
|
|
@@ -380,15 +405,28 @@ python eval/scripts/run_security_eval.py --exfil-battery -v \
|
|
|
380
405
|
|
|
381
406
|
Results go to `eval/runs/<timestamp>/` (gitignored).
|
|
382
407
|
|
|
383
|
-
**Best result: 150/150 completion (100%)
|
|
408
|
+
**Best result: 150/150 completion (100%) AND 150/150 accuracy (100%)** — Inkling 2 Mini
|
|
409
|
+
(26.4B/3.4B-active looped MoE) on 2026-11-22, 69.5 min wall (4172.8s), every one of the 30
|
|
410
|
+
scenarios 5/5. It is the slowest full run on record — 1.83x the 2026-08-15 Qwen3.8-27B
|
|
411
|
+
baseline (37.9 min) and 2.05x Qwen3.8-27B+DSpine (33.8 min) — and it holds tool-calling to
|
|
412
|
+
98K ctx (10/10) where base Qwen3.8-27B fell back to prose at ~20-27K. See docs/models.md
|
|
413
|
+
— Inkling 2 Mini, and its wall-time table for the baseline caveat.
|
|
414
|
+
|
|
415
|
+
Previous bests — **150/150 completion (100%)** on Forge 30-scenario eval (Qwen3.5-9B + Ornith,
|
|
384
416
|
re-eval 2026-08-08 under v0.16.1 — the old "93% (140/150)" included the v0.7.4 respond() bug
|
|
385
417
|
that zeroed tool_selection for every model). Qwen3.6-27B: 149/150 (99.3%); LFM2.5: 139/150
|
|
386
418
|
(92.7%). Accuracy: Ornith 95%, Qwen3.6-27B 94%, Qwen3.5-9B 92%, LFM2.5 71%.
|
|
387
|
-
**Qwen3.8-27B + DSpine (v0.26.0 default): 150/150 completion, 149/150 correctness (99%),
|
|
419
|
+
**Qwen3.8-27B + DSpine (v0.26.0 default; v0.27.0 adds the pi LoRA on top): 150/150 completion, 149/150 correctness (99%),
|
|
388
420
|
33.8 min wall on the shipped profile (120K ctx, q4_0 KV, managed proxy) / 29.7 min at 98K
|
|
389
|
-
q8_0 KV** — the same
|
|
390
|
-
(150/150, 146/150), so DSpine is
|
|
421
|
+
q8_0 KV** — the same-day control without the drafter takes 64.1 min
|
|
422
|
+
(150/150, 146/150), so DSpine is **1.89x against its own control**, 2.16x at 98K,
|
|
391
423
|
at no accuracy cost.
|
|
424
|
+
⚠️ **Do not compare that 1.89x across dates.** The 2026-08-15 un-drafted run of the
|
|
425
|
+
same model took **37.9 min** (2276.9s), i.e. the un-drafted baseline itself is 1.69x
|
|
426
|
+
slower by 11-09 — with identical iteration counts, so it is per-turn cost (prompt
|
|
427
|
+
length / context budget / KV type), not retries. `run.json` does not record ctx,
|
|
428
|
+
context_budget or boot flags, so the cause is **unattributed**; only compare runs whose
|
|
429
|
+
profile you can state. Wall-time table with run IDs: docs/models.md.
|
|
392
430
|
**MiniCPM5-2B (added v0.20.0 as the tiny tier): 150/150 (100%) completion, 128/150 (85%) accuracy**
|
|
393
431
|
— tool_selection 5/5 (+stateful) and data_gap_recovery_extended 100% fix LFM2.5's genuine gaps;
|
|
394
432
|
only argument_transformation (0-20%) and inconsistent_api_recovery (40%) stay weak.
|
|
@@ -414,7 +452,7 @@ files — re-sync `tests/eval` from forge upstream after any vendor refresh.
|
|
|
414
452
|
## Development Guidelines
|
|
415
453
|
|
|
416
454
|
- **Do NOT hack Forge source** — extend via public API, subclassing, wrapping
|
|
417
|
-
- All
|
|
455
|
+
- All 810 unit tests must pass
|
|
418
456
|
- No hardcoded scenario-specific logic
|
|
419
457
|
- Block responses must return **text**, not empty tool calls
|
|
420
458
|
- Enforcement prompts must mention `respond()` as the exit tool
|
|
@@ -445,7 +483,7 @@ Every release follows these steps **in order**. Do not skip any step.
|
|
|
445
483
|
|
|
446
484
|
```bash
|
|
447
485
|
source .venv/bin/activate
|
|
448
|
-
pytest tests/unit/ -q # All
|
|
486
|
+
pytest tests/unit/ -q # All 810 tests MUST pass
|
|
449
487
|
```
|
|
450
488
|
|
|
451
489
|
If any test fails → **stop**, fix, re-run. Do not proceed.
|
|
@@ -1,5 +1,29 @@
|
|
|
1
1
|
# Session History
|
|
2
2
|
|
|
3
|
+
## 2026-11-19 — Qwen3.8-27B-pi-adapted (LoRA) becomes the default; thinking-on; 160K context
|
|
4
|
+
|
|
5
|
+
- **New profile `Qwen3.8-27B-pi-adapted`** — the pi-adapted LoRA from harness-hub
|
|
6
|
+
(319 MB F16) over the existing `UD-Q3_K_XL` base, i.e. no second 13 GB download. RL-trained
|
|
7
|
+
inside unmodified pi. Card (corrected): SWE-bench Pro hard-51 39.2% -> 64.7%,
|
|
8
|
+
Terminal-Bench 2.1 39.5% -> 55.8%, turns per solved task 23 vs 59. Sibling
|
|
9
|
+
`Qwen3.8-27B-pi-adapted-nothink` pins `--reasoning off`.
|
|
10
|
+
- **LoRA plumbing** mirrors the drafter path: `ModelProfile.lora` -> `LORAS` registry ->
|
|
11
|
+
`--lora` in `build_argv` -> `download_lora`. Missing adapter warns loudly and serves the
|
|
12
|
+
base model rather than refusing to boot.
|
|
13
|
+
- **Thinking-on by default** (`--auto-no-thinking` flipped to off in `cli.py`,
|
|
14
|
+
`proxy/server.py` and `proxy/handler.py`): the proxy no longer forces
|
|
15
|
+
`enable_thinking=false` on no-tool requests, including for direct `_handle` callers.
|
|
16
|
+
Driven by the new default model, whose reasoning is the point.
|
|
17
|
+
- **Context 120000 -> 160000** for the adapted profile after a local TTS server freed ~1 GB
|
|
18
|
+
(~26 KB VRAM per context token; 160K leaves ~1.4 GB free, 192K only 0.7 GB).
|
|
19
|
+
- **`context_budget` 12000 -> 120000 for the adapted profile only.** The base model's
|
|
20
|
+
measured 2026-08-31 tool-call cliff (~20-27K prompt tokens) does NOT reproduce on the
|
|
21
|
+
adapter: 24/24 tool calls at 16K-98K backend-direct, 19/20 through the proxy at a 120K
|
|
22
|
+
budget. The base profile keeps 12000 — its cliff is real.
|
|
23
|
+
- Verified: `cg server start -m Qwen3.8-27B-pi-adapted` boots adapter scale 1.0 + DSpine
|
|
24
|
+
n_max=4 together at 160K ctx (21.1 GB, ~1.4 GB free); end-to-end `pi --print` fixed a
|
|
25
|
+
failing 4-test repo in 67 s. Decode 60-63 tok/s vs 70.6 without the adapter. 810 tests.
|
|
26
|
+
|
|
3
27
|
## 2026-10-28 — v0.24.0: LFM3-1.7B (256K long-context tiny tier) + llama.cpp pin bump
|
|
4
28
|
- **Pin bump**: `d77256999` → `39febd2ad` (llama.cpp 0.8.0-dev) for `LLM_ARCH_LFM3`
|
|
5
29
|
(#31252 text+VL, #31257 audio+omni, #31283 video input). Rebuilt, `cg server version`
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: coding-guardrails
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.28.0
|
|
4
4
|
Summary: A safety + reliability proxy for coding agents on local LLMs. Forge rescue/retry/validation + 13 composable guardrail rules. Measured on the Forge 30-scenario eval: 100% completion, up to 95% accuracy.
|
|
5
5
|
Project-URL: Homepage, https://github.com/stawils/coding-guardrails
|
|
6
6
|
Project-URL: Repository, https://github.com/stawils/coding-guardrails
|
|
@@ -86,9 +86,15 @@ policy. Neither requires rewriting your agent.
|
|
|
86
86
|
pip install coding-guardrails
|
|
87
87
|
|
|
88
88
|
coding-guardrails server build # builds cg's llama-server (pinned commit: llama.cpp 0.9.0-dev + DSpine draft support)
|
|
89
|
-
coding-guardrails server start --model Qwen3.8-27B-
|
|
89
|
+
coding-guardrails server start --model Qwen3.8-27B-pi-adapted # backend on :8080 (default — pi LoRA + DSpine draft, 160K ctx)
|
|
90
90
|
coding-guardrails serve --backend-url http://localhost:8080 \
|
|
91
|
-
--model Qwen3.8-27B-
|
|
91
|
+
--model Qwen3.8-27B-pi-adapted --port 8081 --context-budget 120000 # proxy on :8081
|
|
92
|
+
#
|
|
93
|
+
# Other tiers: Qwen3.8-27B-UD-Q3_K_XL (base, no adapter), flint-UD-Q3_K_XL,
|
|
94
|
+
# MiniCPM5-2B-Q4_K_M, LFM3-1.7B-Q8_0, Ornith-1.0-9B-Q8_0.
|
|
95
|
+
# The pi-adapted profile is a 319 MB LoRA over the Qwen3.8-27B-UD-Q3_K_XL GGUF
|
|
96
|
+
# (harness-hub, RL-trained inside the pi harness): fewer turns, terser tool calls.
|
|
97
|
+
# Thinking is ON by default; Qwen3.8-27B-pi-adapted-nothink pins it off.
|
|
92
98
|
|
|
93
99
|
# Point your agent at http://localhost:8081/v1
|
|
94
100
|
```
|
|
@@ -104,10 +110,10 @@ decoding, context size) per model — no flag archaeology.
|
|
|
104
110
|
|
|
105
111
|
| Flag | Default | What it does |
|
|
106
112
|
|------|---------|--------------|
|
|
107
|
-
| `--auto-no-thinking` |
|
|
113
|
+
| `--auto-no-thinking` | **off** | When ON, plain (no-tool) requests get `enable_thinking=false` forced on the wire. Default flipped to off 2026-11-19: the shipped model is `Qwen3.8-27B-pi-adapted`, whose reasoning is the point, and auto-stripping it from generation calls was surprising. A request can still opt in either way (`chat_template_kwargs.enable_thinking`, `reasoning_effort`). Tool calls always keep thinking on. Turn it back on for token-bound generation workloads. |
|
|
108
114
|
| `--thinking-budget-tokens` | 4096 | Token cap on the reasoning block for plain-path thinking (0 disables). Prevents reasoning from eating `max_tokens` and returning an empty answer; the empty-after-thinking case still retries once with thinking off. |
|
|
109
115
|
| `--reasoning-replay` | `keep-last` | How much model thinking reaches the agent: `keep-last` (thinking in the `reasoning_content` field), `full` (thinking as message content), `none` (observability only). forge ≥0.7.5 defaults to `none` — set this or thinking silently vanishes from responses. |
|
|
110
|
-
| `--context-budget` | 12000 | Layer-1 compaction budget (tokens)
|
|
116
|
+
| `--context-budget` | 12000 | Layer-1 compaction budget (tokens); `TieredCompact` fires at ~75%. The cliff behind this default is **model-specific**: the base Qwen3.8-27B collapses to prose at ~20–27K prompt tokens (measured 2026-08-31), but the pi-adapted LoRA holds tool calls to at least 98K (measured 2026-11-19: 24/24 backend-direct, 19/20 through the proxy) and ships `context_budget=120000`. Per-model override: `ModelProfile.context_budget`. |
|
|
111
117
|
| `--convergence-nudge-after` | 0 (off) | Experimental finalize-now reminder after N tool-call turns. Measured NOT to fix open-ended task drift — bounded task templates do. |
|
|
112
118
|
|
|
113
119
|
## The 15 guardrail rules
|
|
@@ -333,7 +339,7 @@ limits; workspace root for path safety; secret patterns. Full reference in
|
|
|
333
339
|
|
|
334
340
|
```bash
|
|
335
341
|
source .venv/bin/activate
|
|
336
|
-
pytest tests/unit/ -q #
|
|
342
|
+
pytest tests/unit/ -q # 810 tests (~25s)
|
|
337
343
|
python eval/scripts/run_forge_eval.py --mode proxy --runs 5 # the eval behind the tables above
|
|
338
344
|
```
|
|
339
345
|
|
|
@@ -54,9 +54,15 @@ policy. Neither requires rewriting your agent.
|
|
|
54
54
|
pip install coding-guardrails
|
|
55
55
|
|
|
56
56
|
coding-guardrails server build # builds cg's llama-server (pinned commit: llama.cpp 0.9.0-dev + DSpine draft support)
|
|
57
|
-
coding-guardrails server start --model Qwen3.8-27B-
|
|
57
|
+
coding-guardrails server start --model Qwen3.8-27B-pi-adapted # backend on :8080 (default — pi LoRA + DSpine draft, 160K ctx)
|
|
58
58
|
coding-guardrails serve --backend-url http://localhost:8080 \
|
|
59
|
-
--model Qwen3.8-27B-
|
|
59
|
+
--model Qwen3.8-27B-pi-adapted --port 8081 --context-budget 120000 # proxy on :8081
|
|
60
|
+
#
|
|
61
|
+
# Other tiers: Qwen3.8-27B-UD-Q3_K_XL (base, no adapter), flint-UD-Q3_K_XL,
|
|
62
|
+
# MiniCPM5-2B-Q4_K_M, LFM3-1.7B-Q8_0, Ornith-1.0-9B-Q8_0.
|
|
63
|
+
# The pi-adapted profile is a 319 MB LoRA over the Qwen3.8-27B-UD-Q3_K_XL GGUF
|
|
64
|
+
# (harness-hub, RL-trained inside the pi harness): fewer turns, terser tool calls.
|
|
65
|
+
# Thinking is ON by default; Qwen3.8-27B-pi-adapted-nothink pins it off.
|
|
60
66
|
|
|
61
67
|
# Point your agent at http://localhost:8081/v1
|
|
62
68
|
```
|
|
@@ -72,10 +78,10 @@ decoding, context size) per model — no flag archaeology.
|
|
|
72
78
|
|
|
73
79
|
| Flag | Default | What it does |
|
|
74
80
|
|------|---------|--------------|
|
|
75
|
-
| `--auto-no-thinking` |
|
|
81
|
+
| `--auto-no-thinking` | **off** | When ON, plain (no-tool) requests get `enable_thinking=false` forced on the wire. Default flipped to off 2026-11-19: the shipped model is `Qwen3.8-27B-pi-adapted`, whose reasoning is the point, and auto-stripping it from generation calls was surprising. A request can still opt in either way (`chat_template_kwargs.enable_thinking`, `reasoning_effort`). Tool calls always keep thinking on. Turn it back on for token-bound generation workloads. |
|
|
76
82
|
| `--thinking-budget-tokens` | 4096 | Token cap on the reasoning block for plain-path thinking (0 disables). Prevents reasoning from eating `max_tokens` and returning an empty answer; the empty-after-thinking case still retries once with thinking off. |
|
|
77
83
|
| `--reasoning-replay` | `keep-last` | How much model thinking reaches the agent: `keep-last` (thinking in the `reasoning_content` field), `full` (thinking as message content), `none` (observability only). forge ≥0.7.5 defaults to `none` — set this or thinking silently vanishes from responses. |
|
|
78
|
-
| `--context-budget` | 12000 | Layer-1 compaction budget (tokens)
|
|
84
|
+
| `--context-budget` | 12000 | Layer-1 compaction budget (tokens); `TieredCompact` fires at ~75%. The cliff behind this default is **model-specific**: the base Qwen3.8-27B collapses to prose at ~20–27K prompt tokens (measured 2026-08-31), but the pi-adapted LoRA holds tool calls to at least 98K (measured 2026-11-19: 24/24 backend-direct, 19/20 through the proxy) and ships `context_budget=120000`. Per-model override: `ModelProfile.context_budget`. |
|
|
79
85
|
| `--convergence-nudge-after` | 0 (off) | Experimental finalize-now reminder after N tool-call turns. Measured NOT to fix open-ended task drift — bounded task templates do. |
|
|
80
86
|
|
|
81
87
|
## The 15 guardrail rules
|
|
@@ -301,7 +307,7 @@ limits; workspace root for path safety; secret patterns. Full reference in
|
|
|
301
307
|
|
|
302
308
|
```bash
|
|
303
309
|
source .venv/bin/activate
|
|
304
|
-
pytest tests/unit/ -q #
|
|
310
|
+
pytest tests/unit/ -q # 810 tests (~25s)
|
|
305
311
|
python eval/scripts/run_forge_eval.py --mode proxy --runs 5 # the eval behind the tables above
|
|
306
312
|
```
|
|
307
313
|
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# Guardrail profile for benchmark runs (PLE-Bench / SWE-style evals), not for daily coding.
|
|
2
|
+
#
|
|
3
|
+
# Why this file exists
|
|
4
|
+
# --------------------
|
|
5
|
+
# The default profile is built to protect a developer's real repository. A benchmark
|
|
6
|
+
# task container is the opposite: throwaway, root-owned, and full of absolute paths
|
|
7
|
+
# that sit outside the proxy's cwd. Several default rules therefore either block
|
|
8
|
+
# legitimate work or silently MUTATE the content the agent reads — which corrupts a
|
|
9
|
+
# measurement while looking like model failure.
|
|
10
|
+
#
|
|
11
|
+
# Use with:
|
|
12
|
+
# coding-guardrails serve --backend-url http://localhost:8080 \
|
|
13
|
+
# --model <profile> --port 8081 --serialize \
|
|
14
|
+
# --config configs/ple-bench.yaml \
|
|
15
|
+
# --context-budget 19000
|
|
16
|
+
#
|
|
17
|
+
# (equivalently `--no-guardrails` if you also want path/command/secrets off)
|
|
18
|
+
#
|
|
19
|
+
# Measured 2026-11-19 on PLE dev task `backfeed-energization-reply`:
|
|
20
|
+
# - deliverables live at /filesystem/**, /app/output/**, /app/tools/.apps_data/**
|
|
21
|
+
# i.e. absolute, outside any workspace root -> default path_safety blocks them
|
|
22
|
+
# - the agent writes NEW files with no prior read -> `prerequisites` blocks after 4
|
|
23
|
+
# - the judge payload for one rubric task is 125,175 tokens -> needs --ctx 128000
|
|
24
|
+
# and the Layer-1 budget must not try to "compact" a single oversized message
|
|
25
|
+
|
|
26
|
+
# No workspace root: benchmark tasks are absolute-path-native and throwaway.
|
|
27
|
+
# With path_safety disabled this value is unused; kept explicit for readers.
|
|
28
|
+
workspace: "/"
|
|
29
|
+
|
|
30
|
+
# ── Disabled: they block or rewrite legitimate benchmark traffic ──
|
|
31
|
+
|
|
32
|
+
# Deliverables are absolute paths (/filesystem, /app/output) outside cwd.
|
|
33
|
+
# Keep this OFF for benchmarks. The container is disposable; there is nothing
|
|
34
|
+
# to protect that the harness does not already own.
|
|
35
|
+
path_safety:
|
|
36
|
+
enabled: false
|
|
37
|
+
|
|
38
|
+
# Blocks sudo / rm -rf / / mkfs / dd if= / pipe-to-shell / inline PATH=.
|
|
39
|
+
# Harmless to keep in a throwaway container and still catches a runaway agent.
|
|
40
|
+
command_safety:
|
|
41
|
+
enabled: true
|
|
42
|
+
strength: hard
|
|
43
|
+
blocked_commands:
|
|
44
|
+
- "rm -rf /"
|
|
45
|
+
- "rm -rf /*"
|
|
46
|
+
- "mkfs"
|
|
47
|
+
- ":(){ :|:& };:"
|
|
48
|
+
- "shutdown"
|
|
49
|
+
- "reboot"
|
|
50
|
+
- "init 0"
|
|
51
|
+
- "init 6"
|
|
52
|
+
|
|
53
|
+
# Masking keys is free and never wrong.
|
|
54
|
+
secrets:
|
|
55
|
+
enabled: true
|
|
56
|
+
strength: soft # mask and warn; never block a benchmark deliverable
|
|
57
|
+
mask_value: "[REDACTED]"
|
|
58
|
+
|
|
59
|
+
# OFF: blocks any tool call that echoes the planted token. A deliverable memo
|
|
60
|
+
# that quotes the system prompt would be blocked mid-write.
|
|
61
|
+
canary:
|
|
62
|
+
enabled: false
|
|
63
|
+
|
|
64
|
+
# OFF — this is the dangerous one. mode: mark INSERTS a spotlighting warning into
|
|
65
|
+
# tool results, so the agent reads text that is not in the task files. On a
|
|
66
|
+
# benchmark that is silent content corruption.
|
|
67
|
+
input_scanning:
|
|
68
|
+
enabled: false
|
|
69
|
+
mode: off
|
|
70
|
+
|
|
71
|
+
# OFF: requires a prior read before write/edit, blocks after 4 violations.
|
|
72
|
+
# Benchmark tasks require writing brand-new deliverables.
|
|
73
|
+
prerequisites:
|
|
74
|
+
enabled: false
|
|
75
|
+
|
|
76
|
+
# OFF: nudges "run tests" after every edit. Pure token waste here.
|
|
77
|
+
sequencing:
|
|
78
|
+
enabled: false
|
|
79
|
+
|
|
80
|
+
# Keep: soft nudge when a tool returns empty. Helps weak local models recover.
|
|
81
|
+
tool_resolution:
|
|
82
|
+
enabled: true
|
|
83
|
+
strength: soft
|
|
84
|
+
|
|
85
|
+
# Keep the runaway backstop, but far above any real benchmark session.
|
|
86
|
+
session_budget:
|
|
87
|
+
enabled: true
|
|
88
|
+
max_file_ops: 20000
|
|
89
|
+
max_commands: 20000
|
|
90
|
+
max_reads: 0
|
|
91
|
+
warn_at: 0.9
|
|
92
|
+
|
|
93
|
+
# OFF: lints every edited file. Benchmark deliverables are .py/.xlsx/.docx the
|
|
94
|
+
# agent is still writing; lint noise competes with the actual task.
|
|
95
|
+
lint:
|
|
96
|
+
enabled: false
|
|
97
|
+
workspace: "/"
|
|
98
|
+
mode: nudge
|
|
@@ -127,19 +127,25 @@ Any agent that supports custom OpenAI API base URLs should work. Set:
|
|
|
127
127
|
|
|
128
128
|
## Thinking mode (plain/text requests)
|
|
129
129
|
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
models like Qwen3.8-27B
|
|
133
|
-
|
|
134
|
-
|
|
130
|
+
**Thinking is on by default, on both paths** (changed 2026-11-19). Tool-call
|
|
131
|
+
requests always kept thinking; plain (no-tool) requests used to have it forced
|
|
132
|
+
off, because reasoning models like the base Qwen3.8-27B would burn the budget on
|
|
133
|
+
reasoning that was then discarded. The shipped model is now
|
|
134
|
+
`Qwen3.8-27B-pi-adapted`, whose reasoning is the point, so `--auto-no-thinking`
|
|
135
|
+
defaults to **off** and the proxy no longer rewrites `enable_thinking`.
|
|
135
136
|
|
|
136
|
-
|
|
137
|
+
Consequence to know about: a plain request now spends tokens thinking. Give a
|
|
138
|
+
reasoning model a real `max_tokens` — with a small cap the model can think its
|
|
139
|
+
way to an empty answer, and the proxy's one retry (thinking off) is what saves
|
|
140
|
+
it, at the cost of the reasoning block.
|
|
141
|
+
|
|
142
|
+
To force thinking OFF (token-bound generation), opt out per request (any one works):
|
|
137
143
|
|
|
138
144
|
```bash
|
|
139
145
|
# 1) Simplest — model-native thinking on
|
|
140
146
|
curl -s http://127.0.0.1:8081/v1/chat/completions \
|
|
141
147
|
-H 'Content-Type: application/json' \
|
|
142
|
-
-d '{"model":"Qwen3.8-27B-
|
|
148
|
+
-d '{"model":"Qwen3.8-27B-pi-adapted",
|
|
143
149
|
"messages":[{"role":"user","content":"Design a migration plan for X"}],
|
|
144
150
|
"chat_template_kwargs":{"enable_thinking":true}}'
|
|
145
151
|
|
|
@@ -164,7 +170,7 @@ Server-wide knobs:
|
|
|
164
170
|
|
|
165
171
|
| Flag | Effect |
|
|
166
172
|
|---|---|
|
|
167
|
-
| `--
|
|
173
|
+
| `--auto-no-thinking` | Force `enable_thinking=false` on plain requests (default **off** since 2026-11-19) |
|
|
168
174
|
| `--thinking-budget-tokens N` | Reasoning token cap for plain thinking (default 4096) |
|
|
169
175
|
| `--reasoning-replay {full,keep-last,none}` | How reasoning is delivered: `keep-last` → `reasoning_content` field (default), `full` → merged into content, `none` → dropped |
|
|
170
176
|
|
|
@@ -54,12 +54,17 @@ Response flows back through both layers to agent
|
|
|
54
54
|
|
|
55
55
|
Forge handles the "make it actually work" layer:
|
|
56
56
|
|
|
57
|
-
- **Context budget (
|
|
58
|
-
`min(profile_ctx, profile.context_budget or --context-budget)
|
|
59
|
-
|
|
60
|
-
prose at ~20-27K prompt tokens
|
|
61
|
-
|
|
62
|
-
|
|
57
|
+
- **Context budget (model-specific cliff):** the serving `ContextManager` budgets
|
|
58
|
+
`min(profile_ctx, profile.context_budget or --context-budget)`, default 12000
|
|
59
|
+
tokens, and TieredCompact fires at ~75%. That default encodes a *measurement of
|
|
60
|
+
one model*: the base Qwen3.8-27B collapses to prose at ~20-27K prompt tokens
|
|
61
|
+
(2026-08-31; solid below ~11K; flaky 13-19K regardless of temperature).
|
|
62
|
+
**It does not generalise.** Re-probed 2026-11-19 on `Qwen3.8-27B-pi-adapted`
|
|
63
|
+
(LoRA over the same base): 4/4 tool calls at 16K/24K/32K/49K/65K/98K prompt
|
|
64
|
+
tokens backend-direct (24/24 total), 19/20 through the proxy at a 120000 budget,
|
|
65
|
+
the single miss uncorrelated with length. That profile ships
|
|
66
|
+
`context_budget=120000` (compaction at ~90K). Long-horizon quality at 90K is
|
|
67
|
+
unmeasured; the probe only rules out prose fallback. `--reasoning-replay keep-last` (default) delivers thinking to the
|
|
63
68
|
agent in `reasoning_content` — forge >=0.7.5's converter default `none`
|
|
64
69
|
would silently drop it from the wire.
|
|
65
70
|
- **Rescue parsing** — When the model outputs tool calls as text (not JSON),
|