drydock-cli 2.8.6__tar.gz → 2.8.8__tar.gz

drydock_cli-2.8.8/.auto_release.lock

@@ -0,0 +1 @@
+4191814

drydock_cli-2.8.8/DEEP_NOIR_PRD.md

@@ -0,0 +1,111 @@
+# Deep Noir activation-patching PRD — reasoning steering, transformers-backed sidecar
+
+**Status:** Draft, 2026-05-09. Phase 3 of HLE_PRD.
+
+The v0 token-bias path (`LogitBiasSteeringApplier`) hit its expected ceiling: phase1_steered scored 1/20 = 5% on the worked-example seed, identical to the unsteered baseline. This PRD scopes the next-tier applier — real activation patching at a configurable decoder layer — using a `transformers`-backed sidecar process so we can validate the architecture before the bigger vLLM forward-hook port. Goal: get one steering vector to lift Phase 1 above 5/20 on the existing 20-question seed, proving reasoning steering works at all on quantized Gemma 4. Then port to vLLM for production speed.
+
+## Why a sidecar (vs. patching the production server)
+
+Production inference is llama.cpp Docker (remus, b9014). It does not expose a per-layer hidden-state hook, and rebuilding it with a custom hook costs weeks. vLLM has a `custom_ops` extension point but our prod stack moved off vLLM in May; rebuilding it for steering only would block other work.
+
+A separate `transformers`-backed Python process gives us full forward-pass control today (`model.layers[L].register_forward_hook`), accepts the existing OpenAI-compatible request shape via a thin FastAPI wrapper, and runs on the same GPU pool. It will be **~10× slower than llama.cpp** because there's no batching/quant kernel, but research speed is the bottleneck right now, not throughput. Once one vector lifts Phase 1 we know the work is worth porting; until then, simple wins.
+
+## Architecture
+
+```
+client (drydock TUI / hle_eval)
+   │ chat/completions
+   ▼
+llm_balancer (:8001)
+   │ routes by header X-Drydock-Steering
+   ├─ no header  → llama.cpp :8000 (current prod, fast, no steering)
+   └─ has header → drydock-steered :8002 (new sidecar, slow, hooks active)
+```
+
+- **`X-Drydock-Steering: show_work@18×0.6,verify@22×0.4`** — comma-separated `mode@layer×scale` triples. Empty/missing → balancer falls through to llama.cpp.
+- **Sidecar process** (`drydock/steering/sidecar/server.py`):
+  - Loads Gemma 4 26B via `transformers.AutoModelForCausalLM` with the AWQ-4bit weights from `/data3/Models/Gemma-4-26B-A4B-it-AWQ-4bit/`.
+  - Pre-registers forward hooks on `model.layers[0..N]`. Each hook reads the requested mode/layer/scale from a thread-local context var and adds `scale × vector` to the residual stream.
+  - Exposes `/v1/chat/completions` (OpenAI shape) so the existing balancer can route to it transparently.
+- **`agent_loop.py` change**: when a `SteeringDecision` is active, set the request header `X-Drydock-Steering: <serialized>`. No payload change.
+
+The wire-level result: one HTTP header is the entire integration surface between the harness and the steered inference path. Adding/removing modes is a config change, not a code change.
+
+## Vector training
+
+For each candidate mode, the data pipeline is:
+
+1. **Extract contrastive pairs** from `~/.drydock/admiral_history.log` and stress run logs. A pair = `(prompt, derailed_chain, good_chain)` where derailed = stalled/looped/wrong-answer turn, good = same-prompt turn that resolved correctly. Tag schema lives in `drydock/admiral/schema.py` (already records turn-level outcome).
+2. **Capture residual streams**. Run each pair through the unsteered sidecar with hooks recording `hidden_states[L]` at every layer for the assistant's first 50 tokens. Save as `.pt` per pair.
+3. **Compute the direction**. For layer L, the vector is `mean(good_streams[L]) - mean(derailed_streams[L])`, L1-normalized. Stored as `.npy` + manifest `.toml` in `~/.drydock/steering/vectors/<mode>/<layer>.npy`.
+4. **Validate via sandbox**. The existing `drydock/steering/sandbox.py` harness runs a fixed prompt set with steering on/off and diffs outputs. Pass = no regression on a control set + measurable behavior change on the target set.
+
+Initial three modes to train (chosen because they have the most signal in admiral_history):
+
+| Mode | Direction | Target layer | Hypothesis |
+|---|---|---|---|
+| `show_work` | derailed = single-line answers; good = step-by-step intermediate computations | 16-20 | Push the model to write intermediate steps before committing to a final answer |
+| `verify` | derailed = wrong-answer-then-stop; good = wrong-answer-then-recheck-then-corrected | 18-22 | Push toward second-pass verification before emitting `ANSWER:` |
+| `cite_source` | derailed = made-up specifics; good = "per the chunk above," explicit quote | 14-18 | When retrieval surfaces an authoritative chunk, push toward grounding rather than re-deriving |
+
+`show_work` is the obvious first one to validate against Phase 1's 1/20 floor.
+
+## Milestones
+
+| # | Deliverable | Sized at |
+|---|---|---|
+| 1 | `drydock/steering/sidecar/server.py` — bare FastAPI + transformers loader on :8002, no hooks yet, returns identical outputs to llama.cpp on a smoke prompt. Container or systemd unit. | 1 day |
+| 2 | Forward-hook injection scaffolding. Header parsing, context-var dispatch, vector load on demand. Hook is a no-op (adds zero) so we verify the integration end-to-end with no behavior change. | 1 day |
+| 3 | Vector capture pipeline. Script that takes `(prompt, completion)` pairs, runs them through sidecar with capture-mode hooks, saves residual streams. | 1 day |
+| 4 | Vector training. Compute direction per layer from captured streams, write `.npy` + `.toml`. Two pass: a quick "any non-zero direction works" pre-flight, then real training on real pairs. | 1 day |
+| 5 | First end-to-end run: `show_work@18×0.5` against Phase 1's 20 questions. | 0.5 day (eval is 2-3h compute) |
+| 6 | If lift > 0/20, train two more (`verify`, `cite_source`) and re-run. If lift = 0/20, iterate on layer choice and scale before adding modes. | open-ended |
+| 7 | Optional follow-on: vLLM forward-hook port for production speed, once at least one vector lifts a real benchmark. | 1-2 weeks, only after #6 succeeds |
+
+Total to first signal: ~4-5 days of focused work. The biggest single risk is #4 producing junk vectors because the contrastive pair set is too small or too noisy.
+
+## Success criteria
+
+- **Required:** end-to-end loop works. A request with `X-Drydock-Steering: show_work@18×0.5` returns a different completion than the same request with no header, on the same prompt, deterministic seed. Sandbox eval reports a structured diff. (This is the architecture validation — proves the wiring is real, independent of whether any vector helps.)
+- **Aspirational:** `show_work` lifts Phase 1's 1/20 baseline by at least +1 question (≥10%). Confirms reasoning steering is a real lever for HLE-style tasks on a quantized 26B.
+- **Stretch:** Two of the three trained modes lift Phase 1 ≥10% individually; combined ≥3/20. Suggests vector composition isn't catastrophic and the path scales.
+
+## Constraints / non-goals
+
+- **AWQ-4bit only.** No fp16/bf16 path. Vectors are calibrated against the production runtime — no drift between sandbox and (eventual) prod.
+- **No vLLM port in this PRD.** Mentioned in milestone #7 as a follow-on; only triggers if the sidecar proves the architecture.
+- **Sidecar runs on existing GPU pool.** Either remus or romulus, but only one at a time — competes with stress run for VRAM. Schedule eval runs around stress windows.
+- **No "real-time learning."** Vectors are trained offline from logs and shipped as `.npy` files. The sidecar's only runtime job is the forward-pass injection; it does not adapt during a request.
+- **No mainline drydock dependency on the sidecar.** Sidecar can be down; balancer falls back to llama.cpp; production unaffected.
+
+## Decisions (locked 2026-05-09)
+
+- **Pair source: admiral_history only for v1.** ~3 weeks of stress + autonomous_review traces is the contrastive set. No synthetic pair generation in this PRD. If volume turns out to be insufficient (vector training produces near-zero directions), revisit in a follow-up.
+- **Layer sweep: informed picks first, full sweep as fallback.** Start with the ranges suggested in the modes table (`show_work` 16-20, `verify` 18-22, `cite_source` 14-18). Pick the midpoint of each range as the first attempt. If the informed pick fails to lift Phase 1 by ≥10%, escalate to a full sweep across all layers for that mode before declaring the mode dead.
+- **Stress run flips to steering-on by default once one vector lifts Phase 1.** Confirmed-working vectors become the new baseline for stress; the unsteered run becomes the control measured in periodic A/B comparisons rather than the default. Set via env var `DRYDOCK_STEERING_MODES=<mode1>,<mode2>` in `stress_babysitter.sh`'s spawn line.
+
+## File map (planned)
+
+```
+drydock/steering/
+├── sidecar/
+│   ├── server.py        # FastAPI + transformers, port :8002
+│   ├── hooks.py         # forward_hook implementations
+│   └── header_parser.py # X-Drydock-Steering parsing + validation
+├── train/
+│   ├── extract_pairs.py # admiral_history → contrastive .jsonl
+│   ├── capture.py       # streams pairs through sidecar in capture mode
+│   ├── compute_vector.py# good_mean - derailed_mean per layer
+│   └── README.md        # training pipeline runbook
+└── (existing applier.py, registry.py, vectors.py, sandbox.py — unchanged)
+```
+
+scripts/
+├── start_steering_sidecar.sh    # systemd unit equivalent for the sidecar
+└── train_steering_vector.sh     # one-shot wrapper around the train/ pipeline
+
+## Reference signals (from current scaffolding)
+
+- `drydock/steering/applier.py:14` — comment block already calls out `VllmSidecarSteeringApplier (TODO)` as the planned successor. This PRD picks up that thread and implements it as a transformers sidecar first.
+- `drydock/core/agent_loop.py:2310` — the steering hook is already wired per-request and env-gated. Adding the header path is a 5-line change.
+- `HLE_PRD.md` Phase 3 plan — names the three candidate modes (verify-before-answer, show-work-explicitly, consider-units, minimal-patch). This PRD adopts those as starting modes.

{drydock_cli-2.8.6 → drydock_cli-2.8.8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: drydock-cli
-Version: 2.8.6
+Version: 2.8.8
 Summary: Local-first CLI coding agent — tested with Gemma 4 26B via vLLM
 Project-URL: Homepage, https://github.com/fbobe321/drydock
 Project-URL: Repository, https://github.com/fbobe321/drydock

{drydock_cli-2.8.6 → drydock_cli-2.8.8}/drydock/core/tools/builtins/grep.py

@@ -118,28 +118,36 @@ class Grep(
         self, args: GrepArgs, ctx: InvokeContext | None = None
     ) -> AsyncGenerator[ToolStreamEvent | GrepResult, None]:
         backend = self._detect_backend()
+        args, regex_note = self._coerce_pattern(args)
         self._validate_args(args)
 
         exclude_patterns = self._collect_exclude_patterns()
         cmd = self._build_command(args, exclude_patterns, backend)
         stdout = await self._execute_search(cmd)
 
-        yield self._parse_output(
+        result = self._parse_output(
             stdout, args.max_matches or self.config.default_max_matches
         )
+        if regex_note:
+            result.matches = regex_note + result.matches
+        yield result
 
-    def _validate_args(self, args: GrepArgs) -> None:
-        if not args.pattern.strip():
-            raise ToolError("Empty search pattern provided.")
-
+    def _coerce_pattern(self, args: GrepArgs) -> tuple[GrepArgs, str]:
+        """Auto-escape invalid regex to avoid ToolError loops; return note if coerced."""
         try:
             _re.compile(args.pattern)
-        except _re.error as regex_err:
+            return args, ""
+        except _re.error:
             escaped = _re.escape(args.pattern)
-            raise ToolError(
-                f"Invalid regex pattern: {regex_err}\n"
-                f"To search for this as literal text, use pattern={escaped!r}"
-            ) from None
+            note = (
+                f"NOTE: '{args.pattern}' is not valid regex; searching as literal text"
+                f" (pattern='{escaped}').\n"
+            )
+            return args.model_copy(update={"pattern": escaped}), note
+
+    def _validate_args(self, args: GrepArgs) -> None:
+        if not args.pattern.strip():
+            raise ToolError("Empty search pattern provided.")
 
         path_obj = Path(args.path).expanduser()
         if not path_obj.is_absolute():

drydock_cli-2.8.8/drydock/steering/sidecar/__init__.py

@@ -0,0 +1,28 @@
+"""Deep Noir activation-steering sidecar — transformers-backed inference.
+
+A separate Python process (default port 8002) that loads Gemma 4 via
+`transformers.AutoModelForCausalLM` so we have full forward-pass control
+for activation patching at configurable decoder layers. Speaks the
+OpenAI `/v1/chat/completions` shape so the existing `llm_balancer`
+can route to it transparently when an `X-Drydock-Steering` header is
+present on the request.
+
+Why a sidecar (not a vLLM patch): vLLM doesn't expose per-layer hidden-
+state hooks without a custom forward-pass build, and our production
+stack moved off vLLM in May 2026. transformers gives us
+`model.layers[L].register_forward_hook(...)` for free, at the cost of
+~10× lower throughput. Acceptable for research-velocity vector
+training and Phase 1 ablation eval; not for production stress.
+
+Public surface:
+    from drydock.steering.sidecar.server import build_app
+    app = build_app()                     # FastAPI app, lazy model load
+    # Then: uvicorn ...:app --port 8002
+
+CLI:
+    bash scripts/start_steering_sidecar.sh
+    bash scripts/start_steering_sidecar.sh --port 8002 --model /path/to/weights
+
+See DEEP_NOIR_PRD.md for the architecture, training pipeline, and
+milestones.
+"""

drydock_cli-2.8.8/drydock/steering/sidecar/header_parser.py

@@ -0,0 +1,74 @@
+"""Parse the `X-Drydock-Steering` HTTP header into structured directives.
+
+Wire format:
+    X-Drydock-Steering: <mode>@<layer>×<scale>[,<mode>@<layer>×<scale>...]
+
+The Unicode multiplication sign `×` is the separator between layer and
+scale. ASCII `x` is also accepted (for shells and clients that mangle
+Unicode).
+
+Examples:
+    X-Drydock-Steering: show_work@18×0.6
+    X-Drydock-Steering: show_work@18×0.6,verify@22×0.4
+    X-Drydock-Steering: show_work@18x0.6   (ASCII fallback)
+
+A missing or empty header → no steering, sidecar returns identical
+output to llama.cpp on the same prompt. Malformed entries are
+skipped with a logged warning rather than 400'd — the sidecar is
+defensive in depth, like the existing `agent_loop.py` steering hook.
+"""
+from __future__ import annotations
+
+import logging
+import re
+from dataclasses import dataclass
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass(frozen=True)
+class SteeringDirective:
+    """One mode/layer/scale triple parsed from the header."""
+    mode: str
+    layer: int
+    scale: float
+
+
+# `mode@layer×scale` or `mode@layerxscale`. mode is restricted to
+# [a-z0-9_-] to keep the wire format safe across HTTP intermediaries.
+_DIRECTIVE_RE = re.compile(
+    r"^(?P<mode>[a-z0-9_-]+)@(?P<layer>\d+)[×x](?P<scale>-?\d+(?:\.\d+)?)$"
+)
+
+
+def parse_header(value: str | None) -> list[SteeringDirective]:
+    """Parse the X-Drydock-Steering header value.
+
+    Returns the list of valid directives. Missing/empty header → []. A
+    malformed entry is dropped with a warning rather than raising; we
+    never want a header parse error to 5xx an otherwise-good request.
+    """
+    if not value:
+        return []
+    out: list[SteeringDirective] = []
+    for raw in value.split(","):
+        entry = raw.strip()
+        if not entry:
+            continue
+        m = _DIRECTIVE_RE.match(entry)
+        if not m:
+            logger.warning("steering header: ignoring malformed entry %r", entry)
+            continue
+        try:
+            out.append(
+                SteeringDirective(
+                    mode=m.group("mode"),
+                    layer=int(m.group("layer")),
+                    scale=float(m.group("scale")),
+                )
+            )
+        except (ValueError, OverflowError) as e:
+            logger.warning(
+                "steering header: ignoring %r (parse error: %s)", entry, e
+            )
+    return out

drydock_cli-2.8.8/drydock/steering/sidecar/server.py

@@ -0,0 +1,251 @@
+"""FastAPI server hosting a transformers-backed Gemma 4, OpenAI-compat shape.
+
+Milestone 1 deliverable per DEEP_NOIR_PRD.md:
+- Loads Gemma 4 from a local path via `transformers.AutoModelForCausalLM`.
+- Speaks `POST /v1/chat/completions` (non-streaming first cut) and
+  `GET /v1/models` so the existing `llm_balancer.py` can route to it
+  the same way it routes to llama.cpp.
+- NO forward hooks yet (Milestone 2). Returns identical outputs to
+  llama.cpp on a smoke prompt — that's the M1 acceptance check.
+
+Lazy model load: the model is NOT loaded at import time. The first
+inference request triggers `_load_model()` which holds a lock and
+caches the result. This lets the FastAPI app start in <1s and reserve
+its port even when GPU memory is tight; the heavy load only happens
+when something actually wants steered inference.
+
+Run:
+    bash scripts/start_steering_sidecar.sh
+or:
+    /home/bobef/miniconda3/bin/python3 -m uvicorn \\
+        drydock.steering.sidecar.server:app --host 0.0.0.0 --port 8002
+"""
+from __future__ import annotations
+
+import logging
+import os
+import threading
+import time
+import uuid
+from typing import Any
+
+from fastapi import FastAPI, HTTPException, Request
+from fastapi.responses import JSONResponse
+
+logger = logging.getLogger(__name__)
+
+# Default to the AWQ-4bit Gemma 4 weights (matches CLAUDE.md "active
+# model" path). Override with DRYDOCK_STEERING_SIDECAR_MODEL_PATH.
+_DEFAULT_MODEL_PATH = "/data3/Models/Gemma-4-26B-A4B-it-AWQ-4bit"
+_MODEL_PATH = os.environ.get(
+    "DRYDOCK_STEERING_SIDECAR_MODEL_PATH", _DEFAULT_MODEL_PATH
+)
+# transformers picks the device automatically when device_map="auto",
+# but operators often want to pin to a single GPU to avoid sharing
+# with llama.cpp. CUDA_VISIBLE_DEVICES handles that at the env layer.
+_DEVICE_MAP = os.environ.get("DRYDOCK_STEERING_SIDECAR_DEVICE_MAP", "auto")
+# Reported model name in /v1/models. Matches what `start_gemma4.sh`
+# reports so clients see a consistent name regardless of which
+# backend serves them.
+_REPORTED_MODEL_NAME = os.environ.get(
+    "DRYDOCK_STEERING_SIDECAR_MODEL_NAME", "gemma4"
+)
+
+
+_MODEL_LOCK = threading.Lock()
+_MODEL = None        # type: ignore[var-annotated]
+_TOKENIZER = None    # type: ignore[var-annotated]
+
+
+def _load_model() -> tuple[Any, Any]:
+    """Lazy + thread-safe model load. Cached after the first call.
+
+    Returns `(model, tokenizer)`. Raises RuntimeError on any underlying
+    transformers/torch failure — the FastAPI handler converts that into
+    a 503 so the balancer can fall back to llama.cpp.
+    """
+    global _MODEL, _TOKENIZER
+    if _MODEL is not None and _TOKENIZER is not None:
+        return _MODEL, _TOKENIZER
+    with _MODEL_LOCK:
+        if _MODEL is not None and _TOKENIZER is not None:
+            return _MODEL, _TOKENIZER
+        logger.info(
+            "steering sidecar: loading model from %s (device_map=%s)",
+            _MODEL_PATH, _DEVICE_MAP,
+        )
+        t0 = time.perf_counter()
+        try:
+            import torch  # local import — only need it when actually loading
+            from transformers import AutoModelForCausalLM, AutoTokenizer
+        except ImportError as e:
+            raise RuntimeError(
+                f"steering sidecar: missing dependency ({e}). Install "
+                "transformers + torch (CUDA build) to use the sidecar."
+            ) from e
+
+        try:
+            tokenizer = AutoTokenizer.from_pretrained(
+                _MODEL_PATH, trust_remote_code=True
+            )
+            model = AutoModelForCausalLM.from_pretrained(
+                _MODEL_PATH,
+                device_map=_DEVICE_MAP,
+                torch_dtype="auto",      # let the quant_config drive dtype
+                trust_remote_code=True,
+            )
+            model.eval()
+        except Exception as e:
+            raise RuntimeError(
+                f"steering sidecar: model load failed: {type(e).__name__}: {e}"
+            ) from e
+
+        elapsed = time.perf_counter() - t0
+        logger.info(
+            "steering sidecar: model loaded in %.1fs (param dtype=%s)",
+            elapsed,
+            getattr(next(iter(model.parameters()), torch.tensor(0)), "dtype", "?"),
+        )
+        _MODEL = model
+        _TOKENIZER = tokenizer
+        return _MODEL, _TOKENIZER
+
+
+def _now() -> int:
+    return int(time.time())
+
+
+def build_app() -> FastAPI:
+    app = FastAPI(
+        title="Drydock Deep Noir Steering Sidecar",
+        version="0.1.0-milestone1",
+    )
+
+    @app.get("/health")
+    async def health() -> dict[str, Any]:
+        return {
+            "status": "ok",
+            "model_loaded": _MODEL is not None,
+            "model_path": _MODEL_PATH,
+        }
+
+    @app.get("/v1/models")
+    async def list_models() -> dict[str, Any]:
+        return {
+            "object": "list",
+            "data": [
+                {
+                    "id": _REPORTED_MODEL_NAME,
+                    "object": "model",
+                    "created": _now(),
+                    "owned_by": "drydock-steering-sidecar",
+                }
+            ],
+        }
+
+    @app.post("/v1/chat/completions")
+    async def chat_completions(request: Request) -> JSONResponse:
+        try:
+            payload = await request.json()
+        except Exception as e:
+            raise HTTPException(status_code=400, detail=f"invalid json: {e}")
+        messages = payload.get("messages")
+        if not isinstance(messages, list) or not messages:
+            raise HTTPException(status_code=400, detail="missing messages[]")
+
+        # Milestone 1: ignore the steering header entirely. Just verify
+        # we can serve a completion. Hooks land in M2.
+        steering_header = request.headers.get("x-drydock-steering", "")
+        if steering_header:
+            logger.info(
+                "steering header received but no hooks active in M1: %r",
+                steering_header,
+            )
+
+        # Lazy load — pays the cost only on the first real request.
+        try:
+            model, tokenizer = _load_model()
+        except RuntimeError as e:
+            logger.error("model load failed: %s", e)
+            return JSONResponse(
+                status_code=503,
+                content={"error": {"message": str(e), "type": "model_load_failed"}},
+            )
+
+        # Apply the chat template — Gemma 4 has its own.
+        try:
+            prompt_ids = tokenizer.apply_chat_template(
+                messages,
+                add_generation_prompt=True,
+                return_tensors="pt",
+            )
+        except Exception as e:
+            raise HTTPException(
+                status_code=400, detail=f"chat template failed: {e}"
+            )
+
+        import torch
+
+        prompt_ids = prompt_ids.to(model.device)
+        max_new_tokens = int(payload.get("max_tokens") or 512)
+        # Gemma 4 anti-loop sampling defaults — match what we ship in
+        # DEFAULT_MODELS for the local model. Caller can override via
+        # the JSON payload.
+        temperature = float(payload.get("temperature", 1.0))
+        top_p = float(payload.get("top_p", 0.95))
+        top_k = int(payload.get("top_k", 40))
+        do_sample = temperature > 0
+
+        t0 = time.perf_counter()
+        with torch.no_grad():
+            output = model.generate(
+                prompt_ids,
+                max_new_tokens=max_new_tokens,
+                do_sample=do_sample,
+                temperature=temperature if do_sample else 1.0,
+                top_p=top_p if do_sample else 1.0,
+                top_k=top_k if do_sample else 0,
+                pad_token_id=tokenizer.eos_token_id,
+            )
+        gen_time = time.perf_counter() - t0
+
+        # Decode only the newly-generated portion.
+        new_tokens = output[0][prompt_ids.shape[-1]:]
+        completion = tokenizer.decode(new_tokens, skip_special_tokens=True)
+        prompt_token_count = int(prompt_ids.shape[-1])
+        completion_token_count = int(new_tokens.shape[-1])
+
+        logger.info(
+            "completion: prompt=%d completion=%d gen=%.2fs",
+            prompt_token_count, completion_token_count, gen_time,
+        )
+
+        return JSONResponse(
+            content={
+                "id": f"chatcmpl-{uuid.uuid4().hex[:24]}",
+                "object": "chat.completion",
+                "created": _now(),
+                "model": _REPORTED_MODEL_NAME,
+                "choices": [
+                    {
+                        "index": 0,
+                        "message": {
+                            "role": "assistant",
+                            "content": completion,
+                        },
+                        "finish_reason": "stop",
+                    }
+                ],
+                "usage": {
+                    "prompt_tokens": prompt_token_count,
+                    "completion_tokens": completion_token_count,
+                    "total_tokens": prompt_token_count + completion_token_count,
+                },
+            }
+        )
+
+    return app
+
+
+# uvicorn looks for `app` by default
+app = build_app()

{drydock_cli-2.8.6 → drydock_cli-2.8.8}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "drydock-cli"
-version = "2.8.6"
+version = "2.8.8"
 description = "Local-first CLI coding agent — tested with Gemma 4 26B via vLLM"
 readme = "README.md"
 requires-python = ">=3.12"

drydock_cli-2.8.8/scripts/start_steering_sidecar.sh

@@ -0,0 +1,52 @@
+#!/bin/bash
+# Start the Deep Noir activation-steering sidecar.
+#
+# Default port 8002. The model loads lazily on the first inference
+# request, so this script returns quickly even though the actual
+# Gemma 4 26B AWQ-4bit load takes ~30-60s of GPU time.
+#
+# Override defaults via env:
+#   DRYDOCK_STEERING_SIDECAR_MODEL_PATH=/data3/Models/<other>
+#   DRYDOCK_STEERING_SIDECAR_DEVICE_MAP=cuda:1   # pin to GPU 1
+#   DRYDOCK_STEERING_SIDECAR_MODEL_NAME=gemma4   # /v1/models id
+#
+# Usage:
+#   bash scripts/start_steering_sidecar.sh [--port 8002]
+#
+# The sidecar competes with llama.cpp for VRAM. Confirm
+# `nvidia-smi` shows ≥16 GB free before starting.
+set -euo pipefail
+
+PORT=8002
+HOST=0.0.0.0
+PYTHON=/home/bobef/miniconda3/bin/python3
+
+while [[ $# -gt 0 ]]; do
+    case $1 in
+        --port) PORT=$2; shift 2 ;;
+        --host) HOST=$2; shift 2 ;;
+        *) echo "unknown arg: $1" >&2; exit 2 ;;
+    esac
+done
+
+cd /data3/drydock
+
+LOGFILE=/data3/drydock/logs/steering_sidecar.log
+PIDFILE=/tmp/steering_sidecar.pid
+
+if [[ -f $PIDFILE ]] && kill -0 "$(cat "$PIDFILE")" 2>/dev/null; then
+    echo "steering sidecar already running (pid $(cat "$PIDFILE"))"
+    exit 0
+fi
+
+echo "[$(date)] starting steering sidecar on $HOST:$PORT (log: $LOGFILE)"
+nohup "$PYTHON" -m uvicorn \
+    drydock.steering.sidecar.server:app \
+    --host "$HOST" --port "$PORT" \
+    --log-level info \
+    >> "$LOGFILE" 2>&1 &
+
+echo $! > "$PIDFILE"
+echo "[$(date)] pid=$(cat "$PIDFILE")"
+echo "Tail: tail -f $LOGFILE"
+echo "Stop: kill \$(cat $PIDFILE)"