drydock-cli 2.7.42__tar.gz → 2.7.44__tar.gz

drydock_cli-2.7.44/.auto_release.lock

@@ -0,0 +1 @@
+2997693

drydock_cli-2.7.44/.gitignore

@@ -0,0 +1,222 @@
+# General
+.DS_Store
+.AppleDouble
+.LSOverride
+
+# Icon must end with two \r
+Icon
+
+
+# Thumbnails
+._*
+
+# Files that might appear in the root of a volume
+.DocumentRevisions-V100
+.fseventsd
+.Spotlight-V100
+.TemporaryItems
+.Trashes
+.VolumeIcon.icns
+.com.apple.timemachine.donotpresent
+
+# Directories potentially created on remote AFP share
+.AppleDB
+.AppleDesktop
+Network Trash Folder
+Temporary Items
+.apdisk
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.*cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/latest/usage/project/#working-with-version-control
+.pdm.toml
+.pdm-python
+.pdm-build/
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+.idea/
+
+# nix / direnv
+.direnv/
+result
+result-*
+
+# Vibe runtime/session files; keep tools as part of repo when needed
+.vibe/*
+
+# Tests run the agent in the playground, we don't need to keep the session files
+tests/playground/*
+.
+logs/
+test_bank_results/
+.pause_auto_release
+.pause_vllm_failover
+
+# Research loop: per-experiment outputs are high-churn and regenerate
+# from results.tsv + config_best.toml. Stop sentinel is intentional state.
+research/staged/
+research/traces/
+research/STOP
+
+# HLE eval outputs: per-run TUI logs, session symlinks, and copyrighted
+# HLE dataset content. Never commit.
+hle_results/
+
+# Untracked development clutter that shouldn't go to PyPI
+nohup.out
+.perf_baseline_done
+test/
+perf_results/baseline_*.json
+hle_results_iter/

drydock_cli-2.7.44/DEPLOYMENT.md

@@ -0,0 +1,276 @@
+# DryDock Deployment Guide
+
+The known-working configuration DryDock is developed and tested against.
+If you hit TUI loops, tool errors, or model hangs that look different
+from what the README describes, your setup is probably drifting from
+this baseline. This document is kept deliberately concrete — it mirrors
+a real working machine, not aspirational recommendations.
+
+## Hardware baseline
+
+| Component | What works |
+|---|---|
+| GPU | 2× NVIDIA RTX 4060 Ti 16 GB (32 GB total VRAM) |
+| Model | [Gemma-4-26B-A4B-it-AWQ-4bit](https://huggingface.co/casperhansen/gemma-4-26b-a4b-it-AWQ-4bit) |
+| RAM | 64 GB recommended (32 GB minimum) |
+| Disk | SSD, ≥100 GB free for model + session logs |
+| OS | Ubuntu 22.04, kernel 6.8 |
+
+A single 24 GB GPU can run the model with slightly reduced context.
+The AWQ quant is what makes 16 GB cards viable — non-quantized
+Gemma 4 26B needs ≥40 GB.
+
+## llama.cpp (recommended for tool-using workflows)
+
+Use this stack if drydock's TUI is your primary workflow. The `--jinja`
+flag is the loop-fix — without it, Gemma 4 enters infinite tool-call
+retry loops on multi-turn sessions (the 400 Bad Request loop fixed
+in v2.7.39, GH #14).
+
+```bash
+# 1. Download Unsloth GGUF (Q3_K_M is the article-recommended quant;
+#    UD-Q4_K_M is higher quality if you have ~17GB VRAM per slot)
+huggingface-cli download unsloth/gemma-4-26B-A4B-it-GGUF \
+    --include "gemma-4-26B-A4B-it-UD-Q3_K_M.gguf" \
+    --local-dir /path/to/models
+
+# 2. Build llama.cpp with CUDA, OR use the prebuilt Docker image
+#    ghcr.io/ggml-org/llama.cpp:server-cuda
+
+git clone https://github.com/ggerganov/llama.cpp.git
+cd llama.cpp
+cmake -B build -DGGML_CUDA=ON -DLLAMA_CURL=OFF
+cmake --build build --config Release -j8 --target llama-server
+
+# 3. Start the server (native binary)
+./build/bin/llama-server \
+    -m /path/to/models/gemma-4-26B-A4B-it-UD-Q3_K_M.gguf \
+    --host 0.0.0.0 --port 8000 \
+    -ngl 99 -c 32768 -np 1 \
+    --jinja \
+    -ctk q8_0 -ctv q8_0 \
+    --alias gemma4
+```
+
+**Don't omit** any of these flags — they're all load-bearing:
+
+| Flag | Why |
+|---|---|
+| `--jinja` | **The loop-fix.** Required for tool-using workflows. The GGUF's bundled chat template handles `tool` turns natively; without `--jinja`, tool results inject without proper turn markers and the model loops or returns empty assistants. |
+| `-ngl 99` | Offload all transformer layers to GPU. Anything less than full offload tanks decode speed on this model. |
+| `-c 32768` | 32K context. Fits in 16GB VRAM with q8 KV cache; raise to 65536 if you have headroom. |
+| `-ctk q8_0 -ctv q8_0` | Quantize K/V cache to q8 for longer contexts. f16 is ~2× slower per token at this size. |
+| `-np 1` | Single slot. Concurrent requests serialize — fine for one user, queue under load. |
+| `--alias gemma4` | What the API reports as the `model` field. Match this to your config.toml's `[[models]] name`. |
+
+**Critical client-side requirement:** temperature MUST be 1.0. Lower
+temperatures reinforce tool-call loops on Q3_K_M quants. The
+auto-detect path in v2.7.39+ (`local_detect.py`) bakes this in
+automatically when it sees a llama.cpp endpoint at first launch.
+
+## vLLM (alternative — higher decode throughput)
+
+Use this stack for batch eval / non-interactive workloads where you
+need ~70 tok/s decode (vs llama.cpp's ~15–17). The drydock-side
+empty-assistant filter (v2.7.39, GH #14) catches the loops vLLM is
+prone to without `--jinja`.
+
+Run via Docker:
+
+```bash
+docker run -d \
+    --gpus all \
+    --name gemma4 \
+    -p 8000:8000 \
+    -v /path/to/models:/models \
+    --ipc=host \
+    vllm/vllm-openai:gemma4 \
+    --model /models/Gemma-4-26B-A4B-it-AWQ-4bit \
+    --quantization compressed-tensors \
+    --tensor-parallel-size 2 \
+    --max-model-len 131072 \
+    --max-num-seqs 2 \
+    --gpu-memory-utilization 0.95 \
+    --kv-cache-dtype fp8 \
+    --served-model-name gemma4 \
+    --trust-remote-code \
+    --tool-call-parser gemma4 \
+    --enable-auto-tool-choice \
+    --attention-backend TRITON_ATTN
+```
+
+**Don't omit** any of these flags — they're all load-bearing:
+
+| Flag | Why |
+|---|---|
+| `--tool-call-parser gemma4` | Without this, Gemma 4 emits tool calls in text that the OpenAI-compat layer can't route. Tool calling silently breaks. |
+| `--enable-auto-tool-choice` | Required so `tool_choice="auto"` actually works. Without it the model can't decide between text and tool call. |
+| `--attention-backend TRITON_ATTN` | The default backend has periodic OOM spikes on 16 GB cards. TRITON is stable. |
+| `--kv-cache-dtype fp8` | Halves KV cache memory; required to reach 131K context on 32 GB total VRAM. |
+| `--max-num-seqs 2` | Prevents OOM under concurrent load. Raise on larger cards. |
+| `--gpu-memory-utilization 0.95` | Squeezes the last bit of VRAM. Drop to 0.85 if you see allocator warnings. |
+
+Single-GPU variant: drop `--tensor-parallel-size 2` and lower
+`--max-model-len` to something like 65536.
+
+### Sanity-check
+
+```bash
+curl http://localhost:8000/v1/models
+# Should return: {"data": [{"id": "gemma4", ...}]}
+
+curl http://localhost:8000/v1/chat/completions \
+    -H "Content-Type: application/json" \
+    -d '{"model":"gemma4","messages":[{"role":"user","content":"hi"}]}'
+```
+
+## Optional: load balancer / failover
+
+For long sessions, a thin HTTP proxy in front of vLLM survives brief
+vLLM restarts. Point DryDock at the proxy, not vLLM directly. See
+`scripts/llm_balancer.py` for reference; typical setup is:
+
+- Proxy on port 8001 → vLLM on port 8000
+- Keepalive cron respawns the balancer if it dies
+- `scripts/vllm_failover.sh` on a 5-min cron restarts vLLM if it's unreachable
+
+The development machine uses this setup. Skip it for first-time setup;
+return to it when you start hitting multi-hour sessions.
+
+## `~/.drydock/config.toml`
+
+The known-working DryDock config. Copy into `~/.drydock/config.toml`
+after first run (DryDock creates a stub on first launch; overwrite it):
+
+```toml
+active_model = "local"
+auto_compact_threshold = 120000
+api_timeout = 600.0
+system_prompt_id = "ralph"           # Gemma 4's simplified prompt
+slim_system_prompt = true
+enable_telemetry = true
+enable_notifications = true
+
+[[providers]]
+name = "vllm"
+api_base = "http://localhost:8000/v1"   # or :8001 if using llm_balancer
+api_style = "openai"
+backend = "generic"
+reasoning_field_name = "reasoning_content"
+
+[[models]]
+name = "gemma4"
+provider = "vllm"
+alias = "local"                       # matches active_model above
+temperature = 0.2
+thinking = "high"
+
+[session_logging]
+save_dir = "~/.vibe/logs/session"     # drydock writes sessions here
+session_prefix = "session"
+enabled = true
+
+[project_context]
+max_chars = 40000
+max_doc_bytes = 32768
+max_files = 1000
+max_depth = 3
+timeout_seconds = 2.0
+
+[tools.bash]
+permission = "always"
+max_output_bytes = 16000
+default_timeout = 300
+# See repo config for the full allowlist/denylist.
+
+[tools.read_file]
+permission = "always"
+max_read_bytes = 64000
+
+[tools.write_file]
+permission = "always"
+max_write_bytes = 64000
+create_parent_dirs = true
+
+[tools.search_replace]
+permission = "always"
+max_content_size = 100000
+fuzzy_threshold = 0.9
+
+[tools.grep]
+permission = "always"
+max_output_bytes = 64000
+default_max_matches = 100
+default_timeout = 60
+
+[tools.todo]
+permission = "always"
+max_todos = 100
+
+[tools.task]
+permission = "always"
+allowlist = ["explore", "diagnostic", "planner", "builder"]
+
+[tools.web_fetch]
+permission = "ask"         # don't auto-approve network egress
+default_timeout = 30
+max_content_bytes = 512000
+
+[tools.web_search]
+permission = "ask"
+```
+
+Critical fields:
+
+- `temperature = 0.2` — Gemma 4 gets unstable above ~0.4. Don't raise.
+- `thinking = "high"` — required for planning. DryDock adaptively
+  lowers it per-turn; setting it here is the default ceiling.
+- `system_prompt_id = "ralph"` — the Gemma-4-tuned prompt. The default
+  cli prompt confuses Gemma 4 into planning instead of acting.
+- `auto_compact_threshold = 120000` — below Gemma 4's 131K max context,
+  with headroom for the response.
+- `api_timeout = 600.0` — Gemma 4's first-turn thinking can legitimately
+  take 60–120 s. The default timeout is too short.
+
+## Environment variables
+
+| Var | Default | Why you'd change it |
+|---|---|---|
+| `DRYDOCK_AUTO_CONTINUE_DISABLE` | unset | Set to `1` to stop the auto-"Continue." injection that keeps Gemma 4 chaining tool calls. Useful for **stress test runs** and any workflow where text-only turns should cleanly end the user turn. Leave unset for production use unless you see Continue-loops. |
+| `DRYDOCK_INSECURE` | unset | Set to `1` to accept self-signed TLS (local LAN deployments). |
+| `DRYDOCK_LOCAL_URL` | `http://localhost:8000/v1` | Override the default local endpoint. |
+| `DRYDOCK_LOCAL_MODEL` | `local` | Model alias to target. |
+
+## Known-good network topology
+
+```
+user        drydock TUI           llm_balancer         vLLM
+cli   ───►  :stdin           ───►  :8001      ────►  :8000
+                                  │                    │
+                                  └── keepalive cron   └── keeps one
+                                       (5-min)              container up
+```
+
+## Diagnosing bad installs
+
+If DryDock acts up, compare your runtime to this baseline before
+filing an issue:
+
+1. **`curl localhost:8000/v1/models`** returns Gemma 4? If not, vLLM
+   isn't running or isn't on the expected port.
+2. **`python3 -c "from drydock.core.agent_loop import AgentLoop; import inspect; print('aafa090' in inspect.getsource(AgentLoop._sanitize_message_ordering))"`** — checks the should_break_loop fix is in your install. If `False`, reinstall with `pip install --upgrade drydock-cli`.
+3. **Check the session log.** `tail ~/.vibe/logs/session/session_*/messages.jsonl` — look for `"user": "Continue."` repeating. If you see that, set `DRYDOCK_AUTO_CONTINUE_DISABLE=1`.
+4. **`docker logs gemma4 | tail -50`** — vLLM errors show up here. Common: `out of memory` (drop `--max-num-seqs` or `--max-model-len`), `unknown tool-call parser` (you're on an older vLLM image without the `gemma4` parser).
+
+## Tested versions
+
+Everything in this document was exercised on:
+
+- drydock-cli ≥ 2.6.145
+- vllm/vllm-openai:gemma4 Docker image (built from vLLM 0.7.x + Gemma 4 parser patch)
+- Python 3.12 (dev) / 3.14 (runtime)
+- Ubuntu 22.04, kernel 6.8, Docker 24.x with nvidia-container-toolkit
+- NVIDIA driver 550.x
+
+Older vLLM images may lack `--tool-call-parser gemma4`. Build from the
+gemma4 branch or wait for upstream.

drydock_cli-2.7.44/HLE_PRD.md

@@ -0,0 +1,204 @@
+# HLE PRD — drydock + GraphRAG + Deep Noir vs Humanity's Last Exam
+
+**Status as of 2026-05-04 00:40 UTC.** Update on resume.
+
+## Thesis
+
+> drydock + GraphRAG + Deep Noir working together should be able to solve
+> any problem.
+
+HLE is the hardest possible PRD — explicitly designed to defeat frontier
+models. Reaching a defensible local-26B score against it validates that
+the three-leg architecture generalizes. Failures expose which leg is weak.
+
+**Critical rule:** drydock IS the harness. Eval drives questions through
+the real TUI. Never wrap the model directly. Every failure is a drydock-
+or-leg bug to fix, not a harness limitation. See
+`memory/feedback_drydock_is_the_harness.md`.
+
+## The three legs and their current strength
+
+(updated 2026-05-05 — post v2.7.39, post llama.cpp swap on .21 + .22)
+
+| Leg | What it owns | State | Evidence |
+|-----|--------------|-------|----------|
+| **drydock** | agent loop, tool use, prompts, harness fixes | **strongest** | 70% SWE-bench file match; 49/52 PRD functional tests; 16K+ harness queue actively drained; v2.7.39 landed empty-assistant filter (#14), `extra_params` config seam (#15), markdown indent rescue (#16), GraphRAG auto-prefetch hook |
+| **GraphRAG** | factual recall over indexed code/text | **medium-strong** | infra works; **iter12 PROVEN** (commit 8a0ed75): synthetic-tool-call auto-prefetch lifts a wrong→right HLE answer when corpus is curated. Still needs: a real general-knowledge corpus (Wikipedia/arXiv) instead of code-only |
+| **Deep Noir** | reasoning-mode steering | **weakest** | scaffolding shipped (`drydock/steering/`), `LogitBiasSteeringApplier` wired, hook in agent_loop honors `DRYDOCK_STEERING_APPLIER` env; **zero vectors trained** — hook still a no-op until vectors land |
+
+## Serving infrastructure (post 2026-05-05 swaps)
+
+| Box | Endpoint | Stack | Status |
+|-----|----------|-------|--------|
+| remus (.22, this box) | `localhost:8000` | llama.cpp Docker (`ghcr.io/ggml-org/llama.cpp:server-cuda`), `--jinja`, Q3_K_M GGUF, restart=unless-stopped | ✅ running, fingerprint `b9014-d4b0c22f9` |
+| romulus (.21) | `192.168.50.21:8000` | llama.cpp native build, `--jinja`, same recipe | ✅ running, fingerprint `b1-e77056f` |
+| Jetson (.19) | `192.168.50.19:8080` | unknown (responds at /v1/models with Ollama-style schema; /v1/chat/completions hangs after 90s with no response) | 🔒 **NOT in pool** — needs SSH access + diagnosis. Re-add commented line in `scripts/llm_balancer.py:BACKENDS` once fixed |
+| balancer (port 8001) | round-robin remus + romulus | failover: `(idx + 1) % len(BACKENDS)` | ✅ running |
+
+## Phase plan
+
+### Phase 1 — baseline (DONE 2026-05-04, RE-RUNNING ON LLAMA.CPP)
+
+Goal: bare drydock + bare GraphRAG (cwd only) + zero Deep Noir vs HLE.
+This number is the floor.
+
+**v1 baseline result (vLLM + drydock pre-#14-fix):**
+- 10/200 = **5.0% raw**, 8.62% effective (84/200 thinking-stalls)
+- 22h runtime, avg 396s/question
+- Per category: Humanities 21%, Math 4% (92 questions, dominant), Physics 0%, Chemistry 0%
+- Archived at `/data3/drydock/hle_results_v1_baseline/`
+
+**v2 baseline running 2026-05-05 (llama.cpp + v2.7.39):**
+- PID `/tmp/hle_n20_v2.pid`, log `/tmp/hle_n20_v2.log`
+- N=20 (seed=42 — same first 20 as v1 baseline, apples-to-apples)
+- Tests whether `--jinja` chat template + empty-assistant filter
+  reduces stall rate
+- Output `/data3/drydock/hle_results/run_*/`
+
+Realistic baseline expectation per pre-run analysis:
+- Bare 26B-A4B without retrieval/steering: ~5–10% on HLE
+- Reaching ~22–25% would be defensible vs ~25–30% frontier scores
+- **SOTA reference: 45.9%** (per user, 2026-05-04)
+
+### Phase 2 — GraphRAG with knowledge corpus
+
+Goal: ingest enough general knowledge that retrieve answers fact-recall
+questions. Re-run, measure delta.
+
+- ⏳ Identify corpus (Wikipedia subset / arXiv abstracts / textbook chunks)
+- ⏳ Extend `drydock/graphrag/code_indexer.py` ingest path for non-code text
+      (it already supports text via `text_indexer.py`; just needs corpus)
+- ⏳ Bulk ingest into `~/.drydock/graphrag.sqlite`
+- ⏳ Re-run HLE Phase 1 with corpus loaded; measure delta
+- Expected delta: +3 to +7 points if corpus is good, +0 if not
+
+### Phase 3 — Deep Noir reasoning vectors
+
+Goal: train activation-steering vectors on reasoning-failure pairs from
+admiral_history; deposit into `~/.drydock/steering/vectors/`; the
+existing hook applies them. This is the user's research domain.
+
+- ⏳ Extract pairs from admiral_history (model-output / correct-intervention)
+- ⏳ Train vectors per direction: "verify-before-answer", "show-work-explicitly",
+      "consider-units", "minimal-patch"
+- ⏳ Deposit `.npy + .toml` per mode under `~/.drydock/steering/vectors/<mode>/`
+- ⏳ Set `DRYDOCK_STEERING_MODES=<mode1>,<mode2>` env at TUI launch
+- ⏳ Re-run HLE; measure delta vs Phase 2
+- Expected delta: 0 to +3 points; high variance, open research
+
+## Currently running / in flight (2026-05-05)
+
+| Thing | PID | Log | Notes |
+|-------|-----|-----|-------|
+| HLE N=20 v2 | `/tmp/hle_n20_v2.pid` | `/tmp/hle_n20_v2.log` | re-baseline on llama.cpp; ~2h |
+| stress harness | `/tmp/stress_pid.txt` | `/tmp/stress_*.log` | fresh round started post-completion of v9 |
+| llm_balancer | `/tmp/llm_balancer.pid` | `/data3/drydock/logs/balancer.log` | :8001, 2 backends now |
+| llamacpp-gemma4 (remus) | docker | `docker logs llamacpp-gemma4` | restart=unless-stopped |
+| llama-server (romulus) | `/tmp/llama_server.pid` on 192.168.50.21 | ssh `tail /data2/logs/llama-server.log` | nohup, no auto-restart yet |
+
+## Resume checklist (if connection dropped)
+
+```bash
+# 1. Where are we?
+date -u
+git -C /data3/drydock log --oneline --since="6 hours ago"
+git -C /data3/drydock describe --tags --abbrev=0     # current PyPI tag
+
+# 2. HLE run status
+ps -p $(cat /tmp/hle_overnight.pid 2>/dev/null) -o pid,etime,comm
+ls /data3/drydock/hle_results/
+tail -50 /tmp/hle_overnight.log
+n_done=$(wc -l < /data3/drydock/hle_results/run_*/results.jsonl 2>/dev/null)
+echo "completed: $n_done / 200"
+
+# 3. If HLE crashed mid-flight, RESUME (skip already-done IDs)
+RUN_DIR=$(ls -td /data3/drydock/hle_results/run_* | head -1)
+nohup /home/bobef/miniconda3/bin/python3 /data3/drydock/scripts/hle_eval.py \
+    --source hle --limit 200 --shuffle --seed 42 --resume "$RUN_DIR" \
+    > /tmp/hle_resume.log 2>&1 &
+
+# 4. If complete, see the score
+cat /data3/drydock/hle_results/run_*/summary.json | python3 -m json.tool
+
+# 5. Infra health
+ps -p 2462362 -o pid,etime
+curl -s --max-time 3 http://localhost:8001/v1/models | head -1
+gh issue list --repo fbobe321/drydock --state open --limit 5
+```
+
+## Known issues + workarounds
+
+1. **`web_search` tool requires permission approval** by default
+   (`ToolPermission.ASK`). In batch eval the harness can't see/respond
+   to the prompt → session stalls. **Workaround:**
+   `--dangerously-skip-permissions` flag passed by `hle_eval.py`.
+   **Real fix:** auto-approve read-only tools when stdin is non-TTY
+   or `DRYDOCK_BATCH_MODE=1`. Memory:
+   `memory/project_hle_phase1_findings.md`.
+
+2. **TUI input handler corrupts rapid char-by-char multi-line pexpect
+   input.** Internal `\n` chars get partially eaten + spurious newlines
+   inserted on long prompts. **Workaround:** single-line prompts in
+   `hle_eval.py`. **Real fix:** debug `drydock/cli/textual_ui/` input
+   buffer.
+
+3. **Auto_release at 06:00/12:00/18:00/00:00 UTC overwrites site-packages.**
+   In-flight HLE runs survive (each new question is a fresh TUI spawn that
+   picks up the new binary), but if you direct-edit site-packages your
+   changes vanish. Always commit to source. Pause via:
+   `touch /data3/drydock/.pause_auto_release`.
+
+4. **PRD contamination: model edits `PRD.md` mid-session.** HLE doesn't
+   touch this — every HLE question gets a fresh empty cwd. Not a concern
+   for HLE; relevant for shakedown PRD runs.
+
+5. **Sessions take real time.** HLE questions involve web_search +
+   multi-step reasoning. Seed q's took 30-60s; HLE q's appear to take
+   5+ min each (q1: 5 min in and still working). Expect N=20 to take
+   1-2 hours, N=100 overnight.
+
+## Sentinels currently set
+
+- `/data3/drydock_test_projects/.pause_watchdog` — watchdog cron paused
+- `/data3/drydock/.pause_auto_release` — NOT set (auto_release is active)
+- `/data3/drydock/research/STOP` — research loop sentinel (per gitignore)
+
+## Tomorrow morning's first action
+
+If overnight run completed:
+1. `cat /data3/drydock/hle_results/run_*/summary.json` for the baseline number
+2. If <10%, the diagnosis is in the per-question `verdict` and `judge_reasoning`
+   fields. Sort by category to see if math/physics/CS dominate the failures.
+3. Commit the results JSON (with HLE content redacted to just IDs+verdicts —
+   never commit the question text per HLE license).
+4. Decide Phase 2 corpus based on category distribution of failures.
+
+If overnight run crashed:
+1. `--resume` flag re-enters where it stopped (skip-by-id from results.jsonl).
+2. Likely failure modes: vLLM OOM on a long-thinking question, balancer
+   crash from a port conflict, drydock TUI hang on a tool error.
+   Investigate via the per-question `tui_logs/<id>.tui.log` files.
+
+## File map
+
+```
+/data3/drydock/
+├── scripts/
+│   ├── hle_eval.py                 # the orchestrator (this PRD's main artifact)
+│   ├── hle_eval_seed.jsonl         # 7 hand-crafted seed questions
+│   └── consume_retrieval_queue.py  # GraphRAG-leg autonomy (28× perf-fixed)
+├── hle_results/                    # gitignored — per-run outputs
+│   └── run_<ts>/
+│       ├── config.json
+│       ├── results.jsonl
+│       ├── summary.json
+│       ├── tui_logs/<id>.tui.log
+│       └── work/<id>/              # per-question fresh cwd
+├── HLE_PRD.md                      # this file
+└── ~/.config/drydock/hf_token      # gated cais/hle access
+```
+
+## What "done" looks like for Phase 1
+
+A number. With distribution. Per-category and overall. Writeup in this
+PRD. Commit the writeup, not the questions. Decide Phase 2 from there.