scout-essentials 1.9.1 → 1.9.3

data/research/lock-namespace-blindness.md

@@ -0,0 +1,73 @@
+# Lock liveness is PID-namespace-blind
+
+**Status:** Non-normative investigation artifact. May be outdated.
+
+**Origin:** promoted from the Cortex claim
+`claims/persist_lock_theft_across_agents.md` (investigation of
+2026-09-08, after concurrent agents observed `StolenLockError` storms).
+
+## The mechanism
+
+Every lock liveness check in scout-essentials reduces to
+`Process.kill(0, pid)` (`Lockfile#alive?`,
+`lib/scout/open/lock/lockfile.rb:379-387`):
+
+```ruby
+def alive? pid
+  pid = Integer("#{ pid }")
+  Process.kill 0, pid       # ESRCH ⇒ dead, anything else ⇒ alive
+rescue Errno::ESRCH
+  false
+end
+```
+
+`Process.kill` resolves pids in the **PID namespace of the calling
+process**. In the normal case — all processes on one host — a recorded
+pid either belongs to a live process (signal permitted/delivered ⇒ alive)
+or does not (ESRCH ⇒ dead), and the check is correct.
+
+In a PID namespace (bubblewrap/sandboxed agents, containers) the caller
+sees only its own namespace. A lock recorded by a *host* process carries
+a pid that simply does not exist in the caller's namespace, so
+`Process.kill 0, <host pid>` raises `Errno::ESRCH` and the holder is
+reported **dead while it is alive**. Verified from inside a bwrap
+sandbox (host pid 3151964 recorded in a live
+`var·cache·ask·….persist.lock`): `Process.kill(0, 3151964)` →
+`Errno::ESRCH`, while `Process.kill(0, <namespace pid 2>)` succeeds.
+
+Two consumers act on that verdict:
+
+1. **`Lockfile#sweep`** (`lib/scout/open/lock/lockfile.rb:341-376`) runs
+   on every `Open.lock` in the directory of the lock target and removes
+   `.*lck` temp files whose embedded pid it believes dead. In a
+   directory shared across namespaces (e.g. `~/.scout/tmp/persist_locks`)
+   a sandboxed caller can therefore delete the lock bookkeeping of live
+   host holders.
+2. **`scout system clean`** (scout-gear `scout_commands/system/clean`) —
+   not in this repository, but it iterates `Scout.lock_info` over the
+   same `tmp/persist_locks` directory and unlinks any lock whose pid
+   `Misc.pid_alive?` reports dead, with the same namespace blindness.
+   The scout-gear implementation additionally has an inverted condition
+   in its persists/sensiblewrites sections (unlinks when the pid **is**
+   alive), which makes it more aggressive still.
+
+The victim observes this as `Lockfile::StolenLockError` raised by the
+refresher thread of every live holder at once (`new_refresher`,
+`lib/scout/open/lock/lockfile.rb:414-433`) — matching the reported
+"all agents hit simultaneously" signature.
+
+## Inferred rule
+
+> Do not share `Persist.lock_dir` / produce / sensible_write lock
+> directories across PID namespaces. Under bwrap or containers, pin
+> `HOME` (and the `tmp` path map) per sandbox, so each namespace sweeps
+> only its own locks.
+
+A more robust library-side fix would treat `/proc/<pid>` unreadable —
+or more generally "pid outside my namespace" — as *alive*, at the cost
+of keeping genuinely orphaned locks when the last namespace dies.
+
+## Regression test idea (not implemented)
+
+Holder takes a `Persist` lock; a child in a PID namespace runs the
+sweep; the holder must not receive `StolenLockError`.

data/research/probing-guide.md

@@ -0,0 +1,311 @@
+# Probing guide — general advice for empirical Scout investigation
+
+Promoted from the Cortex artifact `guides/probing.md` (v3, 2026-09-09).
+Source of truth for future revisions remains the Cortex artifact; this
+file is the in-research copy for agents that do not have the Cortex
+workspace mounted. Cortex addresses cited below resolve in the
+scout-essentials / scout-gear Cortex path maps.
+
+Audience: any coding agent probing behavior in **any** Scout package or
+workflow (scout-gear, scout-essentials, scout-ai, scout-camp, scout-rig,
+installed workflows). This guide distills the frictions and successes of
+the 2026-09-07/09 CortexCoder documentation-and-probing campaigns
+(scout-essentials + scout-gear, 106 probe artifacts, 8 claims documents,
+~7,600 bash and 60 `cortex_entity_property` calls across the session
+chats), plus one new probe written to pin down the timeout mechanics.
+
+Evidence is cited by Cortex address throughout. The guide is deliberately
+general: subsystem-specific facts stay in the `scout-gear/*` claims
+artifacts.
+
+## 1. The ladder (unchanged, it works)
+
+```text
+uncertainty
+ → tmp/ experiment (throwaway)
+ → standalone run under an escalating timeout guard
+ → probe/ artifact (versioned, commented, JSON-returning)
+ → Observation/probe execution (receipt + registry record)
+ → claims/ artifact (proposition, observed vs inferred, citations)
+ → research / doc / test promotion
+```
+
+This was executed end-to-end twice and is the single most reliable
+pattern in the corpus (`harness/cortex-coder-first-episode-review.md` F5).
+Nothing below replaces it; everything below is how to avoid the
+frictions the same corpus recorded.
+
+## 2. Before writing a probe
+
+- **Name the question first.** One probe answers one sharply-stated
+  question ("does `WorkQueue#abort` leave `join` blocked?"), not "learn
+  about concurrency". A probe that cannot be titled is a browsing
+  session, not an investigation.
+- **Check what is already known.** `cortex_list type=artifacts prefix=probe/`,
+  `cortex_search`, and `cortex_activity` for the entity. Campaigns left
+  106 probes behind; re-probing an answered question is the most common
+  wasted motion.
+- **Read the docs and the code path before probing.** Probes are for
+  *resolving* disagreement between doc, code and expectation — state in
+  the probe header which doc sentence the question came from.
+  (`probe/persist_engine_selection.rb`, `probe/assoc_field_specs.rb` are
+  good templates: they quote the doc claim being tested.)
+- **Record the environment you are testing.** Live checkout vs installed
+  gem (`-Ilib` vs plain `ruby`), HEAD hash, RUBY_VERSION. Campaign
+  artifacts drifted between batches because line numbers and HEAD moved
+  (`scout-gear/overview.md` open item 6); re-pin before quoting line
+  numbers.
+
+## 3. Probe anatomy
+
+Follow the doctrine header, proven in the corpus:
+
+```ruby
+# Observation: <snake_case_name>
+#
+# What it investigates
+# Why it matters
+# Assumptions / conditions   (environment, isolation, what is NOT tested)
+# Result kind                (JSON object with these keys: ...)
+```
+
+Rules that emerged as load-bearing:
+
+- **Return one JSON-serializable object with string keys.** Every key you
+  intend to cite from a claim should be a distinct result key; claims
+  cite `Observation/probe/<job>.json` *key names*
+  (`scout-gear/concurrency.md` does this per claim).
+- **Rescue each section independently** so one failure does not hide the
+  rest; collect per-section errors into an `errors` key instead of
+  letting the probe die (`probe/entity_persisted_property_invalidation.rb`).
+- **Put the `puts JSON.pretty_generate` at the end AND return the
+  object** — standalone runs need stdout, property runs need the return
+  value.
+- **Errors are results.** Capture `"#{e.class}: #{e.message[0, 60]}"`,
+  not just a boolean. The class/message pair is usually the interesting
+  fact (see `fork_each_raises` in the semaphore probe).
+
+## 4. Determinism and isolation
+
+- **No network. Fixed tmpdirs. Close everything you open** (databases,
+  semaphores, sockets, queue dirs). Scratch under `tmp/probe_<name>_<pid>`
+  or `Dir.mktmpdir`, and remove it in the same run.
+- **Prefer sleep-free observation.** Where you need timing facts, read
+  recorded trace timestamps (`Workflow.trace_job_times`) instead of
+  sleeping (`probe/deploy_local_end_to_end.rb`). Where you need
+  cache-hit vs recompute, use `rand` inside blocks so the two are
+  distinguishable without delays.
+- **Isolate global state before declaring anything.** If the library
+  keeps process-global caches (`Entity.entity_property_cache`,
+  `Entity.formats`), reassign them to a tmpdir *before* registering
+  modules/properties — several defaults are captured at declaration
+  time (`probe/entity_persisted_property_invalidation.rb`).
+- **Isolate the user environment.** For CLI/load-path probes pass
+  `HOME` = fresh tmpdir so per-user config cannot mask the failure you
+  are testing (`probe/cli_process_trace_broken_commands.rb`).
+- **Normalize environment artifacts before claiming.** Anonymous module
+  names (`#<Module::…>::M9::AA`), PIDs, tmpdir paths, timestamps differ
+  between runs — compare shapes and counts, not raw strings
+  (`probe/wf_remote_step_api.rb` notes this explicitly: "an environment
+  artifact, not a behavioral difference").
+- **Scope claims to what you ran.** Report `RUBY_VERSION` and the
+  interpreter (MRI/GVL) when concurrency semantics depend on it
+  (`probe/entity_formats_registry_concurrency.rb`).
+- **Make rare interleavings deterministic by widening the window** (alias
+  patch around a check-compute-write-back), and *label it as an exhibit*
+  in both the probe and the claim — an exhibit proves the interleaving
+  exists, not that it occurs at stock timing.
+
+## 5. The execution environment (bwrap / property sandbox)
+
+Probe bodies are `eval`ed by `Observation/probe` with `self` = a Scout
+Step, and tasks run under bwrap. Facts that cost the campaigns real
+debugging time:
+
+- **`include` is unavailable** at top level (`NoMethodError: undefined
+  method 'include' for #<Step>`). Use modules explicitly:
+  `FU = FileUtils; FU.mkdir_p …` (`probe/persist_engine_selection.rb`).
+- **`Kernel#include`-style sugar and some constants resolve differently.**
+  Prefer explicit `Process.clock_gettime(Process::CLOCK_MONOTONIC)`
+  forms over constants; define helpers as plain `def`s (they work).
+- **Modifier-rescue precedence can make a body unparseable** in the
+  eval context: `File.read(lp).split… rescue []` inside a hash literal
+  broke `cli_preprocess_flags` until rewritten flat. Keep hash-literal
+  contents rescue-free (`scout-gear/cli.md` §7, v3→v4 fix).
+- **Sandbox and plain-ruby values can differ.** The first episode
+  recorded `join_pid_fail*` differing between bwrap and plain ruby.
+  Never claim a value you only measured in one environment when the
+  question is about the other; state both, or say which one you ran.
+- **Writability is layered**: the Ruby allowlist (read/write tasks) and
+  the bwrap mounts (bash/ruby/python) are different sets. When a path is
+  rejected, call `sandbox_paths` instead of guessing
+  (the campaign repeatedly re-discovered which repo tmp dirs were bound).
+
+## 6. Blocking, forking, and timeouts — the F1 class
+
+The one friction that hung real executions. General rules:
+
+- **Single-process by default.** Do not fork/join in probe bodies unless
+  the question *is* about forking. Static/no-fork variants answer most
+  questions (`wq_lifecycle_static` vs `wq_fork_run` split).
+- **A forking probe must graduate:** standalone first, 3/3 clean runs
+  under the escalating guard, every blocking `join` boxed in
+  `Timeout.timeout(k) { … }`, executed through the property exactly once
+  (`probe/wq_fork_run.rb` header is the template;
+  `scout-gear/concurrency.md` "Probe discipline note").
+- **Always use an escalating guard, never bare `timeout N`:**
+  `timeout --kill-after 5 60 ruby probe.rb`. Measured fact: against a
+  child that traps SIGTERM — exactly what forked Scout workers and
+  rescuing loops do — plain `timeout 2` let the child run to natural
+  exit (8.12 s under a 2 s bound) **while still reporting exit 124**, so
+  the guard lies; `--kill-after 1 2` killed it at 3.01 s with SIGKILL
+  (`Observation/probe/shell_timeout_kill_after_…3940752058….json`,
+  `claims/timeout_kill_after.md`). Keep the escalation short relative to
+  the bound.
+- **Put the guard on the leaf command.** Nested timeouts do not compose:
+  the outer KILL reaches only its direct child, and a trapping
+  grandchild survives (measured: 8.12 s under a 2 s outer guard). Do not
+  wrap a wrapper.
+- **Do not trust exit status 124 as proof of a bounded run** — check
+  elapsed time or wrap the measurement yourself.
+- **Property execution has its own bound** (default 3600 s, no progress
+  signal). For probe-shaped bodies pass a short per-call `timeout:` to
+  `cortex_entity_property`. A hung probe inside a property is invisible
+  from the outside until the bound fires; the first-episode branch that
+  hung simply stopped making tool progress (F1).
+- **Hung probes vanish from the transcript.** Killed executions leave no
+  completed tool output, so chat-level latency analysis shows nothing —
+  the evidence of a hang lives in harness reviews and forced-re-run
+  records. If a run "disappeared", suspect a hang, not a skip.
+
+## 7. Property execution and caching subtleties
+
+- **Standalone first, then property.** The standalone run is your
+  pre-flight (parses, terminates, produces JSON); the property run is
+  your receipt.
+- **Editing a probe artifact does NOT auto-invalidate its property job.**
+  The Step is content-addressed on entity + arguments + definition
+  identity — not on artifact contents, which are read by `eval` at run
+  time. After editing `probe/<name>.rb`, re-run with `update: true` or
+  you will be reading a stale cached result while believing you tested
+  the new body (observed live this session: v3 result served after the
+  v4 edit until `update: true` forced recompute).
+- **Check the receipt:** `definition_version`, `definition_digest`,
+  `property_job` and the artifact `version` in `content` should match
+  what you wrote. A `version` in the receipt that lags your edit is the
+  stale-cache signature.
+- **Registry records accumulate**: `cortex_list type=properties` /
+  `cortex_activity` show run counts and last-run times — cheap way to
+  check whether a probe has ever been executed and against which
+  definition version.
+
+## 8. Negative probes
+
+Deliberate failure probes are first-class evidence (77 of the first
+episode's tool-call failures were intentional):
+
+- Test the documented error path, not only the happy path: unknown
+  workflow/task, missing args, broken `require`s, worker-block raises,
+  `abort` after `close`. Capture exit status + first stderr signature
+  (`probe/cli_dispatch_and_surface.rb`, `cli_process_trace_broken_commands`).
+- A `require` that fails in a stock install is a stronger statement than
+  "has a bug" — say which level failed.
+- Keep negative probes as *separate, tiny* probes or clearly-keyed
+  sections, so a claim can cite "exit_status" keys without ambiguity.
+
+## 9. Debugging a failed property run
+
+- **Bisect wrapper pattern:** when a probe works standalone but fails in
+  the property context, run its lines cumulatively in the property
+  context to find the exact failing statement
+  (`probe/tmp_open3_sanity.rb` — returns the failing line number, code
+  and error).
+- Read the Step files the receipt points at before re-running anything;
+  re-running blind is the expensive habit.
+- The two-layer sandbox (§5) explains most "works in bash, fails in
+  read" mysteries — `sandbox_paths` settles it in one call.
+
+## 10. Evidence discipline
+
+- **Never transcribe numbers a probe can return.** Claims cite the
+  property job and the result keys; humans (and later agents) re-run
+  rather than trust copy-paste.
+- **Separate observed from inferred.** The claim format that worked:
+  proposition → *Observed* (probe evidence, exact keys) → *Inferred
+  rule* (scoped, justified) → *Docs impact* (gap/correct/wrong).
+  Code-reading-only findings are labelled as such and never mixed with
+  runtime receipts (`scout-gear/concurrency.md` Claim marked
+  "unprobed at runtime: cross-process effect").
+- **One observation ≠ one universal claim.** Scope the rule to the
+  environment, Ruby version, and code revision you measured.
+
+## 11. Lifecycle
+
+- Probes that support a durable claim stay under `probe/`; scratch and
+  bisect helpers get deleted once their question is answered.
+- A probe with a crisp failure signature can be promoted **into a
+  regression test** in the target repo — prefer that over keeping an
+  out-of-cortex behavior pinned only in Cortex.
+- When the claims are promoted out of Cortex (docs/tests), the probe may
+  be deleted; the property registry keeps the run history either way.
+- Record cross-session harness learnings (hangs, concurrency, provenance
+  gaps) under `harness/` — that is where the F1–F5 evidence lives and
+  where this guide's timeout section came from.
+
+## 12. Quick checklist before executing any new probe
+
+```text
+[ ] one named question, doc/code disagreement stated in the header
+[ ] string-keyed JSON result; sections individually rescued
+[ ] no network; fixed tmpdir; globals reassigned to scratch; cleanup
+[ ] sleep-free where a trace/timestamp alternative exists
+[ ] environment recorded (checkout vs gem, HEAD, RUBY_VERSION)
+[ ] standalone run OK under: timeout --kill-after 5 60 ruby probe.rb
+[ ] forking only if the question needs it; joins boxed in Timeout.timeout
+[ ] property run with a short per-call timeout; receipt version matches
+[ ] after editing the artifact: re-run with update: true
+[ ] claim cites property_job + result keys; observed vs inferred separated
+```
+
+## Evidence index
+
+- `claims/timeout_kill_after.md` +
+  `Observation/probe/shell_timeout_kill_after_3940752058e155e185253a096bb3d07f.json`
+  — the `--kill-after` mechanics (control / trap / kill-after / nested
+  guard), written for this guide.
+- `harness/cortex-coder-first-episode-review.md` — F1 (hang), F2
+  (artifact edit races), F3–F4 (accounting/provenance), F5 (the ladder
+  executed correctly).
+- `harness/cortex-coder-second-episode-record.md` — corpus consumption
+  and promotion decisions.
+- `probe/wq_fork_run.rb`, `probe/wq_lifecycle_static.rb`,
+  `probe/entity_formats_registry_concurrency.rb`,
+  `probe/entity_persisted_property_invalidation.rb`,
+  `probe/persist_engine_selection.rb`,
+  `probe/cli_process_trace_broken_commands.rb`,
+  `probe/deploy_local_end_to_end.rb`, `probe/tmp_open3_sanity.rb` —
+  the pattern templates referenced above.
+- `scout-gear/concurrency.md` (probe discipline note, abort/join hazard),
+  `scout-gear/cli.md` §7 (sandbox parse failure),
+  `scout-gear/overview.md` §5 (106-probe inventory & citation audit),
+  `scout-gear/consolidation-survey.md` (rev drift, HEAD pinning).
+
+### Where the session evidence lives (for future ChatAnalyst passes)
+
+The `~/git/<repo>/chats/<name>` paths are not visible from inside the
+sandbox; the same sessions persist as root chats under
+`~/.scout/tmp/tmp-*.chat` (this campaign: `tmp-772567689` scout-gear CLI
+owner, `tmp-869423640` scout-gear gap-closing, `tmp-725846580` /
+`tmp-908860370` / `tmp-879155263` scout-essentials doc-review owners,
+`tmp-171141240` scout-gear CLI batch 4, `tmp-373370430` scout-gear
+consolidation, `tmp-72174883` synthesis). `chat_overview` /
+`chat_tool_calls` / `provenance_relationships` work on those files
+directly. Corpus stats extracted for this guide: `timeout` durations
+used — 60 s ×350 (the norm), 30 ×82, 120 ×38, 90 ×9, 150 ×5, 300 ×2;
+~207 non-zero bash exit statuses out of ~3,900 recorded (status 1 ×207
+plus stray 231/235/240/246 — 246 is the WorkQueue abort exit code),
+overwhelmingly deliberate negative probes; bash dominates the tool mix
+(~7,600 calls) with `cortex_read` (384) second — probing in this stack
+is a bash + Cortex discipline. No `--kill-after` appears anywhere in
+the corpus: the guard discipline existed, the escalation half did not,
+which is precisely the friction that motivated this guide.

data/research/testing-availability-scout-ai-handoff.md

@@ -0,0 +1,235 @@
+# Testing availability handoff — scout-ai
+
+Provenance: salvage packet produced during the scout-essentials layering
+cleanup (2026-10). scout-ai keeps the model/accelerator/endpoint probes that
+were incorrectly drafted into scout-essentials. This note is self-contained:
+verified call-site inventory, retention list, the 7-defect list with the
+correction for each, a delegation example, the compat-adapter option, and
+constraints.
+
+## Verified call-site inventory (read-only pass over /home/mvazque2/git/scout-ai)
+
+- `python_modules?`
+  - `test/scout/model/python/test_torch.rb`
+  - `test/scout/model/python/huggingface/test_classification.rb`
+  - `test/scout/model/python/huggingface/causal/test_next_token.rb`
+- `cuda?`
+  - `test/scout/model/python/torch/test_helpers.rb`
+- `hf_model_cached?`
+  - `test/scout/model/python/huggingface/test_causal.rb`
+  - `test/scout/model/python/huggingface/test_classification.rb`
+  - `test/scout/model/python/huggingface/causal/test_next_token.rb`
+- `hf_model_reason` — same three files as `hf_model_cached?` (omit messages).
+- `rscript?`, `r_packages?`, `endpoint_reachable?` — **zero** call sites
+  outside `test/support/availability.rb` itself.
+- `endpoint_configured?('test')` — `test/integration/scout/llm/test_infrastructure.rb:30`.
+- `test/support/infrastructure_probes.rb` — used by
+  `test/integration/scout/llm/backends/test_endpoints.rb` and
+  `test/integration/scout/llm/test_infrastructure.rb`;
+  `test/integration/scout/model/test_base.rb` also uses `Availability`
+  (specifically `r_packages?('e1071')` / `r_packages_reason`).
+- `SCOUT_TEST_PYTHON` appears **only** inside `availability.rb`.
+- `test/test_helper.rb` requires `support/fixtures` and `support/mock_backend`
+  but **not** `support/availability`; each test file that needs it requires it
+  directly (e.g. `test_causal.rb` documents the dependency).
+
+## Layer table (what stays where)
+
+| Probe / method | Layer | Destination |
+|---|---|---|
+| `executable?`, `run_command`, `tcp_reachable?` | scout-essentials | `Scout::Testing::Availability` (delivered, stdlib-only) |
+| `cuda?` | scout-ai | `test/support/availability.rb` |
+| `hf_model_cached?` / `hf_model_reason` (must honor `HF_HUB_CACHE`) | scout-ai | same |
+| `endpoint_reachable?`, `endpoint_configured?`, `endpoint_reason`, `configured_endpoints`, `real_endpoints`, `endpoint_available?` | scout-ai | same |
+| `InfrastructureProbes` | scout-ai | `test/support/infrastructure_probes.rb` — unchanged |
+| `python?`, `python_modules?`(+reason), `rscript?`, `r_packages?`(+reason) | scout-rig | see `testing-availability-scout-rig-handoff.md`; scout-ai may still call them until its own suite stops depending on them (see note on `test_base.rb`) |
+
+## Retention list (do not delete in scout-ai)
+
+- `cuda?` — one live call site.
+- `hf_model_cached?` + `hf_model_reason` — three live call sites; must honor
+  `HF_HUB_CACHE`.
+- `endpoint_reachable?`, `endpoint_configured?`, `endpoint_reason`,
+  `configured_endpoints`, `real_endpoints`, `endpoint_available?` — endpoint
+  family; memoization must key on the timeout argument.
+- `InfrastructureProbes` — used by two integration suites; keep unchanged.
+
+`rscript?`/`r_packages?` have exactly one live consumer
+(`test/integration/scout/model/test_base.rb:12`, e1071) and no R integration
+in scout-rig; keep them (reference-only, as in the rig handoff) until that
+call site is either guarded by an R-availability decision or removed.
+
+## The 7-item defect list with corrections
+
+1. **`python?` ignores subprocess exit status.** A broken interpreter can
+   still print. Decide on `exit_status == 0`, not on output presence.
+2. **`python_modules_reason` reads stdout.** Import tracebacks go to stderr;
+   read `result.stderr`, falling back to a synthesized reason when it is
+   empty.
+3. **`memo` rescues `Exception`.** Rescue `StandardError` only; timeout and
+   launch failure are handled deliberately (recorded on the result), never
+   swallowed. Never cache a `SignalException` as "unavailable".
+4. **`run_bounded` drains stdout before stderr** (pipe deadlock with chatty
+   children), **spawns before its rescue** (launch failure escapes), and the
+   **`capture:false` path drains neither pipe**. Corrected shape: spawn inside
+   the rescue scope; drain both pipes concurrently (reader threads) with a
+   deadline; on timeout TERM the process group, wait a grace, then KILL;
+   report launch failure as a structured result.
+5. **`endpoint_reachable?` memoization ignores the timeout argument.** Key the
+   memo on `(endpoint_or_host, port, timeout)` — a 2s probe and a 30s probe
+   must not share an entry.
+6. **HF cache lookup ignores `HF_HUB_CACHE`.** Resolve the cache root as
+   `ENV['HF_HUB_CACHE']` (if set) else the default
+   `~/.cache/huggingface/hub`, then check for the model directory
+   (`models--<org>--<name>` with a snapshots dir) beneath it.
+7. **Module/package names interpolated unsafely into Python/R source.** Pass
+   them as argv to a fixed probe script, or use `Shellwords.escape` when a
+   source string is unavoidable.
+
+The corrected generic primitives (`executable?`, `run_command`,
+`tcp_reachable?`) now live in scout-essentials'
+`Scout::Testing::Availability`; defects 3 and 4 are already fixed there, so
+the cheapest correct move for scout-ai is delegation (below) rather than a
+local `run_bounded`.
+
+## Delegation example
+
+Reimplement scout-ai's internals on the primitives; keep behavior, drop the
+local runner:
+
+```ruby
+# scout-ai test/support/availability.rb (revised)
+require 'scout/testing/availability' # scout-essentials primitives
+
+module Availability
+  PROBE_TIMEOUT = 30
+  CACHE_TIMEOUT = 300
+
+  class << self
+    # defect 3: StandardError only; explicit reset.
+    def reset!
+      @memo = {}
+    end
+
+    def memoized(key)
+      @memo ||= {}
+      return @memo[key] if @memo.key?(key)
+
+      @memo[key] = yield
+    rescue StandardError => e
+      "probe error: #{e.class}: #{e.message}"
+    end
+
+    # defect 1 + 7: exit status decides; module names as argv.
+    def python_modules?(*modules)
+      !python_modules_reason(*modules)
+    end
+
+    def python_modules_reason(*modules)
+      python = ENV['SCOUT_TEST_PYTHON'].to_s.strip
+      python = 'python3' if python.empty? &&
+                            Scout::Testing::Availability.executable?('python3')
+      return 'no python executable found' if python.nil? || python.empty?
+
+      probe = <<~PY
+        import importlib, sys
+        missing = []
+        for name in sys.argv[1:]:
+            try:
+                importlib.import_module(name)
+            except Exception as e:
+                missing.append("%s: %s: %s" % (name, type(e).__name__, e))
+        if missing:
+            sys.stderr.write("; ".join(missing))
+            sys.exit(1)
+      PY
+      result = Scout::Testing::Availability.run_command(
+        [python, '-c', probe, *modules], timeout: PROBE_TIMEOUT
+      )
+      return nil if result.success? && result.exit_status == 0
+
+      # Launch failure and timeout are the concrete reasons; otherwise the
+      # trimmed stderr (defect 2), else the exit status.
+      detail = result.launch_error
+      detail ||= "timed out after #{PROBE_TIMEOUT}s" if result.timed_out
+      detail ||=
+        if result.stderr.to_s.strip.empty?
+          "python exited #{result.exit_status}"
+        else
+          result.stderr.to_s.strip
+        end
+      detail
+    end
+
+    # defect 5: memo key includes the timeout.
+    def endpoint_reachable?(name, timeout: PROBE_TIMEOUT)
+      endpoint = configured_endpoints[name]
+      return "no endpoint '#{name}' configured" unless endpoint
+
+      host, port = endpoint[:host], endpoint[:port]
+      key = [:tcp, host, port, timeout] # <-- timeout in the key
+      hit = memoized(key) { Scout::Testing::Availability.tcp_reachable?(host, port, timeout: timeout) }
+      hit ? nil : "endpoint '#{name}' (#{host}:#{port}) not reachable within #{timeout}s"
+    end
+
+    # defect 6: honor HF_HUB_CACHE.
+    def hf_model_cached?(model)
+      !hf_model_reason(model)
+    end
+
+    def hf_model_reason(model)
+      cache = ENV['HF_HUB_CACHE'].to_s.strip
+      cache = File.join(ENV['HOME'].to_s, '.cache', 'huggingface', 'hub') if cache.empty?
+      dir = File.join(cache, 'models--' + model.tr('/', '--'))
+      return "model #{model} not in cache (#{dir})" unless File.directory?(dir)
+
+      snapshots = File.join(dir, 'snapshots')
+      return "model #{model} cached but empty" unless File.directory?(snapshots) &&
+                                                      Dir.children(snapshots).any?
+
+      nil
+    end
+  end
+end
+```
+
+The reason-selection block in `python_modules_reason` picks, in order: the
+launch error, the timeout note, the trimmed stderr, and finally the exit
+status.
+
+## Compat-adapter option
+
+If touching call sites is out of scope for the first pass, keep the existing
+public method names and reimplement the internals:
+
+```ruby
+module Availability
+  def self.endpoint_reachable?(endpoint_name, timeout: PROBE_TIMEOUT)
+    host, port = resolve(endpoint_name)
+    key = [:tcp, host, port, timeout]
+    memoized(key) { Scout::Testing::Availability.tcp_reachable?(host, port, timeout: timeout) }
+  end
+  # same for any local bounded execution: delegate to
+  # Scout::Testing::Availability.run_command
+end
+```
+
+The four files with `python_modules?`/`cuda?`/`hf_model_cached?` call sites
+keep working unchanged; the internals become the fixed primitives.
+
+## Constraints
+
+- **Never download models**: `hf_model_cached?` inspects the local cache only;
+  absence reports a reason, it never fetches.
+- Ordinary unit tests stay **offline**; integration tests may probe
+  **configured** endpoints (that is their purpose).
+- Product-failure assertions must never become skips: probes omit only on
+  genuinely absent optional dependencies.
+- Do not reintroduce interpreter/package probes into scout-essentials; the
+  Python/R probes belong to scout-rig (see its handoff note), and the
+  primitives layer is frozen at `executable?`/`run_command`/`tcp_reachable?`.
+- scout-ai's `test/support/availability.rb` must keep the same public surface
+  (`python?`, `python_modules?`/`_reason`, `cuda?`, `hf_model_cached?`/
+  `_reason`, `endpoint_reachable?`, `endpoint_configured?`, `endpoint_reason`,
+  `configured_endpoints`, `real_endpoints`, `endpoint_available?`,
+  `InfrastructureProbes` unchanged) so the seven call-site files need no edits.