stata-code 0.9.0__tar.gz → 0.11.0__tar.gz

{stata_code-0.9.0 → stata_code-0.11.0}/.gitignore

@@ -238,3 +238,5 @@ stata_code_test_export.png
 
 # Demo / scratch notebooks (real tests live under tests/)
 demo-tests/*.ipynb
+
+社媒文件/

{stata_code-0.9.0 → stata_code-0.11.0}/CHANGELOG.md

@@ -4,6 +4,338 @@ All notable changes to `stata-code` are documented here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); the project adheres
 to semver-major.minor for the result schema (see `SCHEMA.md` §6).
 
+## [Unreleased]
+
+## 0.11.0 — 2026-07-28
+
+An agent-ergonomics release, driven by a report from an agent that used the MCP
+server to write a full empirical paper. Each item below is a cost that agent
+actually paid.
+
+**Minor, not patch:** `include_results` defaults to `"scalars"`, so `r()` / `e()`
+matrices now arrive as `matrix://` stubs instead of inline values. Pass
+`include_results="full"` for the previous shape. `results.estimation` is
+unaffected — and is now *more* accurate, since two silent numeric bugs in it are
+fixed. The Jupyter kernel and the `stata-code run` CLI keep `"full"`.
+
+### Agent-ergonomics pass
+
+#### Result payloads are now bounded (and correct)
+
+- **`include_results`** (`"none" | "scalars" | "full"`, default **`"scalars"`**).
+  A single estimation used to encode the same numbers four times over — `e(b)`,
+  `e(V)`'s row/column labels, `e(beta)`, and `r(table)` — on top of
+  `results.estimation`. Under the new default, scalars and macros stay inline
+  and every matrix becomes a `matrix://` stub carrying only its shape
+  (`n_rows` / `n_cols`); values and labels remain retrievable with
+  `get_matrix(ref)`. A 123-term regression's envelope drops from ~57 KB to
+  ~28 KB. **This is a behaviour change** — pass `include_results="full"` for
+  the previous shape. The Jupyter kernel keeps `"full"`, since a notebook has
+  no token budget to defend.
+- **`include_estimation`** (`"none" | "summary" | "full"`, default `"full"`)
+  and **`max_coefficients`**. `"summary"` keeps the model-level block and
+  diagnostics but drops per-term rows — the right setting for specifications
+  dominated by `i.year i.firm` nuisance terms. `estimation.n_coefficients`
+  always reports the model's true term count and
+  `estimation.coefficients_truncated` flags any cut, so trimmed output can
+  never be mistaken for a smaller model.
+
+#### Fixed: two silent numeric-correctness bugs
+
+- **Standard errors were blanked whenever `e(V)` was returned by reference.**
+  Any model whose VCV exceeded the inline cell cap (roughly >100 terms) came
+  back with `se`, `statistic`, `p_value` and both CI bounds `null` for *every*
+  coefficient, while still reporting `source: "e_b_v"` as though inference had
+  been computed. The estimation builder now resolves `matrix://` refs, so the
+  wire representation no longer degrades the numbers. `e(b)` by reference is
+  resolved too, where it previously suppressed the estimation block entirely.
+- **`r()` did not survive to the next call.** The runner's own probes are all
+  r-class (`graph dir` before a run, `graph export` after), so the canonical
+  two-call pattern — `summarize price`, then `display r(mean)` — read back a
+  missing value. All internal housekeeping now runs inside a Stata
+  `_return hold` / `_return restore` pair. Pre-existing; the new `log query`
+  probe would have widened it to runs with `include_graphs="none"`.
+- **Stata's missing values leaked as `8.988e+307`.** `sfi` returns system
+  missing (`.`) and extended missings (`.a`–`.z`) as ordinary doubles at or
+  above `2^1023`. These reached the wire unconverted in `r()`/`e()` scalars and
+  in every matrix cell — an omitted base level's standard error arrived as a
+  number an agent would format into a table. All numerics now normalize to
+  `null`, matching what SCHEMA.md already required of scalars.
+
+#### Errors inside `do`-files are now located
+
+- `error.line` and `error.context` are resolved **inside the invoked script**.
+  A failing `do "analysis.do"` previously produced `line: null`, empty
+  before/after context, and `message: "end of do-file"` — Stata's epilogue
+  rather than the diagnosis. The transcript parser now skips structural
+  boilerplate (`end of do-file`, `--Break--`), folds `///` continuation
+  fragments back into one logical command, and resolves the failing command
+  against the script on disk. New `error.source_file` names the file that
+  `error.line` indexes into.
+- **A failed run now produces a log.** pystata raises the whole transcript as
+  its exception message and leaves stdout empty, so failures used to come back
+  with no `log.head`, no `log://` ref and nothing for `search_log` to search —
+  exactly when the transcript matters most.
+
+#### A failed run can no longer poison its session
+
+- New **`auto_close_logs`** (default `true`) closes log handles that a *failed*
+  run opened. A script aborting between `log using` and `log close` previously
+  left the handle dangling, and every subsequent run in that session died with
+  r(604) — an error about a different run, with no suggestions attached.
+  Handles opened by earlier runs are left alone; a `log_closed` warning records
+  what was closed.
+- r(604) and r(606) are now the typed `ErrorKind.LOG_STATE` with Stata's
+  official labels and rc-specific fixes (`capture log close _all` for 604,
+  `capture log close` guarding for 606), and `recovery.retriable = true`.
+
+#### Long runs no longer block the caller
+
+- **`timeout_ms` is an advertised `stata_run` argument.** It was enforced but
+  not exposed, so passing it was an input-validation error.
+- **`run_in_background`** returns a `job_id` immediately; poll with the new
+  **`stata_run_status`** tool (`wait_ms` blocks up to 60 s instead of
+  busy-polling) and enumerate with **`list_background_runs`**. Intended for
+  bootstraps, permutation tests and grouped estimation loops.
+- **Fixed: the worker lock was acquired without a deadline**, so a call queued
+  behind a long run blocked indefinitely no matter what `timeout_ms` said.
+  `timeout_ms` now budgets the whole call, queueing included, and contention
+  returns the new `rc=-5` / `error.kind="session_busy"` — distinct from a
+  timeout because nothing was submitted to Stata and the healthy worker is
+  *not* killed.
+
+#### Inline graphs are visible again
+
+- `include_graphs: "inline"` now delivers graph bytes as MCP `ImageContent`
+  blocks instead of base64 inside a JSON string. The old form was unreadable to
+  a vision-capable client and was pure token cost. `graphs[].inline` is cleared
+  from the structured body so the bytes cross the wire once, and
+  `inline_delivered` records the outcome. At most 4 images per response (the
+  rest stay reachable via `get_graph(ref)`, reported as an
+  `inline_graphs_truncated` warning); `pdf` is never sent as an image block.
+
+#### Generated files are reported
+
+- New top-level **`outputs`** lists files a run created or modified in its
+  working directory (`esttab` tables, exported graphs, saved `.dta`), with
+  `path` / `bytes` / `created`. Previously this required
+  `persist_log_files=true` *and* `origin_path` — a dependency that was not
+  obvious from `persist_generated_files` defaulting to `true`, so the field was
+  always `null` in practice. Controlled by the new `track_output_files`
+  (default `true`); working directories above 5,000 files skip detection and
+  say so with an `output_tracking_skipped` warning rather than reporting a
+  partial answer.
+
+### Added
+
+- The VS Code extension now accepts Stata's familiar `Ctrl+D` shortcut
+  (`Cmd+D` on macOS) for running the selection or current line, while retaining
+  `Cmd/Ctrl+Enter`. The extension README also documents how to customize the
+  `stataCode.runSelection` keybinding and resolve conflicts.
+
+## 0.10.0 — 2026-07-24
+
+Surface-coverage and safety additions so every Stata usage scenario — MCP,
+Jupyter, VS Code, and now a plain Bash/terminal — routes through the same typed
+engine, and so an autonomous agent loop cannot escape to the OS. Also broadens
+execution to **Stata 13+ without pystata** (console backend) and adds a
+zero-Python standalone binary and one-click VS Code onboarding. The default
+README is now Chinese (`README.md`); the English README moves to `README.en.md`.
+
+### Added
+
+- **Command-safety guard** (`stata_code.core.policy`). OS-escape / file-deletion
+  commands (`shell`, `winexec`, `erase`, `rm`, `rmdir`, and the `!` shell escape)
+  are screened out of submitted code *before* Stata runs, at both the
+  subprocess-pool boundary and the in-process runner. A blocked run returns
+  `ok=false`, `rc=-4`, `error.kind="policy_blocked"` (a new 32nd `ErrorKind`)
+  without touching Stata. Configurable from the environment so it crosses the
+  worker boundary: `STATA_CODE_COMMAND_POLICY` (`enforce` default / `warn` /
+  `off`), `STATA_CODE_POLICY_ALLOW`, `STATA_CODE_POLICY_BLOCK`. It is a guard
+  rail, not a sandbox.
+- **`stata-code run`** — execute a `.do` file, one or more `-e` snippets, or
+  stdin through the subprocess pool and print the `RunResult` (text summary or
+  `--json`), with `--session`, `--timeout-ms`, `--full-log`, and `--graphs DIR`.
+  Exit code is `0` on success, `1` on a Stata/adapter error. This is the
+  Bash / plain-terminal surface: any agent that can shell out gets the full
+  structured error loop without MCP.
+- **`stata-code setup`** — opt-in, config-mutating counterpart to `doctor`.
+  Writes the `stata-code` MCP server entry into Claude Code (`.mcp.json`),
+  Cursor (`.cursor/mcp.json`), and VS Code (`.vscode/mcp.json`), preserving other
+  servers and backing up any file it overwrites (`--dry-run`, `--python`,
+  `--json`). Codex (TOML) and Claude Desktop are emitted as copy-paste snippets.
+- **Static do-file linting** (`stata_code.core.lint`), exposed as the `lint_do`
+  MCP tool (19th tool) and `stata-code lint`. A Stata-free syntactic check —
+  unbalanced braces, a `program` / `mata` / `python` block with no `end`, a stray
+  `end`, and a dangling `///` — so an agent can catch a class of mistakes before
+  spending a run. Advisory: a clean result is not a correctness guarantee.
+- Public API: `lint_code`, `LintFinding`, `CommandPolicy`, `Violation`,
+  `policy_from_env`.
+- **Console (batch) backend** (`stata_code.core.console`, `Backend.CONSOLE`).
+  Drives the Stata command-line executable in batch mode and parses the log plus
+  a marker-delimited results dump into the same v1.0 `RunResult` — typed
+  `r()`/`e()` scalars and macros, the estimation matrices (`e(b)`/`e(V)`/
+  `r(table)`), the error taxonomy, warnings, and dataset metadata. Works with
+  **Stata 13+** and **no pystata**. Exposed as `stata_code.run_console()`,
+  `stata-code run --backend {auto,pystata,console}`, and a `doctor` check. Stateless
+  per call; graphs not captured yet. Public API adds `run_console`,
+  `console_available`, `find_stata_cli`, `ConsoleNotAvailable`.
+- **Zero-Python standalone binary** — `scripts/build_standalone.py` (PyInstaller)
+  and a CI workflow template (`packaging/standalone.github-workflow.yml`, copy to
+  `.github/workflows/`) build a self-contained `stata-code` executable per OS.
+  Paired with `--backend console` it needs no Python on the target machine.
+- **One-click VS Code onboarding** — when the MCP server is missing, the extension
+  offers to create a workspace `.venv` and install the package; also available as
+  the command-palette entry "Stata: Set Up MCP Server (create .venv)".
+- **Benchmark harness** (`benchmarks/`) — a tool-agnostic scaffold measuring
+  iterations-to-correct and tokens for typed vs. raw-log execution, with an offline
+  mock runner wired end-to-end and CI-covered.
+- Chinese quickstart ([docs/quickstart.zh.md](docs/quickstart.zh.md)).
+
+## 0.9.1 — 2026-07-22
+
+Quality-hardening pass: correctness fixes from an adversarial code review,
+a large offline test expansion, and a documentation accuracy audit. No new
+features, no API or schema changes.
+
+A second review round found that three of the first round's fixes were
+incomplete or had introduced a regression of their own; those are the first
+four entries below. The common failure was testing a fix in isolation while
+the real call path stayed uncovered — the new tests go through `SessionPool`
+rather than `WorkerProcess` wherever the pool is what carries the bug, and
+each was confirmed to fail against the pre-fix source.
+
+### Fixed
+
+- **A read-only status query no longer kills a healthy mid-run worker.**
+  Once `send_simple_op` learned to bail out rather than block on the worker
+  lock, that bail-out surfaced as a `_WorkerTimeout` — and
+  `SessionPool.stata_info` treated "busy" and "dead" identically: it
+  SIGTERMed the worker and dropped it from the pool. Calling the
+  `stata_info` tool during a long `bootstrap`/`reghdfe` therefore returned
+  the user's run as `adapter_crash` and wiped the session's loaded data, and
+  because the MCP layer swallows the exception the kill was invisible in the
+  tool response. Busy is now a distinct `_WorkerBusy` signal (a
+  `_WorkerTimeout` subclass, so warn-only callers are unaffected) that never
+  kills.
+- **A timed-out read no longer steals the next request's response.** Each
+  request spawned its own `readline()` thread and abandoned it on deadline
+  overrun; the orphan stayed blocked on the pipe and consumed the *following*
+  response. The next `execute()` then burned its entire timeout (default
+  600 s) waiting for a reply that had already been eaten, after which the
+  pool killed a perfectly healthy worker. Reads now come off one long-lived
+  pump thread per worker, which cannot orphan, and responses carrying a
+  stale request id are discarded rather than mis-matched.
+- **`send_simple_op` honors its full timeout budget.** The read deadline was
+  computed *after* acquiring the worker lock, so a call that spent most of
+  its budget waiting for the lock got a second full budget for the read —
+  worst case 2x `timeout_ms`, and `list_session_info_detailed` iterates
+  workers serially, so N busy workers stalled the status tool for
+  `2 * timeout * N`. The deadline is now fixed up front.
+- **Repeated identical notes are no longer double-counted.** The previous
+  de-dup short-circuited before recording the matched span, so the second and
+  later copies of an identical `note:` line went unclaimed and were picked up
+  again by the generic-note pass — the exact double-count the fix was meant
+  to remove, reachable from any `foreach` loop re-running the same
+  regression.
+- **`stata_info`'s worker eviction is identity-checked.** It popped the
+  session's worker unconditionally, so a concurrent `execute()` that had
+  already noticed the death and registered a replacement got *its* worker
+  evicted — leaking a live pystata subprocess that `request_cancel` /
+  `kill_session` / `shutdown` could no longer reach.
+- **`kill()`'s already-dead fast path is identity-checked** too, matching its
+  own slow path: it could otherwise clear a handle a concurrent `execute()`
+  had just respawned into, stranding a live subprocess mid-request.
+- **`TypeError` is no longer blanket-classified as a caller error.** The
+  guard wrapped the whole `execute()` call, not just argument binding, so a
+  `TypeError` raised deep inside result collection told the agent its
+  arguments were wrong when nothing was — and routed the failure out of the
+  `adapter_crash` path that would have recycled the wedged worker. Malformed
+  options are now caught by signature binding before the call; deeper
+  `TypeError`s stay in the `worker_error` bucket.
+- **Session ids longer than Stata's 32-character name cap are mapped.** They
+  were passed through verbatim as a frame name, so `frame create` failed with
+  rc 198; they now route through the hashed mapped-frame path.
+
+- **Worker stderr pipe is now drained continuously.** The subprocess pool
+  only read a worker's stderr after death, so a worker whose cumulative
+  stderr output (pystata banners, Stata C-side messages, warnings) exceeded
+  the ~64 KB OS pipe buffer blocked mid-write and never responded — the
+  parent then misreported the healthy worker as a `timeout` and killed it
+  (or hung forever with `timeout_ms=None`). A per-worker daemon thread now
+  drains stderr into a bounded tail buffer, which also enriches crash
+  messages.
+- **A worker-reported failure no longer destroys the session.** A live
+  worker answering `{ok:false, error_kind:"worker_error"}` was handled like
+  a dead process: killed and dropped from the pool, wiping the session's
+  loaded dataset and `r()`/`e()` state — reachable from a mere argument
+  type typo. Such failures now surface as structured errors while the
+  worker (and the session's data) survives; the worker also classifies
+  `TypeError` as `invalid_request`.
+- **LRU eviction skips busy workers.** `last_used` is stale for a mid-run
+  worker, so capacity pressure from an unrelated new session preferred the
+  longest-running request as its eviction victim (SIGTERM mid-run →
+  `adapter_crash`, session data lost). Eviction now skips in-flight
+  workers, bumps `last_used` at request start, and kills victims outside
+  the pool lock; timeout/error cleanup no longer removes a newer worker
+  handle registered in the interim.
+- **Status queries no longer hang behind a long run.** `send_simple_op`
+  blocked unboundedly on the worker lock held by an in-flight `execute()`;
+  lock acquisition now counts against the same timeout budget and surfaces
+  as a per-session "busy" warning.
+- **`rc` is taken from the last `r(NNN);` in a failure transcript.** An
+  earlier successful command echoing a literal `r(NNN);` (display string,
+  help output) previously hijacked the return code — and therefore the
+  `ErrorKind` classification, suggestions, and recovery contract.
+- **Indented notes are no longer double-counted as warnings.** An indented
+  `note: … omitted because of collinearity` line produced both an
+  `omitted_collinear` and a generic `note` warning.
+- **A session literally named `"default"` no longer aliases `"main"`.** In
+  in-process mode it silently shared `main`'s Stata frame (one dataset for
+  two nominally distinct sessions); it now routes through the private
+  mapped-frame path.
+- **Matrices with missing sfi row/col names are no longer dropped.**
+  Positional names (`r1…`/`c1…`) are synthesized instead of letting the
+  shape validator silently discard successfully read values.
+- **`get_graph(format=…)` is no longer a silent no-op.** Requesting a
+  format different from the stored one now returns `invalid_request`
+  instead of returning mismatched bytes.
+- **Post-run file snapshots survive escaping symlinks.** A symlink inside
+  the working dir pointing outside it crashed `changed_output_files` with
+  an uncaught `ValueError`; it is now skipped.
+
+### Added
+
+- **≈300 new offline tests** (467 → 724 passing without Stata); coverage
+  73% → 90% overall (`mcp/server` 74→99%, `kernel` 73→98%, `_pool` 68→96%,
+  `_refs` 68→100%, `log_artifacts` 76→96%, `runner` 29→72%). The package is
+  now fully mypy-clean, including the previously unchecked top-level
+  modules.
+- **Community health files**: `CONTRIBUTING.md` (dev setup + the exact CI
+  gates), `CODE_OF_CONDUCT.md` (Contributor Covenant 2.1), `SECURITY.md`,
+  and GitHub issue templates that ask for `stata-code doctor` output.
+- **Second-round regression tests** for the eight fixes listed above, each
+  verified to fail against the pre-fix source (823 passing locally with
+  Stata). They drive real subprocess workers through `SessionPool` — a
+  status query racing a live run, a worker that drops its first request, a
+  worker that releases the lock partway — rather than asserting on a mocked
+  seam, which is what let the first round's regressions through.
+
+### Docs
+
+- Regenerated the skill's `rc → kind` reference from `errors.py` — it still
+  documented the pre-0.9.0 mappings the taxonomy audit had corrected; fixed
+  the "32 kinds" claims (the enum has 31).
+- `plugin.json` install verify no longer uses `stata-code-mcp --help`
+  (which hangs); it runs `stata-code doctor --no-stata-probe`.
+- PUBLISHING.md / AGENTS.md version-bump lists now cover all eight version
+  literals the release gate checks; removed the stale claim that a built
+  `.vsix` ships in the repo.
+- SCHEMA.md documents `EstimationResult.command_family` and `.diagnostics`;
+  status lines updated v0.8 → v0.9; README.zh.md re-synced with the English
+  README (plugin-marketplace install, per-client MCP config table, Open VSX
+  and first-activation notes, cell/section conventions).
+
 ## 0.9.0 — 2026-06-23
 
 ### Fixed
@@ -226,8 +558,8 @@ to semver-major.minor for the result schema (see `SCHEMA.md` §6).
 
 ### Added
 
-- **Claude Code plugin marketplace manifest.** `.claude-plugin/marketplace.json`
-  + `.claude-plugin/plugin.json` expose the repo as a single-plugin
+- **Claude Code plugin marketplace manifest.** `.claude-plugin/marketplace.json` +
+  `.claude-plugin/plugin.json` expose the repo as a single-plugin
   marketplace, so users can install everything (MCP server config + agent
   skill) with `claude plugin marketplace add brycewang-stanford/stata-code`
   followed by `claude plugin install stata-code`.
@@ -668,6 +1000,9 @@ Aggregated from the prior `Unreleased` section; covers 0.6.1 and 0.6.2.
   `vscode/package.json`, and the VSCode MCP-client handshake all
   declare `0.3.0`.
 
+- **MCP server tool count is now 8** (added `get_matrix`,
+  `cancel_session`).
+
 ### Added
 
 - **VSCode extension v0.3 — full UI surface** (`vscode/`). Beyond the
@@ -750,11 +1085,6 @@ Aggregated from the prior `Unreleased` section; covers 0.6.1 and 0.6.2.
   in-process and has no clean cancel primitive). Hard interruption
   remains deferred to the subprocess-based runtime planned for v0.3+.
 
-### Changed
-
-- **MCP server tool count is now 8** (added `get_matrix`,
-  `cancel_session`).
-
 ## [0.2.0] — 2026-05-07
 
 The first release that actually ships an end-to-end Stata pipeline. The v1.0

{stata_code-0.9.0 → stata_code-0.11.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: stata-code
-Version: 0.9.0
+Version: 0.11.0
 Summary: Agent-native Stata bridge — one core, multiple frontends (MCP, Jupyter, VSCode)
 Project-URL: Homepage, https://github.com/brycewang-stanford/stata-code
 Project-URL: Repository, https://github.com/brycewang-stanford/stata-code
@@ -42,7 +42,7 @@ Description-Content-Type: text/markdown
 </p>
 
 <p align="center">
-  <a href="README.md"><strong>English</strong></a> | <a href="README.zh.md">中文</a>
+  <a href="README.en.md"><strong>English</strong></a> | <a href="README.md">中文</a>
 </p>
 
 # stata-code
@@ -99,13 +99,13 @@ Then just ask:
 
 `stata-code` writes the do-file, runs it, returns the table, and interprets the result — and can re-estimate the same ATT with [StatsPAI](https://github.com/brycewang-stanford/StatsPAI) to confirm the two stacks agree. These workflows ship as one-call MCP prompts (`did_event_study`, `iv_2sls`, `rdd`, `publication_table`, `cross_validate_did`) backed by an on-demand [recipe library](skills/stata-code/references/recipes/).
 
-**Why `stata-code`:** MIT-licensed · ships as an MCP server, a bundled agent skill, a Jupyter kernel, **and** a VS Code extension · one structured, token-economy result schema (typed errors, native `r()` / `e()`) · cross-stack validation with StatsPAI for the Cunningham check.
+**Why `stata-code`:** MIT-licensed · ships as an MCP server, a bundled agent skill, a Jupyter kernel, a VS Code extension, **and** a plain-terminal CLI (`stata-code run`) · one structured, token-economy result schema (typed errors, native `r()` / `e()`) · runs on **Stata 17+ via pystata *or* Stata 13+ via a console backend that needs no pystata** · a zero-Python standalone binary · a default command-safety guard for unattended agents · cross-stack validation with StatsPAI for the Cunningham check.
 
 ```text
                     ┌────────────────────────────────────────┐
                     │     stata-code core (Python)           │
                     │                                        │
-                    │   • pystata adapter (Stata 17+)        │
+                    │   • pystata 17+ / console 13+ backends │
                     │   • v1.0 unified result schema         │
                     │   • token-economy defaults             │
                     │   • multi-session via Stata frames     │
@@ -118,10 +118,13 @@ Then just ask:
               └─────────────┘  └────────────┘  └─────────────────┘
 ```
 
-**Status: v0.8 (June 2026)** — the core, MCP server, Jupyter kernel, and VS Code extension work end-to-end against Stata 18 MP. The test suite covers schema, runner, MCP, kernel, notebook, run-index, subprocess-pool, and VS Code modules; CI also checks linting, type safety, schema generation, package metadata, and VSIX packaging. License: **MIT**.
+A fourth frontend, a **plain-terminal CLI** (`stata-code run` / `lint` / `setup`), gives any agent that can shell out — or a bare terminal — the same typed `RunResult`. And the core runs two backends: **pystata** (Stata 17+, in-memory sessions) or a **console backend** (Stata 13+ in batch mode, no pystata) — both returning the identical schema.
 
-Three workflows the current tree explicitly supports for end users and agents:
+**Status: v0.10 (July 2026)** — the core, MCP server, Jupyter kernel, VS Code extension, and CLI work end-to-end against Stata 18 MP; the console backend broadens coverage to Stata 13+ without pystata. The test suite covers schema, runner, console parser, MCP, kernel, notebook, run-index, subprocess-pool, command policy, linter, and VS Code modules; CI also checks linting, type safety, schema generation, package metadata, and VSIX packaging. License: **MIT**.
 
+Four workflows the current tree explicitly supports for end users and agents:
+
+- **Run Stata from a plain terminal or a zero-Python binary.** `stata-code run analysis.do` (or `-e "code"`, or stdin) prints the same structured `RunResult` — text or `--json` — so any agent that can shell out gets the typed error loop without MCP. `--backend console` runs **Stata 13+ with no pystata**, and a standalone binary needs no Python install at all. See [From the Command Line](#from-the-command-line-bash).
 - **Run Stata code from a Jupyter notebook.** `pip install "stata-code[kernel]"` + `stata-code-kernel install --user` registers a **Stata** kernel that the Jupyter Notebook UI, JupyterLab, and the VS Code Jupyter extension all pick up by name. Cells render Stata logs, graphs, and warnings inline (the kernel logo bundled since v0.5 makes it appear in VS Code's kernel picker too). See [As a Jupyter Kernel](#as-a-jupyter-kernel).
 - **Optional agent "fix and rerun" loop.** `stata_run` returns typed `error.kind/line/context` plus `suggestions` on every failure. By default Claude Code only reports diagnostics — but if you explicitly say "fix this and rerun until it passes", the agent uses the same fields to edit your `.do` file and re-call `stata_run` until the run is green. The repair loop is **opt-in**: failed runs are diagnostics first, not automatic rewrite permission. See [Error Recovery in Agent Workflows](#error-recovery-in-agent-workflows).
 - **Economist workflow guides.** The bundled skill and cookbook now cover
@@ -222,6 +225,72 @@ else:
         print("hint:", s.action)               # "Did you mean `mpg`?"
 ```
 
+### From the Command Line (Bash)
+
+Any agent or script that can shell out gets the same structured engine — no MCP
+required. `stata-code run` executes a `.do` file, one or more `-e` snippets, or
+code piped on stdin, and prints the `RunResult`:
+
+```bash
+stata-code run analysis.do                 # run a do-file, text summary
+stata-code run -e "sysuse auto" -e "regress mpg weight"
+stata-code run analysis.do --json          # full RunResult JSON (for agents)
+echo "summarize price" | stata-code run -   # read code from stdin
+stata-code run model.do --graphs out/       # also export graphs to out/
+stata-code run job.do --session modelA --timeout-ms 120000
+```
+
+Exit code is `0` on success and `1` on a Stata / adapter error, so it drops into
+CI and scripted fix-and-rerun loops. `stata-code lint analysis.do` runs the
+static checker (unbalanced braces, missing `end`, dangling `///`) without
+touching Stata.
+
+**Backends — Stata 13+ without pystata.** `--backend` selects how code runs:
+`pystata` (Stata 17+, in-memory sessions), `console` (Stata 13+ batch, no pystata,
+stateless), or `auto` (default: pystata when available, else console). The console
+backend drives the Stata command-line executable and parses the log into the same
+typed `RunResult` — typed `r()`/`e()`, the estimation table, and the error
+taxonomy — so older Stata and pystata-free environments are first-class:
+
+```bash
+stata-code run analysis.do --backend console
+export STATA_CODE_STATA_CLI=/usr/local/stata18/stata-mp   # if not auto-found
+```
+
+**Zero-Python binary.** A standalone `stata-code` executable (built by
+[`scripts/build_standalone.py`](scripts/build_standalone.py), with a ready-to-use
+CI workflow template at
+[`packaging/standalone.github-workflow.yml`](packaging/standalone.github-workflow.yml))
+bundles the runtime and needs no Python install. Paired with `--backend console`,
+it is a fully Python-free path to typed Stata results.
+
+### One-Command Client Setup
+
+`stata-code setup` writes the MCP server entry into a client's config — the
+opt-in, mutating counterpart to the read-only `doctor`. It preserves other
+servers and backs up any file it overwrites:
+
+```bash
+stata-code setup --all                 # Claude Code, Cursor, VS Code (project)
+stata-code setup --claude --dry-run    # preview without writing
+stata-code setup --vscode --python .venv/bin/python   # pin an interpreter
+stata-code setup --codex               # print a copy-paste TOML snippet
+```
+
+### Command Safety
+
+By default the runner blocks OS-escape and file-deletion commands (`shell`,
+`winexec`, `erase`, `rm`, `rmdir`, and the `!` shell escape) *before* they reach
+Stata, so an autonomous agent loop can't delete files or run arbitrary shell
+commands. A block returns a `policy_blocked` result (`rc=-4`) rather than
+running. It is a guard rail, not a sandbox; tune it with environment variables:
+
+```bash
+STATA_CODE_COMMAND_POLICY=off      # disable the guard entirely
+STATA_CODE_POLICY_ALLOW=shell      # allow specific commands (comma-separated)
+STATA_CODE_POLICY_BLOCK=python     # block additional commands
+```
+
 ### As an MCP Server
 
 After `pip install "stata-code[mcp]"`, the `stata-code-mcp` binary is on your `PATH`. You can wire it into Claude Code, Cursor, Claude Desktop, or any other MCP-compatible client.
@@ -243,7 +312,7 @@ claude mcp add stata-code --scope local -- stata-code-mcp
 claude mcp add stata-code --scope project -- stata-code-mcp
 ```
 
-Then launch `claude` and type `/mcp` to confirm `stata-code` shows up with its 18 tools (`stata_run`, `stata_info`, `get_log`, `search_log`, `get_graph`, `get_matrix`, `inspect_data`, `install_package`, `list_sessions`, `cancel_session`, `reset_session`, `notebook_outline`, `notebook_get_cell`, `notebook_locate`, `notebook_edit_cell`, `notebook_insert_cell`, `notebook_delete_cell`, `list_runs`).
+Then launch `claude` and type `/mcp` to confirm `stata-code` shows up with its 21 tools (`stata_run`, `stata_run_status`, `list_background_runs`, `stata_info`, `get_log`, `search_log`, `get_graph`, `get_matrix`, `inspect_data`, `lint_do`, `install_package`, `list_sessions`, `cancel_session`, `reset_session`, `notebook_outline`, `notebook_get_cell`, `notebook_locate`, `notebook_edit_cell`, `notebook_insert_cell`, `notebook_delete_cell`, `list_runs`).
 
 #### Error Recovery in Agent Workflows
 
@@ -331,17 +400,20 @@ If an OpenAI-backed client reports `API Error: 400 Invalid schema for function
 upgrade to `stata-code>=0.6.5`, then restart the MCP client. Older server
 processes keep advertising the stale schema until they are restarted.
 
-The MCP server registers 18 tools:
+The MCP server registers 21 tools:
 
 | Tool | Purpose |
 | --- | --- |
-| `stata_run` | Execute Stata code and return a v1.0 RunResult JSON |
+| `stata_run` | Execute Stata code and return a v1.0 RunResult JSON; `include_results` / `include_estimation` bound the payload, `timeout_ms` enforces a hard deadline, `run_in_background` returns a job id |
+| `stata_run_status` | Poll a background run's status and result; `wait_ms` blocks up to a bounded time |
+| `list_background_runs` | List background runs tracked by this server |
 | `stata_info` | Report Stata edition, version, and capabilities |
 | `get_log` | Fetch the full log behind a `log://` ref |
 | `search_log` | Search matching lines inside a stored `log://` payload |
 | `get_graph` | Fetch graph bytes behind a `graph://` ref (`ImageContent`) |
 | `get_matrix` | Fetch matrix payloads behind a `matrix://` ref |
 | `inspect_data` | Run `describe` + `codebook` and return compact dataset metadata |
+| `lint_do` | Statically check do-file source (unbalanced braces, missing `end`, dangling `///`) before spending a run |
 | `install_package` | Install an SSC or explicit `net install` package and verify it resolves |
 | `list_sessions` | Enumerate live sessions |
 | `cancel_session` | Cancel a session; the subprocess-backed path terminates in-flight runs and short-circuits pending ones |
@@ -485,7 +557,9 @@ stata_code/
 │   ├── runner.py      # in-process execute(); collects everything via sfi
 │   └── _pool.py       # subprocess workers for public API / MCP hard timeouts
 ├── mcp/
-│   └── server.py      # MCP server (18 tools)
+│   ├── server.py      # MCP server (21 tools)
+│   └── ...
+├── core/console.py    # console (batch) backend — Stata 13+, no pystata
 └── kernel/
     └── kernel.py      # Jupyter kernel
 ```
@@ -496,18 +570,33 @@ stata_code/
 
 ## Comparison
 
-| | stata-code | SepineTam/stata-mcp | hanlulong/stata-mcp | nbstata |
-| --- | --- | --- | --- | --- |
-| License | **MIT** | AGPL-3.0 | MIT | GPL-3.0 |
-| Standalone MCP | ✓ | ✓ | bundled with VS Code | — |
-| Jupyter kernel | ✓ | — | — | ✓ |
-| Unified result schema | ✓ ([SCHEMA.md](SCHEMA.md)) | per-tool | per-tool | per-tool |
-| Token-economy defaults | ✓ (log refs, graph refs) | — | — | — |
-| Typed errors + suggestions | ✓ (31 kinds) | — | — | — |
-| Multi-session | ✓ (Stata frames) | partial | — | — |
-| Mature ecosystem | early | ✓ (statamcp.com, cookbook) | ✓ (11k installs) | ✓ |
-
-`stata-code` is the younger, MIT-licensed, agent-native alternative in this problem space. Among the AGPL options, SepineTam's `stata-mcp` is currently more mature; `stata-code` is aimed at cases where copyleft contagion is unacceptable and agents need structured results.
+| | stata-code | SepineTam/stata-mcp | hanlulong/stata-mcp | stata-all-in-one | nbstata |
+| --- | --- | --- | --- | --- | --- |
+| License | **MIT** | AGPL-3.0 | MIT | MIT | GPL-3.0 |
+| Standalone MCP | ✓ | ✓ | bundled with VS Code | — (bespoke HTTP + copy-paste) | — |
+| Jupyter kernel | ✓ | — | — | — | ✓ |
+| Unified result schema | ✓ ([SCHEMA.md](SCHEMA.md)) | per-tool | per-tool | raw log to the agent | per-tool |
+| Typed errors + suggestions | ✓ (32 kinds) | — | — | — | — |
+| Token-economy defaults | ✓ (log refs, graph refs) | — | — | — | — |
+| Command-safety guard | ✓ (`shell`/`erase`/`rmdir`/`!`) | ✓ (27-rule) | — | — | — |
+| Bash / plain-terminal CLI | ✓ (`stata-code run`) | ✓ | — | — | — |
+| One-command client setup | ✓ (`stata-code setup`) | ✓ (`install --all`) | bundled | — | — |
+| Static pre-run lint | ✓ (`lint_do`) | — | — | — | — |
+| Stata 13–16 (no pystata) | ✓ (console backend) | ✓ | — | ✓ (COM/dylib) | — |
+| Zero-Python install | ✓ (standalone binary) | — | — | ✓ (VS Code-native) | — |
+| Human IDE polish (data viewer, inline graphs) | growing | — | ✓ | ✓ (strongest) | ✓ |
+| Multi-session | ✓ (Stata frames) | partial | — | — | — |
+| Mature ecosystem | early | ✓ (statamcp.com) | ✓ (11k installs) | ✓ (distributor-backed) | ✓ |
+
+`stata-code` owns the **agent-native, typed-contract** lane: one structured
+`RunResult` schema across MCP, Jupyter, VS Code, and a plain-terminal CLI; a
+32-kind error taxonomy with recovery contracts; token-economy refs; and now a
+console backend (Stata 13+) plus a zero-Python binary. Editor-first tools like
+`stata-all-in-one` lead on human IDE polish and hand the agent raw log text;
+`stata-code` matches their onboarding/version reach while keeping the typed
+execution contract they don't have. See
+[docs/competitive-landscape.md](docs/competitive-landscape.md) for the full
+teardown.
 
 ---
 
@@ -522,8 +611,13 @@ stata_code/
 - Graph capture: `png` / `svg` / `pdf` with ref store and source-command attribution
 - Log truncation with ref store
 - Warning extraction: 5 categories + generic notes
-- 31-kind error taxonomy with canonical suggestions
-- MCP server: 18 tools, including notebook navigation / search / atomic edits, the run-bundle index (`list_runs`), log grep (`search_log`), dataset inspection (`inspect_data`), and package installation (`install_package`)
+- 32-kind error taxonomy with canonical suggestions
+- MCP server: 21 tools, including notebook navigation / search / atomic edits, the run-bundle index (`list_runs`), log grep (`search_log`), dataset inspection (`inspect_data`), static linting (`lint_do`), and package installation (`install_package`)
+- Command-safety guard: OS-escape / file-deletion commands (`shell`, `winexec`, `erase`, `rm`, `rmdir`, `!`) are blocked before Stata runs; configurable via `STATA_CODE_COMMAND_POLICY` / `STATA_CODE_POLICY_ALLOW` / `STATA_CODE_POLICY_BLOCK`
+- Bash / plain-terminal surface: `stata-code run` (a `.do` file, `-e` snippets, or stdin) prints the same structured `RunResult` any agent that can shell out can consume; `stata-code lint` runs the linter; `stata-code setup` writes MCP client configs
+- Console (batch) backend (`core/console.py`, `--backend console`, `run_console()`): drives the Stata command-line executable, parses the log into the same typed `RunResult`, and supports **Stata 13+ with no pystata**
+- Zero-Python standalone binary ([`scripts/build_standalone.py`](scripts/build_standalone.py) + CI workflow template [`packaging/standalone.github-workflow.yml`](packaging/standalone.github-workflow.yml)); with `--backend console` it is a fully Python-free path to typed results
+- One-click VS Code onboarding: the extension offers to create a workspace `.venv` and install the server (command palette: “Stata: Set Up MCP Server”)
 - Jupyter kernel: rewired to the v1.0 pipeline, kernel logos bundled
 - Matrix size cap + `get_matrix(ref)` for large matrices (>10k cells)
 - Subprocess-backed hard timeout and cancellation for the public Python API and MCP server: `timeout_ms`, `cancel(session_id)`, and MCP `cancel_session`
@@ -541,12 +635,16 @@ stata_code/
 
 ### Next Up
 
-- Console fallback for Stata 11–16, re-implemented against the v1.0 schema
-- Decide whether to move the Jupyter kernel from the direct in-process runner to the subprocess pool, or keep documenting the current interactivity-first tradeoff
-- Extra VS Code polish: extension-host end-to-end tests, first-run diagnostics, and command palette UX
+- Streaming / progress for long runs (`log.complete:false`, incremental log lines) so 20-minute `boottest` / `csdid` jobs report before they finish
+- Hard timeout / cancellation for the Jupyter kernel (move it from the direct in-process runner to the subprocess pool, or an equivalent)
+- Console backend: graph capture and richer matrix coverage (values currently materialized for the estimation matrices; state is per-call)
+- Human IDE polish to match editor-first tools: inline graph rendering + DPI export, a scalable data viewer, and an optional "attach to a running Stata" backend
+- Publish the reliability/token [benchmark](benchmarks/) results (typed contract vs. raw-log tools) as evidence, not a claim
 - **v1.0** — Stable schema, broader Stata edition coverage
 
-See [SCHEMA.md §7](SCHEMA.md) for explicitly out-of-scope items.
+See [docs/competitive-landscape.md](docs/competitive-landscape.md) for how these
+priorities line up against comparable tools, and [SCHEMA.md §7](SCHEMA.md) for
+explicitly out-of-scope items.
 
 ---
 
@@ -565,10 +663,17 @@ The `stata_required` marker tags the real-Stata integration tests. CI uses `pyte
 
 ## Contributing
 
+Contributions are welcome — see [CONTRIBUTING.md](CONTRIBUTING.md) for the
+development setup, the checks CI runs, and PR guidelines. In short:
+
 - Read [LICENSE-POLICY.md](LICENSE-POLICY.md) before opening a PR.
 - Add a one-line acknowledgement to your first PR description; the template is in the policy file.
 - Tests are required for any new schema field or runner behavior.
 
+Bug reports and support questions go through
+[GitHub issues](https://github.com/brycewang-stanford/stata-code/issues);
+security reports go through [SECURITY.md](SECURITY.md).
+
 ---
 
 ## License