stata-code 0.12.1__tar.gz → 0.13.0__tar.gz

{stata_code-0.12.1 → stata_code-0.13.0}/CHANGELOG.md

@@ -6,6 +6,148 @@ to semver-major.minor for the result schema (see `SCHEMA.md` §6).
 
 ## [Unreleased]
 
+## 0.13.0 — 2026-10-03
+
+A data viewer for `.dta` files. Opening a dataset in VS Code used to mean
+either launching Stata or converting to CSV and losing the variable labels,
+value labels, formats and notes on the way. The extension now reads `.dta`
+directly and shows all of it.
+
+### Added
+
+- **VS Code: a Stata-free `.dta` data viewer.** Double-click any `.dta` file
+  and it opens in a read-only grid, registered as the default editor for
+  `*.dta`. The file is parsed in the extension itself, from StataCorp's
+  published format documentation, so it needs neither Stata, Python, nor the
+  MCP server. What the viewer keeps that a CSV round trip drops:
+  - variable labels (under each column name and in the variables panel);
+  - value labels, shown in place of their codes, with a *Value labels* toggle
+    to see the underlying numbers;
+  - display formats — `%td` / `%tc` / `%tm` / `%tq` / `%th` / `%tw` / `%ty`
+    dates including custom detail codes (`%tdCCYY-NN-DD`), `%w.df`, `%w.de`,
+    `%w.dg`, comma variants — printed as Stata prints them;
+  - the 27 missing values as `.`, `.a` … `.z`, including value labels attached
+    to extended missing values;
+  - dataset and variable notes, the dataset label, sort order and timestamp;
+  - `strL` variables and non-ASCII text.
+
+  Observations are fixed-width records, so the viewer reads only the rows on
+  screen; a 3-million-row file opens and jumps to its last row as fast as a
+  5-row one. A variables panel filters by name or label and jumps to a column;
+  *Go to row*, arrow / Page / Home / End keys and `Cmd/Ctrl+C` work as
+  expected; the viewer reloads when Stata re-saves the file.
+
+  Formats covered: 113–115 (Stata 8–12), 117 (Stata 13), 118 / 119
+  (Stata 14+), 120 / 121 (Stata 18 alias variables), in either byte order.
+  The parser is tested against fixtures written by a real Stata, and its
+  formatted cells are asserted equal to Stata's own `list` output.
+- **VS Code: `stataCode.dtaLegacyEncoding` setting** (default `auto`) for
+  `.dta` formats older than 118, which predate UTF-8. `auto` tries UTF-8,
+  then GB18030, then Windows-1252.
+- **`dataset.variables[*].format` and `.value_label`.** Each variable now
+  reports its display format when that differs from the storage type's
+  default, and the name of its value label when one is attached — so a
+  consumer can tell that an integer is a `%td` date or that `foreign` is
+  labelled by `origin` without running `describe`. Both keys are *omitted*
+  (not `null`) when they have nothing to say, so the common variable costs no
+  extra tokens. Additive; `schema_version` stays `1.0`. Reported by both the
+  pystata and the console backend.
+- **VS Code: the Data sidebar shows those two fields** next to each variable
+  (`long %td · Sale date`, `byte · [origin] · Car origin`).
+
+### Changed
+
+- **VS Code: "View data preview" opens the data viewer instead of a text
+  listing.** The first `stataCode.dataPreviewObs` observations in memory are
+  copied through a scratch frame into a temporary `.dta` and shown in the same
+  grid as a file on disk, so in-memory data gets labels, formats and notes
+  too. The user's frame is not touched — not its data, sort order,
+  `c(filename)`, `c(changed)`, nor `r()` — and the temporary file is deleted
+  when the panel closes. One panel per session; previewing again refreshes it.
+  On Stata 15 or older (no frames) and with the console backend the command
+  falls back to the text listing below.
+- **VS Code: `stataCode.dataPreviewObs` now defaults to `1000`** (was `50`)
+  and accepts up to `100000` (was `10000`): the grid scrolls, a text document
+  does not. The text fallback is capped at 200 rows regardless.
+- **VS Code: the fallback text listing is shorter and no longer line-wrapped.**
+  `list` is run with a widened `linesize` (restored afterwards) so wide
+  datasets stop folding into `>` continuation lines; the command echo is
+  stripped from the body; the header collapses to a session line plus a
+  `74 obs x 12 vars - showing all 74` summary and the dataset path; and the
+  variable list is column-aligned instead of tab-separated.
+- **VS Code: utility runs (data preview, `pwd`, `cd`) no longer hijack the
+  Output panel.** They used to force it visible and echo their whole log into
+  it. They now log one status line on success and only surface the panel on
+  failure.
+
+### Fixed
+
+- **VS Code: "View data preview" failed on every dataset with fewer than 100
+  observations.** The preview listed rows with `list in 1/100`, and an `in`
+  range whose upper bound exceeds `_N` is an error in Stata, so the command
+  returned `r(198) observation numbers out of range` for `auto.dta` (74 obs)
+  and most other teaching datasets — while the document header still claimed
+  `showing: 0 of 74 observations`. The preview now selects rows with
+  `if _n <= N`, which lists whatever is there (including nothing at all), and
+  a failed preview reports its `rc` and message instead of a row count.
+
+## 0.12.2 — 2026-08-11
+
+Correctness release. `results.estimation` could report confidence intervals
+and p-values that disagreed with the Stata log printed beside them; if you
+read coefficients out of `results.estimation` rather than the log, upgrade.
+
+### Fixed
+
+- **`results.estimation` reported normal-approximation inference for `t` models
+  whenever `r(table)` had been cleared.** `r(table)` only survives until the
+  next command, so any block that runs something after its estimation —
+  `graph export`, `esttab`, a final `summarize` — fell back to rebuilding the
+  table from `e(b)` / `e(V)`, and that path hardcoded a normal approximation.
+  The result was a payload that quietly disagreed with the log beside it:
+
+  ```text
+  regress price_k mpg weight foreign
+  scatter price_k mpg          // clears r(table)
+
+  log:                mpg   95% CI  [-.1261758,  .169883]   P>|t| 0.769
+  results.estimation: mpg   95% CI  [-0.12362,   0.16732]   p     0.7684
+  ```
+
+  The fallback now follows Stata's own rule — a *t* table on `e(df_r)` degrees
+  of freedom when that scalar is set, *z* otherwise — so the rebuilt table
+  matches the printed one to the last digit. `ci_level` also honours `e(level)`
+  instead of assuming 95, so `regress, level(90)` is no longer mislabelled.
+  Agents told to prefer `structuredContent` over the log were the ones exposed
+  to this; the `r(table)` path was always correct.
+
+  A new `estimation_from_e_b_v` warning records when the rebuild happened. It
+  fires only for an estimation produced by the current run, since `e()` is
+  session-global and would otherwise re-warn on every later call.
+
+- **`include_results: "none"` hollowed out `results.estimation`.** `n_obs`,
+  `df_model`, `df_resid` and `model_stats` are all read from `e()` scalars, so
+  suppressing the `r()` / `e()` *echo* also removed the model-level numbers —
+  even though `include_estimation` is the documented knob for that block, and
+  SCHEMA.md §4 already promised `include_results` "never affects
+  `results.estimation`". The scalars are now always read when estimation is
+  wanted, and only withheld from the wire.
+
+- **A finished background run reported a duration that kept growing.**
+  `Job.elapsed_ms` recomputed `now - submitted` on every poll instead of
+  freezing at completion, so a 1.7 s bootstrap polled ten minutes later
+  reported ten minutes. Callers attributing wall-clock time to Stata were
+  reading their own polling delay. The value is now frozen before the terminal
+  status is published, on both the success and error paths.
+
+### Changed
+
+- **Macro values longer than 256 characters are elided on the wire** with an
+  explicit `… (N more chars elided)` marker. This is aimed at `e(rngstate)`,
+  roughly 2 KB of hex emitted on every `bootstrap` / `permute` / `simulate`
+  run that no consumer can act on. Names are always kept, and the estimation
+  contract is still built from the uncapped values.
+
 ## 0.12.1 — 2026-07-28
 
 ### Fixed

{stata_code-0.12.1 → stata_code-0.13.0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: stata-code
-Version: 0.12.1
+Version: 0.13.0
 Summary: Agent-native Stata bridge — one core, multiple frontends (MCP, Jupyter, VSCode)
 Project-URL: Homepage, https://github.com/brycewang-stanford/stata-code
 Project-URL: Repository, https://github.com/brycewang-stanford/stata-code
@@ -527,7 +527,7 @@ Then open Jupyter Notebook / JupyterLab (or a `.ipynb` in VS Code), pick **Stata
 
 ### As a VS Code Extension
 
-The companion extension is on the Marketplace as [`brycewang-stanford.stata-code-vscode`](https://marketplace.visualstudio.com/items?itemName=brycewang-stanford.stata-code-vscode). It spawns `stata-code-mcp` as a child process and adds syntax highlighting, an Outline view for `**#` sections and `program define` blocks, code-lens "Run cell" and "Run section" actions on `.do` files, a **seven-view sidebar** (sessions / last result / **data variables** / run history / logs / graphs / **outputs**) — including an agent-native equivalent of Stata's **Variables window** and an **Outputs** panel that surfaces the `esttab` tables and `export` files each run writes to disk — status-bar indicators, completions, help lookup, conservative variable rename, and inline diagnostics from the v1.0 typed errors.
+The companion extension is on the Marketplace as [`brycewang-stanford.stata-code-vscode`](https://marketplace.visualstudio.com/items?itemName=brycewang-stanford.stata-code-vscode). It spawns `stata-code-mcp` as a child process and adds syntax highlighting, an Outline view for `**#` sections and `program define` blocks, code-lens "Run cell" and "Run section" actions on `.do` files, a **seven-view sidebar** (sessions / last result / **data variables** / run history / logs / graphs / **outputs**) — including an agent-native equivalent of Stata's **Variables window** and an **Outputs** panel that surfaces the `esttab` tables and `export` files each run writes to disk — status-bar indicators, completions, help lookup, conservative variable rename, and inline diagnostics from the v1.0 typed errors. It also registers a **Stata-free `.dta` data viewer**: double-click any `.dta` file to browse it in a grid with its variable labels, value labels, display formats, notes and missing-value codes intact, reading only the rows on screen so multi-gigabyte files open instantly.
 
 ```bash
 # from the VS Code CLI

{stata_code-0.12.1 → stata_code-0.13.0}/README.en.md

@@ -488,7 +488,7 @@ Then open Jupyter Notebook / JupyterLab (or a `.ipynb` in VS Code), pick **Stata
 
 ### As a VS Code Extension
 
-The companion extension is on the Marketplace as [`brycewang-stanford.stata-code-vscode`](https://marketplace.visualstudio.com/items?itemName=brycewang-stanford.stata-code-vscode). It spawns `stata-code-mcp` as a child process and adds syntax highlighting, an Outline view for `**#` sections and `program define` blocks, code-lens "Run cell" and "Run section" actions on `.do` files, a **seven-view sidebar** (sessions / last result / **data variables** / run history / logs / graphs / **outputs**) — including an agent-native equivalent of Stata's **Variables window** and an **Outputs** panel that surfaces the `esttab` tables and `export` files each run writes to disk — status-bar indicators, completions, help lookup, conservative variable rename, and inline diagnostics from the v1.0 typed errors.
+The companion extension is on the Marketplace as [`brycewang-stanford.stata-code-vscode`](https://marketplace.visualstudio.com/items?itemName=brycewang-stanford.stata-code-vscode). It spawns `stata-code-mcp` as a child process and adds syntax highlighting, an Outline view for `**#` sections and `program define` blocks, code-lens "Run cell" and "Run section" actions on `.do` files, a **seven-view sidebar** (sessions / last result / **data variables** / run history / logs / graphs / **outputs**) — including an agent-native equivalent of Stata's **Variables window** and an **Outputs** panel that surfaces the `esttab` tables and `export` files each run writes to disk — status-bar indicators, completions, help lookup, conservative variable rename, and inline diagnostics from the v1.0 typed errors. It also registers a **Stata-free `.dta` data viewer**: double-click any `.dta` file to browse it in a grid with its variable labels, value labels, display formats, notes and missing-value codes intact, reading only the rows on screen so multi-gigabyte files open instantly.
 
 ```bash
 # from the VS Code CLI

{stata_code-0.12.1 → stata_code-0.13.0}/README.md

@@ -456,7 +456,7 @@ jupyter kernelspec list
 
 ### 作为 VS Code 扩展
 
-配套扩展已发布到 Marketplace：[`brycewang-stanford.stata-code-vscode`](https://marketplace.visualstudio.com/items?itemName=brycewang-stanford.stata-code-vscode)。它会以子进程方式启动 `stata-code-mcp`，并提供语法高亮、`**#` section 和 `program define` 的 Outline、`.do` 文件的 code-lens "Run cell" / "Run section"、**七视图侧边栏**（sessions / last result / **data 变量浏览器** / run history / logs / graphs / **outputs**）——其中包含一个 agent-native 版的 Stata **变量窗口**，以及一个把每次运行写到磁盘的 `esttab` 表格和 `export` 文件呈现出来的 **Outputs** 面板——状态栏指示器、补全、帮助跳转、保守变量重命名，以及来自 v1.0 typed errors 的内联诊断。
+配套扩展已发布到 Marketplace：[`brycewang-stanford.stata-code-vscode`](https://marketplace.visualstudio.com/items?itemName=brycewang-stanford.stata-code-vscode)。它会以子进程方式启动 `stata-code-mcp`，并提供语法高亮、`**#` section 和 `program define` 的 Outline、`.do` 文件的 code-lens "Run cell" / "Run section"、**七视图侧边栏**（sessions / last result / **data 变量浏览器** / run history / logs / graphs / **outputs**）——其中包含一个 agent-native 版的 Stata **变量窗口**，以及一个把每次运行写到磁盘的 `esttab` 表格和 `export` 文件呈现出来的 **Outputs** 面板——状态栏指示器、补全、帮助跳转、保守变量重命名，以及来自 v1.0 typed errors 的内联诊断。扩展还注册了一个**不依赖 Stata 的 `.dta` 数据浏览器**：双击任意 `.dta` 文件即可在表格中浏览，变量标签、值标签、显示格式、notes 和缺失值代码全部保留；只读取屏幕上可见的行，几个 GB 的文件也能瞬间打开。
 
 ```bash
 # 从 VS Code 命令行

{stata_code-0.12.1 → stata_code-0.13.0}/SCHEMA.md

@@ -343,7 +343,7 @@ Stata's `r()` and `e()` return dictionaries, structurally separated. Each follow
 | Sub-field | Type | Notes |
 | --- | --- | --- |
 | `scalars` | `dict<str, number \| null>` | Native floats / ints. Stata's system missing (`.`) → JSON `null`. Extended missings (`.a`–`.z`) → `null` with information loss. |
-| `macros` | `dict<str, string>` | Stata macro values verbatim. |
+| `macros` | `dict<str, string>` | Stata macro values verbatim, except that a value longer than 256 characters is truncated and suffixed with `… (N more chars elided)`. The cap exists for macros like `e(rngstate)`, which is ~2 KB of hex on every `bootstrap` / `permute` / `simulate` run and carries nothing a consumer can act on. The name is always kept, and `results.estimation` is derived from the uncapped values. |
 | `matrices` | `dict<str, Matrix>` | See `Matrix` below. |
 
 **`Matrix` shape:**
@@ -383,9 +383,9 @@ Stata's `r()` and `e()` return dictionaries, structurally separated. Each follow
 | `n_obs` | `int \| null` | Integer form of `e(N)` when available. |
 | `df_model` | `number \| null` | Mirrors `e(df_m)`. |
 | `df_resid` | `number \| null` | Mirrors `e(df_r)`. |
-| `statistic_kind` | `"t" \| "z"` | Which statistic fills each coefficient's `statistic` field. |
-| `source` | `"r_table" \| "e_b_v"` | `r_table` means values were copied from Stata's displayed `r(table)` after verifying its columns and `b` row match `e(b)`; `e_b_v` means point estimates come from `e(b)` and inference, when present, is computed from `e(V)` with a normal approximation. A matrix returned by `ref` is resolved before use, so a deferred `e(V)` still yields standard errors. |
-| `ci_level` | `number` | Confidence level used for `ci_low` / `ci_high`; currently `95.0`. |
+| `statistic_kind` | `"t" \| "z"` | Which statistic fills each coefficient's `statistic` field, and which distribution produced `p_value` / `ci_low` / `ci_high`. On the `e_b_v` path this follows Stata's own rule — `t` on `df_resid` degrees of freedom when `e(df_r)` is set, `z` otherwise — so a rebuilt table agrees with the printed log rather than reporting a normal-approximation interval next to a `P>\|t\|` column. |
+| `source` | `"r_table" \| "e_b_v"` | `r_table` means values were copied from Stata's displayed `r(table)` after verifying its columns and `b` row match `e(b)`; `e_b_v` means point estimates come from `e(b)` and inference, when present, is computed from `e(V)`. A matrix returned by `ref` is resolved before use, so a deferred `e(V)` still yields standard errors. |
+| `ci_level` | `number` | Confidence level used for `ci_low` / `ci_high`. Mirrors `e(level)` when the command stored it, so `regress, level(90)` reports `90.0`; defaults to `95.0` otherwise. |
 | `coefficients` | `array<Coefficient>` | One row per term in `e(b)`, subject to the caller's `include_estimation` / `max_coefficients` budget. |
 | `n_coefficients` | `int` | The model's true term count. Equals `coefficients.length` unless the caller trimmed the table, so `12` rows out of `n_coefficients: 141` is never mistaken for a 12-term model. |
 | `coefficients_truncated` | `bool` | `true` when rows were dropped to satisfy the budget. |
@@ -425,9 +425,19 @@ A summary of the active Stata frame *after* the command ran. Always populated.
 
 ```json
 { "name": "mpg", "type": "int", "label": "Mileage (mpg)" }
+{ "name": "foreign", "type": "byte", "label": "Car origin", "value_label": "origin" }
+{ "name": "saledate", "type": "long", "label": "Sale date", "format": "%td" }
 ```
 
-`type` is Stata's storage type (`byte`, `int`, `long`, `float`, `double`, `str#`, `strL`). `label` is the variable label string, or `""` if none.
+| Field | Type | Notes |
+| --- | --- | --- |
+| `name` | `string` | Variable name. |
+| `type` | `string` | Stata's storage type (`byte`, `int`, `long`, `float`, `double`, `str#`, `strL`). |
+| `label` | `string` | The variable label, or `""` if none. |
+| `format` | `string`, optional | The display format, **only when it differs from the storage type's default** (`%8.0g` for `byte`/`int`, `%12.0g` for `long`, `%9.0g` for `float`, `%10.0g` for `double`, `%{max(9,#)}s` for `str#`, `%9s` for `strL`). Absent means "the default". This is how a consumer learns that an integer is a date (`%td`, `%tc`, `%tm`, …). |
+| `value_label` | `string`, optional | Name of the value label attached to the variable. Absent when none is attached. The mapping itself is not shipped; run `label list <name>`. |
+
+`format` and `value_label` are **omitted, not `null`**, when they have nothing to say: the variable list rides along on every run, so a null on most variables would cost tokens to carry no information. Added in 0.13 (additive; `schema_version` stays `"1.0"`). Consumers must treat a missing key and `null` alike.
 
 When `n_vars` is large (default cap: 200), the producer truncates `variables` to the first 200 entries and emits a warning of kind `dataset_variables_truncated`. Agents wanting all variables should call `describe` directly.
 
@@ -564,11 +574,21 @@ The rc-to-kind table is approximate and lives in code (`stata_code.core.errors`)
 
 | Field | Type | Notes |
 | --- | --- | --- |
-| `kind` | `string` | Open enum. Common values: `convergence`, `singular`, `boundary`, `omitted_collinear`, `non_finite`, `dataset_variables_truncated`, `unknown`. |
+| `kind` | `string` | Open enum. Common values: `convergence`, `singular`, `boundary`, `omitted_collinear`, `non_finite`, `dataset_variables_truncated`, `log_closed`, `output_tracking_skipped`, `estimation_from_e_b_v`, `unknown`. |
 | `message` | `string` | Human-readable, single line. Truncated to 1,024 characters. |
 
 Warnings are de-duplicated by `(kind, message)`.
 
+`estimation_from_e_b_v` reports that this run performed an estimation whose
+`r(table)` was already gone by the time results were read — a later command in
+the same submission cleared it — so `results.estimation` was rebuilt from
+`e(b)` / `e(V)`. The rebuilt numbers still match the printed log (see
+`statistic_kind` in §3.5), so this is provenance rather than a correctness
+alarm; putting the estimation last in the block restores the `r_table` path.
+It is emitted only when the estimation was produced by *this* run: `e()` is
+session-global, so a later `summarize` in the same session keeps reporting the
+inherited table through the same fallback without re-warning.
+
 ### 3.9 `origin`
 
 ```json
@@ -607,7 +627,7 @@ The schema also dictates what callers may *ask for*. Every frontend exposes the
 | `include_graphs` | `"ref" \| "inline" \| "none"` | `"ref"` | `"none"` skips graph capture entirely (cheapest); `"ref"` captures and returns refs; `"inline"` base64-encodes bytes into `inline`. |
 | `graph_format` | `"png" \| "svg" \| "pdf"` | `"png"` | Render format. |
 | `include_dataset_variables` | `bool` | `true` | Set `false` to omit `dataset.variables`. |
-| `include_results` | `"none" \| "scalars" \| "full"` | `"scalars"` | Payload budget for `results.r` / `results.e`. `"scalars"` inlines scalars and macros and emits every matrix as a stub (§3.4); `"full"` inlines matrix values up to the ~10,000-cell cap; `"none"` omits `r()` / `e()` entirely. Never affects `results.estimation`. |
+| `include_results` | `"none" \| "scalars" \| "full"` | `"scalars"` | Payload budget for `results.r` / `results.e`. `"scalars"` inlines scalars and macros and emits every matrix as a stub (§3.4); `"full"` inlines matrix values up to the ~10,000-cell cap; `"none"` omits `r()` / `e()` entirely. Never affects `results.estimation`: the model-level fields (`n_obs`, `df_model`, `df_resid`, `model_stats`, `depvar`) are read from `e()` regardless of this setting and merely withheld from the wire, so `"none"` does not hollow out the estimation contract. Use `include_estimation` to trim that block. |
 | `include_estimation` | `"none" \| "summary" \| "full"` | `"full"` | Payload budget for `results.estimation`. `"summary"` keeps the model-level block and drops per-term rows. |
 | `max_coefficients` | `int \| null` | `null` | Cap on `estimation.coefficients` rows. `n_coefficients` still reports the true count. |
 | `timeout_ms` | `int \| null` | `600000` (10 min) | Hard timeout. `null` disables. On expiry, returns `ok: false`, `error.kind: "timeout"`, `rc: -2`. The budget covers **queueing**: a call waiting on a session whose Stata process is mid-run returns `rc: -5`, `error.kind: "session_busy"` rather than blocking past its deadline. Frontends MAY override the default if their use case demands. |

{stata_code-0.12.1 → stata_code-0.13.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "stata-code"
-version = "0.12.1"
+version = "0.13.0"
 description = "Agent-native Stata bridge — one core, multiple frontends (MCP, Jupyter, VSCode)"
 # The repo default README (README.md) is Chinese; keep the PyPI long description
 # English by pointing at README.en.md.

{stata_code-0.12.1 → stata_code-0.13.0}/schema/run_result.schema.json

@@ -1145,6 +1145,18 @@
     "VariableInfo": {
       "additionalProperties": true,
       "properties": {
+        "format": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null,
+          "title": "Format"
+        },
         "label": {
           "default": "",
           "title": "Label",
@@ -1157,6 +1169,18 @@
         "type": {
           "title": "Type",
           "type": "string"
+        },
+        "value_label": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null,
+          "title": "Value Label"
         }
       },
       "required": [

{stata_code-0.12.1 → stata_code-0.13.0}/stata_code/__init__.py

@@ -260,7 +260,7 @@ def is_available() -> bool:
     return True
 
 
-__version__ = "0.12.1"
+__version__ = "0.13.0"
 
 __all__ = [
     # Primary entry points

{stata_code-0.12.1 → stata_code-0.13.0}/stata_code/core/console.py

@@ -57,6 +57,7 @@ from stata_code.core.schema import (
     StataInfo,
     StataReturns,
     VariableInfo,
+    default_display_format,
 )
 
 
@@ -217,6 +218,7 @@ MARK_SECTION = f"{_M}|SECTION|"
 MARK_MATRIX = f"{_M}|MATRIX|"
 MARK_DS = f"{_M}|DS|"
 MARK_VAR = f"{_M}|VAR|"
+MARK_VARFMT = f"{_M}|VARFMT|"
 MARK_BEGIN = f"{_M}|BEGIN"
 MARK_END = f"{_M}|END"
 
@@ -274,6 +276,9 @@ def build_wrapper_do(code: str, *, working_dir: str | None = None) -> str:
         # Stata: display "<MARK>`__v'|`: type `__v''|`: variable label `__v''"
         # Built by concatenation to avoid f-string quote/backtick collisions.
         "        display \"" + MARK_VAR + "`__v'|`: type `__v''|`: variable label `__v''\"",
+        # A second line rather than two more fields on the first: a variable
+        # label may itself contain "|", so it has to stay the last field.
+        "        display \"" + MARK_VARFMT + "`__v'|`: format `__v''|`: value label `__v''\"",
         "    }",
         "}",
         f'display "{MARK_END}"',
@@ -298,6 +303,7 @@ _SCALAR_RE = re.compile(r"^\s*[re]\(([A-Za-z_][A-Za-z0-9_]*)\)\s*=\s*(.+?)\s*$")
 _MACRO_RE = re.compile(r'^\s*[re]\(([A-Za-z_][A-Za-z0-9_]*)\)\s*:\s*"?(.*?)"?\s*$')
 _DS_RE = re.compile(r"^\s*" + re.escape(MARK_DS) + r"(\w+)\|(.*)$")
 _VAR_RE = re.compile(r"^\s*" + re.escape(MARK_VAR) + r"(.+?)\|(.*?)\|(.*)$")
+_VARFMT_RE = re.compile(r"^\s*" + re.escape(MARK_VARFMT) + r"(.+?)\|(.*?)\|(.*)$")
 # `matrix list` dimension header:  e(b)[1,2]  or  symmetric e(V)[2,2]
 _MATRIX_HEADER_RE = re.compile(r"^\s*(symmetric\s+)?[A-Za-z_][A-Za-z0-9_]*\([^)]*\)\[\d+,\d+\]")
 
@@ -536,6 +542,16 @@ def _parse_dataset(block: str) -> DatasetInfo:
                     label=v.group(3).strip(),
                 )
             )
+            continue
+        x = _VARFMT_RE.search(raw)
+        if x and variables and variables[-1].name == x.group(1).strip():
+            # Same contract as the pystata backend: report a format only when
+            # it is not the storage type's default, a value label only when set.
+            var = variables[-1]
+            fmt = x.group(2).strip()
+            if fmt and fmt != default_display_format(var.type):
+                var.format = fmt
+            var.value_label = x.group(3).strip() or None
     return DatasetInfo(
         frame="default",
         n_obs=n_obs,

{stata_code-0.12.1 → stata_code-0.13.0}/stata_code/core/estimation.py

@@ -9,8 +9,18 @@ parse from log prose:
   exactly what Stata printed, so we copy them verbatim.
 * ``e(b)`` / ``e(V)`` — the coefficient vector and its variance–covariance
   matrix. Always present after an estimation command. We use these as a
-  fallback, computing ``se`` from ``diag(V)`` and ``z``/``p``/CI under a normal
-  approximation (clearly flagged via ``source`` / ``statistic_kind``).
+  fallback, computing ``se`` from ``diag(V)`` and the statistic / p-value / CI
+  ourselves (flagged via ``source="e_b_v"``).
+
+The fallback matters more than "fallback" suggests: ``r(table)`` is wiped by
+the *next* command, so any block that ends in ``graph export`` / ``esttab`` /
+``summarize`` after its estimation lands here. It therefore has to reproduce
+what Stata displayed, not merely something defensible. Stata's own rule (see
+``_coef_table``) is distributional: when ``e(df_r)`` is set the table is a
+*t* table on that many residual degrees of freedom, and otherwise a *z* table.
+We follow exactly that rule, so the fallback CI agrees with the printed log
+digit for digit instead of quietly reporting a normal-approximation interval
+alongside a log that says ``P>|t|``.
 
 The result is :class:`EstimationResult`, attached to ``RunResult.results.
 estimation`` so every frontend (MCP, kernel, VS Code) gets the same typed
@@ -22,6 +32,7 @@ from __future__ import annotations
 
 import math
 from collections.abc import Callable
+from functools import lru_cache
 
 from stata_code.core.schema import (
     Coefficient,
@@ -162,6 +173,111 @@ def _two_sided_normal_p(z: float) -> float:
     return 2.0 * (1.0 - _normal_cdf(abs(z)))
 
 
+def _betacf(a: float, b: float, x: float) -> float:
+    """Continued fraction for the incomplete beta function (Lentz's method)."""
+    tiny = 1e-30
+    qab, qap, qam = a + b, a + 1.0, a - 1.0
+    c = 1.0
+    d = 1.0 - qab * x / qap
+    if abs(d) < tiny:
+        d = tiny
+    d = 1.0 / d
+    h = d
+    for m in range(1, 301):
+        m2 = 2 * m
+        aa = m * (b - m) * x / ((qam + m2) * (a + m2))
+        d = 1.0 + aa * d
+        if abs(d) < tiny:
+            d = tiny
+        c = 1.0 + aa / c
+        if abs(c) < tiny:
+            c = tiny
+        d = 1.0 / d
+        h *= d * c
+        aa = -(a + m) * (qab + m) * x / ((a + m2) * (qap + m2))
+        d = 1.0 + aa * d
+        if abs(d) < tiny:
+            d = tiny
+        c = 1.0 + aa / c
+        if abs(c) < tiny:
+            c = tiny
+        d = 1.0 / d
+        delta = d * c
+        h *= delta
+        if abs(delta - 1.0) < 1e-15:
+            break
+    return h
+
+
+def _betainc_reg(a: float, b: float, x: float) -> float:
+    """Regularized incomplete beta ``I_x(a, b)``, stdlib only."""
+    if x <= 0.0:
+        return 0.0
+    if x >= 1.0:
+        return 1.0
+    ln_front = (
+        math.lgamma(a + b) - math.lgamma(a) - math.lgamma(b) + a * math.log(x) + b * math.log1p(-x)
+    )
+    front = math.exp(ln_front)
+    if x < (a + 1.0) / (a + b + 2.0):
+        return front * _betacf(a, b, x) / a
+    return 1.0 - front * _betacf(b, a, 1.0 - x) / b
+
+
+def _two_sided_t_p(t: float, df: float) -> float:
+    """Two-sided p-value for a t statistic on ``df`` degrees of freedom.
+
+    ``P(|T| > |t|) = I_{df/(df+t²)}(df/2, 1/2)`` — exact, no table lookup.
+    """
+    if df <= 0.0:
+        return _two_sided_normal_p(t)
+    if math.isinf(t):
+        return 0.0
+    return _betainc_reg(df / 2.0, 0.5, df / (df + t * t))
+
+
+@lru_cache(maxsize=256)
+def _t_crit(df: float, level: float) -> float:
+    """Two-sided critical value of the t distribution for a CI ``level`` in %.
+
+    Found by bisection on :func:`_two_sided_t_p`, which is strictly decreasing
+    in ``|t|`` — robust for any ``df`` without shipping a quantile table.
+    """
+    alpha = 1.0 - level / 100.0
+    if df <= 0.0 or not (0.0 < alpha < 1.0):
+        return _z_crit(level)
+    lo, hi = 0.0, 1.0
+    while _two_sided_t_p(hi, df) > alpha and hi < 1e12:
+        hi *= 2.0
+    for _ in range(200):
+        mid = 0.5 * (lo + hi)
+        if _two_sided_t_p(mid, df) > alpha:
+            lo = mid
+        else:
+            hi = mid
+        if hi - lo < 1e-13 * max(1.0, hi):
+            break
+    return 0.5 * (lo + hi)
+
+
+@lru_cache(maxsize=64)
+def _z_crit(level: float) -> float:
+    """Two-sided standard-normal critical value for a CI ``level`` in %."""
+    alpha = 1.0 - level / 100.0
+    if not (0.0 < alpha < 1.0):
+        return _Z_CRIT_95
+    lo, hi = 0.0, 40.0
+    for _ in range(200):
+        mid = 0.5 * (lo + hi)
+        if _two_sided_normal_p(mid) > alpha:
+            lo = mid
+        else:
+            hi = mid
+        if hi - lo < 1e-14 * max(1.0, hi):
+            break
+    return 0.5 * (lo + hi)
+
+
 def _cell(row: list[float | None] | None, j: int) -> float | None:
     if row is None or j >= len(row):
         return None
@@ -284,13 +400,16 @@ def _from_b_v(
     v: Matrix | None,
     resolve: MatrixResolver | None = None,
     b_values: list[list[float | None]] | None = None,
-) -> list[Coefficient]:
+    df_resid: float | None = None,
+    level: float = 95.0,
+) -> tuple[list[Coefficient], str]:
     """Compute coefficient rows from e(b) (and e(V) when available).
 
     ``se``/``statistic``/``p_value``/CI are filled only when e(V) is
     obtainable — inline or through its ref; otherwise just the point estimates
-    are returned. Inference uses the normal approximation — callers flag this
-    via ``source="e_b_v"``.
+    are returned. Returns ``(coefficients, statistic_kind)``: a *t* table on
+    ``df_resid`` degrees of freedom when Stata set ``e(df_r)``, matching what
+    the command printed, and a *z* table otherwise.
     """
     if b_values is None:
         b_values = _values(b, resolve)
@@ -305,6 +424,10 @@ def _from_b_v(
     else:
         v_diag = [None] * len(terms)
 
+    df = float(df_resid) if df_resid is not None and df_resid > 0.0 else None
+    stat_kind = "t" if df is not None else "z"
+    crit = _t_crit(df, level) if df is not None else _z_crit(level)
+
     coeffs: list[Coefficient] = []
     for j, term in enumerate(terms):
         b_val = _cell(b_row, j)
@@ -316,9 +439,9 @@ def _from_b_v(
         ci_high: float | None = None
         if b_val is not None and se is not None and se > 0.0:
             stat = b_val / se
-            p_val = _two_sided_normal_p(stat)
-            ci_low = b_val - _Z_CRIT_95 * se
-            ci_high = b_val + _Z_CRIT_95 * se
+            p_val = _two_sided_t_p(stat, df) if df is not None else _two_sided_normal_p(stat)
+            ci_low = b_val - crit * se
+            ci_high = b_val + crit * se
         coeffs.append(
             Coefficient(
                 term=term,
@@ -330,7 +453,7 @@ def _from_b_v(
                 ci_high=ci_high,
             )
         )
-    return coeffs
+    return coeffs, stat_kind
 
 
 def _model_stats(scalars: dict[str, float | None]) -> dict[str, float | None]:
@@ -406,14 +529,30 @@ def build_estimation_from_returns(
     statistic_kind: str
     source: str
 
+    df_resid = e.scalars.get("df_r")
+    # e(level) is the CI level the command actually displayed; absent means the
+    # Stata default of 95. Both the critical value and the reported ci_level
+    # follow it, so `regress, level(90)` does not come back labelled 95.
+    level_raw = e.scalars.get("level")
+    ci_level = float(level_raw) if level_raw is not None else 95.0
+
     table = r.matrices.get("table")
     parsed = _from_r_table(table, resolve_matrix) if table is not None else None
     if parsed is not None and table is not None and _r_table_matches_b(table, b, resolve_matrix):
         coeffs, statistic_kind = parsed
         source = "r_table"
     else:
-        coeffs = _from_b_v(b, v, resolve_matrix, b_values)
-        statistic_kind = "z"
+        # r(table) is gone — cleared by whatever command ran after the
+        # estimation. Rebuild the table from e(b)/e(V) on the same
+        # distribution Stata used, so the numbers still match the log.
+        coeffs, statistic_kind = _from_b_v(
+            b,
+            v,
+            resolve_matrix,
+            b_values,
+            df_resid=float(df_resid) if df_resid is not None else None,
+            level=ci_level,
+        )
         source = "e_b_v"
 
     n_obs: int | None = None
@@ -430,9 +569,10 @@ def build_estimation_from_returns(
         depvar=depvar,
         n_obs=n_obs,
         df_model=e.scalars.get("df_m"),
-        df_resid=e.scalars.get("df_r"),
+        df_resid=df_resid,
         statistic_kind=statistic_kind,  # type: ignore[arg-type]
         source=source,  # type: ignore[arg-type]
+        ci_level=ci_level,
         coefficients=coeffs,
         model_stats=_model_stats(e.scalars),
         diagnostics=_command_diagnostics(command, e.scalars),