sourcecode 4.5.1__tar.gz → 4.5.3__tar.gz

{sourcecode-4.5.1 → sourcecode-4.5.3}/CHANGELOG.md

@@ -2,6 +2,151 @@
 
 ## [Unreleased]
 
+## [4.5.3] — 2026-08-04
+
+Patch: the fixable half of field evaluation #10 (7,5/10, CI reliability **4/10**), in one
+battery. Five defects, one commit each, every one reproduced on a fixture before it was
+touched. No new capability, no published field removed, no exit code moved.
+
+Three of the evaluation's thirteen complaints are **not** fixed here because they did not
+survive verification against HEAD — `spring-audit --compact` has shipped since 3.7.0 (the
+second time that closed row has been re-derived as missing), the mojibake diagnosis is
+wrong (`_force_utf8_streams` already reconfigures both streams; this is the **fourth**
+field report of console mojibake and none has reproduced under measurement — the one
+variable they share is the platform with no runner, C3-39), and the "dead surface"
+capability ships as `endpoints --client-usage`. Full record and the F-A…F-E capability
+queue in `.planning/FIELD-EVAL-10-2026-08-04.md`.
+
+### Fixed
+
+- **`verify --init` derived a contract asserting an invariant the same build reported as
+  violated** (C1-24, **critical**). It proposed `no-sec-004` — *"SEC-004 fires nowhere in
+  this repository"* — on a tree where `spring-audit` reported SEC-004 `high`. Committed
+  without `--dry-run`, that gate is red on the commit it was derived from, on the
+  `core`-tier command whose promise is *"safe to gate CI on"*. Cause: the derivation
+  computed "fires nowhere" from the CIR pattern audit alone, and SEC-004…SEC-007 are
+  produced by `security_config_scan`, the second authority `spring-audit` also runs, which
+  returns nothing without a `root`. The root is now threaded through, so the derivation,
+  `verify`, `verify-edit` and the self-check all evaluate the same audit. **The safety net
+  had the same hole** — `execute_candidates` runs every candidate through the real engine
+  before publishing it, and it too evaluated without a root, so it approved the rule it
+  exists to stop. Same row, second half: SEC-005/006/007 appeared in **neither** the
+  proposed nor the rejected list, because the catalogue was built from the pattern
+  registries, which stop at SEC-004; it now comes from `rule_catalog.RULES`, the authority
+  `--help` and the README already render from.
+- **The security configuration scan could not tell code from a comment** (E-2). `SEC-004`
+  and `SEC-007` fired identically on a live statement and on a commented-out block —
+  **31 % of the `high` findings** in the field, and the count understates it, because the
+  correlation engine wrote a persuasive paragraph about shared credentials over a block
+  that was switched off. Confident prose about dead code is worse than a terse warning: it
+  invites trust. This is E-1's class in a module written after E-1 closed, so the stripper
+  is now one authority (`source_text`) wired at the **three points a file is read** rather
+  than inside each rule — a rule added later is covered by construction. Comments are
+  **blanked, not deleted**, so offsets survive and every observation still points at the
+  line a reader can open. String literals are deliberately kept: these rules carry their
+  evidence *inside* the literal. A/B with the cache cold over BroadleafCommerce,
+  openmrs-core, keycloak and spring-petclinic: **byte-identical security findings** — the
+  fix removes false positives and loses no true one.
+- **Two path conventions inside one document** (C2-24). `servlet_surface.mappings[].source_file`
+  emitted the platform separator while every other payload emits POSIX. The reported
+  emitter was one of **four**: checking for the recurrence found `client_calls` (the
+  `--client-usage` join, whose entire product is a join on path) and the file-too-large
+  branch of `ast_extractor`, which spelled the path differently from the two lines directly
+  above it; a static guard over every module then found a fourth in `token_estimate`. All
+  four fixed, and the guard now fails any `str(x.relative_to(y))` without normalisation.
+- **An untracked source file was not counted as a change** (C3-40). `cache freshness`
+  reported `has_uncommitted_changes: False` with untracked files present, contradicting the
+  root scan's `--changed-only` help in the same CLI. The measurement underneath was worse
+  than the contradiction: an untracked `.java` file **is** in the IR, so the tree reported
+  `fresh: true` for a snapshot that never contained that file — a false *fresh*. Bound to
+  `baseline_autocapture.worktree_dirty`, which already had to answer this exactly right for
+  the architectural history; gitignored build output and cache directories are still not
+  modifications.
+- **`data-exposure` measured nothing and offered no way to measure** (C4-14). It answered
+  `answered: false` and stopped, on the payload aimed at a regulated buyer — ninth
+  appearance of the C4 class. It now carries the registry remedy plus a `declare` block
+  (file, key, copy-pasteable example), all read from the module that parses them. The
+  example is **executed** in the battery — written to disk, parsed by the real loader, and
+  re-run to confirm the answer changes — because an example that has drifted from the schema
+  teaches a buyer the wrong shape.
+
+### Changed
+
+- Cache-hit provenance moved from inside the `git_context` payload block to `_meta`
+  (`stale_fields_refreshed`). Surfaced by C3-40: written into the payload it changed that
+  block's *shape* depending on whether the answer came from cache, so one request answered
+  two ways compared unequal on a key describing the run rather than the repository. The
+  previous key was undocumented, unschema'd, underscore-prefixed and reachable only on one
+  cache path — treated as internal provenance rather than a published field, which is why
+  this is a patch and not a major.
+
+### Still open from this evaluation, on purpose
+
+- **C1-25** — reachability is measured at type scope and attributed at member scope
+  (~1 700× over-attribution measured in one direction; global `@Configuration` defects
+  degraded to `internal` in the other). Feature-sized, and it **gates** the `exposure` join
+  (F-C): that table built on this model would put the over-attribution behind a
+  CISO-facing headline.
+- **P-8** — a pricing decision, and the third independent arrival at the per-repository band.
+
+## [4.5.2] — 2026-08-04
+
+Patch: two affordances that shipped command by command, and therefore drifted, become
+properties of the CLI. No analysis changes, no published field is added or removed, no
+exit code moves, and every payload is byte-identical to 4.5.1.
+
+### Fixed
+
+- **60 of 84 commands did not accept `--copy` / `-c`.** Among them `verify`, `risk`,
+  `posture`, `verify-edit`, `schema`, `archetype`, `cold-start`, `enrich`,
+  `data-exposure`, `migrate-recipe`, `baseline capture`, every `cache`/`mcp` status
+  command, `version`, `auth status`, `telemetry status`, and all 29 `retrieve` intents.
+  The flag had been added one command at a time since it was introduced, so what a
+  reader could copy depended on which command they happened to be using. It is now a
+  property of the CLI: every command that answers a question publishes it, declared
+  through one factory (`cli._copy_option()`) so all of them bind `-c` and describe it
+  with the same sentence. The text answers gained it too — `ask version -c`,
+  `ask cache status -c`, `ask mcp status -c` — via `_emit_text_output`/`_TextAnswer`,
+  which collect the lines as they are printed so the clipboard holds exactly what the
+  reader saw.
+- **Four commands published `--copy` and then dropped it on their empty-population
+  path.** `spring-audit`, `impact-chain`, `pr-impact` and the root `--changed-only`
+  scan passed a hard-coded `False` into the output seam when there were no Java files
+  or no changed files: the flag parsed, exited 0, and copied nothing. A shorter answer
+  is still the answer. Found by the new battery, not by hand.
+- **65 of 84 commands ran silently, including the slowest ones.** Measured on
+  BroadleafCommerce: `ask risk` is 15.2 s, `verify-edit` 11.8 s, `posture` 5.9 s,
+  `migrate-recipe` 5.0 s, `archetype` 5.1 s, `retrieve <intent>` 4.6 s — none of them
+  wrote a byte while they worked, which is indistinguishable from a hang. They now draw
+  the existing `Progress` spinner on stderr, naming the phase. `retrieve`'s 29 intents
+  take it once in the adapter they share rather than in 29 copies. Every spinner stops
+  in a `finally`, so a failing run never prints its error underneath a live frame.
+  Nothing changes for a non-interactive consumer: `Progress` already disables itself
+  off-TTY, in CI, under `NO_COLOR` and on `TERM=dumb`, and only ever writes to stderr.
+
+### Added
+
+- **`tests/test_output_affordances.py` — the mechanism, not a promise in a document.**
+  426 checks whose population comes from the CLI registry itself: a command must
+  publish `--copy` or appear in `cli.COMMANDS_WITHOUT_COPY`, must start a spinner or
+  appear in `cli.COMMANDS_WITHOUT_PROGRESS`, must bind `-c` to `--copy`, and must
+  describe the flag with the shared help string. It also rejects a declared `--copy`
+  wired to a constant `False`, and rejects an exemption naming a command that no longer
+  exists — a stale entry would silently excuse the next command to take that name. The
+  same C4 remedy this project applies everywhere: the catalogue is generated, never
+  maintained by hand.
+- Both affordances are now written down where the next change is made:
+  `CLAUDE.md` (new), `CONTRIBUTING.md` § *Adding a new command*, and a *Two things
+  every command does* section in `docs/USER_GUIDE.md` and `docs/MANUAL-USUARIO.md`.
+
+### Known limitation
+
+- `ask chunk-file -c` still means `--chunk`, not `--copy`: it claimed the letter before
+  `--copy` existed, and removing it would break `ask chunk-file Big.java -c 3`, which
+  its `supported` tier does not permit inside a major. `--copy` works in full there. The
+  exception is recorded in `cli.COMMANDS_WITHOUT_COPY_SHORTHAND` rather than tolerated
+  silently, and the shorthand moves to `--copy` at the next major.
+
 ## [4.5.1] — 2026-08-03
 
 Patch: documentation only. No engine change, no published field, no exit code.

sourcecode-4.5.3/CLAUDE.md

@@ -0,0 +1,74 @@
+# Working on this repository
+
+## Non-negotiable: every new command ships both output affordances
+
+When you add a CLI command — or a new answer, subcommand or output path on an
+existing one — it **must** publish both of these. They are not polish, they are
+part of what a command is here. Both are enforced by
+`tests/test_output_affordances.py`, which derives its population from the CLI
+registry, so a new command fails the suite until each has been decided.
+
+### 1. `--copy` / `-c` — the answer is copyable
+
+Declare it with the shared factory, never a fresh `typer.Option`:
+
+```python
+copy: bool = _copy_option(),
+```
+
+Then **wire it through**. Declaring the flag and passing a hard-coded `False`
+into `_emit_command_output` is the exact defect the battery was written for:
+
+```python
+_emit_command_output(payload, output_path, copy, success_msg=...)   # payload commands
+_emit_text_output(text, copy)                                       # single-block text
+_TextAnswer()  # → .say(line) per line, .done(copy) at the end      # line-by-line answers
+```
+
+Wire **every** return path, including the early ones (no Java files, no changed
+files, nothing to report). A shorter answer is still the answer.
+
+### 2. `Progress()` — the wait is visible
+
+If the command analyses a repository, start a spinner and stop it in `finally`,
+so a raised `typer.Exit` never prints its error under a live spinner frame:
+
+```python
+_prog = Progress()
+_prog.start("building IR")
+try:
+    result = analyse(target)
+finally:
+    _prog.stop()          # .finish() only when a "✓ done (3.2s)" line is wanted
+```
+
+`Progress` writes to stderr only and disables itself off-TTY, in CI, under
+`NO_COLOR` and on `TERM=dumb` — piped output, CI logs and MCP sessions stay
+byte-identical. When commands delegate (`retrieve`'s 29 intents →
+`_run_retrieval_intent`; `onboard`/`review-pr`/`fix-bug` → `prepare-context`),
+put the spinner in the shared helper, not in 29 copies.
+
+### Opting out is a decision, not an omission
+
+Add the command to `cli.COMMANDS_WITHOUT_COPY` or
+`cli.COMMANDS_WITHOUT_PROGRESS` **with an inline comment stating why**:
+
+- no `--copy` when stdout is a confirmation that an effect happened rather than
+  an answer worth keeping (`cache clear`, `mcp init`, `telemetry enable`)
+- no spinner when the command reads local metadata, one named file, or a stored
+  history and returns in milliseconds (`version`, `cache status`, `trend`)
+
+Anything that touches a repository gets both, however fast it is on a small one:
+the same command runs on a 3 300-file monolith.
+
+## Related conventions this repo already enforces
+
+- **One authority per fact.** A hand-written catalogue beside a generated one
+  drifts. Populations come from the CLI registry (`COMMAND_TIERS`,
+  `cli.HELP_PANELS`, the doc batteries), never from a list maintained by hand.
+- **Never a confident falsehood.** An axis that could not be measured is
+  `unknown`, never `0` and never "not reachable".
+- **Vendor-agnostic inference.** Logic never branches on a proprietary name;
+  names are evidence, never the rule.
+- Run the suite with `-o addopts=""` to see the summary line; the entry point for
+  local runs is `run_cli.py`, not `python -m sourcecode.cli`.

{sourcecode-4.5.1 → sourcecode-4.5.3}/CONTRIBUTING.md

@@ -84,6 +84,62 @@ tests/
 3. Register it in `detectors/__init__.py` → `build_default_detectors()`
 4. Add tests in `tests/test_detector_<language>.py`
 
+### Adding a new command (or a new answer to an existing one)
+
+**Every command that answers a question ships two affordances. Both are
+mandatory, both are enforced by `tests/test_output_affordances.py`, and the
+battery derives its population from the CLI registry — so a new command fails
+these tests until a decision has been taken about each.**
+
+**1. The answer is copyable.** Declare the flag with the shared factory, never a
+fresh `typer.Option`, so all ~75 commands describe it identically and all bind
+the same shorthand:
+
+```python
+@app.command("my-command")
+def my_command_cmd(
+    path: Path = typer.Argument(Path(".")),
+    output_path: Optional[Path] = typer.Option(None, "--output", "-o"),
+    copy: bool = _copy_option(),          # ← --copy / -c
+) -> None:
+    ...
+    _emit_command_output(payload, output_path, copy, success_msg=...)
+```
+
+Then *wire it*. A declared `--copy` that reaches `_emit_command_output` as a
+hard-coded `False` parses fine and does nothing; the battery fails on that
+specific shape, because it is how four commands shipped a flag that silently
+dropped the answer on their empty-repository path. Commands that answer in lines
+rather than in a payload use `_emit_text_output(...)` or `_TextAnswer` instead.
+
+**2. The wait is visible.** If the command analyses a repository, start a spinner
+and stop it in a `finally`, so an error is never printed underneath a live
+spinner frame:
+
+```python
+_prog = Progress()
+_prog.start("building IR")
+try:
+    result = analyse(target)
+finally:
+    _prog.stop()          # .finish() when a "✓ done (3.2s)" line is wanted
+```
+
+`Progress` costs nothing when nobody is watching: it disables itself off-TTY, in
+CI, under `NO_COLOR`, and on `TERM=dumb`, and it only ever writes to stderr —
+piped output and MCP sessions stay byte-identical. Put the spinner in the shared
+helper when several commands delegate to one (`retrieve`'s 29 intents share
+`_run_retrieval_intent`; `onboard`/`review-pr`/`fix-bug` delegate to
+`prepare-context`), and the battery will see it through one level of call.
+
+**Opting out** is allowed and must be explicit: add the command to
+`cli.COMMANDS_WITHOUT_COPY` or `cli.COMMANDS_WITHOUT_PROGRESS` **with an inline
+comment giving the reason**. A command whose stdout is a confirmation of an
+effect (`cache clear`) does not need `--copy`; one that reads local metadata and
+returns in milliseconds (`version`) does not need a spinner. Every other command
+does. The exemption sets are checked against the registry too, so a renamed or
+deleted command cannot leave a stale entry that silently excuses its successor.
+
 ### Modifying the output schema
 
 The schema is defined in `schema.py` as Python dataclasses. All fields must have defaults so that existing consumers are not broken. Add new fields at the end of the relevant dataclass with `Optional[T] = None` or `list[T] = field(default_factory=list)`.

{sourcecode-4.5.1 → sourcecode-4.5.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sourcecode
-Version: 4.5.1
+Version: 4.5.3
 Summary: Persistent structural context and ultra-fast repeated analysis for AI coding agents
 License-File: LICENSE
 Keywords: agents,ai,codebase,context,developer-tools,llm
@@ -42,7 +42,7 @@ Description-Content-Type: text/markdown
 
 **Context · Impact · Migration · Architecture · Review — everything from one structural model.**
 
-![Version](https://img.shields.io/badge/version-4.5.1-blue)
+![Version](https://img.shields.io/badge/version-4.5.3-blue)
 ![Python](https://img.shields.io/badge/python-3.9%2B-green)
 
 > **ASK Engine** is the product. The CLI command is **`ask`**. The legacy **`sourcecode`**
@@ -97,7 +97,7 @@ brew tap haroundominique/sourcecode && brew install sourcecode
 # pip / pipx
 pipx install sourcecode        # or: pip install sourcecode
 
-ask version                    # ask 4.5.1 — and, on a build that has aged,
+ask version                    # ask 4.5.3 — and, on a build that has aged,
                                # how many releases have probably shipped since
 ```
 
@@ -309,7 +309,7 @@ from it.
 | `risk` | experimental | what each defect actually costs, once reach, access, write effect and the shape of the input path are in it | `defect_severity × reachability × auth_verdict × write_effect × query_construction × input_constraints`; every factor names its authority |
 | `enrich` | experimental | another scanner's SARIF findings, ranked by what this repository does with them | `--sarif <log>`; the same composition as `risk`, with the scanner as the severity authority |
 | `migrate-recipe` | experimental | the migration report as the OpenRewrite recipe that applies it | only recipes a finding named; the manual remainder published beside them; writes nothing without `--write`, and never a runnable command for an empty recipe list |
-| `data-exposure` | experimental | which routes can carry the data you labelled, and who reaches them | labels declared in `sourcecode.config.json` — never inferred from a name; `signature` and `call_reach` evidence published apart, field-level flow out of scope (NC-008) |
+| `data-exposure` | experimental | which routes can carry the data you labelled, and who reaches them | labels declared in `sourcecode.config.json` — never inferred from a name; `signature` and `call_reach` evidence published apart, field-level flow out of scope (NC-008). With nothing declared, `ask data-exposure /path/to/repo` answers `answered: false` and hands back the file, the key and an example to declare — it never reports zero exposed routes |
 | `endpoints` | core | every REST endpoint, effective path, security policy, confidence | Spring MVC + JAX-RS (~65 % recall on JAX-RS sub-resource locators). `--compact` answers the exposure census without the rows; `--servlets` lists the servlet-mounted surface as its own population; `--client-usage` says which routes the TS/JS client in this repository actually calls |
 | `spring-audit` | core | transactional anomalies + security surface + validation gaps | `--ci`, `-f github-comment` |
 | `migrate-check` | core | Boot 2→3 readiness: located blockers, per-dimension score, effort | `--blast-radius` orders the re-test plan |

{sourcecode-4.5.1 → sourcecode-4.5.3}/README.md

@@ -4,7 +4,7 @@
 
 **Context · Impact · Migration · Architecture · Review — everything from one structural model.**
 
-![Version](https://img.shields.io/badge/version-4.5.1-blue)
+![Version](https://img.shields.io/badge/version-4.5.3-blue)
 ![Python](https://img.shields.io/badge/python-3.9%2B-green)
 
 > **ASK Engine** is the product. The CLI command is **`ask`**. The legacy **`sourcecode`**
@@ -59,7 +59,7 @@ brew tap haroundominique/sourcecode && brew install sourcecode
 # pip / pipx
 pipx install sourcecode        # or: pip install sourcecode
 
-ask version                    # ask 4.5.1 — and, on a build that has aged,
+ask version                    # ask 4.5.3 — and, on a build that has aged,
                                # how many releases have probably shipped since
 ```
 
@@ -271,7 +271,7 @@ from it.
 | `risk` | experimental | what each defect actually costs, once reach, access, write effect and the shape of the input path are in it | `defect_severity × reachability × auth_verdict × write_effect × query_construction × input_constraints`; every factor names its authority |
 | `enrich` | experimental | another scanner's SARIF findings, ranked by what this repository does with them | `--sarif <log>`; the same composition as `risk`, with the scanner as the severity authority |
 | `migrate-recipe` | experimental | the migration report as the OpenRewrite recipe that applies it | only recipes a finding named; the manual remainder published beside them; writes nothing without `--write`, and never a runnable command for an empty recipe list |
-| `data-exposure` | experimental | which routes can carry the data you labelled, and who reaches them | labels declared in `sourcecode.config.json` — never inferred from a name; `signature` and `call_reach` evidence published apart, field-level flow out of scope (NC-008) |
+| `data-exposure` | experimental | which routes can carry the data you labelled, and who reaches them | labels declared in `sourcecode.config.json` — never inferred from a name; `signature` and `call_reach` evidence published apart, field-level flow out of scope (NC-008). With nothing declared, `ask data-exposure /path/to/repo` answers `answered: false` and hands back the file, the key and an example to declare — it never reports zero exposed routes |
 | `endpoints` | core | every REST endpoint, effective path, security policy, confidence | Spring MVC + JAX-RS (~65 % recall on JAX-RS sub-resource locators). `--compact` answers the exposure census without the rows; `--servlets` lists the servlet-mounted surface as its own population; `--client-usage` says which routes the TS/JS client in this repository actually calls |
 | `spring-audit` | core | transactional anomalies + security surface + validation gaps | `--ci`, `-f github-comment` |
 | `migrate-check` | core | Boot 2→3 readiness: located blockers, per-dimension score, effort | `--blast-radius` orders the re-test plan |

{sourcecode-4.5.1 → sourcecode-4.5.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "sourcecode"
-version = "4.5.1"
+version = "4.5.3"
 description = "Persistent structural context and ultra-fast repeated analysis for AI coding agents"
 readme = "README.md"
 requires-python = ">=3.9"

{sourcecode-4.5.1 → sourcecode-4.5.3}/src/sourcecode/__init__.py

@@ -4,4 +4,4 @@ ASK Engine is the product. ``ask`` is the canonical CLI command; ``sourcecode``
 the legacy compatibility alias and the Python/PyPI package name. See
 docs/PRODUCT_IDENTITY.md (normative)."""
 
-__version__ = "4.5.1"
+__version__ = "4.5.3"

{sourcecode-4.5.1 → sourcecode-4.5.3}/src/sourcecode/ast_extractor.py

@@ -1252,7 +1252,7 @@ class AstExtractor:
             stat = path.stat()
             if stat.st_size > self.max_file_size:
                 return FileContract(
-                    path=str(path.relative_to(root)) if root else path.name,
+                    path=rel_path,
                     language=language,
                     extraction_method="heuristic",
                     limitations=[f"file_too_large: {stat.st_size} bytes > {self.max_file_size}"],