sourcecode 4.5.2__tar.gz → 4.5.3__tar.gz

{sourcecode-4.5.2 → sourcecode-4.5.3}/CHANGELOG.md

@@ -2,6 +2,93 @@
 
 ## [Unreleased]
 
+## [4.5.3] — 2026-08-04
+
+Patch: the fixable half of field evaluation #10 (7,5/10, CI reliability **4/10**), in one
+battery. Five defects, one commit each, every one reproduced on a fixture before it was
+touched. No new capability, no published field removed, no exit code moved.
+
+Three of the evaluation's thirteen complaints are **not** fixed here because they did not
+survive verification against HEAD — `spring-audit --compact` has shipped since 3.7.0 (the
+second time that closed row has been re-derived as missing), the mojibake diagnosis is
+wrong (`_force_utf8_streams` already reconfigures both streams; this is the **fourth**
+field report of console mojibake and none has reproduced under measurement — the one
+variable they share is the platform with no runner, C3-39), and the "dead surface"
+capability ships as `endpoints --client-usage`. Full record and the F-A…F-E capability
+queue in `.planning/FIELD-EVAL-10-2026-08-04.md`.
+
+### Fixed
+
+- **`verify --init` derived a contract asserting an invariant the same build reported as
+  violated** (C1-24, **critical**). It proposed `no-sec-004` — *"SEC-004 fires nowhere in
+  this repository"* — on a tree where `spring-audit` reported SEC-004 `high`. Committed
+  without `--dry-run`, that gate is red on the commit it was derived from, on the
+  `core`-tier command whose promise is *"safe to gate CI on"*. Cause: the derivation
+  computed "fires nowhere" from the CIR pattern audit alone, and SEC-004…SEC-007 are
+  produced by `security_config_scan`, the second authority `spring-audit` also runs, which
+  returns nothing without a `root`. The root is now threaded through, so the derivation,
+  `verify`, `verify-edit` and the self-check all evaluate the same audit. **The safety net
+  had the same hole** — `execute_candidates` runs every candidate through the real engine
+  before publishing it, and it too evaluated without a root, so it approved the rule it
+  exists to stop. Same row, second half: SEC-005/006/007 appeared in **neither** the
+  proposed nor the rejected list, because the catalogue was built from the pattern
+  registries, which stop at SEC-004; it now comes from `rule_catalog.RULES`, the authority
+  `--help` and the README already render from.
+- **The security configuration scan could not tell code from a comment** (E-2). `SEC-004`
+  and `SEC-007` fired identically on a live statement and on a commented-out block —
+  **31 % of the `high` findings** in the field, and the count understates it, because the
+  correlation engine wrote a persuasive paragraph about shared credentials over a block
+  that was switched off. Confident prose about dead code is worse than a terse warning: it
+  invites trust. This is E-1's class in a module written after E-1 closed, so the stripper
+  is now one authority (`source_text`) wired at the **three points a file is read** rather
+  than inside each rule — a rule added later is covered by construction. Comments are
+  **blanked, not deleted**, so offsets survive and every observation still points at the
+  line a reader can open. String literals are deliberately kept: these rules carry their
+  evidence *inside* the literal. A/B with the cache cold over BroadleafCommerce,
+  openmrs-core, keycloak and spring-petclinic: **byte-identical security findings** — the
+  fix removes false positives and loses no true one.
+- **Two path conventions inside one document** (C2-24). `servlet_surface.mappings[].source_file`
+  emitted the platform separator while every other payload emits POSIX. The reported
+  emitter was one of **four**: checking for the recurrence found `client_calls` (the
+  `--client-usage` join, whose entire product is a join on path) and the file-too-large
+  branch of `ast_extractor`, which spelled the path differently from the two lines directly
+  above it; a static guard over every module then found a fourth in `token_estimate`. All
+  four fixed, and the guard now fails any `str(x.relative_to(y))` without normalisation.
+- **An untracked source file was not counted as a change** (C3-40). `cache freshness`
+  reported `has_uncommitted_changes: False` with untracked files present, contradicting the
+  root scan's `--changed-only` help in the same CLI. The measurement underneath was worse
+  than the contradiction: an untracked `.java` file **is** in the IR, so the tree reported
+  `fresh: true` for a snapshot that never contained that file — a false *fresh*. Bound to
+  `baseline_autocapture.worktree_dirty`, which already had to answer this exactly right for
+  the architectural history; gitignored build output and cache directories are still not
+  modifications.
+- **`data-exposure` measured nothing and offered no way to measure** (C4-14). It answered
+  `answered: false` and stopped, on the payload aimed at a regulated buyer — ninth
+  appearance of the C4 class. It now carries the registry remedy plus a `declare` block
+  (file, key, copy-pasteable example), all read from the module that parses them. The
+  example is **executed** in the battery — written to disk, parsed by the real loader, and
+  re-run to confirm the answer changes — because an example that has drifted from the schema
+  teaches a buyer the wrong shape.
+
+### Changed
+
+- Cache-hit provenance moved from inside the `git_context` payload block to `_meta`
+  (`stale_fields_refreshed`). Surfaced by C3-40: written into the payload it changed that
+  block's *shape* depending on whether the answer came from cache, so one request answered
+  two ways compared unequal on a key describing the run rather than the repository. The
+  previous key was undocumented, unschema'd, underscore-prefixed and reachable only on one
+  cache path — treated as internal provenance rather than a published field, which is why
+  this is a patch and not a major.
+
+### Still open from this evaluation, on purpose
+
+- **C1-25** — reachability is measured at type scope and attributed at member scope
+  (~1 700× over-attribution measured in one direction; global `@Configuration` defects
+  degraded to `internal` in the other). Feature-sized, and it **gates** the `exposure` join
+  (F-C): that table built on this model would put the over-attribution behind a
+  CISO-facing headline.
+- **P-8** — a pricing decision, and the third independent arrival at the per-repository band.
+
 ## [4.5.2] — 2026-08-04
 
 Patch: two affordances that shipped command by command, and therefore drifted, become

{sourcecode-4.5.2 → sourcecode-4.5.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sourcecode
-Version: 4.5.2
+Version: 4.5.3
 Summary: Persistent structural context and ultra-fast repeated analysis for AI coding agents
 License-File: LICENSE
 Keywords: agents,ai,codebase,context,developer-tools,llm
@@ -42,7 +42,7 @@ Description-Content-Type: text/markdown
 
 **Context · Impact · Migration · Architecture · Review — everything from one structural model.**
 
-![Version](https://img.shields.io/badge/version-4.5.2-blue)
+![Version](https://img.shields.io/badge/version-4.5.3-blue)
 ![Python](https://img.shields.io/badge/python-3.9%2B-green)
 
 > **ASK Engine** is the product. The CLI command is **`ask`**. The legacy **`sourcecode`**
@@ -97,7 +97,7 @@ brew tap haroundominique/sourcecode && brew install sourcecode
 # pip / pipx
 pipx install sourcecode        # or: pip install sourcecode
 
-ask version                    # ask 4.5.2 — and, on a build that has aged,
+ask version                    # ask 4.5.3 — and, on a build that has aged,
                                # how many releases have probably shipped since
 ```
 
@@ -309,7 +309,7 @@ from it.
 | `risk` | experimental | what each defect actually costs, once reach, access, write effect and the shape of the input path are in it | `defect_severity × reachability × auth_verdict × write_effect × query_construction × input_constraints`; every factor names its authority |
 | `enrich` | experimental | another scanner's SARIF findings, ranked by what this repository does with them | `--sarif <log>`; the same composition as `risk`, with the scanner as the severity authority |
 | `migrate-recipe` | experimental | the migration report as the OpenRewrite recipe that applies it | only recipes a finding named; the manual remainder published beside them; writes nothing without `--write`, and never a runnable command for an empty recipe list |
-| `data-exposure` | experimental | which routes can carry the data you labelled, and who reaches them | labels declared in `sourcecode.config.json` — never inferred from a name; `signature` and `call_reach` evidence published apart, field-level flow out of scope (NC-008) |
+| `data-exposure` | experimental | which routes can carry the data you labelled, and who reaches them | labels declared in `sourcecode.config.json` — never inferred from a name; `signature` and `call_reach` evidence published apart, field-level flow out of scope (NC-008). With nothing declared, `ask data-exposure /path/to/repo` answers `answered: false` and hands back the file, the key and an example to declare — it never reports zero exposed routes |
 | `endpoints` | core | every REST endpoint, effective path, security policy, confidence | Spring MVC + JAX-RS (~65 % recall on JAX-RS sub-resource locators). `--compact` answers the exposure census without the rows; `--servlets` lists the servlet-mounted surface as its own population; `--client-usage` says which routes the TS/JS client in this repository actually calls |
 | `spring-audit` | core | transactional anomalies + security surface + validation gaps | `--ci`, `-f github-comment` |
 | `migrate-check` | core | Boot 2→3 readiness: located blockers, per-dimension score, effort | `--blast-radius` orders the re-test plan |

{sourcecode-4.5.2 → sourcecode-4.5.3}/README.md

@@ -4,7 +4,7 @@
 
 **Context · Impact · Migration · Architecture · Review — everything from one structural model.**
 
-![Version](https://img.shields.io/badge/version-4.5.2-blue)
+![Version](https://img.shields.io/badge/version-4.5.3-blue)
 ![Python](https://img.shields.io/badge/python-3.9%2B-green)
 
 > **ASK Engine** is the product. The CLI command is **`ask`**. The legacy **`sourcecode`**
@@ -59,7 +59,7 @@ brew tap haroundominique/sourcecode && brew install sourcecode
 # pip / pipx
 pipx install sourcecode        # or: pip install sourcecode
 
-ask version                    # ask 4.5.2 — and, on a build that has aged,
+ask version                    # ask 4.5.3 — and, on a build that has aged,
                                # how many releases have probably shipped since
 ```
 
@@ -271,7 +271,7 @@ from it.
 | `risk` | experimental | what each defect actually costs, once reach, access, write effect and the shape of the input path are in it | `defect_severity × reachability × auth_verdict × write_effect × query_construction × input_constraints`; every factor names its authority |
 | `enrich` | experimental | another scanner's SARIF findings, ranked by what this repository does with them | `--sarif <log>`; the same composition as `risk`, with the scanner as the severity authority |
 | `migrate-recipe` | experimental | the migration report as the OpenRewrite recipe that applies it | only recipes a finding named; the manual remainder published beside them; writes nothing without `--write`, and never a runnable command for an empty recipe list |
-| `data-exposure` | experimental | which routes can carry the data you labelled, and who reaches them | labels declared in `sourcecode.config.json` — never inferred from a name; `signature` and `call_reach` evidence published apart, field-level flow out of scope (NC-008) |
+| `data-exposure` | experimental | which routes can carry the data you labelled, and who reaches them | labels declared in `sourcecode.config.json` — never inferred from a name; `signature` and `call_reach` evidence published apart, field-level flow out of scope (NC-008). With nothing declared, `ask data-exposure /path/to/repo` answers `answered: false` and hands back the file, the key and an example to declare — it never reports zero exposed routes |
 | `endpoints` | core | every REST endpoint, effective path, security policy, confidence | Spring MVC + JAX-RS (~65 % recall on JAX-RS sub-resource locators). `--compact` answers the exposure census without the rows; `--servlets` lists the servlet-mounted surface as its own population; `--client-usage` says which routes the TS/JS client in this repository actually calls |
 | `spring-audit` | core | transactional anomalies + security surface + validation gaps | `--ci`, `-f github-comment` |
 | `migrate-check` | core | Boot 2→3 readiness: located blockers, per-dimension score, effort | `--blast-radius` orders the re-test plan |

{sourcecode-4.5.2 → sourcecode-4.5.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "sourcecode"
-version = "4.5.2"
+version = "4.5.3"
 description = "Persistent structural context and ultra-fast repeated analysis for AI coding agents"
 readme = "README.md"
 requires-python = ">=3.9"

{sourcecode-4.5.2 → sourcecode-4.5.3}/src/sourcecode/__init__.py

@@ -4,4 +4,4 @@ ASK Engine is the product. ``ask`` is the canonical CLI command; ``sourcecode``
 the legacy compatibility alias and the Python/PyPI package name. See
 docs/PRODUCT_IDENTITY.md (normative)."""
 
-__version__ = "4.5.2"
+__version__ = "4.5.3"

{sourcecode-4.5.2 → sourcecode-4.5.3}/src/sourcecode/ast_extractor.py

@@ -1252,7 +1252,7 @@ class AstExtractor:
             stat = path.stat()
             if stat.st_size > self.max_file_size:
                 return FileContract(
-                    path=str(path.relative_to(root)) if root else path.name,
+                    path=rel_path,
                     language=language,
                     extraction_method="heuristic",
                     limitations=[f"file_too_large: {stat.st_size} bytes > {self.max_file_size}"],

{sourcecode-4.5.2 → sourcecode-4.5.3}/src/sourcecode/cli.py

@@ -2533,9 +2533,17 @@ def main(
                                 [l for l in _uc_r.stdout.splitlines() if l.strip()]
                             )
                             _patched["git_context"]["uncommitted_files"] = _uc_count
-                            _patched["git_context"]["_stale_fields_refreshed"] = [
-                                "uncommitted_files"
-                            ]
+                            # Provenance goes in `_meta`, never inside the payload
+                            # block. Written into `git_context` it changed that
+                            # block's *shape* depending on whether the answer came
+                            # from cache, so one request answered two ways compared
+                            # unequal on a key that describes the run rather than
+                            # the repository (surfaced by C3-40, which made an
+                            # untracked file dirty again and put this path back in
+                            # reach).
+                            _meta_gc = _patched.setdefault("_meta", {})
+                            if isinstance(_meta_gc, dict):
+                                _meta_gc["stale_fields_refreshed"] = ["git_context.uncommitted_files"]
                             _cache_hit_content = _json_gc.dumps(
                                 _patched, indent=2, ensure_ascii=False
                             )

{sourcecode-4.5.2 → sourcecode-4.5.3}/src/sourcecode/client_calls.py

@@ -185,9 +185,12 @@ def extract_client_calls(
         except OSError:
             continue
         try:
-            relative = str(file_path.relative_to(base))
+            # POSIX like every other published path (C2-24). A client-usage row
+            # exists to be joined against a route's declaring file, and a join
+            # on path fails silently when the two sides spell it differently.
+            relative = file_path.relative_to(base).as_posix()
         except ValueError:
-            relative = str(file_path)
+            relative = Path(file_path).as_posix()
         for match in _VERB_CALL.finditer(text):
             receiver = match.group("receiver") or ""
             if not _HTTP_RECEIVER.search(receiver):

{sourcecode-4.5.2 → sourcecode-4.5.3}/src/sourcecode/contract_init.py

@@ -125,24 +125,52 @@ def _security_rules(cir: Any) -> tuple[list[dict], list[dict]]:
     return proposed, rejected
 
 
-def _finding_rules(cir: Any) -> tuple[list[dict], list[dict]]:
-    """`forbid_finding` for audit patterns that fire nowhere today."""
-    from sourcecode.spring_security_audit import _DEFAULT_SECURITY_PATTERNS
+def _rule_catalogue() -> "list[tuple[str, str]]":
+    """Every rule this build ships, as `(pattern_id, severity)`.
+
+    Read from `rule_catalog.RULES` — the one authority `--help`, the README and
+    the rule reference already render from, and the one a test binds to the
+    pattern registries so a new rule cannot exist without a row.
+
+    C1-24's second half: this list used to be built from the two *pattern
+    registries* alone, which stop at SEC-004. SEC-005, SEC-006 and SEC-007 are
+    produced by `security_config_scan` and appeared in **neither** the proposed
+    nor the rejected list — a rule could be silently outside contract derivation
+    with nothing saying so, which is the failure mode a "not proposed, and why"
+    list exists to prevent.
+    """
+    from sourcecode.rule_catalog import RULES
+    from sourcecode.spring_security_audit import (
+        _DEFAULT_SECURITY_PATTERNS,
+        SecurityScanner,
+    )
     from sourcecode.spring_tx_analyzer import _DEFAULT_TX_PATTERNS
+
+    severities: "dict[str, str]" = {}
+    for pattern in (*_DEFAULT_TX_PATTERNS, *_DEFAULT_SECURITY_PATTERNS):
+        pattern_id = getattr(pattern, "pattern_id", "")
+        if pattern_id:
+            severities[pattern_id] = getattr(pattern, "severity", "low")
+    # The configuration rules carry their severity on the scanner that emits
+    # them, not on a pattern object; same authority, different shape.
+    severities.update(getattr(SecurityScanner, "_SEVERITY", {}) or {})
+    return sorted(
+        (rule.id, severities.get(rule.id, "low")) for rule in RULES if rule.id
+    )
+
+
+def _finding_rules(cir: Any, root: "Optional[Path]" = None) -> tuple[list[dict], list[dict]]:
+    """`forbid_finding` for audit patterns that fire nowhere today."""
     from sourcecode.verify_rules import _audit_findings
 
-    findings = _audit_findings(cir)
+    # Same `root` the gate evaluates with: without it the configuration rules
+    # cannot fire, and every one of them would be proposed as "fires nowhere".
+    findings = _audit_findings(cir, root)
     firing = Counter(f.pattern_id for f in findings)
 
     proposed: list[dict] = []
     rejected: list[dict] = []
-    catalogue = [
-        (getattr(p, "pattern_id", ""), getattr(p, "severity", "low"))
-        for p in (*_DEFAULT_TX_PATTERNS, *_DEFAULT_SECURITY_PATTERNS)
-    ]
-    for pattern_id, severity in sorted(set(catalogue)):
-        if not pattern_id:
-            continue
+    for pattern_id, severity in _rule_catalogue():
         count = firing.get(pattern_id, 0)
         if count:
             rejected.append({
@@ -283,7 +311,9 @@ def _slug(annotation: str) -> str:
     return annotation.lstrip("@").replace(".", "-").lower()
 
 
-def execute_candidates(candidates: list[dict], cir: Any) -> tuple[list[dict], list[dict]]:
+def execute_candidates(
+    candidates: list[dict], cir: Any, root: "Optional[Path]" = None
+) -> tuple[list[dict], list[dict]]:
     """``(kept, dropped)`` — run every candidate before publishing it.
 
     The check that makes this a baseline rather than a wish list, and the one
@@ -292,6 +322,12 @@ def execute_candidates(candidates: list[dict], cir: Any) -> tuple[list[dict], li
     the engine `verify` runs and drops anything that disagrees, so a derivation
     that grows a blind spot cannot ship a red gate on the commit it was
     generated from.
+
+    C1-24: this net had the hole it was meant to catch. It evaluated without a
+    `root`, so the security *configuration* rules could not fire here either, and
+    `no-sec-004` passed the check that exists to stop exactly that rule from
+    being published. A safety net blind in the same way as the thing it guards
+    is not a second opinion.
     """
     from sourcecode.verify_rules import evaluate_all, parse_rules
 
@@ -300,7 +336,7 @@ def execute_candidates(candidates: list[dict], cir: Any) -> tuple[list[dict], li
     for rule in candidates:
         payload = {k: v for k, v in rule.items() if not k.startswith("_")}
         try:
-            violations = evaluate_all(parse_rules({"contracts": [payload]}), cir)
+            violations = evaluate_all(parse_rules({"contracts": [payload]}), cir, root)
         except Exception as exc:  # a rule this module built must never break the run
             dropped.append({"id": rule.get("id", "?"), "reason": f"did not parse: {exc}"})
             continue
@@ -333,12 +369,17 @@ def derive_contracts(root: Path, cir: Optional[Any] = None) -> dict:
 
     proposed: list[dict] = []
     rejected: list[dict] = []
+    # `_finding_rules` needs the root; the graph-only derivations do not. Passed
+    # by name rather than positionally so adding a third file-reading derivation
+    # is a one-word change instead of a silent blind spot (C1-24).
     for derive in (_security_rules, _finding_rules, _edge_rules):
-        rules, skipped = derive(cir)
+        rules, skipped = (
+            derive(cir, root) if derive is _finding_rules else derive(cir)
+        )
         proposed.extend(rules)
         rejected.extend(skipped)
 
-    kept, dropped = execute_candidates(proposed, cir)
+    kept, dropped = execute_candidates(proposed, cir, root)
 
     return {
         "schema_version": SCHEMA_VERSION,

{sourcecode-4.5.2 → sourcecode-4.5.3}/src/sourcecode/data_exposure.py

@@ -146,15 +146,44 @@ def build_data_exposure(
     if not decl.labels:
         # No declaration is not "nothing is exposed" — it is a question nobody asked
         # yet, and the counts stay null rather than reading as a clean bill (I-3/R9).
+        #
+        # C4-14: the statement said what to do and the payload gave the reader no
+        # way to do it — "cero valor out-of-the-box, cero guía de onboarding en la
+        # propia salida", on the payload aimed at a regulated buyer. The file name,
+        # the key and the shape are read from `data_labels`, the module that parses
+        # them, so an example that stops being valid fails the build instead of
+        # teaching a buyer the wrong schema.
+        from sourcecode.data_labels import CONFIG_FILENAME, CONFIG_KEY
+        from sourcecode.remedies import remedy as _remedy
+
         payload.update(
             {
                 "answered": False,
                 "statement": (
                     "No data labels are declared, so this run measured nothing. A "
                     "label is a judgement about a domain and is never inferred from a "
-                    "field name: declare one under `dataLabels` in "
-                    "sourcecode.config.json and re-run."
+                    f"field name: declare one under `{CONFIG_KEY}` in "
+                    f"{CONFIG_FILENAME} and re-run."
                 ),
+                "remedy": _remedy("no_data_labels_declared").to_dict(),
+                "declare": {
+                    "file": CONFIG_FILENAME,
+                    "key": CONFIG_KEY,
+                    "example": {
+                        CONFIG_KEY: [
+                            {
+                                "label": "pii",
+                                "types": ["com.example.Person"],
+                                "fields": ["com.example.Account#iban"],
+                            }
+                        ]
+                    },
+                    "note": (
+                        "`types` labels every route whose signature names the type; "
+                        "`fields` labels the declaring type through the member. Both "
+                        "are fully-qualified, and neither is matched by name shape."
+                    ),
+                },
                 "summary": {
                     "labels": 0,
                     "seeds": 0,

{sourcecode-4.5.2 → sourcecode-4.5.3}/src/sourcecode/remedies.py

@@ -75,6 +75,23 @@ REMEDIES: "dict[str, Remedy]" = {
             ),
             headline="which profile set actually runs",
         ),
+        Remedy(
+            key="no_data_labels_declared",
+            command="data-exposure",
+            # The next move is a declaration, not a flag: nothing this command
+            # could be told on the command line would make it *infer* that a
+            # field is sensitive, because that is a judgement about a domain.
+            # The invocation is the re-run, and `declare` beside it in the
+            # payload carries the file, the key and a copy-pasteable example.
+            option="",
+            answers=(
+                "no data labels are declared, so nothing was measured — declare which "
+                "types and fields are sensitive under `dataLabels` in "
+                "sourcecode.config.json (a label is a judgement about a domain and is "
+                "never inferred from a field name), then re-run"
+            ),
+            headline="declare the labels, then re-run",
+        ),
         Remedy(
             key="no_contracts_declared",
             command="verify",

{sourcecode-4.5.2 → sourcecode-4.5.3}/src/sourcecode/ris.py

@@ -451,24 +451,34 @@ def _tree_signature(repo_root: Path) -> str:
 
 
 def _has_uncommitted_changes(repo_root: Path) -> bool:
-    """Return True if working tree has staged or unstaged changes to tracked files.
-
-    Uses ``git status --porcelain --untracked-files=no`` so that untracked
-    files (e.g. legacy .sourcecode-cache/ directories) do not produce false
-    positives.  Returns False on any error (non-git dirs, etc.).
+    """True when the working tree differs from HEAD in a way the analysis sees.
+
+    C3-40: this used to pass ``--untracked-files=no``, to keep legacy
+    `.sourcecode-cache/` directories from reading as modifications. That excused
+    far more than the noise it was aimed at — **an untracked `.java` file is in
+    the IR**, so a repository with a new source file reported
+    `has_uncommitted_changes: False` and, through it, `fresh: true` for a snapshot
+    that did not contain that file. A false *fresh* is the confident-falsehood
+    direction. It also contradicted the root scan's own `--changed-only`, whose
+    help promises "staged, unstaged, **untracked**" in the same CLI.
+
+    One authority now answers it: `baseline_autocapture.worktree_dirty`, which
+    already had to get this exactly right for the architectural history. It
+    honours `.gitignore` (so build output and any gitignored cache directory are
+    not modifications — the original concern, handled properly) and excuses only
+    this tool's own footprint under `.ask/`, for the reason recorded there: the
+    first capture leaves an untracked file behind, and counting it would leave
+    every history one entry long.
+
+    Returns False when the state cannot be determined (non-git directories), which
+    is the pre-existing contract of this helper — `worktree_dirty` answers `None`
+    there, and the RIS freshness fields have no way to carry it.
     """
-    try:
-        result = subprocess.run(
-            ["git", "-C", str(repo_root), "status", "--porcelain", "--untracked-files=no"],
-            capture_output=True,
-            text=True,
-            timeout=2,
-        )
-        if result.returncode == 0:
-            return bool(result.stdout.strip())
-    except Exception:
-        pass
-    return False
+    from sourcecode.baseline_autocapture import HISTORY_DIRNAME, worktree_dirty
+
+    root = Path(repo_root)
+    own = (root / HISTORY_DIRNAME, root / HISTORY_DIRNAME.parts[0])
+    return bool(worktree_dirty(root, ignore=own))
 
 
 def get_cold_start_context(repo_root: Path) -> dict:

{sourcecode-4.5.2 → sourcecode-4.5.3}/src/sourcecode/security_config_scan.py

@@ -338,7 +338,22 @@ def scan_security_configuration(
     java_files: "Optional[list[str]]" = None,
     config_files: "Optional[list[str]]" = None,
 ) -> "list[SecurityConfigObservation]":
-    """Every textbook weakness under ``root``. Best-effort per file, never fatal."""
+    """Every textbook weakness under ``root``. Best-effort per file, never fatal.
+
+    E-2: every rule here matches on raw text, and until 4.5.3 none of them could
+    tell code from a comment — `SEC-004` and `SEC-007` fired identically on a live
+    statement and on a commented-out block, which cost 31 % of the `high` findings
+    in a field evaluation. Comments are blanked **here**, at the three points a
+    file is read, rather than inside each rule: a rule added later is covered by
+    construction instead of by whoever remembers. `source_text` preserves offsets,
+    so the line each observation reports is still the line in the real file.
+    """
+    from sourcecode.source_text import (
+        blank_hash_comments,
+        blank_java_comments,
+        blank_xml_comments,
+    )
+
     root = Path(root)
     if java_files is None:
         try:
@@ -353,7 +368,7 @@ def scan_security_configuration(
             source = (root / rel).read_text(encoding="utf-8", errors="replace")
         except OSError:
             continue
-        out.extend(_scan_java(source, rel))
+        out.extend(_scan_java(blank_java_comments(source), rel))
 
     if config_files is None:
         config_files = []
@@ -367,7 +382,7 @@ def scan_security_configuration(
             text = (root / rel).read_text(encoding="utf-8", errors="replace")
         except OSError:
             continue
-        out.extend(_scan_config(text, rel))
+        out.extend(_scan_config(blank_hash_comments(text), rel))
 
     # CL-10 — the descriptors. Discovered the same way `environment_resolution`
     # classifies them, so the two surfaces cannot disagree about which files are
@@ -386,7 +401,7 @@ def scan_security_configuration(
             text = (root / rel).read_text(encoding="utf-8", errors="replace")
         except OSError:
             continue
-        out.extend(_scan_descriptor(text, rel))
+        out.extend(_scan_descriptor(blank_xml_comments(text), rel))
 
     return sorted(
         _shared_across_environments(out), key=lambda o: (o.rule, o.file, o.line)

{sourcecode-4.5.2 → sourcecode-4.5.3}/src/sourcecode/servlet_surface.py

@@ -144,9 +144,15 @@ def build_servlet_surface(root: Path, *, limit: Optional[int] = None) -> dict:
         if any(part in _SKIP_DIRS for part in path.parts) or not path.is_file():
             continue
         try:
-            relative = str(path.relative_to(root))
+            # POSIX, like every other path this CLI publishes. C2-18 fixed this
+            # for `spring_profiles.conditional_beans[].source_file`; this module
+            # shipped two releases later without the convention, so one response
+            # could carry `src/main/java/...` beside `src\main\options\...`
+            # and defeat a consumer joining on path — which is what a servlet
+            # population is for (C2-24).
+            relative = path.relative_to(root).as_posix()
         except ValueError:
-            relative = str(path)
+            relative = Path(path).as_posix()
         if path.name == "web.xml":
             mappings.extend(_from_web_xml(path, relative))
         elif path.suffix == ".java":

sourcecode-4.5.3/src/sourcecode/source_text.py

@@ -0,0 +1,103 @@
+"""source_text.py — the one authority for "this text, with comments neutralised".
+
+A static analyzer that matches raw text cannot tell code from a note about code.
+This project has now paid for that three times:
+
+* **E-1 (3.7.0)** — a comment containing `class` before an annotated declaration
+  swallowed the real declaration, and the file left the graph entirely.
+* **E-2 (4.5.3)** — the security configuration scan reported `SEC-004` and
+  `SEC-007` on commented-out code, which cost **31 % of the `high` findings** in a
+  field evaluation. Worse than the count: the correlation engine wrote a
+  persuasive paragraph about credentials in a block that was switched off.
+  Confident prose about dead code is worse than a terse warning, because it
+  invites trust.
+
+**Offsets are preserved exactly.** Comment bodies are overwritten with spaces and
+every newline is kept, so the result has the same length and the same line
+structure as the input. A match found in the blanked text has the same
+`start()`, and therefore the same reported line, as it would in the original —
+which is what lets a scanner keep quoting the source file it read.
+
+**String literals are deliberately kept.** This is the difference between these
+functions and `hibernate_strat._strip_comments_strings`, which blanks literal
+*content* on purpose so that a pattern cannot match a substring living inside a
+string. The security rules need the opposite: `new MessageDigestPasswordEncoder(
+"SHA-1")` and `<param-value>admin:{SHA-256}…</param-value>` carry their evidence
+*in* the literal. Two different questions, two functions, neither a copy of the
+other.
+"""
+
+from __future__ import annotations
+
+import re
+
+#: Java/C-family lexical tokens, in precedence order. String and character
+#: literals are matched **first** so that a `//` or `/*` living inside one — as in
+#: `String url = "http://example.com";` — is never mistaken for a comment.
+_JAVA_TOKENS = re.compile(
+    r'"""(?:\\.|[^\\])*?"""'      # text block (Java 15+), before the plain string
+    r'|"(?:\\.|[^"\\\n])*"'       # string literal
+    r"|'(?:\\.|[^'\\\n])*'"       # char literal
+    r"|(?P<line>//[^\n]*)"        # line comment
+    r"|(?P<block>/\*.*?\*/)",     # block comment
+    re.DOTALL,
+)
+
+#: XML/HTML comments. Unterminated ones run to end of input on purpose: an
+#: unclosed `<!--` comments out the rest of the document for a parser too.
+_XML_COMMENT = re.compile(r"<!--.*?(?:-->|\Z)", re.DOTALL)
+
+
+def _blank(text: str) -> str:
+    """Same length, same newlines, no content."""
+    return "".join("\n" if ch == "\n" else " " for ch in text)
+
+
+def blank_java_comments(source: str) -> str:
+    """Java source with comment bodies replaced by spaces, offsets preserved.
+
+    String and character literals are left intact — see the module docstring.
+    """
+    def _replace(match: "re.Match[str]") -> str:
+        if match.lastgroup in ("line", "block"):
+            return _blank(match.group(0))
+        return match.group(0)  # a literal: untouched
+
+    return _JAVA_TOKENS.sub(_replace, source)
+
+
+def blank_xml_comments(source: str) -> str:
+    """XML/HTML with comment bodies replaced by spaces, offsets preserved."""
+    return _XML_COMMENT.sub(lambda m: _blank(m.group(0)), source)
+
+
+#: A `#` comment in `.properties`/YAML, but only where it opens the line. A `#`
+#: further along may be a comment in YAML and is part of the value in
+#: `.properties` (`password=abc#123`), and blanking that would corrupt evidence to
+#: avoid a false positive — the wrong trade for a scan that quotes what it found.
+#: The fully-commented-out block E-2 is about is covered by the line-start form.
+_HASH_COMMENT_LINE = re.compile(r"^[ \t]*#[^\n]*", re.MULTILINE)
+
+
+def blank_hash_comments(source: str) -> str:
+    """`.properties`/YAML with whole-line `#` comments blanked, offsets preserved."""
+    return _HASH_COMMENT_LINE.sub(lambda m: _blank(m.group(0)), source)
+
+
+def commented_spans(source: str, *, xml: bool) -> "list[tuple[int, int]]":
+    """`(start, end)` offsets of every comment — what was switched off, not what runs.
+
+    The inverse of the functions above, and the reason they blank rather than
+    delete. A control that is present but commented out is not noise: somebody
+    decided it was needed and then turned it off, which is the most actionable
+    thing a reader can be told about a security surface. E-2's fix removes those
+    matches from the findings; this is what a future rule reports them *as*
+    (DEAD-001, queued as F-A). Nothing consumes it yet — it exists so the two
+    halves cannot drift apart when it does.
+    """
+    pattern = _XML_COMMENT if xml else _JAVA_TOKENS
+    spans: "list[tuple[int, int]]" = []
+    for match in pattern.finditer(source):
+        if xml or match.lastgroup in ("line", "block"):
+            spans.append((match.start(), match.end()))
+    return spans