sourcecode 5.9.17__tar.gz → 5.9.18__tar.gz

{sourcecode-5.9.17 → sourcecode-5.9.18}/CHANGELOG.md

@@ -1,5 +1,20 @@
 # Changelog
 
+## [5.9.18] — 2026-08-31
+
+*Windows reliability remediation follow-up.*
+
+### Fixed
+
+- `pack gate` opens child stderr as UTF-8 with replacement behavior, so a Windows code-page mismatch cannot terminate its forwarding thread and block a child on an undrained pipe.
+- The shared parse-worker factory rejects Windows process pools as a second deterministic barrier; repository parsing uses the existing sequential path there instead of risking `DuplicateHandle` during worker startup.
+- Expected CLI validation exits, including invalid `--profile` input, now pass through the analysis-error boundary unchanged as `INVALID_INPUT` / exit 2.
+
+### Verification Boundary
+
+- Focused pack, parallel, validation and six-command reliability-matrix tests pass on the engineering host. Native Windows repetition and process-tree evidence remain required before closing `AUD-621-R` and `AUD-621-WIN-PROC`.
+- The 24,073-file context-analysis threshold remains open pending profiling and an intermediate-size benchmark.
+
 ## [5.9.17] — 2026-08-31
 
 *Reliability and monorepo performance follow-up after the 5.9.16 external audit.*

{sourcecode-5.9.17 → sourcecode-5.9.18}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sourcecode
-Version: 5.9.17
+Version: 5.9.18
 Summary: Persistent structural context and ultra-fast repeated analysis for AI coding agents
 Project-URL: Homepage, https://github.com/HarounDominique/sourcecode
 Project-URL: Repository, https://github.com/HarounDominique/sourcecode
@@ -59,7 +59,7 @@ Description-Content-Type: text/markdown
 > question. All analysis is local and deterministic; see [Privacy](#configuration--privacy)
 > before inspecting a repository whose telemetry policy you need to control.
 
-![Version](https://img.shields.io/badge/version-5.9.17-blue)
+![Version](https://img.shields.io/badge/version-5.9.18-blue)
 ![Python](https://img.shields.io/badge/python-3.9%2B-green)
 
 Copyright (c) 2026 Dominique Haroun. All rights reserved. ASK Engine 5.7.0 and
@@ -267,7 +267,7 @@ brew tap haroundominique/sourcecode && brew install sourcecode
 # pip / pipx
 pipx install sourcecode        # or: pip install sourcecode
 
-ask version                    # ask 5.9.17 — and, on a build that has aged,
+ask version                    # ask 5.9.18 — and, on a build that has aged,
                                # how many releases have probably shipped since
 ```
 

{sourcecode-5.9.17 → sourcecode-5.9.18}/README.md

@@ -10,7 +10,7 @@
 > question. All analysis is local and deterministic; see [Privacy](#configuration--privacy)
 > before inspecting a repository whose telemetry policy you need to control.
 
-![Version](https://img.shields.io/badge/version-5.9.17-blue)
+![Version](https://img.shields.io/badge/version-5.9.18-blue)
 ![Python](https://img.shields.io/badge/python-3.9%2B-green)
 
 Copyright (c) 2026 Dominique Haroun. All rights reserved. ASK Engine 5.7.0 and
@@ -218,7 +218,7 @@ brew tap haroundominique/sourcecode && brew install sourcecode
 # pip / pipx
 pipx install sourcecode        # or: pip install sourcecode
 
-ask version                    # ask 5.9.17 — and, on a build that has aged,
+ask version                    # ask 5.9.18 — and, on a build that has aged,
                                # how many releases have probably shipped since
 ```
 

sourcecode-5.9.18/docs/AUDIT-2026-08-31-EXTERNAL-5.9.17.md

@@ -0,0 +1,42 @@
+# External Audit — 5.9.17 Re-audit
+
+**Received:** 2026-08-31  
+**Scope:** repositories under `testing/`  
+**Evidence status:** four independent Windows samples; conflicting earlier
+success samples are superseded by the multi-session result below.
+
+## Summary
+
+The command-surface recovery from `5.9.16` remains confirmed: all nine restored
+commands, Bug F, Bug Q, baseline behavior and endpoint/posture validation are
+healthy in this second independent round. The root compact command is also
+healthy when its target directory is verified before execution.
+
+The shared Windows worker defect is not closed. The report records low or zero
+success rates across all affected commands, with substantial variation between
+sessions. This makes single-session success evidence insufficient for closure.
+
+## Governing Findings
+
+| ID | Finding | Required closure |
+|---|---|---|
+| `AUD-621-R` | `risk --limit != 0` and Markdown `audit-report` were 0% successful; `verify`, `spring-audit`, `migrate-check` and `migrate-apply` were 0-27% successful across independent samples. The observed mechanism remains Windows `DuplicateHandle` failure during worker spawn. | Fail closed or fall back deterministically to sequential work; validate >=25 repetitions per command in >=2 native Windows sessions, with target identity, exit status and byte counts recorded. |
+| `AUD-621-PACK-STDERR` | `pack gate --since` can deadlock because `_forward_stderr` decodes child UTF-8 output as CP1252, raises `UnicodeDecodeError`, and stops draining stderr. | Use UTF-8 with replacement behavior, keep the forwarding loop alive after decode failures, and cover CP1252 and UTF-8 child output. |
+| `AUD-621-CONTEXT` | `analyzing repository context` completed in seconds through roughly 4,772 Java files but stopped advancing beyond 190 seconds on the 24,073-file corpus. The same phase precedes `cache warm` agent-view exit 143. | Profile the phase and add an intermediate-size fixture or subset benchmark to locate the threshold before optimizing. |
+| `AUD-621-WIN-PROC` | Four descendant `ask.exe` processes remained. | Native Windows/Git Bash process-tree tests for normal completion, timeout and interruption. |
+
+## Confirmed Improvements
+
+| Area | External result |
+|---|---|
+| Command surface | All nine commands restored in `5.9.16` remain present and functional. |
+| Manifest parsing | `tutorials` manifest parsing remained improved at 18.8 seconds. |
+| `review-pr` | No recurrence in five further runs. |
+| Kotlin detection | A second real Kotlin repository was classified correctly. |
+
+## Operating Position
+
+Use ASK in bounded, supervised workflows only. Do not represent `5.9.17` as
+reliable for production CI-gating or enterprise-scale Windows analysis until the
+worker, stderr-forwarding and process-lifecycle findings close with repeatable
+native evidence.

sourcecode-5.9.18/docs/AUDIT-2026-08-31-REMEDIATION-5.9.18.md

@@ -0,0 +1,20 @@
+# Release 5.9.18 — AUD-621 Remediation
+
+**Date:** 2026-08-31  
+**Scope:** fixes applied after the external `5.9.17` audit.
+
+## Implemented Fixes
+
+| Finding | Change | Evidence in this checkout |
+|---|---|---|
+| `AUD-621-PACK-STDERR` | Pack child stderr uses UTF-8 with replacement; the forwarding loop cannot terminate on a decode exception. | Pack progress and manifest tests pass; a regression test asserts the subprocess decoding contract. |
+| `AUD-621-R` | Windows parse-worker construction is forbidden, backing up the existing sequential fallback. | Parallel Windows-contract tests pass; the six-command reliability matrix completes on the engineering host. |
+| Validation regression found during verification | Expected `typer.Exit` validation exits bypass the analysis-failure wrapper. | `risk` and `audit-report` invalid-profile regressions pass with `INVALID_INPUT` and exit 2. |
+
+## Unchanged Closure Gates
+
+- `AUD-621-R`: at least 25 repetitions per affected command in at least two native Windows sessions.
+- `AUD-621-WIN-PROC`: native Windows/Git Bash process-tree tests for successful, timed-out and interrupted commands.
+- `AUD-621-CONTEXT`: phase profile and intermediate-size benchmark before a performance claim for the 24,073-file corpus.
+
+Release `5.9.18` is therefore suitable only for bounded, supervised use until those gates are met.

{sourcecode-5.9.17 → sourcecode-5.9.18}/docs/DEFECT-LEDGER.md

@@ -14,6 +14,39 @@ Status values: `open` · `in-progress` · `closed <version>` · `by-design` (wit
 The facts these rows are keyed to are published: `ask schema facts-v1` prints the registry
 (authority, definition, consumers, parity test) that the release battery enforces.
 
+## Current Synchronization — External Audit `AUD-621`
+
+**Release `5.9.18` remediation.** `AUD-621-PACK-STDERR` is corrected in code: the child pipe is read as UTF-8 with replacement and the forwarding loop retains a decode-failure guard. `AUD-621-R` has a second Windows barrier at process-pool construction, complementing the sequential fallback. Both findings remain open until their required native Windows evidence is recorded; `AUD-621-CONTEXT` and `AUD-621-WIN-PROC` are unchanged.
+
+**Re-audit of release `5.9.17`, received 2026-08-31.** Four independent
+Windows measurements overturn the apparent partial recovery of the shared
+worker failure: `risk` with a non-zero limit and Markdown `audit-report` had
+zero successful runs; `verify`, `spring-audit`, `migrate-check` and
+`migrate-apply` succeeded only intermittently (roughly 0-27% by command and
+session). The known `PermissionError: [WinError 5]` in `DuplicateHandle` remains
+the governing mechanism. A successful run from one session is not closure
+evidence for this race.
+
+The audit also isolates a new, high-confidence Windows defect in `pack gate`:
+the `_forward_stderr` thread decodes child output using the active codepage,
+raises `UnicodeDecodeError` on UTF-8 bytes, then stops draining the pipe. The
+child can block once that pipe fills. Two independent forks captured the same
+traceback. The full evidence is
+[`AUDIT-2026-08-31-EXTERNAL-5.9.17.md`](AUDIT-2026-08-31-EXTERNAL-5.9.17.md).
+
+| ID | Severity | Status | Evidence and required closure |
+|---|---:|---|---|
+| `AUD-621-R` | Critical maximum | **open, P0** | All direct shared-worker commands remain unreliable on Windows: `risk --limit != 0` and Markdown `audit-report` were 0% successful; `verify`, `spring-audit`, `migrate-check` and plan-mode `migrate-apply` were 0-27%. Detect pool startup failure, fall back to one process or emit structured non-zero failure, and validate >=25 repetitions in >=2 native Windows sessions per command. |
+| `AUD-621-PACK-STDERR` | Critical | **open, P0** | `packs.py:_forward_stderr` uses the Windows default codepage and crashes on UTF-8 progress bytes; the unread stderr pipe then deadlocks `pack gate`. Open the child stream with UTF-8 and `errors="replace"`, keep draining on decode errors, and test CP1252 plus UTF-8 output. |
+| `AUD-621-CONTEXT` | High | **open, P1** | `analyzing repository context` is stable below about 5,000 Java files but ceases to advance on the 24,073-file corpus for more than 190 seconds. It also explains the `cache warm` agent-view exit 143. Profile this one phase and locate its threshold with 8,000-12,000-file subsets or a synthetic fixture. |
+| `AUD-621-WIN-PROC` | Medium | **open, platform-blocked** | Four `ask.exe` descendants remained after the audit run. Native Windows process-tree evidence is still required. |
+| `AUD-621-REVIEW-PR` | Medium | **closed/no-repro** | No recurrence in five further runs; retain the regression test. |
+
+The restored command surface, Bug F, Bug Q, baseline behavior and endpoint/posture
+validation are now independently stable across multiple rounds. They remain
+closed. Production CI-gating and enterprise-scale claims remain blocked by
+`AUD-621-R`, `AUD-621-PACK-STDERR` and the Windows lifecycle evidence.
+
 ## Current Synchronization — External Audit `AUD-620`
 
 **Re-audit of release `5.9.16`, received 2026-08-31.** This audit confirms that

{sourcecode-5.9.17 → sourcecode-5.9.18}/docs/DEVELOPMENT-ROADMAP.md

@@ -1,5 +1,39 @@
 # ASK — Development Roadmap
 
+### Release 5.9.18 — `AUD-621` remediation
+
+Release `5.9.18` adds two deterministic Windows safeguards: no shared parse worker can be constructed on Windows, and pack-child stderr is decoded as UTF-8 with replacement so the forwarding pipe remains drained. It also preserves expected validation exits through the structured analysis-error boundary.
+
+These code changes do not replace the required native evidence. `AUD-621-R` and `AUD-621-WIN-PROC` remain open until the repetition matrix and process-tree cases pass on native Windows; `AUD-621-CONTEXT` remains profiling work.
+
+### External Audit Intake — `AUD-621` re-audit, release 5.9.17
+
+The 2026-08-31 re-audit corrects the optimistic interpretation of the previous
+round. Four independent Windows samples show that the shared `DuplicateHandle`
+worker-start failure is still active: `risk --limit != 0` and Markdown
+`audit-report` had no successful runs, while `verify`, `spring-audit`,
+`migrate-check` and `migrate-apply` were only intermittently successful. The
+current structured-error boundary is useful, but it does not make a failed or
+blocked worker pool reliable.
+
+**Engineering decision: fix the pool path before further feature work.** Make
+worker startup fail closed or fall back deterministically to a single process;
+instrument pool creation with process/handle context; then validate each affected
+command in at least 25 repetitions across two or more native Windows sessions.
+One clean session, however large, cannot close this defect.
+
+| ID | Priority | Next action and closure boundary |
+|---|---:|---|
+| `AUD-621-R` | P0 | Implement deterministic sequential fallback at every shared-worker entry, with bounded waits and structured failure evidence. Compare direct risk computation with `audit-report --from-risk`, then collect the multi-session Windows matrix. |
+| `AUD-621-PACK-STDERR` | P0 | Open child stderr as UTF-8 with replacement behavior and ensure `_forward_stderr` cannot die without draining the pipe. Add a CP1252/UTF-8 subprocess regression for `pack gate`. |
+| `AUD-621-CONTEXT` | P1 | Profile `analyzing repository context`; it is a threshold failure, not a uniform slowdown, and also causes the agent-view warm timeout. Add an intermediate-size fixture or subset benchmark before optimizing. |
+| `AUD-621-WIN-PROC` | P1 | Measure descendant cleanup around successful, timed-out and interrupted commands on native Windows/Git Bash. |
+
+The root compact command itself, the recovered command surface, redirection and
+validation fixes remain healthy. Commercially, limit use to bounded pilots and
+do not claim production CI gating or enterprise-scale reliability. Full evidence
+is in [`AUDIT-2026-08-31-EXTERNAL-5.9.17.md`](AUDIT-2026-08-31-EXTERNAL-5.9.17.md).
+
 ### External Audit Intake — `AUD-620` re-audit, release 5.9.16
 
 The external re-audit received 2026-08-31 confirms a major recovery from the

{sourcecode-5.9.17 → sourcecode-5.9.18}/docs/EXECUTIVE-ROADMAP.md

@@ -5,6 +5,20 @@
 > `pack assessment` and `pack gate`, and make evidence and safe change the
 > product surface. This roadmap must not introduce a competing product thesis.
 
+> **CURRENT UPDATE — release `5.9.18`, remediation.** Windows parse workers are now forbidden at their construction point and `pack gate` drains UTF-8 child stderr safely. Expected validation exits remain typed input errors. These are code fixes, not native-Windows closure evidence: keep bounded pilots until the required repetition and process-lifecycle measurements complete.
+
+> **CURRENT UPDATE — release `5.9.17`, external re-audit.** The restored command
+> surface and prior validation fixes remain stable, but the shared Windows worker
+> failure is confirmed active, not partially closed. `risk --limit != 0` and
+> Markdown `audit-report` failed throughout the independent samples; `verify`,
+> `spring-audit`, `migrate-check` and `migrate-apply` remain highly intermittent.
+> A new `pack gate` deadlock is also traced to CP1252 decoding in its stderr
+> forwarding thread. The immediate engineering focus is deterministic worker
+> fallback, safe stderr decoding, and native Windows evidence. Keep bounded
+> pilots only; the working commercial hypothesis is `$20-28/developer/month` and
+> no production CI-gating or enterprise-scale reliability promise is authorized.
+> See [`AUDIT-2026-08-31-EXTERNAL-5.9.17.md`](AUDIT-2026-08-31-EXTERNAL-5.9.17.md).
+
 > **CURRENT UPDATE — release `5.9.16`, external re-audit.** The defective
 > `5.9.15` command surface has been recovered: all nine missing commands are
 > restored and independently invoked, and the prior `/dev/null`, option-parser,

{sourcecode-5.9.17 → sourcecode-5.9.18}/docs/USER_GUIDE.md

@@ -46,16 +46,16 @@ CLI commands        — impact, endpoints, spring-audit, explain, … each a pro
 The key idea: the extraction is **content-addressed**. Commands reuse the parse cache and,
 where their analysed scope matches, the shared Canonical IR; `ask cache model` names what a
 warm buys for each command rather than implying that every projection costs the same. In
-5.9.17, `validation` enters through that shared CIR and `data-exposure` reuses one semantic
+5.9.18, `validation` enters through that shared CIR and `data-exposure` reuses one semantic
 model across all declared label seeds. (The extraction and consumption contract is fixed in
 the architecture ADRs 0001–0004 under `docs/architecture/`.)
 
 The current external pack audit intake is tracked in
 [`AUDIT-2026-08-26-EXTERNAL-5.8.28-PACK.md`](AUDIT-2026-08-26-EXTERNAL-5.8.28-PACK.md);
-the two pack fixes are included in release 5.9.17, with release-artifact verification
+the two pack fixes are included in release 5.9.18, with release-artifact verification
 remaining for detached execution and pack evidence.
 
-### Reliability contracts in 5.9.17
+### Reliability contracts in 5.9.18
 
 **Failures must be visible.** `risk`, `verify`, `audit-report` and
 `migrate-check` turn unexpected analysis failures into structured
@@ -64,6 +64,13 @@ The release battery includes a repeatable six-command Windows matrix; it records
 exit codes and output sizes, but native Windows evidence remains required before
 closing platform-specific findings.
 
+**Current Windows limitation.** The `5.9.17` external audit reproduced the
+shared worker-start failure in `risk`, `verify`, `audit-report`, `spring-audit`,
+`migrate-check` and `migrate-apply`. Do not use these commands as a production
+CI gate on Windows until the native matrix closes them. `pack gate --since` now
+uses safe UTF-8 stderr forwarding, but still requires native repeat evidence. See the
+[development roadmap](DEVELOPMENT-ROADMAP.md#external-audit-intake--aud-621-re-audit-release-5917).
+
 **A deadline the root analysis honours.** `ASK_MAX_ANALYSIS_SECONDS=<seconds>` now
 stops `ask <repo>` at a measured phase boundary. A run that spends its budget
 returns a structured `partial-answer-v1` payload naming the phases that completed,

{sourcecode-5.9.17 → sourcecode-5.9.18}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "sourcecode"
-version = "5.9.17"
+version = "5.9.18"
 description = "Persistent structural context and ultra-fast repeated analysis for AI coding agents"
 readme = "README.md"
 requires-python = ">=3.9"

{sourcecode-5.9.17 → sourcecode-5.9.18}/src/sourcecode/__init__.py

@@ -4,4 +4,4 @@ ASK Engine is the product. ``ask`` is the canonical CLI command; ``sourcecode``
 the legacy compatibility alias and the Python/PyPI package name. See
 docs/PRODUCT_IDENTITY.md (normative)."""
 
-__version__ = "5.9.17"
+__version__ = "5.9.18"

{sourcecode-5.9.17 → sourcecode-5.9.18}/src/sourcecode/cli.py

@@ -13422,6 +13422,8 @@ def verify_cmd(
                 hint=f"--fail-on accepts: {' | '.join(_vr.FAIL_ON_CHOICES)}.",
             )
             raise typer.Exit(code=2 if ci else 0)
+        except typer.Exit:
+            raise
         except Exception as exc:
             _prog.stop()
             _analysis_failed("verify", "verifying contracts", exc)
@@ -13691,6 +13693,8 @@ def risk_cmd(
                 ci=ci,
                 allow_partial=allow_partial,
             )
+        except typer.Exit:
+            raise
         except Exception as exc:
             _analysis_failed("risk", phase, exc)
         finally:
@@ -14194,6 +14198,8 @@ def audit_report_cmd(
                 checkpoint=_run.checkpoint,
                 partial_status=lambda **cut: _run.status(_WHY_BUDGET, **cut),
             )
+        except typer.Exit:
+            raise
         except Exception as exc:
             _analysis_failed("audit-report", _phase, exc)
         finally:
@@ -15596,6 +15602,8 @@ def migrate_check_cmd(
                 timings=_mig_timings,
             )
             report.include_hygiene = include_hygiene
+        except typer.Exit:
+            raise
         except Exception as exc:
             _analysis_failed("migrate-check", phase, exc)
         finally:

{sourcecode-5.9.17 → sourcecode-5.9.18}/src/sourcecode/packs.py

@@ -719,17 +719,27 @@ def _run_command(command: str, root: Path, output: Path, extra: list[str]) -> di
         stdout=subprocess.PIPE,
         stderr=subprocess.PIPE,
         text=True,
+        # A Windows console often defaults to a legacy code page while ASK emits
+        # UTF-8 diagnostics.  Decoding strictly lets the forwarding thread die,
+        # leaving stderr undrained and the child blocked on a full pipe.
+        encoding="utf-8",
+        errors="replace",
         cwd=str(root),
     )
     stderr_lines: list[str] = []
 
     def _forward_stderr() -> None:
         assert process.stderr is not None
-        for line in process.stderr:
-            stderr_lines.append(line)
-            # Child diagnostics are operational progress/errors, never pack data.
-            sys.stderr.write(line)
-            sys.stderr.flush()
+        try:
+            for line in process.stderr:
+                stderr_lines.append(line)
+                # Child diagnostics are operational progress/errors, never pack data.
+                sys.stderr.write(line)
+                sys.stderr.flush()
+        except UnicodeDecodeError:
+            # `encoding`/`errors` above prevents this for ASK children. Keep the
+            # pack process alive even if a custom stream violates that contract.
+            stderr_lines.append("[ask] stderr contained undecodable bytes\n")
 
     stderr_thread = threading.Thread(target=_forward_stderr, daemon=True)
     stderr_thread.start()

{sourcecode-5.9.17 → sourcecode-5.9.18}/src/sourcecode/parallel.py

@@ -113,7 +113,14 @@ def _reset_for_tests() -> None:
 
 
 def _executor(max_workers: int):
-    """Construct the process executor through one injectable seam."""
+    """Construct the process executor through one injectable seam.
+
+    Windows uses the sequential parse path.  Keep that invariant at the only
+    construction point as well as at the caller, so a future caller cannot
+    reintroduce a ``DuplicateHandle`` worker-start failure.
+    """
+    if os.name == "nt":
+        raise RuntimeError("Windows process pools are disabled; parse sequentially")
     from concurrent.futures import ProcessPoolExecutor
 
     return ProcessPoolExecutor(max_workers=max_workers)