patchshuttle 0.1.0a3__tar.gz → 0.1.0a4__tar.gz

{patchshuttle-0.1.0a3 → patchshuttle-0.1.0a4}/CHANGELOG.md

@@ -5,6 +5,73 @@ uses semantic versioning, including Python-compatible pre-release versions.
 
 ## Unreleased
 
+## 0.1.0a4 - 2026-08-27
+
+Third alpha release, focused on bounded Python source discovery, compact
+structure output, and owner-controlled architecture ratchets.
+
+### Added
+- Owner-controlled Python architecture ratchets evaluate the planner's virtual
+  file state before execution. Defaults warn at 500 lines per module, 15 direct
+  Python files per package, five new Python files, or one new package; hard
+  limits are 1000, 25, 10, and three respectively. Stable bounded `ARCH` findings
+  appear in plans and logs, while unchanged or improving legacy violations remain
+  allowed.
+
+- Read-only `python_structure` uses the standard-library AST to report a
+  bounded declaration map for one Python file or directory, including
+  imports, classes, functions, methods, parameters, decorators, bases,
+  source ranges, parse errors, and explicit file and symbol limit signals,
+  without importing or executing project code. Its opt-in `compact: true`
+  mode retains navigational symbol identities and ranges while omitting
+  detailed import and declaration metadata; full output remains the default.
+
+- Read-only `search_context` returns bounded physical-line context around
+  literal matches, and `read_symbol` returns one exactly resolved
+  decorator-aware Python symbol with its physical range and canonical
+  LF/UTF-8 SHA-256.
+- Guarded `replace_symbol` replaces one exactly resolved Python class,
+  function, method, or nested symbol using the hash returned by
+  `read_symbol` and the planner's sequential simulated file state.
+- The built-in `ruff: {}` check runs fixed Ruff `F` rules without fixes
+  over only the Python files changed by the current patch.
+- `logs --last --ai` and `logs --last --ai-json` render bounded
+  deterministic text and JSON views of the newest stored log without
+  changing it or executing project code.
+- Run and early-failure logs include evidence-only
+  `PYTHON_DISCOVERY_EVALUATION` telemetry for the current job. It records
+  explicit Python paths, audit action/output volume, targeting styles, and
+  relevant failure signals without estimating tokens or deciding whether a
+  project symbol index should be implemented.
+- Python discovery telemetry also aggregates reported file and search
+  matches, result-limit signals, and durations already present in bounded
+  audit results. Newly generated workspace policy covers conventional
+  `.venv`, `venv`, and `node_modules` trees at any depth; backup directories
+  remain owner-configured exclusions.
+- Best-effort whole-log redaction recognizes common sensitive Python
+  assignment identifiers, including prefixed passwords, API and access keys,
+  secret and private keys, tokens, and one-line `SECRET_KEY_FALLBACKS`
+  collections. Safe annotations, same-name references, and loader calls remain
+  visible; the redaction guarantee remains `NONE`.
+- Explicit project-local Django warning baselines classify known and new
+  W-class system-check warnings without changing the check exit status.
+- Optional owner-controlled `[execution].python_executable` selects the
+  interpreter for Python-based project checks and `{python}` local
+  profiles. Jobs cannot set it, and omitted configuration retains the
+  existing interpreter behavior.
+
+### Changed
+
+- AI handoffs use a deterministic capability hash, generated handoff trees
+  omit legacy backup and archive artifacts, and source audit output avoids
+  the confirmed token-name redaction false positive.
+
+### Fixed
+
+- `replace_symbol` execution now enters the same guarded file-change
+  transaction as other planned text modifications, including revalidation,
+  backups, checks, rollback, logging, and workspace comparison.
+
 ## 0.1.0a3 - 2026-08-23
 
 Second alpha release, focused on AI-facing planning diagnostics, legacy-project

{patchshuttle-0.1.0a3 → patchshuttle-0.1.0a4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: patchshuttle
-Version: 0.1.0a3
+Version: 0.1.0a4
 Summary: Local, auditable patch workflows for AI-assisted software development
 License-Expression: MIT
 License-File: LICENSE
@@ -23,6 +23,7 @@ Requires-Dist: filelock<4,>=3.30
 Requires-Dist: isort<7,>=5.13
 Requires-Dist: pydantic<3,>=2.6
 Requires-Dist: pyyaml<7,>=6
+Requires-Dist: ruff<1,>=0.16
 Requires-Dist: tomli<3,>=2; python_version < '3.11'
 Provides-Extra: dev
 Requires-Dist: build<2,>=1.2; extra == 'dev'
@@ -42,7 +43,7 @@ describes a small job, the user reviews and runs it locally, and PatchShuttle
 records the result for the next iteration.
 
 > [!IMPORTANT]
-> PatchShuttle is alpha software. Version `0.1.0a3` implements the complete
+> PatchShuttle is alpha software. Version `0.1.0a4` implements the complete
 > local v0.1 workflow. It executes bounded read-only
 > audits, approved patch transactions, and approved one-pass verification jobs
 > under a project lock. Patch jobs retain backups, run controlled checks, apply
@@ -53,8 +54,9 @@ records the result for the next iteration.
 > Trusted Publishing workflows are implemented. Every versioned release must
 > pass its own local qualification, GitHub-hosted Windows/Ubuntu matrix,
 > TestPyPI installation, production publication, and post-release smoke gates.
-> The immutable `0.1.0a2` evidence and documented ChatGPT end-to-end workflow
-> are retained in [docs/RELEASE.md](docs/RELEASE.md).
+> The immutable `0.1.0a2` and `0.1.0a3` qualification evidence, plus the
+> separately scoped `0.1.0a2` ChatGPT end-to-end workflow, are retained in
+> [docs/RELEASE.md](docs/RELEASE.md).
 
 > [!NOTE]
 > Compared with `0.1.0a2`, version `0.1.0a3` adds AI-facing planner diagnostics,
@@ -64,6 +66,16 @@ records the result for the next iteration.
 > cleanup, and guarded physical-line range actions with a canonical read-only
 > range hash.
 
+> [!NOTE]
+> The current development branch also contains unreleased work completed after
+> `0.1.0a3`: `search_context`, `read_symbol`, guarded `replace_symbol`, a fixed
+> Ruff F-only check, compact text and JSON AI log views, explicit Django warning
+> baselines, owner-controlled project-check Python selection, evidence-only
+> Python discovery telemetry, expanded Python secret redaction, nested
+> dependency-environment exclusions, and bounded declaration-only
+> `python_structure` discovery. These features are available from the source
+> branch and are not part of the immutable `0.1.0a3` PyPI artifacts.
+
 ## Design goals
 
 - Keep every change local and explicitly initiated by the user.
@@ -82,7 +94,7 @@ After publication, install the exact alpha release from PyPI:
 ```bash
 python -m venv .venv
 python -m pip install --upgrade pip
-python -m pip install "patchshuttle==0.1.0a3"
+python -m pip install "patchshuttle==0.1.0a4"
 ```
 
 For development from a local checkout:
@@ -95,7 +107,7 @@ To enable opt-in HTML template linting in an installed release, install the
 `html` extra:
 
 ```bash
-python -m pip install "patchshuttle[html]==0.1.0a3"
+python -m pip install "patchshuttle[html]==0.1.0a4"
 ```
 
 For a local development checkout, install both development and HTML extras:
@@ -158,6 +170,7 @@ initializing a workspace or reading protected generated files:
 patchshuttle capabilities
 patchshuttle schema
 patchshuttle explain replace_exact
+patchshuttle explain replace_symbol
 patchshuttle explain replace_range
 patchshuttle explain hash_range
 patchshuttle explain apply_diff
@@ -299,9 +312,14 @@ checks:
       modules: [email_client.views, email_client.urls]
 ```
 
-PatchShuttle runs the current interpreter with `manage.py shell -c` and
-internally generated import code. The YAML accepts only bounded dotted module
-identifiers, not an arbitrary expression or shell command.
+PatchShuttle runs the owner-selected project interpreter with
+`manage.py shell -c` and internally generated import code. The optional local
+`[execution] python_executable` value in `patches/patchshuttle.toml` may be an
+absolute path or a path relative to the workspace root. When it is omitted,
+PatchShuttle preserves the current `sys.executable` behavior. Jobs cannot set
+or override the interpreter, and PatchShuttle does not auto-detect a virtual
+environment. The YAML accepts only bounded dotted module identifiers, not an
+arbitrary expression or shell command.
 
 Load a job from disk with the same validation contract:
 
@@ -329,10 +347,11 @@ patchshuttle plan patches/examples/PATCH-EXAMPLE.psh.yaml
 patchshuttle plan patches/examples/PATCH-EXAMPLE.psh.yaml --diff
 ```
 
-The command displays the normalized job hash, sequential action dispositions,
-files and directories that would be created or modified, requested checks,
-Python formatting scope, a per-file isort/Black decision matrix, optional HTML
-lint scope, successful quality preflight records, protected-path result, backup
+The command displays the normalized job hash, effective project Python
+interpreter when one is used, sequential action dispositions, files and
+directories that would be created or modified, requested checks, Python
+formatting scope, a per-file isort/Black decision matrix, optional HTML lint
+scope, successful quality preflight records, protected-path result, backup
 destination template, rollback policy, and whether later execution will
 require confirmation.
 `--diff` also prints a bounded unified preview of the final resolved bytes
@@ -450,12 +469,18 @@ locations, the archived job copy, initial check results, formatter results,
 retained formatted-file states, final check results, audit observations, and a
 workspace comparison where applicable.
 
-An audit executes `tree`, `read`, literal `search`, `find_files`, `file_info`,
-SHA-256 `hash`, canonical `hash_range`, `git_status`, and `environment`.
-Traversal, file reads, match counts, and recorded output are bounded by local
-policy. Protected and ignored paths are skipped, source content appears only
-when explicitly requested by a `read` or `search` action, and a before/after
-inventory verifies that the audit did not modify the workspace.
+An audit executes `tree`, `read`, literal `search`, bounded `search_context`,
+Python `read_symbol`, `find_files`, `file_info`, SHA-256 `hash`, canonical
+`hash_range`, `git_status`, and `environment`. Traversal, file reads, match
+counts, and recorded output are bounded by local policy. Protected and ignored
+paths are skipped, source content appears only when explicitly requested by a
+content-reading audit action, and a before/after inventory verifies that the
+audit did not modify the workspace.
+
+`search_context` reports bounded physical-line windows around literal matches.
+`read_symbol` parses Python source without importing it, resolves exactly one
+class, function, method, or nested symbol, includes its decorators, and reports
+the canonical source range and SHA-256 used by `replace_symbol`.
 
 A verify job runs its controlled checks once, does not create a backup or run
 formatters, and compares the workspace before and after. A successful check
@@ -464,9 +489,9 @@ project checks can have external effects, PatchShuttle reports but does not
 automatically undo those changes.
 
 A patch plan may contain `create_directory`, `create_file`, `replace_exact`,
-`insert_before`, `insert_after`, `delete_exact`, `replace_range`,
-`delete_range`, `insert_at_line`, or `apply_diff` actions and applies the final
-bytes already computed by the planner.
+`replace_symbol`, `insert_before`, `insert_after`, `delete_exact`,
+`replace_range`, `delete_range`, `insert_at_line`, or `apply_diff` actions and
+applies the final bytes already computed by the planner.
 
 The optional line-range actions are strict guarded operations for cases where
 a current audit already established an exact physical range. Lines are 1-based
@@ -477,6 +502,13 @@ and SHA-256 over UTF-8 bytes. PatchShuttle never fuzzes, relocates, partially
 applies, or automatically shifts a stale range. Use an audit `hash_range` when
 a digest is more compact than repeating a large old block.
 
+A `replace_symbol` action names a dotted Python symbol, supplies the canonical
+SHA-256 returned by `read_symbol`, and provides complete replacement source.
+Planning resolves the symbol against the current sequential simulated file,
+rejects missing, duplicate, syntactically invalid, or stale targets, and does
+not use fuzzy relocation or automatic line shifting. Exact desired symbol
+content is an idempotent `NO_CHANGE`.
+
 For a supported plan, the implemented sequence is:
 
 1. validate the workspace lock file and acquire a non-blocking cross-platform
@@ -532,18 +564,21 @@ overwritten. Rollback validates retained originals before restoring them,
 refuses to follow symbolic links or remove foreign non-empty directories, and
 does not claim success when a tracked path cannot be restored.
 
-The internal check runner supports `compileall`, `pytest`, `unittest`, Django
-checks and tests, plain validated module imports, Django-aware validated module
-imports through `manage.py shell -c`, and locally configured profiles. It
-inherits the current process environment because project checks execute project
-code and PatchShuttle is not an operating-system sandbox. PatchShuttle overrides
-`PYTHONPYCACHEPREFIX` with a fresh temporary directory for each check and
-removes that directory afterward, keeping ordinary generated Python bytecode
-outside the workspace. A defensive cache ledger also handles `.pyc` files
-created inside newly added packages despite that override. Formatter order is
-fixed to isort then Black for protocol 1; local exact-path exclusions are
-resolved separately for each tool. Non-Python jobs skip formatting and do not
-repeat checks. Optional djLint uses a locally selected template profile,
+The internal check runner supports the fixed `ruff: {}` F-only check,
+`compileall`, `pytest`, `unittest`, Django checks and tests, plain validated
+module imports, Django-aware validated module imports through
+`manage.py shell -c`, and locally configured profiles. Ruff receives only the
+immutable ordered Python scope derived from current non-no-change patch actions;
+a job cannot supply paths, rules, fixes, or arbitrary Ruff arguments.
+Project checks inherit the current process environment because they execute
+project code and PatchShuttle is not an operating-system sandbox. PatchShuttle
+overrides `PYTHONPYCACHEPREFIX` with a fresh temporary directory for each check
+and removes that directory afterward, keeping ordinary generated Python
+bytecode outside the workspace. A defensive cache ledger also handles `.pyc`
+files created inside newly added packages despite that override. Formatter
+order is fixed to isort then Black for protocol 1; local exact-path exclusions
+are resolved separately for each tool. Non-Python jobs skip formatting and do
+not repeat checks. Optional djLint uses a locally selected template profile,
 targets only changed `.html` files, never reformats them, and triggers the same
 rollback path on failure. It reads content through stdin from an isolated
 temporary configuration root, so project djLint configuration cannot weaken
@@ -592,9 +627,12 @@ Every job run log contains all standard sections in a fixed order, including
 `LINT_HTML`, using `NOT_APPLICABLE` where a stage did not run, and ends with
 `PATCHSHUTTLE_AI_HANDOFF`. Check and formatter output is bounded by local
 policy, as is HTML linter output. Common password, token, API-key,
-authorization-header, and private-key shapes are masked when redaction is
-enabled. Redaction is best-effort and is not a guarantee that a log contains no
-secrets; review a log before uploading it to any AI service.
+authorization-header, private-key, and sensitive Python assignment shapes are
+masked when redaction is enabled. Python assignment matching includes prefixed
+passwords, API and access keys, secret and private keys, tokens, and one-line
+`SECRET_KEY_FALLBACKS` collections while preserving loader calls such as
+`config("SECRET_KEY")`. Redaction is best-effort and is not a guarantee that a
+log contains no secrets; review a log before uploading it to any AI service.
 
 Early `VALIDATION_FAILED` and `PLAN_FAILED` logs use a smaller fixed attempt
 format with `SUMMARY` and `PATCHSHUTTLE_AI_HANDOFF` sections. They record no
@@ -602,16 +640,162 @@ project changes, backup, job archive, or registry update. An explicitly
 declined reviewed plan continues to use the full job log and exact failed-job
 archive with result `USER_DECLINED`.
 
-Find the latest upload-friendly log:
+Django system-check warnings can use an explicit project-local baseline:
+
+```bash
+patchshuttle warnings
+patchshuttle warnings --add urls.W005
+patchshuttle warnings --remove urls.W005
+```
+
+Only explicit W-class IDs are accepted. A `django_check` retains its normal
+exit code and full log output while adding `known_warnings`, `new_warnings`,
+and complete new-warning details. If captured output is truncated,
+classification is reported as incomplete. Existing workspaces without the
+protected baseline file start with an empty baseline; `patchshuttle init`
+creates the missing scaffold file without overwriting existing state.
+Find or compactly render the latest upload-friendly log:
 
 ```bash
 patchshuttle logs --last
+patchshuttle logs --last --ai
+patchshuttle logs --last --ai-json
 ```
 
-This selects the newest recorded run, snapshot, handoff, or failure-attempt
-log. Commands that intentionally produce no artifact, including successful
-`validate`, successful `plan`, `version`, `capabilities`, `schema`, and
-`explain`, do not replace it.
+The default prints the path. `--ai` prints a bounded deterministic text view,
+while `--ai-json` prints the same selected data as compact JSON. These modes
+read the already stored log, preserve it unchanged, and never execute project
+code. They are mutually exclusive and retain useful audit outputs and failed
+check diagnostics within explicit bounds.
+
+Run and early validation/planning failure logs also expose
+`PYTHON_DISCOVERY_EVALUATION`. The section records only current-job evidence:
+explicit Python paths, executed audit action counts and bounded output size
+before whole-log redaction,
+declared text/symbol/line targeting, and relevant failure signals.
+`index_assessment: NOT_EVALUATED` is intentional: the report neither estimates
+AI token savings nor decides whether a project symbol index is needed.
+
+### Discover Python structure before targeted reads
+
+The read-only `python_structure` audit action collects a bounded declaration
+map before targeted reads. Start with the narrowest useful directory or one
+`.py` file because a broad package can produce a large audit result.
+
+A complete YAML audit job looks like this:
+
+```yaml
+protocol: 1
+project_id: PSH-0123456789ABCDEF
+id: AUDIT-PYTHON-STRUCTURE-001
+kind: audit
+actions:
+  - python_structure:
+      path: src/example
+      max_files: 50
+      max_symbols: 400
+      compact: true
+checks: []
+```
+
+`path` defaults to `.`, `max_files` defaults to `300`, `max_symbols`
+defaults to `2000`, and `compact` defaults to `false`. A file target must end
+in `.py`. Directory traversal respects the workspace's protected and ignored
+paths. The standard-library AST is used without importing or executing project
+code. Use `compact: true` for a navigational overview and omit it when detailed
+imports, parameters, decorators, and bases are required.
+
+Compact output begins with the same collection counts, parse errors, and limit
+signals as full output, followed by per-file records containing only counts and
+navigational symbol metadata:
+
+```text
+schema: patchshuttle.python_structure_collection.compact.v1
+path: src/example
+glob: *.py
+files_considered: 1
+files_parsed: 1
+parse_errors: 0
+imports_reported: 3
+symbols_available: 2
+symbols_reported: 2
+file_limit_reached: false
+symbol_limit_reached: false
+
+schema: patchshuttle.python_structure.compact.v1
+file: "src/example/service.py"
+imports: 3
+symbols: 2
+symbol: {"kind":"class","lines":[10,42],"qualified_name":"Service"}
+symbol: {"kind":"method","lines":[14,31],"qualified_name":"Service.run"}
+```
+
+With `compact: false`, the existing
+`patchshuttle.python_structure_collection.v1` and
+`patchshuttle.python_structure.v1` schemas remain unchanged and additionally
+include top-level import records, parameters, decorators, and bases.
+
+Use the reported qualified name for a smaller follow-up audit:
+
+```yaml
+actions:
+  - read_symbol:
+      path: src/example/service.py
+      symbol: Service.run
+```
+
+If a change is needed, copy the canonical SHA-256 returned by `read_symbol`
+into a guarded patch action:
+
+```yaml
+actions:
+  - replace_symbol:
+      path: src/example/service.py
+      symbol: Service.run
+      expected_sha256: 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
+      new_content: |
+        def run(self, value):
+            return value
+```
+
+The same audit can be constructed through the public Python adapter:
+
+```python
+from patchshuttle import Job
+from patchshuttle.actions import python_structure
+
+job = Job(
+    protocol=1,
+    project_id="PSH-0123456789ABCDEF",
+    id="AUDIT-PYTHON-STRUCTURE-001",
+    kind="audit",
+    actions=[
+        python_structure(
+            "src/example", max_files=50, max_symbols=400, compact=True
+        )
+    ],
+)
+```
+
+Full `python_structure` output reports declarations, top-level imports,
+parameters, decorators, bases, source ranges, parse errors, and explicit limit
+signals. Compact output retains file and symbol counts, symbol kinds, qualified
+names, ranges, parse errors, and limit signals. Both modes exclude source values,
+calls, references, inferred types, caching, persistence, and automatic patch
+scope.
+
+The same section reports aggregate Python file/search matches, whether file
+discovery or searches reached their declared result bounds, and total duration
+of executed Python-targeted audit actions. Newly initialized workspaces ignore
+and protect conventional `.venv`, `venv`, and `node_modules` trees at any
+depth. Existing `patches/patchshuttle.toml` files are never rewritten by
+`init`; owners must add nested environment patterns and project-specific
+backup paths to `project.ignored_paths` when needed.
+
+The command selects the newest recorded run, snapshot, handoff, or
+failure-attempt log. Commands that intentionally produce no artifact,
+including successful `validate`, successful `plan`, `version`,
+`capabilities`, `schema`, and `explain`, do not replace it.
 
 Inspect all registered jobs or one job ID:
 
@@ -687,17 +871,64 @@ python -m coverage report --fail-under=100
 python -m build
 python -m twine check dist/*
 python tools/release_checks.py dist
-python tools/wheel_smoke.py dist/patchshuttle-0.1.0a3-py3-none-any.whl --version 0.1.0a3
+python tools/wheel_smoke.py dist/patchshuttle-0.1.0a4-py3-none-any.whl --version 0.1.0a4
 ```
 
-The `0.1.0a2` alpha release passed the required Ubuntu and Windows
-compatibility matrix, TestPyPI qualification, production PyPI Trusted
-Publishing, and a clean post-release installation. The exact release, workflow
-links, artifact hashes, and ChatGPT end-to-end evidence are recorded in
-[docs/RELEASE.md](docs/RELEASE.md). Follow that guide in order for future
-releases, including `0.1.0a3`, and append separate immutable evidence after
-every gate completes. Rerun CI after every release-candidate change.
+The `0.1.0a2` and `0.1.0a3` alpha releases each passed the required Ubuntu and
+Windows compatibility matrix, TestPyPI qualification, production PyPI Trusted
+Publishing, and a clean post-release installation. Their exact release and
+workflow links and artifact hashes, together with the separately scoped
+`0.1.0a2` ChatGPT end-to-end evidence, are recorded in
+[docs/RELEASE.md](docs/RELEASE.md). Follow that guide in order for every future
+release and append separate immutable evidence after each gate completes.
+Rerun CI after every release-candidate change.
+
+## Python architecture policy
+
+PatchShuttle can stop an AI-generated patch from making an already crowded Python
+codebase worse. The policy is local owner configuration, not a job option, and is
+enabled by default for the fixed `modular-monolith` / `package-by-feature`
+profile in `ratchet` mode.
+
+```toml
+[architecture]
+enabled = true
+profile = "modular-monolith"
+organization = "package-by-feature"
+mode = "ratchet"
+exclude = ["**/migrations/**", "**/generated/**"]
+max_report_items = 50
+
+[architecture.module]
+warning_lines = 500
+max_lines = 1000
+
+[architecture.package]
+warning_python_files = 15
+max_python_files = 25
+
+[architecture.patch]
+warning_new_python_files = 5
+max_new_python_files = 10
+warning_new_packages = 1
+max_new_packages = 3
+```
 
+The planner evaluates its in-memory result before touching the workspace. A file
+already above a limit may stay the same size or shrink; making it larger is a new
+regression and is blocked at the hard limit. The same rule applies to direct
+Python-file counts in touched directories. Patch-wide file and package budgets
+catch the kind of broad root-level file growth that is difficult to review.
+
+Plans and stored logs expose the profile, status, measured counts, bounded
+findings, and whether the report was truncated. `ARCH001`/`ARCH002` cover module
+size, `ARCH010`/`ARCH011` package size, and `ARCH020`/`ARCH021` patch growth.
+Warnings remain visible but do not block execution; errors fail planning before
+confirmation, backup, or project checks.
+
+This is deliberately a structural guard. It does not infer feature boundaries,
+build dependency or call graphs, detect cycles, judge semantic responsibility,
+reorganize files, or change the patch scope selected by the AI.
 ## Manual workflow
 
 The implemented local cycle is: