yam-flow 2.3.2 → 2.5.0

package/AGENTS.md

@@ -24,18 +24,20 @@
 
 ## Latest Update Trigger
 
-When the user says exactly `yam 최신 업데이트 사항 반영해줘` in a chat for this project, treat that phrase as explicit authorization to update the globally installed package and refresh the installed yam skills.
+When the user says exactly `yam 최신 업데이트 사항 반영해줘` in a chat for this project, treat that phrase as explicit authorization to check and apply reviewed updates for yam, Scrapling, and Insane Search.
 
 Run these commands sequentially from the yam project directory:
 
 ```bash
-npm install -g yam-flow
-yam install
+yam update check --json
+yam update apply --all --json
 yam version
 yam status
 ```
 
 - Request any environment approval required for network access or writes outside the workspace.
 - Stop at the first failed command, report the blocker, and do not claim that later steps ran.
-- After all four commands pass, report the installed version and skill status, then tell the user that Codex can be restarted to reload the updated skills.
+- Treat that exact phrase as one-time authorization for this run only; do not create an unattended updater, hook, or automation.
+- Never edit `.codex/plugins/cache` directly. If Insane Search cannot be updated in place through the official Codex CLI, report `manual_plugin_update_required` and stop.
+- After all four commands pass, summarize every component receipt, report the installed yam version and skill status, then tell the user that Codex can be restarted to reload updated skills/plugins.
 - Do not restart Codex automatically.

package/CHANGELOG.md

@@ -2,6 +2,56 @@
 
 All notable changes to yam-flow are documented here.
 
+## v2.5.0 - 2026-07-30
+
+### Added
+
+- Added read-only `yam update check [--json]` for yam, Scrapling, and Insane Search with official registry/manifest versions and source-revision metadata.
+- Added explicitly authorized component/all update application with per-component receipts, a concurrent-run lock, exact stable versions, verification evidence, and rollback guidance.
+- Added isolated Scrapling version environments with `pip check`, HTTP extraction, browser extraction, and atomic executable switching; the previous environment remains available for rollback.
+- Added deterministic temp-only updater smoke coverage, including receipt-failure symlink rollback, manual Codex plugin fallback, strict no-remove/cache behavior, and yam-last `--all` ordering.
+
+### Improved
+
+- The build now preserves executable mode on `dist/bin/yam.js`, and dist freshness fails if a global npm install would expose a non-executable `yam` target.
+
+### Security
+
+- Insane Search updates pin the official manifest to the exact observed Git revision and use only official Codex CLI commands. A missing safe in-place path returns `manual_plugin_update_required`; an unreadable post-upgrade state fails closed before `add`; yam never removes the plugin first or writes `.codex/plugins/cache` directly.
+- External commands use argument arrays without a shell, version/component allowlists, redacted captured output, and fail-closed apply locking.
+- Existing Scrapling links require a matching yam ownership marker, and an unwritable receipt stops `--all` before any later component mutation.
+
+### Verification
+
+- `npm run external-updates-smoke`
+- `npm run typecheck`
+- `npm run verify:self`
+- `npm run cli-smoke`
+- `npm run dist:freshness`
+- `npm run package-boundary:check`
+
+## v2.4.0 - 2026-07-27
+
+### Added
+
+- Added ownership-aware skill replacement: an existing active skill is replaced only when the previous yam receipt and complete regular-file inventory and content hashes prove yam ownership, unless the operator explicitly names that skill with `--replace-user-skill`.
+- Added a central CLI help contract so `--help` and `-h` return usage before any command dispatch, including install and uninstall.
+- Added deterministic smoke coverage for user-owned conflicts, explicit replacement, prior-version upgrades, local drift, safe uninstall and rollback, preserved legacy/mirror entries, and help-path state immutability.
+- Added `yam detect [dir] --json` changed-file verification recommendations for dirty, staged, and untracked Git paths. Each advisory records its matched rule, confidence, reason, fallback, and source files; its command plan deduplicates available project commands while retaining rule/file provenance, and unavailable Git scope cannot masquerade as a clean project.
+- Added a deterministic packaged-hook latency and read-only regression smoke for zero, one, and many changed files, including the eight-file prompt cap and bounded Study Note `Stop` behavior.
+
+### Improved
+
+- `yam uninstall` now requires a valid receipt and matching active skill hashes, then removes the verified set transactionally. User-owned, locally modified, legacy, retired, and Codex mirror entries that cannot be proven yam-owned are preserved.
+- Install ownership is checked before staging and again immediately before mutation so conflicts fail closed without replacing the existing skill or receipt.
+- `yam doctor` reports safely preserved legacy/retired/mirror entries as informational unproven entries instead of failing with an ineffective reinstall recommendation.
+- `yam cleanup scan` now includes a bounded, advisory-only AGENTS/SKILL directive-duplication budget with exact normalized matching, digests, redacted limited previews, and file/line provenance. It remains non-destructive and never becomes a hard gate.
+- Study Note check, prompt, and `Stop` paths now preserve Git change-scope availability. A non-Git or failed Git probe reports partial truth and a manual-inspection warning instead of silently claiming that no artifacts changed.
+
+### Verification
+
+- Passed TypeScript typecheck, clean build, hook latency/read-only smoke, the 174-file install/uninstall transaction smoke, packaged CLI smoke, self-verification, forbidden-name scan, distribution freshness, package-boundary validation, and diff whitespace checks.
+
 ## v2.3.2 - 2026-07-21
 
 ### Added

package/COMMANDS.md

@@ -143,6 +143,19 @@ yam measure quick --files 3 --commands 1 --report-lines 5 --seconds 40
 yam template tuning
 ```
 
+## External Updates
+
+```bash
+yam update check
+yam update check --json
+yam update apply --component scrapling --json
+yam update apply --component insane-search --json
+yam update apply --component yam --json
+yam update apply --all --json
+```
+
+`yam update check` is read-only. Every `apply` needs explicit authorization, uses a concurrent-run lock, records a component receipt, and stops on failed/manual results. `--all` runs yam last. Insane Search is never remove-first and `.codex/plugins/cache` is never edited directly.
+
 ## Tool Doctor / Proof / Safety
 
 Read-only readiness and evidence helpers. These do not install, deploy, query databases, or run verification by themselves.
@@ -152,6 +165,7 @@ yam tools doctor /path/to/project
 yam tools doctor /path/to/project --json
 yam context pressure /path/to/project --json
 yam cleanup scan /path/to/project --json
+yam detect /path/to/project --json
 yam study-note check /path/to/project --text "Study Note: The hook handler's role is final-report validation. It runs at Codex Stop. Before it only reminded; after it requests one correction pass. Expected behavior is a complete note. Structure insight: decision=block creates a continuation prompt. Verification checked the packaged CLI. Limits: a restarted real task remains to be observed." --json
 yam loop report --route quick --intent "fix release readiness" --stage "inspect:passed:read release report" --evidence "typecheck passed" --evidence-level local --evidence-stamp "sha256:release-report" --touched-file src/bin/yam.ts --read-file README.md --verified-file scripts/cli-smoke.mjs --skipped-check "npm publish skipped by design" --stop-condition "stop after readiness evidence is recorded" --resume-hint "rerun release report after npm auth refresh" --readiness-state blocked --covered-requirement "release report is read-only" --uncovered-requirement "npm auth verified" --blocked-kind auth_blocked --failure-cause auth_token_invalid --safe-retry "retry after npm whoami succeeds" --recovery-hint "refresh npm auth, then rerun readiness checks" --fix-first-item "npm auth must be verified before publish" --remaining-task "rerun release report after auth refresh" --recommended-direction "fix npm auth first, then publish manually" --implementation-note "keep loop report read-only" --why-this-next "auth blocks public release claims" --blocked-by "npm whoami E401" --owner-route deep --owner-scope "release readiness only" --scope-owner deep --side-effect "no publish attempted" --avoidance-note "do not retry publish before npm auth is proven" --issue-code "src/bin/yam.ts release report" --issue-role "summarizes release readiness without publishing" --issue-symptom "npm auth failure needs clearer next action" --changed-code "yam loop report" --changed-role "records loop evidence and learning note" --change-summary "added a read-only loop artifact" --why-important "it helps users learn what changed without overclaiming verification" --learning-note "fix blockers before claiming done" --json
 yam proof /path/to/project
@@ -213,7 +227,8 @@ Supported report shapes:
 - Ueye design brief/anti-slop: brief dimensions, constraints, invented metrics, placeholder copy, generic visuals, and custom P0 blockers.
 - Ueye deep visual review: acceptance criteria, touched/read/verified files, skipped checks, residual risks, stop condition, resume hint, design-system evidence, implementation evidence, and state matrix.
 - Context pressure: project pack age/size, memory summary, dirty scope, scripts, instruction surfaces, and route hint.
-- Cleanup scan: advisory-only list of confusing surfaces with destructive=false.
+- Changed-file verification detector: dirty/staged/untracked Git paths grouped into advisory rules with confidence, reason, fallback, source files, and available project commands; the command plan is deduplicated while retaining rule/file provenance, Git-unavailable scope stays low confidence, and no recommendation is executed.
+- Cleanup scan: advisory-only list of confusing surfaces with `destructive=false`, plus a bounded exact-normalized AGENTS/SKILL cross-surface duplication budget with file/line provenance.
 
 ## Hooks
 

package/README.md

@@ -97,7 +97,7 @@ yam hook enable study-note --global
 yam hook status --global
 ```
 
-The profile installs `UserPromptSubmit` and `Stop` handlers. At `Stop`, yam checks the latest assistant message with the same read-only Study Note Guard used by `yam study-note check`. If changed artifacts lack the required role, execution point, before/after, expected behavior, syntax/structure, verification, limits, or relevant architecture hygiene, Codex receives one correction prompt. A second failed check warns but does not loop forever. The hook never writes the Study Note or runs verification for the agent.
+The profile installs `UserPromptSubmit` and `Stop` handlers. At `Stop`, yam checks the latest assistant message with the same read-only Study Note Guard used by `yam study-note check`. If changed artifacts lack the required role, execution point, before/after, expected behavior, syntax/structure, verification, limits, or relevant architecture hygiene, Codex receives one correction prompt. A second failed check warns but does not loop forever. If Git change scope is unavailable, the guard reports partial truth and keeps a manual-inspection warning visible instead of claiming the project is clean. The hook never writes the Study Note or runs verification for the agent.
 
 `yam hook status` reports stale or missing command targets as `broken` and exits nonzero. Run `yam hook enable <profile> --global` again to create a timestamped backup, preserve unrelated hooks, and migrate that profile to the current installed path and event coverage. Restart Codex or open a new task after changing hooks.
 
@@ -110,16 +110,74 @@ cd ~/Documents/Codex/tools/yam
 yam install
 ```
 
-`yam install` stages every managed skill plus the shared `references/` directory, verifies the complete staged file manifest with SHA-256, and only then replaces the active set in `~/.agents/skills/`. If any commit or post-install verification step fails, yam restores the previous managed skill set and receipt.
+`yam install` first checks ownership of every existing active skill. A skill is replaced automatically only when the previous yam receipt names it and its complete regular-file inventory and content hashes still match the receipt. The installer then stages every managed skill plus the shared `references/` directory, verifies the complete staged file manifest with SHA-256, and replaces the verified active set in `~/.agents/skills/`. If any commit or post-install verification step fails, yam restores the previous managed skill set and receipt.
+
+If an existing active directory is user-owned, locally modified, or cannot be proven yam-owned, installation stops before staging and preserves it. After reviewing the files, replace only the intended conflicting name explicitly:
+
+```bash
+yam install --replace-user-skill quick
+```
+
+Repeat `--replace-user-skill <name>` for each reviewed conflict. Unproven legacy, retired, and Codex mirror directories are preserved rather than deleted by name.
 
 A successful install writes `~/.agents/skills/.yam-flow-install-receipt.json`. The receipt records the package version, source identity, install timestamp, destination, managed skill inventory, and per-file hashes. `yam status` independently recomputes source and installed hashes, so a stale package install, local file change, unexpected file, or modified receipt is reported as drift.
 
+The receipt is local integrity provenance, not a cryptographic signature or a security boundary against another process running as the same OS user. An actor that can rewrite both the receipt and every recorded file can forge local ownership evidence.
+
 An ordinary install error rolls back automatically. An abrupt process or machine termination can leave `.yam-flow-install.lock` and a hidden transaction directory; `yam status` surfaces the unfinished transaction and yam refuses another mutation while recovery state remains. Remove the lock only after confirming no install is running, and preserve any transaction directory until the previous installation state has been inspected.
 
 Manual copying can still make a skill loadable, but it intentionally remains unverified: without a matching receipt, `yam status` exits non-zero and recommends `yam install`.
 
 Restart Codex after installing so skills reload.
 
+## External Updates
+
+Update checks are read-only and never authorize a mutation by themselves:
+
+```bash
+yam update check
+yam update check --json
+```
+
+Apply requires an explicit component or `--all`:
+
+```bash
+yam update apply --component scrapling
+yam update apply --component insane-search
+yam update apply --component yam
+yam update apply --all --json
+```
+
+`--all` processes Scrapling, then Insane Search, then yam so the running updater replaces itself last. It stops at the first failed or manual-only component. Each attempt writes a component receipt under `~/.local/state/yam/update-receipts/` with the old/new version, source revision, checks, side effects, outcome, and rollback guidance. A lock prevents concurrent apply runs.
+
+Scrapling updates install an exact stable PyPI version into a new versioned virtual environment. `pip check`, ordinary HTTP extraction, and headless browser extraction must pass before the executable symlink changes atomically; the previous environment is retained. An existing environment must have a matching `.yam-scrapling-install.json` ownership marker before yam will replace its link. Insane Search uses only the official `codex plugin` commands. yam never edits `.codex/plugins/cache`, never removes the plugin first, and returns `manual_plugin_update_required` when the installed Codex CLI cannot update it safely in place.
+
+There is no unattended updater. The exact project trigger in `AGENTS.md` grants one-time authorization for a checked `--all` run, and Codex is never restarted automatically.
+
+### Scrapling Usage
+
+The isolated executable can be used directly from the shell:
+
+```bash
+scrapling --version
+scrapling extract get https://example.com example.txt --css-selector h1
+scrapling extract stealthy-fetch https://example.com example-browser.txt --css-selector h1 --headless
+```
+
+From Python inside the installed environment:
+
+```python
+from scrapling.fetchers import Fetcher, StealthyFetcher
+
+page = Fetcher.get("https://example.com")
+print(page.css("h1::text").get())
+
+browser_page = StealthyFetcher.fetch("https://example.com", headless=True)
+print(browser_page.css("h1::text").get())
+```
+
+Use `Fetcher` for ordinary HTTP pages and `StealthyFetcher` only when a real browser or stronger anti-bot behavior is needed; the browser path is heavier.
+
 ## Uninstall
 
 ```bash
@@ -127,6 +185,8 @@ cd ~/Documents/Codex/tools/yam
 yam uninstall
 ```
 
+`yam uninstall` verifies the receipt, complete active inventory, and current file hashes before deleting anything. A missing receipt or locally modified active skill blocks the whole uninstall. Verified active skills and the receipt are removed transactionally; unproven legacy, retired, and Codex mirror entries remain untouched.
+
 No hooks, automations, or global config files are installed.
 
 ## Manage
@@ -140,6 +200,7 @@ yam tools doctor /path/to/project
 yam tools doctor /path/to/project --json
 yam context pressure /path/to/project --json
 yam cleanup scan /path/to/project --json
+yam detect /path/to/project --json
 yam study-note check /path/to/project --text "Study Note: ..." --json
 yam proof /path/to/project
 yam proof write /path/to/project --route quick --truth verified --command "npm run verify:self: pass"
@@ -161,7 +222,6 @@ yam mission gate --expected-thread reviewer-1 --receipt .yam/mission/reviewer-1.
 yam loop report --route quick --intent "fix release readiness" --stage "inspect:passed:read release report" --evidence "typecheck passed" --evidence-level local --evidence-stamp "sha256:release-report" --touched-file src/bin/yam.ts --read-file README.md --verified-file scripts/cli-smoke.mjs --skipped-check "npm publish skipped by design" --stop-condition "stop after readiness evidence is recorded" --resume-hint "rerun release report after npm auth refresh" --readiness-state blocked --covered-requirement "release report is read-only" --uncovered-requirement "npm auth verified" --blocked-kind auth_blocked --failure-cause auth_token_invalid --safe-retry "retry after npm whoami succeeds" --recovery-hint "refresh npm auth, then rerun readiness checks" --fix-first-item "npm auth must be verified before publish" --remaining-task "rerun release report after auth refresh" --recommended-direction "fix npm auth first, then publish manually" --implementation-note "keep loop report read-only" --why-this-next "auth blocks public release claims" --blocked-by "npm whoami E401" --owner-route deep --owner-scope "release readiness only" --scope-owner deep --side-effect "no publish attempted" --avoidance-note "do not retry publish before npm auth is proven" --issue-code "src/bin/yam.ts release report" --issue-role "summarizes release readiness without publishing" --issue-symptom "npm auth failure needs clearer next action" --changed-code "yam loop report" --changed-role "records loop evidence and learning note" --change-summary "added a read-only loop artifact" --why-important "it helps users learn what changed without overclaiming verification" --learning-note "fix blockers before claiming done" --json
 yam release report --json
 yam safety "supabase db reset"
-yam detect /path/to/project
 yam pack /path/to/project
 yam hook status --global
 yam hook enable lite --global
@@ -246,7 +306,8 @@ These shapes keep reports machine-readable without turning normal work into a he
 - Publish blocker evidence: release reports classify common npm auth, permission, OTP, immutable-version, cache, registry, and tarball failures into safe next actions.
 - Publish readiness evidence: release reports run read-only registry/auth/version probes, redact account/token details, keep npm publish outside the report, and include a `yam.release-readiness-receipt.v1` basis for the readiness judgment.
 - Context pressure: `yam context pressure` explains when project context is getting stale, broad, or confusing enough to summarize, refresh, narrow scope, or deepen route.
-- Cleanup scan: `yam cleanup scan` is read-only and advisory; it reports confusing hooks, local skill folders, stale traces, or old proof artifacts without deleting anything.
+- Changed-file verification detector: `yam detect [dir] --json` reads dirty, staged, and untracked Git paths and returns advisory `matched_rule`, `confidence`, `reason`, `fallback`, source files, and only commands that exist in the project. Its deduplicated command plan retains the contributing rules and files, but never runs the commands. If Git status is unavailable, it reports a low-confidence fallback instead of claiming the project is clean.
+- Cleanup scan: `yam cleanup scan` is read-only and advisory; it reports confusing hooks, local skill folders, stale traces, or old proof artifacts without deleting anything. Its bounded AGENTS/SKILL duplication budget scans at most 24 files and 128 KB per file, reports exact normalized cross-surface directive matches with redacted previews and provenance, and never becomes a hard gate.
 - Benchmark optimization loop lite: use `yam measure <route>` and `yam template tuning` to record baseline, one route wording change, rerun result, keep/revert decision, and stop condition.
 - Structured diagnostic next action: every doctor/release diagnostic should say what was observed, what evidence supports it, what to do next, owner route, severity, and truth status.
 - Ueye review continuity/comparison report: `$ueye` run reports can link a previous review, current evidence, comparison delta, design quality result, and next action.

package/ROADMAP.md

@@ -51,8 +51,8 @@ Tasks:
 
 - Read `package.json` scripts.
 - Detect common commands: typecheck, lint, test, build. Done.
-- Map changed files to likely checks. Partial: route-level suggestions added.
-- Keep this advisory, not mandatory.
+- Map changed files to likely checks. Done: dirty, staged, and untracked paths now map to focused rules with confidence, reason, fallback, and available commands.
+- Keep this advisory, not mandatory. Done.
 
 ### 4. Token Budget Reporter
 
@@ -97,6 +97,7 @@ Tasks:
 - Add symlink-friendly local install.
 - Add version command. Done.
 - Add staged transactional install, rollback, receipt, and drift-aware status. Done.
+- Add checked component updates for yam, Scrapling, and Insane Search with explicit authorization, receipts, and rollback guidance. Done.
 - Consider npm packaging only after route behavior stabilizes.
 
 ### 8. Scout / Research Workflow