@mohammadhprp/system-prompt 0.11.1 → 0.11.2

package/framework/skills/tlc-spec-driven/references/validate.md

@@ -0,0 +1,355 @@
+# Execute: Validate & Verify
+
+**Goal**: Verify implementation meets spec AND coding principles. This is NOT a separate phase - verification is part of every task's completion within Execute.
+
+**Three levels of verification:**
+
+1. **Per-task verification (always, author self-check):** After implementing each task, verify its "Done when" criteria before committing. This is mandatory and automatic. The implementer runs it.
+
+2. **Feature-level validation (independent Verifier sub-agent, always-on, never prompted):** After all tasks for a feature (or priority group) are done, validation runs automatically - the orchestrator dispatches a **fresh Verifier sub-agent** (see [sub-agents.md](sub-agents.md)). Do NOT ask the user whether to run it; it is the safety net, not an opt-in. User interaction is limited to interactive UAT (for user-facing features) and acting on a FAIL verdict ("fix these gaps now?"). The Verifier:
+   - Runs **read-only** over the real implementation and tests - mutations run in a scratch/throwaway state only (see Discrimination Sensor section)
+   - Scopes coverage to the feature's **git diff surface** (not the full repository)
+   - Re-derives coverage independently using **evidence-or-zero**: every AC must be traced to a `file:line` + assertion expression; a criterion with no `file:line` citation counts as NOT covered
+   - Runs the **spec-anchored outcome check** and the **discrimination sensor** (both described below)
+   - Writes `.specs/features/[feature]/validation.md` with the full evidence report
+   - Returns a compact verdict + ranked gap list to the orchestrator in chat
+   - Gaps become **fix tasks** routed back to an implementer; re-verification follows with a maximum of **3 fix→re-verify iterations** before escalating to the user
+
+3. **Interactive UAT (for user-facing features only):** The feature has complex user-facing behavior where human judgment matters (UI flows, interaction patterns, visual design). For backend-only or infrastructure work, automated checks are sufficient.
+
+**Trigger for explicit validation:** "Validate", "verify work", "UAT", "test with me", "walk me through it"
+
+---
+
+## Process
+
+### 1. Check Completed Tasks
+
+Go through tasks.md:
+
+- [ ] All tasks marked done?
+- [ ] Any blocked or partial?
+
+### 2. Spec-Anchored Acceptance Criteria Check
+
+For each acceptance criterion in `spec.md`, the Verifier re-derives the **spec-defined expected outcome** and confirms the test's actual assertion matches it:
+
+```markdown
+### P1: [Story Title]
+
+**Acceptance Criteria**:
+
+| Criterion (WHEN X THEN Y) | Spec-defined outcome | `file:line` + assertion expression | Result |
+| ------------------------- | -------------------- | ---------------------------------- | ------ |
+| WHEN [X] THEN [Y]         | [precise value/state from spec] | `path/to/test.ts:42` - `expect(result.field).toBe(expected)` | ✅ PASS / ❌ GAP / ⚠️ Spec-precision gap |
+```
+
+**Rules:**
+
+- Where the spec defines a precise outcome (specific status code, field value, error message, state), the test assertion MUST target that exact outcome - not just that an assertion exists.
+- Where the spec does NOT define a precise outcome, mark as **⚠️ Spec-precision gap** and flag it in the report. Do NOT silently pass a vague assertion.
+- Evidence-or-zero: a criterion with no `file:line` citation counts as NOT covered.
+
+### 3. Check Edge Cases
+
+From spec.md edge cases:
+
+- [ ] [Edge case 1] handled correctly
+- [ ] [Edge case 2] handled correctly
+
+### 4. Run Build-Level Gate Check (MANDATORY)
+
+Run the Build-level gate check from the **Gate Check Commands** section in tasks.md. This is NOT optional.
+
+1. Run: `[Build gate command from the Gate Check Commands section in tasks.md]`
+2. Non-zero exit code = STOP. Do not proceed to Code Quality Check.
+3. Record results:
+   - Total test count: [N]
+   - Passed: [N]
+   - Failed: [list]
+   - Skipped: [list - each skip must be justified]
+
+**Test Integrity Check:**
+
+- Compare current test count against the count before this feature was implemented
+- If test count DECREASED: investigate why. Tests should only be deleted with explicit justification.
+- If assertions were weakened (less specific than before): flag as potential regression
+
+### 5. Discrimination Sensor (MANDATORY - always runs after gate check passes)
+
+The sensor provides the empirical guarantee that the tests can actually detect regressions. It runs in a scratch/throwaway state - the real working tree is never modified.
+
+**How it works:**
+
+1. **Prepare an isolated scratch.** Never mutate the real worktree. Choose one:
+   - Preferred: a temporary git worktree (`git worktree add <scratch-path> HEAD`), mutate and run tests there, then `git worktree remove --force <scratch-path>`.
+   - Fallback (no git / worktree unavailable): copy only the affected file(s) to a temp directory, mutate the copies, point the test runner at those copies (or restore originals from the copies' backups), then delete the temp directory.
+   - **Forbidden:** `git stash` / `git stash pop`. A stash records state *before* the mutation; popping it does not reverse a mutation applied afterward, and on a clean tree `git stash` creates no entry at all - so the fault is left in the real worktree.
+2. **Capture a baseline.** Record `git status --porcelain` (or equivalent) of the real worktree *before* any sensor work. It must be unchanged after cleanup.
+3. **Inject a behavior-level fault** into the scratch copy of the new code introduced by this feature. Choose a mutation proportional to the code's risk:
+   - Flip a boolean condition (`if (x)` → `if (!x)`, `>` → `>=`)
+   - Change a return value (return a wrong status code, wrong field, zero instead of a computed value)
+   - Off-by-one (shift a loop bound, change a slice index)
+   - Remove a required side effect (delete a method call that the spec requires)
+4. **Run the tests** that cover the mutated code (against the scratch). Use the Quick or Full gate command from tasks.md.
+5. **Confirm the mutant is killed** (tests FAIL). Discard the scratch (remove worktree or delete temp copies).
+6. **Verify isolation.** Re-run `git status --porcelain` on the real worktree and confirm it matches the baseline from step 2. If it differs, STOP - restore the real tree before continuing, and treat the sensor run as invalid.
+7. **If a mutant survives** (tests still pass after the fault), the tests are not discriminating for that behavior - add a fix task to strengthen the assertion.
+
+**Tiering (proportional, not optional):**
+
+| Context | Sensor depth |
+| ------- | ------------ |
+| Default (all features) | Lightweight fault-injection: 1-3 targeted behavior-level mutations per feature, focused on the highest-risk new code |
+| P0 / critical paths (payment, auth, data integrity) | Full mutation run: use language-appropriate mutation tooling if available (e.g., Stryker, mutmut, cargo-mutants, pitest); otherwise increase the number of manual fault-injection mutations to ≥5 covering all branches |
+
+**Stack-agnostic:** The sensor targets behavior-level semantics (what the code does), not a specific tool. Any language, any framework.
+
+**Report:** Record killed/survived for each mutation attempt. Surviving mutants → create fix tasks before marking the feature done.
+
+### 6. Code Quality Check (MANDATORY)
+
+For each changed file, verify against [coding-principles.md](coding-principles.md):
+
+| Check                                | Pass? |
+| ------------------------------------ | ----- |
+| No features beyond what was asked    |       |
+| No abstractions for single-use code  |       |
+| No unnecessary "flexibility" added   |       |
+| Only touched files required for task |       |
+| Didn't "improve" unrelated code      |       |
+| Matches existing patterns/style      |       |
+| Would senior engineer approve?       |       |
+| Tests map to acceptance criteria and are non-shallow (spot-check one story) | |
+| Spec-anchored outcome check: each test's asserted value matches the spec-defined outcome (or gap flagged) | |
+| Per-layer Coverage Expectation met: domain logic has 1:1 AC mapping; routes/e2e cover happy + edge + error paths for every route in scope | |
+| Every test in scope maps to a spec AC, listed edge case, or Done-when criterion (no unclaimed tests) | |
+| Documented project quality/testing guidelines followed (cite guideline file, or "none - strong defaults applied") | |
+
+❌ Any "No"? → Fix before marking complete.
+
+### 7. Interactive UAT (if user-facing feature)
+
+For each testable deliverable, present one test at a time:
+
+```
+Test [N]: [Test Name]
+
+Expected: [What should happen - specific and observable]
+
+→ Does this work? Describe what you see.
+```
+
+Wait for user response:
+
+| User says                      | Interpret as            |
+| ------------------------------ | ----------------------- |
+| "yes", "pass", "works", "next" | ✅ Pass                 |
+| "skip", "can't test", "n/a"    | ⏭️ Skip                 |
+| Anything else                  | ❌ Issue - log verbatim |
+
+**Severity inference (never ask the user for severity):**
+
+| User description contains               | Inferred severity |
+| --------------------------------------- | ----------------- |
+| crash, error, exception, fails, broken  | Blocker           |
+| doesn't work, wrong, missing, can't     | Major             |
+| slow, weird, off, minor, small          | Minor             |
+| color, font, spacing, alignment, visual | Cosmetic          |
+| (unclear)                               | Major (default)   |
+
+### 8. Generate Fix Plans (if issues found)
+
+For each issue found during UAT or from the Verifier:
+
+1. **Diagnose** - Analyze the codebase to find root cause
+2. **Create fix task** - Write a task definition with:
+   - What: The specific fix
+   - Where: File paths
+   - Verify: How to prove the fix works
+   - Done when: Acceptance criteria for the fix
+3. **Present fix plan** - Show all fix tasks to user for approval
+
+Fix tasks follow the same format as regular tasks and can be executed with the implement phase.
+
+**Guardrail:** Maximum 3 diagnostic iterations per issue. If root cause isn't found after 3 attempts, flag for human investigation. The same 3-iteration bound applies to the Verifier's fix→re-verify cycle: if gaps persist after 3 rounds, escalate to the user rather than continuing to loop.
+
+### 9. Write Validation Report File + Return Chat Summary (MANDATORY)
+
+After all checks complete, the Verifier MUST:
+
+1. **Write the persisted report** to `.specs/features/[feature]/validation.md` (see template below). This file is the evidence artifact - it survives the session and can be referenced by CI, reviewers, or future agents.
+2. **Return a compact summary in chat** to the orchestrator (see Compact Chat Summary section below). The orchestrator surfaces it to the user and routes any ranked gaps to fix tasks.
+
+**Deterministic backing (run it, do not eyeball it).** After writing the report, run `python3 <skill-dir>/scripts/validate_state.py <feature>`. It confirms the report is real - present, verdict filled to PASS, and backed by at least one `file:line` evidence citation - so a missing, hollow, placeholder, or FAIL report cannot slip through as done. A non-zero exit means the feature is NOT done: repair the report or route the FAIL gaps to fix tasks, then re-run. This is the closing gate of Execute and runs automatically, the same way the lessons layer runs at distillation; it is never a manual step. If no code-execution tool is available, confirm the same by reading `validation.md`.
+
+### 10. Distill Lessons (MANDATORY when validation.md has signal)
+
+This is the closing action of validation - not a separate phase. Immediately after the report is written, turn its grounded failures into reusable, project-local guidance by following [lessons.md](lessons.md). In short: for each surviving mutant, spec-precision gap, failed/uncovered AC, or `// SPEC_DEVIATION`, record one terse general lesson via `python3 <skill-dir>/scripts/lessons.py add` (the script enforces grounding and owns all bookkeeping). A clean PASS with no signal → record nothing. Run the self-check: if there was signal but no lesson was recorded, say so in chat. See [lessons.md](lessons.md) for the exact commands, phrasing rules, scope discipline, and the no-script fallback.
+
+---
+
+## Compact Chat Summary (returned in chat after validation)
+
+The Verifier returns this block to the orchestrator after completing all checks:
+
+```markdown
+## Validation: [Feature] - [PASS ✅ | FAIL ❌]
+
+**Spec-anchored check**: [N/N ACs matched spec outcome | M spec-precision gaps flagged]
+**Gate**: [X passed, 0 failed]
+**Sensor**: [N mutations injected, N killed, N survived]
+**Report**: `.specs/features/[feature]/validation.md`
+
+**Ranked gaps** (if FAIL):
+1. [Gap description] - [AC or criterion] - [file:line or "no evidence"]
+2. ...
+```
+
+---
+
+## Validation Report Template (`.specs/features/[feature]/validation.md`)
+
+```markdown
+# [Feature] Validation
+
+**Date**: [YYYY-MM-DD]
+**Spec**: `.specs/features/[feature]/spec.md`
+**Diff range**: [commit range or branch..HEAD]
+**Verifier**: independent sub-agent (author ≠ verifier)
+
+---
+
+## Task Completion
+
+| Task | Status     | Notes   |
+| ---- | ---------- | ------- |
+| T1   | ✅ Done    | -       |
+| T2   | ✅ Done    | -       |
+| T3   | ⚠️ Partial | [Issue] |
+
+---
+
+## Spec-Anchored Acceptance Criteria
+
+| Criterion (WHEN X THEN Y) | Spec-defined outcome | `file:line` + assertion | Result |
+| ------------------------- | -------------------- | ----------------------- | ------ |
+| WHEN X THEN Y             | [precise value/state from spec] | `path/to/test.ts:42` - `expect(result.field).toBe(expected)` | ✅ PASS |
+| WHEN A THEN B             | [expected value]     | `path/to/test.ts:88` - `expect(res.status).toBe(400)` | ✅ PASS |
+| WHEN C THEN D             | not precisely defined in spec | - | ⚠️ Spec-precision gap |
+
+**Status**: ✅ All ACs covered / ❌ Gaps present / ⚠️ Spec-precision gaps flagged
+
+---
+
+## Discrimination Sensor
+
+| Mutation | File:line | Description | Killed? |
+| -------- | --------- | ----------- | ------- |
+| 1        | `src/service.ts:42` | Flipped condition `x > 0` → `x >= 0` | ✅ Killed |
+| 2        | `src/service.ts:88` | Changed return value `status: 'active'` → `status: 'inactive'` | ✅ Killed |
+| 3        | `src/handler.ts:15` | Removed side-effect call to `notify()` | ❌ Survived → fix task created |
+
+**Sensor depth**: [lightweight / P0-full]
+**Result**: [N/N killed] - [PASS ✅ | FAIL ❌]
+
+---
+
+## Interactive UAT Results (if performed)
+
+| #   | Test        | Result   | Details                                         |
+| --- | ----------- | -------- | ----------------------------------------------- |
+| 1   | [Test name] | ✅ Pass  | -                                               |
+| 2   | [Test name] | ❌ Issue | [Verbatim user response] - Severity: [inferred] |
+| 3   | [Test name] | ⏭️ Skip  | [Reason]                                        |
+
+---
+
+## Code Quality
+
+| Principle        | Status |
+| ---------------- | ------ |
+| Minimum code     | ✅     |
+| Surgical changes | ✅     |
+| No scope creep   | ✅     |
+| Matches patterns | ✅     |
+| Spec-anchored outcome check (asserted values match spec) | ✅ |
+| Per-layer Coverage Expectation met (domain 1:1 ACs; routes happy+edge+error) | ✅ |
+| Every test maps to a spec requirement - no unclaimed tests | ✅ |
+| Documented guidelines followed: [file(s) or "none - strong defaults applied"] | ✅ |
+
+---
+
+## Edge Cases
+
+- [x] Edge case 1: Handled correctly
+- [ ] Edge case 2: NOT handled - needs fix
+
+---
+
+## Gate Check
+
+- **Gate command**: [Build gate command from the Gate Check Commands section in tasks.md]
+- **Result**: [X] passed, [Y] failed, [Z] skipped
+- **Test count before feature**: [N]
+- **Test count after feature**: [M]
+- **Delta**: [+(M - N) new tests]
+- **Skipped tests**: [list with justification for each]
+- **Failures**: [list with details]
+
+---
+
+## Fix Plans (if issues found)
+
+### Fix 1: [Issue description]
+
+- **Root cause**: [What's actually wrong]
+- **Fix task**: [Task definition]
+- **Priority**: [Blocker/Major/Minor/Cosmetic]
+
+---
+
+## Requirement Traceability Update
+
+Update spec.md requirement statuses:
+
+| Requirement | Previous Status | New Status   |
+| ----------- | --------------- | ------------ |
+| [FEAT]-01   | Implementing    | ✅ Verified  |
+| [FEAT]-02   | Implementing    | ❌ Needs Fix |
+
+---
+
+## Summary
+
+**Overall**: ✅ Ready | ⚠️ Issues | ❌ Not Ready
+
+**Spec-anchored check**: [N/N ACs matched spec outcome | M spec-precision gaps]
+**Sensor**: [N/N mutations killed]
+**Gate**: [X passed]
+
+**What works**: [List]
+
+**Issues found**: [Issue 1: How to fix]
+
+**Next steps**: [Action]
+```
+
+---
+
+## Tips
+
+- **Validation is never prompted** - it always runs after the last task; do not ask the user whether to run it
+- **Spec-anchored, not just covered** - "there is an assertion" is not enough; the assertion must target the spec-defined outcome
+- **Sensor in scratch only** - never mutate the real tree; use a temp worktree or file copies (never `git stash`), run, discard, then confirm porcelain matches the pre-sensor baseline
+- **Surviving mutants are fix tasks** - do not mark the feature done if the sensor found weak tests
+- **P1 first** - MVP must work before P2/P3
+- **WHEN/THEN = Test** - Each criterion is a test case
+- **Be specific** - "Doesn't work" isn't helpful
+- **Recommend fixes** - Don't just report problems, create fix tasks
+- **Quality check is mandatory** - Not optional
+- **Infer severity** - Never ask the user "how bad is this?"
+- **Max 3 diagnostic iterations** - Prevents infinite investigation loops
+- **Update traceability** - Every verified requirement updates spec.md status
+- **Always write the report file** - `.specs/features/[feature]/validation.md` is the persisted evidence artifact
+- **Distill after writing** - turn grounded failures into lessons via `scripts/lessons.py` ([lessons.md](lessons.md)); clean PASS → no lesson

package/framework/skills/tlc-spec-driven/scripts/check_commit.py

@@ -0,0 +1,115 @@
+#!/usr/bin/env python3
+"""
+check_commit.py - deterministic Conventional Commits validation.
+
+The per-task atomic-commit rule mandates Conventional Commits 1.0.0. This makes
+that rule checkable instead of trusting the model to remember the format. Pure
+standard library, zero dependencies, agent-agnostic.
+
+It reads the message from (in priority order): a positional file path, --message,
+or stdin. The file-path form matches how git passes the message file to a
+`commit-msg` hook, so this doubles as an optional git-level guard WITHOUT
+coupling the skill to any AI agent:
+
+    ln -s <skill-dir>/scripts/check_commit.py .git/hooks/commit-msg && chmod +x .git/hooks/commit-msg
+
+What it checks:
+  ERROR  - header does not match  type(scope)!: description
+  ERROR  - type is not one of the allowed Conventional Commits types
+  ERROR  - description is empty, starts uppercase, or ends with a period
+  ERROR  - `!` breaking marker present but no `BREAKING CHANGE:` footer
+  WARN   - header longer than 72 characters
+
+Usage:
+  python3 <skill-dir>/scripts/check_commit.py [msgfile]
+  python3 <skill-dir>/scripts/check_commit.py --message "feat(auth): add email validation"
+  echo "fix(cart): prevent negative quantity" | python3 <skill-dir>/scripts/check_commit.py
+
+Exit codes: 0 pass, 1 violation, 2 usage error.
+"""
+
+import argparse
+import re
+import sys
+
+TYPES = ["feat", "fix", "refactor", "docs", "test", "style", "perf", "build", "ci", "chore"]
+HEADER_RE = re.compile(r"^(?P<type>\w+)(?:\((?P<scope>[^)]+)\))?(?P<bang>!)?: (?P<desc>.+)$")
+
+
+def read_message(args):
+    if args.message is not None:
+        return args.message
+    if args.msgfile:
+        with open(args.msgfile, "r", encoding="utf-8") as f:
+            return f.read()
+    if not sys.stdin.isatty():
+        return sys.stdin.read()
+    return ""
+
+
+def check(message):
+    errors, warnings = [], []
+    # Ignore comment lines (git puts '#' comments in the message file).
+    lines = [ln for ln in message.splitlines() if not ln.lstrip().startswith("#")]
+    # Trim leading blank lines.
+    while lines and not lines[0].strip():
+        lines.pop(0)
+    if not lines:
+        return (["empty commit message"], warnings)
+
+    header = lines[0].rstrip()
+    if len(header) > 72:
+        warnings.append(f"header is {len(header)} chars (>72): {header[:60]}...")
+
+    m = HEADER_RE.match(header)
+    if not m:
+        errors.append(f"header does not match 'type(scope): description': {header!r}")
+        return (errors, warnings)
+
+    ctype = m.group("type")
+    desc = m.group("desc")
+    bang = m.group("bang")
+
+    if ctype not in TYPES:
+        errors.append(f"type '{ctype}' is not one of: {', '.join(TYPES)}")
+    if not desc.strip():
+        errors.append("description is empty")
+    else:
+        if desc[:1].isupper():
+            errors.append(f"description should start lowercase: '{desc[:30]}'")
+        if desc.rstrip().endswith("."):
+            errors.append("description should not end with a period")
+
+    body = "\n".join(lines[1:])
+    breaking_footer = bool(re.search(r"^BREAKING CHANGE:", body, re.MULTILINE))
+    if bang and not breaking_footer:
+        errors.append("'!' breaking marker present but no 'BREAKING CHANGE:' footer")
+
+    return (errors, warnings)
+
+
+def main(argv=None):
+    p = argparse.ArgumentParser(prog="check_commit.py", description="Validate a Conventional Commits message.")
+    p.add_argument("msgfile", nargs="?", default=None, help="path to a commit message file (as git passes to commit-msg)")
+    p.add_argument("--message", default=None, help="the commit message as a string")
+    args = p.parse_args(argv)
+
+    message = read_message(args)
+    if not message.strip():
+        print("check_commit: no message provided (pass a file, --message, or pipe via stdin).", file=sys.stderr)
+        return 2
+
+    errors, warnings = check(message)
+    for w in warnings:
+        print(f"  WARN  {w}")
+    for e in errors:
+        print(f"  ERROR {e}")
+    if errors:
+        print("\ncheck_commit: FAIL - see https://www.conventionalcommits.org/en/v1.0.0/")
+        return 1
+    print("check_commit: OK")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())