all-hands-cli 0.1.9 → 0.1.11

package/.allhands/skills/harness-maintenance/references/validation-tooling.md

@@ -17,6 +17,16 @@ This is how validation compounds. Every domain has both a stochastic dimension (
 
 A validation suite must have a meaningful stochastic dimension to justify existing. Deterministic-only tools (type checking, linting, formatting) are test commands referenced directly in acceptance criteria and CI/CD — they are NOT suites.
 
+## Repository Agnosticism
+
+This reference file is a generic rule file that ships with the harness. It MUST NOT contain references to project-specific validation suites, commands, or infrastructure. All examples must either:
+- Reference existing default validation suites shipped with this repo (currently: xcode-automation, browser-automation)
+- Use generic/hypothetical descriptions that any target repository can map to their own context
+
+When examples are needed, use **snippets from the existing default suites** rather than naming suites or commands that belong to a specific target project. Target repositories create their own suites for their domains — this file teaches how to create and structure them, not what they should be called.
+
+**Why**: Target repositories consume this file as authoritative guidance. Project-specific references create confusion (agents look for suites that don't exist), couple the harness to a single project, and violate the principle that this file teaches patterns, not inventories. If a pattern needs a concrete example, draw it from xcode-automation or browser-automation.
+
 ## Creating Validation Tooling
 
 Follow `.allhands/flows/shared/CREATE_VALIDATION_TOOLING_SPEC.md` for the full process. This creates a spec, not an implementation.
@@ -80,6 +90,113 @@ Prompt files reference validation suites in their `validation_suites` frontmatte
 2. Agent runs suite's **Deterministic Integration** section for acceptance criteria gating
 3. Validation review (`PROMPT_VALIDATION_REVIEW.md`) confirms pass/fail
 
+## Command Documentation Principle
+
+Two categories of commands exist in validation suites, each requiring different documentation approaches:
+
+**External tooling commands — Document explicitly**: Commands from external tools (`xctrace`, `xcrun simctl`, `agent-browser`, `playwright`, `curl`, etc.) are stable, unfamiliar to agents by default, and unlikely to change with codebase evolution. Document specific commands, flags, and use cases inline with motivations. Example from xcode-automation: `xcrun xctrace record --template 'Time Profiler' --device '<UDID>' --attach '<PID>'` — the flags, ordering constraints, and PID discovery method are all external tool knowledge that the suite documents explicitly.
+
+**Internal codebase commands — Document patterns, not inventories**: Project-specific scripts, test commands, and codebase-specific CLI wrappers evolve rapidly. Instead:
+1. **Document core infrastructure commands explicitly** — commands that boot services, manage environments, and are foundational to validation in the target project. These are stable and essential per-project, but suites should teach agents how to discover them (e.g., "check `package.json` scripts" or "run `--help`"), not hardcode specific script names.
+2. **Teach patterns for everything else** — naming conventions, where to discover project commands, what categories mean, and how to build upon them.
+3. **Document motivations** — why different test categories exist, when to use which, what confidence each provides.
+
+Per **Frontier Models are Capable**: An agent given patterns + motivations + discovery instructions outperforms one given stale command inventories. Suites that teach patterns age gracefully; suites that enumerate commands require maintenance on every change.
+
+## Decision Tree Requirement
+
+Every validation suite MUST include a decision tree that routes agents to the correct validation approach based on their situation. Decision trees:
+- Distinguish which instructions are relevant to which validation scenario (e.g., UI-only test vs full E2E with native code changes)
+- Show where/when stochastic vs deterministic testing applies
+- Surface deterministic branch points where other validation suites must be utilized (e.g., "Does this branch have native code changes? → Yes → follow xcode-automation decision tree")
+- Cleanly articulate multiple expected use cases within a single suite
+
+The decision tree replaces flat prerequisite lists with structured routing. An agent reads the tree and follows the branch matching their situation, skipping irrelevant setup and finding the right cross-references.
+
+## tmux Session Management Standard
+
+All suites that require long-running processes (dev servers, Expo servers, Flask API, Metro bundler) MUST use the tmux approach proven in xcode-automation:
+
+```bash
+# CRITICAL: -t $TMUX_PANE pins split to agent's window, not user's focused window
+tmux split-window -h -d -t $TMUX_PANE \
+  -c /path/to/repo '<command>'
+```
+
+**Observability**: Agents MUST verify processes are running correctly via tmux pane capture (`tmux capture-pane -p -t <pane_id>`) before proceeding with validation. This prevents silent failures where a dev server fails to start but the agent proceeds to test against nothing.
+
+**Teardown**: Reverse order of setup. Kill processes via `tmux send-keys -t <pane_id> C-c` or kill the pane.
+
+**Worktree isolation**: Each worktree uses unique ports (via `.env.local`), so tmux sessions in different worktrees don't conflict. Agents must use the correct repo path (`-c`) for the worktree they're operating in.
+
+Reference xcode-automation as the canonical tmux pattern.
+
+## Hypothesis-First Validation Workflow
+
+New suites should be drafted, then tested hands-on on a feature branch before guidance is marked as proven. This aligns with the Proven vs Untested Guidance principle:
+
+1. **Draft**: Write suite files based on plan and codebase analysis (mark unverified practices as hypotheses)
+2. **Test on feature branch**: Check out a feature branch and exercise each suite's practices hands-on — boot services, run commands, verify workflows, test worktree isolation
+3. **Verify & adjust**: Document what works, what doesn't, what needs adjustment. Worktree-specific concerns get explicit verification.
+4. **Solidify**: Only after verification do practices become authoritative guidance. Unverified practices stay framed as motivations per the Proven vs Untested Guidance principle.
+
+The plan/handoff document persists as the hypothesis record. If implementation runs long, it serves as the handoff document for future work.
+
+## Cross-Referencing Between Suites
+
+**Reference** when complex multi-step setup is involved (e.g., simulator setup spanning multiple tools) — point to the authoritative suite's decision tree rather than duplicating instructions.
+
+**Inline** when the command is simple and stable (e.g., `xcrun simctl boot <UDID>`) — no need to send agents to another document for a single command.
+
+Decision trees are the natural place for cross-references — branch points that route to another suite's decision tree. Example from browser-automation: "Does the change affect native iOS rendering? → Yes → follow xcode-automation decision tree for build and simulator verification."
+
+## Testing Scenario Matrix
+
+Target repositories should build a scenario matrix mapping their validation scenarios to suite combinations. The matrix documents which suites apply to which types of changes, so agents can quickly determine what validation is needed. Structure as a table:
+
+| Scenario | Suite(s) | Notes |
+|----------|----------|-------|
+| _Description of change type_ | _Which suites apply_ | _Any special setup or cross-references_ |
+
+Example using this repo's default suites:
+
+| Scenario | Suite(s) | Notes |
+|----------|----------|-------|
+| Browser UI changes only | browser-automation | Dev server must be running |
+| Native iOS/macOS changes | xcode-automation | Simulator setup via session defaults |
+| Cross-platform changes (web + native) | browser-automation + xcode-automation | Each suite's decision tree routes to the relevant validation path |
+
+When a suite serves as a shared dependency for multiple scenarios (e.g., a database management suite referenced by both API and front-end suites), it should be cross-referenced via decision tree branch points rather than duplicated.
+
+## Environment Management Patterns
+
+Validation suites that depend on environment configuration should document these patterns for their domain:
+
+**ENV injection**: Document how the target project injects environment variables for different contexts (local development, testing, production). Suites should teach the pattern (e.g., "check for `.env.*` files and wrapper scripts") rather than hardcoding specific variable names.
+
+**Service isolation**: When validation requires running services (dev servers, databases, bundlers), document how to avoid port conflicts across concurrent worktrees or parallel agent sessions. Reference the suite's ENV Configuration table for relevant variables.
+
+**Worktree isolation**: Each worktree should use unique ports and isolated service instances where possible. Suites should document which resources need isolation and how to configure it (e.g., xcode-automation documents simulator isolation via dedicated simulator clones and derived data paths).
+
+## Suite Creation Guidance
+
+When creating a new validation suite for a new domain:
+
+**Engineer provides**: Testing scenarios, tooling requirements, CI/CD integration needs, cross-references to existing suites.
+
+**Suite author follows**:
+1. Follow the validation suite schema (`ah schema validation-suite`)
+2. Validate the stochastic dimension meets the existence threshold
+3. Apply the Command Documentation Principle — external tools explicit, internal commands via patterns + discovery
+4. Include a Decision Tree routing agents to the correct validation path
+5. Use tmux Session Management Standard for long-running processes
+6. Document proven vs untested guidance per the Hypothesis-First Validation Workflow
+7. Cross-reference other suites at decision tree branch points
+
+**Structural templates** (reference the existing default suites for patterns):
+- xcode-automation — external-tool-heavy suite (MCP tools, xctrace, simctl). Reference for suites that primarily wrap external CLI tools with agent-driven exploration.
+- browser-automation — dual-dimension suite (agent-browser stochastic, Playwright deterministic). Reference for suites that have both agent-driven exploration and scripted CI-gated tests.
+
 ## Related References
 
 - [`tools-commands-mcp-hooks.md`](tools-commands-mcp-hooks.md) — When validation uses hooks, CLI commands, or MCP research tools

package/bin/sync-cli.js

@@ -7245,6 +7245,16 @@ function collectFilesToPush(cwd, finalIncludes, finalExcludes) {
   const upstreamFiles = manifest.getDistributableFiles();
   const filesToPush = [];
   const localGitFiles = new Set(getGitFiles(cwd));
+  const deletedFiles = /* @__PURE__ */ new Set();
+  for (const args of [
+    ["diff", "--name-only", "--diff-filter=D"],
+    ["diff", "--cached", "--name-only", "--diff-filter=D"]
+  ]) {
+    const result = git(args, cwd);
+    if (result.success && result.stdout) {
+      result.stdout.split("\n").filter(Boolean).forEach((f) => deletedFiles.add(f));
+    }
+  }
   for (const relPath of upstreamFiles) {
     if (manifest.isInitOnly(relPath)) {
       continue;
@@ -7255,15 +7265,19 @@ function collectFilesToPush(cwd, finalIncludes, finalExcludes) {
     if (finalExcludes.some((pattern) => minimatch(relPath, pattern, { dot: true }))) {
       continue;
     }
-    if (!localGitFiles.has(relPath)) {
+    if (!localGitFiles.has(relPath) && !deletedFiles.has(relPath)) {
       continue;
     }
     const localFile = join9(cwd, relPath);
     const upstreamFile = join9(allhandsRoot, relPath);
-    if (existsSync11(localFile) && filesAreDifferent(localFile, upstreamFile)) {
-      if (wasModifiedByTargetRepo(cwd, relPath, allhandsRoot)) {
-        filesToPush.push({ path: relPath, type: "M" });
+    if (existsSync11(localFile)) {
+      if (filesAreDifferent(localFile, upstreamFile)) {
+        if (wasModifiedByTargetRepo(cwd, relPath, allhandsRoot)) {
+          filesToPush.push({ path: relPath, type: "M" });
+        }
       }
+    } else if (deletedFiles.has(relPath)) {
+      filesToPush.push({ path: relPath, type: "D" });
     }
   }
   for (const pattern of finalIncludes) {
@@ -7335,12 +7349,16 @@ async function createPullRequest(cwd, ghUser, filesToPush, title, body) {
       console.error("Error creating branch:", checkoutResult.stderr);
       return 1;
     }
-    console.log("Copying files...");
+    console.log("Applying changes...");
     for (const file of filesToPush) {
-      const src = join9(cwd, file.path);
-      const dest = join9(tempDir, file.path);
-      mkdirSync2(dirname8(dest), { recursive: true });
-      copyFileSync2(src, dest);
+      if (file.type === "D") {
+        git(["rm", "--ignore-unmatch", file.path], tempDir);
+      } else {
+        const src = join9(cwd, file.path);
+        const dest = join9(tempDir, file.path);
+        mkdirSync2(dirname8(dest), { recursive: true });
+        copyFileSync2(src, dest);
+      }
     }
     const addResult = git(["add", "."], tempDir);
     if (!addResult.success) {
@@ -7409,8 +7427,8 @@ async function cmdPush(include, exclude, dryRun, titleArg, bodyArg) {
   }
   console.log("\nFiles to be included in PR:");
   for (const file of filesToPush.sort((a, b) => a.path.localeCompare(b.path))) {
-    const marker = file.type === "M" ? "M" : "A";
-    const label = file.type === "M" ? "modified" : "included";
+    const marker = file.type;
+    const label = { D: "deleted", M: "modified", A: "included" }[file.type];
     console.log(`  ${marker} ${file.path} (${label})`);
   }
   console.log();

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "all-hands-cli",
-  "version": "0.1.9",
+  "version": "0.1.11",
   "description": "Agentic harness for model-first software development",
   "type": "module",
   "bin": {

package/src/commands/push.ts

@@ -17,7 +17,7 @@ interface SyncConfig {
 
 interface FileEntry {
   path: string;
-  type: 'M' | 'A';
+  type: 'M' | 'A' | 'D';
 }
 
 interface PrerequisiteResult {
@@ -124,6 +124,18 @@ function collectFilesToPush(
   // Get non-ignored files in user's repo (respects .gitignore)
   const localGitFiles = new Set(getGitFiles(cwd));
 
+  // Get files deleted locally (both staged and unstaged deletions)
+  const deletedFiles = new Set<string>();
+  for (const args of [
+    ['diff', '--name-only', '--diff-filter=D'],
+    ['diff', '--cached', '--name-only', '--diff-filter=D'],
+  ]) {
+    const result = git(args, cwd);
+    if (result.success && result.stdout) {
+      result.stdout.split('\n').filter(Boolean).forEach((f) => deletedFiles.add(f));
+    }
+  }
+
   for (const relPath of upstreamFiles) {
     if (manifest.isInitOnly(relPath)) {
       continue;
@@ -134,18 +146,22 @@ function collectFilesToPush(
     if (finalExcludes.some((pattern) => minimatch(relPath, pattern, { dot: true }))) {
       continue;
     }
-    // Skip files that are gitignored in user's repo
-    if (!localGitFiles.has(relPath)) {
+    // Skip files that are gitignored in user's repo (but allow deleted files through)
+    if (!localGitFiles.has(relPath) && !deletedFiles.has(relPath)) {
       continue;
     }
 
     const localFile = join(cwd, relPath);
     const upstreamFile = join(allhandsRoot, relPath);
 
-    if (existsSync(localFile) && filesAreDifferent(localFile, upstreamFile)) {
-      if (wasModifiedByTargetRepo(cwd, relPath, allhandsRoot)) {
-        filesToPush.push({ path: relPath, type: 'M' });
+    if (existsSync(localFile)) {
+      if (filesAreDifferent(localFile, upstreamFile)) {
+        if (wasModifiedByTargetRepo(cwd, relPath, allhandsRoot)) {
+          filesToPush.push({ path: relPath, type: 'M' });
+        }
       }
+    } else if (deletedFiles.has(relPath)) {
+      filesToPush.push({ path: relPath, type: 'D' });
     }
   }
 
@@ -239,12 +255,16 @@ async function createPullRequest(
       return 1;
     }
 
-    console.log('Copying files...');
+    console.log('Applying changes...');
     for (const file of filesToPush) {
-      const src = join(cwd, file.path);
-      const dest = join(tempDir, file.path);
-      mkdirSync(dirname(dest), { recursive: true });
-      copyFileSync(src, dest);
+      if (file.type === 'D') {
+        git(['rm', '--ignore-unmatch', file.path], tempDir);
+      } else {
+        const src = join(cwd, file.path);
+        const dest = join(tempDir, file.path);
+        mkdirSync(dirname(dest), { recursive: true });
+        copyFileSync(src, dest);
+      }
     }
 
     const addResult = git(['add', '.'], tempDir);
@@ -328,8 +348,8 @@ export async function cmdPush(
 
   console.log('\nFiles to be included in PR:');
   for (const file of filesToPush.sort((a, b) => a.path.localeCompare(b.path))) {
-    const marker = file.type === 'M' ? 'M' : 'A';
-    const label = file.type === 'M' ? 'modified' : 'included';
+    const marker = file.type;
+    const label = { D: 'deleted', M: 'modified', A: 'included' }[file.type];
     console.log(`  ${marker} ${file.path} (${label})`);
   }
   console.log();