onecrawl 4.0.0-beta.1 → 4.0.0-beta.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -17,43 +17,70 @@ Run a structured, future-proof decision process when a task may affect public co
17
17
  | Surface | Breaking Impact | Examples |
18
18
  |---------|----------------|----------|
19
19
  | **CLI flags/args** | HIGH | Renaming `--headless` → `--head`, removing `--native` |
20
- | **MCP tool names/schemas** | HIGH | Renaming tool actions, changing JSON schemas |
20
+ | **MCP tool names/schemas** | HIGH | Renaming tool actions, changing JSON input schemas |
21
21
  | **NAPI/PyO3 method signatures** | HIGH | Changing function names, param types, return types |
22
- | **Config keys** | MEDIUM | Renaming `config.toml` keys, changing defaults |
22
+ | **Config keys** (`config.toml`) | MEDIUM | Renaming keys, changing defaults, removing options |
23
23
  | **Session file format** | MEDIUM | Changing `/tmp/onecrawl-session-*.json` structure |
24
- | **Daemon protocol** | MEDIUM | Changing HTTP/WS API between CLI and daemon |
25
- | **Internal crate APIs** | LOW | Changing `pub` functions in workspace crates |
26
- | **CDP layer** | LOW | Internal CDP wrapper changes |
24
+ | **Daemon HTTP/WS protocol** | MEDIUM | Changing API between CLI daemon communication |
25
+ | **Profile file format** | MEDIUM | Changing profile storage schema |
26
+ | **Internal crate `pub` APIs** | LOW | Changing `pub fn` in workspace crates (internal consumers only) |
27
+ | **CDP layer wrappers** | LOW | Internal CDP abstractions (no external consumers) |
28
+
29
+ ## Concrete OneCrawl Examples
30
+
31
+ ### Non-Breaking Change Examples
32
+ ```
33
+ ✅ Adding new CLI flag: `onecrawl session start --timeout 30`
34
+ ✅ Adding new MCP action: `onecrawl run browser get_accessibility_snapshot`
35
+ ✅ New config key with default: config.toml gains `stealth.level = "standard"`
36
+ ✅ Adding optional field to session JSON: `{"pid": 1234, "created_at": "..."}`
37
+ ✅ New NAPI export function (additive)
38
+ ```
39
+
40
+ ### Breaking Change Examples
41
+ ```
42
+ ⛔ Renaming `onecrawl run browser` → `onecrawl browser exec`
43
+ ⛔ Removing `--native` flag from session start
44
+ ⛔ Changing MCP tool "browser" schema (existing clients break)
45
+ ⛔ Changing session JSON key: "session_name" → "name"
46
+ ⛔ Changing PyO3 class method signature
47
+ ```
27
48
 
28
49
  ## Decision Framework
29
50
 
30
51
  ### Non-Breaking Path
31
52
  - Add new flags/options alongside existing ones
32
- - Deprecation warnings before removal (minimum 2 alpha releases)
53
+ - Deprecation warnings before removal (minimum 2 beta releases)
33
54
  - Backward-compatible config: new keys get defaults, old keys still work
34
55
  - Additive MCP actions (new actions, not renamed ones)
56
+ - Session JSON: add optional fields, never remove/rename existing
35
57
 
36
58
  ### Breaking Path (requires justification)
37
- - Must document: what breaks, who is affected, migration steps
38
- - Must provide: migration guide or automated migration tool
39
- - Must bump: version increment that signals the break
40
- - Quality gates unchanged: unit/integration/E2E/non-regression must still pass
59
+ - [ ] Document: what breaks, who is affected, migration steps
60
+ - [ ] Migration: provide guide or automated migration in code
61
+ - [ ] Version: bump that signals the break (beta → beta.N+1 minimum)
62
+ - [ ] CHANGELOG: explicit "BREAKING" section with before/after examples
63
+ - [ ] Quality gates unchanged: completion-gate must still pass
41
64
 
42
65
  ## Procedure
43
- 1. Identify all contract surfaces affected.
44
- 2. For each, classify as non-breaking or breaking.
45
- 3. Present options via `ask_user` with the compatibility triad:
46
- - Non-Breaking Path (additive/compatible)
47
- - Breaking Path (with migration plan)
48
- - Alternative Structural Path (redesign to avoid the break)
66
+ 1. Identify all contract surfaces affected (use table above).
67
+ 2. For each surface, classify impact as non-breaking or breaking.
68
+ 3. If any HIGH-impact surface is affected, present options to user:
69
+ - **Non-Breaking Path** (additive/compatible — deprecate first)
70
+ - **Breaking Path** (with migration plan)
71
+ - **Alternative Structural Path** (redesign to avoid the break)
49
72
  4. Document decision in commit message and CHANGELOG.
73
+ 5. If breaking: add `BREAKING:` prefix to commit message.
50
74
 
51
75
  ## Done Criteria
52
76
  - Decision documented with rationale.
53
77
  - Migration steps provided for any breaking change.
54
- - All quality gates pass.
78
+ - All quality gates pass (completion-gate skill).
79
+ - CHANGELOG updated with breaking change notice if applicable.
55
80
 
56
81
  ## Anti-patterns
57
- - Silent breaking changes (no documentation)
82
+ - Silent breaking changes (no documentation, no CHANGELOG entry)
58
83
  - Breaking changes without version bump
59
- - Assuming internal changes are safe (NAPI/PyO3 are public)
84
+ - Assuming internal crate changes are safe (NAPI/PyO3 expose them)
85
+ - Removing deprecated items before 2-release grace period
86
+ - Breaking MCP schemas without client-side migration path
@@ -21,37 +21,74 @@ Enforce a mandatory quality gate for every Issue, Milestone, and PR with zero-er
21
21
 
22
22
  ## OneCrawl Gate Commands
23
23
 
24
- Run these in order. Both must pass **twice consecutively** with zero output:
24
+ Run in order. All must pass **twice consecutively** with zero output:
25
25
 
26
26
  ```bash
27
- # 1. Clippy (lint + static analysis) - 0 warnings required
27
+ # 1. Clippy (lint + static analysis) 0 warnings required
28
28
  cargo clippy --workspace --exclude onecrawl-e2e --exclude onecrawl-python -- -W clippy::all
29
29
 
30
30
  # 2. Build check (fast compilation verification)
31
- cargo check --workspace
31
+ cargo check -p onecrawl-cli-rs -p onecrawl-mcp-rs
32
32
 
33
- # 3. Test suite (all tests, single-threaded for determinism)
33
+ # 3. Test suite (573 tests, single-threaded for determinism)
34
34
  cargo test --workspace --exclude onecrawl-e2e --exclude onecrawl-python -- --test-threads=1
35
35
 
36
- # 4. Health check (binary runs correctly)
36
+ # 4. Release build (binary artifact)
37
+ cargo build --release -p onecrawl-cli-rs
38
+
39
+ # 5. Health check (binary runs correctly)
37
40
  ./target/release/onecrawl health
38
41
  ./target/release/onecrawl --version
39
42
  ```
40
43
 
41
- Acceptable warnings: `onecrawl-browser` vendor crate (1 warning, do not touch).
44
+ Acceptable warnings: `onecrawl-browser` vendor crate (1 pre-existing clippy warning do not touch).
45
+
46
+ ## Gate Execution Checklist
47
+
48
+ - [ ] **Pass 1 — Clippy**: 0 warnings in owned crates
49
+ - [ ] **Pass 1 — Build**: `cargo check` exits 0
50
+ - [ ] **Pass 1 — Tests**: 573 tests passed, 0 failed
51
+ - [ ] **Pass 1 — Binary**: `onecrawl --version` prints `v4.0.0-beta.1`
52
+ - [ ] **Pass 2 — Clippy**: identical to pass 1
53
+ - [ ] **Pass 2 — Tests**: identical to pass 1
54
+ - [ ] **No regressions**: test count ≥ baseline recorded before changes
55
+ - [ ] **Commit**: use `git commit -F /tmp/commit-msg.txt` with Co-authored-by trailer
56
+
57
+ ## CI Verification (release.yml + rust-ci.yml)
58
+
59
+ After push, verify CI on GitHub:
60
+
61
+ ```bash
62
+ # Check CI status for current branch
63
+ gh run list --repo giulio-leone/onecrawl --branch "$(git branch --show-current)" --limit 5
64
+
65
+ # Watch a specific run
66
+ gh run watch <run-id> --repo giulio-leone/onecrawl
67
+
68
+ # Get failed job logs
69
+ gh run view <run-id> --repo giulio-leone/onecrawl --log-failed
70
+ ```
71
+
72
+ **rust-ci.yml** gates: clippy, test (--test-threads=1), build check.
73
+ **release.yml** gates: 5-platform matrix (linux-x64, linux-arm64, macos-x64, macos-arm64, windows-x64).
42
74
 
43
75
  ## Procedure
44
- 1. Run all gate commands above.
45
- 2. If any error/warning exists in owned crates, fix and restart from step 1.
46
- 3. Repeat until two clean consecutive passes are recorded.
47
- 4. Only then mark status `done`/merge.
76
+ 1. Record baseline: `cargo test ... 2>&1 | tail -1` (note test count).
77
+ 2. Run all gate commands above.
78
+ 3. If any error/warning exists in owned crates, fix and restart from step 1.
79
+ 4. Repeat until two clean consecutive passes are recorded.
80
+ 5. Verify CI is green after push (`gh run list`).
81
+ 6. Only then mark status `done`/merge.
48
82
 
49
83
  ## Done Criteria
50
84
  - Two consecutive clean review passes are documented.
51
85
  - No unresolved pre-existing issues in touched scope.
86
+ - CI (rust-ci.yml) is green on the pushed branch.
52
87
 
53
88
  ## Anti-patterns
54
89
  - Single-pass approval
55
- - Ignoring warnings
90
+ - Ignoring warnings ("it's just one warning")
56
91
  - Deferring known issues to "later"
57
92
  - Excluding tests with `#[ignore]` to pass the gate
93
+ - Pushing without verifying CI status
94
+ - Using `cargo check --workspace` when `cargo check -p <crate>` suffices (slower)
@@ -13,12 +13,13 @@ Ensure critical user flows work end-to-end with deterministic, CI-compatible exe
13
13
  ## OneCrawl E2E Architecture
14
14
 
15
15
  E2E tests live in `packages/onecrawl-rust/crates/onecrawl-e2e/` and require a running Chrome instance.
16
+ Session state files: `/tmp/onecrawl-session*.json` (per-PID markers for multi-agent isolation).
16
17
 
17
18
  ```bash
18
19
  # Run E2E tests (requires Chrome)
19
20
  cargo test -p onecrawl-e2e -- --test-threads=1
20
21
 
21
- # Manual E2E verification (quick smoke test)
22
+ # Manual E2E smoke test
22
23
  onecrawl session start -H # Headless Chrome
23
24
  onecrawl navigate https://example.com # Navigate
24
25
  onecrawl get title # Verify page loaded
@@ -34,20 +35,59 @@ onecrawl session close # Cleanup
34
35
  5. **Config management**: config set → config show → verify change persists
35
36
  6. **Auth persistence**: auth-state save → session close → session start → auth-state load
36
37
  7. **Stealth**: session start → stealth detection-audit → verify 0% headless detection
38
+ 8. **MCP server**: onecrawl mcp → tool discovery → action execution → clean shutdown
37
39
 
38
- ## Checklist
40
+ ## Multi-Agent E2E Test Template
41
+
42
+ Tests verifying process-level isolation between concurrent agents:
43
+
44
+ ```bash
45
+ # Agent A (PID-based session isolation)
46
+ onecrawl session start -H -s "agent-$$-a"
47
+ onecrawl navigate https://example.com
48
+ TITLE_A=$(onecrawl get title)
49
+
50
+ # Agent B (separate session, same daemon)
51
+ onecrawl session start -H -s "agent-$$-b"
52
+ onecrawl navigate https://httpbin.org
53
+ TITLE_B=$(onecrawl get title)
54
+
55
+ # Verify isolation — each session sees its own page
56
+ [ "$TITLE_A" != "$TITLE_B" ] && echo "PASS: isolation OK"
57
+
58
+ # Cleanup both
59
+ onecrawl session close -s "agent-$$-a"
60
+ onecrawl session close -s "agent-$$-b"
61
+ ```
62
+
63
+ ## Pre-Test Checklist
64
+ - [ ] Chrome/Chromium installed and accessible
65
+ - [ ] No stale sessions: `ls /tmp/onecrawl-session*.json` (should be empty)
66
+ - [ ] No orphan Chrome processes: `pgrep -f "chrome.*remote-debugging" | head`
67
+ - [ ] Release binary built: `cargo build --release -p onecrawl-cli-rs`
68
+
69
+ ## Test Execution Checklist
39
70
  - [ ] Happy path covered for each critical flow
40
71
  - [ ] Edge cases: invalid input, missing Chrome, concurrent sessions
41
- - [ ] Cleanup: sessions closed, temp files removed, profiles deleted
42
- - [ ] Deterministic: no timing-dependent assertions
72
+ - [ ] Deterministic: no timing-dependent assertions (use wait-for, not sleep)
43
73
  - [ ] Stable selectors: use `data-testid` or semantic selectors
44
74
  - [ ] CI-compatible: headless mode, no GUI dependencies
45
75
 
76
+ ## Post-Test Cleanup Checklist
77
+ - [ ] All sessions closed: `onecrawl session close` or `onecrawl session close -s <name>`
78
+ - [ ] Temp files removed: `rm -f /tmp/onecrawl-session*.json`
79
+ - [ ] No orphan Chrome: verify `pgrep -f "chrome.*remote-debugging"` returns empty
80
+ - [ ] Profiles cleaned: `onecrawl profile delete <test-profile>` if created
81
+ - [ ] Daemon stopped (if started for test): `onecrawl daemon stop`
82
+
46
83
  ## Done Criteria
47
84
  - All critical flows pass in headless mode.
48
85
  - No leaked browser processes after test completion.
86
+ - No residual session files in `/tmp/`.
49
87
 
50
88
  ## Anti-patterns
51
89
  - Hard-coded sleep/timeouts instead of wait-for conditions
52
- - Tests that depend on network state
90
+ - Tests that depend on network state or external services
53
91
  - Shared mutable state between test cases
92
+ - Forgetting to close sessions (leaks Chrome processes)
93
+ - Using `--test-threads=N` where N > 1 (session conflicts)
@@ -11,43 +11,113 @@ Keep local plan and GitHub project artifacts perfectly aligned.
11
11
  - Creating or updating a plan
12
12
  - Changing issue status
13
13
  - Completing milestones
14
+ - Preparing a release
14
15
 
15
16
  ## OneCrawl Repository
16
17
  - Owner: `giulio-leone`
17
18
  - Repo: `onecrawl`
18
- - Branch: `main`
19
- - Tags: `v4.0.0-alpha.XX`
19
+ - Default branch: `main`
20
+ - Current tag: `v4.0.0-beta.1`
21
+ - CI workflows: `release.yml` (5-platform matrix), `rust-ci.yml`
20
22
 
21
23
  ## Naming Rules
22
24
  - Milestones: `MX: <Title>` (e.g., `M14: Deep Audit`)
23
25
  - Issues: `<type>: <description>` (e.g., `fix: session name path traversal`)
24
26
  - Branches: `<type>/<short-description>` (e.g., `fix/session-path-traversal`)
25
- - Tags: `v4.0.0-alpha.XX` (SemVer pre-release)
27
+ - Tags: SemVer pre-release (e.g., `v4.0.0-beta.2`)
28
+ - Commits: use `git commit -F /tmp/commit-msg.txt` (not `-m`), always include Co-authored-by trailer
26
29
 
27
30
  ## Required Labels
28
31
  - Priority: `P0-critical`, `P1-high`, `P2-medium`, `P3-low`
29
32
  - Type: `bug`, `feature`, `refactor`, `docs`, `chore`
30
33
  - Status: `todo`, `in-progress`, `review`, `done`, `blocked`
31
34
 
35
+ ## gh CLI Commands
36
+
37
+ ```bash
38
+ # Create milestone
39
+ gh api repos/giulio-leone/onecrawl/milestones -f title="M15: Title" -f state="open"
40
+
41
+ # Create issue with labels and milestone
42
+ gh issue create --repo giulio-leone/onecrawl \
43
+ --title "fix: description" \
44
+ --body "Details..." \
45
+ --label "bug,P1-high,in-progress" \
46
+ --milestone "M15: Title"
47
+
48
+ # Update issue status
49
+ gh issue edit <number> --repo giulio-leone/onecrawl --remove-label "in-progress" --add-label "done"
50
+
51
+ # Close issue
52
+ gh issue close <number> --repo giulio-leone/onecrawl
53
+
54
+ # List open issues for a milestone
55
+ gh issue list --repo giulio-leone/onecrawl --milestone "M15: Title" --state open
56
+
57
+ # Check CI status
58
+ gh run list --repo giulio-leone/onecrawl --branch main --limit 5
59
+ ```
60
+
32
61
  ## Procedure
33
62
  1. Create GitHub milestone matching local plan milestone.
34
63
  2. Create issues for each task, linked to milestone.
35
64
  3. Apply labels (priority + type + status).
36
- 4. Update issue status as work progresses.
37
- 5. Close milestone when all issues are done.
65
+ 4. Update issue status labels as work progresses.
66
+ 5. Close issues when completion gate passes.
67
+ 6. Close milestone when all issues are done.
68
+
69
+ ## OneCrawl Release Workflow
70
+
71
+ ### Pre-Release Checklist
72
+ - [ ] All milestone issues closed
73
+ - [ ] Completion gate passed (2 consecutive clean runs)
74
+ - [ ] CHANGELOG.md updated with release notes
75
+ - [ ] Version bumped in `packages/onecrawl-rust/Cargo.toml`
76
+ - [ ] `cargo check --workspace` (propagates version to all crates)
77
+
78
+ ### Release Steps
79
+ ```bash
80
+ # 1. Version bump
81
+ # Edit packages/onecrawl-rust/Cargo.toml — update version field
82
+ cargo check --workspace
83
+
84
+ # 2. Update CHANGELOG.md — add release section
85
+
86
+ # 3. Commit
87
+ echo "chore: release v4.0.0-beta.2
88
+
89
+ - <summary of changes>
90
+
91
+ Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>" > /tmp/release-msg.txt
92
+ git add -A && git commit -F /tmp/release-msg.txt
93
+
94
+ # 4. Tag and push
95
+ git tag v4.0.0-beta.2
96
+ git push origin main --tags
97
+
98
+ # 5. npm publish (from packages/onecrawl/)
99
+ cd packages/onecrawl
100
+ npm version 4.0.0-beta.2 --no-git-tag-version
101
+ npm publish --tag beta --access public
102
+
103
+ # 6. Verify release.yml CI (5-platform matrix)
104
+ gh run list --repo giulio-leone/onecrawl --workflow release.yml --limit 1
105
+ ```
38
106
 
39
- ## OneCrawl Release Sync
40
- After each alpha release:
41
- 1. Tag: `git tag v4.0.0-alpha.XX && git push origin main --tags`
42
- 2. npm: `npm publish --tag alpha --access public`
43
- 3. CHANGELOG: update with release notes
107
+ ### Post-Release Verification
108
+ - [ ] `gh run view <id>` — all 5 platforms green
109
+ - [ ] `npm view onecrawl versions --json | tail` version published
110
+ - [ ] GitHub tag visible: `gh release list --repo giulio-leone/onecrawl`
44
111
 
45
112
  ## Done Criteria
46
113
  - All plan items have matching GitHub artifacts.
47
114
  - Labels are applied consistently.
48
115
  - Milestones reflect actual progress.
116
+ - Releases are tagged, CI-verified, and npm-published.
49
117
 
50
118
  ## Anti-patterns
51
119
  - Local plan diverges from GitHub state
52
120
  - Missing labels on issues
53
121
  - Stale milestones with no issues
122
+ - Pushing tags before CI verification
123
+ - Using `git commit -m` instead of `git commit -F`
@@ -13,14 +13,14 @@ Enforce a strict iterative decision loop with consistent user checkpoints and ex
13
13
  - Completing an autonomous run
14
14
 
15
15
  ## Default 5-Option Structure
16
- 1. **Recommended Development Path** (mark with star as most future-proof)
16
+ 1. **Recommended Development Path** (mark with as most future-proof)
17
17
  2. **Alternative Development Path A**
18
18
  3. **Alternative Development Path B**
19
19
  4. **Freeform** — set enum value to `custom` (allows free-text input)
20
20
  5. **Autonomous Mode**
21
21
 
22
22
  ## Escalation (Compatibility Triad)
23
- Use ONLY when there is a concrete compatibility/contract impact:
23
+ Use ONLY when there is a concrete compatibility/contract impact (see breaking-change-paths skill):
24
24
  1. Non-Breaking Path
25
25
  2. Breaking Path
26
26
  3. Alternative Structural Path
@@ -31,17 +31,62 @@ Use ONLY when there is a concrete compatibility/contract impact:
31
31
  Each option must include:
32
32
  `<Title> — Why: <reason> | Leads to: <next step> | Risk: <low|medium|high>`
33
33
 
34
+ ### OneCrawl Example Cards
35
+ ```
36
+ ⭐ 1. Add new --timeout flag (non-breaking)
37
+ Why: Users need session timeouts | Leads to: CLI + daemon changes | Risk: low
38
+
39
+ 2. Replace --headless with --head (breaking)
40
+ Why: Simpler flag name | Leads to: CLI + docs + migration | Risk: high
41
+
42
+ 3. Config-only timeout (no CLI flag)
43
+ Why: Minimal surface change | Leads to: config.toml + docs | Risk: low
44
+
45
+ 4. Freeform — describe your preferred approach
46
+
47
+ 5. Autonomous Mode — I'll implement the recommended path and report back
48
+ ```
49
+
50
+ ## Copilot CLI Implementation
51
+
52
+ In Copilot CLI runtime, the interaction loop uses natural language responses (not tool calls).
53
+ The agent presents options as a numbered list in the response and the user replies with their choice.
54
+
55
+ ### Autonomous Mode Behavior
56
+ When the user selects Autonomous Mode (option 5):
57
+ 1. Agent implements the recommended path (option 1) without further prompts.
58
+ 2. Follows all applicable skills (planning-tracking, testing-policy, completion-gate).
59
+ 3. On completion, presents a summary with:
60
+ - What was done
61
+ - Test results (exact counts)
62
+ - Files changed
63
+ - Asks: "Rate this result (1-5), suggest next action, or say 'I am satisfied'."
64
+ 4. Loop continues until the exact phrase **"I am satisfied"** is received.
65
+
66
+ ### Decision Tracking
67
+ Record each decision point in the session database:
68
+ ```sql
69
+ INSERT INTO session_state (key, value) VALUES
70
+ ('decision_1', 'Option 1: Add --timeout flag (non-breaking) — user chose at 2024-01-15T10:30Z');
71
+ ```
72
+
34
73
  ## Rules
35
74
  - One clear question per iteration
36
75
  - Exactly 5 options every time
37
- - Option 4 is ALWAYS Freeform (hard invariant)
76
+ - Option 4 is ALWAYS Freeform (hard invariant — label "Freeform", enum value `custom`)
38
77
  - Options 1-3 carry forward from previous iteration unless explicitly replaced
39
- - Mark recommendation with star; explain changes
78
+ - Mark recommendation with ⭐; explain if recommendation changes between iterations
40
79
  - At autonomous completion: ask for rating, next action, satisfaction
41
- - Loop stops ONLY on exact phrase: "I am satisfied"
80
+ - Loop stops ONLY on exact phrase: **"I am satisfied"**
81
+
82
+ ## Done Criteria
83
+ - User has explicitly ended the loop with "I am satisfied".
84
+ - All decisions are documented in session log.
42
85
 
43
86
  ## Anti-patterns
44
87
  - Multi-question prompts in one iteration
45
- - Missing or renaming Freeform option
88
+ - Missing or renaming Freeform option (option 4)
46
89
  - Stopping without explicit "I am satisfied"
47
- - Using compatibility triad as default (it's an escalation)
90
+ - Using compatibility triad as default (it's an escalation for breaking changes)
91
+ - Implementing without presenting options first
92
+ - Changing recommendation without explaining why
@@ -19,42 +19,85 @@ interface Milestone { id: string; description: string; priority: "critical"|"hig
19
19
  interface Issue { id: string; task: string; priority: "critical"|"high"|"medium"|"low"; status: "todo"|"in_progress"|"review"|"done"|"blocked"; depends_on: string[]; children: Record<string, Issue>; }
20
20
  ```
21
21
 
22
+ ## SQL-Based Tracking
23
+
24
+ Use the session database for operational tracking:
25
+
26
+ ```sql
27
+ -- Create plan items
28
+ INSERT INTO todos (id, title, description, status) VALUES
29
+ ('m1-cdp-refactor', 'Refactor CDP error handling', 'Update onecrawl-cdp error types to use thiserror, propagate to onecrawl-mcp-rs', 'pending'),
30
+ ('m1-cli-flags', 'Add --timeout flag', 'Add session timeout to onecrawl-cli-rs session start', 'pending'),
31
+ ('m1-tests', 'Add timeout tests', 'Unit tests for session timeout in cli-rs and cdp', 'pending');
32
+
33
+ -- Declare dependencies
34
+ INSERT INTO todo_deps (todo_id, depends_on) VALUES
35
+ ('m1-cli-flags', 'm1-cdp-refactor'), -- CLI depends on CDP changes
36
+ ('m1-tests', 'm1-cli-flags'); -- Tests depend on implementation
37
+
38
+ -- Find ready items (no pending dependencies)
39
+ SELECT t.id, t.title FROM todos t
40
+ WHERE t.status = 'pending'
41
+ AND NOT EXISTS (
42
+ SELECT 1 FROM todo_deps td
43
+ JOIN todos dep ON td.depends_on = dep.id
44
+ WHERE td.todo_id = t.id AND dep.status != 'done'
45
+ );
46
+
47
+ -- Update status as work progresses
48
+ UPDATE todos SET status = 'in_progress' WHERE id = 'm1-cdp-refactor';
49
+ UPDATE todos SET status = 'done' WHERE id = 'm1-cdp-refactor';
50
+ ```
51
+
22
52
  ## OneCrawl Workspace Parallelism Rules
23
53
 
24
54
  Safe to parallelize:
25
- - Changes in different crates (e.g., `onecrawl-cdp` + `onecrawl-parser`)
55
+ - Changes in independent crates (e.g., `onecrawl-crypto` + `onecrawl-parser`)
26
56
  - NAPI + PyO3 bindings (independent build targets)
27
57
  - Documentation + code changes
58
+ - Tests in different crates (with `--test-threads=1` per crate)
28
59
 
29
60
  NOT safe to parallelize:
30
61
  - Changes in a crate + its dependents (e.g., `onecrawl-cdp` + `onecrawl-cli-rs`)
31
62
  - Multiple changes to the same file
32
- - Version bumps (must be sequential: Cargo.toml then package.json)
63
+ - Version bumps (must be sequential: Cargo.toml package.json → tag)
64
+ - Any two changes that touch session state or `/tmp/onecrawl-session*.json`
33
65
 
34
66
  ## OneCrawl Crate Dependency Graph
35
67
  ```
36
- onecrawl-cli-rs
37
- -> onecrawl-mcp-rs -> onecrawl-cdp -> onecrawl-browser
38
- -> onecrawl-server -> onecrawl-cdp
39
- -> onecrawl-core
40
- -> onecrawl-crypto
41
- -> onecrawl-parser
42
- -> onecrawl-storage
68
+ onecrawl-cli-rs (binary)
69
+ ├── onecrawl-mcp-rs ──→ onecrawl-cdp ──→ onecrawl-browser (vendor)
70
+ ├── onecrawl-server ──→ onecrawl-cdp
71
+ ├── onecrawl-core
72
+ ├── onecrawl-crypto
73
+ ├── onecrawl-parser
74
+ └── onecrawl-storage
75
+
76
+ Independent leaf crates (safe to change in parallel):
77
+ onecrawl-core, onecrawl-crypto, onecrawl-parser, onecrawl-storage
78
+
79
+ High-impact crates (changes cascade):
80
+ onecrawl-cdp (affects: mcp-rs, server, cli-rs)
81
+ onecrawl-browser (affects: cdp, mcp-rs, server, cli-rs)
43
82
  ```
44
83
 
45
84
  ## Procedure
46
- 1. Build plan before implementation.
47
- 2. Assign unique IDs to milestones/issues.
48
- 3. Declare dependencies for every issue.
49
- 4. Execute by dependency order, then priority (critical then high then medium then low).
50
- 5. Run independent same-priority milestones in parallel when safe.
51
- 6. Update statuses continuously.
85
+ 1. Build plan before implementation (use SQL todos).
86
+ 2. Assign unique IDs to milestones/issues (kebab-case: `m1-feature-name`).
87
+ 3. Declare dependencies using `todo_deps` table.
88
+ 4. Execute by dependency order, then priority (critical high medium low).
89
+ 5. Run independent same-priority items in parallel when safe per rules above.
90
+ 6. Update statuses continuously (`pending` → `in_progress` → `done`).
91
+ 7. Sync to GitHub when milestones complete (github-sync skill).
52
92
 
53
93
  ## Done Criteria
54
- - Plan exists, is up-to-date, and reflects actual execution state.
55
- - Dependencies are respected and parallel work is safe.
94
+ - Plan exists in SQL todos, is up-to-date, and reflects actual execution state.
95
+ - Dependencies are respected and parallel work follows safety rules.
96
+ - GitHub artifacts are synced (github-sync skill).
56
97
 
57
98
  ## Anti-patterns
58
99
  - Starting implementation without a plan
59
- - Missing dependency declarations
100
+ - Missing dependency declarations (causes parallel conflicts)
60
101
  - Running blocked items in parallel
102
+ - Changing `onecrawl-cdp` and `onecrawl-cli-rs` simultaneously
103
+ - Using free-form notes instead of structured SQL tracking