@manhquy/flow-skill 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +11 -8
- package/README_VN.md +7 -4
- package/package.json +1 -1
- package/skills/flow/SKILL.md +1 -1
- package/skills/flow/harness/GAP-MATRIX-0.1.17.md +4 -3
- package/skills/flow/harness/README.md +29 -2
- package/skills/flow/harness/_db.py +175 -6
- package/skills/flow/harness/flow_harness.py +165 -9
- package/skills/flow/harness/graph_executor.py +1205 -0
- package/skills/flow/harness/graph_ids.py +35 -0
- package/skills/flow/harness/graph_predicates.py +48 -0
- package/skills/flow/harness/pins/flow-topology.sha256 +1 -0
- package/skills/flow/harness/schema/014-graph-executor.sql +67 -0
- package/skills/flow/references/auto-run.md +29 -2
- package/skills/flow/references/command-dispatch.md +1 -0
- package/skills/flow/references/concierge.md +1 -0
- package/skills/flow/references/flow-topology.json +146 -0
- package/skills/flow/runner/flow.sh +216 -5
- package/skills-manifest.json +8 -2
package/README.md
CHANGED
|
@@ -10,20 +10,24 @@ One-command installer that copies the [flow](https://github.com/manhquydev/flow-
|
|
|
10
10
|
## Install
|
|
11
11
|
|
|
12
12
|
```
|
|
13
|
-
#
|
|
13
|
+
# Stable (latest) — current: v0.1.0, ships skill v0.24.0
|
|
14
|
+
npx @manhquy/flow-skill
|
|
15
|
+
|
|
16
|
+
# Pre-release channel (opt-in)
|
|
14
17
|
npx @manhquy/flow-skill@rc
|
|
15
18
|
```
|
|
16
19
|
|
|
17
20
|
An interactive prompt asks which agents to install to. Pick one or more, confirm, done.
|
|
18
21
|
|
|
19
|
-
> **
|
|
22
|
+
> **Stable is on `latest`**: bare `npx @manhquy/flow-skill` (or `@0.1.x`) resolves the GA release.
|
|
23
|
+
> Pin `@rc` only if you want the pre-release channel. See [SECURITY.md](./SECURITY.md).
|
|
20
24
|
>
|
|
21
25
|
> **`npm i` alone is not enough.** `npm install @manhquy/flow-skill` only adds the installer
|
|
22
|
-
> package to `node_modules`. You must **run** it (`npx @manhquy/flow-skill
|
|
26
|
+
> package to `node_modules`. You must **run** it (`npx @manhquy/flow-skill` or
|
|
23
27
|
> `npx flow-skill` after install) to copy the skill into agent homes.
|
|
24
28
|
>
|
|
25
|
-
> **Two version axes:** package `version` in this folder is the **installer** (e.g. `0.1.0
|
|
26
|
-
> Skill product version lives in `skills/flow/SKILL.md` (`metadata.version`, e.g. `0.
|
|
29
|
+
> **Two version axes:** package `version` in this folder is the **installer** (e.g. `0.1.0`).
|
|
30
|
+
> Skill product version lives in `skills/flow/SKILL.md` (`metadata.version`, e.g. `0.24.0`).
|
|
27
31
|
> `--help` and the JSONL `plan` event expose both (`version` + `skillVersion`).
|
|
28
32
|
|
|
29
33
|
## Non-interactive
|
|
@@ -93,8 +97,7 @@ rm -rf <project>/.claude/skills/flow
|
|
|
93
97
|
|
|
94
98
|
- **Windows `EBUSY` / `EPERM` mid-install**: an agent (Claude Code, Codex, Antigravity IDE) is holding a file inside the destination. Close the agent and re-run. The installer already retries with 100/300/900 ms backoff before surfacing the error.
|
|
95
99
|
- **Stale advisory lock**: a prior run crashed. The next run detects the dead PID and reclaims the lock automatically. If it does not (very rare — the recorded PID was recycled by another live process), delete `<parent-of-dest>/.flow-skill.installing.lock`.
|
|
96
|
-
- **`No matching version found` on `@0.
|
|
97
|
-
- **`No matching version found` on `@0.22.0`**: that is the **skill product** version, not the npm package version. Use `@rc` / `@0.1.0-rc.N` (installer) — skill content version is printed by `--help` as `ships skill v…`.
|
|
100
|
+
- **`No matching version found` on `@0.24.0`**: that is the **skill product** version, not the npm package version. Install the installer by its own version (bare / `@0.1.x` / `@rc`) — skill content version is printed by `--help` as `ships skill v…`.
|
|
98
101
|
- **Installed package but agent has no `/flow`**: `npm i` does not run the installer. Run `npx @manhquy/flow-skill@rc` (or `npx flow-skill` if already installed) and restart the agent.
|
|
99
102
|
- **Node too old** (`requires Node.js >=22.14.0`): upgrade with your preferred version manager (`nvm install 22`, `fnm install 22`, or Node's official installer). Node 20 reached end-of-life April 2026; npm OIDC Trusted Publishing needs npm >=11.5.1 which bundles with Node 22.14+.
|
|
100
103
|
|
|
@@ -103,7 +106,7 @@ rm -rf <project>/.claude/skills/flow
|
|
|
103
106
|
`--json` streams one JSON object per line:
|
|
104
107
|
|
|
105
108
|
```jsonl
|
|
106
|
-
{"event":"plan","version":"0.1.0
|
|
109
|
+
{"event":"plan","version":"0.1.0","skillVersion":"0.24.0","dryRun":false,"scope":"global","targets":["claude","codex"]}
|
|
107
110
|
{"event":"install:start","target":"claude","dests":["~/.claude/skills/flow"]}
|
|
108
111
|
{"event":"install:done","target":"claude","dests":["~/.claude/skills/flow"],"result":"success","error":null,"warnings":[]}
|
|
109
112
|
{"event":"summary","success":true,"total":2,"attempted":2,"installed":2,"failed":0,"skipped":0,"aborted":false}
|
package/README_VN.md
CHANGED
|
@@ -10,15 +10,18 @@ Trình cài đặt một-lệnh sao chép skill [flow](https://github.com/manhqu
|
|
|
10
10
|
## Cài đặt
|
|
11
11
|
|
|
12
12
|
```
|
|
13
|
-
#
|
|
13
|
+
# Stable (latest) — hiện tại: v0.1.0, ships skill v0.24.0
|
|
14
|
+
npx @manhquy/flow-skill
|
|
15
|
+
|
|
16
|
+
# Kênh pre-release (tùy chọn)
|
|
14
17
|
npx @manhquy/flow-skill@rc
|
|
15
18
|
```
|
|
16
19
|
|
|
17
20
|
Prompt tương tác hỏi cài vào agent nào. Chọn một hoặc nhiều, xác nhận, xong.
|
|
18
21
|
|
|
19
|
-
> **
|
|
22
|
+
> **Stable ở `latest`**: bare `npx @manhquy/flow-skill` (hoặc `@0.1.x`) lấy bản GA. Pin `@rc` nếu muốn kênh pre-release. Xem [SECURITY.md](./SECURITY.md).
|
|
20
23
|
>
|
|
21
|
-
> **`npm i` alone chưa đủ** — phải **chạy** CLI (`npx @manhquy/flow-skill
|
|
24
|
+
> **`npm i` alone chưa đủ** — phải **chạy** CLI (`npx @manhquy/flow-skill`). Hai trục version: package npm vs skill product (`ships skill v…` / `skillVersion` trong JSONL).
|
|
22
25
|
|
|
23
26
|
## Non-interactive
|
|
24
27
|
|
|
@@ -97,7 +100,7 @@ Hợp đồng additive trong `0.1.x` — field mới có thể thêm, field cũ
|
|
|
97
100
|
|
|
98
101
|
- **`EBUSY`/`EPERM` giữa install trên Windows**: một agent đang giữ handle vào file trong destination. Đóng agent + re-run. Installer đã retry 100/300/900 ms trước khi báo error.
|
|
99
102
|
- **Advisory lock cũ**: run trước crash. Run mới tự phát hiện PID dead → reclaim. Trường hợp hiếm (PID được recycle bởi process khác đang chạy): xóa `<parent-of-dest>/.flow-skill.installing.lock`.
|
|
100
|
-
- **`No matching version found` với `@0.
|
|
103
|
+
- **`No matching version found` với `@0.24.0`**: đó là version **skill product**, không phải version npm package. Cài installer theo version của nó (bare / `@0.1.x` / `@rc`) — version skill in ra ở `--help` (`ships skill v…`).
|
|
101
104
|
- **Node quá cũ** (`requires Node.js >=22.14.0`): update bằng `nvm install 22`, `fnm install 22`, hoặc installer chính thức của Node. Node 20 hết vòng đời từ 4/2026; npm OIDC Trusted Publishing cần npm >=11.5.1 (bundle theo Node 22.14+).
|
|
102
105
|
|
|
103
106
|
## Yêu cầu
|
package/package.json
CHANGED
package/skills/flow/SKILL.md
CHANGED
|
@@ -8,7 +8,7 @@ keywords: [flow, buildflow, gate, build, ship, scope, prd, contract, card, deplo
|
|
|
8
8
|
license: MIT
|
|
9
9
|
metadata:
|
|
10
10
|
author: flow-skill
|
|
11
|
-
version: "0.
|
|
11
|
+
version: "0.25.0"
|
|
12
12
|
attribution: "Methodology from ai20k-build-phase/buildflow (Tony, arealisticdreamer.com); harness/agent layers from repository-harness, claudekit-engineer, BMAD-METHOD. v0.22 concierge routing pattern adapted from BMAD-METHOD's bmad-help; forge-idea ritual adapted from BMAD-METHOD's bmad-forge-idea (both MIT, BMad Code LLC)."
|
|
13
13
|
---
|
|
14
14
|
|
|
@@ -14,7 +14,8 @@ Supersedes: brainstorm 260703 Thread1 (“nothing to upgrade”).
|
|
|
14
14
|
| 005 | present | present | File hash may differ on comments only — do not overclaim DDL fork |
|
|
15
15
|
| 006–008 | **absent** (reserved gap) | changesets / deps / hierarchy | Not adopted this plan |
|
|
16
16
|
| 009–012 | **usage/accessed** (flow-only) | improvement identity / links / closure | **Semantic collision** — same numbers, different DDL |
|
|
17
|
-
| 013 | absent | changeset content sha | Not adopted |
|
|
17
|
+
| 013 | absent | changeset content sha | Not adopted — stays reserved/absent |
|
|
18
|
+
| 014+ | **graph-executor** (flow-owned band) | absent | Flow-owned since the 2026-07-26 supersession (plan `260726-1718-harness-graph-executor-langgraph-port`) |
|
|
18
19
|
|
|
19
20
|
## Commands / invariants
|
|
20
21
|
|
|
@@ -25,7 +26,7 @@ Supersedes: brainstorm 260703 Thread1 (“nothing to upgrade”).
|
|
|
25
26
|
| `story verify` | optional shell verify_command | required for shell-proven complete | shell=True = operator-authored only |
|
|
26
27
|
| `query sql` | **not exposed** | read-only connection | Do not add mutating SQL |
|
|
27
28
|
| Rust forward | refused on flow-lineage DB | n/a | `FLOW_HARNESS_BACKEND=rust` → exit 2 when usage mirror / schema≥9 |
|
|
28
|
-
| Changesets / work-graph |
|
|
29
|
+
| Changesets / work-graph | **flow-owned graph executor** (schema band 014+) | 006–013 | **Superseded 2026-07-26** (was: out of scope, FOMO red line) — flow builds its own work-graph; upstream 006–013 will NOT be adopted. Decision record: plan `260726-1718-harness-graph-executor-langgraph-port` |
|
|
29
30
|
| `query contract --json` | not on Python CLI | protocol v1 discovery | Optional external binary smoke only |
|
|
30
31
|
|
|
31
32
|
## Pins (consumer)
|
|
@@ -38,4 +39,4 @@ DO_NOT_USE = harness-cli-v0.1.16 # no assets / failed promotion
|
|
|
38
39
|
|
|
39
40
|
## Rust refuse-forward
|
|
40
41
|
|
|
41
|
-
Any DB with `usage_event` or `MAX(schema_version) >= 9` is **flow-lineage**. Python entrypoint must refuse forwarding to external `harness-cli` (exit 2).
|
|
42
|
+
Any DB with `usage_event` or `MAX(schema_version) >= 9` is **flow-lineage**. Python entrypoint must refuse forwarding to external `harness-cli` (exit 2). `graph` verbs are NEVER forwarded regardless of DB state (flow-only surface, band 014+). The former "schema re-home ≥014 (separate epic)" note is superseded 2026-07-26: 014+ is now the flow-owned graph-executor band; 013 remains reserved/absent.
|
|
@@ -32,6 +32,30 @@ message). Use the Python backend for flow projects. The seam stays in code as a
|
|
|
32
32
|
power-path for non-flow DBs; flow does not build or ship the binary.
|
|
33
33
|
|
|
34
34
|
Disable the durable layer entirely with `FLOW_HARNESS_DISABLE=1` (engine still runs).
|
|
35
|
+
**Python floor: 3.7** (measured, not asserted — `subprocess.run(capture_output=, text=)`;
|
|
36
|
+
no walrus/match/builtin generics). `scripts/release-preflight.sh` enforces it, and
|
|
37
|
+
`docs/migration-v1.md` records what a python-mandatory v1 would change.
|
|
38
|
+
|
|
39
|
+
**Topology pin.** `pins/flow-topology.sha256` pins the canonical-JSON sha256 of
|
|
40
|
+
`references/flow-topology.json`; graph verbs refuse to run on a mismatch (topology is
|
|
41
|
+
executable-adjacent data). After an INTENTIONAL topology edit, run `graph lint` and
|
|
42
|
+
regenerate from the repo root:
|
|
43
|
+
|
|
44
|
+
```
|
|
45
|
+
PY="$(command -v python3 || command -v python)" # same cascade flow.sh _python() uses
|
|
46
|
+
"$PY" -c "import json,hashlib;print(hashlib.sha256(json.dumps(json.load(open('skills/flow/references/flow-topology.json')),sort_keys=True,separators=(',',':')).encode()).hexdigest()+' flow-topology.json')" > skills/flow/harness/pins/flow-topology.sha256
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
**DB location.** `<FLOW_PROJECT_ROOT>/.flow/harness.db`, with two graph-era rules: a root inside
|
|
50
|
+
a LINKED git worktree is translated to its main-worktree equivalent (all parallel cards share one
|
|
51
|
+
DB). Not translated — each worktree keeps its own DB — for submodules and for
|
|
52
|
+
`--separate-git-dir` repos, where git reports the separate git dir as the main worktree and the
|
|
53
|
+
checkout path is not discoverable; guessing there would put durable state inside `.git`.
|
|
54
|
+
`FLOW_HARNESS_DB=<path>`
|
|
55
|
+
is a narrow test/tool override of the DB path only — never repurpose `FLOW_PROJECT_ROOT` for
|
|
56
|
+
this. Worktree telemetry merges at workspace-remove time via
|
|
57
|
+
`rollup --src <sink> --src-key "<main events.jsonl>#<branch>#<created_at>"` (lifecycle-unique;
|
|
58
|
+
prefixed with the destination sink so `usage`/`prune` see the rows).
|
|
35
59
|
|
|
36
60
|
### STRICT durable writes (`flow.sh`)
|
|
37
61
|
|
|
@@ -147,5 +171,8 @@ stage_from→to, card, project_type, mode, flow_version, tier, host, read_only.
|
|
|
147
171
|
- `schema/00N-*.sql` — DDL. **001-005 are a faithful port of repository-harness** (005 =
|
|
148
172
|
tool-extensions, kind-aware tool registry). **009-012 are flow-specific** (accessed-count +
|
|
149
173
|
usage-log mirror), re-homed off 005-008 so the upstream 005 number is free and the lineages no
|
|
150
|
-
longer collide.
|
|
151
|
-
|
|
174
|
+
longer collide. **014+ is the flow-owned graph-executor band** (2026-07-26 supersession of the
|
|
175
|
+
work-graph red line — see GAP-MATRIX; 013 stays reserved/absent). Migrations are
|
|
176
|
+
column-idempotent and safe to re-run; an old DB built under the pre-005 numbering is
|
|
177
|
+
reconciled automatically on the next `init`.
|
|
178
|
+
- `graph_ids.py` — monotonic, lexicographically sortable ids for graph-executor rows (stdlib).
|
|
@@ -1,19 +1,113 @@
|
|
|
1
1
|
"""SQLite durable layer for the flow harness (stdlib sqlite3 only).
|
|
2
2
|
|
|
3
3
|
Schema lives in schema/00N-*.sql (001-005 a faithful port of repository-harness; 009-012
|
|
4
|
-
flow-specific: accessed-count + usage-log mirror
|
|
5
|
-
migration bumps schema_version and statements are
|
|
6
|
-
DB path defaults to
|
|
4
|
+
flow-specific: accessed-count + usage-log mirror; 014+ flow-owned graph-executor band) and is
|
|
5
|
+
applied by missing-version set; each migration bumps schema_version and statements are
|
|
6
|
+
idempotent, so init/upgrade is safe to re-run. DB path defaults to
|
|
7
|
+
<FLOW_PROJECT_ROOT>/.flow/harness.db, translated to the MAIN worktree when the root sits
|
|
8
|
+
inside a linked git worktree so every parallel card shares one DB.
|
|
7
9
|
"""
|
|
8
10
|
|
|
11
|
+
import contextlib
|
|
9
12
|
import os
|
|
10
13
|
import re
|
|
11
14
|
import sqlite3
|
|
15
|
+
import subprocess
|
|
16
|
+
import time
|
|
12
17
|
|
|
13
18
|
SCHEMA_DIR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "schema")
|
|
14
19
|
|
|
15
20
|
|
|
21
|
+
_worktree_main_cache = {}
|
|
22
|
+
|
|
23
|
+
|
|
24
|
+
def _git_env():
|
|
25
|
+
"""Strip inherited GIT_DIR/GIT_WORK_TREE (exported by every git hook, `rebase -x`,
|
|
26
|
+
`bisect run`, `submodule foreach`): with them set, git answers about a FOREIGN repo
|
|
27
|
+
and the worktree translation silently splits the journal."""
|
|
28
|
+
env = dict(os.environ)
|
|
29
|
+
for k in ("GIT_DIR", "GIT_WORK_TREE", "GIT_COMMON_DIR", "GIT_INDEX_FILE",
|
|
30
|
+
"GIT_OBJECT_DIRECTORY", "GIT_ALTERNATE_OBJECT_DIRECTORIES"):
|
|
31
|
+
env.pop(k, None)
|
|
32
|
+
return env
|
|
33
|
+
|
|
34
|
+
|
|
35
|
+
def _linked_worktree_main_root(root):
|
|
36
|
+
"""Map `root` inside a LINKED git worktree to its main-worktree equivalent path, else None.
|
|
37
|
+
|
|
38
|
+
Card worktrees are full checkouts (they contain flow/ + cards/), so without this
|
|
39
|
+
translation every parallel card mints its own throwaway .flow/harness.db that
|
|
40
|
+
`git worktree remove` then deletes. Keys on the RESOLVED ROOT (never CWD — flow.sh
|
|
41
|
+
passes FLOW_PROJECT_ROOT per call and the two are independent). Cached per process:
|
|
42
|
+
this runs on every connect and an uncached `git` spawn multiplies on Windows.
|
|
43
|
+
Any failure (git absent, git < 2.31, not a repo) returns None = current behavior.
|
|
44
|
+
"""
|
|
45
|
+
if root in _worktree_main_cache:
|
|
46
|
+
return _worktree_main_cache[root]
|
|
47
|
+
out = None
|
|
48
|
+
try:
|
|
49
|
+
r = subprocess.run(
|
|
50
|
+
["git", "-C", root, "rev-parse", "--path-format=absolute",
|
|
51
|
+
"--git-dir", "--git-common-dir", "--show-toplevel"],
|
|
52
|
+
capture_output=True, text=True, timeout=10, env=_git_env(),
|
|
53
|
+
)
|
|
54
|
+
git_dir = common_dir = worktree_top = None
|
|
55
|
+
if r.returncode == 0:
|
|
56
|
+
vals = [ln.strip() for ln in r.stdout.splitlines() if ln.strip()]
|
|
57
|
+
if len(vals) >= 3:
|
|
58
|
+
git_dir, common_dir, worktree_top = vals[0], vals[1], vals[2]
|
|
59
|
+
elif "not a git repository" not in (r.stderr or "").lower():
|
|
60
|
+
# git < 2.31 lacks --path-format (declared floor); best-effort re-derive.
|
|
61
|
+
# Gated on the failure NOT being "not a repo" so non-git roots cost one spawn.
|
|
62
|
+
def _rp(flag):
|
|
63
|
+
q = subprocess.run(["git", "-C", root, "rev-parse", flag], env=_git_env(),
|
|
64
|
+
capture_output=True, text=True, timeout=10)
|
|
65
|
+
return q.stdout.strip() if q.returncode == 0 and q.stdout.strip() else None
|
|
66
|
+
git_dir = _rp("--absolute-git-dir")
|
|
67
|
+
common_dir = _rp("--git-common-dir")
|
|
68
|
+
worktree_top = _rp("--show-toplevel")
|
|
69
|
+
if common_dir and not os.path.isabs(common_dir):
|
|
70
|
+
common_dir = os.path.abspath(os.path.join(root, common_dir))
|
|
71
|
+
if git_dir and common_dir and worktree_top:
|
|
72
|
+
# Linked worktree iff the per-worktree git dir differs from the shared common
|
|
73
|
+
# dir. Submodules and --separate-git-dir repos have git_dir == common_dir (both
|
|
74
|
+
# live OUTSIDE the worktree) and must never be translated - dirname(common_dir)
|
|
75
|
+
# is git internals there, not a project root. realpath everywhere: macOS mktemp
|
|
76
|
+
# and symlinked checkouts otherwise fail the containment compare silently.
|
|
77
|
+
if os.path.realpath(git_dir) != os.path.realpath(common_dir):
|
|
78
|
+
wt = subprocess.run(["git", "-C", root, "worktree", "list", "--porcelain"], env=_git_env(),
|
|
79
|
+
capture_output=True, text=True, timeout=10)
|
|
80
|
+
first = next((ln for ln in wt.stdout.splitlines()
|
|
81
|
+
if ln.startswith("worktree ")), "") if wt.returncode == 0 else ""
|
|
82
|
+
main_top = first[len("worktree "):].strip() # main worktree is always first
|
|
83
|
+
# --separate-git-dir: the main `.git` is a FILE, so git itself reports the
|
|
84
|
+
# separate git dir as the "main worktree" and does not record the checkout
|
|
85
|
+
# anywhere discoverable (core.worktree is unset). Translating would put the
|
|
86
|
+
# DB inside git internals, so REFUSE instead of guessing: the worktree keeps
|
|
87
|
+
# its own DB (pre-graph behavior), documented in README, rather than
|
|
88
|
+
# corrupting state. A real checkout always has a `.git` entry.
|
|
89
|
+
if main_top and not os.path.exists(os.path.join(main_top, ".git")):
|
|
90
|
+
main_top = ""
|
|
91
|
+
rr = os.path.realpath(root)
|
|
92
|
+
wtop = os.path.realpath(worktree_top)
|
|
93
|
+
inside = rr == wtop or rr.startswith(wtop + os.sep)
|
|
94
|
+
if main_top and inside:
|
|
95
|
+
main_top = os.path.realpath(main_top)
|
|
96
|
+
rel = os.path.relpath(rr, wtop)
|
|
97
|
+
out = main_top if rel == "." else os.path.join(main_top, rel)
|
|
98
|
+
except Exception:
|
|
99
|
+
out = None # best-effort: never break the python path over the resolver
|
|
100
|
+
_worktree_main_cache[root] = out
|
|
101
|
+
return out
|
|
102
|
+
|
|
103
|
+
|
|
16
104
|
def default_db_path(root=None):
|
|
105
|
+
# Narrow override for tests/tools: DB path ONLY. Never FLOW_PROJECT_ROOT — that is
|
|
106
|
+
# the global root override and repurposing it would hijack CARDS_DIR/lock/DEBT
|
|
107
|
+
# resolution for every flow.sh call in a worktree.
|
|
108
|
+
override = os.environ.get("FLOW_HARNESS_DB")
|
|
109
|
+
if override:
|
|
110
|
+
return override
|
|
17
111
|
root = root or os.environ.get("FLOW_PROJECT_ROOT") or os.getcwd()
|
|
18
112
|
# On native Windows Python, translate a Git Bash POSIX root like /c/proj -> c:/proj
|
|
19
113
|
# so the db lands under the project (not relative to the current drive root). Only
|
|
@@ -22,6 +116,9 @@ def default_db_path(root=None):
|
|
|
22
116
|
m = re.match(r"^/([a-zA-Z])/(.*)$", root)
|
|
23
117
|
if m:
|
|
24
118
|
root = m.group(1) + ":/" + m.group(2)
|
|
119
|
+
main_equiv = _linked_worktree_main_root(root)
|
|
120
|
+
if main_equiv:
|
|
121
|
+
root = main_equiv
|
|
25
122
|
return os.path.join(root, ".flow", "harness.db")
|
|
26
123
|
|
|
27
124
|
|
|
@@ -100,6 +197,9 @@ def connect(db_path=None, root=None, auto_migrate=True):
|
|
|
100
197
|
con = sqlite3.connect(db_path)
|
|
101
198
|
con.row_factory = sqlite3.Row
|
|
102
199
|
con.execute("PRAGMA foreign_keys = ON")
|
|
200
|
+
# Explicit 5s wait for the write lock: two parallel card workers share one DB
|
|
201
|
+
# (worktree-translated path), so contention must queue, not raise `database is locked`.
|
|
202
|
+
con.execute("PRAGMA busy_timeout = 5000")
|
|
103
203
|
if auto_migrate:
|
|
104
204
|
migrate(con)
|
|
105
205
|
return con
|
|
@@ -126,9 +226,25 @@ def _apply_migration(con, sql):
|
|
|
126
226
|
pragmas = [s for s in stmts if s.upper().startswith("PRAGMA")]
|
|
127
227
|
ddl = [s for s in stmts if not s.upper().startswith("PRAGMA")]
|
|
128
228
|
for p in pragmas:
|
|
129
|
-
|
|
229
|
+
# Belt-and-braces retry (a pragma can still meet a busy DB on a cold start).
|
|
230
|
+
for attempt in range(20):
|
|
231
|
+
try:
|
|
232
|
+
con.execute(p)
|
|
233
|
+
break
|
|
234
|
+
except sqlite3.OperationalError as e:
|
|
235
|
+
if "locked" not in str(e).lower() and "busy" not in str(e).lower():
|
|
236
|
+
raise
|
|
237
|
+
if attempt == 19:
|
|
238
|
+
raise
|
|
239
|
+
time.sleep(0.05 * (attempt + 1))
|
|
130
240
|
try:
|
|
131
|
-
|
|
241
|
+
# IMMEDIATE, not deferred: a deferred BEGIN takes SHARED on the first read probe
|
|
242
|
+
# (_idempotent_statement inspects sqlite_master/table_info) and the later DDL must
|
|
243
|
+
# upgrade SHARED->RESERVED - an upgrade SQLite refuses with SQLITE_BUSY IMMEDIATELY,
|
|
244
|
+
# bypassing the busy handler, so busy_timeout never applies. Taking RESERVED up
|
|
245
|
+
# front puts the wait where busy_timeout can cover it. This is what made cold-start
|
|
246
|
+
# parallel card workers lose records.
|
|
247
|
+
con.execute("BEGIN IMMEDIATE")
|
|
132
248
|
for s in ddl:
|
|
133
249
|
stmt = _idempotent_statement(con, s)
|
|
134
250
|
if stmt is None:
|
|
@@ -136,7 +252,10 @@ def _apply_migration(con, sql):
|
|
|
136
252
|
con.execute(stmt)
|
|
137
253
|
con.execute("COMMIT")
|
|
138
254
|
except Exception:
|
|
139
|
-
|
|
255
|
+
# rollback(), not execute("ROLLBACK"): if BEGIN IMMEDIATE itself failed there is no
|
|
256
|
+
# active transaction, and the raw statement would raise "cannot rollback", masking
|
|
257
|
+
# the original lock error.
|
|
258
|
+
con.rollback()
|
|
140
259
|
raise
|
|
141
260
|
|
|
142
261
|
|
|
@@ -219,6 +338,56 @@ def update(con, table, id_col, id_val, **cols):
|
|
|
219
338
|
return cur.rowcount
|
|
220
339
|
|
|
221
340
|
|
|
341
|
+
def update_where(con, table, where, **cols):
|
|
342
|
+
# SECURITY INVARIANT: `table` and every key in `where`/`cols` must be code literals,
|
|
343
|
+
# never user input. Composite-key variant of update() for the graph tables whose
|
|
344
|
+
# primary keys span (execution_id, ns, ...); update()'s single id_col cannot address them.
|
|
345
|
+
sets, vals = [], []
|
|
346
|
+
for k, v in cols.items():
|
|
347
|
+
if v is not None:
|
|
348
|
+
sets.append(f"{k} = ?")
|
|
349
|
+
vals.append(v)
|
|
350
|
+
if not sets:
|
|
351
|
+
return 0
|
|
352
|
+
conds = []
|
|
353
|
+
for k, v in where.items():
|
|
354
|
+
if v is None:
|
|
355
|
+
# `col = NULL` never matches in SQL - a None key would silently update nothing.
|
|
356
|
+
raise ValueError(f"update_where: where[{k!r}] is None")
|
|
357
|
+
conds.append(f"{k} = ?")
|
|
358
|
+
vals.append(v)
|
|
359
|
+
sql = f"UPDATE {table} SET {', '.join(sets)} WHERE {' AND '.join(conds)}"
|
|
360
|
+
cur = con.execute(sql, vals)
|
|
361
|
+
con.commit()
|
|
362
|
+
return cur.rowcount
|
|
363
|
+
|
|
364
|
+
|
|
365
|
+
@contextlib.contextmanager
|
|
366
|
+
def transaction(con):
|
|
367
|
+
"""Multi-row atomicity: BEGIN IMMEDIATE ... COMMIT/ROLLBACK.
|
|
368
|
+
|
|
369
|
+
IMMEDIATE takes the write lock up front so concurrent worktree writers queue on
|
|
370
|
+
busy_timeout instead of failing mid-transaction. Do NOT call insert()/update()/
|
|
371
|
+
update_where() inside this block — their per-call con.commit() would end the
|
|
372
|
+
transaction early; use con.execute directly. An already-open transaction at entry
|
|
373
|
+
RAISES (this helper never commits or discards a caller's pending work). The
|
|
374
|
+
except-path uses con.rollback(), a no-op when SQLite already auto-rolled-back
|
|
375
|
+
(disk full / IO error), so the ORIGINAL exception always propagates.
|
|
376
|
+
"""
|
|
377
|
+
if con.in_transaction:
|
|
378
|
+
raise sqlite3.ProgrammingError(
|
|
379
|
+
"transaction(): connection already has an open transaction - "
|
|
380
|
+
"commit or roll it back before entering this block"
|
|
381
|
+
)
|
|
382
|
+
con.execute("BEGIN IMMEDIATE")
|
|
383
|
+
try:
|
|
384
|
+
yield con
|
|
385
|
+
con.commit()
|
|
386
|
+
except Exception:
|
|
387
|
+
con.rollback()
|
|
388
|
+
raise
|
|
389
|
+
|
|
390
|
+
|
|
222
391
|
def rows(con, sql, params=()):
|
|
223
392
|
return [dict(r) for r in con.execute(sql, params).fetchall()]
|
|
224
393
|
|