@surlymochan/evalid 0.0.0-stage → 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (192) hide show
  1. package/CHANGELOG.md +74 -0
  2. package/LICENSE +21 -0
  3. package/README.md +315 -2
  4. package/RELEASE_NOTES.md +102 -0
  5. package/atoms/registry.json +758 -0
  6. package/bin/xflow.js +7208 -0
  7. package/docs/adoption/README.md +128 -0
  8. package/docs/adoption/openflow-release-hardening.md +61 -0
  9. package/docs/benchmark-openflow-vs-gstack-superpowers-openspec-2026-05-25.md +81 -0
  10. package/docs/compatibility.md +52 -0
  11. package/docs/competitive-benchmark.md +108 -0
  12. package/docs/corps-operator-guide.md +75 -0
  13. package/docs/demo-proof.md +126 -0
  14. package/docs/examples-gallery.md +115 -0
  15. package/docs/fixtures/tracker-item.json +7 -0
  16. package/docs/goal-vs-codex.md +129 -0
  17. package/docs/gstack-comparison.md +88 -0
  18. package/docs/install-upgrade.md +133 -0
  19. package/docs/integrations.md +49 -0
  20. package/docs/launch-demo.md +123 -0
  21. package/docs/launch-dossier.md +180 -0
  22. package/docs/methodology.md +149 -0
  23. package/docs/npm-publish-handoff.md +83 -0
  24. package/docs/openspec-migration.md +106 -0
  25. package/docs/public-benchmark.md +68 -0
  26. package/docs/public-release.md +145 -0
  27. package/docs/quality-assessment.md +139 -0
  28. package/docs/quickstart.md +156 -0
  29. package/docs/reviewer-guide.md +43 -0
  30. package/docs/spec-kit-benchmark.md +73 -0
  31. package/docs/super-assistant-comparison.md +106 -0
  32. package/docs/superpowers-comparison.md +82 -0
  33. package/docs/team-adoption.md +123 -0
  34. package/docs/tooling-matrix.md +91 -0
  35. package/docs/walkthrough.md +76 -0
  36. package/docs/workflow-manual.md +1125 -0
  37. package/package.json +83 -4
  38. package/prompts/pencil-draft.md +46 -0
  39. package/prompts/pencil-refine.md +46 -0
  40. package/prompts/phase-artifact.md +24 -0
  41. package/prompts/phase-execution.md +32 -0
  42. package/schemas/competitor-reconstruction-contract.schema.json +91 -0
  43. package/schemas/llm-design-review-aggregate.schema.json +83 -0
  44. package/schemas/visual-benchmark.schema.json +978 -0
  45. package/schemas/workflow.schema.json +100 -0
  46. package/scripts/run-tests.mjs +23 -0
  47. package/scripts/takein_digest.py +139 -0
  48. package/specs/README.md +118 -0
  49. package/specs/architecture.md +36 -0
  50. package/specs/product.md +33 -0
  51. package/specs/workflow.md +349 -0
  52. package/src/agent_team/atoms/codex-app-bridge.js +8 -0
  53. package/src/agent_team/atoms/design/competitor-reconstruction-review.js +44 -0
  54. package/src/agent_team/atoms/design/contract-freeze.js +8 -0
  55. package/src/agent_team/atoms/design/generation-contract.js +442 -0
  56. package/src/agent_team/atoms/design/reconstruction-pack.js +246 -0
  57. package/src/agent_team/atoms/design/reference-surface-lock.js +49 -0
  58. package/src/agent_team/atoms/patch-challenge.js +19 -0
  59. package/src/agent_team/atoms/pencil/accept.js +9 -0
  60. package/src/agent_team/atoms/pencil/draft.js +365 -0
  61. package/src/agent_team/atoms/pencil/refine.js +229 -0
  62. package/src/agent_team/atoms/phase-run-dispatch.js +8 -0
  63. package/src/agent_team/atoms/team-run.js +380 -0
  64. package/src/agent_team/atoms/visual/aesthetic-review.js +231 -0
  65. package/src/agent_team/atoms/visual/benchmark-repair-plan.js +291 -0
  66. package/src/agent_team/atoms/visual/benchmark.js +1652 -0
  67. package/src/agent_team/atoms/visual/design-selection.js +23 -0
  68. package/src/agent_team/atoms/visual/direction-synthesis.js +124 -0
  69. package/src/agent_team/atoms/visual/dom-rect-export.js +129 -0
  70. package/src/agent_team/atoms/visual/image-diff.js +186 -0
  71. package/src/agent_team/atoms/visual/layout-competition.js +8 -0
  72. package/src/agent_team/atoms/visual/page-evidence-capture.js +806 -0
  73. package/src/agent_team/atoms/visual/playwright-runtime.js +8 -0
  74. package/src/agent_team/atoms/visual/review-aggregate.js +87 -0
  75. package/src/agent_team/atoms/visual/review.js +77 -0
  76. package/src/agent_team/atoms/visual/visual-report.js +197 -0
  77. package/src/agent_team/atoms/visual/visual-token-export.js +417 -0
  78. package/src/agent_team/common.js +311 -0
  79. package/src/core/atoms/agent-invoke.js +684 -0
  80. package/src/core/atoms/event/ingest.js +30 -0
  81. package/src/core/atoms/experience/digest.js +55 -0
  82. package/src/core/atoms/gate/artifact-verify.js +83 -0
  83. package/src/core/atoms/gate/competitor-contract-validate.js +119 -0
  84. package/src/core/atoms/gate/evidence-collect.js +64 -0
  85. package/src/core/atoms/gate/payload-from-phase-run.js +78 -0
  86. package/src/core/atoms/gate/visual-benchmark-validate.js +183 -0
  87. package/src/core/atoms/gate/workbench.js +79 -0
  88. package/src/core/atoms/intake/classify.js +15 -0
  89. package/src/core/atoms/intake/design-skill-select.js +16 -0
  90. package/src/core/atoms/intake/preview-or-refine.js +19 -0
  91. package/src/core/atoms/intake/recommend.js +20 -0
  92. package/src/core/atoms/intake/research-same-category.js +19 -0
  93. package/src/core/atoms/intervention/create.js +30 -0
  94. package/src/core/atoms/memory/manage.js +27 -0
  95. package/src/core/atoms/memory/mount.js +13 -0
  96. package/src/core/atoms/mission/create.js +19 -0
  97. package/src/core/atoms/mission/list.js +14 -0
  98. package/src/core/atoms/mission/show.js +26 -0
  99. package/src/core/atoms/notify/checkpoint.js +46 -0
  100. package/src/core/atoms/phase-run/complete.js +42 -0
  101. package/src/core/atoms/phase-run/fail.js +40 -0
  102. package/src/core/atoms/phase-run/persist.js +39 -0
  103. package/src/core/atoms/phase-run/start.js +30 -0
  104. package/src/core/atoms/plugin/run.js +76 -0
  105. package/src/core/atoms/policy/gate.js +90 -0
  106. package/src/core/atoms/policy/resolve.js +11 -0
  107. package/src/core/atoms/storage/sync.js +57 -0
  108. package/src/core/atoms/workflow/advance.js +25 -0
  109. package/src/core/atoms/workflow/guard.js +127 -0
  110. package/src/core/atoms/workflow/state-load.js +37 -0
  111. package/src/core/bindings.js +37 -0
  112. package/src/core/change-artifacts.js +134 -0
  113. package/src/core/competitive-score.js +265 -0
  114. package/src/core/concurrency-policy.js +35 -0
  115. package/src/core/execution-log.js +108 -0
  116. package/src/core/intake-recommendation.js +134 -0
  117. package/src/core/memory-state.js +62 -0
  118. package/src/core/mission-state.js +780 -0
  119. package/src/core/pending-gates.js +66 -0
  120. package/src/core/phase-cost.js +116 -0
  121. package/src/core/phase-run-read.js +27 -0
  122. package/src/core/policy-overlay.js +180 -0
  123. package/src/core/state-store.js +201 -0
  124. package/src/core/workflow-executor.js +998 -0
  125. package/src/core/workflow-integrity.js +111 -0
  126. package/src/core/workflow-loader.js +163 -0
  127. package/src/core/workflow-rerun.js +84 -0
  128. package/src/core/workflow-state.js +119 -0
  129. package/src/server.js +1307 -0
  130. package/workflows/corps.yaml +479 -0
  131. package/workflows/yolo.yaml +233 -0
  132. package/xflow/README.md +76 -0
  133. package/xflow/aha/SKILL.md +82 -0
  134. package/xflow/aha/scripts/append_aha.py +50 -0
  135. package/xflow/archive/scripts/archive_change.py +338 -0
  136. package/xflow/archive/templates/merge-design.md +54 -0
  137. package/xflow/archive/templates/merge-workflow.md +26 -0
  138. package/xflow/atoms/__init__.py +6 -0
  139. package/xflow/atoms/a1_issue_create_or_patch.py +70 -0
  140. package/xflow/atoms/a2_branch_create_and_link.py +42 -0
  141. package/xflow/atoms/a3_repo_doctor.py +11 -0
  142. package/xflow/atoms/a4_project_set_status.py +11 -0
  143. package/xflow/atoms/a5_archive_commit_push_close.py +11 -0
  144. package/xflow/atoms/a6_pr_create.py +72 -0
  145. package/xflow/atoms/b1_change_scaffold.py +11 -0
  146. package/xflow/atoms/b2_status_read.py +16 -0
  147. package/xflow/atoms/b2b_status_write.py +20 -0
  148. package/xflow/atoms/b3_status_transition.py +27 -0
  149. package/xflow/atoms/b4_handoff_scaffold_or_refresh.py +11 -0
  150. package/xflow/atoms/b5_aha_append.py +11 -0
  151. package/xflow/atoms/b6_takein_digest.py +11 -0
  152. package/xflow/atoms/b6b_lookback_digest.py +11 -0
  153. package/xflow/atoms/b7_change_mission_bind.py +43 -0
  154. package/xflow/atoms/c1_mem.py +54 -0
  155. package/xflow/atoms/e6_gate_local_precheck.py +23 -0
  156. package/xflow/atoms/h1_design_lite_gate.py +74 -0
  157. package/xflow/atoms/i6a_tdd_run.py +104 -0
  158. package/xflow/atoms/i6b_tdd_proof_validate.py +165 -0
  159. package/xflow/atoms/i6c_tdd_quality_review.py +357 -0
  160. package/xflow/atoms/j1_tests_run.py +129 -0
  161. package/xflow/atoms/j4a_spec_consistency_structural.py +105 -0
  162. package/xflow/atoms/j4b_spec_delta_review.py +316 -0
  163. package/xflow/atoms/j4c_openspec_migration_map.py +121 -0
  164. package/xflow/atoms/j4d_spec_clarify_gate.py +373 -0
  165. package/xflow/atoms/j6_cr_review.py +443 -0
  166. package/xflow/atoms/k1_artifacts_complete_check.py +39 -0
  167. package/xflow/atoms/k2_merge_snippets_apply.py +11 -0
  168. package/xflow/atoms/k3_handoff_refresh.py +14 -0
  169. package/xflow/atoms/k4_mem_lesson_persist.py +65 -0
  170. package/xflow/atoms/k5_archive_publish.py +21 -0
  171. package/xflow/atoms/k6_aha_merge.py +42 -0
  172. package/xflow/atoms/tdd_lib.py +747 -0
  173. package/xflow/corps/SKILL.md +109 -0
  174. package/xflow/exec/scripts/set_issue_in_progress.py +159 -0
  175. package/xflow/goal/SKILL.md +150 -0
  176. package/xflow/handoff/SKILL.md +95 -0
  177. package/xflow/handoff/scripts/scaffold_handoff.py +304 -0
  178. package/xflow/lookback/SKILL.md +66 -0
  179. package/xflow/lookback/scripts/lookback_digest.py +46 -0
  180. package/xflow/openissue/scripts/open_issue_flow.py +438 -0
  181. package/xflow/plan/SKILL.md +38 -0
  182. package/xflow/ralph/SKILL.md +123 -0
  183. package/xflow/scripts/check_installed_xflow_skill_sync.sh +44 -0
  184. package/xflow/scripts/common.py +480 -0
  185. package/xflow/scripts/doctor.py +278 -0
  186. package/xflow/scripts/sync_installed_xflow_skill.sh +134 -0
  187. package/xflow/takein/SKILL.md +81 -0
  188. package/xflow/takein/scripts/takein_digest.py +88 -0
  189. package/xflow/yolo/SKILL.md +66 -0
  190. package/xflow/yolo/scripts/scaffold_specs_change.py +673 -0
  191. package/xflow/yolo/scripts/yolo_gate.py +312 -0
  192. package/xflow/yolo/templates/DESIGN.md +179 -0
@@ -0,0 +1,128 @@
1
+ # Adoption Evidence
2
+
3
+ Use this folder for real external adoption proof. Do not use it for fixture-only
4
+ demo runs, synthetic screenshots, or private anecdotes that cannot be reviewed.
5
+
6
+ `xflow launch audit --json` intentionally treats public launch as incomplete
7
+ until at least one adoption artifact exists here or a linked public PR provides
8
+ equivalent evidence.
9
+
10
+ Validate records before using them in launch claims:
11
+
12
+ ```bash
13
+ xflow adoption brief --name <team-or-project> --source <tracker-or-pr> --track yolo
14
+ xflow adoption trial --name <team-or-project> --source <tracker-or-pr> --track yolo
15
+ xflow adoption kit --name <team-or-project> --source <tracker-or-pr> --track yolo
16
+ xflow adoption init --name <team-or-project> --source <tracker-or-pr> --track yolo
17
+ xflow adoption status --json
18
+ xflow adoption validate --json
19
+ xflow adoption validate --splash --json
20
+ ```
21
+
22
+ `xflow adoption status` is the non-failing release-owner view: it reports
23
+ ordinary adoption status, splash third-party status, blocker ids, and the next
24
+ external trial command without treating missing evidence as a CLI error.
25
+
26
+ `xflow adoption brief` prints a sendable third-party ask: what to ask for, what
27
+ acceptance bar to hold, and which maintainer follow-up commands close the loop.
28
+ Use it when you need to message an external team instead of just reading raw
29
+ commands yourself.
30
+
31
+ `xflow adoption trial` prints a copy-paste external trial sequence without
32
+ writing files. Use it when a third-party team asks what to run before you create
33
+ or send a packet.
34
+
35
+ `xflow adoption kit` creates `docs/adoption/trial-packets/<team-or-project>.md`.
36
+ Share that packet with the external project before the trial. It lists the
37
+ commands to run, the sendable ask text, the artifacts to collect, the
38
+ acceptance bar, and the validation commands that must pass after evidence is
39
+ converted into an adoption record.
40
+
41
+ `xflow adoption init` only creates a draft. The draft intentionally contains
42
+ placeholders and must fail validation until a real trial fills in evidence,
43
+ outcome, and redaction details.
44
+
45
+ ## Required Shape
46
+
47
+ Each adoption record should be a short markdown file named after the project,
48
+ team, or tracker source:
49
+
50
+ ```text
51
+ docs/adoption/<team-or-project>.md
52
+ ```
53
+
54
+ The record must include:
55
+
56
+ - **Context**: what project, team, or external workflow tried xflow.
57
+ - **Goal**: the `.xflow/GOAL.md` direction or equivalent project objective.
58
+ - **Track**: yolo, corps, or both.
59
+ - **Evidence**: commands run, artifacts produced, and links or paths to proof.
60
+ - **Outcome**: what improved, what failed, and whether xflow would be used again.
61
+ - **Redactions**: what was sanitized and why.
62
+
63
+ ## Acceptance Bar
64
+
65
+ An adoption record is strong enough for launch only when it proves all of these:
66
+
67
+ 1. The work came from outside the fixture path in `docs/fixtures/`.
68
+ 2. A real goal or tracker item shaped the delivery.
69
+ 3. At least one xflow command produced reviewable artifacts.
70
+ 4. The record names a concrete benefit or a concrete failure.
71
+ 5. A reviewer can follow the evidence without private machine state.
72
+
73
+ For industry-splash claims, the bar is stricter: at least one passing record
74
+ must come from a third-party project, public PR, external repository, or named
75
+ external team beyond maintainer dogfooding. Use `xflow launch audit --splash
76
+ --strict --json` before saying broad third-party adoption is proven.
77
+
78
+ ## Template
79
+
80
+ ```markdown
81
+ # Adoption: <team-or-project>
82
+
83
+ Date: YYYY-MM-DD
84
+ Source: <tracker, repository, team workflow, or public PR>
85
+ Track: <yolo|corps|both>
86
+
87
+ ## Context
88
+
89
+ <What external project or team workflow tried xflow?>
90
+
91
+ Use `third-party`, `public PR`, `external project`, `external repository`,
92
+ `customer`, `partner`, `team workflow`, or an `https://` source when the record
93
+ is intended to satisfy splash-launch third-party evidence.
94
+
95
+ ## Goal
96
+
97
+ <What durable project direction or tracker objective guided the work?>
98
+
99
+ ## Commands
100
+
101
+ ```bash
102
+ xflow goal show --json
103
+ xflow assess --json
104
+ xflow workflow validate yolo --project-root .
105
+ # or: xflow corps ... && xflow proof --track corps --change-id <id>
106
+ ```
107
+
108
+ ## Evidence
109
+
110
+ - `<path-or-link-to-status.json>`
111
+ - `<path-or-link-to-red-green-quality-proof>`
112
+ - `<path-or-link-to-corps_proof.json>`
113
+ - `<path-or-link-to-PR-or-review>`
114
+
115
+ At least one evidence bullet must be reviewable by someone outside the authoring
116
+ machine. Prefer public PRs, external repository links, sanitized tracker exports,
117
+ or checked-in proof artifacts.
118
+
119
+ ## Outcome
120
+
121
+ <What got better, what failed, and whether the team would use xflow again?>
122
+
123
+ ## Redactions
124
+
125
+ <What was sanitized?>
126
+
127
+ State `None` only when the record and evidence are already public.
128
+ ```
@@ -0,0 +1,61 @@
1
+ # Adoption: OpenFlow release hardening
2
+
3
+ Date: 2026-05-25
4
+ Source: OpenFlow maintainer release-hardening workflow, public commit 9344633
5
+ Track: corps
6
+
7
+ ## Context
8
+
9
+ The OpenFlow maintainer workflow used xflow itself to extract and harden the
10
+ public launch surface for the heavy `corps` track. The work was not a fixture
11
+ run: it produced a public source repository, documentation, CLI evidence,
12
+ workflow checks, benchmark reporting, and CI verification.
13
+
14
+ ## Goal
15
+
16
+ Evaluate whether `xflow:goal`, `xflow:yolo`, and `xflow:corps` can provide a
17
+ credible open-source project-delivery layer, then keep the public boundary
18
+ honest about what is proven locally versus what still needs external adoption
19
+ or npm registry proof.
20
+
21
+ ## Commands
22
+
23
+ ```bash
24
+ xflow goal show --json
25
+ xflow goal audit --json
26
+ xflow corps --explain --json
27
+ xflow assess --json
28
+ xflow score --json
29
+ node --test test/cli-usability.test.js test/competitive-readiness.test.js test/workflow-drift-scan.test.js
30
+ SKILL_SYNC_TARGET_DIRS="$HOME/.agents/skills
31
+ $HOME/.codex/skills
32
+ $HOME/.config/opencode/skills" npm run skill:sync
33
+ npm run release:pack
34
+ xflow launch audit --strict --json
35
+ ```
36
+
37
+ ## Evidence
38
+
39
+ - `https://github.com/surlymochan/openflow/commit/9344633`
40
+ - `docs/corps-operator-guide.md`
41
+ - `bin/xflow.js`
42
+ - `test/cli-usability.test.js`
43
+ - `test/competitive-readiness.test.js`
44
+ - `docs/quality-assessment.md`
45
+ - `docs/public-benchmark.md`
46
+
47
+ ## Outcome
48
+
49
+ The release review got better because the public extraction has a first-time
50
+ operator guide, a CLI `--explain` path, score evidence, CI, and tests that fail
51
+ if core launch documentation disappears. The workflow also kept claim safety
52
+ explicit: source-checkout quality can be proven now, while npm publication and
53
+ third-party adoption claims require separate evidence. A future release owner
54
+ would use this gate again because it separates local quality from distribution
55
+ and ecosystem evidence.
56
+
57
+ ## Redactions
58
+
59
+ Local home-directory paths and machine-specific command output were summarized.
60
+ No secrets, tokens, private customer data, or private tracker content were
61
+ included.
@@ -0,0 +1,81 @@
1
+ # OpenFlow vs gstack vs Superpowers vs OpenSpec Benchmark
2
+
3
+ Date: 2026-05-25
4
+
5
+ This benchmark is a source-checkout, command-driven comparison. It is not a
6
+ neutral industry certification. Scores use OpenFlow's repo-owned delivery
7
+ runtime rubric: executable workflow state, verification gates, handoff,
8
+ auditability, packaging, and claim safety.
9
+
10
+ ## Tested Revisions
11
+
12
+ | Project | Source | Revision / Version |
13
+ | --- | --- | --- |
14
+ | OpenFlow | local public extraction | `openflow@0.1.0` |
15
+ | gstack | https://github.com/garrytan/gstack | `920a13a17f463c28a3db75cc27482affb13a4fee`, package `1.44.0.0` |
16
+ | Superpowers | https://github.com/obra/superpowers | `f2cbfbefebbfef77321e4c9abc9e949826bea9d7`, package `5.1.0` |
17
+ | OpenSpec | https://github.com/Fission-AI/OpenSpec | `e441287b1fdd719bfa4518936c79e03e91c8d3c9`, package `@fission-ai/openspec@1.3.1` |
18
+
19
+ ## Commands Actually Run
20
+
21
+ | Project | Command | Result | Notes |
22
+ | --- | --- | --- | --- |
23
+ | OpenFlow | `npm ci` | Pass | Installed local package dependencies. |
24
+ | OpenFlow | `node bin/xflow.js doctor --project-root . --json` | Pass | 12/12 readiness checks passed. |
25
+ | OpenFlow | `node bin/xflow.js workflow validate yolo --project-root . --json` | Pass | Public yolo workflow validates. |
26
+ | OpenFlow | `node bin/xflow.js score --json` | Pass | `100/100`. |
27
+ | OpenFlow | `node bin/xflow.js compare openspec --json` | Pass | OpenFlow `93`, OpenSpec `73`. |
28
+ | OpenFlow | `node bin/xflow.js compare superpowers --json` | Pass | OpenFlow `91`, Superpowers `78`. |
29
+ | OpenFlow | `node bin/xflow.js compare gstack --json` | Pass | OpenFlow `96`, gstack `80`. |
30
+ | OpenFlow | `npm run drift:scan` | Pass | 9/9 drift checks passed. |
31
+ | OpenFlow | `node --test test/cli-usability.test.js` | Pass | 67/67 tests passed after public-boundary updates. |
32
+ | OpenFlow | `node --test test/competitive-readiness.test.js` | Pass | 16/16 tests passed after public-boundary updates. |
33
+ | OpenFlow | `gitleaks detect --no-git --redact --source .` | Pass | No leaks found in the extracted tree. |
34
+ | gstack | `./setup --host openclaw` | Pass | Safe non-mutating host path prints integration instructions. |
35
+ | gstack | `bun run skill:check` | Fail | Skill command validation mostly passed, but generated host skill freshness failed and `claude/SKILL.md` was missing. |
36
+ | gstack | `bun install --frozen-lockfile` | Inconclusive | No output for an extended wait; killed to avoid blocking the benchmark. |
37
+ | Superpowers | `npm pack --dry-run --json` | Pass | Package dry run succeeded. |
38
+ | Superpowers | `CLAUDE_PLUGIN_ROOT=<repo> bash hooks/session-start` | Pass | Hook produced valid JSON context injection. |
39
+ | OpenSpec | `npm ci` | Pass | Installed 304 packages and built successfully. |
40
+ | OpenSpec | `node bin/openspec.js --help` | Pass | CLI exposed command surface. |
41
+ | OpenSpec | `node bin/openspec.js validate --all --json` | Fail | 49/54 items passed; 5 change proposals lacked spec deltas. |
42
+ | OpenSpec | `npm test` | Pass | 76 test files and 1512 tests passed. |
43
+
44
+ ## Scorecard
45
+
46
+ | Dimension | Weight | OpenFlow | gstack | Superpowers | OpenSpec |
47
+ | --- | ---: | ---: | ---: | ---: | ---: |
48
+ | First-run / install check | 10 | 9 | 6 | 8 | 9 |
49
+ | Executable workflow state | 15 | 15 | 9 | 8 | 10 |
50
+ | TDD / verification gates | 15 | 14 | 10 | 14 | 9 |
51
+ | Spec durability | 10 | 10 | 7 | 7 | 10 |
52
+ | Repo-local auditability | 15 | 15 | 10 | 9 | 9 |
53
+ | Cross-host portability | 10 | 9 | 9 | 10 | 10 |
54
+ | Release / claim gates | 15 | 13 | 8 | 7 | 8 |
55
+ | Public-boundary hygiene | 10 | 9 | 7 | 8 | 8 |
56
+ | **Total** | **100** | **94** | **66** | **71** | **73** |
57
+
58
+ ## Interpretation
59
+
60
+ OpenFlow leads this benchmark because it combines a runnable workflow engine,
61
+ machine-checked gates, release checks, handoff continuity, and explicit claim
62
+ boundaries from source checkout.
63
+
64
+ gstack remains strong for role-rich operator workflows and browser/QA rituals,
65
+ but the tested checkout exposed generated-skill freshness failures.
66
+
67
+ Superpowers remains the strongest low-ceremony behavior-discipline layer across
68
+ agent hosts, especially for planning, TDD, debugging, and review. It is less of
69
+ a repo-owned execution runtime.
70
+
71
+ OpenSpec remains strongest for lightweight spec-driven development. Its own test
72
+ suite passed, but repository-wide `validate --all` surfaced five incomplete
73
+ change deltas in the tested checkout.
74
+
75
+ ## Public Claim Boundary
76
+
77
+ Safe claim: OpenFlow is ahead in this benchmark for repo-owned delivery closure,
78
+ auditability, release gates, and cross-session workflow evidence.
79
+
80
+ Unsafe claim: OpenFlow universally makes gstack, Superpowers, or OpenSpec
81
+ obsolete. Each tool remains strong in its narrower operating model.
@@ -0,0 +1,52 @@
1
+ # evalid Compatibility Notes
2
+
3
+ `evalid` is designed to work as a workflow runtime below multiple agent tools. The agent can change, but the project artifacts, workflow YAML, gates, and archive rules stay stable.
4
+
5
+ For installation snippets by tool, see `docs/tooling-matrix.md`.
6
+
7
+ ## Codex
8
+
9
+ - Use project-local instructions to call `xflow:takein`, `xflow:plan`, and `xflow:yolo`.
10
+ - Run `xflow doctor` before substantial work in a new checkout.
11
+ - Use `npm run drift:scan` before editing active workflow or skill surfaces.
12
+
13
+ ## Claude Code
14
+
15
+ - Treat `docs/quickstart.md` as the first-run path.
16
+ - Keep the active plan in `specs/changes/<change-id>/plan.md`.
17
+ - Run `xflow workflow validate yolo` or `xflow workflow validate corps` before executing a workflow.
18
+
19
+ ## Cursor
20
+
21
+ - Put xflow commands in project rules or agent instructions rather than duplicating workflow prose.
22
+ - Keep `.as-xflow/config.json` in the project root as the local contract.
23
+ - Use `xflow doctor --json` for machine-readable readiness checks.
24
+ - Let `.as-xflow/config.json` carry workflow aliases, issue routing defaults, and skill sync defaults across tools.
25
+
26
+ ## OpenCode
27
+
28
+ - Prefer CLI commands over editor-specific state: `xflow init`, `xflow doctor`, `xflow workflow validate yolo`.
29
+ - Keep verification in package scripts so OpenCode tasks can run the same checks as Codex or Claude Code.
30
+
31
+ ## Generic CLI Agents
32
+
33
+ - Required commands are intentionally plain shell commands.
34
+ - Minimum adoption path:
35
+ - `xflow init --project-root .`
36
+ - `xflow doctor --project-root .`
37
+ - `xflow workflow validate yolo`
38
+ - `npm run drift:scan`
39
+
40
+ ## Gemini CLI
41
+
42
+ - Use the universal CLI contract from `docs/tooling-matrix.md`.
43
+ - Treat xflow as the workflow source of truth and Gemini as the executor.
44
+ - Validate workflow data with `xflow workflow validate yolo --project-root .` before running implementation.
45
+
46
+ ## Invariants Across Tools
47
+
48
+ - `xflow:plan` remains track-neutral.
49
+ - `xflow:yolo` and `xflow:corps` reuse a current plan instead of re-planning.
50
+ - `xmem` remains outside the xflow namespace.
51
+ - Archive publish stays owned by `A5.archive.commit_push_close` before `A6.pr.create`.
52
+ - Installed skill sync goes through the evalid wrapper and verifies with `npm run skill:diff`.
@@ -0,0 +1,108 @@
1
+ # evalid Competitive Benchmark
2
+
3
+ Date: 2026-04-19
4
+
5
+ ## Goal
6
+
7
+ Make `evalid` stronger than adjacent agent workflow systems in every dimension that matters for serious delivery work: planning, execution, verification, portability, team handoff, and operator confidence.
8
+
9
+ This benchmark is intentionally stricter than a marketing comparison. A category is only considered won when `evalid` has a named project surface, a command or artifact operators can use, and local tests or drift scans that protect it.
10
+
11
+ ## Reference Systems
12
+
13
+ - Superpowers: strong skill distribution, agent behavior shaping, and test-driven coding guidance.
14
+ - Super-Assistant: broad enterprise harness with team constraints, checkpoint notifications, plugin execution, experience indexing, and S3-backed sync.
15
+ - OpenSpec: strong spec-driven workflow, low-friction command surface, and cross-tool positioning.
16
+ - gstack: strong role-specialist command surface, browser QA, and host-native release rituals for Codex / Claude Code.
17
+ - spec-kit: strong specification-driven development lifecycle, broad public extension surface, and ecosystem credibility.
18
+
19
+ ## Score Dimensions
20
+
21
+ | Dimension | Win Condition | Current evalid Surface | Current Status |
22
+ | --- | --- | --- | --- |
23
+ | Local delivery closure | One command family can move from plan through archive with verification evidence. | `xflow:plan`, `xflow:yolo`, `xflow:corps`, `workflows/*.yaml`, archive atoms | Strong |
24
+ | Goal alignment | Project direction survives across threads, tools, handoffs, and workflow tracks. | `xflow goal set`, `xflow goal audit`, `xflow compare codex-goal`, `xflow:goal`, `.xflow/GOAL.md`, `docs/goal-vs-codex.md`, yolo/corps/Ralph goal-alignment rules | Strong |
25
+ | Executable workflow data | Workflow order is data, validated by schema/tests, not only prose. | `workflows/yolo.yaml`, `workflows/corps.yaml`, `schemas/workflow.schema.json` | Strong |
26
+ | Verification rigor | Drift checks, tests, gates, and sync checks catch regressions before release. | `npm test`, `npm run drift:scan`, `npm run skill:diff`, deterministic gates | Strong |
27
+ | TDD / code quality push | Red runs before implementation, green runs after implementation, and changed tests are checked for quality anti-patterns. | `I6a.tdd.run`, `I6b.tdd.proof_validate`, `I6c.tdd.quality_review`, `test/tdd-proof.test.js`, `verify_proof.json` | Strong |
28
+ | Operator onboarding | First commands explain the human loop, initialize a project, and report whether the local install is ready. | `xflow quickstart`, `xflow guide`, `xflow init`, `xflow doctor`, `npm run doctor` | Strong |
29
+ | One-shot evaluation | External evaluators can see quality score, Codex goal / Superpowers / Super-Assistant positioning, plus OpenSpec / gstack comparisons, launch blockers, release-owner status, claim boundaries, next actions, and first-run path from one command family. | `xflow evaluate`, `xflow evaluate --json`, `xflow release status`, `xflow release status --json`, `xflow compare codex-goal --json`, `xflow compare superpowers --json`, `xflow compare super-assistant --json`, `xflow compare openspec --json`, `xflow compare gstack --json`, `xflow assess --json`, `xflow launch audit --json`, `xflow launch claims --json`, `xflow launch copy --json` | Strong |
30
+ | Spec durability | Plans, proposal artifacts, workflow deltas, AHA, and handoff survive thread changes. | `specs/changes/*`, `specs/workflow.md`, `AHA.md`, `HANDOFF.md` | Strong |
31
+ | Cross-tool portability | Skills and workflows can be installed without personal machine assumptions. | `docs/compatibility.md`, `.as-xflow/config.json`, skillhub sync wrapper, installed skill diff | Strong |
32
+ | Public product clarity | New users can understand the system without reading implementation internals. | README badges, `xflow quickstart`, `xflow guide`, `xflow evaluate`, `xflow assess`, `xflow demo launch`, `xflow launch dossier`, `xflow launch claims`, `xflow launch copy`, `xflow launch audit`, `xflow adoption status`, `xflow adoption trial`, `xflow package status`, `xflow package preflight`, `xflow package audit`, `xflow score`, `docs/quality-assessment.md`, `docs/launch-demo.md`, `docs/examples-gallery.md`, `docs/adoption/README.md`, `docs/methodology.md`, `docs/public-benchmark.md`, `RELEASE_NOTES.md`, `docs/fixtures/tracker-item.json`, this benchmark | Strong |
33
+ | Team collaboration | Issues, branches, PR/archive state, handoff, and reviewer checks are explicit and auditable. | `docs/reviewer-guide.md`, openissue/archive atoms, current-state handoff block | Strong |
34
+ | Control-plane observability | Operators can inspect run state, gates, orphaned runs, and aggregate signals. | `xflow serve`, server tests, timeline analytics | Strong |
35
+ | Ecosystem packaging | External users can install, initialize, upgrade, inspect package readiness, and import external tracker context with minimal manual wiring. | `docs/install-upgrade.md`, `docs/public-release.md`, `docs/demo-proof.md`, `docs/fixtures/tracker-item.json`, `.github/workflows/ci.yml`, `package.json` bin/files, `npm run publish:check`, `xflow adapter import-file`, `xflow adapter github-issue` | Strong |
36
+
37
+ ## Current Judgment
38
+
39
+ `evalid` now beats reference systems on executable workflow depth, local delivery closure, archive discipline, observable heavy-track control, first-run commands, split red/green TDD proof semantics, TDD test-quality review, deterministic requirements-level spec delta review, OpenSpec migration mapping, public release checks, real adapter seams, examples gallery, public benchmark proof, methodology guidance, and reviewer-facing collaboration guidance.
40
+
41
+ Against Superpowers, xflow should not claim behavior-guidance superiority.
42
+ Superpowers remains stronger as lightweight behavior discipline; xflow's
43
+ stronger claim is repo-local workflow evidence, cross-tool state, and launch
44
+ gates.
45
+
46
+ Against Super-Assistant, xflow should not claim broader concrete enterprise
47
+ connector inventory. Super-Assistant remains a useful reference for DingTalk,
48
+ plugin, checkpoint, experience, and S3 patterns. xflow's stronger claim is
49
+ absorbing those patterns as a policy overlay and thin atoms, keeping workflow
50
+ authority in local YAML, gates, and proof artifacts.
51
+
52
+ Against gstack, xflow should not claim better role choreography or browser
53
+ rituals. The stronger xflow claim is repo-owned workflow truth: checked-in YAML,
54
+ goal alignment, handoff/AHA continuity, and launch gates that can be audited
55
+ from source checkout without assuming a specific host command pack.
56
+
57
+ Against Codex native goal, xflow wins only at the project layer: Codex goal is
58
+ excellent thread memory, while `.xflow/GOAL.md` becomes stronger when yolo,
59
+ corps, Ralph, handoff, and takein all consume it and report alignment. The
60
+ score dimension above is intentionally product-facing: a public user should be
61
+ able to see why `xflow:goal` is not a duplicate prompt reminder.
62
+ CLI users should be able to prove the same story without knowing the skill
63
+ system first: `xflow goal set` writes the durable anchor, and `xflow goal show
64
+ --json` exposes it to any agent runtime. `xflow goal audit --json` then proves
65
+ the core skill family still consumes the anchor instead of leaving it as a
66
+ decorative file.
67
+
68
+ The remaining product moat work is no longer the core engine; it is distribution and ecosystem proof. The current repo now includes a root README with public badges, install/upgrade page, tooling matrix, demo proof page, walkthrough, integration matrix, public release gate, release notes, public benchmark, examples gallery, spec-kit benchmark, and CI-backed minimal adoption example so the external story is testable instead of aspirational.
69
+
70
+ The highest-leverage path is not more workflow phases. The next win is authenticated public registry publish plus third-party adoption data from real teams, recorded through `docs/adoption/README.md`.
71
+
72
+ ## Proof Commands
73
+
74
+ ```bash
75
+ xflow score
76
+ xflow quickstart
77
+ xflow evaluate --json
78
+ xflow assess --json
79
+ xflow demo launch --json
80
+ xflow launch dossier
81
+ xflow launch claims --json
82
+ xflow launch copy --json
83
+ xflow launch audit --strict --json
84
+ xflow adoption kit --name <team-or-project> --source <tracker-or-pr> --track yolo
85
+ xflow adoption init --name <team-or-project> --source <tracker-or-pr> --track yolo
86
+ xflow adoption validate --json
87
+ xflow package preflight --check-registry --check-auth --json
88
+ xflow launch audit --pre-publish --strict --json
89
+ xflow package audit --check-registry --json
90
+ xflow goal show --json
91
+ xflow goal audit --json
92
+ xflow corps --explain --json
93
+ xflow compare codex-goal
94
+ xflow compare gstack
95
+ xflow compare superpowers
96
+ xflow compare super-assistant
97
+ xflow compare openspec
98
+ xflow compare spec-kit
99
+ xflow adapter import-file --input docs/fixtures/tracker-item.json --json
100
+ xflow spec delta --change-id <change-id>
101
+ npm run release:pack
102
+ npm run publish:check
103
+ ```
104
+
105
+ ## Required Next Wins
106
+
107
+ 1. Run `npm run publish:check`, confirm package ownership and npm identity with `xflow package preflight --check-registry --check-auth --json`, require `xflow launch audit --pre-publish --strict --json` before the irreversible publish, publish the package to the public registry, and verify it with `xflow package audit --check-registry --json`.
108
+ 2. Extend the current non-fixture maintainer adoption record with at least one third-party tracker workflow using `xflow adoption status`, `xflow adoption trial`, `xflow adoption kit`, `xflow adoption init`, and `docs/adoption/README.md`, then gate it with `xflow adoption validate --json`.
@@ -0,0 +1,75 @@
1
+ # Corps Operator Guide
2
+
3
+ Use this page when `xflow:corps` feels too large to start. The short version:
4
+ corps is the heavy track for product/UI risk, not a replacement for yolo.
5
+
6
+ ## Verdict
7
+
8
+ Use corps when the work needs governed product proof:
9
+
10
+ - new user-facing product surface or interaction pattern
11
+ - competitor-led UI or visual fidelity matters
12
+ - multiple agents should propose, critique, or repair
13
+ - completion must be accepted from artifacts instead of chat summary
14
+
15
+ Do not use corps for ordinary backend, docs, or infrastructure changes. Use
16
+ `xflow:yolo` for that path.
17
+
18
+ ## First-Time Sequence
19
+
20
+ ```bash
21
+ xflow corps --explain
22
+ xflow goal show --json
23
+ xflow goal audit --json
24
+ xflow corps --title "<product or UI change>" --change-type frontend --change-id <change-id> --dry-run --json
25
+ xflow corps --title "<product or UI change>" --change-type frontend --change-id <change-id>
26
+ xflow proof --track corps --change-id <change-id>
27
+ ```
28
+
29
+ The dry run explains the proof contract. The real run is not complete until
30
+ `xflow proof --track corps` writes `corps_proof.json` with `ok=true`.
31
+
32
+ ## Proof Contract
33
+
34
+ Corps completion requires all of this:
35
+
36
+ - `specs/changes/<change-id>/corps_proof.json` reports `ok=true`
37
+ - the built-in governed corps workflow manifest is used
38
+ - hash-linked execution logs contain every canonical phase, atom, and gate
39
+ witness
40
+ - required product, visual, QA, and archive artifacts exist
41
+ - no `stub`, `task_queued`, or `pencil_stubbed` runtime fallback appears
42
+ - the operator confirms the proof aligns with `.xflow/GOAL.md`
43
+
44
+ ## Competitor-Led UI Inputs
45
+
46
+ Competitor-led UI requires a primary reference surface and exactly one benchmark
47
+ evidence path before execution:
48
+
49
+ ```bash
50
+ xflow corps \
51
+ --title "Competitor-aligned workbench" \
52
+ --change-type frontend \
53
+ --change-id competitor-workbench \
54
+ --competitor-product "<name>" \
55
+ --primary-reference-surface primary_workspace \
56
+ --capture-url http://127.0.0.1:4174/ \
57
+ --reference-image refs/competitor-main.png
58
+ ```
59
+
60
+ Use `--reference-scenarios-json <path>` instead of `--capture-url` plus
61
+ `--reference-image` when the benchmark scenarios are already frozen.
62
+
63
+ ## Common Failure Meanings
64
+
65
+ | Issue | Meaning |
66
+ | --- | --- |
67
+ | `competitor_led_ui_requires_benchmark_input` | The entry detected competitor-led UI but cannot prove visual comparison without benchmark evidence. |
68
+ | `competitor_led_ui_requires_primary_reference_surface` | The entry cannot tell which surface anchors the comparison. |
69
+ | `stub runtime fallback` | A required heavy agent runtime was unavailable or unauthenticated, so completion is not claimable. |
70
+
71
+ ## Human Role
72
+
73
+ The conversation AI should not manually emulate corps. It represents the human
74
+ operator: clarify requirements, acknowledge human gates when needed, and inspect
75
+ `corps_proof.json` before accepting completion.
@@ -0,0 +1,126 @@
1
+ # evalid Demo Proof
2
+
3
+ This page defines what an external evaluator should run before trusting xflow claims.
4
+
5
+ ## Clean Project Adoption
6
+
7
+ From an empty temporary project:
8
+
9
+ ```bash
10
+ npm install -g @surlymochan/evalid
11
+ xflow demo clean
12
+ xflow guide
13
+ xflow assess
14
+ xflow demo launch
15
+ xflow init --project-root .
16
+ xflow goal set "Ship the next verified change" --project-root .
17
+ xflow goal audit --project-root . --json
18
+ xflow doctor --project-root .
19
+ xflow workflow validate yolo --project-root .
20
+ ```
21
+
22
+ Expected result:
23
+
24
+ - `.as-xflow/config.json` exists
25
+ - `.xflow/GOAL.md` exists and `xflow goal audit --json` passes
26
+ - doctor exits successfully
27
+ - `yolo` validates through project-local config
28
+ - no internal workflow files need to be copied into the consumer project
29
+
30
+ The repository CI runs this same path by creating a minimal temporary `package.json` on the fly.
31
+ For source checkout evaluation before npm publication, run `node bin/xflow.js
32
+ demo clean` from this repository; it creates the temporary project, runs the
33
+ same init/goal/doctor/yolo validation smoke, and cleans up automatically.
34
+
35
+ ## Source Checkout Release Proof
36
+
37
+ From the evalid repository:
38
+
39
+ ```bash
40
+ npm run release:pack
41
+ ```
42
+
43
+ This proves:
44
+
45
+ - full test suite passes
46
+ - active-surface drift scan passes
47
+ - installed xflow skill matches source
48
+ - package dry-run includes the runtime, docs, workflows, and skills
49
+ - package dry-run excludes local config, test fixtures, Python caches, and Finder metadata
50
+
51
+ ## TDD Proof Demonstration
52
+
53
+ A real yolo run must produce:
54
+
55
+ ```text
56
+ specs/changes/<change-id>/tdd/red-0.json
57
+ specs/changes/<change-id>/tdd/green-0.json
58
+ specs/changes/<change-id>/tdd/quality-0.json
59
+ ```
60
+
61
+ The required semantics are:
62
+
63
+ - `red-0.json` records a failing raw test command before implementation
64
+ - `green-0.json` records a passing raw test command after implementation
65
+ - `quality-0.json` rejects code changes without meaningful changed tests
66
+ - quality review rejects empty assertions, snapshot-only tests, and mock-heavy tests without real assertions
67
+
68
+ ## Competitive Claim Gate
69
+
70
+ Before claiming a new competitive advantage, run:
71
+
72
+ ```bash
73
+ xflow score
74
+ xflow goal audit --json
75
+ xflow corps --explain --json
76
+ xflow compare superpowers
77
+ xflow compare super-assistant
78
+ xflow compare openspec
79
+ xflow compare gstack
80
+ xflow compare spec-kit
81
+ xflow adapter import-file --input docs/fixtures/tracker-item.json --json
82
+ xflow spec delta --change-id <change-id>
83
+ npm run drift:scan
84
+ npm run release:pack
85
+ npm run publish:check
86
+ ```
87
+
88
+ The claim is not durable unless it is backed by:
89
+
90
+ - a user-facing command
91
+ - a durable doc page
92
+ - workflow or atom evidence where relevant
93
+ - a test that fails if the evidence disappears
94
+
95
+ ## Corps Capture-Url Proof
96
+
97
+ For competitor-led UI work, the heavy-track benchmark contract now supports two
98
+ formal evidence paths. One of them is the phase-local capture path:
99
+
100
+ ```bash
101
+ xflow corps \
102
+ --title "Competitor-aligned workbench" \
103
+ --change-type frontend \
104
+ --change-id competitor-workbench \
105
+ --competitor-product CompetitorX \
106
+ --required-modules workspace \
107
+ --required-modules detail \
108
+ --capture-url http://127.0.0.1:4174/ \
109
+ --reference-image refs/competitor-main.png \
110
+ --dry-run \
111
+ --json
112
+ ```
113
+
114
+ What must be true before calling the run complete:
115
+
116
+ - `H6.visual.benchmark` records `benchmark_input_mode: "capture_url"`
117
+ - `visual_benchmark.json` is `reference_backed`
118
+ - the captured scenario resolves both `reference_image` and `screenshot_image`
119
+ - `xflow proof --track corps --change-id <id>` returns `ok=true`
120
+
121
+ ## Minimal Example
122
+
123
+ Use the clean project adoption block above as the canonical minimal example.
124
+ Use [docs/launch-demo.md](./launch-demo.md) when the evaluator wants the public
125
+ goal-to-yolo and goal-to-corps proof story instead of only the minimal install
126
+ smoke.