@rpamis/comet 0.4.0-rc.1 → 0.4.0-rc.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -21
- package/README.md +3 -650
- package/assets/manifest.json +1 -1
- package/assets/skills/comet/scripts/comet-archive.mjs +74 -74
- package/assets/skills/comet/scripts/comet-guard.mjs +100 -100
- package/assets/skills/comet/scripts/comet-handoff.mjs +77 -77
- package/assets/skills/comet/scripts/comet-hook-guard.mjs +58 -59
- package/assets/skills/comet/scripts/comet-hook-router.mjs +103 -104
- package/assets/skills/comet/scripts/comet-intent.mjs +6 -6
- package/assets/skills/comet/scripts/comet-resume-probe.mjs +2 -2
- package/assets/skills/comet/scripts/comet-runtime.mjs +174 -175
- package/assets/skills/comet/scripts/comet-state.mjs +105 -105
- package/assets/skills/comet/scripts/comet-yaml-validate.mjs +2 -2
- package/assets/skills/comet-native/SKILL.md +5 -4
- package/assets/skills/comet-native/reference/commands.md +4 -4
- package/assets/skills/comet-native/scripts/comet-native-archive.mjs +108 -102
- package/assets/skills/comet-native/scripts/comet-native-doctor.mjs +130 -124
- package/assets/skills/comet-native/scripts/comet-native-hook-guard.mjs +96 -89
- package/assets/skills/comet-native/scripts/comet-native-init.mjs +120 -114
- package/assets/skills/comet-native/scripts/comet-native-new.mjs +112 -106
- package/assets/skills/comet-native/scripts/comet-native-next.mjs +121 -115
- package/assets/skills/comet-native/scripts/comet-native-root.mjs +100 -94
- package/assets/skills/comet-native/scripts/comet-native-runtime.mjs +164 -157
- package/assets/skills/comet-native/scripts/comet-native-select.mjs +113 -107
- package/assets/skills/comet-native/scripts/comet-native-show.mjs +106 -100
- package/assets/skills/comet-native/scripts/comet-native-spec.mjs +94 -88
- package/assets/skills/comet-native/scripts/comet-native-status.mjs +110 -104
- package/assets/skills-zh/comet-native/SKILL.md +5 -4
- package/assets/skills-zh/comet-native/reference/commands.md +4 -4
- package/dist/app/commands/status.d.ts.map +1 -1
- package/dist/app/commands/status.js +49 -9
- package/dist/app/commands/status.js.map +1 -1
- package/dist/app/commands/update.d.ts.map +1 -1
- package/dist/app/commands/update.js +1 -44
- package/dist/app/commands/update.js.map +1 -1
- package/dist/domains/comet-classic/classic-archive.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-archive.js +15 -0
- package/dist/domains/comet-classic/classic-archive.js.map +1 -1
- package/dist/domains/comet-classic/classic-cli.d.ts +8 -0
- package/dist/domains/comet-classic/classic-cli.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-cli.js +10 -0
- package/dist/domains/comet-classic/classic-cli.js.map +1 -1
- package/dist/domains/comet-classic/classic-guard.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-guard.js +20 -0
- package/dist/domains/comet-classic/classic-guard.js.map +1 -1
- package/dist/domains/comet-classic/classic-handoff.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-handoff.js +13 -0
- package/dist/domains/comet-classic/classic-handoff.js.map +1 -1
- package/dist/domains/comet-classic/classic-hook-guard.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-hook-guard.js +14 -15
- package/dist/domains/comet-classic/classic-hook-guard.js.map +1 -1
- package/dist/domains/comet-classic/classic-output-language.d.ts +73 -0
- package/dist/domains/comet-classic/classic-output-language.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-output-language.js +175 -0
- package/dist/domains/comet-classic/classic-output-language.js.map +1 -0
- package/dist/domains/comet-classic/classic-script-entry.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-script-entry.js +10 -0
- package/dist/domains/comet-classic/classic-script-entry.js.map +1 -1
- package/dist/domains/comet-classic/classic-state-command.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-state-command.js +43 -4
- package/dist/domains/comet-classic/classic-state-command.js.map +1 -1
- package/dist/domains/comet-entry/project-status.d.ts.map +1 -1
- package/dist/domains/comet-entry/project-status.js +15 -3
- package/dist/domains/comet-entry/project-status.js.map +1 -1
- package/dist/domains/comet-entry/types.d.ts +5 -1
- package/dist/domains/comet-entry/types.d.ts.map +1 -1
- package/dist/domains/comet-native/native-children.d.ts +16 -0
- package/dist/domains/comet-native/native-children.d.ts.map +1 -1
- package/dist/domains/comet-native/native-children.js +73 -7
- package/dist/domains/comet-native/native-children.js.map +1 -1
- package/dist/domains/comet-native/native-cli-help.d.ts.map +1 -1
- package/dist/domains/comet-native/native-cli-help.js +2 -1
- package/dist/domains/comet-native/native-cli-help.js.map +1 -1
- package/dist/domains/comet-native/native-cli-shared.d.ts +7 -1
- package/dist/domains/comet-native/native-cli-shared.d.ts.map +1 -1
- package/dist/domains/comet-native/native-cli-shared.js +51 -4
- package/dist/domains/comet-native/native-cli-shared.js.map +1 -1
- package/dist/domains/comet-native/native-cli.d.ts.map +1 -1
- package/dist/domains/comet-native/native-cli.js +4 -2
- package/dist/domains/comet-native/native-cli.js.map +1 -1
- package/dist/domains/comet-native/native-loop-runtime.d.ts +4 -1
- package/dist/domains/comet-native/native-loop-runtime.d.ts.map +1 -1
- package/dist/domains/comet-native/native-loop-runtime.js +18 -3
- package/dist/domains/comet-native/native-loop-runtime.js.map +1 -1
- package/dist/domains/comet-native/native-next-command.d.ts.map +1 -1
- package/dist/domains/comet-native/native-next-command.js +1 -0
- package/dist/domains/comet-native/native-next-command.js.map +1 -1
- package/dist/domains/comet-native/native-output-language.d.ts +69 -0
- package/dist/domains/comet-native/native-output-language.d.ts.map +1 -0
- package/dist/domains/comet-native/native-output-language.js +369 -0
- package/dist/domains/comet-native/native-output-language.js.map +1 -0
- package/dist/domains/comet-native/native-portable-continuation.d.ts.map +1 -1
- package/dist/domains/comet-native/native-portable-continuation.js +44 -0
- package/dist/domains/comet-native/native-portable-continuation.js.map +1 -1
- package/dist/domains/comet-native/native-portable-runtime.d.ts +1 -0
- package/dist/domains/comet-native/native-portable-runtime.d.ts.map +1 -1
- package/dist/domains/comet-native/native-portable-runtime.js +1 -1
- package/dist/domains/comet-native/native-portable-runtime.js.map +1 -1
- package/dist/domains/comet-native/native-portable-state.d.ts.map +1 -1
- package/dist/domains/comet-native/native-portable-state.js +5 -0
- package/dist/domains/comet-native/native-portable-state.js.map +1 -1
- package/dist/domains/comet-native/native-portable-types.d.ts +3 -0
- package/dist/domains/comet-native/native-portable-types.d.ts.map +1 -1
- package/dist/domains/comet-native/native-portable-types.js.map +1 -1
- package/dist/domains/comet-native/native-runner-input.d.ts +30 -28
- package/dist/domains/comet-native/native-runner-input.d.ts.map +1 -1
- package/dist/domains/comet-native/native-runner-input.js +5 -0
- package/dist/domains/comet-native/native-runner-input.js.map +1 -1
- package/dist/domains/comet-native/native-script-entry.d.ts +5 -4
- package/dist/domains/comet-native/native-script-entry.d.ts.map +1 -1
- package/dist/domains/comet-native/native-script-entry.js +8 -16
- package/dist/domains/comet-native/native-script-entry.js.map +1 -1
- package/dist/domains/comet-native/native-status-discovery.d.ts +31 -1
- package/dist/domains/comet-native/native-status-discovery.d.ts.map +1 -1
- package/dist/domains/comet-native/native-status-discovery.js +46 -8
- package/dist/domains/comet-native/native-status-discovery.js.map +1 -1
- package/dist/domains/dashboard/native-collector.d.ts.map +1 -1
- package/dist/domains/dashboard/native-collector.js +2 -0
- package/dist/domains/dashboard/native-collector.js.map +1 -1
- package/dist/domains/dashboard/web/index.html +14 -14
- package/dist/domains/workflow-contract/output-envelope.d.ts +59 -0
- package/dist/domains/workflow-contract/output-envelope.d.ts.map +1 -0
- package/dist/domains/workflow-contract/output-envelope.js +108 -0
- package/dist/domains/workflow-contract/output-envelope.js.map +1 -0
- package/dist/platform/version/version.d.ts +8 -0
- package/dist/platform/version/version.d.ts.map +1 -1
- package/dist/platform/version/version.js +58 -8
- package/dist/platform/version/version.js.map +1 -1
- package/eval/local/tasks/comet-native-wave-d-stagnation-stop/instruction.md +2 -0
- package/eval/local/tasks/comet-native-wave-d-stagnation-stop/task.toml +1 -0
- package/eval/local/tasks/comet-native-wave-d-stagnation-stop/validation/test_native_wave_d_stagnation_stop.py +37 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -155,21 +155,12 @@ npm install -g @rpamis/comet
|
|
|
155
155
|
|
|
156
156
|
## Quick Start
|
|
157
157
|
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
```bash
|
|
161
|
-
comet init --scope global
|
|
162
|
-
cd your-project
|
|
163
|
-
# Invoke /comet in the host
|
|
164
|
-
```
|
|
165
|
-
|
|
166
|
-
The first `/comet` invocation in an unconfigured project snapshots the defaults from `~/.comet/config.yaml` into the project's `.comet/config.yaml` and creates artifacts only inside that project. Later global-default changes do not rewrite activated projects. Hooks then recognize the project config and `.comet/current-change.json`; Hook events, `comet status`, and ordinary read-only resolution never initialize a project by themselves.
|
|
167
|
-
|
|
168
|
-
Run project initialization only when you want project-local copies of Skills, Rules, and Hooks, or explicit project overrides:
|
|
158
|
+
Initialize Comet in the project where you want to use it:
|
|
169
159
|
|
|
170
160
|
```bash
|
|
171
161
|
cd your-project
|
|
172
162
|
comet init
|
|
163
|
+
# Invoke /comet in the host
|
|
173
164
|
```
|
|
174
165
|
|
|
175
166
|
Interactive setup explains and offers Native, Classic, or both. Native is for strong models that can implement and verify autonomously; Classic is for tasks that need full Spec/TDD phase constraints; Both installs two independent entries while keeping `/comet` on Native by default. Non-interactive new projects default to Native, and project configuration is unified at `.comet/config.yaml`:
|
|
@@ -261,249 +252,9 @@ Comet Eval's automated dual-agent architecture can integrate online with LangSmi
|
|
|
261
252
|
</p>
|
|
262
253
|
<p align="center">Trace custom Rubric metrics with Pytest in LangSmith</p>
|
|
263
254
|
|
|
264
|
-
## Commands
|
|
265
|
-
|
|
266
|
-
<details>
|
|
267
|
-
<summary><code>comet init [path]</code> — Initialize Comet workflow</summary>
|
|
268
|
-
|
|
269
|
-
Initializes Comet for selected AI coding platforms. Interactive setup selects Native, Classic, or both; new non-interactive projects default to self-contained Native, while projects with existing Classic state remain Classic. Pass `--platform` to initialize only that platform; unknown but valid platform ids are treated as project-scoped custom platforms installed under `.<platform>/` with the selected workflow's skills, scripts, rules, and hooks. The workflows keep independent entries, state, artifacts, and Guards. Each platform installs one `comet-workflow-guard` Rule, and platforms with Hook support install only `comet-hook-router.mjs`. The Router uses `.comet/current-change.json` to send each write to exactly one current Native or Classic Guard. Only Classic installs OpenSpec and Superpowers; Native depends on no external Skill.
|
|
270
|
-
|
|
271
|
-
| Option | Description |
|
|
272
|
-
| ------------------- | ------------------------------------------------------------------------------ |
|
|
273
|
-
| `--yes` | Non-interactive mode, auto-select detected platforms (or all if none detected) |
|
|
274
|
-
| `--scope <scope>` | Install scope: `project` or `global` |
|
|
275
|
-
| `--language <lang>` | Skill language: `en` or `zh` (skips interactive language prompt) |
|
|
276
|
-
| `--platform <platform>` | Initialize only this platform; project scope accepts custom platform ids |
|
|
277
|
-
| `--workflow <mode>` | Workflows to initialize: `native`, `classic`, or `both` |
|
|
278
|
-
| `--root <path>` | Project-relative Native artifact root; global scope stores it as a project default |
|
|
279
|
-
| `--codegraph <action>` | Non-interactive project index action: explicitly choose `init` or `skip` |
|
|
280
|
-
| `--skip-existing` | Skip already installed components |
|
|
281
|
-
| `--overwrite` | Overwrite already installed components |
|
|
282
|
-
| `--json` | Output structured JSON |
|
|
283
|
-
|
|
284
|
-
When multiple existing components are found on the same platform, interactive init offers one bulk choice: overwrite
|
|
285
|
-
all, skip all, or choose per component.
|
|
286
|
-
|
|
287
|
-
</details>
|
|
288
|
-
|
|
289
|
-
<details>
|
|
290
|
-
<summary><code>comet status [path]</code> — Show active changes and next workflow command</summary>
|
|
291
|
-
|
|
292
|
-
Displays the configured default entry point and separate Native, Classic, and unmanaged OpenSpec sections. Native entries include phase, approval, verification, selection, and next-step data; Classic entries retain workflow, phase, runtime mode, current step, evidence, and recovery diagnostics.
|
|
293
|
-
|
|
294
|
-
| Option | Description |
|
|
295
|
-
| -------- | ------------------------------------------------------------------------- |
|
|
296
|
-
| `--json` | Output active changes with `nextCommand`, `currentStep`, and runtime data |
|
|
297
|
-
|
|
298
|
-
</details>
|
|
299
|
-
|
|
300
|
-
<details>
|
|
301
|
-
<summary><code>comet resume-probe [path]</code> — Decide whether an active Comet workflow should resume</summary>
|
|
302
|
-
|
|
303
|
-
Read-only probe that resolves the configured default workflow and inspects only that side. It returns `auto_resume`, `ask_user`, `out_of_scope`, or `none`, and routes Native to `/comet-native` or Classic to `/comet-classic`. A malformed root configuration stops recovery instead of falling back or scanning the other workflow. `comet init/update` still merges a `<comet-ambient-resume>` managed block into `AGENTS.md` and `CLAUDE.md` while preserving user-authored rules.
|
|
304
|
-
|
|
305
|
-
</details>
|
|
306
|
-
|
|
307
|
-
<details>
|
|
308
|
-
<summary><code>comet dashboard [path]</code> — Launch local read-only dashboard server</summary>
|
|
309
|
-
|
|
310
|
-
Starts a local HTTP server that displays a visual dashboard with active changes, phase status, task progress, and archive history. Auto-opens in your browser by default.
|
|
311
|
-
|
|
312
|
-
<p align="center">
|
|
313
|
-
<img src="https://github.com/rpamis/comet/blob/master/img/dashboard-light.png" alt="Comet Dashboard Light" width="800">
|
|
314
|
-
</p>
|
|
315
|
-
<p align="center">
|
|
316
|
-
<img src="https://github.com/rpamis/comet/blob/master/img/dashboard-dark.png" alt="Comet Dashboard Dark" width="800">
|
|
317
|
-
</p>
|
|
318
|
-
<p align="center">Active change overview with phase indicators, task progress, and archive history</p>
|
|
319
|
-
|
|
320
|
-
| Option | Description |
|
|
321
|
-
| ----------- | --------------------------------------------------------------------------- |
|
|
322
|
-
| `--port` | Server port (default: auto-selects available port) |
|
|
323
|
-
| `--no-open` | Don't auto-open the dashboard in browser |
|
|
324
|
-
| `--json` | Collect single snapshot and print JSON to stdout (for scripting/inspection) |
|
|
325
|
-
|
|
326
|
-
</details>
|
|
327
|
-
|
|
328
|
-
<details>
|
|
329
|
-
<summary><code>comet doctor [path]</code> — Diagnose Comet installation health</summary>
|
|
330
|
-
|
|
331
|
-
Checks project/global installation health, working directories, installed skills, scripts, CodeGraph indexes, and active change
|
|
332
|
-
diagnostics. `comet doctor` reports diagnostic status for malformed `.comet.yaml` files, current step / runtime mode
|
|
333
|
-
for valid changes, and runtime evidence gaps that block safe resume. In a Git secondary worktree, it separately reports
|
|
334
|
-
the current worktree, primary worktree, and global fallback installation state. Ignored assets in the primary worktree
|
|
335
|
-
are inspected only for classification and are never executed across worktrees.
|
|
336
|
-
CodeGraph diagnostics separately report whether the CLI is installed, whether the current project index is current, whether
|
|
337
|
-
MCP is registered with a supported Agent, and whether each Agent has effective capability; a healthy project index does not
|
|
338
|
-
mean that an Agent has CodeGraph MCP registration.
|
|
339
|
-
|
|
340
|
-
| Option | Description |
|
|
341
|
-
| ----------------- | --------------------------------------------------------------------------------------------- |
|
|
342
|
-
| `--json` | Output structured diagnostics, including CodeGraph state and the effective runtime source |
|
|
343
|
-
| `--scope <scope>` | Diagnose `auto`, `project`, or `global` scope (default: `auto`) |
|
|
344
|
-
| `--repair` | Repair deterministic managed-install and state problems |
|
|
345
|
-
| `--yes` | Use with `--repair` to authorize potentially expensive CodeGraph initialization, rebuild, or sync |
|
|
346
|
-
|
|
347
|
-
</details>
|
|
348
|
-
|
|
349
|
-
<details>
|
|
350
|
-
<summary><code>comet update [path]</code> — Update Comet package and skills</summary>
|
|
351
|
-
|
|
352
|
-
Refreshes installed Comet skills in detected project/global targets. Pass `--platform` to refresh only that platform, including project-scoped custom platforms. A current-project refresh does not mutate any npm installation, whether global or project-local, by default; pass `--self-update` explicitly when the CLI should be upgraded too. Explicit `--scope global` deterministically uses the current-project asset scope, never opens the all-projects selector, and still does not update the npm package implicitly. Self-update compares full semver values, including prereleases, refuses downgrades, and validates the candidate package's version, Workflow command, and Native command in isolation before installation. A failed install attempts to restore the exact installed version.
|
|
353
|
-
|
|
354
|
-
| Option | Description |
|
|
355
|
-
| -------------------- | ------------------------------------------------ |
|
|
356
|
-
| `--json` | Output npm and skill update results as JSON |
|
|
357
|
-
| `--language <lang>` | Override detected skill language (`en`, `zh`) |
|
|
358
|
-
| `--platform <platform>` | Refresh only this platform; project scope accepts custom platform ids |
|
|
359
|
-
| `--scope <scope>` | Update only the `global` or `project` install scope |
|
|
360
|
-
| `--current-project` | Refresh only the current project |
|
|
361
|
-
| `--all-projects` | Refresh all registered project installations |
|
|
362
|
-
| `--self-update` | Explicitly update the Comet npm package first |
|
|
363
|
-
| `--skip-self-update` | Explicitly skip Comet npm package self-update |
|
|
364
|
-
|
|
365
|
-
</details>
|
|
366
|
-
|
|
367
|
-
<details>
|
|
368
|
-
<summary><code>comet uninstall [path]</code> — Remove Comet skills, rules, and hooks</summary>
|
|
369
|
-
|
|
370
|
-
Safely removes Comet-distributed skills, rules, and hooks from all detected platforms. Preserves user-defined hooks and non-Comet configuration.
|
|
371
|
-
|
|
372
|
-
| Option | Description |
|
|
373
|
-
| ----------------- | ------------------------------------------ |
|
|
374
|
-
| `--force` | Skip confirmation prompt |
|
|
375
|
-
| `--scope <scope>` | Uninstall only `global` or `project` scope |
|
|
376
|
-
| `--json` | Output removal results as JSON |
|
|
377
|
-
|
|
378
|
-
```bash
|
|
379
|
-
comet uninstall # Interactive — shows targets, asks for confirmation
|
|
380
|
-
comet uninstall --force # Non-interactive — removes everything immediately
|
|
381
|
-
comet uninstall --scope project # Only remove project-level installations
|
|
382
|
-
```
|
|
383
|
-
|
|
384
|
-
</details>
|
|
385
|
-
|
|
386
|
-
<details>
|
|
387
|
-
<summary><code>comet eval [target]</code> — Evaluate Skills through the shared eval harness</summary>
|
|
388
|
-
|
|
389
|
-
`comet eval` answers a simple question: does this Skill actually work reliably on standard tasks?
|
|
390
|
-
|
|
391
|
-
The primary path is evaluating a local Skill that you already own. You do not need to run
|
|
392
|
-
`/comet-any` first or create `comet/eval.yaml`:
|
|
393
|
-
|
|
394
|
-
```bash
|
|
395
|
-
comet eval ./my-skill --collect
|
|
396
|
-
comet eval ./my-skill --html
|
|
397
|
-
comet eval ./my-skill --quick --html
|
|
398
|
-
```
|
|
399
|
-
|
|
400
|
-
`--collect` performs static discovery and configuration checks without starting an Agent, Docker, plugins,
|
|
401
|
-
credential checks, or network requests. A normal run generates and caches 2–4 tasks when the Skill has no
|
|
402
|
-
`comet/eval.yaml`; `--quick` uses the fixed `generic-skill-smoke` task. The report path is printed by the command
|
|
403
|
-
and is written under `.comet/eval/runs/<experiment-id>/` in the Skill or the project passed with `--project`.
|
|
404
|
-
|
|
405
|
-
If the Skill contains `comet/eval.yaml`, passing its directory discovers it automatically. You can also pass a
|
|
406
|
-
manifest or `SKILL.md` directly:
|
|
407
|
-
|
|
408
|
-
```bash
|
|
409
|
-
comet eval ./my-skill/comet/eval.yaml --collect
|
|
410
|
-
comet eval ./my-skill/SKILL.md --html
|
|
411
|
-
```
|
|
412
|
-
|
|
413
|
-
Configure the main Agent, model, and API URL directly. LLM-as-Judge has an independent Agent, model, API URL, and
|
|
414
|
-
credential boundary:
|
|
415
|
-
|
|
416
|
-
```yaml
|
|
417
|
-
execution:
|
|
418
|
-
agent: codex
|
|
419
|
-
model: subject-model
|
|
420
|
-
baseUrl: https://subject.example/v1
|
|
421
|
-
judge:
|
|
422
|
-
agent: claude-code
|
|
423
|
-
model: judge-model
|
|
424
|
-
baseUrl: https://judge.example/v1
|
|
425
|
-
```
|
|
426
|
-
|
|
427
|
-
`/comet-any` may still produce a compatible manifest, but it is an optional Skill producer, never a prerequisite
|
|
428
|
-
for `comet eval`.
|
|
429
|
-
|
|
430
|
-
### Reading Local Eval
|
|
431
|
-
|
|
432
|
-
Local eval is the normal path for day-to-day development and pre-release checks. In the HTML report, look first at:
|
|
433
|
-
|
|
434
|
-
- whether pass/fail and rubric scores match expectations
|
|
435
|
-
- whether failures are attributed to the Skill, workflow, task, model, or environment/harness
|
|
436
|
-
- whether expected artifacts are missing
|
|
437
|
-
- whether token use, cost, or duration look unusual
|
|
438
|
-
- whether the result is clean enough, or a specific task/treatment should be rerun
|
|
439
|
-
|
|
440
|
-
If the report says `Insufficient clean data` or `Inconclusive due to data quality`, check auth, rate limits,
|
|
441
|
-
Docker/container setup, network, and other environment issues before treating the run as a Skill-quality verdict.
|
|
442
|
-
|
|
443
|
-
### LangSmith Eval
|
|
444
|
-
|
|
445
|
-
Use the LangSmith suite when you want to sync eval results to LangSmith, or when your team wants to inspect runs, rubric
|
|
446
|
-
feedback, costs, and Claude Code trajectories together. It reuses the same tasks, treatments, rubric, and
|
|
447
|
-
`comet/eval.yaml`; the difference is that results are uploaded to LangSmith.
|
|
448
|
-
|
|
449
|
-
Prepare dependencies and environment variables once:
|
|
450
|
-
|
|
451
|
-
```bash
|
|
452
|
-
cd eval
|
|
453
|
-
uv sync --extra langsmith
|
|
454
|
-
```
|
|
455
|
-
|
|
456
|
-
```bash
|
|
457
|
-
LANGSMITH_API_KEY=lsv2_pt_...
|
|
458
|
-
LANGSMITH_PROJECT=comet-skill-eval
|
|
459
|
-
LANGSMITH_TRACING=true
|
|
460
|
-
```
|
|
461
|
-
|
|
462
|
-
Then select the LangSmith suite through the same `comet eval` entry point:
|
|
463
|
-
|
|
464
|
-
```bash
|
|
465
|
-
comet eval ./my-skill --suite langsmith --html
|
|
466
|
-
```
|
|
467
|
-
|
|
468
|
-
`--suite` defaults to `local`. With `langsmith`, the CLI starts the LangSmith runner, prints `Suite: langsmith` in its
|
|
469
|
-
launch details, and writes reports under the project's `.comet/eval/runs/`. In PowerShell, set
|
|
470
|
-
`$env:LANGSMITH_API_KEY`, `$env:LANGSMITH_PROJECT`, and `$env:LANGSMITH_TRACING`, or place them in `eval/.env`. See
|
|
471
|
-
[eval/langsmith/README.md](eval/langsmith/README.md) for plugin cache and trajectory tracing details.
|
|
472
|
-
|
|
473
|
-
### Which Path To Use
|
|
474
|
-
|
|
475
|
-
- Local Skill: run `comet eval ./my-skill --collect`, then `comet eval ./my-skill --html`
|
|
476
|
-
- Low-cost smoke: `comet eval ./my-skill --quick --html`
|
|
477
|
-
- `/comet-any` output: pass its directory for automatic manifest discovery, or pass `comet/eval.yaml` directly
|
|
478
|
-
- Team tracing and side-by-side comparison: `comet eval ./my-skill --suite langsmith --html`
|
|
479
|
-
|
|
480
|
-
For full task, treatment, report, and troubleshooting details, see the [Eval usage guide](docs/operations/EVAL-USAGE.md).
|
|
481
|
-
|
|
482
|
-
</details>
|
|
483
|
-
|
|
484
|
-
<details>
|
|
485
|
-
<summary><code>/comet-any</code> / <code>comet creator</code> / <code>comet publish</code> — Create, evaluate, and publish Skills</summary>
|
|
486
|
-
|
|
487
|
-
`/comet-any` is an optional Skill creation and composition path: Create or optimize a reusable Skill → validate it with `comet eval` → review and
|
|
488
|
-
distribute it, until it becomes a stable composed Skill. For resume and release, use `comet creator`,
|
|
489
|
-
`comet creator status` / `comet creator next`, `comet publish`, and `comet publish distribute --preview`. The README
|
|
490
|
-
does not expand the backend command list; see the [Skill creation guide](docs/operations/SKILL-CREATION.md) for Advanced Bundle backend and Advanced Engine Run details, including `comet skill run` / `comet skill continue`.
|
|
491
|
-
|
|
492
|
-
</details>
|
|
493
|
-
|
|
494
|
-
<details>
|
|
495
|
-
<summary><code>comet --help</code> / <code>comet --version</code> — Basic information</summary>
|
|
496
|
-
|
|
497
|
-
| Command | Description |
|
|
498
|
-
| ----------------- | ------------ |
|
|
499
|
-
| `comet --help` | Show help |
|
|
500
|
-
| `comet --version` | Show version |
|
|
501
|
-
|
|
502
|
-
</details>
|
|
503
|
-
|
|
504
255
|
## Supported Platforms
|
|
505
256
|
|
|
506
|
-
`comet init` supports
|
|
257
|
+
`comet init` supports 37 AI coding platforms:
|
|
507
258
|
|
|
508
259
|
<details>
|
|
509
260
|
<summary>View full platform list</summary>
|
|
@@ -531,404 +282,6 @@ does not expand the backend command list; see the [Skill creation guide](docs/op
|
|
|
531
282
|
|
|
532
283
|
</details>
|
|
533
284
|
|
|
534
|
-
## Skills
|
|
535
|
-
|
|
536
|
-
`comet init` installs skills for the selected workflow. Native depends only on Comet-owned skills; Classic additionally installs OpenSpec and Superpowers.
|
|
537
|
-
|
|
538
|
-
### Comet Skills
|
|
539
|
-
|
|
540
|
-
<details>
|
|
541
|
-
<summary>View Comet skills</summary>
|
|
542
|
-
|
|
543
|
-
| Skill | Description |
|
|
544
|
-
| ---------------- | -------------------------------------------------------------------------------------------- |
|
|
545
|
-
| `/comet` | Shared entry — routes to the configured Native or Classic workflow from `.comet/config.yaml` |
|
|
546
|
-
| `/comet-native` | Permanent Native entry — self-contained, recoverable Shape, Build, Verify, and Archive |
|
|
547
|
-
| `/comet-classic` | Permanent Classic entry — the five-phase OpenSpec + Superpowers workflow |
|
|
548
|
-
| `/comet-review` | Read-only manual review of the current Native or Classic change without advancing its phase |
|
|
549
|
-
| `/comet-open` | Classic phase 1: Open a change (proposal, design, task breakdown) |
|
|
550
|
-
| `/comet-design` | Classic phase 2: Deep design (brainstorming, Design Doc) |
|
|
551
|
-
| `/comet-build` | Classic phase 3: Plan and build (implementation plan, code commits) |
|
|
552
|
-
| `/comet-verify` | Classic phase 4: Verify and finish (testing, verification report) |
|
|
553
|
-
| `/comet-archive` | Classic phase 5: Archive (delta spec sync, status annotation) |
|
|
554
|
-
| `/comet-hotfix` | Classic preset: Quick bug fix |
|
|
555
|
-
| `/comet-tweak` | Classic preset: OpenSpec-chained medium change |
|
|
556
|
-
| `/comet-any` | Comet Skill Creator — Create or optimize a reusable Skill |
|
|
557
|
-
|
|
558
|
-
</details>
|
|
559
|
-
|
|
560
|
-
### Guard & Automation Scripts
|
|
561
|
-
|
|
562
|
-
<details>
|
|
563
|
-
<summary>View script list</summary>
|
|
564
|
-
|
|
565
|
-
| Script | Purpose |
|
|
566
|
-
| ------------------------- | ---------------------------------------------------------------------------------------------------------- |
|
|
567
|
-
| `comet-env.mjs` | Script discovery helper — prints the bundled scripts directory so skills can resolve sibling command paths |
|
|
568
|
-
| `comet-guard.mjs` | Phase transition guard — validates exit conditions, `--apply` auto-updates `.comet.yaml` |
|
|
569
|
-
| `comet-handoff.mjs` | Design handoff — generates deterministic context packages from OpenSpec artifacts with SHA256 tracing |
|
|
570
|
-
| `comet-archive.mjs` | One-command archive — validates state, syncs specs, moves to archive, updates status |
|
|
571
|
-
| `comet-yaml-validate.mjs` | Schema validator — validates `.comet.yaml` structure and field values |
|
|
572
|
-
| `comet-state.mjs` | Unified state management — init/set/get/check/scale, agents' exclusive YAML interface |
|
|
573
|
-
| `comet-hook-router.mjs` | The platform's only Hook entry — routes by current ownership to one workflow Guard |
|
|
574
|
-
| `comet-hook-guard.mjs` | Classic Guard launcher — calls Classic runtime when selected by the Router |
|
|
575
|
-
| `comet-native-runtime.mjs` | Native state, checks, archive, and recovery runtime |
|
|
576
|
-
| `comet-native-hook-guard.mjs` | Native Guard launcher — calls Native runtime when selected by the Router |
|
|
577
|
-
|
|
578
|
-
Both Native and Classic runtimes are generated from TypeScript and run through `node` on every platform. Native does
|
|
579
|
-
not depend on OpenSpec, Superpowers, Bash, Git Bash, or WSL.
|
|
580
|
-
|
|
581
|
-
</details>
|
|
582
|
-
|
|
583
|
-
## Workflow
|
|
584
|
-
|
|
585
|
-
### Native Workflow
|
|
586
|
-
|
|
587
|
-
`/comet-native` uses Shape → Build → Verify → Archive. Shape owns clarification, the brief, complete target specs, and
|
|
588
|
-
user approval; the agent chooses its own planning and implementation methods. User-readable state, specifications, and
|
|
589
|
-
verification material live under `docs/comet/` by default; machine Runtime, locks, transactions, and recovery data live
|
|
590
|
-
under the project-local `.comet/runtime/native/`.
|
|
591
|
-
|
|
592
|
-
<details>
|
|
593
|
-
<summary>View the Native phase flow</summary>
|
|
594
|
-
|
|
595
|
-
```text
|
|
596
|
-
/comet-native (or /comet when default_workflow: native)
|
|
597
|
-
Shape ──confirm requirements──> Build ──Builder handoff──> Verify ──pass──> Archive
|
|
598
|
-
^ │
|
|
599
|
-
└────────── fail ──────────────┘
|
|
600
|
-
```
|
|
601
|
-
|
|
602
|
-
| Phase | Main work | Required result |
|
|
603
|
-
| ------- | ------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
|
|
604
|
-
| Shape | Inspect the environment, clarify in rounds, and write the brief plus complete target specs | No blocking questions; the user confirms the current outcome |
|
|
605
|
-
| Build | The agent plans, implements, self-tests, and submits a concise Builder handoff | Every acceptance item is addressed; ordinary source writes occur in Build |
|
|
606
|
-
| Verify | Runtime runs necessary checks and dispatches a fresh read-only Verifier | Pass advances to Archive; failure returns to Build with specific gaps |
|
|
607
|
-
| Archive | Sync complete target specs and move the change without rerunning Build or Verify checks | The recoverable archive transaction commits |
|
|
608
|
-
|
|
609
|
-
Shape supports `clarification_mode: sequential` and `clarification_mode: batch`. Sequential asks one upstream question per round; Batch asks every currently answerable question whose prerequisites are settled and is the default for new projects. In both modes, the agent investigates discoverable facts and writes user decisions back into the brief and complete target specs.
|
|
610
|
-
|
|
611
|
-
</details>
|
|
612
|
-
|
|
613
|
-
<details>
|
|
614
|
-
<summary>View Native state and artifacts</summary>
|
|
615
|
-
|
|
616
|
-
| File or directory | Purpose |
|
|
617
|
-
| ----------------------------------------------- | ------------------------------------------------------------------------------- |
|
|
618
|
-
| `.comet/config.yaml` | Select workflows, artifact root, language, clarification mode, and project-local Hook write allow paths |
|
|
619
|
-
| `.comet/current-change.json` | Shared Native/Classic ownership; each write routes to one workflow |
|
|
620
|
-
| `docs/comet/changes/<name>/comet-state.yaml` | Portable phase, Loop, acceptance results, blockers, and next action |
|
|
621
|
-
| `docs/comet/changes/<name>/brief.md` | Outcome, scope, non-goals, examples, constraints, decisions, and open questions |
|
|
622
|
-
| `docs/comet/changes/<name>/specs/` | Complete target behavior for each capability after archive |
|
|
623
|
-
| `docs/comet/changes/<name>/verification.md` | Human-readable Runtime checks, acceptance results, risks, and conclusion |
|
|
624
|
-
| `.comet/runtime/native/changes/<name>/` | Local `state.json` execution overlay and on-demand logs; rebuildable from portable state |
|
|
625
|
-
| `.comet/runtime/native/locks/` / `transactions/` | Project-local locks and recoverable transactions |
|
|
626
|
-
| `docs/comet/specs/` / `archive/` | Applied specs, archived changes, and user-readable verification material |
|
|
627
|
-
|
|
628
|
-
Native can keep multiple active changes at the same time. `comet status` lists the candidates, while `.comet/current-change.json` selects ownership for the current request; it does not limit the project to one change. Missing, stale, or ambiguous selection stops resume and writes for an explicit choice instead of guessing another change or switching to Classic.
|
|
629
|
-
|
|
630
|
-
Runtime owns `comet-state.yaml` and machine state under `.comet/runtime/native/`. Requirement edits return the change to Shape; implementation edits return it to Build before a fresh Verifier reviews the candidate. Do not hand-edit state to skip a phase.
|
|
631
|
-
|
|
632
|
-
Repositories that require their own PR template or validation script can opt into a Native repository-command provider in `.comet/config.yaml`:
|
|
633
|
-
|
|
634
|
-
```yaml
|
|
635
|
-
native:
|
|
636
|
-
finish:
|
|
637
|
-
pull_request:
|
|
638
|
-
provider: repository-command
|
|
639
|
-
command: [pwsh, -NoProfile, -File, scripts/comet-create-pr.ps1]
|
|
640
|
-
timeout_ms: 120000
|
|
641
|
-
```
|
|
642
|
-
|
|
643
|
-
After a `pull-request` Archive finish is selected, Comet still owns commit, push, existing-PR observation, base/head/head-SHA verification, idempotent recovery, and safe cleanup. The repository command owns only title/body/template policy and repository-specific remote validation. Its executable must be either a bare name resolved through PATH or a project-relative path. It runs from the project root, receives `comet.native.pull-request-finish-input.v1` JSON on stdin, and must return `comet.native.pull-request-finish-result.v1` JSON on stdout. GitHub CLI (`gh`) must still be installed and authenticated because Comet independently verifies remote state with `gh pr list` and `gh pr view`. Projects without this configuration keep the compatible `gh pr create --fill` behavior.
|
|
644
|
-
|
|
645
|
-
</details>
|
|
646
|
-
|
|
647
|
-
<details>
|
|
648
|
-
<summary>View Native reliability and recovery</summary>
|
|
649
|
-
|
|
650
|
-
1. **Clarification blocking** — Shape persists unresolved implementation decisions as blocking questions. Batch mode also requires explicit shared-understanding confirmation before Build.
|
|
651
|
-
2. **Builders cannot self-verify** — Build submits an implementation handoff. Runtime runs necessary checks and gives the brief, target specs, acceptance items, and results to a fresh read-only Verifier.
|
|
652
|
-
3. **Complete acceptance review** — The Verifier must return passed, failed, or blocked with a reason for every item. Missing, duplicate, and unknown items cannot pass.
|
|
653
|
-
4. **Bounded repair Loop** — Failure returns to Build. Repeated execution errors or no-progress results stop and return control to the user instead of looping indefinitely.
|
|
654
|
-
5. **Lightweight recovery** — `comet-state.yaml` keeps the stable phase, Loop, and next action. Interrupted local work reruns only necessary checks, not Shape, Build, or an already completed matching check.
|
|
655
|
-
6. **Protected state and transactions** — Runtime uses atomic state writes and short locks. Archive uses a recoverable transaction and does not rerun verification on the normal path.
|
|
656
|
-
7. **Unified guards without merged workflows** — Each platform installs one Rule and one Hook Router. The Router invokes Native Guard from current ownership; Native permits ordinary implementation writes only in Build and keeps its phases, schema, directories, and Guard independent from Classic.
|
|
657
|
-
|
|
658
|
-
</details>
|
|
659
|
-
|
|
660
|
-
### Classic Five-Phase Workflow
|
|
661
|
-
|
|
662
|
-
<details>
|
|
663
|
-
<summary>View the Classic five-phase flow</summary>
|
|
664
|
-
|
|
665
|
-
```
|
|
666
|
-
/comet-classic (or /comet when default_workflow: classic)
|
|
667
|
-
↓
|
|
668
|
-
/comet-open --> /comet-design --> /comet-build --> /comet-verify --> /comet-archive
|
|
669
|
-
(OpenSpec) (Superpowers) (Superpowers) (Both) (OpenSpec)
|
|
670
|
-
|
|
671
|
-
/comet-hotfix (preset path, skips brainstorming)
|
|
672
|
-
open --> build --> verify --> archive
|
|
673
|
-
|
|
674
|
-
/comet-tweak (lightweight preset, chains OpenSpec)
|
|
675
|
-
open --> build --> verify --> archive
|
|
676
|
-
```
|
|
677
|
-
|
|
678
|
-
#### Five Phases
|
|
679
|
-
|
|
680
|
-
| Phase | Command | Owner | Artifacts |
|
|
681
|
-
| ------------------ | ---------------- | ----------- | ------------------------------------ |
|
|
682
|
-
| 1. Open | `/comet-open` | OpenSpec | proposal.md, design.md, tasks.md |
|
|
683
|
-
| 2. Deep Design | `/comet-design` | Superpowers | Design Doc, delta spec |
|
|
684
|
-
| 3. Plan & Build | `/comet-build` | Superpowers | Implementation plan, code commits |
|
|
685
|
-
| 4. Verify & Finish | `/comet-verify` | Both | Verification report, branch handling |
|
|
686
|
-
| 5. Archive | `/comet-archive` | OpenSpec | delta→main spec sync, archive |
|
|
687
|
-
|
|
688
|
-
</details>
|
|
689
|
-
|
|
690
|
-
### Classic State Management
|
|
691
|
-
|
|
692
|
-
Classic uses a decoupled state architecture with separate files:
|
|
693
|
-
|
|
694
|
-
<details>
|
|
695
|
-
<summary>View Classic state management</summary>
|
|
696
|
-
|
|
697
|
-
| File | Owner | Purpose |
|
|
698
|
-
| ----------------------------------------- | -------- | --------------------------------------------------- |
|
|
699
|
-
| `.openspec.yaml` | OpenSpec | Spec lifecycle, change metadata |
|
|
700
|
-
| `<classic-change-dir>/.comet.yaml` | Comet | Workflow phase, execution mode, verification status |
|
|
701
|
-
| `.comet/run-state.json` | Engine | Run identity and execution state (machine-owned) |
|
|
702
|
-
| `.comet/state-events.jsonl` | Comet | Append-only state transition audit log |
|
|
703
|
-
|
|
704
|
-
Each change-level `.comet.yaml` stores Classic workflow state and only keeps `run_id` as the link to the Engine Run.
|
|
705
|
-
`<classic-change-dir>` is resolved from the project's `classic.artifact_layout`; use `comet classic root show` to inspect the current OpenSpec root.
|
|
706
|
-
Machine-owned Engine state lives in the change's `.comet/run-state.json` with camelCase fields such as `currentStep`,
|
|
707
|
-
`status`, and `iteration`. Legacy Run fields left in YAML are migrated after compatibility reads, and `skill` is no
|
|
708
|
-
longer a valid current `.comet.yaml` field. Project defaults live in `.comet/config.yaml`.
|
|
709
|
-
|
|
710
|
-
Phase progression is handled consistently by the TypeScript transition table, `comet-state transition`,
|
|
711
|
-
`comet-guard --apply`, and archive commands. Each successful progression appends an audit event to
|
|
712
|
-
`.comet/state-events.jsonl` with the source, before/after state, and actual field changes.
|
|
713
|
-
|
|
714
|
-
This keeps Skill text focused on guiding the agent while scripts own state writes, phase checks, auditability, and
|
|
715
|
-
breakpoint recovery. Agents can use Comet commands to know which phase the current Spec is in.
|
|
716
|
-
|
|
717
|
-
</details>
|
|
718
|
-
|
|
719
|
-
<details>
|
|
720
|
-
<summary>View key Classic change .comet.yaml fields</summary>
|
|
721
|
-
|
|
722
|
-
**Key Fields in change `.comet.yaml`:**
|
|
723
|
-
|
|
724
|
-
```yaml
|
|
725
|
-
workflow: full # Workflow type: full | tweak | hotfix
|
|
726
|
-
phase: build # Current phase: open | design | build | verify | archive
|
|
727
|
-
context_compression: off # Context compression: off | beta
|
|
728
|
-
auto_transition: true # Auto-invoke the next Skill after phase completion
|
|
729
|
-
base_ref: <git-sha-or-null> # Baseline commit captured at init; may be null
|
|
730
|
-
created_at: YYYY-MM-DD # Creation date written by comet-state.mjs init
|
|
731
|
-
run_id: <uuid> # Links to .comet/run-state.json only; Run details stay out of YAML
|
|
732
|
-
review_mode: standard # Automatic review strength: off | standard | thorough
|
|
733
|
-
build_mode: subagent-driven-development # Build mode: subagent-driven-development | executing-plans | direct
|
|
734
|
-
build_pause: null # `build_pause` records an internal build-phase pause point: null none, `plan-ready` means the plan has been generated
|
|
735
|
-
subagent_dispatch: null # Dispatch confirmation; confirm before verify
|
|
736
|
-
tdd_mode: null # Full-workflow build choice: tdd | direct
|
|
737
|
-
isolation: branch # Isolation mode: current | branch | worktree
|
|
738
|
-
bound_branch: null # Git branch bound by current/branch/worktree isolation; branch drift blocks progress
|
|
739
|
-
verify_mode: null # Verification mode: light | full
|
|
740
|
-
design_doc: docs/superpowers/specs/<design-doc>.md # Design doc path
|
|
741
|
-
plan: docs/superpowers/plans/YYYY-MM-DD-feature.md # Implementation plan path
|
|
742
|
-
verify_result: pending # Verification result: pending | pass | fail
|
|
743
|
-
verification_report: null # Verification report path; must exist before verify-pass
|
|
744
|
-
branch_status: pending # Branch handling status: pending | handled
|
|
745
|
-
verified_at: null # Verification timestamp; null before verification passes
|
|
746
|
-
archived: false # Archived changes are blocked from further mutation
|
|
747
|
-
direct_override: null # Must be true when a full workflow chooses direct build
|
|
748
|
-
handoff_context: null # Design handoff context path written by comet-handoff.mjs
|
|
749
|
-
handoff_hash: null # SHA256 for handoff_context; 64 hex chars when present
|
|
750
|
-
classic_profile: full # Machine-maintained Classic profile
|
|
751
|
-
classic_migration: 1 # Machine-maintained migration version
|
|
752
|
-
```
|
|
753
|
-
|
|
754
|
-
Current change `.comet.yaml` no longer contains `skill`; legacy Run fields in YAML are migrated to `.comet/run-state.json`.
|
|
755
|
-
|
|
756
|
-
</details>
|
|
757
|
-
|
|
758
|
-
### Classic Reliability and Shared Guards
|
|
759
|
-
|
|
760
|
-
The first six items below describe Classic phase automation; the final item covers the Router/Rule boundary shared by
|
|
761
|
-
Native and Classic:
|
|
762
|
-
|
|
763
|
-
<details>
|
|
764
|
-
<summary>View Classic reliability and shared guards</summary>
|
|
765
|
-
|
|
766
|
-
1. **Entry Verification** — Each phase validates preconditions before execution
|
|
767
|
-
- Checks file existence, state consistency, and phase transitions
|
|
768
|
-
- Outputs `[HARD STOP]` with actionable suggestions if validation fails
|
|
769
|
-
|
|
770
|
-
2. **Automated State Transitions** — `comet-guard.mjs --apply` updates `.comet.yaml` automatically
|
|
771
|
-
- All phase transitions (open → design/build → verify → archive) use `guard --apply`
|
|
772
|
-
- No manual state editing required — eliminates write-verification errors
|
|
773
|
-
- `comet-state.mjs` is the agents' exclusive interface for state operations
|
|
774
|
-
- Guard and archive scripts use `comet-state.mjs` internally for state management
|
|
775
|
-
|
|
776
|
-
3. **Schema Validation** — `comet-yaml-validate.mjs` ensures data integrity
|
|
777
|
-
- Validates required and optional fields
|
|
778
|
-
- Validates enum values, including `direct_override`
|
|
779
|
-
- Validates `design_doc`, `plan`, and `handoff_context` paths exist, plus `handoff_hash` format
|
|
780
|
-
- Detects unknown/typos fields
|
|
781
|
-
|
|
782
|
-
4. **Build Decision Enforcement** — Guard and state transitions both block skipped build choices
|
|
783
|
-
- `isolation` must be `current`, `branch`, or `worktree`
|
|
784
|
-
- `isolation: current`, `branch`, and `worktree` bind the current Git branch, and later entry checks block accidental branch drift
|
|
785
|
-
- `build_mode` must be selected before leaving build
|
|
786
|
-
- `build_pause: plan-ready` is a recoverable pause after plan generation, not a `build_mode`
|
|
787
|
-
- Full workflow `build_mode: direct` requires `direct_override: true`
|
|
788
|
-
|
|
789
|
-
5. **Verification Evidence** — Guard enforces proof before phase advance
|
|
790
|
-
- `verify-pass` transition requires `verification_report` pointing to an existing report file
|
|
791
|
-
- `branch_status` must be `handled` before verify can pass
|
|
792
|
-
- Guard checks `verification_report exists` and `branch_status=handled` as hard prerequisites
|
|
793
|
-
- Prevents false phase advances when verification or branch handling was skipped
|
|
794
|
-
|
|
795
|
-
6. **Archive Automation** — `comet-archive.mjs` handles the full archive flow in one command
|
|
796
|
-
- Validates entry state, merges delta specs into main specs through OpenSpec
|
|
797
|
-
- Annotates design doc and plan frontmatter
|
|
798
|
-
- Moves change to archive directory and updates `archived: true`
|
|
799
|
-
- Supports `--dry-run` for preview
|
|
800
|
-
|
|
801
|
-
7. **Anti-drift Phase Guards** — Phase awareness for long-context sessions
|
|
802
|
-
- Rule layer: each platform installs one `comet-workflow-guard`, resolves enabled, default, and current ownership
|
|
803
|
-
from `.comet/current-change.json`, and applies only one phase model
|
|
804
|
-
- Hook layer: platforms with Hook support install only `comet-hook-router.mjs`; one write reaches at most one Guard,
|
|
805
|
-
and ambiguous or stale selection fails closed
|
|
806
|
-
- Guard layer: Native and Classic keep independent phases and allowed paths. Native permits ordinary implementation
|
|
807
|
-
writes only in Build; Classic permits them in Build and Verify
|
|
808
|
-
|
|
809
|
-
</details>
|
|
810
|
-
|
|
811
|
-
## Project Structure
|
|
812
|
-
|
|
813
|
-
<details>
|
|
814
|
-
<summary>Native project structure (default <code>docs/comet/</code>)</summary>
|
|
815
|
-
|
|
816
|
-
```text
|
|
817
|
-
your-project/
|
|
818
|
-
├── .comet/
|
|
819
|
-
│ ├── config.yaml # Shared config and Native artifact_root
|
|
820
|
-
│ ├── current-change.json # Optional current ownership shared by Native and Classic
|
|
821
|
-
│ └── runtime/native/ # Git-ignored Native machine Runtime
|
|
822
|
-
│ ├── changes/<name>/ # Local state.json and on-demand logs
|
|
823
|
-
│ ├── locks/ # Project-local locks
|
|
824
|
-
│ └── transactions/ # Recoverable transactions
|
|
825
|
-
├── .claude/skills/ # Claude Code shown as the platform example
|
|
826
|
-
│ ├── comet/SKILL.md # Configuration-driven shared entry
|
|
827
|
-
│ └── comet-native/
|
|
828
|
-
│ ├── SKILL.md # Permanent Native entry
|
|
829
|
-
│ ├── reference/ # Artifact, command, and recovery contracts
|
|
830
|
-
│ └── scripts/ # Native runtime and Guard launcher
|
|
831
|
-
└── docs/comet/ # native.artifact_root: docs
|
|
832
|
-
├── specs/<capability>/spec.md # Archived canonical specs
|
|
833
|
-
├── changes/<name>/
|
|
834
|
-
│ ├── comet-state.yaml # Native change state
|
|
835
|
-
│ ├── brief.md # Outcome, scope, decisions, and acceptance
|
|
836
|
-
│ ├── specs/<capability>/spec.md # Complete target specification
|
|
837
|
-
│ ├── verification.md # Verification report
|
|
838
|
-
└── archive/YYYY-MM-DD-<name>/ # Archived changes
|
|
839
|
-
```
|
|
840
|
-
|
|
841
|
-
</details>
|
|
842
|
-
|
|
843
|
-
<details>
|
|
844
|
-
<summary>Classic project structure</summary>
|
|
845
|
-
|
|
846
|
-
```
|
|
847
|
-
your-project/
|
|
848
|
-
├── .comet/
|
|
849
|
-
│ └── config.yaml # Shared project config (workflows, Ambient Resume, language, Classic defaults)
|
|
850
|
-
├── .claude/skills/ # Platform skills dir (Comet + OpenSpec + Superpowers)
|
|
851
|
-
│ ├── comet/SKILL.md
|
|
852
|
-
│ │ └── scripts/
|
|
853
|
-
│ │ ├── comet-guard.mjs # Phase transition guard (--apply auto-updates state)
|
|
854
|
-
│ │ ├── comet-env.mjs # Script discovery helper
|
|
855
|
-
│ │ ├── comet-handoff.mjs # Design handoff (OpenSpec → Superpowers context tracing)
|
|
856
|
-
│ │ ├── comet-archive.mjs # One-command archive automation
|
|
857
|
-
│ │ ├── comet-yaml-validate.mjs # Schema validator
|
|
858
|
-
│ │ ├── comet-hook-router.mjs # The platform's only Hook entry (routes the current workflow Guard)
|
|
859
|
-
│ │ ├── comet-hook-guard.mjs # Classic Guard (not installed as a separate platform Hook)
|
|
860
|
-
│ │ └── comet-state.mjs # Unified state management (init/set/get/check/scale)
|
|
861
|
-
│ ├── comet-*/SKILL.md
|
|
862
|
-
│ ├── openspec-*/SKILL.md
|
|
863
|
-
│ └── brainstorming/SKILL.md
|
|
864
|
-
└── docs/
|
|
865
|
-
├── openspec/ # OpenSpec — WHAT (classic.artifact_layout: docs)
|
|
866
|
-
│ ├── config.yaml
|
|
867
|
-
│ ├── specs/<capability>/spec.md # Main specs after archive
|
|
868
|
-
│ └── changes/
|
|
869
|
-
│ ├── archive/YYYY-MM-DD-<name>/ # Archived changes
|
|
870
|
-
│ └── <name>/
|
|
871
|
-
│ ├── .openspec.yaml # OpenSpec state
|
|
872
|
-
│ ├── .comet.yaml # Comet workflow state (Classic fields + run_id link)
|
|
873
|
-
│ ├── .comet/
|
|
874
|
-
│ │ ├── run-state.json # Engine Run state (machine-owned, auto-migrated)
|
|
875
|
-
│ │ └── state-events.jsonl # State transition audit log (append-only)
|
|
876
|
-
│ ├── proposal.md
|
|
877
|
-
│ ├── design.md
|
|
878
|
-
│ ├── specs/<capability>/spec.md
|
|
879
|
-
│ └── tasks.md
|
|
880
|
-
└── superpowers/ # Superpowers — HOW
|
|
881
|
-
├── specs/ # Design documents
|
|
882
|
-
├── plans/ # Implementation plans
|
|
883
|
-
└── reports/ # Verification reports
|
|
884
|
-
```
|
|
885
|
-
|
|
886
|
-
New Classic and dual-workflow projects default to `classic.artifact_layout: docs`. Projects that retain the legacy layout keep the OpenSpec root at repository-level `openspec/`, while Superpowers artifacts remain under `docs/superpowers/`. Use `comet classic root show` to inspect the current layout. To migrate, run `comet classic root move docs --dry-run` first and then apply the returned plan with `--apply`; do not edit the configuration field by itself.
|
|
887
|
-
|
|
888
|
-
</details>
|
|
889
|
-
|
|
890
|
-
<details>
|
|
891
|
-
<summary>Classic Context Compression (Beta)</summary>
|
|
892
|
-
|
|
893
|
-
Comet supports context compression at the Design → Build handoff. When enabled, `comet-handoff.mjs` generates a compact
|
|
894
|
-
context package that reduces Build-phase input tokens by **25–30%** without affecting implementation correctness.
|
|
895
|
-
|
|
896
|
-
| Mode | Behavior | Token Savings |
|
|
897
|
-
| ------ | ---------------------------------------- | ------------- |
|
|
898
|
-
| `off` | Full Spec excerpts in handoff context | Baseline |
|
|
899
|
-
| `beta` | Design Doc + SHA256 hash references only | ~25–30% |
|
|
900
|
-
|
|
901
|
-
Key findings from benchmark testing:
|
|
902
|
-
|
|
903
|
-
- **Test pass rate**: 100% across all tiers (compression does not affect correctness)
|
|
904
|
-
- **Spec coverage**: 100% (off) vs 95% (beta) — minor edge-case detail loss
|
|
905
|
-
- **Scaling**: Larger tasks yield higher absolute savings (up to 15,000 tokens for large-tier tasks)
|
|
906
|
-
|
|
907
|
-
Enable `context_compression: beta` under the `classic:` block in `.comet/config.yaml`.
|
|
908
|
-
|
|
909
|
-
See [CONTEXT-COMPRESSION.md](docs/CONTEXT-COMPRESSION.md) for the full benchmark report, compression principles, and
|
|
910
|
-
reproduction steps.
|
|
911
|
-
|
|
912
|
-
</details>
|
|
913
|
-
|
|
914
|
-
<details>
|
|
915
|
-
<summary>Classic Auto Transition</summary>
|
|
916
|
-
|
|
917
|
-
`auto_transition` controls whether Comet automatically invokes the next skill after a phase completes, or pauses for
|
|
918
|
-
manual handoff. Phase advancement itself always happens — this setting only affects skill invocation.
|
|
919
|
-
|
|
920
|
-
| Value | Behavior |
|
|
921
|
-
| ------- | ------------------------------------------------------------- |
|
|
922
|
-
| `true` | Auto-invoke the next skill after each phase (default) |
|
|
923
|
-
| `false` | Pause after each phase; user manually triggers the next skill |
|
|
924
|
-
|
|
925
|
-
Three-layer configuration with precedence: `COMET_AUTO_TRANSITION` env var > `classic.auto_transition` in
|
|
926
|
-
`.comet/config.yaml` (project) > change `.comet.yaml`.
|
|
927
|
-
|
|
928
|
-
See [AUTO-TRANSITION.md](docs/AUTO-TRANSITION.md) for configuration details, workflow mapping, and FAQ.
|
|
929
|
-
|
|
930
|
-
</details>
|
|
931
|
-
|
|
932
285
|
## Development
|
|
933
286
|
|
|
934
287
|
See [CONTRIBUTING.md](CONTRIBUTING.md) | [中文版](CONTRIBUTING-zh.md) for development setup, commit
|