lightcone-cli 0.3.4__tar.gz → 0.3.5__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (75) hide show
  1. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/.gitignore +9 -0
  2. lightcone_cli-0.3.5/PKG-INFO +71 -0
  3. lightcone_cli-0.3.5/README.md +50 -0
  4. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/claude/lightcone/scripts/session-start.sh +13 -6
  5. lightcone_cli-0.3.5/claude/lightcone/skills/README.md +43 -0
  6. lightcone_cli-0.3.4/claude/lightcone/guides/astra-reference.md → lightcone_cli-0.3.5/claude/lightcone/skills/astra/SKILL.md +28 -1
  7. lightcone_cli-0.3.5/claude/lightcone/skills/check-sentence-by-sentence/SKILL.md +369 -0
  8. lightcone_cli-0.3.5/claude/lightcone/skills/figure-comparison/SKILL.md +578 -0
  9. lightcone_cli-0.3.4/claude/lightcone/guides/lightcone-cli-reference.md → lightcone_cli-0.3.5/claude/lightcone/skills/lc-cli/SKILL.md +31 -3
  10. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/claude/lightcone/skills/lc-feedback/SKILL.md +1 -2
  11. {lightcone_cli-0.3.4/claude/lightcone/skills/lc-migrate → lightcone_cli-0.3.5/claude/lightcone/skills/lc-from-code}/SKILL.md +25 -13
  12. lightcone_cli-0.3.5/claude/lightcone/skills/lc-from-paper/SKILL.md +164 -0
  13. lightcone_cli-0.3.5/claude/lightcone/skills/lc-from-paper/references/architect.md +112 -0
  14. lightcone_cli-0.3.5/claude/lightcone/skills/lc-from-paper/references/compare.md +118 -0
  15. lightcone_cli-0.3.5/claude/lightcone/skills/lc-from-paper/references/implement.md +109 -0
  16. lightcone_cli-0.3.5/claude/lightcone/skills/lc-from-paper/references/literature.md +199 -0
  17. lightcone_cli-0.3.5/claude/lightcone/skills/lc-from-paper/references/orient.md +235 -0
  18. lightcone_cli-0.3.5/claude/lightcone/skills/lc-from-paper/references/review.md +108 -0
  19. lightcone_cli-0.3.5/claude/lightcone/skills/lc-from-paper/references/run.md +57 -0
  20. lightcone_cli-0.3.5/claude/lightcone/skills/lc-from-paper/references/specify.md +198 -0
  21. lightcone_cli-0.3.5/claude/lightcone/skills/lc-from-paper/templates/CLAUDE.md +36 -0
  22. lightcone_cli-0.3.5/claude/lightcone/skills/lc-from-paper/templates/constitution.md +45 -0
  23. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/claude/lightcone/skills/lc-new/SKILL.md +5 -8
  24. lightcone_cli-0.3.5/claude/lightcone/skills/narrative/SKILL.md +228 -0
  25. lightcone_cli-0.3.5/claude/lightcone/skills/narrative/references/co-drafting.md +79 -0
  26. lightcone_cli-0.3.5/claude/lightcone/skills/narrative/references/existing-analysis.md +50 -0
  27. lightcone_cli-0.3.5/claude/lightcone/skills/narrative/references/paper-reproduction.md +118 -0
  28. lightcone_cli-0.3.5/claude/lightcone/skills/paper-extraction/SKILL.md +242 -0
  29. lightcone_cli-0.3.5/claude/lightcone/skills/paper-extraction/examples/unions-bmodes-astra.yaml +106 -0
  30. lightcone_cli-0.3.5/claude/lightcone/skills/paper-extraction/references/arxiv-source.md +47 -0
  31. lightcone_cli-0.3.5/claude/lightcone/skills/paper-extraction/references/pdf-fallback.md +66 -0
  32. lightcone_cli-0.3.5/claude/lightcone/skills/paper-extraction/scripts/extract-paper-substrate.py +1475 -0
  33. lightcone_cli-0.3.5/claude/lightcone/skills/ralph/SKILL.md +195 -0
  34. lightcone_cli-0.3.5/claude/lightcone/skills/ralph/references/constitution.md +133 -0
  35. lightcone_cli-0.3.5/claude/lightcone/skills/ralph/references/crafting.md +181 -0
  36. lightcone_cli-0.3.5/claude/lightcone/skills/ralph/scripts/ralph +145 -0
  37. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/claude/lightcone/templates/CLAUDE.md +2 -5
  38. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/pyproject.toml +16 -3
  39. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/cli/commands.py +233 -38
  40. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/engine/manifest.py +4 -0
  41. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/engine/snakefile.py +34 -0
  42. lightcone_cli-0.3.5/src/lightcone/engine/wrroc.py +885 -0
  43. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/eval/build.py +2 -2
  44. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/eval/harness.py +3 -3
  45. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/eval/sandbox.py +3 -6
  46. lightcone_cli-0.3.4/PKG-INFO +0 -19
  47. lightcone_cli-0.3.4/claude/lightcone/hooks/langfuse_git_commit_hook.py +0 -303
  48. lightcone_cli-0.3.4/claude/lightcone/hooks/langfuse_hook.py +0 -894
  49. lightcone_cli-0.3.4/claude/lightcone/hooks/langfuse_prepare_commit_msg.py +0 -142
  50. lightcone_cli-0.3.4/claude/lightcone/hooks/langfuse_session_init_hook.py +0 -83
  51. lightcone_cli-0.3.4/claude/lightcone/hooks/langfuse_utils.py +0 -457
  52. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/LICENSE +0 -0
  53. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/claude/lightcone/agents/lc-extractor.md +0 -0
  54. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/claude/lightcone/hooks.json +0 -0
  55. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/claude/lightcone/scripts/activate-venv.sh +0 -0
  56. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/claude/lightcone/scripts/validate-on-save.sh +0 -0
  57. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/cli/__init__.py +0 -0
  58. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/cli/plugin.py +0 -0
  59. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/engine/__init__.py +0 -0
  60. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/engine/container.py +0 -0
  61. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/engine/dask_cluster.py +0 -0
  62. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/engine/runner.py +0 -0
  63. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/engine/scratch.py +0 -0
  64. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/engine/site_registry.py +0 -0
  65. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/engine/status.py +0 -0
  66. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/engine/tree.py +0 -0
  67. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/engine/validation.py +0 -0
  68. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/engine/verify.py +0 -0
  69. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/eval/__init__.py +0 -0
  70. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/eval/cli.py +0 -0
  71. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/eval/graders.py +0 -0
  72. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/eval/models.py +0 -0
  73. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/lightcone/eval/report.py +0 -0
  74. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/snakemake_executor_plugin_dask/__init__.py +0 -0
  75. {lightcone_cli-0.3.4 → lightcone_cli-0.3.5}/src/snakemake_executor_plugin_dask/executor.py +0 -0
@@ -213,3 +213,12 @@ eval-results
213
213
  # UV
214
214
  ## lockfile should be versioned for applications but not for libraries
215
215
  uv.lock
216
+
217
+ # wrangler files
218
+ .wrangler
219
+ .dev.vars*
220
+ !.dev.vars.example
221
+ !.env.example
222
+
223
+ # macOS
224
+ .DS_Store
@@ -0,0 +1,71 @@
1
+ Metadata-Version: 2.4
2
+ Name: lightcone-cli
3
+ Version: 0.3.5
4
+ Summary: Lightcone command-line toolchain
5
+ Author: Lightcone Research
6
+ License: BSD-3-Clause
7
+ License-File: LICENSE
8
+ Requires-Python: >=3.11
9
+ Requires-Dist: astra-tools>=0.2.5
10
+ Requires-Dist: click>=8.0
11
+ Requires-Dist: dask>=2024.1
12
+ Requires-Dist: distributed>=2024.1
13
+ Requires-Dist: jinja2>=3.0
14
+ Requires-Dist: pyyaml>=6.0
15
+ Requires-Dist: rich>=13.0
16
+ Requires-Dist: rocrate>=0.11
17
+ Requires-Dist: snakemake-interface-common>=1.14
18
+ Requires-Dist: snakemake-interface-executor-plugins>=9.0
19
+ Requires-Dist: snakemake>=9.0
20
+ Description-Content-Type: text/markdown
21
+
22
+ # lightcone-cli
23
+
24
+ [![License](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)
25
+ [![Python](https://img.shields.io/pypi/pyversions/lightcone-cli)](https://pypi.org/project/lightcone-cli/)
26
+ [![Tests](https://github.com/LightconeResearch/lightcone-cli/actions/workflows/tests.yml/badge.svg)](https://github.com/LightconeResearch/lightcone-cli/actions/workflows/tests.yml)
27
+ [![PyPI](https://img.shields.io/pypi/v/lightcone-cli)](https://pypi.org/project/lightcone-cli/)
28
+
29
+ <!-- [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) -->
30
+
31
+ **lightcone-cli** (`lc`) is the agentic execution layer for
32
+ [ASTRA](https://astra-spec.org/latest/) (Agentic Schema for Transparent
33
+ Research Analysis). Describe your analysis to an AI agent and `lc` takes
34
+ care of the rest — specification, execution, and provenance.
35
+
36
+ ## Quick Start
37
+
38
+ ```bash
39
+ uv tool install lightcone-cli
40
+ lc init my-analysis
41
+ cd my-analysis
42
+ claude
43
+ ```
44
+
45
+ Then tell the agent what you have to start from — a research question
46
+ (`/lc-new`), existing code (`/lc-from-code`), or a paper to reproduce
47
+ (`/lc-from-paper`).
48
+
49
+ → [Full getting-started guide](https://docs.lightconeresearch.org/user/getting-started/)
50
+
51
+ ## Skills
52
+
53
+ | Skill | What it does |
54
+ |---|---|
55
+ | [`/lc-new`](https://docs.lightconeresearch.org/skills/lc-new/) | Scope a new analysis from a research question into a full `astra.yaml` spec |
56
+ | [`/lc-from-code`](https://docs.lightconeresearch.org/skills/lc-from-code/) | Bring an existing codebase into ASTRA |
57
+ | [`/lc-from-paper`](https://docs.lightconeresearch.org/skills/lc-from-paper/) | Reproduce a published paper end-to-end |
58
+ | [`/lc-feedback`](https://docs.lightconeresearch.org/skills/lc-feedback/) | File a bug report with version and error context auto-collected |
59
+
60
+ ## Capabilities
61
+
62
+ - **Multiverse analysis** — define methodological decisions with multiple options; `lc` runs your analysis across all defensible paths automatically
63
+ - **Provenance integrity** — every output gets a content-addressed manifest; `lc verify` detects tampering or broken chains
64
+ - **HPC-ready execution** — Snakemake-backed DAG dispatch with SLURM and container support (Docker, Podman, Apptainer) out of the box
65
+ - **Reproducible publishing** — `lc export wrroc` emits a [Workflow Run RO-Crate](https://www.researchobject.org/workflow-run-crate/) bundle ready for Zenodo or WorkflowHub
66
+
67
+ → [Full documentation](https://docs.lightconeresearch.org)
68
+
69
+ ## License
70
+
71
+ BSD 3-Clause — see [LICENSE](LICENSE) for details.
@@ -0,0 +1,50 @@
1
+ # lightcone-cli
2
+
3
+ [![License](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)
4
+ [![Python](https://img.shields.io/pypi/pyversions/lightcone-cli)](https://pypi.org/project/lightcone-cli/)
5
+ [![Tests](https://github.com/LightconeResearch/lightcone-cli/actions/workflows/tests.yml/badge.svg)](https://github.com/LightconeResearch/lightcone-cli/actions/workflows/tests.yml)
6
+ [![PyPI](https://img.shields.io/pypi/v/lightcone-cli)](https://pypi.org/project/lightcone-cli/)
7
+
8
+ <!-- [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) -->
9
+
10
+ **lightcone-cli** (`lc`) is the agentic execution layer for
11
+ [ASTRA](https://astra-spec.org/latest/) (Agentic Schema for Transparent
12
+ Research Analysis). Describe your analysis to an AI agent and `lc` takes
13
+ care of the rest — specification, execution, and provenance.
14
+
15
+ ## Quick Start
16
+
17
+ ```bash
18
+ uv tool install lightcone-cli
19
+ lc init my-analysis
20
+ cd my-analysis
21
+ claude
22
+ ```
23
+
24
+ Then tell the agent what you have to start from — a research question
25
+ (`/lc-new`), existing code (`/lc-from-code`), or a paper to reproduce
26
+ (`/lc-from-paper`).
27
+
28
+ → [Full getting-started guide](https://docs.lightconeresearch.org/user/getting-started/)
29
+
30
+ ## Skills
31
+
32
+ | Skill | What it does |
33
+ |---|---|
34
+ | [`/lc-new`](https://docs.lightconeresearch.org/skills/lc-new/) | Scope a new analysis from a research question into a full `astra.yaml` spec |
35
+ | [`/lc-from-code`](https://docs.lightconeresearch.org/skills/lc-from-code/) | Bring an existing codebase into ASTRA |
36
+ | [`/lc-from-paper`](https://docs.lightconeresearch.org/skills/lc-from-paper/) | Reproduce a published paper end-to-end |
37
+ | [`/lc-feedback`](https://docs.lightconeresearch.org/skills/lc-feedback/) | File a bug report with version and error context auto-collected |
38
+
39
+ ## Capabilities
40
+
41
+ - **Multiverse analysis** — define methodological decisions with multiple options; `lc` runs your analysis across all defensible paths automatically
42
+ - **Provenance integrity** — every output gets a content-addressed manifest; `lc verify` detects tampering or broken chains
43
+ - **HPC-ready execution** — Snakemake-backed DAG dispatch with SLURM and container support (Docker, Podman, Apptainer) out of the box
44
+ - **Reproducible publishing** — `lc export wrroc` emits a [Workflow Run RO-Crate](https://www.researchobject.org/workflow-run-crate/) bundle ready for Zenodo or WorkflowHub
45
+
46
+ → [Full documentation](https://docs.lightconeresearch.org)
47
+
48
+ ## License
49
+
50
+ BSD 3-Clause — see [LICENSE](LICENSE) for details.
@@ -1,11 +1,12 @@
1
1
  #!/bin/bash
2
2
  # SessionStart hook: surface a terse project status to the agent.
3
3
  #
4
- # Reports validation status, materialization counts, and pointers to the
5
- # canonical reference docs. Project name / decision count / universe count
6
- # are intentionally omitted -- they are trivia the agent reads from
7
- # astra.yaml and CLAUDE.md when needed, and they cost against the 10k
8
- # additionalContext budget.
4
+ # Reports validation status, materialization counts, and a tight CLI
5
+ # primer so the agent knows what substrate commands exist and which
6
+ # reference skills carry the depth. Project name / decision count /
7
+ # universe count are intentionally omitted -- they are trivia the agent
8
+ # reads from astra.yaml and CLAUDE.md when needed, and they cost against
9
+ # the 10k additionalContext budget.
9
10
 
10
11
  input=$(cat)
11
12
  cwd=$(echo "$input" | jq -r '.cwd // empty')
@@ -48,7 +49,13 @@ fi
48
49
  summary="$summary
49
50
  Materialization: ok=$ok_count stale=$stale_count missing=$missing_count alias=$alias_count
50
51
 
51
- References: .claude/guides/astra-reference.md (spec) and .claude/guides/lightcone-cli-reference.md (CLI)."
52
+ Substrate CLIs (use --help on any):
53
+ lc init / lc run / lc status / lc verify / lc build / lc export wrroc
54
+ astra validate / astra paper add / astra universe generate
55
+
56
+ Reference skills (invoke when the surface above isn't enough):
57
+ /astra — astra.yaml spec: decisions, prior_insights, findings, evidence, sub-analyses, narrative anchors
58
+ /lc-cli — lc workflow: spec-code invariant, status interpretation, failure diagnosis"
52
59
 
53
60
  if [ "$validation_ok" -ne 0 ]; then
54
61
  # tail rather than head -- the leading lines are success markers
@@ -0,0 +1,43 @@
1
+ # lightcone-cli skills
2
+
3
+ Each subdirectory is one Claude Code skill: `SKILL.md` plus optional `references/`, `assets/`, and `scripts/`. `lc init` copies these into a project's `.claude/skills/` so they are discoverable to Claude Code sessions.
4
+
5
+ ## Project lifecycle skills
6
+
7
+ | Skill | Role |
8
+ |---|---|
9
+ | `lc-new` | Scaffold a new ASTRA-shaped project from a research question. |
10
+ | `lc-from-code` | Bring an existing codebase into ASTRA — scan, spec, parameterize. |
11
+ | `lc-from-paper` | Reproduce a published paper in ASTRA (paper-reproduction bundle entry point — see below). |
12
+ | `lc-feedback` | Report bugs and feature requests upstream. |
13
+ | `ralph` | Author a constitution and run a ralph loop against it (authoring + launching + iterating in one skill). `lc-from-paper` uses this for the long middle of a reproduction; standalone for any other long-running work. |
14
+
15
+ ## Reference skills
16
+
17
+ Not direct entry points — Claude invokes these (or other skills invoke them) to load reference content into the session. The session-start hook primes their names so they're discoverable from turn one.
18
+
19
+ | Skill | Role |
20
+ |---|---|
21
+ | `astra` | Reference for the `astra.yaml` spec: structure, decisions, options, prior insights, findings, evidence, sub-analyses, narrative anchors, composition mechanics. |
22
+ | `lc-cli` | Reference for `lc` workflow: commands, the Spec-Code Invariant, status interpretation, failure diagnosis, multiverse runs, WRROC export. |
23
+
24
+ ## Paper-reproduction bundle
25
+
26
+ A self-contained toolkit for reproducing published papers in ASTRA. The bundle is co-located so a single `lc init` brings the full toolkit into a project — no plugin marketplace, no separate installs.
27
+
28
+ | Skill | Role |
29
+ |---|---|
30
+ | [`lc-from-paper`](lc-from-paper/SKILL.md) | **Reproduction driver.** ORIENT-first; one pre-loop phase in the user's main session that asks for the paper, runs `/paper-extraction` inline, interviews the user (grounded in the paper), clones the reference code and runs `/lc-from-code` scan-only (when a repo exists), and drafts the per-paper `constitution.md` + `CLAUDE.md`. Then hands off to a ralph loop whose iterations carry the long middle: ARCHITECT → SPECIFY → LITERATURE → IMPLEMENT → RUN → COMPARE. When the loop closes (constitution `status: closed` after COMPARE returns `pass`), REVIEW runs back in the user's main session. Fidelity intent — captured as prose at ORIENT — is what every iteration reads when sizing its next move, and what COMPARE grades opportunities against. |
31
+ | [`ralph`](ralph/SKILL.md) | The loop substrate. `lc-from-paper`'s ORIENT invokes `/ralph`'s Authoring mode to draft the per-paper constitution; the loop launcher hands off after ORIENT lands. Each iteration runs `/ralph`'s Loop protocol against the constitution. |
32
+ | [`narrative`](narrative/SKILL.md) | Author the `narrative:` prose and decision `rationale:` in `astra.yaml`. Invoked by `lc-from-paper`'s ARCHITECT (for the structural narrative) and SPECIFY (for anchored content narrative). |
33
+ | [`paper-extraction`](paper-extraction/SKILL.md) | Turn an arXiv ID or DOI into a standardized `work/reference/` directory: structural index (figures, tables, outline, citations with resolved DOIs) plus a stub `astra.yaml` for the paper. Primary acquisition path for `lc-from-paper`'s ORIENT (Stage 2); also invoked per cited paper by LITERATURE. |
34
+ | [`check-sentence-by-sentence`](check-sentence-by-sentence/SKILL.md) | Audit paper claims against code locations (`file:line` or `NOT FOUND`). Invoked from `lc-from-paper`'s REVIEW close-out (opt-in); also user-invokable directly. |
35
+ | [`figure-comparison`](figure-comparison/SKILL.md) | Build a self-contained HTML side-by-side: original figures/tables/numerics vs replicated. Invoked from `lc-from-paper`'s REVIEW close-out (mandatory); also user-invokable directly. |
36
+
37
+ The full reproduction story spans these skills. `lc-from-paper`'s `SKILL.md` names each by role and tells the agent when to invoke them; the siblings stand alone and don't know about `lc-from-paper`.
38
+
39
+ ### Why bundle (not depend on plugin install)
40
+
41
+ - **Testability.** We want to verify `lc-from-paper` invokes its sibling skills correctly. That only works when all are in the same checkout.
42
+ - **Single install path.** `lc init` brings the full toolkit. Adding a separate plugin-marketplace step is friction we don't need.
43
+ - **Future consolidation is open.** The long-run shape may be `astra` ships skills in `astra`, `lc` ships skills in `lightcone-cli`, plus a centralized external-skills list. Today: bundle it all. See [[lightcone/skills-location-policy]].
@@ -1,3 +1,17 @@
1
+ ---
2
+ name: astra
3
+ description: >
4
+ Comprehensive reference for the `astra.yaml` specification — top-level
5
+ structure, sub-analyses, inputs/outputs, decisions and options, prior
6
+ insights and findings, evidence and quote verification, narrative
7
+ anchors, and composition mechanics. Invoke whenever reading, writing,
8
+ validating, or debugging an `astra.yaml` spec; whenever working with
9
+ decisions, options, prior_insights, findings, or evidence; or whenever
10
+ the user asks about ASTRA schema, spec syntax, or sub-analysis
11
+ composition.
12
+ allowed-tools: Read, Glob, Grep, Bash(astra:*)
13
+ ---
14
+
1
15
  # ASTRA Reference
2
16
 
3
17
  ## What an ASTRA Analysis Is
@@ -100,6 +114,17 @@ A decision is a methodological choice where a different defensible option could
100
114
 
101
115
  Decisions may carry an optional `tags:` list for grouping (e.g. `[preprocessing]`, `[physics]`, `[stats]`). Keep the tag vocabulary **small and consolidated** -- reuse existing tags rather than minting new ones, since tags are mostly useful for cross-cutting views over a shared decision space, and that view fragments quickly when every decision invents its own label.
102
116
 
117
+ ### Options
118
+
119
+ Each decision must have at least one option. Options are `key: { ... }` entries:
120
+
121
+ - `label:` (required) -- short human-readable name for compact rendering.
122
+ - `description:` (optional) -- longer prose explaining what the option means.
123
+ - `insights:` (optional) -- list of `prior_insights:` IDs that justify this option; back-references the supporting evidence (see [Prior Insights and Findings](#prior-insights-and-findings)).
124
+ - `excluded:` + `excluded_reason:` -- option considered but rejected. See [Constraints](#constraints).
125
+
126
+ `label:` and `options:` are required on the decision itself. An aliased decision (one that points at another via `from: ../decisions.foo` -- see [Composition Mechanics](#composition-mechanics)) inherits both from its source and doesn't redeclare them.
127
+
103
128
  ### Parameterization
104
129
 
105
130
  **Every decision must be parameterized in code** -- never hardcode a decision value. The recipe's `command:` template references it via `{decisions.<id>}` (see [Command Template Substitution](#command-template-substitution)).
@@ -137,6 +162,8 @@ Set `container:` at analysis level (all recipes inherit); per-recipe `container:
137
162
 
138
163
  Runners expand `{...}` placeholders in `command:` before invoking it: `{inputs.<id>}` (input path), `{inputs}` (all input paths, declared order), `{decisions.<id>}` (active option ID), `{output}` (artifact path), `{{`/`}}` (literal braces). Every `{inputs.<id>}` and `{decisions.<id>}` must name something declared in the parent Output's `inputs:`/`decisions:` lists -- always **local IDs** (no `../`; bridging is declared once at the Input/Decision via `from:`).
139
164
 
165
+ Text outside `{...}` is literal command text and isn't validated. Static constants (`--max-iter 1000`), per-output specialisations when fan-out is unrolled into one Output per value (`--tracer lrg1`), and shell features (`${VAR}`, pipes, redirects) all live as plain text. Only values that vary across the multiverse need to be `{decisions.<id>}` placeholders -- there is no separate `params` channel, and Snakemake-style wildcards (`{chunk_id}`, `{block_i}`) have no spec-level analogue: either inline the value, unroll the fan-out into one Output per value, or describe only the aggregated artifact.
166
+
140
167
  ### Conditional Outputs
141
168
 
142
169
  Outputs can have `when` conditions -- the output only exists when the condition is met for a given universe. Uses the same syntax as decision `when` (negation with `~`, lists AND'd).
@@ -171,7 +198,7 @@ Two kinds of insight, distinguished by direction:
171
198
  - **Prior insights** (`prior_insights:`) — knowledge from outside the analysis that informs decisions. From literature (by DOI) or artifacts from a prior/parent analysis.
172
199
  - **Findings** (`findings:`) — conclusions from the analysis itself, backed by its own output artifacts.
173
200
 
174
- Both use the same Insight model: `id`, `label` (optional), `claim`, `created_at`, `evidence`, plus optional `derived` (true if synthesized/inferred from multiple sources), `scope` (applicability conditions), `tags`, `notes`. Placement determines direction.
201
+ Both use the same Insight model. Required: `id`, `claim`, `created_at` (ISO 8601 datetime — e.g. `"2025-02-01T14:00:00"`), `evidence`. Optional: `label`, `derived` (true if synthesized/inferred from multiple sources), `scope` (applicability conditions), `tags`, `notes`. Placement determines direction.
175
202
 
176
203
  Each evidence item has its own fields: `id`, exactly one of `doi` (literature) or `artifact` (output ID), and either a `quote` (TextQuoteSelector with required `exact`, optional `prefix`/`suffix`) or `location` (FragmentSelector with `value` like `"page=6"` and/or 1-indexed `page`). DOI evidence may add `version` (arXiv version). Artifact evidence may add `snapshot` (path to an immutable artifact copy) and `source_commit` (git commit that produced it).
177
204
 
@@ -0,0 +1,369 @@
1
+ ---
2
+ name: check-sentence-by-sentence
3
+ description: >
4
+ Sentence-by-sentence audit of a paper against an ASTRA project's code. For
5
+ every claim about implementation or results in the methodology, results,
6
+ discussion, and appendices, locate the corresponding code (file:line) or
7
+ mark NOT FOUND. Only the user can invoke this skill, though this skill can be suggested for the user to invoke during paper reproduction. Other skills may mention this skill as an optional follow-up, but should not invoke it themselves. Run from the project folder containing astra.yaml. In lc-from-paper projects, read paper sources from
8
+ work/reference/: prefer arXiv TeX under work/reference/source/, fall back to
9
+ Docling/Pandoc markdown at work/reference/document.md.
10
+ argument-hint: "[path to paper source, e.g. work/reference/source/main.tex or work/reference/document.md]"
11
+ ---
12
+
13
+ # /check-sentence-by-sentence
14
+
15
+ Audit a paper against the code in this ASTRA project, sentence by sentence.
16
+ Every sentence that asserts an implementation detail or a numerical/empirical
17
+ result is located in the code (`file:line`) or marked NOT FOUND. The agent
18
+ does NOT run any code -- this is a static reading audit.
19
+
20
+ In lc-from-paper projects, the paper substrate comes from `work/reference/`.
21
+ Path A is arXiv source at `work/reference/source/`; Path B is the parsed
22
+ markdown fallback at `work/reference/document.md`, produced by Docling or
23
+ Pandoc.
24
+
25
+ ## Setup
26
+
27
+ 1. **Confirm project root.** Read `astra.yaml` in the current working
28
+ directory. If it is missing, ask the user:
29
+
30
+ > "I do not see an `astra.yaml` in the current directory. Please point me
31
+ > to the ASTRA project folder, or `cd` there and re-invoke."
32
+
33
+ Stop until resolved.
34
+
35
+ 2. **Confirm paper source.** The user may have passed a path as an
36
+ argument. Resolve it in this order:
37
+
38
+ 1. If the argument is a `.tex` file, use it in `tex` mode.
39
+ 2. If the argument is `work/reference/` or another directory, first look
40
+ for TeX source under `<dir>/source/`, then for `<dir>/document.md`.
41
+ 3. If no argument was supplied, prefer the lc-from-paper layout:
42
+ - `work/reference/source/<main>.tex` if TeX source exists. Identify the
43
+ main file with `grep -l '\\documentclass' work/reference/source/*.tex`;
44
+ if exactly one file matches, use it. If multiple files match, ask the
45
+ user which one is the main paper file. After identifying the main
46
+ file, expand its local `\input{...}` and `\include{...}` files before
47
+ section enumeration; many arXiv papers keep most prose outside the
48
+ main TeX wrapper.
49
+ - `work/reference/document.md` if there is no TeX source. This is the
50
+ Docling/Pandoc fallback and should be audited in `markdown` mode.
51
+ 4. Only after those lc-from-paper paths fail, look for an obvious legacy
52
+ `.tex` source in cwd: a top-level `*.tex`, or one inside `paper/`,
53
+ `tex/`, or a similarly named subdirectory. If exactly one obvious
54
+ candidate is found, use it in `tex` mode.
55
+
56
+ If no usable source is found, ask:
57
+
58
+ > "Which paper source should I audit? Please give me a `.tex` path or
59
+ > `work/reference/document.md`."
60
+
61
+ If only `work/reference/paper.pdf` exists, ask the user to run the PARSE
62
+ phase first so `work/reference/document.md` exists. Do not audit PDFs
63
+ directly.
64
+
65
+ ## Section enumeration
66
+
67
+ This is **your job in the main agent** -- do it carefully so each subagent
68
+ gets a precise line range. Do NOT read full section content; only enough to
69
+ identify boundaries.
70
+
71
+ 1. Enumerate sections according to source mode:
72
+ - In `tex` mode, first build the ordered audit source list. Start with the
73
+ main TeX file, scan it for local `\input{...}` and `\include{...}` paths,
74
+ normalize missing `.tex` suffixes, and include those files when they
75
+ exist under the same source tree. Recurse one level deeper when an
76
+ included file itself includes local TeX files. Ignore package/style
77
+ imports (`\usepackage`, `.sty`, `.cls`) and remote/generated files. If
78
+ the main file is mostly a wrapper, the leaf included files will carry
79
+ most audit units.
80
+ - For every file in the TeX audit source list, use `grep -n` for
81
+ `^\\section`, `^\\subsection`, and `^\\appendix`. Record each match's
82
+ file path, line number, and label.
83
+ - In `markdown` mode, use `grep -n` for markdown headings
84
+ (`^#`, `^##`, `^###`, etc.) in `work/reference/document.md`. Treat
85
+ heading depth the way TeX treats section/subsection. If Docling emitted
86
+ unnumbered headings, use their text labels.
87
+ 2. Get the file's total line count with `wc -l`.
88
+ 3. Compute each section's line range: **start = the section's own line
89
+ number; end = (next section/subsection or same/lower heading-depth start
90
+ minus 1 in the same source file), or that source file's last line for the
91
+ final section in that file.** For a section that contains subsections,
92
+ each subsection's range runs from its own line to (next subsection
93
+ start − 1), and the section's pre-subsection prose (if any) becomes its
94
+ own audit unit covering (section line + 1) to (first subsection − 1) if
95
+ that span is non-trivial.
96
+ 4. Mark sections appearing after `\appendix` (TeX) or after an `Appendix` /
97
+ `Appendices` heading (markdown) as appendices regardless of label.
98
+
99
+ Identify the audit-relevant sections:
100
+
101
+ - Methodology (often `Methods`, `Analysis`, `Data`, `Sample selection`)
102
+ - Results
103
+ - Discussion (often `Discussion and Conclusions`)
104
+ - Appendices (every section after `\appendix`)
105
+
106
+ Skip Abstract, Introduction, Acknowledgements, References, author lists.
107
+
108
+ For each retained section, check whether it has subsections. **Spin up one
109
+ subagent per leaf (sub)section** -- a section with subsections becomes one
110
+ subagent per subsection (plus optionally one for any pre-subsection prose
111
+ span); a section without subsections becomes one subagent for the whole
112
+ section. Spawn them all in a single message so they run in parallel.
113
+
114
+ ## Subagent prompt
115
+
116
+ Use `Agent(subagent_type="general-purpose", ...)`. Pass each subagent:
117
+
118
+ - The absolute path to the paper source file for this section
119
+ - The paper source mode: `tex` or `markdown`
120
+ - The exact section/subsection label and the line range in the source file
121
+ it covers (so it knows where to read)
122
+ - The absolute path to the project root (which contains `astra.yaml`)
123
+ - The instructions below, verbatim
124
+
125
+ ```
126
+ You are auditing one (sub)section of a paper against an ASTRA project's
127
+ code. Your job is mechanical and exhaustive.
128
+
129
+ INPUTS
130
+ - Paper source file: <path>
131
+ - Source mode: <tex|markdown>
132
+ - Section: <name>, lines <start>-<end>
133
+ - Project root: <path>
134
+
135
+ PROCEDURE
136
+ 1. Read the assigned section of the paper. Split it into sentences using
137
+ common sense, not naive period-splitting. In `tex` mode, use TeX-aware
138
+ splitting; in `markdown` mode, preserve Docling/Pandoc math blocks,
139
+ captions, and headings as source text. Treat `e.g.`, `i.e.`, `et al.`,
140
+ `Fig.`, `Eq.`, `Sec.`, `Dr.`, decimals (`0.5`), inline math `$...$`,
141
+ and citation commands (`\citep{...}`, `\citet{...}`) as part of the
142
+ surrounding sentence, not boundaries. Display equations belong to
143
+ whichever sentence introduces them.
144
+ 2. For each sentence, decide using common sense: does it make a concrete
145
+ claim about an IMPLEMENTATION DETAIL (a method, parameter, threshold,
146
+ formula, data cut, model choice, sample definition, algorithmic step)
147
+ or a RESULTS DETAIL (a numerical value, plot, fitted parameter,
148
+ statistical outcome)? If neither -- pure motivation, citation prose,
149
+ or generic framing -- skip it.
150
+ 3. Before searching, **read `astra.yaml` once** -- it is a pre-built
151
+ paper↔code map maintained by the project. Harvest specifically:
152
+ - `narrative.methods` — links paper methodology concepts to decision
153
+ IDs (e.g. paper prose "the chosen <method>" → `#decisions.<id>`)
154
+ - `narrative.findings` — links paper claims/values to result anchors
155
+ - `prior_insights` (if present) — extracted paper quotes already tied
156
+ to decisions
157
+ - per-decision `evidence` quotes and `description` fields
158
+ Treat these as your translation table: paper prose → decision/output
159
+ IDs → script files. Do not re-derive what the spec already encodes.
160
+
161
+ For everything not covered by the spec, use common sense to translate
162
+ concepts. In general:
163
+ - A quality cut stated as a ratio or threshold may appear in code
164
+ under an inverted form or a different variable name -- map by
165
+ meaning, not by symbol.
166
+ - A named model or distribution will usually appear as a function
167
+ whose name describes its shape or role, not as the paper's prose
168
+ phrasing.
169
+ - A cited constant from a referenced paper will usually appear as a
170
+ module-level constant or as an option value in a decision.
171
+ Grep for the underlying concept, not just the paper's wording.
172
+ 4. For every claim-bearing sentence, search the project code (`scripts/`,
173
+ source files, `universes/`, `astra.yaml`, `results/`) for where the
174
+ claim is implemented or computed. Use Grep, Glob, and Read.
175
+ 5. Record one of:
176
+ - (quote, path/file.py:LINE, optional <10-word note)
177
+ when the sentence's claim is implemented or computed at that location
178
+ - (quote, NOT FOUND, optional <10-word note)
179
+ when no implementation or matching computation is present
180
+
181
+ CONSTRAINTS
182
+ - Do NOT run any code. No Bash beyond ls/grep/find/wc for searching.
183
+ - Do NOT read the paper outside the assigned line range.
184
+ - Quote the sentence verbatim, trimmed to a single sentence. If the
185
+ sentence is long, you may include just the claim-bearing clause but
186
+ preserve enough text to identify it.
187
+ - file:line should point to the most specific line that implements or
188
+ states the claim (the function call, parameter assignment, or computed
189
+ value -- not just the file).
190
+ - Notes must be under 10 words. Use them for nuance like "approximate
191
+ match", "different constant", "implemented but commented out",
192
+ "value computed at runtime, not statically comparable", "produced as
193
+ figure but printed value not stored".
194
+ - For numerical results that the paper states as a final number, point
195
+ at the line that computes the value and use a note like "value
196
+ computed at runtime" -- you cannot verify numerical agreement without
197
+ executing code, and that is fine.
198
+
199
+ OUTPUT
200
+ Return a JSON-ish list, one entry per sentence, in paper order:
201
+
202
+ [
203
+ {"quote": "...", "location": "scripts/foo.py:142", "note": "..."},
204
+ {"quote": "...", "location": "NOT FOUND", "note": "..."},
205
+ ...
206
+ ]
207
+
208
+ Return nothing else.
209
+ ```
210
+
211
+ ## Aggregation
212
+
213
+ When all subagents return, you receive raw entries from every claim-bearing
214
+ sentence each subagent kept. **Do not just concatenate and print them.**
215
+ Two filtering passes happen here, in this order:
216
+
217
+ ### Pass 1 — drop non-computational sentences
218
+
219
+ Subagents are deliberately generous about what they keep, so the raw list
220
+ contains a long tail of sentences that quote the paper but do not actually
221
+ correspond to anything you would expect to find in code. **Drop any entry
222
+ whose sentence is:**
223
+
224
+ - **Framing / motivation** — sentences whose job is to set up the next
225
+ step, e.g. "the first step is...", "to investigate this...", "we want
226
+ to look at...", "for this reason..."
227
+ - **Citation prose / literature comparison** — sentences that compare to
228
+ or quote prior literature, e.g. "agrees with values typical of previous
229
+ measurements...", "much like Author+YYYY they show...", "in particular,
230
+ Author found <value>..."
231
+ - **Theoretical framing or derivations** — sentences asserting a property
232
+ expected from theory rather than implemented in code, and restatements
233
+ of textbook identities used only to introduce the next equation
234
+ - **Rhetorical / interpretive claims** — qualitative readings of a
235
+ figure or trend, e.g. "the trend clearly has an oscillatory
236
+ behaviour", "the trend seems to be independent of <variable>", "this
237
+ supports that..."
238
+ - **Conclusions / justifications / qualitative observations** —
239
+ "thus we conclude that...", "we choose not to include this
240
+ because...", "by and large the trends are similar"
241
+ - **Future work / speculation** — "this could be improved by...", "the
242
+ discrepancy could be explained by..."
243
+ - **Forward/backward references with no claim** — "we discuss this in
244
+ Sec X below", "as described in Sec Y above"
245
+ - **NOT FOUND entries that fall in any of the above categories** — most
246
+ framing/motivation sentences will land as NOT FOUND because there is
247
+ nothing to find. Drop them silently; they are noise, not gaps.
248
+
249
+ Keep an entry only if it asserts something a reader would expect to be
250
+ implemented or computed: a parameter value, a cut, a formula, an
251
+ algorithmic step, a fitted/measured value, a figure that the project
252
+ should produce, a sample size after a specific cut.
253
+
254
+ When in doubt about a NOT FOUND, ask: "if this sentence is not in the
255
+ code, is that a real gap?" If no, drop it.
256
+
257
+ ### Pass 2 — deduplicate / merge near-duplicates
258
+
259
+ Subagents do not see each other, and the same claim is often restated
260
+ across sentences within a (sub)section -- e.g. a prose statement of a
261
+ cut followed by a sentence asserting "this is the only cut we make", or
262
+ two sub-equations of one larger formula that map to the same line.
263
+ Collapse these:
264
+
265
+ - If two adjacent sentences make the same claim and resolve to the same
266
+ `file:line`, keep one entry whose quote is the more specific or
267
+ formula-bearing of the two, and append the other in a short
268
+ parenthetical only if it adds information.
269
+ - If a paper-text claim and an explicit equation/quoted code map to the
270
+ same line, prefer the equation/quoted-code form.
271
+ - Do not merge across (sub)sections.
272
+ - Do not merge if the two sentences resolve to different `file:line`
273
+ locations -- they may look similar but are doing different things.
274
+
275
+ ### Pass 3 — render
276
+
277
+ After filtering and deduplication, present the result to the user as
278
+ markdown, organized by section -> subsection -> sentence, in paper order:
279
+
280
+ ```
281
+ # Sentence-by-sentence reproduction audit
282
+
283
+ Paper: <path>
284
+ Project: <path>
285
+
286
+ ## <Section>
287
+
288
+ ### <Subsection> (omit if no subsections)
289
+
290
+ - "<sentence quote>"
291
+ → ✅ `scripts/foo.py:142` -- <note if any>
292
+
293
+ - "<sentence quote>"
294
+ → ❌ NOT FOUND -- <note if any>
295
+ - ...
296
+ ```
297
+
298
+ Use `→ ✅ \`file:line\`` for found entries and `→ ❌ NOT FOUND` for
299
+ missing ones. Notes are optional; only include the trailing `-- <note>`
300
+ when the subagent supplied one.
301
+
302
+ End with a one-line summary:
303
+
304
+ > N sentences audited across M sections. K implemented, J not found.
305
+
306
+ ### Follow-up suggestion (conditional)
307
+
308
+ After the summary, scan the NOT FOUND entries and **cluster them**. A
309
+ cluster is a group of NOT FOUND sentences that all relate to the same
310
+ missing piece of work (a missing analysis, a missing diagnostic, an
311
+ unimplemented model variant) -- usually a few consecutive sentences in
312
+ one (sub)section, or sentences that all reference the same concept across
313
+ sections.
314
+
315
+ **Only emit the follow-up block if there is at least one major
316
+ unimplemented cluster** -- a cluster of genuine missing computation
317
+ substantial enough to be worth offering to add (rule of thumb: ≥3
318
+ sentences of related missing-computation claims, or a single
319
+ heavyweight missing artifact like an entire missing analysis or
320
+ figure). If every NOT FOUND is isolated framing, motivation, or
321
+ qualitative interpretation -- or if the only clusters are tiny -- stop
322
+ after the one-line summary. Do not pad with a follow-up just to have
323
+ one.
324
+
325
+ When the threshold is met, write a short follow-up block in this shape:
326
+
327
+ > Major unimplemented clusters: (1) `<short description of cluster 1>`
328
+ > (`<§section>`, ~`<N>` sentences), and (2) `<short description of
329
+ > cluster 2>` (`<§section>`, ~`<N>` sentences). The rest of the NOT
330
+ > FOUND entries are pure framing/motivation/qualitative interpretation,
331
+ > not computational claims. Worth considering as a follow-up if you
332
+ > want full coverage — want me to add `<concrete artifact 1>` and
333
+ > `<concrete artifact 2>`?
334
+
335
+ Rules for this block:
336
+ - Only call out clusters that look like genuine missing computation, not
337
+ rhetoric.
338
+ - Keep it to 1–3 clusters. Do not enumerate every NOT FOUND entry.
339
+ - The closing offer must name **concrete artifacts** the user could add
340
+ (a new output ID, a new script filename, a new decision option, a new
341
+ figure) -- not vague promises like "fill in the gaps".
342
+ - Cite the section reference in the project's own notation (`§2.1`,
343
+ `Appendix B`, etc.) and an approximate sentence count.
344
+ - One short paragraph; do not pad.
345
+
346
+ ## Restrictions
347
+
348
+ - You MUST NOT run project code, recipes, or `lc run`. This is static.
349
+ - You MUST NOT read the paper source wholesale into the main context;
350
+ delegate to subagents.
351
+ - You MUST NOT modify any project file. Read-only.
352
+ - You MUST NOT fabricate `file:line` locations -- if a subagent's location
353
+ looks suspicious, ask it to re-verify rather than guessing.
354
+ - You MUST spawn one subagent per leaf (sub)section, in parallel.
355
+
356
+ ## Anti-patterns
357
+
358
+ - **Auditing intro/abstract** -- skip narrative-only sections; only
359
+ methodology, results, discussion, and appendices.
360
+ - **Bundling sentences** -- one entry per sentence. Do not collapse
361
+ multiple claims into one row even if they share a citation or location.
362
+ - **Vague locations** -- a bare filename (`scripts/foo.py`) is not
363
+ enough; a line number is required for found entries.
364
+ - **Long notes** -- the 10-word cap is a hard limit; reserve notes for
365
+ signal, not commentary.
366
+ - **Running code to verify** -- this skill is a reading audit. If a claim
367
+ cannot be verified by reading code alone, mark it found at the
368
+ computing line and note "value computed at runtime" rather than
369
+ executing anything.