litplan 0.0.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (204) hide show
  1. litplan-0.0.1/.dockerignore +9 -0
  2. litplan-0.0.1/.env.example +71 -0
  3. litplan-0.0.1/.github/workflows/ci.yml +84 -0
  4. litplan-0.0.1/.github/workflows/publish-pypi.yml +39 -0
  5. litplan-0.0.1/.github/workflows/workflowgen-engines.yml +100 -0
  6. litplan-0.0.1/.gitignore +11 -0
  7. litplan-0.0.1/Dockerfile +23 -0
  8. litplan-0.0.1/LICENSE +21 -0
  9. litplan-0.0.1/PKG-INFO +264 -0
  10. litplan-0.0.1/README.md +215 -0
  11. litplan-0.0.1/alembic/env.py +69 -0
  12. litplan-0.0.1/alembic/script.py.mako +26 -0
  13. litplan-0.0.1/alembic/versions/5899d8e46cfa_t3_3_pipeline_spec_revisions_and_runs.py +73 -0
  14. litplan-0.0.1/alembic/versions/c7f2a1b9e4d0_t6_3_pipeline_run_node_checkpoints.py +40 -0
  15. litplan-0.0.1/alembic/versions/d8e3f0a1b2c4_t10_2_plan_approval_records.py +50 -0
  16. litplan-0.0.1/alembic/versions/e9a1c2d3f4b5_t10_4_eval_provenance_annotations.py +48 -0
  17. litplan-0.0.1/alembic.ini +45 -0
  18. litplan-0.0.1/docker-compose.grobid.yml +12 -0
  19. litplan-0.0.1/docs/CAMELINE_REQUEST_FLOW_DEVELOPER_GUIDE.md +553 -0
  20. litplan-0.0.1/docs/CONTEXT_POISONING_POLICY.md +15 -0
  21. litplan-0.0.1/docs/DEVS_README.md +323 -0
  22. litplan-0.0.1/docs/PRODUCT_SCENARIOS.md +277 -0
  23. litplan-0.0.1/docs/adr/0002-librarian-chunk-index.md +30 -0
  24. litplan-0.0.1/docs/adr/0003-grobid-only-pdf-ingestion.md +46 -0
  25. litplan-0.0.1/docs/adr/0004-mcp-vs-cli.md +93 -0
  26. litplan-0.0.1/docs/assets/litplan_streamlit_ui.png +0 -0
  27. litplan-0.0.1/docs/manual/MANUAL_CLI_MCP_UI_WORKFLOW.md +459 -0
  28. litplan-0.0.1/docs/workflowgen/ENVIRONMENT_FINGERPRINT.md +96 -0
  29. litplan-0.0.1/docs/workflowgen/GENERATION_POLICY.md +101 -0
  30. litplan-0.0.1/docs/workflowgen/IDEMPOTENCY_AND_RESUME.md +99 -0
  31. litplan-0.0.1/docs/workflowgen/PATH_MODEL.md +124 -0
  32. litplan-0.0.1/docs/workflowgen/SECURITY_REVIEW_CHECKLIST.md +60 -0
  33. litplan-0.0.1/docs/workflowgen/WORKFLOW_EXPORT.md +103 -0
  34. litplan-0.0.1/evals/.gitkeep +0 -0
  35. litplan-0.0.1/evals/t10_experiment_scenario_demo.py +308 -0
  36. litplan-0.0.1/evals/t8_tool_use/dataset.json +114 -0
  37. litplan-0.0.1/evals/t8_tool_use/score_tool_use_eval.py +368 -0
  38. litplan-0.0.1/examples/paper_repro_sketch_pdf.py +210 -0
  39. litplan-0.0.1/examples/paper_repro_sketch_tei.py +200 -0
  40. litplan-0.0.1/fixtures/experiments/synth_dft_bulk_si/expected/recovery_targets.yaml +40 -0
  41. litplan-0.0.1/fixtures/experiments/synth_dft_bulk_si/synth_dft_bulk_si.tei.xml +47 -0
  42. litplan-0.0.1/fixtures/papers/FIXTURES.md +31 -0
  43. litplan-0.0.1/fixtures/papers/context_poisoning_minimal.tei.xml +15 -0
  44. litplan-0.0.1/fixtures/papers/pdf/batatia-2022-mace-force-fields-arxiv-2206.07697v2.pdf +0 -0
  45. litplan-0.0.1/fixtures/papers/pdf/yu-2024-systematic-assessment-universal-mlip-arxiv-2403.05729v3.pdf +0 -0
  46. litplan-0.0.1/fixtures/tei/README.md +42 -0
  47. litplan-0.0.1/fixtures/tei/structural_minimal.tei.xml +45 -0
  48. litplan-0.0.1/fixtures/workflowgen/airflow_golden/airflow_parameters_quoting.litplan_dag.py +35 -0
  49. litplan-0.0.1/fixtures/workflowgen/airflow_golden/diamond.litplan_dag.py +78 -0
  50. litplan-0.0.1/fixtures/workflowgen/airflow_golden/edges_unsorted_in_file.litplan_dag.py +63 -0
  51. litplan-0.0.1/fixtures/workflowgen/airflow_golden/empty.litplan_dag.py +33 -0
  52. litplan-0.0.1/fixtures/workflowgen/airflow_golden/linear_chain.litplan_dag.py +63 -0
  53. litplan-0.0.1/fixtures/workflowgen/airflow_golden/parallel_roots.litplan_dag.py +63 -0
  54. litplan-0.0.1/fixtures/workflowgen/environment_sidecars/apptainer.stub.def +7 -0
  55. litplan-0.0.1/fixtures/workflowgen/environment_sidecars/conda.env.stub.yml +8 -0
  56. litplan-0.0.1/fixtures/workflowgen/environment_sidecars/nextflow.env_hints.stub.config +4 -0
  57. litplan-0.0.1/fixtures/workflowgen/kind_maps/artifact_fork_selective.yaml +19 -0
  58. litplan-0.0.1/fixtures/workflowgen/nextflow_golden/diamond.main.nf +86 -0
  59. litplan-0.0.1/fixtures/workflowgen/nextflow_golden/diamond.nextflow.config +12 -0
  60. litplan-0.0.1/fixtures/workflowgen/nextflow_golden/edges_unsorted_in_file.main.nf +69 -0
  61. litplan-0.0.1/fixtures/workflowgen/nextflow_golden/edges_unsorted_in_file.nextflow.config +12 -0
  62. litplan-0.0.1/fixtures/workflowgen/nextflow_golden/empty.main.nf +32 -0
  63. litplan-0.0.1/fixtures/workflowgen/nextflow_golden/empty.nextflow.config +12 -0
  64. litplan-0.0.1/fixtures/workflowgen/nextflow_golden/linear_chain.main.nf +69 -0
  65. litplan-0.0.1/fixtures/workflowgen/nextflow_golden/linear_chain.nextflow.config +12 -0
  66. litplan-0.0.1/fixtures/workflowgen/nextflow_golden/parallel_roots.main.nf +69 -0
  67. litplan-0.0.1/fixtures/workflowgen/nextflow_golden/parallel_roots.nextflow.config +12 -0
  68. litplan-0.0.1/fixtures/workflowgen/pipeline_specs/airflow_parameters_quoting.json +23 -0
  69. litplan-0.0.1/fixtures/workflowgen/pipeline_specs/airflow_write_missing_content.json +14 -0
  70. litplan-0.0.1/fixtures/workflowgen/pipeline_specs/artifact_chain_linear.json +26 -0
  71. litplan-0.0.1/fixtures/workflowgen/pipeline_specs/artifact_fork_selective.json +24 -0
  72. litplan-0.0.1/fixtures/workflowgen/pipeline_specs/artifact_merge_diamond.json +24 -0
  73. litplan-0.0.1/fixtures/workflowgen/pipeline_specs/diamond.json +15 -0
  74. litplan-0.0.1/fixtures/workflowgen/pipeline_specs/edges_unsorted_in_file.json +12 -0
  75. litplan-0.0.1/fixtures/workflowgen/pipeline_specs/empty.json +5 -0
  76. litplan-0.0.1/fixtures/workflowgen/pipeline_specs/linear_chain.json +12 -0
  77. litplan-0.0.1/fixtures/workflowgen/pipeline_specs/parallel_roots.json +12 -0
  78. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/artifact_chain_linear.Snakefile +61 -0
  79. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/artifact_chain_linear.config.yaml +4 -0
  80. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/artifact_fork_selective.Snakefile +43 -0
  81. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/artifact_fork_selective.config.yaml +4 -0
  82. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/artifact_merge_diamond.Snakefile +43 -0
  83. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/artifact_merge_diamond.config.yaml +4 -0
  84. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/diamond.Snakefile +35 -0
  85. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/diamond.config.yaml +4 -0
  86. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/edges_unsorted_in_file.Snakefile +27 -0
  87. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/edges_unsorted_in_file.config.yaml +4 -0
  88. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/empty.Snakefile +10 -0
  89. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/empty.config.yaml +4 -0
  90. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/linear_chain.Snakefile +27 -0
  91. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/linear_chain.config.yaml +4 -0
  92. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/parallel_roots.Snakefile +27 -0
  93. litplan-0.0.1/fixtures/workflowgen/snakemake_golden/parallel_roots.config.yaml +4 -0
  94. litplan-0.0.1/profiles/workflowgen/default_kind_map.yaml +43 -0
  95. litplan-0.0.1/pyproject.toml +62 -0
  96. litplan-0.0.1/src/litplan/__init__.py +199 -0
  97. litplan-0.0.1/src/litplan/audit_bundle.py +154 -0
  98. litplan-0.0.1/src/litplan/chunking.py +341 -0
  99. litplan-0.0.1/src/litplan/chunking_policy.py +120 -0
  100. litplan-0.0.1/src/litplan/context_safety.py +119 -0
  101. litplan-0.0.1/src/litplan/document_ir.py +107 -0
  102. litplan-0.0.1/src/litplan/document_ir_artifacts.py +158 -0
  103. litplan-0.0.1/src/litplan/eval_annotations.py +113 -0
  104. litplan-0.0.1/src/litplan/experiment_scenarios.py +29 -0
  105. litplan-0.0.1/src/litplan/graph/__init__.py +39 -0
  106. litplan-0.0.1/src/litplan/graph/checkpoint.py +40 -0
  107. litplan-0.0.1/src/litplan/graph/pipeline_hitl.py +177 -0
  108. litplan-0.0.1/src/litplan/graph/planner_critic.py +276 -0
  109. litplan-0.0.1/src/litplan/graph/structured_plan.py +85 -0
  110. litplan-0.0.1/src/litplan/ingest/__init__.py +24 -0
  111. litplan-0.0.1/src/litplan/ingest/grobid_client.py +340 -0
  112. litplan-0.0.1/src/litplan/ingest/grobid_settings.py +68 -0
  113. litplan-0.0.1/src/litplan/ingest/pdf.py +69 -0
  114. litplan-0.0.1/src/litplan/ingest/tei_to_document_ir.py +388 -0
  115. litplan-0.0.1/src/litplan/json_tools.py +1295 -0
  116. litplan-0.0.1/src/litplan/librarian/__init__.py +21 -0
  117. litplan-0.0.1/src/litplan/librarian/embeddings.py +35 -0
  118. litplan-0.0.1/src/litplan/librarian/exceptions.py +13 -0
  119. litplan-0.0.1/src/litplan/librarian/sqlite_vec_index.py +305 -0
  120. litplan-0.0.1/src/litplan/llm/__init__.py +62 -0
  121. litplan-0.0.1/src/litplan/llm/anthropic_provider.py +18 -0
  122. litplan-0.0.1/src/litplan/llm/config.py +233 -0
  123. litplan-0.0.1/src/litplan/llm/factory.py +67 -0
  124. litplan-0.0.1/src/litplan/llm/gemini.py +18 -0
  125. litplan-0.0.1/src/litplan/llm/grok.py +23 -0
  126. litplan-0.0.1/src/litplan/llm/openai_provider.py +18 -0
  127. litplan-0.0.1/src/litplan/llm/structured.py +95 -0
  128. litplan-0.0.1/src/litplan/llm/stub.py +106 -0
  129. litplan-0.0.1/src/litplan/llm/usage_metrics.py +149 -0
  130. litplan-0.0.1/src/litplan/log_redaction.py +97 -0
  131. litplan-0.0.1/src/litplan/pipeline_compile.py +281 -0
  132. litplan-0.0.1/src/litplan/pipeline_serialization.py +54 -0
  133. litplan-0.0.1/src/litplan/pipeline_spec.py +155 -0
  134. litplan-0.0.1/src/litplan/prefect_chaos.py +93 -0
  135. litplan-0.0.1/src/litplan/prefect_node_policies.py +106 -0
  136. litplan-0.0.1/src/litplan/prefect_pipeline.py +311 -0
  137. litplan-0.0.1/src/litplan/run_status.py +299 -0
  138. litplan-0.0.1/src/litplan/store/__init__.py +65 -0
  139. litplan-0.0.1/src/litplan/store/base.py +9 -0
  140. litplan-0.0.1/src/litplan/store/engine.py +44 -0
  141. litplan-0.0.1/src/litplan/store/eval_provenance_annotations.py +93 -0
  142. litplan-0.0.1/src/litplan/store/models.py +124 -0
  143. litplan-0.0.1/src/litplan/store/node_checkpoint.py +112 -0
  144. litplan-0.0.1/src/litplan/store/plan_approval_records.py +91 -0
  145. litplan-0.0.1/src/litplan/store/repository.py +208 -0
  146. litplan-0.0.1/src/litplan/store/urls.py +122 -0
  147. litplan-0.0.1/src/litplan/telemetry.py +107 -0
  148. litplan-0.0.1/src/litplan/tool_contracts.py +129 -0
  149. litplan-0.0.1/src/litplan/tool_registry.py +262 -0
  150. litplan-0.0.1/src/litplan/tools.py +597 -0
  151. litplan-0.0.1/src/litplan/ui/__init__.py +1 -0
  152. litplan-0.0.1/src/litplan/ui/plan_draft_app.py +642 -0
  153. litplan-0.0.1/src/litplan/ui/plan_draft_ui.py +61 -0
  154. litplan-0.0.1/src/litplan/workflowgen/__init__.py +80 -0
  155. litplan-0.0.1/src/litplan/workflowgen/airflow_gen.py +309 -0
  156. litplan-0.0.1/src/litplan/workflowgen/default_kind_map.yaml +38 -0
  157. litplan-0.0.1/src/litplan/workflowgen/kind_map.py +224 -0
  158. litplan-0.0.1/src/litplan/workflowgen/nextflow_gen.py +504 -0
  159. litplan-0.0.1/src/litplan/workflowgen/normalized_dag.py +72 -0
  160. litplan-0.0.1/src/litplan/workflowgen/snakemake_gen.py +308 -0
  161. litplan-0.0.1/src/litplan/workflowgen/spec_json.py +47 -0
  162. litplan-0.0.1/src/litplan_cli/__init__.py +1 -0
  163. litplan-0.0.1/src/litplan_cli/main.py +521 -0
  164. litplan-0.0.1/src/litplan_mcp/__init__.py +5 -0
  165. litplan-0.0.1/src/litplan_mcp/__main__.py +6 -0
  166. litplan-0.0.1/src/litplan_mcp/server.py +246 -0
  167. litplan-0.0.1/tests/.gitkeep +0 -0
  168. litplan-0.0.1/tests/conftest.py +23 -0
  169. litplan-0.0.1/tests/golden/document_ir_structural_minimal.json +131 -0
  170. litplan-0.0.1/tests/golden/t8_compile_expected.json +59 -0
  171. litplan-0.0.1/tests/golden/t8_log_redaction_lines.json +44 -0
  172. litplan-0.0.1/tests/golden/t8_pipeline_spec_linear.json +46 -0
  173. litplan-0.0.1/tests/golden/t8_prefect_checkpoint_a.json +7 -0
  174. litplan-0.0.1/tests/golden/t8_prefect_checkpoint_b.json +8 -0
  175. litplan-0.0.1/tests/golden/t8_tool_args_invalid.json +30 -0
  176. litplan-0.0.1/tests/golden/t8_tool_args_valid.json +47 -0
  177. litplan-0.0.1/tests/integration/__init__.py +1 -0
  178. litplan-0.0.1/tests/integration/test_t214_chunk_e2e.py +355 -0
  179. litplan-0.0.1/tests/test_context_poisoning_fixtures.py +100 -0
  180. litplan-0.0.1/tests/test_experiment_recovery_targets.py +66 -0
  181. litplan-0.0.1/tests/test_export_workflow_mcp_cli_parity.py +380 -0
  182. litplan-0.0.1/tests/test_grobid_client.py +114 -0
  183. litplan-0.0.1/tests/test_http_mock_httpserver.py +110 -0
  184. litplan-0.0.1/tests/test_llm_provider_config.py +52 -0
  185. litplan-0.0.1/tests/test_phase3_exit_criteria.py +198 -0
  186. litplan-0.0.1/tests/test_phase4_exit_criteria.py +241 -0
  187. litplan-0.0.1/tests/test_phase5_exit_criteria.py +177 -0
  188. litplan-0.0.1/tests/test_phase6_exit_criteria.py +205 -0
  189. litplan-0.0.1/tests/test_phase7_mcp_cli_parity.py +610 -0
  190. litplan-0.0.1/tests/test_phase8_golden_path.py +214 -0
  191. litplan-0.0.1/tests/test_phase8_log_redaction.py +35 -0
  192. litplan-0.0.1/tests/test_pipeline_spec.py +78 -0
  193. litplan-0.0.1/tests/test_smoke.py +36 -0
  194. litplan-0.0.1/tests/test_t7_tool_contracts.py +174 -0
  195. litplan-0.0.1/tests/test_t810_plan_compile_mcp_cli_parity.py +325 -0
  196. litplan-0.0.1/tests/test_t812_grounded_plan_compile_experiments_stub.py +82 -0
  197. litplan-0.0.1/tests/test_t88_merge_semantics.py +288 -0
  198. litplan-0.0.1/tests/test_t8_otel_compile_execute_trace_tree.py +153 -0
  199. litplan-0.0.1/tests/test_tei_document_ir_golden.py +73 -0
  200. litplan-0.0.1/tests/test_workflowgen_airflow_golden.py +142 -0
  201. litplan-0.0.1/tests/test_workflowgen_nextflow_golden.py +118 -0
  202. litplan-0.0.1/tests/test_workflowgen_normalized_dag.py +102 -0
  203. litplan-0.0.1/tests/test_workflowgen_snakemake_golden.py +71 -0
  204. litplan-0.0.1/uv.lock +4321 -0
@@ -0,0 +1,9 @@
1
+ .venv
2
+ .env
3
+ __pycache__
4
+ *.pyc
5
+ .git
6
+ .pytest_cache
7
+ .ruff_cache
8
+ .mypy_cache
9
+ runs
@@ -0,0 +1,71 @@
1
+ # Litplan — copy to `.env` locally (never commit `.env`).
2
+ # API keys and tokens MUST come from the environment (or your secret store),
3
+ # never from committed prompts, fixtures, or application source.
4
+
5
+ # --- LLM (provider-agnostic factory; see README "Configuration & secrets") ---
6
+
7
+ # Selects which adapter the factory uses. Code default when unset is `gemini` (needs GOOGLE_API_KEY).
8
+ # Supported: stub, gemini, openai, anthropic, grok (xAI OpenAI-compatible endpoint).
9
+ # Use `stub` in this file for offline copy-paste until you add a key.
10
+ LITPLAN_LLM_PROVIDER=stub
11
+
12
+ # Model id passed to the active provider. Leave empty to use the built-in default for that provider
13
+ # (e.g. Gemini: gemini-3.1-flash-lite-preview, OpenAI: gpt-4.1, Anthropic: claude-sonnet-4-20250514, Grok: grok-3-mini).
14
+ LITPLAN_LLM_MODEL=
15
+
16
+ # --- Keys (set only for the provider you use) ---
17
+
18
+ # gemini: Google AI Studio / Gemini API. Either name works (GOOGLE_API_KEY, else GEMINI_API_KEY).
19
+ # GOOGLE_API_KEY=
20
+ # GEMINI_API_KEY=
21
+
22
+ # openai: Chat Completions API (OPENAI_API_KEY).
23
+ # OPENAI_API_KEY=
24
+
25
+ # anthropic: Messages API (ANTHROPIC_API_KEY).
26
+ # ANTHROPIC_API_KEY=
27
+
28
+ # grok: xAI OpenAI-compatible API (XAI_API_KEY). Optional override for the base URL (default https://api.x.ai/v1).
29
+ # XAI_API_KEY=
30
+ # LITPLAN_XAI_BASE_URL=
31
+
32
+ # --- GROBID (PDF → TEI; run the official container, then point the app at it) ---
33
+
34
+ # Base URL only (no trailing path). Example: http://127.0.0.1:8070
35
+ # LITPLAN_GROBID_URL=http://127.0.0.1:8070
36
+
37
+ # Optional digest/tag of the GROBID image used for this parse (manifest / reproducibility).
38
+ # Pin matches docker-compose.grobid.yml (CRF-only; avoid *-full* unless you intend the ~8 GB image).
39
+ # LITPLAN_GROBID_IMAGE_REF=grobid/grobid:0.8.2.1-crf
40
+
41
+ # Connect vs read/write timeouts (seconds). Full-text can be slow on large PDFs.
42
+ # LITPLAN_GROBID_CONNECT_TIMEOUT_S=30
43
+ # LITPLAN_GROBID_TIMEOUT_S=300
44
+
45
+ # Retries for transient failures (timeouts, connection errors, 5xx). Total attempts = 1 + this value.
46
+ # LITPLAN_GROBID_MAX_RETRIES=2
47
+
48
+ # Base seconds for exponential backoff between retries (0 = immediate retry).
49
+ # LITPLAN_GROBID_RETRY_BACKOFF_BASE_S=1
50
+
51
+ # --- Chunking (DocumentIR → list[DocumentChunk]; see docs/DEVS_README.md) ---
52
+
53
+ # Max estimated tokens per chunk (rough count; default_token_count in chunking.py).
54
+ # LITPLAN_CHUNK_MAX_TOKENS=512
55
+
56
+ # Token overlap carried into the next chunk (default 0).
57
+ # LITPLAN_CHUNK_OVERLAP_TOKENS=0
58
+
59
+ # If true, do not pack segments from different sections into one chunk (default true).
60
+ # LITPLAN_CHUNK_RESPECT_SECTION_BOUNDARIES=true
61
+
62
+ # --- Prefect node chaos (T6.4; dev / simulation only) ---
63
+
64
+ # Integer seed enables simulated latency + random task failures (see README "Prefect execution & chaos").
65
+ # LITPLAN_CHAOS_SEED=1
66
+
67
+ # Optional: failure probability [0, 1], default 0.2
68
+ # LITPLAN_CHAOS_FAILURE_PROB=0.2
69
+
70
+ # Optional: max sleep ms before failure roll, default 250 (0 = no sleep)
71
+ # LITPLAN_CHAOS_LATENCY_MAX_MS=250
@@ -0,0 +1,84 @@
1
+ name: CI
2
+
3
+ on:
4
+ push:
5
+ branches: [main]
6
+ pull_request:
7
+
8
+ permissions:
9
+ contents: read
10
+
11
+ env:
12
+ FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
13
+
14
+ jobs:
15
+ pytest:
16
+ name: Pytest
17
+ runs-on: ubuntu-latest
18
+ steps:
19
+ - uses: actions/checkout@v6
20
+
21
+ - name: Set up Python
22
+ uses: actions/setup-python@v6
23
+ with:
24
+ python-version: "3.11"
25
+
26
+ - name: Install uv
27
+ uses: astral-sh/setup-uv@v7
28
+ with:
29
+ enable-cache: true
30
+ cache-suffix: ci-pytest
31
+
32
+ - name: Sync project with dev dependencies
33
+ run: uv sync --frozen --group dev
34
+
35
+ - name: Run pytest
36
+ run: uv run pytest
37
+
38
+ build:
39
+ name: Build package
40
+ runs-on: ubuntu-latest
41
+ steps:
42
+ - uses: actions/checkout@v6
43
+
44
+ - name: Set up Python
45
+ uses: actions/setup-python@v6
46
+ with:
47
+ python-version: "3.11"
48
+
49
+ - name: Install uv
50
+ uses: astral-sh/setup-uv@v7
51
+ with:
52
+ enable-cache: true
53
+ cache-suffix: ci-build
54
+
55
+ - name: Build source and wheel distributions
56
+ run: uv build
57
+
58
+ migrations:
59
+ name: Alembic smoke
60
+ runs-on: ubuntu-latest
61
+ env:
62
+ LITPLAN_HOME: ${{ github.workspace }}/.litplan-home-ci
63
+ steps:
64
+ - uses: actions/checkout@v6
65
+
66
+ - name: Set up Python
67
+ uses: actions/setup-python@v6
68
+ with:
69
+ python-version: "3.11"
70
+
71
+ - name: Install uv
72
+ uses: astral-sh/setup-uv@v7
73
+ with:
74
+ enable-cache: true
75
+ cache-suffix: ci-migrations
76
+
77
+ - name: Sync project runtime dependencies
78
+ run: uv sync --frozen
79
+
80
+ - name: Upgrade database to head
81
+ run: uv run alembic upgrade head
82
+
83
+ - name: Verify SQLite database was created
84
+ run: test -f "${LITPLAN_HOME}/runs/litplan.db"
@@ -0,0 +1,39 @@
1
+ # Publish to PyPI via Trusted Publishing (OIDC). Trigger: push a version tag only (e.g. v0.0.1).
2
+ # Configure the publisher on PyPI to match this repo, workflow file, and optional environment:
3
+ # https://docs.pypi.org/trusted-publishers/using-a-publisher/#github-actions
4
+ name: Publish to PyPI
5
+
6
+ on:
7
+ push:
8
+ tags:
9
+ - "v*"
10
+
11
+ permissions:
12
+ contents: read
13
+
14
+ jobs:
15
+ pypi-publish:
16
+ name: Upload release to PyPI
17
+ runs-on: ubuntu-latest
18
+ # Strongly recommended by PyPI; must match the "environment" in your PyPI trusted publisher config.
19
+ environment: pypi
20
+ permissions:
21
+ # Required for Trusted Publishing (OIDC).
22
+ id-token: write
23
+
24
+ steps:
25
+ - uses: actions/checkout@v6
26
+
27
+ - name: Set up Python
28
+ uses: actions/setup-python@v6
29
+ with:
30
+ python-version: "3.11"
31
+
32
+ - name: Install uv
33
+ uses: astral-sh/setup-uv@v7
34
+
35
+ - name: Build source and wheel distributions
36
+ run: uv build
37
+
38
+ - name: Publish package distributions to PyPI
39
+ uses: pypa/gh-action-pypi-publish@release/v1
@@ -0,0 +1,100 @@
1
+ name: Workflowgen engine integration
2
+
3
+ on:
4
+ push:
5
+ branches: [main]
6
+ pull_request:
7
+ workflow_dispatch:
8
+
9
+ permissions:
10
+ contents: read
11
+
12
+ jobs:
13
+ workflowgen:
14
+ runs-on: ubuntu-latest
15
+ strategy:
16
+ fail-fast: false
17
+ matrix:
18
+ engine: [snakemake, nextflow, airflow]
19
+ steps:
20
+ - uses: actions/checkout@v6
21
+
22
+ # Nextflow uses Java plus the upstream installer; other legs use the project env via uv.
23
+ - name: Install uv
24
+ if: matrix.engine != 'nextflow'
25
+ uses: astral-sh/setup-uv@v7
26
+ with:
27
+ enable-cache: true
28
+ cache-suffix: workflowgen-${{ matrix.engine }}
29
+
30
+ - name: Sync Python env (Snakemake only)
31
+ if: matrix.engine == 'snakemake'
32
+ run: uv sync --group dev
33
+
34
+ - name: Set up Java (Nextflow)
35
+ if: matrix.engine == 'nextflow'
36
+ uses: actions/setup-java@v5
37
+ with:
38
+ distribution: temurin
39
+ java-version: "17"
40
+
41
+ - name: Install Nextflow
42
+ if: matrix.engine == 'nextflow'
43
+ run: |
44
+ set -euo pipefail
45
+ mkdir -p "${HOME}/.local/bin"
46
+ curl -fsSL https://get.nextflow.io | bash
47
+ test -f nextflow
48
+ mv nextflow "${HOME}/.local/bin/nextflow"
49
+ chmod +x "${HOME}/.local/bin/nextflow"
50
+ echo "${HOME}/.local/bin" >> "${GITHUB_PATH}"
51
+
52
+ - name: Snakemake dry-run on golden bundles
53
+ if: matrix.engine == 'snakemake'
54
+ run: |
55
+ set -euo pipefail
56
+ ROOT="${GITHUB_WORKSPACE}"
57
+ export PROJECT_HOME="${ROOT}"
58
+ export RUN_ID="ci-workflowgen"
59
+ mkdir -p "${ROOT}/runs/${RUN_ID}/workspace"
60
+
61
+ for name in empty linear_chain diamond parallel_roots edges_unsorted_in_file artifact_chain_linear artifact_merge_diamond artifact_fork_selective; do
62
+ TMP="$(mktemp -d)"
63
+ cp "${ROOT}/fixtures/workflowgen/snakemake_golden/${name}.Snakefile" "${TMP}/Snakefile"
64
+ envsubst < "${ROOT}/fixtures/workflowgen/snakemake_golden/${name}.config.yaml" > "${TMP}/config.yaml"
65
+ echo "=== snakemake -n: ${name} ==="
66
+ cd "${ROOT}"
67
+ uv run snakemake -n --snakefile "${TMP}/Snakefile" --configfile "${TMP}/config.yaml"
68
+ done
69
+
70
+ - name: Nextflow config on golden bundles
71
+ if: matrix.engine == 'nextflow'
72
+ env:
73
+ GITHUB_WORKSPACE: ${{ github.workspace }}
74
+ run: |
75
+ set -euo pipefail
76
+ command -v nextflow >/dev/null 2>&1
77
+ ROOT="${GITHUB_WORKSPACE}"
78
+ for name in empty linear_chain diamond parallel_roots edges_unsorted_in_file; do
79
+ TMP="$(mktemp -d)"
80
+ cp "${ROOT}/fixtures/workflowgen/nextflow_golden/${name}.main.nf" "${TMP}/main.nf"
81
+ cp "${ROOT}/fixtures/workflowgen/nextflow_golden/${name}.nextflow.config" "${TMP}/nextflow.config"
82
+ echo "=== nextflow config: ${name} ==="
83
+ cd "${TMP}"
84
+ nextflow config .
85
+ done
86
+
87
+ - name: Airflow DagBag on golden DAG files
88
+ if: matrix.engine == 'airflow'
89
+ run: |
90
+ set -euo pipefail
91
+ ROOT="${GITHUB_WORKSPACE}"
92
+ for name in empty linear_chain diamond parallel_roots edges_unsorted_in_file; do
93
+ TMP="$(mktemp -d)"
94
+ cp "${ROOT}/fixtures/workflowgen/airflow_golden/${name}.litplan_dag.py" "${TMP}/litplan_dag.py"
95
+ echo "=== airflow DagBag: ${name} ==="
96
+ cd "${ROOT}"
97
+ uv run --with 'apache-airflow>=2.8,<3' python -c \
98
+ 'import sys; from pathlib import Path; from airflow.models import DagBag; f=Path(sys.argv[1]); db=DagBag(dag_folder=str(f), include_examples=False); assert not db.import_errors, db.import_errors; assert db.dags, "no DAGs"; print("ok", len(db.dags))' \
99
+ "${TMP}"
100
+ done
@@ -0,0 +1,11 @@
1
+ .venv/
2
+ .env
3
+ runs/
4
+ __pycache__/
5
+ .ruff_cache/
6
+ .pytest_cache/
7
+ *.py[cod]
8
+ dev
9
+ evals/t8_tool_use/report.json
10
+ .DS_Store
11
+ manual-work
@@ -0,0 +1,23 @@
1
+ # Litplan — Python 3.11 + uv, dependencies from uv.lock.
2
+ # Add apt packages only if a wheel build fails for a dependency.
3
+ FROM python:3.11-slim-bookworm
4
+
5
+ RUN apt-get update \
6
+ && apt-get install -y --no-install-recommends curl ca-certificates \
7
+ && rm -rf /var/lib/apt/lists/*
8
+
9
+ ENV UV_LINK_MODE=copy
10
+
11
+ WORKDIR /app
12
+
13
+ RUN curl -LsSf https://astral.sh/uv/install.sh | sh
14
+ ENV PATH="/root/.local/bin:${PATH}"
15
+
16
+ COPY pyproject.toml uv.lock README.md ./
17
+ COPY src ./src
18
+
19
+ RUN uv sync --frozen --group dev
20
+
21
+ COPY tests ./tests
22
+
23
+ CMD ["/bin/bash"]
litplan-0.0.1/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 litplan
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
litplan-0.0.1/PKG-INFO ADDED
@@ -0,0 +1,264 @@
1
+ Metadata-Version: 2.4
2
+ Name: litplan
3
+ Version: 0.0.1
4
+ Summary: Litplan: ingest papers, compile pipeline IR, run multi-agent workflows.
5
+ License: MIT License
6
+
7
+ Copyright (c) 2026 litplan
8
+
9
+ Permission is hereby granted, free of charge, to any person obtaining a copy
10
+ of this software and associated documentation files (the "Software"), to deal
11
+ in the Software without restriction, including without limitation the rights
12
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
13
+ copies of the Software, and to permit persons to whom the Software is
14
+ furnished to do so, subject to the following conditions:
15
+
16
+ The above copyright notice and this permission notice shall be included in all
17
+ copies or substantial portions of the Software.
18
+
19
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
20
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
21
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
22
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
23
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
24
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
25
+ SOFTWARE.
26
+ License-File: LICENSE
27
+ Requires-Python: >=3.11
28
+ Requires-Dist: alembic>=1.18.4
29
+ Requires-Dist: click>=8.1.0
30
+ Requires-Dist: httpx>=0.28.1
31
+ Requires-Dist: jsonschema>=4.23.0
32
+ Requires-Dist: langchain-anthropic>=0.3.0
33
+ Requires-Dist: langchain-core>=0.3.0
34
+ Requires-Dist: langchain-google-genai>=2.0.0
35
+ Requires-Dist: langchain-openai>=0.3.0
36
+ Requires-Dist: langgraph-checkpoint-sqlite>=3.0.3
37
+ Requires-Dist: langgraph>=1.1.3
38
+ Requires-Dist: lxml>=6.0.2
39
+ Requires-Dist: mcp>=1.2.0
40
+ Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.40.0
41
+ Requires-Dist: opentelemetry-sdk>=1.40.0
42
+ Requires-Dist: prefect>=3.0
43
+ Requires-Dist: pydantic-settings>=2.13.1
44
+ Requires-Dist: pydantic>=2.12.5
45
+ Requires-Dist: pyyaml>=6.0
46
+ Requires-Dist: sqlalchemy>=2.0.49
47
+ Requires-Dist: sqlite-vec>=0.1.6
48
+ Description-Content-Type: text/markdown
49
+
50
+ # Litplan
51
+
52
+ Litplan turns a research paper into structured document data, a checked and versioned `PipelineSpec`, optional approval records, and either a reference execution path or workflow files for your own runtime. It is the governed-plan layer, not a replacement for Snakemake, Nextflow, Airflow, or your HPC stack.
53
+
54
+ ## Start Here
55
+
56
+ ### Prerequisites
57
+
58
+ - Python 3.11+
59
+ - [uv](https://docs.astral.sh/uv/)
60
+ - Docker, only if you want live PDF ingest via GROBID
61
+ - An LLM API key, only if you want live goal-driven planning
62
+
63
+ Run these commands from the repository root:
64
+
65
+ ```bash
66
+ uv sync
67
+ cp .env.example .env
68
+ uv run litplan environment
69
+ uv run litplan ingest fixtures/tei/structural_minimal.tei.xml \
70
+ --parser-route tei_fixture_replay \
71
+ --document-ir
72
+ ```
73
+
74
+ What to expect:
75
+
76
+ - `uv sync` installs the package and CLI from `uv.lock`.
77
+ - `cp .env.example .env` gives you a local config file; it defaults to `LITPLAN_LLM_PROVIDER=stub`, so this path stays offline.
78
+ - `uv run litplan environment` prints JSON hints about paths and config resolution.
79
+ - The offline `ingest` command should return JSON with `"ok": true` and inline `DocumentIR` output.
80
+
81
+ ## Choose Your Path
82
+
83
+
84
+ | Path | Use it when | Needs | First step |
85
+ | --------------- | ---------------------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------------------------- |
86
+ | Offline demo | You want a first successful run with no external services | `uv` | Run the `tei_fixture_replay` ingest command above |
87
+ | Live PDF ingest | You want to parse a real PDF through GROBID | Docker + GROBID | `docker compose -f docker-compose.grobid.yml up -d` |
88
+ | LLM planning | You want `plan-compile` to draft a pipeline from a goal | A real provider in `.env` plus a matching API key | Set `LITPLAN_LLM_PROVIDER` and provider key in `.env` |
89
+ | Workflow export | You want Snakemake, Nextflow, or Airflow files from a stored run | A persisted run in the local DB | Compile with `--create-run`, then `approve`, then `export-workflow` |
90
+
91
+
92
+ ## Fastest Successful Runs
93
+
94
+ ### 1. Offline ingest demo
95
+
96
+ This is the fastest self-contained path and works in a fresh checkout after `uv sync`.
97
+
98
+ ```bash
99
+ uv run litplan ingest fixtures/tei/structural_minimal.tei.xml \
100
+ --parser-route tei_fixture_replay \
101
+ --document-ir
102
+ ```
103
+
104
+ Expected outcome:
105
+
106
+ - You should get JSON output with `"ok": true`.
107
+ - You should get inline `DocumentIR` JSON.
108
+ - No Docker, database, or API key is required.
109
+
110
+ ### 2. Persist a local run
111
+
112
+ Use this when you want approvals, execution, status, or export commands to work against the local SQLite store.
113
+
114
+ ```bash
115
+ export LITPLAN_HOME="$(pwd)/.litplan-home"
116
+ uv run alembic upgrade head
117
+ RUN_ID="examples-linear-chain-run-$(date +%s)"
118
+ uv run litplan compile fixtures/workflowgen/pipeline_specs/linear_chain.json \
119
+ --pipeline-id examples.linear_chain \
120
+ --lockfile uv.lock \
121
+ --run-id "$RUN_ID" \
122
+ --create-run
123
+ ```
124
+
125
+ Expected outcome:
126
+
127
+ - This creates `.litplan-home/` and initializes the SQLite schema there.
128
+ - The compile command should return `"ok": true`.
129
+ - The run and compiled revision are now persisted for later `approve`, `run-status`, `execute`, and export commands.
130
+
131
+ ### 3. Live PDF ingest with GROBID
132
+
133
+ If PDF ingest is the feature you care about, this is the shortest working setup.
134
+
135
+ Start GROBID:
136
+
137
+ ```bash
138
+ docker compose -f docker-compose.grobid.yml up -d
139
+ curl -fsSL http://127.0.0.1:8070/api/isalive
140
+ ```
141
+
142
+ Then set `LITPLAN_GROBID_URL=http://127.0.0.1:8070` in `.env` and run:
143
+
144
+ ```bash
145
+ uv run litplan ingest \
146
+ fixtures/papers/pdf/batatia-2022-mace-force-fields-arxiv-2206.07697v2.pdf \
147
+ --document-ir
148
+ ```
149
+
150
+ Expected outcome:
151
+
152
+ - The health check should print an `alive` response from GROBID.
153
+ - The ingest command should return `"ok": true`.
154
+ - You should get parsed document metadata and, with `--document-ir`, inline `DocumentIR` JSON for the PDF.
155
+
156
+ ### 4. Live LLM planning
157
+
158
+ For this path, change `.env` from the default `stub` provider to a real provider and add its key. Example:
159
+
160
+ ```bash
161
+ LITPLAN_LLM_PROVIDER=gemini
162
+ GOOGLE_API_KEY=your-real-key
163
+ ```
164
+
165
+ Then run:
166
+
167
+ ```bash
168
+ uv run litplan plan-compile "build linear noop pipeline" \
169
+ --pipeline-id examples.goal_demo \
170
+ --lockfile uv.lock \
171
+ --expanded-spec
172
+ ```
173
+
174
+ Expected outcome:
175
+
176
+ - The command should return `"ok": true` when the provider is configured correctly.
177
+ - You should get a compiled pipeline result and expanded spec JSON.
178
+ - Use `compile` instead of `plan-compile` when you already have full `PipelineSpec` JSON and do not want an LLM involved.
179
+
180
+ ### 5. Workflow export
181
+
182
+ Once you have a persisted run, you can approve it and generate engine-native workflow files.
183
+
184
+ ```bash
185
+ uv run litplan approve "$RUN_ID"
186
+ uv run litplan export-workflow "$RUN_ID" snakemake ./tmp/workflow-out
187
+ ```
188
+
189
+ Expected outcome:
190
+
191
+ - `approve` records a non-pending run status for `RUN_ID`.
192
+ - `export-workflow` writes files under `./tmp/workflow-out`.
193
+ - You can switch `snakemake` to `nextflow` or `airflow`.
194
+
195
+ ## Main Commands
196
+
197
+
198
+ | Command | What it does |
199
+ | ----------------------------- | ------------------------------------------------------------------------- |
200
+ | `litplan ingest` | Parse a PDF or TEI file into `DocumentIR`; can include inline JSON output |
201
+ | `litplan compile` | Validate and compile a `PipelineSpec` JSON input |
202
+ | `litplan plan-compile` | Use a goal-driven planner, then run the same compiler |
203
+ | `litplan approve` | Record approval or other status transitions for a run |
204
+ | `litplan execute` | Execute a persisted approved run through the reference Prefect path |
205
+ | `litplan run-status` | Show unified run, planning, and execution status |
206
+ | `litplan export-audit-bundle` | Write a reproducibility-oriented bundle for a run |
207
+ | `litplan export-workflow` | Generate Snakemake, Nextflow, or Airflow files from a stored revision |
208
+ | `litplan environment` | Print non-secret environment and path hints |
209
+
210
+
211
+ Run `uv run litplan --help` for the full CLI and option details.
212
+
213
+ ## Project Surfaces
214
+
215
+ - CLI: `uv run litplan`
216
+ - MCP server: `uv run litplan-mcp`
217
+ - Python library: `import litplan`
218
+ - Streamlit UI (optional): `uv run streamlit run src/litplan/ui/plan_draft_app.py`
219
+
220
+ The CLI and MCP server call the same shared JSON tool layer, so they expose the same core operations.
221
+
222
+ ### Streamlit UI
223
+
224
+ The optional Streamlit app is a browser UI on top of the same SQLite project database. You can edit **PlanDraft** text and compare it to a prior draft, append **approval** records, append **hallucination** flags with chunk provenance, and load **unified run status** (LangGraph checkpoints, run timeline, execute-phase node checkpoints, OpenTelemetry hints). Ingest, compile, plan-compile, approve, execute, export, and other pipeline steps still run from the CLI or MCP; the UI does not replace those tools.
225
+
226
+ ![Litplan Streamlit PlanDraft UI](docs/assets/litplan_streamlit_ui.png)
227
+
228
+ From the repository root, after `uv sync` and `uv run alembic upgrade head` for your chosen `LITPLAN_HOME`:
229
+
230
+ ```bash
231
+ export LITPLAN_HOME="$(pwd)/.litplan-home"
232
+ uv run streamlit run src/litplan/ui/plan_draft_app.py
233
+ ```
234
+
235
+ For a full step-by-step (including diff panels, approvals, flags, and run explorer), see **Part 3** in [MANUAL_CLI_MCP_UI_WORKFLOW.md](docs/manual/MANUAL_CLI_MCP_UI_WORKFLOW.md).
236
+
237
+ ## Examples
238
+
239
+ Library-first examples live in `examples/`:
240
+
241
+ - `examples/paper_repro_sketch_tei.py`: saved TEI -> `DocumentIR` -> chunking -> hand-authored `PipelineSpec` -> compile
242
+ - `examples/paper_repro_sketch_pdf.py`: PDF path -> live GROBID ingest -> the same chunk/plan/compile flow
243
+
244
+ Run them from the repository root:
245
+
246
+ ```bash
247
+ uv run python examples/paper_repro_sketch_tei.py
248
+ uv run python examples/paper_repro_sketch_pdf.py \
249
+ fixtures/papers/pdf/batatia-2022-mace-force-fields-arxiv-2206.07697v2.pdf
250
+ ```
251
+
252
+ ## What To Expect
253
+
254
+ - The package version is **0.0.1** (see `pyproject.toml`); pre-1.0 semver releases may still evolve public APIs.
255
+ - Offline and stub-backed flows remain supported for local development and tests.
256
+ - The bundled `execute` path demonstrates orchestration, retries, checkpoints, and bookkeeping; use your own workflow engine or cluster stack for production-scale execution.
257
+
258
+ ## Where To Go Next
259
+
260
+ - [Product scenarios, no-technical overview](docs/PRODUCT_SCENARIOS.md)
261
+ - [Full configuration, env vars, architecture, and command reference](docs/DEVS_README.md)
262
+ - [End-to-end operator walkthrough across CLI, MCP, and UI](docs/manual/MANUAL_CLI_MCP_UI_WORKFLOW.md)
263
+ - [Workflow export details, kind maps, and safety model](docs/workflowgen/)
264
+ - [Paper fixture notes](fixtures/papers/FIXTURES.md)