litplan 0.0.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- litplan-0.0.1/.dockerignore +9 -0
- litplan-0.0.1/.env.example +71 -0
- litplan-0.0.1/.github/workflows/ci.yml +84 -0
- litplan-0.0.1/.github/workflows/publish-pypi.yml +39 -0
- litplan-0.0.1/.github/workflows/workflowgen-engines.yml +100 -0
- litplan-0.0.1/.gitignore +11 -0
- litplan-0.0.1/Dockerfile +23 -0
- litplan-0.0.1/LICENSE +21 -0
- litplan-0.0.1/PKG-INFO +264 -0
- litplan-0.0.1/README.md +215 -0
- litplan-0.0.1/alembic/env.py +69 -0
- litplan-0.0.1/alembic/script.py.mako +26 -0
- litplan-0.0.1/alembic/versions/5899d8e46cfa_t3_3_pipeline_spec_revisions_and_runs.py +73 -0
- litplan-0.0.1/alembic/versions/c7f2a1b9e4d0_t6_3_pipeline_run_node_checkpoints.py +40 -0
- litplan-0.0.1/alembic/versions/d8e3f0a1b2c4_t10_2_plan_approval_records.py +50 -0
- litplan-0.0.1/alembic/versions/e9a1c2d3f4b5_t10_4_eval_provenance_annotations.py +48 -0
- litplan-0.0.1/alembic.ini +45 -0
- litplan-0.0.1/docker-compose.grobid.yml +12 -0
- litplan-0.0.1/docs/CAMELINE_REQUEST_FLOW_DEVELOPER_GUIDE.md +553 -0
- litplan-0.0.1/docs/CONTEXT_POISONING_POLICY.md +15 -0
- litplan-0.0.1/docs/DEVS_README.md +323 -0
- litplan-0.0.1/docs/PRODUCT_SCENARIOS.md +277 -0
- litplan-0.0.1/docs/adr/0002-librarian-chunk-index.md +30 -0
- litplan-0.0.1/docs/adr/0003-grobid-only-pdf-ingestion.md +46 -0
- litplan-0.0.1/docs/adr/0004-mcp-vs-cli.md +93 -0
- litplan-0.0.1/docs/assets/litplan_streamlit_ui.png +0 -0
- litplan-0.0.1/docs/manual/MANUAL_CLI_MCP_UI_WORKFLOW.md +459 -0
- litplan-0.0.1/docs/workflowgen/ENVIRONMENT_FINGERPRINT.md +96 -0
- litplan-0.0.1/docs/workflowgen/GENERATION_POLICY.md +101 -0
- litplan-0.0.1/docs/workflowgen/IDEMPOTENCY_AND_RESUME.md +99 -0
- litplan-0.0.1/docs/workflowgen/PATH_MODEL.md +124 -0
- litplan-0.0.1/docs/workflowgen/SECURITY_REVIEW_CHECKLIST.md +60 -0
- litplan-0.0.1/docs/workflowgen/WORKFLOW_EXPORT.md +103 -0
- litplan-0.0.1/evals/.gitkeep +0 -0
- litplan-0.0.1/evals/t10_experiment_scenario_demo.py +308 -0
- litplan-0.0.1/evals/t8_tool_use/dataset.json +114 -0
- litplan-0.0.1/evals/t8_tool_use/score_tool_use_eval.py +368 -0
- litplan-0.0.1/examples/paper_repro_sketch_pdf.py +210 -0
- litplan-0.0.1/examples/paper_repro_sketch_tei.py +200 -0
- litplan-0.0.1/fixtures/experiments/synth_dft_bulk_si/expected/recovery_targets.yaml +40 -0
- litplan-0.0.1/fixtures/experiments/synth_dft_bulk_si/synth_dft_bulk_si.tei.xml +47 -0
- litplan-0.0.1/fixtures/papers/FIXTURES.md +31 -0
- litplan-0.0.1/fixtures/papers/context_poisoning_minimal.tei.xml +15 -0
- litplan-0.0.1/fixtures/papers/pdf/batatia-2022-mace-force-fields-arxiv-2206.07697v2.pdf +0 -0
- litplan-0.0.1/fixtures/papers/pdf/yu-2024-systematic-assessment-universal-mlip-arxiv-2403.05729v3.pdf +0 -0
- litplan-0.0.1/fixtures/tei/README.md +42 -0
- litplan-0.0.1/fixtures/tei/structural_minimal.tei.xml +45 -0
- litplan-0.0.1/fixtures/workflowgen/airflow_golden/airflow_parameters_quoting.litplan_dag.py +35 -0
- litplan-0.0.1/fixtures/workflowgen/airflow_golden/diamond.litplan_dag.py +78 -0
- litplan-0.0.1/fixtures/workflowgen/airflow_golden/edges_unsorted_in_file.litplan_dag.py +63 -0
- litplan-0.0.1/fixtures/workflowgen/airflow_golden/empty.litplan_dag.py +33 -0
- litplan-0.0.1/fixtures/workflowgen/airflow_golden/linear_chain.litplan_dag.py +63 -0
- litplan-0.0.1/fixtures/workflowgen/airflow_golden/parallel_roots.litplan_dag.py +63 -0
- litplan-0.0.1/fixtures/workflowgen/environment_sidecars/apptainer.stub.def +7 -0
- litplan-0.0.1/fixtures/workflowgen/environment_sidecars/conda.env.stub.yml +8 -0
- litplan-0.0.1/fixtures/workflowgen/environment_sidecars/nextflow.env_hints.stub.config +4 -0
- litplan-0.0.1/fixtures/workflowgen/kind_maps/artifact_fork_selective.yaml +19 -0
- litplan-0.0.1/fixtures/workflowgen/nextflow_golden/diamond.main.nf +86 -0
- litplan-0.0.1/fixtures/workflowgen/nextflow_golden/diamond.nextflow.config +12 -0
- litplan-0.0.1/fixtures/workflowgen/nextflow_golden/edges_unsorted_in_file.main.nf +69 -0
- litplan-0.0.1/fixtures/workflowgen/nextflow_golden/edges_unsorted_in_file.nextflow.config +12 -0
- litplan-0.0.1/fixtures/workflowgen/nextflow_golden/empty.main.nf +32 -0
- litplan-0.0.1/fixtures/workflowgen/nextflow_golden/empty.nextflow.config +12 -0
- litplan-0.0.1/fixtures/workflowgen/nextflow_golden/linear_chain.main.nf +69 -0
- litplan-0.0.1/fixtures/workflowgen/nextflow_golden/linear_chain.nextflow.config +12 -0
- litplan-0.0.1/fixtures/workflowgen/nextflow_golden/parallel_roots.main.nf +69 -0
- litplan-0.0.1/fixtures/workflowgen/nextflow_golden/parallel_roots.nextflow.config +12 -0
- litplan-0.0.1/fixtures/workflowgen/pipeline_specs/airflow_parameters_quoting.json +23 -0
- litplan-0.0.1/fixtures/workflowgen/pipeline_specs/airflow_write_missing_content.json +14 -0
- litplan-0.0.1/fixtures/workflowgen/pipeline_specs/artifact_chain_linear.json +26 -0
- litplan-0.0.1/fixtures/workflowgen/pipeline_specs/artifact_fork_selective.json +24 -0
- litplan-0.0.1/fixtures/workflowgen/pipeline_specs/artifact_merge_diamond.json +24 -0
- litplan-0.0.1/fixtures/workflowgen/pipeline_specs/diamond.json +15 -0
- litplan-0.0.1/fixtures/workflowgen/pipeline_specs/edges_unsorted_in_file.json +12 -0
- litplan-0.0.1/fixtures/workflowgen/pipeline_specs/empty.json +5 -0
- litplan-0.0.1/fixtures/workflowgen/pipeline_specs/linear_chain.json +12 -0
- litplan-0.0.1/fixtures/workflowgen/pipeline_specs/parallel_roots.json +12 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/artifact_chain_linear.Snakefile +61 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/artifact_chain_linear.config.yaml +4 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/artifact_fork_selective.Snakefile +43 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/artifact_fork_selective.config.yaml +4 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/artifact_merge_diamond.Snakefile +43 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/artifact_merge_diamond.config.yaml +4 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/diamond.Snakefile +35 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/diamond.config.yaml +4 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/edges_unsorted_in_file.Snakefile +27 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/edges_unsorted_in_file.config.yaml +4 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/empty.Snakefile +10 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/empty.config.yaml +4 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/linear_chain.Snakefile +27 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/linear_chain.config.yaml +4 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/parallel_roots.Snakefile +27 -0
- litplan-0.0.1/fixtures/workflowgen/snakemake_golden/parallel_roots.config.yaml +4 -0
- litplan-0.0.1/profiles/workflowgen/default_kind_map.yaml +43 -0
- litplan-0.0.1/pyproject.toml +62 -0
- litplan-0.0.1/src/litplan/__init__.py +199 -0
- litplan-0.0.1/src/litplan/audit_bundle.py +154 -0
- litplan-0.0.1/src/litplan/chunking.py +341 -0
- litplan-0.0.1/src/litplan/chunking_policy.py +120 -0
- litplan-0.0.1/src/litplan/context_safety.py +119 -0
- litplan-0.0.1/src/litplan/document_ir.py +107 -0
- litplan-0.0.1/src/litplan/document_ir_artifacts.py +158 -0
- litplan-0.0.1/src/litplan/eval_annotations.py +113 -0
- litplan-0.0.1/src/litplan/experiment_scenarios.py +29 -0
- litplan-0.0.1/src/litplan/graph/__init__.py +39 -0
- litplan-0.0.1/src/litplan/graph/checkpoint.py +40 -0
- litplan-0.0.1/src/litplan/graph/pipeline_hitl.py +177 -0
- litplan-0.0.1/src/litplan/graph/planner_critic.py +276 -0
- litplan-0.0.1/src/litplan/graph/structured_plan.py +85 -0
- litplan-0.0.1/src/litplan/ingest/__init__.py +24 -0
- litplan-0.0.1/src/litplan/ingest/grobid_client.py +340 -0
- litplan-0.0.1/src/litplan/ingest/grobid_settings.py +68 -0
- litplan-0.0.1/src/litplan/ingest/pdf.py +69 -0
- litplan-0.0.1/src/litplan/ingest/tei_to_document_ir.py +388 -0
- litplan-0.0.1/src/litplan/json_tools.py +1295 -0
- litplan-0.0.1/src/litplan/librarian/__init__.py +21 -0
- litplan-0.0.1/src/litplan/librarian/embeddings.py +35 -0
- litplan-0.0.1/src/litplan/librarian/exceptions.py +13 -0
- litplan-0.0.1/src/litplan/librarian/sqlite_vec_index.py +305 -0
- litplan-0.0.1/src/litplan/llm/__init__.py +62 -0
- litplan-0.0.1/src/litplan/llm/anthropic_provider.py +18 -0
- litplan-0.0.1/src/litplan/llm/config.py +233 -0
- litplan-0.0.1/src/litplan/llm/factory.py +67 -0
- litplan-0.0.1/src/litplan/llm/gemini.py +18 -0
- litplan-0.0.1/src/litplan/llm/grok.py +23 -0
- litplan-0.0.1/src/litplan/llm/openai_provider.py +18 -0
- litplan-0.0.1/src/litplan/llm/structured.py +95 -0
- litplan-0.0.1/src/litplan/llm/stub.py +106 -0
- litplan-0.0.1/src/litplan/llm/usage_metrics.py +149 -0
- litplan-0.0.1/src/litplan/log_redaction.py +97 -0
- litplan-0.0.1/src/litplan/pipeline_compile.py +281 -0
- litplan-0.0.1/src/litplan/pipeline_serialization.py +54 -0
- litplan-0.0.1/src/litplan/pipeline_spec.py +155 -0
- litplan-0.0.1/src/litplan/prefect_chaos.py +93 -0
- litplan-0.0.1/src/litplan/prefect_node_policies.py +106 -0
- litplan-0.0.1/src/litplan/prefect_pipeline.py +311 -0
- litplan-0.0.1/src/litplan/run_status.py +299 -0
- litplan-0.0.1/src/litplan/store/__init__.py +65 -0
- litplan-0.0.1/src/litplan/store/base.py +9 -0
- litplan-0.0.1/src/litplan/store/engine.py +44 -0
- litplan-0.0.1/src/litplan/store/eval_provenance_annotations.py +93 -0
- litplan-0.0.1/src/litplan/store/models.py +124 -0
- litplan-0.0.1/src/litplan/store/node_checkpoint.py +112 -0
- litplan-0.0.1/src/litplan/store/plan_approval_records.py +91 -0
- litplan-0.0.1/src/litplan/store/repository.py +208 -0
- litplan-0.0.1/src/litplan/store/urls.py +122 -0
- litplan-0.0.1/src/litplan/telemetry.py +107 -0
- litplan-0.0.1/src/litplan/tool_contracts.py +129 -0
- litplan-0.0.1/src/litplan/tool_registry.py +262 -0
- litplan-0.0.1/src/litplan/tools.py +597 -0
- litplan-0.0.1/src/litplan/ui/__init__.py +1 -0
- litplan-0.0.1/src/litplan/ui/plan_draft_app.py +642 -0
- litplan-0.0.1/src/litplan/ui/plan_draft_ui.py +61 -0
- litplan-0.0.1/src/litplan/workflowgen/__init__.py +80 -0
- litplan-0.0.1/src/litplan/workflowgen/airflow_gen.py +309 -0
- litplan-0.0.1/src/litplan/workflowgen/default_kind_map.yaml +38 -0
- litplan-0.0.1/src/litplan/workflowgen/kind_map.py +224 -0
- litplan-0.0.1/src/litplan/workflowgen/nextflow_gen.py +504 -0
- litplan-0.0.1/src/litplan/workflowgen/normalized_dag.py +72 -0
- litplan-0.0.1/src/litplan/workflowgen/snakemake_gen.py +308 -0
- litplan-0.0.1/src/litplan/workflowgen/spec_json.py +47 -0
- litplan-0.0.1/src/litplan_cli/__init__.py +1 -0
- litplan-0.0.1/src/litplan_cli/main.py +521 -0
- litplan-0.0.1/src/litplan_mcp/__init__.py +5 -0
- litplan-0.0.1/src/litplan_mcp/__main__.py +6 -0
- litplan-0.0.1/src/litplan_mcp/server.py +246 -0
- litplan-0.0.1/tests/.gitkeep +0 -0
- litplan-0.0.1/tests/conftest.py +23 -0
- litplan-0.0.1/tests/golden/document_ir_structural_minimal.json +131 -0
- litplan-0.0.1/tests/golden/t8_compile_expected.json +59 -0
- litplan-0.0.1/tests/golden/t8_log_redaction_lines.json +44 -0
- litplan-0.0.1/tests/golden/t8_pipeline_spec_linear.json +46 -0
- litplan-0.0.1/tests/golden/t8_prefect_checkpoint_a.json +7 -0
- litplan-0.0.1/tests/golden/t8_prefect_checkpoint_b.json +8 -0
- litplan-0.0.1/tests/golden/t8_tool_args_invalid.json +30 -0
- litplan-0.0.1/tests/golden/t8_tool_args_valid.json +47 -0
- litplan-0.0.1/tests/integration/__init__.py +1 -0
- litplan-0.0.1/tests/integration/test_t214_chunk_e2e.py +355 -0
- litplan-0.0.1/tests/test_context_poisoning_fixtures.py +100 -0
- litplan-0.0.1/tests/test_experiment_recovery_targets.py +66 -0
- litplan-0.0.1/tests/test_export_workflow_mcp_cli_parity.py +380 -0
- litplan-0.0.1/tests/test_grobid_client.py +114 -0
- litplan-0.0.1/tests/test_http_mock_httpserver.py +110 -0
- litplan-0.0.1/tests/test_llm_provider_config.py +52 -0
- litplan-0.0.1/tests/test_phase3_exit_criteria.py +198 -0
- litplan-0.0.1/tests/test_phase4_exit_criteria.py +241 -0
- litplan-0.0.1/tests/test_phase5_exit_criteria.py +177 -0
- litplan-0.0.1/tests/test_phase6_exit_criteria.py +205 -0
- litplan-0.0.1/tests/test_phase7_mcp_cli_parity.py +610 -0
- litplan-0.0.1/tests/test_phase8_golden_path.py +214 -0
- litplan-0.0.1/tests/test_phase8_log_redaction.py +35 -0
- litplan-0.0.1/tests/test_pipeline_spec.py +78 -0
- litplan-0.0.1/tests/test_smoke.py +36 -0
- litplan-0.0.1/tests/test_t7_tool_contracts.py +174 -0
- litplan-0.0.1/tests/test_t810_plan_compile_mcp_cli_parity.py +325 -0
- litplan-0.0.1/tests/test_t812_grounded_plan_compile_experiments_stub.py +82 -0
- litplan-0.0.1/tests/test_t88_merge_semantics.py +288 -0
- litplan-0.0.1/tests/test_t8_otel_compile_execute_trace_tree.py +153 -0
- litplan-0.0.1/tests/test_tei_document_ir_golden.py +73 -0
- litplan-0.0.1/tests/test_workflowgen_airflow_golden.py +142 -0
- litplan-0.0.1/tests/test_workflowgen_nextflow_golden.py +118 -0
- litplan-0.0.1/tests/test_workflowgen_normalized_dag.py +102 -0
- litplan-0.0.1/tests/test_workflowgen_snakemake_golden.py +71 -0
- litplan-0.0.1/uv.lock +4321 -0
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
# Litplan — copy to `.env` locally (never commit `.env`).
|
|
2
|
+
# API keys and tokens MUST come from the environment (or your secret store),
|
|
3
|
+
# never from committed prompts, fixtures, or application source.
|
|
4
|
+
|
|
5
|
+
# --- LLM (provider-agnostic factory; see README "Configuration & secrets") ---
|
|
6
|
+
|
|
7
|
+
# Selects which adapter the factory uses. Code default when unset is `gemini` (needs GOOGLE_API_KEY).
|
|
8
|
+
# Supported: stub, gemini, openai, anthropic, grok (xAI OpenAI-compatible endpoint).
|
|
9
|
+
# Use `stub` in this file for offline copy-paste until you add a key.
|
|
10
|
+
LITPLAN_LLM_PROVIDER=stub
|
|
11
|
+
|
|
12
|
+
# Model id passed to the active provider. Leave empty to use the built-in default for that provider
|
|
13
|
+
# (e.g. Gemini: gemini-3.1-flash-lite-preview, OpenAI: gpt-4.1, Anthropic: claude-sonnet-4-20250514, Grok: grok-3-mini).
|
|
14
|
+
LITPLAN_LLM_MODEL=
|
|
15
|
+
|
|
16
|
+
# --- Keys (set only for the provider you use) ---
|
|
17
|
+
|
|
18
|
+
# gemini: Google AI Studio / Gemini API. Either name works (GOOGLE_API_KEY, else GEMINI_API_KEY).
|
|
19
|
+
# GOOGLE_API_KEY=
|
|
20
|
+
# GEMINI_API_KEY=
|
|
21
|
+
|
|
22
|
+
# openai: Chat Completions API (OPENAI_API_KEY).
|
|
23
|
+
# OPENAI_API_KEY=
|
|
24
|
+
|
|
25
|
+
# anthropic: Messages API (ANTHROPIC_API_KEY).
|
|
26
|
+
# ANTHROPIC_API_KEY=
|
|
27
|
+
|
|
28
|
+
# grok: xAI OpenAI-compatible API (XAI_API_KEY). Optional override for the base URL (default https://api.x.ai/v1).
|
|
29
|
+
# XAI_API_KEY=
|
|
30
|
+
# LITPLAN_XAI_BASE_URL=
|
|
31
|
+
|
|
32
|
+
# --- GROBID (PDF → TEI; run the official container, then point the app at it) ---
|
|
33
|
+
|
|
34
|
+
# Base URL only (no trailing path). Example: http://127.0.0.1:8070
|
|
35
|
+
# LITPLAN_GROBID_URL=http://127.0.0.1:8070
|
|
36
|
+
|
|
37
|
+
# Optional digest/tag of the GROBID image used for this parse (manifest / reproducibility).
|
|
38
|
+
# Pin matches docker-compose.grobid.yml (CRF-only; avoid *-full* unless you intend the ~8 GB image).
|
|
39
|
+
# LITPLAN_GROBID_IMAGE_REF=grobid/grobid:0.8.2.1-crf
|
|
40
|
+
|
|
41
|
+
# Connect vs read/write timeouts (seconds). Full-text can be slow on large PDFs.
|
|
42
|
+
# LITPLAN_GROBID_CONNECT_TIMEOUT_S=30
|
|
43
|
+
# LITPLAN_GROBID_TIMEOUT_S=300
|
|
44
|
+
|
|
45
|
+
# Retries for transient failures (timeouts, connection errors, 5xx). Total attempts = 1 + this value.
|
|
46
|
+
# LITPLAN_GROBID_MAX_RETRIES=2
|
|
47
|
+
|
|
48
|
+
# Base seconds for exponential backoff between retries (0 = immediate retry).
|
|
49
|
+
# LITPLAN_GROBID_RETRY_BACKOFF_BASE_S=1
|
|
50
|
+
|
|
51
|
+
# --- Chunking (DocumentIR → list[DocumentChunk]; see docs/DEVS_README.md) ---
|
|
52
|
+
|
|
53
|
+
# Max estimated tokens per chunk (rough count; default_token_count in chunking.py).
|
|
54
|
+
# LITPLAN_CHUNK_MAX_TOKENS=512
|
|
55
|
+
|
|
56
|
+
# Token overlap carried into the next chunk (default 0).
|
|
57
|
+
# LITPLAN_CHUNK_OVERLAP_TOKENS=0
|
|
58
|
+
|
|
59
|
+
# If true, do not pack segments from different sections into one chunk (default true).
|
|
60
|
+
# LITPLAN_CHUNK_RESPECT_SECTION_BOUNDARIES=true
|
|
61
|
+
|
|
62
|
+
# --- Prefect node chaos (T6.4; dev / simulation only) ---
|
|
63
|
+
|
|
64
|
+
# Integer seed enables simulated latency + random task failures (see README "Prefect execution & chaos").
|
|
65
|
+
# LITPLAN_CHAOS_SEED=1
|
|
66
|
+
|
|
67
|
+
# Optional: failure probability [0, 1], default 0.2
|
|
68
|
+
# LITPLAN_CHAOS_FAILURE_PROB=0.2
|
|
69
|
+
|
|
70
|
+
# Optional: max sleep ms before failure roll, default 250 (0 = no sleep)
|
|
71
|
+
# LITPLAN_CHAOS_LATENCY_MAX_MS=250
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
name: CI
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
branches: [main]
|
|
6
|
+
pull_request:
|
|
7
|
+
|
|
8
|
+
permissions:
|
|
9
|
+
contents: read
|
|
10
|
+
|
|
11
|
+
env:
|
|
12
|
+
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
|
|
13
|
+
|
|
14
|
+
jobs:
|
|
15
|
+
pytest:
|
|
16
|
+
name: Pytest
|
|
17
|
+
runs-on: ubuntu-latest
|
|
18
|
+
steps:
|
|
19
|
+
- uses: actions/checkout@v6
|
|
20
|
+
|
|
21
|
+
- name: Set up Python
|
|
22
|
+
uses: actions/setup-python@v6
|
|
23
|
+
with:
|
|
24
|
+
python-version: "3.11"
|
|
25
|
+
|
|
26
|
+
- name: Install uv
|
|
27
|
+
uses: astral-sh/setup-uv@v7
|
|
28
|
+
with:
|
|
29
|
+
enable-cache: true
|
|
30
|
+
cache-suffix: ci-pytest
|
|
31
|
+
|
|
32
|
+
- name: Sync project with dev dependencies
|
|
33
|
+
run: uv sync --frozen --group dev
|
|
34
|
+
|
|
35
|
+
- name: Run pytest
|
|
36
|
+
run: uv run pytest
|
|
37
|
+
|
|
38
|
+
build:
|
|
39
|
+
name: Build package
|
|
40
|
+
runs-on: ubuntu-latest
|
|
41
|
+
steps:
|
|
42
|
+
- uses: actions/checkout@v6
|
|
43
|
+
|
|
44
|
+
- name: Set up Python
|
|
45
|
+
uses: actions/setup-python@v6
|
|
46
|
+
with:
|
|
47
|
+
python-version: "3.11"
|
|
48
|
+
|
|
49
|
+
- name: Install uv
|
|
50
|
+
uses: astral-sh/setup-uv@v7
|
|
51
|
+
with:
|
|
52
|
+
enable-cache: true
|
|
53
|
+
cache-suffix: ci-build
|
|
54
|
+
|
|
55
|
+
- name: Build source and wheel distributions
|
|
56
|
+
run: uv build
|
|
57
|
+
|
|
58
|
+
migrations:
|
|
59
|
+
name: Alembic smoke
|
|
60
|
+
runs-on: ubuntu-latest
|
|
61
|
+
env:
|
|
62
|
+
LITPLAN_HOME: ${{ github.workspace }}/.litplan-home-ci
|
|
63
|
+
steps:
|
|
64
|
+
- uses: actions/checkout@v6
|
|
65
|
+
|
|
66
|
+
- name: Set up Python
|
|
67
|
+
uses: actions/setup-python@v6
|
|
68
|
+
with:
|
|
69
|
+
python-version: "3.11"
|
|
70
|
+
|
|
71
|
+
- name: Install uv
|
|
72
|
+
uses: astral-sh/setup-uv@v7
|
|
73
|
+
with:
|
|
74
|
+
enable-cache: true
|
|
75
|
+
cache-suffix: ci-migrations
|
|
76
|
+
|
|
77
|
+
- name: Sync project runtime dependencies
|
|
78
|
+
run: uv sync --frozen
|
|
79
|
+
|
|
80
|
+
- name: Upgrade database to head
|
|
81
|
+
run: uv run alembic upgrade head
|
|
82
|
+
|
|
83
|
+
- name: Verify SQLite database was created
|
|
84
|
+
run: test -f "${LITPLAN_HOME}/runs/litplan.db"
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# Publish to PyPI via Trusted Publishing (OIDC). Trigger: push a version tag only (e.g. v0.0.1).
|
|
2
|
+
# Configure the publisher on PyPI to match this repo, workflow file, and optional environment:
|
|
3
|
+
# https://docs.pypi.org/trusted-publishers/using-a-publisher/#github-actions
|
|
4
|
+
name: Publish to PyPI
|
|
5
|
+
|
|
6
|
+
on:
|
|
7
|
+
push:
|
|
8
|
+
tags:
|
|
9
|
+
- "v*"
|
|
10
|
+
|
|
11
|
+
permissions:
|
|
12
|
+
contents: read
|
|
13
|
+
|
|
14
|
+
jobs:
|
|
15
|
+
pypi-publish:
|
|
16
|
+
name: Upload release to PyPI
|
|
17
|
+
runs-on: ubuntu-latest
|
|
18
|
+
# Strongly recommended by PyPI; must match the "environment" in your PyPI trusted publisher config.
|
|
19
|
+
environment: pypi
|
|
20
|
+
permissions:
|
|
21
|
+
# Required for Trusted Publishing (OIDC).
|
|
22
|
+
id-token: write
|
|
23
|
+
|
|
24
|
+
steps:
|
|
25
|
+
- uses: actions/checkout@v6
|
|
26
|
+
|
|
27
|
+
- name: Set up Python
|
|
28
|
+
uses: actions/setup-python@v6
|
|
29
|
+
with:
|
|
30
|
+
python-version: "3.11"
|
|
31
|
+
|
|
32
|
+
- name: Install uv
|
|
33
|
+
uses: astral-sh/setup-uv@v7
|
|
34
|
+
|
|
35
|
+
- name: Build source and wheel distributions
|
|
36
|
+
run: uv build
|
|
37
|
+
|
|
38
|
+
- name: Publish package distributions to PyPI
|
|
39
|
+
uses: pypa/gh-action-pypi-publish@release/v1
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
name: Workflowgen engine integration
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
branches: [main]
|
|
6
|
+
pull_request:
|
|
7
|
+
workflow_dispatch:
|
|
8
|
+
|
|
9
|
+
permissions:
|
|
10
|
+
contents: read
|
|
11
|
+
|
|
12
|
+
jobs:
|
|
13
|
+
workflowgen:
|
|
14
|
+
runs-on: ubuntu-latest
|
|
15
|
+
strategy:
|
|
16
|
+
fail-fast: false
|
|
17
|
+
matrix:
|
|
18
|
+
engine: [snakemake, nextflow, airflow]
|
|
19
|
+
steps:
|
|
20
|
+
- uses: actions/checkout@v6
|
|
21
|
+
|
|
22
|
+
# Nextflow uses Java plus the upstream installer; other legs use the project env via uv.
|
|
23
|
+
- name: Install uv
|
|
24
|
+
if: matrix.engine != 'nextflow'
|
|
25
|
+
uses: astral-sh/setup-uv@v7
|
|
26
|
+
with:
|
|
27
|
+
enable-cache: true
|
|
28
|
+
cache-suffix: workflowgen-${{ matrix.engine }}
|
|
29
|
+
|
|
30
|
+
- name: Sync Python env (Snakemake only)
|
|
31
|
+
if: matrix.engine == 'snakemake'
|
|
32
|
+
run: uv sync --group dev
|
|
33
|
+
|
|
34
|
+
- name: Set up Java (Nextflow)
|
|
35
|
+
if: matrix.engine == 'nextflow'
|
|
36
|
+
uses: actions/setup-java@v5
|
|
37
|
+
with:
|
|
38
|
+
distribution: temurin
|
|
39
|
+
java-version: "17"
|
|
40
|
+
|
|
41
|
+
- name: Install Nextflow
|
|
42
|
+
if: matrix.engine == 'nextflow'
|
|
43
|
+
run: |
|
|
44
|
+
set -euo pipefail
|
|
45
|
+
mkdir -p "${HOME}/.local/bin"
|
|
46
|
+
curl -fsSL https://get.nextflow.io | bash
|
|
47
|
+
test -f nextflow
|
|
48
|
+
mv nextflow "${HOME}/.local/bin/nextflow"
|
|
49
|
+
chmod +x "${HOME}/.local/bin/nextflow"
|
|
50
|
+
echo "${HOME}/.local/bin" >> "${GITHUB_PATH}"
|
|
51
|
+
|
|
52
|
+
- name: Snakemake dry-run on golden bundles
|
|
53
|
+
if: matrix.engine == 'snakemake'
|
|
54
|
+
run: |
|
|
55
|
+
set -euo pipefail
|
|
56
|
+
ROOT="${GITHUB_WORKSPACE}"
|
|
57
|
+
export PROJECT_HOME="${ROOT}"
|
|
58
|
+
export RUN_ID="ci-workflowgen"
|
|
59
|
+
mkdir -p "${ROOT}/runs/${RUN_ID}/workspace"
|
|
60
|
+
|
|
61
|
+
for name in empty linear_chain diamond parallel_roots edges_unsorted_in_file artifact_chain_linear artifact_merge_diamond artifact_fork_selective; do
|
|
62
|
+
TMP="$(mktemp -d)"
|
|
63
|
+
cp "${ROOT}/fixtures/workflowgen/snakemake_golden/${name}.Snakefile" "${TMP}/Snakefile"
|
|
64
|
+
envsubst < "${ROOT}/fixtures/workflowgen/snakemake_golden/${name}.config.yaml" > "${TMP}/config.yaml"
|
|
65
|
+
echo "=== snakemake -n: ${name} ==="
|
|
66
|
+
cd "${ROOT}"
|
|
67
|
+
uv run snakemake -n --snakefile "${TMP}/Snakefile" --configfile "${TMP}/config.yaml"
|
|
68
|
+
done
|
|
69
|
+
|
|
70
|
+
- name: Nextflow config on golden bundles
|
|
71
|
+
if: matrix.engine == 'nextflow'
|
|
72
|
+
env:
|
|
73
|
+
GITHUB_WORKSPACE: ${{ github.workspace }}
|
|
74
|
+
run: |
|
|
75
|
+
set -euo pipefail
|
|
76
|
+
command -v nextflow >/dev/null 2>&1
|
|
77
|
+
ROOT="${GITHUB_WORKSPACE}"
|
|
78
|
+
for name in empty linear_chain diamond parallel_roots edges_unsorted_in_file; do
|
|
79
|
+
TMP="$(mktemp -d)"
|
|
80
|
+
cp "${ROOT}/fixtures/workflowgen/nextflow_golden/${name}.main.nf" "${TMP}/main.nf"
|
|
81
|
+
cp "${ROOT}/fixtures/workflowgen/nextflow_golden/${name}.nextflow.config" "${TMP}/nextflow.config"
|
|
82
|
+
echo "=== nextflow config: ${name} ==="
|
|
83
|
+
cd "${TMP}"
|
|
84
|
+
nextflow config .
|
|
85
|
+
done
|
|
86
|
+
|
|
87
|
+
- name: Airflow DagBag on golden DAG files
|
|
88
|
+
if: matrix.engine == 'airflow'
|
|
89
|
+
run: |
|
|
90
|
+
set -euo pipefail
|
|
91
|
+
ROOT="${GITHUB_WORKSPACE}"
|
|
92
|
+
for name in empty linear_chain diamond parallel_roots edges_unsorted_in_file; do
|
|
93
|
+
TMP="$(mktemp -d)"
|
|
94
|
+
cp "${ROOT}/fixtures/workflowgen/airflow_golden/${name}.litplan_dag.py" "${TMP}/litplan_dag.py"
|
|
95
|
+
echo "=== airflow DagBag: ${name} ==="
|
|
96
|
+
cd "${ROOT}"
|
|
97
|
+
uv run --with 'apache-airflow>=2.8,<3' python -c \
|
|
98
|
+
'import sys; from pathlib import Path; from airflow.models import DagBag; f=Path(sys.argv[1]); db=DagBag(dag_folder=str(f), include_examples=False); assert not db.import_errors, db.import_errors; assert db.dags, "no DAGs"; print("ok", len(db.dags))' \
|
|
99
|
+
"${TMP}"
|
|
100
|
+
done
|
litplan-0.0.1/.gitignore
ADDED
litplan-0.0.1/Dockerfile
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# Litplan — Python 3.11 + uv, dependencies from uv.lock.
|
|
2
|
+
# Add apt packages only if a wheel build fails for a dependency.
|
|
3
|
+
FROM python:3.11-slim-bookworm
|
|
4
|
+
|
|
5
|
+
RUN apt-get update \
|
|
6
|
+
&& apt-get install -y --no-install-recommends curl ca-certificates \
|
|
7
|
+
&& rm -rf /var/lib/apt/lists/*
|
|
8
|
+
|
|
9
|
+
ENV UV_LINK_MODE=copy
|
|
10
|
+
|
|
11
|
+
WORKDIR /app
|
|
12
|
+
|
|
13
|
+
RUN curl -LsSf https://astral.sh/uv/install.sh | sh
|
|
14
|
+
ENV PATH="/root/.local/bin:${PATH}"
|
|
15
|
+
|
|
16
|
+
COPY pyproject.toml uv.lock README.md ./
|
|
17
|
+
COPY src ./src
|
|
18
|
+
|
|
19
|
+
RUN uv sync --frozen --group dev
|
|
20
|
+
|
|
21
|
+
COPY tests ./tests
|
|
22
|
+
|
|
23
|
+
CMD ["/bin/bash"]
|
litplan-0.0.1/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 litplan
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
litplan-0.0.1/PKG-INFO
ADDED
|
@@ -0,0 +1,264 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: litplan
|
|
3
|
+
Version: 0.0.1
|
|
4
|
+
Summary: Litplan: ingest papers, compile pipeline IR, run multi-agent workflows.
|
|
5
|
+
License: MIT License
|
|
6
|
+
|
|
7
|
+
Copyright (c) 2026 litplan
|
|
8
|
+
|
|
9
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
10
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
11
|
+
in the Software without restriction, including without limitation the rights
|
|
12
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
13
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
14
|
+
furnished to do so, subject to the following conditions:
|
|
15
|
+
|
|
16
|
+
The above copyright notice and this permission notice shall be included in all
|
|
17
|
+
copies or substantial portions of the Software.
|
|
18
|
+
|
|
19
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
20
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
21
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
22
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
23
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
24
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
25
|
+
SOFTWARE.
|
|
26
|
+
License-File: LICENSE
|
|
27
|
+
Requires-Python: >=3.11
|
|
28
|
+
Requires-Dist: alembic>=1.18.4
|
|
29
|
+
Requires-Dist: click>=8.1.0
|
|
30
|
+
Requires-Dist: httpx>=0.28.1
|
|
31
|
+
Requires-Dist: jsonschema>=4.23.0
|
|
32
|
+
Requires-Dist: langchain-anthropic>=0.3.0
|
|
33
|
+
Requires-Dist: langchain-core>=0.3.0
|
|
34
|
+
Requires-Dist: langchain-google-genai>=2.0.0
|
|
35
|
+
Requires-Dist: langchain-openai>=0.3.0
|
|
36
|
+
Requires-Dist: langgraph-checkpoint-sqlite>=3.0.3
|
|
37
|
+
Requires-Dist: langgraph>=1.1.3
|
|
38
|
+
Requires-Dist: lxml>=6.0.2
|
|
39
|
+
Requires-Dist: mcp>=1.2.0
|
|
40
|
+
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.40.0
|
|
41
|
+
Requires-Dist: opentelemetry-sdk>=1.40.0
|
|
42
|
+
Requires-Dist: prefect>=3.0
|
|
43
|
+
Requires-Dist: pydantic-settings>=2.13.1
|
|
44
|
+
Requires-Dist: pydantic>=2.12.5
|
|
45
|
+
Requires-Dist: pyyaml>=6.0
|
|
46
|
+
Requires-Dist: sqlalchemy>=2.0.49
|
|
47
|
+
Requires-Dist: sqlite-vec>=0.1.6
|
|
48
|
+
Description-Content-Type: text/markdown
|
|
49
|
+
|
|
50
|
+
# Litplan
|
|
51
|
+
|
|
52
|
+
Litplan turns a research paper into structured document data, a checked and versioned `PipelineSpec`, optional approval records, and either a reference execution path or workflow files for your own runtime. It is the governed-plan layer, not a replacement for Snakemake, Nextflow, Airflow, or your HPC stack.
|
|
53
|
+
|
|
54
|
+
## Start Here
|
|
55
|
+
|
|
56
|
+
### Prerequisites
|
|
57
|
+
|
|
58
|
+
- Python 3.11+
|
|
59
|
+
- [uv](https://docs.astral.sh/uv/)
|
|
60
|
+
- Docker, only if you want live PDF ingest via GROBID
|
|
61
|
+
- An LLM API key, only if you want live goal-driven planning
|
|
62
|
+
|
|
63
|
+
Run these commands from the repository root:
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
uv sync
|
|
67
|
+
cp .env.example .env
|
|
68
|
+
uv run litplan environment
|
|
69
|
+
uv run litplan ingest fixtures/tei/structural_minimal.tei.xml \
|
|
70
|
+
--parser-route tei_fixture_replay \
|
|
71
|
+
--document-ir
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
What to expect:
|
|
75
|
+
|
|
76
|
+
- `uv sync` installs the package and CLI from `uv.lock`.
|
|
77
|
+
- `cp .env.example .env` gives you a local config file; it defaults to `LITPLAN_LLM_PROVIDER=stub`, so this path stays offline.
|
|
78
|
+
- `uv run litplan environment` prints JSON hints about paths and config resolution.
|
|
79
|
+
- The offline `ingest` command should return JSON with `"ok": true` and inline `DocumentIR` output.
|
|
80
|
+
|
|
81
|
+
## Choose Your Path
|
|
82
|
+
|
|
83
|
+
|
|
84
|
+
| Path | Use it when | Needs | First step |
|
|
85
|
+
| --------------- | ---------------------------------------------------------------- | ------------------------------------------------- | ------------------------------------------------------------------- |
|
|
86
|
+
| Offline demo | You want a first successful run with no external services | `uv` | Run the `tei_fixture_replay` ingest command above |
|
|
87
|
+
| Live PDF ingest | You want to parse a real PDF through GROBID | Docker + GROBID | `docker compose -f docker-compose.grobid.yml up -d` |
|
|
88
|
+
| LLM planning | You want `plan-compile` to draft a pipeline from a goal | A real provider in `.env` plus a matching API key | Set `LITPLAN_LLM_PROVIDER` and provider key in `.env` |
|
|
89
|
+
| Workflow export | You want Snakemake, Nextflow, or Airflow files from a stored run | A persisted run in the local DB | Compile with `--create-run`, then `approve`, then `export-workflow` |
|
|
90
|
+
|
|
91
|
+
|
|
92
|
+
## Fastest Successful Runs
|
|
93
|
+
|
|
94
|
+
### 1. Offline ingest demo
|
|
95
|
+
|
|
96
|
+
This is the fastest self-contained path and works in a fresh checkout after `uv sync`.
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
uv run litplan ingest fixtures/tei/structural_minimal.tei.xml \
|
|
100
|
+
--parser-route tei_fixture_replay \
|
|
101
|
+
--document-ir
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
Expected outcome:
|
|
105
|
+
|
|
106
|
+
- You should get JSON output with `"ok": true`.
|
|
107
|
+
- You should get inline `DocumentIR` JSON.
|
|
108
|
+
- No Docker, database, or API key is required.
|
|
109
|
+
|
|
110
|
+
### 2. Persist a local run
|
|
111
|
+
|
|
112
|
+
Use this when you want approvals, execution, status, or export commands to work against the local SQLite store.
|
|
113
|
+
|
|
114
|
+
```bash
|
|
115
|
+
export LITPLAN_HOME="$(pwd)/.litplan-home"
|
|
116
|
+
uv run alembic upgrade head
|
|
117
|
+
RUN_ID="examples-linear-chain-run-$(date +%s)"
|
|
118
|
+
uv run litplan compile fixtures/workflowgen/pipeline_specs/linear_chain.json \
|
|
119
|
+
--pipeline-id examples.linear_chain \
|
|
120
|
+
--lockfile uv.lock \
|
|
121
|
+
--run-id "$RUN_ID" \
|
|
122
|
+
--create-run
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Expected outcome:
|
|
126
|
+
|
|
127
|
+
- This creates `.litplan-home/` and initializes the SQLite schema there.
|
|
128
|
+
- The compile command should return `"ok": true`.
|
|
129
|
+
- The run and compiled revision are now persisted for later `approve`, `run-status`, `execute`, and export commands.
|
|
130
|
+
|
|
131
|
+
### 3. Live PDF ingest with GROBID
|
|
132
|
+
|
|
133
|
+
If PDF ingest is the feature you care about, this is the shortest working setup.
|
|
134
|
+
|
|
135
|
+
Start GROBID:
|
|
136
|
+
|
|
137
|
+
```bash
|
|
138
|
+
docker compose -f docker-compose.grobid.yml up -d
|
|
139
|
+
curl -fsSL http://127.0.0.1:8070/api/isalive
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
Then set `LITPLAN_GROBID_URL=http://127.0.0.1:8070` in `.env` and run:
|
|
143
|
+
|
|
144
|
+
```bash
|
|
145
|
+
uv run litplan ingest \
|
|
146
|
+
fixtures/papers/pdf/batatia-2022-mace-force-fields-arxiv-2206.07697v2.pdf \
|
|
147
|
+
--document-ir
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
Expected outcome:
|
|
151
|
+
|
|
152
|
+
- The health check should print an `alive` response from GROBID.
|
|
153
|
+
- The ingest command should return `"ok": true`.
|
|
154
|
+
- You should get parsed document metadata and, with `--document-ir`, inline `DocumentIR` JSON for the PDF.
|
|
155
|
+
|
|
156
|
+
### 4. Live LLM planning
|
|
157
|
+
|
|
158
|
+
For this path, change `.env` from the default `stub` provider to a real provider and add its key. Example:
|
|
159
|
+
|
|
160
|
+
```bash
|
|
161
|
+
LITPLAN_LLM_PROVIDER=gemini
|
|
162
|
+
GOOGLE_API_KEY=your-real-key
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
Then run:
|
|
166
|
+
|
|
167
|
+
```bash
|
|
168
|
+
uv run litplan plan-compile "build linear noop pipeline" \
|
|
169
|
+
--pipeline-id examples.goal_demo \
|
|
170
|
+
--lockfile uv.lock \
|
|
171
|
+
--expanded-spec
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
Expected outcome:
|
|
175
|
+
|
|
176
|
+
- The command should return `"ok": true` when the provider is configured correctly.
|
|
177
|
+
- You should get a compiled pipeline result and expanded spec JSON.
|
|
178
|
+
- Use `compile` instead of `plan-compile` when you already have full `PipelineSpec` JSON and do not want an LLM involved.
|
|
179
|
+
|
|
180
|
+
### 5. Workflow export
|
|
181
|
+
|
|
182
|
+
Once you have a persisted run, you can approve it and generate engine-native workflow files.
|
|
183
|
+
|
|
184
|
+
```bash
|
|
185
|
+
uv run litplan approve "$RUN_ID"
|
|
186
|
+
uv run litplan export-workflow "$RUN_ID" snakemake ./tmp/workflow-out
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
Expected outcome:
|
|
190
|
+
|
|
191
|
+
- `approve` records a non-pending run status for `RUN_ID`.
|
|
192
|
+
- `export-workflow` writes files under `./tmp/workflow-out`.
|
|
193
|
+
- You can switch `snakemake` to `nextflow` or `airflow`.
|
|
194
|
+
|
|
195
|
+
## Main Commands
|
|
196
|
+
|
|
197
|
+
|
|
198
|
+
| Command | What it does |
|
|
199
|
+
| ----------------------------- | ------------------------------------------------------------------------- |
|
|
200
|
+
| `litplan ingest` | Parse a PDF or TEI file into `DocumentIR`; can include inline JSON output |
|
|
201
|
+
| `litplan compile` | Validate and compile a `PipelineSpec` JSON input |
|
|
202
|
+
| `litplan plan-compile` | Use a goal-driven planner, then run the same compiler |
|
|
203
|
+
| `litplan approve` | Record approval or other status transitions for a run |
|
|
204
|
+
| `litplan execute` | Execute a persisted approved run through the reference Prefect path |
|
|
205
|
+
| `litplan run-status` | Show unified run, planning, and execution status |
|
|
206
|
+
| `litplan export-audit-bundle` | Write a reproducibility-oriented bundle for a run |
|
|
207
|
+
| `litplan export-workflow` | Generate Snakemake, Nextflow, or Airflow files from a stored revision |
|
|
208
|
+
| `litplan environment` | Print non-secret environment and path hints |
|
|
209
|
+
|
|
210
|
+
|
|
211
|
+
Run `uv run litplan --help` for the full CLI and option details.
|
|
212
|
+
|
|
213
|
+
## Project Surfaces
|
|
214
|
+
|
|
215
|
+
- CLI: `uv run litplan`
|
|
216
|
+
- MCP server: `uv run litplan-mcp`
|
|
217
|
+
- Python library: `import litplan`
|
|
218
|
+
- Streamlit UI (optional): `uv run streamlit run src/litplan/ui/plan_draft_app.py`
|
|
219
|
+
|
|
220
|
+
The CLI and MCP server call the same shared JSON tool layer, so they expose the same core operations.
|
|
221
|
+
|
|
222
|
+
### Streamlit UI
|
|
223
|
+
|
|
224
|
+
The optional Streamlit app is a browser UI on top of the same SQLite project database. You can edit **PlanDraft** text and compare it to a prior draft, append **approval** records, append **hallucination** flags with chunk provenance, and load **unified run status** (LangGraph checkpoints, run timeline, execute-phase node checkpoints, OpenTelemetry hints). Ingest, compile, plan-compile, approve, execute, export, and other pipeline steps still run from the CLI or MCP; the UI does not replace those tools.
|
|
225
|
+
|
|
226
|
+

|
|
227
|
+
|
|
228
|
+
From the repository root, after `uv sync` and `uv run alembic upgrade head` for your chosen `LITPLAN_HOME`:
|
|
229
|
+
|
|
230
|
+
```bash
|
|
231
|
+
export LITPLAN_HOME="$(pwd)/.litplan-home"
|
|
232
|
+
uv run streamlit run src/litplan/ui/plan_draft_app.py
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
For a full step-by-step (including diff panels, approvals, flags, and run explorer), see **Part 3** in [MANUAL_CLI_MCP_UI_WORKFLOW.md](docs/manual/MANUAL_CLI_MCP_UI_WORKFLOW.md).
|
|
236
|
+
|
|
237
|
+
## Examples
|
|
238
|
+
|
|
239
|
+
Library-first examples live in `examples/`:
|
|
240
|
+
|
|
241
|
+
- `examples/paper_repro_sketch_tei.py`: saved TEI -> `DocumentIR` -> chunking -> hand-authored `PipelineSpec` -> compile
|
|
242
|
+
- `examples/paper_repro_sketch_pdf.py`: PDF path -> live GROBID ingest -> the same chunk/plan/compile flow
|
|
243
|
+
|
|
244
|
+
Run them from the repository root:
|
|
245
|
+
|
|
246
|
+
```bash
|
|
247
|
+
uv run python examples/paper_repro_sketch_tei.py
|
|
248
|
+
uv run python examples/paper_repro_sketch_pdf.py \
|
|
249
|
+
fixtures/papers/pdf/batatia-2022-mace-force-fields-arxiv-2206.07697v2.pdf
|
|
250
|
+
```
|
|
251
|
+
|
|
252
|
+
## What To Expect
|
|
253
|
+
|
|
254
|
+
- The package version is **0.0.1** (see `pyproject.toml`); pre-1.0 semver releases may still evolve public APIs.
|
|
255
|
+
- Offline and stub-backed flows remain supported for local development and tests.
|
|
256
|
+
- The bundled `execute` path demonstrates orchestration, retries, checkpoints, and bookkeeping; use your own workflow engine or cluster stack for production-scale execution.
|
|
257
|
+
|
|
258
|
+
## Where To Go Next
|
|
259
|
+
|
|
260
|
+
- [Product scenarios, no-technical overview](docs/PRODUCT_SCENARIOS.md)
|
|
261
|
+
- [Full configuration, env vars, architecture, and command reference](docs/DEVS_README.md)
|
|
262
|
+
- [End-to-end operator walkthrough across CLI, MCP, and UI](docs/manual/MANUAL_CLI_MCP_UI_WORKFLOW.md)
|
|
263
|
+
- [Workflow export details, kind maps, and safety model](docs/workflowgen/)
|
|
264
|
+
- [Paper fixture notes](fixtures/papers/FIXTURES.md)
|