calkit-python 0.47.7__tar.gz → 0.47.9__tar.gz

{calkit_python-0.47.7 → calkit_python-0.47.9}/.pre-commit-config.yaml

@@ -43,6 +43,38 @@ repos:
         pass_filenames: false
         files: \.pyi?$
 
+  # Generated docs are regenerated here rather than only by 'make format',
+  # so a change to what they are generated from can't be committed while
+  # they still describe the old behavior. Each runs only when something it
+  # reads has changed, and they run in this order because the README
+  # embeds docs that the earlier ones write.
+  - repo: local
+    hooks:
+      - id: generate-schema
+        name: generate calkit.yaml schema
+        language: system
+        entry: bash -c 'uv run calkit describe schema -o docs/schemas/calkit.json && cp docs/schemas/calkit.json vscode-ext/schemas/calkit.json'
+        pass_filenames: false
+        files: ^calkit/models/.*\.py$
+      - id: generate-docs-references
+        name: generate docs references
+        language: system
+        entry: bash -c 'uv run python scripts/generate-docs-references.py'
+        pass_filenames: false
+        files: ^(calkit/.*\.py|scripts/generate-docs-references\.py)$
+      - id: sync-docs
+        name: sync docs into README
+        language: system
+        entry: bash -c 'uv run python scripts/sync-docs.py'
+        pass_filenames: false
+        files: ^(docs/.*\.md|README\.md|scripts/sync-docs\.py)$
+      - id: sync-resources
+        name: sync project resources
+        language: system
+        entry: bash -c 'uv run python scripts/sync-resources.py'
+        pass_filenames: false
+        files: ^(calkit/resources/.*|actions/run/.*|scripts/sync-resources\.py)$
+
   - repo: https://github.com/pre-commit/mirrors-prettier
     rev: "v3.0.3"
     hooks:

{calkit_python-0.47.7 → calkit_python-0.47.9}/Makefile

@@ -24,7 +24,7 @@ frontend-client: ## Regenerate the hub frontend's API client.
 	@$(MAKE) -C hub/frontend client
 
 .PHONY: format
-format: sync-docs sync-resources ## Automatically format files.
+format: ## Automatically format files and regenerate what's generated.
 	@echo "🚀 Linting code with pre-commit"
 	@uv run pre-commit run -a
 

{calkit_python-0.47.7 → calkit_python-0.47.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: calkit-python
-Version: 0.47.7
+Version: 0.47.9
 Summary: Reproducibility simplified.
 Project-URL: Homepage, https://calkit.org
 Project-URL: Issues, https://github.com/calkit/calkit/issues
@@ -214,6 +214,14 @@ which is a critical feature to have when using generative AI.
   Simply call `calkit run`.
   Compose your pipeline from many different kinds of stages,
   including simple scripts, commands, Jupyter Notebooks, LaTeX, and more.
+- Tools to automatically build a reproducible pipeline from work you're
+  already doing.
+  Run a script, notebook, or LaTeX document with
+  [`calkit xr`](https://docs.calkit.org/pipeline#automatic-stage-and-environment-detection)
+  and it will be added as a pipeline stage,
+  with its environment, inputs, and outputs detected automatically.
+  This way, an existing project can be made reproducible
+  one step at a time.
 - A complementary self-hostable and GitHub-integrated
   [hub](https://github.com/calkit/calkit/tree/main/hub)
   web app to facilitate backup, collaboration,
@@ -357,7 +365,7 @@ This re-execs the CLI via `uvx --from calkit-python@<version> calkit`,
 so it requires [uv](https://docs.astral.sh/uv/) on `PATH`.
 You can also declare a minimum version in `calkit.yaml`;
 see
-[Pinning the Calkit CLI version](https://docs.calkit.org/requirements.md#pinning-the-calkit-cli-version).
+[Pinning the Calkit CLI version](https://docs.calkit.org/requirements#pinning-the-calkit-cli-version).
 
 ### Calkit Assistant
 
@@ -382,102 +390,226 @@ Download the executable from the latest
 > `ck` is an abbreviated alias for the `calkit` executable.
 > All `calkit` commands can be run as `ck` instead, e.g., `ck save -am "..."`.
 
-### From an existing project
+Research projects exist to answer questions.
+They involve collecting data,
+analyzing it,
+creating figures and calculating numbers,
+and finally writing about the answers and why they make sense.
+Here we'll see what's involved in setting that up.
 
-If you want to use Calkit with an existing project,
-navigate into its working directory and use the `xr` command to start
-executing and recording your scripts, notebooks, LaTeX files, etc.,
-as reproducible pipeline stages.
-For example:
+### Create a project
+
+First, create a project from a template:
 
 ```sh
-calkit xr scripts/analyze.py
+calkit new project phd \
+    --title "PhD research" \
+    --template calkit/example-basic
+```
+
+The title can be changed later, but the name is more difficult to change,
+so it's a good idea to keep the name general.
+A single project can hold multiple investigations,
+e.g., a grad student might use one project for their entire PhD,
+with a question and a paper for each study.
+
+Add the `--hub` flag to create the project on
+[a Calkit hub](https://docs.calkit.org/hub) for backup and sharing.
+This requires an account,
+but can be set up later,
+so we'll leave it off for now.
+
+### Look at the question
+
+Open up `calkit.yaml`.
+Near the top you'll see the project's research question:
+
+```yaml
+questions:
+  - question: How does the system respond to increasing $x$?
+    hypothesis: The value of $y$ increases linearly with $x$.
+    answer: $y$ increases quadratically with $x$, not linearly
+      ($R^2 = {r2:.3f}$ for the quadratic fit).
+    evidence:
+      - kind: figure
+        path: figures/x-vs-y.png
+      - kind: value
+        path: results/summary.json
+        key: r_squared_quadratic
+        name: r2
+```
 
-calkit xr notebooks/plot.ipynb
+The answer lists the evidence that supports it,
+and `{r2}` is read from one of those files rather than typed in by hand.
+We can see the rendered answer with:
 
-calkit xr paper/main.tex
+```sh
+calkit list questions
 ```
 
-Calkit will attempt to detect environments, inputs, and outputs and
-save them in `calkit.yaml`.
-If successful,
-you'll be able to run the full pipeline with:
+```
+1. question: How does the system respond to increasing $x$?
+    hypothesis: The value of $y$ increases linearly with $x$.
+    answer: $y$ increases quadratically with $x$, not linearly ($R^2 = 0.985$ for the quadratic fit).
+    evidence:
+      - kind: figure
+        path: figures/x-vs-y.png
+      - kind: value
+        path: results/summary.json
+        key: r_squared_quadratic
+        name: r2
+```
+
+The evidence files are produced by the pipeline,
+so if the data changes,
+the figure and number become stale,
+and so does the answer that depends on them.
+
+We recommend writing questions down first in your own projects as well.
+It makes clear what evidence you need to produce,
+and everything else in the project exists to produce it.
+
+### Run the pipeline
 
 ```sh
+cd phd
 calkit run
 ```
 
-Next, make a change to e.g., a script and look at the output of
-`calkit status`.
-You'll see that the pipeline has a stage that is out-of-date:
+```
+💻 Getting system information
+🔗 Checking system-level requirements
+📦 Checking environments
+🔀 Compiling DVC pipeline
+Running stage 'collect-data':
+Running stage 'analyze':
+Running stage 'figs-to-paper':
+Running stage 'results-to-tex':
+Running stage 'build-paper':
+Pipeline completed successfully ✅
+```
+
+The pipeline has five stages:
+collect the data,
+analyze it to produce a figure and a results file,
+copy the figure into the paper folder,
+convert the results to LaTeX,
+and compile the paper.
+You should now have a `paper/main.pdf` file.
+
+Calkit created the Python environment and pulled the LaTeX Docker image
+automatically.
+If something it needs isn't installed, e.g., Docker,
+it will show you the command to install it and offer to run it:
+
+```
+App 'docker' is not installed.
+  brew install --cask docker
+Run this to install 'docker'? [Y/n]
+```
+
+You can also run this check on its own with `calkit check reqs`.
+
+### Make a change
+
+Next, make an edit to `scripts/analyze.py` and check the project status:
 
 ```sh
+calkit status
+```
+
+```
+--------------------------- Questions ----------------------------
+1 question, 1 with stale evidence
+Run 'calkit check questions' for detail.
+
 ---------------------------- Pipeline ----------------------------
 Stale stages:
         analyze:
           stale outputs:
-            data/results.csv
+            figures/x-vs-y.png
+            results/summary.json
           modified inputs:
             scripts/analyze.py
 ```
 
-This can be fixed with another call to `calkit run`.
-
-You can save (add and commit) all changes with:
+The figure and results file are stale because the script that produces
+them changed,
+and the answer is stale because its evidence is.
+We can see more detail with:
 
 ```sh
-calkit save -am "Add to pipeline"
+calkit check questions
 ```
 
-### Fresh from a Calkit project template
-
-Create a new project from the
-[`calkit/example-basic`](https://github.com/calkit/example-basic)
-template with:
-
-```sh
-calkit new project my-research \
-    --title "My research" \
-    --template calkit/example-basic \
-    --hub
 ```
+1. [stale] How does the system respond to increasing $x$?
+     figure figures/x-vs-y.png [stale] -- stage 'analyze' is out of date; run the pipeline
+     value results/summary.json:r_squared_quadratic [stale] -- stage 'analyze' is out of date; run the pipeline
 
-Note the `--hub` flag requires [a hub connection](https://docs.calkit.org/hub)
-to be set up, but can be omitted if the project doesn't need to be backed up to
-the hub or shared with collaborators.
-Hub integration can also be set up later.
-
-Next, move into the project folder and run the pipeline,
-which consists of several stages defined in `calkit.yaml`:
+Questions answered: 1/1
+Answers backed by current evidence: 0/1 ❌
+```
 
-<!-- TODO: This takes a long time to pull the image -->
+So, editing the script made the project's answer stale,
+and we were able to verify this from the CLI.
+To bring everything back up to date, run the pipeline again:
 
 ```sh
-cd my-research
 calkit run
 ```
 
-Next, make some edits to a script or LaTeX file and run `calkit status` to
-see what stages are out-of-date.
-For example:
+Only the stages affected by the change will rerun.
+The paper is rebuilt with the new figure and numbers,
+and the answer is backed by current evidence again.
+This edit-and-run loop is the main way of working in a Calkit project,
+and it keeps the paper in sync with the analysis.
+
+### Save the project
 
 ```sh
----------------------------- Pipeline ----------------------------
-Stale stages:
-        build-paper:
-          stale outputs:
-            paper/paper.pdf
-          modified inputs:
-            paper/paper.tex
+calkit save -am "Tweak the fit"
 ```
 
-Execute `calkit run` again to bring everything up-to-date.
+This adds and commits all changes in one step,
+putting code in Git and data and outputs in DVC,
+so you don't need to decide which goes where.
 
-To back up or save the project, call:
+At this point the project only exists on your machine,
+so there's nowhere to push to.
+Calkit will offer to set that up:
 
-```sh
-calkit save -am "Run pipeline"
 ```
+This project isn't connected to a hub, so there's nowhere to push its
+code and data.
+Connect it now? [Y/n]
+```
+
+Answering yes creates the project on a [hub](https://docs.calkit.org/hub),
+which backs it up,
+stores data and outputs that are too big for Git,
+and makes it available to collaborators,
+if you choose to add them.
+After that, `calkit save` will push everything to the right place,
+and a collaborator can clone the project and reproduce it with
+`calkit run`.
+
+Answering no is fine too.
+The commit has already been made,
+and the project can be connected later with `calkit update hub`.
+
+### Next steps
+
+Add your own data and a script to process it with
+[`calkit xr`](https://docs.calkit.org/pipeline),
+which executes a command and records it as a pipeline stage.
+To use Calkit with an existing project rather than a template, see
+[the tutorial on existing projects](https://docs.calkit.org/tutorials/existing-project).
+
+As the research goes on, add more questions.
+Each entry in `questions` lists its own evidence,
+so a new study can be a new question and the stages that answer it,
+rather than a new project.
 
 ### With an AI coding agent
 
@@ -487,7 +619,7 @@ Simply tell the [AI agent](https://docs.calkit.org/ai-tools):
 
 or
 
-> Create me a new Calkit project for investigating...
+> Create me a new Calkit project that answers the question: ...
 
 <!-- END INCLUDE -->
 

{calkit_python-0.47.7 → calkit_python-0.47.9}/README.md

@@ -152,6 +152,14 @@ which is a critical feature to have when using generative AI.
   Simply call `calkit run`.
   Compose your pipeline from many different kinds of stages,
   including simple scripts, commands, Jupyter Notebooks, LaTeX, and more.
+- Tools to automatically build a reproducible pipeline from work you're
+  already doing.
+  Run a script, notebook, or LaTeX document with
+  [`calkit xr`](https://docs.calkit.org/pipeline#automatic-stage-and-environment-detection)
+  and it will be added as a pipeline stage,
+  with its environment, inputs, and outputs detected automatically.
+  This way, an existing project can be made reproducible
+  one step at a time.
 - A complementary self-hostable and GitHub-integrated
   [hub](https://github.com/calkit/calkit/tree/main/hub)
   web app to facilitate backup, collaboration,
@@ -295,7 +303,7 @@ This re-execs the CLI via `uvx --from calkit-python@<version> calkit`,
 so it requires [uv](https://docs.astral.sh/uv/) on `PATH`.
 You can also declare a minimum version in `calkit.yaml`;
 see
-[Pinning the Calkit CLI version](https://docs.calkit.org/requirements.md#pinning-the-calkit-cli-version).
+[Pinning the Calkit CLI version](https://docs.calkit.org/requirements#pinning-the-calkit-cli-version).
 
 ### Calkit Assistant
 
@@ -320,102 +328,226 @@ Download the executable from the latest
 > `ck` is an abbreviated alias for the `calkit` executable.
 > All `calkit` commands can be run as `ck` instead, e.g., `ck save -am "..."`.
 
-### From an existing project
+Research projects exist to answer questions.
+They involve collecting data,
+analyzing it,
+creating figures and calculating numbers,
+and finally writing about the answers and why they make sense.
+Here we'll see what's involved in setting that up.
 
-If you want to use Calkit with an existing project,
-navigate into its working directory and use the `xr` command to start
-executing and recording your scripts, notebooks, LaTeX files, etc.,
-as reproducible pipeline stages.
-For example:
+### Create a project
+
+First, create a project from a template:
 
 ```sh
-calkit xr scripts/analyze.py
+calkit new project phd \
+    --title "PhD research" \
+    --template calkit/example-basic
+```
+
+The title can be changed later, but the name is more difficult to change,
+so it's a good idea to keep the name general.
+A single project can hold multiple investigations,
+e.g., a grad student might use one project for their entire PhD,
+with a question and a paper for each study.
+
+Add the `--hub` flag to create the project on
+[a Calkit hub](https://docs.calkit.org/hub) for backup and sharing.
+This requires an account,
+but can be set up later,
+so we'll leave it off for now.
+
+### Look at the question
+
+Open up `calkit.yaml`.
+Near the top you'll see the project's research question:
+
+```yaml
+questions:
+  - question: How does the system respond to increasing $x$?
+    hypothesis: The value of $y$ increases linearly with $x$.
+    answer: $y$ increases quadratically with $x$, not linearly
+      ($R^2 = {r2:.3f}$ for the quadratic fit).
+    evidence:
+      - kind: figure
+        path: figures/x-vs-y.png
+      - kind: value
+        path: results/summary.json
+        key: r_squared_quadratic
+        name: r2
+```
 
-calkit xr notebooks/plot.ipynb
+The answer lists the evidence that supports it,
+and `{r2}` is read from one of those files rather than typed in by hand.
+We can see the rendered answer with:
 
-calkit xr paper/main.tex
+```sh
+calkit list questions
 ```
 
-Calkit will attempt to detect environments, inputs, and outputs and
-save them in `calkit.yaml`.
-If successful,
-you'll be able to run the full pipeline with:
+```
+1. question: How does the system respond to increasing $x$?
+    hypothesis: The value of $y$ increases linearly with $x$.
+    answer: $y$ increases quadratically with $x$, not linearly ($R^2 = 0.985$ for the quadratic fit).
+    evidence:
+      - kind: figure
+        path: figures/x-vs-y.png
+      - kind: value
+        path: results/summary.json
+        key: r_squared_quadratic
+        name: r2
+```
+
+The evidence files are produced by the pipeline,
+so if the data changes,
+the figure and number become stale,
+and so does the answer that depends on them.
+
+We recommend writing questions down first in your own projects as well.
+It makes clear what evidence you need to produce,
+and everything else in the project exists to produce it.
+
+### Run the pipeline
 
 ```sh
+cd phd
 calkit run
 ```
 
-Next, make a change to e.g., a script and look at the output of
-`calkit status`.
-You'll see that the pipeline has a stage that is out-of-date:
+```
+💻 Getting system information
+🔗 Checking system-level requirements
+📦 Checking environments
+🔀 Compiling DVC pipeline
+Running stage 'collect-data':
+Running stage 'analyze':
+Running stage 'figs-to-paper':
+Running stage 'results-to-tex':
+Running stage 'build-paper':
+Pipeline completed successfully ✅
+```
+
+The pipeline has five stages:
+collect the data,
+analyze it to produce a figure and a results file,
+copy the figure into the paper folder,
+convert the results to LaTeX,
+and compile the paper.
+You should now have a `paper/main.pdf` file.
+
+Calkit created the Python environment and pulled the LaTeX Docker image
+automatically.
+If something it needs isn't installed, e.g., Docker,
+it will show you the command to install it and offer to run it:
+
+```
+App 'docker' is not installed.
+  brew install --cask docker
+Run this to install 'docker'? [Y/n]
+```
+
+You can also run this check on its own with `calkit check reqs`.
+
+### Make a change
+
+Next, make an edit to `scripts/analyze.py` and check the project status:
 
 ```sh
+calkit status
+```
+
+```
+--------------------------- Questions ----------------------------
+1 question, 1 with stale evidence
+Run 'calkit check questions' for detail.
+
 ---------------------------- Pipeline ----------------------------
 Stale stages:
         analyze:
           stale outputs:
-            data/results.csv
+            figures/x-vs-y.png
+            results/summary.json
           modified inputs:
             scripts/analyze.py
 ```
 
-This can be fixed with another call to `calkit run`.
-
-You can save (add and commit) all changes with:
+The figure and results file are stale because the script that produces
+them changed,
+and the answer is stale because its evidence is.
+We can see more detail with:
 
 ```sh
-calkit save -am "Add to pipeline"
+calkit check questions
 ```
 
-### Fresh from a Calkit project template
-
-Create a new project from the
-[`calkit/example-basic`](https://github.com/calkit/example-basic)
-template with:
-
-```sh
-calkit new project my-research \
-    --title "My research" \
-    --template calkit/example-basic \
-    --hub
 ```
+1. [stale] How does the system respond to increasing $x$?
+     figure figures/x-vs-y.png [stale] -- stage 'analyze' is out of date; run the pipeline
+     value results/summary.json:r_squared_quadratic [stale] -- stage 'analyze' is out of date; run the pipeline
 
-Note the `--hub` flag requires [a hub connection](https://docs.calkit.org/hub)
-to be set up, but can be omitted if the project doesn't need to be backed up to
-the hub or shared with collaborators.
-Hub integration can also be set up later.
-
-Next, move into the project folder and run the pipeline,
-which consists of several stages defined in `calkit.yaml`:
+Questions answered: 1/1
+Answers backed by current evidence: 0/1 ❌
+```
 
-<!-- TODO: This takes a long time to pull the image -->
+So, editing the script made the project's answer stale,
+and we were able to verify this from the CLI.
+To bring everything back up to date, run the pipeline again:
 
 ```sh
-cd my-research
 calkit run
 ```
 
-Next, make some edits to a script or LaTeX file and run `calkit status` to
-see what stages are out-of-date.
-For example:
+Only the stages affected by the change will rerun.
+The paper is rebuilt with the new figure and numbers,
+and the answer is backed by current evidence again.
+This edit-and-run loop is the main way of working in a Calkit project,
+and it keeps the paper in sync with the analysis.
+
+### Save the project
 
 ```sh
----------------------------- Pipeline ----------------------------
-Stale stages:
-        build-paper:
-          stale outputs:
-            paper/paper.pdf
-          modified inputs:
-            paper/paper.tex
+calkit save -am "Tweak the fit"
 ```
 
-Execute `calkit run` again to bring everything up-to-date.
+This adds and commits all changes in one step,
+putting code in Git and data and outputs in DVC,
+so you don't need to decide which goes where.
 
-To back up or save the project, call:
+At this point the project only exists on your machine,
+so there's nowhere to push to.
+Calkit will offer to set that up:
 
-```sh
-calkit save -am "Run pipeline"
 ```
+This project isn't connected to a hub, so there's nowhere to push its
+code and data.
+Connect it now? [Y/n]
+```
+
+Answering yes creates the project on a [hub](https://docs.calkit.org/hub),
+which backs it up,
+stores data and outputs that are too big for Git,
+and makes it available to collaborators,
+if you choose to add them.
+After that, `calkit save` will push everything to the right place,
+and a collaborator can clone the project and reproduce it with
+`calkit run`.
+
+Answering no is fine too.
+The commit has already been made,
+and the project can be connected later with `calkit update hub`.
+
+### Next steps
+
+Add your own data and a script to process it with
+[`calkit xr`](https://docs.calkit.org/pipeline),
+which executes a command and records it as a pipeline stage.
+To use Calkit with an existing project rather than a template, see
+[the tutorial on existing projects](https://docs.calkit.org/tutorials/existing-project).
+
+As the research goes on, add more questions.
+Each entry in `questions` lists its own evidence,
+so a new study can be a new question and the stages that answer it,
+rather than a new project.
 
 ### With an AI coding agent
 
@@ -425,7 +557,7 @@ Simply tell the [AI agent](https://docs.calkit.org/ai-tools):
 
 or
 
-> Create me a new Calkit project for investigating...
+> Create me a new Calkit project that answers the question: ...
 
 <!-- END INCLUDE -->