calkit-python 0.28.3__tar.gz → 0.29.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (180) hide show
  1. {calkit_python-0.28.3 → calkit_python-0.29.0}/PKG-INFO +1 -1
  2. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/__init__.py +1 -1
  3. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/cli/config.py +14 -4
  4. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/cli/main.py +65 -16
  5. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/cli/new.py +31 -3
  6. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/cli/notebooks.py +59 -14
  7. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/dvc.py +23 -13
  8. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/models/pipeline.py +38 -15
  9. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/models/test_pipeline.py +16 -0
  10. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/notebooks.md +1 -1
  11. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/quickstart.md +1 -1
  12. calkit_python-0.29.0/docs/tutorials/github-actions.md +83 -0
  13. calkit_python-0.29.0/docs/tutorials/img/actions-repo-secrets.png +0 -0
  14. calkit_python-0.29.0/docs/tutorials/img/latex-codespaces/new-token.png +0 -0
  15. calkit_python-0.29.0/docs/tutorials/img/quick-actions.png +0 -0
  16. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/index.md +1 -0
  17. {calkit_python-0.28.3 → calkit_python-0.29.0}/mkdocs.yml +1 -0
  18. calkit_python-0.28.3/docs/tutorials/img/latex-codespaces/new-token.png +0 -0
  19. {calkit_python-0.28.3 → calkit_python-0.29.0}/.github/FUNDING.yml +0 -0
  20. {calkit_python-0.28.3 → calkit_python-0.29.0}/.github/workflows/docs.yml +0 -0
  21. {calkit_python-0.28.3 → calkit_python-0.29.0}/.github/workflows/format.yml +0 -0
  22. {calkit_python-0.28.3 → calkit_python-0.29.0}/.github/workflows/publish-test.yml +0 -0
  23. {calkit_python-0.28.3 → calkit_python-0.29.0}/.github/workflows/publish.yml +0 -0
  24. {calkit_python-0.28.3 → calkit_python-0.29.0}/.github/workflows/test.yml +0 -0
  25. {calkit_python-0.28.3 → calkit_python-0.29.0}/.gitignore +0 -0
  26. {calkit_python-0.28.3 → calkit_python-0.29.0}/.pre-commit-config.yaml +0 -0
  27. {calkit_python-0.28.3 → calkit_python-0.29.0}/.python-version +0 -0
  28. {calkit_python-0.28.3 → calkit_python-0.29.0}/CITATION.cff +0 -0
  29. {calkit_python-0.28.3 → calkit_python-0.29.0}/CONTRIBUTING.md +0 -0
  30. {calkit_python-0.28.3 → calkit_python-0.29.0}/LICENSE +0 -0
  31. {calkit_python-0.28.3 → calkit_python-0.29.0}/Makefile +0 -0
  32. {calkit_python-0.28.3 → calkit_python-0.29.0}/README.md +0 -0
  33. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/__main__.py +0 -0
  34. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/calc.py +0 -0
  35. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/check.py +0 -0
  36. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/cli/__init__.py +0 -0
  37. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/cli/check.py +0 -0
  38. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/cli/cloud.py +0 -0
  39. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/cli/core.py +0 -0
  40. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/cli/describe.py +0 -0
  41. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/cli/import_.py +0 -0
  42. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/cli/list.py +0 -0
  43. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/cli/office.py +0 -0
  44. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/cli/overleaf.py +0 -0
  45. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/cli/update.py +0 -0
  46. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/cloud.py +0 -0
  47. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/conda.py +0 -0
  48. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/config.py +0 -0
  49. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/core.py +0 -0
  50. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/datasets.py +0 -0
  51. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/docker.py +0 -0
  52. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/environments.py +0 -0
  53. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/git.py +0 -0
  54. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/github.py +0 -0
  55. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/gui.py +0 -0
  56. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/jupyter.py +0 -0
  57. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/magics.py +0 -0
  58. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/matlab.py +0 -0
  59. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/models/__init__.py +0 -0
  60. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/models/core.py +0 -0
  61. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/models/io.py +0 -0
  62. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/models/iteration.py +0 -0
  63. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/notebooks.py +0 -0
  64. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/office.py +0 -0
  65. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/ops.py +0 -0
  66. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/pipeline.py +0 -0
  67. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/releases.py +0 -0
  68. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/server.py +0 -0
  69. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/templates/__init__.py +0 -0
  70. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/templates/core.py +0 -0
  71. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/templates/latex/__init__.py +0 -0
  72. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/templates/latex/article/paper.tex +0 -0
  73. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/templates/latex/core.py +0 -0
  74. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/templates/latex/jfm/jfm.bst +0 -0
  75. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/templates/latex/jfm/jfm.cls +0 -0
  76. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/templates/latex/jfm/lineno-FLM.sty +0 -0
  77. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/templates/latex/jfm/paper.tex +0 -0
  78. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/templates/latex/jfm/upmath.sty +0 -0
  79. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/__init__.py +0 -0
  80. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/cli/__init__.py +0 -0
  81. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/cli/test_check.py +0 -0
  82. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/cli/test_config.py +0 -0
  83. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/cli/test_list.py +0 -0
  84. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/cli/test_main.py +0 -0
  85. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/cli/test_new.py +0 -0
  86. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/cli/test_notebooks.py +0 -0
  87. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/models/__init__.py +0 -0
  88. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/models/test_iteration.py +0 -0
  89. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/test_calc.py +0 -0
  90. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/test_check.py +0 -0
  91. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/test_conda.py +0 -0
  92. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/test_core.py +0 -0
  93. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/test_dvc.py +0 -0
  94. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/test_jupyter.py +0 -0
  95. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/test_magics.py +0 -0
  96. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/test_notebooks.py +0 -0
  97. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/test_pipeline.py +0 -0
  98. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/tests/test_templates.py +0 -0
  99. {calkit_python-0.28.3 → calkit_python-0.29.0}/calkit/zenodo.py +0 -0
  100. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/CNAME +0 -0
  101. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/apps.md +0 -0
  102. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/calculations.md +0 -0
  103. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/calkit-yaml.md +0 -0
  104. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/cli-reference.md +0 -0
  105. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/cloud-integration.md +0 -0
  106. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/datasets.md +0 -0
  107. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/dependencies.md +0 -0
  108. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/environments.md +0 -0
  109. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/examples.md +0 -0
  110. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/help.md +0 -0
  111. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/img/c-to-the-k-white.svg +0 -0
  112. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/img/calkit-no-bg.png +0 -0
  113. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/img/connect-zenodo.png +0 -0
  114. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/img/jupyterlab-params.png +0 -0
  115. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/img/vscode-nb-params.png +0 -0
  116. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/index.md +0 -0
  117. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/installation.md +0 -0
  118. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/local-server.md +0 -0
  119. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/overleaf.md +0 -0
  120. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/pipeline/index.md +0 -0
  121. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/pipeline/manual-steps.md +0 -0
  122. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/pipeline/running-and-logging.md +0 -0
  123. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/references.md +0 -0
  124. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/releases.md +0 -0
  125. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/adding-latex-pub-docker.md +0 -0
  126. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/conda-envs.md +0 -0
  127. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/existing-project.md +0 -0
  128. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/first-project.md +0 -0
  129. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/latex-codespaces/building-codespace.png +0 -0
  130. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/latex-codespaces/codespaces-secrets-2.png +0 -0
  131. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/latex-codespaces/editor-split.png +0 -0
  132. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/latex-codespaces/go-to-linked-code.png +0 -0
  133. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/latex-codespaces/issue-from-selection.png +0 -0
  134. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/latex-codespaces/new-project.png +0 -0
  135. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/latex-codespaces/new-pub-2.png +0 -0
  136. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/latex-codespaces/paper.tex.png +0 -0
  137. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/latex-codespaces/project-home-3.png +0 -0
  138. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/latex-codespaces/push.png +0 -0
  139. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/latex-codespaces/stage.png +0 -0
  140. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/anakin-excel.jpg +0 -0
  141. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/chart-more-rows.png +0 -0
  142. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/create-project.png +0 -0
  143. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/elsevier-research-data-guidelines.png +0 -0
  144. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/excel-chart.png +0 -0
  145. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/excel-data.png +0 -0
  146. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/insert-link-to-file.png +0 -0
  147. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/needs-clone.png +0 -0
  148. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/new-stage.png +0 -0
  149. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/phd-comics-version-control.webp +0 -0
  150. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/pipeline-out-of-date.png +0 -0
  151. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/status-more-rows.png +0 -0
  152. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/uncommitted-changes.png +0 -0
  153. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/untracked-data.png +0 -0
  154. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/updated-publication.png +0 -0
  155. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/word-to-pdf-stage-2.png +0 -0
  156. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/office/workflow-page.png +0 -0
  157. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/openfoam/clone.png +0 -0
  158. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/openfoam/create-project.png +0 -0
  159. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/openfoam/datasets-page.png +0 -0
  160. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/openfoam/figure-on-website-updated.png +0 -0
  161. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/openfoam/figure-on-website.png +0 -0
  162. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/openfoam/new-token.png +0 -0
  163. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/openfoam/reclone.png +0 -0
  164. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/openfoam/status-after-import-dataset.png +0 -0
  165. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/img/run-proc.png +0 -0
  166. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/latex-codespaces.md +0 -0
  167. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/matlab.md +0 -0
  168. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/notebook-pipeline.md +0 -0
  169. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/office.md +0 -0
  170. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/openfoam.md +0 -0
  171. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/tutorials/procedures.md +0 -0
  172. {calkit_python-0.28.3 → calkit_python-0.29.0}/docs/version-control.md +0 -0
  173. {calkit_python-0.28.3 → calkit_python-0.29.0}/pyproject.toml +0 -0
  174. {calkit_python-0.28.3 → calkit_python-0.29.0}/scripts/install.ps1 +0 -0
  175. {calkit_python-0.28.3 → calkit_python-0.29.0}/scripts/install.sh +0 -0
  176. {calkit_python-0.28.3 → calkit_python-0.29.0}/test/nb-params.ipynb +0 -0
  177. {calkit_python-0.28.3 → calkit_python-0.29.0}/test/nb-subdir.ipynb +0 -0
  178. {calkit_python-0.28.3 → calkit_python-0.29.0}/test/pipeline.ipynb +0 -0
  179. {calkit_python-0.28.3 → calkit_python-0.29.0}/test/test-log.log +0 -0
  180. {calkit_python-0.28.3 → calkit_python-0.29.0}/uv.lock +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: calkit-python
3
- Version: 0.28.3
3
+ Version: 0.29.0
4
4
  Summary: Reproducibility simplified.
5
5
  Project-URL: Homepage, https://calkit.org
6
6
  Project-URL: Issues, https://github.com/calkit/calkit/issues
@@ -1,4 +1,4 @@
1
- __version__ = "0.28.3"
1
+ __version__ = "0.29.0"
2
2
 
3
3
  from .core import * # noqa: F403, I001
4
4
  from . import git # noqa: F401
@@ -2,6 +2,7 @@
2
2
 
3
3
  from __future__ import annotations
4
4
 
5
+ import os
5
6
  import subprocess
6
7
 
7
8
  import git
@@ -82,11 +83,17 @@ def setup_remote(
82
83
  configure_remote()
83
84
  set_remote_auth()
84
85
  except subprocess.CalledProcessError:
85
- raise_error("DVC remote config failed; have you run `dvc init`?")
86
+ if not os.path.isfile(".dvc/config"):
87
+ raise_error(
88
+ "DVC remote config failed; have you run `calkit init`?"
89
+ )
90
+ raise_error(
91
+ "Failed to configure DVC remote; check DVC config for errors"
92
+ )
86
93
  except InvalidGitRepositoryError:
87
94
  raise_error("Current directory is not a Git repository")
88
- except ValueError as e:
89
- raise_error(e)
95
+ except (ValueError, RuntimeError) as e:
96
+ raise_error(f"Failed to set up DVC remote: {e}")
90
97
  if not no_commit:
91
98
  repo = git.Repo()
92
99
  repo.git.add(".dvc/config")
@@ -101,7 +108,10 @@ def setup_remote_auth():
101
108
  """Store a Calkit cloud token in the local DVC config for all Calkit
102
109
  remotes.
103
110
  """
104
- remotes = get_remotes()
111
+ try:
112
+ remotes = get_remotes()
113
+ except Exception:
114
+ raise_error("Cannot list DVC remotes; check DVC config for errors")
105
115
  for name, url in remotes.items():
106
116
  if name == "calkit" or name.startswith("calkit:"):
107
117
  typer.echo(f"Setting up authentication for DVC remote: {name}")
@@ -981,22 +981,23 @@ def run(
981
981
  except Exception as e:
982
982
  os.environ.pop("CALKIT_PIPELINE_RUNNING", None)
983
983
  raise_error(f"Pipeline compilation failed: {e}")
984
- # Get status of Git repo before running
985
- repo = git.Repo()
986
- git_rev = repo.head.commit.hexsha
987
- try:
988
- git_branch = repo.active_branch.name
989
- except TypeError:
990
- # If no branch is checked out, we are in a detached HEAD state
991
- git_branch = None
992
- git_changed_files_before = calkit.git.get_changed_files(repo=repo)
993
- git_staged_files_before = calkit.git.get_staged_files(repo=repo)
994
- git_untracked_files_before = calkit.git.get_untracked_files(repo=repo)
995
- # Get status of DVC repo before running
996
- dvc_repo = dvc.repo.Repo()
997
- dvc_status_before = dvc_repo.status()
998
- dvc_data_status_before = dvc_repo.data_status()
999
- dvc_data_status_before.pop("git", None) # Remove git status
984
+ if save_log:
985
+ # Get status of Git repo before running
986
+ repo = git.Repo()
987
+ git_rev = repo.head.commit.hexsha
988
+ try:
989
+ git_branch = repo.active_branch.name
990
+ except TypeError:
991
+ # If no branch is checked out, we are in a detached HEAD state
992
+ git_branch = None
993
+ git_changed_files_before = calkit.git.get_changed_files(repo=repo)
994
+ git_staged_files_before = calkit.git.get_staged_files(repo=repo)
995
+ git_untracked_files_before = calkit.git.get_untracked_files(repo=repo)
996
+ # Get status of DVC repo before running
997
+ dvc_repo = dvc.repo.Repo()
998
+ dvc_status_before = dvc_repo.status()
999
+ dvc_data_status_before = dvc_repo.data_status()
1000
+ dvc_data_status_before.pop("git", None) # Remove git status
1000
1001
  if targets is None:
1001
1002
  targets = []
1002
1003
  args = deepcopy(targets)
@@ -1855,3 +1856,51 @@ def run_jupyter(
1855
1856
  """Run a command with the Jupyter CLI."""
1856
1857
  process = subprocess.run([sys.executable, "-m", "jupyter"] + sys.argv[2:])
1857
1858
  sys.exit(process.returncode)
1859
+
1860
+
1861
+ @app.command(name="latexmk")
1862
+ def run_latexmk(
1863
+ tex_file: Annotated[str, typer.Argument(help="The .tex file to compile.")],
1864
+ environment: Annotated[
1865
+ str | None,
1866
+ typer.Option(
1867
+ "--env",
1868
+ "-e",
1869
+ help=("Environment in which to run latexmk, if applicable."),
1870
+ ),
1871
+ ] = None,
1872
+ ):
1873
+ """Compile a LaTeX document with latexmk.
1874
+
1875
+ If a Calkit environment is not specified, latexmk will be run in the
1876
+ system environment if available. If not available, a TeX Live Docker
1877
+ container will be used.
1878
+ """
1879
+ latexmk_cmd = [
1880
+ "latexmk",
1881
+ "-pdf",
1882
+ "-cd",
1883
+ "-silent",
1884
+ "-synctex=1",
1885
+ "-interaction=nonstopmode",
1886
+ tex_file,
1887
+ ]
1888
+ if environment is not None:
1889
+ cmd = ["calkit", "xenv", "--name", environment] + latexmk_cmd
1890
+ elif calkit.check_dep_exists("latexmk"):
1891
+ cmd = latexmk_cmd
1892
+ else:
1893
+ cmd = [
1894
+ "docker",
1895
+ "run",
1896
+ "--rm",
1897
+ "-v",
1898
+ f"{os.getcwd()}:/work",
1899
+ "-w",
1900
+ "/work",
1901
+ "texlive/texlive:latest-full",
1902
+ ] + latexmk_cmd
1903
+ try:
1904
+ subprocess.check_call(cmd)
1905
+ except subprocess.CalledProcessError:
1906
+ raise_error("latexmk failed")
@@ -157,10 +157,16 @@ def new_project(
157
157
  repo.git.commit(["-m", "Initialize DVC"])
158
158
  ck_info_fpath = os.path.join(abs_path, "calkit.yaml")
159
159
  if os.path.isfile(ck_info_fpath) and not overwrite:
160
- raise_error(
160
+ ck_info = calkit.load_calkit_info(wdir=abs_path)
161
+ name = ck_info.get("name", name)
162
+ title = ck_info.get("title", title)
163
+ description = ck_info.get("description", description)
164
+ typer.echo(
161
165
  "Destination is already a Calkit project; "
162
- "use --overwrite to continue"
166
+ "will use existing project info where possible"
163
167
  )
168
+ else:
169
+ ck_info = {}
164
170
  if os.path.isdir(abs_path) and os.listdir(abs_path) and repo is None:
165
171
  warn(f"{abs_path} is not empty")
166
172
  if name is None:
@@ -224,7 +230,29 @@ def new_project(
224
230
  )
225
231
  repo.git.add("calkit.yaml")
226
232
  if not no_commit:
227
- repo.git.commit(["-m", "Create calkit.yaml"])
233
+ repo.git.commit(
234
+ ["calkit.yaml", "-m", "Create calkit.yaml"]
235
+ )
236
+ else:
237
+ # Merge with existing project info in calkit.yaml
238
+ typer.echo("Updating existing calkit.yaml file")
239
+ ck_info = (
240
+ dict(
241
+ owner=resp["owner_account_name"],
242
+ name=resp["name"],
243
+ title=resp["title"],
244
+ description=resp["description"],
245
+ git_repo_url=resp["git_repo_url"],
246
+ )
247
+ | ck_info
248
+ )
249
+ with open(calkit_fpath, "w") as f:
250
+ ryaml.dump(ck_info, f)
251
+ repo.git.add("calkit.yaml")
252
+ if not no_commit and repo.git.diff("--staged"):
253
+ repo.git.commit(
254
+ ["calkit.yaml", "-m", "Update calkit.yaml"]
255
+ )
228
256
  try:
229
257
  calkit.dvc.configure_remote(wdir=abs_path)
230
258
  except Exception:
@@ -163,6 +163,17 @@ def execute_notebook(
163
163
  ),
164
164
  ),
165
165
  ] = None,
166
+ language: Annotated[
167
+ str,
168
+ typer.Option(
169
+ "--language",
170
+ "-l",
171
+ help=(
172
+ "Notebook language; if 'matlab', MATLAB kernel must be "
173
+ "available in environment."
174
+ ),
175
+ ),
176
+ ] = "python",
166
177
  verbose: Annotated[
167
178
  bool, typer.Option("--verbose", "-v", help="Print verbose output.")
168
179
  ] = False,
@@ -176,10 +187,24 @@ def execute_notebook(
176
187
 
177
188
  if os.path.isabs(path):
178
189
  raise ValueError("Path must be relative")
190
+ if language.lower() not in ["python", "matlab", "julia"]:
191
+ raise ValueError(
192
+ "Language must be one of 'python', 'matlab', or 'julia'"
193
+ )
179
194
  # First, ensure the specified environment has a kernel we can use
180
- kernel_name = check_env_kernel(
181
- env_name=env_name, no_check=no_check, verbose=verbose
182
- )
195
+ # We need to check the environment type and create the kernel if needed
196
+ if language.lower() == "python":
197
+ kernel_name = check_env_kernel(
198
+ env_name=env_name, no_check=no_check, verbose=verbose
199
+ )
200
+ elif language.lower() == "matlab":
201
+ kernel_name = "jupyter_matlab_kernel"
202
+ else:
203
+ raise_error(f"Language '{language}' not yet supported")
204
+ # We can't handle parameters unless language is Python
205
+ if language.lower() != "python":
206
+ if params or params_json is not None or params_base64 is not None:
207
+ raise_error("Parameters can only be passed to Python notebooks")
183
208
  # Parse parameters
184
209
  if params:
185
210
  try:
@@ -214,14 +239,34 @@ def execute_notebook(
214
239
  typer.echo(f"Using kernel: {kernel_name}")
215
240
  typer.echo(f"Running with cwd: {notebook_dir}")
216
241
  typer.echo(f"Output will be saved to: {fpath_out_exec}")
217
- papermill.execute_notebook(
218
- input_path=path,
219
- output_path=fpath_out_exec,
220
- kernel_name=kernel_name,
221
- log_output=True,
222
- parameters=parsed_params,
223
- cwd=notebook_dir,
224
- )
242
+ # If this is a Python or Julia notebook, we can use Papermill
243
+ # If it's a MATLAB notebook, we need to use the MATLAB kernel inside the
244
+ # specified environment
245
+ if language.lower() in ["python", "julia"]:
246
+ papermill.execute_notebook(
247
+ input_path=path,
248
+ output_path=fpath_out_exec,
249
+ kernel_name=kernel_name,
250
+ log_output=True,
251
+ parameters=parsed_params,
252
+ cwd=notebook_dir,
253
+ )
254
+ elif language.lower() == "matlab":
255
+ # Use nbconvert to execute the notebook with the MATLAB kernel
256
+ cmd = [
257
+ "python",
258
+ "-m",
259
+ "jupyter",
260
+ "nbconvert",
261
+ "--to",
262
+ "notebook",
263
+ "--execute",
264
+ f"--ExecutePreprocessor.kernel_name={kernel_name}",
265
+ "--output",
266
+ fpath_out_exec,
267
+ path,
268
+ ]
269
+ run_in_env(cmd, env_name=env_name, no_check=no_check, verbose=verbose)
225
270
  for to_fmt in to:
226
271
  if to_fmt != "notebook":
227
272
  try:
@@ -250,6 +295,6 @@ def execute_notebook(
250
295
  fname_out,
251
296
  ]
252
297
  typer.echo(f"Exporting {to_fmt}")
253
- run_in_env(
254
- cmd=cmd, env_name=env_name, no_check=True, verbose=verbose
255
- )
298
+ p = subprocess.run(cmd)
299
+ if p.returncode != 0:
300
+ raise_error(f"nbconvert failed for format '{to_fmt}'")
@@ -18,7 +18,7 @@ logger = logging.getLogger(__package__)
18
18
  logger.setLevel(logging.INFO)
19
19
 
20
20
 
21
- def configure_remote(wdir: str = None):
21
+ def configure_remote(wdir: str | None = None):
22
22
  try:
23
23
  project_name = calkit.detect_project_name(wdir=wdir)
24
24
  except ValueError as e:
@@ -70,7 +70,9 @@ def configure_remote(wdir: str = None):
70
70
 
71
71
 
72
72
  def set_remote_auth(
73
- remote_name: str = None, always_auth: bool = False, wdir: str = None
73
+ remote_name: str | None = None,
74
+ always_auth: bool = False,
75
+ wdir: str | None = None,
74
76
  ):
75
77
  """Get a token and set it in the local DVC config so we can interact with
76
78
  the cloud as an HTTP remote.
@@ -85,7 +87,7 @@ def set_remote_auth(
85
87
  )["access_token"]
86
88
  settings.dvc_token = token
87
89
  settings.write()
88
- subprocess.check_call(
90
+ p1 = subprocess.run(
89
91
  [
90
92
  sys.executable,
91
93
  "-m",
@@ -99,7 +101,7 @@ def set_remote_auth(
99
101
  ],
100
102
  cwd=wdir,
101
103
  )
102
- subprocess.check_call(
104
+ p2 = subprocess.run(
103
105
  [
104
106
  sys.executable,
105
107
  "-m",
@@ -113,6 +115,10 @@ def set_remote_auth(
113
115
  ],
114
116
  cwd=wdir,
115
117
  )
118
+ if p1.returncode != 0 or p2.returncode != 0:
119
+ raise RuntimeError(
120
+ f"Failed to set DVC remote authentication for {remote_name}"
121
+ )
116
122
 
117
123
 
118
124
  def add_external_remote(owner_name: str, project_name: str) -> dict:
@@ -159,13 +165,15 @@ def get_remotes(wdir: str | None = None) -> dict[str, str]:
159
165
  """Get a dictionary of DVC remotes, keyed by name, with URL as the
160
166
  value.
161
167
  """
162
- out = (
163
- subprocess.check_output(
164
- [sys.executable, "-m", "dvc", "remote", "list"], cwd=wdir
165
- )
166
- .decode()
167
- .strip()
168
+ p = subprocess.run(
169
+ [sys.executable, "-m", "dvc", "remote", "list"],
170
+ cwd=wdir,
171
+ capture_output=True,
172
+ text=True,
168
173
  )
174
+ if p.returncode != 0:
175
+ raise RuntimeError(f"Error getting DVC remotes: {p.stderr.strip()}")
176
+ out = p.stdout.strip()
169
177
  if not out:
170
178
  return {}
171
179
  resp = {}
@@ -182,12 +190,14 @@ def get_remotes(wdir: str | None = None) -> dict[str, str]:
182
190
  return resp
183
191
 
184
192
 
185
- def list_paths(wdir: str = None, recursive=False) -> list[str]:
193
+ def list_paths(wdir: str | None = None, recursive=False) -> list[str]:
186
194
  """List paths tracked with DVC."""
187
- return [p.get("path") for p in list_files(wdir=wdir, recursive=recursive)]
195
+ return [
196
+ p.get("path", "") for p in list_files(wdir=wdir, recursive=recursive)
197
+ ]
188
198
 
189
199
 
190
- def list_files(wdir: str = None, recursive=True) -> list[dict]:
200
+ def list_files(wdir: str | None = None, recursive=True) -> list[dict]:
191
201
  """Return a list with all files in DVC, including their path and md5
192
202
  checksum.
193
203
  """
@@ -113,6 +113,7 @@ class Stage(BaseModel):
113
113
  "python-script",
114
114
  "latex",
115
115
  "matlab-script",
116
+ "matlab-command",
116
117
  "docker-command",
117
118
  "shell-command",
118
119
  "shell-script",
@@ -164,7 +165,9 @@ class Stage(BaseModel):
164
165
 
165
166
  @property
166
167
  def xenv_cmd(self) -> str:
167
- return f"calkit xenv -n {self.environment} --no-check"
168
+ if self.environment == "_system":
169
+ return ""
170
+ return f"calkit xenv -n {self.environment} --no-check --"
168
171
 
169
172
  def to_dvc(self) -> dict:
170
173
  """Convert to a DVC stage.
@@ -193,7 +196,7 @@ class PythonScriptStage(Stage):
193
196
 
194
197
  @property
195
198
  def dvc_cmd(self) -> str:
196
- cmd = f"{self.xenv_cmd} -- python {self.script_path}"
199
+ cmd = f"{self.xenv_cmd} python {self.script_path}"
197
200
  for arg in self.args:
198
201
  cmd += f" {arg}"
199
202
  return cmd
@@ -212,7 +215,7 @@ class LatexStage(Stage):
212
215
 
213
216
  @property
214
217
  def dvc_cmd(self) -> str:
215
- cmd = f"{self.xenv_cmd} -- latexmk -cd -interaction=nonstopmode"
218
+ cmd = f"{self.xenv_cmd} latexmk -cd -interaction=nonstopmode"
216
219
  if not self.verbose:
217
220
  cmd += " -silent"
218
221
  if self.force:
@@ -247,7 +250,26 @@ class MatlabScriptStage(Stage):
247
250
 
248
251
  @property
249
252
  def dvc_cmd(self) -> str:
250
- return f"{self.xenv_cmd} -- \"run('{self.script_path}');\""
253
+ cmd = self.xenv_cmd
254
+ if self.environment == "_system":
255
+ cmd += "matlab -batch"
256
+ cmd += f" \"run('{self.script_path}');\""
257
+ return cmd
258
+
259
+
260
+ class MatlabCommandStage(Stage):
261
+ kind: Literal["matlab-command"] = "matlab-command"
262
+ command: str
263
+
264
+ @property
265
+ def dvc_cmd(self) -> str:
266
+ # We need to escape quotes in the command
267
+ matlab_cmd = self.command.replace('"', '\\"')
268
+ cmd = self.xenv_cmd
269
+ if self.environment == "_system":
270
+ cmd += "matlab -batch"
271
+ cmd += f' "{matlab_cmd}"'
272
+ return cmd
251
273
 
252
274
 
253
275
  class ShellCommandStage(Stage):
@@ -257,14 +279,12 @@ class ShellCommandStage(Stage):
257
279
 
258
280
  @property
259
281
  def dvc_cmd(self) -> str:
260
- cmd = ""
261
- if self.environment != "_system":
262
- cmd = f"{self.xenv_cmd} -- "
282
+ cmd = self.xenv_cmd
263
283
  if self.shell == "zsh":
264
284
  norc_args = "-f"
265
285
  else:
266
286
  norc_args = "--noprofile --norc"
267
- cmd += f'{self.shell} {norc_args} -c "{self.command}"'
287
+ cmd += f' {self.shell} {norc_args} -c "{self.command}"'
268
288
  return cmd
269
289
 
270
290
 
@@ -280,14 +300,12 @@ class ShellScriptStage(Stage):
280
300
 
281
301
  @property
282
302
  def dvc_cmd(self) -> str:
283
- cmd = ""
284
- if self.environment != "_system":
285
- cmd = f"{self.xenv_cmd} -- "
303
+ cmd = self.xenv_cmd
286
304
  if self.shell == "zsh":
287
305
  norc_args = "-f"
288
306
  else:
289
307
  norc_args = "--noprofile --norc"
290
- cmd += f"{self.shell} {norc_args} {self.script_path}"
308
+ cmd += f" {self.shell} {norc_args} {self.script_path}"
291
309
  for arg in self.args:
292
310
  cmd += f" {arg}"
293
311
  return cmd
@@ -327,7 +345,7 @@ class JuliaScriptStage(Stage):
327
345
 
328
346
  @property
329
347
  def dvc_cmd(self) -> str:
330
- cmd = f'{self.xenv_cmd} -- "include(\\"{self.script_path}\\")"'
348
+ cmd = f'{self.xenv_cmd} "include(\\"{self.script_path}\\")"'
331
349
  return cmd
332
350
 
333
351
  @property
@@ -343,7 +361,7 @@ class JuliaCommandStage(Stage):
343
361
  def dvc_cmd(self) -> str:
344
362
  # We need to escape quotes in the command
345
363
  julia_cmd = self.command.replace('"', '\\"')
346
- cmd = f'{self.xenv_cmd} -- "{julia_cmd}"'
364
+ cmd = f'{self.xenv_cmd} "{julia_cmd}"'
347
365
  return cmd
348
366
 
349
367
 
@@ -373,6 +391,7 @@ class JupyterNotebookStage(Stage):
373
391
  executed_ipynb_storage: Literal["git", "dvc"] | None = "dvc"
374
392
  html_storage: Literal["git", "dvc"] | None = "dvc"
375
393
  parameters: dict[str, Any] = {}
394
+ language: Literal["python", "matlab", "julia"] = "python"
376
395
 
377
396
  def update_parameters(self, params: dict) -> None:
378
397
  """If we have any templated parameters, update those, e.g., from
@@ -430,7 +449,10 @@ class JupyterNotebookStage(Stage):
430
449
 
431
450
  @property
432
451
  def dvc_cmd(self) -> str:
433
- cmd = f"calkit nb execute --environment {self.environment} --no-check"
452
+ cmd = (
453
+ f"calkit nb execute --environment {self.environment} "
454
+ f"--no-check --language {self.language}"
455
+ )
434
456
  if self.html_storage:
435
457
  cmd += " --to html"
436
458
  if self.parameters:
@@ -533,6 +555,7 @@ class Pipeline(BaseModel):
533
555
  PythonScriptStage
534
556
  | LatexStage
535
557
  | MatlabScriptStage
558
+ | MatlabCommandStage
536
559
  | ShellCommandStage
537
560
  | ShellScriptStage
538
561
  | DockerCommandStage
@@ -7,6 +7,7 @@ from calkit.models.pipeline import (
7
7
  JuliaCommandStage,
8
8
  JupyterNotebookStage,
9
9
  LatexStage,
10
+ MatlabCommandStage,
10
11
  PythonScriptStage,
11
12
  StageIteration,
12
13
  WordToPdfStage,
@@ -131,3 +132,18 @@ def test_juliacommandstage():
131
132
  assert sd["cmd"] == (
132
133
  'calkit xenv -n j1 --no-check -- "println(\\"sup\\")"'
133
134
  )
135
+
136
+
137
+ def test_matlabcommandstage():
138
+ s = MatlabCommandStage(environment="m1", command='disp("Hello, MATLAB!");')
139
+ sd = s.to_dvc()
140
+ print(sd)
141
+ assert sd["cmd"] == (
142
+ 'calkit xenv -n m1 --no-check -- "disp(\\"Hello, MATLAB!\\");"'
143
+ )
144
+ s = MatlabCommandStage(
145
+ environment="_system", command='disp("Hello, MATLAB!");'
146
+ )
147
+ sd = s.to_dvc()
148
+ print(sd)
149
+ assert sd["cmd"] == 'matlab -batch "disp(\\"Hello, MATLAB!\\");"'
@@ -322,4 +322,4 @@ pipeline:
322
322
  - results/param1={param1}/param2={param2}/results.csv
323
323
  ```
324
324
 
325
- For more information, see [pipeline iteration](/pipeline#iteration).
325
+ For more information, see [pipeline iteration](pipeline/index.md#iteration).
@@ -25,7 +25,7 @@ If you're using Conda for environment management,
25
25
  e.g., with an `environment.yml` file,
26
26
  you can use the `calkit new conda-env` command.
27
27
 
28
- Next, we can start building our [pipeline](pipeline.md).
28
+ Next, we can start building our [pipeline](pipeline/index.md).
29
29
  Let's say we have a Jupyter notebook called `collect-data.ipynb`
30
30
  that produces raw data at `data/raw.h5`.
31
31
  We can add a pipeline stage to run this notebook in the `main` environment
@@ -0,0 +1,83 @@
1
+ # Running Calkit in GitHub Actions
2
+
3
+ A project can be set up to automatically run the pipeline every time a
4
+ change is pushed, either to the main branch or on a pull request,
5
+ using GitHub Actions.
6
+ The latter allows for inspection of outputs before merging into main.
7
+
8
+ To get started, generate a DVC token and set it in your GitHub Actions
9
+ secrets as `CALKIT_DVC_TOKEN`,
10
+ either at your account or project level.
11
+ On calkit.io, there are shortcuts on the project page for managing both
12
+ Calkit tokens and GitHub Actions secrets:
13
+
14
+ ![Project quick actions links](img/quick-actions.png)
15
+
16
+ ![DVC token](img/latex-codespaces/new-token.png)
17
+
18
+ ![Actions repo secrets](img/actions-repo-secrets.png)
19
+
20
+ Next, add a workflow to the project in the `.github/workflows` folder.
21
+ For example, we can put the content below into `.github/workflows/run.yml`:
22
+
23
+ ```yaml
24
+ name: Run pipeline
25
+
26
+ on:
27
+ push:
28
+ branches:
29
+ - main
30
+ pull_request:
31
+
32
+ permissions:
33
+ contents: write
34
+
35
+ # Make sure we only ever run one per branch so we don't have issues pushing
36
+ # after running the pipeline
37
+ concurrency:
38
+ group: calkit-run-${{ github.ref }}
39
+ cancel-in-progress: false
40
+
41
+ jobs:
42
+ main:
43
+ name: Run
44
+ runs-on: ubuntu-latest
45
+ steps:
46
+ - uses: actions/checkout@v4
47
+ with:
48
+ # For PRs, checkout the head ref to avoid detached HEAD
49
+ ref: ${{ github.head_ref || github.ref_name }}
50
+ token: ${{ secrets.GITHUB_TOKEN }}
51
+ - name: Configure Git credentials
52
+ run: |
53
+ git config user.name github-actions[bot]
54
+ git config user.email 41898282+github-actions[bot]@users.noreply.github.com
55
+ - name: Setup uv
56
+ uses: astral-sh/setup-uv@v5
57
+ - name: Install Calkit
58
+ run: uv tool install calkit-python
59
+ - name: Run Calkit
60
+ uses: calkit/run-action@v1
61
+ with:
62
+ dvc_token: ${{ secrets.CALKIT_DVC_TOKEN }}
63
+ ```
64
+
65
+ This particular example installs Calkit with uv,
66
+ meaning `uv` and `uv-venv` environment types will work without any additional
67
+ configuration.
68
+ Docker is also installed by default on the `ubuntu-latest` machine.
69
+ If other environment types are used in the project,
70
+ setup steps may be necessary for those, e.g.,
71
+ [`setup-miniconda`](https://github.com/marketplace/actions/setup-miniconda)
72
+ or [`install-juliaup`](https://github.com/marketplace/actions/install-juliaup).
73
+
74
+ By default, the Calkit GitHub Action will run the pipeline and save results.
75
+ This is why the workflow needs write permissions and configures Git
76
+ credentials to act as the GitHub Actions bot.
77
+ The workflow also limits concurrency so multiple jobs don't attempt to push
78
+ to the same branch at the same time.
79
+
80
+ It's possible to configure the action to not save results, e.g., if you
81
+ just want to check that the pipeline can run without errors.
82
+ See the [documentation](https://github.com/marketplace/actions/run-calkit)
83
+ for all available options.
@@ -10,3 +10,4 @@
10
10
  - [Adding a new LaTeX-based publication with its own Docker build environment](adding-latex-pub-docker.md)
11
11
  - [A reproducible workflow using Microsoft Office (Word and Excel)](office.md)
12
12
  - [Using Calkit with MATLAB](matlab.md)
13
+ - [Automation with GitHub Actions](github-actions.md)