calkit-python 0.22.2__tar.gz → 0.22.3__tar.gz

{calkit_python-0.22.2 → calkit_python-0.22.3}/.pre-commit-config.yaml

@@ -12,7 +12,6 @@ repos:
     hooks:
       - id: ruff
         args: [--exit-non-zero-on-fix, --config=pyproject.toml]
-        exclude: calkit
       - id: ruff-format
         args: [--config=pyproject.toml]
 

{calkit_python-0.22.2 → calkit_python-0.22.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: calkit-python
-Version: 0.22.2
+Version: 0.22.3
 Summary: Reproducibility simplified.
 Project-URL: Homepage, https://calkit.org
 Project-URL: Issues, https://github.com/calkit/calkit/issues
@@ -54,16 +54,60 @@ Description-Content-Type: text/markdown
   </a>
 </p>
 
-Calkit is a framework and toolkit for reproducible research projects.
-It acts as a top-level layer to integrate and simplify the use of enabling
-technologies such as
-[Git](https://git-scm.com/),
-[DVC](https://dvc.org/),
-[Conda](https://docs.conda.io/en/latest/),
-and [Docker](https://docker.com).
-Calkit also adds a domain-specific data model
-such that all aspects of the research process can be fully described in a
-single repository and therefore easily consumed by others.
+Calkit is a language-agnostic project framework and toolkit
+to make your research or analytics project
+reproducible to the highest standard,
+which means:
+
+> Inputs and process definitions are provided and sufficiently described
+> such that anyone can easily verify that they produced the outputs
+> used to support the conclusions.
+
+"Easily" means that after obtaining your project files,
+it should only require executing a single command
+(like "pressing a single button" in
+[Claerbout and Karrenbach (1992)](https://doi.org/10.1190/1.1822162)),
+which should finish in less than 15 minutes
+(suggested by
+[Vandewalle et al. (2009)](https://doi.org/10.1109/MSP.2009.932122)).
+
+If the processes are too expensive to rerun in under 15 minutes,
+it should be possible to confirm that none of the input data
+or process definitions (e.g., environment specifications, scripts)
+have changed since saving the current versions of each output artifact
+(figure, table, dataset, publication, etc.)
+
+When your project is reproducible,
+you'll be able to iterate more quickly and more often,
+easily onboard collaborators,
+make fewer mistakes,
+and feel confident sharing all of your project materials
+with your research articles,
+because you'll know the code will actually run!
+This will allow others to reuse parts of your project in their own research,
+accelerating the pace of discovery.
+
+Working at this level of automation, discipline, and rigor may sound like
+a lot of effort,
+but Calkit makes it easy!
+
+## Features
+
+- A schema to store structured metadata describing the
+  project's important outputs (in its `calkit.yaml` file)
+  and how they are created
+  (its computational environments and pipeline).
+- A CLI to run the project's pipeline to verify it's reproducible,
+  regenerating outputs as needed and
+  ensuring all
+  computational environments (e.g., [Conda](https://docs.conda.io/en/latest/), [Docker](https://docker.com)) match their specification.
+- A command line interface (CLI) to simplify keeping code, text, and larger
+  data files backed up in the same project repo using both
+  [Git](https://git-scm.com/) and [DVC](https://dvc.org/).
+- A complementary
+  [cloud system](https://github.com/calkit/calkit-cloud)
+  to facilitate backup, collaboration,
+  and sharing throughout the entire research lifecycle.
 
 ## Installation
 
@@ -150,47 +194,6 @@ This will commit and push to both GitHub and the Calkit Cloud.
 We welcome all kinds of contributions!
 See [CONTRIBUTING.md](CONTRIBUTING.md) to learn how to get involved.
 
-## Why does reproducibility matter?
-
-If your work is reproducible, that means that someone else can "run" it and
-calculate the same results or outputs.
-This is a major step towards addressing
-[the replication crisis](https://en.wikipedia.org/wiki/Replication_crisis)
-and has some major benefits for both you as an individual and the research
-community:
-
-1. You will avoid mistakes caused by, e.g., running an old version of a script
-   and including a figure that wasn't created after fixing a bug in the data
-   processing pipeline.
-2. Since your project is "runnable," it's more likely that someone else will be
-   able to reuse part of your work to run it in a different context, thereby
-   producing a bigger impact and accelerating the pace of discovery.
-   If someone can take what you've done and use it to calculate a
-   prediction, you have just produced truly useful knowledge.
-
-## Why another tool/platform?
-
-Git, GitHub, DVC, Docker et al. are amazing tools/platforms, but their
-use involves multiple fairly difficult learning curves,
-and tying them together might mean developing something new for each project.
-Our goal is to provide a single tool and platform to unify all of these so
-that there is a single, gentle learning curve.
-However, it is not our goal to hide or replace these underlying components.
-Advanced users can use them directly, but new users aren't forced to, which
-helps them get up and running with less effort and training.
-Calkit should help users understand what is going on under the hood without
-forcing them to work at that lower level of abstraction.
-
-## How it works
-
-Calkit creates a simple human-readable "database" inside the `calkit.yaml`
-file, which serves as a way to store important information about the project,
-e.g., what question(s) it seeks to answer,
-what files should be considered datasets, figures, publications, etc.
-The Calkit cloud reads this database and registers the various entities
-as part of the entire ecosystem such that if a project is made public,
-other researchers can find and reuse your work to accelerate their own.
-
 ## Design/UX principles
 
 1. Be opinionated. Users should not be forced to make unimportant decisions.

{calkit_python-0.22.2 → calkit_python-0.22.3}/README.md

@@ -17,16 +17,60 @@
   </a>
 </p>
 
-Calkit is a framework and toolkit for reproducible research projects.
-It acts as a top-level layer to integrate and simplify the use of enabling
-technologies such as
-[Git](https://git-scm.com/),
-[DVC](https://dvc.org/),
-[Conda](https://docs.conda.io/en/latest/),
-and [Docker](https://docker.com).
-Calkit also adds a domain-specific data model
-such that all aspects of the research process can be fully described in a
-single repository and therefore easily consumed by others.
+Calkit is a language-agnostic project framework and toolkit
+to make your research or analytics project
+reproducible to the highest standard,
+which means:
+
+> Inputs and process definitions are provided and sufficiently described
+> such that anyone can easily verify that they produced the outputs
+> used to support the conclusions.
+
+"Easily" means that after obtaining your project files,
+it should only require executing a single command
+(like "pressing a single button" in
+[Claerbout and Karrenbach (1992)](https://doi.org/10.1190/1.1822162)),
+which should finish in less than 15 minutes
+(suggested by
+[Vandewalle et al. (2009)](https://doi.org/10.1109/MSP.2009.932122)).
+
+If the processes are too expensive to rerun in under 15 minutes,
+it should be possible to confirm that none of the input data
+or process definitions (e.g., environment specifications, scripts)
+have changed since saving the current versions of each output artifact
+(figure, table, dataset, publication, etc.)
+
+When your project is reproducible,
+you'll be able to iterate more quickly and more often,
+easily onboard collaborators,
+make fewer mistakes,
+and feel confident sharing all of your project materials
+with your research articles,
+because you'll know the code will actually run!
+This will allow others to reuse parts of your project in their own research,
+accelerating the pace of discovery.
+
+Working at this level of automation, discipline, and rigor may sound like
+a lot of effort,
+but Calkit makes it easy!
+
+## Features
+
+- A schema to store structured metadata describing the
+  project's important outputs (in its `calkit.yaml` file)
+  and how they are created
+  (its computational environments and pipeline).
+- A CLI to run the project's pipeline to verify it's reproducible,
+  regenerating outputs as needed and
+  ensuring all
+  computational environments (e.g., [Conda](https://docs.conda.io/en/latest/), [Docker](https://docker.com)) match their specification.
+- A command line interface (CLI) to simplify keeping code, text, and larger
+  data files backed up in the same project repo using both
+  [Git](https://git-scm.com/) and [DVC](https://dvc.org/).
+- A complementary
+  [cloud system](https://github.com/calkit/calkit-cloud)
+  to facilitate backup, collaboration,
+  and sharing throughout the entire research lifecycle.
 
 ## Installation
 
@@ -113,47 +157,6 @@ This will commit and push to both GitHub and the Calkit Cloud.
 We welcome all kinds of contributions!
 See [CONTRIBUTING.md](CONTRIBUTING.md) to learn how to get involved.
 
-## Why does reproducibility matter?
-
-If your work is reproducible, that means that someone else can "run" it and
-calculate the same results or outputs.
-This is a major step towards addressing
-[the replication crisis](https://en.wikipedia.org/wiki/Replication_crisis)
-and has some major benefits for both you as an individual and the research
-community:
-
-1. You will avoid mistakes caused by, e.g., running an old version of a script
-   and including a figure that wasn't created after fixing a bug in the data
-   processing pipeline.
-2. Since your project is "runnable," it's more likely that someone else will be
-   able to reuse part of your work to run it in a different context, thereby
-   producing a bigger impact and accelerating the pace of discovery.
-   If someone can take what you've done and use it to calculate a
-   prediction, you have just produced truly useful knowledge.
-
-## Why another tool/platform?
-
-Git, GitHub, DVC, Docker et al. are amazing tools/platforms, but their
-use involves multiple fairly difficult learning curves,
-and tying them together might mean developing something new for each project.
-Our goal is to provide a single tool and platform to unify all of these so
-that there is a single, gentle learning curve.
-However, it is not our goal to hide or replace these underlying components.
-Advanced users can use them directly, but new users aren't forced to, which
-helps them get up and running with less effort and training.
-Calkit should help users understand what is going on under the hood without
-forcing them to work at that lower level of abstraction.
-
-## How it works
-
-Calkit creates a simple human-readable "database" inside the `calkit.yaml`
-file, which serves as a way to store important information about the project,
-e.g., what question(s) it seeks to answer,
-what files should be considered datasets, figures, publications, etc.
-The Calkit cloud reads this database and registers the various entities
-as part of the entire ecosystem such that if a project is made public,
-other researchers can find and reuse your work to accelerate their own.
-
 ## Design/UX principles
 
 1. Be opinionated. Users should not be forced to make unimportant decisions.

calkit_python-0.22.3/calkit/__init__.py

@@ -0,0 +1,17 @@
+__version__ = "0.22.3"
+
+from .core import *  # noqa: F403, I001
+from . import git  # noqa: F401
+from . import dvc  # noqa: F401
+from . import cloud  # noqa: F401
+from . import jupyter  # noqa: F401
+from . import config  # noqa: F401
+from . import models  # noqa: F401
+from . import office  # noqa: F401
+from . import templates  # noqa: F401
+from . import conda  # noqa: F401
+from . import calc  # noqa: F401
+from . import check  # noqa: F401
+from . import github  # noqa: F401
+from . import zenodo  # noqa: F401
+from . import releases  # noqa: F401

{calkit_python-0.22.2 → calkit_python-0.22.3}/calkit/check.py

@@ -6,8 +6,8 @@ from typing import Callable
 import git
 from git.exc import InvalidGitRepositoryError
 from pydantic import BaseModel, computed_field
-import calkit
 
+import calkit
 
 INSTRUCTIONS_NOTE = (
     "Note that these could be as simple as telling the user to "
@@ -194,7 +194,7 @@ def check_reproducibility(
     readme_path = os.path.join(wdir, "README.md")
     if os.path.isfile(readme_path):
         res["has_readme"] = True
-        with open(readme_path) as f:
+        with open(readme_path, encoding="utf-8") as f:
             readme_txt = f.read().lower()
         res["instructions_in_readme"] = (
             ("getting started" in readme_txt)

{calkit_python-0.22.2 → calkit_python-0.22.3}/calkit/cli/__init__.py

@@ -1,4 +1,4 @@
-from .core import *
+from .core import *  # noqa: F403
 
 
 def run() -> None:

{calkit_python-0.22.2 → calkit_python-0.22.3}/calkit/cli/import_.py

@@ -221,11 +221,11 @@ def import_environment(
         ),
     ],
     dest_path: Annotated[
-        str,
+        str | None,
         typer.Option("--path", help="Output path at which to save."),
     ] = None,
     dest_name: Annotated[
-        str,
+        str | None,
         typer.Option(
             "--name", "-n", help="Name to use in the destination project."
         ),
@@ -254,15 +254,15 @@ def import_environment(
         raise_error("Invalid source environment specification")
     if os.path.isdir(project):
         typer.echo(f"Importing from local project directory: {project}")
-        src_ck_info = calkit.load_calkit_info(
-            wdir=project, process_includes=True
+        src_ck_info = dict(
+            calkit.load_calkit_info(wdir=project, process_includes=True)
         )
         environments = src_ck_info.get("environments", {})
         if env_name not in environments:
             raise_error(f"Environment {env_name} not found in project")
         src_env = environments[env_name]
         if "path" in src_env:
-            env_path = src_env["path"]
+            env_path = src_env["path"]  # noqa: F841 TODO: Use this variable
         try:
             src_project_name = calkit.detect_project_name(project)
         except Exception as e:
@@ -270,7 +270,7 @@ def import_environment(
     else:
         typer.echo("Importing from Cloud project")
         try:
-            resp = calkit.cloud.get(
+            resp = calkit.cloud.get(  # noqa: F841 TODO: Use this variable
                 f"/projects/{project}/environments/{env_name}"
             )
         except Exception as e:
@@ -278,7 +278,7 @@ def import_environment(
         src_project_name = project
         # TODO: Parse information we need from the response
     # Write environment into current Calkit info
-    ck_info = calkit.load_calkit_info()
+    ck_info = dict(calkit.load_calkit_info())
     environments = ck_info.get("environments", {})
     # Check if an environment with this name already exists
     if dest_name is None:

{calkit_python-0.22.2 → calkit_python-0.22.3}/calkit/cli/main.py

@@ -10,6 +10,7 @@ import posixpath
 import subprocess
 import sys
 import time
+from pathlib import PurePath
 
 import dotenv
 import dvc.repo
@@ -632,6 +633,9 @@ def ignore(
 ):
     """Ignore a file, i.e., keep it out of version control."""
     repo = git.Repo()
+    path = PurePath(
+        path
+    ).as_posix()  # gitignore expects / (not \) regardless of OS
     if repo.ignored(path):
         typer.echo(f"{path} is already ignored")
         return

{calkit_python-0.22.2 → calkit_python-0.22.3}/calkit/cli/new.py

@@ -1474,16 +1474,17 @@ def new_stage(
     ] = False,
 ):
     """Create a new pipeline stage."""
-    ck_info = calkit.load_calkit_info(process_includes="environments")
+    ck_info = dict(calkit.load_calkit_info(process_includes="environments"))
+    environments = ck_info.get("environments", {})
     if environment is None:
         warn("No environment is specified")
         cmd = ""
     else:
-        if environment not in ck_info["environments"] and not no_check:
+        if environment not in environments and not no_check:
             raise_error(f"Environment '{environment}' does not exist")
         cmd = f"calkit xenv -n {environment} -- "
         # Add environment path as a dependency if applicable
-        env_path = ck_info["environments"].get(environment, {}).get("path")
+        env_path = environments.get(environment, {}).get("path")
         if env_path is not None and env_path not in deps:
             deps = [env_path] + deps
     if not os.path.exists(target) and not no_check:

{calkit_python-0.22.2 → calkit_python-0.22.3}/calkit/releases.py

@@ -152,8 +152,10 @@ def populate_dvc_cache():
             if not os.path.isfile(dvc_cache_fpath):
                 # TODO: Download file from Zenodo and save in the cache
                 release_fpath = f".calkit/releases/{name}/files/{obj['path']}"
+                print(release_fpath)
                 zip_file = obj.get("zipfile")
                 if zip_file is not None:
                     zip_fpath = f".calkit/releases/{name}/files/{zip_file}"
+                    print(zip_fpath)
                 # Extract out of ZIP file if necessary
                 # TODO: Check MD5 before inserting into the cache

calkit_python-0.22.3/calkit/templates/__init__.py

@@ -0,0 +1 @@
+from .core import *  # noqa: F403

calkit_python-0.22.3/calkit/templates/latex/__init__.py

@@ -0,0 +1 @@
+from .core import *  # noqa: F403

{calkit_python-0.22.2 → calkit_python-0.22.3}/calkit/tests/test_calc.py

@@ -43,7 +43,7 @@ def test_formula():
 
 
 def test_lookuptable():
-    calc = calkit.calc.LookupTable(
+    _ = calkit.calc.LookupTable(
         inputs=["x"],
         output="something",
         params=calkit.calc.LookupTableParams(

calkit_python-0.22.3/docs/index.md

@@ -0,0 +1,60 @@
+# Home
+
+Calkit is a language-agnostic project framework and toolkit
+to make your research or analytics project
+reproducible to the highest standard,
+which means:
+
+> Inputs and process definitions are provided and sufficiently described
+> such that anyone can easily verify that they produced the outputs
+> used to support the conclusions.
+
+"Easily" means that after obtaining your project files,
+it should only require executing a single command
+(like "pressing a single button" in
+[Claerbout and Karrenbach (1992)](https://doi.org/10.1190/1.1822162)),
+which should finish in less than 15 minutes
+(suggested by
+[Vandewalle et al. (2009)](https://doi.org/10.1109/MSP.2009.932122)).
+
+If the processes are too expensive to rerun in under 15 minutes,
+it should be possible to confirm that none of the input data
+or process definitions (e.g., environment specifications, scripts)
+have changed since saving the current versions of each output artifact
+(figure, table, dataset, publication, etc.)
+
+When your project is reproducible,
+you'll be able to iterate more quickly and more often,
+easily onboard collaborators,
+make fewer mistakes,
+and feel confident sharing all of your project materials
+with your research articles,
+because you'll know the code will actually run!
+This will allow others to reuse parts of your project in their own research,
+accelerating the pace of discovery.
+
+Working at this level of automation, discipline, and rigor may sound like
+a lot of effort,
+but Calkit makes it easy!
+
+## Features
+
+- A schema to store structured metadata describing the
+  project's important outputs (in its `calkit.yaml` file)
+  and how they are created
+  (its computational environments and pipeline).
+- A CLI to run the project's pipeline to verify it's reproducible,
+  regenerating outputs as needed and
+  ensuring all
+  computational environments (e.g., [Conda](https://docs.conda.io/en/latest/), [Docker](https://docker.com)) match their specification.
+- A command line interface (CLI) to simplify keeping code, text, and larger
+  data files backed up in the same project repo using both
+  [Git](https://git-scm.com/) and [DVC](https://dvc.org/).
+- A complementary
+  [cloud system](https://github.com/calkit/calkit-cloud)
+  to facilitate backup, collaboration,
+  and sharing throughout the entire research lifecycle.
+
+## Installation
+
+See [installation](installation.md).

{calkit_python-0.22.2 → calkit_python-0.22.3}/pyproject.toml

@@ -90,7 +90,7 @@ warn_unused_ignores = true
 show_error_codes = true
 
 [tool.ruff]
-target-version = "py39"
+target-version = "py310"
 line-length = 79
 fix = true
 extend-select = ["I"]

calkit_python-0.22.2/calkit/__init__.py

@@ -1,17 +0,0 @@
-__version__ = "0.22.2"
-
-from .core import *
-from . import git
-from . import dvc
-from . import cloud
-from . import jupyter
-from . import config
-from . import models
-from . import office
-from . import templates
-from . import conda
-from . import calc
-from . import check
-from . import github
-from . import zenodo
-from . import releases

calkit_python-0.22.2/calkit/templates/__init__.py

@@ -1 +0,0 @@
-from .core import *

calkit_python-0.22.2/calkit/templates/latex/__init__.py

@@ -1 +0,0 @@
-from .core import *

calkit_python-0.22.2/docs/index.md

@@ -1,51 +0,0 @@
-# Home
-
-Calkit is an open source
-framework and toolkit for reproducible research projects.
-It acts as a top-level layer to integrate and simplify the use of enabling
-technologies such as
-[Git](https://git-scm.com/),
-[DVC](https://dvc.org/),
-[Conda](https://docs.conda.io/en/latest/),
-and [Docker](https://docker.com).
-Calkit also adds a domain-specific data model
-such that all aspects of the research process can be fully described in a
-single repository and therefore easily consumed by others.
-
-Our goal is to make reproducibility easier so it becomes more common.
-To do this, we try to make it easy for users to follow two simple rules:
-
-1. **Keep everything in version control.** This includes large files like
-   datasets, enabled by DVC.
-   The [Calkit Cloud](https://github.com/calkit/calkit-cloud),
-   hosted at [calkit.io](https://calkit.io),
-   serves as a simple default DVC remote storage location for those who do not
-   want to manage their own infrastructure.
-2. **Generate all important artifacts with a single pipeline.** There should be
-   no special instructions required to reproduce a project's artifacts.
-   It should be as simple as calling `calkit run`.
-   The DVC pipeline (in a project's `dvc.yaml` file) is therefore the main
-   thing to "build" throughout a research project.
-   Calkit provides helper functionality to build pipeline stages that
-   keep computational environments up-to-date and label their outputs for
-   convenient reuse.
-
-## Features
-
-- A [version control interface](version-control.md)
-  that unifies and simplifies interaction with Git and DVC.
-- Automated [environment management](environments.md).
-- A [project metadata model](calkit-yaml.md)
-  to declare global dependencies, environments,
-  and artifacts like datasets, figures, notebooks, and publications
-  to facilitate searchability and reuse.
-- A complementary [cloud platform](https://calkit.io) to interact with
-  the project and its artifacts, which also serves as a DVC remote.
-- Templates for projects, publications, and more.
-- The ability to declare, execute, and track
-  [manual procedures](tutorials/procedures.md) and
-  pipeline stages with [manual steps](pipeline/manual-steps.md).
-- A Jupyter cell magic to
-  [use notebook cells as pipeline stages](tutorials/notebook-pipeline.md).
-- Tools to help improve the reproducibility of workflows that depend on
-  [Microsoft Office](tutorials/office.md).