lightcone-cli 0.4.2__tar.gz → 0.5.0rc2__tar.gz

lightcone_cli-0.5.0rc2/PKG-INFO

@@ -0,0 +1,83 @@
+Metadata-Version: 2.5
+Name: lightcone-cli
+Version: 0.5.0rc2
+Summary: Lightcone Research's agentic execution layer
+Project-URL: Repository, https://github.com/LightconeResearch/lightcone-cli
+Author: Lightcone Research
+License: BSD-3-Clause
+License-File: LICENSE
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering
+Requires-Python: >=3.11
+Requires-Dist: astra-tools==0.2.17
+Requires-Dist: click>=8.0
+Requires-Dist: distributed>=2026.7
+Requires-Dist: git-annex>=10.2026
+Requires-Dist: rich>=13.0
+Requires-Dist: rocrate>=0.15
+Description-Content-Type: text/markdown
+
+# lightcone-cli
+
+[![License](https://img.shields.io/badge/License-BSD_3--Clause-426b78.svg?style=flat)](https://opensource.org/licenses/BSD-3-Clause)
+[![Python](https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13-4e5a70?style=flat)](https://pypi.org/project/lightcone-cli/)
+[![PyPI](https://img.shields.io/pypi/v/lightcone-cli?style=flat&color=f8f7f3)](https://pypi.org/project/lightcone-cli/)
+[![Tests](https://img.shields.io/github/actions/workflow/status/LightconeResearch/lightcone-cli/tests.yml?style=flat&color=darkgreen)](https://github.com/LightconeResearch/lightcone-cli/actions/workflows/tests.yml)
+
+**lightcone-cli** (`lc`) is the execution layer for
+[ASTRA](https://astra-spec.org/latest/) (Agentic Schema for Transparent
+Research Analysis). Describe your analysis in an `astra.yaml`
+specification and `lc` takes care of the rest — execution, environments,
+and provenance.
+
+## Quick Start
+
+**lightcone-cli** only requires you to have `uv` installed on your environment, and will take care of everything else. See how to install uv here: [https://docs.astral.sh/uv/getting-started/installation](https://docs.astral.sh/uv/getting-started/installation). 
+
+Then to install **lightcone-cli**:
+```bash
+uv tool install lightcone-cli
+```
+
+Once the CLI is installed, you can use it to create an ASTRA project and generate outputs like so:
+
+```bash
+lc init my-analysis
+cd my-analysis
+# describe your analysis in astra.yaml, write your scripts,
+# declare what they import through normal uv interactions:
+uv add numpy
+# When you are done with your edits, commit:
+git add -A && git commit -m "First analysis"
+# Use the lightcone CLI to generate your outputs with full provenance tracking
+lc materialize
+```
+
+ASTRA specs are plain, structured YAML — they work well hand-written or
+drafted with any AI coding assistant.
+
+→ [Full getting-started guide](https://docs.lightconeresearch.org/user/getting-started/)
+
+## Capabilities
+
+- **Multiverse analysis** — declare methodological decisions with multiple defensible options; `lc` materializes your analysis across every universe you define
+- **Provenance by construction** — every output is committed to git together with a content-addressed manifest and a re-runnable run record; git-annex carries the bytes, so results travel with the repository
+- **Locked, isolated execution** — a project's environment is `pyproject.toml` + `uv.lock`; recipes run in it under a sandbox (Landlock on Linux, Seatbelt on macOS) that keeps undeclared files out and stray writes contained
+- **Containers and HPC** — declare `[tool.lightcone.image]` and recipes run in a content-addressed image archived in the repository itself; a SLURM allocation is detected and used automatically, every node included
+- **Publication view** — declare a license and `lc materialize` maintains an [RO-Crate](https://www.researchobject.org/ro-crate/) of the project and its provenance, ready to archive or deposit
+
+→ [Full documentation](https://docs.lightconeresearch.org)
+
+## License
+
+BSD 3-Clause — see [LICENSE](LICENSE) for details.

lightcone_cli-0.5.0rc2/README.md

@@ -0,0 +1,54 @@
+# lightcone-cli
+
+[![License](https://img.shields.io/badge/License-BSD_3--Clause-426b78.svg?style=flat)](https://opensource.org/licenses/BSD-3-Clause)
+[![Python](https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13-4e5a70?style=flat)](https://pypi.org/project/lightcone-cli/)
+[![PyPI](https://img.shields.io/pypi/v/lightcone-cli?style=flat&color=f8f7f3)](https://pypi.org/project/lightcone-cli/)
+[![Tests](https://img.shields.io/github/actions/workflow/status/LightconeResearch/lightcone-cli/tests.yml?style=flat&color=darkgreen)](https://github.com/LightconeResearch/lightcone-cli/actions/workflows/tests.yml)
+
+**lightcone-cli** (`lc`) is the execution layer for
+[ASTRA](https://astra-spec.org/latest/) (Agentic Schema for Transparent
+Research Analysis). Describe your analysis in an `astra.yaml`
+specification and `lc` takes care of the rest — execution, environments,
+and provenance.
+
+## Quick Start
+
+**lightcone-cli** only requires you to have `uv` installed on your environment, and will take care of everything else. See how to install uv here: [https://docs.astral.sh/uv/getting-started/installation](https://docs.astral.sh/uv/getting-started/installation). 
+
+Then to install **lightcone-cli**:
+```bash
+uv tool install lightcone-cli
+```
+
+Once the CLI is installed, you can use it to create an ASTRA project and generate outputs like so:
+
+```bash
+lc init my-analysis
+cd my-analysis
+# describe your analysis in astra.yaml, write your scripts,
+# declare what they import through normal uv interactions:
+uv add numpy
+# When you are done with your edits, commit:
+git add -A && git commit -m "First analysis"
+# Use the lightcone CLI to generate your outputs with full provenance tracking
+lc materialize
+```
+
+ASTRA specs are plain, structured YAML — they work well hand-written or
+drafted with any AI coding assistant.
+
+→ [Full getting-started guide](https://docs.lightconeresearch.org/user/getting-started/)
+
+## Capabilities
+
+- **Multiverse analysis** — declare methodological decisions with multiple defensible options; `lc` materializes your analysis across every universe you define
+- **Provenance by construction** — every output is committed to git together with a content-addressed manifest and a re-runnable run record; git-annex carries the bytes, so results travel with the repository
+- **Locked, isolated execution** — a project's environment is `pyproject.toml` + `uv.lock`; recipes run in it under a sandbox (Landlock on Linux, Seatbelt on macOS) that keeps undeclared files out and stray writes contained
+- **Containers and HPC** — declare `[tool.lightcone.image]` and recipes run in a content-addressed image archived in the repository itself; a SLURM allocation is detected and used automatically, every node included
+- **Publication view** — declare a license and `lc materialize` maintains an [RO-Crate](https://www.researchobject.org/ro-crate/) of the project and its provenance, ready to archive or deposit
+
+→ [Full documentation](https://docs.lightconeresearch.org)
+
+## License
+
+BSD 3-Clause — see [LICENSE](LICENSE) for details.

lightcone_cli-0.5.0rc2/pyproject.toml

@@ -0,0 +1,111 @@
+[project]
+name = "lightcone-cli"
+dynamic = ["version"]
+description = "Lightcone Research's agentic execution layer"
+readme = "README.md"
+license = { text = "BSD-3-Clause" }
+requires-python = ">=3.11"
+authors = [
+    { name = "Lightcone Research" }
+]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Environment :: Console",
+    "Intended Audience :: Science/Research",
+    "License :: OSI Approved :: BSD License",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3.14",
+    "Programming Language :: Python :: 3 :: Only",
+    "Topic :: Scientific/Engineering",
+]
+
+dependencies = [
+    "astra-tools==0.2.17",
+    "click>=8.0",
+    "rich>=13.0",
+    "git-annex>=10.2026",
+    "distributed>=2026.7",
+    "rocrate>=0.15",
+]
+
+[dependency-groups]
+dev = [
+    "pytest>=8.0",
+    "pytest-cov",
+    "ruff",
+    "mypy",
+    "datalad",
+    "roc-validator>=0.11.3",
+]
+docs = [
+    "zensical>=0.0.33",
+    # squidfunk's mike fork — required by zensical's versioning provider.
+    # Not on PyPI; install from git.
+    "mike @ git+https://github.com/squidfunk/mike.git ; python_version >= '3.10'",
+]
+
+
+[project.urls]
+Repository = "https://github.com/LightconeResearch/lightcone-cli"
+
+[project.scripts]
+lc = "lightcone.cli:main"
+# The git-annex wheel's own entry points, re-declared verbatim: an
+# installer links only the requested package's executables, and a plain
+# `uv tool install lightcone-cli` must put git-annex on the shell's PATH
+# — `filter=annex` makes the researcher's own `git add` need it. The
+# suite pins these against the wheel's, so an upstream change fails
+# loudly here rather than silently at install time.
+git-annex = "git_annex:cli"
+git-annex-shell = "git_annex:cli"
+git-remote-annex = "git_annex:cli"
+git-remote-tor-annex = "git_annex:cli"
+
+[build-system]
+requires = ["hatchling", "hatch-vcs"]
+build-backend = "hatchling.build"
+
+[tool.hatch.version]
+source = "vcs"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/lightcone"]
+
+[tool.hatch.build.targets.sdist]
+include = ["src/lightcone"]
+
+[tool.ruff]
+target-version = "py311"
+line-length = 100
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "N", "W", "UP"]
+
+[tool.mypy]
+python_version = "3.11"
+strict = true
+namespace_packages = true
+explicit_package_bases = true
+mypy_path = "src"
+
+[[tool.mypy.overrides]]
+module = ["astra.*", "rocrate.*"]
+ignore_missing_imports = true
+follow_untyped_imports = true
+
+# rocrate ships no type information, and engine/crate.py is a client of
+# little else — one module-scoped relaxation beats an ignore on every call.
+[[tool.mypy.overrides]]
+module = ["lightcone.engine.crate"]
+disallow_untyped_calls = false
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+# The container smoke tests leave ~200 MB image archives in each test's
+# tmp_path; keeping every passing run's directories exhausts a tmpfs.
+tmp_path_retention_policy = "failed"
+tmp_path_retention_count = 1

lightcone_cli-0.5.0rc2/src/lightcone/_sandbox_exec.py

@@ -0,0 +1,328 @@
+"""The Landlock exec shim — ``python -m lightcone._sandbox_exec``.
+
+Landlock is a *self*-restriction: a process narrows its own access rights
+and can never widen them again, and the restriction is inherited by
+everything it execs. There is no way to restrict *another* process, so
+there is no ``sandbox-exec``-shaped command to wrap a recipe in. This
+module is that missing command: it reads a policy, restricts itself,
+and ``execvp``'s the real command. That is what lets the engine treat
+Landlock and Seatbelt as the same thing — a pure argv rewrite (see
+:mod:`lightcone.engine.sandbox`).
+
+The policy arrives as **JSON on argv**, not as an inherited ruleset FD —
+the shape the codex CLI uses. A Landlock FD cannot be reopened, so
+whether one would survive ``uv run``'s spawn/exec chain is a question
+serializing the policy never has to answer, and it is what keeps the
+engine's argv rewrite a pure function.
+
+Two properties of this module are load-bearing and pinned by tests:
+
+- **Stdlib only, and no lightcone imports.** ``lightcone`` is a PEP 420
+  namespace package with no ``__init__``, so ``-m lightcone._sandbox_exec``
+  executes this file and nothing else. It runs on every sandboxed exec;
+  it must not drag the engine in.
+- **It never proceeds unsandboxed.** Every setup failure exits
+  :data:`SETUP_FAILURE_EXIT` with a distinguishable message, so a broken
+  sandbox can never be mistaken for a working one (the "silent
+  best-effort" trap this design exists to avoid).
+"""
+
+from __future__ import annotations
+
+import ctypes
+import errno
+import functools
+import json
+import os
+import stat
+import sys
+
+# --- the kernel interface --------------------------------------------------
+
+# asm-generic syscall numbers — identical on x86_64 and aarch64.
+SYS_LANDLOCK_CREATE_RULESET = 444
+SYS_LANDLOCK_ADD_RULE = 445
+SYS_LANDLOCK_RESTRICT_SELF = 446
+
+_LANDLOCK_CREATE_RULESET_VERSION = 1 << 0
+_RULE_PATH_BENEATH = 1
+_PR_SET_NO_NEW_PRIVS = 38
+
+# Access-right bits, grouped by the ABI that introduced them.
+ACCESS_FS_EXECUTE = 1 << 0
+ACCESS_FS_WRITE_FILE = 1 << 1
+ACCESS_FS_READ_FILE = 1 << 2
+ACCESS_FS_READ_DIR = 1 << 3
+_ABI1_ALL = (1 << 13) - 1  # bits 0..12: read, write, and every make/remove
+ACCESS_FS_REFER = 1 << 13  # ABI ≥ 2
+ACCESS_FS_TRUNCATE = 1 << 14  # ABI ≥ 3
+
+#: Rights the kernel accepts on a rule whose parent is a regular file.
+#: Everything else is directory-only and makes ``add_rule`` fail EINVAL.
+_FILE_ONLY_BITS = (
+    ACCESS_FS_EXECUTE | ACCESS_FS_WRITE_FILE | ACCESS_FS_READ_FILE | ACCESS_FS_TRUNCATE
+)
+
+READ_BITS = ACCESS_FS_READ_FILE | ACCESS_FS_READ_DIR
+EXEC_BITS = ACCESS_FS_EXECUTE | ACCESS_FS_READ_FILE
+
+#: Landlock's syscall numbers are asm-generic, but the ABI is not defined
+#: for every architecture; refuse rather than issue syscall 444 blind.
+_SUPPORTED_ARCHES = frozenset({"x86_64", "aarch64", "arm64"})
+
+#: Reserved exit code for a failure to *set up* the sandbox — never a
+#: command's own. The engine renders it as an lc problem, not a recipe's.
+SETUP_FAILURE_EXIT = 97
+
+class _RulesetAttr(ctypes.Structure):
+    _fields_ = [
+        ("handled_access_fs", ctypes.c_uint64),
+        ("handled_access_net", ctypes.c_uint64),
+    ]
+
+
+class _PathBeneathAttr(ctypes.Structure):
+    # The kernel struct is packed (u64 followed by s32). `_layout_` is the
+    # 3.14 spelling and is ignored by older ctypes, which honor `_pack_`.
+    _pack_ = 1
+    _layout_ = "ms"
+    _fields_ = [
+        ("allowed_access", ctypes.c_uint64),
+        ("parent_fd", ctypes.c_int32),
+    ]
+
+
+@functools.cache
+def _libc() -> ctypes.CDLL:
+    """The already-loaded libc.
+
+    Cached, and `CDLL(None)` rather than `find_library("c")`: this is
+    called once per rule, and `find_library` forks `ldconfig -p` every
+    time — 74 subprocesses for a 71-path policy, which measured at ~740 ms
+    of the shim's ~780 ms. `dlopen(NULL)` reaches the libc already mapped
+    into this process, needs no search, and drops `ctypes.util` (and the
+    `tempfile`/`random` chain behind it) off the import path.
+    """
+    return ctypes.CDLL(None, use_errno=True)
+
+
+def abi() -> int:
+    """Probe the Landlock ABI this kernel supports.
+
+    Returns:
+        The ABI level, or 0 if the kernel supports none.
+    """
+    if os.uname().machine not in _SUPPORTED_ARCHES:
+        return 0
+    try:
+        result = _libc().syscall(
+            SYS_LANDLOCK_CREATE_RULESET, None, ctypes.c_size_t(0), ctypes.c_uint32(
+                _LANDLOCK_CREATE_RULESET_VERSION
+            )
+        )
+    except OSError:  # pragma: no cover - no libc to load
+        return 0
+    return int(result) if result > 0 else 0
+
+
+def handled_access(abi_level: int) -> int:
+    """Build the mask of rights the ruleset declares it governs.
+
+    Args:
+        abi_level: The ABI reported by :func:`abi`.
+
+    Returns:
+        Every right this ABI knows about. A right the kernel does not
+        know is a ruleset it rejects outright.
+    """
+    handled = _ABI1_ALL
+    if abi_level >= 2:
+        handled |= ACCESS_FS_REFER
+    if abi_level >= 3:
+        handled |= ACCESS_FS_TRUNCATE
+    return handled
+
+
+def write_bits(abi_level: int) -> int:
+    """Build the mask a writable root is granted.
+
+    Args:
+        abi_level: The ABI reported by :func:`abi`.
+
+    Returns:
+        Everything but EXECUTE, which is granted per file.
+    """
+    return (handled_access(abi_level) & ~ACCESS_FS_EXECUTE) | READ_BITS
+
+
+def create_ruleset(handled_fs: int) -> int:
+    """Create an empty Landlock ruleset.
+
+    Args:
+        handled_fs: The rights the ruleset governs.
+
+    Returns:
+        A ruleset file descriptor.
+
+    Raises:
+        OSError: If the kernel refuses the ruleset.
+    """
+    attr = _RulesetAttr(handled_fs, 0)
+    fd = _libc().syscall(
+        SYS_LANDLOCK_CREATE_RULESET,
+        ctypes.byref(attr),
+        ctypes.c_size_t(ctypes.sizeof(attr)),
+        ctypes.c_uint32(0),
+    )
+    if fd < 0:
+        raise OSError(ctypes.get_errno(), "landlock_create_ruleset failed")
+    return int(fd)
+
+
+def add_path_rule(ruleset_fd: int, path: str, access: int) -> None:
+    """Grant access beneath a path.
+
+    Args:
+        ruleset_fd: The ruleset to add to.
+        path: The directory or file to grant on.
+        access: The rights to grant, masked to what the path can bear.
+
+    Raises:
+        OSError: If the path cannot be opened or the rule is refused.
+    """
+    parent_fd = os.open(path, os.O_PATH | os.O_CLOEXEC)
+    try:
+        # `stat.S_ISDIR`, not a bitwise test: S_IFSOCK (0o140000) and
+        # S_IFBLK (0o060000) both carry the S_IFDIR bit, so a hand-rolled
+        # mask calls a socket a directory, leaves directory-only rights
+        # unmasked, and turns a declared unix-socket input into EINVAL —
+        # surfacing as a sandbox *setup* failure rather than a grant.
+        if not stat.S_ISDIR(os.fstat(parent_fd).st_mode):
+            access &= _FILE_ONLY_BITS
+        if not access:
+            return
+        attr = _PathBeneathAttr(access, parent_fd)
+        result = _libc().syscall(
+            SYS_LANDLOCK_ADD_RULE,
+            ctypes.c_int(ruleset_fd),
+            ctypes.c_uint32(_RULE_PATH_BENEATH),
+            ctypes.byref(attr),
+            ctypes.c_uint32(0),
+        )
+        if result != 0:
+            code = ctypes.get_errno()
+            raise OSError(code, f"landlock_add_rule({path}): {errno.errorcode.get(code, code)}")
+    finally:
+        os.close(parent_fd)
+
+
+def build_ruleset(policy: dict[str, object], abi_level: int) -> int:
+    """Build a ruleset realizing a policy document.
+
+    Args:
+        policy: The read, write and execute path lists from argv.
+        abi_level: The ABI reported by :func:`abi`.
+
+    Returns:
+        A ruleset file descriptor.
+
+    Raises:
+        OSError: If the ruleset cannot be created or a rule refused.
+    """
+    fd = create_ruleset(handled_access(abi_level))
+    try:
+        for key, access in (
+            ("read", READ_BITS),
+            ("write", write_bits(abi_level)),
+            ("execute", EXEC_BITS),
+        ):
+            for path in _paths(policy, key):
+                try:
+                    add_path_rule(fd, path, access)
+                except FileNotFoundError:
+                    continue
+    except BaseException:
+        os.close(fd)
+        raise
+    return fd
+
+
+def restrict_self(ruleset_fd: int) -> None:
+    """Apply a ruleset to this process, irreversibly.
+
+    Args:
+        ruleset_fd: The ruleset to enter.
+
+    Raises:
+        OSError: If ``no_new_privs`` or the restriction is refused.
+    """
+    libc = _libc()
+    if libc.prctl(_PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) != 0:
+        raise OSError(ctypes.get_errno(), "prctl(PR_SET_NO_NEW_PRIVS) failed")
+    if libc.syscall(SYS_LANDLOCK_RESTRICT_SELF, ctypes.c_int(ruleset_fd), ctypes.c_uint32(0)) != 0:
+        raise OSError(ctypes.get_errno(), "landlock_restrict_self failed")
+
+
+# --- the entry point -------------------------------------------------------
+
+
+def _paths(policy: dict[str, object], key: str) -> list[str]:
+    value = policy.get(key, [])
+    if not isinstance(value, list) or not all(isinstance(p, str) for p in value):
+        raise ValueError(f"policy field {key!r} must be a list of strings")
+    return list(value)
+
+
+def _fail(message: str) -> None:
+    sys.stderr.write(f"lc sandbox setup failed: {message}\n")
+    raise SystemExit(SETUP_FAILURE_EXIT)
+
+
+def main(argv: list[str] | None = None) -> None:
+    """Restrict this process, then exec the command it was given.
+
+    ``--policy <json> -- <command>...``. Never falls through to running
+    the command unsandboxed: a setup failure exits with a reserved code
+    that nothing a command could return will collide with.
+
+    Args:
+        argv: Defaults to the process arguments.
+    """
+    args = list(sys.argv[1:] if argv is None else argv)
+    if len(args) < 2 or args[0] != "--policy":
+        _fail("usage: -m lightcone._sandbox_exec --policy <json> -- <command>...")
+    raw = args[1]
+    rest = args[2:]
+    if not rest or rest[0] != "--":
+        _fail("missing `--` before the command")
+    command = rest[1:]
+    if not command:
+        _fail("no command after `--`")
+
+    try:
+        policy = json.loads(raw)
+        if not isinstance(policy, dict):
+            raise ValueError("policy must be a JSON object")
+        abi_level = abi()
+        if abi_level == 0:
+            raise ValueError("landlock unavailable (kernel < 5.13, or blocked by seccomp)")
+        fd = build_ruleset(policy, abi_level)
+        restrict_self(fd)
+        os.close(fd)
+    except (ValueError, OSError) as e:
+        _fail(str(e))
+
+    try:
+        os.execvp(command[0], command)
+    except OSError as e:
+        # Not a setup failure: we are already restricted, so this is very
+        # likely the sandbox denying the exec. Report it the way a shell
+        # would — 126 "cannot execute", 127 "not found" — so the denial
+        # classifier sees a message and an exit code it recognizes, and
+        # exit 97 keeps meaning "lc's own sandbox setup broke".
+        sys.stderr.write(f"lc sandbox: {command[0]}: {e.strerror}\n")
+        raise SystemExit(127 if e.errno == errno.ENOENT else 126) from e
+
+
+if __name__ == "__main__":
+    main()

{lightcone_cli-0.4.2 → lightcone_cli-0.5.0rc2}/src/lightcone/cli/__init__.py

@@ -2,15 +2,13 @@
 
 from __future__ import annotations
 
-try:
-    from importlib.metadata import version
-
-    __version__ = version("lightcone-cli")
-except Exception:
-    __version__ = "0.0.0.dev"
-
 
 def main() -> None:
+    """Run the CLI.
+
+    The console-script entry point. Imports the command module lazily, so
+    the cost of click and the engine is paid only once a command runs.
+    """
     from lightcone.cli.commands import main as _main
 
     _main()