nbdantic 0.1.0__tar.gz

nbdantic-0.1.0/PKG-INFO

@@ -0,0 +1,109 @@
+Metadata-Version: 2.4
+Name: nbdantic
+Version: 0.1.0
+Requires-Dist: ansiwrap>=0.8.4
+Requires-Dist: nbformat>=5.10.4
+Requires-Dist: tomlkit>=0.13.0
+Summary: notebook parsing minimum DSL
+Requires-Python: >=3.13
+Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
+Project-URL: Documentation, https://github.com/ivanbelenky/nbdantic#readme
+Project-URL: Homepage, https://github.com/ivanbelenky/nbdantic
+Project-URL: Issues, https://github.com/ivanbelenky/nbdantic/issues
+Project-URL: Repository, https://github.com/ivanbelenky/nbdantic
+
+# nbdantic ◬
+
+fully typed context-free grammar for jupyter notebooks. declare expected structure, validate against it.
+
+## install
+
+```
+uv add nbdantic
+```
+
+## grammar
+
+| production | rule | note |
+|---|---|---|
+| Notebook | Cell\* Sequence Cell_terminal? | |
+| Sequence | (Element …)+ \| ε | `empty=True` allows ε |
+| Element | Markdown \| Code \| Sequence \| Choice \| Maybe | |
+| Markdown | cell\<markdown\> | terminal |
+| Code | cell\<code\> | terminal |
+| Choice | Element \| Element \| … | first match wins |
+| Maybe | Element \| ε | |
+
+### grammar elements
+
+```python
+
+type ParseElement = Markdown | Code | Sequence | Choice | Maybe
+
+# Markdown and Code are self explanatory
+# Sequence is None one or more elements (enabling nesting of structures/sequences)
+# Choice lets you pick from a set of Elements (it could be arbitrary sets of nested pattenrs)
+# Maybe allows for expressing Optiona<Element>
+```
+
+
+### usage
+
+```python
+import ast # for custom validation function
+
+from nbdantic import Code, Markdown, Maybe, Sequence, JupyterNotebook
+from nbdantic.validators import valid_python, not_empty, at_least
+
+
+def only_imports(c):
+    tree = ast.parse(c)
+    if not all(
+        isinstance(n, (ast.Import, ast.ImportFrom, ast.alias, ast.Module)) for n in ast.walk(tree)
+    ):
+        raise ValueError("code should only contain import statements")
+
+
+class Paper(JupyterNotebook):
+    structure = Sequence("root", [
+        Markdown("title", validators=[not_empty]),
+        Code("imports", validators=[valid_python, only_imports]),
+        Maybe("abstract", Markdown("abstract_md")),
+        Sequence("sections", [
+            Markdown("heading"),
+            Code("body", validators=[valid_python]),
+        ], validators=[at_least(2)], empty=True),
+    ])
+
+result = Paper.from_file("paper.ipynb").validate()
+result.raise_if_failed()
+result.cells_map
+```
+
+
+### extra features
+
+#### built-in validators
+
+`nbdantic.validators` has some common checks, some of them are simple and example like, some of them may prove useful, like
+- `ty_check`
+- `ruff_check`
+- `line_warning`
+
+
+#### script runner
+
+`nbdantic.script_runner.UVScriptRunner` spins up isolated uv environments for running code, linting, type checking, whatever you need. validators like `ruff_check` and `ty_check` use it under the hood.
+
+```python
+from nbdantic.script_runner import UVScriptRunner
+
+with UVScriptRunner(packages=["numpy>=1.24"], python_version="3.13") as runner:
+    result = runner.oneshot_python("import numpy; print(numpy.__version__)")
+    print(result.stdout)
+```
+
+
+### notes
+pep-508/440 parsing backed by [uv-pep508 & uv-pep440 crates](https://github.com/astral-sh/uv) via pyo3. Thanks uv for everything.
+

nbdantic-0.1.0/README.md

@@ -0,0 +1,94 @@
+# nbdantic ◬
+
+fully typed context-free grammar for jupyter notebooks. declare expected structure, validate against it.
+
+## install
+
+```
+uv add nbdantic
+```
+
+## grammar
+
+| production | rule | note |
+|---|---|---|
+| Notebook | Cell\* Sequence Cell_terminal? | |
+| Sequence | (Element …)+ \| ε | `empty=True` allows ε |
+| Element | Markdown \| Code \| Sequence \| Choice \| Maybe | |
+| Markdown | cell\<markdown\> | terminal |
+| Code | cell\<code\> | terminal |
+| Choice | Element \| Element \| … | first match wins |
+| Maybe | Element \| ε | |
+
+### grammar elements
+
+```python
+
+type ParseElement = Markdown | Code | Sequence | Choice | Maybe
+
+# Markdown and Code are self explanatory
+# Sequence is None one or more elements (enabling nesting of structures/sequences)
+# Choice lets you pick from a set of Elements (it could be arbitrary sets of nested pattenrs)
+# Maybe allows for expressing Optiona<Element>
+```
+
+
+### usage
+
+```python
+import ast # for custom validation function
+
+from nbdantic import Code, Markdown, Maybe, Sequence, JupyterNotebook
+from nbdantic.validators import valid_python, not_empty, at_least
+
+
+def only_imports(c):
+    tree = ast.parse(c)
+    if not all(
+        isinstance(n, (ast.Import, ast.ImportFrom, ast.alias, ast.Module)) for n in ast.walk(tree)
+    ):
+        raise ValueError("code should only contain import statements")
+
+
+class Paper(JupyterNotebook):
+    structure = Sequence("root", [
+        Markdown("title", validators=[not_empty]),
+        Code("imports", validators=[valid_python, only_imports]),
+        Maybe("abstract", Markdown("abstract_md")),
+        Sequence("sections", [
+            Markdown("heading"),
+            Code("body", validators=[valid_python]),
+        ], validators=[at_least(2)], empty=True),
+    ])
+
+result = Paper.from_file("paper.ipynb").validate()
+result.raise_if_failed()
+result.cells_map
+```
+
+
+### extra features
+
+#### built-in validators
+
+`nbdantic.validators` has some common checks, some of them are simple and example like, some of them may prove useful, like
+- `ty_check`
+- `ruff_check`
+- `line_warning`
+
+
+#### script runner
+
+`nbdantic.script_runner.UVScriptRunner` spins up isolated uv environments for running code, linting, type checking, whatever you need. validators like `ruff_check` and `ty_check` use it under the hood.
+
+```python
+from nbdantic.script_runner import UVScriptRunner
+
+with UVScriptRunner(packages=["numpy>=1.24"], python_version="3.13") as runner:
+    result = runner.oneshot_python("import numpy; print(numpy.__version__)")
+    print(result.stdout)
+```
+
+
+### notes
+pep-508/440 parsing backed by [uv-pep508 & uv-pep440 crates](https://github.com/astral-sh/uv) via pyo3. Thanks uv for everything.

nbdantic-0.1.0/pyproject.toml

@@ -0,0 +1,51 @@
+[build-system]
+requires = ["maturin>=1.0,<2.0"]
+build-backend = "maturin"
+
+[project]
+name = "nbdantic"
+version = "0.1.0"
+description = "notebook parsing minimum DSL"
+readme = "README.md"
+requires-python = ">=3.13"
+dependencies = [
+    "ansiwrap>=0.8.4",
+    "nbformat>=5.10.4",
+    "tomlkit>=0.13.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/ivanbelenky/nbdantic"
+Repository = "https://github.com/ivanbelenky/nbdantic"
+Issues = "https://github.com/ivanbelenky/nbdantic/issues"
+Documentation = "https://github.com/ivanbelenky/nbdantic#readme"
+
+[tool.maturin]
+manifest-path = "src-rs/Cargo.toml"
+python-source = "src"
+module-name = "nbdantic.nbrs"
+features = ["pyo3/extension-module"]
+
+[tool.ruff]
+indent-width = 4
+line-length = 120
+
+[tool.ruff.lint]
+select = ["E", "F", "A"]
+ignore = ["E501"]
+
+[dependency-groups]
+dev = [
+    "ipython>=8.31.0",
+    "coverage>=7.6.12",
+    "maturin>=1.0,<2.0",
+    "pre-commit>=4.0.1",
+    "ipykernel>=6.30.1",
+    "pyright>=1.1.407",
+    "ty>=0.0.1a7",
+    "pytest>=9.0.2",
+]
+
+
+[tool.pytest.ini_options]
+norecursedirs = ["scratch"]

nbdantic-0.1.0/src/nbdantic/__init__.py

@@ -0,0 +1,35 @@
+from .grammar import (
+    Cell,
+    Choice,
+    Code,
+    Markdown,
+    Maybe,
+    Sequence,
+)
+from .logger import set_nocolor
+from .parser import (
+    JupyterNotebook,
+    ValidationFailure,
+    ValidationResult,
+)
+from .types import (
+    CellNode,
+    CellsMap,
+    ChoiceResult,
+)
+
+__all__ = [
+    "Cell",
+    "CellNode",
+    "CellsMap",
+    "Choice",
+    "ChoiceResult",
+    "Code",
+    "JupyterNotebook",
+    "Markdown",
+    "Maybe",
+    "Sequence",
+    "ValidationFailure",
+    "ValidationResult",
+    "set_nocolor",
+]

nbdantic-0.1.0/src/nbdantic/grammar.py

@@ -0,0 +1,194 @@
+from __future__ import annotations
+
+import inspect
+from copy import deepcopy
+from textwrap import dedent
+from typing import Any, Callable, ClassVar, Final, Iterator, cast
+
+from .logger import cstr
+from .types import CellContent, CellValidator, Label, SequenceValidator
+
+
+def _validator_name(v: Callable[..., Any]) -> str:
+    return getattr(v, "__name__", type(v).__name__)
+
+
+def _check_cell_validator(v: CellValidator) -> None:
+    try:
+        sig = inspect.signature(v)
+    except (ValueError, TypeError):
+        return
+    params = [
+        p
+        for p in sig.parameters.values()
+        if p.default is inspect.Parameter.empty
+        and p.kind not in (inspect.Parameter.VAR_POSITIONAL, inspect.Parameter.VAR_KEYWORD)
+    ]
+    if len(params) != 1:
+        raise TypeError(
+            f"CellValidator '{_validator_name(v)}' must accept exactly 1 positional "
+            f"parameter (CellContent), got {len(params)}"
+        )
+
+
+def _check_sequence_validator(v: SequenceValidator) -> None:
+    try:
+        sig = inspect.signature(v)
+    except (ValueError, TypeError):
+        return
+    params = [
+        p
+        for p in sig.parameters.values()
+        if p.default is inspect.Parameter.empty
+        and p.kind not in (inspect.Parameter.VAR_POSITIONAL, inspect.Parameter.VAR_KEYWORD)
+    ]
+    if len(params) != 1:
+        raise TypeError(
+            f"SequenceValidator '{_validator_name(v)}' must accept exactly 1 positional "
+            f"parameter (SequenceContent), got {len(params)}"
+        )
+
+
+class CellValidationError(Exception):
+    def __init__(self, message: str, validator_name: str | None = None) -> None:
+        super().__init__(message)
+        self.validator_name = validator_name
+
+
+class SequenceError(Exception):
+    pass
+
+
+class Cell:
+    cell_type: ClassVar[str]
+
+    def __init__(
+        self,
+        label: Label,
+        validators: list[CellValidator] | None = None,
+        source: CellContent = "",
+    ) -> None:
+        self.label: Final[Label] = label
+        self._source = source
+        self.validators: list[CellValidator] = []
+        for v in validators or []:
+            _check_cell_validator(v)
+            self.validators.append(v)
+
+    @property
+    def content(self) -> CellContent:
+        return self._source
+
+    def validate(self, x: CellContent) -> None:
+        for v in self.validators:
+            try:
+                v(deepcopy(x))
+            except Exception as e:
+                doc = f"({v.__doc__})" if v.__doc__ else ""
+                raise CellValidationError(
+                    dedent(f"""
+                    validation failed for cell '{cstr("yellow", self.label, True)}':\n
+                    {cstr("magenta", _validator_name(v), True)} {doc} - {e!s}
+                    """),
+                    validator_name=_validator_name(v),
+                )
+
+    def __or__(self, other: Choice | Sequence | Cell | Maybe) -> Choice:
+        if isinstance(other, Choice):
+            return Choice(
+                label=f"{self.label}_or_{'_or_'.join(o.label for o in other.options)}",
+                options=cast(list, [self] + other.options),
+            )
+        return Choice(
+            label=f"{self.label}_or_{other.label}",
+            options=cast(list, [self, other]),
+        )
+
+
+class Markdown(Cell):
+    cell_type = "markdown"
+
+
+class Code(Cell):
+    cell_type = "code"
+
+
+class Choice:
+    def __init__(
+        self,
+        label: Label,
+        options: list[Markdown | Code | Sequence | Choice | Maybe],
+    ) -> None:
+        self.label: Final[Label] = label
+        self.options: list[Markdown | Code | Sequence | Choice | Maybe] = []
+        for opt in options:
+            if isinstance(opt, Choice):
+                self.options.extend(opt.options)
+            else:
+                self.options.append(opt)
+
+    def __or__(self, other: Choice | Sequence | Cell | Maybe) -> Choice:
+        if isinstance(other, Choice):
+            return Choice(
+                label=f"{self.label}_or_{other.label}",
+                options=self.options + other.options,
+            )
+        return Choice(
+            label=f"{self.label}_or_{other.label}",
+            options=cast(list, self.options + [other]),
+        )
+
+
+class Maybe:
+    def __init__(
+        self,
+        label: Label,
+        element: Markdown | Code | Sequence | Choice,
+    ) -> None:
+        self.label: Final[Label] = label
+        self.element: Final[Markdown | Code | Sequence | Choice] = element
+
+    def __or__(self, other: Choice | Sequence | Cell | Maybe) -> Choice:
+        if isinstance(other, Choice):
+            return Choice(
+                label=f"{self.label}_or_{'_or_'.join(o.label for o in other.options)}",
+                options=cast(list, [self] + other.options),
+            )
+        return Choice(
+            label=f"{self.label}_or_{other.label}",
+            options=cast(list, [self, other]),
+        )
+
+
+class Sequence:
+    def __init__(
+        self,
+        label: Label,
+        structure: list[Markdown | Code | Sequence | Choice | Maybe],
+        validators: list[SequenceValidator] | None = None,
+        empty: bool = False,
+    ) -> None:
+        self.label: Final[Label] = label
+        self.empty: Final[bool] = empty
+        self.structure: Final[list[Markdown | Code | Sequence | Choice | Maybe]] = structure
+        self.validators: list[SequenceValidator] = []
+        for v in validators or []:
+            _check_sequence_validator(v)
+            self.validators.append(v)
+
+    def __iter__(self) -> Iterator[Markdown | Code | Sequence | Choice | Maybe]:
+        yield from self.structure
+
+    def __or__(self, other: Choice | Sequence | Cell | Maybe) -> Choice:
+        if isinstance(other, Choice):
+            return Choice(
+                label=f"{self.label}_or_{'_or_'.join(o.label for o in other.options)}",
+                options=cast(list, [self] + other.options),
+            )
+        return Choice(
+            label=f"{self.label}_or_{other.label}",
+            options=cast(list, [self, other]),
+        )
+
+
+type ParseElement = Markdown | Code | Sequence | Choice | Maybe

nbdantic-0.1.0/src/nbdantic/logger.py

@@ -0,0 +1,181 @@
+import importlib
+import importlib.util
+import logging
+import re
+import sys
+from enum import Enum
+
+try:
+
+    def _find_module(module: str) -> list[str]:
+        spec = importlib.util.find_spec(module)
+        return [spec.name] if spec else []
+
+    def _load_module(_: str, module: str) -> object:
+        return importlib.import_module(module)
+
+    setattr(importlib, "find_module", _find_module)
+    setattr(importlib, "load_module", _load_module)
+    sys.modules["imp"] = importlib
+
+    from ansiwrap import wrap as _wrap
+except ImportError:
+    from textwrap import wrap as _wrap
+
+STATUS_LEN = 88
+
+_ANSI: dict[str, str] = {
+    "red": "31",
+    "green": "32",
+    "yellow": "33",
+    "blue": "34",
+    "magenta": "35",
+    "cyan": "36",
+    "white": "37",
+    "black": "30",
+    "orange": "38;5;208",
+    "purple": "38;5;141",
+    "pink": "38;5;217",
+}
+
+_STATUS_COLORS: dict[str, str] = {
+    "STARTED": "yellow",
+    "PASSED": "green",
+    "FAILED": "red",
+    "WARNING": "orange",
+    "INFO": "purple",
+}
+
+_nocolor: bool = False
+
+
+def set_nocolor(enabled: bool = True) -> None:
+    global _nocolor
+    _nocolor = enabled
+    _handler.setFormatter(PlainFormatter() if enabled else StatusFormatter())
+
+
+class LogStatus(Enum):
+    STARTED = "STARTED"
+    PASSED = "PASSED"
+    FAILED = "FAILED"
+    WARNING = "WARNING"
+    INFO = "INFO"
+
+
+# NOTE: inspired always by geohotz tinygrad/helpers.py `colored()``
+def cstr(color: str | None, text: object, bold: bool = False) -> str:
+    if _nocolor or color is None or color not in _ANSI:
+        return str(text)
+    b = "1;" if bold else ""
+    return f"\033[{b}{_ANSI[color]}m{text}\033[0m"
+
+
+def ansistrip(s: str) -> str:
+    return re.sub(r"\x1b\[(K|.*?m)", "", s)
+
+
+def ansilen(s: str) -> int:
+    return len(ansistrip(s))
+
+
+def word_wrap(s: str, width: int = 80) -> str:
+    if ansilen(s) <= width:
+        return s
+    lines = s.splitlines()
+    if len(lines) > 1:
+        return "\n".join(word_wrap(line, width) for line in lines)
+    i = 0
+    while ansilen(s[:i]) < width and i < len(s):
+        i += 1
+    return s[:i] + "\n" + word_wrap(s[i:], width)
+
+
+class PlainFormatter(logging.Formatter):
+    def format(self, record: logging.LogRecord) -> str:
+        status: LogStatus | None = getattr(record, "status", None)
+        if status is None:
+            return super().format(record)
+
+        msg = record.getMessage()
+        lines = msg.split("\n")
+        all_chunks: list[str] = []
+        for line in lines:
+            chunks = _wrap(line, width=STATUS_LEN - 1, break_long_words=False) if line.strip() else [""]
+            all_chunks.extend(chunks)
+
+        parts: list[str] = []
+        if status == LogStatus.WARNING:
+            parts.append(f"{'-' * (STATUS_LEN - 1)} [{status.value}]")
+
+        first_line = f"{all_chunks[0]:<{STATUS_LEN - 1}}"
+        if status == LogStatus.WARNING:
+            parts.append(first_line)
+        else:
+            parts.append(f"{first_line} [{status.value}]")
+
+        for chunk in all_chunks[1:]:
+            parts.append(chunk)
+
+        if status == LogStatus.WARNING:
+            parts.append("-" * (STATUS_LEN - 1))
+
+        return "\n".join(parts)
+
+
+class StatusFormatter(logging.Formatter):
+    def format(self, record: logging.LogRecord) -> str:
+        status: LogStatus | None = getattr(record, "status", None)
+        nocolor: bool = getattr(record, "nocolor", False)
+
+        if status is None:
+            return super().format(record)
+
+        msg = record.getMessage()
+        color = _STATUS_COLORS.get(status.value)
+        status_tag = cstr(color, f" [{status.value}]", bold=True)
+
+        lines = msg.split("\n")
+        all_chunks: list[str] = []
+        for line in lines:
+            chunks = _wrap(line, width=STATUS_LEN - 1, break_long_words=False) if line.strip() else [""]
+            all_chunks.extend(chunks)
+
+        text_color = "orange" if status == LogStatus.WARNING and not nocolor else None
+        parts: list[str] = []
+
+        if status == LogStatus.WARNING:
+            parts.append(f"{'-' * (STATUS_LEN - 1)}{status_tag}")
+
+        visible_len = ansilen(all_chunks[0])
+        pad = max(STATUS_LEN - 1 - visible_len, 0)
+        first_line = all_chunks[0] + " " * pad
+        if status == LogStatus.WARNING:
+            parts.append(cstr(text_color, first_line))
+        else:
+            parts.append(cstr(text_color, first_line) + status_tag)
+
+        for chunk in all_chunks[1:]:
+            parts.append(cstr(text_color, chunk))
+
+        if status == LogStatus.WARNING:
+            parts.append("-" * (STATUS_LEN - 1))
+
+        return "\n".join(parts)
+
+
+class NbdanticLogger(logging.Logger):
+    def status_log(self, msg: str, status: LogStatus, nocolor: bool = False) -> None:
+        level = logging.WARNING if status == LogStatus.WARNING else logging.INFO
+        self.log(level, msg, extra={"status": status, "nocolor": nocolor})
+
+
+logging.setLoggerClass(NbdanticLogger)
+
+logger: NbdanticLogger = logging.getLogger("nbdantic")  # type: ignore
+
+_handler = logging.StreamHandler()
+_handler.setFormatter(StatusFormatter())
+logger.addHandler(_handler)
+logger.setLevel(logging.INFO)
+logger.propagate = False

nbdantic-0.1.0/src/nbdantic/nbrs.pyi

@@ -0,0 +1,27 @@
+from enum import IntEnum
+
+class VersionT(IntEnum):
+    Url = ...
+    VerSpec = ...
+
+class VersionSpecifier:
+    @property
+    def operator(self) -> str: ...
+    @property
+    def version(self) -> str: ...
+    def __repr__(self) -> str: ...
+    def __str__(self) -> str: ...
+
+class UVRequirement:
+    @property
+    def package_name(self) -> str: ...
+    @property
+    def version_type(self) -> VersionT | None: ...
+    @property
+    def pkg_spec(self) -> list[VersionSpecifier] | None: ...
+    @property
+    def version_str(self) -> str | None: ...
+    def __repr__(self) -> str: ...
+
+def version_specs_from_string(spec_str: str) -> list[VersionSpecifier]: ...
+def package_spec_from_string(spec_str: str) -> UVRequirement: ...