labnotes 0.0.13__tar.gz → 0.0.14__tar.gz

{labnotes-0.0.13 → labnotes-0.0.14}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: labnotes
-Version: 0.0.13
+Version: 0.0.14
 Summary: Browse experiment designs, notes, and run data from a folder.
 Project-URL: Repository, https://github.com/thekevinscott/labnotes
 Author-email: Kevin Scott <me@thekevinscott.com>
@@ -13,6 +13,7 @@ Requires-Dist: jinja2>=3.1.6
 Requires-Dist: jupyter-client>=8.10.0
 Requires-Dist: markdown-it-py>=4.2.0
 Requires-Dist: pyyaml>=6.0.3
+Requires-Dist: rich>=14
 Requires-Dist: starlette>=1.8.0
 Requires-Dist: uvicorn>=0.54.0
 Description-Content-Type: text/markdown
@@ -25,20 +26,52 @@ Browse experiment designs, notes, and run data from a folder.
 uvx labnotes server
 ```
 
-labnotes reads two folders, named in a `.labnotes.toml` at the project root:
+A project's root is the folder holding its `.labnotes.toml`. `labnotes init`
+writes one with the default layout:
 
 ```toml
-experiments = "experiments"
-data = "data"
+[experiments]
+path = "{experiment}"
+
+[files]
+path = "{experiment}/files/"
+
+[runs]
+path = "{experiment}/runs/{run}"
+```
+
+Each path is relative to the root. `{experiment}` and `{run}` stand for folder
+names: the experiments path must use `{experiment}`, and the runs path both.
+labnotes reads the file from the working folder, or from the path given with
+`-c`.
+
+`labnotes experiments` reads three tables from those paths:
+
+- `experiments`: `name`, and `config`, the experiment's `experiment.toml` as
+  JSON. An experiment is a folder matching the experiments path that holds an
+  `experiment.toml`, files or runs.
+- `files`: `experiment`, `filename`, `path`, `ctime`, `mtime`.
+- `runs`: `experiment`, `id`, `folder`, `created`, `ended`, `exit_code`,
+  `error`, `stdout`, `stderr`, and `params` as JSON, read from each run
+  folder's `run.json`.
+
+Keys come from folder names, so renaming a folder renames its rows on the next
+read. Paths are relative to the root, and times are UTC ISO 8601.
+
+```sh
+labnotes experiments            # name, last_run, created, files, runs
+labnotes experiments foo        # the same, plus config
+labnotes experiments foo runs   # foo's runs, oldest first
 ```
 
-Both keys are optional and default to the values above. Paths are relative to
-`.labnotes.toml`, or absolute, or start with `~`. labnotes reads the file from
-the working folder, or from the path given with `-c`.
+Each takes `--format table` (the default) or `--format json`.
+
+`labnotes server` doesn't read `.labnotes.toml` yet. Give it the two folders
+it uses with `--experiments-folder` and `--data-folder`:
 
-- `experiments` holds one subfolder per experiment, each with a
+- `--experiments-folder` holds one subfolder per experiment, each with a
   `design.yaml` (with a `study` key) and any number of markdown notes.
-- `data` holds the run data and a `.dirsql.toml` that declares a
+- `--data-folder` holds the run data and a `.dirsql.toml` that declares a
   `runs` table with a `study` column. An experiment's runs are the rows whose
   `study` matches its design's.
 
@@ -54,9 +87,8 @@ of values), `runs_per_cell`, and `completed_run`, a SQL condition over the
 `-h`/`--host` and `-p`/`--port` change where it listens; `-h 0.0.0.0` (or a
 Tailscale IP) serves beyond localhost. Every other uvicorn option passes
 through: `--reload`, `--workers`, `--proxy-headers`, `--root-path`, the
-`--ssl-*` set, and the rest. `labnotes server --help` lists them. `--experiments-folder`
-and `--data-folder`, or `LABNOTES_EXPERIMENTS_FOLDER` and
-`LABNOTES_DATA_FOLDER`, override `.labnotes.toml`.
+`--ssl-*` set, and the rest. `labnotes server --help` lists them. `LABNOTES_EXPERIMENTS_FOLDER` and
+`LABNOTES_DATA_FOLDER` can stand in for the folder flags.
 
 ## Agent skill
 

labnotes-0.0.14/README.md

@@ -0,0 +1,80 @@
+# labnotes
+
+Browse experiment designs, notes, and run data from a folder.
+
+```sh
+uvx labnotes server
+```
+
+A project's root is the folder holding its `.labnotes.toml`. `labnotes init`
+writes one with the default layout:
+
+```toml
+[experiments]
+path = "{experiment}"
+
+[files]
+path = "{experiment}/files/"
+
+[runs]
+path = "{experiment}/runs/{run}"
+```
+
+Each path is relative to the root. `{experiment}` and `{run}` stand for folder
+names: the experiments path must use `{experiment}`, and the runs path both.
+labnotes reads the file from the working folder, or from the path given with
+`-c`.
+
+`labnotes experiments` reads three tables from those paths:
+
+- `experiments`: `name`, and `config`, the experiment's `experiment.toml` as
+  JSON. An experiment is a folder matching the experiments path that holds an
+  `experiment.toml`, files or runs.
+- `files`: `experiment`, `filename`, `path`, `ctime`, `mtime`.
+- `runs`: `experiment`, `id`, `folder`, `created`, `ended`, `exit_code`,
+  `error`, `stdout`, `stderr`, and `params` as JSON, read from each run
+  folder's `run.json`.
+
+Keys come from folder names, so renaming a folder renames its rows on the next
+read. Paths are relative to the root, and times are UTC ISO 8601.
+
+```sh
+labnotes experiments            # name, last_run, created, files, runs
+labnotes experiments foo        # the same, plus config
+labnotes experiments foo runs   # foo's runs, oldest first
+```
+
+Each takes `--format table` (the default) or `--format json`.
+
+`labnotes server` doesn't read `.labnotes.toml` yet. Give it the two folders
+it uses with `--experiments-folder` and `--data-folder`:
+
+- `--experiments-folder` holds one subfolder per experiment, each with a
+  `design.yaml` (with a `study` key) and any number of markdown notes.
+- `--data-folder` holds the run data and a `.dirsql.toml` that declares a
+  `runs` table with a `study` column. An experiment's runs are the rows whose
+  `study` matches its design's.
+
+That serves a local site at http://127.0.0.1:8000: a table of experiments,
+and a page per experiment with its parameters, its notes, progress per cell of
+the design grid, and every run so far. Both folders are watched, so a reload
+shows new runs and edited notes.
+
+`design.yaml` keys the site reads: `study`, `fixed`, `varies` (each key a list
+of values), `runs_per_cell`, and `completed_run`, a SQL condition over the
+`runs` table that marks a run as complete. Anything else shows up as-is.
+
+`-h`/`--host` and `-p`/`--port` change where it listens; `-h 0.0.0.0` (or a
+Tailscale IP) serves beyond localhost. Every other uvicorn option passes
+through: `--reload`, `--workers`, `--proxy-headers`, `--root-path`, the
+`--ssl-*` set, and the rest. `labnotes server --help` lists them. `LABNOTES_EXPERIMENTS_FOLDER` and
+`LABNOTES_DATA_FOLDER` can stand in for the folder flags.
+
+## Agent skill
+
+```sh
+npx skills add thekevinscott/labnotes
+```
+
+The skill tells a coding agent to run `labnotes context`, which prints a usage
+guide for the installed version.

{labnotes-0.0.13 → labnotes-0.0.14}/pyproject.toml

@@ -18,6 +18,7 @@ dependencies = [
     "jupyter-client>=8.10.0",
     "markdown-it-py>=4.2.0",
     "pyyaml>=6.0.3",
+    "rich>=14",
     "starlette>=1.8.0",
     "uvicorn>=0.54.0",
 ]
@@ -51,7 +52,7 @@ exclude = ["*_test.py"]
 packages = ["src/labnotes"]
 
 [tool.pytest.ini_options]
-testpaths = ["src/labnotes"]
+testpaths = ["src/labnotes", "tests"]
 python_files = "*_test.py"
 
 [tool.ruff.lint]

{labnotes-0.0.13 → labnotes-0.0.14}/src/labnotes/cli.py

@@ -3,6 +3,8 @@ from pathlib import Path
 import click
 
 from labnotes.context_guide import context_guide
+from labnotes.experiments import experiments
+from labnotes.init import init
 from labnotes.load_config import load_config
 from labnotes.read_config import CONFIG_FILE
 from labnotes.server import server
@@ -35,4 +37,6 @@ def context(ctx: click.Context) -> None:
     click.echo(context_guide(load_config(ctx), Path.cwd()), nl=False)
 
 
+main.add_command(experiments)
+main.add_command(init)
 main.add_command(server)

labnotes-0.0.14/src/labnotes/context.md

@@ -0,0 +1,47 @@
+## Usage
+
+A project that uses labnotes has a `.labnotes.toml` at its root; the folder holding it is the root. labnotes reads it from the working folder, so run labnotes from the project root, or pass `-c PATH`. The lines above this guide name the one found and its paths, or say there is none. `labnotes init` writes one with the default layout:
+
+```toml
+[experiments]
+path = "{experiment}"            # one folder per experiment
+
+[files]
+path = "{experiment}/files/"     # an experiment's notes and other files
+
+[runs]
+path = "{experiment}/runs/{run}" # one folder per run
+```
+
+Paths are relative to the root. `{experiment}` and `{run}` stand for folder names; the experiments and files paths use `{experiment}`, the runs path both. A project laid out differently keeps its folders and changes the paths: look at where its experiments, notes and runs live, then write the templates that match.
+
+`labnotes experiments` reads three tables from those paths:
+
+- `experiments`: `name`, and `config`, the experiment's `experiment.toml` as
+  JSON. An experiment is a folder matching the experiments path that holds an
+  `experiment.toml`, files or runs.
+- `files`: `experiment`, `filename`, `path`, `ctime`, `mtime`.
+- `runs`: `experiment`, `id`, `folder`, `created`, `ended`, `exit_code`,
+  `error`, `stdout`, `stderr`, and `params` as JSON, read from each run
+  folder's `run.json`.
+
+Keys come from folder names, so renaming a folder renames its rows on the next
+read. Paths are relative to the root, and times are UTC ISO 8601.
+
+```sh
+labnotes experiments            # name, last_run, created, files, runs
+labnotes experiments foo        # the same, plus config
+labnotes experiments foo runs   # foo's runs, oldest first
+```
+
+Each takes `--format table` (the default) or `--format json`; use `--format json` when you parse the output.
+
+`labnotes server` doesn't read `.labnotes.toml` yet: pass `--experiments-folder` and `--data-folder`, or set `LABNOTES_EXPERIMENTS_FOLDER` and `LABNOTES_DATA_FOLDER`. The experiments folder has one subfolder per experiment, each with a `design.yaml` and markdown notes. The data folder holds run data and a `.dirsql.toml` declaring a `runs` table with a `study` column.
+
+The run data lives in the data folder. Query it with [dirsql](https://dirsql.dev) and the data folder's `.dirsql.toml`, which declares the `runs` table:
+
+```sh
+uvx dirsql -c data/.dirsql.toml "SELECT * FROM runs LIMIT 1" --format json
+```
+
+`uvx labnotes server` serves the experiments and their runs as a local site.

{labnotes-0.0.13 → labnotes-0.0.14}/src/labnotes/context_guide.py

@@ -16,7 +16,9 @@ def context_guide(config: LabnotesConfig | None, cwd: Path) -> str:
     else:
         project = (
             f"Config: {config.path}\n"
-            f"Experiments folder: {config.experiments_folder}\n"
-            f"Data folder: {config.data_folder}"
+            f"Root: {config.root}\n"
+            f"Experiments: {config.experiments}\n"
+            f"Files: {config.files}\n"
+            f"Runs: {config.runs}"
         )
     return f"labnotes {version('labnotes')}\n\n{project}\n\n{guide}"

labnotes-0.0.14/src/labnotes/default_config_text.py

@@ -0,0 +1,7 @@
+from labnotes.read_path import DEFAULT_PATHS
+
+
+def default_config_text() -> str:
+    return "\n".join(
+        f'[{table}]\npath = "{template}"\n' for table, template in DEFAULT_PATHS.items()
+    )

labnotes-0.0.14/src/labnotes/experiments.py

@@ -0,0 +1,33 @@
+import asyncio
+
+import click
+
+from labnotes.load_config import load_config
+from labnotes.print_rows import print_rows
+from labnotes.read_config import CONFIG_FILE
+from labnotes.tables.experiment_rows import experiment_rows
+
+
+@click.command(help="List experiments, show one, or list its runs.")
+@click.argument("name", required=False)
+@click.argument("action", required=False, type=click.Choice(["runs"]))
+@click.option(
+    "--format",
+    "output_format",
+    type=click.Choice(["table", "json"]),
+    default="table",
+    show_default=True,
+)
+@click.pass_context
+def experiments(
+    ctx: click.Context, name: str | None, action: str | None, output_format: str
+) -> None:
+    config = load_config(ctx)
+    if config is None:
+        raise click.ClickException(
+            f"No {CONFIG_FILE} here. Run labnotes init, or pass -c PATH."
+        )
+    rows = asyncio.run(experiment_rows(config, name, action))
+    if rows is None:
+        raise click.ClickException(f"No experiment named {name}.")
+    print_rows(rows, output_format)

labnotes-0.0.14/src/labnotes/init.py

@@ -0,0 +1,17 @@
+from pathlib import Path
+
+import click
+
+from labnotes.default_config_text import default_config_text
+from labnotes.read_config import CONFIG_FILE
+
+
+@click.command(help=f"Write a {CONFIG_FILE} with the default layout here.")
+def init() -> None:
+    path = Path.cwd() / CONFIG_FILE
+    try:
+        with path.open("x", encoding="utf-8") as file:
+            file.write(default_config_text())
+    except FileExistsError as error:
+        raise click.ClickException(f"{path} already exists.") from error
+    click.echo(f"Wrote {path}")

{labnotes-0.0.13 → labnotes-0.0.14}/src/labnotes/models.py

@@ -13,5 +13,7 @@ class RunView:
 @dataclass(frozen=True)
 class LabnotesConfig:
     path: Path
-    experiments_folder: Path
-    data_folder: Path
+    root: Path
+    experiments: str
+    files: str
+    runs: str

labnotes-0.0.14/src/labnotes/print_rows.py

@@ -0,0 +1,19 @@
+import json
+
+import click
+from rich.console import Console
+from rich.table import Table
+
+
+def print_rows(rows: list[dict[str, object]], output_format: str) -> None:
+    match output_format:
+        case "json":
+            click.echo(json.dumps(rows, indent=2))
+            return
+    if not rows:
+        click.echo("No rows.")
+        return
+    table = Table(*rows[0].keys())
+    for row in rows:
+        table.add_row(*("" if value is None else str(value) for value in row.values()))
+    Console().print(table)

labnotes-0.0.14/src/labnotes/read_config.py

@@ -0,0 +1,27 @@
+import tomllib
+from pathlib import Path
+
+from labnotes.models import LabnotesConfig
+from labnotes.read_path import DEFAULT_PATHS, read_path
+
+CONFIG_FILE = ".labnotes.toml"
+
+
+def read_config(path: Path) -> LabnotesConfig:
+    try:
+        with path.open("rb") as file:
+            values = tomllib.load(file)
+    except tomllib.TOMLDecodeError as error:
+        raise ValueError(f"{path}: {error}") from error
+    unknown = sorted(values.keys() - DEFAULT_PATHS.keys())
+    if unknown:
+        raise ValueError(f"{path}: unknown tables {', '.join(unknown)}")
+    path = path.resolve()
+    return LabnotesConfig(
+        path,
+        path.parent,
+        **{
+            table: read_path(path, table, values.get(table, {}))
+            for table in DEFAULT_PATHS
+        },
+    )

labnotes-0.0.14/src/labnotes/read_path.py

@@ -0,0 +1,34 @@
+import re
+from pathlib import Path, PurePosixPath
+
+DEFAULT_PATHS = {
+    "experiments": "{experiment}",
+    "files": "{experiment}/files/",
+    "runs": "{experiment}/runs/{run}",
+}
+PLACEHOLDERS = {
+    "experiments": {"experiment"},
+    "files": {"experiment"},
+    "runs": {"experiment", "run"},
+}
+PLACEHOLDER = re.compile(r"\{([^}]*)\}")
+
+
+def read_path(config: Path, table: str, section: object) -> str:
+    if not isinstance(section, dict):
+        raise ValueError(f"{config}: {table} must be a table, like [{table}]")
+    unknown = sorted(section.keys() - {"path"})
+    if unknown:
+        raise ValueError(f"{config}: unknown keys in [{table}]: {', '.join(unknown)}")
+    template = section.get("path", DEFAULT_PATHS[table])
+    if not isinstance(template, str):
+        raise ValueError(f"{config}: [{table}] path must be a string")
+    if PurePosixPath(template).is_absolute() or template.startswith("~"):
+        raise ValueError(
+            f"{config}: [{table}] path must be relative to the config's folder"
+        )
+    required = PLACEHOLDERS[table]
+    if set(PLACEHOLDER.findall(template)) != required:
+        expected = " and ".join(f"{{{name}}}" for name in sorted(required))
+        raise ValueError(f"{config}: [{table}] path must use exactly {expected}")
+    return template

{labnotes-0.0.13 → labnotes-0.0.14}/src/labnotes/server.py

@@ -4,8 +4,6 @@ from pathlib import Path
 import click
 from uvicorn.main import main as uvicorn_main
 
-from labnotes.load_config import load_config
-from labnotes.read_config import CONFIG_FILE
 from labnotes.serve_app import DATA_FOLDER, EXPERIMENTS_FOLDER, KERNEL_IDLE_MINUTES
 
 # labnotes owns these; the rest of uvicorn's options pass straight through.
@@ -17,12 +15,14 @@ OWN_OPTIONS = {"app", "factory", "app_dir", "version", "host", "port"}
     "--experiments-folder",
     type=click.Path(path_type=Path),
     envvar=EXPERIMENTS_FOLDER,
+    required=True,
     help="One subfolder per experiment, each with a design.yaml and markdown notes.",
 )
 @click.option(
     "--data-folder",
     type=click.Path(path_type=Path),
     envvar=DATA_FOLDER,
+    required=True,
     help="Run data, with a .dirsql.toml declaring a runs table that has a study column.",
 )
 @click.option(
@@ -44,19 +44,11 @@ OWN_OPTIONS = {"app", "factory", "app_dir", "version", "host", "port"}
 @click.pass_context
 def server(
     ctx: click.Context,
-    experiments_folder: Path | None,
-    data_folder: Path | None,
+    experiments_folder: Path,
+    data_folder: Path,
     kernel_idle_minutes: int,
     **server_options: object,
 ) -> None:
-    config = load_config(ctx)
-    if config is not None:
-        experiments_folder = experiments_folder or config.experiments_folder
-        data_folder = data_folder or config.data_folder
-    if experiments_folder is None or data_folder is None:
-        raise click.UsageError(
-            f"--experiments-folder and --data-folder are required without a {CONFIG_FILE}."
-        )
     # Through the environment so --reload and --workers child processes see them.
     os.environ[EXPERIMENTS_FOLDER] = str(experiments_folder.resolve())
     os.environ[DATA_FOLDER] = str(data_folder.resolve())

labnotes-0.0.14/src/labnotes/tables/experiment_rows.py

@@ -0,0 +1,17 @@
+from labnotes.models import LabnotesConfig
+from labnotes.tables.list_experiment_runs import list_experiment_runs
+from labnotes.tables.open_tables import open_tables
+from labnotes.tables.summarize_experiments import summarize_experiments
+
+
+async def experiment_rows(
+    config: LabnotesConfig, name: str | None, action: str | None
+) -> list[dict[str, object]] | None:
+    tables = open_tables(config)
+    rows = await summarize_experiments(tables, name)
+    if name is not None and not rows:
+        return None
+    match name, action:
+        case str(), "runs":
+            return await list_experiment_runs(tables, name)
+    return rows

labnotes-0.0.14/src/labnotes/tables/file_prefix.py

@@ -0,0 +1,9 @@
+import re
+
+GLOB = re.compile(r"[*?\[]")
+
+
+def file_prefix(template: str) -> str:
+    glob = GLOB.search(template)
+    head = template[: glob.start()] if glob else template
+    return head if head.endswith("/") else head[: head.rfind("/") + 1]

labnotes-0.0.14/src/labnotes/tables/iso_time.py

@@ -0,0 +1,5 @@
+from datetime import UTC, datetime
+
+
+def iso_time(seconds: float) -> str:
+    return datetime.fromtimestamp(seconds, UTC).strftime("%Y-%m-%dT%H:%M:%SZ")

labnotes-0.0.14/src/labnotes/tables/list_experiment_runs.py

@@ -0,0 +1,10 @@
+from dirsql import DirSQL
+
+from labnotes.sql_quote import quote_string
+
+
+async def list_experiment_runs(tables: DirSQL, name: str) -> list[dict[str, object]]:
+    return await tables.query(
+        "SELECT id, created, ended, exit_code, error, params, folder FROM runs"
+        f" WHERE experiment = {quote_string(name)} ORDER BY created, id"
+    )

labnotes-0.0.14/src/labnotes/tables/open_tables.py

@@ -0,0 +1,63 @@
+from functools import partial
+
+from dirsql import DirSQL, Table
+
+from labnotes.models import LabnotesConfig
+from labnotes.tables.path_glob import path_glob
+from labnotes.tables.read_experiment import read_experiment
+from labnotes.tables.read_file import read_file
+from labnotes.tables.read_run import read_run
+
+# experiments is a view, so files and runs can't declare a foreign key to it.
+EXPERIMENTS = """
+CREATE TABLE experiment_configs (experiment TEXT PRIMARY KEY, config TEXT);
+CREATE VIEW experiments AS
+  SELECT names.name, experiment_configs.config
+  FROM (
+    SELECT experiment AS name FROM experiment_configs
+    UNION SELECT experiment FROM files
+    UNION SELECT experiment FROM runs
+  ) AS names
+  LEFT JOIN experiment_configs ON experiment_configs.experiment = names.name;
+"""
+FILES = """
+CREATE TABLE files (
+  experiment TEXT, filename TEXT, path TEXT PRIMARY KEY, ctime TEXT, mtime TEXT
+)
+"""
+RUNS = """
+CREATE TABLE runs (
+  experiment TEXT, id TEXT, folder TEXT, created TEXT, ended TEXT,
+  exit_code INTEGER, error TEXT, stdout TEXT, stderr TEXT, params TEXT,
+  PRIMARY KEY (experiment, id)
+)
+"""
+
+
+def open_tables(config: LabnotesConfig) -> DirSQL:
+    root = config.root
+    return DirSQL(
+        str(root),
+        tables=[
+            Table(
+                name="files",
+                ddl=FILES,
+                glob=path_glob(config.files),
+                on_file=partial(read_file, root, config.files),
+            ),
+            Table(
+                name="runs",
+                ddl=RUNS,
+                glob=path_glob(f"{config.runs}/run.json"),
+                on_file=partial(read_run, root, config.runs),
+            ),
+            Table(
+                name="experiment_configs",
+                ddl=EXPERIMENTS,
+                glob=path_glob(f"{config.experiments}/experiment.toml"),
+                on_file=partial(read_experiment, root, config.experiments),
+            ),
+        ],
+        # Run folders are often gitignored (DVC, large outputs); scan them anyway.
+        no_ignore=True,
+    )

labnotes-0.0.14/src/labnotes/tables/path_glob.py

@@ -0,0 +1,8 @@
+import re
+
+PLACEHOLDER = re.compile(r"\{[^}]*\}")
+
+
+def path_glob(template: str) -> str:
+    glob = PLACEHOLDER.sub("*", template)
+    return f"{glob}**" if glob.endswith("/") else glob

labnotes-0.0.14/src/labnotes/tables/path_pattern.py

@@ -0,0 +1,23 @@
+import re
+
+TOKEN = re.compile(r"\{([^}]*)\}|\*\*|\*|\?|[^{*?]+")
+
+
+def path_pattern(template: str) -> re.Pattern[str]:
+    parts = []
+    for found in TOKEN.finditer(template):
+        if found.group(1) is not None:
+            parts.append(f"(?P<{found.group(1)}>[^/]+)")
+            continue
+        match found.group(0):
+            case "**":
+                parts.append(".*")
+            case "*":
+                parts.append("[^/]*")
+            case "?":
+                parts.append("[^/]")
+            case token:
+                parts.append(re.escape(token))
+    if template.endswith("/"):
+        parts.append(".+")
+    return re.compile("".join(parts))

labnotes-0.0.14/src/labnotes/tables/read_experiment.py

@@ -0,0 +1,15 @@
+import json
+import tomllib
+from pathlib import Path
+
+from labnotes.tables.path_pattern import path_pattern
+
+
+def read_experiment(root: Path, template: str, path: str) -> list[dict[str, object]]:
+    relative = Path(path).relative_to(root).as_posix()
+    found = path_pattern(f"{template}/experiment.toml").fullmatch(relative)
+    if found is None:
+        return []
+    with Path(path).open("rb") as file:
+        config = tomllib.load(file)
+    return [{"experiment": found["experiment"], "config": json.dumps(config)}]

labnotes-0.0.14/src/labnotes/tables/read_file.py

@@ -0,0 +1,23 @@
+from pathlib import Path
+
+from labnotes.tables.file_prefix import file_prefix
+from labnotes.tables.iso_time import iso_time
+from labnotes.tables.path_pattern import path_pattern
+
+
+def read_file(root: Path, template: str, path: str) -> list[dict[str, object]]:
+    relative = Path(path).relative_to(root).as_posix()
+    found = path_pattern(template).fullmatch(relative)
+    prefix = path_pattern(file_prefix(template).rstrip("/")).match(relative)
+    if found is None or prefix is None:
+        return []
+    stat = Path(path).stat()
+    return [
+        {
+            "experiment": found["experiment"],
+            "filename": relative[prefix.end() :].lstrip("/"),
+            "path": relative,
+            "ctime": iso_time(stat.st_ctime),
+            "mtime": iso_time(stat.st_mtime),
+        }
+    ]

labnotes-0.0.14/src/labnotes/tables/read_run.py

@@ -0,0 +1,27 @@
+import json
+from pathlib import Path
+
+from labnotes.tables.path_pattern import path_pattern
+
+
+def read_run(root: Path, template: str, path: str) -> list[dict[str, object]]:
+    relative = Path(path).relative_to(root).as_posix()
+    found = path_pattern(f"{template}/run.json").fullmatch(relative)
+    if found is None:
+        return []
+    run = json.loads(Path(path).read_text(encoding="utf-8"))
+    folder = relative.removesuffix("/run.json")
+    return [
+        {
+            "experiment": found["experiment"],
+            "id": found["run"],
+            "folder": folder,
+            "created": run.get("created"),
+            "ended": run.get("ended"),
+            "exit_code": run.get("exit_code"),
+            "error": run.get("error"),
+            "stdout": f"{folder}/stdout.log",
+            "stderr": f"{folder}/stderr.log",
+            "params": json.dumps(run.get("params", {})),
+        }
+    ]

labnotes-0.0.14/src/labnotes/tables/summarize_experiments.py

@@ -0,0 +1,27 @@
+from dirsql import DirSQL
+
+from labnotes.sql_quote import quote_string
+
+SUMMARY = """
+SELECT
+  e.name,{config}
+  (SELECT max(created) FROM runs WHERE runs.experiment = e.name) AS last_run,
+  (
+    SELECT min(t) FROM (
+      SELECT ctime AS t FROM files WHERE files.experiment = e.name
+      UNION ALL SELECT created FROM runs WHERE runs.experiment = e.name
+    )
+  ) AS created,
+  (SELECT count(*) FROM files WHERE files.experiment = e.name) AS files,
+  (SELECT count(*) FROM runs WHERE runs.experiment = e.name) AS runs
+FROM experiments AS e
+"""
+
+
+async def summarize_experiments(
+    tables: DirSQL, name: str | None = None
+) -> list[dict[str, object]]:
+    if name is None:
+        return await tables.query(SUMMARY.format(config="") + " ORDER BY e.name")
+    sql = SUMMARY.format(config=" e.config,")
+    return await tables.query(f"{sql} WHERE e.name = {quote_string(name)}")

{labnotes-0.0.13 → labnotes-0.0.14}/uv.lock

@@ -460,6 +460,7 @@ dependencies = [
     { name = "jupyter-client" },
     { name = "markdown-it-py" },
     { name = "pyyaml" },
+    { name = "rich" },
     { name = "starlette" },
     { name = "uvicorn" },
 ]
@@ -482,6 +483,7 @@ requires-dist = [
     { name = "jupyter-client", specifier = ">=8.10.0" },
     { name = "markdown-it-py", specifier = ">=4.2.0" },
     { name = "pyyaml", specifier = ">=6.0.3" },
+    { name = "rich", specifier = ">=14" },
     { name = "starlette", specifier = ">=1.8.0" },
     { name = "uvicorn", specifier = ">=0.54.0" },
 ]
@@ -1069,6 +1071,19 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/6e/97/bc4f0edefb992df4fdebcf9f0cc40f631cd4ed277e1ed59ef2cd99a5c8c5/pyzmq-27.2.0-cp315-cp315t-win_arm64.whl", hash = "sha256:a843094b4d3d633bc3623e47a2ff50742d6af02bc1f7606aa2e67e971e21878d", size = 581985, upload-time = "2026-08-20T19:07:34.19Z" },
 ]
 
+[[package]]
+name = "rich"
+version = "15.0.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "markdown-it-py" },
+    { name = "pygments" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/c0/8f/0722ca900cc807c13a6a0c696dacf35430f72e0ec571c4275d2371fca3e9/rich-15.0.0.tar.gz", hash = "sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36", size = 230680, upload-time = "2026-04-12T08:24:00.75Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl", hash = "sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", size = 310654, upload-time = "2026-04-12T08:24:02.83Z" },
+]
+
 [[package]]
 name = "ruff"
 version = "0.8.4"

labnotes-0.0.13/README.md

@@ -1,49 +0,0 @@
-# labnotes
-
-Browse experiment designs, notes, and run data from a folder.
-
-```sh
-uvx labnotes server
-```
-
-labnotes reads two folders, named in a `.labnotes.toml` at the project root:
-
-```toml
-experiments = "experiments"
-data = "data"
-```
-
-Both keys are optional and default to the values above. Paths are relative to
-`.labnotes.toml`, or absolute, or start with `~`. labnotes reads the file from
-the working folder, or from the path given with `-c`.
-
-- `experiments` holds one subfolder per experiment, each with a
-  `design.yaml` (with a `study` key) and any number of markdown notes.
-- `data` holds the run data and a `.dirsql.toml` that declares a
-  `runs` table with a `study` column. An experiment's runs are the rows whose
-  `study` matches its design's.
-
-That serves a local site at http://127.0.0.1:8000: a table of experiments,
-and a page per experiment with its parameters, its notes, progress per cell of
-the design grid, and every run so far. Both folders are watched, so a reload
-shows new runs and edited notes.
-
-`design.yaml` keys the site reads: `study`, `fixed`, `varies` (each key a list
-of values), `runs_per_cell`, and `completed_run`, a SQL condition over the
-`runs` table that marks a run as complete. Anything else shows up as-is.
-
-`-h`/`--host` and `-p`/`--port` change where it listens; `-h 0.0.0.0` (or a
-Tailscale IP) serves beyond localhost. Every other uvicorn option passes
-through: `--reload`, `--workers`, `--proxy-headers`, `--root-path`, the
-`--ssl-*` set, and the rest. `labnotes server --help` lists them. `--experiments-folder`
-and `--data-folder`, or `LABNOTES_EXPERIMENTS_FOLDER` and
-`LABNOTES_DATA_FOLDER`, override `.labnotes.toml`.
-
-## Agent skill
-
-```sh
-npx skills add thekevinscott/labnotes
-```
-
-The skill tells a coding agent to run `labnotes context`, which prints a usage
-guide for the installed version.

labnotes-0.0.13/src/labnotes/context.md

@@ -1,18 +0,0 @@
-## Usage
-
-labnotes reads two folders. The experiments folder has one subfolder per experiment, each with a `design.yaml` and markdown notes. The data folder holds run data and a `.dirsql.toml` declaring a `runs` table with a `study` column. A project that uses labnotes has a `.labnotes.toml` at its root; labnotes reads it from the working folder, so run labnotes from the project root, or pass `-c PATH`. The lines above this guide name the one found and the folders it resolves to, or say there is none. Its keys are optional:
-
-```toml
-experiments = "experiments"  # the default
-data = "data"                # relative to .labnotes.toml, absolute, or ~
-```
-
-`--experiments-folder` and `--data-folder`, or `LABNOTES_EXPERIMENTS_FOLDER` and `LABNOTES_DATA_FOLDER`, override either key. Without a `.labnotes.toml`, pass both.
-
-The run data lives in the data folder. Query it with [dirsql](https://dirsql.dev) and the data folder's `.dirsql.toml`, which declares the `runs` table:
-
-```sh
-uvx dirsql -c data/.dirsql.toml "SELECT * FROM runs LIMIT 1" --format json
-```
-
-`uvx labnotes server` serves the experiments and their runs as a local site.

labnotes-0.0.13/src/labnotes/read_config.py

@@ -1,23 +0,0 @@
-import tomllib
-from pathlib import Path
-
-from labnotes.models import LabnotesConfig
-
-CONFIG_FILE = ".labnotes.toml"
-DEFAULTS = {"experiments": "experiments", "data": "data"}
-
-
-def read_config(path: Path) -> LabnotesConfig:
-    with path.open("rb") as file:
-        values = tomllib.load(file)
-    unknown = sorted(values.keys() - DEFAULTS.keys())
-    if unknown:
-        raise ValueError(f"{path}: unknown keys {', '.join(unknown)}")
-    folder = path.resolve().parent
-    folders = {}
-    for key, default in DEFAULTS.items():
-        value = values.get(key, default)
-        if not isinstance(value, str):
-            raise ValueError(f"{path}: {key} must be a string")
-        folders[key] = folder / Path(value).expanduser()
-    return LabnotesConfig(path.resolve(), folders["experiments"], folders["data"])