ontodag-fs 0.0.1__tar.gz

ontodag_fs-0.0.1/PKG-INFO

@@ -0,0 +1,12 @@
+Metadata-Version: 2.4
+Name: ontodag-fs
+Version: 0.0.1
+Summary: Stateless fsspec adapter presenting an OntoDAG concept lattice as a browsable filesystem, with bytes on Ethereum Swarm via swarmfs.
+Requires-Python: >=3.11
+Requires-Dist: fsspec
+Requires-Dist: ontodag
+Requires-Dist: swarmfs
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: pytest-asyncio; extra == "dev"
+Requires-Dist: hypothesis; extra == "dev"

ontodag_fs-0.0.1/README.md

@@ -0,0 +1,125 @@
+# ontodag-fs
+
+**Browse your knowledge, not your folders.** ontodag-fs presents an
+[OntoDAG](https://github.com/petfold/ontodag) category lattice as a real,
+mountable filesystem, with file content stored on
+[Ethereum Swarm](https://www.ethswarm.org/) via
+[swarmfs](https://github.com/petfold/swarmfs).
+
+It is a modern descendant of Gifford's *Semantic File System* (SOSP '91):
+directory names are interpreted as **queries**, not locations — but with a
+concept lattice instead of flat attributes, and content-addressed storage
+instead of a local disk.
+
+```console
+$ odag-fs tree /
+/
+├── animal/
+│   ├── dog/
+│   │   └── rex.txt
+│   ├── pet/
+│   │   └── rex.txt
+│   └── spider/
+│       ├── document/
+│       │   └── web-study.md
+│       └── charlotte.txt
+└── document/
+    ├── spider/
+    │   └── web-study.md
+    ├── DESIGN_DECISIONS.md
+    └── SPEC.md
+```
+
+The same file appears under every path it *belongs* to — `web-study.md` is
+both a spider thing and a document, so it lives at `/animal/spider/document/`,
+`/document/spider/`, and every reordering of those. No copies, no symlinks:
+one object, several true names.
+
+## The ideas in five lines
+
+- **Paths are queries.** `/pet/dog` means "everything that is a pet AND a
+  dog". Order never matters: `/dog/pet` is the same place.
+- **Directories are concepts.** Subdirectories are the categories that
+  meaningfully refine what you're looking at.
+- **Files are classified objects.** A file's identity is its Swarm content
+  address; its name is just a display label.
+- **The ontology does the work.** File something under `dog` and it is
+  automatically under `mammal`, `animal`, `pet` — subsumption is free.
+- **References cannot dangle.** Content addressing means a classification
+  can never point at a file that "moved" — there is nowhere to move to.
+  (Tag filesystems on top of paths fight this forever; here it is
+  structurally impossible.)
+
+## Quick start
+
+```console
+$ pip install swarmfs   # on PyPI (as are recordstore and swarmlite)
+$ pip install \
+    "ontodag[swarm] @ git+https://github.com/petfold/ontodag.git" \
+    "ontodag-fs @ git+https://github.com/petfold/ontodag-fs.git"
+$ odag-fs set store swarm:my-store     # once — the same setting odag uses
+$ odag-fs tree /
+$ odag-fs cat /pet/dog/rex.txt
+$ odag-fs                              # interactive: cd/ls/cat with a > prompt
+$ pip install fusepy && odag-fs mount ~/mnt
+```
+
+New here? Read the **[User Guide](docs/USER_GUIDE.md)** — a tutorial that
+takes you from an empty machine to a mounted, browsable ontology, with
+worked examples of every capability.
+
+## Status
+
+**v0 — read-only view.** Browsing (`ls`, `tree`, `cat`, `info`, FUSE mount)
+is complete and tested. Filing through the filesystem (`cp` into a concept
+directory, `rm` as reclassification, `mv` between concepts) is **v0.1**, in
+progress; today filing is done with a short Python helper (see the User
+Guide). The lattice itself (creating categories) is edited through OntoDAG's
+own API, never through the mount. See [ROADMAP.md](ROADMAP.md).
+
+## Architecture
+
+```
+            FUSE mount (fsspec.fuse — a deployment mode, not architecture)
+                 │
+        OntoDAGFileSystem (this repo: fsspec AbstractFileSystem, stateless glue)
+           │                │
+        OntoDAG           swarmfs
+     (classifier/index)  (bytestore: fsspec backend for Swarm)
+           │                │
+       recordstore        Bee node
+     (persistence of the DAG)
+```
+
+This repo owns **no state**: classifications live in OntoDAG, the DAG
+persists through recordstore, bytes live on Swarm. `OntoDAGFileSystem` is a
+pure [fsspec](https://filesystem-spec.readthedocs.io/) backend, so besides
+the CLI and FUSE you also get the whole fsspec ecosystem (pandas, pyarrow,
+DuckDB, …) for free.
+
+| Repo | Role |
+|---|---|
+| [ontodag](https://github.com/petfold/ontodag) | the category DAG — index and classifier |
+| [swarmfs](https://github.com/petfold/swarmfs) | fsspec backend for Swarm — the bytestore |
+| [recordstore](https://github.com/petfold/recordstore) | versioned key→record store over Swarm — DAG persistence |
+| [mdl-fca](https://github.com/petfold/mdl-fca) | MDL/FCA learning over the same DAG (upstream, not a dependency) |
+
+## Documentation
+
+- [User Guide](docs/USER_GUIDE.md) — tutorial, setup, worked examples
+- [SPEC.md](SPEC.md) — the precise v0/v0.1 contract, method by method
+- [DESIGN_DECISIONS.md](DESIGN_DECISIONS.md) — prior art and every decision, with reasoning
+- [ROADMAP.md](ROADMAP.md) — what's in scope now vs later
+
+## Development
+
+```console
+$ git clone https://github.com/petfold/ontodag-fs && cd ontodag-fs
+$ python3 -m venv .venv && .venv/bin/pip install -e ".[dev]"
+$ .venv/bin/pytest
+```
+
+The test suite runs entirely offline — no Bee node, no FUSE — and every
+test runs against both the in-memory reference index and the real OntoDAG
+adapter, so the two cannot drift apart. Path-resolution invariants are
+property-based (hypothesis); see SPEC §6 for the invariant list.

ontodag_fs-0.0.1/pyproject.toml

@@ -0,0 +1,37 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "ontodag-fs"
+version = "0.0.1"
+description = "Stateless fsspec adapter presenting an OntoDAG concept lattice as a browsable filesystem, with bytes on Ethereum Swarm via swarmfs."
+requires-python = ">=3.11"
+dependencies = [
+    "fsspec",
+    "ontodag",
+    "swarmfs",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest",
+    "pytest-asyncio",
+    "hypothesis",
+]
+
+[project.entry-points."fsspec.specs"]
+ontodag = "ontodag_fs.fs:OntoDAGFileSystem"
+
+[project.scripts]
+odag-fs = "ontodag_fs.__main__:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+markers = [
+    "fuse: FUSE integration tests (require libfuse; skipped by default)",
+]
+addopts = "-m 'not fuse'"

ontodag_fs-0.0.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

ontodag_fs-0.0.1/src/ontodag_fs/__init__.py

@@ -0,0 +1,15 @@
+"""ontodag-fs: an OntoDAG concept lattice as an fsspec filesystem over Swarm."""
+
+from .fs import OntoDAGFileSystem
+from .index import ConceptIndex, ObjectInfo, UnknownAttributeError
+from .memory import InMemoryIndex
+from .ontodag_index import OntoDAGIndex
+
+__all__ = [
+    "ConceptIndex",
+    "InMemoryIndex",
+    "ObjectInfo",
+    "OntoDAGFileSystem",
+    "OntoDAGIndex",
+    "UnknownAttributeError",
+]

ontodag_fs-0.0.1/src/ontodag_fs/__main__.py

@@ -0,0 +1,368 @@
+"""Thin browse CLI for the v0 manual milestone (the full CLI is ROADMAP v1).
+
+Loads a DAG through odag's store machinery — the same store specs odag
+uses: a path to a native/OWL file, or ``swarm:NAME`` — and serves it
+read-only with OntoDAGFileSystem over swarmfs.
+
+    odag-fs [-s STORE] [--bee-api URL] [COMMAND [args]]
+
+Commands: ls, tree, cat, info, cd, pwd, mount, set, help. With no command,
+odag-fs follows odag's convention: it reads commands from a pipe, or opens
+an interactive ``>`` prompt on a terminal — with a current directory, so
+paths may be relative (``cd pet`` then ``ls``). The store is hydrated once
+per session, so repeated commands are fast.
+
+``set`` reads and writes the same ``~/.ontodag/config`` as odag (keys:
+store, bee_api, bee_batch), so ``odag set store swarm:NAME`` and
+``odag-fs set store swarm:NAME`` are interchangeable and no ``-s`` is
+needed once a default store is set. With nothing configured the default
+is the local ``~/.ontodag`` store. ``mount`` uses fsspec's generic FUSE
+wrapper (needs fusepy; deployment mode, not architecture).
+"""
+
+from __future__ import annotations
+
+import argparse
+import os
+import posixpath
+import shlex
+import sys
+
+try:
+    from importlib.metadata import PackageNotFoundError, version
+
+    try:
+        __version__ = version("ontodag-fs")
+    except PackageNotFoundError:
+        __version__ = "dev"
+except Exception:  # pragma: no cover
+    __version__ = "dev"
+
+
+HELP_TEXT = """\
+Usage: odag-fs [-s STORE] [--bee-api URL] [<command> [args]]
+
+Commands:
+  ls [-l] [PATH]           list a concept directory
+  tree [PATH] [--depth N]  recursive listing of the lattice
+  cat PATH                 print an object's bytes
+  info PATH                show details for a path
+  cd [PATH]                change the current directory (interactive mode)
+  pwd                      print the current directory
+  mount MOUNTPOINT         FUSE-mount the view (needs fusepy)
+  set [KEY [VALUE]]        show settings, or set one (store, bee_api, bee_batch)
+  help                     show this help
+
+With no command odag-fs reads commands from a pipe, or opens an interactive
+`>` prompt on a terminal — the same convention as odag. In that mode paths
+may be relative to the current directory (`cd pet` then `ls`), and the
+store is loaded once for the whole session.
+
+Settings are shared with odag (~/.ontodag/config): `set store swarm:NAME`
+in either tool makes -s unnecessary everywhere. With nothing configured
+the default is the local ~/.ontodag store.
+
+Options:
+  -s, --store STORE   one-off store override: a file path or swarm:NAME
+  --bee-api URL       Bee API URL (default: $BEE_API, configured bee_api,
+                      or localhost)
+"""
+
+
+# ----------------------------------------------------------------- session
+
+
+class Session:
+    """One loaded store + a current directory, shared across commands."""
+
+    def __init__(self, store_spec: str | None, bee_api: str | None):
+        self.store_spec = store_spec
+        self.bee_api = bee_api
+        self.cwd = "/"
+        self._fs = None
+
+    @property
+    def fs(self):
+        if self._fs is None:
+            self._fs = _build_fs(self.store_spec, self.bee_api)
+        return self._fs
+
+    def switch(self) -> None:
+        """Drop the loaded store (after `set store`); reload lazily."""
+        self._fs = None
+        self.store_spec = None  # the new config default takes over
+        self.cwd = "/"
+
+    def resolve(self, path: str) -> str:
+        """Make a path absolute against the current directory (supports
+        `.` and `..`, which operate on the typed path, not the lattice)."""
+        if not path.startswith("/"):
+            path = posixpath.join(self.cwd, path)
+        norm = posixpath.normpath(path)
+        return "/" if norm in (".", "//") else norm
+
+
+def _build_fs(store_spec: str | None, bee_api: str | None):
+    # odag's CLI module is the authority on store specs/config; reusing its
+    # (private) helpers is accepted milestone tooling — the real CLI (v1)
+    # gets a public seam.
+    from ontodag.__main__ import _make_backend, _read_config, _resolve_store
+
+    from swarmfs import SwarmFileSystem
+
+    from . import OntoDAGFileSystem, OntoDAGIndex
+
+    dag = _make_backend(_resolve_store(store_spec)).load()
+    api = bee_api or os.environ.get("BEE_API") or _read_config().get("bee_api")
+    swarm = SwarmFileSystem(api_url=api) if api else SwarmFileSystem()
+    return OntoDAGFileSystem(index=OntoDAGIndex(dag), swarm=swarm)
+
+
+# ---------------------------------------------------------------- commands
+
+
+def _basename(entry: dict) -> str:
+    return entry["name"].rstrip("/").rsplit("/", 1)[-1]
+
+
+def cmd_ls(session: Session, args) -> None:
+    for e in session.fs.ls(session.resolve(args.path), detail=True):
+        base = _basename(e)
+        if e["type"] == "directory":
+            print(base + "/")
+        elif args.long:
+            print(f"{base}  [{e.get('swarm_ref', '')[:8]}]  {sorted(e.get('intent', []))}")
+        else:
+            print(base)
+
+
+def cmd_tree(session: Session, args) -> None:
+    path = session.resolve(args.path)
+    print(path)
+    _tree(session.fs, path, args.depth)
+
+
+def _tree(fs, path, depth, prefix="") -> None:
+    entries = fs.ls(path, detail=True)
+    entries.sort(key=lambda e: (e["type"] != "directory", e["name"]))
+    for i, e in enumerate(entries):
+        last = i == len(entries) - 1
+        connector = "└── " if last else "├── "
+        base = _basename(e)
+        if e["type"] == "directory":
+            print(prefix + connector + base + "/")
+            # .swarm is unenumerable; .all repeats what the tree already
+            # shows — descend the lattice and .unfiled only
+            if depth > 1 and base not in (".all", ".swarm"):
+                _tree(fs, e["name"], depth - 1, prefix + ("    " if last else "│   "))
+        else:
+            ref = e.get("swarm_ref", "")
+            print(prefix + connector + base + (f"  [{ref[:8]}]" if ref else ""))
+
+
+def cmd_cat(session: Session, args) -> None:
+    sys.stdout.buffer.write(session.fs.cat_file(session.resolve(args.path)))
+    sys.stdout.buffer.flush()
+
+
+def cmd_info(session: Session, args) -> None:
+    for key, value in session.fs.info(session.resolve(args.path)).items():
+        print(f"{key}: {value}")
+
+
+def cmd_cd(session: Session, args) -> None:
+    path = session.resolve(args.path)
+    if not session.fs.isdir(path):
+        raise FileNotFoundError(f"not a directory: {path}")
+    session.cwd = path
+
+
+def cmd_pwd(session: Session, args) -> None:
+    print(session.cwd)
+
+
+def cmd_mount(session: Session, args) -> None:
+    try:
+        from fsspec.fuse import run
+    except ImportError:
+        raise ValueError("odag-fs mount needs fusepy: pip install fusepy") from None
+    print(f"mounting ontodag view at {args.mountpoint} — Ctrl-C or "
+          f"`fusermount -u {args.mountpoint}` to unmount")
+    run(session.fs, "/", args.mountpoint)
+
+
+def cmd_set(session: Session, args) -> None:
+    """Show or change settings — same keys and config file as odag's `set`
+    (~/.ontodag/config), so either tool's `set store` configures both."""
+    from ontodag.__main__ import (
+        _SETTINGS,
+        _normalize_spec,
+        _read_config,
+        _resolve_store,
+        _write_config,
+    )
+
+    def effective(key: str) -> str:
+        cfg = _read_config()
+        if key == "store":
+            return session.store_spec or _resolve_store(None)
+        if key == "bee_api":
+            return os.environ.get("BEE_API") or cfg.get("bee_api") or "http://localhost:1633"
+        if key == "bee_batch":
+            return os.environ.get("BEE_BATCH") or cfg.get("bee_batch") or ""
+        return cfg.get(key, "")
+
+    if not args.key:
+        for key in _SETTINGS:
+            print(f"{key} = {effective(key)}")
+        return
+    if args.key not in _SETTINGS:
+        raise ValueError(f"unknown setting: {args.key} "
+                         f"(known: {', '.join(_SETTINGS)})")
+    if args.value is None:
+        print(f"{args.key} = {effective(args.key)}")
+        return
+    cfg = _read_config()
+    cfg[args.key] = _normalize_spec(args.value) if args.key == "store" else args.value
+    _write_config(cfg)
+    if args.key == "store":
+        session.switch()
+
+
+def cmd_help(session: Session, args) -> None:
+    sys.stdout.write(HELP_TEXT)
+
+
+# ------------------------------------------------------------------ parser
+
+
+def _build_command_parser(with_globals: bool) -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(prog="odag-fs", add_help=with_globals)
+    if with_globals:
+        parser.add_argument("-s", "--store", default=None,
+                            help="one-off store override: a file path or "
+                                 "swarm:NAME (default: the configured store; "
+                                 "see `odag-fs set`)")
+        parser.add_argument("--bee-api", default=None,
+                            help="Bee API URL for the bytestore (default: "
+                                 "$BEE_API, configured bee_api, or localhost)")
+    sub = parser.add_subparsers(dest="command", metavar="<command>")
+
+    p = sub.add_parser("ls", help="list a concept directory")
+    p.add_argument("path", nargs="?", default=".")
+    p.add_argument("-l", "--long", action="store_true")
+    p.set_defaults(func=cmd_ls)
+
+    p = sub.add_parser("tree", help="recursive listing of the lattice")
+    p.add_argument("path", nargs="?", default=".")
+    p.add_argument("--depth", type=int, default=4)
+    p.set_defaults(func=cmd_tree)
+
+    p = sub.add_parser("cat", help="print an object's bytes")
+    p.add_argument("path")
+    p.set_defaults(func=cmd_cat)
+
+    p = sub.add_parser("info", help="show info for a path")
+    p.add_argument("path")
+    p.set_defaults(func=cmd_info)
+
+    p = sub.add_parser("cd", help="change the current directory")
+    p.add_argument("path", nargs="?", default="/")
+    p.set_defaults(func=cmd_cd)
+
+    p = sub.add_parser("pwd", help="print the current directory")
+    p.set_defaults(func=cmd_pwd)
+
+    p = sub.add_parser("mount", help="FUSE-mount the view (needs fusepy)")
+    p.add_argument("mountpoint")
+    p.set_defaults(func=cmd_mount)
+
+    p = sub.add_parser("set", help="show or change settings (shared with odag)")
+    p.add_argument("key", nargs="?")
+    p.add_argument("value", nargs="?")
+    p.set_defaults(func=cmd_set)
+
+    p = sub.add_parser("help", help="show help")
+    p.set_defaults(func=cmd_help)
+
+    return parser
+
+
+_LINE_PARSER = _build_command_parser(with_globals=False)
+
+
+def dispatch(tokens: list[str], session: Session) -> int:
+    """Parse one command line and run it. Returns a process-style exit code."""
+    try:
+        args = _LINE_PARSER.parse_args(tokens)
+    except SystemExit as exc:  # argparse handled --help or a usage error
+        return exc.code or 0
+    if args.command is None:
+        return 0
+    try:
+        args.func(session, args)
+        return 0
+    except (ValueError, OSError) as exc:
+        print(f"odag-fs: {exc}", file=sys.stderr)
+        return 1
+
+
+# ----------------------------------------------- interactive / batch modes
+
+
+def run_stream(session: Session, stream, interactive: bool) -> int:
+    """Read commands line by line — `>` prompt on a tty, silently from a
+    pipe. Errors are reported and the loop continues (odag convention);
+    the exit code says whether every line succeeded."""
+    if interactive:
+        print(f"odag-fs {__version__} - type help for help")
+    failed = False
+    while True:
+        if interactive:
+            try:
+                line = input(f"{session.cwd}> " if session.cwd != "/" else "> ")
+            except EOFError:
+                print()
+                break
+        else:
+            line = stream.readline()
+            if not line:
+                break
+        line = line.strip()
+        if not line or line.startswith("#"):
+            continue
+        try:
+            tokens = shlex.split(line)
+        except ValueError as exc:
+            print(f"odag-fs: {exc}", file=sys.stderr)
+            failed = True
+            continue
+        if tokens[0] in ("quit", "exit"):
+            break
+        if dispatch(tokens, session) != 0:
+            failed = True
+    return 1 if failed and not interactive else 0
+
+
+# ------------------------------------------------------------- entry point
+
+
+def main(argv=None) -> None:
+    parser = _build_command_parser(with_globals=True)
+    args = parser.parse_args(argv)
+    session = Session(getattr(args, "store", None), getattr(args, "bee_api", None))
+
+    if args.command is None:
+        sys.exit(run_stream(session, sys.stdin, interactive=sys.stdin.isatty()))
+
+    code = 0
+    try:
+        args.func(session, args)
+    except (ValueError, OSError) as exc:
+        print(f"odag-fs: {exc}", file=sys.stderr)
+        code = 1
+    sys.exit(code)
+
+
+if __name__ == "__main__":
+    main()