gistfs 0.1.0__tar.gz

gistfs-0.1.0/.gitignore

@@ -0,0 +1,32 @@
+# Python
+__pycache__/
+*.py[cod]
+*.so
+*.egg
+*.egg-info/
+dist/
+build/
+*.whl
+
+# Virtual environments
+.venv/
+venv/
+
+# Environment
+.env
+
+# IDE
+.idea/
+.vscode/
+*.swp
+
+# Testing
+.pytest_cache/
+.coverage
+htmlcov/
+
+# macOS
+.DS_Store
+
+# Project
+junk/

gistfs-0.1.0/CLAUDE.md

@@ -0,0 +1,38 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project overview
+
+gistfs is a Python library that uses GitHub Gists as a persistent key-value filesystem, designed for AI agent memory. It provides a layered architecture:
+
+- **GistFS** (`gistfs/core.py`) — Low-level filesystem abstraction over a single gist. Handles CRUD on gist files via the GitHub API, with local caching and a context manager interface. Also provides `GistFile`, a `io.StringIO` subclass for file-like read/write/append.
+- **GistMemory** (`gistfs/memory.py`) — Higher-level key-value store built on GistFS. Supports collections (each collection = one JSON file in the gist).
+- **Integrations** (`gistfs/integrations/`) — Adapters that implement third-party store interfaces:
+  - `GistKVStore` (LlamaIndex `BaseKVStore`) — delegates to GistMemory
+  - `GistStore` (LangGraph `BaseStore`) — uses GistFS directly, maps namespace tuples to filenames (`("user", "prefs")` → `user__prefs.json`)
+
+## Commands
+
+```bash
+# Install (uses uv, hatchling build backend)
+uv pip install -e ".[dev,all]"
+
+# Run all tests (requires GITHUB_TOKEN — tests hit the live GitHub API)
+uv run pytest
+
+# Run a single test file or test
+uv run pytest tests/test_core.py
+uv run pytest tests/test_core.py::test_write_and_read -v
+```
+
+## Testing
+
+Tests are **live integration tests** — they create a real gist, run operations, then delete it. The `test_gist` session-scoped fixture in `tests/conftest.py` handles gist lifecycle. A `GITHUB_TOKEN` env var with `gist` scope is required; tests skip if it's missing. A `.env` file is loaded via `python-dotenv`.
+
+## Key design decisions
+
+- All gist file content is JSON-serialized. Non-JSON content is stored/returned as raw strings.
+- GistFS uses an in-memory `_cache` dict synced from the API. The cache is populated on `sync()` / context manager entry and cleared on exit.
+- Write operations (`write`, `delete`) hit the GitHub API immediately then update the local cache — there is no batching or deferred flush.
+- `GistFile.close()` flushes writes to the gist; the context manager calls `close()` automatically.

gistfs-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Hervé Mignot
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

gistfs-0.1.0/Makefile

@@ -0,0 +1,17 @@
+.PHONY: test build clean publish-test publish
+
+test:
+	uv run pytest
+
+build: clean
+	uv build
+
+clean:
+	rm -rf dist/
+
+publish-test: build
+	UV_PUBLISH_TOKEN=$(UV_PUBLISH_TOKEN_TEST)
+	uv publish --publish-url https://test.pypi.org/legacy/
+
+publish: build
+	uv publish

gistfs-0.1.0/PKG-INFO

@@ -0,0 +1,142 @@
+Metadata-Version: 2.4
+Name: gistfs
+Version: 0.1.0
+Summary: Use GitHub Gists as a persistent key-value filesystem — ideal for AI agent memory
+Project-URL: Homepage, https://github.com/HerveMignot/gistfs
+Project-URL: Repository, https://github.com/HerveMignot/gistfs
+Project-URL: Issues, https://github.com/HerveMignot/gistfs/issues
+Author: Hervé Mignot
+License-Expression: MIT
+License-File: LICENSE
+Keywords: agent,ai,filesystem,gist,github,llm,memory
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries
+Requires-Python: >=3.10
+Requires-Dist: requests>=2.28
+Provides-Extra: all
+Requires-Dist: langgraph-checkpoint>=2.0; extra == 'all'
+Requires-Dist: llama-index-core>=0.11; extra == 'all'
+Provides-Extra: dev
+Requires-Dist: pytest-asyncio>=1.0; extra == 'dev'
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Requires-Dist: python-dotenv>=1.0; extra == 'dev'
+Provides-Extra: langgraph
+Requires-Dist: langgraph-checkpoint>=2.0; extra == 'langgraph'
+Provides-Extra: llamaindex
+Requires-Dist: llama-index-core>=0.11; extra == 'llamaindex'
+Description-Content-Type: text/markdown
+
+# gistfs
+
+Use GitHub Gists as a persistent key-value filesystem — ideal for AI agent memory.
+
+## Install
+
+```bash
+pip install gistfs
+```
+
+With optional integrations:
+
+```bash
+pip install gistfs[llamaindex]   # LlamaIndex KVStore
+pip install gistfs[langgraph]    # LangGraph BaseStore
+pip install gistfs[all]          # everything
+```
+
+## Quick start
+
+### Creating a new gist
+
+No need to create a gist manually — bootstrap one from code:
+
+```python
+from gistfs import GistFS
+
+gfs = GistFS.create(description="my agent memory")
+print(gfs.gist_id)  # save this for later
+
+# Or with GistMemory:
+from gistfs import GistMemory
+mem = GistMemory.create(description="my agent memory")
+```
+
+### As a filesystem (context manager)
+
+```python
+from gistfs import GistFS
+
+with GistFS(gist_id="your_gist_id") as gfs:
+    gfs.write("config.json", {"model": "gpt-4", "temperature": 0.7})
+    config = gfs.read("config.json")
+    print(gfs.list_files())
+    gfs.delete("config.json")
+```
+
+### File-like interface
+
+```python
+with GistFS(gist_id="your_gist_id") as gfs:
+    with gfs.open("notes.txt", "w") as f:
+        f.write("hello world")
+
+    with gfs.open("notes.txt", "r") as f:
+        content = f.read()
+
+    with gfs.open("notes.txt", "a") as f:
+        f.write("\nappended line")
+```
+
+### As AI agent memory
+
+```python
+from gistfs import GistMemory
+
+with GistMemory(gist_id="your_gist_id") as mem:
+    mem.put("conversation_1", {"messages": [{"role": "user", "content": "hi"}]})
+    history = mem.get("conversation_1")
+    all_data = mem.get_all()
+    mem.delete("conversation_1")
+```
+
+### LlamaIndex integration
+
+```python
+from gistfs.integrations.llamaindex import GistKVStore
+
+store = GistKVStore(gist_id="your_gist_id")
+store.put("doc1", {"text": "hello world"}, collection="docstore")
+doc = store.get("doc1", collection="docstore")
+```
+
+### LangGraph integration
+
+```python
+from gistfs.integrations.langgraph import GistStore
+
+store = GistStore(gist_id="your_gist_id")
+store.put(("user", "prefs"), "theme", {"value": "dark"})
+item = store.get(("user", "prefs"), "theme")
+```
+
+## Authentication
+
+Set the `GITHUB_TOKEN` environment variable with a GitHub personal access token that has `gist` scope. Read-only operations on public gists work without a token.
+
+```bash
+export GITHUB_TOKEN=ghp_your_token_here
+```
+
+Or pass it directly:
+
+```python
+gfs = GistFS(gist_id="abc123", token="ghp_...")
+```

gistfs-0.1.0/README.md

@@ -0,0 +1,106 @@
+# gistfs
+
+Use GitHub Gists as a persistent key-value filesystem — ideal for AI agent memory.
+
+## Install
+
+```bash
+pip install gistfs
+```
+
+With optional integrations:
+
+```bash
+pip install gistfs[llamaindex]   # LlamaIndex KVStore
+pip install gistfs[langgraph]    # LangGraph BaseStore
+pip install gistfs[all]          # everything
+```
+
+## Quick start
+
+### Creating a new gist
+
+No need to create a gist manually — bootstrap one from code:
+
+```python
+from gistfs import GistFS
+
+gfs = GistFS.create(description="my agent memory")
+print(gfs.gist_id)  # save this for later
+
+# Or with GistMemory:
+from gistfs import GistMemory
+mem = GistMemory.create(description="my agent memory")
+```
+
+### As a filesystem (context manager)
+
+```python
+from gistfs import GistFS
+
+with GistFS(gist_id="your_gist_id") as gfs:
+    gfs.write("config.json", {"model": "gpt-4", "temperature": 0.7})
+    config = gfs.read("config.json")
+    print(gfs.list_files())
+    gfs.delete("config.json")
+```
+
+### File-like interface
+
+```python
+with GistFS(gist_id="your_gist_id") as gfs:
+    with gfs.open("notes.txt", "w") as f:
+        f.write("hello world")
+
+    with gfs.open("notes.txt", "r") as f:
+        content = f.read()
+
+    with gfs.open("notes.txt", "a") as f:
+        f.write("\nappended line")
+```
+
+### As AI agent memory
+
+```python
+from gistfs import GistMemory
+
+with GistMemory(gist_id="your_gist_id") as mem:
+    mem.put("conversation_1", {"messages": [{"role": "user", "content": "hi"}]})
+    history = mem.get("conversation_1")
+    all_data = mem.get_all()
+    mem.delete("conversation_1")
+```
+
+### LlamaIndex integration
+
+```python
+from gistfs.integrations.llamaindex import GistKVStore
+
+store = GistKVStore(gist_id="your_gist_id")
+store.put("doc1", {"text": "hello world"}, collection="docstore")
+doc = store.get("doc1", collection="docstore")
+```
+
+### LangGraph integration
+
+```python
+from gistfs.integrations.langgraph import GistStore
+
+store = GistStore(gist_id="your_gist_id")
+store.put(("user", "prefs"), "theme", {"value": "dark"})
+item = store.get(("user", "prefs"), "theme")
+```
+
+## Authentication
+
+Set the `GITHUB_TOKEN` environment variable with a GitHub personal access token that has `gist` scope. Read-only operations on public gists work without a token.
+
+```bash
+export GITHUB_TOKEN=ghp_your_token_here
+```
+
+Or pass it directly:
+
+```python
+gfs = GistFS(gist_id="abc123", token="ghp_...")
+```

gistfs-0.1.0/gistfs/__init__.py

@@ -0,0 +1,7 @@
+"""gistfs — Use GitHub Gists as a persistent key-value filesystem for AI agents."""
+
+from .core import GistFS, GistFile
+from .memory import GistMemory
+
+__all__ = ["GistFS", "GistFile", "GistMemory"]
+__version__ = "0.1.0"

gistfs-0.1.0/gistfs/core.py

@@ -0,0 +1,330 @@
+"""Core GistFS class — use GitHub Gists as a persistent key-value filesystem."""
+
+from __future__ import annotations
+
+import io
+import json
+import os
+from typing import Any
+
+import requests
+
+
+GITHUB_API_GISTS = "https://api.github.com/gists/{gist_id}"
+GITHUB_API_GISTS_BASE = "https://api.github.com/gists"
+
+
+class GistFS:
+    """A filesystem-like interface backed by a single GitHub Gist.
+
+    Each "file" in the gist stores a JSON-serialized value, giving you a
+    persistent key-value store accessible from anywhere with an internet
+    connection.
+
+    Usage::
+
+        with GistFS(gist_id="abc123") as gfs:
+            gfs.write("state.json", {"counter": 1})
+            data = gfs.read("state.json")
+            print(gfs.list_files())
+            gfs.delete("state.json")
+
+    The token is read from the ``GITHUB_TOKEN`` environment variable by
+    default, or can be passed explicitly.  Read-only operations on public
+    gists work without a token.
+    """
+
+    def __init__(
+        self,
+        gist_id: str,
+        token: str | None = None,
+        *,
+        auto_sync: bool = True,
+    ) -> None:
+        self.gist_id = gist_id
+        self.token = token or os.environ.get("GITHUB_TOKEN", "")
+        self.auto_sync = auto_sync
+        self._url = GITHUB_API_GISTS.format(gist_id=gist_id)
+        self._cache: dict[str, Any] | None = None
+
+    # ── factory ───────────────────────────────────────────────────────
+
+    @classmethod
+    def create(
+        cls,
+        description: str = "",
+        *,
+        public: bool = False,
+        token: str | None = None,
+        init_files: dict[str, Any] | None = None,
+    ) -> GistFS:
+        """Create a new GitHub Gist and return a :class:`GistFS` bound to it.
+
+        Parameters
+        ----------
+        description:
+            Gist description shown on GitHub.
+        public:
+            If ``True`` the gist is public; otherwise it is secret (default).
+        token:
+            GitHub personal access token.  Falls back to ``GITHUB_TOKEN``.
+        init_files:
+            Optional mapping of ``{filename: content}`` to seed the gist with.
+            If *None*, a single placeholder file ``_gistfs.json`` is created.
+
+        Returns
+        -------
+        GistFS
+            A new instance already synced with the freshly created gist.
+        """
+        resolved_token = token or os.environ.get("GITHUB_TOKEN", "")
+        if not resolved_token:
+            raise ValueError(
+                "A GitHub token is required to create a gist. "
+                "Set GITHUB_TOKEN or pass token=."
+            )
+
+        if init_files:
+            files_payload = {
+                fname: {"content": json.dumps(data, ensure_ascii=False, default=str)}
+                for fname, data in init_files.items()
+            }
+        else:
+            files_payload = {"_gistfs.json": {"content": "{}"}}
+
+        payload: dict[str, Any] = {
+            "description": description,
+            "public": public,
+            "files": files_payload,
+        }
+        headers = {
+            "Accept": "application/vnd.github+json",
+            "Authorization": f"Bearer {resolved_token}",
+        }
+        resp = requests.post(GITHUB_API_GISTS_BASE, headers=headers, json=payload)
+        resp.raise_for_status()
+
+        gist_id = resp.json()["id"]
+        instance = cls(gist_id, token=resolved_token)
+        instance.sync()
+        return instance
+
+    # ── context manager ──────────────────────────────────────────────
+
+    def __enter__(self) -> GistFS:
+        self.sync()
+        return self
+
+    def __exit__(self, exc_type, exc_val, exc_tb) -> None:  # noqa: ANN001
+        self._cache = None
+        return None
+
+    # ── public API ───────────────────────────────────────────────────
+
+    def sync(self) -> None:
+        """Fetch the latest gist state from GitHub."""
+        resp = self._get_request()
+        resp.raise_for_status()
+        gist = resp.json()
+        self._cache = {}
+        for fname, fdata in gist.get("files", {}).items():
+            try:
+                self._cache[fname] = json.loads(fdata["content"])
+            except (json.JSONDecodeError, TypeError):
+                self._cache[fname] = fdata["content"]
+
+    def read(self, filename: str) -> Any:
+        """Read and return the parsed content of *filename*."""
+        self._ensure_synced()
+        assert self._cache is not None
+        if filename not in self._cache:
+            raise FileNotFoundError(f"{filename!r} not found in gist {self.gist_id}")
+        return self._cache[filename]
+
+    def write(self, filename: str, data: Any) -> None:
+        """Write *data* (JSON-serializable) to *filename* in the gist."""
+        content = json.dumps(data, ensure_ascii=False, default=str)
+        payload = {"files": {filename: {"content": content}}}
+        resp = self._patch_request(payload)
+        resp.raise_for_status()
+        # Update local cache
+        self._ensure_synced()
+        assert self._cache is not None
+        self._cache[filename] = data
+
+    def delete(self, filename: str) -> bool:
+        """Delete *filename* from the gist.  Returns True if it existed."""
+        self._ensure_synced()
+        assert self._cache is not None
+        if filename not in self._cache:
+            return False
+        # GitHub API: setting content to empty string with null deletes the file
+        payload = {"files": {filename: None}}
+        resp = self._patch_request(payload)
+        resp.raise_for_status()
+        self._cache.pop(filename, None)
+        return True
+
+    def list_files(self) -> list[str]:
+        """Return a list of filenames in the gist."""
+        self._ensure_synced()
+        assert self._cache is not None
+        return list(self._cache.keys())
+
+    def exists(self, filename: str) -> bool:
+        """Check whether *filename* exists in the gist."""
+        self._ensure_synced()
+        assert self._cache is not None
+        return filename in self._cache
+
+    def read_all(self) -> dict[str, Any]:
+        """Return all files as a ``{filename: content}`` dict."""
+        self._ensure_synced()
+        assert self._cache is not None
+        return dict(self._cache)
+
+    def open(self, filename: str, mode: str = "r") -> GistFile:
+        """Open a file in the gist, returning a file-like object.
+
+        Supported modes: ``"r"`` (read), ``"w"`` (write), ``"a"`` (append),
+        ``"r+"`` (read and write).
+
+        Usage::
+
+            with gfs.open("notes.txt", "w") as f:
+                f.write("hello world")
+
+            with gfs.open("notes.txt", "r") as f:
+                content = f.read()
+        """
+        if mode not in ("r", "w", "a", "r+"):
+            raise ValueError(f"Unsupported mode {mode!r} — use 'r', 'w', 'a', or 'r+'")
+        return GistFile(self, filename, mode)
+
+    # ── helpers ──────────────────────────────────────────────────────
+
+    def _ensure_synced(self) -> None:
+        if self._cache is None:
+            if self.auto_sync:
+                self.sync()
+            else:
+                raise RuntimeError("GistFS not synced — call .sync() first")
+
+    def _headers(self, *, auth: bool = True) -> dict[str, str]:
+        headers = {"Accept": "application/vnd.github+json"}
+        if auth and self.token:
+            headers["Authorization"] = f"Bearer {self.token}"
+        return headers
+
+    def _get_request(self) -> requests.Response:
+        return requests.get(self._url, headers=self._headers(auth=bool(self.token)))
+
+    def _patch_request(self, payload: dict) -> requests.Response:
+        if not self.token:
+            raise ValueError(
+                "A GitHub token is required for write operations. "
+                "Set GITHUB_TOKEN or pass token= to GistFS()."
+            )
+        return requests.patch(self._url, headers=self._headers(), json=payload)
+
+    def __repr__(self) -> str:
+        return f"GistFS(gist_id={self.gist_id!r})"
+
+
+class GistFile(io.StringIO):
+    """File-like object for a single file in a gist.
+
+    Wraps :class:`io.StringIO` so that standard file operations
+    (``read``, ``write``, ``seek``, ``tell``, iteration) work as expected.
+    On :meth:`close` (or exiting the context manager), any writes are
+    flushed back to the gist.
+
+    Do not instantiate directly — use :meth:`GistFS.open` instead.
+    """
+
+    def __init__(self, gfs: GistFS, filename: str, mode: str) -> None:
+        self._gfs = gfs
+        self._filename = filename
+        self._mode = mode
+        self._writable = mode in ("w", "a", "r+")
+
+        # Seed the buffer with existing content for read / append / r+
+        initial = ""
+        if mode in ("r", "a", "r+"):
+            try:
+                content = gfs.read(filename)
+                if isinstance(content, str):
+                    initial = content
+                else:
+                    initial = json.dumps(content, ensure_ascii=False, default=str)
+            except FileNotFoundError:
+                if mode == "r":
+                    raise
+                # "a" and "r+" start empty if file doesn't exist
+
+        super().__init__(initial)
+
+        # Position cursor at end for append, at start for read/r+
+        if mode == "a":
+            self.seek(0, io.SEEK_END)
+        else:
+            self.seek(0)
+
+    @property
+    def name(self) -> str:
+        return self._filename
+
+    @property
+    def mode(self) -> str:
+        return self._mode
+
+    def writable(self) -> bool:
+        return self._writable
+
+    def readable(self) -> bool:
+        return self._mode in ("r", "r+")
+
+    def write(self, s: str) -> int:
+        if not self._writable:
+            raise io.UnsupportedOperation("not writable")
+        return super().write(s)
+
+    def read(self, size: int = -1) -> str:
+        if self._mode == "w":
+            raise io.UnsupportedOperation("not readable")
+        return super().read(size)
+
+    def readline(self, size: int = -1) -> str:
+        if self._mode == "w":
+            raise io.UnsupportedOperation("not readable")
+        return super().readline(size)
+
+    def readlines(self, hint: int = -1) -> list[str]:
+        if self._mode == "w":
+            raise io.UnsupportedOperation("not readable")
+        return super().readlines(hint)
+
+    def close(self) -> None:
+        if not self.closed and self._writable:
+            self._flush_to_gist()
+        super().close()
+
+    def _flush_to_gist(self) -> None:
+        """Push the buffer content to the gist as a raw string."""
+        content = self.getvalue()
+        if not content:
+            return
+        payload = {"files": {self._filename: {"content": content}}}
+        resp = self._gfs._patch_request(payload)
+        resp.raise_for_status()
+        # Update cache
+        self._gfs._ensure_synced()
+        assert self._gfs._cache is not None
+        try:
+            self._gfs._cache[self._filename] = json.loads(content)
+        except (json.JSONDecodeError, ValueError):
+            self._gfs._cache[self._filename] = content
+
+    def __repr__(self) -> str:
+        state = "closed" if self.closed else self._mode
+        return f"GistFile({self._filename!r}, {state!r})"