qhaway 0.1.0__tar.gz

qhaway-0.1.0/PKG-INFO

@@ -0,0 +1,167 @@
+Metadata-Version: 2.4
+Name: qhaway
+Version: 0.1.0
+Summary: A truncation-proof projection of a Markdown memory index: regenerate MEMORY.md to fit the loader's budget and declare what it sets aside, instead of being silently cut.
+Keywords: memory,index,markdown,llm,agent,sqlite,mcp
+Author: Tony
+Author-email: Tony <fsgeek@cs.ubc.ca>
+License-Expression: MIT
+Classifier: Development Status :: 2 - Pre-Alpha
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Utilities
+Requires-Dist: mcp[cli]>=1.28.0
+Requires-Dist: pyyaml
+Requires-Python: >=3.14
+Description-Content-Type: text/markdown
+
+# qhaway
+
+*Quechua: "to see / to watch over."* The name states the cure — make the whole
+memory record **visible** instead of silently truncated.
+
+`qhaway` keeps a Markdown memory index from being silently cut off when it grows
+past the size limit of the system that loads it.
+
+## The problem
+
+Some agents and tools maintain memory as a directory of small Markdown files plus
+a single curated index (`MEMORY.md`) that points at them. The index is loaded into
+context on startup so the agent boots with a map of what it knows.
+
+That index grows. When it grows past the loader's size limit, it is **silently
+truncated** — cut off with no error raised. The agent boots a *partial self* and
+doesn't know it: everything past the cut is invisible, and a pointer to a file
+that no longer exists rides along just as silently. The honest record is there on
+disk; the loaded view of it is a lie of omission.
+
+This was observed live: a 36.8KB / 137-entry index against a ~24.4KB load limit,
+with the entire latest section — including the pointer to the most recent state —
+falling past the cut.
+
+## The fix
+
+qhaway regenerates `MEMORY.md` itself as a **truncation-proof projection** of the
+memory files:
+
+- **Files stay the write surface.** You keep writing topic `.md` files exactly as
+  you do today. There is no schema to learn and no "save" API to call. qhaway only
+  changes *who writes the index* — a machine, not a hand.
+- **It fits the budget.** The regenerated index is guaranteed to come in under the
+  loader's limit, so it is never silently cut.
+- **No silent loss — ever.** When the index can't fit everything, it doesn't drop
+  entries quietly. It **declares the omission**:
+
+  ```
+  +47 project memories not shown — run: qhaway index --type project
+  ```
+
+  Truncation becomes *visible selection*. You always know what was set aside and
+  how to see it.
+
+The loader keeps reading `MEMORY.md` exactly as before — now complete-for-what-it-
+claims and guaranteed under budget. Nothing downstream changes.
+
+## Install
+
+```sh
+uv tool install qhaway
+# or
+pipx install qhaway
+```
+
+Embedded and zero-infra: it uses stdlib SQLite (WAL mode) as a single local
+file. No server, no database to provision, no credentials.
+
+## Usage
+
+```sh
+# Regenerate MEMORY.md from the memory directory (the main command)
+qhaway index
+
+# See a specific slice — including entries the default index declared as omitted
+qhaway index --type project
+qhaway index --role <role>
+qhaway index --status superseded
+
+# Set a custom budget
+qhaway index --budget <bytes>
+
+# Inspect without writing: would it overflow? any broken links? any leftover files?
+qhaway index --check
+
+# Print the projection without writing the file
+qhaway index --dry-run
+```
+
+To record a memory: **write a topic `.md` file, then run `qhaway index`.** Don't
+hand-edit `MEMORY.md` — it is fully derived, and any hand edit is preserved (see
+below) but won't survive into the index unless it lives in a topic file.
+
+## MCP spine (remember / recall)
+
+The spine lets a Claude Code instance reach its memory through MCP tools instead
+of hand-writing files. `MEMORY.md` becomes a managed, read-only **redirect** into
+the SQLite-derived index; the topic files stay the source of truth.
+
+```sh
+# Run the MCP server over a memory directory (reconciles once at startup)
+qhaway serve --dir <memory_dir>
+
+# Sync the index from the files (alias: qhaway index)
+qhaway reconcile --dir <memory_dir>
+
+# Inspect: broken wikilinks, orphan backups, low topic count, would-overflow
+qhaway check --dir <memory_dir>
+```
+
+Two verbs are exposed to the model:
+
+- `recall(type?, role?, status?)` — pure read; returns the budgeted projection
+  (omit args for the working set).
+- `remember(type, title, body, description?, links?)` — writes a topic file then
+  reconciles. Files stay truth; the DB is a derived, rebuildable view.
+
+`MEMORY.md` is written born-read-only (`0o444`) as a friction signal — not a hard
+barrier — so the reflexive hand-edit is deflected toward the tools. qhaway's own
+writer updates it via atomic temp-file + replace.
+
+## How it works
+
+```
+qhaway index
+  → scan the memory directory
+  → parse each file into a node (frontmatter type, filename role, links, body)
+  → build an index of nodes + links in SQLite
+  → project the working set under the byte budget,
+    appending a declared-omissions footer for anything set aside
+  → write MEMORY.md
+```
+
+The memory files are the single source of truth. The index is rebuilt from scratch
+on every run, so it can never drift from the files. The same files always produce
+a byte-identical index.
+
+### What's preserved
+
+`MEMORY.md` is fully machine-derived — there are no hand-maintained regions. If
+qhaway ever finds that the index was edited by hand since it last wrote it, it does
+**not** overwrite the edit: it renames the existing file to a timestamped
+`MEMORY-<timestamp>.md` and writes a fresh index. Your edit is preserved verbatim;
+the index rebuilds from the files. Nothing is interpreted, merged, or lost.
+
+## Design philosophy
+
+One pain, fixed completely: **truncation**. Full-text search, deep audit, write
+tooling, and ranking sophistication are deliberately *not* in this version — each
+is a real later idea, none is this version's job.
+
+The wager is simple: a structured index built *over* an existing pile of files —
+without replacing the pile — makes the whole thing measurably work better. The
+proof is use. If it removes felt pain for skeptical users who'll drop it the moment
+it's more friction than value, it ships; if it removes the same pain for strangers
+feeling the same sprawl, it spreads. Propagation is the measurement.
+
+## Status
+
+Early (`v0.1.0`). The design is specified in
+[`docs/superpowers/specs/2026-06-20-qhaway-mvp-design.md`](docs/superpowers/specs/2026-06-20-qhaway-mvp-design.md).

qhaway-0.1.0/README.md

@@ -0,0 +1,151 @@
+# qhaway
+
+*Quechua: "to see / to watch over."* The name states the cure — make the whole
+memory record **visible** instead of silently truncated.
+
+`qhaway` keeps a Markdown memory index from being silently cut off when it grows
+past the size limit of the system that loads it.
+
+## The problem
+
+Some agents and tools maintain memory as a directory of small Markdown files plus
+a single curated index (`MEMORY.md`) that points at them. The index is loaded into
+context on startup so the agent boots with a map of what it knows.
+
+That index grows. When it grows past the loader's size limit, it is **silently
+truncated** — cut off with no error raised. The agent boots a *partial self* and
+doesn't know it: everything past the cut is invisible, and a pointer to a file
+that no longer exists rides along just as silently. The honest record is there on
+disk; the loaded view of it is a lie of omission.
+
+This was observed live: a 36.8KB / 137-entry index against a ~24.4KB load limit,
+with the entire latest section — including the pointer to the most recent state —
+falling past the cut.
+
+## The fix
+
+qhaway regenerates `MEMORY.md` itself as a **truncation-proof projection** of the
+memory files:
+
+- **Files stay the write surface.** You keep writing topic `.md` files exactly as
+  you do today. There is no schema to learn and no "save" API to call. qhaway only
+  changes *who writes the index* — a machine, not a hand.
+- **It fits the budget.** The regenerated index is guaranteed to come in under the
+  loader's limit, so it is never silently cut.
+- **No silent loss — ever.** When the index can't fit everything, it doesn't drop
+  entries quietly. It **declares the omission**:
+
+  ```
+  +47 project memories not shown — run: qhaway index --type project
+  ```
+
+  Truncation becomes *visible selection*. You always know what was set aside and
+  how to see it.
+
+The loader keeps reading `MEMORY.md` exactly as before — now complete-for-what-it-
+claims and guaranteed under budget. Nothing downstream changes.
+
+## Install
+
+```sh
+uv tool install qhaway
+# or
+pipx install qhaway
+```
+
+Embedded and zero-infra: it uses stdlib SQLite (WAL mode) as a single local
+file. No server, no database to provision, no credentials.
+
+## Usage
+
+```sh
+# Regenerate MEMORY.md from the memory directory (the main command)
+qhaway index
+
+# See a specific slice — including entries the default index declared as omitted
+qhaway index --type project
+qhaway index --role <role>
+qhaway index --status superseded
+
+# Set a custom budget
+qhaway index --budget <bytes>
+
+# Inspect without writing: would it overflow? any broken links? any leftover files?
+qhaway index --check
+
+# Print the projection without writing the file
+qhaway index --dry-run
+```
+
+To record a memory: **write a topic `.md` file, then run `qhaway index`.** Don't
+hand-edit `MEMORY.md` — it is fully derived, and any hand edit is preserved (see
+below) but won't survive into the index unless it lives in a topic file.
+
+## MCP spine (remember / recall)
+
+The spine lets a Claude Code instance reach its memory through MCP tools instead
+of hand-writing files. `MEMORY.md` becomes a managed, read-only **redirect** into
+the SQLite-derived index; the topic files stay the source of truth.
+
+```sh
+# Run the MCP server over a memory directory (reconciles once at startup)
+qhaway serve --dir <memory_dir>
+
+# Sync the index from the files (alias: qhaway index)
+qhaway reconcile --dir <memory_dir>
+
+# Inspect: broken wikilinks, orphan backups, low topic count, would-overflow
+qhaway check --dir <memory_dir>
+```
+
+Two verbs are exposed to the model:
+
+- `recall(type?, role?, status?)` — pure read; returns the budgeted projection
+  (omit args for the working set).
+- `remember(type, title, body, description?, links?)` — writes a topic file then
+  reconciles. Files stay truth; the DB is a derived, rebuildable view.
+
+`MEMORY.md` is written born-read-only (`0o444`) as a friction signal — not a hard
+barrier — so the reflexive hand-edit is deflected toward the tools. qhaway's own
+writer updates it via atomic temp-file + replace.
+
+## How it works
+
+```
+qhaway index
+  → scan the memory directory
+  → parse each file into a node (frontmatter type, filename role, links, body)
+  → build an index of nodes + links in SQLite
+  → project the working set under the byte budget,
+    appending a declared-omissions footer for anything set aside
+  → write MEMORY.md
+```
+
+The memory files are the single source of truth. The index is rebuilt from scratch
+on every run, so it can never drift from the files. The same files always produce
+a byte-identical index.
+
+### What's preserved
+
+`MEMORY.md` is fully machine-derived — there are no hand-maintained regions. If
+qhaway ever finds that the index was edited by hand since it last wrote it, it does
+**not** overwrite the edit: it renames the existing file to a timestamped
+`MEMORY-<timestamp>.md` and writes a fresh index. Your edit is preserved verbatim;
+the index rebuilds from the files. Nothing is interpreted, merged, or lost.
+
+## Design philosophy
+
+One pain, fixed completely: **truncation**. Full-text search, deep audit, write
+tooling, and ranking sophistication are deliberately *not* in this version — each
+is a real later idea, none is this version's job.
+
+The wager is simple: a structured index built *over* an existing pile of files —
+without replacing the pile — makes the whole thing measurably work better. The
+proof is use. If it removes felt pain for skeptical users who'll drop it the moment
+it's more friction than value, it ships; if it removes the same pain for strangers
+feeling the same sprawl, it spreads. Propagation is the measurement.
+
+## Status
+
+Early (`v0.1.0`). The design is specified in
+[`docs/superpowers/specs/2026-06-20-qhaway-mvp-design.md`](docs/superpowers/specs/2026-06-20-qhaway-mvp-design.md).

qhaway-0.1.0/pyproject.toml

@@ -0,0 +1,34 @@
+[project]
+name = "qhaway"
+version = "0.1.0"
+description = "A truncation-proof projection of a Markdown memory index: regenerate MEMORY.md to fit the loader's budget and declare what it sets aside, instead of being silently cut."
+readme = "README.md"
+requires-python = ">=3.14"
+license = "MIT"
+authors = [{ name = "Tony", email = "fsgeek@cs.ubc.ca" }]
+keywords = ["memory", "index", "markdown", "llm", "agent", "sqlite", "mcp"]
+classifiers = [
+    "Development Status :: 2 - Pre-Alpha",
+    "Programming Language :: Python :: 3",
+    "Topic :: Utilities",
+]
+dependencies = [
+    "mcp[cli]>=1.28.0",
+    "pyyaml",
+]
+
+[project.scripts]
+qhaway = "qhaway.cli:main"
+
+[build-system]
+requires = ["uv_build>=0.10,<0.11"]
+build-backend = "uv_build"
+
+[tool.uv.build-backend]
+module-root = "src"
+
+# Publishing (deferred — no release until the implementation exists and its tests
+# pass). When ready, tokens are read from the environment at publish time and are
+# NEVER stored in this file or committed:
+#   TestPyPI: uv publish --index testpypi   (token: PYPI_DEPLOY_KEY_TEST)
+#   PyPI:     uv publish                     (token: PYPI_DEPLOY_KEY_REAL)

qhaway-0.1.0/src/qhaway/__init__.py

@@ -0,0 +1,6 @@
+"""qhaway — a truncation-proof projection of a Markdown memory index.
+
+See the design spec: docs/superpowers/specs/2026-06-20-qhaway-mvp-design.md
+"""
+
+__version__ = "0.1.0"

qhaway-0.1.0/src/qhaway/cli.py

@@ -0,0 +1,238 @@
+"""Entry point for the `qhaway` command."""
+
+from __future__ import annotations
+
+import argparse
+import os
+import sys
+from pathlib import Path
+
+from qhaway import model, parse, project, server
+from qhaway import reconcile as reconcile_mod
+from qhaway.reconcile import reconcile
+
+MEMORY_NAME = "MEMORY.md"
+
+
+def main(args: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(prog="qhaway")
+    sub = parser.add_subparsers(dest="command", required=True)
+    for name in ("reconcile", "check", "serve", "index", "exit"):
+        p = sub.add_parser(name)
+        p.add_argument("--dir")
+        p.add_argument("--budget", type=int, default=project.DEFAULT_BUDGET)
+        p.add_argument("--type", dest="content_type")
+        p.add_argument("--role")
+        p.add_argument("--status", default="live")
+        p.add_argument("--dry-run", action="store_true")
+        p.add_argument("--check", action="store_true")  # deprecated alias on index
+        p.add_argument("--emit", action="store_true")
+
+    ns = parser.parse_args(args)
+    directory = _resolve_dir(ns)
+
+    if ns.command == "serve":
+        return _serve(directory)
+    if ns.command == "exit":
+        return _exit(directory, ns.budget)
+    if ns.command == "check" or (ns.command == "index" and ns.check):
+        return _check(directory, ns.budget)
+    if ns.command == "index" and ns.dry_run:
+        return _dry_run(directory, ns)
+
+    # reconcile, and index-as-reconcile-alias
+    try:
+        reconcile(directory)
+    except FileNotFoundError as exc:
+        sys.stderr.write(f"{exc}\n")
+        return 1
+    if getattr(ns, "emit", False):
+        conn = model.get_connection(directory)
+        try:
+            sys.stdout.write(project.project_slice(conn, budget=ns.budget))
+        finally:
+            conn.close()
+    return 0
+
+
+def _resolve_dir(ns) -> str:
+    return ns.dir or os.environ.get("QHAWAY_MEMORY_DIR") or "."
+
+
+def _serve(directory: str) -> int:
+    if not os.path.isdir(directory):
+        sys.stderr.write(f"memory directory is not readable: {directory}\n")
+        return 1
+    server.run(directory)
+    return 0
+
+
+def _dry_run(directory: str, ns) -> int:
+    if not os.path.isdir(directory):
+        sys.stderr.write(f"memory directory is not readable: {directory}\n")
+        return 1
+    conn = model.get_connection(directory)
+    try:
+        output = project.project_slice(
+            conn,
+            budget=ns.budget,
+            content_type=ns.content_type,
+            role=ns.role,
+            status=ns.status,
+        )
+    finally:
+        conn.close()
+    sys.stdout.write(output)
+    return 0
+
+
+def _exit(directory: str, budget: int) -> int:
+    """SessionEnd: leave a current, self-sufficient, truncation-proof index in
+    place — NOT the pre-install original. qhaway borrows MEMORY.md while enabled
+    and returns a current honest index when it leaves; the original is preserved
+    under its distinguished name (MEMORY.preinstall.md) for an explicit uninstall,
+    never handed back here. Worst case (a future loader truncates the file), the
+    index degrades gracefully and declares what it set aside; the raw original
+    would truncate into silent staleness — the exact failure qhaway prevents.
+
+    The index is budgeted (the footer's bytes are reserved within the budget, not
+    appended past it) and carries no recall()/remember() instructions, since the
+    hooks are not guaranteed to run once the plugin is disabled.
+    """
+    memory_dir = Path(directory)
+    if not memory_dir.is_dir():
+        sys.stderr.write(f"memory directory is not readable: {memory_dir}\n")
+        return 1
+
+    reconcile(directory)
+    conn = model.get_connection(directory)
+    try:
+        total = len(model.topic_files(memory_dir))
+
+        def compose_footer(omitted: int) -> str:
+            return (
+                f"\n\n---\n_qhaway exit index — {total} memories under {budget} "
+                f"bytes; {omitted} set aside. Self-sufficient static index "
+                "(qhaway disabled)._\n"
+            )
+
+        # Probe at full budget only to SIZE the reserve (footer + signature bytes);
+        # the displayed "set aside" count comes from the FINAL reduced-budget
+        # projection, so the footer reports what the shipped file actually omits —
+        # honest declaration is the whole point. The count's digit width is bounded
+        # (a few hundred memories at most), so any drift between probe and final
+        # count is sub-byte against the reserve and never pushes over budget.
+        probe = project.project_slice_with_overflow(conn, budget=budget)
+        reserve = (
+            len(compose_footer(sum(probe.overflow.omitted_counts.values())).encode("utf-8"))
+            + len(reconcile_mod.signature_line(""))
+            + 2
+        )
+        result = project.project_slice_with_overflow(conn, budget=max(0, budget - reserve))
+        footer = compose_footer(sum(result.overflow.omitted_counts.values()))
+    finally:
+        conn.close()
+    reconcile_mod.write_readonly(
+        memory_dir / MEMORY_NAME, reconcile_mod.embed_signature(result.markdown + footer)
+    )
+    return 0
+
+
+def _check(directory: str, budget: int) -> int:
+    memory_dir = Path(directory)
+    if not memory_dir.is_dir():
+        sys.stderr.write(f"memory directory is not readable: {memory_dir}\n")
+        return 1
+
+    exit_code = 0
+    topic_count = len(model.topic_files(memory_dir))
+    if topic_count <= 2:
+        sys.stderr.write(f"warning: low topic file count ({topic_count}) in {memory_dir}\n")
+
+    orphans = _orphan_files(memory_dir)
+    if orphans:
+        sys.stdout.write(f"{len(orphans)} orphan MEMORY backups found:\n")
+        for orphan in orphans:
+            sys.stdout.write(f"- {orphan.name}\n")
+
+    conn = model.get_connection(directory)
+    try:
+        dangling = _dangling_links(conn)
+        stale_drift = _stale_drift(conn)
+        full_projection = project.project_slice(conn, budget=10**12)
+    finally:
+        conn.close()
+
+    if dangling:
+        exit_code = 1
+        sys.stdout.write("dangling topic wikilinks found:\n")
+        for src_file, dst_slug in dangling:
+            sys.stdout.write(f"- {src_file} -> [[{dst_slug}]]\n")
+
+    if stale_drift:
+        exit_code = 1
+        sys.stdout.write(
+            "live memories whose body announces supersession but whose name: was "
+            "never redirected (they leak into the working set):\n"
+        )
+        for file_name, marker in stale_drift:
+            sys.stdout.write(f"- {file_name} (body says {marker}; retire it: set name: 'SUPERSEDED — see ...')\n")
+
+    if len(full_projection.encode("utf-8")) > budget:
+        overflow = len(full_projection.encode("utf-8")) - budget
+        exit_code = 1
+        sys.stderr.write(f"corpus exceeds budget by {overflow} bytes before projection\n")
+
+    if exit_code == 0 and not orphans and topic_count > 2:
+        sys.stdout.write("qhaway check passed\n")
+    return exit_code
+
+
+def _stale_drift(conn) -> list[tuple[str, str]]:
+    """Find live nodes whose body announces supersession but whose name: was
+    never rewritten to the redirect form — so parse left status=live and the
+    projector serves them as current. This is the silent-staleness leak: the
+    conscientious in-body 'SUPERSEDED' annotation never reaches the one field
+    the retire path keys on. Conservative by design (a tombstone word as a
+    leading/emphasized token on its own line, not a passing mention) so a
+    correctly-live memory that merely discusses supersession is not nagged.
+    """
+    drift: list[tuple[str, str]] = []
+    for file_name, status, body in conn.execute(
+        "SELECT file, status, body FROM nodes ORDER BY file"
+    ).fetchall():
+        if status != "live":
+            continue
+        marker = _body_supersession_marker(body or "")
+        if marker:
+            drift.append((file_name, marker))
+    return drift
+
+
+def _body_supersession_marker(body: str) -> str | None:
+    for raw in body.splitlines():
+        line = raw.strip().lstrip("*_# ").strip()
+        upper = line.upper()
+        for word in parse.TOMBSTONE_NAMES:
+            if upper.startswith(word):
+                return word
+    return None
+
+
+def _dangling_links(conn) -> list[tuple[str, str]]:
+    stems = {row[0].removesuffix(".md") for row in conn.execute("SELECT file FROM nodes").fetchall()}
+    dangling: list[tuple[str, str]] = []
+    for src_file, dst_slug in conn.execute(
+        "SELECT src_file, dst_slug FROM edges ORDER BY src_file, dst_slug"
+    ).fetchall():
+        if dst_slug not in stems:
+            dangling.append((src_file, dst_slug))
+    return dangling
+
+
+def _orphan_files(memory_dir: Path) -> list[Path]:
+    return sorted(memory_dir.glob("MEMORY-*.md"), key=lambda path: path.name)
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())