sqlite-graph-memory 0.3.2__tar.gz → 0.3.3__tar.gz

{sqlite_graph_memory-0.3.2 → sqlite_graph_memory-0.3.3}/CHANGELOG.md

@@ -4,6 +4,28 @@ What shipped, in plain words. Entries below v0.1.3 were written on 2026-09-05 fr
 release notes that already existed — the file itself did not exist until then, so those three are
 recorded after the fact rather than backdated to look contemporaneous.
 
+## v0.3.3 — 2026-10-26
+
+**Hidden note files are skipped like hidden folders.** A note whose filename starts with a
+dot, such as `.ideas.md` or `.draft.md`, used to be indexed when it sat in the vault root or a
+visible folder, while the same file inside `.drafts/` was skipped. Now both are skipped, so
+after re-indexing those notes stop showing up in answers. `BRAIN_INDEX_HIDDEN=1` brings back
+the old behaviour and indexes hidden files and folders alike. `eval/build_gold.py` now walks the
+vault with the indexer's own filter, so it follows the same rule and the same variable instead
+of keeping its own copy (issue #25).
+
+**The MCP server works from a source checkout and finds your index where you launch it**
+(issue #24, reported by @Kaap10). `sgm-mcp` used to run the recall child with its working
+directory set to the package folder, so a relative `BRAIN_INDEX_DIR=index` resolved inside
+site-packages and your own index was never found; from a plain clone without `pip install -e`,
+`python src/sqlite_graph_memory/mcp_server.py --test` failed with ModuleNotFoundError. Recall now
+runs in the server's own working directory, and CI reproduces the user's launch: installed wheel,
+relative index, started from another directory.
+
+**A three-question feedback form.** "What were you trying to do, what broke, which machine": an
+issue template, a README section, and a link printed next to the missing-index error, because
+this project collects no telemetry and that form is how we learn what is missing.
+
 ## v0.3.2 — 2026-10-24
 
 **The PageRank lane the README credits is now in the package** (#51). `ppr.py` runs

{sqlite_graph_memory-0.3.2 → sqlite_graph_memory-0.3.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: sqlite-graph-memory
-Version: 0.3.2
+Version: 0.3.3
 Summary: Associative memory for LLM agents: Graph RAG over a folder of markdown notes, in one SQLite file, with no server to run.
 Project-URL: Homepage, https://github.com/tonydzi/sqlite-graph-memory
 Project-URL: Repository, https://github.com/tonydzi/sqlite-graph-memory
@@ -126,7 +126,7 @@ points, the `[[wikilinks]]` you already wrote by hand are the graph, and a cross
 reranks what the hop dragged in.
 SQLite is the only database ([`schema.sql`](src/sqlite_graph_memory/schema.sql)).
 
-Status: **pilot**, and the word is load-bearing: it runs daily in one real setup (a ~100k-note
+Status: **pilot**, and the word is load-bearing: it runs daily in one real setup (an index of ~28k curated notes, ~93k chunks, rebuilt nightly from a ~270k-file
 Obsidian vault driven by Claude Code), but it is deliberately minimal and makes no attempt
 to be general — see [What's intentionally missing](#whats-intentionally-missing), which is
 where the uneven test coverage is described.
@@ -252,7 +252,7 @@ present, and CPU works fine for small corpora.
 
 ### What `--ab` looks like
 
-Output of `brain_ask.py` with `--ab`, on a toy 4-note corpus (real runs use a ~100k-note vault):
+Output of `brain_ask.py` with `--ab`, on a toy 4-note corpus (real runs use an index of ~28k notes / ~93k chunks):
 
 ```
 A/B RECALL: how should agent memory work
@@ -491,6 +491,16 @@ Questions, war stories, or you want to run this on your own fleet:
 - 🌐 [palo-alto.ai](https://palo-alto.ai) · [Palo Alto AI Research Lab](https://github.com/tonydzi)
 - 🧪 **Engineers: want to test-drive this setup?** Message me — I hand out free starter seeds to engineers who test and report back.
 
+## Tell us what is missing
+
+You tried this on your own notes and something was missing or broke. That is the most useful
+thing anyone can tell us.
+
+- **You started and it did not work:** [open the 3-question form](https://github.com/tonydzi/sqlite-graph-memory/issues/new?template=feature_request.yml) (what you tried, what broke, which machine).
+- **Not sure it is a bug, or just an idea:** [Discussions → Ideas](https://github.com/tonydzi/sqlite-graph-memory/discussions/categories/ideas).
+
+Nothing in this package phones home. There is no telemetry, so we only know what you write to us.
+
 ## Contributors welcome — and there is a queue
 
 Issues labelled [`accepted`](https://github.com/tonydzi/sqlite-graph-memory/issues?q=is%3Aissue+is%3Aopen+label%3Aaccepted)

{sqlite_graph_memory-0.3.2 → sqlite_graph_memory-0.3.3}/README.md

@@ -75,7 +75,7 @@ points, the `[[wikilinks]]` you already wrote by hand are the graph, and a cross
 reranks what the hop dragged in.
 SQLite is the only database ([`schema.sql`](src/sqlite_graph_memory/schema.sql)).
 
-Status: **pilot**, and the word is load-bearing: it runs daily in one real setup (a ~100k-note
+Status: **pilot**, and the word is load-bearing: it runs daily in one real setup (an index of ~28k curated notes, ~93k chunks, rebuilt nightly from a ~270k-file
 Obsidian vault driven by Claude Code), but it is deliberately minimal and makes no attempt
 to be general — see [What's intentionally missing](#whats-intentionally-missing), which is
 where the uneven test coverage is described.
@@ -201,7 +201,7 @@ present, and CPU works fine for small corpora.
 
 ### What `--ab` looks like
 
-Output of `brain_ask.py` with `--ab`, on a toy 4-note corpus (real runs use a ~100k-note vault):
+Output of `brain_ask.py` with `--ab`, on a toy 4-note corpus (real runs use an index of ~28k notes / ~93k chunks):
 
 ```
 A/B RECALL: how should agent memory work
@@ -440,6 +440,16 @@ Questions, war stories, or you want to run this on your own fleet:
 - 🌐 [palo-alto.ai](https://palo-alto.ai) · [Palo Alto AI Research Lab](https://github.com/tonydzi)
 - 🧪 **Engineers: want to test-drive this setup?** Message me — I hand out free starter seeds to engineers who test and report back.
 
+## Tell us what is missing
+
+You tried this on your own notes and something was missing or broke. That is the most useful
+thing anyone can tell us.
+
+- **You started and it did not work:** [open the 3-question form](https://github.com/tonydzi/sqlite-graph-memory/issues/new?template=feature_request.yml) (what you tried, what broke, which machine).
+- **Not sure it is a bug, or just an idea:** [Discussions → Ideas](https://github.com/tonydzi/sqlite-graph-memory/discussions/categories/ideas).
+
+Nothing in this package phones home. There is no telemetry, so we only know what you write to us.
+
 ## Contributors welcome — and there is a queue
 
 Issues labelled [`accepted`](https://github.com/tonydzi/sqlite-graph-memory/issues?q=is%3Aissue+is%3Aopen+label%3Aaccepted)

{sqlite_graph_memory-0.3.2 → sqlite_graph_memory-0.3.3}/bench/README.md

@@ -140,7 +140,7 @@ has seen.
 
 ## Limits, stated up front
 
-- 202 notes is small. Dense retrieval at this size is easier than in a 100k-note vault,
+- 202 notes is small. Dense retrieval at this size is easier than in the ~28k-note production index,
   so the bench measures failure modes, and it does not show how a system scales.
 - The text is English only and comes from templates. The sentences are cleaner than real
   notes, and that flatters extractive systems.

{sqlite_graph_memory-0.3.2 → sqlite_graph_memory-0.3.3}/eval/build_gold.py

@@ -40,6 +40,9 @@ import sys
 from collections import Counter
 from pathlib import Path
 
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'src'))
+from sqlite_graph_memory.index_notes import iter_notes  # noqa: E402
+
 WIKILINK_RX = re.compile(r'\[\[([^\]\|#]+)(?:[|#][^\]]*)?\]\]')
 CYRILLIC_RX = re.compile(r'[а-яё]', re.I)
 
@@ -121,10 +124,10 @@ def read_notes(notes_dir, stems, folders=None):
     """Every indexed note, parsed into the fields the four classes need."""
     root = Path(notes_dir)
     paths = []
-    for p in sorted(root.rglob('*.md')):
+    # The indexer's own walk: hidden files and dirs, conflict copies and BRAIN_INDEX_HIDDEN
+    # are decided in one place, so a gold answer is never a note the indexer would skip.
+    for p in iter_notes(root):
         rel = p.relative_to(root)
-        if any(part.startswith('.') for part in rel.parts[:-1]): continue
-        if 'sync-conflict' in p.name: continue
         if folders and (not rel.parts[:-1] or rel.parts[0] not in folders): continue
         if p.stem.lower() not in stems: continue
         paths.append(p)

{sqlite_graph_memory-0.3.2 → sqlite_graph_memory-0.3.3}/examples/README.md

@@ -109,7 +109,7 @@ All components share standard configuration variables:
 | `BRAIN_INDEX_DIR` | `./index` | Directory where `index_notes.py` writes and `brain_ask.py` reads `_brain_e5.npy` and metadata. |
 | `TURNSTATE_DB` | `./turnstate.db` | SQLite database holding per-turn ledger (`turns`) and A/B telemetry (`ab_recall`). |
 | `BRAIN_ANSWER_OUT` | `./_brain_answer.txt` | File where the formatted context bundle or answer is mirrored. |
-| `BRAIN_INDEX_HIDDEN` | `0` | If `1`, indexes hidden dot-directories too (by default skipped). |
+| `BRAIN_INDEX_HIDDEN` | `0` | If `1`, indexes hidden dot-files (`.draft.md`) and dot-directories too (by default both are skipped). |
 | `BRAIN_AB_JSON` | `None` | If set, `--ab` mode writes a structured JSON diff to this file path. |
 | `AB_SOURCE` | `work` | Telemetry label written to `ab_recall.source` (`work`, `eval`, etc.). |
 

{sqlite_graph_memory-0.3.2 → sqlite_graph_memory-0.3.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "sqlite-graph-memory"
-version = "0.3.2"
+version = "0.3.3"
 description = "Associative memory for LLM agents: Graph RAG over a folder of markdown notes, in one SQLite file, with no server to run."
 readme = "README.md"
 requires-python = ">=3.9"

{sqlite_graph_memory-0.3.2 → sqlite_graph_memory-0.3.3}/src/sqlite_graph_memory/__init__.py

@@ -21,6 +21,6 @@ the indexer and the reranker will tell you what is missing instead of failing
 on an ``ImportError`` at startup.
 """
 
-__version__ = "0.3.2"
+__version__ = "0.3.3"
 
 __all__ = ["__version__"]

{sqlite_graph_memory-0.3.2 → sqlite_graph_memory-0.3.3}/src/sqlite_graph_memory/brain_ask.py

@@ -31,6 +31,7 @@ for _s in (sys.stdout, sys.stderr):
     try: _s.reconfigure(encoding='utf-8')
     except Exception: pass
 
+FEEDBACK_URL = 'https://github.com/tonydzi/sqlite-graph-memory/discussions/categories/ideas'
 INDEX_DIR = Path(os.getenv('BRAIN_INDEX_DIR', 'index'))
 AB_DB = Path(os.getenv('TURNSTATE_DB', 'turnstate.db'))
 EMB = INDEX_DIR / '_brain_e5.npy'
@@ -299,6 +300,8 @@ def main():
     query = ' '.join(a for a in args if a not in flags)
     if not query or not EMB.exists():
         sys.stderr.write('need an embedding index (%s) and a query — run index_notes.py first\n' % EMB)
+        if query and not EMB.exists():
+            print('Stuck? Tell us what you were trying to do: %s' % FEEDBACK_URL, file=sys.stderr)
         raise SystemExit(1)
 
     dev = pick_device()

{sqlite_graph_memory-0.3.2 → sqlite_graph_memory-0.3.3}/src/sqlite_graph_memory/index_notes.py

@@ -12,9 +12,9 @@ USAGE:
   python index_notes.py <notes_dir>
 Config (env, optional):
   BRAIN_INDEX_DIR     output dir (default: ./index)
-  BRAIN_INDEX_HIDDEN  1 = also index dot-directories (default: skip them,
-                      so .stversions/.obsidian/.git and Syncthing conflict
-                      copies never enter the index)
+  BRAIN_INDEX_HIDDEN  1 = also index dot-files and dot-directories (default:
+                      skip them, so .draft.md, .stversions/.obsidian/.git and
+                      Syncthing conflict copies never enter the index)
 """
 import os, re, sys, pickle
 from pathlib import Path
@@ -77,15 +77,15 @@ CONFLICT_MARKER = 'sync-conflict'
 def iter_notes(notes_dir):
     """Files that count as a note under ``notes_dir``.
 
-    Skips dot-directories and Syncthing conflict copies. Set ``BRAIN_INDEX_HIDDEN=1``
-    to index hidden folders too — someone will legitimately keep notes in one.
+    Skips dot-directories, dot-files (``.draft.md``) and Syncthing conflict copies.
+    Set ``BRAIN_INDEX_HIDDEN=1`` to index hidden paths too — someone will legitimately keep notes in one.
     """
     root = Path(notes_dir)
     include_hidden = os.getenv('BRAIN_INDEX_HIDDEN', '') not in ('', '0', 'false', 'no')
     out = []
     for p in root.rglob('*.md'):
         rel = p.relative_to(root)
-        if not include_hidden and any(part.startswith('.') for part in rel.parts[:-1]):
+        if not include_hidden and any(part.startswith('.') for part in rel.parts):
             continue
         if CONFLICT_MARKER in p.name:
             continue

{sqlite_graph_memory-0.3.2 → sqlite_graph_memory-0.3.3}/src/sqlite_graph_memory/mcp_server.py

@@ -131,7 +131,6 @@ def run_recall(query: str, mode: str = "associative") -> tuple[str, bool]:
         call_env["PYTHONPATH"] = f"{src_dir}{os.pathsep}{existing_pp}" if existing_pp else src_dir
         res = subprocess.run(
             cmd,
-            cwd=str(ROOT_DIR),
             env=call_env,
             capture_output=True,
             text=True,

{sqlite_graph_memory-0.3.2 → sqlite_graph_memory-0.3.3}/tests/test_eval.py

@@ -158,6 +158,25 @@ def test_a_bridge_question_records_how_many_links_were_cut(tmp_path):
     assert bridge['gold_truncated'] == 4
 
 
+def test_gold_skips_hidden_notes_exactly_like_the_indexer(tmp_path, monkeypatch):
+    """build_gold walks the vault with the indexer's own filter (issue #25).
+
+    A root-level `.draft.md` is never indexed, so it must never become a question either,
+    even when a stale index still lists its stem. `BRAIN_INDEX_HIDDEN=1` lets it into the
+    index, so it must let it into the gold set too: one rule, one place.
+    """
+    vault = _vault(tmp_path)
+    (vault / '.draft.md').write_text('---\ntitle: "Hidden draft"\n---\n\n' + PARA + '\n',
+                                     encoding='utf-8')
+    stems = INDEXED | {'.draft'}
+
+    monkeypatch.delenv('BRAIN_INDEX_HIDDEN', raising=False)
+    assert '.draft' not in {n['stem'] for n in read_notes(vault, stems)}
+
+    monkeypatch.setenv('BRAIN_INDEX_HIDDEN', '1')
+    assert '.draft' in {n['stem'] for n in read_notes(vault, stems)}
+
+
 def test_a_filename_living_in_two_folders_is_reported_as_ambiguous(tmp_path):
     """Found by the review panel: `[[foo]]` names both files, so neither can be gold."""
     vault = _vault(tmp_path)

sqlite_graph_memory-0.3.3/tests/test_feedback_link.py

@@ -0,0 +1,29 @@
+"""A missing index must say where to tell us about it (stderr), without needing a model."""
+import sys
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "src"))
+
+from sqlite_graph_memory import brain_ask  # noqa: E402
+
+
+def test_missing_index_prints_feedback_link(tmp_path, monkeypatch, capsys):
+    monkeypatch.setattr(brain_ask, "EMB", tmp_path / "nope.npy")
+    monkeypatch.setattr(sys, "argv", ["sgm-ask", "anything"])
+    with pytest.raises(SystemExit) as exc:
+        brain_ask.main()
+    assert exc.value.code == 1
+    out = capsys.readouterr()
+    assert "need an embedding index" in out.err
+    assert brain_ask.FEEDBACK_URL in out.err
+
+
+def test_empty_query_does_not_nag(tmp_path, monkeypatch, capsys):
+    monkeypatch.setattr(brain_ask, "EMB", tmp_path / "nope.npy")
+    monkeypatch.setattr(sys, "argv", ["sgm-ask"])
+    with pytest.raises(SystemExit) as exc:
+        brain_ask.main()
+    assert exc.value.code == 1
+    assert brain_ask.FEEDBACK_URL not in capsys.readouterr().err

sqlite_graph_memory-0.3.3/tests/test_index_notes_dotfiles.py

@@ -0,0 +1,81 @@
+# -*- coding: utf-8 -*-
+"""Dot-FILES are skipped the same way dot-DIRECTORIES are (issue #25).
+
+`iter_notes` filtered hidden *directories* by checking `rel.parts[:-1]` — every
+path component except the last. The last component is the filename, so a note
+called `.draft.md` or `.trash.md` sitting in the vault root (or in any visible
+folder) was indexed as if it were a real note, while the same file one level
+down inside `.drafts/` was correctly skipped. Reproduced 2026-10-13 with
+`.secret.md` next to `ok.md`: `iter_notes()` returned both.
+
+RED-FIRST: restore `rel.parts[:-1]` on the filter line and
+`test_dot_file_in_root_is_skipped` fails with `.draft.md` in the result.
+
+Both halves are pinned on purpose — the directory case already worked and must
+keep working, the file case is the fix — and `BRAIN_INDEX_HIDDEN=1` must still
+let both through, because someone legitimately keeps notes in a hidden folder.
+
+No model download, no network: pure file work.
+"""
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
+from sqlite_graph_memory.index_notes import iter_notes  # noqa: E402
+
+
+def _vault(tmp_path: Path) -> Path:
+    (tmp_path / "ok.md").write_text("visible note", encoding="utf-8")
+    (tmp_path / ".draft.md").write_text("hidden file in root", encoding="utf-8")
+    (tmp_path / "sub").mkdir()
+    (tmp_path / "sub" / ".trash.md").write_text("hidden file in visible dir", encoding="utf-8")
+    (tmp_path / ".drafts").mkdir()
+    (tmp_path / ".drafts" / "inside.md").write_text("visible name, hidden dir", encoding="utf-8")
+    return tmp_path
+
+
+def _names(root: Path) -> set[str]:
+    # as_posix(): the assertions spell paths with "/". str() gives a backslash on
+    # Windows, so "sub/.trash.md" would never match and the skip test would pass
+    # for the wrong reason.
+    return {p.relative_to(root).as_posix() for p in iter_notes(root)}
+
+
+def test_dot_file_in_root_is_skipped(tmp_path, monkeypatch):
+    monkeypatch.delenv("BRAIN_INDEX_HIDDEN", raising=False)
+    assert ".draft.md" not in _names(_vault(tmp_path))
+
+
+def test_dot_file_in_visible_dir_is_skipped(tmp_path, monkeypatch):
+    monkeypatch.delenv("BRAIN_INDEX_HIDDEN", raising=False)
+    assert "sub/.trash.md" not in _names(_vault(tmp_path))
+
+
+def test_dot_dir_still_skipped_and_visible_note_still_kept(tmp_path, monkeypatch):
+    """The pre-existing behaviour must survive the fix."""
+    monkeypatch.delenv("BRAIN_INDEX_HIDDEN", raising=False)
+    names = _names(_vault(tmp_path))
+    assert ".drafts/inside.md" not in names
+    assert "ok.md" in names
+
+
+def test_hidden_flag_admits_dot_files_and_dot_dirs(tmp_path, monkeypatch):
+    """Opt-in must cover both kinds of hidden path, not just directories."""
+    monkeypatch.setenv("BRAIN_INDEX_HIDDEN", "1")
+    names = _names(_vault(tmp_path))
+    assert {"ok.md", ".draft.md", "sub/.trash.md", ".drafts/inside.md"} <= names
+
+
+def test_vault_root_that_is_itself_a_dot_directory_is_indexed(tmp_path, monkeypatch):
+    """Only path components BELOW the vault root decide what is hidden.
+
+    A vault kept at ``~/.vault`` or ``~/.notes`` is ordinary. A filter that checks
+    the absolute path (``p.parts``) instead of the relative one (``rel.parts``)
+    sees ``.vault`` in every path and silently returns nothing.
+    """
+    monkeypatch.delenv("BRAIN_INDEX_HIDDEN", raising=False)
+    root = tmp_path / ".vault"
+    root.mkdir()
+    names = _names(_vault(root))
+    assert "ok.md" in names
+    assert ".draft.md" not in names and ".drafts/inside.md" not in names

sqlite_graph_memory-0.3.3/tests/test_mcp_script_mode.py

@@ -0,0 +1,59 @@
+# -*- coding: utf-8 -*-
+"""Script-mode and working-directory behaviour of the MCP server.
+
+Kept in its own file so it does not collide with edits to test_mcp_server.py.
+Runs offline: no model, no index, no network.
+"""
+import json
+import os
+import subprocess
+import sys
+from pathlib import Path
+
+SRC = Path(__file__).resolve().parents[1] / "src"
+sys.path.insert(0, str(SRC))
+from sqlite_graph_memory import __version__  # noqa: E402
+from sqlite_graph_memory import mcp_server  # noqa: E402
+
+SCRIPT = SRC / "sqlite_graph_memory" / "mcp_server.py"
+
+
+def test_script_mode_starts_without_installed_package(tmp_path):
+    """`python src/sqlite_graph_memory/mcp_server.py` must not need the package on sys.path.
+
+    Python puts the script's own directory (not src/) on sys.path, so an
+    absolute `from sqlite_graph_memory import ...` at import time crashed with
+    ModuleNotFoundError before the server answered anything (issue #24).
+    `-S` drops site-packages so an installed or editable copy cannot mask it.
+    """
+    env = {k: v for k, v in os.environ.items() if k != "PYTHONPATH"}
+    req = {"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {}}
+    res = subprocess.run(
+        [sys.executable, "-S", str(SCRIPT)],
+        input=json.dumps(req) + "\n",
+        capture_output=True, text=True, cwd=str(tmp_path), env=env, timeout=30,
+    )
+    assert res.returncode == 0, res.stderr
+    resp = json.loads(res.stdout.strip().splitlines()[-1])
+    assert resp["result"]["serverInfo"]["version"] == __version__
+
+
+def test_recall_subprocess_inherits_server_cwd(tmp_path, monkeypatch):
+    """Relative BRAIN_INDEX_DIR / TURNSTATE_DB must resolve against the user's cwd.
+
+    On an installed wheel the package dir is site-packages/sqlite_graph_memory;
+    running brain_ask there made `BRAIN_INDEX_DIR=index` miss the user's ./index.
+    """
+    work = tmp_path / "work"
+    work.mkdir()
+    stub = tmp_path / "stub_brain_ask.py"
+    stub.write_text(
+        "import os, pathlib\n"
+        "pathlib.Path(os.environ['BRAIN_ANSWER_OUT']).write_text(os.getcwd(), encoding='utf-8')\n",
+        encoding="utf-8",
+    )
+    monkeypatch.setattr(mcp_server, "_brain_ask_cmd", lambda: [sys.executable, str(stub)])
+    monkeypatch.chdir(work)
+    text, is_error = mcp_server.run_recall("anything")
+    assert is_error is False, text
+    assert Path(text).resolve() == work.resolve()