ghostjournal 0.1.1__tar.gz

ghostjournal-0.1.1/CHANGELOG.md

@@ -0,0 +1,25 @@
+# Changelog
+
+## 0.1.1
+
+Hardening companion for deeprem 0.1.0. Stop every 0.1.0 writer before upgrading;
+the lock protocol is not mixed-version compatible.
+
+- Reject path traversal, separators, glob syntax and unsafe IDs on append/get.
+- Enforce global ID uniqueness across date directories and validate indexed paths.
+- Replace age-stealing lockfiles with OS-held filelock locks; serialize reads,
+  reindex and append, and explicitly close SQLite connections.
+- Recover retry keys from original JSON after crash-before-index and rebuild a
+  missing index at open. Original payload still wins on a reused client_key.
+- Publish immutable JSON with no-clobber hard links and POSIX file/directory fsync.
+- Reject non-finite JSON and boolean/float schema versions; harden type errors.
+- Reject symlinks inside canonical storage and validate original file layout.
+- Require explicit NN enablement; normal model loading is CPU/cache-only with
+  remote-code trust disabled. Add explicit download-model setup command.
+- Emit bounded escaped JSON from prompt_context, retaining its historical-data
+  label. This is an output-format change, not a prompt-injection guarantee.
+- Add regression, crash-window, multiprocess and concurrent reindex tests.
+
+Valid original entries and old manifest bytes are not migrated or rewritten.
+This release does not itself sign or encrypt a journal. Real NN model loading
+and a non-Linux support matrix still require maintainer testing.

ghostjournal-0.1.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Shelleyguitar
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ghostjournal-0.1.1/MANIFEST.in

@@ -0,0 +1,7 @@
+include README.md
+include LICENSE
+recursive-include ghostjournal/schemas *.json
+recursive-include examples *.py
+
+include CHANGELOG.md PUBLISHING.md
+recursive-include tests *.py

ghostjournal-0.1.1/PKG-INFO

@@ -0,0 +1,209 @@
+Metadata-Version: 2.4
+Name: ghostjournal
+Version: 0.1.1
+Summary: A local-first, append-only reflective journal substrate for LLM agents.
+Author: Shelleyguitar
+License-Expression: MIT
+Project-URL: Homepage, https://pypi.org/project/ghostjournal/
+Keywords: agents,journal,memory,embeddings,local-first,llm
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: filelock<4,>=3.16
+Provides-Extra: nn
+Requires-Dist: numpy>=1.24; extra == "nn"
+Requires-Dist: sentence-transformers>=3.0; extra == "nn"
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: build>=1.2; extra == "dev"
+Dynamic: license-file
+
+# ghostjournal
+
+**0.1.1 hardening release.** Stop all 0.1.0 writers before upgrading. The lock
+protocol changed from age-based lockfiles to OS-held advisory locks. Do not mix
+old and new writers on one root. Existing valid entry JSON and existing manifest
+bytes are preserved; new roots receive a journal UUID. Read CHANGELOG.md.
+
+`ghostjournal` is boring, local-first infrastructure for agents that need a durable reflective journal rather than a chat-log dump.
+
+Each journal entry is immutable JSON. SQLite, FTS, and optional embedding vectors are **derived state**: delete `index/`, run `ghostjournal reindex`, and the searchable journal is rebuilt from `entries/` without rewriting history.
+
+## Install
+
+```bash
+pip install -e .
+```
+
+For local sentence embeddings:
+
+```bash
+pip install -e '.[nn]'
+```
+
+The NN extra uses `sentence-transformers` with
+`sentence-transformers/all-MiniLM-L6-v2` by default. The base runtime dependency is
+`filelock`; SQLite/FTS5 supplies lexical search. NN is now explicitly opt-in even
+when the extra is installed. Ordinary encoder operations use CPU and
+`local_files_only=True`, with `trust_remote_code=False`. Download the selected
+model only with this explicit setup command:
+
+```sh
+ghostjournal --root ./journal download-model
+```
+
+That command requires the NN extra and network access. Later `--nn` operations
+use the cached model and do not intentionally fetch model files. In strictly
+offline deployments also set `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1` and
+enforce network policy at the OS boundary. No GPU or API key is required for the
+default public model. Actual model download/inference was not exercised in this
+hardening build; the lexical path and adapter configuration were tested.
+
+## API
+
+```python
+from ghostjournal import Journal
+
+journal = Journal("./journal", enable_nn=False)
+
+entry = journal.append({
+    "kind": "pulse",
+    "agent": "motoko",
+    "voice": "I keep using practical light to explain where work happens.",
+    "theme": "light as labor",
+    "tags": ["light", "places"],
+    "signals": {"focus": "light", "novelty": 0.5, "continuity_refs": []},
+    "client_key": "motoko:pulse:2026-09-05T16",
+    "meta": {"source_routine": "pulse-4h"},
+})
+
+print(journal.get(entry.id).to_dict())
+print(journal.list(kind="pulse", limit=20))
+print(journal.search("what did I care about in lighting?", k=8))
+print(journal.relate(entry.id, k=8))
+print(journal.digest())
+print(journal.prompt_context("what production habits keep returning?", k=6))
+```
+
+`append()` supplies `id`, timezone-aware UTC `ts`, top-level `schema_version`, `tags`, and `meta.schema_version` when omitted. All supplied fields are validated strictly. Unknown top-level fields are rejected.
+
+`client_key` is optional. When reused, `append()` returns the previously stored
+entry, allowing cron retries without duplicate pulses. For compatibility, the
+original payload wins even if a retry supplies different prose. This differs
+from deeprem's strict retry-key conflict rule. All generated IDs are UUIDs.
+Invalid/non-finite JSON values, invalid timestamps, and unsafe IDs are rejected.
+
+## CLI
+
+```bash
+ghostjournal --root ./journal init
+
+echo '{
+  "kind": "pulse",
+  "agent": "motoko",
+  "voice": "The crane silhouette is becoming a landmark.",
+  "theme": "recurring landmarks",
+  "tags": ["places", "continuity"],
+  "meta": {"source_routine": "pulse-4h"}
+}' | ghostjournal --root ./journal append
+
+ghostjournal --root ./journal list --kind pulse --limit 20
+ghostjournal --root ./journal search "recurring places" -k 8
+ghostjournal --root ./journal relate ENTRY_ID -k 8
+ghostjournal --root ./journal digest
+ghostjournal --root ./journal prompt-context "what did I care about last week?" -k 6
+ghostjournal --root ./journal reindex
+```
+
+Add `--nn` before the subcommand to enable semantic embeddings when `ghostjournal[nn]` is installed:
+
+```bash
+ghostjournal --root ./journal --nn search "what identity am I developing?"
+```
+
+## On-disk layout
+
+```text
+journal/
+  manifest.json
+  entries/
+    YYYY/MM/DD/<uuid>.json
+  index/
+    journal.sqlite3
+  models/
+  .write.lock
+```
+
+The JSON entry files are canonical. SQLite contains metadata, FTS content, idempotency keys, and—when enabled—float32 embedding blobs. Keeping vectors in SQLite avoids an additional vector database and makes the derived index transactional and simple to rebuild. `models/` is reserved for encoder/cache integrations; model caching itself follows the sentence-transformers/Hugging Face cache configuration.
+
+Reads, writes, and reindex are serialized with an OS-held advisory lock. No live
+lock is stolen based on age. Entry files are flushed/fsynced and published with
+an atomic no-clobber hard link; newly created parent directories are synced on
+POSIX. SQLite connections are explicitly closed. The journal's canonical files,
+not a stale index, determine ID uniqueness and retry keys. An exact retry after
+a crash between file publication and indexing repairs the index. Digests remain
+computed views unless the caller explicitly appends a new `kind="digest"` entry.
+
+Use a local filesystem supporting OS locks, hard links and atomic replacement.
+Network filesystems and hostile same-account processes are not a supported
+security boundary. This package is not encrypted or cryptographically signed;
+use deeprem for evidence seals and review decisions, and encrypted storage for
+source-journal confidentiality. Symlink entry paths and path/glob syntax in IDs
+are rejected. `prompt_context()` now emits bounded, escaped historical JSON data,
+not a privileged instruction block. It is not a complete prompt-injection defense.
+
+## Entry schema v1
+
+The packaged JSON Schema lives at `ghostjournal/schemas/entry-v1.schema.json`.
+
+Core fields:
+
+- `schema_version`: `1`
+- `id`: UUID by default; custom IDs must match `[A-Za-z0-9][A-Za-z0-9_-]{0,127}`
+- `ts`: timezone-aware ISO8601 timestamp
+- `kind`: `pulse | evening | note | digest`
+- `agent`: agent identity
+- `voice`: reflective prose
+- `theme`: short theme label
+- `tags`: unique strings
+- `mood`: optional short string
+- `anchors`: optional `{type, ref, label?}` external references; ghostjournal never fetches them
+- `signals`: optional `focus`, `novelty`, and `continuity_refs`
+- `embedding`: optional numeric vector; normally embeddings are stored separately in the index
+- `client_key`: optional idempotency key
+- `meta`: extensible metadata object with required `schema_version` and optional `source_routine`
+
+## Why structured JSON helps a “ghost” develop
+
+Raw markdown preserves prose but forces every later agent run to rediscover what the prose means. A ghostjournal entry preserves both levels at once: `voice` keeps the subjective record, while stable machine fields expose theme, focus, novelty, provenance, tags, and explicit continuity links.
+
+That gives retrieval more than a transcript. An agent can ask for semantically similar past thoughts, restrict by entry kind or time, aggregate recurring themes, and carry compact “past-you” context into its next reflective prompt. The library does not claim to create identity and does not call an LLM; it makes the agent’s self-observations durable, addressable, and comparable over time.
+
+## Recovery contract
+
+The `entries/` tree is the source of truth. With writers stopped, remove the entire `index/` directory and run:
+
+```bash
+ghostjournal --root ./journal reindex
+```
+
+All searchable metadata and optional vectors are recreated solely from immutable entry JSON. Historical JSON files are not modified. Opening an existing journal with a missing index also rebuilds it automatically. NN reindex requires the selected encoder to already be cached.
+
+## Development
+
+```bash
+python -m pytest
+python examples/motoko_sim.py
+python -m build
+```
+
+## License
+
+MIT

ghostjournal-0.1.1/PUBLISHING.md

@@ -0,0 +1,18 @@
+# Publishing ghostjournal 0.1.1
+
+This is the patched dependency for deeprem 0.1.0. Publish it first. No upload has
+been performed. Verify project ownership, name, licensing and metadata before
+public release. Review the accompanying deeprem PUBLISHING.md and release-kit
+verification report for exact checks and remaining release gates.
+
+```sh
+python -m pip install -e '.[dev]'
+python -m pytest
+python -m build
+python -m twine check --strict dist/*
+```
+
+Prefer owner-controlled PyPI Trusted Publishing. Expected files are
+`ghostjournal-0.1.1-py3-none-any.whl` and `ghostjournal-0.1.1.tar.gz`. Do not upload
+the old 0.1.0 artifacts as the hardened version. Stop old writers before any
+runtime upgrade, since 0.1.0 and 0.1.1 locking protocols differ.

ghostjournal-0.1.1/README.md

@@ -0,0 +1,182 @@
+# ghostjournal
+
+**0.1.1 hardening release.** Stop all 0.1.0 writers before upgrading. The lock
+protocol changed from age-based lockfiles to OS-held advisory locks. Do not mix
+old and new writers on one root. Existing valid entry JSON and existing manifest
+bytes are preserved; new roots receive a journal UUID. Read CHANGELOG.md.
+
+`ghostjournal` is boring, local-first infrastructure for agents that need a durable reflective journal rather than a chat-log dump.
+
+Each journal entry is immutable JSON. SQLite, FTS, and optional embedding vectors are **derived state**: delete `index/`, run `ghostjournal reindex`, and the searchable journal is rebuilt from `entries/` without rewriting history.
+
+## Install
+
+```bash
+pip install -e .
+```
+
+For local sentence embeddings:
+
+```bash
+pip install -e '.[nn]'
+```
+
+The NN extra uses `sentence-transformers` with
+`sentence-transformers/all-MiniLM-L6-v2` by default. The base runtime dependency is
+`filelock`; SQLite/FTS5 supplies lexical search. NN is now explicitly opt-in even
+when the extra is installed. Ordinary encoder operations use CPU and
+`local_files_only=True`, with `trust_remote_code=False`. Download the selected
+model only with this explicit setup command:
+
+```sh
+ghostjournal --root ./journal download-model
+```
+
+That command requires the NN extra and network access. Later `--nn` operations
+use the cached model and do not intentionally fetch model files. In strictly
+offline deployments also set `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1` and
+enforce network policy at the OS boundary. No GPU or API key is required for the
+default public model. Actual model download/inference was not exercised in this
+hardening build; the lexical path and adapter configuration were tested.
+
+## API
+
+```python
+from ghostjournal import Journal
+
+journal = Journal("./journal", enable_nn=False)
+
+entry = journal.append({
+    "kind": "pulse",
+    "agent": "motoko",
+    "voice": "I keep using practical light to explain where work happens.",
+    "theme": "light as labor",
+    "tags": ["light", "places"],
+    "signals": {"focus": "light", "novelty": 0.5, "continuity_refs": []},
+    "client_key": "motoko:pulse:2026-09-05T16",
+    "meta": {"source_routine": "pulse-4h"},
+})
+
+print(journal.get(entry.id).to_dict())
+print(journal.list(kind="pulse", limit=20))
+print(journal.search("what did I care about in lighting?", k=8))
+print(journal.relate(entry.id, k=8))
+print(journal.digest())
+print(journal.prompt_context("what production habits keep returning?", k=6))
+```
+
+`append()` supplies `id`, timezone-aware UTC `ts`, top-level `schema_version`, `tags`, and `meta.schema_version` when omitted. All supplied fields are validated strictly. Unknown top-level fields are rejected.
+
+`client_key` is optional. When reused, `append()` returns the previously stored
+entry, allowing cron retries without duplicate pulses. For compatibility, the
+original payload wins even if a retry supplies different prose. This differs
+from deeprem's strict retry-key conflict rule. All generated IDs are UUIDs.
+Invalid/non-finite JSON values, invalid timestamps, and unsafe IDs are rejected.
+
+## CLI
+
+```bash
+ghostjournal --root ./journal init
+
+echo '{
+  "kind": "pulse",
+  "agent": "motoko",
+  "voice": "The crane silhouette is becoming a landmark.",
+  "theme": "recurring landmarks",
+  "tags": ["places", "continuity"],
+  "meta": {"source_routine": "pulse-4h"}
+}' | ghostjournal --root ./journal append
+
+ghostjournal --root ./journal list --kind pulse --limit 20
+ghostjournal --root ./journal search "recurring places" -k 8
+ghostjournal --root ./journal relate ENTRY_ID -k 8
+ghostjournal --root ./journal digest
+ghostjournal --root ./journal prompt-context "what did I care about last week?" -k 6
+ghostjournal --root ./journal reindex
+```
+
+Add `--nn` before the subcommand to enable semantic embeddings when `ghostjournal[nn]` is installed:
+
+```bash
+ghostjournal --root ./journal --nn search "what identity am I developing?"
+```
+
+## On-disk layout
+
+```text
+journal/
+  manifest.json
+  entries/
+    YYYY/MM/DD/<uuid>.json
+  index/
+    journal.sqlite3
+  models/
+  .write.lock
+```
+
+The JSON entry files are canonical. SQLite contains metadata, FTS content, idempotency keys, and—when enabled—float32 embedding blobs. Keeping vectors in SQLite avoids an additional vector database and makes the derived index transactional and simple to rebuild. `models/` is reserved for encoder/cache integrations; model caching itself follows the sentence-transformers/Hugging Face cache configuration.
+
+Reads, writes, and reindex are serialized with an OS-held advisory lock. No live
+lock is stolen based on age. Entry files are flushed/fsynced and published with
+an atomic no-clobber hard link; newly created parent directories are synced on
+POSIX. SQLite connections are explicitly closed. The journal's canonical files,
+not a stale index, determine ID uniqueness and retry keys. An exact retry after
+a crash between file publication and indexing repairs the index. Digests remain
+computed views unless the caller explicitly appends a new `kind="digest"` entry.
+
+Use a local filesystem supporting OS locks, hard links and atomic replacement.
+Network filesystems and hostile same-account processes are not a supported
+security boundary. This package is not encrypted or cryptographically signed;
+use deeprem for evidence seals and review decisions, and encrypted storage for
+source-journal confidentiality. Symlink entry paths and path/glob syntax in IDs
+are rejected. `prompt_context()` now emits bounded, escaped historical JSON data,
+not a privileged instruction block. It is not a complete prompt-injection defense.
+
+## Entry schema v1
+
+The packaged JSON Schema lives at `ghostjournal/schemas/entry-v1.schema.json`.
+
+Core fields:
+
+- `schema_version`: `1`
+- `id`: UUID by default; custom IDs must match `[A-Za-z0-9][A-Za-z0-9_-]{0,127}`
+- `ts`: timezone-aware ISO8601 timestamp
+- `kind`: `pulse | evening | note | digest`
+- `agent`: agent identity
+- `voice`: reflective prose
+- `theme`: short theme label
+- `tags`: unique strings
+- `mood`: optional short string
+- `anchors`: optional `{type, ref, label?}` external references; ghostjournal never fetches them
+- `signals`: optional `focus`, `novelty`, and `continuity_refs`
+- `embedding`: optional numeric vector; normally embeddings are stored separately in the index
+- `client_key`: optional idempotency key
+- `meta`: extensible metadata object with required `schema_version` and optional `source_routine`
+
+## Why structured JSON helps a “ghost” develop
+
+Raw markdown preserves prose but forces every later agent run to rediscover what the prose means. A ghostjournal entry preserves both levels at once: `voice` keeps the subjective record, while stable machine fields expose theme, focus, novelty, provenance, tags, and explicit continuity links.
+
+That gives retrieval more than a transcript. An agent can ask for semantically similar past thoughts, restrict by entry kind or time, aggregate recurring themes, and carry compact “past-you” context into its next reflective prompt. The library does not claim to create identity and does not call an LLM; it makes the agent’s self-observations durable, addressable, and comparable over time.
+
+## Recovery contract
+
+The `entries/` tree is the source of truth. With writers stopped, remove the entire `index/` directory and run:
+
+```bash
+ghostjournal --root ./journal reindex
+```
+
+All searchable metadata and optional vectors are recreated solely from immutable entry JSON. Historical JSON files are not modified. Opening an existing journal with a missing index also rebuilds it automatically. NN reindex requires the selected encoder to already be cached.
+
+## Development
+
+```bash
+python -m pytest
+python examples/motoko_sim.py
+python -m build
+```
+
+## License
+
+MIT

ghostjournal-0.1.1/examples/motoko_sim.py

@@ -0,0 +1,75 @@
+from __future__ import annotations
+
+from pathlib import Path
+from tempfile import TemporaryDirectory
+
+from ghostjournal import Journal
+
+
+def main() -> None:
+    with TemporaryDirectory(prefix="motoko-journal-") as tmp:
+        journal = Journal(Path(tmp), enable_nn=False)
+
+        pulses = [
+            {
+                "kind": "pulse",
+                "agent": "motoko",
+                "voice": "I keep favoring pools of practical light over decorative glow. The dock feels more believable when light explains the work.",
+                "theme": "light as labor",
+                "tags": ["light", "places"],
+                "mood": "curious",
+                "signals": {"focus": "light", "novelty": 0.55, "continuity_refs": []},
+                "client_key": "motoko:pulse:1",
+                "meta": {"source_routine": "pulse-4h"},
+            },
+            {
+                "kind": "pulse",
+                "agent": "motoko",
+                "voice": "Mechanical detail reads better when I imply repeated maintenance instead of drawing every bolt.",
+                "theme": "economical machinery",
+                "tags": ["process", "effects"],
+                "mood": "focused",
+                "signals": {"focus": "process", "novelty": 0.35, "continuity_refs": []},
+                "client_key": "motoko:pulse:2",
+                "meta": {"source_routine": "pulse-4h"},
+            },
+            {
+                "kind": "pulse",
+                "agent": "motoko",
+                "voice": "The same crane silhouette keeps returning. It may be turning into a landmark rather than a prop.",
+                "theme": "recurring landmarks",
+                "tags": ["places", "continuity"],
+                "mood": "attentive",
+                "signals": {"focus": "places", "novelty": 0.7, "continuity_refs": []},
+                "client_key": "motoko:pulse:3",
+                "meta": {"source_routine": "pulse-4h"},
+            },
+        ]
+        saved = [journal.append(item) for item in pulses]
+
+        journal.append({
+            "kind": "evening",
+            "agent": "motoko",
+            "voice": "Today I trusted recurring shapes more than novelty. Light, maintenance, and the crane all became ways of showing that a place has habits. I want tomorrow's work to keep that restraint.",
+            "theme": "places have habits",
+            "tags": ["continuity", "light", "process"],
+            "mood": "settled",
+            "signals": {
+                "focus": "self",
+                "novelty": 0.45,
+                "continuity_refs": [entry.id for entry in saved],
+            },
+            "client_key": "motoko:evening:1",
+            "meta": {"source_routine": "evening-journal"},
+        })
+
+        print("SEARCH")
+        for hit in journal.search("what matters about light and recurring places?", k=3):
+            print(f"{hit.score:.3f}  {hit.entry_id}  {hit.snippet}")
+
+        print("\nPROMPT CONTEXT")
+        print(journal.prompt_context("what production identity am I developing?", k=4))
+
+
+if __name__ == "__main__":
+    main()

ghostjournal-0.1.1/ghostjournal/__init__.py

@@ -0,0 +1,6 @@
+from .exceptions import EntryNotFound, GhostJournalError, ValidationError
+from .journal import Journal
+from .models import Digest, Entry, Hit
+
+__all__ = ["Journal", "Entry", "Hit", "Digest", "GhostJournalError", "ValidationError", "EntryNotFound"]
+__version__ = "0.1.1"

ghostjournal-0.1.1/ghostjournal/__main__.py

@@ -0,0 +1,3 @@
+from .cli import main
+
+raise SystemExit(main())

ghostjournal-0.1.1/ghostjournal/cli.py

@@ -0,0 +1,102 @@
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from pathlib import Path
+from typing import Any
+
+from .journal import Journal
+from .embedding import DEFAULT_MODEL, download_model
+
+
+def _dump(value: Any) -> None:
+    if hasattr(value, "to_dict"):
+        value = value.to_dict()
+    print(json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True))
+
+
+def _entry_dict(entry):
+    return entry.to_dict()
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(prog="ghostjournal", description="Local-first journal substrate for agents")
+    parser.add_argument("--root", default="journal", help="journal root directory (default: ./journal)")
+    parser.add_argument("--nn", action="store_true", help="enable local sentence embeddings (requires ghostjournal[nn])")
+    parser.add_argument("--model", default=DEFAULT_MODEL, help="local sentence-transformer model name")
+    sub = parser.add_subparsers(dest="command", required=True)
+
+    sub.add_parser("init", help="create/open a journal root")
+    sub.add_parser("download-model", help="explicit network opt-in to cache the local encoder")
+
+    ap = sub.add_parser("append", help="append one JSON entry")
+    ap.add_argument("--json", dest="json_text", help="entry JSON; if omitted, read JSON from stdin")
+
+    gp = sub.add_parser("get", help="get an entry by id")
+    gp.add_argument("id")
+
+    lp = sub.add_parser("list", help="list entries")
+    lp.add_argument("--since")
+    lp.add_argument("--until")
+    lp.add_argument("--kind", choices=["pulse", "evening", "note", "digest"])
+    lp.add_argument("--limit", type=int, default=100)
+
+    sp = sub.add_parser("search", help="hybrid lexical/semantic search")
+    sp.add_argument("query")
+    sp.add_argument("-k", type=int, default=8)
+
+    rp = sub.add_parser("relate", help="find related entries")
+    rp.add_argument("id")
+    rp.add_argument("-k", type=int, default=8)
+
+    dp = sub.add_parser("digest", help="roll up themes/tags/mood/focus")
+    dp.add_argument("--since")
+    dp.add_argument("--until")
+
+    pc = sub.add_parser("prompt-context", help="render compact past-self context")
+    pc.add_argument("query")
+    pc.add_argument("-k", type=int, default=6)
+    pc.add_argument("--max-chars", type=int, default=6000)
+
+    sub.add_parser("reindex", help="rebuild all derived indexes from immutable entry JSON")
+    return parser
+
+
+def main(argv: list[str] | None = None) -> int:
+    args = build_parser().parse_args(argv)
+    journal = Journal(Path(args.root), enable_nn=args.nn, model_name=args.model)
+
+    if args.command == "download-model":
+        download_model(args.model, cache_folder=str(journal.models_dir))
+        _dump({"model": args.model, "cache": str(journal.models_dir)})
+    elif args.command == "init":
+        _dump({"root": str(journal.root), "manifest": str(journal.manifest_path)})
+    elif args.command == "append":
+        raw = args.json_text if args.json_text is not None else sys.stdin.read()
+        _dump(journal.append(json.loads(raw)))
+    elif args.command == "get":
+        _dump(journal.get(args.id))
+    elif args.command == "list":
+        _dump([_entry_dict(e) for e in journal.list(since=args.since, until=args.until, kind=args.kind, limit=args.limit)])
+    elif args.command == "search":
+        _dump([
+            {"entry_id": h.entry_id, "score": h.score, "lexical_score": h.lexical_score, "semantic_score": h.semantic_score, "snippet": h.snippet}
+            for h in journal.search(args.query, k=args.k)
+        ])
+    elif args.command == "relate":
+        _dump([
+            {"entry_id": h.entry_id, "score": h.score, "snippet": h.snippet}
+            for h in journal.relate(args.id, k=args.k)
+        ])
+    elif args.command == "digest":
+        _dump(journal.digest(since=args.since, until=args.until))
+    elif args.command == "prompt-context":
+        print(journal.prompt_context(args.query, k=args.k, max_chars=args.max_chars))
+    elif args.command == "reindex":
+        _dump({"indexed": journal.reindex()})
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())