@simbimbo/memory-ocmemog 0.1.6 → 0.1.8

package/CHANGELOG.md

@@ -1,5 +1,24 @@
 # Changelog
 
+## 0.1.8 — 2026-03-19
+
+Documentation and release follow-through after the llama.cpp migration and repo grooming pass.
+
+### Highlights
+- documented the stable local runtime architecture (gateway/sidecar/text/embed split)
+- published the repo in a llama.cpp-first state with fixed ports and cleaned installers/scripts
+- kept compatibility hooks only where still useful instead of leaving Ollama as the implied primary path
+
+## 0.1.7 — 2026-03-19
+
+llama.cpp-first cleanup after the 0.1.6 runtime cutover.
+
+### Highlights
+- made llama.cpp / local OpenAI-compatible endpoints the primary documented and scripted local runtime path
+- reduced misleading Ollama-first defaults in installers, sidecar scripts, docs, and helper tooling
+- aligned context/distill/runtime helpers with the fixed local model architecture (`17890` gateway, `17891` sidecar, `18080` text, `18081` embeddings)
+- kept compatibility hooks only where still useful for rollback or mixed environments
+
 ## 0.1.6 — 2026-03-19
 
 Port-separation and publish-solid follow-up.
@@ -8,6 +27,7 @@ Port-separation and publish-solid follow-up.
 - Split ocmemog sidecar onto dedicated loopback port `17891` to avoid collision with the OpenClaw gateway/dashboard on `17890`
 - Restored the plain realtime dashboard on `/dashboard` and fixed the `local_html` template crash
 - Updated plugin/runtime defaults, scripts, and documentation to use the dedicated sidecar endpoint on `17891`
+- Switched repo-facing local-runtime defaults to llama.cpp-first endpoints on `18080`/`18081` with Qwen2.5 text and `nomic-embed-text-v1.5` embeddings, while keeping Ollama as explicit legacy fallback only
 - Added governance retrieval/governance-policy hardening plus expanded regression coverage for duplicate, contradiction, supersession, queue, audit, rollback, and auto-resolve flows
 - Aligned package/version metadata across npm, Python, and FastAPI surfaces
 
@@ -16,7 +36,7 @@ Port-separation and publish-solid follow-up.
 Repair and hardening follow-up after the 0.1.4 publish.
 
 ### Highlights
-- Fixed vector reindex defaults so repair scripts use provider-backed Ollama embeddings instead of silently rebuilding weak local/hash vectors
+- Fixed vector reindex defaults so repair scripts use provider-backed local embeddings instead of silently rebuilding weak local/hash vectors
 - Added battery-aware sidecar defaults for macOS laptops (`OCMEMOG_LAPTOP_MODE=auto|ac|battery`)
 - Fixed `record_reinforcement()` so new experiences preserve `memory_reference`, and added integrity repair to backfill legacy missing references
 - Added incremental vector backfill tooling (`scripts/ocmemog-backfill-vectors.py`) for non-destructive backlog repair

package/README.md

@@ -14,6 +14,9 @@ Architecture at a glance:
 - **FastAPI sidecar (`ocmemog/sidecar/`)** exposes memory and continuity APIs
 - **SQLite-backed runtime (`brain/runtime/memory/`)** powers storage, hydration, checkpoints, salience ranking, and pondering
 
+Current local runtime architecture note:
+- `docs/architecture/local-runtime-2026-03-19.md`
+
 ## Repo layout
 
 - `openclaw.plugin.json`, `index.ts`, `package.json`: OpenClaw plugin package and manifest.
@@ -78,20 +81,24 @@ Optional environment variables:
 - `OCMEMOG_OPENAI_API_BASE` (default: `https://api.openai.com/v1`)
 - `OCMEMOG_OPENAI_EMBED_MODEL` (default: `text-embedding-3-small`)
 - `BRAIN_EMBED_MODEL_LOCAL` (`simple` by default)
-- `BRAIN_EMBED_MODEL_PROVIDER` (`openai` to enable provider embeddings)
+- `BRAIN_EMBED_MODEL_PROVIDER` (`local-openai` to use the local llama.cpp embedding endpoint; `openai` remains available for hosted embeddings)
 - `OCMEMOG_TRANSCRIPT_WATCHER` (`true` to auto-start transcript watcher inside the sidecar)
 - `OCMEMOG_TRANSCRIPT_ROOTS` (comma-separated allowed roots for transcript context retrieval; default: `~/.openclaw/workspace/memory`)
 - `OCMEMOG_API_TOKEN` (optional; if set, requests must include `x-ocmemog-token` or `Authorization: Bearer ...`)
 - `OCMEMOG_AUTO_HYDRATION` (`true` to re-enable prompt-time continuity prepending; defaults to `false` as a safety guard until the host runtime is verified not to persist prepended context into session history)
 - `OCMEMOG_LAPTOP_MODE` (`auto` by default; on macOS battery power this slows watcher polling, reduces ingest batch size, and disables sentiment reinforcement unless explicitly overridden)
-- `OCMEMOG_USE_OLLAMA` (`true` to use Ollama for distill/inference)
-- `OCMEMOG_OLLAMA_HOST` (default: `http://127.0.0.1:11434`)
-- `OCMEMOG_OLLAMA_MODEL` (default: `phi3:latest`; lightweight local fallback / cheap cognition)
-- `OCMEMOG_OLLAMA_EMBED_MODEL` (default: `nomic-embed-text:latest`)
+- `OCMEMOG_LOCAL_LLM_BASE_URL` (default: `http://127.0.0.1:18080/v1`; local OpenAI-compatible text endpoint, e.g. llama.cpp)
+- `OCMEMOG_LOCAL_LLM_MODEL` (default: `qwen2.5-7b-instruct`; matches the active Qwen2.5-7B-Instruct GGUF runtime)
+- `OCMEMOG_LOCAL_EMBED_BASE_URL` (default: `http://127.0.0.1:18081/v1`; local OpenAI-compatible embedding endpoint)
+- `OCMEMOG_LOCAL_EMBED_MODEL` (default: `nomic-embed-text-v1.5`)
+- `OCMEMOG_USE_OLLAMA` (`true` to force legacy Ollama local inference path)
+- `OCMEMOG_OLLAMA_HOST` (default: `http://127.0.0.1:11434`; legacy fallback)
+- `OCMEMOG_OLLAMA_MODEL` (default: `qwen2.5:7b`; legacy fallback for machines that still use Ollama)
+- `OCMEMOG_OLLAMA_EMBED_MODEL` (default: `nomic-embed-text:latest`; legacy embedding fallback)
 - `OCMEMOG_PROMOTION_THRESHOLD` (default: `0.5`)
 - `OCMEMOG_DEMOTION_THRESHOLD` (default: `0.2`)
 - `OCMEMOG_PONDER_ENABLED` (default: `true`)
-- `OCMEMOG_PONDER_MODEL` (default via launcher: `qwen2.5:7b`; recommended for structured local memory refinement)
+- `OCMEMOG_PONDER_MODEL` (default via launcher: `local-openai:qwen2.5-7b-instruct`; recommended for structured local memory refinement)
 - `OCMEMOG_LESSON_MINING_ENABLED` (default: `true`)
 
 ## Security
@@ -129,12 +136,13 @@ This installer will try to:
 - install Python requirements
 - install/enable the OpenClaw plugin when the `openclaw` CLI is available
 - install/load LaunchAgents via `scripts/ocmemog-install.sh`
-- pull required local Ollama models when Ollama is already installed
+- verify the local llama.cpp runtime and expected text/embed endpoints
 - validate `/healthz`
 
 Notes:
-- If `OCMEMOG_INSTALL_PREREQS=true` and Homebrew is present, the installer will try to install missing `ollama` and `ffmpeg` automatically.
-- If Ollama is not installed and prereq auto-install is off or unavailable, the installer warns and continues; local model support will remain unavailable until Ollama is installed.
+- If `OCMEMOG_INSTALL_PREREQS=true` and Homebrew is present, the installer will try to install missing `llama.cpp` and `ffmpeg` automatically.
+- The installer no longer pulls local models. It assumes your llama.cpp text endpoint is on `127.0.0.1:18080` and your embedding endpoint is on `127.0.0.1:18081`.
+- Legacy Ollama compatibility remains available only when you explicitly opt into it with `OCMEMOG_USE_OLLAMA=true`.
 - If package install is unavailable in the local OpenClaw build, the installer falls back to local-path plugin install.
 - Advanced flags are available for local debugging/CI (`--skip-plugin-install`, `--skip-launchagents`, `--skip-model-pulls`, `--endpoint`, `--repo-url`).
 
@@ -154,7 +162,7 @@ launchctl bootstrap gui/$UID scripts/launchagents/com.openclaw.ocmemog.guard.pli
 
 ## Recent changes
 
-### 0.1.5 (current main)
+### 0.1.6 (current main)
 
 Package ownership + runtime safety release:
 - Publish package under `@simbimbo/memory-ocmemog` instead of the unauthorized `@openclaw` scope

package/brain/runtime/config.py

@@ -9,8 +9,13 @@ OCMEMOG_MEMORY_MODEL = os.environ.get("OCMEMOG_MEMORY_MODEL", "gpt-4o-mini")
 OCMEMOG_OPENAI_API_BASE = os.environ.get("OCMEMOG_OPENAI_API_BASE", "https://api.openai.com/v1")
 OCMEMOG_OPENAI_EMBED_MODEL = os.environ.get("OCMEMOG_OPENAI_EMBED_MODEL", "text-embedding-3-small")
 
+OCMEMOG_LOCAL_LLM_BASE_URL = os.environ.get("OCMEMOG_LOCAL_LLM_BASE_URL", "http://127.0.0.1:18080/v1")
+OCMEMOG_LOCAL_LLM_MODEL = os.environ.get("OCMEMOG_LOCAL_LLM_MODEL", "qwen2.5-7b-instruct")
+OCMEMOG_LOCAL_EMBED_BASE_URL = os.environ.get("OCMEMOG_LOCAL_EMBED_BASE_URL", "http://127.0.0.1:18081/v1")
+OCMEMOG_LOCAL_EMBED_MODEL = os.environ.get("OCMEMOG_LOCAL_EMBED_MODEL", "nomic-embed-text-v1.5")
+
 OCMEMOG_OLLAMA_HOST = os.environ.get("OCMEMOG_OLLAMA_HOST", "http://127.0.0.1:11434")
-OCMEMOG_OLLAMA_MODEL = os.environ.get("OCMEMOG_OLLAMA_MODEL", "phi3:latest")
+OCMEMOG_OLLAMA_MODEL = os.environ.get("OCMEMOG_OLLAMA_MODEL", "qwen2.5:7b")
 OCMEMOG_OLLAMA_EMBED_MODEL = os.environ.get("OCMEMOG_OLLAMA_EMBED_MODEL", "nomic-embed-text:latest")
 
 OCMEMOG_PROMOTION_THRESHOLD = float(os.environ.get("OCMEMOG_PROMOTION_THRESHOLD", "0.5"))

package/brain/runtime/inference.py

@@ -11,6 +11,35 @@ from brain.runtime.instrumentation import emit_event
 LOGFILE = state_store.reports_dir() / "brain_memory.log.jsonl"
 
 
+def _infer_openai_compatible(prompt: str, *, base_url: str, model: str, api_key: str | None = None, provider_label: str = "openai-compatible") -> dict[str, str]:
+    url = f"{base_url.rstrip('/')}/chat/completions"
+    payload = {
+        "model": model,
+        "messages": [{"role": "user", "content": prompt}],
+        "temperature": 0.2,
+    }
+    data = json.dumps(payload).encode("utf-8")
+    req = urllib.request.Request(url, data=data, method="POST")
+    if api_key:
+        req.add_header("Authorization", f"Bearer {api_key}")
+    req.add_header("Content-Type", "application/json")
+
+    try:
+        with urllib.request.urlopen(req, timeout=30) as resp:
+            response = json.loads(resp.read().decode("utf-8"))
+    except Exception as exc:
+        emit_event(LOGFILE, "brain_infer_error", status="error", provider=provider_label, error=str(exc))
+        return {"status": "error", "error": f"request_failed:{exc}"}
+
+    try:
+        output = response["choices"][0]["message"]["content"]
+    except Exception as exc:
+        emit_event(LOGFILE, "brain_infer_error", status="error", provider=provider_label, error=str(exc))
+        return {"status": "error", "error": "invalid_response"}
+
+    return {"status": "ok", "output": str(output).strip()}
+
+
 def _infer_ollama(prompt: str, model: str | None = None) -> dict[str, str]:
     payload = {
         "model": model or config.OCMEMOG_OLLAMA_MODEL,
@@ -33,6 +62,21 @@ def _infer_ollama(prompt: str, model: str | None = None) -> dict[str, str]:
     return {"status": "ok", "output": str(output).strip()}
 
 
+def _looks_like_local_openai_model(name: str) -> bool:
+    if not name:
+        return False
+    lowered = name.strip().lower()
+    return lowered.startswith("local-openai:") or lowered.startswith("local_openai:") or lowered.startswith("llamacpp:")
+
+
+def _normalize_local_model_name(name: str) -> str:
+    lowered = (name or "").strip()
+    for prefix in ("local-openai:", "local_openai:", "llamacpp:"):
+        if lowered.lower().startswith(prefix):
+            return lowered[len(prefix):]
+    return lowered
+
+
 def _looks_like_ollama_model(name: str) -> bool:
     if not name:
         return False
@@ -69,41 +113,37 @@ def infer(prompt: str, provider_name: str | None = None) -> dict[str, str]:
 
     use_ollama = os.environ.get("OCMEMOG_USE_OLLAMA", "").lower() in {"1", "true", "yes"}
     model_override = provider_name or config.OCMEMOG_MEMORY_MODEL
+    if _looks_like_local_openai_model(model_override):
+        model = _normalize_local_model_name(model_override) or config.OCMEMOG_LOCAL_LLM_MODEL
+        return _infer_openai_compatible(
+            prompt,
+            base_url=config.OCMEMOG_LOCAL_LLM_BASE_URL,
+            model=model,
+            api_key=os.environ.get("OCMEMOG_LOCAL_LLM_API_KEY") or os.environ.get("LOCAL_LLM_API_KEY"),
+            provider_label="local-openai",
+        )
     if use_ollama or _looks_like_ollama_model(model_override):
         model = model_override.split(":", 1)[-1] if model_override.startswith("ollama:") else model_override
         return _infer_ollama(prompt, model)
 
     api_key = os.environ.get("OCMEMOG_OPENAI_API_KEY") or os.environ.get("OPENAI_API_KEY")
     if not api_key:
-        # fall back to local ollama if configured
-        return _infer_ollama(prompt, config.OCMEMOG_OLLAMA_MODEL)
+        return _infer_openai_compatible(
+            prompt,
+            base_url=config.OCMEMOG_LOCAL_LLM_BASE_URL,
+            model=config.OCMEMOG_LOCAL_LLM_MODEL,
+            api_key=os.environ.get("OCMEMOG_LOCAL_LLM_API_KEY") or os.environ.get("LOCAL_LLM_API_KEY"),
+            provider_label="local-openai",
+        )
 
     model = model_override
-    url = f"{config.OCMEMOG_OPENAI_API_BASE.rstrip('/')}/chat/completions"
-    payload = {
-        "model": model,
-        "messages": [{"role": "user", "content": prompt}],
-        "temperature": 0.2,
-    }
-    data = json.dumps(payload).encode("utf-8")
-    req = urllib.request.Request(url, data=data, method="POST")
-    req.add_header("Authorization", f"Bearer {api_key}")
-    req.add_header("Content-Type", "application/json")
-
-    try:
-        with urllib.request.urlopen(req, timeout=30) as resp:
-            response = json.loads(resp.read().decode("utf-8"))
-    except Exception as exc:
-        emit_event(LOGFILE, "brain_infer_error", status="error", provider="openai", error=str(exc))
-        return {"status": "error", "error": f"request_failed:{exc}"}
-
-    try:
-        output = response["choices"][0]["message"]["content"]
-    except Exception as exc:
-        emit_event(LOGFILE, "brain_infer_error", status="error", provider="openai", error=str(exc))
-        return {"status": "error", "error": "invalid_response"}
-
-    return {"status": "ok", "output": str(output).strip()}
+    return _infer_openai_compatible(
+        prompt,
+        base_url=config.OCMEMOG_OPENAI_API_BASE,
+        model=model,
+        api_key=api_key,
+        provider_label="openai",
+    )
 
 
 def parse_operator_name(text: str) -> dict[str, str] | None:

package/brain/runtime/memory/api.py

@@ -316,7 +316,10 @@ def _model_contradiction_hint(left: str, right: str) -> Optional[Dict[str, Any]]
         f"Statement A: {left}\n"
         f"Statement B: {right}\n"
     )
-    result = inference.infer(prompt, provider_name="qwen2.5:7b")
+    result = inference.infer(
+        prompt,
+        provider_name=os.environ.get("OCMEMOG_PONDER_MODEL", "local-openai:qwen2.5-7b-instruct"),
+    )
     if result.get("status") != "ok":
         return None
     try:

package/brain/runtime/memory/context_builder.py

@@ -53,7 +53,7 @@ def _groom_queries(prompt: str, limit: int = 3) -> List[str]:
         return []
     if _should_skip_query_grooming(cleaned):
         return _heuristic_queries(cleaned, limit=limit)
-    model = os.environ.get("OCMEMOG_PONDER_MODEL", "qwen2.5:7b")
+    model = os.environ.get("OCMEMOG_PONDER_MODEL", "local-openai:qwen2.5-7b-instruct")
     ask = (
         "Rewrite this raw memory request into up to 3 short search queries. "
         "Return strict JSON as {\"queries\":[\"...\"]}. "

package/brain/runtime/memory/distill.py

@@ -43,7 +43,7 @@ def _local_distill_summary(text: str) -> str:
         f"Experience:\n{text}\n\n"
         "Summary:"
     )
-    model = os.environ.get("OCMEMOG_PONDER_MODEL", "qwen2.5:7b")
+    model = os.environ.get("OCMEMOG_PONDER_MODEL", "local-openai:qwen2.5-7b-instruct")
     try:
         result = inference.infer(prompt, provider_name=model)
     except Exception:

package/brain/runtime/model_router.py

@@ -17,6 +17,8 @@ def get_provider_for_role(role: str) -> ModelSelection:
     provider = (config.BRAIN_EMBED_MODEL_PROVIDER or "").strip().lower()
     if provider in {"openai", "openai_compatible", "openai-compatible"}:
         return ModelSelection(provider_id="openai", model=config.OCMEMOG_OPENAI_EMBED_MODEL)
+    if provider in {"local-openai", "local_openai", "llamacpp", "llama.cpp"}:
+        return ModelSelection(provider_id="local-openai", model=config.OCMEMOG_LOCAL_EMBED_MODEL)
     if provider in {"ollama", "local-ollama"}:
         return ModelSelection(provider_id="ollama", model=config.OCMEMOG_OLLAMA_EMBED_MODEL)
     return ModelSelection()

package/brain/runtime/providers.py

@@ -14,25 +14,34 @@ class ProviderExecute:
     def execute_embedding_call(self, selection, text: str) -> dict[str, object]:
         provider_id = getattr(selection, "provider_id", "") or ""
         model = getattr(selection, "model", "") or config.OCMEMOG_OPENAI_EMBED_MODEL
-        if provider_id == "openai":
-            api_key = os.environ.get("OCMEMOG_OPENAI_API_KEY") or os.environ.get("OPENAI_API_KEY")
-            if not api_key:
-                return {}
-            url = f"{config.OCMEMOG_OPENAI_API_BASE.rstrip('/')}/embeddings"
+        if provider_id in {"openai", "local-openai"}:
+            api_key = None
+            url_base = config.OCMEMOG_OPENAI_API_BASE
+            provider_label = "openai"
+            if provider_id == "openai":
+                api_key = os.environ.get("OCMEMOG_OPENAI_API_KEY") or os.environ.get("OPENAI_API_KEY")
+                if not api_key:
+                    return {}
+            else:
+                url_base = config.OCMEMOG_LOCAL_EMBED_BASE_URL
+                api_key = os.environ.get("OCMEMOG_LOCAL_EMBED_API_KEY") or os.environ.get("LOCAL_EMBED_API_KEY")
+                provider_label = "local-openai"
+            url = f"{url_base.rstrip('/')}/embeddings"
             payload = json.dumps({"model": model, "input": text}).encode("utf-8")
             req = urllib.request.Request(url, data=payload, method="POST")
-            req.add_header("Authorization", f"Bearer {api_key}")
+            if api_key:
+                req.add_header("Authorization", f"Bearer {api_key}")
             req.add_header("Content-Type", "application/json")
             try:
                 with urllib.request.urlopen(req, timeout=20) as resp:
                     data = json.loads(resp.read().decode("utf-8"))
             except Exception as exc:
-                emit_event(LOGFILE, "brain_embedding_provider_error", status="error", provider="openai", error=str(exc))
+                emit_event(LOGFILE, "brain_embedding_provider_error", status="error", provider=provider_label, error=str(exc))
                 return {}
             try:
                 embedding = data["data"][0]["embedding"]
             except Exception as exc:
-                emit_event(LOGFILE, "brain_embedding_provider_error", status="error", provider="openai", error=str(exc))
+                emit_event(LOGFILE, "brain_embedding_provider_error", status="error", provider=provider_label, error=str(exc))
                 return {}
             return {"embedding": embedding}
 

package/docs/architecture/local-runtime-2026-03-19.md

@@ -0,0 +1,33 @@
+# Local Runtime Architecture — 2026-03-19
+
+This repo is now documented and operated with a **llama.cpp-first** local runtime architecture.
+
+## Stable loopback-only service split
+
+- OpenClaw gateway/dashboard: `127.0.0.1:17890`
+- ocmemog sidecar/dashboard: `127.0.0.1:17891`
+- llama.cpp text inference: `127.0.0.1:18080`
+- llama.cpp embeddings: `127.0.0.1:18081`
+
+## Active local models
+
+- Text: `Qwen2.5-7B-Instruct-Q4_K_M.gguf`
+- Embeddings: `nomic-embed-text-v1.5.Q4_K_M.gguf`
+
+## Configuration direction
+
+Primary local envs:
+
+- `OCMEMOG_LOCAL_LLM_BASE_URL=http://127.0.0.1:18080/v1`
+- `OCMEMOG_LOCAL_LLM_MODEL=qwen2.5-7b-instruct`
+- `OCMEMOG_LOCAL_EMBED_BASE_URL=http://127.0.0.1:18081/v1`
+- `OCMEMOG_LOCAL_EMBED_MODEL=nomic-embed-text-v1.5`
+
+Legacy Ollama knobs may remain in code for compatibility/rollback, but they are **not the primary runtime path**.
+
+## Operational notes
+
+- The sidecar should remain loopback-only by default.
+- The old plain dashboard lives at `http://127.0.0.1:17891/dashboard`.
+- Memory search and pondering should target the sidecar, not the OpenClaw gateway port.
+- Avoid reusing `17890` for the sidecar; that previously caused a routing collision with the OpenClaw dashboard/gateway.

package/docs/notes/2026-03-18-memory-repair-and-backfill.md

@@ -12,8 +12,8 @@ This pass focused on turning `ocmemog` from a noisy/fragile memory stack into a
 ## Changes landed
 
 ### Embedding and rebuild behavior
-- Fixed the vector reindex entrypoint so it defaults to provider-backed Ollama embeddings instead of silently rebuilding weak hash/simple vectors.
-- Confirmed local Ollama embeddings (`nomic-embed-text:latest`) are available and produce 768-dim vectors.
+- Fixed the vector reindex entrypoint so it defaults to provider-backed local embeddings instead of silently rebuilding weak hash/simple vectors.
+- At the time this landed, the provider-backed path used Ollama-hosted `nomic-embed-text:latest`; the current repo default is the llama.cpp embedding endpoint on `127.0.0.1:18081` with `nomic-embed-text-v1.5`.
 - Added a new incremental repair path:
   - `backfill_missing_vectors()` in `brain/runtime/memory/vector_index.py`
   - `scripts/ocmemog-backfill-vectors.py`
@@ -62,7 +62,7 @@ For laptop-friendly backlog burn-down, use staged backfills in roughly this orde
 6. knowledge last
 
 ## Commits from this sweep
-- `f3d3dd9` — fix: default vector reindex to ollama embeddings
+- `f3d3dd9` — fix: default vector reindex to provider-backed embeddings
 - `759d23d` — feat: add battery-aware sidecar defaults
 - `4a102eb` — fix: clean memory freshness summaries
 - `9ee7966` — fix: report duplicate promotion counts accurately

package/docs/notes/local-model-role-matrix-2026-03-18.md

@@ -1,8 +1,10 @@
 # Local model role matrix — 2026-03-18
 
+Historical note: this bakeoff was recorded before the local-runtime cutover from Ollama to llama.cpp. Keep the conclusions, but map them onto the current llama.cpp-served GGUF models when using this repo today.
+
 Purpose: document which installed local model is best suited for which `ocmemog` task so background cognition can be smarter without putting heavy/slow models on every path.
 
-Installed local models observed:
+Installed local models observed at the time:
 - `phi3:latest`
 - `qwen2.5:7b`
 - `llama3.1:8b`
@@ -45,6 +47,8 @@ Installed local models observed:
 - richer optional background cognition: `llama3.1:8b`
 
 ## Operational recommendation
-- Keep `OCMEMOG_OLLAMA_MODEL=phi3:latest` for lightweight local fallback behavior.
-- Set `OCMEMOG_PONDER_MODEL=qwen2.5:7b` for unresolved-state rewrite, lesson extraction, and cluster recommendation shaping.
+- Current llama.cpp-first equivalent for this repo:
+- Set `OCMEMOG_LOCAL_LLM_MODEL=qwen2.5-7b-instruct` and `OCMEMOG_PONDER_MODEL=local-openai:qwen2.5-7b-instruct` for unresolved-state rewrite, lesson extraction, and cluster recommendation shaping.
+- Set `OCMEMOG_LOCAL_EMBED_MODEL=nomic-embed-text-v1.5` for embeddings on the `18081` endpoint.
+- If you intentionally keep Ollama on another machine, prefer `OCMEMOG_OLLAMA_MODEL=qwen2.5:7b` instead of `phi3`.
 - Consider `llama3.1:8b` for optional deeper background cognition passes where latency is acceptable.

package/docs/usage.md

@@ -2,10 +2,10 @@
 
 ## Current operating model
 
-ocmemog is a repo-local OpenClaw memory sidecar backed by SQLite. It is not a full brAIn runtime clone. The safe assumption is:
+ocmemog is a repo-local OpenClaw memory sidecar backed by SQLite with llama.cpp-first local inference and embeddings. It is not a full brAIn runtime clone. The safe assumption is:
 
 - search/get over local memory are supported
-- heuristic embeddings are supported by default
+- provider-backed local embeddings are the primary path
 - several advanced brAIn memory flows are copied in but still degraded by missing runtime dependencies
 
 ## Running the sidecar
@@ -47,8 +47,12 @@ export OCMEMOG_MEMORY_MODEL=gpt-4o-mini
 export OCMEMOG_OPENAI_API_KEY=sk-...
 export OCMEMOG_OPENAI_API_BASE=https://api.openai.com/v1
 export OCMEMOG_OPENAI_EMBED_MODEL=text-embedding-3-small
+export OCMEMOG_LOCAL_LLM_BASE_URL=http://127.0.0.1:18080/v1
+export OCMEMOG_LOCAL_LLM_MODEL=qwen2.5-7b-instruct
+export OCMEMOG_LOCAL_EMBED_BASE_URL=http://127.0.0.1:18081/v1
+export OCMEMOG_LOCAL_EMBED_MODEL=nomic-embed-text-v1.5
 export BRAIN_EMBED_MODEL_LOCAL=simple
-export BRAIN_EMBED_MODEL_PROVIDER=openai
+export BRAIN_EMBED_MODEL_PROVIDER=local-openai
 export OCMEMOG_TRANSCRIPT_DIR=$HOME/.openclaw/workspace/memory/transcripts
 export OCMEMOG_TRANSCRIPT_GLOB=*.log
 export OCMEMOG_TRANSCRIPT_POLL_SECONDS=1
@@ -182,8 +186,8 @@ Notes:
 - `brain/runtime/memory/api.py`
   - It targets missing/legacy tables and columns.
 - Provider-backed embeddings
-  - Available when `BRAIN_EMBED_MODEL_PROVIDER=openai` and `OCMEMOG_OPENAI_API_KEY` is set.
-  - Falls back to local embeddings when missing.
+  - Available when `BRAIN_EMBED_MODEL_PROVIDER=local-openai` and the local embedding endpoint is reachable.
+  - Legacy OpenAI-hosted embeddings remain available when `BRAIN_EMBED_MODEL_PROVIDER=openai` and `OCMEMOG_OPENAI_API_KEY` is set.
 - Model-backed distillation
   - Available when `OCMEMOG_OPENAI_API_KEY` is set; otherwise falls back to heuristic distill.
 - Role-prioritized context building

package/ocmemog/sidecar/app.py

@@ -19,7 +19,7 @@ from ocmemog.sidecar.transcript_watcher import watch_forever
 
 DEFAULT_CATEGORIES = ("knowledge", "reflections", "directives", "tasks", "runbooks", "lessons")
 
-app = FastAPI(title="ocmemog sidecar", version="0.1.6")
+app = FastAPI(title="ocmemog sidecar", version="0.1.8")
 
 API_TOKEN = os.environ.get("OCMEMOG_API_TOKEN")
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@simbimbo/memory-ocmemog",
-  "version": "0.1.6",
+  "version": "0.1.8",
   "description": "Advanced OpenClaw memory plugin with durable recall, transcript-backed continuity, and sidecar APIs",
   "license": "MIT",
   "repository": {

package/scripts/install-ocmemog.sh

@@ -8,7 +8,9 @@ PLUGIN_PACKAGE="@simbimbo/memory-ocmemog"
 PLUGIN_ID="memory-ocmemog"
 ENDPOINT="${OCMEMOG_ENDPOINT:-http://127.0.0.1:17891}"
 TIMEOUT_MS="${OCMEMOG_TIMEOUT_MS:-30000}"
-DEFAULT_OLLAMA_MODEL="${OCMEMOG_OLLAMA_MODEL:-phi3:latest}"
+DEFAULT_LOCAL_LLM_MODEL="${OCMEMOG_LOCAL_LLM_MODEL:-qwen2.5-7b-instruct}"
+DEFAULT_LOCAL_EMBED_MODEL="${OCMEMOG_LOCAL_EMBED_MODEL:-nomic-embed-text-v1.5}"
+DEFAULT_OLLAMA_MODEL="${OCMEMOG_OLLAMA_MODEL:-qwen2.5:7b}"
 DEFAULT_OLLAMA_EMBED_MODEL="${OCMEMOG_OLLAMA_EMBED_MODEL:-nomic-embed-text:latest}"
 INSTALL_PREREQS="${OCMEMOG_INSTALL_PREREQS:-false}"
 SKIP_PLUGIN_INSTALL="false"
@@ -27,10 +29,10 @@ Arguments:
 
 Options:
   --help                     Show this help text.
-  --install-prereqs          Auto-install missing ollama/ffmpeg via Homebrew.
+  --install-prereqs          Auto-install missing llama.cpp/ffmpeg via Homebrew.
   --skip-plugin-install      Skip OpenClaw plugin install/enable.
   --skip-launchagents        Skip LaunchAgent install/load.
-  --skip-model-pulls         Skip local Ollama model pulls.
+  --skip-model-pulls         Skip local llama.cpp runtime checks.
   --dry-run                  Print what would happen without making changes.
   --endpoint URL             Override sidecar endpoint (default: http://127.0.0.1:17891).
   --timeout-ms N             Override plugin timeout summary value (default: 30000).
@@ -38,8 +40,10 @@ Options:
 
 Environment:
   OCMEMOG_INSTALL_PREREQS=true   Same as --install-prereqs.
-  OCMEMOG_OLLAMA_MODEL           Default local model to pull.
-  OCMEMOG_OLLAMA_EMBED_MODEL     Default local embedding model to pull.
+  OCMEMOG_LOCAL_LLM_MODEL        Default local llama.cpp/OpenAI-compatible text model.
+  OCMEMOG_LOCAL_EMBED_MODEL      Default local llama.cpp/OpenAI-compatible embedding model.
+  OCMEMOG_OLLAMA_MODEL           Legacy Ollama text model fallback.
+  OCMEMOG_OLLAMA_EMBED_MODEL     Legacy Ollama embedding model fallback.
 EOF
 }
 
@@ -125,9 +129,9 @@ maybe_install_prereqs() {
     warn "Homebrew not found; cannot auto-install prerequisites"
     return
   fi
-  if ! have ollama; then
-    log "Installing Ollama via Homebrew"
-    run_cmd brew install ollama || warn "brew install ollama failed"
+  if ! have llama-server; then
+    log "Installing llama.cpp via Homebrew"
+    run_cmd brew install llama.cpp || warn "brew install llama.cpp failed"
   fi
   if ! have ffmpeg; then
     log "Installing ffmpeg via Homebrew"
@@ -206,23 +210,18 @@ install_launchagents() {
   run_cmd "$ROOT_DIR/scripts/ocmemog-install.sh"
 }
 
-ensure_ollama_models() {
+ensure_local_runtime() {
   if [[ "$SKIP_MODEL_PULLS" == "true" ]]; then
-    log "Skipping local model pulls by request"
+    log "Skipping local llama.cpp runtime checks by request"
     return
   fi
-  if ! have ollama; then
-    warn "Ollama not found. Install from https://ollama.com/download to enable local models."
+  if ! have llama-server; then
+    warn "llama-server not found. Install llama.cpp or provide your own local OpenAI-compatible endpoints."
     return
   fi
-  if ! ollama list | rg -q "$(printf '%s' "$DEFAULT_OLLAMA_MODEL" | sed 's/:.*$//')"; then
-    log "Pulling local model $DEFAULT_OLLAMA_MODEL"
-    run_cmd ollama pull "$DEFAULT_OLLAMA_MODEL"
-  fi
-  if ! ollama list | rg -q "$(printf '%s' "$DEFAULT_OLLAMA_EMBED_MODEL" | sed 's/:.*$//')"; then
-    log "Pulling local embed model $DEFAULT_OLLAMA_EMBED_MODEL"
-    run_cmd ollama pull "$DEFAULT_OLLAMA_EMBED_MODEL"
-  fi
+  log "Detected llama.cpp runtime via llama-server"
+  log "Expect local text endpoint at http://127.0.0.1:18080/v1 using model $DEFAULT_LOCAL_LLM_MODEL"
+  log "Expect local embed endpoint at http://127.0.0.1:18081/v1 using model $DEFAULT_LOCAL_EMBED_MODEL"
 }
 
 validate_install() {
@@ -252,12 +251,13 @@ ocmemog install summary
 - repo: $ROOT_DIR
 - endpoint: $ENDPOINT
 - timeoutMs: $TIMEOUT_MS
-- local model: $DEFAULT_OLLAMA_MODEL
-- embed model: $DEFAULT_OLLAMA_EMBED_MODEL
+- local text model: $DEFAULT_LOCAL_LLM_MODEL
+- local embed model: $DEFAULT_LOCAL_EMBED_MODEL
+- legacy Ollama fallback model: $DEFAULT_OLLAMA_MODEL
 - install prereqs automatically: $INSTALL_PREREQS
 - skip plugin install: $SKIP_PLUGIN_INSTALL
 - skip LaunchAgents: $SKIP_LAUNCHAGENTS
-- skip model pulls: $SKIP_MODEL_PULLS
+- skip local runtime checks: $SKIP_MODEL_PULLS
 - dry run: $DRY_RUN
 
 Next checks:
@@ -272,6 +272,6 @@ maybe_install_prereqs
 ensure_python
 install_plugin
 install_launchagents
-ensure_ollama_models
+ensure_local_runtime
 validate_install
 print_summary

package/scripts/ocmemog-backfill-vectors.py

@@ -9,10 +9,12 @@ from pathlib import Path
 REPO_ROOT = Path(__file__).resolve().parents[1]
 sys.path.insert(0, str(REPO_ROOT))
 
-os.environ.setdefault("OCMEMOG_USE_OLLAMA", "true")
-os.environ.setdefault("OCMEMOG_OLLAMA_MODEL", "phi3:latest")
-os.environ.setdefault("OCMEMOG_OLLAMA_EMBED_MODEL", "nomic-embed-text:latest")
-os.environ.setdefault("BRAIN_EMBED_MODEL_PROVIDER", "ollama")
+os.environ.setdefault("OCMEMOG_USE_OLLAMA", "false")
+os.environ.setdefault("OCMEMOG_LOCAL_LLM_BASE_URL", "http://127.0.0.1:18080/v1")
+os.environ.setdefault("OCMEMOG_LOCAL_LLM_MODEL", "qwen2.5-7b-instruct")
+os.environ.setdefault("OCMEMOG_LOCAL_EMBED_BASE_URL", "http://127.0.0.1:18081/v1")
+os.environ.setdefault("OCMEMOG_LOCAL_EMBED_MODEL", "nomic-embed-text-v1.5")
+os.environ.setdefault("BRAIN_EMBED_MODEL_PROVIDER", "local-openai")
 os.environ.setdefault("BRAIN_EMBED_MODEL_LOCAL", "")
 os.environ.setdefault("OCMEMOG_STATE_DIR", str(REPO_ROOT / ".ocmemog-state"))
 

package/scripts/ocmemog-demo.py

@@ -50,7 +50,7 @@ def demo_precision() -> dict:
         "synology nas",
         "openclaw status --deep",
         "gateway bind loopback",
-        "ollama embeddings",
+        "llama.cpp embeddings",
         "memory pipeline",
         "jira projects",
         "calix arden",

package/scripts/ocmemog-install.sh

@@ -66,21 +66,13 @@ for plist in "$ROOT_DIR"/scripts/launchagents/com.openclaw.ocmemog.{sidecar,pond
   echo "Loaded $label"
 done
 
-if ! command -v ollama >/dev/null 2>&1; then
-  echo "Ollama not found. Install from: https://ollama.com/download"
-  echo "Then run: ollama pull phi3 && ollama pull nomic-embed-text"
+if ! command -v llama-server >/dev/null 2>&1; then
+  echo "llama.cpp not found. Install with: brew install llama.cpp"
   exit 0
 fi
 
-if ! ollama list | rg -q "phi3"; then
-  echo "Pulling phi3..."
-  ollama pull phi3
-fi
-
-if ! ollama list | rg -q "nomic-embed-text"; then
-  echo "Pulling nomic-embed-text..."
-  ollama pull nomic-embed-text
-fi
+echo "Expect local llama.cpp text endpoint at http://127.0.0.1:18080/v1"
+echo "Expect local llama.cpp embed endpoint at http://127.0.0.1:18081/v1"
 
 if ! command -v ffmpeg >/dev/null 2>&1; then
   echo "ffmpeg not found. Install with: brew install ffmpeg"

package/scripts/ocmemog-load-test.py

@@ -17,7 +17,7 @@ QUERIES = [
     "ssh key policy",
     "synology nas",
     "openclaw status --deep",
-    "ollama embeddings",
+    "llama.cpp embeddings",
     "memory pipeline",
     "calix arden",
     "gateway bind loopback",

package/scripts/ocmemog-recall-test.py

@@ -11,7 +11,7 @@ QUERIES = [
     "ssh key policy",
     "synology nas",
     "openclaw status --deep",
-    "ollama embeddings",
+    "llama.cpp embeddings",
     "memory pipeline",
     "calix arden",
 ]

package/scripts/ocmemog-reindex-vectors.py

@@ -8,10 +8,12 @@ from pathlib import Path
 REPO_ROOT = Path(__file__).resolve().parents[1]
 sys.path.insert(0, str(REPO_ROOT))
 
-os.environ.setdefault("OCMEMOG_USE_OLLAMA", "true")
-os.environ.setdefault("OCMEMOG_OLLAMA_MODEL", "phi3:latest")
-os.environ.setdefault("OCMEMOG_OLLAMA_EMBED_MODEL", "nomic-embed-text:latest")
-os.environ.setdefault("BRAIN_EMBED_MODEL_PROVIDER", "ollama")
+os.environ.setdefault("OCMEMOG_USE_OLLAMA", "false")
+os.environ.setdefault("OCMEMOG_LOCAL_LLM_BASE_URL", "http://127.0.0.1:18080/v1")
+os.environ.setdefault("OCMEMOG_LOCAL_LLM_MODEL", "qwen2.5-7b-instruct")
+os.environ.setdefault("OCMEMOG_LOCAL_EMBED_BASE_URL", "http://127.0.0.1:18081/v1")
+os.environ.setdefault("OCMEMOG_LOCAL_EMBED_MODEL", "nomic-embed-text-v1.5")
+os.environ.setdefault("BRAIN_EMBED_MODEL_PROVIDER", "local-openai")
 os.environ.setdefault("BRAIN_EMBED_MODEL_LOCAL", "")
 os.environ.setdefault("OCMEMOG_STATE_DIR", str(REPO_ROOT / ".ocmemog-state"))
 

package/scripts/ocmemog-sidecar.sh

@@ -31,12 +31,16 @@ if [[ "$LAPTOP_MODE" == "auto" ]]; then
 fi
 export OCMEMOG_LAPTOP_MODE="$LAPTOP_MODE"
 
-# defaults for local ollama-backed inference/embeddings
-export OCMEMOG_USE_OLLAMA="${OCMEMOG_USE_OLLAMA:-true}"
-export OCMEMOG_OLLAMA_MODEL="${OCMEMOG_OLLAMA_MODEL:-phi3:latest}"
+# defaults for local llama.cpp / OpenAI-compatible inference and embeddings
+export OCMEMOG_USE_OLLAMA="${OCMEMOG_USE_OLLAMA:-false}"
+export OCMEMOG_LOCAL_LLM_BASE_URL="${OCMEMOG_LOCAL_LLM_BASE_URL:-http://127.0.0.1:18080/v1}"
+export OCMEMOG_LOCAL_LLM_MODEL="${OCMEMOG_LOCAL_LLM_MODEL:-qwen2.5-7b-instruct}"
+export OCMEMOG_LOCAL_EMBED_BASE_URL="${OCMEMOG_LOCAL_EMBED_BASE_URL:-http://127.0.0.1:18081/v1}"
+export OCMEMOG_LOCAL_EMBED_MODEL="${OCMEMOG_LOCAL_EMBED_MODEL:-nomic-embed-text-v1.5}"
+export OCMEMOG_OLLAMA_MODEL="${OCMEMOG_OLLAMA_MODEL:-qwen2.5:7b}"
 export OCMEMOG_OLLAMA_EMBED_MODEL="${OCMEMOG_OLLAMA_EMBED_MODEL:-nomic-embed-text:latest}"
-export OCMEMOG_PONDER_MODEL="${OCMEMOG_PONDER_MODEL:-qwen2.5:7b}"
-export BRAIN_EMBED_MODEL_PROVIDER="${BRAIN_EMBED_MODEL_PROVIDER:-ollama}"
+export OCMEMOG_PONDER_MODEL="${OCMEMOG_PONDER_MODEL:-local-openai:qwen2.5-7b-instruct}"
+export BRAIN_EMBED_MODEL_PROVIDER="${BRAIN_EMBED_MODEL_PROVIDER:-local-openai}"
 export BRAIN_EMBED_MODEL_LOCAL="${BRAIN_EMBED_MODEL_LOCAL:-}"
 
 # battery-aware transcript watcher defaults

package/scripts/ocmemog-test-rig.py

@@ -153,8 +153,9 @@ def _distill_batches(endpoint: str, target: int, batch_sizes: list[int], timeout
 
 def _enable_local_embeddings() -> None:
     os.environ.setdefault("BRAIN_EMBED_MODEL_LOCAL", "")
-    os.environ.setdefault("BRAIN_EMBED_MODEL_PROVIDER", "ollama")
-    os.environ.setdefault("OCMEMOG_OLLAMA_EMBED_MODEL", "nomic-embed-text:latest")
+    os.environ.setdefault("BRAIN_EMBED_MODEL_PROVIDER", "local-openai")
+    os.environ.setdefault("OCMEMOG_LOCAL_EMBED_BASE_URL", "http://127.0.0.1:18081/v1")
+    os.environ.setdefault("OCMEMOG_LOCAL_EMBED_MODEL", "nomic-embed-text-v1.5")
 
 
 def main() -> int: