@gotcos/glasses-server 6.21.35 → 6.22.0

package/CHANGELOG.md

@@ -1,3 +1,88 @@
+## 6.22.0
+
+Memory and Threads now work without a Python bridge, a venv, or a vector
+database. Point COS Data at a folder holding `memory/` or `threads/` markdown and
+they are browsable immediately.
+
+This is the move that made Meetings adoptable, applied to context: the
+requirement collapses to markdown files in folders. Any nesting, any filename,
+front matter optional. `type` comes from front matter, else the containing folder
+name, else `note`; ordering from a front-matter date, else a date in the filename,
+else mtime.
+
+- **Backwards compatible by construction, not by promise.** The file tier is
+  reachable only from the branch `callPython` takes when the bridge is ABSENT. An
+  install with a working venv and `cos_api_bridge.py` never executes a line of it,
+  so its behaviour cannot change. There is no merged resolver to get wrong, no
+  migration, no reindex, and no profile edit. A test reads `python-bridge.ts` and
+  fails if the bridge path ever references the file tier.
+- **The routes were the real gate.** `/api/memory`, `/api/memory/:id`,
+  `/api/memory/overview`, `/api/threads` and `/api/threads/:id` returned 503
+  before `callPython` was ever called, so a fallback inside the bridge would have
+  changed nothing observable. They now gate on whether ANY source can answer.
+- **`/api/context/status` reports a file store as available**, with
+  `source: "bridge" | "files"` so a client can say which tier it is showing
+  instead of implying a vector store that is not there. Absent on older servers.
+  `stale` is 0 for file threads because nothing computed staleness — that is the
+  truth, not a default.
+- **File ids are namespaced `file_`,** disjoint from `mem_`, so a reference is
+  never ambiguous about which store it addresses. `MEMORY_ID_PATTERN` accepts
+  both; it previously required `mem_`, which silently dropped every file-backed
+  row from the list and returned a 200 containing nothing.
+- **`COS_CONTEXT_DIR`** points the tier anywhere and is exclusive when set.
+  Otherwise `COS_OPERATIONS_DIR`, `COS_MEETINGS_ROOT` and its parent, the parent
+  of `COS_SCRIPTS_DIR`, then `~/.cos-glasses` — so `mkdir ~/.cos-glasses/memory`
+  is a complete setup.
+- Amendments are NOT in this release. Reads only.
+
+## 6.21.36
+
+Security and accuracy fixes for the 6.21.35 context browser. Two claims in that
+release's notes were false; both are corrected here and the claims restated
+honestly.
+
+- **Local filesystem paths were reaching the lens.** 6.21.35 said "no filesystem
+  paths are exposed." Three shapes leaked, each reproduced against the live store:
+  a tilde path lost only its `~/` and shipped the rest
+  (`~/.cos-glasses/data/voice-profiles.json`); a path with no whitespace before it
+  never matched at all (`KEY=/Users/...`, `>/Users/...`, `,/Users/...`); and a path
+  containing spaces stopped at the first space. The pattern is now anchored to real
+  filesystem roots, which also stops it corrupting API routes — `/api/health` and
+  `/v1/chat/completions` were being replaced with `[local path hidden]`, and that
+  same redacted string is sent to the model as evidence, so a follow-up about a
+  route lost its subject. UNC paths are covered too.
+
+- **Eleven credential families were passing through**, each demonstrated with a
+  correctly-shaped value: HubSpot `pat-`, GitHub `github_pat_`, Slack `xapp-` and
+  `xoxd-`, Google `GOCSPX-`, `npm_`, Stripe `(sk|rk|pk)_live_`, AWS `ASIA` key ids
+  and unprefixed 40-char secret keys, GitLab `glpat-`, SSH2/PuTTY key headers, and
+  `redis://:password@host` where the username is empty. `PWD=` and `PASS=` are now
+  treated as credential names.
+
+  Conversely `MAX_THINKING_TOKENS=31999` was being redacted as a secret — a bare
+  integer is not a credential, and redacting a real setting corrupts evidence
+  without protecting anything.
+
+- **Malformed bridge protocols were relabeled as protocol 1** — precisely what
+  6.21.35's notes said it prevented. `finiteInteger` ran `Math.trunc(Number(v))`
+  before the comparison, so `'1'`, `1.5`, `1.9`, `true` and `[1]` all became `1`
+  and were served as compatible, with the reported field rewritten to `1` so
+  Control could not see what it had been handed. The check is now strict and
+  pre-coercion, and an incompatible protocol is reported as its raw integer or 0,
+  never a truncated 1.
+
+- **Three browse routes could CREATE the Qdrant collection.** `/api/memory`,
+  `/api/memory/:id` and `/api/memory/overview` called `ensure_collection()`, so a
+  read-only G2 browse against a machine with no collection wrote a new empty one
+  and made a broken setup look healthy. They now pass `ensure=False`.
+  `get_summary_stats` already documented this exact reasoning and deliberately
+  omitted the call; the browse paths did not follow it.
+
+Tests were written to fail first: 21 new assertions reproduced real leaks against
+6.21.35 before any pattern changed. One pre-existing assertion changed by a single
+character — the old pattern's segment class included a comma and reported
+`file.txt,` as the filename.
+
 ## 6.21.35
 
 - Adds authenticated `/api/context/status` proof so Control and the companion

package/README.md

@@ -307,8 +307,58 @@ complete Bot Memory count/type split, bounded recent summaries, exact logical
 memory IDs, and existing tracked/manual threads. Exact detail requests are
 resolved by stable ID so a spoken follow-up can carry the selected snapshot as
 context. Embeddings, vector-store point IDs, cache files, secrets, and local
-paths never cross the API boundary. Standalone installs report the feature as
-unavailable without affecting messages, meetings, transcription, or agents.
+paths never cross the API boundary.
+
+### Memory and Threads from plain markdown (6.22.0)
+
+You do not need a Python bridge, a virtual environment, or a vector database.
+Make a folder with a `memory/` or `threads/` subfolder, put markdown files in it,
+and point COS Data at it (or set `COS_CONTEXT_DIR`):
+
+```
+notes/
+  memory/                      any nesting, any filenames
+    2026-08-09-hiring-call.md
+    decisions/pricing.md       folder name becomes the type
+  threads/
+    website-rebuild.md
+```
+
+Front matter is optional and every field degrades rather than rejecting:
+
+```markdown
+---
+type: decision          # else the containing folder name, else "note"
+date: 2026-08-09        # else a YYYY-MM-DD in the filename, else file mtime
+status: resolved        # threads only
+---
+# Held Rain POS for v25
+
+Body text. The first heading becomes the summary.
+```
+
+Two tiers, and the bridge always wins when it is present:
+
+| Tier | Requires | Provides |
+| --- | --- | --- |
+| Files | a folder of markdown | Browse, read, and reference memories and threads |
+| Bridge | + venv, `cos_api_bridge.py`, vector store | Adds semantic recall, dedup, type statistics, retention |
+
+`/api/context/status` reports `source: "bridge"` or `source: "files"` so a client
+can say which tier it is showing. Fields a file-backed record cannot have are
+empty rather than invented: a file thread has `velocity: ""`, `meeting_count: 0`
+and `stale: 0` because nothing computed them.
+
+Root resolution, first match wins: `COS_CONTEXT_DIR` (exclusive when set), then
+`COS_OPERATIONS_DIR`, `COS_MEETINGS_ROOT` and its parent, the parent of
+`COS_SCRIPTS_DIR`, then `~/.cos-glasses` — so `mkdir ~/.cos-glasses/memory` is a
+complete setup. The file tier is read from the code path taken only when no
+bridge is configured, so adding it cannot change the behaviour of an install that
+already has one.
+
+The API is read-only in both tiers. Standalone installs with neither a bridge nor
+a notes folder report the feature as unavailable without affecting messages,
+meetings, transcription, or agents.
 
 The first server start downloads the real-time turbo model. True HQ additionally
 requires the full `ggml-large-v3.bin` model (about 3.1 GB):

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@gotcos/glasses-server",
-  "version": "6.21.35",
+  "version": "6.22.0",
   "description": "COS Glasses — self-hosted AI heads-up-display server for Even G2 smart glasses, powered by Claude Code, Codex, or Cursor Agent CLI",
   "type": "module",
   "bin": {

package/server/lib/context-files.ts

@@ -0,0 +1,390 @@
+// Memory and Threads from plain markdown, with no venv, bridge, or vector store.
+//
+// WHY. Meetings became adoptable when the requirement collapsed to "markdown files
+// in folders" — any domain name, any capitalisation, spaces allowed, with the only
+// structural demand being a YYYY-MM directory because that is the shape the lister
+// reads. Memory and Threads never got that treatment: the COS Data picker requires
+// `operations/scripts/cos_api_bridge.py` AND an executable `venv/bin/python3`, and
+// below that sit Docker and OpenAI embeddings. So the honest answer to "how do I
+// start using memory?" was "clone a workspace and run a vector database."
+//
+// BACKWARDS COMPATIBILITY IS STRUCTURAL, NOT PROMISED. This module is only ever
+// reached from `standaloneNoop` — the branch `callPython` takes when the bridge is
+// ABSENT. An install with a working bridge never enters this code at all, so its
+// behaviour cannot change. That is deliberate: the alternative, a resolver that
+// merges both sources, would put new code in the path of an existing setup.
+//
+// The files are also the IMPORT surface. Someone brings notes in their own shape,
+// COS reads them immediately, and later indexes those same files into their own
+// vector store. The files were always the substrate — which is exactly why an
+// amendment must write only here and never to the derived store.
+
+import { existsSync, readFileSync, readdirSync, statSync } from 'node:fs'
+import { homedir } from 'node:os'
+import { basename, dirname, extname, join, resolve } from 'node:path'
+
+/** Where a file-tier install keeps its notes, relative to the chosen COS Data dir. */
+const MEMORY_DIRS = ['memory', 'memories'] as const
+const THREAD_DIRS = ['threads', 'thread'] as const
+
+/** Bounded so a large folder cannot make a G2 browse slow or a payload huge. */
+const MAX_FILES_SCANNED = 2_000
+const MAX_BODY_CHARS = 32_000
+const SUMMARY_CHARS = 240
+
+/**
+ * Date embedded in a filename, e.g. `2026-08-09-decision.md` or
+ * `2026-08-09_thing.md`. Used for ordering before falling back to mtime, because a
+ * filename date survives a file copy and mtime does not.
+ */
+const FILENAME_DATE = /(\d{4})-(\d{2})-(\d{2})/
+
+export interface FileMemory {
+  id: string
+  type: string
+  summary: string
+  content: string
+  created_at: string
+  domain: string
+  refs: Record<string, string[]>
+  reference_available: true
+}
+
+export interface FileThread {
+  id: string
+  name: string
+  domain: string
+  is_manual: boolean
+  topics: string[]
+  meeting_count: number
+  first_seen: string
+  last_seen: string
+  velocity: string
+  age_days: number
+  is_stale: boolean
+  is_resolved: boolean
+  meetings: Array<{ name: string; date: string }>
+  manual_updates: Array<{ content: string; timestamp: string; source: string }>
+  stakeholders: string[]
+  milestones: string[]
+  sources: string[]
+  target_date: string
+  serves_goal: string
+  created_at: string
+  created_by: string
+  access_count: number
+  reference_available: true
+}
+
+/** First existing candidate directory, or null. */
+function findDir(root: string, names: readonly string[]): string | null {
+  for (const name of names) {
+    const candidate = join(root, name)
+    try { if (statSync(candidate).isDirectory()) return candidate } catch { /* next */ }
+  }
+  return null
+}
+
+/**
+ * Every markdown file under `dir`, recursively, bounded.
+ *
+ * Any nesting is accepted on purpose — a user's notes may be organised by year, by
+ * project, or not at all, and rejecting a shape is what made COS Data unusable for
+ * anyone but its author.
+ */
+function walkMarkdown(dir: string, budget = { left: MAX_FILES_SCANNED }): string[] {
+  const out: string[] = []
+  let entries: import('node:fs').Dirent[]
+  try { entries = readdirSync(dir, { withFileTypes: true }) } catch { return out }
+  for (const entry of entries) {
+    if (budget.left <= 0) break
+    if (entry.name.startsWith('.')) continue
+    const full = join(dir, entry.name)
+    if (entry.isDirectory()) {
+      out.push(...walkMarkdown(full, budget))
+    } else if (entry.isFile() && ['.md', '.markdown', '.txt'].includes(extname(entry.name).toLowerCase())) {
+      budget.left -= 1
+      out.push(full)
+    }
+  }
+  return out
+}
+
+/** Minimal YAML-ish front matter. Absent or malformed degrades to {}. */
+function parseFrontMatter(text: string): { data: Record<string, string>; body: string } {
+  if (!text.startsWith('---')) return { data: {}, body: text }
+  const end = text.indexOf('\n---', 3)
+  if (end === -1) return { data: {}, body: text }
+  const raw = text.slice(3, end)
+  const data: Record<string, string> = {}
+  for (const line of raw.split('\n')) {
+    const i = line.indexOf(':')
+    if (i <= 0) continue
+    const key = line.slice(0, i).trim().toLowerCase()
+    const value = line.slice(i + 1).trim().replace(/^["']|["']$/g, '')
+    if (key) data[key] = value
+  }
+  return { data, body: text.slice(end + 4).replace(/^\n+/, '') }
+}
+
+/** ISO date for ordering: front matter, else filename, else mtime. */
+function resolveDate(path: string, front: Record<string, string>): string {
+  const explicit = front.date || front.created_at || front.created
+  if (explicit && /^\d{4}-\d{2}-\d{2}/.test(explicit)) return explicit
+  const m = FILENAME_DATE.exec(basename(path))
+  if (m) return `${m[1]}-${m[2]}-${m[3]}`
+  try { return new Date(statSync(path).mtimeMs).toISOString() } catch { return '' }
+}
+
+/**
+ * Stable id for a file-backed record.
+ *
+ * Derived from the path so it survives restarts and can be resolved back to a file
+ * without an index. Prefixed `file_` so it can never be mistaken for a `mem_` id
+ * from the vector store — two different stores must not share an id space.
+ */
+function fileId(root: string, path: string): string {
+  const rel = path.startsWith(root) ? path.slice(root.length).replace(/^\//, '') : basename(path)
+  const safe = rel.replace(/[^A-Za-z0-9._-]+/g, '_')
+  // MEMORY_ID_PATTERN caps the id at 120 characters after the prefix and
+  // `cleanContextText` truncates at 128, so a deep path must be shortened HERE.
+  // Truncating alone would collide across two long sibling paths and resolve the
+  // wrong note, so the discarded head is folded into a short hash.
+  if (safe.length <= 100) return `file_${safe}`
+  let hash = 5381
+  for (let i = 0; i < safe.length; i++) hash = ((hash * 33) ^ safe.charCodeAt(i)) >>> 0
+  return `file_${hash.toString(36)}_${safe.slice(-88)}`
+}
+
+function firstHeading(body: string): string {
+  for (const line of body.split('\n')) {
+    const m = /^#{1,3}\s+(.+)$/.exec(line.trim())
+    if (m) return m[1].trim()
+  }
+  return ''
+}
+
+function summarise(body: string): string {
+  const flat = body.replace(/^#{1,6}\s+/gm, '').replace(/\s+/g, ' ').trim()
+  return flat.length <= SUMMARY_CHARS ? flat : `${flat.slice(0, SUMMARY_CHARS)}…`
+}
+
+/** Newest first, by resolved date then path for determinism. */
+function byNewest<T extends { created_at: string; id: string }>(a: T, b: T): number {
+  return b.created_at.localeCompare(a.created_at) || a.id.localeCompare(b.id)
+}
+
+function envPath(name: string): string | null {
+  const raw = process.env[name]?.trim()
+  return raw ? resolve(raw) : null
+}
+
+/**
+ * Where this install keeps plain-text notes, or null.
+ *
+ * Candidates in priority order, first one holding a `memory/` or `threads/`
+ * folder wins. Env is read on every call rather than captured at import, so a
+ * COS Control settings change followed by a restart-free re-read behaves, and so
+ * tests can point it somewhere without module-load ordering games.
+ *
+ * `COS_MEETINGS_ROOT` contributes its PARENT as well as itself, because a direct
+ * meetings library is `.../personal/meetings` while notes sit beside it at
+ * `.../personal/memory`.
+ *
+ * The data home is last and is the zero-configuration answer: someone with no
+ * COS repo at all can `mkdir ~/.cos-glasses/memory` and start browsing. It is
+ * also the only candidate guaranteed to survive a server update, which is where
+ * anything durable belongs.
+ */
+export function resolveContextFilesRoot(): string | null {
+  // An EXPLICIT setting is exclusive, including when it turns out to hold nothing.
+  // Falling through from an empty explicit root to the data home would mean a
+  // user who pointed us at one folder could be served notes from another, and it
+  // would make every test's answer depend on whether `~/.cos-glasses/memory`
+  // happens to exist on the machine running it.
+  const explicit = envPath('COS_CONTEXT_DIR')
+  if (explicit) {
+    return hasFileMemory(explicit) || hasFileThreads(explicit) ? explicit : null
+  }
+  const meetings = envPath('COS_MEETINGS_ROOT')
+  const scripts = envPath('COS_SCRIPTS_DIR')
+  const dataHome = envPath('COS_DATA_DIR') ?? join(homedir(), '.cos-glasses')
+  const candidates = [
+    envPath('COS_OPERATIONS_DIR'),
+    meetings,
+    meetings ? dirname(meetings) : null,
+    scripts ? dirname(scripts) : null,
+    dataHome,
+  ]
+  for (const candidate of candidates) {
+    if (candidate && (hasFileMemory(candidate) || hasFileThreads(candidate))) return candidate
+  }
+  return null
+}
+
+/** Markdown files under `dir` with content, counted without reading any of them. */
+function countMarkdown(dir: string | null): number {
+  if (!dir) return 0
+  let total = 0
+  for (const path of walkMarkdown(dir)) {
+    // `size > 0` instead of a read: this runs on the status route, which the
+    // companion polls, and reading every note to produce one integer is how a
+    // browse surface becomes a performance problem. A whitespace-only file is
+    // counted here but skipped by the readers, so the total can exceed the
+    // browsable list by the number of blank files. That is the only known
+    // discrepancy and it is bounded by files the user created empty.
+    try { if (statSync(path).size > 0) total += 1 } catch { /* vanished */ }
+  }
+  return total
+}
+
+/**
+ * Counts for the status surface.
+ *
+ * Memory is counted by path. Threads are READ, because the status header needs
+ * the active/resolved split and `status: resolved` lives in front matter — and a
+ * thread store is one file per project, so tens of files, not thousands. The
+ * MAX_FILES_SCANNED bound still applies.
+ */
+export function fileTierStatus(root: string | null): {
+  memory: { present: boolean; total: number }
+  threads: { present: boolean; total: number; active: number; resolved: number }
+} {
+  const memoryDir = root ? findDir(root, MEMORY_DIRS) : null
+  const threads = root && hasFileThreads(root) ? readFileThreads(root, MAX_FILES_SCANNED) : []
+  return {
+    memory: { present: !!memoryDir, total: countMarkdown(memoryDir) },
+    threads: {
+      present: !!root && hasFileThreads(root),
+      total: threads.length,
+      active: threads.filter(t => !t.is_resolved).length,
+      resolved: threads.filter(t => t.is_resolved).length,
+    },
+  }
+}
+
+/** Is a file-backed memory store present under this root? */
+export function hasFileMemory(root: string | null): boolean {
+  return !!root && findDir(root, MEMORY_DIRS) !== null
+}
+
+/** Is a file-backed thread store present under this root? */
+export function hasFileThreads(root: string | null): boolean {
+  return !!root && findDir(root, THREAD_DIRS) !== null
+}
+
+/**
+ * Memories from markdown, newest first.
+ *
+ * `type` comes from front matter when present, else the containing folder name,
+ * else 'note' — a folder called `decisions/` is a reasonable signal and asking the
+ * user to add front matter before anything works would defeat the point.
+ */
+export function readFileMemories(root: string, limit = 30): FileMemory[] {
+  const dir = findDir(root, MEMORY_DIRS)
+  if (!dir) return []
+  const out: FileMemory[] = []
+  for (const path of walkMarkdown(dir)) {
+    let text: string
+    try { text = readFileSync(path, 'utf-8') } catch { continue }
+    const { data, body } = parseFrontMatter(text)
+    const trimmed = body.trim()
+    if (!trimmed) continue
+    const parent = basename(join(path, '..'))
+    out.push({
+      id: data.id || fileId(root, path),
+      type: (data.type || (parent && parent !== basename(dir) ? parent : '') || 'note').toLowerCase(),
+      summary: data.summary || firstHeading(trimmed) || summarise(trimmed),
+      content: trimmed.slice(0, MAX_BODY_CHARS),
+      created_at: resolveDate(path, data),
+      domain: data.domain || '',
+      refs: {},
+      reference_available: true,
+    })
+  }
+  return out.sort(byNewest).slice(0, Math.max(1, limit))
+}
+
+/** One file-backed memory by its `file_` id, or null. */
+export function readFileMemoryById(root: string, id: string): FileMemory | null {
+  // Scanning rather than path-reconstructing: the id is sanitised, so mapping it
+  // back to a path would mean trusting a lossy transform to address the filesystem.
+  return readFileMemories(root, MAX_FILES_SCANNED).find(m => m.id === id) ?? null
+}
+
+/** Total count and per-type split, for the overview surface. */
+export function fileMemoryOverview(root: string): { total: number; by_type: Record<string, number> } {
+  const all = readFileMemories(root, MAX_FILES_SCANNED)
+  const by_type: Record<string, number> = {}
+  for (const m of all) by_type[m.type] = (by_type[m.type] ?? 0) + 1
+  return { total: all.length, by_type }
+}
+
+/**
+ * Threads from markdown, newest first.
+ *
+ * Every derived field a vector-tier thread carries is present and honestly empty —
+ * velocity '', meeting_count 0 — rather than absent, so the same display code
+ * renders both tiers without branching. A file thread has no computed velocity
+ * because nothing computed it; that is a real difference, not a missing value to
+ * be invented.
+ */
+export function readFileThreads(root: string, limit = 30): FileThread[] {
+  const dir = findDir(root, THREAD_DIRS)
+  if (!dir) return []
+  const out: FileThread[] = []
+  for (const path of walkMarkdown(dir)) {
+    let text: string
+    try { text = readFileSync(path, 'utf-8') } catch { continue }
+    const { data, body } = parseFrontMatter(text)
+    const trimmed = body.trim()
+    if (!trimmed) continue
+    const created = resolveDate(path, data)
+    const topics = (data.topics || data.tags || '')
+      .split(/[,;]/).map(t => t.trim()).filter(Boolean)
+    out.push({
+      id: data.id || fileId(root, path),
+      name: data.name || data.title || firstHeading(trimmed) || basename(path, extname(path)),
+      domain: data.domain || '',
+      is_manual: true,
+      topics,
+      meeting_count: 0,
+      first_seen: created,
+      last_seen: created,
+      velocity: '',
+      age_days: 0,
+      is_stale: false,
+      is_resolved: (data.status || '').toLowerCase() === 'resolved',
+      meetings: [],
+      manual_updates: [{ content: trimmed.slice(0, MAX_BODY_CHARS), timestamp: created, source: 'file' }],
+      stakeholders: (data.stakeholders || '').split(/[,;]/).map(s => s.trim()).filter(Boolean),
+      milestones: [],
+      sources: [],
+      target_date: data.target_date || '',
+      serves_goal: data.goal || data.serves_goal || '',
+      created_at: created,
+      created_by: data.created_by || 'file',
+      access_count: 0,
+      reference_available: true,
+    })
+  }
+  return out.sort(byNewest).slice(0, Math.max(1, limit))
+}
+
+/** One file-backed thread by id, or null. */
+export function readFileThreadById(root: string, id: string): FileThread | null {
+  return readFileThreads(root, MAX_FILES_SCANNED).find(t => t.id === id) ?? null
+}
+
+/** Does this root hold anything the file tier can serve? */
+export function fileTierState(root: string | null): 'ready' | 'empty' | 'absent' {
+  if (!root || !existsSync(root)) return 'absent'
+  const dirs = [findDir(root, MEMORY_DIRS), findDir(root, THREAD_DIRS)].filter((d): d is string => !!d)
+  if (dirs.length === 0) return 'absent'
+  // Existence, not content: `walkMarkdown` stops at the first hit, so this stays
+  // a couple of readdir calls even on a large tree. 'empty' is a HEALTHY state —
+  // a configured store with nothing in it yet — and must be distinguishable from
+  // 'absent', which is what the picker and the wizard branch on.
+  const any = dirs.some(dir => walkMarkdown(dir, { left: 1 }).length > 0)
+  return any ? 'ready' : 'empty'
+}

package/server/lib/cos-context-browser.ts

@@ -1,16 +1,98 @@
-export const MEMORY_ID_PATTERN = /^mem_[A-Za-z0-9_:-]{1,120}$/
+// `mem_` addresses the vector store, `file_` addresses a note on disk. Two stores,
+// one id space, disjoint prefixes — so a reference is never ambiguous about which
+// one it means. `.` is permitted because a file id carries its extension; it
+// cannot traverse, since `/` is excluded and a bare `..` cannot match the prefix.
+export const MEMORY_ID_PATTERN = /^(?:mem|file)_[A-Za-z0-9._:-]{1,120}$/
 export const THREAD_ID_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$/
 
 const CONTROL_CHARS = /[\u0000-\u0008\u000b\u000c\u000e-\u001f\u007f]/g
-const ABSOLUTE_PATH = /(^|[\s("'`])(?:~\/|\/(?!\/)(?:[^/\s)"'`]+\/)+[^/\s)"'`]+\/?)/g
+/**
+ * Filesystem roots. Anchoring to real roots does two jobs at once: it catches the
+ * shapes the old pattern missed, and it stops redacting API routes.
+ *
+ * The old pattern was `(^|[\s("'`])(?:~\/|\/(?!\/)…)`. Three failures, all
+ * reproduced against 6.21.35 on the live store before this rewrite:
+ *   1. `~\/` had no segment requirement, so `~/.cos-glasses/data/x.json` lost only
+ *      the two characters "~/" and shipped the rest to the lens.
+ *   2. The leading `(^|[\s("'`])` meant `KEY=/Users/...`, `>/Users/...` and
+ *      `,/Users/...` never matched at all.
+ *   3. Being root-agnostic, it redacted `/api/context/status` and `/v1/chat/…`
+ *      — and that same redacted string is sent to the model as evidence, so a
+ *      follow-up about a route lost its subject.
+ */
+const FS_ROOTS = 'Users|home|root|opt|usr|var|private|tmp|Volumes|etc|Library|System|Applications'
+/**
+ * Greedy to a hard delimiter, then trimmed back in the replacer. A path may
+ * legitimately contain spaces — this machine's own repo root does — and no regex
+ * can tell "space inside a path" from "space before the next word", so the
+ * boundary decision is made in code where it can be tested.
+ */
+const ABSOLUTE_PATH = new RegExp(`(?:~|/(?:${FS_ROOTS}))(?:/[^,;)"'\`\r\n]*)?`, 'g')
+const UNC_PATH = /\\\\[A-Za-z0-9._-]+\\(?:[^\\\r\n]+\\?)*/g
+
+/**
+ * Trim a greedy path match back to its last plausible component.
+ *
+ * Keeps `…/Ukaoma Chief Of Staff/MU/ops/comp.md` whole (every space is followed by
+ * more path) while not swallowing the trailing prose in `/Users/me/x.md here`.
+ * A component qualifies if it is followed by a `/` or contains a dot.
+ */
+function pathMatchEnd(match: string): number {
+  const lastSlash = match.lastIndexOf('/')
+  if (lastSlash <= 0) return match.length
+  const tail = match.slice(lastSlash + 1)
+  const space = tail.indexOf(' ')
+  // No space in the final component: the whole match is path.
+  if (space === -1) return match.length
+  // A dotted filename before the space is the end of the path.
+  const beforeSpace = tail.slice(0, space)
+  return beforeSpace.length > 0 ? lastSlash + 1 + beforeSpace.length : lastSlash
+}
 const SECRET_TOKEN = /\b(?:sk-[A-Za-z0-9_-]{12,}|(?:bearer|token|api[_ -]?key)\s*[:=]\s*[A-Za-z0-9._-]{12,})\b/gi
 const WINDOWS_PATH = /\b[A-Za-z]:\\(?:[^\\\r\n]+\\)*[^\\\r\n]*/g
-const PEM_BLOCK = /-----BEGIN [^-\r\n]+(?:PRIVATE KEY|KEY)[^-\r\n]*-----[\s\S]*?(?:-----END [^-\r\n]+-----|$)/g
+// 4 OR 5 dashes: SSH2/PuTTY exports use four, and PuTTY uses a bare header line.
+const PEM_BLOCK = /-{4,5} ?BEGIN [^-\r\n]*(?:PRIVATE KEY|KEY)[^-\r\n]*-{4,5}[\s\S]*?(?:-{4,5} ?END [^-\r\n]+-{4,5}|$)/g
+const PUTTY_KEY = /\bPuTTY-User-Key-File-\d+:.*/g
 const JWT_TOKEN = /\beyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\b/g
-const PREFIXED_SECRET = /\b(?:gh[pousr]_[A-Za-z0-9]{20,}|xox[baprs]-[A-Za-z0-9-]{20,}|AIza[A-Za-z0-9_-]{20,}|AKIA[A-Z0-9]{16})\b/g
-const URL_CREDENTIALS = /(\b[a-z][a-z0-9+.-]{0,20}:\/\/)[^\s/@:]+:[^\s/@]+@/gi
+/**
+ * Credential prefixes. Every family added here was demonstrated LEAKING against
+ * 6.21.35 with a correctly-shaped fabricated value. `pat-` is the one that
+ * mattered most: HubSpot PAK rotation is a recurring topic in this store, and
+ * `/end-session` writes rotation notes into it.
+ */
+const PREFIXED_SECRET = new RegExp([
+  'gh[pousr]_[A-Za-z0-9]{20,}',
+  'github_pat_[A-Za-z0-9_]{30,}',
+  'xox[baprsde]-[A-Za-z0-9-]{15,}',      // +d (cookie) +e (refresh)
+  'xapp-[A-Za-z0-9-]{15,}',
+  'AIza[A-Za-z0-9_-]{20,}',
+  'AKIA[A-Z0-9]{16}',
+  'ASIA[A-Z0-9]{16}',
+  'GOCSPX-[A-Za-z0-9_-]{16,}',
+  'npm_[A-Za-z0-9]{30,}',
+  '(?:sk|rk|pk)_(?:live|test)_[A-Za-z0-9]{16,}',
+  'pat-[a-z0-9]{2,4}-[0-9a-fA-F-]{20,}',
+  'glpat-[A-Za-z0-9_-]{16,}',
+].join('|'), 'g')
+/**
+ * AWS secret access keys have no prefix: exactly 40 chars of the base64 alphabet.
+ *
+ * A `/` or `+` is REQUIRED, which is a deliberate trade-off. Without it, any
+ * 40-char alphanumeric run matches — and a SHA-1 hex digest is exactly 40 chars,
+ * so every commit sha in this store would be redacted as a credential. The cost
+ * is that an all-alphanumeric AWS key is missed; the alternative corrupts far more
+ * real evidence than it protects.
+ */
+const AWS_SECRET_KEY = /(?<![A-Za-z0-9/+])(?=[A-Za-z0-9/+]{40}(?![A-Za-z0-9/+]))(?=[A-Za-z0-9]*[/+])[A-Za-z0-9/+]{40}/g
+// Username may be EMPTY — `redis://:password@host` defeated the old `+`.
+const URL_CREDENTIALS = /(\b[a-z][a-z0-9+.-]{0,20}:\/\/)[^\s/@:]*:[^\s/@]+@/gi
 const QUERY_SECRET = /([?&](?:access_token|api_key|apikey|token|secret|password|key)=)[^&#\s]+/gi
-const ENV_SECRET = /\b([A-Z0-9_]*(?:SECRET|TOKEN|PASSWORD|PASSWD|API_KEY|PRIVATE_KEY|DATABASE_URL)[A-Z0-9_]*\s*=)\s*[^\s]+/gi
+/**
+ * Env-style assignments. Added PWD and PASS; requires a value that is not purely
+ * numeric, because `MAX_THINKING_TOKENS=31999` is a real setting in this store and
+ * redacting it corrupted evidence without protecting anything.
+ */
+const ENV_SECRET = /\b([A-Z0-9_]*(?:SECRET|TOKEN|PASSWORD|PASSWD|PWD|PASS|API_KEY|PRIVATE_KEY|DATABASE_URL)[A-Z0-9_]*\s*=)\s*(?!\d+\b)[^\s]+/gi
 const BEARER_SECRET = /\b(Bearer\s+)[A-Za-z0-9._~+\/-]{12,}/gi
 const FILE_URI = /\bfile:\/\/(?:localhost)?\/(?:[^\s)"'`]+\/?)+/gi
 const LABELED_PATH = /\b(path|file|folder|directory)\s*[:=]\s*(?:~\/|\/(?!\/)[^\s,;)}\]"'`]+)/gi
@@ -64,6 +146,7 @@ export function cleanContextText(value: unknown, limit: number): string {
   text = text.slice(0, Math.max(limit, Math.min(64_000, limit * 2)))
   text = text.replace(CONTROL_CHARS, '')
   text = text.replace(PEM_BLOCK, '[secret hidden]')
+  text = text.replace(PUTTY_KEY, '[secret hidden]')
   text = text.replace(JWT_TOKEN, '[secret hidden]')
   text = text.replace(PREFIXED_SECRET, '[secret hidden]')
   text = text.replace(URL_CREDENTIALS, '$1[credentials hidden]@')
@@ -73,7 +156,15 @@ export function cleanContextText(value: unknown, limit: number): string {
   text = text.replace(FILE_URI, '[local path hidden]')
   text = text.replace(LABELED_PATH, '$1: [local path hidden]')
   text = text.replace(WINDOWS_PATH, '[local path hidden]')
-  text = text.replace(ABSOLUTE_PATH, (_match, prefix: string) => `${prefix}[local path hidden]`)
+  text = text.replace(UNC_PATH, '[local path hidden]')
+  // Trimmed in code, not by the regex: a path may contain spaces, so the greedy
+  // match is walked back to its last plausible component.
+  text = text.replace(ABSOLUTE_PATH, (match: string) => {
+    const end = pathMatchEnd(match)
+    return `[local path hidden]${match.slice(end)}`
+  })
+  // AFTER the path rules, so a 40-char path segment cannot be mistaken for a key.
+  text = text.replace(AWS_SECRET_KEY, '[secret hidden]')
   text = text.replace(SECRET_TOKEN, '[secret hidden]')
   return text.trim().slice(0, limit)
 }
@@ -265,6 +356,13 @@ export interface ContextBrowserStatus {
   available: boolean
   protocol: number
   state?: string
+  /**
+   * WHICH tier answered: 'bridge' is a Python/vector pipeline, 'files' is plain
+   * markdown on disk. Present so a client can say what it is showing instead of
+   * implying a vector store that may not exist. Absent means the field was not
+   * supplied, which is every response from a server older than 6.22.0.
+   */
+  source?: 'bridge' | 'files'
   memory: { available: boolean; total: number; state: string; reason?: string }
   threads: { available: boolean; total: number; active: number; stale: number; resolved: number; state: string; reason?: string }
 }
@@ -277,8 +375,16 @@ export function normalizeContextBrowserStatus(value: unknown): ContextBrowserSta
   const threads = source.threads && typeof source.threads === 'object' && !Array.isArray(source.threads)
     ? source.threads as Record<string, unknown> : {}
   const cleanState = (candidate: unknown, fallback: string) => cleanContextText(candidate, 64) || fallback
-  const protocol = finiteInteger(source.protocol)
-  const protocolCompatible = protocol === 1
+  // STRICT and PRE-COERCION. finiteInteger() does Math.trunc(Number(v)), so
+  // 6.21.35 turned '1', 1.5, 1.9, true and [1] all into 1 and served them as
+  // compatible — and rewrote the reported field to 1, so Control could not even
+  // see what it had been handed. The changelog claimed the opposite.
+  const protocolCompatible = source.protocol === 1
+  // Report the RAW value when it is a clean integer, else 0. Truncating 1.5 to 1
+  // is what let an incompatible bridge look compatible in Control's own display.
+  const protocol = protocolCompatible
+    ? 1
+    : (Number.isInteger(source.protocol) ? source.protocol as number : 0)
   const memoryAvailable = protocolCompatible && memory.available === true
   const threadsAvailable = protocolCompatible && threads.available === true
   const incompatibleState = protocolCompatible ? '' : 'bridge_outdated'
@@ -290,6 +396,11 @@ export function normalizeContextBrowserStatus(value: unknown): ContextBrowserSta
     ...(!protocolCompatible
       ? { state: 'bridge_outdated' }
       : source.state ? { state: cleanState(source.state, 'unavailable') } : {}),
+    // Allowlisted, not passed through: an unrecognised value would let a future
+    // or hostile payload put arbitrary text on a surface a client renders.
+    ...(protocolCompatible && (source.source === 'bridge' || source.source === 'files')
+      ? { source: source.source }
+      : {}),
     memory: {
       available: memoryAvailable,
       total: memoryAvailable ? finiteInteger(memory.total) : 0,

package/server/lib/python-bridge.ts

@@ -1,6 +1,18 @@
 import { execFile } from 'node:child_process'
 import { resolve } from 'node:path'
 import { existsSync } from 'node:fs'
+import {
+  fileMemoryOverview,
+  fileTierState,
+  fileTierStatus,
+  hasFileMemory,
+  hasFileThreads,
+  readFileMemories,
+  readFileMemoryById,
+  readFileThreadById,
+  readFileThreads,
+  resolveContextFilesRoot,
+} from './context-files.js'
 
 // Optional COS pipeline bridge.
 //
@@ -40,6 +52,21 @@ export function pythonBridgeState(): 'ready' | 'pipeline_missing' | 'bridge_miss
   return COS_SCRIPTS_DIR ? 'bridge_missing' : 'pipeline_missing'
 }
 
+/**
+ * Which context source can answer a memory/threads request: the Python bridge,
+ * plain files, or nothing.
+ *
+ * The routes gate on THIS rather than on `pythonBridgeAvailable()`, which is what
+ * made the file tier unreachable in the first draft — every route returned 503
+ * before `callPython` was ever called, so wiring the fallback into the bridge
+ * alone changed nothing observable. Bridge first, always: an install with a venv
+ * behaves exactly as it did.
+ */
+export function contextSourceAvailable(): 'bridge' | 'files' | null {
+  if (pythonAvailable) return 'bridge'
+  return fileTierState(resolveContextFilesRoot()) === 'absent' ? null : 'files'
+}
+
 if (pythonAvailable) {
   console.log('[python-bridge] COS pipeline detected — sourcing live context')
 } else if (COS_SCRIPTS_DIR) {
@@ -58,37 +85,119 @@ export function callPython(args: string[], timeoutMs = 30_000): Promise<unknown>
   return Promise.resolve(standaloneNoop(args))
 }
 
-/** Empty shapes that the context builder tolerates (no crash, no live data). */
+/**
+ * No bridge configured. Try the FILE tier first, then fall back to empty shapes.
+ *
+ * This function is the only door to the file tier, and it is only reached when
+ * `pythonAvailable` is false — so an install with a working venv and bridge never
+ * executes any of it. That is what makes the file tier backwards compatible by
+ * construction rather than by promise: there is no merged resolver to get wrong.
+ *
+ * A user with a `memory/` or `threads/` folder gets browsable content with no venv,
+ * no bridge, and no vector store. When they later build the pipeline, the SAME files
+ * become the thing it indexes — nothing to refactor, because the files were always
+ * the substrate.
+ */
 function standaloneNoop(args: string[]): unknown {
+  const root = resolveContextFilesRoot()
   switch (args[0]) {
     case 'calendar': return { events: [] }
     case 'tasks': return {}
-    case 'threads': return { threads: [], active_count: 0, stale_count: 0, resolved_count: 0 }
-    case 'thread-detail': return { error: 'cos_pipeline_not_configured' }
+    case 'threads': {
+      if (root && hasFileThreads(root)) {
+        const threads = readFileThreads(root, argLimit(args, 30))
+        return {
+          threads,
+          active_count: threads.filter(t => !t.is_resolved).length,
+          stale_count: 0,
+          resolved_count: threads.filter(t => t.is_resolved).length,
+          source: 'files',
+        }
+      }
+      return { threads: [], active_count: 0, stale_count: 0, resolved_count: 0 }
+    }
+    case 'thread-detail': {
+      if (root && hasFileThreads(root)) {
+        const hit = readFileThreadById(root, args[1] ?? '')
+        if (hit) return hit
+      }
+      return { error: 'cos_pipeline_not_configured' }
+    }
     case 'context-status': {
-      const state = pythonBridgeState()
+      const bridgeState = pythonBridgeState()
+      const tier = fileTierState(root)
+      // 'files' is a real, working configuration — not a degraded bridge. Reporting
+      // it as unavailable is what made the G2 render "Unavailable." for a user who
+      // had perfectly readable notes.
+      if (tier !== 'absent') {
+        const counts = fileTierStatus(root)
+        return {
+          available: true,
+          protocol: 1,
+          state: tier,
+          source: 'files',
+          memory: {
+            available: counts.memory.present,
+            total: counts.memory.total,
+            state: counts.memory.present ? tier : 'absent',
+          },
+          threads: {
+            available: counts.threads.present,
+            total: counts.threads.total,
+            active: counts.threads.active,
+            // Staleness is a computed property of meeting cadence. Nothing
+            // computed it for a file thread, so 0 is the truth, not a default.
+            stale: 0,
+            resolved: counts.threads.resolved,
+            state: counts.threads.present ? tier : 'absent',
+          },
+        }
+      }
       return {
         available: false,
         protocol: 1,
-        state,
-        memory: { available: false, total: 0, state },
-        threads: { available: false, total: 0, active: 0, stale: 0, resolved: 0, state },
+        state: bridgeState,
+        memory: { available: false, total: 0, state: bridgeState },
+        threads: { available: false, total: 0, active: 0, stale: 0, resolved: 0, state: bridgeState },
       }
     }
-    case 'memory': return []
-    case 'memory-overview': return {
-      available: false,
-      collection: 'cos_memory',
-      total: 0,
-      by_type: {},
-      reason: 'cos_pipeline_not_configured',
+    case 'memory': {
+      if (root && hasFileMemory(root)) return readFileMemories(root, argLimit(args, 30))
+      return []
+    }
+    case 'memory-overview': {
+      if (root && hasFileMemory(root)) {
+        const o = fileMemoryOverview(root)
+        return { available: true, collection: 'files', total: o.total, by_type: o.by_type, source: 'files' }
+      }
+      return {
+        available: false,
+        collection: 'cos_memory',
+        total: 0,
+        by_type: {},
+        reason: 'cos_pipeline_not_configured',
+      }
+    }
+    case 'memory-detail': {
+      if (root && hasFileMemory(root)) {
+        const hit = readFileMemoryById(root, args[1] ?? '')
+        if (hit) return hit
+      }
+      return { error: 'cos_pipeline_not_configured' }
     }
-    case 'memory-detail': return { error: 'cos_pipeline_not_configured' }
     case 'badges': return {}
     default: return {}
   }
 }
 
+/** `--limit N` from a bridge-style argv, bounded. */
+function argLimit(args: string[], fallback: number): number {
+  const i = args.indexOf('--limit')
+  if (i === -1) return fallback
+  const n = Number(args[i + 1])
+  return Number.isFinite(n) && n > 0 ? Math.min(Math.trunc(n), 50) : fallback
+}
+
 /** Full Python bridge — requires the user's venv + cos_api_bridge.py. */
 function callPythonDirect(args: string[], timeoutMs: number): Promise<unknown> {
   return new Promise((resolvePromise, reject) => {

package/server/routes/memory.ts

@@ -1,5 +1,17 @@
 import { Router } from 'express'
-import { callPython, pythonBridgeAvailable, pythonBridgeState } from '../lib/python-bridge.js'
+import { callPython, contextSourceAvailable, pythonBridgeState } from '../lib/python-bridge.js'
+
+/**
+ * Is there anything to serve — a Python bridge OR plain files on disk?
+ *
+ * These routes used to gate on `pythonBridgeAvailable()`, which returned 503
+ * before `callPython` was ever reached. That is why "I selected COS Memory and
+ * the G2 says Unavailable" was the experience for anyone without a venv and a
+ * vector database: the answer was decided two layers above the data.
+ */
+function contextConfigured(): boolean {
+  return contextSourceAvailable() !== null
+}
 import {
   MEMORY_ID_PATTERN,
   normalizeMemoryDetail,
@@ -12,7 +24,7 @@ export const memoryRouter = Router()
 let overviewCache: { expiresAt: number; value: ReturnType<typeof normalizeMemoryOverview> } | null = null
 
 memoryRouter.get('/context/status', async (_req, res) => {
-  if (!pythonBridgeAvailable()) {
+  if (!contextConfigured()) {
     const state = pythonBridgeState()
     res.json(normalizeContextBrowserStatus({
       available: false, protocol: 1, state,
@@ -40,7 +52,7 @@ function boundedInteger(value: unknown, fallback: number, min: number, max: numb
 }
 
 memoryRouter.get('/memory/overview', async (_req, res) => {
-  if (!pythonBridgeAvailable()) {
+  if (!contextConfigured()) {
     res.status(503).json(normalizeMemoryOverview({
       available: false, reason: pythonBridgeState(), total: 0, by_type: {},
     }))
@@ -71,7 +83,7 @@ memoryRouter.get('/memory/:id', async (req, res) => {
     res.status(400).json({ error: 'invalid_memory_id' })
     return
   }
-  if (!pythonBridgeAvailable()) {
+  if (!contextConfigured()) {
     res.status(503).json({ error: pythonBridgeState() })
     return
   }
@@ -95,7 +107,7 @@ memoryRouter.get('/memory/:id', async (req, res) => {
 memoryRouter.get('/memory', async (req, res) => {
   const days = boundedInteger(req.query.days, 30, 1, 3650)
   const limit = boundedInteger(req.query.limit, 20, 1, 50)
-  if (!pythonBridgeAvailable()) {
+  if (!contextConfigured()) {
     res.status(503).json({ error: pythonBridgeState() })
     return
   }

package/server/routes/threads.ts

@@ -1,5 +1,5 @@
 import { Router } from 'express'
-import { callPython, pythonBridgeAvailable, pythonBridgeState } from '../lib/python-bridge.js'
+import { callPython, contextSourceAvailable, pythonBridgeState } from '../lib/python-bridge.js'
 import { THREAD_ID_PATTERN, normalizeThreadDetail, normalizeThreads } from '../lib/cos-context-browser.js'
 
 export const threadsRouter = Router()
@@ -9,7 +9,7 @@ threadsRouter.get('/threads/:id', async (req, res) => {
     res.status(400).json({ error: 'invalid_thread_id' })
     return
   }
-  if (!pythonBridgeAvailable()) {
+  if (contextSourceAvailable() === null) {
     res.status(503).json({ error: pythonBridgeState() })
     return
   }
@@ -33,7 +33,7 @@ threadsRouter.get('/threads/:id', async (req, res) => {
 threadsRouter.get('/threads', async (req, res) => {
   const parsed = Number(req.query.limit)
   const limit = Number.isFinite(parsed) ? Math.max(1, Math.min(50, Math.trunc(parsed))) : 30
-  if (!pythonBridgeAvailable()) {
+  if (contextSourceAvailable() === null) {
     res.status(503).json({
       error: pythonBridgeState(), available: false,
       generated_at: '', active_count: 0, stale_count: 0, resolved_count: 0, threads: [],