ko-qmd 2.8.3-ko.3 → 2.8.3-ko.4

package/CHANGELOG.md

@@ -2,6 +2,135 @@
 
 ## [Unreleased]
 
+## [2.8.3-ko.4] - 2026-09-24
+
+Korean documents are now chunked at their real token budget, and Hangul
+spellings of technical loanwords (`서치`, `웹 코어`) reach notes written in Latin
+script. This release also syncs tobi/qmd up to `04e4dbd` — metadata filtering,
+the oxlint fence and the REST JSON body guard — and adds `qmd grep`, `path`/
+`since`/`until` scoping on MCP and REST, and a daemon that keeps the text index
+fresh on its own. Existing indexes need one `qmd embed` run to pick up the new
+chunking, and SDK callers that passed `filter` for path/time scoping must rename
+it to `scope`.
+
+### Breaking
+
+- SDK: ko-qmd's path/time narrowing option is now `scope` (was `filter`) on
+  `search()`, `searchLex()` and `searchVector()`, and the trailing `scope`
+  parameter of `searchFTS`/`searchVec`. `filter` is upstream's metadata filter.
+  CLI `--path/--since/--until` and REST/MCP `path/since/until` are unchanged,
+  and a request may pass both — a result has to satisfy each.
+
+### Upstream sync
+
+- Merged tobi/qmd `main` up to `04e4dbd` (24 commits past v2.8.3): oxlint
+  fence (`bun run lint` is oxlint; the tsc check is `bun run test:types`),
+  metadata filtering, separator-aware FTS terms (quoted `"PIO-1384"` now
+  matches), REST JSON body guard, Windows index names, unreadable-gguf
+  diagnostics and the store-local embedding tokenizer. ko-qmd behaviour is
+  kept: the Hangul query branch runs ahead of the separator split, a metadata
+  filter gets the same anti-starvation scan as `--path`, and bm25 on the
+  ko-vault bench is identical per query.
+
+### Added
+
+- Added Oxlint lint fence.
+- Document metadata and metadata filtering. Markdown documents can opt into typed metadata through a namespaced frontmatter block (`qmd.metadata` with strings, numbers, booleans, or flat homogeneous arrays), and every search surface — CLI `search`/`vsearch`/`query` via `--filter <json>`, the SDK's `filter` option on `search()`/`searchLex()`/`searchVector()`, the MCP `query` tool, and HTTP `POST /query` and `/search` — accepts one shared recursive filter AST discriminated by `operator`: `and`/`or`/`not` logical groups, `eq`/`ne`/`gt`/`gte`/`lt`/`lte` comparisons, `in`/`nin`/`all` membership, and `exists` presence. Every returned result satisfies the filter (applied before RRF fusion and reranking); like collection filtering, highly selective filters remain best-effort for top-K completeness. Frontmatter stays ordinary searchable content — no chunking, embedding, snippet, or line-number changes — and documents without `qmd.metadata` behave exactly as before. JSON/SDK/MCP/HTTP results now include each document's indexed metadata, and `qmd status` reports how many documents still need metadata extraction (a normal `qmd update` backfills existing indexes).
+
+### Fixed
+
+- Embedding generation and legacy fingerprint adoption now tokenize documents
+  with the store-selected embedding model instead of the global default. This
+  keeps chunk boundaries aligned with the model that creates and verifies the
+  stored vectors without initializing an unrelated provider.
+
+### Changes
+
+- Plain Hangul lex terms that spell a common technical loanword (`서치`, `웹`,
+  `코어`, `마이그레이션`, about 80 entries in a built-in table) also match the
+  Latin spelling, so `하이브리드 서치` finds a note that only says
+  `hybrid-search` and `레몬 웹 코어` reaches `lemon-web-core`. The lookup runs
+  on particle-stripped stems too (`서치를`), single-syllable entries (`웹`, `훅`)
+  are bridged when they stand alone (not `웹을`), and terms with no entry produce the same FTS5 query as before.
+  Query-side only; no re-index. On the ko-vault bench the eleven new loanword
+  queries go from bm25_r5 0.0000 to 1.0000 while the original 52 hold at
+  0.9519 per query.
+- REST `POST /query` and the MCP `query` tool attach `vec_score` (cosine
+  similarity) and `fts_score` (normalized BM25) to each result on the
+  `rerank:false` path, where `score` is the 1/rank fusion position and cannot
+  be thresholded. Each is the max over the sub-queries that returned the hit
+  and is omitted, never 0, when that backend did not return it. The daemon
+  query log (`QMD_QUERY_LOG`) records the same two fields per result; the row
+  schema stays `v: 1` and `rerank:true` responses are unchanged.
+- Korean documents are chunked at their real token budget instead of the
+  English-tuned 3.0 chars/token: `estimateCharsPerToken` blends CJK and other
+  scripts as a harmonic mean (measured ko 1.64 / en 6.08 chars per token on
+  Qwen3-Embedding-0.6B-Q8_0, 2026-09-21), and indexing plus both search-path
+  best-chunk selections size chunks with it. A 20KB Korean document now
+  first-passes into ~900-token chunks instead of being split ~855 and Korean
+  rerank chunks shrink from ~3600 to ~1440 chars. Non-CJK text keeps the ratio
+  each call site already used — 3.0 when indexing, 4.0 on the search paths — so
+  English chunk boundaries do not move, and the ko-vault bench holds
+  (bm25_r5=0.9519).
+- Re-embedding a document whose chunk count shrank now prunes its stale
+  higher-`seq` vectors before inserting, so one `qmd embed` run converges
+  instead of waiting for a second run. The embedding fingerprint gains a
+  `chunk_chars_estimator` term, so `qmd status` reports re-embedding as needed
+  for existing indexes; run `qmd embed` once to pick up the new chunking.
+
+- `search`, `vsearch` and `query` take `--path`, `--since` and `--until`, so a
+  search can be narrowed to part of a collection or to what changed recently.
+  `--path` matches globs against `collection/path` and is repeatable, with a
+  `!` prefix to exclude; `--since` / `--until` take a span (`7d`) or a date.
+  The filter narrows the corpus rather than the result list — excluded
+  documents are gone before ranking, on the vector path as well as BM25, so a
+  narrow filter returns its own best matches instead of whatever survived the
+  global top-K. A span that does not parse is an error, and a filter that
+  leaves nothing to search says so instead of looking like "no match".
+
+- `qmd query --explain` now lists every expansion sub-query that placed the
+  document, with that sub-query's own rank, contribution, backend score and the
+  text it ran. It previously showed the top three contributions collapsed onto
+  one line without the query text, which could not distinguish a document that
+  ranked first in one sub-query and nowhere else from one that ranked middling
+  everywhere — two situations that call for different fixes.
+
+- `qmd grep <pattern>` matches document bodies exactly, by string or regular
+  expression, with no ranking and no LLM: every matching line, grouped by file,
+  with line numbers that `qmd get <file>:<n>:<count>` accepts. It is the escape
+  hatch for a query that finds nothing because tokenization never produced the
+  term the document contains — the failure this fork keeps hitting with Korean
+  text — and for finding an identifier or error string verbatim. It searches
+  exactly what the index holds and honours collection scoping, `--path`,
+  `--since` and `--until`. Smart case: an uppercase letter in the pattern makes
+  it case-sensitive, and Hangul has no case so Korean patterns are unaffected.
+  `-F` searches for a literal string. An MCP `grep` tool exposes the same thing
+  to agents, which is the reason it exists: a client whose only tools are
+  `query`, `get` and `multi_get` has no other way to match a string exactly.
+
+- The MCP `query` tool and the REST `/query` endpoint take the same `path`,
+  `since` and `until` parameters, so an agent can scope a search to a folder or
+  to recent changes without shelling out. A span that does not parse comes back
+  as an error rather than an unfiltered search, and an empty result under a
+  filter reports how many documents the filter left — an agent reading a bare
+  "No results" would otherwise conclude the corpus has nothing on the topic.
+
+- The HTTP and stdio MCP daemon keeps the text index current on its own. Before
+  each search it checks the collections that search covers — at most once every
+  30 seconds each — and re-indexes the ones whose files changed, so a note
+  written while the daemon runs is findable on the next query without `qmd
+  update`. It updates keyword search only; vectors still wait for `qmd embed`.
+  It does not run a collection's `update:` hook, does not clear the LLM cache
+  the way `qmd update` does, and does not touch a collection whose folder has
+  vanished or emptied, since reindexing would deactivate every document in it.
+
+- The daemon's query log now covers the MCP `query` tool, not just the REST
+  endpoints. A search an agent runs through MCP over HTTP lands in the same
+  `queries-YYYY-MM.jsonl` with `via: "mcp"`, the same result path spelling and
+  the same `X-QMD-*` header handling, so a dogfooding session's real traffic is
+  in one file. A plain `query` argument is recorded as one search of type
+  `auto`. stdio MCP calls stay unlogged — they carry no request to annotate.
+
 ## [2.8.3-ko.3] - 2026-09-18
 
 The reranker now decides which document ranks first. The blend's retrieval-position

package/README.ko.md

@@ -11,13 +11,15 @@
 1. 패키징: 이름·repository·플러그인 marketplace owner, `publish.yml` 은 `ko-qmd` 로 발행, CI에 `windows-latest`와 Electron 스모크 잡 추가
 2. Hangul FTS (`src/hangul.ts`, `store.ts` 접점 2곳)
    - 질의: 따옴표 없는 한글 단어는 끝 조사 하나 또는 `기`를 뗀 어간도 매칭한다(`검색을` → `검색`, 어간 2음절 이상). 따옴표 구문·한자·가나는 그대로.
+   - 외래어 다리: 기술 용어의 한글 외래어 표기(`서치`·`웹`·`코어`·`마이그레이션` 등 약 80개, `hangul.ts` 내장 표)는 영문 표기를 OR 로 함께 찾는다(`하이브리드 서치` → `hybrid search`). 어간에도 적용(`서치를` → `search`), 영문과 붙은 한글 구간도(`qmd서치`). 붙여 쓴 한글 복합어(`레몬웹코어`)와 1음절 항목에 조사가 붙은 꼴(`웹을`)은 대상 아님. 질의 쪽만 바뀌므로 재색인 불필요.
    - 색인: 한글 구간의 음절 bigram을 필드 끝에 덧붙인다. `FTS_CJK_NORMALIZED_VERSION`이 `"2"`라 기존 인덱스는 처음 열 때 FTS를 한 번 다시 만든다.
 3. 한국어 기본값
    - 기본 임베딩 모델: `hf:Qwen/Qwen3-Embedding-0.6B-GGUF/Qwen3-Embedding-0.6B-Q8_0.gguf`(업스트림은 embeddinggemma-300M). [README.md](README.md)의 모델 표·"Custom Embedding Model"·"Model Configuration" 절도 이 기본값 기준으로 고쳐 뒀다. 리랭커·질의 확장 모델은 업스트림 기본값 그대로.
    - 스킬 `skills/qmd/SKILL.md`에 "Korean queries" 절(`lex:`에 어간·alias·영문 용어, `vec:`에 한국어 패러프레이즈).
 
-4. 데몬 질의 로그 (`src/query-log.ts`, `server.ts` 접점 = REST 핸들러·`status` 툴): `QMD_QUERY_LOG=1` 로 띄운 HTTP 데몬이 REST 검색마다 `queries-YYYY-MM.jsonl` 에 한 줄을 남긴다(결과 경로·점수, 스니펫 없음). 헤더 `X-QMD-Tag`·`X-QMD-Qid`·`X-QMD-Role`·`X-QMD-No-Log`. 계약은 [README.md § Query log](README.md#query-log-ko-qmd). 업스트림 PR 대상 아님.
+4. 데몬 질의 로그 (`src/query-log.ts`, `server.ts` 접점 = REST 핸들러·MCP `query` 툴·`status` 툴): `QMD_QUERY_LOG=1` 로 띄운 HTTP 데몬이 REST 검색과 HTTP MCP `query` 툴 호출마다 `queries-YYYY-MM.jsonl` 에 한 줄을 남긴다(stdio MCP 는 남기지 않는다)(결과 경로·점수, 스니펫 없음). `rerank:false` 응답·로그 행에는 결과마다 원시 점수 `vec_score`(코사인 유사도)·`fts_score`(정규화 BM25)가 붙는다 — 그 경로의 `score`는 1/순위라 임계값을 걸 수 없다. 헤더 `X-QMD-Tag`·`X-QMD-Qid`·`X-QMD-Role`·`X-QMD-No-Log`. 계약은 [README.md § Query log](README.md#query-log-ko-qmd). 업스트림 PR 대상 아님.
 5. 모델 idle 타임아웃 env (`src/llm.ts` `LlamaCpp` 생성자): `QMD_LLM_IDLE_TIMEOUT_MS` 가 `createStore()` 의 5분 고정값보다 우선한다. `0` 이면 모델을 내리지 않는다 — 상주 데몬의 첫 검색이 모델을 다시 올리지 않게 하는 용도다.
+6. 경로·시간 범위 필터 (`src/filters.ts`): CLI `--path`·`--since`·`--until`, REST·MCP `path`·`since`·`until`. 걸러진 코퍼스 안에서 순위를 매긴다(전역 top-K 뒤 후처리가 아님). SDK 옵션 이름은 `scope`다 — `filter`는 업스트림의 metadata 필터(`--filter <json>`)가 쓰고, 둘을 같이 주면 AND. ko-qmd 에서는 metadata 필터도 같은 전체 스캔·정확 스캔 경로를 타므로, 업스트림 README 의 "선택적 필터는 best-effort" 한계가 적용되지 않는다.
 
 ### 기본 임베딩 변경 후 재임베딩
 
@@ -44,7 +46,9 @@ bash scripts/dogfood.sh --check "RESULT bm25_r5=…"   # 게이트 판정만
 
 ### ko-vault 벤치
 
-`bash scripts/bench-ko.sh` (픽스처 `test/fixtures/ko-vault/`, 문서 28·질의 36, 임베딩은 픽스처 `models.yml`의 embeddinggemma로 고정). 표와 run별 수치는 [BASELINE.md](test/fixtures/ko-vault/BASELINE.md).
+`bash scripts/bench-ko.sh` (픽스처 `test/fixtures/ko-vault/`, 문서 28·질의 63, 임베딩은 픽스처 `models.yml`의 Qwen3-Embedding-0.6B-Q8_0으로 고정 — 패키지 기본값과 같다). run별 수치는 [BASELINE.md](test/fixtures/ko-vault/BASELINE.md).
+
+아래 표는 질의 36·embeddinggemma 시절의 단계별 기록이다.
 
 | 단계 | bm25_r5 | vector_r5 | full_r5 |
 |---|---|---|---|
@@ -52,7 +56,7 @@ bash scripts/dogfood.sh --check "RESULT bm25_r5=…"   # 게이트 판정만
 | + 질의 조사 처리 (B2) | 0.6528 | 0.9861 | 1.0000 |
 | + bigram 색인 (B3) | 0.6528 | 0.9861 | 1.0000 |
 
-Qwen3-Embedding 기본값의 벤치 수치는 미측정이다(design §6 라운드 M1).
+Qwen3-Embedding 기본값에서 질의 52는 bm25_r5 0.9519 · vector_r5 1.0000 · full_r5 1.0000이다. 질의 63은 외래어 표기 질의 11건(`하이브리드 서치` → `hybrid-search.md`)을 더한 것으로, 그 11건의 bm25_r5는 BASELINE.md 2026-09-23 절에 따로 적는다.
 
 ## 설치
 
@@ -91,13 +95,20 @@ Windows CI 결과·리빌드 소요 시간은 `docs/vault-search/WINDOWS.md` 에
 
 ## 업스트림 동기화
 
+`develop` 에서 `chore/upstream-sync-*` 브랜치를 따고, 업스트림의 first-parent 머지 지점을 하나씩 `--no-ff` 로 머지한다. 대상은 태그든 `main` 스냅샷이든 된다.
+
 ```sh
-git fetch upstream
-git rebase upstream/<tag>   # on ko
-npm run test:unit && bash scripts/bench-ko.sh
+git fetch upstream --tags
+git log --first-parent --oneline develop..upstream/main   # 머지할 지점 목록
+bash scripts/bench-ko.sh                                  # 머지 전 기준값 (tmp/bench-ko/bench.json 보관)
+git merge --no-ff <지점>                                   # 지점마다 반복, 충돌은 이 머지 커밋에서 해결
+bun run lint && bun run test:types && bun run test:unit && bash scripts/bench-ko.sh
 ```
 
-통과하면 `v<업스트림 버전>-ko.N` 태그를 push한다.
+- 벤치는 전체 수치가 아니라 bm25 recall@5·MRR 을 질의별로 머지 전과 비교한다(hybrid 는 실행마다 한 질의쯤 흔들린다 — BASELINE.md 2026-09-23).
+- PR 은 squash 가 아니라 **머지 커밋**으로 들인다. 업스트림 커밋이 조상으로 남아야 다음 동기화가 새 커밋만 본다. 같은 이유로 이 브랜치는 squash·rebase 하지 않는다.
+- `develop`·`main` 을 upstream 위로 rebase 하지 않는다(보호 브랜치 force push).
+- 업스트림 태그 위에 있을 때만 `v<업스트림 버전>-ko.N` 태그를 쓴다. `main` 스냅샷이면 버전은 `2.8.3-ko.N` 식으로 마지막 태그 기준이다.
 
 ## 라이선스
 

package/README.md

@@ -158,21 +158,41 @@ runs in a container and a liveness probe connects from a non-loopback address.
 
 The HTTP server exposes two endpoints:
 - `POST /mcp` — MCP Streamable HTTP (JSON responses, stateless)
-- `POST /query` (alias `/search`) — structured search without the MCP protocol
+- `POST /query` (alias `/search`) — structured search without the MCP protocol. Accepts the same optional `filter` object as the `query` tool (invalid filters return `400`); see [Metadata Filtering](#metadata-filtering)
 - `GET /health` — liveness check with uptime
 
+##### Index refresh (ko-qmd)
+
+Before each search, the daemon checks the collections that search covers and
+re-indexes the ones that changed on disk, so a note written a minute ago is
+findable without running `qmd update`. Each collection is checked at most once
+every 30 seconds. The refresh updates the text index only: a new or edited note
+is found by keyword search right away and by vector search after the next
+`qmd embed`. It never runs a collection's `update:` command, never clears the
+LLM cache, and leaves a collection alone when its folder has vanished or turned
+up empty — an unmounted drive is not a reason to drop its documents. Collections
+outside the project of a project-local `.qmd` config are never refreshed.
+CLI searches do not refresh; run `qmd update` there.
+
 ##### Query log (ko-qmd)
 
 Start the daemon with `QMD_QUERY_LOG=1` (`true`/`yes` also work; anything else
-is off) and every search served by `POST /query` or `/search` is appended as
-one JSON line to `$XDG_CACHE_HOME/qmd/queries-YYYY-MM.jsonl` (default
-`~/.cache/qmd`, one file per local month, mode `0600`, never pruned). CLI
-searches and MCP tool calls are not logged.
+is off) and every search it serves — `POST /query`, `/search`, and the MCP
+`query` tool over HTTP — is appended as one JSON line to
+`$XDG_CACHE_HOME/qmd/queries-YYYY-MM.jsonl` (default `~/.cache/qmd`, one file
+per local month, mode `0600`, never pruned). CLI searches are not logged, and
+neither is an MCP call over stdio: it carries no request to annotate.
 
 A row records the qmd version and commit, an index fingerprint (document count
-and last update), the searches, collections, limit, rerank flag, each result's
+and last update), `via` (`rest` or `mcp`), the searches (a plain MCP `query`
+argument is recorded as one search of type `auto`), collections, limit, rerank
+flag, each result's
 `<collection>/<path>`, score and rank, and the elapsed milliseconds — no
-snippets or document text. Clients can annotate rows with request headers:
+snippets or document text. On `rerank:false` rows each result also carries the
+raw backend scores the response returned, `vec_score` (cosine similarity) and
+`fts_score` (normalized BM25), since `score` there is only the 1/rank fusion
+position; each is absent, never 0, when that backend did not return the hit,
+and the row schema stays `v: 1`. Clients can annotate rows with request headers:
 
 | Header | Row field |
 |--------|-----------|
@@ -218,11 +238,15 @@ Point any MCP client at `http://localhost:8181/mcp` to connect.
 |------|-----------|------|-------|
 | `query` | `searches` | array | Typed sub-queries (`lex`/`vec`/`hyde`), 1–10. **Required.** First gets 2x weight. |
 | `query` | `collections` | string[] | Filter by collection names (OR). **Array only** — singular `collection` is silently ignored. |
+| `query` | `filter` | object | Metadata filter (recursive `operator`-discriminated JSON AST; see [Metadata Filtering](#metadata-filtering)) |
 | `query` | `intent` | string | Disambiguation context (does not search on its own) |
 | `query` | `limit` | number | Max results (default 10) |
 | `query` | `minScore` | number | Minimum relevance 0–1 (default 0) |
 | `query` | `candidateLimit` | number | Max candidates to rerank (default 40) |
 | `query` | `rerank` | boolean | Run LLM reranking (default **true**); set false for RRF-only |
+| `query` | `path` | string[] | ko-qmd: globs against `collection/path`; `!` prefix excludes |
+| `query` | `since` | string | ko-qmd: span (`7d`) or date — documents modified at or after |
+| `query` | `until` | string | ko-qmd: same formats — documents modified at or before |
 | `get` | `file` | string | Path, docid (`#abc123`), or `path:from:count` (e.g. `#abc123:120:40`) |
 | `get` | `fromLine` | number | Start line (1-indexed); overrides the `:from` suffix |
 | `get` | `maxLines` | number | Limit returned lines |
@@ -231,11 +255,27 @@ Point any MCP client at `http://localhost:8181/mcp` to connect.
 | `multi_get` | `maxBytes` | number | Skip files larger than N (default 10240) |
 | `multi_get` | `maxLines` | number | Limit lines per file |
 | `multi_get` | `lineNumbers` | boolean | Prefix lines with numbers (default **true**) |
+| `grep` | `pattern` | string | ko-qmd: regex, or a literal when `fixedString` is set. **Required.** |
+| `grep` | `collections` | string[] | Filter by collection names (OR) |
+| `grep` | `path` / `since` / `until` | — | Same as on `query` |
+| `grep` | `limit` | number | Max files (default 20) |
+| `grep` | `maxLinesPerFile` | number | Max reported lines per file (default 20) |
+| `grep` | `caseSensitive` | boolean | Override smart case |
+| `grep` | `fixedString` | boolean | Treat the pattern as a literal string |
 
 Unknown parameters are silently ignored (not rejected) — double-check names if
 results seem unscoped. The HTTP `/query` and `/search` endpoints return
 `qmd://collection/path` URIs in the `file` field, matching the CLI and MCP output.
 
+`path`, `since` and `until` are the MCP side of the CLI's filtering flags —
+see [Path and Time Filtering](#path-and-time-filtering-ko-qmd) for their
+semantics. They narrow the corpus before ranking, so a narrow filter returns
+its own best matches rather than whatever survived the global top-K. A `since`
+or `until` that does not parse is an error (a tool error on MCP, HTTP 400 on
+`/query`) rather than a dropped filter, and an empty result under a filter
+reports how much corpus the filter left so an agent can tell "nothing matched"
+from "nothing to match against". The REST endpoints accept the same three.
+
 ### SDK / Library Usage
 
 Use QMD as a library in your own Node.js or Bun applications.
@@ -323,6 +363,20 @@ const results3 = await store.search({
 
 // Skip reranking for faster results
 const fast = await store.search({ query: "auth", rerank: false })
+
+// Metadata filter — every returned result satisfies it (also available on
+// searchLex() and searchVector()); results expose indexed metadata via
+// r.metadata. See "Metadata Filtering" for the full grammar.
+const published = await store.search({
+  query: "authentication flow",
+  filter: {
+    operator: "and",
+    operands: [
+      { key: "topics", operator: "all", value: ["typescript"] },
+      { key: "status", operator: "ne", value: "draft" },
+    ],
+  },
+})
 ```
 
 For direct backend access:
@@ -891,8 +945,13 @@ and `deep-search` (→ `query`).
 --full             # Show full document content
 --line-numbers     # Add line numbers to output
 --explain          # Include retrieval score traces (query, JSON/CLI output)
+                   # ko-qmd: also lists each sub-query's own rank for the result
+--filter <json>    # Metadata filter (recursive JSON AST; see Metadata Filtering)
 --index <name>     # Use named index
 --intent "<text>"  # Disambiguation context (e.g. "web page load times")
+--path <glob>      # ko-qmd: only documents matching the glob (repeatable)
+--since <when>     # ko-qmd: only documents modified at or after this point
+--until <when>     # ko-qmd: only documents modified at or before this point
 --no-rerank        # Skip LLM reranking (RRF scores only; faster on CPU)
 -C, --candidate-limit <n>  # Max candidates to rerank (default: 40)
 --full-path        # Emit on-disk filesystem paths instead of qmd:// URIs
@@ -934,6 +993,175 @@ explicitly with `-c`.
 > then filtered. If one collection dominates the rankings, matches from smaller
 > collections may not appear at the default limit — raise `-n` or use `--all`.
 
+### Reading `--explain` (ko-qmd)
+
+`qmd query <q> --explain` prints, under the fused score, one line per expansion
+sub-query that placed the document — with that sub-query's own rank, its RRF
+contribution, its backend score and the text it actually ran:
+
+```
+Explain: fts=[0.6700, 0.5100] vec=[0.4400]
+  RRF: total=0.0889 base=0.0389 bonus=0.0500 rank=1
+  Blend: 75%*1.0000 + 25%*0.8200 = 0.9550
+  Sub-query ranks (3 of the expansion's lists placed this document):
+    fts/original   #1  rrf=0.0328 backend=0.6700  한글 토큰화
+    vec/hyde       #2  rrf=0.0161 backend=0.4400  형태소 분석기는 어절을 나눌 때...
+    fts/lex        #9  rrf=0.0061 backend=0.5100  토큰화 형태소
+```
+
+Expansion turns one query into a dozen and reports one number. That number
+cannot distinguish "first in the hyde list and nowhere else" from "middling
+everywhere", and those two call for different fixes — the first says the
+expansion invented a good query, the second says the document is broadly
+relevant. Previously only the top three contributions were shown, collapsed
+onto one line without the sub-query text.
+
+### `qmd grep` (ko-qmd)
+
+```sh
+qmd grep 'TODO'                          # smart case: lowercase matches either
+qmd grep 'TODO: (call|ship)'             # a regular expression
+qmd grep 'v2.8.3-ko' -F                  # a literal string, regex chars and all
+qmd grep '놓친 문자열' --path 'journals/**'
+```
+
+Exact string and regular-expression matching over indexed document bodies: no
+ranking, no LLM, every matching line grouped by file with line numbers that
+`qmd get <file>:<n>:<count>` accepts.
+
+This is the escape hatch for a query that finds nothing because tokenization
+never produced the term the document contains — the failure Korean text keeps
+hitting — and the way to find an identifier, error string or config key
+verbatim. Ranked search answers "what is this about"; grep answers "where does
+this string appear".
+
+It is not a replacement for ripgrep. At a terminal with a shell, ripgrep is
+usually the right tool and faster. `qmd grep` earns its place for the caller
+with no shell — an MCP client whose only tools are `query`, `get` and
+`multi_get` — and for the caller who wants one corpus definition: it searches
+exactly what the index holds, honours collection exclusion and the `--path` /
+`--since` / `--until` filters, and returns `qmd://` paths and docids the rest
+of qmd accepts.
+
+Smart case: a pattern containing an uppercase letter matches case-sensitively,
+otherwise either case. Hangul has no case, so Korean patterns are unaffected
+either way. `-i` and `-S` override it, `-F` treats the pattern as a literal
+string, `--max-lines` caps reported lines per file, and `-n` caps files.
+
+Patterns are capped at 1000 characters and matched line by line, with long
+lines tested in overlapping slices. That bound is what keeps a pathological
+pattern from hanging the daemon: a catastrophic backtrack blocks the only
+thread, so a timeout could never fire to stop it.
+
+### Path and Time Filtering (ko-qmd)
+
+`--path`, `--since` and `--until` narrow the corpus a search runs against, on
+`search`, `vsearch` and `query` alike. They are not a filter over the results:
+the documents they exclude are gone before anything is ranked, so a narrow
+filter returns its own best matches rather than whatever survived the global
+top-K.
+
+```sh
+qmd query "retry policy" --since 7d                  # touched in the last week
+qmd query "retry policy" --since 2026-09-01          # or an explicit date
+qmd search "auth" --path 'notes/journals/**'         # only under this path
+qmd search "auth" --path 'notes/**' --path '!notes/archive/**'
+```
+
+`--path` matches globs against `collection/path` — the same string results
+print — and is repeatable. A `!` prefix excludes, and an exclude beats an
+include. A bare directory (`--path notes/journals`) means everything under it.
+
+`--since` / `--until` take either a span back from now (`30m`, `3h`, `7d`,
+`2w`, `6mo`, `1y`) or a date (`2026-09-01`, or a full ISO timestamp). A bare
+date starts at that local day. Anything else is an error rather than a silently
+ignored filter, because a search that quietly dropped your filter returns a
+full-corpus answer that looks exactly like a narrow one.
+
+They compare against the index's `modified_at`, which is when the document was
+last indexed with changed content — not the file's mtime. The daemon's index
+refresh keeps those within 30 seconds of each other; for CLI searches, `--since`
+is only as current as your last `qmd update`.
+
+When a filter leaves nothing to search, the empty result says so rather than
+looking like "no match":
+
+```
+No results found — path 'notes/nowhere/**' matched 0 of 232 documents.
+No results found in the 4 of 232 documents matching since 2026-09-13T12:00:00.000Z.
+```
+
+These flags combine with `--filter` (Metadata Filtering, below): a result has to
+pass both. In the SDK the path/time narrowing is the `scope` option (build it
+with `buildDocumentFilter`), because `filter` is upstream's metadata filter; the
+REST and MCP `path` / `since` / `until` parameters are unchanged.
+
+### Metadata Filtering
+
+Documents can opt into typed metadata through a namespaced frontmatter block. A document without `qmd.metadata` behaves exactly as before, and the frontmatter stays ordinary searchable content (no chunking, embedding, or line-number changes):
+
+```markdown
+---
+qmd:
+  metadata:
+    topics:
+      - typescript
+      - programming
+    status: published
+    priority: 3
+    reviewed: true
+---
+
+# Document body starts here
+```
+
+Supported values are strings, numbers, booleans, and flat homogeneous arrays of one of those. Nested objects, nulls, empty arrays, and mixed-type arrays are rejected (the document still indexes; it is excluded from filtered search until corrected). Metadata keys are user-defined data — `tags`, `topics`, and `labels` are all ordinary keys with no special semantics.
+
+Every search surface (CLI, SDK, MCP, HTTP) accepts the same recursive filter, a JSON AST discriminated by `operator`:
+
+```sh
+# One condition
+qmd search "authentication" \
+  --filter '{"key":"status","operator":"eq","value":"published"}'
+
+# Composed conditions — works with search, vsearch, and query
+qmd query "dependency injection" --filter '{
+  "operator": "and",
+  "operands": [
+    { "key": "topics", "operator": "all", "value": ["typescript", "programming"] },
+    { "key": "status", "operator": "nin", "value": ["draft", "archived"] },
+    { "operator": "or", "operands": [
+      { "key": "priority", "operator": "gte", "value": 3 },
+      { "key": "reviewed", "operator": "eq", "value": true }
+    ] },
+    { "operator": "not", "operand": { "key": "audience", "operator": "eq", "value": "internal" } }
+  ]
+}'
+```
+
+| Node | Shape |
+|------|-------|
+| Logical group | `{ "operator": "and" \| "or", "operands": […] }` |
+| Negation | `{ "operator": "not", "operand": {…} }` |
+| Comparison | `{ "key", "operator": "eq" \| "ne" \| "gt" \| "gte" \| "lt" \| "lte", "value" }` |
+| Membership | `{ "key", "operator": "in" \| "nin" \| "all", "value": […] }` |
+| Presence | `{ "key", "operator": "exists", "value": true \| false }` |
+
+Semantics:
+
+- Matching is typed and exact — no string/number/boolean coercion, and a type mismatch never matches (including `ne` and `nin`).
+- Array-valued metadata is a set: a condition matches when any element satisfies it, `all` requires every filter value to be present.
+- Missing keys do not match `ne`/`nin`; combine with `{ "operator": "exists", "value": false }` in an `or` group to include them.
+- Multiple conditions require an explicit `and` group — there is no implicit AND, and no `$`-prefixed shorthand.
+
+Guarantees and limits:
+
+- Every returned result satisfies the filter, before RRF fusion and reranking.
+- Like collection filtering, highly selective filters are best-effort for top-K completeness: backends over-fetch and post-filter, so a very selective filter can return fewer than `limit` results. (ko-qmd: a metadata filter takes the same full-scan / exact-scan path as `--path`, so a selective filter still returns its own best matches.)
+- Filtered search only considers documents whose metadata has been extracted (run `qmd update` after upgrading; `qmd status` shows the pending count).
+
+JSON output (`--format json`), the SDK, MCP structured results, and the HTTP endpoints include each result's indexed metadata.
+
 ### Output Format
 
 Default output is colorized CLI format (respects `NO_COLOR` env).
@@ -1180,7 +1408,7 @@ llm_cache       -- Cached LLM responses (query expansion, rerank scores)
 | `QMD_CONFIG_DIR` | unset | Override the config directory outright (takes precedence over `XDG_CONFIG_HOME`) |
 | `QMD_LLAMA_GPU` | `auto` | Force llama.cpp GPU backend (`metal`, `vulkan`, `cuda`) or disable GPU with `false` |
 | `QMD_FORCE_CPU` | unset | Set to `1`/`true` to force CPU mode before any CUDA/Vulkan/Metal probing. Equivalent CLI flag: `--no-gpu`. |
-| `QMD_QUERY_LOG` | unset | ko-qmd: set to `1`/`true`/`yes` on the HTTP daemon to append each search to `$XDG_CACHE_HOME/qmd/queries-YYYY-MM.jsonl` (see [Query log](#query-log-ko-qmd)) |
+| `QMD_QUERY_LOG` | unset | ko-qmd: set to `1`/`true`/`yes` on the HTTP daemon to append each REST or MCP search to `$XDG_CACHE_HOME/qmd/queries-YYYY-MM.jsonl` (see [Query log](#query-log-ko-qmd)) |
 | `QMD_LLM_IDLE_TIMEOUT_MS` | `300000` | ko-qmd: idle time in ms before loaded models and contexts are unloaded; `0` never unloads. Overrides the SDK's `createStore()` timeout, so a long-running HTTP daemon can skip the model reload on its first search after a quiet spell, at the cost of keeping that memory in use. |
 | `QMD_EMBED_PARALLELISM` | automatic | Override embedding/reranking context parallelism (1-8). Windows CUDA defaults to `1` because parallel CUDA contexts can crash with `ggml-cuda.cu:98`; use Vulkan or raise this only if your driver is stable. |
 

package/dist/cli/build-info.json

@@ -1,4 +1,4 @@
 {
-  "commit": "d4da081",
-  "builtAt": "2026-09-18T03:59:04.070Z"
+  "commit": "bec590b",
+  "builtAt": "2026-09-24T04:13:20.171Z"
 }

package/dist/cli/qmd.d.ts

@@ -1,3 +1,4 @@
+import { type RRFContributionTrace } from "../store.js";
 import { type OutputFormat } from "./formatter.js";
 type CliLifecycleWritable = {
     write(chunk: string | Uint8Array, callback?: (error?: Error | null) => void): boolean;
@@ -38,6 +39,15 @@ export declare function finishSuccessfulCliCommand(options: FinishSuccessfulCliC
 export declare function resolveEmbedModelForCli(): string;
 export declare function resolveGenerateModelForCli(): string;
 export declare function resolveRerankModelForCli(): string;
+/**
+ * Which sub-query found this document, and where did it place in that sub-query's own list?
+ *
+ * Expansion turns one query into a dozen, fuses their lists and reports one number. A single
+ * fused rank cannot say whether a result placed first in the hyde list and nowhere else, or
+ * middling everywhere — and those call for different fixes. Each expansion gets a line, with
+ * the sub-query text it ran, so the fused score can be read back to the query that earned it.
+ */
+export declare function explainGroupLines(contributions: readonly RRFContributionTrace[]): string[];
 export declare function buildEditorUri(template: string, absolutePath: string, line: number, col: number): string;
 export declare function termLink(text: string, url: string, isTTY?: boolean): string;
 export {};