okf 1.8.0 → 1.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 6c6e09a9ce22d65f6ffe880b9e8ad3925bb648e4c985e2249ad13a83c17cc620
4
- data.tar.gz: 177086b26883d9004852181dbc5f935f5112ad2b5fde3277fb29a5db766b650a
3
+ metadata.gz: 24f81d695477c347d78e44d89131ebdb7c5eec4f380c2f8ba7596292bf043e39
4
+ data.tar.gz: 14bc57ccda962eb9abfbcb9906e95c74b9fb808d2e46850dbfec234aafb933f7
5
5
  SHA512:
6
- metadata.gz: d39fa2b520b3c33e76dd3c03111da96a7668809101c035e494ad47df31f993bf661ed3bb5aa0d3b3e9cba81b5087ef95a4398082954033949a20a91ff152feb5
7
- data.tar.gz: 6109c082b1d89144bd6f60ddedf93a70af41ca23914d5cf9cc243acad3b418211317c988cd2e28fa23632187c79ab629b1b6473a8d6f0d711700db4736c3898f
6
+ metadata.gz: d746db4e46fd17179f8eb82275749a1a2d1077c58e0d892d2e694dcd15f052e40dfe5526e17656ba2b21adb1fc63d1fa4885d06dce44c9edb584028445703444
7
+ data.tar.gz: 5f41214c5b0829c6d50a99cde236d5b768256d062ea6a6faa13270bc5ab3c2416fc6ff562be34c5c7067157da36fbd91f547db4120950dc2e9a8b6376a551f8c
data/CHANGELOG.md CHANGED
@@ -1,6 +1,223 @@
1
1
  # Changelog
2
2
 
3
- ## [Unreleased]
3
+ ## [1.9.0] - 2026-07-19
4
+
5
+ - **`okf search` gains an opt-in full-text index engine.** `--engine index` — and
6
+ `--fuzzy`, which implies it — routes to
7
+ [minifts](https://github.com/serradura/minifts), the pure-Ruby port of the same
8
+ MiniSearch build the graph page loads. It is the gem's third runtime
9
+ dependency, admitted because it costs the footprint nothing the first two were
10
+ chosen to protect: no native extension, no dependency tree of its own, the same
11
+ Ruby 2.4 floor. Three things it adds, and nothing else does:
12
+ - **BM25+ relevance ranking**, where the default scores by summed field weight;
13
+ - **`--fuzzy`** — typo tolerance at edit distance `0.2 × term length`, the
14
+ browser's own setting. Search stays exact unless you ask;
15
+ - **parity with the graph page**, which runs the same MiniSearch build, so the
16
+ two rank identically when the index is named.
17
+ - **The default search is unchanged** — literal, case-insensitive substring
18
+ matching over the same fields with the same weights as 1.8.0. The index is
19
+ opt-in rather than default because a one-shot CLI builds an index, asks one
20
+ question, and exits: end to end, **3.00 s against 0.24 s at 1,000 concepts**,
21
+ the build accounting for ~95% of that. The ~44–56× per-query throughput that
22
+ recommends minifts is the right measure for a long-lived index — a page, a
23
+ server — and the wrong one for a process that exits. A cached prebuilt index is
24
+ what would change that arithmetic.
25
+ - **Know what the index costs before naming it.** Its tokenizer splits on
26
+ punctuation, so `customer_id` becomes `customer` + `id` and `7.2.0` becomes
27
+ `7`, `2`, `0`; an infix (`ustomer`) finds nothing; and a backtick is Unicode
28
+ `Sk` rather than punctuation, so a word inside a code span indexes as
29
+ `` `minifts` `` and the query `minifts` does not match it — 409 such tokens
30
+ on this repo's own bundle. Ranking does not rescue it: BM25 normalizes by
31
+ field length, so a short concept dense in `7`, `2` and `0` can outrank the one
32
+ that actually says `7.2.0`. The default has none of these, because raw-text
33
+ matching has no tokenizer.
34
+ - **`--engine NAME` picks the engine outright**, for the case a capability flag
35
+ cannot express: a matching *model* requires nothing, so no flag selects one.
36
+ Naming an engine that cannot do what was also asked is a usage error naming one
37
+ that can (`--engine index -e` → *try --engine scan*), and an unknown name lists
38
+ what is available. `--help` reads the registry, so an addon's engine appears
39
+ without the CLI knowing it exists.
40
+ - **Search engines are adapters.** `OKF::Bundle::Search` became a facade over N
41
+ engines instead of one class with a `regexp ? scan : index` branch. The facade
42
+ keeps everything that defines a result — documents, the row and its key order,
43
+ the snippet, the sort — and an engine answers only which documents match, how
44
+ well, and where. The built-ins are `Search::Scan` (raw text, the default,
45
+ `regexp`) and `Search::Index` (minifts, `fuzzy`/`prefix`).
46
+ - **Selection is by capability when the query requires one** — `--fuzzy`
47
+ requires `:fuzzy`, so it routes to the index without naming it — and that
48
+ routing prints **nothing**: no note, no header change, no new JSON key.
49
+ - **`Search.register` is a published extension point** — append-only,
50
+ idempotent by id, capabilities checked against a fixed vocabulary. This is
51
+ the seam a future SQLite/FTS5 addon plugs into; no addon code ships here.
52
+ - **A shared conformance suite replaces the "kernel is the oracle" rule**,
53
+ which multiple engines made impossible: the index and the scan disagree about
54
+ match sets by design, so neither can be the oracle. Every registered engine
55
+ runs the same contract, with capability-gated blocks for its own semantics,
56
+ and a registered engine with no conformance class fails the suite.
57
+ - **Cross-bundle search ranks one corpus under `--engine index`.** BM25 prices a
58
+ term by how rare it is, so ranking each bundle separately and interleaving the
59
+ lists would produce a ranking that looks sorted and compares nothing; the
60
+ searched bundles are indexed together instead. The visible consequence, under
61
+ that engine only: a score is relative to the whole answer, so the same concept
62
+ scores lower searched beside other bundles than alone. The default's scores are
63
+ absolute and need no such treatment.
64
+
65
+ - **The graph can draw the index layer, under any layout.** The §6 map was
66
+ visible only inside file-tree mode, where a folder node stood in for a
67
+ directory's `index.md`. **Show indexes** makes it a layer: each map is a tile
68
+ edged to the concepts it lists and the maps below it, dressed by the same
69
+ selector as file-tree mode's folder node, because the two are the same thing
70
+ twice over — clicking either opens that directory's `index.md`. Both are accent
71
+ squares with dashed edges into them, so colour separates *kinds* rather than
72
+ modes: a directory is not a concept and no longer reads as one. Authorship shows
73
+ as form — solid where an author wrote a map, hollow and dashed where the bundle
74
+ only implies one — so the toggle reads as curation as much as navigation.
75
+ - **Moving between the modes lands in one click.** Tearing the layer down ran
76
+ its own layout while file-tree mode ran `breadthfirst` a beat later, two
77
+ layouts racing the same canvas; and because the layer is fetched, a promise
78
+ resolving after a mode change could land inside file-tree mode. A `relayout`
79
+ flag settles the first, a per-toggle ticket the second.
80
+ - **File-tree mode disables the toggle** rather than doubling the folders it
81
+ already draws.
82
+ - **One label on every file's graph button.** It read "Explore the knowledge
83
+ graph" on the root index and "Open core/ in graph" on a nested one, which made
84
+ a single action look like three. The question is the same whatever is open, so
85
+ the label is too — and it lives in the markup, where it cannot go stale.
86
+ - **Opening a map from the reader keeps the reader's graph.** It forced
87
+ file-tree mode, discarding whatever layout was running, and dimmed the canvas
88
+ to the map's immediate neighbours. It now switches the *layer* on rather than
89
+ the *mode* and leaves the layout alone. A reader already in file-tree mode
90
+ stays there.
91
+ - **Selecting anything emphasises it the same way.** A concept dimmed the graph
92
+ to its neighbourhood, a map did nothing at all, and a folder node did nothing
93
+ either — three meanings for one gesture. One `focusNode` now serves all three.
94
+ - **Drawn, never modelled.** `index.md` is reserved, so these nodes are built
95
+ from `/index` straight onto the canvas; `NODES`, `/catalog` and the type and
96
+ tag indexes never learn they exist. Filters pass them over — a map has no type
97
+ or tags — but a map whose concepts are all filtered away leaves with them.
98
+ - **Collapsing the root folds the file list away** on phones and tablets, where
99
+ the list is stacked on top of the reader and closing the root otherwise left a
100
+ single row above a column of nothing. Reopening the list undoes that collapse,
101
+ so it is one gesture rather than two states to dig out of — the fold remembers
102
+ *why* it happened, and a list folded because a file was opened comes back
103
+ exactly as it was left.
104
+ - **The bundle names its own root.** `(root)` and `/` are what a filesystem calls
105
+ it, not what a reader does. The tree's root row, file-tree mode's root node, the
106
+ index layer's root map and the inspector's directory map now all carry the name
107
+ the page header already shows, `--title` included. `areaOf` keeps its own
108
+ `(root)`: that is the area vocabulary `okf stats --by area` and `tags --by area`
109
+ print, not a UI label.
110
+ - **The Indexes tab dissolves into the file tree.** The authored layer lived on a
111
+ second tab as a flat list of paths, which put a directory's own map somewhere
112
+ other than the directory. `index.md` and `log.md` are rows now, at the top of
113
+ the folder they document, and **Indexes only** is a toggle over the same tree —
114
+ same rows, fewer of them, structure intact. The toggle yields only when it would
115
+ hide what was just opened — a map stays under it, a concept releases it — so
116
+ browsing the authored layer no longer destroys the list being browsed. A log
117
+ offers no graph button at all: it is a chronology, not a place in the graph, and
118
+ the button had been opening the root index's node.
119
+ Narrowed, a folder owns exactly one row, so the row stands where the folder
120
+ header stood — at that folder's depth, carrying the path — rather than nesting
121
+ a single child under a header.
122
+ - **The rail's Index becomes an action, not a fake view.** It had no
123
+ `#view-index` behind it — the files view showing its other tab — so
124
+ `activeRail()` answered a question of view *and* tab. The shortcut stays,
125
+ opening the root map through the same `readIndex()` the first-visit note
126
+ uses; `activeRail()` answers with the view it lands on, so Files highlights
127
+ and nothing invents a place for Index to be. `?view=index` resolves to the
128
+ same action.
129
+ - **Fixed on the way:** the reader header rendered empty — an unlabelled badge
130
+ and a graph button pointing nowhere — whenever no file was open, because
131
+ `.fp-head{display:flex}` outranks the UA sheet's `[hidden]{display:none}`.
132
+ - **A first-visit note tells a newcomer the index exists.** The `index.md` an
133
+ author wrote to be read first was reachable only by finding the Indexes tab and
134
+ clicking a row, so a reader meeting a bundle for the first time met unlabelled
135
+ dots with no way in. The page still opens on the graph — it is what makes a
136
+ bundle legible at a glance, at every width — and a dismissible note at the
137
+ bottom now says what the picture is, how to touch it, and where the index is.
138
+ **Read the index** goes straight there; the dismissal is remembered.
139
+ - **It absorbed the old mobile-only tip** rather than stacking a second banner
140
+ under it, and it is written for a finger throughout, since a phone is where a
141
+ first-time reader is least oriented.
142
+ - **The wording follows the device on two gates, not one.** What a reader does
143
+ follows `(pointer:coarse)` — a touch tablet in landscape is wider than 768px
144
+ and still taps; a narrow desktop window is narrower and still clicks. What a
145
+ reader can reach follows `(max-width:768px)`, because that is when the rail
146
+ collapses behind `☰`. Short viewports tighten; short *and* wide puts the
147
+ question beside the button, taking a landscape phone from half the screen to
148
+ under a third.
149
+ - **A second note points at `☰`** on compact layouts only, anchored under the
150
+ button it names rather than at the bottom of the screen. It fires on leaving
151
+ the graph by any route, so dismissing the first note does not cost it, and
152
+ opening `☰` answers it — but only once it is on screen, since `☰` is the only
153
+ way off the graph there and the first tap always comes first.
154
+ - **Deep links are unaffected**, and `?select=`/`#hash` now switch to the graph
155
+ before selecting, since the page can be standing elsewhere when they are read.
156
+ - **The file tree nests.** Directories were a sorted list of full paths, which
157
+ made `core` and `core/configurations` read as two unrelated folders and left
158
+ the shape of a bundle invisible. Each row is now one path segment indented by
159
+ depth, folders before files, and collapsing a folder takes its subtree with it.
160
+ A directory holding nothing but directories still renders, so the chain to its
161
+ children never breaks.
162
+ - **"Collapse all" folds into the root, not over it** — everything inside the
163
+ root closes and the root stays open, so the click leaves the top-level
164
+ folders standing instead of a single `(root)` row. Unfolding clears the whole
165
+ set, root included, so a root closed by hand is still reversible from there.
166
+
167
+ - The graph page's search box grows a full-text index. One MiniSearch index —
168
+ lazy-loaded from the CDN on first search, pinned to the `7.2.0` the Ruby
169
+ MiniSearch port tracks so an `okf search --engine index` result and the
170
+ browser's rank identically — now backs the graph, catalog and files views: ranked, multi-term
171
+ (`AND`), prefix (as-you-type) and typo-tolerant, over title, id, type, tags and
172
+ **description** — plus each concept's **body** wherever the page already holds
173
+ it (`okf render` bakes every body in, so a static file searches bodies offline;
174
+ the live server keeps bodies lazy, so its index stays metadata-only until a
175
+ backend body index arrives). The graph could not be searched by a leaf's
176
+ description before; now it can. The Files view's **Indexes** tab gets its own
177
+ full-text index too, over each `index.md`/`log.md`'s body — not just its
178
+ filename. Until an index loads — or if the CDN is unreachable — each view falls
179
+ back to its own substring filter, so the box is never dead.
180
+ - The Files tree's folder collapse works during a search. An active search or
181
+ type/tag filter used to force every folder open, so fold clicks did nothing;
182
+ folders now honor their collapsed state always (a collapsed group still shows
183
+ its header, so a match is never hidden). A **fold/unfold-all** control in the
184
+ Files tab header collapses or expands every visible group at once.
185
+ - Clustering no longer leaves phantom empty boxes. When a filter or a search hid
186
+ every concept in an area, the cluster's labelled box lingered as an empty
187
+ rectangle; the box now hides when no child survives and returns when one does —
188
+ the same rule the fit already used to leave stale boxes out of view, now
189
+ applied to what is drawn.
190
+ - `Esc` clears the graph selection. A dense graph leaves almost no empty canvas
191
+ to click for deselecting; `Esc` now drops the highlight (and lets the URL hash
192
+ forget the node) the same way tapping empty canvas does.
193
+ - A title-less concept now wears one name in every view. `catalog` and the §6
194
+ index listing fell back a concept with no `title` to its full id — `area/thing`
195
+ — while the graph node fell back blank-aware to the basename — `thing` — so the
196
+ same concept answered to two labels across two views of one bundle, and a
197
+ `title: ""` slipped past the nil-only `||` to catalog as an empty string. Both
198
+ now fall back the graph's way (`File.basename`, blank-aware), so the label is
199
+ the same wherever the concept appears.
200
+ - `okf render` stops baking a redundant description map. The static page derived
201
+ its `/node/meta` fragments from a separate `meta` payload that held nothing but
202
+ each concept's description, HTML-escaped — data the embedded `catalog` already
203
+ carries raw. The page now escapes the catalog's description on the client (the
204
+ same escape the server applies at `/node/meta`), so the `meta` key leaves the
205
+ baked payload and the description lives in one place. Both XSS guards are
206
+ unchanged; `okf server` is untouched.
207
+ - The bare not-a-directory error now teaches the registry grammar. A verb given
208
+ a target that is neither a directory nor an `@ref` moved from
209
+ `error: <arg> is not a directory` to
210
+ `… is not a directory or a registry ref (@slug names a registered bundle, @ the default; okf registry list)`,
211
+ so a consumer who typed a query or a bad path meets `@slug` addressing at the
212
+ error instead of hunting for it. (`@all` stays out of the message — it is
213
+ `search`'s alone, and the error seam is shared by every verb.)
214
+ - The bundled skill teaches `@slug` as a first-class target and stops probing for
215
+ the CLI. `SKILL.md`'s "Which directory?" is now "Which target?" — a leading `@`
216
+ is a registry ref routed straight to `okf <verb> @slug`, with the fallback
217
+ "no bundle in the cwd → `okf registry list`" — and the consume/search playbooks
218
+ name `@slug` in their orientation steps. The per-run `command -v okf` presence
219
+ probe is gone: run the verb, and treat a shell `command not found` as the only
220
+ signal to install, so the common case pays no guard round.
4
221
 
5
222
  ## [1.8.0] - 2026-07-17
6
223
 
data/README.md CHANGED
@@ -35,7 +35,7 @@ The package, end to end:
35
35
  <p align="center">
36
36
  <picture>
37
37
  <source media="(prefers-color-scheme: dark)" srcset=".github/overview-dark.png">
38
- <img src=".github/overview-light.png" width="760" alt="The package: the Agent Skill (your coding agent authors and curates, you stay the editor) writes and maintains the bundle, a folder of Markdown + YAML in your repo where one concept is one file and links between files are the knowledge graph. The bundle is read by the CLI/Lib (validate: legal OKF per section 9; lint: well-curated and fresh; search: ranked retrieval; require okf for Ruby objects) and by the Graph, in four modes: okf server (a live local server), okf render (the same page exported as one static, self-contained HTML file you can host anywhere), okf registry (every registered bundle behind one hub), and OKF::Server::App (the Rack app mounted in a Rails route). One gem, 100% local, Ruby 2.4 or newer, only rack and webrick as dependencies.">
38
+ <img src=".github/overview-light.png" width="760" alt="The package: the Agent Skill (your coding agent authors and curates, you stay the editor) writes and maintains the bundle, a folder of Markdown + YAML in your repo where one concept is one file and links between files are the knowledge graph. The bundle is read by the CLI/Lib (validate: legal OKF per section 9; lint: well-curated and fresh; search: ranked retrieval; require okf for Ruby objects) and by the Graph, in four modes: okf server (a live local server), okf render (the same page exported as one static, self-contained HTML file you can host anywhere), okf registry (every registered bundle behind one hub), and OKF::Server::App (the Rack app mounted in a Rails route). One gem, 100% local, Ruby 2.4 or newer, only rack, webrick and minifts as dependencies.">
39
39
  </picture>
40
40
  </p>
41
41
 
@@ -72,10 +72,13 @@ It is deliberately light so it runs on the Ruby your OS already ships:
72
72
 
73
73
  - works on every Ruby since 2.4, the same floor as [rack](https://github.com/rack/rack),
74
74
  its core dependency;
75
- - only two runtime dependencies: `rack` (the server is a mountable Rack app)
76
- and `webrick` (unbundled from Ruby in 3.0);
77
- - no ActiveSupport, no build step, no JavaScript toolchain the
78
- [design constraints](.okf/design/) that hold this line are enforced by tests.
75
+ - only three runtime dependencies: `rack` (the server is a mountable Rack app),
76
+ `webrick` (unbundled from Ruby in 3.0), and
77
+ [`minifts`](https://github.com/serradura/minifts) (the search enginepure
78
+ Ruby, no dependencies of its own, same 2.4 floor);
79
+ - no ActiveSupport, no native extension, no build step, no JavaScript
80
+ toolchain — the [design constraints](.okf/design/) that hold this line are
81
+ enforced by tests.
79
82
 
80
83
  That range is not aspirational: CI runs the full test suite and RuboCop on every
81
84
  one of these on each push.
@@ -225,7 +228,7 @@ docker run --rm -v "$PWD:/data" -p 8808:8808 ghcr.io/serradura/okf server . --bi
225
228
  Then open <http://127.0.0.1:8808>. Images are published for `linux/amd64` and
226
229
  `linux/arm64` on
227
230
  [ghcr.io](https://github.com/serradura/okf-gem/pkgs/container/okf): `:latest`
228
- tracks the newest release, or pin a version like `:1.8.0`.
231
+ tracks the newest release, or pin a version like `:1.9.0`.
229
232
 
230
233
  Tired of the long line? Install a Docker-backed [`okf` command](https://docker.okfgem.com),
231
234
  so every verb drops the `docker run` prefix and reads exactly like the native CLI
@@ -247,7 +250,7 @@ instead: `irm https://docker.okfgem.com/install.ps1 | iex`.
247
250
  okf validate <dir|@slug> [--json] # check OKF v0.1 conformance (§9)
248
251
  okf lint <dir|@slug> [--json] [--fail-on warn] [...] # report curation-quality issues
249
252
  okf loose <dir|@slug> [--json] # list files with no graph links, by folder
250
- okf search <dir|@slug…|@all> <term…> [-e] [...] # ranked retrieval; @slugs or @all span bundles
253
+ okf search <dir|@slug…|@all> <term…> [-e|--fuzzy] [...] # ranked retrieval; @slugs or @all span bundles
251
254
  okf index <dir|@slug> [--json] [--area A] [--no-body] # progressive-disclosure map (§6): bodies, rollups, listings
252
255
  okf server [DIR|@slug…] [-p PORT] [--bind ADDR] [...] # serve one bundle, or many behind a hub (⌘K to switch)
253
256
  okf render <dir|@slug> [-o FILE] [--layout NAME] [...] # export the graph as one static, self-contained HTML file
@@ -50,6 +50,16 @@ module OKF
50
50
  @bundle.directory_index
51
51
  end
52
52
 
53
+ # Every log.md with its content, root scope first — read live from disk so a
54
+ # just-appended entry shows without a reload; the reserved snapshot is the
55
+ # fallback if the file has since vanished. Shared by `okf render`'s bake
56
+ # (OKF::Render::Graph.payload) and OKF::Server::App's /log endpoint.
57
+ def log_entries
58
+ @bundle.log_files.sort_by { |path| [ path == "log.md" ? 0 : 1, path ] }.map do |path|
59
+ { path: path, dir: File.dirname(path), content: log_content(path) }
60
+ end
61
+ end
62
+
53
63
  # Human-readable "parent/dir" name — the default HTML title.
54
64
  # The bundle's display label, "parent/dir" — path arithmetic, no disk. It
55
65
  # is a class method so a caller that only wants the label (the registry
@@ -96,6 +106,12 @@ module OKF
96
106
  .select { |entry| File.basename(entry.path) == basename }
97
107
  .each_with_object({}) { |entry, hash| hash[entry.path] = entry.content }
98
108
  end
109
+
110
+ def log_content(path)
111
+ File.read(File.join(@root, path), encoding: "UTF-8")
112
+ rescue SystemCallError
113
+ @bundle.reserved_content(path)
114
+ end
99
115
  end
100
116
  end
101
117
  end
@@ -0,0 +1,65 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "minifts"
4
+ require "okf/bundle/search"
5
+
6
+ module OKF
7
+ class Bundle
8
+ class Search
9
+ # The default engine: a MiniFTS full-text index — the same engine, and the
10
+ # same BM25+ arithmetic, the browser page already runs as MiniSearch, so a
11
+ # Ruby-built index and the page's rank identically.
12
+ #
13
+ # Matching is by *token*: a term matches a whole word or a word it prefixes
14
+ # ("dedup" reaches "deduplication"), and `fuzzy:` opts into typo tolerance.
15
+ # The index is built per call — see .okf/capabilities/search.md for why that
16
+ # ceiling stands and what lifts it.
17
+ module Index
18
+ CAPABILITIES = %i[fuzzy prefix].freeze
19
+
20
+ class << self
21
+ def id
22
+ :index
23
+ end
24
+
25
+ def capabilities
26
+ CAPABILITIES
27
+ end
28
+
29
+ # minifts is a hard runtime dependency with no native extension, so it
30
+ # is here whenever the gem is. An addon backed by a native build is the
31
+ # case this predicate exists for.
32
+ def available?
33
+ true
34
+ end
35
+
36
+ # `fields:` narrows where a term may hit, so a field the caller excluded
37
+ # can neither match nor be credited. The hit's `terms` are MiniFTS's
38
+ # matched *document* terms — already lowercased, and present in the text
39
+ # verbatim even when the query only prefixed them.
40
+ def call(documents, terms, fields:, fuzzy: false, **_options)
41
+ index = MiniFTS.new(fields: FIELDS, id_field: "key")
42
+ index.add_all(documents)
43
+
44
+ options = { combine_with: "AND", prefix: true, boost: WEIGHTS, fields: fields }
45
+ options[:fuzzy] = FUZZY_DISTANCE if fuzzy
46
+
47
+ index.search(terms.join(" "), options).map do |hit|
48
+ { key: hit[:id], matched: matched_in(hit), score: hit[:score], terms: hit[:terms] }
49
+ end
50
+ end
51
+
52
+ private
53
+
54
+ # The union of fields any term hit, in WEIGHTS order. MiniFTS reports it
55
+ # per query term as { term => [field, …] }.
56
+ def matched_in(hit)
57
+ FIELDS.select { |field| hit[:match].any? { |_term, found| found.include?(field) } }
58
+ end
59
+ end
60
+
61
+ Search.register(self)
62
+ end
63
+ end
64
+ end
65
+ end
@@ -0,0 +1,89 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "okf/bundle/search"
4
+
5
+ module OKF
6
+ class Bundle
7
+ class Search
8
+ # The linear engine: terms matched against raw field text, one document at a
9
+ # time. No index, so nothing is tokenized and nothing is normalized — a
10
+ # phrase stays a phrase, `7.2.0` stays one string, and an infix matches.
11
+ # That is the exactness a token index gives up, and the reason this engine
12
+ # survived the swap.
13
+ #
14
+ # Two readings of a term, and the engine is the *raw text* half of the split
15
+ # rather than the regexp half:
16
+ #
17
+ # `regexp: false` — literal substring, which is what this engine did
18
+ # before the index landed, and what `--engine scan`
19
+ # restores. Terms are escaped, so `7.2.0` does not
20
+ # match `7x2y0` and `[draft]` is not a character class.
21
+ # `regexp: true` — the term is a pattern, opted into with `-e`.
22
+ #
23
+ # Conflating the two would make choosing the engine silently change what the
24
+ # terms mean, and turn an ordinary term like `review (pending` into exit 2.
25
+ #
26
+ # Scoring is the summed weight of the fields that matched: absolute, and so
27
+ # comparable across bundles without a corpus to normalize against.
28
+ module Scan
29
+ CAPABILITIES = %i[regexp].freeze
30
+
31
+ class << self
32
+ def id
33
+ :scan
34
+ end
35
+
36
+ def capabilities
37
+ CAPABILITIES
38
+ end
39
+
40
+ # No backing store to fail: the engine is Regexp and Enumerable.
41
+ def available?
42
+ true
43
+ end
44
+
45
+ # Raises RegexpError on an invalid pattern under `regexp: true` — the
46
+ # caller owns turning that into a usage error. A literal term cannot
47
+ # raise, because it is escaped before it is compiled. The hit's `terms`
48
+ # are the compiled patterns, which is what the facade points its snippet
49
+ # window at.
50
+ def call(documents, terms, fields:, regexp: false, **_options)
51
+ patterns = terms.map do |term|
52
+ Regexp.new(regexp ? term : Regexp.escape(term), Regexp::IGNORECASE)
53
+ end
54
+
55
+ hits = []
56
+ documents.each do |document|
57
+ matched = matched_fields(document, patterns, fields)
58
+ next if matched.nil?
59
+
60
+ hits << {
61
+ key: document["key"],
62
+ matched: matched,
63
+ score: matched.map { |field| WEIGHTS[field] }.reduce(0, :+),
64
+ terms: patterns
65
+ }
66
+ end
67
+ hits
68
+ end
69
+
70
+ private
71
+
72
+ # The union of fields any pattern hit, in WEIGHTS order — or nil when
73
+ # some pattern hit nothing (terms are ANDed).
74
+ def matched_fields(document, patterns, fields)
75
+ hits = patterns.map do |pattern|
76
+ found = fields.select { |field| pattern.match?(document[field]) }
77
+ return nil if found.empty?
78
+
79
+ found
80
+ end
81
+ FIELDS.select { |field| hits.any? { |found| found.include?(field) } }
82
+ end
83
+ end
84
+
85
+ Search.register(self)
86
+ end
87
+ end
88
+ end
89
+ end