okf 1.8.0 → 1.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +218 -1
- data/README.md +10 -7
- data/lib/okf/bundle/folder.rb +16 -0
- data/lib/okf/bundle/search/index.rb +65 -0
- data/lib/okf/bundle/search/scan.rb +89 -0
- data/lib/okf/bundle/search.rb +262 -66
- data/lib/okf/bundle.rb +2 -2
- data/lib/okf/cli.rb +95 -25
- data/lib/okf/{server → render}/graph/template.html.erb +535 -116
- data/lib/okf/{server → render}/graph.rb +39 -7
- data/lib/okf/server/app.rb +7 -44
- data/lib/okf/skill/SKILL.md +27 -16
- data/lib/okf/skill/playbooks/consume.md +5 -3
- data/lib/okf/skill/playbooks/maintain.md +1 -1
- data/lib/okf/skill/playbooks/search.md +47 -7
- data/lib/okf/skill/reference/authoring.md +3 -2
- data/lib/okf/skill/reference/cli.md +76 -17
- data/lib/okf/version.rb +1 -1
- data/lib/okf.rb +8 -0
- metadata +19 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 24f81d695477c347d78e44d89131ebdb7c5eec4f380c2f8ba7596292bf043e39
|
|
4
|
+
data.tar.gz: 14bc57ccda962eb9abfbcb9906e95c74b9fb808d2e46850dbfec234aafb933f7
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: d746db4e46fd17179f8eb82275749a1a2d1077c58e0d892d2e694dcd15f052e40dfe5526e17656ba2b21adb1fc63d1fa4885d06dce44c9edb584028445703444
|
|
7
|
+
data.tar.gz: 5f41214c5b0829c6d50a99cde236d5b768256d062ea6a6faa13270bc5ab3c2416fc6ff562be34c5c7067157da36fbd91f547db4120950dc2e9a8b6376a551f8c
|
data/CHANGELOG.md
CHANGED
|
@@ -1,6 +1,223 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
-
## [
|
|
3
|
+
## [1.9.0] - 2026-07-19
|
|
4
|
+
|
|
5
|
+
- **`okf search` gains an opt-in full-text index engine.** `--engine index` — and
|
|
6
|
+
`--fuzzy`, which implies it — routes to
|
|
7
|
+
[minifts](https://github.com/serradura/minifts), the pure-Ruby port of the same
|
|
8
|
+
MiniSearch build the graph page loads. It is the gem's third runtime
|
|
9
|
+
dependency, admitted because it costs the footprint nothing the first two were
|
|
10
|
+
chosen to protect: no native extension, no dependency tree of its own, the same
|
|
11
|
+
Ruby 2.4 floor. Three things it adds, and nothing else does:
|
|
12
|
+
- **BM25+ relevance ranking**, where the default scores by summed field weight;
|
|
13
|
+
- **`--fuzzy`** — typo tolerance at edit distance `0.2 × term length`, the
|
|
14
|
+
browser's own setting. Search stays exact unless you ask;
|
|
15
|
+
- **parity with the graph page**, which runs the same MiniSearch build, so the
|
|
16
|
+
two rank identically when the index is named.
|
|
17
|
+
- **The default search is unchanged** — literal, case-insensitive substring
|
|
18
|
+
matching over the same fields with the same weights as 1.8.0. The index is
|
|
19
|
+
opt-in rather than default because a one-shot CLI builds an index, asks one
|
|
20
|
+
question, and exits: end to end, **3.00 s against 0.24 s at 1,000 concepts**,
|
|
21
|
+
the build accounting for ~95% of that. The ~44–56× per-query throughput that
|
|
22
|
+
recommends minifts is the right measure for a long-lived index — a page, a
|
|
23
|
+
server — and the wrong one for a process that exits. A cached prebuilt index is
|
|
24
|
+
what would change that arithmetic.
|
|
25
|
+
- **Know what the index costs before naming it.** Its tokenizer splits on
|
|
26
|
+
punctuation, so `customer_id` becomes `customer` + `id` and `7.2.0` becomes
|
|
27
|
+
`7`, `2`, `0`; an infix (`ustomer`) finds nothing; and a backtick is Unicode
|
|
28
|
+
`Sk` rather than punctuation, so a word inside a code span indexes as
|
|
29
|
+
`` `minifts` `` and the query `minifts` does not match it — 409 such tokens
|
|
30
|
+
on this repo's own bundle. Ranking does not rescue it: BM25 normalizes by
|
|
31
|
+
field length, so a short concept dense in `7`, `2` and `0` can outrank the one
|
|
32
|
+
that actually says `7.2.0`. The default has none of these, because raw-text
|
|
33
|
+
matching has no tokenizer.
|
|
34
|
+
- **`--engine NAME` picks the engine outright**, for the case a capability flag
|
|
35
|
+
cannot express: a matching *model* requires nothing, so no flag selects one.
|
|
36
|
+
Naming an engine that cannot do what was also asked is a usage error naming one
|
|
37
|
+
that can (`--engine index -e` → *try --engine scan*), and an unknown name lists
|
|
38
|
+
what is available. `--help` reads the registry, so an addon's engine appears
|
|
39
|
+
without the CLI knowing it exists.
|
|
40
|
+
- **Search engines are adapters.** `OKF::Bundle::Search` became a facade over N
|
|
41
|
+
engines instead of one class with a `regexp ? scan : index` branch. The facade
|
|
42
|
+
keeps everything that defines a result — documents, the row and its key order,
|
|
43
|
+
the snippet, the sort — and an engine answers only which documents match, how
|
|
44
|
+
well, and where. The built-ins are `Search::Scan` (raw text, the default,
|
|
45
|
+
`regexp`) and `Search::Index` (minifts, `fuzzy`/`prefix`).
|
|
46
|
+
- **Selection is by capability when the query requires one** — `--fuzzy`
|
|
47
|
+
requires `:fuzzy`, so it routes to the index without naming it — and that
|
|
48
|
+
routing prints **nothing**: no note, no header change, no new JSON key.
|
|
49
|
+
- **`Search.register` is a published extension point** — append-only,
|
|
50
|
+
idempotent by id, capabilities checked against a fixed vocabulary. This is
|
|
51
|
+
the seam a future SQLite/FTS5 addon plugs into; no addon code ships here.
|
|
52
|
+
- **A shared conformance suite replaces the "kernel is the oracle" rule**,
|
|
53
|
+
which multiple engines made impossible: the index and the scan disagree about
|
|
54
|
+
match sets by design, so neither can be the oracle. Every registered engine
|
|
55
|
+
runs the same contract, with capability-gated blocks for its own semantics,
|
|
56
|
+
and a registered engine with no conformance class fails the suite.
|
|
57
|
+
- **Cross-bundle search ranks one corpus under `--engine index`.** BM25 prices a
|
|
58
|
+
term by how rare it is, so ranking each bundle separately and interleaving the
|
|
59
|
+
lists would produce a ranking that looks sorted and compares nothing; the
|
|
60
|
+
searched bundles are indexed together instead. The visible consequence, under
|
|
61
|
+
that engine only: a score is relative to the whole answer, so the same concept
|
|
62
|
+
scores lower searched beside other bundles than alone. The default's scores are
|
|
63
|
+
absolute and need no such treatment.
|
|
64
|
+
|
|
65
|
+
- **The graph can draw the index layer, under any layout.** The §6 map was
|
|
66
|
+
visible only inside file-tree mode, where a folder node stood in for a
|
|
67
|
+
directory's `index.md`. **Show indexes** makes it a layer: each map is a tile
|
|
68
|
+
edged to the concepts it lists and the maps below it, dressed by the same
|
|
69
|
+
selector as file-tree mode's folder node, because the two are the same thing
|
|
70
|
+
twice over — clicking either opens that directory's `index.md`. Both are accent
|
|
71
|
+
squares with dashed edges into them, so colour separates *kinds* rather than
|
|
72
|
+
modes: a directory is not a concept and no longer reads as one. Authorship shows
|
|
73
|
+
as form — solid where an author wrote a map, hollow and dashed where the bundle
|
|
74
|
+
only implies one — so the toggle reads as curation as much as navigation.
|
|
75
|
+
- **Moving between the modes lands in one click.** Tearing the layer down ran
|
|
76
|
+
its own layout while file-tree mode ran `breadthfirst` a beat later, two
|
|
77
|
+
layouts racing the same canvas; and because the layer is fetched, a promise
|
|
78
|
+
resolving after a mode change could land inside file-tree mode. A `relayout`
|
|
79
|
+
flag settles the first, a per-toggle ticket the second.
|
|
80
|
+
- **File-tree mode disables the toggle** rather than doubling the folders it
|
|
81
|
+
already draws.
|
|
82
|
+
- **One label on every file's graph button.** It read "Explore the knowledge
|
|
83
|
+
graph" on the root index and "Open core/ in graph" on a nested one, which made
|
|
84
|
+
a single action look like three. The question is the same whatever is open, so
|
|
85
|
+
the label is too — and it lives in the markup, where it cannot go stale.
|
|
86
|
+
- **Opening a map from the reader keeps the reader's graph.** It forced
|
|
87
|
+
file-tree mode, discarding whatever layout was running, and dimmed the canvas
|
|
88
|
+
to the map's immediate neighbours. It now switches the *layer* on rather than
|
|
89
|
+
the *mode* and leaves the layout alone. A reader already in file-tree mode
|
|
90
|
+
stays there.
|
|
91
|
+
- **Selecting anything emphasises it the same way.** A concept dimmed the graph
|
|
92
|
+
to its neighbourhood, a map did nothing at all, and a folder node did nothing
|
|
93
|
+
either — three meanings for one gesture. One `focusNode` now serves all three.
|
|
94
|
+
- **Drawn, never modelled.** `index.md` is reserved, so these nodes are built
|
|
95
|
+
from `/index` straight onto the canvas; `NODES`, `/catalog` and the type and
|
|
96
|
+
tag indexes never learn they exist. Filters pass them over — a map has no type
|
|
97
|
+
or tags — but a map whose concepts are all filtered away leaves with them.
|
|
98
|
+
- **Collapsing the root folds the file list away** on phones and tablets, where
|
|
99
|
+
the list is stacked on top of the reader and closing the root otherwise left a
|
|
100
|
+
single row above a column of nothing. Reopening the list undoes that collapse,
|
|
101
|
+
so it is one gesture rather than two states to dig out of — the fold remembers
|
|
102
|
+
*why* it happened, and a list folded because a file was opened comes back
|
|
103
|
+
exactly as it was left.
|
|
104
|
+
- **The bundle names its own root.** `(root)` and `/` are what a filesystem calls
|
|
105
|
+
it, not what a reader does. The tree's root row, file-tree mode's root node, the
|
|
106
|
+
index layer's root map and the inspector's directory map now all carry the name
|
|
107
|
+
the page header already shows, `--title` included. `areaOf` keeps its own
|
|
108
|
+
`(root)`: that is the area vocabulary `okf stats --by area` and `tags --by area`
|
|
109
|
+
print, not a UI label.
|
|
110
|
+
- **The Indexes tab dissolves into the file tree.** The authored layer lived on a
|
|
111
|
+
second tab as a flat list of paths, which put a directory's own map somewhere
|
|
112
|
+
other than the directory. `index.md` and `log.md` are rows now, at the top of
|
|
113
|
+
the folder they document, and **Indexes only** is a toggle over the same tree —
|
|
114
|
+
same rows, fewer of them, structure intact. The toggle yields only when it would
|
|
115
|
+
hide what was just opened — a map stays under it, a concept releases it — so
|
|
116
|
+
browsing the authored layer no longer destroys the list being browsed. A log
|
|
117
|
+
offers no graph button at all: it is a chronology, not a place in the graph, and
|
|
118
|
+
the button had been opening the root index's node.
|
|
119
|
+
Narrowed, a folder owns exactly one row, so the row stands where the folder
|
|
120
|
+
header stood — at that folder's depth, carrying the path — rather than nesting
|
|
121
|
+
a single child under a header.
|
|
122
|
+
- **The rail's Index becomes an action, not a fake view.** It had no
|
|
123
|
+
`#view-index` behind it — the files view showing its other tab — so
|
|
124
|
+
`activeRail()` answered a question of view *and* tab. The shortcut stays,
|
|
125
|
+
opening the root map through the same `readIndex()` the first-visit note
|
|
126
|
+
uses; `activeRail()` answers with the view it lands on, so Files highlights
|
|
127
|
+
and nothing invents a place for Index to be. `?view=index` resolves to the
|
|
128
|
+
same action.
|
|
129
|
+
- **Fixed on the way:** the reader header rendered empty — an unlabelled badge
|
|
130
|
+
and a graph button pointing nowhere — whenever no file was open, because
|
|
131
|
+
`.fp-head{display:flex}` outranks the UA sheet's `[hidden]{display:none}`.
|
|
132
|
+
- **A first-visit note tells a newcomer the index exists.** The `index.md` an
|
|
133
|
+
author wrote to be read first was reachable only by finding the Indexes tab and
|
|
134
|
+
clicking a row, so a reader meeting a bundle for the first time met unlabelled
|
|
135
|
+
dots with no way in. The page still opens on the graph — it is what makes a
|
|
136
|
+
bundle legible at a glance, at every width — and a dismissible note at the
|
|
137
|
+
bottom now says what the picture is, how to touch it, and where the index is.
|
|
138
|
+
**Read the index** goes straight there; the dismissal is remembered.
|
|
139
|
+
- **It absorbed the old mobile-only tip** rather than stacking a second banner
|
|
140
|
+
under it, and it is written for a finger throughout, since a phone is where a
|
|
141
|
+
first-time reader is least oriented.
|
|
142
|
+
- **The wording follows the device on two gates, not one.** What a reader does
|
|
143
|
+
follows `(pointer:coarse)` — a touch tablet in landscape is wider than 768px
|
|
144
|
+
and still taps; a narrow desktop window is narrower and still clicks. What a
|
|
145
|
+
reader can reach follows `(max-width:768px)`, because that is when the rail
|
|
146
|
+
collapses behind `☰`. Short viewports tighten; short *and* wide puts the
|
|
147
|
+
question beside the button, taking a landscape phone from half the screen to
|
|
148
|
+
under a third.
|
|
149
|
+
- **A second note points at `☰`** on compact layouts only, anchored under the
|
|
150
|
+
button it names rather than at the bottom of the screen. It fires on leaving
|
|
151
|
+
the graph by any route, so dismissing the first note does not cost it, and
|
|
152
|
+
opening `☰` answers it — but only once it is on screen, since `☰` is the only
|
|
153
|
+
way off the graph there and the first tap always comes first.
|
|
154
|
+
- **Deep links are unaffected**, and `?select=`/`#hash` now switch to the graph
|
|
155
|
+
before selecting, since the page can be standing elsewhere when they are read.
|
|
156
|
+
- **The file tree nests.** Directories were a sorted list of full paths, which
|
|
157
|
+
made `core` and `core/configurations` read as two unrelated folders and left
|
|
158
|
+
the shape of a bundle invisible. Each row is now one path segment indented by
|
|
159
|
+
depth, folders before files, and collapsing a folder takes its subtree with it.
|
|
160
|
+
A directory holding nothing but directories still renders, so the chain to its
|
|
161
|
+
children never breaks.
|
|
162
|
+
- **"Collapse all" folds into the root, not over it** — everything inside the
|
|
163
|
+
root closes and the root stays open, so the click leaves the top-level
|
|
164
|
+
folders standing instead of a single `(root)` row. Unfolding clears the whole
|
|
165
|
+
set, root included, so a root closed by hand is still reversible from there.
|
|
166
|
+
|
|
167
|
+
- The graph page's search box grows a full-text index. One MiniSearch index —
|
|
168
|
+
lazy-loaded from the CDN on first search, pinned to the `7.2.0` the Ruby
|
|
169
|
+
MiniSearch port tracks so an `okf search --engine index` result and the
|
|
170
|
+
browser's rank identically — now backs the graph, catalog and files views: ranked, multi-term
|
|
171
|
+
(`AND`), prefix (as-you-type) and typo-tolerant, over title, id, type, tags and
|
|
172
|
+
**description** — plus each concept's **body** wherever the page already holds
|
|
173
|
+
it (`okf render` bakes every body in, so a static file searches bodies offline;
|
|
174
|
+
the live server keeps bodies lazy, so its index stays metadata-only until a
|
|
175
|
+
backend body index arrives). The graph could not be searched by a leaf's
|
|
176
|
+
description before; now it can. The Files view's **Indexes** tab gets its own
|
|
177
|
+
full-text index too, over each `index.md`/`log.md`'s body — not just its
|
|
178
|
+
filename. Until an index loads — or if the CDN is unreachable — each view falls
|
|
179
|
+
back to its own substring filter, so the box is never dead.
|
|
180
|
+
- The Files tree's folder collapse works during a search. An active search or
|
|
181
|
+
type/tag filter used to force every folder open, so fold clicks did nothing;
|
|
182
|
+
folders now honor their collapsed state always (a collapsed group still shows
|
|
183
|
+
its header, so a match is never hidden). A **fold/unfold-all** control in the
|
|
184
|
+
Files tab header collapses or expands every visible group at once.
|
|
185
|
+
- Clustering no longer leaves phantom empty boxes. When a filter or a search hid
|
|
186
|
+
every concept in an area, the cluster's labelled box lingered as an empty
|
|
187
|
+
rectangle; the box now hides when no child survives and returns when one does —
|
|
188
|
+
the same rule the fit already used to leave stale boxes out of view, now
|
|
189
|
+
applied to what is drawn.
|
|
190
|
+
- `Esc` clears the graph selection. A dense graph leaves almost no empty canvas
|
|
191
|
+
to click for deselecting; `Esc` now drops the highlight (and lets the URL hash
|
|
192
|
+
forget the node) the same way tapping empty canvas does.
|
|
193
|
+
- A title-less concept now wears one name in every view. `catalog` and the §6
|
|
194
|
+
index listing fell back a concept with no `title` to its full id — `area/thing`
|
|
195
|
+
— while the graph node fell back blank-aware to the basename — `thing` — so the
|
|
196
|
+
same concept answered to two labels across two views of one bundle, and a
|
|
197
|
+
`title: ""` slipped past the nil-only `||` to catalog as an empty string. Both
|
|
198
|
+
now fall back the graph's way (`File.basename`, blank-aware), so the label is
|
|
199
|
+
the same wherever the concept appears.
|
|
200
|
+
- `okf render` stops baking a redundant description map. The static page derived
|
|
201
|
+
its `/node/meta` fragments from a separate `meta` payload that held nothing but
|
|
202
|
+
each concept's description, HTML-escaped — data the embedded `catalog` already
|
|
203
|
+
carries raw. The page now escapes the catalog's description on the client (the
|
|
204
|
+
same escape the server applies at `/node/meta`), so the `meta` key leaves the
|
|
205
|
+
baked payload and the description lives in one place. Both XSS guards are
|
|
206
|
+
unchanged; `okf server` is untouched.
|
|
207
|
+
- The bare not-a-directory error now teaches the registry grammar. A verb given
|
|
208
|
+
a target that is neither a directory nor an `@ref` moved from
|
|
209
|
+
`error: <arg> is not a directory` to
|
|
210
|
+
`… is not a directory or a registry ref (@slug names a registered bundle, @ the default; okf registry list)`,
|
|
211
|
+
so a consumer who typed a query or a bad path meets `@slug` addressing at the
|
|
212
|
+
error instead of hunting for it. (`@all` stays out of the message — it is
|
|
213
|
+
`search`'s alone, and the error seam is shared by every verb.)
|
|
214
|
+
- The bundled skill teaches `@slug` as a first-class target and stops probing for
|
|
215
|
+
the CLI. `SKILL.md`'s "Which directory?" is now "Which target?" — a leading `@`
|
|
216
|
+
is a registry ref routed straight to `okf <verb> @slug`, with the fallback
|
|
217
|
+
"no bundle in the cwd → `okf registry list`" — and the consume/search playbooks
|
|
218
|
+
name `@slug` in their orientation steps. The per-run `command -v okf` presence
|
|
219
|
+
probe is gone: run the verb, and treat a shell `command not found` as the only
|
|
220
|
+
signal to install, so the common case pays no guard round.
|
|
4
221
|
|
|
5
222
|
## [1.8.0] - 2026-07-17
|
|
6
223
|
|
data/README.md
CHANGED
|
@@ -35,7 +35,7 @@ The package, end to end:
|
|
|
35
35
|
<p align="center">
|
|
36
36
|
<picture>
|
|
37
37
|
<source media="(prefers-color-scheme: dark)" srcset=".github/overview-dark.png">
|
|
38
|
-
<img src=".github/overview-light.png" width="760" alt="The package: the Agent Skill (your coding agent authors and curates, you stay the editor) writes and maintains the bundle, a folder of Markdown + YAML in your repo where one concept is one file and links between files are the knowledge graph. The bundle is read by the CLI/Lib (validate: legal OKF per section 9; lint: well-curated and fresh; search: ranked retrieval; require okf for Ruby objects) and by the Graph, in four modes: okf server (a live local server), okf render (the same page exported as one static, self-contained HTML file you can host anywhere), okf registry (every registered bundle behind one hub), and OKF::Server::App (the Rack app mounted in a Rails route). One gem, 100% local, Ruby 2.4 or newer, only rack and
|
|
38
|
+
<img src=".github/overview-light.png" width="760" alt="The package: the Agent Skill (your coding agent authors and curates, you stay the editor) writes and maintains the bundle, a folder of Markdown + YAML in your repo where one concept is one file and links between files are the knowledge graph. The bundle is read by the CLI/Lib (validate: legal OKF per section 9; lint: well-curated and fresh; search: ranked retrieval; require okf for Ruby objects) and by the Graph, in four modes: okf server (a live local server), okf render (the same page exported as one static, self-contained HTML file you can host anywhere), okf registry (every registered bundle behind one hub), and OKF::Server::App (the Rack app mounted in a Rails route). One gem, 100% local, Ruby 2.4 or newer, only rack, webrick and minifts as dependencies.">
|
|
39
39
|
</picture>
|
|
40
40
|
</p>
|
|
41
41
|
|
|
@@ -72,10 +72,13 @@ It is deliberately light so it runs on the Ruby your OS already ships:
|
|
|
72
72
|
|
|
73
73
|
- works on every Ruby since 2.4, the same floor as [rack](https://github.com/rack/rack),
|
|
74
74
|
its core dependency;
|
|
75
|
-
- only
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
75
|
+
- only three runtime dependencies: `rack` (the server is a mountable Rack app),
|
|
76
|
+
`webrick` (unbundled from Ruby in 3.0), and
|
|
77
|
+
[`minifts`](https://github.com/serradura/minifts) (the search engine — pure
|
|
78
|
+
Ruby, no dependencies of its own, same 2.4 floor);
|
|
79
|
+
- no ActiveSupport, no native extension, no build step, no JavaScript
|
|
80
|
+
toolchain — the [design constraints](.okf/design/) that hold this line are
|
|
81
|
+
enforced by tests.
|
|
79
82
|
|
|
80
83
|
That range is not aspirational: CI runs the full test suite and RuboCop on every
|
|
81
84
|
one of these on each push.
|
|
@@ -225,7 +228,7 @@ docker run --rm -v "$PWD:/data" -p 8808:8808 ghcr.io/serradura/okf server . --bi
|
|
|
225
228
|
Then open <http://127.0.0.1:8808>. Images are published for `linux/amd64` and
|
|
226
229
|
`linux/arm64` on
|
|
227
230
|
[ghcr.io](https://github.com/serradura/okf-gem/pkgs/container/okf): `:latest`
|
|
228
|
-
tracks the newest release, or pin a version like `:1.
|
|
231
|
+
tracks the newest release, or pin a version like `:1.9.0`.
|
|
229
232
|
|
|
230
233
|
Tired of the long line? Install a Docker-backed [`okf` command](https://docker.okfgem.com),
|
|
231
234
|
so every verb drops the `docker run` prefix and reads exactly like the native CLI
|
|
@@ -247,7 +250,7 @@ instead: `irm https://docker.okfgem.com/install.ps1 | iex`.
|
|
|
247
250
|
okf validate <dir|@slug> [--json] # check OKF v0.1 conformance (§9)
|
|
248
251
|
okf lint <dir|@slug> [--json] [--fail-on warn] [...] # report curation-quality issues
|
|
249
252
|
okf loose <dir|@slug> [--json] # list files with no graph links, by folder
|
|
250
|
-
okf search <dir|@slug…|@all> <term…> [-e] [...]
|
|
253
|
+
okf search <dir|@slug…|@all> <term…> [-e|--fuzzy] [...] # ranked retrieval; @slugs or @all span bundles
|
|
251
254
|
okf index <dir|@slug> [--json] [--area A] [--no-body] # progressive-disclosure map (§6): bodies, rollups, listings
|
|
252
255
|
okf server [DIR|@slug…] [-p PORT] [--bind ADDR] [...] # serve one bundle, or many behind a hub (⌘K to switch)
|
|
253
256
|
okf render <dir|@slug> [-o FILE] [--layout NAME] [...] # export the graph as one static, self-contained HTML file
|
data/lib/okf/bundle/folder.rb
CHANGED
|
@@ -50,6 +50,16 @@ module OKF
|
|
|
50
50
|
@bundle.directory_index
|
|
51
51
|
end
|
|
52
52
|
|
|
53
|
+
# Every log.md with its content, root scope first — read live from disk so a
|
|
54
|
+
# just-appended entry shows without a reload; the reserved snapshot is the
|
|
55
|
+
# fallback if the file has since vanished. Shared by `okf render`'s bake
|
|
56
|
+
# (OKF::Render::Graph.payload) and OKF::Server::App's /log endpoint.
|
|
57
|
+
def log_entries
|
|
58
|
+
@bundle.log_files.sort_by { |path| [ path == "log.md" ? 0 : 1, path ] }.map do |path|
|
|
59
|
+
{ path: path, dir: File.dirname(path), content: log_content(path) }
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
|
|
53
63
|
# Human-readable "parent/dir" name — the default HTML title.
|
|
54
64
|
# The bundle's display label, "parent/dir" — path arithmetic, no disk. It
|
|
55
65
|
# is a class method so a caller that only wants the label (the registry
|
|
@@ -96,6 +106,12 @@ module OKF
|
|
|
96
106
|
.select { |entry| File.basename(entry.path) == basename }
|
|
97
107
|
.each_with_object({}) { |entry, hash| hash[entry.path] = entry.content }
|
|
98
108
|
end
|
|
109
|
+
|
|
110
|
+
def log_content(path)
|
|
111
|
+
File.read(File.join(@root, path), encoding: "UTF-8")
|
|
112
|
+
rescue SystemCallError
|
|
113
|
+
@bundle.reserved_content(path)
|
|
114
|
+
end
|
|
99
115
|
end
|
|
100
116
|
end
|
|
101
117
|
end
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "minifts"
|
|
4
|
+
require "okf/bundle/search"
|
|
5
|
+
|
|
6
|
+
module OKF
|
|
7
|
+
class Bundle
|
|
8
|
+
class Search
|
|
9
|
+
# The default engine: a MiniFTS full-text index — the same engine, and the
|
|
10
|
+
# same BM25+ arithmetic, the browser page already runs as MiniSearch, so a
|
|
11
|
+
# Ruby-built index and the page's rank identically.
|
|
12
|
+
#
|
|
13
|
+
# Matching is by *token*: a term matches a whole word or a word it prefixes
|
|
14
|
+
# ("dedup" reaches "deduplication"), and `fuzzy:` opts into typo tolerance.
|
|
15
|
+
# The index is built per call — see .okf/capabilities/search.md for why that
|
|
16
|
+
# ceiling stands and what lifts it.
|
|
17
|
+
module Index
|
|
18
|
+
CAPABILITIES = %i[fuzzy prefix].freeze
|
|
19
|
+
|
|
20
|
+
class << self
|
|
21
|
+
def id
|
|
22
|
+
:index
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
def capabilities
|
|
26
|
+
CAPABILITIES
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
# minifts is a hard runtime dependency with no native extension, so it
|
|
30
|
+
# is here whenever the gem is. An addon backed by a native build is the
|
|
31
|
+
# case this predicate exists for.
|
|
32
|
+
def available?
|
|
33
|
+
true
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
# `fields:` narrows where a term may hit, so a field the caller excluded
|
|
37
|
+
# can neither match nor be credited. The hit's `terms` are MiniFTS's
|
|
38
|
+
# matched *document* terms — already lowercased, and present in the text
|
|
39
|
+
# verbatim even when the query only prefixed them.
|
|
40
|
+
def call(documents, terms, fields:, fuzzy: false, **_options)
|
|
41
|
+
index = MiniFTS.new(fields: FIELDS, id_field: "key")
|
|
42
|
+
index.add_all(documents)
|
|
43
|
+
|
|
44
|
+
options = { combine_with: "AND", prefix: true, boost: WEIGHTS, fields: fields }
|
|
45
|
+
options[:fuzzy] = FUZZY_DISTANCE if fuzzy
|
|
46
|
+
|
|
47
|
+
index.search(terms.join(" "), options).map do |hit|
|
|
48
|
+
{ key: hit[:id], matched: matched_in(hit), score: hit[:score], terms: hit[:terms] }
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
private
|
|
53
|
+
|
|
54
|
+
# The union of fields any term hit, in WEIGHTS order. MiniFTS reports it
|
|
55
|
+
# per query term as { term => [field, …] }.
|
|
56
|
+
def matched_in(hit)
|
|
57
|
+
FIELDS.select { |field| hit[:match].any? { |_term, found| found.include?(field) } }
|
|
58
|
+
end
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
Search.register(self)
|
|
62
|
+
end
|
|
63
|
+
end
|
|
64
|
+
end
|
|
65
|
+
end
|
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "okf/bundle/search"
|
|
4
|
+
|
|
5
|
+
module OKF
|
|
6
|
+
class Bundle
|
|
7
|
+
class Search
|
|
8
|
+
# The linear engine: terms matched against raw field text, one document at a
|
|
9
|
+
# time. No index, so nothing is tokenized and nothing is normalized — a
|
|
10
|
+
# phrase stays a phrase, `7.2.0` stays one string, and an infix matches.
|
|
11
|
+
# That is the exactness a token index gives up, and the reason this engine
|
|
12
|
+
# survived the swap.
|
|
13
|
+
#
|
|
14
|
+
# Two readings of a term, and the engine is the *raw text* half of the split
|
|
15
|
+
# rather than the regexp half:
|
|
16
|
+
#
|
|
17
|
+
# `regexp: false` — literal substring, which is what this engine did
|
|
18
|
+
# before the index landed, and what `--engine scan`
|
|
19
|
+
# restores. Terms are escaped, so `7.2.0` does not
|
|
20
|
+
# match `7x2y0` and `[draft]` is not a character class.
|
|
21
|
+
# `regexp: true` — the term is a pattern, opted into with `-e`.
|
|
22
|
+
#
|
|
23
|
+
# Conflating the two would make choosing the engine silently change what the
|
|
24
|
+
# terms mean, and turn an ordinary term like `review (pending` into exit 2.
|
|
25
|
+
#
|
|
26
|
+
# Scoring is the summed weight of the fields that matched: absolute, and so
|
|
27
|
+
# comparable across bundles without a corpus to normalize against.
|
|
28
|
+
module Scan
|
|
29
|
+
CAPABILITIES = %i[regexp].freeze
|
|
30
|
+
|
|
31
|
+
class << self
|
|
32
|
+
def id
|
|
33
|
+
:scan
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def capabilities
|
|
37
|
+
CAPABILITIES
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
# No backing store to fail: the engine is Regexp and Enumerable.
|
|
41
|
+
def available?
|
|
42
|
+
true
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
# Raises RegexpError on an invalid pattern under `regexp: true` — the
|
|
46
|
+
# caller owns turning that into a usage error. A literal term cannot
|
|
47
|
+
# raise, because it is escaped before it is compiled. The hit's `terms`
|
|
48
|
+
# are the compiled patterns, which is what the facade points its snippet
|
|
49
|
+
# window at.
|
|
50
|
+
def call(documents, terms, fields:, regexp: false, **_options)
|
|
51
|
+
patterns = terms.map do |term|
|
|
52
|
+
Regexp.new(regexp ? term : Regexp.escape(term), Regexp::IGNORECASE)
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
hits = []
|
|
56
|
+
documents.each do |document|
|
|
57
|
+
matched = matched_fields(document, patterns, fields)
|
|
58
|
+
next if matched.nil?
|
|
59
|
+
|
|
60
|
+
hits << {
|
|
61
|
+
key: document["key"],
|
|
62
|
+
matched: matched,
|
|
63
|
+
score: matched.map { |field| WEIGHTS[field] }.reduce(0, :+),
|
|
64
|
+
terms: patterns
|
|
65
|
+
}
|
|
66
|
+
end
|
|
67
|
+
hits
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
private
|
|
71
|
+
|
|
72
|
+
# The union of fields any pattern hit, in WEIGHTS order — or nil when
|
|
73
|
+
# some pattern hit nothing (terms are ANDed).
|
|
74
|
+
def matched_fields(document, patterns, fields)
|
|
75
|
+
hits = patterns.map do |pattern|
|
|
76
|
+
found = fields.select { |field| pattern.match?(document[field]) }
|
|
77
|
+
return nil if found.empty?
|
|
78
|
+
|
|
79
|
+
found
|
|
80
|
+
end
|
|
81
|
+
FIELDS.select { |field| hits.any? { |found| found.include?(field) } }
|
|
82
|
+
end
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
Search.register(self)
|
|
86
|
+
end
|
|
87
|
+
end
|
|
88
|
+
end
|
|
89
|
+
end
|