winnower 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +5 -0
- data/LICENSE.txt +21 -0
- data/README.md +132 -0
- data/examples/palette.rb +14 -0
- data/lib/winnower/index.rb +166 -0
- data/lib/winnower/matcher.rb +448 -0
- data/lib/winnower/version.rb +6 -0
- data/lib/winnower.rb +36 -0
- data/sig/winnower.rbs +84 -0
- metadata +52 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 7d86f2133bcaff61c25bfd647d603eb7b9bf1c2516c71517935cbac5738d0cf5
|
|
4
|
+
data.tar.gz: 495543a26aa5e2190e76974dbec94e375bad1e66dc6b467f77115898302272ad
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 11830ab5d0e8c497f4099c20688845909dce976935711b04ca8db51722f2bac6fd22c7ed62d267b31894db0aff325bda48a57d1871b8bc940de9934bc82877a1
|
|
7
|
+
data.tar.gz: a858f85cf086c7fd661d5e490c0882ce23c443fd179302e297f854a4d39f7f816e4ebf18ac3e90fc5a810d5415de79c9bd5e78d726fdfbb3cd5ef9f346e76d31
|
data/CHANGELOG.md
ADDED
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
The MIT License (MIT)
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 ydah
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
|
13
|
+
all copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,132 @@
|
|
|
1
|
+
# Winnower
|
|
2
|
+
|
|
3
|
+
A pure Ruby fuzzy subsequence matcher with scores, highlight positions and incremental filtering for command palettes and quick-open lists.
|
|
4
|
+
|
|
5
|
+
Edit-distance similarity is not enough for a palette: `amu` should find `app/models/user.rb`, rank its alignment, and tell the UI which characters to highlight. Winnower combines that contract with reusable candidate preprocessing and query history. It requires Ruby 3.1+ and has no runtime gems or native extensions.
|
|
6
|
+
|
|
7
|
+
## Five-line example
|
|
8
|
+
|
|
9
|
+
```ruby
|
|
10
|
+
require "winnower"
|
|
11
|
+
index = Winnower::Index.new(["app/models/user.rb", "app/models/order.rb", "README.md"])
|
|
12
|
+
session = index.session
|
|
13
|
+
session.query = "amu"
|
|
14
|
+
p session.top(50).map { |match| [match.candidate, match.score, match.positions] }
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
## Installation
|
|
18
|
+
|
|
19
|
+
```sh
|
|
20
|
+
gem install winnower
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## Stateless or incremental
|
|
24
|
+
|
|
25
|
+
```ruby
|
|
26
|
+
Winnower.score("amf", "app/models/foo.rb") # Float; -Float::INFINITY if absent
|
|
27
|
+
match = Winnower.match("amu", "app/models/user.rb")
|
|
28
|
+
match.positions # [0, 4, 11]
|
|
29
|
+
match.score # optimal weighted alignment score
|
|
30
|
+
Winnower.match("xyz", "README.md") # nil
|
|
31
|
+
Winnower.filter("srcm", candidates, limit: 50) # Array<Match>
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Positions are Unicode **character offsets**, not bytes or grapheme-cluster indices. Results, their strings/positions, and returned top arrays are immutable. `Match#text` aliases `candidate`; `index` is the stable registration ID.
|
|
35
|
+
|
|
36
|
+
```ruby
|
|
37
|
+
index = Winnower::Index.new(paths)
|
|
38
|
+
session = index.session
|
|
39
|
+
session.query = "c"; session.top(50)
|
|
40
|
+
session.query = "co"; session.top(50) # only previous matches are rescored
|
|
41
|
+
session.query = "con"; session.top(50)
|
|
42
|
+
session.query = "co"; session.top(50) # cached match set and top results
|
|
43
|
+
|
|
44
|
+
index.add(["new/file.rb"])
|
|
45
|
+
index.remove(["deleted/file.rb"])
|
|
46
|
+
session.top(50) # index generation invalidates old history
|
|
47
|
+
session.count # all matches, not only the top 50
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Indices deduplicate identical text; removing and re-adding a path gives it a new registration ID. Original input strings can be edited without changing indexed records. A session retains the active prefix chain; unrelated pasted queries restart from the full index. Limiting the displayed top list never discards candidates needed for the next query.
|
|
51
|
+
|
|
52
|
+
A larger `top(limit)` recomputes partial selection; a repeated or smaller limit reuses cached results. Build one Index per candidate collection and one Session per independent palette. Candidate records and Options are frozen, while Index mutations and Session operations are explicitly stateful: serialize mutations, and do not share a Session between concurrent callers.
|
|
53
|
+
|
|
54
|
+
## Scoring and options
|
|
55
|
+
|
|
56
|
+
The score is an optimal fzy-style subsequence alignment for inputs within the configured ceilings. A sparse D/M recurrence visits matching positions, carrying the best intervening-gap score instead of materializing every matrix cell. Score-only scans reuse two rows; highlight backtracking runs only for selected results. One-/two-character queries have equivalent scalar recurrences, and a contiguous match can stop early only when it reaches a proven global score upper bound.
|
|
57
|
+
|
|
58
|
+
The subsequence precheck also scores an alignment directly when every remaining query character has exactly one occurrence. Ambiguous alignments still use the full recurrence. Membership masks and DP scratch rows are prepared only when needed, avoiding unused preprocessing in stateless calls.
|
|
59
|
+
|
|
60
|
+
```ruby
|
|
61
|
+
options = Winnower::Options.new(
|
|
62
|
+
case_sensitivity: :smart, # :smart, :insensitive, :sensitive
|
|
63
|
+
path_mode: true,
|
|
64
|
+
limit: 100,
|
|
65
|
+
tie_break: :shorter
|
|
66
|
+
)
|
|
67
|
+
index = Winnower::Index.new(paths, options:)
|
|
68
|
+
Winnower.match("cf", "core/File.rb", options:)
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Smart case is case-insensitive unless the query contains an uppercase character. Default ties are resolved by score descending, candidate character length ascending, then registration ID ascending. `tie_break: :index` skips the length criterion. Exact matches score positive infinity; an empty query matches every candidate with score zero.
|
|
72
|
+
|
|
73
|
+
Default weights preserve the upstream public fzy ranking cases:
|
|
74
|
+
|
|
75
|
+
| Option | Default |
|
|
76
|
+
| --- | ---: |
|
|
77
|
+
| `consecutive` | 1.0 |
|
|
78
|
+
| `slash` (also start of candidate / Windows separator) | 0.9 |
|
|
79
|
+
| `boundary` (after dash, underscore or space) | 0.8 |
|
|
80
|
+
| `camel` | 0.7 |
|
|
81
|
+
| `dot` | 0.6 |
|
|
82
|
+
| `leading_gap` / `trailing_gap` | -0.005 |
|
|
83
|
+
| `inner_gap` | -0.01 |
|
|
84
|
+
| `case_bonus` (exact-case micro-bonus, opt-in) | 0.0 |
|
|
85
|
+
| `basename_bonus` (used when `path_mode: true`) | 0.2 |
|
|
86
|
+
|
|
87
|
+
Path mode adds a configurable bonus to nonconsecutive matches within the basename. Its default is off for fzy-compatible ranking. All weights are configurable finite real numbers; millipoint-exact weights use integer internal arithmetic so mathematically equal scores do not flicker from floating-point accumulation order.
|
|
88
|
+
|
|
89
|
+
The index precomputes folded ASCII strings, a 128-bit membership mask plus Unicode membership, boundary codes and basename offsets. Non-ASCII text keeps one lowercase mapping per original character. This preserves highlight positions through expanding lowercase mappings, but deliberately does not perform full Unicode case folding, normalization or grapheme matching: `ss` is not a substitute for `ß`, and composed/decomposed text is not silently normalized.
|
|
90
|
+
|
|
91
|
+
Candidates longer than `max_length: 1024` or queries longer than `max_query: 256` use bounded-memory greedy alignment when an exact fast path is unavailable. They remain valid subsequence matches, but their score/positions are not promised optimal. No edit distance, token rearrangement, transliteration or phonetic matching is performed. Invalid text/options raise `ArgumentError`.
|
|
92
|
+
|
|
93
|
+
## Verification
|
|
94
|
+
|
|
95
|
+
```sh
|
|
96
|
+
bundle install
|
|
97
|
+
bundle exec rake
|
|
98
|
+
bundle exec rake test:oracle
|
|
99
|
+
BUDGET=1 bundle exec rake bench
|
|
100
|
+
rbs -I sig validate
|
|
101
|
+
yard doc
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
Tests include 2,000 Unicode position properties, 3,000 comparisons against an independent dense recurrence with varied weights, randomized heap/top and session consistency, encoding/case behavior, deterministic ties, cache invalidation and malformed options.
|
|
105
|
+
|
|
106
|
+
The optional native oracle compiles the pinned, unmodified [upstream fzy scorer](test/vendor/fzy/README.md) in a temporary directory. It checks **all eight public ranking assertions** plus 500 seeded ASCII score comparisons. A compiler is only needed for this development oracle; the library works with `ruby --disable-gems`. `FZY_REQUIRED=1` makes a missing compiler fail instead of skip. Linux CI requires the oracle; macOS/Windows run it when a compiler is available. Isolation checks build/install the gem into a temporary GEM_HOME and emit results without development/application dependencies.
|
|
107
|
+
|
|
108
|
+
## Measured performance
|
|
109
|
+
|
|
110
|
+
Ruby 4.0.0 + YJIT, arm64 macOS; median of five warmed runs. The representative corpus has 100,000 paths, of which the first query `c` retains 10,000; the next query scans those 10,000. Timings include `query=` and `top(50)`.
|
|
111
|
+
|
|
112
|
+
| Workload | Measured | Goal |
|
|
113
|
+
| --- | ---: | ---: |
|
|
114
|
+
| Build 100,000-candidate index | 87.83ms | <400ms |
|
|
115
|
+
| First key, 100,000 → 10,000 matches | 4.93ms | <40ms |
|
|
116
|
+
| Second key `co`, 10,000 candidates | 2.39ms | <5ms |
|
|
117
|
+
| Noncontiguous `cm`, 10,000 candidates | 3.13ms | — |
|
|
118
|
+
| Longer `component_123` query | 2.82ms | — |
|
|
119
|
+
| Cached backspace | 0.002ms | <1ms |
|
|
120
|
+
| Stateless `score("amf", "app/models/foo.rb")` | 2.18µs | <3µs |
|
|
121
|
+
| Adversarial first key matching all 100,000 | 15.23ms | — |
|
|
122
|
+
| Adversarial second key still matching all 100,000 | 11.54ms | — |
|
|
123
|
+
|
|
124
|
+
The 5ms second-key target applies to the specified **10,000 remaining candidates**, not 100,000 matches. The benchmark retains and reports the all-hit stress case separately. Performance depends on candidate/query distribution and hardware; these measurements are not worst-case guarantees.
|
|
125
|
+
|
|
126
|
+
A subsequent stateless-score CI regression check compared `4af16c3` with the unique-alignment/lazy-preparation fix on Ruby 4.0.6 + YJIT, Linux arm64, Bundler 4.0.19. Three alternating `BUDGET=1 bundle exec rake bench` pairs (each reporting five warmed runs) reduced the median stateless call from 2.62µs to 1.20µs; all gates passed in all three corrected runs. The same-call allocation count fell from 15 to 6 objects. The original failing GitHub x86_64 runner measured 5.17µs; these local measurements are not a rerun on that hardware. Neither the 3µs limit nor the benchmark workload was changed.
|
|
127
|
+
|
|
128
|
+
Retained candidate records measured 34.33MiB, about 360 bytes per candidate, excluding the Index hash and input array. This exceeds the design estimate of roughly 100 bytes per candidate / 10MB for 100,000; it is a known Ruby object-overhead tradeoff, not a passed memory target. `bench/search.rb` reports both timing gates and retained-size measurements.
|
|
129
|
+
|
|
130
|
+
## Name and license
|
|
131
|
+
|
|
132
|
+
A winnower separates grain from chaff; this library separates useful palette matches from a large candidate list. MIT, see [LICENSE.txt](LICENSE.txt). Test-only fzy sources retain their [upstream MIT license](test/vendor/fzy/LICENSE).
|
data/examples/palette.rb
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "winnower"
|
|
4
|
+
|
|
5
|
+
index = Winnower::Index.new(%w[app/models/user.rb app/models/order.rb app/controllers/users_controller.rb README.md])
|
|
6
|
+
session = index.session
|
|
7
|
+
(ARGV.empty? ? %w[a am amu am] : ARGV).each do |query|
|
|
8
|
+
session.query = query
|
|
9
|
+
puts "#{query.inspect}: #{session.count} matches"
|
|
10
|
+
session.top(5).each do |match|
|
|
11
|
+
highlighted = match.candidate.each_char.with_index.map { |character, position| match.positions.include?(position) ? "[#{character}]" : character }.join
|
|
12
|
+
puts " #{highlighted} (#{match.score})"
|
|
13
|
+
end
|
|
14
|
+
end
|
|
@@ -0,0 +1,166 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Winnower
|
|
4
|
+
# Mutable registry; immutable candidate records can be shared by sessions.
|
|
5
|
+
class Index
|
|
6
|
+
attr_reader :options, :generation
|
|
7
|
+
|
|
8
|
+
# @param candidates [Array<String>] initial candidate collection
|
|
9
|
+
# @param options [Options] shared immutable scoring policy
|
|
10
|
+
def initialize(candidates = [], options: DEFAULT_OPTIONS, **settings)
|
|
11
|
+
@options = settings.empty? ? options : Options.new(**options.to_h.merge(settings))
|
|
12
|
+
@entries, @sequence, @generation = {}, 0, 0
|
|
13
|
+
add(candidates)
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
# Add unique text, retaining monotonic registration IDs.
|
|
17
|
+
# @return [self]
|
|
18
|
+
def add(candidates)
|
|
19
|
+
Array(candidates).each do |text|
|
|
20
|
+
next if @entries.key?(text)
|
|
21
|
+
candidate = Candidate.new(text, @sequence)
|
|
22
|
+
next if @entries.key?(candidate.text)
|
|
23
|
+
@entries[candidate.text] = candidate
|
|
24
|
+
@sequence += 1
|
|
25
|
+
@generation += 1
|
|
26
|
+
end
|
|
27
|
+
self
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
# Remove text and invalidate active session histories via generation.
|
|
31
|
+
# @return [self]
|
|
32
|
+
def remove(candidates)
|
|
33
|
+
Array(candidates).each do |text|
|
|
34
|
+
raise ArgumentError, "candidate must be valid text" unless text.is_a?(String) && text.valid_encoding?
|
|
35
|
+
@generation += 1 if @entries.delete(text.encode(Encoding::UTF_8))
|
|
36
|
+
end
|
|
37
|
+
self
|
|
38
|
+
rescue EncodingError => error
|
|
39
|
+
raise ArgumentError, "candidate encoding: #{error.message}"
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
# @return [Integer] number of registered unique candidates
|
|
43
|
+
def size = @entries.size
|
|
44
|
+
# @return [Array<Candidate>] immutable records in registration order
|
|
45
|
+
def candidates = @entries.values
|
|
46
|
+
# @return [Session] independent query history over this index
|
|
47
|
+
def session = Session.new(self)
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
# Incremental match sets and cached top results. A session is thread-confined;
|
|
51
|
+
# separate sessions keep independent query history and reusable DP scratch.
|
|
52
|
+
class Session
|
|
53
|
+
# Session-owned cached match set, scorer and largest requested top list.
|
|
54
|
+
Snapshot = Struct.new(:candidates, :scores, :matcher, :top_limit, :top_matches)
|
|
55
|
+
private_constant :Snapshot
|
|
56
|
+
attr_reader :query
|
|
57
|
+
|
|
58
|
+
# @param index [Index] registry observed for generation changes
|
|
59
|
+
def initialize(index)
|
|
60
|
+
@index, @query, @generation = index, "", -1
|
|
61
|
+
@history = {}
|
|
62
|
+
@shorter = index.options.tie_break == :shorter
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
# Narrow the previous match set or restore a cached query prefix.
|
|
66
|
+
# @param query [String] new query (copied/frozen internally)
|
|
67
|
+
# @return [String] caller's query
|
|
68
|
+
def query=(query)
|
|
69
|
+
raise ArgumentError, "query must be valid text" unless query.is_a?(String) && query.encoding.ascii_compatible? && query.valid_encoding?
|
|
70
|
+
@query = query.encode(Encoding::UTF_8).freeze
|
|
71
|
+
refresh
|
|
72
|
+
query
|
|
73
|
+
rescue EncodingError => error
|
|
74
|
+
raise ArgumentError, "query encoding: #{error.message}"
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
# Select only the best candidates, then reconstruct their highlights.
|
|
78
|
+
# @return [Array<Match>] frozen deterministic result list
|
|
79
|
+
def top(limit = @index.options.limit)
|
|
80
|
+
raise ArgumentError, "limit must be nonnegative" unless limit.is_a?(Integer) && limit >= 0
|
|
81
|
+
refresh
|
|
82
|
+
state = @history.fetch(@query)
|
|
83
|
+
limit = [limit, state.candidates.length].min
|
|
84
|
+
if state.top_matches && limit <= state.top_limit
|
|
85
|
+
return limit == state.top_limit ? state.top_matches : state.top_matches.first(limit).freeze
|
|
86
|
+
end
|
|
87
|
+
candidates, scores = state.candidates, state.scores
|
|
88
|
+
heap = []
|
|
89
|
+
unless limit.zero?
|
|
90
|
+
candidates.each_index do |index|
|
|
91
|
+
if heap.length < limit
|
|
92
|
+
heap << index
|
|
93
|
+
child = heap.length - 1
|
|
94
|
+
while child.positive?
|
|
95
|
+
parent = (child - 1) / 2
|
|
96
|
+
break unless better?(heap[parent], heap[child], candidates, scores)
|
|
97
|
+
heap[parent], heap[child] = heap[child], heap[parent]
|
|
98
|
+
child = parent
|
|
99
|
+
end
|
|
100
|
+
elsif better?(index, heap[0], candidates, scores)
|
|
101
|
+
heap[0] = index
|
|
102
|
+
parent = 0
|
|
103
|
+
loop do
|
|
104
|
+
left = parent * 2 + 1
|
|
105
|
+
break if left >= heap.length
|
|
106
|
+
right = left + 1
|
|
107
|
+
child = right < heap.length && better?(heap[left], heap[right], candidates, scores) ? right : left
|
|
108
|
+
break unless better?(heap[parent], heap[child], candidates, scores)
|
|
109
|
+
heap[parent], heap[child] = heap[child], heap[parent]
|
|
110
|
+
parent = child
|
|
111
|
+
end
|
|
112
|
+
end
|
|
113
|
+
end
|
|
114
|
+
end
|
|
115
|
+
heap.sort! { |a, b| compare(a, b, candidates, scores) }
|
|
116
|
+
state.top_limit = limit
|
|
117
|
+
state.top_matches = heap.map { |index| state.matcher.match(candidates[index]) }.freeze
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
# @return [Integer] complete match count, independent from display limit
|
|
121
|
+
def count
|
|
122
|
+
refresh
|
|
123
|
+
@history.fetch(@query).candidates.length
|
|
124
|
+
end
|
|
125
|
+
|
|
126
|
+
private
|
|
127
|
+
|
|
128
|
+
def better?(a, b, candidates, scores)
|
|
129
|
+
left, right = scores[a], scores[b]
|
|
130
|
+
return left > right unless left == right
|
|
131
|
+
x, y = candidates[a], candidates[b]
|
|
132
|
+
return x.length < y.length if @shorter && x.length != y.length
|
|
133
|
+
x.index < y.index
|
|
134
|
+
end
|
|
135
|
+
|
|
136
|
+
def compare(a, b, candidates, scores)
|
|
137
|
+
comparison = scores[b] <=> scores[a]
|
|
138
|
+
return comparison unless comparison.zero?
|
|
139
|
+
x, y = candidates[a], candidates[b]
|
|
140
|
+
comparison = x.length <=> y.length if @shorter
|
|
141
|
+
comparison.zero? ? x.index <=> y.index : comparison
|
|
142
|
+
end
|
|
143
|
+
|
|
144
|
+
def refresh
|
|
145
|
+
if @generation != @index.generation
|
|
146
|
+
@history.clear
|
|
147
|
+
@generation = @index.generation
|
|
148
|
+
end
|
|
149
|
+
return if @history.key?(@query)
|
|
150
|
+
prefix = @history.keys.select { |key| @query.start_with?(key) }.max_by(&:length)
|
|
151
|
+
candidates = prefix ? @history[prefix].candidates : @index.candidates
|
|
152
|
+
matcher = Matcher.new(@query, @index.options)
|
|
153
|
+
matched, scores = [], []
|
|
154
|
+
candidates.each do |candidate|
|
|
155
|
+
score = matcher.score(candidate)
|
|
156
|
+
next unless score
|
|
157
|
+
matched << candidate
|
|
158
|
+
scores << score
|
|
159
|
+
end
|
|
160
|
+
# Keep only the current prefix chain, including cached top lists so a
|
|
161
|
+
# backspace never has to score or select those results again.
|
|
162
|
+
@history.delete_if { |key, _| !@query.start_with?(key) }
|
|
163
|
+
@history[@query] = Snapshot.new(matched.freeze, scores.freeze, matcher)
|
|
164
|
+
end
|
|
165
|
+
end
|
|
166
|
+
end
|
|
@@ -0,0 +1,448 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Winnower
|
|
4
|
+
# Immutable scoring policy. Defaults preserve fzy's ranking.
|
|
5
|
+
Options = Struct.new(:case_sensitivity, :path_mode, :limit, :tie_break,
|
|
6
|
+
:consecutive, :boundary, :slash, :camel, :dot, :basename_bonus,
|
|
7
|
+
:leading_gap, :inner_gap, :trailing_gap, :case_bonus,
|
|
8
|
+
:max_length, :max_query, keyword_init: true) do
|
|
9
|
+
attr_reader :scoring
|
|
10
|
+
def initialize(case_sensitivity: :smart, path_mode: false, limit: 100,
|
|
11
|
+
tie_break: :shorter, consecutive: 1.0, boundary: 0.8,
|
|
12
|
+
slash: 0.9, camel: 0.7, dot: 0.6, basename_bonus: 0.2,
|
|
13
|
+
leading_gap: -0.005, inner_gap: -0.01, trailing_gap: -0.005,
|
|
14
|
+
case_bonus: 0.0, max_length: 1024, max_query: 256)
|
|
15
|
+
super
|
|
16
|
+
raise ArgumentError, "invalid case sensitivity" unless %i[smart sensitive insensitive].include?(case_sensitivity)
|
|
17
|
+
raise ArgumentError, "invalid tie break" unless %i[shorter index].include?(tie_break)
|
|
18
|
+
raise ArgumentError, "path_mode must be boolean" unless path_mode == true || path_mode == false
|
|
19
|
+
raise ArgumentError, "limit must be nonnegative" unless limit.is_a?(Integer) && limit >= 0
|
|
20
|
+
%i[max_length max_query].each { |name| raise ArgumentError, "#{name} must be positive" unless self[name].is_a?(Integer) && self[name].positive? }
|
|
21
|
+
%i[consecutive boundary slash camel dot basename_bonus leading_gap inner_gap trailing_gap case_bonus].each do |name|
|
|
22
|
+
number = self[name]
|
|
23
|
+
raise ArgumentError, "#{name} must be finite and real" unless number.is_a?(Numeric) && number.real? && number.to_f.finite?
|
|
24
|
+
self[name] = number.to_f
|
|
25
|
+
end
|
|
26
|
+
bonuses = [0.0, self.slash, self.boundary, self.dot, self.camel]
|
|
27
|
+
path = path_mode ? self.basename_bonus : 0.0
|
|
28
|
+
weights = [*bonuses, self.leading_gap, self.inner_gap, self.trailing_gap, self.consecutive, self.case_bonus, path]
|
|
29
|
+
scale = weights.all? { |weight| (weight * 1000).finite? && weight * 1000 == (weight * 1000).round } ? 1000.0 : 1.0
|
|
30
|
+
weights.map! { |weight| (weight * scale).round } if scale == 1000.0
|
|
31
|
+
bonuses = weights.shift(5).freeze
|
|
32
|
+
leading, inner, trailing, consecutive_weight, case_weight, path_weight = weights
|
|
33
|
+
max_bonus = bonuses.max + [case_weight, 0].max + [path_weight, 0].max
|
|
34
|
+
contiguous = leading == trailing && inner <= leading && consecutive_weight >= bonuses.max + [path_weight, 0].max
|
|
35
|
+
@scoring = [scale, bonuses, leading, inner, trailing, consecutive_weight, case_weight, path_weight, max_bonus, contiguous].freeze
|
|
36
|
+
freeze
|
|
37
|
+
end
|
|
38
|
+
end
|
|
39
|
+
# Shared immutable fzy-compatible default policy.
|
|
40
|
+
DEFAULT_OPTIONS = Options.new
|
|
41
|
+
|
|
42
|
+
# Immutable UI result; positions are Unicode character offsets, not bytes.
|
|
43
|
+
Match = Struct.new(:candidate, :score, :positions, :index, keyword_init: true) do
|
|
44
|
+
alias text candidate
|
|
45
|
+
# @return [String] candidate text
|
|
46
|
+
def to_s = candidate
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
# Compact ASCII representation; Unicode keeps per-character folds so an
|
|
50
|
+
# expanding lowercase mapping never shifts the original highlight offsets.
|
|
51
|
+
class Candidate
|
|
52
|
+
# Shared byte used to allocate compact per-character boundary codes.
|
|
53
|
+
ZERO_BYTE = "\0".b.freeze
|
|
54
|
+
attr_reader :text, :chars, :folded, :mask, :unicode, :index, :basename,
|
|
55
|
+
:bonuses, :length, :ascii
|
|
56
|
+
|
|
57
|
+
# @param text [String] candidate text, copied unless already immutable
|
|
58
|
+
# @param index [Integer] registration ID for stable tie resolution
|
|
59
|
+
# @param precompute_mask [Boolean] cache memberships for repeated queries
|
|
60
|
+
def initialize(text, index, precompute_mask: true)
|
|
61
|
+
raise ArgumentError, "candidate must be valid text" unless text.is_a?(String) && text.encoding.ascii_compatible? && text.valid_encoding?
|
|
62
|
+
@text = text.encoding == Encoding::UTF_8 ? (text.frozen? ? text : text.dup.freeze) : text.encode(Encoding::UTF_8).freeze
|
|
63
|
+
@index = index
|
|
64
|
+
@ascii = @text.ascii_only?
|
|
65
|
+
@basename = 0
|
|
66
|
+
low = high = 0
|
|
67
|
+
if @ascii
|
|
68
|
+
@chars = @text
|
|
69
|
+
@folded = @text.match?(/[A-Z]/) ? @text.downcase.freeze : @text
|
|
70
|
+
@length = @text.bytesize
|
|
71
|
+
@unicode = nil
|
|
72
|
+
@bonuses = ZERO_BYTE * @length
|
|
73
|
+
previous = 47
|
|
74
|
+
position = 0
|
|
75
|
+
while position < @length
|
|
76
|
+
original = @text.getbyte(position)
|
|
77
|
+
if precompute_mask
|
|
78
|
+
folded = @folded.getbyte(position)
|
|
79
|
+
if folded < 64
|
|
80
|
+
low |= 1 << folded
|
|
81
|
+
else
|
|
82
|
+
high |= 1 << (folded - 64)
|
|
83
|
+
end
|
|
84
|
+
end
|
|
85
|
+
kind = if previous == 47 || previous == 92
|
|
86
|
+
1
|
|
87
|
+
elsif previous == 45 || previous == 95 || previous == 32
|
|
88
|
+
2
|
|
89
|
+
elsif previous == 46
|
|
90
|
+
3
|
|
91
|
+
elsif previous >= 97 && previous <= 122 && original >= 65 && original <= 90
|
|
92
|
+
4
|
|
93
|
+
else
|
|
94
|
+
0
|
|
95
|
+
end
|
|
96
|
+
@bonuses.setbyte(position, kind)
|
|
97
|
+
@basename = position + 1 if original == 47 || original == 92
|
|
98
|
+
previous = original
|
|
99
|
+
position += 1
|
|
100
|
+
end
|
|
101
|
+
else
|
|
102
|
+
@chars = @text.each_char.map(&:freeze).freeze
|
|
103
|
+
@folded = @chars.map { |character| character.downcase.freeze }.freeze
|
|
104
|
+
@length = @chars.length
|
|
105
|
+
@unicode = precompute_mask ? {} : nil
|
|
106
|
+
@bonuses = ZERO_BYTE * @length
|
|
107
|
+
previous = "/"
|
|
108
|
+
@chars.each_with_index do |original, position|
|
|
109
|
+
folded = @folded[position]
|
|
110
|
+
if precompute_mask
|
|
111
|
+
if folded.ascii_only?
|
|
112
|
+
code = folded.ord
|
|
113
|
+
code < 64 ? low |= 1 << code : high |= 1 << (code - 64)
|
|
114
|
+
else
|
|
115
|
+
@unicode[folded] = true
|
|
116
|
+
end
|
|
117
|
+
end
|
|
118
|
+
kind = if previous == "/" || previous == "\\"
|
|
119
|
+
1
|
|
120
|
+
elsif ["-", "_", " "].include?(previous)
|
|
121
|
+
2
|
|
122
|
+
elsif previous == "."
|
|
123
|
+
3
|
|
124
|
+
elsif previous.match?(/\p{Lower}/) && original.match?(/\p{Upper}/)
|
|
125
|
+
4
|
|
126
|
+
else
|
|
127
|
+
0
|
|
128
|
+
end
|
|
129
|
+
@bonuses.setbyte(position, kind)
|
|
130
|
+
@basename = position + 1 if original == "/" || original == "\\"
|
|
131
|
+
previous = original
|
|
132
|
+
end
|
|
133
|
+
@unicode.freeze
|
|
134
|
+
end
|
|
135
|
+
@mask = precompute_mask ? low | (high << 64) : nil
|
|
136
|
+
@bonuses.freeze
|
|
137
|
+
freeze
|
|
138
|
+
rescue EncodingError => error
|
|
139
|
+
raise ArgumentError, "candidate encoding: #{error.message}"
|
|
140
|
+
end
|
|
141
|
+
end
|
|
142
|
+
|
|
143
|
+
# Sparse form of the fzy D/M recurrence. Only matching character positions
|
|
144
|
+
# need a cell; the maximum across intervening gaps is carried analytically.
|
|
145
|
+
class Matcher
|
|
146
|
+
# Sentinel for unreachable DP states.
|
|
147
|
+
NEGATIVE_INFINITY = -Float::INFINITY
|
|
148
|
+
# Immutable one-byte query strings shared across scorers.
|
|
149
|
+
ASCII_CHARACTERS = Array.new(128) { |code| code.chr(Encoding::UTF_8).freeze }.freeze
|
|
150
|
+
# Shared empty non-ASCII membership list.
|
|
151
|
+
EMPTY_CHARACTERS = [].freeze
|
|
152
|
+
|
|
153
|
+
# @param query [String] query to preprocess once
|
|
154
|
+
# @param options [Options] scoring policy
|
|
155
|
+
def initialize(query, options = DEFAULT_OPTIONS)
|
|
156
|
+
raise ArgumentError, "query must be valid text" unless query.is_a?(String) && query.encoding.ascii_compatible? && query.valid_encoding?
|
|
157
|
+
query = query.encoding == Encoding::UTF_8 ? (query.frozen? ? query : query.dup.freeze) : query.encode(Encoding::UTF_8).freeze
|
|
158
|
+
@original_text = query
|
|
159
|
+
@options = options
|
|
160
|
+
@sensitive = options.case_sensitivity == :sensitive ||
|
|
161
|
+
(options.case_sensitivity == :smart && query.match?(/\p{Upper}/))
|
|
162
|
+
@scale, @bonus_values, @leading, @inner, @trailing, @consecutive, @case_bonus, @path_bonus, @max_bonus, @contiguous_bound = options.scoring
|
|
163
|
+
if query.ascii_only?
|
|
164
|
+
@query_text = @sensitive ? query : query.downcase
|
|
165
|
+
@query = []
|
|
166
|
+
@query_text.each_byte { |code| @query << ASCII_CHARACTERS[code] }
|
|
167
|
+
@original = query.each_byte.map { |code| ASCII_CHARACTERS[code] } unless @case_bonus.zero?
|
|
168
|
+
else
|
|
169
|
+
@original = query.each_char.to_a
|
|
170
|
+
folded = @original.map(&:downcase)
|
|
171
|
+
@query = @sensitive ? @original : folded
|
|
172
|
+
@query_text = @query.join.freeze
|
|
173
|
+
end
|
|
174
|
+
@size = @query.length
|
|
175
|
+
# Keep one object layout even when these caches remain unused.
|
|
176
|
+
@mask = @unicode = @positions_a = @positions_b = @scores_a = @scores_b = nil
|
|
177
|
+
rescue EncodingError => error
|
|
178
|
+
raise ArgumentError, "query encoding: #{error.message}"
|
|
179
|
+
end
|
|
180
|
+
|
|
181
|
+
# @return [Float, nil] optimal score, or nil for a non-match
|
|
182
|
+
def score(candidate)
|
|
183
|
+
value = evaluate(candidate, false)
|
|
184
|
+
raise ArgumentError, "scoring overflow; reduce option weights" if value.is_a?(Float) && value.nan?
|
|
185
|
+
value / @scale if value
|
|
186
|
+
end
|
|
187
|
+
|
|
188
|
+
# Backtracking is only performed for results actually returned to the UI.
|
|
189
|
+
# @return [Match, nil]
|
|
190
|
+
def match(candidate)
|
|
191
|
+
value = evaluate(candidate, true)
|
|
192
|
+
return unless value
|
|
193
|
+
raise ArgumentError, "scoring overflow; reduce option weights" if value.is_a?(Float) && value.nan?
|
|
194
|
+
Match.new(candidate: candidate.text, score: value / @scale, positions: @aligned.freeze, index: candidate.index).freeze
|
|
195
|
+
end
|
|
196
|
+
|
|
197
|
+
private
|
|
198
|
+
|
|
199
|
+
# Membership masks only help indexed candidates. Stateless calls and short
|
|
200
|
+
# queries never inspect them, so avoid building their arbitrary-size integers.
|
|
201
|
+
def prepare_mask
|
|
202
|
+
@unicode = @query_text.ascii_only? ? EMPTY_CHARACTERS : []
|
|
203
|
+
low = high = 0
|
|
204
|
+
@query.each do |character|
|
|
205
|
+
character = character.downcase if @sensitive
|
|
206
|
+
if character.ascii_only?
|
|
207
|
+
code = character.ord
|
|
208
|
+
code < 64 ? low |= 1 << code : high |= 1 << (code - 64)
|
|
209
|
+
else
|
|
210
|
+
@unicode << character
|
|
211
|
+
end
|
|
212
|
+
end
|
|
213
|
+
@mask = low.zero? ? high << 64 : low | (high << 64)
|
|
214
|
+
end
|
|
215
|
+
|
|
216
|
+
def find(chars, needle, offset)
|
|
217
|
+
return chars.index(needle, offset) if chars.is_a?(String)
|
|
218
|
+
while offset < chars.length
|
|
219
|
+
return offset if chars[offset] == needle
|
|
220
|
+
offset += 1
|
|
221
|
+
end
|
|
222
|
+
nil
|
|
223
|
+
end
|
|
224
|
+
|
|
225
|
+
def bonus(candidate, position)
|
|
226
|
+
value = @bonus_values[candidate.bonuses.getbyte(position)]
|
|
227
|
+
@path_bonus.zero? || position < candidate.basename ? value : value + @path_bonus
|
|
228
|
+
end
|
|
229
|
+
|
|
230
|
+
def case_bonus(candidate, position, query_index)
|
|
231
|
+
return 0.0 if @case_bonus.zero?
|
|
232
|
+
character = candidate.ascii ? candidate.text.getbyte(position) : candidate.chars[position]
|
|
233
|
+
expected = candidate.ascii ? @original[query_index].ord : @original[query_index]
|
|
234
|
+
character == expected ? @case_bonus : 0.0
|
|
235
|
+
end
|
|
236
|
+
|
|
237
|
+
def evaluate(candidate, backtrack)
|
|
238
|
+
n, m = candidate.length, @size
|
|
239
|
+
if m.zero?
|
|
240
|
+
@aligned = [] if backtrack
|
|
241
|
+
return 0.0
|
|
242
|
+
end
|
|
243
|
+
return if m > n
|
|
244
|
+
if m > 2 && candidate.mask
|
|
245
|
+
prepare_mask unless @mask
|
|
246
|
+
return if (candidate.mask & @mask) != @mask || @unicode.any? { |character| !candidate.unicode || !candidate.unicode[character] }
|
|
247
|
+
end
|
|
248
|
+
chars = @sensitive ? candidate.chars : candidate.folded
|
|
249
|
+
# A consecutive run starting at the strongest possible boundary reaches
|
|
250
|
+
# the global DP upper bound. This is an exact shortcut, not a heuristic.
|
|
251
|
+
if m > 1 && candidate.ascii && @query_text.ascii_only? && @contiguous_bound
|
|
252
|
+
position = chars.index(@query_text)
|
|
253
|
+
if position && bonus(candidate, position) + @case_bonus == @max_bonus && (@case_bonus.zero? || candidate.text.index(@original_text, position) == position)
|
|
254
|
+
@aligned = (position...(position + m)).to_a if backtrack
|
|
255
|
+
return Float::INFINITY if m == n
|
|
256
|
+
return (n - m) * @leading + bonus(candidate, position) + (m - 1) * @consecutive + m * @case_bonus
|
|
257
|
+
end
|
|
258
|
+
end
|
|
259
|
+
if m == 1
|
|
260
|
+
position = find(chars, @query[0], 0)
|
|
261
|
+
return unless position
|
|
262
|
+
if n == 1
|
|
263
|
+
@aligned = [0] if backtrack
|
|
264
|
+
return Float::INFINITY
|
|
265
|
+
end
|
|
266
|
+
best, finish = NEGATIVE_INFINITY, nil
|
|
267
|
+
while position
|
|
268
|
+
value = bonus(candidate, position)
|
|
269
|
+
value += case_bonus(candidate, position, 0) unless @case_bonus.zero?
|
|
270
|
+
value += @leading == @trailing ? (n - 1) * @leading : position * @leading + (n - position - 1) * @trailing
|
|
271
|
+
if value >= best
|
|
272
|
+
best, finish = value, position
|
|
273
|
+
end
|
|
274
|
+
break if !backtrack && @leading == @trailing && value >= (n - 1) * @leading + @max_bonus
|
|
275
|
+
position = find(chars, @query[0], position + 1)
|
|
276
|
+
end
|
|
277
|
+
@aligned = [finish] if backtrack
|
|
278
|
+
return best
|
|
279
|
+
end
|
|
280
|
+
return two_letters(candidate, chars, backtrack) if m == 2
|
|
281
|
+
# Cheap C-level subsequence checks precede the sparse DP.
|
|
282
|
+
cursor = 0
|
|
283
|
+
unique = n <= @options.max_length && m <= @options.max_query
|
|
284
|
+
total = 0
|
|
285
|
+
aligned = [] if backtrack
|
|
286
|
+
i = 0
|
|
287
|
+
while i < m
|
|
288
|
+
character = @query[i]
|
|
289
|
+
position = find(chars, character, cursor)
|
|
290
|
+
return unless position
|
|
291
|
+
# If each remaining character occurs only once, there is no alignment
|
|
292
|
+
# choice for DP to resolve. Stop checking as soon as one is ambiguous.
|
|
293
|
+
unique &&= find(chars, character, position + 1).nil?
|
|
294
|
+
if unique
|
|
295
|
+
value = bonus(candidate, position)
|
|
296
|
+
value = @consecutive if i.positive? && position == cursor && @consecutive > value
|
|
297
|
+
total += value + (i.zero? ? position * @leading : (position - cursor) * @inner)
|
|
298
|
+
total += case_bonus(candidate, position, i) unless @case_bonus.zero?
|
|
299
|
+
aligned << position if backtrack
|
|
300
|
+
end
|
|
301
|
+
cursor = position + 1
|
|
302
|
+
i += 1
|
|
303
|
+
end
|
|
304
|
+
if m == n
|
|
305
|
+
@aligned = (0...m).to_a if backtrack
|
|
306
|
+
return Float::INFINITY
|
|
307
|
+
end
|
|
308
|
+
if unique
|
|
309
|
+
@aligned = aligned if backtrack
|
|
310
|
+
return total + (n - cursor) * @trailing
|
|
311
|
+
end
|
|
312
|
+
# ponytail: exceptionally long text uses greedy alignment; configurable
|
|
313
|
+
# ceilings bound memory/time without dropping valid subsequence matches.
|
|
314
|
+
return greedy(candidate, chars, backtrack) if n > @options.max_length || m > @options.max_query
|
|
315
|
+
|
|
316
|
+
previous_positions, current_positions = (@positions_a ||= []), (@positions_b ||= [])
|
|
317
|
+
previous_scores, current_scores = (@scores_a ||= []), (@scores_b ||= [])
|
|
318
|
+
previous_positions.clear
|
|
319
|
+
previous_scores.clear
|
|
320
|
+
rows = [] if backtrack
|
|
321
|
+
parents = [] if backtrack
|
|
322
|
+
i = 0
|
|
323
|
+
while i < m
|
|
324
|
+
current_positions.clear
|
|
325
|
+
current_scores.clear
|
|
326
|
+
parent_row = [] if backtrack
|
|
327
|
+
position = find(chars, @query[i], i)
|
|
328
|
+
previous_cursor = 0
|
|
329
|
+
maximum = NEGATIVE_INFINITY
|
|
330
|
+
maximum_index = nil
|
|
331
|
+
while position && position <= n - m + i
|
|
332
|
+
if i.zero?
|
|
333
|
+
value = position * @leading + bonus(candidate, position)
|
|
334
|
+
parent_index = nil
|
|
335
|
+
else
|
|
336
|
+
while previous_cursor < previous_positions.length && previous_positions[previous_cursor] < position
|
|
337
|
+
adjusted = previous_scores[previous_cursor] - previous_positions[previous_cursor] * @inner
|
|
338
|
+
if adjusted >= maximum
|
|
339
|
+
maximum, maximum_index = adjusted, previous_cursor
|
|
340
|
+
end
|
|
341
|
+
previous_cursor += 1
|
|
342
|
+
end
|
|
343
|
+
if maximum_index.nil?
|
|
344
|
+
position = find(chars, @query[i], position + 1)
|
|
345
|
+
next
|
|
346
|
+
end
|
|
347
|
+
separated = maximum + (position - 1) * @inner + bonus(candidate, position)
|
|
348
|
+
adjacent = previous_cursor.positive? && previous_positions[previous_cursor - 1] == position - 1 ? previous_scores[previous_cursor - 1] + @consecutive : NEGATIVE_INFINITY
|
|
349
|
+
if adjacent >= separated
|
|
350
|
+
value, parent_index = adjacent, previous_cursor - 1
|
|
351
|
+
else
|
|
352
|
+
value, parent_index = separated, maximum_index
|
|
353
|
+
end
|
|
354
|
+
end
|
|
355
|
+
value += case_bonus(candidate, position, i) unless @case_bonus.zero?
|
|
356
|
+
current_positions << position
|
|
357
|
+
current_scores << value
|
|
358
|
+
parent_row << parent_index if backtrack
|
|
359
|
+
position = find(chars, @query[i], position + 1)
|
|
360
|
+
end
|
|
361
|
+
if backtrack
|
|
362
|
+
rows << current_positions.dup
|
|
363
|
+
parents << parent_row
|
|
364
|
+
end
|
|
365
|
+
previous_positions, current_positions = current_positions, previous_positions
|
|
366
|
+
previous_scores, current_scores = current_scores, previous_scores
|
|
367
|
+
i += 1
|
|
368
|
+
end
|
|
369
|
+
best, finish = NEGATIVE_INFINITY, nil
|
|
370
|
+
previous_positions.each_with_index do |position, index|
|
|
371
|
+
value = previous_scores[index] + (n - position - 1) * @trailing
|
|
372
|
+
if value >= best
|
|
373
|
+
best, finish = value, index
|
|
374
|
+
end
|
|
375
|
+
end
|
|
376
|
+
if backtrack
|
|
377
|
+
@aligned = Array.new(m)
|
|
378
|
+
(m - 1).downto(0) do |row|
|
|
379
|
+
@aligned[row] = rows[row][finish]
|
|
380
|
+
finish = parents[row][finish]
|
|
381
|
+
end
|
|
382
|
+
end
|
|
383
|
+
best
|
|
384
|
+
end
|
|
385
|
+
|
|
386
|
+
# Two rows collapse to scalar prefix maxima: no temporary arrays, while
|
|
387
|
+
# retaining every alignment and the exact same D/M recurrence.
|
|
388
|
+
def two_letters(candidate, chars, backtrack)
|
|
389
|
+
first = find(chars, @query[0], 0)
|
|
390
|
+
return unless first
|
|
391
|
+
second = find(chars, @query[1], first + 1)
|
|
392
|
+
return unless second
|
|
393
|
+
if candidate.length == 2
|
|
394
|
+
@aligned = [0, 1] if backtrack
|
|
395
|
+
return Float::INFINITY
|
|
396
|
+
end
|
|
397
|
+
maximum = best = NEGATIVE_INFINITY
|
|
398
|
+
maximum_position = last_position = last_score = finish_first = finish_second = nil
|
|
399
|
+
while second
|
|
400
|
+
while first && first < second
|
|
401
|
+
score = first * @leading + bonus(candidate, first)
|
|
402
|
+
score += case_bonus(candidate, first, 0) unless @case_bonus.zero?
|
|
403
|
+
adjusted = score - first * @inner
|
|
404
|
+
if adjusted >= maximum
|
|
405
|
+
maximum, maximum_position = adjusted, first
|
|
406
|
+
end
|
|
407
|
+
last_position, last_score = first, score
|
|
408
|
+
first = find(chars, @query[0], first + 1)
|
|
409
|
+
end
|
|
410
|
+
separated = maximum + (second - 1) * @inner + bonus(candidate, second)
|
|
411
|
+
adjacent = last_position == second - 1 ? last_score + @consecutive : NEGATIVE_INFINITY
|
|
412
|
+
if adjacent >= separated
|
|
413
|
+
score, start = adjacent, last_position
|
|
414
|
+
else
|
|
415
|
+
score, start = separated, maximum_position
|
|
416
|
+
end
|
|
417
|
+
score += case_bonus(candidate, second, 1) unless @case_bonus.zero?
|
|
418
|
+
score += (candidate.length - second - 1) * @trailing
|
|
419
|
+
if score >= best
|
|
420
|
+
best, finish_first, finish_second = score, start, second
|
|
421
|
+
end
|
|
422
|
+
second = find(chars, @query[1], second + 1)
|
|
423
|
+
end
|
|
424
|
+
@aligned = [finish_first, finish_second] if backtrack
|
|
425
|
+
best
|
|
426
|
+
end
|
|
427
|
+
|
|
428
|
+
def greedy(candidate, chars, backtrack)
|
|
429
|
+
cursor = 0
|
|
430
|
+
previous = nil
|
|
431
|
+
total = 0.0
|
|
432
|
+
@aligned = [] if backtrack
|
|
433
|
+
@query.each_with_index do |character, i|
|
|
434
|
+
position = find(chars, character, cursor)
|
|
435
|
+
@aligned << position if backtrack
|
|
436
|
+
total += if previous && position == previous + 1
|
|
437
|
+
@consecutive
|
|
438
|
+
else
|
|
439
|
+
bonus(candidate, position) + (previous ? (position - previous - 1) * @inner : position * @leading)
|
|
440
|
+
end
|
|
441
|
+
total += case_bonus(candidate, position, i)
|
|
442
|
+
previous = position
|
|
443
|
+
cursor = position + 1
|
|
444
|
+
end
|
|
445
|
+
total + (candidate.length - previous - 1) * @trailing
|
|
446
|
+
end
|
|
447
|
+
end
|
|
448
|
+
end
|
data/lib/winnower.rb
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "winnower/version"
|
|
4
|
+
require_relative "winnower/matcher"
|
|
5
|
+
require_relative "winnower/index"
|
|
6
|
+
|
|
7
|
+
# Fuzzy subsequence search for palettes and quick-open interfaces.
|
|
8
|
+
module Winnower
|
|
9
|
+
# Base error class reserved for library-level failures.
|
|
10
|
+
class Error < StandardError; end
|
|
11
|
+
|
|
12
|
+
# Match a query subsequence and return its score and character positions.
|
|
13
|
+
# @param query [String] valid query text
|
|
14
|
+
# @param candidate [String] valid candidate text
|
|
15
|
+
# @param options [Options] immutable scoring policy
|
|
16
|
+
# @return [Match, nil] optimal alignment, or nil if there is no subsequence
|
|
17
|
+
# @raise [ArgumentError] on invalid text/options
|
|
18
|
+
def self.match(query, candidate, options: DEFAULT_OPTIONS, **settings)
|
|
19
|
+
options = Options.new(**options.to_h.merge(settings)) unless settings.empty?
|
|
20
|
+
Matcher.new(query, options).match(Candidate.new(candidate, 0, precompute_mask: false))
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
# Compute a score without reconstructing highlighted positions.
|
|
24
|
+
# @return [Float] score, or negative infinity if the query does not match
|
|
25
|
+
def self.score(query, candidate, options: DEFAULT_OPTIONS, **settings)
|
|
26
|
+
options = Options.new(**options.to_h.merge(settings)) unless settings.empty?
|
|
27
|
+
Matcher.new(query, options).score(Candidate.new(candidate, 0, precompute_mask: false)) || -Float::INFINITY
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
# One-shot ranked filtering. Reuse Index/Session for repeated queries.
|
|
31
|
+
# @return [Array<Match>] at most limit results, sorted deterministically
|
|
32
|
+
def self.filter(query, candidates, limit: nil, **settings)
|
|
33
|
+
index = Index.new(candidates, **settings)
|
|
34
|
+
index.session.tap { |session| session.query = query }.top(limit || index.options.limit)
|
|
35
|
+
end
|
|
36
|
+
end
|
data/sig/winnower.rbs
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
module Winnower
|
|
2
|
+
VERSION: String
|
|
3
|
+
DEFAULT_OPTIONS: Options
|
|
4
|
+
|
|
5
|
+
class Error < StandardError
|
|
6
|
+
end
|
|
7
|
+
|
|
8
|
+
def self.match: (String query, String candidate, ?options: Options, **untyped settings) -> Match?
|
|
9
|
+
def self.score: (String query, String candidate, ?options: Options, **untyped settings) -> Float
|
|
10
|
+
def self.filter: (String query, Array[String] candidates, ?limit: Integer?, **untyped settings) -> Array[Match]
|
|
11
|
+
|
|
12
|
+
class Options < Struct[untyped]
|
|
13
|
+
attr_reader case_sensitivity: Symbol
|
|
14
|
+
attr_reader path_mode: bool
|
|
15
|
+
attr_reader limit: Integer
|
|
16
|
+
attr_reader tie_break: Symbol
|
|
17
|
+
attr_reader consecutive: Float
|
|
18
|
+
attr_reader boundary: Float
|
|
19
|
+
attr_reader slash: Float
|
|
20
|
+
attr_reader camel: Float
|
|
21
|
+
attr_reader dot: Float
|
|
22
|
+
attr_reader basename_bonus: Float
|
|
23
|
+
attr_reader leading_gap: Float
|
|
24
|
+
attr_reader inner_gap: Float
|
|
25
|
+
attr_reader trailing_gap: Float
|
|
26
|
+
attr_reader case_bonus: Float
|
|
27
|
+
attr_reader max_length: Integer
|
|
28
|
+
attr_reader max_query: Integer
|
|
29
|
+
attr_reader scoring: Array[untyped]
|
|
30
|
+
def initialize: (?case_sensitivity: Symbol, ?path_mode: bool, ?limit: Integer, ?tie_break: Symbol,
|
|
31
|
+
?consecutive: Numeric, ?boundary: Numeric, ?slash: Numeric, ?camel: Numeric, ?dot: Numeric,
|
|
32
|
+
?basename_bonus: Numeric, ?leading_gap: Numeric, ?inner_gap: Numeric, ?trailing_gap: Numeric,
|
|
33
|
+
?case_bonus: Numeric, ?max_length: Integer, ?max_query: Integer) -> void
|
|
34
|
+
def to_h: () -> Hash[Symbol, untyped]
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
class Match < Struct[untyped]
|
|
38
|
+
attr_reader candidate: String
|
|
39
|
+
attr_reader score: Float
|
|
40
|
+
attr_reader positions: Array[Integer]
|
|
41
|
+
attr_reader index: Integer
|
|
42
|
+
alias text candidate
|
|
43
|
+
def to_s: () -> String
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
class Candidate
|
|
47
|
+
attr_reader text: String
|
|
48
|
+
attr_reader chars: String | Array[String]
|
|
49
|
+
attr_reader folded: String | Array[String]
|
|
50
|
+
attr_reader mask: Integer?
|
|
51
|
+
attr_reader unicode: Hash[String, bool]?
|
|
52
|
+
attr_reader index: Integer
|
|
53
|
+
attr_reader basename: Integer
|
|
54
|
+
attr_reader bonuses: String
|
|
55
|
+
attr_reader length: Integer
|
|
56
|
+
attr_reader ascii: bool
|
|
57
|
+
def initialize: (String text, Integer index, ?precompute_mask: bool) -> void
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
class Matcher
|
|
61
|
+
def initialize: (String query, ?Options options) -> void
|
|
62
|
+
def score: (Candidate candidate) -> Float?
|
|
63
|
+
def match: (Candidate candidate) -> Match?
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
class Index
|
|
67
|
+
attr_reader options: Options
|
|
68
|
+
attr_reader generation: Integer
|
|
69
|
+
def initialize: (?Array[String] candidates, ?options: Options, **untyped settings) -> void
|
|
70
|
+
def add: (String | Array[String] candidates) -> self
|
|
71
|
+
def remove: (String | Array[String] candidates) -> self
|
|
72
|
+
def size: () -> Integer
|
|
73
|
+
def candidates: () -> Array[Candidate]
|
|
74
|
+
def session: () -> Session
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
class Session
|
|
78
|
+
attr_reader query: String
|
|
79
|
+
def initialize: (Index index) -> void
|
|
80
|
+
def query=: (String query) -> String
|
|
81
|
+
def top: (?Integer limit) -> Array[Match]
|
|
82
|
+
def count: () -> Integer
|
|
83
|
+
end
|
|
84
|
+
end
|
metadata
ADDED
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: winnower
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- ydah
|
|
8
|
+
bindir: exe
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies: []
|
|
12
|
+
email:
|
|
13
|
+
- t.yudai92@gmail.com
|
|
14
|
+
executables: []
|
|
15
|
+
extensions: []
|
|
16
|
+
extra_rdoc_files: []
|
|
17
|
+
files:
|
|
18
|
+
- CHANGELOG.md
|
|
19
|
+
- LICENSE.txt
|
|
20
|
+
- README.md
|
|
21
|
+
- examples/palette.rb
|
|
22
|
+
- lib/winnower.rb
|
|
23
|
+
- lib/winnower/index.rb
|
|
24
|
+
- lib/winnower/matcher.rb
|
|
25
|
+
- lib/winnower/version.rb
|
|
26
|
+
- sig/winnower.rbs
|
|
27
|
+
homepage: https://github.com/rubifex/winnower
|
|
28
|
+
licenses:
|
|
29
|
+
- MIT
|
|
30
|
+
metadata:
|
|
31
|
+
source_code_uri: https://github.com/rubifex/winnower
|
|
32
|
+
changelog_uri: https://github.com/rubifex/winnower/blob/main/CHANGELOG.md
|
|
33
|
+
allowed_push_host: https://rubygems.org
|
|
34
|
+
rubygems_mfa_required: 'true'
|
|
35
|
+
rdoc_options: []
|
|
36
|
+
require_paths:
|
|
37
|
+
- lib
|
|
38
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
39
|
+
requirements:
|
|
40
|
+
- - ">="
|
|
41
|
+
- !ruby/object:Gem::Version
|
|
42
|
+
version: '3.1'
|
|
43
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
44
|
+
requirements:
|
|
45
|
+
- - ">="
|
|
46
|
+
- !ruby/object:Gem::Version
|
|
47
|
+
version: '0'
|
|
48
|
+
requirements: []
|
|
49
|
+
rubygems_version: 4.0.19
|
|
50
|
+
specification_version: 4
|
|
51
|
+
summary: Deterministic fuzzy subsequence matching with incremental filtering
|
|
52
|
+
test_files: []
|