sec_id 7.0.0 → 7.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +6 -0
- data/README.md +38 -2
- data/lib/sec_id/cei.rb +1 -0
- data/lib/sec_id/concerns/suggestable.rb +168 -0
- data/lib/sec_id/cusip.rb +1 -0
- data/lib/sec_id/dti.rb +1 -0
- data/lib/sec_id/figi.rb +1 -0
- data/lib/sec_id/iban.rb +1 -0
- data/lib/sec_id/isin.rb +1 -0
- data/lib/sec_id/lei.rb +1 -0
- data/lib/sec_id/sedol.rb +1 -0
- data/lib/sec_id/suggestion.rb +59 -0
- data/lib/sec_id/upi.rb +1 -0
- data/lib/sec_id/version.rb +1 -1
- data/lib/sec_id.rb +34 -0
- data/sec_id.gemspec +4 -3
- data/sig/sec_id/base.rbs +9 -3
- data/sig/sec_id/concerns/checkable.rbs +2 -1
- data/sig/sec_id/concerns/suggestable.rbs +32 -0
- data/sig/sec_id/suggestion.rbs +23 -0
- data/sig/sec_id.rbs +2 -0
- metadata +9 -4
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2ce3f9d548d4530dae973ee4ee4a7cf8e18898aaa36396423efe5ece7dd091f4
|
|
4
|
+
data.tar.gz: 5b7950a09cfc4afd59358b230fd659a9f26a1e58e50577ed16267f4763142a7f
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 34447319dc249c2fb3b7150cc45cc4a2aa8eed8e11ab16b726d9ee73d7aa2ad01daa2eed846f0cc788fb9f955488b161a91112ea6ec683268af7fa3fdf5267e0
|
|
7
|
+
data.tar.gz: 155fd8fe486ea96af91949ddd1adacb04310f50c820a2a5c1d98f1f55075545886c3935adfa5dacf2eedc9975ca7f1879c0e1140a8165dcd60736f24a0bcea94
|
data/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,12 @@ and [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/).
|
|
|
8
8
|
|
|
9
9
|
## [Unreleased]
|
|
10
10
|
|
|
11
|
+
## [7.1.0] - 2026-07-15
|
|
12
|
+
|
|
13
|
+
### Added
|
|
14
|
+
|
|
15
|
+
- `suggest` — a checksum-anchored diagnostic-repair verb that turns the checksum from a gatekeeper into a repair engine. For a structurally-valid but checksum-**failing** identifier, `SecID::<Type>.suggest(str)` and `SecID.suggest(str, types:)` enumerate the plausible single-character human errors — visual/OCR homoglyph substitutions (`O`↔`0`, `I`↔`1`, `5`↔`S`, `8`↔`B`, …) and adjacent transpositions — keep only those that re-validate (`valid?` is the oracle, so no false candidate ever escapes), and return them as confidence-ranked `SecID::Suggestion` value objects reporting *what changed* (edit kind, position, from/to characters, confidence tier). Available for all 9 checksum types (ISIN, CUSIP, SEDOL, FIGI, LEI, IBAN, CEI, DTI, UPI) through one shared `Suggestable` concern. Candidates rank `:high` (homoglyph) then `:medium` (transposition) then a `:checksum` recompute fallback last; there is no `:low` tier (coincidental substitutions are never generated), keeping results small and high-precision. `suggest` never mutates its input and never presents a candidate as an authoritative correction. Non-checksum types (CIK, OCC, WKN, Valoren, CFI, FISN, BIC) have no checksum oracle and are unsupported; `SecID.suggest` silently skips them. Known limitations: the mistyped character must be in the type's charset to be reachable (the vowel-free SEDOL/FIGI/DTI/UPI can't repair an `O`-for-`0`/`I`-for-`1` typo), only a single body error is in scope, and wrong-length input returns `[]`. In a seeded simulation (`benchmark/suggest_precision.rb`), every reachable single-error homoglyph or transposition that yields a checksum-failing identifier is recovered (the correct identifier is always among the returned candidates), the top-ranked body candidate ~89% of the time for homoglyph errors
|
|
16
|
+
|
|
11
17
|
## [7.0.0] - 2026-07-14
|
|
12
18
|
|
|
13
19
|
### Added
|
data/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# SecID [](https://rubygems.org/gems/sec_id) [](https://github.com/svyatov/sec_id/actions/workflows/main.yml) [](https://codecov.io/gh/svyatov/sec_id) [](https://rubydoc.info/gems/sec_id) [](https://www.ruby-lang.org) [](https://github.com/svyatov/sec_id/tree/main/sig)
|
|
2
2
|
|
|
3
|
-
> A Ruby toolkit for securities identifiers — validate, parse, normalize, detect, convert, generate, and
|
|
3
|
+
> A Ruby toolkit for securities identifiers — validate, parse, normalize, detect, convert, generate, classify, and repair.
|
|
4
4
|
|
|
5
5
|
## Table of Contents
|
|
6
6
|
|
|
@@ -13,6 +13,7 @@
|
|
|
13
13
|
- [Structured Validation](#structured-validation) - detailed error codes and messages
|
|
14
14
|
- [Pattern Matching](#pattern-matching) - destructure identifiers with `case/in`
|
|
15
15
|
- [Generating Test Fixtures](#generating-test-fixtures) - produce valid identifiers for tests
|
|
16
|
+
- [Repairing Typos](#repairing-typos) - suggest corrections for checksum-failing identifiers
|
|
16
17
|
- [ISIN](#isin) - International Securities Identification Number
|
|
17
18
|
- [CUSIP](#cusip) - Committee on Uniform Securities Identification Procedures
|
|
18
19
|
- [CEI](#cei) - CUSIP Entity Identifier
|
|
@@ -47,7 +48,7 @@ Ruby 3.2+ is required.
|
|
|
47
48
|
Add this line to your application's Gemfile:
|
|
48
49
|
|
|
49
50
|
```ruby
|
|
50
|
-
gem 'sec_id', '~> 7.
|
|
51
|
+
gem 'sec_id', '~> 7.1'
|
|
51
52
|
```
|
|
52
53
|
|
|
53
54
|
And then execute:
|
|
@@ -328,6 +329,41 @@ SecID::LEI.generate(random: Random.new(42)) == SecID::LEI.generate(random: Rando
|
|
|
328
329
|
> randomly selected (from the values each standard permits) and do not map to real-world
|
|
329
330
|
> instruments. Use them as test fixtures, not as references to actual securities.
|
|
330
331
|
|
|
332
|
+
### Repairing Typos
|
|
333
|
+
|
|
334
|
+
Turn the checksum from a gatekeeper into a repair engine. For a checksum-**failing** identifier, `suggest` enumerates the plausible single-character human errors — visual/OCR homoglyph substitutions (`O`↔`0`, `I`↔`1`, `5`↔`S`, `8`↔`B`, …) and adjacent transpositions — keeps only the edits that re-validate, and returns them as confidence-ranked `SecID::Suggestion` candidates that report *what changed*. Available for all 9 checksum types (ISIN, CUSIP, SEDOL, FIGI, LEI, IBAN, CEI, DTI, UPI) per class and via the central dispatcher:
|
|
335
|
+
|
|
336
|
+
```ruby
|
|
337
|
+
# A letter O typed where a 0 belongs — 'US5949181O45' should be 'US5949181045'
|
|
338
|
+
top = SecID::ISIN.suggest('US5949181O45').first
|
|
339
|
+
top.to_s # => 'US5949181045' (the corrected identifier)
|
|
340
|
+
top.edit # => :substitution
|
|
341
|
+
top.position # => 9
|
|
342
|
+
top.from # => 'O'
|
|
343
|
+
top.to # => '0'
|
|
344
|
+
top.confidence # => :high
|
|
345
|
+
top.identifier # => #<SecID::ISIN ...> (parsed and valid — call .country_code, .to_h, etc.)
|
|
346
|
+
|
|
347
|
+
# Module-level: infers every format-compatible checksum type (like parse / detect)
|
|
348
|
+
SecID.suggest('US5949181O45') # => [#<SecID::Suggestion type=:isin ...>, ...]
|
|
349
|
+
SecID.suggest('US5949181O45', types: [:isin]) # => restrict to specific types
|
|
350
|
+
```
|
|
351
|
+
|
|
352
|
+
Each candidate carries the corrected `identifier` (a parsed, valid instance), the `edit` kind, its `position`, the `from`/`to` characters, and a `confidence` tier. Candidates are ranked by confidence: `:high` homoglyph substitutions first, then `:medium` adjacent transpositions, then the `:checksum` recompute (body assumed correct, wrong check character) last as a fallback hypothesis. **There is no `:low` tier** — coincidental substitutions that merely satisfy the checksum are never generated, keeping the result small and high-precision.
|
|
353
|
+
|
|
354
|
+
> **`suggest` returns candidates, never authoritative corrections, and never mutates its input.**
|
|
355
|
+
> Every returned candidate fully re-validates (`valid?` is the oracle), so no false candidate escapes —
|
|
356
|
+
> but the `confidence` tier is how *you* decide what to trust. This matters for financial identifiers.
|
|
357
|
+
|
|
358
|
+
Notes and limitations:
|
|
359
|
+
|
|
360
|
+
- **Never empty for structurally-valid input** — a parseable but checksum-failing identifier always yields at least the `:checksum` fallback. Only wrong-length or illegal-charset input (which fails the format gate) returns `[]`.
|
|
361
|
+
- **Vowel-free reachability** — SEDOL, FIGI, DTI, and UPI exclude vowels from their charset, so an `O`-for-`0` or `I`-for-`1` typo is unparseable and therefore unrepairable (it fails the format gate before enumeration). The mistyped character must be *in* the type's charset to be reachable.
|
|
362
|
+
- **Single body error only** — two or more wrong body characters, or a dropped/doubled character (insertion/deletion), are out of scope; such input returns only the `:checksum` fallback, never additional body candidates.
|
|
363
|
+
- **Non-checksum types are unsupported** — CIK, OCC, WKN, Valoren, CFI, FISN, and BIC have no checksum oracle, so they have no `suggest`; `SecID.suggest` silently skips them.
|
|
364
|
+
|
|
365
|
+
**Precision** (simulated over 1,000 seeded samples per checksum type — see [`benchmark/suggest_precision.rb`](benchmark/suggest_precision.rb)): every reachable, single-error homoglyph or transposition that yields a checksum-failing identifier is recovered — the correct identifier is **always** among the returned candidates (100%), and is the top-ranked body candidate ~89% of the time for homoglyph errors. In-charset homoglyph reachability averages ~96% (100% for the letter-permitting types, lower for the four vowel-free ones).
|
|
366
|
+
|
|
331
367
|
### ISIN
|
|
332
368
|
|
|
333
369
|
> [International Securities Identification Number](https://en.wikipedia.org/wiki/International_Securities_Identification_Number) - a 12-character alphanumeric code that uniquely identifies a security.
|
data/lib/sec_id/cei.rb
CHANGED
|
@@ -0,0 +1,168 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module SecID
|
|
4
|
+
# Repair engine for checksum-failing identifiers: enumerates plausible single-character
|
|
5
|
+
# edits (homoglyph/OCR substitutions and adjacent transpositions), keeps only those that
|
|
6
|
+
# re-validate, and returns them as confidence-ranked {Suggestion} objects.
|
|
7
|
+
#
|
|
8
|
+
# Included by the 9 checksum types. The candidate space is the human-error net —
|
|
9
|
+
# {HOMOGLYPHS} plus adjacent transpositions — never a full coincidental net. `valid?`
|
|
10
|
+
# is the oracle, so no false candidate ever escapes; the {HOMOGLYPHS} table bounds recall.
|
|
11
|
+
#
|
|
12
|
+
# @see SecID::Suggestion
|
|
13
|
+
#
|
|
14
|
+
# @example
|
|
15
|
+
# SecID::ISIN.suggest('US5949181O45')
|
|
16
|
+
# #=> [#<SecID::Suggestion edit=:substitution ...>, #<SecID::Suggestion edit=:checksum ...>]
|
|
17
|
+
module Suggestable
|
|
18
|
+
# Homoglyph / OCR confusion table (KTD5 seed): each character maps to the characters
|
|
19
|
+
# it is commonly mistyped as or misread from. A bidirectional expansion of the seed
|
|
20
|
+
# pairs {0/O, 0/Q, 0/D, 1/I, 1/L, I/L, 2/Z, 5/S, 6/G, 8/B}. Membership bounds recall —
|
|
21
|
+
# widen it (see benchmark/suggest_precision.rb) to raise it.
|
|
22
|
+
HOMOGLYPHS = {
|
|
23
|
+
'0' => %w[O Q D], 'O' => %w[0], 'Q' => %w[0], 'D' => %w[0],
|
|
24
|
+
'1' => %w[I L], 'I' => %w[1 L], 'L' => %w[1 I],
|
|
25
|
+
'2' => %w[Z], 'Z' => %w[2], '5' => %w[S], 'S' => %w[5],
|
|
26
|
+
'6' => %w[G], 'G' => %w[6], '8' => %w[B], 'B' => %w[8]
|
|
27
|
+
}.freeze
|
|
28
|
+
|
|
29
|
+
# Rank weight per edit kind: body substitutions first, then adjacent transpositions,
|
|
30
|
+
# then the checksum-recompute fallback last (R6/R7).
|
|
31
|
+
RANK = { substitution: 0, transposition: 1, checksum: 2 }.freeze
|
|
32
|
+
|
|
33
|
+
# @api private
|
|
34
|
+
def self.included(base)
|
|
35
|
+
base.extend(ClassMethods)
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
# Class methods added when Suggestable is included.
|
|
39
|
+
module ClassMethods
|
|
40
|
+
# Returns confidence-ranked repair candidates for a checksum-failing identifier.
|
|
41
|
+
#
|
|
42
|
+
# @param str [String] the identifier string to repair
|
|
43
|
+
# @return [Array<Suggestion>] ranked candidates; empty when `str` is already valid
|
|
44
|
+
# or fails the type's format (wrong length or charset)
|
|
45
|
+
def suggest(str)
|
|
46
|
+
new(str).suggest
|
|
47
|
+
end
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
# Enumerates plausible edits and returns the re-validating ones, confidence-ranked.
|
|
51
|
+
#
|
|
52
|
+
# @return [Array<Suggestion>] ranked candidates; empty when already valid or unparseable
|
|
53
|
+
def suggest
|
|
54
|
+
return [] if valid?
|
|
55
|
+
return [] unless valid_format?
|
|
56
|
+
|
|
57
|
+
surviving = candidate_strings.uniq.map { |candidate| self.class.new(candidate) }.select(&:valid?)
|
|
58
|
+
surviving.filter_map { |candidate| classify(candidate) }.sort_by { |suggestion| RANK.fetch(suggestion.edit) }
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
private
|
|
62
|
+
|
|
63
|
+
# Homoglyph substitutions + adjacent transpositions + the checksum recompute (R4, R6).
|
|
64
|
+
#
|
|
65
|
+
# @return [Array<String>] candidate identifier strings (unfiltered, may include duplicates)
|
|
66
|
+
def candidate_strings
|
|
67
|
+
homoglyph_candidates + transposition_candidates + [restore]
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
# @return [Array<String>]
|
|
71
|
+
def homoglyph_candidates
|
|
72
|
+
full_id.each_char.with_index.flat_map do |char, index|
|
|
73
|
+
Array(HOMOGLYPHS[char]).map { |replacement| replace_at(full_id, index, replacement) }
|
|
74
|
+
end
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
# @return [Array<String>]
|
|
78
|
+
def transposition_candidates
|
|
79
|
+
(0...(full_id.length - 1)).filter_map do |index|
|
|
80
|
+
swap_at(full_id, index) unless full_id[index] == full_id[index + 1]
|
|
81
|
+
end
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
# @param str [String]
|
|
85
|
+
# @param index [Integer]
|
|
86
|
+
# @param char [String] the replacement character
|
|
87
|
+
# @return [String]
|
|
88
|
+
def replace_at(str, index, char)
|
|
89
|
+
str.dup.tap { |copy| copy[index] = char }
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
# @param str [String]
|
|
93
|
+
# @param index [Integer] left index of the adjacent pair to swap
|
|
94
|
+
# @return [String]
|
|
95
|
+
def swap_at(str, index)
|
|
96
|
+
str.dup.tap { |copy| copy[index], copy[index + 1] = copy[index + 1], copy[index] }
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
# Tags a surviving candidate by comparing bodies then diffing the strings (KTD3):
|
|
100
|
+
# equal bodies mean only the check characters changed.
|
|
101
|
+
#
|
|
102
|
+
# @param candidate [SecID::Base] a re-validated candidate instance
|
|
103
|
+
# @return [Suggestion, nil] nil when the diff is neither a single substitution nor an adjacent swap
|
|
104
|
+
def classify(candidate)
|
|
105
|
+
return checksum_suggestion(candidate) if candidate.identifier == identifier
|
|
106
|
+
|
|
107
|
+
changed = changed_indices(candidate)
|
|
108
|
+
case changed.size
|
|
109
|
+
when 1 then substitution_suggestion(candidate, changed.first.to_i)
|
|
110
|
+
when 2 then transposition_suggestion(candidate, changed)
|
|
111
|
+
end
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
# @param candidate [SecID::Base]
|
|
115
|
+
# @return [Array<Integer>] indices where the candidate's string differs from the input's
|
|
116
|
+
def changed_indices(candidate)
|
|
117
|
+
other = candidate.full_id
|
|
118
|
+
(0...full_id.length).reject { |index| full_id[index] == other[index] }
|
|
119
|
+
end
|
|
120
|
+
|
|
121
|
+
# @param candidate [SecID::Base]
|
|
122
|
+
# @param index [Integer]
|
|
123
|
+
# @return [Suggestion]
|
|
124
|
+
def substitution_suggestion(candidate, index)
|
|
125
|
+
Suggestion.new(
|
|
126
|
+
type: self.class.type_key, identifier: candidate, edit: :substitution,
|
|
127
|
+
position: index, from: full_id[index].to_s, to: candidate.full_id[index].to_s, confidence: :high
|
|
128
|
+
)
|
|
129
|
+
end
|
|
130
|
+
|
|
131
|
+
# @param candidate [SecID::Base]
|
|
132
|
+
# @param indices [Array<Integer>] the two differing indices
|
|
133
|
+
# @return [Suggestion, nil] nil unless the two indices are an adjacent, mutual swap
|
|
134
|
+
def transposition_suggestion(candidate, indices)
|
|
135
|
+
left = indices.min.to_i
|
|
136
|
+
right = indices.max.to_i
|
|
137
|
+
return unless right == left + 1 && swap?(candidate, left, right)
|
|
138
|
+
|
|
139
|
+
Suggestion.new(
|
|
140
|
+
type: self.class.type_key, identifier: candidate, edit: :transposition,
|
|
141
|
+
position: left, from: full_id[left, 2].to_s, to: candidate.full_id[left, 2].to_s, confidence: :medium
|
|
142
|
+
)
|
|
143
|
+
end
|
|
144
|
+
|
|
145
|
+
# @param candidate [SecID::Base]
|
|
146
|
+
# @param left [Integer]
|
|
147
|
+
# @param right [Integer]
|
|
148
|
+
# @return [Boolean] true when the two positions are a mutual swap of the input's characters
|
|
149
|
+
def swap?(candidate, left, right)
|
|
150
|
+
full_id[left] == candidate.full_id[right] && full_id[right] == candidate.full_id[left]
|
|
151
|
+
end
|
|
152
|
+
|
|
153
|
+
# @param candidate [SecID::Base]
|
|
154
|
+
# @return [Suggestion]
|
|
155
|
+
def checksum_suggestion(candidate)
|
|
156
|
+
Suggestion.new(
|
|
157
|
+
type: self.class.type_key, identifier: candidate, edit: :checksum, position: nil,
|
|
158
|
+
from: rendered_checksum(checksum), to: rendered_checksum(candidate.checksum), confidence: nil
|
|
159
|
+
)
|
|
160
|
+
end
|
|
161
|
+
|
|
162
|
+
# @param value [Integer, String, nil] a checksum value
|
|
163
|
+
# @return [String] the check-character string, zero-padded to the checksum width ('' when absent)
|
|
164
|
+
def rendered_checksum(value)
|
|
165
|
+
value.nil? ? '' : value.to_s.rjust(checksum_width, '0')
|
|
166
|
+
end
|
|
167
|
+
end
|
|
168
|
+
end
|
data/lib/sec_id/cusip.rb
CHANGED
data/lib/sec_id/dti.rb
CHANGED
data/lib/sec_id/figi.rb
CHANGED
data/lib/sec_id/iban.rb
CHANGED
data/lib/sec_id/isin.rb
CHANGED
data/lib/sec_id/lei.rb
CHANGED
data/lib/sec_id/sedol.rb
CHANGED
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module SecID
|
|
4
|
+
# Immutable value object representing one repair candidate returned by `suggest`.
|
|
5
|
+
#
|
|
6
|
+
# Carries the corrected, re-validated identifier plus the what-changed diff (edit
|
|
7
|
+
# kind, position, from/to characters) and a confidence tier — the honest surface
|
|
8
|
+
# that keeps a candidate from reading as an authoritative correction.
|
|
9
|
+
#
|
|
10
|
+
# Coordinate system by edit kind:
|
|
11
|
+
# - `:substitution` — `position` is the 0-based index of the changed character;
|
|
12
|
+
# `from`/`to` are the single characters before/after.
|
|
13
|
+
# - `:transposition` — `position` is the left index of the swapped pair;
|
|
14
|
+
# `from`/`to` are the two-character adjacent substring before/after the swap.
|
|
15
|
+
# - `:checksum` — `position` is `nil`; `from`/`to` are the old/new check-character
|
|
16
|
+
# string. `confidence` is `nil` (it is the fallback hypothesis, ranked last).
|
|
17
|
+
#
|
|
18
|
+
# @!attribute [r] type
|
|
19
|
+
# @return [Symbol] the identifier type key (e.g. :isin)
|
|
20
|
+
# @!attribute [r] identifier
|
|
21
|
+
# @return [SecID::Base] the valid, parsed corrected identifier
|
|
22
|
+
# @!attribute [r] edit
|
|
23
|
+
# @return [Symbol] :substitution, :transposition, or :checksum
|
|
24
|
+
# @!attribute [r] position
|
|
25
|
+
# @return [Integer, nil] the edit position (nil for :checksum)
|
|
26
|
+
# @!attribute [r] from
|
|
27
|
+
# @return [String] the original character(s)
|
|
28
|
+
# @!attribute [r] to
|
|
29
|
+
# @return [String] the replacement character(s)
|
|
30
|
+
# @!attribute [r] confidence
|
|
31
|
+
# @return [Symbol, nil] :high (homoglyph), :medium (transposition), or nil (:checksum)
|
|
32
|
+
#
|
|
33
|
+
# @example
|
|
34
|
+
# suggestion = SecID::ISIN.suggest('US5949181O45').first
|
|
35
|
+
# suggestion.to_s #=> 'US5949181045'
|
|
36
|
+
# suggestion.edit #=> :substitution
|
|
37
|
+
# suggestion.confidence #=> :high
|
|
38
|
+
Suggestion = Data.define(:type, :identifier, :edit, :position, :from, :to, :confidence)
|
|
39
|
+
|
|
40
|
+
# Reopened to add the domain enumerations and serialization helpers per the repo
|
|
41
|
+
# convention (see {Base#as_json}).
|
|
42
|
+
class Suggestion
|
|
43
|
+
# The edit kinds `edit` can take, in rank order (see {Suggestable::RANK}).
|
|
44
|
+
EDIT_KINDS = %i[substitution transposition checksum].freeze
|
|
45
|
+
|
|
46
|
+
# The non-nil confidence tiers `confidence` can take; `:checksum` candidates carry `nil`.
|
|
47
|
+
CONFIDENCE_LEVELS = %i[high medium].freeze
|
|
48
|
+
|
|
49
|
+
# @return [Hash] the symbol-keyed field hash (JSON-compatible)
|
|
50
|
+
def as_json(*)
|
|
51
|
+
to_h
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
# @return [String] the corrected identifier string
|
|
55
|
+
def to_s
|
|
56
|
+
identifier.to_s
|
|
57
|
+
end
|
|
58
|
+
end
|
|
59
|
+
end
|
data/lib/sec_id/upi.rb
CHANGED
data/lib/sec_id/version.rb
CHANGED
data/lib/sec_id.rb
CHANGED
|
@@ -143,8 +143,40 @@ module SecID
|
|
|
143
143
|
self[key].generate(random: random)
|
|
144
144
|
end
|
|
145
145
|
|
|
146
|
+
# Returns confidence-ranked repair candidates for a checksum-failing identifier,
|
|
147
|
+
# inferred across all format-compatible checksum types (or the given `types:`).
|
|
148
|
+
#
|
|
149
|
+
# @note Repair is only defined for checksum types; non-checksum types have no
|
|
150
|
+
# oracle and are silently skipped.
|
|
151
|
+
#
|
|
152
|
+
# @param str [String, nil] the identifier string to repair
|
|
153
|
+
# @param types [Array<Symbol>, nil] restrict to specific types (non-checksum types are ignored)
|
|
154
|
+
# @return [Array<Suggestion>] cross-type ranked candidates, each carrying its `type`;
|
|
155
|
+
# empty when no format-compatible checksum type matches
|
|
156
|
+
# @raise [ArgumentError] if any key in types is unknown
|
|
157
|
+
def suggest(str, types: nil)
|
|
158
|
+
input = str.to_s.strip.upcase
|
|
159
|
+
suggestable_types(input, types)
|
|
160
|
+
.flat_map { |klass| klass.suggest(input) }
|
|
161
|
+
.sort_by { |suggestion| [Suggestable::RANK.fetch(suggestion.edit), self[suggestion.type].detection_priority.last] }
|
|
162
|
+
end
|
|
163
|
+
|
|
146
164
|
private
|
|
147
165
|
|
|
166
|
+
# Resolves the checksum types eligible to repair `input` (KTD4): the given `types:`
|
|
167
|
+
# narrowed to checksum types, or every checksum type whose length and charset admit it.
|
|
168
|
+
#
|
|
169
|
+
# @param input [String] the normalized (stripped, upcased) input
|
|
170
|
+
# @param types [Array<Symbol>, nil]
|
|
171
|
+
# @return [Array<Class>]
|
|
172
|
+
def suggestable_types(input, types)
|
|
173
|
+
return types.map { |key| self[key] }.select(&:has_checksum?) if types
|
|
174
|
+
|
|
175
|
+
identifier_list.select do |klass|
|
|
176
|
+
klass.has_checksum? && klass.length_values.include?(input.length) && input.match?(klass::VALID_CHARS_REGEX)
|
|
177
|
+
end
|
|
178
|
+
end
|
|
179
|
+
|
|
148
180
|
# @return [void]
|
|
149
181
|
def register_identifier(klass)
|
|
150
182
|
identifier_map[klass.type_key] = klass
|
|
@@ -206,7 +238,9 @@ require 'sec_id/concerns/normalizable'
|
|
|
206
238
|
require 'sec_id/concerns/validatable'
|
|
207
239
|
require 'sec_id/concerns/checkable'
|
|
208
240
|
require 'sec_id/concerns/generatable'
|
|
241
|
+
require 'sec_id/concerns/suggestable'
|
|
209
242
|
require 'sec_id/base'
|
|
243
|
+
require 'sec_id/suggestion'
|
|
210
244
|
require 'sec_id/detector'
|
|
211
245
|
require 'sec_id/scanner'
|
|
212
246
|
require 'sec_id/isin'
|
data/sec_id.gemspec
CHANGED
|
@@ -11,9 +11,10 @@ Gem::Specification.new do |spec|
|
|
|
11
11
|
spec.email = ['leonid@svyatov.ru']
|
|
12
12
|
|
|
13
13
|
spec.summary = 'A Ruby toolkit for securities identifiers — validate, parse, normalize, detect, convert, ' \
|
|
14
|
-
'generate, and
|
|
15
|
-
spec.description = 'Validate, normalize, parse, convert, generate, and
|
|
16
|
-
'Auto-detect identifier type from any string. Calculate and restore checksums.
|
|
14
|
+
'generate, classify, and repair.'
|
|
15
|
+
spec.description = 'Validate, normalize, parse, convert, generate, classify, and repair securities identifiers. ' \
|
|
16
|
+
'Auto-detect identifier type from any string. Calculate and restore checksums. Suggest ' \
|
|
17
|
+
'confidence-ranked corrections for fat-fingered or OCR-mangled identifiers. Generate ' \
|
|
17
18
|
'format-valid identifiers as test fixtures. Decode CFI codes to full ISO 10962:2021 ' \
|
|
18
19
|
'classifications. Includes an opt-in ActiveModel/Rails validator that adds no runtime ' \
|
|
19
20
|
'dependency. Supports ISIN, CUSIP, CEI, SEDOL, FIGI, LEI, IBAN, CIK, OCC, WKN, Valoren, ' \
|
data/sig/sec_id/base.rbs
CHANGED
|
@@ -43,6 +43,9 @@ module SecID
|
|
|
43
43
|
def self.length_specificity: () -> (Integer | Float | nil)
|
|
44
44
|
def self.example: () -> String
|
|
45
45
|
def self.has_checksum?: () -> bool
|
|
46
|
+
# Only Suggestable (checksum) types implement `.suggest`; declared here so the
|
|
47
|
+
# module-level `SecID.suggest` can call it on any registered class object.
|
|
48
|
+
def self.suggest: (SecID::input str) -> Array[Suggestion]
|
|
46
49
|
|
|
47
50
|
# @deprecated v7 bridge alias for `has_checksum?`; removed in v8.
|
|
48
51
|
def self.has_check_digit?: () -> bool
|
|
@@ -58,12 +61,15 @@ module SecID
|
|
|
58
61
|
def as_json: (*untyped) -> Hash[Symbol, untyped]
|
|
59
62
|
def deconstruct_keys: (::Array[Symbol]? _keys) -> Hash[Symbol, untyped]
|
|
60
63
|
|
|
61
|
-
# Checksum surface. Only the types that `include Checkable`
|
|
62
|
-
# (they raise/NoMethodError otherwise);
|
|
63
|
-
#
|
|
64
|
+
# Checksum / repair surface. Only the types that `include Checkable` /
|
|
65
|
+
# `include Suggestable` implement these (they raise/NoMethodError otherwise);
|
|
66
|
+
# declared on Base so `Generatable#generate`, `Checkable::ClassMethods`, and the
|
|
67
|
+
# Base-self-typed `Suggestable` body type-check without narrowing on `has_checksum?`.
|
|
64
68
|
def restore: () -> String
|
|
65
69
|
def restore!: () -> self
|
|
66
70
|
def calculate_checksum: () -> (Integer | String)
|
|
71
|
+
attr_reader checksum: (Integer | String)?
|
|
72
|
+
def suggest: () -> Array[Suggestion]
|
|
67
73
|
|
|
68
74
|
def comparison_id: () -> String
|
|
69
75
|
|
|
@@ -6,7 +6,8 @@ module SecID
|
|
|
6
6
|
|
|
7
7
|
def self.included: (Module base) -> void
|
|
8
8
|
|
|
9
|
-
attr_reader checksum
|
|
9
|
+
# `attr_reader checksum` is declared on Base (the repair surface) to keep the
|
|
10
|
+
# Base-self-typed Suggestable body type-checking; not redeclared here.
|
|
10
11
|
|
|
11
12
|
# Class methods added when Checkable is included.
|
|
12
13
|
module ClassMethods : SecID::_IdentifierClass
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
module SecID
|
|
2
|
+
# Repair engine for checksum-failing identifiers: enumerates plausible edits and
|
|
3
|
+
# returns the re-validating ones as confidence-ranked Suggestion objects.
|
|
4
|
+
module Suggestable : SecID::Base
|
|
5
|
+
HOMOGLYPHS: Hash[String, Array[String]]
|
|
6
|
+
RANK: Hash[Symbol, Integer]
|
|
7
|
+
|
|
8
|
+
def self.included: (Module base) -> void
|
|
9
|
+
|
|
10
|
+
# Class methods added when Suggestable is included.
|
|
11
|
+
module ClassMethods : SecID::_IdentifierClass
|
|
12
|
+
def suggest: (SecID::input str) -> Array[Suggestion]
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
def suggest: () -> Array[Suggestion]
|
|
16
|
+
|
|
17
|
+
private
|
|
18
|
+
|
|
19
|
+
def candidate_strings: () -> Array[String]
|
|
20
|
+
def homoglyph_candidates: () -> Array[String]
|
|
21
|
+
def transposition_candidates: () -> Array[String]
|
|
22
|
+
def replace_at: (String str, Integer index, String char) -> String
|
|
23
|
+
def swap_at: (String str, Integer index) -> String
|
|
24
|
+
def classify: (Base candidate) -> Suggestion?
|
|
25
|
+
def changed_indices: (Base candidate) -> Array[Integer]
|
|
26
|
+
def substitution_suggestion: (Base candidate, Integer index) -> Suggestion
|
|
27
|
+
def transposition_suggestion: (Base candidate, Array[Integer] indices) -> Suggestion?
|
|
28
|
+
def swap?: (Base candidate, Integer left, Integer right) -> bool
|
|
29
|
+
def checksum_suggestion: (Base candidate) -> Suggestion
|
|
30
|
+
def rendered_checksum: ((Integer | String)? value) -> String
|
|
31
|
+
end
|
|
32
|
+
end
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
module SecID
|
|
2
|
+
# Immutable value object representing one repair candidate returned by `suggest`.
|
|
3
|
+
class Suggestion < Data
|
|
4
|
+
EDIT_KINDS: Array[Symbol]
|
|
5
|
+
CONFIDENCE_LEVELS: Array[Symbol]
|
|
6
|
+
|
|
7
|
+
attr_reader type: Symbol
|
|
8
|
+
attr_reader identifier: Base
|
|
9
|
+
attr_reader edit: Symbol
|
|
10
|
+
attr_reader position: Integer?
|
|
11
|
+
attr_reader from: String
|
|
12
|
+
attr_reader to: String
|
|
13
|
+
attr_reader confidence: Symbol?
|
|
14
|
+
|
|
15
|
+
def self.new: (type: Symbol, identifier: Base, edit: Symbol, position: Integer?, from: String, to: String, confidence: Symbol?) -> instance
|
|
16
|
+
| (Symbol type, Base identifier, Symbol edit, Integer? position, String from, String to, Symbol? confidence) -> instance
|
|
17
|
+
def self.[]: (type: Symbol, identifier: Base, edit: Symbol, position: Integer?, from: String, to: String, confidence: Symbol?) -> instance
|
|
18
|
+
| (Symbol type, Base identifier, Symbol edit, Integer? position, String from, String to, Symbol? confidence) -> instance
|
|
19
|
+
|
|
20
|
+
def as_json: (*untyped) -> Hash[Symbol, untyped]
|
|
21
|
+
def to_s: () -> String
|
|
22
|
+
end
|
|
23
|
+
end
|
data/sig/sec_id.rbs
CHANGED
|
@@ -38,9 +38,11 @@ module SecID
|
|
|
38
38
|
# against the `:first`/`:raise` arm's `Base?` and fail. Widening here keeps `result` untyped.
|
|
39
39
|
def self.parse!: (SecID::input str, ?types: Array[Symbol]?, ?on_ambiguous: Symbol) -> (Base | Array[Base])
|
|
40
40
|
def self.generate: (Symbol key, ?random: Random) -> Base
|
|
41
|
+
def self.suggest: (SecID::input str, ?types: Array[Symbol]?) -> Array[Suggestion]
|
|
41
42
|
|
|
42
43
|
# Singleton-method privacy must be declared inline (`private def self.…`); a bare
|
|
43
44
|
# `private` section only affects instance methods, leaving these public.
|
|
45
|
+
private def self.suggestable_types: (String input, Array[Symbol]? types) -> Array[singleton(Base)]
|
|
44
46
|
private def self.register_identifier: (singleton(Base) klass) -> void
|
|
45
47
|
private def self.parse_any: (SecID::input str) -> Base?
|
|
46
48
|
private def self.parse_from: (SecID::input str, Array[Symbol] types) -> Base?
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: sec_id
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 7.
|
|
4
|
+
version: 7.1.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Leonid Svyatov
|
|
@@ -9,9 +9,10 @@ bindir: bin
|
|
|
9
9
|
cert_chain: []
|
|
10
10
|
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
11
|
dependencies: []
|
|
12
|
-
description: Validate, normalize, parse, convert, generate, and
|
|
12
|
+
description: Validate, normalize, parse, convert, generate, classify, and repair securities
|
|
13
13
|
identifiers. Auto-detect identifier type from any string. Calculate and restore
|
|
14
|
-
checksums.
|
|
14
|
+
checksums. Suggest confidence-ranked corrections for fat-fingered or OCR-mangled
|
|
15
|
+
identifiers. Generate format-valid identifiers as test fixtures. Decode CFI codes
|
|
15
16
|
to full ISO 10962:2021 classifications. Includes an opt-in ActiveModel/Rails validator
|
|
16
17
|
that adds no runtime dependency. Supports ISIN, CUSIP, CEI, SEDOL, FIGI, LEI, IBAN,
|
|
17
18
|
CIK, OCC, WKN, Valoren, CFI, FISN, BIC, DTI, and UPI.
|
|
@@ -41,6 +42,7 @@ files:
|
|
|
41
42
|
- lib/sec_id/concerns/checkable.rb
|
|
42
43
|
- lib/sec_id/concerns/generatable.rb
|
|
43
44
|
- lib/sec_id/concerns/normalizable.rb
|
|
45
|
+
- lib/sec_id/concerns/suggestable.rb
|
|
44
46
|
- lib/sec_id/concerns/validatable.rb
|
|
45
47
|
- lib/sec_id/cusip.rb
|
|
46
48
|
- lib/sec_id/deep_freeze.rb
|
|
@@ -58,6 +60,7 @@ files:
|
|
|
58
60
|
- lib/sec_id/railtie.rb
|
|
59
61
|
- lib/sec_id/scanner.rb
|
|
60
62
|
- lib/sec_id/sedol.rb
|
|
63
|
+
- lib/sec_id/suggestion.rb
|
|
61
64
|
- lib/sec_id/upi.rb
|
|
62
65
|
- lib/sec_id/valoren.rb
|
|
63
66
|
- lib/sec_id/version.rb
|
|
@@ -78,6 +81,7 @@ files:
|
|
|
78
81
|
- sig/sec_id/concerns/checkable.rbs
|
|
79
82
|
- sig/sec_id/concerns/generatable.rbs
|
|
80
83
|
- sig/sec_id/concerns/normalizable.rbs
|
|
84
|
+
- sig/sec_id/concerns/suggestable.rbs
|
|
81
85
|
- sig/sec_id/concerns/validatable.rbs
|
|
82
86
|
- sig/sec_id/cusip.rbs
|
|
83
87
|
- sig/sec_id/deep_freeze.rbs
|
|
@@ -94,6 +98,7 @@ files:
|
|
|
94
98
|
- sig/sec_id/occ.rbs
|
|
95
99
|
- sig/sec_id/scanner.rbs
|
|
96
100
|
- sig/sec_id/sedol.rbs
|
|
101
|
+
- sig/sec_id/suggestion.rbs
|
|
97
102
|
- sig/sec_id/upi.rbs
|
|
98
103
|
- sig/sec_id/valoren.rbs
|
|
99
104
|
- sig/sec_id/version.rbs
|
|
@@ -124,5 +129,5 @@ requirements: []
|
|
|
124
129
|
rubygems_version: 4.0.12
|
|
125
130
|
specification_version: 4
|
|
126
131
|
summary: A Ruby toolkit for securities identifiers — validate, parse, normalize, detect,
|
|
127
|
-
convert, generate, and
|
|
132
|
+
convert, generate, classify, and repair.
|
|
128
133
|
test_files: []
|