sec_id 7.1.0 → 8.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +86 -30
- data/README.md +118 -63
- data/lib/sec_id/active_model.rb +5 -4
- data/lib/sec_id/bic/country_codes.rb +1 -2
- data/lib/sec_id/cei.rb +1 -1
- data/lib/sec_id/cfi/attribute_set.rb +4 -1
- data/lib/sec_id/cfi/tables.rb +1 -2
- data/lib/sec_id/concerns/checkable.rb +46 -62
- data/lib/sec_id/concerns/generatable.rb +4 -0
- data/lib/sec_id/concerns/normalizable.rb +4 -0
- data/lib/sec_id/concerns/suggestable.rb +6 -1
- data/lib/sec_id/concerns/validatable.rb +4 -0
- data/lib/sec_id/cusip.rb +1 -1
- data/lib/sec_id/detector.rb +13 -10
- data/lib/sec_id/figi.rb +1 -1
- data/lib/sec_id/iban/country_rules.rb +1 -2
- data/lib/sec_id/iban.rb +1 -1
- data/lib/sec_id/isin.rb +1 -1
- data/lib/sec_id/lei.rb +1 -1
- data/lib/sec_id/scanner.rb +12 -7
- data/lib/sec_id/suggestion.rb +4 -0
- data/lib/sec_id/version.rb +1 -1
- data/lib/sec_id.rb +3 -1
- data/sec_id.gemspec +2 -2
- data/sig/sec_id/cfi/attribute_set.rbs +2 -2
- data/sig/sec_id/concerns/checkable.rbs +7 -6
- data/sig/sec_id/detector.rbs +1 -1
- data/sig/sec_id/scanner.rbs +1 -1
- metadata +5 -5
data/README.md
CHANGED
|
@@ -1,19 +1,27 @@
|
|
|
1
|
-
# SecID
|
|
1
|
+
# SecID
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
A Ruby toolkit for securities identifiers: validate, parse, normalize, detect, convert, generate, classify, and repair.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
[](https://rubygems.org/gems/sec_id) [](https://github.com/svyatov/sec_id/actions/workflows/main.yml) [](https://codecov.io/gh/svyatov/sec_id) [](https://rubydoc.info/gems/sec_id) [](https://www.ruby-lang.org) [](https://github.com/svyatov/sec_id/tree/main/sig)
|
|
6
6
|
|
|
7
|
-
-
|
|
7
|
+
- **Nothing to call, nothing to sign up for.** SecID validates all 16 identifier standards offline. No registry lookup, no API key, no network call anywhere in [`lib/`](lib/).
|
|
8
|
+
- **9 of the 16 carry a checksum.** SecID validates it, restores a missing one, and repairs a wrong one.
|
|
9
|
+
- **Zero runtime dependencies.** Runs on Ruby 3.3 or newer.
|
|
10
|
+
- **Rails, if you want it.** An opt-in ActiveModel validator adds `validates :isin, sec_id: true`. It stays off the default require path, so the dependency count stays at zero.
|
|
11
|
+
- **Typed.** Ships RBS signatures for the whole public API, checked by Steep in strict mode.
|
|
12
|
+
|
|
13
|
+
## Table of contents
|
|
14
|
+
|
|
15
|
+
- [Supported Ruby versions](#supported-ruby-versions)
|
|
8
16
|
- [Installation](#installation)
|
|
9
|
-
- [Supported
|
|
10
|
-
- [Metadata
|
|
11
|
-
- [Text
|
|
12
|
-
- [Debugging
|
|
13
|
-
- [Structured
|
|
14
|
-
- [Pattern
|
|
15
|
-
- [Generating
|
|
16
|
-
- [Repairing
|
|
17
|
+
- [Supported standards and usage](#supported-standards-and-usage)
|
|
18
|
+
- [Metadata registry](#metadata-registry) - enumerate, filter, look up, and detect identifier types
|
|
19
|
+
- [Text scanning](#text-scanning) - find identifiers in freeform text
|
|
20
|
+
- [Debugging detection](#debugging-detection) - understand why strings match or don't
|
|
21
|
+
- [Structured validation](#structured-validation) - detailed error codes and messages
|
|
22
|
+
- [Pattern matching](#pattern-matching) - destructure identifiers with `case/in`
|
|
23
|
+
- [Generating test fixtures](#generating-test-fixtures) - produce valid identifiers for tests
|
|
24
|
+
- [Repairing typos](#repairing-typos) - suggest corrections for checksum-failing identifiers
|
|
17
25
|
- [ISIN](#isin) - International Securities Identification Number
|
|
18
26
|
- [CUSIP](#cusip) - Committee on Uniform Securities Identification Procedures
|
|
19
27
|
- [CEI](#cei) - CUSIP Entity Identifier
|
|
@@ -30,25 +38,26 @@
|
|
|
30
38
|
- [BIC](#bic) - Business Identifier Code / SWIFT code
|
|
31
39
|
- [DTI](#dti) - Digital Token Identifier
|
|
32
40
|
- [UPI](#upi) - Unique Product Identifier
|
|
33
|
-
- [ActiveModel / Rails
|
|
34
|
-
- [Lookup
|
|
35
|
-
- [Type
|
|
41
|
+
- [ActiveModel / Rails validator](#activemodel--rails-validator) - declarative `validates :isin, sec_id: {...}`
|
|
42
|
+
- [Lookup service integration](#lookup-service-integration)
|
|
43
|
+
- [Type signatures (RBS)](#type-signatures-rbs)
|
|
36
44
|
- [Development](#development)
|
|
45
|
+
- [Support and status](#support-and-status)
|
|
37
46
|
- [Contributing](#contributing)
|
|
38
47
|
- [Changelog](#changelog)
|
|
39
48
|
- [Versioning](#versioning)
|
|
40
49
|
- [License](#license)
|
|
41
50
|
|
|
42
|
-
## Supported Ruby
|
|
51
|
+
## Supported Ruby versions
|
|
43
52
|
|
|
44
|
-
Ruby 3.
|
|
53
|
+
Ruby 3.3+ is required.
|
|
45
54
|
|
|
46
55
|
## Installation
|
|
47
56
|
|
|
48
57
|
Add this line to your application's Gemfile:
|
|
49
58
|
|
|
50
59
|
```ruby
|
|
51
|
-
gem 'sec_id', '~>
|
|
60
|
+
gem 'sec_id', '~> 8.0'
|
|
52
61
|
```
|
|
53
62
|
|
|
54
63
|
And then execute:
|
|
@@ -65,7 +74,7 @@ gem install sec_id
|
|
|
65
74
|
|
|
66
75
|
**Upgrading from v4?** See [MIGRATION.md](MIGRATION.md) for a step-by-step guide.
|
|
67
76
|
|
|
68
|
-
## Supported
|
|
77
|
+
## Supported standards and usage
|
|
69
78
|
|
|
70
79
|
All identifier classes provide `valid?`, `errors`, `validate`, `validate!` methods at both class and instance levels.
|
|
71
80
|
|
|
@@ -90,7 +99,7 @@ SecID::ISIN.new('INVALID').to_h
|
|
|
90
99
|
# valid: false, components: { country_code: nil, nsin: nil, checksum: nil } }
|
|
91
100
|
```
|
|
92
101
|
|
|
93
|
-
**All identifiers** support value equality
|
|
102
|
+
**All identifiers** support value equality. Two instances of the same type with the same normalized form are equal:
|
|
94
103
|
|
|
95
104
|
```ruby
|
|
96
105
|
a = SecID::ISIN.new('US5949181045')
|
|
@@ -109,7 +118,7 @@ Set.new([a, b]).size # => 1
|
|
|
109
118
|
- `restore!` / `.restore!` - restores checksum in place and returns `self` / instance
|
|
110
119
|
- `checksum` / `calculate_checksum` - calculates and returns the checksum
|
|
111
120
|
|
|
112
|
-
### Metadata
|
|
121
|
+
### Metadata registry
|
|
113
122
|
|
|
114
123
|
All identifier classes are registered automatically and can be enumerated, filtered, and looked up by symbol key:
|
|
115
124
|
|
|
@@ -165,7 +174,7 @@ SecID.parse('514000', on_ambiguous: :all) # => [#<SecID::WKN>, #<SecID:
|
|
|
165
174
|
SecID.parse('US5949181045', on_ambiguous: :raise) # => #<SecID::ISIN> (unambiguous, no error)
|
|
166
175
|
```
|
|
167
176
|
|
|
168
|
-
### Text
|
|
177
|
+
### Text scanning
|
|
169
178
|
|
|
170
179
|
Find identifiers embedded in freeform text:
|
|
171
180
|
|
|
@@ -191,12 +200,12 @@ match.identifier.normalized # => "US5949181045"
|
|
|
191
200
|
|
|
192
201
|
> **Known limitations:** Format-only types (CIK, Valoren, WKN, BIC) can false-positive on
|
|
193
202
|
> common numbers and short words in prose (a BIC8 is 8 letters with a valid country code in the
|
|
194
|
-
> middle)
|
|
203
|
+
> middle). Use the `types:` filter to restrict scanning when
|
|
195
204
|
> this is a concern. Identifiers prefixed with special characters (e.g. `#US5949181045`) may be
|
|
196
205
|
> consumed as a single token by CUSIP's `*@#` character class and fail validation, preventing
|
|
197
206
|
> the embedded identifier from being found.
|
|
198
207
|
|
|
199
|
-
### Debugging
|
|
208
|
+
### Debugging detection
|
|
200
209
|
|
|
201
210
|
Understand why a string matches or doesn't match specific identifier types:
|
|
202
211
|
|
|
@@ -210,7 +219,7 @@ isin[:errors].first[:error] # => :invalid_checksum
|
|
|
210
219
|
SecID.explain('US5949181045', types: %i[isin cusip])
|
|
211
220
|
```
|
|
212
221
|
|
|
213
|
-
### Structured
|
|
222
|
+
### Structured validation
|
|
214
223
|
|
|
215
224
|
All identifier classes provide a Rails-like `#errors` API for detailed error reporting:
|
|
216
225
|
|
|
@@ -266,7 +275,7 @@ SecID::FIGI.new('BSG000BLNNH6').validate!
|
|
|
266
275
|
isin = SecID::ISIN.validate!('US5949181045') # => #<SecID::ISIN>
|
|
267
276
|
```
|
|
268
277
|
|
|
269
|
-
### Pattern
|
|
278
|
+
### Pattern matching
|
|
270
279
|
|
|
271
280
|
Every identifier destructures in `case/in` via its parsed components. Since `SecID.parse` returns `nil` for
|
|
272
281
|
anything invalid, an `in nil` branch is a complete validity guard:
|
|
@@ -306,11 +315,11 @@ CIK, WKN, and Valoren have no sub-fields: they match a bare `in SecID::CIK` but
|
|
|
306
315
|
|
|
307
316
|
Watch the `:type` key: on a `SecID::Match` it is the registry symbol (`:occ`), while on an `OCC` instance it is the
|
|
308
317
|
OSI option type (`'C'` or `'P'`), so `in SecID::OCC[type: :occ]` never matches. Other types have no `:type` component
|
|
309
|
-
at all
|
|
318
|
+
at all. `#to_h`'s envelope keys (`:type`, `:full_id`, `:normalized`, `:valid`) are not part of the pattern surface.
|
|
310
319
|
|
|
311
|
-
### Generating
|
|
320
|
+
### Generating test fixtures
|
|
312
321
|
|
|
313
|
-
Generate syntactically valid identifiers
|
|
322
|
+
Generate syntactically valid identifiers, with correct checksum where applicable, for use as test fixtures. Available per class and via the central dispatcher:
|
|
314
323
|
|
|
315
324
|
```ruby
|
|
316
325
|
SecID::ISIN.generate # => #<SecID::ISIN ...>
|
|
@@ -324,17 +333,17 @@ SecID.generate(:nope) # => raises ArgumentError: Unknown identifier type
|
|
|
324
333
|
SecID::LEI.generate(random: Random.new(42)) == SecID::LEI.generate(random: Random.new(42)) # => true
|
|
325
334
|
```
|
|
326
335
|
|
|
327
|
-
> **Generated identifiers are valid in format only
|
|
336
|
+
> **Generated identifiers are valid in format only. They are not real, registered securities.**
|
|
328
337
|
> Country codes, FIGI prefixes, OCC expiry dates, and CFI category/group/attribute choices are
|
|
329
338
|
> randomly selected (from the values each standard permits) and do not map to real-world
|
|
330
339
|
> instruments. Use them as test fixtures, not as references to actual securities.
|
|
331
340
|
|
|
332
|
-
### Repairing
|
|
341
|
+
### Repairing typos
|
|
333
342
|
|
|
334
|
-
Turn the checksum from a gatekeeper into a repair engine. For a checksum-**failing** identifier, `suggest` enumerates the plausible single-character human errors
|
|
343
|
+
Turn the checksum from a gatekeeper into a repair engine. For a checksum-**failing** identifier, `suggest` enumerates the plausible single-character human errors (visual/OCR homoglyph substitutions such as `O`↔`0`, `I`↔`1`, `5`↔`S`, `8`↔`B`, plus adjacent transpositions), keeps only the edits that re-validate, and returns them as confidence-ranked `SecID::Suggestion` candidates that report *what changed*. Available for all 9 checksum types (ISIN, CUSIP, SEDOL, FIGI, LEI, IBAN, CEI, DTI, UPI) per class and via the central dispatcher:
|
|
335
344
|
|
|
336
345
|
```ruby
|
|
337
|
-
# A letter O typed where a 0 belongs
|
|
346
|
+
# A letter O typed where a 0 belongs: 'US5949181O45' should be 'US5949181045'
|
|
338
347
|
top = SecID::ISIN.suggest('US5949181O45').first
|
|
339
348
|
top.to_s # => 'US5949181045' (the corrected identifier)
|
|
340
349
|
top.edit # => :substitution
|
|
@@ -342,27 +351,28 @@ top.position # => 9
|
|
|
342
351
|
top.from # => 'O'
|
|
343
352
|
top.to # => '0'
|
|
344
353
|
top.confidence # => :high
|
|
345
|
-
top.identifier # => #<SecID::ISIN ...> (parsed and valid
|
|
354
|
+
top.identifier # => #<SecID::ISIN ...> (parsed and valid; call .country_code, .to_h, etc.)
|
|
346
355
|
|
|
347
356
|
# Module-level: infers every format-compatible checksum type (like parse / detect)
|
|
348
357
|
SecID.suggest('US5949181O45') # => [#<SecID::Suggestion type=:isin ...>, ...]
|
|
349
358
|
SecID.suggest('US5949181O45', types: [:isin]) # => restrict to specific types
|
|
350
359
|
```
|
|
351
360
|
|
|
352
|
-
Each candidate carries the corrected `identifier` (a parsed, valid instance), the `edit` kind, its `position`, the `from`/`to` characters, and a `confidence` tier. Candidates are ranked by confidence: `:high` homoglyph substitutions first, then `:medium` adjacent transpositions, then the `:checksum` recompute (body assumed correct, wrong check character) last as a fallback hypothesis. **There is no `:low` tier
|
|
361
|
+
Each candidate carries the corrected `identifier` (a parsed, valid instance), the `edit` kind, its `position`, the `from`/`to` characters, and a `confidence` tier. Candidates are ranked by confidence: `:high` homoglyph substitutions first, then `:medium` adjacent transpositions, then the `:checksum` recompute (body assumed correct, wrong check character) last as a fallback hypothesis. **There is no `:low` tier.** Coincidental substitutions that merely satisfy the checksum are never generated, keeping the result small and high-precision.
|
|
353
362
|
|
|
354
363
|
> **`suggest` returns candidates, never authoritative corrections, and never mutates its input.**
|
|
355
|
-
> Every returned candidate fully re-validates (`valid?` is the oracle), so no
|
|
356
|
-
> but the `confidence` tier is how *you* decide what to
|
|
364
|
+
> Every returned candidate fully re-validates (`valid?` is the oracle), so no checksum-invalid candidate escapes,
|
|
365
|
+
> but a valid candidate is not necessarily *the* correction, so the `confidence` tier is how *you* decide what to
|
|
366
|
+
> trust. This matters for financial identifiers.
|
|
357
367
|
|
|
358
368
|
Notes and limitations:
|
|
359
369
|
|
|
360
|
-
- **Never empty for structurally-valid input
|
|
361
|
-
- **Vowel-free reachability
|
|
362
|
-
- **Single body error only
|
|
363
|
-
- **Non-checksum types are unsupported
|
|
370
|
+
- **Never empty for structurally-valid input.** A parseable but checksum-failing identifier always yields at least the `:checksum` fallback. Only wrong-length or illegal-charset input (which fails the format gate), or an already-valid identifier (nothing to repair), returns `[]`.
|
|
371
|
+
- **Vowel-free reachability.** SEDOL, FIGI, DTI, and UPI exclude vowels from their charset, so an `O`-for-`0` or `I`-for-`1` typo is unparseable and therefore unrepairable (it fails the format gate before enumeration). The mistyped character must be *in* the type's charset to be reachable.
|
|
372
|
+
- **Single body error only.** Two or more wrong body characters, or a dropped/doubled character (insertion/deletion), are out of scope; such input returns only the `:checksum` fallback, never additional body candidates.
|
|
373
|
+
- **Non-checksum types are unsupported.** CIK, OCC, WKN, Valoren, CFI, FISN, and BIC have no checksum oracle, so they have no `suggest`; `SecID.suggest` silently skips them.
|
|
364
374
|
|
|
365
|
-
**Precision** (simulated over 1,000 seeded samples per checksum type
|
|
375
|
+
**Precision** (simulated over 1,000 seeded samples per checksum type; see [`benchmark/suggest_precision.rb`](benchmark/suggest_precision.rb)): every reachable, single-error homoglyph or transposition that yields a checksum-failing identifier is recovered: the correct identifier is **always** among the returned candidates (100%), and is the top-ranked body candidate ~89% of the time for homoglyph errors. In-charset homoglyph reachability averages ~96% (100% for the letter-permitting types, lower for the four vowel-free ones).
|
|
366
376
|
|
|
367
377
|
### ISIN
|
|
368
378
|
|
|
@@ -702,7 +712,7 @@ SecID::CFI.new('QQXXXX').decode # => nil (decode returns nil for an invalid
|
|
|
702
712
|
|
|
703
713
|
CFI is validated strictly against the ISO 10962:2021 code tables for all 14 categories: the category (position 1), the group (position 2), and every attribute (positions 3-6) must be a value the standard defines for that group. `X` means "not applicable" and is accepted in every position; `Strategies` (`K`) codes carry no attributes and require `XXXX`. An impermissible attribute letter raises `InvalidStructureError` (`:invalid_attribute`).
|
|
704
714
|
|
|
705
|
-
> **Migration from < 6.0:** the old category-wide equity predicates (`cfi.voting?`, `cfi.fully_paid?`, …) are removed. Use `cfi.decode` and its scoped fields instead
|
|
715
|
+
> **Migration from < 6.0:** the old category-wide equity predicates (`cfi.voting?`, `cfi.fully_paid?`, …) are removed. Use `cfi.decode` and its scoped fields instead. A predicate now lives on the field whose domain defines it: `cfi.voting?` → `cfi.decode.attributes.voting_right.voting?`. Two do not map name-for-name: `cfi.equity?` → `cfi.decode.category.equity?` (or `cfi.category == :equity`), and `cfi.no_restrictions?` → `cfi.decode.attributes.ownership_restrictions.free_of_restrictions?`. Several group letters and symbols also changed to match ISO 10962:2021 (e.g. non-listed options `H` are now classified by underlying, and `LS` → `:securities_lending`, `TI` → `:indices`).
|
|
706
716
|
|
|
707
717
|
```ruby
|
|
708
718
|
# Introspect valid codes
|
|
@@ -757,7 +767,7 @@ SecID::BIC.valid?('DEUTZZFF') # => false ('ZZ' is not a recognized country
|
|
|
757
767
|
SecID::BIC.countries # => ['AD', 'AE', 'AF', ...] (sorted, includes 'XK')
|
|
758
768
|
```
|
|
759
769
|
|
|
760
|
-
BIC validation confirms structure and a real country code only. It does **not** verify that the institution, location, or branch corresponds to a registered SWIFT participant
|
|
770
|
+
BIC validation confirms structure and a real country code only. It does **not** verify that the institution, location, or branch corresponds to a registered SWIFT participant. That requires the licensed SWIFT registry.
|
|
761
771
|
|
|
762
772
|
### DTI
|
|
763
773
|
|
|
@@ -781,7 +791,7 @@ dti.restore! # => #<SecID::DTI> (mutates instance)
|
|
|
781
791
|
dti.calculate_checksum # => 'S'
|
|
782
792
|
```
|
|
783
793
|
|
|
784
|
-
DTI accepts exactly 9 characters: an 8-character base (first character never `0`) plus 1 check character, both drawn from a 30-symbol alphabet
|
|
794
|
+
DTI accepts exactly 9 characters: an 8-character base (first character never `0`) plus 1 check character, both drawn from a 30-symbol alphabet: digits `0`-`9` and consonants (vowels and `Y` never appear). Unlike most checksum types in this gem, `checksum` and `calculate_checksum` return a `String`, not an `Integer` (as does UPI). The check character is computed fully offline via ISO 7064 hybrid MOD 31,30, with no registry lookup or paywalled ISO 24165-1 spec required.
|
|
785
795
|
|
|
786
796
|
> **Grandfathered code:** Bitcoin's registered code (`4H95J0R2X`) predates the algorithm and fails the MOD 31,30 computation (which yields `4H95J0R2T`). A frozen exception map honors the registry's assignment across `valid?`, `restore`, and `checksum` alike:
|
|
787
797
|
>
|
|
@@ -812,15 +822,15 @@ upi.restore! # => #<SecID::UPI> (mutates instance)
|
|
|
812
822
|
upi.calculate_checksum # => '2'
|
|
813
823
|
```
|
|
814
824
|
|
|
815
|
-
UPI accepts exactly 12 characters: a fixed `QZ` prefix, a 9-character body, and 1 check character, all drawn from the same 30-symbol alphabet as DTI
|
|
825
|
+
UPI accepts exactly 12 characters: a fixed `QZ` prefix, a 9-character body, and 1 check character, all drawn from the same 30-symbol alphabet as DTI: digits `0`-`9` and consonants (vowels and `Y` never appear). Like DTI, `checksum` and `calculate_checksum` return a `String`, not an `Integer`. The check character is computed fully offline via ISO 7064 hybrid MOD 31,30 over the 11 preceding characters, with no DSB registry lookup or paywalled ISO 4914 spec required.
|
|
816
826
|
|
|
817
|
-
> **Coexistence with ISIN:** a UPI shares the 12-character length bucket with ISIN. A UPI whose digit check character also satisfies ISIN's Luhn detects as both (`SecID.detect('QZXKR05S3DL1') # => [:isin, :upi]`), with ISIN ranked first; `SecID.parse(..., on_ambiguous: :raise)` surfaces the collision. UPI validation itself is fully offline and existence is **not** verified
|
|
827
|
+
> **Coexistence with ISIN:** a UPI shares the 12-character length bucket with ISIN. A UPI whose digit check character also satisfies ISIN's Luhn detects as both (`SecID.detect('QZXKR05S3DL1') # => [:isin, :upi]`), with ISIN ranked first; `SecID.parse(..., on_ambiguous: :raise)` surfaces the collision. UPI validation itself is fully offline and existence is **not** verified. That requires the licensed DSB registry.
|
|
818
828
|
|
|
819
|
-
## ActiveModel / Rails
|
|
829
|
+
## ActiveModel / Rails validator
|
|
820
830
|
|
|
821
|
-
SecID ships an opt-in [ActiveModel](https://api.rubyonrails.org/classes/ActiveModel/Validations.html) validator, registered as `sec_id`, for declarative validation of any supported identifier type. It adds **no runtime dependency
|
|
831
|
+
SecID ships an opt-in [ActiveModel](https://api.rubyonrails.org/classes/ActiveModel/Validations.html) validator, registered as `sec_id`, for declarative validation of any supported identifier type. It adds **no runtime dependency**: `require 'sec_id'` loads none of it, and ActiveModel is a development/test dependency only.
|
|
822
832
|
|
|
823
|
-
**In Rails it just works.** A Railtie loads the validator automatically after the framework boots, so `gem 'sec_id'` in your `Gemfile` is enough
|
|
833
|
+
**In Rails it just works.** A Railtie loads the validator automatically after the framework boots, so `gem 'sec_id'` in your `Gemfile` is enough, with no `require:` option and no initializer:
|
|
824
834
|
|
|
825
835
|
```ruby
|
|
826
836
|
class Security < ApplicationRecord
|
|
@@ -837,13 +847,13 @@ require 'sec_id/active_model'
|
|
|
837
847
|
### Validation modes
|
|
838
848
|
|
|
839
849
|
```ruby
|
|
840
|
-
# Single type
|
|
850
|
+
# Single type: the value must be a valid ISIN
|
|
841
851
|
validates :isin, sec_id: { type: :isin }
|
|
842
852
|
|
|
843
|
-
# Allowlist
|
|
853
|
+
# Allowlist: valid as at least one of the listed types
|
|
844
854
|
validates :ref, sec_id: { types: %i[isin cusip] }
|
|
845
855
|
|
|
846
|
-
# Type-agnostic
|
|
856
|
+
# Type-agnostic: valid as any supported type
|
|
847
857
|
validates :ref, sec_id: true
|
|
848
858
|
```
|
|
849
859
|
|
|
@@ -862,16 +872,16 @@ With `normalize: true` in allowlist or agnostic mode, a value valid as more than
|
|
|
862
872
|
|
|
863
873
|
### Error messages and `details:`
|
|
864
874
|
|
|
865
|
-
On failure the validator adds one error under the `:sec_id` key with a type-aware default ("is not a valid ISIN" for a single type, "is not a valid securities identifier" for an allowlist/agnostic). Override the message in either of two ways: pass the standard `message:` option (the simplest, per-validation override), or define the attribute-scoped i18n key `activemodel.errors.models.<model>.attributes.<attribute>.sec_id` in your locale files. (The generic `activemodel.errors.messages.sec_id` key is not consulted, because the built-in default is supplied as ActiveModel's `message:` fallback.) Pass `details: true` (with a single `type
|
|
875
|
+
On failure the validator adds one error under the `:sec_id` key with a type-aware default ("is not a valid ISIN" for a single type, "is not a valid securities identifier" for an allowlist/agnostic). Override the message in either of two ways: pass the standard `message:` option (the simplest, per-validation override), or define the attribute-scoped i18n key `activemodel.errors.models.<model>.attributes.<attribute>.sec_id` in your locale files. (The generic `activemodel.errors.messages.sec_id` key is not consulted, because the built-in default is supplied as ActiveModel's `message:` fallback.) Pass `details: true` (with a single `type:`; it is ignored for an allowlist/agnostic) to surface sec_id's specific reason instead of the generic text:
|
|
866
876
|
|
|
867
877
|
```ruby
|
|
868
878
|
validates :isin, sec_id: { type: :isin, details: true }
|
|
869
879
|
# a bad checksum reports e.g. "Checksum '4' is invalid, expected '5'"
|
|
870
880
|
```
|
|
871
881
|
|
|
872
|
-
Standard `EachValidator` options
|
|
882
|
+
Standard `EachValidator` options (`allow_nil`, `allow_blank`, `if`, `unless`, `on`) work as usual. Tested against Rails 7.2, 8.0, and 8.1.
|
|
873
883
|
|
|
874
|
-
## Lookup
|
|
884
|
+
## Lookup service integration
|
|
875
885
|
|
|
876
886
|
SecID validates identifiers but does not include HTTP clients. The [`docs/guides/`](docs/guides/) directory provides integration patterns for external lookup services using only stdlib (`net/http`, `json`):
|
|
877
887
|
|
|
@@ -884,10 +894,10 @@ SecID validates identifiers but does not include HTTP clients. The [`docs/guides
|
|
|
884
894
|
|
|
885
895
|
Each guide includes a complete adapter class and a [runnable example](examples/).
|
|
886
896
|
|
|
887
|
-
## Type
|
|
897
|
+
## Type signatures (RBS)
|
|
888
898
|
|
|
889
899
|
sec_id ships hand-written [RBS](https://github.com/ruby/rbs) signatures under `sig/`,
|
|
890
|
-
packaged in the gem
|
|
900
|
+
packaged in the gem, so if you use [Steep](https://github.com/soutaro/steep) or an
|
|
891
901
|
RBS-aware editor, sec_id's types resolve automatically on install, no `rbs collection`
|
|
892
902
|
entry required. The only standard-library signature referenced is `date`, declared in
|
|
893
903
|
`sig/manifest.yaml`.
|
|
@@ -912,16 +922,47 @@ committed signatures fall out of sync with the tables.
|
|
|
912
922
|
|
|
913
923
|
## Development
|
|
914
924
|
|
|
915
|
-
|
|
916
|
-
|
|
917
|
-
|
|
925
|
+
Install [Betterleaks](https://github.com/betterleaks/betterleaks) and make its `betterleaks` command available on your `PATH`.
|
|
926
|
+
After checking out the repo, run `bin/setup` to install dependencies and Git hooks.
|
|
927
|
+
Then run `bundle exec rake` to run RuboCop, validate the RBS signatures, and run the specs.
|
|
928
|
+
`bin/console` gives you an interactive prompt to experiment in.
|
|
929
|
+
|
|
930
|
+
Pre-commit scans staged changes for secrets and whitespace errors, and safely corrects staged Ruby files with RuboCop.
|
|
931
|
+
Corrected Ruby files are staged automatically.
|
|
932
|
+
Pre-push runs static type checks, type coverage, specs, runtime signature checks, and documentation coverage.
|
|
933
|
+
Commit messages must follow Conventional Commits.
|
|
934
|
+
Put personal hook overrides in the ignored `lefthook-local.yml` file.
|
|
935
|
+
|
|
936
|
+
Run all pre-push jobs manually with `bundle exec lefthook run pre-push`.
|
|
937
|
+
|
|
938
|
+
CI checks library methods with [crap4ruby](https://github.com/svyatov/crap4ruby), using fresh test coverage and a maximum CRAP score of 10.
|
|
939
|
+
Run the same check locally:
|
|
940
|
+
|
|
941
|
+
```sh
|
|
942
|
+
COVERAGE=1 bundle exec rake spec crap
|
|
943
|
+
```
|
|
944
|
+
|
|
945
|
+
To use your working copy from another project, point its Gemfile at the checkout:
|
|
946
|
+
`gem 'sec_id', path: '/path/to/sec_id'`.
|
|
947
|
+
|
|
948
|
+
Releases are published from CI by pushing a tag, so there is no local publish task.
|
|
949
|
+
|
|
950
|
+
## Support and status
|
|
918
951
|
|
|
919
|
-
|
|
952
|
+
Ask questions in [Discussions](https://github.com/svyatov/sec_id/discussions). Report bugs in the [issue tracker](https://github.com/svyatov/sec_id/issues). Both are public and searchable, so the next reader with your question finds the answer.
|
|
953
|
+
|
|
954
|
+
One person actively maintains SecID. They read bug reports and pull requests, and they track each identifier standard against its published specification. No response time is promised. See [Governance](CONTRIBUTING.md#governance) for who decides and what happens if they stop.
|
|
920
955
|
|
|
921
956
|
## Contributing
|
|
922
957
|
|
|
923
958
|
Bug reports and pull requests are welcome on [GitHub](https://github.com/svyatov/sec_id). See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, code style, and PR guidelines.
|
|
924
959
|
|
|
960
|
+
For checksum throughput and allocation comparisons, run `ruby -Ilib benchmark/checksums.rb`. It covers Luhn checksums, LEI, and IBAN, including numeric and letter-heavy inputs. It prints repeated samples as JSON with garbage collection enabled. Run the same script with `-I/path/to/baseline/lib` to compare another checkout on the same Ruby version.
|
|
961
|
+
|
|
962
|
+
For scanner comparisons, run `ruby -Ilib benchmark/scanner.rb`. It measures paragraphs, dense identifiers, noise, type filters, ambiguous identifiers, and compound patterns. JSON output includes matches to check correctness alongside timings and allocations. Use the same baseline option to compare another checkout.
|
|
963
|
+
|
|
964
|
+
For detection and parsing comparisons, run `ruby -Ilib benchmark/detector.rb`. It measures valid identifiers, ambiguous inputs, invalid checksums, and unsupported lengths. JSON output includes results, timings, and allocations with garbage collection enabled. Use the same baseline option to compare another checkout.
|
|
965
|
+
|
|
925
966
|
This project follows the [Contributor Covenant Code of Conduct](CODE_OF_CONDUCT.md).
|
|
926
967
|
|
|
927
968
|
## Changelog
|
|
@@ -930,7 +971,21 @@ See [CHANGELOG.md](CHANGELOG.md) for a detailed history of changes, following [K
|
|
|
930
971
|
|
|
931
972
|
## Versioning
|
|
932
973
|
|
|
933
|
-
This project follows [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html)
|
|
974
|
+
This project follows [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html).
|
|
975
|
+
|
|
976
|
+
The public API that version promise covers is everything documented in this README and in the [API docs](https://rubydoc.info/gems/sec_id), except anything marked `@api private` in YARD. Concretely, it covers:
|
|
977
|
+
|
|
978
|
+
- the `SecID` module methods: `[]`, `identifiers`, `valid?`, `detect`, `parse`, `parse!`, `extract`, `scan`, `explain`, `generate`, and `suggest`
|
|
979
|
+
- every identifier class and its documented instance and class methods
|
|
980
|
+
- the `SecID::Errors`, `SecID::Suggestion`, `SecID::Scanner::Match`, and `SecID::CFI::Classification` value objects, and the exception hierarchy under `SecID::Error`
|
|
981
|
+
- the `sec_id` ActiveModel validator and its options
|
|
982
|
+
- the RBS signatures under `sig/`
|
|
983
|
+
|
|
984
|
+
Marked `@api private` and outside the promise: `SecID::Detector`, `SecID::Scanner` itself, `SecID::CFI::Tables`, `SecID::IBAN::CountryRules`, `SecID::BIC::CountryCodes`, `Base.detection_priority`, and the internals of the four concerns. These can change in any release.
|
|
985
|
+
|
|
986
|
+
Raising the minimum Ruby version is a breaking change and ships in a MAJOR release.
|
|
987
|
+
|
|
988
|
+
Removals follow a deprecation window; see [Deprecation policy](CONTRIBUTING.md#deprecation-policy).
|
|
934
989
|
|
|
935
990
|
## License
|
|
936
991
|
|
data/lib/sec_id/active_model.rb
CHANGED
|
@@ -5,11 +5,11 @@ require 'sec_id'
|
|
|
5
5
|
begin
|
|
6
6
|
require 'active_model'
|
|
7
7
|
rescue LoadError => e
|
|
8
|
-
# :
|
|
8
|
+
# simplecov:disable ActiveModel is always loadable in the test suite
|
|
9
9
|
raise LoadError, 'sec_id/active_model requires ActiveModel, which could not be loaded. Add ' \
|
|
10
10
|
'`gem "activemodel"` to your Gemfile (or use this inside a Rails app) before ' \
|
|
11
11
|
"requiring 'sec_id/active_model'. (#{e.message})"
|
|
12
|
-
# :
|
|
12
|
+
# simplecov:enable
|
|
13
13
|
end
|
|
14
14
|
|
|
15
15
|
# ActiveModel validator for securities identifiers, registered under the `sec_id` key.
|
|
@@ -115,9 +115,10 @@ class SecIdValidator < ActiveModel::EachValidator
|
|
|
115
115
|
# @return [String, nil] the message of the SecID::Error raised when normalizing `value` as `type`
|
|
116
116
|
def capture_reason(type, value)
|
|
117
117
|
SecID[type].normalize(value)
|
|
118
|
-
# :
|
|
118
|
+
# simplecov:disable unreachable — only called after normalize already raised in valid_value?,
|
|
119
|
+
# kept as a safe fallback
|
|
119
120
|
nil
|
|
120
|
-
# :
|
|
121
|
+
# simplecov:enable
|
|
121
122
|
rescue SecID::Error => e
|
|
122
123
|
e.message
|
|
123
124
|
end
|
|
@@ -10,7 +10,7 @@ module SecID
|
|
|
10
10
|
#
|
|
11
11
|
# @api private
|
|
12
12
|
# @see https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2
|
|
13
|
-
# rubocop:disable Metrics/CollectionLiteralLength
|
|
13
|
+
# rubocop:disable-next Metrics/CollectionLiteralLength
|
|
14
14
|
COUNTRY_CODES = Set.new(
|
|
15
15
|
%w[
|
|
16
16
|
AD AE AF AG AI AL AM AO AQ AR AS AT AU AW AX AZ
|
|
@@ -41,6 +41,5 @@ module SecID
|
|
|
41
41
|
XK
|
|
42
42
|
]
|
|
43
43
|
).freeze
|
|
44
|
-
# rubocop:enable Metrics/CollectionLiteralLength
|
|
45
44
|
end
|
|
46
45
|
end
|
data/lib/sec_id/cei.rb
CHANGED
|
@@ -55,7 +55,7 @@ module SecID
|
|
|
55
55
|
# @raise [InvalidFormatError] if the CEI format is invalid
|
|
56
56
|
def calculate_checksum
|
|
57
57
|
validate_format_for_calculation!
|
|
58
|
-
mod10(
|
|
58
|
+
mod10(luhn_sum_single(identifier))
|
|
59
59
|
end
|
|
60
60
|
|
|
61
61
|
# Generates a random CEI body: 1 letter + 1 digit + 7 alphanumeric characters.
|
|
@@ -36,8 +36,11 @@ module SecID
|
|
|
36
36
|
end
|
|
37
37
|
|
|
38
38
|
# @return [Hash{Symbol => Hash}]
|
|
39
|
+
# @raise [ArgumentError] if a field has no meaning (category/group fields; never the case for attributes)
|
|
39
40
|
def to_h
|
|
40
|
-
@fields.to_h
|
|
41
|
+
@fields.to_h do |field|
|
|
42
|
+
[field.meaning || raise(ArgumentError, "attribute field #{field.code} has no meaning"), field.to_h]
|
|
43
|
+
end
|
|
41
44
|
end
|
|
42
45
|
|
|
43
46
|
# @return [Hash{Symbol => Hash}]
|
data/lib/sec_id/cfi/tables.rb
CHANGED
|
@@ -21,7 +21,7 @@ module SecID
|
|
|
21
21
|
# +:not_applicable+, so it is never listed in a value map.
|
|
22
22
|
#
|
|
23
23
|
# @api private
|
|
24
|
-
# rubocop:disable Metrics/ModuleLength
|
|
24
|
+
# rubocop:disable-next Metrics/ModuleLength
|
|
25
25
|
module Tables
|
|
26
26
|
# Marker for a pure not-applicable attribute position (accepts only +X+).
|
|
27
27
|
NA = nil
|
|
@@ -1028,6 +1028,5 @@ module SecID
|
|
|
1028
1028
|
GROUPS.dig(category_code, group_code)
|
|
1029
1029
|
end
|
|
1030
1030
|
end
|
|
1031
|
-
# rubocop:enable Metrics/ModuleLength
|
|
1032
1031
|
end
|
|
1033
1032
|
end
|
|
@@ -22,26 +22,12 @@ module SecID
|
|
|
22
22
|
#
|
|
23
23
|
# def calculate_checksum
|
|
24
24
|
# validate_format_for_calculation!
|
|
25
|
-
# mod10(luhn_sum_standard(
|
|
25
|
+
# mod10(luhn_sum_standard(identifier))
|
|
26
26
|
# end
|
|
27
27
|
# end
|
|
28
28
|
#
|
|
29
29
|
# @see https://en.wikipedia.org/wiki/Luhn_algorithm
|
|
30
30
|
module Checkable
|
|
31
|
-
# Character-to-digit mapping for Luhn algorithm variants.
|
|
32
|
-
# Maps alphanumeric characters to digit arrays for multi-digit expansion.
|
|
33
|
-
# Used by ISIN for checksum calculation.
|
|
34
|
-
CHAR_TO_DIGITS = {
|
|
35
|
-
'0' => 0, '1' => 1, '2' => 2, '3' => 3, '4' => 4,
|
|
36
|
-
'5' => 5, '6' => 6, '7' => 7, '8' => 8, '9' => 9,
|
|
37
|
-
'A' => [1, 0], 'B' => [1, 1], 'C' => [1, 2], 'D' => [1, 3], 'E' => [1, 4],
|
|
38
|
-
'F' => [1, 5], 'G' => [1, 6], 'H' => [1, 7], 'I' => [1, 8], 'J' => [1, 9],
|
|
39
|
-
'K' => [2, 0], 'L' => [2, 1], 'M' => [2, 2], 'N' => [2, 3], 'O' => [2, 4],
|
|
40
|
-
'P' => [2, 5], 'Q' => [2, 6], 'R' => [2, 7], 'S' => [2, 8], 'T' => [2, 9],
|
|
41
|
-
'U' => [3, 0], 'V' => [3, 1], 'W' => [3, 2], 'X' => [3, 3], 'Y' => [3, 4], 'Z' => [3, 5],
|
|
42
|
-
'*' => [3, 6], '@' => [3, 7], '#' => [3, 8]
|
|
43
|
-
}.freeze
|
|
44
|
-
|
|
45
31
|
# Character-to-digit mapping for single-digit conversion.
|
|
46
32
|
# Maps alphanumeric characters to values 0-38 (A=10, B=11, ..., Z=35, *=36, @=37, #=38).
|
|
47
33
|
# Used by CUSIP, FIGI, SEDOL, LEI, and IBAN for checksum calculations.
|
|
@@ -56,6 +42,19 @@ module SecID
|
|
|
56
42
|
'*' => 36, '@' => 37, '#' => 38
|
|
57
43
|
}.freeze
|
|
58
44
|
|
|
45
|
+
# ASCII byte lookup avoids allocating a String for each checksum character.
|
|
46
|
+
BYTE_TO_DIGIT = CHAR_TO_DIGIT.transform_keys(&:ord).freeze
|
|
47
|
+
|
|
48
|
+
# Decimal expansions reused by LEI and IBAN without allocating replacement strings.
|
|
49
|
+
DECIMAL_VALUES = CHAR_TO_DIGIT.transform_values { |value| value.to_s.freeze }.freeze
|
|
50
|
+
|
|
51
|
+
# Doubled decimal digits with their digits summed, for expanded ISIN Luhn.
|
|
52
|
+
LUHN_DOUBLE = [0, 2, 4, 6, 8, 1, 3, 5, 7, 9].freeze
|
|
53
|
+
|
|
54
|
+
# Extends the including identifier class with the concern's class methods and checksum reader.
|
|
55
|
+
#
|
|
56
|
+
# @param base [Class] the identifier class including this concern
|
|
57
|
+
# @return [void]
|
|
59
58
|
# @api private
|
|
60
59
|
def self.included(base)
|
|
61
60
|
base.attr_reader :checksum
|
|
@@ -158,61 +157,46 @@ module SecID
|
|
|
158
157
|
|
|
159
158
|
private
|
|
160
159
|
|
|
161
|
-
# CUSIP/CEI
|
|
162
|
-
#
|
|
163
|
-
# then summing both digit's div10mod10 values.
|
|
164
|
-
#
|
|
165
|
-
# @param digits [Array<Integer>] reversed array of digit values
|
|
166
|
-
# @return [Integer] the Luhn sum
|
|
167
|
-
def luhn_sum_double_add_double(digits)
|
|
168
|
-
digits.each_slice(2).reduce(0) do |sum, (even, odd)|
|
|
169
|
-
double_even = (even || 0) * 2
|
|
170
|
-
sum + div10mod10(double_even) + div10mod10(odd || 0)
|
|
171
|
-
end
|
|
172
|
-
end
|
|
173
|
-
|
|
174
|
-
# FIGI style: index-based doubling algorithm.
|
|
175
|
-
# Doubles odd-indexed digits (from right), then sums div10mod10 values.
|
|
160
|
+
# CUSIP/CEI double the rightmost body character; FIGI starts one position left.
|
|
161
|
+
# Input must already have passed the identifier's ASCII format validation.
|
|
176
162
|
#
|
|
177
|
-
# @param
|
|
163
|
+
# @param id [String] validated identifier body
|
|
164
|
+
# @param double_first [Boolean] whether to double the rightmost value
|
|
178
165
|
# @return [Integer] the Luhn sum
|
|
179
|
-
def
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
166
|
+
def luhn_sum_single(id, double_first: true)
|
|
167
|
+
# @type var id: String
|
|
168
|
+
sum = 0
|
|
169
|
+
double = double_first
|
|
170
|
+
(id.bytesize - 1).downto(0) do |index|
|
|
171
|
+
value = BYTE_TO_DIGIT.fetch(id.getbyte(index))
|
|
172
|
+
value *= 2 if double
|
|
173
|
+
sum += div10mod10(value)
|
|
174
|
+
double = !double
|
|
183
175
|
end
|
|
176
|
+
sum
|
|
184
177
|
end
|
|
185
178
|
|
|
186
|
-
# ISIN
|
|
187
|
-
#
|
|
188
|
-
# subtracting 9 if result > 9.
|
|
179
|
+
# ISIN expands letters into two decimal digits before applying Luhn.
|
|
180
|
+
# A letter consumes two positions, so only a numeric character flips parity.
|
|
189
181
|
#
|
|
190
|
-
# @param
|
|
182
|
+
# @param id [String] validated identifier body
|
|
191
183
|
# @return [Integer] the Luhn sum
|
|
192
|
-
def luhn_sum_standard(
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
184
|
+
def luhn_sum_standard(id)
|
|
185
|
+
# @type var id: String
|
|
186
|
+
sum = 0
|
|
187
|
+
double = true
|
|
188
|
+
(id.bytesize - 1).downto(0) do |index|
|
|
189
|
+
value = BYTE_TO_DIGIT.fetch(id.getbyte(index))
|
|
190
|
+
units = value % 10
|
|
191
|
+
sum += double ? LUHN_DOUBLE.fetch(units) : units
|
|
192
|
+
if value >= 10
|
|
193
|
+
tens = value / 10
|
|
194
|
+
sum += double ? tens : LUHN_DOUBLE.fetch(tens)
|
|
195
|
+
else
|
|
196
|
+
double = !double
|
|
197
|
+
end
|
|
197
198
|
end
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
# Converts identifier characters to reversed digit array using single-digit mapping.
|
|
201
|
-
# Used by CUSIP, CEI, FIGI, and SEDOL.
|
|
202
|
-
#
|
|
203
|
-
# @param id [String] the identifier string
|
|
204
|
-
# @return [Array<Integer>] reversed array of digit values
|
|
205
|
-
def reversed_digits_single(id)
|
|
206
|
-
id.each_char.map { |c| CHAR_TO_DIGIT.fetch(c) }.reverse!
|
|
207
|
-
end
|
|
208
|
-
|
|
209
|
-
# Converts identifier characters to reversed digit array using multi-digit mapping.
|
|
210
|
-
# Used by ISIN where letters expand to two digits.
|
|
211
|
-
#
|
|
212
|
-
# @param id [String] the identifier string
|
|
213
|
-
# @return [Array<Integer>] reversed array of digit values
|
|
214
|
-
def reversed_digits_multi(id)
|
|
215
|
-
id.each_char.flat_map { |c| CHAR_TO_DIGITS.fetch(c) }.reverse!
|
|
199
|
+
sum
|
|
216
200
|
end
|
|
217
201
|
|
|
218
202
|
# Returns error codes including checksum validation.
|
|
@@ -28,6 +28,10 @@ module SecID
|
|
|
28
28
|
# Alphanumeric characters (digits then uppercase letters).
|
|
29
29
|
ALPHANUMERIC = (DIGITS + ALPHA).freeze
|
|
30
30
|
|
|
31
|
+
# Extends the including identifier class with the concern's class methods.
|
|
32
|
+
#
|
|
33
|
+
# @param base [Class] the identifier class including this concern
|
|
34
|
+
# @return [void]
|
|
31
35
|
# @api private
|
|
32
36
|
def self.included(base)
|
|
33
37
|
base.extend(ClassMethods)
|