sec_id 7.0.0 → 7.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +72 -29
- data/README.md +117 -48
- data/lib/sec_id/active_model.rb +5 -4
- data/lib/sec_id/cei.rb +1 -0
- data/lib/sec_id/concerns/checkable.rb +4 -0
- data/lib/sec_id/concerns/generatable.rb +4 -0
- data/lib/sec_id/concerns/normalizable.rb +4 -0
- data/lib/sec_id/concerns/suggestable.rb +173 -0
- data/lib/sec_id/concerns/validatable.rb +4 -0
- data/lib/sec_id/cusip.rb +1 -0
- data/lib/sec_id/dti.rb +1 -0
- data/lib/sec_id/figi.rb +1 -0
- data/lib/sec_id/iban.rb +1 -0
- data/lib/sec_id/isin.rb +1 -0
- data/lib/sec_id/lei.rb +1 -0
- data/lib/sec_id/sedol.rb +1 -0
- data/lib/sec_id/suggestion.rb +63 -0
- data/lib/sec_id/upi.rb +1 -0
- data/lib/sec_id/version.rb +1 -1
- data/lib/sec_id.rb +36 -0
- data/sec_id.gemspec +5 -4
- data/sig/sec_id/base.rbs +9 -3
- data/sig/sec_id/concerns/checkable.rbs +2 -1
- data/sig/sec_id/concerns/suggestable.rbs +32 -0
- data/sig/sec_id/suggestion.rbs +23 -0
- data/sig/sec_id.rbs +2 -0
- metadata +11 -6
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: be4a59e18a04226f78492df3eb2ec1318af8e7c99044005bda0f7b9652ac7a33
|
|
4
|
+
data.tar.gz: 6edd9226b4cc7b293a7fe1950f32aa0673ea0b44507b36ecc253da98f372d3c2
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: b4b95d04179e0aee0ff4c45dbc2e169a024d3933a2c8234de2319422d19773d34247d83b8f457317af93d536ac431406fca9455a454cc7159ae2a9bad8982955
|
|
7
|
+
data.tar.gz: f479c160b3229763197b1ac1fe3b7d67f8a6fc3281d3607adf59fece50e89cc8dd2160eb2d4f94d32954d136bc2e8c8998b19bfcdf1e24f4e32a2a035f9e972a
|
data/CHANGELOG.md
CHANGED
|
@@ -8,58 +8,76 @@ and [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/).
|
|
|
8
8
|
|
|
9
9
|
## [Unreleased]
|
|
10
10
|
|
|
11
|
+
## [7.1.1] - 2026-07-31
|
|
12
|
+
|
|
13
|
+
Documentation and packaging only. No identifier, checksum, or public method changed.
|
|
14
|
+
|
|
15
|
+
This is the first release published from CI through RubyGems trusted publishing, signed with sigstore and pushed with a build provenance attestation.
|
|
16
|
+
|
|
17
|
+
### Changed
|
|
18
|
+
|
|
19
|
+
- README declares the public API that Semantic Versioning covers, names the `@api private` internals excluded from it, and links where to ask a question and where to report a defect
|
|
20
|
+
- CHANGELOG version headings resolve to comparison links, and 2.0.1 has the entry it never had
|
|
21
|
+
- Gem summary punctuation matches the README tagline
|
|
22
|
+
|
|
23
|
+
## [7.1.0] - 2026-07-15
|
|
24
|
+
|
|
25
|
+
### Added
|
|
26
|
+
|
|
27
|
+
- `suggest`, a checksum-anchored diagnostic-repair verb that turns the checksum from a gatekeeper into a repair engine. For a structurally-valid but checksum-**failing** identifier, `SecID::<Type>.suggest(str)` and `SecID.suggest(str, types:)` enumerate the plausible single-character human errors (visual/OCR homoglyph substitutions such as `O`↔`0`, `I`↔`1`, `5`↔`S`, `8`↔`B`, plus adjacent transpositions), keep only those that re-validate (`valid?` is the oracle, so no checksum-invalid candidate ever escapes, though a valid candidate is not necessarily the intended correction), and return them as confidence-ranked `SecID::Suggestion` value objects reporting *what changed* (edit kind, position, from/to characters, confidence tier). Available for all 9 checksum types (ISIN, CUSIP, SEDOL, FIGI, LEI, IBAN, CEI, DTI, UPI) through one shared `Suggestable` concern. Candidates rank `:high` (homoglyph) then `:medium` (transposition) then a `:checksum` recompute fallback last; there is no `:low` tier (coincidental substitutions are never generated), keeping results small and high-precision. `suggest` never mutates its input and never presents a candidate as an authoritative correction. Non-checksum types (CIK, OCC, WKN, Valoren, CFI, FISN, BIC) have no checksum oracle and are unsupported; `SecID.suggest` silently skips them. Known limitations: the mistyped character must be in the type's charset to be reachable (the vowel-free SEDOL/FIGI/DTI/UPI can't repair an `O`-for-`0`/`I`-for-`1` typo), only a single body error is in scope, and wrong-length input returns `[]`. In a seeded simulation (`benchmark/suggest_precision.rb`), every reachable single-error homoglyph or transposition that yields a checksum-failing identifier is recovered (the correct identifier is always among the returned candidates), the top-ranked body candidate ~89% of the time for homoglyph errors
|
|
28
|
+
|
|
11
29
|
## [7.0.0] - 2026-07-14
|
|
12
30
|
|
|
13
31
|
### Added
|
|
14
32
|
|
|
15
|
-
- UPI (ISO 4914, Unique Product Identifier) support via `SecID::UPI
|
|
33
|
+
- UPI (ISO 4914, Unique Product Identifier) support via `SecID::UPI`: the gem's 16th identifier type, and the first offline UPI validator in any language. Validates the 12-character code (fixed `QZ` prefix, 9-character body, 1 check character) issued by the ANNA Derivatives Service Bureau for OTC-derivatives reporting (CFTC, EMIR, and other global mandates). The check character is computed fully offline via ISO 7064 hybrid MOD 31,30 over the same 30-symbol alphabet as DTI (digits plus consonants; vowels and `Y` never appear), pinned empirically against DSB-issued vectors, with no DSB registry lookup or paywalled ISO 4914 spec required. Like DTI, `checksum`/`calculate_checksum` return a `String` rather than an `Integer`, since UPI check characters can be letters. A UPI shares the 12-character length bucket with ISIN and can double-detect (ISIN ranked first) when its digit check character also satisfies ISIN's Luhn checksum.
|
|
16
34
|
|
|
17
35
|
### Changed
|
|
18
36
|
|
|
19
|
-
- BREAKING: Renamed the check-digit concept to **checksum** across the entire public API, because the old name was wrong on two axes
|
|
37
|
+
- BREAKING: Renamed the check-digit concept to **checksum** across the entire public API, because the old name was wrong on two axes. DTI's and UPI's check value is a `String` (it can be a letter, not a digit), and LEI and IBAN carry a two-character check value. The instance and class methods `check_digit` → `checksum`, `calculate_check_digit` → `calculate_checksum`, and `has_check_digit?` → `has_checksum?`; the error class `SecID::InvalidCheckDigitError` → `SecID::InvalidChecksumError`; the error code `:invalid_check_digit` → `:invalid_checksum` (a **hard flip with no dual emission**: dual would duplicate `errors.details` entries, so update any `errors.details`/`explain`/ActiveModel `details: true` matcher immediately); and the `:check_digit` components key → `:checksum` in `to_h`/`deconstruct_keys` across all nine checkable types. `restore`/`restore!` and the `Checkable` concern name are unchanged. No validation or checksum-arithmetic behavior changed: every identifier that was valid stays valid and every computed value is byte-identical. The old method names, the `InvalidCheckDigitError` constant, and the `:check_digit` components key remain as deprecated bridges through v7 (removed in v8); the error code is the only surface with no bridge. See MIGRATION.md for the full upgrade guide
|
|
20
38
|
|
|
21
39
|
### Deprecated
|
|
22
40
|
|
|
23
|
-
- The pre-rename check-digit names, kept as v7 bridges and removed in v8: the `check_digit` / `calculate_check_digit` / `has_check_digit?` methods (instance and class level)
|
|
41
|
+
- The pre-rename check-digit names, kept as v7 bridges and removed in v8: the `check_digit` / `calculate_check_digit` / `has_check_digit?` methods (instance and class level). Each warns via `Kernel#warn` on every call, visible at Ruby's default verbosity and silenceable with `-W0` / `$VERBOSE = nil` or an app-level `Warning` override; the `SecID::InvalidCheckDigitError` constant, a same-object alias of `InvalidChecksumError` so `rescue` under either name keeps working; and the `:check_digit` key, still present alongside `:checksum` in `components` / `to_h` / `deconstruct_keys`
|
|
24
42
|
|
|
25
43
|
## [6.1.0] - 2026-07-10
|
|
26
44
|
|
|
27
45
|
### Added
|
|
28
46
|
|
|
29
|
-
- `SecID::Base#deconstruct_keys
|
|
30
|
-
- `SecID::Base.type_key
|
|
47
|
+
- `SecID::Base#deconstruct_keys`: every identifier now destructures in `case/in` patterns, exposing the same parsed fields `#to_h` reports under `:components`. The `keys` argument is ignored, no new keys are introduced, and validity is not part of the protocol: `SecID.parse` returning `nil` remains the validity guard, and an instance built from unparseable input binds `nil` for each component. No `deconstruct` (array-pattern) method is defined
|
|
48
|
+
- `SecID::Base.type_key`: the memoized class-level registry symbol for an identifier type, joining the existing `short_name` / `full_name` / `id_length` metadata surface. It round-trips through the registry: `SecID[SecID::ISIN.type_key] == SecID::ISIN`. It is now the single authority for that symbol; the registry, `#to_h`, `SecID.explain`, the ActiveModel validator, the detector, and the scanner all read it instead of each deriving it from the class name
|
|
31
49
|
|
|
32
50
|
## [6.0.0] - 2026-07-08
|
|
33
51
|
|
|
34
52
|
### Added
|
|
35
53
|
|
|
36
|
-
- Hand-written RBS type signatures under `sig/`, shipped in the gem, so consumers running Steep or an RBS-aware editor resolve sec_id's types on install. The core library is checked by Steep in strict mode with zero errors and verified at runtime against the specs via RBS::Test. New dev/test tooling (`rbs` and `steep`, `require: false
|
|
37
|
-
- `SecID::CFI#decode` returns a frozen `SecID::CFI::Classification` value object for any valid CFI (`nil` for invalid). Its category, group, and each attribute are `Field` objects carrying the CFI letter (`#code`), semantic symbol (`#name`), and authoritative ISO 10962 label (`#label`), plus `<name>?` predicates scoped to the field's own domain
|
|
38
|
-
- `SecID::CFI` now validates the full ISO 10962:2021 attribute matrix
|
|
39
|
-
- Opt-in ActiveModel / Rails validator (`require 'sec_id/active_model'`), registered as `sec_id`, for declarative validation: `validates :isin, sec_id: { type: :isin }`. Validates a single type (`type:`), an allowlist (`types:`), or any supported type (`sec_id: true`); a bad type raises `ArgumentError` at class-load. Opt-in `normalize: true` accepts separatored input and rewrites the attribute to canonical form on success; opt-in `details: true` surfaces the specific failure reason. Auto-activates in Rails via a Railtie (no `require:`/initializer needed) and adds **no runtime dependency
|
|
40
|
-
- BIC / SWIFT code (ISO 9362) support via `SecID::BIC
|
|
41
|
-
- DTI (ISO 24165, Digital Token Identifier) support via `SecID::DTI
|
|
42
|
-
- `.generate` on all 15 identifier types plus a central `SecID.generate(:type)` dispatcher for producing syntactically valid identifiers (with correct check digits where applicable) as test fixtures
|
|
43
|
-
- 100% YARD documentation coverage of the public API, enforced in CI via `rake yard:stats`. The gem now ships a `.yardopts` so `rubydoc.info` renders the public API cleanly (private internals hidden, README/CHANGELOG/LICENSE as pages) and exposes `documentation_uri` metadata (a Documentation link on the RubyGems page). Adds a `yard` dev/test dependency (`require: false
|
|
54
|
+
- Hand-written RBS type signatures under `sig/`, shipped in the gem, so consumers running Steep or an RBS-aware editor resolve sec_id's types on install. The core library is checked by Steep in strict mode with zero errors and verified at runtime against the specs via RBS::Test. New dev/test tooling (`rbs` and `steep`, `require: false`; the gem stays zero-runtime-dependency) and rake tasks: `rake steep` (strict type check), `rake steep:coverage` (untyped-call gate), `rake rbs:test` (runtime signature verification), `rake rbs` (validate signatures, in the default task), and `rake sig:cfi` (regenerate the CFI dynamic-method signatures from `SecID::CFI::Tables`, guarded by a drift spec). CI gains a `types` job on Ruby 4.0. The optional ActiveModel adapter and Railtie are excluded from the typed scope
|
|
55
|
+
- `SecID::CFI#decode` returns a frozen `SecID::CFI::Classification` value object for any valid CFI (`nil` for invalid). Its category, group, and each attribute are `Field` objects carrying the CFI letter (`#code`), semantic symbol (`#name`), and authoritative ISO 10962 label (`#label`), plus `<name>?` predicates scoped to the field's own domain, so `decode.category.equity?` and `decode.attributes.voting_right.voting?` answer, while an out-of-domain predicate (e.g. `category.voting?`) raises `NoMethodError`. `#attributes` is an `Enumerable` of the attribute fields keyed by each position's group meaning (`attributes.voting_right`, nil-safe `attributes[:form]`). Ships a human-readable `#to_s` and `#to_h`/`#as_json` for serialization. `X` decodes to `:not_applicable`; pure-N/A positions are omitted
|
|
56
|
+
- `SecID::CFI` now validates the full ISO 10962:2021 attribute matrix: positions 3 to 6 are checked per-group, `Strategies` (`K`) codes require `XXXX`, pure-N/A positions accept only `X`, and the `ED` (depositary receipts on equities) cross-position rule is enforced. A new `:invalid_attribute` error code (mapped to `InvalidStructureError`) names the offending positions and group
|
|
57
|
+
- Opt-in ActiveModel / Rails validator (`require 'sec_id/active_model'`), registered as `sec_id`, for declarative validation: `validates :isin, sec_id: { type: :isin }`. Validates a single type (`type:`), an allowlist (`types:`), or any supported type (`sec_id: true`); a bad type raises `ArgumentError` at class-load. Opt-in `normalize: true` accepts separatored input and rewrites the attribute to canonical form on success; opt-in `details: true` surfaces the specific failure reason. Auto-activates in Rails via a Railtie (no `require:`/initializer needed) and adds **no runtime dependency**: nothing on the default `require 'sec_id'` path loads Rails or ActiveModel. Tested across Rails 7.2, 8.0, 8.1, and main
|
|
58
|
+
- BIC / SWIFT code (ISO 9362) support via `SecID::BIC`: validate, normalize, parse, detect, extract, and generate 8- or 11-character Business Identifier Codes. Exposes `bank_code`, `country_code`, `location_code`, and `branch_code` (`nil` for a BIC8) components. Validates the embedded country code against a frozen ISO 3166-1 / SWIFT-recognized set (`SecID::BIC.countries`), raising `InvalidStructureError` (`:invalid_country`) for unrecognized codes. Validation confirms structure and a real country code only. It does not verify registration in the licensed SWIFT registry
|
|
59
|
+
- DTI (ISO 24165, Digital Token Identifier) support via `SecID::DTI`: the gem's 15th identifier type, and the first offline DTI validator in any language. Validates the ISO 7064 hybrid MOD 31,30 check character over the 30-symbol DTI alphabet (digits plus consonants; vowels and `Y` never appear), identified empirically from public registry data (1,595 of 1,596 snapshot codes plus 2 post-snapshot registrations). Bitcoin's hand-assigned code (`4H95J0R2X`, which fails the algorithm) is honored via a frozen exception map consulted by `valid?`, `restore`, and `check_digit` alike. Unlike every other check-digit type in the gem, `check_digit`/`calculate_check_digit` return a `String` rather than an `Integer`, since DTI check characters can be letters
|
|
60
|
+
- `.generate` on all 15 identifier types plus a central `SecID.generate(:type)` dispatcher for producing syntactically valid identifiers (with correct check digits where applicable) as test fixtures. Accepts an optional `random:` keyword (a Ruby `Random`) for reproducible output. Generated values are valid in format only and are not real, registered securities
|
|
61
|
+
- 100% YARD documentation coverage of the public API, enforced in CI via `rake yard:stats`. The gem now ships a `.yardopts` so `rubydoc.info` renders the public API cleanly (private internals hidden, README/CHANGELOG/LICENSE as pages) and exposes `documentation_uri` metadata (a Documentation link on the RubyGems page). Adds a `yard` dev/test dependency (`require: false`; the gem stays zero-runtime-dependency) and a `rake yard` task for local HTML docs
|
|
44
62
|
|
|
45
63
|
### Changed
|
|
46
64
|
|
|
47
65
|
- Internal `@api private` IBAN country-data module dissolved into the `SecID::IBAN` class, so constant names match their file paths per Ruby convention: `SecID::IBANCountryRules::COUNTRY_RULES` / `LENGTH_ONLY_COUNTRIES` are now `SecID::IBAN::COUNTRY_RULES` / `SecID::IBAN::LENGTH_ONLY_COUNTRIES`
|
|
48
|
-
- `SecID.valid?` and `SecID.parse` are faster and allocate less
|
|
49
|
-
- **BREAKING:** `SecID::CFI.valid?` is now strict at the attribute level for all 14 categories (including the derivative categories `S`, `H`, `J`)
|
|
50
|
-
- **BREAKING:** `SecID::CFI` group tables corrected to ISO 10962:2021
|
|
66
|
+
- `SecID.valid?` and `SecID.parse` are faster and allocate less. `SecID.valid?` short-circuits on the first matching type instead of running a full detect-and-sort, and `SecID.parse` reuses the instance built during detection instead of instantiating the matched type a second time
|
|
67
|
+
- **BREAKING:** `SecID::CFI.valid?` is now strict at the attribute level for all 14 categories (including the derivative categories `S`, `H`, `J`). Codes with attribute letters outside the ISO 10962:2021 tables, previously accepted, are now invalid (e.g. `CFI.valid?('ESZZZZ')` returns `false`). There is no leniency option, matching every other identifier type in the gem. The ActiveModel validator inherits this strictness automatically
|
|
68
|
+
- **BREAKING:** `SecID::CFI` group tables corrected to ISO 10962:2021: six categories carried wrong group letters. Non-listed options (`H`) are now classified by underlying (`HR` Rates, `HT` Commodities, `HE` Equity, `HC` Credit, `HF` FX, `HM` Others) instead of the copied listed-option call/put shape; `D` gains `DE` (structured products without capital protection) and corrects `DG`/`DA`/`DN` (dropping the invented `municipal_notes`); `L` becomes `LL`/`LR`/`LS`, `T` becomes `TC`/`TT`/`TR`/`TI`/…, and the phantom `FM`/`IM`/`JM`/`LM` groups are removed. Group symbols renamed accordingly (e.g. `LS` → `:securities_lending`, `TI` → `:indices`, `MM` → `:other_assets`). Real `H`-category codes that were rejected now validate; codes relying on the old wrong letters no longer do
|
|
51
69
|
|
|
52
70
|
### Removed
|
|
53
71
|
|
|
54
|
-
- **BREAKING:** the 12 hardcoded equity predicate helpers on `SecID::CFI` (`equity?`, `voting?`, `non_voting?`, `restricted_voting?`, `enhanced_voting?`, `restrictions?`, `no_restrictions?`, `fully_paid?`, `nil_paid?`, `partly_paid?`, `bearer?`, `registered?`). They were category-wide and semantically wrong for non-equity groups. Migration: use `cfi.decode` and its scoped field predicates
|
|
72
|
+
- **BREAKING:** the 12 hardcoded equity predicate helpers on `SecID::CFI` (`equity?`, `voting?`, `non_voting?`, `restricted_voting?`, `enhanced_voting?`, `restrictions?`, `no_restrictions?`, `fully_paid?`, `nil_paid?`, `partly_paid?`, `bearer?`, `registered?`). They were category-wide and semantically wrong for non-equity groups. Migration: use `cfi.decode` and its scoped field predicates, so `cfi.voting?` becomes `cfi.decode.attributes.voting_right.voting?`, which answers only on the field whose ISO domain defines the concept. Two names do not carry over name-for-name: migrate `cfi.equity?` to `cfi.decode.category.equity?` (or `cfi.category == :equity`, unchanged), and `cfi.no_restrictions?` to `cfi.decode.attributes.ownership_restrictions.free_of_restrictions?` (the ISO value name)
|
|
55
73
|
|
|
56
74
|
## [5.2.0] - 2026-02-24
|
|
57
75
|
|
|
58
76
|
### Added
|
|
59
77
|
|
|
60
|
-
- `SecID.scan` and `SecID.extract` methods for finding identifiers in freeform text
|
|
61
|
-
- `SecID.explain` method for debugging identifier detection
|
|
62
|
-
- `on_ambiguous:` option for `SecID.parse` and `SecID.parse
|
|
78
|
+
- `SecID.scan` and `SecID.extract` methods for finding identifiers in freeform text. Returns `Scanner::Match` objects (`Data.define(:type, :raw, :range, :identifier)`) with the validated identifier instance; supports `types:` filtering, hyphenated identifiers, and compound patterns (OCC with spaces, FISN with slashes)
|
|
79
|
+
- `SecID.explain` method for debugging identifier detection. Returns per-type validation results showing exactly why each type matched or rejected the input
|
|
80
|
+
- `on_ambiguous:` option for `SecID.parse` and `SecID.parse!`: `:first` (default, existing behavior), `:raise` (raises `AmbiguousMatchError`), `:all` (returns array of all matching instances)
|
|
63
81
|
- `SecID::AmbiguousMatchError` exception class for ambiguous identifier detection
|
|
64
82
|
- `#as_json` method on all identifier types (delegates to `#to_h`) and on `Errors` (delegates to `#details`) for JSON serialization compatibility
|
|
65
83
|
- `SecID::IBAN.supported_countries` class method returning sorted array of all supported country codes
|
|
@@ -71,9 +89,9 @@ and [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/).
|
|
|
71
89
|
|
|
72
90
|
### Added
|
|
73
91
|
|
|
74
|
-
- `#==`, `#eql?`, and `#hash` methods on all identifier types
|
|
75
|
-
- `#to_h` method on all identifier types for consistent hash serialization
|
|
76
|
-
- `#to_pretty_s` and `.to_pretty_s` display formatting methods on all identifier types, returning a human-readable string or `nil` for invalid input
|
|
92
|
+
- `#==`, `#eql?`, and `#hash` methods on all identifier types. Two instances of the same type with the same normalized form are equal and usable as Hash keys / in Sets
|
|
93
|
+
- `#to_h` method on all identifier types for consistent hash serialization. Returns `{ type:, full_id:, normalized:, valid:, components: }` with type-specific component hashes (e.g. ISIN: `country_code`, `nsin`, `check_digit`)
|
|
94
|
+
- `#to_pretty_s` and `.to_pretty_s` display formatting methods on all identifier types, returning a human-readable string or `nil` for invalid input, with type-specific formats for IBAN (4-char groups), LEI (4-char groups), ISIN (CC + NSIN + CD), CUSIP (cusip6 + issue + CD), FIGI (prefix+G + random + CD), OCC (space-separated components), and Valoren (thousands grouping)
|
|
77
95
|
- Lookup service integration guides and runnable examples for OpenFIGI, SEC EDGAR, GLEIF, and Eurex APIs (`docs/guides/`, `examples/`)
|
|
78
96
|
- GitHub community standards files: Code of Conduct, Contributing guide, Security policy, issue templates, and PR template
|
|
79
97
|
|
|
@@ -111,15 +129,15 @@ and [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/).
|
|
|
111
129
|
|
|
112
130
|
### Removed
|
|
113
131
|
|
|
114
|
-
- Class-level `.normalize!` on CIK, OCC, and Valoren
|
|
132
|
+
- Class-level `.normalize!` on CIK, OCC, and Valoren, replaced by `.normalize`
|
|
115
133
|
- `upcase` keyword parameter from `Base#parse`
|
|
116
134
|
- `#valid_format?` instance method (now private) and `.valid_format?` class method
|
|
117
|
-
- `OCC#full_symbol` method
|
|
135
|
+
- `OCC#full_symbol` method. Use `#full_id` instead
|
|
118
136
|
|
|
119
137
|
### Fixed
|
|
120
138
|
|
|
121
|
-
- `to_str` now always returns the same value as `to_s` across all identifier types
|
|
122
|
-
- OCC `#date` memoization for invalid dates
|
|
139
|
+
- `to_str` now always returns the same value as `to_s` across all identifier types. Previously LEI, IBAN, and Checkable identifiers could return divergent strings due to Ruby `alias` resolving to the parent class method
|
|
140
|
+
- OCC `#date` memoization for invalid dates. Previously it re-attempted parsing on every call instead of caching `nil`
|
|
123
141
|
- LEI `restore` and `to_s` now correctly pad single-digit check digits to 2 characters
|
|
124
142
|
- `Valoren#to_isin` no longer mutates the source instance
|
|
125
143
|
|
|
@@ -153,7 +171,7 @@ and [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/).
|
|
|
153
171
|
### Fixed
|
|
154
172
|
|
|
155
173
|
- Allow Crown Dependencies (GG, IM, JE) and Overseas Territories (FK) in SEDOL/ISIN conversions ([@wtn](https://github.com/wtn), [#117](https://github.com/svyatov/sec_id/pull/117))
|
|
156
|
-
- Removed BR (Brazil) from CGS country codes
|
|
174
|
+
- Removed BR (Brazil) from CGS country codes. Brazil never used CINS numbers and Brazilian ISINs cannot be converted to CUSIP ([@wtn](https://github.com/wtn), [#110](https://github.com/svyatov/sec_id/pull/110))
|
|
157
175
|
|
|
158
176
|
## [4.3.0] - 2025-01-13
|
|
159
177
|
|
|
@@ -168,7 +186,7 @@ and [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/).
|
|
|
168
186
|
- Refactored CIK and OCC to inherit from Base class with `has_check_digit?` hook for cleaner architecture
|
|
169
187
|
- Added `Normalizable` module for consistent `normalize!` class method across identifiers
|
|
170
188
|
- Added `validate_format_for_calculation!` helper method to Base class to reduce code duplication
|
|
171
|
-
- Added
|
|
189
|
+
- Added YARD documentation to all classes (public and private methods)
|
|
172
190
|
- Applied Stepdown Rule to method ordering throughout codebase
|
|
173
191
|
- Created shared RSpec examples for edge cases (nil, empty, whitespace inputs)
|
|
174
192
|
- Created shared RSpec examples for check-digit and normalization identifiers
|
|
@@ -226,6 +244,10 @@ and [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/).
|
|
|
226
244
|
- TravisCI config updated: dropped Ruby 2.3 and 2.4, added Ruby 2.7
|
|
227
245
|
- Rubocop's Ruby target version changed to 2.5
|
|
228
246
|
|
|
247
|
+
## [2.0.1] - 2019-02-04
|
|
248
|
+
|
|
249
|
+
Gem metadata only: corrected the gemspec description and README wording. No library change.
|
|
250
|
+
|
|
229
251
|
## [2.0.0] - 2019-02-03
|
|
230
252
|
|
|
231
253
|
### Added
|
|
@@ -262,3 +284,24 @@ and [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/).
|
|
|
262
284
|
### Added
|
|
263
285
|
|
|
264
286
|
- ISIN numbers support: `SecID::ISIN`
|
|
287
|
+
|
|
288
|
+
[Unreleased]: https://github.com/svyatov/sec_id/compare/v7.1.1...HEAD
|
|
289
|
+
[7.1.1]: https://github.com/svyatov/sec_id/compare/v7.1.0...v7.1.1
|
|
290
|
+
[7.1.0]: https://github.com/svyatov/sec_id/compare/v7.0.0...v7.1.0
|
|
291
|
+
[7.0.0]: https://github.com/svyatov/sec_id/compare/v6.1.0...v7.0.0
|
|
292
|
+
[6.1.0]: https://github.com/svyatov/sec_id/compare/v6.0.0...v6.1.0
|
|
293
|
+
[6.0.0]: https://github.com/svyatov/sec_id/compare/v5.2.0...v6.0.0
|
|
294
|
+
[5.2.0]: https://github.com/svyatov/sec_id/compare/v5.1.0...v5.2.0
|
|
295
|
+
[5.1.0]: https://github.com/svyatov/sec_id/compare/v5.0.0...v5.1.0
|
|
296
|
+
[5.0.0]: https://github.com/svyatov/sec_id/compare/v4.4.1...v5.0.0
|
|
297
|
+
[4.4.1]: https://github.com/svyatov/sec_id/compare/v4.4.0...v4.4.1
|
|
298
|
+
[4.4.0]: https://github.com/svyatov/sec_id/compare/v4.3.0...v4.4.0
|
|
299
|
+
[4.3.0]: https://github.com/svyatov/sec_id/compare/v4.2.0...v4.3.0
|
|
300
|
+
[4.2.0]: https://github.com/svyatov/sec_id/compare/v4.1.0...v4.2.0
|
|
301
|
+
[4.1.0]: https://github.com/svyatov/sec_id/compare/v4.0.0...v4.1.0
|
|
302
|
+
[4.0.0]: https://github.com/svyatov/sec_id/compare/v3.0.0...v4.0.0
|
|
303
|
+
[3.0.0]: https://github.com/svyatov/sec_id/compare/v2.0.1...v3.0.0
|
|
304
|
+
[2.0.1]: https://github.com/svyatov/sec_id/compare/v2.0.0...v2.0.1
|
|
305
|
+
[2.0.0]: https://github.com/svyatov/sec_id/compare/v1.1.0...v2.0.0
|
|
306
|
+
[1.1.0]: https://github.com/svyatov/sec_id/compare/v1.0.0...v1.1.0
|
|
307
|
+
[1.0.0]: https://github.com/svyatov/sec_id/releases/tag/v1.0.0
|
data/README.md
CHANGED
|
@@ -1,18 +1,27 @@
|
|
|
1
|
-
# SecID
|
|
1
|
+
# SecID
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
A Ruby toolkit for securities identifiers: validate, parse, normalize, detect, convert, generate, classify, and repair.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
[](https://rubygems.org/gems/sec_id) [](https://github.com/svyatov/sec_id/actions/workflows/main.yml) [](https://codecov.io/gh/svyatov/sec_id) [](https://rubydoc.info/gems/sec_id) [](https://www.ruby-lang.org) [](https://github.com/svyatov/sec_id/tree/main/sig)
|
|
6
6
|
|
|
7
|
-
-
|
|
7
|
+
- **Nothing to call, nothing to sign up for.** SecID validates all 16 identifier standards offline. No registry lookup, no API key, no network call anywhere in [`lib/`](lib/).
|
|
8
|
+
- **9 of the 16 carry a checksum.** SecID validates it, restores a missing one, and repairs a wrong one.
|
|
9
|
+
- **Zero runtime dependencies.** Runs on Ruby 3.2 or newer.
|
|
10
|
+
- **Rails, if you want it.** An opt-in ActiveModel validator adds `validates :isin, sec_id: true`. It stays off the default require path, so the dependency count stays at zero.
|
|
11
|
+
- **Typed.** Ships RBS signatures for the whole public API, checked by Steep in strict mode.
|
|
12
|
+
|
|
13
|
+
## Table of contents
|
|
14
|
+
|
|
15
|
+
- [Supported Ruby versions](#supported-ruby-versions)
|
|
8
16
|
- [Installation](#installation)
|
|
9
|
-
- [Supported
|
|
10
|
-
- [Metadata
|
|
11
|
-
- [Text
|
|
12
|
-
- [Debugging
|
|
13
|
-
- [Structured
|
|
14
|
-
- [Pattern
|
|
15
|
-
- [Generating
|
|
17
|
+
- [Supported standards and usage](#supported-standards-and-usage)
|
|
18
|
+
- [Metadata registry](#metadata-registry) - enumerate, filter, look up, and detect identifier types
|
|
19
|
+
- [Text scanning](#text-scanning) - find identifiers in freeform text
|
|
20
|
+
- [Debugging detection](#debugging-detection) - understand why strings match or don't
|
|
21
|
+
- [Structured validation](#structured-validation) - detailed error codes and messages
|
|
22
|
+
- [Pattern matching](#pattern-matching) - destructure identifiers with `case/in`
|
|
23
|
+
- [Generating test fixtures](#generating-test-fixtures) - produce valid identifiers for tests
|
|
24
|
+
- [Repairing typos](#repairing-typos) - suggest corrections for checksum-failing identifiers
|
|
16
25
|
- [ISIN](#isin) - International Securities Identification Number
|
|
17
26
|
- [CUSIP](#cusip) - Committee on Uniform Securities Identification Procedures
|
|
18
27
|
- [CEI](#cei) - CUSIP Entity Identifier
|
|
@@ -29,16 +38,17 @@
|
|
|
29
38
|
- [BIC](#bic) - Business Identifier Code / SWIFT code
|
|
30
39
|
- [DTI](#dti) - Digital Token Identifier
|
|
31
40
|
- [UPI](#upi) - Unique Product Identifier
|
|
32
|
-
- [ActiveModel / Rails
|
|
33
|
-
- [Lookup
|
|
34
|
-
- [Type
|
|
41
|
+
- [ActiveModel / Rails validator](#activemodel--rails-validator) - declarative `validates :isin, sec_id: {...}`
|
|
42
|
+
- [Lookup service integration](#lookup-service-integration)
|
|
43
|
+
- [Type signatures (RBS)](#type-signatures-rbs)
|
|
35
44
|
- [Development](#development)
|
|
45
|
+
- [Support and status](#support-and-status)
|
|
36
46
|
- [Contributing](#contributing)
|
|
37
47
|
- [Changelog](#changelog)
|
|
38
48
|
- [Versioning](#versioning)
|
|
39
49
|
- [License](#license)
|
|
40
50
|
|
|
41
|
-
## Supported Ruby
|
|
51
|
+
## Supported Ruby versions
|
|
42
52
|
|
|
43
53
|
Ruby 3.2+ is required.
|
|
44
54
|
|
|
@@ -47,7 +57,7 @@ Ruby 3.2+ is required.
|
|
|
47
57
|
Add this line to your application's Gemfile:
|
|
48
58
|
|
|
49
59
|
```ruby
|
|
50
|
-
gem 'sec_id', '~> 7.
|
|
60
|
+
gem 'sec_id', '~> 7.1'
|
|
51
61
|
```
|
|
52
62
|
|
|
53
63
|
And then execute:
|
|
@@ -64,7 +74,7 @@ gem install sec_id
|
|
|
64
74
|
|
|
65
75
|
**Upgrading from v4?** See [MIGRATION.md](MIGRATION.md) for a step-by-step guide.
|
|
66
76
|
|
|
67
|
-
## Supported
|
|
77
|
+
## Supported standards and usage
|
|
68
78
|
|
|
69
79
|
All identifier classes provide `valid?`, `errors`, `validate`, `validate!` methods at both class and instance levels.
|
|
70
80
|
|
|
@@ -89,7 +99,7 @@ SecID::ISIN.new('INVALID').to_h
|
|
|
89
99
|
# valid: false, components: { country_code: nil, nsin: nil, checksum: nil } }
|
|
90
100
|
```
|
|
91
101
|
|
|
92
|
-
**All identifiers** support value equality
|
|
102
|
+
**All identifiers** support value equality. Two instances of the same type with the same normalized form are equal:
|
|
93
103
|
|
|
94
104
|
```ruby
|
|
95
105
|
a = SecID::ISIN.new('US5949181045')
|
|
@@ -108,7 +118,7 @@ Set.new([a, b]).size # => 1
|
|
|
108
118
|
- `restore!` / `.restore!` - restores checksum in place and returns `self` / instance
|
|
109
119
|
- `checksum` / `calculate_checksum` - calculates and returns the checksum
|
|
110
120
|
|
|
111
|
-
### Metadata
|
|
121
|
+
### Metadata registry
|
|
112
122
|
|
|
113
123
|
All identifier classes are registered automatically and can be enumerated, filtered, and looked up by symbol key:
|
|
114
124
|
|
|
@@ -164,7 +174,7 @@ SecID.parse('514000', on_ambiguous: :all) # => [#<SecID::WKN>, #<SecID:
|
|
|
164
174
|
SecID.parse('US5949181045', on_ambiguous: :raise) # => #<SecID::ISIN> (unambiguous, no error)
|
|
165
175
|
```
|
|
166
176
|
|
|
167
|
-
### Text
|
|
177
|
+
### Text scanning
|
|
168
178
|
|
|
169
179
|
Find identifiers embedded in freeform text:
|
|
170
180
|
|
|
@@ -190,12 +200,12 @@ match.identifier.normalized # => "US5949181045"
|
|
|
190
200
|
|
|
191
201
|
> **Known limitations:** Format-only types (CIK, Valoren, WKN, BIC) can false-positive on
|
|
192
202
|
> common numbers and short words in prose (a BIC8 is 8 letters with a valid country code in the
|
|
193
|
-
> middle)
|
|
203
|
+
> middle). Use the `types:` filter to restrict scanning when
|
|
194
204
|
> this is a concern. Identifiers prefixed with special characters (e.g. `#US5949181045`) may be
|
|
195
205
|
> consumed as a single token by CUSIP's `*@#` character class and fail validation, preventing
|
|
196
206
|
> the embedded identifier from being found.
|
|
197
207
|
|
|
198
|
-
### Debugging
|
|
208
|
+
### Debugging detection
|
|
199
209
|
|
|
200
210
|
Understand why a string matches or doesn't match specific identifier types:
|
|
201
211
|
|
|
@@ -209,7 +219,7 @@ isin[:errors].first[:error] # => :invalid_checksum
|
|
|
209
219
|
SecID.explain('US5949181045', types: %i[isin cusip])
|
|
210
220
|
```
|
|
211
221
|
|
|
212
|
-
### Structured
|
|
222
|
+
### Structured validation
|
|
213
223
|
|
|
214
224
|
All identifier classes provide a Rails-like `#errors` API for detailed error reporting:
|
|
215
225
|
|
|
@@ -265,7 +275,7 @@ SecID::FIGI.new('BSG000BLNNH6').validate!
|
|
|
265
275
|
isin = SecID::ISIN.validate!('US5949181045') # => #<SecID::ISIN>
|
|
266
276
|
```
|
|
267
277
|
|
|
268
|
-
### Pattern
|
|
278
|
+
### Pattern matching
|
|
269
279
|
|
|
270
280
|
Every identifier destructures in `case/in` via its parsed components. Since `SecID.parse` returns `nil` for
|
|
271
281
|
anything invalid, an `in nil` branch is a complete validity guard:
|
|
@@ -305,11 +315,11 @@ CIK, WKN, and Valoren have no sub-fields: they match a bare `in SecID::CIK` but
|
|
|
305
315
|
|
|
306
316
|
Watch the `:type` key: on a `SecID::Match` it is the registry symbol (`:occ`), while on an `OCC` instance it is the
|
|
307
317
|
OSI option type (`'C'` or `'P'`), so `in SecID::OCC[type: :occ]` never matches. Other types have no `:type` component
|
|
308
|
-
at all
|
|
318
|
+
at all. `#to_h`'s envelope keys (`:type`, `:full_id`, `:normalized`, `:valid`) are not part of the pattern surface.
|
|
309
319
|
|
|
310
|
-
### Generating
|
|
320
|
+
### Generating test fixtures
|
|
311
321
|
|
|
312
|
-
Generate syntactically valid identifiers
|
|
322
|
+
Generate syntactically valid identifiers, with correct checksum where applicable, for use as test fixtures. Available per class and via the central dispatcher:
|
|
313
323
|
|
|
314
324
|
```ruby
|
|
315
325
|
SecID::ISIN.generate # => #<SecID::ISIN ...>
|
|
@@ -323,11 +333,47 @@ SecID.generate(:nope) # => raises ArgumentError: Unknown identifier type
|
|
|
323
333
|
SecID::LEI.generate(random: Random.new(42)) == SecID::LEI.generate(random: Random.new(42)) # => true
|
|
324
334
|
```
|
|
325
335
|
|
|
326
|
-
> **Generated identifiers are valid in format only
|
|
336
|
+
> **Generated identifiers are valid in format only. They are not real, registered securities.**
|
|
327
337
|
> Country codes, FIGI prefixes, OCC expiry dates, and CFI category/group/attribute choices are
|
|
328
338
|
> randomly selected (from the values each standard permits) and do not map to real-world
|
|
329
339
|
> instruments. Use them as test fixtures, not as references to actual securities.
|
|
330
340
|
|
|
341
|
+
### Repairing typos
|
|
342
|
+
|
|
343
|
+
Turn the checksum from a gatekeeper into a repair engine. For a checksum-**failing** identifier, `suggest` enumerates the plausible single-character human errors (visual/OCR homoglyph substitutions such as `O`↔`0`, `I`↔`1`, `5`↔`S`, `8`↔`B`, plus adjacent transpositions), keeps only the edits that re-validate, and returns them as confidence-ranked `SecID::Suggestion` candidates that report *what changed*. Available for all 9 checksum types (ISIN, CUSIP, SEDOL, FIGI, LEI, IBAN, CEI, DTI, UPI) per class and via the central dispatcher:
|
|
344
|
+
|
|
345
|
+
```ruby
|
|
346
|
+
# A letter O typed where a 0 belongs: 'US5949181O45' should be 'US5949181045'
|
|
347
|
+
top = SecID::ISIN.suggest('US5949181O45').first
|
|
348
|
+
top.to_s # => 'US5949181045' (the corrected identifier)
|
|
349
|
+
top.edit # => :substitution
|
|
350
|
+
top.position # => 9
|
|
351
|
+
top.from # => 'O'
|
|
352
|
+
top.to # => '0'
|
|
353
|
+
top.confidence # => :high
|
|
354
|
+
top.identifier # => #<SecID::ISIN ...> (parsed and valid; call .country_code, .to_h, etc.)
|
|
355
|
+
|
|
356
|
+
# Module-level: infers every format-compatible checksum type (like parse / detect)
|
|
357
|
+
SecID.suggest('US5949181O45') # => [#<SecID::Suggestion type=:isin ...>, ...]
|
|
358
|
+
SecID.suggest('US5949181O45', types: [:isin]) # => restrict to specific types
|
|
359
|
+
```
|
|
360
|
+
|
|
361
|
+
Each candidate carries the corrected `identifier` (a parsed, valid instance), the `edit` kind, its `position`, the `from`/`to` characters, and a `confidence` tier. Candidates are ranked by confidence: `:high` homoglyph substitutions first, then `:medium` adjacent transpositions, then the `:checksum` recompute (body assumed correct, wrong check character) last as a fallback hypothesis. **There is no `:low` tier.** Coincidental substitutions that merely satisfy the checksum are never generated, keeping the result small and high-precision.
|
|
362
|
+
|
|
363
|
+
> **`suggest` returns candidates, never authoritative corrections, and never mutates its input.**
|
|
364
|
+
> Every returned candidate fully re-validates (`valid?` is the oracle), so no checksum-invalid candidate escapes,
|
|
365
|
+
> but a valid candidate is not necessarily *the* correction, so the `confidence` tier is how *you* decide what to
|
|
366
|
+
> trust. This matters for financial identifiers.
|
|
367
|
+
|
|
368
|
+
Notes and limitations:
|
|
369
|
+
|
|
370
|
+
- **Never empty for structurally-valid input.** A parseable but checksum-failing identifier always yields at least the `:checksum` fallback. Only wrong-length or illegal-charset input (which fails the format gate), or an already-valid identifier (nothing to repair), returns `[]`.
|
|
371
|
+
- **Vowel-free reachability.** SEDOL, FIGI, DTI, and UPI exclude vowels from their charset, so an `O`-for-`0` or `I`-for-`1` typo is unparseable and therefore unrepairable (it fails the format gate before enumeration). The mistyped character must be *in* the type's charset to be reachable.
|
|
372
|
+
- **Single body error only.** Two or more wrong body characters, or a dropped/doubled character (insertion/deletion), are out of scope; such input returns only the `:checksum` fallback, never additional body candidates.
|
|
373
|
+
- **Non-checksum types are unsupported.** CIK, OCC, WKN, Valoren, CFI, FISN, and BIC have no checksum oracle, so they have no `suggest`; `SecID.suggest` silently skips them.
|
|
374
|
+
|
|
375
|
+
**Precision** (simulated over 1,000 seeded samples per checksum type; see [`benchmark/suggest_precision.rb`](benchmark/suggest_precision.rb)): every reachable, single-error homoglyph or transposition that yields a checksum-failing identifier is recovered: the correct identifier is **always** among the returned candidates (100%), and is the top-ranked body candidate ~89% of the time for homoglyph errors. In-charset homoglyph reachability averages ~96% (100% for the letter-permitting types, lower for the four vowel-free ones).
|
|
376
|
+
|
|
331
377
|
### ISIN
|
|
332
378
|
|
|
333
379
|
> [International Securities Identification Number](https://en.wikipedia.org/wiki/International_Securities_Identification_Number) - a 12-character alphanumeric code that uniquely identifies a security.
|
|
@@ -666,7 +712,7 @@ SecID::CFI.new('QQXXXX').decode # => nil (decode returns nil for an invalid
|
|
|
666
712
|
|
|
667
713
|
CFI is validated strictly against the ISO 10962:2021 code tables for all 14 categories: the category (position 1), the group (position 2), and every attribute (positions 3-6) must be a value the standard defines for that group. `X` means "not applicable" and is accepted in every position; `Strategies` (`K`) codes carry no attributes and require `XXXX`. An impermissible attribute letter raises `InvalidStructureError` (`:invalid_attribute`).
|
|
668
714
|
|
|
669
|
-
> **Migration from < 6.0:** the old category-wide equity predicates (`cfi.voting?`, `cfi.fully_paid?`, …) are removed. Use `cfi.decode` and its scoped fields instead
|
|
715
|
+
> **Migration from < 6.0:** the old category-wide equity predicates (`cfi.voting?`, `cfi.fully_paid?`, …) are removed. Use `cfi.decode` and its scoped fields instead. A predicate now lives on the field whose domain defines it: `cfi.voting?` → `cfi.decode.attributes.voting_right.voting?`. Two do not map name-for-name: `cfi.equity?` → `cfi.decode.category.equity?` (or `cfi.category == :equity`), and `cfi.no_restrictions?` → `cfi.decode.attributes.ownership_restrictions.free_of_restrictions?`. Several group letters and symbols also changed to match ISO 10962:2021 (e.g. non-listed options `H` are now classified by underlying, and `LS` → `:securities_lending`, `TI` → `:indices`).
|
|
670
716
|
|
|
671
717
|
```ruby
|
|
672
718
|
# Introspect valid codes
|
|
@@ -721,7 +767,7 @@ SecID::BIC.valid?('DEUTZZFF') # => false ('ZZ' is not a recognized country
|
|
|
721
767
|
SecID::BIC.countries # => ['AD', 'AE', 'AF', ...] (sorted, includes 'XK')
|
|
722
768
|
```
|
|
723
769
|
|
|
724
|
-
BIC validation confirms structure and a real country code only. It does **not** verify that the institution, location, or branch corresponds to a registered SWIFT participant
|
|
770
|
+
BIC validation confirms structure and a real country code only. It does **not** verify that the institution, location, or branch corresponds to a registered SWIFT participant. That requires the licensed SWIFT registry.
|
|
725
771
|
|
|
726
772
|
### DTI
|
|
727
773
|
|
|
@@ -745,7 +791,7 @@ dti.restore! # => #<SecID::DTI> (mutates instance)
|
|
|
745
791
|
dti.calculate_checksum # => 'S'
|
|
746
792
|
```
|
|
747
793
|
|
|
748
|
-
DTI accepts exactly 9 characters: an 8-character base (first character never `0`) plus 1 check character, both drawn from a 30-symbol alphabet
|
|
794
|
+
DTI accepts exactly 9 characters: an 8-character base (first character never `0`) plus 1 check character, both drawn from a 30-symbol alphabet: digits `0`-`9` and consonants (vowels and `Y` never appear). Unlike most checksum types in this gem, `checksum` and `calculate_checksum` return a `String`, not an `Integer` (as does UPI). The check character is computed fully offline via ISO 7064 hybrid MOD 31,30, with no registry lookup or paywalled ISO 24165-1 spec required.
|
|
749
795
|
|
|
750
796
|
> **Grandfathered code:** Bitcoin's registered code (`4H95J0R2X`) predates the algorithm and fails the MOD 31,30 computation (which yields `4H95J0R2T`). A frozen exception map honors the registry's assignment across `valid?`, `restore`, and `checksum` alike:
|
|
751
797
|
>
|
|
@@ -776,15 +822,15 @@ upi.restore! # => #<SecID::UPI> (mutates instance)
|
|
|
776
822
|
upi.calculate_checksum # => '2'
|
|
777
823
|
```
|
|
778
824
|
|
|
779
|
-
UPI accepts exactly 12 characters: a fixed `QZ` prefix, a 9-character body, and 1 check character, all drawn from the same 30-symbol alphabet as DTI
|
|
825
|
+
UPI accepts exactly 12 characters: a fixed `QZ` prefix, a 9-character body, and 1 check character, all drawn from the same 30-symbol alphabet as DTI: digits `0`-`9` and consonants (vowels and `Y` never appear). Like DTI, `checksum` and `calculate_checksum` return a `String`, not an `Integer`. The check character is computed fully offline via ISO 7064 hybrid MOD 31,30 over the 11 preceding characters, with no DSB registry lookup or paywalled ISO 4914 spec required.
|
|
780
826
|
|
|
781
|
-
> **Coexistence with ISIN:** a UPI shares the 12-character length bucket with ISIN. A UPI whose digit check character also satisfies ISIN's Luhn detects as both (`SecID.detect('QZXKR05S3DL1') # => [:isin, :upi]`), with ISIN ranked first; `SecID.parse(..., on_ambiguous: :raise)` surfaces the collision. UPI validation itself is fully offline and existence is **not** verified
|
|
827
|
+
> **Coexistence with ISIN:** a UPI shares the 12-character length bucket with ISIN. A UPI whose digit check character also satisfies ISIN's Luhn detects as both (`SecID.detect('QZXKR05S3DL1') # => [:isin, :upi]`), with ISIN ranked first; `SecID.parse(..., on_ambiguous: :raise)` surfaces the collision. UPI validation itself is fully offline and existence is **not** verified. That requires the licensed DSB registry.
|
|
782
828
|
|
|
783
|
-
## ActiveModel / Rails
|
|
829
|
+
## ActiveModel / Rails validator
|
|
784
830
|
|
|
785
|
-
SecID ships an opt-in [ActiveModel](https://api.rubyonrails.org/classes/ActiveModel/Validations.html) validator, registered as `sec_id`, for declarative validation of any supported identifier type. It adds **no runtime dependency
|
|
831
|
+
SecID ships an opt-in [ActiveModel](https://api.rubyonrails.org/classes/ActiveModel/Validations.html) validator, registered as `sec_id`, for declarative validation of any supported identifier type. It adds **no runtime dependency**: `require 'sec_id'` loads none of it, and ActiveModel is a development/test dependency only.
|
|
786
832
|
|
|
787
|
-
**In Rails it just works.** A Railtie loads the validator automatically after the framework boots, so `gem 'sec_id'` in your `Gemfile` is enough
|
|
833
|
+
**In Rails it just works.** A Railtie loads the validator automatically after the framework boots, so `gem 'sec_id'` in your `Gemfile` is enough, with no `require:` option and no initializer:
|
|
788
834
|
|
|
789
835
|
```ruby
|
|
790
836
|
class Security < ApplicationRecord
|
|
@@ -801,13 +847,13 @@ require 'sec_id/active_model'
|
|
|
801
847
|
### Validation modes
|
|
802
848
|
|
|
803
849
|
```ruby
|
|
804
|
-
# Single type
|
|
850
|
+
# Single type: the value must be a valid ISIN
|
|
805
851
|
validates :isin, sec_id: { type: :isin }
|
|
806
852
|
|
|
807
|
-
# Allowlist
|
|
853
|
+
# Allowlist: valid as at least one of the listed types
|
|
808
854
|
validates :ref, sec_id: { types: %i[isin cusip] }
|
|
809
855
|
|
|
810
|
-
# Type-agnostic
|
|
856
|
+
# Type-agnostic: valid as any supported type
|
|
811
857
|
validates :ref, sec_id: true
|
|
812
858
|
```
|
|
813
859
|
|
|
@@ -826,16 +872,16 @@ With `normalize: true` in allowlist or agnostic mode, a value valid as more than
|
|
|
826
872
|
|
|
827
873
|
### Error messages and `details:`
|
|
828
874
|
|
|
829
|
-
On failure the validator adds one error under the `:sec_id` key with a type-aware default ("is not a valid ISIN" for a single type, "is not a valid securities identifier" for an allowlist/agnostic). Override the message in either of two ways: pass the standard `message:` option (the simplest, per-validation override), or define the attribute-scoped i18n key `activemodel.errors.models.<model>.attributes.<attribute>.sec_id` in your locale files. (The generic `activemodel.errors.messages.sec_id` key is not consulted, because the built-in default is supplied as ActiveModel's `message:` fallback.) Pass `details: true` (with a single `type
|
|
875
|
+
On failure the validator adds one error under the `:sec_id` key with a type-aware default ("is not a valid ISIN" for a single type, "is not a valid securities identifier" for an allowlist/agnostic). Override the message in either of two ways: pass the standard `message:` option (the simplest, per-validation override), or define the attribute-scoped i18n key `activemodel.errors.models.<model>.attributes.<attribute>.sec_id` in your locale files. (The generic `activemodel.errors.messages.sec_id` key is not consulted, because the built-in default is supplied as ActiveModel's `message:` fallback.) Pass `details: true` (with a single `type:`; it is ignored for an allowlist/agnostic) to surface sec_id's specific reason instead of the generic text:
|
|
830
876
|
|
|
831
877
|
```ruby
|
|
832
878
|
validates :isin, sec_id: { type: :isin, details: true }
|
|
833
879
|
# a bad checksum reports e.g. "Checksum '4' is invalid, expected '5'"
|
|
834
880
|
```
|
|
835
881
|
|
|
836
|
-
Standard `EachValidator` options
|
|
882
|
+
Standard `EachValidator` options (`allow_nil`, `allow_blank`, `if`, `unless`, `on`) work as usual. Tested against Rails 7.2, 8.0, and 8.1.
|
|
837
883
|
|
|
838
|
-
## Lookup
|
|
884
|
+
## Lookup service integration
|
|
839
885
|
|
|
840
886
|
SecID validates identifiers but does not include HTTP clients. The [`docs/guides/`](docs/guides/) directory provides integration patterns for external lookup services using only stdlib (`net/http`, `json`):
|
|
841
887
|
|
|
@@ -848,10 +894,10 @@ SecID validates identifiers but does not include HTTP clients. The [`docs/guides
|
|
|
848
894
|
|
|
849
895
|
Each guide includes a complete adapter class and a [runnable example](examples/).
|
|
850
896
|
|
|
851
|
-
## Type
|
|
897
|
+
## Type signatures (RBS)
|
|
852
898
|
|
|
853
899
|
sec_id ships hand-written [RBS](https://github.com/ruby/rbs) signatures under `sig/`,
|
|
854
|
-
packaged in the gem
|
|
900
|
+
packaged in the gem, so if you use [Steep](https://github.com/soutaro/steep) or an
|
|
855
901
|
RBS-aware editor, sec_id's types resolve automatically on install, no `rbs collection`
|
|
856
902
|
entry required. The only standard-library signature referenced is `date`, declared in
|
|
857
903
|
`sig/manifest.yaml`.
|
|
@@ -877,10 +923,19 @@ committed signatures fall out of sync with the tables.
|
|
|
877
923
|
## Development
|
|
878
924
|
|
|
879
925
|
After checking out the repo, run `bin/setup` to install dependencies.
|
|
880
|
-
Then
|
|
881
|
-
|
|
926
|
+
Then run `bundle exec rake` to run RuboCop, validate the RBS signatures, and run the specs.
|
|
927
|
+
`bin/console` gives you an interactive prompt to experiment in.
|
|
928
|
+
|
|
929
|
+
To use your working copy from another project, point its Gemfile at the checkout:
|
|
930
|
+
`gem 'sec_id', path: '/path/to/sec_id'`.
|
|
931
|
+
|
|
932
|
+
Releases are published from CI by pushing a tag, so there is no local publish task.
|
|
933
|
+
|
|
934
|
+
## Support and status
|
|
935
|
+
|
|
936
|
+
Ask questions in [Discussions](https://github.com/svyatov/sec_id/discussions). Report bugs in the [issue tracker](https://github.com/svyatov/sec_id/issues). Both are public and searchable, so the next reader with your question finds the answer.
|
|
882
937
|
|
|
883
|
-
|
|
938
|
+
One person actively maintains SecID. They read bug reports and pull requests, and they track each identifier standard against its published specification. No response time is promised. See [Governance](CONTRIBUTING.md#governance) for who decides and what happens if they stop.
|
|
884
939
|
|
|
885
940
|
## Contributing
|
|
886
941
|
|
|
@@ -894,7 +949,21 @@ See [CHANGELOG.md](CHANGELOG.md) for a detailed history of changes, following [K
|
|
|
894
949
|
|
|
895
950
|
## Versioning
|
|
896
951
|
|
|
897
|
-
This project follows [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html)
|
|
952
|
+
This project follows [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html).
|
|
953
|
+
|
|
954
|
+
The public API that version promise covers is everything documented in this README and in the [API docs](https://rubydoc.info/gems/sec_id), except anything marked `@api private` in YARD. Concretely, it covers:
|
|
955
|
+
|
|
956
|
+
- the `SecID` module methods: `[]`, `identifiers`, `valid?`, `detect`, `parse`, `parse!`, `extract`, `scan`, `explain`, `generate`, and `suggest`
|
|
957
|
+
- every identifier class and its documented instance and class methods
|
|
958
|
+
- the `SecID::Errors`, `SecID::Suggestion`, `SecID::Scanner::Match`, and `SecID::CFI::Classification` value objects, and the exception hierarchy under `SecID::Error`
|
|
959
|
+
- the `sec_id` ActiveModel validator and its options
|
|
960
|
+
- the RBS signatures under `sig/`
|
|
961
|
+
|
|
962
|
+
Marked `@api private` and outside the promise: `SecID::Detector`, `SecID::Scanner` itself, `SecID::CFI::Tables`, `SecID::IBAN::CountryRules`, `SecID::BIC::CountryCodes`, `Base.detection_priority`, and the internals of the four concerns. These can change in any release.
|
|
963
|
+
|
|
964
|
+
Raising the minimum Ruby version is a breaking change and ships in a MAJOR release.
|
|
965
|
+
|
|
966
|
+
Removals follow a deprecation window; see [Deprecation policy](CONTRIBUTING.md#deprecation-policy).
|
|
898
967
|
|
|
899
968
|
## License
|
|
900
969
|
|
data/lib/sec_id/active_model.rb
CHANGED
|
@@ -5,11 +5,11 @@ require 'sec_id'
|
|
|
5
5
|
begin
|
|
6
6
|
require 'active_model'
|
|
7
7
|
rescue LoadError => e
|
|
8
|
-
# :
|
|
8
|
+
# simplecov:disable ActiveModel is always loadable in the test suite
|
|
9
9
|
raise LoadError, 'sec_id/active_model requires ActiveModel, which could not be loaded. Add ' \
|
|
10
10
|
'`gem "activemodel"` to your Gemfile (or use this inside a Rails app) before ' \
|
|
11
11
|
"requiring 'sec_id/active_model'. (#{e.message})"
|
|
12
|
-
# :
|
|
12
|
+
# simplecov:enable
|
|
13
13
|
end
|
|
14
14
|
|
|
15
15
|
# ActiveModel validator for securities identifiers, registered under the `sec_id` key.
|
|
@@ -115,9 +115,10 @@ class SecIdValidator < ActiveModel::EachValidator
|
|
|
115
115
|
# @return [String, nil] the message of the SecID::Error raised when normalizing `value` as `type`
|
|
116
116
|
def capture_reason(type, value)
|
|
117
117
|
SecID[type].normalize(value)
|
|
118
|
-
# :
|
|
118
|
+
# simplecov:disable unreachable — only called after normalize already raised in valid_value?,
|
|
119
|
+
# kept as a safe fallback
|
|
119
120
|
nil
|
|
120
|
-
# :
|
|
121
|
+
# simplecov:enable
|
|
121
122
|
rescue SecID::Error => e
|
|
122
123
|
e.message
|
|
123
124
|
end
|
data/lib/sec_id/cei.rb
CHANGED
|
@@ -56,6 +56,10 @@ module SecID
|
|
|
56
56
|
'*' => 36, '@' => 37, '#' => 38
|
|
57
57
|
}.freeze
|
|
58
58
|
|
|
59
|
+
# Extends the including identifier class with the concern's class methods and checksum reader.
|
|
60
|
+
#
|
|
61
|
+
# @param base [Class] the identifier class including this concern
|
|
62
|
+
# @return [void]
|
|
59
63
|
# @api private
|
|
60
64
|
def self.included(base)
|
|
61
65
|
base.attr_reader :checksum
|
|
@@ -28,6 +28,10 @@ module SecID
|
|
|
28
28
|
# Alphanumeric characters (digits then uppercase letters).
|
|
29
29
|
ALPHANUMERIC = (DIGITS + ALPHA).freeze
|
|
30
30
|
|
|
31
|
+
# Extends the including identifier class with the concern's class methods.
|
|
32
|
+
#
|
|
33
|
+
# @param base [Class] the identifier class including this concern
|
|
34
|
+
# @return [void]
|
|
31
35
|
# @api private
|
|
32
36
|
def self.included(base)
|
|
33
37
|
base.extend(ClassMethods)
|
|
@@ -8,6 +8,10 @@ module SecID
|
|
|
8
8
|
# Characters stripped during normalization (whitespace and hyphens); classes may override.
|
|
9
9
|
SEPARATORS = /[\s-]/
|
|
10
10
|
|
|
11
|
+
# Extends the including identifier class with the concern's class methods.
|
|
12
|
+
#
|
|
13
|
+
# @param base [Class] the identifier class including this concern
|
|
14
|
+
# @return [void]
|
|
11
15
|
# @api private
|
|
12
16
|
def self.included(base)
|
|
13
17
|
base.extend(ClassMethods)
|
|
@@ -0,0 +1,173 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module SecID
|
|
4
|
+
# Repair engine for checksum-failing identifiers: enumerates plausible single-character
|
|
5
|
+
# edits (homoglyph/OCR substitutions and adjacent transpositions), keeps only those that
|
|
6
|
+
# re-validate, and returns them as confidence-ranked {Suggestion} objects.
|
|
7
|
+
#
|
|
8
|
+
# Included by the 9 checksum types. The candidate space is the human-error net —
|
|
9
|
+
# {HOMOGLYPHS} plus adjacent transpositions — never a full coincidental net. `valid?`
|
|
10
|
+
# is the oracle, so no checksum-invalid candidate ever escapes (a valid candidate is not
|
|
11
|
+
# necessarily the intended correction); the {HOMOGLYPHS} table bounds recall.
|
|
12
|
+
#
|
|
13
|
+
# @see SecID::Suggestion
|
|
14
|
+
#
|
|
15
|
+
# @example
|
|
16
|
+
# SecID::ISIN.suggest('US5949181O45')
|
|
17
|
+
# #=> [#<SecID::Suggestion edit=:substitution ...>, #<SecID::Suggestion edit=:checksum ...>]
|
|
18
|
+
module Suggestable
|
|
19
|
+
# Homoglyph / OCR confusion table (KTD5 seed): each character maps to the characters
|
|
20
|
+
# it is commonly mistyped as or misread from. A bidirectional expansion of the seed
|
|
21
|
+
# pairs {0/O, 0/Q, 0/D, 1/I, 1/L, I/L, 2/Z, 5/S, 6/G, 8/B}. Membership bounds recall —
|
|
22
|
+
# widen it (see benchmark/suggest_precision.rb) to raise it.
|
|
23
|
+
HOMOGLYPHS = {
|
|
24
|
+
'0' => %w[O Q D], 'O' => %w[0], 'Q' => %w[0], 'D' => %w[0],
|
|
25
|
+
'1' => %w[I L], 'I' => %w[1 L], 'L' => %w[1 I],
|
|
26
|
+
'2' => %w[Z], 'Z' => %w[2], '5' => %w[S], 'S' => %w[5],
|
|
27
|
+
'6' => %w[G], 'G' => %w[6], '8' => %w[B], 'B' => %w[8]
|
|
28
|
+
}.freeze
|
|
29
|
+
|
|
30
|
+
# Rank weight per edit kind: body substitutions first, then adjacent transpositions,
|
|
31
|
+
# then the checksum-recompute fallback last (R6/R7).
|
|
32
|
+
RANK = { substitution: 0, transposition: 1, checksum: 2 }.freeze
|
|
33
|
+
|
|
34
|
+
# Extends the including identifier class with the concern's class methods.
|
|
35
|
+
#
|
|
36
|
+
# @param base [Class] the identifier class including this concern
|
|
37
|
+
# @return [void]
|
|
38
|
+
# @api private
|
|
39
|
+
def self.included(base)
|
|
40
|
+
base.extend(ClassMethods)
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
# Class methods added when Suggestable is included.
|
|
44
|
+
module ClassMethods
|
|
45
|
+
# Returns confidence-ranked repair candidates for a checksum-failing identifier.
|
|
46
|
+
#
|
|
47
|
+
# @param str [String] the identifier string to repair
|
|
48
|
+
# @return [Array<Suggestion>] ranked candidates; empty when `str` is already valid
|
|
49
|
+
# or fails the type's format (wrong length or charset)
|
|
50
|
+
def suggest(str)
|
|
51
|
+
new(str).suggest
|
|
52
|
+
end
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
# Enumerates plausible edits and returns the re-validating ones, confidence-ranked.
|
|
56
|
+
#
|
|
57
|
+
# @return [Array<Suggestion>] ranked candidates; empty when already valid or unparseable
|
|
58
|
+
def suggest
|
|
59
|
+
return [] if valid?
|
|
60
|
+
return [] unless valid_format?
|
|
61
|
+
|
|
62
|
+
surviving = candidate_strings.uniq.map { |candidate| self.class.new(candidate) }.select(&:valid?)
|
|
63
|
+
surviving.filter_map { |candidate| classify(candidate) }.sort_by { |suggestion| RANK.fetch(suggestion.edit) }
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
private
|
|
67
|
+
|
|
68
|
+
# Homoglyph substitutions + adjacent transpositions + the checksum recompute (R4, R6).
|
|
69
|
+
#
|
|
70
|
+
# @return [Array<String>] candidate identifier strings (unfiltered, may include duplicates)
|
|
71
|
+
def candidate_strings
|
|
72
|
+
homoglyph_candidates + transposition_candidates + [restore]
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
# @return [Array<String>]
|
|
76
|
+
def homoglyph_candidates
|
|
77
|
+
full_id.each_char.with_index.flat_map do |char, index|
|
|
78
|
+
Array(HOMOGLYPHS[char]).map { |replacement| replace_at(full_id, index, replacement) }
|
|
79
|
+
end
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
# @return [Array<String>]
|
|
83
|
+
def transposition_candidates
|
|
84
|
+
(0...(full_id.length - 1)).filter_map do |index|
|
|
85
|
+
swap_at(full_id, index) unless full_id[index] == full_id[index + 1]
|
|
86
|
+
end
|
|
87
|
+
end
|
|
88
|
+
|
|
89
|
+
# @param str [String]
|
|
90
|
+
# @param index [Integer]
|
|
91
|
+
# @param char [String] the replacement character
|
|
92
|
+
# @return [String]
|
|
93
|
+
def replace_at(str, index, char)
|
|
94
|
+
str.dup.tap { |copy| copy[index] = char }
|
|
95
|
+
end
|
|
96
|
+
|
|
97
|
+
# @param str [String]
|
|
98
|
+
# @param index [Integer] left index of the adjacent pair to swap
|
|
99
|
+
# @return [String]
|
|
100
|
+
def swap_at(str, index)
|
|
101
|
+
str.dup.tap { |copy| copy[index], copy[index + 1] = copy[index + 1], copy[index] }
|
|
102
|
+
end
|
|
103
|
+
|
|
104
|
+
# Tags a surviving candidate by comparing bodies then diffing the strings (KTD3):
|
|
105
|
+
# equal bodies mean only the check characters changed.
|
|
106
|
+
#
|
|
107
|
+
# @param candidate [SecID::Base] a re-validated candidate instance
|
|
108
|
+
# @return [Suggestion, nil] nil when the diff is neither a single substitution nor an adjacent swap
|
|
109
|
+
def classify(candidate)
|
|
110
|
+
return checksum_suggestion(candidate) if candidate.identifier == identifier
|
|
111
|
+
|
|
112
|
+
changed = changed_indices(candidate)
|
|
113
|
+
case changed.size
|
|
114
|
+
when 1 then substitution_suggestion(candidate, changed.first.to_i)
|
|
115
|
+
when 2 then transposition_suggestion(candidate, changed)
|
|
116
|
+
end
|
|
117
|
+
end
|
|
118
|
+
|
|
119
|
+
# @param candidate [SecID::Base]
|
|
120
|
+
# @return [Array<Integer>] indices where the candidate's string differs from the input's
|
|
121
|
+
def changed_indices(candidate)
|
|
122
|
+
other = candidate.full_id
|
|
123
|
+
(0...full_id.length).reject { |index| full_id[index] == other[index] }
|
|
124
|
+
end
|
|
125
|
+
|
|
126
|
+
# @param candidate [SecID::Base]
|
|
127
|
+
# @param index [Integer]
|
|
128
|
+
# @return [Suggestion]
|
|
129
|
+
def substitution_suggestion(candidate, index)
|
|
130
|
+
Suggestion.new(
|
|
131
|
+
type: self.class.type_key, identifier: candidate, edit: :substitution,
|
|
132
|
+
position: index, from: full_id[index].to_s, to: candidate.full_id[index].to_s, confidence: :high
|
|
133
|
+
)
|
|
134
|
+
end
|
|
135
|
+
|
|
136
|
+
# @param candidate [SecID::Base]
|
|
137
|
+
# @param indices [Array<Integer>] the two differing indices
|
|
138
|
+
# @return [Suggestion, nil] nil unless the two indices are an adjacent, mutual swap
|
|
139
|
+
def transposition_suggestion(candidate, indices)
|
|
140
|
+
left = indices.min.to_i
|
|
141
|
+
right = indices.max.to_i
|
|
142
|
+
return unless right == left + 1 && swap?(candidate, left, right)
|
|
143
|
+
|
|
144
|
+
Suggestion.new(
|
|
145
|
+
type: self.class.type_key, identifier: candidate, edit: :transposition,
|
|
146
|
+
position: left, from: full_id[left, 2].to_s, to: candidate.full_id[left, 2].to_s, confidence: :medium
|
|
147
|
+
)
|
|
148
|
+
end
|
|
149
|
+
|
|
150
|
+
# @param candidate [SecID::Base]
|
|
151
|
+
# @param left [Integer]
|
|
152
|
+
# @param right [Integer]
|
|
153
|
+
# @return [Boolean] true when the two positions are a mutual swap of the input's characters
|
|
154
|
+
def swap?(candidate, left, right)
|
|
155
|
+
full_id[left] == candidate.full_id[right] && full_id[right] == candidate.full_id[left]
|
|
156
|
+
end
|
|
157
|
+
|
|
158
|
+
# @param candidate [SecID::Base]
|
|
159
|
+
# @return [Suggestion]
|
|
160
|
+
def checksum_suggestion(candidate)
|
|
161
|
+
Suggestion.new(
|
|
162
|
+
type: self.class.type_key, identifier: candidate, edit: :checksum, position: nil,
|
|
163
|
+
from: rendered_checksum(checksum), to: rendered_checksum(candidate.checksum), confidence: nil
|
|
164
|
+
)
|
|
165
|
+
end
|
|
166
|
+
|
|
167
|
+
# @param value [Integer, String, nil] a checksum value
|
|
168
|
+
# @return [String] the check-character string, zero-padded to the checksum width ('' when absent)
|
|
169
|
+
def rendered_checksum(value)
|
|
170
|
+
value.nil? ? '' : value.to_s.rjust(checksum_width, '0')
|
|
171
|
+
end
|
|
172
|
+
end
|
|
173
|
+
end
|
|
@@ -18,6 +18,10 @@ module SecID
|
|
|
18
18
|
invalid_country: InvalidStructureError
|
|
19
19
|
}.freeze
|
|
20
20
|
|
|
21
|
+
# Extends the including identifier class with the concern's class methods.
|
|
22
|
+
#
|
|
23
|
+
# @param base [Class] the identifier class including this concern
|
|
24
|
+
# @return [void]
|
|
21
25
|
# @api private
|
|
22
26
|
def self.included(base)
|
|
23
27
|
base.extend(ClassMethods)
|
data/lib/sec_id/cusip.rb
CHANGED
data/lib/sec_id/dti.rb
CHANGED
data/lib/sec_id/figi.rb
CHANGED
data/lib/sec_id/iban.rb
CHANGED
data/lib/sec_id/isin.rb
CHANGED
data/lib/sec_id/lei.rb
CHANGED
data/lib/sec_id/sedol.rb
CHANGED
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module SecID
|
|
4
|
+
# Immutable value object representing one repair candidate returned by `suggest`.
|
|
5
|
+
#
|
|
6
|
+
# Carries the corrected, re-validated identifier plus the what-changed diff (edit
|
|
7
|
+
# kind, position, from/to characters) and a confidence tier — the honest surface
|
|
8
|
+
# that keeps a candidate from reading as an authoritative correction.
|
|
9
|
+
#
|
|
10
|
+
# Coordinate system by edit kind:
|
|
11
|
+
# - `:substitution` — `position` is the 0-based index of the changed character;
|
|
12
|
+
# `from`/`to` are the single characters before/after.
|
|
13
|
+
# - `:transposition` — `position` is the left index of the swapped pair;
|
|
14
|
+
# `from`/`to` are the two-character adjacent substring before/after the swap.
|
|
15
|
+
# - `:checksum` — `position` is `nil`; `from`/`to` are the old/new check-character
|
|
16
|
+
# string. `confidence` is `nil` (it is the fallback hypothesis, ranked last).
|
|
17
|
+
#
|
|
18
|
+
# @!attribute [r] type
|
|
19
|
+
# @return [Symbol] the identifier type key (e.g. :isin)
|
|
20
|
+
# @!attribute [r] identifier
|
|
21
|
+
# @return [SecID::Base] the valid, parsed corrected identifier
|
|
22
|
+
# @!attribute [r] edit
|
|
23
|
+
# @return [Symbol] :substitution, :transposition, or :checksum
|
|
24
|
+
# @!attribute [r] position
|
|
25
|
+
# @return [Integer, nil] the edit position (nil for :checksum)
|
|
26
|
+
# @!attribute [r] from
|
|
27
|
+
# @return [String] the original character(s)
|
|
28
|
+
# @!attribute [r] to
|
|
29
|
+
# @return [String] the replacement character(s)
|
|
30
|
+
# @!attribute [r] confidence
|
|
31
|
+
# @return [Symbol, nil] :high (homoglyph), :medium (transposition), or nil (:checksum)
|
|
32
|
+
#
|
|
33
|
+
# @example
|
|
34
|
+
# suggestion = SecID::ISIN.suggest('US5949181O45').first
|
|
35
|
+
# suggestion.to_s #=> 'US5949181045'
|
|
36
|
+
# suggestion.edit #=> :substitution
|
|
37
|
+
# suggestion.confidence #=> :high
|
|
38
|
+
Suggestion = Data.define(:type, :identifier, :edit, :position, :from, :to, :confidence)
|
|
39
|
+
|
|
40
|
+
# Reopened to add the domain enumerations and serialization helpers per the repo
|
|
41
|
+
# convention (see {Base#as_json}).
|
|
42
|
+
class Suggestion
|
|
43
|
+
# The edit kinds `edit` can take, in rank order (see {Suggestable::RANK}).
|
|
44
|
+
EDIT_KINDS = %i[substitution transposition checksum].freeze
|
|
45
|
+
|
|
46
|
+
# The non-nil confidence tiers `confidence` can take; `:checksum` candidates carry `nil`.
|
|
47
|
+
CONFIDENCE_LEVELS = %i[high medium].freeze
|
|
48
|
+
|
|
49
|
+
# Returns a JSON-compatible hash representation of the suggestion.
|
|
50
|
+
#
|
|
51
|
+
# @return [Hash] the symbol-keyed field hash (JSON-compatible)
|
|
52
|
+
def as_json(*)
|
|
53
|
+
to_h
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
# Returns the corrected identifier as a string.
|
|
57
|
+
#
|
|
58
|
+
# @return [String] the corrected identifier string
|
|
59
|
+
def to_s
|
|
60
|
+
identifier.to_s
|
|
61
|
+
end
|
|
62
|
+
end
|
|
63
|
+
end
|
data/lib/sec_id/upi.rb
CHANGED
data/lib/sec_id/version.rb
CHANGED
data/lib/sec_id.rb
CHANGED
|
@@ -143,8 +143,42 @@ module SecID
|
|
|
143
143
|
self[key].generate(random: random)
|
|
144
144
|
end
|
|
145
145
|
|
|
146
|
+
# Returns confidence-ranked repair candidates for a checksum-failing identifier,
|
|
147
|
+
# inferred across all format-compatible checksum types (or the given `types:`).
|
|
148
|
+
#
|
|
149
|
+
# @note Repair is only defined for checksum types; non-checksum types have no
|
|
150
|
+
# oracle and are silently skipped.
|
|
151
|
+
#
|
|
152
|
+
# @param str [String, nil] the identifier string to repair
|
|
153
|
+
# @param types [Array<Symbol>, nil] restrict to specific types (non-checksum types are ignored)
|
|
154
|
+
# @return [Array<Suggestion>] cross-type ranked candidates, each carrying its `type`;
|
|
155
|
+
# empty when no format-compatible checksum type matches
|
|
156
|
+
# @raise [ArgumentError] if any key in types is unknown
|
|
157
|
+
def suggest(str, types: nil)
|
|
158
|
+
input = str.to_s.strip.upcase
|
|
159
|
+
suggestable_types(input, types)
|
|
160
|
+
.flat_map { |klass| klass.suggest(input) }
|
|
161
|
+
.sort_by do |suggestion|
|
|
162
|
+
[Suggestable::RANK.fetch(suggestion.edit), self[suggestion.type].detection_priority.last]
|
|
163
|
+
end
|
|
164
|
+
end
|
|
165
|
+
|
|
146
166
|
private
|
|
147
167
|
|
|
168
|
+
# Resolves the checksum types eligible to repair `input` (KTD4): the given `types:`
|
|
169
|
+
# narrowed to checksum types, or every checksum type whose length and charset admit it.
|
|
170
|
+
#
|
|
171
|
+
# @param input [String] the normalized (stripped, upcased) input
|
|
172
|
+
# @param types [Array<Symbol>, nil]
|
|
173
|
+
# @return [Array<Class>]
|
|
174
|
+
def suggestable_types(input, types)
|
|
175
|
+
return types.map { |key| self[key] }.select(&:has_checksum?) if types
|
|
176
|
+
|
|
177
|
+
identifier_list.select do |klass|
|
|
178
|
+
klass.has_checksum? && klass.length_values.include?(input.length) && input.match?(klass::VALID_CHARS_REGEX)
|
|
179
|
+
end
|
|
180
|
+
end
|
|
181
|
+
|
|
148
182
|
# @return [void]
|
|
149
183
|
def register_identifier(klass)
|
|
150
184
|
identifier_map[klass.type_key] = klass
|
|
@@ -206,7 +240,9 @@ require 'sec_id/concerns/normalizable'
|
|
|
206
240
|
require 'sec_id/concerns/validatable'
|
|
207
241
|
require 'sec_id/concerns/checkable'
|
|
208
242
|
require 'sec_id/concerns/generatable'
|
|
243
|
+
require 'sec_id/concerns/suggestable'
|
|
209
244
|
require 'sec_id/base'
|
|
245
|
+
require 'sec_id/suggestion'
|
|
210
246
|
require 'sec_id/detector'
|
|
211
247
|
require 'sec_id/scanner'
|
|
212
248
|
require 'sec_id/isin'
|
data/sec_id.gemspec
CHANGED
|
@@ -10,10 +10,11 @@ Gem::Specification.new do |spec|
|
|
|
10
10
|
spec.authors = ['Leonid Svyatov']
|
|
11
11
|
spec.email = ['leonid@svyatov.ru']
|
|
12
12
|
|
|
13
|
-
spec.summary = 'A Ruby toolkit for securities identifiers
|
|
14
|
-
'generate, and
|
|
15
|
-
spec.description = 'Validate, normalize, parse, convert, generate, and
|
|
16
|
-
'Auto-detect identifier type from any string. Calculate and restore checksums.
|
|
13
|
+
spec.summary = 'A Ruby toolkit for securities identifiers: validate, parse, normalize, detect, convert, ' \
|
|
14
|
+
'generate, classify, and repair.'
|
|
15
|
+
spec.description = 'Validate, normalize, parse, convert, generate, classify, and repair securities identifiers. ' \
|
|
16
|
+
'Auto-detect identifier type from any string. Calculate and restore checksums. Suggest ' \
|
|
17
|
+
'confidence-ranked corrections for fat-fingered or OCR-mangled identifiers. Generate ' \
|
|
17
18
|
'format-valid identifiers as test fixtures. Decode CFI codes to full ISO 10962:2021 ' \
|
|
18
19
|
'classifications. Includes an opt-in ActiveModel/Rails validator that adds no runtime ' \
|
|
19
20
|
'dependency. Supports ISIN, CUSIP, CEI, SEDOL, FIGI, LEI, IBAN, CIK, OCC, WKN, Valoren, ' \
|
data/sig/sec_id/base.rbs
CHANGED
|
@@ -43,6 +43,9 @@ module SecID
|
|
|
43
43
|
def self.length_specificity: () -> (Integer | Float | nil)
|
|
44
44
|
def self.example: () -> String
|
|
45
45
|
def self.has_checksum?: () -> bool
|
|
46
|
+
# Only Suggestable (checksum) types implement `.suggest`; declared here so the
|
|
47
|
+
# module-level `SecID.suggest` can call it on any registered class object.
|
|
48
|
+
def self.suggest: (SecID::input str) -> Array[Suggestion]
|
|
46
49
|
|
|
47
50
|
# @deprecated v7 bridge alias for `has_checksum?`; removed in v8.
|
|
48
51
|
def self.has_check_digit?: () -> bool
|
|
@@ -58,12 +61,15 @@ module SecID
|
|
|
58
61
|
def as_json: (*untyped) -> Hash[Symbol, untyped]
|
|
59
62
|
def deconstruct_keys: (::Array[Symbol]? _keys) -> Hash[Symbol, untyped]
|
|
60
63
|
|
|
61
|
-
# Checksum surface. Only the types that `include Checkable`
|
|
62
|
-
# (they raise/NoMethodError otherwise);
|
|
63
|
-
#
|
|
64
|
+
# Checksum / repair surface. Only the types that `include Checkable` /
|
|
65
|
+
# `include Suggestable` implement these (they raise/NoMethodError otherwise);
|
|
66
|
+
# declared on Base so `Generatable#generate`, `Checkable::ClassMethods`, and the
|
|
67
|
+
# Base-self-typed `Suggestable` body type-check without narrowing on `has_checksum?`.
|
|
64
68
|
def restore: () -> String
|
|
65
69
|
def restore!: () -> self
|
|
66
70
|
def calculate_checksum: () -> (Integer | String)
|
|
71
|
+
attr_reader checksum: (Integer | String)?
|
|
72
|
+
def suggest: () -> Array[Suggestion]
|
|
67
73
|
|
|
68
74
|
def comparison_id: () -> String
|
|
69
75
|
|
|
@@ -6,7 +6,8 @@ module SecID
|
|
|
6
6
|
|
|
7
7
|
def self.included: (Module base) -> void
|
|
8
8
|
|
|
9
|
-
attr_reader checksum
|
|
9
|
+
# `attr_reader checksum` is declared on Base (the repair surface) to keep the
|
|
10
|
+
# Base-self-typed Suggestable body type-checking; not redeclared here.
|
|
10
11
|
|
|
11
12
|
# Class methods added when Checkable is included.
|
|
12
13
|
module ClassMethods : SecID::_IdentifierClass
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
module SecID
|
|
2
|
+
# Repair engine for checksum-failing identifiers: enumerates plausible edits and
|
|
3
|
+
# returns the re-validating ones as confidence-ranked Suggestion objects.
|
|
4
|
+
module Suggestable : SecID::Base
|
|
5
|
+
HOMOGLYPHS: Hash[String, Array[String]]
|
|
6
|
+
RANK: Hash[Symbol, Integer]
|
|
7
|
+
|
|
8
|
+
def self.included: (Module base) -> void
|
|
9
|
+
|
|
10
|
+
# Class methods added when Suggestable is included.
|
|
11
|
+
module ClassMethods : SecID::_IdentifierClass
|
|
12
|
+
def suggest: (SecID::input str) -> Array[Suggestion]
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
def suggest: () -> Array[Suggestion]
|
|
16
|
+
|
|
17
|
+
private
|
|
18
|
+
|
|
19
|
+
def candidate_strings: () -> Array[String]
|
|
20
|
+
def homoglyph_candidates: () -> Array[String]
|
|
21
|
+
def transposition_candidates: () -> Array[String]
|
|
22
|
+
def replace_at: (String str, Integer index, String char) -> String
|
|
23
|
+
def swap_at: (String str, Integer index) -> String
|
|
24
|
+
def classify: (Base candidate) -> Suggestion?
|
|
25
|
+
def changed_indices: (Base candidate) -> Array[Integer]
|
|
26
|
+
def substitution_suggestion: (Base candidate, Integer index) -> Suggestion
|
|
27
|
+
def transposition_suggestion: (Base candidate, Array[Integer] indices) -> Suggestion?
|
|
28
|
+
def swap?: (Base candidate, Integer left, Integer right) -> bool
|
|
29
|
+
def checksum_suggestion: (Base candidate) -> Suggestion
|
|
30
|
+
def rendered_checksum: ((Integer | String)? value) -> String
|
|
31
|
+
end
|
|
32
|
+
end
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
module SecID
|
|
2
|
+
# Immutable value object representing one repair candidate returned by `suggest`.
|
|
3
|
+
class Suggestion < Data
|
|
4
|
+
EDIT_KINDS: Array[Symbol]
|
|
5
|
+
CONFIDENCE_LEVELS: Array[Symbol]
|
|
6
|
+
|
|
7
|
+
attr_reader type: Symbol
|
|
8
|
+
attr_reader identifier: Base
|
|
9
|
+
attr_reader edit: Symbol
|
|
10
|
+
attr_reader position: Integer?
|
|
11
|
+
attr_reader from: String
|
|
12
|
+
attr_reader to: String
|
|
13
|
+
attr_reader confidence: Symbol?
|
|
14
|
+
|
|
15
|
+
def self.new: (type: Symbol, identifier: Base, edit: Symbol, position: Integer?, from: String, to: String, confidence: Symbol?) -> instance
|
|
16
|
+
| (Symbol type, Base identifier, Symbol edit, Integer? position, String from, String to, Symbol? confidence) -> instance
|
|
17
|
+
def self.[]: (type: Symbol, identifier: Base, edit: Symbol, position: Integer?, from: String, to: String, confidence: Symbol?) -> instance
|
|
18
|
+
| (Symbol type, Base identifier, Symbol edit, Integer? position, String from, String to, Symbol? confidence) -> instance
|
|
19
|
+
|
|
20
|
+
def as_json: (*untyped) -> Hash[Symbol, untyped]
|
|
21
|
+
def to_s: () -> String
|
|
22
|
+
end
|
|
23
|
+
end
|
data/sig/sec_id.rbs
CHANGED
|
@@ -38,9 +38,11 @@ module SecID
|
|
|
38
38
|
# against the `:first`/`:raise` arm's `Base?` and fail. Widening here keeps `result` untyped.
|
|
39
39
|
def self.parse!: (SecID::input str, ?types: Array[Symbol]?, ?on_ambiguous: Symbol) -> (Base | Array[Base])
|
|
40
40
|
def self.generate: (Symbol key, ?random: Random) -> Base
|
|
41
|
+
def self.suggest: (SecID::input str, ?types: Array[Symbol]?) -> Array[Suggestion]
|
|
41
42
|
|
|
42
43
|
# Singleton-method privacy must be declared inline (`private def self.…`); a bare
|
|
43
44
|
# `private` section only affects instance methods, leaving these public.
|
|
45
|
+
private def self.suggestable_types: (String input, Array[Symbol]? types) -> Array[singleton(Base)]
|
|
44
46
|
private def self.register_identifier: (singleton(Base) klass) -> void
|
|
45
47
|
private def self.parse_any: (SecID::input str) -> Base?
|
|
46
48
|
private def self.parse_from: (SecID::input str, Array[Symbol] types) -> Base?
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: sec_id
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 7.
|
|
4
|
+
version: 7.1.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Leonid Svyatov
|
|
@@ -9,9 +9,10 @@ bindir: bin
|
|
|
9
9
|
cert_chain: []
|
|
10
10
|
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
11
|
dependencies: []
|
|
12
|
-
description: Validate, normalize, parse, convert, generate, and
|
|
12
|
+
description: Validate, normalize, parse, convert, generate, classify, and repair securities
|
|
13
13
|
identifiers. Auto-detect identifier type from any string. Calculate and restore
|
|
14
|
-
checksums.
|
|
14
|
+
checksums. Suggest confidence-ranked corrections for fat-fingered or OCR-mangled
|
|
15
|
+
identifiers. Generate format-valid identifiers as test fixtures. Decode CFI codes
|
|
15
16
|
to full ISO 10962:2021 classifications. Includes an opt-in ActiveModel/Rails validator
|
|
16
17
|
that adds no runtime dependency. Supports ISIN, CUSIP, CEI, SEDOL, FIGI, LEI, IBAN,
|
|
17
18
|
CIK, OCC, WKN, Valoren, CFI, FISN, BIC, DTI, and UPI.
|
|
@@ -41,6 +42,7 @@ files:
|
|
|
41
42
|
- lib/sec_id/concerns/checkable.rb
|
|
42
43
|
- lib/sec_id/concerns/generatable.rb
|
|
43
44
|
- lib/sec_id/concerns/normalizable.rb
|
|
45
|
+
- lib/sec_id/concerns/suggestable.rb
|
|
44
46
|
- lib/sec_id/concerns/validatable.rb
|
|
45
47
|
- lib/sec_id/cusip.rb
|
|
46
48
|
- lib/sec_id/deep_freeze.rb
|
|
@@ -58,6 +60,7 @@ files:
|
|
|
58
60
|
- lib/sec_id/railtie.rb
|
|
59
61
|
- lib/sec_id/scanner.rb
|
|
60
62
|
- lib/sec_id/sedol.rb
|
|
63
|
+
- lib/sec_id/suggestion.rb
|
|
61
64
|
- lib/sec_id/upi.rb
|
|
62
65
|
- lib/sec_id/valoren.rb
|
|
63
66
|
- lib/sec_id/version.rb
|
|
@@ -78,6 +81,7 @@ files:
|
|
|
78
81
|
- sig/sec_id/concerns/checkable.rbs
|
|
79
82
|
- sig/sec_id/concerns/generatable.rbs
|
|
80
83
|
- sig/sec_id/concerns/normalizable.rbs
|
|
84
|
+
- sig/sec_id/concerns/suggestable.rbs
|
|
81
85
|
- sig/sec_id/concerns/validatable.rbs
|
|
82
86
|
- sig/sec_id/cusip.rbs
|
|
83
87
|
- sig/sec_id/deep_freeze.rbs
|
|
@@ -94,6 +98,7 @@ files:
|
|
|
94
98
|
- sig/sec_id/occ.rbs
|
|
95
99
|
- sig/sec_id/scanner.rbs
|
|
96
100
|
- sig/sec_id/sedol.rbs
|
|
101
|
+
- sig/sec_id/suggestion.rbs
|
|
97
102
|
- sig/sec_id/upi.rbs
|
|
98
103
|
- sig/sec_id/valoren.rbs
|
|
99
104
|
- sig/sec_id/version.rbs
|
|
@@ -121,8 +126,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
121
126
|
- !ruby/object:Gem::Version
|
|
122
127
|
version: '0'
|
|
123
128
|
requirements: []
|
|
124
|
-
rubygems_version: 4.0.
|
|
129
|
+
rubygems_version: 4.0.16
|
|
125
130
|
specification_version: 4
|
|
126
|
-
summary: A Ruby toolkit for securities identifiers
|
|
127
|
-
convert, generate, and
|
|
131
|
+
summary: 'A Ruby toolkit for securities identifiers: validate, parse, normalize, detect,
|
|
132
|
+
convert, generate, classify, and repair.'
|
|
128
133
|
test_files: []
|