active_sanction 1.0.1 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: b6c9a31bd72dfdc43d5614499a19aab837b55996b9644ae145b89da18b93bea6
4
- data.tar.gz: 40cbea7b7fcf458307171ca953fb61864dce06da7733ece2ac17e4d27a43215d
3
+ metadata.gz: ffb2d77bf3207e3ef97a16269dcf91737e20d48871cc2aec9e747d4185793e52
4
+ data.tar.gz: eeca2cf2bb69972d83e15e982c2daea80969a636e4c88e86cbd29b670420098c
5
5
  SHA512:
6
- metadata.gz: 892209a20496c2eea1185776a83725cb89ae6c7b40ba2385becf7870b78d8e1a6cf88734e29afec40153418dfa73ac275a3270b0bb16a80e56c3de6605e58652
7
- data.tar.gz: e3330620e4c4f6c31c6a7e0c58b53bda9972628e9d6965798d56f666f7301b83ccc2edb42931d15b18b161c9480df96b5147edf0e94b77461e90e0958944ed46
6
+ metadata.gz: 748d59d2d06bb502eefd684a71c1abc74beb6b03d677eaf9a85c4563945c03fb7c9d69474988319d0ef1f03d4a68c328efd7d63ad14a62d33fa74ef4989e361b
7
+ data.tar.gz: dc9c4247f9e3ce24772a8045c4e7e0ea992ae7fbeb17c8f2f123f8a9ccf39fd741d7617a98cd67eef5968a16811c2cfc1b42a0c433df3da94d3b8e4f3f624f40
data/CHANGELOG.md CHANGED
@@ -17,6 +17,58 @@ screening decision would come out as today.
17
17
 
18
18
  Nothing yet.
19
19
 
20
+ ## [1.1.0] - 2026-09-14
21
+
22
+ **A minor, because it adds.** Nothing that existed changed: no behaviour under `lib/`
23
+ moved, `MATCHER_VERSION` stays at `1`, and a name scores today exactly what it scored
24
+ under 1.0.0. Instrumentation is additive and off unless a host asks for it.
25
+
26
+ ### Added
27
+
28
+ - **Instrumentation: six structured events, so a host can measure this library without
29
+ monkeypatching it** (#59). `ActiveSanction.configure { |c| c.instrumenter = ... }` takes
30
+ anything answering `#call(event)` and is handed a finished
31
+ `ActiveSanction::Instrumentation::Event` for each of `:fetch`, `:parse`, `:store`,
32
+ `:"index.build"`, `:screen` and `:sync` — every one carrying a duration and the ids
33
+ needed to correlate it, with no anonymous timings. The event names and every payload key
34
+ are public API, enumerated in [`docs/api_stability.md`](docs/api_stability.md) and on the
35
+ site's [instrumentation reference](https://babystep.tech/active_sanction/reference/instrumentation/),
36
+ and covered by the deprecation path from here.
37
+
38
+ **It is a measurement of the work and never part of it.** A subscriber that raises has its
39
+ exception caught, reported once through the configured logger, and dropped; the sync it
40
+ was watching finishes and returns the report it was going to return. The converse holds
41
+ too: a stage that raises emits its event with `error:` set and then the exception
42
+ continues exactly as if nothing were listening.
43
+
44
+ **Nothing is listening by default, and that costs nothing.** `nil` is a branch taken
45
+ before anything is allocated rather than a no-op object that gets called, so an
46
+ uninstrumented screening call builds no event and allocates no payload — which is the
47
+ only way a per-query event could be affordable at all. Measured rather than asserted:
48
+ building a matcher over 47,051 names allocates 262 more objects than before, out of
49
+ 5.79 million, and `rake benchmark:latency` reports the same p50 either side of the change
50
+ (12.1–12.8 ms against a run-to-run spread that was already that wide).
51
+
52
+ Rails hosts get `ActiveSanction::Instrumentation::Notifications`, which republishes every
53
+ event into `ActiveSupport::Notifications` under `<event>.active_sanction`. It is an
54
+ adapter and not a dependency: **nothing in this gem requires ActiveSupport**, and building
55
+ one in a process that has not loaded it raises `ConfigurationError` rather than quietly
56
+ instrumenting nothing.
57
+
58
+ The issue asked for `ActiveSanction.instrumenter = ...`, and this is a configuration
59
+ setting instead. #55 ended process-global configuration deliberately, and a module-level
60
+ writer would have rebuilt the default client — dropping the matcher it had indexed every
61
+ stored list into — as a side effect of naming a subscriber. A `Matcher` takes its
62
+ instrumenter at build and freezes it with its weights, so a subscriber swapped halfway
63
+ through a batch cannot make half of it instrumented; that is the rule every other setting
64
+ on the query path already follows.
65
+
66
+ - **`Sources::Base#warnings`**, defaulting to none. Every shipped adapter already exposed
67
+ it and the adapter rules already required it of a new one, but the base class never said
68
+ so — and the `:parse` event counts warnings for every source, which a count that is
69
+ sometimes a `NoMethodError` cannot do. An optional hook with a default implementation, so
70
+ no adapter outside this repository has to change.
71
+
20
72
  ## [1.0.1] - 2026-09-14
21
73
 
22
74
  A packaging and release-tooling release. **Nothing about screening changes**: no behaviour
@@ -600,6 +652,7 @@ summarized here because they are what a reader of a first release most needs:
600
652
  - Recall at the default threshold is 0.939 overall on the labeled set, and every record this
601
653
  version misses is named in the committed accuracy report.
602
654
 
603
- [Unreleased]: https://github.com/Babystep-Technologies/active_sanction/compare/v1.0.1...main
655
+ [Unreleased]: https://github.com/Babystep-Technologies/active_sanction/compare/v1.1.0...main
656
+ [1.1.0]: https://github.com/Babystep-Technologies/active_sanction/compare/v1.0.1...v1.1.0
604
657
  [1.0.1]: https://github.com/Babystep-Technologies/active_sanction/compare/v1.0.0...v1.0.1
605
658
  [1.0.0]: https://github.com/Babystep-Technologies/active_sanction/releases/tag/v1.0.0
@@ -4,7 +4,7 @@ What this gem promises not to break, and what it reserves the right to change
4
4
  in any release.
5
5
 
6
6
  **The public surface is enumerated below, not inferred.** A constant being
7
- reachable does not make it public; nearly 500 of them are reachable and 137 are
7
+ reachable does not make it public; over 500 of them are reachable and 142 are
8
8
  promised. Everything else is marked `@api private` in the source, is hidden
9
9
  from the rendered documentation, and may be renamed, moved or deleted in a
10
10
  patch release without a note anywhere. If you need something that is not on
@@ -107,7 +107,7 @@ on working.
107
107
 
108
108
  ## Contracts that are not constants
109
109
 
110
- Four promises here are about behaviour rather than about a name, and none of
110
+ Five promises here are about behaviour rather than about a name, and none of
111
111
  them is enforceable by the surface spec.
112
112
 
113
113
  **The error hierarchy.** Within a major version an error does not move to a
@@ -128,6 +128,35 @@ anyone may produce or consume one, in any language.
128
128
  store written against one schema version keeps being readable; the file layout
129
129
  underneath a shipped store is not public and may change.
130
130
 
131
+ **The instrumentation events.** `ActiveSanction::Instrumentation::EVENTS`
132
+ names the six, and each one's payload keys are promised the same way a method
133
+ signature is: within a major version a key is not removed, renamed, or made to
134
+ mean something else, and a dashboard written against one keeps working. Keys
135
+ may be **added** to an event — that is how a new measurement ships without a
136
+ major version — so a subscriber reads the keys it knows and ignores the rest,
137
+ and must not assume the set is closed.
138
+
139
+ Every event carries `name`, `started_at` and `duration`, plus the keys below.
140
+ `error` is present only when the stage raised, in which case the keys it had
141
+ not reached yet are **absent rather than zero**.
142
+
143
+ | Event | Payload keys |
144
+ |---|---|
145
+ | `:fetch` | `source`, `key`, `url`, `forced`, `conditional`, `status`, `not_modified`, `bytes` |
146
+ | `:parse` | `source`, `bytes`, `records`, `warnings` |
147
+ | `:store` | `source`, `snapshot_id`, `entities`, `store`, `imported` (on an import only) |
148
+ | `:"index.build"` | `store`, `sources`, `snapshots`, `entities`, `names`, `keys`, `postings`, `bytes` |
149
+ | `:screen` | `candidates`, `scored`, `results`, `threshold`, `limit`, `sources`, `snapshots` |
150
+ | `:sync` | `sources`, `forced`, `concurrency`, `outcomes`, `updated`, `unchanged`, `failed`, `records` |
151
+
152
+ Two things about them are contracts rather than incidental. **A `:fetch` event
153
+ is one HTTP round trip** — a file served out of the payload cache after a 304
154
+ costs no request and emits nothing, and a source re-fetching one because its
155
+ cache was empty emits a second event rather than amending the first. And
156
+ **`bytes` on `:"index.build"` is an estimate**, documented as one on
157
+ `Index#profile`; it is good to within a factor a dashboard cares about and is
158
+ not a heap measurement.
159
+
131
160
  **A `MatchResult` is reproducible.** The snapshot checksum, matcher version,
132
161
  weights and query it stamps are what let a screening decision be re-derived
133
162
  years later. Fields may be added to that stamp; the meaning of an existing one
@@ -138,6 +167,12 @@ does not change under it.
138
167
  Named here because their absence from the list is a decision rather than an
139
168
  oversight:
140
169
 
170
+ - **The instrumenter contract's other half.** A subscriber is anything
171
+ answering `#call(event)`, and that is public. `Instrumentation.instrument`
172
+ and `.emit`, which the library's own stages call, are not: where an event is
173
+ emitted from is an implementation detail of the stage, and a host that
174
+ wanted to emit one of these names itself would be publishing a measurement
175
+ of something this library did not do.
141
176
  - **The matching internals** — `Index`, `Similarity`, `Phonetics`, and the
142
177
  scorer's `Adjustments` and `NameScore`. These are where accuracy work
143
178
  happens, and accuracy work that had to preserve a signature would stop.
@@ -193,6 +228,16 @@ ActiveSanction::Configuration::DEFAULT_USER_AGENT
193
228
  ActiveSanction::Configuration::DEFAULT_XML_BACKEND
194
229
  ```
195
230
 
231
+ ### Instrumentation
232
+
233
+ ```
234
+ ActiveSanction::Instrumentation
235
+ ActiveSanction::Instrumentation::EVENTS
236
+ ActiveSanction::Instrumentation::Event
237
+ ActiveSanction::Instrumentation::Notifications
238
+ ActiveSanction::Instrumentation::Notifications::NAMESPACE
239
+ ```
240
+
196
241
  ### The canonical record
197
242
 
198
243
  ```
@@ -5,6 +5,7 @@ require "sorbet-runtime"
5
5
 
6
6
  require "active_sanction/configuration"
7
7
  require "active_sanction/error"
8
+ require "active_sanction/instrumentation"
8
9
  require "active_sanction/match_result"
9
10
 
10
11
  module ActiveSanction
@@ -157,7 +158,8 @@ module ActiveSanction
157
158
  @lock.synchronize do
158
159
  @matcher ||= with_configuration do
159
160
  Matcher.build(storage, sources: configuration.sources, weights: configuration.scorer_weights,
160
- candidate_limit: configuration.candidate_limit, backend: backend)
161
+ candidate_limit: configuration.candidate_limit, backend: backend,
162
+ instrumenter: configuration.instrumenter)
161
163
  end
162
164
  end
163
165
  end
@@ -286,7 +288,14 @@ module ActiveSanction
286
288
  sig { params(path: T.untyped, verify_with: T.untyped).returns(Snapshot) }
287
289
  def import(path, verify_with: nil)
288
290
  snapshot = ::File.open(path.to_s, "rb") { |io| Snapshot::Bundle.read(io, verify_with: verify_with) }
289
- with_configuration { storage.write_snapshot(snapshot) }
291
+ # The same `:store` event a sync emits, because it is the same fact: a
292
+ # list version was written to this store, and a host watching data
293
+ # freshness should not have to know which of the two ways it arrived.
294
+ fields = { source: snapshot.source, snapshot_id: snapshot.checksum, entities: snapshot.record_count,
295
+ store: storage.class.name, imported: true }
296
+ Instrumentation.instrument(configuration.instrumenter, :store, fields) do
297
+ with_configuration { storage.write_snapshot(snapshot) }
298
+ end
290
299
  reload!
291
300
  snapshot
292
301
  end
@@ -4,6 +4,7 @@
4
4
  require "sorbet-runtime"
5
5
 
6
6
  require "active_sanction/error"
7
+ require "active_sanction/instrumentation"
7
8
  require "active_sanction/normalizer"
8
9
  require "active_sanction/scorer/weights"
9
10
  require "active_sanction/version"
@@ -245,6 +246,11 @@ module ActiveSanction
245
246
  sig { returns(T.untyped) }
246
247
  attr_reader :logger
247
248
 
249
+ # Anything answering `#call(event)`, or nil for the default, which is that
250
+ # nothing is listening. See #instrumenter= and Instrumentation.
251
+ sig { returns(T.untyped) }
252
+ attr_reader :instrumenter
253
+
248
254
  sig { void }
249
255
  def initialize
250
256
  @user_agent = T.let(DEFAULT_USER_AGENT, String)
@@ -267,6 +273,7 @@ module ActiveSanction
267
273
  @normalizer_dictionary = T.let(Normalizer::Dictionary.default, Normalizer::Dictionary)
268
274
  @scorer_weights = T.let(Scorer::Weights.default, Scorer::Weights)
269
275
  @logger = T.let(nil, T.untyped)
276
+ @instrumenter = T.let(nil, T.untyped)
270
277
  @storage = T.let(nil, T.nilable(Storage::Base))
271
278
  @default_storage = T.let(nil, T.nilable(Storage::Base))
272
279
  end
@@ -569,6 +576,38 @@ module ActiveSanction
569
576
  @logger = value
570
577
  end
571
578
 
579
+ # Where this library's structured events go: a lambda, a Method, or any
580
+ # object answering `#call(event)`.
581
+ #
582
+ # c.instrumenter = ->(event) { StatsD.timing("sanctions.#{event.name}", event.duration_ms) }
583
+ # c.instrumenter = ActiveSanction::Instrumentation::Notifications.new # a Rails host
584
+ #
585
+ # Six events -- `:fetch`, `:parse`, `:store`, `:"index.build"`, `:screen`
586
+ # and `:sync` -- each carrying a duration and the ids needed to correlate
587
+ # it. Their payload keys are public API; see Instrumentation, which is
588
+ # where all of it is documented, and docs/api_stability.md, which
589
+ # enumerates the keys.
590
+ #
591
+ # The default is nil, and nil is a branch rather than a no-op object: an
592
+ # installation that instruments nothing pays nothing, which is the only
593
+ # way a per-query event could be affordable at all.
594
+ #
595
+ # This is read where a stage is *built* rather than where it runs -- a
596
+ # Matcher takes its instrumenter at `build` and freezes it, the way it
597
+ # freezes its weights -- so changing it here affects the next matcher,
598
+ # sync or fetcher and never one already running. That is the same rule
599
+ # every other setting on the query path follows, and it is what keeps a
600
+ # batch from being half one set of numbers and half another.
601
+ sig { params(value: T.untyped).void }
602
+ def instrumenter=(value)
603
+ unless value.nil? || value.respond_to?(:call)
604
+ raise ConfigurationError,
605
+ "instrumenter must respond to #call(event), got #{value.class}. See ActiveSanction::Instrumentation."
606
+ end
607
+
608
+ @instrumenter = value
609
+ end
610
+
572
611
  sig { returns(String) }
573
612
  def self.default_storage_dir
574
613
  -File.expand_path(File.join(Dir.home, DEFAULT_STORAGE_DIRNAME))
@@ -5,6 +5,7 @@ require "sorbet-runtime"
5
5
 
6
6
  require "active_sanction/error"
7
7
  require "active_sanction/http_client"
8
+ require "active_sanction/instrumentation"
8
9
  require "active_sanction/validators"
9
10
  require "active_sanction/validator_store"
10
11
  require "active_sanction/fetcher/result"
@@ -64,20 +65,28 @@ module ActiveSanction
64
65
  sig { returns(T.untyped) }
65
66
  attr_reader :logger
66
67
 
68
+ # Where the `:fetch` event goes, or nil for nothing listening. See
69
+ # Instrumentation.
70
+ sig { returns(T.untyped) }
71
+ attr_reader :instrumenter
72
+
67
73
  # The store defaults to disk, so the second run of a cron job benefits and
68
74
  # not merely the second call in one process. A caller that would rather
69
75
  # keep nothing between runs passes ValidatorStore::Memory.new.
70
76
  sig do
71
- params(client: HttpClient, store: T.untyped, stale_after: T.nilable(Numeric), logger: T.untyped).void
77
+ params(client: HttpClient, store: T.untyped, stale_after: T.nilable(Numeric), logger: T.untyped,
78
+ instrumenter: T.untyped).void
72
79
  end
73
80
  def initialize(client: HttpClient.new,
74
81
  store: ValidatorStore::FileSystem.new,
75
82
  stale_after: ActiveSanction.config.stale_after,
76
- logger: ActiveSanction.config.logger)
83
+ logger: ActiveSanction.config.logger,
84
+ instrumenter: ActiveSanction.config.instrumenter)
77
85
  @client = T.let(client, HttpClient)
78
86
  @store = T.let(store, T.untyped)
79
87
  @stale_after = T.let(stale_after, T.nilable(Numeric))
80
88
  @logger = T.let(logger, T.untyped)
89
+ @instrumenter = T.let(instrumenter, T.untyped)
81
90
  end
82
91
 
83
92
  # Fetches conditionally and buffers the body, like HttpClient#get.
@@ -91,12 +100,17 @@ module ActiveSanction
91
100
  # `force: true` sends no validators, so the publisher has no way to answer
92
101
  # 304. For the operator who suspects the cached copy is wrong and wants the
93
102
  # bytes regardless of what the ETag says.
103
+ # `source:` is which list this file belongs to, and is only ever read by
104
+ # instrumentation: a multi-file source files its validators under
105
+ # `:"ofac_sdn-sdn"` and the rest, and a dashboard asking which *list* is
106
+ # degrading wants `:ofac_sdn`. Defaults to the fetch key, which is right
107
+ # for every single-file source and for a caller fetching a bare URL.
94
108
  sig do
95
- params(url: T.untyped, key: T.untyped, force: T::Boolean, headers: T::Hash[T.untyped, T.untyped])
96
- .returns(Result)
109
+ params(url: T.untyped, key: T.untyped, force: T::Boolean, headers: T::Hash[T.untyped, T.untyped],
110
+ source: T.untyped).returns(Result)
97
111
  end
98
- def fetch(url, key: url, force: false, headers: {})
99
- conditional(url, key, force, headers) { |request| client.get(url, headers: request) }
112
+ def fetch(url, key: url, force: false, headers: {}, source: nil)
113
+ conditional(url, key, force, headers, source) { |request| client.get(url, headers: request) }
100
114
  end
101
115
 
102
116
  # Streams conditionally to disk, like HttpClient#download. A 304 writes
@@ -104,10 +118,10 @@ module ActiveSanction
104
118
  # is left exactly as the last download left it.
105
119
  sig do
106
120
  params(url: T.untyped, to: T.untyped, key: T.untyped, force: T::Boolean,
107
- headers: T::Hash[T.untyped, T.untyped]).returns(Result)
121
+ headers: T::Hash[T.untyped, T.untyped], source: T.untyped).returns(Result)
108
122
  end
109
- def download(url, to:, key: url, force: false, headers: {})
110
- conditional(url, key, force, headers) { |request| client.download(url, to: to, headers: request) }
123
+ def download(url, to:, key: url, force: false, headers: {}, source: nil)
124
+ conditional(url, key, force, headers, source) { |request| client.download(url, to: to, headers: request) }
111
125
  end
112
126
 
113
127
  # Whether a sync is due, answered locally and without a request.
@@ -149,14 +163,26 @@ module ActiveSanction
149
163
 
150
164
  sig do
151
165
  params(url: T.untyped, key: T.untyped, force: T::Boolean, headers: T::Hash[T.untyped, T.untyped],
166
+ source: T.untyped,
152
167
  block: T.proc.params(request: T::Hash[T.untyped, T.untyped]).returns(HttpClient::Response))
153
168
  .returns(Result)
154
169
  end
155
- def conditional(url, key, force, headers, &block)
170
+ def conditional(url, key, force, headers, source, &block)
156
171
  stored = force ? nil : usable(key, url)
157
172
  log_request(key, url, stored, force)
158
- response = block.call(merge(headers, stored))
159
- record(key, url, stored, response)
173
+ # One event per HTTP round trip, which is why `forced` is on it: a file
174
+ # served out of the payload cache after a 304 costs no request and emits
175
+ # nothing, and a source re-fetching one because its cache was empty
176
+ # emits a second event rather than amending the first.
177
+ fields = { source: source || key, key: key, url: url.to_s, forced: force }
178
+ Instrumentation.instrument(instrumenter, :fetch, fields) do |event|
179
+ response = block.call(merge(headers, stored))
180
+ event[:status] = response.status
181
+ event[:not_modified] = response.not_modified?
182
+ event[:bytes] = response.body.to_s.bytesize
183
+ event[:conditional] = !stored.nil?
184
+ record(key, url, stored, response)
185
+ end
160
186
  end
161
187
 
162
188
  # Validators stored against a different URL are not merely useless, they
@@ -144,6 +144,28 @@ module ActiveSanction
144
144
  # numbers again on another machine.
145
145
  POSTINGS_BUDGET = T.let(5_000, Integer)
146
146
 
147
+ # One posting: an Integer in an Array slot, which on a 64-bit CRuby is a
148
+ # tagged immediate and costs the slot and nothing else. See #profile,
149
+ # which is where these three are used and where their crudeness is
150
+ # admitted.
151
+ #
152
+ # @api private
153
+ POSTING_BYTES = T.let(8, Integer)
154
+
155
+ # One distinct feature: a short String -- a token, a trigram, a metaphone
156
+ # key -- plus its slot in a posting Hash and the empty Array header the
157
+ # slot points at. Measured as an order of magnitude rather than counted.
158
+ #
159
+ # @api private
160
+ FEATURE_BYTES = T.let(120, Integer)
161
+
162
+ # One indexed name: an Entry and the Normalizer::Form it holds, both small
163
+ # objects of references. The Entity and Name they point at belong to the
164
+ # snapshot and are not counted here.
165
+ #
166
+ # @api private
167
+ ENTRY_BYTES = T.let(200, Integer)
168
+
147
169
  # Entries, in the order they were indexed. The posting lists hold
148
170
  # positions in this array.
149
171
  sig { returns(T::Array[Entry]).checked(:tests) }
@@ -247,6 +269,45 @@ module ActiveSanction
247
269
  }
248
270
  end
249
271
 
272
+ # How big this index is, as the `index.build` event reports it: names,
273
+ # distinct features, postings, and an estimate of what it all weighs.
274
+ #
275
+ # Deliberately not #stats, and the difference is the one number #stats
276
+ # carries that this does not. Counting distinct entities means walking
277
+ # every entry and uniquing 47,000 ids, which is a fine thing to do in a
278
+ # benchmark and not a thing to do at the end of every index build -- it
279
+ # allocates at exactly the moment the heap is already at its peak, with
280
+ # the snapshot, the builder and the finished index all still alive. The
281
+ # event reports entity count from the snapshots it read instead, where it
282
+ # is already known.
283
+ #
284
+ # **`bytes` is an estimate, and deliberately a crude one.** Ruby offers no
285
+ # way to ask what an object graph weighs that does not either walk every
286
+ # object in the heap or lie, and an index is tens of millions of small
287
+ # objects. What it counts is the three things that dominate: a posting is
288
+ # an Integer in an Array slot, a distinct feature is a String plus its
289
+ # slot in a Hash, and an entry is a small object holding references. What
290
+ # it deliberately does not count is the Entities and Names themselves --
291
+ # those are the snapshot's, shared with it rather than owned here, and
292
+ # counting them would report the same megabytes twice to a host watching
293
+ # both.
294
+ #
295
+ # Good to within a factor a dashboard cares about, which is what "is the
296
+ # index growing?" and "will another list fit?" actually need. Anything
297
+ # finer wants a real heap profiler.
298
+ sig { returns(T::Hash[Symbol, Integer]).checked(:tests) }
299
+ def profile
300
+ keys = @tokens.size + @trigrams.size + @phonetics.size
301
+ # `each_value` rather than a two-parameter block over the Hash: iterating
302
+ # a Hash as pairs allocates a two-element Array per distinct feature,
303
+ # which is twenty thousand short-lived objects at the exact moment the
304
+ # heap is at its peak -- the snapshot, the builder and the finished
305
+ # index all still alive. #stats can afford that and a build cannot.
306
+ postings = [@tokens, @trigrams, @phonetics].sum { |space| space.each_value.sum(&:size) }
307
+ { names: entries.size, keys: keys, postings: postings,
308
+ bytes: (postings * POSTING_BYTES) + (keys * FEATURE_BYTES) + (entries.size * ENTRY_BYTES) }
309
+ end
310
+
250
311
  sig { returns(String) }
251
312
  def inspect = "#<#{self.class} #{size} names>"
252
313
 
@@ -0,0 +1,122 @@
1
+ # typed: strict
2
+ # frozen_string_literal: true
3
+
4
+ require "sorbet-runtime"
5
+
6
+ module ActiveSanction
7
+ module Instrumentation
8
+ # One thing this library did, after it finished doing it.
9
+ #
10
+ # ActiveSanction.configure do |c|
11
+ # c.instrumenter = ->(event) { StatsD.timing("sanctions.#{event.name}", event.duration_ms) }
12
+ # end
13
+ #
14
+ # event.name # => :fetch
15
+ # event.duration # => 2.418, seconds
16
+ # event[:source] # => :ofac_sdn
17
+ # event[:bytes] # => 12_845_056
18
+ #
19
+ # A subscriber is handed a finished event and never a running one. That is
20
+ # the difference between this and a block-based instrumenter, and it is
21
+ # deliberate: a subscriber that cannot wrap the work cannot retry it,
22
+ # cannot swallow its exception, and cannot leave a `begin` half-entered if
23
+ # it raises. The cost is that nothing here can time a stage from the
24
+ # outside -- which nothing needs to, since every event already carries its
25
+ # own duration.
26
+ #
27
+ # ### Every event carries a duration and enough to correlate it
28
+ #
29
+ # There are no anonymous timings. `duration` is monotonic seconds, taken
30
+ # across the stage this event describes, and `started_at` is the wall
31
+ # clock at its start -- both, because one answers "how long did it take"
32
+ # and the other answers "when, in the log I am reading beside this". Every
33
+ # event but `screen` names a `source`; `screen` names the snapshot
34
+ # checksums it consulted, which is the same question asked of a query.
35
+ #
36
+ # ### An event is emitted for work that raised
37
+ #
38
+ # With `error` set to the exception, and with whichever payload keys the
39
+ # stage had filled in before it failed. A publisher that starts timing out
40
+ # is exactly what a host is watching for, and an instrumentation layer
41
+ # that only reports successes cannot see it. See Instrumentation.
42
+ #
43
+ # Frozen, and its payload with it, so a subscriber cannot edit what the
44
+ # next one is handed.
45
+ class Event
46
+ extend T::Sig
47
+
48
+ # The stage this event describes -- one of Instrumentation::EVENTS.
49
+ sig { returns(Symbol) }
50
+ attr_reader :name
51
+
52
+ # What the stage measured, keyed as documented for each event name in
53
+ # docs/api_stability.md. Frozen.
54
+ sig { returns(T::Hash[Symbol, T.untyped]) }
55
+ attr_reader :payload
56
+
57
+ # The wall clock when the stage started, UTC. For lining an event up
58
+ # against a log; `duration` is what to measure with.
59
+ sig { returns(Time) }
60
+ attr_reader :started_at
61
+
62
+ # Seconds the stage took, from a monotonic clock, so it is not moved by
63
+ # a clock adjustment mid-stage.
64
+ sig { returns(Float) }
65
+ attr_reader :duration
66
+
67
+ sig do
68
+ params(name: Symbol, payload: T::Hash[Symbol, T.untyped], started_at: Time, duration: Float).void
69
+ end
70
+ def initialize(name:, payload:, started_at:, duration:)
71
+ @name = T.let(name, Symbol)
72
+ @payload = T.let(payload.freeze, T::Hash[Symbol, T.untyped])
73
+ @started_at = T.let(started_at, Time)
74
+ @duration = T.let(duration, Float)
75
+ freeze
76
+ end
77
+
78
+ # One payload key, or nil for one this event does not carry.
79
+ sig { params(key: Symbol).returns(T.untyped) }
80
+ def [](key) = payload[key]
81
+
82
+ # Which list this was about, or nil for an event that is not about one
83
+ # -- `screen`, which is about a query, and `sync`, which is about a run.
84
+ sig { returns(T.nilable(Symbol)) }
85
+ def source = payload[:source]
86
+
87
+ # The exception the stage raised, or nil for one that finished. An event
88
+ # carrying one is a partial measurement: the keys the stage had not
89
+ # reached are absent rather than zero.
90
+ sig { returns(T.nilable(StandardError)) }
91
+ def error = payload[:error]
92
+
93
+ sig { returns(T::Boolean) }
94
+ def failed? = !error.nil?
95
+
96
+ # The wall clock when the stage finished. Derived from `started_at` and
97
+ # the monotonic duration rather than read again, so the two cannot
98
+ # disagree about how long this took.
99
+ sig { returns(Time) }
100
+ def finished_at = started_at + duration
101
+
102
+ # Milliseconds, which is what a metrics backend usually wants.
103
+ sig { returns(Float) }
104
+ def duration_ms = (duration * 1_000).round(3).to_f
105
+
106
+ # The whole event as one Hash, for a subscriber that forwards it
107
+ # somewhere structured. The payload is spread in at the top level, and
108
+ # its keys win over nothing -- `name`, `started_at` and `duration` are
109
+ # not payload keys on any event this library emits.
110
+ sig { returns(T::Hash[Symbol, T.untyped]) }
111
+ def to_h
112
+ { name: name, started_at: started_at, duration: duration }.merge(payload)
113
+ end
114
+
115
+ sig { returns(String) }
116
+ def to_s = "#{name} #{format("%.3f", duration)}s#{" #{source}" if source}#{" failed" if failed?}"
117
+
118
+ sig { returns(String) }
119
+ def inspect = "#<#{self.class} #{name} #{payload.inspect} #{format("%.3f", duration)}s>"
120
+ end
121
+ end
122
+ end