active_sanction 1.0.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 63d579349c5ed0f731edbb3811cea143c2a1bdf7f591b319fb290ab6fb28c8c6
-  data.tar.gz: e6ad0e3c72fdca696dc2d9e2e1d528a4a43e638054362f1ebf0931def9b309c8
+  metadata.gz: ffb2d77bf3207e3ef97a16269dcf91737e20d48871cc2aec9e747d4185793e52
+  data.tar.gz: eeca2cf2bb69972d83e15e982c2daea80969a636e4c88e86cbd29b670420098c
 SHA512:
-  metadata.gz: '086eb3b75aa22d1cf3b976b269f28f49a0b5a734a22bba52f0a25119125042c40d41a35fc25b694069814d7f3d4e24426e51ef5da15f5d91f24c33519eff97f4'
-  data.tar.gz: d5cf18bc582c46e84b1255328e5a8411bb6e35c5444c8660c2687b7be5621411333f0d3a782fd352ad0963ab96e72ed93f1abd3fb7349bccc073f4f02e2e4452
+  metadata.gz: 748d59d2d06bb502eefd684a71c1abc74beb6b03d677eaf9a85c4563945c03fb7c9d69474988319d0ef1f03d4a68c328efd7d63ad14a62d33fa74ef4989e361b
+  data.tar.gz: dc9c4247f9e3ce24772a8045c4e7e0ea992ae7fbeb17c8f2f123f8a9ccf39fd741d7617a98cd67eef5968a16811c2cfc1b42a0c433df3da94d3b8e4f3f624f40

data/CHANGELOG.md

@@ -13,6 +13,123 @@ algorithms or the scorer could move a score. **A change that moves `MATCHER_VERS
 called out here as such**, because it is the one kind of change that alters what a past
 screening decision would come out as today.
 
+## [Unreleased]
+
+Nothing yet.
+
+## [1.1.0] - 2026-09-14
+
+**A minor, because it adds.** Nothing that existed changed: no behaviour under `lib/`
+moved, `MATCHER_VERSION` stays at `1`, and a name scores today exactly what it scored
+under 1.0.0. Instrumentation is additive and off unless a host asks for it.
+
+### Added
+
+- **Instrumentation: six structured events, so a host can measure this library without
+  monkeypatching it** (#59). `ActiveSanction.configure { |c| c.instrumenter = ... }` takes
+  anything answering `#call(event)` and is handed a finished
+  `ActiveSanction::Instrumentation::Event` for each of `:fetch`, `:parse`, `:store`,
+  `:"index.build"`, `:screen` and `:sync` — every one carrying a duration and the ids
+  needed to correlate it, with no anonymous timings. The event names and every payload key
+  are public API, enumerated in [`docs/api_stability.md`](docs/api_stability.md) and on the
+  site's [instrumentation reference](https://babystep.tech/active_sanction/reference/instrumentation/),
+  and covered by the deprecation path from here.
+
+  **It is a measurement of the work and never part of it.** A subscriber that raises has its
+  exception caught, reported once through the configured logger, and dropped; the sync it
+  was watching finishes and returns the report it was going to return. The converse holds
+  too: a stage that raises emits its event with `error:` set and then the exception
+  continues exactly as if nothing were listening.
+
+  **Nothing is listening by default, and that costs nothing.** `nil` is a branch taken
+  before anything is allocated rather than a no-op object that gets called, so an
+  uninstrumented screening call builds no event and allocates no payload — which is the
+  only way a per-query event could be affordable at all. Measured rather than asserted:
+  building a matcher over 47,051 names allocates 262 more objects than before, out of
+  5.79 million, and `rake benchmark:latency` reports the same p50 either side of the change
+  (12.1–12.8 ms against a run-to-run spread that was already that wide).
+
+  Rails hosts get `ActiveSanction::Instrumentation::Notifications`, which republishes every
+  event into `ActiveSupport::Notifications` under `<event>.active_sanction`. It is an
+  adapter and not a dependency: **nothing in this gem requires ActiveSupport**, and building
+  one in a process that has not loaded it raises `ConfigurationError` rather than quietly
+  instrumenting nothing.
+
+  The issue asked for `ActiveSanction.instrumenter = ...`, and this is a configuration
+  setting instead. #55 ended process-global configuration deliberately, and a module-level
+  writer would have rebuilt the default client — dropping the matcher it had indexed every
+  stored list into — as a side effect of naming a subscriber. A `Matcher` takes its
+  instrumenter at build and freezes it with its weights, so a subscriber swapped halfway
+  through a batch cannot make half of it instrumented; that is the rule every other setting
+  on the query path already follows.
+
+- **`Sources::Base#warnings`**, defaulting to none. Every shipped adapter already exposed
+  it and the adapter rules already required it of a new one, but the base class never said
+  so — and the `:parse` event counts warnings for every source, which a count that is
+  sometimes a `NoMethodError` cannot do. An optional hook with a default implementation, so
+  no adapter outside this repository has to change.
+
+## [1.0.1] - 2026-09-14
+
+A packaging and release-tooling release. **Nothing about screening changes**: no behaviour
+in `lib/` moved, `MATCHER_VERSION` is unchanged at `1`, and a name scores today exactly what
+it scored under 1.0.0.
+
+### Added
+
+- **Releases publish themselves from a tag**, through
+  [`.github/workflows/release.yml`](.github/workflows/release.yml). Pushing `v1.2.3` re-runs
+  the three gates against the tagged tree, checks that the tag and `VERSION` agree, that the
+  tag is an ancestor of `main` and that the changelog has a section for it, then builds the
+  gem, publishes it and writes the GitHub release from that section. A tag failing any of
+  those publishes nothing.
+
+  **No API key exists to leak.** It authenticates by
+  [trusted publishing](https://guides.rubygems.org/trusted-publishing): a short-lived OIDC
+  token, verified by rubygems.org as naming this repository and this workflow file, exchanged
+  for a credential that expires with the job. The alternative is a long-lived key in a public
+  repository's settings, one leak away from someone else publishing under this gem's name --
+  and unlike a bad deploy, a bad gem is already installed by the time anyone could be warned.
+  Same reasoning that keeps a service-account key out of the documentation deploy.
+
+  The gem pushed is the gem that was verified, carried between the two jobs as an artifact
+  rather than rebuilt -- a second build is a second thing, however identical it looks.
+
+  **The publish is confirmed against rubygems.org** (#136), because `gem push` exiting 0 says
+  the upload was accepted rather than that a `bundle install` will find the version. The job
+  that writes the GitHub release waits on that confirmation, so a version rubygems.org did not
+  end up serving is never announced.
+
+- **A gem badge on the README** (#136), read live from rubygems.org rather than generated into
+  a file. What it states is what is installable, which is a different question from what was
+  last tagged -- and a version written into the repository would be stale the moment the next
+  one published. Same argument as #104, applied to the one fact about this gem that lives
+  somewhere else entirely.
+
+- **A release can be cut from the Actions tab**, without tagging by hand (#139). The
+  workflow's "Run workflow" button takes a tag and a checkbox: ticked, it runs every gate
+  against `main` and writes the tag only once they have all passed. That is the safer order
+  than `git tag && git push`, which makes a tag public before anything has checked the tree
+  under it — and this workflow will not move a tag somebody may already have fetched, so the
+  repair for that is a new version number. A tag it writes is annotated but unsigned; push
+  the tag yourself when you want your own signature on it. See
+  [`CONTRIBUTING.md`](CONTRIBUTING.md#releasing).
+
+### Fixed
+
+- **`rake canary:refresh` regenerates the catalogue page's data as well as the baselines.**
+  `site/src/data/sources.json` is derived from `.github/baselines` (#104), so accepting new
+  numbers without rebuilding it left the two disagreeing and
+  [`spec/site_sources_data_spec.rb`](spec/site_sources_data_spec.rb) red. The rolling
+  baseline pull request the canary opens had failed on this every run since it started
+  opening one, on all six Rubies. Chained onto the task rather than added as a step to each
+  caller: a derived file that callers have to remember to rebuild is stale by the third
+  caller.
+
+- **The canary signs its baseline commit off.** Nothing exempts a bot from
+  [`.github/workflows/dco.yml`](.github/workflows/dco.yml), which skips merge commits and
+  nothing else, so the rolling pull request failed the DCO check as well.
+
 ## [1.0.0] - 2026-09-13
 
 The first release. Everything below is in it.
@@ -535,4 +652,7 @@ summarized here because they are what a reader of a first release most needs:
 - Recall at the default threshold is 0.939 overall on the labeled set, and every record this
   version misses is named in the committed accuracy report.
 
+[Unreleased]: https://github.com/Babystep-Technologies/active_sanction/compare/v1.1.0...main
+[1.1.0]: https://github.com/Babystep-Technologies/active_sanction/compare/v1.0.1...v1.1.0
+[1.0.1]: https://github.com/Babystep-Technologies/active_sanction/compare/v1.0.0...v1.0.1
 [1.0.0]: https://github.com/Babystep-Technologies/active_sanction/releases/tag/v1.0.0

data/CONTRIBUTING.md

@@ -232,11 +232,75 @@ comment restating the name would be noise.
 
 ## Releasing
 
-`bundle exec rake install` installs the gem locally. A release is: bump
-`VERSION` in [`lib/active_sanction/version.rb`](lib/active_sanction/version.rb),
-move the `Unreleased` section of [`CHANGELOG.md`](CHANGELOG.md) under the new
-version with its date, then `bundle exec rake release`, which tags, pushes
-and publishes to [rubygems.org](https://rubygems.org).
+`bundle exec rake install` installs the gem locally.
+
+A release is a tag. Bump `VERSION` in
+[`lib/active_sanction/version.rb`](lib/active_sanction/version.rb), move the
+`Unreleased` section of [`CHANGELOG.md`](CHANGELOG.md) under the new version
+with its date, and merge that.
+
+Then the tag, which you can let the workflow write or write yourself.
+
+**From the Actions tab**, which is the shorter path and the safer order. Open
+[`Release`](https://github.com/Babystep-Technologies/active_sanction/actions/workflows/release.yml),
+press **Run workflow**, enter `v1.2.3`, and tick **Create the tag on main**.
+Every gate runs against `main` first, and the tag is written only once they
+have all passed — so a release that turns out not to be publishable leaves no
+tag behind.
+
+**Or from a terminal**, which is the path to take when you want your own
+signature on the tag:
+
+    $ git tag -s v1.2.3 -m "Release 1.2.3"
+    $ git push origin v1.2.3
+
+The cost of that order is that the tag is public before anything has checked
+the tree under it. If a gate then fails, the tag stays where it is: this
+workflow will not move a tag somebody may already have fetched, so the repair
+is a new version rather than a retagged one.
+
+Either way, [`.github/workflows/release.yml`](.github/workflows/release.yml)
+does the rest. It re-runs the three gates against the tree being released,
+checks that the tag and `VERSION` agree, that the commit is on `main`, and
+that the changelog has a section for it — then builds the gem, publishes it to
+[rubygems.org](https://rubygems.org), confirms rubygems.org is serving that
+version, and writes the GitHub release from that changelog section. A release
+that fails any of those checks publishes nothing, and a push rubygems.org did
+not end up serving is never announced (#136).
+
+**Leave the box unticked to re-run a publish.** A dispatch with `Create the
+tag` unticked publishes a tag that already exists — one pushed before this
+workflow did, or one whose publish failed after the tag was public — and
+nothing in this workflow ever moves or deletes a tag. Ticking the box against
+a tag that already exists is not an error either; it publishes that tag and
+says it created nothing.
+
+**A tag the workflow writes is annotated but not signed.** It has no key of
+yours, and signing with a machine key would say less than saying nothing. The
+tag names the commit every gate ran against, and the provenance that matters
+for what users install is on the other side: the gem is published by [trusted
+publishing](https://guides.rubygems.org/trusted-publishing) from this
+repository and this workflow file. Sign the tag yourself, with the terminal
+path above, when you want the tag to carry it too.
+
+The [gem badge](https://rubygems.org/gems/active_sanction) at the top of the
+README reads from rubygems.org rather than from anything in this repository,
+so it states what is installable rather than what was last tagged.
+
+**Nobody needs a RubyGems API key, including the person tagging.** The
+workflow authenticates by [trusted
+publishing](https://guides.rubygems.org/trusted-publishing): it mints a
+short-lived OIDC token, rubygems.org verifies the token names this repository
+and this workflow file, and hands back a credential that expires with the job.
+There is no publishing secret in this repository's settings, for the same
+reason there is no service-account key for the documentation deploy. It is
+configured once, on rubygems.org, under the gem's **Trusted Publishers**:
+owner `Babystep-Technologies`, repository `active_sanction`, workflow
+`release.yml`, environment `rubygems`.
+
+`bundle exec rake release` — the manual path that tags, pushes and publishes
+in one step — still exists and still works for anyone with push rights to the
+gem. It is not how this gem is released, and using it skips every check above.
 
 `MATCHER_VERSION` in the same file is bumped on a different occasion and for
 a different reason — whenever a change to the normalizer, the index, the

data/README.md

@@ -4,6 +4,7 @@
 
 # ActiveSanction
 
+[![Gem](https://img.shields.io/gem/v/active_sanction?label=gem&color=CC342D)](https://rubygems.org/gems/active_sanction)
 [![CI](https://github.com/Babystep-Technologies/active_sanction/actions/workflows/ci.yml/badge.svg)](https://github.com/Babystep-Technologies/active_sanction/actions/workflows/ci.yml)
 
 **[Full documentation, guides and the source catalogue →](https://babystep.tech/active_sanction/)**

data/docs/api_stability.md

@@ -4,7 +4,7 @@ What this gem promises not to break, and what it reserves the right to change
 in any release.
 
 **The public surface is enumerated below, not inferred.** A constant being
-reachable does not make it public; nearly 500 of them are reachable and 137 are
+reachable does not make it public; over 500 of them are reachable and 142 are
 promised. Everything else is marked `@api private` in the source, is hidden
 from the rendered documentation, and may be renamed, moved or deleted in a
 patch release without a note anywhere. If you need something that is not on
@@ -107,7 +107,7 @@ on working.
 
 ## Contracts that are not constants
 
-Four promises here are about behaviour rather than about a name, and none of
+Five promises here are about behaviour rather than about a name, and none of
 them is enforceable by the surface spec.
 
 **The error hierarchy.** Within a major version an error does not move to a
@@ -128,6 +128,35 @@ anyone may produce or consume one, in any language.
 store written against one schema version keeps being readable; the file layout
 underneath a shipped store is not public and may change.
 
+**The instrumentation events.** `ActiveSanction::Instrumentation::EVENTS`
+names the six, and each one's payload keys are promised the same way a method
+signature is: within a major version a key is not removed, renamed, or made to
+mean something else, and a dashboard written against one keeps working. Keys
+may be **added** to an event — that is how a new measurement ships without a
+major version — so a subscriber reads the keys it knows and ignores the rest,
+and must not assume the set is closed.
+
+Every event carries `name`, `started_at` and `duration`, plus the keys below.
+`error` is present only when the stage raised, in which case the keys it had
+not reached yet are **absent rather than zero**.
+
+| Event | Payload keys |
+|---|---|
+| `:fetch` | `source`, `key`, `url`, `forced`, `conditional`, `status`, `not_modified`, `bytes` |
+| `:parse` | `source`, `bytes`, `records`, `warnings` |
+| `:store` | `source`, `snapshot_id`, `entities`, `store`, `imported` (on an import only) |
+| `:"index.build"` | `store`, `sources`, `snapshots`, `entities`, `names`, `keys`, `postings`, `bytes` |
+| `:screen` | `candidates`, `scored`, `results`, `threshold`, `limit`, `sources`, `snapshots` |
+| `:sync` | `sources`, `forced`, `concurrency`, `outcomes`, `updated`, `unchanged`, `failed`, `records` |
+
+Two things about them are contracts rather than incidental. **A `:fetch` event
+is one HTTP round trip** — a file served out of the payload cache after a 304
+costs no request and emits nothing, and a source re-fetching one because its
+cache was empty emits a second event rather than amending the first. And
+**`bytes` on `:"index.build"` is an estimate**, documented as one on
+`Index#profile`; it is good to within a factor a dashboard cares about and is
+not a heap measurement.
+
 **A `MatchResult` is reproducible.** The snapshot checksum, matcher version,
 weights and query it stamps are what let a screening decision be re-derived
 years later. Fields may be added to that stamp; the meaning of an existing one
@@ -138,6 +167,12 @@ does not change under it.
 Named here because their absence from the list is a decision rather than an
 oversight:
 
+- **The instrumenter contract's other half.** A subscriber is anything
+  answering `#call(event)`, and that is public. `Instrumentation.instrument`
+  and `.emit`, which the library's own stages call, are not: where an event is
+  emitted from is an implementation detail of the stage, and a host that
+  wanted to emit one of these names itself would be publishing a measurement
+  of something this library did not do.
 - **The matching internals** — `Index`, `Similarity`, `Phonetics`, and the
   scorer's `Adjustments` and `NameScore`. These are where accuracy work
   happens, and accuracy work that had to preserve a signature would stop.
@@ -193,6 +228,16 @@ ActiveSanction::Configuration::DEFAULT_USER_AGENT
 ActiveSanction::Configuration::DEFAULT_XML_BACKEND
 ```
 
+### Instrumentation
+
+```
+ActiveSanction::Instrumentation
+ActiveSanction::Instrumentation::EVENTS
+ActiveSanction::Instrumentation::Event
+ActiveSanction::Instrumentation::Notifications
+ActiveSanction::Instrumentation::Notifications::NAMESPACE
+```
+
 ### The canonical record
 
 ```

data/lib/active_sanction/client.rb

@@ -5,6 +5,7 @@ require "sorbet-runtime"
 
 require "active_sanction/configuration"
 require "active_sanction/error"
+require "active_sanction/instrumentation"
 require "active_sanction/match_result"
 
 module ActiveSanction
@@ -157,7 +158,8 @@ module ActiveSanction
       @lock.synchronize do
         @matcher ||= with_configuration do
           Matcher.build(storage, sources: configuration.sources, weights: configuration.scorer_weights,
-                                 candidate_limit: configuration.candidate_limit, backend: backend)
+                                 candidate_limit: configuration.candidate_limit, backend: backend,
+                                 instrumenter: configuration.instrumenter)
         end
       end
     end
@@ -286,7 +288,14 @@ module ActiveSanction
     sig { params(path: T.untyped, verify_with: T.untyped).returns(Snapshot) }
     def import(path, verify_with: nil)
       snapshot = ::File.open(path.to_s, "rb") { |io| Snapshot::Bundle.read(io, verify_with: verify_with) }
-      with_configuration { storage.write_snapshot(snapshot) }
+      # The same `:store` event a sync emits, because it is the same fact: a
+      # list version was written to this store, and a host watching data
+      # freshness should not have to know which of the two ways it arrived.
+      fields = { source: snapshot.source, snapshot_id: snapshot.checksum, entities: snapshot.record_count,
+                 store: storage.class.name, imported: true }
+      Instrumentation.instrument(configuration.instrumenter, :store, fields) do
+        with_configuration { storage.write_snapshot(snapshot) }
+      end
       reload!
       snapshot
     end

data/lib/active_sanction/configuration.rb

@@ -4,6 +4,7 @@
 require "sorbet-runtime"
 
 require "active_sanction/error"
+require "active_sanction/instrumentation"
 require "active_sanction/normalizer"
 require "active_sanction/scorer/weights"
 require "active_sanction/version"
@@ -245,6 +246,11 @@ module ActiveSanction
     sig { returns(T.untyped) }
     attr_reader :logger
 
+    # Anything answering `#call(event)`, or nil for the default, which is that
+    # nothing is listening. See #instrumenter= and Instrumentation.
+    sig { returns(T.untyped) }
+    attr_reader :instrumenter
+
     sig { void }
     def initialize
       @user_agent = T.let(DEFAULT_USER_AGENT, String)
@@ -267,6 +273,7 @@ module ActiveSanction
       @normalizer_dictionary = T.let(Normalizer::Dictionary.default, Normalizer::Dictionary)
       @scorer_weights = T.let(Scorer::Weights.default, Scorer::Weights)
       @logger = T.let(nil, T.untyped)
+      @instrumenter = T.let(nil, T.untyped)
       @storage = T.let(nil, T.nilable(Storage::Base))
       @default_storage = T.let(nil, T.nilable(Storage::Base))
     end
@@ -569,6 +576,38 @@ module ActiveSanction
       @logger = value
     end
 
+    # Where this library's structured events go: a lambda, a Method, or any
+    # object answering `#call(event)`.
+    #
+    #   c.instrumenter = ->(event) { StatsD.timing("sanctions.#{event.name}", event.duration_ms) }
+    #   c.instrumenter = ActiveSanction::Instrumentation::Notifications.new   # a Rails host
+    #
+    # Six events -- `:fetch`, `:parse`, `:store`, `:"index.build"`, `:screen`
+    # and `:sync` -- each carrying a duration and the ids needed to correlate
+    # it. Their payload keys are public API; see Instrumentation, which is
+    # where all of it is documented, and docs/api_stability.md, which
+    # enumerates the keys.
+    #
+    # The default is nil, and nil is a branch rather than a no-op object: an
+    # installation that instruments nothing pays nothing, which is the only
+    # way a per-query event could be affordable at all.
+    #
+    # This is read where a stage is *built* rather than where it runs -- a
+    # Matcher takes its instrumenter at `build` and freezes it, the way it
+    # freezes its weights -- so changing it here affects the next matcher,
+    # sync or fetcher and never one already running. That is the same rule
+    # every other setting on the query path follows, and it is what keeps a
+    # batch from being half one set of numbers and half another.
+    sig { params(value: T.untyped).void }
+    def instrumenter=(value)
+      unless value.nil? || value.respond_to?(:call)
+        raise ConfigurationError,
+              "instrumenter must respond to #call(event), got #{value.class}. See ActiveSanction::Instrumentation."
+      end
+
+      @instrumenter = value
+    end
+
     sig { returns(String) }
     def self.default_storage_dir
       -File.expand_path(File.join(Dir.home, DEFAULT_STORAGE_DIRNAME))

data/lib/active_sanction/fetcher.rb

@@ -5,6 +5,7 @@ require "sorbet-runtime"
 
 require "active_sanction/error"
 require "active_sanction/http_client"
+require "active_sanction/instrumentation"
 require "active_sanction/validators"
 require "active_sanction/validator_store"
 require "active_sanction/fetcher/result"
@@ -64,20 +65,28 @@ module ActiveSanction
     sig { returns(T.untyped) }
     attr_reader :logger
 
+    # Where the `:fetch` event goes, or nil for nothing listening. See
+    # Instrumentation.
+    sig { returns(T.untyped) }
+    attr_reader :instrumenter
+
     # The store defaults to disk, so the second run of a cron job benefits and
     # not merely the second call in one process. A caller that would rather
     # keep nothing between runs passes ValidatorStore::Memory.new.
     sig do
-      params(client: HttpClient, store: T.untyped, stale_after: T.nilable(Numeric), logger: T.untyped).void
+      params(client: HttpClient, store: T.untyped, stale_after: T.nilable(Numeric), logger: T.untyped,
+             instrumenter: T.untyped).void
     end
     def initialize(client: HttpClient.new,
                    store: ValidatorStore::FileSystem.new,
                    stale_after: ActiveSanction.config.stale_after,
-                   logger: ActiveSanction.config.logger)
+                   logger: ActiveSanction.config.logger,
+                   instrumenter: ActiveSanction.config.instrumenter)
       @client = T.let(client, HttpClient)
       @store = T.let(store, T.untyped)
       @stale_after = T.let(stale_after, T.nilable(Numeric))
       @logger = T.let(logger, T.untyped)
+      @instrumenter = T.let(instrumenter, T.untyped)
     end
 
     # Fetches conditionally and buffers the body, like HttpClient#get.
@@ -91,12 +100,17 @@ module ActiveSanction
     # `force: true` sends no validators, so the publisher has no way to answer
     # 304. For the operator who suspects the cached copy is wrong and wants the
     # bytes regardless of what the ETag says.
+    # `source:` is which list this file belongs to, and is only ever read by
+    # instrumentation: a multi-file source files its validators under
+    # `:"ofac_sdn-sdn"` and the rest, and a dashboard asking which *list* is
+    # degrading wants `:ofac_sdn`. Defaults to the fetch key, which is right
+    # for every single-file source and for a caller fetching a bare URL.
     sig do
-      params(url: T.untyped, key: T.untyped, force: T::Boolean, headers: T::Hash[T.untyped, T.untyped])
-        .returns(Result)
+      params(url: T.untyped, key: T.untyped, force: T::Boolean, headers: T::Hash[T.untyped, T.untyped],
+             source: T.untyped).returns(Result)
     end
-    def fetch(url, key: url, force: false, headers: {})
-      conditional(url, key, force, headers) { |request| client.get(url, headers: request) }
+    def fetch(url, key: url, force: false, headers: {}, source: nil)
+      conditional(url, key, force, headers, source) { |request| client.get(url, headers: request) }
     end
 
     # Streams conditionally to disk, like HttpClient#download. A 304 writes
@@ -104,10 +118,10 @@ module ActiveSanction
     # is left exactly as the last download left it.
     sig do
       params(url: T.untyped, to: T.untyped, key: T.untyped, force: T::Boolean,
-             headers: T::Hash[T.untyped, T.untyped]).returns(Result)
+             headers: T::Hash[T.untyped, T.untyped], source: T.untyped).returns(Result)
     end
-    def download(url, to:, key: url, force: false, headers: {})
-      conditional(url, key, force, headers) { |request| client.download(url, to: to, headers: request) }
+    def download(url, to:, key: url, force: false, headers: {}, source: nil)
+      conditional(url, key, force, headers, source) { |request| client.download(url, to: to, headers: request) }
     end
 
     # Whether a sync is due, answered locally and without a request.
@@ -149,14 +163,26 @@ module ActiveSanction
 
     sig do
       params(url: T.untyped, key: T.untyped, force: T::Boolean, headers: T::Hash[T.untyped, T.untyped],
+             source: T.untyped,
              block: T.proc.params(request: T::Hash[T.untyped, T.untyped]).returns(HttpClient::Response))
         .returns(Result)
     end
-    def conditional(url, key, force, headers, &block)
+    def conditional(url, key, force, headers, source, &block)
       stored = force ? nil : usable(key, url)
       log_request(key, url, stored, force)
-      response = block.call(merge(headers, stored))
-      record(key, url, stored, response)
+      # One event per HTTP round trip, which is why `forced` is on it: a file
+      # served out of the payload cache after a 304 costs no request and emits
+      # nothing, and a source re-fetching one because its cache was empty
+      # emits a second event rather than amending the first.
+      fields = { source: source || key, key: key, url: url.to_s, forced: force }
+      Instrumentation.instrument(instrumenter, :fetch, fields) do |event|
+        response = block.call(merge(headers, stored))
+        event[:status] = response.status
+        event[:not_modified] = response.not_modified?
+        event[:bytes] = response.body.to_s.bytesize
+        event[:conditional] = !stored.nil?
+        record(key, url, stored, response)
+      end
     end
 
     # Validators stored against a different URL are not merely useless, they

data/lib/active_sanction/index.rb

@@ -144,6 +144,28 @@ module ActiveSanction
     # numbers again on another machine.
     POSTINGS_BUDGET = T.let(5_000, Integer)
 
+    # One posting: an Integer in an Array slot, which on a 64-bit CRuby is a
+    # tagged immediate and costs the slot and nothing else. See #profile,
+    # which is where these three are used and where their crudeness is
+    # admitted.
+    #
+    # @api private
+    POSTING_BYTES = T.let(8, Integer)
+
+    # One distinct feature: a short String -- a token, a trigram, a metaphone
+    # key -- plus its slot in a posting Hash and the empty Array header the
+    # slot points at. Measured as an order of magnitude rather than counted.
+    #
+    # @api private
+    FEATURE_BYTES = T.let(120, Integer)
+
+    # One indexed name: an Entry and the Normalizer::Form it holds, both small
+    # objects of references. The Entity and Name they point at belong to the
+    # snapshot and are not counted here.
+    #
+    # @api private
+    ENTRY_BYTES = T.let(200, Integer)
+
     # Entries, in the order they were indexed. The posting lists hold
     # positions in this array.
     sig { returns(T::Array[Entry]).checked(:tests) }
@@ -247,6 +269,45 @@ module ActiveSanction
       }
     end
 
+    # How big this index is, as the `index.build` event reports it: names,
+    # distinct features, postings, and an estimate of what it all weighs.
+    #
+    # Deliberately not #stats, and the difference is the one number #stats
+    # carries that this does not. Counting distinct entities means walking
+    # every entry and uniquing 47,000 ids, which is a fine thing to do in a
+    # benchmark and not a thing to do at the end of every index build -- it
+    # allocates at exactly the moment the heap is already at its peak, with
+    # the snapshot, the builder and the finished index all still alive. The
+    # event reports entity count from the snapshots it read instead, where it
+    # is already known.
+    #
+    # **`bytes` is an estimate, and deliberately a crude one.** Ruby offers no
+    # way to ask what an object graph weighs that does not either walk every
+    # object in the heap or lie, and an index is tens of millions of small
+    # objects. What it counts is the three things that dominate: a posting is
+    # an Integer in an Array slot, a distinct feature is a String plus its
+    # slot in a Hash, and an entry is a small object holding references. What
+    # it deliberately does not count is the Entities and Names themselves --
+    # those are the snapshot's, shared with it rather than owned here, and
+    # counting them would report the same megabytes twice to a host watching
+    # both.
+    #
+    # Good to within a factor a dashboard cares about, which is what "is the
+    # index growing?" and "will another list fit?" actually need. Anything
+    # finer wants a real heap profiler.
+    sig { returns(T::Hash[Symbol, Integer]).checked(:tests) }
+    def profile
+      keys = @tokens.size + @trigrams.size + @phonetics.size
+      # `each_value` rather than a two-parameter block over the Hash: iterating
+      # a Hash as pairs allocates a two-element Array per distinct feature,
+      # which is twenty thousand short-lived objects at the exact moment the
+      # heap is at its peak -- the snapshot, the builder and the finished
+      # index all still alive. #stats can afford that and a build cannot.
+      postings = [@tokens, @trigrams, @phonetics].sum { |space| space.each_value.sum(&:size) }
+      { names: entries.size, keys: keys, postings: postings,
+        bytes: (postings * POSTING_BYTES) + (keys * FEATURE_BYTES) + (entries.size * ENTRY_BYTES) }
+    end
+
     sig { returns(String) }
     def inspect = "#<#{self.class} #{size} names>"