probabilistic_machines 0.1.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 8c5cfb3b4e94463e5c61a5a8181c1bd713d0135b87839176d55b1a4ac9ed1fe9
+  data.tar.gz: a086f4a5d9cc1505c740bd4fb7be35b84283fae0fbbc5888e04ad44d0333ced9
+SHA512:
+  metadata.gz: 2619d85be86268533cc95d101d332ceb71e50139a24487d1fecb9f281d501a511fda27bdb9191e865b5a1dae265faa80f2e632bb5b1a5586a9817f7a817d85e2
+  data.tar.gz: 73164970f1cfa50eb8a1a976eddff1e053865af8f122393bcdd4b5cf8db133c1360ea03b09407d457a8bf245bf85685206962c70821f3e63e5d6493088a7b943

data/CHANGELOG.md

@@ -0,0 +1,58 @@
+# Changelog
+
+## [0.1.1](https://github.com/seuros/probabilistic_machines/compare/probabilistic_machines-v0.1.0...probabilistic_machines/v0.1.1) (2026-11-01)
+
+
+### Features
+
+* HeavyHitters codec and an infallible exact-size encode ([1cf9442](https://github.com/seuros/probabilistic_machines/commit/1cf944208e191f1cc7847221239204e14f0ad9a9))
+* probabilistic data structures in no_std Rust ([e8c9711](https://github.com/seuros/probabilistic_machines/commit/e8c97117a19e39ecfb2eff588724d9bd3d33be48))
+* Ruby gem and magnus FFI for the five structures ([83144bd](https://github.com/seuros/probabilistic_machines/commit/83144bd4553f2ea51fd8437ab966142b50122cec))
+
+
+### Bug Fixes
+
+* a loaded t-digest honours the compression its constructor requires ([dae877e](https://github.com/seuros/probabilistic_machines/commit/dae877eb1161dfc16f5238329c8c6fdd4b1c78e5))
+* a source install can build, and a failed gem push fails the release ([4b18e42](https://github.com/seuros/probabilistic_machines/commit/4b18e42a9b4c7d6264cc3cc9f685e0cb597de409))
+* a zero cluster budget spun the t-digest fold loop forever ([8f4f146](https://github.com/seuros/probabilistic_machines/commit/8f4f1466ca579a943d8a23aec4075b53314e3824))
+* expose and persist count-min saturation ([0ac83ac](https://github.com/seuros/probabilistic_machines/commit/0ac83ac77d02ddee5d7614d73e95e5d103e823e8))
+* fold t-digest clusters without overflowing their span ([2e33539](https://github.com/seuros/probabilistic_machines/commit/2e33539b0dbcc0eb8d288eedc936c0ece8d7f7bd))
+* interpolate a subnormal t-digest span without underflowing it ([1b05f4e](https://github.com/seuros/probabilistic_machines/commit/1b05f4e5515fa4468730bec3708daf8f57052726))
+* interpolate the t-digest cdf without overflowing the span ([48cd8a1](https://github.com/seuros/probabilistic_machines/commit/48cd8a11f1b2fdd56403af46614fd6a56b8b8309))
+* keep t-digest total weight finite and the cluster count bounded ([0a37d4b](https://github.com/seuros/probabilistic_machines/commit/0a37d4b52cd5a0abd7de862fe0f52e4f987c956e))
+* refuse a fractional compression too ([379c001](https://github.com/seuros/probabilistic_machines/commit/379c0012d2e5c21131ec8b9fb92534e796997be9))
+* refuse a fractional count instead of truncating it to zero ([0b88ae8](https://github.com/seuros/probabilistic_machines/commit/0b88ae8204ebbb761e8aab42df3e7d014501d135))
+* reject a compression a t-digest cannot honour ([f53e07b](https://github.com/seuros/probabilistic_machines/commit/f53e07b3af5377565253e14c7327b90c9975eb95))
+* reject a hyperloglog rank its precision cannot reach ([c41210b](https://github.com/seuros/probabilistic_machines/commit/c41210b66ccfc305240244fcf151990ea317392e))
+* reject a t-digest frame that lost whole clusters ([2e7e0b5](https://github.com/seuros/probabilistic_machines/commit/2e7e0b5594ff14297d98cb5c32380beed1f8192a))
+* reject a t-digest frame whose total weight cannot exist ([4bdf21f](https://github.com/seuros/probabilistic_machines/commit/4bdf21fa69949159aa30a8d400a8c7d11e64fbf1))
+* reject count-min counters its total cannot explain ([4413fe9](https://github.com/seuros/probabilistic_machines/commit/4413fe965cfdad617f46d9f2f2720e13a3aee6d0))
+* reject heavy-hitter records no stream could leave ([3a4172e](https://github.com/seuros/probabilistic_machines/commit/3a4172e730fd7c46b06d39bde2ef2c6034be19d4))
+* ship the core's test files so the packaged crate builds ([b567b39](https://github.com/seuros/probabilistic_machines/commit/b567b39144fd403f92efcb7ac48120be228d3b6f))
+* t-digest overflow and NaN, and report count-min saturation ([f3aa87c](https://github.com/seuros/probabilistic_machines/commit/f3aa87c018465de0618125b3f2f84e6848e57f37))
+* the t-digest arithmetic builds without std ([8009555](https://github.com/seuros/probabilistic_machines/commit/800955549c872ff7db8d257fbe63b18d326c1aa2))
+* validate HLL registers on decode, and the same hash collision ([496c077](https://github.com/seuros/probabilistic_machines/commit/496c0771f9a2c06766c1296c7123ea3b6bd7f4ae))
+* widen heavy-hitter bounds for what a merged table evicted ([41b0a9c](https://github.com/seuros/probabilistic_machines/commit/41b0a9c33f7433b915f448820628604c107d1cad))
+
+## 0.1.0
+
+- Initial release.
+- `ProbabilisticMachines::HyperLogLog`: distinct counts in 1024, 4096, 16384 or
+  65536 registers, with `merge` for union and `standard_error` for the bound.
+- `ProbabilisticMachines::CountMinSketch`: per-item frequencies in a width of
+  1024, 4096 or 16384 by a depth of 4 or 8.
+- `ProbabilisticMachines::TDigest`: quantiles and CDF from clusters, with a
+  runtime compression.
+- `ProbabilisticMachines::CuckooFilter`: membership that can forget, in 1024,
+  4096, 16384 or 65536 buckets, raising `ProbabilisticMachines::Full` when a
+  fingerprint has nowhere to go.
+- `ProbabilisticMachines::HeavyHitters`: Space-Saving top-k over 16, 64 or 256
+  counters, returning `Hitter` objects that carry the overestimate.
+- `dump`/`load` round-trip every sketch through the byte format of the
+  `probabilistic-machines` crate. A `HeavyHitters` blob is the crate's frame
+  followed by the extension's key table, so the tracked strings survive a
+  restart instead of loading back as hashes.
+- RBS signatures for the whole public API.
+- The `probabilistic-machines` crate codes `HeavyHitters<u64, K>`, and with
+  `alloc` hands every structure's frame back from `encoded` instead of filling
+  a caller's buffer, with `decode_into` to read one back in place.

data/Cargo.lock

@@ -0,0 +1,294 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 4
+
+[[package]]
+name = "aho-corasick"
+version = "1.1.5"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c982642fa9e8606056828ee9a8505737230110bb1099153c79efe865c59d12ba"
+dependencies = [
+ "memchr",
+]
+
+[[package]]
+name = "bindgen"
+version = "0.72.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "993776b509cfb49c750f11b8f07a46fa23e0a1386ffc01fb1e7d343efc387895"
+dependencies = [
+ "bitflags",
+ "cexpr",
+ "clang-sys",
+ "itertools",
+ "proc-macro2",
+ "quote",
+ "regex",
+ "rustc-hash",
+ "shlex",
+ "syn",
+]
+
+[[package]]
+name = "bitflags"
+version = "2.13.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3ded4057c258ba199e2d26386d3af3780957ecaee6c4ef4041c6b4b8b97c0b06"
+
+[[package]]
+name = "bytes"
+version = "1.12.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04"
+
+[[package]]
+name = "cexpr"
+version = "0.6.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "6fac387a98bb7c37292057cffc56d62ecb629900026402633ae9160df93a8766"
+dependencies = [
+ "nom",
+]
+
+[[package]]
+name = "cfg-if"
+version = "1.0.5"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "4e7648175b45a9a48536d676f68d918270699102aa8dab5496df06904c914600"
+
+[[package]]
+name = "clang-sys"
+version = "1.9.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "157a8ba7b480713b56f4c09fd13fc3e0a22a5dfab8097ba61cbc5feef950788a"
+dependencies = [
+ "glob",
+ "libc",
+ "libloading",
+]
+
+[[package]]
+name = "either"
+version = "1.19.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0e9c71c2167ca323c882b99918929403426e2373ea17242ff5653e0d5e1058be"
+
+[[package]]
+name = "glob"
+version = "0.3.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "e4eba85ea1d0a966a983acd07deee566e67395d2d96b6fb39e62b5a833f1eb0b"
+
+[[package]]
+name = "itertools"
+version = "0.13.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "413ee7dfc52ee1a4949ceeb7dbc8a33f2d6c088194d9f922fb8318faf1f01186"
+dependencies = [
+ "either",
+]
+
+[[package]]
+name = "lazy_static"
+version = "1.5.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "20870f649af7073d53e38067b2a84312175d56ea15217e1b15bc83506ec50afb"
+
+[[package]]
+name = "libc"
+version = "0.2.191"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "65cd87c9a6d60536580b316ad985726104730a5151c99845f2490428ef91240a"
+
+[[package]]
+name = "libloading"
+version = "0.8.9"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d7c4b02199fee7c5d21a5ae7d8cfa79a6ef5bb2fc834d6e9058e89c825efdc55"
+dependencies = [
+ "cfg-if",
+ "windows-link",
+]
+
+[[package]]
+name = "magnus"
+version = "0.9.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "40ef424404d166cbb71c8b8d19c9c589c748c4547c1afa1928c5504299f6c713"
+dependencies = [
+ "bytes",
+ "magnus-macros",
+ "rb-sys",
+ "rb-sys-env",
+ "seq-macro",
+]
+
+[[package]]
+name = "magnus-macros"
+version = "0.9.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "9dafde2eeb66f090e0f326112323eb8324b12cb32b5d1210e541d3df6b0ce77e"
+dependencies = [
+ "proc-macro2",
+ "quote",
+ "syn",
+]
+
+[[package]]
+name = "memchr"
+version = "2.8.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98"
+
+[[package]]
+name = "minimal-lexical"
+version = "0.2.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
+
+[[package]]
+name = "nom"
+version = "7.1.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a"
+dependencies = [
+ "memchr",
+ "minimal-lexical",
+]
+
+[[package]]
+name = "probabilistic-machines"
+version = "0.2.0"
+
+[[package]]
+name = "probabilistic_machines_native"
+version = "0.1.1"
+dependencies = [
+ "bytes",
+ "magnus",
+ "probabilistic-machines",
+]
+
+[[package]]
+name = "proc-macro2"
+version = "1.0.107"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9"
+dependencies = [
+ "unicode-ident",
+]
+
+[[package]]
+name = "quote"
+version = "1.0.47"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001"
+dependencies = [
+ "proc-macro2",
+]
+
+[[package]]
+name = "rb-sys"
+version = "0.9.130"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "02faf625bb10ba893e3ae620f19c9fb1b5f8fcae0fe4eb86bb3f2230fad75edb"
+dependencies = [
+ "rb-sys-build",
+]
+
+[[package]]
+name = "rb-sys-build"
+version = "0.9.130"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "05be6f9c86fe5482808162826f6c047d093b3670582296c770de1d94f8066694"
+dependencies = [
+ "bindgen",
+ "lazy_static",
+ "proc-macro2",
+ "quote",
+ "regex",
+ "shell-words",
+ "syn",
+]
+
+[[package]]
+name = "rb-sys-env"
+version = "0.2.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "cca7ad6a7e21e72151d56fe2495a259b5670e204c3adac41ee7ef676ea08117a"
+
+[[package]]
+name = "regex"
+version = "1.13.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f020237b6c8eed93db2e2cb53c00c60a8e1bc73da7d073199a1180401450218d"
+dependencies = [
+ "aho-corasick",
+ "memchr",
+ "regex-automata",
+ "regex-syntax",
+]
+
+[[package]]
+name = "regex-automata"
+version = "0.4.18"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ad8553b9b26413251cbf30e620595c7a41b3887f03da04579c0e6b0d6a06b4b2"
+dependencies = [
+ "aho-corasick",
+ "memchr",
+ "regex-syntax",
+]
+
+[[package]]
+name = "regex-syntax"
+version = "0.8.11"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
+
+[[package]]
+name = "rustc-hash"
+version = "2.1.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "6b1e7f9a428571be2dc5bc0505c13fb6bf936822b894ec87abf8a08a4e51742d"
+
+[[package]]
+name = "seq-macro"
+version = "0.3.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "1bc711410fbe7399f390ca1c3b60ad0f53f80e95c5eb935e52268a0e2cd49acc"
+
+[[package]]
+name = "shell-words"
+version = "1.1.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "dc6fe69c597f9c37bfeeeeeb33da3530379845f10be461a66d16d03eca2ded77"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "syn"
+version = "2.0.119"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297"
+dependencies = [
+ "proc-macro2",
+ "quote",
+ "unicode-ident",
+]
+
+[[package]]
+name = "unicode-ident"
+version = "1.0.27"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a2c754d6c33795a1c324727428e5a7dedb5b06195f9890bdbcba760d3e246563"
+
+[[package]]
+name = "windows-link"
+version = "0.2.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"

data/Cargo.toml

@@ -0,0 +1,24 @@
+[workspace]
+members = ["ext/probabilistic_machines_native/core", "ext/probabilistic_machines_native/ffi"]
+resolver = "3"
+
+[workspace.package]
+edition = "2024"
+rust-version = "1.99"
+
+[workspace.dependencies]
+probabilistic-machines = { path = "ext/probabilistic_machines_native/core" }
+
+[workspace.lints.rust]
+unsafe_code = "forbid"
+unused_must_use = "warn"
+
+[workspace.lints.clippy]
+all = { level = "warn", priority = -1 }
+pedantic = { level = "warn", priority = -1 }
+nursery = { level = "warn", priority = -1 }
+expect_used = "deny"
+unwrap_used = "deny"
+uninlined_format_args = "deny"
+needless_borrow = "deny"
+redundant_clone = "deny"

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2025 Abdelkader Boudih
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,203 @@
+# ProbabilisticMachines 🔭
+
+> **Survey the whole stream on a fixed fuel budget.** The tender that answers how
+> many, how often and how extreme without carrying the cargo it measured.
+
+Distinct counts, quantiles and frequencies in memory you pick up front.
+
+Counting exactly means keeping every key: a `Set` of user ids, a `Hash` of path
+counts, an array of latencies to sort. These five structures answer the same
+questions from state that was sized at construction time and never grows.
+**AMS Flajolet** takes a reading and discards the sample, which is why its hold
+is the same size on the way home as it was at launch.
+
+---
+
+## 🚀 Launch Sequence
+
+```ruby
+require 'probabilistic_machines'
+
+visitors = ProbabilisticMachines::HyperLogLog.new(registers: 4096)   # 4 KiB
+50_000.times { |id| visitors.add("user-#{id % 12_000}") }
+
+visitors.estimate       # => 12_361.2
+visitors.standard_error # => 0.01625
+```
+
+---
+
+## 🔩 The Native Extension Is Required
+
+There is no pure-Ruby fallback and no `DISABLE_*` switch. HyperLogLog, Count-Min
+Sketch, t-digest, cuckoo hashing and Space-Saving are five algorithms, and a
+second implementation would mean maintaining each of them twice, with two sets
+of accuracy tests and two byte formats to keep in step. The gem is a binding to
+the `probabilistic-machines` crate in `ext/probabilistic_machines_native/core`,
+and it needs Ruby 4.0 and a Rust toolchain (or a platform gem).
+
+---
+
+## 📐 Sizes Are a Fixed Menu
+
+The crate sizes every structure by a const generic, so each size is a separately
+compiled type. The extension carries these and raises `ArgumentError` for
+anything else:
+
+| Class            | Keyword              | Values                        |
+| ---------------- | -------------------- | ----------------------------- |
+| `HyperLogLog`    | `registers:`         | 1024, 4096, 16384, 65536      |
+| `CountMinSketch` | `width:` by `depth:` | 1024, 4096 by 4 or 8          |
+| `CuckooFilter`   | `buckets:`           | 1024, 4096, 16384, 65536      |
+| `HeavyHitters`   | `k:`                 | 16, 64, 256                   |
+| `TDigest`        | `compression:`       | any, it is a runtime argument |
+
+```ruby
+ProbabilisticMachines::HyperLogLog.new(registers: 3000)
+# ArgumentError: registers 3000 is not supported, expected one of [1024, 4096, 16384, 65536]
+```
+
+---
+
+## 🌌 How Many Distinct
+
+```ruby
+hll = ProbabilisticMachines::HyperLogLog.new(registers: 4096)
+hll.add('user-1')
+hll.add_hash(0x9e37_79b9_7f4a_7c15) # for streams that ship hashes, not keys
+hll.estimate                        # Float
+hll.standard_error                  # 1.04 / sqrt(registers)
+hll.merge(other)                    # exact union
+```
+
+Memory is one byte per register, so 4096 registers cost 4 KiB whether the stream
+held ten ids or ten billion.
+
+---
+
+## 📊 How Often Each
+
+```ruby
+cms = ProbabilisticMachines::CountMinSketch.new(width: 4096, depth: 4)
+cms.add('/login', count: 3)
+cms.count('/login')  # never below the truth
+cms.total            # every count added
+cms.merge(other)     # exact sum
+```
+
+A count is over the truth by at most `e * total / width`, with probability
+`1 - e ** -depth` of staying inside it. Width buys accuracy, depth buys
+confidence.
+
+---
+
+## 📏 Quantiles
+
+```ruby
+td = ProbabilisticMachines::TDigest.new(compression: 100)
+td.add(12.5)
+td.add(12.5, weight: 1_000.0) # one point standing for a thousand observations
+td.quantile(0.99)
+td.cdf(12.5)
+td.centroids                  # Array of frozen Centroid(mean:, weight:)
+td.minimum                    # exact, as is maximum
+td.merge(other)
+```
+
+A quantile is off by about `1 / compression` of the distribution in the middle
+and is exact at the extremes, in `compression * ln(n)` clusters rather than `n`
+samples. An empty digest answers `Float::NAN`.
+
+---
+
+## 🛰️ Have I Seen This Contact Before
+
+```ruby
+cf = ProbabilisticMachines::CuckooFilter.new(buckets: 4096)
+cf.insert('device-9')      # raises ProbabilisticMachines::Full when wedged
+cf.include?('device-9')    # contains? is the same question
+cf.remove('device-9')      # unlike a Bloom filter, it can forget
+```
+
+`buckets * 4` items in `buckets * 4` bytes. Never a false negative; about 3%
+false positives. Only remove items you inserted, or you take a colliding item's
+fingerprint with it. A cuckoo filter cannot merge, so there is no `merge` here.
+
+---
+
+## 👑 Who Dominates the Stream
+
+```ruby
+hh = ProbabilisticMachines::HeavyHitters.new(k: 64)
+hh.add('10.0.0.1')
+hh.add('10.0.0.1', count: 5)
+hh.top(10)   # [Hitter(key:, count:, overestimate:), ...] most frequent first
+hh.ranked    # every tracked key
+hh.observed  # observations fed in
+hh.merge(other)
+```
+
+After `n` observations every key seen more than `n / k` times is tracked, in `k`
+counters and no heap per key. A counter can inherit the count of the key it
+evicted, or what a merged-in table's own evictions may have hidden, so the true
+count lies between `count - overestimate` and `count`.
+
+Keys are UTF-8 strings here: Space-Saving needs `Copy + Eq` keys in Rust, so the
+table counts 64-bit key hashes and the extension keeps the string each tracked
+hash came from, to hand back from `top` and `ranked`. The other four classes
+take any byte string.
+
+---
+
+## 📦 Transmitting a Reading
+
+`dump` returns the crate's own byte frame, `load` reads one back. The frame
+names its own shape, so `load` needs no keyword and rejects a blob written with
+a size this build does not carry. It also reads the frame's fields against each
+other and refuses a state no stream could have left behind, so an edited blob
+raises instead of handing back a sketch that counts more than it ever saw.
+
+```ruby
+blob = hll.dump                                 # binary String
+ProbabilisticMachines::HyperLogLog.load(blob)
+ProbabilisticMachines::TDigest.load(blob)       # ArgumentError: codec: wrong structure tag
+```
+
+Anything that reads the crate reads these frames, so an embedded device can ship
+a sketch a Rails app folds in. `HeavyHitters` is the one blob that is not purely
+the crate's: the crate's frame counts 64-bit key hashes, so the extension appends
+its key table after it, one record of a 64-bit byte length and that many UTF-8
+bytes per tracked key. `load` rehashes each name, and rejects a blob whose key
+table is malformed or does not name every key the frame tracks.
+
+```ruby
+restored = ProbabilisticMachines::HeavyHitters.load(hh.dump)
+restored.top(1).first.key # => '10.0.0.1', the string, not its hash
+```
+
+---
+
+## ⚙️ The Rust Core
+
+The sketches themselves are the `probabilistic-machines` crate under
+`ext/probabilistic_machines_native/core`: `no_std`, zero dependencies, allocating
+only for `TDigest` and for the `encoded` frames this extension dumps, and
+verified on bare metal. The binding crate beside it under
+`ffi/` is the Ruby surface and is not published.
+
+---
+
+## 📡 Credits
+
+**AMS Flajolet**, NCC-0185, *Flajolet*-class tender.
+
+Named for Philippe Flajolet and G. Nigel Martin's *Probabilistic Counting
+Algorithms for Data Base Applications* (1985), and for Flajolet, Fusy, Gandouet
+and Meunier's *HyperLogLog* (2007), which established that you can count what
+you refuse to store.
+
+---
+
+## License
+
+MIT.

data/ext/probabilistic_machines_native/core/Cargo.toml

@@ -0,0 +1,24 @@
+[package]
+name = "probabilistic-machines"
+version = "0.2.0"
+edition.workspace = true
+rust-version.workspace = true
+authors = ["Abdelkader Boudih <oss@seuros.com>"]
+license = "MIT"
+description = "Sketches for distinct counts, quantiles and frequencies - no_std, zero dependencies"
+repository = "https://github.com/seuros/probabilistic_machines"
+keywords = ["hyperloglog", "t-digest", "count-min", "sketch", "no-std"]
+categories = ["no-std", "algorithms", "data-structures"]
+readme = "README.md"
+
+[lib]
+name = "probabilistic_machines"
+crate-type = ["lib"]
+
+[features]
+default = ["std"]
+std = ["alloc"]
+alloc = []
+
+[lints]
+workspace = true