albex 0.3.0 → 0.6.1

package/CHANGELOG.md

@@ -5,6 +5,472 @@ All notable changes to Albex are documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and Albex follows [Semantic Versioning](https://semver.org/).
 
+## [Unreleased]
+
+### Changed
+
+- **IDF-aware relevance scoring — exact matches no longer saturate.** The old
+  `rich_score` set `base = 1000 − avg_errors·200`, so every 0-error hit hit the
+  1000 ceiling and `.min(1000)` discarded *all* the relevance bonuses (term
+  frequency, proximity, location, word-boundary AND idf). The practical effect:
+  on a real corpus every exact match tied at 1000 and the final ordering among
+  them was arbitrary — the chunk holding the rare, decisive query term was no
+  more likely to rank first than dozens that merely shared a common word.
+  Scores are now laid out as **error tiers with relevance ranking inside each
+  tier** (0 errors → `[750,1000]`, 1 → `[500,750]`, 2 → `[250,500]`, 3 →
+  `[0,250]`): a cleaner match still always beats a noisier one, but within a
+  tier the **IDF bonus (now the largest, up to 150)** lifts rare-term matches
+  decisively above common-term ones. Measured downstream (the Noesis hybrid-RAG
+  eval over a 293-doc / 12.4k-chunk corpus): lexical-leg MRR **0.19 → 0.40**,
+  R@5 **23% → 53%**, and end-to-end hybrid MRR **0.42 → 0.53** — with no change
+  on small corpora. `SearchResult.score` keeps its `0..=1000` range, so
+  `setThreshold` is unaffected for typical values; very weak 3-error fuzzy
+  matches (bonus-starved, score < the default threshold 250) are now filtered
+  where they previously slipped through.
+- **Fixed a latent result-ordering bug** exposed by the scoring change. The
+  heap-sort already left `RESULTS` in descending order; a trailing reverse then
+  flipped it to *ascending*. It was invisible while all exact scores tied at
+  1000 and was masked from API consumers by the TypeScript-side re-sort, but the
+  WASM ABI now returns correctly descending results on its own.
+
+### Added
+
+- **Dynamic capacity (ABI 7, decision A16 executed).** The compile-time
+  capacity tiers are gone for good: the engine pools (chunk table, trigram
+  signatures, text pool, doc table, content hashes, name pool, tombstone
+  bitset, GPU candidate mask) moved from BSS `static mut` arrays to heap
+  allocations sized at runtime by the new export
+  `initWithCapacity(maxDocs, maxChunks, textPoolBytes, namePoolBytes) → 1|0`.
+  - `init()` is now a wrapper over `initWithCapacity` with the historical
+    std defaults (128 docs · 100k chunks · 16 MB text · 32 KB names) —
+    default behaviour is identical to every previous release.
+  - Hard ceilings (validated, return 0 — never trap): 65 536 docs, 4 M
+    chunks, 1 GiB text, 16 MiB names; allocation failure (memory.grow
+    refused) also returns 0 cleanly with the engine in a safe empty state.
+  - Re-init with the same capacities is a plain reset (no realloc);
+    different capacities free + re-allocate (no leak; the linear-memory
+    high-water mark stays, as WASM memory never shrinks).
+  - **Snapshots are admitted by CONTENT, not by writer capacity**: the
+    restore gate compares the snapshot's counters against the live runtime
+    capacities, so a snapshot saved by a `'large'` engine loads into a
+    `'std'` engine whenever its contents fit — and fails CLEANLY (previous
+    index intact) when they don't. v4 headers additionally record the
+    writer's capacities in the previously-reserved bytes [40..56], for
+    diagnostics only.
+  - **TS option `capacity`**: `'std'` (default) · `'large'`
+    (1 024 docs / 800k chunks / 128 MB text / 256 KB names — the old "pro"
+    tier) · custom `{ maxDocs?, maxChunks?, textPoolBytes?, namePoolBytes? }`
+    (partials completed from std scaled by std's ratios: chunks = docs×782,
+    text = chunks×168 B, names = docs×256 B, with floors; explicit
+    maxChunks below maxDocs is clamped up). Forwarded to workers and pool
+    shards (structured-clone-safe). `reset()` re-inits with the CONFIGURED
+    capacity, not the std defaults.
+  - `AlbexCapacityError` gains `max` — the runtime numeric limit of the
+    pool named by `limit` (e.g. `4` for `capacity: { maxDocs: 4 }`); it
+    survives the worker boundary.
+  - New large-scale bench (`bench/large.bench.ts`): 1 000 docs / 200k
+    chunks / 15.8 MB text in a `'large'` engine — search 141-769 ms
+    (rare-token → fuzzy), snapshot save 24 ms / restore 175 ms for a
+    ~20 MB snapshot. Numbers in `bench/README.md`. Binary cost of heap
+    pools: baseline 43 396 → 47 446 B (gate 48 KB still green).
+
+- **Batch frontier reads (ABI 6).** Three groups of exports collapse the
+  JS↔WASM call count on the hot read paths:
+  - `getResultsPtr()` + `getResultStride()`: `_collectResults` now reads every
+    numeric field of all results with ONE `DataView` pass over the
+    `#[repr(C)]` `RESULTS` array (layout documented in the export's
+    doc-comment and compile-time asserted with `offset_of!`) instead of
+    ~12-15 frontier calls per result. Doc names are resolved once per
+    distinct document via its table slot — the per-result
+    `getResultDocName` (O(doc_count) inside WASM) is no longer called.
+  - `getPatternBloomLo()/Hi()`: the query Bloom the GPU pre-filter needs is
+    now computed in `setPattern` through the exact pipeline `searchBegin`
+    uses (split → optional Spanish stemming → fold). The third TypeScript
+    copy of the fold (`computePatternBloom`) is deleted; with stemming on it
+    silently diverged from the CPU pattern.
+  - `listChunksBatch(slot, startOrd, maxChunks)`: packs consecutive chunk
+    records `[u32 text_len][u32 location][text]` into the scratchpad;
+    `listChunks` now makes ~1 frontier call per 64 KB batch instead of 2-3
+    per chunk (an embeddings pipeline over 100k chunks drops from ~300k
+    calls to ~1.3k).
+  `abiVersion()` bumped 5 → 6; host pin updated. Snapshot format unchanged.
+- **UTF-16 spans: `SearchResult.snippetStart/snippetEnd`.** The existing
+  `matchStart/matchEnd`/`matches[]` are UTF-8 BYTE offsets (now documented as
+  such) and mis-highlight as soon as the snippet contains accents. The new
+  fields are the primary span as UTF-16 code-unit indices over the decoded
+  snippet — safe for `snippet.slice()`. Byte fields are unchanged for
+  backwards compatibility.
+- **Worker parity: `replaceDocument` + `takeDiagnostics`** added to the
+  worker protocol and `AlbexEngineWorker`. The class JSDoc now documents that
+  `attachOcr` cannot exist in the worker (functions don't cross postMessage)
+  and that scanned PDFs index with 0 chunks there — with the diagnostic
+  explaining why readable through `takeDiagnostics`, and the recommendation
+  to OCR on the main-thread engine and `save()`/`load()` the snapshot.
+- **CI with teeth:** `cargo clippy -D warnings` for every crate (host pass
+  for core/ingest, wasm32 pass for the wasm and pdf crates), an advisory
+  `cargo fmt --check`, a hard 48 KB size gate on the baseline `.wasm`, a
+  build of the SIMD variant, and a manual-only (`workflow_dispatch`) bench
+  job with no thresholds.
+
+### Removed
+
+- **`tier` option and tier plumbing (breaking, pre-1.0).** The deprecated
+  `AlbexOptions.tier` (ignored since 0.5.0), `AlbexEngine.tier`,
+  `AlbexPool.tier`, `EngineStats.tier` and the WASM `getTier` export are
+  gone — superseded by the runtime `capacity` option (ABI 7, decision
+  A16). `EngineStats` now reports the real runtime capacities (`maxDocs`,
+  `maxChunks`, `textCapacity`, plus new `namePoolBytes`). The deprecated
+  `pickTier`/`Tier` re-exports from `profile.ts` remain for source
+  compatibility. The `tier-mini`/`tier-std`/`tier-pro` Cargo features are
+  deleted; the only build-time switch left is `simd`.
+
+### Changed
+
+- **Trigram windows no longer cross word boundaries.** `build_sig` /
+  `token_trigram_bits` only extend the sliding window on code points that
+  fold to an ASCII letter or digit; whitespace/punctuation/symbols break it
+  on BOTH the index and query sides (previously the fold passed ASCII
+  whitespace through, so signatures contained cross-word trigrams that
+  diluted selectivity — the doc-comment claimed otherwise). Sound under
+  fuzzy matching (same ≤3-windows-per-edit bound); signatures are rebuilt
+  on every restore/compact, so existing snapshots are unaffected.
+- **`wee_alloc` replaced (RUSTSEC-2022-0054).** The main no_std module now
+  uses `dlmalloc` (`global` feature) — baseline binary 36 732 → 42 654 bytes
+  (+5.8 KB, under the 48 KB gate). `pdf-wasm` drops its custom allocator and
+  uses std's default (also dlmalloc): 1 193 074 → 1 199 910 bytes (+6.8 KB on
+  1.2 MB). In pdf-wasm the allocator serves all of lopdf, so the known
+  wee_alloc leak surface was real there.
+
+### Fixed (this pass)
+
+- **`getSnippetWindow` can no longer open or close mid code point.** When the
+  word-boundary snap found no space within 20 bytes, the window edge could
+  land on a UTF-8 continuation byte → U+FFFD at the snippet border and every
+  span visually off. Both edges now back off continuation bytes to the
+  code-point lead (same pattern as the chunker's hard cut).
+
+### Added (previous pass)
+
+- **Authoritative chunk enumeration (ABI 4).** New `AlbexEngine.listChunks(docId)`
+  (and `AlbexEngineWorker.listChunks`) returns the exact chunks Albex indexed for
+  a document — `{ docId, location, ord, sub, text, byteLen, id }` — so a host can
+  mirror Albex's chunking for a parallel index (e.g. embeddings) keyed on the
+  `compact()`-stable id `"<docId>::<ord>"`. Backed by four read-only
+  WASM exports (`getDocChunkBase`, `getChunkLocationAt`, `getChunkByteLenAt`,
+  `getChunkTextAt`); `abiVersion()` bumped 3 → 4. No change to search, scoring or
+  snapshot formats; v3 snapshots still load.
+- **`SearchResult` now carries `docId` and `chunkId`.** `chunkId` (`"<docId>::<ord>"`)
+  is identical to the matching `AuthoritativeChunk.id`, so a host can fuse search
+  hits with a parallel index on a single stable key — no `(name, location)`
+  heuristics, and unambiguous when a long paragraph splits into sub-chunks.
+- **`maxFileBytes` option (default 256 MiB).** `indexFile` now rejects oversized
+  inputs with a typed `AlbexCapacityError` (`limit: 'file'`) by checking
+  `File.size` BEFORE reading — a 2 GB file no longer gets fully buffered and
+  hashed just to fail later. Enforced by the engine, the worker wrapper and the
+  pool coordinator.
+
+### Fixed
+
+- **GPU pre-filter no longer corrupts the search pattern.** `setCandidateMask`
+  pushes the candidate bitset through the scratchpad, overwriting the pattern
+  staged by `selectQueryBranch`; `searchBegin` then compiled garbage tokens out
+  of the mask bytes, so every GPU-assisted cooperative search silently returned
+  wrong results. The active branch is now re-selected after the pre-filter.
+- **GPU bloom upload invalidation is now mutation-driven.** The upload used to
+  be skipped when the chunk count matched the last upload — but `compact()` can
+  reorder chunks while keeping the count identical, leaving the GPU mask
+  filtering the wrong chunks (silent false negatives). A dirty flag set by every
+  index mutation (index/remove/compact/reset/load) now forces the re-upload.
+- **`AlbexEngineWorker.init` forwards all engine options.** Previously only
+  `wasmUrl` and `pdfWasmUrl` crossed the worker boundary; `wasmBaseUrl`, `simd`,
+  `gpu`, `gpuThreshold` and `maxFileBytes` were silently dropped, making the
+  SIMD variant and the GPU policy unreachable in workers.
+- **OR-branch dedup keyed on `chunkId`.** The previous
+  `(doc, location, matchStart)` key collided when two sub-chunks of the same
+  location hit at the same relative offset, dropping a legitimate result.
+- **`AlbexPool.search` now actually caps and dedups after the merge.** The doc
+  promised "capped to setMaxResults AFTER merge" but the coordinator returned
+  the raw flattened buckets. Results are now deduped, sorted and capped to the
+  last `setMaxResults` value (default 50); per-shard search stats are captured
+  race-free in the same posting batch as the search itself.
+- Corrected the `listChunks` docstring (and the entry above): the canonical
+  chunk id format is `"<docId>::<ord>"`, not `"<docId>::<location>.<sub>"`.
+
+## [0.6.0] — 2026-05-31
+
+Release de auditoría algorítmica + robustez. Cierra los hallazgos #1–#8 de
+la revisión externa en profundidad: sube la selectividad del pre-filtro de
+forma drástica, elimina dos clases de corrupción/pérdida silenciosa de
+datos, y de paso corrige una falta de _soundness_ del filtro Bloom bajo
+matching difuso que existía desde el principio. El binario crece ~2 KB
+(33 KB baseline · 37 KB SIMD) por el código del pre-filtro de trigramas.
+
+### Breaking changes
+
+- **ABI WASM v2 → v3** (hallazgo #7). El binario principal exporta ahora
+  `getLastIndexOverflow` y el host fija el rango ABI aceptado a `[3, 3]`
+  (antes `[1, 2]`, con un mínimo inalcanzable que la lista de exports
+  requeridos ya hacía imposible). Un `.wasm` cacheado de 0.5.x falla con
+  `AlbexAbiMismatchError` claro en `init()` en vez de petar más tarde.
+
+### Added
+
+- **Pre-filtro de trigramas q-gram** (hallazgo #1). El Bloom de 64 bits
+  (un bit por `c & 0x3F`) está saturado en chunks de prosa de 512 bytes —
+  cada chunk contiene casi todo el alfabeto, así que no poda y el Bitap
+  acababa ejecutándose sobre todo el corpus. Se añade una firma de 256
+  bits de los **trigramas** de cada chunk (`CHUNK_SIG`, BSS, no infla el
+  `.wasm`) como segundo pre-filtro mucho más selectivo. Es **sound bajo
+  matching difuso**: una ocurrencia con `e` errores conserva ≥ `N − 3e`
+  de los trigramas exactos del token (lema q-gram), y la firma no tiene
+  falsos negativos, así que nunca descarta un match real — solo poda
+  chunks que demostrablemente no pueden contenerlo. El Bitap confirma
+  cada superviviente: la corrección no cambia, solo se encoge el conjunto
+  candidato.
+- **`AlbexCapacityError` con campo `limit`** (`'chunks' | 'text' | 'docs'
+  | 'names'`). El nuevo export `getLastIndexOverflow()` señaliza qué pool
+  se llenó durante el último `begin..endDocument`.
+
+### Fixed
+
+- **Pérdida silenciosa de datos al agotar capacidad** (hallazgo #3). Los
+  pools del WASM se llenaban en silencio y `indexFile` devolvía un
+  documento a medio indexar indistinguible de uno completo (`getStats()`
+  mentía). Ahora `indexFile` lee el overflow y lanza `AlbexCapacityError`.
+  El documento que desborda se **revierte atómicamente** en el WASM
+  (`endDocument` restaura `chunk_count`/`text_used`/`name_used` al inicio
+  del doc): indexación all-or-nothing por fichero, sin chunks huérfanos
+  sin nombre.
+- **Re-entrancy en el path async** (hallazgo #2). Una única instancia WASM
+  con estado global y ops async que ceden al scheduler entre slices: dos
+  operaciones solapadas se corrompían (un `searchBegin` nuevo reseteaba el
+  cursor de una búsqueda cooperativa en vuelo). Las ops async se serializan
+  ahora con una cola interna; los mutadores/búsquedas síncronos rechazan
+  ejecutarse a media operación (`AlbexError` kind `'busy'`) en vez de
+  corromper. El worker-runtime procesa mensajes estrictamente en orden.
+- **Bloom no-sound bajo fuzzy** (corolario del #1). El filtro Bloom de
+  caracteres rechazaba un match aproximado cuando el carácter sustituido
+  no estaba en el chunk (bug latente pre-0.6.0). Ahora el Bloom solo se
+  aplica a tokens exactos (`eff_errors == 0`); los tokens difusos se
+  filtran solo con el conteo de trigramas, que sí es sound.
+- **Frase + `windowed`** (hallazgo #7). El post-filtro de frase corría
+  contra el snippet recortado, así que `{ windowed: true }` podía descartar
+  un match válido cuyo segundo término caía fuera de la ventana. Ahora la
+  comprobación de adyacencia corre contra el **texto completo** del chunk;
+  el windowing solo afecta a la visualización.
+- **GPU + OR**: el hash del pre-filtro GPU usaba siempre el patrón de la
+  rama 0, generando una máscara de candidatos errónea para las ramas i≠0 de
+  una query OR y descartando sus hits en silencio (hallazgo #6).
+- **Corte de chunk a mitad de codepoint UTF-8** (hallazgo #7): el corte duro
+  a 512 bytes podía partir una secuencia multibyte; ahora retrocede a la
+  frontera de codepoint y el snippet del borde no renderiza `�`.
+- **`replaceDocument` sin reclamar espacio** (hallazgo #7): repetidos
+  replaces dejaban tombstones en el text pool; ahora se compacta de forma
+  oportunista bajo presión.
+
+### Performance
+
+- `prepareQuery` ya no hace un `memset` de 64 KB en pila por query (hallazgo
+  #4). La query de trabajo se acota a `MAX_QUERY_BYTES` (1 KB).
+
+### Tooling
+
+- `npm run relaunch`: limpia artefactos, recompila WASM (baseline + SIMD) +
+  PDF + TS + OCR, corre los tests, empaqueta la librería (`npm pack`) y
+  levanta el demo en `http://localhost:5173/demo/`. Scripts auxiliares
+  `clean` / `clean:all` / `build:ocr`.
+- Cobertura: +16 tests (110 en total) — selectividad del trigram, soundness
+  fuzzy, supervivencia de firmas tras compact/restore, error de capacidad,
+  guard de concurrencia, frase+windowed.
+
+## [0.5.0] — 2026-05-30
+
+Release de endurecimiento — cierra las cinco clases de bugs accionables
+de la auditoría externa de código. Sin nuevas features que pidan datos
+reales para ser correctas. El binario crece ~2 KB respecto a 0.4.0 por
+la lógica de query parsing en Rust + ABI version.
+
+### Breaking changes
+
+- **`@albex/ocr` ahora requiere `engine.attachOcr()`** (audit 3.8). El
+  patrón anterior de mutar `engine.ocrImage = ...` y
+  `engine.ocrConfig = ...` directamente queda eliminado. Para
+  integradores que usaban `@albex/ocr`, **el cambio es transparente** —
+  `enableOcr(engine)` sigue siendo la misma llamada. Para quien tuviera
+  un adaptador manual: usar `engine.attachOcr({ recognize, options })`.
+- **Tiers eliminados** (audit 4.1). Mini/std/pro × baseline/SIMD (6
+  binarios) consolidados a **baseline + SIMD** (2 binarios). El
+  parámetro `tier` de `AlbexOptions` queda como noop deprecado. El alias
+  `albex_wasm_bg.wasm` se mantiene para compatibilidad con `0.4.x`.
+- `pickTier(profile)` siempre devuelve `'std'` ahora. La función queda
+  exportada para compatibilidad de código fuente, no de comportamiento.
+
+### Fixed
+
+- **Validación runtime de la ABI WASM** (audit 3.2). `asAlbexExports` y
+  `asAlbexPdfExports` ya no son `as unknown as` ceremoniales — verifican
+  que `memory` sea una `WebAssembly.Memory`, que cada export requerido
+  exista, y que `abiVersion()` esté en el rango soportado. Lanzan
+  `AlbexAbiMismatchError` con la lista de exports faltantes cuando algo
+  no encaja. Antes, un binario incompatible instanciaba en silencio y
+  petaba en el primer call site que tocaba la función ausente.
+- **`makePdfWasmImports` falla rápido** ante imports desconocidos
+  (compatibilidad con el cambio anterior de 0.3.1; ya estaba pero ahora
+  alineado con la ABI version del módulo).
+
+### Added
+
+- **Canal de diagnósticos estructurado** (audit 3.6). Los `console.warn`
+  diseminados por el path de indexación desaparecen — los reemplaza un
+  buffer interno de `AlbexDiagnostic[]` consultable con
+  `engine.takeDiagnostics()`. Cada entrada es `{kind, stage, message, file?,
+  page?}` con `kind` en `'recovered' | 'skipped' | 'fallback' | 'info'`.
+  Cubre: fallback PDF→OCR, OCR fail por imagen, GPU caída a CPU, descarga
+  PDF WASM en red restringida. Capped a 256 entradas para no explotar en
+  corpus muy corruptos. La API de "best-effort" se mantiene, pero el
+  caller ahora puede inspeccionar qué se perdió.
+- **`engine.attachOcr(adapter)`** — extension point formal. Devuelve un
+  `OcrHandle` con `dispose()`. El motor valida el contrato y rechaza
+  un segundo `attachOcr` mientras haya uno activo. La propiedad pública
+  `engine.ocrImage` se mantiene como getter de feature-detect pero no es
+  asignable — para evitar el patrón anti-encapsulación del audit.
+- **`abiVersion()` exportada por ambos módulos WASM**. Main = v2 (incluye
+  query parser nuevo); PDF = v3 (incluye image extraction). El validador
+  TS rechaza binarios fuera de rango.
+
+### Architecture — query parsing moves to WASM (audit "two truths")
+
+Pre-0.5.0 el TypeScript dueño de `parseQuery`, `tokenize`,
+`tokensToWasmQuery`, mientras Rust tokenizaba al indexar. Dos verdades
+sobre qué era un "token".
+
+- Nuevos exports WASM: `prepareQuery`, `getQueryKind`,
+  `getQueryBranchCount`, `getQueryBranchPattern`, `selectQueryBranch`.
+- Hasta **8 branches OR** soportadas, **4 tokens por branch**, **256
+  bytes por pattern compilado** — todo en static BSS, sin alocación.
+- `containsPhrase` queda en TS porque opera sobre snippets (output del
+  WASM), no sobre la query — no es divergencia de tokenizer.
+- `parseQuery`, `tokenize`, `tokensToWasmQuery` eliminados del TS.
+- Un único algoritmo de "qué es un token" entre indexación y querying.
+
+### Build & maintenance
+
+- **`prepublishOnly` rebuildea WASM + corre tests** desde 0.3.1, ya
+  garantizado en 0.5.0.
+- Build pipeline simplificada: `scripts/build-wasm.mjs` produce solo
+  dos binarios. `npm pack --dry-run` muestra 4 archivos `.wasm` en lugar
+  de 8.
+- `wasm/Cargo.toml` añade `wee_alloc` (~1 KB) para el staging Vec del
+  restore atómico de 0.4.0.
+
+### Tests
+
+- 94 vitest cases verdes (era 88 en 0.4.0). Cinco tests del tier matrix
+  eliminados (mini/std/pro ya no existen). Tests nuevos:
+  - `tests/abi-validation.test.ts` (5): valida que `AlbexAbiMismatchError`
+    se lanza ante exports faltantes, abiVersion fuera de rango, memory
+    inválida.
+  - `tests/diagnostics.test.ts` (4): valida `takeDiagnostics()` drena,
+    cap a 256, reset limpia.
+
+### Postponed con razón
+
+Cosas del audit que NO se cierran en 0.5.0 porque cerrarlas sin datos
+reales sería adivinar:
+
+- **3.5 OCR paralelización**: optimización sin profiling no es ingeniería.
+- **3.9 Adaptive runtime con métricas reales**: requiere corpus y uso
+  reales para validar decisiones.
+- **4.3 GPU equivalence test**: requiere corpus >20k chunks que aún no
+  está checked in.
+- **7 parsers lite a WASM**: ~3 semanas serias. Separable. No es bug
+  fix, es mejora arquitectural más limpia con tiempo dedicado.
+
+## [0.4.0] — 2026-05-30
+
+Cierre de dos clases enteras de bugs identificadas por la auditoría externa
+de código. Sin cambios cosméticos — el binario crece ~4 KB por la lógica
+de atomicidad y el allocator necesario para el staging buffer.
+
+### Fixed — atomic snapshot restore (audit 3.4)
+
+- **Snapshot v3 con formato por campos**. Reemplaza la copia byte a byte
+  de los structs internos `Chunk`/`DocEntry` (`from_raw_parts`) por un
+  encoding explícito little-endian. El formato deja de depender del
+  layout en memoria de Rust, del target, del padding o de cambios en
+  los tipos. Lo que va al disco es un contrato.
+- **`restoreCommit()` — protocolo de 3 fases atómico**. El antiguo
+  `restoreBegin` reseteaba el estado y escribía los counters antes de
+  recibir un solo byte del payload. Si `restoreFeed` fallaba a mitad,
+  el corpus previo quedaba destruido. v3 acumula todo el payload en un
+  staging buffer y solo aplica al estado vivo cuando `restoreCommit`
+  valida que el tamaño completo coincide con el header. Un commit
+  fallido deja el motor con el corpus previo intacto.
+- **Compatibilidad backwards**. v1 y v2 siguen cargando — para ellos
+  `restoreBegin` mantiene la semántica vieja (no-atómica) y
+  `restoreCommit` es no-op que devuelve 1. El primer `save()` tras
+  cargar un snapshot viejo lo reescribe como v3.
+- Binarios crecen ~4 KB por la lógica nueva y por `wee_alloc` (única
+  fuente de alocación en el módulo, usada por el staging Vec).
+
+### Fixed — single source of truth for content hash (audit "two truths")
+
+- **FNV-1a 64-bit ahora vive en Rust**. La implementación TypeScript que
+  duplicaba el algoritmo desaparece. Tres nuevos exports
+  (`hashBegin`/`hashFeed`/`hashFinish`) implementan el hash en streaming
+  para archivos mayores que el scratchpad. El método privado del engine
+  `_contentHash` produce exactamente el mismo string hex de 16
+  caracteres que devolvía la versión TS — ningún caller cambia.
+
+### Added — tests
+
+- `tests/load-restores-docs.test.ts`: nuevo test "a v3 restore that
+  never commits leaves the previous index intact". Verifica
+  explícitamente la atomicidad: trunca el payload de un snapshot al
+  75 %, intenta cargarlo, verifica que `load()` devuelve `false` y que
+  el corpus previo sigue indexado y consultable.
+- `tests/hash.test.ts`: reescrito para validar el hash WASM contra el
+  engine real (la versión vieja era una re-implementación TS standalone
+  comparándose consigo misma). Cubre shape, determinismo, sensibilidad
+  a un byte, FNV offset basis, streaming sobre 96 KB (> scratchpad).
+- 88 tests verdes (era 85 en 0.3.1).
+
+### Postponed
+
+- Mover el tokenizador y query parser a WASM (audit "wrapper TS hace
+  demasiado") se traslada a 0.5.0. Es mejora arquitectural, no cierre
+  de bug — y tiene suficientes trade-offs de diseño (semánticas de OR,
+  post-filter de phrase) como para no publicar una API a medio cocer.
+
+## [0.3.1] — 2026-05-30
+
+Hardening pass after an external code audit. No new features; three
+specific issues addressed.
+
+### Fixed
+
+- **Debug logs removed from the indexing hot path.** Three `console.log`
+  statements added during the OCR-worker-abort diagnostic session were
+  firing on every PDF (hybrid OCR decision) and every embedded image
+  (kind / len / magic-byte trace). They are gone; the legitimate
+  `console.warn` messages for actual failures stay.
+
+- **`makePdfWasmImports` now fails fast on unknown imports.** Previously
+  any unrecognised import was satisfied with a `console.warn` stub,
+  which let the module instantiate and defer the real failure to an
+  arbitrary call inside `extractPdf`. The loader now throws
+  `AlbexInitError` at boot with a clear "rebuild your binary" message.
+  An unknown import means the wasm-bindgen / lopdf / getrandom graph
+  drifted from what this loader was written for; better to surface that
+  immediately than to hang or crash mid-extraction.
+
+- **`prepublishOnly` now rebuilds every WASM artifact and runs the
+  entire test suite.** It was running only `tsc + banner.mjs`, which
+  meant the WASM binaries published to npm could be out of sync with
+  the current Rust source. The script is now `npm run build:all && npm
+  test`. Publishing takes longer, but the package is guaranteed to
+  contain binaries reproducible from the source it ships.
+
 ## [0.3.0] — 2026-05-30
 
 ### Hybrid PDF OCR (opt-in)

package/README.md

@@ -69,7 +69,7 @@ That's the entire onboarding. Read on for what else the engine can do.
 - **Bundler-friendly default** — `new AlbexEngine()` works without extra
   configuration in bundlers that recognise the `new URL(..., import.meta.url)`
   asset pattern (see the "Install" section for the tested matrix).
-- **Fuzzy matching** — finds `"contrato"` even if you type `"conttrato"` (Bitap with adaptive edit distance).
+- **Fuzzy matching** — finds `"contrato"` even if you type `"conttrato"` (Bitap with adaptive edit distance). Sound under a two-stage pre-filter (character Bloom for exact tokens, a 256-bit **trigram q-gram signature** for everything) that prunes the candidate set ~10× on prose without ever dropping a real approximate match.
 - **Accent-insensitive** — `"accion"` matches `"acción"`, `"espana"` matches `"España"`, plus Latin Extended (Polish, Czech, Slovak, Turkish…).
 - **11 formats with varying depth** — DOCX · XLSX · PDF · HTML · MD · JSON · CSV · EML · RTF · TXT · XML. See the support table below; several formats are deliberately "lite" (CSV is RFC-4180-lite, EML is MIME-lite, RTF is regex-stripped, etc.).
 - **Phrase + OR queries** — `"contrato marco"` and `contrato | acuerdo` work out of the box.
@@ -81,8 +81,11 @@ That's the entire onboarding. Read on for what else the engine can do.
 - **WebGPU pre-filter** — experimental, opt-in (`gpu: 'auto'`). Implemented for corpora over 20 k chunks; no reproducible speedup number yet — the bench in this repo runs on a 200-document synthetic corpus only.
 - **SIMD opportunistic** — picks a SIMD-accelerated variant when the host supports v128.
 - **Tiered storage** — `TieredStore` keeps recent docs hot, evicts cold ones to OPFS, promotes on demand.
+- **Runtime capacity** — one binary, pools sized at init: `capacity: 'std'` (default, 128 docs / 100k chunks / 16 MB text), `'large'` (1 024 docs / 800k chunks / 128 MB text) or a custom `{ maxDocs, maxChunks, textPoolBytes, namePoolBytes }`.
+- **Capacity-safe** — when a pool fills (`docs`/`chunks`/`text`/`names`), `indexFile` throws `AlbexCapacityError` with `limit` (which pool) and `max` (the runtime limit) instead of silently truncating the corpus.
+- **Re-entrancy-safe** — async operations on one engine serialize; sync `search`/`compact`/`reset` refuse to run mid-operation (`AlbexError` kind `busy`) rather than corrupting the shared WASM state. Use `searchCooperative` for overlapping search-as-you-type.
 - **Typed errors** — `AlbexParseError`, `AlbexUnsupportedFormatError`, `AlbexCapacityError`, `AlbexInitError`. All extend `AlbexError`.
-- **Tiny core** — main WASM 24 KB (27 KB SIMD). PDF module (~1.2 MB) loads on demand. The OCR companion (`@albex/ocr`) is a separate package and pulls Tesseract.js (~3.5 MB) only when you call `enableOcr()`.
+- **Tiny core** — main WASM ~47 KB (~19 KB gzipped); the SIMD build is ~54 KB (~21 KB gzipped). PDF module (~1.2 MB, ~510 KB gzipped) loads on demand. The OCR companion (`@albex/ocr`) is a separate package and pulls Tesseract.js (~3.5 MB) only when you call `enableOcr()`.
 
 ---
 
@@ -197,7 +200,7 @@ const results = await pool.search('contrato');  // map-reduce
 
 ## Big corpora — tiered storage
 
-For workloads that exceed the tier's RAM capacity:
+For workloads that exceed the engine's RAM capacity:
 
 ```ts
 import { AlbexEngine, TieredStore } from 'albex';
@@ -221,29 +224,40 @@ Hot tier = engine. Warm tier = original files in OPFS. LRU eviction is automatic
 `new AlbexEngine()` covers the default case. The options below address
 specific deployment needs:
 
-### Tier auto-selection (`mini` / `std` / `pro` based on `deviceMemory`)
+### Capacity (runtime, single binary)
 
-Albex ships **six** WASM variants of the main engine (3 tiers × baseline/SIMD).
-By default it loads the std-baseline binary that comes with the npm package.
-If you want runtime tier auto-selection, serve the variants yourself and
-pass `wasmBaseUrl`:
+Capacity is a **runtime** parameter — there is one engine binary (plus its
+SIMD variant) and the pools are heap-allocated at `init()` to the size you
+ask for. The old compile-time tiers (and the `tier` option) are gone:
 
 ```ts
 const engine = new AlbexEngine({
-  wasmBaseUrl: '/assets',          // directory containing the 6 .wasm files
-  tier: 'auto',                    // picks mini/std/pro by deviceMemory
+  capacity: 'large',               // or 'std' (default), or a custom object
   simd: 'auto',                    // picks baseline/simd by WASM probe
   gpu:  'auto',                    // engages WebGPU when corpus > 20k chunks
 });
 ```
 
-Tier capacities:
+Presets and cost (≈ `maxChunks × 64 B + textPool + namePool`; WASM memory
+never shrinks, so the largest capacity initialised stays committed):
 
-| Tier  | Max docs | Max chunks | Max text | Working set |
-|-------|---------:|-----------:|---------:|------------:|
-| mini  | 32       | 25 000     | 4 MB     | ~5 MB       |
-| std   | 128      | 100 000    | 16 MB    | ~20 MB      |
-| pro   | 1 024    | 800 000    | 128 MB   | ~160 MB     |
+| Capacity  | Max docs | Max chunks | Max text | Working set |
+|-----------|---------:|-----------:|---------:|------------:|
+| `'std'`   | 128      | 100 000    | 16 MB    | ~22 MB      |
+| `'large'` | 1 024    | 800 000    | 128 MB   | ~180 MB     |
+| custom    | ≤ 65 536 | ≤ 4 M      | ≤ 1 GiB  | as configured |
+
+Custom objects may be partial — missing fields are completed from the std
+ratios (`maxChunks = maxDocs × 782`, `textPoolBytes = maxChunks × 168 B`,
+`namePoolBytes = maxDocs × 256 B`, with sane floors):
+
+```ts
+const tiny = new AlbexEngine({ capacity: { maxDocs: 16 } });
+```
+
+Snapshots are admitted by **content**: a snapshot saved with `'large'`
+loads into a `'std'` engine whenever its counters fit, and fails cleanly
+(previous index intact) when they don't.
 
 ### Custom CDN
 
@@ -297,14 +311,13 @@ PDF support requires `albex_pdf.wasm` to be served with MIME type `application/w
 rustup target add wasm32-unknown-unknown
 
 npm install
-npm run build:all         # 6 main variants + PDF + TypeScript
+npm run build:all         # main (baseline + SIMD) + PDF + TypeScript
 ```
 
 Partial builds:
 
 ```bash
-npm run build:wasm        # std baseline only
-npm run build:wasm:tiers  # all 6 variants
+npm run build:wasm        # main module (baseline + SIMD)
 npm run build:pdf-wasm    # PDF module
 npm run build             # TypeScript only
 ```