fpembed 0.2.0__tar.gz → 0.2.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (43) hide show
  1. {fpembed-0.2.0 → fpembed-0.2.2}/PKG-INFO +104 -22
  2. fpembed-0.2.0/src/fpembed.egg-info/PKG-INFO → fpembed-0.2.2/README.md +596 -537
  3. {fpembed-0.2.0 → fpembed-0.2.2}/pyproject.toml +5 -0
  4. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed/__init__.py +1 -1
  5. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed/compression.py +30 -2
  6. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed/compression_projection.py +21 -8
  7. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed/embedding_cache.py +6 -4
  8. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed/generator.py +19 -1
  9. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed/generator_smiles.py +21 -1
  10. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed/hashing.py +6 -0
  11. fpembed-0.2.2/src/fpembed/input_validation.py +74 -0
  12. fpembed-0.2.2/src/fpembed/manifest.py +170 -0
  13. fpembed-0.2.0/README.md → fpembed-0.2.2/src/fpembed.egg-info/PKG-INFO +619 -517
  14. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed.egg-info/SOURCES.txt +2 -0
  15. {fpembed-0.2.0 → fpembed-0.2.2}/LICENSE +0 -0
  16. {fpembed-0.2.0 → fpembed-0.2.2}/setup.cfg +0 -0
  17. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed/compression_blockwise.py +0 -0
  18. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed/compression_projection_cache.py +0 -0
  19. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed/dtype_support.py +0 -0
  20. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed/py.typed +0 -0
  21. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed/skfp_factory.py +0 -0
  22. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed/smiles_utils.py +0 -0
  23. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed.egg-info/dependency_links.txt +0 -0
  24. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed.egg-info/requires.txt +0 -0
  25. {fpembed-0.2.0 → fpembed-0.2.2}/src/fpembed.egg-info/top_level.txt +0 -0
  26. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_cache.py +0 -0
  27. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_compression.py +0 -0
  28. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_compression_blockwise.py +0 -0
  29. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_compression_projection.py +0 -0
  30. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_dtype.py +0 -0
  31. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_dtype_properties.py +0 -0
  32. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_embedding_cache.py +0 -0
  33. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_error_transparency.py +0 -0
  34. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_fingerprints.py +0 -0
  35. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_generator.py +0 -0
  36. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_hashing.py +0 -0
  37. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_input_validation.py +0 -0
  38. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_new_fp_type_matrix.py +0 -0
  39. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_packaging.py +0 -0
  40. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_properties.py +0 -0
  41. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_properties_050.py +0 -0
  42. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_resources_immutability.py +0 -0
  43. {fpembed-0.2.0 → fpembed-0.2.2}/tests/test_smiles_utils.py +0 -0
@@ -1,8 +1,11 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: fpembed
3
- Version: 0.2.0
3
+ Version: 0.2.2
4
4
  Summary: Generalized fingerprint embedding library
5
5
  License-Expression: MIT
6
+ Project-URL: Homepage, https://github.com/Sciencealone/FPembed
7
+ Project-URL: Commercial & Research Support, https://github.com/Sciencealone/FPembed/blob/main/COMMERCIAL_SUPPORT.md
8
+ Project-URL: Issue Tracker, https://github.com/Sciencealone/FPembed/issues
6
9
  Requires-Python: >=3.10
7
10
  Description-Content-Type: text/markdown
8
11
  License-File: LICENSE
@@ -45,6 +48,7 @@ Dynamic: license-file
45
48
  - [Code Examples](#code-examples)
46
49
  - [Choosing a Method](#choosing-a-method)
47
50
  - [Output Dtype](#output-dtype)
51
+ - [Representation Manifest](#representation-manifest)
48
52
  - [Compression Limits](#compression-limits)
49
53
  - [Performance Characteristics](#performance-characteristics)
50
54
  - [Project Structure](#project-structure)
@@ -52,6 +56,7 @@ Dynamic: license-file
52
56
  - [Datasets](#datasets)
53
57
  - [License](#license)
54
58
  - [Legal & Trademark Notice](#legal--trademark-notice)
59
+ - [Commercial & Research Services](#commercial--research-services)
55
60
  - [AI disclosure](#ai-disclosure)
56
61
  - [Support](#support)
57
62
 
@@ -168,6 +173,21 @@ print(embeddings.shape) # (3, 128) - 3 valid molecules
168
173
  print(invalid_indices) # [3] - index of invalid SMILES
169
174
  ```
170
175
 
176
+ ### Bounded-Memory Iteration
177
+
178
+ The batch methods above collect every valid embedding before stacking them into one matrix. At scale that peak allocation dominates: one million molecules at D=1024 needs about 4.1 GB as float32 or 8.2 GB as float64 for the final matrix alone, before counting the individual arrays held while it is built. When the input does not fit in memory, iterate instead:
179
+
180
+ ```python
181
+ for index, emb in gen.IterFingerprintsFromSmiles(smiles_iterable):
182
+ if emb is None:
183
+ continue # normal invalid input at this index
184
+ store(index, emb) # consume or write one row at a time
185
+ ```
186
+
187
+ `IterFingerprints`, `IterFingerprintsFromSmiles`, and `IterFingerprintsFromSelfies` accept any iterable, pull one input at a time, and yield `(index, embedding_or_none)`. Indices match input order and include invalid entries, which yield `None`; for `IterFingerprints`, a `None` molecule also yields `None`. Unexpected computation errors surface when the item is consumed, matching the single-molecule methods. The iterator never materializes the input or accumulates rows, so peak memory is yours to bound.
188
+
189
+ **Memory contract.** These APIs are not streaming in the sense of a total process cap. The `random_projection` matrix cache is bounded by retained bytes, but a matrix referenced by a live generator stays resident, so the cache budget is not a process-wide limit. Embeddings you keep are yours to release, and `dtype` reduces the returned and stored size, not the peak transient memory during compression.
190
+
171
191
  ### Raw Fingerprint (No Compression)
172
192
 
173
193
  ```python
@@ -189,6 +209,8 @@ emb = compress_fingerprint(fp, size=16)
189
209
  print(emb.shape) # (1, 128)
190
210
  ```
191
211
 
212
+ **Input contract.** `compress_fingerprint` accepts 1-D `(L,)` and 2-D `(N, L)` arrays of real numeric dtype: boolean, signed or unsigned integer, or floating. Non-real dtypes (complex, object, string, datetime) raise `TypeError`; floating input containing NaN or either infinity raises `ValueError`. Integer output (`dtype="uint16"`) additionally requires binary input and represents integer codes or counts, not normalized floats. Neither validation nor compression mutates the array you pass in.
213
+
192
214
  ### Parameter Hashing
193
215
 
194
216
  ```python
@@ -216,6 +238,12 @@ print(gen.cache_info()) # CacheInfo(hits=1, misses=1, maxsize=1024, currsize=1)
216
238
  gen.clear_cache()
217
239
  ```
218
240
 
241
+ ### SMILES Canonicalization and Chemical Standardization
242
+
243
+ FPembed canonicalizes SMILES spelling before keying and before parsing, so the same molecule written two ways produces the same embedding: the pipeline works with the canonical form (atom ordering and RDKit's aromatic form). That makes results spelling-invariant for atom-order-sensitive backends, and it is why the cache key can be built from the canonical string.
244
+
245
+ Canonicalization is not chemical standardization. FPembed does not remove salts, normalize tautomers, pick a protonation state, or impose a stereochemistry policy, because those decisions change scientific meaning and are the caller's to make. Two inputs that differ by a counter-ion or a tautomer stay distinct. If your workflow needs standardization, apply it before calling FPembed and describe it with `representation_manifest(preprocessing=...)` (see [Representation Manifest](#representation-manifest)).
246
+
219
247
  ## Why Use Embedded Fingerprints?
220
248
 
221
249
  Predictive accuracy is one axis of comparison between raw and embedded fingerprints - and the gap can be narrow, especially on large datasets where raw fingerprints have enough data to exploit all 2048 bits directly. However, accuracy is not the only metric that matters. The one guaranteed advantage is dimensionality: an embedded vector has D = L / C features instead of L, and the stored byte count follows directly from D and the chosen dtype (calculated exactly in [Storage Size](#storage-size)). The operational benefits that follow — training and inference speed, memory during training, sample efficiency — are consequences of fewer features, but their magnitude is workload-, model-, and hardware-dependent, not a fixed guarantee.
@@ -236,7 +264,7 @@ The output dimensionality (D = L / C) is deterministic and independent of datase
236
264
 
237
265
  A word on the baseline. Comparing an embedding against a raw fingerprint *stored as float64* overstates the storage gain, because a raw fingerprint is binary and would sensibly be bit-packed. Against **bit-packed** raw fingerprints, the `geometric` embedding at C=16 in float64 is actually *larger* (1 KB vs 256 B per molecule at L=2048): geometric at C=16 is a reversible re-encoding — it packs each 16-bit block into a single wide value, carrying the same information in fewer but larger numbers. Storing that embedding as `uint16` (see [Output Dtype](#output-dtype)) brings it back to parity with bit-packed input.
238
266
 
239
- So raw byte savings are **not** the reason to embed. The genuine, unconditional advantage is the **downstream feature count**: the embedded vector has D = L / C features instead of L, and every downstream cost — model training, inference, memory, distance computation — scales with feature count, not with how the bits were packed on disk. That dimensionality reduction is what the rest of this section quantifies.
267
+ So raw byte savings are **not** the reason to embed. The genuine, unconditional advantage is the **downstream feature count**: the embedded vector has D = L / C features instead of L. Downstream training, inference, and the in-memory feature matrix then shrink roughly in proportion to the feature count, and that reduction is what the rest of this section quantifies. Keep the scaling directional rather than universal: similarity on bit-packed fingerprints works on packed words, tree learners subsample features and pay fixed costs, and approximate-neighbor indexes change the comparison, so none of these costs follows the feature count exactly.
240
268
 
241
269
  ### ML Training and Inference Speed
242
270
 
@@ -244,7 +272,7 @@ The downstream ML model operates on the feature vector. Fewer features means fas
244
272
 
245
273
  - **Tree-based models (Random Forest, XGBoost)**: Feature splitting cost is proportional to the number of features. Going from 2048 to 128 features means each tree split considers ~16x fewer candidates. For hyperparameter searches (e.g., Optuna with hundreds of trials), this compounds into significant wall-clock savings.
246
274
  - **Neural networks**: The first dense layer's weight matrix shrinks from `(2048 x hidden)` to `(128 x hidden)` - 16x fewer parameters and 16x fewer multiply-adds per forward pass.
247
- - **Distance-based methods (k-NN, similarity search)**: Pairwise distance computation is O(N² x D). Reducing D from 2048 to 128 gives a direct 16x speedup.
275
+ - **Distance-based methods (k-NN, similarity search)**: Pairwise distance computation is O(N² x D). Reducing D from 2048 to 128 shrinks the distance computation itself by 16x; index build and query overhead, memory traffic, and candidate re-ranking sit on top of that and reduce the realized speedup.
248
276
 
249
277
  ### Memory During ML Training
250
278
 
@@ -252,12 +280,14 @@ During model training, the feature matrix for N=100K molecules occupies `(100000
252
280
 
253
281
  ### Compression Overhead
254
282
 
255
- The compression step itself is negligible for block-wise methods (~1 ms per 1000 molecules). The total pipeline cost is:
283
+ The compression step itself is small for block-wise methods, but its cost is not universal. Under warm-cache conditions, and on the machine where the project measured it, block-wise compression added on the order of 1 ms per 1000 molecules. Treat that as an order-of-magnitude indication with `L`, dtype, and hardware attached, not as a constant. Cold-cache cost is higher: the projection methods build or fetch a projection artifact on first use, and `random_projection` additionally pays to draw and store its `(L, D)` matrix.
284
+
285
+ The total pipeline cost is:
256
286
 
257
287
  - **Raw**: `skfp generation time`
258
- - **Embedded**: `skfp generation time + ~1 ms per 1000 molecules` (block-wise)
288
+ - **Embedded**: `skfp generation time + compression time`
259
289
 
260
- The downstream ML speedup from 128 vs 2048 features far exceeds this overhead.
290
+ Measure both phases on the target hardware before relying on the overhead being negligible. The downstream ML saving from 128 versus 2048 features is usually larger, but that too is workload-dependent.
261
291
 
262
292
  ### Sample Efficiency
263
293
 
@@ -289,8 +319,8 @@ The guaranteed columns are the feature count and its exact consequences (paramet
289
319
  | Layered | `layered` | `min_path` (1), `max_path` (7) |
290
320
  | Pattern | `pattern` | (none) |
291
321
  | Avalon | `avalon` | (none) |
292
- | SECFP | `secfp` | `radius` (default 3) |
293
- | MHFP | `mhfp` | `radius` (default 3) |
322
+ | SECFP | `secfp` | `radius` (3), `min_radius` (1) |
323
+ | MHFP | `mhfp` | `radius` (3), `min_radius` (1) |
294
324
  | MAP4 | `map` | `radius` (default 2) |
295
325
 
296
326
  ## Compression Methods
@@ -303,10 +333,10 @@ FPembed supports six compression methods, selectable via the `method` parameter
303
333
  |-------------------------|----------|-----------------|---------------------------------------|------------|
304
334
  | `geometric` | block-wise | `interleave` (bool) | (2**C − 1):1 dynamic range (e.g. 65,535:1 at C=16) | O(L) |
305
335
  | `linear` | block-wise | `interleave` (bool) | C:1 dynamic range | O(L) |
306
- | `log` | block-wise | `interleave` (bool) | ~4.1:1 dynamic range | O(L) |
336
+ | `log` | block-wise | `interleave` (bool) | ~log2(C+1):1 (~4.1:1 at C=16) | O(L) |
307
337
  | `uniform` | block-wise | `interleave` (bool) | 1:1 (mean pooling) | O(L) |
308
- | `hadamard` | global | `seed` (int) | orthogonal projection | O(L log L) |
309
- | `random_projection` | global | `seed` (int) + `sparse` (bool) | JL distance preservation | O(L D) |
338
+ | `hadamard` | global | `seed` (int) | seeded, truncated Hadamard projection (expected squared norm D/L) | O(L log L) |
339
+ | `random_projection` | global | `seed` (int) + `sparse` (bool) | JL-style distance preservation for the projection itself | O(L D) |
310
340
 
311
341
  ### The `method` Parameter
312
342
 
@@ -316,7 +346,7 @@ Pass `method` to the Generator constructor to select a compression strategy.
316
346
 
317
347
  - **Block-wise methods** (`geometric`, `linear`, `log`, `uniform`): accept `interleave` (bool, default `False`). When `True`, bits are assigned to blocks by stride (`bit[i] -> block[i % n_blocks]`) instead of contiguous partitioning, breaking hash clustering artifacts.
318
348
  - **`hadamard`**: accepts `seed` (int, default `42`). Controls the random sign flips applied before the Fast Walsh-Hadamard Transform.
319
- - **`random_projection`**: accepts `seed` (int, default `42`) and `sparse` (bool, default `False`). The `sparse` option uses the Achlioptas variant with approximately 2/3 zero entries for faster computation at comparable quality.
349
+ - **`random_projection`**: accepts `seed` (int, default `42`) and `sparse` (bool, default `False`). `sparse=True` draws the Achlioptas distribution, whose entries are about 2/3 zero, but the matrix is stored densely, so it saves no memory. On the binary-input fast path the work is set by the number of set bits in the input row, not by the zeros in the matrix, so `sparse` does not automatically make that path faster; any arithmetic saving applies to the dense-product fallback used for non-binary input. It also changes the projection values, so compare the two settings on your task.
320
350
 
321
351
  Seed-based methods (`hadamard`, `random_projection`) are fully deterministic given the same seed and NumPy version. The default seed is `42`.
322
352
 
@@ -367,7 +397,19 @@ print(emb.shape) # (1, 128)
367
397
 
368
398
  ### Choosing a Method
369
399
 
370
- Block-wise methods (`geometric`, `linear`, `log`, `uniform`) are fast (O(L)) and simple - use them when speed matters or compression ratios are modest. Among these, `geometric` preserves the most dynamic range while `uniform` treats all bits equally (mean pooling). Global projection methods (`hadamard`, `random_projection`) mix information across all input bits, which helps retain more information at high compression ratios. `hadamard` is efficient (O(L log L)) and requires power-of-2 fingerprint sizes; `random_projection` offers the strongest theoretical distance-preservation guarantees (Johnson-Lindenstrauss lemma) at the cost of O(L D) complexity.
400
+ Block-wise methods (`geometric`, `linear`, `log`, `uniform`) are fast (O(L)) and simple, so reach for them when speed or modest compression ratios matter most. `geometric` keeps the widest code range per block, while `uniform` treats every bit equally (mean pooling). Global projection methods (`hadamard`, `random_projection`) mix information across all input bits, which tends to retain more information at high compression ratios. `hadamard` is efficient (O(L log L)) and requires power-of-2 fingerprint sizes; `random_projection` carries the strongest theoretical distance-preservation statement (Johnson-Lindenstrauss) at the cost of O(L D) work.
401
+
402
+ Method rankings depend on the downstream model, so treat any single recommendation as a starting point rather than an optimum. The table below records the direction found in the project's own experiments (see `dev_docs/investigation_foundation_model_embedding_compression.md`); those measurements were taken on specific datasets and models and do not transfer as a guarantee. The recorded configuration ECFP with `fp_size=16384` and `C=16` is a starting candidate, not a proven optimum, and it does not change the default method.
403
+
404
+ | Application | Starting point | What to check before relying on it |
405
+ |---|---|---|
406
+ | Tree-based molecular property prediction | `geometric`, `C=16` | Compare against raw fingerprints and other block-wise methods; tree feature subsampling changes the realized speedup |
407
+ | Linear prediction | `random_projection` | Re-tune regularization after changing the representation |
408
+ | Approximate similarity, clustering, or retrieval | `random_projection` | Measure neighborhood preservation and `recall@k` under the intended metric; cosine correlation with a reference embedding is not enough |
409
+ | Exact binary similarity (Tanimoto) | Bit-packed raw fingerprints | Compressed-space Euclidean distance is not binary Tanimoto |
410
+ | Dense continuous embeddings | Projection methods | The geometric binary-code argument no longer applies |
411
+
412
+ Retrieval and similarity use cases need task-specific retrieval evaluation. A high correlation with a reference embedding does not establish that nearest neighbors are preserved.
371
413
 
372
414
  ### Output Dtype
373
415
 
@@ -375,7 +417,7 @@ Both `EmbeddedFingerprintGenerator` (constructor argument `dtype`, applied to ev
375
417
 
376
418
  | `dtype` | Bytes/value | Semantics |
377
419
  |---|---|---|
378
- | `float64` (default) | 8 | Exact, unchanged. What every existing caller gets. |
420
+ | `float64` (default) | 8 | The values existing callers already get, computed exactly as before. For `geometric`, the value equals the integer code divided by `2**C - 1`, apart from the float64 round-off of the accumulation path. |
379
421
  | `float32` | 4 | **Information-preserving** for `geometric` at `C <= 16` (see below). ~1e-7 relative error for the projection methods. Permitted for every method. |
380
422
  | `uint16` | 2 | The **exact integer code**. Permitted only for `geometric` and `uniform`, and only for binary input. |
381
423
 
@@ -404,6 +446,19 @@ The stored `float32` value is **not** bit-equal to the `float64` output — it d
404
446
 
405
447
  `interleave=True` is supported identically — interleaving changes which bits form a block, not the recovery formula.
406
448
 
449
+ #### Reversibility is not distance preservation
450
+
451
+ At supported precision, `geometric` recovers the original bits exactly, but that does not mean the decoded values preserve the geometry of those bits, or that a model can reach them equally easily. A two-bit counterexample, with `C=2` so `2**C - 1 = 3`:
452
+
453
+ - Codes `1` and `2` differ in both bits, yet their decoded values differ by only `1/3`.
454
+ - Codes `0` and `2` differ in one bit, yet their decoded values differ by `2/3`.
455
+
456
+ Ranking pairs by Euclidean distance in the decoded space therefore does not reproduce ranking by Hamming distance. Higher-order bits also map to larger positional values than lower-order bits, which makes them easier for a threshold-based model to reach; injectivity alone does not make all bits equally learnable. Exact binary similarity (Tanimoto on the original bits) stays a separate operation from comparing compressed vectors.
457
+
458
+ Pooling methods (`linear`, `log`, `uniform`) additionally collide distinct blocks, so their suitability has to be measured on the task rather than inferred from the weight range.
459
+
460
+ Keep the sources of loss separate: fingerprint hashing collisions come from the fingerprint itself, block pooling from the compression step, float rounding from the stored dtype, and any accuracy change from the downstream model. Only pooling and rounding are compression concerns.
461
+
407
462
  **Which combinations raise `ValueError`:**
408
463
 
409
464
  - `uint16` with `linear`, `log`, `hadamard`, or `random_projection` — these have no exact integer form.
@@ -420,13 +475,30 @@ fp = np.random.randint(0, 2, size=2048).astype(np.uint8)
420
475
  # Exact integer code, 4x smaller than float64
421
476
  codes = compress_fingerprint(fp, size=16, dtype="uint16")
422
477
 
423
- # Recover the float64 embedding exactly
424
- recovered = codes / (2**16 - 1) # geometric
478
+ # Recover the geometric float64 value (the code / (2**16 - 1), up to float64 round-off)
479
+ recovered = codes / (2**16 - 1)
425
480
  ```
426
481
 
482
+ ### Representation Manifest
483
+
484
+ `fp_params_hash()` is a cache key: it covers the fingerprint type, its parameters, and the compression method. It is deliberately narrow and must not change, so it is not a complete identity for an array you persist. `EmbeddedFingerprintGenerator.representation_manifest()` returns a fresh JSON-compatible mapping for that purpose:
485
+
486
+ ```python
487
+ import json
488
+
489
+ manifest = gen.representation_manifest()
490
+ print(json.dumps(manifest, indent=2))
491
+ ```
492
+
493
+ It records the resolved fingerprint type, width, and parameters (omitted and explicitly-default values resolve to the same settings), whether compression is active together with its factor, method and effective method parameters including `seed` and `interleave`, the output dimension, dtype, and physical encoding, the built-in canonicalization policy, an optional declarative `preprocessing` mapping you supply, and the installed `fpembed`, `numpy`, `rdkit`, `scikit-fingerprints`, and `selfies` versions. `schema_version` describes the manifest format and moves independently of the package version.
494
+
495
+ The `output.encoding` field distinguishes what the values physically are: `raw_binary` for an uncompressed fingerprint, `normalized_block_value` and `mean_block_value` for geometric and uniform float output, `integer_code` and `integer_count` for the exact `uint16` codes, `weighted_block_value` for linear and log, and `projection_value` for Hadamard and random projection. A `uint16` geometric array and its `float64` counterpart are therefore not interchangeable: the recovery formula relates them, but only the integer form is exact.
496
+
497
+ Built-in canonicalization and external preprocessing are reported separately. The manifest stores a caller-supplied `preprocessing` mapping verbatim under `preprocessing.external` and never executes it; supplying one does not make the pipeline standardized. The manifest is additive: it does not change `fp_params_hash`, cache keys, or returned arrays. Implementations may change the layout between versions, so re-derive a manifest rather than comparing its bytes across `fpembed` releases.
498
+
427
499
  ### Compression Limits
428
500
 
429
- `geometric` compression is capped at `C <= 32`; requesting `compression > 32` with `geometric` (at either entry point) raises `ValueError`. Geometric block weights span `2**-C`, so the block integer survives only while it fits the float64 mantissa. The einsum accumulates roughly `C * 2**-52` relative error against a code spacing of `2**-C`, so recovery starts failing well before the naive 53-bit limit — at `C = 64` the low ~11 bits of every block are silently absorbed. `C <= 32` is the largest power of two comfortably inside the safe range (verified exact for `C <= 16`). A `ValueError`, not a warning, is used, because warnings can be suppressed in evaluation runs — precisely where silent bit loss does the most damage.
501
+ `geometric` compression is capped at `C <= 32`; requesting `compression > 32` with `geometric` (at either entry point) raises `ValueError`. Geometric block weights span `2**-C`, so the block integer survives only while it fits the float64 mantissa. The einsum accumulates roughly `C · 2**-52` relative error against a code spacing of `2**-C`, so recovery starts failing well before the naive 53-bit limit — at `C = 64` the low ~11 bits of every block are silently absorbed. `C <= 32` is the largest power of two comfortably inside the safe range (verified exact for `C <= 16`). A `ValueError`, not a warning, is used, because warnings can be suppressed in evaluation runs — precisely where silent bit loss does the most damage.
430
502
 
431
503
  The cap applies only to `geometric` and only when compression is active (`compression=None` or `0` is exempt). The other methods (`linear`, `log`, `uniform`, `hadamard`, `random_projection`) have bounded weight ratios or use no block weights, so none approaches the mantissa limit even at large `C` and none is capped.
432
504
 
@@ -437,15 +509,17 @@ All methods produce the same output dimensionality (D = L / compression) but dif
437
509
  | Method | Speed | Precomputed Memory | Best For |
438
510
  |--------|------------------------------------------------------------------------------------|--------------------------------------|----------|
439
511
  | Block-wise (all four) | Fastest - single vectorized einsum, O(L) | Negligible (C-length weight vector) | Default choice; large batches |
440
- | Hadamard (SRHT) | Fast - pruned fold-then-FWHT: the input is folded down to length D *before* transforming, so the discarded outputs are never computed. Exact — bit-identical to the full transform for binary input. | L-length sign vector (~16 KB) | High compression ratios |
441
- | Random projection | Fast - for binary input, gathers and sums only the matrix rows selected by the set bits (exploiting fingerprint sparsity) instead of a full dense matmul; non-binary input falls back to the dense product. The gather result is mathematically equivalent to the dense product **within float64 round-off** (the two sum the same terms in a different order), not bit-identical. One canonical C-contiguous `(L, D)` matrix is cached per parameter set. | (L, D) matrix (~2 MB for L=2048, D=128) | Best theoretical guarantees (JL lemma) |
512
+ | Hadamard (seeded, truncated) | Fast: pruned fold-then-FWHT, where the input is folded down to length D *before* transforming, so the discarded outputs are never computed. The retained first-D outputs are bit-identical to truncating the full transform for binary input. Correctness against that reference does not by itself prove neighborhood or distance preservation. | L-length sign vector (~16 KB) | High compression ratios |
513
+ | Random projection | Fast - for binary input, gathers and sums only the matrix rows selected by the set bits (exploiting fingerprint sparsity) instead of a full dense matmul; non-binary input falls back to the dense product. The gather result is mathematically equivalent to the dense product **within float64 round-off** (the two sum the same terms in a different order), not bit-identical. One canonical C-contiguous `(L, D)` matrix is cached per parameter set. | (L, D) matrix (~2 MB for L=2048, D=128) | High compression ratios; JL-style guarantee for the projection itself |
442
514
 
443
515
  Both projection methods gained fast paths — the Hadamard fold is bit-identical to the full transform for binary input, and the random-projection gather is equivalent to the dense product within float64 round-off. Measured wall-time improvements (L=16384, D=1024, on one machine):
444
516
 
445
517
  - **Hadamard** (pruned fold-then-FWHT vs full-length transform then truncate): roughly **3x** at N=1 and **6x** at N=16.
446
518
  - **Random projection** (binary row-gather vs dense matmul, at ECFP-like sparsity): roughly **28x** at N=1 and **2.5x** at N=16.
447
519
 
448
- These are measured wall times on one machine, not a theoretical claim — the pruned FWHT reduces butterfly *operation count* by about 9x, but the code is already vectorized and largely bandwidth-bound, so wall-time gain is materially less than the operation-count reduction. Block-wise methods remain the fastest overall choice. Random projection's matrix memory still grows with `L * D`; note that `sparse=True` (the Achlioptas variant) stores its matrix **densely** and therefore saves no memory despite the name — it trades roughly 2/3 zero entries for faster arithmetic, not a smaller footprint.
520
+ These are measured wall times on one machine, not a theoretical claim: the pruned FWHT reduces butterfly *operation count* by about 9x, but the code is already vectorized and largely bandwidth-bound, so the wall-time gain is materially less than the operation-count reduction. Block-wise methods remain the fastest overall choice. Random projection's matrix memory still grows with `L × D`. Note that `sparse=True` (the Achlioptas variant) stores its matrix **densely** and therefore saves no memory despite the name; it trades roughly 2/3 zero entries for a denser arithmetic pattern in the dense-product path, not a smaller footprint, and the binary-input gather selects matrix rows by the input's set bits rather than by the matrix nonzeros.
521
+
522
+ **Hadamard specifics.** The implementation applies seeded random sign flips and keeps the first `D` outputs in Sylvester order. It does not randomly choose which `D` of the `L` coordinates to keep, so it is a seeded, truncated Hadamard projection rather than conventional randomly subsampled SRHT. With the retained `1/sqrt(L)` normalization, averaging over the random signs gives an expected squared norm of `D / L` times the input squared norm; a distance-scale correction would multiply the outputs by `sqrt(L / D)`, but that does not remove the fixed-coordinate selection. Read the fold's bit-identity with the full transform as correctness of this transform, not as evidence of neighborhood preservation.
449
523
 
450
524
  **Projection matrix cache.** The random-projection matrix is cached to avoid redrawing it on every call. Only a **single canonical orientation** is retained per parameter set — one C-contiguous `(L, D)` matrix; the `(D, L)` orientation is served as a non-owning `.T` view of the same buffer, so no second owning copy exists. The cache is bounded by **total retained bytes**, not entry count: the budget defaults to 512 MiB (`RP_CACHE_BYTE_BUDGET` in `compression_projection_cache.py`), with a small entry-count floor so a single oversized matrix cannot evict itself. Least-recently-used matrices are dropped once the accumulated bytes exceed the budget.
451
525
 
@@ -459,11 +533,13 @@ fpembed/
459
533
  │ ├── generator_smiles.py # SMILES/SELFIES/Mol input mixin
460
534
  │ ├── compression.py # compress_fingerprint (orchestrator)
461
535
  │ ├── compression_blockwise.py # block-wise weight schemes
462
- │ ├── compression_projection.py # Hadamard SRHT + random projection
536
+ │ ├── compression_projection.py # seeded truncated Hadamard + random projection
463
537
  │ ├── compression_projection_cache.py # byte-bounded projection-matrix cache
464
538
  │ ├── dtype_support.py # output dtype validation and casting
465
539
  │ ├── embedding_cache.py # BoundedCache for embedding lookups
466
540
  │ ├── hashing.py # fp_params_hash
541
+ │ ├── input_validation.py # real/finite input boundary guard
542
+ │ ├── manifest.py # representation manifest builder
467
543
  │ ├── skfp_factory.py # scikit-fingerprints backend factory
468
544
  │ ├── smiles_utils.py # parse_smiles, canonicalize_smiles
469
545
  │ └── py.typed # PEP 561 marker
@@ -472,7 +548,7 @@ fpembed/
472
548
  │ ├── datasets/ # molecular datasets (RedDB, NFA, QM9)
473
549
  │ └── nicegui_app/ # NiceGUI demo application
474
550
  ├── pyproject.toml # package metadata and dependency ranges
475
- ├── requirements.txt # third-party app lock (no fpembed self-pin)
551
+ ├── requirements.txt # third-party app lock (no FPembed self-pin)
476
552
  ├── environment.yml # authoritative conda env (installs -e .)
477
553
  └── README.md
478
554
  ```
@@ -529,6 +605,12 @@ The name **"FPembed"**, its branding, and identifiers are the intellectual prope
529
605
  * **Commercial Branding:** The unauthorized use of the name "FPembed" to brand, market, or promote commercial software, corporate AI engines, or proprietary services within the fields of bioinformatics, chemoinformatics, and AI drug discovery is strictly prohibited.
530
606
  * **Community Protection:** This notice is established to prevent public confusion and to protect the open-source community from misleading corporate misrepresentations. For licensing queries regarding the project name, please open an issue or contact the author directly.
531
607
 
608
+ ## Commercial & Research Services
609
+
610
+ For the project onboarding, corporate deployments, high-performance compute scaling (HPC / Databricks / Ray), or custom model fine-tuning for pharma R&D and research grants (Horizon Europe / Art. 60 LOSU / AEI) as well as other related tasks, official technical support and contract services are available.
611
+
612
+ **[View Full Service Offerings & Contact Details](COMMERCIAL_SUPPORT.md)**
613
+
532
614
  ## AI disclosure
533
615
 
534
616
  AI usage during project development is declared in [aidecl.yaml](aidecl.yaml) following the [AI Declaration Format](https://ai-declaration.org/).