hypertopos 0.6.2__tar.gz → 0.6.4__tar.gz

{hypertopos-0.6.2 → hypertopos-0.6.4}/CHANGELOG.md

@@ -7,6 +7,86 @@ Versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
 
+## [0.6.4] — 2026-05-07
+
+### Fixed
+- Parallel chain extraction (`extract_chains` via `ProcessPoolExecutor`) emitted `chain_id`s that collided across workers. Each worker built its chain list locally and used `len(chains)` as the id counter, so the merged output assigned the same `CHAIN-XXXXXX` value to up to N_workers distinct chains. Chains in the points table were then no longer keyed uniquely by `primary_key`. Post-merge dedup now reassigns `chain_id` from the global merged index, restoring uniqueness. **Spheres built before this fix carry colliding chain_ids; rebuild the chain pattern (or the full sphere) to restore unique ids.** A defensive raise in `anomaly_propagation_in_chain` surfaces the ambiguity on legacy spheres so investigations don't silently return the wrong chain's hops. Chain extraction now also asserts post-dedup chain_id uniqueness as a build-time safety net so any future regression in the merge logic fails loudly instead of silently corrupting the chain pattern points table.
+
+### Added
+- `find_motif_by_hops` gains an `anomaly_seed_filter: bool = False` parameter. When `True`, the BFS starting frontier is intersected with the anomaly subset of the resolved anchor companion pattern (entities with `is_anomaly=True`) — replaces the implicit "all keys" frontier when `seed_keys=None`, intersects with the explicit list otherwise. On large populations with a small anomaly fraction, the BFS traversal is collapsed proportionally to the prune ratio, reducing wall-clock for population-sweep motif queries that are bounded on frontier size. Result dict gains `seed_filter_summary` with `requested`, `anomaly`, and `filtered` counts so callers can verify the prune. Raises `GDSNavigationError` when no anchor companion pattern is configured.
+- `classify_chain_typology(chain_id, pattern_id, *, anchor_pattern_id)` — new navigator inspector that wraps `anomaly_propagation_in_chain` and labels the chain along five operational axes: `shape` (monotone-rising / monotone-falling / peak-in-middle / peak-at-start / peak-at-end / flat / single-hop / no-anomalous-run), `peak_position` within the run, `position_in_chain` (leading / transit / terminal / full-chain), `extension_signals` (`forward` / `backward` booleans indicating whether the next-hop or pre-run hop is in an elevated rank band), `pre_run_rank_bucket` and `breakpoint_rank_bucket`, plus the chain-wide `dominant_top_dim`. Returns the chain's longest-run summary alongside the typology block.
+- `extend_chain(chain_id, pattern_id, *, anchor_pattern_id, direction="forward"|"backward", max_results=20)` — new navigator inspector that suggests extension entities at the boundary of a chain's anomalous run. Forward looks at entities that follow the run-end key in OTHER chains in the same chain pattern (via the chain reverse index); backward looks at entities that precede the run-start key. Candidates are returned with `is_anomaly`, `delta_norm`, `delta_rank_pct`, source chain ids, and ranked by `(is_anomaly DESC, delta_norm DESC, n_source_chains DESC)`.
+- `anomaly_propagation_in_chain(chain_id, pattern_id, *, anchor_pattern_id)` — new navigator inspector primitive complementary to `find_chains_with_coherent_anomaly`. Takes a single chain id and returns the chain's hop-by-hop anomaly trace: for each entity in the chain's keys sequence, returns `is_anomaly`, `delta_norm`, the dominant delta dimension (sigma-normalised argmax over `|delta|`) when anomalous, and `delta_rank_pct`. Plus a summary with `n_hops`, `n_anomalous`, `max_run_length_same_top_dim`, and `dominant_top_dim`. Use after a population sweep flags a chain to drill into how the anomaly accumulates and where it breaks. Pure query-side, additive within sphere format 2.4.
+
+### Changed
+- `find_motif_by_hops` no longer routes seeded queries (`seed_keys=<list>`) through a scoped per-call edge-table read path. Scoped routing was uniformly slower than the cached global `AdjacencyIndex` + per-call seed filter at every measured seed count from 10 up; the BFS enumerator already filtered the starting frontier by `seed_keys`, so the scoped branch added cost without benefit. After the change all queries — seeded or unseeded — use the cached global adjacency. The cold first call still pays the full O(E) adjacency build; warm calls amortise to BFS-only cost. The internal `_build_scoped_adjacency_for_motif` helper is removed.
+
+### Added
+- `find_chains_with_coherent_anomaly(pattern_id, *, anchor_pattern_id, min_hops=3, max_results=100)` — new navigator primitive that surfaces chains where `>=min_hops` strictly consecutive entity-anchor positions are individually flagged as `is_anomaly=True` AND share the same dominant delta dimension (argmax over sigma-normalised |delta|). Returns ranked runs sorted by `(run_length DESC, max_delta_norm DESC)` with `chain_id`, `run_start_idx`, `run_length`, `top_dim`, `run_keys`, `max_delta_norm` per match plus diagnostics (`n_chains_total`, `n_anomaly_entities`, `elapsed_ms`). Pure query-side, additive within sphere format 2.4. Distinct from `find_anomalies` on a chain pattern, which scores chain-level features (hop count, time span, amount decay); the new primitive scores chain composition — coherent cascades through structurally-similar anomalous entities. Useful when chain shape looks normal but consecutive hops go through entities that all share the same anomaly driver.
+
+## [0.6.3] — 2026-05-06
+
+### Changed
+- Build-time compute paths for `edge_dim_aggregations:` and the edge feature
+  catalog (`compute_pair_edge_count`, `compute_time_since_pair_last_edge`,
+  `compute_pair_amount_zscore`, `parse_timestamps_to_epoch`) optimised —
+  pure-Python per-row loops replaced with vectorised pyarrow / numpy
+  paths. The `_p95` aggregate now returns the exact 95-th percentile
+  element (previously a t-digest sketch approximation).
+
+### Added
+- `aggregates:` per-source-dim subset selector on `edge_dim_aggregations:`.
+  Source dims can now declare which aggregates to emit (`mean` / `max` /
+  `std` / `p95` / `count_above_threshold`) instead of always emitting all
+  five. The existing `dims: [a, b]` list form continues to expand to all
+  five aggregates per dim; the new mapping form
+  `dims: {a: [count_above_threshold], b: [mean, max]}` selects subsets.
+  Polygon-dim layout follows source-dim insertion order × canonical
+  aggregate order, so reordering the user-supplied aggregate list does
+  not flip `schema_hash`. Spheres rebuilt with subset selectors carry an
+  `aggregates_per_dim` block under each pattern's `edge_dim_aggregations:`
+  in `sphere.json`; pre-selector spheres keep emitting the full five-tuple
+  on read.
+- Per-source-dim `_count_above_threshold` thresholds are now persisted into
+  the calibration epoch JSON under `_gds_meta/calibration_history/<pid>/v={N}.json`
+  alongside `mu` / `sigma` / `theta`. Each calibration epoch carries the
+  thresholds used during its build, so re-builds are deterministic and
+  multi-epoch comparisons see threshold drift. `compare_calibrations`
+  output gains a new `edge_dim_threshold_drift` field — a per-source-dim
+  `{from, to, delta}` map populated when both compared epochs declared
+  `edge_dim_aggregations:`. Legacy epochs (built before this release)
+  deserialize with `edge_dim_thresholds=None` and silently omit the
+  drift block.
+- `edge_dim_aggregations:` on composite anchors with `len(key_cols) > 2`
+  (tripartite and beyond). The previous k=2 (pair) restriction is lifted —
+  composite anchors registered via `composite_lines:` with three or more
+  key columns now compose with `edge_dim_aggregations:` the same way k=2
+  composite anchors did. The anchor primary key is constructed positionally
+  from `key_cols` joined by the `composite_lines.<id>.separator`, matching
+  the convention used to register the composite line itself. Aggregated
+  per-anchor `<source_dim>_mean` / `<source_dim>_max` columns surface in
+  every existing anchor-pattern primitive (`find_anomalies`,
+  `explain_anomaly`, `find_clusters`, `find_calibration_influencers`,
+  `decompose_drift`) via the same `dimension_kinds` source-of-truth as the
+  earlier `single` / `pair` / `chain` regimes. No new tools, no MCP API
+  change, no sphere format bump.
+- Three new aggregates on `edge_dim_aggregations:` — `_std`
+  (per-anchor standard deviation of the source dim across the anchor's
+  edges), `_p95` (95th percentile, tail-mass tracker without `_max`'s
+  extreme-outlier sensitivity), and `_count_above_threshold` (number
+  of edges where the source dim crosses a per-dim threshold). Extend
+  the prior `_mean` / `_max` set on every anchor regime (single, pair,
+  k>2, chain). Surface automatically alongside `_mean` / `_max` on
+  every anchor primitive (`find_anomalies`, `explain_anomaly`,
+  `find_clusters`, `find_calibration_influencers`, `decompose_drift`).
+  Per-dim threshold for `_count_above_threshold` defaults to the
+  population p95 of the source dim; user can override by passing a
+  `thresholds` dict keyed by source dim name. `_std` adds variance
+  signal that `_mean` / `_max` don't capture; `_p95` separates
+  tail-mass from extreme outliers; `_count_above_threshold` flags
+  anchors with anomalous concentration of high-value edges. No new
+  YAML keywords, no new tools, no sphere format bump.
+
 ## [0.6.2] — 2026-05-05
 
 ### Added

{hypertopos-0.6.2 → hypertopos-0.6.4}/LICENSE.md

@@ -26,7 +26,7 @@ For purposes of this License:
 - **Internal use** within an organization is not a competitive offering. Your
   organization includes entities under common control.
 
-**Change Date:** 2029-05-05
+**Change Date:** 2029-05-07
 
 **Change License:** Apache License, Version 2.0
 

{hypertopos-0.6.2 → hypertopos-0.6.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hypertopos
-Version: 0.6.2
+Version: 0.6.4
 Summary: Understand the structure of your data — without training machine learning models
 Project-URL: Homepage, https://github.com/hypertopos/hypertopos-py
 Project-URL: Documentation, https://github.com/hypertopos/hypertopos-py/tree/main/docs

{hypertopos-0.6.2 → hypertopos-0.6.4}/README.md

@@ -8,7 +8,7 @@
 [![PyArrow](https://img.shields.io/badge/format-PyArrow-red.svg)](https://arrow.apache.org/docs/python/)
 [![Lance](https://img.shields.io/badge/storage-Lance-blueviolet.svg)](https://github.com/lance-format/lance)
 [![MCP](https://img.shields.io/badge/MCP-compatible-green.svg)](https://modelcontextprotocol.io)
-[![Version](https://img.shields.io/badge/version-0.6.2-%235500FF.svg)](pyproject.toml)
+[![Version](https://img.shields.io/badge/version-0.6.4-%235500FF.svg)](pyproject.toml)
 
 hypertopos is not a database, and not a machine learning model. It is a layer that turns relational data into a coordinate system where every entity gets a position derived from its relationships and the population around it.
 
@@ -69,6 +69,9 @@ Each capability below emerges from treating entities as points in a shared, popu
 | Richer hop predicates | `HopPredicate.amount_ratio_to_prev` (decreasing-chain ratio) and `require_anomalous_entity` (filter chains routing through calibrated-anomalous nodes) extend the declarative motif API | Closed-vocab motif library has fixed amount thresholds and no anomaly-routing filter | `0.6.1` |
 | Event-aware motif scoring | `find_motif_by_hops(score=True)` ranks motifs by the product of event-aware `edge_potential` across edges (uses both the anchor companion's per-entity geometry and the event pattern's per-transaction polygons); distinct transactions between the same accounts produce distinct scores | Pure node-pair scoring collapses ranks when motifs share a node sequence | `0.6.1` |
 | Chain-anchor aggregation | Chain anchor patterns auto-emitted from `chain_lines:` declare `edge_dim_aggregations:` to bake per-event sidecar signals into per-chain `_mean` / `_max` columns; closes the third `anchor_kind` after `single` and `pair` | Per-chain manual aggregation outside the geometry, or no chain-level edge-dim summary at all | `0.6.2` |
+| Expanded `edge_dim_aggregations:` surface | Three additional canonical aggregates per source dim (`_std`, `_p95`, `_count_above_threshold` with population p95 cutoff persisted in calibration epoch JSON); k>2 composite anchor support (tripartite and beyond); per-source-dim subset selector — `dims:` accepts list (sugar = all five aggregates) or mapping `{dim: [agg, …]}` (per-dim subset); cross-epoch `edge_dim_threshold_drift` surface on `compare_calibrations` | Manual rebuild + re-engineering whenever the aggregate vocabulary or the per-anchor breadth changes | `0.6.3` |
+| Chain-coherent investigative loop | Four primitives compose into a complete chain investigation: `find_chains_with_coherent_anomaly` (population sweep — chains where consecutive entity-anchor positions are individually anomalous on the same dominant delta dim), `anomaly_propagation_in_chain` (per-chain hop-by-hop trace), `classify_chain_typology` (five-axis label: shape / peak_position / position_in_chain / extension_signals / dominant_top_dim), `extend_chain` (boundary-extension suggester via the chain reverse index) | Manual SQL over chain pattern + ad-hoc Python scoring + no per-chain typology label | `0.6.4` |
+| Anomaly-anchored seed prune for motifs | `find_motif_by_hops(anomaly_seed_filter=True)` intersects the BFS starting frontier with the anomaly subset of the resolved anchor companion (replaces "all keys" frontier when `seed_keys=None`, intersects with explicit list otherwise); result dict carries `seed_filter_summary` (`{requested, anomaly, filtered}`) | Manual pre-filter on every call, no built-in convergence on anomaly-anchored seeds | `0.6.4` |
 
 ### What changes in practice
 

{hypertopos-0.6.2 → hypertopos-0.6.4}/ROADMAP.md

@@ -9,6 +9,8 @@ Architecture-emergent analytics, properly enabled — calibration is now version
 - **0.6.0** — multi-epoch calibration retention, `compare_calibrations`, intrinsic / extrinsic drift decomposition, hidden-influencer matrix (`find_calibration_influencers`), cross-pattern temporal lead-lag (`find_lead_lag`), edge-derived event-pattern dimensions, joint density gap detection (`find_density_gaps`), declarative motif API (`find_motif_by_hops`, bounded MVP). Sphere format 2.4.
 - **0.6.1** — anchor-pattern aggregation of edge-derived dimensions (`edge_dim_aggregations:`); BFS-by-level enumerator on `find_motif_by_hops` with hop count cap raised 6→8 and global `time_window_hours`; declarative motif API completion via `HopPredicate.amount_ratio_to_prev` and `require_anomalous_entity`; functional anchor-companion + event-aware scoring on `find_motif_by_hops(score=True)`; broadcast-safe `Pattern.dim_labels` on patterns with aggregated edge-dim names. Additive only — no sphere format bump, no new tools.
 - **0.6.2** — chain-anchor `edge_dim_aggregations:` regime closes the third `anchor_kind` after single (account-style) and pair (k=2 composite); vectorized `AdjacencyIndex` load path (pyarrow `group_by` + lazy per-key materialization) for cold-start latency cleanup; column-selective `neighbors_*_window` accessors with motif enumerator migration for `bipartite_burst`, `fan_out`, `fan_in`, `cycle_2`; new project skill `profile-before-plan` gating hot-path refactors on real-data cProfile. Additive only — sphere format stays at 2.4 (the originally proposed persisted-artifact path was empirically refuted; vectorized loader ships without any on-disk change).
+- **0.6.3** — `edge_dim_aggregations:` expansion along four axes: three additional canonical aggregates per source dim (`_std`, `_p95`, `_count_above_threshold` with population p95 cutoff); k>2 composite anchor support (tripartite and beyond) via positional `key_cols` to edge endpoints; per-source-dim subset selector (`dims:` accepts list form for back-compat all-five sugar or mapping form `{dim: [agg, ...]}` for explicit per-dim subset); `_count_above_threshold` cutoffs persisted into the calibration epoch JSON with cross-epoch drift surfacing through `compare_calibrations.edge_dim_threshold_drift`. Spheres declaring `edge_dim_aggregations:` must rebuild; sphere format stays at 2.4.
+- **0.6.4** — chain-coherent investigative loop: `find_chains_with_coherent_anomaly` flags chains where consecutive entity-anchor positions are individually anomalous on the same dominant delta dim (chain composition vs chain shape — orthogonal to `find_anomalies` on the chain pattern); `anomaly_propagation_in_chain` returns the per-chain hop-by-hop anomaly trace; `classify_chain_typology` labels each chain along five operational axes (shape, peak_position, position_in_chain, extension_signals, dominant_top_dim); `extend_chain` suggests boundary extension entities via the chain reverse index. Plus `find_motif_by_hops.anomaly_seed_filter` parameter to prune the BFS frontier to anomalous entities, and a routing fix that drops the per-call scoped edge-table read in favour of the cached global adjacency. Critical correctness fix in parallel chain extraction: worker-local `chain_id` counters were producing colliding ids in the merged output — post-merge dedup now reassigns from the global merged index, restoring `primary_key` uniqueness in the chain pattern points table. Spheres built before the fix carry colliding chain_ids; rebuild required. Sphere format stays at 2.4.
 
 See [CHANGELOG](CHANGELOG.md) for the full feature list.
 

{hypertopos-0.6.2 → hypertopos-0.6.4}/SECURITY.md

@@ -1,6 +1,6 @@
 # Security
 
-## Current version: 0.6.2
+## Current version: 0.6.4
 
 hypertopos runs locally. No network services, no auth layer, no multi-tenancy.