hypertopos 0.7.2__tar.gz → 0.8.0__tar.gz

{hypertopos-0.7.2 → hypertopos-0.8.0}/CHANGELOG.md

@@ -7,6 +7,45 @@ Versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
 
+## [0.8.0] — 2026-05-30
+
+### Added
+
+- `hypertopos sphere health <path>` — composes the population summary with the geometric health checks into a single status (`ok` / `warning` / `critical`); `--exit-code-on-critical` exits `2` on a critical status so a CI shell gate can fail a deploy, and `--json` emits a `{status, sphere_path, overview, alerts}` document.
+- `hypertopos sphere validate <path> [--strict] [--json]` — structural integrity check over a built sphere (sphere.json parses, each materialized line has a points directory, each pattern has a geometry directory); `--strict` promotes calibration-health and dimension-quality warnings to errors, exiting `1` when invalid.
+- `hypertopos sphere diff <old> <new> [--json]` — pre-deploy diff reporting the pattern-inventory delta (added / removed / common) and per-shared-pattern calibration drift between two built spheres; patterns whose calibration schema differs are marked `not_comparable`.
+- `hypertopos sphere ingest <sphere> --points <file>` — incremental ingest CLI: append a new-/changed-entities table (Arrow IPC, Parquet, or CSV) to one pattern's geometry without a full rebuild. `--pattern` selects the target pattern (optional when the sphere has a single one); `--reindex` forces an immediate ANN index rebuild; `--finalize` recomputes the global rank percentile and rebuilds the ANN index once at the end of a batched ingest session; `--json` emits an `{added, modified, deleted, population_size, drift_pct, geometry_version_before, geometry_version_after, reindexed, finalized}` summary for pipeline consumption.
+- Production-ready incremental ingest: `GDSBuilder.incremental_update` adds, modifies, and deletes entities on an existing sphere without a full rebuild, and newly added entities are immediately visible to anomaly search. Patterns whose dimensions are relations, event dimensions, edge-dim aggregations, and tracked properties are now fully supported (previously such patterns raised a width-mismatch error). The appended rows now also enter the ANN vector index — pass `reindex=True` to rebuild it immediately. For batched ingestion of many small appends, pass `recompute_ranks=False` to defer the population percentile recompute and call the new `GDSBuilder.finalize_incremental` once at the end of the session.
+
+### Changed
+
+- Per-tag calibration-epoch timestamps now resolve directly from the dataset tag listing (a single lookup) instead of scanning version history, with read-side fallback for spheres written by older tooling.
+- Batch point reads (`read_points_batch`) issue a single indexed lookup over the whole key set on a reused dataset handle, replacing the per-key access loop — repeated lookups of the same entities are dramatically faster.
+- `pylance` dependency floor bumped to 7.x — Lance dataset reads, writes, scalar-index builds, and `merge_insert` upsert behavior verified equivalent to the prior 6.x floor.
+- Builder skips the trajectory ANN index when a pattern has too few rows to train it, shortening sphere builds for patterns with few entities; `attract_trajectory` (π10) and `find_drifting_similar` still return correct nearest-trajectory results via an exact full-scan fallback when no index is present.
+
+## [0.7.3] — 2026-05-27
+
+### Added
+
+- `GDSNavigator.π12_attract_regime_change` (and its cache fast-path) — each detected changepoint now carries a `near_data_boundary: bool` field. Fires `True` when the changepoint timestamp index falls in the first two or last two temporal buckets of the filtered range — agents can suppress these as low-confidence detections sitting on insufficient one-sided context (the buckets at the data range edges have asymmetric neighbours, which inflates the shift magnitude). Applies to both the cache fast-path (`_pi12_from_cache`) and the cold-scan path; no schema change.
+- `Pattern.signed_percentiles` — per-pattern percentile cache (`p1`, `p5`, `p50`, `p95`, `p99`) over the `delta_norm_signed` Lance column, populated at build time when label-aware calibration fires. Drives the new `signed_tail_concentration` dim_quality warning that flags one-sided extreme tails (`|p99| / max(|p50|, 1e-9) > 50`) on the Fisher LDA-projected delta distribution; suppressed when `Pattern.label_aware_n_pos < 30` to avoid noise from undersampled positive classes.
+- `GDSEngine.decompose_displacement(delta, label_direction)` — pure-math helper that splits a polygon delta vector along (`intrinsic`) and orthogonal to (`extrinsic`) any label-discriminating axis, satisfying `intrinsic^2 + extrinsic^2 == ||delta||^2`. Pattern-level means `intrinsic_displacement_mean` / `extrinsic_displacement_mean` are persisted on `Pattern` at build time when label-aware calibration fires.
+- `Pattern.label_aware_n_pos` / `Pattern.label_aware_n_neg` — positive / negative labelled sample counts captured at build time, co-populated with `label_aware_calibration`.
+- `GDSNavigator.chain_signed_confidence_rollup(chain_id, chain_pattern=..., anchor_pattern=...)` — chain-level reliability rollup derived from per-member `signed_confidence` ranking on the anchor pattern. Returns `chain_mean_signed_confidence`, `chain_n_low_confidence_members`, `chain_n_single_dim_driven_members`, and `chain_confidence_verdict` ∈ {high, medium, low, label-aware-unavailable}. Falls back cleanly with verdict `"label-aware-unavailable"` and null numeric fields when the anchor pattern lacks `label_aware_calibration`.
+- `GDSNavigator.aggregate` per-entity result rows gain `anomaly_rate: float | null` — fraction of an entity's events flagged anomalous (`delta_norm >= theta_diag`). `null` when entity has zero events or pattern lacks anomaly threshold.
+- `GDSNavigator.sphere_overview` gains `cross_pattern_discrepancy: dict | null` — pairwise Jaccard overlap of anomalous primary_keys across cover-overlapping patterns. Lists pairs with `pattern_a`, `pattern_b`, `shared_line`, the four anomaly-bucket counts, and `jaccard_anomaly_overlap`. `null` on spheres with fewer than two cover-overlapping patterns.
+- `GDSNavigator.find_anomalies(..., sample_size: int | None = None, boundary_aware: bool = False)` — opt-in sampling for the in-process scan path. `sample_size` caps geometry rows scored; `boundary_aware=True` allocates half the budget to entities within the `[0.8 * theta_norm, 1.2 * theta_norm]` band around the decision threshold, the other half drawn from the rest of the population. Defaults preserve the existing full-scan behaviour. `boundary_aware=True` requires `sample_size` to be set.
+- `engine.topology.classify_trajectory(solid_table, *, sample_size=10000)` — wraps `trajectory_continuous_score` with a deterministic 4-category classifier (`outlier` / `lagging` / `leading` / `typical`). Combines DTW distance against the population-median trajectory with a first-derivative slope comparison via `scipy.stats.linregress`. Returns a list of `{primary_key, dtw_distance, category, category_evidence}` per entity, with `category_evidence` carrying the signed deviation that drove the classification.
+- `GDSNavigator.classify_trajectory(primary_key, pattern_id, sample_size=10000)` — per-entity navigator wrapper that streams temporal `shape_snapshot` rows, projects to `delta_snapshot` via the pattern's `(mu, sigma_diag)`, and returns the classification for the requested entity.
+- `sphere.yaml` `graph_features.features:` accepts a new token — `community_id` — Louvain modularity-optimizing community detection on the anchor-pattern graph. `igraph.Graph.community_multilevel` when igraph is installed; `networkx.algorithms.community.louvain_communities` otherwise; gracefully emits an empty mapping (caller fills nulls) when neither backend is available. Community IDs are renumbered so the largest community is `0`.
+- `graph_features.features:` `pagerank` and `connected_component` now route through pure-numpy helpers (damped PageRank with damping `0.85`, max `100` iterations, L2 convergence threshold `1e-6`; Union-Find connected components with path compression and union-by-rank). No external graph library is required for these two; igraph remains optional and is used only for the other features in the block.
+- `engine.graph.compute_pagerank`, `compute_louvain_community`, `compute_connected_components`, `compute_from_adjacency` — pure helpers exposed for direct use on an `AdjacencyIndex`. PageRank parameters (`damping`, `max_iter`, `tol`) are configurable per call.
+- `GDSNavigator.find_calibration_influencers(..., auto_discover: bool = False, auto_k: int = 10)` — auto-discovery mode runs k-means on the pattern's geometry, surfaces top-K candidate influencers from cluster centroids, then ranks each by existing μ-impact algorithm. Returns the existing shape extended with per-candidate `cluster_size` and `cluster_centroid_distance`. Default `auto_discover=False` preserves manual mode requiring caller-supplied candidate_keys.
+- `GDSNavigator.calibration_influencer_history(primary_key, *, pattern_id)` — per-epoch μ-impact history for a known influencer. Lazily populated write-through cache at `_gds_meta/calibration_history/<pattern>/influencer_<primary_key>.json`. Returns chronological list of `{epoch, calibrated_at, mu_impact, delta_norm_impact}` entries. Empty list + warning when cache absent.
+- `engine.topology.local_trajectory_shape(delta_norms)` — pure-function classifier that categorises an entity's own `delta_norm` time-series as `arch` / `V` / `linear` / `flat` without requiring a population reference. Returns `None` when fewer than three samples are available. Used by `dive_solid` to surface trajectory shape on the entity's temporal slices.
+- `Navigator.find_similar_entities(..., with_neighbor_anomaly=False)` — opt-in flag triggers a single point lookup over the returned top-N neighbour keys and populates the `SimilarityResult.is_anomaly_map` sidecar with stored anomaly flags. Existing 2-tuple iteration is preserved.
+
 ## [0.7.2] — 2026-05-21
 
 ### Added

{hypertopos-0.7.2 → hypertopos-0.8.0}/LICENSE.md

@@ -26,7 +26,7 @@ For purposes of this License:
 - **Internal use** within an organization is not a competitive offering. Your
   organization includes entities under common control.
 
-**Change Date:** 2029-05-21
+**Change Date:** 2029-05-30
 
 **Change License:** Apache License, Version 2.0
 

{hypertopos-0.7.2 → hypertopos-0.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hypertopos
-Version: 0.7.2
+Version: 0.8.0
 Summary: Understand the structure of your data — without training machine learning models
 Project-URL: Homepage, https://github.com/hypertopos/hypertopos-py
 Project-URL: Documentation, https://github.com/hypertopos/hypertopos-py/tree/main/docs
@@ -14,7 +14,7 @@ Requires-Dist: numpy>=1.26
 Requires-Dist: packaging>=23.0
 Requires-Dist: pandas>=2.0
 Requires-Dist: pyarrow>=14.0
-Requires-Dist: pylance<7.0,>=6.0
+Requires-Dist: pylance<8.0,>=7.0
 Requires-Dist: pyyaml>=6.0
 Requires-Dist: scipy>=1.11
 Provides-Extra: dev

{hypertopos-0.7.2 → hypertopos-0.8.0}/README.md

@@ -9,7 +9,7 @@
 [![PyArrow](https://img.shields.io/badge/format-PyArrow-red.svg)](https://arrow.apache.org/docs/python/)
 [![Lance](https://img.shields.io/badge/storage-Lance-blueviolet.svg)](https://github.com/lance-format/lance)
 [![MCP](https://img.shields.io/badge/MCP-compatible-green.svg)](https://modelcontextprotocol.io)
-[![Version](https://img.shields.io/badge/version-0.7.2-%235500FF.svg)](pyproject.toml)
+[![Version](https://img.shields.io/badge/version-0.8.0-%235500FF.svg)](pyproject.toml)
 
 hypertopos is not a database, and not a machine learning model. It is a layer that turns relational data into a coordinate system where every entity gets a position derived from its relationships and the population around it.
 
@@ -85,6 +85,10 @@ Each capability below emerges from treating entities as points in a shared, popu
 | Counterfactual: dimension override | `simulate_dimension_change(primary_key, pattern_id, line_id, set_dimension)` — what-if recomputation of `delta_norm` under hypothetical raw shape-vector values for one or more dimensions; reports the before/after norms, the anomaly-flag flip, and the new top witness dims. Companion to `simulate_edge_removal` for non-edge dimensions | Manual reconstruction of `shape = delta * sigma + mu`, point-edit, and norm recompute against the right cholesky path | `0.7.1` |
 | Multi-hypothesis anomaly explainer | `find_diverse_explanations(primary_key, pattern_id, n_hypotheses=3)` — K strict-disjoint dim sets explaining the anomaly via greedy selection over per-dim Bregman contributions (each dim in at most one hypothesis); graceful degradation when an entity is single-dim driven; `validate=True` opt-in counterfactual confirmation | Single explanation from `explain_anomaly` only — investigator manually picks "what else could explain this" from the full dim ranking | `0.7.0` |
 | Declarative compliance rules + violations query | `Pattern.conformance_rules` (AST predicate language in `sphere.yaml`) + `find_conformance_violations(pattern_id, rule_id, severity_min, top_n)` — builder evaluates rules vectorized at build time, sidecar Lance dataset persists violations, query primitive returns ranked violations with sub-second filter pushdown | Manual SQL ad-hoc on entity table + bespoke compliance scripts; no integration with anomaly geometry | `0.7.0` |
+| Incremental ingest | `GDSBuilder.incremental_update` adds, modifies, and deletes entities on an existing sphere without a full rebuild; new entities are immediately visible to anomaly search; `reindex=True` rebuilds the ANN vector index, and `recompute_ranks=False` + `GDSBuilder.finalize_incremental` batch the percentile recompute across many small appends. Works on the existing sphere format — no migration | Full rebuild from source on every new batch of entities | `0.8.0` |
+| Agent-correctness reports | Five composer tools synthesise a verdict an agent can act on: `assess_anomaly_certainty` (calibrated certainty + rationale), `consensus_classification` (multi-detector agreement), `calibration_drift_report` (cross-epoch drift verdict), `diverse_explanations` (multi-cause robustness verdict), `theta_sensitivity_report` (recalibration-safety verdict) | Manual stitching of several lower-level tool calls plus bespoke interpretation logic | `0.8.0` |
+| Vector-index health observability | `audit_pattern_dims` returns a `vector_index_health` block (index presence, indexed/unindexed row counts, staleness flag, reindex recommendation) and `check_alerts` emits a `stale_vector_index` alert, so an agent knows whether recently-added entities are covered by the ANN index before relying on trajectory tools | No agent-visible signal that the ANN index lagged behind incrementally-added rows | `0.8.0` |
+| Cloud-ops CLI | `hypertopos sphere health` (composes population summary + geometric checks into an `ok`/`warning`/`critical` status with `--exit-code-on-critical` for CI gates and `--json` output), `hypertopos sphere validate [--strict] [--json]` (structural integrity check), and `hypertopos sphere diff [--json]` (pattern-inventory + calibration drift between two built spheres) | Hand-rolled validation scripts and no built-in CI deploy gate | `0.8.0` |
 
 ### What changes in practice
 

{hypertopos-0.7.2 → hypertopos-0.8.0}/ROADMAP.md

@@ -1,14 +1,24 @@
 # hypertopos — Roadmap
 
 > Planned direction. Priorities may change based on feedback.
+> See [CHANGELOG](CHANGELOG.md) for the full feature list per release.
+
+## 0.8.0
+
+Lance 7.0 single-process performance, incremental ingest, agent-correctness composers, and operations tooling — on the existing sphere format 3.x, no rebuild required.
+
+- **0.8.0** — Lance 7.0 storage with native batched reads; incremental sphere ingest (`hypertopos sphere ingest`) without a full rebuild; agent-correctness composers (`assess_anomaly_certainty`, `consensus_classification`, `calibration_drift_report`, `diverse_explanations`, `theta_sensitivity_report`); vector-index health in `audit_pattern_dims`; cloud-ops CLI (`sphere health` / `validate` / `diff`); trajectory ANN index auto-skipped on small populations.
+
+---
 
 ## 0.7.0
 
 Detector composition and counterfactual primitives on native Lance MVCC — multi-detector consensus, per-edge / joint / per-counterparty influence math, topological cycle persistence, multi-resolution FDR, and the entity-side investigation orchestrator land on a modernized Lance 6.0 storage substrate with sphere format 3.0.
 
-- **0.7.0** — Lance 4.0 → 6.0 bump; sphere format 2.4 → 3.0 (breaking — single Lance dataset per pattern, calibration epochs tagged `epoch_<N>`, no migration tool); Wilson HMP combiner + `combine_anomaly_pvalues` + `classify_detector_consensus` over five orthogonal anomaly axes; counterfactual family (`simulate_edge_removal`, `select_minimal_joint_edge_removal`, `simulate_counterparty_removal`); `find_topological_anomalies` H_1 cycle persistence + `find_topological_trajectory_anomalies`; `find_graph_geometry_tension` 2×2 behavioural-vs-graph cross-tab; multi-resolution FDR with `Pattern.fdr_hierarchy` + `fdr_temporal_hierarchy`; per-dim FDR axis + `rank_by="min_q_per_dim"`; `investigate_entity` orchestrator; `edge_curvature_frc` Forman-Ricci edge dimension; `reliability_flags` triage on five primitive outputs; `composite_risk` switches Fisher → HMP.
-
-See [CHANGELOG](CHANGELOG.md) for the full feature list.
+- **0.7.0** — Lance 6.0 + sphere format 3.0; detector consensus (Wilson HMP), counterfactual family, topological cycle persistence, multi-resolution FDR, `investigate_entity` orchestrator.
+- **0.7.1** — label-aware calibration foundation: `label_audit:` YAML, per-class Cohen's d + Fisher LDA on `Pattern`, `delta_norm_signed` Lance column.
+- **0.7.2** — composition orchestrators: `find_anomalies(rank_by="signed_confidence")`, `chain_full_loop_summary`, `audit_label_alignment`, counterfactual frozen-population on `dive_solid`.
+- **0.7.3** — investigation-orchestrator surface: per-dim AUROC, intrinsic/extrinsic decomposition, chain reliability rollup, DTW trajectory classifier, Louvain `community_id`, calibration-influencer auto-discovery, MCP ergonomics on `passive_scan` / `find_similar_entities` / `dive_solid`.
 
 ---
 
@@ -25,8 +35,6 @@ Architecture-emergent analytics, properly enabled — calibration is now version
 - **0.6.6** — agent-side close of the chain investigative loop: smart-mode chain routing, `sphere_overview.suggested_queries` entry-point, R9 cheatsheet; `theta_sensitivity` calibration introspection; length- and volume-stratified correlation gates.
 - **0.6.7** — close the investigation→SAR pipeline: `dimension_weights` runtime override on `find_anomalies`, `chain_investigation_summary` triage, `investigate_chain` one-shot R9 orchestrator, `generate_sar_rationale` template-based SAR narrative draft; `dim_quality_warnings` (dead / sparse dim) on `sphere_overview`; external-chains-as-anchor-line cookbook.
 
-See [CHANGELOG](CHANGELOG.md) for the full feature list.
-
 ---
 
 ## 0.5.0
@@ -77,12 +85,19 @@ First public release — full GDS stack, π1–π12, builder, MCP server, valida
 
 ## Future
 
+> Next major direction (planned): **0.9.0 — cloud foundations**, making hypertopos hostable. Priorities may change based on feedback.
+
+**Cloud foundations (0.9.0 direction)**
+- Session-keyed MCP server state with per-session isolation, so concurrent HTTP clients and multi-tenant hosting are safe.
+- URI-addressable object storage (S3 / ADLS / GCS) for spheres, with a credential profile and a read-path prefetch cache.
+- Cold-open scalar-index caches for faster first-query latency, with no on-disk format change.
+- Production-path architecture decision records (data freshness, multi-tenancy, permission passthrough) and a committed surface model: MCP for agents, library API for embedding, CLI for ops / batch / cloud-compute.
+
+**Adoption & product**
+- `scan <file>` auto-builder for one-command onboarding, plus a deterministic (no-LLM) interpretation layer producing human-readable, audit-ready anomaly explanations.
+
 **Detection quality**
 - Robust estimators, multi-scale resolution — trimmed means / MAD-based σ, hierarchical build at daily/weekly/full granularity.
-- Chains as first-class geometric entities — external chains declared as anchor lines with a membership table, plus chain-aware geometric primitives (anomaly propagation inside a chain, coherent-anomaly filter, chain drift trajectory).
-
-**Builder evolution**
-- Incremental rebuild — geometry-only without `--force` wipe.
 
 **PassiveScanner evolution**
 - Native temporal source support for direct temporal inputs, without requiring manual dataset plumbing in benchmark scripts.
@@ -90,7 +105,7 @@ First public release — full GDS stack, π1–π12, builder, MCP server, valida
 - **SphereProfiler** — autonomous sphere scanner that profiles all patterns, runs calibration sweeps across source combinations, proposes optimal PassiveScanner composition for Layer 1 surveillance.
 
 **Code refactoring**
-- Break up oversized modules into smaller, domain-focused components (post-0.5.0 `navigator.py` is over 11k lines and is the obvious first target; per-pattern parallelism and AdjacencyIndex reuse in 0.4.0/0.5.0 reduced some coupling but the module itself did not shrink).
+- Break up oversized modules into smaller, domain-focused components — `navigator.py` is now well over 20k lines and is the primary target for a mixin-based split that preserves the public single-class API.
 - Consolidate repeated orchestration logic into shared helpers.
 
 **Cross-sphere capabilities**

{hypertopos-0.7.2 → hypertopos-0.8.0}/SECURITY.md

@@ -1,6 +1,6 @@
 # Security
 
-## Current version: 0.7.2
+## Current version: 0.8.0
 
 hypertopos runs locally. No network services, no auth layer, no multi-tenancy.
 

{hypertopos-0.7.2 → hypertopos-0.8.0}/docs/api-reference.md

@@ -218,7 +218,8 @@ is the always-available primary path.
 | `anomaly_summary(pattern_id, max_clusters)` | Anomaly population breakdown with geometric clustering |
 | `aggregate_anomalies(pattern_id, group_by)` | Group anomalies by a property column with per-group rates |
 | `aggregate(event_pattern_id, group_by_line)` | Aggregate event polygons by group with metric computation |
-| `check_alerts(pattern_id=None)` | Implicit health checks: anomaly rate spikes, population shocks, calibration staleness |
+| `check_alerts(pattern_id=None)` | Implicit health checks: anomaly rate spikes, population shocks, calibration staleness, stale ANN vector index |
+| `vector_index_health(pattern_id, line_id=None)` | ANN (IVF) index staleness for a pattern's geometry: indexed/unindexed row counts, `is_stale`, recommendation (metadata-only read) |
 | `hub_score_stats(pattern_id)` | Hub score distribution statistics |
 | `check_anomaly_batch(pattern_id, primary_keys)` | Batch anomaly status check for multiple entities |
 | `temporal_quality_summary(pattern_id)` | Temporal anomaly persistence metrics |
@@ -256,7 +257,8 @@ builder = GDSBuilder(
 | `add_chain_line(line_id, chains, features)` | Create anchor line from extracted chain dicts |
 | `add_alias(alias_id, base_pattern_id, cutting_plane_dimension, cutting_plane_threshold)` | Register an alias with a cutting plane for sub-population analysis |
 | `build(temporal_configs=None)` | Validate, compute statistics, write all files. Pass `temporal_configs` to run geometry→temporal pipeline per pattern. Returns output path |
-| `incremental_update(pattern_id, changed_entities, deleted_keys)` | Update geometry incrementally with drift tracking |
+| `incremental_update(pattern_id, changed_entities, deleted_keys, recalibrate, reindex, recompute_ranks)` | Add/modify/delete entities without a full rebuild; new entities are immediately visible to anomaly search. Supports patterns whose dimensions are relations, event dimensions, edge-dim aggregations, and tracked properties. `reindex=True` rebuilds the ANN index immediately; `recompute_ranks=False` defers the global `delta_rank_pct` recompute for batched ingestion |
+| `finalize_incremental(pattern_id)` | Recompute the global `delta_rank_pct` percentile once and rebuild the ANN index. Call at the end of a batched ingestion session that used `recompute_ranks=False` |
 | `build_temporal(time_col, time_window)` | Generate temporal snapshots from time-windowed event data. Call after `build()` when not using pipeline mode |
 
 ### RelationSpec

{hypertopos-0.7.2 → hypertopos-0.8.0}/docs/configuration.md

@@ -60,6 +60,77 @@ Print summary of a built sphere.
 hypertopos info my_sphere/
 ```
 
+### `hypertopos sphere` — cloud-ops verbs
+
+Cloud-ops checks over a **built** sphere directory (the one `info` reads),
+shaped for CI gates and pre-deploy pipelines. Each verb supports `--json`
+for machine-readable output; in JSON mode stdout carries only the JSON
+document (diagnostics and errors go to stderr).
+
+#### `hypertopos sphere health`
+
+```bash
+hypertopos sphere health my_sphere/ [--json] [--exit-code-on-critical]
+```
+
+Composes the population summary (per-pattern entity count, anomaly rate,
+calibration health) with the geometric health checks. The derived `status`
+is `critical` when any HIGH-severity alert is present, `warning` for
+MEDIUM-only, else `ok`. With `--exit-code-on-critical` the process exits `2`
+on a critical status (and `0` otherwise) so a `set -e` shell gate can fail a
+deploy. JSON payload: `{status, sphere_path, overview, alerts}`.
+
+#### `hypertopos sphere validate`
+
+```bash
+hypertopos sphere validate my_sphere/ [--strict] [--json]
+```
+
+Structural integrity check over a built sphere: `sphere.json` parses, each
+materialized line has a `points/` directory, each pattern has a `geometry/`
+directory. `--strict` additionally promotes calibration-health (`suspect` /
+`poor`) and dimension-quality warnings to errors. Exits `0` when valid, `1`
+when invalid. JSON payload: `{valid, strict, errors, warnings}`.
+
+> This validates a **built** sphere on disk; `hypertopos validate --config`
+> validates a `sphere.yaml` before building. They are distinct commands.
+
+#### `hypertopos sphere diff`
+
+```bash
+hypertopos sphere diff old_sphere/ new_sphere/ [--json]
+```
+
+Pre-deploy diff between two built spheres: pattern-inventory delta (`added` /
+`removed` / `common` pattern ids) and per-shared-pattern calibration drift
+(`overall_drift_rms` plus the top-drifted dimensions, comparing each sphere's
+latest calibration epoch). Patterns whose calibration schema differs between
+the two spheres are marked `not_comparable` rather than crashing. Identical
+paths yield `identical: true`.
+
+#### `hypertopos sphere ingest`
+
+```bash
+hypertopos sphere ingest my_sphere/ --points changed.arrow \
+  [--pattern ID] [--recalibrate {auto,force,never}] \
+  [--reindex] [--finalize] [--json]
+```
+
+Incrementally appends a new-/changed-entities table to one pattern's geometry
+without a full rebuild. The points table may be Arrow IPC (`.arrow` /
+`.arrows`), Parquet (`.parquet` / `.pq`), or CSV (`.csv` / `.csv.gz`) and must
+carry a `primary_key` column plus the columns the pattern's dimensions
+reconstruct from (relations, event dimensions, edge-dim aggregations, or
+property columns). `--pattern` is optional when the sphere has a single
+pattern and required otherwise. `--reindex` forces an ANN index rebuild so the
+appended rows are immediately searchable; `--finalize` recomputes the global
+`delta_rank_pct` and rebuilds the ANN index once at the end of a batched
+ingest session. JSON payload: `{pattern_id, added, modified, deleted,
+population_size, drift_pct, geometry_version_before, geometry_version_after,
+reindexed, finalized}`. Patterns carrying generalized dimension blocks (geo /
+metric / semantic) cannot be reconstructed incrementally and exit 1 with a
+clear message directing you to a full rebuild.
+
 ---
 
 ## YAML Reference — `sphere.yaml`

hypertopos-0.8.0/docs/container-deployment.md

@@ -0,0 +1,112 @@
+# Container Deployment
+
+> Running hypertopos CLI verbs as one-shot container jobs.
+
+The `hypertopos` CLI is designed to run as a one-shot command inside a
+container — build a sphere, gate a deploy on its health, or diff two
+spheres before promoting one to production. This page covers the
+`ENTRYPOINT ["hypertopos"]` contract and the cloud-ops verbs that pair
+with it.
+
+## ENTRYPOINT contract
+
+Set the console script as the image entrypoint and pass the verb plus its
+arguments as the container command:
+
+```dockerfile
+FROM python:3.11-slim
+
+RUN pip install --no-cache-dir hypertopos
+
+ENTRYPOINT ["hypertopos"]
+```
+
+With `ENTRYPOINT ["hypertopos"]`, the container command becomes the verb
+and its flags:
+
+```bash
+# Build a sphere from a mounted config
+docker run --rm -v "$PWD:/work" -w /work my-image \
+  build --config sphere.yaml --output gds_sales/
+
+# Health-check a built sphere, failing the run on a critical status
+docker run --rm -v "$PWD:/work" -w /work my-image \
+  sphere health gds_sales/ --exit-code-on-critical
+```
+
+Because each invocation is a single short-lived process, no long-running
+server is started — the container runs the verb, prints its result, and
+exits with a meaningful status code.
+
+## Exit codes
+
+The cloud-ops verbs return process exit codes that shell gates and
+orchestrators (`set -e`, Kubernetes Job `backoffLimit`, Airflow
+`BashOperator`) can act on directly:
+
+| Verb | Exit 0 | Exit 1 | Exit 2 |
+|------|--------|--------|--------|
+| `sphere health … --exit-code-on-critical` | status `ok` / `warning` | path is not a sphere | status `critical` (HIGH alert) |
+| `sphere validate …` | sphere valid | sphere invalid or path not a sphere | — |
+| `sphere diff <old> <new>` | diff produced | either path not a sphere | — |
+| `sphere ingest <path> --points <file>` | rows ingested | path not a sphere, points file missing/unsupported, unknown pattern, or unreconstructable geometry | — |
+
+Without `--exit-code-on-critical`, `sphere health` always exits 0 on a
+readable sphere (a non-sphere path still exits 1) — useful when you only
+want to capture the JSON report and decide what to do with it yourself.
+
+## JSON output for pipelines
+
+Every cloud-ops verb accepts `--json`. In JSON mode, stdout carries only
+the JSON document — human-readable diagnostics and error messages go to
+stderr — so a pipeline step can parse the result without scraping log
+lines:
+
+```bash
+# Capture the health report, branch on the status field
+status=$(hypertopos sphere health gds_sales/ --json | jq -r .status)
+if [ "$status" = "critical" ]; then
+  echo "Refusing to deploy a critical sphere" >&2
+  exit 1
+fi
+```
+
+```bash
+# Pre-deploy: diff the candidate sphere against the live one
+hypertopos sphere diff gds_live/ gds_candidate/ --json > diff.json
+jq '.pattern_inventory' diff.json
+```
+
+```bash
+# Incremental ingest: append a changed-entities table to one pattern's
+# geometry, then finalize the rank recompute + ANN rebuild once at the end
+# of the batch. --pattern is optional when the sphere has a single pattern.
+hypertopos sphere ingest gds_sales/ \
+  --points changed_customers.arrow --finalize --json | jq '{added, population_size}'
+```
+
+The points table may be Arrow IPC (`.arrow`/`.arrows`), Parquet
+(`.parquet`/`.pq`), or CSV (`.csv`/`.csv.gz`) and must carry a
+`primary_key` column. The JSON summary reports `added`, `modified`,
+`deleted`, `population_size`, the geometry Lance dataset version
+before/after, and whether the ANN index was rebuilt (`reindexed`) and the
+batch finalized (`finalized`).
+
+## CI gate example
+
+A health check as a deploy gate inside a CI job:
+
+```yaml
+- name: Gate on sphere health
+  run: hypertopos sphere health gds_sales/ --exit-code-on-critical
+```
+
+The step fails (exit 2) and stops the pipeline when the sphere has a
+critical alert, and passes otherwise.
+
+## See also
+
+- [Configuration](configuration.md) — the full CLI command reference,
+  including `build`, `validate`, `info`, and the `sphere` cloud-ops verbs.
+- [Quick Start](quickstart.md) — building a sphere from a relational
+  dataset.

{hypertopos-0.7.2 → hypertopos-0.8.0}/docs/data-format.md

@@ -15,7 +15,7 @@ gds_{sphere_id}/
 ├── _gds_meta/
 │   ├── sphere.json              # central config (lines, patterns, aliases, storage)
 │   ├── geometry_stats/          # precomputed population summaries
-│   ├── trajectory/              # ANN index for trajectory similarity search
+│   ├── trajectory/              # per-entity trajectory summary vectors (optional IVF_FLAT ANN index — skipped for too-few-entity patterns, which then use a brute-force scan)
 │   ├── temporal_centroids/      # cached population centroids per time window
 │   ├── edge_stats/              # per-event-pattern edge table summary cache (row count, unique from/to, ts/amount range)
 │   ├── edge_features/           # per-edge derived dim sidecar (event patterns with edge_dimensions: in YAML)

{hypertopos-0.7.2 → hypertopos-0.8.0}/hypertopos/__init__.py

@@ -44,4 +44,4 @@ __all__ = [
     "IntrinsicExtrinsicReport",
     "LeadLagReport",
 ]
-__version__ = "0.7.2"
+__version__ = "0.8.0"