vector-engine 1.1.1__tar.gz → 1.2.0__tar.gz

vector_engine-1.2.0/CITATION.cff

@@ -0,0 +1,11 @@
+cff-version: 1.2.0
+title: "Vector Engine"
+message: "If you use Vector Engine in research, please cite this project."
+type: software
+authors:
+  - family-names: "Panchal"
+    given-names: "Neel"
+repository-code: "https://github.com/neelpanchal11/Vector-Engine"
+license: MIT
+version: "1.2.0"
+date-released: 2026-08-13

vector_engine-1.2.0/MANIFEST.in

@@ -0,0 +1,4 @@
+include CITATION.cff
+recursive-include docs/releases *.md
+recursive-include notebooks *.ipynb
+recursive-include scripts *.py

{vector_engine-1.1.1 → vector_engine-1.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: vector-engine
-Version: 1.1.1
+Version: 1.2.0
 Summary: ML-first vector computation and retrieval engine.
 Author: Neel Panchal
 License-Expression: MIT
@@ -30,7 +30,7 @@ Provides-Extra: dev
 Requires-Dist: pytest<9,>=8.2; extra == "dev"
 Dynamic: license-file
 
-# Vector Engine v1.1.1
+# Vector Engine v1.2.0
 
 Reproducibility-first vector retrieval toolkit for local ML and IR workflows.
 
@@ -116,6 +116,7 @@ Each notebook opens directly from this repo, so it always matches the current re
 - Semantic search: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neelpanchal11/Vector-Engine/blob/main/notebooks/01_semantic_search.ipynb)
 - kNN baseline classifier: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neelpanchal11/Vector-Engine/blob/main/notebooks/02_knn_baseline.ipynb)
 - Item-item recommender similarity: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neelpanchal11/Vector-Engine/blob/main/notebooks/03_recommender_similarity.ipynb)
+- IVF ANN backend (recall vs. speed): [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neelpanchal11/Vector-Engine/blob/main/notebooks/05_ann_backend.ipynb)
 
 `notebooks/04_stability_runs.ipynb` analyzes locally generated benchmark artifacts (see [Reproducibility and Evidence](#reproducibility-and-evidence)) and is not a standalone Colab demo.
 
@@ -180,13 +181,16 @@ Bundle outputs include:
 
 ## Backends
 
-| Backend | Search | Add | Save/Load | Custom Metric |
-| --- | ---: | ---: | ---: | ---: |
-| `bruteforce` | yes | yes | yes | yes |
-| `faiss` | yes | yes | yes | no |
+| Backend | Search | Add | Save/Load | Custom Metric | Deps |
+| --- | ---: | ---: | ---: | ---: | --- |
+| `bruteforce` | yes | yes | yes | yes | numpy only |
+| `ivf` | yes | yes | yes | yes | numpy only |
+| `faiss` | yes | yes | yes | no | `faiss-cpu` (optional; no wheel on macOS arm64) |
 
 FAISS is optional. The required reproducibility path is bruteforce-safe.
 
+`ivf` is a pure-numpy approximate backend (coarse-quantize with k-means, probe the `nprobe` nearest clusters at search time) — it needs no extra install and works on hosts where `faiss-cpu` has no wheel. Tune it via `backend_config={"n_clusters": ..., "nprobe": ...}`. Search batches queries by probed cluster to reduce per-query Python overhead. It still trades recall for speed as `nprobe` changes; see `docs/releases/v1.2.0.md` for the original recall/latency sweep and its implementation details.
+
 ## Reproducibility and Evidence
 
 Recommended release evidence flow:
@@ -194,6 +198,8 @@ Recommended release evidence flow:
 ```bash
 python scripts/repro_smoke.py --output-dir artifacts/repro_smoke
 python scripts/benchmark_matrix.py --mode exact --warmup 2 --loops 8 --seed 7 --output-dir artifacts/benchmark_matrix
+python scripts/benchmark_ivf_batching.py --n 5000 --d 64 --nq 200 --k 10 --n-clusters 32 --nprobe-options 1,4,16,32 --loops 5 --warmup 1 --seed 22 --output artifacts/ivf_benchmark/ivf_batching_comparison.json
+python scripts/benchmark_ivf.py --n 5000 --d 64 --nq 100 --k 10 --n-clusters 32 --nprobe-options 1,4,8,16,32 --loops 3 --seed 7 --output artifacts/ivf_benchmark/ivf_batched_recall_sweep.json
 python scripts/publishable_results.py --matrix-summary artifacts/benchmark_matrix/matrix_summary.json --stability-summary artifacts/testing_runs/stability_summary_bruteforce_200.json --output artifacts/benchmark_matrix/publishable_results.v1.json
 python scripts/credibility_audit.py --matrix-summary artifacts/benchmark_matrix/matrix_summary.json --stability-summary artifacts/testing_runs/stability_summary_bruteforce_200.json --publishable-summary artifacts/benchmark_matrix/publishable_results.v1.json --output artifacts/audit/credibility_audit.v1.json
 ```
@@ -206,6 +212,7 @@ python scripts/credibility_audit.py --matrix-summary artifacts/benchmark_matrix/
 - `notebooks/01_semantic_search.ipynb`
 - `notebooks/02_knn_baseline.ipynb`
 - `notebooks/03_recommender_similarity.ipynb`
+- `notebooks/05_ann_backend.ipynb`
 
 ## Troubleshooting
 
@@ -216,6 +223,7 @@ python scripts/credibility_audit.py --matrix-summary artifacts/benchmark_matrix/
 
 ## Project Links
 
+- `docs/releases/v1.2.0.md`
 - `docs/releases/v1.1.0.md`
 - `docs/releases/v1.1.0-checklist.md`
 - `docs/reproducibility.md`

{vector_engine-1.1.1 → vector_engine-1.2.0}/README.md

@@ -1,4 +1,4 @@
-# Vector Engine v1.1.1
+# Vector Engine v1.2.0
 
 Reproducibility-first vector retrieval toolkit for local ML and IR workflows.
 
@@ -84,6 +84,7 @@ Each notebook opens directly from this repo, so it always matches the current re
 - Semantic search: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neelpanchal11/Vector-Engine/blob/main/notebooks/01_semantic_search.ipynb)
 - kNN baseline classifier: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neelpanchal11/Vector-Engine/blob/main/notebooks/02_knn_baseline.ipynb)
 - Item-item recommender similarity: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neelpanchal11/Vector-Engine/blob/main/notebooks/03_recommender_similarity.ipynb)
+- IVF ANN backend (recall vs. speed): [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neelpanchal11/Vector-Engine/blob/main/notebooks/05_ann_backend.ipynb)
 
 `notebooks/04_stability_runs.ipynb` analyzes locally generated benchmark artifacts (see [Reproducibility and Evidence](#reproducibility-and-evidence)) and is not a standalone Colab demo.
 
@@ -148,13 +149,16 @@ Bundle outputs include:
 
 ## Backends
 
-| Backend | Search | Add | Save/Load | Custom Metric |
-| --- | ---: | ---: | ---: | ---: |
-| `bruteforce` | yes | yes | yes | yes |
-| `faiss` | yes | yes | yes | no |
+| Backend | Search | Add | Save/Load | Custom Metric | Deps |
+| --- | ---: | ---: | ---: | ---: | --- |
+| `bruteforce` | yes | yes | yes | yes | numpy only |
+| `ivf` | yes | yes | yes | yes | numpy only |
+| `faiss` | yes | yes | yes | no | `faiss-cpu` (optional; no wheel on macOS arm64) |
 
 FAISS is optional. The required reproducibility path is bruteforce-safe.
 
+`ivf` is a pure-numpy approximate backend (coarse-quantize with k-means, probe the `nprobe` nearest clusters at search time) — it needs no extra install and works on hosts where `faiss-cpu` has no wheel. Tune it via `backend_config={"n_clusters": ..., "nprobe": ...}`. Search batches queries by probed cluster to reduce per-query Python overhead. It still trades recall for speed as `nprobe` changes; see `docs/releases/v1.2.0.md` for the original recall/latency sweep and its implementation details.
+
 ## Reproducibility and Evidence
 
 Recommended release evidence flow:
@@ -162,6 +166,8 @@ Recommended release evidence flow:
 ```bash
 python scripts/repro_smoke.py --output-dir artifacts/repro_smoke
 python scripts/benchmark_matrix.py --mode exact --warmup 2 --loops 8 --seed 7 --output-dir artifacts/benchmark_matrix
+python scripts/benchmark_ivf_batching.py --n 5000 --d 64 --nq 200 --k 10 --n-clusters 32 --nprobe-options 1,4,16,32 --loops 5 --warmup 1 --seed 22 --output artifacts/ivf_benchmark/ivf_batching_comparison.json
+python scripts/benchmark_ivf.py --n 5000 --d 64 --nq 100 --k 10 --n-clusters 32 --nprobe-options 1,4,8,16,32 --loops 3 --seed 7 --output artifacts/ivf_benchmark/ivf_batched_recall_sweep.json
 python scripts/publishable_results.py --matrix-summary artifacts/benchmark_matrix/matrix_summary.json --stability-summary artifacts/testing_runs/stability_summary_bruteforce_200.json --output artifacts/benchmark_matrix/publishable_results.v1.json
 python scripts/credibility_audit.py --matrix-summary artifacts/benchmark_matrix/matrix_summary.json --stability-summary artifacts/testing_runs/stability_summary_bruteforce_200.json --publishable-summary artifacts/benchmark_matrix/publishable_results.v1.json --output artifacts/audit/credibility_audit.v1.json
 ```
@@ -174,6 +180,7 @@ python scripts/credibility_audit.py --matrix-summary artifacts/benchmark_matrix/
 - `notebooks/01_semantic_search.ipynb`
 - `notebooks/02_knn_baseline.ipynb`
 - `notebooks/03_recommender_similarity.ipynb`
+- `notebooks/05_ann_backend.ipynb`
 
 ## Troubleshooting
 
@@ -184,6 +191,7 @@ python scripts/credibility_audit.py --matrix-summary artifacts/benchmark_matrix/
 
 ## Project Links
 
+- `docs/releases/v1.2.0.md`
 - `docs/releases/v1.1.0.md`
 - `docs/releases/v1.1.0-checklist.md`
 - `docs/reproducibility.md`

vector_engine-1.2.0/docs/releases/v0.2.0-alpha.md

@@ -0,0 +1,61 @@
+# Vector Engine v0.2.0-alpha (Draft)
+
+## Highlights
+
+- Expanded clustering API with richer outputs:
+  - labels
+  - centers
+  - inertia
+  - iteration count
+- Hard-negative mining strategies:
+  - `top1`
+  - `topk_sample`
+  - `distance_band`
+  - plus exclusion support (`exclude_ids`, `exclude_mask`)
+- Retrieval evaluation enhancements:
+  - `retrieval_report_detailed` (summary + per-query)
+  - `batch_metrics_summary` for multi-run aggregation
+- Real-corpus evaluation script:
+  - `scripts/rag_real_corpus_eval.py`
+  - supports retrieval thresholds and latency gates
+- Public demo template included in `demo_repo_template/`
+
+## Validation Snapshot
+
+- RAG baseline artifact generation validated.
+- Real-corpus style 3-run reports generated in `artifacts/real_corpus_runs/`:
+  - quality stable across runs (`recall@1/3/6 = 1.0`, `ndcg@1/3/6 = 1.0`)
+  - p95 latency envelope: `0.0376-0.0717 ms`
+  - thresholds pass in all runs (`recall >= 0.75`, `ndcg >= 0.70`, `p95 <= 120 ms`)
+- Faiss Flat exact-equivalence checks generated in `artifacts/faiss_equivalence/`:
+  - `overlap_vs_bruteforce = 1.0` in all 3 runs with `--min-flat-overlap 0.99`
+  - p95 latency ranges:
+    - bruteforce: `29.99-37.63 ms`
+    - faiss_flat: `4.17-15.03 ms`
+- 200-run stability study generated:
+  - `artifacts/testing_runs/stability_runs_bruteforce_200.jsonl`
+  - `artifacts/testing_runs/stability_summary_bruteforce_200.json`
+  - p95 mean `0.0255 ms` (95% interval `0.0203-0.0547 ms`)
+  - qps mean `188,097` (95% interval `117,499-214,111`)
+- Stability analysis notebook: `notebooks/04_stability_runs.ipynb`.
+
+## Known Limitations
+
+- Real-corpus evaluation requires user-provided embeddings/ground-truth files.
+- Public demo template is included locally; publish as separate repo for outreach.
+- Mock/public-safe corpus numbers are not a substitute for production-scale private corpus benchmarks.
+
+## Release Checklist
+
+1. Run `python3 -m pytest -q`
+2. Run `python3 scripts/rag_baseline.py --output-dir artifacts --k 3`
+3. Run exact equivalence:
+   - `python3 benchmarks/compare_bruteforce_vs_faiss.py --mode exact --min-flat-overlap 0.99 --output artifacts/benchmark_exact.json`
+4. Run real corpus eval:
+   - `python3 scripts/rag_real_corpus_eval.py --embeddings ... --query-embeddings ... --ids ... --ground-truth ...`
+5. Run stability study:
+   - `python3 scripts/stability_runs.py --embeddings ... --query-embeddings ... --ids ... --ground-truth ... --backend bruteforce --run-count 200 --output-dir artifacts/testing_runs`
+6. Update README benchmark and quality numbers
+7. Tag and release:
+   - `git tag -a v0.2.0-alpha -m "Vector Engine v0.2.0-alpha"`
+   - `git push origin v0.2.0-alpha`

vector_engine-1.2.0/docs/releases/v0.3.0.md

@@ -0,0 +1,57 @@
+# Vector Engine v0.3.0 (Draft)
+
+## Release Positioning
+
+v0.3.0 is a stability-oriented integration release. It focuses on adoption and reproducibility workflows on top of the core feature surfaces introduced in v0.2.0-alpha.
+
+## Highlights
+
+- Public-ready demo workflow under `demo_repo_template/`:
+  - semantic retrieval demo
+  - item similarity demo
+  - benchmark demo with bruteforce vs faiss_flat comparison
+  - quickstart verification script (`scripts/verify_quickstart.py`)
+- Stable integration guidance:
+  - `docs/integration_guides.md`
+  - `docs/reproducibility.md`
+- Reproducibility-first command workflow and artifact policy:
+  - baseline report
+  - real-corpus evaluation report
+  - repeated stability runs
+  - exact-equivalence benchmark checks
+
+## Migration from v0.2.0-alpha
+
+- No breaking API changes are introduced in core `vector_engine` entry points.
+- Existing v0.2.0-alpha workflows continue to run.
+- Documentation language now positions the project as v0.3.0 stable.
+- Integration and release workflows are now organized into explicit guides and publish/private artifact policy.
+
+## Validation Evidence Snapshot
+
+- Real-corpus style 3-run results remain stable (see `artifacts/real_corpus_runs/`).
+- Faiss Flat exact-equivalence checks pass overlap gating (`overlap_vs_bruteforce >= 0.99` target, observed `1.0` in recorded runs).
+- 200-run stability study generated with summary statistics and plot:
+  - `artifacts/testing_runs/stability_runs_bruteforce_200.jsonl`
+  - `artifacts/testing_runs/stability_summary_bruteforce_200.json`
+  - `artifacts/testing_runs/stability_plot_p95_qps.png`
+- Stability notebook:
+  - `notebooks/04_stability_runs.ipynb`
+
+## Known Limits
+
+- Real private-corpus embeddings and raw IDs should remain outside the public repo.
+- Published benchmark numbers are configuration-dependent; include hardware/runtime notes with external reports.
+- Faiss benchmark paths require local Faiss installation (`faiss-cpu`).
+
+## Release Checklist
+
+1. Run tests: `python3 -m pytest -q`
+2. Generate baseline: `python3 scripts/rag_baseline.py --output-dir artifacts --k 3`
+3. Run real-corpus evaluation: `python3 scripts/rag_real_corpus_eval.py --embeddings ... --query-embeddings ... --ids ... --ground-truth ...`
+4. Run stability workflow: `python3 scripts/stability_runs.py --embeddings ... --query-embeddings ... --ids ... --ground-truth ... --backend bruteforce --run-count 200 --output-dir artifacts/testing_runs`
+5. Run exact benchmark: `python3 benchmarks/compare_bruteforce_vs_faiss.py --mode exact --min-flat-overlap 0.99 --output artifacts/faiss_equivalence/run_1.json`
+6. Verify docs links and artifact policy language in `README.md`.
+7. Tag and release:
+   - `git tag -a v0.3.0 -m "Vector Engine v0.3.0"`
+   - `git push origin v0.3.0`

vector_engine-1.2.0/docs/releases/v1.0.0-checklist.md

@@ -0,0 +1,40 @@
+# v1.0.0 Final Checklist
+
+## Core quality
+
+- [ ] `python -m pytest -q` completes successfully in `.venv312` created via `requirements/constraints-macos-arm64-py312.txt`.
+- [ ] API stability tests pass (`tests/test_api_stability.py`).
+
+## Performance evidence
+
+- [ ] Optional: exact-equivalence benchmark generated when Faiss is available (`artifacts/faiss_equivalence/run_*.json`).
+- [ ] Stability summary generated (`artifacts/testing_runs/stability_summary_*.json`).
+- [ ] Matrix summary generated (`artifacts/benchmark_matrix/matrix_summary.json`).
+- [ ] Local medium profile summary generated (`artifacts/local_profile/profile_local_summary.json`).
+- [ ] Performance table in `docs/releases/v1.0.0.md` populated from artifacts.
+
+## Reproducibility
+
+- [ ] Protocol settings are fixed and documented (`seed`, `warmup`, `loops`, matrix configs).
+- [ ] Hardware/runtime notes captured in machine-readable artifacts.
+- [ ] Public/private artifact policy respected (no confidential embeddings committed).
+
+## Packaging
+
+- [ ] `README.md` includes v1.0.0 install/start, ingest-to-eval, and benchmark matrix commands.
+- [ ] `docs/reproducibility.md` includes matrix workflow + artifact locations.
+- [ ] Canonical py312 macOS arm64 bootstrap command is documented in README and reproducibility docs.
+- [ ] `docs/api.md` includes v1.0 API stability contract.
+
+## Release actions
+
+- [ ] Commit final release changes on `main`.
+- [ ] Tag `v1.0.0` on final release commit.
+- [ ] Publish GitHub release notes from `docs/releases/v1.0.0.md`.
+- [ ] Build and publish PyPI package.
+
+## Next minor cadence gate (`v1.x.0`)
+
+- [ ] Tier-1 KPI review complete (`docs/kpi_charter.md`).
+- [ ] Credibility audit status is `pass`.
+- [ ] Release bundle manifest has `ready_for_submission=true`.

vector_engine-1.2.0/docs/releases/v1.0.0.md

@@ -0,0 +1,88 @@
+# Vector Engine v1.0.0
+
+## Release summary
+
+v1.0.0 establishes Vector Engine as a reproducibility-first Python library for local retrieval experimentation, with stable API contracts, contract-validated artifacts, and publication-oriented evaluation workflows.
+
+## Patch notes
+
+### v1.0.2
+
+- `retrieval_report_detailed(..., include_error_buckets=True)` now computes `zero_hit_rate@k` and `perfect_recall_rate@k` over queries that have non-empty ground truth.
+- `no_ground_truth_rate@k` continues to use total-query denominator so missing-label prevalence stays visible.
+- This is a semantics correction for mixed query sets (labeled + unlabeled) and does not change API names or output keys.
+
+## Stable public surface
+
+- Core: `VectorArray`, `VectorIndex`, `Metric`, `SearchResult`
+- ML: `kmeans`, `KMeansResult`, `knn_classify`, `knn_regress`
+- Training: `mine_hard_negatives`, `TripletBatch`
+- Eval: `retrieval_report`, `retrieval_report_detailed`, `batch_metrics_summary`, `retrieval_cohort_report`
+
+See `docs/api_stability.md` and `tests/test_api_stability.py`.
+
+## Highlights
+
+- Reproducibility contract pipeline via `artifact_contract_version`
+- Dataset ingest/connectors pipeline with reproducible bundle manifests
+- Matrix benchmarks with protocol and environment metadata
+- Stability-study summaries with spread metrics (`std`, `cv`, intervals)
+- Credibility audit flow for evidence-quality checks
+
+## Evidence and artifact flow
+
+1. benchmark matrix outputs
+2. repeated stability outputs
+3. publishable summary composition
+4. credibility audit report
+5. release bundle manifest
+
+Recommended commands:
+
+```bash
+python scripts/repro_smoke.py --output-dir artifacts/repro_smoke
+python scripts/credibility_audit.py --matrix-summary artifacts/benchmark_matrix/matrix_summary.json --stability-summary artifacts/testing_runs/stability_summary_bruteforce_200.json --publishable-summary artifacts/benchmark_matrix/publishable_results.v1.json --real-corpus-report artifacts/real_corpus_runs/run_1.json --output artifacts/audit/credibility_audit.v1.json
+python scripts/build_release_bundle.py --output-dir artifacts/release_bundle
+```
+
+## Legal and citation
+
+- `LICENSE`
+- `CITATION.cff`
+
+## Caveats and scope boundaries
+
+- This project does not claim a new ANN algorithm.
+- Performance outcomes are configuration and hardware dependent.
+- Private embeddings and sensitive metadata must remain outside public artifacts.
+
+See `docs/limitations.md` and `docs/credibility_audit.md`.
+
+## Migration notes (from v0.3.x)
+
+- No intentional breaking changes in core public API.
+- v1.0.0 formalizes API stability and reproducibility contracts.
+- Existing v0.3 workflows remain usable, with additional artifact validation and release packaging.
+
+## Reproduce release evidence
+
+```bash
+python3.12 -m venv .venv312
+source .venv312/bin/activate
+python -m pip install --upgrade pip setuptools wheel
+python -m pip install -c requirements/constraints-macos-arm64-py312.txt -e ".[dev,ml]"
+python scripts/env_diagnostics.py
+python scripts/repro_smoke.py --output-dir artifacts/repro_smoke
+python scripts/rag_baseline.py --output-dir artifacts --k 3
+python scripts/rag_real_corpus_eval.py --embeddings artifacts/repro_smoke/real_corpus_inputs/embeddings.npy --query-embeddings artifacts/repro_smoke/real_corpus_inputs/query_embeddings.npy --ids artifacts/repro_smoke/real_corpus_inputs/ids.json --ground-truth artifacts/repro_smoke/real_corpus_inputs/ground_truth.json --metadata artifacts/repro_smoke/real_corpus_inputs/metadata.json --output artifacts/real_corpus_runs/run_1.json --backend bruteforce --k 10 --ks 1,5,10 --loops 5
+python scripts/stability_runs.py --embeddings artifacts/repro_smoke/real_corpus_inputs/embeddings.npy --query-embeddings artifacts/repro_smoke/real_corpus_inputs/query_embeddings.npy --ids artifacts/repro_smoke/real_corpus_inputs/ids.json --ground-truth artifacts/repro_smoke/real_corpus_inputs/ground_truth.json --metadata artifacts/repro_smoke/real_corpus_inputs/metadata.json --backend bruteforce --run-count 200 --output-dir artifacts/testing_runs
+python benchmarks/compare_bruteforce_vs_faiss.py --mode exact --min-flat-overlap 0.99 --output artifacts/faiss_equivalence/run_1.json
+python scripts/benchmark_matrix.py --mode exact --warmup 2 --loops 8 --seed 7 --min-flat-overlap 0.99 --output-dir artifacts/benchmark_matrix
+python scripts/publishable_results.py --matrix-summary artifacts/benchmark_matrix/matrix_summary.json --stability-summary artifacts/testing_runs/stability_summary_bruteforce_200.json --output artifacts/benchmark_matrix/publishable_results.v1.json
+python scripts/credibility_audit.py --matrix-summary artifacts/benchmark_matrix/matrix_summary.json --stability-summary artifacts/testing_runs/stability_summary_bruteforce_200.json --publishable-summary artifacts/benchmark_matrix/publishable_results.v1.json --real-corpus-report artifacts/real_corpus_runs/run_1.json --output artifacts/audit/credibility_audit.v1.json
+python scripts/build_release_bundle.py --output-dir artifacts/release_bundle
+```
+
+## Release checklist
+
+See `docs/releases/v1.0.0-checklist.md`.

vector_engine-1.2.0/docs/releases/v1.1.0-checklist.md

@@ -0,0 +1,40 @@
+# v1.1.0 Final Checklist
+
+## Core quality
+
+- [x] `python -m pytest -q` completes successfully in `.venv312` created via `requirements/constraints-macos-arm64-py312.txt`. (71 passed, 1 skipped — optional faiss test, faiss not installed on this macOS arm64 host)
+- [x] API stability tests pass (`tests/test_api_stability.py`).
+
+## Performance evidence
+
+- [x] Optional: exact-equivalence benchmark generated when Faiss is available (`artifacts/faiss_equivalence/run_*.json`). Faiss is unavailable on this host (macOS arm64, no faiss-cpu wheel); existing committed artifacts stand, regeneration deferred to a faiss-capable host.
+- [x] Stability summary generated (`artifacts/testing_runs/stability_summary_*.json`).
+- [x] Matrix summary generated (`artifacts/benchmark_matrix/matrix_summary.json`).
+- [x] Local medium profile summary generated (`artifacts/local_profile/profile_local_summary.json`).
+- [x] Performance table in `docs/releases/v1.1.0.md` populated from artifacts.
+
+## Reproducibility
+
+- [x] Protocol settings are fixed and documented (`seed`, `warmup`, `loops`, matrix configs).
+- [x] Hardware/runtime notes captured in machine-readable artifacts.
+- [x] Public/private artifact policy respected (no confidential embeddings committed).
+
+## Packaging
+
+- [x] `README.md` includes v1.1.0 install/start, ingest-to-eval, and benchmark matrix commands.
+- [x] `docs/reproducibility.md` includes matrix workflow + artifact locations.
+- [x] Canonical py312 macOS arm64 bootstrap command is documented in README and reproducibility docs.
+- [x] `docs/api.md` includes v1.0 API stability contract.
+
+## Release actions
+
+- [ ] Commit final release changes on `main`.
+- [ ] Tag `v1.1.0` on final release commit.
+- [ ] Publish GitHub release notes from `docs/releases/v1.1.0.md`.
+- [ ] Build and publish PyPI package.
+
+## Next minor cadence gate (`v1.x.0`)
+
+- [x] Tier-1 KPI review complete (`docs/kpi_charter.md`).
+- [x] Credibility audit status is `pass`.
+- [x] Release bundle manifest has `ready_for_submission=true`.

vector_engine-1.2.0/docs/releases/v1.1.0.md

@@ -0,0 +1,114 @@
+# Vector Engine v1.1.0
+
+## Release summary
+
+v1.1.0 continues Vector Engine as a reproducibility-first Python library for local retrieval experimentation, with stable API contracts, contract-validated artifacts, and publication-oriented evaluation workflows.
+
+## Patch notes
+
+### Included fixes from v1.0.2
+
+- `retrieval_report_detailed(..., include_error_buckets=True)` computes `zero_hit_rate@k` and `perfect_recall_rate@k` over queries that have non-empty ground truth.
+- `no_ground_truth_rate@k` continues to use total-query denominator so missing-label prevalence stays visible.
+- This is a semantics correction for mixed query sets (labeled + unlabeled) and does not change API names or output keys.
+
+## Stable public surface
+
+- Core: `VectorArray`, `VectorIndex`, `Metric`, `SearchResult`
+- ML: `kmeans`, `KMeansResult`, `knn_classify`, `knn_regress`
+- Training: `mine_hard_negatives`, `TripletBatch`
+- Eval: `retrieval_report`, `retrieval_report_detailed`, `batch_metrics_summary`, `retrieval_cohort_report`
+
+See `docs/api_stability.md` and `tests/test_api_stability.py`.
+
+## Highlights
+
+- Reproducibility contract pipeline via `artifact_contract_version`
+- Dataset ingest/connectors pipeline with reproducible bundle manifests
+- Matrix benchmarks with protocol and environment metadata
+- Stability-study summaries with spread metrics (`std`, `cv`, intervals)
+- Credibility audit flow for evidence-quality checks
+
+## Performance table
+
+Environment: macOS-arm64, Python 3.12.0, backend `bruteforce`, mode `exact`.
+
+### Stability study (`run-count=200`, k=10)
+
+| Metric | Mean | CV |
+| --- | ---: | ---: |
+| Latency p50 (ms) | 0.037 | 0.031 |
+| Latency p95 (ms) | 0.041 | 0.070 |
+| QPS | 313,036 | 0.036 |
+
+All stability CVs are well under the v1.x gate (`<= 0.35`).
+
+### Benchmark matrix (`profile=medium`, warmup=2, loops=8, seed=7)
+
+| Config | n | d | nq | k | Latency p95 (ms) | QPS |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| s_small | 5,000 | 64 | 100 | 10 | 17.98 | 8,041 |
+| m_balanced | 10,000 | 128 | 200 | 10 | 31.64 | 6,978 |
+| l_wide | 25,000 | 256 | 300 | 20 | 146.23 | 2,377 |
+
+`overlap_vs_bruteforce = 1.0` for all configs (exact mode, self-comparison against bruteforce).
+
+Source artifacts: `artifacts/testing_runs/stability_summary_bruteforce_200.json`, `artifacts/benchmark_matrix/matrix_summary.json`, `artifacts/benchmark_matrix/publishable_results.v1.json`.
+
+## Evidence and artifact flow
+
+1. benchmark matrix outputs
+2. repeated stability outputs
+3. publishable summary composition
+4. credibility audit report
+5. release bundle manifest
+
+Recommended commands:
+
+```bash
+python scripts/repro_smoke.py --output-dir artifacts/repro_smoke
+python scripts/credibility_audit.py --matrix-summary artifacts/benchmark_matrix/matrix_summary.json --stability-summary artifacts/testing_runs/stability_summary_bruteforce_200.json --publishable-summary artifacts/benchmark_matrix/publishable_results.v1.json --real-corpus-report artifacts/real_corpus_runs/run_1.json --output artifacts/audit/credibility_audit.v1.json
+python scripts/build_release_bundle.py --output-dir artifacts/release_bundle
+```
+
+## Legal and citation
+
+- `LICENSE`
+- `CITATION.cff`
+
+## Caveats and scope boundaries
+
+- This project does not claim a new ANN algorithm.
+- Performance outcomes are configuration and hardware dependent.
+- Private embeddings and sensitive metadata must remain outside public artifacts.
+
+See `docs/limitations.md` and `docs/credibility_audit.md`.
+
+## Migration notes (from v0.3.x)
+
+- No intentional breaking changes in core public API.
+- v1.1.0 continues API stability and reproducibility contracts.
+- Existing v0.3 workflows remain usable, with additional artifact validation and release packaging.
+
+## Reproduce release evidence
+
+```bash
+python3.12 -m venv .venv312
+source .venv312/bin/activate
+python -m pip install --upgrade pip setuptools wheel
+python -m pip install -c requirements/constraints-macos-arm64-py312.txt -e ".[dev,ml]"
+python scripts/env_diagnostics.py
+python scripts/repro_smoke.py --output-dir artifacts/repro_smoke
+python scripts/rag_baseline.py --output-dir artifacts --k 3
+python scripts/rag_real_corpus_eval.py --embeddings artifacts/repro_smoke/real_corpus_inputs/embeddings.npy --query-embeddings artifacts/repro_smoke/real_corpus_inputs/query_embeddings.npy --ids artifacts/repro_smoke/real_corpus_inputs/ids.json --ground-truth artifacts/repro_smoke/real_corpus_inputs/ground_truth.json --metadata artifacts/repro_smoke/real_corpus_inputs/metadata.json --output artifacts/real_corpus_runs/run_1.json --backend bruteforce --k 10 --ks 1,5,10 --loops 5
+python scripts/stability_runs.py --embeddings artifacts/repro_smoke/real_corpus_inputs/embeddings.npy --query-embeddings artifacts/repro_smoke/real_corpus_inputs/query_embeddings.npy --ids artifacts/repro_smoke/real_corpus_inputs/ids.json --ground-truth artifacts/repro_smoke/real_corpus_inputs/ground_truth.json --metadata artifacts/repro_smoke/real_corpus_inputs/metadata.json --backend bruteforce --run-count 200 --output-dir artifacts/testing_runs
+python benchmarks/compare_bruteforce_vs_faiss.py --mode exact --min-flat-overlap 0.99 --output artifacts/faiss_equivalence/run_1.json
+python scripts/benchmark_matrix.py --mode exact --warmup 2 --loops 8 --seed 7 --min-flat-overlap 0.99 --output-dir artifacts/benchmark_matrix
+python scripts/publishable_results.py --matrix-summary artifacts/benchmark_matrix/matrix_summary.json --stability-summary artifacts/testing_runs/stability_summary_bruteforce_200.json --output artifacts/benchmark_matrix/publishable_results.v1.json
+python scripts/credibility_audit.py --matrix-summary artifacts/benchmark_matrix/matrix_summary.json --stability-summary artifacts/testing_runs/stability_summary_bruteforce_200.json --publishable-summary artifacts/benchmark_matrix/publishable_results.v1.json --real-corpus-report artifacts/real_corpus_runs/run_1.json --output artifacts/audit/credibility_audit.v1.json
+python scripts/build_release_bundle.py --output-dir artifacts/release_bundle
+```
+
+## Release checklist
+
+See `docs/releases/v1.1.0-checklist.md`.

vector_engine-1.2.0/docs/releases/v1.1.1.md

@@ -0,0 +1,21 @@
+# Vector Engine v1.1.1
+
+## Release summary
+
+v1.1.1 is a packaging and documentation patch on top of v1.1.0. No public API surface, behavior, or evidence artifacts changed; see `docs/releases/v1.1.0.md` for the full evidence package (stability, matrix, and reproducibility contracts, which remain valid for this patch).
+
+## Changes
+
+- Added PyPI `classifiers` and `[project.urls]` (Homepage, Repository, Issues, Documentation) to `pyproject.toml`.
+- Closed out the v1.1.0 release checklist and populated the performance table in `docs/releases/v1.1.0.md` from regenerated evidence artifacts.
+- Colab demo notebooks now install standalone via `pip` and link directly from GitHub instead of static Drive copies.
+- Fixed a build failure under newer setuptools caused by a redundant `License ::` classifier conflicting with the SPDX `license = "MIT"` field (PEP 639).
+
+## Stable public surface
+
+Unchanged from v1.1.0. See `docs/api_stability.md` and `tests/test_api_stability.py`.
+
+## Legal and citation
+
+- `LICENSE`
+- `CITATION.cff`

vector_engine-1.2.0/docs/releases/v1.2.0-checklist.md

@@ -0,0 +1,48 @@
+# v1.2.0 Final Checklist
+
+## Core quality
+
+- [x] Full suite passes on Python 3.12 (80 passed, 3 skipped) and Python 3.14 (82 passed, 1 skipped).
+- [x] New `tests/test_ivf_backend.py` covers: search output shapes, self-hit at full `nprobe`, recall improves monotonically with `nprobe`, `add()` + `save`/`load` round-trip equivalence, invalid-config validation.
+- [x] IVF cluster-batched scoring preserves result IDs and scores for L2, cosine, inner-product, and custom metrics; empty-cluster fallback and result padding are covered.
+- [x] API stability tests pass (`tests/test_api_stability.py`); `docs/api_stability.md` backend surface section updated with `"ivf"`.
+
+## New feature: IVF backend
+
+- [x] `vector_engine/backends/ivf.py` implements the existing `BaseBackend` protocol (`build`/`add`/`search`/`save`/`load`/`capabilities`/`get_runtime_stats`).
+- [x] Pure numpy — no new dependency; works on hosts without a `faiss-cpu` wheel (e.g. macOS arm64).
+- [x] Registered as `"ivf"` in `vector_engine/backends/__init__.py` / `registry.py`.
+
+## Performance evidence
+
+- [x] Updated recall/latency/QPS sweep generated (`scripts/benchmark_ivf.py` → `artifacts/ivf_benchmark/ivf_batched_recall_sweep.json`).
+- [x] Current batched-IVF recall sweep and same-data previous-loop comparison generated with reproducible scripts and recorded environment metadata.
+- [x] Original recall/latency sweep retained and updated release notes document same-data timing comparison for the cluster-batched scoring improvement (`docs/releases/v1.2.0.md`).
+- [x] Sweep is honest about tradeoffs, not just favorable numbers — matches this project's reproducibility-first standard.
+
+## Reproducibility
+
+- [x] Benchmark protocol is fixed and documented (`n`, `d`, `nq`, `k`, `n_clusters`, `nprobe` sweep, `loops`, `seed`).
+- [x] Benchmark is independently re-runnable via the documented CLI command in `docs/releases/v1.2.0.md`.
+- [x] Release-bundle manifest reports v1.2.0 and includes both current IVF benchmark reports.
+
+## Packaging and docs
+
+- [x] `README.md` backend table includes `ivf` with dependency footprint and tuning knobs.
+- [x] `README.md` title, Colab Demos, and Examples sections updated with `notebooks/05_ann_backend.ipynb`.
+- [x] All five notebooks have separate markdown and code cells, valid Python code, and current release guidance; local demo cells execute with documented artifacts. The semantic notebook's embedding-model call is exercised with a deterministic test double because the optional model dependency and remote model weights are not installed here.
+- [x] Source distribution includes the release notebooks, benchmark scripts, and v1.2.0 notes; the wheel contains only the importable Python package.
+- [x] `docs/releases/v1.2.0.md` documents the design, measured batching improvement, benchmark tables, environment, and reproduction commands.
+- [x] `notebooks/05_ann_backend.ipynb` created; code cells verified to execute end-to-end and reproduce the documented numbers.
+
+## Release actions
+
+- [x] Commit final release changes on `main` and point tag `v1.2.0` to that commit.
+- [x] Publish updated GitHub release notes from `docs/releases/v1.2.0.md` (https://github.com/neelpanchal11/Vector-Engine/releases/tag/v1.2.0).
+- [x] Rebuild and validate v1.2.0 wheel and source distributions, including the cluster-batched IVF search change.
+- [ ] Publish the validated v1.2.0 package to PyPI.
+
+## Next minor cadence gate (`v1.x.0`)
+
+- [x] Credibility/evidence artifacts for this feature stand on their own sweep (`artifacts/ivf_benchmark/`), independent of the v1.1.0 bruteforce/faiss evidence pipeline (`artifacts/benchmark_matrix/`, `artifacts/audit/credibility_audit.v1.json`), which remains valid and untouched for the existing backends.
+- [x] No breaking changes — minor version bump per `docs/api_stability.md` compatibility guarantees.