vecscope 0.1.0__tar.gz

vecscope-0.1.0/.agents/skills/vecscope/SKILL.md

@@ -0,0 +1,27 @@
+---
+name: vecscope
+description: Audit an embedding matrix or vector index before trusting retrieval results. Use when building or debugging RAG, semantic search, or recommendation retrieval, when "the same chunks always come back", when similarity scores all look alike, or before choosing ANN index parameters.
+---
+
+# vecscope
+
+Run `vecscope audit <embeddings> --json` (or the MCP tool `audit_embeddings`; `pip install vecscope` if missing) and read `verdict`. For files above 100k rows a seeded sample is audited; pass `--sample 0` when ANN recall must be scored on every row.
+
+- `fail` on **anisotropy** or **effective_rank**: center/whiten the embeddings or switch models before tuning retrieval; cosine scores are not discriminative. `details.mean_random_cosine_after_centering` shows what centering alone would achieve; if it is near 0, centering is enough and the model can stay.
+- `fail` on **hubness**: use mutual k-NN, local scaling, or re-rank; raw cosine top-k will over-return hub items.
+- `fail` on **collapse**: deduplicate the corpus first; top-k is wasted on copies.
+- `fail` on **ann_recall**: raise `ef_search`/`nprobe` or rebuild the index; the model is not the problem.
+- `fail` on **rogue_dimensions**: standardize each dimension (z-score) or drop `details.rogue_dims` before indexing; cosine is being decided by a few coordinates.
+- `fail` on **norms**: L2-normalize before indexing and drop zero vectors (usually empty or failed inputs). Check **metric_mismatch** next.
+- `fail` on **metric_mismatch**: the index metric (cosine vs inner product) changes the results; pick one deliberately and normalize if cosine is intended.
+- `fail` on **distance_concentration**: nearest neighbors are barely closer than random items; reduce dimensionality, change the model, or add a re-ranker. Expect poor ANN recall too.
+- `fail` on **outliers**: inspect `details.sample_ids`; they are usually junk, empty or foreign-language items. Remove or re-embed them.
+- `fail` on **ood_queries** (needs `--queries`): the queries do not live where the corpus lives. Check that queries use the same model and the model's query prefix/instruction; inspect `details.ood_sample_ids` for topics the corpus does not cover.
+- `fail` on **query_hubness**: a few corpus items answer most queries; re-rank or apply hubness reduction before trusting top-k.
+- `fail` on **connectivity**: the k-NN graph has islands; raise HNSW `M`/`ef_construction` or add bridging content, otherwise whole regions are unreachable.
+
+Every finding carries `healthy_range`, `warn_at` and `fail_at`; quote them when explaining a verdict.
+
+Pass `--clusters 10` (or `clusters=10`) to get a per-cluster map with sample ids when you need to tell the user *where* a problem lives. Run `vecscope sweep <embeddings>` (MCP: `sweep_index`) before choosing HNSW/IVF parameters; take the recommended configuration and quote its recall and qps. The sweep uses the GPU automatically when the faiss build has one (IVF only; HNSW is CPU-only); mention the device from the result when comparing qps.
+
+Report the numbers verbatim to the user. Do not guess at embedding quality without this evidence.

vecscope-0.1.0/.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,27 @@
+name: Bug report
+description: A finding, verdict, loader or CLI behaviour that looks wrong
+labels: [bug]
+body:
+  - type: textarea
+    id: what
+    attributes:
+      label: What happened
+      description: Paste the command (or MCP call) and the output. `--json` output is best.
+    validations:
+      required: true
+  - type: textarea
+    id: expected
+    attributes:
+      label: What you expected
+    validations:
+      required: true
+  - type: input
+    id: shape
+    attributes:
+      label: Input shape and format
+      placeholder: "n=120000, dim=768, .parquet, float32"
+  - type: input
+    id: version
+    attributes:
+      label: vecscope and Python version
+      placeholder: "vecscope 0.1.0, Python 3.12, Windows 11"

vecscope-0.1.0/.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,5 @@
+blank_issues_enabled: true
+contact_links:
+  - name: Security report
+    url: https://github.com/076923/vecscope/blob/main/SECURITY.md
+    about: Please do not open public issues for security problems.

vecscope-0.1.0/.github/ISSUE_TEMPLATE/feature_request.yml

@@ -0,0 +1,20 @@
+name: Feature request
+description: A new finding, input format, or agent integration
+labels: [enhancement]
+body:
+  - type: textarea
+    id: problem
+    attributes:
+      label: What retrieval failure would this help an agent see?
+      description: Describe the symptom first ("same chunks for every query"), then the measurement.
+    validations:
+      required: true
+  - type: textarea
+    id: evidence
+    attributes:
+      label: Proposed evidence
+      description: The metric, how it is computed, and a reference if one exists.
+  - type: textarea
+    id: verdict
+    attributes:
+      label: What should the agent do when it fails?

vecscope-0.1.0/.github/pull_request_template.md

@@ -0,0 +1,16 @@
+## What this changes
+
+<!-- One paragraph. If it adds or changes a finding, say what symptom it makes visible. -->
+
+## Evidence
+
+<!-- For a new or changed diagnostic: the synthetic case it detects, the healthy case it leaves
+     at `ok`, and the reference (paper or doc) for the measurement, if any. -->
+
+## Checklist
+
+- [ ] `pytest -q` passes
+- [ ] `ruff check src tests examples` passes
+- [ ] Output is still deterministic (fixed seeds, no learned components)
+- [ ] README / SKILL.md updated if the CLI, MCP tool or verdicts changed
+- [ ] CHANGELOG.md has an entry under Unreleased

vecscope-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,39 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, windows-latest, macos-latest]
+        python: ["3.10", "3.13"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python }}
+      - run: python -m pip install -e ".[dev,mcp,index]"
+      - run: ruff check src tests examples
+      - run: pytest -q
+      - name: CLI smoke test
+        run: |
+          python examples/make_examples.py
+          vecscope audit examples/healthy.npy
+          vecscope --version
+
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: python -m pip install build twine
+      - run: python -m build
+      - run: python -m twine check dist/*

vecscope-0.1.0/.github/workflows/release.yml

@@ -0,0 +1,37 @@
+name: Release to PyPI
+
+# Publishes when a tag like v0.1.0 is pushed. Uses PyPI trusted publishing:
+# register this repository/workflow at https://pypi.org/manage/account/publishing/
+# (environment name: pypi) so no API token is stored in the repository.
+
+on:
+  push:
+    tags: ["v*"]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: python -m pip install build
+      - run: python -m build
+      - uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    needs: build
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+      - uses: pypa/gh-action-pypi-publish@release/v1

vecscope-0.1.0/.gitignore

@@ -0,0 +1,9 @@
+__pycache__/
+*.pyc
+.venv/
+dist/
+build/
+*.egg-info/
+.pytest_cache/
+.ruff_cache/
+*.npy

vecscope-0.1.0/AGENTS.md

@@ -0,0 +1,31 @@
+# Repository guidelines
+
+`vecscope` gives agents deterministic evidence about embedding spaces and vector indexes.
+These rules apply to any agent or person changing this repository.
+
+## Product rules
+
+- Every finding reports a measured value, the verdict, the threshold that produced it, and an
+  explanation that states what the number means and what healthy looks like.
+- Output is deterministic: fixed seeds, exact neighbors, no learned components, no network.
+  Two runs on the same input must produce identical JSON.
+- The core depends on NumPy only. Loaders and the MCP server may use optional extras.
+- Never guess at embedding quality in docs or messages. Point at the finding.
+
+## Layout
+
+- `src/vecscope/diagnostics/`: one pure function per finding, returning plain Python numbers
+- `src/vecscope/evidence.py`: thresholds, `Finding`/`Evidence` models, the `audit` entry point
+- `src/vecscope/sweep.py`: HNSW/IVF parameter sweep on top of faiss (optional extra)
+- `src/vecscope/cli.py`, `src/vecscope/mcp_server.py`: thin adapters over `audit` and `sweep`
+- `.agents/skills/vecscope/SKILL.md`: what an agent should do with each verdict
+
+## Checks before a change is done
+
+```bash
+pytest -q
+ruff check src tests examples
+```
+
+Update `README.md`, `docs/readme/README_ko.md`, the skill file and `CHANGELOG.md` when the
+CLI, MCP tool, findings or verdicts change. Keep the English and Korean READMEs in sync.

vecscope-0.1.0/CHANGELOG.md

@@ -0,0 +1,29 @@
+# Changelog
+
+All notable changes to this project are documented here. The format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses
+[Semantic Versioning](https://semver.org/).
+
+## [Unreleased]
+
+## [0.1.0] - 2026-10-25
+
+### Added
+
+- `vecscope audit` CLI with text and `--json` output, `--fail-on` gate and non-zero exit on FAIL
+- Findings: hubness, antihubs, anisotropy, effective rank, collapse, rogue dimensions, norms,
+  metric mismatch, distance concentration, outliers, k-NN graph connectivity, ANN recall
+- Seeded sampling (`--sample`, default 100k rows) so large matrices stay tractable
+- MCP server (`vecscope-mcp`) exposing `audit_embeddings`
+- Query-side findings (`--queries`): out-of-distribution queries and query hubness
+- Per-cluster report with sample ids (`--clusters N`)
+- `vecscope sweep` / MCP `sweep_index`: HNSW and IVF parameter sweep with recall, build time and
+  QPS, plus a recommended configuration (optional `faiss-cpu` extra); IVF and the exact
+  baseline run on the GPU automatically with a faiss GPU build (`--device auto|cpu|gpu`)
+- Loaders for `.npy`, `.npz`, `.parquet`, `.jsonl`, `.json`, `.csv`
+- Agent skill file describing when to run the audit and how to act on verdicts
+- Each finding reports `healthy_range`, `warn_at`, `fail_at`; `anisotropy` also reports the
+  mean-vector norm and the random-pair cosine after centering
+- Thresholds calibrated on public embeddings from eight models (OpenAI ada-002 and 3-large,
+  Cohere embed-v3, all-MiniLM-L6-v2, bge-m3, mxbai-embed-large-v1, gte-base-en-v1.5);
+  `examples/real_models.py` reproduces the table in the README

vecscope-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,44 @@
+# Contributing to vecscope
+
+Thanks for helping agents see embedding spaces. This file explains how to work on the code
+and what a good change looks like.
+
+## Setup
+
+```bash
+git clone https://github.com/076923/vecscope
+cd vecscope
+python -m pip install -e ".[dev,mcp,index]"
+pytest -q
+ruff check src tests examples
+```
+
+`python examples/make_examples.py` writes three small matrices into `examples/` so you can try
+`vecscope audit examples/anisotropic.npy` right away.
+
+## Scope
+
+`vecscope` produces **deterministic evidence** about an embedding matrix or a vector index for
+an agent to act on. A change fits when it:
+
+- makes a retrieval failure visible that an agent could not see before, or
+- makes an existing finding more accurate, cheaper, or easier to act on.
+
+It does not fit when it needs a learned component, a network call, or a model download at
+audit time, or when it makes two runs on the same input produce different output.
+
+## Adding a finding
+
+1. Put the measurement in `src/vecscope/diagnostics/<name>.py` as a pure function returning a
+   `dict` of plain Python numbers. Reuse the shared exact k-NN from `audit` when you can.
+2. Register the threshold pair in `THRESHOLDS` and build the `Finding` in
+   `src/vecscope/evidence.py`. The explanation string must state the number, what it means, and
+   what "healthy" looks like.
+3. Add a test with a synthetic case that fails and a healthy case that stays `ok`.
+4. Describe the finding in the table in `README.md` and the action in
+   `.agents/skills/vecscope/SKILL.md`.
+
+## Pull requests
+
+Keep PRs focused on one change. Fill in the template, especially the evidence section.
+Dependencies stay at NumPy for the core; optional extras are fine for loaders and the MCP server.

vecscope-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Daehee Yun
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.