compass-kb-validation 1.0.0__py3-none-any.whl

compass_kb_validation-1.0.0.dist-info/METADATA

@@ -0,0 +1,305 @@
+Metadata-Version: 2.4
+Name: compass-kb-validation
+Version: 1.0.0
+Summary: Knowledge-base readiness validation for RAG & AI agents — config-driven, deterministic pipeline integrity checks.
+Author: COMPASS
+License: MIT
+Project-URL: Homepage, https://github.com/TestAutomationArchitect/compass
+Project-URL: Repository, https://github.com/TestAutomationArchitect/compass
+Project-URL: Issues, https://github.com/TestAutomationArchitect/compass/issues
+Keywords: rag,knowledge-base,vector-database,data-validation,pipeline-testing,llm-infrastructure
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Quality Assurance
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.31.0
+Requires-Dist: boto3>=1.34.0
+Requires-Dist: pyyaml>=6.0.1
+Requires-Dist: python-dotenv>=1.0.0
+Requires-Dist: redis>=5.0.0
+Provides-Extra: dashboard
+Requires-Dist: fastapi>=0.104.0; extra == "dashboard"
+Requires-Dist: uvicorn>=0.24.0; extra == "dashboard"
+Provides-Extra: pdf
+Requires-Dist: PyPDF2>=3.0.0; extra == "pdf"
+Provides-Extra: ui
+Requires-Dist: playwright>=1.40.0; extra == "ui"
+Provides-Extra: qdrant
+Requires-Dist: qdrant-client>=1.7.0; extra == "qdrant"
+Provides-Extra: pgvector
+Requires-Dist: psycopg[binary]>=3.1; extra == "pgvector"
+Provides-Extra: pinecone
+Requires-Dist: pinecone>=5.0.0; extra == "pinecone"
+Provides-Extra: weaviate
+Requires-Dist: weaviate-client>=4.5.0; extra == "weaviate"
+Provides-Extra: azure
+Requires-Dist: azure-storage-blob>=12.0.0; extra == "azure"
+Requires-Dist: azure-identity>=1.15.0; extra == "azure"
+Provides-Extra: gcs
+Requires-Dist: google-cloud-storage>=2.10.0; extra == "gcs"
+Provides-Extra: all
+Requires-Dist: fastapi>=0.104.0; extra == "all"
+Requires-Dist: uvicorn>=0.24.0; extra == "all"
+Requires-Dist: PyPDF2>=3.0.0; extra == "all"
+Requires-Dist: playwright>=1.40.0; extra == "all"
+Requires-Dist: qdrant-client>=1.7.0; extra == "all"
+Requires-Dist: psycopg[binary]>=3.1; extra == "all"
+Requires-Dist: pinecone>=5.0.0; extra == "all"
+Requires-Dist: weaviate-client>=4.5.0; extra == "all"
+Requires-Dist: azure-storage-blob>=12.0.0; extra == "all"
+Requires-Dist: azure-identity>=1.15.0; extra == "all"
+Requires-Dist: google-cloud-storage>=2.10.0; extra == "all"
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+Requires-Dist: pyflakes>=3.0.0; extra == "dev"
+Requires-Dist: pyright==1.1.409; extra == "dev"
+Requires-Dist: bandit>=1.7.5; extra == "dev"
+Requires-Dist: pip-audit>=2.6.0; extra == "dev"
+Dynamic: license-file
+
+# COMPASS — Knowledge-Base Readiness Validation for RAG & AI Agents
+
+[![CI](https://github.com/TestAutomationArchitect/compass/actions/workflows/ci.yml/badge.svg)](https://github.com/TestAutomationArchitect/compass/actions/workflows/ci.yml)
+
+**COMPASS** — **COM**prehensive **P**ipeline **A**nalysis & **S**tructure **S**earch
+
+A config-driven, **deterministic** framework that proves an AI/RAG knowledge base is correctly **built, intact, and retrievable** — *before* AI agents consume it. It's the integrity gate beneath answer-quality evaluation: not "are the answers good?" but **"is the knowledge base itself sound?"**
+
+## What COMPASS Does
+
+COMPASS validates a document's journey through your ingestion pipeline and verifies the resulting knowledge base is agent-ready. The pipeline is **declarative** — you choose which steps run, in what order, in YAML — and ships with these built-in steps:
+
+```
+API Upload → Object Storage → Knowledge Graph → Vector Store → Retrieval → UI Visibility
+```
+
+Enable only the steps your setup has (no object store? no graph? drop them), reorder them, or register your own. For each enabled step it deterministically checks, for example:
+
+- **Ingestion** — the document was accepted and assigned an id
+- **Object storage** — it persisted with the right content type, size, and integrity (S3, MinIO, R2, Azure Blob, GCS…)
+- **Knowledge graph** — required metadata predicates are present and well-formed (GraphDB, Neptune, Neo4j…)
+- **Vector store** — chunks exist with valid embeddings, correct dimensions/metric, sane ordering/overlap, no cross-document contamination (Redis, Qdrant, Pinecone, pgvector…) — **28 toggleable checks**
+- **Retrieval** — the document's own content is actually retrievable (recall@K, MRR — informational)
+- **UI visibility** — it surfaces in the end-user application
+
+Two modes: **live** validation as documents are ingested, and **post-hoc discovery** of a corpus that's already loaded.
+
+> Scope note: COMPASS validates *pipeline & data integrity*. It does **not** judge answer quality — that's the evaluation layer's job. COMPASS is the gate that runs first.
+
+## Key Features
+
+✅ **Declarative pipeline** — choose/reorder/extend steps in YAML; no fixed shape  
+✅ **Config-Driven** — zero code changes per project; all customization in YAML  
+✅ **Pluggable backends** — storage / vector store / knowledge graph via a provider registry (one file to add a vendor)  
+✅ **Pluggable steps** — register custom validators in a step registry; the report adapts automatically  
+✅ **Deterministic & cheap** — structural/byte/schema checks, no LLM, CI-gateable  
+✅ **Interactive reporting** — self-contained HTML dashboard + JSON + JUnit XML  
+✅ **Trend Tracking** — SQLite historical performance analysis  
+✅ **Notifications** — Slack/Teams webhooks for alerts  
+✅ **Parallel Execution** — process many documents concurrently  
+✅ **Graceful Degradation** — non-critical failures don't crash the pipeline  
+
+## Quick Start
+
+Install the package (core deps only; add extras per backend — see below):
+
+```bash
+pip install compass-kb-validation
+# backends/features on demand, e.g.:
+pip install "compass-kb-validation[qdrant,pgvector]"
+```
+
+This installs two commands: `compass-validate` (live) and `compass-discover` (post-hoc).
+Create a project config (`config/projects/<name>.yaml`) and, for live validation,
+a `test_data/<name>.yaml` listing the documents to push. Then:
+
+```bash
+# Live ingestion — validate documents as they're uploaded through your API
+compass-validate --project my-kb --output-format all
+
+# Post-hoc discovery — validate a knowledge base that's already populated
+compass-discover --project my-kb --include-retrieval
+
+# Validate connectivity/config only (no backends hit)
+compass-validate --project my-kb --dry-run
+```
+
+> From source instead of PyPI: `git clone … && cd compass && pip install -e ".[all]"`.
+> Releases are published via [PUBLISHING.md](PUBLISHING.md).
+
+Reports are written to `reports/` (HTML dashboard + JSON + JUnit XML). Add
+`--live` to open a real-time progress dashboard, `--trends` to record history,
+`--parallel` for concurrency.
+
+### Try it locally (no cloud needed)
+
+A self-contained Docker stack runs the full pipeline against MinIO + Redis Stack +
+a mock ingestion API — see **[integration/README.md](integration/README.md)**.
+
+## Project Structure
+
+```
+compass/
+├── run_validation.py              # Entry point — live ingestion validation
+├── discover_and_validate.py       # Entry point — post-hoc discovery
+├── ingestion_validation/          # Main package
+│   ├── models/                    # StepResult / PipelineResult (stdlib dataclasses)
+│   ├── config/                    # Layered YAML config + typed Settings
+│   ├── utils/                     # Run-id, logging, polling w/ backoff
+│   ├── providers/                 # Pluggable backends via a registry:
+│   │   ├── storage.py             #   object storage (S3-family…)
+│   │   ├── vectorstore.py         #   vector store (Redis/RediSearch…)
+│   │   ├── graph.py               #   knowledge graph (SPARQL, Neptune…)
+│   │   └── source.py              #   document discovery (storage|index|manifest)
+│   ├── validators/                # BaseValidator + step registry + the steps
+│   │   └── registry.py            #   declarative pipeline (StepSpec registry)
+│   ├── pipeline/                  # Orchestrator (context propagation, halting)
+│   ├── report/                    # Self-contained HTML dashboard + JSON/JUnit
+│   ├── corpus.py                  # Corpus-level KB readiness analysis
+│   ├── live_dashboard.py          # FastAPI + SSE real-time dashboard
+│   ├── notifications.py           # Slack / Teams webhooks
+│   └── trend_tracker.py           # SQLite historical trends
+├── config/{base.yaml, projects/_template.yaml}
+├── test_data/_template.yaml       # Documents to validate (live mode)
+├── integration/                   # Local Docker integration stack
+├── tests/                         # Unit tests (backends faked — no live services)
+└── reports/                       # Generated reports (gitignored)
+```
+
+## Documentation
+
+- **[ARCHITECTURE.md](ARCHITECTURE.md)** — how COMPASS is built: layers, data flow, the registries, the two run modes, extension points
+- **[EXTENDING-BACKENDS.md](EXTENDING-BACKENDS.md)** — add a new backend (storage/vector/graph/source) or a new pipeline step
+- **[integration/README.md](integration/README.md)** — run the full pipeline locally on Docker
+- [CONTRIBUTING.md](CONTRIBUTING.md) · [SECURITY.md](SECURITY.md)
+
+## Configuration
+
+All behaviour lives in YAML; credentials are referenced by **environment-variable
+name** and resolved at runtime (never stored in config). `config/base.yaml` holds
+shared defaults; each `config/projects/<name>.yaml` overlays only what differs and
+selects a `dev`/`staging`/`prod` environment block.
+
+```yaml
+# config/projects/my-kb.yaml
+display_name: "My Knowledge Base"
+
+# Declarative pipeline — choose which steps run, in what order (registry keys).
+# Omit to use the full default pipeline.
+pipeline:
+  steps: [api_upload, s3_storage, redis_chunks, retrieval_quality]
+  critical_steps: ["API Upload", "S3 Storage"]   # failure here halts the run
+
+# Where post-hoc discovery finds documents: storage | vectorstore | manifest
+discovery:
+  provider: vectorstore           # enumerate the KB straight from the index
+
+environments:
+  dev:
+    api:
+      base_url_env: API_BASE_URL          # env-var NAME, not the value
+      graphql_mutation: |
+        mutation ($file: Upload!, $metadata: JSON) { uploadDocument(file:$file, metadata:$metadata){ status document_id } }
+    s3:
+      provider: s3                        # s3 | minio | r2 | azure_blob | gcs …
+      bucket_name: my-documents
+      prefix: knowledge-base/
+    redis:
+      provider: redis                     # redis | qdrant | pinecone | pgvector …
+      search_index: my_embeddings
+      expected_embedding_dim: 1536
+      expected_distance_metric: COSINE
+```
+
+## Advanced Usage
+
+### Add a custom pipeline step
+
+Steps are declarative — register a validator and reference it in `pipeline.steps`:
+
+```python
+from ingestion_validation.validators.base import BaseValidator
+from ingestion_validation.validators.registry import register_step, StepSpec
+from ingestion_validation.models import StepResult, StepStatus
+
+class PiiRedactionValidator(BaseValidator):
+    step_name = "PII Redaction"
+    def __init__(self, config): self.config = config
+    def validate(self, context: dict) -> StepResult:
+        return StepResult(self.step_name, StepStatus.PASSED, "no PII leaked")
+
+register_step(StepSpec(
+    key="pii_redaction", name="PII Redaction",
+    factory=lambda settings, shared: PiiRedactionValidator(settings.s3),
+    abbr="PII", order=25,
+))
+# then:  pipeline.steps: [api_upload, pii_redaction, redis_chunks]
+```
+
+The orchestrator, live dashboard, and HTML report adapt automatically. See
+**[EXTENDING-BACKENDS.md](EXTENDING-BACKENDS.md)** for adding backends too.
+
+### CI/CD integration — the readiness gate
+
+COMPASS is built to run **before** your agents or eval consume the knowledge
+base: wire it into the pipeline that publishes the KB and fail the build when
+the KB is not retrieval-ready. Every entry point returns a CI-friendly exit
+code — **0 = gate passed, 1 = gate failed, 2 = config error** — and emits JUnit
+XML for any CI system.
+
+**Gate strictness** (discovery mode) is yours to choose:
+
+```bash
+# Strictest: any failed document fails the build (default)
+compass-discover --project my-kb
+
+# Tolerate a few failures: require >= 95% of documents to pass
+compass-discover --project my-kb --fail-under 95
+
+# Also require the corpus to be READY (completeness/coverage/dedup verdict)
+compass-discover --project my-kb --require-ready
+```
+
+**GitHub Actions** — drop the reusable action into your KB pipeline:
+
+```yaml
+# .github/workflows/kb-readiness.yml
+jobs:
+  kb-readiness:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4              # your repo holds config/projects/<name>.yaml
+      - uses: TestAutomationArchitect/compass@v1
+        with:
+          project: my-kb
+          require-ready: "true"                # block promotion unless READY
+          # fail-under: "95"                   # ...or tolerate a bounded failure rate
+        env:
+          REDIS_HOST: ${{ secrets.REDIS_HOST }}   # config references env-var names; supply them here
+```
+
+A full example you can copy is in **[examples/kb-readiness.yml](examples/kb-readiness.yml)**.
+
+**Container** — no Python setup needed; works in any CI or a self-hosted runner:
+
+```bash
+docker build -t compass:latest .
+docker run --rm -v "$PWD:/work" --env-file .env \
+  compass:latest compass-discover --project my-kb --require-ready
+```
+
+## Applicable Domains
+
+Enterprise document management · legal/compliance · healthcare · financial
+research · customer-support KBs · internal wikis · e-commerce catalogs · academic
+repositories — any RAG/agent knowledge base built from a document pipeline.
+
+## License
+
+MIT License — see [LICENSE](LICENSE).

compass_kb_validation-1.0.0.dist-info/RECORD

@@ -0,0 +1,38 @@
+discover_and_validate.py,sha256=vwiBA2HCCu-judb8wDTcXvmV8WRGuYDTLm2BD4pvXes,8551
+run_validation.py,sha256=HNUlQHRV6mXIp9TK3_N3j3fSXnQAM1XUneeJj5eHV3A,7475
+compass_kb_validation-1.0.0.dist-info/licenses/LICENSE,sha256=AX0mhitXCN2WT51Z0Kjs20MuBLcKzyFKGlBBtHpzEyE,1077
+ingestion_validation/__init__.py,sha256=1sLkwhuyn9XugsH1LrzOEs6tC8uMUqZQfJmxd5Zjb_I,287
+ingestion_validation/corpus.py,sha256=j-kTjOsDbYmfUa7gtyf__zJKgTKykT7JjOor63mbOOs,4386
+ingestion_validation/live_dashboard.py,sha256=OUW2ZuF_sXdD9GI4RLZYFw98ZpKn2oCzBp2jWv1JGX4,6799
+ingestion_validation/notifications.py,sha256=jSIbT1s8S9y5Hx4TLECE5MuGUdmzWOdlgyFmxvgckbw,3881
+ingestion_validation/remediation.py,sha256=_drCAmtsMiE4OXGF2truNWl6KATvO43vX9UUitt6GEM,4591
+ingestion_validation/trend_tracker.py,sha256=ncAU7U6rlfCYYlpZNLiVzUKf-I2WsWyjW-HwQXbMNQA,6720
+ingestion_validation/config/__init__.py,sha256=TSo01o5pcmZ0bKU8GoZvHQeUkuI1ojasSVypTW88fnU,15752
+ingestion_validation/models/__init__.py,sha256=ooJvC1lEicE0Y9Xqj9VWpPMjJCU4tsuf5_7PXe8f4uc,3814
+ingestion_validation/pipeline/__init__.py,sha256=PZjXg93NE-Pp05sJf69_TP1Vwqm-nibj9FYFXc-P5Ik,5474
+ingestion_validation/providers/__init__.py,sha256=4KLw23ve_XsePegY1b510l8iFun5gs4sESoHeY-upis,3285
+ingestion_validation/providers/graph.py,sha256=y_rQfUOT5jBpYtrl9r-cZ18b8p_GdxcrekklLsmRWmY,6398
+ingestion_validation/providers/source.py,sha256=maK7uV4wfN3VYusSObw6c0ooh2puHkW7cs8mED8YUxA,5848
+ingestion_validation/providers/storage.py,sha256=us_q1U678dFeCOQau-7asv7LmxquoEM7kUQCKJ5r8no,8938
+ingestion_validation/providers/vectorstore.py,sha256=qUFSkaOmV6obAFUP1JcF-US9-OrWDT0cXg_KaxuhTGw,33305
+ingestion_validation/report/__init__.py,sha256=wkmkuKnPADdINqbedcC-IsRkx851g_X8jm2GyAp733I,64344
+ingestion_validation/report/_report_css.py,sha256=tZN840yp_TsUMkgpkQaCQ9LL9ExFYJKBGEKBwvghX_M,35427
+ingestion_validation/report/_report_js.py,sha256=SE3N0epDcXgDMnxgBdi5KAcu7JlhM-XfqheEPnbtluM,19588
+ingestion_validation/report/export.py,sha256=zqnEsni94jxs5qA2oYKD4OucIPEZuQYqtdlQZ6LYMCY,4280
+ingestion_validation/utils/__init__.py,sha256=6x1_weVXPjKPtmIJYNuBEStr_7dpNMcaxi5fnW1J9BE,3161
+ingestion_validation/validators/__init__.py,sha256=qaJw5YmGVElKQU02GYZtdcGdzsVD-M9MhjV85BOXHsQ,898
+ingestion_validation/validators/api_upload.py,sha256=cXIb0jUzIcI1knIbYl7I55yGhNepp00T1OtDWTt9iN0,8991
+ingestion_validation/validators/base.py,sha256=5ysAKz74c1NrNe5m4XwqHX1dwTJ_-iygFIJ6Q6hv0Ts,849
+ingestion_validation/validators/base_sparql.py,sha256=-VE6G2_9CnptULgSwLDeA03Z43UevCOKX8nMqOY24Nk,10830
+ingestion_validation/validators/graphdb_metadata.py,sha256=ZVAkWbpsuaC1PLdaXtyXE2oGDhHN3_GdI_zycwrJTbA,1762
+ingestion_validation/validators/neptune_metadata.py,sha256=cLnnat_R-dYDsDRNlNZxHtr_DFj1XOEn99SQCk-fAT0,2192
+ingestion_validation/validators/redis_chunks.py,sha256=yo1l1ijgYvFS7vqD4FbraqnAo_laAfJH4KhZ--kovJs,27783
+ingestion_validation/validators/registry.py,sha256=cBRybiGEKRLqCbTOrroQ5_q-6k-kDcogPyNZOSoULI0,6594
+ingestion_validation/validators/retrieval_quality.py,sha256=Vsbmy62nafnXy6zMWjaMrMdJ8q9v2hg2lCmEfrmlhiU,7662
+ingestion_validation/validators/s3_storage.py,sha256=acMxFgIOl-3pHNgQA8K3rQ9RWhw_14jWqOXfUN8npKY,10268
+ingestion_validation/validators/ui_document.py,sha256=0xS6qVidpzi3vPCAUL9aFdFNr_QJsIN9IxNzR8I6Rx8,7318
+compass_kb_validation-1.0.0.dist-info/METADATA,sha256=_jC78Lkb5B86Y-DCuUvUgB4dHUCx9pqWY5CuY0-XYng,14183
+compass_kb_validation-1.0.0.dist-info/WHEEL,sha256=aeYiig01lYGDzBgS8HxWXOg3uV61G9ijOsup-k9o1sk,91
+compass_kb_validation-1.0.0.dist-info/entry_points.txt,sha256=t_DKfJzRTwqnae8hMcOwxY-okVg4F144Nr393cxFvYk,103
+compass_kb_validation-1.0.0.dist-info/top_level.txt,sha256=Hx3kAZme27zH7yk0N6UcPoD9hn6xl5ZinwGDmq6olJU,58
+compass_kb_validation-1.0.0.dist-info/RECORD,,

compass_kb_validation-1.0.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (82.0.1)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

compass_kb_validation-1.0.0.dist-info/entry_points.txt

@@ -0,0 +1,3 @@
+[console_scripts]
+compass-discover = discover_and_validate:main
+compass-validate = run_validation:main

compass_kb_validation-1.0.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 COMPASS Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

compass_kb_validation-1.0.0.dist-info/top_level.txt

@@ -0,0 +1,3 @@
+discover_and_validate
+ingestion_validation
+run_validation

discover_and_validate.py

@@ -0,0 +1,184 @@
+#!/usr/bin/env python3
+"""COMPASS — post-hoc discovery & validation entry point.
+
+Scans object storage for existing documents and validates their vector-store
+chunks (and optionally graph metadata / retrieval quality) without re-ingesting.
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+
+try:
+    from dotenv import load_dotenv
+except Exception:  # pragma: no cover
+    load_dotenv = None  # type: ignore
+
+from ingestion_validation.config import load_settings
+from ingestion_validation.pipeline import IngestionPipeline
+from ingestion_validation.providers import get_adapter
+from ingestion_validation.report import export_json, export_junit, generate_batch_dashboard
+from ingestion_validation.utils import configure_logging, new_run_id
+
+
+EXIT_OK, EXIT_GATE_FAILED, EXIT_CONFIG = 0, 1, 2
+
+
+def parse_args(argv=None) -> argparse.Namespace:
+    p = argparse.ArgumentParser(description="Discover storage documents and validate vector chunks")
+    p.add_argument("--project", required=True, help="Project name")
+    p.add_argument("--env", default="dev", help="Environment")
+    p.add_argument("--folder", help="Filter to a specific storage subfolder")
+    p.add_argument("--limit", type=int, help="Max documents to validate")
+    p.add_argument("--dry-run", action="store_true", help="List documents without validating")
+    p.add_argument("--output-format", choices=["html", "json", "junit", "all"], default="html")
+    p.add_argument("--include-graphdb", action="store_true", help="Also validate graph metadata")
+    p.add_argument("--include-retrieval", action="store_true", help="Also run retrieval quality")
+    p.add_argument("--no-trends", action="store_true", help="Do not record/report historical trends (on by default)")
+    # Gate strictness (for CI). Default gate: fail if any document fails.
+    p.add_argument("--fail-under", type=float, metavar="PCT",
+                   help="Fail unless the document pass-rate is >= PCT (0-100). "
+                        "Replaces the default 'fail on any failure' rule.")
+    p.add_argument("--require-ready", action="store_true",
+                   help="Also fail unless the corpus readiness verdict is READY.")
+    return p.parse_args(argv)
+
+
+def gate_failed(passed: int, total: int, verdict: str, fail_under: float | None,
+                require_ready: bool) -> tuple[bool, str]:
+    """Decide whether the readiness gate fails, and why.
+
+    Returns (failed, reason). The gate is the product's contract with a CI
+    pipeline: a non-zero exit blocks promotion of a knowledge base that is not
+    retrieval-ready. Strictness is caller-controlled:
+      * default            -- fail if any document failed validation
+      * --fail-under PCT    -- fail if pass-rate < PCT (tolerates a few failures)
+      * --require-ready     -- additionally fail if the corpus verdict != READY
+    """
+    rate = (passed / total * 100.0) if total else 100.0
+    if fail_under is not None:
+        if rate < fail_under:
+            return True, f"pass-rate {rate:.1f}% < required {fail_under:.1f}%"
+    elif passed < total:
+        return True, f"{total - passed} document(s) failed validation"
+    if require_ready and verdict != "READY":
+        return True, f"corpus readiness verdict is {verdict} (READY required)"
+    return False, "all gate conditions met"
+
+
+def _record_trends(results: list[dict], project: str, environment: str, run_id: str,
+                   mode: str, disabled: bool) -> list[dict] | None:
+    """Record this run to the trend DB and return this project's recent history."""
+    if disabled:
+        return None
+    from ingestion_validation.trend_tracker import record_run
+    rows = record_run(results, environment=environment, run_id=run_id, mode=mode, project=project)
+    print("  Trends recorded." if rows is not None else "  (trend tracking skipped)")
+    return rows
+
+
+def discover_documents(settings, folder_filter: str | None = None, limit: int | None = None) -> list[dict]:
+    """Discover documents via the configured source provider.
+
+    ``settings.discovery.provider`` selects where documents come from:
+    storage (object store), vectorstore (the index itself), manifest (a list
+    file), or any custom source. Records are normalised to carry document_id +
+    file_name (+ optional s3_key/s3_etag/s3_size/folder).
+    """
+    source = get_adapter("source", settings.discovery.provider, settings=settings)
+    return source.discover(folder=folder_filter, limit=limit)
+
+
+def validate_documents(settings, documents: list[dict], skip_steps: list[str]) -> list[dict]:
+    results = []
+    for i, doc in enumerate(documents, 1):
+        folder = doc.get("folder", "")
+        name = f"[{folder}] {doc['file_name']}" if folder else doc["file_name"]
+        print(f"  [{i}/{len(documents)}] {name}")
+        # Pass through normalised identity + any storage hints the source provided.
+        extra = {"file_name": doc["file_name"], "document_id": doc["document_id"]}
+        for k in ("s3_key", "s3_etag", "s3_size"):
+            if doc.get(k) is not None:
+                extra[k] = doc[k]
+        pipeline = IngestionPipeline(settings, skip_steps=skip_steps)
+        result = pipeline.run(file_path="", extra_context=extra, case_name=name)
+        results.append({"name": name, "result": result})
+    return results
+
+
+def main(argv=None) -> int:
+    args = parse_args(argv)
+    configure_logging()
+    if load_dotenv:
+        load_dotenv()
+    run_id = new_run_id()
+    try:
+        settings = load_settings(project=args.project, env=args.env)
+    except Exception as exc:  # noqa: BLE001 - surface config errors as exit 2 for CI
+        print(f"  Config error: {exc}")
+        return EXIT_CONFIG
+
+    print("=" * 60)
+    print(f"  Discovery Validation - {args.project} ({args.env})")
+    print(f"  Run ID: {run_id}")
+    print("=" * 60)
+
+    documents = discover_documents(settings, folder_filter=args.folder, limit=args.limit)
+    print(f"\n  Found {len(documents)} document(s)\n")
+    if not documents:
+        print("  No documents found. Check storage prefix and supported_extensions.")
+        return 0
+
+    if args.dry_run:
+        for doc in documents:
+            folder = doc.get("folder", "")
+            size = doc.get("s3_size")
+            tag = f"[{folder}] " if folder else ""
+            suffix = f" ({size:,} bytes)" if isinstance(size, int) and size else ""
+            print(f"    {tag}{doc['file_name']}{suffix}")
+        return 0
+
+    skip_steps = ["API Upload", "UI Visibility"]
+    if not args.include_graphdb:
+        skip_steps += ["GraphDB Metadata", "Neptune Metadata"]
+    if not args.include_retrieval:
+        skip_steps.append("Retrieval Quality")
+
+    results = validate_documents(settings, documents, skip_steps=skip_steps)
+
+    # Corpus-level knowledge-base readiness (completeness, dedup, coverage).
+    from ingestion_validation.corpus import analyze_corpus
+
+    corpus = analyze_corpus(
+        documents, results,
+        expected_documents=settings.discovery.extra.get("expected_documents"),
+        expected_folders=settings.discovery.extra.get("expected_folders"),
+    )
+    print(f"  KB readiness: {corpus['verdict']} ({corpus['passed']}/{corpus['total']} passed, "
+          f"{len(corpus['issues'])} issue(s))")
+
+    # Maintain historical trends (stdlib sqlite; on by default) and embed them in the report.
+    trend_rows = _record_trends(results, args.project, args.env, run_id, "Discovery", disabled=args.no_trends)
+
+    passed = sum(1 for r in results if r["result"].all_passed)
+    failed = len(results) - passed
+    base = f"reports/{args.project}-discovery-report"
+    if args.output_format in ("html", "all"):
+        print(f"\n  HTML report: {generate_batch_dashboard(results, base + '.html', project=args.project, environment=args.env, run_id=run_id, trends=trend_rows, corpus=corpus, mode='Discovery')}")
+    if args.output_format in ("json", "all"):
+        print(f"  JSON report: {export_json(results, base + '.json')}")
+    if args.output_format in ("junit", "all"):
+        print(f"  JUnit report: {export_junit(results, base + '.xml')}")
+
+    failed_gate, reason = gate_failed(passed, len(results), corpus["verdict"],
+                                      args.fail_under, args.require_ready)
+    print("=" * 60)
+    print(f"  Results: {passed} passed, {failed} failed ({len(results)} total)")
+    print(f"  Gate: {'FAILED' if failed_gate else 'PASSED'} - {reason}")
+    print("=" * 60)
+    return EXIT_GATE_FAILED if failed_gate else EXIT_OK
+
+
+if __name__ == "__main__":
+    sys.exit(main())

ingestion_validation/__init__.py

@@ -0,0 +1,8 @@
+"""COMPASS — Comprehensive Pipeline Analysis & Structure Search.
+
+A config-driven framework for validating AI/RAG document ingestion pipelines
+end-to-end: API upload -> object storage -> knowledge graph -> vector store ->
+retrieval quality -> UI visibility.
+"""
+
+__version__ = "1.0.0"