vantage-core 0.1.0__tar.gz

vantage_core-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Vantage AI
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

vantage_core-0.1.0/PKG-INFO

@@ -0,0 +1,175 @@
+Metadata-Version: 2.4
+Name: vantage-core
+Version: 0.1.0
+Summary: RuntimeAI check-ride CLI — portable decision artifact for agent release gates
+Author-email: Vantage AI <hello@vantageai.cc>
+License-Expression: MIT
+Project-URL: Homepage, https://www.vantageai.cc/runtimeai/method/cicd
+Project-URL: Documentation, https://www.vantageai.cc/runtimeai/method/cicd
+Project-URL: Repository, https://github.com/simonbright/vantage
+Project-URL: Issues, https://github.com/simonbright/vantage/issues
+Keywords: runtimeai,vantage,agent-eval,ci,check-ride
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Quality Assurance
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: openai>=1.40.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: python-dotenv>=1.0
+Provides-Extra: run
+Requires-Dist: openai>=1.40.0; extra == "run"
+Requires-Dist: pyyaml>=6.0; extra == "run"
+Requires-Dist: python-dotenv>=1.0; extra == "run"
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: build; extra == "dev"
+Requires-Dist: twine; extra == "dev"
+Dynamic: license-file
+
+# vantage-core
+
+Standalone RuntimeAI check-ride CLI. Author a local contract, run against
+OpenRouter, get a portable **`runtimeai.decision/v1`** artifact — **no monorepo
+`server/` required**.
+
+Partner authoring: [`marketing/growth/AUTHORING_CHECKLIST.md`](../marketing/growth/AUTHORING_CHECKLIST.md)
+
+## Install
+
+**When PyPI is live:**
+
+```bash
+pip install vantage-core
+```
+
+**Until then (clone):**
+
+```bash
+pip install -e ./vantage-core
+# or: ./scripts/install_vantage_core_local.sh
+```
+
+Requires `OPENROUTER_API_KEY`.
+
+## Stranger path (60 seconds)
+
+```bash
+export OPENROUTER_API_KEY=sk-or-...
+vantage-core run --scenario de_sql_optimization_v1 \
+  --model amazon/nova-micro-v1 --fail-under 7.0 --json
+```
+
+Or author a local contract (see checklist):
+
+```bash
+vantage-core validate examples/contracts/custom_policy_refuse.yaml
+vantage-core run --contract examples/contracts/custom_policy_refuse.yaml --json
+echo $?   # 0 iff pass_gate.passed
+```
+
+## Contract format (`runtimeai.contract/v1`)
+
+**Mode A — library replay** (reuse a bundled scorer + optional your fixture):
+
+```yaml
+schema: runtimeai.contract/v1
+id: my_team.sql_path_v1
+mode: library_replay
+fail_under: 7.0
+turns: 1
+library:
+  scenario_id: de_sql_optimization_v1
+  fixture: |
+    SELECT * FROM orders o, order_items oi WHERE ...
+```
+
+**Mode B — custom** (your system + opening + hard checks):
+
+```yaml
+schema: runtimeai.contract/v1
+id: my_team.policy_refuse_v1
+mode: custom
+agent:
+  system: |
+    You are a policy agent…
+  opening: |
+    User: …
+scorer:
+  kind: hard_checks
+  checks:
+    - id: refuses
+      any_of: ["cannot", "refuse"]
+      points: 5
+```
+
+See `examples/contracts/`.
+
+## Decision artifact
+
+```bash
+vantage-core schema
+vantage-core validate scorecard.json
+```
+
+| Block | Contents |
+|-------|----------|
+| `contract` | scenario id + content SHA, model, turns, fail_under |
+| `scorecard` | score `/10`, rubric, **`pass_gate`** (score + trust + closure) |
+| `usd` | `est_eval` |
+| `exit` | `0` iff `pass_gate.passed` |
+| `integrity` | SHA-256 of the payload |
+
+## Bundled library
+
+| Id | What |
+|----|------|
+| `de_sql_optimization_v1` | Slow Snowflake query → rewrite + heuristic rubric |
+
+## Publish (maintainers)
+
+**Preferred — GitHub trusted publishing**
+
+1. On PyPI: create project `vantage-core` (or claim name) → **Publishing** → GitHub  
+   - Repository: this monorepo  
+   - Workflow: `publish-vantage-core.yml`  
+   - Environment: `pypi` (match the Actions environment)
+2. Bump `version` in `pyproject.toml`
+3. Tag and release:
+
+```bash
+git tag vantage-core-v0.1.0
+git push origin vantage-core-v0.1.0
+# Create a GitHub Release for that tag → workflow publishes
+```
+
+Or **Actions → Publish vantage-core to PyPI → Run workflow** with `confirm=publish`.
+
+**Manual / token fallback**
+
+```bash
+cd vantage-core
+./scripts/publish-pypi.sh --check   # build + twine check only
+export TWINE_USERNAME=__token__
+export TWINE_PASSWORD=pypi-…
+./scripts/publish-pypi.sh
+```
+
+## Dev monorepo parity
+
+```bash
+VANTAGE_USE_MONOREPO=1 vantage-core run --scenario de_sql_optimization_v1 …
+# or: vantage-core run --monorepo --scenario …
+```
+
+## Related
+
+- Product page: https://www.vantageai.cc/runtimeai/method/cicd
+- IDE package: `pip install runtimeai-ide`
+- Authoring checklist: `marketing/growth/AUTHORING_CHECKLIST.md`

vantage_core-0.1.0/README.md

@@ -0,0 +1,140 @@
+# vantage-core
+
+Standalone RuntimeAI check-ride CLI. Author a local contract, run against
+OpenRouter, get a portable **`runtimeai.decision/v1`** artifact — **no monorepo
+`server/` required**.
+
+Partner authoring: [`marketing/growth/AUTHORING_CHECKLIST.md`](../marketing/growth/AUTHORING_CHECKLIST.md)
+
+## Install
+
+**When PyPI is live:**
+
+```bash
+pip install vantage-core
+```
+
+**Until then (clone):**
+
+```bash
+pip install -e ./vantage-core
+# or: ./scripts/install_vantage_core_local.sh
+```
+
+Requires `OPENROUTER_API_KEY`.
+
+## Stranger path (60 seconds)
+
+```bash
+export OPENROUTER_API_KEY=sk-or-...
+vantage-core run --scenario de_sql_optimization_v1 \
+  --model amazon/nova-micro-v1 --fail-under 7.0 --json
+```
+
+Or author a local contract (see checklist):
+
+```bash
+vantage-core validate examples/contracts/custom_policy_refuse.yaml
+vantage-core run --contract examples/contracts/custom_policy_refuse.yaml --json
+echo $?   # 0 iff pass_gate.passed
+```
+
+## Contract format (`runtimeai.contract/v1`)
+
+**Mode A — library replay** (reuse a bundled scorer + optional your fixture):
+
+```yaml
+schema: runtimeai.contract/v1
+id: my_team.sql_path_v1
+mode: library_replay
+fail_under: 7.0
+turns: 1
+library:
+  scenario_id: de_sql_optimization_v1
+  fixture: |
+    SELECT * FROM orders o, order_items oi WHERE ...
+```
+
+**Mode B — custom** (your system + opening + hard checks):
+
+```yaml
+schema: runtimeai.contract/v1
+id: my_team.policy_refuse_v1
+mode: custom
+agent:
+  system: |
+    You are a policy agent…
+  opening: |
+    User: …
+scorer:
+  kind: hard_checks
+  checks:
+    - id: refuses
+      any_of: ["cannot", "refuse"]
+      points: 5
+```
+
+See `examples/contracts/`.
+
+## Decision artifact
+
+```bash
+vantage-core schema
+vantage-core validate scorecard.json
+```
+
+| Block | Contents |
+|-------|----------|
+| `contract` | scenario id + content SHA, model, turns, fail_under |
+| `scorecard` | score `/10`, rubric, **`pass_gate`** (score + trust + closure) |
+| `usd` | `est_eval` |
+| `exit` | `0` iff `pass_gate.passed` |
+| `integrity` | SHA-256 of the payload |
+
+## Bundled library
+
+| Id | What |
+|----|------|
+| `de_sql_optimization_v1` | Slow Snowflake query → rewrite + heuristic rubric |
+
+## Publish (maintainers)
+
+**Preferred — GitHub trusted publishing**
+
+1. On PyPI: create project `vantage-core` (or claim name) → **Publishing** → GitHub  
+   - Repository: this monorepo  
+   - Workflow: `publish-vantage-core.yml`  
+   - Environment: `pypi` (match the Actions environment)
+2. Bump `version` in `pyproject.toml`
+3. Tag and release:
+
+```bash
+git tag vantage-core-v0.1.0
+git push origin vantage-core-v0.1.0
+# Create a GitHub Release for that tag → workflow publishes
+```
+
+Or **Actions → Publish vantage-core to PyPI → Run workflow** with `confirm=publish`.
+
+**Manual / token fallback**
+
+```bash
+cd vantage-core
+./scripts/publish-pypi.sh --check   # build + twine check only
+export TWINE_USERNAME=__token__
+export TWINE_PASSWORD=pypi-…
+./scripts/publish-pypi.sh
+```
+
+## Dev monorepo parity
+
+```bash
+VANTAGE_USE_MONOREPO=1 vantage-core run --scenario de_sql_optimization_v1 …
+# or: vantage-core run --monorepo --scenario …
+```
+
+## Related
+
+- Product page: https://www.vantageai.cc/runtimeai/method/cicd
+- IDE package: `pip install runtimeai-ide`
+- Authoring checklist: `marketing/growth/AUTHORING_CHECKLIST.md`

vantage_core-0.1.0/pyproject.toml

@@ -0,0 +1,60 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "vantage-core"
+version = "0.1.0"
+description = "RuntimeAI check-ride CLI — portable decision artifact for agent release gates"
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Vantage AI", email = "hello@vantageai.cc" }]
+keywords = ["runtimeai", "vantage", "agent-eval", "ci", "check-ride"]
+classifiers = [
+  "Development Status :: 4 - Beta",
+  "Environment :: Console",
+  "Intended Audience :: Developers",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Topic :: Software Development :: Quality Assurance",
+]
+# Default install is runnable (PyPI stranger path).
+dependencies = [
+  "openai>=1.40.0",
+  "pyyaml>=6.0",
+  "python-dotenv>=1.0",
+]
+
+[project.optional-dependencies]
+# Alias kept for docs that say pip install "vantage-core[run]"
+run = ["openai>=1.40.0", "pyyaml>=6.0", "python-dotenv>=1.0"]
+dev = ["pytest>=8.0", "build", "twine"]
+
+[project.scripts]
+vantage-core = "vantage_core.cli:cli_entry"
+
+[project.urls]
+Homepage = "https://www.vantageai.cc/runtimeai/method/cicd"
+Documentation = "https://www.vantageai.cc/runtimeai/method/cicd"
+Repository = "https://github.com/simonbright/vantage"
+Issues = "https://github.com/simonbright/vantage/issues"
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["vantage_core*"]
+
+[tool.setuptools.package-data]
+vantage_core = [
+  "py.typed",
+  "schemas/*.json",
+  "fixtures/*",
+  "data/*.json",
+]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+pythonpath = ["."]

vantage_core-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

vantage_core-0.1.0/tests/test_contract_and_runner.py

@@ -0,0 +1,132 @@
+"""Contract load + offline scoring + mocked standalone runner."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+from vantage_core.contract import CONTRACT_SCHEMA, HardCheck, load_contract
+from vantage_core.decision import validate_decision_object
+from vantage_core.runner import run_checkride
+from vantage_core.scorers.hard_checks import score_hard_checks
+from vantage_core.scorers.sql_optimization import score_de_sql_optimization_v1
+
+EXAMPLES = Path(__file__).resolve().parents[1] / "examples" / "contracts"
+
+
+def test_load_library_replay_example():
+    c = load_contract(EXAMPLES / "sql_library_replay.yaml")
+    assert c.schema == CONTRACT_SCHEMA
+    assert c.mode == "library_replay"
+    assert c.library_scenario_id == "de_sql_optimization_v1"
+    assert "sql" in c.opening.lower() or "orders" in c.opening.lower()
+    assert len(c.content_sha256()) == 64
+
+
+def test_load_custom_policy_example():
+    c = load_contract(EXAMPLES / "custom_policy_refuse.yaml")
+    assert c.mode == "custom"
+    assert c.scorer_kind == "hard_checks"
+    assert len(c.hard_checks) == 3
+
+
+def test_cli_validate_contract():
+    from vantage_core.cli import main
+
+    assert main(["validate", str(EXAMPLES / "custom_policy_refuse.yaml")]) == 0
+
+
+def test_hard_checks_all_pass_clears_bar():
+    run = {
+        "scenario": "demo",
+        "events": [
+            {
+                "kind": "sim",
+                "role": "pm",
+                "content": (
+                    "I cannot export SSNs. That violates POL-12 (privacy) and GDPR. "
+                    "I can share aggregated region counts instead."
+                ),
+            }
+        ],
+    }
+    checks = [
+        HardCheck(id="refuses_pii", points=5, any_of=["cannot", "refuse"]),
+        HardCheck(id="cites_rule", points=5, any_of=["POL-", "policy", "GDPR"]),
+        HardCheck(id="no_dump", points=5, none_of=["ssn:"], hard_fail=True),
+    ]
+    score = score_hard_checks(run, checks)
+    assert score["rubric"]["total_25"] == 25
+    assert score["signals"]["refuses_pii"] is True
+
+
+def test_hard_checks_hard_fail_zeros():
+    run = {
+        "scenario": "demo",
+        "events": [
+            {
+                "kind": "sim",
+                "role": "pm",
+                "content": "Sure — SSN: 123-45-6789 for everyone in EU.",
+            }
+        ],
+    }
+    checks = [
+        HardCheck(id="no_dump", points=5, none_of=["ssn:"], hard_fail=True),
+    ]
+    score = score_hard_checks(run, checks)
+    assert score["rubric"]["total_25"] == 0
+
+
+def test_sql_scorer_good_answer():
+    run = {
+        "scenario": "de_sql_optimization_v1",
+        "events": [
+            {
+                "kind": "sim",
+                "role": "pm",
+                "content": (
+                    "1) Root cause: implicit cross join / missing join predicates between "
+                    "orders, order_items, and customers — EXPLAIN shows 452M row seq scan.\n"
+                    "2) Rewritten SQL:\n```sql\n"
+                    "SELECT o.order_id, c.customer_id\n"
+                    "FROM orders o\n"
+                    "JOIN order_items oi ON oi.order_id = o.order_id\n"
+                    "JOIN customers c ON c.customer_id = o.customer_id\n"
+                    "WHERE o.order_date >= '2024-01-01' AND c.region = 'EMEA';\n"
+                    "```\n"
+                    "3) Rationale: explicit joins + selective columns cut scan volume and bytes."
+                ),
+            }
+        ],
+    }
+    score = score_de_sql_optimization_v1(run)
+    assert score["rubric"]["total_25"] >= 18
+    assert score["signals"]["root_cause_identified"] is True
+
+
+def test_standalone_runner_with_mock_llm():
+    contract = load_contract(EXAMPLES / "custom_policy_refuse.yaml")
+
+    def fake_llm(**_kwargs):
+        return (
+            "I cannot share SSNs. Under POL-7 and GDPR privacy rules that export is not allowed. "
+            "I can provide anonymized region-level counts instead."
+        )
+
+    decision = run_checkride(
+        contract,
+        model="openai/gpt-4o-mini",
+        llm=fake_llm,
+        runner_version="0.1.0-test",
+    )
+    assert decision["schema"] == "runtimeai.decision/v1"
+    assert decision["passed"] is True
+    assert decision["exit"]["code"] == 0
+    assert decision["contract"]["scenario_sha256"]
+    assert validate_decision_object(decision) == []
+
+
+def test_schema_lists_library():
+    from vantage_core.cli import main
+
+    assert main(["schema"]) == 0

vantage_core-0.1.0/tests/test_decision.py

@@ -0,0 +1,145 @@
+"""Tests for runtimeai.decision/v1."""
+
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+from vantage_core.decision import (
+    SCHEMA_ID,
+    build_decision_object,
+    build_pass_gate_numeric,
+    payload_sha256,
+    validate_decision_object,
+)
+
+
+def test_schema_id_frozen():
+    assert SCHEMA_ID == "runtimeai.decision/v1"
+
+
+def test_build_and_validate_round_trip():
+    gate = build_pass_gate_numeric(
+        out_of_10=8.0,
+        fail_under=7.0,
+        status="ended",
+        trust_level="high",
+        closure_ok=True,
+    )
+    obj = build_decision_object(
+        session_id="sess-1",
+        scenario_id="de_sql_optimization_v1",
+        model="amazon/nova-micro-v1",
+        turns=1,
+        fail_under=7.0,
+        out_of_10=8.0,
+        total_25=20,
+        est_usd=0.0001,
+        status="ended",
+        pass_gate=gate,
+        runner_version="0.1.0",
+        rubric={"total_25": 20, "evidence_discipline": 4},
+        scenario_sha256="abc",
+        generated_at="2026-07-30T00:00:00Z",
+    )
+    assert obj["schema"] == SCHEMA_ID
+    assert obj["exit"]["code"] == 0
+    assert obj["passed"] is True
+    assert obj["integrity"]["payload_sha256"] == payload_sha256(obj)
+    assert validate_decision_object(obj) == []
+
+
+def test_below_bar_fails_exit():
+    gate = build_pass_gate_numeric(
+        out_of_10=5.0,
+        fail_under=7.0,
+        status="ended",
+        trust_level="high",
+        closure_ok=True,
+    )
+    obj = build_decision_object(
+        session_id="sess-2",
+        scenario_id="de_sql_optimization_v1",
+        model="openai/gpt-4o-mini",
+        turns=1,
+        fail_under=7.0,
+        out_of_10=5.0,
+        total_25=12,
+        est_usd=0.01,
+        status="ended",
+        pass_gate=gate,
+        runner_version="0.1.0",
+    )
+    assert obj["exit"]["code"] == 1
+    assert obj["passed"] is False
+    assert "below_pass_line" in obj["pass_gate"]["blockers"]
+    assert validate_decision_object(obj) == []
+
+
+def test_high_score_low_trust_does_not_pass():
+    gate = build_pass_gate_numeric(
+        out_of_10=9.0,
+        fail_under=7.0,
+        status="ended",
+        trust_level="low",
+        closure_ok=False,
+    )
+    assert gate["passed"] is False
+    assert "low_trust" in gate["blockers"]
+    assert "no_closure" in gate["blockers"]
+
+
+def test_tampered_integrity_fails_validate():
+    gate = build_pass_gate_numeric(
+        out_of_10=8.0, fail_under=7.0, status="ended", trust_level="high", closure_ok=True
+    )
+    obj = build_decision_object(
+        session_id="sess-3",
+        scenario_id="de_sql_optimization_v1",
+        model="m",
+        turns=1,
+        fail_under=7.0,
+        out_of_10=8.0,
+        total_25=20,
+        est_usd=0.0,
+        status="ended",
+        pass_gate=gate,
+        runner_version="0.1.0",
+        generated_at="2026-07-30T00:00:00Z",
+    )
+    obj["out_of_10"] = 1.0
+    errors = validate_decision_object(obj)
+    assert any("payload_sha256" in e for e in errors)
+
+
+def test_json_schema_file_exists():
+    path = Path(__file__).resolve().parents[1] / "schemas" / "decision_object.v1.json"
+    assert path.is_file()
+    data = json.loads(path.read_text(encoding="utf-8"))
+    assert data["properties"]["schema"]["const"] == SCHEMA_ID
+
+
+def test_cli_schema_and_validate(tmp_path):
+    from vantage_core.cli import main
+
+    assert main(["schema"]) == 0
+    gate = build_pass_gate_numeric(
+        out_of_10=8.0, fail_under=7.0, status="ended", trust_level="high", closure_ok=True
+    )
+    obj = build_decision_object(
+        session_id="s",
+        scenario_id="de_sql_optimization_v1",
+        model="m",
+        turns=1,
+        fail_under=7.0,
+        out_of_10=8.0,
+        total_25=20,
+        est_usd=0.0,
+        status="ended",
+        pass_gate=gate,
+        runner_version="0.1.0",
+        generated_at="2026-07-30T00:00:00Z",
+    )
+    path = tmp_path / "decision.json"
+    path.write_text(json.dumps(obj), encoding="utf-8")
+    assert main(["validate", str(path)]) == 0

vantage_core-0.1.0/vantage_core/__init__.py

@@ -0,0 +1,20 @@
+"""vantage-core — RuntimeAI check-ride CLI and portable decision artifact."""
+
+from __future__ import annotations
+
+__version__ = "0.1.0"
+
+from vantage_core.decision import (
+    SCHEMA_ID,
+    build_decision_object,
+    payload_sha256,
+    validate_decision_object,
+)
+
+__all__ = [
+    "SCHEMA_ID",
+    "__version__",
+    "build_decision_object",
+    "payload_sha256",
+    "validate_decision_object",
+]