codi-api-agent 0.3.1__py3-none-any.whl

codi_api_agent-0.3.1.dist-info/METADATA

@@ -0,0 +1,260 @@
+Metadata-Version: 2.4
+Name: codi-api-agent
+Version: 0.3.1
+Summary: Read-only, spec-driven natural-language agent over any API (OpenAPI/Swagger, GraphQL, or auto-converted Postman/RAML/API Blueprint), with citations and a faithfulness check.
+Author: API Agent
+License: MIT
+Project-URL: Documentation, https://github.com/your-org/api-agent/blob/main/HOW_IT_WORKS.md
+Keywords: openapi,graphql,llm,agent,api,postman,read-only,rag
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Developers
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: openai>=1.30
+Requires-Dist: requests>=2.31
+Requires-Dist: python-dotenv>=1.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: graphql-core>=3.2
+Provides-Extra: ui
+Requires-Dist: streamlit>=1.30; extra == "ui"
+Provides-Extra: embeddings
+Requires-Dist: sentence-transformers>=3.0; extra == "embeddings"
+Requires-Dist: numpy>=1.26; extra == "embeddings"
+Provides-Extra: sql
+Requires-Dist: psycopg[binary]>=3.1; extra == "sql"
+Provides-Extra: all
+Requires-Dist: streamlit>=1.30; extra == "all"
+Requires-Dist: sentence-transformers>=3.0; extra == "all"
+Requires-Dist: numpy>=1.26; extra == "all"
+Provides-Extra: test
+Requires-Dist: pytest>=8; extra == "test"
+Provides-Extra: evals
+Requires-Dist: evals>=3.0.1; extra == "evals"
+Requires-Dist: pyyaml>=6; extra == "evals"
+
+# API Agent
+
+Load an **API description** — OpenAPI/Swagger or GraphQL natively, a Postman
+collection / RAML / API Blueprint (auto-converted on load), or even a **prose API
+reference doc** — ask a question in natural language, and the agent picks the
+right operations, calls them (strictly read-only), and returns an **answer with
+citations** plus a multi-signal **evaluation** (grounding, sufficiency,
+responsiveness). Ships with a Streamlit UI with live steps, streaming answers,
+and token/cost tracking.
+
+```
+Load spec(s) / reference doc ──▶ Catalog of operations
+                     │
+User question ──▶ Router + intent (narrow to relevant ops) ──▶ Executor (call operations)
+                     ──▶ Synthesis + Citation ──▶ Self-review ──▶ Evaluator ──▶ Answer
+```
+
+The LLM provider is an **OpenAI-compatible endpoint**, so the model is a config
+value — Groq, HuggingFace, Ollama and OpenAI are all swappable without code
+changes. The **generator**, **judge** and **router** roles are configured separately.
+Full internals in [How_It_Works.md](How_It_Works.md).
+
+## Install as a pip package (share it / minimal setup)
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install "codi-api-agent[all]"                          # from a package index
+# or from a wheel someone shared with you:
+# pip install "codi_api_agent-0.3.1-py3-none-any.whl[all]"
+export LLM_API_KEY="your-groq-or-openai-key"
+api-agent                                                  # opens the UI at http://localhost:8501
+```
+
+The distribution is `codi-api-agent`; the import name stays `api_agent`
+(`from api_agent import Agent`).
+
+Then add a free no-auth demo spec in the sidebar (Countries GraphQL or the SWAPI Postman
+collection) and ask away. **Full step-by-step in [TUTORIAL.md](TUTORIAL.md).** Build the wheel
+yourself from a checkout with `pip install build && python -m build` (→ `dist/`).
+
+## Setup (from a source checkout)
+
+```bash
+python3 -m venv .venv && source .venv/bin/activate
+pip install -e ".[all]"       # editable install with UI + embeddings
+cp .env.example .env          # then edit .env with your provider + key
+```
+
+| Provider | `LLM_BASE_URL` | Key | Notes |
+|---|---|---|---|
+| **Groq** (recommended) | `https://api.groq.com/openai/v1` | free API key | fast, good tool-calling |
+| **HuggingFace** | `https://router.huggingface.co/v1` | HF token | many open models |
+| **Ollama** (local) | `http://localhost:11434/v1` | any non-empty string | offline; pick a tool-calling model |
+| **OPENAI** | `https://api.openai.com/v1` | api-key | open ai models |
+
+Set `GENERATOR_MODEL` / `JUDGE_MODEL` to models your provider serves **that support
+tool/function calling** (e.g. on Groq, `llama-3.3-70b-versatile`).
+
+**Rate-limit resilience:** give several keys (`LLM_API_KEYS=key1,key2,…`) and/or whole
+backends (`LLM_POOL='[{"base_url":…,"api_key":…,"model":…}, …]'`) and the client rotates
+over them per query and **fails over on a 429** with a cooldown — useful on free tiers.
+
+**Cost controls (for paid, per-token providers):** set `MAX_RESPONSE_TOKENS` to cap tokens per
+response — the agent stops early and returns a partial answer once the ceiling is hit (0 = unlimited).
+Each response also shows an estimated **$ cost**; override the built-in per-model prices with
+`MODEL_PRICING` (JSON per 1M tokens, e.g. `MODEL_PRICING='{"gpt-4o":[2.5,10]}'`) or the sidebar
+**💲 Budget & cost** fields. Unknown/free models simply show no cost.
+
+## Run
+
+```bash
+streamlit run app/streamlit_app.py
+```
+
+It opens empty — add an API via the sidebar's **📚 Load an API** panel (set
+`DEFAULT_SPEC=<url-or-path>` in `.env` to auto-load one on startup). Free, no-auth
+demos to try: the **Countries GraphQL** endpoint `https://countries.trevorblades.com/`
+(tick **GraphQL API**), or the Petstore spec
+`https://petstore.swagger.io/v2/swagger.json` → *“fetch all pets that are sold”*.
+
+## Load any API spec
+
+Sidebar → **📚 Load an API** → paste a spec **URL or file path** (or upload files) →
+**➕ Add spec**. OpenAPI/Swagger and GraphQL load natively; a Postman collection,
+RAML 0.8, or API Blueprint file is **detected from its content and auto-converted**
+to OpenAPI on load. You can add **several sources** (any mix of formats) — each keeps
+its own base URL and auth, and one question can span all of them.
+
+Every `GET`/`HEAD` operation (or GraphQL **query** field) becomes a callable tool
+(write operations are excluded — the read-only guardrail; they remain *describable*
+in documentation mode). Each operation maps: `operationId` → name,
+`summary`/`description` → routing text, `parameters` → arguments, `servers`
+(or Swagger-2.0 `host`+`basePath`) → base URL.
+
+**Authenticated (private) APIs:** open the **Auth (optional)** expander and set a
+header before loading — e.g. `Authorization` = `Bearer <token>`, or `X-API-Key` =
+`<key>`. It's attached to every call. If an API needs auth and none is set, the
+agent says so instead of guessing.
+
+**No spec?** Write a small one from the
+[minimal template](examples/minimal_rest_template.openapi.yaml) — describe just the `GET`
+endpoints you care about (copy a block per endpoint), then load it as a file path. You don't
+need to be an OpenAPI expert; the `summary`/`description` you write are what the agent routes on.
+
+## Other formats (RAML / API Blueprint / Postman)
+
+The agent's pipeline is format-agnostic — only the *loader* speaks OpenAPI — so these are
+**converted to OpenAPI automatically when you load them** (in the UI or via `load_catalog`).
+To pre-convert from the terminal instead:
+
+```bash
+python scripts/convert_spec.py path/to/api.apib  -o specs/api.openapi.json   # API Blueprint
+python scripts/convert_spec.py path/to/api.raml  --base-url https://your-host.com   # RAML 0.8
+python scripts/convert_postman.py your.postman_collection.json --base-url https://your-host.com
+```
+
+| Input | Converts to | Notes |
+|---|---|---|
+| **API Blueprint** (`.apib`) | Swagger 2.0 | read natively |
+| **RAML 0.8** (`.raml`) | OpenAPI 3.0 | RAML **1.0** has no good free CLI converter — convert it to 0.8/OpenAPI first |
+| **Postman collection** | OpenAPI 3.0 | see [scripts/convert_postman.py](scripts/convert_postman.py) |
+
+Requires Node/npx (the converters are npm tools, fetched on first use).
+
+## No spec at all? Load a prose API reference doc
+
+Sidebar → **📚 Load an API** → source type **API reference doc** → point it at a
+reference page (URL, file, or upload) → **🔍 Extract endpoints**. The documented
+`METHOD /path` lines, curl examples, path and query params are extracted from the
+doc's **literal text** (free, deterministic — it cannot invent an endpoint); an
+optional checkbox lets the LLM also enrich param types or handle prose-only docs.
+You then **review and approve** the extracted endpoints before any become callable —
+GETs load as tools, writes as documentation-only.
+
+Inspect what a spec produces from the terminal:
+
+```bash
+python scripts/load_openapi_demo.py                          # default: Petstore
+python scripts/load_openapi_demo.py --source your_spec.yaml --no-call
+```
+
+## Verified working spec URLs (no auth)
+
+| API | Spec URL |
+|---|---|
+| Petstore v2 (pets) — *default* | `https://petstore.swagger.io/v2/swagger.json` |
+| APIs.guru (API directory) | `https://api.apis.guru/v2/specs/apis.guru/2.2.0/openapi.json` |
+| ExchangeRate-API (FX rates) | `https://api.apis.guru/v2/specs/exchangerate-api.com/4/openapi.json` |
+| Color Name API | `https://api.apis.guru/v2/specs/color.pizza/1.0.0/openapi.json` |
+
+## Query your own GitHub repos
+
+GitHub publishes its OpenAPI spec, so you can ask about *your* account:
+
+1. **Load the public spec** (sidebar → **📚 Load an API** → *URL or file path*):
+   `https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json`
+   It has ~624 GET operations; the default **Max operations to load** (1000) loads them
+   all and the router narrows per query.
+2. **Set a token** in the **Auth** expander *before loading*: header `Authorization`,
+   value `Bearer <your GitHub PAT>` (a token with `repo` / read scope).
+3. **Ask:** *“show my repositories”* → routes to `GET /user/repos` and lists your repos.
+
+More once loaded + authed: *“who am I on GitHub?”* (`GET /user`), *“list my open issues”*.
+
+> The big GitHub spec takes a few seconds to fetch/parse on load. Without a token,
+> authed endpoints return 401 and the agent tells you a key is needed.
+
+## Routing (large specs)
+
+When a spec has many operations, the **router** narrows them to the most relevant
+per query before the agent runs: **hybrid** recall — lexical (idf-weighted, stemmed)
+fused with **local embeddings** (`all-MiniLM-L6-v2`, free/offline; catches paraphrases
+with zero shared words) via Reciprocal Rank Fusion — then a fast LLM makes the final
+pick **and classifies the request intent** (write/doc/data) in the same call. Falls
+back to lexical-only if embeddings are unavailable. Specs with ≤ `router_min_tools`
+(default 6) operations skip routing. Configure in the sidebar or via
+`ROUTER_ENABLED` / `ROUTER_TOP_K` / `ROUTER_MODEL` / `ROUTER_MIN_TOOLS` /
+`EMBEDDING_MODEL`.
+
+## Testing & performance report
+
+Two ways to check the agent is behaving:
+
+```bash
+./scripts/run_checks.sh              # deterministic regression suite (no LLM/network) — see tests/README.md
+python evals/run_eval.py --offline   # reliability report (from the suite), zero cost
+python evals/run_eval.py             # full report: reliability + live accuracy on public APIs (needs LLM_API_KEY)
+```
+
+The eval harness writes a stakeholder-facing `evals/report.md` + `report.json` covering guardrail
+reliability, per-category accuracy, faithfulness, read-only/PII safety, abstention, latency, and
+(with `--repeats N`) run-to-run consistency. Details in [evals/README.md](evals/README.md).
+
+## Project layout
+
+```
+api_agent/
+  config.py          # env-driven settings (provider, per-component models, router, budget)
+  llm.py             # OpenAI-compatible client + multi-LLM rotation + tool-call recovery
+  openapi_loader.py  # OpenAPI/Swagger spec -> callable Tools; compaction + PII redaction
+  graphql_loader.py  # GraphQL introspection/SDL + the load_catalog format dispatcher
+  spec_convert.py    # Postman / RAML / API Blueprint -> OpenAPI (auto, via npx)
+  doc_extract.py     # prose API reference doc -> draft OpenAPI (structural; LLM optional)
+  catalog.py         # Tool + Catalog (operation registry + result cache + name recovery)
+  router.py          # hybrid lexical+embedding routing per query
+  schemas.py         # Evidence, Citation, ToolCall, Faithfulness, AgentResult, Usage
+  agent.py           # pipeline: cache -> route -> execute -> synthesize -> review -> evaluate
+  ui.py              # Streamlit app: live steps, streaming, Stop, token/cost, doc review gate
+app/streamlit_app.py # entry point for the UI
+scripts/load_openapi_demo.py  # CLI: inspect a spec + one live call
+tests/               # deterministic regression suite (no LLM/network)
+evals/               # reliability + live-accuracy report harness
+```
+
+## Notes & limitations
+
+- **Read-only:** only `GET`/`HEAD` operations are exposed; write endpoints are never called.
+- **Honest failures:** if an operation needs a key (401/403), is unreachable, or the
+  server returns 5xx, the agent reports that clearly instead of guessing.
+- **Not production-hardened:** the loader fetches the given spec URL and calls
+  endpoints as-is — SSRF egress controls and per-user credential scoping are
+  follow-ups; fine for dev against trusted specs.
+- Public demo servers (e.g. Petstore **v3**) are often flaky — prefer v2 / a spec
+  whose server you control.

codi_api_agent-0.3.1.dist-info/RECORD

@@ -0,0 +1,31 @@
+api_agent/__init__.py,sha256=SRr4n87J_1fiYWtA9eVSdxDyF3v8qiEomYwn_oXCBMc,1819
+api_agent/__main__.py,sha256=cm4GAplCdbTKqB4q4GHQ46XSveEvil-aXo0pEUMQdgM,161
+api_agent/_version.py,sha256=Q5frm4VFOXY1kVW1GZfGSmZjoxaajJ26mTtEH0P19P8,107
+api_agent/agent.py,sha256=VQFI0rqLUBd8cOHs74wJHxQUKF2vjLsd5K01GlZqTts,409724
+api_agent/catalog.py,sha256=c_yASjMu16Adwnr9jWxydWt8vsf3WkDSh2kwtpkTc10,7201
+api_agent/chart.py,sha256=EniCGc5GQcaazBlpNG45trem3EXae2YXT_5QfTwyK8c,7558
+api_agent/cli.py,sha256=exarta1dns3eoABAc7dJnhc6OxDBslumSTachRmLhyA,1101
+api_agent/config.py,sha256=ZD-rBzgrH8WX5Ld3d7G5_HOQVam0mxNW0MCNq74zDpc,18978
+api_agent/doc_extract.py,sha256=f8-xjlqEK5hLu7c0Dios22Tcy3zswd8mSxVs2cZD7OM,20919
+api_agent/graphql_loader.py,sha256=fDK2E0gNt3N_m1wF-BLCrOpS8P3zGW_KDrgA4fcvA8w,14718
+api_agent/llm.py,sha256=JTkUnbdP-WNFx2tEydOxY6bWkh3NzolU_DaYd7Q8GhQ,21736
+api_agent/log.py,sha256=7-DxreeRbpOt1umtkXtGfM4ZJ1xugMjddpzaFto983E,6067
+api_agent/metrics.py,sha256=nd72YB94erPYvFxKpsOKbnjgcdFzdTwbKPKspGYpNMs,54181
+api_agent/openapi_loader.py,sha256=c3iSbAulqM7I6CQQ78I_z-zHuxl1G2oLmrTWCAj_bH0,27877
+api_agent/router.py,sha256=m3uRrwiyL_nABDQS3P9ciwSRiPHSkS5ovKZMWFrATLs,14341
+api_agent/schemas.py,sha256=jORxThmadYswOAcOMzwsQqjwllde8SkRrYnwcsQbPmM,12576
+api_agent/spec_convert.py,sha256=bqQ_qrEOFbVuX_PAsoeTZhY-s8qRVliiMTyAggqZhDA,3901
+api_agent/sql_loader.py,sha256=Os6-qcd8woT2Bm3xwglGuW14FI6cJelZPeIO6WSc_lU,67459
+api_agent/supervisor.py,sha256=VolOu-V57BUKtEZ_HgFXtPXh3C0zSmK8b8PCxyt8GnU,8846
+api_agent/ui.py,sha256=FYVY1yrGfANKEtMenVCIcn1vGH7G-aM7nJfVkgj_na4,51473
+api_agent/prompts/__init__.py,sha256=4sA6Dpg1ef8W-vS8OOz-MA38PXBzz2XrPQvnTfmbnL4,1414
+api_agent/prompts/advisory.py,sha256=rU0cd03gaGTFUP7zSo_wuD8cz6vETTX_vlMGwc8FZvU,3924
+api_agent/prompts/executor.py,sha256=TcOwSWwCE5cPRxAUMKBRoryFLARPfMTldvKtk-04fsc,14119
+api_agent/prompts/judges.py,sha256=TJxqOqC9LxZrnZHcRl3jSpkB1J7qg1rrnrxlIvmzpZs,15954
+api_agent/prompts/support.py,sha256=ISAqipjK7-cMllEWxhzzyPiPa2he2SL8mBtMM3EXusk,4634
+api_agent/prompts/synthesis.py,sha256=0fY-VOOe4dFhPRX4H_8ud0k4hEkuuAREHuorXY-1ky0,29069
+codi_api_agent-0.3.1.dist-info/METADATA,sha256=MiAX1A-EpyqwvX26_CGaIUj_N-JclgFav_lpIGiCzEg,13250
+codi_api_agent-0.3.1.dist-info/WHEEL,sha256=YVMoNqKzERt-wjUZwJ33xBGAwnFl-4cqbYkTtWa4itE,91
+codi_api_agent-0.3.1.dist-info/entry_points.txt,sha256=15AwPe9GGrAW7MpgyHqSfMWHS2lcIvKFQJVKHZMMrqQ,49
+codi_api_agent-0.3.1.dist-info/top_level.txt,sha256=H84mDK01gK9siU_tWSmMMa4U8gZo3nduyH9-O0T4LZc,10
+codi_api_agent-0.3.1.dist-info/RECORD,,

codi_api_agent-0.3.1.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (84.0.0)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

codi_api_agent-0.3.1.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+api-agent = api_agent.cli:main

codi_api_agent-0.3.1.dist-info/top_level.txt

@@ -0,0 +1 @@
+api_agent