@kirrosh/apitool 0.4.3

package/.github/workflows/ci.yml

@@ -0,0 +1,27 @@
+name: CI
+
+on:
+  push:
+    branches: [main, dev]
+  pull_request:
+    branches: [main]
+
+jobs:
+  check:
+    name: Typecheck & Test
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: oven-sh/setup-bun@v2
+        with:
+          bun-version: latest
+
+      - run: bun install
+
+      - name: Typecheck
+        run: bunx tsc --noEmit
+
+      - name: Tests
+        run: bun run test

package/.github/workflows/release.yml

@@ -0,0 +1,97 @@
+name: Release
+
+on:
+  push:
+    tags:
+      - "v*"
+
+permissions:
+  contents: write
+
+jobs:
+  build:
+    name: Build (${{ matrix.target }})
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        include:
+          - os: ubuntu-latest
+            target: linux-x64
+            artifact: apitool
+            archive: tar.gz
+          - os: macos-latest
+            target: darwin-arm64
+            artifact: apitool
+            archive: tar.gz
+          - os: windows-latest
+            target: win-x64
+            artifact: apitool.exe
+            archive: zip
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: oven-sh/setup-bun@v2
+        with:
+          bun-version: latest
+
+      - run: bun install
+
+      - name: Build binary
+        run: bun build --compile src/cli/index.ts --outfile apitool
+
+      - name: Package (tar.gz)
+        if: matrix.archive == 'tar.gz'
+        run: tar -czf apitool-${{ matrix.target }}.tar.gz ${{ matrix.artifact }}
+
+      - name: Package (zip)
+        if: matrix.archive == 'zip'
+        shell: pwsh
+        run: Compress-Archive -Path ${{ matrix.artifact }} -DestinationPath apitool-${{ matrix.target }}.zip
+
+      - uses: actions/upload-artifact@v4
+        with:
+          name: apitool-${{ matrix.target }}
+          path: apitool-${{ matrix.target }}.${{ matrix.archive }}
+
+  release:
+    name: Publish Release
+    needs: build
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          merge-multiple: true
+
+      - name: Create GitHub Release
+        uses: softprops/action-gh-release@v2
+        with:
+          generate_release_notes: true
+          files: |
+            apitool-linux-x64.tar.gz
+            apitool-darwin-arm64.tar.gz
+            apitool-win-x64.zip
+
+  publish:
+    name: Publish to npm
+    needs: release
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: "20"
+          registry-url: "https://registry.npmjs.org"
+
+      - uses: oven-sh/setup-bun@v2
+        with:
+          bun-version: latest
+
+      - run: bun install
+
+      - run: npm publish --access public
+        env:
+          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

package/.mcp.json

@@ -0,0 +1,9 @@
+{
+  "mcpServers": {
+    "apitool": {
+      "command": "bun",
+      "args": ["run", "C:/projects/apitool/src/cli/index.ts", "mcp", "--dir", "C:/projects/apitool"],
+      "cwd": "C:/projects/apitool"
+    }
+  }
+}

package/APITOOL.md

@@ -0,0 +1,195 @@
+# APITOOL
+
+**AI-native API testing tool** — OpenAPI spec → test generation → execution → diagnostics. One binary. Zero config.
+
+- **MCP** — primary interface for AI agents (Claude Code, Cursor, Windsurf)
+- **CLI** — for humans and CI/CD
+- **WebUI** — dashboard for viewing results
+
+---
+
+## Quick Start
+
+```bash
+apitool update                    # install / update
+apitool add-api myapi --spec openapi.json   # register API
+apitool ai-generate --api myapi --prompt "test all CRUD endpoints"
+apitool run --api myapi --env staging
+apitool serve --port 4000         # web dashboard
+```
+
+### MCP Setup
+
+```json
+{ "mcpServers": { "apitool": { "command": "apitool", "args": ["mcp"] } } }
+```
+
+Then ask your AI agent: *"Test the API from openapi.json"*
+
+---
+
+## Architecture
+
+```
+src/
+├── core/
+│   ├── parser/       YAML → TestSuite (schema, variables, generators)
+│   ├── runner/       HTTP execution, captures, assertions
+│   ├── generator/    OpenAPI reader, coverage scanner, AI generation
+│   ├── reporter/     Console, JSON, JUnit XML
+│   └── agent/        AI Chat (AI SDK v6, tool calling)
+├── db/               SQLite (runs, collections, environments)
+├── mcp/              MCP Server (15 tools)
+├── web/              Hono + HTMX dashboard
+└── cli/              16 CLI commands
+```
+
+### Stack
+
+Bun runtime, TypeScript, SQLite (`bun:sqlite`), Hono + HTMX, `bun build --compile` → single binary.
+
+---
+
+## Modules
+
+### Parser
+Reads YAML test files → `TestSuite[]`. Schema validation (Zod), variable interpolation (`{{base_url}}`), built-in generators (`{{$randomEmail}}`, `{{$uuid}}`, etc.), nested body assertion flattening.
+
+### Runner
+Executes test steps sequentially within a suite. Native `fetch`, captures (`{ capture: "token" }`), assertions (equals, type, contains, matches, gt/lt, exists), nested body paths (`category.name`), root body checks (`_body: { type: "array" }`).
+
+### Generator
+Reads OpenAPI specs (`@readme/openapi-parser`), compresses schemas for LLM context, scans test coverage, AI-based test generation (Ollama/OpenAI/Anthropic).
+
+### Reporter
+Console (colored, tags display), JSON, JUnit XML (CI-compatible).
+
+### Agent
+Interactive AI chat (`apitool chat`). AI SDK v6, tool calling, multi-provider (Ollama, OpenAI, Anthropic). Safe mode (GET-only).
+
+### DB
+SQLite auto-created. Tables: `collections`, `runs`, `results`, `environments`, `ai_generations`. Schema version 5.
+
+### WebUI
+Single-page dashboard: API selector → env selector → Run Tests → results + coverage + history. JUnit/JSON export. Hono + HTMX.
+
+### MCP Server
+15 tools for AI agent integration. Primary test generation flow:
+
+```
+generate_tests_guide → [agent writes YAML] → save_test_suite → run_tests → diagnose_failure → ci_init
+```
+
+### CI/CD
+`apitool ci init` scaffolds GitHub Actions or GitLab CI workflow. Supports schedule, repository_dispatch, manual triggers. See [docs/ci.md](docs/ci.md).
+
+---
+
+## MCP Tools
+
+| Tool | Description |
+|------|-------------|
+| `generate_tests_guide` | Full API spec + generation algorithm. Use **before** writing tests |
+| `generate_missing_tests` | Guide for only uncovered endpoints |
+| `save_test_suite` | Validate YAML + save file. Returns coverage hint |
+| `run_tests` | Execute tests, return summary with failures |
+| `query_db` | List collections, runs, results, diagnose failures |
+| `explore_api` | Browse OpenAPI spec (`includeSchemas=true` for schemas) |
+| `coverage_analysis` | Compare spec vs existing tests |
+| `validate_tests` | Check YAML syntax without running |
+| `send_request` | Ad-hoc HTTP request with variable interpolation |
+| `setup_api` | Register API (dirs + spec + env + collection) |
+| `manage_environment` | CRUD for environments |
+| `manage_server` | Start/stop WebUI server |
+| `ci_init` | Generate CI/CD workflow (GitHub Actions / GitLab CI) |
+
+## CLI Commands
+
+| Command | Description | Key flags |
+|---------|-------------|-----------|
+| `add-api <name>` | Register new API | `--spec`, `--dir`, `--env key=value` |
+| `run <path>` | Run tests | `--api`, `--env`, `--report`, `--safe`, `--tag`, `--bail` |
+| `ai-generate` | AI test generation | `--api`, `--from`, `--prompt`, `--provider`, `--model` |
+| `validate` | Validate YAML tests | |
+| `envs` | Environment management | `list\|get\|set\|delete\|import\|export`, `--api` |
+| `runs [id]` | Run history | `--limit` |
+| `coverage` | API test coverage | `--api`, `--spec`, `--tests` |
+| `collections` | List collections | |
+| `serve` | Web dashboard | `--port`, `--watch` |
+| `chat` | Interactive AI agent | `--provider`, `--model`, `--safe` |
+| `mcp` | Start MCP server | `--db` |
+| `ci init` | Generate CI/CD workflow | `--github`, `--gitlab`, `--dir`, `--force` |
+| `init` | Scaffold new project | |
+| `doctor` | Diagnostics | |
+| `update` | Self-update | |
+
+---
+
+## YAML Test Format
+
+```yaml
+name: Users CRUD
+description: "Full lifecycle test"
+tags: [users, crud]
+base_url: "{{base_url}}"
+headers:
+  Authorization: "Bearer {{auth_token}}"
+
+tests:
+  - name: "Create user"
+    POST: /users
+    json:
+      name: "{{$randomName}}"
+      email: "{{$randomEmail}}"
+    expect:
+      status: 201
+      body:
+        id: { capture: user_id, type: integer }
+
+  - name: "Get user"
+    GET: /users/{{user_id}}
+    expect:
+      status: 200
+      body:
+        id: { equals: "{{user_id}}" }
+
+  - name: "Delete user"
+    DELETE: /users/{{user_id}}
+    expect:
+      status: 204
+```
+
+### Assertions
+
+`equals`, `type`, `capture`, `contains`, `matches`, `gt`, `lt`, `exists` (boolean). Nested: `category.name: { equals: "Dogs" }`. Root body: `_body: { type: "array" }`.
+
+### Generators
+
+`{{$randomInt}}`, `{{$uuid}}`, `{{$timestamp}}`, `{{$randomEmail}}`, `{{$randomString}}`, `{{$randomName}}`
+
+### Environments
+
+```yaml
+# .env.staging.yaml
+base_url: https://staging.example.com/api
+token: staging-token
+```
+
+`apitool run tests/ --env staging`
+
+---
+
+## Build
+
+```bash
+bun run build    # → apitool binary (standalone)
+bun test         # run test suite
+```
+
+## Principles
+
+1. **One file** — download binary, run. No Docker, no npm.
+2. **Tests as code** — YAML in git, code review, CI/CD.
+3. **OpenAPI-first** — spec exists → tests generate.
+4. **AI-native** — MCP for agents, CLI for humans, same engine.
+5. **SQLite by default** — history works out of the box.

package/BACKLOG.md

@@ -0,0 +1,62 @@
+# Backlog
+
+## High Priority
+
+### CI/CD Improvements
+- **`--fail-on-coverage` flag** — fail CI if coverage below threshold: `apitool run --fail-on-coverage 80`
+- **`--env-var` flag** — pass secrets from CI without env files: `apitool run --env-var "token=$API_TOKEN"`
+- **GitHub Action** — `uses: kirrosh/apitool-action@v1` composite action (separate repo)
+
+### One-shot test generation (`generate_and_save`)
+New MCP tool that accepts spec path + optional endpoint filter and produces ready-to-save YAML test suites in one call. Eliminates the 3-step chain: `generate_tests_guide` → agent assembles YAML → `save_test_suite`.
+
+---
+
+## Medium Priority
+
+### Batch `save_test_suite`
+Accept an array of `{filePath, content}` pairs in a single call. Reduces token usage and round-trips for bulk generation.
+
+### Split format guide from endpoint data
+`generate_tests_guide` returns the full YAML format reference every call. Split into static format + dynamic endpoints, or cache format after first call.
+
+### `apitool compare` command
+Compare two test runs — regression detection in CI.
+
+### Timestamp capture pattern
+When a `timestamp` field is detected in a request body schema (common in OAuth, AWS Sig, Ably tokens), add a hint in the guide: "consider GET /time before this step to capture the server timestamp".
+
+### Per-suite env resolution
+When running `apitool run apis/` (directory with subdirs), resolve `.env.<name>.yaml` from each suite's directory, not from `dirname(path)`.
+
+---
+
+## Low Priority
+
+### Summary after batch generation
+After generating many suites, return a summary (created N files, total coverage %).
+
+### Env file location in MCP output
+`setup_api` should return the path to `.env.default.yaml` and a brief instruction for adding the API key in its response, so the agent knows exactly where to write credentials immediately after registration.
+
+### Comment preservation
+Parser preserves YAML comments when reading/writing (currently lost).
+
+### `apitool docs` command
+Generate markdown documentation from YAML tests: descriptions + examples.
+
+### Multipart/form-data support
+Runner support for file upload endpoints.
+
+---
+
+## Not Doing
+
+| Item | Reason |
+|------|--------|
+| GraphQL / gRPC / WebSocket | REST + OpenAPI = 80% of market |
+| Load testing | Use k6 instead |
+| WebUI polish (themes, animations) | Not a selling point |
+| Plugins / marketplace | Requires large team |
+| Team features / RBAC | Different product category |
+| Docker image | Single binary is simpler |

package/CHANGELOG.md

@@ -0,0 +1,88 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+## [Unreleased]
+
+### Added
+
+- **MCP feedback improvements**
+  - `diagnose_failure` now includes `response_headers` in failure output (e.g. `X-Ably-ErrorMessage`)
+  - `generate_tests_guide`: annotates `any`-typed request bodies with a warning comment
+  - `generate_tests_guide`: added 204 No Content tips in Practical Tips and Common Mistakes sections
+  - `schema-utils.ts`: added `isAnySchema()` helper
+  - DB schema v6: `results.response_headers TEXT` column
+
+- **M22: MCP-first smart test generation**
+  - `generate_tests_guide` MCP tool — returns full API spec with schemas + step-by-step generation algorithm
+  - `save_test_suite` MCP tool — validates YAML and saves test files with structured error reporting
+  - `explore_api` enhanced — new `includeSchemas` parameter for full request/response body schemas
+  - `schema-utils.ts` — extracted `compressSchema()` and `formatParam()` as shared utilities
+  - Improved MCP tool descriptions with "when to use" guidance
+
+### Removed
+
+- `list_environments` MCP tool — duplicated by `manage_environment(action: "list")`
+
+---
+
+## [0.3.0] - Unreleased (post-M21)
+
+### Added
+
+- **Environment management in WebUI** — full CRUD for environments (`/environments`)
+- **Key-value editor** — add/remove variables with inline JavaScript
+- **Environment selector** — `<select name="env">` dropdown in collection "Run Tests" form
+- **DB queries** — `getEnvironmentById()`, `deleteEnvironment()`, `listEnvironmentRecords()`
+- **Navigation** — "Environments" link in navbar
+- **Improved runs filter** — environment dropdown merges defined environments + run history
+- **Self-documented API** — routes use `@hono/zod-openapi`, `GET /api/openapi.json` serves spec
+- **Incremental generation** — `apitool generate` skips already-covered endpoints
+- **Dogfooding** — integration tests run against apitool's own API
+- **Generator: `additionalProperties`** — Record types generate sample key-value pairs instead of `{}`
+- **CI: typecheck** — `tsc --noEmit` step added to CI pipeline
+
+### Changed
+
+- **Auth-flow test** — rewritten with inline OpenAPI server (no external `test-server/` dependency)
+
+### Removed
+
+- **`test-server/`** — replaced by inline test servers in integration tests
+- **Duplicate spec files** — `openapi-self.json`, `self-tests-spec.json` removed from project root
+
+### Fixed
+
+- **Type errors** — `z.coerce.number()` in schemas, `c.body()` return type in export route
+- **Environments CRUD skeleton** — `variables` field now generates test data correctly
+
+## [0.1.0] - 2025-02-27
+
+Initial public release.
+
+### Features
+
+- **YAML test definitions** — declarative API tests with steps, assertions, variables, and captures
+- **Test runner** — sequential HTTP execution with variable substitution, chained captures, and configurable timeouts
+- **Assertions** — status code, JSON body (exact, contains, path), headers, response time
+- **Environment files** — `.env.<name>.yaml` for per-environment variables (base URLs, tokens, etc.)
+- **OpenAPI test generator** — generate skeleton YAML tests from OpenAPI 3.x specs (CRUD operations, auth-aware)
+- **AI-powered test generation** — generate tests using LLM providers (Ollama, OpenAI, Anthropic, custom)
+- **Reporters** — console (colored), JSON, JUnit XML output formats
+- **SQLite storage** — persist test runs, results, and collections in `apitool.db`
+- **WebUI dashboard** — Hono + HTMX web interface with:
+  - Run history with filters and trend charts
+  - Suite detail view with per-step results
+  - API Explorer with request builder and authorization panel
+  - Collection management with drill-down
+  - AI test generation UI
+  - Result export (JSON, JUnit)
+- **CLI commands**:
+  - `apitool run <path>` — execute tests with env, reporter, timeout, bail options
+  - `apitool validate <path>` — validate YAML test files
+  - `apitool generate --from <spec>` — generate tests from OpenAPI
+  - `apitool ai-generate --from <spec> --prompt "..."` — AI test generation
+  - `apitool serve` — start web dashboard
+  - `apitool collections` — list test collections
+- **Multi-auth support** — Basic, Bearer, API Key auth in CLI (`--auth-token`) and WebUI
+- **Standalone binary** — single-file executable via `bun build --compile`

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 apitool contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,105 @@
+# @kirrosh/apitool
+
+AI-native API testing tool. OpenAPI spec in, tests out. One binary, zero config.
+
+[![Install in Cursor](https://cursor.com/deeplink/mcp-install-dark.svg)](https://cursor.com/en/install-mcp?name=apitool&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBraXJyb3NoL2FwaXRvb2wiLCJtY3AiXX0=)
+
+## Install
+
+```bash
+# Option 1: via npx (recommended — works everywhere with Node.js)
+npx -y @kirrosh/apitool --version
+
+# Option 2: Binary (no Node.js required)
+# macOS / Linux
+curl -fsSL https://raw.githubusercontent.com/kirrosh/apitool/master/install.sh | sh
+
+# Windows
+iwr https://raw.githubusercontent.com/kirrosh/apitool/master/install.ps1 | iex
+```
+
+[All releases](https://github.com/kirrosh/apitool/releases) (Linux x64, macOS ARM, Windows x64)
+
+## Quick Start
+
+```bash
+# Register API + generate + run — all in one flow via MCP or CLI
+apitool add-api petstore --spec https://petstore.swagger.io/v2/swagger.json
+apitool run apis/petstore/tests/ --env default
+apitool serve   # web dashboard at http://localhost:8080
+```
+
+Or let your AI agent do it — just say: *"Test the API from openapi.json"*
+
+## MCP Setup (Cursor / Claude Code / Windsurf)
+
+Click the badge above, or add manually:
+
+```json
+{
+  "mcpServers": {
+    "apitool": {
+      "command": "npx",
+      "args": ["-y", "@kirrosh/apitool", "mcp", "--dir", "${workspaceFolder}"]
+    }
+  }
+}
+```
+
+**Where to put this:**
+
+| Editor | Config file |
+|--------|-------------|
+| Cursor | Settings > MCP, or `.cursor/mcp.json` in project root |
+| Claude Code | `.mcp.json` in project root |
+| Windsurf | `.windsurfrules/mcp.json` or settings |
+
+14 MCP tools: `setup_api`, `generate_tests_guide`, `save_test_suite`, `run_tests`, `query_db`, `explore_api`, `coverage_analysis`, `generate_missing_tests`, `validate_tests`, `send_request`, `manage_environment`, `manage_server`. Full reference in [APITOOL.md](APITOOL.md).
+
+## YAML Test Format
+
+```yaml
+name: Users CRUD
+description: Full user lifecycle
+tags: [users, crud, smoke]
+base_url: "{{base_url}}"
+
+tests:
+  - name: Create user
+    POST: /users
+    json:
+      name: "{{$randomName}}"
+      email: "{{$randomEmail}}"
+    expect:
+      status: 201
+      body:
+        id: { capture: user_id, type: integer }
+
+  - name: Get user
+    GET: /users/{{user_id}}
+    expect:
+      status: 200
+
+  - name: Delete user
+    DELETE: /users/{{user_id}}
+    expect:
+      status: 204
+```
+
+## CLI
+
+```
+apitool run <path>           Run tests (--env, --safe, --report json|junit)
+apitool add-api <name>       Register API (--spec <openapi>)
+apitool serve                Web dashboard (--port 8080)
+apitool mcp                  Start MCP server
+apitool coverage             API test coverage (--spec, --tests)
+apitool chat                 AI chat agent (--provider ollama|openai|anthropic)
+apitool doctor               Diagnostics
+```
+
+Full docs: [APITOOL.md](APITOOL.md)
+
+## License
+
+[MIT](LICENSE)