reverse-api-engineer 0.2.8__tar.gz → 0.2.10__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/.claude/settings.local.json +20 -1
- reverse_api_engineer-0.2.10/.claude-plugin/marketplace.json +14 -0
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/.gitignore +3 -1
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/CHANGELOG.md +51 -0
- reverse_api_engineer-0.2.10/CLAUDE.md +220 -0
- reverse_api_engineer-0.2.10/INTERVIEW.md +5 -0
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/PKG-INFO +39 -19
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/README.md +37 -18
- reverse_api_engineer-0.2.10/examples/apple/INDEX.md +247 -0
- reverse_api_engineer-0.2.10/examples/apple/QUICKSTART.md +245 -0
- reverse_api_engineer-0.2.10/examples/apple/README.md +409 -0
- reverse_api_engineer-0.2.10/examples/apple/SUMMARY.md +334 -0
- reverse_api_engineer-0.2.10/examples/apple/api_client.py +416 -0
- reverse_api_engineer-0.2.10/examples/apple/extract_job_fields.py +188 -0
- reverse_api_engineer-0.2.10/examples/apple/main.py +36 -0
- reverse_api_engineer-0.2.10/examples/apple/quick_example.py +108 -0
- reverse_api_engineer-0.2.10/examples/apple/requirements.txt +1 -0
- reverse_api_engineer-0.2.10/examples/ashby/API_SUMMARY.txt +143 -0
- reverse_api_engineer-0.2.10/examples/ashby/QUICKSTART.md +80 -0
- reverse_api_engineer-0.2.10/examples/ashby/README.md +702 -0
- reverse_api_engineer-0.2.10/examples/ashby/api_client.py +880 -0
- reverse_api_engineer-0.2.10/examples/ashby/example_usage.py +186 -0
- reverse_api_engineer-0.2.10/examples/ashby/requirements.txt +2 -0
- reverse_api_engineer-0.2.10/examples/ikea/README.md +350 -0
- reverse_api_engineer-0.2.10/examples/ikea/api_client.py +564 -0
- reverse_api_engineer-0.2.10/examples/uber/API_ANALYSIS_SUMMARY.md +255 -0
- reverse_api_engineer-0.2.10/examples/uber/README.md +321 -0
- reverse_api_engineer-0.2.10/examples/uber/api_client.py +480 -0
- reverse_api_engineer-0.2.10/examples/uber/example_fetch_all_jobs.py +192 -0
- reverse_api_engineer-0.2.10/examples/uber/quick_start.py +82 -0
- reverse_api_engineer-0.2.10/examples/uber/requirements.txt +1 -0
- reverse_api_engineer-0.2.10/llm-docs/building-skill/OVERVIEW.md +130 -0
- reverse_api_engineer-0.2.10/llm-docs/building-skill/SPECS.md +262 -0
- reverse_api_engineer-0.2.10/llm-docs/building-skill/WHAT-ARE-SKILLS.md +82 -0
- reverse_api_engineer-0.2.10/llm-docs/claude-agent-sdk/QUICKSTART.md +2017 -0
- reverse_api_engineer-0.2.10/llm-docs/claude-agent-sdk/TODO_LIST.md +176 -0
- reverse_api_engineer-0.2.10/llm-docs/claude-agent-sdk/TOOLS.md +0 -0
- reverse_api_engineer-0.2.10/llm-docs/crawler/DOCS.md +6 -0
- reverse_api_engineer-0.2.10/plugins/reverse-api-engineer/.claude-plugin/plugin.json +38 -0
- reverse_api_engineer-0.2.10/plugins/reverse-api-engineer/CHANGELOG.md +23 -0
- reverse_api_engineer-0.2.10/plugins/reverse-api-engineer/LICENSE +21 -0
- reverse_api_engineer-0.2.10/plugins/reverse-api-engineer/README.md +211 -0
- reverse_api_engineer-0.2.10/plugins/reverse-api-engineer/agents/api-reverse-engineer.md +338 -0
- reverse_api_engineer-0.2.10/plugins/reverse-api-engineer/commands/agent.md +443 -0
- reverse_api_engineer-0.2.10/plugins/reverse-api-engineer/commands/engineer.md +398 -0
- reverse_api_engineer-0.2.10/plugins/reverse-api-engineer/commands/manual.md +324 -0
- reverse_api_engineer-0.2.10/plugins/reverse-api-engineer/skills/reverse-engineering-api/CHANGELOG.md +33 -0
- reverse_api_engineer-0.2.10/plugins/reverse-api-engineer/skills/reverse-engineering-api/LICENSE +21 -0
- reverse_api_engineer-0.2.10/plugins/reverse-api-engineer/skills/reverse-engineering-api/SKILL.md +301 -0
- reverse_api_engineer-0.2.10/plugins/reverse-api-engineer/skills/reverse-engineering-api/references/AUTH_PATTERNS.md +575 -0
- reverse_api_engineer-0.2.10/plugins/reverse-api-engineer/skills/reverse-engineering-api/references/HAR_ANALYSIS.md +381 -0
- reverse_api_engineer-0.2.10/plugins/reverse-api-engineer/skills/reverse-engineering-api/templates/api_client.py +464 -0
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/pyproject.toml +47 -2
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/src/reverse_api/__init__.py +1 -1
- reverse_api_engineer-0.2.10/src/reverse_api/auto_engineer.py +554 -0
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/src/reverse_api/base_engineer.py +90 -12
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/src/reverse_api/browser.py +61 -65
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/src/reverse_api/cli.py +624 -180
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/src/reverse_api/config.py +10 -20
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/src/reverse_api/engineer.py +51 -37
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/src/reverse_api/messages.py +7 -8
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/src/reverse_api/opencode_engineer.py +81 -63
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/src/reverse_api/opencode_ui.py +56 -46
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/src/reverse_api/pricing.py +21 -9
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/src/reverse_api/session.py +6 -5
- reverse_api_engineer-0.2.10/src/reverse_api/sync.py +282 -0
- reverse_api_engineer-0.2.10/src/reverse_api/tools/crawler.py +417 -0
- reverse_api_engineer-0.2.10/src/reverse_api/tools/firecrawl_out_join_com.json +4908 -0
- reverse_api_engineer-0.2.10/src/reverse_api/tools/mapper.py +407 -0
- reverse_api_engineer-0.2.10/src/reverse_api/tools/mapping_.json +160 -0
- reverse_api_engineer-0.2.10/src/reverse_api/tools/sitemap.py +301 -0
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/src/reverse_api/tui.py +41 -22
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/src/reverse_api/utils.py +55 -9
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/uv.lock +191 -3
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/.python-version +0 -0
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/CONTRIBUTING.md +0 -0
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/LICENSE +0 -0
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/RELEASING.md +0 -0
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/assets/reverse-api-banner.svg +0 -0
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/assets/reverse-api-engineer.gif +0 -0
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/assets/reverse-api-logo.svg +0 -0
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/llm-docs/OPENCODE_API_SUMMARY.md +0 -0
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/llm-docs/opencode-api.json +0 -0
- {reverse_api_engineer-0.2.8 → reverse_api_engineer-0.2.10}/scripts/clean_build.sh +0 -0
|
@@ -51,7 +51,26 @@
|
|
|
51
51
|
"WebFetch(domain:localhost)",
|
|
52
52
|
"Bash(curl:*)",
|
|
53
53
|
"WebFetch(domain:docs.litellm.ai)",
|
|
54
|
-
"Bash(git reset:*)"
|
|
54
|
+
"Bash(git reset:*)",
|
|
55
|
+
"Bash(ls:*)",
|
|
56
|
+
"Bash(if [ -n \"$PYPI_TOKEN\" ])",
|
|
57
|
+
"Bash(then)",
|
|
58
|
+
"Bash(elif [ -n \"$UV_PUBLISH_TOKEN\" ])",
|
|
59
|
+
"Bash(else)",
|
|
60
|
+
"Bash(fi)",
|
|
61
|
+
"Bash(python -m pytest:*)",
|
|
62
|
+
"WebFetch(domain:opencode.ai)",
|
|
63
|
+
"Skill(plugin-dev:plugin-structure)",
|
|
64
|
+
"Bash(tree:*)",
|
|
65
|
+
"Bash(mkdir:*)",
|
|
66
|
+
"Skill(plugin-dev:command-development)",
|
|
67
|
+
"mcp__plugin_reverse-api-engineer_rae-playwright-mcp__browser_navigate",
|
|
68
|
+
"mcp__plugin_reverse-api-engineer_rae-playwright-mcp__browser_close",
|
|
69
|
+
"mcp__plugin_reverse-api-engineer_rae-playwright-mcp__browser_click",
|
|
70
|
+
"mcp__plugin_reverse-api-engineer_rae-playwright-mcp__browser_snapshot",
|
|
71
|
+
"mcp__plugin_reverse-api-engineer_rae-playwright-mcp__browser_run_code",
|
|
72
|
+
"mcp__plugin_reverse-api-engineer_rae-playwright-mcp__browser_network_requests",
|
|
73
|
+
"Skill(reverse-api-engineer:reverse-engineering-api)"
|
|
55
74
|
]
|
|
56
75
|
}
|
|
57
76
|
}
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "reverse-api-engineer",
|
|
3
|
+
"owner": {
|
|
4
|
+
"name": "Kalil Sama Bouzigues",
|
|
5
|
+
"email": "kalil.bouzigues@gmail.com"
|
|
6
|
+
},
|
|
7
|
+
"plugins": [
|
|
8
|
+
{
|
|
9
|
+
"name": "reverse-api-engineer",
|
|
10
|
+
"source": "./plugins/reverse-api-engineer",
|
|
11
|
+
"description": "A plugin to reverse engineer APIs from websites"
|
|
12
|
+
}
|
|
13
|
+
]
|
|
14
|
+
}
|
|
@@ -5,6 +5,57 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [0.2.10] - 2026-01-03
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
- **Claude Code Plugin**: Official plugin for seamless integration with Claude Code CLI
|
|
12
|
+
- Three operation modes: manual browser capture, autonomous agent browsing, and re-engineering from HAR files
|
|
13
|
+
- Comprehensive skill system with progressive disclosure of reverse engineering techniques
|
|
14
|
+
- Slash commands for quick access: `/agent`, `/engineer`, `/manual`
|
|
15
|
+
- Reference documentation for HAR analysis and authentication patterns
|
|
16
|
+
- API client templates for common patterns
|
|
17
|
+
- **Example API Clients**: Added production-ready examples for major platforms
|
|
18
|
+
- Apple Jobs API client with field extraction utilities
|
|
19
|
+
- Ashby Jobs API client with comprehensive endpoint coverage
|
|
20
|
+
- Ikea API client for product search and catalog browsing
|
|
21
|
+
- Uber Careers API client with pagination support
|
|
22
|
+
- **Engineer Tagging System**: Enhanced metadata tracking for generated API clients
|
|
23
|
+
- Automatic tagging of runs with descriptive identifiers
|
|
24
|
+
- Improved organization and searchability of reverse-engineered APIs
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
- **Enhanced Auto Mode**: Improved MCP browser integration with better error handling
|
|
28
|
+
- **Better Sync Fallback**: More robust file synchronization with fallback mechanisms
|
|
29
|
+
- **Code Quality**: Comprehensive formatting and linting improvements across codebase
|
|
30
|
+
|
|
31
|
+
### Fixed
|
|
32
|
+
- **Agent Mode Screenshots**: Reduced unnecessary screenshot captures in agent mode
|
|
33
|
+
- **Sync Error Handling**: Fixed sync fallback when primary sync method fails
|
|
34
|
+
- **Path Handling**: Corrected CLAUDE.md documentation paths
|
|
35
|
+
- **Import Errors**: Fixed missing imports in various modules
|
|
36
|
+
|
|
37
|
+
## [0.2.9] - 2025-12-30
|
|
38
|
+
|
|
39
|
+
### Added
|
|
40
|
+
- **Real-time File Sync**: Watch and automatically sync generated scripts to local directory
|
|
41
|
+
- Debounced file watching with configurable delay (default 500ms)
|
|
42
|
+
- Visual feedback for sync operations in terminal UI
|
|
43
|
+
- Prevents overwriting existing directories by appending counter suffix
|
|
44
|
+
- **MCP Browser Integration**: Native integration with `rae-playwright-mcp` for auto mode
|
|
45
|
+
- Seamless browser automation via Model Context Protocol
|
|
46
|
+
- Works with both Claude SDK and OpenCode SDK
|
|
47
|
+
- Combines browser control and real-time reverse engineering in single workflow
|
|
48
|
+
- **CLAUDE.md Autogeneration**: Automatic generation of project documentation for Claude Code
|
|
49
|
+
|
|
50
|
+
### Changed
|
|
51
|
+
- **Enhanced Settings Management**: Improved settings configuration and UI
|
|
52
|
+
- **Better Sync Error Handling**: Improved error handling and resource cleanup for sync operations
|
|
53
|
+
|
|
54
|
+
### Fixed
|
|
55
|
+
- **Sync Directory Overwrite**: Fixed issue where sync would overwrite existing directories
|
|
56
|
+
- **Sync Resource Leaks**: Fixed memory leaks when sync errors occurred
|
|
57
|
+
- **UI Improvements**: Various UI fixes and enhancements
|
|
58
|
+
|
|
8
59
|
## [0.2.8] - 2025-12-28
|
|
9
60
|
|
|
10
61
|
### Added
|
|
@@ -0,0 +1,220 @@
|
|
|
1
|
+
# CLAUDE.md
|
|
2
|
+
|
|
3
|
+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
4
|
+
|
|
5
|
+
## Project Overview
|
|
6
|
+
|
|
7
|
+
Reverse API Engineer is a CLI tool that captures browser traffic (HAR files) and uses AI to automatically generate production-ready Python API clients. It supports three modes: manual browser capture, autonomous AI agent browsing, and re-engineering from previous captures.
|
|
8
|
+
|
|
9
|
+
## Core Architecture
|
|
10
|
+
|
|
11
|
+
### SDK Dispatch Pattern
|
|
12
|
+
The project uses a **dual-SDK architecture** controlled by `config.json`:
|
|
13
|
+
|
|
14
|
+
- **Claude SDK** (`engineer.py` → `ClaudeEngineer`): Uses `claude-agent-sdk` for direct Claude API integration
|
|
15
|
+
- **OpenCode SDK** (`opencode_engineer.py` → `OpenCodeEngineer`): Uses httpx to communicate with local OpenCode server (http://127.0.0.1:4096)
|
|
16
|
+
|
|
17
|
+
Both inherit from `BaseEngineer` (abstract base class) which defines the common interface and shared prompt building logic. The SDK selection is determined by the `sdk` field in config.
|
|
18
|
+
|
|
19
|
+
### Key Components
|
|
20
|
+
|
|
21
|
+
- **CLI Layer** (`cli.py`): Main entry point with mode cycling (manual/engineer/agent), slash commands, and interactive prompts
|
|
22
|
+
- **Browser** (`browser.py`): Playwright-based HAR recording with stealth mode and anti-detection measures
|
|
23
|
+
- `ManualBrowser`: User-controlled browser with HAR capture
|
|
24
|
+
- `run_agent_browser()`: Autonomous agent browsing (browser-use or stagehand)
|
|
25
|
+
- **Configuration** (`config.py`): JSON-based config manager at `~/.reverse-api/config.json`
|
|
26
|
+
- **Session Management** (`session.py`): Tracks run history with costs and metadata
|
|
27
|
+
- **Message Store** (`messages.py`): Persists full conversation logs per run
|
|
28
|
+
- **UI** (`tui.py`, `opencode_ui.py`): Rich-based terminal UI for progress tracking
|
|
29
|
+
|
|
30
|
+
### Data Flow
|
|
31
|
+
1. Browser captures HAR → saved to `~/.reverse-api/runs/har/{run_id}`
|
|
32
|
+
1. Browser captures HAR → saved to `~/.reverse-api/runs/har/{run_id}`
|
|
33
|
+
2. Engineer analyzes HAR with LLM → generates Python scripts
|
|
34
|
+
3. Scripts saved to:
|
|
35
|
+
- `~/.reverse-api/runs/scripts/{run_id}` (permanent)
|
|
36
|
+
- `~/.reverse-api/runs/scripts/{run_id}` (permanent)
|
|
37
|
+
- `./scripts/{descriptive_name}/` (local copy)
|
|
38
|
+
|
|
39
|
+
## Development Commands
|
|
40
|
+
|
|
41
|
+
### Setup
|
|
42
|
+
```bash
|
|
43
|
+
# Clone and install dependencies
|
|
44
|
+
git clone https://github.com/kalil0321/reverse-api-engineer.git
|
|
45
|
+
cd reverse-api-engineer
|
|
46
|
+
uv sync
|
|
47
|
+
playwright install chromium
|
|
48
|
+
|
|
49
|
+
# Run locally (development mode)
|
|
50
|
+
uv run reverse-api-engineer
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
### Building
|
|
54
|
+
```bash
|
|
55
|
+
# Clean build (removes all caches and artifacts)
|
|
56
|
+
./scripts/clean_build.sh
|
|
57
|
+
|
|
58
|
+
# Manual build
|
|
59
|
+
uv build
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
### Testing
|
|
63
|
+
```bash
|
|
64
|
+
# Test as installed package (creates isolated venv)
|
|
65
|
+
python -m venv test_env
|
|
66
|
+
test_env/bin/pip install dist/*.whl
|
|
67
|
+
test_env/bin/reverse-api-engineer
|
|
68
|
+
|
|
69
|
+
# Test with uv tool
|
|
70
|
+
uv tool install .
|
|
71
|
+
reverse-api-engineer
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
### Linting & Static Analysis
|
|
75
|
+
```bash
|
|
76
|
+
# Run all checks
|
|
77
|
+
./scripts/lint.sh
|
|
78
|
+
|
|
79
|
+
# Run specific tools
|
|
80
|
+
uv run ruff check src # Linter
|
|
81
|
+
uv run ruff format src # Formatter
|
|
82
|
+
uv run mypy src # Static analysis
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
## Release Process
|
|
86
|
+
|
|
87
|
+
**Single source of truth: `pyproject.toml`**
|
|
88
|
+
|
|
89
|
+
1. Update version in `pyproject.toml` only
|
|
90
|
+
2. Update `CHANGELOG.md`
|
|
91
|
+
3. Run `./scripts/clean_build.sh`
|
|
92
|
+
4. Commit and tag:
|
|
93
|
+
```bash
|
|
94
|
+
git add pyproject.toml CHANGELOG.md
|
|
95
|
+
git commit -m "release: vX.Y.Z - description"
|
|
96
|
+
git tag vX.Y.Z
|
|
97
|
+
git push origin main
|
|
98
|
+
git push origin vX.Y.Z
|
|
99
|
+
```
|
|
100
|
+
5. Publish to PyPI:
|
|
101
|
+
```bash
|
|
102
|
+
source .env # contains UV_PUBLISH_TOKEN or PYPI_TOKEN
|
|
103
|
+
uv publish
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
**Important:** Never edit `src/reverse_api/__init__.py` for versioning—it reads from package metadata via `importlib.metadata`.
|
|
107
|
+
|
|
108
|
+
## Configuration System
|
|
109
|
+
|
|
110
|
+
### Config Location
|
|
111
|
+
`~/.reverse-api/config.json` with these fields:
|
|
112
|
+
|
|
113
|
+
```json
|
|
114
|
+
{
|
|
115
|
+
"claude_code_model": "claude-sonnet-4-5", // For Claude SDK
|
|
116
|
+
"opencode_provider": "anthropic", // For OpenCode SDK
|
|
117
|
+
"opencode_model": "claude-sonnet-4-5", // For OpenCode SDK
|
|
118
|
+
"sdk": "claude", // "claude" or "opencode"
|
|
119
|
+
"agent_provider": "browser-use", // "browser-use" or "stagehand"
|
|
120
|
+
"browser_use_model": "bu-llm", // Browser-use agent model
|
|
121
|
+
"stagehand_model": "openai/computer-use-preview-2025-03-11",
|
|
122
|
+
"output_dir": null // Custom output dir (null = use ~/.reverse-api/runs)
|
|
123
|
+
}
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
### Model Naming
|
|
127
|
+
- **Claude models**: `claude-sonnet-4-5`, `claude-opus-4-5`, `claude-haiku-4-5`
|
|
128
|
+
- **Gemini models**: `gemini-3-flash`, `gemini-3-pro`, `gemini-3-pro-low`, `gemini-3-pro-high`
|
|
129
|
+
- **Antigravity models** (free tier): Require OpenCode SDK with `opencode_provider: "google"`
|
|
130
|
+
- **Agent models**: Format varies by provider
|
|
131
|
+
- Browser-use: `bu-llm` or `{provider}/{model}` (e.g., `openai/gpt-4`)
|
|
132
|
+
- Stagehand: `{provider}/{model}` (e.g., `openai/computer-use-preview-2025-03-11`)
|
|
133
|
+
|
|
134
|
+
## Pricing System
|
|
135
|
+
|
|
136
|
+
### Three-tier fallback for cost calculation:
|
|
137
|
+
1. **Local pricing** (`pricing.py`): Built-in prices for Claude/Gemini models
|
|
138
|
+
2. **LiteLLM pricing** (optional `[pricing]` extra): Extended coverage for 100+ models
|
|
139
|
+
3. **Default pricing**: Falls back to Claude Sonnet 4.5 pricing
|
|
140
|
+
|
|
141
|
+
Pricing tracks:
|
|
142
|
+
- Input/output tokens
|
|
143
|
+
- Cache creation/read tokens
|
|
144
|
+
- Reasoning tokens (for thinking models)
|
|
145
|
+
|
|
146
|
+
## Agent Mode
|
|
147
|
+
|
|
148
|
+
### Requirements
|
|
149
|
+
Agent mode requires browser-use from specific git commit (has HAR recording support):
|
|
150
|
+
```bash
|
|
151
|
+
uv tool install 'reverse-api-engineer[agent]' --with 'browser-use @ git+https://github.com/browser-use/browser-use.git@49a345fb19e9f12befc5cc1658e0033873892455'
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
### Agent Providers
|
|
155
|
+
- **browser-use**: Multi-model support (Browser-Use LLM, OpenAI, Google)
|
|
156
|
+
- **stagehand**: Computer Use models only (OpenAI, Anthropic)
|
|
157
|
+
|
|
158
|
+
API keys via environment variables: `BROWSER_USE_API_KEY`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GOOGLE_API_KEY`
|
|
159
|
+
|
|
160
|
+
## Code Style Conventions
|
|
161
|
+
|
|
162
|
+
- **Type hints**: Use throughout (`from typing import Optional, Dict, Any`)
|
|
163
|
+
- **Async/await**: Engineers use async for LLM streaming
|
|
164
|
+
- **Error handling**: Wrap external calls in try-except with user-friendly messages
|
|
165
|
+
- **UI consistency**: Use Rich for all terminal output (no plain print statements)
|
|
166
|
+
- **Config migration**: `config.py` includes backward compatibility logic for renamed keys
|
|
167
|
+
|
|
168
|
+
## Common Tasks
|
|
169
|
+
|
|
170
|
+
### Adding a new model
|
|
171
|
+
1. Add pricing to `pricing.py` MODEL_PRICING dict (if not in LiteLLM)
|
|
172
|
+
2. Update model choices in `tui.py` get_model_choices() if needed
|
|
173
|
+
3. Update README.md model list
|
|
174
|
+
|
|
175
|
+
### Adding a new agent provider
|
|
176
|
+
1. Add provider logic to `browser.py` run_agent_browser()
|
|
177
|
+
2. Add config fields to `config.py` DEFAULT_CONFIG
|
|
178
|
+
3. Update settings menu in `cli.py`
|
|
179
|
+
4. Document in README.md
|
|
180
|
+
|
|
181
|
+
### Debugging
|
|
182
|
+
- **OpenCode**: Set `OPENCODE_DEBUG=1` for detailed logs
|
|
183
|
+
- **Stagehand logs**: Suppressed by `_suppress_stagehand_logs()` in `browser.py`
|
|
184
|
+
- **Message logs**: All LLM interactions saved to `~/.reverse-api/runs/messages/{run_id}.jsonl`
|
|
185
|
+
|
|
186
|
+
## File Structure Notes
|
|
187
|
+
|
|
188
|
+
```
|
|
189
|
+
src/reverse_api/
|
|
190
|
+
├── cli.py # Main entry point, mode cycling, slash commands
|
|
191
|
+
├── base_engineer.py # Abstract base for SDK implementations
|
|
192
|
+
├── engineer.py # Claude SDK implementation
|
|
193
|
+
├── opencode_engineer.py # OpenCode SDK implementation
|
|
194
|
+
├── browser.py # Playwright HAR capture + agent mode
|
|
195
|
+
├── config.py # Configuration manager
|
|
196
|
+
├── session.py # Run history and metadata
|
|
197
|
+
├── messages.py # Message persistence
|
|
198
|
+
├── pricing.py # Model pricing database
|
|
199
|
+
├── tui.py # Terminal UI (Rich-based)
|
|
200
|
+
├── opencode_ui.py # OpenCode-specific UI extensions
|
|
201
|
+
└── utils.py # Path helpers, run ID generation
|
|
202
|
+
|
|
203
|
+
~/.reverse-api/
|
|
204
|
+
├── config.json # User configuration
|
|
205
|
+
├── history.json # Run history with costs
|
|
206
|
+
└── runs/ # Organized by data type
|
|
207
|
+
├── har/{run_id}/ # Captured traffic per run
|
|
208
|
+
├── scripts/{run_id}/ # Generated API clients per run
|
|
209
|
+
└── messages/ # LLM conversation logs
|
|
210
|
+
└── {run_id}.jsonl
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
## Dependencies Management
|
|
214
|
+
|
|
215
|
+
Uses `uv` for dependency management:
|
|
216
|
+
- **Core deps**: playwright, claude-agent-sdk, rich, questionary, anthropic
|
|
217
|
+
- **Optional [agent]**: stagehand (browser-use must be installed separately from git)
|
|
218
|
+
- **Optional [pricing]**: litellm (for extended model pricing)
|
|
219
|
+
|
|
220
|
+
Always use `uv sync` after pulling changes.
|
|
@@ -0,0 +1,5 @@
|
|
|
1
|
+
Read this @SPEC.md and interview me in detail using the AskUserQuestionTool about
|
|
2
|
+
literally anything: technical implementation, UI & UX, concerns, tradeoffs, etc.
|
|
3
|
+
but make sure the questions are not obvious.
|
|
4
|
+
|
|
5
|
+
Be very in-depth and continue interviewing me continually until it’s complete, then write the spec to the file.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: reverse-api-engineer
|
|
3
|
-
Version: 0.2.
|
|
3
|
+
Version: 0.2.10
|
|
4
4
|
Summary: A tool to capture browser traffic for API reverse engineering
|
|
5
5
|
Project-URL: Homepage, https://github.com/kalil0321/reverse-api-engineer
|
|
6
6
|
Project-URL: Repository, https://github.com/kalil0321/reverse-api-engineer
|
|
@@ -29,6 +29,7 @@ Requires-Dist: playwright>=1.40.0
|
|
|
29
29
|
Requires-Dist: questionary>=2.0.0
|
|
30
30
|
Requires-Dist: requests>=2.32.5
|
|
31
31
|
Requires-Dist: rich>=13.0.0
|
|
32
|
+
Requires-Dist: watchdog>=3.0.0
|
|
32
33
|
Provides-Extra: agent
|
|
33
34
|
Requires-Dist: stagehand; extra == 'agent'
|
|
34
35
|
Provides-Extra: pricing
|
|
@@ -36,18 +37,16 @@ Requires-Dist: litellm>=1.0.0; extra == 'pricing'
|
|
|
36
37
|
Description-Content-Type: text/markdown
|
|
37
38
|
|
|
38
39
|
<div align="center">
|
|
39
|
-
<img src="assets/reverse-api-banner.svg" alt="Reverse API Engineer Banner">
|
|
40
|
+
<img src="https://raw.githubusercontent.com/kalil0321/reverse-api-engineer/main/assets/reverse-api-banner.svg" alt="Reverse API Engineer Banner">
|
|
40
41
|
</div>
|
|
41
42
|
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
[](https://badge.fury.io/py/reverse-api-engineer)
|
|
43
|
+

|
|
45
44
|
[](https://www.python.org/downloads/)
|
|
46
45
|
[](https://opensource.org/licenses/MIT)
|
|
47
46
|
|
|
48
47
|
CLI tool that captures browser traffic and automatically generates production-ready Python API clients. No more manual reverse engineering—just browse, capture, and get clean API code.
|
|
49
48
|
|
|
50
|
-

|
|
49
|
+

|
|
51
50
|
|
|
52
51
|
## Table of Contents
|
|
53
52
|
|
|
@@ -60,10 +59,11 @@ CLI tool that captures browser traffic and automatically generates production-re
|
|
|
60
59
|
- [Agent Mode](#agent-mode)
|
|
61
60
|
- [Configuration](#-configuration)
|
|
62
61
|
- [Model Selection](#model-selection)
|
|
63
|
-
- [💸 Free Models with Antigravity](
|
|
62
|
+
- [💸 Free Models with Antigravity](#free-model-options-with-antigravity)
|
|
64
63
|
- [Agent Configuration](#agent-configuration)
|
|
65
64
|
- [SDK Selection](#sdk-selection)
|
|
66
65
|
- [CLI Commands](#-cli-commands)
|
|
66
|
+
- [Claude Code Plugin](#-claude-code-plugin)
|
|
67
67
|
- [Examples](#-examples)
|
|
68
68
|
- [Development](#-development)
|
|
69
69
|
- [Contributing](#-contributing)
|
|
@@ -71,7 +71,7 @@ CLI tool that captures browser traffic and automatically generates production-re
|
|
|
71
71
|
## ✨ Features
|
|
72
72
|
|
|
73
73
|
- 🌐 **Browser Automation**: Built on Playwright with stealth mode for realistic browsing
|
|
74
|
-
- 🤖 **Autonomous Agent Mode**: Fully automated browser interaction using AI agents (browser-use, stagehand)
|
|
74
|
+
- 🤖 **Autonomous Agent Mode**: Fully automated browser interaction using AI agents (auto mode with MCP, browser-use, stagehand)
|
|
75
75
|
- 📊 **HAR Recording**: Captures all network traffic in HTTP Archive format
|
|
76
76
|
- 🧠 **AI-Powered Generation**: Uses Claude 4.5 to analyze traffic and generate clean Python code
|
|
77
77
|
- 🔌 **Multi-SDK Support**: Native integration with Claude and OpenCode SDKs
|
|
@@ -141,7 +141,7 @@ reverse-api-engineer
|
|
|
141
141
|
The CLI has three modes (cycle with **Shift+Tab**):
|
|
142
142
|
- **manual**: Browser capture + AI generation
|
|
143
143
|
- **engineer**: Re-process existing captures
|
|
144
|
-
- **agent**: Autonomous AI browser agent
|
|
144
|
+
- **agent**: Autonomous AI browser agent (default: auto mode with MCP-based browser + real-time reverse engineering)
|
|
145
145
|
|
|
146
146
|
Example workflow:
|
|
147
147
|
```bash
|
|
@@ -185,15 +185,20 @@ reverse-api-engineer engineer <run_id>
|
|
|
185
185
|
|
|
186
186
|
Fully automated browser interaction using AI agents:
|
|
187
187
|
|
|
188
|
-
1.
|
|
189
|
-
2.
|
|
190
|
-
3.
|
|
191
|
-
4.
|
|
192
|
-
5.
|
|
193
|
-
6.
|
|
194
|
-
|
|
188
|
+
1. Start CLI and switch to agent mode (Shift+Tab)
|
|
189
|
+
2. Enter task description (e.g., "Click on the first job listing")
|
|
190
|
+
3. Optionally provide starting URL
|
|
191
|
+
4. Agent automatically navigates and interacts
|
|
192
|
+
5. HAR captured automatically
|
|
193
|
+
6. API client generated automatically
|
|
194
|
+
|
|
195
|
+
**Agent Provider Options:**
|
|
195
196
|
|
|
196
|
-
**
|
|
197
|
+
- **auto** (default): Uses MCP-based browser automation with Claude Agent SDK & Opencode. Combines browser control and real-time reverse engineering in a single workflow. No additional installation required beyond the base package.
|
|
198
|
+
- **browser-use**: Uses browser-use library for browser automation. Requires installation with `[agent]` extra and browser-use from specific git commit (includes HAR recording support).
|
|
199
|
+
- **stagehand**: Uses Stagehand for browser automation with Computer Use models.
|
|
200
|
+
|
|
201
|
+
Change agent provider in `/settings` → "agent provider".
|
|
197
202
|
|
|
198
203
|
## 🔧 Configuration
|
|
199
204
|
|
|
@@ -204,7 +209,7 @@ Settings stored in `~/.reverse-api/config.json`:
|
|
|
204
209
|
"opencode_provider": "anthropic",
|
|
205
210
|
"opencode_model": "claude-sonnet-4-5",
|
|
206
211
|
"sdk": "claude",
|
|
207
|
-
"agent_provider": "
|
|
212
|
+
"agent_provider": "auto",
|
|
208
213
|
"browser_use_model": "bu-llm",
|
|
209
214
|
"stagehand_model": "openai/computer-use-preview-2025-03-11",
|
|
210
215
|
"output_dir": null
|
|
@@ -223,6 +228,8 @@ Change in `/settings` or via CLI:
|
|
|
223
228
|
reverse-api-engineer manual --model claude-sonnet-4-5
|
|
224
229
|
```
|
|
225
230
|
|
|
231
|
+
If you use Opencode, look at the [models](https://models.dev). Some are free like `opencode/grok-code`).
|
|
232
|
+
|
|
226
233
|
### Free Model Options with Antigravity
|
|
227
234
|
|
|
228
235
|
You can use free models via [Antigravity](https://github.com/NoeFabris/opencode-antigravity-auth) for API generation. See [available models](https://github.com/NoeFabris/opencode-antigravity-auth#available-models) for the full list.
|
|
@@ -252,7 +259,8 @@ You can use free models via [Antigravity](https://github.com/NoeFabris/opencode-
|
|
|
252
259
|
Configure AI agents for autonomous browser automation.
|
|
253
260
|
|
|
254
261
|
**Agent Providers:**
|
|
255
|
-
- **
|
|
262
|
+
- **auto** (default): MCP-based browser automation with real-time reverse engineering. Uses Claude Agent SDK with browser MCP tools. Combines browser control and API reverse engineering in a single unified workflow. Works with Claude SDK (default) or OpenCode SDK.
|
|
263
|
+
- **browser-use**: Supports Browser-Use LLM, OpenAI, and Google models. Requires installation with `[agent]` extra.
|
|
256
264
|
- **stagehand**: Supports OpenAI and Anthropic Computer Use models
|
|
257
265
|
|
|
258
266
|
**Agent Models:**
|
|
@@ -294,6 +302,18 @@ Use these slash commands while in the CLI:
|
|
|
294
302
|
- `/help` - Show all commands
|
|
295
303
|
- `/exit` - Quit
|
|
296
304
|
|
|
305
|
+
## 🔌 Claude Code Plugin
|
|
306
|
+
|
|
307
|
+
Install the plugin in [Claude Code](https://claude.com/claude-code):
|
|
308
|
+
|
|
309
|
+
```bash
|
|
310
|
+
claude # Open REPL
|
|
311
|
+
/plugin marketplace add kalil0321/reverse-api-engineer
|
|
312
|
+
/plugin install reverse-api-engineer@reverse-api-engineer
|
|
313
|
+
```
|
|
314
|
+
|
|
315
|
+
See [plugin documentation](plugins/reverse-api-engineer/README.md) for commands, agents, skills, and usage examples.
|
|
316
|
+
|
|
297
317
|
## 💡 Examples
|
|
298
318
|
|
|
299
319
|
### Example: Reverse Engineering a Job Board API
|
|
@@ -1,16 +1,14 @@
|
|
|
1
1
|
<div align="center">
|
|
2
|
-
<img src="assets/reverse-api-banner.svg" alt="Reverse API Engineer Banner">
|
|
2
|
+
<img src="https://raw.githubusercontent.com/kalil0321/reverse-api-engineer/main/assets/reverse-api-banner.svg" alt="Reverse API Engineer Banner">
|
|
3
3
|
</div>
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
[](https://badge.fury.io/py/reverse-api-engineer)
|
|
5
|
+

|
|
8
6
|
[](https://www.python.org/downloads/)
|
|
9
7
|
[](https://opensource.org/licenses/MIT)
|
|
10
8
|
|
|
11
9
|
CLI tool that captures browser traffic and automatically generates production-ready Python API clients. No more manual reverse engineering—just browse, capture, and get clean API code.
|
|
12
10
|
|
|
13
|
-

|
|
11
|
+

|
|
14
12
|
|
|
15
13
|
## Table of Contents
|
|
16
14
|
|
|
@@ -23,10 +21,11 @@ CLI tool that captures browser traffic and automatically generates production-re
|
|
|
23
21
|
- [Agent Mode](#agent-mode)
|
|
24
22
|
- [Configuration](#-configuration)
|
|
25
23
|
- [Model Selection](#model-selection)
|
|
26
|
-
- [💸 Free Models with Antigravity](
|
|
24
|
+
- [💸 Free Models with Antigravity](#free-model-options-with-antigravity)
|
|
27
25
|
- [Agent Configuration](#agent-configuration)
|
|
28
26
|
- [SDK Selection](#sdk-selection)
|
|
29
27
|
- [CLI Commands](#-cli-commands)
|
|
28
|
+
- [Claude Code Plugin](#-claude-code-plugin)
|
|
30
29
|
- [Examples](#-examples)
|
|
31
30
|
- [Development](#-development)
|
|
32
31
|
- [Contributing](#-contributing)
|
|
@@ -34,7 +33,7 @@ CLI tool that captures browser traffic and automatically generates production-re
|
|
|
34
33
|
## ✨ Features
|
|
35
34
|
|
|
36
35
|
- 🌐 **Browser Automation**: Built on Playwright with stealth mode for realistic browsing
|
|
37
|
-
- 🤖 **Autonomous Agent Mode**: Fully automated browser interaction using AI agents (browser-use, stagehand)
|
|
36
|
+
- 🤖 **Autonomous Agent Mode**: Fully automated browser interaction using AI agents (auto mode with MCP, browser-use, stagehand)
|
|
38
37
|
- 📊 **HAR Recording**: Captures all network traffic in HTTP Archive format
|
|
39
38
|
- 🧠 **AI-Powered Generation**: Uses Claude 4.5 to analyze traffic and generate clean Python code
|
|
40
39
|
- 🔌 **Multi-SDK Support**: Native integration with Claude and OpenCode SDKs
|
|
@@ -104,7 +103,7 @@ reverse-api-engineer
|
|
|
104
103
|
The CLI has three modes (cycle with **Shift+Tab**):
|
|
105
104
|
- **manual**: Browser capture + AI generation
|
|
106
105
|
- **engineer**: Re-process existing captures
|
|
107
|
-
- **agent**: Autonomous AI browser agent
|
|
106
|
+
- **agent**: Autonomous AI browser agent (default: auto mode with MCP-based browser + real-time reverse engineering)
|
|
108
107
|
|
|
109
108
|
Example workflow:
|
|
110
109
|
```bash
|
|
@@ -148,15 +147,20 @@ reverse-api-engineer engineer <run_id>
|
|
|
148
147
|
|
|
149
148
|
Fully automated browser interaction using AI agents:
|
|
150
149
|
|
|
151
|
-
1.
|
|
152
|
-
2.
|
|
153
|
-
3.
|
|
154
|
-
4.
|
|
155
|
-
5.
|
|
156
|
-
6.
|
|
157
|
-
|
|
150
|
+
1. Start CLI and switch to agent mode (Shift+Tab)
|
|
151
|
+
2. Enter task description (e.g., "Click on the first job listing")
|
|
152
|
+
3. Optionally provide starting URL
|
|
153
|
+
4. Agent automatically navigates and interacts
|
|
154
|
+
5. HAR captured automatically
|
|
155
|
+
6. API client generated automatically
|
|
156
|
+
|
|
157
|
+
**Agent Provider Options:**
|
|
158
158
|
|
|
159
|
-
**
|
|
159
|
+
- **auto** (default): Uses MCP-based browser automation with Claude Agent SDK & Opencode. Combines browser control and real-time reverse engineering in a single workflow. No additional installation required beyond the base package.
|
|
160
|
+
- **browser-use**: Uses browser-use library for browser automation. Requires installation with `[agent]` extra and browser-use from specific git commit (includes HAR recording support).
|
|
161
|
+
- **stagehand**: Uses Stagehand for browser automation with Computer Use models.
|
|
162
|
+
|
|
163
|
+
Change agent provider in `/settings` → "agent provider".
|
|
160
164
|
|
|
161
165
|
## 🔧 Configuration
|
|
162
166
|
|
|
@@ -167,7 +171,7 @@ Settings stored in `~/.reverse-api/config.json`:
|
|
|
167
171
|
"opencode_provider": "anthropic",
|
|
168
172
|
"opencode_model": "claude-sonnet-4-5",
|
|
169
173
|
"sdk": "claude",
|
|
170
|
-
"agent_provider": "
|
|
174
|
+
"agent_provider": "auto",
|
|
171
175
|
"browser_use_model": "bu-llm",
|
|
172
176
|
"stagehand_model": "openai/computer-use-preview-2025-03-11",
|
|
173
177
|
"output_dir": null
|
|
@@ -186,6 +190,8 @@ Change in `/settings` or via CLI:
|
|
|
186
190
|
reverse-api-engineer manual --model claude-sonnet-4-5
|
|
187
191
|
```
|
|
188
192
|
|
|
193
|
+
If you use Opencode, look at the [models](https://models.dev). Some are free like `opencode/grok-code`).
|
|
194
|
+
|
|
189
195
|
### Free Model Options with Antigravity
|
|
190
196
|
|
|
191
197
|
You can use free models via [Antigravity](https://github.com/NoeFabris/opencode-antigravity-auth) for API generation. See [available models](https://github.com/NoeFabris/opencode-antigravity-auth#available-models) for the full list.
|
|
@@ -215,7 +221,8 @@ You can use free models via [Antigravity](https://github.com/NoeFabris/opencode-
|
|
|
215
221
|
Configure AI agents for autonomous browser automation.
|
|
216
222
|
|
|
217
223
|
**Agent Providers:**
|
|
218
|
-
- **
|
|
224
|
+
- **auto** (default): MCP-based browser automation with real-time reverse engineering. Uses Claude Agent SDK with browser MCP tools. Combines browser control and API reverse engineering in a single unified workflow. Works with Claude SDK (default) or OpenCode SDK.
|
|
225
|
+
- **browser-use**: Supports Browser-Use LLM, OpenAI, and Google models. Requires installation with `[agent]` extra.
|
|
219
226
|
- **stagehand**: Supports OpenAI and Anthropic Computer Use models
|
|
220
227
|
|
|
221
228
|
**Agent Models:**
|
|
@@ -257,6 +264,18 @@ Use these slash commands while in the CLI:
|
|
|
257
264
|
- `/help` - Show all commands
|
|
258
265
|
- `/exit` - Quit
|
|
259
266
|
|
|
267
|
+
## 🔌 Claude Code Plugin
|
|
268
|
+
|
|
269
|
+
Install the plugin in [Claude Code](https://claude.com/claude-code):
|
|
270
|
+
|
|
271
|
+
```bash
|
|
272
|
+
claude # Open REPL
|
|
273
|
+
/plugin marketplace add kalil0321/reverse-api-engineer
|
|
274
|
+
/plugin install reverse-api-engineer@reverse-api-engineer
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
See [plugin documentation](plugins/reverse-api-engineer/README.md) for commands, agents, skills, and usage examples.
|
|
278
|
+
|
|
260
279
|
## 💡 Examples
|
|
261
280
|
|
|
262
281
|
### Example: Reverse Engineering a Job Board API
|