auto-api-discovery 1.0.0 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -21
- package/README.md +28 -55
- package/package.json +1 -1
package/LICENSE
CHANGED
|
@@ -1,21 +1,21 @@
|
|
|
1
|
-
MIT License
|
|
2
|
-
|
|
3
|
-
Copyright (c) 2026 Anooj Shete
|
|
4
|
-
|
|
5
|
-
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
-
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
-
in the Software without restriction, including without limitation the rights
|
|
8
|
-
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
-
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
-
furnished to do so, subject to the following conditions:
|
|
11
|
-
|
|
12
|
-
The above copyright notice and this permission notice shall be included in all
|
|
13
|
-
copies or substantial portions of the Software.
|
|
14
|
-
|
|
15
|
-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
-
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
-
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
-
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
-
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
-
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
-
SOFTWARE.
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Anooj Shete
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,79 +1,52 @@
|
|
|
1
|
-
#
|
|
1
|
+
# ApiGen (auto-api-discovery)
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
[](https://www.npmjs.com/package/auto-api-discovery)
|
|
4
|
+
[](https://opensource.org/licenses/MIT)
|
|
4
5
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
6
|
+
A CLI tool for automated API discovery. ApiGen intercepts browser network traffic (XHR, Fetch, GraphQL) to automatically build OpenAPI 3.0 specifications.
|
|
9
7
|
|
|
10
|
-
|
|
8
|
+
## Installation
|
|
11
9
|
|
|
12
|
-
|
|
10
|
+
Install the package globally via npm:
|
|
13
11
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
---
|
|
19
|
-
|
|
20
|
-
## 🏗️ Architecture
|
|
21
|
-
|
|
22
|
-
ApiGen utilizes a **Hybrid API Discovery Approach** consisting of:
|
|
23
|
-
|
|
24
|
-
1. **Passive Recording (Capture Mode):** Uses a headful instance of Playwright. You navigate a target web application as a real user—bypassing CAPTCHAs, 2FA, and complex login flows. Under the hood, ApiGen intercepts all network requests/responses and logs them securely into a local SQLite database. Upon closing the browser, it actively exports your authenticated session state into `.apigen-session.json`.
|
|
25
|
-
2. **Auto-Spidering (Crawl Mode):** Takes the previously captured authenticated session state and launches a headless Playwright Breadth-First-Search (BFS) spider. It organically navigates the application just like an authenticated user would, discovering and hitting protected routes, intelligently avoiding generic WAF traps using randomized latency offsets, and feeding new underlying API traffic deeply into the local SQLite store.
|
|
26
|
-
3. **OpenAPI Generation (Export):** A local robust schema inference engine processes thousands of SQLite records. It aggressively deduplicates dynamic URLs (folding Object IDs and UUIDs into neat path parameters), accurately infers nested JSON payload structures recursively, and compiles them to a strict, standard OpenAPI 3.0 json specification document.
|
|
27
|
-
|
|
28
|
-
---
|
|
29
|
-
|
|
30
|
-
## ⚙️ Tech Stack
|
|
31
|
-
|
|
32
|
-
- **[Node.js](https://nodejs.org/en/) & [TypeScript](https://www.typescriptlang.org/)** - For robust type-safe runtime execution.
|
|
33
|
-
- **[Playwright](https://playwright.dev/)** - For complete DOM navigation, session persistence, and native underlying browser CDP network interception workflows.
|
|
34
|
-
- **[Better-SQLite3](https://github.com/WiseLibs/better-sqlite3)** - Highly scalable, local WAL-mode database used as the high-throughput primary storage layer.
|
|
35
|
-
- **[Commander.js](https://github.com/tj/commander.js/)** - For building the intuitive Command Line Interface.
|
|
36
|
-
- **[Chalk](https://github.com/chalk/chalk)** - For beautiful and clear terminal logging feedback.
|
|
37
|
-
|
|
38
|
-
---
|
|
39
|
-
|
|
40
|
-
## 🛠️ Getting Started
|
|
12
|
+
```bash
|
|
13
|
+
npm install -g auto-api-discovery
|
|
14
|
+
```
|
|
41
15
|
|
|
42
|
-
|
|
16
|
+
*(Note: The `postinstall` script will automatically download the Chromium binaries required by Playwright.)*
|
|
43
17
|
|
|
44
|
-
|
|
18
|
+
Alternatively, you can run it directly via `npx`:
|
|
45
19
|
|
|
46
20
|
```bash
|
|
47
|
-
|
|
48
|
-
cd auto-api-discovery
|
|
49
|
-
npm install
|
|
50
|
-
npx playwright install chromium
|
|
51
|
-
npm run build
|
|
21
|
+
npx auto-api-discovery capture https://example.com
|
|
52
22
|
```
|
|
53
23
|
|
|
54
|
-
|
|
24
|
+
## Usage
|
|
55
25
|
|
|
56
|
-
|
|
26
|
+
Once installed, the `apigen` CLI becomes available.
|
|
57
27
|
|
|
58
|
-
|
|
59
|
-
|
|
28
|
+
### 1. Capture API Traffic
|
|
29
|
+
Launch an interactive browser. You can manually log in, navigate the app, and bypass captchas. ApiGen intercepts the underlying API requests and saves them to a local SQLite database.
|
|
60
30
|
|
|
61
31
|
```bash
|
|
62
|
-
|
|
32
|
+
apigen capture https://example.com
|
|
63
33
|
```
|
|
64
34
|
|
|
65
|
-
|
|
66
|
-
|
|
35
|
+
### 2. Crawl Connected Pages (Auto-Spidering)
|
|
36
|
+
Use your previously captured authenticated session to automatically run a headless crawler. It discovers and logs additional API endpoints in the background.
|
|
67
37
|
|
|
68
38
|
```bash
|
|
69
|
-
|
|
70
|
-
npm run apigen crawl https://example.com --depth 3 --pages 50
|
|
39
|
+
apigen crawl https://example.com --depth 3 --pages 50
|
|
71
40
|
```
|
|
72
41
|
|
|
73
|
-
|
|
74
|
-
|
|
42
|
+
### 3. Export OpenAPI Schema
|
|
43
|
+
Convert the recorded API traffic into a unified OpenAPI 3.0 specification document. Dynamic IDs and UUIDs are automatically folded into path parameters.
|
|
75
44
|
|
|
76
45
|
```bash
|
|
77
|
-
|
|
78
|
-
npm run apigen export ./openapi.json --base-url https://api.example.com
|
|
46
|
+
apigen export ./openapi.json --base-url https://api.example.com
|
|
79
47
|
```
|
|
48
|
+
|
|
49
|
+
## How It Works
|
|
50
|
+
- Uses **Playwright** to proxy network requests and manage sessions.
|
|
51
|
+
- Stores metadata and traffic locally using **Better-SQLite3**.
|
|
52
|
+
- Automatically infers JSON payload structures and route schemas.
|