aicodinggym-cli 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- aicodinggym_cli-0.1.0/PKG-INFO +217 -0
- aicodinggym_cli-0.1.0/README.md +198 -0
- aicodinggym_cli-0.1.0/__init__.py +3 -0
- aicodinggym_cli-0.1.0/aicodinggym_cli.egg-info/PKG-INFO +217 -0
- aicodinggym_cli-0.1.0/aicodinggym_cli.egg-info/SOURCES.txt +18 -0
- aicodinggym_cli-0.1.0/aicodinggym_cli.egg-info/dependency_links.txt +1 -0
- aicodinggym_cli-0.1.0/aicodinggym_cli.egg-info/entry_points.txt +2 -0
- aicodinggym_cli-0.1.0/aicodinggym_cli.egg-info/requires.txt +2 -0
- aicodinggym_cli-0.1.0/aicodinggym_cli.egg-info/top_level.txt +1 -0
- aicodinggym_cli-0.1.0/api.py +132 -0
- aicodinggym_cli-0.1.0/cli.py +641 -0
- aicodinggym_cli-0.1.0/config.py +80 -0
- aicodinggym_cli-0.1.0/git_ops.py +175 -0
- aicodinggym_cli-0.1.0/pyproject.toml +39 -0
- aicodinggym_cli-0.1.0/setup.cfg +4 -0
|
@@ -0,0 +1,217 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: aicodinggym-cli
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: CLI tool for AI Coding Gym platform
|
|
5
|
+
Author-email: AICodingGym Team <datasmithlab@gmail.com>
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://aicodinggym.com
|
|
8
|
+
Keywords: cli,coding-gym,swebench,mlebench,ai
|
|
9
|
+
Classifier: Development Status :: 3 - Alpha
|
|
10
|
+
Classifier: Intended Audience :: Developers
|
|
11
|
+
Classifier: Programming Language :: Python :: 3
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
15
|
+
Requires-Python: >=3.10
|
|
16
|
+
Description-Content-Type: text/markdown
|
|
17
|
+
Requires-Dist: click>=8.0.0
|
|
18
|
+
Requires-Dist: requests>=2.31.0
|
|
19
|
+
|
|
20
|
+
# aicodinggym-cli API Design
|
|
21
|
+
|
|
22
|
+
## Overview
|
|
23
|
+
|
|
24
|
+
CLI tool for the AI Coding Gym platform (`https://aicodinggym.com`).
|
|
25
|
+
Supports two benchmarks: **SWE-bench** (code bug fixes) and **MLE-bench** (ML competitions).
|
|
26
|
+
|
|
27
|
+
**Install:** `pip install aicodinggym-cli`
|
|
28
|
+
**Entry point:** `aicodinggym`
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## Commands
|
|
33
|
+
|
|
34
|
+
### `aicodinggym configure`
|
|
35
|
+
|
|
36
|
+
One-time setup. Generates SSH key, registers with server.
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
aicodinggym configure --user-id USER_ID [--workspace-dir DIR]
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
| Option | Required | Description |
|
|
43
|
+
|---|---|---|
|
|
44
|
+
| `--user-id` | Yes | Your AI Coding Gym user ID |
|
|
45
|
+
| `--workspace-dir` | No | Default workspace directory (default: cwd) |
|
|
46
|
+
|
|
47
|
+
**Backend API:** `POST /api/configure`
|
|
48
|
+
```json
|
|
49
|
+
// Request
|
|
50
|
+
{"user_id": "alice123", "public_key": "ssh-rsa AAAA..."}
|
|
51
|
+
// Response
|
|
52
|
+
{"repo_name": "alice123-swebench"}
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
**Local storage:** `~/.aicodinggym/config.json`, `~/.aicodinggym/{user_id}_id_rsa`
|
|
56
|
+
|
|
57
|
+
---
|
|
58
|
+
|
|
59
|
+
### `aicodinggym swe` — SWE-bench Commands
|
|
60
|
+
|
|
61
|
+
#### `aicodinggym swe fetch PROBLEM_ID`
|
|
62
|
+
|
|
63
|
+
Fetch a problem and clone the repo locally (shallow clone).
|
|
64
|
+
|
|
65
|
+
```
|
|
66
|
+
aicodinggym swe fetch PROBLEM_ID [--user-id ID] [--workspace-dir DIR]
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
**Backend API:** `POST /api/fetch-problem`
|
|
70
|
+
```json
|
|
71
|
+
// Request
|
|
72
|
+
{"user_id": "alice123", "problem_id": "django__django-10097"}
|
|
73
|
+
// Response
|
|
74
|
+
{"branch_name": "django__django-10097-alice123", "repo_url": "git@...", "message": "..."}
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
**Local effect:** Clones `<workspace>/<problem_id>/` via SSH.
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
#### `aicodinggym swe submit PROBLEM_ID`
|
|
82
|
+
|
|
83
|
+
Commit all changes and push to remote. Notifies backend.
|
|
84
|
+
|
|
85
|
+
```
|
|
86
|
+
aicodinggym swe submit PROBLEM_ID [--message MSG] [--force] [--user-id ID] [--workspace-dir DIR]
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
| Option | Description |
|
|
90
|
+
|---|---|
|
|
91
|
+
| `--message, -m` | Commit message (auto-generated if omitted) |
|
|
92
|
+
| `--force` | Force push with `--force-with-lease` |
|
|
93
|
+
|
|
94
|
+
**Local effect:** `git add -A`, `git commit`, `git push`
|
|
95
|
+
|
|
96
|
+
**Backend API:** `POST /api/submissions`
|
|
97
|
+
```json
|
|
98
|
+
// Request
|
|
99
|
+
{
|
|
100
|
+
"problem_id": "django__django-10097",
|
|
101
|
+
"user_id": "alice123",
|
|
102
|
+
"commit_hash": "abc123...",
|
|
103
|
+
"branch": "django__django-10097-alice123",
|
|
104
|
+
"commit_message": "Fix auth bug",
|
|
105
|
+
"timestamp": "2026-03-05T10:30:00"
|
|
106
|
+
}
|
|
107
|
+
// Response
|
|
108
|
+
{"status": "success", "message": "Submission received"}
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
---
|
|
112
|
+
|
|
113
|
+
#### `aicodinggym swe reset PROBLEM_ID`
|
|
114
|
+
|
|
115
|
+
Reset repo to original setup commit. Destructive — discards all local changes.
|
|
116
|
+
|
|
117
|
+
```
|
|
118
|
+
aicodinggym swe reset PROBLEM_ID [--user-id ID] [--workspace-dir DIR]
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
**Local effect:** `git reset --hard <setup_commit>`, `git clean -fd`
|
|
122
|
+
|
|
123
|
+
---
|
|
124
|
+
|
|
125
|
+
### `aicodinggym mle` — MLE-bench Commands
|
|
126
|
+
|
|
127
|
+
#### `aicodinggym mle download COMPETITION_ID`
|
|
128
|
+
|
|
129
|
+
Download dataset files (train/test/sample_submission CSVs).
|
|
130
|
+
|
|
131
|
+
```
|
|
132
|
+
aicodinggym mle download COMPETITION_ID [--user-id ID] [--output-dir DIR]
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
| Option | Description |
|
|
136
|
+
|---|---|
|
|
137
|
+
| `--output-dir` | Save location (default: `./<competition_id>/data/`) |
|
|
138
|
+
|
|
139
|
+
**Backend API:** `POST /api/mlebench/download`
|
|
140
|
+
```json
|
|
141
|
+
// Request
|
|
142
|
+
{"user_id": "alice123", "competition_id": "spaceship-titanic"}
|
|
143
|
+
// Response (single archive)
|
|
144
|
+
{"download_url": "https://...", "filename": "spaceship-titanic_data.zip", "message": "..."}
|
|
145
|
+
// Response (multiple files)
|
|
146
|
+
{"files": [{"name": "train.csv", "url": "https://..."}, ...], "message": "..."}
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
---
|
|
150
|
+
|
|
151
|
+
#### `aicodinggym mle submit COMPETITION_ID -F FILE`
|
|
152
|
+
|
|
153
|
+
Upload prediction CSV for scoring.
|
|
154
|
+
|
|
155
|
+
```
|
|
156
|
+
aicodinggym mle submit COMPETITION_ID -F FILE [--user-id ID] [--message MSG]
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
| Option | Required | Description |
|
|
160
|
+
|---|---|---|
|
|
161
|
+
| `-F` | Yes | Path to prediction CSV file |
|
|
162
|
+
| `--message, -m` | No | Submission description |
|
|
163
|
+
|
|
164
|
+
**Backend API:** `POST /api/mlebench/submit` (multipart form)
|
|
165
|
+
```
|
|
166
|
+
POST /api/mlebench/submit
|
|
167
|
+
Content-Type: multipart/form-data
|
|
168
|
+
|
|
169
|
+
user_id=alice123
|
|
170
|
+
competition_id=spaceship-titanic
|
|
171
|
+
file=@predictions.csv
|
|
172
|
+
```
|
|
173
|
+
```json
|
|
174
|
+
// Response
|
|
175
|
+
{"message": "Submission received", "score": 0.85}
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
---
|
|
179
|
+
|
|
180
|
+
## File Structure
|
|
181
|
+
|
|
182
|
+
```
|
|
183
|
+
src/aicodinggym/
|
|
184
|
+
├── __init__.py # Version
|
|
185
|
+
├── cli.py # Click CLI commands (entry point)
|
|
186
|
+
├── config.py # Config + credentials persistence (~/.aicodinggym/)
|
|
187
|
+
├── api.py # HTTP client for aicodinggym.com/api
|
|
188
|
+
└── git_ops.py # SSH key generation, git clone/commit/push/reset
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
## Configuration Files
|
|
192
|
+
|
|
193
|
+
| File | Purpose |
|
|
194
|
+
|---|---|
|
|
195
|
+
| `~/.aicodinggym/config.json` | Global config (user_id, repo_name, key path, workspace) |
|
|
196
|
+
| `~/.aicodinggym/credentials.json` | Per-problem credentials (repo_url, branch, cached after fetch) |
|
|
197
|
+
| `~/.aicodinggym/{user_id}_id_rsa` | SSH private key |
|
|
198
|
+
| `~/.aicodinggym/{user_id}_id_rsa.pub` | SSH public key |
|
|
199
|
+
|
|
200
|
+
## Error Handling
|
|
201
|
+
|
|
202
|
+
All errors print actionable messages guiding the user (or LLM agent) to the correct next step:
|
|
203
|
+
|
|
204
|
+
- **Not configured** → "Run 'aicodinggym configure --user-id YOUR_USER_ID' first."
|
|
205
|
+
- **Not fetched** → "You must fetch the problem first: aicodinggym swe fetch PROBLEM_ID"
|
|
206
|
+
- **API unreachable** → "Cannot connect to https://aicodinggym.com/api. Check your internet connection."
|
|
207
|
+
- **User ID mismatch** → "Problem was fetched by 'X', not 'Y'. Either use --user-id X or re-fetch."
|
|
208
|
+
|
|
209
|
+
## Backend API Summary
|
|
210
|
+
|
|
211
|
+
| Endpoint | Method | Used By |
|
|
212
|
+
|---|---|---|
|
|
213
|
+
| `/api/configure` | POST | `configure` |
|
|
214
|
+
| `/api/fetch-problem` | POST | `swe fetch` |
|
|
215
|
+
| `/api/submissions` | POST | `swe submit` |
|
|
216
|
+
| `/api/mlebench/download` | POST | `mle download` |
|
|
217
|
+
| `/api/mlebench/submit` | POST | `mle submit` |
|
|
@@ -0,0 +1,198 @@
|
|
|
1
|
+
# aicodinggym-cli API Design
|
|
2
|
+
|
|
3
|
+
## Overview
|
|
4
|
+
|
|
5
|
+
CLI tool for the AI Coding Gym platform (`https://aicodinggym.com`).
|
|
6
|
+
Supports two benchmarks: **SWE-bench** (code bug fixes) and **MLE-bench** (ML competitions).
|
|
7
|
+
|
|
8
|
+
**Install:** `pip install aicodinggym-cli`
|
|
9
|
+
**Entry point:** `aicodinggym`
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Commands
|
|
14
|
+
|
|
15
|
+
### `aicodinggym configure`
|
|
16
|
+
|
|
17
|
+
One-time setup. Generates SSH key, registers with server.
|
|
18
|
+
|
|
19
|
+
```
|
|
20
|
+
aicodinggym configure --user-id USER_ID [--workspace-dir DIR]
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
| Option | Required | Description |
|
|
24
|
+
|---|---|---|
|
|
25
|
+
| `--user-id` | Yes | Your AI Coding Gym user ID |
|
|
26
|
+
| `--workspace-dir` | No | Default workspace directory (default: cwd) |
|
|
27
|
+
|
|
28
|
+
**Backend API:** `POST /api/configure`
|
|
29
|
+
```json
|
|
30
|
+
// Request
|
|
31
|
+
{"user_id": "alice123", "public_key": "ssh-rsa AAAA..."}
|
|
32
|
+
// Response
|
|
33
|
+
{"repo_name": "alice123-swebench"}
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
**Local storage:** `~/.aicodinggym/config.json`, `~/.aicodinggym/{user_id}_id_rsa`
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
### `aicodinggym swe` — SWE-bench Commands
|
|
41
|
+
|
|
42
|
+
#### `aicodinggym swe fetch PROBLEM_ID`
|
|
43
|
+
|
|
44
|
+
Fetch a problem and clone the repo locally (shallow clone).
|
|
45
|
+
|
|
46
|
+
```
|
|
47
|
+
aicodinggym swe fetch PROBLEM_ID [--user-id ID] [--workspace-dir DIR]
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
**Backend API:** `POST /api/fetch-problem`
|
|
51
|
+
```json
|
|
52
|
+
// Request
|
|
53
|
+
{"user_id": "alice123", "problem_id": "django__django-10097"}
|
|
54
|
+
// Response
|
|
55
|
+
{"branch_name": "django__django-10097-alice123", "repo_url": "git@...", "message": "..."}
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
**Local effect:** Clones `<workspace>/<problem_id>/` via SSH.
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
#### `aicodinggym swe submit PROBLEM_ID`
|
|
63
|
+
|
|
64
|
+
Commit all changes and push to remote. Notifies backend.
|
|
65
|
+
|
|
66
|
+
```
|
|
67
|
+
aicodinggym swe submit PROBLEM_ID [--message MSG] [--force] [--user-id ID] [--workspace-dir DIR]
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
| Option | Description |
|
|
71
|
+
|---|---|
|
|
72
|
+
| `--message, -m` | Commit message (auto-generated if omitted) |
|
|
73
|
+
| `--force` | Force push with `--force-with-lease` |
|
|
74
|
+
|
|
75
|
+
**Local effect:** `git add -A`, `git commit`, `git push`
|
|
76
|
+
|
|
77
|
+
**Backend API:** `POST /api/submissions`
|
|
78
|
+
```json
|
|
79
|
+
// Request
|
|
80
|
+
{
|
|
81
|
+
"problem_id": "django__django-10097",
|
|
82
|
+
"user_id": "alice123",
|
|
83
|
+
"commit_hash": "abc123...",
|
|
84
|
+
"branch": "django__django-10097-alice123",
|
|
85
|
+
"commit_message": "Fix auth bug",
|
|
86
|
+
"timestamp": "2026-03-05T10:30:00"
|
|
87
|
+
}
|
|
88
|
+
// Response
|
|
89
|
+
{"status": "success", "message": "Submission received"}
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
#### `aicodinggym swe reset PROBLEM_ID`
|
|
95
|
+
|
|
96
|
+
Reset repo to original setup commit. Destructive — discards all local changes.
|
|
97
|
+
|
|
98
|
+
```
|
|
99
|
+
aicodinggym swe reset PROBLEM_ID [--user-id ID] [--workspace-dir DIR]
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
**Local effect:** `git reset --hard <setup_commit>`, `git clean -fd`
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
### `aicodinggym mle` — MLE-bench Commands
|
|
107
|
+
|
|
108
|
+
#### `aicodinggym mle download COMPETITION_ID`
|
|
109
|
+
|
|
110
|
+
Download dataset files (train/test/sample_submission CSVs).
|
|
111
|
+
|
|
112
|
+
```
|
|
113
|
+
aicodinggym mle download COMPETITION_ID [--user-id ID] [--output-dir DIR]
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
| Option | Description |
|
|
117
|
+
|---|---|
|
|
118
|
+
| `--output-dir` | Save location (default: `./<competition_id>/data/`) |
|
|
119
|
+
|
|
120
|
+
**Backend API:** `POST /api/mlebench/download`
|
|
121
|
+
```json
|
|
122
|
+
// Request
|
|
123
|
+
{"user_id": "alice123", "competition_id": "spaceship-titanic"}
|
|
124
|
+
// Response (single archive)
|
|
125
|
+
{"download_url": "https://...", "filename": "spaceship-titanic_data.zip", "message": "..."}
|
|
126
|
+
// Response (multiple files)
|
|
127
|
+
{"files": [{"name": "train.csv", "url": "https://..."}, ...], "message": "..."}
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
---
|
|
131
|
+
|
|
132
|
+
#### `aicodinggym mle submit COMPETITION_ID -F FILE`
|
|
133
|
+
|
|
134
|
+
Upload prediction CSV for scoring.
|
|
135
|
+
|
|
136
|
+
```
|
|
137
|
+
aicodinggym mle submit COMPETITION_ID -F FILE [--user-id ID] [--message MSG]
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
| Option | Required | Description |
|
|
141
|
+
|---|---|---|
|
|
142
|
+
| `-F` | Yes | Path to prediction CSV file |
|
|
143
|
+
| `--message, -m` | No | Submission description |
|
|
144
|
+
|
|
145
|
+
**Backend API:** `POST /api/mlebench/submit` (multipart form)
|
|
146
|
+
```
|
|
147
|
+
POST /api/mlebench/submit
|
|
148
|
+
Content-Type: multipart/form-data
|
|
149
|
+
|
|
150
|
+
user_id=alice123
|
|
151
|
+
competition_id=spaceship-titanic
|
|
152
|
+
file=@predictions.csv
|
|
153
|
+
```
|
|
154
|
+
```json
|
|
155
|
+
// Response
|
|
156
|
+
{"message": "Submission received", "score": 0.85}
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
---
|
|
160
|
+
|
|
161
|
+
## File Structure
|
|
162
|
+
|
|
163
|
+
```
|
|
164
|
+
src/aicodinggym/
|
|
165
|
+
├── __init__.py # Version
|
|
166
|
+
├── cli.py # Click CLI commands (entry point)
|
|
167
|
+
├── config.py # Config + credentials persistence (~/.aicodinggym/)
|
|
168
|
+
├── api.py # HTTP client for aicodinggym.com/api
|
|
169
|
+
└── git_ops.py # SSH key generation, git clone/commit/push/reset
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
## Configuration Files
|
|
173
|
+
|
|
174
|
+
| File | Purpose |
|
|
175
|
+
|---|---|
|
|
176
|
+
| `~/.aicodinggym/config.json` | Global config (user_id, repo_name, key path, workspace) |
|
|
177
|
+
| `~/.aicodinggym/credentials.json` | Per-problem credentials (repo_url, branch, cached after fetch) |
|
|
178
|
+
| `~/.aicodinggym/{user_id}_id_rsa` | SSH private key |
|
|
179
|
+
| `~/.aicodinggym/{user_id}_id_rsa.pub` | SSH public key |
|
|
180
|
+
|
|
181
|
+
## Error Handling
|
|
182
|
+
|
|
183
|
+
All errors print actionable messages guiding the user (or LLM agent) to the correct next step:
|
|
184
|
+
|
|
185
|
+
- **Not configured** → "Run 'aicodinggym configure --user-id YOUR_USER_ID' first."
|
|
186
|
+
- **Not fetched** → "You must fetch the problem first: aicodinggym swe fetch PROBLEM_ID"
|
|
187
|
+
- **API unreachable** → "Cannot connect to https://aicodinggym.com/api. Check your internet connection."
|
|
188
|
+
- **User ID mismatch** → "Problem was fetched by 'X', not 'Y'. Either use --user-id X or re-fetch."
|
|
189
|
+
|
|
190
|
+
## Backend API Summary
|
|
191
|
+
|
|
192
|
+
| Endpoint | Method | Used By |
|
|
193
|
+
|---|---|---|
|
|
194
|
+
| `/api/configure` | POST | `configure` |
|
|
195
|
+
| `/api/fetch-problem` | POST | `swe fetch` |
|
|
196
|
+
| `/api/submissions` | POST | `swe submit` |
|
|
197
|
+
| `/api/mlebench/download` | POST | `mle download` |
|
|
198
|
+
| `/api/mlebench/submit` | POST | `mle submit` |
|
|
@@ -0,0 +1,217 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: aicodinggym-cli
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: CLI tool for AI Coding Gym platform
|
|
5
|
+
Author-email: AICodingGym Team <datasmithlab@gmail.com>
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://aicodinggym.com
|
|
8
|
+
Keywords: cli,coding-gym,swebench,mlebench,ai
|
|
9
|
+
Classifier: Development Status :: 3 - Alpha
|
|
10
|
+
Classifier: Intended Audience :: Developers
|
|
11
|
+
Classifier: Programming Language :: Python :: 3
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
15
|
+
Requires-Python: >=3.10
|
|
16
|
+
Description-Content-Type: text/markdown
|
|
17
|
+
Requires-Dist: click>=8.0.0
|
|
18
|
+
Requires-Dist: requests>=2.31.0
|
|
19
|
+
|
|
20
|
+
# aicodinggym-cli API Design
|
|
21
|
+
|
|
22
|
+
## Overview
|
|
23
|
+
|
|
24
|
+
CLI tool for the AI Coding Gym platform (`https://aicodinggym.com`).
|
|
25
|
+
Supports two benchmarks: **SWE-bench** (code bug fixes) and **MLE-bench** (ML competitions).
|
|
26
|
+
|
|
27
|
+
**Install:** `pip install aicodinggym-cli`
|
|
28
|
+
**Entry point:** `aicodinggym`
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## Commands
|
|
33
|
+
|
|
34
|
+
### `aicodinggym configure`
|
|
35
|
+
|
|
36
|
+
One-time setup. Generates SSH key, registers with server.
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
aicodinggym configure --user-id USER_ID [--workspace-dir DIR]
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
| Option | Required | Description |
|
|
43
|
+
|---|---|---|
|
|
44
|
+
| `--user-id` | Yes | Your AI Coding Gym user ID |
|
|
45
|
+
| `--workspace-dir` | No | Default workspace directory (default: cwd) |
|
|
46
|
+
|
|
47
|
+
**Backend API:** `POST /api/configure`
|
|
48
|
+
```json
|
|
49
|
+
// Request
|
|
50
|
+
{"user_id": "alice123", "public_key": "ssh-rsa AAAA..."}
|
|
51
|
+
// Response
|
|
52
|
+
{"repo_name": "alice123-swebench"}
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
**Local storage:** `~/.aicodinggym/config.json`, `~/.aicodinggym/{user_id}_id_rsa`
|
|
56
|
+
|
|
57
|
+
---
|
|
58
|
+
|
|
59
|
+
### `aicodinggym swe` — SWE-bench Commands
|
|
60
|
+
|
|
61
|
+
#### `aicodinggym swe fetch PROBLEM_ID`
|
|
62
|
+
|
|
63
|
+
Fetch a problem and clone the repo locally (shallow clone).
|
|
64
|
+
|
|
65
|
+
```
|
|
66
|
+
aicodinggym swe fetch PROBLEM_ID [--user-id ID] [--workspace-dir DIR]
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
**Backend API:** `POST /api/fetch-problem`
|
|
70
|
+
```json
|
|
71
|
+
// Request
|
|
72
|
+
{"user_id": "alice123", "problem_id": "django__django-10097"}
|
|
73
|
+
// Response
|
|
74
|
+
{"branch_name": "django__django-10097-alice123", "repo_url": "git@...", "message": "..."}
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
**Local effect:** Clones `<workspace>/<problem_id>/` via SSH.
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
#### `aicodinggym swe submit PROBLEM_ID`
|
|
82
|
+
|
|
83
|
+
Commit all changes and push to remote. Notifies backend.
|
|
84
|
+
|
|
85
|
+
```
|
|
86
|
+
aicodinggym swe submit PROBLEM_ID [--message MSG] [--force] [--user-id ID] [--workspace-dir DIR]
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
| Option | Description |
|
|
90
|
+
|---|---|
|
|
91
|
+
| `--message, -m` | Commit message (auto-generated if omitted) |
|
|
92
|
+
| `--force` | Force push with `--force-with-lease` |
|
|
93
|
+
|
|
94
|
+
**Local effect:** `git add -A`, `git commit`, `git push`
|
|
95
|
+
|
|
96
|
+
**Backend API:** `POST /api/submissions`
|
|
97
|
+
```json
|
|
98
|
+
// Request
|
|
99
|
+
{
|
|
100
|
+
"problem_id": "django__django-10097",
|
|
101
|
+
"user_id": "alice123",
|
|
102
|
+
"commit_hash": "abc123...",
|
|
103
|
+
"branch": "django__django-10097-alice123",
|
|
104
|
+
"commit_message": "Fix auth bug",
|
|
105
|
+
"timestamp": "2026-03-05T10:30:00"
|
|
106
|
+
}
|
|
107
|
+
// Response
|
|
108
|
+
{"status": "success", "message": "Submission received"}
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
---
|
|
112
|
+
|
|
113
|
+
#### `aicodinggym swe reset PROBLEM_ID`
|
|
114
|
+
|
|
115
|
+
Reset repo to original setup commit. Destructive — discards all local changes.
|
|
116
|
+
|
|
117
|
+
```
|
|
118
|
+
aicodinggym swe reset PROBLEM_ID [--user-id ID] [--workspace-dir DIR]
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
**Local effect:** `git reset --hard <setup_commit>`, `git clean -fd`
|
|
122
|
+
|
|
123
|
+
---
|
|
124
|
+
|
|
125
|
+
### `aicodinggym mle` — MLE-bench Commands
|
|
126
|
+
|
|
127
|
+
#### `aicodinggym mle download COMPETITION_ID`
|
|
128
|
+
|
|
129
|
+
Download dataset files (train/test/sample_submission CSVs).
|
|
130
|
+
|
|
131
|
+
```
|
|
132
|
+
aicodinggym mle download COMPETITION_ID [--user-id ID] [--output-dir DIR]
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
| Option | Description |
|
|
136
|
+
|---|---|
|
|
137
|
+
| `--output-dir` | Save location (default: `./<competition_id>/data/`) |
|
|
138
|
+
|
|
139
|
+
**Backend API:** `POST /api/mlebench/download`
|
|
140
|
+
```json
|
|
141
|
+
// Request
|
|
142
|
+
{"user_id": "alice123", "competition_id": "spaceship-titanic"}
|
|
143
|
+
// Response (single archive)
|
|
144
|
+
{"download_url": "https://...", "filename": "spaceship-titanic_data.zip", "message": "..."}
|
|
145
|
+
// Response (multiple files)
|
|
146
|
+
{"files": [{"name": "train.csv", "url": "https://..."}, ...], "message": "..."}
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
---
|
|
150
|
+
|
|
151
|
+
#### `aicodinggym mle submit COMPETITION_ID -F FILE`
|
|
152
|
+
|
|
153
|
+
Upload prediction CSV for scoring.
|
|
154
|
+
|
|
155
|
+
```
|
|
156
|
+
aicodinggym mle submit COMPETITION_ID -F FILE [--user-id ID] [--message MSG]
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
| Option | Required | Description |
|
|
160
|
+
|---|---|---|
|
|
161
|
+
| `-F` | Yes | Path to prediction CSV file |
|
|
162
|
+
| `--message, -m` | No | Submission description |
|
|
163
|
+
|
|
164
|
+
**Backend API:** `POST /api/mlebench/submit` (multipart form)
|
|
165
|
+
```
|
|
166
|
+
POST /api/mlebench/submit
|
|
167
|
+
Content-Type: multipart/form-data
|
|
168
|
+
|
|
169
|
+
user_id=alice123
|
|
170
|
+
competition_id=spaceship-titanic
|
|
171
|
+
file=@predictions.csv
|
|
172
|
+
```
|
|
173
|
+
```json
|
|
174
|
+
// Response
|
|
175
|
+
{"message": "Submission received", "score": 0.85}
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
---
|
|
179
|
+
|
|
180
|
+
## File Structure
|
|
181
|
+
|
|
182
|
+
```
|
|
183
|
+
src/aicodinggym/
|
|
184
|
+
├── __init__.py # Version
|
|
185
|
+
├── cli.py # Click CLI commands (entry point)
|
|
186
|
+
├── config.py # Config + credentials persistence (~/.aicodinggym/)
|
|
187
|
+
├── api.py # HTTP client for aicodinggym.com/api
|
|
188
|
+
└── git_ops.py # SSH key generation, git clone/commit/push/reset
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
## Configuration Files
|
|
192
|
+
|
|
193
|
+
| File | Purpose |
|
|
194
|
+
|---|---|
|
|
195
|
+
| `~/.aicodinggym/config.json` | Global config (user_id, repo_name, key path, workspace) |
|
|
196
|
+
| `~/.aicodinggym/credentials.json` | Per-problem credentials (repo_url, branch, cached after fetch) |
|
|
197
|
+
| `~/.aicodinggym/{user_id}_id_rsa` | SSH private key |
|
|
198
|
+
| `~/.aicodinggym/{user_id}_id_rsa.pub` | SSH public key |
|
|
199
|
+
|
|
200
|
+
## Error Handling
|
|
201
|
+
|
|
202
|
+
All errors print actionable messages guiding the user (or LLM agent) to the correct next step:
|
|
203
|
+
|
|
204
|
+
- **Not configured** → "Run 'aicodinggym configure --user-id YOUR_USER_ID' first."
|
|
205
|
+
- **Not fetched** → "You must fetch the problem first: aicodinggym swe fetch PROBLEM_ID"
|
|
206
|
+
- **API unreachable** → "Cannot connect to https://aicodinggym.com/api. Check your internet connection."
|
|
207
|
+
- **User ID mismatch** → "Problem was fetched by 'X', not 'Y'. Either use --user-id X or re-fetch."
|
|
208
|
+
|
|
209
|
+
## Backend API Summary
|
|
210
|
+
|
|
211
|
+
| Endpoint | Method | Used By |
|
|
212
|
+
|---|---|---|
|
|
213
|
+
| `/api/configure` | POST | `configure` |
|
|
214
|
+
| `/api/fetch-problem` | POST | `swe fetch` |
|
|
215
|
+
| `/api/submissions` | POST | `swe submit` |
|
|
216
|
+
| `/api/mlebench/download` | POST | `mle download` |
|
|
217
|
+
| `/api/mlebench/submit` | POST | `mle submit` |
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
README.md
|
|
2
|
+
__init__.py
|
|
3
|
+
api.py
|
|
4
|
+
cli.py
|
|
5
|
+
config.py
|
|
6
|
+
git_ops.py
|
|
7
|
+
pyproject.toml
|
|
8
|
+
./__init__.py
|
|
9
|
+
./api.py
|
|
10
|
+
./cli.py
|
|
11
|
+
./config.py
|
|
12
|
+
./git_ops.py
|
|
13
|
+
aicodinggym_cli.egg-info/PKG-INFO
|
|
14
|
+
aicodinggym_cli.egg-info/SOURCES.txt
|
|
15
|
+
aicodinggym_cli.egg-info/dependency_links.txt
|
|
16
|
+
aicodinggym_cli.egg-info/entry_points.txt
|
|
17
|
+
aicodinggym_cli.egg-info/requires.txt
|
|
18
|
+
aicodinggym_cli.egg-info/top_level.txt
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
aicodinggym
|