astoncli 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- astoncli-0.1.0/PKG-INFO +883 -0
- astoncli-0.1.0/README.md +863 -0
- astoncli-0.1.0/aston/__init__.py +0 -0
- astoncli-0.1.0/aston/axios.py +525 -0
- astoncli-0.1.0/aston/config.py +88 -0
- astoncli-0.1.0/aston/docker.py +402 -0
- astoncli-0.1.0/aston/exceptions.py +49 -0
- astoncli-0.1.0/aston/ingestion.py +1105 -0
- astoncli-0.1.0/aston/llm.py +182 -0
- astoncli-0.1.0/aston/pipeline.py +749 -0
- astoncli-0.1.0/aston/poller.py +201 -0
- astoncli-0.1.0/aston/relevance.py +175 -0
- astoncli-0.1.0/aston/repo_verification.py +243 -0
- astoncli-0.1.0/aston/rule_validation.py +437 -0
- astoncli-0.1.0/aston/scanner.py +211 -0
- astoncli-0.1.0/aston/watcher.py +66 -0
- astoncli-0.1.0/astoncli.egg-info/PKG-INFO +883 -0
- astoncli-0.1.0/astoncli.egg-info/SOURCES.txt +22 -0
- astoncli-0.1.0/astoncli.egg-info/dependency_links.txt +1 -0
- astoncli-0.1.0/astoncli.egg-info/entry_points.txt +2 -0
- astoncli-0.1.0/astoncli.egg-info/requires.txt +13 -0
- astoncli-0.1.0/astoncli.egg-info/top_level.txt +1 -0
- astoncli-0.1.0/pyproject.toml +31 -0
- astoncli-0.1.0/setup.cfg +4 -0
astoncli-0.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,883 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: astoncli
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Automated Python repository migration CLI tool
|
|
5
|
+
Requires-Python: >=3.11
|
|
6
|
+
Description-Content-Type: text/markdown
|
|
7
|
+
Requires-Dist: PyYAML>=6.0
|
|
8
|
+
Requires-Dist: libcst>=1.0.0
|
|
9
|
+
Requires-Dist: packaging>=21.0
|
|
10
|
+
Requires-Dist: jedi>=0.18.0
|
|
11
|
+
Requires-Dist: argparse>=1.4.0
|
|
12
|
+
Requires-Dist: rich>=13.0.0
|
|
13
|
+
Requires-Dist: requests>=2.28.0
|
|
14
|
+
Requires-Dist: pydantic>=2.0.0
|
|
15
|
+
Requires-Dist: openai>=1.0.0
|
|
16
|
+
Requires-Dist: mistralai>=1.0.0
|
|
17
|
+
Requires-Dist: anthropic>=0.20.0
|
|
18
|
+
Requires-Dist: google-genai>=0.1.0
|
|
19
|
+
Requires-Dist: python-dotenv>=1.0.0
|
|
20
|
+
|
|
21
|
+
```python
|
|
22
|
+
█████╗ ███████╗████████╗ ██████╗ ███╗ ██╗
|
|
23
|
+
██╔══██╗██╔════╝╚══██╔══╝██╔═══██╗████╗ ██║
|
|
24
|
+
███████║███████╗ ██║ ██║ ██║██╔██╗ ██║
|
|
25
|
+
██╔══██║╚════██║ ██║ ██║ ██║██║╚██╗██║
|
|
26
|
+
██║ ██║███████║ ██║ ╚██████╔╝██║ ╚████║
|
|
27
|
+
╚═╝ ╚═╝╚══════╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝
|
|
28
|
+
|
|
29
|
+
██████╗██╗ ██╗
|
|
30
|
+
██╔════╝██║ ██║
|
|
31
|
+
██║ ██║ ██║
|
|
32
|
+
██║ ██║ ██║
|
|
33
|
+
╚██████╗███████╗██║
|
|
34
|
+
╚═════╝╚══════╝╚═╝
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
https://github.com/user-attachments/assets/812a1990-13c3-4c45-8a8e-080f602c7a3f
|
|
38
|
+
|
|
39
|
+
### Automated dependency migration, from breaking-change detection to a verified pull request.
|
|
40
|
+
|
|
41
|
+

|
|
42
|
+

|
|
43
|
+

|
|
44
|
+

|
|
45
|
+
|
|
46
|
+
**[Quick Start](#quick-start)** •
|
|
47
|
+
**[Live Demo](#aston-migration-demonstration)** •
|
|
48
|
+
**[How It Works](#how-it-works)** •
|
|
49
|
+
**[Commands](#command-reference)** •
|
|
50
|
+
**[Architecture](#project-architecture)** •
|
|
51
|
+
**[FAQ](#faq--troubleshooting)** •
|
|
52
|
+
**[Contributing](#contributing)**
|
|
53
|
+
|
|
54
|
+
*Making dependency migrations less manual.*
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## Table of Contents
|
|
59
|
+
|
|
60
|
+
<details>
|
|
61
|
+
<summary><strong>Click to expand</strong></summary>
|
|
62
|
+
|
|
63
|
+
1. The problem ASTon solves
|
|
64
|
+
2. Quick start
|
|
65
|
+
3. ASTon migration demonstration
|
|
66
|
+
4. How it works
|
|
67
|
+
5. The 10 stages in detail
|
|
68
|
+
6. Command reference
|
|
69
|
+
7. Run the complete workflow
|
|
70
|
+
8. Example migration
|
|
71
|
+
9. Project architecture
|
|
72
|
+
10. Generated files and directories
|
|
73
|
+
11. Installation
|
|
74
|
+
12. Design principles
|
|
75
|
+
13. Current scope & roadmap
|
|
76
|
+
14. FAQ & troubleshooting
|
|
77
|
+
15. Contributing
|
|
78
|
+
|
|
79
|
+
</details>
|
|
80
|
+
|
|
81
|
+
---
|
|
82
|
+
|
|
83
|
+
## The problem ASTon solves
|
|
84
|
+
|
|
85
|
+
Your project uses an older version of a Python library. A new version ships and some APIs change.
|
|
86
|
+
|
|
87
|
+
```python
|
|
88
|
+
# Old API
|
|
89
|
+
import openai
|
|
90
|
+
|
|
91
|
+
openai.ChatCompletion.create(...)
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
```python
|
|
95
|
+
# New API
|
|
96
|
+
from openai import OpenAI
|
|
97
|
+
|
|
98
|
+
client = OpenAI()
|
|
99
|
+
client.chat.completions.create(...)
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
Simply bumping the version can break your app. Traditionally you would have to:
|
|
103
|
+
|
|
104
|
+
1. Read release notes and migration guides
|
|
105
|
+
2. Find the breaking changes relevant to *your* code
|
|
106
|
+
3. Search your codebase for affected APIs
|
|
107
|
+
4. Rewrite the affected code
|
|
108
|
+
5. Run tests and debug failures
|
|
109
|
+
6. Commit and open a pull request
|
|
110
|
+
|
|
111
|
+
**ASTon automates that entire loop** and hands you a reviewable PR.
|
|
112
|
+
|
|
113
|
+
---
|
|
114
|
+
|
|
115
|
+
## Quick start
|
|
116
|
+
|
|
117
|
+
Three commands, from update detection to pull request:
|
|
118
|
+
|
|
119
|
+
```bash
|
|
120
|
+
aston watcher --path demo # 1. detect dependency updates
|
|
121
|
+
aston check --run --path demo # 2. run the migration pipeline
|
|
122
|
+
aston pr --path aston/succ_migrations --repo demo # 3. open the PR
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
> Not sure what's available? Run `aston --help` or `aston <command> --help` at any time.
|
|
126
|
+
|
|
127
|
+
<details>
|
|
128
|
+
<summary><strong>Prerequisites checklist</strong></summary>
|
|
129
|
+
|
|
130
|
+
- [ ] Python installed
|
|
131
|
+
- [ ] Docker installed and running
|
|
132
|
+
- [ ] Git installed
|
|
133
|
+
- [ ] GitHub access (for PR creation)
|
|
134
|
+
- [ ] API key for your configured LLM provider
|
|
135
|
+
- [ ] A local clone of the target repository (e.g. `demo/`)
|
|
136
|
+
|
|
137
|
+
</details>
|
|
138
|
+
|
|
139
|
+
---
|
|
140
|
+
|
|
141
|
+
## ASTon Migration Demonstration
|
|
142
|
+
|
|
143
|
+
This repository demonstrates ASTon's end-to-end dependency migration workflow using a Python application built with the legacy OpenAI Python SDK.
|
|
144
|
+
|
|
145
|
+
ASTon analyzed the repository, identified deprecated API usage across multiple Python modules, generated migration changes, verified the updated code, and submitted a GitHub pull request that was subsequently merged.
|
|
146
|
+
|
|
147
|
+
The demo recording is at the top of this README.
|
|
148
|
+
|
|
149
|
+
### Migration details
|
|
150
|
+
|
|
151
|
+
- **Source dependency:** `openai==0.28.1`
|
|
152
|
+
- **Target dependency:** `openai==3.41.0`
|
|
153
|
+
- **Migration scope:** Multiple Python modules
|
|
154
|
+
- **Verification:** Docker-based verification and repository-wide tests
|
|
155
|
+
|
|
156
|
+
### Migration evidence
|
|
157
|
+
|
|
158
|
+
| Artifact | Link |
|
|
159
|
+
|---|---|
|
|
160
|
+
| Original legacy code | [Initial commit](https://github.com/2022n02511-stack/demo_aston/commit/10cce48d300d828c774e56ed2e72066c2e993b33) |
|
|
161
|
+
| ASTon-generated changes | [Migration commit](https://github.com/2022n02511-stack/demo_aston/commit/66547808b4b4868a6684718c2d095928a5b6605a) |
|
|
162
|
+
| Merged migration | [Merge commit](https://github.com/2022n02511-stack/demo_aston/commit/caaea1080b9c7e603c6917fa0fd16c77ee93c8a2) |
|
|
163
|
+
| Complete code comparison | [View before-and-after diff](https://github.com/2022n02511-stack/demo_aston/compare/10cce48d300d828c774e56ed2e72066c2e993b33...caaea1080b9c7e603c6917fa0fd16c77ee93c8a2) |
|
|
164
|
+
| ASTon source code | [GitHub repository](https://github.com/2022n02511-stack/ASTon) |
|
|
165
|
+
|
|
166
|
+
### What this demonstrates
|
|
167
|
+
|
|
168
|
+
1. Breaking-change discovery from dependency documentation.
|
|
169
|
+
2. Identification of affected code across multiple modules.
|
|
170
|
+
3. Automated migration rule generation and code modification.
|
|
171
|
+
4. Docker-based verification and repository-wide testing.
|
|
172
|
+
5. GitHub pull request creation and merging.
|
|
173
|
+
|
|
174
|
+
<details>
|
|
175
|
+
<summary><strong>Reproduce this demo</strong></summary>
|
|
176
|
+
|
|
177
|
+
<br>
|
|
178
|
+
|
|
179
|
+
Clone the demo project next to ASTon and run the same three commands from the quick start:
|
|
180
|
+
|
|
181
|
+
```bash
|
|
182
|
+
git clone https://github.com/2022n02511-stack/demo_aston.git demo
|
|
183
|
+
|
|
184
|
+
aston watcher --path demo
|
|
185
|
+
aston check --run --path demo
|
|
186
|
+
aston pr --path aston/succ_migrations --repo demo
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
> The demo project starts on the legacy `openai==0.28.1` code, so it shows the same before-and-after the links above describe.
|
|
190
|
+
|
|
191
|
+
</details>
|
|
192
|
+
|
|
193
|
+
**Note:** This is a demonstration project designed to showcase ASTon's migration capabilities, not a production customer deployment.
|
|
194
|
+
|
|
195
|
+
---
|
|
196
|
+
|
|
197
|
+
## How it works
|
|
198
|
+
|
|
199
|
+
### The 30-second view
|
|
200
|
+
|
|
201
|
+
```text
|
|
202
|
+
┌──────────────────────────────────────────────────────────────────────────┐
|
|
203
|
+
│ ASTon MIGRATION PIPELINE │
|
|
204
|
+
└──────────────────────────────────────────────────────────────────────────┘
|
|
205
|
+
|
|
206
|
+
① WATCH ② INGEST ③ EXTRACT ④ SCAN
|
|
207
|
+
┌─────────┐ ┌───────────┐ ┌────────────┐ ┌────────────┐
|
|
208
|
+
│ new pkg │───▶│ changelog │───▶│ LLM builds │───▶│ AST usage │
|
|
209
|
+
│ version │ │ + guides │ │ rules.json │ │ map of repo│
|
|
210
|
+
└─────────┘ └───────────┘ └────────────┘ └─────┬──────┘
|
|
211
|
+
│
|
|
212
|
+
⑧ VERIFY ⑦ GENERATE ⑥ VALIDATE ⑤ RELEVANCE
|
|
213
|
+
┌──────────┐ ┌────────────┐ ┌────────────┐ ┌──────┴──────┐
|
|
214
|
+
│ Docker │◀──│ candidate │◀──│ rules vs │◀──│ which rules │
|
|
215
|
+
│ per file │ │ migrations │ │ target ver │ │ hit my code?│
|
|
216
|
+
└────┬─────┘ └────────────┘ └────────────┘ └─────────────┘
|
|
217
|
+
│
|
|
218
|
+
├── fail ──▶ REPAIR ──▶ (back to ⑧)
|
|
219
|
+
│
|
|
220
|
+
└── pass
|
|
221
|
+
│
|
|
222
|
+
▼
|
|
223
|
+
⑨ FULL-REPO VERIFY ⑩ DELIVER
|
|
224
|
+
┌─────────────────────┐ ┌──────────────────────┐
|
|
225
|
+
│ syntax + test cmd │──pass─▶│ branch → commit → │──▶ Human review
|
|
226
|
+
│ on combined project │ │ push → GitHub PR │ & merge
|
|
227
|
+
└──────────┬──────────┘ └──────────────────────┘
|
|
228
|
+
│
|
|
229
|
+
fail
|
|
230
|
+
▼
|
|
231
|
+
Verification report
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
### The full workflow (Mermaid)
|
|
235
|
+
|
|
236
|
+
<details open>
|
|
237
|
+
<summary><strong>View the complete migration workflow</strong></summary>
|
|
238
|
+
|
|
239
|
+
```mermaid
|
|
240
|
+
flowchart TD
|
|
241
|
+
A["Start ASTon"] --> B["Watch for dependency updates"]
|
|
242
|
+
B --> C["Discover new package version"]
|
|
243
|
+
C --> D["Collect release notes and migration documentation"]
|
|
244
|
+
D --> E["Extract migration rules"]
|
|
245
|
+
E --> F["Build repository usage map"]
|
|
246
|
+
F --> G["Identify relevant breaking changes"]
|
|
247
|
+
G --> H["Validate migration rules"]
|
|
248
|
+
H --> I["Generate code migrations"]
|
|
249
|
+
I --> J["Verify individual files in Docker"]
|
|
250
|
+
J --> K{"Migration successful?"}
|
|
251
|
+
K -->|No| L["Repair failed migration"]
|
|
252
|
+
L --> J
|
|
253
|
+
K -->|Yes| M["Save migrated files"]
|
|
254
|
+
M --> N["Verify complete repository"]
|
|
255
|
+
N --> O{"Repository verification passed?"}
|
|
256
|
+
O -->|No| P["Report verification failure"]
|
|
257
|
+
O -->|Yes| Q["Create GitHub pull request"]
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
</details>
|
|
261
|
+
|
|
262
|
+
### Which CLI command runs which stages?
|
|
263
|
+
|
|
264
|
+
| Command | Stages it drives | Output |
|
|
265
|
+
|---|---|---|
|
|
266
|
+
| `aston watcher` | ① Dependency monitoring | Detected updates, version state |
|
|
267
|
+
| `aston check --run` | ②–⑨ Ingest → extract → scan → relevance → validate → generate → verify → repair → full-repo verify | `succ_migrations/`, `final_reports/` |
|
|
268
|
+
| `aston pr` | ⑩ Pull request creation | A GitHub PR for review |
|
|
269
|
+
|
|
270
|
+
```text
|
|
271
|
+
aston watcher ───▶ aston check --run ───▶ aston pr
|
|
272
|
+
(stage 1) (stages 2 – 9) (stage 10)
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
---
|
|
276
|
+
|
|
277
|
+
## The 10 stages in detail
|
|
278
|
+
|
|
279
|
+
> Click any stage to expand it.
|
|
280
|
+
|
|
281
|
+
<details>
|
|
282
|
+
<summary><strong>① Dependency monitoring</strong> — find out when a dependency has a newer version</summary>
|
|
283
|
+
|
|
284
|
+
<br>
|
|
285
|
+
|
|
286
|
+
ASTon includes a watcher that monitors dependencies and identifies available updates.
|
|
287
|
+
|
|
288
|
+
```text
|
|
289
|
+
Current dependency: openai==0.28.1
|
|
290
|
+
Available version: 3.41.0
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
It also keeps version-related state so previously processed updates can be tracked.
|
|
294
|
+
|
|
295
|
+
**Run it:**
|
|
296
|
+
|
|
297
|
+
```bash
|
|
298
|
+
aston watcher --path demo
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
</details>
|
|
302
|
+
|
|
303
|
+
<details>
|
|
304
|
+
<summary><strong>② Change discovery & documentation ingestion</strong> — understand what changed</summary>
|
|
305
|
+
|
|
306
|
+
<br>
|
|
307
|
+
|
|
308
|
+
When an update is detected, ASTon collects:
|
|
309
|
+
|
|
310
|
+
- Release notes
|
|
311
|
+
- Changelogs
|
|
312
|
+
- Migration guides
|
|
313
|
+
- Relevant documentation links
|
|
314
|
+
- Breaking-change discussions and references
|
|
315
|
+
|
|
316
|
+
It follows relevant links to gather extra migration context, so extraction doesn't rely on version numbers alone.
|
|
317
|
+
|
|
318
|
+
```mermaid
|
|
319
|
+
flowchart TD
|
|
320
|
+
A["New package version"] --> B["Find release information"]
|
|
321
|
+
B --> C["Collect changelog"]
|
|
322
|
+
C --> D["Find migration documentation"]
|
|
323
|
+
D --> E["Follow relevant documentation links"]
|
|
324
|
+
E --> F["Combine collected information"]
|
|
325
|
+
F --> G["Documentation for rule extraction"]
|
|
326
|
+
```
|
|
327
|
+
|
|
328
|
+
</details>
|
|
329
|
+
|
|
330
|
+
<details>
|
|
331
|
+
<summary><strong>③ Migration rule extraction</strong> — turn docs into structured rules</summary>
|
|
332
|
+
|
|
333
|
+
<br>
|
|
334
|
+
|
|
335
|
+
Documentation is written for humans. ASTon uses an LLM to convert it into structured migration rules:
|
|
336
|
+
|
|
337
|
+
```json
|
|
338
|
+
{
|
|
339
|
+
"package": "openai",
|
|
340
|
+
"old_api": "openai.ChatCompletion.create",
|
|
341
|
+
"new_api": "client.chat.completions.create",
|
|
342
|
+
"migration_type": "rename"
|
|
343
|
+
}
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
Rules can describe:
|
|
347
|
+
|
|
348
|
+
- API renames
|
|
349
|
+
- Removed APIs
|
|
350
|
+
- Configuration changes
|
|
351
|
+
- New instantiation requirements
|
|
352
|
+
- Changes in how an API is accessed
|
|
353
|
+
|
|
354
|
+
> Extracted rules are **not** automatically assumed to be applicable. They are evaluated against the target version first.
|
|
355
|
+
|
|
356
|
+
</details>
|
|
357
|
+
|
|
358
|
+
<details>
|
|
359
|
+
<summary><strong>④ Repository scanning & usage mapping</strong> — see how your project really uses the library</summary>
|
|
360
|
+
|
|
361
|
+
<br>
|
|
362
|
+
|
|
363
|
+
ASTon uses Python's Abstract Syntax Tree (AST) instead of plain text search.
|
|
364
|
+
|
|
365
|
+
```python
|
|
366
|
+
from openai import OpenAI as Client
|
|
367
|
+
|
|
368
|
+
client = Client()
|
|
369
|
+
|
|
370
|
+
response = client.chat.completions.create(
|
|
371
|
+
model="example-model",
|
|
372
|
+
messages=[]
|
|
373
|
+
)
|
|
374
|
+
```
|
|
375
|
+
|
|
376
|
+
The scanner captures: imported package, imported symbol, local alias, instantiated objects, API usage, source line, and relevant keyword arguments.
|
|
377
|
+
|
|
378
|
+
```mermaid
|
|
379
|
+
flowchart TD
|
|
380
|
+
A["Python source files"] --> B["Python AST parser"]
|
|
381
|
+
B --> C["Inspect imports"]
|
|
382
|
+
B --> D["Inspect function calls"]
|
|
383
|
+
B --> E["Inspect object instantiation"]
|
|
384
|
+
C --> F["Usage map"]
|
|
385
|
+
D --> F
|
|
386
|
+
E --> F
|
|
387
|
+
F --> G["Repository dependency usage"]
|
|
388
|
+
```
|
|
389
|
+
|
|
390
|
+
</details>
|
|
391
|
+
|
|
392
|
+
<details>
|
|
393
|
+
<summary><strong>⑤ Change relevance analysis</strong> — keep only the rules that matter to you</summary>
|
|
394
|
+
|
|
395
|
+
<br>
|
|
396
|
+
|
|
397
|
+
A library can introduce hundreds of changes. Most won't touch your code.
|
|
398
|
+
|
|
399
|
+
| Extracted change | Used in project? |
|
|
400
|
+
|---|---|
|
|
401
|
+
| `ChatCompletion.create` changed | Yes |
|
|
402
|
+
| `Embedding.create` changed | Yes |
|
|
403
|
+
| An unrelated API was removed | No |
|
|
404
|
+
|
|
405
|
+
The result is a smaller set of potentially relevant migration rules.
|
|
406
|
+
|
|
407
|
+
</details>
|
|
408
|
+
|
|
409
|
+
<details>
|
|
410
|
+
<summary><strong>⑥ Migration rule validation</strong> — don't apply stale rules</summary>
|
|
411
|
+
|
|
412
|
+
<br>
|
|
413
|
+
|
|
414
|
+
A rule that was valid for an earlier upgrade may not be valid for a later one.
|
|
415
|
+
|
|
416
|
+
```text
|
|
417
|
+
Version 0 → Version 1 API A → API B
|
|
418
|
+
Version 1 → Version 2 API B → API C
|
|
419
|
+
```
|
|
420
|
+
|
|
421
|
+
Applying the first rule directly to a project targeting Version 2 would leave it on an outdated API. ASTon validates rules against the **target version** before code generation.
|
|
422
|
+
|
|
423
|
+
Rules that can't be validated are **not** silently treated as confirmed. They remain unresolved for further analysis.
|
|
424
|
+
|
|
425
|
+
</details>
|
|
426
|
+
|
|
427
|
+
<details>
|
|
428
|
+
<summary><strong>⑦ Code migration generation</strong> — produce candidate code</summary>
|
|
429
|
+
|
|
430
|
+
<br>
|
|
431
|
+
|
|
432
|
+
```python
|
|
433
|
+
# Before
|
|
434
|
+
import openai
|
|
435
|
+
|
|
436
|
+
response = openai.ChatCompletion.create(...)
|
|
437
|
+
```
|
|
438
|
+
|
|
439
|
+
```python
|
|
440
|
+
# After
|
|
441
|
+
from openai import OpenAI
|
|
442
|
+
|
|
443
|
+
client = OpenAI()
|
|
444
|
+
response = client.chat.completions.create(...)
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
The output is a **candidate**, not an accepted result. It must pass verification.
|
|
448
|
+
|
|
449
|
+
</details>
|
|
450
|
+
|
|
451
|
+
<details>
|
|
452
|
+
<summary><strong>⑧ Individual file verification & repair</strong> — Docker-isolated checks</summary>
|
|
453
|
+
|
|
454
|
+
<br>
|
|
455
|
+
|
|
456
|
+
Each migrated file is verified inside a Docker container with the target dependency installed. Verification distinguishes between:
|
|
457
|
+
|
|
458
|
+
- Successful migrations
|
|
459
|
+
- Migration errors
|
|
460
|
+
- Environment / dependency installation errors
|
|
461
|
+
- External service errors
|
|
462
|
+
- Timeouts
|
|
463
|
+
|
|
464
|
+
```mermaid
|
|
465
|
+
flowchart TD
|
|
466
|
+
A["Generated candidate"] --> B["Create Docker environment"]
|
|
467
|
+
B --> C["Install target requirements"]
|
|
468
|
+
C --> D["Run verification"]
|
|
469
|
+
D --> E{"Passed?"}
|
|
470
|
+
E -->|Yes| F["Save successful migration"]
|
|
471
|
+
E -->|No| G["Analyze verification error"]
|
|
472
|
+
G --> H["Generate repaired candidate"]
|
|
473
|
+
H --> B
|
|
474
|
+
```
|
|
475
|
+
|
|
476
|
+
**Repair process:** the failed candidate plus its real verification feedback are used to attempt a *better* migration, not just repeat the same replacement. Successful files are saved to the migration output directory.
|
|
477
|
+
|
|
478
|
+
</details>
|
|
479
|
+
|
|
480
|
+
<details>
|
|
481
|
+
<summary><strong>⑨ Full-repository verification</strong> — make sure it all works together</summary>
|
|
482
|
+
|
|
483
|
+
<br>
|
|
484
|
+
|
|
485
|
+
Passing per-file checks doesn't guarantee the whole project works (one migrated file may depend on another).
|
|
486
|
+
|
|
487
|
+
The verification environment combines:
|
|
488
|
+
|
|
489
|
+
- Original repository files that were not migrated
|
|
490
|
+
- Successfully migrated files
|
|
491
|
+
- Updated target requirements
|
|
492
|
+
|
|
493
|
+
Then it runs Python syntax checks and the configured test command.
|
|
494
|
+
|
|
495
|
+
```mermaid
|
|
496
|
+
flowchart TD
|
|
497
|
+
A["Original repository"] --> C["Build verification copy"]
|
|
498
|
+
B["Successful migrated files"] --> C
|
|
499
|
+
D["Target requirements"] --> C
|
|
500
|
+
C --> E["Install dependencies"]
|
|
501
|
+
E --> F["Check Python syntax"]
|
|
502
|
+
F --> G["Run repository tests"]
|
|
503
|
+
G --> H{"Passed?"}
|
|
504
|
+
H -->|Yes| I["Ready for pull request"]
|
|
505
|
+
H -->|No| J["Report verification failure"]
|
|
506
|
+
```
|
|
507
|
+
|
|
508
|
+
</details>
|
|
509
|
+
|
|
510
|
+
<details>
|
|
511
|
+
<summary><strong>⑩ GitHub pull request creation</strong> — reviewable delivery</summary>
|
|
512
|
+
|
|
513
|
+
<br>
|
|
514
|
+
|
|
515
|
+
The PR contains the migrated files and updated dependency requirements. The original source repository is kept separate from ASTon's output during migration, so you stay in control.
|
|
516
|
+
|
|
517
|
+
```mermaid
|
|
518
|
+
flowchart TD
|
|
519
|
+
A["Verified migration output"] --> B["Prepare migration branch"]
|
|
520
|
+
B --> C["Copy migrated files"]
|
|
521
|
+
C --> D["Commit changes"]
|
|
522
|
+
D --> E["Push branch"]
|
|
523
|
+
E --> F["Create GitHub pull request"]
|
|
524
|
+
F --> G["Developer reviews changes"]
|
|
525
|
+
G --> H["Merge when approved"]
|
|
526
|
+
```
|
|
527
|
+
|
|
528
|
+
</details>
|
|
529
|
+
|
|
530
|
+
---
|
|
531
|
+
|
|
532
|
+
## Command reference
|
|
533
|
+
|
|
534
|
+
### 1. Check available commands
|
|
535
|
+
|
|
536
|
+
```bash
|
|
537
|
+
aston --help
|
|
538
|
+
```
|
|
539
|
+
|
|
540
|
+
### 2. Start the dependency watcher
|
|
541
|
+
|
|
542
|
+
```bash
|
|
543
|
+
aston watcher --path demo
|
|
544
|
+
```
|
|
545
|
+
|
|
546
|
+
View watcher-specific options:
|
|
547
|
+
|
|
548
|
+
```bash
|
|
549
|
+
aston watcher --help
|
|
550
|
+
```
|
|
551
|
+
|
|
552
|
+
### 3. Run the migration pipeline
|
|
553
|
+
|
|
554
|
+
```bash
|
|
555
|
+
aston check --run --path demo
|
|
556
|
+
```
|
|
557
|
+
|
|
558
|
+
View migration command options:
|
|
559
|
+
|
|
560
|
+
```bash
|
|
561
|
+
aston check --help
|
|
562
|
+
```
|
|
563
|
+
|
|
564
|
+
### 4. Create a GitHub pull request
|
|
565
|
+
|
|
566
|
+
```bash
|
|
567
|
+
aston pr \
|
|
568
|
+
--path aston/succ_migrations \
|
|
569
|
+
--repo demo \
|
|
570
|
+
--branch demo_aston \
|
|
571
|
+
--title "Migrate OpenAI dependency" \
|
|
572
|
+
--body "Automated dependency migration generated and verified by ASTon."
|
|
573
|
+
```
|
|
574
|
+
|
|
575
|
+
View PR command options:
|
|
576
|
+
|
|
577
|
+
```bash
|
|
578
|
+
aston pr --help
|
|
579
|
+
```
|
|
580
|
+
|
|
581
|
+
<details>
|
|
582
|
+
<summary><strong>What do the flags mean?</strong></summary>
|
|
583
|
+
|
|
584
|
+
<br>
|
|
585
|
+
|
|
586
|
+
| Command | Flag | Meaning |
|
|
587
|
+
|---|---|---|
|
|
588
|
+
| `aston watcher` | `--path` | Path to the project whose dependencies are monitored |
|
|
589
|
+
| `aston check` | `--run` | Execute the migration pipeline |
|
|
590
|
+
| `aston check` | `--path` | Path to the project to analyze and migrate |
|
|
591
|
+
| `aston pr` | `--path` | ASTon's successful migration output (e.g. `aston/succ_migrations`) |
|
|
592
|
+
| `aston pr` | `--repo` | Local clone of the target GitHub repository |
|
|
593
|
+
| `aston pr` | `--branch` | Name of the migration branch (optional) |
|
|
594
|
+
| `aston pr` | `--title` | Pull request title |
|
|
595
|
+
| `aston pr` | `--body` | Pull request description |
|
|
596
|
+
|
|
597
|
+
> Options can vary between versions. Always confirm with `aston <command> --help`.
|
|
598
|
+
|
|
599
|
+
</details>
|
|
600
|
+
|
|
601
|
+
---
|
|
602
|
+
|
|
603
|
+
## Run the complete workflow
|
|
604
|
+
|
|
605
|
+
Chain everything in one go. Each command runs **only if the previous one succeeds** (`&&`).
|
|
606
|
+
|
|
607
|
+
```bash
|
|
608
|
+
aston watcher --path demo && \
|
|
609
|
+
aston check --run --path demo && \
|
|
610
|
+
aston pr \
|
|
611
|
+
--path aston/succ_migrations \
|
|
612
|
+
--repo demo \
|
|
613
|
+
--branch demo_aston \
|
|
614
|
+
--title "Migrate OpenAI dependency" \
|
|
615
|
+
--body "Automated dependency migration generated and verified by ASTon."
|
|
616
|
+
```
|
|
617
|
+
|
|
618
|
+
```text
|
|
619
|
+
┌──────────────┐ ok ┌──────────────────┐ ok ┌────────────┐
|
|
620
|
+
│ aston watcher│──────▶│ aston check --run│──────▶│ aston pr │──▶ PR opened
|
|
621
|
+
└──────┬───────┘ └────────┬─────────┘ └─────┬──────┘
|
|
622
|
+
│ fail │ fail │ fail
|
|
623
|
+
▼ ▼ ▼
|
|
624
|
+
stop stop stop
|
|
625
|
+
```
|
|
626
|
+
|
|
627
|
+
---
|
|
628
|
+
|
|
629
|
+
## Example migration
|
|
630
|
+
|
|
631
|
+
This is the migration shown in the [demonstration above](#aston-migration-demonstration).
|
|
632
|
+
|
|
633
|
+
Your project uses:
|
|
634
|
+
|
|
635
|
+
```text
|
|
636
|
+
openai==0.28.1
|
|
637
|
+
```
|
|
638
|
+
|
|
639
|
+
You want to move to:
|
|
640
|
+
|
|
641
|
+
```text
|
|
642
|
+
openai==3.41.0
|
|
643
|
+
```
|
|
644
|
+
|
|
645
|
+
```mermaid
|
|
646
|
+
flowchart TD
|
|
647
|
+
A["openai 0.28.1"] --> B["Discover target 3.41.0"]
|
|
648
|
+
B --> C["Collect migration documentation"]
|
|
649
|
+
C --> D["Extract API migration rules"]
|
|
650
|
+
D --> E["Scan project for OpenAI usage"]
|
|
651
|
+
E --> F["Identify relevant rules"]
|
|
652
|
+
F --> G["Validate rules for 3.41.0"]
|
|
653
|
+
G --> H["Generate migrated Python files"]
|
|
654
|
+
H --> I["Verify and repair"]
|
|
655
|
+
I --> J["Verify complete repository"]
|
|
656
|
+
J --> K["Create pull request"]
|
|
657
|
+
```
|
|
658
|
+
|
|
659
|
+
Instead of manually hunting down every affected API, you receive a **pull request with the proposed changes**.
|
|
660
|
+
|
|
661
|
+
---
|
|
662
|
+
|
|
663
|
+
## Project architecture
|
|
664
|
+
|
|
665
|
+
<details>
|
|
666
|
+
<summary><strong>Main modules</strong></summary>
|
|
667
|
+
|
|
668
|
+
<br>
|
|
669
|
+
|
|
670
|
+
| Module | Responsibility |
|
|
671
|
+
|---|---|
|
|
672
|
+
| `watcher.py` | Monitors dependency updates |
|
|
673
|
+
| `poller.py` | Supports checking for available updates |
|
|
674
|
+
| `ingestion.py` | Collects release and migration documentation |
|
|
675
|
+
| `changes/chain.py` | Handles migration boundaries across versions |
|
|
676
|
+
| `axios.py` | Code-generation and related integration logic |
|
|
677
|
+
| `scanner.py` | Analyzes Python source code and dependency usage |
|
|
678
|
+
| `relevance.py` | Evaluates whether changes affect the repository |
|
|
679
|
+
| `rule_validation.py` | Validates migration rules against the target |
|
|
680
|
+
| `pipeline.py` | Coordinates migration and repair operations |
|
|
681
|
+
| `docker.py` | Verifies individual migrated files |
|
|
682
|
+
| `repo_verification.py` | Verifies the complete migrated repository |
|
|
683
|
+
| `repo_index/repo_map.py` | Maps the repository and coordinates repository migration |
|
|
684
|
+
| `github_integrations/pr.py` | Creates GitHub pull requests |
|
|
685
|
+
| `cli/main.py` | Defines the command-line interface |
|
|
686
|
+
| `cli/console.py` | Provides CLI messages and output |
|
|
687
|
+
| `cli_ui/rich.py` | Provides the Rich terminal interface |
|
|
688
|
+
|
|
689
|
+
</details>
|
|
690
|
+
|
|
691
|
+
<details>
|
|
692
|
+
<summary><strong>Module → pipeline map</strong></summary>
|
|
693
|
+
|
|
694
|
+
<br>
|
|
695
|
+
|
|
696
|
+
```text
|
|
697
|
+
WATCH watcher.py ── poller.py
|
|
698
|
+
│
|
|
699
|
+
INGEST ingestion.py ── changes/chain.py
|
|
700
|
+
│
|
|
701
|
+
EXTRACT axios.py (LLM) ──▶ rules.json
|
|
702
|
+
│
|
|
703
|
+
SCAN scanner.py ──▶ repo_index/repo_map.py ──▶ repo_index.json
|
|
704
|
+
│
|
|
705
|
+
FILTER relevance.py ── rule_validation.py
|
|
706
|
+
│
|
|
707
|
+
MIGRATE pipeline.py ◀──▶ docker.py (generate ⇄ verify ⇄ repair)
|
|
708
|
+
│
|
|
709
|
+
VERIFY repo_verification.py
|
|
710
|
+
│
|
|
711
|
+
DELIVER github_integrations/pr.py
|
|
712
|
+
|
|
713
|
+
INTERFACE cli/main.py ── cli/console.py ── cli_ui/rich.py
|
|
714
|
+
```
|
|
715
|
+
|
|
716
|
+
</details>
|
|
717
|
+
|
|
718
|
+
---
|
|
719
|
+
|
|
720
|
+
## Generated files and directories
|
|
721
|
+
|
|
722
|
+
| Path | Purpose |
|
|
723
|
+
|---|---|
|
|
724
|
+
| `artifacts/runs/` | Individual migration run artifacts and attempts |
|
|
725
|
+
| `succ_migrations/` | Successfully migrated files and target requirements |
|
|
726
|
+
| `final_reports/` | Migration analysis and result reports |
|
|
727
|
+
| `rule_cache/` | Cached migration-rule information |
|
|
728
|
+
| `repo_index.json` | Repository indexing information |
|
|
729
|
+
| `migrations.json` | Migration-related information |
|
|
730
|
+
| `changes.json` | Collected change information |
|
|
731
|
+
| `rules.json` | Extracted migration rules |
|
|
732
|
+
| `temp_requirements.txt` | Temporary target requirements used during migration |
|
|
733
|
+
|
|
734
|
+
---
|
|
735
|
+
|
|
736
|
+
## Installation
|
|
737
|
+
|
|
738
|
+
```bash
|
|
739
|
+
# 1. Clone
|
|
740
|
+
git clone https://github.com/2022n02511-stack/ASTon.git
|
|
741
|
+
cd ASTon
|
|
742
|
+
|
|
743
|
+
# 2. Install the project
|
|
744
|
+
pip install -e .
|
|
745
|
+
|
|
746
|
+
# 3. Install dependencies if needed
|
|
747
|
+
pip install -r requirements.txt
|
|
748
|
+
|
|
749
|
+
# 4. Verify the CLI works
|
|
750
|
+
aston --help
|
|
751
|
+
```
|
|
752
|
+
|
|
753
|
+
Then configure the environment variables required by your selected **LLM provider** and **GitHub integration**. A template is provided in `.env.example`.
|
|
754
|
+
|
|
755
|
+
---
|
|
756
|
+
|
|
757
|
+
## Design principles
|
|
758
|
+
|
|
759
|
+
| # | Principle | Idea |
|
|
760
|
+
|---|---|---|
|
|
761
|
+
| 1 | **Documentation-driven migration** | Decisions come from real release notes and migration docs |
|
|
762
|
+
| 2 | **Repository-aware changes** | Only changes relevant to your usage are considered |
|
|
763
|
+
| 3 | **Version-aware validation** | Rules must be compatible with the requested target version |
|
|
764
|
+
| 4 | **Verification before acceptance** | Generated code is tested, not trusted because it looks right |
|
|
765
|
+
| 5 | **Isolated execution** | Docker isolates dependency installs and verification |
|
|
766
|
+
| 6 | **Reviewable delivery** | Migrations arrive as PRs so you control what merges |
|
|
767
|
+
|
|
768
|
+
---
|
|
769
|
+
|
|
770
|
+
## Current scope & roadmap
|
|
771
|
+
|
|
772
|
+
ASTon is under **active development**. The current implementation focuses on **Python** dependency migrations, with the **OpenAI Python SDK** as the test case demonstrated above. It has been run end to end on one demo project so far.
|
|
773
|
+
|
|
774
|
+
The architecture separates ingestion, rule extraction, repository analysis, code generation, verification, and delivery, so more packages can be supported without tying the system to one library.
|
|
775
|
+
|
|
776
|
+
- [x] Dependency watching
|
|
777
|
+
- [x] Documentation ingestion & rule extraction
|
|
778
|
+
- [x] AST-based repository usage mapping
|
|
779
|
+
- [x] Docker-isolated per-file verification with repair
|
|
780
|
+
- [x] Full-repository verification
|
|
781
|
+
- [x] GitHub pull request creation
|
|
782
|
+
- [ ] Version-aware rule validation across more version chains (in progress)
|
|
783
|
+
- [ ] Reliability across more real-world repositories
|
|
784
|
+
- [ ] Support for additional Python packages
|
|
785
|
+
- [ ] `pip install` distribution
|
|
786
|
+
- [ ] Broader language ecosystem support
|
|
787
|
+
|
|
788
|
+
---
|
|
789
|
+
|
|
790
|
+
## FAQ & troubleshooting
|
|
791
|
+
|
|
792
|
+
<details>
|
|
793
|
+
<summary><strong>Does ASTon change my repository directly?</strong></summary>
|
|
794
|
+
|
|
795
|
+
<br>
|
|
796
|
+
|
|
797
|
+
No. Migration output is kept separate (in `succ_migrations/`) and only reaches your repo as a pull request that you review and merge.
|
|
798
|
+
|
|
799
|
+
</details>
|
|
800
|
+
|
|
801
|
+
<details>
|
|
802
|
+
<summary><strong>Why is a rule "unresolved"?</strong></summary>
|
|
803
|
+
|
|
804
|
+
<br>
|
|
805
|
+
|
|
806
|
+
Rules that can't be validated against the target version are never silently treated as confirmed. They stay unresolved so they can be analyzed further instead of risking an incorrect migration.
|
|
807
|
+
|
|
808
|
+
</details>
|
|
809
|
+
|
|
810
|
+
<details>
|
|
811
|
+
<summary><strong>A file failed verification. What now?</strong></summary>
|
|
812
|
+
|
|
813
|
+
<br>
|
|
814
|
+
|
|
815
|
+
ASTon sends it through the repair loop, using the real failure output to generate a corrected candidate. Check `artifacts/runs/` for each attempt and `final_reports/` for the result.
|
|
816
|
+
|
|
817
|
+
</details>
|
|
818
|
+
|
|
819
|
+
<details>
|
|
820
|
+
<summary><strong>Verification failed on dependency installation or timeout.</strong></summary>
|
|
821
|
+
|
|
822
|
+
<br>
|
|
823
|
+
|
|
824
|
+
ASTon separates environment, external-service, and timeout failures from true migration errors. Confirm that Docker is running and has network access, then re-run `aston check --run --path demo`.
|
|
825
|
+
|
|
826
|
+
</details>
|
|
827
|
+
|
|
828
|
+
<details>
|
|
829
|
+
<summary><strong>The full-repository check failed even though every file passed.</strong></summary>
|
|
830
|
+
|
|
831
|
+
<br>
|
|
832
|
+
|
|
833
|
+
Individual success doesn't guarantee cross-file compatibility. Review the verification report in `final_reports/` to see whether syntax checks or your configured test command failed.
|
|
834
|
+
|
|
835
|
+
</details>
|
|
836
|
+
|
|
837
|
+
<details>
|
|
838
|
+
<summary><strong>The <code>aston pr</code> command can't push or create the PR.</strong></summary>
|
|
839
|
+
|
|
840
|
+
<br>
|
|
841
|
+
|
|
842
|
+
Check that:
|
|
843
|
+
|
|
844
|
+
- [ ] `--repo` points to a valid local clone of the target repository
|
|
845
|
+
- [ ] Your GitHub credentials/environment variables are configured
|
|
846
|
+
- [ ] `--path` points to `aston/succ_migrations` (the successful migration output)
|
|
847
|
+
- [ ] You have push access to the repository
|
|
848
|
+
|
|
849
|
+
</details>
|
|
850
|
+
|
|
851
|
+
<details>
|
|
852
|
+
<summary><strong>A command or flag isn't recognized.</strong></summary>
|
|
853
|
+
|
|
854
|
+
<br>
|
|
855
|
+
|
|
856
|
+
Flags can differ between versions. Run `aston --help` and `aston <command> --help` to see what your installed version supports.
|
|
857
|
+
|
|
858
|
+
</details>
|
|
859
|
+
|
|
860
|
+
---
|
|
861
|
+
|
|
862
|
+
## Contributing
|
|
863
|
+
|
|
864
|
+
Contributions, feedback, and ideas are welcome!
|
|
865
|
+
|
|
866
|
+
If you hit a migration issue, please include:
|
|
867
|
+
|
|
868
|
+
- [ ] The dependency and source version
|
|
869
|
+
- [ ] The target version
|
|
870
|
+
- [ ] The affected code
|
|
871
|
+
- [ ] The migration report or verification output
|
|
872
|
+
|
|
873
|
+
These details make problems far easier to reproduce and investigate.
|
|
874
|
+
|
|
875
|
+
---
|
|
876
|
+
|
|
877
|
+
<div align="center">
|
|
878
|
+
|
|
879
|
+
**ASTon — Making dependency migrations less manual.**
|
|
880
|
+
|
|
881
|
+
If ASTon saves you time, consider starring the repo.
|
|
882
|
+
|
|
883
|
+
</div>
|