aiopt 0.2.3 → 0.2.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +68 -71
- package/dist/cli.js +496 -26
- package/dist/cli.js.map +1 -1
- package/package.json +5 -3
package/README.md
CHANGED
|
@@ -1,55 +1,73 @@
|
|
|
1
|
-
# AIOpt —
|
|
1
|
+
# AIOpt — pre-deploy LLM cost accident guardrail (local-only)
|
|
2
2
|
|
|
3
3
|
[](https://www.npmjs.com/package/aiopt)
|
|
4
4
|
[](https://github.com/tkddlr0716-collab/aiopt/actions/workflows/ci.yml)
|
|
5
5
|
[](./LICENSE)
|
|
6
6
|
|
|
7
|
-
**
|
|
7
|
+
AIOpt is a **pre-deploy cost accident guardrail** for LLM changes.
|
|
8
|
+
- baseline = your observed local usage log (`usage.jsonl` / `usage.csv`)
|
|
9
|
+
- candidate = estimated change (model/provider/context/output/retry/traffic deltas)
|
|
10
|
+
- output = deterministic verdict + monthly impact estimate + confidence
|
|
11
|
+
|
|
12
|
+
No server. No upload. No account. No payments inside the CLI.
|
|
13
|
+
|
|
14
|
+
## 3-line quickstart (Guardrail)
|
|
8
15
|
```bash
|
|
9
16
|
npx aiopt install --force
|
|
10
|
-
|
|
11
|
-
npx aiopt
|
|
17
|
+
# baseline: your existing usage log
|
|
18
|
+
npx aiopt guard --context-mult 1.2
|
|
12
19
|
```
|
|
13
20
|
|
|
14
|
-
|
|
15
|
-
-
|
|
16
|
-
-
|
|
17
|
-
-
|
|
21
|
+
Exit codes:
|
|
22
|
+
- `0` OK
|
|
23
|
+
- `2` WARN (cost accident possible)
|
|
24
|
+
- `3` FAIL (merge-blocking)
|
|
25
|
+
|
|
26
|
+
Common knobs:
|
|
27
|
+
- `--context-mult <n>` (prompt/context grows)
|
|
28
|
+
- `--output-mult <n>` (output grows)
|
|
29
|
+
- `--retries-delta <n>` (more retries/attempts)
|
|
30
|
+
- `--call-mult <n>` (traffic spike / call volume)
|
|
31
|
+
|
|
32
|
+
## CI integration (GitHub Actions)
|
|
33
|
+
You can run `aiopt guard` in CI to catch accidental cost blow-ups before merge.
|
|
34
|
+
|
|
35
|
+
### 1) Non-blocking (report only)
|
|
36
|
+
```yaml
|
|
37
|
+
- name: AI cost guard (non-blocking)
|
|
38
|
+
run: |
|
|
39
|
+
npx aiopt guard --input ./aiopt-output/usage.jsonl --context-mult 1.2 || true
|
|
40
|
+
```
|
|
18
41
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
42
|
+
### 2) Merge-blocking (fail on high risk)
|
|
43
|
+
```yaml
|
|
44
|
+
- name: AI cost guard (blocking)
|
|
45
|
+
run: |
|
|
46
|
+
npx aiopt guard --input ./aiopt-output/usage.jsonl --context-mult 1.2
|
|
47
|
+
```
|
|
22
48
|
|
|
23
|
-
|
|
24
|
-
After `scan`, you will have:
|
|
25
|
-
1) `./aiopt-output/analysis.json` (top cost by feature/model)
|
|
26
|
-
2) `./aiopt-output/report.md` (WHAT TO CHANGE)
|
|
27
|
-
3) `./aiopt-output/report.json` (machine-readable summary)
|
|
28
|
-
4) `./aiopt-output/patches/*` (policy patch stubs)
|
|
29
|
-
5) `./aiopt-output/cost-policy.json` (policy file)
|
|
49
|
+
Tip: print guard output into the GitHub Actions **Step Summary** so you don’t need to scroll logs.
|
|
30
50
|
|
|
31
|
-
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
절감 가능 금액(Estimated): $0.21
|
|
35
|
-
절감 근거 3줄:
|
|
36
|
-
a) 모델 라우팅 절감(추정): $0.07
|
|
37
|
-
b) 컨텍스트 감축(추정): $0.02 (상위 20% input에 25% 감축 가정)
|
|
38
|
-
c) 재시도/오류 낭비(상한 적용): $0.13
|
|
51
|
+
## Optional: local dashboard
|
|
52
|
+
```bash
|
|
53
|
+
npx aiopt dashboard --port 3010
|
|
39
54
|
```
|
|
55
|
+
- Binds to **127.0.0.1** (local-only)
|
|
56
|
+
- Reads local JSONL/CSV → renders locally
|
|
40
57
|
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
58
|
+
## Optional: deeper local analysis (`scan`)
|
|
59
|
+
`scan` generates a more detailed local report + patch stubs (still local-only).
|
|
60
|
+
|
|
61
|
+
After `scan`, you will have:
|
|
62
|
+
1) `./aiopt-output/analysis.json`
|
|
63
|
+
2) `./aiopt-output/report.md`
|
|
64
|
+
3) `./aiopt-output/report.json`
|
|
65
|
+
4) `./aiopt-output/patches/*`
|
|
66
|
+
5) `./aiopt-output/cost-policy.json`
|
|
49
67
|
|
|
50
68
|
## Input (default)
|
|
51
69
|
- Default path: `./aiopt-output/usage.jsonl`
|
|
52
|
-
- Change: `npx aiopt
|
|
70
|
+
- Change: `npx aiopt guard --input <path>`
|
|
53
71
|
- Supported: JSONL (1 event per line), CSV
|
|
54
72
|
|
|
55
73
|
AIOpt supports both:
|
|
@@ -66,45 +84,24 @@ For wrapper logs:
|
|
|
66
84
|
Optional:
|
|
67
85
|
- `billed_cost` (aggregated) or `cost_usd` (wrapper)
|
|
68
86
|
|
|
69
|
-
### JSONL example (5 lines)
|
|
70
|
-
```jsonl
|
|
71
|
-
{"ts":"2026-02-07T00:00:01Z","provider":"openai","model":"gpt-5-mini","input_tokens":12000,"output_tokens":1800,"feature_tag":"summarize","retries":0,"status":"ok"}
|
|
72
|
-
{"ts":"2026-02-07T00:00:02Z","provider":"openai","model":"gpt-5.2","input_tokens":35000,"output_tokens":3000,"feature_tag":"coding","retries":1,"status":"ok"}
|
|
73
|
-
{"ts":"2026-02-07T00:00:03Z","provider":"anthropic","model":"claude-sonnet","input_tokens":22000,"output_tokens":2500,"feature_tag":"classify","retries":0,"status":"ok"}
|
|
74
|
-
{"ts":"2026-02-07T00:00:04Z","provider":"gemini","model":"gemini-1.5-flash","input_tokens":9000,"output_tokens":1200,"feature_tag":"translate","retries":0,"status":"ok"}
|
|
75
|
-
{"ts":"2026-02-07T00:00:05Z","provider":"openai","model":"unknown-model-x","input_tokens":8000,"output_tokens":1000,"feature_tag":"summarize","retries":2,"status":"error"}
|
|
76
|
-
```
|
|
77
|
-
|
|
78
|
-
## Outputs
|
|
79
|
-
- `analysis.json`
|
|
80
|
-
- `total_cost`
|
|
81
|
-
- `by_model_top` (top 10)
|
|
82
|
-
- `by_feature_top` (top 10)
|
|
83
|
-
- `unknown_models`
|
|
84
|
-
- `rate_table_version`, `rate_table_date`
|
|
85
|
-
- `report.md`
|
|
86
|
-
- WHAT TO CHANGE (file paths + keys)
|
|
87
|
-
- confidence + assumptions
|
|
88
|
-
- `report.json`
|
|
89
|
-
- summary + warnings + assumptions
|
|
90
|
-
- `patches/*`
|
|
91
|
-
- `policies.updated.*` stubs
|
|
92
|
-
- `cost-policy.json`
|
|
93
|
-
- `version, default_provider, rules, budgets, generated_from`
|
|
94
|
-
|
|
95
87
|
## Rate table
|
|
96
88
|
- `./rates/rate_table.json`
|
|
97
|
-
- Unknown models/providers
|
|
89
|
+
- Unknown models/providers may be marked as `Estimated`.
|
|
98
90
|
- `provider=local` (or `ollama`/`vllm`) is treated as **$0** by default (CPU/GPU cost not included).
|
|
99
91
|
|
|
100
|
-
##
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
## Local dev
|
|
92
|
+
## License (offline)
|
|
93
|
+
If you have a signed license key:
|
|
104
94
|
```bash
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
node dist/cli.js doctor
|
|
109
|
-
node dist/cli.js scan
|
|
95
|
+
npx aiopt license activate <LICENSE_KEY>
|
|
96
|
+
npx aiopt license status
|
|
97
|
+
npx aiopt license verify
|
|
110
98
|
```
|
|
99
|
+
|
|
100
|
+
## Docs
|
|
101
|
+
- Install-first guide: `aiopt/README.md`
|
|
102
|
+
- Platform support / caveats: `docs/PLATFORM_SUPPORT.md`
|
|
103
|
+
- Pricing/flow (internal): `docs/PRODUCT_PRICING_AND_FLOW.md`
|
|
104
|
+
- Payment/license ops: `docs/PAYMENT.md`
|
|
105
|
+
|
|
106
|
+
## Contact
|
|
107
|
+
- Instagram: **@sangikpp**
|