@autobe/estimate 0.30.0-dev.20260315 → 0.30.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/LICENSE +661 -661
  2. package/README.md +201 -261
  3. package/package.json +14 -5
package/README.md CHANGED
@@ -1,261 +1,201 @@
1
- # AutoBE - AI backend builder for prototype to production
2
-
3
- [![GitHub License](https://img.shields.io/badge/license-AGPL--3.0-blue.svg)](https://github.com/wrtnlabs/autobe/blob/master/LICENSE)
4
- [![NPM Version](https://img.shields.io/npm/v/@autobe/agent.svg)](https://www.npmjs.com/package/@autobe/agent)
5
- [![NPM Downloads](https://img.shields.io/npm/dm/@autobe/agent.svg)](https://www.npmjs.com/package/@autobe/agent)
6
- [![Build Status](https://github.com/wrtnlabs/autobe/workflows/build/badge.svg?branch=main)](https://github.com/wrtnlabs/autobe/actions?query=workflow%3Abuild)
7
- [![Guide Documents](https://img.shields.io/badge/Guide-Documents-forestgreen)](https://autobe.dev/docs/)
8
- [![Discord Badge](https://dcbadge.limes.pink/api/server/https://discord.gg/aMhRmzkqCx?style=flat)](https://discord.gg/aMhRmzkqCx)
9
-
10
- Describe your backend requirements in natural language through AutoBE's chat interface.
11
-
12
- AutoBE will analyze your requirements and build the backend application for you. The generated backend application is designed to be 100% buildable by AI-friendly compilers and ensures stability through powerful e2e test functions.
13
-
14
- With such AutoBE, build your first backend application quickly, then maintain and extend it with AI code assistants like Claude Code for enhanced productivity and stability.
15
-
16
- AutoBE will generate complete specifications, detailed database and API documentation, comprehensive test coverage for stability, and clean implementation logic that serves as a learning foundation for juniors while significantly improving senior developer productivity.
17
-
18
- Check out these complete backend application examples generated by AutoBE:
19
-
20
- https://github.com/user-attachments/assets/b995dd2a-23bd-43c9-96cb-96d5c805f19f
21
-
22
- 1. **To Do List**: [`todo`](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/todo)
23
- 2. **Discussion Board**: [`bbs`](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/bbs)
24
- 3. **Reddit Community**: [`reddit](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/reddit)
25
- 4. **E-Commerce**: [`shopping`](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/shopping)
26
- - Requirements Analysis: [Report](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/shopping/docs/analysis)
27
- - Database Design: [Entity Relationship Diagram](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/shopping/docs/ERD.md) / [Prisma Schema](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/shopping/prisma/schema)
28
- - API Design: [API Controllers](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/shopping/src/controllers) / [DTO Structures](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/shopping/src/api/structures)
29
- - E2E Test Functions: [`test/features/api`](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/shopping/test/features/api)
30
- - API Implementations: [`src/providers`](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/shopping/src/providers)
31
-
32
- ## Getting Started
33
-
34
- ```bash
35
- git clone https://github.com/wrtnlabs/autobe --depth=1
36
- cd autobe
37
- pnpm install
38
- pnpm run playground
39
- ```
40
-
41
- To use AutoBE, clone the repository and run the playground application locally. This allows you to chat with AutoBE's AI agents, manage multiple sessions, and use various LLM providers including local models like `qwen3-next-80b-a3b-instruct`.
42
-
43
- After installation, the playground will be available at http://localhost:5713. You can interact with AutoBE through a chat interface - simply describe what you want to build, and AutoBE will generate the backend application for you.
44
-
45
- Here's an example conversation script that guides AutoBE to create an "Economic/Political Discussion Board":
46
-
47
- 1. **Requirements Analysis**: "I want to create an economic/political discussion board. Since I'm not familiar with programming, please write a requirements analysis report as you see fit."
48
- 2. **Database Design**: "Design the database schema."
49
- 3. **API Specification**: "Create the API interface specification."
50
- 4. **Testing**: "Make the e2e test functions."
51
- 5. **Implementation**: "Implement API functions."
52
-
53
- ![Compilation Success Dashboard](https://autobe.dev/images/demonstrate/replay-z-ai-glm-5.png)
54
-
55
- > The playground includes a replay feature at http://localhost:5713/replay/index.html where you can view chat sessions from the AutoBE development team's testing and benchmarks.
56
-
57
- ## Documentation Resources
58
-
59
- Find comprehensive resources at our [official website](https://autobe.dev).
60
-
61
- ### 🏠 Home
62
-
63
- - 🙋🏻‍♂️ [Introduction](https://autobe.dev/docs)
64
- - 📦 [Setup](https://autobe.dev/docs/setup)
65
- - 🔍 Concepts
66
- - [Waterfall Model](https://autobe.dev/docs/concepts/waterfall)
67
- - [Compiler Strategy](https://autobe.dev/docs/concepts/compiler)
68
- - [AI Function Calling](https://autobe.dev/docs/concepts/function-calling)
69
-
70
- ### 📖 Features
71
-
72
- - 🤖 Agent Library
73
- - [Facade Controller](https://autobe.dev/docs/agent/facade)
74
- - [Configuration](https://autobe.dev/docs/agent/config)
75
- - [Event Handling](https://autobe.dev/docs/agent/event)
76
- - [Prompt Histories](https://autobe.dev/docs/agent/history)
77
- - 📡 WebSocket Protocol
78
- - [Remote Procedure Call](https://autobe.dev/docs/websocket/rpc)
79
- - [NestJS Server](https://autobe.dev/docs/websocket/nestjs)
80
- - [NodeJS Server](https://autobe.dev/docs/websocket/nodejs)
81
- - [Client Application](https://autobe.dev/docs/websocket/client)
82
- - 🛠️ Backend Stack
83
- - [TypeScript](https://autobe.dev/docs/stack/typescript)
84
- - [Prisma ORM](https://autobe.dev/docs/stack/prisma)
85
- - [NestJS Framework](https://autobe.dev/docs/stack/nestjs)
86
-
87
- ### 🔗 Appendix
88
-
89
- - 🌐 [No-Code Ecosystem](https://autobe.dev/docs/ecosystem)
90
- - 📅 Roadmap
91
- - [Alpha Release (done)](https://autobe.dev/docs/roadmap/alpha)
92
- - [Beta Release (done)](https://autobe.dev/docs/roadmap/beta)
93
- - [Gamma Release (done)](https://autobe.dev/docs/roadmap/gamma)
94
- - [Delta Release (active)](https://autobe.dev/docs/roadmap/delta)
95
- - 🔧 [API Documentation](https://autobe.dev/api)
96
-
97
- ## How AutoBE Works
98
-
99
- ```mermaid
100
- flowchart
101
- subgraph "Backend Coding Agent"
102
- coder("Facade Controller")
103
- end
104
- subgraph "Functional Agents"
105
- coder --"Requirements Analysis"--> analyze("Analyze")
106
- coder --"ERD"--> database("Database")
107
- coder --"API Design"--> interface("Interface")
108
- coder --"Test Codes" --> test("Test")
109
- coder --"Main Program" --> realize("Realize")
110
- end
111
- subgraph "Compiler Feedback"
112
- database --"validates" --> prismaCompiler("<a href="https://github.com/wrtnlabs/autobe/blob/main/packages/interface/src/database/AutoBeDatabase.ts" target="_blank">Prisma Compiler</a>")
113
- interface --"generates" --> openapiCompiler("<a href="https://github.com/wrtnlabs/autobe/blob/main/packages/interface/src/openapi/AutoBeOpenApi.ts" target="_blank">OpenAPI Compiler</a>")
114
- test --"analyzes" --> testCompiler("<a href="https://github.com/wrtnlabs/autobe/blob/main/packages/interface/src/test/AutoBeTest.ts" target="_blank">Test Compiler</a>")
115
- realize --"compiles" --> realizeCompiler("TypeScript Compiler")
116
- end
117
- ```
118
-
119
- AutoBE follows a waterfall methodology to generate backend applications, with 40+ specialized agents handling each phase. The agents work in coordinated teams throughout the development process.
120
-
121
- Each waterfall stage includes AI-friendly compilers that guarantee type safety of the generated code. Rather than generating code directly, AutoBE's agents first construct language-neutral Abstract Syntax Trees using predefined schemas. Each AST node undergoes validation against type rules before any code generation occurs, catching structural errors at the conceptual level rather than during compilation.
122
-
123
- This approach is designed to ensure that the final generated TypeScript and Prisma code is 100% buildable. Based on our testing with multiple example projects including e-commerce platforms, discussion boards, and task management systems, AutoBE maintains this compilation guarantee across diverse application types.
124
-
125
- To illustrate this process, here are the phase outputs from our "Economic/Political Discussion Board" example:
126
-
127
- 1. **Requirements Analysis**: [Report](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/bbs/docs/analysis)
128
- 2. **Database Design**: [Entity Relationship Diagram](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/bbs/docs/ERD.md) / [Prisma Schema](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/bbs/prisma/schema)
129
- 3. **API Specification**: [API Controllers](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/bbs/src/controllers) / [DTO Structures](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/bbs/src/api/structures)
130
- 4. **E2E Test Functions**: [`test/features/api`](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/bbs/test/features/api)
131
- 5. **API Implementations**: [`src/providers`](https://github.com/wrtnlabs/autobe-examples/tree/main/openai/gpt-4.1/bbs/src/providers)
132
-
133
- Also, you don't need to use all phases - stop at any stage that fits your needs. Whether you want just requirements analysis, database design, API specification, or e2e testing, AutoBE adapts to your workflow.
134
-
135
- Additionally, if you're skipping the full pipeline because of language preference rather than workflow needs, this capability is in development - AutoBE's language-neutral AST structure will soon support additional programming languages beyond TypeScript.
136
-
137
- ## Type-Safe Client SDK
138
-
139
- Every AutoBE-generated backend automatically includes a type-safe client SDK, making frontend integration seamless and error-free. This SDK provides:
140
-
141
- - **Zero Configuration**: SDK is auto-generated alongside your backend - no manual setup required
142
- - **100% Type Safety**: Full TypeScript support with autocomplete and compile-time validation
143
- - **Framework Agnostic**: Works with React, Vue, Angular, or any TypeScript/JavaScript project
144
- - **E2E Test Integration**: Powers AI-generated test suites for comprehensive backend testing
145
-
146
- ```typescript
147
- import api, { IPost } from "autobe-generated-sdk";
148
-
149
- // Type-safe API calls with full autocomplete
150
- const connection: api.IConnection = {
151
- host: "http://localhost:1234",
152
- };
153
- await api.functional.users.login(connection, {
154
- body: {
155
- email: "user@example.com",
156
- password: "secure-password",
157
- },
158
- });
159
-
160
- // TypeScript catches errors at compile time
161
- const post: IPost = await api.functional.posts.create(connection, {
162
- body: {
163
- title: "Hello World",
164
- content: "My first post",
165
- // authorId: "123" <- TypeScript error if this field is missing!
166
- },
167
- });
168
- ```
169
-
170
- This SDK eliminates the traditional pain points of API integration - no more manual type definitions, no more runtime surprises, and no more API documentation lookups. Your frontend developers can focus on building features, not wrestling with API contracts.
171
-
172
- **Beyond Frontend Integration**: The SDK powers both frontend development and E2E test generation. AutoBE uses the same type-safe SDK internally to generate comprehensive test suites, ensuring every API endpoint is thoroughly tested. This creates a robust feedback loop that enhances backend stability - AI writes tests using the SDK, the SDK ensures type safety, and your backend becomes more reliable with every generated test.
173
-
174
- ## Roadmap Schedule
175
-
176
- ```mermaid
177
- gantt
178
- dateFormat YYYY-MM-DD
179
- title AutoBE Delta Roadmap Timeline (2026 Q1)
180
-
181
- section Local LLM Benchmark
182
- Qwen3 Database Phase : done, 2026-01-01, 31d
183
- Qwen3 Interface Phase : done, 2026-01-16, 31d
184
- Qwen3 Test Phase : done, 2026-02-01, 21d
185
- Qwen3 Realize Phase : done, 2026-02-01, 59d
186
-
187
- section Validation Logic Enhancement
188
- Dynamic Function Calling Schema : done, 2026-01-01, 7d
189
- Validation Feedback Stringify : done, 2026-01-08, 12d
190
- JSON Schema Validator : done, 2026-01-08, 12d
191
- Schema Review Validation Logic : done, 2026-01-20, 35d
192
- Test Mapping Plan Enhancement : done, 2026-02-05, 24d
193
- Realize Mapping Plan Enhancement : done, 2026-02-15, 45d
194
-
195
- section RAG Optimization
196
- Hybrid Search (Vector + BM25) : done, 2026-01-01, 14d
197
- Dynamic K Retrieval : done, 2026-01-15, 14d
198
- RAG Preliminary Prompting : done, 2026-01-15, 28d
199
- RAG Benchmark & Tuning : done, 2026-02-01, 28d
200
- Analyze Agent Restructuring : done, 2026-02-01, 28d
201
-
202
- section Design Integrity
203
- DB Coverage Agent : done, 2026-01-15, 28d
204
- API Endpoint Coverage Agent : done, 2026-01-22, 28d
205
- Schema Relation Agent : done, 2026-02-01, 28d
206
- Schema Structure Agent : done, 2026-02-01, 28d
207
- Schema Content Agent : done, 2026-03-01, 28d
208
-
209
- section Multi-lingual Support
210
- Java Compiler PoC : done, 2026-01-01, 30d
211
- Java Database : done, 2026-01-01, 14d
212
- Java Interface : done, 2026-01-15, 21d
213
- Java Test : active, 2026-02-05, 28d
214
- Java Realize : active, 2026-03-01, 31d
215
-
216
- section Human Modification Support
217
- Database Schema Parser : active, 2026-02-15, 28d
218
- Interface Schema Parser : active, 2026-02-22, 28d
219
- Requirements Sync Agent : planned, 2026-03-08, 24d
220
-
221
- section Miscellaneous
222
- System Prompt Simplification : done, 2026-02-01, 28d
223
- Estimation Agent : done, 2026-02-01, 28d
224
- Playground Service Enhancement : active, 2026-02-15, 28d
225
- PR Articles Writing : active, 2026-02-15, 30d
226
- ```
227
-
228
- AutoBE has successfully completed Alpha, Beta, and Gamma development phases, establishing a solid foundation with **100% compilation success rate**. The current **Delta Release** focuses on transitioning from horizontal expansion to vertical deepening.
229
-
230
- **Strategic Shift**: In Gamma, we rapidly implemented features like RAG, Modularization, and Complementation under a "just ship it" philosophy. Delta fills the stability gaps that remained by systematically discovering and fixing hidden defects through Local LLM benchmarks.
231
-
232
- **Key Focus Areas**:
233
-
234
- - **Local LLM Benchmark**: Using open-source models like Qwen3 as a touchstone to discover hidden defects that commercial models mask, ensuring more robust operation across all model types
235
- - **Validation Logic Enhancement**: Strengthening schemas and validation logic through dynamic function calling schemas, JSON Schema validators, and progressive validation pipelines
236
- - **RAG Optimization**: Completing the Hybrid Search system (Vector + BM25) with dynamic K retrieval and comprehensive benchmark tuning
237
- - **Design Integrity**: Building mechanisms to verify and ensure design consistency between Database and Interface phases through coverage and schema review agents
238
- - **Multi-lingual Support**: Launching Java/Spring code generation alongside TypeScript/NestJS, with language-neutral AST structures enabling future language additions
239
- - **Human Modification Support**: Enabling maintenance continuity by parsing user-modified code back into AutoBE's internal AST representation, ensuring AutoBE remains useful beyond initial generation
240
-
241
- This roadmap prioritizes stability and depth over feature breadth, informed by real-world production experience from Gamma.
242
-
243
- ## Current Limitations
244
-
245
- While AutoBE achieves 100% compilation success, please note these current limitations:
246
-
247
- **Runtime Behavior**: Generated applications compile successfully, but runtime behavior may require testing and refinement. Unexpected runtime errors can occur during server execution, such as database connection issues, API endpoint failures, or business logic exceptions that weren't caught during compilation. We strongly recommend thorough testing in development environments before deploying to production. Our v1.0 release targets 100% runtime success to address these issues.
248
-
249
- **Design Interpretation**: AutoBE's database and API designs may differ from your expectations. We recommend thoroughly reviewing generated specifications before proceeding with implementation, especially before production deployment.
250
-
251
- **Token Consumption**: AutoBE requires significant AI token usage for complex projects. Based on our testing, projects typically consume 30M-250M+ tokens depending on complexity (simple todo apps use ~4M tokens, while complex e-commerce platforms may require 250M+ tokens). We are working on RAG optimization to reduce this overhead in future releases.
252
-
253
- **Maintenance**: AutoBE focuses on initial generation and does not provide ongoing maintenance capabilities. Once your backend is generated, you'll need to handle bug fixes, feature additions, performance optimizations, and security updates manually. We recommend establishing a development workflow that combines the generated codebase with AI coding assistants like Claude Code for efficient ongoing development and maintenance tasks.
254
-
255
- ![Entity Relationship Diagram Sample](https://autobe.dev/images/demonstrate/erd.png)
256
-
257
- ## License
258
-
259
- AutoBE is licensed under the [GNU Affero General Public License v3.0 (AGPL-3.0)](LICENSE). If you modify AutoBE itself or offer it as a network service, you must make your source code available under the same license.
260
-
261
- However, backend applications generated by AutoBE can be relicensed under any license you choose, such as MIT. This means you can freely use AutoBE-generated code in commercial projects without open source obligations, similar to how other code generation tools work.
1
+ # @autobe/estimate
2
+
3
+ A CLI tool that evaluates code quality for AutoBE-generated projects.
4
+
5
+ ## Quick Start
6
+
7
+ ```bash
8
+ # 1. Build (only needed once)
9
+ cd packages/estimate
10
+ npx tsc --build
11
+
12
+ # 2. Set up environment
13
+ cp .env.example .env
14
+ # Fill in OPENROUTER_API_KEY if you want AI agent evaluation
15
+
16
+ # 3. Run a single evaluation
17
+ npx tsx dist/bin/estimate.js -i /path/to/project -o ./reports
18
+
19
+ # 4. Run the full benchmark suite
20
+ ./run-benchmark.sh
21
+ ```
22
+
23
+ ## CLI Usage
24
+
25
+ ```bash
26
+ npx tsx dist/bin/estimate.js -i <path> -o <path> [options]
27
+ ```
28
+
29
+ | Option | Description |
30
+ |--------|-------------|
31
+ | `-i, --input <path>` | Path to the project to evaluate (required) |
32
+ | `-o, --output <path>` | Directory to save reports (required) |
33
+ | `-v, --verbose` | Show detailed logs |
34
+ | `--continue-on-gate-failure` | Continue evaluation even if gate fails |
35
+ | `--use-agent` | Enable AI agent evaluation (30% of score) |
36
+ | `--provider <provider>` | LLM provider: `claude`, `openai`, `openrouter` |
37
+ | `--api-key <key>` | API key (or set `OPENROUTER_API_KEY` env var) |
38
+ | `--auto-fix` | Auto-fix simple issues (TS1161, TS7006) |
39
+ | `--run-tests` | Start Docker server and run e2e tests |
40
+ | `--golden` | Run Golden Set evaluation |
41
+ | `--project <project>` | Project type for Golden Set: `todo`, `bbs`, `reddit`, `shopping` |
42
+
43
+ ## Scoring System
44
+
45
+ ### Gate Check (pass/fail)
46
+
47
+ If the code doesn't compile, you get a 0.
48
+
49
+ - **Source file check**: No TypeScript files in `src/` means instant failure (GATE001)
50
+ - **TypeScript compilation**: Uses `AutoBeTypeScriptCompiler` (in-memory)
51
+ - **Prisma schema validation**: Uses `AutoBeDatabaseCompiler` (in-memory)
52
+
53
+ ### Scoring Phases (70% of total)
54
+
55
+ | Phase | Weight | What we check |
56
+ |-------|--------|---------------|
57
+ | Document Quality | 10% | Presence of `docs/analysis/`, README |
58
+ | Requirements Coverage | 25% | Controllers, providers, DTOs coverage |
59
+ | Test Coverage | 30% | Test count, assertion quality, stub detection |
60
+ | Logic Completeness | 25% | TODOs, FIXMEs, empty methods, stub returns |
61
+ | API Completeness | 10% | Endpoint implementation, provider delegation |
62
+
63
+ ### Penalties
64
+
65
+ | Penalty | Trigger | Max Deduction |
66
+ |---------|---------|---------------|
67
+ | Warning | Warning ratio > 50% | -10 |
68
+ | Duplication | > 50 duplicate blocks | -5 |
69
+ | JSDoc | > 10% missing | -5 |
70
+ | Schema Sync (SYNC001) | > 5 empty types in DTOs | -5 |
71
+ | Schema Sync (SYNC002) | >= 3 Prisma-Structure mismatches | -5 |
72
+ | Mapping ratio (REQ006) | < 50% controller-provider coverage | -40 |
73
+
74
+ ### Reference Info (no score impact)
75
+
76
+ - **Complexity**: Functions with cyclomatic complexity > 15
77
+ - **Duplication**: Blocks of 10+ identical lines
78
+ - **Naming**: PascalCase violations
79
+ - **JSDoc**: Missing documentation comments
80
+ - **Schema Sync**: Empty interfaces (SYNC001) + Prisma-Structure property mismatches (SYNC002)
81
+
82
+ ### AI Agent Evaluation (30% of total)
83
+
84
+ Enable with `--use-agent`:
85
+
86
+ - **SecurityAgent**: OWASP Top 10 security analysis
87
+ - **LLMQualityAgent**: Detects hallucinations, incomplete implementations, logic errors
88
+
89
+ ### Scoring Formula
90
+
91
+ ```
92
+ Raw Phase Score = sum(phase_score * phase_weight)
93
+ Penalties = warning + duplication + jsdoc + schemaSync + mapping (max ~65)
94
+ Adjusted Phase = Raw Phase - Penalties
95
+
96
+ Without agents: Final Score = Adjusted Phase (100%)
97
+ With agents: Final Score = (Adjusted Phase * 70%) + (Agent Average * 30%)
98
+ ```
99
+
100
+ ## Grading
101
+
102
+ | Grade | Score | Meaning |
103
+ |-------|-------|---------|
104
+ | A | 90-100 | Production ready |
105
+ | B | 80-89 | Minor improvements needed |
106
+ | C | 70-79 | Several issues to address |
107
+ | D | 60-69 | Significant problems |
108
+ | F | 0-59 | Major issues or gate failure |
109
+
110
+ ## Benchmarking
111
+
112
+ Run evaluations across all models and projects:
113
+
114
+ ```bash
115
+ cd packages/estimate
116
+
117
+ # Scoring only (no LLM calls, fast)
118
+ ./run-benchmark.sh
119
+
120
+ # With AI agents (requires OPENROUTER_API_KEY)
121
+ ./run-benchmark.sh agent
122
+
123
+ # Full mode (agents + runtime tests + golden set)
124
+ ./run-benchmark.sh full
125
+ ```
126
+
127
+ Results are saved to `reports/benchmark/<model>/<project>/`.
128
+
129
+ ### Compare
130
+
131
+ Compare multiple projects side by side:
132
+
133
+ ```bash
134
+ npx tsx dist/bin/estimate.js compare \
135
+ -p "model-a:/path/to/a" "model-b:/path/to/b" \
136
+ -o ./reports/comparison
137
+ ```
138
+
139
+ ## Environment Variables
140
+
141
+ Create a `.env` file in `packages/estimate/`:
142
+
143
+ ```bash
144
+ OPENROUTER_API_KEY=sk-or-...
145
+
146
+ # Optional: Langfuse telemetry
147
+ LANGFUSE_PUBLIC_KEY=pk-lf-...
148
+ LANGFUSE_SECRET_KEY=sk-lf-...
149
+ LANGFUSE_HOST=https://cloud.langfuse.com
150
+ ```
151
+
152
+ ## Output
153
+
154
+ Each evaluation produces two files:
155
+
156
+ - `estimate-report.md` — Human-readable summary with score breakdown
157
+ - `estimate-report.json` — Machine-readable for CI/CD integration
158
+
159
+ ## Sample Output
160
+
161
+ ```
162
+ Scoring Phases (70% of total score):
163
+ -------------------------------------
164
+ Gate: Pass
165
+ Document Quality 100/100
166
+ Requirements Coverage 90/100
167
+ Test Coverage 61/100
168
+ Logic Completeness 100/100
169
+ API Completeness 100/100
170
+ -------------------------------------
171
+
172
+ Reference Info (no score impact):
173
+ -------------------------------------
174
+ Complexity: 2 complex functions (max: 22)
175
+ Duplication: 102 duplicate blocks
176
+ Naming: 0 issues
177
+ JSDoc: 36 missing
178
+ Schema Sync: 0/35 empty types, 0 mismatched
179
+ -------------------------------------
180
+
181
+ Final Score: 85/100 (Grade: B)
182
+ ```
183
+
184
+ ## Troubleshooting
185
+
186
+ **Gate keeps failing**
187
+ - Use `--continue-on-gate-failure` to see all issues
188
+ - Gate uses in-memory compilers — unresolved external modules like `@nestjs/common` are expected
189
+
190
+ **AI agent errors**
191
+ - Check your API key
192
+ - OpenRouter model IDs use `provider/model-name` format
193
+ - Rate limits are retried automatically
194
+
195
+ **Build not working**
196
+ - Run `npx tsc --build` first
197
+ - Make sure `dist/` directory exists
198
+
199
+ ## License
200
+
201
+ AGPL-3.0
package/package.json CHANGED
@@ -1,7 +1,16 @@
1
1
  {
2
2
  "name": "@autobe/estimate",
3
- "version": "0.30.0-dev.20260315",
3
+ "version": "0.30.2",
4
4
  "description": "Code quality evaluation system for AutoBE generated code",
5
+ "author": "Wrtn Technologies",
6
+ "license": "AGPL-3.0",
7
+ "repository": {
8
+ "type": "git",
9
+ "url": "https://github.com/wrtnlabs/autobe"
10
+ },
11
+ "bugs": {
12
+ "url": "https://github.com/wrtnlabs/autobe/issues"
13
+ },
5
14
  "main": "dist/index.js",
6
15
  "types": "dist/index.d.ts",
7
16
  "bin": {
@@ -25,10 +34,10 @@
25
34
  "langfuse": "^3.38.6",
26
35
  "openai": "^6.15.0",
27
36
  "typescript": "~5.9.3",
28
- "@autobe/compiler": "0.30.0-dev.20260315",
29
- "@autobe/interface": "0.30.0-dev.20260315",
30
- "@autobe/utils": "0.30.0-dev.20260315",
31
- "@autobe/filesystem": "0.30.0-dev.20260315"
37
+ "@autobe/interface": "0.30.2",
38
+ "@autobe/compiler": "0.30.2",
39
+ "@autobe/utils": "0.30.2",
40
+ "@autobe/filesystem": "0.30.2"
32
41
  },
33
42
  "devDependencies": {
34
43
  "@types/node": "^20.10.0",