@workos/oagen 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +12 -0
- package/LICENSE.txt +21 -0
- package/README.md +197 -0
- package/dist/chunk-CIX5NOBF.js +263 -0
- package/dist/chunk-IJ5E5AE7.js +1040 -0
- package/dist/chunk-P5XUDUGY.js +1748 -0
- package/dist/cli/index.js +5603 -0
- package/dist/compat/index.d.ts +195 -0
- package/dist/compat/index.js +4453 -0
- package/dist/index.d.ts +300 -0
- package/dist/index.js +1733 -0
- package/dist/scripts/smoke/baseline.js +1751 -0
- package/dist/scripts/smoke/sdk-test.js +93 -0
- package/dist/smoke.js +1963 -0
- package/dist/types-B5t9O9Wm.d.ts +367 -0
- package/dist/verify/index.d.ts +76 -0
- package/dist/verify/index.js +336 -0
- package/docs/advanced/index.md +11 -0
- package/docs/advanced/oagen-ignore.md +144 -0
- package/docs/agents/architecture.md +53 -0
- package/docs/agents/emitters.md +69 -0
- package/docs/agents/testing.md +60 -0
- package/docs/architecture/dependency-layers.md +75 -0
- package/docs/architecture/emitter-contract.md +160 -0
- package/docs/architecture/extractor-contract.md +227 -0
- package/docs/architecture/ir-types.md +275 -0
- package/docs/architecture/non-additive-changes.md +76 -0
- package/docs/architecture/pipeline.md +110 -0
- package/docs/architecture/workflows.md +215 -0
- package/docs/cli.md +156 -0
- package/docs/contributor/index.md +10 -0
- package/docs/contributor/public-api.md +59 -0
- package/docs/contributor/versioning.md +48 -0
- package/docs/core/index.md +12 -0
- package/docs/core/minimal-emitter.md +134 -0
- package/docs/core/quickstart.md +142 -0
- package/docs/diagrams/generate-sdk-workflow.md +50 -0
- package/docs/index.md +40 -0
- package/examples/reference-emitter/README.md +41 -0
- package/examples/reference-emitter/oagen.config.ts +7 -0
- package/examples/reference-emitter/spec/github-subset.yml +400 -0
- package/examples/reference-emitter/src/emitter.ts +52 -0
- package/examples/reference-emitter/src/generators/client.ts +37 -0
- package/examples/reference-emitter/src/generators/config.ts +19 -0
- package/examples/reference-emitter/src/generators/enums.ts +16 -0
- package/examples/reference-emitter/src/generators/errors.ts +39 -0
- package/examples/reference-emitter/src/generators/models.ts +29 -0
- package/examples/reference-emitter/src/generators/resources.ts +60 -0
- package/examples/reference-emitter/src/generators/tests.ts +7 -0
- package/examples/reference-emitter/src/index.ts +1 -0
- package/examples/reference-emitter/src/naming.ts +21 -0
- package/examples/reference-emitter/src/type-mapper.ts +39 -0
- package/examples/reference-emitter/tsconfig.json +19 -0
- package/package.json +102 -0
- package/skills/generate-emitter/SKILL.md +393 -0
- package/skills/generate-emitter/references/common-pitfalls.md +23 -0
- package/skills/generate-emitter/references/generator-guide.md +201 -0
- package/skills/generate-emitter/references/project-scaffold.md +138 -0
- package/skills/generate-extractor/SKILL.md +200 -0
- package/skills/generate-sdk/SKILL.md +155 -0
- package/skills/generate-smoke-test/SKILL.md +185 -0
- package/skills/integrate/SKILL.md +108 -0
- package/skills/verify-compat/SKILL.md +151 -0
- package/skills/verify-smoke-test/SKILL.md +176 -0
|
@@ -0,0 +1,275 @@
|
|
|
1
|
+
# IR Type System Reference
|
|
2
|
+
|
|
3
|
+
Source: `src/ir/types.ts`
|
|
4
|
+
|
|
5
|
+
The intermediate representation (IR) is the contract between the parser and all language emitters. It uses plain TypeScript interfaces (no classes) with a discriminated union type system.
|
|
6
|
+
|
|
7
|
+
## Top-Level: ApiSpec
|
|
8
|
+
|
|
9
|
+
```typescript
|
|
10
|
+
interface ServerEntry {
|
|
11
|
+
url: string;
|
|
12
|
+
description?: string;
|
|
13
|
+
}
|
|
14
|
+
|
|
15
|
+
interface ApiSpec {
|
|
16
|
+
name: string; // From info.title
|
|
17
|
+
version: string; // From info.version
|
|
18
|
+
description?: string; // From info.description
|
|
19
|
+
baseUrl: string; // From servers[0].url
|
|
20
|
+
servers?: ServerEntry[]; // All servers from the spec
|
|
21
|
+
services: Service[]; // Grouped operations
|
|
22
|
+
models: Model[]; // Schema objects
|
|
23
|
+
enums: Enum[]; // String enums
|
|
24
|
+
auth?: AuthScheme[]; // Authentication schemes
|
|
25
|
+
}
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
The `servers` array contains all server entries from the OpenAPI spec. `baseUrl` is always set to the first server's URL for backward compatibility.
|
|
29
|
+
|
|
30
|
+
## TypeRef (discriminated union)
|
|
31
|
+
|
|
32
|
+
Every type in the IR is a `TypeRef`, discriminated on the `kind` field:
|
|
33
|
+
|
|
34
|
+
| Kind | Interface | Example |
|
|
35
|
+
| ----------- | --------------- | -------------------------------------------------------------------- |
|
|
36
|
+
| `primitive` | `PrimitiveType` | `{ kind: "primitive", type: "string" }` |
|
|
37
|
+
| `array` | `ArrayType` | `{ kind: "array", items: { kind: "primitive", type: "string" } }` |
|
|
38
|
+
| `model` | `ModelRef` | `{ kind: "model", name: "Organization" }` |
|
|
39
|
+
| `enum` | `EnumRef` | `{ kind: "enum", name: "Status" }` |
|
|
40
|
+
| `union` | `UnionType` | `{ kind: "union", variants: [...], discriminator?: {...} }` |
|
|
41
|
+
| `nullable` | `NullableType` | `{ kind: "nullable", inner: { kind: "primitive", type: "string" } }` |
|
|
42
|
+
| `literal` | `LiteralType` | `{ kind: "literal", value: "active" }` |
|
|
43
|
+
| `map` | `MapType` | `{ kind: "map", valueType: { kind: "primitive", type: "string" } }` |
|
|
44
|
+
|
|
45
|
+
### Primitive Formats
|
|
46
|
+
|
|
47
|
+
The `format` field on `PrimitiveType` carries additional type information:
|
|
48
|
+
|
|
49
|
+
| type | format | Semantic |
|
|
50
|
+
| --------- | ----------- | ----------------- |
|
|
51
|
+
| `string` | (none) | Plain string |
|
|
52
|
+
| `string` | `date` | ISO 8601 date |
|
|
53
|
+
| `string` | `date-time` | ISO 8601 datetime |
|
|
54
|
+
| `string` | `uuid` | UUID string |
|
|
55
|
+
| `string` | `binary` | Binary data |
|
|
56
|
+
| `integer` | (none) | Integer |
|
|
57
|
+
| `number` | (none) | Floating point |
|
|
58
|
+
| `boolean` | (none) | Boolean |
|
|
59
|
+
| `unknown` | (none) | Unknown/any type |
|
|
60
|
+
|
|
61
|
+
### LiteralType
|
|
62
|
+
|
|
63
|
+
```typescript
|
|
64
|
+
interface LiteralType {
|
|
65
|
+
kind: "literal";
|
|
66
|
+
value: string | number | boolean | null;
|
|
67
|
+
}
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
Represents a fixed value. The `null` variant maps to language-specific null literals.
|
|
71
|
+
|
|
72
|
+
### MapType
|
|
73
|
+
|
|
74
|
+
```typescript
|
|
75
|
+
interface MapType {
|
|
76
|
+
kind: "map";
|
|
77
|
+
valueType: TypeRef;
|
|
78
|
+
keyType?: TypeRef; // Defaults to string when absent
|
|
79
|
+
}
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
The optional `keyType` allows non-string map keys (e.g., enum-keyed maps). When absent, emitters should assume `string` keys.
|
|
83
|
+
|
|
84
|
+
### UnionType
|
|
85
|
+
|
|
86
|
+
```typescript
|
|
87
|
+
interface UnionType {
|
|
88
|
+
kind: "union";
|
|
89
|
+
variants: TypeRef[];
|
|
90
|
+
discriminator?: { property: string; mapping: Record<string, string> };
|
|
91
|
+
compositionKind?: "allOf" | "oneOf" | "anyOf";
|
|
92
|
+
}
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
The `compositionKind` field tells emitters which OAS composition keyword produced the union, allowing them to distinguish inheritance (`allOf`) from exclusive union (`oneOf`) from open union (`anyOf`).
|
|
96
|
+
|
|
97
|
+
### TypeParam
|
|
98
|
+
|
|
99
|
+
```typescript
|
|
100
|
+
interface TypeParam {
|
|
101
|
+
name: string;
|
|
102
|
+
default?: TypeRef;
|
|
103
|
+
}
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
Generic type parameters on models. Example: `DirectoryUser<TCustomAttributes = Record<string, unknown>>`.
|
|
107
|
+
|
|
108
|
+
## AuthScheme
|
|
109
|
+
|
|
110
|
+
```typescript
|
|
111
|
+
type AuthScheme =
|
|
112
|
+
| { kind: "bearer" }
|
|
113
|
+
| { kind: "apiKey"; in: "header" | "query" | "cookie"; name: string }
|
|
114
|
+
| { kind: "oauth2"; flows: Record<string, unknown> };
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
Emitters use `AuthScheme` to generate authentication configuration and client constructors. The `cookie` variant supports cookie-based API key authentication.
|
|
118
|
+
|
|
119
|
+
## HttpMethod
|
|
120
|
+
|
|
121
|
+
```typescript
|
|
122
|
+
type HttpMethod =
|
|
123
|
+
| "get"
|
|
124
|
+
| "post"
|
|
125
|
+
| "put"
|
|
126
|
+
| "patch"
|
|
127
|
+
| "delete"
|
|
128
|
+
| "head"
|
|
129
|
+
| "options"
|
|
130
|
+
| "trace";
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
All standard HTTP methods are supported. Emitters should handle `head`, `options`, and `trace` appropriately (e.g., HEAD returns no body).
|
|
134
|
+
|
|
135
|
+
## PaginationMeta
|
|
136
|
+
|
|
137
|
+
```typescript
|
|
138
|
+
interface PaginationMeta {
|
|
139
|
+
strategy: "cursor" | "offset" | "link-header";
|
|
140
|
+
param: string; // Query param name (e.g., "after" for cursor, "offset" for offset)
|
|
141
|
+
limitParam?: string; // Limit param name (offset strategy only, e.g., "limit")
|
|
142
|
+
dataPath?: string; // JSON path to the data array (e.g., "data", "results") — undefined means the response IS the data
|
|
143
|
+
itemType: TypeRef; // Type of each item in the paginated list
|
|
144
|
+
}
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
Present on `Operation` when pagination is detected. The `strategy` discriminant tells emitters whether to generate cursor-based, offset-based, or link-header pagination helpers. The `dataPath` is dynamically detected from the response envelope — when `undefined`, emitters should treat the entire response as the data array or use their own default.
|
|
148
|
+
|
|
149
|
+
The `link-header` strategy is for APIs that use RFC 5988 `Link` headers for pagination (e.g., GitHub API).
|
|
150
|
+
|
|
151
|
+
## Service & Operation
|
|
152
|
+
|
|
153
|
+
Service names are derived from OpenAPI tags when present — the first tag on an operation is converted to PascalCase (e.g., `tags: ["multi-factor-auth"]` → service `MultiFactorAuth`). When no tag is present, the parser falls back to the first path segment (e.g., `/organizations/{id}` → `Organizations`). Emitters can further remap service names via an overlay (e.g., `MultiFactorAuth` → `Mfa`) to match existing SDK class names.
|
|
154
|
+
|
|
155
|
+
```typescript
|
|
156
|
+
interface Service {
|
|
157
|
+
name: string; // e.g., "Organizations"
|
|
158
|
+
description?: string;
|
|
159
|
+
operations: Operation[];
|
|
160
|
+
}
|
|
161
|
+
|
|
162
|
+
interface Operation {
|
|
163
|
+
name: string; // e.g., "listUsers", "getUser", "createUser"
|
|
164
|
+
description?: string;
|
|
165
|
+
httpMethod: HttpMethod; // All 8 standard HTTP methods
|
|
166
|
+
path: string; // e.g., "/organizations/{id}"
|
|
167
|
+
pathParams: Parameter[];
|
|
168
|
+
queryParams: Parameter[];
|
|
169
|
+
headerParams: Parameter[];
|
|
170
|
+
cookieParams?: Parameter[]; // Present only when cookie params exist
|
|
171
|
+
requestBody?: TypeRef;
|
|
172
|
+
requestBodyEncoding?:
|
|
173
|
+
| "json"
|
|
174
|
+
| "form-data"
|
|
175
|
+
| "form-urlencoded"
|
|
176
|
+
| "binary"
|
|
177
|
+
| "text";
|
|
178
|
+
response: TypeRef; // Primary response (lowest 2xx with body)
|
|
179
|
+
successResponses?: SuccessResponse[]; // All 2xx responses (only when multiple exist)
|
|
180
|
+
errors: ErrorResponse[];
|
|
181
|
+
pagination?: PaginationMeta;
|
|
182
|
+
injectIdempotencyKey: boolean; // true when spec declares Idempotency-Key header
|
|
183
|
+
deprecated?: boolean;
|
|
184
|
+
async?: boolean;
|
|
185
|
+
}
|
|
186
|
+
|
|
187
|
+
interface SuccessResponse {
|
|
188
|
+
statusCode: number;
|
|
189
|
+
type: TypeRef;
|
|
190
|
+
}
|
|
191
|
+
|
|
192
|
+
interface Parameter {
|
|
193
|
+
name: string;
|
|
194
|
+
type: TypeRef;
|
|
195
|
+
required: boolean;
|
|
196
|
+
description?: string;
|
|
197
|
+
deprecated?: boolean;
|
|
198
|
+
default?: unknown;
|
|
199
|
+
}
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
### Idempotency Key
|
|
203
|
+
|
|
204
|
+
The `injectIdempotencyKey` flag is spec-driven: it is `true` only when the operation explicitly declares an `Idempotency-Key` header parameter in the OpenAPI spec. This means APIs that don't use idempotency keys (e.g., GitHub) get `false`, while APIs that do (e.g., Stripe, billing APIs) already declare the header in their spec and get `true`.
|
|
205
|
+
|
|
206
|
+
### Request Body Encoding
|
|
207
|
+
|
|
208
|
+
The `requestBodyEncoding` field tells emitters how to serialize the request body:
|
|
209
|
+
|
|
210
|
+
| Value | Content-Type |
|
|
211
|
+
| ----------------- | ----------------------------------- |
|
|
212
|
+
| `json` | `application/json` |
|
|
213
|
+
| `form-data` | `multipart/form-data` |
|
|
214
|
+
| `form-urlencoded` | `application/x-www-form-urlencoded` |
|
|
215
|
+
| `binary` | `application/octet-stream` |
|
|
216
|
+
| `text` | `text/plain` |
|
|
217
|
+
|
|
218
|
+
### Multiple 2xx Responses
|
|
219
|
+
|
|
220
|
+
When an operation has multiple 2xx status codes, `successResponses` contains all of them. The `response` field always points to the primary response (lowest 2xx with a body schema). This is useful for emitters that need to handle different status codes differently (e.g., 200 vs 204).
|
|
221
|
+
|
|
222
|
+
## Model & Field
|
|
223
|
+
|
|
224
|
+
```typescript
|
|
225
|
+
interface Model {
|
|
226
|
+
name: string; // PascalCase, e.g., "Organization"
|
|
227
|
+
description?: string;
|
|
228
|
+
fields: Field[];
|
|
229
|
+
typeParams?: TypeParam[]; // Generic type parameters
|
|
230
|
+
}
|
|
231
|
+
|
|
232
|
+
interface Field {
|
|
233
|
+
name: string; // snake_case from the spec
|
|
234
|
+
type: TypeRef;
|
|
235
|
+
required: boolean;
|
|
236
|
+
description?: string;
|
|
237
|
+
readOnly?: boolean;
|
|
238
|
+
writeOnly?: boolean;
|
|
239
|
+
deprecated?: boolean;
|
|
240
|
+
default?: unknown;
|
|
241
|
+
}
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
## Enum
|
|
245
|
+
|
|
246
|
+
```typescript
|
|
247
|
+
interface Enum {
|
|
248
|
+
name: string; // PascalCase
|
|
249
|
+
values: EnumValue[];
|
|
250
|
+
}
|
|
251
|
+
|
|
252
|
+
interface EnumValue {
|
|
253
|
+
name: string; // UPPER_SNAKE_CASE display name
|
|
254
|
+
value: string | number; // Actual value from spec (preserves numeric types)
|
|
255
|
+
description?: string;
|
|
256
|
+
deprecated?: boolean;
|
|
257
|
+
}
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
## ErrorResponse
|
|
261
|
+
|
|
262
|
+
```typescript
|
|
263
|
+
interface ErrorResponse {
|
|
264
|
+
statusCode: number; // e.g., 401, 404, 422
|
|
265
|
+
type?: TypeRef; // Response body type, if any
|
|
266
|
+
}
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
## Versioning
|
|
270
|
+
|
|
271
|
+
The IR does not currently use a version constant. As a 0.x project with no external consumers yet, formal IR versioning is deferred until post-open-source when external emitters need a stable contract signal.
|
|
272
|
+
|
|
273
|
+
**Compile-time safety:** `assertNever` enforces exhaustive `switch` statements over `TypeRef.kind` at compile time. Adding a new variant causes build failures in any emitter that doesn't handle it — this is the primary compatibility mechanism.
|
|
274
|
+
|
|
275
|
+
**Walker/Mapper utilities:** `walkTypeRef` and `mapTypeRef` handle recursive traversal of `TypeRef` trees. Both visit `MapType.keyType` when present. The `mapTypeRef` mapper callback for `map` receives an optional third argument `mappedKey?: T` for the mapped key type.
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
# Non-Additive Spec Changes: Staleness Detection
|
|
2
|
+
|
|
3
|
+
## Problem
|
|
4
|
+
|
|
5
|
+
The writer (`src/engine/writer.ts`) is additive-only — it never removes or modifies existing symbols in the live SDK. When an OpenAPI spec removes an endpoint, renames a model, or drops a field, the old symbols persist as dead code. The `oagen verify` command with `--scope spec-only` masks this: `specDerivedNames(newSpec)` excludes removed symbols from the baseline, so they silently pass verification.
|
|
6
|
+
|
|
7
|
+
## Solution
|
|
8
|
+
|
|
9
|
+
Staleness detection runs as an optional step in `oagen verify` when `--old-spec`, `--spec`, and `--api-surface` are all provided. It warns about symbols the spec no longer defines but that still exist in the live SDK.
|
|
10
|
+
|
|
11
|
+
All findings route through the existing compat `Violation` system with category `'staleness'` and severity `'warning'`. Stale symbols compile and run — they are dead code, not breakage.
|
|
12
|
+
|
|
13
|
+
## Detection Mechanisms
|
|
14
|
+
|
|
15
|
+
Two complementary mechanisms identify stale symbols:
|
|
16
|
+
|
|
17
|
+
### 1. Name-set difference (top-level symbols and fields)
|
|
18
|
+
|
|
19
|
+
```
|
|
20
|
+
removedNames = specDerivedNames(oldSpec) − specDerivedNames(newSpec)
|
|
21
|
+
removedFieldPaths = specDerivedFieldPaths(oldSpec) − specDerivedFieldPaths(newSpec)
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
This catches:
|
|
25
|
+
|
|
26
|
+
- Removed models, enums, services, and type aliases at the symbol level
|
|
27
|
+
- Removed fields from models that still exist in the new spec
|
|
28
|
+
|
|
29
|
+
### 2. Engine differ changes (operation-level)
|
|
30
|
+
|
|
31
|
+
```
|
|
32
|
+
diffSpecs(oldSpec, newSpec) → operation-removed changes
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
This catches methods removed from services that still exist — e.g., `Users.deleteUser` removed while the `Users` service continues.
|
|
36
|
+
|
|
37
|
+
### Cross-reference with live surface
|
|
38
|
+
|
|
39
|
+
Both mechanisms cross-reference findings against the live SDK surface (`--api-surface`) to confirm the stale symbol actually exists. If a removed spec symbol was never in the SDK (e.g., never generated, or already cleaned up), no warning is emitted.
|
|
40
|
+
|
|
41
|
+
## CLI Usage
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
oagen verify \
|
|
45
|
+
--lang node \
|
|
46
|
+
--output ./sdk \
|
|
47
|
+
--spec openapi-v2.yml \
|
|
48
|
+
--old-spec openapi-v1.yml \
|
|
49
|
+
--api-surface sdk-node-surface.json
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
Flags:
|
|
53
|
+
|
|
54
|
+
- `--old-spec <path>` — previous version of the OpenAPI spec
|
|
55
|
+
- `--spec <path>` — current version of the OpenAPI spec
|
|
56
|
+
- `--api-surface <path>` — extracted API surface of the live SDK
|
|
57
|
+
|
|
58
|
+
All three must be provided for staleness detection to run. When `--diagnostics` is set, staleness findings are included in `verify-diagnostics.json` under `stalenessCheck`.
|
|
59
|
+
|
|
60
|
+
## Behavior
|
|
61
|
+
|
|
62
|
+
- Staleness findings are **warnings only** — they never cause a non-zero exit code on their own
|
|
63
|
+
- Hand-written SDK symbols (never in any spec version) are not flagged
|
|
64
|
+
- Field-level staleness is only reported for models that still exist in the new spec (fully removed models are reported at the symbol level)
|
|
65
|
+
|
|
66
|
+
## Deferred Work
|
|
67
|
+
|
|
68
|
+
**Rename detection** is not implemented. When a spec renames a model (e.g., `Organization` → `Org`), staleness detection flags the old name as stale and compat verification flags the new name as missing. Precise rename detection (linking old → new) requires manifest comparison across spec versions and is deferred to future work.
|
|
69
|
+
|
|
70
|
+
## Implementation
|
|
71
|
+
|
|
72
|
+
- `src/compat/types.ts` — `ViolationCategory` union includes `'staleness'`
|
|
73
|
+
- `src/compat/staleness.ts` — `detectStaleSymbols()` core detection logic
|
|
74
|
+
- `src/cli/verify.ts` — wires staleness detection into the verify command
|
|
75
|
+
- `src/cli/index.ts` — registers `--old-spec` CLI flag
|
|
76
|
+
- `test/compat/staleness.test.ts` — unit tests
|
|
@@ -0,0 +1,110 @@
|
|
|
1
|
+
# Pipeline Architecture
|
|
2
|
+
|
|
3
|
+
## Overview
|
|
4
|
+
|
|
5
|
+
oagen provides a three-stage pipeline for transforming OpenAPI specs into SDK code: Parse → Emit → Write. Emitters are registered plugins that implement the `Emitter` interface.
|
|
6
|
+
|
|
7
|
+
```
|
|
8
|
+
┌──────────────────┐ ┌─────────────────────┐ ┌──────────────┐
|
|
9
|
+
│ OpenAPI Spec │────▶│ Parser Pipeline │────▶│ IR │
|
|
10
|
+
│ (YAML/JSON) │ │ │ │ (ApiSpec) │
|
|
11
|
+
└──────────────────┘ └─────────────────────┘ └──────┬───────┘
|
|
12
|
+
│
|
|
13
|
+
┌──────────────────────┐ │
|
|
14
|
+
│ Emitter (per lang) │◀───────────┘
|
|
15
|
+
│ generator methods │
|
|
16
|
+
└──────────┬───────────┘
|
|
17
|
+
│
|
|
18
|
+
┌──────────▼───────────┐
|
|
19
|
+
│ GeneratedFile[] │
|
|
20
|
+
│ { path, content } │
|
|
21
|
+
└──────────┬───────────┘
|
|
22
|
+
│
|
|
23
|
+
┌──────────▼───────────┐
|
|
24
|
+
│ Writer │
|
|
25
|
+
│ (disk + headers) │
|
|
26
|
+
└──────────────────────┘
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
## Stage 1: Parse (`src/parser/`)
|
|
30
|
+
|
|
31
|
+
Entry point: `parse.ts` → `parseSpec(specPath: string): Promise<ApiSpec>`
|
|
32
|
+
|
|
33
|
+
1. **refs.ts** — Loads the spec file, uses `@redocly/openapi-core` to bundle and resolve all `$ref` pointers
|
|
34
|
+
2. **schemas.ts** — Walks `components.schemas`, extracts `Model[]` and `Enum[]`
|
|
35
|
+
3. **operations.ts** — Walks `paths`, groups endpoints into `Service[]` with `Operation[]`
|
|
36
|
+
4. **pagination.ts** — Detects cursor-based pagination from query params; also detected from response shape in `responses.ts` (either source sets `paginated: true`)
|
|
37
|
+
|
|
38
|
+
### Operation Name Inference
|
|
39
|
+
|
|
40
|
+
If an `operationId` is present, it takes precedence. Names are inferred from HTTP method + path pattern:
|
|
41
|
+
|
|
42
|
+
| Method | Path Pattern | Inferred Name |
|
|
43
|
+
| ------ | ------------- | ------------- |
|
|
44
|
+
| GET | `/users` | `list` |
|
|
45
|
+
| GET | `/users/{id}` | `get` |
|
|
46
|
+
| POST | `/users` | `create` |
|
|
47
|
+
| PUT | `/users/{id}` | `update` |
|
|
48
|
+
| DELETE | `/users/{id}` | `delete` |
|
|
49
|
+
|
|
50
|
+
The collection vs. instance distinction is determined by whether the path ends with a `{param}` segment.
|
|
51
|
+
|
|
52
|
+
## Stage 2: Emit (`src/engine/`)
|
|
53
|
+
|
|
54
|
+
Entry point: `orchestrator.ts` → `generate(spec, emitter, options)`
|
|
55
|
+
|
|
56
|
+
The orchestrator calls each generator method in order:
|
|
57
|
+
|
|
58
|
+
```typescript
|
|
59
|
+
generateModels(models, ctx); // Data classes/structs
|
|
60
|
+
generateEnums(enums, ctx); // Enum types
|
|
61
|
+
generateResources(services, ctx); // API resource classes
|
|
62
|
+
generateClient(spec, ctx); // HTTP client
|
|
63
|
+
generateErrors(ctx); // Error hierarchy
|
|
64
|
+
generateConfig(ctx); // Configuration
|
|
65
|
+
generateTypeSignatures(spec, ctx); // Type annotations (optional)
|
|
66
|
+
generateTests(spec, ctx); // Tests + fixtures
|
|
67
|
+
generateManifest?.(spec, ctx); // Smoke-test manifest (optional)
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
### OperationPlan
|
|
71
|
+
|
|
72
|
+
Before rendering, emitters call `planOperation(op)` from `src/engine/operation-plan.ts` to compute an `OperationPlan` — a flat struct of semantic decisions (`isDelete`, `hasBody`, `isIdempotentPost`, `pathParamsInOptions`, `isPaginated`, `responseModelName`, etc.). This separates _what_ the generated code should do from _how_ it is rendered as a string. The plan is shared across all language emitters.
|
|
73
|
+
|
|
74
|
+
Each method returns `GeneratedFile[]`. The orchestrator:
|
|
75
|
+
|
|
76
|
+
1. Collects all files
|
|
77
|
+
2. Prepends `emitter.fileHeader()` to each file's content
|
|
78
|
+
3. Passes to writer (or prints paths in dry-run mode)
|
|
79
|
+
|
|
80
|
+
### EmitterContext
|
|
81
|
+
|
|
82
|
+
Every generator method receives:
|
|
83
|
+
|
|
84
|
+
- `namespace` — snake_case version of the SDK namespace
|
|
85
|
+
- `namespacePascal` — PascalCase version
|
|
86
|
+
- `spec` — the full IR (for cross-references)
|
|
87
|
+
- `outputDir` — target directory
|
|
88
|
+
|
|
89
|
+
## Stage 3: Write (`src/engine/writer.ts`)
|
|
90
|
+
|
|
91
|
+
- Writes `GeneratedFile[]` to disk under the output directory
|
|
92
|
+
- Respects `skipIfExists` flag — won't overwrite files marked as hand-editable
|
|
93
|
+
- Creates directories as needed
|
|
94
|
+
|
|
95
|
+
## Emitter Registry (`src/engine/registry.ts`)
|
|
96
|
+
|
|
97
|
+
Simple map from language name → `Emitter` instance. Emitters are registered at startup via `oagen.config.ts`:
|
|
98
|
+
|
|
99
|
+
```typescript
|
|
100
|
+
// In oagen.config.ts (emitter project)
|
|
101
|
+
import { nodeEmitter, rubyEmitter } from "./src/index.js";
|
|
102
|
+
|
|
103
|
+
export default {
|
|
104
|
+
emitters: [nodeEmitter, rubyEmitter],
|
|
105
|
+
};
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
## Diff Engine (`src/differ/`)
|
|
109
|
+
|
|
110
|
+
Compares two parsed specs (old vs new) and determines which files need regeneration. Classifies changes as additive, breaking, or mixed.
|
|
@@ -0,0 +1,215 @@
|
|
|
1
|
+
# Workflows
|
|
2
|
+
|
|
3
|
+
Two distinct phases: one-time setup (add a new SDK), then ongoing spec-driven
|
|
4
|
+
updates. The emitter-fixing loop only happens during setup.
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## Phase 1: /generate-sdk (one-time setup)
|
|
9
|
+
|
|
10
|
+
Build and validate a new language emitter. This is the only time emitter code
|
|
11
|
+
gets written or fixed.
|
|
12
|
+
|
|
13
|
+
### Skill sequence
|
|
14
|
+
|
|
15
|
+
**Project setup:** Run `oagen init --lang {language}` to create the emitter project, then implement the emitter methods.
|
|
16
|
+
|
|
17
|
+
**Scenario A** (existing SDK to preserve):
|
|
18
|
+
|
|
19
|
+
1. `/generate-emitter` — scaffold emitter, design doc, tests
|
|
20
|
+
2. `/generate-extractor` — scaffold extractor for API surface extraction
|
|
21
|
+
3. `/verify-compat` — extract baseline, generate with overlay, verify (includes overlay loop)
|
|
22
|
+
4. `/generate-smoke-test` — wire-level HTTP parity tests against output dir
|
|
23
|
+
5. `/verify-smoke-test` — iterate until smoke tests pass
|
|
24
|
+
6. `/integrate` — merge generated code into the live SDK
|
|
25
|
+
|
|
26
|
+
**Scenario B** (fresh, no compat constraints):
|
|
27
|
+
|
|
28
|
+
1. `/generate-emitter` — scaffold emitter, design doc, tests
|
|
29
|
+
2. `/generate-smoke-test` — wire-level HTTP parity tests
|
|
30
|
+
|
|
31
|
+
> **Lifecycle note:** Scenario B is a one-time bootstrap. Once the generated SDK
|
|
32
|
+
> ships, it becomes the live SDK. All future spec updates follow Scenario A —
|
|
33
|
+
> use `--target` to integrate into the live SDK and `--api-surface` to preserve
|
|
34
|
+
> backwards compatibility. Phase 2 is always Scenario A.
|
|
35
|
+
|
|
36
|
+
### Overlay loop (Scenario A only, inside /verify-compat)
|
|
37
|
+
|
|
38
|
+
Mechanical, automated. The overlay patches itself to resolve naming mismatches
|
|
39
|
+
between the generated SDK and the live SDK's API surface. The extractor's
|
|
40
|
+
`hints: LanguageHints` flows through to both the overlay builder and the differ,
|
|
41
|
+
so all type string interpretation (nullability, unions, extraction artifacts) is
|
|
42
|
+
language-appropriate.
|
|
43
|
+
|
|
44
|
+
1. Generate SDK with compat overlay applied (hints from extractor)
|
|
45
|
+
2. Diff generated output against baseline `<output>/sdk-{language}-surface.json` (hints from extractor)
|
|
46
|
+
3. If violations found, `patchOverlay()` adjusts the overlay
|
|
47
|
+
4. Regenerate with the patched overlay
|
|
48
|
+
5. Repeat
|
|
49
|
+
|
|
50
|
+
**Caps:**
|
|
51
|
+
|
|
52
|
+
- `--max-retries <n>` flag on `oagen verify` (default 3, set to 0 for single-pass)
|
|
53
|
+
- Stall detection — if the preservation score doesn't improve between iterations,
|
|
54
|
+
stops early (the remaining violations can't be fixed by overlay patching)
|
|
55
|
+
- Only `public-api` and `export-structure` violations are patchable; `signature`
|
|
56
|
+
and `behavioral` violations require emitter code changes
|
|
57
|
+
|
|
58
|
+
**Guard rails:** The retry loop only activates when BOTH `--api-surface` AND
|
|
59
|
+
`--spec` are provided (the spec is needed to regenerate). Without both,
|
|
60
|
+
verify runs a single pass.
|
|
61
|
+
|
|
62
|
+
**What it can't fix:** Structural emitter issues (e.g., wrong method signature
|
|
63
|
+
shape, async/sync mismatches). Those require changing emitter code — the loop
|
|
64
|
+
exits 1 and the emitter-fixing loop takes over.
|
|
65
|
+
|
|
66
|
+
**Code:** `src/cli/verify.ts` (retry loop + compat check) and `src/compat/overlay.ts`
|
|
67
|
+
(`patchOverlay` — also available as a library export for custom loops)
|
|
68
|
+
|
|
69
|
+
### Emitter-fixing loop (Phase 1 only)
|
|
70
|
+
|
|
71
|
+
This loop only runs during `/generate-sdk` setup, after all skills complete.
|
|
72
|
+
Its purpose is to get the new emitter producing correct output for the first time:
|
|
73
|
+
|
|
74
|
+
1. `oagen generate --spec {spec} --lang {lang} --output {sdk-path}`
|
|
75
|
+
2. `oagen verify --spec {spec} --lang {lang} --output {sdk-path}`
|
|
76
|
+
3. If exit 0 — done, proceed to final validation
|
|
77
|
+
4. If exit 1 — read `smoke-diff-findings.json`, fix emitter or smoke script, go to 1
|
|
78
|
+
5. If exit 2 — read `smoke-compile-errors.json`, fix emitter, go to 1
|
|
79
|
+
|
|
80
|
+
This loop ends when verify exits 0. After that, the emitter is stable and
|
|
81
|
+
Phase 2 (spec-driven updates) takes over.
|
|
82
|
+
|
|
83
|
+
### Final validation
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
npx tsc --noEmit # type check
|
|
87
|
+
npx vitest run # tests
|
|
88
|
+
npx tsup # build
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
---
|
|
92
|
+
|
|
93
|
+
## Phase 2: Spec updates (ongoing)
|
|
94
|
+
|
|
95
|
+
The emitter is stable. Phase 2 is always Scenario A — there is always a live
|
|
96
|
+
SDK to integrate into (either a pre-existing one from Scenario A setup, or the
|
|
97
|
+
output of Scenario B's initial generation).
|
|
98
|
+
|
|
99
|
+
### Pipeline
|
|
100
|
+
|
|
101
|
+
Every spec update uses `--target` to merge into the live SDK and `--api-surface`
|
|
102
|
+
to preserve backwards compatibility:
|
|
103
|
+
|
|
104
|
+
1. **Review changes:** `oagen diff --old v1.yml --new v2.yml --report`
|
|
105
|
+
2. **Regenerate:** `oagen generate --spec v2.yml --lang {lang} --output ./sdk --target {sdk_path} --api-surface sdk-{lang}-surface.json`
|
|
106
|
+
3. **Verify:** `oagen verify --spec v2.yml --lang {lang} --output ./sdk --api-surface sdk-{lang}-surface.json`
|
|
107
|
+
4. **Ship** if verify exits 0
|
|
108
|
+
|
|
109
|
+
**Alternative: incremental generation.** Instead of full `generate`, use
|
|
110
|
+
`diff` in generation mode to only regenerate affected files. This also supports
|
|
111
|
+
`--target` for live SDK integration:
|
|
112
|
+
|
|
113
|
+
```bash
|
|
114
|
+
oagen diff --old v1.yml --new v2.yml --lang ruby --output ./sdk --target {sdk_path} --api-surface sdk-{lang}-surface.json
|
|
115
|
+
oagen verify --spec v2.yml --lang ruby --output ./sdk --api-surface sdk-{lang}-surface.json
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
**External consumers** configure emitters and extractors via `oagen.config.ts`
|
|
119
|
+
instead of modifying CLI source — see the Configuration section in the README.
|
|
120
|
+
|
|
121
|
+
### If verify fails on a spec update
|
|
122
|
+
|
|
123
|
+
This is rare — it means the new spec introduced a pattern the emitter doesn't
|
|
124
|
+
handle yet (a new type, an unusual response shape). This is not the same as the
|
|
125
|
+
emitter-fixing loop from Phase 1. Fix the specific gap in the emitter, re-run
|
|
126
|
+
`generate` + `verify`, and move on.
|
|
127
|
+
|
|
128
|
+
**Diagnostics:** Pass `--diagnostics` to produce `verify-diagnostics.json` with
|
|
129
|
+
a structured breakdown of compat violations and smoke results. This is useful
|
|
130
|
+
for measuring overlay effectiveness per language and tracking preservation scores
|
|
131
|
+
over time.
|
|
132
|
+
|
|
133
|
+
```bash
|
|
134
|
+
oagen verify --spec v2.yml --lang node --output ./sdk --api-surface surface.json --diagnostics
|
|
135
|
+
# → verify-diagnostics.json
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
### Verify Result Types
|
|
139
|
+
|
|
140
|
+
Source: `src/verify/types.ts`, `src/compat/types.ts`, `src/differ/types.ts`
|
|
141
|
+
|
|
142
|
+
The verify pipeline returns structured result types that consumers can use programmatically:
|
|
143
|
+
|
|
144
|
+
```typescript
|
|
145
|
+
interface VerifyDiagnostics {
|
|
146
|
+
compatCheck?: {
|
|
147
|
+
totalBaselineSymbols;
|
|
148
|
+
preservedSymbols;
|
|
149
|
+
preservationScore;
|
|
150
|
+
violationsByCategory;
|
|
151
|
+
violationsBySeverity;
|
|
152
|
+
additions;
|
|
153
|
+
scopedToSpec;
|
|
154
|
+
scopedSymbolCount?;
|
|
155
|
+
};
|
|
156
|
+
smokeCheck?: { passed; findingsCount?; compileErrors?; baselinePath };
|
|
157
|
+
stalenessCheck?: { violations };
|
|
158
|
+
}
|
|
159
|
+
|
|
160
|
+
interface CompatCheckResult {
|
|
161
|
+
passed: boolean; // true if no breaking violations
|
|
162
|
+
diff: DiffResult; // detailed diff between baseline and candidate
|
|
163
|
+
scopedToSpec: boolean; // true if --spec was used to filter
|
|
164
|
+
scopedSymbolCount?: number;
|
|
165
|
+
}
|
|
166
|
+
|
|
167
|
+
interface SmokeCheckResult {
|
|
168
|
+
passed: boolean;
|
|
169
|
+
findingsCount?: number; // number of CRITICAL/WARNING findings
|
|
170
|
+
compileErrors?: boolean; // true if SDK failed type check
|
|
171
|
+
baselinePath: string; // path to the baseline used
|
|
172
|
+
generatedBaseline: boolean;
|
|
173
|
+
}
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
The compat differ returns a `DiffResult`:
|
|
177
|
+
|
|
178
|
+
```typescript
|
|
179
|
+
interface DiffResult {
|
|
180
|
+
preservationScore: number; // 0–100% of baseline symbols preserved
|
|
181
|
+
totalBaselineSymbols: number;
|
|
182
|
+
preservedSymbols: number;
|
|
183
|
+
violations: Violation[]; // breaking changes found
|
|
184
|
+
additions: Addition[]; // new symbols not in baseline
|
|
185
|
+
}
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
Each `Violation` carries a `ViolationCategory` that indicates the kind of breaking change:
|
|
189
|
+
|
|
190
|
+
| ViolationCategory | Meaning |
|
|
191
|
+
| ------------------ | ------------------------------------------------------ |
|
|
192
|
+
| `public-api` | A public class, method, or type was renamed or removed |
|
|
193
|
+
| `signature` | A method's parameter list or return type changed |
|
|
194
|
+
| `export-structure` | A barrel export is missing or reorganized |
|
|
195
|
+
| `behavioral` | A behavioral contract changed (e.g., async to sync) |
|
|
196
|
+
| `staleness` | A symbol was removed from the spec but still exists |
|
|
197
|
+
|
|
198
|
+
The spec differ (used by `oagen diff`) returns a `DiffReport`:
|
|
199
|
+
|
|
200
|
+
```typescript
|
|
201
|
+
interface DiffReport {
|
|
202
|
+
oldVersion: string;
|
|
203
|
+
newVersion: string;
|
|
204
|
+
changes: Change[]; // ModelAdded, ModelRemoved, EnumAdded, OperationAdded, etc.
|
|
205
|
+
summary: { added; removed; modified; breaking; additive };
|
|
206
|
+
}
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
### Exit codes from `oagen verify`
|
|
210
|
+
|
|
211
|
+
| Exit code | Meaning | Findings file |
|
|
212
|
+
| --------- | --------------------------------------------- | --------------------------- |
|
|
213
|
+
| 0 | Clean — all checks passed | — |
|
|
214
|
+
| 1 | Findings — CRITICAL mismatches or missing ops | `smoke-diff-findings.json` |
|
|
215
|
+
| 2 | Compile error — SDK failed type check | `smoke-compile-errors.json` |
|