@workos/oagen 0.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (64) hide show
  1. package/.claude-plugin/plugin.json +12 -0
  2. package/LICENSE.txt +21 -0
  3. package/README.md +197 -0
  4. package/dist/chunk-CIX5NOBF.js +263 -0
  5. package/dist/chunk-IJ5E5AE7.js +1040 -0
  6. package/dist/chunk-P5XUDUGY.js +1748 -0
  7. package/dist/cli/index.js +5603 -0
  8. package/dist/compat/index.d.ts +195 -0
  9. package/dist/compat/index.js +4453 -0
  10. package/dist/index.d.ts +300 -0
  11. package/dist/index.js +1733 -0
  12. package/dist/scripts/smoke/baseline.js +1751 -0
  13. package/dist/scripts/smoke/sdk-test.js +93 -0
  14. package/dist/smoke.js +1963 -0
  15. package/dist/types-B5t9O9Wm.d.ts +367 -0
  16. package/dist/verify/index.d.ts +76 -0
  17. package/dist/verify/index.js +336 -0
  18. package/docs/advanced/index.md +11 -0
  19. package/docs/advanced/oagen-ignore.md +144 -0
  20. package/docs/agents/architecture.md +53 -0
  21. package/docs/agents/emitters.md +69 -0
  22. package/docs/agents/testing.md +60 -0
  23. package/docs/architecture/dependency-layers.md +75 -0
  24. package/docs/architecture/emitter-contract.md +160 -0
  25. package/docs/architecture/extractor-contract.md +227 -0
  26. package/docs/architecture/ir-types.md +275 -0
  27. package/docs/architecture/non-additive-changes.md +76 -0
  28. package/docs/architecture/pipeline.md +110 -0
  29. package/docs/architecture/workflows.md +215 -0
  30. package/docs/cli.md +156 -0
  31. package/docs/contributor/index.md +10 -0
  32. package/docs/contributor/public-api.md +59 -0
  33. package/docs/contributor/versioning.md +48 -0
  34. package/docs/core/index.md +12 -0
  35. package/docs/core/minimal-emitter.md +134 -0
  36. package/docs/core/quickstart.md +142 -0
  37. package/docs/diagrams/generate-sdk-workflow.md +50 -0
  38. package/docs/index.md +40 -0
  39. package/examples/reference-emitter/README.md +41 -0
  40. package/examples/reference-emitter/oagen.config.ts +7 -0
  41. package/examples/reference-emitter/spec/github-subset.yml +400 -0
  42. package/examples/reference-emitter/src/emitter.ts +52 -0
  43. package/examples/reference-emitter/src/generators/client.ts +37 -0
  44. package/examples/reference-emitter/src/generators/config.ts +19 -0
  45. package/examples/reference-emitter/src/generators/enums.ts +16 -0
  46. package/examples/reference-emitter/src/generators/errors.ts +39 -0
  47. package/examples/reference-emitter/src/generators/models.ts +29 -0
  48. package/examples/reference-emitter/src/generators/resources.ts +60 -0
  49. package/examples/reference-emitter/src/generators/tests.ts +7 -0
  50. package/examples/reference-emitter/src/index.ts +1 -0
  51. package/examples/reference-emitter/src/naming.ts +21 -0
  52. package/examples/reference-emitter/src/type-mapper.ts +39 -0
  53. package/examples/reference-emitter/tsconfig.json +19 -0
  54. package/package.json +102 -0
  55. package/skills/generate-emitter/SKILL.md +393 -0
  56. package/skills/generate-emitter/references/common-pitfalls.md +23 -0
  57. package/skills/generate-emitter/references/generator-guide.md +201 -0
  58. package/skills/generate-emitter/references/project-scaffold.md +138 -0
  59. package/skills/generate-extractor/SKILL.md +200 -0
  60. package/skills/generate-sdk/SKILL.md +155 -0
  61. package/skills/generate-smoke-test/SKILL.md +185 -0
  62. package/skills/integrate/SKILL.md +108 -0
  63. package/skills/verify-compat/SKILL.md +151 -0
  64. package/skills/verify-smoke-test/SKILL.md +176 -0
@@ -0,0 +1,275 @@
1
+ # IR Type System Reference
2
+
3
+ Source: `src/ir/types.ts`
4
+
5
+ The intermediate representation (IR) is the contract between the parser and all language emitters. It uses plain TypeScript interfaces (no classes) with a discriminated union type system.
6
+
7
+ ## Top-Level: ApiSpec
8
+
9
+ ```typescript
10
+ interface ServerEntry {
11
+ url: string;
12
+ description?: string;
13
+ }
14
+
15
+ interface ApiSpec {
16
+ name: string; // From info.title
17
+ version: string; // From info.version
18
+ description?: string; // From info.description
19
+ baseUrl: string; // From servers[0].url
20
+ servers?: ServerEntry[]; // All servers from the spec
21
+ services: Service[]; // Grouped operations
22
+ models: Model[]; // Schema objects
23
+ enums: Enum[]; // String enums
24
+ auth?: AuthScheme[]; // Authentication schemes
25
+ }
26
+ ```
27
+
28
+ The `servers` array contains all server entries from the OpenAPI spec. `baseUrl` is always set to the first server's URL for backward compatibility.
29
+
30
+ ## TypeRef (discriminated union)
31
+
32
+ Every type in the IR is a `TypeRef`, discriminated on the `kind` field:
33
+
34
+ | Kind | Interface | Example |
35
+ | ----------- | --------------- | -------------------------------------------------------------------- |
36
+ | `primitive` | `PrimitiveType` | `{ kind: "primitive", type: "string" }` |
37
+ | `array` | `ArrayType` | `{ kind: "array", items: { kind: "primitive", type: "string" } }` |
38
+ | `model` | `ModelRef` | `{ kind: "model", name: "Organization" }` |
39
+ | `enum` | `EnumRef` | `{ kind: "enum", name: "Status" }` |
40
+ | `union` | `UnionType` | `{ kind: "union", variants: [...], discriminator?: {...} }` |
41
+ | `nullable` | `NullableType` | `{ kind: "nullable", inner: { kind: "primitive", type: "string" } }` |
42
+ | `literal` | `LiteralType` | `{ kind: "literal", value: "active" }` |
43
+ | `map` | `MapType` | `{ kind: "map", valueType: { kind: "primitive", type: "string" } }` |
44
+
45
+ ### Primitive Formats
46
+
47
+ The `format` field on `PrimitiveType` carries additional type information:
48
+
49
+ | type | format | Semantic |
50
+ | --------- | ----------- | ----------------- |
51
+ | `string` | (none) | Plain string |
52
+ | `string` | `date` | ISO 8601 date |
53
+ | `string` | `date-time` | ISO 8601 datetime |
54
+ | `string` | `uuid` | UUID string |
55
+ | `string` | `binary` | Binary data |
56
+ | `integer` | (none) | Integer |
57
+ | `number` | (none) | Floating point |
58
+ | `boolean` | (none) | Boolean |
59
+ | `unknown` | (none) | Unknown/any type |
60
+
61
+ ### LiteralType
62
+
63
+ ```typescript
64
+ interface LiteralType {
65
+ kind: "literal";
66
+ value: string | number | boolean | null;
67
+ }
68
+ ```
69
+
70
+ Represents a fixed value. The `null` variant maps to language-specific null literals.
71
+
72
+ ### MapType
73
+
74
+ ```typescript
75
+ interface MapType {
76
+ kind: "map";
77
+ valueType: TypeRef;
78
+ keyType?: TypeRef; // Defaults to string when absent
79
+ }
80
+ ```
81
+
82
+ The optional `keyType` allows non-string map keys (e.g., enum-keyed maps). When absent, emitters should assume `string` keys.
83
+
84
+ ### UnionType
85
+
86
+ ```typescript
87
+ interface UnionType {
88
+ kind: "union";
89
+ variants: TypeRef[];
90
+ discriminator?: { property: string; mapping: Record<string, string> };
91
+ compositionKind?: "allOf" | "oneOf" | "anyOf";
92
+ }
93
+ ```
94
+
95
+ The `compositionKind` field tells emitters which OAS composition keyword produced the union, allowing them to distinguish inheritance (`allOf`) from exclusive union (`oneOf`) from open union (`anyOf`).
96
+
97
+ ### TypeParam
98
+
99
+ ```typescript
100
+ interface TypeParam {
101
+ name: string;
102
+ default?: TypeRef;
103
+ }
104
+ ```
105
+
106
+ Generic type parameters on models. Example: `DirectoryUser<TCustomAttributes = Record<string, unknown>>`.
107
+
108
+ ## AuthScheme
109
+
110
+ ```typescript
111
+ type AuthScheme =
112
+ | { kind: "bearer" }
113
+ | { kind: "apiKey"; in: "header" | "query" | "cookie"; name: string }
114
+ | { kind: "oauth2"; flows: Record<string, unknown> };
115
+ ```
116
+
117
+ Emitters use `AuthScheme` to generate authentication configuration and client constructors. The `cookie` variant supports cookie-based API key authentication.
118
+
119
+ ## HttpMethod
120
+
121
+ ```typescript
122
+ type HttpMethod =
123
+ | "get"
124
+ | "post"
125
+ | "put"
126
+ | "patch"
127
+ | "delete"
128
+ | "head"
129
+ | "options"
130
+ | "trace";
131
+ ```
132
+
133
+ All standard HTTP methods are supported. Emitters should handle `head`, `options`, and `trace` appropriately (e.g., HEAD returns no body).
134
+
135
+ ## PaginationMeta
136
+
137
+ ```typescript
138
+ interface PaginationMeta {
139
+ strategy: "cursor" | "offset" | "link-header";
140
+ param: string; // Query param name (e.g., "after" for cursor, "offset" for offset)
141
+ limitParam?: string; // Limit param name (offset strategy only, e.g., "limit")
142
+ dataPath?: string; // JSON path to the data array (e.g., "data", "results") — undefined means the response IS the data
143
+ itemType: TypeRef; // Type of each item in the paginated list
144
+ }
145
+ ```
146
+
147
+ Present on `Operation` when pagination is detected. The `strategy` discriminant tells emitters whether to generate cursor-based, offset-based, or link-header pagination helpers. The `dataPath` is dynamically detected from the response envelope — when `undefined`, emitters should treat the entire response as the data array or use their own default.
148
+
149
+ The `link-header` strategy is for APIs that use RFC 5988 `Link` headers for pagination (e.g., GitHub API).
150
+
151
+ ## Service & Operation
152
+
153
+ Service names are derived from OpenAPI tags when present — the first tag on an operation is converted to PascalCase (e.g., `tags: ["multi-factor-auth"]` → service `MultiFactorAuth`). When no tag is present, the parser falls back to the first path segment (e.g., `/organizations/{id}` → `Organizations`). Emitters can further remap service names via an overlay (e.g., `MultiFactorAuth` → `Mfa`) to match existing SDK class names.
154
+
155
+ ```typescript
156
+ interface Service {
157
+ name: string; // e.g., "Organizations"
158
+ description?: string;
159
+ operations: Operation[];
160
+ }
161
+
162
+ interface Operation {
163
+ name: string; // e.g., "listUsers", "getUser", "createUser"
164
+ description?: string;
165
+ httpMethod: HttpMethod; // All 8 standard HTTP methods
166
+ path: string; // e.g., "/organizations/{id}"
167
+ pathParams: Parameter[];
168
+ queryParams: Parameter[];
169
+ headerParams: Parameter[];
170
+ cookieParams?: Parameter[]; // Present only when cookie params exist
171
+ requestBody?: TypeRef;
172
+ requestBodyEncoding?:
173
+ | "json"
174
+ | "form-data"
175
+ | "form-urlencoded"
176
+ | "binary"
177
+ | "text";
178
+ response: TypeRef; // Primary response (lowest 2xx with body)
179
+ successResponses?: SuccessResponse[]; // All 2xx responses (only when multiple exist)
180
+ errors: ErrorResponse[];
181
+ pagination?: PaginationMeta;
182
+ injectIdempotencyKey: boolean; // true when spec declares Idempotency-Key header
183
+ deprecated?: boolean;
184
+ async?: boolean;
185
+ }
186
+
187
+ interface SuccessResponse {
188
+ statusCode: number;
189
+ type: TypeRef;
190
+ }
191
+
192
+ interface Parameter {
193
+ name: string;
194
+ type: TypeRef;
195
+ required: boolean;
196
+ description?: string;
197
+ deprecated?: boolean;
198
+ default?: unknown;
199
+ }
200
+ ```
201
+
202
+ ### Idempotency Key
203
+
204
+ The `injectIdempotencyKey` flag is spec-driven: it is `true` only when the operation explicitly declares an `Idempotency-Key` header parameter in the OpenAPI spec. This means APIs that don't use idempotency keys (e.g., GitHub) get `false`, while APIs that do (e.g., Stripe, billing APIs) already declare the header in their spec and get `true`.
205
+
206
+ ### Request Body Encoding
207
+
208
+ The `requestBodyEncoding` field tells emitters how to serialize the request body:
209
+
210
+ | Value | Content-Type |
211
+ | ----------------- | ----------------------------------- |
212
+ | `json` | `application/json` |
213
+ | `form-data` | `multipart/form-data` |
214
+ | `form-urlencoded` | `application/x-www-form-urlencoded` |
215
+ | `binary` | `application/octet-stream` |
216
+ | `text` | `text/plain` |
217
+
218
+ ### Multiple 2xx Responses
219
+
220
+ When an operation has multiple 2xx status codes, `successResponses` contains all of them. The `response` field always points to the primary response (lowest 2xx with a body schema). This is useful for emitters that need to handle different status codes differently (e.g., 200 vs 204).
221
+
222
+ ## Model & Field
223
+
224
+ ```typescript
225
+ interface Model {
226
+ name: string; // PascalCase, e.g., "Organization"
227
+ description?: string;
228
+ fields: Field[];
229
+ typeParams?: TypeParam[]; // Generic type parameters
230
+ }
231
+
232
+ interface Field {
233
+ name: string; // snake_case from the spec
234
+ type: TypeRef;
235
+ required: boolean;
236
+ description?: string;
237
+ readOnly?: boolean;
238
+ writeOnly?: boolean;
239
+ deprecated?: boolean;
240
+ default?: unknown;
241
+ }
242
+ ```
243
+
244
+ ## Enum
245
+
246
+ ```typescript
247
+ interface Enum {
248
+ name: string; // PascalCase
249
+ values: EnumValue[];
250
+ }
251
+
252
+ interface EnumValue {
253
+ name: string; // UPPER_SNAKE_CASE display name
254
+ value: string | number; // Actual value from spec (preserves numeric types)
255
+ description?: string;
256
+ deprecated?: boolean;
257
+ }
258
+ ```
259
+
260
+ ## ErrorResponse
261
+
262
+ ```typescript
263
+ interface ErrorResponse {
264
+ statusCode: number; // e.g., 401, 404, 422
265
+ type?: TypeRef; // Response body type, if any
266
+ }
267
+ ```
268
+
269
+ ## Versioning
270
+
271
+ The IR does not currently use a version constant. As a 0.x project with no external consumers yet, formal IR versioning is deferred until post-open-source when external emitters need a stable contract signal.
272
+
273
+ **Compile-time safety:** `assertNever` enforces exhaustive `switch` statements over `TypeRef.kind` at compile time. Adding a new variant causes build failures in any emitter that doesn't handle it — this is the primary compatibility mechanism.
274
+
275
+ **Walker/Mapper utilities:** `walkTypeRef` and `mapTypeRef` handle recursive traversal of `TypeRef` trees. Both visit `MapType.keyType` when present. The `mapTypeRef` mapper callback for `map` receives an optional third argument `mappedKey?: T` for the mapped key type.
@@ -0,0 +1,76 @@
1
+ # Non-Additive Spec Changes: Staleness Detection
2
+
3
+ ## Problem
4
+
5
+ The writer (`src/engine/writer.ts`) is additive-only — it never removes or modifies existing symbols in the live SDK. When an OpenAPI spec removes an endpoint, renames a model, or drops a field, the old symbols persist as dead code. The `oagen verify` command with `--scope spec-only` masks this: `specDerivedNames(newSpec)` excludes removed symbols from the baseline, so they silently pass verification.
6
+
7
+ ## Solution
8
+
9
+ Staleness detection runs as an optional step in `oagen verify` when `--old-spec`, `--spec`, and `--api-surface` are all provided. It warns about symbols the spec no longer defines but that still exist in the live SDK.
10
+
11
+ All findings route through the existing compat `Violation` system with category `'staleness'` and severity `'warning'`. Stale symbols compile and run — they are dead code, not breakage.
12
+
13
+ ## Detection Mechanisms
14
+
15
+ Two complementary mechanisms identify stale symbols:
16
+
17
+ ### 1. Name-set difference (top-level symbols and fields)
18
+
19
+ ```
20
+ removedNames = specDerivedNames(oldSpec) − specDerivedNames(newSpec)
21
+ removedFieldPaths = specDerivedFieldPaths(oldSpec) − specDerivedFieldPaths(newSpec)
22
+ ```
23
+
24
+ This catches:
25
+
26
+ - Removed models, enums, services, and type aliases at the symbol level
27
+ - Removed fields from models that still exist in the new spec
28
+
29
+ ### 2. Engine differ changes (operation-level)
30
+
31
+ ```
32
+ diffSpecs(oldSpec, newSpec) → operation-removed changes
33
+ ```
34
+
35
+ This catches methods removed from services that still exist — e.g., `Users.deleteUser` removed while the `Users` service continues.
36
+
37
+ ### Cross-reference with live surface
38
+
39
+ Both mechanisms cross-reference findings against the live SDK surface (`--api-surface`) to confirm the stale symbol actually exists. If a removed spec symbol was never in the SDK (e.g., never generated, or already cleaned up), no warning is emitted.
40
+
41
+ ## CLI Usage
42
+
43
+ ```bash
44
+ oagen verify \
45
+ --lang node \
46
+ --output ./sdk \
47
+ --spec openapi-v2.yml \
48
+ --old-spec openapi-v1.yml \
49
+ --api-surface sdk-node-surface.json
50
+ ```
51
+
52
+ Flags:
53
+
54
+ - `--old-spec <path>` — previous version of the OpenAPI spec
55
+ - `--spec <path>` — current version of the OpenAPI spec
56
+ - `--api-surface <path>` — extracted API surface of the live SDK
57
+
58
+ All three must be provided for staleness detection to run. When `--diagnostics` is set, staleness findings are included in `verify-diagnostics.json` under `stalenessCheck`.
59
+
60
+ ## Behavior
61
+
62
+ - Staleness findings are **warnings only** — they never cause a non-zero exit code on their own
63
+ - Hand-written SDK symbols (never in any spec version) are not flagged
64
+ - Field-level staleness is only reported for models that still exist in the new spec (fully removed models are reported at the symbol level)
65
+
66
+ ## Deferred Work
67
+
68
+ **Rename detection** is not implemented. When a spec renames a model (e.g., `Organization` → `Org`), staleness detection flags the old name as stale and compat verification flags the new name as missing. Precise rename detection (linking old → new) requires manifest comparison across spec versions and is deferred to future work.
69
+
70
+ ## Implementation
71
+
72
+ - `src/compat/types.ts` — `ViolationCategory` union includes `'staleness'`
73
+ - `src/compat/staleness.ts` — `detectStaleSymbols()` core detection logic
74
+ - `src/cli/verify.ts` — wires staleness detection into the verify command
75
+ - `src/cli/index.ts` — registers `--old-spec` CLI flag
76
+ - `test/compat/staleness.test.ts` — unit tests
@@ -0,0 +1,110 @@
1
+ # Pipeline Architecture
2
+
3
+ ## Overview
4
+
5
+ oagen provides a three-stage pipeline for transforming OpenAPI specs into SDK code: Parse → Emit → Write. Emitters are registered plugins that implement the `Emitter` interface.
6
+
7
+ ```
8
+ ┌──────────────────┐ ┌─────────────────────┐ ┌──────────────┐
9
+ │ OpenAPI Spec │────▶│ Parser Pipeline │────▶│ IR │
10
+ │ (YAML/JSON) │ │ │ │ (ApiSpec) │
11
+ └──────────────────┘ └─────────────────────┘ └──────┬───────┘
12
+ │
13
+ ┌──────────────────────┐ │
14
+ │ Emitter (per lang) │◀───────────┘
15
+ │ generator methods │
16
+ └──────────┬───────────┘
17
+ │
18
+ ┌──────────▼───────────┐
19
+ │ GeneratedFile[] │
20
+ │ { path, content } │
21
+ └──────────┬───────────┘
22
+ │
23
+ ┌──────────▼───────────┐
24
+ │ Writer │
25
+ │ (disk + headers) │
26
+ └──────────────────────┘
27
+ ```
28
+
29
+ ## Stage 1: Parse (`src/parser/`)
30
+
31
+ Entry point: `parse.ts` → `parseSpec(specPath: string): Promise<ApiSpec>`
32
+
33
+ 1. **refs.ts** — Loads the spec file, uses `@redocly/openapi-core` to bundle and resolve all `$ref` pointers
34
+ 2. **schemas.ts** — Walks `components.schemas`, extracts `Model[]` and `Enum[]`
35
+ 3. **operations.ts** — Walks `paths`, groups endpoints into `Service[]` with `Operation[]`
36
+ 4. **pagination.ts** — Detects cursor-based pagination from query params; also detected from response shape in `responses.ts` (either source sets `paginated: true`)
37
+
38
+ ### Operation Name Inference
39
+
40
+ If an `operationId` is present, it takes precedence. Names are inferred from HTTP method + path pattern:
41
+
42
+ | Method | Path Pattern | Inferred Name |
43
+ | ------ | ------------- | ------------- |
44
+ | GET | `/users` | `list` |
45
+ | GET | `/users/{id}` | `get` |
46
+ | POST | `/users` | `create` |
47
+ | PUT | `/users/{id}` | `update` |
48
+ | DELETE | `/users/{id}` | `delete` |
49
+
50
+ The collection vs. instance distinction is determined by whether the path ends with a `{param}` segment.
51
+
52
+ ## Stage 2: Emit (`src/engine/`)
53
+
54
+ Entry point: `orchestrator.ts` → `generate(spec, emitter, options)`
55
+
56
+ The orchestrator calls each generator method in order:
57
+
58
+ ```typescript
59
+ generateModels(models, ctx); // Data classes/structs
60
+ generateEnums(enums, ctx); // Enum types
61
+ generateResources(services, ctx); // API resource classes
62
+ generateClient(spec, ctx); // HTTP client
63
+ generateErrors(ctx); // Error hierarchy
64
+ generateConfig(ctx); // Configuration
65
+ generateTypeSignatures(spec, ctx); // Type annotations (optional)
66
+ generateTests(spec, ctx); // Tests + fixtures
67
+ generateManifest?.(spec, ctx); // Smoke-test manifest (optional)
68
+ ```
69
+
70
+ ### OperationPlan
71
+
72
+ Before rendering, emitters call `planOperation(op)` from `src/engine/operation-plan.ts` to compute an `OperationPlan` — a flat struct of semantic decisions (`isDelete`, `hasBody`, `isIdempotentPost`, `pathParamsInOptions`, `isPaginated`, `responseModelName`, etc.). This separates _what_ the generated code should do from _how_ it is rendered as a string. The plan is shared across all language emitters.
73
+
74
+ Each method returns `GeneratedFile[]`. The orchestrator:
75
+
76
+ 1. Collects all files
77
+ 2. Prepends `emitter.fileHeader()` to each file's content
78
+ 3. Passes to writer (or prints paths in dry-run mode)
79
+
80
+ ### EmitterContext
81
+
82
+ Every generator method receives:
83
+
84
+ - `namespace` — snake_case version of the SDK namespace
85
+ - `namespacePascal` — PascalCase version
86
+ - `spec` — the full IR (for cross-references)
87
+ - `outputDir` — target directory
88
+
89
+ ## Stage 3: Write (`src/engine/writer.ts`)
90
+
91
+ - Writes `GeneratedFile[]` to disk under the output directory
92
+ - Respects `skipIfExists` flag — won't overwrite files marked as hand-editable
93
+ - Creates directories as needed
94
+
95
+ ## Emitter Registry (`src/engine/registry.ts`)
96
+
97
+ Simple map from language name → `Emitter` instance. Emitters are registered at startup via `oagen.config.ts`:
98
+
99
+ ```typescript
100
+ // In oagen.config.ts (emitter project)
101
+ import { nodeEmitter, rubyEmitter } from "./src/index.js";
102
+
103
+ export default {
104
+ emitters: [nodeEmitter, rubyEmitter],
105
+ };
106
+ ```
107
+
108
+ ## Diff Engine (`src/differ/`)
109
+
110
+ Compares two parsed specs (old vs new) and determines which files need regeneration. Classifies changes as additive, breaking, or mixed.
@@ -0,0 +1,215 @@
1
+ # Workflows
2
+
3
+ Two distinct phases: one-time setup (add a new SDK), then ongoing spec-driven
4
+ updates. The emitter-fixing loop only happens during setup.
5
+
6
+ ---
7
+
8
+ ## Phase 1: /generate-sdk (one-time setup)
9
+
10
+ Build and validate a new language emitter. This is the only time emitter code
11
+ gets written or fixed.
12
+
13
+ ### Skill sequence
14
+
15
+ **Project setup:** Run `oagen init --lang {language}` to create the emitter project, then implement the emitter methods.
16
+
17
+ **Scenario A** (existing SDK to preserve):
18
+
19
+ 1. `/generate-emitter` — scaffold emitter, design doc, tests
20
+ 2. `/generate-extractor` — scaffold extractor for API surface extraction
21
+ 3. `/verify-compat` — extract baseline, generate with overlay, verify (includes overlay loop)
22
+ 4. `/generate-smoke-test` — wire-level HTTP parity tests against output dir
23
+ 5. `/verify-smoke-test` — iterate until smoke tests pass
24
+ 6. `/integrate` — merge generated code into the live SDK
25
+
26
+ **Scenario B** (fresh, no compat constraints):
27
+
28
+ 1. `/generate-emitter` — scaffold emitter, design doc, tests
29
+ 2. `/generate-smoke-test` — wire-level HTTP parity tests
30
+
31
+ > **Lifecycle note:** Scenario B is a one-time bootstrap. Once the generated SDK
32
+ > ships, it becomes the live SDK. All future spec updates follow Scenario A —
33
+ > use `--target` to integrate into the live SDK and `--api-surface` to preserve
34
+ > backwards compatibility. Phase 2 is always Scenario A.
35
+
36
+ ### Overlay loop (Scenario A only, inside /verify-compat)
37
+
38
+ Mechanical, automated. The overlay patches itself to resolve naming mismatches
39
+ between the generated SDK and the live SDK's API surface. The extractor's
40
+ `hints: LanguageHints` flows through to both the overlay builder and the differ,
41
+ so all type string interpretation (nullability, unions, extraction artifacts) is
42
+ language-appropriate.
43
+
44
+ 1. Generate SDK with compat overlay applied (hints from extractor)
45
+ 2. Diff generated output against baseline `<output>/sdk-{language}-surface.json` (hints from extractor)
46
+ 3. If violations found, `patchOverlay()` adjusts the overlay
47
+ 4. Regenerate with the patched overlay
48
+ 5. Repeat
49
+
50
+ **Caps:**
51
+
52
+ - `--max-retries <n>` flag on `oagen verify` (default 3, set to 0 for single-pass)
53
+ - Stall detection — if the preservation score doesn't improve between iterations,
54
+ stops early (the remaining violations can't be fixed by overlay patching)
55
+ - Only `public-api` and `export-structure` violations are patchable; `signature`
56
+ and `behavioral` violations require emitter code changes
57
+
58
+ **Guard rails:** The retry loop only activates when BOTH `--api-surface` AND
59
+ `--spec` are provided (the spec is needed to regenerate). Without both,
60
+ verify runs a single pass.
61
+
62
+ **What it can't fix:** Structural emitter issues (e.g., wrong method signature
63
+ shape, async/sync mismatches). Those require changing emitter code — the loop
64
+ exits 1 and the emitter-fixing loop takes over.
65
+
66
+ **Code:** `src/cli/verify.ts` (retry loop + compat check) and `src/compat/overlay.ts`
67
+ (`patchOverlay` — also available as a library export for custom loops)
68
+
69
+ ### Emitter-fixing loop (Phase 1 only)
70
+
71
+ This loop only runs during `/generate-sdk` setup, after all skills complete.
72
+ Its purpose is to get the new emitter producing correct output for the first time:
73
+
74
+ 1. `oagen generate --spec {spec} --lang {lang} --output {sdk-path}`
75
+ 2. `oagen verify --spec {spec} --lang {lang} --output {sdk-path}`
76
+ 3. If exit 0 — done, proceed to final validation
77
+ 4. If exit 1 — read `smoke-diff-findings.json`, fix emitter or smoke script, go to 1
78
+ 5. If exit 2 — read `smoke-compile-errors.json`, fix emitter, go to 1
79
+
80
+ This loop ends when verify exits 0. After that, the emitter is stable and
81
+ Phase 2 (spec-driven updates) takes over.
82
+
83
+ ### Final validation
84
+
85
+ ```bash
86
+ npx tsc --noEmit # type check
87
+ npx vitest run # tests
88
+ npx tsup # build
89
+ ```
90
+
91
+ ---
92
+
93
+ ## Phase 2: Spec updates (ongoing)
94
+
95
+ The emitter is stable. Phase 2 is always Scenario A — there is always a live
96
+ SDK to integrate into (either a pre-existing one from Scenario A setup, or the
97
+ output of Scenario B's initial generation).
98
+
99
+ ### Pipeline
100
+
101
+ Every spec update uses `--target` to merge into the live SDK and `--api-surface`
102
+ to preserve backwards compatibility:
103
+
104
+ 1. **Review changes:** `oagen diff --old v1.yml --new v2.yml --report`
105
+ 2. **Regenerate:** `oagen generate --spec v2.yml --lang {lang} --output ./sdk --target {sdk_path} --api-surface sdk-{lang}-surface.json`
106
+ 3. **Verify:** `oagen verify --spec v2.yml --lang {lang} --output ./sdk --api-surface sdk-{lang}-surface.json`
107
+ 4. **Ship** if verify exits 0
108
+
109
+ **Alternative: incremental generation.** Instead of full `generate`, use
110
+ `diff` in generation mode to only regenerate affected files. This also supports
111
+ `--target` for live SDK integration:
112
+
113
+ ```bash
114
+ oagen diff --old v1.yml --new v2.yml --lang ruby --output ./sdk --target {sdk_path} --api-surface sdk-{lang}-surface.json
115
+ oagen verify --spec v2.yml --lang ruby --output ./sdk --api-surface sdk-{lang}-surface.json
116
+ ```
117
+
118
+ **External consumers** configure emitters and extractors via `oagen.config.ts`
119
+ instead of modifying CLI source — see the Configuration section in the README.
120
+
121
+ ### If verify fails on a spec update
122
+
123
+ This is rare — it means the new spec introduced a pattern the emitter doesn't
124
+ handle yet (a new type, an unusual response shape). This is not the same as the
125
+ emitter-fixing loop from Phase 1. Fix the specific gap in the emitter, re-run
126
+ `generate` + `verify`, and move on.
127
+
128
+ **Diagnostics:** Pass `--diagnostics` to produce `verify-diagnostics.json` with
129
+ a structured breakdown of compat violations and smoke results. This is useful
130
+ for measuring overlay effectiveness per language and tracking preservation scores
131
+ over time.
132
+
133
+ ```bash
134
+ oagen verify --spec v2.yml --lang node --output ./sdk --api-surface surface.json --diagnostics
135
+ # → verify-diagnostics.json
136
+ ```
137
+
138
+ ### Verify Result Types
139
+
140
+ Source: `src/verify/types.ts`, `src/compat/types.ts`, `src/differ/types.ts`
141
+
142
+ The verify pipeline returns structured result types that consumers can use programmatically:
143
+
144
+ ```typescript
145
+ interface VerifyDiagnostics {
146
+ compatCheck?: {
147
+ totalBaselineSymbols;
148
+ preservedSymbols;
149
+ preservationScore;
150
+ violationsByCategory;
151
+ violationsBySeverity;
152
+ additions;
153
+ scopedToSpec;
154
+ scopedSymbolCount?;
155
+ };
156
+ smokeCheck?: { passed; findingsCount?; compileErrors?; baselinePath };
157
+ stalenessCheck?: { violations };
158
+ }
159
+
160
+ interface CompatCheckResult {
161
+ passed: boolean; // true if no breaking violations
162
+ diff: DiffResult; // detailed diff between baseline and candidate
163
+ scopedToSpec: boolean; // true if --spec was used to filter
164
+ scopedSymbolCount?: number;
165
+ }
166
+
167
+ interface SmokeCheckResult {
168
+ passed: boolean;
169
+ findingsCount?: number; // number of CRITICAL/WARNING findings
170
+ compileErrors?: boolean; // true if SDK failed type check
171
+ baselinePath: string; // path to the baseline used
172
+ generatedBaseline: boolean;
173
+ }
174
+ ```
175
+
176
+ The compat differ returns a `DiffResult`:
177
+
178
+ ```typescript
179
+ interface DiffResult {
180
+ preservationScore: number; // 0–100% of baseline symbols preserved
181
+ totalBaselineSymbols: number;
182
+ preservedSymbols: number;
183
+ violations: Violation[]; // breaking changes found
184
+ additions: Addition[]; // new symbols not in baseline
185
+ }
186
+ ```
187
+
188
+ Each `Violation` carries a `ViolationCategory` that indicates the kind of breaking change:
189
+
190
+ | ViolationCategory | Meaning |
191
+ | ------------------ | ------------------------------------------------------ |
192
+ | `public-api` | A public class, method, or type was renamed or removed |
193
+ | `signature` | A method's parameter list or return type changed |
194
+ | `export-structure` | A barrel export is missing or reorganized |
195
+ | `behavioral` | A behavioral contract changed (e.g., async to sync) |
196
+ | `staleness` | A symbol was removed from the spec but still exists |
197
+
198
+ The spec differ (used by `oagen diff`) returns a `DiffReport`:
199
+
200
+ ```typescript
201
+ interface DiffReport {
202
+ oldVersion: string;
203
+ newVersion: string;
204
+ changes: Change[]; // ModelAdded, ModelRemoved, EnumAdded, OperationAdded, etc.
205
+ summary: { added; removed; modified; breaking; additive };
206
+ }
207
+ ```
208
+
209
+ ### Exit codes from `oagen verify`
210
+
211
+ | Exit code | Meaning | Findings file |
212
+ | --------- | --------------------------------------------- | --------------------------- |
213
+ | 0 | Clean — all checks passed | — |
214
+ | 1 | Findings — CRITICAL mismatches or missing ops | `smoke-diff-findings.json` |
215
+ | 2 | Compile error — SDK failed type check | `smoke-compile-errors.json` |