@techspokes/typescript-wsdl-client 0.15.2 → 0.17.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (79) hide show
  1. package/README.md +5 -0
  2. package/dist/app/generateApp.d.ts.map +1 -1
  3. package/dist/app/generateApp.js +4 -3
  4. package/dist/cli.js +46 -2
  5. package/dist/client/generateClient.d.ts.map +1 -1
  6. package/dist/client/generateClient.js +64 -8
  7. package/dist/client/generateOperations.d.ts.map +1 -1
  8. package/dist/client/generateOperations.js +29 -6
  9. package/dist/client/generateTypes.d.ts.map +1 -1
  10. package/dist/client/generateTypes.js +13 -0
  11. package/dist/compiler/schemaCompiler.d.ts +44 -11
  12. package/dist/compiler/schemaCompiler.d.ts.map +1 -1
  13. package/dist/compiler/schemaCompiler.js +102 -6
  14. package/dist/compiler/shapeResolver.d.ts +18 -0
  15. package/dist/compiler/shapeResolver.d.ts.map +1 -0
  16. package/dist/compiler/shapeResolver.js +280 -0
  17. package/dist/gateway/generateGateway.d.ts.map +1 -1
  18. package/dist/gateway/generateGateway.js +2 -1
  19. package/dist/gateway/generators.d.ts +13 -1
  20. package/dist/gateway/generators.d.ts.map +1 -1
  21. package/dist/gateway/generators.js +98 -13
  22. package/dist/gateway/helpers.d.ts +16 -0
  23. package/dist/gateway/helpers.d.ts.map +1 -1
  24. package/dist/gateway/helpers.js +1 -0
  25. package/dist/index.d.ts +6 -0
  26. package/dist/index.d.ts.map +1 -1
  27. package/dist/index.js +23 -4
  28. package/dist/openapi/generateOpenAPI.d.ts.map +1 -1
  29. package/dist/openapi/generateOpenAPI.js +30 -2
  30. package/dist/openapi/generatePaths.d.ts.map +1 -1
  31. package/dist/openapi/generatePaths.js +4 -2
  32. package/dist/openapi/generateSchemas.d.ts.map +1 -1
  33. package/dist/openapi/generateSchemas.js +20 -5
  34. package/dist/pipeline.d.ts +13 -0
  35. package/dist/pipeline.d.ts.map +1 -1
  36. package/dist/pipeline.js +17 -1
  37. package/dist/runtime/ndjson.d.ts +24 -0
  38. package/dist/runtime/ndjson.d.ts.map +1 -0
  39. package/dist/runtime/ndjson.js +30 -0
  40. package/dist/runtime/streamXml.d.ts +45 -0
  41. package/dist/runtime/streamXml.d.ts.map +1 -0
  42. package/dist/runtime/streamXml.js +212 -0
  43. package/dist/test/generators.d.ts +2 -2
  44. package/dist/test/generators.d.ts.map +1 -1
  45. package/dist/test/generators.js +79 -26
  46. package/dist/test/mockData.d.ts +12 -2
  47. package/dist/test/mockData.d.ts.map +1 -1
  48. package/dist/test/mockData.js +17 -8
  49. package/dist/util/cli.d.ts +3 -0
  50. package/dist/util/cli.d.ts.map +1 -1
  51. package/dist/util/cli.js +6 -1
  52. package/dist/util/runtimeSource.d.ts +2 -0
  53. package/dist/util/runtimeSource.d.ts.map +1 -0
  54. package/dist/util/runtimeSource.js +38 -0
  55. package/dist/util/streamConfig.d.ts +59 -0
  56. package/dist/util/streamConfig.d.ts.map +1 -0
  57. package/dist/util/streamConfig.js +230 -0
  58. package/docs/README.md +1 -0
  59. package/docs/api-reference.md +146 -0
  60. package/docs/architecture.md +27 -5
  61. package/docs/cli-reference.md +30 -0
  62. package/docs/concepts.md +150 -11
  63. package/docs/configuration.md +40 -0
  64. package/docs/decisions/002-streamable-responses.md +308 -0
  65. package/docs/gateway-guide.md +37 -0
  66. package/docs/generated-code.md +21 -0
  67. package/docs/migration-playbook.md +33 -0
  68. package/docs/migration.md +31 -6
  69. package/docs/output-anatomy.md +49 -0
  70. package/docs/production.md +32 -0
  71. package/docs/start-here.md +33 -0
  72. package/docs/supported-patterns.md +29 -0
  73. package/docs/testing.md +14 -0
  74. package/docs/troubleshooting.md +18 -0
  75. package/package.json +9 -6
  76. package/src/runtime/clientStreamMethods.tpl.txt +183 -0
  77. package/src/runtime/ndjson.ts +32 -0
  78. package/src/runtime/operationsStreamHelper.tpl.txt +13 -0
  79. package/src/runtime/streamXml.ts +293 -0
package/docs/concepts.md CHANGED
@@ -34,12 +34,12 @@ interface Price {
34
34
  All XSD numeric and date-time types map to `string` by default. This prevents
35
35
  precision loss at the cost of convenience.
36
36
 
37
- | XSD Type | Default | Override Options | When to Override |
38
- |----------|---------|-----------------|-----------------|
39
- | xs:long | string | number, bigint | Use number if values fit JS range |
40
- | xs:integer | string | number | Use string for arbitrary-size ints |
41
- | xs:decimal | string | number | Use string for precise decimals |
42
- | xs:dateTime | string | Date | Use Date if runtime parsing is okay |
37
+ | XSD Type | Default | Override Options | When to Override |
38
+ |---------------|----------|--------------------|---------------------------------------|
39
+ | `xs:long` | `string` | `number`, `bigint` | Use `number` if values fit JS range |
40
+ | `xs:integer` | `string` | `number` | Use `string` for arbitrary-size ints |
41
+ | `xs:decimal` | `string` | `number` | Use `string` for precise decimals |
42
+ | `xs:dateTime` | `string` | `Date` | Use `Date` if runtime parsing is okay |
43
43
 
44
44
  Override with CLI flags:
45
45
 
@@ -47,6 +47,92 @@ Override with CLI flags:
47
47
  - `--client-decimal-as`
48
48
  - `--client-date-as`
49
49
 
50
+ ## Simple Type Aliases
51
+
52
+ Named `xs:simpleType` declarations generate scalar TypeScript aliases. Enumerated restrictions generate string literal unions.
53
+
54
+ ```xml
55
+ <xs:simpleType name="MyEnum">
56
+ <xs:restriction base="xs:string">
57
+ <xs:enumeration value="Red"/>
58
+ <xs:enumeration value="Green"/>
59
+ </xs:restriction>
60
+ </xs:simpleType>
61
+ ```
62
+
63
+ The generated TypeScript type preserves the enum as a scalar alias:
64
+
65
+ ```typescript
66
+ export type MyEnum = "Red" | "Green";
67
+ ```
68
+
69
+ The generated OpenAPI schema uses the same component name with a scalar enum schema:
70
+
71
+ ```json
72
+ {
73
+ "MyEnum": {
74
+ "type": "string",
75
+ "enum": ["Red", "Green"]
76
+ }
77
+ }
78
+ ```
79
+
80
+ ### Same-Name Global Elements
81
+
82
+ Some WSDLs declare a named simple type and a global element with the same local name. When the element references that simple type, the generator treats the element as the scalar alias rather than creating a wrapper interface.
83
+
84
+ ```xml
85
+ <xs:simpleType name="MyEnum">
86
+ <xs:restriction base="xs:string">
87
+ <xs:enumeration value="Red"/>
88
+ <xs:enumeration value="Green"/>
89
+ </xs:restriction>
90
+ </xs:simpleType>
91
+ <xs:element name="MyEnum" nillable="true" type="tns:MyEnum"/>
92
+ ```
93
+
94
+ The generated TypeScript remains a single declaration:
95
+
96
+ ```typescript
97
+ export type MyEnum = "Red" | "Green";
98
+ ```
99
+
100
+ Operation methods that use `tns:MyEnum` as their root element accept and return `MyEnum` directly:
101
+
102
+ ```typescript
103
+ interface EnumServiceOperations {
104
+ Echo(args: MyEnum): Promise<{ response: MyEnum; headers: unknown }>;
105
+ }
106
+ ```
107
+
108
+ This avoids invalid duplicate declarations such as `type MyEnum` plus `interface MyEnum`. It also keeps OpenAPI request and response schemas pointed at the scalar `MyEnum` component.
109
+
110
+ The same-name scalar element does not create an object wrapper only to carry element metadata. A root element marked `nillable="true"` still uses the scalar alias as the operation type.
111
+
112
+ ### Different-Name Simple Elements
113
+
114
+ When a global element has a different name from the named simple type it references, the element remains a wrapper surface type. The simple type alias is still generated separately.
115
+
116
+ ```xml
117
+ <xs:simpleType name="MyEnum">
118
+ <xs:restriction base="xs:string">
119
+ <xs:enumeration value="Red"/>
120
+ <xs:enumeration value="Green"/>
121
+ </xs:restriction>
122
+ </xs:simpleType>
123
+ <xs:element name="FavoriteColor" type="tns:MyEnum"/>
124
+ ```
125
+
126
+ The generated surface type keeps the element name and stores the scalar value in `$value`:
127
+
128
+ ```typescript
129
+ export type MyEnum = "Red" | "Green";
130
+
131
+ export interface FavoriteColor {
132
+ $value?: MyEnum;
133
+ }
134
+ ```
135
+
50
136
  ## Deterministic Generation
51
137
 
52
138
  All output is stable and diff-friendly for CI/CD pipelines.
@@ -69,6 +155,8 @@ automatically placed alongside generated output.
69
155
 
70
156
  The catalog stores optional human-readable `doc` fields extracted from WSDL/XSD documentation nodes. These fields are additive metadata used by TypeScript, OpenAPI, gateway, and generated-test emitters and do not change runtime behavior.
71
157
 
158
+ The catalog may also store optional `diagnostics.notes` entries. These notes record non-error modeling decisions, such as reusing a same-name simple type alias instead of emitting a duplicate wrapper interface. CLI commands print these entries as `Note:` lines during compilation.
159
+
72
160
  The catalog also stores optional `wsdlDocs` metadata for selected WSDL nodes:
73
161
 
74
162
  - `bindings[]`
@@ -81,11 +169,11 @@ OpenAPI uses operation docs for both `description` and default `summary` values.
81
169
 
82
170
  ### Catalog Locations by Command
83
171
 
84
- | Command | Location |
85
- |---------|----------|
86
- | client | `{client-dir}/catalog.json` |
87
- | openapi | `{openapi-dir}/catalog.json` |
88
- | pipeline | First available output directory |
172
+ | Command | Location |
173
+ |------------|----------------------------------|
174
+ | `client` | `{client-dir}/catalog.json` |
175
+ | `openapi` | `{openapi-dir}/catalog.json` |
176
+ | `pipeline` | First available output directory |
89
177
 
90
178
  ## Response Envelope
91
179
 
@@ -142,6 +230,10 @@ details:
142
230
  }
143
231
  ```
144
232
 
233
+ ### Streaming Bypass
234
+
235
+ Operations opted into streaming with `--stream-config` bypass the success envelope on the `200` response path. The OpenAPI response content is declared as the configured stream media type (default `application/x-ndjson`) and the gateway writes raw NDJSON lines straight to the response body. Error responses (400, 502, and the rest) still use the normal envelope so clients always see structured failures before the first record. See [ADR-002](decisions/002-streamable-responses.md) for the full rationale.
236
+
145
237
  ### Envelope Naming
146
238
 
147
239
  The base envelope is named `${serviceName}ResponseEnvelope`. Override with
@@ -249,3 +341,50 @@ schemas, and detects circular dependencies.
249
341
 
250
342
  Disable with `--openapi-validate false` or `validate: false` in the
251
343
  programmatic API.
344
+
345
+ ## Streaming vs Buffered Responses
346
+
347
+ The generator produces two response execution models. Buffered is the default and only path for operations not listed in a stream config. Streaming is opt-in and operation-scoped; it changes the emitted client method signature, the OpenAPI response description, and the Fastify route shape for that operation only.
348
+
349
+ ### Execution Model Contrast
350
+
351
+ Buffered operations call `node-soap`, wait for the full response to materialize, and return `{ response, headers, responseRaw, requestRaw }`. Streaming operations bypass `node-soap`, POST a hand-built SOAP envelope via `fetch`, and return `StreamOperationResponse<RecordType>` with `records: AsyncIterable<RecordType>`. The SAX parser in `runtime/streamXml.ts` walks the configured `recordPath` and yields each record as its closing tag arrives.
352
+
353
+ The catalog is the source of truth for stream metadata. Each opted-in operation carries an `OperationStreamMetadata` entry, and downstream emitters (client, OpenAPI, gateway, tests) all read from the catalog. OpenAPI carries a derived view via the `x-wsdl-tsc-stream` extension.
354
+
355
+ ### Terminal-Error Policy
356
+
357
+ Errors before the first record use the normal gateway error envelope because the response headers and status have not been committed yet. Errors mid-stream truncate the chunked response without a terminating zero-chunk; consumers detect this as an incomplete HTTP response and must treat it as a failure. NDJSON has no native error frame, and emitting a fake one would conflict with the item schema. This behavior is documented for operators in the [Production Guide](production.md#terminal-error-policy).
358
+
359
+ ## Companion Catalogs and Shape Resolution
360
+
361
+ Some vendor WSDLs split their types across multiple services. The stream wrapper operation lives in one WSDL while the concrete record type lives in a companion WSDL. The stream config's `shapeCatalogs` section names additional WSDL or catalog inputs used only to resolve record shapes.
362
+
363
+ ### How Resolution Works
364
+
365
+ When an operation names a `shapeCatalog`, the compiler loads the companion catalog once, copies the reachable record-type graph into the current compilation, and fails loudly on structural name collisions. Structurally identical types dedupe silently, so two catalogs that share a common base type do not conflict.
366
+
367
+ ```json
368
+ {
369
+ "shapeCatalogs": {
370
+ "main": { "wsdlSource": "https://api.example.com/Main.svc?singleWsdl" }
371
+ },
372
+ "operations": {
373
+ "StreamOp": {
374
+ "recordType": "ConcreteRecordType",
375
+ "recordPath": ["StreamOpResponse", "Records", "Record"],
376
+ "shapeCatalog": "main"
377
+ }
378
+ }
379
+ }
380
+ ```
381
+
382
+ ### Collision Handling
383
+
384
+ A structural collision means two types share a name but differ in fields. The build fails with a diagnostic naming both source catalogs. Rename in the companion source or point `recordType` at a distinct subtree. Silent renames are intentionally disallowed because they would produce ambiguous public APIs.
385
+
386
+ ## xs:any Wildcard Retention
387
+
388
+ XSD wildcards (`<xs:any>`) were silently dropped by earlier compiler versions. Since 0.17.0 the compiler retains them on the compiled type alongside any concrete children. This enables two downstream behaviors: honest stream-candidate detection (a wrapper that contains a wildcard is a likely streaming target) and accurate companion-catalog resolution (the compiler knows which elements are open for record types to slot into).
389
+
390
+ The retained wildcard is metadata only. It does not emit TypeScript `any` or loosen the generated types; concrete fields remain strictly typed.
@@ -65,6 +65,46 @@ Per-operation overrides for method, summary, description, and deprecation.
65
65
  }
66
66
  ```
67
67
 
68
+ ## Stream Configuration
69
+
70
+ The `--stream-config <file>` flag (available on `compile`, `client`, `pipeline`)
71
+ opts selected WSDL operations into streaming. Buffered output is unchanged for
72
+ operations not listed in the file.
73
+
74
+ ```json
75
+ {
76
+ "shapeCatalogs": {
77
+ "main": { "wsdlSource": "https://api.example.com/Main.svc?singleWsdl" }
78
+ },
79
+ "operations": {
80
+ "UnitDescriptiveInfoStream": {
81
+ "format": "ndjson",
82
+ "mediaType": "application/x-ndjson",
83
+ "recordType": "UnitDescriptiveContentType",
84
+ "recordPath": [
85
+ "UnitDescriptiveInfoStream",
86
+ "EVRN_UnitDescriptiveInfoRS",
87
+ "UnitDescriptiveContents",
88
+ "UnitDescriptiveContent"
89
+ ],
90
+ "shapeCatalog": "main"
91
+ }
92
+ }
93
+ }
94
+ ```
95
+
96
+ - `recordType` and `recordPath` are required; `format` defaults to `ndjson` and
97
+ `mediaType` to `application/x-ndjson`.
98
+ - `shapeCatalog` references a `shapeCatalogs` entry and is only needed when
99
+ the record type lives in a different WSDL than the one driving generation.
100
+ - Shape catalogs accept either `wsdlSource` (fetched and compiled on the fly)
101
+ or `catalogFile` (path to a pre-compiled `catalog.json`).
102
+ - Structural name collisions across catalogs fail the build; structurally
103
+ identical types dedupe silently.
104
+
105
+ See [ADR-002](decisions/002-streamable-responses.md) for rationale and the
106
+ terminal-error policy.
107
+
68
108
  ## Example Files
69
109
 
70
110
  Example configuration files are available in the `examples/openapi/` directory:
@@ -0,0 +1,308 @@
1
+ # ADR 002: Streamable SOAP Responses and JSON Gateway Output
2
+
3
+ Proposal for opt-in streamable SOAP response support, with Escapia EVRN Content Service as the concrete driver.
4
+
5
+ See the root [README](../../README.md) for the authoritative documentation index and [Architecture](../architecture.md) for the current generator pipeline.
6
+
7
+ ## Status
8
+
9
+ Accepted (2026-04-20). Implementation shipped in 0.17.0 across five phases
10
+ (Phase 0 research → Phase 5 integration). See
11
+ `scratches/plans/v017/streamable-responses.plan.yaml` for phase-by-phase
12
+ notes and verification gates.
13
+
14
+ ## Context
15
+
16
+ The current generator handles WSDL operations as buffered request and response calls. A generated client method awaits the SOAP runtime callback, receives a fully materialized result object, and returns `{ response, headers, responseRaw, requestRaw }`. The generated OpenAPI document describes each operation as `application/json`. The generated Fastify route calls the client method, optionally unwraps array wrapper types, and returns the standard response envelope.
17
+
18
+ Escapia exposes two related SOAP services:
19
+
20
+ - `EVRNService` contains the normal operation and type model.
21
+ - `EVRNContentService` contains content-oriented operations that are intended for large responses.
22
+
23
+ The Escapia content WSDL includes stream-like operations such as `UnitDescriptiveInfoStream` and `UnitCalendarAvailBatch`. Their output wrapper types contain schema and wildcard payload sections instead of clean concrete response members. The practical record shapes come from the main EVRN service WSDL, for example `UnitDescriptiveContentType` and `UnitCalendarAvailType`.
24
+
25
+ This means the current package has two gaps:
26
+
27
+ - It does not stream SOAP responses through generated clients or gateways.
28
+ - It does not let users map opaque WSDL response wrappers to concrete JSON record shapes from the same or a companion WSDL.
29
+
30
+ ## Current Architecture Review
31
+
32
+ The compile stage produces `catalog.json`, which is the only artifact that understands WSDL types, operation input and output elements, attribute metadata, child type metadata, and primitive mapping options.
33
+
34
+ The client stage generates a `node-soap` wrapper. Every operation is generated as a promise-returning method. The public operations interface mirrors that buffered shape, which is useful for mocks but cannot currently express an `AsyncIterable` or stream response.
35
+
36
+ The OpenAPI stage generates paths from catalog operations. It assumes every operation has a JSON request body and a JSON response body. Since version 0.7.1, successful responses are always wrapped in the standard envelope schema.
37
+
38
+ The gateway stage reads the OpenAPI document first and enriches operation metadata from `catalog.json` when the catalog is available. This lets route generation stay OpenAPI-driven, but stream support needs metadata that OpenAPI alone cannot represent.
39
+
40
+ The app and test generators assume ordinary JSON route responses. Generated tests use mock clients that implement the buffered operations interface.
41
+
42
+ ## Requirements
43
+
44
+ - Existing buffered generation must remain unchanged unless a user opts in.
45
+ - Stream support must be configurable per operation.
46
+ - Users must be able to map an opaque output wrapper to a concrete record type.
47
+ - Record types must be resolvable from the current catalog or from a companion catalog.
48
+ - Gateway output must be able to emit records before the full SOAP response has arrived.
49
+ - Generated routes must respect Node and Fastify backpressure.
50
+ - OpenAPI output must describe the wire media type without pretending that NDJSON is a single JSON object.
51
+ - Error handling must distinguish errors before the first byte from errors after streaming has started.
52
+ - The feature must be testable with a local chunked SOAP server, not only static XML fixtures.
53
+
54
+ ## Proposed Configuration Model
55
+
56
+ Add a stream configuration file passed with `--stream-config`. The same option should be available on `compile`, `client`, `openapi`, `gateway`, and `pipeline` so each stage can be run independently.
57
+
58
+ ```json
59
+ {
60
+ "shapeCatalogs": {
61
+ "evrn": {
62
+ "wsdlSource": "https://api.escapia.com/EVRNService.svc?singleWsdl"
63
+ }
64
+ },
65
+ "operations": {
66
+ "UnitDescriptiveInfoStream": {
67
+ "mode": "stream",
68
+ "format": "ndjson",
69
+ "mediaType": "application/x-ndjson",
70
+ "recordType": "UnitDescriptiveContentType",
71
+ "recordPath": [
72
+ "UnitDescriptiveInfoStream",
73
+ "EVRN_UnitDescriptiveInfoRS",
74
+ "UnitDescriptiveContents",
75
+ "UnitDescriptiveContent"
76
+ ],
77
+ "shapeCatalog": "evrn"
78
+ },
79
+ "UnitCalendarAvailBatch": {
80
+ "mode": "stream",
81
+ "format": "ndjson",
82
+ "mediaType": "application/x-ndjson",
83
+ "recordType": "UnitCalendarAvailType",
84
+ "recordPath": [
85
+ "EVRN_UnitCalendarAvailBatchRS",
86
+ "EVRN_UnitCalendarAvailBatchRS",
87
+ "UnitCalendarAvails",
88
+ "UnitCalendarAvail"
89
+ ],
90
+ "shapeCatalog": "evrn"
91
+ }
92
+ }
93
+ }
94
+ ```
95
+
96
+ `shapeCatalogs` names additional WSDL or catalog inputs used only to resolve stream record shapes.
97
+
98
+ `operations` marks specific WSDL operations as streamed. Operations not listed keep the existing buffered behavior.
99
+
100
+ `recordPath` is an ordered XML element path from the SOAP body payload to the repeated record element. Duplicate local names must be allowed because Escapia uses nested elements with the same name in at least one response.
101
+
102
+ `recordType` is the TypeScript and schema model to emit for each streamed record.
103
+
104
+ `format` should support `ndjson` first. `json-array` can be added later for clients that require a single JSON array response.
105
+
106
+ ## CLI and Programmatic API
107
+
108
+ The CLI should add `--stream-config <file>` to all generation commands. The pipeline command should pass the parsed and normalized configuration to every stage.
109
+
110
+ The programmatic API should add optional stream configuration fields to:
111
+
112
+ ```typescript
113
+ export interface PipelineOptions {
114
+ streamConfigFile?: string;
115
+ streamConfig?: StreamConfig;
116
+ }
117
+
118
+ export interface GenerateOpenAPIOptions {
119
+ streamConfigFile?: string;
120
+ streamConfig?: StreamConfig;
121
+ }
122
+
123
+ export interface GenerateGatewayOptions {
124
+ streamConfigFile?: string;
125
+ streamConfig?: StreamConfig;
126
+ }
127
+ ```
128
+
129
+ The compiler options should accept normalized stream metadata only when compilation needs to persist it into the catalog.
130
+
131
+ ## Catalog Model
132
+
133
+ The compiled catalog should carry normalized stream metadata beside operation metadata:
134
+
135
+ ```typescript
136
+ export interface Operation {
137
+ name: string;
138
+ inputTypeName?: string;
139
+ outputTypeName?: string;
140
+ stream?: OperationStreamMetadata;
141
+ }
142
+
143
+ export interface OperationStreamMetadata {
144
+ mode: "stream";
145
+ format: "ndjson" | "json-array";
146
+ mediaType: string;
147
+ recordPath: string[];
148
+ recordTypeName: string;
149
+ shapeCatalogName?: string;
150
+ sourceOutputTypeName?: string;
151
+ }
152
+ ```
153
+
154
+ The compiler should also represent wildcard schema particles explicitly. Today Escapia stream wrappers compile into misleading `xs:schema` properties and drop the wildcard payload. A better model is to retain both the schema marker and a wildcard marker so diagnostics and stream candidate detection are honest.
155
+
156
+ ## Shape Resolution
157
+
158
+ The first implementation should support companion catalogs in one of two forms:
159
+
160
+ - A `catalogFile` path for a catalog already produced by `wsdl-tsc compile`.
161
+ - A `wsdlSource` path or URL that the pipeline compiles before normalizing stream metadata.
162
+
163
+ For generated types, the MVP should copy the reachable record type graph from the companion catalog into the current generated output when there is no name collision. If a collision occurs, generation should fail with a clear diagnostic instead of silently renaming public API types.
164
+
165
+ A future version can support external type imports from another generated client directory.
166
+
167
+ ## Client Runtime
168
+
169
+ Generated clients should keep the existing buffered method signatures for ordinary operations.
170
+
171
+ Stream operations should return a separate response type:
172
+
173
+ ```typescript
174
+ export type StreamOperationResponse<RecordType, HeadersType = Record<string, unknown>> = {
175
+ records: AsyncIterable<RecordType>;
176
+ headers: HeadersType;
177
+ requestRaw?: string;
178
+ };
179
+ ```
180
+
181
+ The operations interface should use the same stream response type so generated gateway tests and user mocks can implement streaming behavior without importing the concrete SOAP client.
182
+
183
+ The transport layer should be proven by a chunked integration test. If `node-soap` with stream or SAX options can deliver records incrementally, it can be used for the MVP. If it buffers before yielding, the feature must use a dedicated SOAP HTTP transport for stream operations before release.
184
+
185
+ > **Phase 0 research outcome (2026-04-20):** `node-soap` **buffers** the full response before invoking the operation callback. Measured with `test/research/node-soap-streaming.test.ts` against a chunked HTTP fixture: first chunk flushed at ~48 ms, server closed at ~701 ms, node-soap callback fired at ~659 ms (~40 ms before close — that is parse time, not streaming). Generated stream operations therefore use a dedicated streaming HTTP transport emitted into the client runtime; `node-soap` remains the transport for buffered operations only. SAX streaming was validated with `saxes` 6.0.0; `test/research/sax-record-path.test.ts` chunk-fuzzes the Escapia-shaped XML (duplicate `EVRN_UnitDescriptiveInfoRS` wrappers) across every single-byte split and every byte-pair split and produces identical records each time. See `scratches/plans/v017/streamable-responses.plan.yaml` for the full findings.
186
+
187
+ ## XML to JSON Streaming Conversion
188
+
189
+ The runtime needs a streaming XML parser. `fast-xml-parser` is suitable for buffered documents but not for this use case. A SAX-style parser should track the configured `recordPath`, build a record object as the target element closes, and yield each record immediately.
190
+
191
+ The converter must reuse catalog metadata for:
192
+
193
+ - XML attributes and the configured attributes key
194
+ - Child element type lookup
195
+ - Repeated elements
196
+ - Nillable elements
197
+ - Text content and `$value`
198
+ - Primitive mapping
199
+
200
+ The converter should collect SOAP faults, response errors, and warnings when they appear before the first record. After streaming starts, terminal errors cannot be represented as a normal JSON error envelope for NDJSON. The runtime should either emit a final error record with a documented extension shape or abort the stream and log the classified error.
201
+
202
+ ## OpenAPI Output
203
+
204
+ Stream operations should not use the standard success envelope for `200` responses. The response content should use the configured stream media type.
205
+
206
+ ```json
207
+ {
208
+ "description": "Successful streamed SOAP operation response",
209
+ "content": {
210
+ "application/x-ndjson": {
211
+ "schema": { "type": "string" },
212
+ "x-wsdl-tsc-stream": {
213
+ "format": "ndjson",
214
+ "itemSchema": {
215
+ "$ref": "#/components/schemas/UnitDescriptiveContentType"
216
+ }
217
+ }
218
+ }
219
+ }
220
+ }
221
+ ```
222
+
223
+ OpenAPI 3.1 cannot fully describe an NDJSON sequence as a standard JSON Schema document. The `x-wsdl-tsc-stream` extension makes the item schema explicit for generated gateways, documentation tools, and future SDK generators.
224
+
225
+ ## Gateway Output
226
+
227
+ Generated stream routes should call the stream operation and send a Node stream through Fastify.
228
+
229
+ ```typescript
230
+ import { Readable } from "node:stream";
231
+
232
+ handler: async (request, reply) => {
233
+ const client = fastify.escapiaContentClient;
234
+ const result = await client.UnitDescriptiveInfoStream(request.body);
235
+
236
+ reply.type("application/x-ndjson");
237
+ return reply.send(Readable.from(toNdjson(result.records)));
238
+ }
239
+ ```
240
+
241
+ The generated operation schema should keep request validation but omit the Fastify response serialization schema for streamed `200` responses. Fastify cannot serialize an unbounded stream with a normal JSON response schema.
242
+
243
+ The gateway runtime should add `toNdjson()` and, later, `toJsonArrayStream()` helpers. These helpers should be deterministic, backpressure-aware, and safe for large payloads.
244
+
245
+ ## Test Strategy
246
+
247
+ Add unit tests for stream config parsing, operation matching, catalog normalization, wildcard compiler metadata, and record path matching.
248
+
249
+ Add converter tests that split XML chunks across element boundaries to prove the parser does not rely on convenient chunking.
250
+
251
+ Add a local Escapia-like WSDL fixture with `xs:any` output wrappers and a companion WSDL fixture with the concrete record types.
252
+
253
+ Add an integration test with a fake SOAP HTTP server that writes one record, waits, writes another record, and then closes the envelope. The test should assert that the gateway emits the first NDJSON line before the upstream response completes.
254
+
255
+ Add snapshot tests for generated `client.ts`, `operations.ts`, OpenAPI output, gateway route files, and gateway runtime helpers.
256
+
257
+ Run `npm run smoke:pipeline` after implementation to verify ordinary buffered generation is unchanged.
258
+
259
+ ## Implementation Plan
260
+
261
+ 1. Define `StreamConfig`, parser, validation errors, and normalized operation metadata.
262
+ 2. Add `--stream-config` to the CLI and thread it through the programmatic API.
263
+ 3. Update the compiler to preserve wildcard particles and emit stream candidate diagnostics.
264
+ 4. Implement companion catalog loading and record type graph copying.
265
+ 5. Generate stream method signatures in `client.ts` and `operations.ts`.
266
+ 6. Add a streaming XML converter with catalog-aware JSON shape conversion.
267
+ 7. Generate OpenAPI stream responses with `x-wsdl-tsc-stream` metadata.
268
+ 8. Generate Fastify streaming handlers and runtime helpers.
269
+ 9. Add unit, snapshot, and chunked integration tests.
270
+ 10. Update README, CLI reference, configuration docs, gateway guide, testing guide, and supported patterns.
271
+
272
+ ## Acceptance Criteria
273
+
274
+ - Existing generated output is unchanged when no stream config is provided.
275
+ - A stream-configured Escapia content WSDL generates typed stream client methods.
276
+ - The generated gateway emits `application/x-ndjson` records incrementally.
277
+ - The generated OpenAPI document identifies stream operations and record schemas.
278
+ - The converter maps XML attributes, arrays, text values, and nillable values consistently with buffered responses.
279
+ - The chunked integration test proves the first record is sent before the full SOAP response is available.
280
+ - Stream route errors before the first byte use the normal gateway error envelope.
281
+ - Stream route errors after the first byte follow a documented terminal error policy.
282
+
283
+ ## Consequences
284
+
285
+ The generator gains a second response execution model. This increases complexity in the client, OpenAPI, gateway, and test generators, but keeps the complexity opt-in and operation-scoped.
286
+
287
+ The catalog becomes more important as the shared source of truth because OpenAPI alone cannot carry enough stream conversion metadata.
288
+
289
+ NDJSON becomes the recommended stream format because it is simple, broadly consumable, and does not require buffering a complete JSON array before sending data.
290
+
291
+ Companion catalogs are required for vendors that split stream wrappers and concrete record shapes across separate WSDLs.
292
+
293
+ ## Implementation Notes
294
+
295
+ Captured after the 0.17.0 ship for future maintainers:
296
+
297
+ - `--stream-config` is wired onto the `compile`, `client`, and `pipeline` CLI commands. It is intentionally not accepted on `openapi`, `gateway`, or `app` because those commands consume a pre-compiled `catalog.json` that already carries the normalized `OperationStreamMetadata`. The original proposal text that listed the flag on every command reflected the design intent; the shipped surface is narrower for that reason.
298
+ - `GenerateOpenAPIOptions` and `GenerateGatewayOptions` in the programmatic API do not carry stream-config fields for the same reason. `compileWsdlToProject` and `runGenerationPipeline` (PipelineOptions) do.
299
+ - The client stream transport is emitted from two templates (`clientStreamMethods.tpl.txt`, `operationsStreamHelper.tpl.txt`). They embed the `StreamOperationResponse<T>` type and a `callStream()` method that POSTs a hand-built SOAP envelope via global `fetch`, bypassing `node-soap` as required by the phase-0 finding.
300
+ - `saxes ^6.0.0` was promoted from devDependency to runtime dependency on the package, and added as a pinned dependency in the generated app scaffold so stream-enabled consumers install it automatically.
301
+ - The `json-array` format is reserved in the config parser but not yet implemented by the emitters. Entries using `format: "json-array"` parse successfully and can be used to forward-declare intent; they do not currently generate routes or client methods.
302
+
303
+ ## References
304
+
305
+ - Escapia EVRN API documentation: <https://eweb.escapia.com/distribution/api/evrn-api-documentation>
306
+ - Escapia EVRN service WSDL: <https://api.escapia.com/EVRNService.svc?WSDL>
307
+ - Escapia EVRN content service WSDL: <https://api.escapia.com/EVRNContentService.svc?WSDL>
308
+ - Escapia batch API support article: <https://support.escapia.com/articles/en_US/Article/HASW-Batch-API-Methods-EVRN?category=Website_Services>
@@ -81,6 +81,43 @@ export async function registerRoute_v1_weather_getcityforecastbyzip(fastify: Fas
81
81
  }
82
82
  ```
83
83
 
84
+ ### Streaming Handlers
85
+
86
+ Operations opted in via `--stream-config` emit an NDJSON response pipe
87
+ instead of the standard envelope. The generated handler streams records as
88
+ they arrive, and the Fastify response is flushed line-by-line with
89
+ backpressure (ADR-002).
90
+
91
+ ```typescript
92
+ import type { FastifyInstance } from "fastify";
93
+ import type { GetWeatherInformation } from "../../client/types.js";
94
+ import schema from "../schemas/operations/getweatherinformation.json" with { type: "json" };
95
+ import { toNdjson } from "../runtime.js";
96
+
97
+ // Response schema omitted: stream operations emit NDJSON, not a single JSON object
98
+ const { response: _response, ...routeSchema } = schema as Record<string, unknown>;
99
+
100
+ export async function registerRoute_v1_weather_getweatherinformation(fastify: FastifyInstance) {
101
+ fastify.route<{ Body: GetWeatherInformation }>({
102
+ method: "POST",
103
+ url: "/get-weather-information",
104
+ schema: routeSchema,
105
+ handler: async (request, reply) => {
106
+ const client = fastify.weatherClient;
107
+ const result = await client.GetWeatherInformation(request.body as GetWeatherInformation);
108
+ reply.type("application/x-ndjson");
109
+ return reply.send(toNdjson(result.records));
110
+ },
111
+ });
112
+ }
113
+ ```
114
+
115
+ The client method returns `StreamOperationResponse<RecordType>` with a
116
+ `records: AsyncIterable<RecordType>`. Errors raised before the first record
117
+ use the normal error envelope; errors raised mid-stream trip the chunked
118
+ response's truncation — consumers detect these via the absence of a clean
119
+ terminating zero-chunk.
120
+
84
121
  ## Error Handling
85
122
 
86
123
  The centralized error handler (runtime.ts) automatically classifies errors:
@@ -36,6 +36,23 @@ const info = await client.GetWeatherInformation({});
36
36
  console.log(info.GetWeatherInformationResult.WeatherDescriptions);
37
37
  ```
38
38
 
39
+ ## Calling Stream Operations
40
+
41
+ Operations opted into streaming with `--stream-config` return `StreamOperationResponse<RecordType>` instead of the buffered shape. `records` is a single-pass `AsyncIterable<RecordType>` that pulls bytes from the upstream SOAP response on demand, so memory stays bounded regardless of payload size.
42
+
43
+ ```typescript
44
+ const result = await client.UnitDescriptiveInfoStream({});
45
+ for await (const record of result.records) {
46
+ console.log(record);
47
+ }
48
+ ```
49
+
50
+ Because `records` is single-pass, each response can be iterated once. Headers captured before the first record are available on `result.headers`, and the serialized SOAP envelope that was sent upstream is on `result.requestRaw` when populated.
51
+
52
+ Streaming requires the `saxes` runtime dependency. The generated app scaffold pins `saxes ^6.0.0` automatically; consumers that integrate the generated client into their own project must install it explicitly.
53
+
54
+ See [Stream Configuration](configuration.md#stream-configuration) for how to opt specific operations in, and [ADR-002](decisions/002-streamable-responses.md) for the rationale and terminal-error policy.
55
+
39
56
  ## Attributes and Text Content
40
57
 
41
58
  When an element has both attributes and text content, use the $value convention:
@@ -134,6 +151,8 @@ Key features of the generated handlers:
134
151
  - Envelope wrapping: `buildSuccessEnvelope()` wraps the raw SOAP response in the standard `{ status, message, data, error }` envelope
135
152
  - Route file header comments include propagated operation summary and description when present
136
153
 
154
+ Stream-configured operations generate a different handler shape: the response serialization schema is omitted, `reply.type("application/x-ndjson")` is set, and `reply.send(toNdjson(result.records))` streams records with backpressure. See the [Gateway Guide Streaming Handlers section](gateway-guide.md#streaming-handlers) for the full example and terminal-error policy.
155
+
137
156
  See [Gateway Guide](gateway-guide.md) for the full architecture and [CLI Reference](cli-reference.md) for generation flags.
138
157
 
139
158
  ## Operations Interface
@@ -158,4 +177,6 @@ The gateway plugin accepts any `WeatherOperations` implementation, so you can pa
158
177
  app.register(weatherGateway, { client: mock });
159
178
  ```
160
179
 
180
+ For stream operations, mock implementations return an async-iterable `records` field. See the [Testing Guide](testing.md) for the full pattern with async generators.
181
+
161
182
  See [Testing Guide](testing.md) for full integration test patterns.