@onlineapps/conn-orch-validator 3.3.2 → 4.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +358 -0
- package/README.md +46 -6
- package/TESTING_STRATEGY.md +1 -1
- package/docs/DESIGN.md +18 -11
- package/package.json +2 -8
- package/src/CookbookTestRunner.js +155 -26
- package/src/CookbookTestUtils.js +12 -4
- package/src/ServiceReadinessValidator.js +35 -18
- package/src/ValidationOrchestrator.js +223 -126
- package/src/cli/biz-ci-gate.js +356 -7
- package/src/helpers/README.md +3 -55
- package/src/helpers/createServiceReadinessTests.js +5 -1
- package/src/index.js +5 -2
- package/src/mocks/MockMQClient.js +100 -15
- package/src/utils/bizCiGateContract.js +110 -21
- package/src/utils/connectorContract.js +96 -0
- package/src/utils/cookbookFormat.js +95 -0
- package/src/utils/deployContract.js +488 -0
- package/src/utils/envContract.js +417 -0
- package/src/utils/installContract.js +142 -0
- package/src/utils/integrationRun.js +297 -0
- package/src/utils/libCompat.js +158 -0
- package/src/utils/preValidation.js +137 -0
- package/src/utils/setupDatabase.js +154 -0
- package/src/utils/stepFailure.js +73 -0
- package/src/utils/testNamespace.js +104 -0
- package/src/validators/ServiceStructureValidator.js +195 -48
- package/src/config.js +0 -32
- package/src/defaults.js +0 -11
- package/src/helpers/createPreValidationTests.js +0 -326
- package/test-mq-flow.js +0 -72
- package/test-orchestrator.js +0 -95
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,364 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to this package. Follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) format.
|
|
4
4
|
|
|
5
|
+
## [Unreleased]
|
|
6
|
+
|
|
7
|
+
> ⚠ Neither this entry nor the connector-step work of `dcddb862` is published.
|
|
8
|
+
> The last version on NPM is 3.3.2, so **no biz CI run sees any of it yet**.
|
|
9
|
+
|
|
10
|
+
### Fixed — validator findings pointed at an archived document (DÁVKA 51)
|
|
11
|
+
|
|
12
|
+
`ServiceStructureValidator` sent readers of five findings to
|
|
13
|
+
`biz-service-invocation-model.md` — twice by name, three times under its `RFC`
|
|
14
|
+
nickname. The document was archived; following the pointer found nothing, so the
|
|
15
|
+
finding stayed unfixed. `automation-gates.md` §1 requirement 4 asks a gate for
|
|
16
|
+
*the exact command that fixes it*, and a dead citation is the opposite of that.
|
|
17
|
+
|
|
18
|
+
- `INVALID_OPERATIONS_STRUCTURE`, `SCHEMA_VERSION_MISMATCH`,
|
|
19
|
+
`MISSING_OPERATION_FIELD` and `V2_FIELD_PRESENT` now carry the fix in the text
|
|
20
|
+
and name the file to edit (`config/service/operations.json`) — no citation to
|
|
21
|
+
follow at all.
|
|
22
|
+
- The two `@see` headers above that code point at the live owner,
|
|
23
|
+
`api/docs/biz/30-operations/schema-v3.md`.
|
|
24
|
+
- A sweep test resolves every `.md` path in a `message`/`fix` string literal
|
|
25
|
+
against the checkout and fails on the first one that is not there, so the next
|
|
26
|
+
archived document is caught the day it moves.
|
|
27
|
+
|
|
28
|
+
### BREAKING — v1.2 checks the business-error CONTRACT, not the package name (DÁVKA 50)
|
|
29
|
+
|
|
30
|
+
**Major.** The v1.2 verdict changes for every service that still imports its
|
|
31
|
+
error classes from `@onlineapps/service-common`, and for every service whose
|
|
32
|
+
handlers do not reach a business-error declaration at all.
|
|
33
|
+
|
|
34
|
+
Owner decision: `api/docs/governance/confirmations/business-error-contract.md`
|
|
35
|
+
(`20260829-1500-business-error-contract-001`, CONFIRMED).
|
|
36
|
+
|
|
37
|
+
v1.2 used to ask *"does this service depend on `@onlineapps/service-common` and
|
|
38
|
+
import an error class FROM IT?"* — `dep_service_common` and
|
|
39
|
+
`business_error_usage`. Both demanded the hierarchy that DÁVKA 29 retired, so a
|
|
40
|
+
service that had already migrated to the wrapper's classes **failed the gate for
|
|
41
|
+
being correct**. A check keyed on a package name goes stale at every move; a
|
|
42
|
+
check keyed on the contract does not.
|
|
43
|
+
|
|
44
|
+
- **Removed** `dep_service_common`. `@onlineapps/service-common` is no longer
|
|
45
|
+
required in `dependencies` at any standard level.
|
|
46
|
+
- **Removed** `business_error_usage`, replaced by **`business_error_contract`**:
|
|
47
|
+
at least one file under `src/**` imports a `BusinessError` family from
|
|
48
|
+
`@onlineapps/service-wrapper`, **or** declares the brand
|
|
49
|
+
`onlineapps.businessError`. A service's own error class satisfies the check by
|
|
50
|
+
declaring the brand; inheriting from the wrapper's base class is the
|
|
51
|
+
convenient way to get it, not the only permitted one.
|
|
52
|
+
The scope is the whole of `src/`, deliberately — one rule in one sentence
|
|
53
|
+
(`automation-gates.md` §1 requirement 2, *Simple*). A first cut scoped it to
|
|
54
|
+
`src/handlers/**` plus the `src/lib/**` modules a handler requires; measured
|
|
55
|
+
against the eight services it failed `biz-converter` and `biz-ingest`, both of
|
|
56
|
+
which declare the contract in `src/lib/` and reach it through `src/services/`.
|
|
57
|
+
A static check cannot prove a `throw` reaches a handler in any case — it can
|
|
58
|
+
only see the declaration — so the narrower rule bought no accuracy and cost two
|
|
59
|
+
false findings.
|
|
60
|
+
- **Added** `no_retired_error_hierarchy`: no file under `src/**` imports an error
|
|
61
|
+
class from `@onlineapps/service-common`. The finding is
|
|
62
|
+
`[Context] Problem - Expected/Fix` and names **every** offending file with the
|
|
63
|
+
exact replacement import line. Scoped to what
|
|
64
|
+
`service-common/src/errors/BusinessError.js` exports — `ScopedRegistryError`
|
|
65
|
+
belongs to the live scoped-registry helper and is not part of the retired
|
|
66
|
+
hierarchy, so importing it stays legal.
|
|
67
|
+
- A v1.2 check that carries its own `fix` now emits that message verbatim in its
|
|
68
|
+
`STANDARD_LEVEL_GAP` warning, instead of the generic `missing …` wrapper that
|
|
69
|
+
would have buried the file list.
|
|
70
|
+
|
|
71
|
+
**Measured over all eight biz services before the change landed**
|
|
72
|
+
(`automation-gates.md` §3 — a gate arrives with the compliance work, never ahead
|
|
73
|
+
of it): `converter`, `emailer`, `hello-service`, `ingest`, `invoicing`,
|
|
74
|
+
`property` PASS both checks and reach v1.3; `meta` fails both — 8 files still on
|
|
75
|
+
the retired hierarchy, so it declares no wrapper contract either; `pdfgen` fails
|
|
76
|
+
`business_error_contract` alone, having no business-error class at all (every
|
|
77
|
+
throw in `src/handlers/pdf.js` is a plain `Error`, i.e. a masked 500 today). The
|
|
78
|
+
gate reaches the services with the wrapper release that carries the brand, so
|
|
79
|
+
each migrates its imports in the same change that bumps the wrapper.
|
|
80
|
+
|
|
81
|
+
**Defect found by that measurement, fixed here:** the import scanner matched the
|
|
82
|
+
destructuring body with `[\s\S]*?`, which starts at the nearest preceding
|
|
83
|
+
`const {` and swallows whole lines to reach the right `require`. On
|
|
84
|
+
`api_biz/meta/src/handlers/persons.js:10-11` — an unrelated `models`
|
|
85
|
+
destructuring directly above the service-common one — that silently dropped
|
|
86
|
+
`ValidationError` from the finding. The body is now `[^{}]*?`.
|
|
87
|
+
|
|
88
|
+
### Added — the environment contract (F16)
|
|
89
|
+
|
|
90
|
+
**A service declares the environment it needs, in the file where it already
|
|
91
|
+
declares its connectors and its database.** The platform had an enforcement
|
|
92
|
+
engine (`@onlineapps/runtime-config` schemas throw on a missing required key)
|
|
93
|
+
and no declaration: of 56 variables measured across the eight biz services,
|
|
94
|
+
21 were enforced by `ConfigLoader`, 5 by the connector contract and the rest by
|
|
95
|
+
nothing at all. So `EMAIL_PROVIDER_LIVE_MODE` could be read by biz-emailer while
|
|
96
|
+
no template ever set it — live mail was silently off, and nothing said a word.
|
|
97
|
+
|
|
98
|
+
- **`integration-contract.json` gains an optional `env` block** —
|
|
99
|
+
`env.required[]` / `env.optional[]`, items `{ name, why }`. `why` is
|
|
100
|
+
mandatory. `normalizeIntegrationContract()` validates the shape strictly:
|
|
101
|
+
unknown keys, a non-array list, a bare string item, a name that is not
|
|
102
|
+
SCREAMING_SNAKE_CASE, a missing `why`, a duplicate name, and a name already
|
|
103
|
+
covered by a `${VAR}` config placeholder or by a required connector are all
|
|
104
|
+
hard errors naming the fix.
|
|
105
|
+
- **`oa-biz-ci-gate verify-env-contract`** (new subcommand, and folded into
|
|
106
|
+
`verify-contract` between the installation contract and R6) checks
|
|
107
|
+
COMPLETENESS: every environment name the service repository visibly reads is
|
|
108
|
+
declared or covered. It prints its measurement boundary on every run —
|
|
109
|
+
`config/service/*.json` plus literal `process.env.X` / `env.X` /
|
|
110
|
+
`requireEnv('X')` in `src/`, `index.js`, `scripts/`, and the number of dynamic
|
|
111
|
+
`process.env[key]` sites it could not resolve. Environment read inside
|
|
112
|
+
`node_modules` is out of reach and is declared by hand (variant D of the
|
|
113
|
+
design would close that).
|
|
114
|
+
- **`ValidationOrchestrator` step 3, "Environment Contract"** checks PRESENCE
|
|
115
|
+
before any handler runs and long before a connector opens: a declared
|
|
116
|
+
`required` name that is unset fails with
|
|
117
|
+
`[EnvContract] Missing environment variable - <NAME> is required (<why>). Fix: …`.
|
|
118
|
+
This is defect D2 — `SECRETS_MASTER_KEY` used to fail in
|
|
119
|
+
`ServiceWrapper.js:916`, after MQ registration, where the symptom looked like
|
|
120
|
+
a broken registration.
|
|
121
|
+
- **A contract with no `env` block is skipped OUT LOUD** by both consumers. That
|
|
122
|
+
state is legitimate until every repo adopts the declaration; a step that says
|
|
123
|
+
nothing about what it did not check reads as a guarantee it never gave
|
|
124
|
+
(`automation-gates.md` §5).
|
|
125
|
+
|
|
126
|
+
Measured against the eight live repos on 2026-08-27 (read-only): biz-invoicing
|
|
127
|
+
and biz-pdfgen pass with nothing declared; biz-emailer reports 13 undeclared
|
|
128
|
+
reads, biz-hello 6, biz-meta 2, biz-converter / biz-ingest / biz-property 1 each.
|
|
129
|
+
That is the gate doing its job before adoption, not a regression — it lands with
|
|
130
|
+
the declarations, per `automation-gates.md` §3 (no grandfathering).
|
|
131
|
+
|
|
132
|
+
### BREAKING — step numbering
|
|
133
|
+
|
|
134
|
+
- `ValidationOrchestrator` announces `Step n/6`; `results.steps` keys are
|
|
135
|
+
`structure, config, env, operations, cookbooks, connectors`.
|
|
136
|
+
|
|
137
|
+
**Tier-1 drops to five steps: the readiness step is removed, and with it the last `url`.**
|
|
138
|
+
|
|
139
|
+
### BREAKING — `ValidationOrchestrator`
|
|
140
|
+
|
|
141
|
+
- **Step 5 "Service Readiness" removed.** The remaining steps are renumbered and
|
|
142
|
+
the log now reads `Step n/5`. `results.steps.readiness` is no longer emitted;
|
|
143
|
+
`results.steps` keys are `structure, config, operations, cookbooks, connectors`.
|
|
144
|
+
- `ValidationOrchestrator.validateReadiness()` removed; the orchestrator no
|
|
145
|
+
longer constructs a `ServiceReadinessValidator`.
|
|
146
|
+
- `options.serviceUrl` is **no longer required** and is no longer stored. It is
|
|
147
|
+
accepted and ignored, so `ServiceWrapper` and the biz
|
|
148
|
+
`scripts/run-pre-validation.js` need no change. Previously a *required*
|
|
149
|
+
option that no line of code read — a service with `SERVICE_URL` unset failed
|
|
150
|
+
Phase 0.2 for the sake of a value nobody wanted.
|
|
151
|
+
|
|
152
|
+
Why the step went rather than shrinking: its verdict was a strict function of
|
|
153
|
+
steps 2 and 3. Its per-operation rules were step 3's rules, copied loop for
|
|
154
|
+
loop; its two extra guards ("operations must be an object", "no operations
|
|
155
|
+
defined") are step 2's `operations.json has no operations defined`. Across a
|
|
156
|
+
valid baseline plus 13 mutations covering every rule it never once disagreed
|
|
157
|
+
with `step2 && step3`, and against the eight live biz services it emitted a
|
|
158
|
+
single check, scored 80 out of a `maxScore` of 100 this call path could never
|
|
159
|
+
reach, and compared it to a `score >= 60` threshold the one remaining weight had
|
|
160
|
+
made decorative. Measured 0 ms. It could not fail a service the earlier steps
|
|
161
|
+
passed, so it announced a sixth verdict it never independently reached — the
|
|
162
|
+
defect the connector step was fixed for in `dcddb862`, one step earlier.
|
|
163
|
+
|
|
164
|
+
### Changed — nothing lost with it
|
|
165
|
+
|
|
166
|
+
- **Step 3 now warns on an operation with no `description`.** That was the
|
|
167
|
+
readiness step's only non-duplicate signal, and it never reached anyone: the
|
|
168
|
+
step returned `{valid, errors}` and dropped `warnings`. `validateOperations()`
|
|
169
|
+
now returns `warnings` and `runFullValidation()` puts them in
|
|
170
|
+
`results.warnings`.
|
|
171
|
+
|
|
172
|
+
### Fixed
|
|
173
|
+
|
|
174
|
+
- **`runFullValidation()` crashed on a failing connector step.** `dcddb862`
|
|
175
|
+
changed `validateConnectors()` to return `{valid, errors}`; the caller still
|
|
176
|
+
spread `.warnings`. Any service that actually failed the contract — the one
|
|
177
|
+
case the step exists for — aborted with
|
|
178
|
+
`results.steps.connectors.warnings is not iterable` and lost every finding.
|
|
179
|
+
Reproduced against biz-pdfgen and biz-property. Severity is unchanged: the
|
|
180
|
+
connector contract stays non-critical, reported as warnings.
|
|
181
|
+
- **Per-check score is recorded.** `createServiceReadinessTests` prints
|
|
182
|
+
`checkResult.score`; nothing ever assigned it, so every biz bootstrap run
|
|
183
|
+
printed `0 points` beside checks that had just been awarded full weight,
|
|
184
|
+
under a total that contradicted them.
|
|
185
|
+
|
|
186
|
+
### `ServiceReadinessValidator` — kept, `url` gone
|
|
187
|
+
|
|
188
|
+
The class stays: its cookbook and registry checks are used by
|
|
189
|
+
`createServiceReadinessTests`, and through it by `tests/bootstrap/` in
|
|
190
|
+
`api_biz/hello-service`, `api_biz/converter` and `api_biz/ingest`. That helper's
|
|
191
|
+
signature is unchanged, and it never passed a `url`.
|
|
192
|
+
|
|
193
|
+
- `service.url` no longer destructured; `results.serviceUrl` no longer emitted;
|
|
194
|
+
`generateReport()` no longer prints a `URL:` line naming an address nothing
|
|
195
|
+
listens on.
|
|
196
|
+
|
|
197
|
+
### Tests
|
|
198
|
+
|
|
199
|
+
Unit suite 283 → **322 GREEN**. `ServiceReadinessValidator` had no unit tests at
|
|
200
|
+
all — which is how a dead parameter survived the removal of its probe; it now
|
|
201
|
+
has 22, pinning every operations rule explicitly so a future divergence from
|
|
202
|
+
step 3 is visible rather than silent.
|
|
203
|
+
|
|
204
|
+
### Added — the biz `scripts/*` duplication moves into the gate (F15#3)
|
|
205
|
+
|
|
206
|
+
Two roles that lived as copied per-service scripts become subcommands. Nothing
|
|
207
|
+
in any service repo changes until its pin is bumped; activation is F8.
|
|
208
|
+
|
|
209
|
+
- **`verify-install-contract`** — the setup-doc package every repo carries, plus
|
|
210
|
+
the `migrations/BASELINE` + `SEED/{production_like,test_only}` SQL tree and its
|
|
211
|
+
four headers, required of every repo whose contract declares a database.
|
|
212
|
+
Replaces nine copies of `scripts/verify-installation-docs-sql-contract.sh`
|
|
213
|
+
(eight biz repos + `api/`).
|
|
214
|
+
**Whether the SQL half applies is now DERIVED from `integration-contract.json`**
|
|
215
|
+
instead of a hand-maintained `REPO_HAS_DB` literal per copy. The literal had
|
|
216
|
+
drifted: converter, hello-service and ingest declared `false` while their
|
|
217
|
+
contract declares a database, so the SQL half silently checked nothing in three
|
|
218
|
+
repos while still printing `PASS`. `docs/setup/` is the single accepted
|
|
219
|
+
location — property's `docs/80-setup/` does not satisfy it.
|
|
220
|
+
- Folded into **`verify-contract`**, on the precedent of `e538fc8d`: a version
|
|
221
|
+
bump activates it, with no per-repo CI edit to forget.
|
|
222
|
+
- **`run-prevalidation`** — runs the service's cookbooks offline against mocked
|
|
223
|
+
infrastructure and writes `conn-runtime/validation-proof.json`. Replaces five
|
|
224
|
+
copies of `scripts/run-pre-validation.js`: four identical, plus property's
|
|
225
|
+
16-line stand-in that counted `.json` files and printed OK without running a
|
|
226
|
+
cookbook or writing a proof, while wired to the same `npm run test:cookbooks`.
|
|
227
|
+
The legacy HTTP dispatch branch is not carried over — every service is v3, so
|
|
228
|
+
it was unreachable; a non-v3 shape now fails fast instead of silently taking
|
|
229
|
+
the handler path.
|
|
230
|
+
`ConfigLoader` is **not** imported here: it lives in `@onlineapps/service-wrapper`
|
|
231
|
+
(L4) and this package is L3, so the service URL is resolved from the service's
|
|
232
|
+
own installed wrapper and injected (architecture-principles §7).
|
|
233
|
+
|
|
234
|
+
### Fixed — a failed cookbook step said `[object Object]`
|
|
235
|
+
|
|
236
|
+
The scripts interpolated a step's `error` straight into a template literal, so
|
|
237
|
+
an object-shaped failure told the operator nothing. It is now rendered by
|
|
238
|
+
message, then by serialization. On the first real run this turned
|
|
239
|
+
`[object Object]` into `getaddrinfo ENOTFOUND gen_mariadb10.5`.
|
|
240
|
+
|
|
241
|
+
### Tests
|
|
242
|
+
|
|
243
|
+
Suite 388 → **422 GREEN** (25 → 30 suites): 21 unit + 13 integration, the latter
|
|
244
|
+
driving the real CLI as a child process and the real `CookbookTestRunner` over
|
|
245
|
+
real v3 handlers. Both new modules were mutation-falsified; removing the
|
|
246
|
+
`verify-contract` fold-in turns exactly one test red.
|
|
247
|
+
|
|
248
|
+
### Added — the cookbook format check is mandatory, not opt-in (F17)
|
|
249
|
+
|
|
250
|
+
`version` is a REQUIRED cookbook field, `"2.1.0"` or higher
|
|
251
|
+
(`api/docs/biz/40-cookbooks/format.md` § Required fields). Until now that
|
|
252
|
+
sentence was implemented only in `CookbookTestUtils.validateCookbook`, which is
|
|
253
|
+
reachable only from `tests/bootstrap/service-readiness.test.js` — a wrapper 3 of
|
|
254
|
+
8 biz repos happen to have. The Tier-1 path (`CookbookTestRunner`, step 4 of
|
|
255
|
+
`ValidationOrchestrator`) validated steps and never looked at `version`, so
|
|
256
|
+
**14 of 28 cookbooks across 4 repos (emailer, invoicing, pdfgen, property) carry
|
|
257
|
+
no `version` at all** and every boot said PASS.
|
|
258
|
+
|
|
259
|
+
- New `src/utils/cookbookFormat.js` — one owner for the rule:
|
|
260
|
+
`MIN_COOKBOOK_FORMAT_VERSION = '2.1.0'` and `checkCookbookFormatVersion()`
|
|
261
|
+
returning the problem in `Problem - Expected/Fix` form, or `null`.
|
|
262
|
+
- `CookbookTestRunner.validateCookbook()` checks the format version FIRST, and
|
|
263
|
+
rejects a payload that is not a JSON object at all. No signature change.
|
|
264
|
+
- `CookbookTestRunner.runCookbook()` appends `(cookbook file: <path>)` to
|
|
265
|
+
whatever validation rejected, when the cookbook came from a file — a service
|
|
266
|
+
carries a dozen of them and "a cookbook is malformed" named none.
|
|
267
|
+
- `CookbookTestRunner.loadCookbook()` — a syntax error in a cookbook file now
|
|
268
|
+
fails as `[CookbookTestRunner] Cookbook file is not valid JSON …` naming the
|
|
269
|
+
file, instead of a bare `Unexpected token` from `JSON.parse`.
|
|
270
|
+
- `CookbookTestUtils.validateCookbook()` uses the same shared rule, so the
|
|
271
|
+
readiness path stops accepting an outdated version on presence alone.
|
|
272
|
+
- The cookbooks this package itself synthesises (`CookbookTestUtils`
|
|
273
|
+
generators, `createServiceReadinessTests`) said `version: '1.0.0'` — the
|
|
274
|
+
library emitted cookbooks its own gate rejects. They now use the constant.
|
|
275
|
+
|
|
276
|
+
Nothing changes in the biz repos until F8 bumps the pin; the 3 readiness
|
|
277
|
+
wrappers stay until then (removing them ahead of activation would reopen the
|
|
278
|
+
hole this closes).
|
|
279
|
+
|
|
280
|
+
### Tests
|
|
281
|
+
|
|
282
|
+
Suite 422 → **449 GREEN** (30 → 33 suites). RED before the change: a
|
|
283
|
+
version-less and an outdated cookbook both passed Tier-1 step 4
|
|
284
|
+
(`success: true`); the readiness helper scored 100/100 while synthesising a
|
|
285
|
+
`1.0.0` cookbook. New integration suite drives the real
|
|
286
|
+
`ValidationOrchestrator.runCookbookTests()` over real cookbook files on disk,
|
|
287
|
+
and the real `createServiceReadinessTests` over a real service tree — control
|
|
288
|
+
cases assert the existing fixture cookbook still runs unchanged (2 steps, both
|
|
289
|
+
passed) and that the structural messages are untouched.
|
|
290
|
+
|
|
291
|
+
### Fixed — three defects behind biz-hello's oscillating boot verdict (2026-08-29)
|
|
292
|
+
|
|
293
|
+
Reported from the publication wave as two open findings: a `count_stub` that
|
|
294
|
+
failed at 1101–5593 ms and passed at 0–916 ms in the same container, and a
|
|
295
|
+
series of `Validation FAILED` in 104–221 ms that nobody could explain. They are
|
|
296
|
+
one causal chain and three separate defects.
|
|
297
|
+
|
|
298
|
+
- **`expect.duration.max` measured the runner's cold start, not the handler.**
|
|
299
|
+
`executeStep` started its stopwatch before `resolveOperation()` and before
|
|
300
|
+
`require()` of the handler module, so the module-load cost was charged to the
|
|
301
|
+
handler. biz-hello's `count_stub` is a stub returning four constants that
|
|
302
|
+
never touches the DB, but it is the only cookbook step in that service whose
|
|
303
|
+
module pulls in `sequelize` + `src/config/database.js`. Measured in the
|
|
304
|
+
running container with the wrapper already loaded, as at boot: `secretLookup`
|
|
305
|
+
25 ms, `greetings` 8 ms, `externalHttpPing` 5 ms, `storeAndFetchNote` 9 ms,
|
|
306
|
+
**`tenantGreetingCount` 2264 ms**. Against a 1000 ms bar the verdict on a
|
|
307
|
+
healthy service therefore depended on filesystem cache warmth — not
|
|
308
|
+
predictable, so not a gate (`automation-gates.md` §1.1). A step now reports
|
|
309
|
+
three numbers: `setupDurationMs` (the runner's own cost), `handlerDurationMs`
|
|
310
|
+
(the handler's runtime) and `duration` (the unchanged total); the expectation
|
|
311
|
+
compares `handlerDurationMs`, and the load cost is printed on the step's log
|
|
312
|
+
line rather than swept away.
|
|
313
|
+
- **A reused runner accumulated results forever.** `ValidationOrchestrator`
|
|
314
|
+
builds its `CookbookTestRunner` once in the constructor, and
|
|
315
|
+
`ServiceWrapper._ensureValidationProof` reuses ONE orchestrator across its
|
|
316
|
+
whole startup retry loop (six attempts, 30 s → 30 min). `runCookbooks()` only
|
|
317
|
+
ever added to `this.results`, so one transient failure on attempt 1 stayed in
|
|
318
|
+
`failed` for the life of the process and `success: result.failed === 0` could
|
|
319
|
+
never return true again. The whole cycle is legible in biz-hello's own log —
|
|
320
|
+
one process, six attempts, the deficit of 1 riding along and the total
|
|
321
|
+
climbing by 8 each time, while the step that started it passes in single-digit
|
|
322
|
+
milliseconds:
|
|
323
|
+
|
|
324
|
+
```
|
|
325
|
+
7/8 passed ❌ FAILED (3840ms) count_stub FAILED (3297ms) ← cold require
|
|
326
|
+
15/16 passed ❌ FAILED (155ms) count_stub PASSED (5ms)
|
|
327
|
+
23/24 passed ❌ FAILED (92ms) count_stub PASSED (4ms)
|
|
328
|
+
31/32 passed ❌ FAILED (160ms) count_stub PASSED (4ms)
|
|
329
|
+
39/40 passed ❌ FAILED (103ms) count_stub PASSED (2ms)
|
|
330
|
+
47/48 passed ❌ FAILED (94ms) count_stub PASSED (3ms)
|
|
331
|
+
8/8 passed ✅ PASSED (556ms) count_stub PASSED (498ms) ← new process
|
|
332
|
+
```
|
|
333
|
+
|
|
334
|
+
Six attempts is exactly the length of `REVALIDATION_FAST_BACKOFF_MS`. The
|
|
335
|
+
92–221 ms runs are the false verdict in its purest form: full runs in which
|
|
336
|
+
every step passed, reported as failures, for ~18.5 minutes before the process
|
|
337
|
+
gave up and exited. `resetResults()` had existed the whole time, with a unit
|
|
338
|
+
test and **no production caller** (`change-discipline.md` § Removing something
|
|
339
|
+
removes its declaration); `runCookbooks()` is now that caller.
|
|
340
|
+
- **A failed step never said why.** The log line was `Step count_stub: FAILED
|
|
341
|
+
(1101ms)` and step 4's only error was `1 cookbook test(s) failed` — the reason
|
|
342
|
+
lived in `result.validationErrors` / `result.error` and was discarded at both
|
|
343
|
+
ends, which made a failed boot undiagnosable from its own logs
|
|
344
|
+
(`automation-gates.md` §5). New `utils/stepFailure.js` is the single owner of
|
|
345
|
+
that sentence: the runner prints it (with the stack, for a thrown error) and
|
|
346
|
+
the orchestrator's error list names the cookbook, the step, the operation and
|
|
347
|
+
the reason.
|
|
348
|
+
|
|
349
|
+
### Tests
|
|
350
|
+
|
|
351
|
+
Suite 560 → **572 GREEN** (40 → 43 suites), exit 0. Three new integration
|
|
352
|
+
suites, each RED first against real cookbook files, a real `operations.json`
|
|
353
|
+
and real handler modules required out of a freshly copied service tree:
|
|
354
|
+
`cookbookFailureDiagnostics` (3 of 4 RED — measured
|
|
355
|
+
`info Step throwing-step: FAILED (0ms)` with no reason), `cookbookRepeatedRuns`
|
|
356
|
+
(5 of 5 RED — the second run reported `total: 2`), `cookbookStepDuration`
|
|
357
|
+
(3 of 3 RED — a healthy stub behind a 400 ms module load failed a 150 ms bar).
|
|
358
|
+
Control cases assert what must NOT move: a permanently broken cookbook still
|
|
359
|
+
fails on the second run, a healthy one reports identical numbers on every run,
|
|
360
|
+
a genuinely slow handler still fails the same bar, and a passing cookbook logs
|
|
361
|
+
no failure reason at all.
|
|
362
|
+
|
|
5
363
|
## [3.3.2] — 2026-08-17
|
|
6
364
|
|
|
7
365
|
**ServiceStructureValidator refactor for ADR 0005 / Fáze F6 reality.**
|
package/README.md
CHANGED
|
@@ -42,13 +42,52 @@ await wrapper.initialize();
|
|
|
42
42
|
|
|
43
43
|
1. **Service Structure** - directories and files exist
|
|
44
44
|
2. **Config Files** - valid JSON, required fields
|
|
45
|
-
3. **
|
|
46
|
-
4. **
|
|
47
|
-
5. **
|
|
48
|
-
6. **Connector Integration** -
|
|
45
|
+
3. **Environment Contract** - every variable the contract declares `env.required` is set
|
|
46
|
+
4. **Operations Compliance** - follows OPERATIONS.md standard
|
|
47
|
+
5. **Cookbook Tests** - business logic + integration (MOCKED infra)
|
|
48
|
+
6. **Connector Integration** - the connector declarations agree and the environment backs them
|
|
49
49
|
|
|
50
50
|
**Output:** Validation proof saved to `conn-runtime/validation-proof.json`
|
|
51
51
|
|
|
52
|
+
> The list said "Service Readiness — HTTP API works" until 2026-08-27. That step
|
|
53
|
+
> was removed on 2026-08-22 under ADR 0005 and this README kept describing it.
|
|
54
|
+
|
|
55
|
+
---
|
|
56
|
+
|
|
57
|
+
## Environment contract (`env` block)
|
|
58
|
+
|
|
59
|
+
One declaration in `config/service/integration-contract.json`, two consumers:
|
|
60
|
+
|
|
61
|
+
```json
|
|
62
|
+
"env": {
|
|
63
|
+
"required": [{ "name": "SECRETS_MASTER_KEY", "why": "decrypts per-tenant secrets from SecretBox" }],
|
|
64
|
+
"optional": [{ "name": "SES_REGION", "why": "AWS region when the SES provider is live" }]
|
|
65
|
+
}
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
| Consumer | Direction | Where |
|
|
69
|
+
|---|---|---|
|
|
70
|
+
| `oa-biz-ci-gate verify-env-contract` (also folded into `verify-contract`) | COMPLETENESS — nothing the repo reads is undeclared | CI |
|
|
71
|
+
| `ValidationOrchestrator` step 3 | PRESENCE — every `required` name is set | boot, phase 0.2 |
|
|
72
|
+
|
|
73
|
+
`why` is mandatory: a required variable nobody can justify is the one nobody
|
|
74
|
+
dares delete.
|
|
75
|
+
|
|
76
|
+
**The block must not repeat what already covers a name** — a `${VAR}`
|
|
77
|
+
placeholder in `config/service/*.json`, or the endpoint variable of a connector
|
|
78
|
+
the contract declares required. Both are rejected as a duplicate declaration.
|
|
79
|
+
|
|
80
|
+
**Measurement boundary, printed on every run.** The completeness scan reads
|
|
81
|
+
`config/service/*.json` and literal `process.env.X` / `env.X` /
|
|
82
|
+
`requireEnv('X')` in `src/`, `index.js` and `scripts/`. It cannot see dynamic
|
|
83
|
+
access (`process.env[key]` — counted and reported, never guessed) or environment
|
|
84
|
+
read inside an installed library, so a library-level variable such as
|
|
85
|
+
`SECRETS_MASTER_KEY` is declared by hand until libraries export their own env
|
|
86
|
+
schemas. The scan proves one direction only: what it CAN see is declared.
|
|
87
|
+
|
|
88
|
+
A contract with no `env` block is skipped out loud by both consumers — a
|
|
89
|
+
legitimate state until every repo adopts the declaration, and never silent.
|
|
90
|
+
|
|
52
91
|
---
|
|
53
92
|
|
|
54
93
|
## Runtime Directory Structure
|
|
@@ -103,7 +142,7 @@ The validator evaluates each service against cumulative implementation standards
|
|
|
103
142
|
|-------|------|--------|-------|
|
|
104
143
|
| **v1.0** | Base Service Standard | `conn-config/`, `src/app.js`, `index.js`, valid `config.json` + `operations.json`, `@onlineapps/service-wrapper` dep | 2025-06 |
|
|
105
144
|
| **v1.1** | Multitenancy Standard | `wrapper.tenantContext` configured in `config.json` | 2026-03 |
|
|
106
|
-
| **v1.2** | Business Error Handling | `@onlineapps/service-
|
|
145
|
+
| **v1.2** | Business Error Handling | `business_error_contract` (some file under `src/**` imports a `BusinessError` family from `@onlineapps/service-wrapper` or declares the brand `onlineapps.businessError`) + `no_retired_error_hierarchy` (no error class imported from `@onlineapps/service-common` anywhere under `src/`) | 2026-03 |
|
|
107
146
|
|
|
108
147
|
**Key properties:**
|
|
109
148
|
- **Cumulative** — v1.2 requires v1.0 + v1.1 to also pass
|
|
@@ -118,13 +157,14 @@ const { ServiceStructureValidator } = require('@onlineapps/conn-orch-validator/s
|
|
|
118
157
|
const validator = new ServiceStructureValidator('/path/to/service');
|
|
119
158
|
const { level, details } = validator.determineStandardLevel();
|
|
120
159
|
// level = 'v1.2', details = [{ level: 'v1.0', passed: true, checks: [...] }, ...]
|
|
160
|
+
// A failing check may also carry `fix` — a ready-to-apply replacement line.
|
|
121
161
|
|
|
122
162
|
// List all known levels
|
|
123
163
|
ServiceStructureValidator.getStandardLevels();
|
|
124
164
|
// [{ level: 'v1.0', name: 'Base Service Standard', since: '2025-06-01' }, ...]
|
|
125
165
|
```
|
|
126
166
|
|
|
127
|
-
**Related:** [Biz Service Canonical Shape — Implementation Standard Levels](/docs/biz/00-model/service-shape.md#implementation-standard-levels), [Error Handling
|
|
167
|
+
**Related:** [Biz Service Canonical Shape — Implementation Standard Levels](/docs/biz/00-model/service-shape.md#implementation-standard-levels), [Biz Error Handling Contract](/docs/biz/70-contracts/error-handling.md)
|
|
128
168
|
|
|
129
169
|
## Related Documentation
|
|
130
170
|
|
package/TESTING_STRATEGY.md
CHANGED
|
@@ -65,7 +65,7 @@
|
|
|
65
65
|
- ✅ ServiceWrapper E2E test suite created
|
|
66
66
|
- ✅ Test infrastructure setup (Jest, Express for mock services)
|
|
67
67
|
- ✅ Fixed message parsing from AMQP buffer to JSON
|
|
68
|
-
- ✅ Created test-mq-flow.js for testing message flow
|
|
68
|
+
- ✅ Created test-mq-flow.js for testing message flow (script removed 2026-08-26 — no consumer)
|
|
69
69
|
- ✅ Fixed cookbook validation (added type field to steps)
|
|
70
70
|
|
|
71
71
|
### In Progress
|
package/docs/DESIGN.md
CHANGED
|
@@ -8,9 +8,8 @@ Validation framework for OA Drive microservices. Drives two flows:
|
|
|
8
8
|
structure (config files, package.json), readiness (endpoints respond, health
|
|
9
9
|
works) and produces a signed `validation-proof.json` stored under
|
|
10
10
|
`conn-runtime/`.
|
|
11
|
-
2. **Readiness checks** — reusable
|
|
12
|
-
|
|
13
|
-
of individual biz services.
|
|
11
|
+
2. **Readiness checks** — a reusable probe (`createServiceReadinessTests`)
|
|
12
|
+
consumable from the `tests/bootstrap/` suites of individual biz services.
|
|
14
13
|
|
|
15
14
|
The single source of truth for service endpoint metadata is
|
|
16
15
|
[`operations.json`](../../../docs/biz/30-operations/registration-wire.md).
|
|
@@ -24,8 +23,10 @@ surface was removed together with the now-retired `ServiceValidator` /
|
|
|
24
23
|
integration, readiness and proof generation.
|
|
25
24
|
- **Production-ready** — the exact same code runs locally, in CI and during
|
|
26
25
|
wrapper startup.
|
|
27
|
-
- **Fail-fast** — missing logger, missing
|
|
28
|
-
|
|
26
|
+
- **Fail-fast** — missing logger, missing operations.json: immediate throw.
|
|
27
|
+
`serviceUrl` is *not* among them: it fed the retired `/health` probe and
|
|
28
|
+
nothing reads it (ADR 0005). It is accepted and ignored so existing callers
|
|
29
|
+
keep working.
|
|
29
30
|
|
|
30
31
|
## Components
|
|
31
32
|
|
|
@@ -37,9 +38,13 @@ surface was removed together with the now-retired `ServiceValidator` /
|
|
|
37
38
|
|
|
38
39
|
### Production Validation
|
|
39
40
|
|
|
40
|
-
- `ValidationOrchestrator` —
|
|
41
|
-
|
|
42
|
-
|
|
41
|
+
- `ValidationOrchestrator` — 5-step pre-validation pipeline (structure, config,
|
|
42
|
+
operations, cookbooks, connectors), emits proof
|
|
43
|
+
- `ServiceReadinessValidator` — score-based checks (operations 80 / cookbook 15
|
|
44
|
+
/ registry 5). Not used by the orchestrator: its only consumer is
|
|
45
|
+
`createServiceReadinessTests`, and through it the biz repos'
|
|
46
|
+
`tests/bootstrap/` suites. The `endpoints` and `health` checks are gone
|
|
47
|
+
(ADR 0005), and so is the `url` they took.
|
|
43
48
|
- `ServiceStructureValidator` — config layout checks (config/service/*)
|
|
44
49
|
- `ValidationProofGenerator` — fingerprint + codec helpers
|
|
45
50
|
- `CookbookTestRunner` / `WorkflowTestRunner` — cookbook execution probes
|
|
@@ -48,7 +53,6 @@ surface was removed together with the now-retired `ServiceValidator` /
|
|
|
48
53
|
### Test Suite Helpers
|
|
49
54
|
|
|
50
55
|
- `createServiceReadinessTests(options)` — Jest suite generator
|
|
51
|
-
- `createPreValidationTests(options)` — Jest suite generator
|
|
52
56
|
|
|
53
57
|
## Integration Points
|
|
54
58
|
|
|
@@ -57,8 +61,11 @@ surface was removed together with the now-retired `ServiceValidator` /
|
|
|
57
61
|
(30-day validity, invalidated by config fingerprint change — includes
|
|
58
62
|
operations map, see
|
|
59
63
|
[operations-registry-contract.md §3](../../../docs/biz/30-operations/registration-wire.md)).
|
|
60
|
-
-
|
|
61
|
-
`
|
|
64
|
+
- The biz repos import `createServiceReadinessTests` from this package in their
|
|
65
|
+
`tests/bootstrap/service-readiness.test.js` (`converter`, `ingest`,
|
|
66
|
+
`hello-service`).
|
|
67
|
+
- Pre-validation is driven by the `oa-biz-ci-gate run-prevalidation` subcommand,
|
|
68
|
+
not by a Jest suite generator.
|
|
62
69
|
|
|
63
70
|
## What We Test
|
|
64
71
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@onlineapps/conn-orch-validator",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "4.0.1",
|
|
4
4
|
"description": "Validation orchestrator for OA Drive microservices - coordinates validation across all layers (base, infra, orch, business)",
|
|
5
5
|
"main": "src/index.js",
|
|
6
6
|
"bin": {
|
|
@@ -24,15 +24,9 @@
|
|
|
24
24
|
"author": "OnlineApps",
|
|
25
25
|
"license": "PROPRIETARY",
|
|
26
26
|
"dependencies": {
|
|
27
|
-
"@onlineapps/service-validator-core": "1.0.
|
|
28
|
-
"@onlineapps/runtime-config": "1.0.2",
|
|
29
|
-
"ajv": "^8.12.0",
|
|
30
|
-
"ajv-formats": "^2.1.1",
|
|
31
|
-
"amqplib": "^0.10.9",
|
|
32
|
-
"joi": "^17.9.0"
|
|
27
|
+
"@onlineapps/service-validator-core": "1.0.15"
|
|
33
28
|
},
|
|
34
29
|
"devDependencies": {
|
|
35
|
-
"express": "^4.18.0",
|
|
36
30
|
"jest": "^29.5.0"
|
|
37
31
|
},
|
|
38
32
|
"engines": {
|