@onlineapps/mq-client-core 4.0.0 → 4.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,153 @@ All notable changes to this package. Follows [Keep a Changelog](https://keepacha
4
4
 
5
5
  ## [Unreleased]
6
6
 
7
+ ## [4.1.0] — 2026-09-23
8
+
9
+ ### Added — plánovač se dozví KONEC běhu z vlastní fronty (d.818)
10
+
11
+ `queueConfig.scheduler` nese třetí položku, `workflow.events`. Je to fronta,
12
+ kterou si plánovač sám naváže na existující výměnu `monitoring.workflow.fanout`
13
+ a čte si z ní terminální událost běhu (`completed` / `failed` / `discarded`) —
14
+ místo aby se ptal pozorovatele přes HTTP nebo četl databázi, která není jeho
15
+ (rozhodnutí vlastníka, `api/docs/governance/confirmations/scheduler.md` 007).
16
+ Žádný producent se nemění; od knihovny je jen deklarace.
17
+
18
+ TTL je **vyslovená, ne zděděná**: `x-message-ttl: 300000`, stejně jako u
19
+ sourozence `monitoring.workflow` na téže výměně. Fanout neumí filtrovat, takže
20
+ fronta dostane všechny workflow události platformy a plánovač většinu zahodí; co
21
+ stojí výpadek delší než TTL, je zapsané v 007 (události se ztratí a běh uzavře
22
+ per-schedule timeout, který zůstává pojistkou). Dál `x-max-length: 10000` a
23
+ dead-letter přes VÝCHOZÍ výměnu do `scheduler.dlq` — bez deklarované cesty by
24
+ `consume()` frontu vůbec nepřipojil (conf `mq-consumer-contract` 002).
25
+
26
+ Import: `queueConfig.queueName('scheduler', 'workflow.events')` složí jméno,
27
+ `queueConfig.declarationOptions('scheduler.workflow.events')` vrátí argumenty,
28
+ kterými ji služba asserteuje — druhá kopie těch čísel u služby by se s touhle
29
+ potkala jako 406 v den, kdy se rozejdou.
30
+
31
+ ### Added — `workflow_initiated` má jednu kolej: stavitel události vedle jejího přenosu (d.819)
32
+
33
+ `monitoring.buildWorkflowInitiatedEvent()` a `monitoring.publishWorkflowInitiated()`
34
+ — jediný stavitel události, ze které se monitoring dozví, že běh existuje (čte
35
+ z ní tenanta, štítky i spouštěč a teprve jí označí běh za `active`). Dosud ji
36
+ stavěly dvě služby, každá po svém; rozhodnutí vlastníka
37
+ (`api/docs/governance/confirmations/scheduler.md` 009) dalo kolej sem, do
38
+ knihovny, která už vlastní přenosovou půlku téhož drátu.
39
+
40
+ Vstup: `{ workflowId, redactedCookbook, serviceName, timestamp }`. Modul
41
+ **neredaguje** — co je v receptu citlivé, ví jedině spec služby v registru, a
42
+ číst ho by knihovně přidalo závislost na specech operací; volající předává
43
+ observability kopii, kterou si sám postavil. A **nic nevymýšlí**: každý klíč
44
+ `_system` musí existovat (`hasOwnProperty`, ne pravdivost), takže
45
+ `tenant_uuid: null` projde jako výslovná hodnota (conf
46
+ `run-envelope-tenant-uuid` 001) a chybějící klíč je hlášená chyba, ne tiché
47
+ `null`.
48
+
49
+ Publikace jde stávající `publishToMonitoringWorkflow`, tedy fail-safe: ztracená
50
+ kopie se hlásí, běh se kvůli ní neodmítá. Jediná výjimka je chybějící logger —
51
+ bez něj by slib „každé selhání se ohlásí" neplatil.
52
+
53
+ Import: `const { monitoring } = require('@onlineapps/mq-client-core');`
54
+
55
+ ### Added — jméno `monitoring.resource` má vlastníka v jádru (d.619)
56
+
57
+ Topic exchange, po které jezdí události `resource.changed.*`, byla poslední
58
+ stranou téhle linky bez vlastníka jména: publikující strana ji nesla jako
59
+ konstantu modulu (`api/infra/api_monitoring/src/consumer/watchers/
60
+ resourceChangedPublisher.js`, `EXCHANGE_NAME`), bindující strana jako **povinný
61
+ env klíč** (`MONITORING_RESOURCE_EXCHANGE` v delivery-endpointu), zatímco fronta
62
+ na ni navázaná (`delivery.resource.events`) má vlastníka tady od d.270/d.278.
63
+ Jméno, na kterém se musí shodnout tři strany jedné linky, má mít jednoho
64
+ vlastníka — jinak jich má tolik, kolik je míst, která ho píší, a překlep nebo
65
+ override na jedné straně ztrácí události tiše (týž důvod jako u fanout výměny
66
+ `infrastructure.health.events`, d.270).
67
+
68
+ Nově: `queueConfig.monitoring.resource` deklaruje `type: 'exchange'`,
69
+ `exchangeType: 'topic'`, `durable: true` — tvar, který publikující strana
70
+ asserteuje a který vrací i běžící broker — a `topology.monitoringResourceExchange`
71
+ to jméno publikuje, složené z té deklarace (`queueConfig.queueName()`), ne
72
+ přepsané. Import: `const { topology } = require('@onlineapps/mq-client-core');`
73
+
74
+ Doklad proti realitě, ne proti sobě: integrační tier jméno ověřuje na živém
75
+ brokeru (`checkExchange`) a deklarovaný typ `assertExchange`em, s kontrolním
76
+ případem, že tatáž výměna asserteovaná jako `fanout` končí 406 — typ je tedy
77
+ změřený, ne předpokládaný.
78
+
79
+ Beze změny na drátě: žádné jméno se nepřejmenovává, výměna se nezakládá (
80
+ deklarace není pokyn k vytvoření — zakládá ji dál vlastník prefixu). Pro INFRA
81
+ z toho plyne, že povinný klíč `MONITORING_RESOURCE_EXCHANGE` může po re-pinu
82
+ zmizet stejně jako pět jmen před ním.
83
+
84
+ ## [4.0.1] — 2026-09-17
85
+
86
+ ### Fixed — zrušení konzumenta brokerem přestalo být tiché (d.605)
87
+
88
+ `basic.cancel` od brokera — smazaná fronta, přesun fronty při HA failoveru
89
+ (`x-cancel-on-ha-failover`) — doručí amqplib do callbacku `consume()` jako
90
+ zprávu `null`. Klient na ni odpovídal holým `return`: žádný záznam, žádná změna
91
+ stavu, žádné znovupřipojení. Značka konzumenta zůstala nastavená, a protože
92
+ právě ona je jediný fakt, kterým `performHealthCheck()` odpovídá „je konzument
93
+ TOHOTO klienta připojený", hlásil klient `healthy: true` o frontě, ze které už
94
+ nic nečetl. Ticho je vada stejné závažnosti jako špatný výsledek
95
+ (`automation-gates.md` §5).
96
+
97
+ Nově v tomto pořadí: značka se zahodí (týž fakt, který d.386b zavedl pro
98
+ odmítnutou re-registraci — od té chvíle je klient `unhealthy` a jmenuje frontu),
99
+ ztráta se zapíše na úrovni **error** se jménem fronty i zrušené značky, a
100
+ konzument se znovu připojí **kolejí, kterou už používá obnova kanálu**: tělo
101
+ smyčky `_reRegisterConsumers()` dostalo jméno (`_reRegisterConsumer()`) a cancel
102
+ volá tutéž funkci pro jednu frontu. Celá smyčka by znovu připojila i konzumenty,
103
+ kteří o své místo nepřišli.
104
+
105
+ Klasifikaci, deklaraci a prefetch tedy dělá `_prepareQueueForConsume()` —
106
+ vlastněná business fronta se znovu deklaruje, nevlastněná infrastrukturní se jen
107
+ `checkQueue` a její absence je odmítnutí se značkou `CONSUMER_QUEUE_MISSING`,
108
+ nikdy tiché založení. Odmítnuté znovupřipojení nechá konzumenta sledovaného bez
109
+ značky a důvod jede na události, kterou balíček už má —
110
+ `consumer:re-registration:failed`.
111
+
112
+ **Žádný nový veřejný kontrakt**: žádná nová metoda, žádná nová událost, žádná
113
+ změna podpisu. Každý stav, do kterého se cesta může dostat, už balíček hlásí.
114
+
115
+ Ticho zůstává tam, kde zrušení není vada: vlastní `cancelConsumer()` callback
116
+ vůbec nevolá (amqplib odregistruje konzumenta na `basic.cancel-ok`; broker
117
+ neposílá cancel), a cancel, který dorazí během `disconnect()`, je zapsán na
118
+ úrovni debug — úmyslné zavření není porucha.
119
+
120
+ Kdo z toho co má: každá služba, která konzumuje frontu, se po re-pinu na tuto
121
+ verzi dozví, že o konzumenta přišla — dnes to nepozná ani z logu, ani ze
122
+ zdravotní kontroly.
123
+
124
+ ### Fixed — „připojený konzument" má jednu definici, a čte ji i levný reportér (d.613)
125
+
126
+ `getConsumerState()` vracelo jako `active` klíče `_activeConsumers`, tedy
127
+ konzumenty, které klient **sleduje**. `_performHealthCheck()` odpovídá na tutéž
128
+ otázku podle značky (`consumerInfo.consumerTag`), kterou d.386b a d.605 nechávají
129
+ nastavenou jen po dobu skutečného připojení. Jeden soubor, dvě definice slova
130
+ „aktivní" — a rozcházely se přesně tam, kde na tom záleží: konzument, jemuž
131
+ někdo smazal frontu a jehož znovupřipojení broker odmítl (406), zůstává
132
+ sledovaný bez značky, takže zdravotní kontrola hlásila 0 připojených, kdežto
133
+ `getConsumerState()` frontu dál jmenovalo jako aktivní.
134
+
135
+ To není kosmetika. `getConsumerState()` je ta **levná synchronní** odpověď, na
136
+ kterou se čtenář dostane: `docs/standards/rabbitmq-channel-best-practices.md`
137
+ § „Consumers stop receiving after a reconnect" i
138
+ `docs/operations/rabbitmq-connection-recovery-troubleshooting.md` posílají
139
+ operátora právě sem pro tento případ — a dostal odpověď, že konzument je aktivní.
140
+
141
+ Nově obě čtení volají jednu privátní funkci `_isConsumerAttached(consumerInfo)`;
142
+ `active` jmenuje **připojené** fronty a `tracked` zůstává počtem sledovaných,
143
+ takže `tracked > active.length` je celá věta „tento klient z některé fronty nic
144
+ nebere". Tvar návratové hodnoty se nemění (žádný klíč nepřibyl ani nezmizel),
145
+ mění se pravdivost hodnoty `active`; mimo tento balíček ji dnes nečte žádný kód
146
+ (změřeno přes `api`, `api_biz/*`, `fe_adminui`), takže se nic dalšího
147
+ nepřepisuje.
148
+
149
+ Kdo z toho co má: `@onlineapps/conn-infra-mq` to po re-pinu pošle dál ve
150
+ `getHealth().consumers` a `@onlineapps/service-wrapper` z toho odvodí
151
+ `components.mq` — služba se ztraceným konzumentem přestane být v heartbeatu
152
+ `healthy`.
153
+
7
154
  ## [4.0.0] — 2026-09-16
8
155
 
9
156
  ### Changed — značku odmítnutí nese i deklarační rail, jednou funkcí pro obě cesty (d.582)
package/README.md CHANGED
@@ -187,7 +187,10 @@ Owner decision: `api/docs/governance/confirmations/connector-logger-contract.md`
187
187
  three channels, the consumers this client holds, and what is waiting in the
188
188
  publish buffer (`{ size, inMemory }` — there is one buffer). All five refuse
189
189
  before `connect()` has built a transport, rather than answering with an invented
190
- zero.
190
+ zero. `getConsumerState()` answers `{ tracked, active }`: `tracked` counts the
191
+ consumers this client would restore, `active` names the ones ATTACHED right now
192
+ — the same question the health check asks — so `tracked > active.length` says
193
+ this client takes no message off some queue it registered.
191
194
  - `disconnectAll()` (static, on the class and on the package export) - Close every
192
195
  client this module holds; see § Shutdown.
193
196
  - `onError(callback)` - Register error handler
@@ -328,6 +331,39 @@ There is no third meaning. `requeueOnError: false` together with `maxAttempts`
328
331
  above 1 is a contradiction and is **refused where it was written**, rather than
329
332
  one of the two being silently ignored.
330
333
 
334
+ #### A cancel from the BROKER — the consumer that was stopped without asking
335
+
336
+ `basic.cancel` sent by the broker is how a consumer learns its attachment is gone
337
+ while the channel and the connection stay up: the queue was deleted, or the broker
338
+ moved it (`x-cancel-on-ha-failover`). amqplib delivers it to the consume callback
339
+ as a `null` message, and it reaches nothing else in this client — the channel
340
+ handlers listen for `error` and `close`, and both stay silent.
341
+
342
+ It is **not** a silent event. In this order:
343
+
344
+ - the consumer's tag is dropped, because it names an attachment the broker no
345
+ longer holds — so `performHealthCheck()` stops counting that consumer as
346
+ attached the moment the cancel arrives, rather than one recovery later;
347
+ - the loss is logged at **error** level, naming the queue and the tag the broker
348
+ cancelled;
349
+ - the consumer is re-attached on the rail a channel recreate uses, for that queue
350
+ alone: the queue is classified, declared or checked and the prefetch re-applied
351
+ exactly as `consume()` does it. So a business queue the service owns is
352
+ re-declared, and an infrastructure queue this client does not own is only
353
+ checked — a deleted one is refused with `CONSUMER_QUEUE_MISSING`, never
354
+ silently recreated.
355
+
356
+ If that re-attachment is refused, the consumer stays **tracked with no tag** — the
357
+ same state a refused re-registration leaves — so the client reports itself
358
+ unhealthy and names the queue, `consumer:re-registration:failed` carries the
359
+ broker's reason and code, and the next channel recreate tries again.
360
+
361
+ Two cancels that are NOT this: `cancelConsumer()` on this client (the broker
362
+ answers `basic.cancel-ok`, the callback is never invoked, and the consumer is
363
+ dropped from the registry on purpose), and a cancel arriving while
364
+ `disconnect()` is running — an intentional close is not a fault, and it is
365
+ recorded at debug level.
366
+
331
367
  ### Connection loss — the contract
332
368
 
333
369
  When the broker goes away, the transport runs connection-level recovery with
@@ -637,6 +673,53 @@ topology.telemetryLogsQueue = 'mine'; // TypeError — the object is froz
637
673
  - A service that still carries one of these names as a private literal is carrying the
638
674
  spelling of a fact declared here; importing it changes the wire by nothing at all.
639
675
 
676
+ ### monitoring — `workflow_initiated`, the event a run is known by
677
+
678
+ Monitoring learns that a run exists from ONE event and from nowhere else: it reads the run's
679
+ tenancy, its labels and its trigger out of `workflow_initiated` and marks the run active on it.
680
+ A service that starts runs and does not file it starts runs that are invisible for their whole
681
+ life. So the event has one builder, here, beside the transport half that sends it
682
+ ([`scheduler` 009](/docs/governance/confirmations/scheduler.md)).
683
+
684
+ ```javascript
685
+ const { monitoring } = require('@onlineapps/mq-client-core');
686
+
687
+ await monitoring.publishWorkflowInitiated({
688
+ mqClient,
689
+ logger,
690
+ workflowId,
691
+ redactedCookbook, // already redacted by the caller
692
+ serviceName: 'api_gateway', // who files the event
693
+ timestamp: startedAt, // the run's instant, not a second reading of the clock
694
+ context: { correlationId } // optional, for the log lines
695
+ });
696
+ ```
697
+
698
+ `monitoring.buildWorkflowInitiatedEvent({ workflowId, redactedCookbook, serviceName, timestamp })`
699
+ composes the body alone, so a suite can state it without a broker. It carries exactly these ten
700
+ keys: `event_type` (`initiated`), `workflow_id`, `service_name`, `step_index` (`-1`), `step_id`
701
+ (`workflow_start`), `cookbook`, `status` (`initiated`), `tenant_uuid`, `tenant_id`, `timestamp`.
702
+
703
+ What it does NOT do, and both halves are rules rather than omissions:
704
+
705
+ - **It does not redact.** What is sensitive in a cookbook is known only to the service spec in
706
+ the registry, and reading that spec would give this package a dependency on the operation
707
+ specs. The caller hands over the observability copy it built; a path that cannot redact does
708
+ not file the event at all, because an empty set of sensitive fields is silent consent to an
709
+ unredacted copy.
710
+ - **It invents no value.** `service_name` and the instant are the caller's, and every key it
711
+ reads out of `_system` must EXIST — `hasOwnProperty`, not truthiness. `tenant_uuid: null`
712
+ therefore passes through as the explicit value it is (a webhook run proves itself by
713
+ signature and has no personal identity), while a MISSING `_system.tenant_id` or
714
+ `_system.tenant_uuid` is a reported error. A `|| null` here would make the two cases
715
+ identical and hide a broken envelope.
716
+
717
+ `publishWorkflowInitiated()` is **fail-safe**: it never throws. It is observability about a run
718
+ that has already happened, so refusing the run because the observer could not be told would
719
+ trade a real effect for a record of it — a lost copy is reported through the logger and the call
720
+ answers `false`. The one exception is a missing logger, which is the transport half's rule:
721
+ without it the promise "every failure is reported" cannot hold.
722
+
640
723
  ### redactUrl — a connection URL that is safe to log
641
724
 
642
725
  `RABBITMQ_URL` carries the broker account, so anything that renders it verbatim
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@onlineapps/mq-client-core",
3
- "version": "4.0.0",
3
+ "version": "4.1.0",
4
4
  "description": "Core MQ client library for RabbitMQ - shared by infrastructure services and connectors",
5
5
  "oa": {
6
6
  "category": "connector"
package/src/BaseClient.js CHANGED
@@ -1125,9 +1125,14 @@ class BaseClient {
1125
1125
  }
1126
1126
 
1127
1127
  /**
1128
- * The consumers this client currently holds: how many, and on which queues.
1128
+ * The consumers this client currently holds: how many it TRACKS, and which of
1129
+ * them are ATTACHED right now. A consumer whose queue was deleted, or whose
1130
+ * re-registration the broker refused, stays tracked without its tag — so
1131
+ * `tracked > active.length` says this client takes no message off some queue
1132
+ * it registered (d.613; one definition of "attached",
1133
+ * `RabbitMQClient._isConsumerAttached()`).
1129
1134
  *
1130
- * @returns {Object} `{ tracked: <count>, active: [<queue names>] }`
1135
+ * @returns {Object} `{ tracked: <count>, active: [<attached queue names>] }`
1131
1136
  * @throws {ConnectionError} Before `connect()` has built a transport.
1132
1137
  */
1133
1138
  getConsumerState() {
@@ -60,7 +60,8 @@ const SECTION_PREFIXES = Object.freeze({
60
60
  monitoring: 'monitoring',
61
61
  telemetry: 'telemetry',
62
62
  alerts: 'alert',
63
- registry: 'registry'
63
+ registry: 'registry',
64
+ scheduler: 'scheduler'
64
65
  });
65
66
 
66
67
  /**
@@ -81,7 +82,8 @@ const INFRASTRUCTURE_PREFIX_LOOKUPS = Object.freeze({
81
82
  validation: 'getValidationQueueConfig',
82
83
  monitoring: 'getMonitoringQueueConfig',
83
84
  telemetry: 'getTelemetryQueueConfig',
84
- delivery: 'getDeliveryQueueConfig'
85
+ delivery: 'getDeliveryQueueConfig',
86
+ scheduler: 'getSchedulerQueueConfig'
85
87
  });
86
88
 
87
89
  const INFRASTRUCTURE_PREFIXES = Object.freeze(Object.keys(INFRASTRUCTURE_PREFIX_LOOKUPS));
@@ -689,6 +691,33 @@ module.exports = {
689
691
  durable: true
690
692
  },
691
693
 
694
+ /**
695
+ * monitoring.resource - Topic exchange for resource-changed events
696
+ *
697
+ * The monitoring consumer's resource-changed watcher publishes here with a routing
698
+ * key of the shape `resource.changed.<resource_type>`; the delivery endpoint binds
699
+ * `delivery.resource.events` to it by the pattern `resource.changed.#` for WS push.
700
+ * Three sides of ONE wire — the publisher, this exchange and the bound queue — and
701
+ * until d.619 the name had an owner on none of them: a module constant on the
702
+ * publishing side and a required env key on the binding side, so an override or a
703
+ * typo on either side loses every event silently. That is the same reasoning d.270
704
+ * settled for the health-events fanout, and the reason the name lives here and is
705
+ * published by `src/index.js` § topology.
706
+ *
707
+ * `topic`, `durable`, no arguments: what the publisher asserts, and what the running
708
+ * broker answers for it. An exchange holds nothing to dead-letter, so — like every
709
+ * other exchange in this file — it carries no route, even though every QUEUE of this
710
+ * section routes to `monitoring.dlq`.
711
+ *
712
+ * @see api/infra/api_monitoring/src/consumer/watchers/resourceChangedPublisher.js
713
+ * @see api/docs/biz/70-contracts/resource-changed.md
714
+ */
715
+ resource: {
716
+ type: 'exchange',
717
+ exchangeType: 'topic',
718
+ durable: true
719
+ },
720
+
692
721
  /**
693
722
  * monitoring.infrastructure.health.events - the monitoring consumer's own copy of
694
723
  * the infrastructure health events, bound to the infrastructure.health.events
@@ -743,8 +772,8 @@ module.exports = {
743
772
  * Telemetry queue configurations
744
773
  *
745
774
  * The OpenTelemetry intake of the monitoring consumer: every service publishes
746
- * logs, metrics and spans to `telemetry.exchange` with a routing key carrying its
747
- * own name, and these three queues are bound to it
775
+ * logs, metrics and spans to the `telemetry.exchange` declared below, with a routing
776
+ * key carrying its own name, and these three queues are bound to it
748
777
  * (`docs/standards/monitoring-queues.md`, `docs/architecture/monitoring.md`).
749
778
  *
750
779
  * The TTL and the cap are the ones the queues are created with today; this config is
@@ -758,6 +787,45 @@ module.exports = {
758
787
  * @see api/docs/governance/confirmations/mq-consumer-contract.md 003
759
788
  */
760
789
  telemetry: {
790
+ /**
791
+ * telemetry.exchange - the topic exchange every telemetry producer on the platform
792
+ * publishes to, and the one the three queues below are bound to.
793
+ *
794
+ * The widest wire there is: every biz service through `@onlineapps/service-wrapper`,
795
+ * every infrastructure service through `@onlineapps/monitoring-core` (its logger and
796
+ * its OpenTelemetry exporter), and the monitoring consumer, which asserts the
797
+ * exchange, binds the three queues and drains them — and since I16 publishes its own
798
+ * lines onto it as well.
799
+ *
800
+ * Until d.708 the name appeared HERE only in the prose above, while the other three
801
+ * platform exchanges of this file (`infrastructure.health.events`,
802
+ * `monitoring.workflow.fanout`, `monitoring.resource`) each had a template. So one
803
+ * name had three owners — a module constant in the monitoring consumer
804
+ * (`infra/api_monitoring/src/consumer/topology.js`) and a config default in each of
805
+ * monitoring-core's two publishers — which is exactly the shape d.270 gave an owner
806
+ * for the health-events fanout and d.619 for the resource-changed topic
807
+ * (`change-discipline.md` § One rail per concern).
808
+ *
809
+ * `topic`, `durable`, no arguments: what all three of those sides assert today
810
+ * (`assertExchange(<name>, 'topic', { durable: true })`), measured before this entry
811
+ * was written. The broker compares an exchange's type and durability at EVERY
812
+ * declaration, so a value invented here would be `406 PRECONDITION-FAILED` for
813
+ * whichever side booted second. An exchange holds nothing to dead-letter, so — like
814
+ * every other exchange in this file — it carries no route, even though every QUEUE
815
+ * of this section routes to `telemetry.dlq`.
816
+ *
817
+ * Declaring it is not creating it: the exchange is brought into being by the service
818
+ * that asserts it, the same rule as `telemetry.dlq` below.
819
+ *
820
+ * @see api/infra/api_monitoring/src/consumer/index.js — assertMonitoringTopology()
821
+ * @see api/shared/monitoring/monitoring-core/src/exporters/rabbitmq.js
822
+ */
823
+ exchange: {
824
+ type: 'exchange',
825
+ exchangeType: 'topic',
826
+ durable: true
827
+ },
828
+
761
829
  /** telemetry.logs.queue - OpenTelemetry log records, drained into Loki. */
762
830
  'logs.queue': {
763
831
  durable: true,
@@ -932,15 +1000,129 @@ module.exports = {
932
1000
  },
933
1001
 
934
1002
  /**
935
- * registry.changes - Registry event fanout exchange
936
- * (This is an exchange, not a queue, but included for reference)
1003
+ * registry.changes - the fanout exchange registry index changes travel on.
1004
+ *
1005
+ * One module is every side of this wire: `@onlineapps/conn-orch-registry`,
1006
+ * `src/registryEventConsumer.js`. It holds the only spelling of the name (`:92`),
1007
+ * it is the only party that ASSERTS the exchange (`:117`,
1008
+ * `assertExchange(name, 'fanout', { durable: true })`), it binds each service's
1009
+ * `{service}.registry.events` to it with the pattern `''` (`:133`), and it publishes
1010
+ * the snapshot request onto it (`:331`, `routingKey: ''`). The publish path declares
1011
+ * nothing since d.342, so there is no second founder.
1012
+ *
1013
+ * Until d.712 this was the ONE exchange of this file with no `exchangeType`, so the
1014
+ * founder had to decide the kind on its own while the template only said "an
1015
+ * exchange, durable". `type` is one of the two properties the broker compares at
1016
+ * every declaration, which makes it exactly the half that cannot live somewhere
1017
+ * else (`change-discipline.md` § One rail per concern; the same step d.619 made for
1018
+ * `monitoring.resource` and d.708 for `telemetry.exchange`).
1019
+ *
1020
+ * `fanout`, `durable`, no arguments: measured on that founder, not chosen. The
1021
+ * binding pattern `''` only routes on a fanout, so the type is load-bearing rather
1022
+ * than decorative. An exchange holds nothing to dead-letter, so — like every other
1023
+ * exchange in this file — it carries no route.
1024
+ *
1025
+ * Declaring it is not creating it: the exchange is brought into being by the service
1026
+ * whose registry client subscribes to changes.
1027
+ *
1028
+ * @see api/shared/connector/conn-orch-registry/src/registryEventConsumer.js
937
1029
  */
938
1030
  changes: {
939
1031
  type: 'exchange',
1032
+ exchangeType: 'fanout',
940
1033
  durable: true
941
1034
  }
942
1035
  },
943
1036
 
1037
+ /**
1038
+ * Scheduler queue configurations — the platform's one holder of time.
1039
+ *
1040
+ * The scheduler consumes requests for a run: a recurrence that came due, or a deferred
1041
+ * request somebody filed. Both are published by other services and both are the ONLY
1042
+ * record that the run was owed, so the family is shaped for loss-aversion rather than
1043
+ * for freshness (`docs/governance/confirmations/scheduler.md` 001, 004).
1044
+ *
1045
+ * @see api/infra/api_scheduler/README.md
1046
+ */
1047
+ scheduler: {
1048
+ /**
1049
+ * scheduler.request - every request for a run, from every producer.
1050
+ *
1051
+ * NO `x-message-ttl`, deliberately, and it is the one property that differs from the
1052
+ * shape of a workflow queue: a request that waited out a consumer outage is still the
1053
+ * run that was due, and a TTL would turn that outage into runs which silently never
1054
+ * happened. What bounds the queue is its length, and what catches a request the
1055
+ * scheduler could not process is the family's dead-letter queue below — over the
1056
+ * DEFAULT exchange, where the routing key IS the destination name, the same way every
1057
+ * other infrastructure family here routes (the `dlx` exchange belongs to the business
1058
+ * templates, whose queues one service owns).
1059
+ *
1060
+ * Without this declaration `consume()` refuses to attach a consumer at all — a queue
1061
+ * with no declared route would have the broker DROP what the policy rejects
1062
+ * (`docs/governance/confirmations/mq-consumer-contract.md` 002).
1063
+ */
1064
+ request: {
1065
+ durable: true,
1066
+ arguments: {
1067
+ 'x-max-length': 10000,
1068
+ 'x-dead-letter-exchange': '', // Default exchange: routing key IS the queue name
1069
+ 'x-dead-letter-routing-key': 'scheduler.dlq'
1070
+ }
1071
+ },
1072
+
1073
+ /**
1074
+ * scheduler.workflow.events - where the scheduler learns that a run it started ENDED.
1075
+ *
1076
+ * The third queue bound to `monitoring.workflow.fanout`, beside `monitoring.workflow`
1077
+ * and `delivery.workflow.events`. The owner settled it in
1078
+ * `docs/governance/confirmations/scheduler.md` 007: the terminal event of a run
1079
+ * (`completed` / `failed` / `discarded`) already travels on that exchange, so the
1080
+ * scheduler reads it there rather than asking the observer over HTTP or reading a
1081
+ * database that is not its own. No producer changes; the binding is the scheduler's own
1082
+ * act at boot (the pattern `infra/api_delivery_endpoint/src/index.js:164` already uses),
1083
+ * and this file only DECLARES what that queue is.
1084
+ *
1085
+ * THE TTL IS STATED, NOT INHERITED, and it is a property of that decision rather than a
1086
+ * default: a fanout cannot filter, so this queue receives EVERY workflow event of the
1087
+ * platform and the scheduler discards most of them. 300 000 ms is what
1088
+ * `monitoring.workflow` carries on the same exchange - the sibling whose readings this
1089
+ * one follows. What an outage longer than the TTL costs is on record: those events are
1090
+ * lost and the run is closed by the per-schedule timeout, which 007 keeps as the
1091
+ * backstop rather than replacing it.
1092
+ *
1093
+ * The route goes over the DEFAULT exchange, like every other infrastructure family
1094
+ * here, and ends in this family's own `scheduler.dlq`. Without a declared route
1095
+ * `consume()` refuses to attach at all (`BaseClient.js`,
1096
+ * `CONSUMER_DEAD_LETTER_ROUTE_MISSING`; `mq-consumer-contract` 002).
1097
+ *
1098
+ * @see api/docs/governance/confirmations/scheduler.md 007
1099
+ */
1100
+ 'workflow.events': {
1101
+ durable: true,
1102
+ arguments: {
1103
+ 'x-message-ttl': 300000, // Stated by confirmation 007, like monitoring.workflow
1104
+ 'x-max-length': 10000,
1105
+ 'x-dead-letter-exchange': '', // Default exchange: routing key IS the queue name
1106
+ 'x-dead-letter-routing-key': 'scheduler.dlq'
1107
+ }
1108
+ },
1109
+
1110
+ /**
1111
+ * scheduler.dlq - the END of this family's chain.
1112
+ *
1113
+ * Durable, capped higher, no TTL and no onward route — the three properties every
1114
+ * dead-letter queue in this file carries, and the reason the operator's `*.dlq`
1115
+ * dashboard finds it (`mq-consumer-contract` 003).
1116
+ */
1117
+ dlq: {
1118
+ durable: true,
1119
+ arguments: {
1120
+ // No TTL for DLQ - messages should persist
1121
+ 'x-max-length': 50000 // Higher limit for DLQ
1122
+ }
1123
+ }
1124
+ },
1125
+
944
1126
  /**
945
1127
  * The full wire name of a declared entry: the section's prefix + the entry's key.
946
1128
  *
@@ -1219,6 +1401,23 @@ module.exports = {
1219
1401
  return this.getQueueConfig('infrastructure', name);
1220
1402
  },
1221
1403
 
1404
+ /**
1405
+ * Get scheduler queue configuration
1406
+ * @param {string} queueName - Queue name (e.g., 'scheduler.request')
1407
+ * @returns {Object} Queue configuration
1408
+ * @throws {ValidationError} If the name carries another prefix, or nothing declares it.
1409
+ */
1410
+ getSchedulerQueueConfig(queueName) {
1411
+ if (!queueName.startsWith('scheduler.')) {
1412
+ throw new ValidationError(
1413
+ `[queueConfig] Queue ${queueName} is not a scheduler queue - Expected: a name starting with "scheduler.". `
1414
+ + 'Fix: call getInfrastructureQueueConfig(), which routes each prefix to the right lookup.'
1415
+ );
1416
+ }
1417
+ const name = queueName.slice('scheduler.'.length);
1418
+ return this.getQueueConfig('scheduler', name);
1419
+ },
1420
+
1222
1421
  /**
1223
1422
  * Get monitoring queue configuration
1224
1423
  * @param {string} queueName - Queue name (e.g., 'monitoring.workflow')
package/src/index.js CHANGED
@@ -36,6 +36,10 @@ const {
36
36
  publishToMonitoringServices,
37
37
  isQueueUnavailableError,
38
38
  } = require('./monitoring-publish');
39
+ const {
40
+ buildWorkflowInitiatedEvent,
41
+ publishWorkflowInitiated,
42
+ } = require('./monitoring-initiated');
39
43
  const queueConfig = require('./config/queueConfig');
40
44
  const deliveryPolicy = require('./config/deliveryPolicy');
41
45
  const { redactUrl, UNPARSEABLE_PLACEHOLDER } = require('@onlineapps/logger-contract');
@@ -84,6 +88,11 @@ module.exports.monitoring = {
84
88
  publishToMonitoringWorkflow,
85
89
  publishToMonitoringServices,
86
90
  isQueueUnavailableError,
91
+ // d.819 - the one builder of `workflow_initiated`, beside the transport half it uses
92
+ // (`docs/governance/confirmations/scheduler.md` 009). The gateway and the scheduler each
93
+ // built that event in their own words; two rails over one event is a defect by default.
94
+ buildWorkflowInitiatedEvent,
95
+ publishWorkflowInitiated,
87
96
  };
88
97
  /** The only public way to reach the central queue classification; `src/config/queueConfig` is an internal path, not a contract. */
89
98
  module.exports.queueConfig = queueConfig;
@@ -109,6 +118,18 @@ module.exports.topology = Object.freeze({
109
118
  monitoringInfrastructureHealthEventsQueue: queueConfig.queueName('monitoring', 'infrastructure.health.events'),
110
119
  /** The delivery endpoint's queue bound to that exchange, for WS push. */
111
120
  deliveryHealthEventsQueue: queueConfig.queueName('deliveryEvents', 'health.events'),
121
+ /**
122
+ * Topic exchange: the monitoring consumer's resource-changed watcher publishes here,
123
+ * with a routing key of the shape `resource.changed.<resource_type>`.
124
+ *
125
+ * Declared in `queueConfig` since d.619. Until then the name was a module constant on
126
+ * the publishing side and a required env key on the binding side
127
+ * (`MONITORING_RESOURCE_EXCHANGE`) — the third side of this wire with no owner, while
128
+ * the queue bound to it had one here since d.270.
129
+ *
130
+ * @see api/infra/api_monitoring/src/consumer/watchers/resourceChangedPublisher.js
131
+ */
132
+ monitoringResourceExchange: queueConfig.queueName('monitoring', 'resource'),
112
133
  /**
113
134
  * The delivery endpoint's queue for resource-changed events, bound to the
114
135
  * `monitoring.resource` exchange by the routing pattern `resource.changed.#`.
@@ -0,0 +1,187 @@
1
+ 'use strict';
2
+
3
+ /**
4
+ * The `workflow_initiated` event — ONE rail for every service that starts a run.
5
+ *
6
+ * Monitoring learns that a run exists from this event and from nothing else: its intake
7
+ * reads the run's tenancy, labels and trigger out of `workflow_initiated` and marks the
8
+ * run `active` on it (`api/infra/api_monitoring/src/consumer/index.js`). A service that
9
+ * does not file it starts runs that are invisible for their whole life.
10
+ *
11
+ * Until now two services built that event, each in its own words: the gateway
12
+ * (`infra/api_gateway/shared/monitoringInitiatedCopy.js`, with a redacted cookbook) and
13
+ * the scheduler (`infra/api_scheduler/src/publisher/workflowPublisher.js`, inline, the
14
+ * cookbook verbatim and no `tenant_uuid` at all). Two implementations of one concern are
15
+ * a defect by default (`change-discipline.md` § One rail per concern), and the owner
16
+ * settled which rail survives: this one, in the library that already owns the transport
17
+ * half of the same wire (`docs/governance/confirmations/scheduler.md` 009). Both copies
18
+ * are retired by their own services in the change that adopts this module — no period
19
+ * with both (`architecture-principles.md` §11).
20
+ *
21
+ * WHAT THIS MODULE DOES NOT DO, deliberately:
22
+ *
23
+ * 1. **It does not redact.** What is sensitive in a cookbook is known only to the
24
+ * service spec in the registry, and reading that spec would give this library a
25
+ * dependency on the operation specs and on `infrastructure-tools`. The caller passes
26
+ * the observability copy it already built (`cookbookRedaction.js` in the gateway); a
27
+ * path that cannot redact does not file the event at all, because an empty set of
28
+ * sensitive fields is silent consent to an unredacted copy.
29
+ * 2. **It invents no value.** `service_name` and the instant are the caller's, and every
30
+ * key it reads out of `_system` must EXIST — `hasOwnProperty`, not truthiness, so
31
+ * `tenant_uuid: null` passes through as the explicit value it is (a webhook run has
32
+ * no personal identity, and a scheduled run of the pilot writes `null` on purpose —
33
+ * `docs/governance/confirmations/run-envelope-tenant-uuid.md` 001). A `|| null` here
34
+ * would hide a broken envelope instead of reporting it (`architecture-principles.md`
35
+ * §3).
36
+ *
37
+ * @see api/docs/governance/confirmations/scheduler.md 009
38
+ * @see api/docs/biz/70-contracts/monitoring-events.md
39
+ * @see ./monitoring-publish.js — the transport half: the queue, the fanout, the fail-safe
40
+ */
41
+
42
+ const { assertLogger } = require('@onlineapps/logger-contract');
43
+ const { publishToMonitoringWorkflow } = require('./monitoring-publish');
44
+
45
+ /** Starting a run is not a step of the cookbook; it has its own constant coordinates. */
46
+ const STEP_ID = 'workflow_start';
47
+ const STEP_INDEX = -1;
48
+ /** The event's own two words, which the monitoring intake matches on. */
49
+ const EVENT_TYPE = 'initiated';
50
+ const STATUS = 'initiated';
51
+
52
+ /** The keys of `_system` this event carries, each of which must exist on the envelope. */
53
+ const REQUIRED_SYSTEM_KEYS = Object.freeze(['tenant_id', 'tenant_uuid']);
54
+
55
+ /**
56
+ * The body of the `initiated` event — exactly these keys and no others.
57
+ *
58
+ * @param {Object} args
59
+ * @param {string} args.workflowId - the id of the run this event reports.
60
+ * @param {Object} args.redactedCookbook - the observability copy of the unified cookbook,
61
+ * ALREADY redacted by the caller; it travels as it arrived.
62
+ * @param {string} args.serviceName - the platform name of the service filing the event;
63
+ * the monitoring intake refuses an event that names no publisher (`service_name_missing`).
64
+ * @param {string} args.timestamp - the instant the run started, ISO-8601. The caller's,
65
+ * not a second reading of the clock here: the run and the row observing it then say the
66
+ * same thing.
67
+ * @returns {Object} the message for `monitoring.workflow`.
68
+ * @throws {Error} when an argument is missing, or the envelope does not carry what this
69
+ * event states about it.
70
+ */
71
+ function buildWorkflowInitiatedEvent({ workflowId, redactedCookbook, serviceName, timestamp } = {}) {
72
+ if (typeof workflowId !== 'string' || workflowId.trim() === '') {
73
+ throw new Error(
74
+ '[monitoring-initiated] Missing workflow_id - Expected: the id of the run this event '
75
+ + 'reports, as a non-empty string. Fix: pass workflowId of the run being published.'
76
+ );
77
+ }
78
+ if (typeof serviceName !== 'string' || serviceName.trim() === '') {
79
+ throw new Error(
80
+ '[monitoring-initiated] Missing service_name - Expected: the platform name of the service '
81
+ + 'filing this event; the monitoring intake refuses an event that names no publisher '
82
+ + '(service_name_missing). Fix: pass config.serviceName.'
83
+ );
84
+ }
85
+ if (typeof timestamp !== 'string' || timestamp.trim() === '') {
86
+ throw new Error(
87
+ '[monitoring-initiated] Missing timestamp - Expected: the instant the run started, as an '
88
+ + 'ISO-8601 string taken from the run itself, so the run and the row observing it say the '
89
+ + 'same thing. Fix: pass the envelope\'s own instant, not a new Date() read here.'
90
+ );
91
+ }
92
+ if (!redactedCookbook || typeof redactedCookbook !== 'object') {
93
+ throw new Error(
94
+ '[monitoring-initiated] Missing cookbook - Expected: the redacted observability copy of the '
95
+ + 'unified cookbook. Fix: pass the copy the caller built with its own redaction '
96
+ + '(a path that cannot redact does not file this event at all).'
97
+ );
98
+ }
99
+
100
+ const system = redactedCookbook._system;
101
+ if (!system || typeof system !== 'object') {
102
+ throw new Error(
103
+ '[monitoring-initiated] Cookbook carries no _system - Expected: the unified cookbook with '
104
+ + 'its system metadata, which is where the intake reads the run\'s tenancy, labels and '
105
+ + 'trigger. Fix: build the copy from the unified cookbook the run was published with.'
106
+ );
107
+ }
108
+
109
+ for (const key of REQUIRED_SYSTEM_KEYS) {
110
+ if (!Object.prototype.hasOwnProperty.call(system, key)) {
111
+ throw new Error(
112
+ `[monitoring-initiated] Cookbook _system carries no ${key} - Expected: every entry path `
113
+ + 'writes it explicitly (a value, or null where the path has none - a webhook run proves '
114
+ + 'itself by signature, a scheduled run of the pilot has no personal identity). '
115
+ + `Fix: set _system.${key} where the unified cookbook is built.`
116
+ );
117
+ }
118
+ }
119
+
120
+ return {
121
+ event_type: EVENT_TYPE,
122
+ workflow_id: workflowId,
123
+ service_name: serviceName,
124
+ step_index: STEP_INDEX,
125
+ step_id: STEP_ID,
126
+ cookbook: redactedCookbook,
127
+ status: STATUS,
128
+ tenant_uuid: system.tenant_uuid,
129
+ tenant_id: system.tenant_id,
130
+ timestamp
131
+ };
132
+ }
133
+
134
+ /**
135
+ * File the `initiated` event. NEVER throws — a lost copy is reported, not raised.
136
+ *
137
+ * It is observability about a run that has already happened: refusing the run because the
138
+ * observer could not be told would trade a real effect for a record of it. Both services
139
+ * that build this event today made the same decision in their own words, and it moves here
140
+ * with the event. A missing logger is the one exception, and it is the transport half's
141
+ * rule (`monitoring-publish.js`): without a logger the promise "every failure is reported"
142
+ * cannot hold, so the call refuses instead of dropping events in silence.
143
+ *
144
+ * @param {Object} args
145
+ * @param {Object} args.mqClient - the connected `BaseClient`.
146
+ * @param {Object} args.logger - the service logger; REQUIRED.
147
+ * @param {string} args.workflowId
148
+ * @param {Object} args.redactedCookbook
149
+ * @param {string} args.serviceName
150
+ * @param {string} args.timestamp
151
+ * @param {Object} [args.context] - extra context for the log lines.
152
+ * @returns {Promise<boolean>} whether the event reached the broker.
153
+ */
154
+ async function publishWorkflowInitiated({
155
+ mqClient,
156
+ logger,
157
+ workflowId,
158
+ redactedCookbook,
159
+ serviceName,
160
+ timestamp,
161
+ context = {}
162
+ } = {}) {
163
+ assertLogger(
164
+ 'publishWorkflowInitiated',
165
+ logger,
166
+ 'a copy monitoring never received is reported instead of silently returning false',
167
+ 'pass the logger your service already built'
168
+ );
169
+
170
+ try {
171
+ const event = buildWorkflowInitiatedEvent({ workflowId, redactedCookbook, serviceName, timestamp });
172
+ return await publishToMonitoringWorkflow(mqClient, event, logger, { workflowId, ...context });
173
+ } catch (error) {
174
+ logger.warn('[monitoring-initiated] Failed to publish workflow_initiated to monitoring', {
175
+ ...context,
176
+ workflow_id: workflowId,
177
+ service_name: serviceName,
178
+ error: error.message
179
+ });
180
+ return false;
181
+ }
182
+ }
183
+
184
+ module.exports = {
185
+ buildWorkflowInitiatedEvent,
186
+ publishWorkflowInitiated
187
+ };
@@ -525,13 +525,52 @@ class RabbitMQClient extends EventEmitter {
525
525
  }
526
526
 
527
527
  /**
528
- * Get current consumer state
529
- * @returns {Object} Consumer state information
528
+ * THE definition of "this client's consumer is attached to that queue".
529
+ *
530
+ * The tag is the one fact that is about THIS client: the broker gives it to
531
+ * the attachment it created, and since d.386b / d.605 the entry carries it
532
+ * only while that attachment exists — a refused re-registration and a cancel
533
+ * sent by the broker both drop it while the consumer stays TRACKED, because
534
+ * the next recovery must find it on the list.
535
+ *
536
+ * It is a method rather than an expression repeated twice for the reason
537
+ * `change-discipline.md` § One rail per concern states: until d.613
538
+ * `getConsumerState()` answered from `_activeConsumers.keys()` and
539
+ * `_performHealthCheck()` from the tag, so one file held two definitions of
540
+ * "active" — and the cheap synchronous reader, the one a service and an
541
+ * operator actually read, was the one that answered about a consumer the
542
+ * broker had already cancelled.
543
+ *
544
+ * @param {Object|undefined} consumerInfo - The entry `_activeConsumers` holds
545
+ * @returns {boolean}
546
+ * @private
547
+ */
548
+ _isConsumerAttached(consumerInfo) {
549
+ return typeof consumerInfo?.consumerTag === 'string' && consumerInfo.consumerTag !== '';
550
+ }
551
+
552
+ /**
553
+ * Get current consumer state.
554
+ *
555
+ * `tracked` counts the consumers this client would restore; `active` names the
556
+ * ones that are ATTACHED right now (`_isConsumerAttached()`, the same question
557
+ * `_performHealthCheck()` asks). `tracked > active.length` is therefore the
558
+ * cheap, synchronous statement that this client takes no message off some
559
+ * queue it registered — the state
560
+ * `docs/operations/rabbitmq-connection-recovery-troubleshooting.md` sends an
561
+ * operator here to read, and which the queue-name list could not carry.
562
+ *
563
+ * @returns {{tracked: number, active: string[]}} Consumer state information
530
564
  */
531
565
  getConsumerState() {
566
+ const active = [];
567
+ for (const [queue, consumerInfo] of this._activeConsumers.entries()) {
568
+ if (this._isConsumerAttached(consumerInfo)) active.push(queue);
569
+ }
570
+
532
571
  return {
533
572
  tracked: this._activeConsumers.size,
534
- active: Array.from(this._activeConsumers.keys())
573
+ active
535
574
  };
536
575
  }
537
576
 
@@ -1858,54 +1897,160 @@ class RabbitMQClient extends EventEmitter {
1858
1897
  if (this._activeConsumers.size > 0) {
1859
1898
  this._logger.info(`[RabbitMQClient] [mq-client-core] Re-registering ${this._activeConsumers.size} consumers...`);
1860
1899
  for (const [queue, consumerInfo] of this._activeConsumers.entries()) {
1861
- try {
1862
- await this._prepareQueueForConsume(queue, {
1863
- durable: consumerInfo.options.durable,
1864
- queueOptions: consumerInfo.options.queueOptions
1865
- });
1866
-
1867
- // The consumer's prefetch is re-applied here for the same reason the
1868
- // queue step is: the channel is new, and QoS lives on the channel.
1869
- await this._applyConsumerPrefetch(queue, consumerInfo.options.prefetch);
1870
-
1871
- const consumeResult = await this._consumerChannel.consume(
1872
- queue,
1873
- consumerInfo.handler,
1874
- { noAck: consumerInfo.options.noAck }
1875
- );
1876
- consumerInfo.consumerTag = consumeResult.consumerTag;
1900
+ if (await this._reRegisterConsumer(queue, consumerInfo)) {
1877
1901
  reRegistered++;
1878
- this._logger.info(`[RabbitMQClient] [mq-client-core] \u2713 Re-registered consumer for queue: ${queue} (consumerTag: ${consumeResult.consumerTag})`);
1879
- } catch (err) {
1902
+ } else {
1880
1903
  failed++;
1881
- this._logger.error(`[RabbitMQClient] [mq-client-core] Failed to re-register consumer for queue ${queue}:`, err.message);
1882
- // The consumer STAYS tracked. Whatever refused it — a 406 against the
1883
- // owner's declaration, a missing infrastructure queue, a channel that
1884
- // went away mid-flight — is a condition somebody fixes at its source,
1885
- // and the next recreate must find the consumer still on the list.
1886
- //
1887
- // Its TAG does not stay, and the two are not the same fact (d.386b). A
1888
- // consumer tag names an attachment on ONE channel; the channel that
1889
- // issued this one is gone and the new one refused to re-attach, so the
1890
- // string the entry carried names nothing on the broker. Two readers
1891
- // took it for an attachment that exists: `_performHealthCheck()` counts
1892
- // `active` by "is the tag set", so a client whose only consumer had
1893
- // just been refused reported `healthy: true` and `1/1 consumers active`
1894
- // — the false OK d.386 exists to end, one layer below where d.386 could
1895
- // reach it; and `cancelConsumer()` sent `basic.cancel` with it, asking
1896
- // the broker to stop a consumer it does not hold.
1897
- consumerInfo.consumerTag = null;
1898
- // The broker's own code is what a listener acts on (406 = the queue
1899
- // exists with different arguments), and the rail wraps it in a typed
1900
- // error, so it is read from the cause when the wrapper carries none.
1901
- const code = err.code !== undefined ? err.code : (err.cause ? err.cause.code : undefined);
1902
- this.emit('consumer:re-registration:failed', { queue, error: err.message, code });
1903
1904
  }
1904
1905
  }
1905
1906
  }
1906
1907
  return { reRegistered, failed };
1907
1908
  }
1908
1909
 
1910
+ /**
1911
+ * Re-attach ONE tracked consumer on the current consumer channel.
1912
+ *
1913
+ * This is the body of the loop above, given a name in d.605 because a second
1914
+ * event needs exactly it: `basic.cancel` from the broker takes ONE consumer
1915
+ * away while every other one stays attached, so re-running the whole loop
1916
+ * would re-attach consumers that never lost their place — a second
1917
+ * `basic.consume` for each of them on the broker. One rail, one set of
1918
+ * consequences (`change-discipline.md` § One rail per concern).
1919
+ *
1920
+ * Everything a first `consume()` does is done here: the queue is classified,
1921
+ * declared or checked by `_prepareQueueForConsume()` — so an owned business
1922
+ * queue is re-declared and an unowned infrastructure queue is only checked,
1923
+ * its absence answered with `CONSUMER_QUEUE_MISSING` rather than a silent
1924
+ * creation — and the prefetch is re-applied, because QoS lives on the channel.
1925
+ *
1926
+ * @param {string} queue - Queue name
1927
+ * @param {Object} consumerInfo - The entry `_activeConsumers` holds for it
1928
+ * @returns {Promise<boolean>} `true` when the consumer is attached again
1929
+ * @private
1930
+ */
1931
+ async _reRegisterConsumer(queue, consumerInfo) {
1932
+ try {
1933
+ await this._prepareQueueForConsume(queue, {
1934
+ durable: consumerInfo.options.durable,
1935
+ queueOptions: consumerInfo.options.queueOptions
1936
+ });
1937
+
1938
+ // The consumer's prefetch is re-applied here for the same reason the
1939
+ // queue step is: the channel is new, and QoS lives on the channel.
1940
+ await this._applyConsumerPrefetch(queue, consumerInfo.options.prefetch);
1941
+
1942
+ const consumeResult = await this._consumerChannel.consume(
1943
+ queue,
1944
+ consumerInfo.handler,
1945
+ { noAck: consumerInfo.options.noAck }
1946
+ );
1947
+ consumerInfo.consumerTag = consumeResult.consumerTag;
1948
+ this._logger.info(`[RabbitMQClient] [mq-client-core] ✓ Re-registered consumer for queue: ${queue} (consumerTag: ${consumeResult.consumerTag})`);
1949
+ return true;
1950
+ } catch (err) {
1951
+ this._logger.error(`[RabbitMQClient] [mq-client-core] Failed to re-register consumer for queue ${queue}:`, err.message);
1952
+ // The consumer STAYS tracked. Whatever refused it — a 406 against the
1953
+ // owner's declaration, a missing infrastructure queue, a channel that
1954
+ // went away mid-flight — is a condition somebody fixes at its source,
1955
+ // and the next recreate must find the consumer still on the list.
1956
+ //
1957
+ // Its TAG does not stay, and the two are not the same fact (d.386b). A
1958
+ // consumer tag names an attachment on ONE channel; the channel that
1959
+ // issued this one is gone and the new one refused to re-attach, so the
1960
+ // string the entry carried names nothing on the broker. Two readers
1961
+ // took it for an attachment that exists: `_performHealthCheck()` counts
1962
+ // `active` by "is the tag set", so a client whose only consumer had
1963
+ // just been refused reported `healthy: true` and `1/1 consumers active`
1964
+ // — the false OK d.386 exists to end, one layer below where d.386 could
1965
+ // reach it; and `cancelConsumer()` sent `basic.cancel` with it, asking
1966
+ // the broker to stop a consumer it does not hold.
1967
+ consumerInfo.consumerTag = null;
1968
+ // The broker's own code is what a listener acts on (406 = the queue
1969
+ // exists with different arguments), and the rail wraps it in a typed
1970
+ // error, so it is read from the cause when the wrapper carries none.
1971
+ const code = err.code !== undefined ? err.code : (err.cause ? err.cause.code : undefined);
1972
+ this.emit('consumer:re-registration:failed', { queue, error: err.message, code });
1973
+ return false;
1974
+ }
1975
+ }
1976
+
1977
+ /**
1978
+ * The broker cancelled one of this client's consumers.
1979
+ *
1980
+ * `basic.cancel` sent BY THE BROKER is not an error frame and does not touch
1981
+ * the channel or the connection: the queue was deleted, or the broker moved it
1982
+ * (`x-cancel-on-ha-failover`). Nothing else in this client hears it — the
1983
+ * channel handlers listen for `error` and `close`, and both stay silent — so
1984
+ * this method is the whole of the client's answer, and until d.605 that answer
1985
+ * was nothing at all.
1986
+ *
1987
+ * Three things happen, in this order:
1988
+ *
1989
+ * 1. the TAG is dropped, first, because it is the one fact
1990
+ * `_performHealthCheck()` reads to answer "is THIS client's consumer
1991
+ * attached" (d.386b). Dropping it before anything else leaves no window in
1992
+ * which the check reports an attachment the broker has already cancelled;
1993
+ * 2. the loss is stated at ERROR level, naming the queue and the cancelled
1994
+ * tag. A consumer that stops reading a queue is not a debug-level event,
1995
+ * and the record is what tells an operator which queue went quiet;
1996
+ * 3. the consumer is re-attached through `_reRegisterConsumer()`, the rail a
1997
+ * channel recreate already uses. A refusal there keeps the consumer
1998
+ * tracked with no tag and travels on `consumer:re-registration:failed` —
1999
+ * which is why this method introduces no event and no public method of its
2000
+ * own: every state it can reach is a state the package already reports.
2001
+ *
2002
+ * @param {string} queue - The queue whose consumer the broker cancelled
2003
+ * @returns {Promise<void>}
2004
+ * @private
2005
+ */
2006
+ async _onBrokerCancel(queue) {
2007
+ if (this._closedByCaller) {
2008
+ // An intentional close is not a fault — the doctrine `disconnect()` states
2009
+ // and both channel handlers already follow.
2010
+ this._logger.debug(
2011
+ `[RabbitMQClient] [mq-client-core] [CONSUMER] Broker cancelled the consumer for queue "${queue}" while the client was being closed by its caller - no alarm, no re-attachment`
2012
+ );
2013
+ return;
2014
+ }
2015
+
2016
+ const consumerInfo = this._activeConsumers.get(queue);
2017
+ const cancelledTag = consumerInfo ? consumerInfo.consumerTag : null;
2018
+
2019
+ if (consumerInfo) {
2020
+ consumerInfo.consumerTag = null;
2021
+ }
2022
+
2023
+ this._logger.error(
2024
+ `[RabbitMQClient] Consumer cancelled by the broker - the consumer on queue "${queue}" `
2025
+ + `(consumerTag: ${cancelledTag}) was stopped by the BROKER, not by this client: the queue `
2026
+ + 'was deleted, or the broker moved it (x-cancel-on-ha-failover). '
2027
+ + 'Expected: a consumer stays attached until cancelConsumer() or disconnect(). '
2028
+ + 'Fix: none here - the client re-attaches it on the rail a channel recreate uses; if that '
2029
+ + 'is refused, the consumer stays tracked with no tag, performHealthCheck() reports this '
2030
+ + 'client unhealthy and names the queue, and consumer:re-registration:failed carries the '
2031
+ + 'broker reason.',
2032
+ { queue, consumerTag: cancelledTag }
2033
+ );
2034
+
2035
+ if (!consumerInfo) {
2036
+ // The client no longer holds this consumer, so there is nothing to
2037
+ // re-attach and no state left to correct. The loss is still stated above.
2038
+ return;
2039
+ }
2040
+
2041
+ if (!this._isChannelAlive(this._consumerChannel)) {
2042
+ // Re-attaching on a dead channel is not this method's job: the channel
2043
+ // recovery re-registers every tracked consumer when it recreates the
2044
+ // channel (d.262), and the consumer stays tracked for exactly that.
2045
+ this._logger.warn(
2046
+ `[RabbitMQClient] [mq-client-core] [CONSUMER] Consumer for queue "${queue}" stays tracked and unattached - the consumer channel is not alive, so the channel recovery re-registers it`
2047
+ );
2048
+ return;
2049
+ }
2050
+
2051
+ await this._reRegisterConsumer(queue, consumerInfo);
2052
+ }
2053
+
1909
2054
  /**
1910
2055
  * Disconnects: closes channel and connection.
1911
2056
  * @returns {Promise<void>}
@@ -2061,8 +2206,10 @@ class RabbitMQClient extends EventEmitter {
2061
2206
  // this client's and everybody else's alike, so it cannot say whether
2062
2207
  // the one asking is among them. The tag can: it is the name the
2063
2208
  // broker gave THIS client's attachment, and since d.386b it is set
2064
- // only while that attachment exists.
2065
- if (consumerInfo.consumerTag) {
2209
+ // only while that attachment exists. The question is asked through
2210
+ // `_isConsumerAttached()` — the one definition `getConsumerState()`
2211
+ // now reads too (d.613).
2212
+ if (this._isConsumerAttached(consumerInfo)) {
2066
2213
  health.consumers.active++;
2067
2214
  } else {
2068
2215
  health.consumers.inactive.push(queue);
@@ -3290,9 +3437,11 @@ class RabbitMQClient extends EventEmitter {
3290
3437
  // leaves exactly those without a route of their own, because a dead-letter
3291
3438
  // route on a dead-letter queue is a loop or a chain with no end
3292
3439
  // (`mq-consumer-contract` 002 point 2, 003 point 2). Until d.452 the gate
3293
- // asked only the first question, so the operator's own queue was
3294
- // unconsumable and its reader — the delivery dispatcher — could not attach
3295
- // at all. The second question is derived from the templates
3440
+ // asked only the first question, so the operator's own queue could not be
3441
+ // consumed through this library at all. Who reads such a queue is that
3442
+ // service's own fact, not this file's —
3443
+ // @see api/infra/api_monitoring/src/consumer/deadLetterPeek.js
3444
+ // The second question is derived from the templates
3296
3445
  // (`queueConfig.isDeadLetterTarget()`), never from a list of names, so it
3297
3446
  // cannot fall behind them; every other queue with no route is refused
3298
3447
  // exactly as before, with the same sentence.
@@ -3532,7 +3681,21 @@ class RabbitMQClient extends EventEmitter {
3532
3681
  // Wrap handler to track message processing and prefetch utilization
3533
3682
  const messageHandler = async (msg) => {
3534
3683
  if (msg === null) {
3535
- return; // Consumer cancellation
3684
+ // `null` is not a message. It is `basic.cancel` FROM THE BROKER,
3685
+ // handed to this callback by amqplib (`handleCancel()` →
3686
+ // `dispatchMessage(fields, null)`), and it is the only way a consumer
3687
+ // learns that its attachment is gone while the channel and the
3688
+ // connection stay up: the queue was deleted, or the broker moved it
3689
+ // (`x-cancel-on-ha-failover`).
3690
+ //
3691
+ // Until d.605 the answer was this bare `return`. Nothing was logged,
3692
+ // the tag stayed set — so `_performHealthCheck()` went on counting the
3693
+ // consumer as attached — and nothing re-attached it. The service
3694
+ // reported itself healthy while no message was taken off the queue,
3695
+ // which is the failure `automation-gates.md` §5 calls a defect of the
3696
+ // same severity as a wrong result.
3697
+ await this._onBrokerCancel(queue);
3698
+ return;
3536
3699
  }
3537
3700
 
3538
3701
  // IMPORTANT: