@toa.io/extensions.telemetry 1.0.0-alpha.304 → 1.0.0-alpha.306

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/grafana.md CHANGED
@@ -1,7 +1,8 @@
1
1
  # Grafana stack setup
2
2
 
3
- Toa-specific notes for wiring application traces and metrics into Tempo, Prometheus and Grafana.
4
- Reference configuration: [`observability/`](../../observability) in the Toa repository.
3
+ Toa-specific notes for wiring application traces, metrics and logs into Tempo, Prometheus, Loki
4
+ and Grafana. Reference configuration: [`observability/`](../../observability) in the Toa
5
+ repository.
5
6
 
6
7
  ## Local environment
7
8
 
@@ -11,8 +12,11 @@ The application environment needs:
11
12
  (see [`tempo.yaml`](../../observability/tempo.yaml) for the metrics-generator wiring)
12
13
  - Prometheus with `--web.enable-remote-write-receiver`, which is where Tempo's generator
13
14
  writes, and `--web.enable-otlp-receiver`, which is where Toa posts its own metrics
14
- - Grafana with a Prometheus datasource and a Tempo datasource
15
- with `serviceMap.datasourceUid` pointing at it
15
+ - [Loki](https://grafana.com/oss/loki/) with the OTLP receiver (`/otlp/v1/logs`), on the `tsdb`
16
+ store and schema `v13` — structured metadata, which is where record attributes land, is legal
17
+ only there (see [`loki.yaml`](../../observability/loki.yaml))
18
+ - Grafana with a Prometheus datasource, a Tempo datasource with `serviceMap.datasourceUid`
19
+ pointing at it, and a Loki datasource
16
20
  (see [`grafana-datasources.yaml`](../../observability/grafana-datasources.yaml))
17
21
 
18
22
  Enable the export in the application context:
@@ -30,6 +34,10 @@ telemetry:
30
34
  exporters:
31
35
  otlp:
32
36
  endpoint: http://localhost:9090/api/v1/otlp
37
+ logs:
38
+ exporters:
39
+ otlp:
40
+ endpoint: http://localhost:3100/otlp
33
41
  ```
34
42
 
35
43
  [`dashboards/toa.json`](../../observability/dashboards/toa.json) reads those metrics: operations,
@@ -38,11 +46,15 @@ exposition, the outbox, the broker, storage, what is in flight and the event loo
38
46
  ## Production
39
47
 
40
48
  - Point `traces.exporters.otlp.endpoint` at any OTLP/HTTP receiver (`/v1/traces` is appended),
41
- and `metrics.exporters.otlp.endpoint` at one that takes metrics (`/v1/metrics` is appended).
49
+ `metrics.exporters.otlp.endpoint` at one that takes metrics (`/v1/metrics` is appended), and
50
+ `logs.exporters.otlp.endpoint` at one that takes logs (`/v1/logs` is appended).
42
51
  Use `headers` for authentication (e.g. Grafana Cloud `Authorization: Basic ...`).
43
- - Metrics carry the context as `service.namespace` and the environment as
52
+ - Metrics and logs carry the context as `service.namespace` and the environment as
44
53
  `deployment.environment.name`, so several products and several environments reaching one
45
54
  backend separate by `job` rather than by metric name.
55
+ - The console log exporter stays on unless `logs.exporters.console: false` says otherwise. Turn it
56
+ off where the node also scrapes the pod's stdout into the same backend, or every record arrives
57
+ twice.
46
58
  - Skip the usual per-process `service.name` setup (`OTEL_SERVICE_NAME` and the like):
47
59
  Toa attributes spans to component ids (`default.orders`) or `exposition` automatically,
48
60
  even when multiple components run in a single process.
@@ -53,27 +65,34 @@ exposition, the outbox, the broker, storage, what is in flight and the event loo
53
65
  - Sampling is head-based and decided by Toa (see [Sampling](readme.md#sampling)),
54
66
  so no tail sampling is required on the collector side; `sample` and `rate`
55
67
  are the knobs controlling the exported volume.
56
- - Spans are sent in batches (512 spans or every 5 seconds) and flushed on process exit;
57
- abrupt termination (`SIGKILL`) may lose the last batch.
68
+ - Spans and log records are sent in batches (512 of them or every 5 seconds) and flushed on
69
+ process exit; abrupt termination (`SIGKILL`) may lose the last batch.
58
70
  - Metrics are cumulative, so a lost export costs resolution and not totals: the next one carries
59
71
  what the lost one would have.
60
72
 
61
73
  ## Trace to logs
62
74
 
63
75
  Toa stamps `trace_id` and `span_id` into every log entry written within a span
64
- (including unsampled traces), so once pod logs are shipped to Loki,
65
- traces and logs link up with two datasource settings:
76
+ (including unsampled traces), so traces and logs link up with two datasource settings.
66
77
 
67
- - Tempo datasource, _Trace to logs_: select the Loki datasource,
68
- set time shifts (e.g. `-5m`/`+5m`) and a custom query:
78
+ Exported over OTLP, a record's line is the message alone and everything else is a label or
79
+ structured metadata beside it — Loki maps `service.name` to the `service_name` label and the
80
+ record's trace and span ids to `trace_id` and `span_id`:
81
+
82
+ - Tempo datasource, _Trace to logs_: select the Loki datasource, set time shifts (e.g.
83
+ `-5m`/`+5m`), and a custom query, as the trace id is not in the line:
69
84
 
70
85
  ```logql
71
- {namespace="$NAMESPACE"} | json | trace_id = `${__span.traceId}`
86
+ {service_name="$SERVICE"} | trace_id = `${__span.traceId}`
72
87
  ```
73
88
 
74
89
  Append `` | span_id = `${__span.spanId}` `` to narrow down to a single span.
75
90
 
76
- - Loki datasource, _Derived fields_: regex `"trace_id":"([0-9a-f]+)"`,
77
- query `${__value.raw}`, internal link to the Tempo datasource.
91
+ - Loki datasource, _Derived fields_: matcher type _Label_, regex `trace_id`, query
92
+ `${__value.raw}`, internal link to the Tempo datasource.
93
+
94
+ Where the records are scraped off the pod's stdout instead, the line is the whole JSON entry, and
95
+ the two settings read it as such: `| json | trace_id = ...` in the query, and a derived field
96
+ matching the regular expression `"trace_id":"([0-9a-f]+)"`.
78
97
 
79
98
  Logs of unsampled traces carry a `trace_id` that does not exist in Tempo.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@toa.io/extensions.telemetry",
3
- "version": "1.0.0-alpha.304",
3
+ "version": "1.0.0-alpha.306",
4
4
  "type": "module",
5
5
  "description": "Toa Telemetry extension",
6
6
  "author": "temich <tema.gurtovoy@gmail.com>",
@@ -29,10 +29,10 @@
29
29
  "transpile": "tsc"
30
30
  },
31
31
  "dependencies": {
32
- "@toa.io/core": "1.0.0-alpha.304",
33
- "@toa.io/definitions": "1.0.0-alpha.304",
32
+ "@toa.io/core": "1.0.0-alpha.306",
33
+ "@toa.io/definitions": "1.0.0-alpha.306",
34
34
  "@toa.io/generic": "1.0.0-alpha.304",
35
- "openspan": "1.0.0-alpha.304"
35
+ "openspan": "1.0.0-alpha.305"
36
36
  },
37
- "gitHead": "a95a5a5a64922b11b23142d4acbcee1331ed31aa"
37
+ "gitHead": "0693d180a8053a7d07dca879dac2a3c4784d6d45"
38
38
  }
package/readme.md CHANGED
@@ -60,6 +60,31 @@ telemetry:
60
60
  level: debug
61
61
  ```
62
62
 
63
+ ### Logs exporting
64
+
65
+ An entry goes to every configured log exporter. The console writes the JSON line and is on unless
66
+ it is turned off; `otlp` posts entries to an OTLP/HTTP endpoint and is off until one is
67
+ configured. The two are independent: configuring one says nothing about the other.
68
+
69
+ ```yaml
70
+ # context.toa.yaml
71
+
72
+ telemetry:
73
+ logs:
74
+ level: info
75
+ exporters:
76
+ console: false # stops the JSON line being written; on when omitted
77
+ otlp:
78
+ endpoint: http://loki:3100/otlp # POSTs to {endpoint}/v1/logs
79
+ ```
80
+
81
+ The severity threshold is the one above, per component: what a process writes is what it exports.
82
+ A missing or unavailable endpoint costs a single warning and dropped entries, and never the
83
+ throughput or the shutdown.
84
+
85
+ See [Logs](/documentation/logs.md) for what a record carries and the resource it is exported
86
+ under.
87
+
63
88
  ## Tracing
64
89
 
65
90
  Each operation invocation runs within a _span_.
@@ -266,7 +291,9 @@ Logs are not comments or documentation, nor are they a replacement for them.
266
291
  :-1: Don't:
267
292
 
268
293
  ```javascript
269
- context.logs.error('Failed to send the email, please check the email server configuration')
294
+ context.logs.error(
295
+ 'Failed to send the email, please check the email server configuration'
296
+ )
270
297
  ```
271
298
 
272
299
  :+1: Do:
@@ -274,7 +301,7 @@ context.logs.error('Failed to send the email, please check the email server conf
274
301
  ```javascript
275
302
  context.logs.error('Failed to send the email', {
276
303
  reason: 'SMTP error',
277
- status: response.statusCode,
304
+ status: response.statusCode
278
305
  })
279
306
  ```
280
307
 
@@ -315,7 +342,7 @@ context.logs.debug('Configuration', context.configuration)
315
342
  ```javascript
316
343
  context.logs.debug('Limits', {
317
344
  max: context.configuration.limits.max,
318
- min: context.configuration.limits.min,
345
+ min: context.configuration.limits.min
319
346
  })
320
347
  ```
321
348
 
@@ -81,8 +81,14 @@ export class Ready extends Connector {
81
81
  response.writeHead(404).end();
82
82
  return;
83
83
  }
84
- if (this.ready)
84
+ if (this.ready) {
85
+ /*
86
+ * A halted process answers this as a working one does, and says nothing about the halt:
87
+ * there is nothing wrong with it, and whoever watches it must not replace it — one that
88
+ * came up in its place would come up running.
89
+ */
85
90
  response.writeHead(200, { 'cache-control': 'no-store' }).end();
91
+ }
86
92
  else {
87
93
  const remaining = Math.ceil((Date.now() - this.startedAt) / 1000).toString();
88
94
  response.writeHead(503, { 'retry-after': remaining }).end();
@@ -1 +1 @@
1
- {"version":3,"file":"Ready.js","sourceRoot":"","sources":["../source/Ready.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,IAAI,MAAM,WAAW,CAAA;AACjC,OAAO,EAAE,OAAO,EAAE,MAAM,UAAU,CAAA;AAClC,OAAO,EAAE,SAAS,EAAE,MAAM,cAAc,CAAA;AACxC,OAAO,EAAE,WAAW,EAAE,MAAM,iBAAiB,CAAA;AAC7C,OAAO,EACL,kBAAkB,EAClB,SAAS,EAEV,MAAM,0CAA0C,CAAA;AAEjD,MAAM,OAAO,KAAM,SAAQ,SAAS;IAClB,IAAI,GAAG,OAAO,CAAA;IAEb,MAAM,GAAgB,IAAI,CAAC,YAAY,EAAE,CAAA;IACzC,OAAO,CAAc;IAC9B,KAAK,GAAG,KAAK,CAAA;IACb,SAAS,GAAG,CAAC,CAAA;IACb,SAAS,GAAG,KAAK,CAAA;IACjB,OAAO,GAAG,KAAK,CAAA;IAEvB,YAAmB,OAAqB;QACtC,KAAK,EAAE,CAAA;QAEP,IAAI,CAAC,OAAO,GAAG,OAAO,CAAA;QACtB,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,SAAS,EAAE,CAAC,GAAG,EAAE,GAAG,EAAE,EAAE,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,EAAE,GAAG,CAAC,CAAC,CAAA;IACnE,CAAC;IAEM,MAAM,CAAC,MAAM;QAClB,MAAM,OAAO,GAAG,cAAc,EAAE,CAAA;QAEhC,IAAI,OAAO,KAAK,IAAI;YAAE,OAAO,IAAI,CAAA;QAEjC,OAAO,IAAI,KAAK,CAAC,OAAO,CAAC,CAAA;IAC3B,CAAC;IAEM,KAAK,CAAC,MAAM;QACjB,IAAI,IAAI,CAAC,SAAS,IAAI,IAAI,CAAC,OAAO;YAAE,OAAM;QAE1C,IAAI,CAAC,SAAS,GAAG,IAAI,CAAC,GAAG,EAAE,CAAA;QAE3B,IAAI,CAAC;YACH,MAAM,IAAI,OAAO,CAAO,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;gBAC1C,MAAM,OAAO,GAAG,CAAC,KAAY,EAAQ,EAAE;oBACrC,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,WAAW,EAAE,WAAW,CAAC,CAAA;oBACzC,MAAM,CAAC,KAAK,CAAC,CAAA;gBACf,CAAC,CAAA;gBAED,MAAM,WAAW,GAAG,GAAS,EAAE;oBAC7B,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,OAAO,EAAE,OAAO,CAAC,CAAA;oBACjC,OAAO,EAAE,CAAA;gBACX,CAAC,CAAA;gBAED,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,OAAO,EAAE,OAAO,CAAC,CAAA;gBAClC,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,WAAW,EAAE,WAAW,CAAC,CAAA;gBAC1C,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,CAAA;YACvC,CAAC,CAAC,CAAA;QACJ,CAAC;QAAC,OAAO,KAAU,EAAE,CAAC;YACpB,uEAAuE;YACvE,IAAI,KAAK,EAAE,IAAI,KAAK,YAAY,EAAE,CAAC;gBACjC,IAAI,CAAC,OAAO,GAAG,IAAI,CAAA;gBACnB,OAAO,CAAC,IAAI,CAAC,2CAA2C,EAAE;oBACxD,IAAI,EAAE,IAAI,CAAC,OAAO,CAAC,IAAI;iBACxB,CAAC,CAAA;gBAEF,OAAM;YACR,CAAC;YAED,MAAM,KAAK,CAAA;QACb,CAAC;QAED,IAAI,CAAC,SAAS,GAAG,IAAI,CAAA;QAErB,iGAAiG;QACjG,IAAI,CAAC,MAAM,CAAC,KAAK,EAAE,CAAA;IACrB,CAAC;IAEM,KAAK,CAAC,QAAQ;QACnB,MAAM,IAAI,CAAC,MAAM,EAAE,CAAA;QAEnB,IAAI,CAAC,KAAK,GAAG,IAAI,CAAA;QAEjB,OAAO,CAAC,IAAI,CAAC,OAAO,CAAC,CAAA;QAErB,kFAAkF;QAClF,oFAAoF;QACpF,OAAO,CAAC,IAAI,EAAE,CAAC,OAAO,CAAC,CAAA;IACzB,CAAC;IAEkB,KAAK,CAAC,IAAI;QAC3B,MAAM,IAAI,CAAC,MAAM,EAAE,CAAA;IACrB,CAAC;IAEkB,KAAK,CAAC,KAAK;QAC5B,IAAI,CAAC,KAAK,GAAG,KAAK,CAAA;QAElB,IAAI,CAAC,IAAI,CAAC,SAAS;YAAE,OAAM;QAE3B,IAAI,CAAC,SAAS,GAAG,KAAK,CAAA;QAEtB,mFAAmF;QACnF,IAAI,CAAC,MAAM,CAAC,mBAAmB,EAAE,CAAA;QAEjC,MAAM,IAAI,OAAO,CAAO,CAAC,OAAO,EAAE,EAAE,CAAC,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,OAAO,EAAE,CAAC,CAAC,CAAA;IAC1E,CAAC;IAED,SAAS,CAAC,OAA6B,EAAE,QAA6B;QACpE,IAAI,OAAO,CAAC,GAAG,KAAK,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC;YACtC,QAAQ,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,CAAA;YAE7B,OAAM;QACR,CAAC;QAED,IAAI,IAAI,CAAC,KAAK;YAAE,QAAQ,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,eAAe,EAAE,UAAU,EAAE,CAAC,CAAC,GAAG,EAAE,CAAA;aACzE,CAAC;YACJ,MAAM,SAAS,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,GAAG,EAAE,GAAG,IAAI,CAAC,SAAS,CAAC,GAAG,IAAI,CAAC,CAAC,QAAQ,EAAE,CAAA;YAE5E,QAAQ,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,aAAa,EAAE,SAAS,EAAE,CAAC,CAAC,GAAG,EAAE,CAAA;QAC7D,CAAC;IACH,CAAC;CACF;AAED,MAAM,UAAU,cAAc;IAC5B,MAAM,GAAG,GAAG,WAAW,CAAC,GAAG,CAAC,SAAS,CAAC,CAAA;IAEtC,IAAI,GAAG,KAAK,SAAS;QAAE,OAAO,EAAE,GAAG,QAAQ,EAAE,CAAA;IAE7C,MAAM,OAAO,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAgB,CAAA;IAE9C,IAAI,OAAO,KAAK,KAAK,IAAI,OAAO,CAAC,OAAO,KAAK,KAAK;QAAE,OAAO,IAAI,CAAA;IAE/D,OAAO;QACL,IAAI,EAAE,OAAO,CAAC,IAAI,IAAI,QAAQ,CAAC,IAAI;QACnC,IAAI,EAAE,OAAO,CAAC,IAAI,IAAI,QAAQ,CAAC,IAAI;KACpC,CAAA;AACH,CAAC;AAED,MAAM,QAAQ,GAAiB,EAAE,GAAG,kBAAkB,EAAE,CAAA"}
1
+ {"version":3,"file":"Ready.js","sourceRoot":"","sources":["../source/Ready.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,IAAI,MAAM,WAAW,CAAA;AACjC,OAAO,EAAE,OAAO,EAAE,MAAM,UAAU,CAAA;AAClC,OAAO,EAAE,SAAS,EAAE,MAAM,cAAc,CAAA;AACxC,OAAO,EAAE,WAAW,EAAE,MAAM,iBAAiB,CAAA;AAC7C,OAAO,EACL,kBAAkB,EAClB,SAAS,EAEV,MAAM,0CAA0C,CAAA;AAEjD,MAAM,OAAO,KAAM,SAAQ,SAAS;IAClB,IAAI,GAAG,OAAO,CAAA;IAEb,MAAM,GAAgB,IAAI,CAAC,YAAY,EAAE,CAAA;IACzC,OAAO,CAAc;IAC9B,KAAK,GAAG,KAAK,CAAA;IACb,SAAS,GAAG,CAAC,CAAA;IACb,SAAS,GAAG,KAAK,CAAA;IACjB,OAAO,GAAG,KAAK,CAAA;IAEvB,YAAmB,OAAqB;QACtC,KAAK,EAAE,CAAA;QAEP,IAAI,CAAC,OAAO,GAAG,OAAO,CAAA;QACtB,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,SAAS,EAAE,CAAC,GAAG,EAAE,GAAG,EAAE,EAAE,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,EAAE,GAAG,CAAC,CAAC,CAAA;IACnE,CAAC;IAEM,MAAM,CAAC,MAAM;QAClB,MAAM,OAAO,GAAG,cAAc,EAAE,CAAA;QAEhC,IAAI,OAAO,KAAK,IAAI;YAAE,OAAO,IAAI,CAAA;QAEjC,OAAO,IAAI,KAAK,CAAC,OAAO,CAAC,CAAA;IAC3B,CAAC;IAEM,KAAK,CAAC,MAAM;QACjB,IAAI,IAAI,CAAC,SAAS,IAAI,IAAI,CAAC,OAAO;YAAE,OAAM;QAE1C,IAAI,CAAC,SAAS,GAAG,IAAI,CAAC,GAAG,EAAE,CAAA;QAE3B,IAAI,CAAC;YACH,MAAM,IAAI,OAAO,CAAO,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;gBAC1C,MAAM,OAAO,GAAG,CAAC,KAAY,EAAQ,EAAE;oBACrC,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,WAAW,EAAE,WAAW,CAAC,CAAA;oBACzC,MAAM,CAAC,KAAK,CAAC,CAAA;gBACf,CAAC,CAAA;gBAED,MAAM,WAAW,GAAG,GAAS,EAAE;oBAC7B,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,OAAO,EAAE,OAAO,CAAC,CAAA;oBACjC,OAAO,EAAE,CAAA;gBACX,CAAC,CAAA;gBAED,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,OAAO,EAAE,OAAO,CAAC,CAAA;gBAClC,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,WAAW,EAAE,WAAW,CAAC,CAAA;gBAC1C,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,CAAA;YACvC,CAAC,CAAC,CAAA;QACJ,CAAC;QAAC,OAAO,KAAU,EAAE,CAAC;YACpB,uEAAuE;YACvE,IAAI,KAAK,EAAE,IAAI,KAAK,YAAY,EAAE,CAAC;gBACjC,IAAI,CAAC,OAAO,GAAG,IAAI,CAAA;gBACnB,OAAO,CAAC,IAAI,CAAC,2CAA2C,EAAE;oBACxD,IAAI,EAAE,IAAI,CAAC,OAAO,CAAC,IAAI;iBACxB,CAAC,CAAA;gBAEF,OAAM;YACR,CAAC;YAED,MAAM,KAAK,CAAA;QACb,CAAC;QAED,IAAI,CAAC,SAAS,GAAG,IAAI,CAAA;QAErB,iGAAiG;QACjG,IAAI,CAAC,MAAM,CAAC,KAAK,EAAE,CAAA;IACrB,CAAC;IAEM,KAAK,CAAC,QAAQ;QACnB,MAAM,IAAI,CAAC,MAAM,EAAE,CAAA;QAEnB,IAAI,CAAC,KAAK,GAAG,IAAI,CAAA;QAEjB,OAAO,CAAC,IAAI,CAAC,OAAO,CAAC,CAAA;QAErB,kFAAkF;QAClF,oFAAoF;QACpF,OAAO,CAAC,IAAI,EAAE,CAAC,OAAO,CAAC,CAAA;IACzB,CAAC;IAEkB,KAAK,CAAC,IAAI;QAC3B,MAAM,IAAI,CAAC,MAAM,EAAE,CAAA;IACrB,CAAC;IAEkB,KAAK,CAAC,KAAK;QAC5B,IAAI,CAAC,KAAK,GAAG,KAAK,CAAA;QAElB,IAAI,CAAC,IAAI,CAAC,SAAS;YAAE,OAAM;QAE3B,IAAI,CAAC,SAAS,GAAG,KAAK,CAAA;QAEtB,mFAAmF;QACnF,IAAI,CAAC,MAAM,CAAC,mBAAmB,EAAE,CAAA;QAEjC,MAAM,IAAI,OAAO,CAAO,CAAC,OAAO,EAAE,EAAE,CAAC,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,OAAO,EAAE,CAAC,CAAC,CAAA;IAC1E,CAAC;IAED,SAAS,CAAC,OAA6B,EAAE,QAA6B;QACpE,IAAI,OAAO,CAAC,GAAG,KAAK,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC;YACtC,QAAQ,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,GAAG,EAAE,CAAA;YAE7B,OAAM;QACR,CAAC;QAED,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;YACf;;;;eAIG;YACH,QAAQ,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,eAAe,EAAE,UAAU,EAAE,CAAC,CAAC,GAAG,EAAE,CAAA;QAChE,CAAC;aAAM,CAAC;YACN,MAAM,SAAS,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,GAAG,EAAE,GAAG,IAAI,CAAC,SAAS,CAAC,GAAG,IAAI,CAAC,CAAC,QAAQ,EAAE,CAAA;YAE5E,QAAQ,CAAC,SAAS,CAAC,GAAG,EAAE,EAAE,aAAa,EAAE,SAAS,EAAE,CAAC,CAAC,GAAG,EAAE,CAAA;QAC7D,CAAC;IACH,CAAC;CACF;AAED,MAAM,UAAU,cAAc;IAC5B,MAAM,GAAG,GAAG,WAAW,CAAC,GAAG,CAAC,SAAS,CAAC,CAAA;IAEtC,IAAI,GAAG,KAAK,SAAS;QAAE,OAAO,EAAE,GAAG,QAAQ,EAAE,CAAA;IAE7C,MAAM,OAAO,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAgB,CAAA;IAE9C,IAAI,OAAO,KAAK,KAAK,IAAI,OAAO,CAAC,OAAO,KAAK,KAAK;QAAE,OAAO,IAAI,CAAA;IAE/D,OAAO;QACL,IAAI,EAAE,OAAO,CAAC,IAAI,IAAI,QAAQ,CAAC,IAAI;QACnC,IAAI,EAAE,OAAO,CAAC,IAAI,IAAI,QAAQ,CAAC,IAAI;KACpC,CAAA;AACH,CAAC;AAED,MAAM,QAAQ,GAAiB,EAAE,GAAG,kBAAkB,EAAE,CAAA"}
@@ -3,7 +3,7 @@ import type { Locator } from '@toa.io/core';
3
3
  import type { extensions } from '@toa.io/core/types';
4
4
  type Resident = extensions.Resident;
5
5
  export declare class Factory implements extensions.Factory {
6
- private readonly logsOptions;
6
+ private readonly logsConfig;
7
7
  private readonly ready;
8
8
  constructor();
9
9
  aspect(locator: Locator, declaration: Declaration): extensions.Aspect[];
@@ -1,4 +1,4 @@
1
- import { console, metrics, traces } from 'openspan';
1
+ import { console, logs, metrics, traces } from 'openspan';
2
2
  import { environment } from '@toa.io/generic';
3
3
  import { LOGS_PREFIX, METRICS_ENV, TRACES_ENV } from '@toa.io/definitions/extensions.telemetry';
4
4
  import { Logs } from './Logs.js';
@@ -6,14 +6,16 @@ import { Span } from './Span.js';
6
6
  import { Metrics } from './Metrics.js';
7
7
  import { Ready } from './Ready.js';
8
8
  export class Factory {
9
- logsOptions;
9
+ logsConfig;
10
10
  ready;
11
11
  constructor() {
12
12
  const globEnv = environment.get(LOGS_PREFIX);
13
13
  const level = environment.get('TOA_DEV') === '1' ? 'trace' : 'info';
14
- this.logsOptions = globEnv === undefined ? { level } : JSON.parse(globEnv);
15
- this.logsOptions.level ??= level;
16
- console.configure({ level: this.logsOptions.level });
14
+ this.logsConfig = globEnv === undefined ? { level } : JSON.parse(globEnv);
15
+ this.logsConfig.level ??= level;
16
+ console.configure({ level: this.logsConfig.level });
17
+ // after the level, which is what decides whether an entry is written at all
18
+ logs({ exporters: records(this.logsConfig.exporters) });
17
19
  const tracesEnv = environment.get(TRACES_ENV);
18
20
  const options = tracesEnv === undefined ? development() : JSON.parse(tracesEnv);
19
21
  // `openspan` is not Toa's and reads the environment, which no longer holds the context by
@@ -45,19 +47,24 @@ export class Factory {
45
47
  createLogs(locator) {
46
48
  const overEnv = environment.get(`${LOGS_PREFIX}_${locator.uppercase}`);
47
49
  const override = overEnv !== undefined ? JSON.parse(overEnv) : undefined;
48
- const { level } = Object.assign({}, this.logsOptions, override);
50
+ const { level } = Object.assign({}, this.logsConfig, override);
49
51
  return new Logs(locator, { level });
50
52
  }
51
53
  }
52
54
  /**
53
- * Tracing is off unless it is configured. The console exporter is a local development
54
- * mechanism, so it is turned on for `toa dev` and for a boot trace the CLI has already
55
- * asked for (`runtime/boot/src/span.js`), and nowhere else — a deployment that wants
56
- * traces annotates `telemetry.traces.exporters`.
57
- *
58
- * `extensions/exposition/source/Factory.ts` says the same thing for the gateway process,
59
- * which boots without this extension.
55
+ * What the records say they came from. The same resource the series carry, so that one backend
56
+ * holding several products and several environments separates them by the same attributes
57
+ * whichever signal is being read.
60
58
  */
59
+ function records(exporters) {
60
+ if (exporters?.otlp !== undefined) {
61
+ const resource = (exporters.otlp.resource ??= {});
62
+ resource['service.name'] ??= environment.get('TOA_CONTEXT');
63
+ resource['service.namespace'] ??= environment.get('TOA_CONTEXT');
64
+ resource['deployment.environment.name'] ??= environment.get('TOA_ENV');
65
+ }
66
+ return exporters;
67
+ }
61
68
  /**
62
69
  * What the series say they came from. The metric names are the same in every deployment, so what
63
70
  * tells two products apart in one backend is the resource: the context is the namespace, and the
@@ -79,6 +86,15 @@ function measurements() {
79
86
  }
80
87
  return options;
81
88
  }
89
+ /**
90
+ * Tracing is off unless it is configured. The console exporter is a local development
91
+ * mechanism, so it is turned on for `toa dev` and for a boot trace the CLI has already
92
+ * asked for (`runtime/boot/src/span.js`), and nowhere else — a deployment that wants
93
+ * traces annotates `telemetry.traces.exporters`.
94
+ *
95
+ * `extensions/exposition/source/Factory.ts` says the same thing for the gateway process,
96
+ * which boots without this extension.
97
+ */
82
98
  function development() {
83
99
  const local = environment.get('TOA_DEV') === '1' || environment.get('TOA_BOOT_TRACE') === '1';
84
100
  return local ? { exporters: { console: {} } } : {};
@@ -1 +1 @@
1
- {"version":3,"file":"extension.js","sourceRoot":"","sources":["../source/extension.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,UAAU,CAAA;AACnD,OAAO,EAAE,WAAW,EAAE,MAAM,iBAAiB,CAAA;AAC7C,OAAO,EACL,WAAW,EACX,WAAW,EACX,UAAU,EACX,MAAM,0CAA0C,CAAA;AACjD,OAAO,EAAE,IAAI,EAAE,MAAM,WAAW,CAAA;AAChC,OAAO,EAAE,IAAI,EAAE,MAAM,WAAW,CAAA;AAChC,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAA;AACtC,OAAO,EAAE,KAAK,EAAE,MAAM,YAAY,CAAA;AASlC,MAAM,OAAO,OAAO;IACD,WAAW,CAAa;IACxB,KAAK,CAAc;IAEpC;QACE,MAAM,OAAO,GAAG,WAAW,CAAC,GAAG,CAAC,WAAW,CAAC,CAAA;QAC5C,MAAM,KAAK,GAAG,WAAW,CAAC,GAAG,CAAC,SAAS,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAA;QAEnE,IAAI,CAAC,WAAW,GAAG,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAA;QAC1E,IAAI,CAAC,WAAW,CAAC,KAAK,KAAK,KAAK,CAAA;QAEhC,OAAO,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,IAAI,CAAC,WAAW,CAAC,KAAK,EAAE,CAAC,CAAA;QAEpD,MAAM,SAAS,GAAG,WAAW,CAAC,GAAG,CAAC,UAAU,CAAC,CAAA;QAC7C,MAAM,OAAO,GACX,SAAS,KAAK,SAAS,CAAC,CAAC,CAAC,WAAW,EAAE,CAAC,CAAC,CAAE,IAAI,CAAC,KAAK,CAAC,SAAS,CAAmB,CAAA;QAEpF,0FAA0F;QAC1F,+DAA+D;QAC/D,IAAI,OAAO,CAAC,SAAS,EAAE,IAAI,KAAK,SAAS;YACvC,OAAO,CAAC,SAAS,CAAC,IAAI,CAAC,OAAO,KAAK,WAAW,CAAC,GAAG,CAAC,aAAa,CAAC,CAAA;QAEnE,MAAM,CAAC,OAAO,CAAC,CAAA;QACf,OAAO,CAAC,YAAY,EAAE,CAAC,CAAA;QAEvB,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC,MAAM,EAAE,CAAA;IAC7B,CAAC;IAEM,MAAM,CAAC,OAAgB,EAAE,WAAwB;QACtD,MAAM,IAAI,GAAG,IAAI,CAAC,UAAU,CAAC,OAAO,CAAC,CAAA;QACrC,MAAM,IAAI,GAAG,IAAI,IAAI,CAAC,OAAO,CAAC,CAAA;QAC9B,MAAM,OAAO,GAAG,IAAI,OAAO,CAAC,OAAO,EAAE,WAAW,IAAI,EAAE,CAAC,CAAA;QAEvD,OAAO,CAAC,IAAI,EAAE,IAAI,EAAE,OAAO,CAAC,CAAA;IAC9B,CAAC;IAED;;;;;;;;OAQG;IACI,QAAQ;QACb,OAAO,IAAI,CAAC,KAAK,CAAA;IACnB,CAAC;IAEO,UAAU,CAAC,OAAgB;QACjC,MAAM,OAAO,GAAG,WAAW,CAAC,GAAG,CAAC,GAAG,WAAW,IAAI,OAAO,CAAC,SAAS,EAAE,CAAC,CAAA;QACtE,MAAM,QAAQ,GAAG,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,SAAS,CAAA;QAExE,MAAM,EAAE,KAAK,EAAE,GAAG,MAAM,CAAC,MAAM,CAAC,EAAE,EAAE,IAAI,CAAC,WAAW,EAAE,QAAQ,CAAC,CAAA;QAE/D,OAAO,IAAI,IAAI,CAAC,OAAO,EAAE,EAAE,KAAK,EAAE,CAAC,CAAA;IACrC,CAAC;CACF;AAED;;;;;;;;GAQG;AACH;;;;;;;GAOG;AACH,SAAS,YAAY;IACnB,MAAM,GAAG,GAAG,WAAW,CAAC,GAAG,CAAC,WAAW,CAAC,CAAA;IAExC,IAAI,GAAG,KAAK,SAAS;QAAE,OAAO,SAAS,CAAA;IAEvC,MAAM,OAAO,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAmB,CAAA;IAEjD,OAAO,CAAC,MAAM,KAAK,KAAK,CAAA;IAExB,IAAI,OAAO,CAAC,SAAS,EAAE,IAAI,KAAK,SAAS,EAAE,CAAC;QAC1C,MAAM,QAAQ,GAAG,CAAC,OAAO,CAAC,SAAS,CAAC,IAAI,CAAC,QAAQ,KAAK,EAAE,CAAC,CAAA;QAEzD,QAAQ,CAAC,mBAAmB,CAAC,KAAK,WAAW,CAAC,GAAG,CAAC,aAAa,CAAC,CAAA;QAChE,QAAQ,CAAC,6BAA6B,CAAC,KAAK,WAAW,CAAC,GAAG,CAAC,SAAS,CAAC,CAAA;IACxE,CAAC;IAED,OAAO,OAAO,CAAA;AAChB,CAAC;AAED,SAAS,WAAW;IAClB,MAAM,KAAK,GACT,WAAW,CAAC,GAAG,CAAC,SAAS,CAAC,KAAK,GAAG,IAAI,WAAW,CAAC,GAAG,CAAC,gBAAgB,CAAC,KAAK,GAAG,CAAA;IAEjF,OAAO,KAAK,CAAC,CAAC,CAAC,EAAE,SAAS,EAAE,EAAE,OAAO,EAAE,EAAE,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,CAAA;AACpD,CAAC"}
1
+ {"version":3,"file":"extension.js","sourceRoot":"","sources":["../source/extension.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,UAAU,CAAA;AACzD,OAAO,EAAE,WAAW,EAAE,MAAM,iBAAiB,CAAA;AAC7C,OAAO,EACL,WAAW,EACX,WAAW,EACX,UAAU,EACX,MAAM,0CAA0C,CAAA;AACjD,OAAO,EAAE,IAAI,EAAE,MAAM,WAAW,CAAA;AAChC,OAAO,EAAE,IAAI,EAAE,MAAM,WAAW,CAAA;AAChC,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAA;AACtC,OAAO,EAAE,KAAK,EAAE,MAAM,YAAY,CAAA;AASlC,MAAM,OAAO,OAAO;IACD,UAAU,CAAY;IACtB,KAAK,CAAc;IAEpC;QACE,MAAM,OAAO,GAAG,WAAW,CAAC,GAAG,CAAC,WAAW,CAAC,CAAA;QAC5C,MAAM,KAAK,GAAG,WAAW,CAAC,GAAG,CAAC,SAAS,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAA;QAEnE,IAAI,CAAC,UAAU,GAAG,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAA;QACzE,IAAI,CAAC,UAAU,CAAC,KAAK,KAAK,KAAK,CAAA;QAE/B,OAAO,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,IAAI,CAAC,UAAU,CAAC,KAAK,EAAE,CAAC,CAAA;QAEnD,4EAA4E;QAC5E,IAAI,CAAC,EAAE,SAAS,EAAE,OAAO,CAAC,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,EAAE,CAAC,CAAA;QAEvD,MAAM,SAAS,GAAG,WAAW,CAAC,GAAG,CAAC,UAAU,CAAC,CAAA;QAC7C,MAAM,OAAO,GACX,SAAS,KAAK,SAAS,CAAC,CAAC,CAAC,WAAW,EAAE,CAAC,CAAC,CAAE,IAAI,CAAC,KAAK,CAAC,SAAS,CAAmB,CAAA;QAEpF,0FAA0F;QAC1F,+DAA+D;QAC/D,IAAI,OAAO,CAAC,SAAS,EAAE,IAAI,KAAK,SAAS;YACvC,OAAO,CAAC,SAAS,CAAC,IAAI,CAAC,OAAO,KAAK,WAAW,CAAC,GAAG,CAAC,aAAa,CAAC,CAAA;QAEnE,MAAM,CAAC,OAAO,CAAC,CAAA;QACf,OAAO,CAAC,YAAY,EAAE,CAAC,CAAA;QAEvB,IAAI,CAAC,KAAK,GAAG,KAAK,CAAC,MAAM,EAAE,CAAA;IAC7B,CAAC;IAEM,MAAM,CAAC,OAAgB,EAAE,WAAwB;QACtD,MAAM,IAAI,GAAG,IAAI,CAAC,UAAU,CAAC,OAAO,CAAC,CAAA;QACrC,MAAM,IAAI,GAAG,IAAI,IAAI,CAAC,OAAO,CAAC,CAAA;QAC9B,MAAM,OAAO,GAAG,IAAI,OAAO,CAAC,OAAO,EAAE,WAAW,IAAI,EAAE,CAAC,CAAA;QAEvD,OAAO,CAAC,IAAI,EAAE,IAAI,EAAE,OAAO,CAAC,CAAA;IAC9B,CAAC;IAED;;;;;;;;OAQG;IACI,QAAQ;QACb,OAAO,IAAI,CAAC,KAAK,CAAA;IACnB,CAAC;IAEO,UAAU,CAAC,OAAgB;QACjC,MAAM,OAAO,GAAG,WAAW,CAAC,GAAG,CAAC,GAAG,WAAW,IAAI,OAAO,CAAC,SAAS,EAAE,CAAC,CAAA;QACtE,MAAM,QAAQ,GAAG,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,SAAS,CAAA;QAExE,MAAM,EAAE,KAAK,EAAE,GAAG,MAAM,CAAC,MAAM,CAAC,EAAE,EAAE,IAAI,CAAC,UAAU,EAAE,QAAQ,CAAC,CAAA;QAE9D,OAAO,IAAI,IAAI,CAAC,OAAO,EAAE,EAAE,KAAK,EAAE,CAAC,CAAA;IACrC,CAAC;CACF;AAED;;;;GAIG;AACH,SAAS,OAAO,CAAC,SAA8B;IAC7C,IAAI,SAAS,EAAE,IAAI,KAAK,SAAS,EAAE,CAAC;QAClC,MAAM,QAAQ,GAAG,CAAC,SAAS,CAAC,IAAI,CAAC,QAAQ,KAAK,EAAE,CAAC,CAAA;QAEjD,QAAQ,CAAC,cAAc,CAAC,KAAK,WAAW,CAAC,GAAG,CAAC,aAAa,CAAC,CAAA;QAC3D,QAAQ,CAAC,mBAAmB,CAAC,KAAK,WAAW,CAAC,GAAG,CAAC,aAAa,CAAC,CAAA;QAChE,QAAQ,CAAC,6BAA6B,CAAC,KAAK,WAAW,CAAC,GAAG,CAAC,SAAS,CAAC,CAAA;IACxE,CAAC;IAED,OAAO,SAAS,CAAA;AAClB,CAAC;AAED;;;;;;;GAOG;AACH,SAAS,YAAY;IACnB,MAAM,GAAG,GAAG,WAAW,CAAC,GAAG,CAAC,WAAW,CAAC,CAAA;IAExC,IAAI,GAAG,KAAK,SAAS;QAAE,OAAO,SAAS,CAAA;IAEvC,MAAM,OAAO,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAmB,CAAA;IAEjD,OAAO,CAAC,MAAM,KAAK,KAAK,CAAA;IAExB,IAAI,OAAO,CAAC,SAAS,EAAE,IAAI,KAAK,SAAS,EAAE,CAAC;QAC1C,MAAM,QAAQ,GAAG,CAAC,OAAO,CAAC,SAAS,CAAC,IAAI,CAAC,QAAQ,KAAK,EAAE,CAAC,CAAA;QAEzD,QAAQ,CAAC,mBAAmB,CAAC,KAAK,WAAW,CAAC,GAAG,CAAC,aAAa,CAAC,CAAA;QAChE,QAAQ,CAAC,6BAA6B,CAAC,KAAK,WAAW,CAAC,GAAG,CAAC,SAAS,CAAC,CAAA;IACxE,CAAC;IAED,OAAO,OAAO,CAAA;AAChB,CAAC;AAMD;;;;;;;;GAQG;AACH,SAAS,WAAW;IAClB,MAAM,KAAK,GACT,WAAW,CAAC,GAAG,CAAC,SAAS,CAAC,KAAK,GAAG,IAAI,WAAW,CAAC,GAAG,CAAC,gBAAgB,CAAC,KAAK,GAAG,CAAA;IAEjF,OAAO,KAAK,CAAC,CAAC,CAAC,EAAE,SAAS,EAAE,EAAE,OAAO,EAAE,EAAE,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,CAAA;AACpD,CAAC"}