@powerhousedao/switchboard 6.2.3-dev.22 → 6.2.3-dev.24

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. package/.tsbuild/test/boot-create-signature-policy.test.d.ts +2 -0
  2. package/.tsbuild/test/boot-create-signature-policy.test.d.ts.map +1 -0
  3. package/.tsbuild/test/boot-without-renown.test.d.ts +2 -0
  4. package/.tsbuild/test/boot-without-renown.test.d.ts.map +1 -0
  5. package/.tsbuild/test/boot-without-require-signatures.test.d.ts +2 -0
  6. package/.tsbuild/test/boot-without-require-signatures.test.d.ts.map +1 -0
  7. package/.tsbuild/test/boot-worker-trust-policy.test.d.ts +2 -0
  8. package/.tsbuild/test/boot-worker-trust-policy.test.d.ts.map +1 -0
  9. package/.tsbuild/test/builder-defaults.test.d.ts +2 -0
  10. package/.tsbuild/test/builder-defaults.test.d.ts.map +1 -0
  11. package/.tsbuild/test/create-signature-policy.test.d.ts +2 -0
  12. package/.tsbuild/test/create-signature-policy.test.d.ts.map +1 -0
  13. package/.tsbuild/test/fatal-shutdown.test.d.ts +2 -0
  14. package/.tsbuild/test/fatal-shutdown.test.d.ts.map +1 -0
  15. package/.tsbuild/test/renown-trust-policy.test.d.ts +2 -0
  16. package/.tsbuild/test/renown-trust-policy.test.d.ts.map +1 -0
  17. package/.tsbuild/tsconfig.tsbuildinfo +1 -1
  18. package/Auth.md +41 -0
  19. package/CHANGELOG.md +33 -0
  20. package/README.md +14 -0
  21. package/dist/index.mjs +6 -5
  22. package/dist/index.mjs.map +1 -1
  23. package/dist/{server-C6VtW18k.mjs → server-BFbus-at.mjs} +225 -33
  24. package/dist/server-BFbus-at.mjs.map +1 -0
  25. package/dist/server.d.mts +19 -5
  26. package/dist/server.d.mts.map +1 -1
  27. package/dist/server.mjs +4 -4
  28. package/dist/{utils-Baw7rThP.mjs → utils-BBHt2kdo.mjs} +15 -9
  29. package/dist/utils-BBHt2kdo.mjs.map +1 -0
  30. package/dist/utils.d.mts.map +1 -1
  31. package/dist/utils.mjs +3 -3
  32. package/package.json +16 -16
  33. package/test/attachments/auth.test.ts +85 -0
  34. package/test/attachments/index.test.ts +34 -3
  35. package/test/attachments/routes-integration.test.ts +40 -0
  36. package/test/attachments/routes.test.ts +126 -2
  37. package/test/boot-create-signature-policy.test.ts +108 -0
  38. package/test/boot-without-renown.test.ts +81 -0
  39. package/test/boot-without-require-signatures.test.ts +74 -0
  40. package/test/boot-worker-trust-policy.test.ts +90 -0
  41. package/test/builder-defaults.test.ts +80 -0
  42. package/test/create-signature-policy.test.ts +69 -0
  43. package/test/fatal-shutdown.test.ts +102 -0
  44. package/test/renown-trust-policy.test.ts +315 -0
  45. package/dist/server-C6VtW18k.mjs.map +0 -1
  46. package/dist/utils-Baw7rThP.mjs.map +0 -1
package/Auth.md CHANGED
@@ -221,6 +221,47 @@ It requires a caller to be resolvable at all, so it refuses to boot without
221
221
  resolution off, no bearer is ever read and it would reject every caller,
222
222
  including authenticated ones.
223
223
 
224
+ It also covers the attachment routes, which are mounted on the HTTP adapter and
225
+ never pass the GraphQL fetch chain. Their own 401 keys on `AUTH_ENABLED`, so
226
+ without this switch a deployment running `OPEN` serves them to anyone; with it,
227
+ every attachment route refuses a caller it cannot name — `download-target`
228
+ included, even though that route decides per document on its own.
229
+
230
+ ##### Serving one path anonymously
231
+
232
+ Some products have a flow that runs *before* sign-in: previewing an invitation
233
+ from the code in its e-mail, for instance, so the screen can say who the
234
+ invitation is for. That flow has no caller by definition, and the floor above
235
+ would answer it a `401`.
236
+
237
+ ```bash
238
+ export REQUIRE_AUTHENTICATED_CALLER=true
239
+ export REQUIRE_AUTHENTICATED_CALLER_EXEMPT_PATHS=/graphql/public
240
+ ```
241
+
242
+ Comma-separated, so several paths are one variable:
243
+
244
+ ```bash
245
+ export REQUIRE_AUTHENTICATED_CALLER_EXEMPT_PATHS=/graphql/public,/graphql/invites
246
+ ```
247
+
248
+ Each entry is matched against the request's **pathname, in full** — never as a
249
+ prefix, so `/graphql/public` does not also exempt `/graphql/public-admin`. A
250
+ trailing slash on either side is ignored; a leading slash is required, and a
251
+ path without one is refused at boot rather than left as an exemption that
252
+ silently never applies. Configuring exemptions while the floor is off is
253
+ refused too: it would describe a protection the server is not applying.
254
+
255
+ The exempt paths are named in the boot log, because the one question worth
256
+ asking about this floor is what is still open.
257
+
258
+ Every entry is a hole in it. Only ever name a path that serves operations which
259
+ are safe without a caller, and prefer a route mounted for exactly that purpose
260
+ over exempting one that also serves something else. Note what this switch does
261
+ **not** reach, in either configuration: routes registered directly on the HTTP
262
+ adapter (`/health`, `/ready`, `/explorer`, `/d/:drive`) and package HTTP routes,
263
+ which declare their own `auth` per route.
264
+
224
265
  #### Configuration File Method
225
266
 
226
267
  ```json
package/CHANGELOG.md CHANGED
@@ -1,3 +1,36 @@
1
+ ## 6.2.3-dev.24 (2026-09-25)
2
+
3
+ ### 🚀 Features
4
+
5
+ - **reactor-api:** exempt named paths from the authenticated-caller floor, and carry it to the attachment routes ([#3108](https://github.com/powerhouse-inc/powerhouse/pull/3108))
6
+ - **reactor:** action signature integrity ([#3088](https://github.com/powerhouse-inc/powerhouse/pull/3088), [#2894](https://github.com/powerhouse-inc/powerhouse/issues/2894), [#7](https://github.com/powerhouse-inc/powerhouse/issues/7))
7
+ - **reactor-workflow:** refuse the auth and trigger features the engine can't run ([7ce03d28f](https://github.com/powerhouse-inc/powerhouse/commit/7ce03d28f))
8
+ - type-safe piece authoring, upstream connection hooks, and a ph build that typechecks first ([#3082](https://github.com/powerhouse-inc/powerhouse/pull/3082))
9
+
10
+ ### 🩹 Fixes
11
+
12
+ - **workflow:** build after powerhouse-vetra-packages so their tsc --build runs never overlap ([#3101](https://github.com/powerhouse-inc/powerhouse/pull/3101))
13
+
14
+ ### ❤️ Thank You
15
+
16
+ - acaldas
17
+ - Benjamin Jordan
18
+ - Claude Fable 5.1
19
+ - Guillermo Puente Sandoval @gpuente
20
+
21
+ ## 6.2.3-dev.23 (2026-09-24)
22
+
23
+ ### 🩹 Fixes
24
+
25
+ - **switchboard:** leave unhandled rejections to Sentry when it listens ([c6a40108e](https://github.com/powerhouse-inc/powerhouse/commit/c6a40108e))
26
+ - **switchboard:** flush PGlite before exiting on an uncaught error ([#3077](https://github.com/powerhouse-inc/powerhouse/issues/3077))
27
+ - **switchboard:** escape non-ASCII file names in attachment metadata headers ([#3077](https://github.com/powerhouse-inc/powerhouse/issues/3077))
28
+
29
+ ### ❤️ Thank You
30
+
31
+ - Benjamin Jordan
32
+ - Claude Opus 5.5
33
+
1
34
  ## 6.2.3-dev.22 (2026-09-23)
2
35
 
3
36
  ### 🩹 Fixes
package/README.md CHANGED
@@ -186,6 +186,20 @@ exits non-zero instead of waiting out the init timeout.
186
186
 
187
187
  Programmatically: `startSwitchboard({ projectionWorker: { enabled, dbPoolSize } })`.
188
188
 
189
+ ### Creation Signature Policy
190
+
191
+ | Variable | Description | Default |
192
+ | ------------------------- | ------------------------------------------------------------ | ------------- |
193
+ | `CREATE_SIGNATURE_POLICY` | What new documents are created as: `legacy` or `v2-required` | `v2-required` |
194
+
195
+ A v2-required document accepts only v2 action signatures and takes a
196
+ content-addressed id. Set `legacy` while any reactor that syncs with this one
197
+ predates v2-required documents. It only decides what new documents are born
198
+ as; existing documents keep their policy, and it does not change what the
199
+ switchboard accepts. Without a Renown identity the switchboard has no signer,
200
+ so it creates legacy documents and logs a warning. A default drive configured
201
+ with a fixed `id` is always created legacy.
202
+
189
203
  ### Reactor Enforcement Flags
190
204
 
191
205
  | Variable | Description | Default |
package/dist/index.mjs CHANGED
@@ -1,8 +1,8 @@
1
1
  #!/usr/bin/env node
2
2
 
3
- !function(){try{var e="undefined"!=typeof window?window:"undefined"!=typeof global?global:"undefined"!=typeof globalThis?globalThis:"undefined"!=typeof self?self:{},n=(new e.Error).stack;n&&(e._sentryDebugIds=e._sentryDebugIds||{},e._sentryDebugIds[n]="c6c10af1-f4cc-56a5-abf6-fa35cf0de956")}catch(e){}}();
4
- import { a as parseForcePgVersion, r as startSwitchboard } from "./server-C6VtW18k.mjs";
5
- import "./utils-Baw7rThP.mjs";
3
+ !function(){try{var e="undefined"!=typeof window?window:"undefined"!=typeof global?global:"undefined"!=typeof globalThis?globalThis:"undefined"!=typeof self?self:{},n=(new e.Error).stack;n&&(e._sentryDebugIds=e._sentryDebugIds||{},e._sentryDebugIds[n]="2ce93cee-94b7-54c2-9688-4127efd686af")}catch(e){}}();
4
+ import { a as parseForcePgVersion, r as startSwitchboard } from "./server-BFbus-at.mjs";
5
+ import "./utils-BBHt2kdo.mjs";
6
6
  import { metrics } from "@opentelemetry/api";
7
7
  import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
8
8
  import { ExpressInstrumentation } from "@opentelemetry/instrumentation-express";
@@ -246,7 +246,8 @@ try {
246
246
  await startSwitchboard({
247
247
  ...config,
248
248
  migratePglite: cliMigratePglite || config.migratePglite,
249
- forcePgVersion: config.forcePgVersion ?? void 0
249
+ forcePgVersion: config.forcePgVersion ?? void 0,
250
+ fatalErrorShutdown: true
250
251
  });
251
252
  } catch (e) {
252
253
  Sentry.captureException(e);
@@ -258,4 +259,4 @@ try {
258
259
  export {};
259
260
 
260
261
  //# sourceMappingURL=index.mjs.map
261
- //# debugId=c6c10af1-f4cc-56a5-abf6-fa35cf0de956
262
+ //# debugId=2ce93cee-94b7-54c2-9688-4127efd686af
@@ -1 +1 @@
1
- {"version":3,"file":"index.mjs","sources":["../src/metrics.ts","../src/redact-webhook-path.ts","../src/observability.mts","../src/config.ts","../src/profiler.ts","../src/index.mts"],"sourcesContent":["import { OTLPMetricExporter } from \"@opentelemetry/exporter-metrics-otlp-http\";\nimport { resourceFromAttributes } from \"@opentelemetry/resources\";\nimport {\n MeterProvider,\n PeriodicExportingMetricReader,\n} from \"@opentelemetry/sdk-metrics\";\nimport { childLogger } from \"document-model\";\n\nconst logger = childLogger([\"switchboard\", \"metrics\"]);\n\nexport function createMeterProviderFromEnv(env: {\n OTEL_EXPORTER_OTLP_ENDPOINT?: string;\n OTEL_METRIC_EXPORT_INTERVAL?: string;\n OTEL_SERVICE_NAME?: string;\n}): MeterProvider | undefined {\n const endpoint = env.OTEL_EXPORTER_OTLP_ENDPOINT;\n if (!endpoint) return undefined;\n\n const parsed = parseInt(env.OTEL_METRIC_EXPORT_INTERVAL ?? \"\", 10);\n const exportIntervalMillis =\n Number.isFinite(parsed) && parsed > 0 ? parsed : 5_000;\n\n const base = endpoint.replace(/\\/$/, \"\");\n const exporterUrl = base.endsWith(\"/v1/metrics\")\n ? base\n : `${base}/v1/metrics`;\n\n logger.info(`Initializing OpenTelemetry metrics exporter at: ${endpoint}`);\n const meterProvider = new MeterProvider({\n resource: resourceFromAttributes({\n \"service.name\": env.OTEL_SERVICE_NAME ?? \"switchboard\",\n }),\n readers: [\n new PeriodicExportingMetricReader({\n exporter: new OTLPMetricExporter({\n url: exporterUrl,\n }),\n exportIntervalMillis,\n exportTimeoutMillis: Math.max(exportIntervalMillis - 250, 1),\n }),\n ],\n });\n logger.info(`Metrics export enabled (interval: ${exportIntervalMillis}ms)`);\n return meterProvider;\n}\n","const WEBHOOK_SEGMENT = \"/webhooks/\";\n\n/**\n * Strips a webhook's delivery token out of a request path.\n *\n * The token is the endpoint's whole credential and it sits in the path, so it\n * must not reach a log line or a span attribute that ships off-box. The family\n * mounts under the host's base path, so the segment can sit at any depth.\n */\nexport function redactWebhookPath(url: string | undefined): string {\n if (!url) return \"\";\n // Dropped with the token: `http.route` is meant to be a template, and a\n // query string is another place a sender can put a credential.\n const path = url.split(\"?\")[0] ?? url;\n const at = path.indexOf(WEBHOOK_SEGMENT);\n if (at === -1) return url;\n // Nothing after the segment is nothing to redact.\n if (path.length === at + WEBHOOK_SEGMENT.length) return url;\n return `${path.slice(0, at)}${WEBHOOK_SEGMENT}[redacted]`;\n}\n","// Single observability bootstrap: Sentry + OpenTelemetry (tracing + metrics).\n//\n// MUST be imported as the very first module in apps/switchboard/src/index.mts.\n// OpenTelemetry instrumentations register require-time hooks at module load,\n// so http/express/pg/graphql must not be imported (transitively) before this\n// file runs.\n//\n// Replaces three legacy bootstrap sites:\n// - apps/switchboard/src/server.mts top-level Sentry.init\n// - apps/switchboard/src/metrics.ts standalone MeterProvider (still exported\n// here via createMeterProviderFromEnv so its tests keep passing)\n// - packages/reactor-api/src/tracing.ts side-effect NodeSDK\nimport { metrics } from \"@opentelemetry/api\";\nimport { OTLPTraceExporter } from \"@opentelemetry/exporter-trace-otlp-http\";\nimport { ExpressInstrumentation } from \"@opentelemetry/instrumentation-express\";\nimport { GraphQLInstrumentation } from \"@opentelemetry/instrumentation-graphql\";\nimport { HttpInstrumentation } from \"@opentelemetry/instrumentation-http\";\nimport { PgInstrumentation } from \"@opentelemetry/instrumentation-pg\";\nimport { UndiciInstrumentation } from \"@opentelemetry/instrumentation-undici\";\nimport { resourceFromAttributes } from \"@opentelemetry/resources\";\nimport type { MeterProvider } from \"@opentelemetry/sdk-metrics\";\nimport { NodeSDK } from \"@opentelemetry/sdk-node\";\nimport {\n BatchSpanProcessor,\n type SpanProcessor,\n} from \"@opentelemetry/sdk-trace-base\";\nimport {\n ATTR_SERVICE_NAME,\n ATTR_SERVICE_VERSION,\n} from \"@opentelemetry/semantic-conventions\";\nimport {\n registerEventLoopInstrumentation,\n type EventLoopInstrumentation,\n} from \"@powerhousedao/opentelemetry-instrumentation-reactor\";\nimport * as Sentry from \"@sentry/node\";\nimport { SentryPropagator, SentrySpanProcessor } from \"@sentry/opentelemetry\";\nimport { childLogger } from \"document-model\";\nimport type { IncomingMessage } from \"node:http\";\nimport { createMeterProviderFromEnv } from \"./metrics.js\";\nimport { redactWebhookPath } from \"./redact-webhook-path.js\";\n\nconst logger = childLogger([\"switchboard\", \"observability\"]);\n\nconst SERVICE_NAME = process.env.OTEL_SERVICE_NAME || \"switchboard\";\nconst SERVICE_VERSION = process.env.npm_package_version || \"unknown\";\nconst TENANT_ID = process.env.TENANT_ID || \"default\";\nconst DEPLOY_ENV = process.env.NODE_ENV || \"development\";\n\nconst TEMPO_ENDPOINT = process.env.TEMPO_ENDPOINT;\nconst SENTRY_DSN = process.env.SENTRY_DSN;\n\n// Whether to forward TRANSACTIONS (spans) to Sentry. Errors are always sent\n// when SENTRY_DSN is set — this flag only gates APM/tracing into Sentry.\n// Default on (back-compat). Set SENTRY_TRACING_ENABLED=false for an\n// \"errors-only\" deployment: errors still go to Sentry, traces still go to\n// Tempo, but no transactions hit Sentry (no Kafka/ClickHouse/nodestore\n// cost). This is the recommended mode for tenant workloads at scale —\n// Sentry's value there is error grouping; traces live in Tempo.\nconst SENTRY_TRACING_TO_SENTRY =\n Boolean(SENTRY_DSN) && process.env.SENTRY_TRACING_ENABLED !== \"false\";\n\nconst TRACING_REQUESTED =\n process.env.ENABLE_TRACING === \"true\" ||\n process.env.NODE_ENV === \"production\";\nconst HAS_TRACE_DESTINATION = Boolean(TEMPO_ENDPOINT) || Boolean(SENTRY_DSN);\nconst TRACING_ENABLED = TRACING_REQUESTED && HAS_TRACE_DESTINATION;\n\nif (TRACING_REQUESTED && !HAS_TRACE_DESTINATION) {\n logger.warn(\n \"Tracing was requested (NODE_ENV=production or ENABLE_TRACING=true) but \" +\n \"no destination is configured — instrumentation will not run. Set \" +\n \"TEMPO_ENDPOINT (e.g. http://tempo.monitoring.svc.cluster.local:4318/v1/traces) \" +\n \"to export OTLP spans, and/or SENTRY_DSN to forward spans to Sentry.\",\n );\n}\n\n// APM sampling for the Sentry-SDK-managed path (i.e. when TRACING is OFF and\n// @sentry/node runs its own bundled OTel — see skipOpenTelemetrySetup below).\n//\n// IMPORTANT: when TRACING_ENABLED, this value does NOT govern span volume.\n// Our NodeSDK (below) owns the pipeline and is constructed with no explicit\n// `sampler`, so @opentelemetry/sdk-node falls back to buildSamplerFromEnv()\n// and the REAL head-sampling knob is the standard OTEL_TRACES_SAMPLER /\n// OTEL_TRACES_SAMPLER_ARG env (set per-deploy in the k8s chart). That head\n// decision gates spans before any processor runs, so it bounds BOTH the\n// SentrySpanProcessor (→ Sentry transactions) and the Tempo OTLP export.\n// (Wiring @sentry/opentelemetry's SentrySampler here would let this rate\n// drive both backends and make DSC/sample_rand propagation spec-correct, but\n// that only matters for Sentry server-side dynamic sampling — a SaaS feature\n// our self-hosted install doesn't run — so it's intentionally deferred.)\nconst SENTRY_TRACES_SAMPLE_RATE = parseFloat(\n process.env.SENTRY_TRACES_SAMPLE_RATE ?? \"0.1\",\n);\n\nif (SENTRY_DSN) {\n logger.info(\"Initialized Sentry with env: @env\", process.env.SENTRY_ENV);\n Sentry.init({\n dsn: SENTRY_DSN,\n environment: process.env.SENTRY_ENV,\n // Match the version tag uploaded by release-branch.yml so source maps\n // resolve. Populated by the CI (WORKSPACE_VERSION) or npm at runtime.\n release:\n process.env.SENTRY_RELEASE ||\n (process.env.npm_package_version\n ? `v${process.env.npm_package_version}`\n : undefined),\n // 0 in errors-only mode so even @sentry/node's bundled auto-OTel path\n // (used when TRACING_ENABLED is false) produces no transactions.\n tracesSampleRate: SENTRY_TRACING_TO_SENTRY ? SENTRY_TRACES_SAMPLE_RATE : 0,\n // When tracing is on, our NodeSDK below owns the OTel globals and Sentry\n // receives spans via SentrySpanProcessor. Skipping Sentry's bundled OTel\n // setup avoids two TracerProviders fighting over setGlobalTracerProvider.\n // When tracing is off, leave the flag unset so @sentry/node's default\n // auto-OTel still records HTTP transactions for the APM dashboard.\n skipOpenTelemetrySetup: TRACING_ENABLED,\n });\n}\n\nconst meterProvider: MeterProvider | undefined = createMeterProviderFromEnv({\n OTEL_EXPORTER_OTLP_ENDPOINT: process.env.OTEL_EXPORTER_OTLP_ENDPOINT,\n OTEL_METRIC_EXPORT_INTERVAL: process.env.OTEL_METRIC_EXPORT_INTERVAL,\n OTEL_SERVICE_NAME: process.env.OTEL_SERVICE_NAME,\n});\nlet eventLoopInstrumentation: EventLoopInstrumentation | undefined;\nif (meterProvider) {\n // One-way door: must register before any code calls metrics.getMeter() —\n // most notably ReactorInstrumentation inside the reactor module.\n metrics.setGlobalMeterProvider(meterProvider);\n // Host event-loop and CPU gauges (reactor.host.eventloop.*,\n // reactor.host.cpu.utilization). The bench sweep identified the host event\n // loop, not the executor, as the binding constraint on write throughput\n // (bench/test/integration/BASELINE.md, Run 10) — these are the production\n // counterpart of the metrics that finding was made from. Must come after\n // setGlobalMeterProvider or the gauges bind to a no-op meter.\n eventLoopInstrumentation = registerEventLoopInstrumentation();\n}\n\nlet sdk: NodeSDK | undefined;\n\nif (TRACING_ENABLED) {\n logger.info(`Initializing OpenTelemetry tracing for ${SERVICE_NAME}`);\n if (TEMPO_ENDPOINT) logger.info(` Tempo endpoint: ${TEMPO_ENDPOINT}`);\n if (SENTRY_DSN) logger.info(` Sentry span forwarding: enabled`);\n logger.info(` Tenant: ${TENANT_ID}`);\n\n const resource = resourceFromAttributes({\n [ATTR_SERVICE_NAME]: SERVICE_NAME,\n [ATTR_SERVICE_VERSION]: SERVICE_VERSION,\n \"tenant.id\": TENANT_ID,\n \"deployment.environment\": DEPLOY_ENV,\n });\n\n const spanProcessors: SpanProcessor[] = [];\n if (TEMPO_ENDPOINT) {\n spanProcessors.push(\n new BatchSpanProcessor(new OTLPTraceExporter({ url: TEMPO_ENDPOINT })),\n );\n }\n if (SENTRY_TRACING_TO_SENTRY) {\n // Fan the same OTel spans into Sentry — same trace IDs as Tempo, so\n // Sentry transactions cross-link to traces in Grafana. Skipped in\n // errors-only mode (SENTRY_TRACING_ENABLED=false): spans still flow to\n // Tempo via the BatchSpanProcessor above, just not to Sentry.\n spanProcessors.push(new SentrySpanProcessor());\n }\n\n sdk = new NodeSDK({\n resource,\n spanProcessors,\n textMapPropagator: SENTRY_TRACING_TO_SENTRY\n ? new SentryPropagator()\n : undefined,\n instrumentations: [\n new HttpInstrumentation({\n ignoreIncomingRequestHook: (req) =>\n req.url === \"/health\" || req.url === \"/ready\",\n requireParentforIncomingSpans: false,\n requireParentforOutgoingSpans: false,\n requestHook: (span, request) => {\n span.setAttribute(\n \"http.route\",\n redactWebhookPath((request as IncomingMessage).url),\n );\n },\n responseHook: (span, response) => {\n if (response.statusCode) {\n span.setAttribute(\"http.status_code\", response.statusCode);\n }\n },\n }),\n new ExpressInstrumentation({\n requestHook: (span, info) => {\n if (info.route) span.setAttribute(\"http.route\", info.route);\n },\n }),\n // HttpInstrumentation only patches node:http/https; outbound global\n // fetch() goes through undici and is otherwise untraced — e.g. the\n // per-request Renown credential check in AuthService.verifyCredentialExists.\n new UndiciInstrumentation(),\n new GraphQLInstrumentation({ mergeItems: true, allowValues: true }),\n // requireParentSpan: only trace DB queries that run inside a request\n // (HTTP/GraphQL) span. Parentless queries — the management\n // switchboard's background polling loops (vetra-cloud-observability\n // reconcile @60s + clint pull-worker @15s, each writing\n // environment_pods / clint_runtime_endpoints) — would otherwise each\n // become a standalone root transaction. That volume scales O(tenant\n // count) and was ~70% of all Sentry transactions before this change.\n // Dropping it at the instrumentation layer (no span created at all) is\n // cheaper and cleaner than sampling it away downstream.\n new PgInstrumentation({\n enhancedDatabaseReporting: true,\n requireParentSpan: true,\n }),\n ],\n });\n sdk.start();\n if (\n SENTRY_TRACING_TO_SENTRY &&\n typeof Sentry.validateOpenTelemetrySetup === \"function\"\n ) {\n Sentry.validateOpenTelemetrySetup();\n }\n logger.info(\"OpenTelemetry tracing initialized\");\n}\n\nasync function shutdown() {\n // Detach the observable callback before the provider drains, so the final\n // export does not fire a gauge callback against a shutting-down meter.\n eventLoopInstrumentation?.stop();\n await Promise.race([\n Promise.all([\n meterProvider?.shutdown().catch(() => undefined),\n sdk?.shutdown().catch(() => undefined),\n ]),\n new Promise<void>((resolve) => setTimeout(resolve, 5_000)),\n ]);\n}\n\nprocess.on(\"SIGINT\", () => {\n void shutdown().finally(() => process.exit(0));\n});\nprocess.on(\"SIGTERM\", () => {\n void shutdown().finally(() => process.exit(0));\n});\n\nexport { meterProvider, sdk };\n","import dotenv from \"dotenv\";\ndotenv.config();\n\nimport { getConfig } from \"@powerhousedao/config/node\";\nimport { parseForcePgVersion } from \"./pglite-version.js\";\nimport type {\n SwitchboardDriveDocumentType,\n SwitchboardDriveInput,\n} from \"./types.js\";\nconst phConfig = getConfig();\nconst { switchboard } = phConfig;\ninterface Config {\n database: {\n url: string;\n };\n port: number;\n mcp: boolean;\n migratePglite: boolean;\n forcePgVersion: 16 | 17 | null;\n drive: SwitchboardDriveInput;\n}\n\nfunction parseDriveType(\n raw: string | undefined,\n): SwitchboardDriveDocumentType | undefined {\n if (!raw) return undefined;\n if (raw === \"powerhouse/document-drive\" || raw === \"powerhouse/reactor-drive\")\n return raw;\n throw new Error(\n `Invalid PH_DEFAULT_DRIVE_TYPE: ${raw}. Expected \"powerhouse/document-drive\" or \"powerhouse/reactor-drive\".`,\n );\n}\n\nexport const config: Config = {\n database: {\n // url: process.env.PH_SWITCHBOARD_DATABASE_URL ?? switchboard?.database?.url ?? \"dev.db\",\n url:\n process.env.PH_SWITCHBOARD_DATABASE_URL ??\n switchboard?.database?.url ??\n \"dev.db\",\n },\n port:\n process.env.PH_SWITCHBOARD_PORT &&\n !isNaN(Number(process.env.PH_SWITCHBOARD_PORT))\n ? Number(process.env.PH_SWITCHBOARD_PORT)\n : (switchboard?.port ?? 4001),\n mcp: true,\n migratePglite: process.env.PH_MIGRATE_PGLITE === \"true\",\n forcePgVersion: parseForcePgVersion(process.env.PH_FORCE_PG_VERSION),\n drive: {\n id: \"powerhouse\",\n slug: \"powerhouse\",\n documentType: parseDriveType(process.env.PH_DEFAULT_DRIVE_TYPE),\n global: {\n name: \"Powerhouse\",\n icon: \"https://ipfs.io/ipfs/QmcaTDBYn8X2psGaXe7iQ6qd8q6oqHLgxvMX9yXf7f9uP7\",\n },\n local: {\n availableOffline: true,\n listeners: [],\n sharingType: \"public\",\n triggers: [],\n },\n },\n};\n","import type { PyroscopeConfig } from \"@pyroscope/nodejs\";\n\nexport async function initProfilerFromEnv(env: typeof process.env) {\n const {\n PYROSCOPE_SERVER_ADDRESS: serverAddress,\n PYROSCOPE_APPLICATION_NAME: appName,\n PYROSCOPE_USER: basicAuthUser,\n PYROSCOPE_PASSWORD: basicAuthPassword,\n PYROSCOPE_WALL_ENABLED: wallEnabled,\n PYROSCOPE_HEAP_ENABLED: heapEnabled,\n } = env;\n\n const options: PyroscopeConfig = {\n serverAddress,\n appName,\n basicAuthUser,\n basicAuthPassword,\n // Wall profiling captures wall-clock time (includes async I/O waits)\n // This shows GraphQL resolvers even when waiting for database\n wall: {\n samplingDurationMs: 10000, // 10 second sampling windows\n samplingIntervalMicros: 10000, // 10ms sampling interval (100 samples/sec)\n collectCpuTime: true, // Also collect CPU time alongside wall time\n },\n // Heap profiling for memory allocation tracking\n heap: {\n samplingIntervalBytes: 512 * 1024, // Sample every 512KB allocated\n stackDepth: 64, // Capture deeper stacks for better context\n },\n };\n return initProfiler(options, {\n wallEnabled: wallEnabled !== \"false\",\n heapEnabled: heapEnabled === \"true\",\n });\n}\n\ninterface ProfilerFlags {\n wallEnabled?: boolean;\n heapEnabled?: boolean;\n}\n\nexport async function initProfiler(\n options?: PyroscopeConfig,\n flags: ProfilerFlags = { wallEnabled: true, heapEnabled: false },\n) {\n console.log(\"Initializing Pyroscope profiler at:\", options?.serverAddress);\n console.log(\" Wall profiling:\", flags.wallEnabled ? \"enabled\" : \"disabled\");\n console.log(\" Heap profiling:\", flags.heapEnabled ? \"enabled\" : \"disabled\");\n\n const { default: Pyroscope } = await import(\"@pyroscope/nodejs\");\n Pyroscope.init(options);\n\n // Start wall profiling (captures async I/O time - shows resolvers)\n if (flags.wallEnabled) {\n Pyroscope.startWallProfiling();\n }\n\n // Start CPU profiling (captures CPU-bound work)\n Pyroscope.startCpuProfiling();\n\n // Optionally start heap profiling (memory allocations)\n if (flags.heapEnabled) {\n Pyroscope.startHeapProfiling();\n }\n}\n","#!/usr/bin/env node\n// Observability MUST load before any module that imports http/express/pg/graphql\n// so OpenTelemetry's require-time hooks can patch them. It also owns Sentry\n// init and the SIGINT/SIGTERM flush.\nimport \"./observability.mjs\";\n\nimport * as Sentry from \"@sentry/node\";\nimport { childLogger } from \"document-model\";\nimport { config } from \"./config.js\";\nimport { initProfilerFromEnv } from \"./profiler.js\";\nimport { startSwitchboard } from \"./server.mjs\";\n\nconst logger = childLogger([\"switchboard\"]);\n\nfunction ensureNodeVersion(minVersion = \"24\") {\n const version = process.versions.node;\n if (!version) {\n return;\n }\n\n if (version < minVersion) {\n console.error(\n `Node version ${minVersion} or higher is required. Current version: ${version}`,\n );\n process.exit(1);\n }\n}\n// Ensure minimum Node.js version\nensureNodeVersion(\"24\");\n\n// Each subgraph registers its own SIGINT/SIGTERM listeners, and the count\n// scales with dynamically-loaded document models beyond the default cap of 10.\nprocess.setMaxListeners(0);\n\nif (process.env.PYROSCOPE_SERVER_ADDRESS) {\n try {\n await initProfilerFromEnv(process.env);\n } catch (e) {\n Sentry.captureException(e);\n logger.error(\"Error starting profiler: @error\", e);\n }\n}\n\nconst cliMigratePglite = process.argv.slice(2).includes(\"--migrate-pglite\");\n\n// A boot failure leaves no serving process behind, so it must exit non-zero:\n// logging and continuing hides a dead switchboard from the supervisor, which\n// would otherwise never restart it.\ntry {\n await startSwitchboard({\n ...config,\n migratePglite: cliMigratePglite || config.migratePglite,\n forcePgVersion: config.forcePgVersion ?? undefined,\n });\n} catch (e) {\n Sentry.captureException(e);\n logger.error(\"Switchboard failed to start: @error\", e);\n await Sentry.flush(2000).catch(() => undefined);\n process.exit(1);\n}\n"],"names":["logger","logger"],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;AAQA,MAAMA,WAAS,YAAY,CAAC,eAAe,UAAU,CAAC;AAEtD,SAAgB,2BAA2B,KAIb;CAC5B,MAAM,WAAW,IAAI;AACrB,KAAI,CAAC,SAAU,QAAO,KAAA;CAEtB,MAAM,SAAS,SAAS,IAAI,+BAA+B,IAAI,GAAG;CAClE,MAAM,uBACJ,OAAO,SAAS,OAAO,IAAI,SAAS,IAAI,SAAS;CAEnD,MAAM,OAAO,SAAS,QAAQ,OAAO,GAAG;CACxC,MAAM,cAAc,KAAK,SAAS,cAAc,GAC5C,OACA,GAAG,KAAK;AAEZ,UAAO,KAAK,mDAAmD,WAAW;CAC1E,MAAM,gBAAgB,IAAI,cAAc;EACtC,UAAU,uBAAuB,EAC/B,gBAAgB,IAAI,qBAAqB,eAC1C,CAAC;EACF,SAAS,CACP,IAAI,8BAA8B;GAChC,UAAU,IAAI,mBAAmB,EAC/B,KAAK,aACN,CAAC;GACF;GACA,qBAAqB,KAAK,IAAI,uBAAuB,KAAK,EAAE;GAC7D,CAAC,CACH;EACF,CAAC;AACF,UAAO,KAAK,qCAAqC,qBAAqB,KAAK;AAC3E,QAAO;;;;AC3CT,MAAM,kBAAkB;;;;;;;;AASxB,SAAgB,kBAAkB,KAAiC;AACjE,KAAI,CAAC,IAAK,QAAO;CAGjB,MAAM,OAAO,IAAI,MAAM,IAAI,CAAC,MAAM;CAClC,MAAM,KAAK,KAAK,QAAQ,gBAAgB;AACxC,KAAI,OAAO,GAAI,QAAO;AAEtB,KAAI,KAAK,WAAW,KAAK,GAAwB,QAAO;AACxD,QAAO,GAAG,KAAK,MAAM,GAAG,GAAG,GAAG,gBAAgB;;;;ACuBhD,MAAMC,WAAS,YAAY,CAAC,eAAe,gBAAgB,CAAC;AAE5D,MAAM,eAAe,QAAQ,IAAI,qBAAqB;AACtD,MAAM,kBAAkB,QAAQ,IAAI,uBAAuB;AAC3D,MAAM,YAAY,QAAQ,IAAI,aAAa;AAC3C,MAAM,aAAa,QAAQ,IAAI,YAAY;AAE3C,MAAM,iBAAiB,QAAQ,IAAI;AACnC,MAAM,aAAa,QAAQ,IAAI;AAS/B,MAAM,2BACJ,QAAQ,WAAW,IAAI,QAAQ,IAAI,2BAA2B;AAEhE,MAAM,oBACJ,QAAQ,IAAI,mBAAmB,UAC/B,QAAQ,IAAI,aAAa;AAC3B,MAAM,wBAAwB,QAAQ,eAAe,IAAI,QAAQ,WAAW;AAC5E,MAAM,kBAAkB,qBAAqB;AAE7C,IAAI,qBAAqB,CAAC,sBACxB,UAAO,KACL,6RAID;AAiBH,MAAM,4BAA4B,WAChC,QAAQ,IAAI,6BAA6B,MAC1C;AAED,IAAI,YAAY;AACd,UAAO,KAAK,qCAAqC,QAAQ,IAAI,WAAW;AACxE,QAAO,KAAK;EACV,KAAK;EACL,aAAa,QAAQ,IAAI;EAGzB,SACE,QAAQ,IAAI,mBACX,QAAQ,IAAI,sBACT,IAAI,QAAQ,IAAI,wBAChB,KAAA;EAGN,kBAAkB,2BAA2B,4BAA4B;EAMzE,wBAAwB;EACzB,CAAC;;AAGJ,MAAM,gBAA2C,2BAA2B;CAC1E,6BAA6B,QAAQ,IAAI;CACzC,6BAA6B,QAAQ,IAAI;CACzC,mBAAmB,QAAQ,IAAI;CAChC,CAAC;AACF,IAAI;AACJ,IAAI,eAAe;AAGjB,SAAQ,uBAAuB,cAAc;AAO7C,4BAA2B,kCAAkC;;AAG/D,IAAI;AAEJ,IAAI,iBAAiB;AACnB,UAAO,KAAK,0CAA0C,eAAe;AACrE,KAAI,eAAgB,UAAO,KAAK,qBAAqB,iBAAiB;AACtE,KAAI,WAAY,UAAO,KAAK,oCAAoC;AAChE,UAAO,KAAK,aAAa,YAAY;CAErC,MAAM,WAAW,uBAAuB;GACrC,oBAAoB;GACpB,uBAAuB;EACxB,aAAa;EACb,0BAA0B;EAC3B,CAAC;CAEF,MAAM,iBAAkC,EAAE;AAC1C,KAAI,eACF,gBAAe,KACb,IAAI,mBAAmB,IAAI,kBAAkB,EAAE,KAAK,gBAAgB,CAAC,CAAC,CACvE;AAEH,KAAI,yBAKF,gBAAe,KAAK,IAAI,qBAAqB,CAAC;AAGhD,OAAM,IAAI,QAAQ;EAChB;EACA;EACA,mBAAmB,2BACf,IAAI,kBAAkB,GACtB,KAAA;EACJ,kBAAkB;GAChB,IAAI,oBAAoB;IACtB,4BAA4B,QAC1B,IAAI,QAAQ,aAAa,IAAI,QAAQ;IACvC,+BAA+B;IAC/B,+BAA+B;IAC/B,cAAc,MAAM,YAAY;AAC9B,UAAK,aACH,cACA,kBAAmB,QAA4B,IAAI,CACpD;;IAEH,eAAe,MAAM,aAAa;AAChC,SAAI,SAAS,WACX,MAAK,aAAa,oBAAoB,SAAS,WAAW;;IAG/D,CAAC;GACF,IAAI,uBAAuB,EACzB,cAAc,MAAM,SAAS;AAC3B,QAAI,KAAK,MAAO,MAAK,aAAa,cAAc,KAAK,MAAM;MAE9D,CAAC;GAIF,IAAI,uBAAuB;GAC3B,IAAI,uBAAuB;IAAE,YAAY;IAAM,aAAa;IAAM,CAAC;GAUnE,IAAI,kBAAkB;IACpB,2BAA2B;IAC3B,mBAAmB;IACpB,CAAC;GACH;EACF,CAAC;AACF,KAAI,OAAO;AACX,KACE,4BACA,OAAO,OAAO,+BAA+B,WAE7C,QAAO,4BAA4B;AAErC,UAAO,KAAK,oCAAoC;;AAGlD,eAAe,WAAW;AAGxB,2BAA0B,MAAM;AAChC,OAAM,QAAQ,KAAK,CACjB,QAAQ,IAAI,CACV,eAAe,UAAU,CAAC,YAAY,KAAA,EAAU,EAChD,KAAK,UAAU,CAAC,YAAY,KAAA,EAAU,CACvC,CAAC,EACF,IAAI,SAAe,YAAY,WAAW,SAAS,IAAM,CAAC,CAC3D,CAAC;;AAGJ,QAAQ,GAAG,gBAAgB;AACpB,WAAU,CAAC,cAAc,QAAQ,KAAK,EAAE,CAAC;EAC9C;AACF,QAAQ,GAAG,iBAAiB;AACrB,WAAU,CAAC,cAAc,QAAQ,KAAK,EAAE,CAAC;EAC9C;;;AClPF,OAAO,QAAQ;AASf,MAAM,EAAE,gBADS,WAAW;AAa5B,SAAS,eACP,KAC0C;AAC1C,KAAI,CAAC,IAAK,QAAO,KAAA;AACjB,KAAI,QAAQ,+BAA+B,QAAQ,2BACjD,QAAO;AACT,OAAM,IAAI,MACR,kCAAkC,IAAI,uEACvC;;AAGH,MAAa,SAAiB;CAC5B,UAAU,EAER,KACE,QAAQ,IAAI,+BACZ,aAAa,UAAU,OACvB,UACH;CACD,MACE,QAAQ,IAAI,uBACZ,CAAC,MAAM,OAAO,QAAQ,IAAI,oBAAoB,CAAC,GAC3C,OAAO,QAAQ,IAAI,oBAAoB,GACtC,aAAa,QAAQ;CAC5B,KAAK;CACL,eAAe,QAAQ,IAAI,sBAAsB;CACjD,gBAAgB,oBAAoB,QAAQ,IAAI,oBAAoB;CACpE,OAAO;EACL,IAAI;EACJ,MAAM;EACN,cAAc,eAAe,QAAQ,IAAI,sBAAsB;EAC/D,QAAQ;GACN,MAAM;GACN,MAAM;GACP;EACD,OAAO;GACL,kBAAkB;GAClB,WAAW,EAAE;GACb,aAAa;GACb,UAAU,EAAE;GACb;EACF;CACF;;;AC9DD,eAAsB,oBAAoB,KAAyB;CACjE,MAAM,EACJ,0BAA0B,eAC1B,4BAA4B,SAC5B,gBAAgB,eAChB,oBAAoB,mBACpB,wBAAwB,aACxB,wBAAwB,gBACtB;AAoBJ,QAAO,aAlB0B;EAC/B;EACA;EACA;EACA;EAGA,MAAM;GACJ,oBAAoB;GACpB,wBAAwB;GACxB,gBAAgB;GACjB;EAED,MAAM;GACJ,uBAAuB,MAAM;GAC7B,YAAY;GACb;EACF,EAC4B;EAC3B,aAAa,gBAAgB;EAC7B,aAAa,gBAAgB;EAC9B,CAAC;;AAQJ,eAAsB,aACpB,SACA,QAAuB;CAAE,aAAa;CAAM,aAAa;CAAO,EAChE;AACA,SAAQ,IAAI,uCAAuC,SAAS,cAAc;AAC1E,SAAQ,IAAI,qBAAqB,MAAM,cAAc,YAAY,WAAW;AAC5E,SAAQ,IAAI,qBAAqB,MAAM,cAAc,YAAY,WAAW;CAE5E,MAAM,EAAE,SAAS,cAAc,MAAM,OAAO;AAC5C,WAAU,KAAK,QAAQ;AAGvB,KAAI,MAAM,YACR,WAAU,oBAAoB;AAIhC,WAAU,mBAAmB;AAG7B,KAAI,MAAM,YACR,WAAU,oBAAoB;;;;AClDlC,MAAM,SAAS,YAAY,CAAC,cAAc,CAAC;AAE3C,SAAS,kBAAkB,aAAa,MAAM;CAC5C,MAAM,UAAU,QAAQ,SAAS;AACjC,KAAI,CAAC,QACH;AAGF,KAAI,UAAU,YAAY;AACxB,UAAQ,MACN,gBAAgB,WAAW,2CAA2C,UACvE;AACD,UAAQ,KAAK,EAAE;;;AAInB,kBAAkB,KAAK;AAIvB,QAAQ,gBAAgB,EAAE;AAE1B,IAAI,QAAQ,IAAI,yBACd,KAAI;AACF,OAAM,oBAAoB,QAAQ,IAAI;SAC/B,GAAG;AACV,QAAO,iBAAiB,EAAE;AAC1B,QAAO,MAAM,mCAAmC,EAAE;;AAItD,MAAM,mBAAmB,QAAQ,KAAK,MAAM,EAAE,CAAC,SAAS,mBAAmB;AAK3E,IAAI;AACF,OAAM,iBAAiB;EACrB,GAAG;EACH,eAAe,oBAAoB,OAAO;EAC1C,gBAAgB,OAAO,kBAAkB,KAAA;EAC1C,CAAC;SACK,GAAG;AACV,QAAO,iBAAiB,EAAE;AAC1B,QAAO,MAAM,uCAAuC,EAAE;AACtD,OAAM,OAAO,MAAM,IAAK,CAAC,YAAY,KAAA,EAAU;AAC/C,SAAQ,KAAK,EAAE","debug_id":"c6c10af1-f4cc-56a5-abf6-fa35cf0de956"}
1
+ {"version":3,"file":"index.mjs","sources":["../src/metrics.ts","../src/redact-webhook-path.ts","../src/observability.mts","../src/config.ts","../src/profiler.ts","../src/index.mts"],"sourcesContent":["import { OTLPMetricExporter } from \"@opentelemetry/exporter-metrics-otlp-http\";\nimport { resourceFromAttributes } from \"@opentelemetry/resources\";\nimport {\n MeterProvider,\n PeriodicExportingMetricReader,\n} from \"@opentelemetry/sdk-metrics\";\nimport { childLogger } from \"document-model\";\n\nconst logger = childLogger([\"switchboard\", \"metrics\"]);\n\nexport function createMeterProviderFromEnv(env: {\n OTEL_EXPORTER_OTLP_ENDPOINT?: string;\n OTEL_METRIC_EXPORT_INTERVAL?: string;\n OTEL_SERVICE_NAME?: string;\n}): MeterProvider | undefined {\n const endpoint = env.OTEL_EXPORTER_OTLP_ENDPOINT;\n if (!endpoint) return undefined;\n\n const parsed = parseInt(env.OTEL_METRIC_EXPORT_INTERVAL ?? \"\", 10);\n const exportIntervalMillis =\n Number.isFinite(parsed) && parsed > 0 ? parsed : 5_000;\n\n const base = endpoint.replace(/\\/$/, \"\");\n const exporterUrl = base.endsWith(\"/v1/metrics\")\n ? base\n : `${base}/v1/metrics`;\n\n logger.info(`Initializing OpenTelemetry metrics exporter at: ${endpoint}`);\n const meterProvider = new MeterProvider({\n resource: resourceFromAttributes({\n \"service.name\": env.OTEL_SERVICE_NAME ?? \"switchboard\",\n }),\n readers: [\n new PeriodicExportingMetricReader({\n exporter: new OTLPMetricExporter({\n url: exporterUrl,\n }),\n exportIntervalMillis,\n exportTimeoutMillis: Math.max(exportIntervalMillis - 250, 1),\n }),\n ],\n });\n logger.info(`Metrics export enabled (interval: ${exportIntervalMillis}ms)`);\n return meterProvider;\n}\n","const WEBHOOK_SEGMENT = \"/webhooks/\";\n\n/**\n * Strips a webhook's delivery token out of a request path.\n *\n * The token is the endpoint's whole credential and it sits in the path, so it\n * must not reach a log line or a span attribute that ships off-box. The family\n * mounts under the host's base path, so the segment can sit at any depth.\n */\nexport function redactWebhookPath(url: string | undefined): string {\n if (!url) return \"\";\n // Dropped with the token: `http.route` is meant to be a template, and a\n // query string is another place a sender can put a credential.\n const path = url.split(\"?\")[0] ?? url;\n const at = path.indexOf(WEBHOOK_SEGMENT);\n if (at === -1) return url;\n // Nothing after the segment is nothing to redact.\n if (path.length === at + WEBHOOK_SEGMENT.length) return url;\n return `${path.slice(0, at)}${WEBHOOK_SEGMENT}[redacted]`;\n}\n","// Single observability bootstrap: Sentry + OpenTelemetry (tracing + metrics).\n//\n// MUST be imported as the very first module in apps/switchboard/src/index.mts.\n// OpenTelemetry instrumentations register require-time hooks at module load,\n// so http/express/pg/graphql must not be imported (transitively) before this\n// file runs.\n//\n// Replaces three legacy bootstrap sites:\n// - apps/switchboard/src/server.mts top-level Sentry.init\n// - apps/switchboard/src/metrics.ts standalone MeterProvider (still exported\n// here via createMeterProviderFromEnv so its tests keep passing)\n// - packages/reactor-api/src/tracing.ts side-effect NodeSDK\nimport { metrics } from \"@opentelemetry/api\";\nimport { OTLPTraceExporter } from \"@opentelemetry/exporter-trace-otlp-http\";\nimport { ExpressInstrumentation } from \"@opentelemetry/instrumentation-express\";\nimport { GraphQLInstrumentation } from \"@opentelemetry/instrumentation-graphql\";\nimport { HttpInstrumentation } from \"@opentelemetry/instrumentation-http\";\nimport { PgInstrumentation } from \"@opentelemetry/instrumentation-pg\";\nimport { UndiciInstrumentation } from \"@opentelemetry/instrumentation-undici\";\nimport { resourceFromAttributes } from \"@opentelemetry/resources\";\nimport type { MeterProvider } from \"@opentelemetry/sdk-metrics\";\nimport { NodeSDK } from \"@opentelemetry/sdk-node\";\nimport {\n BatchSpanProcessor,\n type SpanProcessor,\n} from \"@opentelemetry/sdk-trace-base\";\nimport {\n ATTR_SERVICE_NAME,\n ATTR_SERVICE_VERSION,\n} from \"@opentelemetry/semantic-conventions\";\nimport {\n registerEventLoopInstrumentation,\n type EventLoopInstrumentation,\n} from \"@powerhousedao/opentelemetry-instrumentation-reactor\";\nimport * as Sentry from \"@sentry/node\";\nimport { SentryPropagator, SentrySpanProcessor } from \"@sentry/opentelemetry\";\nimport { childLogger } from \"document-model\";\nimport type { IncomingMessage } from \"node:http\";\nimport { createMeterProviderFromEnv } from \"./metrics.js\";\nimport { redactWebhookPath } from \"./redact-webhook-path.js\";\n\nconst logger = childLogger([\"switchboard\", \"observability\"]);\n\nconst SERVICE_NAME = process.env.OTEL_SERVICE_NAME || \"switchboard\";\nconst SERVICE_VERSION = process.env.npm_package_version || \"unknown\";\nconst TENANT_ID = process.env.TENANT_ID || \"default\";\nconst DEPLOY_ENV = process.env.NODE_ENV || \"development\";\n\nconst TEMPO_ENDPOINT = process.env.TEMPO_ENDPOINT;\nconst SENTRY_DSN = process.env.SENTRY_DSN;\n\n// Whether to forward TRANSACTIONS (spans) to Sentry. Errors are always sent\n// when SENTRY_DSN is set — this flag only gates APM/tracing into Sentry.\n// Default on (back-compat). Set SENTRY_TRACING_ENABLED=false for an\n// \"errors-only\" deployment: errors still go to Sentry, traces still go to\n// Tempo, but no transactions hit Sentry (no Kafka/ClickHouse/nodestore\n// cost). This is the recommended mode for tenant workloads at scale —\n// Sentry's value there is error grouping; traces live in Tempo.\nconst SENTRY_TRACING_TO_SENTRY =\n Boolean(SENTRY_DSN) && process.env.SENTRY_TRACING_ENABLED !== \"false\";\n\nconst TRACING_REQUESTED =\n process.env.ENABLE_TRACING === \"true\" ||\n process.env.NODE_ENV === \"production\";\nconst HAS_TRACE_DESTINATION = Boolean(TEMPO_ENDPOINT) || Boolean(SENTRY_DSN);\nconst TRACING_ENABLED = TRACING_REQUESTED && HAS_TRACE_DESTINATION;\n\nif (TRACING_REQUESTED && !HAS_TRACE_DESTINATION) {\n logger.warn(\n \"Tracing was requested (NODE_ENV=production or ENABLE_TRACING=true) but \" +\n \"no destination is configured — instrumentation will not run. Set \" +\n \"TEMPO_ENDPOINT (e.g. http://tempo.monitoring.svc.cluster.local:4318/v1/traces) \" +\n \"to export OTLP spans, and/or SENTRY_DSN to forward spans to Sentry.\",\n );\n}\n\n// APM sampling for the Sentry-SDK-managed path (i.e. when TRACING is OFF and\n// @sentry/node runs its own bundled OTel — see skipOpenTelemetrySetup below).\n//\n// IMPORTANT: when TRACING_ENABLED, this value does NOT govern span volume.\n// Our NodeSDK (below) owns the pipeline and is constructed with no explicit\n// `sampler`, so @opentelemetry/sdk-node falls back to buildSamplerFromEnv()\n// and the REAL head-sampling knob is the standard OTEL_TRACES_SAMPLER /\n// OTEL_TRACES_SAMPLER_ARG env (set per-deploy in the k8s chart). That head\n// decision gates spans before any processor runs, so it bounds BOTH the\n// SentrySpanProcessor (→ Sentry transactions) and the Tempo OTLP export.\n// (Wiring @sentry/opentelemetry's SentrySampler here would let this rate\n// drive both backends and make DSC/sample_rand propagation spec-correct, but\n// that only matters for Sentry server-side dynamic sampling — a SaaS feature\n// our self-hosted install doesn't run — so it's intentionally deferred.)\nconst SENTRY_TRACES_SAMPLE_RATE = parseFloat(\n process.env.SENTRY_TRACES_SAMPLE_RATE ?? \"0.1\",\n);\n\nif (SENTRY_DSN) {\n logger.info(\"Initialized Sentry with env: @env\", process.env.SENTRY_ENV);\n Sentry.init({\n dsn: SENTRY_DSN,\n environment: process.env.SENTRY_ENV,\n // Match the version tag uploaded by release-branch.yml so source maps\n // resolve. Populated by the CI (WORKSPACE_VERSION) or npm at runtime.\n release:\n process.env.SENTRY_RELEASE ||\n (process.env.npm_package_version\n ? `v${process.env.npm_package_version}`\n : undefined),\n // 0 in errors-only mode so even @sentry/node's bundled auto-OTel path\n // (used when TRACING_ENABLED is false) produces no transactions.\n tracesSampleRate: SENTRY_TRACING_TO_SENTRY ? SENTRY_TRACES_SAMPLE_RATE : 0,\n // When tracing is on, our NodeSDK below owns the OTel globals and Sentry\n // receives spans via SentrySpanProcessor. Skipping Sentry's bundled OTel\n // setup avoids two TracerProviders fighting over setGlobalTracerProvider.\n // When tracing is off, leave the flag unset so @sentry/node's default\n // auto-OTel still records HTTP transactions for the APM dashboard.\n skipOpenTelemetrySetup: TRACING_ENABLED,\n });\n}\n\nconst meterProvider: MeterProvider | undefined = createMeterProviderFromEnv({\n OTEL_EXPORTER_OTLP_ENDPOINT: process.env.OTEL_EXPORTER_OTLP_ENDPOINT,\n OTEL_METRIC_EXPORT_INTERVAL: process.env.OTEL_METRIC_EXPORT_INTERVAL,\n OTEL_SERVICE_NAME: process.env.OTEL_SERVICE_NAME,\n});\nlet eventLoopInstrumentation: EventLoopInstrumentation | undefined;\nif (meterProvider) {\n // One-way door: must register before any code calls metrics.getMeter() —\n // most notably ReactorInstrumentation inside the reactor module.\n metrics.setGlobalMeterProvider(meterProvider);\n // Host event-loop and CPU gauges (reactor.host.eventloop.*,\n // reactor.host.cpu.utilization). The bench sweep identified the host event\n // loop, not the executor, as the binding constraint on write throughput\n // (bench/test/integration/BASELINE.md, Run 10) — these are the production\n // counterpart of the metrics that finding was made from. Must come after\n // setGlobalMeterProvider or the gauges bind to a no-op meter.\n eventLoopInstrumentation = registerEventLoopInstrumentation();\n}\n\nlet sdk: NodeSDK | undefined;\n\nif (TRACING_ENABLED) {\n logger.info(`Initializing OpenTelemetry tracing for ${SERVICE_NAME}`);\n if (TEMPO_ENDPOINT) logger.info(` Tempo endpoint: ${TEMPO_ENDPOINT}`);\n if (SENTRY_DSN) logger.info(` Sentry span forwarding: enabled`);\n logger.info(` Tenant: ${TENANT_ID}`);\n\n const resource = resourceFromAttributes({\n [ATTR_SERVICE_NAME]: SERVICE_NAME,\n [ATTR_SERVICE_VERSION]: SERVICE_VERSION,\n \"tenant.id\": TENANT_ID,\n \"deployment.environment\": DEPLOY_ENV,\n });\n\n const spanProcessors: SpanProcessor[] = [];\n if (TEMPO_ENDPOINT) {\n spanProcessors.push(\n new BatchSpanProcessor(new OTLPTraceExporter({ url: TEMPO_ENDPOINT })),\n );\n }\n if (SENTRY_TRACING_TO_SENTRY) {\n // Fan the same OTel spans into Sentry — same trace IDs as Tempo, so\n // Sentry transactions cross-link to traces in Grafana. Skipped in\n // errors-only mode (SENTRY_TRACING_ENABLED=false): spans still flow to\n // Tempo via the BatchSpanProcessor above, just not to Sentry.\n spanProcessors.push(new SentrySpanProcessor());\n }\n\n sdk = new NodeSDK({\n resource,\n spanProcessors,\n textMapPropagator: SENTRY_TRACING_TO_SENTRY\n ? new SentryPropagator()\n : undefined,\n instrumentations: [\n new HttpInstrumentation({\n ignoreIncomingRequestHook: (req) =>\n req.url === \"/health\" || req.url === \"/ready\",\n requireParentforIncomingSpans: false,\n requireParentforOutgoingSpans: false,\n requestHook: (span, request) => {\n span.setAttribute(\n \"http.route\",\n redactWebhookPath((request as IncomingMessage).url),\n );\n },\n responseHook: (span, response) => {\n if (response.statusCode) {\n span.setAttribute(\"http.status_code\", response.statusCode);\n }\n },\n }),\n new ExpressInstrumentation({\n requestHook: (span, info) => {\n if (info.route) span.setAttribute(\"http.route\", info.route);\n },\n }),\n // HttpInstrumentation only patches node:http/https; outbound global\n // fetch() goes through undici and is otherwise untraced — e.g. the\n // per-request Renown credential check in AuthService.verifyCredentialExists.\n new UndiciInstrumentation(),\n new GraphQLInstrumentation({ mergeItems: true, allowValues: true }),\n // requireParentSpan: only trace DB queries that run inside a request\n // (HTTP/GraphQL) span. Parentless queries — the management\n // switchboard's background polling loops (vetra-cloud-observability\n // reconcile @60s + clint pull-worker @15s, each writing\n // environment_pods / clint_runtime_endpoints) — would otherwise each\n // become a standalone root transaction. That volume scales O(tenant\n // count) and was ~70% of all Sentry transactions before this change.\n // Dropping it at the instrumentation layer (no span created at all) is\n // cheaper and cleaner than sampling it away downstream.\n new PgInstrumentation({\n enhancedDatabaseReporting: true,\n requireParentSpan: true,\n }),\n ],\n });\n sdk.start();\n if (\n SENTRY_TRACING_TO_SENTRY &&\n typeof Sentry.validateOpenTelemetrySetup === \"function\"\n ) {\n Sentry.validateOpenTelemetrySetup();\n }\n logger.info(\"OpenTelemetry tracing initialized\");\n}\n\nasync function shutdown() {\n // Detach the observable callback before the provider drains, so the final\n // export does not fire a gauge callback against a shutting-down meter.\n eventLoopInstrumentation?.stop();\n await Promise.race([\n Promise.all([\n meterProvider?.shutdown().catch(() => undefined),\n sdk?.shutdown().catch(() => undefined),\n ]),\n new Promise<void>((resolve) => setTimeout(resolve, 5_000)),\n ]);\n}\n\nprocess.on(\"SIGINT\", () => {\n void shutdown().finally(() => process.exit(0));\n});\nprocess.on(\"SIGTERM\", () => {\n void shutdown().finally(() => process.exit(0));\n});\n\nexport { meterProvider, sdk };\n","import dotenv from \"dotenv\";\ndotenv.config();\n\nimport { getConfig } from \"@powerhousedao/config/node\";\nimport { parseForcePgVersion } from \"./pglite-version.js\";\nimport type {\n SwitchboardDriveDocumentType,\n SwitchboardDriveInput,\n} from \"./types.js\";\nconst phConfig = getConfig();\nconst { switchboard } = phConfig;\ninterface Config {\n database: {\n url: string;\n };\n port: number;\n mcp: boolean;\n migratePglite: boolean;\n forcePgVersion: 16 | 17 | null;\n drive: SwitchboardDriveInput;\n}\n\nfunction parseDriveType(\n raw: string | undefined,\n): SwitchboardDriveDocumentType | undefined {\n if (!raw) return undefined;\n if (raw === \"powerhouse/document-drive\" || raw === \"powerhouse/reactor-drive\")\n return raw;\n throw new Error(\n `Invalid PH_DEFAULT_DRIVE_TYPE: ${raw}. Expected \"powerhouse/document-drive\" or \"powerhouse/reactor-drive\".`,\n );\n}\n\nexport const config: Config = {\n database: {\n // url: process.env.PH_SWITCHBOARD_DATABASE_URL ?? switchboard?.database?.url ?? \"dev.db\",\n url:\n process.env.PH_SWITCHBOARD_DATABASE_URL ??\n switchboard?.database?.url ??\n \"dev.db\",\n },\n port:\n process.env.PH_SWITCHBOARD_PORT &&\n !isNaN(Number(process.env.PH_SWITCHBOARD_PORT))\n ? Number(process.env.PH_SWITCHBOARD_PORT)\n : (switchboard?.port ?? 4001),\n mcp: true,\n migratePglite: process.env.PH_MIGRATE_PGLITE === \"true\",\n forcePgVersion: parseForcePgVersion(process.env.PH_FORCE_PG_VERSION),\n drive: {\n id: \"powerhouse\",\n slug: \"powerhouse\",\n documentType: parseDriveType(process.env.PH_DEFAULT_DRIVE_TYPE),\n global: {\n name: \"Powerhouse\",\n icon: \"https://ipfs.io/ipfs/QmcaTDBYn8X2psGaXe7iQ6qd8q6oqHLgxvMX9yXf7f9uP7\",\n },\n local: {\n availableOffline: true,\n listeners: [],\n sharingType: \"public\",\n triggers: [],\n },\n },\n};\n","import type { PyroscopeConfig } from \"@pyroscope/nodejs\";\n\nexport async function initProfilerFromEnv(env: typeof process.env) {\n const {\n PYROSCOPE_SERVER_ADDRESS: serverAddress,\n PYROSCOPE_APPLICATION_NAME: appName,\n PYROSCOPE_USER: basicAuthUser,\n PYROSCOPE_PASSWORD: basicAuthPassword,\n PYROSCOPE_WALL_ENABLED: wallEnabled,\n PYROSCOPE_HEAP_ENABLED: heapEnabled,\n } = env;\n\n const options: PyroscopeConfig = {\n serverAddress,\n appName,\n basicAuthUser,\n basicAuthPassword,\n // Wall profiling captures wall-clock time (includes async I/O waits)\n // This shows GraphQL resolvers even when waiting for database\n wall: {\n samplingDurationMs: 10000, // 10 second sampling windows\n samplingIntervalMicros: 10000, // 10ms sampling interval (100 samples/sec)\n collectCpuTime: true, // Also collect CPU time alongside wall time\n },\n // Heap profiling for memory allocation tracking\n heap: {\n samplingIntervalBytes: 512 * 1024, // Sample every 512KB allocated\n stackDepth: 64, // Capture deeper stacks for better context\n },\n };\n return initProfiler(options, {\n wallEnabled: wallEnabled !== \"false\",\n heapEnabled: heapEnabled === \"true\",\n });\n}\n\ninterface ProfilerFlags {\n wallEnabled?: boolean;\n heapEnabled?: boolean;\n}\n\nexport async function initProfiler(\n options?: PyroscopeConfig,\n flags: ProfilerFlags = { wallEnabled: true, heapEnabled: false },\n) {\n console.log(\"Initializing Pyroscope profiler at:\", options?.serverAddress);\n console.log(\" Wall profiling:\", flags.wallEnabled ? \"enabled\" : \"disabled\");\n console.log(\" Heap profiling:\", flags.heapEnabled ? \"enabled\" : \"disabled\");\n\n const { default: Pyroscope } = await import(\"@pyroscope/nodejs\");\n Pyroscope.init(options);\n\n // Start wall profiling (captures async I/O time - shows resolvers)\n if (flags.wallEnabled) {\n Pyroscope.startWallProfiling();\n }\n\n // Start CPU profiling (captures CPU-bound work)\n Pyroscope.startCpuProfiling();\n\n // Optionally start heap profiling (memory allocations)\n if (flags.heapEnabled) {\n Pyroscope.startHeapProfiling();\n }\n}\n","#!/usr/bin/env node\n// Observability MUST load before any module that imports http/express/pg/graphql\n// so OpenTelemetry's require-time hooks can patch them. It also owns Sentry\n// init and the SIGINT/SIGTERM flush.\nimport \"./observability.mjs\";\n\nimport * as Sentry from \"@sentry/node\";\nimport { childLogger } from \"document-model\";\nimport { config } from \"./config.js\";\nimport { initProfilerFromEnv } from \"./profiler.js\";\nimport { startSwitchboard } from \"./server.mjs\";\n\nconst logger = childLogger([\"switchboard\"]);\n\nfunction ensureNodeVersion(minVersion = \"24\") {\n const version = process.versions.node;\n if (!version) {\n return;\n }\n\n if (version < minVersion) {\n console.error(\n `Node version ${minVersion} or higher is required. Current version: ${version}`,\n );\n process.exit(1);\n }\n}\n// Ensure minimum Node.js version\nensureNodeVersion(\"24\");\n\n// Each subgraph registers its own SIGINT/SIGTERM listeners, and the count\n// scales with dynamically-loaded document models beyond the default cap of 10.\nprocess.setMaxListeners(0);\n\nif (process.env.PYROSCOPE_SERVER_ADDRESS) {\n try {\n await initProfilerFromEnv(process.env);\n } catch (e) {\n Sentry.captureException(e);\n logger.error(\"Error starting profiler: @error\", e);\n }\n}\n\nconst cliMigratePglite = process.argv.slice(2).includes(\"--migrate-pglite\");\n\n// A boot failure leaves no serving process behind, so it must exit non-zero:\n// logging and continuing hides a dead switchboard from the supervisor, which\n// would otherwise never restart it.\ntry {\n await startSwitchboard({\n ...config,\n migratePglite: cliMigratePglite || config.migratePglite,\n forcePgVersion: config.forcePgVersion ?? undefined,\n fatalErrorShutdown: true,\n });\n} catch (e) {\n Sentry.captureException(e);\n logger.error(\"Switchboard failed to start: @error\", e);\n await Sentry.flush(2000).catch(() => undefined);\n process.exit(1);\n}\n"],"names":["logger","logger"],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;AAQA,MAAMA,WAAS,YAAY,CAAC,eAAe,UAAU,CAAC;AAEtD,SAAgB,2BAA2B,KAIb;CAC5B,MAAM,WAAW,IAAI;AACrB,KAAI,CAAC,SAAU,QAAO,KAAA;CAEtB,MAAM,SAAS,SAAS,IAAI,+BAA+B,IAAI,GAAG;CAClE,MAAM,uBACJ,OAAO,SAAS,OAAO,IAAI,SAAS,IAAI,SAAS;CAEnD,MAAM,OAAO,SAAS,QAAQ,OAAO,GAAG;CACxC,MAAM,cAAc,KAAK,SAAS,cAAc,GAC5C,OACA,GAAG,KAAK;AAEZ,UAAO,KAAK,mDAAmD,WAAW;CAC1E,MAAM,gBAAgB,IAAI,cAAc;EACtC,UAAU,uBAAuB,EAC/B,gBAAgB,IAAI,qBAAqB,eAC1C,CAAC;EACF,SAAS,CACP,IAAI,8BAA8B;GAChC,UAAU,IAAI,mBAAmB,EAC/B,KAAK,aACN,CAAC;GACF;GACA,qBAAqB,KAAK,IAAI,uBAAuB,KAAK,EAAE;GAC7D,CAAC,CACH;EACF,CAAC;AACF,UAAO,KAAK,qCAAqC,qBAAqB,KAAK;AAC3E,QAAO;;;;AC3CT,MAAM,kBAAkB;;;;;;;;AASxB,SAAgB,kBAAkB,KAAiC;AACjE,KAAI,CAAC,IAAK,QAAO;CAGjB,MAAM,OAAO,IAAI,MAAM,IAAI,CAAC,MAAM;CAClC,MAAM,KAAK,KAAK,QAAQ,gBAAgB;AACxC,KAAI,OAAO,GAAI,QAAO;AAEtB,KAAI,KAAK,WAAW,KAAK,GAAwB,QAAO;AACxD,QAAO,GAAG,KAAK,MAAM,GAAG,GAAG,GAAG,gBAAgB;;;;ACuBhD,MAAMC,WAAS,YAAY,CAAC,eAAe,gBAAgB,CAAC;AAE5D,MAAM,eAAe,QAAQ,IAAI,qBAAqB;AACtD,MAAM,kBAAkB,QAAQ,IAAI,uBAAuB;AAC3D,MAAM,YAAY,QAAQ,IAAI,aAAa;AAC3C,MAAM,aAAa,QAAQ,IAAI,YAAY;AAE3C,MAAM,iBAAiB,QAAQ,IAAI;AACnC,MAAM,aAAa,QAAQ,IAAI;AAS/B,MAAM,2BACJ,QAAQ,WAAW,IAAI,QAAQ,IAAI,2BAA2B;AAEhE,MAAM,oBACJ,QAAQ,IAAI,mBAAmB,UAC/B,QAAQ,IAAI,aAAa;AAC3B,MAAM,wBAAwB,QAAQ,eAAe,IAAI,QAAQ,WAAW;AAC5E,MAAM,kBAAkB,qBAAqB;AAE7C,IAAI,qBAAqB,CAAC,sBACxB,UAAO,KACL,6RAID;AAiBH,MAAM,4BAA4B,WAChC,QAAQ,IAAI,6BAA6B,MAC1C;AAED,IAAI,YAAY;AACd,UAAO,KAAK,qCAAqC,QAAQ,IAAI,WAAW;AACxE,QAAO,KAAK;EACV,KAAK;EACL,aAAa,QAAQ,IAAI;EAGzB,SACE,QAAQ,IAAI,mBACX,QAAQ,IAAI,sBACT,IAAI,QAAQ,IAAI,wBAChB,KAAA;EAGN,kBAAkB,2BAA2B,4BAA4B;EAMzE,wBAAwB;EACzB,CAAC;;AAGJ,MAAM,gBAA2C,2BAA2B;CAC1E,6BAA6B,QAAQ,IAAI;CACzC,6BAA6B,QAAQ,IAAI;CACzC,mBAAmB,QAAQ,IAAI;CAChC,CAAC;AACF,IAAI;AACJ,IAAI,eAAe;AAGjB,SAAQ,uBAAuB,cAAc;AAO7C,4BAA2B,kCAAkC;;AAG/D,IAAI;AAEJ,IAAI,iBAAiB;AACnB,UAAO,KAAK,0CAA0C,eAAe;AACrE,KAAI,eAAgB,UAAO,KAAK,qBAAqB,iBAAiB;AACtE,KAAI,WAAY,UAAO,KAAK,oCAAoC;AAChE,UAAO,KAAK,aAAa,YAAY;CAErC,MAAM,WAAW,uBAAuB;GACrC,oBAAoB;GACpB,uBAAuB;EACxB,aAAa;EACb,0BAA0B;EAC3B,CAAC;CAEF,MAAM,iBAAkC,EAAE;AAC1C,KAAI,eACF,gBAAe,KACb,IAAI,mBAAmB,IAAI,kBAAkB,EAAE,KAAK,gBAAgB,CAAC,CAAC,CACvE;AAEH,KAAI,yBAKF,gBAAe,KAAK,IAAI,qBAAqB,CAAC;AAGhD,OAAM,IAAI,QAAQ;EAChB;EACA;EACA,mBAAmB,2BACf,IAAI,kBAAkB,GACtB,KAAA;EACJ,kBAAkB;GAChB,IAAI,oBAAoB;IACtB,4BAA4B,QAC1B,IAAI,QAAQ,aAAa,IAAI,QAAQ;IACvC,+BAA+B;IAC/B,+BAA+B;IAC/B,cAAc,MAAM,YAAY;AAC9B,UAAK,aACH,cACA,kBAAmB,QAA4B,IAAI,CACpD;;IAEH,eAAe,MAAM,aAAa;AAChC,SAAI,SAAS,WACX,MAAK,aAAa,oBAAoB,SAAS,WAAW;;IAG/D,CAAC;GACF,IAAI,uBAAuB,EACzB,cAAc,MAAM,SAAS;AAC3B,QAAI,KAAK,MAAO,MAAK,aAAa,cAAc,KAAK,MAAM;MAE9D,CAAC;GAIF,IAAI,uBAAuB;GAC3B,IAAI,uBAAuB;IAAE,YAAY;IAAM,aAAa;IAAM,CAAC;GAUnE,IAAI,kBAAkB;IACpB,2BAA2B;IAC3B,mBAAmB;IACpB,CAAC;GACH;EACF,CAAC;AACF,KAAI,OAAO;AACX,KACE,4BACA,OAAO,OAAO,+BAA+B,WAE7C,QAAO,4BAA4B;AAErC,UAAO,KAAK,oCAAoC;;AAGlD,eAAe,WAAW;AAGxB,2BAA0B,MAAM;AAChC,OAAM,QAAQ,KAAK,CACjB,QAAQ,IAAI,CACV,eAAe,UAAU,CAAC,YAAY,KAAA,EAAU,EAChD,KAAK,UAAU,CAAC,YAAY,KAAA,EAAU,CACvC,CAAC,EACF,IAAI,SAAe,YAAY,WAAW,SAAS,IAAM,CAAC,CAC3D,CAAC;;AAGJ,QAAQ,GAAG,gBAAgB;AACpB,WAAU,CAAC,cAAc,QAAQ,KAAK,EAAE,CAAC;EAC9C;AACF,QAAQ,GAAG,iBAAiB;AACrB,WAAU,CAAC,cAAc,QAAQ,KAAK,EAAE,CAAC;EAC9C;;;AClPF,OAAO,QAAQ;AASf,MAAM,EAAE,gBADS,WAAW;AAa5B,SAAS,eACP,KAC0C;AAC1C,KAAI,CAAC,IAAK,QAAO,KAAA;AACjB,KAAI,QAAQ,+BAA+B,QAAQ,2BACjD,QAAO;AACT,OAAM,IAAI,MACR,kCAAkC,IAAI,uEACvC;;AAGH,MAAa,SAAiB;CAC5B,UAAU,EAER,KACE,QAAQ,IAAI,+BACZ,aAAa,UAAU,OACvB,UACH;CACD,MACE,QAAQ,IAAI,uBACZ,CAAC,MAAM,OAAO,QAAQ,IAAI,oBAAoB,CAAC,GAC3C,OAAO,QAAQ,IAAI,oBAAoB,GACtC,aAAa,QAAQ;CAC5B,KAAK;CACL,eAAe,QAAQ,IAAI,sBAAsB;CACjD,gBAAgB,oBAAoB,QAAQ,IAAI,oBAAoB;CACpE,OAAO;EACL,IAAI;EACJ,MAAM;EACN,cAAc,eAAe,QAAQ,IAAI,sBAAsB;EAC/D,QAAQ;GACN,MAAM;GACN,MAAM;GACP;EACD,OAAO;GACL,kBAAkB;GAClB,WAAW,EAAE;GACb,aAAa;GACb,UAAU,EAAE;GACb;EACF;CACF;;;AC9DD,eAAsB,oBAAoB,KAAyB;CACjE,MAAM,EACJ,0BAA0B,eAC1B,4BAA4B,SAC5B,gBAAgB,eAChB,oBAAoB,mBACpB,wBAAwB,aACxB,wBAAwB,gBACtB;AAoBJ,QAAO,aAlB0B;EAC/B;EACA;EACA;EACA;EAGA,MAAM;GACJ,oBAAoB;GACpB,wBAAwB;GACxB,gBAAgB;GACjB;EAED,MAAM;GACJ,uBAAuB,MAAM;GAC7B,YAAY;GACb;EACF,EAC4B;EAC3B,aAAa,gBAAgB;EAC7B,aAAa,gBAAgB;EAC9B,CAAC;;AAQJ,eAAsB,aACpB,SACA,QAAuB;CAAE,aAAa;CAAM,aAAa;CAAO,EAChE;AACA,SAAQ,IAAI,uCAAuC,SAAS,cAAc;AAC1E,SAAQ,IAAI,qBAAqB,MAAM,cAAc,YAAY,WAAW;AAC5E,SAAQ,IAAI,qBAAqB,MAAM,cAAc,YAAY,WAAW;CAE5E,MAAM,EAAE,SAAS,cAAc,MAAM,OAAO;AAC5C,WAAU,KAAK,QAAQ;AAGvB,KAAI,MAAM,YACR,WAAU,oBAAoB;AAIhC,WAAU,mBAAmB;AAG7B,KAAI,MAAM,YACR,WAAU,oBAAoB;;;;AClDlC,MAAM,SAAS,YAAY,CAAC,cAAc,CAAC;AAE3C,SAAS,kBAAkB,aAAa,MAAM;CAC5C,MAAM,UAAU,QAAQ,SAAS;AACjC,KAAI,CAAC,QACH;AAGF,KAAI,UAAU,YAAY;AACxB,UAAQ,MACN,gBAAgB,WAAW,2CAA2C,UACvE;AACD,UAAQ,KAAK,EAAE;;;AAInB,kBAAkB,KAAK;AAIvB,QAAQ,gBAAgB,EAAE;AAE1B,IAAI,QAAQ,IAAI,yBACd,KAAI;AACF,OAAM,oBAAoB,QAAQ,IAAI;SAC/B,GAAG;AACV,QAAO,iBAAiB,EAAE;AAC1B,QAAO,MAAM,mCAAmC,EAAE;;AAItD,MAAM,mBAAmB,QAAQ,KAAK,MAAM,EAAE,CAAC,SAAS,mBAAmB;AAK3E,IAAI;AACF,OAAM,iBAAiB;EACrB,GAAG;EACH,eAAe,oBAAoB,OAAO;EAC1C,gBAAgB,OAAO,kBAAkB,KAAA;EACzC,oBAAoB;EACrB,CAAC;SACK,GAAG;AACV,QAAO,iBAAiB,EAAE;AAC1B,QAAO,MAAM,uCAAuC,EAAE;AACtD,OAAM,OAAO,MAAM,IAAK,CAAC,YAAY,KAAA,EAAU;AAC/C,SAAQ,KAAK,EAAE","debug_id":"2ce93cee-94b7-54c2-9688-4127efd686af"}