opencode-effect-enforcer 0.2.0

package/skills/effect-observability/SKILL.md

@@ -0,0 +1,719 @@
+---
+name: effect-observability
+description: Implement structured logging, distributed tracing, and metrics in Effect applications. Use this skill when configuring loggers, adding spans/tracing, collecting metrics, or setting up OTLP/Prometheus export.
+---
+
+You are an Effect TypeScript expert specializing in observability — structured logging, distributed tracing, and metrics collection.
+
+## Effect Source Reference
+
+The Effect v4 source is available at `~/.local/share/opencode/repos/github.com/Effect-TS/effect@main/`.
+Browse and read files there directly to look up APIs, types, and implementations.
+
+Reference this for:
+
+- `Logger` module: `packages/effect/src/Logger.ts`
+- `Tracer` module: `packages/effect/src/Tracer.ts`
+- `Metric` module: `packages/effect/src/Metric.ts`
+- OTLP export: `packages/effect/src/unstable/observability/`
+- Observability examples: `ai-docs/src/08_observability/`
+
+---
+
+## 1. Structured Logging
+
+### Log Functions
+
+Effect provides log functions at every level. Each is variadic and accepts one or more message values:
+
+```ts
+import { Effect } from 'effect';
+
+const program = Effect.gen(function* () {
+	yield* Effect.log('general log message');
+	yield* Effect.logDebug('debug-level detail');
+	yield* Effect.logInfo('informational message');
+	yield* Effect.logWarning('something concerning');
+	yield* Effect.logError('something failed');
+});
+```
+
+Pass additional message values after the first. These extra values become part of the log message/body — they are **not** automatically indexed as queryable annotations:
+
+```ts
+yield* Effect.log('User action', { userId: 123, action: 'login' });
+yield* Effect.logInfo('Request processed', { duration: 150, statusCode: 200 });
+```
+
+For **queryable dimensions** (fields you want to filter or group on in your logging/tracing backend), prefer `Effect.annotateLogs` or span annotations (`Effect.annotateCurrentSpan` / `Effect.annotateSpans`) rather than passing objects as message values.
+
+### Log Annotations
+
+Attach key-value metadata to all log lines within a scope:
+
+```ts
+const checkout = Effect.gen(function* () {
+	yield* Effect.logInfo('validating cart');
+	yield* Effect.logWarning('inventory low for one line item');
+	yield* Effect.logError('payment provider timeout');
+}).pipe(
+	Effect.annotateLogs({
+		service: 'checkout-api',
+		route: 'POST /checkout'
+	})
+);
+```
+
+Single annotation shorthand:
+
+```ts
+Effect.annotateLogs('requestId', 'req-abc-123');
+```
+
+### Log Spans
+
+Add duration metadata to log lines — each log will include `label=<elapsed>ms`:
+
+```ts
+const withTiming = myEffect.pipe(Effect.withLogSpan('checkout'));
+// Log output includes: checkout=42ms
+```
+
+---
+
+## 2. Logger Configuration
+
+### Built-in Loggers
+
+Effect v4 provides several logger implementations:
+
+| Logger                     | Output                            | Use Case                       |
+| -------------------------- | --------------------------------- | ------------------------------ |
+| `Logger.defaultLogger`     | Default runtime format            | General use                    |
+| `Logger.consolePretty()`   | Colorized, human-readable         | Development                    |
+| `Logger.consoleJson`       | Single-line JSON to console       | Production / log aggregation   |
+| `Logger.consoleLogFmt`     | logfmt key=value to console       | Production / structured search |
+| `Logger.consoleStructured` | JS object to console              | Development debugging          |
+| `Logger.formatSimple`      | String (no console output)        | Composition / piping           |
+| `Logger.formatJson`        | JSON string (no console output)   | Composition / piping           |
+| `Logger.formatLogFmt`      | logfmt string (no console output) | Composition / piping           |
+| `Logger.formatStructured`  | Structured JS object              | Composition / piping           |
+| `Logger.tracerLogger`      | Emits logs as tracer span events  | Included by default            |
+
+### Installing Loggers via `Logger.layer`
+
+`Logger.layer` **replaces** the current loggers by default:
+
+```ts
+import { Effect, Logger } from 'effect';
+
+// JSON logging for production
+const JsonLoggerLayer = Logger.layer([Logger.consoleJson]);
+
+// Pretty logging for development
+const PrettyLoggerLayer = Logger.layer([Logger.consolePretty()]);
+
+// Multiple loggers — both receive every log entry
+const MultiLoggerLayer = Logger.layer([
+	Logger.consoleJson,
+	Logger.consolePretty()
+]);
+
+// Merge with existing loggers instead of replacing
+const AdditionalLoggerLayer = Logger.layer([Logger.consoleJson], {
+	mergeWithExisting: true
+});
+```
+
+### Log Level Filtering
+
+Control the minimum log level via `References.MinimumLogLevel`:
+
+```ts
+import { Layer, References } from 'effect';
+
+// Only emit Warn and above (skip Debug, Info)
+const WarnAndAbove = Layer.succeed(References.MinimumLogLevel, 'Warn');
+
+// Combine with logger
+const ProductionLoggerLayer = Logger.layer([Logger.consoleJson]).pipe(
+	Layer.provideMerge(WarnAndAbove)
+);
+```
+
+Valid levels: `"All"`, `"Trace"`, `"Debug"`, `"Info"`, `"Warn"`, `"Error"`, `"Fatal"`, `"None"`.
+
+### Custom Loggers
+
+Create loggers with `Logger.make`:
+
+```ts
+import { Logger } from 'effect';
+
+const customLogger = Logger.make((options) => {
+	console.log(`[${options.logLevel}] ${options.message}`);
+});
+
+// options fields: message, logLevel, cause, fiber, date
+```
+
+### Batched Logging
+
+Aggregate log entries over a time window before flushing:
+
+```ts
+import { Effect, Logger } from 'effect';
+
+const batchedLogger = Logger.batched(Logger.formatStructured, {
+	window: '1 second',
+	flush: Effect.fn(function* (batch) {
+		// Send batch to external service
+		console.log(`Flushing ${batch.length} log entries`);
+	})
+});
+
+// batchedLogger is an Effect — use it inside Logger.layer
+const BatchedLoggerLayer = Logger.layer([batchedLogger]);
+```
+
+### File Logging
+
+Write logs directly to a file (requires `FileSystem` from `@effect/platform`):
+
+```ts
+import { NodeFileSystem } from '@effect/platform-node';
+import { Layer, Logger } from 'effect';
+
+const FileLoggerLayer = Logger.layer([
+	Logger.toFile(Logger.formatSimple, 'app.log')
+]).pipe(Layer.provide(NodeFileSystem.layer));
+```
+
+Pipe syntax with options:
+
+```ts
+const fileLogger = Logger.formatJson.pipe(
+	Logger.toFile('/var/log/myapp.log', {
+		flag: 'a',
+		batchWindow: '5 seconds'
+	})
+);
+```
+
+### Environment-based Logger Selection
+
+```ts
+import { Config, Effect, Layer, Logger } from 'effect';
+
+const LoggerLayer = Layer.unwrap(
+	Effect.gen(function* () {
+		const env = yield* Config.string('NODE_ENV').pipe(
+			Config.withDefault('development')
+		);
+		if (env === 'production') {
+			return Logger.layer([Logger.consoleJson]);
+		}
+		return Logger.layer([Logger.consolePretty()]);
+	})
+);
+```
+
+---
+
+## 3. Spans and Tracing
+
+### `Effect.withSpan`
+
+Create a tracing span around any effect:
+
+```ts
+const processOrder = Effect.gen(function* () {
+	yield* chargeCard(orderId);
+	yield* persistOrder(orderId);
+}).pipe(Effect.withSpan('processOrder'));
+```
+
+Spans nest automatically — child effects that also use `withSpan` become child spans.
+
+### `Effect.fn` with Auto-Spans
+
+When you pass a string name to `Effect.fn`, it automatically wraps the function body in a span:
+
+```ts
+import { Effect } from 'effect';
+
+const processCheckout = Effect.fn('Checkout.processCheckout')(function* (
+	orderId: string
+) {
+	yield* Effect.logInfo('starting checkout', { orderId });
+	yield* chargeCard(orderId).pipe(Effect.withSpan('checkout.charge-card'));
+	yield* persistOrder(orderId).pipe(
+		Effect.withSpan('checkout.persist-order')
+	);
+	yield* Effect.logInfo('checkout completed', { orderId });
+});
+// Creates a span named "Checkout.processCheckout" wrapping the entire body
+```
+
+### Span Attributes
+
+Annotate the current span with key-value attributes:
+
+```ts
+// Annotate the current span from inside the effect
+yield* Effect.annotateCurrentSpan('order.id', orderId);
+yield* Effect.annotateCurrentSpan('order.total', 99.95);
+
+// Annotate from outside using pipe
+const withAttributes = myEffect.pipe(
+	Effect.annotateSpans({
+		'checkout.order_id': orderId,
+		'checkout.provider': 'acme-pay'
+	})
+);
+```
+
+### Layer Spans
+
+Attach spans to layer construction:
+
+```ts
+const MyLayer = Layer.effectDiscard(setupEffect).pipe(
+	Layer.withSpan('my-layer-setup')
+);
+```
+
+---
+
+## 4. Metrics
+
+Effect provides five metric types. All metrics are concurrent-safe and integrated into the runtime.
+
+### Counter
+
+Tracks cumulative values that only increase:
+
+```ts
+import { Effect, Metric } from 'effect';
+
+const requestCount = Metric.counter('http_requests_total', {
+	description: 'Total number of HTTP requests'
+});
+
+yield* Metric.update(requestCount, 1);
+```
+
+### Gauge
+
+A single numerical value that can go up or down:
+
+```ts
+const activeConnections = Metric.gauge('active_connections', {
+	description: 'Current active connections'
+});
+
+yield* Metric.update(activeConnections, 42);
+```
+
+### Histogram
+
+Records observations in configurable buckets:
+
+```ts
+const responseTime = Metric.histogram('http_response_time_ms', {
+	description: 'HTTP response time in milliseconds',
+	boundaries: Metric.linearBoundaries({ start: 0, width: 50, count: 20 })
+});
+
+yield* Metric.update(responseTime, 127);
+```
+
+### Summary
+
+Calculates quantiles over a sliding time window:
+
+```ts
+const dbQueryTime = Metric.summary('db_query_duration', {
+	maxAge: '5 minutes',
+	maxSize: 1000,
+	quantiles: [0.5, 0.9, 0.95, 0.99]
+});
+
+yield* Metric.update(dbQueryTime, durationMs);
+```
+
+### Frequency
+
+Counts occurrences of discrete string values:
+
+```ts
+const statusCodes = Metric.frequency('http_status_codes', {
+	description: 'HTTP status code distribution'
+});
+
+yield* Metric.update(statusCodes, '200');
+yield* Metric.update(statusCodes, '404');
+```
+
+### Metric Attributes
+
+Tag metrics with key-value attributes for filtering/grouping:
+
+```ts
+const taggedCounter = Metric.withAttributes(requestCount, {
+	endpoint: '/api/users',
+	method: 'GET'
+});
+
+yield* Metric.update(taggedCounter, 1);
+
+// Or inline
+yield*
+	Metric.update(
+		Metric.withAttributes(requestCount, {
+			endpoint: '/api/posts',
+			method: 'POST'
+		}),
+		1
+	);
+```
+
+### Reading Metric Values
+
+```ts
+// Get current value of a single metric
+const value = yield* Metric.value(requestCount);
+
+// Snapshot all metrics
+const snapshots = yield* Metric.snapshot;
+for (const metric of snapshots) {
+	console.log(`${metric.id}: ${JSON.stringify(metric.state)}`);
+}
+```
+
+---
+
+## 5. OTLP Export
+
+Effect v4 includes built-in OTLP exporters in `effect/unstable/observability`. No external OpenTelemetry SDK needed.
+
+### All-in-One: `Otlp.layerJson`
+
+The simplest setup — exports traces, logs, and metrics to a single OTLP endpoint:
+
+```ts
+import { Layer } from 'effect';
+import { FetchHttpClient } from 'effect/unstable/http';
+import { Otlp } from 'effect/unstable/observability';
+
+const ObservabilityLayer = Otlp.layerJson({
+	baseUrl: 'http://localhost:4318',
+	resource: {
+		serviceName: 'my-api',
+		serviceVersion: '1.0.0',
+		attributes: {
+			'deployment.environment': 'staging'
+		}
+	}
+}).pipe(Layer.provide(FetchHttpClient.layer));
+```
+
+This creates layers for all three signals with standard OTLP paths (`/v1/traces`, `/v1/logs`, `/v1/metrics`).
+
+Variants:
+
+- `Otlp.layerJson` — JSON serialization (simplest, no extra deps)
+- `Otlp.layerProtobuf` — Protobuf serialization (more efficient)
+- `Otlp.layer` — requires you to provide `OtlpSerialization` separately
+
+### Individual OTLP Exporters
+
+For fine-grained control, configure each exporter independently:
+
+```ts
+import { Layer } from 'effect';
+import { FetchHttpClient } from 'effect/unstable/http';
+import {
+	OtlpLogger,
+	OtlpMetrics,
+	OtlpSerialization,
+	OtlpTracer
+} from 'effect/unstable/observability';
+
+const OtlpTracingLayer = OtlpTracer.layer({
+	url: 'http://localhost:4318/v1/traces',
+	resource: {
+		serviceName: 'checkout-api',
+		serviceVersion: '1.0.0',
+		attributes: { 'deployment.environment': 'staging' }
+	}
+});
+
+const OtlpLoggingLayer = OtlpLogger.layer({
+	url: 'http://localhost:4318/v1/logs',
+	resource: {
+		serviceName: 'checkout-api',
+		serviceVersion: '1.0.0'
+	}
+});
+
+const OtlpMetricsLayer = OtlpMetrics.layer({
+	url: 'http://localhost:4318/v1/metrics',
+	resource: {
+		serviceName: 'checkout-api',
+		serviceVersion: '1.0.0'
+	},
+	temporality: 'delta' // or "cumulative" (default)
+});
+
+const ObservabilityLayer = Layer.mergeAll(
+	OtlpTracingLayer,
+	OtlpLoggingLayer,
+	OtlpMetricsLayer
+).pipe(
+	Layer.provide(OtlpSerialization.layerJson),
+	Layer.provide(FetchHttpClient.layer)
+);
+```
+
+### Manual OTLP Flush
+
+Each OTLP signal layer now outputs the shared `OtlpExporter.Flusher` service. Use it before a controlled handoff or shutdown when buffered telemetry must be exported immediately:
+
+```ts
+import { Effect } from 'effect';
+import { OtlpExporter } from 'effect/unstable/observability';
+
+const flushTelemetry = Effect.gen(function* () {
+	const flusher = yield* OtlpExporter.Flusher;
+	yield* flusher.flush.pipe(Effect.timeoutOption('5 seconds'));
+});
+```
+
+All signal layers in the same memoized layer graph share one flusher registry, so one call drains logs, traces, and metrics concurrently. `flush` cannot fail and has no built-in timeout; it waits only for exports it starts, not an export already in flight.
+
+### OTLP Layer Options
+
+Common options for the individual OTLP exporters (`OtlpLogger.layer`, `OtlpTracer.layer`, `OtlpMetrics.layer`):
+
+| Option                    | Default          | Description                              |
+| ------------------------- | ---------------- | ---------------------------------------- |
+| `url`                     | required         | OTLP endpoint URL                        |
+| `resource.serviceName`    | —                | Service name in exported telemetry       |
+| `resource.serviceVersion` | —                | Service version                          |
+| `resource.attributes`     | —                | Additional resource attributes           |
+| `headers`                 | —                | HTTP headers for auth etc.               |
+| `exportInterval`          | signal-specific  | How often to flush batches (see below)   |
+| `maxBatchSize`            | signal-specific  | Max items per export batch (see below)   |
+| `shutdownTimeout`         | `3 seconds`      | Timeout for final flush on shutdown      |
+
+Batch/flush defaults differ by signal:
+
+| Exporter            | `exportInterval` | `maxBatchSize`                 | `shutdownTimeout` |
+| ------------------- | ---------------- | ------------------------------ | ----------------- |
+| `OtlpLogger.layer`  | `1 second`       | `1000`                         | `3 seconds`       |
+| `OtlpTracer.layer`  | `5 seconds`      | `1000`                         | `3 seconds`       |
+| `OtlpMetrics.layer` | `10 seconds`     | disabled (pull-style snapshot) | `3 seconds`       |
+
+The all-in-one `Otlp.layerJson` / `Otlp.layerProtobuf` use **signal-specific** interval option names instead of a single `exportInterval`: `loggerExportInterval`, `metricsExportInterval`, and `tracerExportInterval` (plus a shared `maxBatchSize`, `shutdownTimeout`, and `metricsTemporality`).
+
+`OtlpMetrics.layer` additionally accepts:
+
+- `temporality`: `"cumulative"` (default) or `"delta"` — determines how metric values relate to their time interval
+
+`OtlpLogger.layer` additionally accepts:
+
+- `mergeWithExisting`: `true` (default) — merge with existing loggers instead of replacing
+- `excludeLogSpans`: omit log span annotations from exported logs
+
+### Wiring Into Your App
+
+Provide the observability layer at the outermost level so all spans and logs are captured:
+
+```ts
+import { NodeRuntime } from '@effect/platform-node';
+import { Layer } from 'effect';
+
+const Main = AppLayer.pipe(Layer.provide(ObservabilityLayer));
+
+Layer.launch(Main).pipe(NodeRuntime.runMain);
+```
+
+---
+
+## 6. Prometheus Metrics
+
+Export Effect metrics in Prometheus exposition format:
+
+```ts
+import { Effect, Metric } from 'effect';
+import * as PrometheusMetrics from 'effect/unstable/observability/PrometheusMetrics';
+
+const program = Effect.gen(function* () {
+	const counter = Metric.counter('http_requests_total', {
+		description: 'Total HTTP requests'
+	});
+	yield* Metric.update(counter, 42);
+
+	// Format metrics as Prometheus text
+	const output = yield* PrometheusMetrics.format();
+	// # HELP http_requests_total Total HTTP requests
+	// # TYPE http_requests_total counter
+	// http_requests_total 42
+
+	// With prefix
+	const prefixed = yield* PrometheusMetrics.format({ prefix: 'myapp' });
+	// myapp_http_requests_total 42
+});
+```
+
+### Prometheus HTTP Endpoint
+
+Automatically register a `/metrics` endpoint on your HTTP router:
+
+```ts
+import * as PrometheusMetrics from 'effect/unstable/observability/PrometheusMetrics';
+
+// Default: GET /metrics
+const PrometheusLayer = PrometheusMetrics.layerHttp();
+
+// Custom path and prefix
+const CustomPrometheusLayer = PrometheusMetrics.layerHttp({
+	path: '/prometheus/metrics',
+	prefix: 'myapp'
+});
+```
+
+`layerHttp` requires `HttpRouter.HttpRouter` in the context — it adds a route to your existing router.
+
+---
+
+## 7. Testing Observability
+
+### Testing Logged Output
+
+Use `TestConsole` from `@effect/vitest` to capture and assert on console output:
+
+```ts
+import { Effect } from 'effect';
+import { it } from '@effect/vitest';
+
+it.effect('logs checkout flow', () =>
+	Effect.gen(function* () {
+		yield* myLoggingEffect;
+		// TestConsole captures log output for assertion
+	})
+);
+```
+
+### Testing Metrics
+
+Read metric values directly:
+
+```ts
+it.effect('increments request counter', () =>
+	Effect.gen(function* () {
+		const counter = Metric.counter('test_requests');
+		yield* Metric.update(counter, 5);
+		const state = yield* Metric.value(counter);
+		expect(state.count).toBe(5);
+	})
+);
+```
+
+### Testing Spans
+
+Use a test tracer or assert on span attributes captured via `Effect.withSpan`.
+
+---
+
+## 8. Common Patterns
+
+### Full Observability Stack (Dev + Prod)
+
+```ts
+import { Config, Effect, Layer, Logger, References } from 'effect';
+import { FetchHttpClient } from 'effect/unstable/http';
+import { Otlp } from 'effect/unstable/observability';
+
+const DevObservability = Logger.layer([Logger.consolePretty()]);
+
+const ProdObservability = Layer.mergeAll(
+	Logger.layer([Logger.consoleJson]),
+	Layer.succeed(References.MinimumLogLevel, 'Info'),
+	Otlp.layerJson({
+		baseUrl: 'http://otel-collector:4318',
+		resource: {
+			serviceName: 'my-api',
+			serviceVersion: '1.0.0'
+		}
+	}).pipe(Layer.provide(FetchHttpClient.layer))
+);
+
+const ObservabilityLayer = Layer.unwrap(
+	Effect.gen(function* () {
+		const env = yield* Config.string('NODE_ENV').pipe(
+			Config.withDefault('development')
+		);
+		return env === 'production' ? ProdObservability : DevObservability;
+	})
+);
+```
+
+### Service with Instrumented Methods
+
+```ts
+import { Effect, Layer, Metric, Context } from 'effect';
+
+const requestLatency = Metric.histogram('checkout_latency_ms', {
+	boundaries: Metric.linearBoundaries({ start: 0, width: 25, count: 40 })
+});
+
+class Checkout extends Context.Service<
+	Checkout,
+	{
+		processCheckout(orderId: string): Effect.Effect<void>;
+	}
+>()('app/Checkout') {
+	static readonly layer = Layer.effect(
+		Checkout,
+		Effect.gen(function* () {
+			return Checkout.of({
+				processCheckout: Effect.fn('Checkout.processCheckout')(
+					function* (orderId: string) {
+						yield* Effect.logInfo('starting checkout', { orderId });
+						yield* Effect.annotateCurrentSpan('order.id', orderId);
+
+						yield* chargeCard(orderId).pipe(
+							Effect.withSpan('checkout.charge-card')
+						);
+						yield* persistOrder(orderId).pipe(
+							Effect.withSpan('checkout.persist-order')
+						);
+
+						yield* Effect.logInfo('checkout completed', {
+							orderId
+						});
+					}
+				)
+			});
+		})
+	);
+}
+```
+
+---
+
+## Critical Rules
+
+1. **Use `Logger.layer` to install loggers** — it replaces the default set. Use `{ mergeWithExisting: true }` to add without replacing.
+2. **`Logger.tracerLogger` is included by default** — log messages automatically become span events. If you override loggers, include it explicitly if you want this behavior.
+3. **`Effect.fn("name")` creates auto-spans** — prefer this over manual `Effect.withSpan` for service methods.
+4. **Provide observability layers outermost** — so all application spans and logs are captured for export.
+5. **OTLP exporters require `HttpClient` and `OtlpSerialization`** — `Otlp.layerJson` (and `layerProtobuf`) wires `OtlpSerialization` for you but still requires an `HttpClient`, so provide a client layer such as `FetchHttpClient.layer`. With `Otlp.layer` or the individual exporters, provide both `OtlpSerialization.layerJson` and `FetchHttpClient.layer` manually.
+6. **`References.MinimumLogLevel`** controls filtering — not a logger concern, set it via `Layer.succeed`.
+7. **Metric names should follow conventions** — snake_case with units suffix (e.g., `http_request_duration_ms`).
+8. **`Metric.withAttributes` creates a tagged variant** — it does not mutate the original metric.
+9. **`OtlpMetrics` temporality** — use `"delta"` for backends like Datadog/Dynatrace, `"cumulative"` (default) for Prometheus-style backends.
+10. **All OTLP modules are under `effect/unstable/observability`** — the API may evolve but the patterns are stable.
+11. **Use `OtlpExporter.Flusher` for manual drains** — bound `flusher.flush` with `Effect.timeoutOption` when shutdown latency must be capped.