autotel 5.0.0 → 6.0.0

package/skills/review-otel-patterns/references/processor-pipeline.md

@@ -1,205 +0,0 @@
-# Processor pipeline cookbook
-
-Composable building blocks for the autotel pipeline. Each helper is small enough to reason about in isolation and isolates errors so a single bad processor cannot break the others.
-
-## Primitives
-
-| Helper                       | Type              | Purpose                                                      |
-| ---------------------------- | ----------------- | ------------------------------------------------------------ |
-| `defineConfig(config)`       | identity          | Authoring helper for typed config                            |
-| `composeSpanProcessors([…])` | `SpanProcessor`   | Fan span lifecycle to multiple processors                    |
-| `composePostProcessors([…])` | `PostProcessorFn` | Chain post-processors (each sees the output of the previous) |
-| `composeSubscribers([…])`    | `EdgeSubscriber`  | Fire in-process side effects in order                        |
-
-All from `autotel-edge`.
-
-## Multi-backend export
-
-```typescript
-import { BatchSpanProcessor } from 'autotel/processors';
-import { OTLPHttpJsonExporter } from 'autotel/exporters';
-import { composeSpanProcessors, defineConfig } from 'autotel-edge';
-
-const honeycomb = new BatchSpanProcessor(
-  new OTLPHttpJsonExporter({
-    url: 'https://api.honeycomb.io/v1/traces',
-    headers: { 'x-honeycomb-team': process.env.HONEYCOMB_KEY! },
-  }),
-);
-
-const grafana = new BatchSpanProcessor(
-  new OTLPHttpJsonExporter({
-    url: process.env.GRAFANA_OTLP_URL!,
-    headers: { authorization: `Basic ${process.env.GRAFANA_AUTH!}` },
-  }),
-);
-
-export const config = defineConfig({
-  service: { name: 'checkout' },
-  spanProcessors: composeSpanProcessors([honeycomb, grafana]),
-});
-```
-
-## Tail sampling: keep errors + slow + 10% otherwise
-
-```typescript
-import { TailSamplingProcessor } from 'autotel/processors';
-import { composeSpanProcessors } from 'autotel-edge';
-
-const tail = new TailSamplingProcessor({
-  keep: (trace) => {
-    if (trace.localRootSpan.status?.code === SpanStatusCode.ERROR) return true;
-    if (trace.localRootSpan.duration[0] > 1) return true; // > 1s
-    return Math.random() < 0.1;
-  },
-});
-
-spanProcessors: composeSpanProcessors([new BatchSpanProcessor(otlp), tail]);
-```
-
-## Drop noisy spans before they reach the batcher
-
-```typescript
-import { FilteringSpanProcessor } from 'autotel/processors';
-
-const dropHealth = new FilteringSpanProcessor({
-  exclude: (span) => /^GET \/(healthz|ready)$/.test(span.name),
-});
-
-spanProcessors: composeSpanProcessors([
-  dropHealth,
-  new BatchSpanProcessor(otlp),
-]);
-```
-
-## Bound URL cardinality
-
-```typescript
-import { SpanNameNormalizingProcessor } from 'autotel/processors';
-
-const normalise = new SpanNameNormalizingProcessor({
-  // Replace UUIDs and 24-char hex ids with placeholders
-  replacements: [
-    {
-      match: /[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/g,
-      with: ':id',
-    },
-    { match: /[0-9a-f]{24}/g, with: ':id' },
-  ],
-});
-```
-
-Now `GET /users/123e4567-e89b-12d3-a456-426614174000/orders` becomes `GET /users/:id/orders` in your traces. Fewer unique span names, dramatically faster queries.
-
-## Lift baggage onto every span
-
-```typescript
-import { BaggageSpanProcessor } from 'autotel/processors';
-
-// Anything placed in baggage upstream becomes an attribute on every child span
-const baggage = new BaggageSpanProcessor({ keys: ['tenant', 'feature_flags'] });
-spanProcessors: composeSpanProcessors([baggage, new BatchSpanProcessor(otlp)]);
-```
-
-## Subscribers for in-process side effects
-
-Subscribers run synchronously in the parent context. Ideal for metrics, audit, and cost calculation that you want recorded **before** the span goes to the batcher.
-
-```typescript
-import type { EdgeSubscriber } from 'autotel-edge';
-import { composeSubscribers } from 'autotel-edge';
-
-const metricsSubscriber: EdgeSubscriber = (event) => {
-  if (
-    event.kind === 'span.end' &&
-    event.span.attributes['http.response.status_code'] >= 500
-  ) {
-    metrics.errorCounter.add(1, { route: event.span.name });
-  }
-};
-
-const auditSubscriber: EdgeSubscriber = (event) => {
-  if (event.kind === 'span.end' && event.span.name.startsWith('admin.')) {
-    audit.write({
-      kind: event.span.name,
-      actor: event.span.attributes['user.id'],
-    });
-  }
-};
-
-subscribers: [composeSubscribers([metricsSubscriber, auditSubscriber])];
-```
-
-## Post-processors for last-mile rewrites
-
-Post-processors mutate the array of spans **after** sampling, just before export. Use for redacting stack traces, dropping fields, or annotating with deployment info.
-
-```typescript
-import type { PostProcessorFn } from 'autotel-edge';
-import { composePostProcessors } from 'autotel-edge';
-import { createStringRedactor } from 'autotel';
-
-const redactStacks = createStringRedactor('strict');
-
-const cleanStacks: PostProcessorFn = (spans) =>
-  spans.map((s) => {
-    if (typeof s.attributes['exception.stacktrace'] === 'string') {
-      s.attributes['exception.stacktrace'] = redactStacks(
-        s.attributes['exception.stacktrace'],
-      );
-    }
-    return s;
-  });
-
-const tagDeploy: PostProcessorFn = (spans) =>
-  spans.map((s) => ({
-    ...s,
-    attributes: { ...s.attributes, 'deploy.id': process.env.RELEASE! },
-  }));
-
-postProcessor: composePostProcessors([cleanStacks, tagDeploy]);
-```
-
-## Putting it all together
-
-```typescript
-import {
-  defineConfig,
-  composeSpanProcessors,
-  composeSubscribers,
-  composePostProcessors,
-} from 'autotel-edge';
-
-export const otelConfig = defineConfig({
-  service: { name: 'checkout' },
-  attributeRedactor: 'strict',
-
-  spanProcessors: composeSpanProcessors([
-    dropHealth, // 1. drop spans we never want
-    normaliseUrls, // 2. bound cardinality
-    new BatchSpanProcessor(honeycomb),
-    new BatchSpanProcessor(grafana),
-    tailSampler, // 3. keep errors + slow + 10%
-  ]),
-
-  subscribers: [
-    composeSubscribers([metricsSubscriber, auditSubscriber, aiCostSubscriber]),
-  ],
-
-  postProcessor: composePostProcessors([cleanStacks, tagDeploy]),
-});
-```
-
-## Error isolation
-
-Every compose helper catches errors per item and logs to `console.error` with the helper name. A single bad processor cannot break the others. Important when one of your subscribers is a third-party integration (Datadog, PagerDuty, …) that can rate-limit or 502.
-
-## Choosing between subscribers and post-processors
-
-| You want…                                 | Use                                                        |
-| ----------------------------------------- | ---------------------------------------------------------- |
-| Mutate exported span attributes           | `postProcessor`                                            |
-| Drop spans entirely                       | `FilteringSpanProcessor` (early) or tail sampler           |
-| Update an in-process metric on every span | `subscribers`                                              |
-| Send an audit log to a DB                 | `subscribers` (use `log.fork('audit')` if writes are slow) |
-| Re-emit spans to a second backend         | second `BatchSpanProcessor` in `composeSpanProcessors`     |

package/skills/review-otel-patterns/references/structured-errors.md

@@ -1,102 +0,0 @@
-# Structured errors
-
-`createStructuredError` produces an `Error` carrying enough context to be:
-
-- **Recorded onto the active span** (`exception.type`, `exception.message`, `exception.stacktrace`, `span.status = ERROR`).
-- **Returned to clients safely** (`internal` is stripped by `parseError`).
-- **Self-documenting** (`why` explains the cause, `fix` tells the caller what to do, `link` points at runbook docs).
-
-## Field reference
-
-| Field      | Audience    | Purpose                                                    |
-| ---------- | ----------- | ---------------------------------------------------------- |
-| `message`  | Both        | Short, stable summary                                      |
-| `status`   | Both        | HTTP status (drives client behaviour and span status code) |
-| `why`      | Both        | Human-readable cause (`"Card declined by issuer"`)         |
-| `fix`      | Client      | Remediation hint (`"Use a different payment method"`)      |
-| `link`     | Client      | URL to docs / runbook                                      |
-| `code`     | Both        | Machine-readable code (`"PAYMENT_DECLINED"`)               |
-| `cause`    | Server only | The underlying error                                       |
-| `internal` | Server only | Diagnostic metadata (`{ correlationId, resourceId }`)      |
-| `details`  | Both        | Structured payload (e.g. validation errors per field)      |
-
-## Templates
-
-### Validation (400)
-
-```typescript
-throw createStructuredError({
-  status: 400,
-  code: 'VALIDATION_ERROR',
-  message: 'Invalid request body',
-  why: 'One or more fields failed validation',
-  fix: 'Check the `details` field for per-field errors',
-  details: { email: 'must be a valid email', age: 'must be ≥ 18' },
-});
-```
-
-### Auth (401 / 403)
-
-```typescript
-throw createStructuredError({
-  status: 403,
-  code: 'FORBIDDEN',
-  message: 'Not allowed',
-  why: 'You do not have access to this resource',
-  fix: 'Ask the workspace owner for access',
-  link: 'https://docs.example.com/permissions',
-  internal: { resourceId: 'proj_123', userRole: 'member' },
-});
-```
-
-### Payment (402)
-
-```typescript
-throw createStructuredError({
-  status: 402,
-  code: 'PAYMENT_DECLINED',
-  message: 'Payment declined',
-  why: 'Card declined by issuer — insufficient funds',
-  fix: 'Use a different payment method or contact your bank',
-  link: 'https://docs.example.com/payments/declined',
-  cause: stripeError,
-  internal: { stripeChargeId: 'ch_…', riskScore: stripeError.risk_level },
-});
-```
-
-### Upstream failure (502 / 503 / 504)
-
-```typescript
-throw createStructuredError({
-  status: 502,
-  code: 'UPSTREAM_FAILED',
-  message: 'Inventory service is unavailable',
-  why: 'Could not reach the inventory service',
-  fix: 'Retry in a few minutes',
-  cause: fetchError,
-  internal: { upstream: 'inventory-svc', retryAttempt: 3 },
-});
-```
-
-## At HTTP boundaries
-
-```typescript
-import { parseError } from 'autotel';
-
-app.onError((error, c) => {
-  // span.status is already ERROR with exception fields recorded
-  const parsed = parseError(error);
-  // `internal` and `cause` are stripped here — never leak them to clients
-  return c.json(parsed, parsed.status);
-});
-```
-
-## Anti-patterns
-
-| Anti-pattern                                                 | Fix                                                                                    |
-| ------------------------------------------------------------ | -------------------------------------------------------------------------------------- |
-| `throw new Error('something went wrong')`                    | `createStructuredError({ message, status, why, fix })`                                 |
-| Putting support IDs in `message` (`"Failed for user 42"`)    | Use `internal: { userId: 42 }`                                                         |
-| Returning `details: { error: stack }` to clients             | Stack traces stay in `cause` / span; never serialise them out                          |
-| `console.error(e); throw e`                                  | Just throw: autotel's span will pick up the exception                                  |
-| Two callers throwing different shapes for the same condition | Centralise: `function declined(reason: string) { throw createStructuredError({ … }) }` |

package/skills/review-otel-patterns/references/wide-spans.md

@@ -1,85 +0,0 @@
-# Designing wide spans
-
-A wide span is a single span per logical unit of work (request, job, message, fork) carrying _all_ the fields you'd ever want to filter or group by. autotel lets you build them with `useLogger().set({ … })`. Fields are flattened to OTel attributes with stable dotted keys.
-
-## Anatomy
-
-```typescript
-import { useLogger } from 'autotel';
-
-export const POST = withAutotel(async (request) => {
-  const log = useLogger();
-
-  // Identity
-  log.set({ user: { id: 'usr_123', plan: 'enterprise', role: 'admin' } });
-
-  // Inputs
-  log.set({ cart: { items: 3, total: 14_999, currency: 'USD' } });
-
-  // Decisions / branches
-  log.set({ promo: { applied: 'SUMMER10', discount: 1_500 } });
-
-  // Outputs
-  log.set({
-    payment: { provider: 'stripe', method: 'card', authCode: 'auth_x' },
-  });
-
-  return Response.json({ ok: true });
-});
-```
-
-OTel attributes recorded:
-
-```
-user.id=usr_123
-user.plan=enterprise
-user.role=admin
-cart.items=3
-cart.total=14999
-cart.currency=USD
-promo.applied=SUMMER10
-promo.discount=1500
-payment.provider=stripe
-payment.method=card
-payment.authCode=auth_x
-```
-
-## Rules of thumb
-
-1. **One wide span per logical unit of work.** Many tiny spans hurt query speed; deep call trees can be opt-in (`autotel-drizzle`, `autotel-mongoose`).
-2. **Group with objects.** `{ user: { id, plan } }` not `userId` / `userPlan`. The flatten step keeps the key shape stable.
-3. **Capture decisions, not just inputs.** Which branch ran, which promo applied, which fallback fired.
-4. **Keep cardinality bounded.** Don't put per-request UUIDs in `span.name`; use `SpanNameNormalizingProcessor`. Free-text labels go in attributes.
-5. **Avoid raw bodies.** Pick the shape: `{ user: { id, plan } }`: never `log.set({ user: requestBody })`.
-6. **Trust the redactor.** PII you forgot to think about (emails, JWTs, cards) gets masked in production. See `attributeRedactor: 'default'`.
-
-## When you need correlated child spans
-
-Use `trace()` to wrap discrete sub-operations whose duration matters:
-
-```typescript
-import { trace } from 'autotel';
-
-const fetchInventory = trace(async (sku: string) => {
-  /* … */
-});
-const reserveStock = trace(async (sku: string, qty: number) => {
-  /* … */
-});
-
-await fetchInventory(sku);
-await reserveStock(sku, qty);
-```
-
-Each gets its own span with the function name; both are children of the active request span.
-
-## When you need background work
-
-`log.fork('label', fn)` spawns a child span that emits its own wide event with `_parentCorrelationId` set, even after the parent response has been returned. Pass `lifecycle.onChildEnter / onChildExit` if your framework tracks active loggers (Elysia, etc.).
-
-```typescript
-log.fork('audit-write', async () => {
-  await audit.write({ kind: 'order.created', orderId });
-});
-return Response.json({ ok: true }); // parent returns immediately
-```

package/skills/tune-sampling/SKILL.md

@@ -1,290 +0,0 @@
----
-name: tune-sampling
-description: >
-  Choose a sampling strategy for an autotel-instrumented service. Covers
-  head sampling (per-span-kind rates, parent-based, ratio), tail sampling
-  (keep errors, slow, AI-aware, debug-headers), cost vs cardinality
-  tradeoffs, and the math for picking rates that hit a target spans/second
-  budget. Includes recipes for low-volume admin services, high-volume APIs,
-  AI agents, and Cloudflare Workers.
-license: MIT
----
-
-# Tune sampling
-
-Untuned tracing is either expensive (100 % at scale costs money + drowns dashboards) or unhelpful (1 % loses the failure modes you need to see). The right answer is almost always **head sample most of the boring traffic, tail keep all the interesting traffic**, with explicit overrides for AI calls and customer escalations.
-
-## When to use
-
-- Hitting your observability budget
-- Dashboards too sparse to spot anomalies
-- "We have the trace IDs but the spans are gone" complaints
-- New service launching at scale
-- Long-running AI agents producing 50+ spans per request
-
-## The mental model
-
-```
-Total cost = (spans/sec × $/span) + (storage_GB × $/GB-month)
-                   ↑
-Head sampling reduces this directly.
-```
-
-Head sampling makes a decision **at span start**: fast but coarse, since it cannot know whether the span will fail.
-Tail sampling makes the decision **at span end**: slower and heavier on storage, but precise.
-
-The right mix:
-
-- **Head sample at the entry point** to keep volume tractable.
-- **Tail keep** the high-value subset (errors, slow, AI, debug-headered).
-- **Don't sample audit spans**: separate processor, see [`build-audit-trails`](../build-audit-trails/SKILL.md).
-
-## Head sampling recipes
-
-### Default for a typical web service
-
-```typescript
-init({
-  service: 'my-app',
-  sampling: {
-    rates: {
-      server: 25, // server entry spans — sample ¼
-      client: 5, // outbound HTTP — sample 1/20
-      internal: 5, // internal sub-spans — sample 1/20
-    },
-  },
-});
-```
-
-Children of a sampled root are **all** kept (parent-based propagation is the default). So `server: 25` means 25 % of _user requests_, complete trace each.
-
-### High-volume API (>1 k req/s)
-
-```typescript
-sampling: {
-  rates: { server: 5, client: 1, internal: 1 }, // 5 % → tail keeps errors anyway
-  tail: keepInterestingTraces,
-},
-```
-
-### Low-volume admin / internal service (<10 req/s)
-
-100 % is fine. Don't penalise yourself for a service that produces 1 GB of traces a week.
-
-### Cloudflare Workers (per-colo budget)
-
-Workers run distributed. Head sampling is your friend because there's no central queue:
-
-```typescript
-defineWorkerFetch(
-  {
-    service: { name: 'edge' },
-    sampling: { rates: { server: 10 } }, // 10 % per colo, scales naturally
-  },
-  handler,
-);
-```
-
-## Tail sampling: keep interesting traces
-
-Tail sampling looks at the full trace (root span + children) before deciding. autotel ships `TailSamplingProcessor`:
-
-```typescript
-import { TailSamplingProcessor } from 'autotel/processors';
-import { SpanStatusCode } from '@opentelemetry/api';
-
-const tail = new TailSamplingProcessor({
-  keep: (trace) => {
-    // 1. Always keep errors
-    if (trace.localRootSpan.status?.code === SpanStatusCode.ERROR) return true;
-    if (trace.spans.some((s) => s.status?.code === SpanStatusCode.ERROR))
-      return true;
-
-    // 2. Always keep slow traces (configurable threshold)
-    if (durationMs(trace.localRootSpan) > 1_000) return true;
-
-    // 3. Always keep customer-marked traces
-    if (trace.localRootSpan.attributes['debug.trace'] === true) return true;
-
-    // 4. Always keep AI traces (rare + expensive — full visibility helps)
-    if (
-      trace.spans.some(
-        (s) =>
-          typeof s.attributes['gen_ai.provider.name'] === 'string' ||
-          // legacy: third-party instrumentations may still emit gen_ai.system
-          typeof s.attributes['gen_ai.system'] === 'string',
-      )
-    )
-      return true;
-
-    // 5. Otherwise: respect head sampling decision
-    return false;
-  },
-});
-```
-
-### Combining with multi-backend
-
-```typescript
-spanProcessors: composeSpanProcessors([
-  // Drop nothing here — we want the tail processor to see the full trace
-  new BatchSpanProcessor(localExporter),
-  tail, // filters before remote export
-  new BatchSpanProcessor(expensiveRemoteExporter),
-]);
-```
-
-## AI / LLM-aware sampling
-
-LLM calls produce 5–50 spans per request and are 100× more expensive than a typical handler call. Tradeoffs:
-
-- **Don't head-sample AI handlers below 50 %**: debugging "why did the model loop" requires the full chain.
-- **Always tail-keep AI traces**: the `gen_ai.*` attributes flag them.
-- **Cost-aware sampling**: keep all calls above a $ threshold:
-
-```typescript
-keep: (trace) => {
-  const cost = trace.spans.reduce(
-    (acc, s) =>
-      acc +
-      (typeof s.attributes['gen_ai.usage.cost.usd'] === 'number'
-        ? (s.attributes['gen_ai.usage.cost.usd'] as number)
-        : 0),
-    0,
-  );
-  if (cost > 0.1) return true; // any trace > $0.10 → keep
-  if (cost > 0.01) return Math.random() < 0.5; // > $0.01 → 50 %
-  return Math.random() < 0.1; // < $0.01 → 10 %
-};
-```
-
-## Customer-driven sampling (debug header)
-
-Let support flip on full tracing per request:
-
-```typescript
-const tail = new TailSamplingProcessor({
-  keep: (trace) => trace.localRootSpan.attributes['x-debug-trace'] === '1' || /* … */,
-})
-```
-
-In your middleware:
-
-```typescript
-if (request.headers.get('x-debug-trace') === '1') {
-  useLogger().set({ 'x-debug-trace': '1' });
-}
-```
-
-Now any user can mark a request as "trace this fully" by sending the header. Invaluable for reproducing customer reports.
-
-## Keeping the trace whole across services
-
-`RandomSampler` rolls the dice in each process. The API can keep a trace its
-worker drops, which leaves a waterfall with holes and no way to tell a dropped
-span from a fast one. `DeterministicSampler` hashes a key that travels with the
-request, so every service reaches the same verdict:
-
-```typescript
-import { DeterministicSampler } from 'autotel/sampling';
-import { trace } from '@opentelemetry/api';
-
-init({
-  service: 'checkout',
-  sampler: new DeterministicSampler({
-    sampleRate: 0.1,
-    key: () => trace.getActiveSpan()?.spanContext().traceId,
-  }),
-});
-```
-
-Give every service the same rate and the same key. Different rates break the
-agreement again.
-
-## Per-key target rates
-
-A single rate cannot fit skewed traffic. At 1 % your busiest endpoint still
-floods storage while the rare tenant whose failures you need disappears.
-`KeyTargetRateSampler` counts traffic per key over a rolling window and gives
-each key its own rate, so every key contributes about the same number of
-events:
-
-```typescript
-import { KeyTargetRateSampler } from 'autotel/sampling';
-
-new KeyTargetRateSampler({
-  key: (context) => context.operationName,
-  targetPerKey: 10, // ~10 events per key per window
-  windowMs: 30_000,
-  maxKeys: 1000, // keys past this share one overflow bucket
-});
-```
-
-The first window keeps everything, because no traffic history exists yet. Rates
-apply from the second window on.
-
-## Recording the rate
-
-Sampled counts lie unless you record what you discarded. A query over a 1 %
-sample reports a hundredth of your traffic, and the number looks plausible
-enough to act on.
-
-autotel writes `autotel.sampling.rate` on the span as "1 in N", so
-`COUNT * rate` estimates the population. It appears only when N exceeds 1, so
-fully captured spans stay clean. A custom sampler joins in by implementing
-`sampleRate()`:
-
-```typescript
-import type { Sampler } from 'autotel/sampling';
-
-class TenantSampler implements Sampler {
-  shouldSample(context) {
-    /* … */
-  }
-  sampleRate() {
-    return 20; // each kept event stands for 20
-  }
-}
-```
-
-Reach for `hashUnitInterval` from `autotel/sampling` when you need the same
-spread for your own keys.
-
-## Sizing the rate
-
-Target volume:
-
-```
-spans/sec ≈ requests/sec × spans_per_request × head_rate × tail_keep_rate
-```
-
-Worked example for a 100 req/s API with 8 spans/req:
-
-| Head rate | Tail keep                        | Result                                      |
-| --------- | -------------------------------- | ------------------------------------------- |
-| 100 %     | 100 %                            | 800 spans/sec: expensive                    |
-| 10 %      | 100 % (errors + slow + AI ≈ 5 %) | ≈ 110 spans/sec: sweet spot                 |
-| 1 %       | 100 %                            | ≈ 18 spans/sec: too sparse for p99 alerting |
-
-For per-vendor pricing:
-
-- **Honeycomb**: $0.000005 / event for paid plans. 110 spans/sec × 86 400 s = 9.5 M events/day = $48/day.
-- **Datadog APM**: ~$1.27/M spans ingested (varies by region). Same volume → ~$12/day.
-- **Grafana Cloud**: 100 GB free tier; 110 spans/sec ≈ 5 GB/day.
-
-## Anti-patterns
-
-| Anti-pattern                                 | Fix                                                           |
-| -------------------------------------------- | ------------------------------------------------------------- |
-| 100 % sampling at scale "to be safe"         | You're paying 10–100× without proportional value              |
-| 1 % sampling with no tail keep               | You'll miss every interesting failure                         |
-| Forgetting to tail-keep errors               | Sampled traces with errors → silent customer pain             |
-| Same rate for `server` and `internal`        | Internal sub-spans are 5–20× more numerous; sample harder     |
-| Ratio-based sampling on service entry point  | Use parent-based: children of a sampled trace stay together   |
-| Head-sampling AI calls below 50 %            | Debugging tool loops requires the full chain                  |
-| Audit spans subject to sampling              | Route them to a separate processor (see `build-audit-trails`) |
-| Tail processor before exporter (loses spans) | Tail processor goes between head sampler and remote exporter  |
-| Rate-by-route hand-coded in handlers         | Use head sampler + tail keep: declarative, one place          |
-| Counting sampled spans as the population     | Multiply by `autotel.sampling.rate`                           |
-| `RandomSampler` on a multi-service trace     | `DeterministicSampler` keyed on the trace id keeps it whole   |
-| One flat rate over skewed traffic            | `KeyTargetRateSampler` gives rare keys their own rate         |