@adonis-agora/telescope 0.8.3 → 0.8.5

package/skills/telescope-storage-retention/SKILL.md

@@ -0,0 +1,292 @@
+---
+name: telescope-storage-retention
+description: >-
+  Storage, retention, redaction and sampling in @adonis-agora/telescope — the
+  storage.memory ring buffer vs the storage.lucid SQL driver (optional peer,
+  create_telescope_entries_table migration), the TelescopeStore contract
+  (record/get/list/count/prune/clear), prune {after,keepLast,intervalMs}, the
+  overload guard + setTelescopePaused, redact {keys,perType}, tail-sampling rules
+  {rate,keepErrors,keepSlowMs}, and writing a custom store that resolves
+  traceId/origin. Use for "persistent storage", "entries lost on restart",
+  "table keeps growing", "redact secrets", "sample noisy entry types", "custom
+  store".
+license: MIT
+metadata:
+  type: core
+  library: "@adonis-agora/telescope"
+  library_version: "0.8.4"
+  framework: adonisjs
+sources:
+  - "DavideCarvalho/adonis-telescope:docs/concepts/storage.mdx"
+  - "DavideCarvalho/adonis-telescope:docs/concepts/protection.mdx"
+  - "DavideCarvalho/adonis-telescope:docs/packages/storages.mdx"
+  - "DavideCarvalho/adonis-telescope:docs/recipes/custom-storage.mdx"
+---
+
+# Storage, retention & protection
+
+A store sits between capture and query: watchers `record` into it, and
+`TelescopeService` reads out of it. Storage is **config-driven** — build named
+drivers with the `storage` factory, pick one with `store`. Two ship in the box:
+`memory` (bounded ring buffer) and `lucid` (persistent SQL via an optional
+`@adonisjs/lucid` peer imported lazily only when selected).
+
+## Setup
+
+```ts
+// config/telescope.ts
+import { defineConfig, storage } from '@adonis-agora/telescope'
+
+export default defineConfig({
+  store: 'lucid',
+  stores: {
+    memory: storage.memory({ limit: 5000 }),   // dev/tests — per-process, lost on restart
+    lucid: storage.lucid(),                    // or storage.lucid({ connection: 'pg', tableName: 'tscope' })
+  },
+  // Retention pruner — OFF unless a prune block is present.
+  prune: { after: '24h', keepLast: 50_000, intervalMs: 60_000 },
+  // Event-loop overload guard (on by default): pauses ingestion when p99 lag >= maxEventLoopLagMs.
+  overload: { enabled: true, maxEventLoopLagMs: 200, startupGraceMs: 5_000 },
+})
+```
+
+The lucid driver needs its table; `node ace configure @adonis-agora/telescope`
+publishes it next to the config:
+
+```sh
+node ace migration:run   # create_telescope_entries_table (+ indexes on created_at/type/trace_id/family_hash)
+```
+
+Source: `docs/concepts/storage.mdx`, `docs/packages/storages.mdx`.
+
+## Core patterns
+
+### Pattern 1 — query the store through TelescopeService
+
+Every set field of `EntryQuery` is an AND predicate; results are always
+newest-first.
+
+```ts
+import { TelescopeService } from '@adonis-agora/telescope'
+
+const telescope = await app.container.make(TelescopeService)
+await telescope.list({ type: 'query', search: 'users', limit: 50 })
+await telescope.list({ tag: 'status:500', after: oneHourAgo })
+await telescope.list({ familyHash: 'billing:invoice-paid' })
+```
+
+Source: `docs/concepts/storage.mdx` (Querying).
+
+### Pattern 2 — redaction and per-type bounds
+
+Redaction is **on by default** at the one boundary every watcher records through —
+the store's `record()` — so no watcher can bypass it. Sensitive keys
+(`authorization`, `cookie`, `password`, `token`, `secret`, …) are masked with
+`[REDACTED]` case-insensitively at any depth; the clone is also memory-bounded and
+cycle-safe.
+
+```ts
+// config/telescope.ts
+export default defineConfig({
+  redact: {
+    enabled: true,
+    keys: ['ssn', 'credit_card'],                       // merged with the built-in set
+    // perType raises only the NUMERIC bounds for one type — masking stays uniform:
+    perType: { exception: { maxContentBytes: 64_000 } },
+  },
+})
+```
+
+The same `redact()` function is exported to scrub a value before building custom
+content yourself.
+
+Source: `docs/reference/configuration.mdx` (redact callout),
+`docs/recipes/tags-and-redaction.mdx` (Built-in redaction).
+
+### Pattern 3 — tail-sample noisy types without losing errors
+
+`sampling` runs on the write path. A bare number is a uniform keep-rate; per-type
+rules keep everything that looks like an error or is slow, no matter the rate.
+
+```ts
+export default defineConfig({
+  sampling: {
+    default: 1,                                   // keep everything by default
+    cache: 0.1,                                   // a tenth of cache entries
+    request: { rate: 0.25, keepErrors: true, keepSlowMs: 500 },  // but all errors/slow
+  },
+})
+```
+
+Sampled-out entries are never persisted, and the dashboard's retention indicator
+shows which types record below 100% so counts aren't misread.
+
+Source: `docs/recipes/tags-and-redaction.mdx` (Sampling),
+`docs/packages/ui.mdx` (Retention endpoint).
+
+### Pattern 4 — pause capture around bulk work
+
+`setTelescopePaused(true)` flips the same flag the overload guard uses; every
+ingestion point honours it immediately.
+
+```ts
+import { setTelescopePaused } from '@adonis-agora/telescope'
+
+setTelescopePaused(true)
+try {
+  await importEverything()
+} finally {
+  setTelescopePaused(false)   // always restore: process-global, nothing else flips it back
+}
+```
+
+Source: `docs/concepts/protection.mdx` (Pausing on purpose).
+
+### Pattern 5 — a custom store
+
+Implement the six-method contract and pass an instance as `store`. Two
+responsibilities are easy to miss: `record` must resolve `traceId`/`origin`, and
+`search` needs serialized content text.
+
+```ts
+import {
+  currentTraceId, isBatchOrigin,
+  type BatchOrigin, type Entry, type EntryQuery,
+  type RecordInput, type TelescopeStore,
+} from '@adonis-agora/telescope'
+import { randomUUID } from 'node:crypto'
+
+export class MongoTelescopeStore implements TelescopeStore {
+  async record<T>(input: RecordInput<T>): Promise<Entry<T>> {
+    const traceId = input.traceId !== undefined ? input.traceId : currentTraceId()
+    const origin: BatchOrigin = isBatchOrigin(input.origin) ? input.origin : 'manual'
+    const entry: Entry<T> = {
+      id: randomUUID(), type: input.type, familyHash: input.familyHash ?? null,
+      content: input.content, tags: input.tags ?? [], sequence: this.sequence++,
+      durationMs: input.durationMs ?? null, origin, traceId, createdAt: new Date(),
+    }
+    await this.col.insertOne({ ...entry, createdAtMs: entry.createdAt.getTime(),
+                               contentText: JSON.stringify(entry.content) }) // search target
+    return entry
+  }
+  /* get / list / count / prune(olderThan, keepLast?) / clear */
+}
+
+// config/telescope.ts
+export default defineConfig({ store: new MongoTelescopeStore(col) })
+```
+
+Source: `docs/recipes/custom-storage.mdx`.
+
+## Common mistakes
+
+### HIGH Shipping the memory driver to production
+
+```ts
+// Wrong — per-process ring buffer, lost on every restart/deploy; in a cluster each
+// worker shows a different slice of reality.
+defineConfig({ store: 'memory', stores: { memory: storage.memory({ limit: 1000 }) } })
+```
+
+```ts
+// Correct — durable, cross-query storage via the built-in Lucid driver.
+defineConfig({ store: 'lucid', stores: { lucid: storage.lucid() } })
+```
+
+Mechanism: eviction past the cap drops the oldest entries and nothing survives a
+restart, so "nothing here" after a deploy is indistinguishable from "nothing
+happened".
+Source: `docs/concepts/storage.mdx` (memory driver warn).
+
+### HIGH Switching to lucid without arming retention
+
+```ts
+// Wrong — a persistent SQL table with no time-based bound grows forever.
+defineConfig({ store: 'lucid', stores: { lucid: storage.lucid() } })
+```
+
+```ts
+// Correct — supply a prune block; that is what arms the background pruner.
+defineConfig({
+  store: 'lucid',
+  stores: { lucid: storage.lucid() },
+  prune: { after: '24h', keepLast: 50_000 },
+})
+```
+
+Mechanism: no store auto-prunes by age (the memory driver evicts by count, the
+lucid store never auto-evicts because that would add a query per write); the
+pruner exists only when a `prune` block is supplied.
+Source: `docs/concepts/protection.mdx` (info callout), `docs/packages/storages.mdx` (warn).
+
+### HIGH Custom store skipping traceId/origin resolution
+
+```ts
+// Wrong — traceId left undefined and origin unvalidated:
+traceId: input.traceId, origin: input.origin as BatchOrigin,
+```
+
+```ts
+// Correct — explicit null respected, omitted reads context, junk falls back:
+const traceId = input.traceId !== undefined ? input.traceId : currentTraceId()
+const origin: BatchOrigin = isBatchOrigin(input.origin) ? input.origin : 'manual'
+```
+
+Mechanism: skipping this silently breaks trace correlation for everything
+downstream (`byTrace`, waterfalls, N+1 detection) while every write still succeeds.
+Source: `docs/recipes/custom-storage.mdx` (load-bearing lines).
+
+### MEDIUM Importing services/db inside a provider's boot()
+
+```ts
+// Wrong — the facade default export is undefined until app.booted(), which runs
+// AFTER provider boot.
+import db from '@adonisjs/lucid/services/db'
+const store = new LucidTelescopeStore(db)
+```
+
+```ts
+// Correct — resolve the binding from the container during boot.
+const db = await app.container.make('lucid.db')
+const store = new LucidTelescopeStore(db, { tableName: 'telescope_entries' })
+```
+
+Mechanism: provider boot precedes `app.booted()`; resolving lazily through the
+container gets the live binding instead of `undefined`.
+Source: `docs/packages/storages.mdx` (Programmatic API note).
+
+### MEDIUM Treating sampling as a security control
+
+```ts
+// Wrong — assuming dropping 90% of queries hides the secrets in the other 10%.
+sampling: { query: 0.1 }
+```
+
+```ts
+// Correct — scrub at the store boundary (always on); sample only for retention cost.
+redact: { keys: ['ssn'] },
+sampling: { query: { rate: 0.1, keepErrors: true } },
+```
+
+Mechanism: sampling decides what gets *stored*, not what gets *scrubbed*; redaction
+runs on every recorded entry regardless of rate.
+Source: `docs/packages/advanced.mdx` (sampling warn callout).
+
+### MEDIUM Misreading prune keepLast semantics
+
+```ts
+// Wrong — assuming keepLast caps the WHOLE table at N rows.
+await store.prune(cutoff, 500)   // expected: table trimmed to newest 500 overall
+```
+
+```ts
+// Correct — keepLast retains the newest N of the matched-and-doomed set:
+const deleted = await store.prune(cutoff, 500) // deletes older-than-cutoff EXCEPT its newest 500
+```
+
+Mechanism: the doomed set is computed first (everything older than cutoff), then
+its newest `keepLast` survive; total row count can exceed `keepLast`.
+Source: `docs/concepts/storage.mdx` (Retention and pruning).
+
+See also: `telescope-watchers/SKILL.md` — the capture side these bounds protect;
+`telescope-alerts-ai/SKILL.md` — why sampled-out exceptions never page.

package/skills/telescope-watchers/SKILL.md

@@ -0,0 +1,317 @@
+---
+name: telescope-watchers
+description: >-
+  Capture sources with @adonis-agora/telescope — built-in request/diagnostics/logs
+  watchers, the @adonis-agora/telescope/watchers subpath (LucidQueryWatcher db:query,
+  MailWatcher mail:sent, CacheWatcher cache:* events, HttpClientWatcher fetch wrap,
+  QueueWatcher boringqueue.job.execute, EventsWatcher onAny, RedisWatcher
+  sendCommand, profile()/startProfile(), scheduleTask()/registerSchedule()), and
+  custom watchers via the Watcher contract + safeRecord + currentTraceId. Use for
+  "record SQL queries", "watch outbound HTTP", "no queries recorded", "write a
+  custom watcher", "correlate entries to a trace", "AdonisJS telemetry".
+license: MIT
+metadata:
+  type: core
+  library: "@adonis-agora/telescope"
+  library_version: "0.8.4"
+  framework: adonisjs
+sources:
+  - "DavideCarvalho/adonis-telescope:docs/concepts/capture.mdx"
+  - "DavideCarvalho/adonis-telescope:docs/packages/watchers.mdx"
+  - "DavideCarvalho/adonis-telescope:docs/recipes/custom-watcher.mdx"
+  - "DavideCarvalho/adonis-telescope:packages/core/src/watchers/index.ts"
+---
+
+# Watchers & capture
+
+The core records requests, exceptions and diagnostics automatically. Every other
+source — Lucid SQL, outbound fetch, mail, cache, queue jobs, events, Redis, logs,
+timing spans, scheduled tasks — comes from a watcher. Per-technology watchers live
+in the `@adonis-agora/telescope/watchers` subpath of the same package; nothing extra
+to install.
+
+## Setup
+
+Re-run configure and pick **Watchers**, then tune the published config:
+
+```sh
+node ace configure @adonis-agora/telescope   # pick "Watchers" at the prompt
+```
+
+```ts
+// config/telescope_watchers.ts
+import { defineConfig } from '@adonis-agora/telescope/watchers'
+
+export default defineConfig({
+  enabled: true,
+  watchers: ['query'],   // only the verified Lucid query watcher is on by default
+  query: { slowMs: 500, captureBindings: false, normalize: true },
+  httpClient: { slowMs: 1000, ignoreHosts: [], captureBodies: false }, // body SIZES, never bytes
+})
+```
+
+Enable more watchers by adding names: `'query'`, `'mail'`, `'cache'`,
+`'http-client'`, `'logs'`, `'queue'`, `'events'`, `'redis'`, `'profiling'`,
+`'schedule'`, `'queue-manager'`.
+
+Source: `docs/packages/watchers.mdx` (Configuration).
+
+## Core patterns
+
+### Pattern 1 — record every Lucid query
+
+The query watcher subscribes to Lucid's `db:query` event. Entries group by a
+normalized SQL template hash (`select * from users where id = ?`), so every
+execution of the same statement rolls into one family for `topFamilies()`.
+
+```ts
+// config/telescope_watchers.ts
+export default defineConfig({
+  watchers: ['query'],
+  // Bindings carry PII/secrets: redacted to [REDACTED] placeholders unless you opt in.
+  query: { slowMs: 500, captureBindings: false, normalize: true },
+  // ignoreConnections: ['telescope'] — drop e.g. telescope's own lucid-store writes.
+})
+```
+
+Lucid only **emits** `db:query` when the connection's `debug` flag is on or a
+listener exists at report time; subscribing the watcher is enough to make Lucid
+report, but `debug: true` in your DB config guarantees it:
+
+```ts
+// config/database.ts
+export const databaseConfig = defineConfig({
+  connections: {
+    pg: {
+      connection: { host: '127.0.0.1', database: 'app', user: 'postgres', password: 'secret' },
+      debug: true,   // guarantee db:query flows even before any other listener attaches
+    },
+  },
+})
+```
+
+Source: `docs/packages/watchers.mdx` (Query watcher callouts),
+`docs/getting-started.mdx` Step 5.
+
+### Pattern 2 — user-instrumented spans and scheduled runs
+
+`profile()`/`startProfile()` and `scheduleTask()`/`registerSchedule()` are helpers
+you call around code — nothing is captured until the watcher is enabled AND a
+helper runs. While disabled they are zero-cost no-ops that still execute your
+closure.
+
+```ts
+import {
+  profile,
+  scheduleTask,
+  registerSchedule,
+} from '@adonis-agora/telescope/watchers'
+
+// Timing span — times fn, records completed/failed, re-throws on error.
+const total = await profile('checkout', async (span) => {
+  span.mark('cart-loaded')
+  return await charge(cart)
+})
+
+// Scheduled run — wraps the closure you actually schedule.
+await scheduleTask('prune-sessions', () => sessions.pruneExpired(), {
+  schedule: '0 * * * *',
+  kind: 'cron',
+})
+
+// Register what EXISTS so the dashboard can compute next-run
+// (optional cron-parser peer; without it nextRunAt is null, not an error).
+registerSchedule({ name: 'prune-sessions', schedule: '0 * * * *', kind: 'cron' })
+```
+
+Source: `docs/packages/watchers.mdx` (Profiling watcher, Schedule watcher).
+
+### Pattern 3 — a custom watcher
+
+Implement `{ type, start(emitter), stop() }` over the structural `EmitterLike`
+(`on` returns an unsubscribe function). Record through `safeRecord(input, source)`:
+it resolves the runtime store, backfills `currentTraceId()`, and swallows every
+failure so your watcher cannot break the path it observes.
+
+```ts
+// app/telescope/queue_watcher.ts
+import { EntryType, type RecordInput, currentTraceId } from '@adonis-agora/telescope'
+import { safeRecord, type Watcher, type EmitterLike } from '@adonis-agora/telescope/watchers'
+
+export class QueueWatcher implements Watcher {
+  readonly type = EntryType.Job
+  private unsubscribe: (() => void) | null = null
+
+  start(emitter: EmitterLike): void {
+    if (this.unsubscribe) return                       // idempotent
+    this.unsubscribe = emitter.on('queue:processed', (data) => this.handle(data))
+  }
+
+  stop(): void {
+    this.unsubscribe?.()
+    this.unsubscribe = null
+  }
+
+  private handle(data: unknown): void {
+    if (!isQueueEvent(data)) return                    // tolerate junk on the channel
+    safeRecord(buildJobEntry(data), 'QueueWatcher')    // guarded; never throws
+  }
+}
+
+function isQueueEvent(d: unknown): d is { queue: string; jobId: string; durationMs: number; failed: boolean } {
+  return typeof d === 'object' && d !== null && 'queue' in d && 'jobId' in d
+}
+
+function buildJobEntry(e: { queue: string; jobId: string; durationMs: number; failed: boolean }): RecordInput {
+  const traceId = currentTraceId()
+  return {
+    type: EntryType.Job,
+    familyHash: e.queue,                               // topFamilies(10, 'job') rolls up here
+    durationMs: e.durationMs,
+    traceId,
+    tags: [`queue:${e.queue}`, ...(e.failed ? ['failed'] : [])],
+    content: { queue: e.queue, jobId: e.jobId, failed: e.failed, durationMs: e.durationMs },
+  }
+}
+```
+
+Start it yourself in a provider (`boot()` → resolve `'emitter'` from the container
+→ `watcher.start(emitter)`; `shutdown()` → `stop()`). The watchers config controls
+only the built-ins — a custom watcher is just a class you start.
+
+Source: `docs/recipes/custom-watcher.mdx`.
+
+### Pattern 4 — tag entries by tenant/user
+
+`getContextAccessor()` reads `@adonis-agora/context` structurally; add tags at
+record time so `list({ tag })` pivots on them.
+
+```ts
+import { getContextAccessor, type RecordInput } from '@adonis-agora/telescope'
+
+export function withTenantTag<T>(input: RecordInput<T>): RecordInput<T> {
+  const tenant = getContextAccessor()?.tenantId()
+  if (tenant === undefined) return input
+  return { ...input, tags: [...(input.tags ?? []), `tenant:${tenant}`] }
+}
+```
+
+Source: `docs/recipes/tags-and-redaction.mdx`, `docs/recipes/request-context.mdx`.
+
+## Common mistakes
+
+### CRITICAL Enabling the query watcher without Lucid debug
+
+```ts
+// Wrong — the watcher is listed, yet zero queries appear.
+export default defineConfig({ watchers: ['query'] })
+// config/database.ts has no debug flag and no other db:query listener exists.
+```
+
+```ts
+// Correct — turn on debug for the connection (or attach another db:query listener).
+pg: { /* ... */ debug: true }
+```
+
+Mechanism: Lucid builds the `db:query` event only when `debug` is on or a listener
+exists at report time — otherwise the event is never even constructed, so the
+enabled watcher silently records nothing while looking healthy.
+Source: `docs/packages/watchers.mdx` (Query watcher warn callout),
+`docs/getting-started.mdx` Step 5.
+
+### HIGH Enabling the logs watcher in both config files
+
+```ts
+// Wrong — 'logs' in config/telescope.ts AND config/telescope_watchers.ts.
+// config/telescope.ts
+export default defineConfig({ watchers: ['request', 'diagnostics', 'logs'], logs: { minLevel: 'warn' } })
+// config/telescope_watchers.ts
+export default defineConfig({ watchers: ['query', 'logs'] })
+```
+
+```ts
+// Correct — enable it in exactly one place.
+export default defineConfig({ watchers: ['request', 'diagnostics', 'logs'], logs: { minLevel: 'warn' } })
+export default defineConfig({ watchers: ['query'] })
+```
+
+Mechanism: whichever provider boots first owns the logger tap; the other warns and
+stays inert, so its `minLevel`/`tags` options are silently ignored.
+Source: `docs/reference/configuration.mdx` ('logs' warn callout).
+
+### HIGH Building entry content outside safeRecord
+
+```ts
+// Wrong — content building runs on the HOST thread; one circular payload breaks
+// the very operation being observed.
+emitter.on('queue:processed', (d) => {
+  safeRecord({ type: EntryType.Job, content: JSON.parse(JSON.stringify(d.raw)) }, 'QueueWatcher')
+})
+```
+
+```ts
+// Correct — validate defensively, build plain content, let safeRecord guard the store.
+private handle(data: unknown): void {
+  if (!isQueueEvent(data)) return
+  safeRecord(buildJobEntry(data), 'QueueWatcher')
+}
+```
+
+Mechanism: recording paths are fire-and-forget and fully guarded — a missing store,
+a throwing `record`, or a rejected promise is swallowed and warn-logged, never
+thrown into the request/job/query being watched.
+Source: `docs/concepts/capture.mdx` (warn callout), `docs/recipes/custom-watcher.mdx`.
+
+### MEDIUM Non-idempotent start/leaky stop in a custom watcher
+
+```ts
+// Wrong — double-subscribe duplicates every entry; a leaked subscription keeps
+// recording after shutdown.
+start(emitter: EmitterLike): void {
+  emitter.on('queue:processed', this.handle)
+}
+stop(): void {}
+```
+
+```ts
+// Correct — keep the unsubscribe fn; guard re-entry; release in stop().
+start(emitter: EmitterLike): void {
+  if (this.unsubscribe) return
+  this.unsubscribe = emitter.on('queue:processed', this.handle)
+}
+stop(): void {
+  this.unsubscribe?.()
+  this.unsubscribe = null
+}
+```
+
+Mechanism: `emitter.on` returns an unsubscribe function; a watcher that leaks it
+records entries into a torn-down process and double-starts duplicate everything.
+Source: `docs/recipes/custom-watcher.mdx` (warn callout).
+
+### MEDIUM Waiting on queue/redis watchers whose engines were never installed
+
+```ts
+// Wrong — assuming pnpm installed the engines because the watcher names are valid.
+export default defineConfig({ watchers: ['queue', 'redis'] })
+// package.json has no @adonisjs/queue / @adonisjs/redis entry.
+```
+
+```json
+// Correct — add the packages yourself; they are NOT declared peers.
+{
+  "dependencies": {
+    "@adonisjs/queue": "^...",
+    "@adonisjs/redis": "^..."
+  }
+}
+```
+
+Mechanism: those two are deliberately not declared peers — their watchers tap a
+Node diagnostics channel / wrap a manager they're handed, idle when nobody feeds
+them, and npm will never install them for you.
+Source: `docs/packages/watchers.mdx` (Peer dependencies).
+
+See also: `telescope-storage-retention/SKILL.md` — where captured entries land and
+how bounds/sampling shape what survives;
+`telescope-alerts-ai/SKILL.md` — rules only ever see what was recorded.