@specific.dev/cli 0.1.161 → 0.1.163

package/dist/docs/observability.md

@@ -0,0 +1,192 @@
+# Observability
+
+Every deployed environment streams logs and metrics into a queryable store. Use `specific query` to run SQL against this data — useful for debugging production incidents, investigating regressions, and ad-hoc analytics.
+
+## Running a query
+
+```sh
+# Inline
+specific query "SELECT count() FROM observability.logs"
+
+# Against a specific environment
+specific query --environment staging "SELECT * FROM observability.logs LIMIT 5"
+
+# From a file via stdin
+cat queries/p99.sql | specific query
+specific query - < queries/p99.sql
+```
+
+Flags:
+
+- `-e, --environment <name|id>` — target environment (defaults to the current one).
+
+Queries are automatically scoped to the selected environment. You cannot see data from other environments and you do not need to filter on environment yourself.
+
+Queries are read-only. `INSERT`, `UPDATE`, `DELETE`, and DDL are rejected. Each query is capped at 30 seconds of execution time, returns at most 100,000 rows, and the SQL string itself is limited to 50,000 characters.
+
+## Schema
+
+Two views are exposed in the `observability` database.
+
+### `observability.logs`
+
+Unified log stream from services.
+
+| Column | Type | Notes |
+|---|---|---|
+| `Timestamp` | DateTime64(9) | When the log was emitted; use time arithmetic directly (`Timestamp > now() - INTERVAL 1 HOUR`) |
+| `ServiceName` | String | Name of the service that emitted the log |
+| `SeverityText` | String | |
+| `SeverityNumber` | UInt8 | |
+| `Body` | String | The log message |
+| `LogAttributes` | Map(String, String) | Per-event attributes |
+| `ResourceAttributes` | Map(String, String) | Resource labels (e.g. `service.name`, `service.namespace`, `deployment.environment.name`) |
+| `TraceId` | String | Distributed-tracing trace ID, if present |
+| `SpanId` | String | Distributed-tracing span ID, if present |
+
+Also exposed but rarely queried directly: `EnvironmentId`, `ProjectId` (already used for auto-scoping), `TraceFlags`.
+
+Retention: up to 30 days. Your plan may further restrict how far back queries can read; when that cutoff is in effect, `specific query` prints a one-line note on stderr.
+
+### `observability.metrics`
+
+Unified metrics from services.
+
+| Column | Type | Notes |
+|---|---|---|
+| `TimeUnix` | DateTime64(9) | Measurement timestamp; use time arithmetic directly (`TimeUnix > now() - INTERVAL 1 HOUR`) |
+| `MetricName` | String | e.g. `container.cpu.time` (see list below) |
+| `MetricType` | String | `'gauge'` or `'sum'` |
+| `Value` | Float64 | Measured value |
+| `ServiceName` | String | Service that emitted the metric |
+| `Attributes` | Map(String, String) | Metric dimensions |
+| `ResourceAttributes` | Map(String, String) | Resource labels |
+
+Also exposed but rarely queried directly: `StartTimeUnix` (DateTime64(9)), `EnvironmentId`, `ProjectId`, `Source`.
+
+Retention: up to 90 days. Your plan may further restrict how far back queries can read; when that cutoff is in effect, `specific query` prints a one-line note on stderr.
+
+#### Service metrics
+
+Emitted for every running service container.
+
+| Metric | Type | Notes |
+|---|---|---|
+| `container.cpu.time` | sum | Cumulative CPU seconds consumed; rate it (`avg(Value)` over a window or per-second deltas) for utilisation |
+| `container.cpu.usage` | gauge | Current CPU usage in cores |
+| `container.cpu.limit_utilization` | gauge | Fraction of CPU limit used (0–1) |
+| `container.memory.working_set` | gauge | Memory in active use (bytes) — the right number for "is this service close to OOM" |
+| `container.memory.usage` | gauge | Total memory usage including page cache (bytes) |
+| `container.memory.available` | gauge | Memory still available before the container limit (bytes) |
+| `k8s.volume.available` | gauge | Free space on attached volumes (bytes) |
+| `k8s.volume.capacity` | gauge | Total capacity of attached volumes (bytes) |
+
+## Debugging recipes
+
+### Recent errors for a specific service
+
+```sql
+SELECT Timestamp, Body
+FROM observability.logs
+WHERE ServiceName = 'api'
+  AND SeverityNumber >= 17
+  AND Timestamp >= now() - INTERVAL 1 HOUR
+ORDER BY Timestamp DESC
+LIMIT 50
+```
+
+### Search log bodies for a substring
+
+```sql
+SELECT Timestamp, ServiceName, Body
+FROM observability.logs
+WHERE positionCaseInsensitive(Body, 'connection refused') > 0
+  AND Timestamp >= now() - INTERVAL 6 HOUR
+ORDER BY Timestamp DESC
+LIMIT 100
+```
+
+### Warning-or-worse rate by service
+
+```sql
+SELECT ServiceName, count() AS errs
+FROM observability.logs
+WHERE SeverityNumber >= 13
+  AND Timestamp >= now() - INTERVAL 1 HOUR
+GROUP BY ServiceName
+ORDER BY errs DESC
+```
+
+### Service CPU timeseries (1-minute buckets)
+
+```sql
+SELECT
+  toStartOfMinute(TimeUnix) AS bucket,
+  avg(Value)                AS cpu_cores
+FROM observability.metrics
+WHERE MetricName = 'container.cpu.usage'
+  AND ServiceName = 'api'
+  AND TimeUnix >= now() - INTERVAL 1 HOUR
+GROUP BY bucket
+ORDER BY bucket
+```
+
+### Memory headroom over time
+
+```sql
+SELECT
+  TimeUnix,
+  ServiceName,
+  Value AS working_set_bytes
+FROM observability.metrics
+WHERE MetricName = 'container.memory.working_set'
+  AND TimeUnix >= now() - INTERVAL 30 MINUTE
+ORDER BY TimeUnix
+```
+
+### Correlate all logs belonging to one trace
+
+```sql
+SELECT Timestamp, ServiceName, Body
+FROM observability.logs
+WHERE TraceId = '<trace-id>'
+ORDER BY Timestamp
+```
+
+## Performance
+
+Both views are partitioned and ordered to make environment-scoped, time-bounded queries fast. To stay within the 30-second limit on large environments:
+
+- **Always filter by time** — `Timestamp >=` for logs, `TimeUnix >=` for metrics. Unbounded scans across 30/90 days will time out.
+- **Add `ServiceName` and/or `MetricName`** to narrow the partition further whenever you can.
+- **`SELECT` only the columns you need** rather than `SELECT *`; `Body`, `LogAttributes`, and `ResourceAttributes` are the heavyweight columns.
+- **`LIMIT` exploratory queries** while you're shaping them.
+
+## Discovering data
+
+When you don't know what's there, ask the database:
+
+```sql
+SHOW TABLES FROM observability;
+DESCRIBE observability.logs;
+
+SELECT DISTINCT ServiceName FROM observability.logs LIMIT 50;
+SELECT DISTINCT MetricName FROM observability.metrics LIMIT 100;
+```
+
+## Useful ClickHouse functions
+
+The store is ClickHouse, so the full ClickHouse SQL dialect is available. A few commonly needed functions:
+
+- `now()`, `now64()` — current time
+- `toStartOfMinute(t)`, `toStartOfFiveMinutes(t)`, `toStartOfHour(t)` — bucket timestamps for time-series aggregation
+- `positionCaseInsensitive(haystack, needle)` — case-insensitive substring search
+- `JSONExtractString(s, 'field')`, `JSONExtractInt`, `JSONExtractFloat` — pull fields out of JSON-formatted log bodies
+- `LogAttributes['key']`, `Attributes['key']`, `ResourceAttributes['key']` — read map columns
+
+---
+
+Related topics:
+
+- Run `specific docs services` for how services are defined
+- Run `specific docs postgres` for connecting to your application database (use `specific psql` for application data, not `specific query`)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@specific.dev/cli",
-  "version": "0.1.161",
+  "version": "0.1.163",
   "description": "CLI for Specific infrastructure-as-code",
   "type": "module",
   "main": "dist/cli.js",