@x12i/countex 1.0.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +192 -3
- package/dist/backup/export.d.ts +2 -0
- package/dist/backup/export.d.ts.map +1 -1
- package/dist/backup/export.js +2 -0
- package/dist/backup/export.js.map +1 -1
- package/dist/cli.d.ts +12 -0
- package/dist/cli.d.ts.map +1 -0
- package/dist/cli.js +190 -0
- package/dist/cli.js.map +1 -0
- package/dist/core/buffer.d.ts +4 -3
- package/dist/core/buffer.d.ts.map +1 -1
- package/dist/core/buffer.js +65 -9
- package/dist/core/buffer.js.map +1 -1
- package/dist/core/countex.d.ts.map +1 -1
- package/dist/core/countex.js +30 -3
- package/dist/core/countex.js.map +1 -1
- package/dist/core/extraction.d.ts +14 -2
- package/dist/core/extraction.d.ts.map +1 -1
- package/dist/core/extraction.js +117 -1
- package/dist/core/extraction.js.map +1 -1
- package/dist/core/mapping.d.ts.map +1 -1
- package/dist/core/mapping.js +44 -1
- package/dist/core/mapping.js.map +1 -1
- package/dist/core/timeframe.d.ts +7 -0
- package/dist/core/timeframe.d.ts.map +1 -1
- package/dist/core/timeframe.js +88 -0
- package/dist/core/timeframe.js.map +1 -1
- package/dist/index.d.ts +1 -1
- package/dist/index.d.ts.map +1 -1
- package/dist/insights/advisors/fanout-health.d.ts +3 -0
- package/dist/insights/advisors/fanout-health.d.ts.map +1 -0
- package/dist/insights/advisors/fanout-health.js +3 -0
- package/dist/insights/advisors/fanout-health.js.map +1 -0
- package/dist/insights/advisors/map-health.d.ts +6 -0
- package/dist/insights/advisors/map-health.d.ts.map +1 -0
- package/dist/insights/advisors/map-health.js +154 -0
- package/dist/insights/advisors/map-health.js.map +1 -0
- package/dist/insights/advisors/resource-advisor.d.ts +4 -0
- package/dist/insights/advisors/resource-advisor.d.ts.map +1 -0
- package/dist/insights/advisors/resource-advisor.js +102 -0
- package/dist/insights/advisors/resource-advisor.js.map +1 -0
- package/dist/insights/advisors/retention-advisor.d.ts +2 -0
- package/dist/insights/advisors/retention-advisor.d.ts.map +1 -0
- package/dist/insights/advisors/retention-advisor.js +6 -0
- package/dist/insights/advisors/retention-advisor.js.map +1 -0
- package/dist/insights/advisors-subpath.d.ts +4 -0
- package/dist/insights/advisors-subpath.d.ts.map +1 -0
- package/dist/insights/advisors-subpath.js +10 -0
- package/dist/insights/advisors-subpath.js.map +1 -0
- package/dist/insights/anomaly/baseline.d.ts +4 -0
- package/dist/insights/anomaly/baseline.d.ts.map +1 -0
- package/dist/insights/anomaly/baseline.js +54 -0
- package/dist/insights/anomaly/baseline.js.map +1 -0
- package/dist/insights/anomaly/detect-anomalies.d.ts +6 -0
- package/dist/insights/anomaly/detect-anomalies.d.ts.map +1 -0
- package/dist/insights/anomaly/detect-anomalies.js +245 -0
- package/dist/insights/anomaly/detect-anomalies.js.map +1 -0
- package/dist/insights/anomaly/ewma.d.ts +2 -0
- package/dist/insights/anomaly/ewma.d.ts.map +1 -0
- package/dist/insights/anomaly/ewma.js +15 -0
- package/dist/insights/anomaly/ewma.js.map +1 -0
- package/dist/insights/anomaly/missing-activity.d.ts +3 -0
- package/dist/insights/anomaly/missing-activity.d.ts.map +1 -0
- package/dist/insights/anomaly/missing-activity.js +8 -0
- package/dist/insights/anomaly/missing-activity.js.map +1 -0
- package/dist/insights/anomaly/robust-zscore.d.ts +2 -0
- package/dist/insights/anomaly/robust-zscore.d.ts.map +1 -0
- package/dist/insights/anomaly/robust-zscore.js +9 -0
- package/dist/insights/anomaly/robust-zscore.js.map +1 -0
- package/dist/insights/anomaly-subpath.d.ts +7 -0
- package/dist/insights/anomaly-subpath.d.ts.map +1 -0
- package/dist/insights/anomaly-subpath.js +20 -0
- package/dist/insights/anomaly-subpath.js.map +1 -0
- package/dist/insights/comparison/compare-periods.d.ts +5 -0
- package/dist/insights/comparison/compare-periods.d.ts.map +1 -0
- package/dist/insights/comparison/compare-periods.js +64 -0
- package/dist/insights/comparison/compare-periods.js.map +1 -0
- package/dist/insights/comparison/growth-rate.d.ts +3 -0
- package/dist/insights/comparison/growth-rate.d.ts.map +1 -0
- package/dist/insights/comparison/growth-rate.js +3 -0
- package/dist/insights/comparison/growth-rate.js.map +1 -0
- package/dist/insights/comparison/moving-average.d.ts +2 -0
- package/dist/insights/comparison/moving-average.d.ts.map +1 -0
- package/dist/insights/comparison/moving-average.js +6 -0
- package/dist/insights/comparison/moving-average.js.map +1 -0
- package/dist/insights/comparison/trend.d.ts +5 -0
- package/dist/insights/comparison/trend.d.ts.map +1 -0
- package/dist/insights/comparison/trend.js +84 -0
- package/dist/insights/comparison/trend.js.map +1 -0
- package/dist/insights/errors.d.ts +25 -0
- package/dist/insights/errors.d.ts.map +1 -0
- package/dist/insights/errors.js +54 -0
- package/dist/insights/errors.js.map +1 -0
- package/dist/insights/forecast/budget.d.ts +2 -0
- package/dist/insights/forecast/budget.d.ts.map +1 -0
- package/dist/insights/forecast/budget.js +6 -0
- package/dist/insights/forecast/budget.js.map +1 -0
- package/dist/insights/forecast/capacity.d.ts +2 -0
- package/dist/insights/forecast/capacity.d.ts.map +1 -0
- package/dist/insights/forecast/capacity.js +3 -0
- package/dist/insights/forecast/capacity.js.map +1 -0
- package/dist/insights/forecast/forecast.d.ts +5 -0
- package/dist/insights/forecast/forecast.d.ts.map +1 -0
- package/dist/insights/forecast/forecast.js +103 -0
- package/dist/insights/forecast/forecast.js.map +1 -0
- package/dist/insights/forecast/retention.d.ts +2 -0
- package/dist/insights/forecast/retention.d.ts.map +1 -0
- package/dist/insights/forecast/retention.js +3 -0
- package/dist/insights/forecast/retention.js.map +1 -0
- package/dist/insights/index.d.ts +4 -0
- package/dist/insights/index.d.ts.map +1 -0
- package/dist/insights/index.js +23 -0
- package/dist/insights/index.js.map +1 -0
- package/dist/insights/insights-client.d.ts +33 -0
- package/dist/insights/insights-client.d.ts.map +1 -0
- package/dist/insights/insights-client.js +344 -0
- package/dist/insights/insights-client.js.map +1 -0
- package/dist/insights/llm-subpath.d.ts +4 -0
- package/dist/insights/llm-subpath.d.ts.map +1 -0
- package/dist/insights/llm-subpath.js +10 -0
- package/dist/insights/llm-subpath.js.map +1 -0
- package/dist/insights/optimization/cost-optimizer.d.ts +6 -0
- package/dist/insights/optimization/cost-optimizer.d.ts.map +1 -0
- package/dist/insights/optimization/cost-optimizer.js +98 -0
- package/dist/insights/optimization/cost-optimizer.js.map +1 -0
- package/dist/insights/optimization/latency-optimizer.d.ts +3 -0
- package/dist/insights/optimization/latency-optimizer.d.ts.map +1 -0
- package/dist/insights/optimization/latency-optimizer.js +3 -0
- package/dist/insights/optimization/latency-optimizer.js.map +1 -0
- package/dist/insights/optimization/threshold-advisor.d.ts +2 -0
- package/dist/insights/optimization/threshold-advisor.d.ts.map +1 -0
- package/dist/insights/optimization/threshold-advisor.js +3 -0
- package/dist/insights/optimization/threshold-advisor.js.map +1 -0
- package/dist/insights/optimization/token-optimizer.d.ts +4 -0
- package/dist/insights/optimization/token-optimizer.d.ts.map +1 -0
- package/dist/insights/optimization/token-optimizer.js +133 -0
- package/dist/insights/optimization/token-optimizer.js.map +1 -0
- package/dist/insights/ranking/contribution.d.ts +2 -0
- package/dist/insights/ranking/contribution.d.ts.map +1 -0
- package/dist/insights/ranking/contribution.js +6 -0
- package/dist/insights/ranking/contribution.js.map +1 -0
- package/dist/insights/ranking/helpers.d.ts +22 -0
- package/dist/insights/ranking/helpers.d.ts.map +1 -0
- package/dist/insights/ranking/helpers.js +88 -0
- package/dist/insights/ranking/helpers.js.map +1 -0
- package/dist/insights/ranking/pareto.d.ts +2 -0
- package/dist/insights/ranking/pareto.d.ts.map +1 -0
- package/dist/insights/ranking/pareto.js +6 -0
- package/dist/insights/ranking/pareto.js.map +1 -0
- package/dist/insights/ranking/slices.d.ts +2 -0
- package/dist/insights/ranking/slices.d.ts.map +1 -0
- package/dist/insights/ranking/slices.js +6 -0
- package/dist/insights/ranking/slices.js.map +1 -0
- package/dist/insights/ranking/top-bottom.d.ts +2 -0
- package/dist/insights/ranking/top-bottom.d.ts.map +1 -0
- package/dist/insights/ranking/top-bottom.js +7 -0
- package/dist/insights/ranking/top-bottom.js.map +1 -0
- package/dist/insights/recommendations/evidence.d.ts +2 -0
- package/dist/insights/recommendations/evidence.d.ts.map +1 -0
- package/dist/insights/recommendations/evidence.js +3 -0
- package/dist/insights/recommendations/evidence.js.map +1 -0
- package/dist/insights/recommendations/recommendation.d.ts +6 -0
- package/dist/insights/recommendations/recommendation.d.ts.map +1 -0
- package/dist/insights/recommendations/recommendation.js +25 -0
- package/dist/insights/recommendations/recommendation.js.map +1 -0
- package/dist/insights/recommendations/severity.d.ts +2 -0
- package/dist/insights/recommendations/severity.d.ts.map +1 -0
- package/dist/insights/recommendations/severity.js +3 -0
- package/dist/insights/recommendations/severity.js.map +1 -0
- package/dist/insights/rules/evaluate-rules.d.ts +5 -0
- package/dist/insights/rules/evaluate-rules.d.ts.map +1 -0
- package/dist/insights/rules/evaluate-rules.js +156 -0
- package/dist/insights/rules/evaluate-rules.js.map +1 -0
- package/dist/insights/types.d.ts +405 -0
- package/dist/insights/types.d.ts.map +1 -0
- package/dist/insights/types.js +3 -0
- package/dist/insights/types.js.map +1 -0
- package/dist/insights/utils/formatting.d.ts +4 -0
- package/dist/insights/utils/formatting.d.ts.map +1 -0
- package/dist/insights/utils/formatting.js +21 -0
- package/dist/insights/utils/formatting.js.map +1 -0
- package/dist/insights/utils/grouping.d.ts +2 -0
- package/dist/insights/utils/grouping.d.ts.map +1 -0
- package/dist/insights/utils/grouping.js +8 -0
- package/dist/insights/utils/grouping.js.map +1 -0
- package/dist/insights/utils/query-builder.d.ts +21 -0
- package/dist/insights/utils/query-builder.d.ts.map +1 -0
- package/dist/insights/utils/query-builder.js +107 -0
- package/dist/insights/utils/query-builder.js.map +1 -0
- package/dist/insights/utils/row-value.d.ts +5 -0
- package/dist/insights/utils/row-value.d.ts.map +1 -0
- package/dist/insights/utils/row-value.js +36 -0
- package/dist/insights/utils/row-value.js.map +1 -0
- package/dist/insights/utils/stats.d.ts +28 -0
- package/dist/insights/utils/stats.d.ts.map +1 -0
- package/dist/insights/utils/stats.js +125 -0
- package/dist/insights/utils/stats.js.map +1 -0
- package/dist/query/formatter.d.ts.map +1 -1
- package/dist/query/formatter.js +202 -24
- package/dist/query/formatter.js.map +1 -1
- package/dist/query/planner.d.ts +10 -2
- package/dist/query/planner.d.ts.map +1 -1
- package/dist/query/planner.js +39 -2
- package/dist/query/planner.js.map +1 -1
- package/dist/storage/memory.d.ts.map +1 -1
- package/dist/storage/memory.js +78 -4
- package/dist/storage/memory.js.map +1 -1
- package/dist/storage/mongo.d.ts.map +1 -1
- package/dist/storage/mongo.js +61 -16
- package/dist/storage/mongo.js.map +1 -1
- package/dist/storage/redis.d.ts +6 -7
- package/dist/storage/redis.d.ts.map +1 -1
- package/dist/storage/redis.js +108 -39
- package/dist/storage/redis.js.map +1 -1
- package/dist/types/index.d.ts +88 -4
- package/dist/types/index.d.ts.map +1 -1
- package/docs/INSIGHTS.md +59 -0
- package/docs/INSIGHTS_ANOMALY_DETECTION.md +33 -0
- package/docs/INSIGHTS_LLM_COST_AND_TOKENS.md +26 -0
- package/docs/INSIGHTS_RESOURCE_ADVISOR.md +31 -0
- package/docs/guides/README.md +29 -0
- package/docs/guides/adapter-memory.md +48 -0
- package/docs/guides/adapter-mongo.md +50 -0
- package/docs/guides/adapter-redis.md +47 -0
- package/docs/guides/backup-export.md +39 -0
- package/docs/guides/cardinality.md +50 -0
- package/docs/guides/cli.md +45 -0
- package/docs/guides/client-events.md +49 -0
- package/docs/guides/counter-maps.md +116 -0
- package/docs/guides/idempotency.md +39 -0
- package/docs/guides/ingestion-and-flush.md +67 -0
- package/docs/guides/insights-advisors-and-optimization.md +39 -0
- package/docs/guides/insights-anomalies-and-rules.md +48 -0
- package/docs/guides/insights-comparison-and-trends.md +47 -0
- package/docs/guides/insights-forecasting-and-budget.md +61 -0
- package/docs/guides/insights-overview.md +76 -0
- package/docs/guides/insights-ranking.md +81 -0
- package/docs/guides/measures.md +67 -0
- package/docs/guides/querying.md +79 -0
- package/docs/guides/timeframes-and-bucketing.md +55 -0
- package/docs/internals/README.md +20 -0
- package/docs/internals/internal-architecture-overview.md +65 -0
- package/docs/internals/internal-backup-stream.md +23 -0
- package/docs/internals/internal-buffer-merge-and-flush.md +51 -0
- package/docs/internals/internal-cardinality.md +30 -0
- package/docs/internals/internal-idempotency.md +36 -0
- package/docs/internals/internal-ingest-pipeline.md +52 -0
- package/docs/internals/internal-insights-layer.md +36 -0
- package/docs/internals/internal-mapping-validation-and-extraction.md +59 -0
- package/docs/internals/internal-measures-and-scales.md +35 -0
- package/docs/internals/internal-query-path.md +46 -0
- package/docs/internals/internal-storage-memory.md +13 -0
- package/docs/internals/internal-storage-mongo.md +24 -0
- package/docs/internals/internal-storage-redis.md +31 -0
- package/docs/internals/internal-timeframe-bucketing.md +33 -0
- package/package.json +45 -2
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
# Guide: Measures
|
|
2
|
+
|
|
3
|
+
**Measures** are optional numeric fields per event (cost, latency, tokens, …) aggregated alongside counts.
|
|
4
|
+
|
|
5
|
+
## Step 1: Declare measures on the map
|
|
6
|
+
|
|
7
|
+
```ts
|
|
8
|
+
const map = {
|
|
9
|
+
counterId: "llm_usage",
|
|
10
|
+
scope: { path: "$.scope.id" },
|
|
11
|
+
eventTime: { path: "$.request.createdAt" },
|
|
12
|
+
subjects: [{ type: "app", path: "$.app.id", required: true }],
|
|
13
|
+
metrics: [{ path: "$.request.status", required: true }],
|
|
14
|
+
dimensions: [{ name: "model", path: "$.model.name" }],
|
|
15
|
+
delta: { default: 1 },
|
|
16
|
+
timeframes: ["day"],
|
|
17
|
+
measures: [
|
|
18
|
+
{ name: "cost", path: "$.usage.costCents", aggregations: ["sum"], scale: 100 },
|
|
19
|
+
{ name: "latencyMs", path: "$.usage.latencyMs", aggregations: ["sum", "count", "min", "max"] },
|
|
20
|
+
],
|
|
21
|
+
} as const;
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
### Aggregations
|
|
25
|
+
|
|
26
|
+
- **`sum`**, **`count`**, **`min`**, **`max`** are stored.
|
|
27
|
+
- **`avg`** is derived at query time when both **`sum`** and **`count`** exist for that measure.
|
|
28
|
+
|
|
29
|
+
### Scale
|
|
30
|
+
|
|
31
|
+
`scale` multiplies then truncates for fixed-point storage (for example cents → integer micros).
|
|
32
|
+
|
|
33
|
+
### Missing values
|
|
34
|
+
|
|
35
|
+
Use `onMissing: "skip"` (default) or `"drop"` / `"zero"` per measure.
|
|
36
|
+
|
|
37
|
+
## Step 2: Ingest as usual
|
|
38
|
+
|
|
39
|
+
Each fan-out increment carries the **same** measure snapshot for that event.
|
|
40
|
+
|
|
41
|
+
## Step 3: Query measures
|
|
42
|
+
|
|
43
|
+
```ts
|
|
44
|
+
const result = await countex.query({
|
|
45
|
+
scope: "org_1",
|
|
46
|
+
counterId: "llm_usage",
|
|
47
|
+
timeframe: { type: "day", from: "2026-05-08", to: "2026-05-08" },
|
|
48
|
+
groupBy: ["subject", "metric"],
|
|
49
|
+
measures: ["cost", "latencyMs"],
|
|
50
|
+
format: "table",
|
|
51
|
+
});
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Use `measures: "*"` to include all declared measures.
|
|
55
|
+
|
|
56
|
+
### Ordering
|
|
57
|
+
|
|
58
|
+
`orderBy` can target measure aggregations (for example sort by `latencyMs` max).
|
|
59
|
+
|
|
60
|
+
### Cache vs durable
|
|
61
|
+
|
|
62
|
+
Redis keeps **sum** and **count** for measures. **min** / **max** live on durable; the query planner skips cache when min/max are required.
|
|
63
|
+
|
|
64
|
+
## See also
|
|
65
|
+
|
|
66
|
+
- [Querying](./querying.md)
|
|
67
|
+
- Internal: [Measures & scales](../internals/internal-measures-and-scales.md)
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
# Guide: Querying
|
|
2
|
+
|
|
3
|
+
`countex.query(spec)` reads aggregated cells for a scope and counter, with optional filters and output shaping.
|
|
4
|
+
|
|
5
|
+
## Step 1: Minimal query
|
|
6
|
+
|
|
7
|
+
```ts
|
|
8
|
+
const result = await countex.query({
|
|
9
|
+
scope: "org_1",
|
|
10
|
+
counterId: "access_by_actor_label",
|
|
11
|
+
timeframe: { type: "day", from: "2026-05-01", to: "2026-05-07" },
|
|
12
|
+
});
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
## Step 2: Filter
|
|
16
|
+
|
|
17
|
+
```ts
|
|
18
|
+
const result = await countex.query({
|
|
19
|
+
scope: "org_1",
|
|
20
|
+
counterId: "access_by_actor_label",
|
|
21
|
+
timeframe: { type: "day", from: "2026-05-01", to: "2026-05-07" },
|
|
22
|
+
subject: { type: "actor", ids: [10001, 10002] },
|
|
23
|
+
metrics: [41, 42],
|
|
24
|
+
dimensions: { operation: ["read", "write"] },
|
|
25
|
+
});
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
Subject and metric ids are **numeric dictionary ids** resolved during ingest.
|
|
29
|
+
|
|
30
|
+
## Step 3: Group and format
|
|
31
|
+
|
|
32
|
+
`groupBy` accepts:
|
|
33
|
+
|
|
34
|
+
- `subject`, `subjectType`, `metric`, `timeframeKey`
|
|
35
|
+
- `dim:NAME` for a dimension
|
|
36
|
+
|
|
37
|
+
`format`:
|
|
38
|
+
|
|
39
|
+
- `cells` — raw-ish cell list
|
|
40
|
+
- `table` / `matrix` — aggregated rows
|
|
41
|
+
- `timeseries` — points per bucket (and optional extra grouping)
|
|
42
|
+
|
|
43
|
+
```ts
|
|
44
|
+
const result = await countex.query({
|
|
45
|
+
scope: "org_1",
|
|
46
|
+
counterId: "access_by_actor_label",
|
|
47
|
+
timeframe: { type: "day", from: "2026-05-01", to: "2026-05-07" },
|
|
48
|
+
groupBy: ["subject", "dim:operation"],
|
|
49
|
+
format: "table",
|
|
50
|
+
orderBy: "-count",
|
|
51
|
+
limit: 50,
|
|
52
|
+
});
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
## Step 4: Measures and rates
|
|
56
|
+
|
|
57
|
+
```ts
|
|
58
|
+
await countex.query({
|
|
59
|
+
// ...
|
|
60
|
+
measures: ["cost"],
|
|
61
|
+
orderBy: { measure: "cost", agg: "sum", dir: "desc" },
|
|
62
|
+
rate: { measure: "cost", agg: "sum" },
|
|
63
|
+
});
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Do not use `rate` with timeframe **`all`**.
|
|
67
|
+
|
|
68
|
+
## Step 5: Cache control
|
|
69
|
+
|
|
70
|
+
- `bypassCache: true` forces durable reads.
|
|
71
|
+
- `signal` accepts `AbortSignal` for timeouts.
|
|
72
|
+
|
|
73
|
+
Result includes `source`: `"cache"`, `"durable"`, or `"merged"`.
|
|
74
|
+
|
|
75
|
+
## See also
|
|
76
|
+
|
|
77
|
+
- [Measures](./measures.md)
|
|
78
|
+
- [Timeframes](./timeframes-and-bucketing.md)
|
|
79
|
+
- Internal: [Query path](../internals/internal-query-path.md)
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Guide: Timeframes and bucketing
|
|
2
|
+
|
|
3
|
+
Timeframes control **how event time is bucketed** when Countex writes cells. Queries later ask for a **range of bucket keys** (`from` / `to`).
|
|
4
|
+
|
|
5
|
+
## Step 1: Pick timeframe types on the map
|
|
6
|
+
|
|
7
|
+
```ts
|
|
8
|
+
const map = {
|
|
9
|
+
counterId: "events",
|
|
10
|
+
// ...
|
|
11
|
+
timeframes: ["minute", "hour", "day", "week", "month", "quarter", "year", "all"],
|
|
12
|
+
} as const;
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Only listed timeframes receive writes for each matching event.
|
|
16
|
+
|
|
17
|
+
## Step 2: Understand bucket keys (UTC)
|
|
18
|
+
|
|
19
|
+
| Type | Example key |
|
|
20
|
+
|------|----------------|
|
|
21
|
+
| minute | `2026-05-08T12:30` |
|
|
22
|
+
| hour | `2026-05-08T12` |
|
|
23
|
+
| day | `2026-05-08` |
|
|
24
|
+
| week | `2026-W19` (ISO week) |
|
|
25
|
+
| month | `2026-05` |
|
|
26
|
+
| quarter | `2026-Q2` |
|
|
27
|
+
| year | `2026` |
|
|
28
|
+
| all | `all` |
|
|
29
|
+
|
|
30
|
+
Buckets are derived from **`eventTime`**, not ingestion time, so late data lands in the correct historical bucket.
|
|
31
|
+
|
|
32
|
+
## Step 3: Query a range
|
|
33
|
+
|
|
34
|
+
```ts
|
|
35
|
+
const result = await countex.query({
|
|
36
|
+
scope: "org_1",
|
|
37
|
+
counterId: "events",
|
|
38
|
+
timeframe: { type: "day", from: "2026-05-01", to: "2026-05-08" },
|
|
39
|
+
format: "table",
|
|
40
|
+
});
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
## Step 4: Respect the bucket cap
|
|
44
|
+
|
|
45
|
+
Range expansion is capped at **10,000** buckets. Narrow the window or coarsen the timeframe if you hit limits.
|
|
46
|
+
|
|
47
|
+
## Step 5: Rates
|
|
48
|
+
|
|
49
|
+
`rate` in queries divides counts or measure sums by elapsed seconds in the queried window. **`rate` is invalid when timeframe type is `all`** (no meaningful duration).
|
|
50
|
+
|
|
51
|
+
## See also
|
|
52
|
+
|
|
53
|
+
- [Querying](./querying.md)
|
|
54
|
+
- [Measures](./measures.md)
|
|
55
|
+
- Internal: [Timeframe bucketing](../internals/internal-timeframe-bucketing.md)
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# Countex internals
|
|
2
|
+
|
|
3
|
+
Deep dives into how the engine executes ingest, flush, storage IO, and insights post-processing. These documents are **not** imported as package subpaths; they ship in the npm tarball under `docs/internals/` for repository readers and advanced operators.
|
|
4
|
+
|
|
5
|
+
Start here:
|
|
6
|
+
|
|
7
|
+
| Topic | Document |
|
|
8
|
+
|-------|----------|
|
|
9
|
+
| End-to-end architecture | [Architecture overview](./internal-architecture-overview.md) |
|
|
10
|
+
| Map startup validation & JSON extraction | [Mapping validation & extraction](./internal-mapping-validation-and-extraction.md) |
|
|
11
|
+
| `ingestBatch` phases | [Ingest pipeline](./internal-ingest-pipeline.md) |
|
|
12
|
+
| In-memory merge | [Buffer merge & flush](./internal-buffer-merge-and-flush.md) |
|
|
13
|
+
| Reads | [Query path](./internal-query-path.md) |
|
|
14
|
+
| Bucket keys | [Timeframe bucketing](./internal-timeframe-bucketing.md) |
|
|
15
|
+
| Measures | [Measures & scales](./internal-measures-and-scales.md) |
|
|
16
|
+
| Cardinality | [Cardinality guard](./internal-cardinality.md) |
|
|
17
|
+
| Dedupe | [Idempotency](./internal-idempotency.md) |
|
|
18
|
+
| Adapters | [Mongo](./internal-storage-mongo.md), [Redis](./internal-storage-redis.md), [Memory](./internal-storage-memory.md) |
|
|
19
|
+
| Ops helpers | [Backup streaming](./internal-backup-stream.md) |
|
|
20
|
+
| Insights | [Insights layer](./internal-insights-layer.md) |
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
# Internal: Architecture overview
|
|
2
|
+
|
|
3
|
+
Countex separates **projection** (events → sparse cells), **durability** (authoritative totals), **cache** (hot reads + dedupe), and **query shaping**.
|
|
4
|
+
|
|
5
|
+
## Component diagram
|
|
6
|
+
|
|
7
|
+
```mermaid
|
|
8
|
+
flowchart LR
|
|
9
|
+
subgraph app [Host application]
|
|
10
|
+
CE[Countex client]
|
|
11
|
+
end
|
|
12
|
+
subgraph core [Core engine]
|
|
13
|
+
MAP[CounterMap registry]
|
|
14
|
+
EXT[extractIncrements]
|
|
15
|
+
CARD[CardinalityGuard]
|
|
16
|
+
BUF[DeltaBuffer]
|
|
17
|
+
PLAN[executeQuery]
|
|
18
|
+
FMT[formatResult]
|
|
19
|
+
end
|
|
20
|
+
subgraph io [Storage]
|
|
21
|
+
DUR[DurableStorage]
|
|
22
|
+
CACHE[CacheStorage]
|
|
23
|
+
end
|
|
24
|
+
CE --> MAP
|
|
25
|
+
CE --> EXT
|
|
26
|
+
EXT --> CARD
|
|
27
|
+
CARD --> DUR
|
|
28
|
+
CARD --> BUF
|
|
29
|
+
BUF --> DUR
|
|
30
|
+
BUF --> CACHE
|
|
31
|
+
CE --> PLAN
|
|
32
|
+
PLAN --> CACHE
|
|
33
|
+
PLAN --> DUR
|
|
34
|
+
PLAN --> FMT
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
## Write trace (conceptual)
|
|
38
|
+
|
|
39
|
+
1. **Validate maps** at `createCountex` construction (`validateMaps`).
|
|
40
|
+
2. **Extract** raw increments per map/event (`extractIncrements`).
|
|
41
|
+
3. **Dedupe** optional (`cache.markEventSeen`).
|
|
42
|
+
4. **Cardinality** rewrite or drop (`CardinalityGuard.check`).
|
|
43
|
+
5. **Dictionary** allocate numeric ids (`durable.resolveIds`).
|
|
44
|
+
6. **Buffer** merge by hashed cell identity (`DeltaBuffer.add`).
|
|
45
|
+
7. **Flush** snapshot → `durable.applyDeltas` → best-effort `cache.applyDeltas`.
|
|
46
|
+
|
|
47
|
+
## Read trace (conceptual)
|
|
48
|
+
|
|
49
|
+
1. **Planner** tries cache when allowed (`cacheCanServe`).
|
|
50
|
+
2. On partial/miss, **durable.readCells**.
|
|
51
|
+
3. Optional **cache.warm** with durable rows.
|
|
52
|
+
4. **Formatter** applies `groupBy`, measures, rates, limits.
|
|
53
|
+
|
|
54
|
+
## Source anchors
|
|
55
|
+
|
|
56
|
+
- Client orchestration: `src/core/countex.ts`
|
|
57
|
+
- Extraction: `src/core/extraction.ts`
|
|
58
|
+
- Buffer: `src/core/buffer.ts`
|
|
59
|
+
- Planner: `src/query/planner.ts`
|
|
60
|
+
- Formatter: `src/query/formatter.ts`
|
|
61
|
+
|
|
62
|
+
## See also
|
|
63
|
+
|
|
64
|
+
- [Ingest pipeline](./internal-ingest-pipeline.md)
|
|
65
|
+
- [Query path](./internal-query-path.md)
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# Internal: Backup streaming
|
|
2
|
+
|
|
3
|
+
`exportCells` and `rebuildCache` (`src/backup/export.ts`) are thin wrappers over `durable.streamCells`:
|
|
4
|
+
|
|
5
|
+
```mermaid
|
|
6
|
+
flowchart LR
|
|
7
|
+
DUR[durable.streamCells filter] --> EXPORT[AsyncIterable CounterCell]
|
|
8
|
+
DUR --> BATCH[batch warm loop]
|
|
9
|
+
BATCH --> CACHE[cache.warm batches]
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
## Contract expectations
|
|
13
|
+
|
|
14
|
+
Implementations must honor:
|
|
15
|
+
|
|
16
|
+
- Scope/counter/timeframe bounding
|
|
17
|
+
- Optional `AbortSignal` propagation
|
|
18
|
+
|
|
19
|
+
`rebuildCache` batches to limit Redis/Mongo round trips during warm operations.
|
|
20
|
+
|
|
21
|
+
## See also
|
|
22
|
+
|
|
23
|
+
- [Mongo durable](./internal-storage-mongo.md)
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# Internal: Buffer merge & flush
|
|
2
|
+
|
|
3
|
+
## DeltaBuffer structure
|
|
4
|
+
|
|
5
|
+
`DeltaBuffer` (`src/core/buffer.ts`) stores:
|
|
6
|
+
|
|
7
|
+
- Map key: string from `cellKey(identity)`
|
|
8
|
+
- Value: `{ identity, delta, measures?, measureScales? }`
|
|
9
|
+
|
|
10
|
+
## Merge semantics
|
|
11
|
+
|
|
12
|
+
`add` merges:
|
|
13
|
+
|
|
14
|
+
- **Counts** — integer delta summed.
|
|
15
|
+
- **Measures** — per aggregation field:
|
|
16
|
+
- `sum`, `count` additive
|
|
17
|
+
- `min` / `max` rolling extrema
|
|
18
|
+
|
|
19
|
+
`measureScales` merge retains first-seen scale per measure name (consistent with map declaration).
|
|
20
|
+
|
|
21
|
+
## Flush lifecycle
|
|
22
|
+
|
|
23
|
+
```mermaid
|
|
24
|
+
stateDiagram-v2
|
|
25
|
+
[*] --> Idle
|
|
26
|
+
Idle --> Draining: flush() & buffer non-empty
|
|
27
|
+
Draining --> DurableWrite: snapshot taken
|
|
28
|
+
DurableWrite --> CacheWrite: durable OK
|
|
29
|
+
DurableWrite --> MergeBack: durable error
|
|
30
|
+
MergeBack --> Idle
|
|
31
|
+
CacheWrite --> Idle: cache success or logged warn
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
`flush()` returns the **same promise** while in-flight (`flushing` lock) so concurrent callers coalesce.
|
|
35
|
+
|
|
36
|
+
### Durable failure
|
|
37
|
+
|
|
38
|
+
Snapshot is **`mergeBack`** into the buffer so counts are not lost; `flush_error` emits.
|
|
39
|
+
|
|
40
|
+
### Cache failure
|
|
41
|
+
|
|
42
|
+
Logged as non-fatal — durable already committed.
|
|
43
|
+
|
|
44
|
+
## Event accounting
|
|
45
|
+
|
|
46
|
+
`noteEvent()` increments an event counter used solely for flush thresholds (reset implicitly when draining — payload uses duration only).
|
|
47
|
+
|
|
48
|
+
## See also
|
|
49
|
+
|
|
50
|
+
- [Ingest pipeline](./internal-ingest-pipeline.md)
|
|
51
|
+
- [Measures & scales](./internal-measures-and-scales.md)
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# Internal: Cardinality guard
|
|
2
|
+
|
|
3
|
+
`CardinalityGuard` (`src/core/cardinality.ts`) tracks distinct source keys per `(scopeId, kind, qualifier)` using in-memory `Set`s.
|
|
4
|
+
|
|
5
|
+
## Check algorithm
|
|
6
|
+
|
|
7
|
+
```mermaid
|
|
8
|
+
flowchart TD
|
|
9
|
+
IN[sourceKey] --> KEY[keyFor scope/kind/qualifier]
|
|
10
|
+
KEY --> SET{Set contains key?}
|
|
11
|
+
SET -->|yes| OK[return original or existing overflow mapping]
|
|
12
|
+
SET -->|no| LIM{count < limit?}
|
|
13
|
+
LIM -->|yes| ADD[add to Set -> return key]
|
|
14
|
+
LIM -->|no| POL{onLimit}
|
|
15
|
+
POL -->|overflow| OVF[emit overflow event -> return sentinel]
|
|
16
|
+
POL -->|drop| DROP[return null]
|
|
17
|
+
POL -->|error| ERR[throw CardinalityLimitError]
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
## Dictionary kinds
|
|
21
|
+
|
|
22
|
+
Kinds align with subject/metric/dimension value namespaces; dimension checks include the dimension **name** as qualifier.
|
|
23
|
+
|
|
24
|
+
## Approximation warning
|
|
25
|
+
|
|
26
|
+
Commentary in source notes multi-worker deployments see approximate counts — acceptable because the guard targets runaway growth, not strict quotas.
|
|
27
|
+
|
|
28
|
+
## See also
|
|
29
|
+
|
|
30
|
+
- [Ingest pipeline](./internal-ingest-pipeline.md)
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Internal: Idempotency
|
|
2
|
+
|
|
3
|
+
Idempotency spans three layers:
|
|
4
|
+
|
|
5
|
+
1. **Map config** — `idempotency.enabled` + `eventIdPath`.
|
|
6
|
+
2. **Extraction** — resolves optional `eventId` string (`src/core/extraction.ts`).
|
|
7
|
+
3. **Ingest gate** — parallel `cache.markEventSeen` before buffering (`src/core/countex.ts`).
|
|
8
|
+
|
|
9
|
+
## Sequence
|
|
10
|
+
|
|
11
|
+
```mermaid
|
|
12
|
+
sequenceDiagram
|
|
13
|
+
participant CX as Countex
|
|
14
|
+
participant EXT as extractIncrements
|
|
15
|
+
participant CACHE as CacheStorage
|
|
16
|
+
participant BUF as DeltaBuffer
|
|
17
|
+
CX->>EXT: increments + eventId
|
|
18
|
+
CX->>CACHE: markEventSeen(scope, counterId:eventId, ttl)
|
|
19
|
+
CACHE-->>CX: isNew boolean
|
|
20
|
+
alt duplicate
|
|
21
|
+
CX->>CX: emit duplicate
|
|
22
|
+
CX->>CX: zero increments for this map slice
|
|
23
|
+
else new
|
|
24
|
+
CX->>BUF: add deltas
|
|
25
|
+
end
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
Namespace **`counterId:`** prefix prevents collisions across maps sharing physical cache.
|
|
29
|
+
|
|
30
|
+
## Failure modes
|
|
31
|
+
|
|
32
|
+
Missing event ids produce warnings during extraction but do not automatically dedupe unless your logic skips ingest.
|
|
33
|
+
|
|
34
|
+
## See also
|
|
35
|
+
|
|
36
|
+
- [Ingest pipeline](./internal-ingest-pipeline.md)
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# Internal: Ingest pipeline (`ingestBatch`)
|
|
2
|
+
|
|
3
|
+
`Countex.ingest` delegates to `ingestBatch` (`src/core/countex.ts`). The implementation is intentionally phased so CPU work stays synchronous while IO stays bounded.
|
|
4
|
+
|
|
5
|
+
## Sequence diagram
|
|
6
|
+
|
|
7
|
+
```mermaid
|
|
8
|
+
sequenceDiagram
|
|
9
|
+
participant App
|
|
10
|
+
participant CX as Countex
|
|
11
|
+
participant EXT as extractIncrements
|
|
12
|
+
participant CACHE as CacheStorage
|
|
13
|
+
participant CARD as CardinalityGuard
|
|
14
|
+
participant DUR as DurableStorage
|
|
15
|
+
participant BUF as DeltaBuffer
|
|
16
|
+
App->>CX: ingestBatch(events)
|
|
17
|
+
loop each event × map
|
|
18
|
+
CX->>EXT: extractIncrements(map, ev)
|
|
19
|
+
EXT-->>CX: RawIncrement[]
|
|
20
|
+
end
|
|
21
|
+
CX->>CACHE: markEventSeen (optional, parallel)
|
|
22
|
+
CACHE-->>CX: boolean flags
|
|
23
|
+
CX->>CARD: check subject/metric/dimensions
|
|
24
|
+
CARD-->>CX: rewritten keys or drop
|
|
25
|
+
CX->>DUR: resolveIds(lookups)
|
|
26
|
+
DUR-->>CX: Map lookup → numeric id
|
|
27
|
+
loop each RawIncrement × timeframe
|
|
28
|
+
CX->>BUF: add(identity, delta, measures)
|
|
29
|
+
end
|
|
30
|
+
CX->>CX: maybe flush() on thresholds
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
## Phase notes
|
|
34
|
+
|
|
35
|
+
1. **Extract** — warnings for measures surface as `measure_invalid` events when meta is present.
|
|
36
|
+
2. **Dedupe** — `markEventSeen` receives `${counterId}:${eventId}` to isolate maps.
|
|
37
|
+
3. **Cardinality** — dimension iteration may drop with `dropped` reason `cardinality:dimension`.
|
|
38
|
+
4. **Dictionary** — only **subjects** and **metrics** resolve via `resolveIds`; dimension values remain literal on cells.
|
|
39
|
+
5. **Buffer key** — identity includes `dimensionsHash` (`src/utils/hash.ts`).
|
|
40
|
+
6. **Timeframe expansion** — `bucketKey(tf, eventTime)` invoked per configured timeframe.
|
|
41
|
+
|
|
42
|
+
## Flush coupling
|
|
43
|
+
|
|
44
|
+
Threshold triggers call `flush()`:
|
|
45
|
+
|
|
46
|
+
- `onOverflow: "block"` awaits flush (back-pressure).
|
|
47
|
+
- `"drop"` schedules flush without awaiting.
|
|
48
|
+
|
|
49
|
+
## See also
|
|
50
|
+
|
|
51
|
+
- [Buffer merge & flush](./internal-buffer-merge-and-flush.md)
|
|
52
|
+
- [Cardinality](./internal-cardinality.md)
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Internal: Insights layer
|
|
2
|
+
|
|
3
|
+
`CountexInsights` (`src/insights/insights-client.ts`) wraps an existing `Countex` instance.
|
|
4
|
+
|
|
5
|
+
## Pattern
|
|
6
|
+
|
|
7
|
+
```mermaid
|
|
8
|
+
flowchart LR
|
|
9
|
+
IN[Insight input] --> QB[baseToQuerySpec +\nparseMeasureSelector]
|
|
10
|
+
QB --> Q[countex.query]
|
|
11
|
+
Q --> POST[ranking / anomaly /\nforecast helpers]
|
|
12
|
+
POST --> OUT[typed result]
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Shared utilities:
|
|
16
|
+
|
|
17
|
+
- `utils/query-builder.ts` translates insight inputs → `QuerySpec`.
|
|
18
|
+
- Ranking helpers sort rows, compute totals, enforce `maxGroups`.
|
|
19
|
+
- Anomaly pipeline fetches **eval** + **baseline** timeseries concurrently (`detectAnomalies`).
|
|
20
|
+
- Forecast/budget/token modules emit recommendations and bubble events.
|
|
21
|
+
|
|
22
|
+
## Event bridge
|
|
23
|
+
|
|
24
|
+
Methods emit:
|
|
25
|
+
|
|
26
|
+
- `insight_query` — performance telemetry
|
|
27
|
+
- `anomaly`, `recommendation`, `insufficient_data` — operator-facing signals
|
|
28
|
+
|
|
29
|
+
## Resource advisor
|
|
30
|
+
|
|
31
|
+
`estimateResourcesFromActuals` bypasses normal query formatting and streams cells via `durable.streamCells`, requiring the durable reference injected at construction.
|
|
32
|
+
|
|
33
|
+
## See also
|
|
34
|
+
|
|
35
|
+
- [Query path](./internal-query-path.md)
|
|
36
|
+
- `docs/INSIGHTS.md` for product-level narrative
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# Internal: Mapping validation & extraction
|
|
2
|
+
|
|
3
|
+
## Startup validation
|
|
4
|
+
|
|
5
|
+
`validateMaps` runs once inside the `Countex` constructor (`src/core/mapping.ts`):
|
|
6
|
+
|
|
7
|
+
- Ensures unique `counterId` values.
|
|
8
|
+
- Parses every configured JSONPath with `parsePath`.
|
|
9
|
+
- Verifies required structural rules (non-empty subjects/metrics/timeframes, valid timeframe enums, measure aggregation enums, idempotency consistency).
|
|
10
|
+
|
|
11
|
+
Failures throw **`MappingError`** immediately.
|
|
12
|
+
|
|
13
|
+
## Extraction pipeline
|
|
14
|
+
|
|
15
|
+
`extractIncrements(map, event)` (`src/core/extraction.ts`) executes in order:
|
|
16
|
+
|
|
17
|
+
```mermaid
|
|
18
|
+
flowchart TD
|
|
19
|
+
A[extract scope] -->|missing| SKIP[skipped event]
|
|
20
|
+
A --> B[extract eventTime]
|
|
21
|
+
B -->|invalid| SKIP
|
|
22
|
+
B --> C[optional eventId for idempotency]
|
|
23
|
+
C --> D[resolve delta]
|
|
24
|
+
D -->|zero| SKIP
|
|
25
|
+
D --> E[fan-out subjects]
|
|
26
|
+
E -->|none| SKIP
|
|
27
|
+
E --> F[fan-out metrics]
|
|
28
|
+
F -->|none| SKIP
|
|
29
|
+
F --> G[dimensions]
|
|
30
|
+
G -->|required missing| SKIP
|
|
31
|
+
G --> H[measures]
|
|
32
|
+
H -->|drop policy| SKIP
|
|
33
|
+
H --> I[cross product increments]
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
### Fan-out
|
|
37
|
+
|
|
38
|
+
Nested loops build `RawIncrement` rows:
|
|
39
|
+
|
|
40
|
+
```
|
|
41
|
+
for subject in subjectFanout:
|
|
42
|
+
for metricKey in metricFanout:
|
|
43
|
+
push RawIncrement(... timeframes: map.timeframes ...)
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
Each increment carries **copies** of dimensions and measures for that Cartesian tuple.
|
|
47
|
+
|
|
48
|
+
### Measures
|
|
49
|
+
|
|
50
|
+
`extractMeasuresForEvent` coerces numbers, applies `scale` truncation rules, and maps declared aggregations into pending contributions via `rawMeasuresToPendingDelta` during buffering.
|
|
51
|
+
|
|
52
|
+
### JSON paths
|
|
53
|
+
|
|
54
|
+
`extract` (`src/utils/jsonpath.ts`) evaluates the supported subset: properties, indices, and `[*]` expansion.
|
|
55
|
+
|
|
56
|
+
## See also
|
|
57
|
+
|
|
58
|
+
- [Ingest pipeline](./internal-ingest-pipeline.md)
|
|
59
|
+
- [Measures & scales](./internal-measures-and-scales.md)
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# Internal: Measures & scales
|
|
2
|
+
|
|
3
|
+
## Effective aggregations
|
|
4
|
+
|
|
5
|
+
`effectiveMeasureAggregations` (`src/core/extraction.ts`) ensures **`count`** exists whenever **`sum`**, **`min`**, or **`max`** are present so averages remain computable later.
|
|
6
|
+
|
|
7
|
+
## Pending deltas
|
|
8
|
+
|
|
9
|
+
`rawMeasuresToPendingDelta` converts each extracted measure into:
|
|
10
|
+
|
|
11
|
+
- `sum` contribution
|
|
12
|
+
- `count` increment (`1` when enabled)
|
|
13
|
+
- `min` / `max` seeds
|
|
14
|
+
|
|
15
|
+
These mirror what storage adapters must merge atomically.
|
|
16
|
+
|
|
17
|
+
## Buffer merging
|
|
18
|
+
|
|
19
|
+
`mergeMeasureDelta` (`src/core/buffer.ts`) applies associative merges identical to eventual durable semantics.
|
|
20
|
+
|
|
21
|
+
## Scale metadata
|
|
22
|
+
|
|
23
|
+
Maps may declare `scale` per measure. During ingest, numeric extraction truncates after multiply; `measureScales` propagate into buffer entries so adapters interpret stored integers consistently.
|
|
24
|
+
|
|
25
|
+
## Query-time averages
|
|
26
|
+
|
|
27
|
+
Formatter / row builders derive **`avg = sum / count`** when both components exist.
|
|
28
|
+
|
|
29
|
+
## Cache limitation
|
|
30
|
+
|
|
31
|
+
Planner denies cache for queries needing min/max fidelity — see `queryNeedsMinMax` in `src/query/planner.ts`.
|
|
32
|
+
|
|
33
|
+
## See also
|
|
34
|
+
|
|
35
|
+
- [Query path](./internal-query-path.md)
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Internal: Query path
|
|
2
|
+
|
|
3
|
+
`Countex.query` delegates to `executeQuery` (`src/query/planner.ts`) with optional measure metadata from `buildMeasureAggIndex(map)`.
|
|
4
|
+
|
|
5
|
+
## Decision flow
|
|
6
|
+
|
|
7
|
+
```mermaid
|
|
8
|
+
flowchart TD
|
|
9
|
+
Q[QuerySpec] --> C{cache configured\nand not bypassed\nand cacheCanServe?}
|
|
10
|
+
C -->|yes| R1[cache.readCells]
|
|
11
|
+
R1 -->|coverage full| F[formatResult source=cache]
|
|
12
|
+
R1 -->|partial/miss| D[durable.readCells]
|
|
13
|
+
C -->|no| D
|
|
14
|
+
D --> W{cache && rows && not bypass?}
|
|
15
|
+
W -->|yes| WARM[cache.warm best-effort]
|
|
16
|
+
W -->|no| F2[formatResult source=merged or durable]
|
|
17
|
+
WARM --> F2
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
## Cache eligibility (`cacheCanServe`)
|
|
21
|
+
|
|
22
|
+
When the counter map is known, queries that require **min/max** measure accuracy skip cache:
|
|
23
|
+
|
|
24
|
+
- `orderBy` targeting min/max aggregation.
|
|
25
|
+
- Explicit measure list referencing min/max-maintained measures.
|
|
26
|
+
- `measures: "*"` while any declared measure tracks min/max.
|
|
27
|
+
|
|
28
|
+
This avoids stale extrema because Redis stores sum/count only for measures.
|
|
29
|
+
|
|
30
|
+
## Formatter responsibilities (`formatResult`)
|
|
31
|
+
|
|
32
|
+
`src/query/formatter.ts`:
|
|
33
|
+
|
|
34
|
+
- Groups cells by `groupBy` keys.
|
|
35
|
+
- Builds `table` / `matrix` / `timeseries` projections.
|
|
36
|
+
- Applies **`rate`** using `bucketDurationSeconds` except timeframe `all`.
|
|
37
|
+
- Sorts and truncates via `orderBy` / `limit`.
|
|
38
|
+
|
|
39
|
+
## Bucket expansion
|
|
40
|
+
|
|
41
|
+
Range queries call `bucketRange` (`src/core/timeframe.ts`) capped at **10,000** buckets.
|
|
42
|
+
|
|
43
|
+
## See also
|
|
44
|
+
|
|
45
|
+
- [Timeframe bucketing](./internal-timeframe-bucketing.md)
|
|
46
|
+
- [Measures & scales](./internal-measures-and-scales.md)
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# Internal: Memory adapters
|
|
2
|
+
|
|
3
|
+
`MemoryDurable` and `MemoryCache` (`src/storage/memory.ts`) mirror production contracts using `Map` structures:
|
|
4
|
+
|
|
5
|
+
- Dictionary resolution mimics monotonic ids.
|
|
6
|
+
- Cells keyed similarly to durable identity tuples.
|
|
7
|
+
- Cache implements simplified coverage semantics for tests.
|
|
8
|
+
|
|
9
|
+
Used heavily by unit tests (`tests/unit/*.test.ts`) because they require no external services.
|
|
10
|
+
|
|
11
|
+
## See also
|
|
12
|
+
|
|
13
|
+
- [Architecture overview](./internal-architecture-overview.md)
|