@x12i/countex 1.0.0 → 1.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (317) hide show
  1. package/README.md +215 -4
  2. package/dist/backup/export.d.ts +2 -0
  3. package/dist/backup/export.d.ts.map +1 -1
  4. package/dist/backup/export.js +2 -0
  5. package/dist/backup/export.js.map +1 -1
  6. package/dist/backup/manifest.d.ts +49 -0
  7. package/dist/backup/manifest.d.ts.map +1 -0
  8. package/dist/backup/manifest.js +103 -0
  9. package/dist/backup/manifest.js.map +1 -0
  10. package/dist/backup/snapshot-encoding.d.ts +8 -0
  11. package/dist/backup/snapshot-encoding.d.ts.map +1 -0
  12. package/dist/backup/snapshot-encoding.js +27 -0
  13. package/dist/backup/snapshot-encoding.js.map +1 -0
  14. package/dist/cli.d.ts +12 -0
  15. package/dist/cli.d.ts.map +1 -0
  16. package/dist/cli.js +206 -0
  17. package/dist/cli.js.map +1 -0
  18. package/dist/core/buffer.d.ts +4 -3
  19. package/dist/core/buffer.d.ts.map +1 -1
  20. package/dist/core/buffer.js +65 -9
  21. package/dist/core/buffer.js.map +1 -1
  22. package/dist/core/countex.d.ts +14 -1
  23. package/dist/core/countex.d.ts.map +1 -1
  24. package/dist/core/countex.js +171 -48
  25. package/dist/core/countex.js.map +1 -1
  26. package/dist/core/extraction.d.ts +14 -6
  27. package/dist/core/extraction.d.ts.map +1 -1
  28. package/dist/core/extraction.js +117 -1
  29. package/dist/core/extraction.js.map +1 -1
  30. package/dist/core/mapping.d.ts +5 -0
  31. package/dist/core/mapping.d.ts.map +1 -1
  32. package/dist/core/mapping.js +71 -1
  33. package/dist/core/mapping.js.map +1 -1
  34. package/dist/core/timeframe.d.ts +7 -0
  35. package/dist/core/timeframe.d.ts.map +1 -1
  36. package/dist/core/timeframe.js +88 -0
  37. package/dist/core/timeframe.js.map +1 -1
  38. package/dist/index.d.ts +4 -1
  39. package/dist/index.d.ts.map +1 -1
  40. package/dist/index.js +13 -1
  41. package/dist/index.js.map +1 -1
  42. package/dist/insights/advisors/fanout-health.d.ts +3 -0
  43. package/dist/insights/advisors/fanout-health.d.ts.map +1 -0
  44. package/dist/insights/advisors/fanout-health.js +3 -0
  45. package/dist/insights/advisors/fanout-health.js.map +1 -0
  46. package/dist/insights/advisors/map-health.d.ts +6 -0
  47. package/dist/insights/advisors/map-health.d.ts.map +1 -0
  48. package/dist/insights/advisors/map-health.js +154 -0
  49. package/dist/insights/advisors/map-health.js.map +1 -0
  50. package/dist/insights/advisors/resource-advisor.d.ts +4 -0
  51. package/dist/insights/advisors/resource-advisor.d.ts.map +1 -0
  52. package/dist/insights/advisors/resource-advisor.js +102 -0
  53. package/dist/insights/advisors/resource-advisor.js.map +1 -0
  54. package/dist/insights/advisors/retention-advisor.d.ts +2 -0
  55. package/dist/insights/advisors/retention-advisor.d.ts.map +1 -0
  56. package/dist/insights/advisors/retention-advisor.js +6 -0
  57. package/dist/insights/advisors/retention-advisor.js.map +1 -0
  58. package/dist/insights/advisors-subpath.d.ts +4 -0
  59. package/dist/insights/advisors-subpath.d.ts.map +1 -0
  60. package/dist/insights/advisors-subpath.js +10 -0
  61. package/dist/insights/advisors-subpath.js.map +1 -0
  62. package/dist/insights/anomaly/baseline.d.ts +4 -0
  63. package/dist/insights/anomaly/baseline.d.ts.map +1 -0
  64. package/dist/insights/anomaly/baseline.js +54 -0
  65. package/dist/insights/anomaly/baseline.js.map +1 -0
  66. package/dist/insights/anomaly/detect-anomalies.d.ts +6 -0
  67. package/dist/insights/anomaly/detect-anomalies.d.ts.map +1 -0
  68. package/dist/insights/anomaly/detect-anomalies.js +245 -0
  69. package/dist/insights/anomaly/detect-anomalies.js.map +1 -0
  70. package/dist/insights/anomaly/ewma.d.ts +2 -0
  71. package/dist/insights/anomaly/ewma.d.ts.map +1 -0
  72. package/dist/insights/anomaly/ewma.js +15 -0
  73. package/dist/insights/anomaly/ewma.js.map +1 -0
  74. package/dist/insights/anomaly/missing-activity.d.ts +3 -0
  75. package/dist/insights/anomaly/missing-activity.d.ts.map +1 -0
  76. package/dist/insights/anomaly/missing-activity.js +8 -0
  77. package/dist/insights/anomaly/missing-activity.js.map +1 -0
  78. package/dist/insights/anomaly/robust-zscore.d.ts +2 -0
  79. package/dist/insights/anomaly/robust-zscore.d.ts.map +1 -0
  80. package/dist/insights/anomaly/robust-zscore.js +9 -0
  81. package/dist/insights/anomaly/robust-zscore.js.map +1 -0
  82. package/dist/insights/anomaly-subpath.d.ts +7 -0
  83. package/dist/insights/anomaly-subpath.d.ts.map +1 -0
  84. package/dist/insights/anomaly-subpath.js +20 -0
  85. package/dist/insights/anomaly-subpath.js.map +1 -0
  86. package/dist/insights/comparison/compare-periods.d.ts +5 -0
  87. package/dist/insights/comparison/compare-periods.d.ts.map +1 -0
  88. package/dist/insights/comparison/compare-periods.js +64 -0
  89. package/dist/insights/comparison/compare-periods.js.map +1 -0
  90. package/dist/insights/comparison/growth-rate.d.ts +3 -0
  91. package/dist/insights/comparison/growth-rate.d.ts.map +1 -0
  92. package/dist/insights/comparison/growth-rate.js +3 -0
  93. package/dist/insights/comparison/growth-rate.js.map +1 -0
  94. package/dist/insights/comparison/moving-average.d.ts +2 -0
  95. package/dist/insights/comparison/moving-average.d.ts.map +1 -0
  96. package/dist/insights/comparison/moving-average.js +6 -0
  97. package/dist/insights/comparison/moving-average.js.map +1 -0
  98. package/dist/insights/comparison/trend.d.ts +5 -0
  99. package/dist/insights/comparison/trend.d.ts.map +1 -0
  100. package/dist/insights/comparison/trend.js +84 -0
  101. package/dist/insights/comparison/trend.js.map +1 -0
  102. package/dist/insights/errors.d.ts +25 -0
  103. package/dist/insights/errors.d.ts.map +1 -0
  104. package/dist/insights/errors.js +54 -0
  105. package/dist/insights/errors.js.map +1 -0
  106. package/dist/insights/forecast/budget.d.ts +2 -0
  107. package/dist/insights/forecast/budget.d.ts.map +1 -0
  108. package/dist/insights/forecast/budget.js +6 -0
  109. package/dist/insights/forecast/budget.js.map +1 -0
  110. package/dist/insights/forecast/capacity.d.ts +2 -0
  111. package/dist/insights/forecast/capacity.d.ts.map +1 -0
  112. package/dist/insights/forecast/capacity.js +3 -0
  113. package/dist/insights/forecast/capacity.js.map +1 -0
  114. package/dist/insights/forecast/forecast.d.ts +5 -0
  115. package/dist/insights/forecast/forecast.d.ts.map +1 -0
  116. package/dist/insights/forecast/forecast.js +103 -0
  117. package/dist/insights/forecast/forecast.js.map +1 -0
  118. package/dist/insights/forecast/retention.d.ts +2 -0
  119. package/dist/insights/forecast/retention.d.ts.map +1 -0
  120. package/dist/insights/forecast/retention.js +3 -0
  121. package/dist/insights/forecast/retention.js.map +1 -0
  122. package/dist/insights/index.d.ts +4 -0
  123. package/dist/insights/index.d.ts.map +1 -0
  124. package/dist/insights/index.js +23 -0
  125. package/dist/insights/index.js.map +1 -0
  126. package/dist/insights/insights-client.d.ts +33 -0
  127. package/dist/insights/insights-client.d.ts.map +1 -0
  128. package/dist/insights/insights-client.js +344 -0
  129. package/dist/insights/insights-client.js.map +1 -0
  130. package/dist/insights/llm-subpath.d.ts +4 -0
  131. package/dist/insights/llm-subpath.d.ts.map +1 -0
  132. package/dist/insights/llm-subpath.js +10 -0
  133. package/dist/insights/llm-subpath.js.map +1 -0
  134. package/dist/insights/optimization/cost-optimizer.d.ts +6 -0
  135. package/dist/insights/optimization/cost-optimizer.d.ts.map +1 -0
  136. package/dist/insights/optimization/cost-optimizer.js +98 -0
  137. package/dist/insights/optimization/cost-optimizer.js.map +1 -0
  138. package/dist/insights/optimization/latency-optimizer.d.ts +3 -0
  139. package/dist/insights/optimization/latency-optimizer.d.ts.map +1 -0
  140. package/dist/insights/optimization/latency-optimizer.js +3 -0
  141. package/dist/insights/optimization/latency-optimizer.js.map +1 -0
  142. package/dist/insights/optimization/threshold-advisor.d.ts +2 -0
  143. package/dist/insights/optimization/threshold-advisor.d.ts.map +1 -0
  144. package/dist/insights/optimization/threshold-advisor.js +3 -0
  145. package/dist/insights/optimization/threshold-advisor.js.map +1 -0
  146. package/dist/insights/optimization/token-optimizer.d.ts +4 -0
  147. package/dist/insights/optimization/token-optimizer.d.ts.map +1 -0
  148. package/dist/insights/optimization/token-optimizer.js +133 -0
  149. package/dist/insights/optimization/token-optimizer.js.map +1 -0
  150. package/dist/insights/ranking/contribution.d.ts +2 -0
  151. package/dist/insights/ranking/contribution.d.ts.map +1 -0
  152. package/dist/insights/ranking/contribution.js +6 -0
  153. package/dist/insights/ranking/contribution.js.map +1 -0
  154. package/dist/insights/ranking/helpers.d.ts +22 -0
  155. package/dist/insights/ranking/helpers.d.ts.map +1 -0
  156. package/dist/insights/ranking/helpers.js +88 -0
  157. package/dist/insights/ranking/helpers.js.map +1 -0
  158. package/dist/insights/ranking/pareto.d.ts +2 -0
  159. package/dist/insights/ranking/pareto.d.ts.map +1 -0
  160. package/dist/insights/ranking/pareto.js +6 -0
  161. package/dist/insights/ranking/pareto.js.map +1 -0
  162. package/dist/insights/ranking/slices.d.ts +2 -0
  163. package/dist/insights/ranking/slices.d.ts.map +1 -0
  164. package/dist/insights/ranking/slices.js +6 -0
  165. package/dist/insights/ranking/slices.js.map +1 -0
  166. package/dist/insights/ranking/top-bottom.d.ts +2 -0
  167. package/dist/insights/ranking/top-bottom.d.ts.map +1 -0
  168. package/dist/insights/ranking/top-bottom.js +7 -0
  169. package/dist/insights/ranking/top-bottom.js.map +1 -0
  170. package/dist/insights/recommendations/evidence.d.ts +2 -0
  171. package/dist/insights/recommendations/evidence.d.ts.map +1 -0
  172. package/dist/insights/recommendations/evidence.js +3 -0
  173. package/dist/insights/recommendations/evidence.js.map +1 -0
  174. package/dist/insights/recommendations/recommendation.d.ts +6 -0
  175. package/dist/insights/recommendations/recommendation.d.ts.map +1 -0
  176. package/dist/insights/recommendations/recommendation.js +25 -0
  177. package/dist/insights/recommendations/recommendation.js.map +1 -0
  178. package/dist/insights/recommendations/severity.d.ts +2 -0
  179. package/dist/insights/recommendations/severity.d.ts.map +1 -0
  180. package/dist/insights/recommendations/severity.js +3 -0
  181. package/dist/insights/recommendations/severity.js.map +1 -0
  182. package/dist/insights/rules/evaluate-rules.d.ts +5 -0
  183. package/dist/insights/rules/evaluate-rules.d.ts.map +1 -0
  184. package/dist/insights/rules/evaluate-rules.js +156 -0
  185. package/dist/insights/rules/evaluate-rules.js.map +1 -0
  186. package/dist/insights/types.d.ts +405 -0
  187. package/dist/insights/types.d.ts.map +1 -0
  188. package/dist/insights/types.js +3 -0
  189. package/dist/insights/types.js.map +1 -0
  190. package/dist/insights/utils/formatting.d.ts +4 -0
  191. package/dist/insights/utils/formatting.d.ts.map +1 -0
  192. package/dist/insights/utils/formatting.js +21 -0
  193. package/dist/insights/utils/formatting.js.map +1 -0
  194. package/dist/insights/utils/grouping.d.ts +2 -0
  195. package/dist/insights/utils/grouping.d.ts.map +1 -0
  196. package/dist/insights/utils/grouping.js +8 -0
  197. package/dist/insights/utils/grouping.js.map +1 -0
  198. package/dist/insights/utils/query-builder.d.ts +21 -0
  199. package/dist/insights/utils/query-builder.d.ts.map +1 -0
  200. package/dist/insights/utils/query-builder.js +107 -0
  201. package/dist/insights/utils/query-builder.js.map +1 -0
  202. package/dist/insights/utils/row-value.d.ts +5 -0
  203. package/dist/insights/utils/row-value.d.ts.map +1 -0
  204. package/dist/insights/utils/row-value.js +36 -0
  205. package/dist/insights/utils/row-value.js.map +1 -0
  206. package/dist/insights/utils/stats.d.ts +28 -0
  207. package/dist/insights/utils/stats.d.ts.map +1 -0
  208. package/dist/insights/utils/stats.js +125 -0
  209. package/dist/insights/utils/stats.js.map +1 -0
  210. package/dist/query/formatter.d.ts +2 -2
  211. package/dist/query/formatter.d.ts.map +1 -1
  212. package/dist/query/formatter.js +211 -30
  213. package/dist/query/formatter.js.map +1 -1
  214. package/dist/query/planner.d.ts +10 -2
  215. package/dist/query/planner.d.ts.map +1 -1
  216. package/dist/query/planner.js +55 -4
  217. package/dist/query/planner.js.map +1 -1
  218. package/dist/simulation/context.d.ts +5 -0
  219. package/dist/simulation/context.d.ts.map +1 -0
  220. package/dist/simulation/context.js +56 -0
  221. package/dist/simulation/context.js.map +1 -0
  222. package/dist/simulation/events.d.ts +3 -0
  223. package/dist/simulation/events.d.ts.map +1 -0
  224. package/dist/simulation/events.js +67 -0
  225. package/dist/simulation/events.js.map +1 -0
  226. package/dist/simulation/index.d.ts +7 -0
  227. package/dist/simulation/index.d.ts.map +1 -0
  228. package/dist/simulation/index.js +16 -0
  229. package/dist/simulation/index.js.map +1 -0
  230. package/dist/simulation/prng.d.ts +4 -0
  231. package/dist/simulation/prng.d.ts.map +1 -0
  232. package/dist/simulation/prng.js +18 -0
  233. package/dist/simulation/prng.js.map +1 -0
  234. package/dist/simulation/resource-estimate.d.ts +6 -0
  235. package/dist/simulation/resource-estimate.d.ts.map +1 -0
  236. package/dist/simulation/resource-estimate.js +11 -0
  237. package/dist/simulation/resource-estimate.js.map +1 -0
  238. package/dist/simulation/simulator.d.ts +15 -0
  239. package/dist/simulation/simulator.d.ts.map +1 -0
  240. package/dist/simulation/simulator.js +67 -0
  241. package/dist/simulation/simulator.js.map +1 -0
  242. package/dist/simulation/tracker.d.ts +8 -0
  243. package/dist/simulation/tracker.d.ts.map +1 -0
  244. package/dist/simulation/tracker.js +18 -0
  245. package/dist/simulation/tracker.js.map +1 -0
  246. package/dist/simulation/types.d.ts +49 -0
  247. package/dist/simulation/types.d.ts.map +1 -0
  248. package/dist/simulation/types.js +21 -0
  249. package/dist/simulation/types.js.map +1 -0
  250. package/dist/storage/memory.d.ts +3 -0
  251. package/dist/storage/memory.d.ts.map +1 -1
  252. package/dist/storage/memory.js +113 -4
  253. package/dist/storage/memory.js.map +1 -1
  254. package/dist/storage/mongo.d.ts +5 -0
  255. package/dist/storage/mongo.d.ts.map +1 -1
  256. package/dist/storage/mongo.js +129 -17
  257. package/dist/storage/mongo.js.map +1 -1
  258. package/dist/storage/redis.d.ts +12 -9
  259. package/dist/storage/redis.d.ts.map +1 -1
  260. package/dist/storage/redis.js +126 -47
  261. package/dist/storage/redis.js.map +1 -1
  262. package/dist/types/index.d.ts +139 -4
  263. package/dist/types/index.d.ts.map +1 -1
  264. package/docs/DEPLOYMENT.md +40 -2
  265. package/docs/INSIGHTS.md +59 -0
  266. package/docs/INSIGHTS_ANOMALY_DETECTION.md +33 -0
  267. package/docs/INSIGHTS_LLM_COST_AND_TOKENS.md +26 -0
  268. package/docs/INSIGHTS_RESOURCE_ADVISOR.md +31 -0
  269. package/docs/countex_gap_analysis_complete_checklist.md +1340 -0
  270. package/docs/guides/README.md +34 -0
  271. package/docs/guides/adapter-memory.md +48 -0
  272. package/docs/guides/adapter-mongo.md +62 -0
  273. package/docs/guides/adapter-redis.md +53 -0
  274. package/docs/guides/backup-export.md +83 -0
  275. package/docs/guides/cardinality.md +50 -0
  276. package/docs/guides/cli.md +53 -0
  277. package/docs/guides/client-events.md +49 -0
  278. package/docs/guides/counter-maps.md +122 -0
  279. package/docs/guides/idempotency.md +39 -0
  280. package/docs/guides/ingestion-and-flush.md +67 -0
  281. package/docs/guides/insights-advisors-and-optimization.md +39 -0
  282. package/docs/guides/insights-anomalies-and-rules.md +48 -0
  283. package/docs/guides/insights-comparison-and-trends.md +47 -0
  284. package/docs/guides/insights-forecasting-and-budget.md +61 -0
  285. package/docs/guides/insights-overview.md +82 -0
  286. package/docs/guides/insights-ranking.md +81 -0
  287. package/docs/guides/measures.md +71 -0
  288. package/docs/guides/querying.md +119 -0
  289. package/docs/guides/simulation-profiles.md +33 -0
  290. package/docs/guides/simulation-resource-estimation.md +30 -0
  291. package/docs/guides/simulation.md +36 -0
  292. package/docs/guides/timeframes-and-bucketing.md +55 -0
  293. package/docs/internals/README.md +21 -0
  294. package/docs/internals/internal-architecture-overview.md +65 -0
  295. package/docs/internals/internal-backup-stream.md +23 -0
  296. package/docs/internals/internal-buffer-merge-and-flush.md +51 -0
  297. package/docs/internals/internal-cardinality.md +30 -0
  298. package/docs/internals/internal-dimension-hash.md +34 -0
  299. package/docs/internals/internal-idempotency.md +36 -0
  300. package/docs/internals/internal-ingest-pipeline.md +52 -0
  301. package/docs/internals/internal-insights-layer.md +36 -0
  302. package/docs/internals/internal-mapping-validation-and-extraction.md +59 -0
  303. package/docs/internals/internal-measures-and-scales.md +35 -0
  304. package/docs/internals/internal-query-path.md +46 -0
  305. package/docs/internals/internal-storage-memory.md +13 -0
  306. package/docs/internals/internal-storage-mongo.md +24 -0
  307. package/docs/internals/internal-storage-redis.md +31 -0
  308. package/docs/internals/internal-timeframe-bucketing.md +33 -0
  309. package/docs/use-cases/README.md +56 -0
  310. package/docs/use-cases/audit-security-log-counts.md +550 -0
  311. package/docs/use-cases/llm-cost-and-token-metering.md +255 -0
  312. package/docs/use-cases/multi-tenant-isolation.md +203 -0
  313. package/docs/use-cases/ops-dashboard-daily-rollups.md +255 -0
  314. package/docs/use-cases/processing-workflow-activity.md +165 -0
  315. package/docs/use-cases/product-feature-usage.md +240 -0
  316. package/docs/use-cases/webhook-idempotent-ingestion.md +188 -0
  317. package/package.json +61 -2
@@ -0,0 +1,39 @@
1
+ # Guide: Insights — advisors and resource estimation
2
+
3
+ Advisors inspect **map shape**, **fan-out health**, and optionally **stream durable cells** to estimate capacity-style metrics.
4
+
5
+ ## Step 1: Map health
6
+
7
+ Requires `mapsByCounterId` passed to `createCountexInsights`.
8
+
9
+ ```ts
10
+ const health = await insights.analyzeMapHealth({
11
+ scope: "org_1",
12
+ counterId: "usage",
13
+ timeframe: { type: "day", from: "2026-05-08", to: "2026-05-08" },
14
+ });
15
+ ```
16
+
17
+ ## Step 2: Resource estimation from actuals
18
+
19
+ Requires the **`durable`** reference in `createCountexInsights`.
20
+
21
+ ```ts
22
+ const resources = await insights.estimateResourcesFromActuals({
23
+ scope: "org_1",
24
+ counterId: "usage",
25
+ timeframe: { type: "day", from: "2026-05-08", to: "2026-05-08" },
26
+ });
27
+ ```
28
+
29
+ This path uses `durable.streamCells` — ensure your adapter implements streaming efficiently.
30
+
31
+ ## Events
32
+
33
+ Recommendations attach to `recommendation` emissions with severity for downstream routing (pager, ticket, etc.).
34
+
35
+ ## See also
36
+
37
+ - `docs/INSIGHTS_RESOURCE_ADVISOR.md`
38
+ - [Insights overview](./insights-overview.md)
39
+ - Internal: [Insights layer](../internals/internal-insights-layer.md)
@@ -0,0 +1,48 @@
1
+ # Guide: Insights — anomalies and rules
2
+
3
+ Detect unusual periods relative to a baseline and evaluate declarative **rules** over stored aggregates.
4
+
5
+ ## Step 1: Detect anomalies
6
+
7
+ ```ts
8
+ const { anomalies } = await insights.detectAnomalies({
9
+ scope: "org_1",
10
+ counterId: "usage",
11
+ timeframe: { type: "day", from: "2026-05-01", to: "2026-05-14" },
12
+ groupBy: [],
13
+ measure: "count",
14
+ baseline: {
15
+ timeframe: { type: "day", from: "2026-04-01", to: "2026-04-30" },
16
+ minBuckets: 12,
17
+ },
18
+ });
19
+ ```
20
+
21
+ Notes:
22
+
23
+ - **`timeframe.type: "all"`** is not supported; you’ll receive an empty list and an `insufficient_data` event.
24
+ - Emits **`anomaly`** events with severity for each finding.
25
+
26
+ ## Step 2: Evaluate rules
27
+
28
+ ```ts
29
+ const rules = await insights.evaluateRules({
30
+ scope: "org_1",
31
+ counterId: "usage",
32
+ timeframe: { type: "week", from: "2026-W18", to: "2026-W19" },
33
+ rules: [
34
+ {
35
+ id: "spike_cost",
36
+ when: { measure: "cost.sum", op: "gt", value: 10_000 },
37
+ recommend: { title: "Investigate cost spike" },
38
+ },
39
+ ],
40
+ });
41
+ ```
42
+
43
+ Rule evaluation reuses the insights query helper so timeouts and group caps stay consistent.
44
+
45
+ ## See also
46
+
47
+ - `docs/INSIGHTS_ANOMALY_DETECTION.md`
48
+ - Internal: [Insights layer](../internals/internal-insights-layer.md)
@@ -0,0 +1,47 @@
1
+ # Guide: Insights — comparison and trends
2
+
3
+ Compare two time windows or derive simple **trend** statistics from a timeseries query.
4
+
5
+ ## Step 1: Compare periods
6
+
7
+ ```ts
8
+ const cmp = await insights.comparePeriods({
9
+ scope: "org_1",
10
+ counterId: "usage",
11
+ current: { type: "day", from: "2026-05-08", to: "2026-05-08" },
12
+ previous: { type: "day", from: "2026-05-07", to: "2026-05-07" },
13
+ groupBy: ["metric"],
14
+ measure: "count",
15
+ minPercentChange: 10,
16
+ minAbsoluteChange: 5,
17
+ });
18
+ ```
19
+
20
+ The helper issues **two** queries (current and previous) and merges rows by group keys.
21
+
22
+ ## Step 2: Trend from timeseries
23
+
24
+ ```ts
25
+ const trend = await insights.trend({
26
+ scope: "org_1",
27
+ counterId: "usage",
28
+ timeframe: { type: "day", from: "2026-05-01", to: "2026-05-14" },
29
+ groupBy: [],
30
+ measure: "cost.sum",
31
+ method: "linear",
32
+ movingAverageWindow: 3,
33
+ });
34
+ ```
35
+
36
+ Underlying query uses `format: "timeseries"`; post-processing computes growth / smoothing per configuration.
37
+
38
+ ## Options shared with other insights
39
+
40
+ - `filters` — subject/metric/dimension filters compatible with core queries
41
+ - `bypassCache`, `signal`
42
+ - `maxBuckets`, `maxGroups`
43
+
44
+ ## See also
45
+
46
+ - [Timeframes](./timeframes-and-bucketing.md)
47
+ - Internal: [Insights layer](../internals/internal-insights-layer.md)
@@ -0,0 +1,61 @@
1
+ # Guide: Insights — forecasting and budget
2
+
3
+ Forecast simple forward curves and relate measures to **budgets**, **cost drivers**, and **token ceilings**.
4
+
5
+ ## Step 1: Forecast
6
+
7
+ ```ts
8
+ const fc = await insights.forecast({
9
+ scope: "org_1",
10
+ counterId: "usage",
11
+ timeframe: { type: "day", from: "2026-05-01", to: "2026-05-14" },
12
+ measure: "count",
13
+ horizonBuckets: 7,
14
+ });
15
+ ```
16
+
17
+ Recommendations may emit `recommendation` events on the insights client.
18
+
19
+ ## Step 2: Budget projection
20
+
21
+ ```ts
22
+ const budget = await insights.budgetProjection({
23
+ scope: "org_1",
24
+ counterId: "usage",
25
+ timeframe: { type: "month", from: "2026-05", to: "2026-05" },
26
+ measure: "cost.sum",
27
+ budget: 50_000,
28
+ });
29
+ ```
30
+
31
+ ## Step 3: Cost drivers
32
+
33
+ ```ts
34
+ const drivers = await insights.costDrivers({
35
+ scope: "org_1",
36
+ counterId: "usage",
37
+ timeframe: { type: "month", from: "2026-05", to: "2026-05" },
38
+ groupBy: ["subject", "dim:model"],
39
+ measure: "cost.sum",
40
+ });
41
+ ```
42
+
43
+ ## Step 4: Token recommendations
44
+
45
+ ```ts
46
+ const tokens = await insights.recommendMaxTokens({
47
+ scope: "org_1",
48
+ counterId: "usage",
49
+ timeframe: { type: "day", from: "2026-05-08", to: "2026-05-08" },
50
+ groupBy: ["dim:model"],
51
+ outputTokenMeasure: "outputTokens",
52
+ safetyFactor: 1.2,
53
+ });
54
+ ```
55
+
56
+ Measure names must exist on your map / cells.
57
+
58
+ ## See also
59
+
60
+ - `docs/INSIGHTS_LLM_COST_AND_TOKENS.md`
61
+ - Internal: [Insights layer](../internals/internal-insights-layer.md)
@@ -0,0 +1,82 @@
1
+ # Guide: Insights overview
2
+
3
+ The **`@x12i/countex/insights`** module layers ranking, comparison, anomaly detection, forecasting, and advisory helpers on top of ordinary `countex.query` results.
4
+
5
+ ## Step 1: Install / import
6
+
7
+ ```ts
8
+ import { createCountexInsights } from "@x12i/countex/insights";
9
+ ```
10
+
11
+ No extra peer dependencies beyond what your core Countex setup already uses.
12
+
13
+ ## Step 2: Construct insights with your Countex client
14
+
15
+ ```ts
16
+ const insights = createCountexInsights({
17
+ countex,
18
+ durable, // optional; required only for estimateResourcesFromActuals
19
+ mapsByCounterId: new Map(maps.map((m) => [m.counterId, m])),
20
+ defaults: {
21
+ maxGroups: 10_000,
22
+ maxBuckets: 10_000,
23
+ queryTimeoutMs: 10_000,
24
+ minBaselineBuckets: 12,
25
+ resultLimit: 1_000,
26
+ },
27
+ });
28
+ ```
29
+
30
+ Passing **`mapsByCounterId`** enables map-aware advisors (for example map health).
31
+
32
+ ## Step 3: Run an insight
33
+
34
+ Each method ultimately builds a `QuerySpec`, calls `client.query`, then post-processes rows.
35
+
36
+ ```ts
37
+ const top = await insights.topN({
38
+ scope: "org_1",
39
+ counterId: "llm_usage",
40
+ timeframe: { type: "month", from: "2026-05", to: "2026-05" },
41
+ groupBy: ["subject"],
42
+ measure: "cost.sum",
43
+ n: 10,
44
+ });
45
+ ```
46
+
47
+ ## Step 4: Observe insight lifecycle events
48
+
49
+ `CountexInsights` emits:
50
+
51
+ - `insight_query` — timing per underlying query
52
+ - `anomaly`, `recommendation`, `insufficient_data` — depending on method
53
+
54
+ ```ts
55
+ insights.on("insight_query", (e) => console.log(e));
56
+ ```
57
+
58
+ ## Step 5: Respect limits
59
+
60
+ Insights honor **`maxBuckets`** (aligned with core query expansion) and **`maxGroups`**. Prefer narrower filters when grouping heavily.
61
+
62
+ ## Distributions, percentiles, and histograms
63
+
64
+ Core Countex cells store **additive** measure aggregates (`sum`, `count`, `min`, `max`) per sparse cell — not full distributions. Insights therefore reason about **rankings, deltas, thresholds, and derived averages**, not arbitrary percentile curves across arbitrary slices unless you add **explicit** histogram or sketch storage elsewhere.
65
+
66
+ Token and latency **advisors** use those summaries (and sometimes robust z-scores on bucketed series); treat percentile-style language in recommendations as **heuristic** unless you extend storage.
67
+
68
+ ## Feature map
69
+
70
+ | Area | Methods |
71
+ |------|---------|
72
+ | Ranking | `topN`, `bottomN`, `topPercent`, `bottomPercent`, `pareto`, `contribution` |
73
+ | Comparison | `comparePeriods`, `trend` |
74
+ | Anomalies & rules | `detectAnomalies`, `evaluateRules` |
75
+ | Forecast & budget | `forecast`, `budgetProjection`, `costDrivers`, `recommendMaxTokens` |
76
+ | Advisors | `analyzeMapHealth`, `estimateResourcesFromActuals` |
77
+
78
+ ## See also
79
+
80
+ - Domain-focused narratives: `docs/INSIGHTS.md`, `docs/INSIGHTS_ANOMALY_DETECTION.md`, etc.
81
+ - [Ranking](./insights-ranking.md)
82
+ - Internal: [Insights layer](../internals/internal-insights-layer.md)
@@ -0,0 +1,81 @@
1
+ # Guide: Insights — ranking
2
+
3
+ Ranking helpers turn grouped query rows into **top/bottom lists**, **percent slices**, **Pareto** views, and **contribution** summaries.
4
+
5
+ ## Prerequisites
6
+
7
+ - A `CountexInsights` instance — see [Insights overview](./insights-overview.md).
8
+ - A measure selector string such as `"count"` or `"cost.sum"`.
9
+
10
+ ## Step 1: Top N
11
+
12
+ ```ts
13
+ const out = await insights.topN({
14
+ scope: "org_1",
15
+ counterId: "usage",
16
+ timeframe: { type: "day", from: "2026-05-01", to: "2026-05-07" },
17
+ groupBy: ["metric"],
18
+ measure: "count",
19
+ n: 10,
20
+ includeTotal: true,
21
+ });
22
+ ```
23
+
24
+ ## Step 2: Bottom N
25
+
26
+ ```ts
27
+ const out = await insights.bottomN({
28
+ scope: "org_1",
29
+ counterId: "usage",
30
+ timeframe: { type: "day", from: "2026-05-01", to: "2026-05-07" },
31
+ groupBy: ["dim:region"],
32
+ measure: "latencyMs.avg",
33
+ n: 5,
34
+ includeZeroes: false,
35
+ });
36
+ ```
37
+
38
+ Sparse counters omit zero rows unless you opt in.
39
+
40
+ ## Step 3: Top / bottom percent slices
41
+
42
+ ```ts
43
+ const slice = await insights.topPercent({
44
+ scope: "org_1",
45
+ counterId: "usage",
46
+ timeframe: { type: "month", from: "2026-05", to: "2026-05" },
47
+ groupBy: ["subject"],
48
+ measure: "cost.sum",
49
+ percent: 20,
50
+ });
51
+ ```
52
+
53
+ ## Step 4: Pareto
54
+
55
+ ```ts
56
+ const pareto = await insights.pareto({
57
+ scope: "org_1",
58
+ counterId: "usage",
59
+ timeframe: { type: "month", from: "2026-05", to: "2026-05" },
60
+ groupBy: ["subject"],
61
+ measure: "cost.sum",
62
+ targetShare: 0.8,
63
+ });
64
+ ```
65
+
66
+ ## Step 5: Contribution
67
+
68
+ ```ts
69
+ const contrib = await insights.contribution({
70
+ scope: "org_1",
71
+ counterId: "usage",
72
+ timeframe: { type: "month", from: "2026-05", to: "2026-05" },
73
+ groupBy: ["dim:model"],
74
+ measure: "tokens.sum",
75
+ });
76
+ ```
77
+
78
+ ## See also
79
+
80
+ - [Querying](./querying.md)
81
+ - Internal: [Insights layer](../internals/internal-insights-layer.md)
@@ -0,0 +1,71 @@
1
+ # Guide: Measures
2
+
3
+ **Measures** are optional numeric fields per event (cost, latency, tokens, …) aggregated alongside counts.
4
+
5
+ ## Step 1: Declare measures on the map
6
+
7
+ ```ts
8
+ const map = {
9
+ counterId: "llm_usage",
10
+ scope: { path: "$.scope.id" },
11
+ eventTime: { path: "$.request.createdAt" },
12
+ subjects: [{ type: "app", path: "$.app.id", required: true }],
13
+ metrics: [{ path: "$.request.status", required: true }],
14
+ dimensions: [{ name: "model", path: "$.model.name" }],
15
+ delta: { default: 1 },
16
+ timeframes: ["day"],
17
+ measures: [
18
+ { name: "cost", path: "$.usage.costCents", aggregations: ["sum"], scale: 100 },
19
+ { name: "latencyMs", path: "$.usage.latencyMs", aggregations: ["sum", "count", "min", "max"] },
20
+ ],
21
+ } as const;
22
+ ```
23
+
24
+ ### Aggregations
25
+
26
+ - **`sum`**, **`count`**, **`min`**, **`max`** are stored.
27
+ - **`avg`** is derived at query time when both **`sum`** and **`count`** exist for that measure.
28
+
29
+ ### Scale
30
+
31
+ `scale` multiplies then truncates for fixed-point storage (for example cents → integer micros).
32
+
33
+ ### Missing values
34
+
35
+ Use `onMissing: "skip"` (default) or `"drop"` / `"zero"` per measure.
36
+
37
+ ### Unique / distinct counts are not additive sums
38
+
39
+ Countex **counts** and **measure aggregations** (`sum`, `count`, `min`, `max`) behave like standard additive counters over cells. They do **not** answer “how many **distinct** users appeared in this bucket?” unless you model that explicitly (for example one subject id per event and interpret cell counts accordingly). Do **not** sum counts across overlapping populations and treat the result as a unique count. For large-cardinality uniques, plan on a dedicated approach (for example optional future HLL-style storage — see deployment notes); vanilla cells are the wrong tool for deduplicated set cardinality across arbitrary queries.
40
+
41
+ ## Step 2: Ingest as usual
42
+
43
+ Each fan-out increment carries the **same** measure snapshot for that event.
44
+
45
+ ## Step 3: Query measures
46
+
47
+ ```ts
48
+ const result = await countex.query({
49
+ scope: "org_1",
50
+ counterId: "llm_usage",
51
+ timeframe: { type: "day", from: "2026-05-08", to: "2026-05-08" },
52
+ groupBy: ["subject", "metric"],
53
+ measures: ["cost", "latencyMs"],
54
+ format: "table",
55
+ });
56
+ ```
57
+
58
+ Use `measures: "*"` to include all declared measures.
59
+
60
+ ### Ordering
61
+
62
+ `orderBy` can target measure aggregations (for example sort by `latencyMs` max).
63
+
64
+ ### Cache vs durable
65
+
66
+ Redis keeps **sum** and **count** for measures. **min** / **max** live on durable; the query planner skips cache when min/max are required.
67
+
68
+ ## See also
69
+
70
+ - [Querying](./querying.md)
71
+ - Internal: [Measures & scales](../internals/internal-measures-and-scales.md)
@@ -0,0 +1,119 @@
1
+ # Guide: Querying
2
+
3
+ `countex.query(spec)` reads aggregated cells for a scope and counter, with optional filters and output shaping.
4
+
5
+ ## Step 1: Minimal query
6
+
7
+ ```ts
8
+ const result = await countex.query({
9
+ scope: "org_1",
10
+ counterId: "access_by_actor_label",
11
+ timeframe: { type: "day", from: "2026-05-01", to: "2026-05-07" },
12
+ });
13
+ ```
14
+
15
+ ## Step 2: Filter
16
+
17
+ ```ts
18
+ const result = await countex.query({
19
+ scope: "org_1",
20
+ counterId: "access_by_actor_label",
21
+ timeframe: { type: "day", from: "2026-05-01", to: "2026-05-07" },
22
+ subject: { type: "actor", ids: [10001, 10002] },
23
+ metrics: [41, 42],
24
+ dimensions: { operation: ["read", "write"] },
25
+ });
26
+ ```
27
+
28
+ Subject and metric ids are **numeric dictionary ids** resolved during ingest.
29
+
30
+ ## Step 3: Group and format
31
+
32
+ `groupBy` accepts:
33
+
34
+ - `subject`, `subjectType`, `metric`, `timeframeKey`
35
+ - `dim:NAME` for a dimension
36
+
37
+ `format`:
38
+
39
+ - `cells` — raw-ish cell list
40
+ - `table` / `matrix` — aggregated rows
41
+ - `timeseries` — points per bucket (and optional extra grouping)
42
+
43
+ ```ts
44
+ const result = await countex.query({
45
+ scope: "org_1",
46
+ counterId: "access_by_actor_label",
47
+ timeframe: { type: "day", from: "2026-05-01", to: "2026-05-07" },
48
+ groupBy: ["subject", "dim:operation"],
49
+ format: "table",
50
+ orderBy: "-count",
51
+ limit: 50,
52
+ });
53
+ ```
54
+
55
+ ## Step 4: Measures and rates
56
+
57
+ ```ts
58
+ await countex.query({
59
+ // ...
60
+ measures: ["cost"],
61
+ orderBy: { measure: "cost", agg: "sum", dir: "desc" },
62
+ rate: { measure: "cost", agg: "sum" },
63
+ });
64
+ ```
65
+
66
+ Do not use `rate` with timeframe **`all`**.
67
+
68
+ ## Step 5: Cache control
69
+
70
+ - `bypassCache: true` forces durable reads.
71
+ - `signal` accepts `AbortSignal` for timeouts.
72
+
73
+ Result includes `source`: `"cache"`, `"durable"`, or `"merged"`.
74
+
75
+ ### Query diagnostics
76
+
77
+ `result.diagnostics` (when present) adds planner detail:
78
+
79
+ | Field | Meaning |
80
+ |-------|---------|
81
+ | `cacheAttempted` | `true` if the engine consulted the cache adapter (cache configured, not `bypassCache`, and the planner allowed cache for this query shape). |
82
+ | `cacheCoverage` | When `cacheAttempted` is true: `"full"` if the cache answered completely; `"partial"` or `"miss"` if the planner fell through to durable for authoritative cells. |
83
+ | `cellsMaterialized` | Number of counter cells loaded from storage before grouping/formatting. |
84
+
85
+ Use this for tests, dashboards, and verifying Redis vs Mongo paths.
86
+
87
+ ## Step 6: Totals, event volume, and “distinct entities”
88
+
89
+ ### Grand total (sum all matching cells)
90
+
91
+ Omit **`groupBy`** (or use an empty list). For **`format: "table"`**, the engine merges every matching cell into **one row**; **`rows[0].count`** is the sum of **`count`** across those cells. **`result.total`** matches that sum for the returned rows (`sumRowCounts` in `src/query/formatter.ts`).
92
+
93
+ Use this for **total increment volume** in the query window. If one source event fans out into many cells (multiple subjects × metrics × timeframes), this total is **not** the same as “number of source events.”
94
+
95
+ ### Distinct subjects / metrics as row cardinality
96
+
97
+ Countex does **not** store arbitrary **`COUNT(DISTINCT …)`** queries. **Exactly what to do** when you need “how many unique actors had activity?”:
98
+
99
+ 1. **`subject: { type: "actor" }`** (optional **`ids`**) to restrict the subject family.
100
+ 2. **`dimensions: { situation: ["external"] }`** (example) to restrict literals stored on cells.
101
+ 3. **`groupBy: ["subjectType", "subject"]`** so **each row aggregates one dictionary subject id**.
102
+ 4. **`rows.length`** after the query = number of distinct actors **with non-zero merged activity** in that slice (sparse — actors who never appeared produce no row).
103
+
104
+ **Critical:** if you set **`limit`**, **`rows.length ≤ limit`** and **`result.total`** only sum **returned** rows — you **cannot** infer global distinct counts from a capped query.
105
+
106
+ ### Logical “one row per source event” totals
107
+
108
+ Maintain a **separate counter map** engineered for **one increment per deduped message** (single timeframe granularity, single synthetic metric field, minimal subjects). See [`docs/use-cases/audit-security-log-counts.md`](../use-cases/audit-security-log-counts.md) §10.6.
109
+
110
+ ### Metric filters at rest
111
+
112
+ `MongoDurable` / `MemoryDurable` apply **`metrics`** filters using **numeric metric ids** parsed from `QuerySpec.metrics`. Pure string labels that are not finite numbers do **not** constrain the database filter — resolve ids from `countex_dict` or filter after read.
113
+
114
+ ## See also
115
+
116
+ - [Measures](./measures.md)
117
+ - [Timeframes](./timeframes-and-bucketing.md)
118
+ - [Audit use case — analytics recipes](../use-cases/audit-security-log-counts.md) §10
119
+ - Internal: [Query path](../internals/internal-query-path.md)
@@ -0,0 +1,33 @@
1
+ # Guide: Simulation profiles
2
+
3
+ Profiles configure **what kind of events** the simulator emits and **how much fan-out** they produce.
4
+
5
+ ## Built-in profile ids
6
+
7
+ | Profile id | Intent |
8
+ |------------|--------|
9
+ | `audit-log` | High-volume audit-style records: actors, resources, operations, optional label fan-out. |
10
+ | `product-activity` | Product analytics–style events: account scope, user subject, named event metric, feature/plan dimensions. |
11
+ | `processing-activity` | Jobs / pipelines / workflows: job id subject, task/step metrics, status dimensions, duration measures. |
12
+ | `llm-usage` | Token and cost measures with model/provider dimensions. |
13
+ | `custom` | Supply your own event factory; simulator handles scheduling and pools only. |
14
+
15
+ ## Profile options (conceptual)
16
+
17
+ - **`durationMs`** — simulated wall-clock span of the run (logical time advancement between events).
18
+ - **`eventsPerTick`** — throughput knob for batch generators.
19
+ - **`poolSizes`** — caps on distinct subjects/metrics/dimension values (cardinality stress).
20
+ - **`weights`** — skew toward hot keys (Pareto-style) vs uniform draws.
21
+
22
+ Exact fields match `SimulationProfile` in `@x12i/countex/simulation` TypeScript exports.
23
+
24
+ ## Choosing a profile
25
+
26
+ - Mirror the **same map** you plan for production; mismatched profiles hide JSONPath bugs.
27
+ - Start **uniform** pools, then add **skew** to reproduce hotspot behavior.
28
+ - Combine with **`validateMap(map, samples)`** from core so paths resolve on representative payloads.
29
+
30
+ ## See also
31
+
32
+ - [Simulation overview](./simulation.md)
33
+ - [Simulation resource estimation](./simulation-resource-estimation.md)
@@ -0,0 +1,30 @@
1
+ # Guide: Simulation resource estimation
2
+
3
+ Use simulation outputs to reason about **Mongo**, **Redis**, and **process memory** before cutover.
4
+
5
+ ## Signals from a run
6
+
7
+ After a simulator run (or CLI report):
8
+
9
+ - **Raw events** and **raw increments** (before cardinality drops) — multiply by your peak events/sec for ingress sizing.
10
+ - **Distinct dictionary keys** per kind (`subject`, `metric`, `dimensionValue`) — drives Mongo dictionary collection growth.
11
+ - **Distinct cells touched** — upper bound on sparse matrix cardinality for the window.
12
+ - **Flush batch sizes** — depends on buffer settings (`maxBufferedCells`, flush interval).
13
+
14
+ ## Mapping to infrastructure
15
+
16
+ | Concern | What to measure |
17
+ |---------|-----------------|
18
+ | Mongo durable | Cell documents + dictionary docs; index sizes (see deployment guide). |
19
+ | Redis | Working set for hot timeframes; TTL/retention policy per timeframe type. |
20
+ | Application RAM | In-flight buffer + Node overhead; simulate worst-case burst with `onOverflow: "block"`. |
21
+
22
+ ## Limits
23
+
24
+ Simulation **does not** replace load testing of your MongoDB cluster or Redis failover behavior; it estimates **logical** counter volume from maps and profiles.
25
+
26
+ ## See also
27
+
28
+ - [Simulation overview](./simulation.md)
29
+ - [Ingestion & flush](./ingestion-and-flush.md)
30
+ - [`docs/DEPLOYMENT.md`](../DEPLOYMENT.md)
@@ -0,0 +1,36 @@
1
+ # Guide: Simulation overview
2
+
3
+ **Simulation** helps before production traffic exists: generate deterministic synthetic events, stress-test maps and cardinality, estimate storage and flush load, and validate that aggregates match expectations.
4
+
5
+ ## Runtime entry point
6
+
7
+ When using the published package:
8
+
9
+ ```ts
10
+ import { createCountexSimulator } from "@x12i/countex/simulation";
11
+ ```
12
+
13
+ The simulator is optional tooling on top of the core engine; it does not change ingest or query semantics.
14
+
15
+ ## What simulation covers
16
+
17
+ - **Seeded randomness** — reproducible runs for CI and regression tests.
18
+ - **Pools & relations** — weighted entities (users, documents, models) with stable ids across events.
19
+ - **Profiles** — named scenarios (audit-style logs, product activity, LLM metering, processing pipelines); see [Simulation profiles](./simulation-profiles.md).
20
+ - **Expected aggregates** — optional tracker that compares simulated totals to counter reads for a window.
21
+ - **Reports** — structured summary (events emitted, fan-out estimates, errors) for sizing discussions.
22
+
23
+ ## Relationship to core Countex
24
+
25
+ 1. Build or load a `CounterMap` (same as production).
26
+ 2. Instantiate `createCountexSimulator({ map, profile, seed, ... })`.
27
+ 3. Drive **`generateEvents(n)`** or **`run()`** into `countex.ingest` / `ingestBatch` in tests or benchmarks.
28
+ 4. Optionally attach **`ExpectedAggregateTracker`** and assert after flush + query.
29
+
30
+ See [Simulation resource estimation](./simulation-resource-estimation.md) for capacity hooks.
31
+
32
+ ## See also
33
+
34
+ - [CLI](./cli.md) — `countex simulate:demo` for scripted dry-runs without a config module
35
+ - [Counter maps](./counter-maps.md)
36
+ - [Cardinality](./cardinality.md)