@toa.io/extensions.chronicles 0.0.0-stage → 1.0.0-alpha.333

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (83) hide show
  1. package/LICENSE +22 -0
  2. package/components/chronicles.encounters/manifest.toa.yaml +108 -0
  3. package/components/chronicles.encounters/operations/anonymous.js +4 -0
  4. package/components/chronicles.encounters/operations/describe.js +4 -0
  5. package/components/chronicles.encounters/operations/identified.js +4 -0
  6. package/components/chronicles.encounters/types/index.d.ts +3 -0
  7. package/components/chronicles.encounters/types/toa.d.ts +36 -0
  8. package/components/chronicles.queries/manifest.toa.yaml +139 -0
  9. package/components/chronicles.queries/operations/define.js +3 -0
  10. package/components/chronicles.queries/operations/describe.js +3 -0
  11. package/components/chronicles.queries/operations/drop.js +3 -0
  12. package/components/chronicles.queries/operations/query.js +3 -0
  13. package/components/chronicles.queries/types/index.d.ts +3 -0
  14. package/components/chronicles.queries/types/toa.d.ts +49 -0
  15. package/components/chronicles.relay/manifest.toa.yaml +43 -0
  16. package/components/chronicles.relay/operations/encounter.js +4 -0
  17. package/components/chronicles.relay/operations/record.js +4 -0
  18. package/components/chronicles.relay/types/index.d.ts +3 -0
  19. package/components/chronicles.relay/types/toa.d.ts +24 -0
  20. package/guides/clickhouse.md +19 -0
  21. package/guides/log.md +52 -0
  22. package/package.json +39 -4
  23. package/readme.md +686 -0
  24. package/store/clickhouse.scope.sql +29 -0
  25. package/store/clickhouse.sql +99 -0
  26. package/transpiled/Client.d.ts +32 -0
  27. package/transpiled/Client.js +173 -0
  28. package/transpiled/Client.js.map +1 -0
  29. package/transpiled/Factory.d.ts +24 -0
  30. package/transpiled/Factory.js +62 -0
  31. package/transpiled/Factory.js.map +1 -0
  32. package/transpiled/Provider.d.ts +142 -0
  33. package/transpiled/Provider.js +4 -0
  34. package/transpiled/Provider.js.map +1 -0
  35. package/transpiled/Publisher.d.ts +50 -0
  36. package/transpiled/Publisher.js +169 -0
  37. package/transpiled/Publisher.js.map +1 -0
  38. package/transpiled/Reader.d.ts +28 -0
  39. package/transpiled/Reader.js +166 -0
  40. package/transpiled/Reader.js.map +1 -0
  41. package/transpiled/Recorder.d.ts +38 -0
  42. package/transpiled/Recorder.js +107 -0
  43. package/transpiled/Recorder.js.map +1 -0
  44. package/transpiled/Relay.d.ts +14 -0
  45. package/transpiled/Relay.js +27 -0
  46. package/transpiled/Relay.js.map +1 -0
  47. package/transpiled/Sender.d.ts +25 -0
  48. package/transpiled/Sender.js +86 -0
  49. package/transpiled/Sender.js.map +1 -0
  50. package/transpiled/Service.d.ts +26 -0
  51. package/transpiled/Service.js +70 -0
  52. package/transpiled/Service.js.map +1 -0
  53. package/transpiled/address.d.ts +7 -0
  54. package/transpiled/address.js +27 -0
  55. package/transpiled/address.js.map +1 -0
  56. package/transpiled/agent.d.ts +16 -0
  57. package/transpiled/agent.js +48 -0
  58. package/transpiled/agent.js.map +1 -0
  59. package/transpiled/clickhouse/Client.d.ts +35 -0
  60. package/transpiled/clickhouse/Client.js +71 -0
  61. package/transpiled/clickhouse/Client.js.map +1 -0
  62. package/transpiled/clickhouse/Provider.d.ts +35 -0
  63. package/transpiled/clickhouse/Provider.js +169 -0
  64. package/transpiled/clickhouse/Provider.js.map +1 -0
  65. package/transpiled/clickhouse/ddl.d.ts +9 -0
  66. package/transpiled/clickhouse/ddl.js +15 -0
  67. package/transpiled/clickhouse/ddl.js.map +1 -0
  68. package/transpiled/clickhouse/views.d.ts +11 -0
  69. package/transpiled/clickhouse/views.js +97 -0
  70. package/transpiled/clickhouse/views.js.map +1 -0
  71. package/transpiled/contract.d.ts +12 -0
  72. package/transpiled/contract.js +58 -0
  73. package/transpiled/contract.js.map +1 -0
  74. package/transpiled/index.d.ts +15 -0
  75. package/transpiled/index.js +2 -0
  76. package/transpiled/index.js.map +1 -0
  77. package/transpiled/settings.d.ts +13 -0
  78. package/transpiled/settings.js +48 -0
  79. package/transpiled/settings.js.map +1 -0
  80. package/transpiled/store.d.ts +5 -0
  81. package/transpiled/store.js +11 -0
  82. package/transpiled/store.js.map +1 -0
  83. package/README.md +0 -3
package/readme.md ADDED
@@ -0,0 +1,686 @@
1
+ # Toa Chronicles
2
+
3
+ A log of what happens in a context. Components write to it by the events they declare and the
4
+ facts they send, clients by a route of the gateway; the service `chronicles-relay` publishes every
5
+ row of it to an analytics broker, and reads it back from a store, by an
6
+ [HTTP API](#reading-the-log) — which the gateway also publishes as [MCP](#over-mcp) tools.
7
+
8
+ Writing and reading meet in nothing Toa does: what takes the rows off the broker into the store,
9
+ and who may read what there, is the [store's](#the-store) own setup.
10
+
11
+ ## The log
12
+
13
+ Three tables, each row of which has the `scope` of the context that wrote it — the context's name,
14
+ followed by `TOA_SUFFIX` where one is given — and a reader sees the rows of its own scope.
15
+
16
+ **`events`** holds a row for everything a component did: an [event](#events-and-states) it
17
+ declares, under `<component>.<event>`, with the entity that changed and its version; and a
18
+ [fact](#facts) it sends, under `<component>.<name>`, with neither.
19
+
20
+ | `label` | `component` | `entity` | `version` | `payload` |
21
+ | ----------------------- | -------------- | -------- | --------- | --------------------------------- |
22
+ | `store.orders.created` | `store.orders` | `9c1f…` | `0` | `{ total: 120, status: "new" }` |
23
+ | `store.search.searched` | `store.search` | `NULL` | `NULL` | `{ query: "teapot", results: 3 }` |
24
+
25
+ Every row also has `id`, the same however often it is delivered, so a row is kept once;
26
+ `identity`, whose it is, where something [says so](#identity); `time`, when it
27
+ [happened](#time); and `trace`, the traceparent of the call that caused it.
28
+
29
+ **`states`** holds each entity as it stands: for every component that declares a
30
+ [state](#events-and-states), a row per entity — `component`, `entity`, `version`, `identity`,
31
+ `time` and `payload` — which a higher version replaces. It is the current state and
32
+ nothing before it: what an entity was yesterday is read from its events, so an event a question
33
+ of history depends on has to carry what that question asks.
34
+
35
+ **`encounters`** holds what [clients send](#encounters): `id`, `name`, `time`, `received`,
36
+ `visitor`, `session`, `identity`, `ip`, `country`, `origin`, `agent`, `browser`, `os`, `device` and
37
+ `payload`.
38
+
39
+ Nothing is ordered: order by `time`, or by `version` within one entity.
40
+
41
+ ### Time
42
+
43
+ Every time is the one its source says — when it happened — and none is when the store wrote it:
44
+
45
+ | | `time` |
46
+ | --------- | ----------------------------------------------------------------------------------------- |
47
+ | an event | the payload's [timestamp](#when-a-change-was-made), or when the component made the change |
48
+ | a state | likewise |
49
+ | a fact | when the operation sent it |
50
+ | encounter | when the gateway received it, less how long before sending it the client says it happened |
51
+
52
+ So a change the relay takes an hour late is written at the time it was made, and the hours it
53
+ was away fill in as it catches up: a count of a recent period may still grow. An encounter's
54
+ `received` is when the gateway received its batch.
55
+
56
+ ## Events and states
57
+
58
+ A component declares what it logs, and nothing else of it is written:
59
+
60
+ ```yaml
61
+ # manifest.toa.yaml
62
+ chronicles:
63
+ events:
64
+ paid:
65
+ description: An order's payment completed
66
+ properties:
67
+ total:
68
+ type: integer
69
+ description: The amount paid, in cents
70
+ shipping:
71
+ type: object
72
+ description: Where the order goes
73
+ properties:
74
+ country:
75
+ type: string
76
+ description: ISO 3166 alpha-2
77
+ state:
78
+ event: snapshot
79
+ description: An order as it stands
80
+ properties:
81
+ status:
82
+ type: string
83
+ enum: [new, paid, shipped]
84
+ description: Where the order is in its life
85
+ ```
86
+
87
+ Each label under `events` is an event of the component — its own, or the prototype's `created`,
88
+ `updated`, `deleted` and `sync` — and `state.event` is the one whose payload is the entity. An
89
+ event is written as it renders: where its `condition` does not hold, nothing is written; where it
90
+ has a `payload`, what the payload returns is what is written. An event that exists for the log
91
+ alone is an event file like any other, and nothing else has to consume it.
92
+
93
+ ### The contract
94
+
95
+ `description` says what the label means, and `properties` what its payload holds, as
96
+ [JSON Schema](https://json-schema.org) of the types [below](#types). Every property, at every
97
+ depth, has a `description`; a declaration that leaves one out, or uses what has no type there, is
98
+ refused when the component is read.
99
+
100
+ What is written is the declared properties of the payload, at every depth, and nothing else: a
101
+ property the declaration leaves out — an email, a document, a field added to the entity since —
102
+ does not reach the store. A payload whose declared property is of another type is not written,
103
+ and is kept in the parking queue of its consumer.
104
+
105
+ A state's payload carries `id` and `VERSION`, which say which entity it is and which version: one
106
+ that does not is refused, and parked likewise. A row of `events` takes its `entity` and `version`
107
+ from the payload's `id` and `VERSION` where it carries them, and is written without them where it
108
+ does not. These, the [timestamp](#when-a-change-was-made) and the [identity](#identity) are read
109
+ before the contract applies, and are not declared. An event that is both is named in both.
110
+
111
+ A committed change is written however long the relay or the broker is away, and once however often it is
112
+ delivered. One committed before the relay first ran in a deployment is not.
113
+
114
+ ### Types
115
+
116
+ | declared | read as |
117
+ | ---------------------------------------- | ------------------------ |
118
+ | `string` | `String` |
119
+ | `string` with `enum` | `LowCardinality(String)` |
120
+ | `integer` | `Int64` |
121
+ | `number` | `Float64` |
122
+ | `boolean` | `Bool` |
123
+ | `string`, `format: date-time` | `DateTime64(3, 'UTC')` |
124
+ | `string`, `format: date` | `Date32` |
125
+ | `string`, `format: time` | `Time64(3)` |
126
+ | `string`, `format: uuid`, `ipv4`, `ipv6` | `UUID`, `IPv4`, `IPv6` |
127
+ | `integer`, `format: int32` | `Int32` |
128
+ | `number`, `format: float` | `Float32` |
129
+ | any other `format` | as its `type` |
130
+ | `object` with `properties` | `Tuple` of them, by name |
131
+ | `array` with `items` of any of these | `Array` of it |
132
+
133
+ ### Reading what is declared
134
+
135
+ `describe` tells, of every declared field of a label or a state, its type and how to read it from
136
+ the table in that type:
137
+
138
+ ```yaml
139
+ - label: store.orders.paid
140
+ fields:
141
+ - path: total
142
+ type: Int64
143
+ description: The amount paid, in cents
144
+ read: JSONExtract(ifNull(toJSONString(payload.total), 'null'), 'Nullable(Int64)')
145
+ ```
146
+
147
+ ```sql
148
+ SELECT JSONExtract(ifNull(toJSONString(payload.total), 'null'), 'Nullable(Int64)') AS total, count()
149
+ FROM events FINAL WHERE label = 'store.orders.paid' GROUP BY total
150
+ ```
151
+
152
+ A value that is not of its type reads as `NULL`; the row keeps it as it was written. A contract
153
+ that changes changes what `describe` tells, and the rows written before it are read the same way.
154
+
155
+ ### When a change was made
156
+
157
+ `time` is the payload's `UPDATED`, which the entity as it stands carries, where it is a number,
158
+ and where it is not, the moment the component made the change, which every event carries:
159
+
160
+ ```yaml
161
+ # manifest.toa.yaml
162
+ chronicles:
163
+ timestamp: placed # the property of the payloads that says when, in milliseconds
164
+ events: …
165
+ ```
166
+
167
+ A component names another property, or none with `timestamp: ~`, when the moment of the change
168
+ is always the time. One property serves every event of the component and its state.
169
+
170
+ ### Identity
171
+
172
+ `identity` is the payload's `identity`, where it is a string:
173
+
174
+ ```yaml
175
+ # manifest.toa.yaml
176
+ chronicles:
177
+ identity: customer # the property of the payloads that names whose a row is
178
+ events: …
179
+ ```
180
+
181
+ A component names another property, or none with `identity: ~`. One property serves every event of
182
+ the component and its state, and a payload that leaves it out is a row of no identity. A property
183
+ the component names is one of its entity.
184
+
185
+ ## Facts
186
+
187
+ ```yaml
188
+ # manifest.toa.yaml
189
+ chronicles:
190
+ facts:
191
+ searched:
192
+ description: A customer searched the catalogue
193
+ properties:
194
+ query:
195
+ type: string
196
+ description: What was typed
197
+ results:
198
+ type: integer
199
+ description: How many products were found
200
+ ```
201
+
202
+ ```typescript
203
+ await context.chronicles.send('searched', { query: input.query, results: results.length })
204
+ ```
205
+
206
+ The fact is a row of `events` under `<component>.searched`, with what was sent of its contract as
207
+ its payload. `send` throws for a name the component does not declare, or a payload a declared
208
+ property of which is of another type; it resolves once the broker has it, and it then waits for
209
+ the relay as an event does.
210
+
211
+ An operation that knows whose the fact is says so; the extension does not know who called it:
212
+
213
+ ```typescript
214
+ await context.chronicles.send(
215
+ 'searched',
216
+ { query: input.query },
217
+ { identity: input.customer }
218
+ )
219
+ ```
220
+
221
+ `context.chronicles` is given to a component that declares `chronicles`.
222
+
223
+ ## Encounters
224
+
225
+ What clients send: pages viewed, buttons pressed, anything a client tells of itself. Served where
226
+ the context says `encounters: true`, at two routes: one of an identity, and one of no one. Who may
227
+ send to which is in [Access](#access).
228
+
229
+ ### `POST /chronicles/encounters/:identity/`
230
+
231
+ What a signed-in client sends, as the identity it is.
232
+
233
+ ```http
234
+ POST /chronicles/encounters/9c1f0a6d5b7e4f3a8d2c1b0a9e8f7d6c/ HTTP/1.1
235
+ authorization: Token …
236
+ content-type: application/json
237
+
238
+ {
239
+ "sent": 1761200004000,
240
+ "visitor": "7f3a…",
241
+ "session": "01c2…",
242
+ "events": [{ "id": "0192f3a1-…", "name": "page_viewed", "time": 1761200000000, "payload": { "path": "/pricing" } }]
243
+ }
244
+ ```
245
+
246
+ ```http
247
+ 202 Accepted
248
+
249
+ accepted: 1
250
+ ```
251
+
252
+ The rows are that identity's, whatever the body says.
253
+
254
+ ### `POST /chronicles/encounters/`
255
+
256
+ What anyone sends, before signing in or without it. The same body, and rows of no identity; a body
257
+ that names one is refused.
258
+
259
+ A form whose `events` is their JSON, beside `sent`, `visitor` and `session`, is read as the JSON
260
+ body, so `navigator.sendBeacon`, which sends no `authorization`, can send a batch here as the page
261
+ closes:
262
+
263
+ ```javascript
264
+ navigator.sendBeacon(
265
+ '/chronicles/encounters/',
266
+ new URLSearchParams({
267
+ sent: Date.now(),
268
+ visitor,
269
+ session,
270
+ events: JSON.stringify(events)
271
+ })
272
+ )
273
+ ```
274
+
275
+ A signed-in client sends with `fetch(…, { keepalive: true })` instead.
276
+
277
+ ### The body
278
+
279
+ `{ sent, visitor?, session?, events }`:
280
+
281
+ | property | |
282
+ | --------- | ---------------------------------------------------------------------------------------------- |
283
+ | `sent` | when the client sends the batch, in milliseconds, by its clock |
284
+ | `visitor` | the client's own id of itself, kept across sessions; at most 64 characters, or `null` for none |
285
+ | `session` | the id of one period of the client's activity; at most 64 characters, or `null` for none |
286
+ | `events` | at most 100 of them |
287
+
288
+ Each event:
289
+
290
+ | property | |
291
+ | --------- | ------------------------------------------------------------------- |
292
+ | `id` | the client's own, at most 64 characters; a retry sends the same one |
293
+ | `name` | lowercase letters, digits and underscores from a letter, at most 64 |
294
+ | `time` | when it happened, in milliseconds, by the client's clock, by `sent` |
295
+ | `payload` | an object, at most 16 KiB of JSON |
296
+
297
+ The client keeps its `visitor` — in `localStorage`, say — and sends the same one before signing
298
+ in and after. The rows of the route of an identity have both, so the visitors of an identity are
299
+ a query. A visitor is not a person: one identity may have many, and a browser that clears its
300
+ storage is a new one.
301
+
302
+ A request is answered once its events are written. A batch one event of which breaks any of the
303
+ above is written not at all, and answered `422` with the code `INVALID`, or `OVERSIZED` for a
304
+ payload too large, and the `index` of the event in `cause`.
305
+
306
+ A payload is written as it was sent, whatever its shape: a client is deployed once and runs for
307
+ months, so a shape that changes is a new `name` — `checkout_started_v2` — and the old one stays as
308
+ old clients send it.
309
+
310
+ A client's clock is its own, and may be off by anything; what it measures between two of its
311
+ moments is not. So an encounter's `time` is when the gateway received the batch, less `sent`
312
+ minus its own `time`: the gateway's clock, at the distance the client measured. A clock that is
313
+ wrong moves nothing, and a batch kept offline for an hour is written an hour back. A row is kept
314
+ and expires by `received`.
315
+
316
+ ### Address and country
317
+
318
+ Every encounter has the `ip` and the `country` of the request, as the gateway reads them from the
319
+ headers [`exposition.ip` and `exposition.country`](/extensions/exposition/documentation/ip.md)
320
+ name, and `NULL` where it has none; a body that names either is overwritten. An address is
321
+ personal data, and the log erases nothing, so of an address the log keeps its network — a `/24`
322
+ of IPv4, a `/48` of IPv6 — unless the annotation says `ip: full`.
323
+
324
+ ### Origin and agent
325
+
326
+ Every encounter has the `origin` of the page that sent it and the client's `agent`, from the
327
+ request's `Origin` and `User-Agent` headers, and `NULL` where it has none; a body that names either
328
+ is overwritten, and a body cannot say one the request does not have. A browser sends `Origin` with
329
+ a request to another origin, and with every `POST`, so it tells which of the context's sites a
330
+ batch came from.
331
+
332
+ What the agent says the client is, is three columns of its own, by name and without versions:
333
+
334
+ | | |
335
+ | --------- | ----------------------------------------------------------------------------- |
336
+ | `browser` | `Edge`, `Opera`, `Samsung Internet`, `Firefox`, `Chrome`, `Safari`, or `NULL` |
337
+ | `os` | `Windows`, `iOS`, `macOS`, `Android`, `ChromeOS`, `Linux`, or `NULL` |
338
+ | `device` | `bot`, `tablet`, `mobile` or `desktop`, and `NULL` where there is no agent |
339
+
340
+ `bot` is a crawler, a headless browser or a program — what a question of who visits leaves out.
341
+ Safari on an iPad says it is a Mac, and is a `desktop` running `macOS` here. A version is read from
342
+ `agent` where a question needs one, knowing that browsers freeze what they say of the system:
343
+ Safari on iOS 26 says iOS 18, and Chrome on Android says Android 10.
344
+
345
+ ### What the encounters are
346
+
347
+ What a client sends is told in the configuration of `chronicles.encounters`, by name:
348
+
349
+ ```yaml
350
+ # context.toa.yaml
351
+ configuration:
352
+ chronicles.encounters:
353
+ page_viewed:
354
+ description: A page was shown
355
+ fields:
356
+ path: The URL path, without the query
357
+ referrer: The page the visitor came from
358
+ ```
359
+
360
+ It is what `describe` tells an agent of each encounter's fields, and what whoever writes a client
361
+ reads:
362
+
363
+ ```http
364
+ GET /chronicles/encounters/ HTTP/1.1
365
+ authorization: Token …
366
+ accept: application/yaml
367
+ ```
368
+
369
+ ```http
370
+ 200 OK
371
+
372
+ page_viewed:
373
+ description: A page was shown
374
+ fields:
375
+ path: The URL path, without the query
376
+ referrer: The page the visitor came from
377
+ ```
378
+
379
+ It documents, and holds nothing back: a name it does not tell is written as any other.
380
+
381
+ ## Reading the log
382
+
383
+ A refusal is answered `422` with `{ code, cause }`. Who may read is in [Access](#access).
384
+
385
+ ### `GET /chronicles/queries/`
386
+
387
+ What the log holds: a guide to reading it, the labels of `events`, the components
388
+ with a state, the encounters, and the views.
389
+
390
+ ```http
391
+ GET /chronicles/queries/ HTTP/1.1
392
+ authorization: Token …
393
+ accept: application/yaml
394
+ ```
395
+
396
+ ```http
397
+ 200 OK
398
+
399
+ guide: |
400
+ # Reading the log
401
+
402
+ The log is three tables, read with `FINAL`.
403
+ …
404
+ labels:
405
+ - label: store.orders.paid
406
+ description: An order's payment completed
407
+ rows: 1204
408
+ first: 2026-10-01 08:12:03.000
409
+ last: 2026-10-24 17:40:51.000
410
+ fields:
411
+ - path: total
412
+ type: Int64
413
+ description: The amount paid, in cents
414
+ read: JSONExtract(ifNull(toJSONString(payload.total), 'null'), 'Nullable(Int64)')
415
+ - path: shipping
416
+ type: Tuple(country String)
417
+ description: Where the order goes
418
+ read: JSONExtract(toJSONString(payload.^shipping), 'Tuple(country Nullable(String))')
419
+ - path: shipping.country
420
+ type: String
421
+ description: ISO 3166 alpha-2
422
+ states:
423
+ - component: store.orders
424
+ description: An order as it stands
425
+ entities: 1180
426
+ last: 2026-10-24 17:40:51.000
427
+ fields:
428
+ - path: status
429
+ type: LowCardinality(String)
430
+ description: Where the order is in its life
431
+ read: JSONExtract(ifNull(toJSONString(payload.status), 'null'), 'LowCardinality(Nullable(String))')
432
+ encounters:
433
+ - name: page_viewed
434
+ description: A page was shown
435
+ rows: 88123
436
+ first: 2026-10-01 00:00:14.000
437
+ last: 2026-10-24 17:41:02.000
438
+ received: 2026-10-24 17:41:03.020
439
+ fields:
440
+ - path: path
441
+ types: [String]
442
+ description: The URL path, without the query
443
+ - path: utm
444
+ types: [Int64, String]
445
+ views:
446
+ - name: paid_daily
447
+ read: shop.paid_daily
448
+ title: Orders paid per day
449
+ description: Orders whose payment completed, counted by the day of `time`
450
+ sql: SELECT toDate(time) AS day, count() AS paid FROM events FINAL WHERE label = 'store.orders.paid' GROUP BY day
451
+ refresh: 3600
452
+ refreshed: 2026-10-24 17:00:03
453
+ ```
454
+
455
+ `guide` is how to read the log — what its tables and columns mean, and what to be careful of —
456
+ and how its store is read, in the store's dialect: what an agent reads before its first query.
457
+ Toa writes it; what is particular to an application is in the descriptions of what it declares
458
+ and documents, and in [`exposition.mcp.instructions`](/extensions/exposition/documentation/mcp.md).
459
+
460
+ A label and a state are told by their declaration, with each field's type and how to
461
+ [read](#reading-what-is-declared) it in that type; a field of a field is read inside the one that
462
+ holds it. A label declared and not yet written is told with no rows. An encounter is
463
+ told by what was sent: the fields of the latest payloads, the types they were sent as — two
464
+ types of one field are two versions of a client — and the description the
465
+ [configuration](#what-the-encounters-are) gives.
466
+
467
+ `rows` counts what the store holds, a row delivered twice possibly twice. `first` and `last` are
468
+ by `time`, the latest change of a state too; an encounter's `received` is the newest batch the
469
+ gateway received, which tells whether clients still send.
470
+
471
+ ### `POST /chronicles/queries/`
472
+
473
+ One statement, in the store's dialect, as the `guide` says.
474
+
475
+ ```http
476
+ POST /chronicles/queries/ HTTP/1.1
477
+ authorization: Token …
478
+ accept: application/yaml
479
+ content-type: application/yaml
480
+
481
+ sql: SELECT status, count() AS n FROM store_orders GROUP BY status
482
+ ```
483
+
484
+ ```http
485
+ 201 Created
486
+
487
+ columns:
488
+ - name: status
489
+ type: Nullable(String)
490
+ - name: n
491
+ type: UInt64
492
+ rows:
493
+ - [new, 31]
494
+ - [paid, 1149]
495
+ truncated: false
496
+ ```
497
+
498
+ A query reads the log and nothing else, and cannot write. It runs for at most 30 seconds, answers
499
+ at most 10,000 rows and 64 MiB, and a reader runs at most 4 queries at once: past the rows it
500
+ answers what it has, with `truncated: true`; past the rest it is refused. It leaves `FORMAT` out.
501
+
502
+ | code | when |
503
+ | --------- | ----------------------------------------------------------------------- |
504
+ | `REFUSED` | the store refused the statement; `cause.message` is what the store said |
505
+
506
+ ### `PUT /chronicles/queries/views/:name/`
507
+
508
+ A view over the log, by a statement a query could run. Defining a name again replaces the view.
509
+
510
+ ```http
511
+ PUT /chronicles/queries/views/paid_daily/ HTTP/1.1
512
+ authorization: Token …
513
+ content-type: application/yaml
514
+
515
+ title: Orders paid per day
516
+ description: Orders whose payment completed, counted by the day of `time`
517
+ sql: SELECT toDate(time) AS day, count() AS paid FROM events FINAL WHERE label = 'store.orders.paid' GROUP BY day
518
+ refresh: 3600
519
+ ```
520
+
521
+ ```http
522
+ 204 No Content
523
+ ```
524
+
525
+ `title`, at most 100 characters, and `description` tell an agent that finds the view what it
526
+ counts. Without `refresh`, a view is computed when it is read. With `refresh`, in seconds and at
527
+ least `3600`, it is computed on that cycle and read as a table, and `describe` tells when it last
528
+ was. A name is lowercase letters, digits and underscores, and a context has at most 50 views. A
529
+ view is kept in the database named after the context's scope, and a query reads it by that name:
530
+ `shop.paid_daily`. Defining one takes a store whose user may create views there; one that may not
531
+ is refused.
532
+
533
+ | code | when |
534
+ | ----------- | -------------------------------------------------------------------- |
535
+ | `REFUSED` | the statement is not one a query could run; `cause.message` says why |
536
+ | `RESERVED` | the name is one of the log's tables |
537
+ | `EXHAUSTED` | the context has 50 views, and this is not one of them |
538
+
539
+ ### `DELETE /chronicles/queries/views/:name/`
540
+
541
+ ```http
542
+ DELETE /chronicles/queries/views/paid_daily/ HTTP/1.1
543
+ authorization: Token …
544
+ ```
545
+
546
+ ```http
547
+ 204 No Content
548
+ ```
549
+
550
+ | code | when |
551
+ | ----------- | --------------------- |
552
+ | `NOT_FOUND` | there is no such view |
553
+
554
+ ### Over MCP
555
+
556
+ Where the context publishes [MCP](/extensions/exposition/documentation/mcp.md), each of these
557
+ routes is a tool, under the name the gateway gives a route, and taking what the route takes:
558
+
559
+ | tool | route |
560
+ | --------------------------------------- | ----------------------------------------- |
561
+ | `chronicles.queries.GET` | `GET /chronicles/queries/` |
562
+ | `chronicles.queries.POST` | `POST /chronicles/queries/` |
563
+ | `chronicles.queries.views._name.PUT` | `PUT /chronicles/queries/views/:name/` |
564
+ | `chronicles.queries.views._name.DELETE` | `DELETE /chronicles/queries/views/:name/` |
565
+ | `chronicles.encounters.GET` | `GET /chronicles/encounters/` |
566
+
567
+ A refusal is a tool result with `isError`.
568
+
569
+ ## Access
570
+
571
+ | route | who may call | limit |
572
+ | ----------------------------------------- | ------------------------------------ | --------------------------- |
573
+ | `POST /chronicles/encounters/` | anyone, with no credentials | 60 a minute per address |
574
+ | `POST /chronicles/encounters/:identity/` | the identity, by its own credentials | 60 a minute per identity |
575
+ | `GET /chronicles/encounters/` | `system:chronicles:encounters` | |
576
+ | `GET /chronicles/queries/` | `system:chronicles:query` | |
577
+ | `POST /chronicles/queries/` | `system:chronicles:query` | the [reader's](#the-reader) |
578
+ | `PUT /chronicles/queries/views/:name/` | `system:chronicles:views` | 50 views |
579
+ | `DELETE /chronicles/queries/views/:name/` | `system:chronicles:views` | |
580
+
581
+ A caller without what a route takes is answered `403`, and past its limit `429`. Over
582
+ [MCP](#over-mcp), a tool is listed to a caller that holds its role, and to no one else.
583
+
584
+ ### Roles
585
+
586
+ | | |
587
+ | ------------------------------ | ------------------------------------------------------------------------------------------- |
588
+ | `system:chronicles:query` | reads every row of the log: the identities, the networks, the origins and agents of clients |
589
+ | `system:chronicles:views` | defines and drops the views every reader reads; a view can read only what a query can |
590
+ | `system:chronicles:encounters` | reads what the configuration documents of encounters, and no row |
591
+ | `system:chronicles` | all three |
592
+
593
+ `query` is access to personal data, and is granted as that. `views` changes what other readers are
594
+ told a view counts, and changes nothing of the log. `encounters` is for whoever writes a client.
595
+ A role is granted as any [role](/extensions/exposition/documentation/access.md) is.
596
+
597
+ ### What a client cannot say
598
+
599
+ A client sends events and nothing that says who or where it is: `identity` is the path's, which the
600
+ credentials have to be the identity's; `ip`, `country`, `origin` and `agent` are what the gateway
601
+ read off the request, and a body that names one is overwritten, or refused where it names
602
+ `identity` on the route of no one. Anything else in a payload is what the client said — a `user`
603
+ there is not a user — so a question that depends on who sent an encounter asks `identity`.
604
+
605
+ ### The reader
606
+
607
+ A query runs as the context's user of the store, which reads what the [store](#the-store) lets it
608
+ — in the reference setup, the rows of its scope and the views of its database — and is never let
609
+ lift its limits there. Toa asks a query to be read-only besides, so a query can change nothing even
610
+ of what its user may: only `define` and `drop` create and remove views.
611
+
612
+ ## Deployment
613
+
614
+ ```yaml
615
+ # context.toa.yaml
616
+ chronicles:
617
+ broker: amqp://analytics.internal
618
+ store:
619
+ provider: clickhouse
620
+ url: http://analytics.internal:8123
621
+ retention: 730
622
+ encounters: true
623
+ ```
624
+
625
+ | | |
626
+ | ------------ | ----------------------------------------------------------------------------------- |
627
+ | `broker` | where the rows are published |
628
+ | `store` | where they are read: the provider, and the address of its server |
629
+ | `retention` | days a row of `events` and of `encounters` is kept; absent, kept |
630
+ | `encounters` | the routes clients send to; absent, there are none |
631
+ | `ip` | `full` keeps a client's whole address; absent, its network |
632
+ | `resources` | what the service may take, as [`resources`](/migrations/262.md) is stated elsewhere |
633
+
634
+ ```shell
635
+ $ toa conceal chronicles broker_username=… broker_password=… store_username=… store_password=…
636
+ ```
637
+
638
+ The service `chronicles-relay` alone is given the secret.
639
+
640
+ The events the components declare travel where the context's `amqp` says, as every event does, and
641
+ the relay takes them from there. A fact and an encounter reach it as tasks. Each is acknowledged
642
+ once the broker has confirmed its row, so while the broker is away they wait where they are, and
643
+ are published once it is back; the relay starts and is ready without it.
644
+
645
+ ## The store
646
+
647
+ What Toa publishes and what it reads; what takes the one into the other is not Toa's.
648
+
649
+ **Published:** one JSON row a message, persistent, to the exchange `chronicles` (`direct`) of the
650
+ broker, under the key of its table — `events`, `states` or `encounters`. A row has the columns of
651
+ its table:
652
+
653
+ | table | row |
654
+ | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
655
+ | `events` | `scope`, `id`, `label`, `component`, `time`, `entity`, `version`, `identity`, `trace`, `payload`, `expires` |
656
+ | `states` | `scope`, `component`, `entity`, `version`, `identity`, `time`, `payload` |
657
+ | `encounters` | `scope`, `id`, `name`, `time`, `received`, `visitor`, `session`, `identity`, `ip`, `country`, `origin`, `agent`, `browser`, `os`, `device`, `payload`, `expires` |
658
+
659
+ A time is `YYYY-MM-DD hh:mm:ss.sss`, in UTC; `expires` is `time` — an encounter's `received` — and
660
+ `retention` days, or `2106-01-01 00:00:00` where there is none. A row may be delivered more than
661
+ once, with the same `id`. The broker's user needs to declare and to write the exchange `chronicles`,
662
+ and nothing else.
663
+
664
+ **Read:** `events`, `states` and `encounters`, with those columns, in the default database of the
665
+ `store` user, and a database named after the scope for its views.
666
+
667
+ The relay writes the scope of its context into every row, and nothing else does: the broker tells
668
+ the store nothing of who published a row. What keeps a context from reading another's rows is the
669
+ store's.
670
+
671
+ ### ClickHouse
672
+
673
+ [`store/clickhouse.sql`](./store/clickhouse.sql) sets a store up: the database `chronicles` with the
674
+ three tables, which are replaced by key in the background — so a query reads them with `FINAL`,
675
+ which collapses what was delivered twice — and a RabbitMQ engine table and a view for each, which
676
+ take the rows off the broker. [`store/clickhouse.scope.sql`](./store/clickhouse.scope.sql) adds the
677
+ user of a context: its row policy, its limits and its database. A user with no row policy reads
678
+ every scope, which is how a platform reads the contexts it runs:
679
+
680
+ ```sql
681
+ SELECT scope, count() FROM encounters FINAL GROUP BY scope
682
+ SELECT visitor, groupUniqArray(identity) FROM encounters FINAL WHERE identity IS NOT NULL GROUP BY visitor
683
+ ```
684
+
685
+ A view with `refresh` is a refreshable materialized view, computed as its definer — the context's
686
+ user — and so of its scope.