@toa.io/extensions.chronicles 1.0.0-alpha.332 → 1.0.0-alpha.335
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +22 -0
- package/components/chronicles.encounters/manifest.toa.yaml +108 -0
- package/components/{chronicles.ingestion → chronicles.encounters}/operations/anonymous.js +1 -1
- package/components/chronicles.encounters/operations/describe.js +4 -0
- package/components/{chronicles.ingestion → chronicles.encounters}/operations/identified.js +1 -1
- package/components/{chronicles.ingestion → chronicles.encounters}/types/toa.d.ts +13 -0
- package/components/chronicles.queries/manifest.toa.yaml +39 -15
- package/components/chronicles.queries/operations/define.js +1 -1
- package/components/chronicles.queries/types/toa.d.ts +8 -3
- package/components/{chronicles.facts → chronicles.relay}/manifest.toa.yaml +14 -3
- package/components/chronicles.relay/operations/encounter.js +4 -0
- package/components/{chronicles.facts → chronicles.relay}/operations/record.js +1 -1
- package/components/{chronicles.facts → chronicles.relay}/types/toa.d.ts +5 -0
- package/guides/clickhouse.md +19 -0
- package/guides/log.md +52 -0
- package/package.json +8 -4
- package/readme.md +459 -142
- package/store/clickhouse.scope.sql +29 -0
- package/store/clickhouse.sql +99 -0
- package/transpiled/Client.d.ts +19 -6
- package/transpiled/Client.js +89 -37
- package/transpiled/Client.js.map +1 -1
- package/transpiled/Factory.d.ts +13 -5
- package/transpiled/Factory.js +32 -23
- package/transpiled/Factory.js.map +1 -1
- package/transpiled/Provider.d.ts +76 -25
- package/transpiled/Provider.js.map +1 -1
- package/transpiled/Publisher.d.ts +50 -0
- package/transpiled/Publisher.js +169 -0
- package/transpiled/Publisher.js.map +1 -0
- package/transpiled/Reader.d.ts +15 -3
- package/transpiled/Reader.js +98 -14
- package/transpiled/Reader.js.map +1 -1
- package/transpiled/Recorder.d.ts +21 -11
- package/transpiled/Recorder.js +49 -24
- package/transpiled/Recorder.js.map +1 -1
- package/transpiled/Relay.d.ts +14 -0
- package/transpiled/Relay.js +27 -0
- package/transpiled/Relay.js.map +1 -0
- package/transpiled/Sender.d.ts +8 -4
- package/transpiled/Sender.js +27 -10
- package/transpiled/Sender.js.map +1 -1
- package/transpiled/Service.d.ts +26 -0
- package/transpiled/Service.js +70 -0
- package/transpiled/Service.js.map +1 -0
- package/transpiled/agent.d.ts +16 -0
- package/transpiled/agent.js +48 -0
- package/transpiled/agent.js.map +1 -0
- package/transpiled/clickhouse/Client.d.ts +5 -8
- package/transpiled/clickhouse/Client.js +12 -14
- package/transpiled/clickhouse/Client.js.map +1 -1
- package/transpiled/clickhouse/Provider.d.ts +20 -18
- package/transpiled/clickhouse/Provider.js +97 -72
- package/transpiled/clickhouse/Provider.js.map +1 -1
- package/transpiled/clickhouse/ddl.d.ts +3 -20
- package/transpiled/clickhouse/ddl.js +6 -79
- package/transpiled/clickhouse/ddl.js.map +1 -1
- package/transpiled/clickhouse/views.d.ts +11 -0
- package/transpiled/clickhouse/views.js +97 -0
- package/transpiled/clickhouse/views.js.map +1 -0
- package/transpiled/contract.d.ts +12 -0
- package/transpiled/contract.js +58 -0
- package/transpiled/contract.js.map +1 -0
- package/transpiled/index.d.ts +9 -3
- package/transpiled/settings.d.ts +6 -4
- package/transpiled/settings.js +27 -14
- package/transpiled/settings.js.map +1 -1
- package/transpiled/store.d.ts +3 -3
- package/transpiled/store.js +3 -3
- package/transpiled/store.js.map +1 -1
- package/components/chronicles.ingestion/manifest.toa.yaml +0 -60
- package/transpiled/Facts.d.ts +0 -9
- package/transpiled/Facts.js +0 -16
- package/transpiled/Facts.js.map +0 -1
- package/transpiled/Scribe.d.ts +0 -25
- package/transpiled/Scribe.js +0 -61
- package/transpiled/Scribe.js.map +0 -1
- package/transpiled/Writer.d.ts +0 -14
- package/transpiled/Writer.js +0 -50
- package/transpiled/Writer.js.map +0 -1
- /package/components/{chronicles.facts → chronicles.encounters}/types/index.d.ts +0 -0
- /package/components/{chronicles.ingestion → chronicles.relay}/types/index.d.ts +0 -0
package/readme.md
CHANGED
|
@@ -1,81 +1,171 @@
|
|
|
1
1
|
# Toa Chronicles
|
|
2
2
|
|
|
3
|
-
A log of what happens in a context
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
through.
|
|
3
|
+
A log of what happens in a context. Components write to it by the events they declare and the
|
|
4
|
+
facts they send, clients by a route of the gateway; the service `chronicles-relay` publishes every
|
|
5
|
+
row of it to an analytics broker, and reads it back from a store, by an
|
|
6
|
+
[HTTP API](#reading-the-log) — which the gateway also publishes as [MCP](#over-mcp) tools.
|
|
8
7
|
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
Two tables.
|
|
8
|
+
Writing and reading meet in nothing Toa does: what takes the rows off the broker into the store,
|
|
9
|
+
and who may read what there, is the [store's](#the-store) own setup.
|
|
12
10
|
|
|
13
|
-
|
|
11
|
+
## The log
|
|
14
12
|
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
| an [event](#events-and-states) a component names | `<component>.<event>` | the entity that changed, and its version | what the event renders |
|
|
18
|
-
| a [fact](#facts) an operation sends | `<component>.<name>` | `NULL` | what was sent |
|
|
19
|
-
| an event a [browser](#from-a-browser) sends | `agent.<name>` | `NULL` | what the browser sent |
|
|
13
|
+
Three tables, each row of which has the `scope` of the context that wrote it — the context's name,
|
|
14
|
+
followed by `TOA_SUFFIX` where one is given — and a reader sees the rows of its own scope.
|
|
20
15
|
|
|
21
|
-
|
|
16
|
+
**`events`** holds a row for everything a component did: an [event](#events-and-states) it
|
|
17
|
+
declares, under `<component>.<event>`, with the entity that changed and its version; and a
|
|
18
|
+
[fact](#facts) it sends, under `<component>.<name>`, with neither.
|
|
22
19
|
|
|
23
|
-
| `label` | `component`
|
|
24
|
-
| ----------------------- |
|
|
25
|
-
| `store.orders.created` | `store.orders`
|
|
26
|
-
| `store.search.searched` | `store.search`
|
|
27
|
-
| `agent.page_left` | `chronicles.ingestion` | `NULL` | `NULL` | `{ path: "/pricing" }` |
|
|
20
|
+
| `label` | `component` | `entity` | `version` | `payload` |
|
|
21
|
+
| ----------------------- | -------------- | -------- | --------- | --------------------------------- |
|
|
22
|
+
| `store.orders.created` | `store.orders` | `9c1f…` | `0` | `{ total: 120, status: "new" }` |
|
|
23
|
+
| `store.search.searched` | `store.search` | `NULL` | `NULL` | `{ query: "teapot", results: 3 }` |
|
|
28
24
|
|
|
29
25
|
Every row also has `id`, the same however often it is delivered, so a row is kept once;
|
|
30
|
-
`identity`, whose it is, where something [says so](#identity); `
|
|
31
|
-
|
|
32
|
-
it; and `trace`, the traceparent of the call that caused it.
|
|
26
|
+
`identity`, whose it is, where something [says so](#identity); `time`, when it
|
|
27
|
+
[happened](#time); and `trace`, the traceparent of the call that caused it.
|
|
33
28
|
|
|
34
|
-
**`states`** holds each entity as it stands: for every component that
|
|
29
|
+
**`states`** holds each entity as it stands: for every component that declares a
|
|
35
30
|
[state](#events-and-states), a row per entity — `component`, `entity`, `version`, `identity`,
|
|
36
|
-
`time
|
|
31
|
+
`time` and `payload` — which a higher version replaces. It is the current state and
|
|
32
|
+
nothing before it: what an entity was yesterday is read from its events, so an event a question
|
|
33
|
+
of history depends on has to carry what that question asks.
|
|
34
|
+
|
|
35
|
+
**`encounters`** holds what [clients send](#encounters): `id`, `name`, `time`, `received`,
|
|
36
|
+
`visitor`, `session`, `identity`, `ip`, `country`, `origin`, `agent`, `browser`, `os`, `device` and
|
|
37
|
+
`payload`.
|
|
37
38
|
|
|
38
39
|
Nothing is ordered: order by `time`, or by `version` within one entity.
|
|
39
40
|
|
|
41
|
+
### Time
|
|
42
|
+
|
|
43
|
+
Every time is the one its source says — when it happened — and none is when the store wrote it:
|
|
44
|
+
|
|
45
|
+
| | `time` |
|
|
46
|
+
| --------- | ----------------------------------------------------------------------------------------- |
|
|
47
|
+
| an event | the payload's [timestamp](#when-a-change-was-made), or when the component made the change |
|
|
48
|
+
| a state | likewise |
|
|
49
|
+
| a fact | when the operation sent it |
|
|
50
|
+
| encounter | when the gateway received it, less how long before sending it the client says it happened |
|
|
51
|
+
|
|
52
|
+
So a change the relay takes an hour late is written at the time it was made, and the hours it
|
|
53
|
+
was away fill in as it catches up: a count of a recent period may still grow. An encounter's
|
|
54
|
+
`received` is when the gateway received its batch.
|
|
55
|
+
|
|
40
56
|
## Events and states
|
|
41
57
|
|
|
58
|
+
A component declares what it logs, and nothing else of it is written:
|
|
59
|
+
|
|
42
60
|
```yaml
|
|
43
61
|
# manifest.toa.yaml
|
|
44
62
|
chronicles:
|
|
45
|
-
events:
|
|
46
|
-
|
|
63
|
+
events:
|
|
64
|
+
paid:
|
|
65
|
+
description: An order's payment completed
|
|
66
|
+
properties:
|
|
67
|
+
total:
|
|
68
|
+
type: integer
|
|
69
|
+
description: The amount paid, in cents
|
|
70
|
+
shipping:
|
|
71
|
+
type: object
|
|
72
|
+
description: Where the order goes
|
|
73
|
+
properties:
|
|
74
|
+
country:
|
|
75
|
+
type: string
|
|
76
|
+
description: ISO 3166 alpha-2
|
|
77
|
+
state:
|
|
78
|
+
event: snapshot
|
|
79
|
+
description: An order as it stands
|
|
80
|
+
properties:
|
|
81
|
+
status:
|
|
82
|
+
type: string
|
|
83
|
+
enum: [new, paid, shipped]
|
|
84
|
+
description: Where the order is in its life
|
|
47
85
|
```
|
|
48
86
|
|
|
49
|
-
Each label is an event of the component — its own, or the prototype's `created`,
|
|
50
|
-
`deleted` and `sync
|
|
51
|
-
|
|
52
|
-
a payload
|
|
87
|
+
Each label under `events` is an event of the component — its own, or the prototype's `created`,
|
|
88
|
+
`updated`, `deleted` and `sync` — and `state.event` is the one whose payload is the entity. An
|
|
89
|
+
event is written as it renders: where its `condition` does not hold, nothing is written; where it
|
|
90
|
+
has a `payload`, what the payload returns is what is written. An event that exists for the log
|
|
91
|
+
alone is an event file like any other, and nothing else has to consume it.
|
|
92
|
+
|
|
93
|
+
### The contract
|
|
94
|
+
|
|
95
|
+
`description` says what the label means, and `properties` what its payload holds, as
|
|
96
|
+
[JSON Schema](https://json-schema.org) of the types [below](#types). Every property, at every
|
|
97
|
+
depth, has a `description`; a declaration that leaves one out, or uses what has no type there, is
|
|
98
|
+
refused when the component is read.
|
|
99
|
+
|
|
100
|
+
What is written is the declared properties of the payload, at every depth, and nothing else: a
|
|
101
|
+
property the declaration leaves out — an email, a document, a field added to the entity since —
|
|
102
|
+
does not reach the store. A payload whose declared property is of another type is not written,
|
|
103
|
+
and is kept in the parking queue of its consumer.
|
|
104
|
+
|
|
105
|
+
A state's payload carries `id` and `VERSION`, which say which entity it is and which version: one
|
|
106
|
+
that does not is refused, and parked likewise. A row of `events` takes its `entity` and `version`
|
|
107
|
+
from the payload's `id` and `VERSION` where it carries them, and is written without them where it
|
|
108
|
+
does not. These, the [timestamp](#when-a-change-was-made) and the [identity](#identity) are read
|
|
109
|
+
before the contract applies, and are not declared. An event that is both is named in both.
|
|
110
|
+
|
|
111
|
+
A committed change is written however long the relay or the broker is away, and once however often it is
|
|
112
|
+
delivered. One committed before the relay first ran in a deployment is not.
|
|
113
|
+
|
|
114
|
+
### Types
|
|
115
|
+
|
|
116
|
+
| declared | read as |
|
|
117
|
+
| ---------------------------------------- | ------------------------ |
|
|
118
|
+
| `string` | `String` |
|
|
119
|
+
| `string` with `enum` | `LowCardinality(String)` |
|
|
120
|
+
| `integer` | `Int64` |
|
|
121
|
+
| `number` | `Float64` |
|
|
122
|
+
| `boolean` | `Bool` |
|
|
123
|
+
| `string`, `format: date-time` | `DateTime64(3, 'UTC')` |
|
|
124
|
+
| `string`, `format: date` | `Date32` |
|
|
125
|
+
| `string`, `format: time` | `Time64(3)` |
|
|
126
|
+
| `string`, `format: uuid`, `ipv4`, `ipv6` | `UUID`, `IPv4`, `IPv6` |
|
|
127
|
+
| `integer`, `format: int32` | `Int32` |
|
|
128
|
+
| `number`, `format: float` | `Float32` |
|
|
129
|
+
| any other `format` | as its `type` |
|
|
130
|
+
| `object` with `properties` | `Tuple` of them, by name |
|
|
131
|
+
| `array` with `items` of any of these | `Array` of it |
|
|
132
|
+
|
|
133
|
+
### Reading what is declared
|
|
134
|
+
|
|
135
|
+
`describe` tells, of every declared field of a label or a state, its type and how to read it from
|
|
136
|
+
the table in that type:
|
|
53
137
|
|
|
54
138
|
```yaml
|
|
55
|
-
|
|
56
|
-
|
|
139
|
+
- label: store.orders.paid
|
|
140
|
+
fields:
|
|
141
|
+
- path: total
|
|
142
|
+
type: Int64
|
|
143
|
+
description: The amount paid, in cents
|
|
144
|
+
read: JSONExtract(ifNull(toJSONString(payload.total), 'null'), 'Nullable(Int64)')
|
|
57
145
|
```
|
|
58
146
|
|
|
59
|
-
|
|
60
|
-
|
|
147
|
+
```sql
|
|
148
|
+
SELECT JSONExtract(ifNull(toJSONString(payload.total), 'null'), 'Nullable(Int64)') AS total, count()
|
|
149
|
+
FROM events FINAL WHERE label = 'store.orders.paid' GROUP BY total
|
|
150
|
+
```
|
|
61
151
|
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
export function payload(event: Event) {
|
|
65
|
-
const { email, address, ...rest } = event.state
|
|
152
|
+
A value that is not of its type reads as `NULL`; the row keeps it as it was written. A contract
|
|
153
|
+
that changes changes what `describe` tells, and the rows written before it are read the same way.
|
|
66
154
|
|
|
67
|
-
|
|
68
|
-
}
|
|
69
|
-
```
|
|
155
|
+
### When a change was made
|
|
70
156
|
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
of `events` takes its `entity`, `version` and `time` from the payload's `id`, `VERSION` and
|
|
74
|
-
`UPDATED` where it carries them, and is written without them, at the time it was taken, where it
|
|
75
|
-
does not. An event that is both is named in both.
|
|
157
|
+
`time` is the payload's `UPDATED`, which the entity as it stands carries, where it is a number,
|
|
158
|
+
and where it is not, the moment the component made the change, which every event carries:
|
|
76
159
|
|
|
77
|
-
|
|
78
|
-
|
|
160
|
+
```yaml
|
|
161
|
+
# manifest.toa.yaml
|
|
162
|
+
chronicles:
|
|
163
|
+
timestamp: placed # the property of the payloads that says when, in milliseconds
|
|
164
|
+
events: …
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
A component names another property, or none with `timestamp: ~`, when the moment of the change
|
|
168
|
+
is always the time. One property serves every event of the component and its state.
|
|
79
169
|
|
|
80
170
|
### Identity
|
|
81
171
|
|
|
@@ -84,9 +174,8 @@ delivered. One committed before the scribe first ran in a deployment is not.
|
|
|
84
174
|
```yaml
|
|
85
175
|
# manifest.toa.yaml
|
|
86
176
|
chronicles:
|
|
87
|
-
events: [created, paid]
|
|
88
|
-
state: snapshot
|
|
89
177
|
identity: customer # the property of the payloads that names whose a row is
|
|
178
|
+
events: …
|
|
90
179
|
```
|
|
91
180
|
|
|
92
181
|
A component names another property, or none with `identity: ~`. One property serves every event of
|
|
@@ -95,12 +184,29 @@ the component names is one of its entity.
|
|
|
95
184
|
|
|
96
185
|
## Facts
|
|
97
186
|
|
|
187
|
+
```yaml
|
|
188
|
+
# manifest.toa.yaml
|
|
189
|
+
chronicles:
|
|
190
|
+
facts:
|
|
191
|
+
searched:
|
|
192
|
+
description: A customer searched the catalogue
|
|
193
|
+
properties:
|
|
194
|
+
query:
|
|
195
|
+
type: string
|
|
196
|
+
description: What was typed
|
|
197
|
+
results:
|
|
198
|
+
type: integer
|
|
199
|
+
description: How many products were found
|
|
200
|
+
```
|
|
201
|
+
|
|
98
202
|
```typescript
|
|
99
203
|
await context.chronicles.send('searched', { query: input.query, results: results.length })
|
|
100
204
|
```
|
|
101
205
|
|
|
102
|
-
The fact is a row of `events` under `<component>.searched`, with what was sent
|
|
103
|
-
`send`
|
|
206
|
+
The fact is a row of `events` under `<component>.searched`, with what was sent of its contract as
|
|
207
|
+
its payload. `send` throws for a name the component does not declare, or a payload a declared
|
|
208
|
+
property of which is of another type; it resolves once the broker has it, and it then waits for
|
|
209
|
+
the relay as an event does.
|
|
104
210
|
|
|
105
211
|
An operation that knows whose the fact is says so; the extension does not know who called it:
|
|
106
212
|
|
|
@@ -112,28 +218,29 @@ await context.chronicles.send(
|
|
|
112
218
|
)
|
|
113
219
|
```
|
|
114
220
|
|
|
115
|
-
`context.chronicles` is given to a component that declares `chronicles`.
|
|
116
|
-
declares it empty:
|
|
117
|
-
|
|
118
|
-
```yaml
|
|
119
|
-
# manifest.toa.yaml
|
|
120
|
-
chronicles: []
|
|
121
|
-
```
|
|
221
|
+
`context.chronicles` is given to a component that declares `chronicles`.
|
|
122
222
|
|
|
123
|
-
##
|
|
223
|
+
## Encounters
|
|
124
224
|
|
|
125
|
-
|
|
225
|
+
What clients send: pages viewed, buttons pressed, anything a client tells of itself. Served where
|
|
226
|
+
the context says `encounters: true`, at two routes: one of an identity, and one of no one. Who may
|
|
227
|
+
send to which is in [Access](#access).
|
|
126
228
|
|
|
127
|
-
### `POST /chronicles/
|
|
229
|
+
### `POST /chronicles/encounters/:identity/`
|
|
128
230
|
|
|
129
231
|
What a signed-in client sends, as the identity it is.
|
|
130
232
|
|
|
131
233
|
```http
|
|
132
|
-
POST /chronicles/
|
|
234
|
+
POST /chronicles/encounters/9c1f0a6d5b7e4f3a8d2c1b0a9e8f7d6c/ HTTP/1.1
|
|
133
235
|
authorization: Token …
|
|
134
236
|
content-type: application/json
|
|
135
237
|
|
|
136
|
-
{
|
|
238
|
+
{
|
|
239
|
+
"sent": 1761200004000,
|
|
240
|
+
"visitor": "7f3a…",
|
|
241
|
+
"session": "01c2…",
|
|
242
|
+
"events": [{ "id": "0192f3a1-…", "name": "page_viewed", "time": 1761200000000, "payload": { "path": "/pricing" } }]
|
|
243
|
+
}
|
|
137
244
|
```
|
|
138
245
|
|
|
139
246
|
```http
|
|
@@ -142,67 +249,143 @@ content-type: application/json
|
|
|
142
249
|
accepted: 1
|
|
143
250
|
```
|
|
144
251
|
|
|
145
|
-
The
|
|
146
|
-
whatever the body says. Throttled to 60 requests a minute per identity.
|
|
252
|
+
The rows are that identity's, whatever the body says.
|
|
147
253
|
|
|
148
|
-
### `POST /chronicles/
|
|
254
|
+
### `POST /chronicles/encounters/`
|
|
149
255
|
|
|
150
256
|
What anyone sends, before signing in or without it. The same body, and rows of no identity; a body
|
|
151
257
|
that names one is refused.
|
|
152
258
|
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
{ "events": [{ "id": "0192f3a2-…", "name": "page_left", "time": 1761200000000 }] }
|
|
158
|
-
```
|
|
159
|
-
|
|
160
|
-
It takes no credentials, and is throttled to 60 requests a minute per address. A form whose
|
|
161
|
-
`events` is their JSON is read as the JSON body, so `navigator.sendBeacon`, which sends no
|
|
162
|
-
`authorization`, can send a batch here as the page closes:
|
|
259
|
+
A form whose `events` is their JSON, beside `sent`, `visitor` and `session`, is read as the JSON
|
|
260
|
+
body, so `navigator.sendBeacon`, which sends no `authorization`, can send a batch here as the page
|
|
261
|
+
closes:
|
|
163
262
|
|
|
164
263
|
```javascript
|
|
165
264
|
navigator.sendBeacon(
|
|
166
|
-
'/chronicles/
|
|
167
|
-
new URLSearchParams({
|
|
265
|
+
'/chronicles/encounters/',
|
|
266
|
+
new URLSearchParams({
|
|
267
|
+
sent: Date.now(),
|
|
268
|
+
visitor,
|
|
269
|
+
session,
|
|
270
|
+
events: JSON.stringify(events)
|
|
271
|
+
})
|
|
168
272
|
)
|
|
169
273
|
```
|
|
170
274
|
|
|
171
|
-
A signed-in client sends with `fetch(…, { keepalive: true })` instead.
|
|
172
|
-
`user` in a payload is what the client said.
|
|
275
|
+
A signed-in client sends with `fetch(…, { keepalive: true })` instead.
|
|
173
276
|
|
|
174
277
|
### The body
|
|
175
278
|
|
|
176
|
-
`{
|
|
279
|
+
`{ sent, visitor?, session?, events }`:
|
|
280
|
+
|
|
281
|
+
| property | |
|
|
282
|
+
| --------- | ---------------------------------------------------------------------------------------------- |
|
|
283
|
+
| `sent` | when the client sends the batch, in milliseconds, by its clock |
|
|
284
|
+
| `visitor` | the client's own id of itself, kept across sessions; at most 64 characters, or `null` for none |
|
|
285
|
+
| `session` | the id of one period of the client's activity; at most 64 characters, or `null` for none |
|
|
286
|
+
| `events` | at most 100 of them |
|
|
287
|
+
|
|
288
|
+
Each event:
|
|
177
289
|
|
|
178
290
|
| property | |
|
|
179
291
|
| --------- | ------------------------------------------------------------------- |
|
|
180
292
|
| `id` | the client's own, at most 64 characters; a retry sends the same one |
|
|
181
293
|
| `name` | lowercase letters, digits and underscores from a letter, at most 64 |
|
|
182
|
-
| `time` | when it happened, in milliseconds, by the client's clock
|
|
294
|
+
| `time` | when it happened, in milliseconds, by the client's clock, by `sent` |
|
|
183
295
|
| `payload` | an object, at most 16 KiB of JSON |
|
|
184
296
|
|
|
297
|
+
The client keeps its `visitor` — in `localStorage`, say — and sends the same one before signing
|
|
298
|
+
in and after. The rows of the route of an identity have both, so the visitors of an identity are
|
|
299
|
+
a query. A visitor is not a person: one identity may have many, and a browser that clears its
|
|
300
|
+
storage is a new one.
|
|
301
|
+
|
|
185
302
|
A request is answered once its events are written. A batch one event of which breaks any of the
|
|
186
303
|
above is written not at all, and answered `422` with the code `INVALID`, or `OVERSIZED` for a
|
|
187
304
|
payload too large, and the `index` of the event in `cause`.
|
|
188
305
|
|
|
306
|
+
A payload is written as it was sent, whatever its shape: a client is deployed once and runs for
|
|
307
|
+
months, so a shape that changes is a new `name` — `checkout_started_v2` — and the old one stays as
|
|
308
|
+
old clients send it.
|
|
309
|
+
|
|
310
|
+
A client's clock is its own, and may be off by anything; what it measures between two of its
|
|
311
|
+
moments is not. So an encounter's `time` is when the gateway received the batch, less `sent`
|
|
312
|
+
minus its own `time`: the gateway's clock, at the distance the client measured. A clock that is
|
|
313
|
+
wrong moves nothing, and a batch kept offline for an hour is written an hour back. A row is kept
|
|
314
|
+
and expires by `received`.
|
|
315
|
+
|
|
189
316
|
### Address and country
|
|
190
317
|
|
|
191
|
-
Every
|
|
192
|
-
|
|
318
|
+
Every encounter has the `ip` and the `country` of the request, as the gateway reads them from the
|
|
319
|
+
headers [`exposition.ip` and `exposition.country`](/extensions/exposition/documentation/ip.md)
|
|
193
320
|
name, and `NULL` where it has none; a body that names either is overwritten. An address is
|
|
194
|
-
personal data, and the log erases nothing, so of an address the log keeps its network — a `/24`
|
|
195
|
-
IPv4, a `/48` of IPv6 — unless the annotation says `ip: full`.
|
|
321
|
+
personal data, and the log erases nothing, so of an address the log keeps its network — a `/24`
|
|
322
|
+
of IPv4, a `/48` of IPv6 — unless the annotation says `ip: full`.
|
|
323
|
+
|
|
324
|
+
### Origin and agent
|
|
325
|
+
|
|
326
|
+
Every encounter has the `origin` of the page that sent it and the client's `agent`, from the
|
|
327
|
+
request's `Origin` and `User-Agent` headers, and `NULL` where it has none; a body that names either
|
|
328
|
+
is overwritten, and a body cannot say one the request does not have. A browser sends `Origin` with
|
|
329
|
+
a request to another origin, and with every `POST`, so it tells which of the context's sites a
|
|
330
|
+
batch came from.
|
|
331
|
+
|
|
332
|
+
What the agent says the client is, is three columns of its own, by name and without versions:
|
|
333
|
+
|
|
334
|
+
| | |
|
|
335
|
+
| --------- | ----------------------------------------------------------------------------- |
|
|
336
|
+
| `browser` | `Edge`, `Opera`, `Samsung Internet`, `Firefox`, `Chrome`, `Safari`, or `NULL` |
|
|
337
|
+
| `os` | `Windows`, `iOS`, `macOS`, `Android`, `ChromeOS`, `Linux`, or `NULL` |
|
|
338
|
+
| `device` | `bot`, `tablet`, `mobile` or `desktop`, and `NULL` where there is no agent |
|
|
339
|
+
|
|
340
|
+
`bot` is a crawler, a headless browser or a program — what a question of who visits leaves out.
|
|
341
|
+
Safari on an iPad says it is a Mac, and is a `desktop` running `macOS` here. A version is read from
|
|
342
|
+
`agent` where a question needs one, knowing that browsers freeze what they say of the system:
|
|
343
|
+
Safari on iOS 26 says iOS 18, and Chrome on Android says Android 10.
|
|
344
|
+
|
|
345
|
+
### What the encounters are
|
|
346
|
+
|
|
347
|
+
What a client sends is told in the configuration of `chronicles.encounters`, by name:
|
|
348
|
+
|
|
349
|
+
```yaml
|
|
350
|
+
# context.toa.yaml
|
|
351
|
+
configuration:
|
|
352
|
+
chronicles.encounters:
|
|
353
|
+
page_viewed:
|
|
354
|
+
description: A page was shown
|
|
355
|
+
fields:
|
|
356
|
+
path: The URL path, without the query
|
|
357
|
+
referrer: The page the visitor came from
|
|
358
|
+
```
|
|
359
|
+
|
|
360
|
+
It is what `describe` tells an agent of each encounter's fields, and what whoever writes a client
|
|
361
|
+
reads:
|
|
362
|
+
|
|
363
|
+
```http
|
|
364
|
+
GET /chronicles/encounters/ HTTP/1.1
|
|
365
|
+
authorization: Token …
|
|
366
|
+
accept: application/yaml
|
|
367
|
+
```
|
|
368
|
+
|
|
369
|
+
```http
|
|
370
|
+
200 OK
|
|
371
|
+
|
|
372
|
+
page_viewed:
|
|
373
|
+
description: A page was shown
|
|
374
|
+
fields:
|
|
375
|
+
path: The URL path, without the query
|
|
376
|
+
referrer: The page the visitor came from
|
|
377
|
+
```
|
|
378
|
+
|
|
379
|
+
It documents, and holds nothing back: a name it does not tell is written as any other.
|
|
196
380
|
|
|
197
381
|
## Reading the log
|
|
198
382
|
|
|
199
|
-
|
|
200
|
-
`system:chronicles` grants both. A refusal is answered `422` with `{ code, cause }`.
|
|
383
|
+
A refusal is answered `422` with `{ code, cause }`. Who may read is in [Access](#access).
|
|
201
384
|
|
|
202
385
|
### `GET /chronicles/queries/`
|
|
203
386
|
|
|
204
|
-
What the log holds:
|
|
205
|
-
with a state, and the views.
|
|
387
|
+
What the log holds: a guide to reading it, the labels of `events`, the components
|
|
388
|
+
with a state, the encounters, and the views.
|
|
206
389
|
|
|
207
390
|
```http
|
|
208
391
|
GET /chronicles/queries/ HTTP/1.1
|
|
@@ -213,29 +396,81 @@ accept: application/yaml
|
|
|
213
396
|
```http
|
|
214
397
|
200 OK
|
|
215
398
|
|
|
216
|
-
|
|
399
|
+
guide: |
|
|
400
|
+
# Reading the log
|
|
401
|
+
|
|
402
|
+
The log is three tables, read with `FINAL`.
|
|
403
|
+
…
|
|
217
404
|
labels:
|
|
218
|
-
- label: store.orders.
|
|
405
|
+
- label: store.orders.paid
|
|
406
|
+
description: An order's payment completed
|
|
219
407
|
rows: 1204
|
|
220
408
|
first: 2026-10-01 08:12:03.000
|
|
221
409
|
last: 2026-10-24 17:40:51.000
|
|
222
|
-
|
|
410
|
+
fields:
|
|
411
|
+
- path: total
|
|
412
|
+
type: Int64
|
|
413
|
+
description: The amount paid, in cents
|
|
414
|
+
read: JSONExtract(ifNull(toJSONString(payload.total), 'null'), 'Nullable(Int64)')
|
|
415
|
+
- path: shipping
|
|
416
|
+
type: Tuple(country String)
|
|
417
|
+
description: Where the order goes
|
|
418
|
+
read: JSONExtract(toJSONString(payload.^shipping), 'Tuple(country Nullable(String))')
|
|
419
|
+
- path: shipping.country
|
|
420
|
+
type: String
|
|
421
|
+
description: ISO 3166 alpha-2
|
|
223
422
|
states:
|
|
224
423
|
- component: store.orders
|
|
424
|
+
description: An order as it stands
|
|
225
425
|
entities: 1180
|
|
226
|
-
|
|
426
|
+
last: 2026-10-24 17:40:51.000
|
|
427
|
+
fields:
|
|
428
|
+
- path: status
|
|
429
|
+
type: LowCardinality(String)
|
|
430
|
+
description: Where the order is in its life
|
|
431
|
+
read: JSONExtract(ifNull(toJSONString(payload.status), 'null'), 'LowCardinality(Nullable(String))')
|
|
432
|
+
encounters:
|
|
433
|
+
- name: page_viewed
|
|
434
|
+
description: A page was shown
|
|
435
|
+
rows: 88123
|
|
436
|
+
first: 2026-10-01 00:00:14.000
|
|
437
|
+
last: 2026-10-24 17:41:02.000
|
|
438
|
+
received: 2026-10-24 17:41:03.020
|
|
439
|
+
fields:
|
|
440
|
+
- path: path
|
|
441
|
+
types: [String]
|
|
442
|
+
description: The URL path, without the query
|
|
443
|
+
- path: utm
|
|
444
|
+
types: [Int64, String]
|
|
227
445
|
views:
|
|
228
|
-
- name:
|
|
229
|
-
|
|
446
|
+
- name: paid_daily
|
|
447
|
+
read: shop.paid_daily
|
|
448
|
+
title: Orders paid per day
|
|
449
|
+
description: Orders whose payment completed, counted by the day of `time`
|
|
450
|
+
sql: SELECT toDate(time) AS day, count() AS paid FROM events FINAL WHERE label = 'store.orders.paid' GROUP BY day
|
|
230
451
|
refresh: 3600
|
|
452
|
+
refreshed: 2026-10-24 17:00:03
|
|
231
453
|
```
|
|
232
454
|
|
|
233
|
-
`
|
|
234
|
-
the
|
|
455
|
+
`guide` is how to read the log — what its tables and columns mean, and what to be careful of —
|
|
456
|
+
and how its store is read, in the store's dialect: what an agent reads before its first query.
|
|
457
|
+
Toa writes it; what is particular to an application is in the descriptions of what it declares
|
|
458
|
+
and documents, and in [`exposition.mcp.instructions`](/extensions/exposition/documentation/mcp.md).
|
|
459
|
+
|
|
460
|
+
A label and a state are told by their declaration, with each field's type and how to
|
|
461
|
+
[read](#reading-what-is-declared) it in that type; a field of a field is read inside the one that
|
|
462
|
+
holds it. A label declared and not yet written is told with no rows. An encounter is
|
|
463
|
+
told by what was sent: the fields of the latest payloads, the types they were sent as — two
|
|
464
|
+
types of one field are two versions of a client — and the description the
|
|
465
|
+
[configuration](#what-the-encounters-are) gives.
|
|
466
|
+
|
|
467
|
+
`rows` counts what the store holds, a row delivered twice possibly twice. `first` and `last` are
|
|
468
|
+
by `time`, the latest change of a state too; an encounter's `received` is the newest batch the
|
|
469
|
+
gateway received, which tells whether clients still send.
|
|
235
470
|
|
|
236
471
|
### `POST /chronicles/queries/`
|
|
237
472
|
|
|
238
|
-
One statement, in the dialect.
|
|
473
|
+
One statement, in the store's dialect, as the `guide` says.
|
|
239
474
|
|
|
240
475
|
```http
|
|
241
476
|
POST /chronicles/queries/ HTTP/1.1
|
|
@@ -243,7 +478,7 @@ authorization: Token …
|
|
|
243
478
|
accept: application/yaml
|
|
244
479
|
content-type: application/yaml
|
|
245
480
|
|
|
246
|
-
sql: SELECT
|
|
481
|
+
sql: SELECT status, count() AS n FROM store_orders GROUP BY status
|
|
247
482
|
```
|
|
248
483
|
|
|
249
484
|
```http
|
|
@@ -251,7 +486,7 @@ sql: SELECT payload.status AS status, count() AS n FROM states FINAL GROUP BY st
|
|
|
251
486
|
|
|
252
487
|
columns:
|
|
253
488
|
- name: status
|
|
254
|
-
type:
|
|
489
|
+
type: Nullable(String)
|
|
255
490
|
- name: n
|
|
256
491
|
type: UInt64
|
|
257
492
|
rows:
|
|
@@ -260,24 +495,26 @@ rows:
|
|
|
260
495
|
truncated: false
|
|
261
496
|
```
|
|
262
497
|
|
|
263
|
-
A query reads the log and nothing else, and cannot write. It runs for at most 30 seconds
|
|
264
|
-
|
|
265
|
-
`FORMAT` out.
|
|
498
|
+
A query reads the log and nothing else, and cannot write. It runs for at most 30 seconds, answers
|
|
499
|
+
at most 10,000 rows and 64 MiB, and a reader runs at most 4 queries at once: past the rows it
|
|
500
|
+
answers what it has, with `truncated: true`; past the rest it is refused. It leaves `FORMAT` out.
|
|
266
501
|
|
|
267
502
|
| code | when |
|
|
268
503
|
| --------- | ----------------------------------------------------------------------- |
|
|
269
504
|
| `REFUSED` | the store refused the statement; `cause.message` is what the store said |
|
|
270
505
|
|
|
271
|
-
### `PUT /chronicles/queries/views/:name
|
|
506
|
+
### `PUT /chronicles/queries/views/:name/`
|
|
272
507
|
|
|
273
508
|
A view over the log, by a statement a query could run. Defining a name again replaces the view.
|
|
274
509
|
|
|
275
510
|
```http
|
|
276
|
-
PUT /chronicles/queries/views/
|
|
511
|
+
PUT /chronicles/queries/views/paid_daily/ HTTP/1.1
|
|
277
512
|
authorization: Token …
|
|
278
513
|
content-type: application/yaml
|
|
279
514
|
|
|
280
|
-
|
|
515
|
+
title: Orders paid per day
|
|
516
|
+
description: Orders whose payment completed, counted by the day of `time`
|
|
517
|
+
sql: SELECT toDate(time) AS day, count() AS paid FROM events FINAL WHERE label = 'store.orders.paid' GROUP BY day
|
|
281
518
|
refresh: 3600
|
|
282
519
|
```
|
|
283
520
|
|
|
@@ -285,9 +522,13 @@ refresh: 3600
|
|
|
285
522
|
204 No Content
|
|
286
523
|
```
|
|
287
524
|
|
|
288
|
-
|
|
289
|
-
`
|
|
290
|
-
|
|
525
|
+
`title`, at most 100 characters, and `description` tell an agent that finds the view what it
|
|
526
|
+
counts. Without `refresh`, a view is computed when it is read. With `refresh`, in seconds and at
|
|
527
|
+
least `3600`, it is computed on that cycle and read as a table, and `describe` tells when it last
|
|
528
|
+
was. A name is lowercase letters, digits and underscores, and a context has at most 50 views. A
|
|
529
|
+
view is kept in the database named after the context's scope, and a query reads it by that name:
|
|
530
|
+
`shop.paid_daily`. Defining one takes a store whose user may create views there; one that may not
|
|
531
|
+
is refused.
|
|
291
532
|
|
|
292
533
|
| code | when |
|
|
293
534
|
| ----------- | -------------------------------------------------------------------- |
|
|
@@ -295,10 +536,10 @@ underscores, and a context has at most 50 views. A query reads a view by its nam
|
|
|
295
536
|
| `RESERVED` | the name is one of the log's tables |
|
|
296
537
|
| `EXHAUSTED` | the context has 50 views, and this is not one of them |
|
|
297
538
|
|
|
298
|
-
### `DELETE /chronicles/queries/views/:name
|
|
539
|
+
### `DELETE /chronicles/queries/views/:name/`
|
|
299
540
|
|
|
300
541
|
```http
|
|
301
|
-
DELETE /chronicles/queries/views/
|
|
542
|
+
DELETE /chronicles/queries/views/paid_daily/ HTTP/1.1
|
|
302
543
|
authorization: Token …
|
|
303
544
|
```
|
|
304
545
|
|
|
@@ -315,55 +556,131 @@ authorization: Token …
|
|
|
315
556
|
Where the context publishes [MCP](/extensions/exposition/documentation/mcp.md), each of these
|
|
316
557
|
routes is a tool, under the name the gateway gives a route, and taking what the route takes:
|
|
317
558
|
|
|
318
|
-
| tool | route
|
|
319
|
-
| --------------------------------------- |
|
|
320
|
-
| `chronicles.queries.GET` | `GET /chronicles/queries/`
|
|
321
|
-
| `chronicles.queries.POST` | `POST /chronicles/queries/`
|
|
322
|
-
| `chronicles.queries.views._name.PUT` | `PUT /chronicles/queries/views/:name
|
|
323
|
-
| `chronicles.queries.views._name.DELETE` | `DELETE /chronicles/queries/views/:name
|
|
559
|
+
| tool | route |
|
|
560
|
+
| --------------------------------------- | ----------------------------------------- |
|
|
561
|
+
| `chronicles.queries.GET` | `GET /chronicles/queries/` |
|
|
562
|
+
| `chronicles.queries.POST` | `POST /chronicles/queries/` |
|
|
563
|
+
| `chronicles.queries.views._name.PUT` | `PUT /chronicles/queries/views/:name/` |
|
|
564
|
+
| `chronicles.queries.views._name.DELETE` | `DELETE /chronicles/queries/views/:name/` |
|
|
565
|
+
| `chronicles.encounters.GET` | `GET /chronicles/encounters/` |
|
|
566
|
+
|
|
567
|
+
A refusal is a tool result with `isError`.
|
|
568
|
+
|
|
569
|
+
## Access
|
|
570
|
+
|
|
571
|
+
| route | who may call | limit |
|
|
572
|
+
| ----------------------------------------- | ------------------------------------ | --------------------------- |
|
|
573
|
+
| `POST /chronicles/encounters/` | anyone, with no credentials | 60 a minute per address |
|
|
574
|
+
| `POST /chronicles/encounters/:identity/` | the identity, by its own credentials | 60 a minute per identity |
|
|
575
|
+
| `GET /chronicles/encounters/` | `system:chronicles:encounters` | |
|
|
576
|
+
| `GET /chronicles/queries/` | `system:chronicles:query` | |
|
|
577
|
+
| `POST /chronicles/queries/` | `system:chronicles:query` | the [reader's](#the-reader) |
|
|
578
|
+
| `PUT /chronicles/queries/views/:name/` | `system:chronicles:views` | 50 views |
|
|
579
|
+
| `DELETE /chronicles/queries/views/:name/` | `system:chronicles:views` | |
|
|
580
|
+
|
|
581
|
+
A caller without what a route takes is answered `403`, and past its limit `429`. Over
|
|
582
|
+
[MCP](#over-mcp), a tool is listed to a caller that holds its role, and to no one else.
|
|
583
|
+
|
|
584
|
+
### Roles
|
|
324
585
|
|
|
325
|
-
|
|
586
|
+
| | |
|
|
587
|
+
| ------------------------------ | ------------------------------------------------------------------------------------------- |
|
|
588
|
+
| `system:chronicles:query` | reads every row of the log: the identities, the networks, the origins and agents of clients |
|
|
589
|
+
| `system:chronicles:views` | defines and drops the views every reader reads; a view can read only what a query can |
|
|
590
|
+
| `system:chronicles:encounters` | reads what the configuration documents of encounters, and no row |
|
|
591
|
+
| `system:chronicles` | all three |
|
|
592
|
+
|
|
593
|
+
`query` is access to personal data, and is granted as that. `views` changes what other readers are
|
|
594
|
+
told a view counts, and changes nothing of the log. `encounters` is for whoever writes a client.
|
|
595
|
+
A role is granted as any [role](/extensions/exposition/documentation/access.md) is.
|
|
596
|
+
|
|
597
|
+
### What a client cannot say
|
|
598
|
+
|
|
599
|
+
A client sends events and nothing that says who or where it is: `identity` is the path's, which the
|
|
600
|
+
credentials have to be the identity's; `ip`, `country`, `origin` and `agent` are what the gateway
|
|
601
|
+
read off the request, and a body that names one is overwritten, or refused where it names
|
|
602
|
+
`identity` on the route of no one. Anything else in a payload is what the client said — a `user`
|
|
603
|
+
there is not a user — so a question that depends on who sent an encounter asks `identity`.
|
|
604
|
+
|
|
605
|
+
### The reader
|
|
606
|
+
|
|
607
|
+
A query runs as the context's user of the store, which reads what the [store](#the-store) lets it
|
|
608
|
+
— in the reference setup, the rows of its scope and the views of its database — and is never let
|
|
609
|
+
lift its limits there. Toa asks a query to be read-only besides, so a query can change nothing even
|
|
610
|
+
of what its user may: only `define` and `drop` create and remove views.
|
|
326
611
|
|
|
327
612
|
## Deployment
|
|
328
613
|
|
|
329
614
|
```yaml
|
|
330
615
|
# context.toa.yaml
|
|
331
616
|
chronicles:
|
|
617
|
+
broker: amqp://analytics.internal
|
|
332
618
|
store:
|
|
333
619
|
provider: clickhouse
|
|
334
|
-
url:
|
|
620
|
+
url: http://analytics.internal:8123
|
|
335
621
|
retention: 730
|
|
336
|
-
|
|
622
|
+
encounters: true
|
|
337
623
|
```
|
|
338
624
|
|
|
339
|
-
|
|
|
340
|
-
|
|
|
341
|
-
| `
|
|
342
|
-
| `
|
|
343
|
-
| `
|
|
344
|
-
| `
|
|
625
|
+
| | |
|
|
626
|
+
| ------------ | ----------------------------------------------------------------------------------- |
|
|
627
|
+
| `broker` | where the rows are published |
|
|
628
|
+
| `store` | where they are read: the provider, and the address of its server |
|
|
629
|
+
| `retention` | days a row of `events` and of `encounters` is kept; absent, kept |
|
|
630
|
+
| `encounters` | the routes clients send to; absent, there are none |
|
|
631
|
+
| `ip` | `full` keeps a client's whole address; absent, its network |
|
|
632
|
+
| `resources` | what the service may take, as [`resources`](/migrations/262.md) is stated elsewhere |
|
|
345
633
|
|
|
346
634
|
```shell
|
|
347
|
-
$ toa conceal chronicles
|
|
635
|
+
$ toa conceal chronicles broker_username=… broker_password=… store_username=… store_password=…
|
|
348
636
|
```
|
|
349
637
|
|
|
350
|
-
The
|
|
351
|
-
|
|
638
|
+
The service `chronicles-relay` alone is given the secret.
|
|
639
|
+
|
|
640
|
+
The events the components declare travel where the context's `amqp` says, as every event does, and
|
|
641
|
+
the relay takes them from there. A fact and an encounter reach it as tasks. Each is acknowledged
|
|
642
|
+
once the broker has confirmed its row, so while the broker is away they wait where they are, and
|
|
643
|
+
are published once it is back; the relay starts and is ready without it.
|
|
644
|
+
|
|
645
|
+
## The store
|
|
646
|
+
|
|
647
|
+
What Toa publishes and what it reads; what takes the one into the other is not Toa's.
|
|
648
|
+
|
|
649
|
+
**Published:** one JSON row a message, persistent, to the exchange `chronicles` (`direct`) of the
|
|
650
|
+
broker, under the key of its table — `events`, `states` or `encounters`. A row has the columns of
|
|
651
|
+
its table:
|
|
652
|
+
|
|
653
|
+
| table | row |
|
|
654
|
+
| ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
655
|
+
| `events` | `scope`, `id`, `label`, `component`, `time`, `entity`, `version`, `identity`, `trace`, `payload`, `expires` |
|
|
656
|
+
| `states` | `scope`, `component`, `entity`, `version`, `identity`, `time`, `payload` |
|
|
657
|
+
| `encounters` | `scope`, `id`, `name`, `time`, `received`, `visitor`, `session`, `identity`, `ip`, `country`, `origin`, `agent`, `browser`, `os`, `device`, `payload`, `expires` |
|
|
658
|
+
|
|
659
|
+
A time is `YYYY-MM-DD hh:mm:ss.sss`, in UTC; `expires` is `time` — an encounter's `received` — and
|
|
660
|
+
`retention` days, or `2106-01-01 00:00:00` where there is none. A row may be delivered more than
|
|
661
|
+
once, with the same `id`. The broker's user needs to declare and to write the exchange `chronicles`,
|
|
662
|
+
and nothing else.
|
|
352
663
|
|
|
353
|
-
|
|
664
|
+
**Read:** `events`, `states` and `encounters`, with those columns, in the default database of the
|
|
665
|
+
`store` user, and a database named after the scope for its views.
|
|
354
666
|
|
|
355
|
-
|
|
667
|
+
The relay writes the scope of its context into every row, and nothing else does: the broker tells
|
|
668
|
+
the store nothing of who published a row. What keeps a context from reading another's rows is the
|
|
669
|
+
store's.
|
|
356
670
|
|
|
357
|
-
|
|
358
|
-
not exist. The user needs `CREATE`, `INSERT` and `access_management`; the reader is
|
|
359
|
-
`<database>_reader`.
|
|
671
|
+
### ClickHouse
|
|
360
672
|
|
|
361
|
-
`
|
|
362
|
-
which
|
|
673
|
+
[`store/clickhouse.sql`](./store/clickhouse.sql) sets a store up: the database `chronicles` with the
|
|
674
|
+
three tables, which are replaced by key in the background — so a query reads them with `FINAL`,
|
|
675
|
+
which collapses what was delivered twice — and a RabbitMQ engine table and a view for each, which
|
|
676
|
+
take the rows off the broker. [`store/clickhouse.scope.sql`](./store/clickhouse.scope.sql) adds the
|
|
677
|
+
user of a context: its row policy, its limits and its database. A user with no row policy reads
|
|
678
|
+
every scope, which is how a platform reads the contexts it runs:
|
|
363
679
|
|
|
364
680
|
```sql
|
|
365
|
-
SELECT
|
|
366
|
-
SELECT
|
|
681
|
+
SELECT scope, count() FROM encounters FINAL GROUP BY scope
|
|
682
|
+
SELECT visitor, groupUniqArray(identity) FROM encounters FINAL WHERE identity IS NOT NULL GROUP BY visitor
|
|
367
683
|
```
|
|
368
684
|
|
|
369
|
-
A view with `refresh` is a refreshable materialized view
|
|
685
|
+
A view with `refresh` is a refreshable materialized view, computed as its definer — the context's
|
|
686
|
+
user — and so of its scope.
|