chiltepin 0.47.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +249 -0
- package/dist/bin.js +3582 -0
- package/dist/bin.js.map +1 -0
- package/package.json +93 -0
- package/templates/chiltepin.config.json +5 -0
- package/templates/demo.md +2161 -0
- package/templates/docs/getting-started.md +155 -0
- package/templates/docs/tutorial.md +559 -0
- package/templates/skill/SKILL.md +172 -0
- package/templates/skill/reference/blocks/INDEX.md +141 -0
- package/templates/skill/reference/blocks/agentic.md +63 -0
- package/templates/skill/reference/blocks/algorithms.md +49 -0
- package/templates/skill/reference/blocks/api.md +40 -0
- package/templates/skill/reference/blocks/architecture.md +94 -0
- package/templates/skill/reference/blocks/business.md +70 -0
- package/templates/skill/reference/blocks/charts-overviews.md +74 -0
- package/templates/skill/reference/blocks/data-model.md +34 -0
- package/templates/skill/reference/blocks/design-system.md +50 -0
- package/templates/skill/reference/blocks/flows.md +74 -0
- package/templates/skill/reference/blocks/narrative.md +65 -0
- package/templates/skill/reference/blocks/planning.md +74 -0
- package/templates/skill/reference/blocks/quality.md +43 -0
- package/templates/skill/reference/blocks/tables-data.md +55 -0
- package/templates/skill/reference/check.md +62 -0
- package/templates/skill/reference/decks.md +198 -0
- package/templates/skill/reference/exemplars/adr.md +87 -0
- package/templates/skill/reference/exemplars/agent-system.md +113 -0
- package/templates/skill/reference/exemplars/api-reference.md +110 -0
- package/templates/skill/reference/exemplars/backend-arch.md +117 -0
- package/templates/skill/reference/exemplars/data-pipeline.md +107 -0
- package/templates/skill/reference/exemplars/frontend-arch.md +93 -0
- package/templates/skill/reference/exemplars/incident-postmortem.md +93 -0
- package/templates/skill/reference/exemplars/migration-plan.md +95 -0
- package/templates/skill/reference/exemplars/onboarding.md +78 -0
- package/templates/skill/reference/exemplars/product-spec.md +81 -0
- package/templates/skill/reference/intake.md +140 -0
- package/templates/skill/reference/mermaid.md +216 -0
- package/templates/skill/reference/organizing.md +118 -0
- package/templates/skill/reference/patterns-design.md +59 -0
- package/templates/skill/reference/patterns.md +167 -0
- package/templates/skill/reference/recipes.md +153 -0
- package/templates/skill/reference/style-ste.md +119 -0
- package/templates/skill/reference/system-design.md +161 -0
- package/templates/skill/reference/writing.md +132 -0
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
```meta
|
|
2
|
+
title: Trips pipeline
|
|
3
|
+
subtitle: How 2.1 billion daily tracker pings become the trips Loxley bills and scores.
|
|
4
|
+
tag: Data · v3
|
|
5
|
+
```
|
|
6
|
+
|
|
7
|
+
Loxley sells fleet telematics: 140,000 delivery vehicles carry a tracker that
|
|
8
|
+
reports position and speed every few seconds. Trackers buffer offline — tunnels,
|
|
9
|
+
depots, dead zones — so pings arrive up to 48 hours late. The pipeline therefore
|
|
10
|
+
streams with a 48-hour dedupe window keyed on `(vehicle_id, recorded_at)`.
|
|
11
|
+
Staleness has a user-facing price: fleet managers review yesterday's trips at
|
|
12
|
+
07:00 local, and per-mile invoices draw on the same rows.
|
|
13
|
+
|
|
14
|
+
## From pings to trips
|
|
15
|
+
|
|
16
|
+
```dfd
|
|
17
|
+
id: ex-pipeline-dfd
|
|
18
|
+
nodes:
|
|
19
|
+
- { id: fleet, col: 1, row: 1, kind: external, name: Tracker fleet }
|
|
20
|
+
- { id: ingest, col: 2, row: 1, kind: process, name: Ingest gateway, num: 1 }
|
|
21
|
+
- { id: raw, col: 3, row: 1, kind: store, name: pings.raw · Kafka }
|
|
22
|
+
- { id: dlq, col: 2, row: 2, kind: store, name: pings.dlq }
|
|
23
|
+
- { id: dedupe, col: 4, row: 1, kind: process, name: Deduper, num: 2 }
|
|
24
|
+
- { id: sess, col: 5, row: 1, kind: process, name: Sessionizer, num: 3 }
|
|
25
|
+
- { id: trips, col: 6, row: 1, kind: store, name: trips-db · Postgres }
|
|
26
|
+
edges:
|
|
27
|
+
- { from: fleet, to: ingest, label: pings }
|
|
28
|
+
- { from: ingest, to: raw }
|
|
29
|
+
- { from: ingest, to: dlq, label: malformed }
|
|
30
|
+
- { from: raw, to: dedupe }
|
|
31
|
+
- { from: dedupe, to: sess, label: unique pings }
|
|
32
|
+
- { from: sess, to: trips, label: closed trips }
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
The sessionizer is the lossy stage by design: it closes a trip after five idle
|
|
36
|
+
minutes. That collapses 2.1 billion pings into about 9.4 million trips a day. Raw
|
|
37
|
+
pings expire after 30 days; safety scores, invoices, and dashboards all read
|
|
38
|
+
trips, never pings.
|
|
39
|
+
|
|
40
|
+
## Trips at rest
|
|
41
|
+
|
|
42
|
+
```erd
|
|
43
|
+
id: ex-pipeline-erd
|
|
44
|
+
entities:
|
|
45
|
+
- name: vehicles
|
|
46
|
+
columns:
|
|
47
|
+
- { name: id, type: uuid, pk: true }
|
|
48
|
+
- { name: fleet_id, type: uuid }
|
|
49
|
+
- { name: tracker_serial, type: text }
|
|
50
|
+
- name: trips
|
|
51
|
+
columns:
|
|
52
|
+
- { name: id, type: uuid, pk: true }
|
|
53
|
+
- { name: vehicle_id, type: uuid, fk: true }
|
|
54
|
+
- { name: started_at, type: timestamptz }
|
|
55
|
+
- { name: ended_at, type: timestamptz }
|
|
56
|
+
- { name: distance_m, type: int }
|
|
57
|
+
- name: trip_points
|
|
58
|
+
columns:
|
|
59
|
+
- { name: trip_id, type: uuid, fk: true }
|
|
60
|
+
- { name: recorded_at, type: timestamptz }
|
|
61
|
+
- { name: speed_kph, type: smallint }
|
|
62
|
+
relations:
|
|
63
|
+
- vehicles ||--o{ trips: drives
|
|
64
|
+
- trips ||--o{ trip_points: samples
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
`distance_m` is computed once, when the trip closes, and never recomputed — an
|
|
68
|
+
invoice printed in March must match the trip row it was billed from.
|
|
69
|
+
|
|
70
|
+
## Replaying a gap
|
|
71
|
+
|
|
72
|
+
```steps
|
|
73
|
+
id: ex-pipeline-replay
|
|
74
|
+
items:
|
|
75
|
+
- title: Size the gap
|
|
76
|
+
body: Compare the ingest tally against the deduper's output for the affected hours.
|
|
77
|
+
code: lox lag --topic pings.raw --by hour
|
|
78
|
+
lang: bash
|
|
79
|
+
- title: Replay inside the window
|
|
80
|
+
body: The deduper drops every ping it has already seen, so a bounded replay is safe.
|
|
81
|
+
code: lox replay --topic pings.raw --from 2026-08-21T06:00Z --to 2026-08-21T09:00Z
|
|
82
|
+
lang: bash
|
|
83
|
+
note: Give both bounds; the tool refuses an unbounded replay.
|
|
84
|
+
- title: Verify counts
|
|
85
|
+
body: Close the incident only when trip counts match the ingest tally within 0.01%.
|
|
86
|
+
code: lox verify trips --day 2026-08-21
|
|
87
|
+
lang: bash
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
```callout
|
|
91
|
+
tone: danger
|
|
92
|
+
title: Never replay past 48 hours
|
|
93
|
+
body: "Beyond the dedupe window the deduper has forgotten the pings, so a replay re-creates trips that invoices already used — miles get billed twice. For older gaps run `lox rebuild --day`, which deletes and re-sessionizes whole days atomically."
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
## Freshness commitments
|
|
97
|
+
|
|
98
|
+
```slo
|
|
99
|
+
id: ex-pipeline-slo
|
|
100
|
+
items:
|
|
101
|
+
- { name: Freshness, sli: Ping visible as trip data within 15 min, target: 99%, current: 99.4%, window: 30d, budget: 0.6 }
|
|
102
|
+
- { name: Completeness, sli: Trip closed within 48 h of its last ping, target: 99.9%, current: 99.95%, window: 30d, budget: 0.5 }
|
|
103
|
+
- { name: Scoring latency, sli: Safety score updated within 24 h of trip close, target: 99.5%, current: 99.1%, window: 7d, budget: 1.8 }
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
The scoring objective is breached at 1.8× budget: the scorer re-shards this
|
|
107
|
+
week, and feature work on it stays frozen until the budget recovers.
|
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
```meta
|
|
2
|
+
title: Terrastride field app
|
|
3
|
+
subtitle: The offline-first inspection app wind-turbine technicians run on tower phones.
|
|
4
|
+
tag: Frontend · SPA
|
|
5
|
+
```
|
|
6
|
+
|
|
7
|
+
Terrastride is a React SPA behind a service worker, not an SSR app.
|
|
8
|
+
Technicians spend up to 9 hours offline inside a tower, and 60% of sessions
|
|
9
|
+
touch the network zero times. Server rendering buys nothing when there is no
|
|
10
|
+
server to reach. The costs we accept: a 180 KB gzip bundle budget on first
|
|
11
|
+
load, and every read and write goes through IndexedDB.
|
|
12
|
+
|
|
13
|
+
## Module tree
|
|
14
|
+
|
|
15
|
+
```frontend
|
|
16
|
+
id: ex-frontend-arch-tree
|
|
17
|
+
nodes:
|
|
18
|
+
- { id: app, kind: root, name: App }
|
|
19
|
+
- { id: sync, parent: app, kind: provider, name: SyncProvider, note: owns the network }
|
|
20
|
+
- { id: shell, parent: app, kind: layout, name: Shell }
|
|
21
|
+
- { id: turbines, parent: shell, kind: page, name: TurbineListPage }
|
|
22
|
+
- { id: turbine, parent: shell, kind: page, name: TurbineDetailPage }
|
|
23
|
+
- { id: inspections, parent: shell, kind: page, name: InspectionListPage }
|
|
24
|
+
- { id: inspection, parent: shell, kind: page, name: InspectionPage }
|
|
25
|
+
- { id: syncpage, parent: shell, kind: page, name: SyncPage }
|
|
26
|
+
- { id: checklist, parent: inspection, kind: component, name: ChecklistForm }
|
|
27
|
+
- { id: photo, parent: inspection, kind: component, name: PhotoCapture }
|
|
28
|
+
- { id: outbox, parent: sync, kind: store, name: outboxStore, note: IndexedDB queue }
|
|
29
|
+
- { id: useoutbox, parent: checklist, kind: hook, name: useOutbox }
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
The rule the tree must keep: pages read IndexedDB and nothing else.
|
|
33
|
+
`SyncProvider` is the only module that imports the network layer, so "does
|
|
34
|
+
this work offline?" is answered by the import graph, not by testing every
|
|
35
|
+
screen.
|
|
36
|
+
|
|
37
|
+
## The inspection screen
|
|
38
|
+
|
|
39
|
+
```wireframe
|
|
40
|
+
id: ex-frontend-arch-screen
|
|
41
|
+
screens:
|
|
42
|
+
- device: phone
|
|
43
|
+
title: "T-114 · Gearbox"
|
|
44
|
+
label: InspectionPage — offline, 4 writes queued
|
|
45
|
+
elements:
|
|
46
|
+
- { type: header, label: "Gearbox inspection" }
|
|
47
|
+
- { type: badge, label: "offline · 4 queued", tone: muted, align: r }
|
|
48
|
+
- { type: card, rows: 3 }
|
|
49
|
+
- { type: input, label: Torque reading }
|
|
50
|
+
- { type: button, label: Add photo }
|
|
51
|
+
- { type: button, label: Complete inspection, tone: accent }
|
|
52
|
+
- { type: tabs, label: "Turbines, Inspections, Sync" }
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
The queued badge is the only sync UI on this screen. Sync state stays
|
|
56
|
+
ambient, and the technician never leaves the checklist to check on it.
|
|
57
|
+
|
|
58
|
+
## Record lifecycle
|
|
59
|
+
|
|
60
|
+
```state
|
|
61
|
+
id: ex-frontend-arch-lifecycle
|
|
62
|
+
states:
|
|
63
|
+
- { id: s0, col: 1, row: 1, kind: start }
|
|
64
|
+
- { id: draft, col: 2, row: 1, kind: active, name: DRAFT }
|
|
65
|
+
- { id: queued, col: 3, row: 1, kind: wait, name: QUEUED }
|
|
66
|
+
- { id: syncing, col: 4, row: 1, kind: active, name: SYNCING }
|
|
67
|
+
- { id: synced, col: 5, row: 1, kind: terminal, name: SYNCED }
|
|
68
|
+
- { id: conflict, col: 4, row: 2, kind: wait, name: CONFLICT }
|
|
69
|
+
transitions:
|
|
70
|
+
- { from: s0, to: draft, event: open inspection }
|
|
71
|
+
- { from: draft, to: queued, event: technician taps Complete }
|
|
72
|
+
- { from: queued, to: syncing, event: radio regained, guard: outbox not empty }
|
|
73
|
+
- { from: syncing, to: synced, event: server ack }
|
|
74
|
+
- { from: syncing, to: conflict, event: server version newer }
|
|
75
|
+
- { from: conflict, to: queued, event: technician merges on SyncPage }
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
Records sit in QUEUED longest — median 3.4 hours, the rest of the tower
|
|
79
|
+
visit. The UI treats QUEUED as success: the badge counts quietly, and a
|
|
80
|
+
queued record never blocks starting the next inspection.
|
|
81
|
+
|
|
82
|
+
## Routes
|
|
83
|
+
|
|
84
|
+
```table
|
|
85
|
+
id: ex-frontend-arch-routes
|
|
86
|
+
columns: [Route, Screen, Offline behavior]
|
|
87
|
+
rows:
|
|
88
|
+
- ["/turbines", TurbineListPage, Served from cache — never blocks on network]
|
|
89
|
+
- ["/turbines/:id", TurbineDetailPage, "Cache first, silent refetch when radio returns"]
|
|
90
|
+
- ["/inspections", InspectionListPage, "Local list from cache + outbox — queued records included"]
|
|
91
|
+
- ["/inspections/:id", InspectionPage, All writes go to the outbox]
|
|
92
|
+
- ["/sync", SyncPage, "Queue, conflicts, manual retry — the only network-aware UI"]
|
|
93
|
+
```
|
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
```meta
|
|
2
|
+
title: SEV-1 · 2026-07-14 — duplicate campaign sends
|
|
3
|
+
subtitle: A 60-second visibility timeout met a 90-second batch job; 1.9M subscribers got the same email twice.
|
|
4
|
+
tag: Postmortem · SEV-1
|
|
5
|
+
```
|
|
6
|
+
|
|
7
|
+
Quillfeed sends about 62 million campaign emails a day through workers that
|
|
8
|
+
consume send-batch jobs from a queue. On 14 July a config refactor silently
|
|
9
|
+
dropped the send queue's visibility-timeout override, and the broker began
|
|
10
|
+
redelivering jobs that were still in flight. All times are UTC.
|
|
11
|
+
|
|
12
|
+
## Timeline
|
|
13
|
+
|
|
14
|
+
```timeline
|
|
15
|
+
id: ex-incident-timeline
|
|
16
|
+
items:
|
|
17
|
+
- "14:12 · Deploy ships · The queue-defaults refactor drops the send queue's 15-minute visibility override to the new 60 s default"
|
|
18
|
+
- "14:22 · First duplicates · The broker redelivers in-flight batch jobs; second workers re-send them"
|
|
19
|
+
- "14:26 · Support signal · Duplicate-email tickets spike; no alert has fired"
|
|
20
|
+
- "14:29 · Pager · The esp-accept-rate anomaly alert fires at 2.3× forecast"
|
|
21
|
+
- "14:37 · Mitigation · On-call correlates with the 14:12 deploy and pauses all send queues"
|
|
22
|
+
- "14:41 · Rollback · Config revert restores the 15-minute timeout"
|
|
23
|
+
- "14:52 · Resume · Sends restart after in-flight jobs are checked against the send ledger"
|
|
24
|
+
- "15:03 · Resolved · Send rate returns to forecast; SEV closed"
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Detection was the slow half: eleven minutes passed between the first customer
|
|
28
|
+
ticket and the queue pause, because no dashboard tied send-rate anomalies to
|
|
29
|
+
deploys. Support saw the incident three minutes before the pager did.
|
|
30
|
+
|
|
31
|
+
## The mechanism
|
|
32
|
+
|
|
33
|
+
```sequence
|
|
34
|
+
id: ex-incident-seq
|
|
35
|
+
actors:
|
|
36
|
+
- { id: Broker, name: Broker, sub: send queue }
|
|
37
|
+
- { id: A, name: Worker A }
|
|
38
|
+
- { id: B, name: Worker B }
|
|
39
|
+
- { id: ESP, name: ESP, sub: email provider, external: true }
|
|
40
|
+
messages:
|
|
41
|
+
- { from: Broker, to: A, label: deliver batch 4411, summary: "500 recipients; a batch takes about 90 seconds end to end." }
|
|
42
|
+
- { from: A, to: A, kind: note, label: ledger check passes, summary: "None of the 500 recipients has a send record yet." }
|
|
43
|
+
- { from: A, to: ESP, label: send 500 emails }
|
|
44
|
+
- { from: Broker, to: B, label: redeliver 4411, kind: error, summary: "No ack after 60 seconds — the new default timeout — so the broker assumes Worker A died." }
|
|
45
|
+
- { from: B, to: B, kind: note, label: ledger check passes again, summary: "Worker A records sends only after the ESP accepts, so the ledger still shows nothing." }
|
|
46
|
+
- { from: B, to: ESP, label: send the same 500 }
|
|
47
|
+
- { from: A, to: Broker, label: ack — 30 s late, kind: response }
|
|
48
|
+
foot:
|
|
49
|
+
- { label: Root cause, value: timeout 60 s < batch 90 s }
|
|
50
|
+
- { label: Amplifier, value: ledger written after the send }
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
The ledger made sends look idempotent without making them idempotent. Workers
|
|
54
|
+
checked it before the send but wrote it only after the ESP accepted — a
|
|
55
|
+
90-second window in which the check lies. The timeout change did not create
|
|
56
|
+
that window; it built a machine that hit it on every batch.
|
|
57
|
+
|
|
58
|
+
## Impact
|
|
59
|
+
|
|
60
|
+
```stats
|
|
61
|
+
id: ex-incident-impact
|
|
62
|
+
stats:
|
|
63
|
+
- { value: 1.9M, label: Duplicate emails, delta: "3.1% of daily volume" }
|
|
64
|
+
- { value: 41 min, label: Impact to resolution, delta: "detection took 7 of them" }
|
|
65
|
+
- { value: "214", label: Campaigns affected }
|
|
66
|
+
- { value: 3.1×, label: Unsubscribe rate on affected campaigns, trend: up, delta: vs. baseline }
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
## Remediation
|
|
70
|
+
|
|
71
|
+
```statustable
|
|
72
|
+
id: ex-incident-remediation
|
|
73
|
+
columns: [Action, Owner, Done / due]
|
|
74
|
+
statuses:
|
|
75
|
+
- { label: shipped, color: success }
|
|
76
|
+
- { label: scheduled, color: neutral }
|
|
77
|
+
rows:
|
|
78
|
+
- { cells: ["Send ledger written before the ESP call; failed writes reconciled hourly", Delivery, 2026-07-16], status: shipped }
|
|
79
|
+
- { cells: ["Visibility timeouts pinned per queue by a config test that fails on defaults", Platform, 2026-07-21], status: shipped }
|
|
80
|
+
- { cells: [Deploy markers on every send-rate dashboard, Observability, 2026-08-01], status: in progress }
|
|
81
|
+
- { cells: ["Duplicate-send canary — pages when the duplicate rate passes 0.1%", Delivery, 2026-08-15], status: scheduled }
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
## Takeaways
|
|
85
|
+
|
|
86
|
+
```takeaways
|
|
87
|
+
id: ex-incident-takeaways
|
|
88
|
+
items:
|
|
89
|
+
- Record intent before the side effect — a ledger written after the send is a race, not a guarantee.
|
|
90
|
+
- Defaults are code — the 15-minute override lived in config nobody tested, and a refactor deleted it silently.
|
|
91
|
+
- text: Users out-detect dashboards on duplicates
|
|
92
|
+
detail: Support tickets led the pager by three minutes; a duplicate-rate canary closes that gap.
|
|
93
|
+
```
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
```meta
|
|
2
|
+
title: Orders store migration
|
|
3
|
+
subtitle: Moving 480M orders off the monolith database, with a rollback path at every phase.
|
|
4
|
+
tag: Plan · Q3–Q4
|
|
5
|
+
```
|
|
6
|
+
|
|
7
|
+
Hollybank's `orders` table is 2.3 TB inside the monolith's shared Postgres.
|
|
8
|
+
Write p95 is 210 ms and climbs about 8 ms a month; autovacuum now runs
|
|
9
|
+
11 hours and blocks schema changes. We move orders to a dedicated cluster,
|
|
10
|
+
partitioned by month, behind the existing `OrdersRepo` interface — application
|
|
11
|
+
code does not change. Every phase is reversible until the old copy is
|
|
12
|
+
destroyed, and that happens no earlier than 30 days after cutover.
|
|
13
|
+
|
|
14
|
+
## Today and target
|
|
15
|
+
|
|
16
|
+
```cvt
|
|
17
|
+
id: ex-migration-cvt
|
|
18
|
+
current:
|
|
19
|
+
label: Monolith DB
|
|
20
|
+
items:
|
|
21
|
+
- orders plus 61 other tables in one Postgres
|
|
22
|
+
- "2.3 TB, 480M rows"
|
|
23
|
+
- "write p95 210 ms, rising"
|
|
24
|
+
- autovacuum runs 11 h and blocks DDL
|
|
25
|
+
target:
|
|
26
|
+
label: orders-db
|
|
27
|
+
items:
|
|
28
|
+
- dedicated cluster, partitioned by month
|
|
29
|
+
- write p95 under 40 ms
|
|
30
|
+
- DDL touches one partition at a time
|
|
31
|
+
- old copy kept warm for 30 days after cutover
|
|
32
|
+
note: OrdersRepo stays the only entry point; just its wiring changes.
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
## Cutover as a state machine
|
|
36
|
+
|
|
37
|
+
```state
|
|
38
|
+
id: ex-migration-state
|
|
39
|
+
states:
|
|
40
|
+
- { id: s0, col: 1, row: 1, kind: start }
|
|
41
|
+
- { id: dual, col: 2, row: 1, kind: active, name: DUAL_WRITE }
|
|
42
|
+
- { id: backfill, col: 3, row: 1, kind: active, name: BACKFILL }
|
|
43
|
+
- { id: shadow, col: 4, row: 1, kind: active, name: SHADOW_READ }
|
|
44
|
+
- { id: readnew, col: 5, row: 1, kind: active, name: READ_NEW }
|
|
45
|
+
- { id: writenew, col: 6, row: 1, kind: active, name: WRITE_NEW }
|
|
46
|
+
- { id: done, col: 7, row: 1, kind: terminal, name: DONE }
|
|
47
|
+
transitions:
|
|
48
|
+
- { from: s0, to: dual, event: flag on }
|
|
49
|
+
- { from: dual, to: backfill, event: writes verified 24 h }
|
|
50
|
+
- { from: backfill, to: shadow, event: history copied, guard: row counts match }
|
|
51
|
+
- { from: shadow, to: readnew, event: 7 clean days, guard: "mismatch < 0.001%" }
|
|
52
|
+
- { from: readnew, to: writenew, event: 7 clean days, guard: read p95 at or under old }
|
|
53
|
+
- { from: writenew, to: done, event: 30 quiet days }
|
|
54
|
+
- { from: shadow, to: dual, event: mismatch spike }
|
|
55
|
+
- { from: readnew, to: shadow, event: read errors or drift }
|
|
56
|
+
- { from: writenew, to: readnew, event: rollback flag, guard: "reverse replication lag < 60 s" }
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
The state names are literal values of the `orders_migration_phase` flag, so the
|
|
60
|
+
diagram, the flag, and the dashboards share one vocabulary. WRITE_NEW is not
|
|
61
|
+
the point of no return — the old database follows through reverse replication
|
|
62
|
+
and can resume as primary in minutes. The only irreversible transition is
|
|
63
|
+
into DONE.
|
|
64
|
+
|
|
65
|
+
## Write-cutover runbook
|
|
66
|
+
|
|
67
|
+
```steps
|
|
68
|
+
id: ex-migration-runbook
|
|
69
|
+
items:
|
|
70
|
+
- title: Freeze schema changes
|
|
71
|
+
body: "The freeze flag rejects DDL on orders in both stores; announce it in #eng first."
|
|
72
|
+
code: hb flags set orders_ddl_freeze on
|
|
73
|
+
lang: bash
|
|
74
|
+
- title: Confirm reverse replication
|
|
75
|
+
body: Rollback depends on it; lag must stay under 60 s before and during the flip.
|
|
76
|
+
code: hb repl status orders-reverse --watch
|
|
77
|
+
lang: bash
|
|
78
|
+
- title: Flip the write primary
|
|
79
|
+
code: hb flags set orders_migration_phase WRITE_NEW
|
|
80
|
+
lang: bash
|
|
81
|
+
note: The flag drains in-flight transactions for up to 5 s — expect a latency blip, not errors.
|
|
82
|
+
- title: Hold the exit criteria for 30 minutes
|
|
83
|
+
body: "Write p95 under 40 ms, zero dual-write mismatches, reverse lag under 60 s. On any breach, flip back to READ_NEW and debug offline — never in place."
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
## Risks
|
|
87
|
+
|
|
88
|
+
```risk
|
|
89
|
+
id: ex-migration-risks
|
|
90
|
+
items:
|
|
91
|
+
- { risk: "Three cron jobs write orders with raw SQL, bypassing OrdersRepo and the dual-write layer", likelihood: high, impact: high, mitigation: "Two rewritten, one deleted; the bypass audit re-runs weekly until DONE.", owner: Monolith, status: mitigating }
|
|
92
|
+
- { risk: Reverse replication breaks silently and the rollback path becomes fiction, likelihood: med, impact: high, mitigation: "Lag over 60 s pages at production severity, day and night.", owner: Storage, status: mitigating }
|
|
93
|
+
- { risk: Diff sampler misses drift inside JSON columns, likelihood: low, impact: high, mitigation: Rows are canonicalized before hashing., owner: Storage, status: closed }
|
|
94
|
+
- { risk: Backfill competes with month-end order load, likelihood: med, impact: med, mitigation: Copy is rate-limited and pauses itself when monolith write p95 passes 150 ms., owner: Storage, status: open }
|
|
95
|
+
```
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
```meta
|
|
2
|
+
title: Ingest team — week one
|
|
3
|
+
subtitle: What a new engineer sets up, reads, and ships in the first five days.
|
|
4
|
+
tag: Onboarding
|
|
5
|
+
```
|
|
6
|
+
|
|
7
|
+
Nocturne's ingest tier accepts 2.1M spans per second across three regions.
|
|
8
|
+
This team owns the gateway, the sampler, and the Kafka topics between them.
|
|
9
|
+
The week has one goal: ship a guarded sampler-config change to staging by
|
|
10
|
+
Friday, with your own hands on every step.
|
|
11
|
+
|
|
12
|
+
## Who to ask
|
|
13
|
+
|
|
14
|
+
```team
|
|
15
|
+
id: ex-onboarding-people
|
|
16
|
+
members:
|
|
17
|
+
- { name: Priya Nair, role: Tech lead, focus: "Sampler, capacity planning", accent: navy }
|
|
18
|
+
- { name: Jonas Weber, role: SRE, focus: "Gateway, on-call rotation, staging access", accent: teal }
|
|
19
|
+
- { name: Mel Torres, role: Backend, focus: "Kafka topics, schema registry", accent: purple }
|
|
20
|
+
- { name: "#ingest-help", initials: IH, role: Slack channel, focus: First stop for any question — median answer 11 minutes, accent: green }
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## Day one — local stack
|
|
24
|
+
|
|
25
|
+
```steps
|
|
26
|
+
id: ex-onboarding-setup
|
|
27
|
+
items:
|
|
28
|
+
- title: Clone and bootstrap
|
|
29
|
+
body: Bootstrap installs the pinned toolchain and takes about 15 minutes on first run.
|
|
30
|
+
code: git clone git@github.com:nocturne/ingest.git && make bootstrap
|
|
31
|
+
lang: bash
|
|
32
|
+
- title: Start the local stack
|
|
33
|
+
body: "Gateway, sampler, and a single-broker Kafka run in Docker."
|
|
34
|
+
code: make stack-up
|
|
35
|
+
lang: bash
|
|
36
|
+
- title: Run the smoke test
|
|
37
|
+
body: The test pushes 400 synthetic spans end to end and checks the drop-rate counter.
|
|
38
|
+
code: make smoke
|
|
39
|
+
lang: bash
|
|
40
|
+
note: "A red smoke test on a clean clone is a bug — report it in #ingest-help the same day."
|
|
41
|
+
- title: Request staging access
|
|
42
|
+
body: Request the "ingest-staging-rw" role in the access portal. Jonas approves same day.
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
```callout
|
|
46
|
+
id: ex-onboarding-replay
|
|
47
|
+
tone: warn
|
|
48
|
+
title: Replay targets staging only
|
|
49
|
+
body: "`make replay` re-sends captured traffic. A production broker accepts replayed spans as real tenant data, and they land in customer dashboards. Check the `KAFKA_BROKERS` value before every replay."
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
## The week
|
|
53
|
+
|
|
54
|
+
```timeline
|
|
55
|
+
id: ex-onboarding-week
|
|
56
|
+
items:
|
|
57
|
+
- "[done] Day 1 · Local stack · Smoke test green; staging access requested"
|
|
58
|
+
- "[done] Day 2 · Read path · Pair with Mel and follow one span from gateway to sampler"
|
|
59
|
+
- "[current] Day 3 · Shadow on-call · Sit in on Jonas's handover; read the two latest incident docs"
|
|
60
|
+
- "[next] Day 4 · First change · Raise tail sampling for the demo tenant behind ingest.sampler.demo_rate"
|
|
61
|
+
- "[next] Day 5 · Ship it · Flag on in staging; watch the drop-rate dashboard for one hour"
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
Day 4's change is deliberately trivial. The point is the path — flag,
|
|
65
|
+
review, staging deploy, dashboard — not the diff, so that your first urgent
|
|
66
|
+
change is not also your first deploy.
|
|
67
|
+
|
|
68
|
+
## Vocabulary
|
|
69
|
+
|
|
70
|
+
```glossary
|
|
71
|
+
id: ex-onboarding-terms
|
|
72
|
+
terms:
|
|
73
|
+
- Span — one timed operation; the unit everything on this team counts.
|
|
74
|
+
- Drop rate — spans the sampler discards as a share of spans received; the team's headline SLI.
|
|
75
|
+
- Head sampling — keep/drop decided at the gateway, before the trace is complete.
|
|
76
|
+
- Tail sampling — decided in the sampler once the full trace arrives; costs memory, saves storage.
|
|
77
|
+
- Tenant — one customer's isolated stream; every dashboard and quota is per-tenant.
|
|
78
|
+
```
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
```meta
|
|
2
|
+
title: Plot transfer & waitlist
|
|
3
|
+
subtitle: How a Loamly garden plot moves from a leaving member to the next person in line.
|
|
4
|
+
tag: Spec · v1
|
|
5
|
+
```
|
|
6
|
+
|
|
7
|
+
Loamly manages 140 community gardens, and the median waitlist is 23 people
|
|
8
|
+
deep. Today a plot changes hands by email between the coordinator and
|
|
9
|
+
whoever answers first — position in line is a suggestion. This feature makes
|
|
10
|
+
the handover self-serve for the member and automatic for the waitlist, with
|
|
11
|
+
the coordinator as an approval gate only.
|
|
12
|
+
|
|
13
|
+
## The story
|
|
14
|
+
|
|
15
|
+
```userstory
|
|
16
|
+
id: ex-product-spec-story
|
|
17
|
+
role: plot holder who is moving away
|
|
18
|
+
want: hand my plot back without emailing the coordinator
|
|
19
|
+
soThat: the next person in line gets it before planting season
|
|
20
|
+
priority: High
|
|
21
|
+
points: 5
|
|
22
|
+
criteria:
|
|
23
|
+
- { given: I hold an active plot, when: I start a transfer, then: the plot enters the offer flow and I keep access for up to 14 days }
|
|
24
|
+
- { given: I am first on the waitlist, when: I receive an offer, then: I have 72 hours to accept before it moves on }
|
|
25
|
+
- { given: I decline an offer, when: the next offer round starts, then: my waitlist position is unchanged }
|
|
26
|
+
links:
|
|
27
|
+
- { ref: "#ex-product-spec-flow", mode: flow, label: Offer flow }
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
## Offer flow
|
|
31
|
+
|
|
32
|
+
```flow
|
|
33
|
+
id: ex-product-spec-flow
|
|
34
|
+
nodes:
|
|
35
|
+
- { id: start, col: 1, row: 1, kind: start, label: Transfer started }
|
|
36
|
+
- { id: approve, col: 2, row: 1, kind: decision, label: Coordinator approves? }
|
|
37
|
+
- { id: keep, col: 2, row: 2, kind: end, label: Plot stays with holder }
|
|
38
|
+
- { id: offer, col: 3, row: 1, kind: process, label: Offer to next in line }
|
|
39
|
+
- { id: accept, col: 4, row: 1, kind: decision, label: "Accepted within 72 h?" }
|
|
40
|
+
- { id: assign, col: 5, row: 1, kind: end, label: Plot assigned }
|
|
41
|
+
- { id: more, col: 4, row: 2, kind: decision, label: More on waitlist? }
|
|
42
|
+
- { id: dormant, col: 5, row: 2, kind: end, label: Dormant — coordinator review }
|
|
43
|
+
edges:
|
|
44
|
+
- start -> approve
|
|
45
|
+
- approve -x-> keep: "no"
|
|
46
|
+
- approve -> offer: "yes"
|
|
47
|
+
- offer -> accept
|
|
48
|
+
- accept -> assign: "yes"
|
|
49
|
+
- accept -> more: "no / expired"
|
|
50
|
+
- more -> offer: "yes"
|
|
51
|
+
- more -x-> dormant: "no, or 3 offers made"
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
The 72-hour window and the three-offer cap exist for the same reason. A
|
|
55
|
+
transfer that drags past two weeks straddles planting season, and a plot
|
|
56
|
+
idle in April is the outcome every party loses on.
|
|
57
|
+
|
|
58
|
+
## Invariants
|
|
59
|
+
|
|
60
|
+
```spec
|
|
61
|
+
id: ex-product-spec-invariants
|
|
62
|
+
accent: green
|
|
63
|
+
rows:
|
|
64
|
+
- { label: One plot per member, value: "A member holds at most one active plot per garden. A transfer that would create a second is refused at start, not at assignment." }
|
|
65
|
+
- { label: Sticky position, value: "Declining an offer keeps your waitlist position. Only accepting a plot removes the entry." }
|
|
66
|
+
- { label: Offer window, value: "72 hours per offer, at most 3 offers per transfer, then the plot goes dormant for coordinator review." }
|
|
67
|
+
- { label: Access overlap, steps: [Transfer starts, "Holder keeps access ≤ 14 days", Access ends at assignment] }
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
## Scope edges
|
|
71
|
+
|
|
72
|
+
```table
|
|
73
|
+
id: ex-product-spec-scope
|
|
74
|
+
columns: [Capability, v1, Why]
|
|
75
|
+
rows:
|
|
76
|
+
- [Member-initiated transfer, { v: In, tone: pos }, The core loop above]
|
|
77
|
+
- [Coordinator-forced reassignment, { v: In, tone: pos }, "Abandoned plots exist; coordinators need the same flow without the member"]
|
|
78
|
+
- [Plot swaps between two members, { v: Out, tone: muted }, "2% of requests — stays manual"]
|
|
79
|
+
- ["Priority rules (seniority, household size)", { v: Out, tone: muted }, "FIFO only in v1; priority differs per garden's bylaws and needs its own spec"]
|
|
80
|
+
- [Plot fees and payments, { v: Out, tone: muted }, Handled off-platform today and out of this feature's blast radius]
|
|
81
|
+
```
|