whodunit-chronicles 0.3.0 β 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +22 -226
- data/LICENSE +1 -1
- data/README.md +96 -599
- data/exe/whodunit-chronicles +6 -0
- data/lib/whodunit/chronicles/chronicler.rb +62 -0
- data/lib/whodunit/chronicles/cli.rb +131 -0
- data/lib/whodunit/chronicles/errors.rb +7 -33
- data/lib/whodunit/chronicles/ledger.rb +69 -0
- data/lib/whodunit/chronicles/ledger_entry.rb +143 -0
- data/lib/whodunit/chronicles/ledger_factory.rb +66 -0
- data/lib/whodunit/chronicles/ledgers/file_ledger.rb +56 -0
- data/lib/whodunit/chronicles/ledgers/memory_ledger.rb +29 -0
- data/lib/whodunit/chronicles/ledgers/sqlite_ledger.rb +172 -0
- data/lib/whodunit/chronicles/version.rb +2 -1
- data/lib/whodunit/chronicles.rb +12 -65
- data/lib/whodunit-chronicles.rb +0 -1
- data/sig/whodunit/chronicles/chronicler.rbs +14 -0
- data/sig/whodunit/chronicles/cli.rbs +17 -0
- data/sig/whodunit/chronicles/errors.rbs +15 -0
- data/sig/whodunit/chronicles/ledger.rbs +13 -0
- data/sig/whodunit/chronicles/ledger_entry.rbs +62 -0
- data/sig/whodunit/chronicles/ledger_factory.rbs +14 -0
- data/sig/whodunit/chronicles/ledgers/file_ledger.rbs +14 -0
- data/sig/whodunit/chronicles/ledgers/memory_ledger.rbs +12 -0
- data/sig/whodunit/chronicles/ledgers/sqlite_ledger.rbs +30 -0
- data/sig/whodunit/chronicles.rbs +5 -0
- metadata +40 -326
- data/.codeclimate.yml +0 -50
- data/.rubocop.yml +0 -93
- data/.yardopts +0 -14
- data/CODE_OF_CONDUCT.md +0 -132
- data/CONTRIBUTING.md +0 -186
- data/Rakefile +0 -18
- data/docker/mysql/init.sql +0 -33
- data/docker/postgres/init.sql +0 -40
- data/docker-compose.yml +0 -138
- data/examples/images/campaign-performance-analytics.png +0 -0
- data/examples/images/candidate-journey-analytics.png +0 -0
- data/examples/images/recruitment-funnel-analytics.png +0 -0
- data/lib/.gitkeep +0 -0
- data/lib/whodunit/chronicles/adapter_loader.rb +0 -69
- data/lib/whodunit/chronicles/adapters/mysql.rb +0 -261
- data/lib/whodunit/chronicles/adapters/postgresql.rb +0 -278
- data/lib/whodunit/chronicles/change_event.rb +0 -201
- data/lib/whodunit/chronicles/composite_processor.rb +0 -86
- data/lib/whodunit/chronicles/configuration.rb +0 -112
- data/lib/whodunit/chronicles/connection.rb +0 -88
- data/lib/whodunit/chronicles/persistence.rb +0 -129
- data/lib/whodunit/chronicles/processor.rb +0 -127
- data/lib/whodunit/chronicles/service.rb +0 -207
- data/lib/whodunit/chronicles/stream_adapter.rb +0 -91
- data/lib/whodunit/chronicles/table.rb +0 -120
- data/whodunit-chronicles.gemspec +0 -79
data/README.md
CHANGED
|
@@ -1,667 +1,164 @@
|
|
|
1
|
-
#
|
|
1
|
+
# whodunit-chronicles
|
|
2
2
|
|
|
3
3
|
[](https://badge.fury.io/rb/whodunit-chronicles)
|
|
4
4
|
[](https://github.com/kanutocd/whodunit-chronicles/actions)
|
|
5
|
-
[](https://www.ruby-lang.org/en/)
|
|
5
|
+
[](https://www.ruby-lang.org/en/)
|
|
7
6
|
[](https://opensource.org/licenses/MIT)
|
|
8
7
|
|
|
9
|
-
|
|
8
|
+
`whodunit-chronicles` is the minimal audit sink for the Ruby CDC ecosystem. It has a runtime dependency on `cdc-core` and consumes `CDC::Core::ChangeEvent` directly.
|
|
10
9
|
|
|
11
|
-
|
|
10
|
+
## Release status
|
|
12
11
|
|
|
13
|
-
|
|
12
|
+
The currently published RubyGems release is `whodunit-chronicles` `0.3.0`, described as "The complete historical record of your data".
|
|
14
13
|
|
|
15
|
-
|
|
14
|
+
The next release is a complete rewrite of the gem. This README describes the rewritten design: a smaller CDC-native audit sink centered on `CDC::Core::ChangeEvent`, immutable ledger entries, lightweight built-in ledgers, and extension gems for heavier storage targets.
|
|
16
15
|
|
|
17
|
-
|
|
18
|
-
- **π Zero Application Overhead**: No Rails callbacks or Active Record hooks required
|
|
19
|
-
- **ποΈ Database Agnostic**: Abstract adapter pattern supports PostgreSQL and MySQL/MariaDB
|
|
20
|
-
- **β‘ Thread-Safe**: Concurrent processing with configurable thread pools
|
|
21
|
-
- **π‘οΈ Resilient**: Built-in error handling, retry logic, and monitoring
|
|
22
|
-
- **π Complete Audit Trail**: Captures INSERT, UPDATE, DELETE with full before/after data
|
|
23
|
-
- **π§ͺ Code Coverage**: 94%+ test coverage with comprehensive error scenarios
|
|
16
|
+
A `Chronicler` writes immutable `LedgerEntry` objects into any object that implements the `Ledger` contract.
|
|
24
17
|
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
---
|
|
34
|
-
|
|
35
|
-
Perfect for applications that need comprehensive change tracking alongside Whodunit's user attribution:
|
|
36
|
-
|
|
37
|
-
```ruby
|
|
38
|
-
# Basic setup for user activity tracking
|
|
39
|
-
class BasicProcessor < Whodunit::Chronicles::Processor
|
|
40
|
-
def build_chronicles_record(change_event)
|
|
41
|
-
super.tap do |record|
|
|
42
|
-
# Add basic business context
|
|
43
|
-
record[:change_category] = categorize_change(change_event)
|
|
44
|
-
record[:business_impact] = assess_impact(change_event)
|
|
45
|
-
end
|
|
46
|
-
end
|
|
47
|
-
|
|
48
|
-
private
|
|
49
|
-
|
|
50
|
-
def categorize_change(change_event)
|
|
51
|
-
case change_event.table_name
|
|
52
|
-
when 'users' then 'user_management'
|
|
53
|
-
when 'posts' then 'content'
|
|
54
|
-
when 'comments' then 'engagement'
|
|
55
|
-
else 'system'
|
|
56
|
-
end
|
|
57
|
-
end
|
|
58
|
-
end
|
|
59
|
-
```
|
|
60
|
-
|
|
61
|
-
**Use Case**: Blog platform tracking user posts and comments for community management and content moderation.
|
|
62
|
-
|
|
63
|
-
#### 2. Advanced Recruitment Analytics
|
|
64
|
-
|
|
65
|
-
Sophisticated business intelligence for talent acquisition platforms:
|
|
66
|
-
|
|
67
|
-
```ruby
|
|
68
|
-
# Advanced processor for recruitment metrics
|
|
69
|
-
class RecruitmentAnalyticsProcessor < Whodunit::Chronicles::Processor
|
|
70
|
-
def build_chronicles_record(change_event)
|
|
71
|
-
super.tap do |record|
|
|
72
|
-
# Add recruitment-specific business metrics
|
|
73
|
-
record[:recruitment_stage] = determine_stage(change_event)
|
|
74
|
-
record[:funnel_position] = calculate_funnel_position(change_event)
|
|
75
|
-
record[:time_to_hire_impact] = assess_time_impact(change_event)
|
|
76
|
-
record[:cost_per_hire_impact] = calculate_cost_impact(change_event)
|
|
77
|
-
|
|
78
|
-
# Campaign attribution
|
|
79
|
-
record[:utm_source] = extract_utm_source(change_event)
|
|
80
|
-
record[:campaign_id] = extract_campaign_id(change_event)
|
|
81
|
-
|
|
82
|
-
# Quality metrics
|
|
83
|
-
record[:candidate_quality_score] = assess_candidate_quality(change_event)
|
|
84
|
-
end
|
|
85
|
-
end
|
|
86
|
-
|
|
87
|
-
def process(change_event)
|
|
88
|
-
record = build_chronicles_record(change_event)
|
|
89
|
-
store_audit_record(record)
|
|
90
|
-
|
|
91
|
-
# Stream to analytics platforms
|
|
92
|
-
stream_to_prometheus(record) if track_metrics?
|
|
93
|
-
update_grafana_dashboard(record)
|
|
94
|
-
trigger_real_time_alerts(record) if alert_worthy?(record)
|
|
95
|
-
end
|
|
96
|
-
|
|
97
|
-
private
|
|
98
|
-
|
|
99
|
-
def determine_stage(change_event)
|
|
100
|
-
return 'unknown' unless change_event.table_name == 'applications'
|
|
101
|
-
|
|
102
|
-
case change_event.new_data&.dig('status')
|
|
103
|
-
when 'submitted' then 'application'
|
|
104
|
-
when 'screening', 'in_review' then 'screening'
|
|
105
|
-
when 'interview_scheduled', 'interviewed' then 'interview'
|
|
106
|
-
when 'offer_extended', 'offer_accepted' then 'offer'
|
|
107
|
-
when 'hired' then 'hire'
|
|
108
|
-
else 'unknown'
|
|
109
|
-
end
|
|
110
|
-
end
|
|
111
|
-
|
|
112
|
-
def stream_to_prometheus(record)
|
|
113
|
-
# Track key recruitment metrics
|
|
114
|
-
RECRUITMENT_APPLICATIONS_TOTAL.increment(
|
|
115
|
-
source: record[:utm_source],
|
|
116
|
-
department: record.dig(:new_data, 'department')
|
|
117
|
-
)
|
|
118
|
-
|
|
119
|
-
if record[:action] == 'UPDATE' && status_changed_to_hired?(record)
|
|
120
|
-
RECRUITMENT_HIRES_TOTAL.increment(
|
|
121
|
-
source: record[:utm_source],
|
|
122
|
-
time_to_hire: record[:time_to_hire_impact]
|
|
123
|
-
)
|
|
124
|
-
end
|
|
125
|
-
end
|
|
126
|
-
|
|
127
|
-
def update_grafana_dashboard(record)
|
|
128
|
-
# Send time-series data for Grafana visualization
|
|
129
|
-
InfluxDB.write_point('recruitment_events', {
|
|
130
|
-
timestamp: record[:occurred_at],
|
|
131
|
-
table: record[:table_name],
|
|
132
|
-
action: record[:action],
|
|
133
|
-
stage: record[:recruitment_stage],
|
|
134
|
-
source: record[:utm_source],
|
|
135
|
-
cost_impact: record[:cost_per_hire_impact],
|
|
136
|
-
quality_score: record[:candidate_quality_score]
|
|
137
|
-
})
|
|
138
|
-
end
|
|
139
|
-
end
|
|
18
|
+
```text
|
|
19
|
+
CDC::Core::ChangeEvent
|
|
20
|
+
β
|
|
21
|
+
Whodunit::Chronicles::Chronicler
|
|
22
|
+
β
|
|
23
|
+
Whodunit::Chronicles::LedgerEntry
|
|
24
|
+
β
|
|
25
|
+
Ledger
|
|
140
26
|
```
|
|
141
27
|
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
#### π Visual Analytics Dashboard
|
|
145
|
-
|
|
146
|
-
The recruitment analytics processor creates comprehensive Grafana dashboards for executive reporting and operational insights:
|
|
147
|
-
|
|
148
|
-
<div align="center">
|
|
149
|
-
|
|
150
|
-
**Campaign Performance Analytics**
|
|
151
|
-
<a href="examples/images/campaign-performance-analytics.png" title="Click to view full size image">
|
|
152
|
-
<img src="examples/images/campaign-performance-analytics.png" width="300" />
|
|
153
|
-
</a>
|
|
154
|
-
_Track campaign ROI, cost-per-hire by channel, and conversion rates across marketing sources_
|
|
155
|
-
|
|
156
|
-
**Candidate Journey Analytics**
|
|
157
|
-
<a href="examples/images/candidate-journey-analytics.png" title="Click to view full size image">
|
|
158
|
-
<img src="examples/images/candidate-journey-analytics.png" width="300" />
|
|
159
|
-
</a>
|
|
160
|
-
_Monitor candidate engagement, funnel conversion rates, and application completion patterns_
|
|
28
|
+
## Core boundary
|
|
161
29
|
|
|
162
|
-
|
|
163
|
-
<a href="examples/images/recruitment-funnel-analytics.png" title="Click to view full size image">
|
|
164
|
-
<img src="examples/images/recruitment-funnel-analytics.png" width="300" />
|
|
165
|
-
</a>
|
|
166
|
-
_Analyze hiring pipeline progression, department performance, and time-series trends_
|
|
30
|
+
Core Chronicles owns:
|
|
167
31
|
|
|
168
|
-
|
|
32
|
+
- `cdc-core` processor integration
|
|
169
33
|
|
|
170
|
-
|
|
34
|
+
- `Chronicler`
|
|
35
|
+
- immutable `LedgerEntry`
|
|
36
|
+
- `Ledger` contract
|
|
37
|
+
- `MemoryLedger`
|
|
38
|
+
- `FileLedger`
|
|
39
|
+
- `SQLiteLedger`
|
|
40
|
+
- ledger lifecycle CLI
|
|
171
41
|
|
|
172
|
-
|
|
42
|
+
Core Chronicles does not own heavyweight storage integrations such as PostgreSQL, MySQL, Oracle, MongoDB, S3, Snowflake, Glacier, Mixpanel, or ClickHouse. Those belong in extension gems that implement the `Ledger` contract.
|
|
173
43
|
|
|
174
|
-
|
|
44
|
+
## Runtime usage
|
|
175
45
|
|
|
176
46
|
```ruby
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
Or install directly:
|
|
47
|
+
ledger = Whodunit::Chronicles::Ledgers::SQLiteLedger.new(path: "chronicles.db")
|
|
48
|
+
chronicler = Whodunit::Chronicles::Chronicler.new(ledger: ledger, prepare: false)
|
|
181
49
|
|
|
182
|
-
|
|
183
|
-
gem install whodunit-chronicles
|
|
50
|
+
chronicler.process(change_event) # change_event is CDC::Core::ChangeEvent
|
|
184
51
|
```
|
|
185
52
|
|
|
186
|
-
|
|
53
|
+
The runtime hot path is intentionally small:
|
|
187
54
|
|
|
188
55
|
```ruby
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
# Database Configuration
|
|
192
|
-
|
|
193
|
-
## PostgreSQL Configuration
|
|
194
|
-
Whodunit::Chronicles.configure do |config|
|
|
195
|
-
config.adapter = :postgresql
|
|
196
|
-
config.database_url = 'postgresql://localhost/myapp_production'
|
|
197
|
-
config.audit_database_url = 'postgresql://localhost/myapp'
|
|
198
|
-
config.publication_name = 'myapp_chronicles'
|
|
199
|
-
config.replication_slot_name = 'myapp_chronicles_slot'
|
|
200
|
-
end
|
|
201
|
-
|
|
202
|
-
## MySQL/MariaDB Configuration
|
|
203
|
-
Whodunit::Chronicles.configure do |config|
|
|
204
|
-
config.adapter = :mysql
|
|
205
|
-
config.database_url = 'mysql://user:password@localhost/myapp_production'
|
|
206
|
-
config.audit_database_url = 'mysql://user:password@localhost/myapp_audit'
|
|
207
|
-
config.mysql_server_id = 1001 # Unique server ID for replication
|
|
208
|
-
end
|
|
209
|
-
|
|
210
|
-
# Create and start the service
|
|
211
|
-
service = Whodunit::Chronicles.service
|
|
212
|
-
service.setup! # Create publication/replication setup
|
|
213
|
-
service.start # Begin streaming changes
|
|
214
|
-
|
|
215
|
-
# Service runs in background threads
|
|
216
|
-
sleep 10
|
|
217
|
-
|
|
218
|
-
# Stop gracefully
|
|
219
|
-
service.stop
|
|
220
|
-
service.teardown! # Clean up database objects
|
|
56
|
+
entry = Whodunit::Chronicles::LedgerEntry.from_change_event(event)
|
|
57
|
+
ledger.append(entry)
|
|
221
58
|
```
|
|
222
59
|
|
|
223
|
-
##
|
|
224
|
-
|
|
225
|
-
Chronicles uses **PostgreSQL logical replication** and **MySQL/MariaDB binary log streaming** to capture database changes without impacting your application:
|
|
226
|
-
|
|
227
|
-
```
|
|
228
|
-
βββββββββββββββββββ ββββββββββββββββββββ βββββββββββββββββββ
|
|
229
|
-
β Application β β Chronicles β β Audit Store β
|
|
230
|
-
β Database βββββΆβ Service βββββΆβ Database β
|
|
231
|
-
β β β β β β
|
|
232
|
-
β β’ Users β β β’ Stream Adapter β β β’ audit_records β
|
|
233
|
-
β β’ Posts β β β’ Event Parser β β β’ Searchable β
|
|
234
|
-
β β’ Comments β β β’ Audit Builder β β β’ Reportable β
|
|
235
|
-
βββββββββββββββββββ ββββββββββββββββββββ βββββββββββββββββββ
|
|
236
|
-
β β
|
|
237
|
-
β ββββββββββΌβββββββββ
|
|
238
|
-
β β PostgreSQL β
|
|
239
|
-
ββββββββββββββββ Logical β
|
|
240
|
-
β Replication β
|
|
241
|
-
βββββββββββββββββββ
|
|
242
|
-
```
|
|
243
|
-
|
|
244
|
-
### Core Components
|
|
245
|
-
|
|
246
|
-
- **StreamAdapter**: Database-specific change streaming (PostgreSQL logical replication, MySQL/MariaDB binary log streaming)
|
|
247
|
-
- **ChangeEvent**: Unified change representation across adapters
|
|
248
|
-
- **Processor**: Transforms changes into searchable audit records
|
|
249
|
-
- **Service**: Orchestrates streaming with error handling and retry logic
|
|
60
|
+
## Ledger contract
|
|
250
61
|
|
|
251
|
-
|
|
62
|
+
Required:
|
|
252
63
|
|
|
253
64
|
```ruby
|
|
254
|
-
|
|
255
|
-
# Database connections
|
|
256
|
-
config.database_url = ENV['DATABASE_URL']
|
|
257
|
-
config.audit_database_url = ENV['AUDIT_DATABASE_URL']
|
|
258
|
-
|
|
259
|
-
# Database adapter (postgresql, mysql, mariadb)
|
|
260
|
-
config.adapter = :postgresql
|
|
261
|
-
|
|
262
|
-
# PostgreSQL-specific settings
|
|
263
|
-
config.publication_name = 'whodunit_chronicles'
|
|
264
|
-
config.replication_slot_name = 'whodunit_chronicles_slot'
|
|
265
|
-
|
|
266
|
-
# Performance tuning
|
|
267
|
-
config.batch_size = 1000
|
|
268
|
-
config.max_retry_attempts = 5
|
|
269
|
-
config.retry_delay = 10
|
|
270
|
-
|
|
271
|
-
# Table filtering
|
|
272
|
-
config.include_tables = %w[users posts comments]
|
|
273
|
-
config.exclude_tables = %w[sessions temp_data]
|
|
274
|
-
config.include_schemas = %w[public app]
|
|
275
|
-
config.exclude_schemas = %w[information_schema pg_catalog]
|
|
276
|
-
end
|
|
65
|
+
ledger.append(entry)
|
|
277
66
|
```
|
|
278
67
|
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
Chronicles creates structured audit records for each database change:
|
|
68
|
+
Optional operational capabilities:
|
|
282
69
|
|
|
283
70
|
```ruby
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
old_data: { "id" => 456, "email" => "old@example.com", "name" => "Old Name" },
|
|
291
|
-
new_data: { "id" => 456, "email" => "new@example.com", "name" => "New Name" },
|
|
292
|
-
changes: { "email" => ["old@example.com", "new@example.com"] },
|
|
293
|
-
user_id: 789, # From creator_id/updater_id/deleter_id columns
|
|
294
|
-
user_type: "User",
|
|
295
|
-
transaction_id: "tx_abc123",
|
|
296
|
-
sequence_number: 42,
|
|
297
|
-
occurred_at: 2025-01-21 10:30:00 UTC,
|
|
298
|
-
created_at: 2025-01-21 10:30:01 UTC,
|
|
299
|
-
metadata: {
|
|
300
|
-
table_schema: "public",
|
|
301
|
-
qualified_table_name: "public.users",
|
|
302
|
-
changed_columns: ["email"],
|
|
303
|
-
chronicles_version: "0.1.0"
|
|
304
|
-
}
|
|
305
|
-
}
|
|
71
|
+
ledger.prepare!
|
|
72
|
+
ledger.migrate!
|
|
73
|
+
ledger.ensure_indexes!
|
|
74
|
+
ledger.verify
|
|
75
|
+
ledger.status
|
|
76
|
+
ledger.partition_for(entry)
|
|
306
77
|
```
|
|
307
78
|
|
|
308
|
-
##
|
|
309
|
-
|
|
310
|
-
### Custom Processors for Analytics & Monitoring
|
|
311
|
-
|
|
312
|
-
**The real power of Chronicles** comes from creating custom processors tailored for your specific analytics needs. While Whodunit captures basic "who changed what," Chronicles lets you build sophisticated data pipelines for tools like **Grafana**, **DataDog**, or **Elasticsearch**.
|
|
313
|
-
|
|
314
|
-
Transform database changes into actionable business intelligence with features like:
|
|
79
|
+
## Built-in ledgers
|
|
315
80
|
|
|
316
|
-
|
|
317
|
-
- **Real-time Dashboards**: Stream data to Grafana for executive reporting and operational monitoring
|
|
318
|
-
- **Smart Alerting**: Trigger notifications based on business rules and thresholds
|
|
319
|
-
- **Multi-destination Streaming**: Send data simultaneously to multiple analytics platforms
|
|
81
|
+
### MemoryLedger
|
|
320
82
|
|
|
321
|
-
|
|
83
|
+
For tests, examples, and short-lived scripts.
|
|
322
84
|
|
|
323
85
|
```ruby
|
|
324
|
-
|
|
325
|
-
def build_chronicles_record(change_event)
|
|
326
|
-
super.tap do |record|
|
|
327
|
-
# Add business metrics
|
|
328
|
-
record[:business_impact] = calculate_business_impact(change_event)
|
|
329
|
-
record[:user_segment] = determine_user_segment(change_event)
|
|
330
|
-
record[:feature_flag] = current_feature_flags
|
|
331
|
-
|
|
332
|
-
# Add performance metrics
|
|
333
|
-
record[:change_size] = calculate_change_size(change_event)
|
|
334
|
-
record[:peak_hours] = during_peak_hours?
|
|
335
|
-
record[:geographic_region] = user_region(change_event)
|
|
336
|
-
|
|
337
|
-
# Add time-series friendly fields for Grafana
|
|
338
|
-
record[:hour_of_day] = Time.current.hour
|
|
339
|
-
record[:day_of_week] = Time.current.wday
|
|
340
|
-
record[:is_weekend] = weekend?
|
|
341
|
-
|
|
342
|
-
# Custom tagging for filtering
|
|
343
|
-
record[:tags] = generate_tags(change_event)
|
|
344
|
-
end
|
|
345
|
-
end
|
|
346
|
-
|
|
347
|
-
private
|
|
348
|
-
|
|
349
|
-
def calculate_business_impact(change_event)
|
|
350
|
-
case change_event.table_name
|
|
351
|
-
when 'orders' then 'revenue_critical'
|
|
352
|
-
when 'users' then 'customer_critical'
|
|
353
|
-
when 'products' then 'inventory_critical'
|
|
354
|
-
else 'standard'
|
|
355
|
-
end
|
|
356
|
-
end
|
|
357
|
-
|
|
358
|
-
def determine_user_segment(change_event)
|
|
359
|
-
return 'anonymous' unless change_event.user_id
|
|
360
|
-
|
|
361
|
-
# Look up user tier from your business logic
|
|
362
|
-
User.find(change_event.user_id)&.tier || 'standard'
|
|
363
|
-
end
|
|
364
|
-
|
|
365
|
-
def generate_tags(change_event)
|
|
366
|
-
tags = [change_event.action.downcase]
|
|
367
|
-
tags << 'bulk_operation' if bulk_operation?(change_event)
|
|
368
|
-
tags << 'api_driven' if api_request?
|
|
369
|
-
tags << 'admin_action' if admin_user?(change_event.user_id)
|
|
370
|
-
tags
|
|
371
|
-
end
|
|
372
|
-
end
|
|
86
|
+
ledger = Whodunit::Chronicles::Ledgers::MemoryLedger.new
|
|
373
87
|
```
|
|
374
88
|
|
|
375
|
-
|
|
89
|
+
### FileLedger
|
|
376
90
|
|
|
377
|
-
|
|
378
|
-
class GrafanaProcessor < Whodunit::Chronicles::Processor
|
|
379
|
-
def build_chronicles_record(change_event)
|
|
380
|
-
{
|
|
381
|
-
# Core metrics for Grafana time series
|
|
382
|
-
timestamp: change_event.occurred_at,
|
|
383
|
-
table_name: change_event.table_name,
|
|
384
|
-
action: change_event.action,
|
|
385
|
-
|
|
386
|
-
# Numerical metrics for graphs
|
|
387
|
-
records_affected: calculate_records_affected(change_event),
|
|
388
|
-
change_magnitude: calculate_change_magnitude(change_event),
|
|
389
|
-
user_session_duration: calculate_session_duration(change_event),
|
|
390
|
-
|
|
391
|
-
# Categorical dimensions for filtering
|
|
392
|
-
environment: Rails.env,
|
|
393
|
-
application_version: app_version,
|
|
394
|
-
database_instance: database_identifier,
|
|
395
|
-
|
|
396
|
-
# Business KPIs
|
|
397
|
-
revenue_impact: calculate_revenue_impact(change_event),
|
|
398
|
-
customer_satisfaction_risk: assess_satisfaction_risk(change_event),
|
|
399
|
-
|
|
400
|
-
# Performance indicators
|
|
401
|
-
query_duration_ms: extract_query_duration(change_event),
|
|
402
|
-
concurrent_users: current_concurrent_users,
|
|
403
|
-
system_load: current_system_load
|
|
404
|
-
}
|
|
405
|
-
end
|
|
406
|
-
end
|
|
407
|
-
```
|
|
408
|
-
|
|
409
|
-
#### Real-Time Alerts Processor
|
|
91
|
+
Append-only newline-delimited JSON file.
|
|
410
92
|
|
|
411
93
|
```ruby
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
record = build_chronicles_record(change_event)
|
|
415
|
-
|
|
416
|
-
# Store the audit record
|
|
417
|
-
store_audit_record(record)
|
|
418
|
-
|
|
419
|
-
# Real-time alerting logic
|
|
420
|
-
send_alert(record) if alert_worthy?(record)
|
|
421
|
-
|
|
422
|
-
# Stream to monitoring systems
|
|
423
|
-
stream_to_datadog(record) if production?
|
|
424
|
-
stream_to_grafana(record)
|
|
425
|
-
end
|
|
426
|
-
|
|
427
|
-
private
|
|
428
|
-
|
|
429
|
-
def alert_worthy?(record)
|
|
430
|
-
# Define your alerting criteria
|
|
431
|
-
record[:business_impact] == 'revenue_critical' ||
|
|
432
|
-
record[:records_affected] > 1000 ||
|
|
433
|
-
record[:action] == 'DELETE' && record[:table_name] == 'orders'
|
|
434
|
-
end
|
|
435
|
-
|
|
436
|
-
def stream_to_grafana(record)
|
|
437
|
-
# Send metrics to Grafana via InfluxDB/Prometheus
|
|
438
|
-
InfluxDB.write_point("chronicles_events", record)
|
|
439
|
-
end
|
|
440
|
-
end
|
|
94
|
+
ledger = Whodunit::Chronicles::Ledgers::FileLedger.new(path: "chronicles.ndjson")
|
|
95
|
+
ledger.prepare!
|
|
441
96
|
```
|
|
442
97
|
|
|
443
|
-
|
|
98
|
+
### SQLiteLedger
|
|
444
99
|
|
|
445
|
-
|
|
446
|
-
# Chain multiple processors for different purposes
|
|
447
|
-
service = Whodunit::Chronicles::Service.new(
|
|
448
|
-
adapter: Adapters::PostgreSQL.new,
|
|
449
|
-
processor: Whodunit::Chronicles::CompositeProcessor.new([
|
|
450
|
-
AnalyticsProcessor.new, # For business intelligence
|
|
451
|
-
AlertingProcessor.new, # For real-time monitoring
|
|
452
|
-
ComplianceProcessor.new, # For regulatory requirements
|
|
453
|
-
ArchivalProcessor.new # For long-term storage
|
|
454
|
-
])
|
|
455
|
-
)
|
|
456
|
-
```
|
|
457
|
-
|
|
458
|
-
**Use Cases:**
|
|
459
|
-
|
|
460
|
-
- **π Business Intelligence**: Track user behavior patterns, feature adoption, revenue impact
|
|
461
|
-
- **π¨ Real-Time Monitoring**: Alert on suspicious activities, bulk operations, data anomalies
|
|
462
|
-
- **π Performance Analytics**: Database performance metrics, query optimization insights
|
|
463
|
-
- **π Compliance Auditing**: Regulatory compliance, data governance, access patterns
|
|
464
|
-
- **π‘ Product Analytics**: Feature usage, A/B testing data, user journey tracking
|
|
465
|
-
|
|
466
|
-
### Service Monitoring
|
|
100
|
+
Embedded durable local ledger.
|
|
467
101
|
|
|
468
102
|
```ruby
|
|
469
|
-
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
status = service.status
|
|
473
|
-
puts "Running: #{status[:running]}"
|
|
474
|
-
puts "Adapter Position: #{status[:adapter_position]}"
|
|
475
|
-
puts "Retry Count: #{status[:retry_count]}"
|
|
476
|
-
puts "Active Threads: #{status[:executor_status][:active_count]}"
|
|
477
|
-
|
|
478
|
-
# Monitor in production
|
|
479
|
-
Thread.new do
|
|
480
|
-
loop do
|
|
481
|
-
status = service.status
|
|
482
|
-
Rails.logger.info "Chronicles Status: #{status}"
|
|
483
|
-
sleep 60
|
|
484
|
-
end
|
|
485
|
-
end
|
|
103
|
+
ledger = Whodunit::Chronicles::Ledgers::SQLiteLedger.new(path: "chronicles.db")
|
|
104
|
+
ledger.prepare!
|
|
105
|
+
ledger.ensure_indexes!
|
|
486
106
|
```
|
|
487
107
|
|
|
488
|
-
|
|
108
|
+
`SQLiteLedger` depends on the `sqlite3` gem. Tests and advanced users may inject a compatible connection.
|
|
489
109
|
|
|
490
|
-
|
|
110
|
+
`ensure_indexes!` creates a unique index on `event_id`. Appending the same event more than once is not idempotent: SQLite rejects the duplicate row, and `SQLiteLedger#append` raises `Whodunit::Chronicles::AppendError`.
|
|
491
111
|
|
|
492
|
-
|
|
112
|
+
## CLI
|
|
493
113
|
|
|
494
|
-
|
|
114
|
+
Ledger evolution is operational work, similar to `db:migrate`. The gem consumer decides when to run it.
|
|
495
115
|
|
|
496
|
-
```
|
|
497
|
-
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
|
|
502
|
-
@service.start
|
|
503
|
-
end
|
|
504
|
-
|
|
505
|
-
def teardown
|
|
506
|
-
@service.stop
|
|
507
|
-
@service.teardown!
|
|
508
|
-
end
|
|
509
|
-
|
|
510
|
-
def test_audit_record_creation
|
|
511
|
-
# Create a user (triggers Whodunit)
|
|
512
|
-
user = User.create!(name: "John", email: "john@example.com")
|
|
513
|
-
|
|
514
|
-
# Wait for Chronicles to process
|
|
515
|
-
sleep 1
|
|
516
|
-
|
|
517
|
-
# Check Chronicles audit record
|
|
518
|
-
audit_record = AuditRecord.find_by(
|
|
519
|
-
table_name: 'users',
|
|
520
|
-
action: 'INSERT',
|
|
521
|
-
record_id: { 'id' => user.id }
|
|
522
|
-
)
|
|
523
|
-
|
|
524
|
-
assert audit_record
|
|
525
|
-
assert_equal 'INSERT', audit_record.action
|
|
526
|
-
assert_equal user.name, audit_record.new_data['name']
|
|
527
|
-
end
|
|
528
|
-
end
|
|
116
|
+
```bash
|
|
117
|
+
whodunit-chronicles ledger prepare config.yml
|
|
118
|
+
whodunit-chronicles ledger migrate config.yml
|
|
119
|
+
whodunit-chronicles ledger ensure-indexes config.yml
|
|
120
|
+
whodunit-chronicles ledger verify config.yml
|
|
121
|
+
whodunit-chronicles ledger status config.yml
|
|
529
122
|
```
|
|
530
123
|
|
|
531
|
-
|
|
124
|
+
Example configs:
|
|
532
125
|
|
|
533
|
-
```
|
|
534
|
-
|
|
535
|
-
|
|
536
|
-
def setup
|
|
537
|
-
@processor = RecruitmentAnalyticsProcessor.new
|
|
538
|
-
end
|
|
539
|
-
|
|
540
|
-
def test_recruitment_stage_determination
|
|
541
|
-
change_event = create_change_event(
|
|
542
|
-
table_name: 'applications',
|
|
543
|
-
action: 'UPDATE',
|
|
544
|
-
new_data: { 'status' => 'hired' }
|
|
545
|
-
)
|
|
546
|
-
|
|
547
|
-
record = @processor.build_chronicles_record(change_event)
|
|
548
|
-
|
|
549
|
-
assert_equal 'hire', record[:recruitment_stage]
|
|
550
|
-
assert record[:cost_per_hire_impact]
|
|
551
|
-
end
|
|
552
|
-
|
|
553
|
-
def test_metrics_streaming
|
|
554
|
-
# Mock Prometheus and Grafana integrations
|
|
555
|
-
assert_difference 'RECRUITMENT_HIRES_TOTAL.get' do
|
|
556
|
-
@processor.stream_to_prometheus(hired_record)
|
|
557
|
-
end
|
|
558
|
-
end
|
|
559
|
-
end
|
|
126
|
+
```yaml
|
|
127
|
+
ledger:
|
|
128
|
+
adapter: memory
|
|
560
129
|
```
|
|
561
130
|
|
|
562
|
-
|
|
563
|
-
|
|
564
|
-
|
|
565
|
-
|
|
566
|
-
```bash
|
|
567
|
-
# Run test suite
|
|
568
|
-
bundle exec rake test
|
|
569
|
-
|
|
570
|
-
# Run with coverage
|
|
571
|
-
bundle exec rake test
|
|
572
|
-
open coverage/index.html
|
|
573
|
-
|
|
574
|
-
# Security scanning
|
|
575
|
-
bundle exec bundler-audit check
|
|
576
|
-
bundle exec brakeman
|
|
131
|
+
```yaml
|
|
132
|
+
ledger:
|
|
133
|
+
adapter: file
|
|
134
|
+
path: chronicles.ndjson
|
|
577
135
|
```
|
|
578
136
|
|
|
579
|
-
|
|
580
|
-
|
|
581
|
-
|
|
582
|
-
|
|
583
|
-
|
|
584
|
-
|
|
585
|
-
- **Memory Efficient**: Streaming processing without loading full datasets
|
|
586
|
-
|
|
587
|
-
## π‘οΈ Security
|
|
588
|
-
|
|
589
|
-
- **Dependency Scanning**: Automated bundler-audit checks
|
|
590
|
-
- **Code Analysis**: GitHub CodeQL integration
|
|
591
|
-
- **Vulnerability Monitoring**: Weekly security scans
|
|
592
|
-
- **Safe Defaults**: Secure configuration out of the box
|
|
593
|
-
|
|
594
|
-
## π€ Contributing
|
|
595
|
-
|
|
596
|
-
We welcome contributions! Chronicles is designed to be extensible and work across different business domains.
|
|
597
|
-
|
|
598
|
-
1. Fork the repository
|
|
599
|
-
2. Set up your development environment:
|
|
600
|
-
```bash
|
|
601
|
-
bundle install
|
|
602
|
-
bundle exec rake test # Ensure tests pass
|
|
603
|
-
```
|
|
604
|
-
3. Create your feature branch (`git checkout -b feature/amazing-feature`)
|
|
605
|
-
4. Make your changes with comprehensive tests
|
|
606
|
-
5. Test your changes:
|
|
607
|
-
- Unit tests: `bundle exec rake test`
|
|
608
|
-
- Code style: `bundle exec rubocop`
|
|
609
|
-
- Security: `bundle exec bundler-audit check`
|
|
610
|
-
6. Commit your changes (`git commit -m 'Add amazing feature'`)
|
|
611
|
-
7. Push to the branch (`git push origin feature/amazing-feature`)
|
|
612
|
-
8. Open a Pull Request with a detailed description
|
|
613
|
-
|
|
614
|
-
### Contributing Custom Processors
|
|
615
|
-
|
|
616
|
-
We especially welcome custom processors for different business domains. Consider contributing processors for:
|
|
617
|
-
|
|
618
|
-
- E-commerce analytics (order tracking, inventory management)
|
|
619
|
-
- Financial services (transaction monitoring, compliance reporting)
|
|
620
|
-
- Healthcare (patient data tracking, regulatory compliance)
|
|
621
|
-
- Education (student progress, course analytics)
|
|
622
|
-
- SaaS metrics (user engagement, feature adoption)
|
|
623
|
-
|
|
624
|
-
## π Requirements
|
|
625
|
-
|
|
626
|
-
- **Ruby**: 3.1.0 or higher
|
|
627
|
-
- **PostgreSQL**: 10.0 or higher (with logical replication enabled)
|
|
628
|
-
- **MySQL/MariaDB**: 5.6+ (with binary logging enabled)
|
|
629
|
-
|
|
630
|
-
## πΊοΈ Roadmap
|
|
631
|
-
|
|
632
|
-
- [ ] **Prometheus Metrics**: Production monitoring integration (with complete codebase included in examples/)
|
|
633
|
-
- [ ] **Advanced Example Apps**: Real-world use cases with complete monitoring stack (with complete codebase included in examples/)
|
|
634
|
-
- [ ] **Custom Analytics Processors**: Business intelligence and real-time monitoring (with complete codebase included in examples/)
|
|
635
|
-
- [x] **MySQL/MariaDB Support**: MySQL/MariaDB databases binlog streaming adapter
|
|
636
|
-
- [ ] **Redis Streams**: Alternative lightweight streaming backend
|
|
637
|
-
- [ ] **Compression**: Optional audit record compression
|
|
638
|
-
- [ ] **Retention Policies**: Automated audit record cleanup
|
|
639
|
-
- [ ] **Web UI**: Management interface for monitoring and configuration
|
|
640
|
-
|
|
641
|
-
## π Documentation
|
|
642
|
-
|
|
643
|
-
- **[API Documentation](https://kanutocd.github.io/whodunit-chronicles/)**
|
|
644
|
-
- [ ] TODO: **Configuration Guide** _(docs/configuration-todo.md)_
|
|
645
|
-
- [ ] TODO: **Architecture Deep Dive** _(docs/architecture-todo.md)_
|
|
646
|
-
- [ ] TODO: **PostgreSQL Setup** _(docs/postgresql-setup-todo.md)_
|
|
647
|
-
- [ ] TODO: **MySQL/MariaDB Setup** _(docs/mysql-setup.md)_
|
|
648
|
-
- [ ] TODO: **Production Deployment** _(docs/production-todo.md)_
|
|
649
|
-
|
|
650
|
-
## π License
|
|
651
|
-
|
|
652
|
-
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
653
|
-
|
|
654
|
-
## π Acknowledgments
|
|
137
|
+
```yaml
|
|
138
|
+
ledger:
|
|
139
|
+
adapter: sqlite
|
|
140
|
+
path: chronicles.db
|
|
141
|
+
table_name: whodunit_chronicles_entries
|
|
142
|
+
```
|
|
655
143
|
|
|
656
|
-
|
|
657
|
-
- **Ruby Community**: For amazing gems and tools that make this possible
|
|
144
|
+
Status can be printed as human-readable lines or JSON:
|
|
658
145
|
|
|
659
|
-
|
|
146
|
+
```bash
|
|
147
|
+
whodunit-chronicles ledger status config.yml
|
|
148
|
+
whodunit-chronicles ledger status config.yml --json
|
|
149
|
+
```
|
|
660
150
|
|
|
661
|
-
|
|
151
|
+
## Extension direction
|
|
662
152
|
|
|
663
|
-
|
|
153
|
+
Future storage targets and compatibility adapters should be separate gems:
|
|
664
154
|
|
|
665
|
-
|
|
155
|
+
```text
|
|
156
|
+
whodunit-chronicles-postgres
|
|
157
|
+
whodunit-chronicles-mysql
|
|
158
|
+
whodunit-chronicles-mongo
|
|
159
|
+
whodunit-chronicles-s3
|
|
160
|
+
whodunit-chronicles-snowflake
|
|
161
|
+
whodunit-chronicles-paper_trail
|
|
162
|
+
```
|
|
666
163
|
|
|
667
|
-
|
|
164
|
+
Storage extensions implement the same `Ledger` contract. Compatibility adapters may translate ledger entries into another ecosystem's expected shape.
|