e11y 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (157) hide show
  1. checksums.yaml +7 -0
  2. data/.rspec +4 -0
  3. data/.rubocop.yml +69 -0
  4. data/CHANGELOG.md +26 -0
  5. data/CODE_OF_CONDUCT.md +64 -0
  6. data/LICENSE.txt +21 -0
  7. data/README.md +179 -0
  8. data/Rakefile +37 -0
  9. data/benchmarks/run_all.rb +33 -0
  10. data/config/README.md +83 -0
  11. data/config/loki-local-config.yaml +35 -0
  12. data/config/prometheus.yml +15 -0
  13. data/docker-compose.yml +78 -0
  14. data/docs/00-ICP-AND-TIMELINE.md +483 -0
  15. data/docs/01-SCALE-REQUIREMENTS.md +858 -0
  16. data/docs/ADR-001-architecture.md +2617 -0
  17. data/docs/ADR-002-metrics-yabeda.md +1395 -0
  18. data/docs/ADR-003-slo-observability.md +3337 -0
  19. data/docs/ADR-004-adapter-architecture.md +2385 -0
  20. data/docs/ADR-005-tracing-context.md +1372 -0
  21. data/docs/ADR-006-security-compliance.md +4143 -0
  22. data/docs/ADR-007-opentelemetry-integration.md +1385 -0
  23. data/docs/ADR-008-rails-integration.md +1911 -0
  24. data/docs/ADR-009-cost-optimization.md +2993 -0
  25. data/docs/ADR-010-developer-experience.md +2166 -0
  26. data/docs/ADR-011-testing-strategy.md +1836 -0
  27. data/docs/ADR-012-event-evolution.md +958 -0
  28. data/docs/ADR-013-reliability-error-handling.md +2750 -0
  29. data/docs/ADR-014-event-driven-slo.md +1533 -0
  30. data/docs/ADR-015-middleware-order.md +1061 -0
  31. data/docs/ADR-016-self-monitoring-slo.md +1234 -0
  32. data/docs/API-REFERENCE-L28.md +914 -0
  33. data/docs/COMPREHENSIVE-CONFIGURATION.md +2366 -0
  34. data/docs/IMPLEMENTATION_NOTES.md +2804 -0
  35. data/docs/IMPLEMENTATION_PLAN.md +1971 -0
  36. data/docs/IMPLEMENTATION_PLAN_ARCHITECTURE.md +586 -0
  37. data/docs/PLAN.md +148 -0
  38. data/docs/QUICK-START.md +934 -0
  39. data/docs/README.md +296 -0
  40. data/docs/design/00-memory-optimization.md +593 -0
  41. data/docs/guides/MIGRATION-L27-L28.md +692 -0
  42. data/docs/guides/PERFORMANCE-BENCHMARKS.md +434 -0
  43. data/docs/guides/README.md +44 -0
  44. data/docs/prd/01-overview-vision.md +440 -0
  45. data/docs/use_cases/README.md +119 -0
  46. data/docs/use_cases/UC-001-request-scoped-debug-buffering.md +813 -0
  47. data/docs/use_cases/UC-002-business-event-tracking.md +1953 -0
  48. data/docs/use_cases/UC-003-pattern-based-metrics.md +1627 -0
  49. data/docs/use_cases/UC-004-zero-config-slo-tracking.md +728 -0
  50. data/docs/use_cases/UC-005-sentry-integration.md +759 -0
  51. data/docs/use_cases/UC-006-trace-context-management.md +905 -0
  52. data/docs/use_cases/UC-007-pii-filtering.md +2648 -0
  53. data/docs/use_cases/UC-008-opentelemetry-integration.md +1153 -0
  54. data/docs/use_cases/UC-009-multi-service-tracing.md +1043 -0
  55. data/docs/use_cases/UC-010-background-job-tracking.md +1018 -0
  56. data/docs/use_cases/UC-011-rate-limiting.md +1906 -0
  57. data/docs/use_cases/UC-012-audit-trail.md +2301 -0
  58. data/docs/use_cases/UC-013-high-cardinality-protection.md +2127 -0
  59. data/docs/use_cases/UC-014-adaptive-sampling.md +1940 -0
  60. data/docs/use_cases/UC-015-cost-optimization.md +735 -0
  61. data/docs/use_cases/UC-016-rails-logger-migration.md +785 -0
  62. data/docs/use_cases/UC-017-local-development.md +867 -0
  63. data/docs/use_cases/UC-018-testing-events.md +1081 -0
  64. data/docs/use_cases/UC-019-tiered-storage-migration.md +562 -0
  65. data/docs/use_cases/UC-020-event-versioning.md +708 -0
  66. data/docs/use_cases/UC-021-error-handling-retry-dlq.md +956 -0
  67. data/docs/use_cases/UC-022-event-registry.md +648 -0
  68. data/docs/use_cases/backlog.md +226 -0
  69. data/e11y.gemspec +76 -0
  70. data/lib/e11y/adapters/adaptive_batcher.rb +207 -0
  71. data/lib/e11y/adapters/audit_encrypted.rb +239 -0
  72. data/lib/e11y/adapters/base.rb +580 -0
  73. data/lib/e11y/adapters/file.rb +224 -0
  74. data/lib/e11y/adapters/in_memory.rb +216 -0
  75. data/lib/e11y/adapters/loki.rb +333 -0
  76. data/lib/e11y/adapters/otel_logs.rb +203 -0
  77. data/lib/e11y/adapters/registry.rb +141 -0
  78. data/lib/e11y/adapters/sentry.rb +230 -0
  79. data/lib/e11y/adapters/stdout.rb +108 -0
  80. data/lib/e11y/adapters/yabeda.rb +370 -0
  81. data/lib/e11y/buffers/adaptive_buffer.rb +339 -0
  82. data/lib/e11y/buffers/base_buffer.rb +40 -0
  83. data/lib/e11y/buffers/request_scoped_buffer.rb +246 -0
  84. data/lib/e11y/buffers/ring_buffer.rb +267 -0
  85. data/lib/e11y/buffers.rb +14 -0
  86. data/lib/e11y/console.rb +122 -0
  87. data/lib/e11y/current.rb +48 -0
  88. data/lib/e11y/event/base.rb +894 -0
  89. data/lib/e11y/event/value_sampling_config.rb +84 -0
  90. data/lib/e11y/events/base_audit_event.rb +43 -0
  91. data/lib/e11y/events/base_payment_event.rb +33 -0
  92. data/lib/e11y/events/rails/cache/delete.rb +21 -0
  93. data/lib/e11y/events/rails/cache/read.rb +23 -0
  94. data/lib/e11y/events/rails/cache/write.rb +22 -0
  95. data/lib/e11y/events/rails/database/query.rb +45 -0
  96. data/lib/e11y/events/rails/http/redirect.rb +21 -0
  97. data/lib/e11y/events/rails/http/request.rb +26 -0
  98. data/lib/e11y/events/rails/http/send_file.rb +21 -0
  99. data/lib/e11y/events/rails/http/start_processing.rb +26 -0
  100. data/lib/e11y/events/rails/job/completed.rb +22 -0
  101. data/lib/e11y/events/rails/job/enqueued.rb +22 -0
  102. data/lib/e11y/events/rails/job/failed.rb +22 -0
  103. data/lib/e11y/events/rails/job/scheduled.rb +23 -0
  104. data/lib/e11y/events/rails/job/started.rb +22 -0
  105. data/lib/e11y/events/rails/log.rb +56 -0
  106. data/lib/e11y/events/rails/view/render.rb +23 -0
  107. data/lib/e11y/events.rb +18 -0
  108. data/lib/e11y/instruments/active_job.rb +201 -0
  109. data/lib/e11y/instruments/rails_instrumentation.rb +141 -0
  110. data/lib/e11y/instruments/sidekiq.rb +175 -0
  111. data/lib/e11y/logger/bridge.rb +205 -0
  112. data/lib/e11y/metrics/cardinality_protection.rb +172 -0
  113. data/lib/e11y/metrics/cardinality_tracker.rb +134 -0
  114. data/lib/e11y/metrics/registry.rb +234 -0
  115. data/lib/e11y/metrics/relabeling.rb +226 -0
  116. data/lib/e11y/metrics.rb +102 -0
  117. data/lib/e11y/middleware/audit_signing.rb +174 -0
  118. data/lib/e11y/middleware/base.rb +140 -0
  119. data/lib/e11y/middleware/event_slo.rb +167 -0
  120. data/lib/e11y/middleware/pii_filter.rb +266 -0
  121. data/lib/e11y/middleware/pii_filtering.rb +280 -0
  122. data/lib/e11y/middleware/rate_limiting.rb +214 -0
  123. data/lib/e11y/middleware/request.rb +163 -0
  124. data/lib/e11y/middleware/routing.rb +157 -0
  125. data/lib/e11y/middleware/sampling.rb +254 -0
  126. data/lib/e11y/middleware/slo.rb +168 -0
  127. data/lib/e11y/middleware/trace_context.rb +131 -0
  128. data/lib/e11y/middleware/validation.rb +118 -0
  129. data/lib/e11y/middleware/versioning.rb +132 -0
  130. data/lib/e11y/middleware.rb +12 -0
  131. data/lib/e11y/pii/patterns.rb +90 -0
  132. data/lib/e11y/pii.rb +13 -0
  133. data/lib/e11y/pipeline/builder.rb +155 -0
  134. data/lib/e11y/pipeline/zone_validator.rb +110 -0
  135. data/lib/e11y/pipeline.rb +12 -0
  136. data/lib/e11y/presets/audit_event.rb +65 -0
  137. data/lib/e11y/presets/debug_event.rb +34 -0
  138. data/lib/e11y/presets/high_value_event.rb +51 -0
  139. data/lib/e11y/presets.rb +19 -0
  140. data/lib/e11y/railtie.rb +138 -0
  141. data/lib/e11y/reliability/circuit_breaker.rb +216 -0
  142. data/lib/e11y/reliability/dlq/file_storage.rb +277 -0
  143. data/lib/e11y/reliability/dlq/filter.rb +117 -0
  144. data/lib/e11y/reliability/retry_handler.rb +207 -0
  145. data/lib/e11y/reliability/retry_rate_limiter.rb +117 -0
  146. data/lib/e11y/sampling/error_spike_detector.rb +225 -0
  147. data/lib/e11y/sampling/load_monitor.rb +161 -0
  148. data/lib/e11y/sampling/stratified_tracker.rb +92 -0
  149. data/lib/e11y/sampling/value_extractor.rb +82 -0
  150. data/lib/e11y/self_monitoring/buffer_monitor.rb +79 -0
  151. data/lib/e11y/self_monitoring/performance_monitor.rb +97 -0
  152. data/lib/e11y/self_monitoring/reliability_monitor.rb +146 -0
  153. data/lib/e11y/slo/event_driven.rb +150 -0
  154. data/lib/e11y/slo/tracker.rb +119 -0
  155. data/lib/e11y/version.rb +9 -0
  156. data/lib/e11y.rb +283 -0
  157. metadata +452 -0
@@ -0,0 +1,867 @@
1
+ # UC-017: Local Development
2
+
3
+ **Status:** MVP Feature
4
+ **Complexity:** Beginner
5
+ **Setup Time:** 5-10 minutes
6
+ **Target Users:** All Developers
7
+
8
+ ---
9
+
10
+ ## 📋 Overview
11
+
12
+ ### Problem Statement
13
+
14
+ **The local development pain:**
15
+ ```ruby
16
+ # ❌ BEFORE: Poor development experience
17
+ # - Events go to production backends (Loki, Sentry)
18
+ # - Can't see events in console (hidden in logs)
19
+ # - No colored output (hard to read)
20
+ # - No pretty-printing (JSON blobs)
21
+ # - Debug events flood console
22
+ # - Can't easily filter what you see
23
+
24
+ # Terminal output:
25
+ # {"event":"order.created","order_id":"123","timestamp":"2026-01-12T10:00:00Z"}
26
+ # {"event":"payment.processing","order_id":"123","timestamp":"2026-01-12T10:00:01Z"}
27
+ # {"event":"debug.sql","query":"SELECT...","timestamp":"2026-01-12T10:00:02Z"}
28
+ # → Hard to read! 😞
29
+ ```
30
+
31
+ ### E11y Solution
32
+
33
+ **Developer-friendly local setup:**
34
+ ```ruby
35
+ # ✅ AFTER: Beautiful, readable output
36
+ E11y.configure do |config|
37
+ if Rails.env.development?
38
+ # Beautiful colored console output
39
+ config.adapters = [
40
+ E11y::Adapters::ConsoleAdapter.new(
41
+ colored: true,
42
+ pretty: true,
43
+ show_payload: true,
44
+ show_context: true
45
+ )
46
+ ]
47
+
48
+ # Show all severities (including debug)
49
+ config.severity = :debug
50
+
51
+ # No rate limiting in dev
52
+ config.rate_limiting.enabled = false
53
+ end
54
+ end
55
+
56
+ # Terminal output (beautiful! 🎨):
57
+ # ╭─────────────────────────────────────────────────────────╮
58
+ # │ 🎉 order.created [SUCCESS] 10:00:00 │
59
+ # ├─────────────────────────────────────────────────────────┤
60
+ # │ order_id: 123 │
61
+ # │ user_id: 456 │
62
+ # │ amount: $99.99 │
63
+ # │ trace_id: abc-123-def │
64
+ # ╰─────────────────────────────────────────────────────────╯
65
+ ```
66
+
67
+ ---
68
+
69
+ ## 🎯 Features
70
+
71
+ > **Implementation:** See [ADR-010: Developer Experience](../ADR-010-developer-experience.md) for complete architecture, including [Section 3: Console Output](../ADR-010-developer-experience.md#3-console-output), [Section 4: Web UI](../ADR-010-developer-experience.md#4-web-ui), [Section 5: Event Registry](../ADR-010-developer-experience.md#5-event-registry), and [Section 6: Debug Helpers](../ADR-010-developer-experience.md#6-debug-helpers).
72
+
73
+ ### 1. Console Adapter (Pretty Output)
74
+
75
+ **Beautiful colored terminal output:**
76
+ ```ruby
77
+ # config/environments/development.rb
78
+ Rails.application.configure do
79
+ config.after_initialize do
80
+ E11y.configure do |config|
81
+ config.adapters = [
82
+ E11y::Adapters::ConsoleAdapter.new(
83
+ # Colors
84
+ colored: true,
85
+ color_scheme: :solarized, # :default, :solarized, :monokai
86
+
87
+ # Formatting
88
+ pretty: true,
89
+ compact: false,
90
+
91
+ # What to show
92
+ show_payload: true,
93
+ show_context: true,
94
+ show_metadata: false, # timestamps, etc.
95
+ show_trace_id: true,
96
+
97
+ # Filtering
98
+ severity_filter: :debug, # Show all
99
+ event_filter: nil, # Show all events
100
+
101
+ # Grouping
102
+ group_by_trace_id: true, # Group events with same trace_id
103
+
104
+ # Performance
105
+ max_payload_length: 1000, # Truncate long payloads
106
+ max_array_items: 10 # Limit array display
107
+ )
108
+ ]
109
+ end
110
+ end
111
+ end
112
+
113
+ # Output examples:
114
+ # ✅ SUCCESS event (green)
115
+ # 🎉 order.created [SUCCESS] 10:00:00
116
+ # order_id: 123
117
+ # amount: $99.99
118
+ # ⚡ Duration: 45ms
119
+
120
+ # ⚠️ WARN event (yellow)
121
+ # ⚠️ payment.retry [WARN] 10:00:05
122
+ # order_id: 123
123
+ # attempt: 2
124
+ # reason: "Card declined"
125
+
126
+ # ❌ ERROR event (red)
127
+ # ❌ payment.failed [ERROR] 10:00:10
128
+ # order_id: 123
129
+ # error: "Insufficient funds"
130
+ # trace_id: abc-123-def
131
+ ```
132
+
133
+ ---
134
+
135
+ ### 2. Event Inspector (Interactive)
136
+
137
+ **Interactive console for exploring events:**
138
+ ```ruby
139
+ # rails console
140
+ > E11y::Inspector.start
141
+ E11y Inspector started. Type 'help' for commands.
142
+
143
+ # Watch events in real-time
144
+ e11y> watch
145
+ Watching events... (Ctrl+C to stop)
146
+ [10:00:00] order.created { order_id: 123 }
147
+ [10:00:01] payment.processing { order_id: 123 }
148
+ [10:00:02] payment.succeeded { transaction_id: 'tx_123' }
149
+
150
+ # Filter by pattern
151
+ e11y> watch pattern: 'order.*'
152
+ Watching events matching 'order.*'...
153
+ [10:00:00] order.created { order_id: 123 }
154
+ [10:00:05] order.shipped { order_id: 123, tracking: 'TRACK123' }
155
+
156
+ # Filter by severity
157
+ e11y> watch severity: :error
158
+ Watching ERROR events...
159
+ [10:00:10] payment.failed { error: "Card declined" }
160
+
161
+ # Show last N events
162
+ e11y> last 10
163
+ Showing last 10 events:
164
+ 1. [10:00:00] order.created
165
+ 2. [10:00:01] payment.processing
166
+ 3. [10:00:02] payment.succeeded
167
+ ...
168
+
169
+ # Search events
170
+ e11y> search order_id: '123'
171
+ Found 5 events:
172
+ 1. [10:00:00] order.created
173
+ 2. [10:00:01] payment.processing
174
+ 3. [10:00:02] payment.succeeded
175
+ 4. [10:00:05] order.shipped
176
+ 5. [10:00:10] order.delivered
177
+
178
+ # Show event details
179
+ e11y> show 1
180
+ Event: order.created
181
+ Severity: SUCCESS
182
+ Timestamp: 2026-01-12 10:00:00
183
+ Trace ID: abc-123-def
184
+ Payload:
185
+ order_id: 123
186
+ user_id: 456
187
+ amount: 99.99
188
+ currency: USD
189
+ Context:
190
+ request_id: req-789
191
+ user_agent: Mozilla/5.0...
192
+ Duration: 45ms
193
+ ```
194
+
195
+ ---
196
+
197
+ ### 3. Debug Helper
198
+
199
+ **Quick debugging methods:**
200
+ ```ruby
201
+ # app/controllers/orders_controller.rb
202
+ class OrdersController < ApplicationController
203
+ def create
204
+ # Quick debug (only in development!)
205
+ E11y.debug("Creating order", order_params)
206
+ # → Pretty-printed to console immediately
207
+
208
+ order = Order.create!(order_params)
209
+
210
+ # Breakpoint with context
211
+ E11y.breakpoint(
212
+ "Order created",
213
+ order: order.attributes,
214
+ user: current_user.attributes
215
+ )
216
+ # → Pauses execution, shows data, waits for Enter
217
+
218
+ # Measure block
219
+ result = E11y.measure("Payment processing") do
220
+ process_payment(order)
221
+ end
222
+ # → Logs duration automatically
223
+
224
+ render json: order
225
+ end
226
+ end
227
+
228
+ # Console output:
229
+ # 🔍 [DEBUG] Creating order
230
+ # user_id: 456
231
+ # items: [...]
232
+ # total: 99.99
233
+ #
234
+ # ⏸️ [BREAKPOINT] Order created
235
+ # order: { id: 123, status: "pending", ... }
236
+ # user: { id: 456, email: "user@example.com", ... }
237
+ # Press Enter to continue...
238
+ #
239
+ # ⏱️ [MEASURE] Payment processing → 1.2s
240
+ ```
241
+
242
+ ---
243
+
244
+ ### 4. Event Recorder (Playback)
245
+
246
+ **Record and replay events for testing:**
247
+ ```ruby
248
+ # Record events during a request
249
+ # rails console
250
+ > recorder = E11y::Recorder.new
251
+ > recorder.start
252
+ Recording events...
253
+
254
+ # Make request
255
+ > app.post '/orders', params: { order: {...} }
256
+
257
+ > recorder.stop
258
+ Recorded 15 events
259
+ Saved to tmp/e11y_recordings/2026-01-12_10-00-00.json
260
+
261
+ # Replay events
262
+ > recorder.replay('tmp/e11y_recordings/2026-01-12_10-00-00.json')
263
+ Replaying 15 events...
264
+ [1/15] order.creation.started
265
+ [2/15] inventory.checked
266
+ [3/15] payment.processing
267
+ ...
268
+ [15/15] order.created
269
+
270
+ # Compare recordings (regression testing)
271
+ > diff = E11y::Recorder.diff(
272
+ 'recordings/baseline.json',
273
+ 'recordings/current.json'
274
+ )
275
+ > puts diff
276
+ + payment.retry (NEW in current)
277
+ - payment.succeeded (MISSING in current)
278
+ ~ payment.processing.duration_ms: 120ms → 1500ms (12.5x slower!)
279
+ ```
280
+
281
+ ---
282
+
283
+ ### 5. Visual Timeline (Web UI)
284
+
285
+ **Mini web UI for development:**
286
+ ```ruby
287
+ # config/routes.rb (development only)
288
+ Rails.application.routes.draw do
289
+ if Rails.env.development?
290
+ mount E11y::Web => '/e11y'
291
+ end
292
+ end
293
+
294
+ # Visit: http://localhost:3000/e11y
295
+ # Features:
296
+ # - Real-time event stream
297
+ # - Timeline view (Gantt chart)
298
+ # - Filtering by severity, pattern
299
+ # - Trace visualization
300
+ # - Event details modal
301
+ # - Export to JSON/CSV
302
+ # - Search & filter
303
+
304
+ # Example UI:
305
+ # ╔══════════════════════════════════════════════════════════╗
306
+ # ║ E11y Event Dashboard 🔄 Auto-refresh ║
307
+ # ╠══════════════════════════════════════════════════════════╣
308
+ # ║ Filters: [All Severities ▾] [All Events ▾] [Search...] ║
309
+ # ╠══════════════════════════════════════════════════════════╣
310
+ # ║ Timeline (Last 5 minutes) ║
311
+ # ║ ┌────────────────────────────────────────────────────┐ ║
312
+ # ║ │ 10:00:00 ████ order.created │ ║
313
+ # ║ │ 10:00:01 ███████ payment.processing │ ║
314
+ # ║ │ 10:00:02 ██ payment.succeeded │ ║
315
+ # ║ │ 10:00:05 ████ shipment.created │ ║
316
+ # ║ └────────────────────────────────────────────────────┘ ║
317
+ # ╠══════════════════════════════════════════════════════════╣
318
+ # ║ Recent Events ║
319
+ # ║ ✅ order.created 10:00:00 trace: abc-123 ║
320
+ # ║ ⏳ payment.processing 10:00:01 trace: abc-123 ║
321
+ # ║ ✅ payment.succeeded 10:00:02 trace: abc-123 ║
322
+ # ╚══════════════════════════════════════════════════════════╝
323
+ ```
324
+
325
+ ---
326
+
327
+ ### 6. Environment-Specific Configuration Recommendations (C14)
328
+
329
+ > **Implementation:** See [ADR-010 Section 2: Development vs Production](../ADR-010-developer-experience.md#2-development-vs-production-configuration) for detailed architecture rationale.
330
+
331
+ **Critical differences between development and production configurations:**
332
+
333
+ **Development: Immediate Feedback (Zero Delay)**
334
+
335
+ ```ruby
336
+ # config/environments/development.rb
337
+ E11y.configure do |config|
338
+ # === BUFFERING: DISABLED (immediate writes) ===
339
+ # ✅ Events appear INSTANTLY in console
340
+ # ✅ No need to wait for flush
341
+ # ⚠️ Trade-off: Slightly slower per-request performance (acceptable in dev)
342
+ config.buffering.enabled = false
343
+
344
+ # OR: Very short interval (near-immediate)
345
+ # config.buffering do
346
+ # enabled true
347
+ # flush_interval 0.1.seconds # Flush every 100ms
348
+ # end
349
+
350
+ # === SAMPLING: DISABLED (keep all events) ===
351
+ # ✅ See EVERY event for complete debugging
352
+ # ✅ No data loss during development
353
+ # ⚠️ Trade-off: More console noise (filter with ignore_events)
354
+ config.sampling.enabled = false
355
+
356
+ # === RATE LIMITING: DISABLED (no throttling) ===
357
+ # ✅ Rapid testing won't hit limits
358
+ config.rate_limiting.enabled = false
359
+
360
+ # === PII FILTERING: DISABLED (optional) ===
361
+ # ✅ See real data for easier debugging
362
+ # ⚠️ Only disable if you're NOT using production data in dev!
363
+ config.pii_filtering.enabled = false
364
+
365
+ # === FLUSH HELPER: Available for manual testing ===
366
+ # Sometimes you want to force-flush buffered events:
367
+ # E11y.flush # ← Forces immediate flush
368
+ end
369
+ ```
370
+
371
+ **Production: Performance & Cost Optimization**
372
+
373
+ ```ruby
374
+ # config/environments/production.rb
375
+ E11y.configure do |config|
376
+ # === BUFFERING: ENABLED (batch writes) ===
377
+ # ✅ 10× performance improvement (batching)
378
+ # ✅ 50% cost reduction (fewer network calls)
379
+ # ⚠️ Trade-off: Events delayed by up to 10s
380
+ config.buffering do
381
+ enabled true
382
+ flush_interval 10.seconds
383
+ max_buffer_size 1000
384
+ end
385
+
386
+ # === SAMPLING: ENABLED (cost savings) ===
387
+ # ✅ 50-80% cost reduction
388
+ # ✅ Errors always kept (100% sampling)
389
+ # ⚠️ Trade-off: Some success events dropped
390
+ config.sampling do
391
+ enabled true
392
+ strategy :adaptive
393
+ base_rate 0.1 # Keep 10% of success events
394
+ error_rate 1.0 # Keep 100% of errors
395
+ end
396
+
397
+ # === RATE LIMITING: ENABLED (DDoS protection) ===
398
+ # ✅ Prevents cost explosions
399
+ # ✅ Protects backend from overload
400
+ config.rate_limiting do
401
+ enabled true
402
+ limit 1000
403
+ window 1.minute
404
+ end
405
+
406
+ # === PII FILTERING: ENABLED (GDPR compliance) ===
407
+ # ✅ GDPR/CCPA compliant
408
+ # ✅ Protects sensitive data
409
+ config.pii_filtering.enabled true
410
+ end
411
+ ```
412
+
413
+ **Comparison Table:**
414
+
415
+ | Feature | Development | Production | Why Different? |
416
+ |---------|-------------|------------|----------------|
417
+ | **Buffering** | ❌ Disabled (immediate) | ✅ Enabled (10s batches) | Dev needs instant feedback, prod needs performance |
418
+ | **Sampling** | ❌ Disabled (100%) | ✅ Enabled (10-50%) | Dev needs complete data, prod needs cost savings |
419
+ | **Rate Limiting** | ❌ Disabled | ✅ Enabled (1000/min) | Dev needs rapid testing, prod needs DDoS protection |
420
+ | **PII Filtering** | ⚠️ Optional (easier debugging) | ✅ Enabled (GDPR) | Dev may use fake data, prod has real user data |
421
+ | **Flush Helper** | ✅ `E11y.flush` available | ⚠️ Available but rarely needed | Dev uses for manual testing |
422
+
423
+ **Manual Flush Helper (Development Testing):**
424
+
425
+ ```ruby
426
+ # Scenario: Testing event delivery in specs
427
+ RSpec.describe 'Order creation' do
428
+ it 'tracks order.created event' do
429
+ # Event is buffered (not sent yet)
430
+ post '/orders', params: { order: {...} }
431
+
432
+ # Force immediate flush (for testing)
433
+ E11y.flush
434
+
435
+ # Now event is available in test adapter
436
+ expect(E11y.test_adapter.events).to include(
437
+ hash_including(event_name: 'order.created')
438
+ )
439
+ end
440
+ end
441
+
442
+ # Rails console manual testing:
443
+ > Events::OrderCreated.track(order_id: 123)
444
+ # → Event buffered, not visible yet
445
+
446
+ > E11y.flush
447
+ # → Forces immediate flush, event now visible in console
448
+
449
+ > E11y.buffer.size
450
+ # => 0 (buffer is empty after flush)
451
+ ```
452
+
453
+ **Trade-offs & Gotchas:**
454
+
455
+ **⚠️ Warning: Development ≠ Production Behavior**
456
+
457
+ ```ruby
458
+ # ❌ GOTCHA: Event appears instantly in dev, delayed in prod
459
+ # Development (buffering disabled):
460
+ Events::OrderCreated.track(order_id: 123)
461
+ # → Appears in console IMMEDIATELY ✅
462
+
463
+ # Production (buffering enabled, 10s flush):
464
+ Events::OrderCreated.track(order_id: 123)
465
+ # → Buffered, appears after 10 seconds ⏱️
466
+ # → Or when buffer full (1000 events)
467
+ # → Or when E11y.flush called manually
468
+
469
+ # Solution: If you need instant delivery in prod (e.g., critical alerts):
470
+ Events::CriticalAlert.track(
471
+ alert_type: 'payment_failure',
472
+ severity: :fatal
473
+ )
474
+ E11y.flush # ← Force immediate delivery (bypasses buffer)
475
+ ```
476
+
477
+ **⚠️ Warning: Sampling Differences**
478
+
479
+ ```ruby
480
+ # ❌ GOTCHA: All events visible in dev, some dropped in prod
481
+ # Development (sampling disabled):
482
+ 100.times { Events::UserLogin.track(user_id: rand(1000)) }
483
+ # → See ALL 100 events in console ✅
484
+
485
+ # Production (sampling enabled, 10% rate):
486
+ 100.times { Events::UserLogin.track(user_id: rand(1000)) }
487
+ # → Only ~10 events reach Loki (90 dropped) ❌
488
+ # → Errors ALWAYS kept (100% sampling) ✅
489
+
490
+ # Solution: Test sampling behavior in staging:
491
+ # config/environments/staging.rb
492
+ config.sampling do
493
+ enabled true # ← Test production-like sampling
494
+ base_rate 0.1
495
+ end
496
+ ```
497
+
498
+ **Staging Environment (Recommended Middle Ground):**
499
+
500
+ ```ruby
501
+ # config/environments/staging.rb
502
+ E11y.configure do |config|
503
+ # Balanced config: production-like but easier to debug
504
+
505
+ # Buffering: Shorter interval (faster feedback than prod)
506
+ config.buffering do
507
+ enabled true
508
+ flush_interval 1.second # vs 10s in prod
509
+ end
510
+
511
+ # Sampling: Higher rate (more data than prod)
512
+ config.sampling do
513
+ enabled true
514
+ base_rate 0.5 # vs 0.1 in prod (keep 50% of events)
515
+ error_rate 1.0 # Always keep errors
516
+ end
517
+
518
+ # Rate limiting: Higher limits (easier testing)
519
+ config.rate_limiting do
520
+ enabled true
521
+ limit 10_000 # vs 1000 in prod
522
+ end
523
+
524
+ # PII: Enabled (test GDPR compliance)
525
+ config.pii_filtering.enabled true
526
+ end
527
+ ```
528
+
529
+ **Key Takeaways:**
530
+
531
+ 1. **Development:** Disable buffering & sampling for instant, complete feedback
532
+ 2. **Production:** Enable buffering & sampling for performance & cost savings
533
+ 3. **Staging:** Middle ground - production-like but easier to debug
534
+ 4. **Use `E11y.flush`:** For manual testing when buffering is enabled
535
+ 5. **Test in staging:** Catch production behavior differences before deployment
536
+
537
+ ---
538
+
539
+ ## 💻 Implementation Examples
540
+
541
+ ### Example 1: Full Development Config
542
+
543
+ ```ruby
544
+ # config/environments/development.rb
545
+ Rails.application.configure do
546
+ config.after_initialize do
547
+ E11y.configure do |config|
548
+ # === CONSOLE OUTPUT ===
549
+ config.adapters = [
550
+ E11y::Adapters::ConsoleAdapter.new(
551
+ colored: true,
552
+ pretty: true,
553
+ show_payload: true,
554
+ show_context: true,
555
+ show_trace_id: true,
556
+ group_by_trace_id: true
557
+ )
558
+ ]
559
+
560
+ # === SEVERITY ===
561
+ # Show everything in development
562
+ config.severity = :debug
563
+
564
+ # === FEATURES ===
565
+ # Disable production features
566
+ config.rate_limiting.enabled = false
567
+ config.sampling.enabled = false
568
+ config.pii_filtering.enabled = false # Easier debugging
569
+
570
+ # === DEBUGGING ===
571
+ # Enable debug helpers
572
+ config.debug_mode = true
573
+
574
+ # === BUFFERING ===
575
+ # Immediate flush (no buffering)
576
+ config.buffer.enabled = false # Or flush_interval: 0.1.seconds
577
+
578
+ # === WEB UI ===
579
+ config.web_ui do
580
+ enabled true
581
+ port 3001 # Or use Rails server
582
+ auto_refresh true
583
+ refresh_interval 2.seconds
584
+ end
585
+
586
+ # === RECORDING ===
587
+ config.recording do
588
+ enabled true
589
+ save_path Rails.root.join('tmp', 'e11y_recordings')
590
+ auto_save_on_error true
591
+ end
592
+
593
+ # === PERFORMANCE ===
594
+ # Verbose self-monitoring
595
+ config.self_monitoring do
596
+ enabled true
597
+ log_internal_events true
598
+ end
599
+ end
600
+ end
601
+ end
602
+ ```
603
+
604
+ ---
605
+
606
+ ### Example 2: Debug Helpers in Code
607
+
608
+ ```ruby
609
+ # app/services/order_processing_service.rb
610
+ class OrderProcessingService
611
+ def call(order_id)
612
+ # Debug checkpoint
613
+ E11y.debug("Starting order processing", order_id: order_id)
614
+
615
+ order = Order.find(order_id)
616
+
617
+ # Show complex data structure
618
+ E11y.inspect(order, depth: 2)
619
+ # → Pretty-printed with colors, max depth 2
620
+
621
+ # Measure performance
622
+ inventory_result = E11y.measure("Inventory check") do
623
+ check_inventory(order)
624
+ end
625
+
626
+ # Conditional breakpoint
627
+ E11y.breakpoint_if(
628
+ -> { inventory_result.low_stock? },
629
+ "Low stock detected!",
630
+ order: order.attributes,
631
+ inventory: inventory_result
632
+ )
633
+
634
+ # Trace execution
635
+ E11y.trace("Processing payment") do
636
+ process_payment(order)
637
+ end
638
+ # → Logs entry/exit automatically
639
+
640
+ # Count invocations
641
+ E11y.count("order_processing")
642
+ # → Logs: "order_processing called 5 times"
643
+
644
+ # Diff objects
645
+ before = order.attributes
646
+ order.update!(status: 'processed')
647
+ E11y.diff(before, order.attributes)
648
+ # → Shows: { status: "pending" → "processed" }
649
+
650
+ order
651
+ end
652
+ end
653
+
654
+ # Console output:
655
+ # 🔍 [DEBUG] Starting order processing
656
+ # order_id: 123
657
+ #
658
+ # 📦 [INSPECT] Order #123
659
+ # id: 123
660
+ # status: "pending"
661
+ # items: [
662
+ # { id: 1, product_id: 456, quantity: 2 },
663
+ # { id: 2, product_id: 789, quantity: 1 }
664
+ # ]
665
+ # total: 99.99
666
+ #
667
+ # ⏱️ [MEASURE] Inventory check → 45ms
668
+ #
669
+ # ⏸️ [BREAKPOINT] Low stock detected!
670
+ # order: { ... }
671
+ # inventory: { low_stock: true, product_id: 456 }
672
+ # Press Enter to continue...
673
+ #
674
+ # 🔀 [TRACE] Processing payment
675
+ # → Entered at 10:00:01
676
+ # → Exited at 10:00:02 (1.2s)
677
+ #
678
+ # 📊 [COUNT] order_processing called 5 times
679
+ #
680
+ # 🔄 [DIFF] Order #123
681
+ # - status: "pending"
682
+ # + status: "processed"
683
+ ```
684
+
685
+ ---
686
+
687
+ ### Example 3: Event Filtering
688
+
689
+ ```ruby
690
+ # config/environments/development.rb
691
+ E11y.configure do |config|
692
+ config.adapters = [
693
+ E11y::Adapters::ConsoleAdapter.new(
694
+ colored: true,
695
+ pretty: true,
696
+
697
+ # === FILTERING ===
698
+ # Only show events matching patterns
699
+ event_filter: ->(event) {
700
+ # Show orders & payments, hide everything else
701
+ event.event_name.match?(/^(order|payment)\./)
702
+ },
703
+
704
+ # Only show warn/error/success
705
+ severity_filter: [:warn, :error, :fatal, :success],
706
+
707
+ # Hide noisy events
708
+ ignore_events: [
709
+ 'health_check',
710
+ 'heartbeat',
711
+ 'metrics.collected'
712
+ ],
713
+
714
+ # Hide debug SQL queries
715
+ ignore_patterns: [
716
+ /^debug\./,
717
+ /\.sql$/
718
+ ]
719
+ )
720
+ ]
721
+ end
722
+
723
+ # Result:
724
+ # ✅ Shows: order.created, payment.succeeded
725
+ # ❌ Hides: debug.sql, health_check, heartbeat
726
+ ```
727
+
728
+ ---
729
+
730
+ ## 🔧 Configuration
731
+
732
+ ### Development Config Template
733
+
734
+ ```ruby
735
+ # config/environments/development.rb
736
+ Rails.application.configure do
737
+ config.after_initialize do
738
+ E11y.configure do |config|
739
+ # === OUTPUT ===
740
+ config.adapters = [
741
+ E11y::Adapters::ConsoleAdapter.new(
742
+ colored: true,
743
+ pretty: true,
744
+ color_scheme: :solarized,
745
+ show_payload: true,
746
+ show_context: true,
747
+ show_trace_id: true,
748
+ group_by_trace_id: true,
749
+ max_payload_length: 1000
750
+ )
751
+ ]
752
+
753
+ # === LEVEL ===
754
+ config.severity = :debug
755
+
756
+ # === FEATURES ===
757
+ config.rate_limiting.enabled = false
758
+ config.sampling.enabled = false
759
+ config.buffering.enabled = false
760
+
761
+ # === DEBUG ===
762
+ config.debug_mode = true
763
+ config.debug_helpers.enabled = true
764
+
765
+ # === WEB UI ===
766
+ config.web_ui.enabled = true
767
+ config.web_ui.auto_refresh = true
768
+
769
+ # === RECORDING ===
770
+ config.recording.enabled = true
771
+ config.recording.auto_save_on_error = true
772
+ end
773
+ end
774
+ end
775
+ ```
776
+
777
+ ---
778
+
779
+ ## 💡 Best Practices
780
+
781
+ ### ✅ DO
782
+
783
+ **1. Use colored console output**
784
+ ```ruby
785
+ # ✅ GOOD: Easy to read
786
+ config.adapters = [
787
+ E11y::Adapters::ConsoleAdapter.new(
788
+ colored: true,
789
+ pretty: true
790
+ )
791
+ ]
792
+ ```
793
+
794
+ **2. Filter noise in development**
795
+ ```ruby
796
+ # ✅ GOOD: Hide irrelevant events
797
+ config.adapters = [
798
+ E11y::Adapters::ConsoleAdapter.new(
799
+ ignore_events: ['health_check', 'heartbeat']
800
+ )
801
+ ]
802
+ ```
803
+
804
+ **3. Use debug helpers**
805
+ ```ruby
806
+ # ✅ GOOD: Quick debugging
807
+ E11y.debug("User logged in", user_id: user.id)
808
+ E11y.breakpoint("Check this", data: complex_data)
809
+ ```
810
+
811
+ ---
812
+
813
+ ### ❌ DON'T
814
+
815
+ **1. Don't use production adapters**
816
+ ```ruby
817
+ # ❌ BAD: Production adapters in development
818
+ config.adapters = [
819
+ E11y::Adapters::LokiAdapter.new(...) # Slow!
820
+ ]
821
+
822
+ # ✅ GOOD: Console adapter
823
+ config.adapters = [
824
+ E11y::Adapters::ConsoleAdapter.new(...)
825
+ ]
826
+ ```
827
+
828
+ **2. Don't forget to disable features**
829
+ ```ruby
830
+ # ❌ BAD: Production features enabled
831
+ config.rate_limiting.enabled = true # Annoying in dev!
832
+ config.sampling.enabled = true # Lose events!
833
+
834
+ # ✅ GOOD: Disable in development
835
+ config.rate_limiting.enabled = false
836
+ config.sampling.enabled = false
837
+ ```
838
+
839
+ ---
840
+
841
+ ## 📚 Related Use Cases
842
+
843
+ - **[UC-016: Rails Logger Migration](./UC-016-rails-logger-migration.md)** - Migration guide
844
+ - **[UC-018: Testing Events](./UC-018-testing-events.md)** - Testing helpers
845
+
846
+ ---
847
+
848
+ ## 🎯 Summary
849
+
850
+ ### Development Experience
851
+
852
+ | Feature | Before | After |
853
+ |---------|--------|-------|
854
+ | **Output** | JSON blobs | Colored, pretty |
855
+ | **Filtering** | None | By severity, pattern |
856
+ | **Debugging** | `puts` | Debug helpers |
857
+ | **Inspection** | Manual | Interactive inspector |
858
+ | **Recording** | None | Record & replay |
859
+ | **Visualization** | None | Web UI |
860
+
861
+ **Setup Time:** 5-10 minutes (one-time config)
862
+
863
+ ---
864
+
865
+ **Document Version:** 1.0
866
+ **Last Updated:** January 12, 2026
867
+ **Status:** ✅ Complete