e11y 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (157) hide show
  1. checksums.yaml +7 -0
  2. data/.rspec +4 -0
  3. data/.rubocop.yml +69 -0
  4. data/CHANGELOG.md +26 -0
  5. data/CODE_OF_CONDUCT.md +64 -0
  6. data/LICENSE.txt +21 -0
  7. data/README.md +179 -0
  8. data/Rakefile +37 -0
  9. data/benchmarks/run_all.rb +33 -0
  10. data/config/README.md +83 -0
  11. data/config/loki-local-config.yaml +35 -0
  12. data/config/prometheus.yml +15 -0
  13. data/docker-compose.yml +78 -0
  14. data/docs/00-ICP-AND-TIMELINE.md +483 -0
  15. data/docs/01-SCALE-REQUIREMENTS.md +858 -0
  16. data/docs/ADR-001-architecture.md +2617 -0
  17. data/docs/ADR-002-metrics-yabeda.md +1395 -0
  18. data/docs/ADR-003-slo-observability.md +3337 -0
  19. data/docs/ADR-004-adapter-architecture.md +2385 -0
  20. data/docs/ADR-005-tracing-context.md +1372 -0
  21. data/docs/ADR-006-security-compliance.md +4143 -0
  22. data/docs/ADR-007-opentelemetry-integration.md +1385 -0
  23. data/docs/ADR-008-rails-integration.md +1911 -0
  24. data/docs/ADR-009-cost-optimization.md +2993 -0
  25. data/docs/ADR-010-developer-experience.md +2166 -0
  26. data/docs/ADR-011-testing-strategy.md +1836 -0
  27. data/docs/ADR-012-event-evolution.md +958 -0
  28. data/docs/ADR-013-reliability-error-handling.md +2750 -0
  29. data/docs/ADR-014-event-driven-slo.md +1533 -0
  30. data/docs/ADR-015-middleware-order.md +1061 -0
  31. data/docs/ADR-016-self-monitoring-slo.md +1234 -0
  32. data/docs/API-REFERENCE-L28.md +914 -0
  33. data/docs/COMPREHENSIVE-CONFIGURATION.md +2366 -0
  34. data/docs/IMPLEMENTATION_NOTES.md +2804 -0
  35. data/docs/IMPLEMENTATION_PLAN.md +1971 -0
  36. data/docs/IMPLEMENTATION_PLAN_ARCHITECTURE.md +586 -0
  37. data/docs/PLAN.md +148 -0
  38. data/docs/QUICK-START.md +934 -0
  39. data/docs/README.md +296 -0
  40. data/docs/design/00-memory-optimization.md +593 -0
  41. data/docs/guides/MIGRATION-L27-L28.md +692 -0
  42. data/docs/guides/PERFORMANCE-BENCHMARKS.md +434 -0
  43. data/docs/guides/README.md +44 -0
  44. data/docs/prd/01-overview-vision.md +440 -0
  45. data/docs/use_cases/README.md +119 -0
  46. data/docs/use_cases/UC-001-request-scoped-debug-buffering.md +813 -0
  47. data/docs/use_cases/UC-002-business-event-tracking.md +1953 -0
  48. data/docs/use_cases/UC-003-pattern-based-metrics.md +1627 -0
  49. data/docs/use_cases/UC-004-zero-config-slo-tracking.md +728 -0
  50. data/docs/use_cases/UC-005-sentry-integration.md +759 -0
  51. data/docs/use_cases/UC-006-trace-context-management.md +905 -0
  52. data/docs/use_cases/UC-007-pii-filtering.md +2648 -0
  53. data/docs/use_cases/UC-008-opentelemetry-integration.md +1153 -0
  54. data/docs/use_cases/UC-009-multi-service-tracing.md +1043 -0
  55. data/docs/use_cases/UC-010-background-job-tracking.md +1018 -0
  56. data/docs/use_cases/UC-011-rate-limiting.md +1906 -0
  57. data/docs/use_cases/UC-012-audit-trail.md +2301 -0
  58. data/docs/use_cases/UC-013-high-cardinality-protection.md +2127 -0
  59. data/docs/use_cases/UC-014-adaptive-sampling.md +1940 -0
  60. data/docs/use_cases/UC-015-cost-optimization.md +735 -0
  61. data/docs/use_cases/UC-016-rails-logger-migration.md +785 -0
  62. data/docs/use_cases/UC-017-local-development.md +867 -0
  63. data/docs/use_cases/UC-018-testing-events.md +1081 -0
  64. data/docs/use_cases/UC-019-tiered-storage-migration.md +562 -0
  65. data/docs/use_cases/UC-020-event-versioning.md +708 -0
  66. data/docs/use_cases/UC-021-error-handling-retry-dlq.md +956 -0
  67. data/docs/use_cases/UC-022-event-registry.md +648 -0
  68. data/docs/use_cases/backlog.md +226 -0
  69. data/e11y.gemspec +76 -0
  70. data/lib/e11y/adapters/adaptive_batcher.rb +207 -0
  71. data/lib/e11y/adapters/audit_encrypted.rb +239 -0
  72. data/lib/e11y/adapters/base.rb +580 -0
  73. data/lib/e11y/adapters/file.rb +224 -0
  74. data/lib/e11y/adapters/in_memory.rb +216 -0
  75. data/lib/e11y/adapters/loki.rb +333 -0
  76. data/lib/e11y/adapters/otel_logs.rb +203 -0
  77. data/lib/e11y/adapters/registry.rb +141 -0
  78. data/lib/e11y/adapters/sentry.rb +230 -0
  79. data/lib/e11y/adapters/stdout.rb +108 -0
  80. data/lib/e11y/adapters/yabeda.rb +370 -0
  81. data/lib/e11y/buffers/adaptive_buffer.rb +339 -0
  82. data/lib/e11y/buffers/base_buffer.rb +40 -0
  83. data/lib/e11y/buffers/request_scoped_buffer.rb +246 -0
  84. data/lib/e11y/buffers/ring_buffer.rb +267 -0
  85. data/lib/e11y/buffers.rb +14 -0
  86. data/lib/e11y/console.rb +122 -0
  87. data/lib/e11y/current.rb +48 -0
  88. data/lib/e11y/event/base.rb +894 -0
  89. data/lib/e11y/event/value_sampling_config.rb +84 -0
  90. data/lib/e11y/events/base_audit_event.rb +43 -0
  91. data/lib/e11y/events/base_payment_event.rb +33 -0
  92. data/lib/e11y/events/rails/cache/delete.rb +21 -0
  93. data/lib/e11y/events/rails/cache/read.rb +23 -0
  94. data/lib/e11y/events/rails/cache/write.rb +22 -0
  95. data/lib/e11y/events/rails/database/query.rb +45 -0
  96. data/lib/e11y/events/rails/http/redirect.rb +21 -0
  97. data/lib/e11y/events/rails/http/request.rb +26 -0
  98. data/lib/e11y/events/rails/http/send_file.rb +21 -0
  99. data/lib/e11y/events/rails/http/start_processing.rb +26 -0
  100. data/lib/e11y/events/rails/job/completed.rb +22 -0
  101. data/lib/e11y/events/rails/job/enqueued.rb +22 -0
  102. data/lib/e11y/events/rails/job/failed.rb +22 -0
  103. data/lib/e11y/events/rails/job/scheduled.rb +23 -0
  104. data/lib/e11y/events/rails/job/started.rb +22 -0
  105. data/lib/e11y/events/rails/log.rb +56 -0
  106. data/lib/e11y/events/rails/view/render.rb +23 -0
  107. data/lib/e11y/events.rb +18 -0
  108. data/lib/e11y/instruments/active_job.rb +201 -0
  109. data/lib/e11y/instruments/rails_instrumentation.rb +141 -0
  110. data/lib/e11y/instruments/sidekiq.rb +175 -0
  111. data/lib/e11y/logger/bridge.rb +205 -0
  112. data/lib/e11y/metrics/cardinality_protection.rb +172 -0
  113. data/lib/e11y/metrics/cardinality_tracker.rb +134 -0
  114. data/lib/e11y/metrics/registry.rb +234 -0
  115. data/lib/e11y/metrics/relabeling.rb +226 -0
  116. data/lib/e11y/metrics.rb +102 -0
  117. data/lib/e11y/middleware/audit_signing.rb +174 -0
  118. data/lib/e11y/middleware/base.rb +140 -0
  119. data/lib/e11y/middleware/event_slo.rb +167 -0
  120. data/lib/e11y/middleware/pii_filter.rb +266 -0
  121. data/lib/e11y/middleware/pii_filtering.rb +280 -0
  122. data/lib/e11y/middleware/rate_limiting.rb +214 -0
  123. data/lib/e11y/middleware/request.rb +163 -0
  124. data/lib/e11y/middleware/routing.rb +157 -0
  125. data/lib/e11y/middleware/sampling.rb +254 -0
  126. data/lib/e11y/middleware/slo.rb +168 -0
  127. data/lib/e11y/middleware/trace_context.rb +131 -0
  128. data/lib/e11y/middleware/validation.rb +118 -0
  129. data/lib/e11y/middleware/versioning.rb +132 -0
  130. data/lib/e11y/middleware.rb +12 -0
  131. data/lib/e11y/pii/patterns.rb +90 -0
  132. data/lib/e11y/pii.rb +13 -0
  133. data/lib/e11y/pipeline/builder.rb +155 -0
  134. data/lib/e11y/pipeline/zone_validator.rb +110 -0
  135. data/lib/e11y/pipeline.rb +12 -0
  136. data/lib/e11y/presets/audit_event.rb +65 -0
  137. data/lib/e11y/presets/debug_event.rb +34 -0
  138. data/lib/e11y/presets/high_value_event.rb +51 -0
  139. data/lib/e11y/presets.rb +19 -0
  140. data/lib/e11y/railtie.rb +138 -0
  141. data/lib/e11y/reliability/circuit_breaker.rb +216 -0
  142. data/lib/e11y/reliability/dlq/file_storage.rb +277 -0
  143. data/lib/e11y/reliability/dlq/filter.rb +117 -0
  144. data/lib/e11y/reliability/retry_handler.rb +207 -0
  145. data/lib/e11y/reliability/retry_rate_limiter.rb +117 -0
  146. data/lib/e11y/sampling/error_spike_detector.rb +225 -0
  147. data/lib/e11y/sampling/load_monitor.rb +161 -0
  148. data/lib/e11y/sampling/stratified_tracker.rb +92 -0
  149. data/lib/e11y/sampling/value_extractor.rb +82 -0
  150. data/lib/e11y/self_monitoring/buffer_monitor.rb +79 -0
  151. data/lib/e11y/self_monitoring/performance_monitor.rb +97 -0
  152. data/lib/e11y/self_monitoring/reliability_monitor.rb +146 -0
  153. data/lib/e11y/slo/event_driven.rb +150 -0
  154. data/lib/e11y/slo/tracker.rb +119 -0
  155. data/lib/e11y/version.rb +9 -0
  156. data/lib/e11y.rb +283 -0
  157. metadata +452 -0
@@ -0,0 +1,934 @@
1
+ # E11y - Quick Start Guide
2
+
3
+ > **TL;DR**: Ruby gem для структурированных бизнес-событий с request-scoped debug buffering, pattern-based метриками и pluggable адаптерами.
4
+
5
+ ---
6
+
7
+ ## 🚀 Installation (5 minutes)
8
+
9
+ ```bash
10
+ # Gemfile
11
+ gem 'e11y', '~> 1.0'
12
+
13
+ bundle install
14
+ rails g e11y:install
15
+ ```
16
+
17
+ ---
18
+
19
+ ## 🎯 Killer Features
20
+
21
+ ### 1. Request-Scoped Debug Buffering
22
+
23
+ **Проблема**: Debug логи в production = шум. Без debug = нет контекста при ошибках.
24
+
25
+ **Решение**: Debug события буферизируются в thread-local storage. При success - дропаются, при error - флашатся.
26
+
27
+ ```ruby
28
+ GET /api/orders/123
29
+ ├─ [debug] Query: SELECT... (buffered, NOT sent)
30
+ ├─ [debug] Cache miss (buffered, NOT sent)
31
+ ├─ [ERROR] Payment failed ← Exception!
32
+ └─> FLUSH все buffered debug события!
33
+ ```
34
+
35
+ **Результат**: Debug логи только когда нужны, zero overhead на happy path.
36
+
37
+ ### 2. :success Pseudo-Severity
38
+
39
+ Новый severity level между :info и :warn для успешных операций.
40
+
41
+ ```ruby
42
+ Events::OrderPaid.track(
43
+ order_id: '123',
44
+ severity: :success # ← легко фильтровать успехи
45
+ )
46
+
47
+ # В Grafana/Kibana:
48
+ severity:success AND event_name:order.paid
49
+ ```
50
+
51
+ **Результат**: Видимость успехов, не только ошибок. Легко построить success rate.
52
+
53
+ ### 3. Pattern-Based Metrics
54
+
55
+ Вместо явных `metric :counter` на каждое событие - паттерны:
56
+
57
+ ```ruby
58
+ E11y.configure do |config|
59
+ config.metrics do
60
+ # Автоматически для всех событий
61
+ counter_for pattern: '*', name: 'events_total'
62
+
63
+ # Histogram для всех оплат
64
+ histogram_for pattern: '*.paid',
65
+ value: ->(e) { e.payload[:amount] }
66
+ end
67
+ end
68
+ ```
69
+
70
+ **Результат**: Метрики без boilerplate.
71
+
72
+ ### 4. Trace Context (OpenTelemetry + Sentry)
73
+
74
+ Автоматическое извлечение trace_id с fallback chain:
75
+
76
+ ```ruby
77
+ # Priority:
78
+ 1. X-Trace-ID header
79
+ 2. X-Request-ID header
80
+ 3. OpenTelemetry span context
81
+ 4. Sentry trace ID
82
+ 5. Generate new UUID v7
83
+ ```
84
+
85
+ **Результат**: Связанные события across services.
86
+
87
+ ### 5. Built-in SLO Tracking (Zero Config!)
88
+
89
+ Включил флаг → получил SLO metrics из коробки:
90
+
91
+ ```ruby
92
+ E11y.configure do |config|
93
+ config.slo_tracking = true # ← ВСЁ!
94
+ end
95
+
96
+ # Автоматически:
97
+ # ✅ HTTP availability + latency
98
+ # ✅ Sidekiq jobs success rate + duration
99
+ # ✅ ActiveJob success rate + duration
100
+ # ✅ Error budget + burn rate
101
+ # ✅ Grafana dashboards (generate)
102
+ # ✅ Prometheus alerts (generate)
103
+ ```
104
+
105
+ **Результат**: Production-ready SLO monitoring без написания middleware.
106
+
107
+ ---
108
+
109
+ ## ⚡ Quick Examples
110
+
111
+ ### Basic Event (v1.1 - Event-Level Configuration!)
112
+
113
+ > **🎯 NEW in v1.1:** Event-level configuration reduces global config from 1400+ to <300 lines!
114
+
115
+ ```ruby
116
+ # 1. Define event (простой Ruby класс + конфигурация)
117
+ class Events::OrderPaid < E11y::Event::Base
118
+ # Schema (dry-schema для валидации)
119
+ schema do
120
+ required(:order_id).filled(:string)
121
+ required(:amount).filled(:decimal)
122
+ required(:currency).filled(:string)
123
+ end
124
+
125
+ # ✨ NEW: Event-level configuration (right next to schema!)
126
+ severity :success
127
+ rate_limit 1000, window: 1.second
128
+ sample_rate 1.0 # Never sample payments
129
+ retention 7.years # Financial records
130
+ adapters [:loki, :sentry, :s3_archive]
131
+
132
+ # Metric definition
133
+ metric :counter,
134
+ name: 'orders.paid.total',
135
+ tags: [:currency]
136
+ end
137
+
138
+ # 2. Track - ТОЛЬКО ТАК, больше никаких вариантов!
139
+ Events::OrderPaid.track(
140
+ order_id: '123',
141
+ amount: 99.99,
142
+ currency: 'USD'
143
+ )
144
+
145
+ # ❌ НЕТ других способов:
146
+ # E11y.track_event(...) - НЕТ!
147
+ # Severity.track(...) - НЕТ!
148
+ ```
149
+
150
+ **90% событий нужен ТОЛЬКО schema (zero config!):**
151
+
152
+ ```ruby
153
+ # Conventions = sensible defaults!
154
+ class Events::OrderCreated < E11y::Event::Base
155
+ schema do
156
+ required(:order_id).filled(:string)
157
+ required(:amount).filled(:decimal)
158
+ end
159
+ # ← That's it! All config from conventions:
160
+ # severity: :success (from name)
161
+ # adapters: [:loki] (from severity)
162
+ # sample_rate: 0.1 (from severity)
163
+ # retention: 30.days (from severity)
164
+ # rate_limit: 1000 (default)
165
+ end
166
+ ```
167
+
168
+ **Inheritance для DRY:**
169
+
170
+ ```ruby
171
+ # Base class для payment events
172
+ module Events
173
+ class BasePaymentEvent < E11y::Event::Base
174
+ severity :success
175
+ sample_rate 1.0 # Never sample
176
+ retention 7.years
177
+ adapters [:loki, :sentry, :s3_archive]
178
+ end
179
+ end
180
+
181
+ # Inherit from base (1-2 lines per event!)
182
+ class Events::PaymentSucceeded < Events::BasePaymentEvent
183
+ schema do
184
+ required(:transaction_id).filled(:string)
185
+ required(:amount).filled(:decimal)
186
+ end
187
+ # ← Inherits ALL config from BasePaymentEvent!
188
+ end
189
+ ```
190
+
191
+ **Preset modules для 1-line includes:**
192
+
193
+ ```ruby
194
+ class Events::PaymentProcessed < E11y::Event::Base
195
+ include E11y::Presets::HighValueEvent # ← All config inherited!
196
+
197
+ schema do
198
+ required(:transaction_id).filled(:string)
199
+ required(:amount).filled(:decimal)
200
+ end
201
+ end
202
+ ```
203
+
204
+ ### With Duration Measurement
205
+
206
+ ```ruby
207
+ Events::OrderProcessing.track(order_id: '123') do
208
+ # Block execution time measured automatically
209
+ process_order(order)
210
+ end
211
+ # → event.duration_ms = 250
212
+ ```
213
+
214
+ ### Request-Scoped Debug Buffering
215
+
216
+ ```ruby
217
+ # Middleware (auto-configured)
218
+ class OrdersController < ApplicationController
219
+ def create
220
+ # Debug events buffered (not sent)
221
+ Events::ValidationStarted.track(severity: :debug)
222
+ Events::DatabaseQuery.track(sql: '...', severity: :debug)
223
+
224
+ order = Order.create!(params)
225
+
226
+ # Success event sent immediately
227
+ Events::OrderCreated.track(order_id: order.id, severity: :success)
228
+
229
+ render json: order
230
+ rescue => e
231
+ # Exception → all buffered debug events flushed with severity :error
232
+ raise
233
+ end
234
+ end
235
+ ```
236
+
237
+ ---
238
+
239
+ ## 🔧 Configuration (v1.1 - Simplified!)
240
+
241
+ > **🎯 NEW in v1.1:** Global config reduced from 1400+ to <300 lines!
242
+ >
243
+ > **Philosophy:** Global config = infrastructure only. Event config = in event classes.
244
+
245
+ ### Global Config (Infrastructure Only)
246
+
247
+ ```ruby
248
+ # config/initializers/e11y.rb (<300 lines!)
249
+ E11y.configure do |config|
250
+ # === ADAPTERS REGISTRATION (infrastructure) ===
251
+ config.register_adapter :loki, E11y::Adapters::LokiAdapter.new(
252
+ url: ENV['LOKI_URL'],
253
+ labels: { env: Rails.env, service: 'api' }
254
+ )
255
+
256
+ config.register_adapter :sentry, E11y::Adapters::SentryAdapter.new(
257
+ dsn: ENV['SENTRY_DSN']
258
+ )
259
+
260
+ config.register_adapter :s3, E11y::Adapters::S3Adapter.new(
261
+ bucket: 'events-archive'
262
+ )
263
+
264
+ # === DEFAULTS (conventions) ===
265
+ config.default_adapters = [:loki] # Most events → Loki
266
+ config.default_sample_rate = 0.1 # 10% sampling
267
+ config.default_rate_limit = 1000 # 1000 events/sec
268
+
269
+ # === REQUEST SCOPE BUFFERING ===
270
+ config.request_scope do
271
+ enabled true
272
+ buffer_limit 100 # max debug events per request
273
+ flush_on :error # :error, :always, :never
274
+ end
275
+
276
+ # === GLOBAL RATE LIMITING (infrastructure) ===
277
+ config.rate_limiting do
278
+ global limit: 10_000, window: 1.minute
279
+ end
280
+
281
+ # === CARDINALITY PROTECTION (infrastructure) ===
282
+ config.cardinality_protection do
283
+ forbidden_labels :user_id, :order_id, :session_id
284
+ default_cardinality_limit 100
285
+ end
286
+
287
+ # === SLO TRACKING (zero config!) ===
288
+ config.slo_tracking = true # ← ВСЁ!
289
+
290
+ # === AUDIT RETENTION (global default) ===
291
+ # Default for audit events, can be overridden per event
292
+ config.audit_retention = case ENV['JURISDICTION']
293
+ when 'EU' then 7.years # GDPR
294
+ when 'US' then 10.years # SOX
295
+ else 5.years
296
+ end
297
+
298
+ # ✅ That's it! No per-event config here anymore!
299
+ end
300
+ ```
301
+
302
+ ### Event-Level Config (Locality of Behavior)
303
+
304
+ ```ruby
305
+ # app/events/order_created.rb
306
+ module Events
307
+ class OrderCreated < E11y::Event::Base
308
+ schema do
309
+ required(:order_id).filled(:string)
310
+ required(:amount).filled(:decimal)
311
+ end
312
+
313
+ # ✨ Event-level config (right next to schema!)
314
+ severity :success
315
+ rate_limit 1000, window: 1.second
316
+ sample_rate 0.1 # 10% sampling
317
+ retention 30.days
318
+ adapters [:loki, :elasticsearch]
319
+
320
+ # Metric definition
321
+ metric :counter,
322
+ name: 'orders.created.total',
323
+ tags: [:currency]
324
+ end
325
+ end
326
+ ```
327
+
328
+ ### Old vs New Config
329
+
330
+ **Before (v1.0): 1400+ lines in global config**
331
+
332
+ ```ruby
333
+ # config/initializers/e11y.rb (1400+ lines!)
334
+ E11y.configure do |config|
335
+ # Adapters registration
336
+ config.register_adapter :loki, Loki.new(...)
337
+ config.register_adapter :sentry, Sentry.new(...)
338
+
339
+ # ❌ Per-event config (100+ events!)
340
+ config.events do
341
+ event 'Events::OrderCreated' do
342
+ severity :success
343
+ adapters [:loki]
344
+ sample_rate 0.1
345
+ retention 30.days
346
+ rate_limit 1000
347
+ end
348
+
349
+ event 'Events::PaymentSucceeded' do
350
+ severity :success
351
+ adapters [:loki, :sentry, :s3]
352
+ sample_rate 1.0
353
+ retention 7.years
354
+ rate_limit 1000
355
+ end
356
+
357
+ # ... 98+ more events ...
358
+ end
359
+ end
360
+ ```
361
+
362
+ **After (v1.1): <300 lines in global config**
363
+
364
+ ```ruby
365
+ # config/initializers/e11y.rb (<300 lines!)
366
+ E11y.configure do |config|
367
+ # ONLY infrastructure
368
+ config.register_adapter :loki, Loki.new(...)
369
+ config.register_adapter :sentry, Sentry.new(...)
370
+
371
+ # Defaults (conventions)
372
+ config.default_adapters = [:loki]
373
+
374
+ # ✅ No per-event config here!
375
+ end
376
+
377
+ # app/events/order_created.rb
378
+ class Events::OrderCreated < E11y::Event::Base
379
+ schema do; required(:order_id).filled(:string); end
380
+ # ← Uses conventions (zero config!)
381
+ end
382
+
383
+ # app/events/payment_succeeded.rb
384
+ class Events::PaymentSucceeded < Events::BasePaymentEvent
385
+ schema do; required(:transaction_id).filled(:string); end
386
+ # ← Inherits config from BasePaymentEvent (DRY!)
387
+ end
388
+ ```
389
+
390
+ ### Migration Path (Backward Compatible)
391
+
392
+ ```ruby
393
+ # v1.1 supports BOTH styles (backward compatible!)
394
+
395
+ # Old style (still works):
396
+ E11y.configure do |config|
397
+ config.events do
398
+ event 'Events::OrderCreated' do
399
+ adapters [:loki]
400
+ end
401
+ end
402
+ end
403
+
404
+ # New style (preferred):
405
+ class Events::OrderCreated < E11y::Event::Base
406
+ adapters [:loki] # ← Event-level (overrides global)
407
+ end
408
+
409
+ # Migrate incrementally (both work together!)
410
+ ```
411
+
412
+ ---
413
+
414
+ ## 🔧 Old Configuration (v1.0 - Deprecated)
415
+
416
+ > **⚠️ Deprecated:** This section shows v1.0 configuration style for reference.
417
+ > Use event-level configuration (above) for new projects.
418
+
419
+ ```ruby
420
+ # config/initializers/e11y.rb (OLD STYLE)
421
+ E11y.configure do |config|
422
+ # === SEVERITY ===
423
+ config.severity = Rails.env.production? ? :info : :debug
424
+
425
+ # === ADAPTERS (old style) ===
426
+ config.adapters = [
427
+ # Loki for logs
428
+ E11y::Adapters::LokiAdapter.new(
429
+ url: ENV['LOKI_URL'],
430
+ labels: { env: Rails.env, service: 'api' }
431
+ ),
432
+
433
+ # Sentry for errors
434
+ E11y::Adapters::SentryAdapter.new(
435
+ severity_filter: [:error, :fatal]
436
+ ),
437
+
438
+ # Stdout for development
439
+ (E11y::Adapters::StdoutAdapter.new if Rails.env.development?)
440
+ ].compact
441
+
442
+ # === PATTERN-BASED METRICS ===
443
+ config.metrics do
444
+ # Counter for all events
445
+ counter_for pattern: '*',
446
+ name: 'business_events_total',
447
+ tags: [:event_name, :severity]
448
+
449
+ # Histogram for payments
450
+ histogram_for pattern: '*.paid',
451
+ name: 'payment_amount',
452
+ value: ->(e) { e.payload[:amount] },
453
+ buckets: [10, 50, 100, 500, 1000]
454
+
455
+ # Success rate auto-metric
456
+ success_rate_for pattern: 'order.*',
457
+ name: 'order_operations_success_rate'
458
+ end
459
+
460
+ # === PII FILTERING (Rails-compatible!) ===
461
+ config.pii_filter do
462
+ # AUTO: Uses Rails.application.config.filter_parameters (default: true)
463
+ use_rails_filter_parameters true
464
+
465
+ # SIMPLE: Add more filters (Rails-style)
466
+ filter_parameters :api_key, :auth_token, /secret/i
467
+
468
+ # WHITELIST: Allow specific IDs even if filtered by Rails
469
+ allow_parameters :user_id, :order_id, :transaction_id
470
+
471
+ # ADVANCED: Pattern-based (beyond Rails)
472
+ filter_pattern /\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/i,
473
+ replacement: '[EMAIL]'
474
+ filter_pattern /\b\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{4}\b/,
475
+ replacement: '[CARD]'
476
+ end
477
+
478
+ # === RATE LIMITING ===
479
+ config.rate_limiting do
480
+ global limit: 10_000, window: 1.minute
481
+ per_event 'user.login.failed', limit: 100, window: 1.minute
482
+ end
483
+
484
+ # === CONTEXT ENRICHMENT ===
485
+ config.context_enricher do |event|
486
+ {
487
+ trace_id: E11y::TraceId.extract,
488
+ user_id: Current.user&.id,
489
+ tenant_id: Current.tenant&.id
490
+ }
491
+ end
492
+ end
493
+
494
+ # Start async workers
495
+ E11y.start!
496
+
497
+ # Graceful shutdown
498
+ at_exit { E11y.stop!(timeout: 5) }
499
+ ```
500
+
501
+ ---
502
+
503
+ ## 📊 Severity Levels
504
+
505
+ ```ruby
506
+ E11y::SEVERITIES = {
507
+ debug: 0, # Detailed diagnostic (buffered in request scope)
508
+ info: 1, # Informational
509
+ success: 2, # ← NEW! Successful operations
510
+ warn: 3, # Warnings
511
+ error: 4, # Errors
512
+ fatal: 5 # Critical failures
513
+ }
514
+ ```
515
+
516
+ **When to use :success?**
517
+
518
+ ```ruby
519
+ # ✅ Use :success for completed operations
520
+ Events::OrderPaid.track(order_id: '123', severity: :success)
521
+ Events::JobCompleted.track(job_id: '456', severity: :success)
522
+ Events::EmailSent.track(user_id: '789', severity: :success)
523
+
524
+ # ✅ Use :info for informational events
525
+ Events::UserLoggedIn.track(user_id: '123', severity: :info)
526
+ Events::SessionStarted.track(session_id: '456', severity: :info)
527
+
528
+ # Why separate :success from :info?
529
+ # → Easy filtering: severity:success = only successful ops
530
+ # → Easy metrics: success_rate = count(:success) / count(:success OR :error)
531
+ ```
532
+
533
+ ---
534
+
535
+ ## 🎭 Middleware (Auto-configured)
536
+
537
+ ### Rails / Rack
538
+
539
+ ```ruby
540
+ # config/application.rb (auto-added by generator)
541
+ config.middleware.use E11y::Middleware::Rack,
542
+ buffer_limit: 100,
543
+ flush_on: :error
544
+ ```
545
+
546
+ ### Sidekiq
547
+
548
+ ```ruby
549
+ # config/initializers/sidekiq.rb (auto-added by generator)
550
+ Sidekiq.configure_server do |config|
551
+ config.server_middleware do |chain|
552
+ chain.add E11y::Middleware::Sidekiq
553
+ end
554
+ end
555
+ ```
556
+
557
+ ### ActiveJob
558
+
559
+ ```ruby
560
+ # Auto-included by E11y (no config needed)
561
+ # trace_id propagated automatically to background jobs
562
+ ```
563
+
564
+ ---
565
+
566
+ ## 🔍 Trace Context Flow
567
+
568
+ ```ruby
569
+ # Service A (API)
570
+ POST /orders
571
+ trace_id: abc-123 (from X-Trace-ID header)
572
+ ├─ Events::OrderValidation.track (trace_id: abc-123)
573
+ ├─ Events::OrderCreated.track (trace_id: abc-123)
574
+ └─ ProcessOrderJob.perform_later(order_id, trace_id: abc-123)
575
+
576
+ # Service B (Background Job)
577
+ ProcessOrderJob
578
+ trace_id: abc-123 (from job args)
579
+ ├─ Events::OrderProcessing.track (trace_id: abc-123)
580
+ └─ HTTP → Service C (headers: X-Trace-ID: abc-123)
581
+
582
+ # Service C (Payment API)
583
+ POST /payments
584
+ trace_id: abc-123 (from X-Trace-ID header)
585
+ └─ Events::PaymentProcessed.track (trace_id: abc-123)
586
+
587
+ # Result: All events linked by trace_id = full visibility
588
+ ```
589
+
590
+ ---
591
+
592
+ ## 📈 Yabeda Integration
593
+
594
+ Events автоматически становятся метриками через pattern-based rules:
595
+
596
+ ```ruby
597
+ # After tracking event:
598
+ Events::OrderPaid.track(amount: 99, currency: 'USD')
599
+
600
+ # Auto-generated metrics:
601
+ yabeda.business_events.events_total{event_name="order.paid",severity="success"} 1
602
+ yabeda.business_events.payment_amount_bucket{currency="USD",le="100"} 1
603
+ yabeda.business_events.order_operations_success 1
604
+ yabeda.business_events.order_operations_total 1
605
+ ```
606
+
607
+ **Prometheus endpoint:**
608
+
609
+ ```ruby
610
+ # config/routes.rb
611
+ mount Yabeda::Prometheus::Exporter => '/metrics'
612
+ ```
613
+
614
+ ---
615
+
616
+ ## 🧪 Testing
617
+
618
+ ```ruby
619
+ # spec/spec_helper.rb
620
+ RSpec.configure do |config|
621
+ config.before(:each) do
622
+ E11y.configure do |c|
623
+ c.adapters = [E11y::Adapters::NullAdapter.new]
624
+ end
625
+ end
626
+ end
627
+
628
+ # spec/controllers/orders_controller_spec.rb
629
+ RSpec.describe OrdersController do
630
+ it 'tracks order creation' do
631
+ expect(Events::OrderCreated).to receive(:track).with(
632
+ hash_including(order_id: anything)
633
+ )
634
+
635
+ post :create, params: { ... }
636
+ end
637
+ end
638
+ ```
639
+
640
+ ---
641
+
642
+ ## 🚀 Performance
643
+
644
+ | Metric | Target | Actual |
645
+ |--------|--------|--------|
646
+ | **Track latency (p99)** | <1ms | ✅ 0.8ms |
647
+ | **Throughput** | 10k events/sec | ✅ 15k/sec |
648
+ | **Memory** | <100MB @ 100k buffer | ✅ 80MB |
649
+ | **CPU overhead** | <5% @ 1k events/sec | ✅ 3% |
650
+
651
+ **Optimizations:**
652
+ - Early severity filtering (<1μs для filtered events)
653
+ - Lock-free ring buffer (SPSC)
654
+ - Async workers (no blocking)
655
+ - Lazy serialization (только перед отправкой)
656
+
657
+ ---
658
+
659
+ ## 🔐 Security
660
+
661
+ ### PII Filtering (Rails-Compatible! 🎉)
662
+
663
+ **ZERO CONFIG**: E11y автоматически использует `Rails.application.config.filter_parameters`!
664
+
665
+ ```ruby
666
+ # config/application.rb
667
+ config.filter_parameters += [:password, :email, :ssn]
668
+
669
+ # E11y автоматически фильтрует эти поля - NO ADDITIONAL CONFIG!
670
+ Events::UserRegistered.track(
671
+ email: 'user@example.com', # → '[FILTERED]'
672
+ password: 'secret123' # → '[FILTERED]'
673
+ )
674
+ ```
675
+
676
+ **EXTENDED**: Добавьте больше фильтров поверх Rails:
677
+
678
+ ```ruby
679
+ # config/initializers/e11y.rb
680
+ E11y.configure do |config|
681
+ config.pii_filter do
682
+ # Inherit Rails filters + add more
683
+ filter_parameters :api_key, :auth_token, /secret/i
684
+
685
+ # Whitelist known-safe fields
686
+ allow_parameters :user_id, :order_id
687
+
688
+ # Pattern-based (content filtering)
689
+ filter_pattern /\b\d{16}\b/, replacement: '[CARD]'
690
+ end
691
+ end
692
+ ```
693
+
694
+ **See full guide**: `e11y-rails-compatible-pii-filtering.md`
695
+
696
+ ### Rate Limiting (защита от DoS)
697
+
698
+ ```ruby
699
+ # Config
700
+ config.rate_limiting do
701
+ global limit: 10_000, window: 1.minute
702
+ per_event 'user.login.failed', limit: 100, window: 1.minute
703
+ end
704
+
705
+ # При превышении - события дропаются
706
+ ```
707
+
708
+ ---
709
+
710
+ ## 🎯 Built-in SLO Tracking (Zero Config!)
711
+
712
+ **Enable one flag → get SLO out of the box!**
713
+
714
+ ```ruby
715
+ # config/initializers/e11y.rb
716
+ E11y.configure do |config|
717
+ config.slo_tracking = true # ← ВСЁ! Этого достаточно!
718
+
719
+ # Опционально: кастомизация
720
+ config.slo do
721
+ # Global defaults
722
+ http_ignore_statuses [404, 401] # 404/401 не ошибки
723
+ latency_target_p95 200 # ms
724
+
725
+ # 🎯 NEW: Per-controller overrides (РЕКОМЕНДУЕТСЯ для Rails!)
726
+ controller 'Api::Admin::BaseController' do
727
+ ignore true # Весь admin не входит в SLO
728
+ end
729
+
730
+ controller 'Api::OrdersController', action: 'show' do
731
+ latency_target_p95 50 # Show должен быть быстрым
732
+ end
733
+
734
+ controller 'Api::OrdersController', action: 'create' do
735
+ latency_target_p95 200 # Create может быть медленнее
736
+ ignore_statuses [422] # 422 Validation = not SLO breach
737
+ end
738
+
739
+ # 🔧 LEGACY: Path-based (для non-Rails apps)
740
+ endpoint '/api/webhooks/*' do
741
+ ignore true
742
+ end
743
+
744
+ # 🎯 Per-job overrides
745
+ job 'ReportGenerationJob' do
746
+ ignore true # Долгие джобы не входят в SLO
747
+ end
748
+
749
+ job 'ProcessPaymentJob' do
750
+ latency_target_p95 1000 # Критичные джобы = строгий SLO
751
+ end
752
+ end
753
+ end
754
+ ```
755
+
756
+ **Автоматически получаем:**
757
+
758
+ ```ruby
759
+ # 🎯 HTTP Metrics (controller#action - автогруппировка!)
760
+ yabeda_slo_http_requests_total{
761
+ status="200",
762
+ method="GET",
763
+ controller="Api::OrdersController",
764
+ action="show"
765
+ } 1234
766
+
767
+ yabeda_slo_http_request_duration_seconds{
768
+ method="GET",
769
+ controller="Api::OrdersController",
770
+ action="show"
771
+ } histogram
772
+
773
+ # Sidekiq Metrics
774
+ yabeda_slo_sidekiq_jobs_total{queue="default", class="ProcessOrderJob", status="success"} 456
775
+ yabeda_slo_sidekiq_job_duration_seconds{queue="default", class="ProcessOrderJob"} histogram
776
+
777
+ # ActiveJob Metrics
778
+ yabeda_slo_active_jobs_total{queue="mailers", class="EmailJob", status="success"} 789
779
+ yabeda_slo_active_job_duration_seconds{queue="mailers", class="EmailJob"} histogram
780
+ ```
781
+
782
+ **Преимущество:** `/orders/123`, `/orders/456` → один `OrdersController#show` (не нужна нормализация path!)
783
+
784
+ **SLO Calculations (PromQL):**
785
+
786
+ ```promql
787
+ # HTTP Availability (30d rolling)
788
+ 100 * (sum(rate(yabeda_slo_http_successes_total[30d])) /
789
+ (sum(rate(yabeda_slo_http_successes_total[30d])) + sum(rate(yabeda_slo_http_errors_total[30d]))))
790
+ # Expected: >= 99.9%
791
+
792
+ # p95 Latency
793
+ histogram_quantile(0.95, rate(yabeda_slo_http_request_duration_seconds_bucket[5m]))
794
+ # Expected: < 200ms
795
+
796
+ # Error Budget Remaining
797
+ 100 * (1 - (sum(rate(yabeda_slo_http_errors_total[30d])) /
798
+ (sum(rate(yabeda_slo_http_successes_total[30d])) + sum(rate(yabeda_slo_http_errors_total[30d])))) / 0.001)
799
+ # 100% = весь бюджет остался, 0% = исчерпан
800
+ ```
801
+
802
+ **Auto-Generated:**
803
+
804
+ ```bash
805
+ # Grafana dashboard
806
+ rails g e11y:grafana_dashboard
807
+ # → config/grafana/e11y_slo_dashboard.json
808
+
809
+ # Prometheus alerts
810
+ rails g e11y:prometheus_alerts
811
+ # → config/prometheus/e11y_slo_alerts.yml
812
+ ```
813
+
814
+ **Что трекается автоматически:**
815
+ - ✅ **Rack middleware** - все HTTP requests (availability, latency)
816
+ - ✅ **Sidekiq middleware** - все jobs (success rate, duration)
817
+ - ✅ **ActiveJob instrumentation** - все jobs (success rate, duration)
818
+ - ✅ **Path normalization** - `/orders/123` → `/orders/:id`
819
+ - ✅ **Error categorization** - configurable (5xx = error, 404 = ignore)
820
+ - ✅ **Heartbeat** - auto-enabled (pod liveness detection)
821
+
822
+ **⚠️ ВАЖНО: Ограничения in-process SLO**
823
+
824
+ E11y SLO работает ВНУТРИ Ruby процесса и НЕ видит:
825
+ - ❌ Network issues (requests не доходят до app)
826
+ - ❌ Load balancer down
827
+ - ❌ All pods crashed (метрик просто нет)
828
+ - ❌ DNS issues
829
+
830
+ **Решение:** Multi-layer monitoring (см. полную документацию):
831
+ 1. **Layer 1**: E11y SLO (in-process)
832
+ 2. **Layer 2**: E11y Heartbeat (pod liveness) - **auto-enabled!**
833
+ 3. **Layer 3**: K8s health probes (`/health/live`, `/health/ready`) - **auto-created!**
834
+ 4. **Layer 4**: External synthetic monitoring (Prometheus Blackbox Exporter)
835
+
836
+ ```ruby
837
+ # Heartbeat автоматически включен с slo_tracking = true
838
+ # Метрики:
839
+ yabeda_e11y_heartbeat_timestamp{pod="pod-1"} 1703500000 # Последний heartbeat
840
+ yabeda_e11y_service_healthy{pod="pod-1"} 1 # 1 = healthy
841
+
842
+ # Alert если pod мертв:
843
+ # (time() - yabeda_e11y_heartbeat_timestamp) > 30s → Pod down!
844
+ ```
845
+
846
+ ---
847
+
848
+ ## 🎯 Migration from Rails.logger
849
+
850
+ ```ruby
851
+ # ❌ Before
852
+ Rails.logger.info "Order #{order.id} paid #{order.amount} #{order.currency}"
853
+ OrderMetrics.increment('orders.paid.total')
854
+ OrderMetrics.observe('orders.paid.amount', order.amount)
855
+
856
+ # ✅ After (1 строка вместо 3!)
857
+ Events::OrderPaid.track(
858
+ order_id: order.id,
859
+ amount: order.amount,
860
+ currency: order.currency
861
+ )
862
+ # → Structured log + auto-metrics + trace context
863
+ ```
864
+
865
+ ---
866
+
867
+ ## 🐛 Troubleshooting
868
+
869
+ ### Events not appearing?
870
+
871
+ ```ruby
872
+ # Check 1: Severity
873
+ E11y.enabled_for?(:debug) # Should be true
874
+
875
+ # Check 2: Adapters
876
+ E11y.config.adapters # Should not be empty
877
+
878
+ # Check 3: Buffer
879
+ E11y.buffer_size # Should be < capacity
880
+
881
+ # Check 4: Circuit breaker
882
+ E11y.circuit_breaker_state # Should be :closed
883
+ ```
884
+
885
+ ### High latency?
886
+
887
+ ```ruby
888
+ # Check metrics
889
+ Yabeda.e11y.track_duration_seconds # p99 should be <1ms
890
+
891
+ # Possible causes:
892
+ # - PII filtering regex too complex
893
+ # - Rate limiter Redis slow
894
+ # - Adapter network slow
895
+ ```
896
+
897
+ ---
898
+
899
+ ## 📚 Full Documentation
900
+
901
+ - **Complete spec**: `severity/e11y-final-spec.md` (2000+ lines)
902
+ - **Old spec**: `severity/tz-improved.md` (reference)
903
+ - **GitHub**: https://github.com/yourorg/e11y
904
+
905
+ ---
906
+
907
+ ## ✅ Checklist
908
+
909
+ - [ ] Install gem
910
+ - [ ] Run generator: `rails g e11y:install`
911
+ - [ ] Configure adapters (Loki/Sentry/ELK)
912
+ - [ ] **Enable SLO tracking**: `config.slo_tracking = true`
913
+ - [ ] Define first event class
914
+ - [ ] Track first event
915
+ - [ ] Check `/metrics` endpoint (Prometheus)
916
+ - [ ] Verify events in Grafana/Kibana
917
+ - [ ] **Verify SLO metrics**: `yabeda_slo_*` in Prometheus
918
+ - [ ] Test request-scoped buffering (raise exception → see debug logs)
919
+ - [ ] Configure PII filtering
920
+ - [ ] Setup rate limiting
921
+ - [ ] Configure pattern-based metrics
922
+ - [ ] **Generate Grafana dashboard**: `rails g e11y:grafana_dashboard`
923
+ - [ ] **Generate Prometheus alerts**: `rails g e11y:prometheus_alerts`
924
+ - [ ] Deploy to staging
925
+ - [ ] Monitor performance
926
+ - [ ] Rollout to production (canary 1% → 10% → 100%)
927
+
928
+ ---
929
+
930
+ **Questions?** See `e11y-final-spec.md` FAQ section or open GitHub issue.
931
+
932
+ **Version**: 1.0.0
933
+ **Last Updated**: 2025-12-24
934
+