e11y 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.rspec +4 -0
- data/.rubocop.yml +69 -0
- data/CHANGELOG.md +26 -0
- data/CODE_OF_CONDUCT.md +64 -0
- data/LICENSE.txt +21 -0
- data/README.md +179 -0
- data/Rakefile +37 -0
- data/benchmarks/run_all.rb +33 -0
- data/config/README.md +83 -0
- data/config/loki-local-config.yaml +35 -0
- data/config/prometheus.yml +15 -0
- data/docker-compose.yml +78 -0
- data/docs/00-ICP-AND-TIMELINE.md +483 -0
- data/docs/01-SCALE-REQUIREMENTS.md +858 -0
- data/docs/ADR-001-architecture.md +2617 -0
- data/docs/ADR-002-metrics-yabeda.md +1395 -0
- data/docs/ADR-003-slo-observability.md +3337 -0
- data/docs/ADR-004-adapter-architecture.md +2385 -0
- data/docs/ADR-005-tracing-context.md +1372 -0
- data/docs/ADR-006-security-compliance.md +4143 -0
- data/docs/ADR-007-opentelemetry-integration.md +1385 -0
- data/docs/ADR-008-rails-integration.md +1911 -0
- data/docs/ADR-009-cost-optimization.md +2993 -0
- data/docs/ADR-010-developer-experience.md +2166 -0
- data/docs/ADR-011-testing-strategy.md +1836 -0
- data/docs/ADR-012-event-evolution.md +958 -0
- data/docs/ADR-013-reliability-error-handling.md +2750 -0
- data/docs/ADR-014-event-driven-slo.md +1533 -0
- data/docs/ADR-015-middleware-order.md +1061 -0
- data/docs/ADR-016-self-monitoring-slo.md +1234 -0
- data/docs/API-REFERENCE-L28.md +914 -0
- data/docs/COMPREHENSIVE-CONFIGURATION.md +2366 -0
- data/docs/IMPLEMENTATION_NOTES.md +2804 -0
- data/docs/IMPLEMENTATION_PLAN.md +1971 -0
- data/docs/IMPLEMENTATION_PLAN_ARCHITECTURE.md +586 -0
- data/docs/PLAN.md +148 -0
- data/docs/QUICK-START.md +934 -0
- data/docs/README.md +296 -0
- data/docs/design/00-memory-optimization.md +593 -0
- data/docs/guides/MIGRATION-L27-L28.md +692 -0
- data/docs/guides/PERFORMANCE-BENCHMARKS.md +434 -0
- data/docs/guides/README.md +44 -0
- data/docs/prd/01-overview-vision.md +440 -0
- data/docs/use_cases/README.md +119 -0
- data/docs/use_cases/UC-001-request-scoped-debug-buffering.md +813 -0
- data/docs/use_cases/UC-002-business-event-tracking.md +1953 -0
- data/docs/use_cases/UC-003-pattern-based-metrics.md +1627 -0
- data/docs/use_cases/UC-004-zero-config-slo-tracking.md +728 -0
- data/docs/use_cases/UC-005-sentry-integration.md +759 -0
- data/docs/use_cases/UC-006-trace-context-management.md +905 -0
- data/docs/use_cases/UC-007-pii-filtering.md +2648 -0
- data/docs/use_cases/UC-008-opentelemetry-integration.md +1153 -0
- data/docs/use_cases/UC-009-multi-service-tracing.md +1043 -0
- data/docs/use_cases/UC-010-background-job-tracking.md +1018 -0
- data/docs/use_cases/UC-011-rate-limiting.md +1906 -0
- data/docs/use_cases/UC-012-audit-trail.md +2301 -0
- data/docs/use_cases/UC-013-high-cardinality-protection.md +2127 -0
- data/docs/use_cases/UC-014-adaptive-sampling.md +1940 -0
- data/docs/use_cases/UC-015-cost-optimization.md +735 -0
- data/docs/use_cases/UC-016-rails-logger-migration.md +785 -0
- data/docs/use_cases/UC-017-local-development.md +867 -0
- data/docs/use_cases/UC-018-testing-events.md +1081 -0
- data/docs/use_cases/UC-019-tiered-storage-migration.md +562 -0
- data/docs/use_cases/UC-020-event-versioning.md +708 -0
- data/docs/use_cases/UC-021-error-handling-retry-dlq.md +956 -0
- data/docs/use_cases/UC-022-event-registry.md +648 -0
- data/docs/use_cases/backlog.md +226 -0
- data/e11y.gemspec +76 -0
- data/lib/e11y/adapters/adaptive_batcher.rb +207 -0
- data/lib/e11y/adapters/audit_encrypted.rb +239 -0
- data/lib/e11y/adapters/base.rb +580 -0
- data/lib/e11y/adapters/file.rb +224 -0
- data/lib/e11y/adapters/in_memory.rb +216 -0
- data/lib/e11y/adapters/loki.rb +333 -0
- data/lib/e11y/adapters/otel_logs.rb +203 -0
- data/lib/e11y/adapters/registry.rb +141 -0
- data/lib/e11y/adapters/sentry.rb +230 -0
- data/lib/e11y/adapters/stdout.rb +108 -0
- data/lib/e11y/adapters/yabeda.rb +370 -0
- data/lib/e11y/buffers/adaptive_buffer.rb +339 -0
- data/lib/e11y/buffers/base_buffer.rb +40 -0
- data/lib/e11y/buffers/request_scoped_buffer.rb +246 -0
- data/lib/e11y/buffers/ring_buffer.rb +267 -0
- data/lib/e11y/buffers.rb +14 -0
- data/lib/e11y/console.rb +122 -0
- data/lib/e11y/current.rb +48 -0
- data/lib/e11y/event/base.rb +894 -0
- data/lib/e11y/event/value_sampling_config.rb +84 -0
- data/lib/e11y/events/base_audit_event.rb +43 -0
- data/lib/e11y/events/base_payment_event.rb +33 -0
- data/lib/e11y/events/rails/cache/delete.rb +21 -0
- data/lib/e11y/events/rails/cache/read.rb +23 -0
- data/lib/e11y/events/rails/cache/write.rb +22 -0
- data/lib/e11y/events/rails/database/query.rb +45 -0
- data/lib/e11y/events/rails/http/redirect.rb +21 -0
- data/lib/e11y/events/rails/http/request.rb +26 -0
- data/lib/e11y/events/rails/http/send_file.rb +21 -0
- data/lib/e11y/events/rails/http/start_processing.rb +26 -0
- data/lib/e11y/events/rails/job/completed.rb +22 -0
- data/lib/e11y/events/rails/job/enqueued.rb +22 -0
- data/lib/e11y/events/rails/job/failed.rb +22 -0
- data/lib/e11y/events/rails/job/scheduled.rb +23 -0
- data/lib/e11y/events/rails/job/started.rb +22 -0
- data/lib/e11y/events/rails/log.rb +56 -0
- data/lib/e11y/events/rails/view/render.rb +23 -0
- data/lib/e11y/events.rb +18 -0
- data/lib/e11y/instruments/active_job.rb +201 -0
- data/lib/e11y/instruments/rails_instrumentation.rb +141 -0
- data/lib/e11y/instruments/sidekiq.rb +175 -0
- data/lib/e11y/logger/bridge.rb +205 -0
- data/lib/e11y/metrics/cardinality_protection.rb +172 -0
- data/lib/e11y/metrics/cardinality_tracker.rb +134 -0
- data/lib/e11y/metrics/registry.rb +234 -0
- data/lib/e11y/metrics/relabeling.rb +226 -0
- data/lib/e11y/metrics.rb +102 -0
- data/lib/e11y/middleware/audit_signing.rb +174 -0
- data/lib/e11y/middleware/base.rb +140 -0
- data/lib/e11y/middleware/event_slo.rb +167 -0
- data/lib/e11y/middleware/pii_filter.rb +266 -0
- data/lib/e11y/middleware/pii_filtering.rb +280 -0
- data/lib/e11y/middleware/rate_limiting.rb +214 -0
- data/lib/e11y/middleware/request.rb +163 -0
- data/lib/e11y/middleware/routing.rb +157 -0
- data/lib/e11y/middleware/sampling.rb +254 -0
- data/lib/e11y/middleware/slo.rb +168 -0
- data/lib/e11y/middleware/trace_context.rb +131 -0
- data/lib/e11y/middleware/validation.rb +118 -0
- data/lib/e11y/middleware/versioning.rb +132 -0
- data/lib/e11y/middleware.rb +12 -0
- data/lib/e11y/pii/patterns.rb +90 -0
- data/lib/e11y/pii.rb +13 -0
- data/lib/e11y/pipeline/builder.rb +155 -0
- data/lib/e11y/pipeline/zone_validator.rb +110 -0
- data/lib/e11y/pipeline.rb +12 -0
- data/lib/e11y/presets/audit_event.rb +65 -0
- data/lib/e11y/presets/debug_event.rb +34 -0
- data/lib/e11y/presets/high_value_event.rb +51 -0
- data/lib/e11y/presets.rb +19 -0
- data/lib/e11y/railtie.rb +138 -0
- data/lib/e11y/reliability/circuit_breaker.rb +216 -0
- data/lib/e11y/reliability/dlq/file_storage.rb +277 -0
- data/lib/e11y/reliability/dlq/filter.rb +117 -0
- data/lib/e11y/reliability/retry_handler.rb +207 -0
- data/lib/e11y/reliability/retry_rate_limiter.rb +117 -0
- data/lib/e11y/sampling/error_spike_detector.rb +225 -0
- data/lib/e11y/sampling/load_monitor.rb +161 -0
- data/lib/e11y/sampling/stratified_tracker.rb +92 -0
- data/lib/e11y/sampling/value_extractor.rb +82 -0
- data/lib/e11y/self_monitoring/buffer_monitor.rb +79 -0
- data/lib/e11y/self_monitoring/performance_monitor.rb +97 -0
- data/lib/e11y/self_monitoring/reliability_monitor.rb +146 -0
- data/lib/e11y/slo/event_driven.rb +150 -0
- data/lib/e11y/slo/tracker.rb +119 -0
- data/lib/e11y/version.rb +9 -0
- data/lib/e11y.rb +283 -0
- metadata +452 -0
data/docs/QUICK-START.md
ADDED
|
@@ -0,0 +1,934 @@
|
|
|
1
|
+
# E11y - Quick Start Guide
|
|
2
|
+
|
|
3
|
+
> **TL;DR**: Ruby gem для структурированных бизнес-событий с request-scoped debug buffering, pattern-based метриками и pluggable адаптерами.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 🚀 Installation (5 minutes)
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
# Gemfile
|
|
11
|
+
gem 'e11y', '~> 1.0'
|
|
12
|
+
|
|
13
|
+
bundle install
|
|
14
|
+
rails g e11y:install
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## 🎯 Killer Features
|
|
20
|
+
|
|
21
|
+
### 1. Request-Scoped Debug Buffering
|
|
22
|
+
|
|
23
|
+
**Проблема**: Debug логи в production = шум. Без debug = нет контекста при ошибках.
|
|
24
|
+
|
|
25
|
+
**Решение**: Debug события буферизируются в thread-local storage. При success - дропаются, при error - флашатся.
|
|
26
|
+
|
|
27
|
+
```ruby
|
|
28
|
+
GET /api/orders/123
|
|
29
|
+
├─ [debug] Query: SELECT... (buffered, NOT sent)
|
|
30
|
+
├─ [debug] Cache miss (buffered, NOT sent)
|
|
31
|
+
├─ [ERROR] Payment failed ← Exception!
|
|
32
|
+
└─> FLUSH все buffered debug события!
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
**Результат**: Debug логи только когда нужны, zero overhead на happy path.
|
|
36
|
+
|
|
37
|
+
### 2. :success Pseudo-Severity
|
|
38
|
+
|
|
39
|
+
Новый severity level между :info и :warn для успешных операций.
|
|
40
|
+
|
|
41
|
+
```ruby
|
|
42
|
+
Events::OrderPaid.track(
|
|
43
|
+
order_id: '123',
|
|
44
|
+
severity: :success # ← легко фильтровать успехи
|
|
45
|
+
)
|
|
46
|
+
|
|
47
|
+
# В Grafana/Kibana:
|
|
48
|
+
severity:success AND event_name:order.paid
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
**Результат**: Видимость успехов, не только ошибок. Легко построить success rate.
|
|
52
|
+
|
|
53
|
+
### 3. Pattern-Based Metrics
|
|
54
|
+
|
|
55
|
+
Вместо явных `metric :counter` на каждое событие - паттерны:
|
|
56
|
+
|
|
57
|
+
```ruby
|
|
58
|
+
E11y.configure do |config|
|
|
59
|
+
config.metrics do
|
|
60
|
+
# Автоматически для всех событий
|
|
61
|
+
counter_for pattern: '*', name: 'events_total'
|
|
62
|
+
|
|
63
|
+
# Histogram для всех оплат
|
|
64
|
+
histogram_for pattern: '*.paid',
|
|
65
|
+
value: ->(e) { e.payload[:amount] }
|
|
66
|
+
end
|
|
67
|
+
end
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
**Результат**: Метрики без boilerplate.
|
|
71
|
+
|
|
72
|
+
### 4. Trace Context (OpenTelemetry + Sentry)
|
|
73
|
+
|
|
74
|
+
Автоматическое извлечение trace_id с fallback chain:
|
|
75
|
+
|
|
76
|
+
```ruby
|
|
77
|
+
# Priority:
|
|
78
|
+
1. X-Trace-ID header
|
|
79
|
+
2. X-Request-ID header
|
|
80
|
+
3. OpenTelemetry span context
|
|
81
|
+
4. Sentry trace ID
|
|
82
|
+
5. Generate new UUID v7
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
**Результат**: Связанные события across services.
|
|
86
|
+
|
|
87
|
+
### 5. Built-in SLO Tracking (Zero Config!)
|
|
88
|
+
|
|
89
|
+
Включил флаг → получил SLO metrics из коробки:
|
|
90
|
+
|
|
91
|
+
```ruby
|
|
92
|
+
E11y.configure do |config|
|
|
93
|
+
config.slo_tracking = true # ← ВСЁ!
|
|
94
|
+
end
|
|
95
|
+
|
|
96
|
+
# Автоматически:
|
|
97
|
+
# ✅ HTTP availability + latency
|
|
98
|
+
# ✅ Sidekiq jobs success rate + duration
|
|
99
|
+
# ✅ ActiveJob success rate + duration
|
|
100
|
+
# ✅ Error budget + burn rate
|
|
101
|
+
# ✅ Grafana dashboards (generate)
|
|
102
|
+
# ✅ Prometheus alerts (generate)
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
**Результат**: Production-ready SLO monitoring без написания middleware.
|
|
106
|
+
|
|
107
|
+
---
|
|
108
|
+
|
|
109
|
+
## ⚡ Quick Examples
|
|
110
|
+
|
|
111
|
+
### Basic Event (v1.1 - Event-Level Configuration!)
|
|
112
|
+
|
|
113
|
+
> **🎯 NEW in v1.1:** Event-level configuration reduces global config from 1400+ to <300 lines!
|
|
114
|
+
|
|
115
|
+
```ruby
|
|
116
|
+
# 1. Define event (простой Ruby класс + конфигурация)
|
|
117
|
+
class Events::OrderPaid < E11y::Event::Base
|
|
118
|
+
# Schema (dry-schema для валидации)
|
|
119
|
+
schema do
|
|
120
|
+
required(:order_id).filled(:string)
|
|
121
|
+
required(:amount).filled(:decimal)
|
|
122
|
+
required(:currency).filled(:string)
|
|
123
|
+
end
|
|
124
|
+
|
|
125
|
+
# ✨ NEW: Event-level configuration (right next to schema!)
|
|
126
|
+
severity :success
|
|
127
|
+
rate_limit 1000, window: 1.second
|
|
128
|
+
sample_rate 1.0 # Never sample payments
|
|
129
|
+
retention 7.years # Financial records
|
|
130
|
+
adapters [:loki, :sentry, :s3_archive]
|
|
131
|
+
|
|
132
|
+
# Metric definition
|
|
133
|
+
metric :counter,
|
|
134
|
+
name: 'orders.paid.total',
|
|
135
|
+
tags: [:currency]
|
|
136
|
+
end
|
|
137
|
+
|
|
138
|
+
# 2. Track - ТОЛЬКО ТАК, больше никаких вариантов!
|
|
139
|
+
Events::OrderPaid.track(
|
|
140
|
+
order_id: '123',
|
|
141
|
+
amount: 99.99,
|
|
142
|
+
currency: 'USD'
|
|
143
|
+
)
|
|
144
|
+
|
|
145
|
+
# ❌ НЕТ других способов:
|
|
146
|
+
# E11y.track_event(...) - НЕТ!
|
|
147
|
+
# Severity.track(...) - НЕТ!
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
**90% событий нужен ТОЛЬКО schema (zero config!):**
|
|
151
|
+
|
|
152
|
+
```ruby
|
|
153
|
+
# Conventions = sensible defaults!
|
|
154
|
+
class Events::OrderCreated < E11y::Event::Base
|
|
155
|
+
schema do
|
|
156
|
+
required(:order_id).filled(:string)
|
|
157
|
+
required(:amount).filled(:decimal)
|
|
158
|
+
end
|
|
159
|
+
# ← That's it! All config from conventions:
|
|
160
|
+
# severity: :success (from name)
|
|
161
|
+
# adapters: [:loki] (from severity)
|
|
162
|
+
# sample_rate: 0.1 (from severity)
|
|
163
|
+
# retention: 30.days (from severity)
|
|
164
|
+
# rate_limit: 1000 (default)
|
|
165
|
+
end
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
**Inheritance для DRY:**
|
|
169
|
+
|
|
170
|
+
```ruby
|
|
171
|
+
# Base class для payment events
|
|
172
|
+
module Events
|
|
173
|
+
class BasePaymentEvent < E11y::Event::Base
|
|
174
|
+
severity :success
|
|
175
|
+
sample_rate 1.0 # Never sample
|
|
176
|
+
retention 7.years
|
|
177
|
+
adapters [:loki, :sentry, :s3_archive]
|
|
178
|
+
end
|
|
179
|
+
end
|
|
180
|
+
|
|
181
|
+
# Inherit from base (1-2 lines per event!)
|
|
182
|
+
class Events::PaymentSucceeded < Events::BasePaymentEvent
|
|
183
|
+
schema do
|
|
184
|
+
required(:transaction_id).filled(:string)
|
|
185
|
+
required(:amount).filled(:decimal)
|
|
186
|
+
end
|
|
187
|
+
# ← Inherits ALL config from BasePaymentEvent!
|
|
188
|
+
end
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
**Preset modules для 1-line includes:**
|
|
192
|
+
|
|
193
|
+
```ruby
|
|
194
|
+
class Events::PaymentProcessed < E11y::Event::Base
|
|
195
|
+
include E11y::Presets::HighValueEvent # ← All config inherited!
|
|
196
|
+
|
|
197
|
+
schema do
|
|
198
|
+
required(:transaction_id).filled(:string)
|
|
199
|
+
required(:amount).filled(:decimal)
|
|
200
|
+
end
|
|
201
|
+
end
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
### With Duration Measurement
|
|
205
|
+
|
|
206
|
+
```ruby
|
|
207
|
+
Events::OrderProcessing.track(order_id: '123') do
|
|
208
|
+
# Block execution time measured automatically
|
|
209
|
+
process_order(order)
|
|
210
|
+
end
|
|
211
|
+
# → event.duration_ms = 250
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
### Request-Scoped Debug Buffering
|
|
215
|
+
|
|
216
|
+
```ruby
|
|
217
|
+
# Middleware (auto-configured)
|
|
218
|
+
class OrdersController < ApplicationController
|
|
219
|
+
def create
|
|
220
|
+
# Debug events buffered (not sent)
|
|
221
|
+
Events::ValidationStarted.track(severity: :debug)
|
|
222
|
+
Events::DatabaseQuery.track(sql: '...', severity: :debug)
|
|
223
|
+
|
|
224
|
+
order = Order.create!(params)
|
|
225
|
+
|
|
226
|
+
# Success event sent immediately
|
|
227
|
+
Events::OrderCreated.track(order_id: order.id, severity: :success)
|
|
228
|
+
|
|
229
|
+
render json: order
|
|
230
|
+
rescue => e
|
|
231
|
+
# Exception → all buffered debug events flushed with severity :error
|
|
232
|
+
raise
|
|
233
|
+
end
|
|
234
|
+
end
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
---
|
|
238
|
+
|
|
239
|
+
## 🔧 Configuration (v1.1 - Simplified!)
|
|
240
|
+
|
|
241
|
+
> **🎯 NEW in v1.1:** Global config reduced from 1400+ to <300 lines!
|
|
242
|
+
>
|
|
243
|
+
> **Philosophy:** Global config = infrastructure only. Event config = in event classes.
|
|
244
|
+
|
|
245
|
+
### Global Config (Infrastructure Only)
|
|
246
|
+
|
|
247
|
+
```ruby
|
|
248
|
+
# config/initializers/e11y.rb (<300 lines!)
|
|
249
|
+
E11y.configure do |config|
|
|
250
|
+
# === ADAPTERS REGISTRATION (infrastructure) ===
|
|
251
|
+
config.register_adapter :loki, E11y::Adapters::LokiAdapter.new(
|
|
252
|
+
url: ENV['LOKI_URL'],
|
|
253
|
+
labels: { env: Rails.env, service: 'api' }
|
|
254
|
+
)
|
|
255
|
+
|
|
256
|
+
config.register_adapter :sentry, E11y::Adapters::SentryAdapter.new(
|
|
257
|
+
dsn: ENV['SENTRY_DSN']
|
|
258
|
+
)
|
|
259
|
+
|
|
260
|
+
config.register_adapter :s3, E11y::Adapters::S3Adapter.new(
|
|
261
|
+
bucket: 'events-archive'
|
|
262
|
+
)
|
|
263
|
+
|
|
264
|
+
# === DEFAULTS (conventions) ===
|
|
265
|
+
config.default_adapters = [:loki] # Most events → Loki
|
|
266
|
+
config.default_sample_rate = 0.1 # 10% sampling
|
|
267
|
+
config.default_rate_limit = 1000 # 1000 events/sec
|
|
268
|
+
|
|
269
|
+
# === REQUEST SCOPE BUFFERING ===
|
|
270
|
+
config.request_scope do
|
|
271
|
+
enabled true
|
|
272
|
+
buffer_limit 100 # max debug events per request
|
|
273
|
+
flush_on :error # :error, :always, :never
|
|
274
|
+
end
|
|
275
|
+
|
|
276
|
+
# === GLOBAL RATE LIMITING (infrastructure) ===
|
|
277
|
+
config.rate_limiting do
|
|
278
|
+
global limit: 10_000, window: 1.minute
|
|
279
|
+
end
|
|
280
|
+
|
|
281
|
+
# === CARDINALITY PROTECTION (infrastructure) ===
|
|
282
|
+
config.cardinality_protection do
|
|
283
|
+
forbidden_labels :user_id, :order_id, :session_id
|
|
284
|
+
default_cardinality_limit 100
|
|
285
|
+
end
|
|
286
|
+
|
|
287
|
+
# === SLO TRACKING (zero config!) ===
|
|
288
|
+
config.slo_tracking = true # ← ВСЁ!
|
|
289
|
+
|
|
290
|
+
# === AUDIT RETENTION (global default) ===
|
|
291
|
+
# Default for audit events, can be overridden per event
|
|
292
|
+
config.audit_retention = case ENV['JURISDICTION']
|
|
293
|
+
when 'EU' then 7.years # GDPR
|
|
294
|
+
when 'US' then 10.years # SOX
|
|
295
|
+
else 5.years
|
|
296
|
+
end
|
|
297
|
+
|
|
298
|
+
# ✅ That's it! No per-event config here anymore!
|
|
299
|
+
end
|
|
300
|
+
```
|
|
301
|
+
|
|
302
|
+
### Event-Level Config (Locality of Behavior)
|
|
303
|
+
|
|
304
|
+
```ruby
|
|
305
|
+
# app/events/order_created.rb
|
|
306
|
+
module Events
|
|
307
|
+
class OrderCreated < E11y::Event::Base
|
|
308
|
+
schema do
|
|
309
|
+
required(:order_id).filled(:string)
|
|
310
|
+
required(:amount).filled(:decimal)
|
|
311
|
+
end
|
|
312
|
+
|
|
313
|
+
# ✨ Event-level config (right next to schema!)
|
|
314
|
+
severity :success
|
|
315
|
+
rate_limit 1000, window: 1.second
|
|
316
|
+
sample_rate 0.1 # 10% sampling
|
|
317
|
+
retention 30.days
|
|
318
|
+
adapters [:loki, :elasticsearch]
|
|
319
|
+
|
|
320
|
+
# Metric definition
|
|
321
|
+
metric :counter,
|
|
322
|
+
name: 'orders.created.total',
|
|
323
|
+
tags: [:currency]
|
|
324
|
+
end
|
|
325
|
+
end
|
|
326
|
+
```
|
|
327
|
+
|
|
328
|
+
### Old vs New Config
|
|
329
|
+
|
|
330
|
+
**Before (v1.0): 1400+ lines in global config**
|
|
331
|
+
|
|
332
|
+
```ruby
|
|
333
|
+
# config/initializers/e11y.rb (1400+ lines!)
|
|
334
|
+
E11y.configure do |config|
|
|
335
|
+
# Adapters registration
|
|
336
|
+
config.register_adapter :loki, Loki.new(...)
|
|
337
|
+
config.register_adapter :sentry, Sentry.new(...)
|
|
338
|
+
|
|
339
|
+
# ❌ Per-event config (100+ events!)
|
|
340
|
+
config.events do
|
|
341
|
+
event 'Events::OrderCreated' do
|
|
342
|
+
severity :success
|
|
343
|
+
adapters [:loki]
|
|
344
|
+
sample_rate 0.1
|
|
345
|
+
retention 30.days
|
|
346
|
+
rate_limit 1000
|
|
347
|
+
end
|
|
348
|
+
|
|
349
|
+
event 'Events::PaymentSucceeded' do
|
|
350
|
+
severity :success
|
|
351
|
+
adapters [:loki, :sentry, :s3]
|
|
352
|
+
sample_rate 1.0
|
|
353
|
+
retention 7.years
|
|
354
|
+
rate_limit 1000
|
|
355
|
+
end
|
|
356
|
+
|
|
357
|
+
# ... 98+ more events ...
|
|
358
|
+
end
|
|
359
|
+
end
|
|
360
|
+
```
|
|
361
|
+
|
|
362
|
+
**After (v1.1): <300 lines in global config**
|
|
363
|
+
|
|
364
|
+
```ruby
|
|
365
|
+
# config/initializers/e11y.rb (<300 lines!)
|
|
366
|
+
E11y.configure do |config|
|
|
367
|
+
# ONLY infrastructure
|
|
368
|
+
config.register_adapter :loki, Loki.new(...)
|
|
369
|
+
config.register_adapter :sentry, Sentry.new(...)
|
|
370
|
+
|
|
371
|
+
# Defaults (conventions)
|
|
372
|
+
config.default_adapters = [:loki]
|
|
373
|
+
|
|
374
|
+
# ✅ No per-event config here!
|
|
375
|
+
end
|
|
376
|
+
|
|
377
|
+
# app/events/order_created.rb
|
|
378
|
+
class Events::OrderCreated < E11y::Event::Base
|
|
379
|
+
schema do; required(:order_id).filled(:string); end
|
|
380
|
+
# ← Uses conventions (zero config!)
|
|
381
|
+
end
|
|
382
|
+
|
|
383
|
+
# app/events/payment_succeeded.rb
|
|
384
|
+
class Events::PaymentSucceeded < Events::BasePaymentEvent
|
|
385
|
+
schema do; required(:transaction_id).filled(:string); end
|
|
386
|
+
# ← Inherits config from BasePaymentEvent (DRY!)
|
|
387
|
+
end
|
|
388
|
+
```
|
|
389
|
+
|
|
390
|
+
### Migration Path (Backward Compatible)
|
|
391
|
+
|
|
392
|
+
```ruby
|
|
393
|
+
# v1.1 supports BOTH styles (backward compatible!)
|
|
394
|
+
|
|
395
|
+
# Old style (still works):
|
|
396
|
+
E11y.configure do |config|
|
|
397
|
+
config.events do
|
|
398
|
+
event 'Events::OrderCreated' do
|
|
399
|
+
adapters [:loki]
|
|
400
|
+
end
|
|
401
|
+
end
|
|
402
|
+
end
|
|
403
|
+
|
|
404
|
+
# New style (preferred):
|
|
405
|
+
class Events::OrderCreated < E11y::Event::Base
|
|
406
|
+
adapters [:loki] # ← Event-level (overrides global)
|
|
407
|
+
end
|
|
408
|
+
|
|
409
|
+
# Migrate incrementally (both work together!)
|
|
410
|
+
```
|
|
411
|
+
|
|
412
|
+
---
|
|
413
|
+
|
|
414
|
+
## 🔧 Old Configuration (v1.0 - Deprecated)
|
|
415
|
+
|
|
416
|
+
> **⚠️ Deprecated:** This section shows v1.0 configuration style for reference.
|
|
417
|
+
> Use event-level configuration (above) for new projects.
|
|
418
|
+
|
|
419
|
+
```ruby
|
|
420
|
+
# config/initializers/e11y.rb (OLD STYLE)
|
|
421
|
+
E11y.configure do |config|
|
|
422
|
+
# === SEVERITY ===
|
|
423
|
+
config.severity = Rails.env.production? ? :info : :debug
|
|
424
|
+
|
|
425
|
+
# === ADAPTERS (old style) ===
|
|
426
|
+
config.adapters = [
|
|
427
|
+
# Loki for logs
|
|
428
|
+
E11y::Adapters::LokiAdapter.new(
|
|
429
|
+
url: ENV['LOKI_URL'],
|
|
430
|
+
labels: { env: Rails.env, service: 'api' }
|
|
431
|
+
),
|
|
432
|
+
|
|
433
|
+
# Sentry for errors
|
|
434
|
+
E11y::Adapters::SentryAdapter.new(
|
|
435
|
+
severity_filter: [:error, :fatal]
|
|
436
|
+
),
|
|
437
|
+
|
|
438
|
+
# Stdout for development
|
|
439
|
+
(E11y::Adapters::StdoutAdapter.new if Rails.env.development?)
|
|
440
|
+
].compact
|
|
441
|
+
|
|
442
|
+
# === PATTERN-BASED METRICS ===
|
|
443
|
+
config.metrics do
|
|
444
|
+
# Counter for all events
|
|
445
|
+
counter_for pattern: '*',
|
|
446
|
+
name: 'business_events_total',
|
|
447
|
+
tags: [:event_name, :severity]
|
|
448
|
+
|
|
449
|
+
# Histogram for payments
|
|
450
|
+
histogram_for pattern: '*.paid',
|
|
451
|
+
name: 'payment_amount',
|
|
452
|
+
value: ->(e) { e.payload[:amount] },
|
|
453
|
+
buckets: [10, 50, 100, 500, 1000]
|
|
454
|
+
|
|
455
|
+
# Success rate auto-metric
|
|
456
|
+
success_rate_for pattern: 'order.*',
|
|
457
|
+
name: 'order_operations_success_rate'
|
|
458
|
+
end
|
|
459
|
+
|
|
460
|
+
# === PII FILTERING (Rails-compatible!) ===
|
|
461
|
+
config.pii_filter do
|
|
462
|
+
# AUTO: Uses Rails.application.config.filter_parameters (default: true)
|
|
463
|
+
use_rails_filter_parameters true
|
|
464
|
+
|
|
465
|
+
# SIMPLE: Add more filters (Rails-style)
|
|
466
|
+
filter_parameters :api_key, :auth_token, /secret/i
|
|
467
|
+
|
|
468
|
+
# WHITELIST: Allow specific IDs even if filtered by Rails
|
|
469
|
+
allow_parameters :user_id, :order_id, :transaction_id
|
|
470
|
+
|
|
471
|
+
# ADVANCED: Pattern-based (beyond Rails)
|
|
472
|
+
filter_pattern /\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/i,
|
|
473
|
+
replacement: '[EMAIL]'
|
|
474
|
+
filter_pattern /\b\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{4}\b/,
|
|
475
|
+
replacement: '[CARD]'
|
|
476
|
+
end
|
|
477
|
+
|
|
478
|
+
# === RATE LIMITING ===
|
|
479
|
+
config.rate_limiting do
|
|
480
|
+
global limit: 10_000, window: 1.minute
|
|
481
|
+
per_event 'user.login.failed', limit: 100, window: 1.minute
|
|
482
|
+
end
|
|
483
|
+
|
|
484
|
+
# === CONTEXT ENRICHMENT ===
|
|
485
|
+
config.context_enricher do |event|
|
|
486
|
+
{
|
|
487
|
+
trace_id: E11y::TraceId.extract,
|
|
488
|
+
user_id: Current.user&.id,
|
|
489
|
+
tenant_id: Current.tenant&.id
|
|
490
|
+
}
|
|
491
|
+
end
|
|
492
|
+
end
|
|
493
|
+
|
|
494
|
+
# Start async workers
|
|
495
|
+
E11y.start!
|
|
496
|
+
|
|
497
|
+
# Graceful shutdown
|
|
498
|
+
at_exit { E11y.stop!(timeout: 5) }
|
|
499
|
+
```
|
|
500
|
+
|
|
501
|
+
---
|
|
502
|
+
|
|
503
|
+
## 📊 Severity Levels
|
|
504
|
+
|
|
505
|
+
```ruby
|
|
506
|
+
E11y::SEVERITIES = {
|
|
507
|
+
debug: 0, # Detailed diagnostic (buffered in request scope)
|
|
508
|
+
info: 1, # Informational
|
|
509
|
+
success: 2, # ← NEW! Successful operations
|
|
510
|
+
warn: 3, # Warnings
|
|
511
|
+
error: 4, # Errors
|
|
512
|
+
fatal: 5 # Critical failures
|
|
513
|
+
}
|
|
514
|
+
```
|
|
515
|
+
|
|
516
|
+
**When to use :success?**
|
|
517
|
+
|
|
518
|
+
```ruby
|
|
519
|
+
# ✅ Use :success for completed operations
|
|
520
|
+
Events::OrderPaid.track(order_id: '123', severity: :success)
|
|
521
|
+
Events::JobCompleted.track(job_id: '456', severity: :success)
|
|
522
|
+
Events::EmailSent.track(user_id: '789', severity: :success)
|
|
523
|
+
|
|
524
|
+
# ✅ Use :info for informational events
|
|
525
|
+
Events::UserLoggedIn.track(user_id: '123', severity: :info)
|
|
526
|
+
Events::SessionStarted.track(session_id: '456', severity: :info)
|
|
527
|
+
|
|
528
|
+
# Why separate :success from :info?
|
|
529
|
+
# → Easy filtering: severity:success = only successful ops
|
|
530
|
+
# → Easy metrics: success_rate = count(:success) / count(:success OR :error)
|
|
531
|
+
```
|
|
532
|
+
|
|
533
|
+
---
|
|
534
|
+
|
|
535
|
+
## 🎭 Middleware (Auto-configured)
|
|
536
|
+
|
|
537
|
+
### Rails / Rack
|
|
538
|
+
|
|
539
|
+
```ruby
|
|
540
|
+
# config/application.rb (auto-added by generator)
|
|
541
|
+
config.middleware.use E11y::Middleware::Rack,
|
|
542
|
+
buffer_limit: 100,
|
|
543
|
+
flush_on: :error
|
|
544
|
+
```
|
|
545
|
+
|
|
546
|
+
### Sidekiq
|
|
547
|
+
|
|
548
|
+
```ruby
|
|
549
|
+
# config/initializers/sidekiq.rb (auto-added by generator)
|
|
550
|
+
Sidekiq.configure_server do |config|
|
|
551
|
+
config.server_middleware do |chain|
|
|
552
|
+
chain.add E11y::Middleware::Sidekiq
|
|
553
|
+
end
|
|
554
|
+
end
|
|
555
|
+
```
|
|
556
|
+
|
|
557
|
+
### ActiveJob
|
|
558
|
+
|
|
559
|
+
```ruby
|
|
560
|
+
# Auto-included by E11y (no config needed)
|
|
561
|
+
# trace_id propagated automatically to background jobs
|
|
562
|
+
```
|
|
563
|
+
|
|
564
|
+
---
|
|
565
|
+
|
|
566
|
+
## 🔍 Trace Context Flow
|
|
567
|
+
|
|
568
|
+
```ruby
|
|
569
|
+
# Service A (API)
|
|
570
|
+
POST /orders
|
|
571
|
+
trace_id: abc-123 (from X-Trace-ID header)
|
|
572
|
+
├─ Events::OrderValidation.track (trace_id: abc-123)
|
|
573
|
+
├─ Events::OrderCreated.track (trace_id: abc-123)
|
|
574
|
+
└─ ProcessOrderJob.perform_later(order_id, trace_id: abc-123)
|
|
575
|
+
|
|
576
|
+
# Service B (Background Job)
|
|
577
|
+
ProcessOrderJob
|
|
578
|
+
trace_id: abc-123 (from job args)
|
|
579
|
+
├─ Events::OrderProcessing.track (trace_id: abc-123)
|
|
580
|
+
└─ HTTP → Service C (headers: X-Trace-ID: abc-123)
|
|
581
|
+
|
|
582
|
+
# Service C (Payment API)
|
|
583
|
+
POST /payments
|
|
584
|
+
trace_id: abc-123 (from X-Trace-ID header)
|
|
585
|
+
└─ Events::PaymentProcessed.track (trace_id: abc-123)
|
|
586
|
+
|
|
587
|
+
# Result: All events linked by trace_id = full visibility
|
|
588
|
+
```
|
|
589
|
+
|
|
590
|
+
---
|
|
591
|
+
|
|
592
|
+
## 📈 Yabeda Integration
|
|
593
|
+
|
|
594
|
+
Events автоматически становятся метриками через pattern-based rules:
|
|
595
|
+
|
|
596
|
+
```ruby
|
|
597
|
+
# After tracking event:
|
|
598
|
+
Events::OrderPaid.track(amount: 99, currency: 'USD')
|
|
599
|
+
|
|
600
|
+
# Auto-generated metrics:
|
|
601
|
+
yabeda.business_events.events_total{event_name="order.paid",severity="success"} 1
|
|
602
|
+
yabeda.business_events.payment_amount_bucket{currency="USD",le="100"} 1
|
|
603
|
+
yabeda.business_events.order_operations_success 1
|
|
604
|
+
yabeda.business_events.order_operations_total 1
|
|
605
|
+
```
|
|
606
|
+
|
|
607
|
+
**Prometheus endpoint:**
|
|
608
|
+
|
|
609
|
+
```ruby
|
|
610
|
+
# config/routes.rb
|
|
611
|
+
mount Yabeda::Prometheus::Exporter => '/metrics'
|
|
612
|
+
```
|
|
613
|
+
|
|
614
|
+
---
|
|
615
|
+
|
|
616
|
+
## 🧪 Testing
|
|
617
|
+
|
|
618
|
+
```ruby
|
|
619
|
+
# spec/spec_helper.rb
|
|
620
|
+
RSpec.configure do |config|
|
|
621
|
+
config.before(:each) do
|
|
622
|
+
E11y.configure do |c|
|
|
623
|
+
c.adapters = [E11y::Adapters::NullAdapter.new]
|
|
624
|
+
end
|
|
625
|
+
end
|
|
626
|
+
end
|
|
627
|
+
|
|
628
|
+
# spec/controllers/orders_controller_spec.rb
|
|
629
|
+
RSpec.describe OrdersController do
|
|
630
|
+
it 'tracks order creation' do
|
|
631
|
+
expect(Events::OrderCreated).to receive(:track).with(
|
|
632
|
+
hash_including(order_id: anything)
|
|
633
|
+
)
|
|
634
|
+
|
|
635
|
+
post :create, params: { ... }
|
|
636
|
+
end
|
|
637
|
+
end
|
|
638
|
+
```
|
|
639
|
+
|
|
640
|
+
---
|
|
641
|
+
|
|
642
|
+
## 🚀 Performance
|
|
643
|
+
|
|
644
|
+
| Metric | Target | Actual |
|
|
645
|
+
|--------|--------|--------|
|
|
646
|
+
| **Track latency (p99)** | <1ms | ✅ 0.8ms |
|
|
647
|
+
| **Throughput** | 10k events/sec | ✅ 15k/sec |
|
|
648
|
+
| **Memory** | <100MB @ 100k buffer | ✅ 80MB |
|
|
649
|
+
| **CPU overhead** | <5% @ 1k events/sec | ✅ 3% |
|
|
650
|
+
|
|
651
|
+
**Optimizations:**
|
|
652
|
+
- Early severity filtering (<1μs для filtered events)
|
|
653
|
+
- Lock-free ring buffer (SPSC)
|
|
654
|
+
- Async workers (no blocking)
|
|
655
|
+
- Lazy serialization (только перед отправкой)
|
|
656
|
+
|
|
657
|
+
---
|
|
658
|
+
|
|
659
|
+
## 🔐 Security
|
|
660
|
+
|
|
661
|
+
### PII Filtering (Rails-Compatible! 🎉)
|
|
662
|
+
|
|
663
|
+
**ZERO CONFIG**: E11y автоматически использует `Rails.application.config.filter_parameters`!
|
|
664
|
+
|
|
665
|
+
```ruby
|
|
666
|
+
# config/application.rb
|
|
667
|
+
config.filter_parameters += [:password, :email, :ssn]
|
|
668
|
+
|
|
669
|
+
# E11y автоматически фильтрует эти поля - NO ADDITIONAL CONFIG!
|
|
670
|
+
Events::UserRegistered.track(
|
|
671
|
+
email: 'user@example.com', # → '[FILTERED]'
|
|
672
|
+
password: 'secret123' # → '[FILTERED]'
|
|
673
|
+
)
|
|
674
|
+
```
|
|
675
|
+
|
|
676
|
+
**EXTENDED**: Добавьте больше фильтров поверх Rails:
|
|
677
|
+
|
|
678
|
+
```ruby
|
|
679
|
+
# config/initializers/e11y.rb
|
|
680
|
+
E11y.configure do |config|
|
|
681
|
+
config.pii_filter do
|
|
682
|
+
# Inherit Rails filters + add more
|
|
683
|
+
filter_parameters :api_key, :auth_token, /secret/i
|
|
684
|
+
|
|
685
|
+
# Whitelist known-safe fields
|
|
686
|
+
allow_parameters :user_id, :order_id
|
|
687
|
+
|
|
688
|
+
# Pattern-based (content filtering)
|
|
689
|
+
filter_pattern /\b\d{16}\b/, replacement: '[CARD]'
|
|
690
|
+
end
|
|
691
|
+
end
|
|
692
|
+
```
|
|
693
|
+
|
|
694
|
+
**See full guide**: `e11y-rails-compatible-pii-filtering.md`
|
|
695
|
+
|
|
696
|
+
### Rate Limiting (защита от DoS)
|
|
697
|
+
|
|
698
|
+
```ruby
|
|
699
|
+
# Config
|
|
700
|
+
config.rate_limiting do
|
|
701
|
+
global limit: 10_000, window: 1.minute
|
|
702
|
+
per_event 'user.login.failed', limit: 100, window: 1.minute
|
|
703
|
+
end
|
|
704
|
+
|
|
705
|
+
# При превышении - события дропаются
|
|
706
|
+
```
|
|
707
|
+
|
|
708
|
+
---
|
|
709
|
+
|
|
710
|
+
## 🎯 Built-in SLO Tracking (Zero Config!)
|
|
711
|
+
|
|
712
|
+
**Enable one flag → get SLO out of the box!**
|
|
713
|
+
|
|
714
|
+
```ruby
|
|
715
|
+
# config/initializers/e11y.rb
|
|
716
|
+
E11y.configure do |config|
|
|
717
|
+
config.slo_tracking = true # ← ВСЁ! Этого достаточно!
|
|
718
|
+
|
|
719
|
+
# Опционально: кастомизация
|
|
720
|
+
config.slo do
|
|
721
|
+
# Global defaults
|
|
722
|
+
http_ignore_statuses [404, 401] # 404/401 не ошибки
|
|
723
|
+
latency_target_p95 200 # ms
|
|
724
|
+
|
|
725
|
+
# 🎯 NEW: Per-controller overrides (РЕКОМЕНДУЕТСЯ для Rails!)
|
|
726
|
+
controller 'Api::Admin::BaseController' do
|
|
727
|
+
ignore true # Весь admin не входит в SLO
|
|
728
|
+
end
|
|
729
|
+
|
|
730
|
+
controller 'Api::OrdersController', action: 'show' do
|
|
731
|
+
latency_target_p95 50 # Show должен быть быстрым
|
|
732
|
+
end
|
|
733
|
+
|
|
734
|
+
controller 'Api::OrdersController', action: 'create' do
|
|
735
|
+
latency_target_p95 200 # Create может быть медленнее
|
|
736
|
+
ignore_statuses [422] # 422 Validation = not SLO breach
|
|
737
|
+
end
|
|
738
|
+
|
|
739
|
+
# 🔧 LEGACY: Path-based (для non-Rails apps)
|
|
740
|
+
endpoint '/api/webhooks/*' do
|
|
741
|
+
ignore true
|
|
742
|
+
end
|
|
743
|
+
|
|
744
|
+
# 🎯 Per-job overrides
|
|
745
|
+
job 'ReportGenerationJob' do
|
|
746
|
+
ignore true # Долгие джобы не входят в SLO
|
|
747
|
+
end
|
|
748
|
+
|
|
749
|
+
job 'ProcessPaymentJob' do
|
|
750
|
+
latency_target_p95 1000 # Критичные джобы = строгий SLO
|
|
751
|
+
end
|
|
752
|
+
end
|
|
753
|
+
end
|
|
754
|
+
```
|
|
755
|
+
|
|
756
|
+
**Автоматически получаем:**
|
|
757
|
+
|
|
758
|
+
```ruby
|
|
759
|
+
# 🎯 HTTP Metrics (controller#action - автогруппировка!)
|
|
760
|
+
yabeda_slo_http_requests_total{
|
|
761
|
+
status="200",
|
|
762
|
+
method="GET",
|
|
763
|
+
controller="Api::OrdersController",
|
|
764
|
+
action="show"
|
|
765
|
+
} 1234
|
|
766
|
+
|
|
767
|
+
yabeda_slo_http_request_duration_seconds{
|
|
768
|
+
method="GET",
|
|
769
|
+
controller="Api::OrdersController",
|
|
770
|
+
action="show"
|
|
771
|
+
} histogram
|
|
772
|
+
|
|
773
|
+
# Sidekiq Metrics
|
|
774
|
+
yabeda_slo_sidekiq_jobs_total{queue="default", class="ProcessOrderJob", status="success"} 456
|
|
775
|
+
yabeda_slo_sidekiq_job_duration_seconds{queue="default", class="ProcessOrderJob"} histogram
|
|
776
|
+
|
|
777
|
+
# ActiveJob Metrics
|
|
778
|
+
yabeda_slo_active_jobs_total{queue="mailers", class="EmailJob", status="success"} 789
|
|
779
|
+
yabeda_slo_active_job_duration_seconds{queue="mailers", class="EmailJob"} histogram
|
|
780
|
+
```
|
|
781
|
+
|
|
782
|
+
**Преимущество:** `/orders/123`, `/orders/456` → один `OrdersController#show` (не нужна нормализация path!)
|
|
783
|
+
|
|
784
|
+
**SLO Calculations (PromQL):**
|
|
785
|
+
|
|
786
|
+
```promql
|
|
787
|
+
# HTTP Availability (30d rolling)
|
|
788
|
+
100 * (sum(rate(yabeda_slo_http_successes_total[30d])) /
|
|
789
|
+
(sum(rate(yabeda_slo_http_successes_total[30d])) + sum(rate(yabeda_slo_http_errors_total[30d]))))
|
|
790
|
+
# Expected: >= 99.9%
|
|
791
|
+
|
|
792
|
+
# p95 Latency
|
|
793
|
+
histogram_quantile(0.95, rate(yabeda_slo_http_request_duration_seconds_bucket[5m]))
|
|
794
|
+
# Expected: < 200ms
|
|
795
|
+
|
|
796
|
+
# Error Budget Remaining
|
|
797
|
+
100 * (1 - (sum(rate(yabeda_slo_http_errors_total[30d])) /
|
|
798
|
+
(sum(rate(yabeda_slo_http_successes_total[30d])) + sum(rate(yabeda_slo_http_errors_total[30d])))) / 0.001)
|
|
799
|
+
# 100% = весь бюджет остался, 0% = исчерпан
|
|
800
|
+
```
|
|
801
|
+
|
|
802
|
+
**Auto-Generated:**
|
|
803
|
+
|
|
804
|
+
```bash
|
|
805
|
+
# Grafana dashboard
|
|
806
|
+
rails g e11y:grafana_dashboard
|
|
807
|
+
# → config/grafana/e11y_slo_dashboard.json
|
|
808
|
+
|
|
809
|
+
# Prometheus alerts
|
|
810
|
+
rails g e11y:prometheus_alerts
|
|
811
|
+
# → config/prometheus/e11y_slo_alerts.yml
|
|
812
|
+
```
|
|
813
|
+
|
|
814
|
+
**Что трекается автоматически:**
|
|
815
|
+
- ✅ **Rack middleware** - все HTTP requests (availability, latency)
|
|
816
|
+
- ✅ **Sidekiq middleware** - все jobs (success rate, duration)
|
|
817
|
+
- ✅ **ActiveJob instrumentation** - все jobs (success rate, duration)
|
|
818
|
+
- ✅ **Path normalization** - `/orders/123` → `/orders/:id`
|
|
819
|
+
- ✅ **Error categorization** - configurable (5xx = error, 404 = ignore)
|
|
820
|
+
- ✅ **Heartbeat** - auto-enabled (pod liveness detection)
|
|
821
|
+
|
|
822
|
+
**⚠️ ВАЖНО: Ограничения in-process SLO**
|
|
823
|
+
|
|
824
|
+
E11y SLO работает ВНУТРИ Ruby процесса и НЕ видит:
|
|
825
|
+
- ❌ Network issues (requests не доходят до app)
|
|
826
|
+
- ❌ Load balancer down
|
|
827
|
+
- ❌ All pods crashed (метрик просто нет)
|
|
828
|
+
- ❌ DNS issues
|
|
829
|
+
|
|
830
|
+
**Решение:** Multi-layer monitoring (см. полную документацию):
|
|
831
|
+
1. **Layer 1**: E11y SLO (in-process)
|
|
832
|
+
2. **Layer 2**: E11y Heartbeat (pod liveness) - **auto-enabled!**
|
|
833
|
+
3. **Layer 3**: K8s health probes (`/health/live`, `/health/ready`) - **auto-created!**
|
|
834
|
+
4. **Layer 4**: External synthetic monitoring (Prometheus Blackbox Exporter)
|
|
835
|
+
|
|
836
|
+
```ruby
|
|
837
|
+
# Heartbeat автоматически включен с slo_tracking = true
|
|
838
|
+
# Метрики:
|
|
839
|
+
yabeda_e11y_heartbeat_timestamp{pod="pod-1"} 1703500000 # Последний heartbeat
|
|
840
|
+
yabeda_e11y_service_healthy{pod="pod-1"} 1 # 1 = healthy
|
|
841
|
+
|
|
842
|
+
# Alert если pod мертв:
|
|
843
|
+
# (time() - yabeda_e11y_heartbeat_timestamp) > 30s → Pod down!
|
|
844
|
+
```
|
|
845
|
+
|
|
846
|
+
---
|
|
847
|
+
|
|
848
|
+
## 🎯 Migration from Rails.logger
|
|
849
|
+
|
|
850
|
+
```ruby
|
|
851
|
+
# ❌ Before
|
|
852
|
+
Rails.logger.info "Order #{order.id} paid #{order.amount} #{order.currency}"
|
|
853
|
+
OrderMetrics.increment('orders.paid.total')
|
|
854
|
+
OrderMetrics.observe('orders.paid.amount', order.amount)
|
|
855
|
+
|
|
856
|
+
# ✅ After (1 строка вместо 3!)
|
|
857
|
+
Events::OrderPaid.track(
|
|
858
|
+
order_id: order.id,
|
|
859
|
+
amount: order.amount,
|
|
860
|
+
currency: order.currency
|
|
861
|
+
)
|
|
862
|
+
# → Structured log + auto-metrics + trace context
|
|
863
|
+
```
|
|
864
|
+
|
|
865
|
+
---
|
|
866
|
+
|
|
867
|
+
## 🐛 Troubleshooting
|
|
868
|
+
|
|
869
|
+
### Events not appearing?
|
|
870
|
+
|
|
871
|
+
```ruby
|
|
872
|
+
# Check 1: Severity
|
|
873
|
+
E11y.enabled_for?(:debug) # Should be true
|
|
874
|
+
|
|
875
|
+
# Check 2: Adapters
|
|
876
|
+
E11y.config.adapters # Should not be empty
|
|
877
|
+
|
|
878
|
+
# Check 3: Buffer
|
|
879
|
+
E11y.buffer_size # Should be < capacity
|
|
880
|
+
|
|
881
|
+
# Check 4: Circuit breaker
|
|
882
|
+
E11y.circuit_breaker_state # Should be :closed
|
|
883
|
+
```
|
|
884
|
+
|
|
885
|
+
### High latency?
|
|
886
|
+
|
|
887
|
+
```ruby
|
|
888
|
+
# Check metrics
|
|
889
|
+
Yabeda.e11y.track_duration_seconds # p99 should be <1ms
|
|
890
|
+
|
|
891
|
+
# Possible causes:
|
|
892
|
+
# - PII filtering regex too complex
|
|
893
|
+
# - Rate limiter Redis slow
|
|
894
|
+
# - Adapter network slow
|
|
895
|
+
```
|
|
896
|
+
|
|
897
|
+
---
|
|
898
|
+
|
|
899
|
+
## 📚 Full Documentation
|
|
900
|
+
|
|
901
|
+
- **Complete spec**: `severity/e11y-final-spec.md` (2000+ lines)
|
|
902
|
+
- **Old spec**: `severity/tz-improved.md` (reference)
|
|
903
|
+
- **GitHub**: https://github.com/yourorg/e11y
|
|
904
|
+
|
|
905
|
+
---
|
|
906
|
+
|
|
907
|
+
## ✅ Checklist
|
|
908
|
+
|
|
909
|
+
- [ ] Install gem
|
|
910
|
+
- [ ] Run generator: `rails g e11y:install`
|
|
911
|
+
- [ ] Configure adapters (Loki/Sentry/ELK)
|
|
912
|
+
- [ ] **Enable SLO tracking**: `config.slo_tracking = true`
|
|
913
|
+
- [ ] Define first event class
|
|
914
|
+
- [ ] Track first event
|
|
915
|
+
- [ ] Check `/metrics` endpoint (Prometheus)
|
|
916
|
+
- [ ] Verify events in Grafana/Kibana
|
|
917
|
+
- [ ] **Verify SLO metrics**: `yabeda_slo_*` in Prometheus
|
|
918
|
+
- [ ] Test request-scoped buffering (raise exception → see debug logs)
|
|
919
|
+
- [ ] Configure PII filtering
|
|
920
|
+
- [ ] Setup rate limiting
|
|
921
|
+
- [ ] Configure pattern-based metrics
|
|
922
|
+
- [ ] **Generate Grafana dashboard**: `rails g e11y:grafana_dashboard`
|
|
923
|
+
- [ ] **Generate Prometheus alerts**: `rails g e11y:prometheus_alerts`
|
|
924
|
+
- [ ] Deploy to staging
|
|
925
|
+
- [ ] Monitor performance
|
|
926
|
+
- [ ] Rollout to production (canary 1% → 10% → 100%)
|
|
927
|
+
|
|
928
|
+
---
|
|
929
|
+
|
|
930
|
+
**Questions?** See `e11y-final-spec.md` FAQ section or open GitHub issue.
|
|
931
|
+
|
|
932
|
+
**Version**: 1.0.0
|
|
933
|
+
**Last Updated**: 2025-12-24
|
|
934
|
+
|