e11y 0.2.0 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.rubocop.yml +130 -10
- data/CHANGELOG.md +56 -1
- data/CLAUDE.md +168 -0
- data/CONTRIBUTING.md +640 -0
- data/README.md +134 -702
- data/RELEASE.md +18 -3
- data/Rakefile +108 -29
- data/config/README.md +1 -1
- data/config/loki-local-config.yaml +12 -0
- data/config/otel-collector-config.yaml +44 -0
- data/cucumber.yml +1 -0
- data/docker-compose.yml +18 -2
- data/docs/ADAPTERS.md +76 -0
- data/docs/ADAPTIVE_SAMPLING.md +59 -0
- data/docs/COMPARISON.md +104 -0
- data/docs/CONFIGURATION.md +52 -0
- data/docs/DISTRIBUTED_TRACING.md +44 -0
- data/docs/LIMITATIONS.md +13 -0
- data/docs/METRICS_DSL.md +84 -0
- data/docs/PERFORMANCE.md +60 -0
- data/docs/PII_FILTERING.md +40 -0
- data/docs/PRESETS.md +65 -0
- data/docs/QUICK-START.md +546 -587
- data/docs/RAILS_INTEGRATION.md +29 -0
- data/docs/SCHEMA_VALIDATION.md +63 -0
- data/docs/SLO-PROMQL-ALERTS.md +161 -0
- data/docs/TESTING.md +69 -0
- data/docs/{ADR-001-architecture.md → architecture/ADR-001-architecture.md} +35 -64
- data/docs/{ADR-002-metrics-yabeda.md → architecture/ADR-002-metrics-yabeda.md} +62 -236
- data/docs/{ADR-003-slo-observability.md → architecture/ADR-003-slo-observability.md} +27 -466
- data/docs/{ADR-004-adapter-architecture.md → architecture/ADR-004-adapter-architecture.md} +163 -146
- data/docs/{ADR-005-tracing-context.md → architecture/ADR-005-tracing-context.md} +10 -9
- data/docs/{ADR-006-security-compliance.md → architecture/ADR-006-security-compliance.md} +184 -191
- data/docs/{ADR-007-opentelemetry-integration.md → architecture/ADR-007-opentelemetry-integration.md} +3 -21
- data/docs/{ADR-008-rails-integration.md → architecture/ADR-008-rails-integration.md} +209 -339
- data/docs/{ADR-009-cost-optimization.md → architecture/ADR-009-cost-optimization.md} +45 -54
- data/docs/architecture/ADR-010-developer-experience.md +522 -0
- data/docs/{ADR-011-testing-strategy.md → architecture/ADR-011-testing-strategy.md} +41 -83
- data/docs/{ADR-013-reliability-error-handling.md → architecture/ADR-013-reliability-error-handling.md} +37 -12
- data/docs/{ADR-014-event-driven-slo.md → architecture/ADR-014-event-driven-slo.md} +12 -24
- data/docs/{ADR-015-middleware-order.md → architecture/ADR-015-middleware-order.md} +23 -41
- data/docs/{ADR-016-self-monitoring-slo.md → architecture/ADR-016-self-monitoring-slo.md} +52 -349
- data/docs/{ADR-017-multi-rails-compatibility.md → architecture/ADR-017-multi-rails-compatibility.md} +4 -11
- data/docs/architecture/ADR-018-memory-optimization.md +366 -0
- data/docs/{ADR-INDEX.md → architecture/ADR-INDEX.md} +11 -6
- data/docs/{00-ICP-AND-TIMELINE.md → prd/00-ICP-AND-TIMELINE.md} +6 -6
- data/docs/{01-SCALE-REQUIREMENTS.md → prd/01-SCALE-REQUIREMENTS.md} +6 -6
- data/docs/prd/01-overview-vision.md +19 -14
- data/docs/use_cases/README.md +22 -23
- data/docs/use_cases/UC-001-request-scoped-debug-buffering.md +50 -44
- data/docs/use_cases/UC-002-business-event-tracking.md +26 -95
- data/docs/use_cases/UC-003-event-metrics.md +66 -0
- data/docs/use_cases/UC-004-zero-config-slo-tracking.md +42 -101
- data/docs/use_cases/UC-005-sentry-integration.md +13 -15
- data/docs/use_cases/UC-006-trace-context-management.md +30 -28
- data/docs/use_cases/UC-007-pii-filtering.md +35 -87
- data/docs/use_cases/UC-008-opentelemetry-integration.md +51 -89
- data/docs/use_cases/UC-009-multi-service-tracing.md +4 -4
- data/docs/use_cases/UC-010-background-job-tracking.md +5 -5
- data/docs/use_cases/UC-011-rate-limiting.md +95 -168
- data/docs/use_cases/UC-012-audit-trail.md +21 -46
- data/docs/use_cases/UC-013-high-cardinality-protection.md +29 -167
- data/docs/use_cases/UC-014-adaptive-sampling.md +2 -2
- data/docs/use_cases/UC-015-cost-optimization.md +46 -99
- data/docs/use_cases/UC-016-rails-logger-migration.md +39 -213
- data/docs/use_cases/UC-017-local-development.md +203 -777
- data/docs/use_cases/UC-018-testing-events.md +3 -3
- data/docs/use_cases/UC-019-retention-based-routing.md +53 -106
- data/docs/use_cases/UC-020-event-versioning.md +8 -9
- data/docs/use_cases/UC-021-error-handling-retry-dlq.md +18 -22
- data/docs/use_cases/UC-022-event-registry.md +15 -21
- data/docs/use_cases/backlog.md +119 -87
- data/e11y.gemspec +2 -2
- data/gems/e11y-devtools/README.md +136 -0
- data/gems/e11y-devtools/config/routes.rb +8 -0
- data/gems/e11y-devtools/e11y-devtools.gemspec +25 -0
- data/gems/e11y-devtools/exe/e11y +34 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/server.rb +96 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tool_base.rb +25 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/clear.rb +31 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/errors.rb +35 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/event_detail.rb +33 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/events_by_trace.rb +33 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/interactions.rb +40 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/recent_events.rb +34 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/search.rb +34 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/stats.rb +30 -0
- data/gems/e11y-devtools/lib/e11y/devtools/overlay/assets/overlay.js +115 -0
- data/gems/e11y-devtools/lib/e11y/devtools/overlay/controller.rb +54 -0
- data/gems/e11y-devtools/lib/e11y/devtools/overlay/engine.rb +26 -0
- data/gems/e11y-devtools/lib/e11y/devtools/overlay/middleware.rb +80 -0
- data/gems/e11y-devtools/lib/e11y/devtools/overlay/rails_controller.rb +42 -0
- data/gems/e11y-devtools/lib/e11y/devtools/tui/app.rb +262 -0
- data/gems/e11y-devtools/lib/e11y/devtools/tui/grouping.rb +66 -0
- data/gems/e11y-devtools/lib/e11y/devtools/tui/widgets/event_detail.rb +62 -0
- data/gems/e11y-devtools/lib/e11y/devtools/tui/widgets/event_list.rb +70 -0
- data/gems/e11y-devtools/lib/e11y/devtools/tui/widgets/interaction_list.rb +47 -0
- data/gems/e11y-devtools/lib/e11y/devtools/version.rb +8 -0
- data/gems/e11y-devtools/lib/e11y/devtools.rb +13 -0
- data/gems/e11y-devtools/spec/e11y/devtools/mcp/tools_spec.rb +107 -0
- data/gems/e11y-devtools/spec/e11y/devtools/overlay/controller_spec.rb +58 -0
- data/gems/e11y-devtools/spec/e11y/devtools/overlay/middleware_spec.rb +46 -0
- data/gems/e11y-devtools/spec/e11y/devtools/tui/app_spec.rb +85 -0
- data/gems/e11y-devtools/spec/e11y/devtools/tui/grouping_spec.rb +64 -0
- data/gems/e11y-devtools/spec/spec_helper.rb +5 -0
- data/gems/e11y-devtools/spec/tui/widgets/event_list_spec.rb +44 -0
- data/gems/e11y-devtools/spec/tui/widgets/interaction_list_spec.rb +62 -0
- data/lib/e11y/adapters/audit_encrypted.rb +53 -11
- data/lib/e11y/adapters/base.rb +33 -34
- data/lib/e11y/adapters/dev_log/file_store.rb +143 -0
- data/lib/e11y/adapters/dev_log/query.rb +219 -0
- data/lib/e11y/adapters/dev_log.rb +118 -0
- data/lib/e11y/adapters/file.rb +3 -6
- data/lib/e11y/adapters/in_memory.rb +52 -5
- data/lib/e11y/adapters/in_memory_test.rb +29 -0
- data/lib/e11y/adapters/loki.rb +58 -23
- data/lib/e11y/adapters/null.rb +82 -0
- data/lib/e11y/adapters/opentelemetry_collector.rb +183 -0
- data/lib/e11y/adapters/otel_logs.rb +136 -23
- data/lib/e11y/adapters/sentry.rb +4 -7
- data/lib/e11y/adapters/stdout.rb +73 -7
- data/lib/e11y/adapters/yabeda.rb +153 -29
- data/lib/e11y/buffers/adaptive_buffer.rb +3 -17
- data/lib/e11y/buffers/{request_scoped_buffer.rb → ephemeral_buffer.rb} +72 -58
- data/lib/e11y/buffers/ring_buffer.rb +3 -16
- data/lib/e11y/configuration.rb +272 -0
- data/lib/e11y/console.rb +10 -17
- data/lib/e11y/current.rb +53 -1
- data/lib/e11y/debug/pipeline_inspector.rb +96 -0
- data/lib/e11y/documentation/generator.rb +48 -0
- data/lib/e11y/event/base.rb +176 -82
- data/lib/e11y/event/value_sampling_config.rb +1 -5
- data/lib/e11y/events/rails/database/query.rb +1 -4
- data/lib/e11y/events/rails/job/failed.rb +2 -0
- data/lib/e11y/instruments/active_job.rb +46 -12
- data/lib/e11y/instruments/rails_instrumentation.rb +49 -24
- data/lib/e11y/instruments/sidekiq.rb +137 -31
- data/lib/e11y/linters/base.rb +11 -0
- data/lib/e11y/linters/pii/pii_declaration_linter.rb +120 -0
- data/lib/e11y/linters/slo/config_consistency_linter.rb +76 -0
- data/lib/e11y/linters/slo/explicit_declaration_linter.rb +36 -0
- data/lib/e11y/linters/slo/slo_status_from_linter.rb +41 -0
- data/lib/e11y/logger/bridge.rb +26 -7
- data/lib/e11y/metrics/cardinality_protection.rb +10 -15
- data/lib/e11y/metrics/cardinality_tracker.rb +16 -6
- data/lib/e11y/metrics/registry.rb +3 -5
- data/lib/e11y/metrics/test_backend.rb +62 -0
- data/lib/e11y/metrics.rb +56 -10
- data/lib/e11y/middleware/adapter_resolver.rb +40 -0
- data/lib/e11y/middleware/audit_signing.rb +43 -6
- data/lib/e11y/middleware/baggage_protection.rb +75 -0
- data/lib/e11y/middleware/dev_log_source.rb +24 -0
- data/lib/e11y/middleware/event_slo.rb +23 -9
- data/lib/e11y/middleware/otel_span.rb +23 -0
- data/lib/e11y/middleware/pii_filter.rb +104 -75
- data/lib/e11y/middleware/rate_limiting.rb +54 -27
- data/lib/e11y/middleware/request.rb +70 -23
- data/lib/e11y/middleware/routing.rb +78 -21
- data/lib/e11y/middleware/sampling.rb +66 -17
- data/lib/e11y/middleware/self_monitoring_emit.rb +39 -0
- data/lib/e11y/middleware/trace_context.rb +45 -10
- data/lib/e11y/middleware/track_latency.rb +34 -0
- data/lib/e11y/middleware/validation.rb +7 -16
- data/lib/e11y/middleware/versioning.rb +26 -22
- data/lib/e11y/opentelemetry/semantic_conventions.rb +109 -0
- data/lib/e11y/opentelemetry/span_creator.rb +142 -0
- data/lib/e11y/pii/patterns.rb +12 -1
- data/lib/e11y/pipeline/builder.rb +1 -1
- data/lib/e11y/presets/audit_event.rb +13 -2
- data/lib/e11y/railtie.rb +52 -15
- data/lib/e11y/registry.rb +306 -0
- data/lib/e11y/reliability/circuit_breaker.rb +19 -21
- data/lib/e11y/reliability/dlq/base.rb +71 -0
- data/lib/e11y/reliability/dlq/file_adapter.rb +301 -0
- data/lib/e11y/reliability/dlq/file_storage.rb +63 -34
- data/lib/e11y/reliability/dlq/filter.rb +37 -54
- data/lib/e11y/reliability/retry_handler.rb +26 -29
- data/lib/e11y/reliability/retry_rate_limiter.rb +3 -11
- data/lib/e11y/sampling/error_spike_detector.rb +0 -2
- data/lib/e11y/sampling/load_monitor.rb +5 -9
- data/lib/e11y/sampling/stratified_tracker.rb +18 -0
- data/lib/e11y/self_monitoring/buffer_monitor.rb +2 -0
- data/lib/e11y/self_monitoring/performance_monitor.rb +19 -61
- data/lib/e11y/self_monitoring/reliability_monitor.rb +4 -74
- data/lib/e11y/slo/config_loader.rb +40 -0
- data/lib/e11y/slo/config_validator.rb +58 -0
- data/lib/e11y/slo/dashboard_generator.rb +122 -0
- data/lib/e11y/slo/event_driven.rb +8 -0
- data/lib/e11y/slo/tracker.rb +31 -4
- data/lib/e11y/testing/have_tracked_event_matcher.rb +190 -0
- data/lib/e11y/testing/rspec_matchers.rb +21 -0
- data/lib/e11y/testing/snapshot_matcher.rb +86 -0
- data/lib/e11y/trace_context/sampler.rb +35 -0
- data/lib/e11y/tracing/faraday_middleware.rb +31 -0
- data/lib/e11y/tracing/net_http_patch.rb +33 -0
- data/lib/e11y/tracing/propagator.rb +116 -0
- data/lib/e11y/tracing.rb +47 -0
- data/lib/e11y/version.rb +1 -1
- data/lib/e11y/versioning/version_extractor.rb +32 -0
- data/lib/e11y.rb +141 -265
- data/lib/generators/e11y/event/event_generator.rb +22 -0
- data/lib/generators/e11y/event/templates/event.rb.tt +16 -0
- data/lib/generators/e11y/grafana_dashboard/grafana_dashboard_generator.rb +30 -0
- data/lib/generators/e11y/grafana_dashboard/templates/e11y_dashboard.json +81 -0
- data/lib/generators/e11y/install/install_generator.rb +34 -0
- data/lib/generators/e11y/install/templates/e11y.rb +239 -0
- data/lib/generators/e11y/prometheus_alerts/prometheus_alerts_generator.rb +29 -0
- data/lib/generators/e11y/prometheus_alerts/templates/e11y_alerts.yml +28 -0
- data/lib/tasks/e11y_docs.rake +30 -0
- data/lib/tasks/e11y_events.rake +71 -0
- data/lib/tasks/e11y_lint.rake +91 -0
- data/lib/tasks/e11y_slo.rake +29 -0
- metadata +129 -39
- data/docs/ADR-010-developer-experience.md +0 -2166
- data/docs/API-REFERENCE-L28.md +0 -914
- data/docs/COMPREHENSIVE-CONFIGURATION.md +0 -2366
- data/docs/CONTRIBUTING.md +0 -312
- data/docs/IMPLEMENTATION_NOTES.md +0 -2804
- data/docs/IMPLEMENTATION_PLAN.md +0 -1971
- data/docs/IMPLEMENTATION_PLAN_ARCHITECTURE.md +0 -586
- data/docs/PLAN.md +0 -148
- data/docs/README.md +0 -296
- data/docs/design/00-memory-optimization.md +0 -593
- data/docs/guides/MIGRATION-L27-L28.md +0 -692
- data/docs/guides/PERFORMANCE-BENCHMARKS.md +0 -434
- data/docs/guides/README.md +0 -44
- data/docs/use_cases/UC-003-pattern-based-metrics.md +0 -1627
- data/lib/e11y/adapters/registry.rb +0 -141
- /data/docs/{ADR-012-event-evolution.md → architecture/ADR-012-event-evolution.md} +0 -0
|
@@ -65,7 +65,7 @@ end
|
|
|
65
65
|
|
|
66
66
|
## 🎯 Features
|
|
67
67
|
|
|
68
|
-
> **Implementation:** See [ADR-007: OpenTelemetry Integration](../ADR-007-opentelemetry-integration.md) for complete architecture, including [Section 3: OTel Collector Adapter](../ADR-007-opentelemetry-integration.md#3-otel-collector-adapter), [Section 4: Semantic Conventions](../ADR-007-opentelemetry-integration.md#4-semantic-conventions), and [Section 5: Logs Signal Export](../ADR-007-opentelemetry-integration.md#5-logs-signal-export).
|
|
68
|
+
> **Implementation:** See [ADR-007: OpenTelemetry Integration](../architecture/ADR-007-opentelemetry-integration.md) for complete architecture, including [Section 3: OTel Collector Adapter](../architecture/ADR-007-opentelemetry-integration.md#3-otel-collector-adapter), [Section 4: Semantic Conventions](../architecture/ADR-007-opentelemetry-integration.md#4-semantic-conventions), and [Section 5: Logs Signal Export](../architecture/ADR-007-opentelemetry-integration.md#5-logs-signal-export).
|
|
69
69
|
|
|
70
70
|
### 1. OpenTelemetry Collector Adapter
|
|
71
71
|
|
|
@@ -73,28 +73,11 @@ end
|
|
|
73
73
|
```ruby
|
|
74
74
|
# config/initializers/e11y.rb
|
|
75
75
|
E11y.configure do |config|
|
|
76
|
-
config.adapters
|
|
76
|
+
config.adapters[:otel] = E11y::Adapters::OpenTelemetryCollector.new(
|
|
77
77
|
endpoint: ENV['OTEL_EXPORTER_OTLP_ENDPOINT'] || 'http://localhost:4318',
|
|
78
|
-
|
|
79
|
-
headers: {
|
|
80
|
-
|
|
81
|
-
},
|
|
82
|
-
|
|
83
|
-
# Signal types
|
|
84
|
-
export_logs: true, # E11y events → OTel Logs Signal
|
|
85
|
-
export_traces: true, # Spans from events → OTel Traces
|
|
86
|
-
export_metrics: false, # Use Yabeda for metrics (better)
|
|
87
|
-
|
|
88
|
-
# Batching
|
|
89
|
-
batch_size: 100,
|
|
90
|
-
flush_interval: 10.seconds,
|
|
91
|
-
|
|
92
|
-
# Compression
|
|
93
|
-
compression: :gzip,
|
|
94
|
-
|
|
95
|
-
# Retry
|
|
96
|
-
retry_enabled: true,
|
|
97
|
-
max_retries: 3
|
|
78
|
+
service_name: 'my-app',
|
|
79
|
+
headers: { 'X-API-Key' => ENV['OTEL_API_KEY'] },
|
|
80
|
+
compress: true # default, gzip on HTTP body
|
|
98
81
|
)
|
|
99
82
|
end
|
|
100
83
|
|
|
@@ -103,14 +86,14 @@ end
|
|
|
103
86
|
# ├─→ Jaeger (traces)
|
|
104
87
|
# ├─→ Loki (logs)
|
|
105
88
|
# ├─→ Prometheus (metrics)
|
|
106
|
-
# └─→
|
|
89
|
+
# └─→ Object storage (archive, via OTel exporter)
|
|
107
90
|
```
|
|
108
91
|
|
|
109
92
|
---
|
|
110
93
|
|
|
111
94
|
### 2. Semantic Conventions Mapping
|
|
112
95
|
|
|
113
|
-
> **Implementation:** See [ADR-007 Section 4: Semantic Conventions](../ADR-007-opentelemetry-integration.md#4-semantic-conventions) for automatic field mapping across HTTP, DB, RPC, Messaging, and Exception patterns.
|
|
96
|
+
> **Implementation:** See [ADR-007 Section 4: Semantic Conventions](../architecture/ADR-007-opentelemetry-integration.md#4-semantic-conventions) for automatic field mapping across HTTP, DB, RPC, Messaging, and Exception patterns.
|
|
114
97
|
|
|
115
98
|
**Automatic field mapping to OTel standards:**
|
|
116
99
|
```ruby
|
|
@@ -171,7 +154,7 @@ Events::HttpRequest.track(
|
|
|
171
154
|
|
|
172
155
|
### 3. Automatic Span Creation
|
|
173
156
|
|
|
174
|
-
> **Implementation:** See [ADR-007 Section 6: Traces Signal Export](../ADR-007-opentelemetry-integration.md#6-traces-signal-export) for automatic span creation rules and parent-child relationships.
|
|
157
|
+
> **Implementation:** See [ADR-007 Section 6: Traces Signal Export](../architecture/ADR-007-opentelemetry-integration.md#6-traces-signal-export) for automatic span creation rules and parent-child relationships.
|
|
175
158
|
|
|
176
159
|
**Create spans from E11y events:**
|
|
177
160
|
```ruby
|
|
@@ -207,7 +190,7 @@ Events::OrderProcessingStarted.track(
|
|
|
207
190
|
|
|
208
191
|
### 4. W3C Trace Context Integration
|
|
209
192
|
|
|
210
|
-
> **Implementation:** See [ADR-007 Section 8: Trace Context Integration](../ADR-007-opentelemetry-integration.md#8-trace-context-integration) for OTel SDK as primary trace context source.
|
|
193
|
+
> **Implementation:** See [ADR-007 Section 8: Trace Context Integration](../architecture/ADR-007-opentelemetry-integration.md#8-trace-context-integration) for OTel SDK as primary trace context source.
|
|
211
194
|
|
|
212
195
|
**Automatic trace context from OpenTelemetry SDK:**
|
|
213
196
|
```ruby
|
|
@@ -240,7 +223,7 @@ Events::OrderCreated.track(order_id: '123')
|
|
|
240
223
|
|
|
241
224
|
### 5. OTel Logs Signal Export
|
|
242
225
|
|
|
243
|
-
> **Implementation:** See [ADR-007 Section 5: Logs Signal Export](../ADR-007-opentelemetry-integration.md#5-logs-signal-export) for OTLP JSON format and trace correlation details.
|
|
226
|
+
> **Implementation:** See [ADR-007 Section 5: Logs Signal Export](../architecture/ADR-007-opentelemetry-integration.md#5-logs-signal-export) for OTLP JSON format and trace correlation details.
|
|
244
227
|
|
|
245
228
|
**Export E11y events as OpenTelemetry Logs:**
|
|
246
229
|
```ruby
|
|
@@ -297,7 +280,7 @@ Events::OrderCreated.track(order_id: '123')
|
|
|
297
280
|
### 6. Baggage PII Protection (C08 Resolution) ⚠️ CRITICAL
|
|
298
281
|
|
|
299
282
|
> **⚠️ CRITICAL: C08 Conflict Resolution - PII Leaking via OpenTelemetry Baggage**
|
|
300
|
-
> **See:** [ADR-006 Section 5.5](../ADR-006-security-compliance.md#55-opentelemetry-baggage-pii-protection-c08-resolution--critical) for detailed architecture and GDPR compliance rationale.
|
|
283
|
+
> **See:** [ADR-006 Section 5.5](../architecture/ADR-006-security-compliance.md#55-opentelemetry-baggage-pii-protection-c08-resolution--critical) for detailed architecture and GDPR compliance rationale.
|
|
301
284
|
> **Problem:** OpenTelemetry Baggage propagates data via HTTP headers (`baggage: key1=value1,key2=value2`), bypassing E11y's PII filtering. If a developer accidentally adds PII to baggage, it leaks across all services.
|
|
302
285
|
> **Solution:** Block ALL baggage keys by default, allow ONLY safe keys via allowlist.
|
|
303
286
|
|
|
@@ -330,33 +313,23 @@ E11y blocks ALL baggage keys by default, allowing ONLY safe keys (no PII):
|
|
|
330
313
|
```ruby
|
|
331
314
|
# config/initializers/e11y.rb
|
|
332
315
|
E11y.configure do |config|
|
|
333
|
-
config.
|
|
334
|
-
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
|
|
343
|
-
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
block_mode :silent # Options: :silent (log), :warn (log+warn), :raise (exception)
|
|
351
|
-
|
|
352
|
-
# Monitoring: Track violations
|
|
353
|
-
on_blocked_key do |key, value, caller_location|
|
|
354
|
-
Yabeda.e11y_baggage_pii_blocked.increment(
|
|
355
|
-
key: key,
|
|
356
|
-
service: ENV['SERVICE_NAME']
|
|
357
|
-
)
|
|
358
|
-
end
|
|
359
|
-
end
|
|
316
|
+
config.security_baggage_protection_enabled = true # ✅ CRITICAL: Always enable in production
|
|
317
|
+
|
|
318
|
+
# Allowlist: ONLY these keys are safe
|
|
319
|
+
config.security_baggage_protection_allowed_keys = [
|
|
320
|
+
'trace_id', # ✅ Safe: Correlation ID
|
|
321
|
+
'span_id', # ✅ Safe: Trace context
|
|
322
|
+
'environment', # ✅ Safe: Deployment context
|
|
323
|
+
'version', # ✅ Safe: Service version
|
|
324
|
+
'service_name', # ✅ Safe: Service identifier
|
|
325
|
+
'request_id', # ✅ Safe: Request identifier
|
|
326
|
+
# Custom safe keys (no PII!):
|
|
327
|
+
'feature_flag_id', # ✅ Safe: Feature flag name
|
|
328
|
+
'ab_test_variant' # ✅ Safe: A/B test group
|
|
329
|
+
]
|
|
330
|
+
|
|
331
|
+
# Block mode: What happens when PII detected?
|
|
332
|
+
config.security_baggage_protection_block_mode = :silent # Options: :silent (log), :warn (log+warn), :raise (exception)
|
|
360
333
|
end
|
|
361
334
|
```
|
|
362
335
|
|
|
@@ -421,11 +394,9 @@ Fail fast in non-production environments:
|
|
|
421
394
|
```ruby
|
|
422
395
|
# config/environments/development.rb
|
|
423
396
|
E11y.configure do |config|
|
|
424
|
-
config.
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
allowed_keys E11y::Middleware::BaggageProtection::ALLOWED_KEYS
|
|
428
|
-
end
|
|
397
|
+
config.security_baggage_protection_enabled = true
|
|
398
|
+
config.security_baggage_protection_block_mode = :raise # ← RAISE exception on blocked keys (fail fast)
|
|
399
|
+
config.security_baggage_protection_allowed_keys = E11y::BAGGAGE_PROTECTION_DEFAULT_ALLOWED_KEYS
|
|
429
400
|
end
|
|
430
401
|
|
|
431
402
|
# Developer tries to set PII:
|
|
@@ -531,18 +502,14 @@ exporters:
|
|
|
531
502
|
prometheus:
|
|
532
503
|
endpoint: 0.0.0.0:8889
|
|
533
504
|
|
|
534
|
-
# Archive
|
|
535
|
-
s3:
|
|
536
|
-
region: us-east-1
|
|
537
|
-
bucket: telemetry-archive
|
|
538
|
-
prefix: logs/
|
|
505
|
+
# Archive: OTel Collector can export to object storage (add exporter config as needed)
|
|
539
506
|
|
|
540
507
|
service:
|
|
541
508
|
pipelines:
|
|
542
509
|
logs:
|
|
543
510
|
receivers: [otlp]
|
|
544
511
|
processors: [batch, resource, filter]
|
|
545
|
-
exporters: [loki
|
|
512
|
+
exporters: [loki]
|
|
546
513
|
|
|
547
514
|
traces:
|
|
548
515
|
receivers: [otlp]
|
|
@@ -558,11 +525,9 @@ service:
|
|
|
558
525
|
```ruby
|
|
559
526
|
# config/initializers/e11y.rb
|
|
560
527
|
E11y.configure do |config|
|
|
561
|
-
config.adapters
|
|
528
|
+
config.adapters[:otel] = E11y::Adapters::OpenTelemetryCollector.new(
|
|
562
529
|
endpoint: 'http://otel-collector:4318',
|
|
563
|
-
|
|
564
|
-
export_logs: true,
|
|
565
|
-
export_traces: true
|
|
530
|
+
service_name: 'my-app'
|
|
566
531
|
)
|
|
567
532
|
end
|
|
568
533
|
|
|
@@ -742,14 +707,11 @@ end
|
|
|
742
707
|
# config/initializers/e11y.rb
|
|
743
708
|
E11y.configure do |config|
|
|
744
709
|
# Single adapter: OTel Collector
|
|
745
|
-
config.adapters =
|
|
746
|
-
|
|
747
|
-
|
|
748
|
-
|
|
749
|
-
|
|
750
|
-
)
|
|
751
|
-
]
|
|
752
|
-
|
|
710
|
+
config.adapters[:otel] = E11y::Adapters::OpenTelemetryCollector.new(
|
|
711
|
+
endpoint: 'http://otel-collector:4318',
|
|
712
|
+
service_name: 'my-app'
|
|
713
|
+
)
|
|
714
|
+
|
|
753
715
|
# OTel Collector handles routing to multiple backends!
|
|
754
716
|
# No need for multiple E11y adapters
|
|
755
717
|
end
|
|
@@ -757,7 +719,7 @@ end
|
|
|
757
719
|
# OTel Collector routes to:
|
|
758
720
|
# - Loki (logs, last 30 days)
|
|
759
721
|
# - Jaeger (traces, last 7 days)
|
|
760
|
-
# -
|
|
722
|
+
# - Object storage (archive, long-term; OTel exporter)
|
|
761
723
|
# - Prometheus (metrics via remote write)
|
|
762
724
|
|
|
763
725
|
# Benefits:
|
|
@@ -931,7 +893,7 @@ end
|
|
|
931
893
|
┌─────────────┐ │ ├─→ Loki (logs)
|
|
932
894
|
│ Sidekiq │─┘ ├─→ Jaeger (traces)
|
|
933
895
|
└─────────────┘ ├─→ Prometheus (metrics)
|
|
934
|
-
├─→
|
|
896
|
+
├─→ Object storage (archive)
|
|
935
897
|
└─→ Datadog (optional)
|
|
936
898
|
```
|
|
937
899
|
|
|
@@ -1048,11 +1010,10 @@ end
|
|
|
1048
1010
|
**1. Use OTel Collector in production**
|
|
1049
1011
|
```ruby
|
|
1050
1012
|
# ✅ GOOD: Central pipeline
|
|
1051
|
-
config.adapters =
|
|
1052
|
-
|
|
1053
|
-
|
|
1054
|
-
|
|
1055
|
-
]
|
|
1013
|
+
config.adapters[:otel] = E11y::Adapters::OpenTelemetryCollector.new(
|
|
1014
|
+
endpoint: 'http://otel-collector:4318',
|
|
1015
|
+
service_name: 'my-app'
|
|
1016
|
+
)
|
|
1056
1017
|
|
|
1057
1018
|
# OTel Collector handles:
|
|
1058
1019
|
# - Sampling
|
|
@@ -1094,13 +1055,14 @@ end
|
|
|
1094
1055
|
config.adapters = [
|
|
1095
1056
|
E11y::Adapters::JaegerAdapter.new(...),
|
|
1096
1057
|
E11y::Adapters::LokiAdapter.new(...),
|
|
1097
|
-
E11y::Adapters::
|
|
1058
|
+
E11y::Adapters::FileAdapter.new(...) # Direct to file (bypasses OTel routing)
|
|
1098
1059
|
]
|
|
1099
1060
|
|
|
1100
1061
|
# ✅ GOOD: Through OTel Collector
|
|
1101
|
-
config.adapters =
|
|
1102
|
-
|
|
1103
|
-
|
|
1062
|
+
config.adapters[:otel] = E11y::Adapters::OpenTelemetryCollector.new(
|
|
1063
|
+
endpoint: ENV['OTEL_EXPORTER_OTLP_ENDPOINT'],
|
|
1064
|
+
service_name: 'my-app'
|
|
1065
|
+
)
|
|
1104
1066
|
```
|
|
1105
1067
|
|
|
1106
1068
|
**2. Don't use custom field names**
|
|
@@ -60,7 +60,7 @@ Events::OrderShipping.track(order_id: '789', tracking: 'TRACK123')
|
|
|
60
60
|
|
|
61
61
|
## 🎯 Features
|
|
62
62
|
|
|
63
|
-
> **Implementation:** See [ADR-005: Tracing Context](../ADR-005-tracing-context.md) for complete architecture, including [Section 5: W3C Trace Context](../ADR-005-tracing-context.md#5-w3c-trace-context), [Section 6.1: HTTP Propagator](../ADR-005-tracing-context.md#61-http-propagator-outgoing-requests), and [Section 8: Context Inheritance](../ADR-005-tracing-context.md#8-context-inheritance-thread-fiber-support).
|
|
63
|
+
> **Implementation:** See [ADR-005: Tracing Context](../architecture/ADR-005-tracing-context.md) for complete architecture, including [Section 5: W3C Trace Context](../architecture/ADR-005-tracing-context.md#5-w3c-trace-context), [Section 6.1: HTTP Propagator](../architecture/ADR-005-tracing-context.md#61-http-propagator-outgoing-requests), and [Section 8: Context Inheritance](../architecture/ADR-005-tracing-context.md#8-context-inheritance-thread-fiber-support).
|
|
64
64
|
|
|
65
65
|
### 1. Automatic W3C Trace Context Propagation
|
|
66
66
|
|
|
@@ -206,7 +206,7 @@ end
|
|
|
206
206
|
|
|
207
207
|
### 3. Background Job Trace Propagation (C17 Resolution) ⚠️
|
|
208
208
|
|
|
209
|
-
> **Implementation:** See [ADR-005 Section 8.3: Background Job Tracing Strategy](../ADR-005-tracing-context.md#83-background-job-tracing-strategy-c17-resolution) for the hybrid tracing model.
|
|
209
|
+
> **Implementation:** See [ADR-005 Section 8.3: Background Job Tracing Strategy](../architecture/ADR-005-tracing-context.md#83-background-job-tracing-strategy-c17-resolution) for the hybrid tracing model.
|
|
210
210
|
|
|
211
211
|
**Hybrid Tracing Model (New Trace + Parent Link):**
|
|
212
212
|
```ruby
|
|
@@ -279,7 +279,7 @@ end
|
|
|
279
279
|
3. **Trace clarity:** Separate timelines for sync (request) vs async (job) operations
|
|
280
280
|
4. **Link preserved:** `parent_trace_id` allows reconstructing full flow
|
|
281
281
|
|
|
282
|
-
See [ADR-005 §8.3](../ADR-005-tracing-context.md#83-background-job-tracing-strategy-c17-resolution) for detailed rationale.
|
|
282
|
+
See [ADR-005 §8.3](../architecture/ADR-005-tracing-context.md#83-background-job-tracing-strategy-c17-resolution) for detailed rationale.
|
|
283
283
|
|
|
284
284
|
**Visual Diagram: Request Trace → Job Trace (with parent link)**
|
|
285
285
|
|
|
@@ -396,7 +396,7 @@ Events::PaymentReceived.track(
|
|
|
396
396
|
|
|
397
397
|
### 5. Service Mesh Integration
|
|
398
398
|
|
|
399
|
-
> **Implementation:** See [ADR-005 Section 5.3: HTTP Header Extraction](../ADR-005-tracing-context.md#53-http-header-extraction-w3c-legacy-headers) for W3C standard header support enabling service mesh compatibility.
|
|
399
|
+
> **Implementation:** See [ADR-005 Section 5.3: HTTP Header Extraction](../architecture/ADR-005-tracing-context.md#53-http-header-extraction-w3c-legacy-headers) for W3C standard header support enabling service mesh compatibility.
|
|
400
400
|
|
|
401
401
|
**Automatic integration with Istio/Linkerd:**
|
|
402
402
|
```ruby
|
|
@@ -72,7 +72,7 @@ end
|
|
|
72
72
|
|
|
73
73
|
## 🎯 Features
|
|
74
74
|
|
|
75
|
-
> **Implementation:** See [ADR-008: Rails Integration](../ADR-008-rails-integration.md) for complete architecture, including [Section 5: Sidekiq Integration](../ADR-008-rails-integration.md#5-sidekiq-integration), [Section 6: ActiveJob Integration](../ADR-008-rails-integration.md#6-activejob-integration), and [Section 5.3: Job-Scoped Buffer](../ADR-008-rails-integration.md#53-job-scoped-buffer).
|
|
75
|
+
> **Implementation:** See [ADR-008: Rails Integration](../architecture/ADR-008-rails-integration.md) for complete architecture, including [Section 5: Sidekiq Integration](../architecture/ADR-008-rails-integration.md#5-sidekiq-integration), [Section 6: ActiveJob Integration](../architecture/ADR-008-rails-integration.md#6-activejob-integration), and [Section 5.3: Job-Scoped Buffer](../architecture/ADR-008-rails-integration.md#53-job-scoped-buffer).
|
|
76
76
|
|
|
77
77
|
### 1. Automatic Instrumentation
|
|
78
78
|
|
|
@@ -204,7 +204,7 @@ end
|
|
|
204
204
|
|
|
205
205
|
### 5. Job-Specific Events
|
|
206
206
|
|
|
207
|
-
> **Note:** E11y supports **job-scoped buffering** similar to [UC-001: Request-Scoped Debug Buffering](./UC-001-request-scoped-debug-buffering.md). Debug events within a job are buffered and only flushed if the job fails. See [ADR-001 Section 3.4: Request-Scoped Buffer](../ADR-001-architecture.md#34-request-scoped-buffer) for implementation details (same architecture applies to jobs).
|
|
207
|
+
> **Note:** E11y supports **job-scoped buffering** similar to [UC-001: Request-Scoped Debug Buffering](./UC-001-request-scoped-debug-buffering.md). Debug events within a job are buffered and only flushed if the job fails. See [ADR-001 Section 3.4: Request-Scoped Buffer](../architecture/ADR-001-architecture.md#34-request-scoped-buffer) for implementation details (same architecture applies to jobs).
|
|
208
208
|
|
|
209
209
|
**Emit custom events within jobs:**
|
|
210
210
|
```ruby
|
|
@@ -506,7 +506,7 @@ end
|
|
|
506
506
|
|
|
507
507
|
### 6. Sidekiq Middleware Implementation (C17, C18 Resolutions) ⚠️
|
|
508
508
|
|
|
509
|
-
> **Reference:** See [ADR-005 §8.3 (C17)](../ADR-005-tracing-context.md#83-background-job-tracing-strategy-c17-resolution) and [ADR-013 §3.6 (C18)](../ADR-013-reliability-error-handling.md#36-event-tracking-in-background-jobs-c18-resolution) for full architecture.
|
|
509
|
+
> **Reference:** See [ADR-005 §8.3 (C17)](../architecture/ADR-005-tracing-context.md#83-background-job-tracing-strategy-c17-resolution) and [ADR-013 §3.6 (C18)](../architecture/ADR-013-reliability-error-handling.md#36-event-tracking-in-background-jobs-c18-resolution) for full architecture.
|
|
510
510
|
|
|
511
511
|
E11y provides two critical Sidekiq middlewares:
|
|
512
512
|
|
|
@@ -697,7 +697,7 @@ end
|
|
|
697
697
|
|
|
698
698
|
```ruby
|
|
699
699
|
# ❌ BAD: E11y failure fails the job
|
|
700
|
-
config.
|
|
700
|
+
config.error_handling_fail_on_error_in_jobs = true
|
|
701
701
|
|
|
702
702
|
# Job fails if Loki is down:
|
|
703
703
|
# - Payment was created successfully
|
|
@@ -706,7 +706,7 @@ config.error_handling.fail_on_error_in_jobs = true
|
|
|
706
706
|
# - Result: Duplicate payments! 💸💸💸
|
|
707
707
|
|
|
708
708
|
# ✅ GOOD: E11y failure doesn't fail the job
|
|
709
|
-
config.
|
|
709
|
+
config.error_handling_fail_on_error_in_jobs = false
|
|
710
710
|
|
|
711
711
|
# Job succeeds even if Loki is down:
|
|
712
712
|
# - Payment created ✅
|