e11y 0.2.0 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (230) hide show
  1. checksums.yaml +4 -4
  2. data/.rubocop.yml +130 -10
  3. data/CHANGELOG.md +56 -1
  4. data/CLAUDE.md +168 -0
  5. data/CONTRIBUTING.md +640 -0
  6. data/README.md +134 -702
  7. data/RELEASE.md +18 -3
  8. data/Rakefile +108 -29
  9. data/config/README.md +1 -1
  10. data/config/loki-local-config.yaml +12 -0
  11. data/config/otel-collector-config.yaml +44 -0
  12. data/cucumber.yml +1 -0
  13. data/docker-compose.yml +18 -2
  14. data/docs/ADAPTERS.md +76 -0
  15. data/docs/ADAPTIVE_SAMPLING.md +59 -0
  16. data/docs/COMPARISON.md +104 -0
  17. data/docs/CONFIGURATION.md +52 -0
  18. data/docs/DISTRIBUTED_TRACING.md +44 -0
  19. data/docs/LIMITATIONS.md +13 -0
  20. data/docs/METRICS_DSL.md +84 -0
  21. data/docs/PERFORMANCE.md +60 -0
  22. data/docs/PII_FILTERING.md +40 -0
  23. data/docs/PRESETS.md +65 -0
  24. data/docs/QUICK-START.md +546 -587
  25. data/docs/RAILS_INTEGRATION.md +29 -0
  26. data/docs/SCHEMA_VALIDATION.md +63 -0
  27. data/docs/SLO-PROMQL-ALERTS.md +161 -0
  28. data/docs/TESTING.md +69 -0
  29. data/docs/{ADR-001-architecture.md → architecture/ADR-001-architecture.md} +35 -64
  30. data/docs/{ADR-002-metrics-yabeda.md → architecture/ADR-002-metrics-yabeda.md} +62 -236
  31. data/docs/{ADR-003-slo-observability.md → architecture/ADR-003-slo-observability.md} +27 -466
  32. data/docs/{ADR-004-adapter-architecture.md → architecture/ADR-004-adapter-architecture.md} +163 -146
  33. data/docs/{ADR-005-tracing-context.md → architecture/ADR-005-tracing-context.md} +10 -9
  34. data/docs/{ADR-006-security-compliance.md → architecture/ADR-006-security-compliance.md} +184 -191
  35. data/docs/{ADR-007-opentelemetry-integration.md → architecture/ADR-007-opentelemetry-integration.md} +3 -21
  36. data/docs/{ADR-008-rails-integration.md → architecture/ADR-008-rails-integration.md} +209 -339
  37. data/docs/{ADR-009-cost-optimization.md → architecture/ADR-009-cost-optimization.md} +45 -54
  38. data/docs/architecture/ADR-010-developer-experience.md +522 -0
  39. data/docs/{ADR-011-testing-strategy.md → architecture/ADR-011-testing-strategy.md} +41 -83
  40. data/docs/{ADR-013-reliability-error-handling.md → architecture/ADR-013-reliability-error-handling.md} +37 -12
  41. data/docs/{ADR-014-event-driven-slo.md → architecture/ADR-014-event-driven-slo.md} +12 -24
  42. data/docs/{ADR-015-middleware-order.md → architecture/ADR-015-middleware-order.md} +23 -41
  43. data/docs/{ADR-016-self-monitoring-slo.md → architecture/ADR-016-self-monitoring-slo.md} +52 -349
  44. data/docs/{ADR-017-multi-rails-compatibility.md → architecture/ADR-017-multi-rails-compatibility.md} +4 -11
  45. data/docs/architecture/ADR-018-memory-optimization.md +366 -0
  46. data/docs/{ADR-INDEX.md → architecture/ADR-INDEX.md} +11 -6
  47. data/docs/{00-ICP-AND-TIMELINE.md → prd/00-ICP-AND-TIMELINE.md} +6 -6
  48. data/docs/{01-SCALE-REQUIREMENTS.md → prd/01-SCALE-REQUIREMENTS.md} +6 -6
  49. data/docs/prd/01-overview-vision.md +19 -14
  50. data/docs/use_cases/README.md +22 -23
  51. data/docs/use_cases/UC-001-request-scoped-debug-buffering.md +50 -44
  52. data/docs/use_cases/UC-002-business-event-tracking.md +26 -95
  53. data/docs/use_cases/UC-003-event-metrics.md +66 -0
  54. data/docs/use_cases/UC-004-zero-config-slo-tracking.md +42 -101
  55. data/docs/use_cases/UC-005-sentry-integration.md +13 -15
  56. data/docs/use_cases/UC-006-trace-context-management.md +30 -28
  57. data/docs/use_cases/UC-007-pii-filtering.md +35 -87
  58. data/docs/use_cases/UC-008-opentelemetry-integration.md +51 -89
  59. data/docs/use_cases/UC-009-multi-service-tracing.md +4 -4
  60. data/docs/use_cases/UC-010-background-job-tracking.md +5 -5
  61. data/docs/use_cases/UC-011-rate-limiting.md +95 -168
  62. data/docs/use_cases/UC-012-audit-trail.md +21 -46
  63. data/docs/use_cases/UC-013-high-cardinality-protection.md +29 -167
  64. data/docs/use_cases/UC-014-adaptive-sampling.md +2 -2
  65. data/docs/use_cases/UC-015-cost-optimization.md +46 -99
  66. data/docs/use_cases/UC-016-rails-logger-migration.md +39 -213
  67. data/docs/use_cases/UC-017-local-development.md +203 -777
  68. data/docs/use_cases/UC-018-testing-events.md +3 -3
  69. data/docs/use_cases/UC-019-retention-based-routing.md +53 -106
  70. data/docs/use_cases/UC-020-event-versioning.md +8 -9
  71. data/docs/use_cases/UC-021-error-handling-retry-dlq.md +18 -22
  72. data/docs/use_cases/UC-022-event-registry.md +15 -21
  73. data/docs/use_cases/backlog.md +119 -87
  74. data/e11y.gemspec +2 -2
  75. data/gems/e11y-devtools/README.md +136 -0
  76. data/gems/e11y-devtools/config/routes.rb +8 -0
  77. data/gems/e11y-devtools/e11y-devtools.gemspec +25 -0
  78. data/gems/e11y-devtools/exe/e11y +34 -0
  79. data/gems/e11y-devtools/lib/e11y/devtools/mcp/server.rb +96 -0
  80. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tool_base.rb +25 -0
  81. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/clear.rb +31 -0
  82. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/errors.rb +35 -0
  83. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/event_detail.rb +33 -0
  84. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/events_by_trace.rb +33 -0
  85. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/interactions.rb +40 -0
  86. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/recent_events.rb +34 -0
  87. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/search.rb +34 -0
  88. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/stats.rb +30 -0
  89. data/gems/e11y-devtools/lib/e11y/devtools/overlay/assets/overlay.js +115 -0
  90. data/gems/e11y-devtools/lib/e11y/devtools/overlay/controller.rb +54 -0
  91. data/gems/e11y-devtools/lib/e11y/devtools/overlay/engine.rb +26 -0
  92. data/gems/e11y-devtools/lib/e11y/devtools/overlay/middleware.rb +80 -0
  93. data/gems/e11y-devtools/lib/e11y/devtools/overlay/rails_controller.rb +42 -0
  94. data/gems/e11y-devtools/lib/e11y/devtools/tui/app.rb +262 -0
  95. data/gems/e11y-devtools/lib/e11y/devtools/tui/grouping.rb +66 -0
  96. data/gems/e11y-devtools/lib/e11y/devtools/tui/widgets/event_detail.rb +62 -0
  97. data/gems/e11y-devtools/lib/e11y/devtools/tui/widgets/event_list.rb +70 -0
  98. data/gems/e11y-devtools/lib/e11y/devtools/tui/widgets/interaction_list.rb +47 -0
  99. data/gems/e11y-devtools/lib/e11y/devtools/version.rb +8 -0
  100. data/gems/e11y-devtools/lib/e11y/devtools.rb +13 -0
  101. data/gems/e11y-devtools/spec/e11y/devtools/mcp/tools_spec.rb +107 -0
  102. data/gems/e11y-devtools/spec/e11y/devtools/overlay/controller_spec.rb +58 -0
  103. data/gems/e11y-devtools/spec/e11y/devtools/overlay/middleware_spec.rb +46 -0
  104. data/gems/e11y-devtools/spec/e11y/devtools/tui/app_spec.rb +85 -0
  105. data/gems/e11y-devtools/spec/e11y/devtools/tui/grouping_spec.rb +64 -0
  106. data/gems/e11y-devtools/spec/spec_helper.rb +5 -0
  107. data/gems/e11y-devtools/spec/tui/widgets/event_list_spec.rb +44 -0
  108. data/gems/e11y-devtools/spec/tui/widgets/interaction_list_spec.rb +62 -0
  109. data/lib/e11y/adapters/audit_encrypted.rb +53 -11
  110. data/lib/e11y/adapters/base.rb +33 -34
  111. data/lib/e11y/adapters/dev_log/file_store.rb +143 -0
  112. data/lib/e11y/adapters/dev_log/query.rb +219 -0
  113. data/lib/e11y/adapters/dev_log.rb +118 -0
  114. data/lib/e11y/adapters/file.rb +3 -6
  115. data/lib/e11y/adapters/in_memory.rb +52 -5
  116. data/lib/e11y/adapters/in_memory_test.rb +29 -0
  117. data/lib/e11y/adapters/loki.rb +58 -23
  118. data/lib/e11y/adapters/null.rb +82 -0
  119. data/lib/e11y/adapters/opentelemetry_collector.rb +183 -0
  120. data/lib/e11y/adapters/otel_logs.rb +136 -23
  121. data/lib/e11y/adapters/sentry.rb +4 -7
  122. data/lib/e11y/adapters/stdout.rb +73 -7
  123. data/lib/e11y/adapters/yabeda.rb +153 -29
  124. data/lib/e11y/buffers/adaptive_buffer.rb +3 -17
  125. data/lib/e11y/buffers/{request_scoped_buffer.rb → ephemeral_buffer.rb} +72 -58
  126. data/lib/e11y/buffers/ring_buffer.rb +3 -16
  127. data/lib/e11y/configuration.rb +272 -0
  128. data/lib/e11y/console.rb +10 -17
  129. data/lib/e11y/current.rb +53 -1
  130. data/lib/e11y/debug/pipeline_inspector.rb +96 -0
  131. data/lib/e11y/documentation/generator.rb +48 -0
  132. data/lib/e11y/event/base.rb +176 -82
  133. data/lib/e11y/event/value_sampling_config.rb +1 -5
  134. data/lib/e11y/events/rails/database/query.rb +1 -4
  135. data/lib/e11y/events/rails/job/failed.rb +2 -0
  136. data/lib/e11y/instruments/active_job.rb +46 -12
  137. data/lib/e11y/instruments/rails_instrumentation.rb +49 -24
  138. data/lib/e11y/instruments/sidekiq.rb +137 -31
  139. data/lib/e11y/linters/base.rb +11 -0
  140. data/lib/e11y/linters/pii/pii_declaration_linter.rb +120 -0
  141. data/lib/e11y/linters/slo/config_consistency_linter.rb +76 -0
  142. data/lib/e11y/linters/slo/explicit_declaration_linter.rb +36 -0
  143. data/lib/e11y/linters/slo/slo_status_from_linter.rb +41 -0
  144. data/lib/e11y/logger/bridge.rb +26 -7
  145. data/lib/e11y/metrics/cardinality_protection.rb +10 -15
  146. data/lib/e11y/metrics/cardinality_tracker.rb +16 -6
  147. data/lib/e11y/metrics/registry.rb +3 -5
  148. data/lib/e11y/metrics/test_backend.rb +62 -0
  149. data/lib/e11y/metrics.rb +56 -10
  150. data/lib/e11y/middleware/adapter_resolver.rb +40 -0
  151. data/lib/e11y/middleware/audit_signing.rb +43 -6
  152. data/lib/e11y/middleware/baggage_protection.rb +75 -0
  153. data/lib/e11y/middleware/dev_log_source.rb +24 -0
  154. data/lib/e11y/middleware/event_slo.rb +23 -9
  155. data/lib/e11y/middleware/otel_span.rb +23 -0
  156. data/lib/e11y/middleware/pii_filter.rb +104 -75
  157. data/lib/e11y/middleware/rate_limiting.rb +54 -27
  158. data/lib/e11y/middleware/request.rb +70 -23
  159. data/lib/e11y/middleware/routing.rb +78 -21
  160. data/lib/e11y/middleware/sampling.rb +66 -17
  161. data/lib/e11y/middleware/self_monitoring_emit.rb +39 -0
  162. data/lib/e11y/middleware/trace_context.rb +45 -10
  163. data/lib/e11y/middleware/track_latency.rb +34 -0
  164. data/lib/e11y/middleware/validation.rb +7 -16
  165. data/lib/e11y/middleware/versioning.rb +26 -22
  166. data/lib/e11y/opentelemetry/semantic_conventions.rb +109 -0
  167. data/lib/e11y/opentelemetry/span_creator.rb +142 -0
  168. data/lib/e11y/pii/patterns.rb +12 -1
  169. data/lib/e11y/pipeline/builder.rb +1 -1
  170. data/lib/e11y/presets/audit_event.rb +13 -2
  171. data/lib/e11y/railtie.rb +52 -15
  172. data/lib/e11y/registry.rb +306 -0
  173. data/lib/e11y/reliability/circuit_breaker.rb +19 -21
  174. data/lib/e11y/reliability/dlq/base.rb +71 -0
  175. data/lib/e11y/reliability/dlq/file_adapter.rb +301 -0
  176. data/lib/e11y/reliability/dlq/file_storage.rb +63 -34
  177. data/lib/e11y/reliability/dlq/filter.rb +37 -54
  178. data/lib/e11y/reliability/retry_handler.rb +26 -29
  179. data/lib/e11y/reliability/retry_rate_limiter.rb +3 -11
  180. data/lib/e11y/sampling/error_spike_detector.rb +0 -2
  181. data/lib/e11y/sampling/load_monitor.rb +5 -9
  182. data/lib/e11y/sampling/stratified_tracker.rb +18 -0
  183. data/lib/e11y/self_monitoring/buffer_monitor.rb +2 -0
  184. data/lib/e11y/self_monitoring/performance_monitor.rb +19 -61
  185. data/lib/e11y/self_monitoring/reliability_monitor.rb +4 -74
  186. data/lib/e11y/slo/config_loader.rb +40 -0
  187. data/lib/e11y/slo/config_validator.rb +58 -0
  188. data/lib/e11y/slo/dashboard_generator.rb +122 -0
  189. data/lib/e11y/slo/event_driven.rb +8 -0
  190. data/lib/e11y/slo/tracker.rb +31 -4
  191. data/lib/e11y/testing/have_tracked_event_matcher.rb +190 -0
  192. data/lib/e11y/testing/rspec_matchers.rb +21 -0
  193. data/lib/e11y/testing/snapshot_matcher.rb +86 -0
  194. data/lib/e11y/trace_context/sampler.rb +35 -0
  195. data/lib/e11y/tracing/faraday_middleware.rb +31 -0
  196. data/lib/e11y/tracing/net_http_patch.rb +33 -0
  197. data/lib/e11y/tracing/propagator.rb +116 -0
  198. data/lib/e11y/tracing.rb +47 -0
  199. data/lib/e11y/version.rb +1 -1
  200. data/lib/e11y/versioning/version_extractor.rb +32 -0
  201. data/lib/e11y.rb +141 -265
  202. data/lib/generators/e11y/event/event_generator.rb +22 -0
  203. data/lib/generators/e11y/event/templates/event.rb.tt +16 -0
  204. data/lib/generators/e11y/grafana_dashboard/grafana_dashboard_generator.rb +30 -0
  205. data/lib/generators/e11y/grafana_dashboard/templates/e11y_dashboard.json +81 -0
  206. data/lib/generators/e11y/install/install_generator.rb +34 -0
  207. data/lib/generators/e11y/install/templates/e11y.rb +239 -0
  208. data/lib/generators/e11y/prometheus_alerts/prometheus_alerts_generator.rb +29 -0
  209. data/lib/generators/e11y/prometheus_alerts/templates/e11y_alerts.yml +28 -0
  210. data/lib/tasks/e11y_docs.rake +30 -0
  211. data/lib/tasks/e11y_events.rake +71 -0
  212. data/lib/tasks/e11y_lint.rake +91 -0
  213. data/lib/tasks/e11y_slo.rake +29 -0
  214. metadata +129 -39
  215. data/docs/ADR-010-developer-experience.md +0 -2166
  216. data/docs/API-REFERENCE-L28.md +0 -914
  217. data/docs/COMPREHENSIVE-CONFIGURATION.md +0 -2366
  218. data/docs/CONTRIBUTING.md +0 -312
  219. data/docs/IMPLEMENTATION_NOTES.md +0 -2804
  220. data/docs/IMPLEMENTATION_PLAN.md +0 -1971
  221. data/docs/IMPLEMENTATION_PLAN_ARCHITECTURE.md +0 -586
  222. data/docs/PLAN.md +0 -148
  223. data/docs/README.md +0 -296
  224. data/docs/design/00-memory-optimization.md +0 -593
  225. data/docs/guides/MIGRATION-L27-L28.md +0 -692
  226. data/docs/guides/PERFORMANCE-BENCHMARKS.md +0 -434
  227. data/docs/guides/README.md +0 -44
  228. data/docs/use_cases/UC-003-pattern-based-metrics.md +0 -1627
  229. data/lib/e11y/adapters/registry.rb +0 -141
  230. /data/docs/{ADR-012-event-evolution.md → architecture/ADR-012-event-evolution.md} +0 -0
@@ -65,7 +65,7 @@ end
65
65
 
66
66
  ## 🎯 Features
67
67
 
68
- > **Implementation:** See [ADR-007: OpenTelemetry Integration](../ADR-007-opentelemetry-integration.md) for complete architecture, including [Section 3: OTel Collector Adapter](../ADR-007-opentelemetry-integration.md#3-otel-collector-adapter), [Section 4: Semantic Conventions](../ADR-007-opentelemetry-integration.md#4-semantic-conventions), and [Section 5: Logs Signal Export](../ADR-007-opentelemetry-integration.md#5-logs-signal-export).
68
+ > **Implementation:** See [ADR-007: OpenTelemetry Integration](../architecture/ADR-007-opentelemetry-integration.md) for complete architecture, including [Section 3: OTel Collector Adapter](../architecture/ADR-007-opentelemetry-integration.md#3-otel-collector-adapter), [Section 4: Semantic Conventions](../architecture/ADR-007-opentelemetry-integration.md#4-semantic-conventions), and [Section 5: Logs Signal Export](../architecture/ADR-007-opentelemetry-integration.md#5-logs-signal-export).
69
69
 
70
70
  ### 1. OpenTelemetry Collector Adapter
71
71
 
@@ -73,28 +73,11 @@ end
73
73
  ```ruby
74
74
  # config/initializers/e11y.rb
75
75
  E11y.configure do |config|
76
- config.adapters << E11y::Adapters::OpenTelemetryCollectorAdapter.new(
76
+ config.adapters[:otel] = E11y::Adapters::OpenTelemetryCollector.new(
77
77
  endpoint: ENV['OTEL_EXPORTER_OTLP_ENDPOINT'] || 'http://localhost:4318',
78
- protocol: :http, # :http or :grpc
79
- headers: {
80
- 'X-API-Key' => ENV['OTEL_API_KEY']
81
- },
82
-
83
- # Signal types
84
- export_logs: true, # E11y events → OTel Logs Signal
85
- export_traces: true, # Spans from events → OTel Traces
86
- export_metrics: false, # Use Yabeda for metrics (better)
87
-
88
- # Batching
89
- batch_size: 100,
90
- flush_interval: 10.seconds,
91
-
92
- # Compression
93
- compression: :gzip,
94
-
95
- # Retry
96
- retry_enabled: true,
97
- max_retries: 3
78
+ service_name: 'my-app',
79
+ headers: { 'X-API-Key' => ENV['OTEL_API_KEY'] },
80
+ compress: true # default, gzip on HTTP body
98
81
  )
99
82
  end
100
83
 
@@ -103,14 +86,14 @@ end
103
86
  # ├─→ Jaeger (traces)
104
87
  # ├─→ Loki (logs)
105
88
  # ├─→ Prometheus (metrics)
106
- # └─→ S3 (archive)
89
+ # └─→ Object storage (archive, via OTel exporter)
107
90
  ```
108
91
 
109
92
  ---
110
93
 
111
94
  ### 2. Semantic Conventions Mapping
112
95
 
113
- > **Implementation:** See [ADR-007 Section 4: Semantic Conventions](../ADR-007-opentelemetry-integration.md#4-semantic-conventions) for automatic field mapping across HTTP, DB, RPC, Messaging, and Exception patterns.
96
+ > **Implementation:** See [ADR-007 Section 4: Semantic Conventions](../architecture/ADR-007-opentelemetry-integration.md#4-semantic-conventions) for automatic field mapping across HTTP, DB, RPC, Messaging, and Exception patterns.
114
97
 
115
98
  **Automatic field mapping to OTel standards:**
116
99
  ```ruby
@@ -171,7 +154,7 @@ Events::HttpRequest.track(
171
154
 
172
155
  ### 3. Automatic Span Creation
173
156
 
174
- > **Implementation:** See [ADR-007 Section 6: Traces Signal Export](../ADR-007-opentelemetry-integration.md#6-traces-signal-export) for automatic span creation rules and parent-child relationships.
157
+ > **Implementation:** See [ADR-007 Section 6: Traces Signal Export](../architecture/ADR-007-opentelemetry-integration.md#6-traces-signal-export) for automatic span creation rules and parent-child relationships.
175
158
 
176
159
  **Create spans from E11y events:**
177
160
  ```ruby
@@ -207,7 +190,7 @@ Events::OrderProcessingStarted.track(
207
190
 
208
191
  ### 4. W3C Trace Context Integration
209
192
 
210
- > **Implementation:** See [ADR-007 Section 8: Trace Context Integration](../ADR-007-opentelemetry-integration.md#8-trace-context-integration) for OTel SDK as primary trace context source.
193
+ > **Implementation:** See [ADR-007 Section 8: Trace Context Integration](../architecture/ADR-007-opentelemetry-integration.md#8-trace-context-integration) for OTel SDK as primary trace context source.
211
194
 
212
195
  **Automatic trace context from OpenTelemetry SDK:**
213
196
  ```ruby
@@ -240,7 +223,7 @@ Events::OrderCreated.track(order_id: '123')
240
223
 
241
224
  ### 5. OTel Logs Signal Export
242
225
 
243
- > **Implementation:** See [ADR-007 Section 5: Logs Signal Export](../ADR-007-opentelemetry-integration.md#5-logs-signal-export) for OTLP JSON format and trace correlation details.
226
+ > **Implementation:** See [ADR-007 Section 5: Logs Signal Export](../architecture/ADR-007-opentelemetry-integration.md#5-logs-signal-export) for OTLP JSON format and trace correlation details.
244
227
 
245
228
  **Export E11y events as OpenTelemetry Logs:**
246
229
  ```ruby
@@ -297,7 +280,7 @@ Events::OrderCreated.track(order_id: '123')
297
280
  ### 6. Baggage PII Protection (C08 Resolution) ⚠️ CRITICAL
298
281
 
299
282
  > **⚠️ CRITICAL: C08 Conflict Resolution - PII Leaking via OpenTelemetry Baggage**
300
- > **See:** [ADR-006 Section 5.5](../ADR-006-security-compliance.md#55-opentelemetry-baggage-pii-protection-c08-resolution--critical) for detailed architecture and GDPR compliance rationale.
283
+ > **See:** [ADR-006 Section 5.5](../architecture/ADR-006-security-compliance.md#55-opentelemetry-baggage-pii-protection-c08-resolution--critical) for detailed architecture and GDPR compliance rationale.
301
284
  > **Problem:** OpenTelemetry Baggage propagates data via HTTP headers (`baggage: key1=value1,key2=value2`), bypassing E11y's PII filtering. If a developer accidentally adds PII to baggage, it leaks across all services.
302
285
  > **Solution:** Block ALL baggage keys by default, allow ONLY safe keys via allowlist.
303
286
 
@@ -330,33 +313,23 @@ E11y blocks ALL baggage keys by default, allowing ONLY safe keys (no PII):
330
313
  ```ruby
331
314
  # config/initializers/e11y.rb
332
315
  E11y.configure do |config|
333
- config.security.baggage_protection do
334
- enabled true # ✅ CRITICAL: Always enable in production
335
-
336
- # Allowlist: ONLY these keys are safe
337
- allowed_keys [
338
- 'trace_id', # ✅ Safe: Correlation ID
339
- 'span_id', # ✅ Safe: Trace context
340
- 'environment', # ✅ Safe: Deployment context
341
- 'version', # ✅ Safe: Service version
342
- 'service_name', # ✅ Safe: Service identifier
343
- 'request_id', # Safe: Request identifier
344
- # Custom safe keys (no PII!):
345
- 'feature_flag_id', # ✅ Safe: Feature flag name
346
- 'ab_test_variant' # ✅ Safe: A/B test group
347
- ]
348
-
349
- # Block mode: What happens when PII detected?
350
- block_mode :silent # Options: :silent (log), :warn (log+warn), :raise (exception)
351
-
352
- # Monitoring: Track violations
353
- on_blocked_key do |key, value, caller_location|
354
- Yabeda.e11y_baggage_pii_blocked.increment(
355
- key: key,
356
- service: ENV['SERVICE_NAME']
357
- )
358
- end
359
- end
316
+ config.security_baggage_protection_enabled = true # ✅ CRITICAL: Always enable in production
317
+
318
+ # Allowlist: ONLY these keys are safe
319
+ config.security_baggage_protection_allowed_keys = [
320
+ 'trace_id', # ✅ Safe: Correlation ID
321
+ 'span_id', # ✅ Safe: Trace context
322
+ 'environment', # ✅ Safe: Deployment context
323
+ 'version', # ✅ Safe: Service version
324
+ 'service_name', # ✅ Safe: Service identifier
325
+ 'request_id', # ✅ Safe: Request identifier
326
+ # Custom safe keys (no PII!):
327
+ 'feature_flag_id', # Safe: Feature flag name
328
+ 'ab_test_variant' # ✅ Safe: A/B test group
329
+ ]
330
+
331
+ # Block mode: What happens when PII detected?
332
+ config.security_baggage_protection_block_mode = :silent # Options: :silent (log), :warn (log+warn), :raise (exception)
360
333
  end
361
334
  ```
362
335
 
@@ -421,11 +394,9 @@ Fail fast in non-production environments:
421
394
  ```ruby
422
395
  # config/environments/development.rb
423
396
  E11y.configure do |config|
424
- config.security.baggage_protection do
425
- enabled true
426
- block_mode :raise # RAISE exception on blocked keys (fail fast)
427
- allowed_keys E11y::Middleware::BaggageProtection::ALLOWED_KEYS
428
- end
397
+ config.security_baggage_protection_enabled = true
398
+ config.security_baggage_protection_block_mode = :raise # ← RAISE exception on blocked keys (fail fast)
399
+ config.security_baggage_protection_allowed_keys = E11y::BAGGAGE_PROTECTION_DEFAULT_ALLOWED_KEYS
429
400
  end
430
401
 
431
402
  # Developer tries to set PII:
@@ -531,18 +502,14 @@ exporters:
531
502
  prometheus:
532
503
  endpoint: 0.0.0.0:8889
533
504
 
534
- # Archive S3
535
- s3:
536
- region: us-east-1
537
- bucket: telemetry-archive
538
- prefix: logs/
505
+ # Archive: OTel Collector can export to object storage (add exporter config as needed)
539
506
 
540
507
  service:
541
508
  pipelines:
542
509
  logs:
543
510
  receivers: [otlp]
544
511
  processors: [batch, resource, filter]
545
- exporters: [loki, s3]
512
+ exporters: [loki]
546
513
 
547
514
  traces:
548
515
  receivers: [otlp]
@@ -558,11 +525,9 @@ service:
558
525
  ```ruby
559
526
  # config/initializers/e11y.rb
560
527
  E11y.configure do |config|
561
- config.adapters << E11y::Adapters::OpenTelemetryCollectorAdapter.new(
528
+ config.adapters[:otel] = E11y::Adapters::OpenTelemetryCollector.new(
562
529
  endpoint: 'http://otel-collector:4318',
563
- protocol: :http,
564
- export_logs: true,
565
- export_traces: true
530
+ service_name: 'my-app'
566
531
  )
567
532
  end
568
533
 
@@ -742,14 +707,11 @@ end
742
707
  # config/initializers/e11y.rb
743
708
  E11y.configure do |config|
744
709
  # Single adapter: OTel Collector
745
- config.adapters = [
746
- E11y::Adapters::OpenTelemetryCollectorAdapter.new(
747
- endpoint: 'http://otel-collector:4318',
748
- export_logs: true,
749
- export_traces: true
750
- )
751
- ]
752
-
710
+ config.adapters[:otel] = E11y::Adapters::OpenTelemetryCollector.new(
711
+ endpoint: 'http://otel-collector:4318',
712
+ service_name: 'my-app'
713
+ )
714
+
753
715
  # OTel Collector handles routing to multiple backends!
754
716
  # No need for multiple E11y adapters
755
717
  end
@@ -757,7 +719,7 @@ end
757
719
  # OTel Collector routes to:
758
720
  # - Loki (logs, last 30 days)
759
721
  # - Jaeger (traces, last 7 days)
760
- # - S3 (archive, long-term storage)
722
+ # - Object storage (archive, long-term; OTel exporter)
761
723
  # - Prometheus (metrics via remote write)
762
724
 
763
725
  # Benefits:
@@ -931,7 +893,7 @@ end
931
893
  ┌─────────────┐ │ ├─→ Loki (logs)
932
894
  │ Sidekiq │─┘ ├─→ Jaeger (traces)
933
895
  └─────────────┘ ├─→ Prometheus (metrics)
934
- ├─→ S3 (archive)
896
+ ├─→ Object storage (archive)
935
897
  └─→ Datadog (optional)
936
898
  ```
937
899
 
@@ -1048,11 +1010,10 @@ end
1048
1010
  **1. Use OTel Collector in production**
1049
1011
  ```ruby
1050
1012
  # ✅ GOOD: Central pipeline
1051
- config.adapters = [
1052
- E11y::Adapters::OpenTelemetryCollectorAdapter.new(
1053
- endpoint: 'http://otel-collector:4318'
1054
- )
1055
- ]
1013
+ config.adapters[:otel] = E11y::Adapters::OpenTelemetryCollector.new(
1014
+ endpoint: 'http://otel-collector:4318',
1015
+ service_name: 'my-app'
1016
+ )
1056
1017
 
1057
1018
  # OTel Collector handles:
1058
1019
  # - Sampling
@@ -1094,13 +1055,14 @@ end
1094
1055
  config.adapters = [
1095
1056
  E11y::Adapters::JaegerAdapter.new(...),
1096
1057
  E11y::Adapters::LokiAdapter.new(...),
1097
- E11y::Adapters::S3Adapter.new(...)
1058
+ E11y::Adapters::FileAdapter.new(...) # Direct to file (bypasses OTel routing)
1098
1059
  ]
1099
1060
 
1100
1061
  # ✅ GOOD: Through OTel Collector
1101
- config.adapters = [
1102
- E11y::Adapters::OpenTelemetryCollectorAdapter.new(...)
1103
- ]
1062
+ config.adapters[:otel] = E11y::Adapters::OpenTelemetryCollector.new(
1063
+ endpoint: ENV['OTEL_EXPORTER_OTLP_ENDPOINT'],
1064
+ service_name: 'my-app'
1065
+ )
1104
1066
  ```
1105
1067
 
1106
1068
  **2. Don't use custom field names**
@@ -60,7 +60,7 @@ Events::OrderShipping.track(order_id: '789', tracking: 'TRACK123')
60
60
 
61
61
  ## 🎯 Features
62
62
 
63
- > **Implementation:** See [ADR-005: Tracing Context](../ADR-005-tracing-context.md) for complete architecture, including [Section 5: W3C Trace Context](../ADR-005-tracing-context.md#5-w3c-trace-context), [Section 6.1: HTTP Propagator](../ADR-005-tracing-context.md#61-http-propagator-outgoing-requests), and [Section 8: Context Inheritance](../ADR-005-tracing-context.md#8-context-inheritance-thread-fiber-support).
63
+ > **Implementation:** See [ADR-005: Tracing Context](../architecture/ADR-005-tracing-context.md) for complete architecture, including [Section 5: W3C Trace Context](../architecture/ADR-005-tracing-context.md#5-w3c-trace-context), [Section 6.1: HTTP Propagator](../architecture/ADR-005-tracing-context.md#61-http-propagator-outgoing-requests), and [Section 8: Context Inheritance](../architecture/ADR-005-tracing-context.md#8-context-inheritance-thread-fiber-support).
64
64
 
65
65
  ### 1. Automatic W3C Trace Context Propagation
66
66
 
@@ -206,7 +206,7 @@ end
206
206
 
207
207
  ### 3. Background Job Trace Propagation (C17 Resolution) ⚠️
208
208
 
209
- > **Implementation:** See [ADR-005 Section 8.3: Background Job Tracing Strategy](../ADR-005-tracing-context.md#83-background-job-tracing-strategy-c17-resolution) for the hybrid tracing model.
209
+ > **Implementation:** See [ADR-005 Section 8.3: Background Job Tracing Strategy](../architecture/ADR-005-tracing-context.md#83-background-job-tracing-strategy-c17-resolution) for the hybrid tracing model.
210
210
 
211
211
  **Hybrid Tracing Model (New Trace + Parent Link):**
212
212
  ```ruby
@@ -279,7 +279,7 @@ end
279
279
  3. **Trace clarity:** Separate timelines for sync (request) vs async (job) operations
280
280
  4. **Link preserved:** `parent_trace_id` allows reconstructing full flow
281
281
 
282
- See [ADR-005 §8.3](../ADR-005-tracing-context.md#83-background-job-tracing-strategy-c17-resolution) for detailed rationale.
282
+ See [ADR-005 §8.3](../architecture/ADR-005-tracing-context.md#83-background-job-tracing-strategy-c17-resolution) for detailed rationale.
283
283
 
284
284
  **Visual Diagram: Request Trace → Job Trace (with parent link)**
285
285
 
@@ -396,7 +396,7 @@ Events::PaymentReceived.track(
396
396
 
397
397
  ### 5. Service Mesh Integration
398
398
 
399
- > **Implementation:** See [ADR-005 Section 5.3: HTTP Header Extraction](../ADR-005-tracing-context.md#53-http-header-extraction-w3c-legacy-headers) for W3C standard header support enabling service mesh compatibility.
399
+ > **Implementation:** See [ADR-005 Section 5.3: HTTP Header Extraction](../architecture/ADR-005-tracing-context.md#53-http-header-extraction-w3c-legacy-headers) for W3C standard header support enabling service mesh compatibility.
400
400
 
401
401
  **Automatic integration with Istio/Linkerd:**
402
402
  ```ruby
@@ -72,7 +72,7 @@ end
72
72
 
73
73
  ## 🎯 Features
74
74
 
75
- > **Implementation:** See [ADR-008: Rails Integration](../ADR-008-rails-integration.md) for complete architecture, including [Section 5: Sidekiq Integration](../ADR-008-rails-integration.md#5-sidekiq-integration), [Section 6: ActiveJob Integration](../ADR-008-rails-integration.md#6-activejob-integration), and [Section 5.3: Job-Scoped Buffer](../ADR-008-rails-integration.md#53-job-scoped-buffer).
75
+ > **Implementation:** See [ADR-008: Rails Integration](../architecture/ADR-008-rails-integration.md) for complete architecture, including [Section 5: Sidekiq Integration](../architecture/ADR-008-rails-integration.md#5-sidekiq-integration), [Section 6: ActiveJob Integration](../architecture/ADR-008-rails-integration.md#6-activejob-integration), and [Section 5.3: Job-Scoped Buffer](../architecture/ADR-008-rails-integration.md#53-job-scoped-buffer).
76
76
 
77
77
  ### 1. Automatic Instrumentation
78
78
 
@@ -204,7 +204,7 @@ end
204
204
 
205
205
  ### 5. Job-Specific Events
206
206
 
207
- > **Note:** E11y supports **job-scoped buffering** similar to [UC-001: Request-Scoped Debug Buffering](./UC-001-request-scoped-debug-buffering.md). Debug events within a job are buffered and only flushed if the job fails. See [ADR-001 Section 3.4: Request-Scoped Buffer](../ADR-001-architecture.md#34-request-scoped-buffer) for implementation details (same architecture applies to jobs).
207
+ > **Note:** E11y supports **job-scoped buffering** similar to [UC-001: Request-Scoped Debug Buffering](./UC-001-request-scoped-debug-buffering.md). Debug events within a job are buffered and only flushed if the job fails. See [ADR-001 Section 3.4: Request-Scoped Buffer](../architecture/ADR-001-architecture.md#34-request-scoped-buffer) for implementation details (same architecture applies to jobs).
208
208
 
209
209
  **Emit custom events within jobs:**
210
210
  ```ruby
@@ -506,7 +506,7 @@ end
506
506
 
507
507
  ### 6. Sidekiq Middleware Implementation (C17, C18 Resolutions) ⚠️
508
508
 
509
- > **Reference:** See [ADR-005 §8.3 (C17)](../ADR-005-tracing-context.md#83-background-job-tracing-strategy-c17-resolution) and [ADR-013 §3.6 (C18)](../ADR-013-reliability-error-handling.md#36-event-tracking-in-background-jobs-c18-resolution) for full architecture.
509
+ > **Reference:** See [ADR-005 §8.3 (C17)](../architecture/ADR-005-tracing-context.md#83-background-job-tracing-strategy-c17-resolution) and [ADR-013 §3.6 (C18)](../architecture/ADR-013-reliability-error-handling.md#36-event-tracking-in-background-jobs-c18-resolution) for full architecture.
510
510
 
511
511
  E11y provides two critical Sidekiq middlewares:
512
512
 
@@ -697,7 +697,7 @@ end
697
697
 
698
698
  ```ruby
699
699
  # ❌ BAD: E11y failure fails the job
700
- config.error_handling.fail_on_error_in_jobs = true
700
+ config.error_handling_fail_on_error_in_jobs = true
701
701
 
702
702
  # Job fails if Loki is down:
703
703
  # - Payment was created successfully
@@ -706,7 +706,7 @@ config.error_handling.fail_on_error_in_jobs = true
706
706
  # - Result: Duplicate payments! 💸💸💸
707
707
 
708
708
  # ✅ GOOD: E11y failure doesn't fail the job
709
- config.error_handling.fail_on_error_in_jobs = false
709
+ config.error_handling_fail_on_error_in_jobs = false
710
710
 
711
711
  # Job succeeds even if Loki is down:
712
712
  # - Payment created ✅