e11y 0.1.0 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (244) hide show
  1. checksums.yaml +4 -4
  2. data/.rspec +1 -0
  3. data/.rubocop.yml +143 -3
  4. data/CHANGELOG.md +206 -13
  5. data/CLAUDE.md +168 -0
  6. data/CONTRIBUTING.md +640 -0
  7. data/README.md +573 -107
  8. data/RELEASE.md +269 -0
  9. data/Rakefile +456 -0
  10. data/benchmarks/OPTIMIZATION.md +246 -0
  11. data/benchmarks/README.md +103 -0
  12. data/benchmarks/allocation_profiling.rb +253 -0
  13. data/benchmarks/e11y_benchmarks.rb +447 -0
  14. data/benchmarks/ruby_baseline_allocations.rb +175 -0
  15. data/benchmarks/run_all.rb +9 -21
  16. data/config/README.md +1 -1
  17. data/config/loki-local-config.yaml +12 -0
  18. data/config/otel-collector-config.yaml +44 -0
  19. data/cucumber.yml +1 -0
  20. data/docker-compose.yml +18 -2
  21. data/docs/ADAPTERS.md +76 -0
  22. data/docs/ADAPTIVE_SAMPLING.md +59 -0
  23. data/docs/COMPARISON.md +104 -0
  24. data/docs/CONFIGURATION.md +52 -0
  25. data/docs/DISTRIBUTED_TRACING.md +44 -0
  26. data/docs/LIMITATIONS.md +13 -0
  27. data/docs/METRICS_DSL.md +84 -0
  28. data/docs/PERFORMANCE.md +60 -0
  29. data/docs/PII_FILTERING.md +40 -0
  30. data/docs/PRESETS.md +65 -0
  31. data/docs/QUICK-START.md +545 -592
  32. data/docs/RAILS_INTEGRATION.md +29 -0
  33. data/docs/SCHEMA_VALIDATION.md +63 -0
  34. data/docs/SLO-PROMQL-ALERTS.md +161 -0
  35. data/docs/TESTING.md +69 -0
  36. data/docs/{ADR-001-architecture.md → architecture/ADR-001-architecture.md} +36 -65
  37. data/docs/{ADR-002-metrics-yabeda.md → architecture/ADR-002-metrics-yabeda.md} +62 -236
  38. data/docs/{ADR-003-slo-observability.md → architecture/ADR-003-slo-observability.md} +27 -466
  39. data/docs/{ADR-004-adapter-architecture.md → architecture/ADR-004-adapter-architecture.md} +405 -141
  40. data/docs/{ADR-005-tracing-context.md → architecture/ADR-005-tracing-context.md} +10 -9
  41. data/docs/{ADR-006-security-compliance.md → architecture/ADR-006-security-compliance.md} +184 -191
  42. data/docs/{ADR-007-opentelemetry-integration.md → architecture/ADR-007-opentelemetry-integration.md} +3 -21
  43. data/docs/{ADR-008-rails-integration.md → architecture/ADR-008-rails-integration.md} +209 -339
  44. data/docs/{ADR-009-cost-optimization.md → architecture/ADR-009-cost-optimization.md} +268 -161
  45. data/docs/architecture/ADR-010-developer-experience.md +522 -0
  46. data/docs/{ADR-011-testing-strategy.md → architecture/ADR-011-testing-strategy.md} +41 -83
  47. data/docs/{ADR-013-reliability-error-handling.md → architecture/ADR-013-reliability-error-handling.md} +37 -12
  48. data/docs/{ADR-014-event-driven-slo.md → architecture/ADR-014-event-driven-slo.md} +12 -24
  49. data/docs/{ADR-015-middleware-order.md → architecture/ADR-015-middleware-order.md} +23 -41
  50. data/docs/{ADR-016-self-monitoring-slo.md → architecture/ADR-016-self-monitoring-slo.md} +52 -349
  51. data/docs/architecture/ADR-017-multi-rails-compatibility.md +96 -0
  52. data/docs/architecture/ADR-018-memory-optimization.md +366 -0
  53. data/docs/architecture/ADR-INDEX.md +104 -0
  54. data/docs/{00-ICP-AND-TIMELINE.md → prd/00-ICP-AND-TIMELINE.md} +8 -8
  55. data/docs/{01-SCALE-REQUIREMENTS.md → prd/01-SCALE-REQUIREMENTS.md} +6 -6
  56. data/docs/prd/01-overview-vision.md +19 -14
  57. data/docs/use_cases/README.md +22 -23
  58. data/docs/use_cases/UC-001-request-scoped-debug-buffering.md +50 -44
  59. data/docs/use_cases/UC-002-business-event-tracking.md +26 -95
  60. data/docs/use_cases/UC-003-event-metrics.md +66 -0
  61. data/docs/use_cases/UC-004-zero-config-slo-tracking.md +42 -101
  62. data/docs/use_cases/UC-005-sentry-integration.md +13 -15
  63. data/docs/use_cases/UC-006-trace-context-management.md +30 -28
  64. data/docs/use_cases/UC-007-pii-filtering.md +35 -87
  65. data/docs/use_cases/UC-008-opentelemetry-integration.md +51 -89
  66. data/docs/use_cases/UC-009-multi-service-tracing.md +4 -4
  67. data/docs/use_cases/UC-010-background-job-tracking.md +5 -5
  68. data/docs/use_cases/UC-011-rate-limiting.md +95 -168
  69. data/docs/use_cases/UC-012-audit-trail.md +21 -46
  70. data/docs/use_cases/UC-013-high-cardinality-protection.md +29 -167
  71. data/docs/use_cases/UC-014-adaptive-sampling.md +2 -2
  72. data/docs/use_cases/UC-015-cost-optimization.md +46 -99
  73. data/docs/use_cases/UC-016-rails-logger-migration.md +39 -213
  74. data/docs/use_cases/UC-017-local-development.md +203 -777
  75. data/docs/use_cases/UC-018-testing-events.md +3 -3
  76. data/docs/use_cases/UC-019-retention-based-routing.md +531 -0
  77. data/docs/use_cases/UC-020-event-versioning.md +8 -9
  78. data/docs/use_cases/UC-021-error-handling-retry-dlq.md +18 -22
  79. data/docs/use_cases/UC-022-event-registry.md +15 -21
  80. data/docs/use_cases/backlog.md +119 -87
  81. data/e11y.gemspec +29 -18
  82. data/gems/e11y-devtools/README.md +136 -0
  83. data/gems/e11y-devtools/config/routes.rb +8 -0
  84. data/gems/e11y-devtools/e11y-devtools.gemspec +25 -0
  85. data/gems/e11y-devtools/exe/e11y +34 -0
  86. data/gems/e11y-devtools/lib/e11y/devtools/mcp/server.rb +96 -0
  87. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tool_base.rb +25 -0
  88. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/clear.rb +31 -0
  89. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/errors.rb +35 -0
  90. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/event_detail.rb +33 -0
  91. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/events_by_trace.rb +33 -0
  92. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/interactions.rb +40 -0
  93. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/recent_events.rb +34 -0
  94. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/search.rb +34 -0
  95. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/stats.rb +30 -0
  96. data/gems/e11y-devtools/lib/e11y/devtools/overlay/assets/overlay.js +115 -0
  97. data/gems/e11y-devtools/lib/e11y/devtools/overlay/controller.rb +54 -0
  98. data/gems/e11y-devtools/lib/e11y/devtools/overlay/engine.rb +26 -0
  99. data/gems/e11y-devtools/lib/e11y/devtools/overlay/middleware.rb +80 -0
  100. data/gems/e11y-devtools/lib/e11y/devtools/overlay/rails_controller.rb +42 -0
  101. data/gems/e11y-devtools/lib/e11y/devtools/tui/app.rb +262 -0
  102. data/gems/e11y-devtools/lib/e11y/devtools/tui/grouping.rb +66 -0
  103. data/gems/e11y-devtools/lib/e11y/devtools/tui/widgets/event_detail.rb +62 -0
  104. data/gems/e11y-devtools/lib/e11y/devtools/tui/widgets/event_list.rb +70 -0
  105. data/gems/e11y-devtools/lib/e11y/devtools/tui/widgets/interaction_list.rb +47 -0
  106. data/gems/e11y-devtools/lib/e11y/devtools/version.rb +8 -0
  107. data/gems/e11y-devtools/lib/e11y/devtools.rb +13 -0
  108. data/gems/e11y-devtools/spec/e11y/devtools/mcp/tools_spec.rb +107 -0
  109. data/gems/e11y-devtools/spec/e11y/devtools/overlay/controller_spec.rb +58 -0
  110. data/gems/e11y-devtools/spec/e11y/devtools/overlay/middleware_spec.rb +46 -0
  111. data/gems/e11y-devtools/spec/e11y/devtools/tui/app_spec.rb +85 -0
  112. data/gems/e11y-devtools/spec/e11y/devtools/tui/grouping_spec.rb +64 -0
  113. data/gems/e11y-devtools/spec/spec_helper.rb +5 -0
  114. data/gems/e11y-devtools/spec/tui/widgets/event_list_spec.rb +44 -0
  115. data/gems/e11y-devtools/spec/tui/widgets/interaction_list_spec.rb +62 -0
  116. data/lib/e11y/adapters/adaptive_batcher.rb +3 -0
  117. data/lib/e11y/adapters/audit_encrypted.rb +60 -12
  118. data/lib/e11y/adapters/base.rb +42 -28
  119. data/lib/e11y/adapters/dev_log/file_store.rb +143 -0
  120. data/lib/e11y/adapters/dev_log/query.rb +219 -0
  121. data/lib/e11y/adapters/dev_log.rb +118 -0
  122. data/lib/e11y/adapters/file.rb +7 -7
  123. data/lib/e11y/adapters/in_memory.rb +58 -5
  124. data/lib/e11y/adapters/in_memory_test.rb +29 -0
  125. data/lib/e11y/adapters/loki.rb +65 -21
  126. data/lib/e11y/adapters/null.rb +82 -0
  127. data/lib/e11y/adapters/opentelemetry_collector.rb +183 -0
  128. data/lib/e11y/adapters/otel_logs.rb +147 -32
  129. data/lib/e11y/adapters/sentry.rb +11 -5
  130. data/lib/e11y/adapters/stdout.rb +73 -7
  131. data/lib/e11y/adapters/yabeda.rb +197 -29
  132. data/lib/e11y/buffers/adaptive_buffer.rb +3 -17
  133. data/lib/e11y/buffers/{request_scoped_buffer.rb → ephemeral_buffer.rb} +72 -58
  134. data/lib/e11y/buffers/ring_buffer.rb +3 -16
  135. data/lib/e11y/buffers.rb +8 -8
  136. data/lib/e11y/configuration.rb +272 -0
  137. data/lib/e11y/console.rb +52 -67
  138. data/lib/e11y/current.rb +53 -1
  139. data/lib/e11y/debug/pipeline_inspector.rb +96 -0
  140. data/lib/e11y/documentation/generator.rb +48 -0
  141. data/lib/e11y/event/base.rb +245 -86
  142. data/lib/e11y/event/value_sampling_config.rb +8 -6
  143. data/lib/e11y/events/rails/database/query.rb +1 -4
  144. data/lib/e11y/events/rails/http/request.rb +1 -1
  145. data/lib/e11y/events/rails/job/failed.rb +2 -0
  146. data/lib/e11y/instruments/active_job.rb +47 -10
  147. data/lib/e11y/instruments/rails_instrumentation.rb +97 -49
  148. data/lib/e11y/instruments/sidekiq.rb +139 -33
  149. data/lib/e11y/linters/base.rb +11 -0
  150. data/lib/e11y/linters/pii/pii_declaration_linter.rb +120 -0
  151. data/lib/e11y/linters/slo/config_consistency_linter.rb +76 -0
  152. data/lib/e11y/linters/slo/explicit_declaration_linter.rb +36 -0
  153. data/lib/e11y/linters/slo/slo_status_from_linter.rb +41 -0
  154. data/lib/e11y/logger/bridge.rb +46 -57
  155. data/lib/e11y/metrics/cardinality_protection.rb +252 -12
  156. data/lib/e11y/metrics/cardinality_tracker.rb +32 -5
  157. data/lib/e11y/metrics/registry.rb +5 -3
  158. data/lib/e11y/metrics/relabeling.rb +0 -56
  159. data/lib/e11y/metrics/test_backend.rb +62 -0
  160. data/lib/e11y/metrics.rb +62 -11
  161. data/lib/e11y/middleware/adapter_resolver.rb +40 -0
  162. data/lib/e11y/middleware/audit_signing.rb +51 -11
  163. data/lib/e11y/middleware/baggage_protection.rb +75 -0
  164. data/lib/e11y/middleware/dev_log_source.rb +24 -0
  165. data/lib/e11y/middleware/event_slo.rb +23 -9
  166. data/lib/e11y/middleware/otel_span.rb +23 -0
  167. data/lib/e11y/middleware/pii_filter.rb +113 -76
  168. data/lib/e11y/middleware/rate_limiting.rb +54 -27
  169. data/lib/e11y/middleware/request.rb +73 -20
  170. data/lib/e11y/middleware/routing.rb +177 -93
  171. data/lib/e11y/middleware/sampling.rb +109 -41
  172. data/lib/e11y/middleware/self_monitoring_emit.rb +39 -0
  173. data/lib/e11y/middleware/trace_context.rb +81 -17
  174. data/lib/e11y/middleware/track_latency.rb +34 -0
  175. data/lib/e11y/middleware/validation.rb +21 -12
  176. data/lib/e11y/middleware/versioning.rb +27 -23
  177. data/lib/e11y/opentelemetry/semantic_conventions.rb +109 -0
  178. data/lib/e11y/opentelemetry/span_creator.rb +142 -0
  179. data/lib/e11y/pii/patterns.rb +12 -1
  180. data/lib/e11y/pii.rb +7 -7
  181. data/lib/e11y/pipeline/builder.rb +1 -1
  182. data/lib/e11y/presets/audit_event.rb +13 -2
  183. data/lib/e11y/railtie.rb +70 -29
  184. data/lib/e11y/registry.rb +306 -0
  185. data/lib/e11y/reliability/circuit_breaker.rb +22 -21
  186. data/lib/e11y/reliability/dlq/base.rb +71 -0
  187. data/lib/e11y/reliability/dlq/file_adapter.rb +301 -0
  188. data/lib/e11y/reliability/dlq/file_storage.rb +71 -31
  189. data/lib/e11y/reliability/dlq/filter.rb +39 -53
  190. data/lib/e11y/reliability/retry_handler.rb +30 -29
  191. data/lib/e11y/reliability/retry_rate_limiter.rb +3 -11
  192. data/lib/e11y/sampling/error_spike_detector.rb +14 -5
  193. data/lib/e11y/sampling/load_monitor.rb +15 -10
  194. data/lib/e11y/sampling/stratified_tracker.rb +18 -0
  195. data/lib/e11y/self_monitoring/buffer_monitor.rb +2 -0
  196. data/lib/e11y/self_monitoring/performance_monitor.rb +19 -61
  197. data/lib/e11y/self_monitoring/reliability_monitor.rb +7 -74
  198. data/lib/e11y/slo/config_loader.rb +40 -0
  199. data/lib/e11y/slo/config_validator.rb +58 -0
  200. data/lib/e11y/slo/dashboard_generator.rb +122 -0
  201. data/lib/e11y/slo/event_driven.rb +8 -0
  202. data/lib/e11y/slo/tracker.rb +31 -4
  203. data/lib/e11y/testing/have_tracked_event_matcher.rb +190 -0
  204. data/lib/e11y/testing/rspec_matchers.rb +21 -0
  205. data/lib/e11y/testing/snapshot_matcher.rb +86 -0
  206. data/lib/e11y/trace_context/sampler.rb +35 -0
  207. data/lib/e11y/tracing/faraday_middleware.rb +31 -0
  208. data/lib/e11y/tracing/net_http_patch.rb +33 -0
  209. data/lib/e11y/tracing/propagator.rb +116 -0
  210. data/lib/e11y/tracing.rb +47 -0
  211. data/lib/e11y/version.rb +1 -1
  212. data/lib/e11y/versioning/version_extractor.rb +32 -0
  213. data/lib/e11y.rb +148 -195
  214. data/lib/generators/e11y/event/event_generator.rb +22 -0
  215. data/lib/generators/e11y/event/templates/event.rb.tt +16 -0
  216. data/lib/generators/e11y/grafana_dashboard/grafana_dashboard_generator.rb +30 -0
  217. data/lib/generators/e11y/grafana_dashboard/templates/e11y_dashboard.json +81 -0
  218. data/lib/generators/e11y/install/install_generator.rb +34 -0
  219. data/lib/generators/e11y/install/templates/e11y.rb +239 -0
  220. data/lib/generators/e11y/prometheus_alerts/prometheus_alerts_generator.rb +29 -0
  221. data/lib/generators/e11y/prometheus_alerts/templates/e11y_alerts.yml +28 -0
  222. data/lib/tasks/e11y_docs.rake +30 -0
  223. data/lib/tasks/e11y_events.rake +71 -0
  224. data/lib/tasks/e11y_lint.rake +91 -0
  225. data/lib/tasks/e11y_slo.rake +29 -0
  226. metadata +207 -72
  227. data/docs/ADR-010-developer-experience.md +0 -2166
  228. data/docs/API-REFERENCE-L28.md +0 -914
  229. data/docs/COMPREHENSIVE-CONFIGURATION.md +0 -2366
  230. data/docs/IMPLEMENTATION_NOTES.md +0 -2804
  231. data/docs/IMPLEMENTATION_PLAN.md +0 -1971
  232. data/docs/IMPLEMENTATION_PLAN_ARCHITECTURE.md +0 -586
  233. data/docs/PLAN.md +0 -148
  234. data/docs/README.md +0 -296
  235. data/docs/design/00-memory-optimization.md +0 -593
  236. data/docs/guides/MIGRATION-L27-L28.md +0 -692
  237. data/docs/guides/PERFORMANCE-BENCHMARKS.md +0 -434
  238. data/docs/guides/README.md +0 -44
  239. data/docs/use_cases/UC-003-pattern-based-metrics.md +0 -1627
  240. data/docs/use_cases/UC-019-tiered-storage-migration.md +0 -562
  241. data/lib/e11y/adapters/registry.rb +0 -141
  242. data/lib/e11y/middleware/pii_filtering.rb +0 -280
  243. data/lib/e11y/middleware/slo.rb +0 -168
  244. /data/docs/{ADR-012-event-evolution.md → architecture/ADR-012-event-evolution.md} +0 -0
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: ae952b91e779ee9e04150e324109586016d6dd0e9831d6acf7867e05f686f579
4
- data.tar.gz: 8c3a957e11bd4cc84b4f5fa9c90eae3248c083b98a518ee328cad76401d5454b
3
+ metadata.gz: 9d7e79dcbcbd63a42a108fff3b2c29f4cea6f07e8e2632145d4cff39f85514ac
4
+ data.tar.gz: b53a0b131efaad3accb9ab20b75c5dcf973274422d384432a3b3562437750e4b
5
5
  SHA512:
6
- metadata.gz: ce56d9787c13db96db175a07418a34b17a43dd11dceb23d463c346abb7082caa1c45151826f6dc59981a069d8f2ec10d0f6eb4516f2da8cbce91754c483bae20
7
- data.tar.gz: b5b650f09a2b634e31fedcebd058965dbc9787960769387b12a84bb5c14c96eb68bcc3245d04393bb75d51011dd290064185040daddcbc975aa8a66f6bda1709
6
+ metadata.gz: 4576c625410acc475dbd5782fa546b29adcc6a815c319d86eae9a9b8e0a700c49a099197db7fdccf3c3ea43f594c24bdd90ccdc689cad82e9a8da11d18d50999
7
+ data.tar.gz: f569447a2c1b532cc39246d163fe245b61f63ec9ee618a2c9cf428936f3eead760549d6774de9b8a5afc09d12a0df7439ad53bf0f41993a7c518af566ebbdc1d
data/.rspec CHANGED
@@ -2,3 +2,4 @@
2
2
  --format documentation
3
3
  --color
4
4
  --order random
5
+ --tag ~benchmark
data/.rubocop.yml CHANGED
@@ -3,8 +3,10 @@
3
3
  # See https://rubystyle.guide/ for Ruby style guide
4
4
  # See https://docs.rubocop.org/rubocop/ for all available cops
5
5
 
6
- require:
6
+ plugins:
7
+ - rubocop-performance
7
8
  - rubocop-rspec
9
+ - rubocop-rspec_rails
8
10
 
9
11
  AllCops:
10
12
  TargetRubyVersion: 3.2
@@ -15,23 +17,89 @@ AllCops:
15
17
  - 'vendor/**/*'
16
18
  - 'node_modules/**/*'
17
19
  - 'tmp/**/*'
20
+ - 'benchmarks/**/*'
18
21
 
19
22
  # Metrics
20
23
  Metrics/BlockLength:
21
24
  Exclude:
22
25
  - 'spec/**/*'
23
26
  - 'Rakefile'
27
+ - 'lib/tasks/**/*'
24
28
  - '*.gemspec'
29
+ - 'lib/e11y/reliability/retry_handler.rb'
30
+ - 'lib/e11y/instruments/active_job.rb'
25
31
 
26
32
  Metrics/MethodLength:
27
- Max: 15
33
+ Max: 20
28
34
  Exclude:
29
35
  - 'spec/**/*'
36
+ - 'lib/e11y/adapters/yabeda.rb'
37
+ - 'lib/e11y/middleware/routing.rb'
38
+ - 'lib/e11y/event/base.rb'
39
+ - 'lib/e11y/middleware/rate_limiting.rb'
40
+ - 'lib/e11y/middleware/sampling.rb'
30
41
 
31
42
  Metrics/ClassLength:
32
43
  Max: 100
33
44
  Exclude:
34
45
  - 'spec/**/*'
46
+ - 'lib/e11y/buffers/ephemeral_buffer.rb'
47
+ - 'lib/e11y/registry.rb'
48
+ - 'lib/e11y/adapters/otel_logs.rb'
49
+ - 'lib/e11y/adapters/opentelemetry_collector.rb'
50
+ - 'lib/e11y/opentelemetry/span_creator.rb'
51
+ - 'lib/e11y/configuration.rb'
52
+
53
+ Metrics/CyclomaticComplexity:
54
+ Max: 15
55
+ Exclude:
56
+ - 'lib/e11y/adapters/yabeda.rb'
57
+ - 'lib/e11y/middleware/event_slo.rb'
58
+ - 'lib/e11y/middleware/routing.rb'
59
+ - 'spec/integration/pattern_metrics_integration_spec.rb'
60
+ - 'spec/integration/slo_tracking_integration_spec.rb'
61
+ - 'spec/support/integration_helpers.rb'
62
+ - 'spec/support/integration/pii_helpers.rb'
63
+ - 'spec/support/loki_helpers.rb'
64
+ - 'spec/support/matchers/pii_matchers.rb'
65
+
66
+ Metrics/AbcSize:
67
+ Max: 20
68
+ Exclude:
69
+ - 'lib/e11y/middleware/event_slo.rb'
70
+ - 'lib/e11y/middleware/routing.rb'
71
+ - 'lib/e11y/adapters/yabeda.rb'
72
+ - 'lib/e11y/event/base.rb'
73
+ - 'lib/e11y/middleware/versioning.rb'
74
+ - 'lib/e11y/reliability/retry_handler.rb'
75
+ - 'lib/e11y/adapters/base.rb'
76
+ - 'lib/e11y/adapters/opentelemetry_collector.rb'
77
+ - 'lib/e11y/adapters/otel_logs.rb'
78
+ - 'lib/e11y/current.rb'
79
+ - 'lib/e11y/instruments/active_job.rb'
80
+ - 'lib/e11y/instruments/sidekiq.rb'
81
+ - 'lib/e11y/linters/pii/pii_declaration_linter.rb'
82
+ - 'lib/e11y/linters/slo/config_consistency_linter.rb'
83
+ - 'lib/e11y/logger/bridge.rb'
84
+ - 'lib/e11y/middleware/pii_filter.rb'
85
+ - 'lib/e11y/middleware/rate_limiting.rb'
86
+ - 'lib/e11y/middleware/sampling.rb'
87
+ - 'lib/e11y/opentelemetry/span_creator.rb'
88
+ - 'spec/integration/slo_tracking_integration_spec.rb'
89
+ - 'spec/support/integration_helpers.rb'
90
+ - 'spec/support/integration/pii_helpers.rb'
91
+ - 'spec/support/loki_helpers.rb'
92
+ - 'spec/support/matchers/pii_matchers.rb'
93
+
94
+ Metrics/PerceivedComplexity:
95
+ Max: 15
96
+ Exclude:
97
+ - 'lib/e11y/middleware/event_slo.rb'
98
+ - 'spec/integration/slo_tracking_integration_spec.rb'
99
+ - 'spec/support/integration_helpers.rb'
100
+ - 'spec/support/integration/pii_helpers.rb'
101
+ - 'spec/support/loki_helpers.rb'
102
+ - 'spec/support/matchers/pii_matchers.rb'
35
103
 
36
104
  # Style
37
105
  Style/Documentation:
@@ -45,25 +113,97 @@ Style/StringLiterals:
45
113
  Style/FrozenStringLiteralComment:
46
114
  Enabled: true
47
115
 
116
+ # Layout
117
+ Layout/LineLength:
118
+ Max: 150
119
+
48
120
  # RSpec
49
121
  RSpec/ExampleLength:
50
- Max: 20 # Allow longer examples for integration tests
122
+ Max: 40 # Allow longer examples for integration tests
123
+ Exclude:
124
+ - "spec/integration/**/*"
51
125
 
52
126
  RSpec/MultipleExpectations:
53
127
  Max: 10 # Allow more expectations for integration tests
54
128
 
129
+ RSpec/MultipleMemoizedHelpers:
130
+ Max: 10 # Allow more memoized helpers for complex tests
131
+
55
132
  RSpec/NestedGroups:
56
133
  Max: 3
134
+ Exclude:
135
+ - "spec/e11y/current_spec.rb"
136
+ - "spec/e11y/middleware/pii_filtering_spec.rb"
57
137
 
58
138
  # Gem-specific: Development dependencies can be in gemspec
59
139
  Gemspec/DevelopmentDependencies:
60
140
  Enabled: false
61
141
 
142
+ Style/OneClassPerFile:
143
+ Exclude:
144
+ - "spec/dummy/config/application.rb"
145
+ - "spec/support/pipeline_debug.rb"
146
+
62
147
  # Allow integration tests without specific class
63
148
  RSpec/DescribeClass:
64
149
  Exclude:
65
150
  - "spec/zeitwerk_spec.rb"
151
+ - "spec/integration/**/*"
152
+ - "spec/e11y/module_api_spec.rb"
153
+ - "spec/e11y/configuration/error_handling_config_spec.rb"
154
+
155
+ # File path conventions (generators use spec/generators/e11y/ structure)
156
+ RSpec/SpecFilePathFormat:
157
+ Exclude:
158
+ - "spec/e11y/event/dsl_additions_spec.rb"
159
+ - "spec/e11y/module_api_spec.rb"
160
+ - "spec/generators/e11y/*_spec.rb"
161
+ - "spec/e11y/opentelemetry/semantic_conventions_spec.rb"
162
+ - "spec/e11y/opentelemetry/span_creator_spec.rb"
163
+
164
+ RSpec/DescribeMethod:
165
+ Exclude:
166
+ - "spec/e11y/module_api_spec.rb"
167
+
168
+ RSpec/MultipleDescribes:
169
+ Exclude:
170
+ - "spec/e11y/adapters/null_adapter_spec.rb"
66
171
 
67
172
  # Allow simple doubles for now (will improve in Phase 1)
68
173
  RSpec/VerifiedDoubles:
69
174
  Enabled: false
175
+
176
+ # MessageSpies: Prefer explicit expect().to receive() for readability.
177
+ # Large codebase uses this pattern consistently (116 occurrences).
178
+ # Will be gradually migrated to have_received in future.
179
+ RSpec/MessageSpies:
180
+ Enabled: false
181
+
182
+ # RSpec/Output: New cop in rubocop-rspec 3.x flagging puts in specs.
183
+ # We use puts for diagnostic output in integration tests and support files.
184
+ RSpec/Output:
185
+ Enabled: false
186
+
187
+ RSpec/RepeatedExample:
188
+ Exclude:
189
+ - "spec/e11y/instruments/rails_instrumentation_spec.rb"
190
+ - "spec/e11y/versioning/version_extractor_spec.rb"
191
+
192
+ RSpec/StubbedMock:
193
+ Exclude:
194
+ - "spec/e11y/module_api_spec.rb"
195
+ - "spec/e11y/slo/config_loader_spec.rb"
196
+
197
+ Metrics/ParameterLists:
198
+ Exclude:
199
+ - "lib/e11y/adapters/opentelemetry_collector.rb"
200
+ - "lib/e11y/adapters/otel_logs.rb"
201
+
202
+ Style/SafeNavigationChainLength:
203
+ Exclude:
204
+ - "lib/e11y/linters/pii/pii_declaration_linter.rb"
205
+
206
+ Performance/CollectionLiteralInLoop:
207
+ Exclude:
208
+ - "lib/e11y/middleware/pii_filter.rb"
209
+
data/CHANGELOG.md CHANGED
@@ -2,25 +2,218 @@
2
2
 
3
3
  All notable changes to this project will be documented in this file.
4
4
 
5
- The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
5
+ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
10
  ### Added
11
- - Initial gem skeleton setup
12
- - Zeitwerk autoloading configuration
13
- - Basic project structure (lib/, spec/, bin/)
14
- - CI/CD pipeline (GitHub Actions)
15
- - Documentation framework (YARD, README)
16
11
 
17
- ## [0.1.0] - 2026-01-17
12
+ ### Changed
13
+
14
+ ### Fixed
15
+
16
+ ### Deprecated
17
+
18
+ ### Removed
19
+
20
+ ### Security
21
+
22
+ ## [1.0.0] - 2026-03-20
23
+
24
+ ### BREAKING: Configuration — Flat config API
25
+
26
+ **Nested config objects removed.** All configuration options are now flat accessors on `config`.
27
+
28
+ **Key migrations:**
29
+ - `config.rails_instrumentation.enabled` → `config.rails_instrumentation_enabled`
30
+ - `config.logger_bridge.track_severities` → `config.logger_bridge_track_severities`
31
+ - `config.rate_limiting { }` removed → use `config.rate_limiting_enabled`, `config.add_rate_limit_per_event(...)`
32
+ - `config.slo { }` removed → use `config.slo_tracking_enabled`, `config.add_slo_controller(...)`
33
+
34
+ **Full mapping:** See `docs/plans/2026-03-13-configuration-design.md`
35
+
36
+ ### BREAKING: Middleware order changed (ADR-015 compliance)
37
+
38
+ **Per ADR-015 and ADR-006:**
39
+ - **Versioning** moved to last (before Routing) — Validation, PII, RateLimiting, Sampling now use original class names
40
+ - **AuditSigning** before PIIFilter — audit events signed with original data (GDPR Art. 30 non-repudiation)
41
+ - **RateLimiting** before Sampling — matches ADR #4, #5
42
+
43
+ **New order:** TraceContext → Validation → AuditSigning → PIIFilter → RateLimiting → Sampling → Versioning → Routing → EventSlo
44
+
45
+ **Migration:** If you custom-configured pipeline order, ensure Versioning is last before Routing. Audit events now receive unfiltered payload at signing.
46
+
47
+ ### BREAKING: Registry.all_events → Registry.event_classes
48
+
49
+ **Renamed for clarity:** Method returns event classes, not event instances or names.
50
+
51
+ **Migration:** `E11y::Registry.all_events` → `E11y::Registry.event_classes`
52
+
53
+ ### BREAKING: RequestScopedBuffer → EphemeralBuffer
54
+
55
+ **Renamed for accuracy:** The buffer works for both HTTP requests and background jobs. "Ephemeral" reflects its temporary lifecycle.
56
+
57
+ **Migration:**
58
+ - `E11y::Buffers::RequestScopedBuffer` → `E11y::Buffers::EphemeralBuffer`
59
+ - `config.request_buffer` → `config.ephemeral_buffer`
60
+ - `Thread.current[:e11y_request_buffer]` → `Thread.current[:e11y_ephemeral_buffer]`
61
+ - Yabeda metric `e11y_request_buffer_total` → `e11y_ephemeral_buffer_total`
62
+
63
+ **Search and replace:** `RequestScopedBuffer` → `EphemeralBuffer`, `request_buffer` → `ephemeral_buffer`
18
64
 
19
65
  ### Added
20
- - Project initialization
21
- - Phase 0: Gem Setup & Best Practices Research complete
22
- - Research documents for 6 successful gems (Devise, Sidekiq, Puma, Dry-rb, Yabeda, Sentry)
23
- - Best practices synthesis for configuration DSL, testing, documentation, CI/CD, release process
24
66
 
25
- [Unreleased]: https://github.com/arturseletskiy/e11y/compare/v0.1.0...HEAD
26
- [0.1.0]: https://github.com/arturseletskiy/e11y/releases/tag/v0.1.0
67
+ ### Changed
68
+
69
+ ### Fixed
70
+
71
+ ### Deprecated
72
+
73
+ ### Removed
74
+
75
+ ### Security
76
+
77
+ ## [0.2.0] - 2026-01-26
78
+
79
+ ### Added
80
+ - Multi-Rails version support (7.0, 7.1, 8.0) (#5)
81
+ - CI matrix testing across Ruby 3.2, 3.3 with Rails 7.0, 7.1, 8.0
82
+ - Dynamic Gemfile dependencies based on RAILS_VERSION env var
83
+ - Support for sqlite3 1.4 (Rails 7.x) and 2.0 (Rails 8.x)
84
+ - Comprehensive test suite documentation in README (#5)
85
+ - Quick commands using rake tasks
86
+ - Manual commands for each test suite
87
+ - Test suite overview with timing and example counts
88
+ - Development commands reference
89
+ - Climate Control gem for ENV manipulation in tests (#5)
90
+
91
+ ### Fixed
92
+ - **RequestScopedBuffer API method names** (#5)
93
+ - `start!` → `initialize!` (correct initialization method)
94
+ - `flush!` → `discard` (for success path - discard buffered events)
95
+ - `flush_on_error!` → `flush_on_error` (remove bang, method doesn't modify in-place)
96
+ - This eliminates warning messages during test execution
97
+ - **BREAKING CHANGE:** If you use `E11y::Buffers::RequestScopedBuffer` directly, update method names
98
+ - Rails instrumentation event namespaces (#5)
99
+ - Fixed: `"Events::Rails::*"` → `"E11y::Events::Rails::*"`
100
+ - Resolves uninitialized constant errors in Rails instrumentation
101
+ - Rails 8.0 exception handling test compatibility (#5)
102
+ - Updated error handling test to support both Rails 7.x and 8.0 behaviors
103
+ - Rails 8.0 changed: exceptions caught and converted to 500 responses
104
+ - Rails 7.x: exceptions raised with show_exceptions = false
105
+ - HTTP request format validation (#5)
106
+ - Now accepts Symbol (e.g., :html, :json) as Rails passes format as Symbol
107
+ - Removed strict :string type check from format field
108
+ - Integration test isolation issues (#5)
109
+ - Moved Rails initialization to spec_helper.rb (prevents FrozenError)
110
+ - Renamed TestJob → DummyTestJob to prevent class conflicts
111
+ - Fixed railtie integration spec tag isolation
112
+ - Added File.exist? check for routes file in railtie tests
113
+ - Floating point precision in stratified sampling tests (#5)
114
+ - Increased upper bound from 0.95 to 0.96 to handle FP precision
115
+ - Fixes intermittent test failures: expected 0.9500000000000001 to be <= 0.95
116
+ - Test isolation in active_job_spec.rb (#5)
117
+ - Store and restore original request_buffer.enabled config
118
+ - Prevents config changes from affecting subsequent tests
119
+ - View rendering instrumentation test (#5)
120
+ - Added posts/list.html.erb template
121
+ - Added posts#list action to render HTML views
122
+ - Implemented complete test for view rendering events (was pending)
123
+
124
+ ### Changed
125
+ - CI workflow now tests against multiple Rails versions (#5)
126
+ - Matrix: Ruby 3.2, 3.3 × Rails 7.0, 7.1, 8.0
127
+ - Separate artifact uploads per Ruby/Rails combination
128
+ - Enhanced test output with Rails version information
129
+ - Improved test execution speed with better organization (#5)
130
+ - Separate rake tasks: spec:unit, spec:integration, spec:railtie
131
+ - Unit tests exclude integration and railtie specs
132
+ - Integration tests run only integration specs
133
+ - Total: 1729 examples (1672 unit + 36 integration + 21 railtie)
134
+ - RuboCop configuration (#5)
135
+ - Exclude spec/integration/**/* from RSpec/DescribeClass cop
136
+ - Integration tests don't always describe a specific class
137
+
138
+ ### Breaking Changes
139
+
140
+ #### RequestScopedBuffer API (affects only direct usage)
141
+
142
+ If you use `E11y::Buffers::RequestScopedBuffer` directly in your code, update method names:
143
+
144
+ **Before (incorrect):**
145
+ ```ruby
146
+ E11y::Buffers::RequestScopedBuffer.start! # ❌ Wrong
147
+ E11y::Buffers::RequestScopedBuffer.flush! # ❌ Wrong
148
+ E11y::Buffers::RequestScopedBuffer.flush_on_error! # ❌ Wrong
149
+ ```
150
+
151
+ **After (correct):**
152
+ ```ruby
153
+ E11y::Buffers::RequestScopedBuffer.initialize! # ✅ Correct
154
+ E11y::Buffers::RequestScopedBuffer.discard # ✅ Correct
155
+ E11y::Buffers::RequestScopedBuffer.flush_on_error # ✅ Correct
156
+ ```
157
+
158
+ **Impact:** LOW - RequestScopedBuffer is an internal API. Most users are not affected as the middleware, ActiveJob, and Sidekiq instrumentations are already updated. Only users who directly call these methods need to update their code.
159
+
160
+ **Migration:** Search your codebase for `RequestScopedBuffer` and update method names if found.
161
+
162
+ ---
163
+
164
+ ## [0.1.0] - 2026-01-17
165
+
166
+ Initial release of E11y - Event-driven observability for Rails applications.
167
+
168
+ ### Features
169
+
170
+ - **Core Event System**
171
+ - Unified event API with dry-schema validation
172
+ - Type-safe event schemas
173
+ - Extensible adapter architecture
174
+ - Request-scoped debug buffering
175
+
176
+ - **Rails Integration**
177
+ - Automatic Rails instrumentation
178
+ - ActiveJob tracking
179
+ - Sidekiq middleware
180
+ - Rails.logger bridge
181
+ - Trace context propagation
182
+
183
+ - **Adapters**
184
+ - Loki (logs)
185
+ - Prometheus (metrics)
186
+ - Sentry (errors)
187
+ - OpenTelemetry (traces)
188
+ - Elasticsearch (search)
189
+ - Redis (fast writes)
190
+ - Audit log (encrypted storage)
191
+
192
+ - **Advanced Features**
193
+ - Event-level metrics (metrics DSL)
194
+ - PII filtering with configurable rules
195
+ - Stratified sampling
196
+ - Rate limiting
197
+ - High-cardinality protection
198
+ - Error handling with retry and DLQ
199
+
200
+ - **Developer Experience**
201
+ - RSpec matchers for testing
202
+ - InMemory test adapter
203
+ - Zero-config defaults
204
+ - Comprehensive documentation
205
+ - 25+ ADRs and use cases
206
+
207
+ ### Supported Versions
208
+
209
+ - Ruby: 3.2, 3.3
210
+ - Rails: 8.0
211
+ - RSpec: 3.13+
212
+
213
+ ### Documentation
214
+
215
+ - 17 Architecture Decision Records (ADRs)
216
+ - 22 Use Cases with examples
217
+ - Complete API documentation
218
+ - Testing guide
219
+ - Migration guides
data/CLAUDE.md ADDED
@@ -0,0 +1,168 @@
1
+ # CLAUDE.md
2
+
3
+ This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
+
5
+ ## Project Overview
6
+
7
+ E11y ("easy telemetry") is a Ruby gem providing observability for Rails apps. Its key differentiator is **request-scoped debug buffering**: debug logs accumulate in memory during a request and flush to storage only if the request fails, cutting noise by ~90%.
8
+
9
+ - **Ruby**: 3.2+, **Rails**: 7.0–8.0 (8.1 excluded due to a sqlite3 bug)
10
+ - **Gem version**: 0.2.0, current branch: `feat/integration-testing`
11
+
12
+ ## Commands
13
+
14
+ ```bash
15
+ # Run all tests
16
+ rake spec:all
17
+
18
+ # Run unit tests only (fast, no Rails required)
19
+ rake spec:unit
20
+
21
+ # Run integration tests (requires --with integration bundle group)
22
+ rake spec:integration
23
+
24
+ # Integration tests for Loki/OTel adapters require services — start with docker compose:
25
+ docker compose up -d loki otel-collector
26
+ INTEGRATION=true bundle exec rspec spec/integration/critical_adapters_integration_spec.rb
27
+
28
+ # Run a single spec file
29
+ bundle exec rspec spec/e11y/adapters/loki_adapter_spec.rb
30
+
31
+ # Run a single example by line number
32
+ bundle exec rspec spec/e11y/adapters/loki_adapter_spec.rb:42
33
+
34
+ # Lint
35
+ bundle exec rubocop
36
+
37
+ # Lint with autocorrect
38
+ bundle exec rubocop -a
39
+
40
+ # Install integration dependencies (needed before rake spec:integration)
41
+ bundle install --with integration
42
+
43
+ # Open a console with the gem loaded
44
+ rake console
45
+
46
+ # Run benchmarks
47
+ rake spec:benchmark
48
+
49
+ # Run Cucumber acceptance tests
50
+ rake cucumber
51
+ # Or: bundle exec cucumber features/
52
+
53
+ # Cucumber with Loki (adapter_configurations.feature): start Loki + OTel first
54
+ docker compose up -d loki otel-collector
55
+ rake cucumber
56
+
57
+ # Run TUI (interactive log viewer)
58
+ bundle exec e11y
59
+
60
+ # Start MCP server (for Cursor / Claude Code)
61
+ bundle exec e11y mcp
62
+
63
+ # Stream events to stdout
64
+ bundle exec e11y tail
65
+ ```
66
+
67
+ ## Architecture
68
+
69
+ ### Event Processing Pipeline
70
+
71
+ Every event flows through a middleware pipeline before reaching an adapter:
72
+
73
+ ```
74
+ Event.track(data)
75
+ → Validation (dry-schema)
76
+ → Sampling (adaptive: error-spike, load, value-based)
77
+ → PII Filtering (mask/hash sensitive fields)
78
+ → Trace Context (attach OTel span/trace IDs)
79
+ → Routing (direct to adapters by severity/type)
80
+ → Rate Limiting
81
+ → Audit Signing
82
+ → Adapter(s): Loki | Sentry | OpenTelemetry | Yabeda | File | Stdout | InMemory
83
+ ```
84
+
85
+ Pipeline is built in `lib/e11y/pipeline/builder.rb`. Middleware order matters — see ADR-015.
86
+
87
+ ### Key Modules
88
+
89
+ | Path | Role |
90
+ |------|------|
91
+ | `lib/e11y.rb` | Public API, `E11y.configure`, `E11y.configuration`, `E11y.logger` |
92
+ | `lib/e11y/event/base.rb` | Base event class; all user events inherit from this |
93
+ | `lib/e11y/adapters/` | Backend adapters (Loki, Sentry, OTel, Yabeda, File, Stdout, InMemory) |
94
+ | `lib/e11y/middleware/` | 11 pipeline stages (validation, sampling, PII, routing, etc.) |
95
+ | `lib/e11y/buffers/` | Request-scoped buffer + adaptive buffer implementations |
96
+ | `lib/e11y/pipeline/builder.rb` | Assembles middleware chain from configuration |
97
+ | `lib/e11y/railtie.rb` | Rails integration entry point |
98
+ | `lib/e11y/pii/` | PII detection patterns and masking/hashing strategies |
99
+ | `lib/e11y/sampling/` | Error-spike, load-based, value-based sampling strategies |
100
+ | `lib/e11y/reliability/` | Circuit breaker, DLQ (dead letter queue), retry logic |
101
+ | `lib/e11y/slo/` | Event-driven SLO tracking |
102
+ | `lib/e11y/metrics/` | Prometheus metrics registry with cardinality protection |
103
+ | `gems/e11y-devtools/` | Developer tools gem (TUI, Browser Overlay, MCP) — dev-only |
104
+ | `gems/e11y-devtools/lib/e11y/devtools/tui/` | ratatui_ruby TUI — interaction-centric log viewer |
105
+ | `gems/e11y-devtools/lib/e11y/devtools/overlay/` | Rails Engine — floating badge + slide-in panel |
106
+ | `gems/e11y-devtools/lib/e11y/devtools/mcp/` | MCP Server — AI integration for Cursor/Claude Code |
107
+ | `lib/e11y/adapters/dev_log.rb` | DevLog adapter — JSONL write+read, shared by all viewers |
108
+
109
+ ### Event Definition Pattern
110
+
111
+ Events are defined as classes inheriting from `E11y::Event::Base`:
112
+
113
+ ```ruby
114
+ class Events::OrderCreated < E11y::Event::Base
115
+ schema do
116
+ required(:order_id).filled(:string)
117
+ required(:amount).filled(:float)
118
+ optional(:user_id).maybe(:string)
119
+ end
120
+
121
+ metrics do
122
+ counter :orders_created_total, "Orders created"
123
+ histogram :order_amount, "Order amount in USD", buckets: [10, 50, 100, 500]
124
+ end
125
+ end
126
+
127
+ # Usage
128
+ Events::OrderCreated.track(order_id: order.id, amount: order.total)
129
+ ```
130
+
131
+ ### Adapter Routing
132
+
133
+ Adapters are registered in configuration and events route to them by severity:
134
+ - `error`/`fatal` → errors tracker (e.g., Sentry)
135
+ - other severities → logs (e.g., Loki)
136
+ - metrics always → Yabeda/Prometheus
137
+
138
+ ### Request-Scoped Buffering
139
+
140
+ The buffer middleware captures debug-level events in a `Concurrent::Array` per request (stored in `Thread.current`). On request success: buffer discarded. On request failure: buffer flushed to configured adapters. Controlled by `config.ephemeral_buffer_enabled`.
141
+
142
+ ## Test Structure
143
+
144
+ - `spec/e11y/` — Unit tests (86 files, ~1672 examples, fast)
145
+ - `spec/integration/` — Integration tests against a real Rails app (~36 examples)
146
+ - `spec/dummy/` — Minimal Rails app used by integration tests
147
+ - `spec/dummy/app/events/events/` — Event class definitions for test fixtures
148
+ - `spec/support/matchers/` — Custom RSpec matchers including PII matchers
149
+ - `spec/fixtures/pii_samples.yml` — PII test data
150
+
151
+ Integration tests use `DatabaseCleaner` and require `--with integration` bundle group.
152
+
153
+ ## Code Conventions
154
+
155
+ - Frozen string literals everywhere (`# frozen_string_literal: true`)
156
+ - Double-quoted strings (enforced by RuboCop)
157
+ - Events use class-level `.track` (not instantiation) — zero-allocation design; data stored in plain Hashes
158
+ - Adapters inherit from `E11y::Adapters::Base` and implement `#deliver(event_data)`
159
+ - Middleware inherits from `E11y::Middleware::Base` and implements `#call(event, pipeline)`
160
+
161
+ ## Architecture Decision Records
162
+
163
+ The `docs/architecture/ADR-*.md` files document design decisions. Key ones:
164
+ - **ADR-004**: Adapter architecture
165
+ - **ADR-011**: Testing strategy
166
+ - **ADR-013**: Reliability and error handling
167
+ - **ADR-015**: Middleware ordering (critical — changing order breaks the pipeline)
168
+ - **ADR-017**: Multi-Rails compatibility approach