e11y 0.2.0 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (288) hide show
  1. checksums.yaml +4 -4
  2. data/.rubocop.yml +130 -10
  3. data/CHANGELOG.md +80 -1
  4. data/CLAUDE.md +168 -0
  5. data/CONTRIBUTING.md +640 -0
  6. data/README.md +165 -701
  7. data/RELEASE.md +41 -12
  8. data/Rakefile +249 -57
  9. data/config/README.md +1 -1
  10. data/config/loki-local-config.yaml +12 -0
  11. data/config/otel-collector-config.yaml +44 -0
  12. data/cucumber.yml +1 -0
  13. data/docker-compose.yml +18 -2
  14. data/docs/ADAPTERS.md +76 -0
  15. data/docs/ADAPTIVE_SAMPLING.md +59 -0
  16. data/docs/COMPARISON.md +104 -0
  17. data/docs/CONFIGURATION.md +52 -0
  18. data/docs/DISTRIBUTED_TRACING.md +44 -0
  19. data/docs/LIMITATIONS.md +13 -0
  20. data/docs/METRICS_DSL.md +84 -0
  21. data/docs/PERFORMANCE.md +60 -0
  22. data/docs/PII_FILTERING.md +40 -0
  23. data/docs/PRESETS.md +65 -0
  24. data/docs/QUICK-START.md +546 -587
  25. data/docs/RAILS_INTEGRATION.md +79 -0
  26. data/docs/SCHEMA_VALIDATION.md +63 -0
  27. data/docs/SLO-PROMQL-ALERTS.md +161 -0
  28. data/docs/TESTING.md +69 -0
  29. data/docs/{ADR-001-architecture.md → architecture/ADR-001-architecture.md} +36 -65
  30. data/docs/{ADR-002-metrics-yabeda.md → architecture/ADR-002-metrics-yabeda.md} +62 -236
  31. data/docs/architecture/ADR-003-slo-observability.md +1402 -0
  32. data/docs/{ADR-004-adapter-architecture.md → architecture/ADR-004-adapter-architecture.md} +163 -146
  33. data/docs/{ADR-005-tracing-context.md → architecture/ADR-005-tracing-context.md} +10 -9
  34. data/docs/{ADR-006-security-compliance.md → architecture/ADR-006-security-compliance.md} +184 -191
  35. data/docs/{ADR-007-opentelemetry-integration.md → architecture/ADR-007-opentelemetry-integration.md} +3 -21
  36. data/docs/{ADR-008-rails-integration.md → architecture/ADR-008-rails-integration.md} +182 -743
  37. data/docs/{ADR-009-cost-optimization.md → architecture/ADR-009-cost-optimization.md} +45 -54
  38. data/docs/architecture/ADR-010-developer-experience.md +522 -0
  39. data/docs/{ADR-011-testing-strategy.md → architecture/ADR-011-testing-strategy.md} +44 -86
  40. data/docs/{ADR-012-event-evolution.md → architecture/ADR-012-event-evolution.md} +11 -11
  41. data/docs/{ADR-013-reliability-error-handling.md → architecture/ADR-013-reliability-error-handling.md} +37 -12
  42. data/docs/{ADR-014-event-driven-slo.md → architecture/ADR-014-event-driven-slo.md} +12 -24
  43. data/docs/{ADR-015-middleware-order.md → architecture/ADR-015-middleware-order.md} +43 -59
  44. data/docs/{ADR-016-self-monitoring-slo.md → architecture/ADR-016-self-monitoring-slo.md} +58 -355
  45. data/docs/{ADR-017-multi-rails-compatibility.md → architecture/ADR-017-multi-rails-compatibility.md} +4 -11
  46. data/docs/architecture/ADR-018-memory-optimization.md +366 -0
  47. data/docs/{ADR-INDEX.md → architecture/ADR-INDEX.md} +11 -6
  48. data/docs/plans/2026-03-20-browser-overlay-svelte.md +281 -0
  49. data/docs/{00-ICP-AND-TIMELINE.md → prd/00-ICP-AND-TIMELINE.md} +6 -6
  50. data/docs/{01-SCALE-REQUIREMENTS.md → prd/01-SCALE-REQUIREMENTS.md} +6 -6
  51. data/docs/prd/01-overview-vision.md +19 -14
  52. data/docs/use_cases/README.md +22 -23
  53. data/docs/use_cases/UC-001-request-scoped-debug-buffering.md +50 -44
  54. data/docs/use_cases/UC-002-business-event-tracking.md +26 -95
  55. data/docs/use_cases/UC-003-event-metrics.md +66 -0
  56. data/docs/use_cases/UC-004-zero-config-slo-tracking.md +33 -684
  57. data/docs/use_cases/UC-005-sentry-integration.md +13 -15
  58. data/docs/use_cases/UC-006-trace-context-management.md +30 -28
  59. data/docs/use_cases/UC-007-pii-filtering.md +35 -87
  60. data/docs/use_cases/UC-008-opentelemetry-integration.md +51 -89
  61. data/docs/use_cases/UC-009-multi-service-tracing.md +30 -178
  62. data/docs/use_cases/UC-010-background-job-tracking.md +24 -91
  63. data/docs/use_cases/UC-011-rate-limiting.md +95 -168
  64. data/docs/use_cases/UC-012-audit-trail.md +21 -46
  65. data/docs/use_cases/UC-013-high-cardinality-protection.md +29 -167
  66. data/docs/use_cases/UC-014-adaptive-sampling.md +2 -2
  67. data/docs/use_cases/UC-015-cost-optimization.md +46 -99
  68. data/docs/use_cases/UC-016-rails-logger-migration.md +39 -213
  69. data/docs/use_cases/UC-017-local-development.md +203 -777
  70. data/docs/use_cases/UC-018-testing-events.md +3 -3
  71. data/docs/use_cases/UC-019-retention-based-routing.md +53 -106
  72. data/docs/use_cases/UC-020-event-versioning.md +8 -9
  73. data/docs/use_cases/UC-021-error-handling-retry-dlq.md +18 -22
  74. data/docs/use_cases/UC-022-event-registry.md +15 -21
  75. data/docs/use_cases/backlog.md +119 -87
  76. data/e11y.gemspec +2 -2
  77. data/gems/e11y-devtools/README.md +158 -0
  78. data/gems/e11y-devtools/config/routes.rb +15 -0
  79. data/gems/e11y-devtools/e11y-devtools.gemspec +25 -0
  80. data/gems/e11y-devtools/exe/e11y +34 -0
  81. data/gems/e11y-devtools/frontend/.gitignore +24 -0
  82. data/gems/e11y-devtools/frontend/README.md +51 -0
  83. data/gems/e11y-devtools/frontend/index.html +14 -0
  84. data/gems/e11y-devtools/frontend/package-lock.json +3707 -0
  85. data/gems/e11y-devtools/frontend/package.json +28 -0
  86. data/gems/e11y-devtools/frontend/public/mocks/v1/events/recent.json +4205 -0
  87. data/gems/e11y-devtools/frontend/public/mocks/v1/interactions.json +194 -0
  88. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/0a2e04027cfa22d014bc22e8b27cd913/events.json +86 -0
  89. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/0e1543af6a630fb3af6b52283154b3e0/events.json +169 -0
  90. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/1838b691faa49564f97db8592ff3978d/events.json +78 -0
  91. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/29f198f6588dacffb687777eb5f8f118/events.json +197 -0
  92. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/34bc3c9c0097de28a7a6f99b90a8e7bc/events.json +194 -0
  93. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/3ba6c20d068ab9cee00e51b180e66444/events.json +184 -0
  94. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/435bfd8f17b9009146a79812d7c3726d/events.json +144 -0
  95. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/4c7676e3fe668e99edb2b94d7d5678a9/events.json +222 -0
  96. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/6daf0d47974bedfc55d5de7004a3ea9f/events.json +194 -0
  97. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/8a81ada42834d15f287bb40010043605/events.json +194 -0
  98. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/8c0a98900edaae105469df8daedccf02/events.json +198 -0
  99. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/8e4f645180f8a7d1dce426b07380466b/events.json +222 -0
  100. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/93db346fa5d44a032605a13b627f4b80/events.json +128 -0
  101. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/98ff6146faf7bd9be8bd03a8275817ba/events.json +223 -0
  102. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/9997ddd0247bc7e25f2ca7a5c415c93d/events.json +197 -0
  103. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/99e35f8ef3baedd798cc4fd085980ad9/events.json +194 -0
  104. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/b4f3095c1909924cbc98889a86c83d6d/events.json +131 -0
  105. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/b54b7fc32b7575a7110de809d11ccda0/events.json +128 -0
  106. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/c0b48033fa06746bcc5886745e053cff/events.json +169 -0
  107. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/c44649ac76701b4558927cd2305ab535/events.json +169 -0
  108. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/d601ae3320057580a39dbdac2edfdf4a/events.json +248 -0
  109. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/e67e724bab422d2b52eeb49635e512e1/events.json +194 -0
  110. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/e6c72765a28f158a8485b35fa63f73da/events.json +194 -0
  111. data/gems/e11y-devtools/frontend/public/mocks/v1/traces/f541b87405c9a54819b18ebe529f6419/events.json +194 -0
  112. data/gems/e11y-devtools/frontend/scripts/generate_mocks.rb +397 -0
  113. data/gems/e11y-devtools/frontend/src/App.svelte +827 -0
  114. data/gems/e11y-devtools/frontend/src/components/Fab.svelte +19 -0
  115. data/gems/e11y-devtools/frontend/src/components/FilterBar.svelte +38 -0
  116. data/gems/e11y-devtools/frontend/src/components/FullscreenPanel.svelte +82 -0
  117. data/gems/e11y-devtools/frontend/src/components/InteractionsTimeline.svelte +264 -0
  118. data/gems/e11y-devtools/frontend/src/components/RecentHistogram.svelte +354 -0
  119. data/gems/e11y-devtools/frontend/src/lib/api.ts +37 -0
  120. data/gems/e11y-devtools/frontend/src/lib/eventIdentity.ts +12 -0
  121. data/gems/e11y-devtools/frontend/src/lib/format.ts +37 -0
  122. data/gems/e11y-devtools/frontend/src/lib/listFilter.ts +43 -0
  123. data/gems/e11y-devtools/frontend/src/lib/recentVolume.ts +80 -0
  124. data/gems/e11y-devtools/frontend/src/lib/router.ts +12 -0
  125. data/gems/e11y-devtools/frontend/src/lib/transitions.ts +34 -0
  126. data/gems/e11y-devtools/frontend/src/lib/viewportOrigin.ts +25 -0
  127. data/gems/e11y-devtools/frontend/src/main.ts +8 -0
  128. data/gems/e11y-devtools/frontend/src/overlay-entry.ts +24 -0
  129. data/gems/e11y-devtools/frontend/src/overlay.css +1080 -0
  130. data/gems/e11y-devtools/frontend/svelte.config.js +2 -0
  131. data/gems/e11y-devtools/frontend/test_puppeteer.js +41 -0
  132. data/gems/e11y-devtools/frontend/test_scale.js +3 -0
  133. data/gems/e11y-devtools/frontend/tsconfig.app.json +21 -0
  134. data/gems/e11y-devtools/frontend/tsconfig.json +7 -0
  135. data/gems/e11y-devtools/frontend/tsconfig.node.json +26 -0
  136. data/gems/e11y-devtools/frontend/vite.config.ts +36 -0
  137. data/gems/e11y-devtools/lib/e11y/devtools/mcp/server.rb +96 -0
  138. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tool_base.rb +25 -0
  139. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/clear.rb +31 -0
  140. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/errors.rb +35 -0
  141. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/event_detail.rb +33 -0
  142. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/events_by_trace.rb +33 -0
  143. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/interactions.rb +40 -0
  144. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/recent_events.rb +34 -0
  145. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/search.rb +34 -0
  146. data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/stats.rb +30 -0
  147. data/gems/e11y-devtools/lib/e11y/devtools/overlay/assets/overlay.js +20 -0
  148. data/gems/e11y-devtools/lib/e11y/devtools/overlay/controller.rb +94 -0
  149. data/gems/e11y-devtools/lib/e11y/devtools/overlay/engine.rb +26 -0
  150. data/gems/e11y-devtools/lib/e11y/devtools/overlay/middleware.rb +80 -0
  151. data/gems/e11y-devtools/lib/e11y/devtools/overlay/rails_controller.rb +67 -0
  152. data/gems/e11y-devtools/lib/e11y/devtools/tui/app.rb +262 -0
  153. data/gems/e11y-devtools/lib/e11y/devtools/tui/grouping.rb +66 -0
  154. data/gems/e11y-devtools/lib/e11y/devtools/tui/widgets/event_detail.rb +62 -0
  155. data/gems/e11y-devtools/lib/e11y/devtools/tui/widgets/event_list.rb +70 -0
  156. data/gems/e11y-devtools/lib/e11y/devtools/tui/widgets/interaction_list.rb +47 -0
  157. data/gems/e11y-devtools/lib/e11y/devtools/version.rb +8 -0
  158. data/gems/e11y-devtools/lib/e11y/devtools.rb +13 -0
  159. data/gems/e11y-devtools/spec/e11y/devtools/mcp/tools_spec.rb +107 -0
  160. data/gems/e11y-devtools/spec/e11y/devtools/overlay/controller_spec.rb +91 -0
  161. data/gems/e11y-devtools/spec/e11y/devtools/overlay/middleware_spec.rb +46 -0
  162. data/gems/e11y-devtools/spec/e11y/devtools/tui/app_spec.rb +85 -0
  163. data/gems/e11y-devtools/spec/e11y/devtools/tui/grouping_spec.rb +64 -0
  164. data/gems/e11y-devtools/spec/spec_helper.rb +5 -0
  165. data/gems/e11y-devtools/spec/tui/widgets/event_list_spec.rb +44 -0
  166. data/gems/e11y-devtools/spec/tui/widgets/interaction_list_spec.rb +62 -0
  167. data/lib/e11y/adapters/audit_encrypted.rb +53 -11
  168. data/lib/e11y/adapters/base.rb +33 -34
  169. data/lib/e11y/adapters/dev_log/file_store.rb +143 -0
  170. data/lib/e11y/adapters/dev_log/query.rb +219 -0
  171. data/lib/e11y/adapters/dev_log.rb +118 -0
  172. data/lib/e11y/adapters/file.rb +3 -6
  173. data/lib/e11y/adapters/in_memory.rb +52 -5
  174. data/lib/e11y/adapters/in_memory_test.rb +29 -0
  175. data/lib/e11y/adapters/loki.rb +58 -23
  176. data/lib/e11y/adapters/null.rb +82 -0
  177. data/lib/e11y/adapters/opentelemetry_collector.rb +183 -0
  178. data/lib/e11y/adapters/otel_logs.rb +136 -23
  179. data/lib/e11y/adapters/sentry.rb +4 -7
  180. data/lib/e11y/adapters/stdout.rb +73 -7
  181. data/lib/e11y/adapters/yabeda.rb +153 -29
  182. data/lib/e11y/buffers/adaptive_buffer.rb +3 -17
  183. data/lib/e11y/buffers/{request_scoped_buffer.rb → ephemeral_buffer.rb} +72 -58
  184. data/lib/e11y/buffers/ring_buffer.rb +3 -16
  185. data/lib/e11y/configuration.rb +272 -0
  186. data/lib/e11y/console.rb +10 -17
  187. data/lib/e11y/current.rb +53 -1
  188. data/lib/e11y/debug/pipeline_inspector.rb +96 -0
  189. data/lib/e11y/documentation/generator.rb +48 -0
  190. data/lib/e11y/event/base.rb +176 -82
  191. data/lib/e11y/event/value_sampling_config.rb +1 -5
  192. data/lib/e11y/events/rails/database/query.rb +1 -4
  193. data/lib/e11y/events/rails/job/failed.rb +2 -0
  194. data/lib/e11y/instruments/active_job.rb +44 -12
  195. data/lib/e11y/instruments/rails_instrumentation.rb +49 -24
  196. data/lib/e11y/instruments/sidekiq.rb +135 -31
  197. data/lib/e11y/linters/base.rb +11 -0
  198. data/lib/e11y/linters/pii/pii_declaration_linter.rb +120 -0
  199. data/lib/e11y/linters/slo/config_consistency_linter.rb +76 -0
  200. data/lib/e11y/linters/slo/explicit_declaration_linter.rb +36 -0
  201. data/lib/e11y/linters/slo/slo_status_from_linter.rb +41 -0
  202. data/lib/e11y/logger/bridge.rb +26 -7
  203. data/lib/e11y/metrics/cardinality_protection.rb +10 -15
  204. data/lib/e11y/metrics/cardinality_tracker.rb +16 -6
  205. data/lib/e11y/metrics/registry.rb +3 -5
  206. data/lib/e11y/metrics/test_backend.rb +62 -0
  207. data/lib/e11y/metrics.rb +56 -10
  208. data/lib/e11y/middleware/adapter_resolver.rb +40 -0
  209. data/lib/e11y/middleware/audit_signing.rb +43 -6
  210. data/lib/e11y/middleware/baggage_protection.rb +75 -0
  211. data/lib/e11y/middleware/dev_log_source.rb +24 -0
  212. data/lib/e11y/middleware/event_slo.rb +23 -9
  213. data/lib/e11y/middleware/otel_span.rb +23 -0
  214. data/lib/e11y/middleware/pii_filter.rb +104 -75
  215. data/lib/e11y/middleware/rate_limiting.rb +54 -27
  216. data/lib/e11y/middleware/request.rb +70 -23
  217. data/lib/e11y/middleware/routing.rb +78 -21
  218. data/lib/e11y/middleware/sampling.rb +66 -17
  219. data/lib/e11y/middleware/self_monitoring_emit.rb +39 -0
  220. data/lib/e11y/middleware/trace_context.rb +45 -10
  221. data/lib/e11y/middleware/track_latency.rb +34 -0
  222. data/lib/e11y/middleware/validation.rb +7 -16
  223. data/lib/e11y/middleware/versioning.rb +26 -22
  224. data/lib/e11y/opentelemetry/semantic_conventions.rb +109 -0
  225. data/lib/e11y/opentelemetry/span_creator.rb +142 -0
  226. data/lib/e11y/pii/patterns.rb +12 -1
  227. data/lib/e11y/pipeline/builder.rb +4 -4
  228. data/lib/e11y/presets/audit_event.rb +13 -2
  229. data/lib/e11y/railtie.rb +52 -14
  230. data/lib/e11y/registry.rb +306 -0
  231. data/lib/e11y/reliability/circuit_breaker.rb +19 -21
  232. data/lib/e11y/reliability/dlq/base.rb +71 -0
  233. data/lib/e11y/reliability/dlq/file_adapter.rb +301 -0
  234. data/lib/e11y/reliability/dlq/file_storage.rb +63 -34
  235. data/lib/e11y/reliability/dlq/filter.rb +37 -54
  236. data/lib/e11y/reliability/retry_handler.rb +26 -29
  237. data/lib/e11y/reliability/retry_rate_limiter.rb +3 -11
  238. data/lib/e11y/sampling/error_spike_detector.rb +0 -2
  239. data/lib/e11y/sampling/load_monitor.rb +5 -9
  240. data/lib/e11y/sampling/stratified_tracker.rb +18 -0
  241. data/lib/e11y/self_monitoring/buffer_monitor.rb +2 -0
  242. data/lib/e11y/self_monitoring/performance_monitor.rb +19 -61
  243. data/lib/e11y/self_monitoring/reliability_monitor.rb +4 -74
  244. data/lib/e11y/slo/config_loader.rb +40 -0
  245. data/lib/e11y/slo/config_validator.rb +58 -0
  246. data/lib/e11y/slo/dashboard_generator.rb +122 -0
  247. data/lib/e11y/slo/event_driven.rb +8 -0
  248. data/lib/e11y/slo/tracker.rb +31 -4
  249. data/lib/e11y/testing/have_tracked_event_matcher.rb +190 -0
  250. data/lib/e11y/testing/rspec_matchers.rb +21 -0
  251. data/lib/e11y/testing/snapshot_matcher.rb +86 -0
  252. data/lib/e11y/trace_context/sampler.rb +35 -0
  253. data/lib/e11y/tracing/faraday_middleware.rb +31 -0
  254. data/lib/e11y/tracing/net_http_patch.rb +33 -0
  255. data/lib/e11y/tracing/propagator.rb +144 -0
  256. data/lib/e11y/tracing.rb +47 -0
  257. data/lib/e11y/version.rb +1 -1
  258. data/lib/e11y/versioning/version_extractor.rb +32 -0
  259. data/lib/e11y.rb +123 -266
  260. data/lib/generators/e11y/event/event_generator.rb +22 -0
  261. data/lib/generators/e11y/event/templates/event.rb.tt +16 -0
  262. data/lib/generators/e11y/grafana_dashboard/grafana_dashboard_generator.rb +30 -0
  263. data/lib/generators/e11y/grafana_dashboard/templates/e11y_dashboard.json +81 -0
  264. data/lib/generators/e11y/install/install_generator.rb +34 -0
  265. data/lib/generators/e11y/install/templates/e11y.rb +239 -0
  266. data/lib/generators/e11y/prometheus_alerts/prometheus_alerts_generator.rb +29 -0
  267. data/lib/generators/e11y/prometheus_alerts/templates/e11y_alerts.yml +28 -0
  268. data/lib/tasks/e11y_docs.rake +30 -0
  269. data/lib/tasks/e11y_events.rake +71 -0
  270. data/lib/tasks/e11y_lint.rake +91 -0
  271. data/lib/tasks/e11y_slo.rake +29 -0
  272. metadata +186 -39
  273. data/docs/ADR-003-slo-observability.md +0 -3337
  274. data/docs/ADR-010-developer-experience.md +0 -2166
  275. data/docs/API-REFERENCE-L28.md +0 -914
  276. data/docs/COMPREHENSIVE-CONFIGURATION.md +0 -2366
  277. data/docs/CONTRIBUTING.md +0 -312
  278. data/docs/IMPLEMENTATION_NOTES.md +0 -2804
  279. data/docs/IMPLEMENTATION_PLAN.md +0 -1971
  280. data/docs/IMPLEMENTATION_PLAN_ARCHITECTURE.md +0 -586
  281. data/docs/PLAN.md +0 -148
  282. data/docs/README.md +0 -296
  283. data/docs/design/00-memory-optimization.md +0 -593
  284. data/docs/guides/MIGRATION-L27-L28.md +0 -692
  285. data/docs/guides/PERFORMANCE-BENCHMARKS.md +0 -434
  286. data/docs/guides/README.md +0 -44
  287. data/docs/use_cases/UC-003-pattern-based-metrics.md +0 -1627
  288. data/lib/e11y/adapters/registry.rb +0 -141
data/docs/QUICK-START.md CHANGED
@@ -1,234 +1,265 @@
1
1
  # E11y - Quick Start Guide
2
2
 
3
- > **TL;DR**: Ruby gem для структурированных бизнес-событий с request-scoped debug buffering, pattern-based метриками и pluggable адаптерами.
3
+ > **TL;DR**: Ruby gem for structured business events with request-scoped debug buffering,
4
+ > schema validation, and pluggable adapters (Loki, Sentry, OpenTelemetry, Yabeda, etc.).
5
+ >
6
+ > **Current version: 0.2.0** — ⚠️ Work in Progress, production validation in progress.
4
7
 
5
8
  ---
6
9
 
7
- ## 🚀 Installation (5 minutes)
10
+ ## 🚀 Installation
8
11
 
9
12
  ```bash
10
13
  # Gemfile
11
- gem 'e11y', '~> 1.0'
14
+ gem 'e11y', '~> 0.2'
12
15
 
13
16
  bundle install
17
+ ```
18
+
19
+ Then run the generator to create the initializer and example event:
20
+
21
+ ```bash
14
22
  rails g e11y:install
15
23
  ```
16
24
 
25
+ This creates:
26
+ - `config/initializers/e11y.rb` — base configuration with comments
27
+ - `app/events/` — directory for event classes (if it doesn't exist)
28
+
29
+ > **Available generators:**
30
+ > - `rails g e11y:install` — initializer + directory scaffold
31
+ > - `rails g e11y:grafana_dashboard` — Grafana dashboard JSON (requires Yabeda/Prometheus)
32
+ > - `rails g e11y:prometheus_alerts` — Prometheus alerting rules
33
+
34
+ Or create the initializer manually:
35
+
36
+ ```bash
37
+ touch config/initializers/e11y.rb
38
+ ```
39
+
17
40
  ---
18
41
 
19
42
  ## 🎯 Killer Features
20
43
 
21
44
  ### 1. Request-Scoped Debug Buffering
22
45
 
23
- **Проблема**: Debug логи в production = шум. Без debug = нет контекста при ошибках.
46
+ **Problem**: Debug logs in production = noise. Without debug = no context when errors occur.
24
47
 
25
- **Решение**: Debug события буферизируются в thread-local storage. При success - дропаются, при error - флашатся.
48
+ **Solution**: Debug events are buffered in thread-local storage. On success discarded.
49
+ On error — flushed with full context.
26
50
 
27
- ```ruby
51
+ ```
28
52
  GET /api/orders/123
29
53
  ├─ [debug] Query: SELECT... (buffered, NOT sent)
30
54
  ├─ [debug] Cache miss (buffered, NOT sent)
31
- ├─ [ERROR] Payment failed ← Exception!
32
- └─> FLUSH все buffered debug события!
55
+ ├─ [ERROR] Payment failed ← 5xx raised
56
+ └─> FLUSH: all buffered debug events are sent to the adapter!
33
57
  ```
34
58
 
35
- **Результат**: Debug логи только когда нужны, zero overhead на happy path.
59
+ > **Note:** By default the buffer flushes only on **5xx server errors** (`flush_on_error = true`).
60
+ > On 4xx responses and successful requests the buffer is discarded.
61
+ >
62
+ > Two independent knobs let you customize this — see [Configuration](#-configuration):
63
+ > - `flush_on_error = false` — disable the 5xx auto-flush
64
+ > - `flush_on_statuses = [403]` — add extra statuses (independent of `flush_on_error`)
65
+
66
+ **Result**: Debug logs only when needed, zero overhead on the happy path.
36
67
 
37
- ### 2. :success Pseudo-Severity
68
+ ### 2. `:success` Pseudo-Severity
38
69
 
39
- Новый severity level между :info и :warn для успешных операций.
70
+ A new severity level between `:info` and `:warn` for successful operations.
40
71
 
41
72
  ```ruby
42
- Events::OrderPaid.track(
43
- order_id: '123',
44
- severity: :success # ← легко фильтровать успехи
45
- )
73
+ Events::OrderPaid.track(order_id: '123', amount: 99.99, currency: 'USD')
74
+ # severity is automatically :success (inferred from class name — "Paid")
46
75
 
47
- # В Grafana/Kibana:
48
- severity:success AND event_name:order.paid
76
+ # Or explicitly:
77
+ Events::SomeEvent.track(order_id: '123', severity: :success)
49
78
  ```
50
79
 
51
- **Результат**: Видимость успехов, не только ошибок. Легко построить success rate.
80
+ **In Grafana/Kibana:**
81
+ ```
82
+ {severity="success", event_name="OrderPaid"}
83
+ ```
52
84
 
53
- ### 3. Pattern-Based Metrics
85
+ **Result**: Visibility into successes, not just errors. Easy to build success rate dashboards.
54
86
 
55
- Вместо явных `metric :counter` на каждое событие - паттерны:
87
+ ### 3. Schema Validation (dry-schema)
56
88
 
57
89
  ```ruby
58
- E11y.configure do |config|
59
- config.metrics do
60
- # Автоматически для всех событий
61
- counter_for pattern: '*', name: 'events_total'
62
-
63
- # Histogram для всех оплат
64
- histogram_for pattern: '*.paid',
65
- value: ->(e) { e.payload[:amount] }
90
+ class Events::OrderPaid < E11y::Event::Base
91
+ schema do
92
+ required(:order_id).filled(:string)
93
+ required(:amount).filled(:float, gt?: 0)
94
+ required(:currency).filled(:string, included_in?: %w[USD EUR GBP])
66
95
  end
67
96
  end
68
- ```
69
97
 
70
- **Результат**: Метрики без boilerplate.
98
+ Events::OrderPaid.track(order_id: nil, amount: -10)
99
+ # => E11y::ValidationError: Validation failed for OrderPaid: {:order_id=>["must be filled"], ...}
100
+ ```
71
101
 
72
- ### 4. Trace Context (OpenTelemetry + Sentry)
102
+ ### 4. Trace Context (W3C + OTel + Sentry fallback)
73
103
 
74
- Автоматическое извлечение trace_id с fallback chain:
104
+ Automatic trace_id extraction with fallback chain:
75
105
 
76
- ```ruby
77
- # Priority:
78
- 1. X-Trace-ID header
106
+ ```
107
+ 1. traceparent header (W3C Trace Context)
79
108
  2. X-Request-ID header
80
- 3. OpenTelemetry span context
81
- 4. Sentry trace ID
82
- 5. Generate new UUID v7
109
+ 3. X-Trace-Id header (custom)
110
+ 4. Generate new UUID
83
111
  ```
84
112
 
85
- **Результат**: Связанные события across services.
86
-
87
- ### 5. Built-in SLO Tracking (Zero Config!)
113
+ **Result**: Correlated events across services with no extra code.
88
114
 
89
- Включил флаг получил SLO metrics из коробки:
115
+ ### 5. Built-in SLO Tracking
90
116
 
91
117
  ```ruby
92
118
  E11y.configure do |config|
93
- config.slo_tracking = true # ← ВСЁ!
119
+ config.rails_instrumentation_enabled = true
120
+ # config.slo_tracking_enabled = true # enabled by default
94
121
  end
95
122
 
96
- # Автоматически:
97
- # HTTP availability + latency
98
- # Sidekiq jobs success rate + duration
99
- # ActiveJob success rate + duration
100
- # ✅ Error budget + burn rate
101
- # ✅ Grafana dashboards (generate)
102
- # ✅ Prometheus alerts (generate)
123
+ # Automatically emits metrics:
124
+ # e11y_http_requests_total, e11y_http_request_duration_seconds
125
+ # e11y_sidekiq_jobs_total, e11y_sidekiq_job_duration_seconds
126
+ # e11y_active_jobs_total, e11y_active_job_duration_seconds
103
127
  ```
104
128
 
105
- **Результат**: Production-ready SLO monitoring без написания middleware.
129
+ > ⚠️ **Caveat:** SLO metrics may be imprecise when adaptive sampling is enabled
130
+ > (sampling correction is planned for Phase 2.8).
131
+
132
+ > 🚧 **Roadmap:** Per-controller/per-job SLO configuration, auto-generated Grafana dashboards
133
+ > and Prometheus alerts — planned for future releases.
106
134
 
107
135
  ---
108
136
 
109
137
  ## ⚡ Quick Examples
110
138
 
111
- ### Basic Event (v1.1 - Event-Level Configuration!)
112
-
113
- > **🎯 NEW in v1.1:** Event-level configuration reduces global config from 1400+ to <300 lines!
139
+ ### Basic Event (minimal configuration)
114
140
 
115
141
  ```ruby
116
- # 1. Define event (простой Ruby класс + конфигурация)
142
+ # 1. Define the event (app/events/order_paid.rb)
117
143
  class Events::OrderPaid < E11y::Event::Base
118
- # Schema (dry-schema для валидации)
119
144
  schema do
120
145
  required(:order_id).filled(:string)
121
- required(:amount).filled(:decimal)
146
+ required(:amount).filled(:float)
122
147
  required(:currency).filled(:string)
123
148
  end
124
-
125
- # ✨ NEW: Event-level configuration (right next to schema!)
126
- severity :success
127
- rate_limit 1000, window: 1.second
128
- sample_rate 1.0 # Never sample payments
129
- retention 7.years # Financial records
130
- adapters [:loki, :sentry, :s3_archive]
131
-
132
- # Metric definition
133
- metric :counter,
134
- name: 'orders.paid.total',
135
- tags: [:currency]
149
+
150
+ # Optional: Prometheus metrics via Yabeda
151
+ # NOTE: The metrics do...end DSL requires the Yabeda adapter to be registered.
152
+ # Without E11y::Adapters::Yabeda.new in config.adapters, metric definitions are
153
+ # stored but never updated. See the Yabeda / Prometheus Integration section below.
154
+ metrics do
155
+ counter :orders_paid_total, tags: [:currency]
156
+ histogram :order_amount, value: :amount, tags: [:currency],
157
+ buckets: [10, 50, 100, 500, 1000]
158
+ end
136
159
  end
137
160
 
138
- # 2. Track - ТОЛЬКО ТАК, больше никаких вариантов!
161
+ # 2. Track only this form, no alternatives
139
162
  Events::OrderPaid.track(
140
163
  order_id: '123',
141
164
  amount: 99.99,
142
165
  currency: 'USD'
143
166
  )
144
-
145
- # ❌ НЕТ других способов:
146
- # E11y.track_event(...) - НЕТ!
147
- # Severity.track(...) - НЕТ!
148
167
  ```
149
168
 
150
- **90% событий нужен ТОЛЬКО schema (zero config!):**
169
+ **90% of events just a schema, everything else from conventions:**
151
170
 
152
171
  ```ruby
153
- # Conventions = sensible defaults!
154
172
  class Events::OrderCreated < E11y::Event::Base
155
173
  schema do
156
174
  required(:order_id).filled(:string)
157
- required(:amount).filled(:decimal)
175
+ required(:amount).filled(:float)
176
+ end
177
+ # severity: :success (auto, inferred from "Created")
178
+ # adapters: [:logs] (auto, from severity via adapter_mapping)
179
+ # sample_rate: 0.1 (auto, from severity)
180
+ # retention_period: 30.days (from config.default_retention_period)
181
+ end
182
+ ```
183
+
184
+ **Explicit configuration at the event-class level:**
185
+
186
+ ```ruby
187
+ class Events::PaymentSucceeded < E11y::Event::Base
188
+ schema do
189
+ required(:transaction_id).filled(:string)
190
+ required(:amount).filled(:float)
158
191
  end
159
- # ← That's it! All config from conventions:
160
- # severity: :success (from name)
161
- # adapters: [:loki] (from severity)
162
- # sample_rate: 0.1 (from severity)
163
- # retention: 30.days (from severity)
164
- # rate_limit: 1000 (default)
192
+
193
+ severity :success
194
+ sample_rate 1.0 # never sample-out payment events
195
+ retention_period 7.years # financial records (also: `retention 7.years` is a valid alias)
196
+ adapters :logs, :errors_tracker
165
197
  end
166
198
  ```
167
199
 
168
- **Inheritance для DRY:**
200
+ **Inheritance for DRY configuration:**
169
201
 
170
202
  ```ruby
171
- # Base class для payment events
172
203
  module Events
173
204
  class BasePaymentEvent < E11y::Event::Base
174
205
  severity :success
175
- sample_rate 1.0 # Never sample
176
- retention 7.years
177
- adapters [:loki, :sentry, :s3_archive]
206
+ sample_rate 1.0
207
+ retention_period 7.years
208
+ adapters :logs, :errors_tracker
178
209
  end
179
210
  end
180
211
 
181
- # Inherit from base (1-2 lines per event!)
182
212
  class Events::PaymentSucceeded < Events::BasePaymentEvent
183
213
  schema do
184
214
  required(:transaction_id).filled(:string)
185
- required(:amount).filled(:decimal)
215
+ required(:amount).filled(:float)
186
216
  end
187
- # Inherits ALL config from BasePaymentEvent!
217
+ # Inherits ALL configuration from BasePaymentEvent
188
218
  end
189
219
  ```
190
220
 
191
- **Preset modules для 1-line includes:**
221
+ **Preset modules:**
192
222
 
193
223
  ```ruby
194
224
  class Events::PaymentProcessed < E11y::Event::Base
195
- include E11y::Presets::HighValueEvent # ← All config inherited!
196
-
225
+ include E11y::Presets::HighValueEvent
226
+ # → severity :success, sample_rate 1.0, adapters [:logs, :errors_tracker]
227
+
197
228
  schema do
198
229
  required(:transaction_id).filled(:string)
199
- required(:amount).filled(:decimal)
230
+ required(:amount).filled(:float)
200
231
  end
201
232
  end
202
- ```
203
233
 
204
- ### With Duration Measurement
234
+ class Events::UserDeleted < E11y::Event::Base
235
+ include E11y::Presets::AuditEvent
236
+ # → sample_rate 1.0, signing enabled, never rate-limited
205
237
 
206
- ```ruby
207
- Events::OrderProcessing.track(order_id: '123') do
208
- # Block execution time measured automatically
209
- process_order(order)
238
+ schema do
239
+ required(:user_id).filled(:string)
240
+ required(:deleted_by).filled(:string)
241
+ end
210
242
  end
211
- # → event.duration_ms = 250
212
243
  ```
213
244
 
214
245
  ### Request-Scoped Debug Buffering
215
246
 
216
247
  ```ruby
217
- # Middleware (auto-configured)
218
248
  class OrdersController < ApplicationController
219
249
  def create
220
- # Debug events buffered (not sent)
221
- Events::ValidationStarted.track(severity: :debug)
250
+ # Debug events are buffered not sent immediately
251
+ Events::ValidationStarted.track(severity: :debug, params: params.keys)
222
252
  Events::DatabaseQuery.track(sql: '...', severity: :debug)
223
-
224
- order = Order.create!(params)
225
-
226
- # Success event sent immediately
227
- Events::OrderCreated.track(order_id: order.id, severity: :success)
228
-
253
+
254
+ order = Order.create!(order_params)
255
+
256
+ # Non-debug events are sent immediately (not buffered)
257
+ Events::OrderCreated.track(order_id: order.id, amount: order.total)
258
+
229
259
  render json: order
260
+ # ← Successful request: debug buffer is discarded
230
261
  rescue => e
231
- # Exception all buffered debug events flushed with severity :error
262
+ # 5xx (or any configured flush status): debug buffer is flushed with error context
232
263
  raise
233
264
  end
234
265
  end
@@ -236,266 +267,186 @@ end
236
267
 
237
268
  ---
238
269
 
239
- ## 🔧 Configuration (v1.1 - Simplified!)
240
-
241
- > **🎯 NEW in v1.1:** Global config reduced from 1400+ to <300 lines!
242
- >
243
- > **Philosophy:** Global config = infrastructure only. Event config = in event classes.
244
-
245
- ### Global Config (Infrastructure Only)
270
+ ## 🔧 Configuration
246
271
 
247
272
  ```ruby
248
- # config/initializers/e11y.rb (<300 lines!)
273
+ # config/initializers/e11y.rb
249
274
  E11y.configure do |config|
250
- # === ADAPTERS REGISTRATION (infrastructure) ===
251
- config.register_adapter :loki, E11y::Adapters::LokiAdapter.new(
275
+ # === Service identity ===
276
+ config.service_name = 'myapp'
277
+ config.environment = Rails.env
278
+
279
+ # === Adapters (key = name, value = instance) ===
280
+ # adapters is a Hash — use [] assignment or register_adapter (both are equivalent):
281
+ config.adapters[:logs] = E11y::Adapters::Loki.new(
252
282
  url: ENV['LOKI_URL'],
253
- labels: { env: Rails.env, service: 'api' }
283
+ batch_size: 100,
284
+ batch_timeout: 5,
285
+ compress: true
254
286
  )
255
-
256
- config.register_adapter :sentry, E11y::Adapters::SentryAdapter.new(
287
+ # Equivalent form:
288
+ # config.register_adapter :logs, E11y::Adapters::Loki.new(url: ENV['LOKI_URL'])
289
+
290
+ config.adapters[:errors_tracker] = E11y::Adapters::Sentry.new(
257
291
  dsn: ENV['SENTRY_DSN']
258
292
  )
259
-
260
- config.register_adapter :s3, E11y::Adapters::S3Adapter.new(
261
- bucket: 'events-archive'
293
+
294
+ # For Prometheus metrics (requires gem 'yabeda' + 'yabeda-prometheus')
295
+ config.adapters[:metrics] = E11y::Adapters::Yabeda.new(
296
+ cardinality_limit: 1000,
297
+ overflow_strategy: :relabel
262
298
  )
263
-
264
- # === DEFAULTS (conventions) ===
265
- config.default_adapters = [:loki] # Most events → Loki
266
- config.default_sample_rate = 0.1 # 10% sampling
267
- config.default_rate_limit = 1000 # 1000 events/sec
268
-
269
- # === REQUEST SCOPE BUFFERING ===
270
- config.request_scope do
271
- enabled true
272
- buffer_limit 100 # max debug events per request
273
- flush_on :error # :error, :always, :never
274
- end
275
-
276
- # === GLOBAL RATE LIMITING (infrastructure) ===
277
- config.rate_limiting do
278
- global limit: 10_000, window: 1.minute
279
- end
280
-
281
- # === CARDINALITY PROTECTION (infrastructure) ===
282
- config.cardinality_protection do
283
- forbidden_labels :user_id, :order_id, :session_id
284
- default_cardinality_limit 100
285
- end
286
-
287
- # === SLO TRACKING (zero config!) ===
288
- config.slo_tracking = true # ← ВСЁ!
289
-
290
- # === AUDIT RETENTION (global default) ===
291
- # Default for audit events, can be overridden per event
292
- config.audit_retention = case ENV['JURISDICTION']
293
- when 'EU' then 7.years # GDPR
294
- when 'US' then 10.years # SOX
295
- else 5.years
296
- end
297
-
298
- # ✅ That's it! No per-event config here anymore!
299
+
300
+ # === Request-scoped buffering ===
301
+ config.ephemeral_buffer_enabled = true
302
+ # flush_on_error (default: true) — flush buffer on any 5xx server error
303
+ # config.ephemeral_buffer_flush_on_error = false # disable 5xx auto-flush
304
+ # flush_on_statuses (default: []) — extra statuses, independent of flush_on_error
305
+ # config.ephemeral_buffer_flush_on_statuses = [403] # also flush on 403 Forbidden
306
+ # config.ephemeral_buffer_flush_on_statuses = [401, 403] # multiple codes
307
+
308
+ # === Rails auto-instrumentation (HTTP, ActiveRecord, ActiveJob, Cache) ===
309
+ config.rails_instrumentation_enabled = true
310
+
311
+ # === SLO tracking (enabled by default) ===
312
+ # config.slo_tracking_enabled = true # already true
313
+
314
+ # === Rate limiting (now in default pipeline!) ===
315
+ # Rate limiting is wired into the default pipeline in v0.2.0.
316
+ # Enable and configure parameters:
317
+ # config.rate_limiting_enabled = true
318
+ # config.rate_limiting_global_limit = 10_000 # events/sec
319
+ # config.rate_limiting_per_event_limit = 1_000 # events/sec per type
320
+ # config.rate_limiting_global_window = 1.0 # seconds
321
+
322
+ # === Retention ===
323
+ config.default_retention_period = 30.days
299
324
  end
325
+
326
+ # Lifecycle methods (v0.2.0):
327
+ # E11y.start! # start background workers (batching, retry, DLQ)
328
+ # at_exit { E11y.stop!(timeout: 5) } # graceful shutdown
300
329
  ```
301
330
 
302
- ### Event-Level Config (Locality of Behavior)
331
+ ### Buffer flush manual trigger
332
+
333
+ `EphemeralBuffer.flush_on_error` is a public method — you can call it directly in custom
334
+ rescue handlers or background jobs:
303
335
 
304
336
  ```ruby
305
- # app/events/order_created.rb
306
- module Events
307
- class OrderCreated < E11y::Event::Base
308
- schema do
309
- required(:order_id).filled(:string)
310
- required(:amount).filled(:decimal)
311
- end
312
-
313
- # ✨ Event-level config (right next to schema!)
314
- severity :success
315
- rate_limit 1000, window: 1.second
316
- sample_rate 0.1 # 10% sampling
317
- retention 30.days
318
- adapters [:loki, :elasticsearch]
319
-
320
- # Metric definition
321
- metric :counter,
322
- name: 'orders.created.total',
323
- tags: [:currency]
324
- end
337
+ # Custom error handler (e.g. Grape API, custom Rack app)
338
+ rescue => e
339
+ E11y::Buffers::EphemeralBuffer.flush_on_error
340
+ raise
325
341
  end
342
+
343
+ # Or flush to a specific adapter target (not yet implemented — placeholder)
344
+ E11y::Buffers::EphemeralBuffer.flush_on_error(target: :errors_tracker)
326
345
  ```
327
346
 
328
- ### Old vs New Config
347
+ ### Severity Adapter mapping
329
348
 
330
- **Before (v1.0): 1400+ lines in global config**
349
+ Default routing:
331
350
 
332
351
  ```ruby
333
- # config/initializers/e11y.rb (1400+ lines!)
352
+ # error/fatal [:logs, :errors_tracker]
353
+ # all others → [:logs]
354
+
355
+ # Override globally:
334
356
  E11y.configure do |config|
335
- # Adapters registration
336
- config.register_adapter :loki, Loki.new(...)
337
- config.register_adapter :sentry, Sentry.new(...)
338
-
339
- # Per-event config (100+ events!)
340
- config.events do
341
- event 'Events::OrderCreated' do
342
- severity :success
343
- adapters [:loki]
344
- sample_rate 0.1
345
- retention 30.days
346
- rate_limit 1000
347
- end
348
-
349
- event 'Events::PaymentSucceeded' do
350
- severity :success
351
- adapters [:loki, :sentry, :s3]
352
- sample_rate 1.0
353
- retention 7.years
354
- rate_limit 1000
355
- end
356
-
357
- # ... 98+ more events ...
358
- end
357
+ config.adapter_mapping[:warn] = [:logs, :errors_tracker]
358
+ end
359
+
360
+ # Override per event:
361
+ class Events::CriticalEvent < E11y::Event::Base
362
+ adapters :logs, :errors_tracker
359
363
  end
360
364
  ```
361
365
 
362
- **After (v1.1): <300 lines in global config**
366
+ ### PII Filtering
363
367
 
364
- ```ruby
365
- # config/initializers/e11y.rb (<300 lines!)
366
- E11y.configure do |config|
367
- # ONLY infrastructure
368
- config.register_adapter :loki, Loki.new(...)
369
- config.register_adapter :sentry, Sentry.new(...)
370
-
371
- # Defaults (conventions)
372
- config.default_adapters = [:loki]
373
-
374
- # ✅ No per-event config here!
375
- end
368
+ **Auto (:rails_filters):** E11y automatically applies `Rails.application.config.filter_parameters`.
376
369
 
377
- # app/events/order_created.rb
378
- class Events::OrderCreated < E11y::Event::Base
379
- schema do; required(:order_id).filled(:string); end
380
- # ← Uses conventions (zero config!)
381
- end
370
+ ```ruby
371
+ # config/application.rb
372
+ config.filter_parameters += [:password, :email, :ssn]
382
373
 
383
- # app/events/payment_succeeded.rb
384
- class Events::PaymentSucceeded < Events::BasePaymentEvent
385
- schema do; required(:transaction_id).filled(:string); end
386
- # ← Inherits config from BasePaymentEvent (DRY!)
387
- end
374
+ # These fields become '[FILTERED]' automatically on track
375
+ Events::UserRegistered.track(email: 'user@example.com', password: 'secret')
388
376
  ```
389
377
 
390
- ### Migration Path (Backward Compatible)
378
+ **Event-level DSL (:explicit_pii):**
391
379
 
392
380
  ```ruby
393
- # v1.1 supports BOTH styles (backward compatible!)
381
+ class Events::PaymentCreated < E11y::Event::Base
382
+ contains_pii true
394
383
 
395
- # Old style (still works):
396
- E11y.configure do |config|
397
- config.events do
398
- event 'Events::OrderCreated' do
399
- adapters [:loki]
400
- end
384
+ pii_filtering do
385
+ masks :card_number # '[FILTERED]'
386
+ hashes :user_email # → SHA256 (preserves searchability)
387
+ partials :phone # → first/last characters visible
388
+ redacts :ssn # → removed completely
389
+ allows :amount # → no filtering
401
390
  end
402
391
  end
392
+ ```
403
393
 
404
- # New style (preferred):
405
- class Events::OrderCreated < E11y::Event::Base
406
- adapters [:loki] # ← Event-level (overrides global)
394
+ **Inheritance:** Use a base class for common rules, child events add or override:
395
+
396
+ ```ruby
397
+ class BaseUserEvent < E11y::Event::Base
398
+ contains_pii true
399
+ pii_filtering do
400
+ masks :password
401
+ hashes :email
402
+ partials :phone
403
+ end
407
404
  end
408
405
 
409
- # Migrate incrementally (both work together!)
406
+ class Events::PaymentCreated < BaseUserEvent
407
+ pii_filtering do
408
+ masks :card_number, :cvv
409
+ end
410
+ end
410
411
  ```
411
412
 
412
- ---
413
-
414
- ## 🔧 Old Configuration (v1.0 - Deprecated)
415
-
416
- > **⚠️ Deprecated:** This section shows v1.0 configuration style for reference.
417
- > Use event-level configuration (above) for new projects.
413
+ ### Adaptive Sampling
418
414
 
419
415
  ```ruby
420
- # config/initializers/e11y.rb (OLD STYLE)
421
416
  E11y.configure do |config|
422
- # === SEVERITY ===
423
- config.severity = Rails.env.production? ? :info : :debug
424
-
425
- # === ADAPTERS (old style) ===
426
- config.adapters = [
427
- # Loki for logs
428
- E11y::Adapters::LokiAdapter.new(
429
- url: ENV['LOKI_URL'],
430
- labels: { env: Rails.env, service: 'api' }
431
- ),
432
-
433
- # Sentry for errors
434
- E11y::Adapters::SentryAdapter.new(
435
- severity_filter: [:error, :fatal]
436
- ),
437
-
438
- # Stdout for development
439
- (E11y::Adapters::StdoutAdapter.new if Rails.env.development?)
440
- ].compact
441
-
442
- # === PATTERN-BASED METRICS ===
443
- config.metrics do
444
- # Counter for all events
445
- counter_for pattern: '*',
446
- name: 'business_events_total',
447
- tags: [:event_name, :severity]
448
-
449
- # Histogram for payments
450
- histogram_for pattern: '*.paid',
451
- name: 'payment_amount',
452
- value: ->(e) { e.payload[:amount] },
453
- buckets: [10, 50, 100, 500, 1000]
454
-
455
- # Success rate auto-metric
456
- success_rate_for pattern: 'order.*',
457
- name: 'order_operations_success_rate'
458
- end
459
-
460
- # === PII FILTERING (Rails-compatible!) ===
461
- config.pii_filter do
462
- # AUTO: Uses Rails.application.config.filter_parameters (default: true)
463
- use_rails_filter_parameters true
464
-
465
- # SIMPLE: Add more filters (Rails-style)
466
- filter_parameters :api_key, :auth_token, /secret/i
467
-
468
- # WHITELIST: Allow specific IDs even if filtered by Rails
469
- allow_parameters :user_id, :order_id, :transaction_id
470
-
471
- # ADVANCED: Pattern-based (beyond Rails)
472
- filter_pattern /\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/i,
473
- replacement: '[EMAIL]'
474
- filter_pattern /\b\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{4}\b/,
475
- replacement: '[CARD]'
476
- end
477
-
478
- # === RATE LIMITING ===
479
- config.rate_limiting do
480
- global limit: 10_000, window: 1.minute
481
- per_event 'user.login.failed', limit: 100, window: 1.minute
482
- end
483
-
484
- # === CONTEXT ENRICHMENT ===
485
- config.context_enricher do |event|
486
- {
487
- trace_id: E11y::TraceId.extract,
488
- user_id: Current.user&.id,
489
- tenant_id: Current.tenant&.id
417
+ config.pipeline.use E11y::Middleware::Sampling,
418
+ default_sample_rate: 0.1,
419
+
420
+ # Error-spike: on error burst → 100% sampling
421
+ error_based_adaptive: true,
422
+ error_spike_config: {
423
+ window: 60,
424
+ absolute_threshold: 100,
425
+ relative_threshold: 3.0,
426
+ spike_duration: 300
427
+ },
428
+
429
+ # Load-based: under high load → reduced sampling
430
+ load_based_adaptive: true,
431
+ load_monitor_config: {
432
+ window: 60,
433
+ thresholds: {
434
+ normal: 1_000,
435
+ high: 10_000,
436
+ very_high: 50_000,
437
+ overload: 100_000
438
+ }
490
439
  }
491
- end
492
440
  end
441
+ ```
493
442
 
494
- # Start async workers
495
- E11y.start!
443
+ **Value-based sampling at the event level:**
496
444
 
497
- # Graceful shutdown
498
- at_exit { E11y.stop!(timeout: 5) }
445
+ ```ruby
446
+ class Events::Payment < E11y::Event::Base
447
+ sample_by_value :amount, greater_than: 1000 # large payments — always tracked
448
+ sample_by_value :total, in_range: 100..500
449
+ end
499
450
  ```
500
451
 
501
452
  ---
@@ -503,64 +454,78 @@ at_exit { E11y.stop!(timeout: 5) }
503
454
  ## 📊 Severity Levels
504
455
 
505
456
  ```ruby
506
- E11y::SEVERITIES = {
507
- debug: 0, # Detailed diagnostic (buffered in request scope)
508
- info: 1, # Informational
509
- success: 2, # NEW! Successful operations
510
- warn: 3, # Warnings
511
- error: 4, # Errors
512
- fatal: 5 # Critical failures
513
- }
457
+ # lib/e11y/event/base.rb
458
+ SEVERITIES = %i[debug info success warn error fatal].freeze
459
+
460
+ # Default sample rates by severity:
461
+ SEVERITY_SAMPLE_RATES = {
462
+ error: 1.0, # always
463
+ fatal: 1.0, # always
464
+ debug: 0.01, # 1%
465
+ info: 0.1, # 10%
466
+ success: 0.1, # 10%
467
+ warn: 0.1 # 10%
468
+ }.freeze
514
469
  ```
515
470
 
516
- **When to use :success?**
471
+ **When to use `:success`:**
517
472
 
518
473
  ```ruby
519
- # Use :success for completed operations
520
- Events::OrderPaid.track(order_id: '123', severity: :success)
521
- Events::JobCompleted.track(job_id: '456', severity: :success)
522
- Events::EmailSent.track(user_id: '789', severity: :success)
474
+ Events::OrderPaid.track(order_id: '123') # :success (explicit)
475
+ Events::JobCompleted.track(job_id: '456') # :success (from name "Completed")
476
+ Events::UserLoggedIn.track(user_id: '123') # :info (default)
477
+ Events::PaymentFailed.track(reason: 'timeout') # :error (from name "Failed")
478
+ ```
523
479
 
524
- # Use :info for informational events
525
- Events::UserLoggedIn.track(user_id: '123', severity: :info)
526
- Events::SessionStarted.track(session_id: '456', severity: :info)
480
+ **Auto-resolved severity (convention over configuration):**
527
481
 
528
- # Why separate :success from :info?
529
- # → Easy filtering: severity:success = only successful ops
530
- # Easy metrics: success_rate = count(:success) / count(:success OR :error)
531
- ```
482
+ | Name contains | Severity |
483
+ |---|---|
484
+ | `Failed`, `Error` | `:error` |
485
+ | `Paid`, `Success`, `Completed` | `:success` |
486
+ | `Warn`, `Warning` | `:warn` |
487
+ | anything else | `:info` |
532
488
 
533
489
  ---
534
490
 
535
- ## 🎭 Middleware (Auto-configured)
491
+ ## 🎭 Middleware (configuration)
536
492
 
537
493
  ### Rails / Rack
538
494
 
539
- ```ruby
540
- # config/application.rb (auto-added by generator)
541
- config.middleware.use E11y::Middleware::Rack,
542
- buffer_limit: 100,
543
- flush_on: :error
544
- ```
495
+ `E11y::Middleware::Request` is automatically inserted by the Railtie when `ephemeral_buffer_enabled` is true.
545
496
 
546
497
  ### Sidekiq
547
498
 
548
499
  ```ruby
549
- # config/initializers/sidekiq.rb (auto-added by generator)
500
+ # Automatically inserted by Railtie when Sidekiq is present.
501
+ # For manual setup:
550
502
  Sidekiq.configure_server do |config|
551
503
  config.server_middleware do |chain|
552
- chain.add E11y::Middleware::Sidekiq
504
+ chain.add E11y::Instruments::Sidekiq::ServerMiddleware
553
505
  end
554
506
  end
555
507
  ```
556
508
 
557
- ### ActiveJob
509
+ ### Default pipeline order
510
+
511
+ ```
512
+ TraceContext → Validation → PIIFilter → AuditSigning → Sampling → RateLimiting → Routing
513
+ ```
514
+
515
+ As of v0.2.0, `RateLimiting` is wired into the default pipeline. To activate it, set
516
+ `config.rate_limiting_enabled = true` (no manual `.use` call needed).
517
+
518
+ **Versioning (opt-in):**
519
+
520
+ `Middleware::Versioning` normalizes event names from CamelCase class names to dot-notation
521
+ (e.g., `OrderPaidEvent` → `order.paid`). It is not in the default pipeline; add it explicitly:
558
522
 
559
523
  ```ruby
560
- # Auto-included by E11y (no config needed)
561
- # trace_id propagated automatically to background jobs
524
+ config.pipeline.use E11y::Middleware::Versioning
562
525
  ```
563
526
 
527
+ Without this middleware, event names in adapters are the raw class name (e.g., `"OrderPaidEvent"`).
528
+
564
529
  ---
565
530
 
566
531
  ## 🔍 Trace Context Flow
@@ -568,40 +533,52 @@ end
568
533
  ```ruby
569
534
  # Service A (API)
570
535
  POST /orders
571
- trace_id: abc-123 (from X-Trace-ID header)
572
- ├─ Events::OrderValidation.track (trace_id: abc-123)
573
- ├─ Events::OrderCreated.track (trace_id: abc-123)
574
- └─ ProcessOrderJob.perform_later(order_id, trace_id: abc-123)
536
+ trace_id: abc-123 # from X-Trace-ID header
537
+ ├─ Events::OrderValidation.track # trace_id: abc-123
538
+ ├─ Events::OrderCreated.track # trace_id: abc-123
539
+ └─ ProcessOrderJob.perform_later(order_id: id, trace_id: 'abc-123')
575
540
 
576
- # Service B (Background Job)
541
+ # Background Job (Sidekiq)
577
542
  ProcessOrderJob
578
- trace_id: abc-123 (from job args)
579
- ├─ Events::OrderProcessing.track (trace_id: abc-123)
580
- └─ HTTP → Service C (headers: X-Trace-ID: abc-123)
543
+ trace_id: abc-123 # propagated through middleware
544
+ └─ Events::OrderProcessed.track # trace_id: abc-123
581
545
 
582
- # Service C (Payment API)
583
- POST /payments
584
- trace_id: abc-123 (from X-Trace-ID header)
585
- └─ Events::PaymentProcessed.track (trace_id: abc-123)
586
-
587
- # Result: All events linked by trace_id = full visibility
546
+ # Result: all events are correlated by trace_id = full visibility
588
547
  ```
589
548
 
590
549
  ---
591
550
 
592
- ## 📈 Yabeda Integration
551
+ ## 📈 Yabeda / Prometheus Integration
593
552
 
594
- Events автоматически становятся метриками через pattern-based rules:
553
+ > **Note:** The `metrics do ... end` DSL requires the Yabeda adapter to be registered.
554
+ > Without `E11y::Adapters::Yabeda.new` in `config.adapters`, metric definitions are stored
555
+ > but never updated. Add the adapter as shown below before defining event metrics.
595
556
 
596
557
  ```ruby
597
- # After tracking event:
598
- Events::OrderPaid.track(amount: 99, currency: 'USD')
558
+ # Gemfile
559
+ gem 'yabeda'
560
+ gem 'yabeda-prometheus'
561
+
562
+ # config/initializers/e11y.rb
563
+ E11y.configure do |config|
564
+ config.adapters[:metrics] = E11y::Adapters::Yabeda.new(
565
+ cardinality_limit: 1000,
566
+ forbidden_labels: [:user_id, :order_id], # additional denylist
567
+ overflow_strategy: :relabel # :drop, :alert, or :relabel
568
+ )
569
+ end
570
+
571
+ # Event with metrics:
572
+ class Events::OrderPaid < E11y::Event::Base
573
+ metrics do
574
+ counter :orders_paid_total, tags: [:currency]
575
+ histogram :order_amount, value: :amount, tags: [:currency]
576
+ end
577
+ end
599
578
 
600
- # Auto-generated metrics:
601
- yabeda.business_events.events_total{event_name="order.paid",severity="success"} 1
602
- yabeda.business_events.payment_amount_bucket{currency="USD",le="100"} 1
603
- yabeda.business_events.order_operations_success 1
604
- yabeda.business_events.order_operations_total 1
579
+ Events::OrderPaid.track(order_id: '123', amount: 99, currency: 'USD')
580
+ # → Yabeda.e11y.orders_paid_total.increment({currency: 'USD'})
581
+ # → Yabeda.e11y.order_amount.measure({currency: 'USD'}, 99)
605
582
  ```
606
583
 
607
584
  **Prometheus endpoint:**
@@ -611,241 +588,164 @@ yabeda.business_events.order_operations_total 1
611
588
  mount Yabeda::Prometheus::Exporter => '/metrics'
612
589
  ```
613
590
 
591
+
614
592
  ---
615
593
 
616
594
  ## 🧪 Testing
617
595
 
618
596
  ```ruby
619
- # spec/spec_helper.rb
597
+ # spec/support/e11y_helper.rb
620
598
  RSpec.configure do |config|
599
+ let(:test_adapter) { E11y::Adapters::InMemory.new }
600
+
621
601
  config.before(:each) do
602
+ # adapters is a Hash — use [] assignment, not Array assignment
622
603
  E11y.configure do |c|
623
- c.adapters = [E11y::Adapters::NullAdapter.new]
604
+ c.adapters[:test] = test_adapter
605
+ # For a no-op adapter that discards all events (no recording overhead):
606
+ # c.adapters[:null] = E11y::Adapters::NullAdapter.new
624
607
  end
625
608
  end
609
+
610
+ config.after(:each) do
611
+ test_adapter.clear!
612
+ end
626
613
  end
627
614
 
628
615
  # spec/controllers/orders_controller_spec.rb
629
616
  RSpec.describe OrdersController do
630
617
  it 'tracks order creation' do
631
- expect(Events::OrderCreated).to receive(:track).with(
632
- hash_including(order_id: anything)
618
+ post :create, params: { order_id: '123', amount: 99.99, currency: 'USD' }
619
+
620
+ events = test_adapter.events
621
+ expect(events).to include(
622
+ a_hash_including(
623
+ event_name: 'OrderCreated',
624
+ payload: hash_including(order_id: '123')
625
+ )
633
626
  )
634
-
635
- post :create, params: { ... }
627
+ end
628
+
629
+ it 'raises on invalid data' do
630
+ expect {
631
+ Events::OrderPaid.track(order_id: nil, amount: -1)
632
+ }.to raise_error(E11y::ValidationError)
636
633
  end
637
634
  end
638
635
  ```
639
636
 
640
- ---
637
+ **InMemory Adapter API:**
641
638
 
642
- ## 🚀 Performance
643
-
644
- | Metric | Target | Actual |
645
- |--------|--------|--------|
646
- | **Track latency (p99)** | <1ms | ✅ 0.8ms |
647
- | **Throughput** | 10k events/sec | ✅ 15k/sec |
648
- | **Memory** | <100MB @ 100k buffer | ✅ 80MB |
649
- | **CPU overhead** | <5% @ 1k events/sec | ✅ 3% |
639
+ ```ruby
640
+ adapter = E11y::Adapters::InMemory.new(max_events: 1000)
650
641
 
651
- **Optimizations:**
652
- - Early severity filtering (<1μs для filtered events)
653
- - Lock-free ring buffer (SPSC)
654
- - Async workers (no blocking)
655
- - Lazy serialization (только перед отправкой)
642
+ adapter.events # => Array<Hash> — all events
643
+ adapter.event_count # => Integer
644
+ adapter.last_event # => Hash last event
645
+ adapter.clear! # reset
646
+ ```
656
647
 
657
648
  ---
658
649
 
659
650
  ## 🔐 Security
660
651
 
661
- ### PII Filtering (Rails-Compatible! 🎉)
652
+ ### PII Filtering what works today
662
653
 
663
- **ZERO CONFIG**: E11y автоматически использует `Rails.application.config.filter_parameters`!
654
+ **Auto (:rails_filters) Rails filter_parameters:**
664
655
 
665
656
  ```ruby
666
657
  # config/application.rb
667
- config.filter_parameters += [:password, :email, :ssn]
668
-
669
- # E11y автоматически фильтрует эти поля - NO ADDITIONAL CONFIG!
670
- Events::UserRegistered.track(
671
- email: 'user@example.com', # → '[FILTERED]'
672
- password: 'secret123' # → '[FILTERED]'
673
- )
658
+ config.filter_parameters += [:password, :email, :ssn, :credit_card]
659
+ # E11y applies this list automatically — no extra config needed
674
660
  ```
675
661
 
676
- **EXTENDED**: Добавьте больше фильтров поверх Rails:
662
+ **Event-level (:explicit_pii):**
677
663
 
678
664
  ```ruby
679
- # config/initializers/e11y.rb
680
- E11y.configure do |config|
681
- config.pii_filter do
682
- # Inherit Rails filters + add more
683
- filter_parameters :api_key, :auth_token, /secret/i
684
-
685
- # Whitelist known-safe fields
686
- allow_parameters :user_id, :order_id
687
-
688
- # Pattern-based (content filtering)
689
- filter_pattern /\b\d{16}\b/, replacement: '[CARD]'
665
+ class Events::UserRegistered < E11y::Event::Base
666
+ contains_pii true
667
+
668
+ pii_filtering do
669
+ masks :password
670
+ hashes :email # SHA256, preserves searchability
671
+ redacts :ssn
672
+ allows :user_id
690
673
  end
691
674
  end
692
675
  ```
693
676
 
694
- **See full guide**: `e11y-rails-compatible-pii-filtering.md`
677
+ ### Rate Limiting — now in default pipeline
695
678
 
696
- ### Rate Limiting (защита от DoS)
679
+ As of v0.2.0, `Middleware::RateLimiting` is included in the default pipeline. Activate it by
680
+ enabling the config (no extra `.use` call required):
697
681
 
698
682
  ```ruby
699
- # Config
700
- config.rate_limiting do
701
- global limit: 10_000, window: 1.minute
702
- per_event 'user.login.failed', limit: 100, window: 1.minute
683
+ E11y.configure do |config|
684
+ config.rate_limiting_enabled = true
685
+ config.rate_limiting_global_limit = 10_000 # events/sec
686
+ config.rate_limiting_per_event_limit = 1_000 # events/sec per type
687
+ config.rate_limiting_global_window = 1.0 # seconds
703
688
  end
704
-
705
- # При превышении - события дропаются
706
689
  ```
707
690
 
708
- ---
691
+ > **Note:** When `config.rate_limiting_enabled = false` (default), the middleware is present in
692
+ > the pipeline but passes all events through without limiting. Set `enabled = true` to activate.
709
693
 
710
- ## 🎯 Built-in SLO Tracking (Zero Config!)
694
+ > 🚧 **Roadmap:** Per-event and per-pattern rate limiting (e.g. `'user.login.failed'` → 100/min)
695
+ > — planned for future releases.
711
696
 
712
- **Enable one flag → get SLO out of the box!**
697
+ ---
713
698
 
714
- ```ruby
715
- # config/initializers/e11y.rb
716
- E11y.configure do |config|
717
- config.slo_tracking = true # ← ВСЁ! Этого достаточно!
718
-
719
- # Опционально: кастомизация
720
- config.slo do
721
- # Global defaults
722
- http_ignore_statuses [404, 401] # 404/401 не ошибки
723
- latency_target_p95 200 # ms
724
-
725
- # 🎯 NEW: Per-controller overrides (РЕКОМЕНДУЕТСЯ для Rails!)
726
- controller 'Api::Admin::BaseController' do
727
- ignore true # Весь admin не входит в SLO
728
- end
729
-
730
- controller 'Api::OrdersController', action: 'show' do
731
- latency_target_p95 50 # Show должен быть быстрым
732
- end
733
-
734
- controller 'Api::OrdersController', action: 'create' do
735
- latency_target_p95 200 # Create может быть медленнее
736
- ignore_statuses [422] # 422 Validation = not SLO breach
737
- end
738
-
739
- # 🔧 LEGACY: Path-based (для non-Rails apps)
740
- endpoint '/api/webhooks/*' do
741
- ignore true
742
- end
743
-
744
- # 🎯 Per-job overrides
745
- job 'ReportGenerationJob' do
746
- ignore true # Долгие джобы не входят в SLO
747
- end
748
-
749
- job 'ProcessPaymentJob' do
750
- latency_target_p95 1000 # Критичные джобы = строгий SLO
751
- end
752
- end
753
- end
754
- ```
699
+ ## 🎯 Built-in SLO Tracking
755
700
 
756
- **Автоматически получаем:**
701
+ **What is tracked automatically** when `config.rails_instrumentation_enabled = true`:
757
702
 
758
703
  ```ruby
759
- # 🎯 HTTP Metrics (controller#action - автогруппировка!)
760
- yabeda_slo_http_requests_total{
761
- status="200",
762
- method="GET",
763
- controller="Api::OrdersController",
764
- action="show"
765
- } 1234
766
-
767
- yabeda_slo_http_request_duration_seconds{
768
- method="GET",
769
- controller="Api::OrdersController",
770
- action="show"
771
- } histogram
704
+ # HTTP Metrics (via Rack middleware)
705
+ e11y_http_requests_total{
706
+ status="200", method="GET",
707
+ controller="Api::OrdersController", action="show"
708
+ }
709
+ e11y_http_request_duration_seconds{ ... } # histogram
772
710
 
773
711
  # Sidekiq Metrics
774
- yabeda_slo_sidekiq_jobs_total{queue="default", class="ProcessOrderJob", status="success"} 456
775
- yabeda_slo_sidekiq_job_duration_seconds{queue="default", class="ProcessOrderJob"} histogram
712
+ e11y_sidekiq_jobs_total{queue="default", class="ProcessOrderJob", status="success"}
713
+ e11y_sidekiq_job_duration_seconds{ ... }
776
714
 
777
715
  # ActiveJob Metrics
778
- yabeda_slo_active_jobs_total{queue="mailers", class="EmailJob", status="success"} 789
779
- yabeda_slo_active_job_duration_seconds{queue="mailers", class="EmailJob"} histogram
716
+ e11y_active_jobs_total{queue="mailers", class="EmailJob", status="success"}
717
+ e11y_active_job_duration_seconds{ ... }
780
718
  ```
781
719
 
782
- **Преимущество:** `/orders/123`, `/orders/456` → один `OrdersController#show` (не нужна нормализация path!)
783
-
784
720
  **SLO Calculations (PromQL):**
785
721
 
786
722
  ```promql
787
723
  # HTTP Availability (30d rolling)
788
- 100 * (sum(rate(yabeda_slo_http_successes_total[30d])) /
789
- (sum(rate(yabeda_slo_http_successes_total[30d])) + sum(rate(yabeda_slo_http_errors_total[30d]))))
790
- # Expected: >= 99.9%
724
+ 100 * (
725
+ sum(rate(e11y_http_requests_total{status=~"2.."}[30d])) /
726
+ sum(rate(e11y_http_requests_total[30d]))
727
+ )
791
728
 
792
729
  # p95 Latency
793
- histogram_quantile(0.95, rate(yabeda_slo_http_request_duration_seconds_bucket[5m]))
794
- # Expected: < 200ms
795
-
796
- # Error Budget Remaining
797
- 100 * (1 - (sum(rate(yabeda_slo_http_errors_total[30d])) /
798
- (sum(rate(yabeda_slo_http_successes_total[30d])) + sum(rate(yabeda_slo_http_errors_total[30d])))) / 0.001)
799
- # 100% = весь бюджет остался, 0% = исчерпан
800
- ```
801
-
802
- **Auto-Generated:**
803
-
804
- ```bash
805
- # Grafana dashboard
806
- rails g e11y:grafana_dashboard
807
- # → config/grafana/e11y_slo_dashboard.json
808
-
809
- # Prometheus alerts
810
- rails g e11y:prometheus_alerts
811
- # → config/prometheus/e11y_slo_alerts.yml
730
+ histogram_quantile(0.95, rate(e11y_http_request_duration_seconds_bucket[5m]))
812
731
  ```
813
732
 
814
- **Что трекается автоматически:**
815
- - ✅ **Rack middleware** - все HTTP requests (availability, latency)
816
- - ✅ **Sidekiq middleware** - все jobs (success rate, duration)
817
- - ✅ **ActiveJob instrumentation** - все jobs (success rate, duration)
818
- - ✅ **Path normalization** - `/orders/123` → `/orders/:id`
819
- - ✅ **Error categorization** - configurable (5xx = error, 404 = ignore)
820
- - ✅ **Heartbeat** - auto-enabled (pod liveness detection)
733
+ **⚠️ In-process SLO limitations:**
821
734
 
822
- **⚠️ ВАЖНО: Ограничения in-process SLO**
735
+ E11y SLO runs inside the Ruby process and **does not see**:
736
+ - Network issues (requests that never reach the app)
737
+ - Load balancer failures
738
+ - All pods down
823
739
 
824
- E11y SLO работает ВНУТРИ Ruby процесса и НЕ видит:
825
- - ❌ Network issues (requests не доходят до app)
826
- - ❌ Load balancer down
827
- - ❌ All pods crashed (метрик просто нет)
828
- - ❌ DNS issues
740
+ Recommended: multi-layer monitoring — E11y SLO + K8s health probes + external synthetic checks.
829
741
 
830
- **Решение:** Multi-layer monitoring (см. полную документацию):
831
- 1. **Layer 1**: E11y SLO (in-process)
832
- 2. **Layer 2**: E11y Heartbeat (pod liveness) - **auto-enabled!**
833
- 3. **Layer 3**: K8s health probes (`/health/live`, `/health/ready`) - **auto-created!**
834
- 4. **Layer 4**: External synthetic monitoring (Prometheus Blackbox Exporter)
835
-
836
- ```ruby
837
- # Heartbeat автоматически включен с slo_tracking = true
838
- # Метрики:
839
- yabeda_e11y_heartbeat_timestamp{pod="pod-1"} 1703500000 # Последний heartbeat
840
- yabeda_e11y_service_healthy{pod="pod-1"} 1 # 1 = healthy
841
-
842
- # Alert если pod мертв:
843
- # (time() - yabeda_e11y_heartbeat_timestamp) > 30s → Pod down!
844
- ```
742
+ > 🚧 **Roadmap:** Per-controller/per-job SLO configuration, auto-generated Grafana dashboards
743
+ > (`rails g e11y:grafana_dashboard`) and Prometheus alerts (`rails g e11y:prometheus_alerts`)
744
+ > planned for future releases.
845
745
 
846
746
  ---
847
747
 
848
- ## 🎯 Migration from Rails.logger
748
+ ## 🔄 Migration from Rails.logger
849
749
 
850
750
  ```ruby
851
751
  # ❌ Before
@@ -853,76 +753,135 @@ Rails.logger.info "Order #{order.id} paid #{order.amount} #{order.currency}"
853
753
  OrderMetrics.increment('orders.paid.total')
854
754
  OrderMetrics.observe('orders.paid.amount', order.amount)
855
755
 
856
- # ✅ After (1 строка вместо 3!)
756
+ # ✅ After (1 line instead of 3, + validation + trace context)
857
757
  Events::OrderPaid.track(
858
758
  order_id: order.id,
859
- amount: order.amount,
759
+ amount: order.amount,
860
760
  currency: order.currency
861
761
  )
862
- # → Structured log + auto-metrics + trace context
863
762
  ```
864
763
 
865
764
  ---
866
765
 
867
766
  ## 🐛 Troubleshooting
868
767
 
869
- ### Events not appearing?
768
+ ### Events not appearing in the adapter?
870
769
 
871
770
  ```ruby
872
- # Check 1: Severity
873
- E11y.enabled_for?(:debug) # Should be true
771
+ # 1. Is E11y enabled?
772
+ E11y.config.enabled # => true
773
+
774
+ # 2. Is the adapter registered?
775
+ E11y.config.adapters # => {:logs=>#<Loki...>, ...} — must not be empty
776
+
777
+ # 3. Is severity routing configured?
778
+ E11y.config.adapter_mapping
779
+ # => {:error=>[:logs, :errors_tracker], :fatal=>[:logs, :errors_tracker], :default=>[:logs]}
874
780
 
875
- # Check 2: Adapters
876
- E11y.config.adapters # Should not be empty
781
+ # 4. Is the adapter healthy?
782
+ E11y.config.adapters[:logs].healthy? # => true
877
783
 
878
- # Check 3: Buffer
879
- E11y.buffer_size # Should be < capacity
784
+ # 5. Metrics not updating? Yabeda adapter must be explicitly configured:
785
+ E11y.config.adapters[:metrics] # should be E11y::Adapters::Yabeda
880
786
 
881
- # Check 4: Circuit breaker
882
- E11y.circuit_breaker_state # Should be :closed
787
+ # 6. Diagnostic helpers (v0.2.0):
788
+ E11y.enabled_for?(:debug) # => true/false — is debug severity active?
789
+ E11y.buffer_size # => Integer — current debug buffer size
790
+ E11y.circuit_breaker_state # => :closed/:open/:half_open
883
791
  ```
884
792
 
885
- ### High latency?
793
+ ### Debug events not flushing on errors?
886
794
 
887
795
  ```ruby
888
- # Check metrics
889
- Yabeda.e11y.track_duration_seconds # p99 should be <1ms
796
+ # Is the buffer enabled?
797
+ E11y.config.ephemeral_buffer_enabled # => true
798
+
799
+ # Is Rails instrumentation enabled?
800
+ E11y.config.rails_instrumentation_enabled # => true
801
+
802
+ # Is 5xx auto-flush enabled?
803
+ E11y.config.ephemeral_buffer_flush_on_error # => true (default)
804
+
805
+ # Any extra statuses configured?
806
+ E11y.config.ephemeral_buffer_flush_on_statuses # => [] (default) or [403] etc.
807
+
808
+ # Note: flush_on_error and flush_on_statuses are independent.
809
+ # flush_on_error=true → flush on any 5xx.
810
+ # flush_on_statuses=[403] → also flush on 403, regardless of flush_on_error.
811
+ ```
890
812
 
813
+ ### High latency?
814
+
815
+ ```ruby
891
816
  # Possible causes:
892
- # - PII filtering regex too complex
893
- # - Rate limiter Redis slow
894
- # - Adapter network slow
817
+ # - Loki/Sentry unreachable circuit breaker will open after 5 errors
818
+ # - PII filtering: complex regexes → simplify
819
+ # - batch_size too large → reduce or lower batch_timeout
820
+
821
+ # Check adapter health:
822
+ E11y.config.adapters[:logs].healthy?
895
823
  ```
896
824
 
897
825
  ---
898
826
 
899
827
  ## 📚 Full Documentation
900
828
 
901
- - **Complete spec**: `severity/e11y-final-spec.md` (2000+ lines)
902
- - **Old spec**: `severity/tz-improved.md` (reference)
903
- - **GitHub**: https://github.com/yourorg/e11y
829
+ - **ADRs**: `docs/architecture/ADR-*.md` architecture decision records
830
+ - **GitHub**: https://github.com/arturseletskiy/e11y
904
831
 
905
832
  ---
906
833
 
907
- ## ✅ Checklist
908
-
909
- - [ ] Install gem
910
- - [ ] Run generator: `rails g e11y:install`
911
- - [ ] Configure adapters (Loki/Sentry/ELK)
912
- - [ ] **Enable SLO tracking**: `config.slo_tracking = true`
913
- - [ ] Define first event class
914
- - [ ] Track first event
915
- - [ ] Check `/metrics` endpoint (Prometheus)
916
- - [ ] Verify events in Grafana/Kibana
917
- - [ ] **Verify SLO metrics**: `yabeda_slo_*` in Prometheus
918
- - [ ] Test request-scoped buffering (raise exception see debug logs)
919
- - [ ] Configure PII filtering
920
- - [ ] Setup rate limiting
921
- - [ ] Configure pattern-based metrics
922
- - [ ] **Generate Grafana dashboard**: `rails g e11y:grafana_dashboard`
923
- - [ ] **Generate Prometheus alerts**: `rails g e11y:prometheus_alerts`
924
- - [ ] Deploy to staging
925
- - [ ] Monitor performance
926
- - [ ] Rollout to production (canary 1% → 10% → 100%)
834
+ ## ✅ Getting Started Checklist
835
+
836
+ - [ ] Add `gem 'e11y', '~> 0.2'` to Gemfile, run `bundle install`
837
+ - [ ] Run `rails g e11y:install` (or create `config/initializers/e11y.rb` manually)
838
+ - [ ] Configure adapters: `config.adapters[:logs] = E11y::Adapters::Loki.new(...)`
839
+ - [ ] For metrics: add `gem 'yabeda'`, set `config.adapters[:metrics] = E11y::Adapters::Yabeda.new`
840
+ - [ ] Enable request buffering: `config.ephemeral_buffer_enabled = true`
841
+ - [ ] Enable Rails instrumentation: `config.rails_instrumentation_enabled = true`
842
+ - [ ] Define first event class in `app/events/`
843
+ - [ ] Use `EventName.track(...)` in a controller or service
844
+ - [ ] Write a test using `E11y::Adapters::InMemory`
845
+ - [ ] Check `/metrics` endpoint (if Yabeda is configured)
846
+ - [ ] Test request buffering: raise an exception → confirm debug events appeared
847
+ - [ ] Configure PII filtering for events that handle personal data
848
+ - [ ] Deploy to staging, monitor performance
927
849
 
850
+ ---
928
851
 
852
+ ## ✅ What's New in v0.2.0
853
+
854
+ | Feature | Notes |
855
+ |---------|-------|
856
+ | `rails g e11y:install` | Generator available: creates initializer + `app/events/` |
857
+ | `E11y.start!` / `E11y.stop!` | Lifecycle methods for graceful startup/shutdown |
858
+ | Rate Limiting in default pipeline | `config.rate_limiting_enabled = true` now works |
859
+ | Event name normalization (`Middleware::Versioning`) | Now in default pipeline |
860
+ | OTelLogs payload attributes | All payload attributes now included in OTel log records |
861
+ | `config.slo_tracking = true` | Boolean coercion now accepted |
862
+ | `retention` / `retention_period` | Both work as aliases on event class |
863
+ | `add_slo_controller` / `add_slo_job` | Helpers on `E11y::Configuration` (stored config; see UC-004) |
864
+ | `config.rate_limiting do` | Rate limiting block DSL |
865
+ | `config.cardinality_protection do` | Cardinality DSL block |
866
+ | `config.register_adapter` | Alias for `config.adapters[name] =` |
867
+ | `NullAdapter` | `E11y::Adapters::NullAdapter.new` for no-op testing |
868
+ | `track() { }` block form | Block form measures duration automatically |
869
+ | `E11y.enabled_for?` / `E11y.buffer_size` | Diagnostic helpers |
870
+ | `metric :counter` single-call DSL | Single metric definition without `metrics do` block |
871
+ | Full block DSLs | All config sections support `do...end` block form |
872
+
873
+ ## 🚧 Roadmap (still not implemented)
874
+
875
+ The following features are **documented in ADRs** but not yet implemented:
876
+
877
+ | Feature | ADR/UC |
878
+ |---------|--------|
879
+ | `rails g e11y:grafana_dashboard` | ADR-003 |
880
+ | `rails g e11y:prometheus_alerts` | ADR-003 |
881
+ | Wire `add_slo_controller` / `add_slo_job` into HTTP/job `Tracker` dimensions | UC-004, ADR-003 |
882
+ | Per-event rate limiting (`rate_limit` DSL on event class) | UC-011 |
883
+ | Tiered storage (archival) | UC-019 — filter by `retention_until` |
884
+ | Cost Tracking / Budget Enforcement | ADR-009, UC-015 |
885
+ | Outgoing HTTP trace propagation (Faraday/Net::HTTP) | UC-009 |
886
+ | Event Registry (`E11y::Registry`) | UC-022 |
887
+ | Key Rotation for AuditEncrypted | ADR-006 |