e11y 0.2.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.rubocop.yml +130 -10
- data/CHANGELOG.md +80 -1
- data/CLAUDE.md +168 -0
- data/CONTRIBUTING.md +640 -0
- data/README.md +165 -701
- data/RELEASE.md +41 -12
- data/Rakefile +249 -57
- data/config/README.md +1 -1
- data/config/loki-local-config.yaml +12 -0
- data/config/otel-collector-config.yaml +44 -0
- data/cucumber.yml +1 -0
- data/docker-compose.yml +18 -2
- data/docs/ADAPTERS.md +76 -0
- data/docs/ADAPTIVE_SAMPLING.md +59 -0
- data/docs/COMPARISON.md +104 -0
- data/docs/CONFIGURATION.md +52 -0
- data/docs/DISTRIBUTED_TRACING.md +44 -0
- data/docs/LIMITATIONS.md +13 -0
- data/docs/METRICS_DSL.md +84 -0
- data/docs/PERFORMANCE.md +60 -0
- data/docs/PII_FILTERING.md +40 -0
- data/docs/PRESETS.md +65 -0
- data/docs/QUICK-START.md +546 -587
- data/docs/RAILS_INTEGRATION.md +79 -0
- data/docs/SCHEMA_VALIDATION.md +63 -0
- data/docs/SLO-PROMQL-ALERTS.md +161 -0
- data/docs/TESTING.md +69 -0
- data/docs/{ADR-001-architecture.md → architecture/ADR-001-architecture.md} +36 -65
- data/docs/{ADR-002-metrics-yabeda.md → architecture/ADR-002-metrics-yabeda.md} +62 -236
- data/docs/architecture/ADR-003-slo-observability.md +1402 -0
- data/docs/{ADR-004-adapter-architecture.md → architecture/ADR-004-adapter-architecture.md} +163 -146
- data/docs/{ADR-005-tracing-context.md → architecture/ADR-005-tracing-context.md} +10 -9
- data/docs/{ADR-006-security-compliance.md → architecture/ADR-006-security-compliance.md} +184 -191
- data/docs/{ADR-007-opentelemetry-integration.md → architecture/ADR-007-opentelemetry-integration.md} +3 -21
- data/docs/{ADR-008-rails-integration.md → architecture/ADR-008-rails-integration.md} +182 -743
- data/docs/{ADR-009-cost-optimization.md → architecture/ADR-009-cost-optimization.md} +45 -54
- data/docs/architecture/ADR-010-developer-experience.md +522 -0
- data/docs/{ADR-011-testing-strategy.md → architecture/ADR-011-testing-strategy.md} +44 -86
- data/docs/{ADR-012-event-evolution.md → architecture/ADR-012-event-evolution.md} +11 -11
- data/docs/{ADR-013-reliability-error-handling.md → architecture/ADR-013-reliability-error-handling.md} +37 -12
- data/docs/{ADR-014-event-driven-slo.md → architecture/ADR-014-event-driven-slo.md} +12 -24
- data/docs/{ADR-015-middleware-order.md → architecture/ADR-015-middleware-order.md} +43 -59
- data/docs/{ADR-016-self-monitoring-slo.md → architecture/ADR-016-self-monitoring-slo.md} +58 -355
- data/docs/{ADR-017-multi-rails-compatibility.md → architecture/ADR-017-multi-rails-compatibility.md} +4 -11
- data/docs/architecture/ADR-018-memory-optimization.md +366 -0
- data/docs/{ADR-INDEX.md → architecture/ADR-INDEX.md} +11 -6
- data/docs/plans/2026-03-20-browser-overlay-svelte.md +281 -0
- data/docs/{00-ICP-AND-TIMELINE.md → prd/00-ICP-AND-TIMELINE.md} +6 -6
- data/docs/{01-SCALE-REQUIREMENTS.md → prd/01-SCALE-REQUIREMENTS.md} +6 -6
- data/docs/prd/01-overview-vision.md +19 -14
- data/docs/use_cases/README.md +22 -23
- data/docs/use_cases/UC-001-request-scoped-debug-buffering.md +50 -44
- data/docs/use_cases/UC-002-business-event-tracking.md +26 -95
- data/docs/use_cases/UC-003-event-metrics.md +66 -0
- data/docs/use_cases/UC-004-zero-config-slo-tracking.md +33 -684
- data/docs/use_cases/UC-005-sentry-integration.md +13 -15
- data/docs/use_cases/UC-006-trace-context-management.md +30 -28
- data/docs/use_cases/UC-007-pii-filtering.md +35 -87
- data/docs/use_cases/UC-008-opentelemetry-integration.md +51 -89
- data/docs/use_cases/UC-009-multi-service-tracing.md +30 -178
- data/docs/use_cases/UC-010-background-job-tracking.md +24 -91
- data/docs/use_cases/UC-011-rate-limiting.md +95 -168
- data/docs/use_cases/UC-012-audit-trail.md +21 -46
- data/docs/use_cases/UC-013-high-cardinality-protection.md +29 -167
- data/docs/use_cases/UC-014-adaptive-sampling.md +2 -2
- data/docs/use_cases/UC-015-cost-optimization.md +46 -99
- data/docs/use_cases/UC-016-rails-logger-migration.md +39 -213
- data/docs/use_cases/UC-017-local-development.md +203 -777
- data/docs/use_cases/UC-018-testing-events.md +3 -3
- data/docs/use_cases/UC-019-retention-based-routing.md +53 -106
- data/docs/use_cases/UC-020-event-versioning.md +8 -9
- data/docs/use_cases/UC-021-error-handling-retry-dlq.md +18 -22
- data/docs/use_cases/UC-022-event-registry.md +15 -21
- data/docs/use_cases/backlog.md +119 -87
- data/e11y.gemspec +2 -2
- data/gems/e11y-devtools/README.md +158 -0
- data/gems/e11y-devtools/config/routes.rb +15 -0
- data/gems/e11y-devtools/e11y-devtools.gemspec +25 -0
- data/gems/e11y-devtools/exe/e11y +34 -0
- data/gems/e11y-devtools/frontend/.gitignore +24 -0
- data/gems/e11y-devtools/frontend/README.md +51 -0
- data/gems/e11y-devtools/frontend/index.html +14 -0
- data/gems/e11y-devtools/frontend/package-lock.json +3707 -0
- data/gems/e11y-devtools/frontend/package.json +28 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/events/recent.json +4205 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/interactions.json +194 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/0a2e04027cfa22d014bc22e8b27cd913/events.json +86 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/0e1543af6a630fb3af6b52283154b3e0/events.json +169 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/1838b691faa49564f97db8592ff3978d/events.json +78 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/29f198f6588dacffb687777eb5f8f118/events.json +197 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/34bc3c9c0097de28a7a6f99b90a8e7bc/events.json +194 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/3ba6c20d068ab9cee00e51b180e66444/events.json +184 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/435bfd8f17b9009146a79812d7c3726d/events.json +144 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/4c7676e3fe668e99edb2b94d7d5678a9/events.json +222 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/6daf0d47974bedfc55d5de7004a3ea9f/events.json +194 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/8a81ada42834d15f287bb40010043605/events.json +194 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/8c0a98900edaae105469df8daedccf02/events.json +198 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/8e4f645180f8a7d1dce426b07380466b/events.json +222 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/93db346fa5d44a032605a13b627f4b80/events.json +128 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/98ff6146faf7bd9be8bd03a8275817ba/events.json +223 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/9997ddd0247bc7e25f2ca7a5c415c93d/events.json +197 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/99e35f8ef3baedd798cc4fd085980ad9/events.json +194 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/b4f3095c1909924cbc98889a86c83d6d/events.json +131 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/b54b7fc32b7575a7110de809d11ccda0/events.json +128 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/c0b48033fa06746bcc5886745e053cff/events.json +169 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/c44649ac76701b4558927cd2305ab535/events.json +169 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/d601ae3320057580a39dbdac2edfdf4a/events.json +248 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/e67e724bab422d2b52eeb49635e512e1/events.json +194 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/e6c72765a28f158a8485b35fa63f73da/events.json +194 -0
- data/gems/e11y-devtools/frontend/public/mocks/v1/traces/f541b87405c9a54819b18ebe529f6419/events.json +194 -0
- data/gems/e11y-devtools/frontend/scripts/generate_mocks.rb +397 -0
- data/gems/e11y-devtools/frontend/src/App.svelte +827 -0
- data/gems/e11y-devtools/frontend/src/components/Fab.svelte +19 -0
- data/gems/e11y-devtools/frontend/src/components/FilterBar.svelte +38 -0
- data/gems/e11y-devtools/frontend/src/components/FullscreenPanel.svelte +82 -0
- data/gems/e11y-devtools/frontend/src/components/InteractionsTimeline.svelte +264 -0
- data/gems/e11y-devtools/frontend/src/components/RecentHistogram.svelte +354 -0
- data/gems/e11y-devtools/frontend/src/lib/api.ts +37 -0
- data/gems/e11y-devtools/frontend/src/lib/eventIdentity.ts +12 -0
- data/gems/e11y-devtools/frontend/src/lib/format.ts +37 -0
- data/gems/e11y-devtools/frontend/src/lib/listFilter.ts +43 -0
- data/gems/e11y-devtools/frontend/src/lib/recentVolume.ts +80 -0
- data/gems/e11y-devtools/frontend/src/lib/router.ts +12 -0
- data/gems/e11y-devtools/frontend/src/lib/transitions.ts +34 -0
- data/gems/e11y-devtools/frontend/src/lib/viewportOrigin.ts +25 -0
- data/gems/e11y-devtools/frontend/src/main.ts +8 -0
- data/gems/e11y-devtools/frontend/src/overlay-entry.ts +24 -0
- data/gems/e11y-devtools/frontend/src/overlay.css +1080 -0
- data/gems/e11y-devtools/frontend/svelte.config.js +2 -0
- data/gems/e11y-devtools/frontend/test_puppeteer.js +41 -0
- data/gems/e11y-devtools/frontend/test_scale.js +3 -0
- data/gems/e11y-devtools/frontend/tsconfig.app.json +21 -0
- data/gems/e11y-devtools/frontend/tsconfig.json +7 -0
- data/gems/e11y-devtools/frontend/tsconfig.node.json +26 -0
- data/gems/e11y-devtools/frontend/vite.config.ts +36 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/server.rb +96 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tool_base.rb +25 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/clear.rb +31 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/errors.rb +35 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/event_detail.rb +33 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/events_by_trace.rb +33 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/interactions.rb +40 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/recent_events.rb +34 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/search.rb +34 -0
- data/gems/e11y-devtools/lib/e11y/devtools/mcp/tools/stats.rb +30 -0
- data/gems/e11y-devtools/lib/e11y/devtools/overlay/assets/overlay.js +20 -0
- data/gems/e11y-devtools/lib/e11y/devtools/overlay/controller.rb +94 -0
- data/gems/e11y-devtools/lib/e11y/devtools/overlay/engine.rb +26 -0
- data/gems/e11y-devtools/lib/e11y/devtools/overlay/middleware.rb +80 -0
- data/gems/e11y-devtools/lib/e11y/devtools/overlay/rails_controller.rb +67 -0
- data/gems/e11y-devtools/lib/e11y/devtools/tui/app.rb +262 -0
- data/gems/e11y-devtools/lib/e11y/devtools/tui/grouping.rb +66 -0
- data/gems/e11y-devtools/lib/e11y/devtools/tui/widgets/event_detail.rb +62 -0
- data/gems/e11y-devtools/lib/e11y/devtools/tui/widgets/event_list.rb +70 -0
- data/gems/e11y-devtools/lib/e11y/devtools/tui/widgets/interaction_list.rb +47 -0
- data/gems/e11y-devtools/lib/e11y/devtools/version.rb +8 -0
- data/gems/e11y-devtools/lib/e11y/devtools.rb +13 -0
- data/gems/e11y-devtools/spec/e11y/devtools/mcp/tools_spec.rb +107 -0
- data/gems/e11y-devtools/spec/e11y/devtools/overlay/controller_spec.rb +91 -0
- data/gems/e11y-devtools/spec/e11y/devtools/overlay/middleware_spec.rb +46 -0
- data/gems/e11y-devtools/spec/e11y/devtools/tui/app_spec.rb +85 -0
- data/gems/e11y-devtools/spec/e11y/devtools/tui/grouping_spec.rb +64 -0
- data/gems/e11y-devtools/spec/spec_helper.rb +5 -0
- data/gems/e11y-devtools/spec/tui/widgets/event_list_spec.rb +44 -0
- data/gems/e11y-devtools/spec/tui/widgets/interaction_list_spec.rb +62 -0
- data/lib/e11y/adapters/audit_encrypted.rb +53 -11
- data/lib/e11y/adapters/base.rb +33 -34
- data/lib/e11y/adapters/dev_log/file_store.rb +143 -0
- data/lib/e11y/adapters/dev_log/query.rb +219 -0
- data/lib/e11y/adapters/dev_log.rb +118 -0
- data/lib/e11y/adapters/file.rb +3 -6
- data/lib/e11y/adapters/in_memory.rb +52 -5
- data/lib/e11y/adapters/in_memory_test.rb +29 -0
- data/lib/e11y/adapters/loki.rb +58 -23
- data/lib/e11y/adapters/null.rb +82 -0
- data/lib/e11y/adapters/opentelemetry_collector.rb +183 -0
- data/lib/e11y/adapters/otel_logs.rb +136 -23
- data/lib/e11y/adapters/sentry.rb +4 -7
- data/lib/e11y/adapters/stdout.rb +73 -7
- data/lib/e11y/adapters/yabeda.rb +153 -29
- data/lib/e11y/buffers/adaptive_buffer.rb +3 -17
- data/lib/e11y/buffers/{request_scoped_buffer.rb → ephemeral_buffer.rb} +72 -58
- data/lib/e11y/buffers/ring_buffer.rb +3 -16
- data/lib/e11y/configuration.rb +272 -0
- data/lib/e11y/console.rb +10 -17
- data/lib/e11y/current.rb +53 -1
- data/lib/e11y/debug/pipeline_inspector.rb +96 -0
- data/lib/e11y/documentation/generator.rb +48 -0
- data/lib/e11y/event/base.rb +176 -82
- data/lib/e11y/event/value_sampling_config.rb +1 -5
- data/lib/e11y/events/rails/database/query.rb +1 -4
- data/lib/e11y/events/rails/job/failed.rb +2 -0
- data/lib/e11y/instruments/active_job.rb +44 -12
- data/lib/e11y/instruments/rails_instrumentation.rb +49 -24
- data/lib/e11y/instruments/sidekiq.rb +135 -31
- data/lib/e11y/linters/base.rb +11 -0
- data/lib/e11y/linters/pii/pii_declaration_linter.rb +120 -0
- data/lib/e11y/linters/slo/config_consistency_linter.rb +76 -0
- data/lib/e11y/linters/slo/explicit_declaration_linter.rb +36 -0
- data/lib/e11y/linters/slo/slo_status_from_linter.rb +41 -0
- data/lib/e11y/logger/bridge.rb +26 -7
- data/lib/e11y/metrics/cardinality_protection.rb +10 -15
- data/lib/e11y/metrics/cardinality_tracker.rb +16 -6
- data/lib/e11y/metrics/registry.rb +3 -5
- data/lib/e11y/metrics/test_backend.rb +62 -0
- data/lib/e11y/metrics.rb +56 -10
- data/lib/e11y/middleware/adapter_resolver.rb +40 -0
- data/lib/e11y/middleware/audit_signing.rb +43 -6
- data/lib/e11y/middleware/baggage_protection.rb +75 -0
- data/lib/e11y/middleware/dev_log_source.rb +24 -0
- data/lib/e11y/middleware/event_slo.rb +23 -9
- data/lib/e11y/middleware/otel_span.rb +23 -0
- data/lib/e11y/middleware/pii_filter.rb +104 -75
- data/lib/e11y/middleware/rate_limiting.rb +54 -27
- data/lib/e11y/middleware/request.rb +70 -23
- data/lib/e11y/middleware/routing.rb +78 -21
- data/lib/e11y/middleware/sampling.rb +66 -17
- data/lib/e11y/middleware/self_monitoring_emit.rb +39 -0
- data/lib/e11y/middleware/trace_context.rb +45 -10
- data/lib/e11y/middleware/track_latency.rb +34 -0
- data/lib/e11y/middleware/validation.rb +7 -16
- data/lib/e11y/middleware/versioning.rb +26 -22
- data/lib/e11y/opentelemetry/semantic_conventions.rb +109 -0
- data/lib/e11y/opentelemetry/span_creator.rb +142 -0
- data/lib/e11y/pii/patterns.rb +12 -1
- data/lib/e11y/pipeline/builder.rb +4 -4
- data/lib/e11y/presets/audit_event.rb +13 -2
- data/lib/e11y/railtie.rb +52 -14
- data/lib/e11y/registry.rb +306 -0
- data/lib/e11y/reliability/circuit_breaker.rb +19 -21
- data/lib/e11y/reliability/dlq/base.rb +71 -0
- data/lib/e11y/reliability/dlq/file_adapter.rb +301 -0
- data/lib/e11y/reliability/dlq/file_storage.rb +63 -34
- data/lib/e11y/reliability/dlq/filter.rb +37 -54
- data/lib/e11y/reliability/retry_handler.rb +26 -29
- data/lib/e11y/reliability/retry_rate_limiter.rb +3 -11
- data/lib/e11y/sampling/error_spike_detector.rb +0 -2
- data/lib/e11y/sampling/load_monitor.rb +5 -9
- data/lib/e11y/sampling/stratified_tracker.rb +18 -0
- data/lib/e11y/self_monitoring/buffer_monitor.rb +2 -0
- data/lib/e11y/self_monitoring/performance_monitor.rb +19 -61
- data/lib/e11y/self_monitoring/reliability_monitor.rb +4 -74
- data/lib/e11y/slo/config_loader.rb +40 -0
- data/lib/e11y/slo/config_validator.rb +58 -0
- data/lib/e11y/slo/dashboard_generator.rb +122 -0
- data/lib/e11y/slo/event_driven.rb +8 -0
- data/lib/e11y/slo/tracker.rb +31 -4
- data/lib/e11y/testing/have_tracked_event_matcher.rb +190 -0
- data/lib/e11y/testing/rspec_matchers.rb +21 -0
- data/lib/e11y/testing/snapshot_matcher.rb +86 -0
- data/lib/e11y/trace_context/sampler.rb +35 -0
- data/lib/e11y/tracing/faraday_middleware.rb +31 -0
- data/lib/e11y/tracing/net_http_patch.rb +33 -0
- data/lib/e11y/tracing/propagator.rb +144 -0
- data/lib/e11y/tracing.rb +47 -0
- data/lib/e11y/version.rb +1 -1
- data/lib/e11y/versioning/version_extractor.rb +32 -0
- data/lib/e11y.rb +123 -266
- data/lib/generators/e11y/event/event_generator.rb +22 -0
- data/lib/generators/e11y/event/templates/event.rb.tt +16 -0
- data/lib/generators/e11y/grafana_dashboard/grafana_dashboard_generator.rb +30 -0
- data/lib/generators/e11y/grafana_dashboard/templates/e11y_dashboard.json +81 -0
- data/lib/generators/e11y/install/install_generator.rb +34 -0
- data/lib/generators/e11y/install/templates/e11y.rb +239 -0
- data/lib/generators/e11y/prometheus_alerts/prometheus_alerts_generator.rb +29 -0
- data/lib/generators/e11y/prometheus_alerts/templates/e11y_alerts.yml +28 -0
- data/lib/tasks/e11y_docs.rake +30 -0
- data/lib/tasks/e11y_events.rake +71 -0
- data/lib/tasks/e11y_lint.rake +91 -0
- data/lib/tasks/e11y_slo.rake +29 -0
- metadata +186 -39
- data/docs/ADR-003-slo-observability.md +0 -3337
- data/docs/ADR-010-developer-experience.md +0 -2166
- data/docs/API-REFERENCE-L28.md +0 -914
- data/docs/COMPREHENSIVE-CONFIGURATION.md +0 -2366
- data/docs/CONTRIBUTING.md +0 -312
- data/docs/IMPLEMENTATION_NOTES.md +0 -2804
- data/docs/IMPLEMENTATION_PLAN.md +0 -1971
- data/docs/IMPLEMENTATION_PLAN_ARCHITECTURE.md +0 -586
- data/docs/PLAN.md +0 -148
- data/docs/README.md +0 -296
- data/docs/design/00-memory-optimization.md +0 -593
- data/docs/guides/MIGRATION-L27-L28.md +0 -692
- data/docs/guides/PERFORMANCE-BENCHMARKS.md +0 -434
- data/docs/guides/README.md +0 -44
- data/docs/use_cases/UC-003-pattern-based-metrics.md +0 -1627
- data/lib/e11y/adapters/registry.rb +0 -141
data/docs/QUICK-START.md
CHANGED
|
@@ -1,234 +1,265 @@
|
|
|
1
1
|
# E11y - Quick Start Guide
|
|
2
2
|
|
|
3
|
-
> **TL;DR**: Ruby gem
|
|
3
|
+
> **TL;DR**: Ruby gem for structured business events with request-scoped debug buffering,
|
|
4
|
+
> schema validation, and pluggable adapters (Loki, Sentry, OpenTelemetry, Yabeda, etc.).
|
|
5
|
+
>
|
|
6
|
+
> **Current version: 0.2.0** — ⚠️ Work in Progress, production validation in progress.
|
|
4
7
|
|
|
5
8
|
---
|
|
6
9
|
|
|
7
|
-
## 🚀 Installation
|
|
10
|
+
## 🚀 Installation
|
|
8
11
|
|
|
9
12
|
```bash
|
|
10
13
|
# Gemfile
|
|
11
|
-
gem 'e11y', '~>
|
|
14
|
+
gem 'e11y', '~> 0.2'
|
|
12
15
|
|
|
13
16
|
bundle install
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Then run the generator to create the initializer and example event:
|
|
20
|
+
|
|
21
|
+
```bash
|
|
14
22
|
rails g e11y:install
|
|
15
23
|
```
|
|
16
24
|
|
|
25
|
+
This creates:
|
|
26
|
+
- `config/initializers/e11y.rb` — base configuration with comments
|
|
27
|
+
- `app/events/` — directory for event classes (if it doesn't exist)
|
|
28
|
+
|
|
29
|
+
> **Available generators:**
|
|
30
|
+
> - `rails g e11y:install` — initializer + directory scaffold
|
|
31
|
+
> - `rails g e11y:grafana_dashboard` — Grafana dashboard JSON (requires Yabeda/Prometheus)
|
|
32
|
+
> - `rails g e11y:prometheus_alerts` — Prometheus alerting rules
|
|
33
|
+
|
|
34
|
+
Or create the initializer manually:
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
touch config/initializers/e11y.rb
|
|
38
|
+
```
|
|
39
|
+
|
|
17
40
|
---
|
|
18
41
|
|
|
19
42
|
## 🎯 Killer Features
|
|
20
43
|
|
|
21
44
|
### 1. Request-Scoped Debug Buffering
|
|
22
45
|
|
|
23
|
-
|
|
46
|
+
**Problem**: Debug logs in production = noise. Without debug = no context when errors occur.
|
|
24
47
|
|
|
25
|
-
|
|
48
|
+
**Solution**: Debug events are buffered in thread-local storage. On success — discarded.
|
|
49
|
+
On error — flushed with full context.
|
|
26
50
|
|
|
27
|
-
```
|
|
51
|
+
```
|
|
28
52
|
GET /api/orders/123
|
|
29
53
|
├─ [debug] Query: SELECT... (buffered, NOT sent)
|
|
30
54
|
├─ [debug] Cache miss (buffered, NOT sent)
|
|
31
|
-
├─ [ERROR] Payment failed ←
|
|
32
|
-
└─> FLUSH
|
|
55
|
+
├─ [ERROR] Payment failed ← 5xx raised
|
|
56
|
+
└─> FLUSH: all buffered debug events are sent to the adapter!
|
|
33
57
|
```
|
|
34
58
|
|
|
35
|
-
|
|
59
|
+
> **Note:** By default the buffer flushes only on **5xx server errors** (`flush_on_error = true`).
|
|
60
|
+
> On 4xx responses and successful requests the buffer is discarded.
|
|
61
|
+
>
|
|
62
|
+
> Two independent knobs let you customize this — see [Configuration](#-configuration):
|
|
63
|
+
> - `flush_on_error = false` — disable the 5xx auto-flush
|
|
64
|
+
> - `flush_on_statuses = [403]` — add extra statuses (independent of `flush_on_error`)
|
|
65
|
+
|
|
66
|
+
**Result**: Debug logs only when needed, zero overhead on the happy path.
|
|
36
67
|
|
|
37
|
-
### 2.
|
|
68
|
+
### 2. `:success` Pseudo-Severity
|
|
38
69
|
|
|
39
|
-
|
|
70
|
+
A new severity level between `:info` and `:warn` for successful operations.
|
|
40
71
|
|
|
41
72
|
```ruby
|
|
42
|
-
Events::OrderPaid.track(
|
|
43
|
-
|
|
44
|
-
severity: :success # ← легко фильтровать успехи
|
|
45
|
-
)
|
|
73
|
+
Events::OrderPaid.track(order_id: '123', amount: 99.99, currency: 'USD')
|
|
74
|
+
# severity is automatically :success (inferred from class name — "Paid")
|
|
46
75
|
|
|
47
|
-
#
|
|
48
|
-
|
|
76
|
+
# Or explicitly:
|
|
77
|
+
Events::SomeEvent.track(order_id: '123', severity: :success)
|
|
49
78
|
```
|
|
50
79
|
|
|
51
|
-
|
|
80
|
+
**In Grafana/Kibana:**
|
|
81
|
+
```
|
|
82
|
+
{severity="success", event_name="OrderPaid"}
|
|
83
|
+
```
|
|
52
84
|
|
|
53
|
-
|
|
85
|
+
**Result**: Visibility into successes, not just errors. Easy to build success rate dashboards.
|
|
54
86
|
|
|
55
|
-
|
|
87
|
+
### 3. Schema Validation (dry-schema)
|
|
56
88
|
|
|
57
89
|
```ruby
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
# Histogram для всех оплат
|
|
64
|
-
histogram_for pattern: '*.paid',
|
|
65
|
-
value: ->(e) { e.payload[:amount] }
|
|
90
|
+
class Events::OrderPaid < E11y::Event::Base
|
|
91
|
+
schema do
|
|
92
|
+
required(:order_id).filled(:string)
|
|
93
|
+
required(:amount).filled(:float, gt?: 0)
|
|
94
|
+
required(:currency).filled(:string, included_in?: %w[USD EUR GBP])
|
|
66
95
|
end
|
|
67
96
|
end
|
|
68
|
-
```
|
|
69
97
|
|
|
70
|
-
|
|
98
|
+
Events::OrderPaid.track(order_id: nil, amount: -10)
|
|
99
|
+
# => E11y::ValidationError: Validation failed for OrderPaid: {:order_id=>["must be filled"], ...}
|
|
100
|
+
```
|
|
71
101
|
|
|
72
|
-
### 4. Trace Context (
|
|
102
|
+
### 4. Trace Context (W3C + OTel + Sentry fallback)
|
|
73
103
|
|
|
74
|
-
|
|
104
|
+
Automatic trace_id extraction with fallback chain:
|
|
75
105
|
|
|
76
|
-
```
|
|
77
|
-
|
|
78
|
-
1. X-Trace-ID header
|
|
106
|
+
```
|
|
107
|
+
1. traceparent header (W3C Trace Context)
|
|
79
108
|
2. X-Request-ID header
|
|
80
|
-
3.
|
|
81
|
-
4.
|
|
82
|
-
5. Generate new UUID v7
|
|
109
|
+
3. X-Trace-Id header (custom)
|
|
110
|
+
4. Generate new UUID
|
|
83
111
|
```
|
|
84
112
|
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
### 5. Built-in SLO Tracking (Zero Config!)
|
|
113
|
+
**Result**: Correlated events across services with no extra code.
|
|
88
114
|
|
|
89
|
-
|
|
115
|
+
### 5. Built-in SLO Tracking
|
|
90
116
|
|
|
91
117
|
```ruby
|
|
92
118
|
E11y.configure do |config|
|
|
93
|
-
config.
|
|
119
|
+
config.rails_instrumentation_enabled = true
|
|
120
|
+
# config.slo_tracking_enabled = true # enabled by default
|
|
94
121
|
end
|
|
95
122
|
|
|
96
|
-
#
|
|
97
|
-
#
|
|
98
|
-
#
|
|
99
|
-
#
|
|
100
|
-
# ✅ Error budget + burn rate
|
|
101
|
-
# ✅ Grafana dashboards (generate)
|
|
102
|
-
# ✅ Prometheus alerts (generate)
|
|
123
|
+
# Automatically emits metrics:
|
|
124
|
+
# e11y_http_requests_total, e11y_http_request_duration_seconds
|
|
125
|
+
# e11y_sidekiq_jobs_total, e11y_sidekiq_job_duration_seconds
|
|
126
|
+
# e11y_active_jobs_total, e11y_active_job_duration_seconds
|
|
103
127
|
```
|
|
104
128
|
|
|
105
|
-
|
|
129
|
+
> ⚠️ **Caveat:** SLO metrics may be imprecise when adaptive sampling is enabled
|
|
130
|
+
> (sampling correction is planned for Phase 2.8).
|
|
131
|
+
|
|
132
|
+
> 🚧 **Roadmap:** Per-controller/per-job SLO configuration, auto-generated Grafana dashboards
|
|
133
|
+
> and Prometheus alerts — planned for future releases.
|
|
106
134
|
|
|
107
135
|
---
|
|
108
136
|
|
|
109
137
|
## ⚡ Quick Examples
|
|
110
138
|
|
|
111
|
-
### Basic Event (
|
|
112
|
-
|
|
113
|
-
> **🎯 NEW in v1.1:** Event-level configuration reduces global config from 1400+ to <300 lines!
|
|
139
|
+
### Basic Event (minimal configuration)
|
|
114
140
|
|
|
115
141
|
```ruby
|
|
116
|
-
# 1. Define event (
|
|
142
|
+
# 1. Define the event (app/events/order_paid.rb)
|
|
117
143
|
class Events::OrderPaid < E11y::Event::Base
|
|
118
|
-
# Schema (dry-schema для валидации)
|
|
119
144
|
schema do
|
|
120
145
|
required(:order_id).filled(:string)
|
|
121
|
-
required(:amount).filled(:
|
|
146
|
+
required(:amount).filled(:float)
|
|
122
147
|
required(:currency).filled(:string)
|
|
123
148
|
end
|
|
124
|
-
|
|
125
|
-
#
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
name: 'orders.paid.total',
|
|
135
|
-
tags: [:currency]
|
|
149
|
+
|
|
150
|
+
# Optional: Prometheus metrics via Yabeda
|
|
151
|
+
# NOTE: The metrics do...end DSL requires the Yabeda adapter to be registered.
|
|
152
|
+
# Without E11y::Adapters::Yabeda.new in config.adapters, metric definitions are
|
|
153
|
+
# stored but never updated. See the Yabeda / Prometheus Integration section below.
|
|
154
|
+
metrics do
|
|
155
|
+
counter :orders_paid_total, tags: [:currency]
|
|
156
|
+
histogram :order_amount, value: :amount, tags: [:currency],
|
|
157
|
+
buckets: [10, 50, 100, 500, 1000]
|
|
158
|
+
end
|
|
136
159
|
end
|
|
137
160
|
|
|
138
|
-
# 2. Track
|
|
161
|
+
# 2. Track — only this form, no alternatives
|
|
139
162
|
Events::OrderPaid.track(
|
|
140
163
|
order_id: '123',
|
|
141
164
|
amount: 99.99,
|
|
142
165
|
currency: 'USD'
|
|
143
166
|
)
|
|
144
|
-
|
|
145
|
-
# ❌ НЕТ других способов:
|
|
146
|
-
# E11y.track_event(...) - НЕТ!
|
|
147
|
-
# Severity.track(...) - НЕТ!
|
|
148
167
|
```
|
|
149
168
|
|
|
150
|
-
**90%
|
|
169
|
+
**90% of events — just a schema, everything else from conventions:**
|
|
151
170
|
|
|
152
171
|
```ruby
|
|
153
|
-
# Conventions = sensible defaults!
|
|
154
172
|
class Events::OrderCreated < E11y::Event::Base
|
|
155
173
|
schema do
|
|
156
174
|
required(:order_id).filled(:string)
|
|
157
|
-
required(:amount).filled(:
|
|
175
|
+
required(:amount).filled(:float)
|
|
176
|
+
end
|
|
177
|
+
# severity: :success (auto, inferred from "Created")
|
|
178
|
+
# adapters: [:logs] (auto, from severity via adapter_mapping)
|
|
179
|
+
# sample_rate: 0.1 (auto, from severity)
|
|
180
|
+
# retention_period: 30.days (from config.default_retention_period)
|
|
181
|
+
end
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
**Explicit configuration at the event-class level:**
|
|
185
|
+
|
|
186
|
+
```ruby
|
|
187
|
+
class Events::PaymentSucceeded < E11y::Event::Base
|
|
188
|
+
schema do
|
|
189
|
+
required(:transaction_id).filled(:string)
|
|
190
|
+
required(:amount).filled(:float)
|
|
158
191
|
end
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
#
|
|
162
|
-
#
|
|
163
|
-
|
|
164
|
-
# rate_limit: 1000 (default)
|
|
192
|
+
|
|
193
|
+
severity :success
|
|
194
|
+
sample_rate 1.0 # never sample-out payment events
|
|
195
|
+
retention_period 7.years # financial records (also: `retention 7.years` is a valid alias)
|
|
196
|
+
adapters :logs, :errors_tracker
|
|
165
197
|
end
|
|
166
198
|
```
|
|
167
199
|
|
|
168
|
-
**Inheritance
|
|
200
|
+
**Inheritance for DRY configuration:**
|
|
169
201
|
|
|
170
202
|
```ruby
|
|
171
|
-
# Base class для payment events
|
|
172
203
|
module Events
|
|
173
204
|
class BasePaymentEvent < E11y::Event::Base
|
|
174
205
|
severity :success
|
|
175
|
-
sample_rate 1.0
|
|
176
|
-
|
|
177
|
-
adapters
|
|
206
|
+
sample_rate 1.0
|
|
207
|
+
retention_period 7.years
|
|
208
|
+
adapters :logs, :errors_tracker
|
|
178
209
|
end
|
|
179
210
|
end
|
|
180
211
|
|
|
181
|
-
# Inherit from base (1-2 lines per event!)
|
|
182
212
|
class Events::PaymentSucceeded < Events::BasePaymentEvent
|
|
183
213
|
schema do
|
|
184
214
|
required(:transaction_id).filled(:string)
|
|
185
|
-
required(:amount).filled(:
|
|
215
|
+
required(:amount).filled(:float)
|
|
186
216
|
end
|
|
187
|
-
#
|
|
217
|
+
# Inherits ALL configuration from BasePaymentEvent
|
|
188
218
|
end
|
|
189
219
|
```
|
|
190
220
|
|
|
191
|
-
**Preset modules
|
|
221
|
+
**Preset modules:**
|
|
192
222
|
|
|
193
223
|
```ruby
|
|
194
224
|
class Events::PaymentProcessed < E11y::Event::Base
|
|
195
|
-
include E11y::Presets::HighValueEvent
|
|
196
|
-
|
|
225
|
+
include E11y::Presets::HighValueEvent
|
|
226
|
+
# → severity :success, sample_rate 1.0, adapters [:logs, :errors_tracker]
|
|
227
|
+
|
|
197
228
|
schema do
|
|
198
229
|
required(:transaction_id).filled(:string)
|
|
199
|
-
required(:amount).filled(:
|
|
230
|
+
required(:amount).filled(:float)
|
|
200
231
|
end
|
|
201
232
|
end
|
|
202
|
-
```
|
|
203
233
|
|
|
204
|
-
|
|
234
|
+
class Events::UserDeleted < E11y::Event::Base
|
|
235
|
+
include E11y::Presets::AuditEvent
|
|
236
|
+
# → sample_rate 1.0, signing enabled, never rate-limited
|
|
205
237
|
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
238
|
+
schema do
|
|
239
|
+
required(:user_id).filled(:string)
|
|
240
|
+
required(:deleted_by).filled(:string)
|
|
241
|
+
end
|
|
210
242
|
end
|
|
211
|
-
# → event.duration_ms = 250
|
|
212
243
|
```
|
|
213
244
|
|
|
214
245
|
### Request-Scoped Debug Buffering
|
|
215
246
|
|
|
216
247
|
```ruby
|
|
217
|
-
# Middleware (auto-configured)
|
|
218
248
|
class OrdersController < ApplicationController
|
|
219
249
|
def create
|
|
220
|
-
# Debug events buffered
|
|
221
|
-
Events::ValidationStarted.track(severity: :debug)
|
|
250
|
+
# Debug events are buffered — not sent immediately
|
|
251
|
+
Events::ValidationStarted.track(severity: :debug, params: params.keys)
|
|
222
252
|
Events::DatabaseQuery.track(sql: '...', severity: :debug)
|
|
223
|
-
|
|
224
|
-
order = Order.create!(
|
|
225
|
-
|
|
226
|
-
#
|
|
227
|
-
Events::OrderCreated.track(order_id: order.id,
|
|
228
|
-
|
|
253
|
+
|
|
254
|
+
order = Order.create!(order_params)
|
|
255
|
+
|
|
256
|
+
# Non-debug events are sent immediately (not buffered)
|
|
257
|
+
Events::OrderCreated.track(order_id: order.id, amount: order.total)
|
|
258
|
+
|
|
229
259
|
render json: order
|
|
260
|
+
# ← Successful request: debug buffer is discarded
|
|
230
261
|
rescue => e
|
|
231
|
-
#
|
|
262
|
+
# ← 5xx (or any configured flush status): debug buffer is flushed with error context
|
|
232
263
|
raise
|
|
233
264
|
end
|
|
234
265
|
end
|
|
@@ -236,266 +267,186 @@ end
|
|
|
236
267
|
|
|
237
268
|
---
|
|
238
269
|
|
|
239
|
-
## 🔧 Configuration
|
|
240
|
-
|
|
241
|
-
> **🎯 NEW in v1.1:** Global config reduced from 1400+ to <300 lines!
|
|
242
|
-
>
|
|
243
|
-
> **Philosophy:** Global config = infrastructure only. Event config = in event classes.
|
|
244
|
-
|
|
245
|
-
### Global Config (Infrastructure Only)
|
|
270
|
+
## 🔧 Configuration
|
|
246
271
|
|
|
247
272
|
```ruby
|
|
248
|
-
# config/initializers/e11y.rb
|
|
273
|
+
# config/initializers/e11y.rb
|
|
249
274
|
E11y.configure do |config|
|
|
250
|
-
# ===
|
|
251
|
-
config.
|
|
275
|
+
# === Service identity ===
|
|
276
|
+
config.service_name = 'myapp'
|
|
277
|
+
config.environment = Rails.env
|
|
278
|
+
|
|
279
|
+
# === Adapters (key = name, value = instance) ===
|
|
280
|
+
# adapters is a Hash — use [] assignment or register_adapter (both are equivalent):
|
|
281
|
+
config.adapters[:logs] = E11y::Adapters::Loki.new(
|
|
252
282
|
url: ENV['LOKI_URL'],
|
|
253
|
-
|
|
283
|
+
batch_size: 100,
|
|
284
|
+
batch_timeout: 5,
|
|
285
|
+
compress: true
|
|
254
286
|
)
|
|
255
|
-
|
|
256
|
-
config.register_adapter :
|
|
287
|
+
# Equivalent form:
|
|
288
|
+
# config.register_adapter :logs, E11y::Adapters::Loki.new(url: ENV['LOKI_URL'])
|
|
289
|
+
|
|
290
|
+
config.adapters[:errors_tracker] = E11y::Adapters::Sentry.new(
|
|
257
291
|
dsn: ENV['SENTRY_DSN']
|
|
258
292
|
)
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
293
|
+
|
|
294
|
+
# For Prometheus metrics (requires gem 'yabeda' + 'yabeda-prometheus')
|
|
295
|
+
config.adapters[:metrics] = E11y::Adapters::Yabeda.new(
|
|
296
|
+
cardinality_limit: 1000,
|
|
297
|
+
overflow_strategy: :relabel
|
|
262
298
|
)
|
|
263
|
-
|
|
264
|
-
# ===
|
|
265
|
-
config.
|
|
266
|
-
|
|
267
|
-
config.
|
|
268
|
-
|
|
269
|
-
#
|
|
270
|
-
config.
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
#
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
#
|
|
282
|
-
config.
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
config.slo_tracking = true # ← ВСЁ!
|
|
289
|
-
|
|
290
|
-
# === AUDIT RETENTION (global default) ===
|
|
291
|
-
# Default for audit events, can be overridden per event
|
|
292
|
-
config.audit_retention = case ENV['JURISDICTION']
|
|
293
|
-
when 'EU' then 7.years # GDPR
|
|
294
|
-
when 'US' then 10.years # SOX
|
|
295
|
-
else 5.years
|
|
296
|
-
end
|
|
297
|
-
|
|
298
|
-
# ✅ That's it! No per-event config here anymore!
|
|
299
|
+
|
|
300
|
+
# === Request-scoped buffering ===
|
|
301
|
+
config.ephemeral_buffer_enabled = true
|
|
302
|
+
# flush_on_error (default: true) — flush buffer on any 5xx server error
|
|
303
|
+
# config.ephemeral_buffer_flush_on_error = false # disable 5xx auto-flush
|
|
304
|
+
# flush_on_statuses (default: []) — extra statuses, independent of flush_on_error
|
|
305
|
+
# config.ephemeral_buffer_flush_on_statuses = [403] # also flush on 403 Forbidden
|
|
306
|
+
# config.ephemeral_buffer_flush_on_statuses = [401, 403] # multiple codes
|
|
307
|
+
|
|
308
|
+
# === Rails auto-instrumentation (HTTP, ActiveRecord, ActiveJob, Cache) ===
|
|
309
|
+
config.rails_instrumentation_enabled = true
|
|
310
|
+
|
|
311
|
+
# === SLO tracking (enabled by default) ===
|
|
312
|
+
# config.slo_tracking_enabled = true # already true
|
|
313
|
+
|
|
314
|
+
# === Rate limiting (now in default pipeline!) ===
|
|
315
|
+
# Rate limiting is wired into the default pipeline in v0.2.0.
|
|
316
|
+
# Enable and configure parameters:
|
|
317
|
+
# config.rate_limiting_enabled = true
|
|
318
|
+
# config.rate_limiting_global_limit = 10_000 # events/sec
|
|
319
|
+
# config.rate_limiting_per_event_limit = 1_000 # events/sec per type
|
|
320
|
+
# config.rate_limiting_global_window = 1.0 # seconds
|
|
321
|
+
|
|
322
|
+
# === Retention ===
|
|
323
|
+
config.default_retention_period = 30.days
|
|
299
324
|
end
|
|
325
|
+
|
|
326
|
+
# Lifecycle methods (v0.2.0):
|
|
327
|
+
# E11y.start! # start background workers (batching, retry, DLQ)
|
|
328
|
+
# at_exit { E11y.stop!(timeout: 5) } # graceful shutdown
|
|
300
329
|
```
|
|
301
330
|
|
|
302
|
-
###
|
|
331
|
+
### Buffer flush — manual trigger
|
|
332
|
+
|
|
333
|
+
`EphemeralBuffer.flush_on_error` is a public method — you can call it directly in custom
|
|
334
|
+
rescue handlers or background jobs:
|
|
303
335
|
|
|
304
336
|
```ruby
|
|
305
|
-
# app
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
required(:order_id).filled(:string)
|
|
310
|
-
required(:amount).filled(:decimal)
|
|
311
|
-
end
|
|
312
|
-
|
|
313
|
-
# ✨ Event-level config (right next to schema!)
|
|
314
|
-
severity :success
|
|
315
|
-
rate_limit 1000, window: 1.second
|
|
316
|
-
sample_rate 0.1 # 10% sampling
|
|
317
|
-
retention 30.days
|
|
318
|
-
adapters [:loki, :elasticsearch]
|
|
319
|
-
|
|
320
|
-
# Metric definition
|
|
321
|
-
metric :counter,
|
|
322
|
-
name: 'orders.created.total',
|
|
323
|
-
tags: [:currency]
|
|
324
|
-
end
|
|
337
|
+
# Custom error handler (e.g. Grape API, custom Rack app)
|
|
338
|
+
rescue => e
|
|
339
|
+
E11y::Buffers::EphemeralBuffer.flush_on_error
|
|
340
|
+
raise
|
|
325
341
|
end
|
|
342
|
+
|
|
343
|
+
# Or flush to a specific adapter target (not yet implemented — placeholder)
|
|
344
|
+
E11y::Buffers::EphemeralBuffer.flush_on_error(target: :errors_tracker)
|
|
326
345
|
```
|
|
327
346
|
|
|
328
|
-
###
|
|
347
|
+
### Severity → Adapter mapping
|
|
329
348
|
|
|
330
|
-
|
|
349
|
+
Default routing:
|
|
331
350
|
|
|
332
351
|
```ruby
|
|
333
|
-
#
|
|
352
|
+
# error/fatal → [:logs, :errors_tracker]
|
|
353
|
+
# all others → [:logs]
|
|
354
|
+
|
|
355
|
+
# Override globally:
|
|
334
356
|
E11y.configure do |config|
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
event 'Events::OrderCreated' do
|
|
342
|
-
severity :success
|
|
343
|
-
adapters [:loki]
|
|
344
|
-
sample_rate 0.1
|
|
345
|
-
retention 30.days
|
|
346
|
-
rate_limit 1000
|
|
347
|
-
end
|
|
348
|
-
|
|
349
|
-
event 'Events::PaymentSucceeded' do
|
|
350
|
-
severity :success
|
|
351
|
-
adapters [:loki, :sentry, :s3]
|
|
352
|
-
sample_rate 1.0
|
|
353
|
-
retention 7.years
|
|
354
|
-
rate_limit 1000
|
|
355
|
-
end
|
|
356
|
-
|
|
357
|
-
# ... 98+ more events ...
|
|
358
|
-
end
|
|
357
|
+
config.adapter_mapping[:warn] = [:logs, :errors_tracker]
|
|
358
|
+
end
|
|
359
|
+
|
|
360
|
+
# Override per event:
|
|
361
|
+
class Events::CriticalEvent < E11y::Event::Base
|
|
362
|
+
adapters :logs, :errors_tracker
|
|
359
363
|
end
|
|
360
364
|
```
|
|
361
365
|
|
|
362
|
-
|
|
366
|
+
### PII Filtering
|
|
363
367
|
|
|
364
|
-
|
|
365
|
-
# config/initializers/e11y.rb (<300 lines!)
|
|
366
|
-
E11y.configure do |config|
|
|
367
|
-
# ONLY infrastructure
|
|
368
|
-
config.register_adapter :loki, Loki.new(...)
|
|
369
|
-
config.register_adapter :sentry, Sentry.new(...)
|
|
370
|
-
|
|
371
|
-
# Defaults (conventions)
|
|
372
|
-
config.default_adapters = [:loki]
|
|
373
|
-
|
|
374
|
-
# ✅ No per-event config here!
|
|
375
|
-
end
|
|
368
|
+
**Auto (:rails_filters):** E11y automatically applies `Rails.application.config.filter_parameters`.
|
|
376
369
|
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
# ← Uses conventions (zero config!)
|
|
381
|
-
end
|
|
370
|
+
```ruby
|
|
371
|
+
# config/application.rb
|
|
372
|
+
config.filter_parameters += [:password, :email, :ssn]
|
|
382
373
|
|
|
383
|
-
#
|
|
384
|
-
|
|
385
|
-
schema do; required(:transaction_id).filled(:string); end
|
|
386
|
-
# ← Inherits config from BasePaymentEvent (DRY!)
|
|
387
|
-
end
|
|
374
|
+
# These fields become '[FILTERED]' automatically on track
|
|
375
|
+
Events::UserRegistered.track(email: 'user@example.com', password: 'secret')
|
|
388
376
|
```
|
|
389
377
|
|
|
390
|
-
|
|
378
|
+
**Event-level DSL (:explicit_pii):**
|
|
391
379
|
|
|
392
380
|
```ruby
|
|
393
|
-
|
|
381
|
+
class Events::PaymentCreated < E11y::Event::Base
|
|
382
|
+
contains_pii true
|
|
394
383
|
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
384
|
+
pii_filtering do
|
|
385
|
+
masks :card_number # → '[FILTERED]'
|
|
386
|
+
hashes :user_email # → SHA256 (preserves searchability)
|
|
387
|
+
partials :phone # → first/last characters visible
|
|
388
|
+
redacts :ssn # → removed completely
|
|
389
|
+
allows :amount # → no filtering
|
|
401
390
|
end
|
|
402
391
|
end
|
|
392
|
+
```
|
|
403
393
|
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
394
|
+
**Inheritance:** Use a base class for common rules, child events add or override:
|
|
395
|
+
|
|
396
|
+
```ruby
|
|
397
|
+
class BaseUserEvent < E11y::Event::Base
|
|
398
|
+
contains_pii true
|
|
399
|
+
pii_filtering do
|
|
400
|
+
masks :password
|
|
401
|
+
hashes :email
|
|
402
|
+
partials :phone
|
|
403
|
+
end
|
|
407
404
|
end
|
|
408
405
|
|
|
409
|
-
|
|
406
|
+
class Events::PaymentCreated < BaseUserEvent
|
|
407
|
+
pii_filtering do
|
|
408
|
+
masks :card_number, :cvv
|
|
409
|
+
end
|
|
410
|
+
end
|
|
410
411
|
```
|
|
411
412
|
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
## 🔧 Old Configuration (v1.0 - Deprecated)
|
|
415
|
-
|
|
416
|
-
> **⚠️ Deprecated:** This section shows v1.0 configuration style for reference.
|
|
417
|
-
> Use event-level configuration (above) for new projects.
|
|
413
|
+
### Adaptive Sampling
|
|
418
414
|
|
|
419
415
|
```ruby
|
|
420
|
-
# config/initializers/e11y.rb (OLD STYLE)
|
|
421
416
|
E11y.configure do |config|
|
|
422
|
-
|
|
423
|
-
|
|
424
|
-
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
|
|
444
|
-
# Counter for all events
|
|
445
|
-
counter_for pattern: '*',
|
|
446
|
-
name: 'business_events_total',
|
|
447
|
-
tags: [:event_name, :severity]
|
|
448
|
-
|
|
449
|
-
# Histogram for payments
|
|
450
|
-
histogram_for pattern: '*.paid',
|
|
451
|
-
name: 'payment_amount',
|
|
452
|
-
value: ->(e) { e.payload[:amount] },
|
|
453
|
-
buckets: [10, 50, 100, 500, 1000]
|
|
454
|
-
|
|
455
|
-
# Success rate auto-metric
|
|
456
|
-
success_rate_for pattern: 'order.*',
|
|
457
|
-
name: 'order_operations_success_rate'
|
|
458
|
-
end
|
|
459
|
-
|
|
460
|
-
# === PII FILTERING (Rails-compatible!) ===
|
|
461
|
-
config.pii_filter do
|
|
462
|
-
# AUTO: Uses Rails.application.config.filter_parameters (default: true)
|
|
463
|
-
use_rails_filter_parameters true
|
|
464
|
-
|
|
465
|
-
# SIMPLE: Add more filters (Rails-style)
|
|
466
|
-
filter_parameters :api_key, :auth_token, /secret/i
|
|
467
|
-
|
|
468
|
-
# WHITELIST: Allow specific IDs even if filtered by Rails
|
|
469
|
-
allow_parameters :user_id, :order_id, :transaction_id
|
|
470
|
-
|
|
471
|
-
# ADVANCED: Pattern-based (beyond Rails)
|
|
472
|
-
filter_pattern /\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/i,
|
|
473
|
-
replacement: '[EMAIL]'
|
|
474
|
-
filter_pattern /\b\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{4}\b/,
|
|
475
|
-
replacement: '[CARD]'
|
|
476
|
-
end
|
|
477
|
-
|
|
478
|
-
# === RATE LIMITING ===
|
|
479
|
-
config.rate_limiting do
|
|
480
|
-
global limit: 10_000, window: 1.minute
|
|
481
|
-
per_event 'user.login.failed', limit: 100, window: 1.minute
|
|
482
|
-
end
|
|
483
|
-
|
|
484
|
-
# === CONTEXT ENRICHMENT ===
|
|
485
|
-
config.context_enricher do |event|
|
|
486
|
-
{
|
|
487
|
-
trace_id: E11y::TraceId.extract,
|
|
488
|
-
user_id: Current.user&.id,
|
|
489
|
-
tenant_id: Current.tenant&.id
|
|
417
|
+
config.pipeline.use E11y::Middleware::Sampling,
|
|
418
|
+
default_sample_rate: 0.1,
|
|
419
|
+
|
|
420
|
+
# Error-spike: on error burst → 100% sampling
|
|
421
|
+
error_based_adaptive: true,
|
|
422
|
+
error_spike_config: {
|
|
423
|
+
window: 60,
|
|
424
|
+
absolute_threshold: 100,
|
|
425
|
+
relative_threshold: 3.0,
|
|
426
|
+
spike_duration: 300
|
|
427
|
+
},
|
|
428
|
+
|
|
429
|
+
# Load-based: under high load → reduced sampling
|
|
430
|
+
load_based_adaptive: true,
|
|
431
|
+
load_monitor_config: {
|
|
432
|
+
window: 60,
|
|
433
|
+
thresholds: {
|
|
434
|
+
normal: 1_000,
|
|
435
|
+
high: 10_000,
|
|
436
|
+
very_high: 50_000,
|
|
437
|
+
overload: 100_000
|
|
438
|
+
}
|
|
490
439
|
}
|
|
491
|
-
end
|
|
492
440
|
end
|
|
441
|
+
```
|
|
493
442
|
|
|
494
|
-
|
|
495
|
-
E11y.start!
|
|
443
|
+
**Value-based sampling at the event level:**
|
|
496
444
|
|
|
497
|
-
|
|
498
|
-
|
|
445
|
+
```ruby
|
|
446
|
+
class Events::Payment < E11y::Event::Base
|
|
447
|
+
sample_by_value :amount, greater_than: 1000 # large payments — always tracked
|
|
448
|
+
sample_by_value :total, in_range: 100..500
|
|
449
|
+
end
|
|
499
450
|
```
|
|
500
451
|
|
|
501
452
|
---
|
|
@@ -503,64 +454,78 @@ at_exit { E11y.stop!(timeout: 5) }
|
|
|
503
454
|
## 📊 Severity Levels
|
|
504
455
|
|
|
505
456
|
```ruby
|
|
506
|
-
|
|
507
|
-
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
error:
|
|
512
|
-
fatal:
|
|
513
|
-
|
|
457
|
+
# lib/e11y/event/base.rb
|
|
458
|
+
SEVERITIES = %i[debug info success warn error fatal].freeze
|
|
459
|
+
|
|
460
|
+
# Default sample rates by severity:
|
|
461
|
+
SEVERITY_SAMPLE_RATES = {
|
|
462
|
+
error: 1.0, # always
|
|
463
|
+
fatal: 1.0, # always
|
|
464
|
+
debug: 0.01, # 1%
|
|
465
|
+
info: 0.1, # 10%
|
|
466
|
+
success: 0.1, # 10%
|
|
467
|
+
warn: 0.1 # 10%
|
|
468
|
+
}.freeze
|
|
514
469
|
```
|
|
515
470
|
|
|
516
|
-
**When to use
|
|
471
|
+
**When to use `:success`:**
|
|
517
472
|
|
|
518
473
|
```ruby
|
|
519
|
-
#
|
|
520
|
-
Events::
|
|
521
|
-
Events::
|
|
522
|
-
Events::
|
|
474
|
+
Events::OrderPaid.track(order_id: '123') # ← :success (explicit)
|
|
475
|
+
Events::JobCompleted.track(job_id: '456') # ← :success (from name "Completed")
|
|
476
|
+
Events::UserLoggedIn.track(user_id: '123') # ← :info (default)
|
|
477
|
+
Events::PaymentFailed.track(reason: 'timeout') # ← :error (from name "Failed")
|
|
478
|
+
```
|
|
523
479
|
|
|
524
|
-
|
|
525
|
-
Events::UserLoggedIn.track(user_id: '123', severity: :info)
|
|
526
|
-
Events::SessionStarted.track(session_id: '456', severity: :info)
|
|
480
|
+
**Auto-resolved severity (convention over configuration):**
|
|
527
481
|
|
|
528
|
-
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
|
|
482
|
+
| Name contains | Severity |
|
|
483
|
+
|---|---|
|
|
484
|
+
| `Failed`, `Error` | `:error` |
|
|
485
|
+
| `Paid`, `Success`, `Completed` | `:success` |
|
|
486
|
+
| `Warn`, `Warning` | `:warn` |
|
|
487
|
+
| anything else | `:info` |
|
|
532
488
|
|
|
533
489
|
---
|
|
534
490
|
|
|
535
|
-
## 🎭 Middleware (
|
|
491
|
+
## 🎭 Middleware (configuration)
|
|
536
492
|
|
|
537
493
|
### Rails / Rack
|
|
538
494
|
|
|
539
|
-
|
|
540
|
-
# config/application.rb (auto-added by generator)
|
|
541
|
-
config.middleware.use E11y::Middleware::Rack,
|
|
542
|
-
buffer_limit: 100,
|
|
543
|
-
flush_on: :error
|
|
544
|
-
```
|
|
495
|
+
`E11y::Middleware::Request` is automatically inserted by the Railtie when `ephemeral_buffer_enabled` is true.
|
|
545
496
|
|
|
546
497
|
### Sidekiq
|
|
547
498
|
|
|
548
499
|
```ruby
|
|
549
|
-
#
|
|
500
|
+
# Automatically inserted by Railtie when Sidekiq is present.
|
|
501
|
+
# For manual setup:
|
|
550
502
|
Sidekiq.configure_server do |config|
|
|
551
503
|
config.server_middleware do |chain|
|
|
552
|
-
chain.add E11y::
|
|
504
|
+
chain.add E11y::Instruments::Sidekiq::ServerMiddleware
|
|
553
505
|
end
|
|
554
506
|
end
|
|
555
507
|
```
|
|
556
508
|
|
|
557
|
-
###
|
|
509
|
+
### Default pipeline order
|
|
510
|
+
|
|
511
|
+
```
|
|
512
|
+
TraceContext → Validation → PIIFilter → AuditSigning → Sampling → RateLimiting → Routing
|
|
513
|
+
```
|
|
514
|
+
|
|
515
|
+
As of v0.2.0, `RateLimiting` is wired into the default pipeline. To activate it, set
|
|
516
|
+
`config.rate_limiting_enabled = true` (no manual `.use` call needed).
|
|
517
|
+
|
|
518
|
+
**Versioning (opt-in):**
|
|
519
|
+
|
|
520
|
+
`Middleware::Versioning` normalizes event names from CamelCase class names to dot-notation
|
|
521
|
+
(e.g., `OrderPaidEvent` → `order.paid`). It is not in the default pipeline; add it explicitly:
|
|
558
522
|
|
|
559
523
|
```ruby
|
|
560
|
-
|
|
561
|
-
# trace_id propagated automatically to background jobs
|
|
524
|
+
config.pipeline.use E11y::Middleware::Versioning
|
|
562
525
|
```
|
|
563
526
|
|
|
527
|
+
Without this middleware, event names in adapters are the raw class name (e.g., `"OrderPaidEvent"`).
|
|
528
|
+
|
|
564
529
|
---
|
|
565
530
|
|
|
566
531
|
## 🔍 Trace Context Flow
|
|
@@ -568,40 +533,52 @@ end
|
|
|
568
533
|
```ruby
|
|
569
534
|
# Service A (API)
|
|
570
535
|
POST /orders
|
|
571
|
-
trace_id: abc-123
|
|
572
|
-
├─ Events::OrderValidation.track
|
|
573
|
-
├─ Events::OrderCreated.track
|
|
574
|
-
└─ ProcessOrderJob.perform_later(order_id, trace_id: abc-123)
|
|
536
|
+
trace_id: abc-123 # from X-Trace-ID header
|
|
537
|
+
├─ Events::OrderValidation.track # trace_id: abc-123
|
|
538
|
+
├─ Events::OrderCreated.track # trace_id: abc-123
|
|
539
|
+
└─ ProcessOrderJob.perform_later(order_id: id, trace_id: 'abc-123')
|
|
575
540
|
|
|
576
|
-
#
|
|
541
|
+
# Background Job (Sidekiq)
|
|
577
542
|
ProcessOrderJob
|
|
578
|
-
trace_id: abc-123
|
|
579
|
-
|
|
580
|
-
└─ HTTP → Service C (headers: X-Trace-ID: abc-123)
|
|
543
|
+
trace_id: abc-123 # propagated through middleware
|
|
544
|
+
└─ Events::OrderProcessed.track # trace_id: abc-123
|
|
581
545
|
|
|
582
|
-
#
|
|
583
|
-
POST /payments
|
|
584
|
-
trace_id: abc-123 (from X-Trace-ID header)
|
|
585
|
-
└─ Events::PaymentProcessed.track (trace_id: abc-123)
|
|
586
|
-
|
|
587
|
-
# Result: All events linked by trace_id = full visibility
|
|
546
|
+
# Result: all events are correlated by trace_id = full visibility
|
|
588
547
|
```
|
|
589
548
|
|
|
590
549
|
---
|
|
591
550
|
|
|
592
|
-
## 📈 Yabeda Integration
|
|
551
|
+
## 📈 Yabeda / Prometheus Integration
|
|
593
552
|
|
|
594
|
-
|
|
553
|
+
> **Note:** The `metrics do ... end` DSL requires the Yabeda adapter to be registered.
|
|
554
|
+
> Without `E11y::Adapters::Yabeda.new` in `config.adapters`, metric definitions are stored
|
|
555
|
+
> but never updated. Add the adapter as shown below before defining event metrics.
|
|
595
556
|
|
|
596
557
|
```ruby
|
|
597
|
-
#
|
|
598
|
-
|
|
558
|
+
# Gemfile
|
|
559
|
+
gem 'yabeda'
|
|
560
|
+
gem 'yabeda-prometheus'
|
|
561
|
+
|
|
562
|
+
# config/initializers/e11y.rb
|
|
563
|
+
E11y.configure do |config|
|
|
564
|
+
config.adapters[:metrics] = E11y::Adapters::Yabeda.new(
|
|
565
|
+
cardinality_limit: 1000,
|
|
566
|
+
forbidden_labels: [:user_id, :order_id], # additional denylist
|
|
567
|
+
overflow_strategy: :relabel # :drop, :alert, or :relabel
|
|
568
|
+
)
|
|
569
|
+
end
|
|
570
|
+
|
|
571
|
+
# Event with metrics:
|
|
572
|
+
class Events::OrderPaid < E11y::Event::Base
|
|
573
|
+
metrics do
|
|
574
|
+
counter :orders_paid_total, tags: [:currency]
|
|
575
|
+
histogram :order_amount, value: :amount, tags: [:currency]
|
|
576
|
+
end
|
|
577
|
+
end
|
|
599
578
|
|
|
600
|
-
|
|
601
|
-
|
|
602
|
-
|
|
603
|
-
yabeda.business_events.order_operations_success 1
|
|
604
|
-
yabeda.business_events.order_operations_total 1
|
|
579
|
+
Events::OrderPaid.track(order_id: '123', amount: 99, currency: 'USD')
|
|
580
|
+
# → Yabeda.e11y.orders_paid_total.increment({currency: 'USD'})
|
|
581
|
+
# → Yabeda.e11y.order_amount.measure({currency: 'USD'}, 99)
|
|
605
582
|
```
|
|
606
583
|
|
|
607
584
|
**Prometheus endpoint:**
|
|
@@ -611,241 +588,164 @@ yabeda.business_events.order_operations_total 1
|
|
|
611
588
|
mount Yabeda::Prometheus::Exporter => '/metrics'
|
|
612
589
|
```
|
|
613
590
|
|
|
591
|
+
|
|
614
592
|
---
|
|
615
593
|
|
|
616
594
|
## 🧪 Testing
|
|
617
595
|
|
|
618
596
|
```ruby
|
|
619
|
-
# spec/
|
|
597
|
+
# spec/support/e11y_helper.rb
|
|
620
598
|
RSpec.configure do |config|
|
|
599
|
+
let(:test_adapter) { E11y::Adapters::InMemory.new }
|
|
600
|
+
|
|
621
601
|
config.before(:each) do
|
|
602
|
+
# adapters is a Hash — use [] assignment, not Array assignment
|
|
622
603
|
E11y.configure do |c|
|
|
623
|
-
c.adapters =
|
|
604
|
+
c.adapters[:test] = test_adapter
|
|
605
|
+
# For a no-op adapter that discards all events (no recording overhead):
|
|
606
|
+
# c.adapters[:null] = E11y::Adapters::NullAdapter.new
|
|
624
607
|
end
|
|
625
608
|
end
|
|
609
|
+
|
|
610
|
+
config.after(:each) do
|
|
611
|
+
test_adapter.clear!
|
|
612
|
+
end
|
|
626
613
|
end
|
|
627
614
|
|
|
628
615
|
# spec/controllers/orders_controller_spec.rb
|
|
629
616
|
RSpec.describe OrdersController do
|
|
630
617
|
it 'tracks order creation' do
|
|
631
|
-
|
|
632
|
-
|
|
618
|
+
post :create, params: { order_id: '123', amount: 99.99, currency: 'USD' }
|
|
619
|
+
|
|
620
|
+
events = test_adapter.events
|
|
621
|
+
expect(events).to include(
|
|
622
|
+
a_hash_including(
|
|
623
|
+
event_name: 'OrderCreated',
|
|
624
|
+
payload: hash_including(order_id: '123')
|
|
625
|
+
)
|
|
633
626
|
)
|
|
634
|
-
|
|
635
|
-
|
|
627
|
+
end
|
|
628
|
+
|
|
629
|
+
it 'raises on invalid data' do
|
|
630
|
+
expect {
|
|
631
|
+
Events::OrderPaid.track(order_id: nil, amount: -1)
|
|
632
|
+
}.to raise_error(E11y::ValidationError)
|
|
636
633
|
end
|
|
637
634
|
end
|
|
638
635
|
```
|
|
639
636
|
|
|
640
|
-
|
|
637
|
+
**InMemory Adapter API:**
|
|
641
638
|
|
|
642
|
-
|
|
643
|
-
|
|
644
|
-
| Metric | Target | Actual |
|
|
645
|
-
|--------|--------|--------|
|
|
646
|
-
| **Track latency (p99)** | <1ms | ✅ 0.8ms |
|
|
647
|
-
| **Throughput** | 10k events/sec | ✅ 15k/sec |
|
|
648
|
-
| **Memory** | <100MB @ 100k buffer | ✅ 80MB |
|
|
649
|
-
| **CPU overhead** | <5% @ 1k events/sec | ✅ 3% |
|
|
639
|
+
```ruby
|
|
640
|
+
adapter = E11y::Adapters::InMemory.new(max_events: 1000)
|
|
650
641
|
|
|
651
|
-
|
|
652
|
-
|
|
653
|
-
|
|
654
|
-
|
|
655
|
-
|
|
642
|
+
adapter.events # => Array<Hash> — all events
|
|
643
|
+
adapter.event_count # => Integer
|
|
644
|
+
adapter.last_event # => Hash — last event
|
|
645
|
+
adapter.clear! # reset
|
|
646
|
+
```
|
|
656
647
|
|
|
657
648
|
---
|
|
658
649
|
|
|
659
650
|
## 🔐 Security
|
|
660
651
|
|
|
661
|
-
### PII Filtering
|
|
652
|
+
### PII Filtering — what works today
|
|
662
653
|
|
|
663
|
-
**
|
|
654
|
+
**Auto (:rails_filters) — Rails filter_parameters:**
|
|
664
655
|
|
|
665
656
|
```ruby
|
|
666
657
|
# config/application.rb
|
|
667
|
-
config.filter_parameters += [:password, :email, :ssn]
|
|
668
|
-
|
|
669
|
-
# E11y автоматически фильтрует эти поля - NO ADDITIONAL CONFIG!
|
|
670
|
-
Events::UserRegistered.track(
|
|
671
|
-
email: 'user@example.com', # → '[FILTERED]'
|
|
672
|
-
password: 'secret123' # → '[FILTERED]'
|
|
673
|
-
)
|
|
658
|
+
config.filter_parameters += [:password, :email, :ssn, :credit_card]
|
|
659
|
+
# E11y applies this list automatically — no extra config needed
|
|
674
660
|
```
|
|
675
661
|
|
|
676
|
-
**
|
|
662
|
+
**Event-level (:explicit_pii):**
|
|
677
663
|
|
|
678
664
|
```ruby
|
|
679
|
-
|
|
680
|
-
|
|
681
|
-
|
|
682
|
-
|
|
683
|
-
|
|
684
|
-
|
|
685
|
-
|
|
686
|
-
|
|
687
|
-
|
|
688
|
-
# Pattern-based (content filtering)
|
|
689
|
-
filter_pattern /\b\d{16}\b/, replacement: '[CARD]'
|
|
665
|
+
class Events::UserRegistered < E11y::Event::Base
|
|
666
|
+
contains_pii true
|
|
667
|
+
|
|
668
|
+
pii_filtering do
|
|
669
|
+
masks :password
|
|
670
|
+
hashes :email # SHA256, preserves searchability
|
|
671
|
+
redacts :ssn
|
|
672
|
+
allows :user_id
|
|
690
673
|
end
|
|
691
674
|
end
|
|
692
675
|
```
|
|
693
676
|
|
|
694
|
-
|
|
677
|
+
### Rate Limiting — now in default pipeline
|
|
695
678
|
|
|
696
|
-
|
|
679
|
+
As of v0.2.0, `Middleware::RateLimiting` is included in the default pipeline. Activate it by
|
|
680
|
+
enabling the config (no extra `.use` call required):
|
|
697
681
|
|
|
698
682
|
```ruby
|
|
699
|
-
|
|
700
|
-
config.
|
|
701
|
-
|
|
702
|
-
|
|
683
|
+
E11y.configure do |config|
|
|
684
|
+
config.rate_limiting_enabled = true
|
|
685
|
+
config.rate_limiting_global_limit = 10_000 # events/sec
|
|
686
|
+
config.rate_limiting_per_event_limit = 1_000 # events/sec per type
|
|
687
|
+
config.rate_limiting_global_window = 1.0 # seconds
|
|
703
688
|
end
|
|
704
|
-
|
|
705
|
-
# При превышении - события дропаются
|
|
706
689
|
```
|
|
707
690
|
|
|
708
|
-
|
|
691
|
+
> **Note:** When `config.rate_limiting_enabled = false` (default), the middleware is present in
|
|
692
|
+
> the pipeline but passes all events through without limiting. Set `enabled = true` to activate.
|
|
709
693
|
|
|
710
|
-
|
|
694
|
+
> 🚧 **Roadmap:** Per-event and per-pattern rate limiting (e.g. `'user.login.failed'` → 100/min)
|
|
695
|
+
> — planned for future releases.
|
|
711
696
|
|
|
712
|
-
|
|
697
|
+
---
|
|
713
698
|
|
|
714
|
-
|
|
715
|
-
# config/initializers/e11y.rb
|
|
716
|
-
E11y.configure do |config|
|
|
717
|
-
config.slo_tracking = true # ← ВСЁ! Этого достаточно!
|
|
718
|
-
|
|
719
|
-
# Опционально: кастомизация
|
|
720
|
-
config.slo do
|
|
721
|
-
# Global defaults
|
|
722
|
-
http_ignore_statuses [404, 401] # 404/401 не ошибки
|
|
723
|
-
latency_target_p95 200 # ms
|
|
724
|
-
|
|
725
|
-
# 🎯 NEW: Per-controller overrides (РЕКОМЕНДУЕТСЯ для Rails!)
|
|
726
|
-
controller 'Api::Admin::BaseController' do
|
|
727
|
-
ignore true # Весь admin не входит в SLO
|
|
728
|
-
end
|
|
729
|
-
|
|
730
|
-
controller 'Api::OrdersController', action: 'show' do
|
|
731
|
-
latency_target_p95 50 # Show должен быть быстрым
|
|
732
|
-
end
|
|
733
|
-
|
|
734
|
-
controller 'Api::OrdersController', action: 'create' do
|
|
735
|
-
latency_target_p95 200 # Create может быть медленнее
|
|
736
|
-
ignore_statuses [422] # 422 Validation = not SLO breach
|
|
737
|
-
end
|
|
738
|
-
|
|
739
|
-
# 🔧 LEGACY: Path-based (для non-Rails apps)
|
|
740
|
-
endpoint '/api/webhooks/*' do
|
|
741
|
-
ignore true
|
|
742
|
-
end
|
|
743
|
-
|
|
744
|
-
# 🎯 Per-job overrides
|
|
745
|
-
job 'ReportGenerationJob' do
|
|
746
|
-
ignore true # Долгие джобы не входят в SLO
|
|
747
|
-
end
|
|
748
|
-
|
|
749
|
-
job 'ProcessPaymentJob' do
|
|
750
|
-
latency_target_p95 1000 # Критичные джобы = строгий SLO
|
|
751
|
-
end
|
|
752
|
-
end
|
|
753
|
-
end
|
|
754
|
-
```
|
|
699
|
+
## 🎯 Built-in SLO Tracking
|
|
755
700
|
|
|
756
|
-
|
|
701
|
+
**What is tracked automatically** when `config.rails_instrumentation_enabled = true`:
|
|
757
702
|
|
|
758
703
|
```ruby
|
|
759
|
-
#
|
|
760
|
-
|
|
761
|
-
status="200",
|
|
762
|
-
|
|
763
|
-
|
|
764
|
-
|
|
765
|
-
} 1234
|
|
766
|
-
|
|
767
|
-
yabeda_slo_http_request_duration_seconds{
|
|
768
|
-
method="GET",
|
|
769
|
-
controller="Api::OrdersController",
|
|
770
|
-
action="show"
|
|
771
|
-
} histogram
|
|
704
|
+
# HTTP Metrics (via Rack middleware)
|
|
705
|
+
e11y_http_requests_total{
|
|
706
|
+
status="200", method="GET",
|
|
707
|
+
controller="Api::OrdersController", action="show"
|
|
708
|
+
}
|
|
709
|
+
e11y_http_request_duration_seconds{ ... } # histogram
|
|
772
710
|
|
|
773
711
|
# Sidekiq Metrics
|
|
774
|
-
|
|
775
|
-
|
|
712
|
+
e11y_sidekiq_jobs_total{queue="default", class="ProcessOrderJob", status="success"}
|
|
713
|
+
e11y_sidekiq_job_duration_seconds{ ... }
|
|
776
714
|
|
|
777
715
|
# ActiveJob Metrics
|
|
778
|
-
|
|
779
|
-
|
|
716
|
+
e11y_active_jobs_total{queue="mailers", class="EmailJob", status="success"}
|
|
717
|
+
e11y_active_job_duration_seconds{ ... }
|
|
780
718
|
```
|
|
781
719
|
|
|
782
|
-
**Преимущество:** `/orders/123`, `/orders/456` → один `OrdersController#show` (не нужна нормализация path!)
|
|
783
|
-
|
|
784
720
|
**SLO Calculations (PromQL):**
|
|
785
721
|
|
|
786
722
|
```promql
|
|
787
723
|
# HTTP Availability (30d rolling)
|
|
788
|
-
100 * (
|
|
789
|
-
|
|
790
|
-
|
|
724
|
+
100 * (
|
|
725
|
+
sum(rate(e11y_http_requests_total{status=~"2.."}[30d])) /
|
|
726
|
+
sum(rate(e11y_http_requests_total[30d]))
|
|
727
|
+
)
|
|
791
728
|
|
|
792
729
|
# p95 Latency
|
|
793
|
-
histogram_quantile(0.95, rate(
|
|
794
|
-
# Expected: < 200ms
|
|
795
|
-
|
|
796
|
-
# Error Budget Remaining
|
|
797
|
-
100 * (1 - (sum(rate(yabeda_slo_http_errors_total[30d])) /
|
|
798
|
-
(sum(rate(yabeda_slo_http_successes_total[30d])) + sum(rate(yabeda_slo_http_errors_total[30d])))) / 0.001)
|
|
799
|
-
# 100% = весь бюджет остался, 0% = исчерпан
|
|
800
|
-
```
|
|
801
|
-
|
|
802
|
-
**Auto-Generated:**
|
|
803
|
-
|
|
804
|
-
```bash
|
|
805
|
-
# Grafana dashboard
|
|
806
|
-
rails g e11y:grafana_dashboard
|
|
807
|
-
# → config/grafana/e11y_slo_dashboard.json
|
|
808
|
-
|
|
809
|
-
# Prometheus alerts
|
|
810
|
-
rails g e11y:prometheus_alerts
|
|
811
|
-
# → config/prometheus/e11y_slo_alerts.yml
|
|
730
|
+
histogram_quantile(0.95, rate(e11y_http_request_duration_seconds_bucket[5m]))
|
|
812
731
|
```
|
|
813
732
|
|
|
814
|
-
|
|
815
|
-
- ✅ **Rack middleware** - все HTTP requests (availability, latency)
|
|
816
|
-
- ✅ **Sidekiq middleware** - все jobs (success rate, duration)
|
|
817
|
-
- ✅ **ActiveJob instrumentation** - все jobs (success rate, duration)
|
|
818
|
-
- ✅ **Path normalization** - `/orders/123` → `/orders/:id`
|
|
819
|
-
- ✅ **Error categorization** - configurable (5xx = error, 404 = ignore)
|
|
820
|
-
- ✅ **Heartbeat** - auto-enabled (pod liveness detection)
|
|
733
|
+
**⚠️ In-process SLO limitations:**
|
|
821
734
|
|
|
822
|
-
|
|
735
|
+
E11y SLO runs inside the Ruby process and **does not see**:
|
|
736
|
+
- Network issues (requests that never reach the app)
|
|
737
|
+
- Load balancer failures
|
|
738
|
+
- All pods down
|
|
823
739
|
|
|
824
|
-
E11y SLO
|
|
825
|
-
- ❌ Network issues (requests не доходят до app)
|
|
826
|
-
- ❌ Load balancer down
|
|
827
|
-
- ❌ All pods crashed (метрик просто нет)
|
|
828
|
-
- ❌ DNS issues
|
|
740
|
+
Recommended: multi-layer monitoring — E11y SLO + K8s health probes + external synthetic checks.
|
|
829
741
|
|
|
830
|
-
|
|
831
|
-
|
|
832
|
-
|
|
833
|
-
3. **Layer 3**: K8s health probes (`/health/live`, `/health/ready`) - **auto-created!**
|
|
834
|
-
4. **Layer 4**: External synthetic monitoring (Prometheus Blackbox Exporter)
|
|
835
|
-
|
|
836
|
-
```ruby
|
|
837
|
-
# Heartbeat автоматически включен с slo_tracking = true
|
|
838
|
-
# Метрики:
|
|
839
|
-
yabeda_e11y_heartbeat_timestamp{pod="pod-1"} 1703500000 # Последний heartbeat
|
|
840
|
-
yabeda_e11y_service_healthy{pod="pod-1"} 1 # 1 = healthy
|
|
841
|
-
|
|
842
|
-
# Alert если pod мертв:
|
|
843
|
-
# (time() - yabeda_e11y_heartbeat_timestamp) > 30s → Pod down!
|
|
844
|
-
```
|
|
742
|
+
> 🚧 **Roadmap:** Per-controller/per-job SLO configuration, auto-generated Grafana dashboards
|
|
743
|
+
> (`rails g e11y:grafana_dashboard`) and Prometheus alerts (`rails g e11y:prometheus_alerts`)
|
|
744
|
+
> — planned for future releases.
|
|
845
745
|
|
|
846
746
|
---
|
|
847
747
|
|
|
848
|
-
##
|
|
748
|
+
## 🔄 Migration from Rails.logger
|
|
849
749
|
|
|
850
750
|
```ruby
|
|
851
751
|
# ❌ Before
|
|
@@ -853,76 +753,135 @@ Rails.logger.info "Order #{order.id} paid #{order.amount} #{order.currency}"
|
|
|
853
753
|
OrderMetrics.increment('orders.paid.total')
|
|
854
754
|
OrderMetrics.observe('orders.paid.amount', order.amount)
|
|
855
755
|
|
|
856
|
-
# ✅ After (1
|
|
756
|
+
# ✅ After (1 line instead of 3, + validation + trace context)
|
|
857
757
|
Events::OrderPaid.track(
|
|
858
758
|
order_id: order.id,
|
|
859
|
-
amount:
|
|
759
|
+
amount: order.amount,
|
|
860
760
|
currency: order.currency
|
|
861
761
|
)
|
|
862
|
-
# → Structured log + auto-metrics + trace context
|
|
863
762
|
```
|
|
864
763
|
|
|
865
764
|
---
|
|
866
765
|
|
|
867
766
|
## 🐛 Troubleshooting
|
|
868
767
|
|
|
869
|
-
### Events not appearing?
|
|
768
|
+
### Events not appearing in the adapter?
|
|
870
769
|
|
|
871
770
|
```ruby
|
|
872
|
-
#
|
|
873
|
-
E11y.
|
|
771
|
+
# 1. Is E11y enabled?
|
|
772
|
+
E11y.config.enabled # => true
|
|
773
|
+
|
|
774
|
+
# 2. Is the adapter registered?
|
|
775
|
+
E11y.config.adapters # => {:logs=>#<Loki...>, ...} — must not be empty
|
|
776
|
+
|
|
777
|
+
# 3. Is severity routing configured?
|
|
778
|
+
E11y.config.adapter_mapping
|
|
779
|
+
# => {:error=>[:logs, :errors_tracker], :fatal=>[:logs, :errors_tracker], :default=>[:logs]}
|
|
874
780
|
|
|
875
|
-
#
|
|
876
|
-
E11y.config.adapters
|
|
781
|
+
# 4. Is the adapter healthy?
|
|
782
|
+
E11y.config.adapters[:logs].healthy? # => true
|
|
877
783
|
|
|
878
|
-
#
|
|
879
|
-
E11y.
|
|
784
|
+
# 5. Metrics not updating? Yabeda adapter must be explicitly configured:
|
|
785
|
+
E11y.config.adapters[:metrics] # should be E11y::Adapters::Yabeda
|
|
880
786
|
|
|
881
|
-
#
|
|
882
|
-
E11y.
|
|
787
|
+
# 6. Diagnostic helpers (v0.2.0):
|
|
788
|
+
E11y.enabled_for?(:debug) # => true/false — is debug severity active?
|
|
789
|
+
E11y.buffer_size # => Integer — current debug buffer size
|
|
790
|
+
E11y.circuit_breaker_state # => :closed/:open/:half_open
|
|
883
791
|
```
|
|
884
792
|
|
|
885
|
-
###
|
|
793
|
+
### Debug events not flushing on errors?
|
|
886
794
|
|
|
887
795
|
```ruby
|
|
888
|
-
#
|
|
889
|
-
|
|
796
|
+
# Is the buffer enabled?
|
|
797
|
+
E11y.config.ephemeral_buffer_enabled # => true
|
|
798
|
+
|
|
799
|
+
# Is Rails instrumentation enabled?
|
|
800
|
+
E11y.config.rails_instrumentation_enabled # => true
|
|
801
|
+
|
|
802
|
+
# Is 5xx auto-flush enabled?
|
|
803
|
+
E11y.config.ephemeral_buffer_flush_on_error # => true (default)
|
|
804
|
+
|
|
805
|
+
# Any extra statuses configured?
|
|
806
|
+
E11y.config.ephemeral_buffer_flush_on_statuses # => [] (default) or [403] etc.
|
|
807
|
+
|
|
808
|
+
# Note: flush_on_error and flush_on_statuses are independent.
|
|
809
|
+
# flush_on_error=true → flush on any 5xx.
|
|
810
|
+
# flush_on_statuses=[403] → also flush on 403, regardless of flush_on_error.
|
|
811
|
+
```
|
|
890
812
|
|
|
813
|
+
### High latency?
|
|
814
|
+
|
|
815
|
+
```ruby
|
|
891
816
|
# Possible causes:
|
|
892
|
-
# -
|
|
893
|
-
# -
|
|
894
|
-
# -
|
|
817
|
+
# - Loki/Sentry unreachable → circuit breaker will open after 5 errors
|
|
818
|
+
# - PII filtering: complex regexes → simplify
|
|
819
|
+
# - batch_size too large → reduce or lower batch_timeout
|
|
820
|
+
|
|
821
|
+
# Check adapter health:
|
|
822
|
+
E11y.config.adapters[:logs].healthy?
|
|
895
823
|
```
|
|
896
824
|
|
|
897
825
|
---
|
|
898
826
|
|
|
899
827
|
## 📚 Full Documentation
|
|
900
828
|
|
|
901
|
-
- **
|
|
902
|
-
- **
|
|
903
|
-
- **GitHub**: https://github.com/yourorg/e11y
|
|
829
|
+
- **ADRs**: `docs/architecture/ADR-*.md` — architecture decision records
|
|
830
|
+
- **GitHub**: https://github.com/arturseletskiy/e11y
|
|
904
831
|
|
|
905
832
|
---
|
|
906
833
|
|
|
907
|
-
## ✅ Checklist
|
|
908
|
-
|
|
909
|
-
- [ ]
|
|
910
|
-
- [ ] Run
|
|
911
|
-
- [ ] Configure adapters (
|
|
912
|
-
- [ ]
|
|
913
|
-
- [ ]
|
|
914
|
-
- [ ]
|
|
915
|
-
- [ ]
|
|
916
|
-
- [ ]
|
|
917
|
-
- [ ]
|
|
918
|
-
- [ ]
|
|
919
|
-
- [ ]
|
|
920
|
-
- [ ]
|
|
921
|
-
- [ ]
|
|
922
|
-
- [ ] **Generate Grafana dashboard**: `rails g e11y:grafana_dashboard`
|
|
923
|
-
- [ ] **Generate Prometheus alerts**: `rails g e11y:prometheus_alerts`
|
|
924
|
-
- [ ] Deploy to staging
|
|
925
|
-
- [ ] Monitor performance
|
|
926
|
-
- [ ] Rollout to production (canary 1% → 10% → 100%)
|
|
834
|
+
## ✅ Getting Started Checklist
|
|
835
|
+
|
|
836
|
+
- [ ] Add `gem 'e11y', '~> 0.2'` to Gemfile, run `bundle install`
|
|
837
|
+
- [ ] Run `rails g e11y:install` (or create `config/initializers/e11y.rb` manually)
|
|
838
|
+
- [ ] Configure adapters: `config.adapters[:logs] = E11y::Adapters::Loki.new(...)`
|
|
839
|
+
- [ ] For metrics: add `gem 'yabeda'`, set `config.adapters[:metrics] = E11y::Adapters::Yabeda.new`
|
|
840
|
+
- [ ] Enable request buffering: `config.ephemeral_buffer_enabled = true`
|
|
841
|
+
- [ ] Enable Rails instrumentation: `config.rails_instrumentation_enabled = true`
|
|
842
|
+
- [ ] Define first event class in `app/events/`
|
|
843
|
+
- [ ] Use `EventName.track(...)` in a controller or service
|
|
844
|
+
- [ ] Write a test using `E11y::Adapters::InMemory`
|
|
845
|
+
- [ ] Check `/metrics` endpoint (if Yabeda is configured)
|
|
846
|
+
- [ ] Test request buffering: raise an exception → confirm debug events appeared
|
|
847
|
+
- [ ] Configure PII filtering for events that handle personal data
|
|
848
|
+
- [ ] Deploy to staging, monitor performance
|
|
927
849
|
|
|
850
|
+
---
|
|
928
851
|
|
|
852
|
+
## ✅ What's New in v0.2.0
|
|
853
|
+
|
|
854
|
+
| Feature | Notes |
|
|
855
|
+
|---------|-------|
|
|
856
|
+
| `rails g e11y:install` | Generator available: creates initializer + `app/events/` |
|
|
857
|
+
| `E11y.start!` / `E11y.stop!` | Lifecycle methods for graceful startup/shutdown |
|
|
858
|
+
| Rate Limiting in default pipeline | `config.rate_limiting_enabled = true` now works |
|
|
859
|
+
| Event name normalization (`Middleware::Versioning`) | Now in default pipeline |
|
|
860
|
+
| OTelLogs payload attributes | All payload attributes now included in OTel log records |
|
|
861
|
+
| `config.slo_tracking = true` | Boolean coercion now accepted |
|
|
862
|
+
| `retention` / `retention_period` | Both work as aliases on event class |
|
|
863
|
+
| `add_slo_controller` / `add_slo_job` | Helpers on `E11y::Configuration` (stored config; see UC-004) |
|
|
864
|
+
| `config.rate_limiting do` | Rate limiting block DSL |
|
|
865
|
+
| `config.cardinality_protection do` | Cardinality DSL block |
|
|
866
|
+
| `config.register_adapter` | Alias for `config.adapters[name] =` |
|
|
867
|
+
| `NullAdapter` | `E11y::Adapters::NullAdapter.new` for no-op testing |
|
|
868
|
+
| `track() { }` block form | Block form measures duration automatically |
|
|
869
|
+
| `E11y.enabled_for?` / `E11y.buffer_size` | Diagnostic helpers |
|
|
870
|
+
| `metric :counter` single-call DSL | Single metric definition without `metrics do` block |
|
|
871
|
+
| Full block DSLs | All config sections support `do...end` block form |
|
|
872
|
+
|
|
873
|
+
## 🚧 Roadmap (still not implemented)
|
|
874
|
+
|
|
875
|
+
The following features are **documented in ADRs** but not yet implemented:
|
|
876
|
+
|
|
877
|
+
| Feature | ADR/UC |
|
|
878
|
+
|---------|--------|
|
|
879
|
+
| `rails g e11y:grafana_dashboard` | ADR-003 |
|
|
880
|
+
| `rails g e11y:prometheus_alerts` | ADR-003 |
|
|
881
|
+
| Wire `add_slo_controller` / `add_slo_job` into HTTP/job `Tracker` dimensions | UC-004, ADR-003 |
|
|
882
|
+
| Per-event rate limiting (`rate_limit` DSL on event class) | UC-011 |
|
|
883
|
+
| Tiered storage (archival) | UC-019 — filter by `retention_until` |
|
|
884
|
+
| Cost Tracking / Budget Enforcement | ADR-009, UC-015 |
|
|
885
|
+
| Outgoing HTTP trace propagation (Faraday/Net::HTTP) | UC-009 |
|
|
886
|
+
| Event Registry (`E11y::Registry`) | UC-022 |
|
|
887
|
+
| Key Rotation for AuditEncrypted | ADR-006 |
|