e11y 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.rspec +4 -0
- data/.rubocop.yml +69 -0
- data/CHANGELOG.md +26 -0
- data/CODE_OF_CONDUCT.md +64 -0
- data/LICENSE.txt +21 -0
- data/README.md +179 -0
- data/Rakefile +37 -0
- data/benchmarks/run_all.rb +33 -0
- data/config/README.md +83 -0
- data/config/loki-local-config.yaml +35 -0
- data/config/prometheus.yml +15 -0
- data/docker-compose.yml +78 -0
- data/docs/00-ICP-AND-TIMELINE.md +483 -0
- data/docs/01-SCALE-REQUIREMENTS.md +858 -0
- data/docs/ADR-001-architecture.md +2617 -0
- data/docs/ADR-002-metrics-yabeda.md +1395 -0
- data/docs/ADR-003-slo-observability.md +3337 -0
- data/docs/ADR-004-adapter-architecture.md +2385 -0
- data/docs/ADR-005-tracing-context.md +1372 -0
- data/docs/ADR-006-security-compliance.md +4143 -0
- data/docs/ADR-007-opentelemetry-integration.md +1385 -0
- data/docs/ADR-008-rails-integration.md +1911 -0
- data/docs/ADR-009-cost-optimization.md +2993 -0
- data/docs/ADR-010-developer-experience.md +2166 -0
- data/docs/ADR-011-testing-strategy.md +1836 -0
- data/docs/ADR-012-event-evolution.md +958 -0
- data/docs/ADR-013-reliability-error-handling.md +2750 -0
- data/docs/ADR-014-event-driven-slo.md +1533 -0
- data/docs/ADR-015-middleware-order.md +1061 -0
- data/docs/ADR-016-self-monitoring-slo.md +1234 -0
- data/docs/API-REFERENCE-L28.md +914 -0
- data/docs/COMPREHENSIVE-CONFIGURATION.md +2366 -0
- data/docs/IMPLEMENTATION_NOTES.md +2804 -0
- data/docs/IMPLEMENTATION_PLAN.md +1971 -0
- data/docs/IMPLEMENTATION_PLAN_ARCHITECTURE.md +586 -0
- data/docs/PLAN.md +148 -0
- data/docs/QUICK-START.md +934 -0
- data/docs/README.md +296 -0
- data/docs/design/00-memory-optimization.md +593 -0
- data/docs/guides/MIGRATION-L27-L28.md +692 -0
- data/docs/guides/PERFORMANCE-BENCHMARKS.md +434 -0
- data/docs/guides/README.md +44 -0
- data/docs/prd/01-overview-vision.md +440 -0
- data/docs/use_cases/README.md +119 -0
- data/docs/use_cases/UC-001-request-scoped-debug-buffering.md +813 -0
- data/docs/use_cases/UC-002-business-event-tracking.md +1953 -0
- data/docs/use_cases/UC-003-pattern-based-metrics.md +1627 -0
- data/docs/use_cases/UC-004-zero-config-slo-tracking.md +728 -0
- data/docs/use_cases/UC-005-sentry-integration.md +759 -0
- data/docs/use_cases/UC-006-trace-context-management.md +905 -0
- data/docs/use_cases/UC-007-pii-filtering.md +2648 -0
- data/docs/use_cases/UC-008-opentelemetry-integration.md +1153 -0
- data/docs/use_cases/UC-009-multi-service-tracing.md +1043 -0
- data/docs/use_cases/UC-010-background-job-tracking.md +1018 -0
- data/docs/use_cases/UC-011-rate-limiting.md +1906 -0
- data/docs/use_cases/UC-012-audit-trail.md +2301 -0
- data/docs/use_cases/UC-013-high-cardinality-protection.md +2127 -0
- data/docs/use_cases/UC-014-adaptive-sampling.md +1940 -0
- data/docs/use_cases/UC-015-cost-optimization.md +735 -0
- data/docs/use_cases/UC-016-rails-logger-migration.md +785 -0
- data/docs/use_cases/UC-017-local-development.md +867 -0
- data/docs/use_cases/UC-018-testing-events.md +1081 -0
- data/docs/use_cases/UC-019-tiered-storage-migration.md +562 -0
- data/docs/use_cases/UC-020-event-versioning.md +708 -0
- data/docs/use_cases/UC-021-error-handling-retry-dlq.md +956 -0
- data/docs/use_cases/UC-022-event-registry.md +648 -0
- data/docs/use_cases/backlog.md +226 -0
- data/e11y.gemspec +76 -0
- data/lib/e11y/adapters/adaptive_batcher.rb +207 -0
- data/lib/e11y/adapters/audit_encrypted.rb +239 -0
- data/lib/e11y/adapters/base.rb +580 -0
- data/lib/e11y/adapters/file.rb +224 -0
- data/lib/e11y/adapters/in_memory.rb +216 -0
- data/lib/e11y/adapters/loki.rb +333 -0
- data/lib/e11y/adapters/otel_logs.rb +203 -0
- data/lib/e11y/adapters/registry.rb +141 -0
- data/lib/e11y/adapters/sentry.rb +230 -0
- data/lib/e11y/adapters/stdout.rb +108 -0
- data/lib/e11y/adapters/yabeda.rb +370 -0
- data/lib/e11y/buffers/adaptive_buffer.rb +339 -0
- data/lib/e11y/buffers/base_buffer.rb +40 -0
- data/lib/e11y/buffers/request_scoped_buffer.rb +246 -0
- data/lib/e11y/buffers/ring_buffer.rb +267 -0
- data/lib/e11y/buffers.rb +14 -0
- data/lib/e11y/console.rb +122 -0
- data/lib/e11y/current.rb +48 -0
- data/lib/e11y/event/base.rb +894 -0
- data/lib/e11y/event/value_sampling_config.rb +84 -0
- data/lib/e11y/events/base_audit_event.rb +43 -0
- data/lib/e11y/events/base_payment_event.rb +33 -0
- data/lib/e11y/events/rails/cache/delete.rb +21 -0
- data/lib/e11y/events/rails/cache/read.rb +23 -0
- data/lib/e11y/events/rails/cache/write.rb +22 -0
- data/lib/e11y/events/rails/database/query.rb +45 -0
- data/lib/e11y/events/rails/http/redirect.rb +21 -0
- data/lib/e11y/events/rails/http/request.rb +26 -0
- data/lib/e11y/events/rails/http/send_file.rb +21 -0
- data/lib/e11y/events/rails/http/start_processing.rb +26 -0
- data/lib/e11y/events/rails/job/completed.rb +22 -0
- data/lib/e11y/events/rails/job/enqueued.rb +22 -0
- data/lib/e11y/events/rails/job/failed.rb +22 -0
- data/lib/e11y/events/rails/job/scheduled.rb +23 -0
- data/lib/e11y/events/rails/job/started.rb +22 -0
- data/lib/e11y/events/rails/log.rb +56 -0
- data/lib/e11y/events/rails/view/render.rb +23 -0
- data/lib/e11y/events.rb +18 -0
- data/lib/e11y/instruments/active_job.rb +201 -0
- data/lib/e11y/instruments/rails_instrumentation.rb +141 -0
- data/lib/e11y/instruments/sidekiq.rb +175 -0
- data/lib/e11y/logger/bridge.rb +205 -0
- data/lib/e11y/metrics/cardinality_protection.rb +172 -0
- data/lib/e11y/metrics/cardinality_tracker.rb +134 -0
- data/lib/e11y/metrics/registry.rb +234 -0
- data/lib/e11y/metrics/relabeling.rb +226 -0
- data/lib/e11y/metrics.rb +102 -0
- data/lib/e11y/middleware/audit_signing.rb +174 -0
- data/lib/e11y/middleware/base.rb +140 -0
- data/lib/e11y/middleware/event_slo.rb +167 -0
- data/lib/e11y/middleware/pii_filter.rb +266 -0
- data/lib/e11y/middleware/pii_filtering.rb +280 -0
- data/lib/e11y/middleware/rate_limiting.rb +214 -0
- data/lib/e11y/middleware/request.rb +163 -0
- data/lib/e11y/middleware/routing.rb +157 -0
- data/lib/e11y/middleware/sampling.rb +254 -0
- data/lib/e11y/middleware/slo.rb +168 -0
- data/lib/e11y/middleware/trace_context.rb +131 -0
- data/lib/e11y/middleware/validation.rb +118 -0
- data/lib/e11y/middleware/versioning.rb +132 -0
- data/lib/e11y/middleware.rb +12 -0
- data/lib/e11y/pii/patterns.rb +90 -0
- data/lib/e11y/pii.rb +13 -0
- data/lib/e11y/pipeline/builder.rb +155 -0
- data/lib/e11y/pipeline/zone_validator.rb +110 -0
- data/lib/e11y/pipeline.rb +12 -0
- data/lib/e11y/presets/audit_event.rb +65 -0
- data/lib/e11y/presets/debug_event.rb +34 -0
- data/lib/e11y/presets/high_value_event.rb +51 -0
- data/lib/e11y/presets.rb +19 -0
- data/lib/e11y/railtie.rb +138 -0
- data/lib/e11y/reliability/circuit_breaker.rb +216 -0
- data/lib/e11y/reliability/dlq/file_storage.rb +277 -0
- data/lib/e11y/reliability/dlq/filter.rb +117 -0
- data/lib/e11y/reliability/retry_handler.rb +207 -0
- data/lib/e11y/reliability/retry_rate_limiter.rb +117 -0
- data/lib/e11y/sampling/error_spike_detector.rb +225 -0
- data/lib/e11y/sampling/load_monitor.rb +161 -0
- data/lib/e11y/sampling/stratified_tracker.rb +92 -0
- data/lib/e11y/sampling/value_extractor.rb +82 -0
- data/lib/e11y/self_monitoring/buffer_monitor.rb +79 -0
- data/lib/e11y/self_monitoring/performance_monitor.rb +97 -0
- data/lib/e11y/self_monitoring/reliability_monitor.rb +146 -0
- data/lib/e11y/slo/event_driven.rb +150 -0
- data/lib/e11y/slo/tracker.rb +119 -0
- data/lib/e11y/version.rb +9 -0
- data/lib/e11y.rb +283 -0
- metadata +452 -0
data/docs/PLAN.md
ADDED
|
@@ -0,0 +1,148 @@
|
|
|
1
|
+
# ✅ TeamTab Plan Successfully Created!
|
|
2
|
+
|
|
3
|
+
**Date:** 2026-01-17
|
|
4
|
+
**Status:** Complete (100%)
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## 📊 CREATED PLANS SUMMARY
|
|
9
|
+
|
|
10
|
+
### **7 Root Tasks (Phases):**
|
|
11
|
+
|
|
12
|
+
1. **FEAT-4711:** PHASE 0: Gem Setup & Best Practices (Week -1)
|
|
13
|
+
- 3 L3 components + 20 L4 subtasks
|
|
14
|
+
- Root task (created separately)
|
|
15
|
+
|
|
16
|
+
2. **FEAT-4738:** PHASE 1: Foundation (Weeks 1-4)
|
|
17
|
+
- 6 L3 components + 19 L4 subtasks
|
|
18
|
+
- Root task (created separately)
|
|
19
|
+
|
|
20
|
+
3. **FEAT-4770:** PHASE 2: Core Features (Weeks 5-10)
|
|
21
|
+
- 4 L3 components + 12 L4 subtasks
|
|
22
|
+
- Root task (created separately)
|
|
23
|
+
|
|
24
|
+
4. **FEAT-4837:** PHASE 2.8: Advanced Sampling Strategies ⚡ *NEW*
|
|
25
|
+
- 5 L3 components + 14 L4 subtasks
|
|
26
|
+
- Root task (created 2026-01-20)
|
|
27
|
+
- **Deferred from L2.7** - Advanced adaptive sampling (error-based, load-based, value-based, stratified)
|
|
28
|
+
|
|
29
|
+
5. **FEAT-4792:** PHASE 3: Rails Integration (Weeks 11-14)
|
|
30
|
+
- 3 L3 components + 6 L4 subtasks
|
|
31
|
+
- Root task (created separately)
|
|
32
|
+
|
|
33
|
+
6. **FEAT-4805:** PHASE 4: Production Hardening (Weeks 15-20)
|
|
34
|
+
- 4 L3 components + 8 L4 subtasks
|
|
35
|
+
- Root task (created separately)
|
|
36
|
+
|
|
37
|
+
7. **FEAT-4822:** PHASE 5: Scale & Optimization (Weeks 21-26)
|
|
38
|
+
- 4 L3 components + 8 L4 subtasks
|
|
39
|
+
- Root task (created separately)
|
|
40
|
+
|
|
41
|
+
---
|
|
42
|
+
|
|
43
|
+
## 📈 STATISTICS
|
|
44
|
+
|
|
45
|
+
**Total Created:**
|
|
46
|
+
- **7 root tasks** (phases) - *+1 added 2026-01-20*
|
|
47
|
+
- **29 L3 components** (main subtasks) - *+5 from Phase 2.8*
|
|
48
|
+
- **87 L4 subtasks** (detailed tasks) - *+14 from Phase 2.8*
|
|
49
|
+
- **~120+ L5-L6 verification tasks** (in task descriptions)
|
|
50
|
+
|
|
51
|
+
**Timeline:**
|
|
52
|
+
- Week -1: Phase 0 (Research & Setup)
|
|
53
|
+
- Weeks 1-4: Phase 1 (Foundation)
|
|
54
|
+
- Weeks 5-10: Phase 2 (Core Features)
|
|
55
|
+
- Weeks 11-14: Phase 3 (Rails Integration)
|
|
56
|
+
- Weeks 15-20: Phase 4 (Production Hardening)
|
|
57
|
+
- Weeks 21-26: Phase 5 (Scale & Optimization)
|
|
58
|
+
- **Total Duration:** 23-27 weeks
|
|
59
|
+
|
|
60
|
+
**Team:**
|
|
61
|
+
- Peak parallelization: 6 developers simultaneously
|
|
62
|
+
- Total estimate: ~1000+ person-hours
|
|
63
|
+
|
|
64
|
+
---
|
|
65
|
+
|
|
66
|
+
## ✅ WHAT'S INCLUDED IN THE PLAN
|
|
67
|
+
|
|
68
|
+
### **Phase 0: Gem Setup** ✅
|
|
69
|
+
- Best practices research (Devise, Sidekiq, Puma, Dry-rb, Yabeda, Sentry)
|
|
70
|
+
- Project skeleton setup (Zeitwerk, CI/CD, Docker Compose)
|
|
71
|
+
- Team alignment
|
|
72
|
+
|
|
73
|
+
### **Phase 1: Foundation** ✅
|
|
74
|
+
- Event::Base (zero-allocation, schema DSL, presets)
|
|
75
|
+
- Buffers (RingBuffer, AdaptiveBuffer, RequestScoped)
|
|
76
|
+
- Middleware Pipeline (zones, ordering)
|
|
77
|
+
|
|
78
|
+
### **Phase 2: Core Features** ✅
|
|
79
|
+
- PII Filtering & Security (3-tier, audit pipeline)
|
|
80
|
+
- Adapter Architecture (6 adapters: Stdout, File, Loki, Sentry, Elasticsearch, InMemory)
|
|
81
|
+
- Metrics & Yabeda (pattern-based, cardinality protection)
|
|
82
|
+
- Sampling & Cost Optimization (Basic - L2.7)
|
|
83
|
+
|
|
84
|
+
### **Phase 2.8: Advanced Sampling Strategies** ⚡ *NEW* ⏳
|
|
85
|
+
- Error-Based Adaptive Sampling (↑ rate during incidents)
|
|
86
|
+
- Load-Based Adaptive Sampling (↓ rate during overload)
|
|
87
|
+
- Value-Based Sampling (↑ rate for high-value events)
|
|
88
|
+
- Stratified Sampling for SLO Accuracy (C11)
|
|
89
|
+
- **Status:** Planned (2026-01-20), awaiting approval to start
|
|
90
|
+
- **Depends On:** L2.7 (Basic Sampling - completed)
|
|
91
|
+
|
|
92
|
+
### **Phase 3: Rails Integration** ✅
|
|
93
|
+
- Railtie & Auto-Configuration
|
|
94
|
+
- Rails Instrumentation (unidirectional ASN → E11y)
|
|
95
|
+
- Sidekiq/ActiveJob Integration (hybrid tracing)
|
|
96
|
+
- Rails.logger Migration
|
|
97
|
+
|
|
98
|
+
### **Phase 4: Production Hardening** ✅
|
|
99
|
+
- Reliability & Error Handling (DLQ, rate limiting, retry)
|
|
100
|
+
- OpenTelemetry Integration (logs, baggage PII protection)
|
|
101
|
+
- Event Versioning & Schema Migrations
|
|
102
|
+
- SLO Tracking & Self-Monitoring
|
|
103
|
+
|
|
104
|
+
### **Phase 5: Scale & Optimization** ✅
|
|
105
|
+
- High Cardinality Protection (dynamic actions)
|
|
106
|
+
- Tiered Storage Migration (hot/warm/cold)
|
|
107
|
+
- Performance Optimization (1K → 10K → 100K events/sec)
|
|
108
|
+
- Documentation & Gem Release
|
|
109
|
+
|
|
110
|
+
---
|
|
111
|
+
|
|
112
|
+
## 🎯 KEY ACHIEVEMENTS
|
|
113
|
+
|
|
114
|
+
✅ **All ADRs covered** (ADR-001 to ADR-016)
|
|
115
|
+
✅ **All Use Cases covered** (UC-001 to UC-022)
|
|
116
|
+
✅ **All critical resolutions** (C01 to C20)
|
|
117
|
+
✅ **Detailed DoD for each task**
|
|
118
|
+
✅ **[P] markers for parallelization**
|
|
119
|
+
✅ **Quality Gates for milestone tasks**
|
|
120
|
+
✅ **Verification steps (L6) in descriptions**
|
|
121
|
+
|
|
122
|
+
---
|
|
123
|
+
|
|
124
|
+
## 🚀 NEXT STEPS
|
|
125
|
+
|
|
126
|
+
### **Current Status (2026-01-20):**
|
|
127
|
+
- ✅ **L2.7 (Basic Sampling)** - COMPLETED
|
|
128
|
+
- ⏳ **FEAT-4837 (Phase 2.8)** - AWAITING APPROVAL
|
|
129
|
+
- Plan created for advanced sampling strategies
|
|
130
|
+
- Need human approval to start execution
|
|
131
|
+
|
|
132
|
+
### **To Continue:**
|
|
133
|
+
1. **Return to main plan:** Continue with original Phase 2 tasks (L2.8+)
|
|
134
|
+
2. **Approve Phase 2.8:** Review and approve FEAT-4837 when ready to implement advanced sampling
|
|
135
|
+
3. **Track progress:** Monitor via TeamTab dashboard
|
|
136
|
+
|
|
137
|
+
---
|
|
138
|
+
|
|
139
|
+
## 📝 REFERENCES
|
|
140
|
+
|
|
141
|
+
- **Main Plan:** `docs/IMPLEMENTATION_PLAN.md`
|
|
142
|
+
- **Architecture:** `docs/IMPLEMENTATION_PLAN_ARCHITECTURE.md`
|
|
143
|
+
- **ADRs:** `docs/ADR-001-*.md` to `docs/ADR-016-*.md`
|
|
144
|
+
- **Use Cases:** `docs/use_cases/UC-001-*.md` to `docs/use_cases/UC-022-*.md`
|
|
145
|
+
|
|
146
|
+
---
|
|
147
|
+
|
|
148
|
+
**🎉 Plan is ready for execution! All 6 phases created with detailed structure, DoD, and clear dependencies.**
|