orfeas_lyra 0.6.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 3d1fc3fab85a30b1ed635864d084fee35468b77ff0ad0ce2aec0b0d004fe43e7
+  data.tar.gz: e050efff730bc551ef67c3d0107dc5c11fb5c5e8ba433970f188c17d3517e3c5
+SHA512:
+  metadata.gz: 5650de44e8ade792aa79bf9fba6e42dd6144e08521a47aa5132e34df94215af9c464faa3d08f87c7d8b98a03649b0f4600e8636a3e334da89100eb0e257a6170
+  data.tar.gz: 03d0c2c5419ffddd950f86d8d97dcc3033be8bbd81dda0a8470ef9ab2c4e743c98484788cb74a59a41cf3933c946496f06201d77341b2c9280e58accd26012f7

data/CHANGELOG.md

@@ -0,0 +1,222 @@
+# Changelog
+
+All notable changes to the Lyra project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.6.0] - 2026-01-05
+
+### Added
+
+#### Core CRUD-to-Event Sourcing Engine
+- Monitor mode for non-intrusive CRUD operation logging
+- Hijack mode for full event sourcing transformation
+- Rails Event Store integration
+- CRUD interceptor using ActiveRecord callbacks
+- Event mapping layer (CRUD operations → Domain events)
+- Command/Aggregate pattern for hijack mode
+- Dual-view comparison (CRUD state vs Event-sourced state)
+- State reconstruction from event streams
+- Audit trail generation
+- Dashboard API endpoints
+- ERB-based dashboard views
+- Pluggable event store backend
+- Custom aggregate support
+- Custom event mapper support
+- Projection system for read models
+- **Custom Metadata Proc** (`metadata_proc`) - Inject custom metadata into every event:
+  - Capture user context from `Current`/`CurrentAttributes`
+  - Support for Devise/Warden authentication
+  - Thread-safe per-request metadata
+  - Graceful error handling (returns `{}` on failure)
+- **Rails Engine Namespaced Model Support** - Properly handle namespaced models:
+  - `MyEngine::Order` → `MyEngineOrderCreated` (sanitized event class name)
+  - Preserves original namespace in `model_class` attribute
+  - Supports deeply nested namespaces (e.g., `MyEngine::Admin::Widget`)
+
+#### Privacy Extraction to PAM DSL
+- **PAM DSL as Optional Dependency** - Privacy features gracefully degrade without PAM DSL
+- **PIIDetector Delegation** - `Lyra::Privacy::PIIDetector` delegates to `PamDsl::PIIDetector`
+- **PIIMasker Delegation** - `Lyra::Privacy::PIIMasker` delegates to `PamDsl::PIIMasker`
+- **GDPRCompliance Delegation** - `Lyra::Privacy::GDPRCompliance` delegates to `PamDsl::GDPRCompliance`
+- **PolicyIntegration Enhancements**:
+  - `use_detector: true` option (default) - Falls back to PIIDetector for fields not in policy
+  - `use_detector: false` - Policy-only mode, no pattern-based detection
+  - `pam_dsl_available?` - Check if PAM DSL is loaded
+  - `source: :policy` or `source: :detector` in detection results
+  - Retention returns `nil` (infinite/manual) when no policy defined
+  - 28 new tests for PolicyIntegration
+
+#### Unified PII Detection System
+- **Consolidated PIIDetector** - Lyra now delegates to `PamDsl::PIIDetector` for unified PII detection
+- **Partial Matching Mode** (default) - Detects PII in compound field names using word boundaries
+  - Matches prefixed fields: `customer_email`, `billing_phone`, `user_name`
+  - Matches suffixed fields: `home_phone`, `work_email`, `shipping_address`
+  - Supports camelCase suffix boundaries: `emailAddress`, `phoneNumber`
+- **Exact Matching Mode** - Optional strict mode for specific known field names only
+- **Configurable via `PamDsl::PIIDetector.partial_match`** - Toggle between matching modes
+- **New PII Types**:
+  - `:identifier` - VAT numbers, tax IDs, passport numbers, AFM (Greek tax ID)
+  - `:financial` - IBAN, bank accounts, salary, income
+  - `:health` - Medical records, diagnosis, prescriptions
+  - `:biometric` - Fingerprints, face ID, retina scans
+  - `:location` - GPS coordinates, latitude/longitude
+- **Exclusion Patterns** - Prevents false positives on:
+  - Timestamps: `*_at`, `*_on`, `*_date`, `*_time`
+  - Counters/amounts: `*_count`, `*_amount`, `*_total`
+  - Flags: `*_verified`, `*_confirmed`, `*_enabled`, `is_*`, `has_*`
+  - Codes: `*_code` (except `postal_code`, `zip_code`)
+  - Security: `*_digest`, `*_token`, `*_hash`, `encrypted_*`
+- **Sensitivity Levels with Legislative Background**:
+  - `:public` - Publicly accessible data
+  - `:internal` - Low risk, basic protection (GDPR Art. 6)
+  - `:confidential` - Medium risk, enhanced protection (GDPR Art. 6, Art. 32)
+  - `:restricted` - High risk, special categories (GDPR Art. 9, PCI DSS)
+- **PII Masking** - Type-specific masking for safe display
+- **56 tests with 188 assertions** for comprehensive coverage
+
+#### Event Schema Validation System
+- **Schema Store** (`lib/lyra/schema/store.rb`) - Versioned schema file persistence in `db/lyra_schemas/`
+- **Schema Generator** (`lib/lyra/schema/generator.rb`) - Generates schemas from monitored ActiveRecord models
+- **Schema Diff** (`lib/lyra/schema/diff.rb`) - Compares schemas with severity classification (BREAKING, WARNING, INFO)
+- **Schema Validator** (`lib/lyra/schema/validator.rb`) - Enforces schema consistency with strict mode support
+- **Schema Reporter** (`lib/lyra/schema/reporter.rb`) - Human-readable model→event mapping reports
+- **Rake Tasks** for schema management:
+  - `rake lyra:schema:create` - Generate initial schema
+  - `rake lyra:schema:update` - Create new schema version
+  - `rake lyra:schema:verify` - Check for schema changes
+  - `rake lyra:schema:report` - Display model→event mappings
+  - `rake lyra:schema:history` - Show version history
+  - `rake lyra:schema:diff[v1,v2]` - Compare two schema versions
+
+#### Event Class Registration for RailsEventStore Compatibility
+- **EventClassRegistrar** (`lib/lyra/schema/event_class_registrar.rb`) - Pre-registers event classes at Rails startup
+- Ensures `Object.const_get()` works when RailsEventStore deserializes events after app restart
+- Registers events from both monitored models configuration and stored schema (for legacy events)
+
+#### Event Sourcing with Disabled Projections (Sixth Mode)
+- **CachedRelation** (`lib/lyra/projections/cached_relation.rb`) - ActiveRecord::Relation-like wrapper for in-memory cached data
+  - Full query interface: `where`, `order`, `limit`, `offset`, `first`, `last`, `find`, `find_by`
+  - Type coercion for string/integer and boolean comparisons (handles params from controllers)
+  - Aggregation methods: `sum`, `average`, `minimum`, `maximum`, `pluck`
+  - Pagination support: `page`, `per` (Kaminari-compatible)
+  - Chainable no-ops for AR methods: `includes`, `joins`, `preload`, `distinct`
+  - Rails 8 Arel predicate value extraction support
+
+- **AssociationInterceptor** (`lib/lyra/interceptors/association_interceptor.rb`) - Patches AR associations for cache-based loading
+  - `BelongsToAssociationPatch` - Intercepts belongs_to association loading
+  - `HasOneAssociationPatch` - Intercepts has_one association loading
+  - `HasManyAssociationPatch` - Intercepts has_many association loading with count/size support
+  - Polymorphic association support for belongs_to
+  - Rails 8 async: keyword argument compatibility
+
+- **EventStoreReader** (`lib/lyra/projections/event_store_reader.rb`) - Reads and caches entity state from event store
+  - Reconstructs current state by replaying events
+  - Solid Cache / Rails.cache integration for fast reads
+  - Cache warming and invalidation on writes
+  - Full query interface via CachedRelation
+
+- **CrudInterceptor first/last overrides** - Proper ordering by primary key
+  - `Model.first` now orders by `primary_key ASC` before returning first record
+  - `Model.last` now orders by `primary_key DESC` before returning first record
+
+#### Benchmark Infrastructure
+- **Mode Comparison Benchmark** (`perf/run_mode_comparison_benchmark.rb`)
+  - Comprehensive benchmarking across all 7 Lyra modes
+  - Scales from 500 to 100,000 operations
+  - Four scenarios: CRUD, batch, query, mixed workloads
+  - Checkpoint/resume support for long-running benchmarks
+  - Automatic chart generation (PNG) via gnuplot
+  - JSON, CSV, and Markdown report output
+  - **Object allocations tracking** - measures Ruby allocations per operation via GC.stat
+  - **Allocations charts** - per-scale bar charts and scaling comparison line chart
+- Warmup phase for JIT/cache priming
+- Cache warming for ES Disabled mode before measurements
+
+#### Testing Infrastructure
+- **Comprehensive CachedRelation test suite** (`test/projections/cached_relation_test.rb`)
+  - 37 tests covering enumerable, finders, where filtering, type coercion, ordering, pagination, aggregations
+  - Isolated unit tests with MockRecord and MockModel classes
+
+- **Unified 6-mode test runner** - All Lyra modes tested together
+  - `rake lyra:test:all_modes` runs all 6 configurations in single execution
+  - Configurations: disabled, monitor, hijack, event_sourcing+sync, event_sourcing+async, event_sourcing+disabled
+  - Comprehensive report with pass/fail status per configuration
+
+#### Configuration Options
+- `strict_schema` - Fail startup if schema changes detected (recommended for production)
+- `schema_path` - Custom path for schema files (defaults to `db/lyra_schemas/`)
+
+#### Database Support
+- PostgreSQL primary database support
+- SQLite alternative for small deployments
+
+#### Example Application
+- Aegean E-Pay Testbed with student registrations
+- Payment processing with multiple channels
+- Complete database schema and seed data
+
+### Fixed
+
+#### StrictDataAccess Compatibility with Event Sourcing Mode
+- Fixed conflict between `StrictDataAccess` and ES mode sync projections
+- Projections now correctly bypass strict access checks (using `Thread.current[:lyra_bypass_strict_access]`)
+- This is safe because projections are internal Lyra operations with events already recorded
+- Fixed require order in `lib/lyra.rb` to load `strict_data_access` before projections
+
+#### CachedRelation Exception Handling in ES Disabled Mode
+- Fixed `CachedRelation.method_missing` swallowing `StrictDataAccessViolation` exceptions
+- `update_all` and `delete_all` on `CachedRelation` now correctly raise violations when strict mode is enabled
+- The generic scope delegation rescue clause now re-raises framework violations instead of suppressing them
+
+#### Automatic Event Creation for `dependent: :nullify` Operations
+- When `dependent: :nullify` triggers `update_all(foreign_key: nil)`, Lyra now creates events for affected child records
+- This ensures the event stream captures foreign key nullification when parent records are destroyed
+- Events include `nullify_source: "dependent_association"` in metadata for traceability
+- Works in both DB-backed modes (plucks from DB) and ES Disabled mode (queries event store cache)
+- Resolves event stream completeness gap where child state changes were previously invisible
+
+#### Automatic Event Creation for Callback-Bypassing Operations
+- `update_columns` and `update_column` now create events when allowed (strict mode off or bypassed)
+- `delete` now creates a "Destroyed" event when allowed
+- Events include `bypass_source: "update_columns"` or `bypass_source: "delete"` in metadata
+- This ensures the event stream remains complete even when developers intentionally bypass callbacks
+- Only applies to Lyra-monitored models when mode is not `:disabled`
+
+#### Test Isolation Issues
+- Fixed test contamination between controller and integration tests
+- Integration tests now create their own event store if none exists
+- Added `create_event_store` helper to `multi_mode_integration_test.rb` and `event_sourcing_mode_test.rb`
+- Fixed `assert_not_nil` → `refute_nil` (Minitest syntax)
+- Added proper `Lyra.config.monitor_model` calls for correct event naming
+
+### Changed
+
+#### Test Coverage Configuration
+- Increased SimpleCov minimum coverage thresholds (50% line, 47% branch)
+- Added coverage groups for Projections and Schema modules
+- All 584 tests now pass with proper isolation
+- Added comprehensive tests for Schema::Diff (column limit changes, event renames, config changes)
+- Added comprehensive tests for Schema::Reporter (report generation, PII summaries, model mappings)
+- Added comprehensive tests for Visualization::Timeline (Mermaid, ASCII, D3.js output)
+- Added comprehensive tests for EventAnalyzer (timeline, operations, privacy analysis)
+- Added comprehensive tests for CommandHandler (event data, metadata, error handling)
+- Added comprehensive tests for DualView (value normalization, type coercion)
+
+### Core Components
+- `CrudInterceptor` - ActiveRecord callback integration
+- `EventMapper` - CRUD to event mapping
+- `Command` - Command pattern implementation
+- `CommandHandler` - Command processing
+- `Aggregate` - Domain aggregate base class
+- `GenericAggregate` - Default aggregate for monitored models
+- `Projection` - Read model projection base
+- `StateProjection` - State reconstruction
+- `AuditProjection` - Audit trail generation
+- `DualView` - CRUD vs Event-sourced comparison
+- `StateAnalyzer` - State analysis and recommendations
+- `EventStoreAdapter` - Pluggable event storage
+- `Configuration` - Centralized configuration

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2019-2026 Michail Pantelelis
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.