support_table_data 1.4.0 → 1.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 57189f00e55af0f6d3476edbe8921bfb7ba341fa9a1c6e51eb0674e66327bd12
-  data.tar.gz: f80d9a8f7ce4877aec13c5f0151ddc017bbcb94ea4b85b2c404b23161d11aaf8
+  metadata.gz: 7fded73fcb55268fa76b155b9cb192667bbe535e4e11f0cf00f8c3b8536fe3ee
+  data.tar.gz: 9e06e50abcba73a18918a7fe8fa0168c93a2d0b39c0d1cfa03f1d512368632af
 SHA512:
-  metadata.gz: df96d9b761bb03c2ddc9bc915bc5441c24f8bf06e20f53fb2d962ae74c19f6828f84f3b4c3f5c1fe6766bceed05c4e64ec6ae458c64c9e6bf549855b4a492222
-  data.tar.gz: 820ebdea69220e6a03c5b0966138285af9c7e3181c73532917d074c392cc0a708d5bb4c67d0d11f937ef0ffd81e828f226f64aa0a632ec7475578d8edc2a53d6
+  metadata.gz: c107aa8e78cfbdfe8fdafd1cc42bf2f25b90075f561e9a7933e643ec2773647f4f96cc047d4ed0b90f8b15cd50ffa1e1f282c7f3abc61e25d70c3fb5150a2958
+  data.tar.gz: 612b88d848c4eb101018bf2c643f8b76825e7bb22b459a6b5998a70238f9eb8627bbe7f6d8533b362b99300dcdce3a905bc3a22ca30c680ca03e4043c8837c7c

data/AGENTS.md

@@ -0,0 +1,140 @@
+# Copilot Instructions for support_table_data
+
+## Project Overview
+
+A Ruby gem providing an ActiveRecord mixin for managing support/lookup tables with canonical data defined in YAML/JSON/CSV files. The gem dynamically generates helper methods to reference specific records naturally in code (e.g., `Status.pending` instead of `Status.find_by(name: 'Pending')`).
+
+**Core concept**: Support tables blur the line between data and code—they contain small canonical datasets that must exist for the application to work.
+
+## Architecture
+
+### Key Components
+
+- **`SupportTableData` module** ([lib/support_table_data.rb](lib/support_table_data.rb)): Main concern mixed into ActiveRecord models
+- **Named instance system**: Dynamically generates class methods (`.pending`), predicate methods (`.pending?`), and attribute helpers (`.pending_id`) from hash-based data files
+- **Data sync engine**: Compares canonical data files with database records, creating/updating as needed in atomic transactions
+- **File parsers**: Supports YAML, JSON, and CSV formats with unified interface
+
+### Data Flow
+
+1. Data files (YAML/JSON/CSV) define canonical records with unique key attributes
+2. `add_support_table_data` registers file paths and triggers method generation for hash-based files
+3. `sync_table_data!` parses files, loads matching DB records, and updates/creates within transactions
+4. Named instance methods are dynamically defined via `class_eval` with memoization
+
+## Development Workflows
+
+### Running Tests
+
+```bash
+bundle exec rspec                    # Run all specs
+bundle exec rspec spec/support_table_data_spec.rb  # Single file
+bundle exec rake appraisals          # Test against all ActiveRecord versions
+```
+
+Uses RSpec with in-memory SQLite database. Test models defined in [spec/models.rb](spec/models.rb), data files in `spec/data/`.
+
+### Testing Against Multiple ActiveRecord Versions
+
+The gem supports ActiveRecord 6.0-8.0. Uses Appraisal for multi-version testing:
+
+```bash
+bundle exec appraisal install        # Install all gemfiles
+bundle exec appraisal rspec          # Run specs against all versions
+```
+
+See `Appraisals` file and `gemfiles/` directory.
+
+### Code Style
+
+Uses Standard Ruby formatter:
+
+```bash
+bundle exec rake standard:fix        # Auto-fix style issues
+```
+
+## Critical Patterns
+
+### Named Instance Method Generation
+
+**Hash-based data files** trigger dynamic method generation. Example from [spec/data/colors/named_colors.yml](spec/data/colors/named_colors.yml):
+
+```yaml
+red:
+  id: 1
+  name: Red
+  value: 16711680
+```
+
+Generates:
+- `Color.red` → finds record by id
+- `color_instance.red?` → tests if `color_instance.id == 1`
+- `Color.red_id` → returns `1` (if `named_instance_attribute_helpers :id` defined)
+
+**Implementation**: See `define_support_table_named_instance_methods` in [lib/support_table_data.rb](lib/support_table_data.rb#L230-L265). Methods are generated using `class_eval` with string interpolation.
+
+### Custom Setters for Associations
+
+Support tables often reference other support tables via named instances. Pattern from [spec/models.rb](spec/models.rb#L72-L74):
+
+```ruby
+def group_name=(value)
+  self.group = Group.named_instance(value)
+end
+```
+
+Allows data files to reference related records by instance name instead of foreign keys.
+
+### Key Attribute Configuration
+
+By default, uses model's `id` column. Override for non-id keys:
+
+```ruby
+self.support_table_key_attribute = :name  # Use 'name' instead of 'id'
+```
+
+Key attributes cannot be updated—changing them creates new records.
+
+### Dependency Resolution
+
+`sync_all!` automatically resolves dependencies via `belongs_to` associations and loads tables in correct order. For complex cases (join tables, indirect dependencies), explicitly declare:
+
+```ruby
+support_table_dependency "OtherModel"
+```
+
+See [lib/support_table_data.rb](lib/support_table_data.rb#L219-L222) and dependency resolution logic.
+
+## Testing Conventions
+
+- **Test data isolation**: Each test deletes all records in `before` block ([spec/spec_helper.rb](spec/spec_helper.rb))
+- **Sync before assertions**: Tests call `sync_table_data!` or `sync_all!` before verifying records exist
+- **Multi-file merging**: Tests verify that multiple data files for same model merge correctly (see `Color` model with 5 data files)
+- **STI handling**: See `Polygon`/`Triangle`/`Rectangle` tests for Single Table Inheritance patterns
+
+## Common Pitfalls
+
+1. **Method name conflicts**: Named instance methods raise `ArgumentError` if method already exists. Instance names must match `/\A[a-z][a-z0-9_]+\z/`
+2. **Array vs hash data**: Only hash-keyed data generates named instance methods. Use arrays or underscore-prefixed keys (`_others`) for records without helpers
+3. **Protected instances**: Records in data files cannot be deleted via `destroy` (though this gem doesn't enforce it—see companion caching gem)
+4. **Transaction safety**: All sync operations wrapped in transactions; changes rollback on failure
+
+## Rails Integration
+
+In Rails apps, the gem automatically:
+- Sets `SupportTableData.data_directory` to `Rails.root/db/support_tables`
+- Provides `rake support_table_data:sync` task ([lib/tasks/support_table_data.rake](lib/tasks/support_table_data.rake))
+- Handles eager loading in both classic and Zeitwerk autoloaders
+
+## File References
+
+- Main module: [lib/support_table_data.rb](lib/support_table_data.rb)
+- Test models: [spec/models.rb](spec/models.rb) - comprehensive examples of patterns
+- Sync task: [lib/tasks/support_table_data.rake](lib/tasks/support_table_data.rake)
+- Architecture docs: [ARCHITECTURE.md](ARCHITECTURE.md) - detailed diagrams and design decisions
+
+## Version Compatibility
+
+- Ruby ≥ 2.5
+- ActiveRecord ≥ 6.0
+- Ruby 3.4+: Requires `csv` gem in Gemfile (removed from stdlib)

data/ARCHITECTURE.md

@@ -0,0 +1,538 @@
+# Support Table Data Architecture
+
+This document describes the architecture and design patterns of the `support_table_data` gem, which provides a robust solution for managing static support tables (lookup tables) in ActiveRecord applications.
+
+## Overview
+
+The SupportTableData gem solves the common problem of managing small, canonical datasets that exist at the intersection of data and code. These support tables contain a limited number of rows with values that are often referenced in application logic, making them critical for proper application function.
+
+## Core Components
+
+### Main Components Overview
+
+```mermaid
+flowchart TB
+    subgraph "Application Layer"
+        Model[ActiveRecord Model]
+        DataFiles[Data Files<br/>YAML/JSON/CSV]
+    end
+
+    subgraph "SupportTableData Gem"
+        Concern[SupportTableData Module]
+        Parser[File Parser]
+        Sync[Data Synchronizer]
+        Helpers[Helper Methods]
+    end
+
+    subgraph "Infrastructure"
+        DB[(Database)]
+        Rails[Rails Environment]
+        Rake[Rake Tasks]
+    end
+
+    Model --> Concern
+    DataFiles --> Parser
+    Parser --> Sync
+    Sync --> DB
+    Concern --> Helpers
+    Rails --> Rake
+    Rake --> Sync
+
+    style Concern fill:#e1f5fe
+    style DataFiles fill:#f3e5f5
+    style DB fill:#e8f5e8
+```
+
+## Data Flow Architecture
+
+The gem follows a clear data flow pattern from static files to database records:
+
+```mermaid
+flowchart LR
+    subgraph "Source Data"
+        YML[YAML Files]
+        JSON[JSON Files]
+        CSV[CSV Files]
+    end
+
+    subgraph "Processing"
+        Parse[File Parser]
+        Validate[Data Validation]
+        Transform[Data Transformation]
+    end
+
+    subgraph "Storage"
+        Memory[In-Memory Cache]
+        DB[(Database Tables)]
+    end
+
+    subgraph "Usage"
+        Helpers[Helper Methods]
+        Queries[Database Queries]
+        Predicates[Predicate Methods]
+    end
+
+    YML --> Parse
+    JSON --> Parse
+    CSV --> Parse
+    Parse --> Validate
+    Validate --> Transform
+    Transform --> Memory
+    Transform --> DB
+    Memory --> Helpers
+    DB --> Queries
+    Helpers --> Predicates
+
+    style Parse fill:#fff3e0
+    style DB fill:#e8f5e8
+    style Helpers fill:#e3f2fd
+```
+
+## Class Structure
+
+### Core Module Design
+
+```mermaid
+classDiagram
+    class SupportTableData {
+        +data_directory : String
+        +sync_all!() : Hash
+        +support_table_classes() : Array
+    }
+
+    class ActiveRecord_Model {
+        +support_table_key_attribute : String
+        +support_table_data_directory : String
+        +add_support_table_data(path)
+        +sync_table_data!() : Array
+        +named_instance(name) : Record
+        +instance_names() : Array
+        +protected_instance?(record) : Boolean
+    }
+
+    class NamedInstanceHelpers {
+        +class_name() : Record
+        +class_name_question() : Boolean
+        +class_name_attribute() : Value
+    }
+
+    class FileParser {
+        +parse_yaml(content) : Hash
+        +parse_json(content) : Hash
+        +parse_csv(content) : Array
+    }
+
+    class DataSynchronizer {
+        +sync_records(data) : Array
+        +create_record(attributes) : Record
+        +update_record(record, attributes) : Record
+    }
+
+    SupportTableData --> ActiveRecord_Model : extends
+    ActiveRecord_Model --> NamedInstanceHelpers : generates
+    ActiveRecord_Model --> FileParser : uses
+    ActiveRecord_Model --> DataSynchronizer : uses
+
+    note for NamedInstanceHelpers "Methods are dynamically generated based on data file content"
+```
+
+## Named Instance System
+
+The gem's most powerful feature is its named instance system, which generates helper methods from data files:
+
+```mermaid
+flowchart TD
+    subgraph "Data File Structure"
+        HashData[Hash-based Data]
+        ArrayData[Array-based Data]
+    end
+
+    subgraph "Method Generation"
+        ClassMethods[Class Methods<br/>Status.pending]
+        InstanceMethods[Instance Methods<br/>status.pending?]
+        AttributeHelpers[Attribute Helpers<br/>Status.pending_id]
+    end
+
+    subgraph "Runtime Usage"
+        FindRecord[Find Specific Record]
+        TestRecord[Test Record Type]
+        GetAttribute[Get Static Attribute]
+    end
+
+    HashData --> ClassMethods
+    HashData --> InstanceMethods
+    ArrayData --> ClassMethods
+    ClassMethods --> FindRecord
+    InstanceMethods --> TestRecord
+    AttributeHelpers --> GetAttribute
+
+    style HashData fill:#f3e5f5
+    style ClassMethods fill:#e3f2fd
+    style InstanceMethods fill:#e8f5e8
+```
+
+## Data Synchronization Process
+
+The synchronization process ensures database consistency with data files:
+
+```mermaid
+sequenceDiagram
+    participant App as Application
+    participant Model as AR Model
+    participant Parser as File Parser
+    participant DB as Database
+
+    App->>Model: sync_table_data!
+    Model->>Parser: Parse data files
+    Parser-->>Model: Canonical data
+
+    Model->>DB: BEGIN TRANSACTION
+    Model->>DB: Find existing records
+    DB-->>Model: Current records
+
+    loop For each record
+        Model->>Model: Compare attributes
+        alt Record changed
+            Model->>DB: UPDATE record
+        end
+    end
+
+    loop For new records
+        Model->>DB: INSERT record
+    end
+
+    Model->>DB: COMMIT TRANSACTION
+    Model-->>App: Changes summary
+```
+
+## File Format Support
+
+The gem supports multiple data file formats with a unified interface:
+
+```mermaid
+flowchart LR
+    subgraph "Input Formats"
+        YAML[YAML<br/>*.yml, *.yaml]
+        JSON[JSON<br/>*.json]
+        CSV[CSV<br/>*.csv]
+    end
+
+    subgraph "Parser Layer"
+        YAMLParser[YAML::safe_load]
+        JSONParser[JSON::parse]
+        CSVParser[CSV Parser<br/>with headers]
+    end
+
+    subgraph "Output Format"
+        Hash[Hash Structure<br/>Named instances]
+        Array[Array Structure<br/>Simple records]
+    end
+
+    YAML --> YAMLParser
+    JSON --> JSONParser
+    CSV --> CSVParser
+
+    YAMLParser --> Hash
+    YAMLParser --> Array
+    JSONParser --> Hash
+    JSONParser --> Array
+    CSVParser --> Array
+
+    style YAML fill:#e8f5e8
+    style JSON fill:#fff3e0
+    style CSV fill:#f3e5f5
+```
+
+## Dependency Resolution
+
+The gem automatically resolves dependencies between support table models:
+
+```mermaid
+flowchart TB
+    subgraph "Dependency Analysis"
+        Belongs[belongs_to Associations]
+        Explicit[Explicit Dependencies]
+        Through[has_many :through]
+    end
+
+    subgraph "Resolution Strategy"
+        Detect[Detect Dependencies]
+        Sort[Topological Sort]
+        Validate[Circular Detection]
+    end
+
+    subgraph "Load Order"
+        Level1[Independent Models]
+        Level2[Models with Dependencies]
+        Level3[Join Table Models]
+    end
+
+    Belongs --> Detect
+    Explicit --> Detect
+    Through --> Detect
+
+    Detect --> Sort
+    Sort --> Validate
+    Validate --> Level1
+    Level1 --> Level2
+    Level2 --> Level3
+
+    style Detect fill:#e3f2fd
+    style Sort fill:#f3e5f5
+    style Level1 fill:#e8f5e8
+```
+
+## Rails Integration
+
+The gem integrates seamlessly with Rails applications:
+
+```mermaid
+flowchart TD
+    subgraph "Rails Boot Process"
+        Boot[Application Boot]
+        Eager[Eager Loading]
+        Routes[Routes Loading]
+    end
+
+    subgraph "Gem Integration"
+        Railtie[SupportTableData::Railtie]
+        Tasks[Rake Tasks]
+        AutoLoad[Auto Discovery]
+    end
+
+    subgraph "Development Workflow"
+        Migrate[db:migrate]
+        Sync[support_table_data:sync]
+        Test[Test Suite Setup]
+    end
+
+    Boot --> Railtie
+    Eager --> AutoLoad
+    Railtie --> Tasks
+    Tasks --> Sync
+    Migrate --> Sync
+    Sync --> Test
+
+    style Railtie fill:#e3f2fd
+    style Sync fill:#e8f5e8
+    style Test fill:#fff3e0
+```
+
+## Error Handling and Validation
+
+The gem includes comprehensive error handling:
+
+```mermaid
+flowchart TD
+    subgraph "Validation Points"
+        FileFormat[File Format Validation]
+        DataStructure[Data Structure Validation]
+        MethodNames[Method Name Validation]
+        KeyAttributes[Key Attribute Validation]
+    end
+
+    subgraph "Error Types"
+        ParseError[File Parse Errors]
+        NameError[Method Name Conflicts]
+        DataError[Data Consistency Errors]
+        TransactionError[Database Transaction Errors]
+    end
+
+    subgraph "Recovery Strategies"
+        Rollback[Transaction Rollback]
+        ErrorReport[Detailed Error Messages]
+        PartialSync[Skip Invalid Records]
+    end
+
+    FileFormat --> ParseError
+    DataStructure --> DataError
+    MethodNames --> NameError
+    KeyAttributes --> DataError
+
+    ParseError --> ErrorReport
+    NameError --> ErrorReport
+    DataError --> Rollback
+    TransactionError --> Rollback
+
+    style ParseError fill:#ffebee
+    style Rollback fill:#e8f5e8
+    style ErrorReport fill:#fff3e0
+```
+
+## Performance Considerations
+
+The gem is designed for optimal performance with small datasets:
+
+```mermaid
+flowchart LR
+    subgraph "Optimization Strategies"
+        SmallData[Small Dataset Focus<br/>&lt; 100 records]
+        Memoization[Method Memoization]
+        Transaction[Atomic Transactions]
+        LazyLoad[Lazy Method Generation]
+    end
+
+    subgraph "Memory Management"
+        ClassVars[Class Variables]
+        Mutex[Thread Safety]
+        WeakRef[Weak References]
+    end
+
+    subgraph "Database Efficiency"
+        BulkOps[Bulk Operations]
+        IndexHints[Key Attribute Indexing]
+        MinQueries[Minimize Queries]
+    end
+
+    SmallData --> ClassVars
+    Memoization --> Mutex
+    Transaction --> BulkOps
+    LazyLoad --> WeakRef
+
+    ClassVars --> MinQueries
+    Mutex --> IndexHints
+    BulkOps --> MinQueries
+
+    style SmallData fill:#e8f5e8
+    style Transaction fill:#e3f2fd
+    style MinQueries fill:#fff3e0
+```
+
+## Extension Points
+
+The gem provides several extension points for customization:
+
+```mermaid
+flowchart TB
+    subgraph "Customization Options"
+        KeyAttr[Custom Key Attributes]
+        DataDir[Custom Data Directories]
+        Parser[Custom File Parsers]
+        Validation[Custom Validations]
+    end
+
+    subgraph "Integration Hooks"
+        BeforeSync[Before Sync Callbacks]
+        AfterSync[After Sync Callbacks]
+        MethodGen[Method Generation Hooks]
+        ErrorHook[Error Handling Hooks]
+    end
+
+    subgraph "External Gems"
+        Cache[SupportTableCache]
+        Observers[ActiveRecord Observers]
+        Auditing[Auditing Gems]
+    end
+
+    KeyAttr --> BeforeSync
+    DataDir --> MethodGen
+    Parser --> ErrorHook
+    Validation --> AfterSync
+
+    BeforeSync --> Cache
+    MethodGen --> Observers
+    ErrorHook --> Auditing
+
+    style Cache fill:#e3f2fd
+    style BeforeSync fill:#f3e5f5
+    style MethodGen fill:#e8f5e8
+```
+
+## Deployment Strategy
+
+The gem follows a deployment-friendly pattern:
+
+```mermaid
+flowchart LR
+    subgraph "Development"
+        DevData[Dev Data Files]
+        DevDB[Dev Database]
+        DevTest[Test Suite]
+    end
+
+    subgraph "CI/CD Pipeline"
+        Build[Build Process]
+        Migration[Run Migrations]
+        DataSync[Sync Support Data]
+        TestRun[Run Tests]
+    end
+
+    subgraph "Production"
+        ProdDB[Production Database]
+        ProdData[Production Data]
+        Monitor[Monitoring]
+    end
+
+    DevData --> Build
+    DevDB --> Migration
+    DevTest --> TestRun
+
+    Build --> Migration
+    Migration --> DataSync
+    DataSync --> TestRun
+    TestRun --> ProdDB
+
+    ProdDB --> ProdData
+    ProdData --> Monitor
+
+    style DataSync fill:#e8f5e8
+    style TestRun fill:#e3f2fd
+    style Monitor fill:#fff3e0
+```
+
+## Best Practices
+
+### Model Organization
+
+```mermaid
+flowchart TD
+    subgraph "File Structure"
+        Models[app/models/]
+        DataFiles[db/support_tables/]
+        Tests[spec/models/]
+    end
+
+    subgraph "Naming Conventions"
+        ModelFile[status.rb]
+        DataFile[statuses.yml]
+        TestFile[status_spec.rb]
+    end
+
+    subgraph "Content Organization"
+        Includes[Include SupportTableData]
+        Config[Configure Directories]
+        AddData[Add Data Files]
+        Helpers[Define Helpers]
+    end
+
+    Models --> ModelFile
+    DataFiles --> DataFile
+    Tests --> TestFile
+
+    ModelFile --> Includes
+    DataFile --> Config
+    TestFile --> AddData
+    Config --> Helpers
+
+    style ModelFile fill:#e3f2fd
+    style DataFile fill:#f3e5f5
+    style Includes fill:#e8f5e8
+```
+
+## Summary
+
+The SupportTableData gem provides a comprehensive solution for managing static support tables through:
+
+1. **Declarative Configuration**: Define support data in version-controlled files
+2. **Automatic Synchronization**: Keep database records in sync with data files
+3. **Generated Helper Methods**: Clean, readable code for accessing specific records
+4. **Dependency Management**: Automatic resolution of inter-table dependencies
+5. **Rails Integration**: Seamless integration with Rails applications and deployment workflows
+6. **Type Safety**: Compile-time method generation with runtime validation
+7. **Performance**: Optimized for small datasets with minimal overhead
+
+The architecture promotes maintainable code by keeping support data close to the application logic while ensuring data consistency across environments.
+
+<function_calls>
+<invoke name="get-syntax-docs-mermaid">
+<parameter name="file">flowchart.md

data/CHANGELOG.md

@@ -4,6 +4,19 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 1.5.0
+
+### Added
+
+- The default data directory in a Rails application can be set with the `config.support_table.data_directory` option in the Rails application configuration.
+- Added rake task `support_table_data:yard_docs:add` for Rails applications that will add YARD documentation to support table models for the named instance helpers. There is also a task `support_table_data:yard_docs:verify` that can be used in a build pipeline to verify that the documentation is up to date. You can also remove the documentation with the `support_table_data:yard_docs:remove` task.
+- The data synchronization task is now automatically attached to several Rails tasks: `db:seed`, `db:seed:replant`, `db:prepare`, `db:test:prepare`, `db:fixtures:load`. Support tables will be synced after running any of these tasks. This can be disabled by setting `config.support_table.auto_sync = false` in the Rails application configuration.
+
+### Changed
+
+- The default data directory is now set in a Railtie and can be overridden with the `config.support_table.data_directory` option in the Rails application configuration.
+- The `support_table_key_attribute` method now returns "id" if not explicitly set instead of implicitly interpreting `nil` as the primary key. This makes the behavior more consistent and explicit and avoids edge cases when running the code in environments where the database connection is not available. This is a breaking change if the table uses a primary key other than "id" and the `support_table_key_attribute` was not explicitly set to that primary key.
+
 ## 1.4.0
 
 ### Fixed