propel_api 0.3.2 → 0.3.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0a2cc174f214b50c6e5f58829c9136661591fc142a1e14e09aaf7221c0e1ddda
-  data.tar.gz: ae2f7764a20f7dc8b22b35486c4d1ca68e4910321d98c624306293800326270b
+  metadata.gz: 04fb55f19379b238226c473c68009234156cc6f788d54eb29a89155db938556e
+  data.tar.gz: 6bb69d00b06c6497d0a6fbbec4d4bafc026cd2b1484eaef48e9b31e9bd20564d
 SHA512:
-  metadata.gz: 4ca81a75e2225c0a2bdff55f17aef1e5f6067c7b19c0e815a074201b39a508a0f46d3afa6bcbc54225e7ebc3314cedf46ecdf18cccfa11c15b2031b0afc8c168
-  data.tar.gz: 2bab3dadb532118c96fbdc76e8fa389be29fa4d58801fc3dc7ab6fbea90cffa9534795035f306dd470f6d89f841b055909a3e582634fbae49c5296cc3f44e335
+  metadata.gz: 49af6bb0bbfcad1d62fdf4aada990189a1c1e850df87f1cee4b284ece041bf7c3c777907deff2d2df332b7b44a4563dba6b6f4de29c75c3e4c2e97609345b8d5
+  data.tar.gz: 87d70201da1cd3e2630cc1135997dfdb8bcf7c17900a341e4f9337d9accdc446d64dea4a06aeafcf1a7ae6e621daa2366ab5ea7c35e5434f594bd5e2fbb9869a

data/CHANGELOG.md

@@ -7,9 +7,75 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.3.4] - 2025-09-29
+
+### Fixed
+- **Custom `has_scope` definitions**: The dynamic scope generator no longer overrides custom `has_scope` definitions, allowing developers to implement their own filtering logic without interference.
+- **`has_scope` parameter filtering**: All declared `has_scope` parameters (both dynamic and custom) are now automatically permitted, fixing an issue where they were being filtered out as unsafe.
+
 ### Planned Features
 - GraphQL adapter support
 
+## [0.3.3] - 2025-01-XX
+
+### 🎉 Major Features Added
+- **Comprehensive Dynamic Filtering System**: Complete implementation of automatic URL query string filtering and scoping using the `has_scope` gem
+  - **Multi-Type Filtering Support**: Full filtering support across all data types (string, numeric, boolean, datetime, date, time)
+  - **Database-Agnostic Implementation**: Works seamlessly across SQLite, PostgreSQL, and MySQL without database-specific code
+  - **Automatic Scope Generation**: Dynamic scope generation based on model column types and filter operators
+  - **Intelligent Facet Integration**: Automatic inclusion of filterable fields in JSON facets for API responses
+
+### 🔍 Advanced Filtering Capabilities
+- **String Filtering**: Exact match (`_eq`), contains (`_contains`), starts with (`_starts_with`), ends with (`_ends_with`), in list (`_in`)
+- **Numeric Filtering**: Comparison operators (`_gt`, `_lt`, `_gte`, `_lte`), range filtering (`_range`), in list (`_in`)
+- **Boolean Filtering**: Multiple value format support (`true`/`false`, `1`/`0`, `yes`/`no`, `on`/`off`)
+- **DateTime Filtering**: Temporal comparisons (`_before`, `_after`), date extraction (`_year`, `_month`, `_day`), date matching (`_date`)
+- **Range Filtering**: Support for both array and string formats (e.g., `"150,350"` or `[150, 350]`)
+
+### 🛠️ Technical Implementation
+- **PropelDynamicScopeGenerator**: New generator class for automatic scope creation based on model columns
+- **PropelModelFiltersConcern**: Model concern for dynamic scope generation and filter parameter validation
+- **PropelControllerFiltersConcern**: Controller concern for has_scope integration and security validation
+- **PropelFilterOperators**: Configuration class defining available operators for each data type
+- **Database-Agnostic Datetime Parsing**: Ruby-based datetime parsing instead of SQL extraction functions
+
+### 🔒 Security & Performance
+- **Input Validation**: Comprehensive validation of filter parameters against allowed field names and operators
+- **SQL Injection Protection**: All parameters properly escaped and parameterized
+- **Multi-Tenancy Integration**: Automatic organization scoping for all filtered queries
+- **Sensitive Field Filtering**: Automatic exclusion of sensitive fields (passwords, tokens, etc.) from filtering
+- **Performance Optimization**: Efficient range queries and indexed field prioritization
+
+### 📊 Enhanced JSON Facet System
+- **Smart Field Inclusion**: Automatic inclusion of boolean and datetime fields in `:short` facets
+- **Exclusion Lists**: Configurable exclusion of specific fields (e.g., `created_at`, `updated_at`, `internal_flag`)
+- **Consistent Serialization**: Ensures filterable fields are available in API responses for client-side filtering
+
+### 🧪 Comprehensive Testing
+- **Test-Driven Development**: Complete test suite with 992 tests covering all filtering scenarios
+- **Fixture-Aware Testing**: Unique test data values to avoid conflicts with fixture data
+- **Multi-Tenancy Testing**: Proper testing of organization-scoped filtering
+- **Edge Case Coverage**: Testing of invalid inputs, empty values, and error conditions
+- **Database Compatibility**: Tests pass across SQLite, PostgreSQL, and MySQL
+
+### 📖 Documentation & Examples
+- **Comprehensive API Documentation**: Complete filtering documentation with examples for all operators
+- **Postman Collection Ready**: Detailed query parameter examples for testing
+- **Error Handling Guide**: Common error scenarios and troubleshooting tips
+- **Performance Guidelines**: Optimization recommendations for large datasets
+
+### 🔧 Configuration & Customization
+- **Flexible Operator Configuration**: Easy addition of new filter operators via `PropelFilterOperators`
+- **Customizable Field Exclusions**: Configurable lists for fields to exclude from facets
+- **Template-Based Generation**: All filtering code generated via ERB templates for easy customization
+- **Backward Compatibility**: Existing applications continue to work without changes
+
+### 🐛 Bug Fixes
+- **Template Syntax Errors**: Fixed ERB template syntax issues with `next` vs `return` in select blocks
+- **Scope Generation Caching**: Resolved issues with cached scopes not updating after template changes
+- **Datetime Parsing Errors**: Fixed datetime scope generation to handle multiple input formats
+- **SQL Compatibility**: Resolved SQLite `EXTRACT` function compatibility issues
+
 ## [0.3.2] - 2025-09-15
 
 ### 🐛 Critical Bug Fixes

data/FILTERING_DOCUMENTATION.md

@@ -0,0 +1,470 @@
+# Propel Rails Filtering System Documentation
+
+## Overview
+
+The Propel Rails filtering system provides automatic URL query string filtering and scoping using the `has_scope` gem. It supports filtering across multiple data types with various operators, making it easy to build powerful search and filter interfaces.
+
+## Base URL Structure
+
+```
+GET {{base_url}}/api/v1/{resource}
+```
+
+## Authentication
+
+All API requests require authentication via JWT token in the Authorization header:
+
+```
+Authorization: Bearer <your_jwt_token>
+```
+
+## Data Types and Supported Operators
+
+### 1. String Fields
+
+String fields support text-based filtering with multiple operators.
+
+#### Supported Operators
+
+| Operator | Description | Example |
+|----------|-------------|---------|
+| `_eq` | Exact match | `name_eq=Acme Corporation` |
+| `_contains` | Contains substring | `title_contains=Project` |
+| `_starts_with` | Starts with substring | `name_starts_with=Acme` |
+| `_ends_with` | Ends with substring | `email_ends_with=@company.com` |
+| `_in` | In list of values | `status_in=active,pending` |
+
+#### Examples
+
+```bash
+# Exact match
+GET /api/v1/meetings?title_eq=Important Meeting
+
+# Contains search
+GET /api/v1/meetings?title_contains=Project
+
+# Starts with
+GET /api/v1/organizations?name_starts_with=Acme
+
+# Ends with
+GET /api/v1/users?email_ends_with=@company.com
+
+# Multiple values
+GET /api/v1/meetings?status_in=active,pending,cancelled
+```
+
+### 2. Numeric Fields (Integer, Decimal, Float)
+
+Numeric fields support mathematical comparisons and range operations.
+
+#### Supported Operators
+
+| Operator | Description | Example |
+|----------|-------------|---------|
+| `_eq` | Equal to | `max_participants_eq=50` |
+| `_gt` | Greater than | `max_participants_gt=100` |
+| `_lt` | Less than | `max_participants_lt=500` |
+| `_gte` | Greater than or equal | `max_participants_gte=200` |
+| `_lte` | Less than or equal | `max_participants_lte=400` |
+| `_min` | Alias for `_gte` | `max_participants_min=100` |
+| `_max` | Alias for `_lte` | `max_participants_max=500` |
+| `_range` | Between two values | `max_participants_range=150,350` |
+| `_in` | In list of values | `room_id_in=1,2,3,4` |
+
+#### Examples
+
+```bash
+# Exact match
+GET /api/v1/meetings?max_participants_eq=50
+
+# Greater than
+GET /api/v1/meetings?max_participants_gt=100
+
+# Less than
+GET /api/v1/meetings?max_participants_lt=500
+
+# Range filtering
+GET /api/v1/meetings?max_participants_range=150,350
+
+# Multiple values
+GET /api/v1/meetings?room_id_in=1,2,3
+
+# Combined numeric filters
+GET /api/v1/meetings?max_participants_gte=20&max_participants_lte=100
+```
+
+### 3. Boolean Fields
+
+Boolean fields support true/false filtering with multiple value formats.
+
+#### Supported Operators
+
+| Operator | Description | Example |
+|----------|-------------|---------|
+| `_eq` | Equal to boolean value | `recording_enabled_eq=true` |
+
+#### Supported Boolean Values
+
+| Value | Description |
+|-------|-------------|
+| `true`, `false` | Standard boolean |
+| `1`, `0` | Numeric boolean |
+| `yes`, `no` | Text boolean |
+| `on`, `off` | Toggle boolean |
+
+#### Examples
+
+```bash
+# Standard boolean
+GET /api/v1/meetings?recording_enabled_eq=true
+GET /api/v1/meetings?ai_research_enabled_eq=false
+
+# Numeric boolean
+GET /api/v1/meetings?auto_agenda_eq=1
+GET /api/v1/meetings?follow_up_enabled_eq=0
+
+# Text boolean
+GET /api/v1/meetings?transcription_enabled_eq=yes
+GET /api/v1/meetings?auto_presentation_eq=no
+```
+
+### 4. DateTime Fields
+
+DateTime fields support temporal filtering with various operators.
+
+#### Supported Operators
+
+| Operator | Description | Example |
+|----------|-------------|---------|
+| `_before` | Before datetime | `start_time_before=2024-12-31T23:59:59Z` |
+| `_after` | After datetime | `start_time_after=2024-01-01T00:00:00Z` |
+| `_year` | Specific year | `start_time_year=2024` |
+| `_month` | Specific month | `start_time_month=6` |
+| `_day` | Specific day | `start_time_day=15` |
+| `_date` | Specific date | `start_time_date=2024-06-15` |
+
+#### Supported DateTime Formats
+
+| Format | Example | Description |
+|--------|---------|-------------|
+| ISO 8601 | `2024-06-15T14:30:00Z` | Full datetime with timezone |
+| Date only | `2024-06-15` | Date without time |
+| Local format | `2024-06-15 14:30:00` | Local datetime format |
+| Array format | `[2024, 6, 15]` | Year, month, day array |
+
+#### Examples
+
+```bash
+# Before specific datetime
+GET /api/v1/meetings?start_time_before=2024-12-31T23:59:59Z
+
+# After specific datetime
+GET /api/v1/meetings?start_time_after=2024-01-01T00:00:00Z
+
+# Specific year
+GET /api/v1/meetings?start_time_year=2024
+
+# Specific month
+GET /api/v1/meetings?start_time_month=6
+
+# Specific day
+GET /api/v1/meetings?start_time_day=15
+
+# Specific date
+GET /api/v1/meetings?start_time_date=2024-06-15
+
+# Expiration filtering
+GET /api/v1/rooms?expiration_date_after=2024-12-31T23:59:59Z
+```
+
+### 5. Date Fields
+
+Date fields support date-specific filtering (without time components).
+
+#### Supported Operators
+
+Same as DateTime fields, but optimized for date-only operations.
+
+#### Examples
+
+```bash
+# Date range
+GET /api/v1/events?event_date_after=2024-01-01&event_date_before=2024-12-31
+
+# Specific year
+GET /api/v1/events?event_date_year=2024
+
+# Specific month
+GET /api/v1/events?event_date_month=6
+```
+
+### 6. Time Fields
+
+Time fields support time-of-day filtering.
+
+#### Supported Operators
+
+Same as DateTime fields, but focused on time components.
+
+#### Examples
+
+```bash
+# Time range
+GET /api/v1/schedules?start_time_after=09:00:00&end_time_before=17:00:00
+
+# Specific hour
+GET /api/v1/schedules?start_time_hour=14
+```
+
+## Sorting
+
+All resources support sorting by any filterable field.
+
+### Sorting Syntax
+
+| Format | Description | Example |
+|--------|-------------|---------|
+| `order_by=field` | Ascending order | `order_by=title` |
+| `order_by=-field` | Descending order | `order_by=-created_at` |
+
+### Examples
+
+```bash
+# Sort by title ascending
+GET /api/v1/meetings?order_by=title
+
+# Sort by start time descending
+GET /api/v1/meetings?order_by=-start_time
+
+# Sort by multiple fields (if supported)
+GET /api/v1/meetings?order_by=status,start_time
+```
+
+## Pagination
+
+All list endpoints support pagination parameters.
+
+### Pagination Parameters
+
+| Parameter | Description | Example |
+|-----------|-------------|---------|
+| `page` | Page number (1-based) | `page=2` |
+| `limit` | Records per page | `limit=10` |
+| `per_page` | Alias for limit | `per_page=25` |
+
+### Examples
+
+```bash
+# First page with 10 records
+GET /api/v1/meetings?page=1&limit=10
+
+# Second page with 25 records
+GET /api/v1/meetings?page=2&per_page=25
+
+# Large page size
+GET /api/v1/meetings?limit=100
+```
+
+## Complex Query Examples
+
+### Multi-Parameter Filtering
+
+```bash
+# Find large meetings with recording enabled, starting after June 1st
+GET /api/v1/meetings?max_participants_gte=50&recording_enabled_eq=true&start_time_after=2024-06-01T00:00:00Z
+
+# Find project meetings in 2024, sorted by start time
+GET /api/v1/meetings?title_contains=Project&start_time_year=2024&order_by=start_time
+
+# Find meetings in specific room range with pagination
+GET /api/v1/meetings?room_id_range=1,5&page=1&limit=20
+```
+
+### Organization Filtering
+
+```bash
+# Find organizations with names starting with "Acme"
+GET /api/v1/organizations?name_starts_with=Acme
+
+# Find organizations with specific names
+GET /api/v1/organizations?name_in=Acme Corporation,Tech Corp
+```
+
+### Action Item Report Filtering
+
+```bash
+# Find reports due before end of year with specific status
+GET /api/v1/action_item_reports?follow_up_date_before=2024-12-31T23:59:59Z&status_in=active,pending
+
+# Find reports from specific year and month
+GET /api/v1/action_item_reports?follow_up_date_year=2024&follow_up_date_month=6
+```
+
+### Room Availability
+
+```bash
+# Find available rooms with sufficient capacity
+GET /api/v1/rooms?expiration_date_after=2024-12-31T23:59:59Z&max_capacity_gte=50&order_by=max_capacity
+```
+
+## Response Format
+
+All filtered responses follow this structure:
+
+```json
+{
+  "data": [
+    {
+      "id": 1,
+      "title": "Meeting Title",
+      "start_time": "2024-06-15T14:30:00Z",
+      "max_participants": 50,
+      "recording_enabled": true,
+      // ... other fields based on facet configuration
+    }
+  ],
+  "pagination": {
+    "current_page": 1,
+    "per_page": 10,
+    "total_pages": 5,
+    "total_count": 50,
+    "next_page": 2,
+    "prev_page": null
+  }
+}
+```
+
+## Error Handling
+
+### Common Error Responses
+
+#### 400 Bad Request
+```json
+{
+  "error": "Invalid filter parameter",
+  "message": "Invalid datetime format for start_time_before",
+  "code": "INVALID_FILTER"
+}
+```
+
+#### 401 Unauthorized
+```json
+{
+  "error": "Authentication required",
+  "message": "Valid JWT token must be provided",
+  "code": "MISSING_AUTH_TOKEN"
+}
+```
+
+#### 403 Forbidden
+```json
+{
+  "error": "Organization context required",
+  "message": "Valid organization_id must be provided in JWT token",
+  "code": "MISSING_ORGANIZATION_CONTEXT"
+}
+```
+
+## Testing Tips
+
+### 1. Start Simple
+Begin with single parameter tests to verify basic functionality.
+
+### 2. Combine Gradually
+Add multiple parameters to test complex filtering scenarios.
+
+### 3. Test Edge Cases
+- Invalid dates: `start_time_before=invalid-date`
+- Empty strings: `title_eq=`
+- Special characters: `title_contains=Project%20Meeting`
+- Large numbers: `max_participants_gt=999999`
+
+### 4. Verify Sorting
+Test both ascending and descending order for each sortable field.
+
+### 5. Check Pagination
+Test different page sizes and verify pagination metadata.
+
+### 6. Test Boolean Variations
+Try different boolean value formats: `true`, `1`, `yes`, `on`
+
+## Performance Considerations
+
+### 1. Use Indexed Fields
+Filtering on indexed fields (like `id`, `organization_id`) is faster than unindexed fields.
+
+### 2. Limit Result Sets
+Use pagination to limit large result sets and improve response times.
+
+### 3. Combine Filters Efficiently
+Multiple filters are combined with AND logic, so more specific filters reduce the result set faster.
+
+### 4. Avoid Wildcard Searches
+`_contains` operations are slower than exact matches (`_eq`) or prefix matches (`_starts_with`).
+
+## Security Notes
+
+### 1. Input Validation
+All filter parameters are validated against allowed field names and operators.
+
+### 2. SQL Injection Protection
+All parameters are properly escaped and parameterized to prevent SQL injection.
+
+### 3. Multi-tenancy
+All queries are automatically scoped to the user's organization context.
+
+### 4. Sensitive Field Filtering
+Sensitive fields (passwords, tokens, etc.) are automatically excluded from filtering.
+
+## Troubleshooting
+
+### Common Issues
+
+#### No Results Returned
+- Check if the field name is correct
+- Verify the operator is supported for that field type
+- Ensure the value format matches the expected type
+
+#### Invalid Filter Error
+- Verify the field name exists on the model
+- Check that the operator is supported for the field type
+- Ensure the value can be parsed correctly
+
+#### Authentication Errors
+- Verify the JWT token is valid and not expired
+- Check that the organization context is properly set
+- Ensure the user has access to the requested resource
+
+#### Performance Issues
+- Use more specific filters to reduce result sets
+- Add database indexes for frequently filtered fields
+- Consider using pagination for large datasets
+
+## API Endpoints
+
+### Available Resources
+
+| Resource | Endpoint | Description |
+|----------|----------|-------------|
+| Meetings | `/api/v1/meetings` | Meeting management |
+| Organizations | `/api/v1/organizations` | Organization data |
+| Users | `/api/v1/users` | User management |
+| Rooms | `/api/v1/rooms` | Room management |
+| Action Item Reports | `/api/v1/action_item_reports` | Action item tracking |
+| Meeting Links | `/api/v1/meeting_links` | Meeting link management |
+| Dossiers | `/api/v1/dossiers` | Document management |
+
+### Field Reference
+
+Each resource has different available fields for filtering. Check the individual resource documentation for the complete list of filterable fields.
+
+## Changelog
+
+### Version 0.3.2
+- Added comprehensive datetime filtering support
+- Implemented database-agnostic datetime parsing
+- Added boolean field filtering with multiple value formats
+- Enhanced string filtering with pattern matching
+- Added range filtering for numeric fields
+- Implemented multi-tenancy filtering
+- Added comprehensive error handling and validation

data/README.md

@@ -1,6 +1,51 @@
 # PropelApi
 
-A comprehensive Rails generator that creates complete API resources with models, controllers, tests, and realistic seed data. Supports both PropelFacets (JSON Facet) and Graphiti serialization engines with **fixed route insertion** and proper indentation.
+A comprehensive Rails generator that creates complete API resources with models, controllers, tests, and realistic seed data. Supports both PropelFacets (JSON Facet) and Graphiti serialization engines with **fixed route insertion**, proper indentation, and **comprehensive dynamic filtering system**.
+
+## 🎉 NEW in v0.3.3: Dynamic Filtering System
+
+PropelApi now includes a complete automatic URL query string filtering and scoping system using the `has_scope` gem. This provides powerful, database-agnostic filtering across all data types with zero configuration required.
+
+### Quick Filtering Examples
+
+```bash
+# String filtering
+GET /api/v1/meetings?title_contains=Project&status_in=active,pending
+
+# Numeric filtering  
+GET /api/v1/meetings?max_participants_gte=50&max_participants_lte=200
+
+# Boolean filtering
+GET /api/v1/meetings?recording_enabled_eq=true&ai_research_enabled_eq=false
+
+# DateTime filtering
+GET /api/v1/meetings?start_time_after=2024-01-01T00:00:00Z&start_time_before=2024-12-31T23:59:59Z
+
+# Range filtering
+GET /api/v1/meetings?max_participants_range=150,350
+
+# Combined filtering with sorting and pagination
+GET /api/v1/meetings?title_contains=Project&max_participants_gte=50&order_by=start_time&page=1&limit=20
+```
+
+### Supported Filter Operators
+
+| Data Type | Operators | Examples |
+|-----------|-----------|----------|
+| **String** | `_eq`, `_contains`, `_starts_with`, `_ends_with`, `_in` | `name_eq=Acme`, `title_contains=Project` |
+| **Numeric** | `_eq`, `_gt`, `_lt`, `_gte`, `_lte`, `_range`, `_in` | `max_participants_gte=50`, `price_range=100,500` |
+| **Boolean** | `_eq` (supports `true`/`false`, `1`/`0`, `yes`/`no`, `on`/`off`) | `recording_enabled_eq=true` |
+| **DateTime** | `_before`, `_after`, `_year`, `_month`, `_day`, `_date` | `start_time_after=2024-01-01`, `created_at_year=2024` |
+| **Date** | Same as DateTime | `event_date_after=2024-01-01` |
+| **Time** | Same as DateTime | `start_time_hour=14` |
+
+### Automatic Features
+
+- **Zero Configuration**: Filtering works automatically for all generated resources
+- **Database Agnostic**: Works with SQLite, PostgreSQL, and MySQL
+- **Security Built-in**: SQL injection protection and input validation
+- **Multi-tenancy Ready**: Automatic organization scoping
+- **Performance Optimized**: Efficient queries with proper indexing support
 
 ## Installation