sequel-duckdb 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (39) hide show
  1. checksums.yaml +7 -0
  2. data/.kiro/specs/advanced-sql-features-implementation/design.md +24 -0
  3. data/.kiro/specs/advanced-sql-features-implementation/requirements.md +43 -0
  4. data/.kiro/specs/advanced-sql-features-implementation/tasks.md +24 -0
  5. data/.kiro/specs/duckdb-sql-syntax-compatibility/design.md +258 -0
  6. data/.kiro/specs/duckdb-sql-syntax-compatibility/requirements.md +84 -0
  7. data/.kiro/specs/duckdb-sql-syntax-compatibility/tasks.md +94 -0
  8. data/.kiro/specs/edge-cases-and-validation-fixes/requirements.md +32 -0
  9. data/.kiro/specs/integration-test-database-setup/design.md +0 -0
  10. data/.kiro/specs/integration-test-database-setup/requirements.md +117 -0
  11. data/.kiro/specs/sequel-duckdb-adapter/design.md +542 -0
  12. data/.kiro/specs/sequel-duckdb-adapter/requirements.md +202 -0
  13. data/.kiro/specs/sequel-duckdb-adapter/tasks.md +247 -0
  14. data/.kiro/specs/sql-expression-handling-fix/design.md +298 -0
  15. data/.kiro/specs/sql-expression-handling-fix/requirements.md +86 -0
  16. data/.kiro/specs/sql-expression-handling-fix/tasks.md +22 -0
  17. data/.kiro/specs/test-infrastructure-improvements/requirements.md +106 -0
  18. data/.kiro/steering/product.md +22 -0
  19. data/.kiro/steering/structure.md +88 -0
  20. data/.kiro/steering/tech.md +124 -0
  21. data/.kiro/steering/testing.md +192 -0
  22. data/.rubocop.yml +103 -0
  23. data/.yardopts +8 -0
  24. data/API_DOCUMENTATION.md +919 -0
  25. data/CHANGELOG.md +131 -0
  26. data/LICENSE +21 -0
  27. data/MIGRATION_EXAMPLES.md +740 -0
  28. data/PERFORMANCE_OPTIMIZATIONS.md +723 -0
  29. data/README.md +692 -0
  30. data/Rakefile +27 -0
  31. data/TASK_10.2_IMPLEMENTATION_SUMMARY.md +164 -0
  32. data/docs/DUCKDB_SQL_PATTERNS.md +410 -0
  33. data/docs/TASK_12_VERIFICATION_SUMMARY.md +122 -0
  34. data/lib/sequel/adapters/duckdb.rb +256 -0
  35. data/lib/sequel/adapters/shared/duckdb.rb +2349 -0
  36. data/lib/sequel/duckdb/version.rb +16 -0
  37. data/lib/sequel/duckdb.rb +43 -0
  38. data/sig/sequel/duckdb.rbs +6 -0
  39. metadata +235 -0
@@ -0,0 +1,124 @@
1
+ # Technology Stack
2
+
3
+ ## Language & Runtime
4
+ - **Ruby**: 3.1+ required
5
+ - **Gem**: Standard Ruby gem structure
6
+
7
+ ## Dependencies
8
+ - **Sequel**: Database toolkit (core dependency)
9
+ - **duckdb**: Official DuckDB client gem for connections
10
+ - **DuckDB**: Target database system
11
+ - **Bundler**: Dependency management
12
+ - **Rake**: Build automation
13
+ - **RuboCop**: Code linting and style enforcement
14
+
15
+ ## Testing Framework
16
+ - **Minitest**: Ruby's built-in testing framework (following sequel-hexspace pattern)
17
+ - **Sequel Mock Database**: For SQL generation testing without database connections
18
+ - **DuckDB In-Memory**: For integration testing with actual database instances
19
+
20
+ ## Development Tools
21
+ - **IRB**: Interactive Ruby console
22
+ - **Git**: Version control
23
+ - **GitHub Actions**: CI/CD (configured)
24
+
25
+ ## Code Style & Quality
26
+ - **RuboCop** configuration enforces:
27
+ - Double quotes for string literals
28
+ - Ruby 3.1 target version
29
+ - Standard Ruby style guidelines
30
+
31
+ ## Common Commands
32
+
33
+ ### Setup & Installation
34
+ ```bash
35
+ # Install dependencies
36
+ bundle install
37
+
38
+ # Setup development environment
39
+ bin/setup
40
+
41
+ # Interactive console
42
+ bin/console
43
+ ```
44
+
45
+ ### Development Workflow
46
+ ```bash
47
+ # Run tests (primary development command)
48
+ bundle exec rake test
49
+ # or
50
+ ruby test/all.rb
51
+
52
+ # Run linting
53
+ bundle exec rake rubocop
54
+ # or simply
55
+ rake
56
+
57
+ # Run both tests and linting
58
+ bundle exec rake
59
+
60
+ # Install gem locally
61
+ bundle exec rake install
62
+
63
+ # Build gem
64
+ bundle exec rake build
65
+
66
+ # Release new version
67
+ bundle exec rake release
68
+ ```
69
+
70
+ ### Testing Commands
71
+ ```bash
72
+ # Run all tests
73
+ ruby test/all.rb
74
+
75
+ # Run specific test file
76
+ ruby test/database_test.rb
77
+
78
+ # Run tests with verbose output
79
+ ruby test/all.rb -v
80
+
81
+ # Run specific test method
82
+ ruby test/database_test.rb -n test_connection
83
+ ```
84
+
85
+ ### Code Quality
86
+ ```bash
87
+ # Run RuboCop with auto-correct
88
+ bundle exec rubocop -a
89
+
90
+ # Check specific files
91
+ bundle exec rubocop lib/
92
+ ```
93
+
94
+ ## Coding Standards & References
95
+
96
+ ### Primary Code Style Reference (in order of precedence)
97
+ 1. **jeremyevans/sequel**: Official Sequel repository - follow all conventions and idioms
98
+ 2. **sequel-hexspace**: Secondary reference for adapter patterns
99
+ 3. **sequel_impala**: Additional reference for implementation approaches
100
+
101
+ ### Implementation Guidelines
102
+ - Study git history of reference projects to understand implementation order
103
+ - Focus on incremental, testable implementations
104
+ - **ALWAYS write tests BEFORE implementing functionality (TDD)**
105
+ - Follow Test-Driven Development (TDD) approach strictly
106
+ - All SQL generation must have corresponding tests
107
+ - Every public method must have test coverage
108
+ - Integration tests must use actual DuckDB instances
109
+ - Unit tests must use Sequel's mock database for SQL generation testing
110
+
111
+ ### Testing Requirements (MANDATORY)
112
+ - **Test Structure**: Follow sequel-hexspace test organization exactly
113
+ - **Test Files**: Mirror sequel-hexspace test files (database_test.rb, dataset_test.rb, schema_test.rb, etc.)
114
+ - **SQL Generation Tests**: Every SQL generation method must have unit tests verifying correct SQL output
115
+ - **Integration Tests**: Database operations must have tests using actual DuckDB databases
116
+ - **Error Handling Tests**: All error conditions must have corresponding test coverage
117
+ - **Type Conversion Tests**: All data type mappings must be thoroughly tested
118
+ - **Mock Database Tests**: Use Sequel's mock database for testing SQL generation without database connections
119
+ - **Test Coverage**: Aim for 100% test coverage of all implemented functionality
120
+
121
+ ### Connection Management
122
+ - Use **duckdb** gem exclusively for database connections
123
+ - Follow Sequel's connection pooling patterns
124
+ - Implement proper error handling and connection lifecycle management
@@ -0,0 +1,192 @@
1
+ # Testing Requirements and Standards
2
+
3
+ ## CRITICAL RULE: Test-Driven Development (TDD) is MANDATORY
4
+
5
+ **Every single piece of code implementation must follow strict Test-Driven Development:**
6
+
7
+ 1. **Write Tests FIRST** - Before writing any implementation code, comprehensive tests must be written
8
+ 2. **Red-Green-Refactor** - Follow the TDD cycle: failing test → minimal implementation → refactor
9
+ 3. **No Code Without Tests** - Implementation without corresponding tests is not acceptable
10
+ 4. **100% Test Coverage** - All implemented functionality must have test coverage
11
+
12
+ ## Test Structure (Following sequel-hexspace Pattern)
13
+
14
+ ### Required Test Files
15
+ ```
16
+ test/
17
+ ├── all.rb # Test runner - loads all test files
18
+ ├── spec_helper.rb # Test configuration, setup, and shared utilities
19
+ ├── database_test.rb # Database connection, transactions, and basic functionality
20
+ ├── dataset_test.rb # Comprehensive SQL generation and query execution tests
21
+ ├── schema_test.rb # Schema operations, introspection, and DDL tests
22
+ ├── prepared_statement_test.rb # Prepared statement functionality and parameter binding
23
+ ├── sql_test.rb # SQL generation syntax and correctness tests
24
+ └── type_test.rb # Data type handling, conversion, and mapping tests
25
+ ```
26
+
27
+ ### Test Categories
28
+
29
+ #### 1. SQL Generation Tests (Unit Tests)
30
+ - **Purpose**: Test SQL generation without database connections
31
+ - **Tool**: Use Sequel's mock database functionality
32
+ - **Requirements**:
33
+ - Every SQL generation method must have tests
34
+ - Verify correct SQL syntax and structure
35
+ - Test edge cases and parameter handling
36
+ - Must be fast and isolated
37
+ - Test all SQL operations: SELECT, INSERT, UPDATE, DELETE, DDL
38
+
39
+ #### 2. Integration Tests
40
+ - **Purpose**: Test actual database operations
41
+ - **Tool**: Use real DuckDB in-memory databases
42
+ - **Requirements**:
43
+ - Test actual database operations end-to-end
44
+ - Verify data persistence and retrieval
45
+ - Test connection management and lifecycle
46
+ - Test transaction behavior (commit, rollback)
47
+ - Test error handling with real database errors
48
+
49
+ #### 3. Schema Tests
50
+ - **Purpose**: Test schema operations and introspection
51
+ - **Requirements**:
52
+ - Test table creation, modification, and deletion
53
+ - Test index operations
54
+ - Test schema introspection accuracy
55
+ - Test constraint handling
56
+ - Test DuckDB-specific schema features
57
+
58
+ #### 4. Type Conversion Tests
59
+ - **Purpose**: Test Ruby ↔ DuckDB type mapping
60
+ - **Requirements**:
61
+ - Test all supported data types
62
+ - Test edge cases and null handling
63
+ - Test precision and scale for numeric types
64
+ - Test date/time handling and timezone considerations
65
+ - Test binary data and text encoding
66
+
67
+ #### 5. Error Handling Tests
68
+ - **Purpose**: Test proper exception mapping and error scenarios
69
+ - **Requirements**:
70
+ - Test Sequel exception mapping
71
+ - Test connection failure scenarios
72
+ - Test SQL syntax error handling
73
+ - Test constraint violation handling
74
+ - Test timeout and resource limit scenarios
75
+
76
+ ## Implementation Workflow
77
+
78
+ ### For Every Task Involving Code:
79
+
80
+ 1. **Step 1: Write Tests First**
81
+ - Create comprehensive test cases covering the functionality
82
+ - Include both positive and negative test cases
83
+ - Test edge cases and error conditions
84
+ - Ensure tests fail initially (Red phase)
85
+
86
+ 2. **Step 2: Minimal Implementation**
87
+ - Write the minimal code needed to make tests pass
88
+ - Focus on making tests green, not on perfect implementation
89
+ - Avoid over-engineering at this stage
90
+
91
+ 3. **Step 3: Refactor**
92
+ - Improve code quality while keeping tests green
93
+ - Optimize performance if needed
94
+ - Ensure code follows style guidelines
95
+
96
+ 4. **Step 4: Verify Coverage**
97
+ - Ensure all implemented functionality has test coverage
98
+ - Add additional tests if gaps are found
99
+
100
+ ## Test Quality Standards
101
+
102
+ ### Test Code Quality
103
+ - Tests must be clear and readable
104
+ - Test names should describe what is being tested
105
+ - Tests should be independent and isolated
106
+ - Tests should be deterministic (no flaky tests)
107
+ - Tests should run quickly (especially unit tests)
108
+
109
+ ### Test Coverage Requirements
110
+ - **100% line coverage** for all implemented functionality
111
+ - **Branch coverage** for all conditional logic
112
+ - **Edge case coverage** for error conditions
113
+ - **Integration coverage** for database operations
114
+
115
+ ### Test Documentation
116
+ - Each test file should have a header explaining its purpose
117
+ - Complex test setups should be documented
118
+ - Test utilities should be well-documented
119
+ - Tests serve as executable documentation
120
+
121
+ ## Tools and Utilities
122
+
123
+ ### Mock Database Testing
124
+ ```ruby
125
+ # Example of mock database testing for SQL generation
126
+ DB = Sequel.mock
127
+ dataset = DB[:users].where(name: 'John')
128
+ assert_equal "SELECT * FROM users WHERE (name = 'John')", dataset.sql
129
+ ```
130
+
131
+ ### Integration Testing Setup
132
+ ```ruby
133
+ # Example of integration testing with real DuckDB
134
+ def setup
135
+ @db = Sequel.connect("duckdb::memory:")
136
+ @db.create_table(:test_table) do
137
+ primary_key :id
138
+ String :name
139
+ end
140
+ end
141
+ ```
142
+
143
+ ## Continuous Integration
144
+
145
+ ### Test Execution
146
+ - All tests must pass before any code is merged
147
+ - Tests should be run on multiple Ruby versions
148
+ - Tests should be run on different operating systems
149
+ - Performance regression tests should be included
150
+
151
+ ### Test Reporting
152
+ - Test results should be clearly reported
153
+ - Coverage reports should be generated
154
+ - Failed tests should provide clear error messages
155
+ - Test execution time should be monitored
156
+
157
+ ## Common Testing Patterns
158
+
159
+ ### Testing SQL Generation
160
+ ```ruby
161
+ def test_select_with_where
162
+ dataset = @db[:users].where(name: 'John')
163
+ assert_equal "SELECT * FROM users WHERE (name = 'John')", dataset.sql
164
+ end
165
+ ```
166
+
167
+ ### Testing Database Operations
168
+ ```ruby
169
+ def test_insert_and_select
170
+ @db[:users].insert(name: 'John', email: 'john@example.com')
171
+ user = @db[:users].where(name: 'John').first
172
+ assert_equal 'john@example.com', user[:email]
173
+ end
174
+ ```
175
+
176
+ ### Testing Error Conditions
177
+ ```ruby
178
+ def test_connection_error
179
+ assert_raises(Sequel::DatabaseConnectionError) do
180
+ Sequel.connect("duckdb:/invalid/path/database.db")
181
+ end
182
+ end
183
+ ```
184
+
185
+ ## Remember: NO CODE WITHOUT TESTS
186
+
187
+ This is not optional. Every implementation task must begin with writing comprehensive tests. This ensures:
188
+ - Functionality works as expected
189
+ - Regressions are caught early
190
+ - Code is maintainable and refactorable
191
+ - Documentation through executable examples
192
+ - Confidence in the codebase
data/.rubocop.yml ADDED
@@ -0,0 +1,103 @@
1
+ plugins:
2
+ - rubocop-minitest
3
+ - rubocop-rake
4
+ - rubocop-sequel
5
+
6
+ AllCops:
7
+ TargetRubyVersion: 3.1
8
+ NewCops: enable
9
+
10
+ Gemspec/DevelopmentDependencies:
11
+ Enabled: false
12
+
13
+ # Disable complexity violations for database adapter code
14
+ Metrics/ModuleLength:
15
+ Enabled: true
16
+ Exclude:
17
+ - lib/sequel/adapters/shared/duckdb.rb
18
+
19
+ Metrics/MethodLength:
20
+ Enabled: true
21
+ Exclude:
22
+ - lib/sequel/adapters/shared/duckdb.rb
23
+ - test/**/*
24
+ - test/type_test.rb
25
+
26
+ Metrics/ClassLength:
27
+ Enabled: true
28
+ Exclude:
29
+ - test/**/*
30
+ - test/type_test.rb
31
+
32
+ Metrics/BlockLength:
33
+ Enabled: true
34
+ Exclude:
35
+ - test/**/*
36
+ - test/type_test.rb
37
+ - sequel-duckdb.gemspec
38
+
39
+ Metrics/AbcSize:
40
+ Enabled: true
41
+ Exclude:
42
+ - lib/sequel/adapters/shared/duckdb.rb
43
+ - test/**/*
44
+ - test/type_test.rb
45
+
46
+ Metrics/CyclomaticComplexity:
47
+ Enabled: true
48
+ Exclude:
49
+ - lib/sequel/adapters/shared/duckdb.rb
50
+ - test/**/*
51
+ - test/type_test.rb
52
+
53
+ Metrics/PerceivedComplexity:
54
+ Enabled: true
55
+ Exclude:
56
+ - lib/sequel/adapters/shared/duckdb.rb
57
+ - test/**/*
58
+ - test/type_test.rb
59
+
60
+ # Disable line length for generated SQL and complex expressions
61
+ Layout/LineLength:
62
+ Enabled: true
63
+ Exclude:
64
+ - test/**/*
65
+ - test/type_test.rb
66
+ - sequel-duckdb.gemspec
67
+
68
+ Minitest/MultipleAssertions:
69
+ Enabled: false
70
+
71
+ # Allow duplicate branches for error handling patterns
72
+ Lint/DuplicateBranch:
73
+ Enabled: true
74
+
75
+ # Allow database adapter naming patterns
76
+ Naming/MethodParameterName:
77
+ Enabled: true
78
+
79
+ Naming/AccessorMethodName:
80
+ Enabled: true
81
+
82
+ Naming/VariableNumber:
83
+ Enabled: true
84
+ Exclude:
85
+ - test/**/*
86
+
87
+ # Allow database adapter patterns
88
+ Lint/EmptyBlock:
89
+ Enabled: true
90
+ Exclude:
91
+ - test/**/*
92
+
93
+ Lint/DuplicateMethods:
94
+ Enabled: true
95
+
96
+ Style/TrivialAccessors:
97
+ Enabled: true
98
+
99
+ Style/StringLiterals:
100
+ EnforcedStyle: double_quotes
101
+
102
+ Style/StringLiteralsInInterpolation:
103
+ EnforcedStyle: double_quotes
data/.yardopts ADDED
@@ -0,0 +1,8 @@
1
+ --markup markdown
2
+ --markup-provider kramdown
3
+ --output-dir doc
4
+ --protected
5
+ --private
6
+ lib/**/*.rb
7
+ README.md
8
+ CHANGELOG.md