sequel-duckdb 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.kiro/specs/advanced-sql-features-implementation/design.md +24 -0
- data/.kiro/specs/advanced-sql-features-implementation/requirements.md +43 -0
- data/.kiro/specs/advanced-sql-features-implementation/tasks.md +24 -0
- data/.kiro/specs/duckdb-sql-syntax-compatibility/design.md +258 -0
- data/.kiro/specs/duckdb-sql-syntax-compatibility/requirements.md +84 -0
- data/.kiro/specs/duckdb-sql-syntax-compatibility/tasks.md +94 -0
- data/.kiro/specs/edge-cases-and-validation-fixes/requirements.md +32 -0
- data/.kiro/specs/integration-test-database-setup/design.md +0 -0
- data/.kiro/specs/integration-test-database-setup/requirements.md +117 -0
- data/.kiro/specs/sequel-duckdb-adapter/design.md +542 -0
- data/.kiro/specs/sequel-duckdb-adapter/requirements.md +202 -0
- data/.kiro/specs/sequel-duckdb-adapter/tasks.md +247 -0
- data/.kiro/specs/sql-expression-handling-fix/design.md +298 -0
- data/.kiro/specs/sql-expression-handling-fix/requirements.md +86 -0
- data/.kiro/specs/sql-expression-handling-fix/tasks.md +22 -0
- data/.kiro/specs/test-infrastructure-improvements/requirements.md +106 -0
- data/.kiro/steering/product.md +22 -0
- data/.kiro/steering/structure.md +88 -0
- data/.kiro/steering/tech.md +124 -0
- data/.kiro/steering/testing.md +192 -0
- data/.rubocop.yml +103 -0
- data/.yardopts +8 -0
- data/API_DOCUMENTATION.md +919 -0
- data/CHANGELOG.md +131 -0
- data/LICENSE +21 -0
- data/MIGRATION_EXAMPLES.md +740 -0
- data/PERFORMANCE_OPTIMIZATIONS.md +723 -0
- data/README.md +692 -0
- data/Rakefile +27 -0
- data/TASK_10.2_IMPLEMENTATION_SUMMARY.md +164 -0
- data/docs/DUCKDB_SQL_PATTERNS.md +410 -0
- data/docs/TASK_12_VERIFICATION_SUMMARY.md +122 -0
- data/lib/sequel/adapters/duckdb.rb +256 -0
- data/lib/sequel/adapters/shared/duckdb.rb +2349 -0
- data/lib/sequel/duckdb/version.rb +16 -0
- data/lib/sequel/duckdb.rb +43 -0
- data/sig/sequel/duckdb.rbs +6 -0
- metadata +235 -0
@@ -0,0 +1,124 @@
|
|
1
|
+
# Technology Stack
|
2
|
+
|
3
|
+
## Language & Runtime
|
4
|
+
- **Ruby**: 3.1+ required
|
5
|
+
- **Gem**: Standard Ruby gem structure
|
6
|
+
|
7
|
+
## Dependencies
|
8
|
+
- **Sequel**: Database toolkit (core dependency)
|
9
|
+
- **duckdb**: Official DuckDB client gem for connections
|
10
|
+
- **DuckDB**: Target database system
|
11
|
+
- **Bundler**: Dependency management
|
12
|
+
- **Rake**: Build automation
|
13
|
+
- **RuboCop**: Code linting and style enforcement
|
14
|
+
|
15
|
+
## Testing Framework
|
16
|
+
- **Minitest**: Ruby's built-in testing framework (following sequel-hexspace pattern)
|
17
|
+
- **Sequel Mock Database**: For SQL generation testing without database connections
|
18
|
+
- **DuckDB In-Memory**: For integration testing with actual database instances
|
19
|
+
|
20
|
+
## Development Tools
|
21
|
+
- **IRB**: Interactive Ruby console
|
22
|
+
- **Git**: Version control
|
23
|
+
- **GitHub Actions**: CI/CD (configured)
|
24
|
+
|
25
|
+
## Code Style & Quality
|
26
|
+
- **RuboCop** configuration enforces:
|
27
|
+
- Double quotes for string literals
|
28
|
+
- Ruby 3.1 target version
|
29
|
+
- Standard Ruby style guidelines
|
30
|
+
|
31
|
+
## Common Commands
|
32
|
+
|
33
|
+
### Setup & Installation
|
34
|
+
```bash
|
35
|
+
# Install dependencies
|
36
|
+
bundle install
|
37
|
+
|
38
|
+
# Setup development environment
|
39
|
+
bin/setup
|
40
|
+
|
41
|
+
# Interactive console
|
42
|
+
bin/console
|
43
|
+
```
|
44
|
+
|
45
|
+
### Development Workflow
|
46
|
+
```bash
|
47
|
+
# Run tests (primary development command)
|
48
|
+
bundle exec rake test
|
49
|
+
# or
|
50
|
+
ruby test/all.rb
|
51
|
+
|
52
|
+
# Run linting
|
53
|
+
bundle exec rake rubocop
|
54
|
+
# or simply
|
55
|
+
rake
|
56
|
+
|
57
|
+
# Run both tests and linting
|
58
|
+
bundle exec rake
|
59
|
+
|
60
|
+
# Install gem locally
|
61
|
+
bundle exec rake install
|
62
|
+
|
63
|
+
# Build gem
|
64
|
+
bundle exec rake build
|
65
|
+
|
66
|
+
# Release new version
|
67
|
+
bundle exec rake release
|
68
|
+
```
|
69
|
+
|
70
|
+
### Testing Commands
|
71
|
+
```bash
|
72
|
+
# Run all tests
|
73
|
+
ruby test/all.rb
|
74
|
+
|
75
|
+
# Run specific test file
|
76
|
+
ruby test/database_test.rb
|
77
|
+
|
78
|
+
# Run tests with verbose output
|
79
|
+
ruby test/all.rb -v
|
80
|
+
|
81
|
+
# Run specific test method
|
82
|
+
ruby test/database_test.rb -n test_connection
|
83
|
+
```
|
84
|
+
|
85
|
+
### Code Quality
|
86
|
+
```bash
|
87
|
+
# Run RuboCop with auto-correct
|
88
|
+
bundle exec rubocop -a
|
89
|
+
|
90
|
+
# Check specific files
|
91
|
+
bundle exec rubocop lib/
|
92
|
+
```
|
93
|
+
|
94
|
+
## Coding Standards & References
|
95
|
+
|
96
|
+
### Primary Code Style Reference (in order of precedence)
|
97
|
+
1. **jeremyevans/sequel**: Official Sequel repository - follow all conventions and idioms
|
98
|
+
2. **sequel-hexspace**: Secondary reference for adapter patterns
|
99
|
+
3. **sequel_impala**: Additional reference for implementation approaches
|
100
|
+
|
101
|
+
### Implementation Guidelines
|
102
|
+
- Study git history of reference projects to understand implementation order
|
103
|
+
- Focus on incremental, testable implementations
|
104
|
+
- **ALWAYS write tests BEFORE implementing functionality (TDD)**
|
105
|
+
- Follow Test-Driven Development (TDD) approach strictly
|
106
|
+
- All SQL generation must have corresponding tests
|
107
|
+
- Every public method must have test coverage
|
108
|
+
- Integration tests must use actual DuckDB instances
|
109
|
+
- Unit tests must use Sequel's mock database for SQL generation testing
|
110
|
+
|
111
|
+
### Testing Requirements (MANDATORY)
|
112
|
+
- **Test Structure**: Follow sequel-hexspace test organization exactly
|
113
|
+
- **Test Files**: Mirror sequel-hexspace test files (database_test.rb, dataset_test.rb, schema_test.rb, etc.)
|
114
|
+
- **SQL Generation Tests**: Every SQL generation method must have unit tests verifying correct SQL output
|
115
|
+
- **Integration Tests**: Database operations must have tests using actual DuckDB databases
|
116
|
+
- **Error Handling Tests**: All error conditions must have corresponding test coverage
|
117
|
+
- **Type Conversion Tests**: All data type mappings must be thoroughly tested
|
118
|
+
- **Mock Database Tests**: Use Sequel's mock database for testing SQL generation without database connections
|
119
|
+
- **Test Coverage**: Aim for 100% test coverage of all implemented functionality
|
120
|
+
|
121
|
+
### Connection Management
|
122
|
+
- Use **duckdb** gem exclusively for database connections
|
123
|
+
- Follow Sequel's connection pooling patterns
|
124
|
+
- Implement proper error handling and connection lifecycle management
|
@@ -0,0 +1,192 @@
|
|
1
|
+
# Testing Requirements and Standards
|
2
|
+
|
3
|
+
## CRITICAL RULE: Test-Driven Development (TDD) is MANDATORY
|
4
|
+
|
5
|
+
**Every single piece of code implementation must follow strict Test-Driven Development:**
|
6
|
+
|
7
|
+
1. **Write Tests FIRST** - Before writing any implementation code, comprehensive tests must be written
|
8
|
+
2. **Red-Green-Refactor** - Follow the TDD cycle: failing test → minimal implementation → refactor
|
9
|
+
3. **No Code Without Tests** - Implementation without corresponding tests is not acceptable
|
10
|
+
4. **100% Test Coverage** - All implemented functionality must have test coverage
|
11
|
+
|
12
|
+
## Test Structure (Following sequel-hexspace Pattern)
|
13
|
+
|
14
|
+
### Required Test Files
|
15
|
+
```
|
16
|
+
test/
|
17
|
+
├── all.rb # Test runner - loads all test files
|
18
|
+
├── spec_helper.rb # Test configuration, setup, and shared utilities
|
19
|
+
├── database_test.rb # Database connection, transactions, and basic functionality
|
20
|
+
├── dataset_test.rb # Comprehensive SQL generation and query execution tests
|
21
|
+
├── schema_test.rb # Schema operations, introspection, and DDL tests
|
22
|
+
├── prepared_statement_test.rb # Prepared statement functionality and parameter binding
|
23
|
+
├── sql_test.rb # SQL generation syntax and correctness tests
|
24
|
+
└── type_test.rb # Data type handling, conversion, and mapping tests
|
25
|
+
```
|
26
|
+
|
27
|
+
### Test Categories
|
28
|
+
|
29
|
+
#### 1. SQL Generation Tests (Unit Tests)
|
30
|
+
- **Purpose**: Test SQL generation without database connections
|
31
|
+
- **Tool**: Use Sequel's mock database functionality
|
32
|
+
- **Requirements**:
|
33
|
+
- Every SQL generation method must have tests
|
34
|
+
- Verify correct SQL syntax and structure
|
35
|
+
- Test edge cases and parameter handling
|
36
|
+
- Must be fast and isolated
|
37
|
+
- Test all SQL operations: SELECT, INSERT, UPDATE, DELETE, DDL
|
38
|
+
|
39
|
+
#### 2. Integration Tests
|
40
|
+
- **Purpose**: Test actual database operations
|
41
|
+
- **Tool**: Use real DuckDB in-memory databases
|
42
|
+
- **Requirements**:
|
43
|
+
- Test actual database operations end-to-end
|
44
|
+
- Verify data persistence and retrieval
|
45
|
+
- Test connection management and lifecycle
|
46
|
+
- Test transaction behavior (commit, rollback)
|
47
|
+
- Test error handling with real database errors
|
48
|
+
|
49
|
+
#### 3. Schema Tests
|
50
|
+
- **Purpose**: Test schema operations and introspection
|
51
|
+
- **Requirements**:
|
52
|
+
- Test table creation, modification, and deletion
|
53
|
+
- Test index operations
|
54
|
+
- Test schema introspection accuracy
|
55
|
+
- Test constraint handling
|
56
|
+
- Test DuckDB-specific schema features
|
57
|
+
|
58
|
+
#### 4. Type Conversion Tests
|
59
|
+
- **Purpose**: Test Ruby ↔ DuckDB type mapping
|
60
|
+
- **Requirements**:
|
61
|
+
- Test all supported data types
|
62
|
+
- Test edge cases and null handling
|
63
|
+
- Test precision and scale for numeric types
|
64
|
+
- Test date/time handling and timezone considerations
|
65
|
+
- Test binary data and text encoding
|
66
|
+
|
67
|
+
#### 5. Error Handling Tests
|
68
|
+
- **Purpose**: Test proper exception mapping and error scenarios
|
69
|
+
- **Requirements**:
|
70
|
+
- Test Sequel exception mapping
|
71
|
+
- Test connection failure scenarios
|
72
|
+
- Test SQL syntax error handling
|
73
|
+
- Test constraint violation handling
|
74
|
+
- Test timeout and resource limit scenarios
|
75
|
+
|
76
|
+
## Implementation Workflow
|
77
|
+
|
78
|
+
### For Every Task Involving Code:
|
79
|
+
|
80
|
+
1. **Step 1: Write Tests First**
|
81
|
+
- Create comprehensive test cases covering the functionality
|
82
|
+
- Include both positive and negative test cases
|
83
|
+
- Test edge cases and error conditions
|
84
|
+
- Ensure tests fail initially (Red phase)
|
85
|
+
|
86
|
+
2. **Step 2: Minimal Implementation**
|
87
|
+
- Write the minimal code needed to make tests pass
|
88
|
+
- Focus on making tests green, not on perfect implementation
|
89
|
+
- Avoid over-engineering at this stage
|
90
|
+
|
91
|
+
3. **Step 3: Refactor**
|
92
|
+
- Improve code quality while keeping tests green
|
93
|
+
- Optimize performance if needed
|
94
|
+
- Ensure code follows style guidelines
|
95
|
+
|
96
|
+
4. **Step 4: Verify Coverage**
|
97
|
+
- Ensure all implemented functionality has test coverage
|
98
|
+
- Add additional tests if gaps are found
|
99
|
+
|
100
|
+
## Test Quality Standards
|
101
|
+
|
102
|
+
### Test Code Quality
|
103
|
+
- Tests must be clear and readable
|
104
|
+
- Test names should describe what is being tested
|
105
|
+
- Tests should be independent and isolated
|
106
|
+
- Tests should be deterministic (no flaky tests)
|
107
|
+
- Tests should run quickly (especially unit tests)
|
108
|
+
|
109
|
+
### Test Coverage Requirements
|
110
|
+
- **100% line coverage** for all implemented functionality
|
111
|
+
- **Branch coverage** for all conditional logic
|
112
|
+
- **Edge case coverage** for error conditions
|
113
|
+
- **Integration coverage** for database operations
|
114
|
+
|
115
|
+
### Test Documentation
|
116
|
+
- Each test file should have a header explaining its purpose
|
117
|
+
- Complex test setups should be documented
|
118
|
+
- Test utilities should be well-documented
|
119
|
+
- Tests serve as executable documentation
|
120
|
+
|
121
|
+
## Tools and Utilities
|
122
|
+
|
123
|
+
### Mock Database Testing
|
124
|
+
```ruby
|
125
|
+
# Example of mock database testing for SQL generation
|
126
|
+
DB = Sequel.mock
|
127
|
+
dataset = DB[:users].where(name: 'John')
|
128
|
+
assert_equal "SELECT * FROM users WHERE (name = 'John')", dataset.sql
|
129
|
+
```
|
130
|
+
|
131
|
+
### Integration Testing Setup
|
132
|
+
```ruby
|
133
|
+
# Example of integration testing with real DuckDB
|
134
|
+
def setup
|
135
|
+
@db = Sequel.connect("duckdb::memory:")
|
136
|
+
@db.create_table(:test_table) do
|
137
|
+
primary_key :id
|
138
|
+
String :name
|
139
|
+
end
|
140
|
+
end
|
141
|
+
```
|
142
|
+
|
143
|
+
## Continuous Integration
|
144
|
+
|
145
|
+
### Test Execution
|
146
|
+
- All tests must pass before any code is merged
|
147
|
+
- Tests should be run on multiple Ruby versions
|
148
|
+
- Tests should be run on different operating systems
|
149
|
+
- Performance regression tests should be included
|
150
|
+
|
151
|
+
### Test Reporting
|
152
|
+
- Test results should be clearly reported
|
153
|
+
- Coverage reports should be generated
|
154
|
+
- Failed tests should provide clear error messages
|
155
|
+
- Test execution time should be monitored
|
156
|
+
|
157
|
+
## Common Testing Patterns
|
158
|
+
|
159
|
+
### Testing SQL Generation
|
160
|
+
```ruby
|
161
|
+
def test_select_with_where
|
162
|
+
dataset = @db[:users].where(name: 'John')
|
163
|
+
assert_equal "SELECT * FROM users WHERE (name = 'John')", dataset.sql
|
164
|
+
end
|
165
|
+
```
|
166
|
+
|
167
|
+
### Testing Database Operations
|
168
|
+
```ruby
|
169
|
+
def test_insert_and_select
|
170
|
+
@db[:users].insert(name: 'John', email: 'john@example.com')
|
171
|
+
user = @db[:users].where(name: 'John').first
|
172
|
+
assert_equal 'john@example.com', user[:email]
|
173
|
+
end
|
174
|
+
```
|
175
|
+
|
176
|
+
### Testing Error Conditions
|
177
|
+
```ruby
|
178
|
+
def test_connection_error
|
179
|
+
assert_raises(Sequel::DatabaseConnectionError) do
|
180
|
+
Sequel.connect("duckdb:/invalid/path/database.db")
|
181
|
+
end
|
182
|
+
end
|
183
|
+
```
|
184
|
+
|
185
|
+
## Remember: NO CODE WITHOUT TESTS
|
186
|
+
|
187
|
+
This is not optional. Every implementation task must begin with writing comprehensive tests. This ensures:
|
188
|
+
- Functionality works as expected
|
189
|
+
- Regressions are caught early
|
190
|
+
- Code is maintainable and refactorable
|
191
|
+
- Documentation through executable examples
|
192
|
+
- Confidence in the codebase
|
data/.rubocop.yml
ADDED
@@ -0,0 +1,103 @@
|
|
1
|
+
plugins:
|
2
|
+
- rubocop-minitest
|
3
|
+
- rubocop-rake
|
4
|
+
- rubocop-sequel
|
5
|
+
|
6
|
+
AllCops:
|
7
|
+
TargetRubyVersion: 3.1
|
8
|
+
NewCops: enable
|
9
|
+
|
10
|
+
Gemspec/DevelopmentDependencies:
|
11
|
+
Enabled: false
|
12
|
+
|
13
|
+
# Disable complexity violations for database adapter code
|
14
|
+
Metrics/ModuleLength:
|
15
|
+
Enabled: true
|
16
|
+
Exclude:
|
17
|
+
- lib/sequel/adapters/shared/duckdb.rb
|
18
|
+
|
19
|
+
Metrics/MethodLength:
|
20
|
+
Enabled: true
|
21
|
+
Exclude:
|
22
|
+
- lib/sequel/adapters/shared/duckdb.rb
|
23
|
+
- test/**/*
|
24
|
+
- test/type_test.rb
|
25
|
+
|
26
|
+
Metrics/ClassLength:
|
27
|
+
Enabled: true
|
28
|
+
Exclude:
|
29
|
+
- test/**/*
|
30
|
+
- test/type_test.rb
|
31
|
+
|
32
|
+
Metrics/BlockLength:
|
33
|
+
Enabled: true
|
34
|
+
Exclude:
|
35
|
+
- test/**/*
|
36
|
+
- test/type_test.rb
|
37
|
+
- sequel-duckdb.gemspec
|
38
|
+
|
39
|
+
Metrics/AbcSize:
|
40
|
+
Enabled: true
|
41
|
+
Exclude:
|
42
|
+
- lib/sequel/adapters/shared/duckdb.rb
|
43
|
+
- test/**/*
|
44
|
+
- test/type_test.rb
|
45
|
+
|
46
|
+
Metrics/CyclomaticComplexity:
|
47
|
+
Enabled: true
|
48
|
+
Exclude:
|
49
|
+
- lib/sequel/adapters/shared/duckdb.rb
|
50
|
+
- test/**/*
|
51
|
+
- test/type_test.rb
|
52
|
+
|
53
|
+
Metrics/PerceivedComplexity:
|
54
|
+
Enabled: true
|
55
|
+
Exclude:
|
56
|
+
- lib/sequel/adapters/shared/duckdb.rb
|
57
|
+
- test/**/*
|
58
|
+
- test/type_test.rb
|
59
|
+
|
60
|
+
# Disable line length for generated SQL and complex expressions
|
61
|
+
Layout/LineLength:
|
62
|
+
Enabled: true
|
63
|
+
Exclude:
|
64
|
+
- test/**/*
|
65
|
+
- test/type_test.rb
|
66
|
+
- sequel-duckdb.gemspec
|
67
|
+
|
68
|
+
Minitest/MultipleAssertions:
|
69
|
+
Enabled: false
|
70
|
+
|
71
|
+
# Allow duplicate branches for error handling patterns
|
72
|
+
Lint/DuplicateBranch:
|
73
|
+
Enabled: true
|
74
|
+
|
75
|
+
# Allow database adapter naming patterns
|
76
|
+
Naming/MethodParameterName:
|
77
|
+
Enabled: true
|
78
|
+
|
79
|
+
Naming/AccessorMethodName:
|
80
|
+
Enabled: true
|
81
|
+
|
82
|
+
Naming/VariableNumber:
|
83
|
+
Enabled: true
|
84
|
+
Exclude:
|
85
|
+
- test/**/*
|
86
|
+
|
87
|
+
# Allow database adapter patterns
|
88
|
+
Lint/EmptyBlock:
|
89
|
+
Enabled: true
|
90
|
+
Exclude:
|
91
|
+
- test/**/*
|
92
|
+
|
93
|
+
Lint/DuplicateMethods:
|
94
|
+
Enabled: true
|
95
|
+
|
96
|
+
Style/TrivialAccessors:
|
97
|
+
Enabled: true
|
98
|
+
|
99
|
+
Style/StringLiterals:
|
100
|
+
EnforcedStyle: double_quotes
|
101
|
+
|
102
|
+
Style/StringLiteralsInInterpolation:
|
103
|
+
EnforcedStyle: double_quotes
|