sequel-duckdb 0.1.0 → 0.2.0

data/plans/schema_management.md

@@ -0,0 +1,383 @@
+# Schema Management Implementation Plan
+
+## Overview
+
+Implement DuckDB schema management functionality in the sequel-duckdb adapter to support creating, dropping, and managing schemas. This plan addresses schema operations that are natively supported by DuckDB.
+
+## Background Research
+
+### DuckDB Schema Support
+
+DuckDB supports SQL schema management with the following capabilities:
+
+1. **CREATE SCHEMA**
+
+   - Basic: `CREATE SCHEMA s1;`
+   - Conditional: `CREATE SCHEMA IF NOT EXISTS s2;`
+   - Replace: `CREATE OR REPLACE SCHEMA s2;`
+   - Default schema is `main`
+
+2. **DROP SCHEMA**
+
+   - Basic: `DROP SCHEMA sch;`
+   - Conditional: `DROP SCHEMA IF EXISTS sch;`
+   - With cascade: `DROP SCHEMA sch CASCADE;`
+   - Dependency tracking for tables, views, functions, indexes, sequences, types, macros
+   - Note: Dependencies are NOT tracked for views (limitation)
+
+3. **Schema Usage**
+
+   - Tables can be created in schemas: `CREATE TABLE s1.t (id INTEGER);`
+   - Cross-schema queries are supported
+   - Schemas provide logical namespaces for organizing database objects
+
+### DuckDB Database Support
+
+**Important Finding**: DuckDB does NOT support traditional `CREATE DATABASE` or `DROP DATABASE` commands.
+
+Instead, DuckDB uses:
+
+- **ATTACH DATABASE**: Attach existing database files to current session
+  - `ATTACH 'file.db' AS alias;`
+  - `ATTACH 'file.db' (READ_ONLY);`
+  - Not persisted between sessions
+- **DETACH DATABASE**: Detach database files from current session
+- Database creation happens implicitly when opening a connection to a file path
+
+### Sequel Schema Support
+
+Sequel provides schema management methods (primarily in PostgreSQL adapter):
+
+1. **create_schema(name, opts={})**
+
+   - Creates a schema
+   - Options: `:if_not_exists`, `:owner` (PostgreSQL-specific)
+
+2. **create_schema_sql(name, opts={})**
+
+   - Generates CREATE SCHEMA SQL
+
+3. **drop_schema(name, opts={})**
+
+   - Drops a schema
+   - Calls `remove_all_cached_schemas` to clear caches
+   - Options: `:if_exists`, `:cascade`
+
+4. **drop_schema_sql(name, opts={})**
+
+   - Generates DROP SCHEMA SQL
+
+5. **rename_schema(name, new_name)** (PostgreSQL-specific)
+
+   - Renames a schema
+
+6. **remove_all_cached_schemas**
+
+   - Clears cached schema information
+   - Resets `@primary_keys`, `@primary_key_sequences`, `@schemas`
+
+## Implementation Plan
+
+### Phase 1: Core Schema Operations
+
+Implement basic schema creation and deletion in `DatabaseMethods` module:
+
+#### 1.1 `create_schema(name, opts={})`
+
+```ruby
+def create_schema(name, opts=OPTS)
+  self << create_schema_sql(name, opts)
+end
+```
+
+**Supported options:**
+
+- `:if_not_exists` - Add IF NOT EXISTS clause
+
+**Not supported (PostgreSQL-specific):**
+
+- `:owner` - DuckDB doesn't support AUTHORIZATION/OWNER syntax
+
+#### 1.2 `create_schema_sql(name, opts={})`
+
+```ruby
+def create_schema_sql(name, opts=OPTS)
+  sql = "CREATE"
+  sql += " OR REPLACE" if opts[:or_replace]
+  sql += " SCHEMA"
+  sql += " IF NOT EXISTS" if opts[:if_not_exists]
+  sql += " #{quote_identifier(name)}"
+  sql
+end
+```
+
+**DuckDB-specific considerations:**
+
+- Support both `IF NOT EXISTS` and `OR REPLACE`
+- Cannot use both simultaneously (mutually exclusive)
+
+#### 1.3 `drop_schema(name, opts={})`
+
+```ruby
+def drop_schema(name, opts=OPTS)
+  self << drop_schema_sql(name, opts)
+  remove_all_cached_schemas
+end
+```
+
+**Supported options:**
+
+- `:if_exists` - Add IF EXISTS clause
+- `:cascade` - Add CASCADE clause to drop dependent objects
+
+#### 1.4 `drop_schema_sql(name, opts={})`
+
+```ruby
+def drop_schema_sql(name, opts=OPTS)
+  sql = "DROP SCHEMA"
+  sql += " IF EXISTS" if opts[:if_exists]
+  sql += " #{quote_identifier(name)}"
+  sql += " CASCADE" if opts[:cascade]
+  sql
+end
+```
+
+#### 1.5 `remove_all_cached_schemas`
+
+```ruby
+def remove_all_cached_schemas
+  @schema_cache = {}
+  @schemas = {}
+  @primary_keys = {}
+  @primary_key_sequences = {}
+end
+```
+
+**Important**: All cache variables MUST be initialized as empty hashes (`{}`), not `nil`. Sequel's internal `remove_cached_schema` method expects `@schemas` to be a Hash and will call `.delete()` on it. Setting it to `nil` will cause a `NoMethodError` when `create_view` or other methods call `remove_cached_schema` internally.
+
+**Note**: DuckDB adapter uses `@schema_cache` for caching schema information (see line 505-510 in shared/duckdb.rb). Need to clear this when schemas are modified.
+
+### Phase 2: Schema Introspection
+
+Implement methods to list and inspect schemas:
+
+#### 2.1 `schemas(opts={})`
+
+```ruby
+def schemas(opts=OPTS)
+  sql = "SELECT schema_name FROM information_schema.schemata"
+  sql += " WHERE catalog_name = ?" if opts[:catalog]
+
+  schemas = []
+  execute(sql, opts[:catalog] ? [opts[:catalog]] : []) do |row|
+    schemas << row[:schema_name].to_sym
+  end
+
+  schemas
+end
+```
+
+#### 2.2 `schema_exists?(name, opts={})`
+
+```ruby
+def schema_exists?(name, opts=OPTS)
+  sql = "SELECT 1 FROM information_schema.schemata WHERE schema_name = ? LIMIT 1"
+
+  result = nil
+  execute(sql, [name.to_s]) do |_row|
+    result = true
+  end
+
+  !!result
+end
+```
+
+### Phase 3: Database File Operations (ATTACH/DETACH)
+
+**Decision**: Do NOT implement `create_database` / `drop_database` because:
+
+1. DuckDB doesn't support these SQL commands
+2. Database creation is implicit (happens when connecting to a file path)
+3. ATTACH/DETACH are session-specific, not DDL operations
+4. These operations are outside the scope of typical Sequel adapter functionality
+
+Instead, document that users should:
+
+- Use connection strings to create/open database files
+- Use ATTACH/DETACH via raw SQL if needed: `db.run("ATTACH 'other.db' AS other")`
+
+### Phase 4: Testing
+
+Create comprehensive tests for schema operations:
+
+#### 4.1 Schema Creation Tests
+
+- Test basic CREATE SCHEMA
+- Test CREATE SCHEMA IF NOT EXISTS
+- Test CREATE OR REPLACE SCHEMA
+- Test error handling for duplicate schemas (without IF NOT EXISTS)
+
+#### 4.2 Schema Deletion Tests
+
+- Test basic DROP SCHEMA (empty schema)
+- Test DROP SCHEMA IF EXISTS
+- Test DROP SCHEMA CASCADE (with dependent objects)
+- Test error handling for non-existent schemas (without IF EXISTS)
+- Test dependency error (dropping schema with objects, without CASCADE)
+
+#### 4.3 Schema Introspection Tests
+
+- Test listing all schemas
+- Test schema_exists? for existing and non-existing schemas
+- Test that default 'main' schema exists
+
+#### 4.4 Schema Usage Tests
+
+- Test creating tables in custom schemas
+- Test cross-schema queries
+- Test that schema cache is properly cleared after drop
+
+#### 4.5 Edge Cases
+
+- Test schema names that require quoting (with special characters)
+- Test schema names that are reserved words
+- Test mutually exclusive options (:if_not_exists vs :or_replace)
+
+### Phase 5: Documentation
+
+#### 5.1 Update README
+
+Add section on schema management:
+
+```markdown
+## Schema Management
+
+sequel-duckdb supports DuckDB's schema functionality:
+
+### Creating Schemas
+db.create_schema(:analytics)
+db.create_schema(:staging, if_not_exists: true)
+db.create_schema(:temp, or_replace: true)
+
+### Dropping Schemas
+db.drop_schema(:analytics)
+db.drop_schema(:staging, if_exists: true)
+db.drop_schema(:temp, cascade: true)  # Drops all objects in schema
+
+### Listing Schemas
+db.schemas  # => [:main, :analytics, :staging]
+
+### Using Schemas
+db.create_table(Sequel[:analytics][:sales]) do
+  primary_key :id
+  String :product
+  Numeric :amount
+end
+
+db[:analytics__sales].insert(product: 'Widget', amount: 99.99)
+```
+
+#### 5.2 Document Limitations
+
+- DuckDB does not support schema ownership/authorization
+- DuckDB does not support renaming schemas (no ALTER SCHEMA RENAME)
+- View dependencies are not tracked by DuckDB
+
+## Implementation Checklist
+
+- [x] Phase 1: Core schema operations
+
+  - [x] Implement `create_schema(name, opts={})`
+  - [x] Implement `create_schema_sql(name, opts={})`
+  - [x] Implement `drop_schema(name, opts={})`
+  - [x] Implement `drop_schema_sql(name, opts={})`
+  - [x] Implement `remove_all_cached_schemas`
+  - [x] Add validation for mutually exclusive options
+
+- [x] Phase 2: Schema introspection
+
+  - [x] Implement `schemas(opts={})`
+  - [x] Implement `schema_exists?(name, opts={})`
+
+- [x] Phase 3: Document ATTACH/DETACH approach
+
+  - [x] Add documentation for database file operations
+  - [x] Explain why create_database/drop_database are not implemented
+
+- [x] Phase 4: Testing
+
+  - [x] Write schema creation tests (6 tests)
+  - [x] Write schema deletion tests (6 tests)
+  - [x] Write schema introspection tests (5 tests)
+  - [x] Write schema usage tests (4 tests)
+  - [x] Write edge case tests (3 tests)
+  - [x] Write cache clearing tests (3 tests)
+  - [x] Add regression test for `remove_cached_schema` compatibility
+
+- [x] Phase 5: Documentation
+
+  - [x] Update README with schema management section
+  - [x] Document limitations
+  - [x] Add examples for common use cases
+  - [x] Document ATTACH/DETACH for multiple database files
+
+## Implementation Summary
+
+### Completed Features
+
+All planned features have been implemented and tested:
+
+1. **Core Schema Operations** (`lib/sequel/adapters/shared/duckdb.rb`)
+
+   - `create_schema(name, opts={})` - Lines 1191-1193
+   - `create_schema_sql(name, opts={})` - Lines 1202-1214
+   - `drop_schema(name, opts={})` - Lines 1227-1230
+   - `drop_schema_sql(name, opts={})` - Lines 1239-1245
+   - `remove_all_cached_schemas` - Lines 1257-1262
+
+2. **Schema Introspection** (`lib/sequel/adapters/shared/duckdb.rb`)
+
+   - `schemas(opts={})` - Lines 1270-1280
+   - `schema_exists?(name, opts={})` - Lines 1291-1300
+
+3. **Comprehensive Test Suite** (`test/schema_management_test.rb`)
+
+   - 38 tests covering all functionality
+   - All tests passing
+   - Regression tests to prevent cache-related bugs
+
+4. **Documentation** (`README.md`)
+
+   - Complete schema management section
+   - Usage examples
+   - Limitations documented
+   - ATTACH/DETACH examples
+
+### Bug Fixes
+
+- **Cache initialization bug**: Fixed `remove_all_cached_schemas` to initialize cache variables as empty hashes instead of `nil`. This prevents `NoMethodError` when Sequel's `remove_cached_schema` is called internally by methods like `create_view`.
+
+### Test Results
+
+- **Schema management tests**: 38 tests, 73 assertions, 0 failures
+- **Full test suite**: 539 tests, 2362 assertions, 0 failures
+
+### Files Modified
+
+- `lib/sequel/adapters/shared/duckdb.rb` - Added schema management methods
+- `test/schema_management_test.rb` - New comprehensive test file
+- `README.md` - Added schema management documentation section
+
+## Related Files
+
+- Implementation: `lib/sequel/adapters/shared/duckdb.rb` (DatabaseMethods module)
+- Tests: `spec/sequel/adapters/duckdb_spec.rb` or new `spec/schema_management_spec.rb`
+- Documentation: `README.md`
+
+## References
+
+- [DuckDB CREATE SCHEMA](https://duckdb.org/docs/stable/sql/statements/create_schema.html)
+- [DuckDB DROP Statement](https://duckdb.org/docs/stable/sql/statements/drop.html)
+- [DuckDB ATTACH/DETACH](https://duckdb.org/docs/stable/sql/statements/attach.html)
+- [Sequel PostgreSQL Adapter](https://github.com/jeremyevans/sequel/blob/master/lib/sequel/adapters/shared/postgres.rb)
+- [Sequel Schema Modification Docs](https://github.com/jeremyevans/sequel/blob/master/doc/schema_modification.rdoc)

data/release-please-config.json

@@ -0,0 +1,14 @@
+{
+  "$schema": "https://raw.githubusercontent.com/googleapis/release-please/main/schemas/config.json",
+  "release-type": "ruby",
+  "include-component-in-tag": false,
+  "bump-minor-pre-major": true,
+  "bump-patch-for-minor-pre-major": true,
+  "bootstrap-sha": "8c6792de032d5ba5f54fcd8e3672b59c250039e2",
+  "packages": {
+    ".": {
+      "package-name": "sequel-duckdb",
+      "version-file": "lib/sequel/duckdb/version.rb"
+    }
+  }
+}

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: sequel-duckdb
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - Ryan Duryea
@@ -9,26 +9,6 @@ bindir: exe
 cert_chain: []
 date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
-- !ruby/object:Gem::Dependency
-  name: duckdb
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '1.0'
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 1.0.0
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '1.0'
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 1.0.0
 - !ruby/object:Gem::Dependency
   name: sequel
   requirement: !ruby/object:Gem::Requirement
@@ -83,14 +63,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '5.0'
+        version: '6.0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '5.0'
+        version: '6.0'
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
@@ -125,14 +105,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.38.2
+        version: 0.40.0
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.38.2
+        version: 0.40.0
 - !ruby/object:Gem::Dependency
   name: rubocop-rake
   requirement: !ruby/object:Gem::Requirement
@@ -161,6 +141,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: 0.4.1
+- !ruby/object:Gem::Dependency
+  name: simplecov
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.0'
 description: A Ruby gem that provides a complete database adapter for the Sequel toolkit
   to work with DuckDB, enabling Ruby applications to connect to and interact with
   DuckDB databases through Sequel's comprehensive ORM and database abstraction interface.
@@ -170,6 +164,16 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
+- ".beads/.beads-credential-key"
+- ".beads/.gitignore"
+- ".beads/README.md"
+- ".beads/config.yaml"
+- ".beads/hooks/post-checkout"
+- ".beads/hooks/post-merge"
+- ".beads/hooks/pre-commit"
+- ".beads/hooks/pre-push"
+- ".beads/hooks/prepare-commit-msg"
+- ".beads/metadata.json"
 - ".kiro/specs/advanced-sql-features-implementation/design.md"
 - ".kiro/specs/advanced-sql-features-implementation/requirements.md"
 - ".kiro/specs/advanced-sql-features-implementation/tasks.md"
@@ -190,22 +194,40 @@ files:
 - ".kiro/steering/structure.md"
 - ".kiro/steering/tech.md"
 - ".kiro/steering/testing.md"
+- ".mdformat.toml"
+- ".release-please-manifest.json"
 - ".rubocop.yml"
+- ".rubocop_todo.yml"
 - ".yardopts"
+- AGENTS.md
 - API_DOCUMENTATION.md
 - CHANGELOG.md
+- FINAL_STATUS.md
 - LICENSE
 - MIGRATION_EXAMPLES.md
 - PERFORMANCE_OPTIMIZATIONS.md
 - README.md
+- REFACTORING_SUMMARY.md
 - Rakefile
 - TASK_10.2_IMPLEMENTATION_SUMMARY.md
 - docs/DUCKDB_SQL_PATTERNS.md
 - docs/TASK_12_VERIFICATION_SUMMARY.md
+- justfile
 - lib/sequel/adapters/duckdb.rb
 - lib/sequel/adapters/shared/duckdb.rb
 - lib/sequel/duckdb.rb
+- lib/sequel/duckdb/helpers/copier.rb
+- lib/sequel/duckdb/helpers/pathifier.rb
 - lib/sequel/duckdb/version.rb
+- plans/date_arithmetic.md
+- plans/engineering/Sequel.md
+- plans/engineering/duckdb.md
+- plans/engineering/sqlite.md
+- plans/mock_connection_bug.md
+- plans/mock_without_driver_gem.md
+- plans/over_engineering_analysis.md
+- plans/schema_management.md
+- release-please-config.json
 - sig/sequel/duckdb.rbs
 homepage: https://github.com/sequel/sequel-duckdb
 licenses:
@@ -222,14 +244,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: 3.1.0
+      version: 3.2.0
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.6.7
+rubygems_version: 4.0.20
 specification_version: 4
 summary: Sequel database adapter for DuckDB
 test_files: []