dami 1.0.0

data/docs/02.Models_and_Fields.md

@@ -0,0 +1,244 @@
+# Chapter 2: The Four Pillars of a Model
+
+In Dami, a "model" is more than just a schema definition; it's a complete representation of a business entity. To keep your code clean, scalable, and easy to reason about, Dami organizes this representation into **four** distinct, purpose-built pillars.
+
+This architectural pattern solves the "fat model" problem, where data structure, business logic, presentation logic, and querying get tangled together. By separating these concerns, Dami guides you toward a cleaner, more maintainable architecture from the start.
+
+The four pillars are:
+
+1.  **`Dami.model`**: The **Structure** - Defines the schema, fields, and relationships.
+2.  **`Dami.behavior`**: The **Write-Side Logic** - Defines validations and protection rules.
+3.  **`Dami.present`**: The **Public Interface** - Defines read-side helpers and computed properties.
+4.  **`Dami.scopes`**: The **Collection Queries** - Defines reusable query shortcuts.
+
+**Enforced by Design:** Dami enforces this separation at the DSL level. Attempting to define validations inside `Dami.model` raises `Dami::InvalidDSLError` with a helpful message pointing you to the correct location. No documentation to memorize—the framework guides you.
+
+-----
+
+## Pillar 1: `Dami.model` - The Structure
+
+The `Dami.model` block is exclusively for defining the **structure** of your data. Its single responsibility is to describe your table's schema and its relationships to other tables.
+
+```ruby
+Dami.model :users do
+  database :default
+  fields do
+    field :first_name, :string
+    field :last_name, :string
+    field :email, :string
+  end
+  relationships do
+    has_many :posts
+  end
+end
+```
+
+### Field Types
+
+Dami supports all common SQL types.
+
+| Type(s) | Generated SQL Type |
+| :--- | :--- |
+| `:string`, `:text` | `TEXT` |
+| `:integer`, `:boolean` | `INTEGER` |
+| `:decimal`, `:float` | `REAL` |
+| `:datetime`, `:date`, `:time` | `DATETIME` |
+| `:json` | `TEXT` (auto-serialized) |
+
+Two field names are special: if a model declares `created_at` and/or `updated_at` (`:datetime`), Dami fills them automatically — both on create, `updated_at` on every update — unless you pass a value yourself. Dates and times are stored as UTC strings and come back as strings.
+
+### Virtual Fields
+
+Virtual fields are defined in your `Dami.model` block but do not have a corresponding column in the database. They are perfect for form data like password confirmations or "terms accepted" checkboxes.
+
+```ruby
+Dami.model :users do
+  fields do
+    field :email, :string
+    field :password_digest, :string
+  end
+  virtual do
+    field :password, :string
+    field :password_confirmation, :string
+  end
+end
+```
+
+Virtual fields are accepted as input and can be used in validations, but they are stripped out before data is sent to the database.
+
+-----
+
+## Pillar 2: `Dami.behavior` - The Write-Side Logic
+
+The `Dami.behavior` block defines all **write-side rules**. Its job is to ensure data integrity and security whenever a record is created or updated.
+
+### Validations
+
+Validations guard write operations and ensure data quality:
+
+```ruby
+Dami.behavior :users do
+  validate do
+    rule :email, :required, :email
+    rule :first_name, :required
+  end
+end
+```
+
+Validations are covered in depth in Chapter 5.
+
+### Protection Rules
+
+Protection prevents mass-assignment vulnerabilities by blocking certain fields from being set via `create` or `update`:
+
+```ruby
+Dami.behavior :users do
+  protection do
+    protect :role        # Cannot be mass-assigned
+    permit :status       # Explicitly allowed
+  end
+end
+```
+
+```ruby
+# This will raise Dami::ProtectionError
+Dami.db(:users).create(
+  first_name: 'Alice',
+  role: 'admin'  # ❌ Protected field
+)
+
+# This works - role is explicitly permitted
+Dami.db(:users).create(
+  first_name: 'Alice',
+  role: 'admin'
+, permit: [:role])
+
+# Status is always allowed (in permit list)
+Dami.db(:users).create(
+  first_name: 'Alice',
+  status: 'active'  # ✅ Permitted field
+)
+```
+
+**Protection Options:**
+
+  - `protect :field` - Block field from mass-assignment
+  - `permit :field` - Allow field (overrides model-level protection)
+  - `permit: [:field]` - Runtime override in `create`/`update`
+  - `protect: false` - Disable all protection for the operation
+
+-----
+
+## Pillar 3: `Dami.present` - The Public Interface
+
+This is the dedicated pillar for all **read-side logic** and presentation. It's where you define computed properties, formatting methods, and any other helpers that don't belong in the database. This keeps your data structure clean while providing a rich, helpful interface for your records.
+
+```ruby
+Dami.present :users do
+  def full_name
+    "#{self[:first_name]} #{self[:last_name]}"
+  end
+  
+  def greeting
+    "Hello, #{full_name}!"
+  end
+
+  def is_admin?
+    self[:role] == 'admin'
+  end
+end
+```
+
+These methods are now available on any `RecordProxy` instance for that model:
+
+```ruby
+user = Dami.db(:users).find(1)
+puts user.full_name  # => "Alice Smith"
+puts user.greeting   # => "Hello, Alice Smith!"
+```
+
+**Field Access: Hash vs Method Style**
+
+  - **Hash style** (`user[:field]`): Direct, raw access to database column data.
+  - **Method style** (`user.field`): Will first look for a defined method in your `Dami.present` block. If no method is found, it falls back to hash access (`self[:field]`).
+
+-----
+
+## Pillar 4: `Dami.scopes` - Collection Queries
+
+The `Dami.scopes` block defines reusable query shortcuts for filtering collections:
+
+```ruby
+Dami.scopes :users do
+  scope :active, -> { where(status: 'active') }
+  scope :admins, -> { where(role: 'admin') }
+  scope :recent, -> { where(created_at: { gt: Time.now - 86400 }) }
+end
+```
+
+```ruby
+# Use scopes like methods
+active_users = Dami.db(:users).active.to_a
+recent_admins = Dami.db(:users).admins.recent.to_a
+```
+
+Scopes are covered in detail in Chapter 8.
+
+-----
+
+## The Full Picture: A Cohesive Model File
+
+For any non-trivial application, you can organize all four pillars for a single entity into one file to see the complete picture:
+
+```ruby
+# models/user.rb
+
+# 1. Structure
+Dami.model :users do
+  fields do
+    field :first_name, :string
+    field :last_name, :string
+    field :email, :string
+    field :status, :string
+    field :role, :string
+  end
+  relationships { has_many :posts }
+end
+
+# 2. Behavior (Write-Side)
+Dami.behavior :users do
+  validate do
+    rule :email, :required, :email
+  _ rule :status, inclusion: %w[active inactive]
+  end
+  protection do
+    protect :role
+    permit :status
+  end
+end
+
+# 3. Presentation (Read-Side)
+Dami.present :users do
+  def full_name
+    "#{self[:first_name]} #{self[:last_name]}"
+  end
+
+  def is_admin?
+    self[:role] == 'admin'
+  end
+end
+
+# 4. Querying (Collection-Side)
+Dami.scopes :users do
+  scope :active, -> { where(status: 'active') }
+  scope :admins, -> { where(role: 'admin') }
+end
+```
+
+## What's Next?
+
+  - **Chapter 3: Querying** - Find, filter, and retrieve data using the query builder.
+  - **Chapter 4: Creating, Updating, and Deleting** - Modify your data.
+  - **Chapter 5: Validation** - Comprehensive guide to the validation DSL.
+  - **Chapter 7: Associations** - Connect models with relationships.
+  - **Chapter 8: Scopes** - Deep dive into reusable queries.

data/docs/03.Querying.md

@@ -0,0 +1,387 @@
+# Chapter 3: Querying
+
+Dami provides a powerful, chainable DSL for building SQL queries. All query methods are **lazy**, meaning they don't execute a database query until you explicitly ask for the results.
+
+## Basic Queries
+
+### Finding by ID
+
+The quickest way to retrieve a single record is by its primary key using `.find`.
+
+```ruby
+user = Dami.db(:users).find(1)
+user[:name]  # => "Alice"
+
+# Returns nil if not found
+user = Dami.db(:users).find(999)  # => nil
+```
+
+### Where Clauses
+
+You can filter records using `.where`. Chaining multiple `.where` calls will combine them with an `AND` operator.
+
+```ruby
+# Single condition
+users = Dami.db(:users).where(status: 'active').to_a
+
+# Multiple conditions (implicit AND)
+users = Dami.db(:users).where(status: 'active', role: 'admin').to_a
+
+# Chained conditions (implicit AND)
+users = Dami.db(:users)
+  .where(status: 'active')
+  .where('age > ?', 30) # You can also use raw SQL fragments
+  .to_a
+```
+
+### OR Conditions
+
+To combine conditions with `OR`, you can use the chainable `.or` method or a block for more complex grouping.
+
+```ruby
+# Simple OR
+# Find users who are admins OR active
+users = Dami.db(:users).where(role: 'admin').or(status: 'active')
+# => WHERE role = 'admin' OR status = 'active'
+
+# 💡 Grouped Conditions with Blocks
+# Find users where (name is 'Bob') AND (age is 30 OR status is 'pending')
+users = Dami.db(:users).where(name: 'Bob').where do |q|
+  q.where(age: 30).or(status: 'pending')
+end
+# => WHERE name = 'Bob' AND (age = 30 OR status = 'pending')
+```
+
+## Operators
+
+For more complex comparisons, `where` accepts a hash with operators.
+
+### Table of Operators
+
+Here is a quick reference for all available operators:
+
+| Operator      | Example                               | Generated SQL            |
+|---------------|---------------------------------------|--------------------------|
+| `gt`          | `{ age: { gt: 18 } }`                 | `age > 18`               |
+| `lt`          | `{ age: { lt: 65 } }`                 | `age < 65`               |
+| `gte`         | `{ age: { gte: 18 } }`                | `age >= 18`              |
+| `lte`         | `{ age: { lte: 65 } }`                | `age <= 65`              |
+| `not`         | `{ status: { not: 'deleted' } }`      | `status != 'deleted'`    |
+| `in`          | `{ id: [1, 2] }`                      | `id IN (1, 2)`           |
+| `not_in`      | `{ role: { not_in: ['a', 'b'] } }`    | `role NOT IN ('a', 'b')` |
+| `starts_with` | `{ name: { starts_with: 'A' } }`      | `name LIKE 'A%'`         |
+| `ends_with`   | `{ name: { ends_with: 'e' } }`        | `name LIKE '%e'`         |
+| `contains`    | `{ name: { contains: 'ice' } }`       | `name LIKE '%ice%'`      |
+
+**Note:** Dami is smart enough to infer an `IN` query without specifying `in:`
+
+```
+# Find users by a list of IDs. Dami automatically uses an IN clause.
+users = Dami.db(:users).where(id: [1, 5, 10])
+```
+
+
+### String Matching and `NOT IN`
+
+```ruby
+# Find users whose names start with 'A'
+users = Dami.db(:users).where(name: { starts_with: 'A' })
+
+# Find users whose names contain 'ice'
+users = Dami.db(:users).where(name: { contains: 'ice' })
+
+# Find all users who are NOT admins or moderators
+users = Dami.db(:users).where(role: { not_in: ['admin', 'moderator'] })
+```
+
+## Ordering and Limiting
+
+### Ordering
+
+Sort your results using `.order`.
+
+```ruby
+# Ascending (default)
+users = Dami.db(:users).order(:name).to_a
+
+# Descending
+users = Dami.db(:users).order(name: :desc).to_a
+
+# Multiple columns
+users = Dami.db(:users).order(status: :asc, created_at: :desc).to_a
+```
+
+### Limit and Offset
+
+Use `.limit` and `.offset` for pagination.
+
+```ruby
+# Get the first 10 users
+users = Dami.db(:users).limit(10).to_a
+
+# Skip the first 20 and get the next 10
+users = Dami.db(:users).order(:id).limit(10).offset(20).to_a
+```
+
+### First and Last
+
+Quickly retrieve a single record.
+
+```ruby
+# Get the first user (ordered by primary key by default)
+user = Dami.db(:users).first
+
+# Get the last active user by creation date
+user = Dami.db(:users).where(status: 'active').order(:created_at).last
+```
+
+## Joins
+
+Combine data from multiple tables using joins.
+
+### Inner Joins
+
+Use `.join` to get records that have matching data in another table.
+
+```ruby
+# Get posts that have an associated user
+posts = Dami.db(:posts).join(:users, { id: :user_id })
+```
+
+[Image of two overlapping circles]
+
+### Left Joins
+
+Use `.left_join` to get all records from the first table, even if they don't have a match in the second.
+
+```ruby
+# Find all users and their post titles, including users who have not posted anything
+users_with_posts = Dami.db(:users)
+  .left_join(:posts, { id: :user_id })
+  .select('users.name', 'posts.title AS post_title')
+  .to_a
+
+# A user with no posts will appear in the results with `post_title: nil`
+```
+
+## Advanced Features
+
+### Selecting Columns
+
+By default, Dami selects all columns (`SELECT *`). Use `.select` to be more efficient.
+
+```ruby
+# Only get the id and email columns
+users = Dami.db(:users).select(:id, :email).to_a
+
+# Use aliases with joins
+posts = Dami.db(:posts)
+  .join(:users, { id: :user_id })
+  .select('posts.title', 'users.name AS author_name')
+  .to_a
+```
+
+### Counting and Existence Checks
+
+Check the number of records or if any exist.
+
+```ruby
+# Get the total count
+total = Dami.db(:users).count
+
+# Get a conditional count
+active_admins = Dami.db(:users).where(status: 'active', role: 'admin').count
+
+# Performant existence checks (often faster than `.count > 0`)
+if Dami.db(:users).where(status: 'pending').any?
+  puts "There are pending users to review."
+end
+
+# `exists?` is a convenient alias
+if Dami.db(:users).exists?(status: 'pending')
+  puts "There are pending users to review."
+end
+```
+
+### Batch Processing
+
+For large datasets, process records in batches to avoid high memory usage.
+
+```ruby
+# Process one user at a time
+Dami.db(:users).find_each(batch_size: 1000) do |user|
+  send_email(user)
+end
+
+# Process 1000 users at a time
+Dami.db(:users).find_in_batches(batch_size: 1000) do |batch_of_users|
+  send_bulk_notifications(batch_of_users)
+end
+```
+
+## Query Execution
+
+Queries are **lazy**. They are only sent to the database when you ask for the data.
+
+✅ A query will be executed when you call any of these methods:
+
+  - `.to_a` (or its alias `.all`)
+  - `.first`
+  - `.last`
+  - `.count`
+  - `.any?` / `.exists?`
+  - `.each`
+  - `.find_each`
+  - `.find_in_batches`
+
+This allows you to build complex queries step-by-step without hitting the database until you're ready.
+
+```ruby
+# No database query has been run yet
+query = Dami.db(:users).where(status: 'active')
+
+# Now the query executes
+users = query.to_a
+```
+
+## Best Practices
+
+Following these guidelines will help you write efficient, readable, and scalable queries.
+
+### 1\. Let the Database Do the Work
+
+Always use `where`, `or`, and `order` to filter and sort your data at the database level. Avoid loading large, unfiltered collections into memory and processing them with Ruby.
+
+```ruby
+# ✅ Good: The database does all the filtering and sorting.
+recent_admins = Dami.db(:users)
+  .where(status: 'active', role: 'admin')
+  .order(created_at: :desc)
+  .limit(10)
+
+# ❌ Avoid: Loading all users into memory to filter with Ruby.
+all_users = Dami.db(:users).to_a
+recent_admins = all_users
+  .select { |u| u[:status] == 'active' && u[:role] == 'admin' }
+  .sort_by { |u| u[:created_at] }
+  .reverse
+  .take(10)
+```
+
+### 2\. Select Only What You Need
+
+For queries that return many records, use `.select` to specify only the columns you need. This reduces the amount of data transferred from the database and lowers memory usage.
+
+```ruby
+# ✅ Good: Only loads id and email for a mailing list.
+mailing_list = Dami.db(:users).select(:id, :email).where(subscribed: true)
+
+# ❌ Avoid: Loading entire user objects when you only need one field.
+users = Dami.db(:users).where(subscribed: true).to_a
+emails = users.map { |u| u[:email] }
+```
+
+### 3\. Use Batch Processing for Large Operations
+
+When you need to iterate over thousands or millions of records, never load them all at once. Use `.find_each` or `.find_in_batches` to process them in manageable chunks.
+
+```ruby
+# ✅ Good: Processes 1000 users at a time with low memory.
+Dami.db(:users).find_each(batch_size: 1000) do |user|
+  user.send_notification!
+end
+
+# ❌ Avoid: Loading the entire table into memory. This will crash on large tables.
+all_users = Dami.db(:users).to_a # This could be millions of records!
+all_users.each do |user|
+  user.send_notification!
+end
+```
+
+### 4\. Build Complex Logic in SQL
+
+Leverage the full power of the DSL—including block-based `where`, `or`, and `joins`—to express complex business logic. This is almost always more performant than writing complicated Ruby logic to combine the results of multiple simple queries.
+
+```ruby
+# ✅ Good: One query to find admins OR active users over 30.
+users = Dami.db(:users).where(role: 'admin').or do |q|
+  q.where(status: 'active').where(age: { gt: 30 })
+end
+
+# ❌ Avoid: Running multiple queries and combining them in Ruby.
+admins = Dami.db(:users).where(role: 'admin').to_a
+active_users = Dami.db(:users).where(status: 'active', age: { gt: 30 }).to_a
+users = (admins + active_users).uniq
+```
+
+-----
+
+## Quick Reference
+
+A concise summary of the Dami query DSL.
+
+### Finding Records
+
+| Method         | Description                                     |
+|----------------|-------------------------------------------------|
+| `.find(id)`    | Retrieves a single record by its primary key.   |
+| `.first`       | Retrieves the first record matching the query.  |
+| `.last`        | Retrieves the last record matching the query.   |
+| `.to_a`, `.all`| Executes the query and returns an array of records. |
+
+### Building Conditions
+
+| Method                 | Example                                     |
+|------------------------|---------------------------------------------|
+| `.where(hash)`         | `where(status: 'active', role: 'admin')`    |
+| `.or(hash)`            | `where(status: 'active').or(status: 'new')` |
+| `.where { \|q\| ... }` | `where(name: 'A').where { \|q\| q.where(age: 30).or(status: 'B') }` |
+
+### Where Operators
+
+| Operator        | Example Usage                              |
+|-----------------|--------------------------------------------|
+| `gt`, `lt`, `gte`, `lte` | `where(age: { gt: 18, lte: 65 })`         |
+| `not`           | `where(status: { not: 'archived' })`       |
+| `in`            | `where(id: [1, 5, 10])`                     |
+| `not_in`        | `where(role: { not_in: ['admin', 'owner'] })` |
+| `starts_with`   | `where(name: { starts_with: 'A' })`        |
+| `contains`      | `where(name: { contains: 'ice' })`         |
+| `ends_with`     | `where(name: { ends_with: 'z' })`          |
+
+### Ordering & Limiting
+
+| Method         | Example                               |
+|----------------|---------------------------------------|
+| `.order()`     | `order(:name)`, `order(name: :desc)` |
+| `.limit()`     | `limit(10)`                           |
+| `.offset()`    | `offset(20)`                          |
+
+### Joins & Selection
+
+| Method         | Example                                     |
+|----------------|---------------------------------------------|
+| `.join()`      | `join(:posts, { id: :user_id })`            |
+| `.left_join()` | `left_join(:profiles, { id: :user_id })`    |
+| `.select()`    | `select(:id, :name)`, `select('users.*')` |
+
+### Execution & Counting
+
+| Method                  | Description                                            |
+|-------------------------|--------------------------------------------------------|
+| `.each { \|rec\| ... }` | Executes the query and yields each record.             |
+| `.count`                | Executes a `COUNT` query for the conditions.           |
+| `.any?`, `.exists?`     | Executes a `LIMIT 1` query to check for existence.     |
+
+### Batch Processing
+
+| Method                   | Description                                    |
+|--------------------------|------------------------------------------------|
+| `.find_each(batch_size:)`  | Yields each record one-by-one, in batches.     |
+| `.find_in_batches(batch_size:)` | Yields an array of records for each batch.     |
+
+## What's Next?
+
+- **Chapter 4: Creating & Updating** - Modify your data
+- **Chapter 5: Validation** - Ensure data quality
+- **Chapter 7: Associations** - Query related records easily