boto3-assist 0.28.0__tar.gz → 0.30.0__tar.gz

{boto3_assist-0.28.0 → boto3_assist-0.30.0}/.gitignore

@@ -171,4 +171,5 @@ cython_debug/
 
 activate.sh
 .pypirc
-.DS_Store
+.DS_Store
+.prompts

boto3_assist-0.30.0/.windsurf/rules/cascade.yaml

@@ -0,0 +1,11 @@
+rules:
+  - name: Defining Models
+    path: docs/defining-models.md
+    description: Model layer design patterns
+  - name: Design Patterns
+    path: docs/design-patterns.md
+    description: Design patterns for building scalable SaaS applications
+  - name: Designing Services
+    path: docs/designing-services.md
+    description: Service layer design patterns
+    

{boto3_assist-0.28.0 → boto3_assist-0.30.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: boto3_assist
-Version: 0.28.0
+Version: 0.30.0
 Summary: Additional boto3 wrappers to make your life a little easier
 Author-email: Eric Wilson <boto3-assist@geekcafe.com>
 License-File: LICENSE-EXPLAINED.txt

boto3_assist-0.30.0/docs/defining-models.md

@@ -0,0 +1,237 @@
+# Defining DynamoDB Models with boto3-assist
+
+This guide explains how to create and define DynamoDB models using the `DynamoDBModelBase` class provided by `boto3-assist`. Following these patterns ensures consistency, scalability, and easy integration with the `boto3-assist` ecosystem.
+
+## 1. Introduction to DynamoDBModelBase
+
+The `DynamoDBModelBase` is the foundation for all DynamoDB models in this framework. It provides a rich set of features out of the box, including:
+
+-   **Automatic Serialization**: Convert model instances to and from DynamoDB-compatible dictionaries.
+-   **Index Management**: A structured way to define primary keys and global secondary indexes (GSIs).
+-   **Data Mapping**: Easily map raw DynamoDB response data to your model instances.
+-   **Helper Utilities**: Access to utility functions for common tasks like timestamp conversion and UUID generation.
+
+## 2. Creating a Basic Model
+
+All models should inherit from `DynamoDBModelBase`. Here is an example of a simple `Product` model:
+
+```python
+import datetime
+from typing import Optional
+from boto3_assist.dynamodb.dynamodb_model_base import DynamoDBModelBase
+from boto3_assist.dynamodb.dynamodb_index import DynamoDBIndex, DynamoDBKey
+
+class Product(DynamoDBModelBase):
+    def __init__(
+        self,
+        id: Optional[str] = None,
+        name: Optional[str] = None,
+        price: float = 0.0,
+        description: Optional[str] = None,
+        sku: Optional[str] = None,
+    ):
+        super().__init__()
+
+        self.id: Optional[str] = id
+        self.name: Optional[str] = name
+        self.price: float = price
+        self.description: Optional[str] = description
+        self.sku: Optional[str] = sku
+
+        # Initialize the indexes
+        self._setup_indexes()
+
+    def _setup_indexes(self):
+        # Index definitions will go here
+        pass
+```
+
+**Key Principles**:
+
+-   **Inheritance**: Your model must inherit from `DynamoDBModelBase`.
+-   **Constructor**: Define your model's attributes in the `__init__` method. Call `super().__init__()` at the beginning.
+-   **Index Setup**: Call a private method (e.g., `_setup_indexes()`) at the end of the constructor to define your keys and indexes.
+
+## 3. Setting Up Indexes
+
+DynamoDB keys and indexes are defined within the `_setup_indexes` method using the `DynamoDBIndex` and `DynamoDBKey` classes.
+
+### Primary Key
+
+Every model must have a primary key. The primary key is defined as a `DynamoDBIndex` and added to the model's `indexes` collection.
+
+```python
+def _setup_indexes(self):
+    primary = DynamoDBIndex()
+    primary.name = "primary"
+    primary.partition_key.attribute_name = "pk"
+    primary.partition_key.value = lambda: DynamoDBKey.build_key(("product", self.id))
+    primary.sort_key.attribute_name = "sk"
+    primary.sort_key.value = lambda: DynamoDBKey.build_key(("product", self.id))
+    self.indexes.add_primary(primary)
+```
+
+-   `DynamoDBIndex()`: Creates a new index definition.
+-   `partition_key` and `sort_key`: Define the attributes and values for your keys.
+-   `attribute_name`: The name of the attribute in the DynamoDB table (e.g., `pk`, `sk`).
+-   `value`: A **lambda function** that dynamically generates the key value. Using a lambda is crucial for ensuring the key is generated with the most current attribute values.
+-   `DynamoDBKey.build_key()`: A helper to construct composite keys with a consistent separator.
+-   `self.indexes.add_primary()`: Registers the index as the primary key.
+
+### Global Secondary Indexes (GSIs)
+
+You can add GSIs to support additional query patterns. GSIs are also defined as `DynamoDBIndex` objects and added using `add_secondary()`.
+
+Here’s how to add a GSI to query all products by name:
+
+```python
+# Inside _setup_indexes method
+
+self.indexes.add_secondary(
+    DynamoDBIndex(
+        index_name="gsi0",
+        partition_key=DynamoDBKey(
+            attribute_name="gsi0_pk",
+            # Use a static value for the partition key to query all products
+            value=lambda: DynamoDBKey.build_key(("products", ""))
+        ),
+        sort_key=DynamoDBKey(
+            attribute_name="gsi0_sk",
+            value=lambda: DynamoDBKey.build_key(("name", self.name))
+        ),
+    )
+)
+```
+
+-   `index_name`: The name of the GSI in your DynamoDB table (e.g., `gsi0`).
+-   `add_secondary()`: Registers the index as a GSI.
+
+### Advanced GSI Patterns
+
+Your models can support more complex query patterns by combining static partition keys with simple or composite sort keys.
+
+#### Querying All Items with a Composite Sort Key
+
+This pattern is useful when you want to retrieve all items of a specific type and sort them by multiple fields. For example, to get all users sorted by `last_name` and then `first_name`.
+
+```python
+# Inside _setup_indexes for a UserModel
+
+# GSI to list all users, sorted by last name, then first name
+gsi_by_lastname = DynamoDBIndex(
+    index_name="gsi2",
+    partition_key=DynamoDBKey(
+        attribute_name="gsi2_pk",
+        # Static partition key to group all users together
+        value=lambda: DynamoDBKey.build_key(("users", None))
+    ),
+    sort_key=DynamoDBKey(
+        attribute_name="gsi2_sk",
+        # Composite sort key
+        value=lambda: DynamoDBKey.build_key(
+            ("lastname", self.last_name), ("firstname", self.first_name)
+        )
+    )
+)
+self.indexes.add_secondary(gsi_by_lastname)
+```
+
+**How it works**:
+-   **Static Partition Key**: `DynamoDBKey.build_key(("users", None))` generates a static partition key (e.g., `"users"`). This forces all user items into the same item collection within the GSI, allowing you to query all of them at once.
+-   **Composite Sort Key**: `DynamoDBKey.build_key(("lastname", self.last_name), ("firstname", self.first_name))` creates a sort key like `lastname#Smith#firstname#John`. This enables lexicographical sorting, first by last name and then by first name.
+
+
+### What the Generated Keys Look Like
+
+It's helpful to visualize what these key definitions produce. Given a `Product` instance:
+
+```python
+product = Product(id='abc-123', name='Mjolnir')
+```
+
+When you serialize this model using `to_resource_dictionary()`, the `boto3-assist` framework will generate the following key attributes based on the `lambda` functions in your index setup:
+
+-   **`pk`**: `product#abc-123`
+-   **`sk`**: `product#abc-123`
+-   **`gsi0_pk`**: `products`
+-   **`gsi0_sk`**: `name#Mjolnir`
+
+Your final item in DynamoDB would look something like this:
+
+```json
+{
+  "pk": "product#abc-123",
+  "sk": "product#abc-123",
+  "gsi0_pk": "products",
+  "gsi0_sk": "name#Mjolnir",
+  "id": "abc-123",
+  "name": "Mjolnir",
+  "price": 0.0,
+  "description": null,
+  "sku": null
+}
+```
+
+This structure allows you to:
+-   Fetch the product directly using its `pk` and `sk`.
+-   Query all products on `gsi0` (where `gsi0_pk` is `"products"`) and sort them by name (`gsi0_sk`).
+
+## 4. Serialization and Deserialization
+
+`DynamoDBModelBase` provides powerful methods for serialization (Python object to dictionary) and deserialization (dictionary to Python object).
+
+### Deserialization with `map()`
+
+The `map()` method is the primary way to populate a model instance from a dictionary. It intelligently handles various DynamoDB response formats.
+
+```python
+# Raw DynamoDB item
+dynamodb_item = {
+    'pk': {'S': 'product#123'},
+    'sk': {'S': 'product#123'},
+    'name': {'S': 'Mjolnir'},
+    'price': {'N': '9999.99'}
+}
+
+# Create an empty model and map the data
+product = Product().map(dynamodb_item)
+
+print(product.name)  # Output: Mjolnir
+print(product.price) # Output: 9999.99
+```
+
+As noted in a previous session, the `map()` method can handle:
+-   Full DynamoDB responses: `{'Item': {...}, 'ResponseMetadata': {...}}`
+-   Item-only responses: `{'Item': {...}}`
+-   Plain dictionaries.
+
+### Serialization
+
+There are several methods to convert a model instance to a dictionary:
+
+-   `to_dictionary()`: Returns a dictionary of the model's attributes, excluding any index attributes. This is useful for general-purpose serialization.
+-   `to_resource_dictionary()`: Returns a dictionary suitable for the Boto3 DynamoDB **Resource** API, including index attributes.
+-   `to_client_dictionary()`: Returns a dictionary suitable for the Boto3 DynamoDB **Client** API, with values serialized into DynamoDB's type format (e.g., `{'S': 'value'}`).
+
+```python
+product = Product(id='456', name='Stormbreaker', price=8500.0)
+
+# Get a simple dictionary of attributes
+plain_dict = product.to_dictionary()
+# {'id': '456', 'name': 'Stormbreaker', 'price': 8500.0, ...}
+
+# Get a dictionary for the DynamoDB Resource API
+resource_dict = product.to_resource_dictionary()
+# {'pk': 'product#456', 'sk': 'product#456', 'id': '456', ...}
+
+# Get a dictionary for the DynamoDB Client API
+client_dict = product.to_client_dictionary()
+# {'pk': {'S': 'product#456'}, 'sk': {'S': 'product#456'}, ...}
+```
+
+## 5. Best Practices
+
+-   **Models as Data Transfer Objects (DTOs)**: Models should only contain data and serialization logic. Avoid adding business logic or direct database calls (e.g., a `save()` method). Keep that logic in your service layer.
+-   **Use Lambda for Keys**: Always use `lambda` functions for key values to ensure they are generated at the time of serialization.
+-   **Consistent Naming**: Follow consistent naming conventions for your indexes and attributes (e.g., `gsi0`, `gsi1_pk`, `gsi1_sk`).
+-   **Call `_setup_indexes` in `__init__`**: Ensure your indexes are defined every time a model is instantiated.

boto3_assist-0.30.0/docs/defining-services.md

@@ -0,0 +1,150 @@
+# Defining Services for DynamoDB Operations
+
+This guide explains how to create a service layer to handle business logic and interact with DynamoDB using `boto3-assist`. The service layer is responsible for all Create, Read, Update, Delete, and List (CRUDL) operations.
+
+## 1. The Role of the Service Layer
+
+The service layer acts as an intermediary between your application's handlers (e.g., API endpoints) and the database. Its primary responsibilities are:
+
+-   **Encapsulating Business Logic**: All logic related to data manipulation resides here.
+-   **Interacting with the Database**: Services are the only part of your application that should directly call the `DynamoDB` class.
+-   **Using Models**: Services use the `DynamoDBModelBase` models to pass data to and from the database.
+-   **Error Handling**: Managing database exceptions and returning consistent responses.
+
+By centralizing database interactions in a service layer, you create a clear separation of concerns, making your application easier to maintain, test, and scale.
+
+## 2. Creating a Basic Service
+
+A service is a Python class that initializes an instance of the `boto3_assist.dynamodb.DynamoDB` class. It also needs the name of the DynamoDB table it will be interacting with, which is typically stored in an environment variable.
+
+Here is a basic `ProductService`:
+
+```python
+import os
+from typing import Optional
+from boto3_assist.dynamodb.dynamodb import DynamoDB
+from ..models.product_model import Product
+
+class ProductService:
+    def __init__(self, db: Optional[DynamoDB] = None):
+        self.db = db or DynamoDB()
+        self.table_name = os.environ.get("APP_TABLE_NAME", "products-table")
+
+    # CRUDL methods will go here
+```
+
+-   **Initialization**: The constructor accepts an optional `DynamoDB` instance, which is useful for dependency injection during testing. If one isn't provided, it creates a new instance.
+-   **Table Name**: The service gets the table name from an environment variable, providing a sensible default.
+
+## 3. Implementing CRUDL Operations
+
+Here’s how to implement the standard CRUDL operations in your service.
+
+### Create
+
+The `create` method uses the `db.save()` method to persist a new item to the database. It takes a model instance, converts it to a dictionary, and saves it.
+
+```python
+def create_product(self, product_data: dict) -> Product:
+    """Creates a new product."""
+    product = Product().map(product_data)
+    
+    # The to_resource_dictionary() method includes the pk, sk, and any GSI keys
+    item_to_save = product.to_resource_dictionary()
+    
+    self.db.save(item=item_to_save, table_name=self.table_name)
+    
+    return product
+```
+
+### Read (Get by ID)
+
+The `get` method retrieves a single item by its primary key. You create a model instance, set its ID, and pass it to the `db.get()` method.
+
+```python
+def get_product_by_id(self, product_id: str) -> Optional[Product]:
+    """Retrieves a product by its ID."""
+    # Create a model with the ID to identify the key
+    model_to_find = Product(id=product_id)
+    
+    response = self.db.get(model=model_to_find, table_name=self.table_name)
+    
+    item = response.get("Item")
+    if not item:
+        return None
+        
+    return Product().map(item)
+```
+
+### Update
+
+Updates are typically handled using a "get-then-save" pattern. You first retrieve the existing item, map the updates to it, and then save it back to the database. This ensures you don't accidentally overwrite data.
+
+```python
+def update_product(self, product_id: str, updates: dict) -> Optional[Product]:
+    """Updates an existing product."""
+    # 1. Get the existing product
+    existing_product = self.get_product_by_id(product_id)
+    if not existing_product:
+        return None
+        
+    # 2. Map the updates to the model
+    existing_product.map(updates)
+    
+    # 3. Save it back to the database
+    item_to_save = existing_product.to_resource_dictionary()
+    self.db.save(item=item_to_save, table_name=self.table_name)
+    
+    return existing_product
+```
+
+### Delete
+
+The `delete` method removes an item from the database using its primary key. Similar to the `get` method, you pass a model instance with the ID set.
+
+```python
+def delete_product(self, product_id: str) -> bool:
+    """Deletes a product by its ID."""
+    product_to_delete = Product(id=product_id)
+    
+    try:
+        self.db.delete(model=product_to_delete, table_name=self.table_name)
+        return True
+    except Exception as e:
+        # Log the error
+        print(f"Error deleting product {product_id}: {e}")
+        return False
+```
+
+### List (Query)
+
+To list items, you typically query a Global Secondary Index (GSI). The `db.query()` method is used for this. You need to provide the index name and a `KeyConditionExpression`.
+
+Here’s how to list all products, sorted by name, using the `gsi0` we defined in the model documentation:
+
+```python
+from boto3.dynamodb.conditions import Key
+
+def list_all_products(self):
+    """Lists all products, sorted by name."""
+    # This key queries for all items where the GSI partition key is 'products'.
+    # This is based on the GSI we defined in the Product model.
+    key_condition = Key('gsi0_pk').eq(Product().get_key('gsi0').partition_key.value())
+
+    response = self.db.query(
+        key=key_condition,
+        index_name="gsi0",
+        table_name=self.table_name,
+        ascending=True
+    )
+    
+    items = response.get("Items", [])
+    return [Product().map(item) for item in items]
+```
+
+## 4. Best Practices
+
+-   **Dependency Injection**: Always allow the `DynamoDB` instance to be injected into your service's constructor. This is critical for testing.
+-   **Environment Variables**: Load sensitive information like table names from environment variables, not hardcoded strings.
+-   **Use Models**: Leverage your `DynamoDBModelBase` models for all data interactions. This ensures your keys are generated correctly and your data is properly serialized.
+-   **Separation of Concerns**: Keep business logic inside the service. API handlers should only be responsible for parsing requests and formatting responses.