moai-adk 0.25.4__py3-none-any.whl → 0.32.8__py3-none-any.whl

moai_adk/templates/.claude/skills/moai-domain-database/SKILL.md

@@ -0,0 +1,320 @@
+---
+name: moai-domain-database
+description: Database specialist covering PostgreSQL, MongoDB, Redis, and advanced data patterns for modern applications
+version: 1.0.0
+category: domain
+tags:
+ - database
+ - postgresql
+ - mongodb
+ - redis
+ - data-patterns
+ - performance
+updated: 2025-11-30
+status: active
+author: MoAI-ADK Team
+---
+
+# Database Domain Specialist
+
+## Quick Reference (30 seconds)
+
+Enterprise Database Expertise - Comprehensive database patterns and implementations covering PostgreSQL, MongoDB, Redis, and advanced data management for scalable modern applications.
+
+Core Capabilities:
+- PostgreSQL: Advanced relational patterns, optimization, and scaling
+- MongoDB: Document modeling, aggregation, and NoSQL performance tuning
+- Redis: In-memory caching, real-time analytics, and distributed systems
+- Multi-Database: Hybrid architectures and data integration patterns
+- Performance: Query optimization, indexing strategies, and scaling
+- Operations: Connection management, migrations, and monitoring
+
+When to Use:
+- Designing database schemas and data models
+- Implementing caching strategies and performance optimization
+- Building scalable data architectures
+- Working with multi-database systems
+- Optimizing database queries and performance
+
+---
+
+## Implementation Guide (5 minutes)
+
+### Quick Start Workflow
+
+Database Stack Initialization:
+```python
+from moai_domain_database import DatabaseManager
+
+# Initialize multi-database stack
+db_manager = DatabaseManager()
+
+# Configure PostgreSQL for relational data
+postgresql = db_manager.setup_postgresql(
+ connection_string="postgresql://...",
+ connection_pool_size=20,
+ enable_query_logging=True
+)
+
+# Configure MongoDB for document storage
+mongodb = db_manager.setup_mongodb(
+ connection_string="mongodb://...",
+ database_name="app_data",
+ enable_sharding=True
+)
+
+# Configure Redis for caching and real-time features
+redis = db_manager.setup_redis(
+ connection_string="redis://...",
+ max_connections=50,
+ enable_clustering=True
+)
+
+# Use unified database interface
+user_data = db_manager.get_user_with_profile(user_id)
+analytics = db_manager.get_user_analytics(user_id, time_range="30d")
+```
+
+Single Database Operations:
+```bash
+# PostgreSQL schema migration
+moai db:migrate --database postgresql --migration-file schema_v2.sql
+
+# MongoDB aggregation pipeline
+moai db:aggregate --collection users --pipeline analytics_pipeline.json
+
+# Redis cache warming
+moai db:cache:warm --pattern "user:*" --ttl 3600
+```
+
+### Core Components
+
+1. PostgreSQL (`modules/postgresql.md`)
+- Advanced schema design and constraints
+- Complex query optimization and indexing
+- Window functions and CTEs
+- Partitioning and materialized views
+- Connection pooling and performance tuning
+
+2. MongoDB (`modules/mongodb.md`)
+- Document modeling and schema design
+- Aggregation pipelines for analytics
+- Indexing strategies and performance
+- Sharding and scaling patterns
+- Data consistency and validation
+
+3. Redis (`modules/redis.md`)
+- Multi-layer caching strategies
+- Real-time analytics and counting
+- Distributed locking and coordination
+- Pub/sub messaging and streams
+- Advanced data structures (HyperLogLog, Geo)
+
+---
+
+## Advanced Patterns (10+ minutes)
+
+### Multi-Database Architecture
+
+Polyglot Persistence Pattern:
+```python
+class DataRouter:
+ def __init__(self):
+ self.postgresql = PostgreSQLConnection()
+ self.mongodb = MongoDBConnection()
+ self.redis = RedisConnection()
+
+ def get_user_profile(self, user_id):
+ # Get structured user data from PostgreSQL
+ user = self.postgresql.get_user(user_id)
+
+ # Get flexible profile data from MongoDB
+ profile = self.mongodb.get_user_profile(user_id)
+
+ # Get real-time status from Redis
+ status = self.redis.get_user_status(user_id)
+
+ return self.merge_user_data(user, profile, status)
+
+ def update_user_data(self, user_id, data):
+ # Route different data types to appropriate databases
+ if 'structured_data' in data:
+ self.postgresql.update_user(user_id, data['structured_data'])
+
+ if 'profile_data' in data:
+ self.mongodb.update_user_profile(user_id, data['profile_data'])
+
+ if 'real_time_data' in data:
+ self.redis.set_user_status(user_id, data['real_time_data'])
+
+ # Invalidate cache across databases
+ self.invalidate_user_cache(user_id)
+```
+
+Data Synchronization:
+```python
+class DataSyncManager:
+ def sync_user_data(self, user_id):
+ # Sync from PostgreSQL to MongoDB for search
+ pg_user = self.postgresql.get_user(user_id)
+ search_document = self.create_search_document(pg_user)
+ self.mongodb.upsert_user_search(user_id, search_document)
+
+ # Update cache in Redis
+ cache_data = self.create_cache_document(pg_user)
+ self.redis.set_user_cache(user_id, cache_data, ttl=3600)
+```
+
+### Performance Optimization
+
+Query Performance Analysis:
+```python
+# PostgreSQL query optimization
+def analyze_query_performance(query):
+ explain_result = postgresql.execute(f"EXPLAIN (ANALYZE, BUFFERS) {query}")
+ return QueryAnalyzer(explain_result).get_optimization_suggestions()
+
+# MongoDB aggregation optimization
+def optimize_aggregation_pipeline(pipeline):
+ optimizer = AggregationOptimizer()
+ return optimizer.optimize_pipeline(pipeline)
+
+# Redis performance monitoring
+def monitor_redis_performance():
+ metrics = redis.info()
+ return PerformanceAnalyzer(metrics).get_recommendations()
+```
+
+Scaling Strategies:
+```python
+# Read replicas for PostgreSQL
+read_replicas = postgresql.setup_read_replicas([
+ "postgresql://replica1...",
+ "postgresql://replica2..."
+])
+
+# Sharding for MongoDB
+mongodb.setup_sharding(
+ shard_key="user_id",
+ num_shards=4
+)
+
+# Redis clustering
+redis.setup_cluster([
+ "redis://node1:7000",
+ "redis://node2:7000",
+ "redis://node3:7000"
+])
+```
+
+---
+
+## Works Well With
+
+Complementary Skills:
+- `moai-domain-backend` - API integration and business logic
+- `moai-foundation-core` - Database migration and schema management
+- `moai-workflow-project` - Database project setup and configuration
+- `moai-platform-baas` - BaaS database integration patterns
+
+Technology Integration:
+- ORMs and ODMs (SQLAlchemy, Mongoose, TypeORM)
+- Connection pooling (PgBouncer, connection pools)
+- Migration tools (Alembic, Flyway)
+- Monitoring (pg_stat_statements, MongoDB Atlas)
+- Cache invalidation and synchronization
+
+---
+
+## Usage Examples
+
+### Database Operations
+```python
+# PostgreSQL advanced queries
+users = postgresql.query(
+ "SELECT * FROM users WHERE created_at > %s ORDER BY activity_score DESC LIMIT 100",
+ [datetime.now() - timedelta(days=30)]
+)
+
+# MongoDB analytics
+analytics = mongodb.aggregate('events', [
+ {"$match": {"timestamp": {"$gte": start_date}}},
+ {"$group": {"_id": "$type", "count": {"$sum": 1}}},
+ {"$sort": {"count": -1}}
+])
+
+# Redis caching operations
+async def get_user_data(user_id):
+ cache_key = f"user:{user_id}"
+ data = await redis.get(cache_key)
+
+ if not data:
+ data = fetch_from_database(user_id)
+ await redis.setex(cache_key, 3600, json.dumps(data))
+
+ return json.loads(data)
+```
+
+### Multi-Database Transactions
+```python
+async def create_user_with_profile(user_data, profile_data):
+ try:
+ # Start transaction across databases
+ async with transaction_manager():
+ # Create user in PostgreSQL
+ user_id = await postgresql.insert_user(user_data)
+
+ # Create profile in MongoDB
+ await mongodb.insert_user_profile(user_id, profile_data)
+
+ # Set initial cache in Redis
+ await redis.set_user_cache(user_id, {
+ "id": user_id,
+ "status": "active",
+ "created_at": datetime.now().isoformat()
+ })
+
+ return user_id
+
+ except Exception as e:
+ # Automatic rollback across databases
+ logger.error(f"User creation failed: {e}")
+ raise
+```
+
+---
+
+## Technology Stack
+
+Relational Database:
+- PostgreSQL 14+ (primary)
+- MySQL 8.0+ (alternative)
+- Connection pooling (PgBouncer, SQLAlchemy)
+
+NoSQL Database:
+- MongoDB 6.0+ (primary)
+- Document modeling and validation
+- Aggregation framework
+- Sharding and replication
+
+In-Memory Database:
+- Redis 7.0+ (primary)
+- Redis Stack for advanced features
+- Clustering and high availability
+- Advanced data structures
+
+Supporting Tools:
+- Migration tools (Alembic, Flyway)
+- Monitoring (Prometheus, Grafana)
+- ORMs/ODMs (SQLAlchemy, Mongoose)
+- Connection management
+
+Performance Features:
+- Query optimization and analysis
+- Index management and strategies
+- Caching layers and invalidation
+- Load balancing and failover
+
+---
+
+*For detailed implementation patterns and database-specific optimizations, see the `modules/` directory.*

moai_adk/templates/.claude/skills/moai-domain-database/modules/README.md

@@ -0,0 +1,53 @@
+# Database Domain Modules
+
+This directory contains specialized modules for advanced database patterns and implementations across multiple database systems.
+
+## Module Structure
+
+### Core Database Systems
+
+- postgresql.md - Advanced PostgreSQL patterns, optimization, and scaling
+- mongodb.md - NoSQL document modeling, aggregation, and performance tuning
+- redis.md - In-memory caching, real-time data structures, and distributed systems
+
+### Usage Patterns
+
+1. Relational Database: Use `postgresql.md` for structured data and complex queries
+2. Document Database: Use `mongodb.md` for flexible schemas and rapid development
+3. In-Memory Store: Use `redis.md` for caching, sessions, and real-time features
+4. Multi-Database: Combine modules for hybrid data architectures
+
+### Integration Guidelines
+
+Each module provides comprehensive patterns for:
+
+```python
+# PostgreSQL integration
+from moai_domain_database.modules.postgresql import AdvancedPostgreSQL
+
+# MongoDB integration
+from moai_domain_database.modules.mongodb import MongoAggregation
+
+# Redis integration
+from moai_domain_database.modules.redis import CacheManager
+
+# Hybrid database architecture
+def setup_database_stack():
+ postgresql = AdvancedPostgreSQL()
+ mongodb = MongoAggregation()
+ redis = CacheManager()
+ return DatabaseStack(postgresql, mongodb, redis)
+```
+
+### Progressive Disclosure
+
+- Quick Start: Use individual database modules for specific needs
+- Implementation: Combine modules for comprehensive data solutions
+- Advanced: Custom implementations based on module patterns
+
+### Dependencies
+
+- PostgreSQL: asyncpg, SQLAlchemy, Psycopg2
+- MongoDB: pymongo, motor, mongoose
+- Redis: redis-py, ioredis, node-redis
+- General: Connection pooling, ORM frameworks, migration tools

moai_adk/templates/.claude/skills/moai-domain-database/modules/mongodb.md

@@ -0,0 +1,231 @@
+# MongoDB Advanced Patterns
+
+## Overview
+Advanced MongoDB patterns covering document modeling, aggregation pipelines, indexing strategies, and performance optimization for scalable NoSQL applications.
+
+## Quick Implementation
+
+### Document Modeling
+
+```javascript
+// User schema with embedded and referenced patterns
+const userSchema = new Schema({
+ _id: ObjectId,
+ username: { type: String, required: true, unique: true },
+ email: { type: String, required: true, unique: true },
+ profile: {
+ displayName: String,
+ bio: String,
+ avatar: String,
+ preferences: {
+ theme: { type: String, enum: ['light', 'dark'], default: 'light' },
+ language: { type: String, default: 'en' },
+ notifications: {
+ email: { type: Boolean, default: true },
+ push: { type: Boolean, default: false }
+ }
+ }
+ },
+ security: {
+ passwordHash: String,
+ lastLoginAt: Date,
+ failedLoginAttempts: { type: Number, default: 0 },
+ lockedUntil: Date
+ },
+ activity: {
+ lastSeenAt: Date,
+ loginCount: { type: Number, default: 0 }
+ }
+}, {
+ timestamps: true,
+ // Optimized indexes
+ index: [
+ { username: 1 },
+ { email: 1 },
+ { 'security.lastLoginAt': -1 },
+ { 'activity.lastSeenAt': -1 }
+ ]
+});
+
+// Post schema with comments embedded for performance
+const postSchema = new Schema({
+ _id: ObjectId,
+ authorId: { type: ObjectId, ref: 'User', required: true },
+ title: { type: String, required: true },
+ content: { type: String, required: true },
+ tags: [String],
+ metadata: {
+ viewCount: { type: Number, default: 0 },
+ likeCount: { type: Number, default: 0 },
+ commentCount: { type: Number, default: 0 }
+ },
+ // Embed recent comments for performance
+ recentComments: [{
+ _id: ObjectId,
+ authorId: ObjectId,
+ authorName: String,
+ content: String,
+ createdAt: { type: Date, default: Date.now }
+ }],
+ status: { type: String, enum: ['draft', 'published', 'archived'], default: 'draft' }
+}, {
+ timestamps: true,
+ index: [
+ { authorId: 1, createdAt: -1 },
+ { status: 1, createdAt: -1 },
+ { tags: 1 },
+ { 'metadata.viewCount': -1 }
+ ]
+});
+```
+
+### Advanced Aggregation Pipelines
+
+```javascript
+// User analytics with complex aggregation
+const getUserAnalytics = async (userId, timeRange) => {
+ return await User.aggregate([
+ // Match specific user
+ { $match: { _id: userId } },
+
+ // Lookup posts and engagement
+ {
+ $lookup: {
+ from: 'posts',
+ localField: '_id',
+ foreignField: 'authorId',
+ as: 'posts'
+ }
+ },
+
+ // Unwind posts for processing
+ { $unwind: '$posts' },
+
+ // Filter by time range
+ {
+ $match: {
+ 'posts.createdAt': {
+ $gte: timeRange.start,
+ $lte: timeRange.end
+ }
+ }
+ },
+
+ // Group by user and calculate metrics
+ {
+ $group: {
+ _id: '$_id',
+ username: { $first: '$username' },
+ totalPosts: { $sum: 1 },
+ totalViews: { $sum: '$posts.metadata.viewCount' },
+ totalLikes: { $sum: '$posts.metadata.likeCount' },
+ avgViewsPerPost: { $avg: '$posts.metadata.viewCount' },
+ tags: { $push: '$posts.tags' }
+ }
+ },
+
+ // Flatten and count tags
+ {
+ $addFields: {
+ allTags: { $reduce: {
+ input: '$tags',
+ initialValue: [],
+ in: { $concatArrays: ['$$value', '$$this'] }
+ }}
+ }
+ },
+
+ {
+ $addFields: {
+ uniqueTags: { $setUnion: ['$allTags', []] },
+ topTags: { $slice: [
+ { $sortArray: { input: { $objectToArray: { $size: '$allTags' } }, sortBy: { count: -1 } } },
+ 5
+ ]}
+ }
+ }
+ ]);
+};
+
+// Time series aggregation for activity tracking
+const getActivityTrends = async (userId, granularity = 'daily') => {
+ const groupFormat = granularity === 'daily' ?
+ { $dateToString: { format: '%Y-%m-%d', date: '$createdAt' } } :
+ { $dateToString: { format: '%Y-%U', date: '$createdAt' } };
+
+ return await Activity.aggregate([
+ { $match: { userId } },
+ {
+ $group: {
+ _id: groupFormat,
+ totalEvents: { $sum: 1 },
+ eventTypes: { $addToSet: '$eventType' },
+ uniqueSessions: { $addToSet: '$sessionId' }
+ }
+ },
+ {
+ $addFields: {
+ eventCount: { $size: '$eventTypes' },
+ sessionCount: { $size: '$uniqueSessions' }
+ }
+ },
+ { $sort: { _id: 1 } }
+ ]);
+};
+```
+
+## Key Features
+
+### 1. Document Design Patterns
+- Embedding vs referencing strategies
+- Denormalization for read performance
+- Schema validation and constraints
+- Array and object modeling
+
+### 2. Indexing Strategies
+- Compound indexes for multi-field queries
+- Text indexes for full-text search
+- Geospatial indexes for location data
+- TTL indexes for automatic expiration
+
+### 3. Aggregation Framework
+- Multi-stage data processing pipelines
+- Lookup operations for joins
+- Group and accumulate operations
+- Window functions for analytics
+
+### 4. Performance Optimization
+- Query pattern analysis
+- Index usage optimization
+- Connection pooling strategies
+- Sharding for horizontal scaling
+
+## Design Patterns
+
+### Embedding Patterns
+- One-to-One: Embed related data
+- One-to-Many: Embed for small arrays
+- Many-to-Many: Use references with lookups
+
+### Reference Patterns
+- Population for related data
+- Manual joins with aggregation
+- Caching frequently accessed data
+
+### Schema Evolution
+- Version-controlled schema changes
+- Backward compatibility strategies
+- Data migration approaches
+
+## Best Practices
+- Design for query patterns
+- Use appropriate data types
+- Implement proper indexing
+- Monitor performance metrics
+- Plan for schema evolution
+
+## Integration Points
+- ORMs (Mongoose, TypeGOOSE)
+- ODMs for Node.js applications
+- Connection pooling libraries
+- Monitoring and analytics tools