moai-adk 0.35.1__py3-none-any.whl

moai_adk/templates/.claude/skills/moai-domain-database/modules/README.md

@@ -0,0 +1,53 @@
+# Database Domain Modules
+
+This directory contains specialized modules for advanced database patterns and implementations across multiple database systems.
+
+## Module Structure
+
+### Core Database Systems
+
+- postgresql.md - Advanced PostgreSQL patterns, optimization, and scaling
+- mongodb.md - NoSQL document modeling, aggregation, and performance tuning
+- redis.md - In-memory caching, real-time data structures, and distributed systems
+
+### Usage Patterns
+
+1. Relational Database: Use `postgresql.md` for structured data and complex queries
+2. Document Database: Use `mongodb.md` for flexible schemas and rapid development
+3. In-Memory Store: Use `redis.md` for caching, sessions, and real-time features
+4. Multi-Database: Combine modules for hybrid data architectures
+
+### Integration Guidelines
+
+Each module provides comprehensive patterns for:
+
+```python
+# PostgreSQL integration
+from moai_domain_database.modules.postgresql import AdvancedPostgreSQL
+
+# MongoDB integration
+from moai_domain_database.modules.mongodb import MongoAggregation
+
+# Redis integration
+from moai_domain_database.modules.redis import CacheManager
+
+# Hybrid database architecture
+def setup_database_stack():
+ postgresql = AdvancedPostgreSQL()
+ mongodb = MongoAggregation()
+ redis = CacheManager()
+ return DatabaseStack(postgresql, mongodb, redis)
+```
+
+### Progressive Disclosure
+
+- Quick Start: Use individual database modules for specific needs
+- Implementation: Combine modules for comprehensive data solutions
+- Advanced: Custom implementations based on module patterns
+
+### Dependencies
+
+- PostgreSQL: asyncpg, SQLAlchemy, Psycopg2
+- MongoDB: pymongo, motor, mongoose
+- Redis: redis-py, ioredis, node-redis
+- General: Connection pooling, ORM frameworks, migration tools

moai_adk/templates/.claude/skills/moai-domain-database/modules/mongodb.md

@@ -0,0 +1,231 @@
+# MongoDB Advanced Patterns
+
+## Overview
+Advanced MongoDB patterns covering document modeling, aggregation pipelines, indexing strategies, and performance optimization for scalable NoSQL applications.
+
+## Quick Implementation
+
+### Document Modeling
+
+```javascript
+// User schema with embedded and referenced patterns
+const userSchema = new Schema({
+ _id: ObjectId,
+ username: { type: String, required: true, unique: true },
+ email: { type: String, required: true, unique: true },
+ profile: {
+ displayName: String,
+ bio: String,
+ avatar: String,
+ preferences: {
+ theme: { type: String, enum: ['light', 'dark'], default: 'light' },
+ language: { type: String, default: 'en' },
+ notifications: {
+ email: { type: Boolean, default: true },
+ push: { type: Boolean, default: false }
+ }
+ }
+ },
+ security: {
+ passwordHash: String,
+ lastLoginAt: Date,
+ failedLoginAttempts: { type: Number, default: 0 },
+ lockedUntil: Date
+ },
+ activity: {
+ lastSeenAt: Date,
+ loginCount: { type: Number, default: 0 }
+ }
+}, {
+ timestamps: true,
+ // Optimized indexes
+ index: [
+ { username: 1 },
+ { email: 1 },
+ { 'security.lastLoginAt': -1 },
+ { 'activity.lastSeenAt': -1 }
+ ]
+});
+
+// Post schema with comments embedded for performance
+const postSchema = new Schema({
+ _id: ObjectId,
+ authorId: { type: ObjectId, ref: 'User', required: true },
+ title: { type: String, required: true },
+ content: { type: String, required: true },
+ tags: [String],
+ metadata: {
+ viewCount: { type: Number, default: 0 },
+ likeCount: { type: Number, default: 0 },
+ commentCount: { type: Number, default: 0 }
+ },
+ // Embed recent comments for performance
+ recentComments: [{
+ _id: ObjectId,
+ authorId: ObjectId,
+ authorName: String,
+ content: String,
+ createdAt: { type: Date, default: Date.now }
+ }],
+ status: { type: String, enum: ['draft', 'published', 'archived'], default: 'draft' }
+}, {
+ timestamps: true,
+ index: [
+ { authorId: 1, createdAt: -1 },
+ { status: 1, createdAt: -1 },
+ { tags: 1 },
+ { 'metadata.viewCount': -1 }
+ ]
+});
+```
+
+### Advanced Aggregation Pipelines
+
+```javascript
+// User analytics with complex aggregation
+const getUserAnalytics = async (userId, timeRange) => {
+ return await User.aggregate([
+ // Match specific user
+ { $match: { _id: userId } },
+
+ // Lookup posts and engagement
+ {
+ $lookup: {
+ from: 'posts',
+ localField: '_id',
+ foreignField: 'authorId',
+ as: 'posts'
+ }
+ },
+
+ // Unwind posts for processing
+ { $unwind: '$posts' },
+
+ // Filter by time range
+ {
+ $match: {
+ 'posts.createdAt': {
+ $gte: timeRange.start,
+ $lte: timeRange.end
+ }
+ }
+ },
+
+ // Group by user and calculate metrics
+ {
+ $group: {
+ _id: '$_id',
+ username: { $first: '$username' },
+ totalPosts: { $sum: 1 },
+ totalViews: { $sum: '$posts.metadata.viewCount' },
+ totalLikes: { $sum: '$posts.metadata.likeCount' },
+ avgViewsPerPost: { $avg: '$posts.metadata.viewCount' },
+ tags: { $push: '$posts.tags' }
+ }
+ },
+
+ // Flatten and count tags
+ {
+ $addFields: {
+ allTags: { $reduce: {
+ input: '$tags',
+ initialValue: [],
+ in: { $concatArrays: ['$$value', '$$this'] }
+ }}
+ }
+ },
+
+ {
+ $addFields: {
+ uniqueTags: { $setUnion: ['$allTags', []] },
+ topTags: { $slice: [
+ { $sortArray: { input: { $objectToArray: { $size: '$allTags' } }, sortBy: { count: -1 } } },
+ 5
+ ]}
+ }
+ }
+ ]);
+};
+
+// Time series aggregation for activity tracking
+const getActivityTrends = async (userId, granularity = 'daily') => {
+ const groupFormat = granularity === 'daily' ?
+ { $dateToString: { format: '%Y-%m-%d', date: '$createdAt' } } :
+ { $dateToString: { format: '%Y-%U', date: '$createdAt' } };
+
+ return await Activity.aggregate([
+ { $match: { userId } },
+ {
+ $group: {
+ _id: groupFormat,
+ totalEvents: { $sum: 1 },
+ eventTypes: { $addToSet: '$eventType' },
+ uniqueSessions: { $addToSet: '$sessionId' }
+ }
+ },
+ {
+ $addFields: {
+ eventCount: { $size: '$eventTypes' },
+ sessionCount: { $size: '$uniqueSessions' }
+ }
+ },
+ { $sort: { _id: 1 } }
+ ]);
+};
+```
+
+## Key Features
+
+### 1. Document Design Patterns
+- Embedding vs referencing strategies
+- Denormalization for read performance
+- Schema validation and constraints
+- Array and object modeling
+
+### 2. Indexing Strategies
+- Compound indexes for multi-field queries
+- Text indexes for full-text search
+- Geospatial indexes for location data
+- TTL indexes for automatic expiration
+
+### 3. Aggregation Framework
+- Multi-stage data processing pipelines
+- Lookup operations for joins
+- Group and accumulate operations
+- Window functions for analytics
+
+### 4. Performance Optimization
+- Query pattern analysis
+- Index usage optimization
+- Connection pooling strategies
+- Sharding for horizontal scaling
+
+## Design Patterns
+
+### Embedding Patterns
+- One-to-One: Embed related data
+- One-to-Many: Embed for small arrays
+- Many-to-Many: Use references with lookups
+
+### Reference Patterns
+- Population for related data
+- Manual joins with aggregation
+- Caching frequently accessed data
+
+### Schema Evolution
+- Version-controlled schema changes
+- Backward compatibility strategies
+- Data migration approaches
+
+## Best Practices
+- Design for query patterns
+- Use appropriate data types
+- Implement proper indexing
+- Monitor performance metrics
+- Plan for schema evolution
+
+## Integration Points
+- ORMs (Mongoose, TypeGOOSE)
+- ODMs for Node.js applications
+- Connection pooling libraries
+- Monitoring and analytics tools

moai_adk/templates/.claude/skills/moai-domain-database/modules/postgresql.md

@@ -0,0 +1,169 @@
+# PostgreSQL Advanced Patterns
+
+## Overview
+Comprehensive PostgreSQL patterns covering advanced schema design, query optimization, indexing strategies, partitioning, and performance tuning for modern applications.
+
+## Quick Implementation
+
+### Advanced Schema Design
+
+```sql
+-- User table with proper constraints and indexes
+CREATE TABLE users (
+ id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
+ email VARCHAR(255) UNIQUE NOT NULL,
+ username VARCHAR(50) UNIQUE,
+ password_hash VARCHAR(255) NOT NULL,
+ email_verified BOOLEAN DEFAULT FALSE,
+ created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
+ updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
+ last_login_at TIMESTAMP WITH TIME ZONE
+);
+
+-- Optimized indexes
+CREATE INDEX idx_users_email ON users (email) WHERE email_verified = TRUE;
+CREATE INDEX idx_users_created_at ON users (created_at DESC);
+CREATE INDEX idx_users_username_trgm ON users USING gin (username gin_trgm_ops);
+
+-- User profiles with JSONB
+CREATE TABLE user_profiles (
+ user_id UUID PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
+ display_name VARCHAR(100),
+ bio TEXT,
+ avatar_url VARCHAR(500),
+ preferences JSONB DEFAULT '{}',
+ metadata JSONB DEFAULT '{}',
+ created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
+ updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
+);
+
+-- GIN index for JSONB queries
+CREATE INDEX idx_user_profiles_preferences ON user_profiles USING gin (preferences);
+```
+
+### Advanced Query Patterns
+
+```sql
+-- Efficient pagination with cursor-based approach
+WITH ranked_users AS (
+ SELECT
+ u.id,
+ u.username,
+ u.email,
+ up.display_name,
+ ROW_NUMBER() OVER (ORDER BY u.created_at DESC) as rn
+ FROM users u
+ LEFT JOIN user_profiles up ON u.id = up.user_id
+ WHERE u.created_at < :cursor OR :cursor IS NULL
+)
+SELECT * FROM ranked_users
+WHERE rn > :offset
+ORDER BY created_at DESC
+LIMIT :limit;
+
+-- Full-text search with trigrams
+SELECT
+ u.id,
+ u.username,
+ up.display_name,
+ similarity(u.username, :query) as username_sim,
+ ts_rank_cd(search_vector, plainto_tsquery('english', :query)) as search_rank
+FROM users u
+LEFT JOIN user_profiles up ON u.id = up.user_id,
+ to_tsvector('english', u.username || ' ' || COALESCE(up.display_name, '')) search_vector
+WHERE u.username % :query
+ OR plainto_tsquery('english', :query) @@ search_vector
+ORDER BY username_sim DESC, search_rank DESC
+LIMIT 20;
+
+-- Window functions for analytics
+SELECT
+ DATE_TRUNC('month', created_at) as month,
+ COUNT(*) as new_users,
+ SUM(COUNT(*)) OVER (ORDER BY DATE_TRUNC('month', created_at)) as cumulative_users,
+ COUNT(*) - LAG(COUNT(*)) OVER (ORDER BY DATE_TRUNC('month', created_at)) as month_over_month_change
+FROM users
+WHERE created_at >= NOW() - INTERVAL '1 year'
+GROUP BY DATE_TRUNC('month', created_at)
+ORDER BY month;
+```
+
+### Performance Optimization
+
+```sql
+-- Partitioning for time-series data
+CREATE TABLE events (
+ id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
+ user_id UUID NOT NULL REFERENCES users(id),
+ event_type VARCHAR(50) NOT NULL,
+ data JSONB,
+ created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
+) PARTITION BY RANGE (created_at);
+
+-- Monthly partitions
+CREATE TABLE events_2024_01 PARTITION OF events
+ FOR VALUES FROM ('2024-01-01') TO ('2024-02-01');
+
+CREATE TABLE events_2024_02 PARTITION OF events
+ FOR VALUES FROM ('2024-02-01') TO ('2024-03-01');
+
+-- Materialized views for analytics
+CREATE MATERIALIZED VIEW user_stats AS
+SELECT
+ u.id,
+ u.email,
+ u.created_at,
+ COUNT(DISTINCT e.id) as event_count,
+ MAX(e.created_at) as last_event_at,
+ COUNT(DISTINCT CASE WHEN e.event_type = 'login' THEN e.id END) as login_count
+FROM users u
+LEFT JOIN events e ON u.id = e.user_id
+GROUP BY u.id, u.email, u.created_at;
+
+-- Refresh materialized view
+CREATE OR REPLACE FUNCTION refresh_user_stats()
+RETURNS void AS $$
+BEGIN
+ REFRESH MATERIALIZED VIEW CONCURRENTLY user_stats;
+END;
+$$ LANGUAGE plpgsql;
+```
+
+## Key Features
+
+### 1. Advanced Indexing
+- Partial indexes for filtered data
+- GIN indexes for full-text search and JSONB
+- Expression indexes for computed values
+- Composite indexes for multi-column queries
+
+### 2. Query Optimization
+- Cursor-based pagination
+- Window functions for analytics
+- CTE (Common Table Expressions)
+- Materialized views for performance
+
+### 3. Data Modeling
+- JSONB for flexible schemas
+- Array types for multi-valued data
+- Range types for temporal data
+- Custom types for domain-specific data
+
+### 4. Performance Tuning
+- Table partitioning
+- Connection pooling
+- Query execution plans
+- Vacuum and analyze strategies
+
+## Best Practices
+- Use appropriate data types and constraints
+- Implement proper indexing strategies
+- Monitor query performance
+- Use transactions for data consistency
+- Implement backup and recovery procedures
+
+## Integration Points
+- ORMs (SQLAlchemy, Django ORM)
+- Connection pools (PgBouncer, connection poolers)
+- Monitoring tools (pg_stat_statements)
+- Migration tools (Alembic, Flyway)

moai_adk/templates/.claude/skills/moai-domain-database/modules/redis.md

@@ -0,0 +1,262 @@
+# Redis Advanced Patterns
+
+## Overview
+Advanced Redis patterns covering caching strategies, data structures, real-time analytics, distributed locking, and performance optimization for high-throughput applications.
+
+## Quick Implementation
+
+### Caching Strategies
+
+```javascript
+class CacheManager {
+ constructor(redisClient) {
+ this.redis = redisClient;
+ }
+
+ // Multi-layer caching with fallback
+ async getWithFallback(key, fetchFunction, ttl = 3600) {
+ // Try memory cache first
+ const memoryCache = this.getMemoryCache(key);
+ if (memoryCache) return memoryCache;
+
+ // Try Redis cache
+ const redisCache = await this.redis.get(key);
+ if (redisCache) {
+ const data = JSON.parse(redisCache);
+ this.setMemoryCache(key, data, ttl / 10); // Shorter memory TTL
+ return data;
+ }
+
+ // Fetch from source
+ const data = await fetchFunction();
+
+ // Set both caches
+ await this.redis.setex(key, ttl, JSON.stringify(data));
+ this.setMemoryCache(key, data, ttl / 10);
+
+ return data;
+ }
+
+ // Write-through caching
+ async setWithWriteThrough(key, data, fetchFunction, ttl = 3600) {
+ // Update source first
+ await fetchFunction(data);
+
+ // Update caches
+ const pipeline = this.redis.pipeline();
+ pipeline.setex(key, ttl, JSON.stringify(data));
+
+ // Invalidate related cache keys
+ const relatedKeys = await this.getRelatedKeys(key);
+ relatedKeys.forEach(relatedKey => {
+ pipeline.del(relatedKey);
+ });
+
+ await pipeline.exec();
+ this.setMemoryCache(key, data, ttl / 10);
+ }
+
+ // Rate limiting with sliding window
+ async checkRateLimit(key, limit, windowMs) {
+ const now = Date.now();
+ const pipeline = this.redis.pipeline();
+
+ // Remove old entries
+ pipeline.zremrangebyscore(key, 0, now - windowMs);
+
+ // Add current request
+ pipeline.zadd(key, now, now);
+
+ // Count current window requests
+ pipeline.zcard(key);
+
+ // Set expiration
+ pipeline.expire(key, Math.ceil(windowMs / 1000));
+
+ const results = await pipeline.exec();
+ const currentCount = results[2][1];
+
+ return {
+ allowed: currentCount <= limit,
+ count: currentCount,
+ remaining: Math.max(0, limit - currentCount),
+ resetTime: now + windowMs
+ };
+ }
+}
+```
+
+### Advanced Data Structures
+
+```javascript
+class RedisDataManager {
+ constructor(redisClient) {
+ this.redis = redisClient;
+ }
+
+ // Leaderboard with time decay
+ async addToTimeDecayLeaderboard(key, memberId, score, decayRate = 0.95) {
+ const timestamp = Date.now();
+ const decayedScore = score * Math.pow(decayRate, (Date.now() - timestamp) / (1000 * 60 * 60));
+
+ await this.redis.zadd(key, decayedScore, memberId);
+
+ // Remove old entries
+ const weekAgo = Date.now() - (7 * 24 * 60 * 60 * 1000);
+ await this.redis.zremrangebyscore(key, 0, weekAgo);
+ }
+
+ // Real-time analytics with HyperLogLog
+ async trackUniqueVisitors(pageKey, userId) {
+ // Track unique users with HyperLogLog
+ await this.redis.pfadd(`${pageKey}:unique`, userId);
+
+ // Track total visits with regular counter
+ await this.redis.incr(`${pageKey}:total`);
+
+ // Track user activity set for recent activity
+ const activityKey = `${pageKey}:activity:${Math.floor(Date.now() / (60 * 1000))}`;
+ await this.redis.sadd(activityKey, userId);
+ await this.redis.expire(activityKey, 300); // 5 minutes
+ }
+
+ async getAnalytics(pageKey) {
+ const pipeline = this.redis.pipeline();
+
+ // Unique visitors estimate
+ pipeline.pfcount(`${pageKey}:unique`);
+
+ // Total page views
+ pipeline.get(`${pageKey}:total`);
+
+ // Recent active users (last 5 minutes)
+ const now = Math.floor(Date.now() / (60 * 1000));
+ for (let i = 0; i < 5; i++) {
+ pipeline.scard(`${pageKey}:activity:${now - i}`);
+ }
+
+ const results = await pipeline.exec();
+
+ const uniqueVisitors = results[0][1];
+ const totalViews = parseInt(results[1][1]) || 0;
+ const recentActivity = results.slice(2).map(r => r[1]).reduce((a, b) => a + b, 0);
+
+ return {
+ uniqueVisitors,
+ totalViews,
+ recentActiveUsers: recentActivity
+ };
+ }
+
+ // Distributed locking with timeout and retry
+ async acquireLock(key, ttl = 30000, retryCount = 3, retryDelay = 100) {
+ const lockKey = `lock:${key}`;
+ const lockValue = `${Date.now()}-${Math.random()}`;
+
+ for (let attempt = 0; attempt < retryCount; attempt++) {
+ const result = await this.redis.set(
+ lockKey,
+ lockValue,
+ 'PX', ttl,
+ 'NX'
+ );
+
+ if (result === 'OK') {
+ return {
+ acquired: true,
+ lockValue,
+ release: async () => await this.releaseLock(lockKey, lockValue)
+ };
+ }
+
+ // Wait before retry
+ if (attempt < retryCount - 1) {
+ await new Promise(resolve => setTimeout(resolve, retryDelay));
+ }
+ }
+
+ return { acquired: false };
+ }
+
+ async releaseLock(lockKey, lockValue) {
+ const script = `
+ if redis.call("get", KEYS[1]) == ARGV[1] then
+ return redis.call("del", KEYS[1])
+ else
+ return 0
+ end
+ `;
+
+ return await this.redis.eval(script, 1, lockKey, lockValue);
+ }
+}
+```
+
+## Key Features
+
+### 1. Caching Patterns
+- Multi-layer caching (memory + Redis)
+- Cache-aside and write-through strategies
+- Cache invalidation and warming
+- Rate limiting and throttling
+
+### 2. Data Structures
+- Strings for simple key-value storage
+- Hashes for object storage
+- Lists for queues and stacks
+- Sets for unique collections
+- Sorted sets for leaderboards
+- HyperLogLog for cardinality estimation
+
+### 3. Real-time Features
+- Pub/sub messaging
+- Streams for event processing
+- Geospatial indexing
+- Bitmap operations
+
+### 4. Performance Optimization
+- Pipeline operations for batch processing
+- Lua scripting for atomic operations
+- Connection pooling strategies
+- Memory optimization techniques
+
+## Advanced Patterns
+
+### Distributed Locking
+- Redlock algorithm for safety
+- Timeout and retry mechanisms
+- Lock release verification
+- Deadlock prevention
+
+### Rate Limiting
+- Sliding window implementation
+- Fixed window counters
+- Token bucket algorithm
+- Distributed rate limiting
+
+### Analytics and Counting
+- Real-time metrics collection
+- Time-series data storage
+- Approximate counting with HyperLogLog
+- Bitmap analytics for user behavior
+
+### Session Management
+- User session storage
+- Shopping cart management
+- Real-time collaboration
+- Presence detection
+
+## Best Practices
+- Use appropriate data structures
+- Implement proper key naming conventions
+- Set appropriate TTLs for cache entries
+- Monitor memory usage and performance
+- Use pipelining for batch operations
+- Implement proper error handling
+
+## Integration Points
+- Connection pooling libraries (ioredis, redis-py)
+- Cache middleware for web frameworks
+- Message queue systems
+- Real-time analytics platforms
+- Session management systems