django-cfg 1.4.46__py3-none-any.whl → 1.5.181__py3-none-any.whl

django_cfg/apps/knowbase/config/README.md

@@ -1,176 +0,0 @@
-# Knowledge Base Configuration
-
-This directory contains the new Pydantic-based configuration system that replaces the removed Constance settings.
-
-## Overview
-
-The configuration system provides:
-- **Type safety** with Pydantic models
-- **Validation** of configuration values
-- **Environment variable support** with `KNOWBASE_` prefix
-- **Backward compatibility** with existing code
-
-## Files
-
-- `settings.py` - **Main Pydantic configuration models** (single source of truth)
-- `constance_fields.py` - **Constance configuration** (automatically synchronized with `settings.py`)
-- `SYNCHRONIZATION.md` - **Detailed explanation** of how synchronization works
-- `README.md` - This file
-
-## Usage
-
-### Basic Usage
-
-```python
-from apps.knowbase.config.settings import get_config
-
-# Get the global configuration
-config = get_config()
-
-# Access settings
-chunk_size = config.chunking.document_chunk_size  # 1000
-embedding_model = config.embedding.model  # "text-embedding-ada-002"
-```
-
-### Convenience Functions
-
-```python
-from apps.knowbase.config.settings import (
-    get_document_chunk_size,
-    get_embedding_model,
-    get_chunking_params_for_type
-)
-
-# Direct access to common settings
-chunk_size = get_document_chunk_size()  # 1000
-model = get_embedding_model()  # "text-embedding-ada-002"
-
-# Get chunking parameters for SemanticChunker
-doc_params = get_chunking_params_for_type('document')
-# Returns: {'chunk_size': 1000, 'overlap': 200}
-
-archive_params = get_chunking_params_for_type('archive')
-# Returns: {'chunk_size': 800, 'overlap': 160}
-```
-
-## Configuration Sections
-
-### EmbeddingConfig
-- `model`: OpenAI embedding model (default: "text-embedding-ada-002")
-- `batch_size`: Chunks per batch (default: 50, range: 1-100)
-- `max_retries`: Max retries for failed embeddings (default: 3)
-- `timeout_seconds`: API timeout (default: 30)
-
-### ChunkingConfig
-- `document_chunk_size`: Document chunk size (default: 1000, range: 100-8000)
-- `document_chunk_overlap`: Document overlap (default: 200)
-- `archive_chunk_size`: Archive chunk size (default: 800, range: 100-8000)
-- `archive_chunk_overlap`: Archive overlap (default: 160)
-
-### SearchConfig
-- `results_limit`: Max search results (default: 10, range: 1-100)
-- `similarity_threshold`: Min similarity score (default: 0.7, range: 0.0-1.0)
-
-### ChatConfig
-- `context_chunks`: Chunks in chat context (default: 5, range: 1-20)
-- `max_tokens`: Max tokens for completion (default: 4000, range: 100-32000)
-- `temperature`: Creativity level (default: 0.7, range: 0.0-2.0)
-
-### ProcessingConfig
-- `max_document_size_mb`: Max document size (default: 10MB, range: 1-100)
-- `max_archive_size_mb`: Max archive size (default: 50MB, range: 1-500)
-- `timeout_minutes`: Processing timeout (default: 30, range: 1-180)
-
-## Environment Variables
-
-You can override any setting using environment variables with the `KNOWBASE_` prefix:
-
-```bash
-# Override document chunk size
-export KNOWBASE_CHUNKING__DOCUMENT_CHUNK_SIZE=1500
-
-# Override embedding model
-export KNOWBASE_EMBEDDING__MODEL=text-embedding-3-small
-
-# Override search results limit
-export KNOWBASE_SEARCH__RESULTS_LIMIT=20
-```
-
-Note the double underscore `__` to separate nested configuration sections.
-
-## Validation
-
-The configuration system automatically validates all values:
-
-```python
-# This will raise a validation error
-config = KnowledgeBaseConfig(
-    chunking={
-        "document_chunk_size": -100  # Must be >= 100
-    }
-)
-
-# This will also raise an error
-config = KnowledgeBaseConfig(
-    chunking={
-        "document_chunk_size": 100,
-        "document_chunk_overlap": 200  # Must be < chunk_size
-    }
-)
-```
-
-## Synchronization with Constance
-
-**Important**: `settings.py` is the **single source of truth**. The `constance_fields.py` automatically pulls default values from the Pydantic configuration to ensure consistency.
-
-```python
-# settings.py (MASTER)
-class ChunkingConfig(BaseModel):
-    document_chunk_size: int = Field(default=1000, ge=100, le=8000)
-
-# constance_fields.py (SYNCHRONIZED)
-def get_knowbase_constance_fields():
-    default_config = KnowledgeBaseConfig()  # ← Pulls from settings.py
-    return [
-        ConstanceField(
-            name="DOCUMENT_CHUNK_SIZE",
-            default=default_config.chunking.document_chunk_size,  # ← Always in sync!
-        )
-    ]
-```
-
-This eliminates duplicate default values and ensures consistency across the application.
-
-## Migration from Constance
-
-The new system is backward compatible. Existing code using:
-
-```python
-from apps.knowbase.utils.chunk_settings import get_document_chunk_size
-```
-
-Will continue to work without changes. The `chunk_settings.py` module now internally uses the new Pydantic configuration.
-
-## Benefits
-
-1. **Type Safety**: All configuration values are properly typed
-2. **Validation**: Invalid values are caught at startup
-3. **Documentation**: Each setting has clear descriptions and ranges
-4. **Environment Support**: Easy deployment configuration
-5. **Performance**: No database queries for configuration access
-6. **Testing**: Easy to mock and test different configurations
-
-## Testing
-
-```python
-from apps.knowbase.config.settings import KnowledgeBaseConfig
-
-# Create test configuration
-test_config = KnowledgeBaseConfig(
-    chunking={"document_chunk_size": 500},
-    embedding={"batch_size": 10}
-)
-
-# Use in tests
-assert test_config.chunking.document_chunk_size == 500
-```

django_cfg/apps/knowbase/config/__init__.py

@@ -1,52 +0,0 @@
-"""
-Knowledge Base configuration module.
-
-This module provides configuration utilities for the knowledge base app,
-including Pydantic-based settings and optional Constance field definitions.
-"""
-
-# Import the main Pydantic configuration
-from .settings import (
-    KnowledgeBaseConfig,
-    get_archive_chunk_size,
-    get_chunking_params_for_type,
-    get_config,
-    get_document_chunk_size,
-    get_embedding_batch_size,
-    get_embedding_model,
-    reload_config,
-)
-
-
-# Constance-related imports are optional and only loaded when needed
-# to avoid django_cfg dependency issues
-def get_django_cfg_knowbase_constance_fields():
-    """Lazy import of Constance fields to avoid dependency issues."""
-    from .constance_fields import get_django_cfg_knowbase_constance_fields as _get_fields
-    return _get_fields()
-
-def get_django_cfg_knowbase_field_validation_rules():
-    """Lazy import of Constance validation rules to avoid dependency issues."""
-    from .constance_fields import get_django_cfg_knowbase_field_validation_rules as _get_rules
-    return _get_rules()
-
-def get_all_django_cfg_knowbase_constance_config():
-    """Lazy import of complete Constance configuration to avoid dependency issues."""
-    from .constance_fields import get_all_django_cfg_knowbase_constance_config as _get_config
-    return _get_config()
-
-__all__ = [
-    # Pydantic configuration
-    "KnowledgeBaseConfig",
-    "get_config",
-    "reload_config",
-    "get_document_chunk_size",
-    "get_archive_chunk_size",
-    "get_embedding_model",
-    "get_embedding_batch_size",
-    "get_chunking_params_for_type",
-    # Constance configuration (lazy loaded)
-    "get_django_cfg_knowbase_constance_fields",
-    "get_django_cfg_knowbase_field_validation_rules",
-    "get_all_django_cfg_knowbase_constance_config",
-]

django_cfg/apps/knowbase/config/constance_fields.py

@@ -1,188 +0,0 @@
-"""
-Constance fields configuration for Knowledge Base app.
-
-This module defines all dynamic settings for the knowledge base system
-that can be configured through Django admin at runtime.
-
-Note: Default values are now sourced from the Pydantic configuration
-to ensure consistency across the application.
-"""
-
-from typing import List
-
-from django_cfg.models.django.constance import ConstanceField
-
-from .settings import KnowledgeBaseConfig, get_openai_api_key, get_openrouter_api_key
-
-
-def get_django_cfg_knowbase_constance_fields() -> List[ConstanceField]:
-    """
-    Get essential Constance fields for Knowledge Base app.
-    
-    Default values are automatically pulled from the Pydantic configuration
-    to ensure consistency.
-    
-    Returns:
-        List of ConstanceField objects for critical knowledge base settings
-    """
-    # Get default values from Pydantic config
-    default_config = KnowledgeBaseConfig()
-
-    return [
-        # === Core Processing Settings ===
-        ConstanceField(
-            name="DOCUMENT_CHUNK_SIZE",
-            default=default_config.chunking.document_chunk_size,
-            help_text=f"Chunk size for document processing (characters). Default: {default_config.chunking.document_chunk_size}. Affects context quality vs memory usage.",
-            field_type="int",
-            group="Knowledge Base",
-        ),
-        ConstanceField(
-            name="ARCHIVE_CHUNK_SIZE",
-            default=default_config.chunking.archive_chunk_size,
-            help_text=f"Chunk size for archive processing (characters). Default: {default_config.chunking.archive_chunk_size}. Smaller for code files.",
-            field_type="int",
-            group="Knowledge Base",
-        ),
-
-        # === Embedding Settings ===
-        ConstanceField(
-            name="EMBEDDING_MODEL",
-            default=default_config.embedding.model,
-            help_text=f"OpenAI embedding model. Default: {default_config.embedding.model}. Options: text-embedding-ada-002, text-embedding-3-small, text-embedding-3-large. API Key: {'✅ Available' if get_openai_api_key() else '❌ Missing'}",
-            field_type="str",
-            group="Knowledge Base",
-        ),
-        ConstanceField(
-            name="EMBEDDING_BATCH_SIZE",
-            default=default_config.embedding.batch_size,
-            help_text=f"Chunks per embedding batch (1-100). Default: {default_config.embedding.batch_size}. Higher = faster but more memory. OpenRouter Key: {'✅ Available' if get_openrouter_api_key() else '❌ Missing'}",
-            field_type="int",
-            group="Knowledge Base",
-        ),
-
-        # === Search Threshold Settings ===
-        ConstanceField(
-            name="DOCUMENT_THRESHOLD",
-            default=default_config.search.document_threshold,
-            help_text=f"Similarity threshold for document chunks (0.0-1.0). Default: {default_config.search.document_threshold}. Higher = more precise, lower = more results.",
-            field_type="float",
-            group="Knowledge Base",
-        ),
-        ConstanceField(
-            name="ARCHIVE_THRESHOLD",
-            default=default_config.search.archive_threshold,
-            help_text=f"Similarity threshold for archive/code chunks (0.0-1.0). Default: {default_config.search.archive_threshold}. Medium precision for code similarity.",
-            field_type="float",
-            group="Knowledge Base",
-        ),
-        # Note: EXTERNAL_DATA_THRESHOLD removed - now configured per-object in ExternalData.similarity_threshold
-
-        # === AI Assistant Settings ===
-        ConstanceField(
-            name="BOT_IDENTITY",
-            default="I am Reforms.ai, an AI assistant specialized in helping with knowledge base queries and technical documentation. I was developed by the Reforms.ai team to provide accurate information based on your uploaded documents and code archives.",
-            help_text="AI assistant identity and description. This text defines who the bot is and what it does. Used in system prompts for all conversations.",
-            field_type="longtext",
-            group="Knowledge Base",
-        ),
-        ConstanceField(
-            name="BOT_NO_CONTEXT_MESSAGE",
-            default="I can help you with questions about your knowledge base, technical documentation, and uploaded content. However, I don't currently have any specific context loaded for this conversation.",
-            help_text="Message shown when AI assistant has no specific context loaded. Explains what the bot can help with when no documents are found.",
-            field_type="longtext",
-            group="Knowledge Base",
-        ),
-    ]
-
-
-def get_django_cfg_knowbase_field_validation_rules() -> dict:
-    """
-    Get validation rules for essential knowledge base Constance fields.
-    
-    Validation rules are now consistent with Pydantic configuration constraints.
-    
-    Returns:
-        Dictionary with field names as keys and validation functions as values
-    """
-    # Get constraints from Pydantic config
-    default_config = KnowledgeBaseConfig()
-
-    def validate_chunk_size(value):
-        """Validate chunk size values using Pydantic constraints."""
-        if not isinstance(value, int):
-            return False, "Chunk size must be an integer"
-        if value < 100:
-            return False, "Chunk size must be at least 100 characters"
-        if value > 8000:
-            return False, "Chunk size should not exceed 8000 characters"
-        return True, ""
-
-    def validate_batch_size(value):
-        """Validate embedding batch size using Pydantic constraints."""
-        if not isinstance(value, int):
-            return False, "Batch size must be an integer"
-        if value < 1:
-            return False, "Batch size must be at least 1"
-        if value > 100:
-            return False, "Batch size should not exceed 100 for stability"
-        return True, ""
-
-    def validate_embedding_model(value):
-        """Validate embedding model name."""
-        if not isinstance(value, str):
-            return False, "Embedding model must be a string"
-        if not value.strip():
-            return False, "Embedding model cannot be empty"
-        valid_models = [
-            "text-embedding-ada-002",
-            "text-embedding-3-small",
-            "text-embedding-3-large"
-        ]
-        if value not in valid_models:
-            return True, f"Warning: '{value}' is not a standard OpenAI model. Valid options: {', '.join(valid_models)}"
-        return True, ""
-
-    def validate_bot_identity(value):
-        """Validate bot identity text."""
-        if not isinstance(value, str):
-            return False, "Bot identity must be a string"
-        if not value.strip():
-            return False, "Bot identity cannot be empty"
-        if len(value) < 10:
-            return False, "Bot identity should be at least 10 characters"
-        if len(value) > 1000:
-            return False, "Bot identity should not exceed 1000 characters"
-        return True, ""
-
-    def validate_bot_message(value):
-        """Validate bot message text."""
-        if not isinstance(value, str):
-            return False, "Bot message must be a string"
-        if not value.strip():
-            return False, "Bot message cannot be empty"
-        if len(value) < 10:
-            return False, "Bot message should be at least 10 characters"
-        if len(value) > 500:
-            return False, "Bot message should not exceed 500 characters"
-        return True, ""
-
-    return {
-        "DOCUMENT_CHUNK_SIZE": validate_chunk_size,
-        "ARCHIVE_CHUNK_SIZE": validate_chunk_size,
-        "EMBEDDING_BATCH_SIZE": validate_batch_size,
-        "EMBEDDING_MODEL": validate_embedding_model,
-        "BOT_IDENTITY": validate_bot_identity,
-        "BOT_NO_CONTEXT_MESSAGE": validate_bot_message,
-    }
-
-
-# Convenience function for easy import
-def get_all_django_cfg_knowbase_constance_config():
-    """
-    Get complete Constance configuration for Knowledge Base.
-    
-    Returns:
-        Tuple of (fields, validation_rules)
-    """
-    return get_django_cfg_knowbase_constance_fields(), get_django_cfg_knowbase_field_validation_rules()

django_cfg/apps/knowbase/config/constance_settings.py

@@ -1,201 +0,0 @@
-"""
-Constance settings accessor for Knowledge Base app.
-
-This module provides a centralized way to access all Constance settings
-with proper fallbacks and type safety using Pydantic 2.
-"""
-
-from typing import Any, Dict
-
-from constance import config
-from pydantic import BaseModel, Field, validator
-
-
-class KnowledgeBaseConstanceSettings(BaseModel):
-    """Pydantic model for Knowledge Base Constance settings with validation."""
-
-    # === AI Assistant Settings ===
-    bot_identity: str = Field(
-        default="I am Reforms.ai, an AI assistant specialized in helping with knowledge base queries and technical documentation. I was developed by the Reforms.ai team to provide accurate information based on your uploaded documents and code archives.",
-        min_length=10,
-        max_length=1000,
-        description="AI assistant identity and description"
-    )
-
-    bot_no_context_message: str = Field(
-        default="I can help you with questions about your knowledge base, technical documentation, and uploaded content. However, I don't currently have any specific context loaded for this conversation.",
-        min_length=10,
-        max_length=500,
-        description="Message shown when AI has no specific context"
-    )
-
-    # === Processing Settings ===
-    document_chunk_size: int = Field(
-        default=1000,
-        ge=100,
-        le=8000,
-        description="Chunk size for document processing (characters)"
-    )
-
-    archive_chunk_size: int = Field(
-        default=800,
-        ge=100,
-        le=8000,
-        description="Chunk size for archive processing (characters)"
-    )
-
-    # === Embedding Settings ===
-    embedding_model: str = Field(
-        default="text-embedding-ada-002",
-        min_length=1,
-        description="OpenAI embedding model name"
-    )
-
-    embedding_batch_size: int = Field(
-        default=50,
-        ge=1,
-        le=100,
-        description="Chunks per embedding batch"
-    )
-
-    # === Search Threshold Settings ===
-    document_threshold: float = Field(
-        default=0.7,
-        ge=0.0,
-        le=1.0,
-        description="Similarity threshold for document chunks"
-    )
-
-    archive_threshold: float = Field(
-        default=0.6,
-        ge=0.0,
-        le=1.0,
-        description="Similarity threshold for archive chunks"
-    )
-
-    # Note: external_data_threshold removed - now configured per-object in ExternalData.similarity_threshold
-
-    @validator('embedding_model')
-    def validate_embedding_model(cls, v):
-        """Validate embedding model name."""
-        valid_models = [
-            "text-embedding-ada-002",
-            "text-embedding-3-small",
-            "text-embedding-3-large"
-        ]
-        if v not in valid_models:
-            # Warning, not error - allow custom models
-            pass
-        return v
-
-    class Config:
-        """Pydantic configuration."""
-        validate_assignment = True
-        extra = "forbid"
-        json_encoders = {
-            # Add custom encoders if needed
-        }
-
-    @classmethod
-    def from_constance(cls) -> 'KnowledgeBaseConstanceSettings':
-        """Create instance from Constance settings with fallbacks."""
-        try:
-            return cls(
-                bot_identity=getattr(config, 'BOT_IDENTITY', cls.model_fields['bot_identity'].default),
-                bot_no_context_message=getattr(config, 'BOT_NO_CONTEXT_MESSAGE', cls.model_fields['bot_no_context_message'].default),
-                document_chunk_size=getattr(config, 'DOCUMENT_CHUNK_SIZE', cls.model_fields['document_chunk_size'].default),
-                archive_chunk_size=getattr(config, 'ARCHIVE_CHUNK_SIZE', cls.model_fields['archive_chunk_size'].default),
-                embedding_model=getattr(config, 'EMBEDDING_MODEL', cls.model_fields['embedding_model'].default),
-                embedding_batch_size=getattr(config, 'EMBEDDING_BATCH_SIZE', cls.model_fields['embedding_batch_size'].default),
-                document_threshold=getattr(config, 'DOCUMENT_THRESHOLD', cls.model_fields['document_threshold'].default),
-                archive_threshold=getattr(config, 'ARCHIVE_THRESHOLD', cls.model_fields['archive_threshold'].default),
-            )
-        except Exception:
-            # If any validation fails, return defaults
-            return cls()
-
-    @classmethod
-    def get_current(cls) -> 'KnowledgeBaseConstanceSettings':
-        """Get current settings instance (cached for performance)."""
-        # Always refresh cache to avoid stale data issues
-        # TODO: Implement smarter cache invalidation based on Constance changes
-        return cls.from_constance()
-
-    @classmethod
-    def refresh_cache(cls):
-        """Refresh cached settings instance."""
-        if hasattr(cls, '_cached_instance'):
-            delattr(cls, '_cached_instance')
-
-    # === Convenience Methods for Backward Compatibility ===
-
-    @classmethod
-    def get_bot_identity(cls) -> str:
-        """Get bot identity from current settings."""
-        return cls.get_current().bot_identity
-
-    @classmethod
-    def get_bot_no_context_message(cls) -> str:
-        """Get bot no-context message from current settings."""
-        return cls.get_current().bot_no_context_message
-
-    @classmethod
-    def get_document_chunk_size(cls) -> int:
-        """Get document chunk size from current settings."""
-        return cls.get_current().document_chunk_size
-
-    @classmethod
-    def get_archive_chunk_size(cls) -> int:
-        """Get archive chunk size from current settings."""
-        return cls.get_current().archive_chunk_size
-
-    @classmethod
-    def get_embedding_model(cls) -> str:
-        """Get embedding model from current settings."""
-        return cls.get_current().embedding_model
-
-    @classmethod
-    def get_embedding_batch_size(cls) -> int:
-        """Get embedding batch size from current settings."""
-        return cls.get_current().embedding_batch_size
-
-    @classmethod
-    def get_document_threshold(cls) -> float:
-        """Get document similarity threshold from current settings."""
-        return cls.get_current().document_threshold
-
-    @classmethod
-    def get_archive_threshold(cls) -> float:
-        """Get archive similarity threshold from current settings."""
-        return cls.get_current().archive_threshold
-
-    # Note: get_external_data_threshold removed - now configured per-object in ExternalData.similarity_threshold
-
-    @classmethod
-    def get_threshold_for_type(cls, content_type: str) -> float:
-        """Get appropriate threshold for content type."""
-        current = cls.get_current()
-        thresholds = {
-            'document': current.document_threshold,
-            'archive': current.archive_threshold,
-            # Note: external_data now uses per-object thresholds from ExternalData.similarity_threshold
-        }
-        return thresholds.get(content_type, 0.7)  # fallback
-
-    @classmethod
-    def get_all_settings(cls) -> Dict[str, Any]:
-        """Get all settings as a dictionary."""
-        return cls.get_current().model_dump()
-
-    @classmethod
-    def validate_settings(cls) -> Dict[str, str]:
-        """Validate current settings and return results."""
-        try:
-            settings = cls.from_constance()
-            return {field: 'OK' for field in settings.model_fields.keys()}
-        except Exception as e:
-            return {'validation_error': str(e)}
-
-
-# Convenience alias for shorter imports
-ConstanceSettings = KnowledgeBaseConstanceSettings