iatoolkit 0.3.9__tar.gz → 0.4.1__tar.gz

{iatoolkit-0.3.9 → iatoolkit-0.4.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: iatoolkit
-Version: 0.3.9
+Version: 0.4.1
 Summary: IAToolkit
 Author: Fernando Libedinsky
 License-Expression: MIT
@@ -207,46 +207,52 @@ Requires-Dist: yarl==1.18.3
 Requires-Dist: zipp==3.21.0
 Requires-Dist: zstandard==0.23.0
 
-# iatoolkit
+
+<div align="center">
+  <h1>IAToolkit</h1>
+  <p><strong>The Open-Source Framework for Building AI Chatbots on Your Private Data.</strong></p>
+</div>
 
 IAToolkit is a comprehensive, open-source framework designed for building enterprise-grade 
 AI chatbots and conversational applications. 
-Built on Flask with dependency injection, it provides a robust foundation for scalable AI solutions.
+With IAToolkit, you can build production-ready, context-aware chatbots and agents that 
+can query relational databases, perform semantic searches on documents, 
+and connect to your internal APIs in minutes.
+
+IAToolkit bridges the gap between powerful LLMs and your company's data. 
+
 
 ## 🚀 Key Features
-- **Universal LLM Integration**: OpenAI GPT, Google Gemini
-- **Template System**: Jinja2-powered prompt templates with variables
-- **Context Management**: Maintain conversation context across sessions
-
-### 🔒 **Enterprise Security**
-- **JWT Authentication**: Secure token-based authentication
-- **Session Management**: Redis-backed secure sessions
-- **CORS Configuration**: Flexible cross-origin resource sharing
-
-### 🛠 **Function Calling & Tools**
-- **Native Function Calls**: Direct integration with LLM function calling
-- **Custom Tools**: Build and register custom tools for your chatbot
-- **SQL Query Generation**: Natural language to SQL conversion
-- **API Integrations**: Connect to external services and APIs
-
-### 🗄 **Database & Storage**
-- **Multi-Database Support**: PostgreSQL, MySQL, SQLite via SQLAlchemy
-- **Vector Store Integration**: Semantic search and retrieval
-- **Document Processing**: PDF, Word, Excel, and text file handling
-
-### 📊 **Analytics & Monitoring**
-- **Query Logging**: Track all LLM interactions
-- **Performance Metrics**: Response times, token usage, costs
-- **Benchmarking**: Compare model performance
-- **Task Management**: Async task processing with status tracking
-
-### 🔧 **Developer Experience**
-- **Dependency Injection**: Clean, testable architecture
-- **CLI Tools**: Command-line interface for common tasks
-- **Hot Reloading**: Development-friendly configuration
-- **Comprehensive Logging**: Debug and monitor easily
-
-## License
-MIT License
 
+*   **🔗 Unified Data Connection**:
+    *   **Natural Language to SQL**: Let your chatbot query relational databases (PostgreSQL, MySQL, SQLite) using everyday language.
+    *   **Semantic Document Search**: Automatically chunk, embed, and search across your private documents (PDFs, Word, etc.) to provide contextually accurate answers.
+
+*   **🏢 Enterprise-Ready Multi-Tenancy**:
+    *   Deploy isolated "Company" modules, each with its own data, tools, and context. Perfect for SaaS products or internal departmental agents.
+
+*   **🧠 LLM Agnostic**:
+    *   Switch between **OpenAI (GPT-*)** and **Google (Gemini-*)** with a single line change in your configuration. No code refactoring needed.
+
+*   **🛠️ Developer-First Experience**:
+    *   Built with a clean, **Dependency Injection** architecture.
+    *   High-quality code base with **90%+ test coverage**.
+    *   Powerful Flask-based **CLI** for database setup, API key generation, and more.
+
+*   **🔒 Security & Observability Built-In**:
+    *   Comes with JWT-based authentication, user management, and secure session handling out of the box.
+    *   Full traceability with detailed logging of all queries, function calls, token usage, and costs.
+
+## ⚡ Quick Start: Create a Custom Tool in 30 Seconds
+
+See how easy it is to give your AI a new skill. Just define a method inside your Company class and describe it. 
+IAToolkit handles the rest.
+
+## 🤝 Contributing
+
+We welcome contributions! Whether it's adding a new feature, improving documentation, or fixing a bug, 
+please feel free to open a pull request.
+
+## 📄 License
 
+IAToolkit is open-source and licensed under the [MIT License](LICENSE).

{iatoolkit-0.3.9 → iatoolkit-0.4.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "iatoolkit"
-version = "0.3.9"
+version = "0.4.1"
 requires-python = ">=3.11"
 description = "IAToolkit"
 readme = "readme.md"

iatoolkit-0.4.1/readme.md

@@ -0,0 +1,49 @@
+
+<div align="center">
+  <h1>IAToolkit</h1>
+  <p><strong>The Open-Source Framework for Building AI Chatbots on Your Private Data.</strong></p>
+</div>
+
+IAToolkit is a comprehensive, open-source framework designed for building enterprise-grade 
+AI chatbots and conversational applications. 
+With IAToolkit, you can build production-ready, context-aware chatbots and agents that 
+can query relational databases, perform semantic searches on documents, 
+and connect to your internal APIs in minutes.
+
+IAToolkit bridges the gap between powerful LLMs and your company's data. 
+
+
+## 🚀 Key Features
+
+*   **🔗 Unified Data Connection**:
+    *   **Natural Language to SQL**: Let your chatbot query relational databases (PostgreSQL, MySQL, SQLite) using everyday language.
+    *   **Semantic Document Search**: Automatically chunk, embed, and search across your private documents (PDFs, Word, etc.) to provide contextually accurate answers.
+
+*   **🏢 Enterprise-Ready Multi-Tenancy**:
+    *   Deploy isolated "Company" modules, each with its own data, tools, and context. Perfect for SaaS products or internal departmental agents.
+
+*   **🧠 LLM Agnostic**:
+    *   Switch between **OpenAI (GPT-*)** and **Google (Gemini-*)** with a single line change in your configuration. No code refactoring needed.
+
+*   **🛠️ Developer-First Experience**:
+    *   Built with a clean, **Dependency Injection** architecture.
+    *   High-quality code base with **90%+ test coverage**.
+    *   Powerful Flask-based **CLI** for database setup, API key generation, and more.
+
+*   **🔒 Security & Observability Built-In**:
+    *   Comes with JWT-based authentication, user management, and secure session handling out of the box.
+    *   Full traceability with detailed logging of all queries, function calls, token usage, and costs.
+
+## ⚡ Quick Start: Create a Custom Tool in 30 Seconds
+
+See how easy it is to give your AI a new skill. Just define a method inside your Company class and describe it. 
+IAToolkit handles the rest.
+
+## 🤝 Contributing
+
+We welcome contributions! Whether it's adding a new feature, improving documentation, or fixing a bug, 
+please feel free to open a pull request.
+
+## 📄 License
+
+IAToolkit is open-source and licensed under the [MIT License](LICENSE).

{iatoolkit-0.3.9 → iatoolkit-0.4.1}/src/iatoolkit/__init__.py

@@ -21,13 +21,15 @@ from services.excel_service import ExcelService
 from services.dispatcher_service import Dispatcher
 from services.document_service import DocumentService
 from services.search_service import SearchService
-from services.query_service import QueryService
+from services.load_documents_service import LoadDocumentsService
 from repositories.profile_repo import ProfileRepo
 from repositories.llm_query_repo import LLMQueryRepo
+from services.query_service import QueryService
+from services.prompt_manager_service import PromptService
 from repositories.database_manager import DatabaseManager
 from infra.call_service import CallServiceClient
 from common.util import Utility
-from repositories.models import Base, Company, Function, TaskType
+from repositories.models import Base, Company, Function, TaskType, Prompt, PromptCategory
 
 
 __all__ = [
@@ -40,10 +42,12 @@ __all__ = [
     'ExcelService',
     'Dispatcher',
     'DocumentService',
-    'QueryService',
     'SearchService',
+    'QueryService',
+    'LoadDocumentsService',
     'ProfileRepo',
     'LLMQueryRepo',
+    'PromptService',
     'DatabaseManager',
     'CallServiceClient',
     'Utility',
@@ -51,4 +55,6 @@ __all__ = [
     'Function',
     'TaskType',
     'Base',
+    'Prompt',
+    'PromptCategory'
 ]

{iatoolkit-0.3.9 → iatoolkit-0.4.1}/src/iatoolkit/cli_commands.py

@@ -24,6 +24,8 @@ def register_core_commands(app):
     def setup_company(company_short_name: str):
         """⚙️ Genera una nueva API key para una compañía ya registrada."""
         try:
+            dispatcher = IAToolkit.get_instance().get_injector().get(Dispatcher)
+            dispatcher.setup_all_companies()
             profile_service = IAToolkit.get_instance().get_injector().get(ProfileService)
             click.echo(f"🔑 Generando API key para '{company_short_name}'...")
             result = profile_service.new_api_key(company_short_name)
@@ -64,14 +66,4 @@ def register_core_commands(app):
             logging.exception(e)
             click.echo(f"Error: {str(e)}")
 
-    @app.cli.command("load")
-    def load_documents():
-        from services.load_documents_service import LoadDocumentsService
 
-        load_documents_service = IAToolkit.get_instance().get_injector().get(LoadDocumentsService)
-        try:
-            result = load_documents_service.load()
-            click.echo(result['message'])
-        except Exception as e:
-            logging.exception(e)
-            click.echo(f"Error: {str(e)}")

{iatoolkit-0.3.9 → iatoolkit-0.4.1}/src/iatoolkit/company_registry.py

@@ -15,27 +15,21 @@ class CompanyRegistry:
     def __init__(self):
         self._company_classes: Dict[str, Type[BaseCompany]] = {}
         self._company_instances: Dict[str, BaseCompany] = {}
-        self._injector = None
 
-    def set_injector(self, injector) -> None:
-        """Establece el injector para crear instancias con dependencias"""
-        self._injector = injector
 
-    def instantiate_companies(self) -> Dict[str, BaseCompany]:
+    def instantiate_companies(self, injector) -> Dict[str, BaseCompany]:
         """
         Instancia todas las empresas registradas con inyección de dependencias.
 
         Returns:
             Dict con instancias de empresas {name: instance}
         """
-        if not self._injector:
-            raise RuntimeError("Injector no configurado. Llame a set_injector() primero.")
 
         for company_key, company_class in self._company_classes.items():
             if company_key not in self._company_instances:
                 try:
                     # use de injector to create the instance
-                    company_instance = self._injector.get(company_class)
+                    company_instance = injector.get(company_class)
                     self._company_instances[company_key] = company_instance
                     logging.info(f"company '{company_key}' created in dispatcher")
 
@@ -46,6 +40,10 @@ class CompanyRegistry:
 
         return self._company_instances.copy()
 
+    def get_all_company_instances(self) -> Dict[str, BaseCompany]:
+        """Devuelve un diccionario con todas las instancias de empresas creadas."""
+        return self._company_instances.copy()
+
     def get_registered_companies(self) -> Dict[str, Type[BaseCompany]]:
         return self._company_classes.copy()
 

{iatoolkit-0.3.9 → iatoolkit-0.4.1}/src/iatoolkit.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: iatoolkit
-Version: 0.3.9
+Version: 0.4.1
 Summary: IAToolkit
 Author: Fernando Libedinsky
 License-Expression: MIT
@@ -207,46 +207,52 @@ Requires-Dist: yarl==1.18.3
 Requires-Dist: zipp==3.21.0
 Requires-Dist: zstandard==0.23.0
 
-# iatoolkit
+
+<div align="center">
+  <h1>IAToolkit</h1>
+  <p><strong>The Open-Source Framework for Building AI Chatbots on Your Private Data.</strong></p>
+</div>
 
 IAToolkit is a comprehensive, open-source framework designed for building enterprise-grade 
 AI chatbots and conversational applications. 
-Built on Flask with dependency injection, it provides a robust foundation for scalable AI solutions.
+With IAToolkit, you can build production-ready, context-aware chatbots and agents that 
+can query relational databases, perform semantic searches on documents, 
+and connect to your internal APIs in minutes.
+
+IAToolkit bridges the gap between powerful LLMs and your company's data. 
+
 
 ## 🚀 Key Features
-- **Universal LLM Integration**: OpenAI GPT, Google Gemini
-- **Template System**: Jinja2-powered prompt templates with variables
-- **Context Management**: Maintain conversation context across sessions
-
-### 🔒 **Enterprise Security**
-- **JWT Authentication**: Secure token-based authentication
-- **Session Management**: Redis-backed secure sessions
-- **CORS Configuration**: Flexible cross-origin resource sharing
-
-### 🛠 **Function Calling & Tools**
-- **Native Function Calls**: Direct integration with LLM function calling
-- **Custom Tools**: Build and register custom tools for your chatbot
-- **SQL Query Generation**: Natural language to SQL conversion
-- **API Integrations**: Connect to external services and APIs
-
-### 🗄 **Database & Storage**
-- **Multi-Database Support**: PostgreSQL, MySQL, SQLite via SQLAlchemy
-- **Vector Store Integration**: Semantic search and retrieval
-- **Document Processing**: PDF, Word, Excel, and text file handling
-
-### 📊 **Analytics & Monitoring**
-- **Query Logging**: Track all LLM interactions
-- **Performance Metrics**: Response times, token usage, costs
-- **Benchmarking**: Compare model performance
-- **Task Management**: Async task processing with status tracking
-
-### 🔧 **Developer Experience**
-- **Dependency Injection**: Clean, testable architecture
-- **CLI Tools**: Command-line interface for common tasks
-- **Hot Reloading**: Development-friendly configuration
-- **Comprehensive Logging**: Debug and monitor easily
-
-## License
-MIT License
 
+*   **🔗 Unified Data Connection**:
+    *   **Natural Language to SQL**: Let your chatbot query relational databases (PostgreSQL, MySQL, SQLite) using everyday language.
+    *   **Semantic Document Search**: Automatically chunk, embed, and search across your private documents (PDFs, Word, etc.) to provide contextually accurate answers.
+
+*   **🏢 Enterprise-Ready Multi-Tenancy**:
+    *   Deploy isolated "Company" modules, each with its own data, tools, and context. Perfect for SaaS products or internal departmental agents.
+
+*   **🧠 LLM Agnostic**:
+    *   Switch between **OpenAI (GPT-*)** and **Google (Gemini-*)** with a single line change in your configuration. No code refactoring needed.
+
+*   **🛠️ Developer-First Experience**:
+    *   Built with a clean, **Dependency Injection** architecture.
+    *   High-quality code base with **90%+ test coverage**.
+    *   Powerful Flask-based **CLI** for database setup, API key generation, and more.
+
+*   **🔒 Security & Observability Built-In**:
+    *   Comes with JWT-based authentication, user management, and secure session handling out of the box.
+    *   Full traceability with detailed logging of all queries, function calls, token usage, and costs.
+
+## ⚡ Quick Start: Create a Custom Tool in 30 Seconds
+
+See how easy it is to give your AI a new skill. Just define a method inside your Company class and describe it. 
+IAToolkit handles the rest.
+
+## 🤝 Contributing
+
+We welcome contributions! Whether it's adding a new feature, improving documentation, or fixing a bug, 
+please feel free to open a pull request.
+
+## 📄 License
 
+IAToolkit is open-source and licensed under the [MIT License](LICENSE).

{iatoolkit-0.3.9 → iatoolkit-0.4.1}/src/services/benchmark_service.py

@@ -65,7 +65,7 @@ class BenchmarkService:
 
         company = self.profile_repo.get_company_by_short_name(company_short_name)
         if not company:
-            raise IAToolkitException(IAToolkitException.ErrorType.CONFIG_ERROR, "Compañía 'maxxa' no encontrada.")
+            raise IAToolkitException(IAToolkitException.ErrorType.CONFIG_ERROR, f"Compañía {company_short_name} no encontrada.")
 
         total_rows = len(df)
         logging.info(f"Iniciando benchmark para {total_rows} casos de prueba desde el archivo: {file_path}")

{iatoolkit-0.3.9 → iatoolkit-0.4.1}/src/services/dispatcher_service.py

@@ -3,7 +3,6 @@
 # Todos los derechos reservados.
 # En trámite de registro en el Registro de Propiedad Intelectual de Chile.
 
-from iatoolkit import current_iatoolkit
 from common.exceptions import IAToolkitException
 from services.prompt_manager_service import PromptService
 from services.api_service import ApiService
@@ -11,10 +10,10 @@ from repositories.llm_query_repo import LLMQueryRepo
 from repositories.models import Company, Function
 from services.excel_service import ExcelService
 from services.mail_service import MailService
-from iatoolkit.company_registry import get_company_registry
 from common.session_manager import SessionManager
 from common.util import Utility
 from injector import inject
+from typing import Dict
 import logging
 import os
 
@@ -37,11 +36,10 @@ class Dispatcher:
         self.system_functions = _FUNCTION_LIST
         self.system_prompts = _SYSTEM_PROMPT
 
-        # Use the global registry
-        self.company_registry = get_company_registry()
+        self._company_registry = None
+        self._company_instances = None
 
         # load into the dispatcher the configured companies
-        self.company_classes = {}
         self.initialize_companies()
 
         # run the statrtup logic for all companies
@@ -53,23 +51,39 @@ class Dispatcher:
             "iat_api_call": self.api_service.call_api
         }
 
+    @property
+    def company_registry(self):
+        """Lazy-loads and returns the CompanyRegistry instance."""
+        if self._company_registry is None:
+            from iatoolkit.company_registry import get_company_registry
+            self._company_registry = get_company_registry()
+        return self._company_registry
+
+    @property
+    def company_instances(self):
+        """Lazy-loads and returns the instantiated company classes."""
+        if self._company_instances is None:
+            self._company_instances = self.company_registry.get_all_company_instances()
+        return self._company_instances
+
     def initialize_companies(self):
+        from iatoolkit import current_iatoolkit
         """
         Initializes and instantiates all registered company classes.
         This method should be called *after* the main injector is fully configured
         and the company registry is populated.
         """
-        if self.company_classes: # Prevent re-initialization
+        if self.company_registry.get_all_company_instances():  # Check if already instantiated
             return
 
         # ✅ NOW it is safe to get the injector and instantiate companies.
         injector = current_iatoolkit().get_injector()
-        self.company_registry.set_injector(injector)
-        self.company_classes = self.company_registry.instantiate_companies()
+        self.company_registry.instantiate_companies(injector)
+
 
     def start_execution(self):
         """Runs the startup logic for all registered companies."""
-        for company_name, company_instance in self.company_classes.items():
+        for company_name, company_instance in self.company_instances.items():
             logging.info(f'Starting execution for company: {company_name}')
             company_instance.start_execution()
 
@@ -100,14 +114,14 @@ class Dispatcher:
                 i += 1
 
         # register in the database  every company class
-        for company in self.company_classes.values():
+        for company in self.company_instances.values():
             company.register_company()
 
     def dispatch(self, company_name: str, action: str, **kwargs) -> str:
         company_key = company_name.lower()
 
-        if company_key not in self.company_classes:
-            available_companies = list(self.company_classes.keys())
+        if company_key not in self.company_instances:
+            available_companies = list(self.company_instances.keys())
             raise IAToolkitException(
                 IAToolkitException.ErrorType.EXTERNAL_SOURCE_ERROR,
                 f"Empresa '{company_name}' no configurada. Empresas disponibles: {available_companies}"
@@ -117,7 +131,7 @@ class Dispatcher:
         if action in self.tool_handlers:
             return  self.tool_handlers[action](**kwargs)
 
-        company_instance = self.company_classes[company_name]
+        company_instance = self.company_instances[company_name]
         try:
             return company_instance.handle_request(action, **kwargs)
         except IAToolkitException as e:
@@ -130,7 +144,7 @@ class Dispatcher:
                                f"Error en function call '{action}': {str(e)}") from e
 
     def get_company_context(self, company_name: str, **kwargs) -> str:
-        if company_name not in self.company_classes:
+        if company_name not in self.company_instances:
             raise IAToolkitException(IAToolkitException.ErrorType.EXTERNAL_SOURCE_ERROR,
                                f"Empresa no configurada: {company_name}")
 
@@ -152,7 +166,7 @@ class Dispatcher:
             filepath = os.path.join(schema_dir, file)
             company_context += self.util.generate_context_for_schema(schema_name, filepath)
 
-        company_instance = self.company_classes[company_name]
+        company_instance = self.company_instances[company_name]
         try:
             return company_context + company_instance.get_company_context(**kwargs)
         except Exception as e:
@@ -180,7 +194,7 @@ class Dispatcher:
         return tools
 
     def get_user_info(self, company_name: str, user_identifier: str, is_local_user: bool) -> dict:
-        if company_name not in self.company_classes:
+        if company_name not in self.company_instances:
             raise IAToolkitException(IAToolkitException.ErrorType.EXTERNAL_SOURCE_ERROR,
                                      f"Empresa no configurada: {company_name}")
 
@@ -190,7 +204,7 @@ class Dispatcher:
             raw_user_data = SessionManager.get('user', {})
         else:
             # source 2: external company user
-            company_instance = self.company_classes[company_name]
+            company_instance = self.company_instances[company_name]
             try:
                 raw_user_data = company_instance.get_user_info(user_identifier)
             except Exception as e:
@@ -226,11 +240,11 @@ class Dispatcher:
         return normalized_user
 
     def get_metadata_from_filename(self, company_name: str, filename: str) -> dict:
-        if company_name not in self.company_classes:
+        if company_name not in self.company_instances:
             raise IAToolkitException(IAToolkitException.ErrorType.EXTERNAL_SOURCE_ERROR,
                                f"Empresa no configurada: {company_name}")
 
-        company_instance = self.company_classes[company_name]
+        company_instance = self.company_instances[company_name]
         try:
             return company_instance.get_metadata_from_filename(filename)
         except Exception as e:
@@ -240,14 +254,14 @@ class Dispatcher:
 
     def get_company_instance(self, company_name: str):
         """Returns the instance for a given company name."""
-        return self.company_classes.get(company_name)
+        return self.company_instances.get(company_name)
 
     def get_registered_companies(self) -> dict:
-        """Obtiene todas las empresas registradas (para debugging/admin)"""
+        """Gets all registered companies (for debugging/admin purposes)"""
         return {
             "registered_classes": list(self.company_registry.get_registered_companies().keys()),
-            "instantiated": list(self.company_classes.keys()),
-            "count": len(self.company_classes)
+            "instantiated": list(self.company_instances.keys()),
+            "count": len(self.company_instances)
         }
 
 

{iatoolkit-0.3.9 → iatoolkit-0.4.1}/src/services/file_processor_service.py

@@ -7,26 +7,45 @@ from infra.connectors.file_connector import FileConnector
 import logging
 import os
 from typing import Optional, Callable, Dict
+from repositories.models import Company
 
 
 class FileProcessorConfig:
+    """Configuration class for the FileProcessor."""
     def __init__(
         self,
         filters: Dict,
-        action: Callable[[str, bytes], None],
+        callback: Callable[[Company, str, bytes, dict], None],
         continue_on_error: bool = True,
         log_file: str = 'file_processor.log',
         echo: bool = False,
-        context: dict = None,
+        context: dict = None
     ):
+        """
+        Initializes the FileProcessor configuration.
+
+        Args:
+            filters (Dict): A dictionary of filters to apply to file names.
+                Example: {'filename_contains': '.pdf'}
+            action (Callable): The function to execute for each processed file.
+                It receives filename (str), content (bytes), and context (dict).
+            continue_on_error (bool): If True, continues processing other files upon an error.
+            log_file (str): The path to the log file.
+            echo (bool): If True, prints progress to the console.
+            context (dict): A context dictionary passed to the action function.
+        """
         self.filters = filters
-        self.action = action
+        self.callback = callback
         self.continue_on_error = continue_on_error
         self.log_file = log_file
         self.echo = echo
         self.context = context or {}
 
 class FileProcessor:
+    """
+    A generic service to process files from a given data source (connector).
+    It lists files, applies filters, and executes a specific action for each one.
+    """
     def __init__(self,
                  connector: FileConnector,
                  config: FileProcessorConfig,
@@ -45,6 +64,7 @@ class FileProcessor:
         return logging.getLogger(__name__)
 
     def process_files(self):
+        # Fetches files from the connector, filters them, and processes them.
         try:
             files = self.connector.list_files()
         except Exception as e:
@@ -67,9 +87,12 @@ class FileProcessor:
 
                 content = self.connector.get_file_content(file_path)
 
-                # execute the action defined
+                # execute the callback function
                 filename = os.path.basename(file_name)
-                self.config.action(filename, content, self.config.context)
+                self.config.callback(company=self.config.context.get('company'),
+                                     filename=filename,
+                                     content=content,
+                                     context=self.config.context)
                 self.processed_files += 1
 
                 self.logger.info(f"Successfully processed file: {file_path}")

{iatoolkit-0.3.9 → iatoolkit-0.4.1}/src/services/load_documents_service.py

@@ -21,6 +21,10 @@ from typing import Dict
 
 
 class LoadDocumentsService:
+    """
+    Orchestrates the process of loading, processing, and storing documents
+    from various sources for different companies.
+    """
     @inject
     def __init__(self,
                  file_connector_factory: FileConnectorFactory,
@@ -38,7 +42,6 @@ class LoadDocumentsService:
         self.vector_store = vector_store
         self.file_connector_factory = file_connector_factory
         self.dispatcher = dispatcher
-        self.company = None
 
         # lower warnings
         logging.getLogger().setLevel(logging.ERROR)
@@ -51,6 +54,17 @@ class LoadDocumentsService:
 
     # load the files for all of the companies.
     def load(self, doc_type: str = None):
+        """
+                Loads documents for all companies based on their configuration.
+                It can load all document types or a specific one if provided.
+
+                Args:
+                    doc_type (str, optional): A specific document type to load.
+                                              If None, all configured types are loaded.
+
+                Returns:
+                    Dict: A dictionary with a summary message.
+                """
         # doc_type: an optional document_type for loading
         files_loaded = 0
         companies = self.profile_repo.get_companies()
@@ -61,7 +75,6 @@ class LoadDocumentsService:
                 continue
 
             print(f"Cargando datos de ** {company.short_name} **")
-            self.company = company
 
             # Si hay configuraciones de tipos de documento específicos
             doc_types_config = load_config.get('document_types', {})
@@ -81,7 +94,34 @@ class LoadDocumentsService:
                     raise IAToolkitException(IAToolkitException.ErrorType.MISSING_PARAMETER,
                                        f"Falta configurar conector en empresa {company.short_name}")
 
-                files_loaded += self.load_data_source(connector)
+                files_loaded += self.load_data_source(company=company,
+                                                      connector_config=connector)
+
+        return {'message': f'{files_loaded} files processed'}
+
+    def load_company_files(self, company: Company,
+                           connector: dict,
+                           predefined_metadata: Dict = None,
+                           filters: Dict = None):
+        """
+        Loads all files for a specific company using a given connector.
+
+        Args:
+            company (Company): The company to load files for.
+            connector (dict): The connector configuration.
+
+        Returns:
+            Dict: A dictionary with a summary message.
+        """
+        if not connector:
+            raise IAToolkitException(IAToolkitException.ErrorType.MISSING_PARAMETER,
+                        f"Falta configurar conector")
+
+        files_loaded = self.load_data_source(
+                            company=company,
+                            connector_config=connector,
+                            predefined_metadata=predefined_metadata,
+                            filters=filters)
 
         return {'message': f'{files_loaded} files processed'}
 
@@ -99,36 +139,42 @@ class LoadDocumentsService:
         # config specific filters
         filters = type_config.get('filters', {"filename_contains": ".pdf"})
 
-        return self.load_data_source(connector, predefined_metadata, filters)
+        return self.load_data_source(company=company,
+                                     connector_config=connector,
+                                     predefined_metadata=predefined_metadata,
+                                     filters=filters)
 
-    def load_data_source(self, connector_config: Dict, predefined_metadata: Dict = None, filters: Dict = None):
+    def load_data_source(self, company: Company, connector_config: Dict, predefined_metadata: Dict = None, filters: Dict = None):
         """
-        Carga archivos desde una fuente de datos usando un conector.
+        Loads files from a data source using a connector and a FileProcessor.
 
         Args:
-            connector_config: Configuración del conector
-            predefined_metadata: Metadatos predefinidos para todos los documentos de esta fuente
-            filters: Filtros específicos para esta carga
+            connector_config (Dict): The configuration for the file connector.
+            predefined_metadata (Dict, optional): Metadata to be added to all documents from this source.
+            filters (Dict, optional): Filters to apply to the files.
 
         Returns:
-            int o dict: Número de archivos procesados o diccionario de error
+            int: The number of processed files.
         """
         try:
-            # Si no se proporcionaron filtros, usar el predeterminado
             if not filters:
                 filters = {"filename_contains": ".pdf"}
 
             # Pasar metadata predefinida como parte del contexto al procesador
-            # para que esté disponible en la función load_file
-            extra_context = {}
+            # para que esté disponible en la función load_file_callback
+            context = {
+                'company': company,
+                'metadata': {}
+            }
+
             if predefined_metadata:
-                extra_context['metadata'] = predefined_metadata
+                context['metadata'] = predefined_metadata
 
             # config the processor
             processor_config = FileProcessorConfig(
-                context=extra_context,
+                callback=self.load_file_callback,
+                context=context,
                 filters=filters,
-                action=self.load_file,
                 continue_on_error=True,
                 echo=True
             )
@@ -144,14 +190,21 @@ class LoadDocumentsService:
             logging.exception("Loading files error: %s", str(e))
             return {"error": str(e)}
 
-    # load an individual filename
-    # this method is set up on the FileProcessorConfig object
-    def load_file(self, filename: str, content: bytes, context: dict = {}, company: Company = None):
-        if not company:
-            company = self.company
+    def load_file_callback(self, company: Company, filename: str, content: bytes, context: dict = {}):
+        """
+        Processes a single file: extracts text, generates metadata, and saves it
+        to the relational database and the vector store.
+        This method is intended to be used as the 'action' for FileProcessor.
+
+        Args:
+            company (Company): The company associated with the file.
+            filename (str): The name of the file.
+            content (bytes): The binary content of the file.
+            context (dict, optional): A context dictionary, may contain predefined metadata.
+        """
 
         # check if file exist in repositories
-        if self.doc_repo.get(company=company,filename=filename):
+        if self.doc_repo.get(company_id=company.id,filename=filename):
             return
 
         try:

{iatoolkit-0.3.9 → iatoolkit-0.4.1}/src/services/prompt_manager_service.py

@@ -74,7 +74,8 @@ class PromptService:
                 raise IAToolkitException(IAToolkitException.ErrorType.DOCUMENT_NOT_FOUND,
                                    f"No se encontró el prompt '{prompt_name}' para la empresa '{company.short_name}'")
 
-            absolute_filepath = os.path.join(execution_dir, user_prompt.filename)
+            prompt_file = f'companies/{company.short_name}/prompts/{user_prompt.filename}'
+            absolute_filepath = os.path.join(execution_dir, prompt_file)
             if not os.path.exists(absolute_filepath):
                 raise IAToolkitException(IAToolkitException.ErrorType.FILE_IO_ERROR,
                                    f"El archivo para el prompt '{prompt_name}' no existe: {absolute_filepath}")

iatoolkit-0.3.9/readme.md

@@ -1,43 +0,0 @@
-# iatoolkit
-
-IAToolkit is a comprehensive, open-source framework designed for building enterprise-grade 
-AI chatbots and conversational applications. 
-Built on Flask with dependency injection, it provides a robust foundation for scalable AI solutions.
-
-## 🚀 Key Features
-- **Universal LLM Integration**: OpenAI GPT, Google Gemini
-- **Template System**: Jinja2-powered prompt templates with variables
-- **Context Management**: Maintain conversation context across sessions
-
-### 🔒 **Enterprise Security**
-- **JWT Authentication**: Secure token-based authentication
-- **Session Management**: Redis-backed secure sessions
-- **CORS Configuration**: Flexible cross-origin resource sharing
-
-### 🛠 **Function Calling & Tools**
-- **Native Function Calls**: Direct integration with LLM function calling
-- **Custom Tools**: Build and register custom tools for your chatbot
-- **SQL Query Generation**: Natural language to SQL conversion
-- **API Integrations**: Connect to external services and APIs
-
-### 🗄 **Database & Storage**
-- **Multi-Database Support**: PostgreSQL, MySQL, SQLite via SQLAlchemy
-- **Vector Store Integration**: Semantic search and retrieval
-- **Document Processing**: PDF, Word, Excel, and text file handling
-
-### 📊 **Analytics & Monitoring**
-- **Query Logging**: Track all LLM interactions
-- **Performance Metrics**: Response times, token usage, costs
-- **Benchmarking**: Compare model performance
-- **Task Management**: Async task processing with status tracking
-
-### 🔧 **Developer Experience**
-- **Dependency Injection**: Clean, testable architecture
-- **CLI Tools**: Command-line interface for common tasks
-- **Hot Reloading**: Development-friendly configuration
-- **Comprehensive Logging**: Debug and monitor easily
-
-## License
-MIT License
-
-