llama-github 0.1.2__tar.gz → 0.1.4__tar.gz

{llama_github-0.1.2 → llama_github-0.1.4}/CHANGELOG.md

@@ -5,6 +5,29 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.1.4] - 2024-10-14
+
+### Improved
+- Optimized `simple_mode`:
+  - Removed dependencies on `Torch` and `Transformers` libraries
+  - Reduced memory footprint
+  - Eliminated related imports
+  - Enhanced compatibility with AWS Lambda environment
+
+## [0.1.3] - 2024-10-14
+
+### Added
+- Modified `LLMManager` class to skip loading embedding and reranker models when `simple_mode` is enabled
+- Updated `retrieve_context` method to use instance's `simple_mode` by default, with option to override
+
+### Improved
+- Faster initialization process when `simple_mode` is enabled, skipping embedding and reranker model loading
+- More flexible usage of `simple_mode` in `retrieve_context`, allowing per-call customization
+
+### Developer Notes
+- When using `simple_mode=True` during GithubRAG initialization, be aware that embedding and reranking functionalities will not be available
+- The `retrieve_context` method now uses a late binding approach for `simple_mode` parameter
+
 ## [0.1.2] - 2024-10-09
 
 ### Added
@@ -49,5 +72,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Basic functionality for retrieving context from GitHub repositories
 - Integration with LLM for processing and generating responses
 
+[0.1.4]: https://github.com/JetXu-LLM/llama-github/compare/v0.1.3...v0.1.4
+[0.1.3]: https://github.com/JetXu-LLM/llama-github/compare/v0.1.2...v0.1.3
+[0.1.2]: https://github.com/JetXu-LLM/llama-github/compare/v0.1.1...v0.1.2
 [0.1.1]: https://github.com/JetXu-LLM/llama-github/compare/v0.1.0...v0.1.1
 [0.1.0]: https://github.com/JetXu-LLM/llama-github/releases/tag/v0.1.0

{llama_github-0.1.2/llama_github.egg-info → llama_github-0.1.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: llama-github
-Version: 0.1.2
+Version: 0.1.4
 Summary: Llama-github is an open-source Python library that empowers LLM Chatbots, AI Agents, and Auto-dev Agents to conduct Retrieval from actively selected GitHub public projects. It Augments through LLMs and Generates context for any coding question, in order to streamline the development of sophisticated AI-driven applications.
 Home-page: https://github.com/JetXu-LLM/llama-github
 Author: Jet Xu
@@ -133,6 +133,29 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.1.4] - 2024-10-14
+
+### Improved
+- Optimized `simple_mode`:
+  - Removed dependencies on `Torch` and `Transformers` libraries
+  - Reduced memory footprint
+  - Eliminated related imports
+  - Enhanced compatibility with AWS Lambda environment
+
+## [0.1.3] - 2024-10-14
+
+### Added
+- Modified `LLMManager` class to skip loading embedding and reranker models when `simple_mode` is enabled
+- Updated `retrieve_context` method to use instance's `simple_mode` by default, with option to override
+
+### Improved
+- Faster initialization process when `simple_mode` is enabled, skipping embedding and reranker model loading
+- More flexible usage of `simple_mode` in `retrieve_context`, allowing per-call customization
+
+### Developer Notes
+- When using `simple_mode=True` during GithubRAG initialization, be aware that embedding and reranking functionalities will not be available
+- The `retrieve_context` method now uses a late binding approach for `simple_mode` parameter
+
 ## [0.1.2] - 2024-10-09
 
 ### Added
@@ -177,5 +200,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Basic functionality for retrieving context from GitHub repositories
 - Integration with LLM for processing and generating responses
 
+[0.1.4]: https://github.com/JetXu-LLM/llama-github/compare/v0.1.3...v0.1.4
+[0.1.3]: https://github.com/JetXu-LLM/llama-github/compare/v0.1.2...v0.1.3
+[0.1.2]: https://github.com/JetXu-LLM/llama-github/compare/v0.1.1...v0.1.2
 [0.1.1]: https://github.com/JetXu-LLM/llama-github/compare/v0.1.0...v0.1.1
 [0.1.0]: https://github.com/JetXu-LLM/llama-github/releases/tag/v0.1.0

{llama_github-0.1.2 → llama_github-0.1.4}/llama_github/github_rag.py

@@ -27,8 +27,10 @@ class GitHubAppCredentials:
 
 class GithubRAG:
     rag_processor: RAGProcessor = None
+    simple_mode: bool = False
 
-    def __init__(self, github_access_token: Optional[str] = None,
+    def __init__(self,
+                 github_access_token: Optional[str] = None,
                  github_app_credentials: Optional[GitHubAppCredentials] = None,
                  openai_api_key: Optional[str] = None,
                  mistral_api_key: Optional[str] = None,
@@ -39,6 +41,7 @@ class GithubRAG:
                      "default_embedding"),
                  rerank_model: Optional[str] = config.get("default_reranker"),
                  llm: Any = None,
+                 simple_mode: bool = False,
                  **kwargs) -> None:
         """
         Initialize the GithubRAG with the provided credentials and configuration.
@@ -47,12 +50,14 @@ class GithubRAG:
         - github_access_token (Optional[str]): GitHub access token for authentication.
         - github_app_credentials (Optional[GitHubAppCredentials]): Credentials for GitHub App authentication.
         - openai_api_key (Optional[str]): API key for OpenAI services -- recommend to use, GPT-4-turbo will be used.
+        - mistral_api_key (Optional[str]): API key for Mistral AI services.
         - huggingface_token (Optional[str]): Token for Hugging Face services -- recommend to fill.
         - jina_api_key (Optional[str]): API key for Jina AI services -- s.jina.ai API will be used
         - open_source_models_hg_dir (Optional[str]): Name of open-source models from Hugging Face to replace OpenAI.
         - embedding_model (Optional[str]): Name of Embedding model from Hugging Face, if you have preferred embedding model to be used.
         - rerank_model (Optional[str]): Name of Rerank model from Hugging Face, if you have preferred rerank model to be used.
         - llm (Any): Any kind of LangChain llm chat object - to replace OpenAI or open-source models from Hugging Face.
+        - simple_mode (bool): If True, skip embedding and rerank model initialization in LLMManager.
         - **kwargs:
             :param repo_cleanup_interval (Optional[int]): How often to run repo cleanup in seconds within RepositoryPool.
             :param repo_max_idle_time (Optional[int]): Keep a repo in cache until max idle time if not used.
@@ -64,6 +69,8 @@ class GithubRAG:
             logger.info("Initializing GithubRAG...")
             logger.debug("Initializing Github Instance...")
 
+            self.simple_mode = simple_mode
+
             self.auth_manager = GitHubAuthManager()
             if github_access_token:
                 self.github_instance = self.auth_manager.authenticate_with_token(
@@ -92,22 +99,26 @@ class GithubRAG:
             logger.debug(
                 "Initializing llm manager, embedding model & reranker model...")
             self.llm_manager = LLMManager(
-                openai_api_key, mistral_api_key, huggingface_token, open_source_models_hg_dir, embedding_model, rerank_model, llm)
+                openai_api_key, mistral_api_key, huggingface_token, open_source_models_hg_dir, embedding_model, rerank_model, llm, simple_mode=self.simple_mode)
             logger.debug(
                 "LLM Manager, Embedding model & Reranker model Initialized.")
 
             self.rag_processor = RAGProcessor(
                 self.github_api_handler, self.llm_manager)
+            logger.info("GithubRAG initialization completed.")
         except Exception as e:
-            logger.error(f"Error initializing GithubRAG: {e}")
-            raise e
+            logger.error(f"Error during GithubRAG initialization: {str(e)}")
+            raise
 
-    async def async_retrieve_context(self, query, simple_mode=False) -> List[str]:
+    async def async_retrieve_context(self, query, simple_mode: Optional[bool] = None) -> List[str]:
         # Implementation of the context retrieval process
         # This will involve using the GitHub API to search for relevant information,
         # augmenting the retrieved data through the RAG methodology, and
         # enhancing it with LLM capabilities.
 
+        if simple_mode is None:
+            simple_mode = self.simple_mode
+
         topn_contexts = []  # This will be the list of context strings
         try:
             logger.info("Retrieving context...")
@@ -176,26 +187,29 @@ class GithubRAG:
             raise e
         return topn_contexts
 
-    def retrieve_context(self, query, simple_mode=False) -> List[str]:
+    def retrieve_context(self, query, simple_mode: Optional[bool] = None) -> List[str]:
         """
         Retrieve context from GitHub code, issue and repo search based on the input query.
 
         Args:
             query (str): The query or question to retrieve context for.
+            simple_mode (Optional[bool]): If provided, overrides the instance's simple_mode setting.
 
         Returns:
             List[str]: A list of context strings retrieved from the specified GitHub repositories.
         """
+        effective_simple_mode = self.simple_mode if simple_mode is None else simple_mode
+
         self.loop = asyncio.get_event_loop()
         ipython = get_ipython()
         if ipython and ipython.has_trait('kernel'):
             logger.debug("Running in Jupyter notebook, nest_asyncio applied.")
             import nest_asyncio
             nest_asyncio.apply()
-            return asyncio.run(self.async_retrieve_context(query, simple_mode=simple_mode))
+            return asyncio.run(self.async_retrieve_context(query, simple_mode=effective_simple_mode))
         if self.loop.is_running():
-            return asyncio.ensure_future(self.async_retrieve_context(query, simple_mode=simple_mode))
-        return self.loop.run_until_complete(self.async_retrieve_context(query, simple_mode=simple_mode))
+            return asyncio.ensure_future(self.async_retrieve_context(query, simple_mode=effective_simple_mode))
+        return self.loop.run_until_complete(self.async_retrieve_context(query, simple_mode=effective_simple_mode))
 
     async def code_search_retrieval(self, query, draft_answer: Optional[str] = None):
         result = []

{llama_github-0.1.2 → llama_github-0.1.4}/llama_github/llm_integration/initial_load.py

@@ -1,5 +1,4 @@
 # initial_load.py
-import torch
 from typing import Optional, Any
 from threading import Lock
 from langchain_openai import ChatOpenAI
@@ -10,20 +9,15 @@ from langchain_mistralai.chat_models import ChatMistralAI
 from llama_github.config.config import config
 from llama_github.logger import logger
 
-from transformers import AutoModel
-from transformers import AutoModelForSequenceClassification
-from transformers import AutoTokenizer
-
-
 class LLMManager:
     _instance_lock = Lock()
     _instance = None
     llm = None
-    # embedding_model = None
     rerank_model = None
     _initialized = False
     llm_simple = None
     tokenizer = None
+    embedding_model = None
 
     def __new__(cls, *args, **kwargs):
         if cls._instance is None:  # First check (unlocked)
@@ -40,12 +34,15 @@ class LLMManager:
                  embedding_model: Optional[str] = config.get(
                      "default_embedding"),
                  rerank_model: Optional[str] = config.get("default_reranker"),
-                 llm: Any = None):
+                 llm: Any = None,
+                 simple_mode: bool = False):
         with self._instance_lock:   # Prevent re-initialization
             if self._initialized:
                 return
             self._initialized = True
 
+        self.simple_mode = simple_mode
+
         # Initialize for OpenAI GPT-4
         if llm is not None:
             self.llm = llm
@@ -69,28 +66,35 @@ class LLMManager:
         elif self.llm is None:
             # default model is phi3_mini_128k
             self.model_type = "Hubgingface"
+            
+        if not self.simple_mode:
+            import torch
+            from transformers import AutoModel
+            from transformers import AutoModelForSequenceClassification
+            from transformers import AutoTokenizer
+            # initial model_kwargs
+            if torch.cuda.is_available():
+                self.device = torch.device('cuda')
+            elif torch.backends.mps.is_available():
+                self.device = torch.device('mps')
+            else:
+                self.device = torch.device('cpu')
 
-        # initial model_kwargs
-        if torch.cuda.is_available():
-            self.device = torch.device('cuda')
-        elif torch.backends.mps.is_available():
-            self.device = torch.device('mps')
-        else:
-            self.device = torch.device('cpu')
-
-        # initial embedding_model
-        if self.tokenizer is None:
-            logger.info(f"Initializing {embedding_model}...")
-            self.tokenizer = AutoTokenizer.from_pretrained(embedding_model)
-            self.embedding_model = AutoModel.from_pretrained(
-                embedding_model, trust_remote_code=True).to(self.device)
+            # initial embedding_model
+            if self.tokenizer is None:
+                logger.info(f"Initializing {embedding_model}...")
+                self.tokenizer = AutoTokenizer.from_pretrained(embedding_model)
+                self.embedding_model = AutoModel.from_pretrained(
+                    embedding_model, trust_remote_code=True).to(self.device)
 
-        # initial rerank_model
-        if self.rerank_model is None:
-            logger.info(f"Initializing {rerank_model}...")
-            self.rerank_model = AutoModelForSequenceClassification.from_pretrained(
-                rerank_model, num_labels=1, trust_remote_code=True
-            ).to(self.device)
+            # initial rerank_model
+            if self.rerank_model is None:
+                logger.info(f"Initializing {rerank_model}...")
+                self.rerank_model = AutoModelForSequenceClassification.from_pretrained(
+                    rerank_model, num_labels=1, trust_remote_code=True
+                ).to(self.device)
+        else:
+            logger.info("Simple mode enabled. Skipping embedding and rerank model initialization.")
 
     def get_llm(self):
         return self.llm
@@ -105,4 +109,4 @@ class LLMManager:
         return self.rerank_model
 
     def get_embedding_model(self):
-        return self.embedding_model
+        return self.embedding_model

llama_github-0.1.4/llama_github/version.py

@@ -0,0 +1 @@
+__version__ = '0.1.4'

{llama_github-0.1.2 → llama_github-0.1.4/llama_github.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: llama-github
-Version: 0.1.2
+Version: 0.1.4
 Summary: Llama-github is an open-source Python library that empowers LLM Chatbots, AI Agents, and Auto-dev Agents to conduct Retrieval from actively selected GitHub public projects. It Augments through LLMs and Generates context for any coding question, in order to streamline the development of sophisticated AI-driven applications.
 Home-page: https://github.com/JetXu-LLM/llama-github
 Author: Jet Xu
@@ -133,6 +133,29 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.1.4] - 2024-10-14
+
+### Improved
+- Optimized `simple_mode`:
+  - Removed dependencies on `Torch` and `Transformers` libraries
+  - Reduced memory footprint
+  - Eliminated related imports
+  - Enhanced compatibility with AWS Lambda environment
+
+## [0.1.3] - 2024-10-14
+
+### Added
+- Modified `LLMManager` class to skip loading embedding and reranker models when `simple_mode` is enabled
+- Updated `retrieve_context` method to use instance's `simple_mode` by default, with option to override
+
+### Improved
+- Faster initialization process when `simple_mode` is enabled, skipping embedding and reranker model loading
+- More flexible usage of `simple_mode` in `retrieve_context`, allowing per-call customization
+
+### Developer Notes
+- When using `simple_mode=True` during GithubRAG initialization, be aware that embedding and reranking functionalities will not be available
+- The `retrieve_context` method now uses a late binding approach for `simple_mode` parameter
+
 ## [0.1.2] - 2024-10-09
 
 ### Added
@@ -177,5 +200,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Basic functionality for retrieving context from GitHub repositories
 - Integration with LLM for processing and generating responses
 
+[0.1.4]: https://github.com/JetXu-LLM/llama-github/compare/v0.1.3...v0.1.4
+[0.1.3]: https://github.com/JetXu-LLM/llama-github/compare/v0.1.2...v0.1.3
+[0.1.2]: https://github.com/JetXu-LLM/llama-github/compare/v0.1.1...v0.1.2
 [0.1.1]: https://github.com/JetXu-LLM/llama-github/compare/v0.1.0...v0.1.1
 [0.1.0]: https://github.com/JetXu-LLM/llama-github/releases/tag/v0.1.0

{llama_github-0.1.2 → llama_github-0.1.4}/setup.cfg

@@ -1,6 +1,6 @@
 [metadata]
 name = llama-github
-version = 0.1.2
+version = 0.1.4
 author = Jet Xu
 author_email = Voldemort.xu@foxmail.com
 description = Llama-github is an open-source Python library that empowers LLM Chatbots, AI Agents, and Auto-dev Agents to conduct Retrieval from actively selected GitHub public projects. It Augments through LLMs and Generates context for any coding question, in order to streamline the development of sophisticated AI-driven applications.

llama_github-0.1.2/llama_github/version.py

@@ -1 +0,0 @@
-__version__ = '0.1.2'