llmcall 0.1.0rc1__tar.gz

llmcall-0.1.0rc1/PKG-INFO

@@ -0,0 +1,111 @@
+Metadata-Version: 2.1
+Name: llmcall
+Version: 0.1.0rc1
+Summary: A lite abstraction layer for LLM calls
+Home-page: https://github.com/rihoneailabs/llmcall
+License: Apache-2.0
+Keywords: llm,ai,litellm,structure-outputs,openai,pydantic
+Author: Ndamulelo Nemakhavhani
+Author-email: info@rihonegroup.com
+Requires-Python: >=3.11,<4.0
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Requires-Dist: environs (>=11.2.1,<12.0.0)
+Requires-Dist: litellm (>=1.56.4,<2.0.0)
+Requires-Dist: openai (>=1.58.1,<2.0.0)
+Requires-Dist: pydantic (>=2.0.0)
+Requires-Dist: pydantic-settings (>=2.0.0)
+Requires-Dist: tenacity (>=9.0.0,<10.0.0)
+Project-URL: Repository, https://github.com/rihoneailabs/llmcall.git
+Description-Content-Type: text/markdown
+
+# LLMCall
+
+A lite abstraction layer for LLM calls.
+
+## Motivation
+
+As AI becomes more prevalent in software development, there's a growing need for simple and intuitive APIs for interacting with AI for quick text generation, decision making, and more. This is especially important now that we have structured outputs, which allow us to seamlessly integrate AI into our application flow.
+
+`llmcall` provides a minimal, batteries-included interface for common LLM operations without unnecessary complexity.
+
+## Installation
+
+```bash
+pip install llmcall
+```
+
+## Example Usage
+
+### Generation
+
+```python
+from llmcall import generate, generate_decision
+from pydantic import BaseModel
+
+# i. Basic generation
+response = generate("Write a story about a fictional holiday to the sun.")
+
+# ii. Structured generation
+class ResponseSchema(BaseModel):
+    story: str
+    tags: list[str]
+    
+response: ResponseSchema = generate("Create a rare story about the history of civilisation.", output_schema=schema)
+
+# iii. Decision making
+decision = generate_decision(
+    "Which is bigger?",
+    options=["apple", "berry", "pumpkin"]
+)
+```
+
+### Extraction
+
+```python
+from llmcall import extract
+from pydantic import BaseModel
+
+class ResponseSchema(BaseModel):
+    email_subject: str
+    email_body: str
+    email_topic: str
+    email_sentiment: str
+
+text = """To whom it may concern,
+
+Request for Admission at Harvard University
+
+I write to plead with the admission board to consider my application for the 2022/2023 academic year. I am a dedicated student with a passion for computer science and a strong desire to make a difference in the world. I believe that Harvard University is the perfect place for me to achieve my dreams and make a positive impact on society."""
+
+response: ResponseSchema = extract(text=text, output_schema=ResponseSchema)
+```
+
+## Configuration
+
+Set environment variables:
+- LLMCALL_API_KEY: Your API key
+- LLMCALL_MODEL: Model to use (default: `openai/gpt-4o-2024-08-06`)
+
+> **Note**: We recommend using `Open AI` as the model provider due to their robust support for structured outputs. You can use other providers by setting the `LLMCALL_MODEL` or changing the [config](./llmcall/core.py) directly. Any model supported by `LiteLLM` can be used.
+
+## Roadmap
+
+- [x] Simple API for generating unstructured text
+- [x] Structured output generation using `Pydantic`
+- [x] Decision making
+- [x] Custom model selection (via `LiteLLM` - See [documentation](https://docs.litellm.ai/docs/providers))
+- [x] Structured text extraction
+- [ ] Structured text extraction from PDF, Docx, etc.
+- [ ] Structured text extraction from Images
+- [ ] Structured text extraction from Websites
+
+## Documentation
+
+Please refer to our comprehensive [documentation](./docs/index.md) to learn more about this tool.
+

llmcall-0.1.0rc1/README.md

@@ -0,0 +1,84 @@
+# LLMCall
+
+A lite abstraction layer for LLM calls.
+
+## Motivation
+
+As AI becomes more prevalent in software development, there's a growing need for simple and intuitive APIs for interacting with AI for quick text generation, decision making, and more. This is especially important now that we have structured outputs, which allow us to seamlessly integrate AI into our application flow.
+
+`llmcall` provides a minimal, batteries-included interface for common LLM operations without unnecessary complexity.
+
+## Installation
+
+```bash
+pip install llmcall
+```
+
+## Example Usage
+
+### Generation
+
+```python
+from llmcall import generate, generate_decision
+from pydantic import BaseModel
+
+# i. Basic generation
+response = generate("Write a story about a fictional holiday to the sun.")
+
+# ii. Structured generation
+class ResponseSchema(BaseModel):
+    story: str
+    tags: list[str]
+    
+response: ResponseSchema = generate("Create a rare story about the history of civilisation.", output_schema=schema)
+
+# iii. Decision making
+decision = generate_decision(
+    "Which is bigger?",
+    options=["apple", "berry", "pumpkin"]
+)
+```
+
+### Extraction
+
+```python
+from llmcall import extract
+from pydantic import BaseModel
+
+class ResponseSchema(BaseModel):
+    email_subject: str
+    email_body: str
+    email_topic: str
+    email_sentiment: str
+
+text = """To whom it may concern,
+
+Request for Admission at Harvard University
+
+I write to plead with the admission board to consider my application for the 2022/2023 academic year. I am a dedicated student with a passion for computer science and a strong desire to make a difference in the world. I believe that Harvard University is the perfect place for me to achieve my dreams and make a positive impact on society."""
+
+response: ResponseSchema = extract(text=text, output_schema=ResponseSchema)
+```
+
+## Configuration
+
+Set environment variables:
+- LLMCALL_API_KEY: Your API key
+- LLMCALL_MODEL: Model to use (default: `openai/gpt-4o-2024-08-06`)
+
+> **Note**: We recommend using `Open AI` as the model provider due to their robust support for structured outputs. You can use other providers by setting the `LLMCALL_MODEL` or changing the [config](./llmcall/core.py) directly. Any model supported by `LiteLLM` can be used.
+
+## Roadmap
+
+- [x] Simple API for generating unstructured text
+- [x] Structured output generation using `Pydantic`
+- [x] Decision making
+- [x] Custom model selection (via `LiteLLM` - See [documentation](https://docs.litellm.ai/docs/providers))
+- [x] Structured text extraction
+- [ ] Structured text extraction from PDF, Docx, etc.
+- [ ] Structured text extraction from Images
+- [ ] Structured text extraction from Websites
+
+## Documentation
+
+Please refer to our comprehensive [documentation](./docs/index.md) to learn more about this tool.

llmcall-0.1.0rc1/llmcall/__init__.py

@@ -0,0 +1,5 @@
+from llmcall.generate import generate, generate_decision
+from llmcall.extract import extract
+from llmcall.core import LLMConfig, config
+
+__all__ = ["generate", "extract", "generate_decision", "config", "LLMConfig"]

llmcall-0.1.0rc1/llmcall/core.py

@@ -0,0 +1,26 @@
+from typing import Optional
+from pydantic_settings import BaseSettings, SettingsConfigDict
+
+
+class ModelConfig(BaseSettings):
+    temperature: float = 0.2
+    stream: bool = False
+    n: Optional[int] = 1
+    max_tokens: int = 1024
+    num_retries: int = 3
+    seed: Optional[int] = 47
+
+
+class LLMConfig(BaseSettings):
+    model_config = SettingsConfigDict(
+        case_sensitive=False,
+        env_prefix="LLMCALL_",
+        extra="ignore",
+    )
+    api_key: str
+    model: str = "openai/gpt-4o-2024-08-06"
+    debug: bool = False
+    llm: ModelConfig = ModelConfig()
+
+
+config = LLMConfig()

llmcall-0.1.0rc1/llmcall/extract.py

@@ -0,0 +1,53 @@
+import json
+import logging
+import time
+from typing import Optional, Union
+from typing_extensions import Annotated
+
+from litellm import completion
+from pydantic import BaseModel
+
+from llmcall.core import config
+
+_logger = logging.getLogger(__name__)
+
+
+def extract(
+    text: Annotated[str, "The unstructured text to extract information from."],
+    output_schema: Annotated[BaseModel, "The Pydantic model to use for response structure validation."],
+    instructions: Annotated[Optional[str], "System metaprompt to condition the model."] = None,
+) -> Union[str, BaseModel]:
+    """Extract structured information from unstructured text using configured LLM."""
+    DEFAULT_SYSTEM_PROMPT = """You are a specialist in organising unstructured data. Given the document below, \
+    your task is to extract the requested information as accurately as you can. \
+    ###
+    <document>{text}</document>
+    ###"""
+
+    if not text:
+        raise ValueError("Text cannot be empty.")
+    
+    start = time.perf_counter()
+    _logger.info(f"Extracting information from text: {text[:50]}")
+
+    if instructions:
+        messages = [
+            {"content": instructions, "role": "system"},
+            {"content": text, "role": "user"},
+        ]
+    else:
+        messages = [
+            {"content": DEFAULT_SYSTEM_PROMPT.format(text=text), "role": "system"},
+        ]
+
+    response = completion(
+        api_key=config.api_key,
+        model=config.model,
+        messages=messages,
+        response_format=output_schema,
+        **config.llm.model_dump(),
+    )
+
+    _logger.info(f"Extraction completed in {time.perf_counter() - start:.2f} seconds.")
+
+    return output_schema.model_validate(json.loads(response.choices[0].message.content), strict=True)

llmcall-0.1.0rc1/llmcall/extract_utils.py

@@ -0,0 +1,20 @@
+
+def extract_from_doc(pdf_path, page_num, pdf_type):
+    """
+    Extracts text from a PDF file.
+    """
+    raise NotImplementedError
+
+
+def extract_from_image(image_path):
+    """
+    Extracts text from an image file.
+    """
+    raise NotImplementedError
+
+
+def extract_from_webpage(url):
+    """
+    Extracts text from a webpage.
+    """
+    raise NotImplementedError

llmcall-0.1.0rc1/llmcall/generate.py

@@ -0,0 +1,125 @@
+import json
+import logging
+import time
+from typing import Optional, Union
+from typing_extensions import Annotated
+
+import litellm
+from litellm import completion
+from litellm import supports_response_schema
+from pydantic import BaseModel
+
+from llmcall.core import config
+
+_logger = logging.getLogger(__name__)
+
+
+class Decision(BaseModel):
+    selection: Annotated[str, "The selected option - MUST be one of the provided options."]
+    prompt: Optional[str] = None
+    options: Optional[list[str]] = None
+    reason: Optional[str] = None
+
+
+def generate(
+    prompt: Annotated[str, "The user prompt which tells the model what to generate."],
+    output_schema: Annotated[
+        Optional[BaseModel], "The Pydantic model to use for response structure validation(optional)"
+    ] = None,
+    instructions: Annotated[Optional[str], "System metaprompt to condition the model."] = None,
+) -> Union[str, BaseModel]:
+    """Generate content using configured LLM."""
+
+    if not prompt:
+        raise ValueError("Prompt cannot be empty.")
+
+    DEFAULT_SYSTEM_PROMPT = "Generate content based on the following: <prompt>{prompt}</prompt>. \
+        Return only the content with no additional information or comments."
+
+    _logger.debug(f"Generating content for prompt: {prompt[:20]}..")
+    start = time.perf_counter()
+
+    if output_schema:
+        if not supports_response_schema(
+            model=config.model.split("/")[1], custom_llm_provider=config.model.split("/")[0]
+        ):
+            raise ValueError(
+                f"Response schema is not supported by the configured model: {config.model}. "
+                "Please use a different model(e.g. openai/gpt-4o-2024-08-06) or remove the output schema."
+            )
+        litellm.enable_json_schema_validation = True
+
+    response = completion(
+        api_key=config.api_key,
+        model=config.model,
+        messages=[
+            {"content": instructions or DEFAULT_SYSTEM_PROMPT, "role": "system"},
+            {"content": prompt, "role": "user"},
+        ],
+        response_format=output_schema,
+        **config.llm.model_dump(),
+    )
+
+    _logger.debug(f"Generated content in {time.perf_counter() - start:.2f}s")
+
+    if output_schema:
+        return output_schema.model_validate(json.loads(response.choices[0].message.content), strict=True)
+
+    return response.choices[0].message.content
+
+
+def generate_decision(
+    prompt: Annotated[str, "The context to consider when making the decision."],
+    options: Annotated[list[str], "List of options to choose from."],
+    instructions: Annotated[Optional[str], "System metaprompt to condition the model."] = None,
+) -> Decision:
+    """Generate a decision from a list of options."""
+
+    if not prompt:
+        raise ValueError("Prompt cannot be empty.")
+
+    DEFAULT_SYSTEM_PROMPT = """You are a specialized computer algorithm designed to make decisions in Control Flow scenarios. \
+        Your task is to analyze the given context and options, then select the most appropriate option based on the context. Here is the context you need to consider: \
+            <context>
+            {{CONTEXT}}
+            </context>
+
+        Here are the options you can choose from:
+            <options>
+            {{OPTIONS}}
+            </options>"""
+
+    _logger.debug(f"Generating decision given options: {options} and prompt: {prompt[:20]}..")
+    start = time.perf_counter()
+
+    if instructions:
+        messages = [
+            {
+                "content": instructions,
+                "role": "system",
+            },
+            {
+                "content": "Pick one of the following options: <options>{options}</options>, given the following query:\n<query>{prompt}</query>.",
+                "role": "user",
+            },
+        ]
+    else:
+        messages = [
+            {
+                "content": DEFAULT_SYSTEM_PROMPT.strip()
+                .replace("{{CONTEXT}}", prompt)
+                .replace("{{OPTIONS}}", "\n".join(options)),
+                "role": "user",
+            },
+        ]
+
+    response = completion(
+        api_key=config.api_key,
+        model=config.model,
+        messages=messages,
+        response_format=Decision,
+        **config.llm.model_dump(),
+    )
+
+    _logger.debug(f"Generated decision in {time.perf_counter() - start:.2f}s")
+    return Decision.model_validate(json.loads(response.choices[0].message.content), strict=True)

llmcall-0.1.0rc1/pyproject.toml

@@ -0,0 +1,40 @@
+[tool.poetry]
+name = "llmcall"
+version = "0.1.0rc1"
+description = "A lite abstraction layer for LLM calls"
+authors = ["Ndamulelo Nemakhavhani <info@rihonegroup.com>"]
+license = "Apache-2.0"
+readme = "README.md"
+keywords = ["llm", "ai", "litellm", "structure-outputs", "openai", "pydantic"]
+repository = "https://github.com/rihoneailabs/llmcall.git"
+homepage = "https://github.com/rihoneailabs/llmcall"
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: Apache Software License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+]
+
+[tool.poetry.dependencies]
+python = ">=3.11, <4.0"
+pydantic = ">=2.0.0"
+pydantic-settings = ">=2.0.0"
+openai = "^1.58.1"
+litellm = "^1.56.4"
+environs = "^11.2.1"
+tenacity = "^9.0.0"
+
+[tool.poetry.group.dev.dependencies]
+pytest = "^8.3.4"
+black = "^24.10.0"
+isort = "^5.13.2"
+mypy = "^1.14.0"
+tox = "^4.23.2"
+pytest-cov = "^6.0.0"
+
+[build-system]
+requires = ["poetry-core"]
+build-backend = "poetry.core.masonry.api"