jehoctor-rag-demo 0.1.1.dev1__py3-none-any.whl → 0.2.1__py3-none-any.whl

jehoctor_rag_demo-0.2.1.dist-info/METADATA

@@ -0,0 +1,125 @@
+Metadata-Version: 2.3
+Name: jehoctor-rag-demo
+Version: 0.2.1
+Summary: Chat with Wikipedia
+Author: James Hoctor
+Author-email: James Hoctor <JEHoctor@protonmail.com>
+Requires-Dist: aiosqlite==0.21.0
+Requires-Dist: bitsandbytes>=0.49.1
+Requires-Dist: chromadb>=1.3.4
+Requires-Dist: datasets>=4.4.1
+Requires-Dist: httpx>=0.28.1
+Requires-Dist: huggingface-hub>=0.36.0
+Requires-Dist: langchain>=1.0.5
+Requires-Dist: langchain-anthropic>=1.0.2
+Requires-Dist: langchain-community>=0.4.1
+Requires-Dist: langchain-huggingface>=1.1.0
+Requires-Dist: langchain-ollama>=1.0.0
+Requires-Dist: langchain-openai>=1.0.2
+Requires-Dist: langgraph-checkpoint-sqlite>=3.0.1
+Requires-Dist: nvidia-ml-py>=13.590.44
+Requires-Dist: ollama>=0.6.0
+Requires-Dist: platformdirs>=4.5.0
+Requires-Dist: psutil>=7.1.3
+Requires-Dist: py-cpuinfo>=9.0.0
+Requires-Dist: pydantic>=2.12.4
+Requires-Dist: pyperclip>=1.11.0
+Requires-Dist: sentence-transformers>=5.2.2
+Requires-Dist: textual>=6.5.0
+Requires-Dist: transformers[torch]>=4.57.6
+Requires-Dist: typer>=0.20.0
+Requires-Dist: llama-cpp-python>=0.3.16 ; extra == 'llamacpp'
+Requires-Python: ~=3.12.0
+Provides-Extra: llamacpp
+Description-Content-Type: text/markdown
+
+# RAG-demo
+
+Chat with (a small portion of) Wikipedia
+
+⚠️ RAG functionality is still under development. ⚠️
+
+![app screenshot](screenshots/screenshot_0.2.0.png "App screenshot")
+
+## Requirements
+
+ 1. The [uv](https://docs.astral.sh/uv/) Python package manager
+    - Installing and updating `uv` is easy by following [the docs](https://docs.astral.sh/uv/getting-started/installation/).
+    - As of 2026-01-25, I'm developing using `uv` version 0.9.26, and using the new experimental `--pytorch-backend` option.
+ 2. A terminal emulator or web browser
+    - Any common web browser will work.
+    - Some terminal emulators will work better than others.
+      See [Notes on terminal emulators](#notes-on-terminal-emulators) below.
+
+### Notes on terminal emulators
+
+Certain terminal emulators will not work with some features of this program.
+In particular, on macOS consider using [iTerm2](https://iterm2.com/) instead of the default Terminal.app ([explanation](https://textual.textualize.io/FAQ/#why-doesnt-textual-look-good-on-macos)).
+On Linux you might want to try [kitty](https://sw.kovidgoyal.net/kitty/), [wezterm](https://wezterm.org/), [alacritty](https://alacritty.org/), or [ghostty](https://ghostty.org/), instead of the terminal that came with your desktop environment ([reason](https://darren.codes/posts/textual-copy-paste/)).
+Windows Terminal should be fine as far as I know.
+
+### Optional dependencies
+
+ 1. [Hugging Face login](https://huggingface.co/docs/huggingface_hub/quick-start#login)
+ 2. API key for your favorite LLM provider (support coming soon)
+ 3. Ollama installed on your system if you have a GPU
+ 4. Run RAG-demo on a more capable (bigger GPU) machine over SSH if you can. It is a terminal app after all.
+ 5. A C compiler if you want to build Llama.cpp from source.
+
+## Run the latest version
+
+Run in a terminal:
+```bash
+uvx --torch-backend=auto --from=jehoctor-rag-demo@latest chat
+```
+
+Or run in a web browser:
+```bash
+uvx --torch-backend=auto --from=jehoctor-rag-demo@latest textual serve chat
+```
+
+## CUDA acceleration via Llama.cpp
+
+If you have an NVIDIA GPU with CUDA and build tools installed, you might be able to get CUDA acceleration without installing Ollama.
+
+```bash
+CMAKE_ARGS="-DGGML_CUDA=on" uv run --extra=llamacpp chat
+```
+
+## Metal acceleration via Llama.cpp (on Apple Silicon)
+
+On an Apple Silicon machine, make sure `uv` runs an ARM interpreter as this should cause it to install Llama.cpp with Metal support.
+Also, run with the extra group `llamacpp`.
+Try this:
+
+```bash
+uvx --python-platform=aarch64-apple-darwin --torch-backend=auto --from=jehoctor-rag-demo[llamacpp]@latest chat
+```
+
+## Ollama on Linux
+
+Remember that you have to keep Ollama up-to-date manually on Linux.
+A recent version of Ollama (v0.11.10 or later) is required to run the [embedding model we use](https://ollama.com/library/embeddinggemma).
+See this FAQ: https://docs.ollama.com/faq#how-can-i-upgrade-ollama.
+
+## Project feature roadmap
+
+- ❌ RAG functionality
+- ❌ torch inference via the Langchain local Hugging Face inference integration
+- ❌ uv automatic torch backend selection (see [the docs](https://docs.astral.sh/uv/guides/integration/pytorch/#automatic-backend-selection))
+- ❌ OpenAI integration
+- ❌ Anthropic integration
+
+## Run from the repository
+
+First, clone this repository. Then, run one of the options below.
+
+Run in a terminal:
+```bash
+uv run chat
+```
+
+Or run in a web browser:
+```bash
+uv run textual serve chat
+```

jehoctor_rag_demo-0.2.1.dist-info/RECORD

@@ -0,0 +1,31 @@
+rag_demo/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+rag_demo/__main__.py,sha256=S0UlQj3EldcXRk3rhH3ONdSOPmeyITYXIZ0o2JSxWbg,1618
+rag_demo/agents/__init__.py,sha256=dsuO3AGcn2yGDq4gkAsZ32pjeTOqAudOL14G_AsEUyc,221
+rag_demo/agents/base.py,sha256=gib6bC8nVKN1s1KPZd1dJVGRXnu7gFQwf3I3_7TSjQo,1312
+rag_demo/agents/hugging_face.py,sha256=VrbGOlMO2z357LmU3sO5aM_yI5P-xbsfKmTAzH9_lFo,4225
+rag_demo/agents/llama_cpp.py,sha256=C0hInc24sXmt5407_k4mP2Y6svqgfUPemhzAL1N6jY0,4272
+rag_demo/agents/ollama.py,sha256=Fmtu8MSPPz91eT7HKvwvbQnA_xGPaD5HBrHEPPtomZA,3317
+rag_demo/app.py,sha256=AVCJjlQ60y5J0v50TcJ3zZoa0ubhd_yKVDfu1ERsMVU,1807
+rag_demo/app.tcss,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+rag_demo/app_protocol.py,sha256=P__Q3KT41uonYazpYmLWmOh1MeoBaCBx4xEydkwK5tk,3292
+rag_demo/constants.py,sha256=5EpyAD6p5Qb0vB5ASMtFSLVVsq3RG5cBn_AYihhDCPc,235
+rag_demo/db.py,sha256=53n662Hj9sTqPNcCI2Q-6Ca_HXv3kBQdAhXU4DLhwBM,3226
+rag_demo/dirs.py,sha256=b0VR76kXRHSRWzaXzmAhfPr3-8WKY3ZLW8aLlaPI3Do,309
+rag_demo/logic.py,sha256=SkF_Hqu1WSLHzwvSd_mJiCMSxZYqDnteYFRpc6oCREY,8236
+rag_demo/markdown.py,sha256=CxzshWfANeiieZkzMlLzpRaz7tBY2_tZQxhs7b2ImKM,551
+rag_demo/modes/__init__.py,sha256=ccvURDWz51_IotzzlO2OH3i4_Ih_MgnGlOK_JCh45dY,91
+rag_demo/modes/_logic_provider.py,sha256=U3J8Fgq8MbNYd92FqENW-5YP_jXqKG3xmMmYoSUzhHo,1343
+rag_demo/modes/chat.py,sha256=2pmKhQ2uYZdjezNnNBINViBMcuTVE5YCom_HEbdJeXg,13607
+rag_demo/modes/chat.tcss,sha256=YANlgYygiOr-e61N9HaGGdRPM36pdr-l4u72G0ozt4o,1032
+rag_demo/modes/config.py,sha256=0A8IdY-GOeqCd0kMs2KMgQEsFFeVXEcnowOugtR_Q84,2609
+rag_demo/modes/config.tcss,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+rag_demo/modes/help.py,sha256=riV8o4WDtsim09R4cRi0xkpYLgj4CL38IrjEz_mrRmk,713
+rag_demo/modes/help.tcss,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+rag_demo/probe.py,sha256=aDD-smNauEXXoBKVgx5xsMawM5tL0QAEBFl07ZGrddc,5101
+rag_demo/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+rag_demo/widgets/__init__.py,sha256=JQ1KQjdYQ4texHw2iT4IyBKgTW0SzNYbNoHAbrdCwtk,44
+rag_demo/widgets/escapable_input.py,sha256=VfFij4NOtQ4uX3YFETg5YPd0_nBMky9Xz-02oRdHu-w,4240
+jehoctor_rag_demo-0.2.1.dist-info/WHEEL,sha256=eh7sammvW2TypMMMGKgsM83HyA_3qQ5Lgg3ynoecH3M,79
+jehoctor_rag_demo-0.2.1.dist-info/entry_points.txt,sha256=-nDSFVcIqdTxzYM4fdveDk3xUKRhmlr_cRuqQechYh4,49
+jehoctor_rag_demo-0.2.1.dist-info/METADATA,sha256=nCXuy3TYPPf67DPFryndW2P6CRdSCcJkXxFXZ-UN4vs,4650
+jehoctor_rag_demo-0.2.1.dist-info/RECORD,,

jehoctor_rag_demo-0.2.1.dist-info/entry_points.txt

@@ -0,0 +1,3 @@
+[console_scripts]
+chat = rag_demo.__main__:main
+

rag_demo/__init__.py

@@ -1,2 +0,0 @@
-def main() -> None:
-    print("Hello from rag-demo!")

rag_demo/__main__.py

@@ -0,0 +1,42 @@
+import time
+
+# Measure the application start time.
+APPLICATION_START_TIME = time.time()
+
+# Disable "module import not at top of file" (aka E402) when importing Typer and other early imports. This is necessary
+# so that the initialization of these modules is included in the application startup time.
+from typing import Annotated  # noqa: E402
+
+import typer  # noqa: E402
+
+from rag_demo.constants import LocalProviderType  # noqa: E402
+
+
+def _main(
+    name: Annotated[str | None, typer.Option(help="The name you want to want the AI to use with you.")] = None,
+    provider: Annotated[LocalProviderType | None, typer.Option(help="The local provider to prefer.")] = None,
+) -> None:
+    """Talk to Wikipedia."""
+    # Import here so that imports run within the typer.run context for prettier stack traces if errors occur.
+    # We ignore PLC0415 because we do not want these imports to be at the top of the module as is usually preferred.
+    import transformers  # noqa: PLC0415
+
+    from rag_demo.app import RAGDemo  # noqa: PLC0415
+    from rag_demo.logic import Logic  # noqa: PLC0415
+
+    # The transformers library likes to print text that interferes with the TUI. Disable it.
+    transformers.logging.set_verbosity(verbosity=transformers.logging.CRITICAL)
+    transformers.logging.disable_progress_bar()
+
+    logic = Logic(username=name, preferred_provider_type=provider, application_start_time=APPLICATION_START_TIME)
+    app = RAGDemo(logic)
+    app.run()
+
+
+def main() -> None:
+    """Entrypoint for the rag demo, specifically the `chat` command."""
+    typer.run(_main)
+
+
+if __name__ == "__main__":
+    main()

rag_demo/agents/__init__.py

@@ -0,0 +1,4 @@
+from .base import Agent, AgentProvider
+from .hugging_face import HuggingFaceAgent, HuggingFaceAgentProvider
+from .llama_cpp import LlamaCppAgent, LlamaCppAgentProvider
+from .ollama import OllamaAgent, OllamaAgentProvider

rag_demo/agents/base.py

@@ -0,0 +1,40 @@
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Final, Protocol
+
+if TYPE_CHECKING:
+    from collections.abc import AsyncIterator
+    from contextlib import AbstractAsyncContextManager
+    from pathlib import Path
+
+    from rag_demo.app_protocol import AppProtocol
+    from rag_demo.constants import LocalProviderType
+
+
+class Agent(Protocol):
+    """An LLM agent that supports streaming responses asynchronously."""
+
+    def astream(self, user_message: str, thread_id: str, app: AppProtocol) -> AsyncIterator[str]:
+        """Stream a response from the agent.
+
+        Args:
+            user_message (str): User's next prompt in the conversation.
+            thread_id (str): Identifier for the current thread/conversation.
+            app (AppProtocol): Application interface, commonly used for logging.
+
+        Yields:
+            str: A token from the agent's response.
+        """
+
+
+class AgentProvider(Protocol):
+    """A strategy for creating LLM agents."""
+
+    type: Final[LocalProviderType]
+
+    def get_agent(self, checkpoints_sqlite_db: str | Path) -> AbstractAsyncContextManager[Agent | None]:
+        """Attempt to create an agent.
+
+        Args:
+            checkpoints_sqlite_db (str | Path): Connection string for SQLite database used for LangChain checkpoints.
+        """

rag_demo/agents/hugging_face.py

@@ -0,0 +1,116 @@
+from __future__ import annotations
+
+import asyncio
+import sqlite3
+from contextlib import asynccontextmanager
+from typing import TYPE_CHECKING, Final
+
+from huggingface_hub import hf_hub_download
+from langchain.agents import create_agent
+from langchain.messages import AIMessageChunk, HumanMessage
+from langchain_huggingface import ChatHuggingFace, HuggingFaceEmbeddings, HuggingFacePipeline
+from langgraph.checkpoint.sqlite import SqliteSaver
+
+from rag_demo.constants import LocalProviderType
+
+if TYPE_CHECKING:
+    from collections.abc import AsyncIterator
+    from pathlib import Path
+
+    from rag_demo.app_protocol import AppProtocol
+
+
+class HuggingFaceAgent:
+    """An LLM agent powered by Hugging Face local pipelines."""
+
+    def __init__(
+        self,
+        checkpoints_sqlite_db: str | Path,
+        model_id: str,
+        embedding_model_id: str,
+    ) -> None:
+        """Initialize the HuggingFaceAgent.
+
+        Args:
+            checkpoints_sqlite_db (str | Path): Connection string for SQLite database used for LangChain checkpoints.
+            model_id (str): Hugging Face model ID for the LLM.
+            embedding_model_id (str): Hugging Face model ID for the embedding model.
+        """
+        self.checkpoints_sqlite_db = checkpoints_sqlite_db
+        self.model_id = model_id
+        self.embedding_model_id = embedding_model_id
+
+        self.llm = ChatHuggingFace(
+            llm=HuggingFacePipeline.from_model_id(
+                model_id=model_id,
+                task="text-generation",
+                device_map="auto",
+                pipeline_kwargs={"max_new_tokens": 4096},
+            ),
+        )
+        self.embed = HuggingFaceEmbeddings(model_name=embedding_model_id)
+        self.agent = create_agent(
+            model=self.llm,
+            system_prompt="You are a helpful assistant.",
+            checkpointer=SqliteSaver(sqlite3.Connection(self.checkpoints_sqlite_db, check_same_thread=False)),
+        )
+
+    async def astream(self, user_message: str, thread_id: str, app: AppProtocol) -> AsyncIterator[str]:
+        """Stream a response from the agent.
+
+        Args:
+            user_message (str): User's next prompt in the conversation.
+            thread_id (str): Identifier for the current thread/conversation.
+            app (AppProtocol): Application interface, commonly used for logging.
+
+        Yields:
+            str: A token from the agent's response.
+        """
+        agent_stream = self.agent.stream(
+            {"messages": [HumanMessage(content=user_message)]},
+            {"configurable": {"thread_id": thread_id}},
+            stream_mode="messages",
+        )
+        for message_chunk, _ in agent_stream:
+            if isinstance(message_chunk, AIMessageChunk):
+                token = message_chunk.content
+                if isinstance(token, str):
+                    yield token
+                else:
+                    app.log.error("Received message content of type", type(token))
+            else:
+                app.log.error("Received message chunk of type", type(message_chunk))
+
+
+def _hf_downloads() -> None:
+    hf_hub_download(
+        repo_id="Qwen/Qwen3-0.6B",  # 1.5GB
+        filename="model.safetensors",
+        revision="c1899de289a04d12100db370d81485cdf75e47ca",
+    )
+    hf_hub_download(
+        repo_id="unsloth/embeddinggemma-300m",  # 1.21GB
+        filename="model.safetensors",
+        revision="bfa3c846ac738e62aa61806ef9112d34acb1dc5a",
+    )
+
+
+class HuggingFaceAgentProvider:
+    """Create LLM agents using Hugging Face local pipelines."""
+
+    type: Final[LocalProviderType] = LocalProviderType.HUGGING_FACE
+
+    @asynccontextmanager
+    async def get_agent(self, checkpoints_sqlite_db: str | Path) -> AsyncIterator[HuggingFaceAgent]:
+        """Create a Hugging Face local pipeline agent.
+
+        Args:
+            checkpoints_sqlite_db (str | Path): Connection string for SQLite database used for LangChain checkpoints.
+        """
+        loop = asyncio.get_running_loop()
+        await loop.run_in_executor(None, _hf_downloads)
+        yield HuggingFaceAgent(
+            checkpoints_sqlite_db,
+            model_id="Qwen/Qwen3-0.6B",
+            embedding_model_id="unsloth/embeddinggemma-300m",
+        )

rag_demo/agents/llama_cpp.py

@@ -0,0 +1,113 @@
+from __future__ import annotations
+
+import asyncio
+from contextlib import asynccontextmanager
+from typing import TYPE_CHECKING, Final
+
+import aiosqlite
+from huggingface_hub import hf_hub_download
+from langchain.agents import create_agent
+from langchain.messages import AIMessageChunk, HumanMessage
+from langchain_community.chat_models import ChatLlamaCpp
+from langchain_community.embeddings import LlamaCppEmbeddings
+from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
+
+from rag_demo import probe
+from rag_demo.constants import LocalProviderType
+
+if TYPE_CHECKING:
+    from collections.abc import AsyncIterator
+    from pathlib import Path
+
+    from rag_demo.app_protocol import AppProtocol
+
+
+class LlamaCppAgent:
+    """An LLM agent powered by Llama.cpp."""
+
+    def __init__(
+        self,
+        checkpoints_conn: aiosqlite.Connection,
+        model_path: str,
+        embedding_model_path: str,
+    ) -> None:
+        """Initialize the LlamaCppAgent.
+
+        Args:
+            checkpoints_conn (aiosqlite.Connection): Connection to SQLite checkpoint database.
+            model_path (str): Path to Llama.cpp model.
+            embedding_model_path (str): Path to Llama.cpp embedding model.
+        """
+        self.checkpoints_conn = checkpoints_conn
+        self.llm = ChatLlamaCpp(model_path=model_path, verbose=False)
+        self.embed = LlamaCppEmbeddings(model_path=embedding_model_path, verbose=False)
+        self.agent = create_agent(
+            model=self.llm,
+            system_prompt="You are a helpful assistant.",
+            checkpointer=AsyncSqliteSaver(self.checkpoints_conn),
+        )
+
+    async def astream(self, user_message: str, thread_id: str, app: AppProtocol) -> AsyncIterator[str]:
+        """Stream a response from the agent.
+
+        Args:
+            user_message (str): User's next prompt in the conversation.
+            thread_id (str): Identifier for the current thread/conversation.
+            app (AppProtocol): Application interface, commonly used for logging.
+
+        Yields:
+            str: A token from the agent's response.
+        """
+        agent_stream = self.agent.astream(
+            {"messages": [HumanMessage(content=user_message)]},
+            {"configurable": {"thread_id": thread_id}},
+            stream_mode="messages",
+        )
+        async for message_chunk, _ in agent_stream:
+            if isinstance(message_chunk, AIMessageChunk):
+                token = message_chunk.content
+                if isinstance(token, str):
+                    yield token
+                else:
+                    app.log.error("Received message content of type", type(token))
+            else:
+                app.log.error("Received message chunk of type", type(message_chunk))
+
+
+def _hf_downloads() -> tuple[str, str]:
+    model_path = hf_hub_download(
+        repo_id="bartowski/google_gemma-3-4b-it-GGUF",
+        filename="google_gemma-3-4b-it-Q6_K_L.gguf",  # 3.35GB
+        revision="71506238f970075ca85125cd749c28b1b0eee84e",
+    )
+    embedding_model_path = hf_hub_download(
+        repo_id="CompendiumLabs/bge-small-en-v1.5-gguf",
+        filename="bge-small-en-v1.5-q8_0.gguf",  # 36.8MB
+        revision="d32f8c040ea3b516330eeb75b72bcc2d3a780ab7",
+    )
+    return model_path, embedding_model_path
+
+
+class LlamaCppAgentProvider:
+    """Create LLM agents using Llama.cpp."""
+
+    type: Final[LocalProviderType] = LocalProviderType.LLAMA_CPP
+
+    @asynccontextmanager
+    async def get_agent(self, checkpoints_sqlite_db: str | Path) -> AsyncIterator[LlamaCppAgent | None]:
+        """Attempt to create a Llama.cpp agent.
+
+        Args:
+            checkpoints_sqlite_db (str | Path): Connection string for SQLite database used for LangChain checkpoints.
+        """
+        if probe.probe_llama_available():
+            loop = asyncio.get_running_loop()
+            model_path, embedding_model_path = await loop.run_in_executor(None, _hf_downloads)
+            async with aiosqlite.connect(database=checkpoints_sqlite_db) as checkpoints_conn:
+                yield LlamaCppAgent(
+                    checkpoints_conn=checkpoints_conn,
+                    model_path=model_path,
+                    embedding_model_path=embedding_model_path,
+                )
+        else:
+            yield None

rag_demo/agents/ollama.py

@@ -0,0 +1,91 @@
+from __future__ import annotations
+
+from contextlib import asynccontextmanager
+from typing import TYPE_CHECKING, Final
+
+import aiosqlite
+import ollama
+from langchain.agents import create_agent
+from langchain.messages import AIMessageChunk, HumanMessage
+from langchain_ollama import ChatOllama, OllamaEmbeddings
+from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
+
+from rag_demo import probe
+from rag_demo.constants import LocalProviderType
+
+if TYPE_CHECKING:
+    from collections.abc import AsyncIterator
+    from pathlib import Path
+
+    from rag_demo.app_protocol import AppProtocol
+
+
+class OllamaAgent:
+    """An LLM agent powered by Ollama."""
+
+    def __init__(self, checkpoints_conn: aiosqlite.Connection) -> None:
+        """Initialize the OllamaAgent.
+
+        Args:
+            checkpoints_conn (aiosqlite.Connection): Asynchronous connection to SQLite db for checkpoints.
+        """
+        self.checkpoints_conn = checkpoints_conn
+        ollama.pull("gemma3:latest")  # 3.3GB
+        ollama.pull("embeddinggemma:latest")  # 621MB
+        self.llm = ChatOllama(
+            model="gemma3:latest",
+            validate_model_on_init=True,
+            temperature=0.5,
+            num_predict=4096,
+        )
+        self.embed = OllamaEmbeddings(model="embeddinggemma:latest")
+        self.agent = create_agent(
+            model=self.llm,
+            system_prompt="You are a helpful assistant.",
+            checkpointer=AsyncSqliteSaver(self.checkpoints_conn),
+        )
+
+    async def astream(self, user_message: str, thread_id: str, app: AppProtocol) -> AsyncIterator[str]:
+        """Stream a response from the agent.
+
+        Args:
+            user_message (str): User's next prompt in the conversation.
+            thread_id (str): Identifier for the current thread/conversation.
+            app (AppProtocol): Application interface, commonly used for logging.
+
+        Yields:
+            str: A token from the agent's response.
+        """
+        agent_stream = self.agent.astream(
+            {"messages": [HumanMessage(content=user_message)]},
+            {"configurable": {"thread_id": thread_id}},
+            stream_mode="messages",
+        )
+        async for message_chunk, _ in agent_stream:
+            if isinstance(message_chunk, AIMessageChunk):
+                token = message_chunk.content
+                if isinstance(token, str):
+                    yield token
+                else:
+                    app.log.error("Received message content of type", type(token))
+            else:
+                app.log.error("Received message chunk of type", type(message_chunk))
+
+
+class OllamaAgentProvider:
+    """Create LLM agents using Ollama."""
+
+    type: Final[LocalProviderType] = LocalProviderType.OLLAMA
+
+    @asynccontextmanager
+    async def get_agent(self, checkpoints_sqlite_db: str | Path) -> AsyncIterator[OllamaAgent | None]:
+        """Attempt to create an Ollama agent.
+
+        Args:
+            checkpoints_sqlite_db (str | Path): Connection string for SQLite database used for LangChain checkpoints.
+        """
+        if probe.probe_ollama() is not None:
+            async with aiosqlite.connect(database=checkpoints_sqlite_db) as checkpoints_conn:
+                yield OllamaAgent(checkpoints_conn=checkpoints_conn)
+        else:
+            yield None

rag_demo/app.py

@@ -0,0 +1,58 @@
+from __future__ import annotations
+
+import asyncio
+from pathlib import Path
+from typing import TYPE_CHECKING, ClassVar
+
+from textual.app import App
+from textual.binding import Binding
+
+from rag_demo.modes import ChatScreen, ConfigScreen, HelpScreen
+
+if TYPE_CHECKING:
+    from rag_demo.logic import Logic, Runtime
+
+
+class RAGDemo(App):
+    """Main application UI.
+
+    This class is responsible for creating the modes of the application, which are defined in :mod:`rag_demo.modes`.
+    """
+
+    TITLE = "RAG Demo"
+    CSS_PATH = Path(__file__).parent / "app.tcss"
+    BINDINGS: ClassVar = [
+        Binding("z", "switch_mode('chat')", "chat"),
+        Binding("c", "switch_mode('config')", "configure"),
+        Binding("h", "switch_mode('help')", "help"),
+    ]
+    MODES: ClassVar = {
+        "chat": ChatScreen,
+        "config": ConfigScreen,
+        "help": HelpScreen,
+    }
+
+    def __init__(self, logic: Logic) -> None:
+        """Initialize the main app.
+
+        Args:
+            logic (Logic): Object implementing the application logic.
+        """
+        super().__init__()
+        self.logic = logic
+        self._runtime_future: asyncio.Future[Runtime] = asyncio.Future()
+
+    async def on_mount(self) -> None:
+        """Set the initial mode to chat and initialize async parts of the logic."""
+        self.switch_mode("chat")
+        self.run_worker(self._hold_runtime())
+
+    async def _hold_runtime(self) -> None:
+        async with self.logic.runtime(app=self) as runtime:
+            self._runtime_future.set_result(runtime)
+            # Pause the task until Textual cancels it when the application closes.
+            await asyncio.Event().wait()
+
+    async def runtime(self) -> Runtime:
+        """Returns the application runtime logic."""
+        return await self._runtime_future