llm_meta_client 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c97f3a183b70b928d55677581100d6774fbf6400140ff9604f883136f4c92c7e
-  data.tar.gz: 5d6078eabeb97d52b07980cb452cdbbf64ce65599d494c662e875249cd6e6f39
+  metadata.gz: b3b7edb6ffb37242cabcfed5f765bd3337565c2e55e1fcce5a6508bb07a53cdf
+  data.tar.gz: c1a289098044870e43c86d2df34b223a33af2fd10f59dceaadef38ae0df68800
 SHA512:
-  metadata.gz: 268562c7e43e59fd50e3cf38dc06d0164e461f98b3e650df580443de556681573dccdb5db5d82963237d230e150bd238ad20a3d03d21d58337f95c2a00247cd1
-  data.tar.gz: 6c7baa9d2c77789861727acbd5f1d40ea391a3045df9b3422a780fbe7978d60473db22d6a2d69bc81b7cb48b207e2a664ba2d31f05b6d4372d802c985860e879
+  metadata.gz: f1fdb013fca840d2ca0762ba8861902402e80282e58491b95b339fcfce1770203d1035af454f6bcaaa36d56e511df45ed2fb92a1e5d602dae769805b031e040d
+  data.tar.gz: 31f06f8324a6626b2b54fe86ef3cb88453ced0c18cccea24e0696721382f793bbee849f6f5505e054188432a5d93df918d77005038c546e6280ec5481fa76418

data/README.md

@@ -2,16 +2,48 @@
 
 A Rails Engine for integrating multiple LLM providers into your application. Provides scaffold and authentication generators to quickly build LLM-powered chat applications with support for OpenAI, Anthropic, Google, and Ollama.
 
+## Architecture
+
+This gem is a **Rails frontend client** that delegates all LLM provider interactions to an external backend service. The gem itself does not implement any LLM provider SDKs directly — instead, it communicates with the external service via REST API to manage API keys, models, and chat completions.
+
+```
+┌─────────────────────┐       REST API         ┌─────────────────────┐
+│  Your Rails App     │ ───────────────────>   │  External LLM       │
+│  + llm_meta_client  │ <───────────────────   │  Service Backend    │
+└─────────────────────┘                        └─────────────────────┘
+                                                  │
+                                                  ├── OpenAI API
+                                                  ├── Anthropic API
+                                                  ├── Google API
+                                                  └── Ollama (local)
+```
+
 ## Features
 
 - **Scaffold generator** — Generates models, controllers, views, and JavaScript controllers for a full chat UI with Turbo Stream support
 - **Authentication generator** — Sets up Devise with Google OAuth2 for user authentication
 - **Core modules** — `ServerQuery`, `ServerResource`, `Helpers`, `ChatManageable`, `HistoryManageable`, and custom exception classes
+- **Hotwire integration** — Real-time UI updates via Turbo Stream with Stimulus controllers
+- **Guest user support** — Ollama can be used without authentication; Google Sign-In is optional
+- **Conversation branching** — Branch from a previous message to explore alternative conversation paths
+- **CSV export** — Download chat history as CSV (single chat or all chats)
+- **Auto-generated titles** — Chat titles are automatically generated by the LLM from the conversation content
+- **Context summarization** — Conversation history is automatically summarized to keep context concise
 
 ## Requirements
 
 - Ruby >= 3.4
 - Rails >= 8.1.1
+- An external LLM service backend (see [External LLM Service](#external-llm-service))
+
+## Dependencies
+
+| Gem | Version | Purpose |
+|---|---|---|
+| `rails` | `~> 8.1, >= 8.1.1` | Rails framework |
+| `httparty` | `~> 0.22` | HTTP client for external service communication |
+| `prompt_navigator` | `~> 0.1` | Prompt execution management and history |
+| `chat_manager` | `~> 0.1` | Chat sidebar, title generation, CSV export |
 
 ## Installation
 
@@ -27,11 +59,9 @@ And then execute:
 $ bundle install
 ```
 
-## Usage
+## Quick Start
 
-### Scaffold Generator
-
-Generates a complete chat interface with LLM integration:
+### 1. Run the Scaffold Generator
 
 ```bash
 $ rails generate llm_meta_client:scaffold
@@ -47,9 +77,9 @@ This creates:
 - **Migrations:** `create_chats`, `create_messages`
 - **Routes:** Chats resource with `clear`, `start_new`, `download_all_csv`, `download_csv`, `update_title` actions
 
-### Authentication Generator
+### 2. (Optional) Run the Authentication Generator
 
-Sets up user authentication with Devise and Google OAuth2:
+If you want user authentication with Google OAuth2:
 
 ```bash
 $ rails generate llm_meta_client:authentication
@@ -64,16 +94,199 @@ This creates:
 - **Migration:** `create_users`
 - **Routes:** Devise routes with OmniAuth callbacks
 
+This generator also adds the following gems to your Gemfile:
+
+- `devise`
+- `omniauth`
+- `omniauth-google-oauth2`
+- `omniauth-rails_csrf_protection`
+
+Run `bundle install` again after the authentication generator.
+
+### 3. Run Migrations
+
+```bash
+$ rails db:migrate
+```
+
+### 4. Configure Rails Credentials
+
+This gem uses **Rails credentials** (`rails credentials:edit`) instead of environment variables for configuration management.
+
+```bash
+$ EDITOR="vim" bin/rails credentials:edit
+```
+
+Add the following entries to your credentials file:
+
+```yaml
+# Required
+llm_service:
+  base_url: "http://localhost:3000"  # URL of your external LLM service
+
+  # Optional
+  summarize_conversation_count: 10  # Number of recent messages for context (default: 10)
+
+# Required only if using the authentication generator
+google:
+  client_id: "your-google-client-id"
+  client_secret: "your-google-client-secret"
+```
+
+### 5. Start the Application
+
+Ensure your external LLM service is running, then:
+
+```bash
+$ rails server -p 3001
+```
+
 ## Configuration
 
-The following environment variables are used:
+All configuration values are managed via **Rails credentials** (`rails credentials:edit`).
+
+| Credential Key | Default | Description |
+|---|---|---|
+| `llm_service.base_url` | `http://localhost:3000` | Base URL of the external LLM service backend |
+| `llm_service.summarize_conversation_count` | `10` | Number of recent messages to include in conversation context |
+| `google.client_id` | — | Google OAuth2 client ID (authentication generator) |
+| `google.client_secret` | — | Google OAuth2 client secret (authentication generator) |
+
+These values are referenced in the generated initializers `config/initializers/llm_service.rb` and `config/initializers/devise.rb`.
+
+## External LLM Service
+
+This gem requires an external LLM service backend that provides the following REST API endpoints:
+
+| Method | Endpoint | Auth | Description |
+|---|---|---|---|
+| `GET` | `/api/llms` | None | List all available LLMs (returns Ollama instances, etc.) |
+| `GET` | `/api/llm_api_keys` | Bearer JWT | List API keys for the authenticated user |
+| `POST` | `/api/llm_api_keys/:uuid/models/:model_id/chats` | Bearer JWT (optional) | Send a prompt and receive an LLM response |
+
+### Expected Response Formats
+
+**`GET /api/llms`**
+
+```json
+{
+  "llms": [
+    {
+      "uuid": "...",
+      "description": "Ollama Local",
+      "family": "ollama",
+      "llm_type": "ollama",
+      "available_models": ["llama3", "mistral"]
+    }
+  ]
+}
+```
+
+**`GET /api/llm_api_keys`**
+
+```json
+{
+  "llm_api_keys": [
+    {
+      "uuid": "...",
+      "description": "My OpenAI Key",
+      "llm_type": "openai",
+      "available_models": ["gpt-4", "gpt-3.5-turbo"]
+    }
+  ]
+}
+```
+
+**`POST /api/llm_api_keys/:uuid/models/:model_id/chats`**
+
+Request body:
+```json
+{
+  "prompt": "Context:..., User Prompt: ..."
+}
+```
+
+Response body:
+```json
+{
+  "response": {
+    "message": "The assistant's response text"
+  }
+}
+```
+
+## Core Classes
+
+### `LlmMetaClient::ServerQuery`
+
+Sends chat prompts to the external LLM service and returns responses. Used internally by the generated `Chat` model.
+
+```ruby
+query = LlmMetaClient::ServerQuery.new
+response = query.call(jwt_token, api_key_uuid, model_id, context, user_content)
+# => "The assistant's response text"
+```
+
+- HTTP timeout: 5 minutes (300 seconds)
+- Responses are synchronous (no streaming)
+
+### `LlmMetaClient::ServerResource`
+
+Fetches available LLM providers and models from the external service.
+
+```ruby
+# Get a flat list of available LLM options
+options = LlmMetaClient::ServerResource.available_llm_options(jwt_token)
+# => [{uuid:, description:, llm_type:, available_models:}, ...]
+
+# Get options grouped by provider family
+families = LlmMetaClient::ServerResource.available_llm_families(jwt_token)
+# => [{name: "OpenAI", llm_type: "openai", api_keys: [...]}, ...]
+```
+
+- Guest users (`jwt_token` is `nil` or blank) receive only Ollama options
+- Logged-in users receive their API keys plus Ollama (if available)
+
+### Concerns
+
+| Module | Purpose |
+|---|---|
+| `LlmMetaClient::Helpers` | View helpers (included automatically via Engine) |
+| `LlmMetaClient::HistoryManageable` | Controller concern for prompt history navigation |
+| `LlmMetaClient::ChatManageable` | Controller concern for chat sidebar management |
+
+## Exception Classes
+
+| Exception | Raised When |
+|---|---|
+| `LlmMetaClient::Exceptions::ServerError` | External LLM service returns a non-2xx HTTP status |
+| `LlmMetaClient::Exceptions::InvalidResponseError` | Response from the LLM service is not valid JSON |
+| `LlmMetaClient::Exceptions::EmptyResponseError` | LLM service returns an empty message |
+| `LlmMetaClient::Exceptions::OllamaUnavailableError` | No Ollama instances are registered in the LLM service |
+
+## How It Works
+
+Each time a user sends a message, the following happens:
+
+1. A `PromptExecution` record and a user `Message` are created
+2. If conversation history exists, it is **summarized by the LLM** to keep context concise
+3. The summarized context + user prompt are sent to the external LLM service
+4. The assistant response is saved as a new `Message`
+5. A chat title is auto-generated by the LLM (if not already set)
+6. Turbo Stream updates the UI in real-time
+
+This means each user message may trigger **up to 3 HTTP requests** to the external LLM service (context summarization, main response, title generation), each with a 5-minute timeout.
+
+**Note:** Streaming is not currently supported. All LLM responses are synchronous.
+
+## Supported Providers
 
-| Variable | Description | Default |
+| Provider | `llm_type` | Notes |
 |---|---|---|
-| `LLM_SERVICE_BASE_URL` | Base URL for the external LLM service | `http://localhost:3000` |
-| `SUMMARIZE_CONVERSATION_COUNT` | Number of messages to include in conversation context | `10` |
-| `GOOGLE_CLIENT_ID` | Google OAuth2 client ID (authentication generator) | — |
-| `GOOGLE_CLIENT_SECRET` | Google OAuth2 client secret (authentication generator) | — |
+| OpenAI | `openai` | Requires API key via external service |
+| Anthropic | `anthropic` | Requires API key via external service |
+| Google | `google` | Requires API key via external service |
+| Ollama | `ollama` | No API key required; usable by guest users |
 
 ## Contributing
 

data/lib/generators/llm_meta_client/authentication/templates/config/initializers/devise.rb

@@ -304,8 +304,8 @@ Devise.setup do |config|
   # access_type: 'offline' - Allows the application to receive refresh tokens
   # include_granted_scopes: true - Enables incremental authorization
   config.omniauth :google_oauth2,
-                  ENV["GOOGLE_CLIENT_ID"],
-                  ENV["GOOGLE_CLIENT_SECRET"],
+                  Rails.application.credentials.dig(:google, :client_id),
+                  Rails.application.credentials.dig(:google, :client_secret),
                   {
                     scope: "email,profile,openid",
                     prompt: "select_account",

data/lib/generators/llm_meta_client/scaffold/templates/config/initializers/llm_service.rb

@@ -4,7 +4,7 @@
 # External LLM service base URL for API key and model management
 Rails.application.configure do
   # Base URL for LLM service
-  # Retrieved from environment variable LLM_SERVICE_BASE_URL, uses default value if not set
-  config.llm_service_base_url = ENV.fetch("LLM_SERVICE_BASE_URL", "http://localhost:3000")
-  config.summarize_conversation_count = ENV.fetch("SUMMARIZE_CONVERSATION_COUNT", 10).to_i
+  # Retrieved from Rails credentials, uses default value if not set
+  config.llm_service_base_url = Rails.application.credentials.dig(:llm_service, :base_url) || "http://localhost:3000"
+  config.summarize_conversation_count = (Rails.application.credentials.dig(:llm_service, :summarize_conversation_count) || 10).to_i
 end

data/lib/llm_meta_client/version.rb

@@ -1,3 +1,3 @@
 module LlmMetaClient
-  VERSION = "0.1.0"
+  VERSION = "0.2.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: llm_meta_client
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - dhq_boiler