llms-py 2.0.25__tar.gz → 2.0.26__tar.gz

{llms_py-2.0.25/llms_py.egg-info → llms_py-2.0.26}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: llms-py
-Version: 2.0.25
+Version: 2.0.26
 Summary: A lightweight CLI tool and OpenAI-compatible server for querying multiple Large Language Model (LLM) providers
 Home-page: https://github.com/ServiceStack/llms
 Author: ServiceStack
@@ -111,56 +111,7 @@ test the response times for all configured providers and models, the results of
 pip install llms-py
 ```
 
-### Using Docker
-
-**a) Simple - Run in a Docker container:**
-
-Run the server on port `8000`:
-
-```bash
-docker run -p 8000:8000 -e GROQ_API_KEY=$GROQ_API_KEY ghcr.io/servicestack/llms:latest
-```
-
-Get the latest version:
-
-```bash
-docker pull ghcr.io/servicestack/llms:latest
-```
-
-Use custom `llms.json` and `ui.json` config files outside of the container (auto created if they don't exist):
-
-```bash
-docker run -p 8000:8000 -e GROQ_API_KEY=$GROQ_API_KEY \
-  -v ~/.llms:/home/llms/.llms \
-  ghcr.io/servicestack/llms:latest
-```
-
-**b) Recommended - Use Docker Compose:**
-
-Download and use [docker-compose.yml](https://raw.githubusercontent.com/ServiceStack/llms/refs/heads/main/docker-compose.yml):
-
-```bash
-curl -O https://raw.githubusercontent.com/ServiceStack/llms/refs/heads/main/docker-compose.yml
-```
-
-Update API Keys in `docker-compose.yml` then start the server:
-
-```bash
-docker-compose up -d
-```
-
-**c) Build and run local Docker image from source:**
-
-```bash
-git clone https://github.com/ServiceStack/llms
-
-docker-compose -f docker-compose.local.yml up -d --build
-```
-
-After the container starts, you can access the UI and API at `http://localhost:8000`.
-
-
-See [DOCKER.md](DOCKER.md) for detailed instructions on customizing configuration files.
+- [Using Docker](#using-docker)
 
 ## Quick Start
 
@@ -224,6 +175,63 @@ llms --disable openrouter_free codestral google_free groq
 llms --enable openrouter anthropic google openai grok z.ai qwen mistral
 ```
 
+## Using Docker
+
+#### a) Simple - Run in a Docker container:
+
+Run the server on port `8000`:
+
+```bash
+docker run -p 8000:8000 -e GROQ_API_KEY=$GROQ_API_KEY ghcr.io/servicestack/llms:latest
+```
+
+Get the latest version:
+
+```bash
+docker pull ghcr.io/servicestack/llms:latest
+```
+
+Use custom `llms.json` and `ui.json` config files outside of the container (auto created if they don't exist):
+
+```bash
+docker run -p 8000:8000 -e GROQ_API_KEY=$GROQ_API_KEY \
+  -v ~/.llms:/home/llms/.llms \
+  ghcr.io/servicestack/llms:latest
+```
+
+#### b) Recommended - Use Docker Compose:
+
+Download and use [docker-compose.yml](https://raw.githubusercontent.com/ServiceStack/llms/refs/heads/main/docker-compose.yml):
+
+```bash
+curl -O https://raw.githubusercontent.com/ServiceStack/llms/refs/heads/main/docker-compose.yml
+```
+
+Update API Keys in `docker-compose.yml` then start the server:
+
+```bash
+docker-compose up -d
+```
+
+#### c) Build and run local Docker image from source:
+
+```bash
+git clone https://github.com/ServiceStack/llms
+
+docker-compose -f docker-compose.local.yml up -d --build
+```
+
+After the container starts, you can access the UI and API at `http://localhost:8000`.
+
+
+See [DOCKER.md](DOCKER.md) for detailed instructions on customizing configuration files.
+
+## GitHub OAuth Authentication
+
+llms.py supports optional GitHub OAuth authentication to secure your web UI and API endpoints. When enabled, users must sign in with their GitHub account before accessing the application.
+
+See [GITHUB_OAUTH_SETUP.md](GITHUB_OAUTH_SETUP.md) for detailed setup instructions.
+
 ## Configuration
 
 The configuration file [llms.json](llms/llms.json) is saved to `~/.llms/llms.json` and defines available providers, models, and default settings. Key sections:

{llms_py-2.0.25 → llms_py-2.0.26}/README.md

@@ -71,56 +71,7 @@ test the response times for all configured providers and models, the results of
 pip install llms-py
 ```
 
-### Using Docker
-
-**a) Simple - Run in a Docker container:**
-
-Run the server on port `8000`:
-
-```bash
-docker run -p 8000:8000 -e GROQ_API_KEY=$GROQ_API_KEY ghcr.io/servicestack/llms:latest
-```
-
-Get the latest version:
-
-```bash
-docker pull ghcr.io/servicestack/llms:latest
-```
-
-Use custom `llms.json` and `ui.json` config files outside of the container (auto created if they don't exist):
-
-```bash
-docker run -p 8000:8000 -e GROQ_API_KEY=$GROQ_API_KEY \
-  -v ~/.llms:/home/llms/.llms \
-  ghcr.io/servicestack/llms:latest
-```
-
-**b) Recommended - Use Docker Compose:**
-
-Download and use [docker-compose.yml](https://raw.githubusercontent.com/ServiceStack/llms/refs/heads/main/docker-compose.yml):
-
-```bash
-curl -O https://raw.githubusercontent.com/ServiceStack/llms/refs/heads/main/docker-compose.yml
-```
-
-Update API Keys in `docker-compose.yml` then start the server:
-
-```bash
-docker-compose up -d
-```
-
-**c) Build and run local Docker image from source:**
-
-```bash
-git clone https://github.com/ServiceStack/llms
-
-docker-compose -f docker-compose.local.yml up -d --build
-```
-
-After the container starts, you can access the UI and API at `http://localhost:8000`.
-
-
-See [DOCKER.md](DOCKER.md) for detailed instructions on customizing configuration files.
+- [Using Docker](#using-docker)
 
 ## Quick Start
 
@@ -184,6 +135,63 @@ llms --disable openrouter_free codestral google_free groq
 llms --enable openrouter anthropic google openai grok z.ai qwen mistral
 ```
 
+## Using Docker
+
+#### a) Simple - Run in a Docker container:
+
+Run the server on port `8000`:
+
+```bash
+docker run -p 8000:8000 -e GROQ_API_KEY=$GROQ_API_KEY ghcr.io/servicestack/llms:latest
+```
+
+Get the latest version:
+
+```bash
+docker pull ghcr.io/servicestack/llms:latest
+```
+
+Use custom `llms.json` and `ui.json` config files outside of the container (auto created if they don't exist):
+
+```bash
+docker run -p 8000:8000 -e GROQ_API_KEY=$GROQ_API_KEY \
+  -v ~/.llms:/home/llms/.llms \
+  ghcr.io/servicestack/llms:latest
+```
+
+#### b) Recommended - Use Docker Compose:
+
+Download and use [docker-compose.yml](https://raw.githubusercontent.com/ServiceStack/llms/refs/heads/main/docker-compose.yml):
+
+```bash
+curl -O https://raw.githubusercontent.com/ServiceStack/llms/refs/heads/main/docker-compose.yml
+```
+
+Update API Keys in `docker-compose.yml` then start the server:
+
+```bash
+docker-compose up -d
+```
+
+#### c) Build and run local Docker image from source:
+
+```bash
+git clone https://github.com/ServiceStack/llms
+
+docker-compose -f docker-compose.local.yml up -d --build
+```
+
+After the container starts, you can access the UI and API at `http://localhost:8000`.
+
+
+See [DOCKER.md](DOCKER.md) for detailed instructions on customizing configuration files.
+
+## GitHub OAuth Authentication
+
+llms.py supports optional GitHub OAuth authentication to secure your web UI and API endpoints. When enabled, users must sign in with their GitHub account before accessing the application.
+
+See [GITHUB_OAUTH_SETUP.md](GITHUB_OAUTH_SETUP.md) for detailed setup instructions.
+
 ## Configuration
 
 The configuration file [llms.json](llms/llms.json) is saved to `~/.llms/llms.json` and defines available providers, models, and default settings. Key sections:

{llms_py-2.0.25 → llms_py-2.0.26}/llms/llms.json

@@ -1,4 +1,12 @@
 {
+    "auth": {
+        "enabled": true,
+        "github": {
+            "client_id": "$GITHUB_CLIENT_ID",
+            "client_secret": "$GITHUB_CLIENT_SECRET",
+            "redirect_uri": "http://localhost:8000/auth/github/callback"
+        }
+    },
     "defaults": {
         "headers": {
             "Content-Type": "application/json",
@@ -113,6 +121,7 @@
                 "mai-ds-r1": "microsoft/mai-ds-r1:free",
                 "llama3.3:70b": "meta-llama/llama-3.3-70b-instruct:free",
                 "nemotron-nano:9b": "nvidia/nemotron-nano-9b-v2:free",
+                "nemotron-nano:12b":"nvidia/nemotron-nano-12b-v2-vl:free",
                 "deepseek-r1-distill-llama:70b": "deepseek/deepseek-r1-distill-llama-70b:free",
                 "gpt-oss:20b": "openai/gpt-oss-20b:free",
                 "mistral-small3.2:24b": "mistralai/mistral-small-3.2-24b-instruct:free",

{llms_py-2.0.25 → llms_py-2.0.26}/llms/main.py

@@ -14,7 +14,8 @@ import mimetypes
 import traceback
 import sys
 import site
-from urllib.parse import parse_qs
+import secrets
+from urllib.parse import parse_qs, urlencode
 
 import aiohttp
 from aiohttp import web
@@ -22,7 +23,7 @@ from aiohttp import web
 from pathlib import Path
 from importlib import resources   # Py≥3.9  (pip install importlib_resources for 3.7/3.8)
 
-VERSION = "2.0.25"
+VERSION = "2.0.26"
 _ROOT = None
 g_config_path = None
 g_ui_path = None
@@ -31,6 +32,8 @@ g_handlers = {}
 g_verbose = False
 g_logprefix=""
 g_default_model=""
+g_sessions = {}  # OAuth session storage: {session_token: {userId, userName, displayName, profileUrl, email, created}}
+g_oauth_states = {}  # CSRF protection: {state: {created, redirect_uri}}
 
 def _log(message):
     """Helper method for logging from the global polling task."""
@@ -1456,9 +1459,61 @@ def main():
             print(f"UI not found at {g_ui_path}")
             exit(1)
 
+        # Validate auth configuration if enabled
+        auth_enabled = g_config.get('auth', {}).get('enabled', False)
+        if auth_enabled:
+            github_config = g_config.get('auth', {}).get('github', {})
+            client_id = github_config.get('client_id', '')
+            client_secret = github_config.get('client_secret', '')
+
+            # Expand environment variables
+            if client_id.startswith('$'):
+                client_id = os.environ.get(client_id[1:], '')
+            if client_secret.startswith('$'):
+                client_secret = os.environ.get(client_secret[1:], '')
+
+            if not client_id or not client_secret:
+                print("ERROR: Authentication is enabled but GitHub OAuth is not properly configured.")
+                print("Please set GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET environment variables,")
+                print("or disable authentication by setting 'auth.enabled' to false in llms.json")
+                exit(1)
+
+            _log("Authentication enabled - GitHub OAuth configured")
+
         app = web.Application()
 
+        # Authentication middleware helper
+        def check_auth(request):
+            """Check if request is authenticated. Returns (is_authenticated, user_data)"""
+            if not auth_enabled:
+                return True, None
+
+            # Check for OAuth session token
+            session_token = request.query.get('session') or request.headers.get('X-Session-Token')
+            if session_token and session_token in g_sessions:
+                return True, g_sessions[session_token]
+
+            # Check for API key
+            auth_header = request.headers.get('Authorization', '')
+            if auth_header.startswith('Bearer '):
+                api_key = auth_header[7:]
+                if api_key:
+                    return True, {"authProvider": "apikey"}
+
+            return False, None
+
         async def chat_handler(request):
+            # Check authentication if enabled
+            is_authenticated, user_data = check_auth(request)
+            if not is_authenticated:
+                return web.json_response({
+                    "error": {
+                        "message": "Authentication required",
+                        "type": "authentication_error",
+                        "code": "unauthorized"
+                    }
+                }, status=401)
+
             try:
                 chat = await request.json()
                 response = await chat_completion(chat)
@@ -1504,6 +1559,198 @@ def main():
             })
         app.router.add_post('/providers/{provider}', provider_handler)
 
+        # OAuth handlers
+        async def github_auth_handler(request):
+            """Initiate GitHub OAuth flow"""
+            if 'auth' not in g_config or 'github' not in g_config['auth']:
+                return web.json_response({"error": "GitHub OAuth not configured"}, status=500)
+
+            auth_config = g_config['auth']['github']
+            client_id = auth_config.get('client_id', '')
+            redirect_uri = auth_config.get('redirect_uri', '')
+
+            # Expand environment variables
+            if client_id.startswith('$'):
+                client_id = os.environ.get(client_id[1:], '')
+            if redirect_uri.startswith('$'):
+                redirect_uri = os.environ.get(redirect_uri[1:], '')
+
+            if not client_id:
+                return web.json_response({"error": "GitHub client_id not configured"}, status=500)
+
+            # Generate CSRF state token
+            state = secrets.token_urlsafe(32)
+            g_oauth_states[state] = {
+                'created': time.time(),
+                'redirect_uri': redirect_uri
+            }
+
+            # Clean up old states (older than 10 minutes)
+            current_time = time.time()
+            expired_states = [s for s, data in g_oauth_states.items() if current_time - data['created'] > 600]
+            for s in expired_states:
+                del g_oauth_states[s]
+
+            # Build GitHub authorization URL
+            params = {
+                'client_id': client_id,
+                'redirect_uri': redirect_uri,
+                'state': state,
+                'scope': 'read:user user:email'
+            }
+            auth_url = f"https://github.com/login/oauth/authorize?{urlencode(params)}"
+
+            return web.HTTPFound(auth_url)
+
+        async def github_callback_handler(request):
+            """Handle GitHub OAuth callback"""
+            code = request.query.get('code')
+            state = request.query.get('state')
+
+            if not code or not state:
+                return web.Response(text="Missing code or state parameter", status=400)
+
+            # Verify state token (CSRF protection)
+            if state not in g_oauth_states:
+                return web.Response(text="Invalid state parameter", status=400)
+
+            state_data = g_oauth_states.pop(state)
+
+            if 'auth' not in g_config or 'github' not in g_config['auth']:
+                return web.json_response({"error": "GitHub OAuth not configured"}, status=500)
+
+            auth_config = g_config['auth']['github']
+            client_id = auth_config.get('client_id', '')
+            client_secret = auth_config.get('client_secret', '')
+            redirect_uri = auth_config.get('redirect_uri', '')
+
+            # Expand environment variables
+            if client_id.startswith('$'):
+                client_id = os.environ.get(client_id[1:], '')
+            if client_secret.startswith('$'):
+                client_secret = os.environ.get(client_secret[1:], '')
+            if redirect_uri.startswith('$'):
+                redirect_uri = os.environ.get(redirect_uri[1:], '')
+
+            if not client_id or not client_secret:
+                return web.json_response({"error": "GitHub OAuth credentials not configured"}, status=500)
+
+            # Exchange code for access token
+            async with aiohttp.ClientSession() as session:
+                token_url = "https://github.com/login/oauth/access_token"
+                token_data = {
+                    'client_id': client_id,
+                    'client_secret': client_secret,
+                    'code': code,
+                    'redirect_uri': redirect_uri
+                }
+                headers = {'Accept': 'application/json'}
+
+                async with session.post(token_url, data=token_data, headers=headers) as resp:
+                    token_response = await resp.json()
+                    access_token = token_response.get('access_token')
+
+                    if not access_token:
+                        error = token_response.get('error_description', 'Failed to get access token')
+                        return web.Response(text=f"OAuth error: {error}", status=400)
+
+                # Fetch user info
+                user_url = "https://api.github.com/user"
+                headers = {
+                    "Authorization": f"Bearer {access_token}",
+                    "Accept": "application/json"
+                }
+
+                async with session.get(user_url, headers=headers) as resp:
+                    user_data = await resp.json()
+
+            # Create session
+            session_token = secrets.token_urlsafe(32)
+            g_sessions[session_token] = {
+                "userId": str(user_data.get('id', '')),
+                "userName": user_data.get('login', ''),
+                "displayName": user_data.get('name', ''),
+                "profileUrl": user_data.get('avatar_url', ''),
+                "email": user_data.get('email', ''),
+                "created": time.time()
+            }
+
+            # Redirect to UI with session token
+            return web.HTTPFound(f"/?session={session_token}")
+
+        async def session_handler(request):
+            """Validate and return session info"""
+            session_token = request.query.get('session') or request.headers.get('X-Session-Token')
+
+            if not session_token or session_token not in g_sessions:
+                return web.json_response({"error": "Invalid or expired session"}, status=401)
+
+            session_data = g_sessions[session_token]
+
+            # Clean up old sessions (older than 24 hours)
+            current_time = time.time()
+            expired_sessions = [token for token, data in g_sessions.items() if current_time - data['created'] > 86400]
+            for token in expired_sessions:
+                del g_sessions[token]
+
+            return web.json_response({
+                **session_data,
+                "sessionToken": session_token
+            })
+
+        async def logout_handler(request):
+            """End OAuth session"""
+            session_token = request.query.get('session') or request.headers.get('X-Session-Token')
+
+            if session_token and session_token in g_sessions:
+                del g_sessions[session_token]
+
+            return web.json_response({"success": True})
+
+        async def auth_handler(request):
+            """Check authentication status and return user info"""
+            # Check for OAuth session token
+            session_token = request.query.get('session') or request.headers.get('X-Session-Token')
+
+            if session_token and session_token in g_sessions:
+                session_data = g_sessions[session_token]
+                return web.json_response({
+                    "userId": session_data.get("userId", ""),
+                    "userName": session_data.get("userName", ""),
+                    "displayName": session_data.get("displayName", ""),
+                    "profileUrl": session_data.get("profileUrl", ""),
+                    "authProvider": "github"
+                })
+
+            # Check for API key in Authorization header
+            # auth_header = request.headers.get('Authorization', '')
+            # if auth_header.startswith('Bearer '):
+            #     # For API key auth, return a basic response
+            #     # You can customize this based on your API key validation logic
+            #     api_key = auth_header[7:]
+            #     if api_key:  # Add your API key validation logic here
+            #         return web.json_response({
+            #             "userId": "1",
+            #             "userName": "apiuser",
+            #             "displayName": "API User",
+            #             "profileUrl": "",
+            #             "authProvider": "apikey"
+            #         })
+
+            # Not authenticated - return error in expected format
+            return web.json_response({
+                "responseStatus": {
+                    "errorCode": "Unauthorized",
+                    "message": "Not authenticated"
+                }
+            }, status=401)
+
+        app.router.add_get('/auth', auth_handler)
+        app.router.add_get('/auth/github', github_auth_handler)
+        app.router.add_get('/auth/github/callback', github_callback_handler)
+        app.router.add_get('/auth/session', session_handler)
+        app.router.add_post('/auth/logout', logout_handler)
+
         async def ui_static(request: web.Request) -> web.Response:
             path = Path(request.match_info["path"])
 
@@ -1543,9 +1790,12 @@ def main():
                 enabled, disabled = provider_status()
                 ui['status'] = {
                     "all": list(g_config['providers'].keys()),
-                    "enabled": enabled, 
-                    "disabled": disabled 
+                    "enabled": enabled,
+                    "disabled": disabled
                 }
+                # Add auth configuration
+                ui['requiresAuth'] = auth_enabled
+                ui['authType'] = 'oauth' if auth_enabled else 'apikey'
                 return web.json_response(ui)
         app.router.add_get('/config', ui_config_handler)
 

{llms_py-2.0.25 → llms_py-2.0.26}/llms/ui/App.mjs

@@ -1,13 +1,18 @@
+import { inject } from "vue"
 import Sidebar from "./Sidebar.mjs"
 
 export default {
     components: {
         Sidebar,
     },
+    setup() {
+        const ai = inject('ai')
+        return { ai }
+    },
     template: `
         <div class="flex h-screen bg-white">
-            <!-- Sidebar -->
-            <div class="w-72 xl:w-80 flex-shrink-0">
+            <!-- Sidebar (hidden when auth required and not authenticated) -->
+            <div v-if="!(ai.requiresAuth && !ai.auth)" class="w-72 xl:w-80 flex-shrink-0">
                 <Sidebar />
             </div>
 

llms_py-2.0.26/llms/ui/Avatar.mjs

@@ -0,0 +1,85 @@
+import { computed, inject, ref, onMounted, onUnmounted } from "vue"
+
+export default {
+    template:`
+        <div v-if="$ai.auth?.profileUrl" class="relative" ref="avatarContainer">
+            <img
+                @click.stop="toggleMenu"
+                :src="$ai.auth.profileUrl"
+                :title="authTitle"
+                class="size-8 rounded-full cursor-pointer hover:ring-2 hover:ring-gray-300"
+            />
+            <div
+                v-if="showMenu"
+                @click.stop
+                class="absolute right-0 mt-2 w-48 bg-white dark:bg-gray-800 rounded-md shadow-lg py-1 z-50 border border-gray-200 dark:border-gray-700"
+            >
+                <div class="px-4 py-2 text-sm text-gray-700 dark:text-gray-300 border-b border-gray-200 dark:border-gray-700">
+                    <div class="font-medium whitespace-nowrap overflow-hidden text-ellipsis">{{ $ai.auth.displayName || $ai.auth.userName }}</div>
+                    <div class="text-xs text-gray-500 dark:text-gray-400 whitespace-nowrap overflow-hidden text-ellipsis">{{ $ai.auth.email }}</div>
+                </div>
+                <button type="button"
+                    @click="handleLogout"
+                    class="w-full text-left px-4 py-2 text-sm text-gray-700 dark:text-gray-300 hover:bg-gray-100 dark:hover:bg-gray-700 flex items-center whitespace-nowrap"
+                >
+                    <svg class="w-4 h-4 mr-2 flex-shrink-0" fill="none" stroke="currentColor" viewBox="0 0 24 24">
+                        <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M17 16l4-4m0 0l-4-4m4 4H7m6 4v1a3 3 0 01-3 3H6a3 3 0 01-3-3V7a3 3 0 013-3h4a3 3 0 013 3v1"></path>
+                    </svg>
+                    Sign Out
+                </button>
+            </div>
+        </div>
+    `,
+    setup() {
+        const ai = inject('ai')
+        const showMenu = ref(false)
+        const avatarContainer = ref(null)
+
+        const authTitle = computed(() => {
+            if (!ai.auth) return ''
+            const { userId, userName, displayName, bearerToken, roles } = ai.auth
+            const name = userName || displayName
+            const prefix = roles && roles.includes('Admin') ? 'Admin' : 'Name'
+            const sb = [
+                name ? `${prefix}: ${name}` : '',
+                `API Key: ${bearerToken}`,
+                `${userId}`,
+            ]
+            return sb.filter(x => x).join('\n')
+        })
+
+        function toggleMenu() {
+            showMenu.value = !showMenu.value
+        }
+
+        async function handleLogout() {
+            showMenu.value = false
+            await ai.signOut()
+            // Reload the page to show sign-in screen
+            window.location.reload()
+        }
+
+        // Close menu when clicking outside
+        const handleClickOutside = (event) => {
+            if (showMenu.value && avatarContainer.value && !avatarContainer.value.contains(event.target)) {
+                showMenu.value = false
+            }
+        }
+
+        onMounted(() => {
+            document.addEventListener('click', handleClickOutside)
+        })
+
+        onUnmounted(() => {
+            document.removeEventListener('click', handleClickOutside)
+        })
+
+        return {
+            authTitle,
+            handleLogout,
+            showMenu,
+            toggleMenu,
+            avatarContainer,
+        }
+    }
+}