lightspeed-stack 0.6.0rc1__tar.gz → 0.6.0rc2__tar.gz

{lightspeed_stack-0.6.0rc1 → lightspeed_stack-0.6.0rc2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: lightspeed-stack
-Version: 0.6.0rc1
+Version: 0.6.0rc2
 Summary: LLM tooling stack
 Keywords: LLM,RAG
 Maintainer-Email: =?utf-8?b?UGF2ZWwgVGnFoW5vdnNrw70=?= <tisnik@centrum.cz>
@@ -259,7 +259,7 @@ Description-Content-Type: text/markdown
 [![License](https://img.shields.io/badge/license-Apache-blue)](https://github.com/lightspeed-core/lightspeed-stack/blob/main/LICENSE)
 [![made-with-python](https://img.shields.io/badge/Made%20with-Python-1f425f.svg)](https://www.python.org/)
 [![Required Python version](https://img.shields.io/python/required-version-toml?tomlFilePath=https%3A%2F%2Fraw.githubusercontent.com%2Flightspeed-core%2Flightspeed-stack%2Frefs%2Fheads%2Fmain%2Fpyproject.toml)](https://www.python.org/)
-[![Tag](https://img.shields.io/github/v/tag/lightspeed-core/lightspeed-stack)](https://github.com/lightspeed-core/lightspeed-stack/releases/tag/0.5.0)
+[![Tag](https://img.shields.io/github/v/tag/lightspeed-core/lightspeed-stack)](https://github.com/lightspeed-core/lightspeed-stack/releases/tag/0.6.0rc2)
 
 Lightspeed Core Stack (LCS) is an AI-powered assistant that provides answers to product questions using backend LLM services, agents, and RAG databases.
 
@@ -272,7 +272,9 @@ The service includes comprehensive user data collection capabilities for various
 * [Prerequisites](#prerequisites)
 * [Installation](#installation)
 * [Run LCS locally](#run-lcs-locally)
+    * [Container Runtime Requirements](#container-runtime-requirements)
 * [Configuration](#configuration)
+    * [Agentic Capabilities](#agentic-capabilities)
     * [LLM Compatibility](#llm-compatibility)
     * [Set LLM provider and model](#set-llm-provider-and-model)
     * [Selecting provider and model](#selecting-provider-and-model)
@@ -281,6 +283,7 @@ The service includes comprehensive user data collection capabilities for various
     * [Supported providers](#supported-providers)
     * [Integration with Llama Stack](#integration-with-llama-stack)
     * [Llama Stack as separate server](#llama-stack-as-separate-server)
+        * [Degraded mode](#degraded-mode)
         * [MCP Server and Tool Configuration](#mcp-server-and-tool-configuration)
             * [Configuring MCP Servers](#configuring-mcp-servers)
             * [Configuring MCP Server Authentication](#configuring-mcp-server-authentication)
@@ -303,6 +306,7 @@ The service includes comprehensive user data collection capabilities for various
         * [System Prompt Literal](#system-prompt-literal)
         * [Custom Profile](#custom-profile)
         * [Control model/provider overrides via authorization](#control-modelprovider-overrides-via-authorization)
+    * [Agent Skills (Upcoming)](#agent-skills-upcoming)
     * [Safety Shields](#safety-shields)
     * [Authentication](#authentication)
     * [CORS](#cors)
@@ -335,6 +339,7 @@ The service includes comprehensive user data collection capabilities for various
     * [Packages on PyPI and Test PyPI](#packages-on-pypi-and-test-pypi)
 * [Contributing](#contributing)
 * [Testing](#testing)
+* [Releasing](#releasing)
 * [License](#license)
 * [Additional tools](#additional-tools)
     * [Utility to generate OpenAPI schema](#utility-to-generate-openapi-schema)
@@ -430,24 +435,39 @@ To quickly get hands on LCS, we can run it using the default configurations prov
    ```bash
    export OPENAI_API_KEY=sk-xxxxx
    ```
-3. start Llama stack server
+3. start LCS server 
    ```bash
-   uv run llama stack run local-run.yaml
-   ```
-4. [Optional] If you're new to Llama stack, run through a quick tutorial to learn the basics of what the server is used for, by running the interactive tutorial script
-   ```bash
-   ./scripts/llama_stack_tutorial.sh
-   ```
-5. check the LCS settings in [lightspeed-stack.yaml](lightspeed-stack.yaml). `llama_stack.url` should be `url: http://localhost:8321`
-6. start LCS server
-    ```
    make run
-    ```
-7. access LCS web UI at [http://localhost:8080/](http://localhost:8080/)
+   ```
+4. access LCS web UI at [http://localhost:8080/](http://localhost:8080/)
+
+**Note**: `make run` uses containerized llama-stack (service mode). To run llama-stack manually instead, see the [Llama Stack as separate server](#llama-stack-as-separate-server) section below.
+
+## Container Runtime Requirements
+
+The Makefile requires either Podman or Docker to launch the Llama Stack container:
+
+- **Podman** (recommended for RHEL/Fedora): `sudo dnf install podman`
+- **Docker**: Install from [docker.com](https://docs.docker.com/get-docker/)
+
+The Makefile will auto-detect which runtime is available.
 
 
 # Configuration
 
+## Agentic Capabilities
+
+Lightspeed Core Stack supports the following agentic features:
+
+| Capability | Status | Description |
+|------------|--------|-------------|
+| MCP Tools | Supported | External tool integration via [Model Context Protocol](https://modelcontextprotocol.io) servers |
+| RAG | Supported | Retrieval-Augmented Generation with vector stores ([RAG Guide](docs/rag_guide.md)) |
+| A2A Protocol (Client) | Supported | Agent-to-Agent communication as client ([A2A Protocol](docs/a2a_protocol.md)) |
+| Conversation History | Supported | Persistent conversation context across requests |
+| Human-in-the-Loop | Upcoming | Interactive approval or confirmation steps |
+| Agent Skills | Upcoming (Q2) | Domain-specific instructions loaded on demand ([Agent Skills Guide](docs/skills_guide.md)) |
+
 ## LLM Compatibility
 
 Lightspeed Core Stack (LCS) provides support for Large Language Model providers. The models listed in the table below represent specific examples that have been tested within LCS.
@@ -578,6 +598,20 @@ user_data_collection:
   transcripts_storage: "/tmp/data/transcripts"
 ```
 
+### Degraded mode
+
+Lightspeed core is able to continue operating in a _degraded but safe_ mode if the LLS service is not started or fails to start. When degraded, the `/health` endpoint report the LLS status and any relevant impacts so operators and automation can detect and respond.
+
+Degraded mode need to be enabled in `lightspeed-stack.yaml` configuration file:
+
+```yaml
+allow_degraded_mode = true
+```
+
+[NOTE] Ability to run in degraded mode is disabled by default.
+
+
+
 ### MCP Server and Tool Configuration
 
 **Note**: The `run.yaml` configuration is currently an implementation detail. In the future, all configuration will be available directly from the lightspeed-core config.
@@ -964,6 +998,15 @@ customization:
 
 By default, clients may specify `model` and `provider` in `/v1/query` and `/v1/streaming_query`. Override is permitted only to callers granted the `MODEL_OVERRIDE` action via the authorization rules. Requests that include `model` or `provider` without this permission are rejected with HTTP 403.
 
+## Agent Skills (Upcoming)
+
+> [!NOTE]
+> Agent Skills is an upcoming feature. The documentation below describes the planned design.
+
+Agent Skills will allow product teams to extend Lightspeed Core with specialized instructions and domain knowledge that the LLM can load on demand. Skills follow the [Agent Skills open standard](https://agentskills.io) and are packaged as portable directories containing a `SKILL.md` file.
+
+For the planned configuration guide, skill authoring instructions, and examples, see the [Agent Skills Guide](docs/skills_guide.md).
+
 ## Safety Shields
 
 A single Llama Stack configuration file can include multiple safety shields, which are utilized in agent
@@ -1083,7 +1126,6 @@ Usage: make <OPTIONS> ... <TARGETS>
 Available targets are:
 
 run                               Run the service locally
-run-llama-stack                   Start Llama Stack with enriched config (for local service mode)
 test-unit                         Run the unit tests
 test-integration                  Run integration tests tests
 test-e2e                          Run end to end tests for the service
@@ -1189,7 +1231,7 @@ docker compose up --build
 
 ### Llama-Stack as Library (Library Mode)
 
-When embedding llama-stack directly in the container, use the existing `Containerfile` directly (this will not build the llama stack service in a separate container). First modify the `lightspeed-stack.yaml` config to use llama stack in library mode.
+When embedding llama-stack directly in the container, use the existing `deploy/lightspeed-stack/Containerfile` directly (this will not build the llama stack service in a separate container). First modify the `lightspeed-stack.yaml` config to use llama stack in library mode.
 
 **Configuration** (`lightspeed-stack.yaml`):
 ```yaml
@@ -1201,7 +1243,7 @@ llama_stack:
 **Build and run**:
 ```bash
 # Build lightspeed-core with embedded llama-stack
-podman build -f Containerfile -t my-lightspeed-core:latest .
+podman build -f deploy/lightspeed-stack/Containerfile -t my-lightspeed-core:latest .
 
 # Run with embedded llama-stack
 podman run \
@@ -1254,7 +1296,7 @@ dependencies = [
 ]
 ```
 
-2. Create `Containerfile` in top-level directory like following. Update it as needed:
+2. Create a `Containerfile` in your repository root like the following (see `deploy/lightspeed-stack/Containerfile` in this repo for the upstream image definition). Update it as needed:
 ```
 # Latest dev image built from the git main branch (consider pinning a digest for reproducibility)
 FROM quay.io/lightspeed-core/lightspeed-stack:dev-latest
@@ -1474,6 +1516,10 @@ If this configuration file does not exist, you will be prompted to specify API t
 
 
 
+# Releasing
+
+* See [releasing](docs/releasing.md) guide.
+
 # License
 
 Published under the Apache 2.0 License
@@ -1581,7 +1627,7 @@ Konflux builds run in **hermetic mode** (air-gapped from the internet), so all d
 
 Update these files when you:
 - Add/remove/update Python packages in the project
-- Add/remove/update RPM packages in the Containerfile
+- Add/remove/update RPM packages in `deploy/lightspeed-stack/Containerfile`
 - Change the base image version
 
 ### Updating Python Dependencies

{lightspeed_stack-0.6.0rc1 → lightspeed_stack-0.6.0rc2}/README.md

@@ -6,7 +6,7 @@
 [![License](https://img.shields.io/badge/license-Apache-blue)](https://github.com/lightspeed-core/lightspeed-stack/blob/main/LICENSE)
 [![made-with-python](https://img.shields.io/badge/Made%20with-Python-1f425f.svg)](https://www.python.org/)
 [![Required Python version](https://img.shields.io/python/required-version-toml?tomlFilePath=https%3A%2F%2Fraw.githubusercontent.com%2Flightspeed-core%2Flightspeed-stack%2Frefs%2Fheads%2Fmain%2Fpyproject.toml)](https://www.python.org/)
-[![Tag](https://img.shields.io/github/v/tag/lightspeed-core/lightspeed-stack)](https://github.com/lightspeed-core/lightspeed-stack/releases/tag/0.5.0)
+[![Tag](https://img.shields.io/github/v/tag/lightspeed-core/lightspeed-stack)](https://github.com/lightspeed-core/lightspeed-stack/releases/tag/0.6.0rc2)
 
 Lightspeed Core Stack (LCS) is an AI-powered assistant that provides answers to product questions using backend LLM services, agents, and RAG databases.
 
@@ -19,7 +19,9 @@ The service includes comprehensive user data collection capabilities for various
 * [Prerequisites](#prerequisites)
 * [Installation](#installation)
 * [Run LCS locally](#run-lcs-locally)
+    * [Container Runtime Requirements](#container-runtime-requirements)
 * [Configuration](#configuration)
+    * [Agentic Capabilities](#agentic-capabilities)
     * [LLM Compatibility](#llm-compatibility)
     * [Set LLM provider and model](#set-llm-provider-and-model)
     * [Selecting provider and model](#selecting-provider-and-model)
@@ -28,6 +30,7 @@ The service includes comprehensive user data collection capabilities for various
     * [Supported providers](#supported-providers)
     * [Integration with Llama Stack](#integration-with-llama-stack)
     * [Llama Stack as separate server](#llama-stack-as-separate-server)
+        * [Degraded mode](#degraded-mode)
         * [MCP Server and Tool Configuration](#mcp-server-and-tool-configuration)
             * [Configuring MCP Servers](#configuring-mcp-servers)
             * [Configuring MCP Server Authentication](#configuring-mcp-server-authentication)
@@ -50,6 +53,7 @@ The service includes comprehensive user data collection capabilities for various
         * [System Prompt Literal](#system-prompt-literal)
         * [Custom Profile](#custom-profile)
         * [Control model/provider overrides via authorization](#control-modelprovider-overrides-via-authorization)
+    * [Agent Skills (Upcoming)](#agent-skills-upcoming)
     * [Safety Shields](#safety-shields)
     * [Authentication](#authentication)
     * [CORS](#cors)
@@ -82,6 +86,7 @@ The service includes comprehensive user data collection capabilities for various
     * [Packages on PyPI and Test PyPI](#packages-on-pypi-and-test-pypi)
 * [Contributing](#contributing)
 * [Testing](#testing)
+* [Releasing](#releasing)
 * [License](#license)
 * [Additional tools](#additional-tools)
     * [Utility to generate OpenAPI schema](#utility-to-generate-openapi-schema)
@@ -177,24 +182,39 @@ To quickly get hands on LCS, we can run it using the default configurations prov
    ```bash
    export OPENAI_API_KEY=sk-xxxxx
    ```
-3. start Llama stack server
+3. start LCS server 
    ```bash
-   uv run llama stack run local-run.yaml
-   ```
-4. [Optional] If you're new to Llama stack, run through a quick tutorial to learn the basics of what the server is used for, by running the interactive tutorial script
-   ```bash
-   ./scripts/llama_stack_tutorial.sh
-   ```
-5. check the LCS settings in [lightspeed-stack.yaml](lightspeed-stack.yaml). `llama_stack.url` should be `url: http://localhost:8321`
-6. start LCS server
-    ```
    make run
-    ```
-7. access LCS web UI at [http://localhost:8080/](http://localhost:8080/)
+   ```
+4. access LCS web UI at [http://localhost:8080/](http://localhost:8080/)
+
+**Note**: `make run` uses containerized llama-stack (service mode). To run llama-stack manually instead, see the [Llama Stack as separate server](#llama-stack-as-separate-server) section below.
+
+## Container Runtime Requirements
+
+The Makefile requires either Podman or Docker to launch the Llama Stack container:
+
+- **Podman** (recommended for RHEL/Fedora): `sudo dnf install podman`
+- **Docker**: Install from [docker.com](https://docs.docker.com/get-docker/)
+
+The Makefile will auto-detect which runtime is available.
 
 
 # Configuration
 
+## Agentic Capabilities
+
+Lightspeed Core Stack supports the following agentic features:
+
+| Capability | Status | Description |
+|------------|--------|-------------|
+| MCP Tools | Supported | External tool integration via [Model Context Protocol](https://modelcontextprotocol.io) servers |
+| RAG | Supported | Retrieval-Augmented Generation with vector stores ([RAG Guide](docs/rag_guide.md)) |
+| A2A Protocol (Client) | Supported | Agent-to-Agent communication as client ([A2A Protocol](docs/a2a_protocol.md)) |
+| Conversation History | Supported | Persistent conversation context across requests |
+| Human-in-the-Loop | Upcoming | Interactive approval or confirmation steps |
+| Agent Skills | Upcoming (Q2) | Domain-specific instructions loaded on demand ([Agent Skills Guide](docs/skills_guide.md)) |
+
 ## LLM Compatibility
 
 Lightspeed Core Stack (LCS) provides support for Large Language Model providers. The models listed in the table below represent specific examples that have been tested within LCS.
@@ -325,6 +345,20 @@ user_data_collection:
   transcripts_storage: "/tmp/data/transcripts"
 ```
 
+### Degraded mode
+
+Lightspeed core is able to continue operating in a _degraded but safe_ mode if the LLS service is not started or fails to start. When degraded, the `/health` endpoint report the LLS status and any relevant impacts so operators and automation can detect and respond.
+
+Degraded mode need to be enabled in `lightspeed-stack.yaml` configuration file:
+
+```yaml
+allow_degraded_mode = true
+```
+
+[NOTE] Ability to run in degraded mode is disabled by default.
+
+
+
 ### MCP Server and Tool Configuration
 
 **Note**: The `run.yaml` configuration is currently an implementation detail. In the future, all configuration will be available directly from the lightspeed-core config.
@@ -711,6 +745,15 @@ customization:
 
 By default, clients may specify `model` and `provider` in `/v1/query` and `/v1/streaming_query`. Override is permitted only to callers granted the `MODEL_OVERRIDE` action via the authorization rules. Requests that include `model` or `provider` without this permission are rejected with HTTP 403.
 
+## Agent Skills (Upcoming)
+
+> [!NOTE]
+> Agent Skills is an upcoming feature. The documentation below describes the planned design.
+
+Agent Skills will allow product teams to extend Lightspeed Core with specialized instructions and domain knowledge that the LLM can load on demand. Skills follow the [Agent Skills open standard](https://agentskills.io) and are packaged as portable directories containing a `SKILL.md` file.
+
+For the planned configuration guide, skill authoring instructions, and examples, see the [Agent Skills Guide](docs/skills_guide.md).
+
 ## Safety Shields
 
 A single Llama Stack configuration file can include multiple safety shields, which are utilized in agent
@@ -830,7 +873,6 @@ Usage: make <OPTIONS> ... <TARGETS>
 Available targets are:
 
 run                               Run the service locally
-run-llama-stack                   Start Llama Stack with enriched config (for local service mode)
 test-unit                         Run the unit tests
 test-integration                  Run integration tests tests
 test-e2e                          Run end to end tests for the service
@@ -936,7 +978,7 @@ docker compose up --build
 
 ### Llama-Stack as Library (Library Mode)
 
-When embedding llama-stack directly in the container, use the existing `Containerfile` directly (this will not build the llama stack service in a separate container). First modify the `lightspeed-stack.yaml` config to use llama stack in library mode.
+When embedding llama-stack directly in the container, use the existing `deploy/lightspeed-stack/Containerfile` directly (this will not build the llama stack service in a separate container). First modify the `lightspeed-stack.yaml` config to use llama stack in library mode.
 
 **Configuration** (`lightspeed-stack.yaml`):
 ```yaml
@@ -948,7 +990,7 @@ llama_stack:
 **Build and run**:
 ```bash
 # Build lightspeed-core with embedded llama-stack
-podman build -f Containerfile -t my-lightspeed-core:latest .
+podman build -f deploy/lightspeed-stack/Containerfile -t my-lightspeed-core:latest .
 
 # Run with embedded llama-stack
 podman run \
@@ -1001,7 +1043,7 @@ dependencies = [
 ]
 ```
 
-2. Create `Containerfile` in top-level directory like following. Update it as needed:
+2. Create a `Containerfile` in your repository root like the following (see `deploy/lightspeed-stack/Containerfile` in this repo for the upstream image definition). Update it as needed:
 ```
 # Latest dev image built from the git main branch (consider pinning a digest for reproducibility)
 FROM quay.io/lightspeed-core/lightspeed-stack:dev-latest
@@ -1221,6 +1263,10 @@ If this configuration file does not exist, you will be prompted to specify API t
 
 
 
+# Releasing
+
+* See [releasing](docs/releasing.md) guide.
+
 # License
 
 Published under the Apache 2.0 License
@@ -1328,7 +1374,7 @@ Konflux builds run in **hermetic mode** (air-gapped from the internet), so all d
 
 Update these files when you:
 - Add/remove/update Python packages in the project
-- Add/remove/update RPM packages in the Containerfile
+- Add/remove/update RPM packages in `deploy/lightspeed-stack/Containerfile`
 - Change the base image version
 
 ### Updating Python Dependencies

{lightspeed_stack-0.6.0rc1 → lightspeed_stack-0.6.0rc2}/pyproject.toml

@@ -56,7 +56,7 @@ dependencies = [
     "python-dotenv>=1.2.2",
     "tiktoken>=0.8.0",
 ]
-version = "0.6.0rc1"
+version = "0.6.0rc2"
 
 [project.license]
 file = "LICENSE"
@@ -174,7 +174,7 @@ dev = [
     "pytest-subtests>=0.14.2",
     "bandit>=1.8.6",
     "pybuild-deps>=0.5.0",
-    "pip==24.3.1",
+    "pip==26.1",
     "pytest-benchmark>=5.2.3",
 ]
 llslibdev = [

{lightspeed_stack-0.6.0rc1 → lightspeed_stack-0.6.0rc2}/src/app/endpoints/conversations_v2.py

@@ -6,6 +6,7 @@ from fastapi import APIRouter, Depends, HTTPException, Request
 
 from authentication import get_auth_dependency
 from authorization.middleware import authorize
+from cache.cache_entry import CacheEntry
 from configuration import configuration
 from log import get_logger
 from models.api.requests import ConversationUpdateRequest
@@ -24,7 +25,6 @@ from models.api.responses.successful import (
     ConversationsListResponseV2,
     ConversationUpdateResponse,
 )
-from models.cache_entry import CacheEntry
 from models.common import (
     ConversationTurn,
     Message,

{lightspeed_stack-0.6.0rc1 → lightspeed_stack-0.6.0rc2}/src/app/endpoints/query.py

@@ -54,7 +54,6 @@ from utils.query import (
     is_context_length_error,
     prepare_input,
     store_query_results,
-    update_azure_token,
     validate_attachments_metadata,
     validate_model_provider_override,
 )
@@ -204,7 +203,7 @@ async def query_endpoint_handler(
         and AzureEntraIDManager().is_token_expired
         and AzureEntraIDManager().refresh_token()
     ):
-        client = await update_azure_token(client)
+        client = await AsyncLlamaStackClientHolder().update_azure_token()
 
     # Retrieve response using Responses API
     turn_summary = await retrieve_response(

{lightspeed_stack-0.6.0rc1 → lightspeed_stack-0.6.0rc2}/src/app/endpoints/responses.py

@@ -60,8 +60,8 @@ from models.api.responses.error import (
 )
 from models.api.responses.successful import ResponsesResponse
 from models.common.moderation import ShieldModerationBlocked
+from models.common.responses.contexts import ResponsesContext
 from models.common.responses.responses_api_params import ResponsesApiParams
-from models.common.responses.responses_context import ResponsesContext
 from models.common.turn_summary import TurnSummary
 from models.config import Action
 from utils.conversations import append_turn_items_to_conversation
@@ -78,7 +78,6 @@ from utils.query import (
     handle_known_apistatus_errors,
     is_context_length_error,
     store_query_results,
-    update_azure_token,
     validate_model_provider_override,
 )
 from utils.quota import check_tokens_available, get_available_quotas
@@ -405,7 +404,7 @@ async def responses_endpoint_handler(
         and AzureEntraIDManager().is_token_expired
         and AzureEntraIDManager().refresh_token()
     ):
-        client = await update_azure_token(client)
+        client = await AsyncLlamaStackClientHolder().update_azure_token()
 
     input_text = (
         original_request.input
@@ -455,12 +454,7 @@ async def responses_endpoint_handler(
             original_request.input, inline_rag_context.context_text
         )
 
-    api_params = ResponsesApiParams.model_validate(
-        {
-            **updated_request.model_dump(exclude={"tools"}),
-            "tools": updated_request.tools,
-        }
-    )
+    api_params = ResponsesApiParams.model_validate(updated_request.model_dump())
     context = ResponsesContext(
         client=client,
         auth=auth,

{lightspeed_stack-0.6.0rc1 → lightspeed_stack-0.6.0rc2}/src/app/endpoints/responses_telemetry.py

@@ -10,8 +10,8 @@ from typing import Optional
 from fastapi import BackgroundTasks
 
 from log import get_logger
+from models.common.responses.contexts import ResponsesContext
 from models.common.responses.responses_api_params import ResponsesApiParams
-from models.common.responses.responses_context import ResponsesContext
 from models.common.turn_summary import TurnSummary
 from observability import ResponsesEventData, build_responses_event
 from observability.splunk import dispatch_splunk_event

{lightspeed_stack-0.6.0rc1 → lightspeed_stack-0.6.0rc2}/src/app/endpoints/rlsapi_v1.py

@@ -20,12 +20,14 @@ from openai._exceptions import APIStatusError as OpenAIAPIStatusError
 import constants
 from authentication import get_auth_dependency
 from authentication.interface import AuthTuple
+from authorization.azure_token_manager import AzureEntraIDManager
 from authorization.middleware import authorize
 from client import AsyncLlamaStackClientHolder
 from configuration import configuration
 from constants import ENDPOINT_PATH_INFER
 from log import get_logger
 from metrics import recording
+from models.api.requests.rlsapi import RlsapiV1InferRequest, RlsapiV1SystemInfo
 from models.api.responses.constants import UNAUTHORIZED_OPENAPI_EXAMPLES
 from models.api.responses.error import (
     ForbiddenResponse,
@@ -37,9 +39,11 @@ from models.api.responses.error import (
     UnauthorizedResponse,
     UnprocessableEntityResponse,
 )
+from models.api.responses.successful.rlsapi import (
+    RlsapiV1InferData,
+    RlsapiV1InferResponse,
+)
 from models.config import Action
-from models.rlsapi.requests import RlsapiV1InferRequest, RlsapiV1SystemInfo
-from models.rlsapi.responses import RlsapiV1InferData, RlsapiV1InferResponse
 from observability import InferenceEventData, build_inference_event, send_splunk_event
 from utils.endpoints import check_configuration_loaded
 from utils.query import (
@@ -327,6 +331,16 @@ async def _call_llm(
     """
     client = AsyncLlamaStackClientHolder().get_client()
     resolved_model_id = model_id or await _get_default_model_id()
+
+    # Handle Azure token refresh if needed
+    if (
+        resolved_model_id.startswith("azure")
+        and AzureEntraIDManager().is_entra_id_configured
+        and AzureEntraIDManager().is_token_expired
+        and AzureEntraIDManager().refresh_token()
+    ):
+        client = await AsyncLlamaStackClientHolder().update_azure_token()
+
     logger.debug("Using model %s for rlsapi v1 inference", resolved_model_id)
 
     response = await client.responses.create(

{lightspeed_stack-0.6.0rc1 → lightspeed_stack-0.6.0rc2}/src/app/endpoints/streaming_query.py

@@ -74,10 +74,10 @@ from models.api.responses.error import (
     UnprocessableEntityResponse,
 )
 from models.api.responses.successful import StreamingQueryResponse
+from models.common.responses.contexts import ResponseGeneratorContext
 from models.common.responses.responses_api_params import ResponsesApiParams
 from models.common.turn_summary import ReferencedDocument, TurnSummary
 from models.config import Action
-from models.context import ResponseGeneratorContext
 from utils.conversations import append_turn_items_to_conversation
 from utils.endpoints import (
     check_configuration_loaded,
@@ -92,7 +92,6 @@ from utils.query import (
     is_context_length_error,
     prepare_input,
     store_query_results,
-    update_azure_token,
     update_conversation_topic_summary,
     validate_attachments_metadata,
     validate_model_provider_override,
@@ -262,7 +261,7 @@ async def streaming_query_endpoint_handler(  # pylint: disable=too-many-locals
         and AzureEntraIDManager().is_token_expired
         and AzureEntraIDManager().refresh_token()
     ):
-        client = await update_azure_token(client)
+        client = await AsyncLlamaStackClientHolder().update_azure_token()
 
     request_id = get_suid()
 

{lightspeed_stack-0.6.0rc1 → lightspeed_stack-0.6.0rc2}/src/app/main.py

@@ -9,7 +9,7 @@ import sentry_sdk  # pyright: ignore[reportMissingImports]
 from fastapi import FastAPI, HTTPException
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.responses import JSONResponse
-from llama_stack_client import APIConnectionError
+from llama_stack_client import APIConnectionError, AsyncLlamaStackClient
 from starlette.routing import Mount, Route, WebSocketRoute
 from starlette.types import ASGIApp, Message, Receive, Scope, Send
 
@@ -77,22 +77,21 @@ async def lifespan(_app: FastAPI) -> AsyncIterator[None]:
 
     initialize_sentry()
 
-    azure_config = configuration.configuration.azure_entra_id
-    if azure_config is not None:
-        AzureEntraIDManager().set_config(azure_config)
-        if not AzureEntraIDManager().refresh_token():
-            logger.warning(
-                "Failed to refresh Azure token at startup. "
-                "Token refresh will be retried on next Azure request."
-            )
-
     llama_stack_config = configuration.configuration.llama_stack
     await AsyncLlamaStackClientHolder().load(llama_stack_config)
-    client = AsyncLlamaStackClientHolder().get_client()
+    client: AsyncLlamaStackClient = AsyncLlamaStackClientHolder().get_client()
+    logger.debug("Llama Stack client initialized, trying to connect to Llama Stack")
     # check if the Llama Stack version is supported by the service
     try:
-        await check_llama_stack_version(client)
+        llama_stack_version = await check_llama_stack_version(
+            client, llama_stack_config.max_retries, llama_stack_config.retry_delay
+        )
+        if llama_stack_version is None:
+            logger.error("Cannot retrieve Llama Stack version, check connection")
+        else:
+            logger.debug("Llama Stack version: %s", llama_stack_version)
     except APIConnectionError as e:
+        # if degraded mode is allowed, simply ignore the exception
         llama_stack_url = llama_stack_config.url
         logger.error(
             "Failed to connect to Llama Stack at '%s'. "
@@ -102,8 +101,16 @@ async def lifespan(_app: FastAPI) -> AsyncIterator[None]:
             llama_stack_url,
             e,
         )
-        raise
+        if llama_stack_config.allow_degraded_mode:
+            logger.info("Entering degraded mode: LCORE running w/o Llama Stack")
+        else:
+            raise
 
+    azure_entra_id_config = configuration.configuration.azure_entra_id
+    if azure_entra_id_config is not None:
+        AzureEntraIDManager().set_config(azure_entra_id_config)
+        azure_base_url = await AsyncLlamaStackClientHolder().get_azure_base_url()
+        AzureEntraIDManager().set_base_url(azure_base_url)
     logger.info("Registering MCP servers")
     await register_mcp_servers_async(logger, configuration.configuration)
     logger.info("App startup complete")

{lightspeed_stack-0.6.0rc1 → lightspeed_stack-0.6.0rc2}/src/authentication/api_key_token.py

@@ -10,8 +10,9 @@ import secrets
 
 from fastapi import HTTPException, Request, status
 
-from authentication.interface import AuthInterface, AuthTuple
+from authentication.interface import NO_AUTH_TUPLE, AuthInterface, AuthTuple
 from authentication.utils import extract_user_token
+from configuration import configuration
 from constants import (
     DEFAULT_USER_NAME,
     DEFAULT_USER_UID,
@@ -23,6 +24,19 @@ from models.config import APIKeyTokenConfiguration
 logger = get_logger(__name__)
 
 
+def _should_skip_auth(request: Request) -> bool:
+    """Check if auth should be skipped for health probe or metrics endpoints."""
+    auth_config = configuration.authentication_configuration
+    if auth_config.skip_for_health_probes and request.url.path in (
+        "/readiness",
+        "/liveness",
+    ):
+        return True
+    if auth_config.skip_for_metrics and request.url.path in ("/metrics",):
+        return True
+    return False
+
+
 class APIKeyTokenAuthDependency(
     AuthInterface
 ):  # pylint: disable=too-few-public-methods
@@ -59,6 +73,10 @@ class APIKeyTokenAuthDependency(
             HTTPException: If the bearer token is missing or
             doesn't match the configured API key (HTTP 401).
         """
+        if not request.headers.get("Authorization"):
+            if _should_skip_auth(request):
+                return NO_AUTH_TUPLE
+
         # try to extract user token from request
         user_token = extract_user_token(request.headers)