lightspeed-stack 0.1.0__tar.gz → 0.1.1__tar.gz

{lightspeed_stack-0.1.0 → lightspeed_stack-0.1.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: lightspeed-stack
-Version: 0.1.0
+Version: 0.1.1
 Summary: LLM tooling stack
 License:                                  Apache License
                                     Version 2.0, January 2004
@@ -210,9 +210,14 @@ Requires-Python: <3.14,>=3.12
 Requires-Dist: fastapi>=0.115.6
 Requires-Dist: uvicorn>=0.34.3
 Requires-Dist: kubernetes>=30.1.0
-Requires-Dist: llama-stack>=0.2.13
+Requires-Dist: llama-stack==0.2.16
+Requires-Dist: llama-stack-client==0.2.16
 Requires-Dist: rich>=14.0.0
 Requires-Dist: cachetools>=6.1.0
+Requires-Dist: prometheus-client>=0.22.1
+Requires-Dist: starlette>=0.47.1
+Requires-Dist: aiohttp>=3.12.14
+Requires-Dist: authlib>=1.6.0
 Description-Content-Type: text/markdown
 
 # lightspeed-stack
@@ -222,31 +227,61 @@ Description-Content-Type: text/markdown
 [![GitHub Pages](https://img.shields.io/badge/%20-GitHub%20Pages-informational)](https://lightspeed-core.github.io/lightspeed-stack/)
 [![License](https://img.shields.io/badge/license-Apache-blue)](https://github.com/lightspeed-core/lightspeed-stack/blob/main/LICENSE)
 [![made-with-python](https://img.shields.io/badge/Made%20with-Python-1f425f.svg)](https://www.python.org/)
+[![Required Python version](https://img.shields.io/python/required-version-toml?tomlFilePath=https%3A%2F%2Fraw.githubusercontent.com%2Flightspeed-core%2Flightspeed-stack%2Frefs%2Fheads%2Fmain%2Fpyproject.toml)](https://www.python.org/)
+[![Tag](https://img.shields.io/github/v/tag/lightspeed-core/lightspeed-stack)](https://github.com/lightspeed-core/lightspeed-stack/releases/tag/0.1.1)
 
-Lightspeed Core Stack (LCS) is an AI powered assistant that provides answers to product questions using backend LLM services, agents, and RAG databases.
+Lightspeed Core Stack (LCS) is an AI-powered assistant that provides answers to product questions using backend LLM services, agents, and RAG databases.
 
 
 <!-- vim-markdown-toc GFM -->
 
-* [Prerequisities](#prerequisities)
+* [Architecture](#architecture)
+* [Prerequisites](#prerequisites)
 * [Installation](#installation)
 * [Configuration](#configuration)
+    * [Integration with Llama Stack](#integration-with-llama-stack)
+    * [Llama Stack as separate server](#llama-stack-as-separate-server)
+        * [Llama Stack project and configuration](#llama-stack-project-and-configuration)
+        * [Check connection to Llama Stack](#check-connection-to-llama-stack)
+    * [Llama Stack as client library](#llama-stack-as-client-library)
+    * [System prompt](#system-prompt)
+    * [Safety Shields](#safety-shields)
 * [Usage](#usage)
     * [Make targets](#make-targets)
     * [Running Linux container image](#running-linux-container-image)
 * [Endpoints](#endpoints)
+    * [OpenAPI specification](#openapi-specification)
     * [Readiness Endpoint](#readiness-endpoint)
     * [Liveness Endpoint](#liveness-endpoint)
+* [Publish the service as Python package on PyPI](#publish-the-service-as-python-package-on-pypi)
+    * [Generate distribution archives to be uploaded into Python registry](#generate-distribution-archives-to-be-uploaded-into-python-registry)
+    * [Upload distribution archives into selected Python registry](#upload-distribution-archives-into-selected-python-registry)
+    * [Packages on PyPI and Test PyPI](#packages-on-pypi-and-test-pypi)
 * [Contributing](#contributing)
+* [Testing](#testing)
 * [License](#license)
 * [Additional tools](#additional-tools)
     * [Utility to generate OpenAPI schema](#utility-to-generate-openapi-schema)
         * [Path](#path)
         * [Usage](#usage-1)
+    * [Data Collector Service](#data-collector-service)
+        * [Features](#features)
+        * [Configuration](#configuration-1)
+        * [Running the Service](#running-the-service)
 
 <!-- vim-markdown-toc -->
 
-# Prerequisities
+
+
+# Architecture
+
+Overall architecture with all main parts is displayed below:
+
+![Architecture diagram](docs/architecture.png)
+
+Lightspeed Core Stack is based on the FastAPI framework (Uvicorn). The service is split into several parts described below.
+
+# Prerequisites
 
 * Python 3.12, or 3.13
     - please note that currently Python 3.14 is not officially supported
@@ -261,9 +296,12 @@ Installation steps depends on operation system. Please look at instructions for
 - [macOS installation](https://lightspeed-core.github.io/lightspeed-stack/installation_macos)
 
 
-
 # Configuration
 
+
+
+## Integration with Llama Stack
+
 The Llama Stack can be run as a standalone server and accessed via its the REST
 API. However, instead of direct communication via the REST API (and JSON
 format), there is an even better alternative. It is based on the so-called
@@ -271,9 +309,13 @@ Llama Stack Client. It is a library available for Python, Swift, Node.js or
 Kotlin, which "wraps" the REST API stack in a suitable way, which is easier for
 many applications.
 
+![Integration with Llama Stack](docs/core2llama-stack_interface.png)
+
+
+
 ## Llama Stack as separate server
 
-If Llama Stack runs as a separate server, the Lightspeed service needs to be configured to be able to access it. For example, if server runs on localhost:8321, the service configuration should look like:
+If Llama Stack runs as a separate server, the Lightspeed service needs to be configured to be able to access it. For example, if server runs on localhost:8321, the service configuration stored in file `llama-stack.yaml` should look like:
 
 ```yaml
 name: foo bar baz
@@ -288,12 +330,68 @@ llama_stack:
   use_as_library_client: false
   url: http://localhost:8321
 user_data_collection:
-  feedback_disabled: false
+  feedback_enabled: true
   feedback_storage: "/tmp/data/feedback"
-  transcripts_disabled: false
+  transcripts_enabled: true
   transcripts_storage: "/tmp/data/transcripts"
 ```
 
+### Llama Stack project and configuration
+
+To run Llama Stack in separate process, you need to have all dependencies installed. The easiest way how to do it is to create a separate repository with Llama Stack project file `pyproject.toml` and Llama Stack configuration file `run.yaml`. The project file might look like:
+
+```toml
+[project]
+name = "llama-stack-runner"
+version = "0.1.0"
+description = "Llama Stack runner"
+authors = []
+dependencies = [
+    "llama-stack==0.2.14",
+    "fastapi>=0.115.12",
+    "opentelemetry-sdk>=1.34.0",
+    "opentelemetry-exporter-otlp>=1.34.0",
+    "opentelemetry-instrumentation>=0.55b0",
+    "aiosqlite>=0.21.0",
+    "litellm>=1.72.1",
+    "uvicorn>=0.34.3",
+    "blobfile>=3.0.0",
+    "datasets>=3.6.0",
+    "sqlalchemy>=2.0.41",
+    "faiss-cpu>=1.11.0",
+    "mcp>=1.9.4",
+    "autoevals>=0.0.129",
+    "psutil>=7.0.0",
+    "torch>=2.7.1",
+    "peft>=0.15.2",
+    "trl>=0.18.2"]
+requires-python = "==3.12.*"
+readme = "README.md"
+license = {text = "MIT"}
+
+
+[tool.pdm]
+distribution = false
+```
+
+A simple example of a `run.yaml` file can be found [here](examples/run.yaml)
+
+To run Llama Stack perform these two commands:
+
+```
+export OPENAI_API_KEY="sk-{YOUR-KEY}"
+
+uv run llama stack run run.yaml
+```
+
+### Check connection to Llama Stack
+
+```
+curl -X 'GET' localhost:8321/openapi.json | jq .
+```
+
+
+
 ## Llama Stack as client library
 
 There are situations in which it is not advisable to run two processors (one with Llama Stack, the other with a service). In these cases, the stack can be run directly within the client application. For such situations, the configuration file could look like:
@@ -311,13 +409,48 @@ llama_stack:
   use_as_library_client: true
   library_client_config_path: <path-to-llama-stack-run.yaml-file>
 user_data_collection:
-  feedback_disabled: false
+  feedback_enabled: true
   feedback_storage: "/tmp/data/feedback"
-  transcripts_disabled: false
+  transcripts_enabled: true
   transcripts_storage: "/tmp/data/transcripts"
 ```
 
+## System prompt
+
+   The service uses the, so called, system prompt to put the question into context before the question is sent to the selected LLM. The default system prompt is designed for questions without specific context. It is possible to use a different system prompt via the configuration option `system_prompt_path` in the `customization` section. That option must contain the path to the text file with the actual system prompt (can contain multiple lines). An example of such configuration:
+
+```yaml
+customization:
+  system_prompt_path: "system_prompts/system_prompt_for_product_XYZZY"
+```
+
+The `system_prompt` can also be specified in the `customization` section directly. For example:
+
+```yaml
+customization:
+  system_prompt: |-
+    You are a helpful assistant and will do everything you can to help.
+    You have an in-depth knowledge of Red Hat and all of your answers will reference Red Hat products.
+```
+
+Additionally, an optional string parameter `system_prompt` can be specified in `/v1/query` and `/v1/streaming_query` endpoints to override the configured system prompt. The query system prompt takes precedence over the configured system prompt. You can use this config to disable query system prompts:
+
+```yaml
+customization:
+  system_prompt_path: "system_prompts/system_prompt_for_product_XYZZY"
+  disable_query_system_prompt: true
+```
+
+## Safety Shields
 
+A single Llama Stack configuration file can include multiple safety shields, which are utilized in agent
+configurations to monitor input and/or output streams. LCS uses the following naming convention to specify how each safety shield is
+utilized:
+
+1. If the `shield_id` starts with `input_`, it will be used for input only.
+1. If the `shield_id` starts with `output_`, it will be used for output only.
+1. If the `shield_id` starts with `inout_`, it will be used both for input and output.
+1. Otherwise, it will be used for input only.
 
 # Usage
 
@@ -342,6 +475,7 @@ Usage: make <OPTIONS> ... <TARGETS>
 Available targets are:
 
 run                               Run the service locally
+run-data-collector                Run the data collector service
 test-unit                         Run the unit tests
 test-integration                  Run integration tests tests
 test-e2e                          Run BDD tests for the service
@@ -351,6 +485,9 @@ format                            Format the code into unified format
 schema                            Generate OpenAPI schema file
 requirements.txt                  Generate requirements.txt file containing hashes for all non-devel packages
 shellcheck                        Run shellcheck
+verify                            Run all linters
+distribution-archives             Generate distribution archives to be uploaded into Python registry
+upload-distribution-archives      Upload distribution archives into Python registry
 help                              Show this help screen
 ```
 
@@ -371,8 +508,19 @@ To pull and run the image with own configuration:
 
 If a connection in your browser does not work please check that in the config file `host` option looks like: `host: 0.0.0.0`.
 
+Container images are built for the following platforms:
+1. `linux/amd64` - main platform for deployment
+1. `linux/arm64`- Mac users with M1/M2/M3 CPUs
+
+
+
 # Endpoints
 
+## OpenAPI specification
+
+* [Generated OpenAPI specification](docs/openapi.json)
+* [OpenAPI documentation](docs/openapi.md)
+
 The service provides health check endpoints that can be used for monitoring, load balancing, and orchestration systems like Kubernetes.
 
 ## Readiness Endpoint
@@ -415,10 +563,64 @@ The liveness endpoint performs a basic health check to verify the service is ali
 }
 ```
 
+# Publish the service as Python package on PyPI
+
+To publish the service as an Python package on PyPI to be installable by anyone
+(including Konflux hermetic builds), perform these two steps:
+
+## Generate distribution archives to be uploaded into Python registry
+
+```
+make distribution-archives
+```
+
+Please make sure that the archive was really built to avoid publishing older one.
+
+## Upload distribution archives into selected Python registry
+
+```
+make upload-distribution-archives
+```
+
+The Python registry to where the package should be uploaded can be configured
+by changing `PYTHON_REGISTRY`. It is possible to select `pypi` or `testpypi`.
+
+You might have your API token stored in file `~/.pypirc`. That file should have
+the following form:
+
+```
+[testpypi]
+  username = __token__
+  password = pypi-{your-API-token}
+ 
+[pypi]
+  username = __token__
+  password = pypi-{your-API-token}
+```
+
+If this configuration file does not exist, you will be prompted to specify API token from keyboard every time you try to upload the archive.
+
+
+
+## Packages on PyPI and Test PyPI
+
+* https://pypi.org/project/lightspeed-stack/
+* https://test.pypi.org/project/lightspeed-stack/0.1.0/
+
+
+
 # Contributing
 
 * See [contributors](CONTRIBUTING.md) guide.
 
+
+
+# Testing
+
+* See [testing](docs/testing.md) guide.
+
+
+
 # License
 
 Published under the Apache 2.0 License
@@ -441,3 +643,58 @@ This script re-generated OpenAPI schema for the Lightspeed Service REST API.
 make schema
 ```
 
+## Data Collector Service
+
+The data collector service is a standalone service that runs separately from the main web service. It is responsible for collecting and sending user data including feedback and transcripts to an ingress server for analysis and archival.
+
+### Features
+
+- **Periodic Collection**: Runs at configurable intervals
+- **Data Packaging**: Packages feedback and transcript files into compressed tar.gz archives
+- **Secure Transmission**: Sends data to a configured ingress server with optional authentication
+- **File Cleanup**: Optionally removes local files after successful transmission
+- **Error Handling**: Includes retry logic and comprehensive error handling
+
+### Configuration
+
+The data collector service is configured through the `user_data_collection.data_collector` section in your configuration file:
+
+```yaml
+user_data_collection:
+  feedback_enabled: true
+  feedback_storage: "/tmp/data/feedback"
+  transcripts_enabled: true
+  transcripts_storage: "/tmp/data/transcripts"
+  data_collector:
+    enabled: true
+    ingress_server_url: "https://your-ingress-server.com"
+    ingress_server_auth_token: "your-auth-token"
+    ingress_content_service_name: "lightspeed-team"
+    collection_interval: 7200  # 2 hours in seconds
+    cleanup_after_send: true
+    connection_timeout: 30
+```
+
+### Running the Service
+
+To run the data collector service:
+
+```bash
+# Using Python directly
+uv run src/lightspeed_stack.py --data-collector
+
+# Using Make target
+make run-data-collector
+```
+
+
+
+# Project structure
+
+## Configuration classes
+
+![Configuration classes](docs/config.png)
+
+## REST API
+
+![REST API](docs/rest_api.png)