variably-sdk 2.6.0__tar.gz → 2.6.1__tar.gz

{variably_sdk-2.6.0 → variably_sdk-2.6.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: variably-sdk
-Version: 2.6.0
+Version: 2.6.1
 Summary: Official Python SDK for Variably feature flags, LLM experimentation, and prompt optimization platform
 Author: Variably
 Author-email: Variably <support@variably.com>
@@ -45,7 +45,7 @@ Dynamic: requires-python
 
 # Variably Python SDK
 
-Official Python SDK for Variably — feature flags, LLM experimentation, and prompt optimization.
+Official Python SDK for Variably — LLM evaluation, experimentation, and prompt optimization.
 
 ## Installation
 
@@ -53,52 +53,124 @@ Official Python SDK for Variably — feature flags, LLM experimentation, and pro
 pip install variably-sdk
 ```
 
-## Quick Start
+## Quick Start — Observe Mode
+
+Add **one line** to your existing LLM app and get **multi-dimension evaluation** across 40+ metrics in 6 categories: Quality, Safety, Semantic, Grounding, Coherence, and Advanced.
+
+No experiment setup. No prompt migration. Just log and see scores.
+
+### 1. Set your environment variables
+
+```bash
+VARIABLY_API_KEY=vbl_your_key_here
+VARIABLY_BASE_URL=https://api.variably.tech
+```
+
+### 2. Add one line after your LLM call
 
 ```python
-from variably import VariablyClient
+from variably import observe
 
-# Initialize the client
-client = VariablyClient({
-    "api_key": "your-api-key",
-    "base_url": "https://api.variably.com",  # optional, defaults to localhost:8080
-    "environment": "production"  # optional
-})
+# Your existing code (unchanged)
+response = your_llm_call(user_query)
 
-# Evaluate a boolean feature flag
-user_context = {
-    "user_id": "user-123",
-    "email": "user@example.com",
-    "country": "US"
-}
+# Add this line:
+observe(prompt=user_query, response=response)
+```
 
-is_feature_enabled = client.evaluate_flag_bool(
-    "new-checkout-flow",
-    False,  # default value
-    user_context
+### Auto-extract tokens & model from provider response
+
+```python
+# OpenAI
+from openai import OpenAI
+from variably import observe
+
+client = OpenAI()
+completion = client.chat.completions.create(
+    model="gpt-4o",
+    messages=[{"role": "user", "content": user_query}],
 )
 
-if is_feature_enabled:
-    # Show new checkout flow
-    pass
+observe(
+    prompt=user_query,
+    response=completion.choices[0].message.content,
+    provider_response=completion,  # auto-extracts model, tokens
+)
+```
 
-# Evaluate a feature gate
-has_access = client.evaluate_gate("premium-features", user_context)
+```python
+# Anthropic
+import anthropic
+from variably import observe
+
+client = anthropic.Anthropic()
+message = client.messages.create(
+    model="claude-sonnet-4-20250514",
+    max_tokens=1024,
+    messages=[{"role": "user", "content": user_query}],
+)
 
-# Track events
-client.track({
-    "name": "button_clicked",
-    "user_id": "user-123",
-    "properties": {
-        "button_name": "checkout",
-        "page": "product-detail"
-    }
-})
+observe(
+    prompt=user_query,
+    response=message.content[0].text,
+    provider_response=message,  # auto-extracts model, tokens
+)
+```
 
-# Clean up resources
-client.close()
+### RAG applications — grounding & hallucination scoring
+
+```python
+observe(
+    prompt=user_query,
+    response=llm_answer,
+    provider_response=completion,
+    reference_materials=[
+        {"id": "chunk-1", "content": "Retrieved text...", "source": "docs.pdf"},
+        {"id": "chunk-2", "content": "Another chunk...", "source": "faq.md"},
+    ],
+    retrieval_query=user_query,
+)
 ```
 
+### Multi-turn chat — coherence scoring
+
+```python
+observe(
+    prompt=latest_user_message,
+    response=llm_answer,
+    provider_response=completion,
+    conversation_history=[
+        {"role": "user", "content": "What is diabetes?"},
+        {"role": "assistant", "content": "Diabetes is a chronic condition..."},
+        {"role": "user", "content": "What are the symptoms?"},
+    ],
+    session_id="conv-123",
+)
+```
+
+### Full parameter reference
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `prompt` | str | Yes | The user's input / question |
+| `response` | str | Yes | The LLM's generated response |
+| `provider_response` | object | No | Raw OpenAI/Anthropic/Google response — auto-extracts model, tokens |
+| `model` | str | No | Model name (auto-extracted if provider_response given) |
+| `provider` | str | No | "openai", "anthropic", etc. (auto-detected) |
+| `latency_ms` | int | No | Response generation time in milliseconds |
+| `prompt_tokens` | int | No | Input token count (auto-extracted if provider_response given) |
+| `completion_tokens` | int | No | Output token count (auto-extracted) |
+| `cost` | float | No | Cost in USD |
+| `reference_materials` | list[dict] | No | RAG chunks: `[{"id", "content", "source"}]` — enables grounding scoring |
+| `retrieval_query` | str | No | Query sent to retriever — enables retrieval quality scoring |
+| `conversation_history` | list[dict] | No | Prior turns: `[{"role", "content"}]` — enables coherence scoring |
+| `tags` | list[str] | No | Grouping labels, e.g. `["production", "rag"]` |
+| `user_id` | str | No | Your user's ID |
+| `session_id` | str | No | Conversation session ID (groups multi-turn) |
+| `metadata` | dict | No | Any extra key-value data |
+
+---
+
 ## Prompt Experimentation
 
 Variably provides two modes for LLM prompt experimentation:

{variably_sdk-2.6.0 → variably_sdk-2.6.1}/README.md

@@ -1,6 +1,6 @@
 # Variably Python SDK
 
-Official Python SDK for Variably — feature flags, LLM experimentation, and prompt optimization.
+Official Python SDK for Variably — LLM evaluation, experimentation, and prompt optimization.
 
 ## Installation
 
@@ -8,52 +8,124 @@ Official Python SDK for Variably — feature flags, LLM experimentation, and pro
 pip install variably-sdk
 ```
 
-## Quick Start
+## Quick Start — Observe Mode
+
+Add **one line** to your existing LLM app and get **multi-dimension evaluation** across 40+ metrics in 6 categories: Quality, Safety, Semantic, Grounding, Coherence, and Advanced.
+
+No experiment setup. No prompt migration. Just log and see scores.
+
+### 1. Set your environment variables
+
+```bash
+VARIABLY_API_KEY=vbl_your_key_here
+VARIABLY_BASE_URL=https://api.variably.tech
+```
+
+### 2. Add one line after your LLM call
 
 ```python
-from variably import VariablyClient
+from variably import observe
 
-# Initialize the client
-client = VariablyClient({
-    "api_key": "your-api-key",
-    "base_url": "https://api.variably.com",  # optional, defaults to localhost:8080
-    "environment": "production"  # optional
-})
+# Your existing code (unchanged)
+response = your_llm_call(user_query)
 
-# Evaluate a boolean feature flag
-user_context = {
-    "user_id": "user-123",
-    "email": "user@example.com",
-    "country": "US"
-}
+# Add this line:
+observe(prompt=user_query, response=response)
+```
 
-is_feature_enabled = client.evaluate_flag_bool(
-    "new-checkout-flow",
-    False,  # default value
-    user_context
+### Auto-extract tokens & model from provider response
+
+```python
+# OpenAI
+from openai import OpenAI
+from variably import observe
+
+client = OpenAI()
+completion = client.chat.completions.create(
+    model="gpt-4o",
+    messages=[{"role": "user", "content": user_query}],
 )
 
-if is_feature_enabled:
-    # Show new checkout flow
-    pass
+observe(
+    prompt=user_query,
+    response=completion.choices[0].message.content,
+    provider_response=completion,  # auto-extracts model, tokens
+)
+```
 
-# Evaluate a feature gate
-has_access = client.evaluate_gate("premium-features", user_context)
+```python
+# Anthropic
+import anthropic
+from variably import observe
+
+client = anthropic.Anthropic()
+message = client.messages.create(
+    model="claude-sonnet-4-20250514",
+    max_tokens=1024,
+    messages=[{"role": "user", "content": user_query}],
+)
 
-# Track events
-client.track({
-    "name": "button_clicked",
-    "user_id": "user-123",
-    "properties": {
-        "button_name": "checkout",
-        "page": "product-detail"
-    }
-})
+observe(
+    prompt=user_query,
+    response=message.content[0].text,
+    provider_response=message,  # auto-extracts model, tokens
+)
+```
 
-# Clean up resources
-client.close()
+### RAG applications — grounding & hallucination scoring
+
+```python
+observe(
+    prompt=user_query,
+    response=llm_answer,
+    provider_response=completion,
+    reference_materials=[
+        {"id": "chunk-1", "content": "Retrieved text...", "source": "docs.pdf"},
+        {"id": "chunk-2", "content": "Another chunk...", "source": "faq.md"},
+    ],
+    retrieval_query=user_query,
+)
 ```
 
+### Multi-turn chat — coherence scoring
+
+```python
+observe(
+    prompt=latest_user_message,
+    response=llm_answer,
+    provider_response=completion,
+    conversation_history=[
+        {"role": "user", "content": "What is diabetes?"},
+        {"role": "assistant", "content": "Diabetes is a chronic condition..."},
+        {"role": "user", "content": "What are the symptoms?"},
+    ],
+    session_id="conv-123",
+)
+```
+
+### Full parameter reference
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `prompt` | str | Yes | The user's input / question |
+| `response` | str | Yes | The LLM's generated response |
+| `provider_response` | object | No | Raw OpenAI/Anthropic/Google response — auto-extracts model, tokens |
+| `model` | str | No | Model name (auto-extracted if provider_response given) |
+| `provider` | str | No | "openai", "anthropic", etc. (auto-detected) |
+| `latency_ms` | int | No | Response generation time in milliseconds |
+| `prompt_tokens` | int | No | Input token count (auto-extracted if provider_response given) |
+| `completion_tokens` | int | No | Output token count (auto-extracted) |
+| `cost` | float | No | Cost in USD |
+| `reference_materials` | list[dict] | No | RAG chunks: `[{"id", "content", "source"}]` — enables grounding scoring |
+| `retrieval_query` | str | No | Query sent to retriever — enables retrieval quality scoring |
+| `conversation_history` | list[dict] | No | Prior turns: `[{"role", "content"}]` — enables coherence scoring |
+| `tags` | list[str] | No | Grouping labels, e.g. `["production", "rag"]` |
+| `user_id` | str | No | Your user's ID |
+| `session_id` | str | No | Conversation session ID (groups multi-turn) |
+| `metadata` | dict | No | Any extra key-value data |
+
+---
+
 ## Prompt Experimentation
 
 Variably provides two modes for LLM prompt experimentation:

{variably_sdk-2.6.0 → variably_sdk-2.6.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "variably-sdk"
-version = "2.6.0"
+version = "2.6.1"
 description = "Official Python SDK for Variably feature flags, LLM experimentation, and prompt optimization platform"
 readme = "README.md"
 requires-python = ">=3.7"

{variably_sdk-2.6.0 → variably_sdk-2.6.1}/src/variably/version.py

@@ -1,3 +1,3 @@
 """Version information for Variably SDK."""
 
-__version__ = "2.6.0"
+__version__ = "2.6.1"

{variably_sdk-2.6.0 → variably_sdk-2.6.1}/src/variably_sdk.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: variably-sdk
-Version: 2.6.0
+Version: 2.6.1
 Summary: Official Python SDK for Variably feature flags, LLM experimentation, and prompt optimization platform
 Author: Variably
 Author-email: Variably <support@variably.com>
@@ -45,7 +45,7 @@ Dynamic: requires-python
 
 # Variably Python SDK
 
-Official Python SDK for Variably — feature flags, LLM experimentation, and prompt optimization.
+Official Python SDK for Variably — LLM evaluation, experimentation, and prompt optimization.
 
 ## Installation
 
@@ -53,52 +53,124 @@ Official Python SDK for Variably — feature flags, LLM experimentation, and pro
 pip install variably-sdk
 ```
 
-## Quick Start
+## Quick Start — Observe Mode
+
+Add **one line** to your existing LLM app and get **multi-dimension evaluation** across 40+ metrics in 6 categories: Quality, Safety, Semantic, Grounding, Coherence, and Advanced.
+
+No experiment setup. No prompt migration. Just log and see scores.
+
+### 1. Set your environment variables
+
+```bash
+VARIABLY_API_KEY=vbl_your_key_here
+VARIABLY_BASE_URL=https://api.variably.tech
+```
+
+### 2. Add one line after your LLM call
 
 ```python
-from variably import VariablyClient
+from variably import observe
 
-# Initialize the client
-client = VariablyClient({
-    "api_key": "your-api-key",
-    "base_url": "https://api.variably.com",  # optional, defaults to localhost:8080
-    "environment": "production"  # optional
-})
+# Your existing code (unchanged)
+response = your_llm_call(user_query)
 
-# Evaluate a boolean feature flag
-user_context = {
-    "user_id": "user-123",
-    "email": "user@example.com",
-    "country": "US"
-}
+# Add this line:
+observe(prompt=user_query, response=response)
+```
 
-is_feature_enabled = client.evaluate_flag_bool(
-    "new-checkout-flow",
-    False,  # default value
-    user_context
+### Auto-extract tokens & model from provider response
+
+```python
+# OpenAI
+from openai import OpenAI
+from variably import observe
+
+client = OpenAI()
+completion = client.chat.completions.create(
+    model="gpt-4o",
+    messages=[{"role": "user", "content": user_query}],
 )
 
-if is_feature_enabled:
-    # Show new checkout flow
-    pass
+observe(
+    prompt=user_query,
+    response=completion.choices[0].message.content,
+    provider_response=completion,  # auto-extracts model, tokens
+)
+```
 
-# Evaluate a feature gate
-has_access = client.evaluate_gate("premium-features", user_context)
+```python
+# Anthropic
+import anthropic
+from variably import observe
+
+client = anthropic.Anthropic()
+message = client.messages.create(
+    model="claude-sonnet-4-20250514",
+    max_tokens=1024,
+    messages=[{"role": "user", "content": user_query}],
+)
 
-# Track events
-client.track({
-    "name": "button_clicked",
-    "user_id": "user-123",
-    "properties": {
-        "button_name": "checkout",
-        "page": "product-detail"
-    }
-})
+observe(
+    prompt=user_query,
+    response=message.content[0].text,
+    provider_response=message,  # auto-extracts model, tokens
+)
+```
 
-# Clean up resources
-client.close()
+### RAG applications — grounding & hallucination scoring
+
+```python
+observe(
+    prompt=user_query,
+    response=llm_answer,
+    provider_response=completion,
+    reference_materials=[
+        {"id": "chunk-1", "content": "Retrieved text...", "source": "docs.pdf"},
+        {"id": "chunk-2", "content": "Another chunk...", "source": "faq.md"},
+    ],
+    retrieval_query=user_query,
+)
 ```
 
+### Multi-turn chat — coherence scoring
+
+```python
+observe(
+    prompt=latest_user_message,
+    response=llm_answer,
+    provider_response=completion,
+    conversation_history=[
+        {"role": "user", "content": "What is diabetes?"},
+        {"role": "assistant", "content": "Diabetes is a chronic condition..."},
+        {"role": "user", "content": "What are the symptoms?"},
+    ],
+    session_id="conv-123",
+)
+```
+
+### Full parameter reference
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `prompt` | str | Yes | The user's input / question |
+| `response` | str | Yes | The LLM's generated response |
+| `provider_response` | object | No | Raw OpenAI/Anthropic/Google response — auto-extracts model, tokens |
+| `model` | str | No | Model name (auto-extracted if provider_response given) |
+| `provider` | str | No | "openai", "anthropic", etc. (auto-detected) |
+| `latency_ms` | int | No | Response generation time in milliseconds |
+| `prompt_tokens` | int | No | Input token count (auto-extracted if provider_response given) |
+| `completion_tokens` | int | No | Output token count (auto-extracted) |
+| `cost` | float | No | Cost in USD |
+| `reference_materials` | list[dict] | No | RAG chunks: `[{"id", "content", "source"}]` — enables grounding scoring |
+| `retrieval_query` | str | No | Query sent to retriever — enables retrieval quality scoring |
+| `conversation_history` | list[dict] | No | Prior turns: `[{"role", "content"}]` — enables coherence scoring |
+| `tags` | list[str] | No | Grouping labels, e.g. `["production", "rag"]` |
+| `user_id` | str | No | Your user's ID |
+| `session_id` | str | No | Conversation session ID (groups multi-turn) |
+| `metadata` | dict | No | Any extra key-value data |
+
+---
+
 ## Prompt Experimentation
 
 Variably provides two modes for LLM prompt experimentation: