ol-llm 0.0.1__tar.gz

ol_llm-0.0.1/.gitignore

@@ -0,0 +1,10 @@
+.venv/
+__pycache__/
+*.pyc
+dist/
+build/
+*.egg-info/
+.mypy_cache/
+.pytest_cache/
+.ruff_cache/
+.env

ol_llm-0.0.1/LICENSE

@@ -0,0 +1,29 @@
+BSD 3-Clause License
+
+Copyright (c) 2026, MIT Open Learning
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice,
+  this list of conditions and the following disclaimer in the documentation
+  and/or other materials provided with the distribution.
+
+* Neither the name of the copyright holder nor the names of its
+  contributors may be used to endorse or promote products derived from
+  this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

ol_llm-0.0.1/PKG-INFO

@@ -0,0 +1,150 @@
+Metadata-Version: 2.5
+Name: ol-llm
+Version: 0.0.1
+Summary: Shared library for LLM calls at MIT Open Learning, with Opik tracing, prompt management, and evaluation built in
+Project-URL: Homepage, https://github.com/mitodl/ol-llm
+Project-URL: Repository, https://github.com/mitodl/ol-llm
+Project-URL: Issues, https://github.com/mitodl/ol-llm/issues
+Author-email: MIT Open Learning <odl-devops@mit.edu>
+License-Expression: BSD-3-Clause
+License-File: LICENSE
+Keywords: llm,observability,opik,pydantic-ai
+Classifier: Development Status :: 2 - Pre-Alpha
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Requires-Dist: httpx>=0.28
+Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.38
+Requires-Dist: opentelemetry-sdk>=1.38
+Requires-Dist: opik==2.2.87
+Provides-Extra: anthropic
+Requires-Dist: pydantic-ai-slim[anthropic]<3,>=2.52; extra == 'anthropic'
+Provides-Extra: bedrock
+Requires-Dist: pydantic-ai-slim[bedrock]<3,>=2.52; extra == 'bedrock'
+Provides-Extra: google
+Requires-Dist: pydantic-ai-slim[google]<3,>=2.52; extra == 'google'
+Provides-Extra: harness
+Requires-Dist: pydantic-ai-harness<0.53,>=0.52; extra == 'harness'
+Provides-Extra: memory
+Requires-Dist: hindsight-client<0.11,>=0.10; extra == 'memory'
+Provides-Extra: mistral
+Requires-Dist: pydantic-ai-slim[mistral]<3,>=2.52; extra == 'mistral'
+Provides-Extra: openai
+Requires-Dist: pydantic-ai-slim[openai]<3,>=2.52; extra == 'openai'
+Provides-Extra: pydantic-ai
+Requires-Dist: pydantic-ai-slim<3,>=2.52; extra == 'pydantic-ai'
+Description-Content-Type: text/markdown
+
+# ol-llm
+
+The library every LLM call site at MIT Open Learning goes through. It owns the Opik
+integration (tracing, prompt management, evaluation), a single timeout and retry policy, and
+the attribution fields that let cost, quality, and traffic join across applications.
+
+Importing `ol_llm` sets `OPIK_SENTRY_ENABLE=false` before anything imports the Opik SDK, which
+otherwise replaces the host application's Sentry client with its own, so import it first.
+
+The design is in [mitodl/ol-infrastructure `docs/plans/shared-llm-library-spec.md`](https://github.com/mitodl/ol-infrastructure/blob/main/docs/plans/shared-llm-library-spec.md).
+The rules the code follows are in [`docs/`](https://github.com/mitodl/ol-llm/tree/main/docs):
+the [error contract](https://github.com/mitodl/ol-llm/blob/main/docs/error-contract.md) and the
+[attribution schema](https://github.com/mitodl/ol-llm/blob/main/docs/attribution.md).
+
+## Usage
+
+Declare each call site once, as a module-level constant, and build its agent from it:
+
+```python
+from pydantic import BaseModel
+
+import ol_llm
+
+SUMMARY = ol_llm.UseCase(
+    app="mit-learn",
+    name="content-summary",
+    profile="batch",
+    model="openai:gpt-4o-mini",
+    fallback_models=("bedrock:global.anthropic.claude-haiku-4-5-20251001-v1:0",),
+)
+
+
+class Summary(BaseModel):
+    text: str
+
+
+summarizer = ol_llm.build_agent(SUMMARY, output_type=Summary, instructions="Summarize.")
+```
+
+The agent's model has a timeout on every request, retries rate limits and outages and
+nothing else, falls back once the primary has spent its retries, and raises `LLMError`
+subclasses (`LLMRateLimited`, `LLMUnavailable`, `LLMRejected`, ...) with the provider's
+exception as the cause. Build agents once, not per request: each build creates new provider
+clients.
+
+Set `OL_LLM_ENVIRONMENT`, or call `ol_llm.configure(ol_llm.Settings(environment=...))` at
+startup. Provider credentials are each SDK's own (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, the AWS
+and Google default credential chains, and so on). Operators can change a use case's limits
+without a release, e.g. `OL_LLM_CONTENT_SUMMARY_TIMEOUT_SECONDS=120`.
+
+Call sites not built on PydanticAI (LangGraph, a plain SDK) apply the same numbers with
+`ol_llm.resolve_policy(use_case)`, which needs no extra.
+
+The entry points that wrap a run in an Opik trace (`run`, `run_stream`, `complete`) are not
+released yet; until they are, `build_agent` gives the policy and errors without the tracing.
+
+### Testing
+
+`ol_llm.testing.override_models` makes `build_model` and `build_agent` use PydanticAI's
+`TestModel` or `FunctionModel` in place of a provider, still under the library's retry and
+error handling, and `tracing_disabled` turns Opik off. Script responses and failures with
+`FunctionModel`; do not commit recorded provider responses.
+
+## Extras
+
+The base install is the Opik SDK and OpenTelemetry, which every call site needs. Everything
+else is opt-in:
+
+| Extra | Installs | For |
+| --- | --- | --- |
+| `pydantic-ai` | `pydantic-ai-slim` | The default construction path for agents and single calls |
+| `openai`, `anthropic`, `bedrock`, `google`, `mistral` | the matching `pydantic-ai-slim` extra | A provider SDK; each also installs PydanticAI |
+| `harness` | `pydantic-ai-harness` | In-session memory, compaction, and conversation search. Each harness release pins PydanticAI exactly |
+| `memory` | `hindsight-client` | The shared memory service's client. Provisional: the backend is still being evaluated |
+
+For example, `ol-llm[bedrock,harness]`.
+
+## Installing without the Opik SDK's heavy dependencies
+
+The `opik` package hard-requires litellm, openai, pytest, and several other packages the
+tracing and prompt paths never import. uv consumers can drop them:
+
+```toml
+[tool.uv]
+override-dependencies = [
+    "boto3-stubs; sys_platform == 'never'",
+    "click; sys_platform == 'never'",
+    "litellm; sys_platform == 'never'",
+    "openai; sys_platform == 'never'",
+    "pytest; sys_platform == 'never'",
+    "rapidfuzz; sys_platform == 'never'",
+    "tree-sitter; sys_platform == 'never'",
+    "tree-sitter-javascript; sys_platform == 'never'",
+    "tree-sitter-typescript; sys_platform == 'never'",
+    "watchfiles; sys_platform == 'never'",
+]
+```
+
+Drop `openai` from that list if the application uses the `openai` extra. Opik's evaluation
+metrics need litellm, so the overrides belong in runtime images, not in an environment that
+runs evaluations. CI checks this exact list on every change (`python/smoke/`).
+
+Applications that cap how new a dependency may be (`exclude-newer` in uv) should exempt this
+package so releases are usable the day they ship:
+
+```toml
+[tool.uv.exclude-newer-package]
+ol-llm = "0d"
+```

ol_llm-0.0.1/README.md

@@ -0,0 +1,109 @@
+# ol-llm
+
+The library every LLM call site at MIT Open Learning goes through. It owns the Opik
+integration (tracing, prompt management, evaluation), a single timeout and retry policy, and
+the attribution fields that let cost, quality, and traffic join across applications.
+
+Importing `ol_llm` sets `OPIK_SENTRY_ENABLE=false` before anything imports the Opik SDK, which
+otherwise replaces the host application's Sentry client with its own, so import it first.
+
+The design is in [mitodl/ol-infrastructure `docs/plans/shared-llm-library-spec.md`](https://github.com/mitodl/ol-infrastructure/blob/main/docs/plans/shared-llm-library-spec.md).
+The rules the code follows are in [`docs/`](https://github.com/mitodl/ol-llm/tree/main/docs):
+the [error contract](https://github.com/mitodl/ol-llm/blob/main/docs/error-contract.md) and the
+[attribution schema](https://github.com/mitodl/ol-llm/blob/main/docs/attribution.md).
+
+## Usage
+
+Declare each call site once, as a module-level constant, and build its agent from it:
+
+```python
+from pydantic import BaseModel
+
+import ol_llm
+
+SUMMARY = ol_llm.UseCase(
+    app="mit-learn",
+    name="content-summary",
+    profile="batch",
+    model="openai:gpt-4o-mini",
+    fallback_models=("bedrock:global.anthropic.claude-haiku-4-5-20251001-v1:0",),
+)
+
+
+class Summary(BaseModel):
+    text: str
+
+
+summarizer = ol_llm.build_agent(SUMMARY, output_type=Summary, instructions="Summarize.")
+```
+
+The agent's model has a timeout on every request, retries rate limits and outages and
+nothing else, falls back once the primary has spent its retries, and raises `LLMError`
+subclasses (`LLMRateLimited`, `LLMUnavailable`, `LLMRejected`, ...) with the provider's
+exception as the cause. Build agents once, not per request: each build creates new provider
+clients.
+
+Set `OL_LLM_ENVIRONMENT`, or call `ol_llm.configure(ol_llm.Settings(environment=...))` at
+startup. Provider credentials are each SDK's own (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, the AWS
+and Google default credential chains, and so on). Operators can change a use case's limits
+without a release, e.g. `OL_LLM_CONTENT_SUMMARY_TIMEOUT_SECONDS=120`.
+
+Call sites not built on PydanticAI (LangGraph, a plain SDK) apply the same numbers with
+`ol_llm.resolve_policy(use_case)`, which needs no extra.
+
+The entry points that wrap a run in an Opik trace (`run`, `run_stream`, `complete`) are not
+released yet; until they are, `build_agent` gives the policy and errors without the tracing.
+
+### Testing
+
+`ol_llm.testing.override_models` makes `build_model` and `build_agent` use PydanticAI's
+`TestModel` or `FunctionModel` in place of a provider, still under the library's retry and
+error handling, and `tracing_disabled` turns Opik off. Script responses and failures with
+`FunctionModel`; do not commit recorded provider responses.
+
+## Extras
+
+The base install is the Opik SDK and OpenTelemetry, which every call site needs. Everything
+else is opt-in:
+
+| Extra | Installs | For |
+| --- | --- | --- |
+| `pydantic-ai` | `pydantic-ai-slim` | The default construction path for agents and single calls |
+| `openai`, `anthropic`, `bedrock`, `google`, `mistral` | the matching `pydantic-ai-slim` extra | A provider SDK; each also installs PydanticAI |
+| `harness` | `pydantic-ai-harness` | In-session memory, compaction, and conversation search. Each harness release pins PydanticAI exactly |
+| `memory` | `hindsight-client` | The shared memory service's client. Provisional: the backend is still being evaluated |
+
+For example, `ol-llm[bedrock,harness]`.
+
+## Installing without the Opik SDK's heavy dependencies
+
+The `opik` package hard-requires litellm, openai, pytest, and several other packages the
+tracing and prompt paths never import. uv consumers can drop them:
+
+```toml
+[tool.uv]
+override-dependencies = [
+    "boto3-stubs; sys_platform == 'never'",
+    "click; sys_platform == 'never'",
+    "litellm; sys_platform == 'never'",
+    "openai; sys_platform == 'never'",
+    "pytest; sys_platform == 'never'",
+    "rapidfuzz; sys_platform == 'never'",
+    "tree-sitter; sys_platform == 'never'",
+    "tree-sitter-javascript; sys_platform == 'never'",
+    "tree-sitter-typescript; sys_platform == 'never'",
+    "watchfiles; sys_platform == 'never'",
+]
+```
+
+Drop `openai` from that list if the application uses the `openai` extra. Opik's evaluation
+metrics need litellm, so the overrides belong in runtime images, not in an environment that
+runs evaluations. CI checks this exact list on every change (`python/smoke/`).
+
+Applications that cap how new a dependency may be (`exclude-newer` in uv) should exempt this
+package so releases are usable the day they ship:
+
+```toml
+[tool.uv.exclude-newer-package]
+ol-llm = "0d"
+```

ol_llm-0.0.1/pyproject.toml

@@ -0,0 +1,63 @@
+[project]
+name = "ol-llm"
+version = "0.0.1"
+description = "Shared library for LLM calls at MIT Open Learning, with Opik tracing, prompt management, and evaluation built in"
+readme = "README.md"
+license = "BSD-3-Clause"
+license-files = ["LICENSE"]
+authors = [{ name = "MIT Open Learning", email = "odl-devops@mit.edu" }]
+requires-python = ">=3.11"
+keywords = ["llm", "opik", "pydantic-ai", "observability"]
+classifiers = [
+    "Development Status :: 2 - Pre-Alpha",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3.14",
+    "Typing :: Typed",
+]
+# The base install is what every call site needs whatever it is built on: the
+# Opik SDK for prompts and the root trace, and OpenTelemetry for the spans a
+# framework emits. Everything framework- or backend-specific is an extra.
+#
+# opik is pinned exactly: consumers slim it with uv override-dependencies, and a
+# release that starts importing a dropped package eagerly would break them at
+# import time. The smoke project in python/smoke/ catches that on every bump.
+dependencies = [
+    "httpx>=0.28",
+    "opentelemetry-exporter-otlp-proto-http>=1.38",
+    "opentelemetry-sdk>=1.38",
+    "opik==2.2.87",
+]
+
+[project.optional-dependencies]
+# The default construction path for agents and single calls.
+pydantic-ai = ["pydantic-ai-slim>=2.52,<3"]
+# Provider SDKs, each through PydanticAI's own extra, so a provider extra also
+# installs PydanticAI.
+anthropic = ["pydantic-ai-slim[anthropic]>=2.52,<3"]
+bedrock = ["pydantic-ai-slim[bedrock]>=2.52,<3"]
+google = ["pydantic-ai-slim[google]>=2.52,<3"]
+mistral = ["pydantic-ai-slim[mistral]>=2.52,<3"]
+openai = ["pydantic-ai-slim[openai]>=2.52,<3"]
+# PydanticAI's first-party capabilities: in-session memory, compaction, and
+# conversation search. Each harness release pins pydantic-ai-slim exactly, so
+# this extra also fixes the PydanticAI version. 0.x: pinned to a minor series.
+harness = ["pydantic-ai-harness>=0.52,<0.53"]
+# Client for the shared memory service (cross-session recall and user-scoped
+# facts). Hindsight is the leading candidate and its PoC is still open, so this
+# extra may change backend before the memory interface ships.
+memory = ["hindsight-client>=0.10,<0.11"]
+
+[project.urls]
+Homepage = "https://github.com/mitodl/ol-llm"
+Repository = "https://github.com/mitodl/ol-llm"
+Issues = "https://github.com/mitodl/ol-llm/issues"
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/ol_llm"]

ol_llm-0.0.1/smoke/pyproject.toml

@@ -0,0 +1,28 @@
+# A consumer-shaped project: ol-llm installed with the overrides a uv consumer
+# copies to drop the opik SDK's heavy dependencies. CI runs smoke_test.py in this
+# environment on every change, so an opik release that starts importing a dropped
+# package eagerly fails here before any app picks it up.
+[project]
+name = "ol-llm-smoke"
+version = "0"
+requires-python = ">=3.11"
+# The pydantic-ai extra, because the smoke test runs an instrumented agent.
+dependencies = ["ol-llm[pydantic-ai]"]
+
+[tool.uv]
+package = false
+override-dependencies = [
+    "boto3-stubs; sys_platform == 'never'",
+    "click; sys_platform == 'never'",
+    "litellm; sys_platform == 'never'",
+    "openai; sys_platform == 'never'",
+    "pytest; sys_platform == 'never'",
+    "rapidfuzz; sys_platform == 'never'",
+    "tree-sitter; sys_platform == 'never'",
+    "tree-sitter-javascript; sys_platform == 'never'",
+    "tree-sitter-typescript; sys_platform == 'never'",
+    "watchfiles; sys_platform == 'never'",
+]
+
+[tool.uv.sources]
+ol-llm = { path = "..", editable = true }

ol_llm-0.0.1/smoke/smoke_test.py

@@ -0,0 +1,91 @@
+"""Import-and-trace check for ol-llm installed with the opik SDK slimmed.
+
+Runs as a plain script, not under pytest: pytest is one of the packages the
+overrides drop. Exits non-zero on any failure.
+
+What it proves, in an environment shaped like a consumer's runtime image:
+- ol_llm and opik import with litellm, openai, pytest, and tree-sitter absent;
+- an @opik.track root with an instrumented agent from ol_llm.build_agent inside it
+  produces OTel spans that OpikSpanProcessor attaches to the tracked trace;
+- importing ol_llm first leaves no Sentry client running.
+
+No Opik server is needed. The SDK is pointed at a closed port, so its background
+sender fails quietly; the assertions read the spans from an in-memory exporter.
+"""
+
+import os
+import sys
+
+os.environ["OPIK_URL_OVERRIDE"] = "http://127.0.0.1:9/api"
+os.environ["OPIK_WORKSPACE"] = "default"
+
+# ol_llm before opik, so OPIK_SENTRY_ENABLE is set before opik reads it.
+import ol_llm
+import opik
+import sentry_sdk
+from ol_llm.models import build_agent, override_models
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import SimpleSpanProcessor
+from opentelemetry.sdk.trace.export.in_memory_span_exporter import (
+    InMemorySpanExporter,
+)
+from opik.integrations.otel import OpikSpanProcessor
+from pydantic_ai.capabilities import Instrumentation
+from pydantic_ai.models.instrumented import InstrumentationSettings
+from pydantic_ai.models.test import TestModel
+
+DROPPED = ("litellm", "openai", "pytest", "tree_sitter", "click")
+OPIK_TRACE_ID = "opik.trace_id"
+SMOKE = ol_llm.UseCase(
+    app="ol-llm-smoke", name="smoke", profile="request", model="openai:smoke"
+)
+
+
+def main() -> int:
+    """Run the checks and return a process exit status."""
+    exporter = InMemorySpanExporter()
+    provider = TracerProvider()
+    provider.add_span_processor(OpikSpanProcessor())
+    provider.add_span_processor(SimpleSpanProcessor(exporter))
+
+    ol_llm.configure(ol_llm.Settings(environment="ci"))
+    # The test model stands in for the provider; the policy wrapper, the agent's
+    # capabilities, and the import graph are the library's own.
+    with override_models(TestModel()):
+        agent = build_agent(
+            SMOKE,
+            capabilities=[
+                Instrumentation(
+                    settings=InstrumentationSettings(tracer_provider=provider)
+                )
+            ],
+        )
+
+    @opik.track(name="smoke", project_name="ol-llm-smoke")
+    def run() -> str:
+        return agent.run_sync("ping").output
+
+    output = run()
+    spans = exporter.get_finished_spans()
+    attached = [s for s in spans if (s.attributes or {}).get(OPIK_TRACE_ID)]
+    imported = [name for name in DROPPED if name in sys.modules]
+    failures = []
+    if not spans:
+        failures.append("the agent emitted no OTel spans")
+    if len(attached) != len(spans):
+        failures.append(
+            f"{len(spans) - len(attached)} of {len(spans)} spans lack {OPIK_TRACE_ID}"
+        )
+    if imported:
+        failures.append(f"dropped packages were imported: {imported}")
+    if sentry_sdk.get_client().dsn is not None:
+        failures.append("a Sentry client is running after importing opik")
+
+    print(f"ol-llm {ol_llm.__version__}: output={output!r} spans={len(spans)}")
+    for failure in failures:
+        print(f"FAIL: {failure}")
+    return 1 if failures else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())