langwatch 0.10.0__tar.gz → 0.10.2__tar.gz

{langwatch-0.10.0 → langwatch-0.10.2}/AGENTS.md

@@ -4,7 +4,7 @@
 |-------------|------------------|
 | Writing E2E tests mixed with unit tests | Keep E2E tests isolated in `tests/e2e/` directory with `@pytest.mark.e2e`. E2E tests verify real API behavior with external services, while examples serve as user demonstrations. |
 | Repeating happy paths in integration/unit tests | Do not duplicate E2E/example coverage in integrations or units. Focus integration tests on edge/error cases, and unit tests on pure logic. |
-| Failing to follow [TESTING.md](../TESTING.md) hierarchy | Always start with a `.feature` file, drive from E2E tests in `tests/e2e/`, then add integration tests in `tests/` and unit tests as outlined. Examples in `examples/` serve as user demonstrations. |
+| Failing to follow [TESTING_PHILOSOPHY.md](../docs/TESTING_PHILOSOPHY.md) hierarchy | Always start with a `.feature` file, drive from E2E tests in `tests/e2e/`, then add integration tests in `tests/` and unit tests as outlined. Examples in `examples/` serve as user demonstrations. |
 | Using `pip` or `python` directly for installing, running, or testing | Always use [`uv`](https://github.com/astral-sh/uv) for all dependency installation, package management, and test running. Do not use `pip` commands or `python -m pip`, prefer `uv` equivalents throughout documentation, scripts, and workflows. |
 
 ## Testing Structure

{langwatch-0.10.0 → langwatch-0.10.2}/CHANGELOG.md

@@ -1,5 +1,37 @@
 # Changelog
 
+## [0.10.2](https://github.com/langwatch/langwatch/compare/python-sdk@v0.10.1...python-sdk@v0.10.2) (2026-02-05)
+
+
+### Bug Fixes
+
+* tests, handle wildcard (*) in spans and metadata trace mappings and allow log_response() without explicit target context  ([#1291](https://github.com/langwatch/langwatch/issues/1291)) ([af5d77f](https://github.com/langwatch/langwatch/commit/af5d77fade37ea9ca157965e7d1ac8e4e73f2dcf))
+
+## [1.0.0](https://github.com/langwatch/langwatch/compare/python-sdk@v0.10.0...python-sdk@v1.0.0) (2026-02-01)
+
+
+### ⚠ BREAKING CHANGES
+
+* Monitors now store evaluation level explicitly in 'level' column
+
+### Features
+
+* **evaluations-v3:** major table performance improvements, prompts to experiment button and other bugfixes ([#1181](https://github.com/langwatch/langwatch/issues/1181)) ([2cbf430](https://github.com/langwatch/langwatch/commit/2cbf4303f670edcd65a81f3af4d7a00a85b13010))
+* new online evaluations and guardrails setup ([#1151](https://github.com/langwatch/langwatch/issues/1151)) ([7c8a804](https://github.com/langwatch/langwatch/commit/7c8a804e265946a9a96e44b110f17266abaafc73))
+* **python-sdk:** add prompts_path configuration to setup() ([#1271](https://github.com/langwatch/langwatch/issues/1271)) ([49cfa7c](https://github.com/langwatch/langwatch/commit/49cfa7c30335a6fd007fbaa156f272f2419af11a))
+* **traces:** add reasoning tokens and effort support for LLM models ([16f1d4a](https://github.com/langwatch/langwatch/commit/16f1d4a80f1425d85c5f700da01287a2415bdd88))
+
+
+### Bug Fixes
+
+* **dspy:** capture full message output including reasoning_content ([8257cae](https://github.com/langwatch/langwatch/commit/8257cae1b9a4dbe8be8e7001fa88d7a0e3de6653))
+* **python-sdk:** resolve prompt path at sdk setup ([#1272](https://github.com/langwatch/langwatch/issues/1272)) ([4daf6d0](https://github.com/langwatch/langwatch/commit/4daf6d023a7ac5bd2b519c0e34173298d1569ea2))
+
+
+### Documentation
+
+* move TESTING.md to docs/TESTING_PHILOSOPHY.md ([#1157](https://github.com/langwatch/langwatch/issues/1157)) ([c475c86](https://github.com/langwatch/langwatch/commit/c475c8692f461dc737a082ca687db101a74be9fb))
+
 ## [0.10.0](https://github.com/langwatch/langwatch/compare/python-sdk@v0.9.0...python-sdk@v0.10.0) (2026-01-21)
 
 

{langwatch-0.10.0 → langwatch-0.10.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: langwatch
-Version: 0.10.0
+Version: 0.10.2
 Summary: LangWatch Python SDK, for monitoring your LLMs
 Author-email: Langwatch Engineers <engineering@langwatch.ai>
 License: MIT

{langwatch-0.10.0 → langwatch-0.10.2}/examples/README.md

@@ -163,5 +163,5 @@ When adding new examples:
 ## 🔗 Related Documentation
 
 - [Main SDK Documentation](../README.md)
-- [Testing Guidelines](../TESTING.md)
+- [Testing Guidelines](../../docs/TESTING_PHILOSOPHY.md)
 - [Agent Guidelines](../AGENTS.md)

{langwatch-0.10.0 → langwatch-0.10.2}/examples/offline_evaluation.ipynb

@@ -12,7 +12,7 @@
     },
     {
       "cell_type": "code",
-      "execution_count": null,
+      "execution_count": 1,
       "id": "e4b3c5de",
       "metadata": {},
       "outputs": [
@@ -20,7 +20,9 @@
           "name": "stdout",
           "output_type": "stream",
           "text": [
-            "LangWatch API key is already set, if you want to login again, please call as langwatch.login(relogin=True)\n"
+            "Please go to https://app.langwatch.ai/authorize to get your API key\n",
+            "2026-01-23 11:42:12,538 - langwatch.client - INFO - Registering atexit handler to flush tracer provider on exit\n",
+            "LangWatch API key set\n"
           ]
         }
       ],
@@ -32,22 +34,32 @@
     },
     {
       "cell_type": "code",
-      "execution_count": 3,
+      "execution_count": null,
       "id": "6f664279",
       "metadata": {},
       "outputs": [
         {
-          "name": "stdout",
+          "name": "stderr",
           "output_type": "stream",
           "text": [
-            "Follow the results at: http://localhost:5560/inbox-narrator/experiments/my-incredible-experiment?runId=eggplant-trout-of-painting\n"
+            "/Users/rchaves/Projects/langwatch-saas/langwatch/langwatch_nlp/.venv/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
+            "  from .autonotebook import tqdm as notebook_tqdm\n",
+            "/var/folders/rp/9_s_f3kd1ssb089myww_p9zw0000gn/T/ipykernel_50410/1400491103.py:38: DeprecationWarning: langwatch.evaluation.init() is deprecated, use langwatch.experiment.init() instead\n",
+            "  evaluation = langwatch.evaluation.init(\"my-incredible-experiment\")\n"
           ]
         },
         {
           "name": "stdout",
           "output_type": "stream",
           "text": [
-            "Evaluating: 100%|██████████| 7/7 [00:42<00:00,  6.05s/it]\n"
+            "Follow the results at: https://app.langwatch.ai/new-new-project-NmkKqM/experiments/my-incredible-experiment?runId=vague-gentle-pudu\n"
+          ]
+        },
+        {
+          "name": "stderr",
+          "output_type": "stream",
+          "text": [
+            "Evaluating: 100%|██████████| 7/7 [00:23<00:00,  3.32s/it]\n"
           ]
         }
       ],
@@ -89,7 +101,7 @@
         "    ]\n",
         ")\n",
         "\n",
-        "evaluation = langwatch.experiment.init(\"my-incredible-experiment\")\n",
+        "evaluation = langwatch.evaluation.init(\"my-incredible-experiment\")\n",
         "\n",
         "\n",
         "@langwatch.trace()\n",
@@ -102,7 +114,7 @@
         "    result = agent(row[\"question\"])  # your code\n",
         "\n",
         "    score = random.randint(0, 80) / 100 + 0.2\n",
-        "    evaluation.log(\"sample_metric\", index=index, score=score, passed=score > 0.5)"
+        "    evaluation.log(\"sample_metric\", index=index, data={\"foo\": \"bar\", \"bar\": \"baz\", \"output\": \"test\"}, score=score, passed=score > 0.5)"
       ]
     },
     {
@@ -115,7 +127,7 @@
     },
     {
       "cell_type": "code",
-      "execution_count": null,
+      "execution_count": 4,
       "id": "8fd7b230",
       "metadata": {},
       "outputs": [
@@ -123,16 +135,15 @@
           "name": "stdout",
           "output_type": "stream",
           "text": [
-            "2026-01-15 12:44:09,318 - langwatch.client - WARNING - An existing global tracer provider was found. Attaching LangWatch exporter to the existing provider. Set `ignore_global_tracer_provider_override_warning=True` to suppress this warning.\n",
-            "2026-01-15 12:44:09,320 - langwatch.client - INFO - Configuring OTLP exporter with endpoint: http://localhost:5560/api/otel/v1/traces\n",
-            "Follow the results at: http://localhost:5560/inbox-narrator/experiments/my-incredible-experiment?runId=prophetic-uakari-of-satiation\n"
+            "2026-01-23 08:19:23,104 - langwatch.client - WARNING - An existing global tracer provider was found. Attaching LangWatch exporter to the existing provider. Set `ignore_global_tracer_provider_override_warning=True` to suppress this warning.\n",
+            "Follow the results at: https://app.langwatch.ai/new-customer-project-22KFDW/experiments/my-incredible-experiment?runId=intelligent-pristine-auk\n"
           ]
         },
         {
-          "name": "stdout",
+          "name": "stderr",
           "output_type": "stream",
           "text": [
-            "Evaluating: 100%|██████████| 7/7 [00:13<00:00,  1.91s/it]\n"
+            "Evaluating: 100%|██████████| 7/7 [00:11<00:00,  1.59s/it]\n"
           ]
         }
       ],

{langwatch-0.10.0 → langwatch-0.10.2}/examples/openai_bot_rag.py

@@ -45,7 +45,7 @@ def rag_retrieval(query: str):
 async def main(message: cl.Message):
     langwatch.get_current_trace().autotrack_openai_calls(client)
     langwatch.get_current_trace().update(
-        metadata={"labels": ["openai", "rag"]},
+        metadata={"labels": ["openai", "rag"], "thread_id": "my-test-thread"},
     )
 
     msg = cl.Message(

{langwatch-0.10.0 → langwatch-0.10.2}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "langwatch"
-version = "0.10.0" # remember to also update it in src/langwatch/__version__.py
+version = "0.10.2" # remember to also update it in src/langwatch/__version__.py
 description = "LangWatch Python SDK, for monitoring your LLMs"
 authors = [{ name = "Langwatch Engineers", email = "engineering@langwatch.ai" }]
 requires-python = ">=3.10,<3.14"

langwatch-0.10.2/src/langwatch/__version__.py

@@ -0,0 +1,3 @@
+"""Version information for LangWatch."""
+
+__version__ = "0.10.2" # x-release-please-version

{langwatch-0.10.0 → langwatch-0.10.2}/src/langwatch/client.py

@@ -50,6 +50,7 @@ class Client(LangWatchClientProtocol):
     _registered_instrumentors: ClassVar[
         dict[opentelemetry.trace.TracerProvider, set[BaseInstrumentor]]
     ] = {}
+    _prompts_path: ClassVar[Optional[str]] = None
 
     # Regular attributes for protocol compatibility
     base_attributes: BaseAttributes
@@ -69,6 +70,7 @@ class Client(LangWatchClientProtocol):
         span_exclude_rules: Optional[List[SpanProcessingExcludeRule]] = None,
         ignore_global_tracer_provider_override_warning: Optional[bool] = None,
         skip_open_telemetry_setup: Optional[bool] = None,
+        prompts_path: Optional[str] = None,
     ) -> "Client":
         """Ensure only one instance of Client exists (singleton pattern)."""
         if cls._instance is None:
@@ -88,6 +90,7 @@ class Client(LangWatchClientProtocol):
         span_exclude_rules: Optional[List[SpanProcessingExcludeRule]] = None,
         ignore_global_tracer_provider_override_warning: Optional[bool] = None,
         skip_open_telemetry_setup: Optional[bool] = None,
+        prompts_path: Optional[str] = None,
     ):
         """
         Initialize the LangWatch tracing client.
@@ -140,6 +143,8 @@ class Client(LangWatchClientProtocol):
                 )
             if skip_open_telemetry_setup is not None:
                 Client._skip_open_telemetry_setup = skip_open_telemetry_setup
+            if prompts_path is not None:
+                Client._prompts_path = prompts_path
             if base_attributes is not None:
                 Client._base_attributes = base_attributes
                 # Ensure required SDK attributes remain present after reconfiguration
@@ -215,6 +220,9 @@ class Client(LangWatchClientProtocol):
         if skip_open_telemetry_setup is not None:
             Client._skip_open_telemetry_setup = skip_open_telemetry_setup
 
+        if prompts_path is not None:
+            Client._prompts_path = prompts_path
+
         if base_attributes is not None:
             Client._base_attributes = base_attributes
         elif not Client._base_attributes:
@@ -284,6 +292,7 @@ class Client(LangWatchClientProtocol):
         span_exclude_rules: Optional[List[SpanProcessingExcludeRule]] = None,
         ignore_global_tracer_provider_override_warning: Optional[bool] = None,
         skip_open_telemetry_setup: Optional[bool] = None,
+        prompts_path: Optional[str] = None,
     ) -> "Client":
         """Create or get the singleton instance of the LangWatch client. Internal use only."""
         if cls._instance is None:
@@ -299,6 +308,7 @@ class Client(LangWatchClientProtocol):
                 span_exclude_rules=span_exclude_rules,
                 ignore_global_tracer_provider_override_warning=ignore_global_tracer_provider_override_warning,
                 skip_open_telemetry_setup=skip_open_telemetry_setup,
+                prompts_path=prompts_path,
             )
         return cls._instance
 
@@ -327,6 +337,7 @@ class Client(LangWatchClientProtocol):
         cls._skip_open_telemetry_setup = False
         cls._tracer_provider = None
         cls._rest_api_client = None
+        cls._prompts_path = None
         cls._registered_instrumentors.clear()
 
     @classmethod
@@ -416,6 +427,11 @@ class Client(LangWatchClientProtocol):
         """Get whether OpenTelemetry setup is skipped."""
         return Client._skip_open_telemetry_setup
 
+    @property
+    def prompts_path(self) -> Optional[str]:
+        """Get the base path for local prompt files."""
+        return Client._prompts_path
+
     @disable_sending.setter
     def disable_sending(self, value: bool) -> None:
         """Set whether sending is disabled. Spans are still created; the exporter conditionally drops them."""

{langwatch-0.10.0 → langwatch-0.10.2}/src/langwatch/domain/__init__.py

@@ -43,6 +43,7 @@ class ChatMessage(TypedDict, total=False):
     tool_calls: Optional[List[ToolCall]]
     tool_call_id: Optional[str]
     name: Optional[str]
+    reasoning_content: Optional[str]
 
 
 class TypedValueChatMessages(TypedDict):
@@ -156,6 +157,7 @@ SpanTypes = Literal[
 class SpanMetrics(TypedDict, total=False):
     prompt_tokens: Optional[int]
     completion_tokens: Optional[int]
+    reasoning_tokens: Optional[int]
     cost: Optional[float]
     first_token_ms: Optional[int]
 
@@ -179,6 +181,7 @@ class SpanParams(TypedDict, total=False):
     functions: Optional[List[Dict[str, Any]]]
     user: Optional[str]
     response_format: Optional[Union[Dict[str, Any], BaseModel]]
+    reasoning_effort: Optional[str]
 
 
 class BaseSpan(TypedDict):

{langwatch-0.10.0 → langwatch-0.10.2}/src/langwatch/dspy/__init__.py

@@ -6,7 +6,8 @@ import warnings
 import dspy
 from typing import Callable, List, Optional, Any, Type, Union
 from langwatch.utils.exceptions import better_raise_for_status
-from langwatch.utils.transformation import truncate_object_recursively
+from langwatch.utils.transformation import SerializableWithStringFallback, truncate_object_recursively
+from langwatch.utils.utils import safe_get
 from langwatch.telemetry.tracing import LangWatchTrace
 from typing_extensions import TypedDict
 import langwatch
@@ -824,6 +825,7 @@ class DSPyTracer:
                 "functions",
                 "user",
                 "response_format",
+                "reasoning_effort",
             ]
             for param in params:
                 if all_kwargs.get(param):
@@ -842,23 +844,44 @@ class DSPyTracer:
 
             result = self.__class__.__original_call__(self, prompt, messages, **kwargs)  # type: ignore
 
+            history = self.history[-1] if len(self.history) > 0 else None
+
             if span:
-                span.update(output=result)
+                # Capture full message from history (includes reasoning_content) instead of just result
+                choices = safe_get(history, "response", "choices")
+
+                if choices and len(choices) > 0:
+                    messages_output = []
+                    for choice in choices:
+                        msg = safe_get(choice, "message")
+                        if msg is not None:
+                            # Convert Pydantic model to dict if needed
+                            if hasattr(msg, "model_dump"):
+                                msg = msg.model_dump(exclude_unset=True)
+                            elif hasattr(msg, "dict"):
+                                msg = msg.dict(exclude_unset=True)
+                            messages_output.append(msg)
+                    if messages_output:
+                        span.update(output=messages_output)
+                    else:
+                        span.update(output=result)
+                else:
+                    span.update(output=result)
 
-            history = self.history[-1] if len(self.history) > 0 else None
-            if (
-                history
-                and "usage" in history
-                and "completion_tokens" in history["usage"]
-                and "prompt_tokens" in history["usage"]
-                and span
-            ):
-                span.update(
-                    metrics={
-                        "completion_tokens": history["usage"]["completion_tokens"],
-                        "prompt_tokens": history["usage"]["prompt_tokens"],
-                    }
+            completion_tokens = safe_get(history, "usage", "completion_tokens")
+            prompt_tokens = safe_get(history, "usage", "prompt_tokens")
+            if span and completion_tokens is not None and prompt_tokens is not None:
+                metrics = {
+                    "completion_tokens": completion_tokens,
+                    "prompt_tokens": prompt_tokens,
+                }
+                # Capture reasoning_tokens if available
+                reasoning_tokens = safe_get(
+                    history, "usage", "completion_tokens_details", "reasoning_tokens"
                 )
+                if reasoning_tokens is not None:
+                    metrics["reasoning_tokens"] = reasoning_tokens
+                span.update(metrics=metrics)
 
             return result
 
@@ -884,26 +907,36 @@ class DSPyTracer:
 
             result = self.__class__.__original_basic_request__(self, prompt, **kwargs)  # type: ignore
 
-            if (
-                span
-                and "choices" in result
-                and len(result["choices"]) == 1
-                and "message" in result["choices"][0]
-            ):
-                span.update(output=[result["choices"][0]["message"]])
-
-            if (
-                span
-                and "usage" in result
-                and "completion_tokens" in result["usage"]
-                and "prompt_tokens" in result["usage"]
-            ):
-                span.update(
-                    metrics={
-                        "completion_tokens": result["usage"]["completion_tokens"],
-                        "prompt_tokens": result["usage"]["prompt_tokens"],
-                    }
+            # Capture full messages from choices (includes reasoning_content)
+            choices = safe_get(result, "choices")
+            if span and choices and len(choices) > 0:
+                messages_output = []
+                for choice in choices:
+                    msg = safe_get(choice, "message")
+                    if msg is not None:
+                        # Convert Pydantic model to dict if needed
+                        if hasattr(msg, "model_dump"):
+                            msg = msg.model_dump(exclude_unset=True)
+                        elif hasattr(msg, "dict"):
+                            msg = msg.dict(exclude_unset=True)
+                        messages_output.append(msg)
+                if messages_output:
+                    span.update(output=messages_output)
+
+            completion_tokens = safe_get(result, "usage", "completion_tokens")
+            prompt_tokens = safe_get(result, "usage", "prompt_tokens")
+            if span and completion_tokens is not None and prompt_tokens is not None:
+                metrics = {
+                    "completion_tokens": completion_tokens,
+                    "prompt_tokens": prompt_tokens,
+                }
+                # Capture reasoning_tokens if available
+                reasoning_tokens = safe_get(
+                    result, "usage", "completion_tokens_details", "reasoning_tokens"
                 )
+                if reasoning_tokens is not None:
+                    metrics["reasoning_tokens"] = reasoning_tokens
+                span.update(metrics=metrics)
 
             return result
 

{langwatch-0.10.0 → langwatch-0.10.2}/src/langwatch/experiment/experiment.py

@@ -302,6 +302,10 @@ class Experiment:
         iter_ctx = IterationContext(index=index, item=item)
         iter_token = _iteration_context.set(iter_ctx)
 
+        # Reset target context at the start of each iteration to prevent pollution
+        # from previous iterations (especially important for implicit Output targets)
+        _target_context.set(None)
+
         # Determine if we should create an iteration trace:
         # - Don't create if evaluation uses targets (each target creates its own trace)
         # - Don't create if we're collecting submit() calls (not in_thread yet)
@@ -340,6 +344,8 @@ class Experiment:
         finally:
             # Reset iteration context
             _iteration_context.reset(iter_token)
+            # Reset target context to prevent pollution to next iteration
+            _target_context.set(None)
 
         # Handle iteration trace cleanup
         # Note: If target() was used, it may have already closed the trace
@@ -715,9 +721,10 @@ class Experiment:
         """
         Log the model's response/output for the current target.
 
-        Must be called inside a `target()` context. The response will be stored
-        in the dataset entry's `predicted` field, which is displayed in the
-        results table.
+        Can be called inside a `target()` context, or outside of one. When called
+        outside a target context, an implicit "Output" target is created automatically.
+        The response will be stored in the dataset entry's `predicted` field, which
+        is displayed in the results table.
 
         Args:
             response: The model's output. Can be a string (will be wrapped as
@@ -725,30 +732,131 @@ class Experiment:
 
         Example:
             ```python
+            # With explicit target
             with evaluation.target("gpt-4", {"model": "openai/gpt-4"}):
                 response = call_gpt4(row["question"])
                 evaluation.log_response(response)  # Store the output
                 evaluation.log("quality", index=index, score=0.95)  # Log metrics
-            ```
 
-        Raises:
-            RuntimeError: If called outside of a target() context.
+            # Without explicit target (creates implicit "Output" target)
+            for index, row in evaluation.loop(df.iterrows()):
+                response = my_model(row["question"])
+                evaluation.log_response(response)  # Creates "Output" target
+                evaluation.log("quality", index=index, score=0.95)
+            ```
         """
         ctx = _target_context.get()
-        if ctx is None:
-            raise RuntimeError(
-                "log_response() must be called inside a target() context. "
-                "Example: with evaluation.target('my-target'): evaluation.log_response(response)"
-            )
 
         # Normalize response to dict format
         if isinstance(response, str):
-            ctx.predicted = {"output": response}
+            predicted = {"output": response}
         elif isinstance(response, dict):
-            ctx.predicted = response
+            predicted = response
         else:
             # Try to convert to string for other types
-            ctx.predicted = {"output": str(response)}
+            predicted = {"output": str(response)}
+
+        if ctx is None:
+            # Create implicit "Output" target and dataset entry immediately
+            self._create_implicit_output_target(predicted)
+        else:
+            # Inside explicit target context - just set predicted
+            ctx.predicted = predicted
+
+    def _create_implicit_output_target(self, predicted: Dict[str, Any]) -> None:
+        """
+        Create an implicit "Output" target when log_response() is called outside
+        a target() context. This enables a simpler API for single-target evaluations.
+
+        Creates the dataset entry immediately with the predicted response.
+        """
+        target_name = "Output"
+
+        # Mark that targets are being used
+        if not self._evaluation_uses_targets:
+            self._evaluation_uses_targets = True
+            # Close the active iteration trace if any
+            if self._active_iteration_trace is not None:
+                self._active_iteration_trace.__exit__(None, None, None)
+                self._active_iteration_trace = None
+
+        self._current_iteration_used_with_target = True
+
+        # Register the target
+        self._register_target(target_name, None)
+
+        # Get index and item from iteration context
+        iter_ctx = _iteration_context.get()
+        if iter_ctx is not None:
+            index = iter_ctx.index
+            current_item = iter_ctx.item
+        else:
+            index = self._current_index
+            current_item = self._current_item
+
+        # Create a trace for this implicit target
+        tracer = trace.get_tracer("langwatch-evaluation")
+        root_context = otel_context.Context()
+
+        # Start span and get trace_id
+        with tracer.start_span(
+            f"evaluation.target.{target_name}",
+            context=root_context,
+            attributes={
+                "evaluation.run_id": self.run_id,
+                "evaluation.index": index,
+                "evaluation.target": target_name,
+            },
+        ) as span:
+            span_context = span.get_span_context()
+            trace_id = format(span_context.trace_id, "032x")
+
+        # Create and set target context (for subsequent log() calls)
+        ctx = TargetContext(
+            target_id=target_name,
+            index=index,
+            trace_id=trace_id,
+            predicted=predicted,
+        )
+        _target_context.set(ctx)
+
+        # Create dataset entry immediately
+        entry_data: Any = (
+            current_item.to_dict()
+            if hasattr(current_item, "to_dict")
+            else (
+                current_item.__dict__
+                if hasattr(current_item, "__dict__")
+                else (
+                    current_item[1].to_dict()
+                    if type(current_item) == tuple
+                    and hasattr(current_item[1], "to_dict")
+                    else (
+                        current_item[1].__dict__
+                        if type(current_item) == tuple
+                        and hasattr(current_item[1], "__dict__")
+                        else {
+                            "entry": json.dumps(
+                                current_item, cls=SerializableWithStringFallback
+                            )
+                        }
+                    )
+                )
+            )
+        )
+
+        batch_entry = BatchEntry(
+            index=index,
+            entry=entry_data,
+            duration=0,  # Duration not tracked for implicit targets
+            error=None,
+            trace_id=trace_id,
+            target_id=target_name,
+            predicted=predicted,
+        )
+
+        with self.lock:
+            self.batch["dataset"].append(batch_entry)
 
     def log(
         self,

{langwatch-0.10.0 → langwatch-0.10.2}/src/langwatch/litellm.py

@@ -246,6 +246,9 @@ class LiteLLMPatch:
                 SpanMetrics(
                     prompt_tokens=safe_get(usage, "prompt_tokens"),
                     completion_tokens=safe_get(usage, "completion_tokens"),
+                    reasoning_tokens=safe_get(
+                        usage, "completion_tokens_details", "reasoning_tokens"
+                    ),
                 )
                 if usage
                 else SpanMetrics()
@@ -281,6 +284,9 @@ class LiteLLMPatch:
             metrics=SpanMetrics(
                 prompt_tokens=safe_get(response, "usage", "prompt_tokens"),
                 completion_tokens=safe_get(response, "usage", "completion_tokens"),
+                reasoning_tokens=safe_get(
+                    response, "usage", "completion_tokens_details", "reasoning_tokens"
+                ),
             ),
             timestamps=timestamps,
             **kwargs,
@@ -338,6 +344,7 @@ class LiteLLMPatch:
             "functions",
             "user",
             "response_format",
+            "reasoning_effort",
         ]
         for param in params:
             if kwargs.get(param):