langfuse-haystack 1.1.0__tar.gz → 1.1.2__tar.gz

{langfuse_haystack-1.1.0 → langfuse_haystack-1.1.2}/CHANGELOG.md

@@ -1,11 +1,26 @@
 # Changelog
 
+## [integrations/langfuse-v1.1.1] - 2025-05-08
+
+### 🚀 Features
+
+- Add more options to LangfuseConnector (#1657)
+
+
+## [integrations/langfuse-v1.1.0] - 2025-05-06
+
+### 🚀 Features
+
+- Enhance Langfuse ToolInvoker span naming (#1682)
+
+
 ## [integrations/langfuse-v1.0.1] - 2025-04-28
 
-### Bug Fixes
+### 🐛 Bug Fixes
 
 - Langfuse - remove warning "Creating a new trace without a parent span is not recommended"
 
+
 ## [integrations/langfuse-v1.0.0] - 2025-04-11
 
 ### 🐛 Bug Fixes

{langfuse_haystack-1.1.0 → langfuse_haystack-1.1.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: langfuse-haystack
-Version: 1.1.0
+Version: 1.1.2
 Summary: Langfuse integration for Haystack
 Project-URL: Documentation, https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/langfuse#readme
 Project-URL: Issues, https://github.com/deepset-ai/haystack-core-integrations/issues
@@ -27,7 +27,10 @@ Description-Content-Type: text/markdown
 [![PyPI - Version](https://img.shields.io/pypi/v/langfuse-haystack.svg)](https://pypi.org/project/langfuse-haystack)
 [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/langfuse-haystack.svg)](https://pypi.org/project/langfuse-haystack)
 
-langfuse-haystack integrates tracing capabilities into [Haystack](https://github.com/deepset-ai/haystack) (2.x) pipelines using [Langfuse](https://langfuse.com/). This package enhances the visibility of pipeline runs by capturing comprehensive details of the execution traces, including API calls, context data, prompts, and more. Whether you're monitoring model performance, pinpointing areas for improvement, or creating datasets for fine-tuning and testing from your pipeline executions, langfuse-haystack is the right tool for you.
+langfuse-haystack integrates tracing capabilities into [Haystack](https://github.com/deepset-ai/haystack) (2.x) pipelines using [Langfuse](https://langfuse.com/). 
+This package enhances the visibility of pipeline runs by capturing comprehensive details of the execution traces, including API calls, context data, prompts, and more. 
+Whether you're monitoring model performance, pinpointing areas for improvement, or creating datasets for fine-tuning and testing from your pipeline executions, langfuse-haystack is the right tool for you.
+
 ## Features
 
 - Easy integration with Haystack pipelines
@@ -51,7 +54,8 @@ To enable tracing in your Haystack pipeline, add the `LangfuseConnector` to your
 You also need to set the `LANGFUSE_SECRET_KEY` and `LANGFUSE_PUBLIC_KEY` environment variables in order to connect to Langfuse account.
 You can get these keys by signing up for an account on the Langfuse website.
 
-⚠️ **Important:** To ensure proper tracing, always set environment variables before importing any Haystack components. This is crucial because Haystack initializes its internal tracing components during import.
+⚠️ **Important:** To ensure proper tracing, always set environment variables before importing any Haystack components. 
+This is crucial because Haystack initializes its internal tracing components during import.
 
 Here's the correct way to set up your script:
 
@@ -93,31 +97,34 @@ from haystack import Pipeline
 
 from haystack_integrations.components.connectors.langfuse import LangfuseConnector
 
-if __name__ == "__main__":
-    pipe = Pipeline()
-    pipe.add_component("tracer", LangfuseConnector("Chat example"))
-    pipe.add_component("prompt_builder", ChatPromptBuilder())
-    pipe.add_component("llm", OpenAIChatGenerator(model="gpt-3.5-turbo"))
+pipe = Pipeline()
+pipe.add_component("tracer", LangfuseConnector("Chat example"))
+pipe.add_component("prompt_builder", ChatPromptBuilder())
+pipe.add_component("llm", OpenAIChatGenerator(model="gpt-3.5-turbo"))
 
-    pipe.connect("prompt_builder.prompt", "llm.messages")
+pipe.connect("prompt_builder.prompt", "llm.messages")
 
-    messages = [
-        ChatMessage.from_system("Always respond in German even if some input data is in other languages."),
-        ChatMessage.from_user("Tell me about {{location}}"),
-    ]
+messages = [
+    ChatMessage.from_system("Always respond in German even if some input data is in other languages."),
+    ChatMessage.from_user("Tell me about {{location}}"),
+]
 
-    response = pipe.run(
-        data={"prompt_builder": {"template_variables": {"location": "Berlin"}, "template": messages}}
-    )
-    print(response["llm"]["replies"][0])
-    print(response["tracer"]["trace_url"])
+response = pipe.run(
+    data={"prompt_builder": {"template_variables": {"location": "Berlin"}, "template": messages}}
+)
+print(response["llm"]["replies"][0])
+print(response["tracer"]["trace_url"])
+print(response["tracer"]["trace_id"])
 ```
 
-In this example, we add the `LangfuseConnector` to the pipeline with the name "tracer". Each run of the pipeline produces one trace viewable on the Langfuse website with a specific URL. The trace captures the entire execution context, including the prompts, completions, and metadata.
+In this example, we add the `LangfuseConnector` to the pipeline with the name "tracer". 
+Each run of the pipeline produces one trace viewable on the Langfuse website with a specific URL. 
+The trace captures the entire execution context, including the prompts, completions, and metadata.
 
 ## Trace Visualization
 
-Langfuse provides a user-friendly interface to visualize and analyze the traces generated by your Haystack pipeline. Login into your Langfuse account and navigate to the trace URL to view the trace details.
+Langfuse provides a user-friendly interface to visualize and analyze the traces generated by your Haystack pipeline. 
+Login into your Langfuse account and navigate to the trace URL to view the trace details.
 
 ## Contributing
 

{langfuse_haystack-1.1.0 → langfuse_haystack-1.1.2}/README.md

@@ -3,7 +3,10 @@
 [![PyPI - Version](https://img.shields.io/pypi/v/langfuse-haystack.svg)](https://pypi.org/project/langfuse-haystack)
 [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/langfuse-haystack.svg)](https://pypi.org/project/langfuse-haystack)
 
-langfuse-haystack integrates tracing capabilities into [Haystack](https://github.com/deepset-ai/haystack) (2.x) pipelines using [Langfuse](https://langfuse.com/). This package enhances the visibility of pipeline runs by capturing comprehensive details of the execution traces, including API calls, context data, prompts, and more. Whether you're monitoring model performance, pinpointing areas for improvement, or creating datasets for fine-tuning and testing from your pipeline executions, langfuse-haystack is the right tool for you.
+langfuse-haystack integrates tracing capabilities into [Haystack](https://github.com/deepset-ai/haystack) (2.x) pipelines using [Langfuse](https://langfuse.com/). 
+This package enhances the visibility of pipeline runs by capturing comprehensive details of the execution traces, including API calls, context data, prompts, and more. 
+Whether you're monitoring model performance, pinpointing areas for improvement, or creating datasets for fine-tuning and testing from your pipeline executions, langfuse-haystack is the right tool for you.
+
 ## Features
 
 - Easy integration with Haystack pipelines
@@ -27,7 +30,8 @@ To enable tracing in your Haystack pipeline, add the `LangfuseConnector` to your
 You also need to set the `LANGFUSE_SECRET_KEY` and `LANGFUSE_PUBLIC_KEY` environment variables in order to connect to Langfuse account.
 You can get these keys by signing up for an account on the Langfuse website.
 
-⚠️ **Important:** To ensure proper tracing, always set environment variables before importing any Haystack components. This is crucial because Haystack initializes its internal tracing components during import.
+⚠️ **Important:** To ensure proper tracing, always set environment variables before importing any Haystack components. 
+This is crucial because Haystack initializes its internal tracing components during import.
 
 Here's the correct way to set up your script:
 
@@ -69,31 +73,34 @@ from haystack import Pipeline
 
 from haystack_integrations.components.connectors.langfuse import LangfuseConnector
 
-if __name__ == "__main__":
-    pipe = Pipeline()
-    pipe.add_component("tracer", LangfuseConnector("Chat example"))
-    pipe.add_component("prompt_builder", ChatPromptBuilder())
-    pipe.add_component("llm", OpenAIChatGenerator(model="gpt-3.5-turbo"))
+pipe = Pipeline()
+pipe.add_component("tracer", LangfuseConnector("Chat example"))
+pipe.add_component("prompt_builder", ChatPromptBuilder())
+pipe.add_component("llm", OpenAIChatGenerator(model="gpt-3.5-turbo"))
 
-    pipe.connect("prompt_builder.prompt", "llm.messages")
+pipe.connect("prompt_builder.prompt", "llm.messages")
 
-    messages = [
-        ChatMessage.from_system("Always respond in German even if some input data is in other languages."),
-        ChatMessage.from_user("Tell me about {{location}}"),
-    ]
+messages = [
+    ChatMessage.from_system("Always respond in German even if some input data is in other languages."),
+    ChatMessage.from_user("Tell me about {{location}}"),
+]
 
-    response = pipe.run(
-        data={"prompt_builder": {"template_variables": {"location": "Berlin"}, "template": messages}}
-    )
-    print(response["llm"]["replies"][0])
-    print(response["tracer"]["trace_url"])
+response = pipe.run(
+    data={"prompt_builder": {"template_variables": {"location": "Berlin"}, "template": messages}}
+)
+print(response["llm"]["replies"][0])
+print(response["tracer"]["trace_url"])
+print(response["tracer"]["trace_id"])
 ```
 
-In this example, we add the `LangfuseConnector` to the pipeline with the name "tracer". Each run of the pipeline produces one trace viewable on the Langfuse website with a specific URL. The trace captures the entire execution context, including the prompts, completions, and metadata.
+In this example, we add the `LangfuseConnector` to the pipeline with the name "tracer". 
+Each run of the pipeline produces one trace viewable on the Langfuse website with a specific URL. 
+The trace captures the entire execution context, including the prompts, completions, and metadata.
 
 ## Trace Visualization
 
-Langfuse provides a user-friendly interface to visualize and analyze the traces generated by your Haystack pipeline. Login into your Langfuse account and navigate to the trace URL to view the trace details.
+Langfuse provides a user-friendly interface to visualize and analyze the traces generated by your Haystack pipeline. 
+Login into your Langfuse account and navigate to the trace URL to view the trace details.
 
 ## Contributing
 

{langfuse_haystack-1.1.0 → langfuse_haystack-1.1.2}/src/haystack_integrations/components/connectors/langfuse/langfuse_connector.py

@@ -21,21 +21,18 @@ class LangfuseConnector:
     LangfuseConnector connects Haystack LLM framework with [Langfuse](https://langfuse.com) in order to enable the
     tracing of operations and data flow within various components of a pipeline.
 
-    Simply add this component to your pipeline, but *do not* connect it to any other component. The LangfuseConnector
-    will automatically trace the operations and data flow within the pipeline.
+    To use LangfuseConnector, add it to your pipeline without connecting it to any other components.
+    It will automatically trace all pipeline operations when tracing is enabled.
 
-    Note that you need to set the `LANGFUSE_SECRET_KEY` and `LANGFUSE_PUBLIC_KEY` environment variables in order
-    to use this component. The `LANGFUSE_SECRET_KEY` and `LANGFUSE_PUBLIC_KEY` are the secret and public keys provided
-    by Langfuse. You can get these keys by signing up for an account on the Langfuse website.
+    **Environment Configuration:**
+    - `LANGFUSE_SECRET_KEY` and `LANGFUSE_PUBLIC_KEY`: Required Langfuse API credentials.
+    - `HAYSTACK_CONTENT_TRACING_ENABLED`: Must be set to `"true"` to enable tracing.
+    - `HAYSTACK_LANGFUSE_ENFORCE_FLUSH`: (Optional) If set to `"false"`, disables flushing after each component.
+      Be cautious: this may cause data loss on crashes unless you manually flush before shutdown.
+      By default, the data is flushed after each component and blocks the thread until the data is sent to Langfuse.
 
-    In addition, you need to set the `HAYSTACK_CONTENT_TRACING_ENABLED` environment variable to `true` in order to
-    enable Haystack tracing in your pipeline.
-
-    Lastly, you may disable flushing the data after each component by setting the `HAYSTACK_LANGFUSE_ENFORCE_FLUSH`
-    environment variable to `false`. By default, the data is flushed after each component and blocks the thread until
-    the data is sent to Langfuse. **Caution**: Disabling this feature may result in data loss if the program crashes
-    before the data is sent to Langfuse. Make sure you will call langfuse.flush() explicitly before the program exits.
-    E.g. by using tracer.actual_tracer.flush():
+    If you disable flushing after each component make sure you will call langfuse.flush() explicitly before the
+    program exits. For example:
 
     ```python
     from haystack.tracing import tracer
@@ -51,13 +48,12 @@ class LangfuseConnector:
 
     # ...
 
-
     @app.on_event("shutdown")
     async def shutdown_event():
         tracer.actual_tracer.flush()
     ```
 
-    Here is an example of how to use it:
+    Here is an example of how to use LangfuseConnector in a pipeline:
 
     ```python
     import os
@@ -72,37 +68,36 @@ class LangfuseConnector:
         LangfuseConnector,
     )
 
-    if __name__ == "__main__":
-        pipe = Pipeline()
-        pipe.add_component("tracer", LangfuseConnector("Chat example"))
-        pipe.add_component("prompt_builder", ChatPromptBuilder())
-        pipe.add_component("llm", OpenAIChatGenerator(model="gpt-3.5-turbo"))
-
-        pipe.connect("prompt_builder.prompt", "llm.messages")
-
-        messages = [
-            ChatMessage.from_system(
-                "Always respond in German even if some input data is in other languages."
-            ),
-            ChatMessage.from_user("Tell me about {{location}}"),
-        ]
-
-        response = pipe.run(
-            data={
-                "prompt_builder": {
-                    "template_variables": {"location": "Berlin"},
-                    "template": messages,
-                }
+    pipe = Pipeline()
+    pipe.add_component("tracer", LangfuseConnector("Chat example"))
+    pipe.add_component("prompt_builder", ChatPromptBuilder())
+    pipe.add_component("llm", OpenAIChatGenerator(model="gpt-4o-mini"))
+
+    pipe.connect("prompt_builder.prompt", "llm.messages")
+
+    messages = [
+        ChatMessage.from_system(
+            "Always respond in German even if some input data is in other languages."
+        ),
+        ChatMessage.from_user("Tell me about {{location}}"),
+    ]
+
+    response = pipe.run(
+        data={
+            "prompt_builder": {
+                "template_variables": {"location": "Berlin"},
+                "template": messages,
             }
-        )
-        print(response["llm"]["replies"][0])
-        print(response["tracer"]["trace_url"])
-        print(response["tracer"]["trace_id"])
+        }
+    )
+    print(response["llm"]["replies"][0])
+    print(response["tracer"]["trace_url"])
+    print(response["tracer"]["trace_id"])
     ```
 
-    For advanced use cases, you can also customize how spans are created and processed by
-    providing a custom SpanHandler. This allows you to add custom metrics, set warning levels,
-    or attach additional metadata to your Langfuse traces:
+    For advanced use cases, you can also customize how spans are created and processed by providing a custom
+    SpanHandler. This allows you to add custom metrics, set warning levels, or attach additional metadata to your
+    Langfuse traces:
 
     ```python
     from haystack_integrations.tracing.langfuse import DefaultSpanHandler, LangfuseSpan
@@ -117,7 +112,6 @@ class LangfuseConnector:
 
     connector = LangfuseConnector(span_handler=CustomSpanHandler())
     ```
-
     """
 
     def __init__(
@@ -128,12 +122,15 @@ class LangfuseConnector:
         secret_key: Optional[Secret] = Secret.from_env_var("LANGFUSE_SECRET_KEY"),  # noqa: B008
         httpx_client: Optional[httpx.Client] = None,
         span_handler: Optional[SpanHandler] = None,
-    ):
+        *,
+        host: Optional[str] = None,
+        langfuse_client_kwargs: Optional[Dict[str, Any]] = None,
+    ) -> None:
         """
         Initialize the LangfuseConnector component.
 
-        :param name: The name of the pipeline or component. This name will be used to identify the tracing run on the
-            Langfuse dashboard.
+        :param name: The name for the trace. This name will be used to identify the tracing run in the Langfuse
+            dashboard.
         :param public: Whether the tracing data should be public or private. If set to `True`, the tracing data will be
             publicly accessible to anyone with the tracing URL. If set to `False`, the tracing data will be private and
             only accessible to the Langfuse account owner. The default is `False`.
@@ -146,18 +143,28 @@ class LangfuseConnector:
             The span handler controls how spans are created and processed, allowing customization of span types
             based on component types and additional processing after spans are yielded. See SpanHandler class for
             details on implementing custom handlers.
+        host: Host of Langfuse API. Can also be set via `LANGFUSE_HOST` environment variable.
+            By default it is set to `https://cloud.langfuse.com`.
+        :param langfuse_client_kwargs: Optional custom configuration for the Langfuse client. This is a dictionary
+            containing any additional configuration options for the Langfuse client. See the Langfuse documentation
+            for more details on available configuration options.
         """
         self.name = name
         self.public = public
         self.secret_key = secret_key
         self.public_key = public_key
         self.span_handler = span_handler
+        self.host = host
+        self.langfuse_client_kwargs = langfuse_client_kwargs
+        resolved_langfuse_client_kwargs = {
+            "secret_key": secret_key.resolve_value() if secret_key else None,
+            "public_key": public_key.resolve_value() if public_key else None,
+            "httpx_client": httpx_client,
+            "host": host,
+            **(langfuse_client_kwargs or {}),
+        }
         self.tracer = LangfuseTracer(
-            tracer=Langfuse(
-                secret_key=secret_key.resolve_value() if secret_key else None,
-                public_key=public_key.resolve_value() if public_key else None,
-                httpx_client=httpx_client,
-            ),
+            tracer=Langfuse(**resolved_langfuse_client_kwargs),
             name=name,
             public=public,
             span_handler=span_handler,
@@ -165,7 +172,7 @@ class LangfuseConnector:
         tracing.enable_tracing(self.tracer)
 
     @component.output_types(name=str, trace_url=str, trace_id=str)
-    def run(self, invocation_context: Optional[Dict[str, Any]] = None):
+    def run(self, invocation_context: Optional[Dict[str, Any]] = None) -> Dict[str, str]:
         """
         Runs the LangfuseConnector component.
 
@@ -191,6 +198,13 @@ class LangfuseConnector:
         :returns: The serialized component as a dictionary.
         """
         span_handler = serialize_class_instance(self.span_handler) if self.span_handler else None
+        if self.langfuse_client_kwargs:
+            # pop httpx_client and mask from self._langfuse_client_kwargs to prevent serialization issues
+            langfuse_client_kwargs = {
+                k: v for k, v in self.langfuse_client_kwargs.items() if k not in ["httpx_client", "mask"]
+            }
+        else:
+            langfuse_client_kwargs = None
         return default_to_dict(
             self,
             name=self.name,
@@ -199,6 +213,8 @@ class LangfuseConnector:
             public_key=self.public_key.to_dict() if self.public_key else None,
             # Note: httpx_client is not serialized as it's not serializable
             span_handler=span_handler,
+            host=self.host,
+            langfuse_client_kwargs=langfuse_client_kwargs,
         )
 
     @classmethod

{langfuse_haystack-1.1.0 → langfuse_haystack-1.1.2}/src/haystack_integrations/tracing/langfuse/tracer.py

@@ -35,6 +35,7 @@ _SUPPORTED_GENERATORS = [
     "HuggingFaceAPIGenerator",
     "HuggingFaceLocalGenerator",
     "CohereGenerator",
+    "OllamaGenerator",
 ]
 _SUPPORTED_CHAT_GENERATORS = [
     "AzureOpenAIChatGenerator",
@@ -198,7 +199,7 @@ class SpanHandler(ABC):
     - Pass your handler to LangfuseConnector init method
     """
 
-    def __init__(self):
+    def __init__(self) -> None:
         self.tracer: Optional[langfuse.Langfuse] = None
 
     def init_tracer(self, tracer: langfuse.Langfuse) -> None:
@@ -408,7 +409,7 @@ class LangfuseTracer(Tracer):
         if self.enforce_flush:
             self.flush()
 
-    def flush(self):
+    def flush(self) -> None:
         self._tracer.flush()
 
     def current_span(self) -> Optional[Span]:

{langfuse_haystack-1.1.0 → langfuse_haystack-1.1.2}/tests/test_langfuse_connector.py

@@ -67,6 +67,8 @@ class TestLangfuseConnector:
                     "strict": True,
                 },
                 "span_handler": None,
+                "host": None,
+                "langfuse_client_kwargs": None,
             },
         }
 
@@ -80,6 +82,8 @@ class TestLangfuseConnector:
             secret_key=Secret.from_env_var("LANGFUSE_SECRET_KEY"),
             public_key=Secret.from_env_var("LANGFUSE_PUBLIC_KEY"),
             span_handler=CustomSpanHandler(),
+            host="https://example.com",
+            langfuse_client_kwargs={"timeout": 30.0},
         )
 
         serialized = langfuse_connector.to_dict()
@@ -105,6 +109,8 @@ class TestLangfuseConnector:
                         "init_parameters": {},
                     },
                 },
+                "host": "https://example.com",
+                "langfuse_client_kwargs": {"timeout": 30.0},
             },
         }
 
@@ -128,6 +134,8 @@ class TestLangfuseConnector:
                     "strict": True,
                 },
                 "span_handler": None,
+                "host": None,
+                "langfuse_client_kwargs": None,
             },
         }
         langfuse_connector = LangfuseConnector.from_dict(data)
@@ -136,6 +144,8 @@ class TestLangfuseConnector:
         assert langfuse_connector.secret_key == Secret.from_env_var("LANGFUSE_SECRET_KEY")
         assert langfuse_connector.public_key == Secret.from_env_var("LANGFUSE_PUBLIC_KEY")
         assert langfuse_connector.span_handler is None
+        assert langfuse_connector.host is None
+        assert langfuse_connector.langfuse_client_kwargs is None
 
     def test_from_dict_with_params(self, monkeypatch):
         monkeypatch.setenv("LANGFUSE_SECRET_KEY", "secret")
@@ -163,6 +173,8 @@ class TestLangfuseConnector:
                         "init_parameters": {},
                     },
                 },
+                "host": "https://example.com",
+                "langfuse_client_kwargs": {"timeout": 30.0},
             },
         }
 
@@ -172,6 +184,8 @@ class TestLangfuseConnector:
         assert langfuse_connector.secret_key == Secret.from_env_var("LANGFUSE_SECRET_KEY")
         assert langfuse_connector.public_key == Secret.from_env_var("LANGFUSE_PUBLIC_KEY")
         assert isinstance(langfuse_connector.span_handler, CustomSpanHandler)
+        assert langfuse_connector.host == "https://example.com"
+        assert langfuse_connector.langfuse_client_kwargs == {"timeout": 30.0}
 
     def test_pipeline_serialization(self, monkeypatch):
         # Set test env vars

{langfuse_haystack-1.1.0 → langfuse_haystack-1.1.2}/tests/test_tracing.py

@@ -57,6 +57,10 @@ def basic_pipeline(llm_class, expected_trace):
     return pipe
 
 
+@pytest.mark.skipif(
+    not all([os.environ.get("LANGFUSE_SECRET_KEY"), os.environ.get("LANGFUSE_PUBLIC_KEY")]),
+    reason="Missing required environment variables: LANGFUSE_SECRET_KEY and LANGFUSE_PUBLIC_KEY",
+)
 @pytest.mark.integration
 @pytest.mark.parametrize(
     "llm_class, env_var, expected_trace",
@@ -68,7 +72,7 @@ def basic_pipeline(llm_class, expected_trace):
 )
 def test_tracing_integration(llm_class, env_var, expected_trace, basic_pipeline):
     if not all([os.environ.get("LANGFUSE_SECRET_KEY"), os.environ.get("LANGFUSE_PUBLIC_KEY"), os.environ.get(env_var)]):
-        pytest.skip(f"Missing required environment variables: LANGFUSE_SECRET_KEY, LANGFUSE_PUBLIC_KEY, or {env_var}")
+        pytest.skip(f"Missing required environment variable: {env_var}")
 
     messages = [
         ChatMessage.from_system("Always respond in German even if some input data is in other languages."),
@@ -102,14 +106,18 @@ def test_tracing_integration(llm_class, env_var, expected_trace, basic_pipeline)
     assert res_json["observations"][0]["type"] == "GENERATION"
 
 
+@pytest.mark.skipif(
+    not all(
+        [
+            os.environ.get("LANGFUSE_SECRET_KEY"),
+            os.environ.get("LANGFUSE_PUBLIC_KEY"),
+            os.environ.get("OPENAI_API_KEY"),
+        ]
+    ),
+    reason="Missing required environment variables: LANGFUSE_SECRET_KEY and LANGFUSE_PUBLIC_KEY",
+)
 @pytest.mark.integration
 def test_tracing_with_sub_pipelines():
-    if not all(
-        [os.environ.get("LANGFUSE_SECRET_KEY"), os.environ.get("LANGFUSE_PUBLIC_KEY"), os.environ.get("OPENAI_API_KEY")]
-    ):
-        pytest.skip(
-            f"Missing required environment variables: LANGFUSE_SECRET_KEY, LANGFUSE_PUBLIC_KEY, or OPENAI_API_KEY"
-        )
 
     @component
     class SubGenerator:
@@ -177,5 +185,7 @@ def test_tracing_with_sub_pipelines():
     # There should be two observations for the haystack.pipeline.run span: one for each sub pipeline
     # Main pipeline is stored under the name "Sub-pipeline example"
     assert len(haystack_pipeline_run_observations) == 2
-    assert "prompt_builder" in haystack_pipeline_run_observations[0]["input"]
-    assert "llm" in haystack_pipeline_run_observations[1]["input"]
+    # Apparently the order of haystack_pipeline_run_observations isn't deterministic
+    component_names = [key for obs in haystack_pipeline_run_observations for key in obs["input"].keys()]
+    assert "prompt_builder" in component_names
+    assert "llm" in component_names