helix.fhir.client.sdk 4.2.8__tar.gz → 4.2.9__tar.gz

{helix_fhir_client_sdk-4.2.8/helix.fhir.client.sdk.egg-info → helix_fhir_client_sdk-4.2.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: helix.fhir.client.sdk
-Version: 4.2.8
+Version: 4.2.9
 Summary: helix.fhir.client.sdk
 Home-page: https://github.com/icanbwell/helix.fhir.client.sdk
 Author: Imran Qureshi
@@ -115,4 +115,49 @@ For FHIR servers that support data streaming (e.g., b.well FHIR server), you can
 The data will be streamed in AsyncGenerators as described above.
 
 # Storage Compression
-The FHIR client SDK natively stores the FHIR resources compressed in memory.  This allows use in environments where you are processing large number of FHIR resources.
+The FHIR client SDK supports two types of compression:
+
+1. **HTTP Compression** (`compress`): Compresses HTTP request body when sending data to the server. Default: **enabled**
+2. **In-Memory Storage** (`storage_mode`): Controls how FHIR resources are stored in memory. Default: **raw (no compression)**
+
+## Disabling HTTP Compression
+HTTP compression (gzip) is enabled by default for request bodies. To disable it:
+
+```python
+from helix_fhir_client_sdk.fhir_client import FhirClient
+
+# Disable HTTP compression for requests
+fhir_client = FhirClient().url("https://fhir.example.com").compress(False)
+```
+
+## In-Memory Storage Modes
+The SDK supports different storage modes for FHIR resources through the `set_storage_mode()` method.
+By default, resources are stored as raw Python dictionaries (no compression).
+
+```python
+from helix_fhir_client_sdk.fhir_client import FhirClient
+from compressedfhir.utilities.compressed_dict.v1.compressed_dict_storage_mode import CompressedDictStorageMode
+
+# Use raw storage (default) - no compression, resources stored as plain Python dicts
+fhir_client = FhirClient().set_storage_mode(CompressedDictStorageMode(storage_type="raw"))
+
+# Use msgpack storage - stores resources in msgpack format
+fhir_client = FhirClient().set_storage_mode(CompressedDictStorageMode(storage_type="msgpack"))
+
+# Use compressed msgpack storage - stores resources in compressed msgpack format
+fhir_client = FhirClient().set_storage_mode(CompressedDictStorageMode(storage_type="compressed_msgpack"))
+```
+
+Available storage types:
+- `raw`: Default. Resources are stored as standard Python dictionaries (no compression)
+- `msgpack`: Resources are serialized using MessagePack for efficient storage
+- `compressed_msgpack`: Resources are serialized using MessagePack and then compressed
+
+## Getting Raw Python Dictionaries
+To completely bypass the `compressedfhir` library and get plain Python dictionaries:
+
+```python
+# Returns plain Python dicts, not FhirResource objects
+result = await fhir_client.get_raw_resources_async()
+resources = result["_resources"]  # list[dict[str, Any]]
+```

{helix_fhir_client_sdk-4.2.8 → helix_fhir_client_sdk-4.2.9}/README.md

@@ -79,4 +79,49 @@ For FHIR servers that support data streaming (e.g., b.well FHIR server), you can
 The data will be streamed in AsyncGenerators as described above.
 
 # Storage Compression
-The FHIR client SDK natively stores the FHIR resources compressed in memory.  This allows use in environments where you are processing large number of FHIR resources.
+The FHIR client SDK supports two types of compression:
+
+1. **HTTP Compression** (`compress`): Compresses HTTP request body when sending data to the server. Default: **enabled**
+2. **In-Memory Storage** (`storage_mode`): Controls how FHIR resources are stored in memory. Default: **raw (no compression)**
+
+## Disabling HTTP Compression
+HTTP compression (gzip) is enabled by default for request bodies. To disable it:
+
+```python
+from helix_fhir_client_sdk.fhir_client import FhirClient
+
+# Disable HTTP compression for requests
+fhir_client = FhirClient().url("https://fhir.example.com").compress(False)
+```
+
+## In-Memory Storage Modes
+The SDK supports different storage modes for FHIR resources through the `set_storage_mode()` method.
+By default, resources are stored as raw Python dictionaries (no compression).
+
+```python
+from helix_fhir_client_sdk.fhir_client import FhirClient
+from compressedfhir.utilities.compressed_dict.v1.compressed_dict_storage_mode import CompressedDictStorageMode
+
+# Use raw storage (default) - no compression, resources stored as plain Python dicts
+fhir_client = FhirClient().set_storage_mode(CompressedDictStorageMode(storage_type="raw"))
+
+# Use msgpack storage - stores resources in msgpack format
+fhir_client = FhirClient().set_storage_mode(CompressedDictStorageMode(storage_type="msgpack"))
+
+# Use compressed msgpack storage - stores resources in compressed msgpack format
+fhir_client = FhirClient().set_storage_mode(CompressedDictStorageMode(storage_type="compressed_msgpack"))
+```
+
+Available storage types:
+- `raw`: Default. Resources are stored as standard Python dictionaries (no compression)
+- `msgpack`: Resources are serialized using MessagePack for efficient storage
+- `compressed_msgpack`: Resources are serialized using MessagePack and then compressed
+
+## Getting Raw Python Dictionaries
+To completely bypass the `compressedfhir` library and get plain Python dictionaries:
+
+```python
+# Returns plain Python dicts, not FhirResource objects
+result = await fhir_client.get_raw_resources_async()
+resources = result["_resources"]  # list[dict[str, Any]]
+```

helix_fhir_client_sdk-4.2.9/VERSION

@@ -0,0 +1 @@
+4.2.9

{helix_fhir_client_sdk-4.2.8 → helix_fhir_client_sdk-4.2.9/helix.fhir.client.sdk.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: helix.fhir.client.sdk
-Version: 4.2.8
+Version: 4.2.9
 Summary: helix.fhir.client.sdk
 Home-page: https://github.com/icanbwell/helix.fhir.client.sdk
 Author: Imran Qureshi
@@ -115,4 +115,49 @@ For FHIR servers that support data streaming (e.g., b.well FHIR server), you can
 The data will be streamed in AsyncGenerators as described above.
 
 # Storage Compression
-The FHIR client SDK natively stores the FHIR resources compressed in memory.  This allows use in environments where you are processing large number of FHIR resources.
+The FHIR client SDK supports two types of compression:
+
+1. **HTTP Compression** (`compress`): Compresses HTTP request body when sending data to the server. Default: **enabled**
+2. **In-Memory Storage** (`storage_mode`): Controls how FHIR resources are stored in memory. Default: **raw (no compression)**
+
+## Disabling HTTP Compression
+HTTP compression (gzip) is enabled by default for request bodies. To disable it:
+
+```python
+from helix_fhir_client_sdk.fhir_client import FhirClient
+
+# Disable HTTP compression for requests
+fhir_client = FhirClient().url("https://fhir.example.com").compress(False)
+```
+
+## In-Memory Storage Modes
+The SDK supports different storage modes for FHIR resources through the `set_storage_mode()` method.
+By default, resources are stored as raw Python dictionaries (no compression).
+
+```python
+from helix_fhir_client_sdk.fhir_client import FhirClient
+from compressedfhir.utilities.compressed_dict.v1.compressed_dict_storage_mode import CompressedDictStorageMode
+
+# Use raw storage (default) - no compression, resources stored as plain Python dicts
+fhir_client = FhirClient().set_storage_mode(CompressedDictStorageMode(storage_type="raw"))
+
+# Use msgpack storage - stores resources in msgpack format
+fhir_client = FhirClient().set_storage_mode(CompressedDictStorageMode(storage_type="msgpack"))
+
+# Use compressed msgpack storage - stores resources in compressed msgpack format
+fhir_client = FhirClient().set_storage_mode(CompressedDictStorageMode(storage_type="compressed_msgpack"))
+```
+
+Available storage types:
+- `raw`: Default. Resources are stored as standard Python dictionaries (no compression)
+- `msgpack`: Resources are serialized using MessagePack for efficient storage
+- `compressed_msgpack`: Resources are serialized using MessagePack and then compressed
+
+## Getting Raw Python Dictionaries
+To completely bypass the `compressedfhir` library and get plain Python dictionaries:
+
+```python
+# Returns plain Python dicts, not FhirResource objects
+result = await fhir_client.get_raw_resources_async()
+resources = result["_resources"]  # list[dict[str, Any]]
+```

{helix_fhir_client_sdk-4.2.8 → helix_fhir_client_sdk-4.2.9}/helix.fhir.client.sdk.egg-info/SOURCES.txt

@@ -146,6 +146,7 @@ helix_fhir_client_sdk/validators/test/__init__.py
 helix_fhir_client_sdk/validators/test/test_async_fhir_validator.py
 tests/__init__.py
 tests/logger_for_test.py
+tests/test_fhir_client_clone.py
 tests/test_get_nested_property.py
 tests/async/__init__.py
 tests/async/test_async_fhir_client_fetch_response_in_chunks.py
@@ -163,6 +164,7 @@ tests/async/test_async_fhir_client_patient_merge_with_validate.py
 tests/async/test_async_fhir_client_patient_update.py
 tests/async/test_async_real_fhir_server_get_patients.py
 tests/async/test_async_real_fhir_server_get_patients_error.py
+tests/async/test_benchmark_compress.py
 tests/async/fhir_server/__init__.py
 tests/async/fhir_server/test_async_real_fhir_server_get_graph_large.py
 tests/async/fhir_server/test_async_real_fhir_server_get_patients_large.py

{helix_fhir_client_sdk-4.2.8 → helix_fhir_client_sdk-4.2.9}/helix_fhir_client_sdk/fhir_client.py

@@ -150,9 +150,10 @@ class FhirClient(
         self._log_all_response_urls: bool = False
         """ If True, logs all response URLs and status codes.  Can take a lot of memory for when there are many responses. """
 
-        self._storage_mode = CompressedDictStorageMode(storage_type="raw")
+        # Default to "raw" storage mode - no in-memory compression, resources stored as plain Python dicts
+        self._storage_mode: CompressedDictStorageMode = CompressedDictStorageMode(storage_type="raw")
 
-        self._create_operation_outcome_for_error = False
+        self._create_operation_outcome_for_error: bool | None = False
 
     def action(self, action: str) -> FhirClient:
         """
@@ -455,9 +456,14 @@ class FhirClient(
 
     def compress(self, compress: bool) -> FhirClient:
         """
-        Sets the compress flag
+        Sets whether to use HTTP compression (gzip) when sending request data to the server.
 
-        :param compress: whether to compress the response
+        This controls compression of the HTTP request body only, not in-memory storage.
+        Default is True (compression enabled).
+
+        To disable all compression, call: .compress(False)
+
+        :param compress: whether to compress HTTP request body (default: True)
         """
         self._compress = compress
         return self
@@ -826,17 +832,17 @@ class FhirClient(
         Whether to ask the server to include the total count in the result
 
 
-        :param include_total: whether to include total count
+        :param include_total: whether to include the total count
         """
         self._include_total = include_total
         return self
 
     def filter(self, filter_: list[BaseFilter]) -> FhirClient:
         """
-        Allows adding in a custom filters that derives from BaseFilter
+        Allows adding in custom filters that derive from BaseFilter
 
 
-        :param filter_: list of custom filter instances that derives from BaseFilter.
+        :param filter_: list of custom filter instances that derive from BaseFilter.
         """
         assert isinstance(filter_, list), "This function requires a list"
         self._filters.extend(filter_)
@@ -889,6 +895,16 @@ class FhirClient(
         fhir_client._time_to_live_in_secs_for_cache = self._time_to_live_in_secs_for_cache
         fhir_client._validation_server_url = self._validation_server_url
         fhir_client._smart_merge = self._smart_merge
+        fhir_client._compress = self._compress
+        fhir_client._storage_mode = self._storage_mode
+        fhir_client._send_data_as_chunked = self._send_data_as_chunked
+        fhir_client._use_post_for_search = self._use_post_for_search
+        fhir_client._maximum_time_to_retry_on_429 = self._maximum_time_to_retry_on_429
+        fhir_client._retry_count = self._retry_count
+        fhir_client._throw_exception_on_error = self._throw_exception_on_error
+        fhir_client._trace_request_function = self._trace_request_function
+        fhir_client._log_all_response_urls = self._log_all_response_urls
+        fhir_client._create_operation_outcome_for_error = self._create_operation_outcome_for_error
         return fhir_client
 
     def set_log_all_response_urls(self, value: bool) -> FhirClient:
@@ -911,18 +927,30 @@ class FhirClient(
 
     def set_storage_mode(self, value: CompressedDictStorageMode) -> FhirClient:
         """
-        Sets the storage mode
+        Sets the in-memory storage mode for FHIR resources.
+
+        This controls how FHIR resources are stored in memory after being received.
+        The default is "raw" (no compression - resources stored as plain Python dicts).
+
+        Available storage types:
+        - "raw": No compression, standard Python dictionaries (default)
+        - "compressed": Zlib/gzip compression in memory
+        - "msgpack": MessagePack binary serialization
+        - "compressed_msgpack": MessagePack + compression
+
+        Note: This is separate from HTTP compression (controlled by .compress()).
+        With default settings, no in-memory compression is applied.
 
-        :param value: storage mode
+        :param value: storage mode (default: raw)
         """
         self._storage_mode = value
         return self
 
-    def set_create_operation_outcome_for_error(self, value: bool) -> FhirClient:
+    def set_create_operation_outcome_for_error(self, value: bool | None) -> FhirClient:
         """
         Sets the create_operation_outcome_for_error flag
 
-        :param value: whether to create operation outcome for error
+        :param value: whether to create an operation outcome for error (True, False, or None)
         """
         self._create_operation_outcome_for_error = value
         return self

{helix_fhir_client_sdk-4.2.8 → helix_fhir_client_sdk-4.2.9}/helix_fhir_client_sdk/fhir_merge_resources_mixin.py

@@ -2,9 +2,7 @@ import json
 import time
 from collections import deque
 from collections.abc import AsyncGenerator
-from typing import (
-    cast,
-)
+from typing import Any, cast
 from urllib import parse
 
 import requests
@@ -45,6 +43,198 @@ from helix_fhir_client_sdk.validators.async_fhir_validator import AsyncFhirValid
 
 
 class FhirMergeResourcesMixin(FhirClientProtocol):
+    async def merge_bundle_uncompressed(
+        self,
+        id_: str | None,
+        bundle: FhirBundle,
+    ) -> FhirMergeResourceResponse:
+        """
+        Optimized variant of :meth:`merge_bundle_async` that bypasses storage-mode handling.
+        Use this method when you do not need storage-mode behavior, or features such as request/response compression.
+        :param id_: id of the resource to merge
+        :param bundle: FHIR Bundle to merge
+        :return: FhirMergeResourceResponse
+        """
+        # Initialize profiling dictionary
+        profiling: dict[str, float] = {
+            "total_time": 0.0,
+            "build_url": 0.0,
+            "get_access_token": 0.0,
+            "prepare_payload": 0.0,
+            "http_post": 0.0,
+            "parse_response": 0.0,
+            "create_response_objects": 0.0,
+        }
+
+        merge_start_time = time.time()
+
+        request_id: str | None = None
+        response_status: int = 500
+
+        # Build URL
+        build_url_start = time.time()
+        full_uri: furl = furl(self._url)
+        full_uri /= self._resource
+
+        # Prepare headers
+        headers = {"Content-Type": "application/fhir+json"}
+        headers.update(self._additional_request_headers)
+        profiling["build_url"] = time.time() - build_url_start
+
+        # Get access token
+        get_token_start = time.time()
+        access_token_result: GetAccessTokenResult = await self.get_access_token_async()
+        access_token: str | None = access_token_result.access_token
+        if access_token:
+            headers["Authorization"] = f"Bearer {access_token}"
+        profiling["get_access_token"] = time.time() - get_token_start
+
+        # Prepare JSON payload
+        prepare_payload_start = time.time()
+        first_resource: FhirResource | None = bundle.entry[0].resource
+        assert first_resource is not None
+        json_payload: str = first_resource.json() if len(bundle.entry) == 1 else bundle.json()
+
+        # Build merge URL
+        obj_id: str = id_ or "1"
+        resource_uri: furl = full_uri / parse.quote(str(obj_id), safe="") / "$merge"
+        profiling["prepare_payload"] = time.time() - prepare_payload_start
+
+        response_text: str | None = None
+        responses: list[dict[str, Any]] = []
+        errors: list[dict[str, Any]] = []
+
+        try:
+            async with RetryableAioHttpClient(
+                fn_get_session=self.create_http_session,
+                refresh_token_func=self._refresh_token_function,
+                tracer_request_func=self._trace_request_function,
+                retries=self._retry_count,
+                exclude_status_codes_from_retry=self._exclude_status_codes_from_retry,
+                use_data_streaming=self._use_data_streaming,
+                send_data_as_chunked=self._send_data_as_chunked,
+                compress=self._compress,
+                throw_exception_on_error=self._throw_exception_on_error,
+                log_all_url_results=self._log_all_response_urls,
+                access_token=self._access_token,
+                access_token_expiry_date=self._access_token_expiry_date,
+                persistent_session=self._persistent_session,
+                use_persistent_session=self._use_persistent_session,
+                close_session_on_exit=self._close_session,
+            ) as client:
+                http_post_start = time.time()
+                response: RetryableAioHttpResponse = await client.post(
+                    url=resource_uri.url,
+                    data=json_payload,
+                    headers=headers,
+                )
+                profiling["http_post"] = time.time() - http_post_start
+
+                response_status = response.status
+                request_id = response.response_headers.get("X-Request-ID", None)
+
+                parse_response_start = time.time()
+                if response.status == 200:
+                    response_text = await response.get_text_async()
+                    if response_text:
+                        try:
+                            # Parse response as plain dicts for speed
+                            parsed_response = json.loads(response_text)
+                            if isinstance(parsed_response, list):
+                                responses = parsed_response
+                            else:
+                                responses = [parsed_response]
+                        except (ValueError, json.JSONDecodeError) as e:
+                            errors.append(
+                                {
+                                    "issue": [
+                                        {
+                                            "severity": "error",
+                                            "code": "exception",
+                                            "diagnostics": f"Failed to parse response: {str(e)}",
+                                        }
+                                    ]
+                                }
+                            )
+                else:
+                    # HTTP error
+                    response_text = await response.get_text_async()
+                    errors.append(
+                        {
+                            "issue": [
+                                {
+                                    "severity": "error",
+                                    "code": "exception",
+                                    "diagnostics": response_text or f"HTTP {response.status}",
+                                }
+                            ]
+                        }
+                    )
+                profiling["parse_response"] = time.time() - parse_response_start
+
+        except requests.exceptions.HTTPError as e:
+            raise FhirSenderException(
+                request_id=request_id,
+                url=resource_uri.url,
+                headers=headers,
+                json_data=json_payload,
+                response_text=response_text,
+                response_status_code=response_status,
+                exception=e,
+                variables=FhirClientLogger.get_variables_to_log(vars(self)),
+                message=f"HttpError: {e}",
+                elapsed_time=time.time() - merge_start_time,
+            ) from e
+        except Exception as e:
+            raise FhirSenderException(
+                request_id=request_id,
+                url=resource_uri.url,
+                headers=headers,
+                json_data=json_payload,
+                response_text=response_text,
+                response_status_code=response_status,
+                exception=e,
+                variables=FhirClientLogger.get_variables_to_log(vars(self)),
+                message=f"Unknown Error: {e}",
+                elapsed_time=time.time() - merge_start_time,
+            ) from e
+
+        # Convert dict responses to proper objects using fast method
+        create_objects_start = time.time()
+        response_entries: deque[BaseFhirMergeResourceResponseEntry] = deque()
+
+        for resp_dict in responses:
+            response_entries.append(FhirMergeResourceResponseEntry.from_dict_uncompressed(resp_dict))
+
+        for error_dict in errors:
+            response_entries.append(FhirMergeResponseEntryError.from_dict(error_dict, storage_mode=self._storage_mode))
+        profiling["create_response_objects"] = time.time() - create_objects_start
+
+        profiling["total_time"] = time.time() - merge_start_time
+
+        # Log profiling information if logger is available
+        if self._logger:
+            self._logger.debug(
+                f"merge_bundle_without_storage profiling: "
+                f"total={profiling['total_time']:.3f}s, "
+                f"build_url={profiling['build_url']:.3f}s, "
+                f"get_token={profiling['get_access_token']:.3f}s, "
+                f"prepare_payload={profiling['prepare_payload']:.3f}s, "
+                f"http_post={profiling['http_post']:.3f}s, "
+                f"parse_response={profiling['parse_response']:.3f}s, "
+                f"create_objects={profiling['create_response_objects']:.3f}s"
+            )
+
+        return FhirMergeResourceResponse(
+            request_id=request_id,
+            url=resource_uri.url,
+            responses=response_entries,
+            error=None if response_status == 200 else (response_text or f"HTTP {response_status}"),
+            access_token=self._access_token,
+            status=response_status,
+            response_text=response_text,
+        )
+
     async def merge_bundle_async(
         self,
         id_: str | None,

{helix_fhir_client_sdk-4.2.8 → helix_fhir_client_sdk-4.2.9}/helix_fhir_client_sdk/responses/merge/fhir_merge_resource_response_entry.py

@@ -75,6 +75,36 @@ class FhirMergeResourceResponseEntry(BaseFhirMergeResourceResponseEntry):
             status=data.get("status"),
         )
 
+    @classmethod
+    def from_dict_uncompressed(cls, data: dict[str, Any]) -> "FhirMergeResourceResponseEntry":
+        """
+        Creates a FhirMergeResourceResponseEntry from a dictionary without storage_mode overhead.
+        Uses FhirResource.construct for faster object creation.
+
+        :param data: Dictionary containing the response entry data
+        :return: FhirMergeResourceResponseEntry instance
+        """
+        resource_payload = data.get("resource")
+        resource_obj: FhirResource | None = (
+            FhirResource.construct(**resource_payload) if isinstance(resource_payload, dict) else None
+        )
+        return FhirMergeResourceResponseEntry(
+            created=data.get("created"),
+            updated=data.get("updated"),
+            deleted=data.get("deleted"),
+            id_=data.get("id"),
+            uuid=data.get("uuid"),
+            resource_type=data.get("resourceType"),
+            source_assigning_authority=data.get("source_assigning_authority"),
+            resource_version=data.get("resource_version"),
+            message=data.get("message"),
+            issue=data.get("issue"),
+            error=data.get("error"),
+            token=data.get("token"),
+            resource=resource_obj,
+            status=data.get("status"),
+        )
+
     @classmethod
     @override
     def from_json(

{helix_fhir_client_sdk-4.2.8 → helix_fhir_client_sdk-4.2.9}/helix_fhir_client_sdk/utilities/retryable_aiohttp_client.py

@@ -8,7 +8,7 @@ import async_timeout
 from aiohttp import ClientError, ClientResponse, ClientResponseError, ClientSession
 from multidict import MultiMapping
 from opentelemetry import trace
-import logging
+
 from helix_fhir_client_sdk.function_types import (
     RefreshTokenFunction,
     RefreshTokenResult,
@@ -130,7 +130,6 @@ class RetryableAioHttpClient:
                     if self.compress:
                         kwargs["compress"] = self.compress
                 assert self.session is not None
-                logging.info(f"Using Session ID: {id(self.session)} for URL: {url}")
                 with TRACER.start_as_current_span(FhirClientSdkOpenTelemetrySpanNames.HTTP_GET) as span:
                     span.set_attribute(
                         FhirClientSdkOpenTelemetryAttributeNames.URL,