dic-helper-toolkit 1.0.2__tar.gz

dic_helper_toolkit-1.0.2/PKG-INFO

@@ -0,0 +1,152 @@
+Metadata-Version: 2.4
+Name: dic-helper-toolkit
+Version: 1.0.2
+Summary: Helper libraries for data integration centers' daily tasks
+Project-URL: Repository, https://github.com/haemka/dic-helper-toolkit
+Project-URL: Issues, https://github.com/haemka/dic-helper-toolkit/issues
+Requires-Python: >=3.13
+Description-Content-Type: text/markdown
+Requires-Dist: requests~=2.34.2
+Requires-Dist: requests-oauthlib~=2.0.0
+Requires-Dist: oauthlib~=3.3.1
+Requires-Dist: fhir.resources~=8.2.0
+Requires-Dist: pytest~=9.0.3
+Requires-Dist: certifi~=2026.5.20
+Requires-Dist: charset-normalizer~=3.4.7
+Requires-Dist: pydantic~=2.13.4
+Requires-Dist: typing-extensions~=4.15.0
+Requires-Dist: typing-inspection~=0.4.2
+Requires-Dist: responses~=0.26.1
+Provides-Extra: xml
+Requires-Dist: lxml~=6.1; extra == "xml"
+Provides-Extra: tests
+Requires-Dist: pytest; extra == "tests"
+Requires-Dist: responses; extra == "tests"
+Requires-Dist: lxml~=6.1; extra == "tests"
+
+# Python DIC Helper Toolkit
+
+This Toolkit contains helper libraries for the data integration centers daily tasks.
+
+## Installation
+
+```shell
+pip install dic-helper-toolkit
+```
+
+The package is published on [PyPI](https://pypi.org/project/dic-helper-toolkit/). To use the FHIR XML reader, install
+the `xml` extra: `pip install dic-helper-toolkit[xml]`.
+
+## Contents
+
+### HTTP client
+
+A wrapper around the `requests` library which allows to create an object for an endpoint holding all needed
+configuration including authentication and certificate verification. It supports Basic Auth and OAuth 2.0.
+
+#### Usage:
+
+```python
+from dic_helpers.http_client import HTTPClient
+
+## No authentication or CA certificate at all
+client = HTTPClient(base_url="https://example.com")
+
+## Basic Auth with CA certificate
+#client = HTTPClient(base_url="https://example.com", username="user", password="P4SSW0RD", ca_file="MyCA.crt")
+
+## OAuth 2.0 with username and password
+#client = HTTPClient(base_url="https://example.com", username="user", password="P4SSW0RD",
+#                    auth_url="https://keycloak.example.com", realm="MyRealm", client_id="MyClient",
+#                    client_secret="00000000-00000000-00000000-00000000", ca_file="MyCA.crt")
+
+with client.get("index.html") as result:
+    print(result)
+```
+
+
+### FHIR client
+
+A simple FHIR client library which uses the HTTP client from above and handles resources directly as `fhir.resources`
+(https://github.com/nazrulworld/fhir.resources) object.
+
+#### Usage:
+
+```python
+from dic_helpers.fhir_client import FHIRClient
+
+client = FHIRClient(base_url="https://example.com", username="user", password="P4SSW0RD",
+                    auth_url="https://keycloak.example.com", realm="MyRealm", client_id="MyClient",
+                    client_secret="00000000-00000000-00000000-00000000", ca_file="MyCA.crt")
+
+# Fetch a patient 
+patients = client.search("Patient", "identifier=0123456789&_lastUpdated=ge2025-01-01"):
+for patient in patients:
+    print(patient)
+```
+*Keep in mind, that this returns an object.* 
+
+### Directory reader
+
+Lists the files of a directory, optionally including all subdirectories, whose names match shell-style patterns.
+
+#### Usage:
+
+```python
+from dic_helpers.directory_reader import DirectoryReader
+
+reader = DirectoryReader("/data/export", patterns=["*.json", "*.ndjson"], recursive=True)
+for path in reader.list_files():
+    print(path)
+```
+
+- **Only regular files** are listed, sorted by path, as `pathlib.Path` objects that start with the directory as given.
+- **`patterns`** (default `"*"`) are matched case-sensitively against the file names (`*`, `?`, `[seq]`). A single
+  pattern can be a string; a file is listed if it matches any pattern. Special characters in the directory's own name
+  don't matter.
+- **`recursive`** (default `False`) also lists the files of all subdirectories.
+- **`include_hidden`** (default `False`): hidden files (names starting with `.`) are skipped and hidden directories
+  are not read, unless this is `True`.
+- **`follow_symlinks`** (default `False`): symbolic links to directories are only read if this is `True`. Every
+  directory is read once at most, so symlink loops end. Symbolic links to files are always listed, broken links never.
+- **Errors:** a missing directory raises `FileNotFoundError`, a file instead of a directory `NotADirectoryError`, and
+  an unreadable directory `PermissionError`. An empty list therefore always means that no file matched.
+
+
+### FHIR file reader
+
+Reads FHIR R4B resources from files as `fhir.resources` objects, together with the file and line they were read from.
+Combined with the directory reader:
+
+```python
+from dic_helpers.directory_reader import DirectoryReader
+from dic_helpers.fhir_file_reader import FHIRFileReader
+
+paths = DirectoryReader("/data/export", patterns=["*.json", "*.ndjson", "*.xml", "*.gz"], recursive=True).list_files()
+for item in FHIRFileReader(paths).resources():
+    print(item.path, item.line, item.resource.get_resource_type(), item.resource.id)
+```
+
+**Formats**, by file extension (case-insensitive):
+
+| Extension | Content | `item.line` |
+|---|---|---|
+| `.json` | one resource | `None` |
+| `.ndjson` | one resource per line (FHIR bulk data), empty lines are skipped | line number, from 1 |
+| `.xml` | one resource in FHIR XML, the resource type comes from the root element | `None` |
+| `.json.gz`, `.ndjson.gz`, `.xml.gz` | the same, gzip compressed | as above |
+
+Other extensions are an error, the format is not guessed. Text is UTF-8, a byte order mark is allowed. Reading XML
+needs `lxml`: install `dic-helper-toolkit[xml]`. External entities are not resolved, an XML file that references one is
+invalid.
+
+**Parameters:**
+- **`paths`**: the files, read in this order; a single path can be given on its own. Resources come in the order they
+  appear in each file. NDJSON files are read line by line, also compressed ones; JSON and XML files as a whole.
+- **`unpack_bundles`** (default `True`): a Bundle gives the resources of its entries, of any Bundle type. Only the
+  outermost Bundle is unpacked, entries without a resource are left out. With `False` the Bundle is returned.
+- **`on_error`** (default `"raise"`): an unreadable or invalid file, NDJSON line or resource raises `FHIRFileError`,
+  whose message starts with the file and line, e.g. `export/Patient.ndjson:17: invalid JSON: …` or
+  `patient.json: invalid Patient: birthDate: Date value string does not match spec regex.`. With `"skip"` the message
+  is logged as a warning, and reading continues with the next line (NDJSON) or the next file. A missing `lxml` is
+  always an error.

dic_helper_toolkit-1.0.2/README.md

@@ -0,0 +1,126 @@
+# Python DIC Helper Toolkit
+
+This Toolkit contains helper libraries for the data integration centers daily tasks.
+
+## Installation
+
+```shell
+pip install dic-helper-toolkit
+```
+
+The package is published on [PyPI](https://pypi.org/project/dic-helper-toolkit/). To use the FHIR XML reader, install
+the `xml` extra: `pip install dic-helper-toolkit[xml]`.
+
+## Contents
+
+### HTTP client
+
+A wrapper around the `requests` library which allows to create an object for an endpoint holding all needed
+configuration including authentication and certificate verification. It supports Basic Auth and OAuth 2.0.
+
+#### Usage:
+
+```python
+from dic_helpers.http_client import HTTPClient
+
+## No authentication or CA certificate at all
+client = HTTPClient(base_url="https://example.com")
+
+## Basic Auth with CA certificate
+#client = HTTPClient(base_url="https://example.com", username="user", password="P4SSW0RD", ca_file="MyCA.crt")
+
+## OAuth 2.0 with username and password
+#client = HTTPClient(base_url="https://example.com", username="user", password="P4SSW0RD",
+#                    auth_url="https://keycloak.example.com", realm="MyRealm", client_id="MyClient",
+#                    client_secret="00000000-00000000-00000000-00000000", ca_file="MyCA.crt")
+
+with client.get("index.html") as result:
+    print(result)
+```
+
+
+### FHIR client
+
+A simple FHIR client library which uses the HTTP client from above and handles resources directly as `fhir.resources`
+(https://github.com/nazrulworld/fhir.resources) object.
+
+#### Usage:
+
+```python
+from dic_helpers.fhir_client import FHIRClient
+
+client = FHIRClient(base_url="https://example.com", username="user", password="P4SSW0RD",
+                    auth_url="https://keycloak.example.com", realm="MyRealm", client_id="MyClient",
+                    client_secret="00000000-00000000-00000000-00000000", ca_file="MyCA.crt")
+
+# Fetch a patient 
+patients = client.search("Patient", "identifier=0123456789&_lastUpdated=ge2025-01-01"):
+for patient in patients:
+    print(patient)
+```
+*Keep in mind, that this returns an object.* 
+
+### Directory reader
+
+Lists the files of a directory, optionally including all subdirectories, whose names match shell-style patterns.
+
+#### Usage:
+
+```python
+from dic_helpers.directory_reader import DirectoryReader
+
+reader = DirectoryReader("/data/export", patterns=["*.json", "*.ndjson"], recursive=True)
+for path in reader.list_files():
+    print(path)
+```
+
+- **Only regular files** are listed, sorted by path, as `pathlib.Path` objects that start with the directory as given.
+- **`patterns`** (default `"*"`) are matched case-sensitively against the file names (`*`, `?`, `[seq]`). A single
+  pattern can be a string; a file is listed if it matches any pattern. Special characters in the directory's own name
+  don't matter.
+- **`recursive`** (default `False`) also lists the files of all subdirectories.
+- **`include_hidden`** (default `False`): hidden files (names starting with `.`) are skipped and hidden directories
+  are not read, unless this is `True`.
+- **`follow_symlinks`** (default `False`): symbolic links to directories are only read if this is `True`. Every
+  directory is read once at most, so symlink loops end. Symbolic links to files are always listed, broken links never.
+- **Errors:** a missing directory raises `FileNotFoundError`, a file instead of a directory `NotADirectoryError`, and
+  an unreadable directory `PermissionError`. An empty list therefore always means that no file matched.
+
+
+### FHIR file reader
+
+Reads FHIR R4B resources from files as `fhir.resources` objects, together with the file and line they were read from.
+Combined with the directory reader:
+
+```python
+from dic_helpers.directory_reader import DirectoryReader
+from dic_helpers.fhir_file_reader import FHIRFileReader
+
+paths = DirectoryReader("/data/export", patterns=["*.json", "*.ndjson", "*.xml", "*.gz"], recursive=True).list_files()
+for item in FHIRFileReader(paths).resources():
+    print(item.path, item.line, item.resource.get_resource_type(), item.resource.id)
+```
+
+**Formats**, by file extension (case-insensitive):
+
+| Extension | Content | `item.line` |
+|---|---|---|
+| `.json` | one resource | `None` |
+| `.ndjson` | one resource per line (FHIR bulk data), empty lines are skipped | line number, from 1 |
+| `.xml` | one resource in FHIR XML, the resource type comes from the root element | `None` |
+| `.json.gz`, `.ndjson.gz`, `.xml.gz` | the same, gzip compressed | as above |
+
+Other extensions are an error, the format is not guessed. Text is UTF-8, a byte order mark is allowed. Reading XML
+needs `lxml`: install `dic-helper-toolkit[xml]`. External entities are not resolved, an XML file that references one is
+invalid.
+
+**Parameters:**
+- **`paths`**: the files, read in this order; a single path can be given on its own. Resources come in the order they
+  appear in each file. NDJSON files are read line by line, also compressed ones; JSON and XML files as a whole.
+- **`unpack_bundles`** (default `True`): a Bundle gives the resources of its entries, of any Bundle type. Only the
+  outermost Bundle is unpacked, entries without a resource are left out. With `False` the Bundle is returned.
+- **`on_error`** (default `"raise"`): an unreadable or invalid file, NDJSON line or resource raises `FHIRFileError`,
+  whose message starts with the file and line, e.g. `export/Patient.ndjson:17: invalid JSON: …` or
+  `patient.json: invalid Patient: birthDate: Date value string does not match spec regex.`. With `"skip"` the message
+  is logged as a warning, and reading continues with the next line (NDJSON) or the next file. A missing `lxml` is
+  always an error.

dic_helper_toolkit-1.0.2/dic_helper_toolkit.egg-info/PKG-INFO

@@ -0,0 +1,152 @@
+Metadata-Version: 2.4
+Name: dic-helper-toolkit
+Version: 1.0.2
+Summary: Helper libraries for data integration centers' daily tasks
+Project-URL: Repository, https://github.com/haemka/dic-helper-toolkit
+Project-URL: Issues, https://github.com/haemka/dic-helper-toolkit/issues
+Requires-Python: >=3.13
+Description-Content-Type: text/markdown
+Requires-Dist: requests~=2.34.2
+Requires-Dist: requests-oauthlib~=2.0.0
+Requires-Dist: oauthlib~=3.3.1
+Requires-Dist: fhir.resources~=8.2.0
+Requires-Dist: pytest~=9.0.3
+Requires-Dist: certifi~=2026.5.20
+Requires-Dist: charset-normalizer~=3.4.7
+Requires-Dist: pydantic~=2.13.4
+Requires-Dist: typing-extensions~=4.15.0
+Requires-Dist: typing-inspection~=0.4.2
+Requires-Dist: responses~=0.26.1
+Provides-Extra: xml
+Requires-Dist: lxml~=6.1; extra == "xml"
+Provides-Extra: tests
+Requires-Dist: pytest; extra == "tests"
+Requires-Dist: responses; extra == "tests"
+Requires-Dist: lxml~=6.1; extra == "tests"
+
+# Python DIC Helper Toolkit
+
+This Toolkit contains helper libraries for the data integration centers daily tasks.
+
+## Installation
+
+```shell
+pip install dic-helper-toolkit
+```
+
+The package is published on [PyPI](https://pypi.org/project/dic-helper-toolkit/). To use the FHIR XML reader, install
+the `xml` extra: `pip install dic-helper-toolkit[xml]`.
+
+## Contents
+
+### HTTP client
+
+A wrapper around the `requests` library which allows to create an object for an endpoint holding all needed
+configuration including authentication and certificate verification. It supports Basic Auth and OAuth 2.0.
+
+#### Usage:
+
+```python
+from dic_helpers.http_client import HTTPClient
+
+## No authentication or CA certificate at all
+client = HTTPClient(base_url="https://example.com")
+
+## Basic Auth with CA certificate
+#client = HTTPClient(base_url="https://example.com", username="user", password="P4SSW0RD", ca_file="MyCA.crt")
+
+## OAuth 2.0 with username and password
+#client = HTTPClient(base_url="https://example.com", username="user", password="P4SSW0RD",
+#                    auth_url="https://keycloak.example.com", realm="MyRealm", client_id="MyClient",
+#                    client_secret="00000000-00000000-00000000-00000000", ca_file="MyCA.crt")
+
+with client.get("index.html") as result:
+    print(result)
+```
+
+
+### FHIR client
+
+A simple FHIR client library which uses the HTTP client from above and handles resources directly as `fhir.resources`
+(https://github.com/nazrulworld/fhir.resources) object.
+
+#### Usage:
+
+```python
+from dic_helpers.fhir_client import FHIRClient
+
+client = FHIRClient(base_url="https://example.com", username="user", password="P4SSW0RD",
+                    auth_url="https://keycloak.example.com", realm="MyRealm", client_id="MyClient",
+                    client_secret="00000000-00000000-00000000-00000000", ca_file="MyCA.crt")
+
+# Fetch a patient 
+patients = client.search("Patient", "identifier=0123456789&_lastUpdated=ge2025-01-01"):
+for patient in patients:
+    print(patient)
+```
+*Keep in mind, that this returns an object.* 
+
+### Directory reader
+
+Lists the files of a directory, optionally including all subdirectories, whose names match shell-style patterns.
+
+#### Usage:
+
+```python
+from dic_helpers.directory_reader import DirectoryReader
+
+reader = DirectoryReader("/data/export", patterns=["*.json", "*.ndjson"], recursive=True)
+for path in reader.list_files():
+    print(path)
+```
+
+- **Only regular files** are listed, sorted by path, as `pathlib.Path` objects that start with the directory as given.
+- **`patterns`** (default `"*"`) are matched case-sensitively against the file names (`*`, `?`, `[seq]`). A single
+  pattern can be a string; a file is listed if it matches any pattern. Special characters in the directory's own name
+  don't matter.
+- **`recursive`** (default `False`) also lists the files of all subdirectories.
+- **`include_hidden`** (default `False`): hidden files (names starting with `.`) are skipped and hidden directories
+  are not read, unless this is `True`.
+- **`follow_symlinks`** (default `False`): symbolic links to directories are only read if this is `True`. Every
+  directory is read once at most, so symlink loops end. Symbolic links to files are always listed, broken links never.
+- **Errors:** a missing directory raises `FileNotFoundError`, a file instead of a directory `NotADirectoryError`, and
+  an unreadable directory `PermissionError`. An empty list therefore always means that no file matched.
+
+
+### FHIR file reader
+
+Reads FHIR R4B resources from files as `fhir.resources` objects, together with the file and line they were read from.
+Combined with the directory reader:
+
+```python
+from dic_helpers.directory_reader import DirectoryReader
+from dic_helpers.fhir_file_reader import FHIRFileReader
+
+paths = DirectoryReader("/data/export", patterns=["*.json", "*.ndjson", "*.xml", "*.gz"], recursive=True).list_files()
+for item in FHIRFileReader(paths).resources():
+    print(item.path, item.line, item.resource.get_resource_type(), item.resource.id)
+```
+
+**Formats**, by file extension (case-insensitive):
+
+| Extension | Content | `item.line` |
+|---|---|---|
+| `.json` | one resource | `None` |
+| `.ndjson` | one resource per line (FHIR bulk data), empty lines are skipped | line number, from 1 |
+| `.xml` | one resource in FHIR XML, the resource type comes from the root element | `None` |
+| `.json.gz`, `.ndjson.gz`, `.xml.gz` | the same, gzip compressed | as above |
+
+Other extensions are an error, the format is not guessed. Text is UTF-8, a byte order mark is allowed. Reading XML
+needs `lxml`: install `dic-helper-toolkit[xml]`. External entities are not resolved, an XML file that references one is
+invalid.
+
+**Parameters:**
+- **`paths`**: the files, read in this order; a single path can be given on its own. Resources come in the order they
+  appear in each file. NDJSON files are read line by line, also compressed ones; JSON and XML files as a whole.
+- **`unpack_bundles`** (default `True`): a Bundle gives the resources of its entries, of any Bundle type. Only the
+  outermost Bundle is unpacked, entries without a resource are left out. With `False` the Bundle is returned.
+- **`on_error`** (default `"raise"`): an unreadable or invalid file, NDJSON line or resource raises `FHIRFileError`,
+  whose message starts with the file and line, e.g. `export/Patient.ndjson:17: invalid JSON: …` or
+  `patient.json: invalid Patient: birthDate: Date value string does not match spec regex.`. With `"skip"` the message
+  is logged as a warning, and reading continues with the next line (NDJSON) or the next file. A missing `lxml` is
+  always an error.

dic_helper_toolkit-1.0.2/dic_helper_toolkit.egg-info/SOURCES.txt

@@ -0,0 +1,16 @@
+README.md
+pyproject.toml
+dic_helper_toolkit.egg-info/PKG-INFO
+dic_helper_toolkit.egg-info/SOURCES.txt
+dic_helper_toolkit.egg-info/dependency_links.txt
+dic_helper_toolkit.egg-info/requires.txt
+dic_helper_toolkit.egg-info/top_level.txt
+dic_helpers/__init__.py
+dic_helpers/directory_reader.py
+dic_helpers/fhir_client.py
+dic_helpers/fhir_file_reader.py
+dic_helpers/http_client.py
+tests/test_directory_reader.py
+tests/test_fhir_client.py
+tests/test_fhir_file_reader.py
+tests/test_http_client.py

dic_helper_toolkit-1.0.2/dic_helper_toolkit.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

dic_helper_toolkit-1.0.2/dic_helper_toolkit.egg-info/requires.txt

@@ -0,0 +1,19 @@
+requests~=2.34.2
+requests-oauthlib~=2.0.0
+oauthlib~=3.3.1
+fhir.resources~=8.2.0
+pytest~=9.0.3
+certifi~=2026.5.20
+charset-normalizer~=3.4.7
+pydantic~=2.13.4
+typing-extensions~=4.15.0
+typing-inspection~=0.4.2
+responses~=0.26.1
+
+[tests]
+pytest
+responses
+lxml~=6.1
+
+[xml]
+lxml~=6.1

dic_helper_toolkit-1.0.2/dic_helper_toolkit.egg-info/top_level.txt

@@ -0,0 +1 @@
+dic_helpers

dic_helper_toolkit-1.0.2/dic_helpers/directory_reader.py

@@ -0,0 +1,80 @@
+"""Listing the files of a directory, optionally including its subdirectories."""
+import fnmatch
+import os
+from pathlib import Path
+from typing import Iterable, List, Set, Tuple, Union
+
+
+class DirectoryReader:
+    """
+    Lists the files of a directory whose names match one of the given patterns.
+    """
+
+    def __init__(self, directory: Union[str, os.PathLike], patterns: Union[str, Iterable[str]] = ("*",),
+                 recursive: bool = False, include_hidden: bool = False, follow_symlinks: bool = False) -> None:
+        """
+        Initialize the DirectoryReader. The directory is only read by list_files().
+
+        :param directory: The directory to read.
+        :type directory: Union[str, os.PathLike]
+        :param patterns: Shell-style patterns (fnmatch: *, ?, [seq]) matched case-sensitively against the file names,
+            e.g. ("*.json", "*.ndjson"). A single pattern can be given as a string. A file is listed if it matches any
+            pattern.
+        :type patterns: Union[str, Iterable[str]]
+        :param recursive: Also list the files of all subdirectories.
+        :type recursive: bool
+        :param include_hidden: Also list hidden files (names starting with '.') and read hidden subdirectories.
+        :type include_hidden: bool
+        :param follow_symlinks: Also read subdirectories that are symbolic links. Every directory is read once at most,
+            so symlink loops end. Symbolic links to files are always listed.
+        :type follow_symlinks: bool
+        """
+        self.directory = Path(directory)
+        self.patterns: Tuple[str, ...] = (patterns,) if isinstance(patterns, str) else tuple(patterns)
+        self.recursive = recursive
+        self.include_hidden = include_hidden
+        self.follow_symlinks = follow_symlinks
+
+    def list_files(self) -> List[Path]:
+        """
+        List the matching files.
+
+        :return: The paths of the matching regular files, sorted. Each path starts with the directory as given.
+        :rtype: List[Path]
+        :raises FileNotFoundError: If the directory doesn't exist.
+        :raises NotADirectoryError: If the path is not a directory.
+        :raises PermissionError: If the directory or one of the subdirectories to read can't be read.
+        """
+        if not self.directory.exists():
+            raise FileNotFoundError(f"Directory not found: {self.directory}")
+        if not self.directory.is_dir():
+            raise NotADirectoryError(f"Not a directory: {self.directory}")
+        files: List[Path] = []
+        self._read(self.directory, files, visited=set())
+        return sorted(files)
+
+    def _read(self, directory: Path, files: List[Path], visited: Set[Tuple[int, int]]) -> None:
+        """
+        Add the matching files of a directory to files, and read its subdirectories if recursive.
+
+        :param directory: The directory to read.
+        :type directory: Path
+        :param files: The list the matching files are added to.
+        :type files: List[Path]
+        :param visited: Device and inode of the directories read so far; directories in it are not read again.
+        :type visited: Set[Tuple[int, int]]
+        """
+        status = directory.stat()
+        visited.add((status.st_dev, status.st_ino))
+        with os.scandir(directory) as entries:
+            for entry in entries:
+                if not self.include_hidden and entry.name.startswith("."):
+                    continue
+                # is_file() follows symbolic links: links to files are listed, broken links are not
+                if entry.is_file():
+                    if any(fnmatch.fnmatchcase(entry.name, pattern) for pattern in self.patterns):
+                        files.append(Path(entry.path))
+                elif self.recursive and entry.is_dir(follow_symlinks=self.follow_symlinks):
+                    status = entry.stat()
+                    if (status.st_dev, status.st_ino) not in visited:
+                        self._read(Path(entry.path), files, visited)