pathling 9.7.1__tar.gz → 9.8.0.dev0__tar.gz

pathling-9.7.1/README.md → pathling-9.8.0.dev0/PKG-INFO

@@ -1,5 +1,27 @@
-Python API for Pathling
-=======================
+Metadata-Version: 2.4
+Name: pathling
+Version: 9.8.0.dev0
+Summary: Python API for Pathling
+Project-URL: Homepage, https://github.com/aehrc/pathling
+Author-email: "Australian e-Health Research Centre, CSIRO" <pathling@csiro.au>
+License: Apache License, version 2.0
+License-File: LICENSE
+Keywords: analytics,fhir,pathling,spark,standards,terminology
+Classifier: Development Status :: 3 - Alpha
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Requires-Python: >=3.9
+Requires-Dist: click>=8.1.7
+Requires-Dist: deprecated>=1.2.13
+Requires-Dist: ipython>=8.18.1
+Requires-Dist: pyspark<4.1.0,>=4.0.0
+Requires-Dist: rich>=13.7.0
+Requires-Dist: tomli>=2.0.1; python_version < '3.11'
+Description-Content-Type: text/markdown
+
+# Python API for Pathling
 
 This is the Python API for [Pathling](https://pathling.csiro.au). It provides a
 set of tools that aid the use of FHIR terminology services and FHIR data within
@@ -16,9 +38,49 @@ Prerequisites:
 To install, run this command:
 
 ```bash
-pip install pathling  
+pip install pathling
 ```
 
+## Command line interface
+
+The package ships a `pathling` console script that surfaces the library's
+functionality - data conversion, SQL on FHIR views, FHIRPath evaluation, bulk
+export, terminology operations, Python scripting (`run`), and an interactive
+console (`console`) - through scriptable commands. It requires a supported
+Java runtime, as does the library itself.
+
+The easiest way to run it is with [uv](https://docs.astral.sh/uv/):
+
+```bash
+# Run without installing.
+uvx pathling --version
+
+# Or install it as a tool.
+uv tool install pathling
+pathling --help
+```
+
+For example, to convert a directory of NDJSON to Parquet, or run a SQL on FHIR
+view:
+
+```bash
+pathling convert data/ --to parquet -o warehouse/
+pathling view data/ --view patients.json --format csv
+```
+
+The `run` and `console` commands execute Python code (a script, stdin, or
+`-c CODE`) or open an interactive IPython session, with `spark` and `pathling`
+variables already configured and in scope:
+
+```bash
+pathling run my_script.py
+pathling run -c "print(spark.version)"
+pathling console
+```
+
+See the [command line interface documentation](https://pathling.csiro.au/docs/libraries/cli)
+for the full list of commands and options.
+
 ## Encoders
 
 The Python library features a set of encoders for converting FHIR data into
@@ -109,7 +171,7 @@ display(result)
 The result of this query would look something like this:
 
 | patient_id | street                     | use  | city       | zip   |
-|------------|----------------------------|------|------------|-------|
+| ---------- | -------------------------- | ---- | ---------- | ----- |
 | 1          | 398 Kautzer Walk Suite 62  | home | Barnstable | 02675 |
 | 1          | 186 Nitzsche Forge         | work | Revere     | 02151 |
 | 2          | 1087 Quitzon Club          | home | Plymouth   | NULL  |
@@ -152,7 +214,7 @@ result.select('CODE', 'DESCRIPTION', 'VIRAL_INFECTION').show()
 Results in:
 
 | CODE      | DESCRIPTION               | VIRAL_INFECTION |
-|-----------|---------------------------|-----------------|
+| --------- | ------------------------- | --------------- |
 | 65363002  | Otitis media              | false           |
 | 16114001  | Fracture of ankle         | false           |
 | 444814009 | Viral sinusitis           | true            |
@@ -177,7 +239,7 @@ result.select('CODE', 'DESCRIPTION', 'READ_CODE').show()
 Results in:
 
 | CODE      | DESCRIPTION               | READ_CODE |
-|-----------|---------------------------|-----------|
+| --------- | ------------------------- | --------- |
 | 65363002  | Otitis media              | X00ik     |
 | 16114001  | Fracture of ankle         | S34..     |
 | 444814009 | Viral sinusitis           | XUjp0     |
@@ -213,7 +275,7 @@ result.select('CODE', 'DESCRIPTION', 'IS_ENT').show()
 Results in:
 
 | CODE      | DESCRIPTION       | IS_ENT |
-|-----------|-------------------|--------|
+| --------- | ----------------- | ------ |
 | 65363002  | Otitis media      | true   |
 | 16114001  | Fracture of ankle | false  |
 | 444814009 | Viral sinusitis   | true   |
@@ -245,7 +307,7 @@ with_displays = exploded_parents.withColumn(
 Results in:
 
 | CODE      | DESCRIPTION       | PARENT    | PARENT_DISPLAY                          |
-|-----------|-------------------|-----------|-----------------------------------------|
+| --------- | ----------------- | --------- | --------------------------------------- |
 | 65363002  | Otitis media      | 43275000  | Otitis                                  |
 | 65363002  | Otitis media      | 68996008  | Disorder of middle ear                  |
 | 16114001  | Fracture of ankle | 125603006 | Injury of ankle                         |
@@ -279,7 +341,7 @@ exploded_synonyms = synonyms.selectExpr(
 Results in:
 
 | CODE      | DESCRIPTION                          | SYNONYM                                    |
-|-----------|--------------------------------------|--------------------------------------------|
+| --------- | ------------------------------------ | ------------------------------------------ |
 | 65363002  | Otitis media                         | OM - Otitis media                          |
 | 16114001  | Fracture of ankle                    | Ankle fracture                             |
 | 16114001  | Fracture of ankle                    | Fracture of distal end of tibia and fibula |
@@ -322,8 +384,8 @@ Maven package. Once the cluster is restarted, the libraries should be available
 for import and use within all notebooks.
 
 By default, Databricks uses Java 8 within its clusters, while Pathling requires
-Java 21. To enable Java 21 support within your cluster, navigate to __Advanced
-Options > Spark > Environment Variables__ and add the following:
+Java 21. To enable Java 21 support within your cluster, navigate to **Advanced
+Options > Spark > Environment Variables** and add the following:
 
 ```bash
 JNAME=zulu21-ca-amd64

pathling-9.7.1/PKG-INFO → pathling-9.8.0.dev0/README.md

@@ -1,24 +1,4 @@
-Metadata-Version: 2.4
-Name: pathling
-Version: 9.7.1
-Summary: Python API for Pathling
-Project-URL: Homepage, https://github.com/aehrc/pathling
-Author-email: "Australian e-Health Research Centre, CSIRO" <pathling@csiro.au>
-License: Apache License, version 2.0
-License-File: LICENSE
-Keywords: analytics,fhir,pathling,spark,standards,terminology
-Classifier: Development Status :: 3 - Alpha
-Classifier: License :: OSI Approved :: Apache Software License
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Requires-Python: >=3.9
-Requires-Dist: deprecated>=1.2.13
-Requires-Dist: pyspark<4.1.0,>=4.0.0
-Description-Content-Type: text/markdown
-
-Python API for Pathling
-=======================
+# Python API for Pathling
 
 This is the Python API for [Pathling](https://pathling.csiro.au). It provides a
 set of tools that aid the use of FHIR terminology services and FHIR data within
@@ -35,9 +15,49 @@ Prerequisites:
 To install, run this command:
 
 ```bash
-pip install pathling  
+pip install pathling
 ```
 
+## Command line interface
+
+The package ships a `pathling` console script that surfaces the library's
+functionality - data conversion, SQL on FHIR views, FHIRPath evaluation, bulk
+export, terminology operations, Python scripting (`run`), and an interactive
+console (`console`) - through scriptable commands. It requires a supported
+Java runtime, as does the library itself.
+
+The easiest way to run it is with [uv](https://docs.astral.sh/uv/):
+
+```bash
+# Run without installing.
+uvx pathling --version
+
+# Or install it as a tool.
+uv tool install pathling
+pathling --help
+```
+
+For example, to convert a directory of NDJSON to Parquet, or run a SQL on FHIR
+view:
+
+```bash
+pathling convert data/ --to parquet -o warehouse/
+pathling view data/ --view patients.json --format csv
+```
+
+The `run` and `console` commands execute Python code (a script, stdin, or
+`-c CODE`) or open an interactive IPython session, with `spark` and `pathling`
+variables already configured and in scope:
+
+```bash
+pathling run my_script.py
+pathling run -c "print(spark.version)"
+pathling console
+```
+
+See the [command line interface documentation](https://pathling.csiro.au/docs/libraries/cli)
+for the full list of commands and options.
+
 ## Encoders
 
 The Python library features a set of encoders for converting FHIR data into
@@ -128,7 +148,7 @@ display(result)
 The result of this query would look something like this:
 
 | patient_id | street                     | use  | city       | zip   |
-|------------|----------------------------|------|------------|-------|
+| ---------- | -------------------------- | ---- | ---------- | ----- |
 | 1          | 398 Kautzer Walk Suite 62  | home | Barnstable | 02675 |
 | 1          | 186 Nitzsche Forge         | work | Revere     | 02151 |
 | 2          | 1087 Quitzon Club          | home | Plymouth   | NULL  |
@@ -171,7 +191,7 @@ result.select('CODE', 'DESCRIPTION', 'VIRAL_INFECTION').show()
 Results in:
 
 | CODE      | DESCRIPTION               | VIRAL_INFECTION |
-|-----------|---------------------------|-----------------|
+| --------- | ------------------------- | --------------- |
 | 65363002  | Otitis media              | false           |
 | 16114001  | Fracture of ankle         | false           |
 | 444814009 | Viral sinusitis           | true            |
@@ -196,7 +216,7 @@ result.select('CODE', 'DESCRIPTION', 'READ_CODE').show()
 Results in:
 
 | CODE      | DESCRIPTION               | READ_CODE |
-|-----------|---------------------------|-----------|
+| --------- | ------------------------- | --------- |
 | 65363002  | Otitis media              | X00ik     |
 | 16114001  | Fracture of ankle         | S34..     |
 | 444814009 | Viral sinusitis           | XUjp0     |
@@ -232,7 +252,7 @@ result.select('CODE', 'DESCRIPTION', 'IS_ENT').show()
 Results in:
 
 | CODE      | DESCRIPTION       | IS_ENT |
-|-----------|-------------------|--------|
+| --------- | ----------------- | ------ |
 | 65363002  | Otitis media      | true   |
 | 16114001  | Fracture of ankle | false  |
 | 444814009 | Viral sinusitis   | true   |
@@ -264,7 +284,7 @@ with_displays = exploded_parents.withColumn(
 Results in:
 
 | CODE      | DESCRIPTION       | PARENT    | PARENT_DISPLAY                          |
-|-----------|-------------------|-----------|-----------------------------------------|
+| --------- | ----------------- | --------- | --------------------------------------- |
 | 65363002  | Otitis media      | 43275000  | Otitis                                  |
 | 65363002  | Otitis media      | 68996008  | Disorder of middle ear                  |
 | 16114001  | Fracture of ankle | 125603006 | Injury of ankle                         |
@@ -298,7 +318,7 @@ exploded_synonyms = synonyms.selectExpr(
 Results in:
 
 | CODE      | DESCRIPTION                          | SYNONYM                                    |
-|-----------|--------------------------------------|--------------------------------------------|
+| --------- | ------------------------------------ | ------------------------------------------ |
 | 65363002  | Otitis media                         | OM - Otitis media                          |
 | 16114001  | Fracture of ankle                    | Ankle fracture                             |
 | 16114001  | Fracture of ankle                    | Fracture of distal end of tibia and fibula |
@@ -341,8 +361,8 @@ Maven package. Once the cluster is restarted, the libraries should be available
 for import and use within all notebooks.
 
 By default, Databricks uses Java 8 within its clusters, while Pathling requires
-Java 21. To enable Java 21 support within your cluster, navigate to __Advanced
-Options > Spark > Environment Variables__ and add the following:
+Java 21. To enable Java 21 support within your cluster, navigate to **Advanced
+Options > Spark > Environment Variables** and add the following:
 
 ```bash
 JNAME=zulu21-ca-amd64

pathling-9.8.0.dev0/pathling/__init__.py

@@ -0,0 +1,127 @@
+#
+# Copyright © 2018-2026 Commonwealth Scientific and Industrial Research
+# Organisation (CSIRO) ABN 41 687 119 230.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""Python API for Pathling.
+
+Public names are exposed lazily (PEP 562) so that importing the ``pathling``
+package itself does not pull in PySpark and the JVM-backed submodules. The
+heavy imports happen only when a public name is first accessed, which keeps the
+command line interface's ``--help`` and ``--version`` paths fast.
+
+Author: John Grimes.
+"""
+
+# The TYPE_CHECKING imports below exist only so that type checkers and IDEs can
+# resolve the lazily-exported public names; they are intentionally unused at
+# runtime, so unused-import checks are disabled for this re-export shim.
+# ruff: noqa: F401
+
+import importlib
+from typing import TYPE_CHECKING, Any
+
+if TYPE_CHECKING:
+    from .coding import Coding
+    from .context import PathlingContext, StorageType
+    from .core import Expression, VariableExpression
+    from .datasource import DataSource, DataSources
+    from .fhir import MimeType, Version
+    from .functions import to_coding, to_ecl_value_set, to_snomed_coding
+    from .udfs import (
+        Equivalence,
+        PropertyType,
+        designation,
+        display,
+        member_of,
+        property_of,
+        subsumed_by,
+        subsumes,
+        translate,
+    )
+
+# Maps each lazily-exported public name to the submodule that defines it.
+_LAZY_EXPORTS = {
+    "Coding": "pathling.coding",
+    "PathlingContext": "pathling.context",
+    "StorageType": "pathling.context",
+    "Expression": "pathling.core",
+    "VariableExpression": "pathling.core",
+    "DataSource": "pathling.datasource",
+    "DataSources": "pathling.datasource",
+    "MimeType": "pathling.fhir",
+    "Version": "pathling.fhir",
+    "to_coding": "pathling.functions",
+    "to_snomed_coding": "pathling.functions",
+    "to_ecl_value_set": "pathling.functions",
+    "member_of": "pathling.udfs",
+    "translate": "pathling.udfs",
+    "subsumes": "pathling.udfs",
+    "subsumed_by": "pathling.udfs",
+    "property_of": "pathling.udfs",
+    "display": "pathling.udfs",
+    "designation": "pathling.udfs",
+    "PropertyType": "pathling.udfs",
+    "Equivalence": "pathling.udfs",
+}
+
+__all__ = list(_LAZY_EXPORTS)
+
+# The package submodules that may be accessed lazily after a bare
+# ``import pathling`` (e.g. ``pathling.udfs.member_of``). This curated allow-list
+# keeps the fallback safe - unknown names still raise ``AttributeError`` - and
+# restores the pre-shim behaviour where these submodules were importable as
+# package attributes, without importing PySpark on a bare ``import pathling``.
+_LAZY_SUBMODULES = (
+    "coding",
+    "context",
+    "core",
+    "datasource",
+    "fhir",
+    "functions",
+    "udfs",
+)
+
+
+def __getattr__(name: str) -> Any:
+    """Resolves a public name or submodule by importing it on first access.
+
+    :param name: the attribute being accessed on the ``pathling`` package.
+    :return: the resolved attribute value.
+    :raises AttributeError: if the name is neither a known public export nor a
+            known package submodule.
+    """
+    module_name = _LAZY_EXPORTS.get(name)
+    if module_name is not None:
+        module = importlib.import_module(module_name)
+        value = getattr(module, name)
+        # Cache on the package so subsequent lookups bypass this hook.
+        globals()[name] = value
+        return value
+    if name in _LAZY_SUBMODULES:
+        submodule = importlib.import_module(f"{__name__}.{name}")
+        # Cache the submodule so subsequent lookups bypass this hook.
+        globals()[name] = submodule
+        return submodule
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+def __dir__() -> list:
+    """Returns the public names and submodules of the package for introspection.
+
+    :return: the sorted list of public export names and lazily-available
+             submodule names.
+    """
+    return sorted(set(__all__) | set(_LAZY_SUBMODULES))

pathling-9.8.0.dev0/pathling/_spark_defaults.py

@@ -0,0 +1,81 @@
+#
+# Copyright © 2018-2026 Commonwealth Scientific and Industrial Research
+# Organisation (CSIRO) ABN 41 687 119 230.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""The single, PySpark-free source of truth for Pathling's managed Spark defaults.
+
+The managed coordinates, the Delta SQL extension, and the Delta catalog that
+every Pathling Spark session requires are defined here, built from the versions
+in :mod:`pathling._version`. Both the session builder
+(:func:`pathling.context._build_spark_session`) and the CLI merge logic
+(:mod:`pathling.cli.sparkconf`) import these values so the two cannot drift
+apart. This module deliberately imports no PySpark, so the CLI configuration
+path can reference the defaults without paying Spark's import cost.
+
+Author: John Grimes.
+"""
+
+from pathling._version import (
+    __delta_version__,
+    __java_version__,
+    __scala_version__,
+)
+
+# The Spark configuration key holding the comma-separated Maven coordinates.
+PACKAGES_KEY = "spark.jars.packages"
+
+# The Spark configuration key holding the comma-separated SQL extension classes.
+EXTENSIONS_KEY = "spark.sql.extensions"
+
+# The Spark configuration key for the session catalog implementation.
+CATALOG_KEY = "spark.sql.catalog.spark_catalog"
+
+# The managed Maven coordinate (group:artifact) for the Pathling library runtime.
+LIBRARY_RUNTIME_COORDINATE = "au.csiro.pathling:library-runtime"
+
+# The managed Maven coordinate (group:artifact) for Delta Lake, which carries the
+# Scala binary version in its artifact identifier.
+DELTA_COORDINATE = f"io.delta:delta-spark_{__scala_version__}"
+
+# The group:artifact identities of the coordinates Pathling manages, used to
+# detect a user-supplied override at a different version.
+MANAGED_COORDINATES = frozenset({LIBRARY_RUNTIME_COORDINATE, DELTA_COORDINATE})
+
+# The Delta SQL extension class that Pathling always requires.
+DELTA_EXTENSION = "io.delta.sql.DeltaSparkSessionExtension"
+
+# The Delta catalog implementation that Pathling always requires.
+DELTA_CATALOG = "org.apache.spark.sql.delta.catalog.DeltaCatalog"
+
+
+def managed_spark_defaults() -> dict:
+    """Returns the Spark configuration that Pathling always requires.
+
+    The packages string lists both managed coordinates at the versions declared
+    in :mod:`pathling._version` and retains a trailing comma, matching the
+    historical inline literal. The extension and catalog are fixed Delta class
+    names.
+
+    :return: a mapping of managed Spark configuration key to value.
+    """
+    return {
+        PACKAGES_KEY: (
+            f"{LIBRARY_RUNTIME_COORDINATE}:{__java_version__},"
+            f"{DELTA_COORDINATE}:{__delta_version__},"
+        ),
+        EXTENSIONS_KEY: DELTA_EXTENSION,
+        CATALOG_KEY: DELTA_CATALOG,
+    }

{pathling-9.7.1 → pathling-9.8.0.dev0}/pathling/_version.py

@@ -2,8 +2,8 @@
 # Auto generated from POM project version.
 # Please do not modify.
 #
-__version__="9.7.1"
-__java_version__="9.7.1"
+__version__="9.8.0.dev0"
+__java_version__="9.8.0-SNAPSHOT"
 __scala_version__="2.13"
 __delta_version__="4.0.0"
 __hadoop_version__="3.4.1"

pathling-9.8.0.dev0/pathling/cli/__init__.py

@@ -0,0 +1,27 @@
+#
+# Copyright © 2018-2026 Commonwealth Scientific and Industrial Research
+# Organisation (CSIRO) ABN 41 687 119 230.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""Command line interface for Pathling.
+
+This subpackage exposes the Pathling Python library through a flat, verb-based
+command tree installed as the ``pathling`` console script. Modules are kept
+free of eager PySpark imports so that ``--help`` and ``--version`` remain fast;
+the Spark session is created lazily by :mod:`pathling.cli.session` only when a
+command needs it.
+
+Author: John Grimes.
+"""