hdx-python-utilities 4.0.7__tar.gz → 4.1.0__tar.gz

{hdx_python_utilities-4.0.7 → hdx_python_utilities-4.1.0}/.pre-commit-config.yaml

@@ -1,5 +1,5 @@
 default_language_version:
-  python: python3.13
+  python: python3
 
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks

hdx_python_utilities-4.1.0/CLAUDE.md

@@ -0,0 +1,81 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+**hdx-python-utilities** is a general-purpose Python utility library (not HDX-specific) providing helpers for: downloading files, retrieving tabular data (CSV/JSON/XLSX), date parsing, JSON/YAML I/O, dict/list manipulation, HTML parsing, file hashing, text processing, error handling, and more.
+
+## Commands
+
+```bash
+# Install dependencies
+uv sync
+
+# Run all tests with coverage
+uv run pytest
+
+# Run a single test file
+uv run pytest tests/hdx/utilities/test_downloader.py
+
+# Run a single test by name
+uv run pytest tests/hdx/utilities/test_downloader.py -k "test_download_file"
+
+# Lint and format
+uv run ruff check
+pre-commit run --all-files
+
+# Build
+uv build
+```
+
+Coverage is written to `coverage.lcov` and JUnit XML to `test-results.xml`.
+
+## Source Layout
+
+All source lives under `src/hdx/utilities/`. Tests mirror the layout under `tests/hdx/utilities/` with fixtures in `tests/fixtures/`.
+
+Key modules:
+- `base_downloader.py` — `BaseDownload` ABC defining the interface (`download_file`, `download_tabular_rows`, etc.) and `DownloadError`
+- `downloader.py` — `Download(BaseDownload)`: HTTP downloading via `requests` + `frictionless` for tabular parsing; supports auth (basic, bearer, extra params), rate limiting, retries, and streaming rows
+- `retriever.py` — `Retrieve(BaseDownload)`: wraps a `Download`, adding save/load-from-disk and static fallback directories; registered instances in `Retrieve.retrievers` dict
+- `session.py` — `get_session()` builds a `requests.Session` with retry logic and auth headers
+- `frictionless_wrapper.py` — thin wrapper around `frictionless.TableResource` for tabular reads
+- `loader.py` / `saver.py` — JSON and YAML I/O (YAML uses `ruamel.yaml` to preserve order/comments)
+- `dateparse.py` — date/time parsing and timezone utilities
+- `dictandlist.py` — dict/list helpers (merge, flatten, key renaming, etc.)
+- `path.py` — `script_dir()`, temp dir helpers, `NotFoundError`
+- `error_handler.py` / `errors_onexit.py` — structured error accumulation and on-exit reporting
+- `text.py` — text processing including `smart_split`
+- `matching.py` — fuzzy/phonetic matching utilities
+- `state.py` — lightweight on-disk state persistence
+- `useragent.py` — user-agent string construction (reads `~/.useragent.yaml` by default)
+
+## Key Patterns
+
+**Download / Retrieve layering** — `Download` handles raw HTTP; `Retrieve` adds a save/use_saved/fallback layer on top. Callers typically use `Retrieve` rather than `Download` directly. Both implement `BaseDownload` so they're interchangeable.
+
+**Tabular row iteration** — `download_tabular_rows()` (on both `Download` and `Retrieve`) yields dicts via `frictionless.TableResource`. The `frictionless_wrapper.py` module handles the `FrictionlessException` boundary. Supports CSV, JSON, XLSX, and XLS.
+
+**User-agent requirement** — `Download` requires a user agent. Either set a global user agent via `UserAgent.set_global(...)` or pass `user_agent=` directly. Missing user agent raises at construction time.
+
+**Context manager usage** — `Download` and `Retrieve` are designed as context managers (`with Download(...) as downloader:`). `__exit__` closes the session.
+
+**Auth precedence** — environment variables `EXTRA_PARAMS`, `BASIC_AUTH`, `BEARER_TOKEN` override constructor arguments. Among constructor args: `auth` > `basic_auth`/`basic_auth_file` > `bearer_token`/`bearer_token_file` > `extra_params_dict` > `extra_params_json`/`extra_params_yaml`.
+
+## Code Style
+
+- Python ≥ 3.10; type hints throughout using `X | Y` union syntax (PEP 604)
+- Google-style docstrings with `Args:` and `Returns:` sections
+- Formatted and linted with `ruff` (rules: E, F, I, UP; E501 ignored)
+- No inline comments unless the *why* is non-obvious
+
+## Collaboration Style
+
+- Be objective, not agreeable. Push back when you disagree, flag tradeoffs honestly.
+- Keep explanations brief and to the point.
+- Don't rely on recalled knowledge for facts that could be stale. Read the actual source first.
+
+## Scope of Changes
+
+When fixing a bug or addressing PR feedback, change only what is necessary to resolve the specific issue. Do not refactor surrounding code, rename variables, adjust formatting, or make improvements in the same commit unless they are directly required by the fix.

{hdx_python_utilities-4.0.7 → hdx_python_utilities-4.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hdx-python-utilities
-Version: 4.0.7
+Version: 4.1.0
 Summary: HDX Python Utilities for streaming tabular data, date and time handling and other helpful functions
 Project-URL: Homepage, https://github.com/OCHA-DAP/hdx-python-utilities
 Author-email: Michael Rans <rans@email.com>
@@ -25,8 +25,8 @@ Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Requires-Python: >=3.10
-Requires-Dist: frictionless>=5.18.0
-Requires-Dist: ijson>=3.2.3
+Requires-Dist: frictionless>=5.19.0
+Requires-Dist: ijson>=3.5.0
 Requires-Dist: jsonlines>=4.0.0
 Requires-Dist: loguru
 Requires-Dist: openpyxl>=3.1.2
@@ -38,7 +38,7 @@ Requires-Dist: ruamel-yaml
 Requires-Dist: tableschema-to-template>=0.0.13
 Requires-Dist: typing-extensions
 Requires-Dist: xlrd>=2.0.1
-Requires-Dist: xlsx2csv
+Requires-Dist: xlsx2csv==0.8.4
 Requires-Dist: xlwt>=1.3.0
 Provides-Extra: diff
 Requires-Dist: cydifflib; extra == 'diff'
@@ -63,7 +63,6 @@ Note that these are not specific to HDX.
 1. Retrieval of data from url with saving to file or from data previously saved
 1. Date utilities
 1. Loading and saving JSON and YAML (maintaining order)
-1. Loading and saving HXLated csv and/or JSON
 1. Dictionary and list utilities
 1. HTML utilities (inc. BeautifulSoup helper)
 1. Compare files (eg. for testing)

{hdx_python_utilities-4.0.7 → hdx_python_utilities-4.1.0}/README.md

@@ -10,7 +10,6 @@ Note that these are not specific to HDX.
 1. Retrieval of data from url with saving to file or from data previously saved
 1. Date utilities
 1. Loading and saving JSON and YAML (maintaining order)
-1. Loading and saving HXLated csv and/or JSON
 1. Dictionary and list utilities
 1. HTML utilities (inc. BeautifulSoup helper)
 1. Compare files (eg. for testing)

{hdx_python_utilities-4.0.7 → hdx_python_utilities-4.1.0}/documentation/index.md

@@ -9,7 +9,6 @@ Python developers. Note that these are not specific to HDX.
 1. [Retrieval of data from url with saving to file or from data previously saved](#retrieving-files)
 1. [Date utilities](#date-utilities)
 1. [Loading and saving JSON and YAML (maintaining order)](#loading-and-saving-json-and-yaml)
-1. [Loading and saving HXLated csv and/or JSON](#loading-and-saving-hxlated-csv-andor-json)
 1. [Dictionary and list utilities](#dictionary-and-list-utilities)
 1. [HTML utilities (inc. BeautifulSoup helper)](#html-utilities)
 1. [Compare files (eg. for testing)](#comparing-files)
@@ -34,6 +33,9 @@ The code for the library is [here](https://github.com/OCHA-DAP/hdx-python-utilit
 The library has detailed API documentation which can be found in the menu at the top.
 
 ## Breaking Changes
+From 4.1.0, save_hxlated_output and Download.hxl_row removed. Parameter use_hxl removed
+from all calls in Download and Retrieve.
+
 From 4.0.4, get_filename_from_url, get_filename_extension_from_url, get_path_for_url,
 get_url_for_get, get_url_params_for_post moved to module url
 
@@ -185,9 +187,6 @@ Other useful functions:
     headers, iterator = downloader.get_tabular_rows_as_list(url)
     for row in iterator:
         ...
-    # Get hxl row
-    assert Download.hxl_row(["a", "b", "c"], {"b": "#b", "c": "#c"}, dict_form=True)
-    # == {"a": "", "b": "#b", "c": "#c"}
     # Build get url from url and dictionary of parameters
     Download.get_url_for_get("http://www.lala.com/hdfa?a=3&b=4",
                              OrderedDict([("c", "e"), ("d", "f")]))
@@ -461,93 +460,6 @@ as a csv or Excel.
 
     save_iterable(xlfilepath, list_of_lists, headers=["h1", "h2", "h3", "h4"], format="xlsx")
 
-## Loading and saving HXLated csv and/or JSON
-
-`save_hxlated_output` is a utility to save HXLated output (currently JSON and/or csv are
-supported) based on a given configuration. Here is an example YAML configuration:
-
-    input:
-      headers:
-        - "Col1"
-        - "Col2"
-        - "Col3"
-      hxltags:
-        - "#tag1"
-        - "#tag2"
-        - "#tag3"
-    process:
-      - header: "tag4"
-        hxltag: "#tag4"
-        expression: "#tag1 * 10"
-    output:
-      csv:
-        filename: "out.csv"
-        hxltags:
-          - "#tag2"
-          - "#tag3"
-      json:
-        filename: "out.json"
-        data: "results"
-        metadata:
-          "#date": "{{today}}"
-          "#mytag": 123
-        hxltags:
-          - "#tag1"
-          - "#tag2"
-
-The `input` section is needed if the rows of data that are passed in are missing either
-headers or HXL hashtags. The `output` section defines what files will be created. If
-`hxltags` are specified, then only those columns are output. CSV output would look like
-this:
-
-    Col2,Col3,tag4
-    #tag2,#tag3,#tag4
-    2,3,10
-    5,6,40
-
-
-For JSON output, if no `metadata` or `data` is specified, the output will look like
-this:
-
-    [
-    {"#tag1":1,"#tag2":"2","#tag4":10},
-    {"#tag1":4,"#tag2":"5","#tag4":40}
-    ]
-
-If only `metadata` was specified, not `data`, then output is like this:
-
-    {"metadata":{"#date":"today!","#mytag":123},"data":[
-    {"#tag1":1,"#tag2":"2","#tag4":10},
-    {"#tag1":4,"#tag2":"5","#tag4":40}
-    ]}
-
-Otherwise, the result is like this:
-
-    {"metadata":{"#date":"today!","#mytag":123},"results":[
-    {"#tag1":1,"#tag2":"2","#tag4":10},
-    {"#tag1":4,"#tag2":"5","#tag4":40}
-    ]}
-
-The utility is called as follows:
-
-    save_hxlated_output(
-        configuration,
-        rows,
-        includes_header=True,
-        includes_hxltags=True,
-        output_dir=output_dir,
-        today="today!",
-    )
-
-The first parameter is the configuration which can come from a YAML file for example.
-The second parameter, `rows` is the data. That data can be a list of lists, tuples or
-dictionaries. If `includes_header` is `True`, headers are taken from `rows`, otherwise
-they must be given by the configuration. If `includes_hxltags` is `True`, HXL hashtags
-are taken from `rows`, otherwise they must be given by the configuration. `output_dir`
-specifies where the output should go and defaults to "". Any other parameters (such as
-`today` in the example above) are used to populate template variables given in the
-configuration for the metadata.
-
 ## Dictionary and list utilities
 
 Examples:

{hdx_python_utilities-4.0.7 → hdx_python_utilities-4.1.0}/pyproject.toml

@@ -39,14 +39,14 @@ requires-python = ">=3.10"
 # Extras for frictionless[excel,json] added explicitly
 # for conda-forge compatibility
 dependencies = [
-    "frictionless>=5.18.0",
+    "frictionless>=5.19.0",
     # frictionless[excel]
     "openpyxl>=3.1.2",
     "tableschema-to-template>=0.0.13",
     "xlrd>=2.0.1",
     "xlwt>=1.3.0",
     # frictionless[json]
-    "ijson>=3.2.3",
+    "ijson>=3.5.0",
     "jsonlines>=4.0.0",
     # /end frictionless extras
     "typing_extensions",
@@ -56,7 +56,7 @@ dependencies = [
     "ratelimit",
     "requests-file",
     "ruamel.yaml",
-    "xlsx2csv",
+    "xlsx2csv==0.8.4",
 ]
 
 [project.optional-dependencies]

hdx_python_utilities-4.1.0/src/hdx/utilities/_version.py

@@ -0,0 +1,24 @@
+# file generated by vcs-versioning
+# don't change, don't track in version control
+from __future__ import annotations
+
+__all__ = [
+    "__version__",
+    "__version_tuple__",
+    "version",
+    "version_tuple",
+    "__commit_id__",
+    "commit_id",
+]
+
+version: str
+__version__: str
+__version_tuple__: tuple[int | str, ...]
+version_tuple: tuple[int | str, ...]
+commit_id: str | None
+__commit_id__: str | None
+
+__version__ = version = '4.1.0'
+__version_tuple__ = version_tuple = (4, 1, 0)
+
+__commit_id__ = commit_id = None

{hdx_python_utilities-4.0.7 → hdx_python_utilities-4.1.0}/src/hdx/utilities/base_downloader.py

@@ -90,7 +90,6 @@ class BaseDownload(ABC):
     def get_tabular_rows(
         self,
         url: Path | str | Sequence[str],
-        has_hxl: bool = False,
         headers: int | Sequence[int] | Sequence[str] = 1,
         dict_form: bool = False,
         *args: Any,
@@ -100,8 +99,6 @@ class BaseDownload(ABC):
         where each row is returned as a list or dictionary depending on the
         dict_rows argument.
 
-        When a list of urls is supplied (in url), then the has_hxl flag indicates if the
-        files are HXLated so that the HXL row is only included from the first file.
         The headers argument is either a row number or list of row numbers (in case of
         multi-line headers) to be considered as headers (rows start counting at 1), or
         the actual headers defined as a list of strings. It defaults to 1.
@@ -110,7 +107,6 @@ class BaseDownload(ABC):
 
         Args:
             url: A single or list of URLs or paths to read from
-            has_hxl: Whether files have HXL hashtags. Ignored for single url. Defaults to False.
             headers: Number of row(s) containing headers or list of headers. Defaults to 1.
             dict_form: Return dict or list for each row. Defaults to False (list)
             *args (Any): Positional arguments

{hdx_python_utilities-4.0.7 → hdx_python_utilities-4.1.0}/src/hdx/utilities/downloader.py

@@ -142,28 +142,6 @@ class Download(BaseDownload):
         preparedrequest = self.session.prepare_request(request)
         return preparedrequest.url
 
-    @staticmethod
-    def hxl_row(
-        headers: Sequence[str],
-        hxltags: dict[str, str],
-        dict_form: bool = False,
-    ) -> list[str] | dict[str, str]:
-        """Return HXL tag row for header row given list of headers and
-        dictionary with header to HXL hashtag mappings. Return list or
-        dictionary depending upon the dict_form argument.
-
-        Args:
-            headers: Headers for which to get HXL hashtags
-            hxltags: Header to HXL hashtag mapping
-            dict_form: Return dict or list. Defaults to False (list)
-
-        Returns:
-            Return either a list or dictionary conating HXL hashtags
-        """
-        if dict_form:
-            return {header: hxltags.get(header, "") for header in headers}
-        return [hxltags.get(header, "") for header in headers]
-
     def normal_setup(
         self,
         url: Path | str,
@@ -674,7 +652,6 @@ class Download(BaseDownload):
     def get_tabular_rows(
         self,
         url: Path | str | Sequence[str],
-        has_hxl: bool = False,
         headers: int | Sequence[int] | Sequence[str] = 1,
         dict_form: bool = False,
         include_headers: bool = False,
@@ -688,8 +665,6 @@ class Download(BaseDownload):
         where each row is returned as a list or dictionary depending on the
         dict_rows argument.
 
-        When a list of urls is supplied (in url), then the has_hxl flag indicates if the
-        files are HXLated so that the HXL row is only included from the first file.
         The headers argument is either a row number or list of row numbers (in case of
         multi-line headers) to be considered as headers (rows start counting at 1), or
         the actual headers defined as a list of strings. It defaults to 1. The dict_form
@@ -705,7 +680,6 @@ class Download(BaseDownload):
 
         Args:
             url: A single or list of URLs or paths to read from
-            has_hxl: Whether files have HXL hashtags. Ignored for single url. Defaults to False.
             headers: Number of row(s) containing headers or list of headers. Defaults to 1.
             dict_form: Return dict or list for each row. Defaults to False (list)
             include_headers: Whether to include headers in iterator. Defaults to False.
@@ -772,8 +746,6 @@ class Download(BaseDownload):
                     row_function,
                     **temp_kwargs,
                 )
-                if has_hxl:
-                    next(iterator)
                 yield from iterator
 
         return outheaders, make_iterator()
@@ -781,7 +753,6 @@ class Download(BaseDownload):
     def get_tabular_rows_as_list(
         self,
         url: Path | str | Sequence[str],
-        has_hxl: bool = False,
         headers: int | Sequence[int] | Sequence[str] = 1,
         include_headers: bool = True,
         ignore_blank_rows: bool = True,
@@ -793,8 +764,6 @@ class Download(BaseDownload):
         """Returns headers and an iterator where each row is returned as a
         list.
 
-        When a list of urls is supplied (in url), then the has_hxl flag indicates if the
-        files are HXLated so that the HXL row is only included from the first file.
         The headers argument is either a row number or list of row numbers (in case of
         multi-line headers) to be considered as headers (rows start counting at 1), or
         the actual headers defined as a list of strings. It defaults to 1 and cannot be
@@ -809,7 +778,6 @@ class Download(BaseDownload):
 
         Args:
             url: A single or list of URLs or paths to read from
-            has_hxl: Whether files have HXL hashtags. Ignored for single url. Defaults to False.
             headers: Number of row(s) containing headers or list of headers. Defaults to 1.
             include_headers: Whether to include headers in iterator. Defaults to True.
             ignore_blank_rows: Whether to ignore blank rows. Defaults to True.
@@ -842,7 +810,6 @@ class Download(BaseDownload):
 
         headers, iterator = self.get_tabular_rows(
             url,
-            has_hxl,
             headers,
             False,
             include_headers,
@@ -857,7 +824,6 @@ class Download(BaseDownload):
     def get_tabular_rows_as_dict(
         self,
         url: Path | str | Sequence[str],
-        has_hxl: bool = False,
         headers: int | Sequence[int] | Sequence[str] = 1,
         ignore_blank_rows: bool = True,
         infer_types: bool = False,
@@ -868,8 +834,6 @@ class Download(BaseDownload):
         """Returns headers and an iterator where each row is returned as a
         dictionary.
 
-        When a list of urls is supplied (in url), then the has_hxl flag indicates if the
-        files are HXLated so that the HXL row is only included from the first file.
         The headers argument is either a row number or list of row numbers (in case of
         multi-line headers) to be considered as headers (rows start counting at 1), or
         the actual headers defined as a list of strings. It defaults to 1 and cannot be
@@ -884,7 +848,6 @@ class Download(BaseDownload):
 
         Args:
             url: A single or list of URLs or paths to read from
-            has_hxl: Whether files have HXL hashtags. Ignored for single url. Defaults to False.
             headers: Number of row(s) containing headers or list of headers. Defaults to 1.
             ignore_blank_rows: Whether to ignore blank rows. Defaults to True.
             infer_types: Whether to infer types. Defaults to False (strings).
@@ -916,7 +879,6 @@ class Download(BaseDownload):
 
         headers, iterator = self.get_tabular_rows(
             url,
-            has_hxl,
             headers,
             True,
             False,
@@ -931,7 +893,6 @@ class Download(BaseDownload):
     def download_tabular_key_value(
         self,
         url: Path | str | Sequence[str],
-        has_hxl: bool = False,
         headers: int | Sequence[int] | Sequence[str] = 1,
         include_headers: bool = True,
         ignore_blank_rows: bool = True,
@@ -943,8 +904,6 @@ class Download(BaseDownload):
         """Download 2 column csv from url and return a dictionary of keys
         (first column) and values (second column).
 
-        When a list of urls is supplied (in url), then the has_hxl flag indicates if the
-        files are HXLated so that the HXL row is only included from the first file.
         The headers argument is either a row number or list of row numbers (in case of
         multi-line headers) to be considered as headers (rows start counting at 1), or
         the actual headers defined as a list of strings. It defaults to 1 and cannot be
@@ -957,10 +916,8 @@ class Download(BaseDownload):
         and row (which will be in dict or list form depending upon the dict_rows
         argument) and outputs a modified row or None to ignore the row.
 
-
         Args:
             url: A single or list of URLs or paths to read from
-            has_hxl: Whether files have HXL hashtags. Ignored for single url. Defaults to False.
             headers: Number of row(s) containing headers or list of headers. Defaults to 1.
             include_headers: Whether to include headers in iterator. Defaults to True.
             ignore_blank_rows: Whether to ignore blank rows. Defaults to True.
@@ -993,7 +950,6 @@ class Download(BaseDownload):
         output_dict = {}
         _, rows = self.get_tabular_rows_as_list(
             url,
-            has_hxl,
             headers,
             include_headers,
             ignore_blank_rows,
@@ -1011,7 +967,6 @@ class Download(BaseDownload):
     def download_tabular_rows_as_dicts(
         self,
         url: Path | str | Sequence[str],
-        has_hxl: bool = False,
         headers: int | Sequence[int] | Sequence[str] = 1,
         keycolumn: int = 1,
         ignore_blank_rows: bool = True,
@@ -1024,8 +979,6 @@ class Download(BaseDownload):
         are first column and values are dictionaries with keys from column
         headers and values from columns beneath.
 
-        When a list of urls is supplied (in url), then the has_hxl flag indicates if the
-        files are HXLated so that the HXL row is only included from the first file.
         The headers argument is either a row number or list of row numbers (in case of
         multi-line headers) to be considered as headers (rows start counting at 1), or
         the actual headers defined as a list of strings. It defaults to 1 and cannot be
@@ -1040,7 +993,6 @@ class Download(BaseDownload):
 
         Args:
             url: A single or list of URLs or paths to read from
-            has_hxl: Whether files have HXL hashtags. Ignored for single url. Defaults to False.
             headers: Number of row(s) containing headers or list of headers. Defaults to 1.
             keycolumn: Number of column to be used for key. Defaults to 1.
             ignore_blank_rows: Whether to ignore blank rows. Defaults to True.
@@ -1073,7 +1025,6 @@ class Download(BaseDownload):
         """
         headers, iterator = self.get_tabular_rows_as_dict(
             url,
-            has_hxl,
             headers,
             ignore_blank_rows,
             infer_types,
@@ -1095,7 +1046,6 @@ class Download(BaseDownload):
     def download_tabular_cols_as_dicts(
         self,
         url: Path | str | Sequence[str],
-        has_hxl: bool = False,
         headers: int | Sequence[int] | Sequence[str] = 1,
         keycolumn: int = 1,
         ignore_blank_rows: bool = True,
@@ -1108,8 +1058,6 @@ class Download(BaseDownload):
         are header names and values are dictionaries with keys from first
         column and values from other columns.
 
-        When a list of urls is supplied (in url), then the has_hxl flag indicates if the
-        files are HXLated so that the HXL row is only included from the first file.
         The headers argument is either a row number or list of row numbers (in case of
         multi-line headers) to be considered as headers (rows start counting at 1), or
         the actual headers defined as a list of strings. It defaults to 1 and cannot be
@@ -1124,7 +1072,6 @@ class Download(BaseDownload):
 
         Args:
             url: A single or list of URLs or paths to read from
-            has_hxl: Whether files have HXL hashtags. Ignored for single url. Defaults to False.
             headers: Number of row(s) containing headers or list of headers. Defaults to 1.
             keycolumn: Number of column to be used for key. Defaults to 1.
             ignore_blank_rows: Whether to ignore blank rows. Defaults to True.
@@ -1157,7 +1104,6 @@ class Download(BaseDownload):
         """
         headers, iterator = self.get_tabular_rows_as_dict(
             url,
-            has_hxl,
             headers,
             ignore_blank_rows,
             infer_types,

{hdx_python_utilities-4.0.7 → hdx_python_utilities-4.1.0}/src/hdx/utilities/path.py

@@ -1,7 +1,6 @@
 """Directory Path Utilities."""
 
 import contextlib
-import inspect
 import logging
 import sys
 from collections.abc import Iterable, Iterator, Sequence
@@ -12,7 +11,6 @@ from os.path import (
 from pathlib import Path
 from shutil import rmtree
 from tempfile import gettempdir
-from typing import Any
 
 from hdx.utilities.loader import load_text
 from hdx.utilities.saver import save_text
@@ -25,45 +23,34 @@ class NotFoundError(Exception):
     pass
 
 
-def script_dir(pyobject: Any, follow_symlinks: bool = True) -> Path:
+def script_dir(*args, **kwargs) -> Path:
     """Get current script's directory.
 
-    Args:
-        pyobject: Any Python object in the script
-        follow_symlinks: Follow symlinks or not. Defaults to True.
-
     Returns:
         Current script's directory
     """
     if getattr(sys, "frozen", False):
-        # Frozen (PyInstaller, etc.): Use the executable path
-        path = Path(sys.executable)  # pragma: no cover
+        # PyInstaller bundle
+        app_path = Path(sys.executable)
     else:
-        # Standard: Use the object's file path
-        path = Path(inspect.getfile(pyobject))
-
-    # Resolve symlinks if requested, otherwise just make absolute
-    path = path.resolve() if follow_symlinks else path.absolute()
+        # Standard Python script
+        app_path = Path(__file__)
 
-    return path.parent
+    # .resolve() ensures we follow symlinks to the real folder,
+    # and makes the path absolute so CWD doesn't matter.
+    return app_path.resolve().parent
 
 
-def script_dir_plus_file(
-    filename: str, pyobject: Any, follow_symlinks: bool = True
-) -> Path:
+def script_dir_plus_file(filename: str, *args, **kwargs) -> Path:
     """Get current script's directory and then append a filename.
 
     Args:
         filename: Filename to append to directory path
-        pyobject: Any Python object in the script
-        return_path: Whether to return a Path object. Defaults to returning str.
-
-        follow_symlinks: Follow symlinks or not. Defaults to True.
 
     Returns:
         Current script's directory and with filename appended
     """
-    return script_dir(pyobject, follow_symlinks) / filename
+    return script_dir() / filename
 
 
 def get_temp_dir(

{hdx_python_utilities-4.0.7 → hdx_python_utilities-4.1.0}/src/hdx/utilities/retriever.py

@@ -385,7 +385,6 @@ class Retrieve(BaseDownload):
     def get_tabular_rows(
         self,
         url: Path | str | Sequence[str],
-        has_hxl: bool = False,
         headers: int | Sequence[int] | Sequence[str] = 1,
         dict_form: bool = False,
         filename: str | None = None,
@@ -397,8 +396,6 @@ class Retrieve(BaseDownload):
         where each row is returned as a list or dictionary depending on the
         dict_rows argument.
 
-        When a list of urls is supplied (in url), then the has_hxl flag indicates if the
-        files are HXLated so that the HXL row is only included from the first file.
         The headers argument is either a row number or list of row numbers (in case of
         multi-line headers) to be considered as headers (rows start counting at 1), or
         the actual headers defined as a list of strings. It defaults to 1.
@@ -407,7 +404,6 @@ class Retrieve(BaseDownload):
 
         Args:
             url: A single or list of URLs or paths to read from
-            has_hxl: Whether files have HXL hashtags. Defaults to False.
             headers: Number of row(s) containing headers or list of headers. Defaults to 1.
             dict_form: Return dict or list for each row. Defaults to False (list)
             filename: Filename of saved file. Defaults to getting from url.
@@ -434,9 +430,7 @@ class Retrieve(BaseDownload):
                 path.append(pth)
 
         kwargs.pop("file_prefix", None)
-        return self.downloader.get_tabular_rows(
-            path, has_hxl, headers, dict_form, **kwargs
-        )
+        return self.downloader.get_tabular_rows(path, headers, dict_form, **kwargs)
 
     @classmethod
     def generate_retrievers(