LSMS_Library 0.10.1__tar.gz → 0.11.0__tar.gz

lsms_library-0.11.0/PKG-INFO

@@ -0,0 +1,179 @@
+Metadata-Version: 2.4
+Name: LSMS_Library
+Version: 0.11.0
+Summary: Abstraction layer for Living Standards Measurement Survey data
+License: CC-BY-4.0
+License-File: LICENSE.txt
+Author: Ethan Ligon
+Author-email: ligon@berkeley.edu
+Requires-Python: >=3.11, <4.0
+Classifier: License :: Other/Proprietary License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Requires-Dist: CFEDemands (>=0.8.2)
+Requires-Dist: PyYAML (>=6.0,<7.0)
+Requires-Dist: cryptography (>=3.4)
+Requires-Dist: dvc[s3] (>=3.67.0,<4.0.0)
+Requires-Dist: folium (>=0.19)
+Requires-Dist: fooddatacentral (>=1.0.10,<2.0.0)
+Requires-Dist: ligonlibrary (>=0.2.0)
+Requires-Dist: matplotlib (>=3.10.0,<4.0.0)
+Requires-Dist: openai (>=1.99.5,<2.0.0)
+Requires-Dist: pandas (>=3.0)
+Requires-Dist: pint (>=0.25.2,<0.26.0)
+Requires-Dist: platformdirs (>=4.0.0,<5.0.0)
+Requires-Dist: pyarrow (>=21.0.0)
+Requires-Dist: pyreadstat (>=1.3.0,<2.0.0)
+Requires-Dist: python-Levenshtein (>=0.27.3,<0.28.0)
+Requires-Dist: python-gnupg (>=0.5,<1.0)
+Requires-Dist: scikit-learn (>=1.7.0,<2.0.0)
+Requires-Dist: typer (>=0.12.0,<1.0.0)
+Project-URL: Repository, https://github.com/ligon/LSMS_Library
+Description-Content-Type: text/plain
+
+#+TITLE: LSMS_Library
+#+AUTHOR: Ethan Ligon
+#+OPTIONS: toc:nil
+
+[[https://doi.org/10.5281/zenodo.17258079][https://img.shields.io/badge/DOI-10.5281%2Fzenodo.17258079-blue.svg]]
+
+A Python library providing a uniform interface to Living Standards Measurement Study (LSMS) household surveys from multiple countries and years, without the data loss typical of traditional harmonization approaches.
+
+* The Problem
+
+LSMS datasets are invaluable for studying poverty, consumption, and household welfare across developing countries. However, each country's survey uses different:
+- Variable names and encodings
+- Food classification systems
+- Questionnaire structures
+- File formats and organization
+
+Researchers typically spend weeks learning each new dataset's idiosyncrasies or use pre-harmonized datasets that sacrifice detail and comparability. Cross-country or longitudinal analyses become prohibitively time-consuming.
+
+* The Solution
+
+LSMS_Library provides an *abstraction layer* that gives you a consistent interface to work with any supported LSMS dataset. Instead of harmonizing the data itself (which loses information), we harmonize the /way you access/ the data.
+
+* Installation
+
+From PyPI (when available):
+
+#+begin_src bash
+pip install LSMS_Library
+#+end_src
+
+From github (current release):
+
+#+begin_src bash
+pip install git+https://github.com/ligon/LSMS_Library.git@v0.7.0
+#+end_src
+
+From a source checkout (for contributors):
+
+#+begin_src bash
+git clone https://github.com/ligon/LSMS_Library.git
+cd LSMS_Library
+poetry install
+#+end_src
+
+* Quick Start
+
+#+begin_src python
+import lsms_library as ll
+
+# Single-country access
+uga = ll.Country('Uganda')
+uga.waves          # ['2005-06', '2009-10', ..., '2019-20']
+uga.data_scheme    # ['food_acquired', 'household_roster', ...]
+food = uga.food_expenditures()   # Standardized DataFrame, all waves
+
+# Cross-country analysis
+roster = ll.Feature('household_roster')
+roster.countries   # ['Burkina_Faso', 'Ethiopia', 'Mali', 'Uganda', ...]
+df = roster()      # Harmonized DataFrame across all countries
+#+end_src
+
+* Data Access
+
+This library abstracts over Living Standards Measurement Study (LSMS) survey data.  The underlying microdata belongs to the respective national statistics offices and the World Bank; users must accept the [[https://microdata.worldbank.org/][World Bank Microdata Library]]'s terms of use before accessing it.
+
+** Authentication: the WB Microdata API key
+
+1. Register at https://microdata.worldbank.org/ (free).
+2. Accept the terms of use for the LSMS collections you want to access.
+3. Get your API key from your account dashboard.
+4. Create =~/.config/lsms_library/config.yml= with:
+
+   #+begin_src yaml
+   microdata_api_key: your_key_here
+   # data_dir: /path/to/override   # same as LSMS_DATA_DIR env var
+   #+end_src
+
+   or set =MICRODATA_API_KEY= as an environment variable.
+5. On =import lsms_library=, the library validates your key against the WB catalog and automatically unlocks access to the S3 read cache.  No further setup is required.
+
+** The S3 cache
+
+Once your WB API key is validated, the library unlocks a read-only S3 cache that mirrors the WB Microdata downloads.  This is a convenience -- the S3 cache is faster than the WB NADA API and reduces load on the WB service -- but it is not a separate access layer.  The WB terms of use are the authoritative gate; the S3 cache just provides the same data faster.
+
+Decrypted plaintext credentials are written to =~/.config/lsms_library/s3_creds= (or the path in the =LSMS_S3_CREDS= environment variable), not into the package tree -- so the library is safe to install from a wheel into a read-only site-packages directory.
+
+*A valid WB API key is sufficient on its own.*  No other software is required: the unlock is pure Python, via =cryptography=, which installs with the library.  Before v0.10.1 it also needed the =gpg= binary -- not a Python package, and absent by default on Windows and macOS -- so a user with a perfectly good key hit =NoCredentialsError= from botocore with nothing pointing at the real cause (GH #741).
+
+If the unlock does fail, the library says so and falls back to downloading directly from the World Bank: slower, but nothing is blocked.  The =gpg= binary is now needed only for the interactive keyless fallback below, and even that prefers the pure-Python path.
+
+** Non-interactive environments
+
+In CI, Docker builds, or other non-interactive contexts, set =LSMS_SKIP_AUTH=1= to suppress the import-time authentication flow.  In that mode you are responsible for ensuring =~/.config/lsms_library/s3_creds= exists (e.g. via a CI secret mount) before the first data access.
+
+** Data cache location
+
+Parquet caches materialize under the platform-appropriate user data directory:
+
+- Linux: =~/.local/share/lsms_library/= by default
+- Override with =LSMS_DATA_DIR= env var or =data_dir= in =config.yml=
+
+See the [[https://ligon.github.io/LSMS_Library/guide/caching/][caching guide]] for details on =assume_cache_fresh=, the =Country= and =Feature= classes, and per-country build methods.
+
+* Documentation
+
+Full documentation is available at [[https://ligon.github.io/LSMS_Library][ligon.github.io/LSMS_Library]], including:
+
+- **[[https://ligon.github.io/LSMS_Library/getting-started/][Getting Started]]** -- installation and first steps
+- **[[https://ligon.github.io/LSMS_Library/guide/country/][Country Guide]]** -- single-country workflows, harmonization pipeline, derived tables
+- **[[https://ligon.github.io/LSMS_Library/guide/feature/][Feature Guide]]** -- cross-country analysis with =ll.Feature=
+- **[[https://ligon.github.io/LSMS_Library/guide/caching/][Caching]]** -- performance tuning, build backends, cache management
+- **[[https://ligon.github.io/LSMS_Library/guide/panel-data/][Panel Data]]** -- longitudinal analysis and ID harmonization
+- **[[https://ligon.github.io/LSMS_Library/api/country/][API Reference]]** -- complete class documentation (auto-generated from source)
+
+* Contributing
+
+See [[file:CONTRIBUTING.org][CONTRIBUTING.org]] for detailed guidelines on adding new datasets using DVC.
+
+* Citation
+
+If you use LSMS_Library in your research, please cite:
+
+#+begin_src bibtex
+@software{ligon25:lsms_library,
+  author =    {Ethan Ligon},
+  title =     {{\tt LSMS_Library}: Abstraction layer for working with Living Standards Measurement Surveys},
+  year =      2025,
+  doi = {10.5281/zenodo.17258079},
+  url = {https://pypi.org/project/lsms_library/}
+}
+#+end_src
+
+* License
+
+See the [[file:LICENSE][LICENSE]] file in the repository for details.
+
+* Acknowledgments
+
+This project builds on data collection efforts by:
+- The World Bank's Living Standards Measurement Study (LSMS) team
+- National statistical offices in participating countries
+- The LSMS-ISA initiative
+

lsms_library-0.11.0/README.org

@@ -0,0 +1,142 @@
+#+TITLE: LSMS_Library
+#+AUTHOR: Ethan Ligon
+#+OPTIONS: toc:nil
+
+[[https://doi.org/10.5281/zenodo.17258079][https://img.shields.io/badge/DOI-10.5281%2Fzenodo.17258079-blue.svg]]
+
+A Python library providing a uniform interface to Living Standards Measurement Study (LSMS) household surveys from multiple countries and years, without the data loss typical of traditional harmonization approaches.
+
+* The Problem
+
+LSMS datasets are invaluable for studying poverty, consumption, and household welfare across developing countries. However, each country's survey uses different:
+- Variable names and encodings
+- Food classification systems
+- Questionnaire structures
+- File formats and organization
+
+Researchers typically spend weeks learning each new dataset's idiosyncrasies or use pre-harmonized datasets that sacrifice detail and comparability. Cross-country or longitudinal analyses become prohibitively time-consuming.
+
+* The Solution
+
+LSMS_Library provides an *abstraction layer* that gives you a consistent interface to work with any supported LSMS dataset. Instead of harmonizing the data itself (which loses information), we harmonize the /way you access/ the data.
+
+* Installation
+
+From PyPI (when available):
+
+#+begin_src bash
+pip install LSMS_Library
+#+end_src
+
+From github (current release):
+
+#+begin_src bash
+pip install git+https://github.com/ligon/LSMS_Library.git@v0.7.0
+#+end_src
+
+From a source checkout (for contributors):
+
+#+begin_src bash
+git clone https://github.com/ligon/LSMS_Library.git
+cd LSMS_Library
+poetry install
+#+end_src
+
+* Quick Start
+
+#+begin_src python
+import lsms_library as ll
+
+# Single-country access
+uga = ll.Country('Uganda')
+uga.waves          # ['2005-06', '2009-10', ..., '2019-20']
+uga.data_scheme    # ['food_acquired', 'household_roster', ...]
+food = uga.food_expenditures()   # Standardized DataFrame, all waves
+
+# Cross-country analysis
+roster = ll.Feature('household_roster')
+roster.countries   # ['Burkina_Faso', 'Ethiopia', 'Mali', 'Uganda', ...]
+df = roster()      # Harmonized DataFrame across all countries
+#+end_src
+
+* Data Access
+
+This library abstracts over Living Standards Measurement Study (LSMS) survey data.  The underlying microdata belongs to the respective national statistics offices and the World Bank; users must accept the [[https://microdata.worldbank.org/][World Bank Microdata Library]]'s terms of use before accessing it.
+
+** Authentication: the WB Microdata API key
+
+1. Register at https://microdata.worldbank.org/ (free).
+2. Accept the terms of use for the LSMS collections you want to access.
+3. Get your API key from your account dashboard.
+4. Create =~/.config/lsms_library/config.yml= with:
+
+   #+begin_src yaml
+   microdata_api_key: your_key_here
+   # data_dir: /path/to/override   # same as LSMS_DATA_DIR env var
+   #+end_src
+
+   or set =MICRODATA_API_KEY= as an environment variable.
+5. On =import lsms_library=, the library validates your key against the WB catalog and automatically unlocks access to the S3 read cache.  No further setup is required.
+
+** The S3 cache
+
+Once your WB API key is validated, the library unlocks a read-only S3 cache that mirrors the WB Microdata downloads.  This is a convenience -- the S3 cache is faster than the WB NADA API and reduces load on the WB service -- but it is not a separate access layer.  The WB terms of use are the authoritative gate; the S3 cache just provides the same data faster.
+
+Decrypted plaintext credentials are written to =~/.config/lsms_library/s3_creds= (or the path in the =LSMS_S3_CREDS= environment variable), not into the package tree -- so the library is safe to install from a wheel into a read-only site-packages directory.
+
+*A valid WB API key is sufficient on its own.*  No other software is required: the unlock is pure Python, via =cryptography=, which installs with the library.  Before v0.10.1 it also needed the =gpg= binary -- not a Python package, and absent by default on Windows and macOS -- so a user with a perfectly good key hit =NoCredentialsError= from botocore with nothing pointing at the real cause (GH #741).
+
+If the unlock does fail, the library says so and falls back to downloading directly from the World Bank: slower, but nothing is blocked.  The =gpg= binary is now needed only for the interactive keyless fallback below, and even that prefers the pure-Python path.
+
+** Non-interactive environments
+
+In CI, Docker builds, or other non-interactive contexts, set =LSMS_SKIP_AUTH=1= to suppress the import-time authentication flow.  In that mode you are responsible for ensuring =~/.config/lsms_library/s3_creds= exists (e.g. via a CI secret mount) before the first data access.
+
+** Data cache location
+
+Parquet caches materialize under the platform-appropriate user data directory:
+
+- Linux: =~/.local/share/lsms_library/= by default
+- Override with =LSMS_DATA_DIR= env var or =data_dir= in =config.yml=
+
+See the [[https://ligon.github.io/LSMS_Library/guide/caching/][caching guide]] for details on =assume_cache_fresh=, the =Country= and =Feature= classes, and per-country build methods.
+
+* Documentation
+
+Full documentation is available at [[https://ligon.github.io/LSMS_Library][ligon.github.io/LSMS_Library]], including:
+
+- **[[https://ligon.github.io/LSMS_Library/getting-started/][Getting Started]]** -- installation and first steps
+- **[[https://ligon.github.io/LSMS_Library/guide/country/][Country Guide]]** -- single-country workflows, harmonization pipeline, derived tables
+- **[[https://ligon.github.io/LSMS_Library/guide/feature/][Feature Guide]]** -- cross-country analysis with =ll.Feature=
+- **[[https://ligon.github.io/LSMS_Library/guide/caching/][Caching]]** -- performance tuning, build backends, cache management
+- **[[https://ligon.github.io/LSMS_Library/guide/panel-data/][Panel Data]]** -- longitudinal analysis and ID harmonization
+- **[[https://ligon.github.io/LSMS_Library/api/country/][API Reference]]** -- complete class documentation (auto-generated from source)
+
+* Contributing
+
+See [[file:CONTRIBUTING.org][CONTRIBUTING.org]] for detailed guidelines on adding new datasets using DVC.
+
+* Citation
+
+If you use LSMS_Library in your research, please cite:
+
+#+begin_src bibtex
+@software{ligon25:lsms_library,
+  author =    {Ethan Ligon},
+  title =     {{\tt LSMS_Library}: Abstraction layer for working with Living Standards Measurement Surveys},
+  year =      2025,
+  doi = {10.5281/zenodo.17258079},
+  url = {https://pypi.org/project/lsms_library/}
+}
+#+end_src
+
+* License
+
+See the [[file:LICENSE][LICENSE]] file in the repository for details.
+
+* Acknowledgments
+
+This project builds on data collection efforts by:
+- The World Bank's Living Standards Measurement Study (LSMS) team
+- National statistical offices in participating countries
+- The LSMS-ISA initiative

lsms_library-0.11.0/lsms_library/__init__.py

@@ -0,0 +1,256 @@
+"""LSMS Library: a harmonized interface to Living Standards Measurement Study data.
+
+The library provides a uniform API across 40+ LSMS-ISA and related
+household surveys from the World Bank Microdata Library.  Its design
+principle is to harmonize the *interface*, not the data: surveys differ
+in structure, and the library preserves survey-specific detail while
+offering a common way to read it.
+
+Typical usage
+-------------
+Instantiate a :class:`Country` and call its table methods as attributes.
+Table names (``food_acquired``, ``household_roster``, ``shocks``, ...)
+come from the country's ``data_scheme.yml``.
+
+.. code-block:: python
+
+    import lsms_library as ll
+
+    uga = ll.Country('Uganda')
+    uga.waves              # ['2005-06', '2009-10', ...]
+    uga.data_scheme        # ['household_roster', 'food_acquired', ...]
+
+    roster = uga.household_roster()    # harmonized DataFrame across waves
+    food   = uga.food_expenditures()   # derived from food_acquired
+    chars  = uga.household_characteristics()  # derived from household_roster
+
+:class:`Feature` assembles the same table across every country that
+declares it, with a ``country`` index level prepended:
+
+.. code-block:: python
+
+    df = ll.Feature('household_roster')()               # every country
+    df = ll.Feature('shocks')(['Ethiopia', 'Niger'])    # subset
+
+Returned DataFrames are indexed by some subset of ``(country, t, v, i,
+pid, j)`` — wave (``t``), cluster (``v``), household (``i``), person
+(``pid``), item (``j``) — depending on the table.
+
+Public API
+----------
+- :class:`Country` — top-level per-country interface. Exposes ``waves``,
+  ``data_scheme``, and one method per table in the scheme. Derived
+  tables (``household_characteristics``, ``food_expenditures``,
+  ``food_prices``, ``food_quantities``) are dispatched via
+  ``__getattr__`` without needing to be in ``data_scheme.yml``.
+- :class:`Wave` — single-wave view of a country, used internally by
+  :class:`Country` but occasionally handy for debugging.
+- :class:`Feature` — cross-country aggregator for a given table name.
+- ``tools`` (alias for :mod:`lsms_library.local_tools`) — ``get_dataframe``
+  for reading ``.dta``/``.csv``/``.parquet`` with local → DVC → WB NADA
+  fallback, ``to_parquet`` for writing caches under ``data_root()``, and
+  the ``df_data_grabber`` helper used by wave-level scripts.
+- :func:`authenticate` — interactive DVC/S3 credential unlock. Usually
+  runs automatically on import when a World Bank Microdata API key is
+  available.
+- :mod:`lsms_library.transformations` — derivation helpers for
+  roster-to-characteristics and food-to-expenditures.
+
+Data access and caching
+-----------------------
+Reads use a fallback chain: local file on disk → DVC filesystem → World
+Bank NADA download. Parquet caches are written under ``data_root()``,
+which defaults to ``~/.local/share/lsms_library/{Country}/`` and can be
+overridden via ``LSMS_DATA_DIR`` or the ``data_dir`` key in
+``~/.config/lsms_library/config.yml``.
+
+As of v0.7.0, :meth:`Country` methods check the parquet cache before
+consulting DVC, giving 10–17× cross-process speedups. The cache has no
+automatic staleness check — set ``LSMS_NO_CACHE=1`` or run
+``lsms-library cache clear --country {Country}`` after editing source
+configs or scripts.
+
+Environment variables
+---------------------
+- ``MICRODATA_API_KEY`` — World Bank Microdata Library API key (also
+  readable from ``~/.config/lsms_library/config.yml``). Required for
+  direct downloads; also unlocks the S3 read cache at import time.
+- ``LSMS_DATA_DIR`` — override the parquet cache root.
+- ``LSMS_NO_CACHE`` — disable the v0.7.0 top-of-function cache read.
+- ``LSMS_BUILD_BACKEND=make`` — force rebuild from source, bypassing
+  both the cache and the DVC stage layer.
+- ``LSMS_SKIP_AUTH`` — suppress the import-time authentication attempt.
+- ``LSMS_BUILD_WORKERS`` — worker processes for a cold wave-parallel build
+  (default: the cgroup-visible CPU count, capped at the number of distinct
+  build targets and by the memory guard below; ``1`` = the serial path).
+  Waves that share a make target build serially in one worker; output is
+  byte-identical to serial.  An explicit value may exceed the memory guard
+  (warns once) but never the CPU count.
+- ``LSMS_BUILD_WORKER_MEM_GB`` — GiB reserved per build worker (default
+  ``4``, covering the largest wave script measured at 3.7 GB).  The default
+  worker count is capped at ``visible memory // this``, where visible memory
+  is the smallest of MemAvailable, the cgroup memory limit and Slurm's
+  ``SLURM_MEM_PER_NODE`` / ``SLURM_MEM_PER_CPU`` allocation.
+- ``LSMS_MAKE_JOBS`` — make ``-j`` parallelism for source rebuilds
+  (default: half the cgroup-visible CPUs -- ``sched_getaffinity`` capped by
+  ``SLURM_CPUS_PER_TASK`` / ``SLURM_CPUS_ON_NODE``, GH #764).  A country
+  build runs one wave build per job, i.e. that many concurrent large S3
+  fetches; set ``LSMS_MAKE_JOBS=1`` to serialize them on hosts where
+  concurrent multipart reads are flaky.  Under ``LSMS_BUILD_WORKERS`` it
+  is a ceiling on each worker's ``-j``.
+- ``LSMS_FETCH_ATTEMPTS`` — retry budget (default ``3``) for a single
+  blob's S3 fetch on a transient TLS error before falling back to streaming.
+
+Further reading
+---------------
+- ``CLAUDE.md`` at the repository root for contributor-facing design
+  notes, the canonical schema (``lsms_library/data_info.yml``), and the
+  list of cross-cutting gotchas.
+- ``.claude/skills/add-feature/`` for adding a new table to a country,
+  with sub-skills for ``sample``, ``shocks``, ``assets``,
+  ``panel-ids``, ``food-acquired``, and ``pp-ph`` (post-planting /
+  post-harvest countries).
+- ``CONTRIBUTING.org`` for the new-wave workflow.
+
+Data terms of use
+-----------------
+Underlying microdata must be obtained from the World Bank Microdata
+Library (https://microdata.worldbank.org/) under their terms of use.
+The library's S3 bucket is a read cache over the authoritative WB NADA
+downloads, unlocked automatically when a valid API key is present.
+"""
+
+from importlib.metadata import PackageNotFoundError
+from importlib.metadata import version as _get_version
+
+try:
+    __version__ = _get_version("LSMS_Library")
+except PackageNotFoundError:
+    __version__ = "0.0.0"
+del _get_version
+
+from pathlib import Path
+import os
+import warnings
+
+import logging
+from sys import stderr
+from . import country
+from .country import Country, Wave
+from .feature import Feature
+from .catalog import countries, features
+from .coverage_matrix import coverage
+from .currency import currency_for
+from .conversion import convert
+from . import local_tools as tools
+from . import population
+from .population import PopulationRecord, population_records
+from . import transformations
+from . import visualizations
+from .visualizations import population_pyramid, coordinate_map
+from .dvc_permissions import authenticate
+try:
+    from dvc.ui import ui as dvc_ui
+    from dvc.logger import LoggerHandler, setup as dvc_log_setup
+    from functools import wraps
+    import sys
+
+    if hasattr(dvc_ui, "rich_console"):
+        try:
+            dvc_ui.rich_console.file = stderr
+        except (AttributeError, ImportError):
+            # Defensive against DVC rich-console API drift; programmer bugs
+            # (TypeError, NameError) propagate.
+            pass
+    if hasattr(dvc_ui, "error_console"):
+        try:
+            dvc_ui.error_console.file = stderr
+        except (AttributeError, ImportError):
+            # Defensive against DVC rich-console API drift.
+            pass
+    original_init = LoggerHandler.__init__
+
+    @wraps(original_init)
+    def _patched_logger_handler_init(self, stream, *args, **kwargs):
+        if stream is sys.stdout:
+            stream = stderr
+        original_init(self, stream, *args, **kwargs)
+
+    LoggerHandler.__init__ = _patched_logger_handler_init  # type: ignore[assignment]
+
+    dvc_log_setup()
+    for logger_name in ("dvc", "dvc_objects", "dvc_data", ""):
+        logger_obj = logging.getLogger(logger_name)
+        for handler in logger_obj.handlers:
+            if isinstance(handler, LoggerHandler) and getattr(handler, "stream", None) is not stderr:
+                handler.stream = stderr
+except (ImportError, AttributeError):
+    # DVC optional or its UI module changed shape; proceed without
+    # UI tweaks.  Programmer bugs surface unchanged.
+    pass
+
+from .config import s3_creds_path as _s3_creds_path
+creds_file = _s3_creds_path()
+
+SKIP_AUTH = os.getenv("LSMS_SKIP_AUTH", "").lower() in {"1", "true", "yes"}
+
+if not SKIP_AUTH and not creds_file.exists():
+    # Try non-interactive S3 unlock first.  This only decrypts the local
+    # s3_reader_creds.gpg — no network call required.  WB API key validation
+    # (which requires a network round-trip to microdata.worldbank.org) is
+    # deferred until the first actual data access via permissions() in
+    # data_access.py.  This keeps ``import lsms_library`` fast.
+    try:
+        from .data_access import _auto_unlock_s3 as _unlock
+        _unlocked = _unlock()
+        if not _unlocked:
+            # Local GPG decryption failed or no .gpg file — fall back to
+            # interactive passphrase prompt.
+            raise RuntimeError("auto-unlock did not produce S3 credentials")
+    except (ImportError, RuntimeError, OSError):
+        # Auto-unlock skipped (missing dep), failed (no .gpg / bad
+        # passphrase / gpg binary missing).  Fall through to interactive
+        # authenticate(); programmer bugs propagate.
+        try:
+            authenticate()
+        except (RuntimeError, OSError) as exc:
+            # Auth flow failure (network, subprocess, missing creds).
+            # Warn but don't crash import.
+            warnings.warn(
+                f"Automatic DVC authentication failed: {exc}. "
+                "Set LSMS_SKIP_AUTH=1 to suppress, or set "
+                "MICRODATA_API_KEY for non-interactive access."
+            )
+elif not SKIP_AUTH and creds_file.exists():
+    # User-config creds are already in place, so the auto-unlock branch
+    # above is short-circuited.  We still need to populate the legacy
+    # in-tree ``lsms_library/countries/.dvc/s3_creds`` path so that
+    # legacy wave scripts using ``dvc.api.open()`` (which don't pass a
+    # ``credentialpath`` override) can find credentials.  Without this,
+    # a fresh clone with valid user-config creds still hits
+    # ``NoCredentialsError`` on Uganda's earnings, enterprise_income,
+    # food_acquired etc. via the legacy DVC API path.
+    try:
+        from .data_access import _sync_legacy_dvc_creds as _sync
+        _sync()
+    except (ImportError, OSError):
+        # Sync failure is non-fatal --- legacy `dvc.api.open()` callers
+        # will surface a clear NoCredentialsError if it matters.
+        pass
+
+if not SKIP_AUTH:
+    # The DVCFS singleton is built when `local_tools` is imported, which is
+    # before either branch above can have written the credentials file.  On a
+    # first-ever import it was therefore built without explicit S3
+    # credentials, leaving it on the `credentialpath` fallback that an
+    # ambient AWS environment silently defeats (see
+    # `local_tools._s3_explicit_credentials`).  Rebuild it once, now that
+    # the file exists.  No-op when the credentials were already present at
+    # import time, which is the common path.
+    try:
+        from .local_tools import refresh_s3_credentials as _refresh_s3
+        _refresh_s3()
+    except (ImportError, OSError, AttributeError):
+        # Best-effort: a failure here leaves the credentialpath fallback,
+        # which is what shipped before.
+        pass