LSMS_Library 0.13.0__tar.gz → 0.15.0__tar.gz

lsms_library-0.15.0/PKG-INFO

@@ -0,0 +1,181 @@
+Metadata-Version: 2.4
+Name: LSMS_Library
+Version: 0.15.0
+Summary: Abstraction layer for Living Standards Measurement Survey data
+License: CC-BY-4.0
+License-File: LICENSE.txt
+Author: Ethan Ligon
+Author-email: ligon@berkeley.edu
+Requires-Python: >=3.11, <4.0
+Classifier: License :: Other/Proprietary License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3.15
+Requires-Dist: CFEDemands (>=0.10.0)
+Requires-Dist: PyYAML (>=6.0,<7.0)
+Requires-Dist: cryptography (>=3.4)
+Requires-Dist: dvc[s3] (>=3.67.0,<4.0.0)
+Requires-Dist: folium (>=0.19)
+Requires-Dist: fooddatacentral (>=1.0.10,<2.0.0)
+Requires-Dist: ligonlibrary (>=0.2.3)
+Requires-Dist: matplotlib (>=3.10.0,<4.0.0)
+Requires-Dist: openai (>=1.99.5,<2.0.0)
+Requires-Dist: pandas (>=3.0)
+Requires-Dist: pint (>=0.25.2,<0.26.0)
+Requires-Dist: platformdirs (>=4.0.0,<5.0.0)
+Requires-Dist: pyarrow (>=21.0.0)
+Requires-Dist: pyreadstat (>=1.3.0,<2.0.0)
+Requires-Dist: python-Levenshtein (>=0.27.3,<0.28.0)
+Requires-Dist: python-gnupg (>=0.5,<1.0)
+Requires-Dist: scikit-learn (>=1.7.0,<2.0.0)
+Requires-Dist: typer (>=0.12.0,<1.0.0)
+Project-URL: Repository, https://github.com/ligon/LSMS_Library
+Description-Content-Type: text/plain
+
+#+TITLE: LSMS_Library
+#+AUTHOR: Ethan Ligon
+#+OPTIONS: toc:nil
+
+[[https://doi.org/10.5281/zenodo.17258079][https://img.shields.io/badge/DOI-10.5281%2Fzenodo.17258079-blue.svg]]
+
+A Python library providing a uniform interface to Living Standards Measurement Study (LSMS) household surveys from multiple countries and years, without the data loss typical of traditional harmonization approaches.
+
+* The Problem
+
+LSMS datasets are invaluable for studying poverty, consumption, and household welfare across developing countries. However, each country's survey uses different:
+- Variable names and encodings
+- Food classification systems
+- Questionnaire structures
+- File formats and organization
+
+Researchers typically spend weeks learning each new dataset's idiosyncrasies or use pre-harmonized datasets that sacrifice detail and comparability. Cross-country or longitudinal analyses become prohibitively time-consuming.
+
+* The Solution
+
+LSMS_Library provides an *abstraction layer* that gives you a consistent interface to work with any supported LSMS dataset. Instead of harmonizing the data itself (which loses information), we harmonize the /way you access/ the data.
+
+* Installation
+
+From PyPI (when available):
+
+#+begin_src bash
+pip install LSMS_Library
+#+end_src
+
+From github (current release):
+
+#+begin_src bash
+pip install git+https://github.com/ligon/LSMS_Library.git@v0.7.0
+#+end_src
+
+From a source checkout (for contributors):
+
+#+begin_src bash
+git clone https://github.com/ligon/LSMS_Library.git
+cd LSMS_Library
+poetry install
+#+end_src
+
+* Quick Start
+
+#+begin_src python
+import lsms_library as ll
+
+# Single-country access
+uga = ll.Country('Uganda')
+uga.waves          # ['2005-06', '2009-10', ..., '2019-20']
+uga.data_scheme    # ['food_acquired', 'household_roster', ...]
+food = uga.food_expenditures()   # Standardized DataFrame, all waves
+
+# Cross-country analysis
+roster = ll.Feature('household_roster')
+roster.countries   # ['Burkina_Faso', 'Ethiopia', 'Mali', 'Uganda', ...]
+df = roster()      # Harmonized DataFrame across all countries
+#+end_src
+
+* Data Access
+
+This library abstracts over Living Standards Measurement Study (LSMS) survey data.  The underlying microdata belongs to the respective national statistics offices and the World Bank; users must accept the [[https://microdata.worldbank.org/][World Bank Microdata Library]]'s terms of use before accessing it.
+
+** Authentication: the WB Microdata API key
+
+1. Register at https://microdata.worldbank.org/ (free).
+2. Accept the terms of use for the LSMS collections you want to access.
+3. Get your API key from your account dashboard.
+4. Create =~/.config/lsms_library/config.yml= with:
+
+   #+begin_src yaml
+   microdata_api_key: your_key_here
+   # data_dir: /path/to/override   # same as LSMS_DATA_DIR env var
+   #+end_src
+
+   or set =MICRODATA_API_KEY= as an environment variable.
+5. On =import lsms_library=, the library validates your key against the WB catalog and automatically unlocks access to the S3 read cache.  No further setup is required.
+
+** The S3 cache
+
+Once your WB API key is validated, the library unlocks a read-only S3 cache that mirrors the WB Microdata downloads.  This is a convenience -- the S3 cache is faster than the WB NADA API and reduces load on the WB service -- but it is not a separate access layer.  The WB terms of use are the authoritative gate; the S3 cache just provides the same data faster.
+
+Decrypted plaintext credentials are written to =~/.config/lsms_library/s3_creds= (or the path in the =LSMS_S3_CREDS= environment variable), not into the package tree -- so the library is safe to install from a wheel into a read-only site-packages directory.
+
+*A valid WB API key is sufficient on its own.*  No other software is required: the unlock is pure Python, via =cryptography=, which installs with the library.  Before v0.10.1 it also needed the =gpg= binary -- not a Python package, and absent by default on Windows and macOS -- so a user with a perfectly good key hit =NoCredentialsError= from botocore with nothing pointing at the real cause (GH #741).
+
+If the unlock does fail, the library says so and falls back to downloading directly from the World Bank: slower, but nothing is blocked.  The =gpg= binary is now needed only for the interactive keyless fallback below, and even that prefers the pure-Python path.
+
+** Non-interactive environments
+
+In CI, Docker builds, or other non-interactive contexts, set =LSMS_SKIP_AUTH=1= to suppress the import-time authentication flow.  In that mode you are responsible for ensuring =~/.config/lsms_library/s3_creds= exists (e.g. via a CI secret mount) before the first data access.
+
+** Data cache location
+
+Parquet caches materialize under the platform-appropriate user data directory:
+
+- Linux: =~/.local/share/lsms_library/= by default
+- Override with =LSMS_DATA_DIR= env var or =data_dir= in =config.yml=
+
+See the [[https://ligon.github.io/LSMS_Library/guide/caching/][caching guide]] for details on =assume_cache_fresh=, the =Country= and =Feature= classes, and per-country build methods.
+
+* Documentation
+
+Full documentation is available at [[https://ligon.github.io/LSMS_Library][ligon.github.io/LSMS_Library]], including:
+
+- **[[https://ligon.github.io/LSMS_Library/getting-started/][Getting Started]]** -- installation and first steps
+- **[[https://ligon.github.io/LSMS_Library/guide/country/][Country Guide]]** -- single-country workflows, harmonization pipeline, derived tables
+- **[[https://ligon.github.io/LSMS_Library/guide/feature/][Feature Guide]]** -- cross-country analysis with =ll.Feature=
+- **[[https://ligon.github.io/LSMS_Library/guide/caching/][Caching]]** -- performance tuning, build backends, cache management
+- **[[https://ligon.github.io/LSMS_Library/guide/panel-data/][Panel Data]]** -- longitudinal analysis and ID harmonization
+- **[[https://ligon.github.io/LSMS_Library/guide/derivations/][Derived Values]]** -- the =Derivation= column: which served numbers are constructions from raw survey answers rather than survey answers, and how to get the raw answers back
+- **[[https://ligon.github.io/LSMS_Library/api/country/][API Reference]]** -- complete class documentation (auto-generated from source)
+
+* Contributing
+
+See [[file:CONTRIBUTING.org][CONTRIBUTING.org]] for detailed guidelines on adding new datasets using DVC.
+
+* Citation
+
+If you use LSMS_Library in your research, please cite:
+
+#+begin_src bibtex
+@software{ligon25:lsms_library,
+  author =    {Ethan Ligon},
+  title =     {{\tt LSMS_Library}: Abstraction layer for working with Living Standards Measurement Surveys},
+  year =      2025,
+  doi = {10.5281/zenodo.17258079},
+  url = {https://pypi.org/project/lsms_library/}
+}
+#+end_src
+
+* License
+
+See the [[file:LICENSE][LICENSE]] file in the repository for details.
+
+* Acknowledgments
+
+This project builds on data collection efforts by:
+- The World Bank's Living Standards Measurement Study (LSMS) team
+- National statistical offices in participating countries
+- The LSMS-ISA initiative
+

lsms_library-0.15.0/lsms_library/Makefile

@@ -0,0 +1,84 @@
+# The venv carries no editable install of this project (GH #957), and this
+# Makefile is a documented entry point in its own right ("make -C lsms_library
+# help").  Its recipes run `python3 -m lsms_library.*` and `python3
+# ../bench/feature_audit/scan.py`, both of which put THIS directory (or the
+# script's) at sys.path[0] -- never the repo root -- so without this they raise
+# ModuleNotFoundError.  Derived from MAKEFILE_LIST, not $(CURDIR), so it is
+# correct under `make -f` and `make -C` alike.
+REPO_ROOT := $(abspath $(dir $(firstword $(MAKEFILE_LIST)))/..)
+export PYTHONPATH := $(REPO_ROOT)$(if $(PYTHONPATH),:$(PYTHONPATH),)
+
+##
+# LSMS_Library
+#
+
+.SILENT: help
+
+empty :=
+space := $(empty) $(empty)
+
+PYTHON ?= python3
+DVC ?= $(PYTHON) -m dvc
+format ?= parquet
+
+slug = $(shell printf '%s' "$(1)" | tr '[:upper:]' '[:lower:]' | tr ' -' '__')
+DEFAULT_WAVE_RAW := $(if $(strip $(wave)),$(firstword $(wave)),all_waves)
+DEFAULT_WAVE := $(subst $(space),_,$(strip $(DEFAULT_WAVE_RAW)))
+OUT_FILE ?= countries/build/$(country)/$(DEFAULT_WAVE)/$(table).$(format)
+stage_name = $(call slug,$(country))_$(call slug,$(DEFAULT_WAVE_RAW))_$(call slug,$(table))
+
+
+COUNTRIES := $(shell ls .. | grep -e ^[A-Z] | grep / | sed -e 's/\//,/g')
+
+help:
+	echo "USAGE:"
+	echo "Either <test> or <build> datasets, <materialize>/<persist> tables, <register> stages, or estimate <demands>"
+	echo "for a particular country using, e.g.,"
+	echo "	make test country=Malawi"
+	echo "or"
+	echo "	make build country=Malawi"
+	echo "or"
+	echo "	make demands country=Malawi market=Region"
+	echo "or"
+	echo "	make materialize country=Malawi wave=2019-20 table=household_roster"
+	echo "or"
+	echo "	make persist country=Malawi wave=2019-20 table=household_roster"
+	echo "or"
+	echo "	make register country=Uganda wave=2023-24 table=household_roster"
+	echo ""
+	echo "where country is one of:"
+	echo $(COUNTRIES)
+
+
+build:
+	echo $(country)
+	+make -C ../$(country)/_
+
+test:
+	$(PYTHON) ../bench/feature_audit/scan.py --countries $(country) --out /tmp/feature_audit_$(call slug,$(country)).jsonl
+
+materialize:
+	$(PYTHON) -m lsms_library.cli materialize \
+		$(if $(strip $(wave)),$(foreach w,$(wave),--wave $(w) ),$(if $(strip $(all_waves)),--all-waves ,))\
+		--country $(country) \
+		--table $(table) \
+		--format $(format) \
+		--out $(OUT_FILE)
+
+register:
+	$(PYTHON) -m lsms_library.cli register \
+		--country $(country) \
+		--wave $(wave) \
+		--table $(table) \
+		--format $(format)
+
+persist:
+	cd countries && $(DVC) repro materialize@$(stage_name)
+
+demands:
+	$(PYTHON) -m lsms_library.demands $(country) $(if $(market),--market $(market),)
+
+clean:
+	+make -C ../$(country)/_ clean
+
+# end

lsms_library-0.15.0/lsms_library/__init__.py

@@ -0,0 +1,344 @@
+"""LSMS Library: a harmonized interface to Living Standards Measurement Study data.
+
+The library provides a uniform API across 40+ LSMS-ISA and related
+household surveys from the World Bank Microdata Library.  Its design
+principle is to harmonize the *interface*, not the data: surveys differ
+in structure, and the library preserves survey-specific detail while
+offering a common way to read it.
+
+Typical usage
+-------------
+Instantiate a :class:`Country` and call its table methods as attributes.
+Table names (``food_acquired``, ``household_roster``, ``shocks``, ...)
+come from the country's ``data_scheme.yml``.
+
+.. code-block:: python
+
+    import lsms_library as ll
+
+    uga = ll.Country('Uganda')
+    uga.waves              # ['2005-06', '2009-10', ...]
+    uga.data_scheme        # ['household_roster', 'food_acquired', ...]
+
+    roster = uga.household_roster()    # harmonized DataFrame across waves
+    food   = uga.food_expenditures()   # derived from food_acquired
+    chars  = uga.household_characteristics()  # derived from household_roster
+
+:class:`Feature` assembles the same table across every country that
+declares it, with a ``country`` index level prepended:
+
+.. code-block:: python
+
+    df = ll.Feature('household_roster')()               # every country
+    df = ll.Feature('shocks')(['Ethiopia', 'Niger'])    # subset
+
+Returned DataFrames are indexed by some subset of ``(country, t, v, i,
+pid, j)`` — wave (``t``), cluster (``v``), household (``i``), person
+(``pid``), item (``j``) — depending on the table.
+
+Public API
+----------
+- :class:`Country` — top-level per-country interface. Exposes ``waves``,
+  ``data_scheme``, and one method per table in the scheme. Derived
+  tables (``household_characteristics``, ``food_expenditures``,
+  ``food_prices``, ``food_quantities``) are dispatched via
+  ``__getattr__`` without needing to be in ``data_scheme.yml``.
+- :class:`Wave` — single-wave view of a country, used internally by
+  :class:`Country` but occasionally handy for debugging.
+- :class:`Feature` — cross-country aggregator for a given table name.
+- ``tools`` (alias for :mod:`lsms_library.local_tools`) — ``get_dataframe``
+  for reading ``.dta``/``.csv``/``.parquet`` with local → DVC → WB NADA
+  fallback, ``to_parquet`` for writing caches under ``data_root()``, and
+  the ``df_data_grabber`` helper used by wave-level scripts.
+- :func:`authenticate` — interactive DVC/S3 credential unlock. Usually
+  runs automatically on import when a World Bank Microdata API key is
+  available.
+- :mod:`lsms_library.transformations` — derivation helpers for
+  roster-to-characteristics and food-to-expenditures.
+
+Data access and caching
+-----------------------
+Reads use a fallback chain: local file on disk → DVC filesystem → World
+Bank NADA download. Parquet caches are written under ``data_root()``,
+which defaults to ``~/.local/share/lsms_library/{Country}/`` and can be
+overridden via ``LSMS_DATA_DIR`` or the ``data_dir`` key in
+``~/.config/lsms_library/config.yml``.
+
+As of v0.7.0, :meth:`Country` methods check the parquet cache before
+consulting DVC, giving 10–17× cross-process speedups. The cache has no
+automatic staleness check — set ``LSMS_NO_CACHE=1`` or run
+``lsms-library cache clear --country {Country}`` after editing source
+configs or scripts.
+
+Environment variables
+---------------------
+- ``MICRODATA_API_KEY`` — World Bank Microdata Library API key (also
+  readable from ``~/.config/lsms_library/config.yml``). Required for
+  direct downloads; also unlocks the S3 read cache at import time.
+- ``LSMS_DATA_DIR`` — override the parquet cache root.
+- ``LSMS_NO_CACHE`` — disable the v0.7.0 top-of-function cache read.
+- ``LSMS_BUILD_BACKEND=make`` — force rebuild from source, bypassing
+  both the cache and the DVC stage layer.
+- ``LSMS_SKIP_AUTH`` — suppress the import-time authentication attempt.
+- ``LSMS_BUILD_WORKERS`` — worker processes for a cold wave-parallel build
+  (default: the cgroup-visible CPU count, capped at the number of distinct
+  build targets and by the memory guard below; ``1`` = the serial path).
+  Waves that share a make target build serially in one worker; output is
+  byte-identical to serial.  An explicit value may exceed the memory guard
+  (warns once) but never the CPU count.
+- ``LSMS_BUILD_WORKER_MEM_GB`` — GiB reserved per build worker (default
+  ``4``, covering the largest wave script measured at 3.7 GB).  The default
+  worker count is capped at ``visible memory // this``, where visible memory
+  is the smallest of MemAvailable, the cgroup memory limit and Slurm's
+  ``SLURM_MEM_PER_NODE`` / ``SLURM_MEM_PER_CPU`` allocation.
+- ``LSMS_MAKE_JOBS`` — make ``-j`` parallelism for source rebuilds
+  (default: half the cgroup-visible CPUs -- ``sched_getaffinity`` capped by
+  ``SLURM_CPUS_PER_TASK`` / ``SLURM_CPUS_ON_NODE``, GH #764).  A country
+  build runs one wave build per job, i.e. that many concurrent large S3
+  fetches; set ``LSMS_MAKE_JOBS=1`` to serialize them on hosts where
+  concurrent multipart reads are flaky.  Under ``LSMS_BUILD_WORKERS`` it
+  is a ceiling on each worker's ``-j``.
+- ``LSMS_FETCH_ATTEMPTS`` — retry budget (default ``3``) for a single
+  blob's S3 fetch on a transient TLS error before falling back to streaming.
+
+Further reading
+---------------
+- ``CLAUDE.md`` at the repository root for contributor-facing design
+  notes, the canonical schema (``lsms_library/data_info.yml``), and the
+  list of cross-cutting gotchas.
+- ``.claude/skills/add-feature/`` for adding a new table to a country,
+  with sub-skills for ``sample``, ``shocks``, ``assets``,
+  ``panel-ids``, ``food-acquired``, and ``pp-ph`` (post-planting /
+  post-harvest countries).
+- ``CONTRIBUTING.org`` for the new-wave workflow.
+
+Data terms of use
+-----------------
+Underlying microdata must be obtained from the World Bank Microdata
+Library (https://microdata.worldbank.org/) under their terms of use.
+The library's S3 bucket is a read cache over the authoritative WB NADA
+downloads, unlocked automatically when a valid API key is present.
+"""
+
+from importlib.metadata import PackageNotFoundError
+from importlib.metadata import version as _get_version
+
+try:
+    __version__ = _get_version("LSMS_Library")
+except PackageNotFoundError:
+    __version__ = "0.0.0"
+del _get_version
+
+from pathlib import Path
+import os
+import warnings
+
+import logging
+from sys import stderr
+from . import country
+from .country import Country, Wave
+from .feature import Feature
+from .catalog import countries, features
+# *** `countries` IS THE THIRD SHADOW, AND IT POINTS THE OTHER WAY.  GH #883. ***
+#
+# `recall` and `derivations` (documented at the rebind site below) are shadows
+# the package WINS: both submodules are imported eagerly during `__init__`, so
+# the callable is bound last and stays bound.  `countries` is the inverse and
+# is strictly worse, because nothing here can lose a race it does not run in:
+# `lsms_library/countries/` is a namespace package imported LAZILY, so the
+# import system rebinds this attribute LONG AFTER `__init__` has finished --
+#
+#     import lsms_library as ll
+#     ll.countries          # -> the function
+#     from lsms_library.countries.GhanaLSS._.ghanalss import derive_...
+#     ll.countries          # -> the MODULE.  TypeError on the next call.
+#
+# That is reachable from ordinary use, not just from inside the package:
+# `DerivationRecord.resolve()` imports whatever module the registry names, and
+# most registered derivations live in `lsms_library.countries.*`, so resolving
+# one silently breaks `ll.countries()` for the rest of the session.  It is also
+# invisible to `tests/test_module_shadowing.py::discover_shadowed`, which reads
+# the attribute at test time -- before any such import has happened -- and so
+# reports a clean bill right up until something triggers it.
+#
+# A `property` is a DATA descriptor, and a data descriptor on the type beats an
+# entry in the instance `__dict__`.  Binding one on this module's own class
+# therefore makes the import system's `setattr(lsms_library, "countries", mod)`
+# a no-op, and the public callable wins permanently -- the same outcome #883
+# chose for `recall`/`derivations`, reached the only way available when the
+# competing binding happens after we have stopped running.
+#
+# `sys.modules["lsms_library.countries"]` is untouched, so every import form
+# still resolves: `import lsms_library.countries`,
+# `from lsms_library.countries.X.Y import Z`, `importlib.import_module(...)`.
+# Only the ATTRIBUTE is pinned.
+def _pin_countries_callable() -> None:
+    import sys as _sys
+    _module = _sys.modules[__name__]
+    _callable = countries
+
+    class _Package(type(_module)):
+        @property
+        def countries(self):
+            return _callable
+
+        @countries.setter
+        def countries(self, _value):
+            # The import system binds the submodule here on first import of
+            # `lsms_library.countries`.  Swallow it: `sys.modules` already
+            # holds the module, which is what every import form actually reads.
+            pass
+
+    _module.__class__ = _Package
+
+
+_pin_countries_callable()
+from .coverage_matrix import coverage
+from .currency import currency_for
+from .conversion import convert
+from . import local_tools as tools
+from . import population
+from .population import PopulationRecord, population_records
+# *** THESE TWO LINES SHADOW A SUBMODULE NAME.  GH #883. ***
+#
+# `ll.recall` and `ll.derivations` are the public TABLE FUNCTIONS, chosen
+# deliberately -- but `lsms_library.recall` and `lsms_library.derivations` are
+# also submodules.  Once the callable is bound here, the package ATTRIBUTE wins
+# over the submodule for `from <package> import <name>`, which prefers an
+# existing attribute.  So:
+#
+#     from lsms_library import recall          # -> the FUNCTION, not the module
+#     from lsms_library import recall as r; r.recall_records
+#                                             # -> AttributeError
+#
+# and that bites CALLERS OUTSIDE THE PACKAGE too, not only modules inside it --
+# it is how the trap was hit a second time, from a one-off script.  The forms
+# that always give you the module:
+#
+#     from lsms_library.recall import recall_records, BASIS_LADDER   # by name
+#     importlib.import_module("lsms_library.recall")                 # the module
+#
+# `sys.modules["lsms_library.recall"]` is still the module either way; only the
+# attribute is rebound.  INSIDE this package, never write
+# `from . import recall` / `from . import derivations` -- import the symbols by
+# name instead (`feature.py` does, and survived only because it is imported
+# above this line).  tests/test_module_shadowing.py pins all of it: the set of
+# shadowed names, the in-package ban, and the escape hatches.
+#
+# Renaming either the public callables or the submodules would remove the trap;
+# both were declined -- the public names are the documented API.
+from . import recall as _recall_mod
+from .recall import RecallRecord, recall_records
+from .recall_table import recall
+from . import derivations as _derivations_mod
+from .derivations import DerivationRecord, derivation_records
+from .derivations import derivations_table as derivations
+from . import transformations
+from . import visualizations
+from .visualizations import population_pyramid, coordinate_map, lorenz_curve
+from .dvc_permissions import authenticate
+try:
+    from dvc.ui import ui as dvc_ui
+    from dvc.logger import LoggerHandler, setup as dvc_log_setup
+    from functools import wraps
+    import sys
+
+    if hasattr(dvc_ui, "rich_console"):
+        try:
+            dvc_ui.rich_console.file = stderr
+        except (AttributeError, ImportError):
+            # Defensive against DVC rich-console API drift; programmer bugs
+            # (TypeError, NameError) propagate.
+            pass
+    if hasattr(dvc_ui, "error_console"):
+        try:
+            dvc_ui.error_console.file = stderr
+        except (AttributeError, ImportError):
+            # Defensive against DVC rich-console API drift.
+            pass
+    original_init = LoggerHandler.__init__
+
+    @wraps(original_init)
+    def _patched_logger_handler_init(self, stream, *args, **kwargs):
+        if stream is sys.stdout:
+            stream = stderr
+        original_init(self, stream, *args, **kwargs)
+
+    LoggerHandler.__init__ = _patched_logger_handler_init  # type: ignore[assignment]
+
+    dvc_log_setup()
+    for logger_name in ("dvc", "dvc_objects", "dvc_data", ""):
+        logger_obj = logging.getLogger(logger_name)
+        for handler in logger_obj.handlers:
+            if isinstance(handler, LoggerHandler) and getattr(handler, "stream", None) is not stderr:
+                handler.stream = stderr
+except (ImportError, AttributeError):
+    # DVC optional or its UI module changed shape; proceed without
+    # UI tweaks.  Programmer bugs surface unchanged.
+    pass
+
+from .config import s3_creds_path as _s3_creds_path
+creds_file = _s3_creds_path()
+
+SKIP_AUTH = os.getenv("LSMS_SKIP_AUTH", "").lower() in {"1", "true", "yes"}
+
+if not SKIP_AUTH and not creds_file.exists():
+    # Try non-interactive S3 unlock first.  This only decrypts the local
+    # s3_reader_creds.gpg — no network call required.  WB API key validation
+    # (which requires a network round-trip to microdata.worldbank.org) is
+    # deferred until the first actual data access via permissions() in
+    # data_access.py.  This keeps ``import lsms_library`` fast.
+    try:
+        from .data_access import _auto_unlock_s3 as _unlock
+        _unlocked = _unlock()
+        if not _unlocked:
+            # Local GPG decryption failed or no .gpg file — fall back to
+            # interactive passphrase prompt.
+            raise RuntimeError("auto-unlock did not produce S3 credentials")
+    except (ImportError, RuntimeError, OSError):
+        # Auto-unlock skipped (missing dep), failed (no .gpg / bad
+        # passphrase / gpg binary missing).  Fall through to interactive
+        # authenticate(); programmer bugs propagate.
+        try:
+            authenticate()
+        except (RuntimeError, OSError) as exc:
+            # Auth flow failure (network, subprocess, missing creds).
+            # Warn but don't crash import.
+            warnings.warn(
+                f"Automatic DVC authentication failed: {exc}. "
+                "Set LSMS_SKIP_AUTH=1 to suppress, or set "
+                "MICRODATA_API_KEY for non-interactive access."
+            )
+elif not SKIP_AUTH and creds_file.exists():
+    # User-config creds are already in place, so the auto-unlock branch
+    # above is short-circuited.  We still need to populate the legacy
+    # in-tree ``lsms_library/countries/.dvc/s3_creds`` path so that
+    # legacy wave scripts using ``dvc.api.open()`` (which don't pass a
+    # ``credentialpath`` override) can find credentials.  Without this,
+    # a fresh clone with valid user-config creds still hits
+    # ``NoCredentialsError`` on Uganda's earnings, enterprise_income,
+    # food_acquired etc. via the legacy DVC API path.
+    try:
+        from .data_access import _sync_legacy_dvc_creds as _sync
+        _sync()
+    except (ImportError, OSError):
+        # Sync failure is non-fatal --- legacy `dvc.api.open()` callers
+        # will surface a clear NoCredentialsError if it matters.
+        pass
+
+if not SKIP_AUTH:
+    # The DVCFS singleton is built when `local_tools` is imported, which is
+    # before either branch above can have written the credentials file.  On a
+    # first-ever import it was therefore built without explicit S3
+    # credentials, leaving it on the `credentialpath` fallback that an
+    # ambient AWS environment silently defeats (see
+    # `local_tools._s3_explicit_credentials`).  Rebuild it once, now that
+    # the file exists.  No-op when the credentials were already present at
+    # import time, which is the common path.
+    try:
+        from .local_tools import refresh_s3_credentials as _refresh_s3
+        _refresh_s3()
+    except (ImportError, OSError, AttributeError):
+        # Best-effort: a failure here leaves the credentialpath fallback,
+        # which is what shipped before.
+        pass