usdata 0.4.0__tar.gz → 0.6.0__tar.gz

{usdata-0.4.0 → usdata-0.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: usdata
-Version: 0.4.0
+Version: 0.6.0
 Summary: Unified Python SDK and CLI for discovering, fetching, and tracking provenance of U.S. public scientific data
 Keywords: noaa,usgs,nasa,open-data,scientific-data,provenance
 Author: Jake Van Slyke
@@ -14,9 +14,11 @@ Requires-Dist: httpx>=0.28.1
 Requires-Dist: pydantic>=2.7
 Requires-Dist: pyyaml>=6.0
 Requires-Dist: typer>=0.12
+Requires-Dist: pandas>=3.0 ; extra == 'pandas'
 Requires-Python: >=3.11
 Project-URL: Homepage, https://github.com/jakeryderv/usdata
 Project-URL: Repository, https://github.com/jakeryderv/usdata
+Provides-Extra: pandas
 Description-Content-Type: text/markdown
 
 # usdata
@@ -24,24 +26,25 @@ Description-Content-Type: text/markdown
 Unified Python SDK and CLI for discovering, fetching, and tracking the
 provenance of U.S. public scientific data (NOAA, USGS, NASA, and more).
 
-> Status: pre-alpha. `noaa:ghcn-daily` and `noaa:nexrad-level2` fetch real
-> data with provenance; other registry entries are stubs.
+> Status: pre-alpha. v0.6 supports GHCN-Daily, NEXRAD Level II, USGS daily
+> values, and CoastWatch SST subsets with provenance, plus Census state/county
+> lookup and optional pandas CSV readers. Other datasets are planned.
 > See [docs/roadmap.md](docs/roadmap.md).
 
 ## Providers
 
 <!-- registry:start -->
-| Provider | Available | Stub | Planned | Next up (0.5) | Datasets |
+| Provider | Available | Stub | Planned | Next up (unassigned) | Datasets |
 |---|---:|---:|---:|---|---|
-| [NOAA](docs/providers/noaa.md) | 2 | 1 | 26 | `gsom`, `gsoy`, `storm-events`, `mrms`, `goes-abi`, `hurdat2`, `ibtracs`, `climate-normals`, `coops-water-levels`, `coastwatch-sst` | `ghcn-daily`, `nexrad-level2`, _coastwatch-sst_, +26 planned |
+| [NOAA](docs/providers/noaa.md) | 3 | 0 | 26 | — | `ghcn-daily`, `nexrad-level2`, `coastwatch-sst`, +26 planned |
+| [USGS](docs/providers/usgs.md) | 1 | 0 | 2 | — | `water-daily`, +2 planned |
 | [Census Bureau](docs/providers/census.md) | 0 | 0 | 1 | — | +1 planned |
 | [EPA](docs/providers/epa.md) | 0 | 0 | 1 | — | +1 planned |
 | [FEMA](docs/providers/fema.md) | 0 | 0 | 1 | — | +1 planned |
 | [NASA](docs/providers/nasa.md) | 0 | 0 | 1 | — | +1 planned |
 | [USDA](docs/providers/usda.md) | 0 | 0 | 1 | — | +1 planned |
-| [USGS](docs/providers/usgs.md) | 0 | 0 | 3 | `water-daily` | +3 planned |
 
-Available datasets are in `code`, stubs in _italics_; planned ones are counted. Each provider page has access notes and full dataset details; [docs/roadmap.md](docs/roadmap.md) lists datasets by target version.
+Available datasets are in `code`, stubs in _italics_; planned ones are counted. Available means implemented in this source checkout; consult the [releases](https://github.com/jakeryderv/usdata/releases) for published support. Each provider page has access notes and full dataset details; [docs/roadmap.md](docs/roadmap.md) lists datasets by target version.
 <!-- registry:end -->
 
 ## Install
@@ -74,6 +77,7 @@ for item in fetch(ds, query):
 
 ```sh
 usdata search "tornado radar" --state OK
+usdata search precipitation --location "Cleveland County, OK"
 usdata info noaa:ghcn-daily
 usdata fetch noaa:ghcn-daily --lat 35.39 --lon -97.60 --radius-km 15 \
     --start 2024-05-06 --end 2024-05-07 --vars PRCP,TMAX
@@ -81,6 +85,10 @@ usdata fetch noaa:ghcn-daily -p stations=USW00013967 --start 2024-01-01 --end 20
 usdata fetch noaa:nexrad-level2 --lat 35.47 --lon -97.52 \
     --start 2024-05-06T20:00 --end 2024-05-06T23:00        # nearest radar (KTLX)
 usdata fetch noaa:nexrad-level2 -p site=KTLX --start 2024-05-06T20:00 --end 2024-05-06T20:30 --dry-run
+usdata fetch usgs:water-daily -p sites=07164500 --vars 00060 \
+    --start 2024-05-06 --end 2024-05-07
+usdata fetch noaa:coastwatch-sst --bbox=-80.08,30.02,-80.02,30.08 \
+    --start 2024-05-06T12:00Z --end 2024-05-06T12:00Z    # four grid cells
 usdata pull dataset.yaml            # resolve, fetch, write dataset.lock.json
 usdata verify dataset.yaml          # exit 1 if any cached input drifted
 ```
@@ -89,6 +97,11 @@ Fetched files land in `~/.cache/usdata/<provider>/<dataset>/` (override with
 `USDATA_CACHE_DIR` or `--cache-dir`), each with a `.provenance.json` sidecar
 recording source URL, retrieval time, checksum, size, and license.
 
+Locations accept state names/postal codes, county/state names, and quoted FIPS
+codes. These select bounding rectangles; see [place lookup](docs/reference/places.md)
+for coverage, ambiguity, and antimeridian limits. CoastWatch CSV includes a
+second header row containing units; see its [access notes](docs/providers/noaa.md#coastwatch-sst).
+
 Manifest and source fields are validated strictly; unknown fields are errors.
 Provider-specific options belong under `params`.
 
@@ -96,8 +109,15 @@ A manifest declares every input a project needs. `pull` resolves each source,
 fetches it, and writes `dataset.lock.json` pinning every asset with its checksum
 and provenance. A second `pull` restores exactly what the lockfile pins without
 re-querying upstream, so the inputs stay reproducible even if the source
-changes. `verify` re-hashes the cached files against the lockfile. Editing the
-manifest after locking requires `pull --force` to re-resolve.
+changes. `verify` checks the manifest checksum and re-hashes cached files
+against the lockfile. Editing the manifest after locking requires `pull --force`
+to re-resolve. A required source matching no assets fails the pull; set
+`allow_empty: true` on a source only when an empty result is intentional.
+
+Checksums detect upstream changes; they cannot recover historical bytes that
+are no longer available. Preserve the cache for long-lived reproducibility.
+See the [manifest reference](docs/reference/manifests.md) and the small
+[NOAA/USGS example](examples/weather-and-streamflow/README.md).
 
 ```yaml
 name: tornado-environment
@@ -112,6 +132,14 @@ sources:
     end: 2024-05-31
 ```
 
+## Opening CSV data
+
+`FetchedAsset.open()` is available since v0.6 with the optional pandas
+extra (`pip install "usdata[pandas]"`). It reads cached CSV into a DataFrame,
+preserves identifier strings, and keeps CoastWatch units as metadata.
+See the [reader reference](docs/reference/readers.md)
+and [fetch → open → analyze example](examples/sst-analysis/README.md).
+
 ## Development
 
 Requires [uv](https://docs.astral.sh/uv/) and [just](https://just.systems/).
@@ -120,11 +148,18 @@ Requires [uv](https://docs.astral.sh/uv/) and [just](https://just.systems/).
 git clone https://github.com/jakeryderv/usdata && cd usdata
 just setup     # install toolchain and dependencies
 just test      # unit tests
-just check     # format, lint, typecheck, tests (what CI runs)
+just check     # format, lint, typecheck, offline tests, generated docs, release notices
+just check-pandas  # install the CSV extra and run the same checks
+just build     # build wheel and sdist
+just smoke     # exercise core and pandas wheel installations outside the checkout
 just run search radar
 ```
 
-Integration tests that hit live services run with `just test-integration`.
+Unit tests mechanically block network connections. Integration tests that hit
+live services run with `just test-integration`. CI checks Python 3.11 and 3.14 on
+Linux, both with and without pandas, and smoke-tests both installed-wheel
+profiles on Linux, macOS, and Windows. The full unit and live-service suites currently run on Linux. `just setup` restores
+a core-only development environment; `just check-pandas` installs the extra.
 
 Releases: `just release minor` opens a version-bump PR; merging it publishes
 to PyPI and creates the tag and GitHub release. See

{usdata-0.4.0 → usdata-0.6.0}/README.md

@@ -3,24 +3,25 @@
 Unified Python SDK and CLI for discovering, fetching, and tracking the
 provenance of U.S. public scientific data (NOAA, USGS, NASA, and more).
 
-> Status: pre-alpha. `noaa:ghcn-daily` and `noaa:nexrad-level2` fetch real
-> data with provenance; other registry entries are stubs.
+> Status: pre-alpha. v0.6 supports GHCN-Daily, NEXRAD Level II, USGS daily
+> values, and CoastWatch SST subsets with provenance, plus Census state/county
+> lookup and optional pandas CSV readers. Other datasets are planned.
 > See [docs/roadmap.md](docs/roadmap.md).
 
 ## Providers
 
 <!-- registry:start -->
-| Provider | Available | Stub | Planned | Next up (0.5) | Datasets |
+| Provider | Available | Stub | Planned | Next up (unassigned) | Datasets |
 |---|---:|---:|---:|---|---|
-| [NOAA](docs/providers/noaa.md) | 2 | 1 | 26 | `gsom`, `gsoy`, `storm-events`, `mrms`, `goes-abi`, `hurdat2`, `ibtracs`, `climate-normals`, `coops-water-levels`, `coastwatch-sst` | `ghcn-daily`, `nexrad-level2`, _coastwatch-sst_, +26 planned |
+| [NOAA](docs/providers/noaa.md) | 3 | 0 | 26 | — | `ghcn-daily`, `nexrad-level2`, `coastwatch-sst`, +26 planned |
+| [USGS](docs/providers/usgs.md) | 1 | 0 | 2 | — | `water-daily`, +2 planned |
 | [Census Bureau](docs/providers/census.md) | 0 | 0 | 1 | — | +1 planned |
 | [EPA](docs/providers/epa.md) | 0 | 0 | 1 | — | +1 planned |
 | [FEMA](docs/providers/fema.md) | 0 | 0 | 1 | — | +1 planned |
 | [NASA](docs/providers/nasa.md) | 0 | 0 | 1 | — | +1 planned |
 | [USDA](docs/providers/usda.md) | 0 | 0 | 1 | — | +1 planned |
-| [USGS](docs/providers/usgs.md) | 0 | 0 | 3 | `water-daily` | +3 planned |
 
-Available datasets are in `code`, stubs in _italics_; planned ones are counted. Each provider page has access notes and full dataset details; [docs/roadmap.md](docs/roadmap.md) lists datasets by target version.
+Available datasets are in `code`, stubs in _italics_; planned ones are counted. Available means implemented in this source checkout; consult the [releases](https://github.com/jakeryderv/usdata/releases) for published support. Each provider page has access notes and full dataset details; [docs/roadmap.md](docs/roadmap.md) lists datasets by target version.
 <!-- registry:end -->
 
 ## Install
@@ -53,6 +54,7 @@ for item in fetch(ds, query):
 
 ```sh
 usdata search "tornado radar" --state OK
+usdata search precipitation --location "Cleveland County, OK"
 usdata info noaa:ghcn-daily
 usdata fetch noaa:ghcn-daily --lat 35.39 --lon -97.60 --radius-km 15 \
     --start 2024-05-06 --end 2024-05-07 --vars PRCP,TMAX
@@ -60,6 +62,10 @@ usdata fetch noaa:ghcn-daily -p stations=USW00013967 --start 2024-01-01 --end 20
 usdata fetch noaa:nexrad-level2 --lat 35.47 --lon -97.52 \
     --start 2024-05-06T20:00 --end 2024-05-06T23:00        # nearest radar (KTLX)
 usdata fetch noaa:nexrad-level2 -p site=KTLX --start 2024-05-06T20:00 --end 2024-05-06T20:30 --dry-run
+usdata fetch usgs:water-daily -p sites=07164500 --vars 00060 \
+    --start 2024-05-06 --end 2024-05-07
+usdata fetch noaa:coastwatch-sst --bbox=-80.08,30.02,-80.02,30.08 \
+    --start 2024-05-06T12:00Z --end 2024-05-06T12:00Z    # four grid cells
 usdata pull dataset.yaml            # resolve, fetch, write dataset.lock.json
 usdata verify dataset.yaml          # exit 1 if any cached input drifted
 ```
@@ -68,6 +74,11 @@ Fetched files land in `~/.cache/usdata/<provider>/<dataset>/` (override with
 `USDATA_CACHE_DIR` or `--cache-dir`), each with a `.provenance.json` sidecar
 recording source URL, retrieval time, checksum, size, and license.
 
+Locations accept state names/postal codes, county/state names, and quoted FIPS
+codes. These select bounding rectangles; see [place lookup](docs/reference/places.md)
+for coverage, ambiguity, and antimeridian limits. CoastWatch CSV includes a
+second header row containing units; see its [access notes](docs/providers/noaa.md#coastwatch-sst).
+
 Manifest and source fields are validated strictly; unknown fields are errors.
 Provider-specific options belong under `params`.
 
@@ -75,8 +86,15 @@ A manifest declares every input a project needs. `pull` resolves each source,
 fetches it, and writes `dataset.lock.json` pinning every asset with its checksum
 and provenance. A second `pull` restores exactly what the lockfile pins without
 re-querying upstream, so the inputs stay reproducible even if the source
-changes. `verify` re-hashes the cached files against the lockfile. Editing the
-manifest after locking requires `pull --force` to re-resolve.
+changes. `verify` checks the manifest checksum and re-hashes cached files
+against the lockfile. Editing the manifest after locking requires `pull --force`
+to re-resolve. A required source matching no assets fails the pull; set
+`allow_empty: true` on a source only when an empty result is intentional.
+
+Checksums detect upstream changes; they cannot recover historical bytes that
+are no longer available. Preserve the cache for long-lived reproducibility.
+See the [manifest reference](docs/reference/manifests.md) and the small
+[NOAA/USGS example](examples/weather-and-streamflow/README.md).
 
 ```yaml
 name: tornado-environment
@@ -91,6 +109,14 @@ sources:
     end: 2024-05-31
 ```
 
+## Opening CSV data
+
+`FetchedAsset.open()` is available since v0.6 with the optional pandas
+extra (`pip install "usdata[pandas]"`). It reads cached CSV into a DataFrame,
+preserves identifier strings, and keeps CoastWatch units as metadata.
+See the [reader reference](docs/reference/readers.md)
+and [fetch → open → analyze example](examples/sst-analysis/README.md).
+
 ## Development
 
 Requires [uv](https://docs.astral.sh/uv/) and [just](https://just.systems/).
@@ -99,11 +125,18 @@ Requires [uv](https://docs.astral.sh/uv/) and [just](https://just.systems/).
 git clone https://github.com/jakeryderv/usdata && cd usdata
 just setup     # install toolchain and dependencies
 just test      # unit tests
-just check     # format, lint, typecheck, tests (what CI runs)
+just check     # format, lint, typecheck, offline tests, generated docs, release notices
+just check-pandas  # install the CSV extra and run the same checks
+just build     # build wheel and sdist
+just smoke     # exercise core and pandas wheel installations outside the checkout
 just run search radar
 ```
 
-Integration tests that hit live services run with `just test-integration`.
+Unit tests mechanically block network connections. Integration tests that hit
+live services run with `just test-integration`. CI checks Python 3.11 and 3.14 on
+Linux, both with and without pandas, and smoke-tests both installed-wheel
+profiles on Linux, macOS, and Windows. The full unit and live-service suites currently run on Linux. `just setup` restores
+a core-only development environment; `just check-pandas` installs the extra.
 
 Releases: `just release minor` opens a version-bump PR; merging it publishes
 to PyPI and creates the tag and GitHub release. See

{usdata-0.4.0 → usdata-0.6.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "usdata"
-version = "0.4.0"
+version = "0.6.0"
 description = "Unified Python SDK and CLI for discovering, fetching, and tracking provenance of U.S. public scientific data"
 readme = "README.md"
 license = "Apache-2.0"
@@ -30,6 +30,9 @@ dependencies = [
 name = "Jake Van Slyke"
 email = "jakervanslyke@gmail.com"
 
+[project.optional-dependencies]
+pandas = ["pandas>=3.0"]
+
 [project.urls]
 Homepage = "https://github.com/jakeryderv/usdata"
 Repository = "https://github.com/jakeryderv/usdata"

{usdata-0.4.0 → usdata-0.6.0}/pyproject.toml.orig

@@ -1,6 +1,6 @@
 [project]
 name = "usdata"
-version = "0.4.0"
+version = "0.6.0"
 description = "Unified Python SDK and CLI for discovering, fetching, and tracking provenance of U.S. public scientific data"
 readme = "README.md"
 license = "Apache-2.0"
@@ -22,6 +22,9 @@ dependencies = [
     "typer>=0.12",
 ]
 
+[project.optional-dependencies]
+pandas = ["pandas>=3.0"]
+
 [project.urls]
 Homepage = "https://github.com/jakeryderv/usdata"
 Repository = "https://github.com/jakeryderv/usdata"

{usdata-0.4.0 → usdata-0.6.0}/src/usdata/cli/app.py

@@ -14,7 +14,7 @@ from usdata.fetch import fetch as fetch_query
 from usdata.manifest import lockfile_path
 from usdata.providers import load_adapter
 from usdata.providers.base import NotImplementedProvider
-from usdata.pull import ManifestChanged, UnknownDatasets
+from usdata.pull import EmptySource, ManifestChanged, UnknownDatasets
 from usdata.pull import pull as pull_manifest
 from usdata.pull import verify as verify_manifest
 from usdata.query import UnknownPlace
@@ -50,7 +50,10 @@ def search(
     provider: Annotated[
         str | None, typer.Option(help="Restrict to one provider, e.g. noaa.")
     ] = None,
-    state: Annotated[str | None, typer.Option(help="State name or postal code.")] = None,
+    state: Annotated[
+        str | None,
+        typer.Option("--location", "--state", help="State, 'County, ST', or quoted FIPS code."),
+    ] = None,
     start: Annotated[str | None, typer.Option(help="ISO date or datetime.")] = None,
     end: Annotated[str | None, typer.Option(help="ISO date or datetime.")] = None,
     planned: Annotated[
@@ -104,7 +107,10 @@ def info(
 @app.command()
 def fetch(
     dataset_id: Annotated[str, typer.Argument(help="Dataset id, e.g. noaa:ghcn-daily")],
-    state: Annotated[str | None, typer.Option(help="State name or postal code.")] = None,
+    state: Annotated[
+        str | None,
+        typer.Option("--location", "--state", help="State, 'County, ST', or quoted FIPS code."),
+    ] = None,
     bbox: Annotated[str | None, typer.Option(help="west,south,east,north in degrees.")] = None,
     lat: Annotated[float | None, typer.Option()] = None,
     lon: Annotated[float | None, typer.Option()] = None,
@@ -190,6 +196,9 @@ def pull(
     """Fetch every source in a manifest and write (or restore from) its lockfile."""
     try:
         result = pull_manifest(manifest, root=cache_dir, force=force)
+    except EmptySource as e:
+        typer.secho(str(e), err=True, fg="yellow")
+        raise typer.Exit(code=1) from None
     except (DatasetNotFound, UnknownDatasets, ManifestChanged, UnknownPlace, ValueError) as e:
         typer.secho(str(e), err=True, fg="red")
         raise typer.Exit(code=2) from None
@@ -218,7 +227,7 @@ def verify(
         raise typer.Exit(code=2)
     try:
         drift = verify_manifest(manifest, root=cache_dir)
-    except (ValueError, OSError) as e:
+    except (ManifestChanged, ValueError, OSError) as e:
         typer.secho(str(e), err=True, fg="red")
         raise typer.Exit(code=2) from None
     for d in drift: