python-midas 0.1.1__tar.gz → 1.0.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (29) hide show
  1. python_midas-1.0.1/.github/workflows/ci.yml +32 -0
  2. python_midas-1.0.1/CHANGELOG.md +62 -0
  3. python_midas-1.0.1/CONTRIBUTING.md +85 -0
  4. python_midas-1.0.1/PKG-INFO +418 -0
  5. python_midas-1.0.1/README.md +387 -0
  6. python_midas-1.0.1/doc/v2-migration.md +56 -0
  7. {python_midas-0.1.1 → python_midas-1.0.1}/pyproject.toml +6 -3
  8. {python_midas-0.1.1 → python_midas-1.0.1}/src/midas/__init__.py +19 -6
  9. {python_midas-0.1.1 → python_midas-1.0.1}/src/midas/auth.py +1 -3
  10. {python_midas-0.1.1 → python_midas-1.0.1}/src/midas/client.py +48 -58
  11. python_midas-1.0.1/src/midas/entities/__init__.py +52 -0
  12. python_midas-1.0.1/src/midas/entities/models.py +261 -0
  13. python_midas-1.0.1/src/midas/enums.py +111 -0
  14. python_midas-1.0.1/src/midas/time.py +101 -0
  15. {python_midas-0.1.1 → python_midas-1.0.1}/tests/test_client.py +69 -85
  16. python_midas-1.0.1/tests/test_entities.py +359 -0
  17. python_midas-1.0.1/tests/test_integration.py +357 -0
  18. python_midas-0.1.1/PKG-INFO +0 -404
  19. python_midas-0.1.1/README.md +0 -374
  20. python_midas-0.1.1/src/midas/entities/__init__.py +0 -59
  21. python_midas-0.1.1/src/midas/entities/models.py +0 -231
  22. python_midas-0.1.1/src/midas/enums.py +0 -41
  23. python_midas-0.1.1/tests/test_entities.py +0 -262
  24. python_midas-0.1.1/tests/test_integration.py +0 -251
  25. {python_midas-0.1.1 → python_midas-1.0.1}/.github/workflows/publish.yml +0 -0
  26. {python_midas-0.1.1 → python_midas-1.0.1}/.gitignore +0 -0
  27. {python_midas-0.1.1 → python_midas-1.0.1}/LICENSE +0 -0
  28. {python_midas-0.1.1 → python_midas-1.0.1}/src/midas/py.typed +0 -0
  29. {python_midas-0.1.1 → python_midas-1.0.1}/tests/test_auth.py +0 -0
@@ -0,0 +1,32 @@
1
+ name: CI
2
+
3
+ on:
4
+ push:
5
+ branches: [main]
6
+ pull_request:
7
+
8
+ jobs:
9
+ lint:
10
+ runs-on: ubuntu-latest
11
+ steps:
12
+ - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
13
+ - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
14
+ with:
15
+ python-version: "3.12"
16
+ - run: pip install ruff
17
+ - run: ruff check .
18
+ - run: ruff format --check .
19
+
20
+ test:
21
+ runs-on: ubuntu-latest
22
+ strategy:
23
+ matrix:
24
+ python-version: ["3.10", "3.11", "3.12", "3.13"]
25
+ steps:
26
+ - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
27
+ - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
28
+ with:
29
+ python-version: ${{ matrix.python-version }}
30
+ - run: pip install -e ".[dev]"
31
+ # Integration tests hit the live MIDAS API; CI runs unit tests only.
32
+ - run: pytest tests/ -v -m "not integration"
@@ -0,0 +1,62 @@
1
+ # Changelog
2
+
3
+ All notable changes to this project will be documented in this file.
4
+
5
+ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html). While the library was in early development (0.x), breaking changes appeared between minor versions when needed to fix correctness issues or align with the MIDAS API. The 1.0.0 release tracks the California Energy Commission's MIDAS v2.0 API.
6
+
7
+ ## [1.0.1] - 2026-06-24
8
+
9
+ Patch release. Regenerating the spec examples against the live API surfaced two wire-shape details the lenient client silently tolerated; both are now handled and guarded by tests.
10
+
11
+ ### Fixed
12
+
13
+ - **Integer day-types are no longer dropped.** Live v2.0 SGIP GHG (MOER) and Flex Alert (ALRT) responses encode `DayStart` / `DayEnd` as integers (1=Monday through 7=Sunday, 8=Holiday: the upload-format code), where v1.0 returned weekday strings. `_parse_day_type` previously tried `DayType(value)` and returned `None` on the resulting `ValueError`, so every MOER/ALRT interval lost its `day_start` / `day_end`. `DayType.from_wire` now accepts the integer code, a digit string, or a weekday string; the electricity-rate wire form (unconfirmed pending utility-data migration) is covered by accepting both.
14
+
15
+ ### Added
16
+
17
+ - **`RateInfo.signal_type` and `RateInfo.description`.** v2.0 rate-values responses carry the per-RIN `SignalType` label and `Description` at the top level (as in the RIN list); these are now surfaced on the entity instead of being silently ignored.
18
+ - **Strict wire-contract validation in the integration suite.** New `TestWireContract` validates raw live responses against the `midas-api-specs` JSON Schemas (via `jsonschema`), and the coerced tests now assert day-type values, so future wire drift fails the suite rather than being absorbed by the lenient runtime models. Point `MIDAS_SPECS_DIR` at a `midas-api-specs` checkout to enable the schema checks (they skip cleanly if absent). Adds `jsonschema` as a dev dependency.
19
+
20
+ ## [1.0.0] — 2026-06-22
21
+
22
+ The California Energy Commission released **MIDAS v2.0 on 2026-06-22**, a breaking change to the live API. v1.0 was removed from the live service that day, so python-midas 1.0.0 is a **v2-only release** — v1.0 compatibility is intentionally dropped. See [doc/v2-migration.md](doc/v2-migration.md) for the v1.0→v2.0 upgrade guide, and the upstream [`midas-api-specs`](https://github.com/grid-coordination/midas-api-specs) `v2` branch for the spec-level delta. Every change below was verified by a live smoke-test against the production v2.0 API on release day. python-midas is a **read-only consumer client**: v2.0 GET endpoints are unauthenticated, so `create_anonymous_client` needs no credentials; the authenticated constructors remain for the utility upload path only.
23
+
24
+ ### Changed
25
+
26
+ - **Breaking: the per-interval value field is read from `Value` (capital V)** instead of v1.0's lowercase `value`. The CEC standardised the casing in v2.0; `ValueData.from_raw` reads `Value`.
27
+ - **Breaking: `ValueData` interval boundaries are a zone-aware `period` tuple.** The four zone-naive fields `date_start`, `date_end`, `time_start`, and `time_end` (`datetime.date` / `datetime.time`) are **removed**; each interval is exposed as `period: tuple[pendulum.DateTime, pendulum.DateTime] | None` — a `(start, end)` pair of zone-aware moments. A bare wall-clock time with no zone is ambiguous, and in v2.0 the wire delivers `DateStart`/`TimeStart`/`DateEnd`/`TimeEnd` in UTC for *every* signal type (v1.0 delivered SGIP GHG and Flex Alert in Pacific Time — an upstream-provider passthrough bug the CEC fixed in v2.0). The pair is composed from the UTC wire date+time and **preserved in UTC** — the library never normalizes to a display zone. Need a wall-clock date or time? Convert an endpoint yourself: `period[0].in_tz("America/Los_Angeles").date()`. The original wire strings remain on `_raw`. (Mirrors `python-oa3`'s `IntervalPeriod.period`.)
28
+ - **Breaking: datetime coercion is zone-aware `pendulum.DateTime`, preserving the wire zone.** Every coerced timestamp carries an honest timezone — `Z`-suffixed fields (`SystemTime_UTC`, `SignupCloseDate`) stay UTC, as does `LastUpdated`, which in v2.0 carries an explicit basic-format offset (`+0000`); `parse_instant` honours it (no Pacific-local shift). The library does not normalize to a single zone; consumers convert with `.in_tz(...)` as needed. This replaces the prior `_parse_datetime` that silently treated every naive field as UTC.
29
+ - **Breaking: `rin_list` / `coerce_rin_list` peel the v2.0 keyed-object response.** v2.0 wraps the entry array in a single-keyed object — on the live API the key is **always `Rates`**, regardless of the requested `SignalType` (the `GHGEmissions`/`FlexAlerts`/`All` keys implied by early design notes do not appear on the wire). `coerce_rin_list(raw, signal_type)` peels the single value without switching on the key name and returns a uniform `list[RinListEntry]`; the new `RinListResponse` model validates the shape.
30
+ - **Breaking: lookup tables return a keyed object.** v2.0 wraps lookup rows in `{table_name, data: [...]}` instead of v1.0's bare array. `lookup_table` / `coerce_lookup_table` peel `data`; the new `LookupTableResponse` model validates the shape. `LookupEntry` gains optional `payload_descriptor` and `unit_type` (extra columns the `Unit` table carries).
31
+ - **Breaking: `SignalType` enum carries the v2.0 long-form wire labels.** `SignalType.RATES` is now `"Electricity Rates"` (was `"Rates"`); new members `GHG_EMISSIONS = "Greenhouse Gas Emissions"` and `FLEX_ALERT = "California Independent System Operator Flex Alert"`. v2.0 always populates the per-entry `SignalType` field (v1.0 returned `null` for GHG/Flex Alert entries); the old labels no longer map.
32
+ - **Breaking: `RateType` enum tracks the v2.0 wire values, which differ by signal type.** Electricity rates return the short `Ratetype` UploadCode, so the enum now uses short codes (`TOU`, `CPP`, `RTP`, `VPP`, `DSR`, `V-D`, `C-D`, `R-D`, `T-D`) — was the long descriptions (`"Time of use"`, …). GHG returns the long Description `"Greenhouse Gas emissions"` (`RateType.GHG`) and Flex Alert returns `"Flex Alert"` (`RateType.FLEX_ALERT`); both are retained. Unknown values still pass through as plain strings.
33
+ - **Breaking: `Unit` reports v2.0 GHG emissions in grams.** New `Unit.G_CO2_PER_KWH = "g/kWh CO2"` — values are **1000× larger** than v1.0's `kg/kWh CO2` for the same physical reading. `Unit.KG_CO2_PER_KWH` is retained for pre-migration historical-archive reads; `MIDASClient.ghg()` recognises both.
34
+ - **Breaking: `get_historical_data` / `historical_data` use the path-param endpoint `/HistoricalData/{rate_id}`** (was the `?id=` query param), and reject a range longer than the v2.0 **6-month** max per call with `ValueError`. The Python signatures are unchanged.
35
+
36
+ ### Added
37
+
38
+ - **`create_anonymous_client`** — an unauthenticated client for the v2.0 GET endpoints (no token acquired, no `Authorization` header sent). This is the default, supported access mode for this read-only consumer; `create_client` / `create_auto_client` remain for the utility upload (POST) path only.
39
+ - **`src/midas/time.py`** — pendulum-based time module shared in spirit with `python-oa3`: the `PendulumDateTime` annotated Pydantic type (parses on input, serialises to ISO 8601 preserving the wire offset) plus `parse_instant` (zone-tagged fields, incl. `LastUpdated`), `parse_local` (attach a documented zone to a bare wall-clock string), and `parse_value_moment` (compose a UTC `ValueInformation` boundary). `MIDAS_ZONE = "America/Los_Angeles"` documents MIDAS's native administrative zone.
40
+ - **`RinListResponse`** and **`LookupTableResponse`** entity models for the v2.0 keyed RIN-list and lookup-table responses, exported from `midas` and `midas.entities`.
41
+ - **`RateType.MOER`** for the v2.0 unified SGIP GHG signal, plus the short electricity `Ratetype` codes (`VPP`, `DSR`, `V-D`, `C-D`, `R-D`, `T-D`).
42
+
43
+ ### Removed
44
+
45
+ - **`get_holidays` / `holidays` / `coerce_holidays` and the `Holiday` model** — v2.0 retired the standalone `/Holiday` endpoint (absent from the CEC's published OpenAPI). Remove any holiday calls; the `Holiday` *day-type* value in rate schedules (`DayType.HOLIDAY`) is unaffected.
46
+ - **`get_historical_list` / `historical_list` / `coerce_historical_list`** — v2.0 retires the `/HistoricalList` endpoint. Use `rin_list(signal_type=0)` for the full active RIN list.
47
+ - The `Holiday` and `TimeZone` **lookup tables** are retired in v2.0 (`?LookupTable=Holiday` / `TimeZone` no longer return data).
48
+
49
+ ## [0.1.1] — 2026-03-20
50
+
51
+ ### Fixed
52
+
53
+ - `RateInfo.id` accepts `null` for empty realtime responses (the live API returns an all-null `RateInfo` when a RIN has no current realtime datapoint).
54
+
55
+ ## [0.1.0] — 2026-03-20
56
+
57
+ Initial implementation. Two-layer raw/coerced data model: raw `httpx.Response` accessors plus coerced Pydantic entities (`RateInfo`, `ValueData`, `RinListEntry`, `Holiday`, `LookupEntry`) with `Decimal` prices and pendulum datetimes, each carrying its original wire dict on `_raw`. httpx-based `MIDASClient` with HTTP Basic → bearer-token auth and transparent auto-refresh (`AutoTokenAuth`); RIN list, rate values, lookup tables, holidays, and historical endpoints; signal-type helpers (`ghg`, `flex_alert`, `flex_alert_active`); domain enums (`SignalType`, `RateType`, `Unit`, `DayType`).
58
+
59
+ [1.0.1]: https://github.com/grid-coordination/python-midas/releases/tag/v1.0.1
60
+ [1.0.0]: https://github.com/grid-coordination/python-midas/releases/tag/v1.0.0
61
+ [0.1.1]: https://github.com/grid-coordination/python-midas/releases/tag/v0.1.1
62
+ [0.1.0]: https://github.com/grid-coordination/python-midas/releases/tag/v0.1.0
@@ -0,0 +1,85 @@
1
+ # Contributing to python-midas
2
+
3
+ Thanks for your interest in contributing! This repo is a Python client library for the California Energy Commission's [MIDAS API](https://midasapi.energy.ca.gov/). It exposes a two-layer raw/coerced data model with Pydantic v2 entities, pendulum time types, and an httpx-based client. Unlike some sibling libraries it does **not** bundle an OpenAPI spec — endpoints are hand-written `httpx` calls, and the spec-level source of truth lives upstream in [`grid-coordination/midas-api-specs`](https://github.com/grid-coordination/midas-api-specs).
4
+
5
+ ## How to contribute
6
+
7
+ ### Discussions
8
+
9
+ Use [Discussions](https://github.com/grid-coordination/python-midas/discussions) for:
10
+
11
+ - Questions about how to use the library — clients, coercion, raw/coerced layering, time and timezone handling, signal-type helpers
12
+ - API and design judgment calls — "should python-midas model X?" / "is this the right shape for Y?"
13
+ - MIDAS API behavior gaps that affect python-midas — when the live API exposes something that doesn't fit the current entity shape and you want to scope what the library should do about it
14
+ - Coordination with the upstream [`midas-api-specs`](https://github.com/grid-coordination/midas-api-specs) spec repo (whose `doc/` notes — RIN structure, datetime/timezone semantics, the v2.0 migration map — this library follows)
15
+ - Sharing what you're building on top of python-midas
16
+
17
+ Discussions are open-ended — a good place to think out loud or scope something before it becomes a concrete change. Aligned outcomes from a Discussion often turn into one or more Issues.
18
+
19
+ ### Issues
20
+
21
+ Use [Issues](https://github.com/grid-coordination/python-midas/issues) for actionable changes:
22
+
23
+ - Bugs in client construction, request building, response parsing, or coercion against the live MIDAS API
24
+ - Coercion or schema gaps surfaced by real API responses (a field the library doesn't handle, or a value that breaks the coerced shape)
25
+ - New endpoints or request parameters when MIDAS exposes them
26
+ - Test failures or unexpected behavior with concrete repro steps
27
+ - Documentation errors, unclear explanations, or stale prose in `README.md` or docstrings
28
+ - Discussion outcomes that have alignment and a clear scope
29
+
30
+ If you're not sure whether something is an Issue or a Discussion, start with a Discussion — we can convert it later.
31
+
32
+ ### Pull requests
33
+
34
+ Pull requests are welcome.
35
+
36
+ - For small fixes (typos, broken links, single-test corrections, single-coercion bug fixes), open a PR directly.
37
+ - For substantive changes (new endpoints, new entity types, new coerced fields, time-handling changes), open a Discussion or Issue first so we can align on scope before you invest the effort.
38
+ - All changes pass `pytest tests/ -m "not integration"` and `ruff check src/ tests/` / `ruff format --check src/ tests/` cleanly.
39
+ - Match the existing tone and structure. The library composes HTTP client → raw response accessors → coerced Pydantic entities as roughly orthogonal layers; patches that fit cleanly into one layer without leaking concerns across them are the easiest to land.
40
+ - One commit per logical change is fine; we don't require squash or any particular branch naming.
41
+
42
+ ## Development
43
+
44
+ ```bash
45
+ pip install -e ".[dev]" # install with dev dependencies
46
+ pytest tests/ -v -m "not integration" # run the offline unit suite
47
+ ruff check src/ tests/ # lint
48
+ ruff format --check src/ tests/ # format check (drop --check to apply)
49
+ ```
50
+
51
+ ### Time and timezones
52
+
53
+ Every coerced datetime is a zone-aware `pendulum.DateTime`, and the library **preserves the honest wire zone** rather than normalizing to one display zone: `Z`-suffixed fields (`SystemTime_UTC`, `SignupCloseDate`) stay UTC; bare administrative fields (`LastUpdated`) are `America/Los_Angeles` local; `ValueData` interval boundaries are exposed as a `period` `(start, end)` tuple composed from the v2.0 UTC wire and kept in UTC. Convert to a zone of your choice yourself with `.in_tz(...)`. The parsing rules live in `src/midas/time.py`; the wire-level semantics they encode are documented in [`midas-api-specs/doc/datetime-and-timezone.md`](https://github.com/grid-coordination/midas-api-specs/blob/main/doc/datetime-and-timezone.md). When changing time handling, keep the "know the zone, convert it yourself" contract intact.
54
+
55
+ ### Integration tests
56
+
57
+ Tests marked `integration` hit the live MIDAS API and require `MIDAS_USERNAME` / `MIDAS_PASSWORD`; they are excluded from CI and from the default offline run above. They target the **v2.0** API, which is live only after the CEC cutover on 2026-06-22 — running them before then (or without credentials) will fail or skip. Run them on release day to smoke-test against production:
58
+
59
+ ```bash
60
+ MIDAS_USERNAME=... MIDAS_PASSWORD=... pytest tests/ -v -m integration
61
+ ```
62
+
63
+ ### Releases
64
+
65
+ `python-midas` is published to PyPI by a GitHub Actions workflow ([`.github/workflows/publish.yml`](.github/workflows/publish.yml)) using PyPI's [Trusted Publisher](https://docs.pypi.org/trusted-publishers/) flow — no API tokens or personal credentials are involved. Pushing a `v*` tag triggers the workflow, which runs the offline test suite, builds the sdist + wheel, and uploads via OpenID Connect.
66
+
67
+ To cut a release:
68
+
69
+ 1. Bump `version` in `pyproject.toml`.
70
+ 2. Move the `## [Unreleased]` entries in [`CHANGELOG.md`](CHANGELOG.md) under a `## [<version>] — <date>` heading (Added / Changed / Fixed / Removed). Update the reference links at the bottom.
71
+ 3. Commit on `main` (e.g. `Bump version to X.Y.Z`).
72
+ 4. Tag and push: `git tag vX.Y.Z && git push origin vX.Y.Z`.
73
+ 5. The `Publish to PyPI` workflow runs the test job, then the publish job in the `pypi` environment. Watch it on the Actions tab.
74
+
75
+ The Trusted Publisher binding is `grid-coordination / python-midas / publish.yml / pypi`, configured at [pypi.org → Manage → Publishing](https://pypi.org/manage/account/publishing/). If a release fails with an OIDC error, verify the binding hasn't drifted (workflow filename, environment name, repo path).
76
+
77
+ Versioning follows [SemVer](https://semver.org/). The 1.0.0 release tracks MIDAS v2.0 and is a v2-only, breaking release (see [`CHANGELOG.md`](CHANGELOG.md)).
78
+
79
+ ## Code of conduct
80
+
81
+ Be respectful and constructive. We're a small project and appreciate everyone who takes the time to file an issue or send a PR.
82
+
83
+ ## Important notice
84
+
85
+ This library is provided on an "as-is" basis. Updates and maintenance, including responses to issues filed on GitHub, will take place on an "as time and resources permit" basis. Library output (raw API responses, coerced entities) is best-effort against the live MIDAS API and the upstream [`midas-api-specs`](https://github.com/grid-coordination/midas-api-specs) documentation. This library is not authoritative for billing, dispatch, or grid operations — independent verification against the MIDAS API's actual responses is recommended for any consumer relying on these results for operational correctness.
@@ -0,0 +1,418 @@
1
+ Metadata-Version: 2.4
2
+ Name: python-midas
3
+ Version: 1.0.1
4
+ Summary: Python client library for the California Energy Commission MIDAS API
5
+ Project-URL: Homepage, https://grid-coordination.energy
6
+ Project-URL: Repository, https://github.com/grid-coordination/python-midas
7
+ Project-URL: Issues, https://github.com/grid-coordination/python-midas/issues
8
+ Author: Clark Communications Corporation
9
+ License-Expression: MIT
10
+ License-File: LICENSE
11
+ Keywords: california,energy,ghg,grid,midas,rates
12
+ Classifier: Development Status :: 5 - Production/Stable
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: License :: OSI Approved :: MIT License
15
+ Classifier: Programming Language :: Python :: 3
16
+ Classifier: Programming Language :: Python :: 3.10
17
+ Classifier: Programming Language :: Python :: 3.11
18
+ Classifier: Programming Language :: Python :: 3.12
19
+ Classifier: Programming Language :: Python :: 3.13
20
+ Classifier: Topic :: Software Development :: Libraries
21
+ Requires-Python: >=3.10
22
+ Requires-Dist: httpx>=0.27
23
+ Requires-Dist: pendulum>=3.0
24
+ Requires-Dist: pydantic>=2.5
25
+ Provides-Extra: dev
26
+ Requires-Dist: jsonschema>=4.18; extra == 'dev'
27
+ Requires-Dist: pytest-httpx>=0.30; extra == 'dev'
28
+ Requires-Dist: pytest>=8.0; extra == 'dev'
29
+ Requires-Dist: ruff>=0.3; extra == 'dev'
30
+ Description-Content-Type: text/markdown
31
+
32
+ # python-midas
33
+
34
+ [![PyPI version](https://img.shields.io/pypi/v/python-midas.svg)](https://pypi.org/project/python-midas/)
35
+ [![Python versions](https://img.shields.io/pypi/pyversions/python-midas.svg)](https://pypi.org/project/python-midas/)
36
+ [![CI](https://github.com/grid-coordination/python-midas/actions/workflows/ci.yml/badge.svg)](https://github.com/grid-coordination/python-midas/actions/workflows/ci.yml)
37
+ [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
38
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
39
+
40
+ Python client library for the California Energy Commission [MIDAS](https://midasapi.energy.ca.gov/) (Market Informed Demand Automation Server) API.
41
+
42
+ MIDAS provides California energy rate data, greenhouse gas (GHG) emissions signals, and Flex Alert status. This library wraps the API with typed Pydantic models and a two-layer data model that preserves raw API responses alongside coerced Python-native types.
43
+
44
+ > **This is a read-only consumer client.** python-midas is built for *consuming* MIDAS data. In v2.0 all public GET endpoints are unauthenticated, so you need **no credentials** — just `create_anonymous_client()`. The authenticated constructors (`create_client` / `create_auto_client`) exist only for utilities that *upload* rate data to the CEC; that path requires CEC-issued utility credentials and is not exercised by this project.
45
+
46
+ Part of the [grid-coordination](https://github.com/grid-coordination) project family, alongside [clj-midas](https://github.com/grid-coordination/clj-midas) (Clojure client) and [midas-api-specs](https://github.com/grid-coordination/midas-api-specs) (OpenAPI specifications).
47
+
48
+ ## Installation
49
+
50
+ ```bash
51
+ pip install python-midas
52
+ ```
53
+
54
+ The distribution is named `python-midas` (the bare `midas` name on PyPI belongs to an unrelated gas-detector driver), but it imports as `midas`:
55
+
56
+ ```python
57
+ import midas
58
+ ```
59
+
60
+ For development:
61
+
62
+ ```bash
63
+ pip install -e ".[dev]"
64
+ ```
65
+
66
+ Requires Python 3.10+.
67
+
68
+ ## Quick Start
69
+
70
+ ```python
71
+ from midas import create_anonymous_client
72
+
73
+ client = create_anonymous_client() # v2.0 GETs need no credentials
74
+
75
+ # List available Rate Identification Numbers (RINs)
76
+ rins = client.rin_list()
77
+ for rin in rins:
78
+ print(f"{rin.id} {rin.signal_type} {rin.description}")
79
+
80
+ # Get rate values for a specific RIN
81
+ rate = client.rate_values(rins[0].id)
82
+ print(f"{rate.name} ({rate.type})")
83
+ for v in rate.values:
84
+ start, end = v.period # zone-aware (start, end) — UTC on the wire
85
+ print(f" {start}–{end}: {v.value} {v.unit}")
86
+ ```
87
+
88
+ ## Authentication
89
+
90
+ In MIDAS **v2.0**, all public GET endpoints (rate values, RIN list, lookup tables, historical data) are **unauthenticated** — use `create_anonymous_client` for read-only access:
91
+
92
+ ```python
93
+ from midas import create_anonymous_client
94
+
95
+ with create_anonymous_client() as client:
96
+ rins = client.rin_list()
97
+ rate = client.rate_values(rins[0].id)
98
+ ```
99
+
100
+ Authentication exists only for **uploads** (LSE rate submission, POST) and requires CEC-issued utility credentials; it is **not exercised by this read-only consumer**. For completeness, the upload path is provided: MIDAS uses HTTP Basic authentication to acquire a short-lived bearer token (valid for 10 minutes), via `create_auto_client` (acquires a token and transparently refreshes it within a 30-second buffer), `create_client` (a single token), or the low-level `get_token` / `token_expired` helpers.
101
+
102
+ ```python
103
+ from midas import create_auto_client, create_client, get_token, token_expired
104
+
105
+ client = create_auto_client("username", "password") # auto-refreshing (uploads)
106
+ client = create_client("username", "password") # single token (~10 min)
107
+
108
+ token_info = get_token("username", "password") # low-level
109
+ # token_info = {"token": "...", "acquired_at": DateTime, "expires_at": DateTime}
110
+ ```
111
+
112
+ ## API Coverage
113
+
114
+ The MIDAS API has a single multiplexed `/ValueData` endpoint that serves different response shapes depending on query parameters, plus a separate endpoint for historical data. All read operations are covered:
115
+
116
+ ### RIN List
117
+
118
+ List available Rate Identification Numbers, optionally filtered by signal type:
119
+
120
+ ```python
121
+ all_rins = client.rin_list() # All signal types
122
+ rate_rins = client.rin_list(signal_type=1) # Rates only
123
+ ghg_rins = client.rin_list(signal_type=2) # GHG only
124
+ flex_rins = client.rin_list(signal_type=3) # Flex Alert only
125
+ ```
126
+
127
+ Each `RinListEntry` has:
128
+
129
+ - `id` — the RIN string (e.g. `"USCA-PGPG-ETOU-0000"`)
130
+ - `signal_type` — `SignalType.RATES`, `SignalType.GHG_EMISSIONS`, or `SignalType.FLEX_ALERT` (v2.0 long-form wire labels)
131
+ - `description` — human-readable description
132
+ - `last_updated` — `pendulum.DateTime` of last data update
133
+
134
+ ### Rate Values
135
+
136
+ Fetch current rate/price data for a specific RIN:
137
+
138
+ ```python
139
+ rate = client.rate_values("USCA-TSTS-TTOU-TEST")
140
+ rate = client.rate_values("USCA-TSTS-TTOU-TEST", query_type="realtime")
141
+ ```
142
+
143
+ The `RateInfo` model contains:
144
+
145
+ - `id` — the RIN
146
+ - `name` — rate name (e.g. `"CEC TEST24HTOU"`)
147
+ - `type` — `RateType` enum or raw string. The wire value is inconsistent across signal types in v2.0: electricity rates return the short Ratetype code (`TOU`, `CPP`, `RTP`, …) while GHG returns `Greenhouse Gas emissions` and Flex Alert returns `Flex Alert`
148
+ - `system_time` — server timestamp as `pendulum.DateTime`
149
+ - `sector`, `end_use` — customer classification
150
+ - `rate_plan_url`, `api_url` — external links (the API's `"None"` string is coerced to `None`)
151
+ - `signup_close` — rate signup deadline as `pendulum.DateTime`
152
+ - `values` — list of `ValueData` intervals
153
+
154
+ Each `ValueData` interval has:
155
+
156
+ - `name` — interval description (e.g. `"winter off peak"`)
157
+ - `period` — `(start, end)` tuple of zone-aware `pendulum.DateTime` moments (or `None` when a boundary is absent). Composed from the v2.0 UTC wire date+time and kept in UTC; convert with `.in_tz(...)`. See [Time and timezones](#time-and-timezones).
158
+ - `day_start`, `day_end` — `DayType` enum (Monday through Sunday, plus Holiday)
159
+ - `value` — `Decimal` (preserves precision for financial data)
160
+ - `unit` — `Unit` enum (`$/kWh`, `$/kW`, `kg/kWh CO2`, `Event`, etc.)
161
+
162
+ ### Lookup Tables
163
+
164
+ Fetch reference data tables:
165
+
166
+ ```python
167
+ energies = client.lookup_table("Energy") # Energy providers
168
+ dists = client.lookup_table("Distribution") # Distribution companies
169
+ units = client.lookup_table("Unit") # Available units
170
+ sectors = client.lookup_table("Sector") # Customer sectors
171
+ ```
172
+
173
+ Available tables: `Country`, `Daytype`, `Distribution`, `Enduse`, `Energy`, `Location`, `Ratetype`, `Sector`, `State`, `Unit`. (v2.0 retired the `Holiday` and `TimeZone` lookup tables.)
174
+
175
+ In v2.0 a lookup response is a keyed object `{table_name, data: [...]}`; the client peels `data` for you. Each `LookupEntry` has `code` and `description`, plus optional `payload_descriptor` and `unit_type` (the `Unit` table carries these extra columns).
176
+
177
+ ### Historical Data
178
+
179
+ Query archived rate data for a RIN over a date range (v2.0 caps each call at a **6-month** range and takes the RIN as a path parameter):
180
+
181
+ ```python
182
+ hist = client.historical_data("USCA-PGPG-ETOU-0000", "2023-01-01", "2023-06-30")
183
+ ```
184
+
185
+ A range longer than six months raises `ValueError` — split it into multiple calls.
186
+
187
+ > The v1.0 `historical_list` / `get_historical_list` methods are **removed** — v2.0 retires the `/HistoricalList` endpoint. For the full active RIN list use `client.rin_list(signal_type=0)`.
188
+
189
+ ## Signal Type Helpers
190
+
191
+ Convenience methods for identifying signal types, matching the [clj-midas](https://github.com/grid-coordination/clj-midas) API:
192
+
193
+ ```python
194
+ rate = client.rate_values("USCA-GHGH-SGHT-0000")
195
+
196
+ client.ghg(rate) # True if GHG signal (by RateType or Unit)
197
+ client.flex_alert(rate) # True if Flex Alert signal
198
+ client.flex_alert_active(rate) # True if Flex Alert with any non-zero value
199
+ ```
200
+
201
+ ## Time and timezones
202
+
203
+ Every coerced datetime is a zone-aware `pendulum.DateTime` — Python's equivalent of Java's `ZonedDateTime` (it carries an IANA zone, not just a fixed offset, so it is DST-correct). The guiding principle, shared with [clj-midas](https://github.com/grid-coordination/clj-midas) and [python-oa3](https://github.com/grid-coordination/python-oa3): **you always know what zone a value is in, and you convert it yourself.** The library preserves the honest wire zone and never normalizes to a single display zone.
204
+
205
+ MIDAS mixes two wire conventions and does not tag the bare ones; python-midas encodes the documented zone for each field (see [midas-api-specs/doc/datetime-and-timezone.md](https://github.com/grid-coordination/midas-api-specs/blob/main/doc/datetime-and-timezone.md)):
206
+
207
+ | Field | Wire form (v2.0) | Coerced as |
208
+ |-------|------------------|------------|
209
+ | `system_time`, `signup_close` | `Z`-suffixed (UTC) | `pendulum.DateTime` in **UTC** — instant preserved |
210
+ | `ValueData.period` (start, end) | bare `DateStart`/`TimeStart`/…, **UTC** in v2.0 | pair of `pendulum.DateTime` in **UTC** |
211
+ | `last_updated` | UTC with basic-format offset (`+0000`) | `pendulum.DateTime` in **UTC** — instant preserved (absent on Flex Alert entries → `None`) |
212
+
213
+ ```python
214
+ rate = client.rate_values("USCA-TSTS-TTOU-TEST")
215
+ start, end = rate.values[0].period
216
+ start # 2026-05-01 07:00:00+00:00 (UTC, as delivered)
217
+ start.in_tz("America/Los_Angeles") # 2026-05-01 00:00:00-07:00 (you convert)
218
+ start.in_tz("America/Los_Angeles").date() # datetime.date(2026, 5, 1) (wall-clock date)
219
+ ```
220
+
221
+ In v2.0 (effective 2026-06-22) MIDAS delivers every `ValueInformation` date/time field in UTC for all signal types — fixing the v1.0 bug where SGIP GHG and Flex Alert timestamps arrived Pacific-local on the wire — and `LastUpdated` now carries an explicit `+0000` offset. The parsing rules live in `midas.time` (`parse_instant`, `parse_local`, `parse_value_moment`, and the `PendulumDateTime` Pydantic type).
222
+
223
+ > The v1.0 zone-naive fields `date_start`, `date_end`, `time_start`, `time_end` (`datetime.date` / `datetime.time`) are **removed** in favour of `period`: a bare wall-clock time with no zone is ambiguous, whereas the `(start, end)` moments are self-describing. The exact wire strings remain on `_raw`.
224
+
225
+ ## Two-Layer Data Model
226
+
227
+ Following the [python-oa3](https://github.com/grid-coordination/python-oa3) pattern, every entity provides two layers:
228
+
229
+ **Raw layer** — the original API JSON dict (PascalCase keys, string values), accessible via `_raw`:
230
+
231
+ ```python
232
+ rate = client.rate_values("USCA-TSTS-TTOU-TEST")
233
+ rate._raw["RateID"] # "USCA-TSTS-TTOU-TEST"
234
+ rate._raw["ValueInformation"][0]["Value"] # 0.1006 (v2.0 capitalises the key)
235
+ rate.values[0]._raw["Unit"] # "$/kWh"
236
+ ```
237
+
238
+ **Coerced layer** — typed Pydantic models with snake_case fields and native Python types:
239
+
240
+ ```python
241
+ rate.id # "USCA-TSTS-TTOU-TEST"
242
+ rate.type # RateType.TOU
243
+ rate.system_time # pendulum.DateTime in UTC (Z-suffixed wire field)
244
+ rate.values[0].value # Decimal("0.1006")
245
+ rate.values[0].unit # Unit.DOLLAR_PER_KWH
246
+ rate.values[0].day_start # DayType.MONDAY
247
+ rate.values[0].period # (DateTime, DateTime) — zone-aware (start, end)
248
+ ```
249
+
250
+ This lets you work with clean, typed data while always being able to fall back to the exact API response when needed.
251
+
252
+ ## Dual-Mode Client
253
+
254
+ Every endpoint is available in two forms:
255
+
256
+ **Raw methods** return `httpx.Response` for full HTTP control:
257
+
258
+ ```python
259
+ resp = client.get_rin_list(signal_type=0)
260
+ resp.status_code # 200
261
+ resp.json() # raw JSON list
262
+
263
+ resp = client.get_rate_values("USCA-TSTS-TTOU-TEST", query_type="alldata")
264
+ resp = client.get_lookup_table("Energy")
265
+ resp = client.get_historical_data("USCA-PGPG-ETOU-0000", "2023-01-01", "2023-06-30")
266
+ ```
267
+
268
+ **Coerced methods** return typed Pydantic models (call `raise_for_status()` internally):
269
+
270
+ ```python
271
+ rins = client.rin_list(signal_type=0) # list[RinListEntry]
272
+ rate = client.rate_values("USCA-TSTS-TTOU-TEST") # RateInfo
273
+ entries = client.lookup_table("Energy") # list[LookupEntry]
274
+ rate = client.historical_data(rin, start, end) # RateInfo (≤ 6-month range)
275
+ ```
276
+
277
+ ## Coercion Functions
278
+
279
+ You can also coerce raw dicts directly, without going through the client:
280
+
281
+ ```python
282
+ from midas import coerce_rate_info, coerce_rin_list, coerce_lookup_table
283
+
284
+ rate = coerce_rate_info({"RateID": "...", "ValueInformation": [...]})
285
+ # v2.0 wraps the RIN list under a single key (always "Rates"); coerce_rin_list peels it.
286
+ rins = coerce_rin_list({"Rates": [{"RateID": "...", "SignalType": "Electricity Rates", ...}]})
287
+ # v2.0 wraps lookup rows under {table_name, data: [...]}; coerce_lookup_table peels data.
288
+ units = coerce_lookup_table({"table_name": "Unit", "data": [{"UploadCode": "...", ...}]})
289
+ ```
290
+
291
+ Available: `coerce_rate_info`, `coerce_rin_list`, `coerce_lookup_table`.
292
+
293
+ ## Enums
294
+
295
+ Domain values are represented as `str` enums, so they compare equal to their string values:
296
+
297
+ ```python
298
+ from midas import SignalType, RateType, Unit, DayType
299
+
300
+ SignalType.RATES # "Electricity Rates"
301
+ SignalType.GHG_EMISSIONS # "Greenhouse Gas Emissions"
302
+ SignalType.FLEX_ALERT # "California Independent System Operator Flex Alert"
303
+
304
+ # Electricity rates return the short Ratetype UploadCode in v2.0:
305
+ RateType.TOU # "TOU"
306
+ RateType.CPP # "CPP"
307
+ RateType.RTP # "RTP"
308
+ # (also VPP, DSR, V-D, C-D, R-D, T-D)
309
+ # GHG and Flex Alert return long Descriptions, not short codes:
310
+ RateType.GHG # "Greenhouse Gas emissions"
311
+ RateType.FLEX_ALERT # "Flex Alert"
312
+ RateType.MOER # "MOER" (v2.0 unified SGIP GHG signal)
313
+
314
+ Unit.DOLLAR_PER_KWH # "$/kWh"
315
+ Unit.DOLLAR_PER_KW # "$/kW"
316
+ Unit.EXPORT_DOLLAR_PER_KWH # "export $/kWh"
317
+ Unit.BACKUP_DOLLAR_PER_KWH # "backup $/kWh"
318
+ Unit.G_CO2_PER_KWH # "g/kWh CO2" (v2.0 GHG — grams, 1000× the v1.0 kg value)
319
+ Unit.KG_CO2_PER_KWH # "kg/kWh CO2" (historical-archive reads)
320
+ Unit.DOLLAR_PER_KVARH # "$/kvarh"
321
+ Unit.EVENT # "Event"
322
+ Unit.LEVEL # "Level"
323
+
324
+ DayType.MONDAY # "Monday"
325
+ # ... through SUNDAY, plus:
326
+ DayType.HOLIDAY # "Holiday"
327
+ ```
328
+
329
+ ## Type Coercion Details
330
+
331
+ The coercion layer applies the following transformations:
332
+
333
+ | API type | Python type | Notes |
334
+ |----------|-------------|-------|
335
+ | Zone-tagged datetime (`"…Z"`) | `pendulum.DateTime` | Instant preserved in UTC (`system_time`, `signup_close`) |
336
+ | `LastUpdated` (`"…+0000"`) | `pendulum.DateTime` | UTC instant; explicit offset honoured |
337
+ | `ValueInformation` date + time | `pendulum.DateTime` pair (`period`) | Composed as UTC in v2.0 — see [Time and timezones](#time-and-timezones) |
338
+ | Numeric values | `Decimal` | Preserves precision for financial data |
339
+ | Signal type strings | `SignalType` enum | `None` passes through as `None` |
340
+ | Rate type strings | `RateType` enum | Unknown values pass through as strings |
341
+ | Unit strings | `Unit` enum | Unknown values pass through as strings |
342
+ | Day type strings | `DayType` enum | `None` passes through (historical data) |
343
+ | `"None"` string (API_Url) | `None` | MIDAS API quirk |
344
+
345
+ ## Context Manager
346
+
347
+ The client supports context manager protocol for clean resource management:
348
+
349
+ ```python
350
+ from midas import create_anonymous_client
351
+
352
+ with create_anonymous_client() as client:
353
+ rins = client.rin_list()
354
+ rate = client.rate_values(rins[0].id)
355
+ # httpx client is closed automatically
356
+ ```
357
+
358
+ ## Project Structure
359
+
360
+ ```
361
+ src/midas/
362
+ __init__.py # Public API re-exports
363
+ py.typed # PEP 561 type-checking marker
364
+ client.py # MIDASClient, create_anonymous_client, create_client, create_auto_client
365
+ auth.py # BearerAuth, BasicAuth, AutoTokenAuth, get_token (upload path)
366
+ enums.py # SignalType, RateType, Unit, DayType
367
+ time.py # pendulum parsing + PendulumDateTime Pydantic type
368
+ entities/
369
+ __init__.py # Coercion dispatch functions
370
+ models.py # Pydantic models: RateInfo, ValueData, RinListEntry, RinListResponse, LookupEntry, LookupTableResponse
371
+ tests/
372
+ test_entities.py # Entity coercion from raw fixture dicts
373
+ test_client.py # HTTP client tests with pytest-httpx
374
+ test_auth.py # Token parsing, expiry, auth headers
375
+ test_integration.py # Live API tests (anonymous, no credentials)
376
+ ```
377
+
378
+ ## Development
379
+
380
+ ```bash
381
+ # Install with dev dependencies
382
+ pip install -e ".[dev]"
383
+
384
+ # Lint
385
+ ruff check src/ tests/
386
+ ```
387
+
388
+ See [CONTRIBUTING.md](CONTRIBUTING.md) for the full contributor workflow and [CHANGELOG.md](CHANGELOG.md) for release history. python-midas `1.0.0` tracks the breaking MIDAS **v2.0** API (live 2026-06-22; v2-only — v1.0 support intentionally dropped) — see [doc/v2-migration.md](doc/v2-migration.md) for the v1.0→v2.0 upgrade guide.
389
+
390
+ ### Tests
391
+
392
+ The test suite has two tiers:
393
+
394
+ **Unit tests** run entirely offline using fixture dicts and mocked HTTP (pytest-httpx):
395
+
396
+ ```bash
397
+ pytest -m "not integration"
398
+ ```
399
+
400
+ **Integration tests** run against the live MIDAS API at `midasapi.energy.ca.gov` using an anonymous client — **no credentials required** (v2.0 GETs are unauthenticated):
401
+
402
+ ```bash
403
+ pytest -m integration
404
+ ```
405
+
406
+ Integration tests exercise every read endpoint (RIN list, rate values, lookup tables, historical data), all entity coercion paths against real response shapes, the v2.0 wire-shape corrections (keyed `Rates` RIN list, keyed `{table_name, data}` lookup tables, UTC `LastUpdated`, signal-type-dependent `RateType`), and the signal type helpers (GHG, Flex Alert detection). Pre-migration historical data may be absent during the v2.0 cutover week, so the historical test skips gracefully on a 404.
407
+
408
+ Note that the MIDAS API server can be slow (5-20+ seconds per request is normal), so the integration suite takes a few minutes to complete. Run everything together with just `pytest`.
409
+
410
+ ## Related Projects
411
+
412
+ - **[midas-api-specs](https://github.com/grid-coordination/midas-api-specs)** — OpenAPI specifications for the MIDAS API, derived from documentation and live API validation
413
+ - **[clj-midas](https://github.com/grid-coordination/clj-midas)** — Clojure client for the MIDAS API (Martian-based, spec-driven)
414
+ - **[python-oa3](https://github.com/grid-coordination/python-oa3)** — Python client for OpenADR 3 (same entity API pattern)
415
+
416
+ ## License
417
+
418
+ MIT