python-esios 2.0.0__py3-none-any.whl → 2.0.2__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,207 @@
1
+ ---
2
+ name: esios
3
+ description: Query Spanish electricity market data (ESIOS/REE). Use when the user asks about electricity prices, generation, demand, I90 files, or any ESIOS indicator.
4
+ version: 2.0.0
5
+ ---
6
+
7
+ # ESIOS Data Assistant
8
+
9
+ You have access to the `python-esios` CLI and library for querying the Spanish electricity market (ESIOS/REE).
10
+
11
+ ## CLI Reference
12
+
13
+ ### Indicators
14
+
15
+ ```bash
16
+ # List all indicators
17
+ esios indicators list
18
+
19
+ # Search by name
20
+ esios indicators search "precio"
21
+
22
+ # Show metadata (unit, granularity, geographies)
23
+ esios indicators meta 600
24
+
25
+ # Historical data
26
+ esios indicators history 600 --start 2025-01-01 --end 2025-01-31
27
+ esios indicators history 600 -s 2025-01-01 -e 2025-01-31 --format csv --output data.csv
28
+ esios indicators history 600 -s 2025-01-01 -e 2025-01-31 --format parquet --output data.parquet
29
+
30
+ # Filter by geography (ID or name)
31
+ esios indicators history 600 -s 2025-01-01 -e 2025-01-31 --geo España
32
+ esios indicators history 600 -s 2025-01-01 -e 2025-01-31 --geo 3
33
+
34
+ # Ad-hoc pandas expressions on fetched data
35
+ esios indicators exec 600 -s 2025-01-01 -e 2025-01-31 --expr "df.describe()"
36
+ esios indicators exec 600 -s 2025-01-01 -e 2025-01-31 --expr "df.resample('D').mean()"
37
+ ```
38
+
39
+ ### Archives
40
+
41
+ ```bash
42
+ # List all available archives
43
+ esios archives list
44
+
45
+ # Download archive files
46
+ esios archives download 34 --start 2025-05-01 --end 2025-05-31 --output ./data
47
+ esios archives download 34 --date 2025-06-01
48
+
49
+ # List sheets (table of contents) in an I90 file
50
+ esios archives sheets 34 --date 2025-06-01
51
+
52
+ # Parse and query archive data (like indicators exec but for archives)
53
+ esios archives exec 34 --sheet I90DIA03 --date 2025-06-01
54
+ esios archives exec 34 --sheet I90DIA03 --date 2025-06-01 -x "df.describe()"
55
+ esios archives exec 34 --sheet I90DIA03 -s 2025-05-05 -e 2025-06-08 \
56
+ -x "df[df['Sentido']=='Bajar'].groupby('Unidad de Programación')['value'].sum().sort_values()"
57
+ esios archives exec 34 --sheet I90DIA26 --date 2025-06-01 --format csv --output pbf.csv
58
+ ```
59
+
60
+ ### Cache Management
61
+
62
+ ```bash
63
+ esios cache status # Path, size, geos registry, catalog info
64
+ esios cache geos # Global geo_id → geo_name registry
65
+ esios cache path # Print cache directory
66
+ esios cache clear # Clear indicator cache
67
+ esios cache clear --all # Clear everything (indicators, archives, geos, catalog)
68
+ esios cache clear --indicator 600 # Clear one indicator
69
+ ```
70
+
71
+ ### Configuration
72
+
73
+ ```bash
74
+ esios config set token <API_KEY>
75
+ esios config show
76
+ ```
77
+
78
+ ## Common Indicator IDs
79
+
80
+ | ID | Name | Description | Geos |
81
+ |----|------|-------------|------|
82
+ | 600 | Precio mercado spot | OMIE spot market price | ES, PT, FR, DE, BE, NL |
83
+ | 1001 | Precio mercado diario | Day-ahead market price | ES |
84
+ | 10033 | Demanda real | Real-time electricity demand | ES |
85
+ | 10034 | Generación eólica | Real-time wind generation | ES |
86
+ | 10035 | Generación solar FV | Real-time solar PV generation | ES |
87
+ | 1293 | Demanda prevista | Forecasted demand | ES |
88
+
89
+ Use `esios indicators search "query"` to find more. Use `esios indicators meta <id>` to see full metadata including geographies and units.
90
+
91
+ ## Multi-Geo Indicators
92
+
93
+ Some indicators (e.g. 600) return data for multiple countries. The output is pivoted so each geography becomes a column:
94
+
95
+ ```
96
+ datetime España Portugal Francia Alemania Bélgica Países Bajos
97
+ 2025-01-01 00:00:00 63.50 63.50 72.10 58.20 58.20 58.20
98
+ 2025-01-01 01:00:00 55.80 55.80 60.30 48.90 48.90 48.90
99
+ ```
100
+
101
+ Filter to specific geos with `--geo`:
102
+ ```bash
103
+ esios indicators history 600 -s 2025-01-01 -e 2025-01-07 --geo España --geo Portugal
104
+ ```
105
+
106
+ ## Geography Reference
107
+
108
+ | geo_id | geo_name |
109
+ |--------|----------|
110
+ | 1 | Portugal |
111
+ | 2 | Francia |
112
+ | 3 | España |
113
+ | 8826 | Alemania |
114
+ | 8827 | Bélgica |
115
+ | 8828 | Países Bajos |
116
+
117
+ The `--geo` flag accepts both IDs and names (case-insensitive substring match):
118
+ - `--geo 3` or `--geo España` or `--geo españa`
119
+ - `--geo "Países Bajos"` or `--geo 8828`
120
+
121
+ ## Python Library
122
+
123
+ ```python
124
+ from esios import ESIOSClient
125
+
126
+ client = ESIOSClient() # reads config file, then ESIOS_API_KEY env var
127
+
128
+ # Get indicator handle
129
+ handle = client.indicators.get(600)
130
+
131
+ # Historical data as DataFrame
132
+ df = handle.historical("2025-01-01", "2025-01-31")
133
+
134
+ # Filter by geo
135
+ df = handle.historical("2025-01-01", "2025-01-31", geo_ids=[3]) # España only
136
+
137
+ # Inspect geographies
138
+ handle.geos # List of {"geo_id": int, "geo_name": str}
139
+ handle.geos_dataframe() # DataFrame with geo_id and geo_name columns
140
+ handle.resolve_geo("España") # Returns 3
141
+
142
+ # Search and compare
143
+ results = client.indicators.search("precio")
144
+ df = client.indicators.compare([600, 10034, 10035], "2025-01-01", "2025-01-07")
145
+ ```
146
+
147
+ ## I90 Settlement Files
148
+
149
+ ### CLI (quickest path)
150
+
151
+ ```bash
152
+ # Discover available sheets
153
+ esios archives sheets 34 --date 2025-06-01
154
+
155
+ # Key I90DIA sheets:
156
+ # I90DIA03 — Restricciones en el Mercado Diario (curtailment)
157
+ # I90DIA08 — Restricciones en Tiempo Real
158
+ # I90DIA26 — Programa Base de Funcionamiento (PBF, generation program)
159
+ # I90DIA01 — Programa PVP
160
+ # I90DIA07 — Regulación Terciaria (mFRR)
161
+
162
+ # Total curtailment by direction
163
+ esios archives exec 34 --sheet I90DIA03 --date 2025-06-01 \
164
+ -x "df.groupby('Sentido')['value'].sum()"
165
+
166
+ # Multi-day curtailment analysis
167
+ esios archives exec 34 --sheet I90DIA03 -s 2025-05-05 -e 2025-06-08 \
168
+ -x "df[df['Sentido']=='Bajar'].groupby('Unidad de Programación')['value'].sum().sort_values().head(20)"
169
+ ```
170
+
171
+ ### Python library
172
+
173
+ ```python
174
+ from esios import ESIOSClient
175
+ from esios.processing.i90 import I90Book
176
+
177
+ # Download + parse in one step
178
+ client = ESIOSClient()
179
+ archive = client.archives.get(34)
180
+ books = I90Book.from_archive(archive, start="2025-05-05", end="2025-06-08")
181
+
182
+ # Access a sheet
183
+ book = books[0]
184
+ sheet = book["I90DIA03"]
185
+ df = sheet.df # Preprocessed DataFrame with datetime index
186
+ print(sheet.frequency) # "hourly" or "hourly-quarterly"
187
+ ```
188
+
189
+ ## Caching Behavior
190
+
191
+ - Data is cached locally as parquet files (`~/.cache/esios/`)
192
+ - Each indicator gets its own directory: `indicators/{id}/data.parquet`
193
+ - Indicator metadata cached in `indicators/{id}/meta.json` (7-day TTL)
194
+ - Indicator catalog cached in `indicators/catalog.json` (24h TTL)
195
+ - Global geo registry at `geos.json` (persisted forever, grows incrementally)
196
+ - Data older than 48h is considered final (won't be re-fetched)
197
+ - Recent data (last 48h) is re-fetched on each request (electricity market corrections)
198
+ - Cache is per-column sparse: fetching `--geo España` only caches that column
199
+
200
+ ## Key Conventions
201
+
202
+ - All timestamps are in Europe/Madrid timezone
203
+ - Date ranges > 3 weeks are auto-chunked into smaller API requests
204
+ - Archives support skip-existing (won't re-download cached files)
205
+ - I90 sheets detect hourly vs quarter-hourly frequency automatically
206
+ - API token resolution: config file (`~/.config/esios/config.toml`) > `ESIOS_API_KEY` env var
207
+ - Custom exceptions: `ESIOSError`, `AuthenticationError`, `APIResponseError`, `NetworkError`
esios/cli/archives.py CHANGED
@@ -64,3 +64,154 @@ def download_archive(
64
64
  output_dir=output,
65
65
  )
66
66
  typer.echo(f"Download complete → {output}")
67
+
68
+
69
+ def _download_files(
70
+ archive_id: int,
71
+ token: str | None,
72
+ date: str | None,
73
+ start: str | None,
74
+ end: str | None,
75
+ ) -> list:
76
+ """Download archive files (shared by sheets and exec commands)."""
77
+ from esios.cli.app import get_client
78
+
79
+ if not date and not (start and end):
80
+ typer.echo("Provide --date or both --start and --end.", err=True)
81
+ raise typer.Exit(1)
82
+
83
+ client = get_client(token)
84
+
85
+ if date:
86
+ files = client.archives.download(archive_id, date=date)
87
+ else:
88
+ files = client.archives.download(archive_id, start=start, end=end)
89
+
90
+ if not files:
91
+ typer.echo("No files found for the given date range.", err=True)
92
+ raise typer.Exit(1)
93
+
94
+ return files
95
+
96
+
97
+ @archives_app.command("sheets")
98
+ def sheets(
99
+ archive_id: int = typer.Argument(..., help="Archive ID (e.g. 34 for I90DIA)"),
100
+ date: Optional[str] = typer.Option(None, "--date", "-d", help="Single date (YYYY-MM-DD)"),
101
+ start: Optional[str] = typer.Option(None, "--start", "-s", help="Start date (YYYY-MM-DD)"),
102
+ end: Optional[str] = typer.Option(None, "--end", "-e", help="End date (YYYY-MM-DD)"),
103
+ token: Optional[str] = typer.Option(None, "--token", "-t"),
104
+ ):
105
+ """List sheets (table of contents) in an archive file.
106
+
107
+ Downloads and parses the first file to show available sheets.
108
+
109
+ \b
110
+ Examples:
111
+ esios archives sheets 34 --date 2025-06-01
112
+ esios archives sheets 34 --start 2025-06-01 --end 2025-06-01
113
+ """
114
+ from esios.processing.i90 import I90Book
115
+
116
+ files = _download_files(archive_id, token, date, start, end)
117
+
118
+ try:
119
+ book = I90Book(files[0])
120
+ except Exception as exc:
121
+ typer.echo(f"Error parsing {files[0].name}: {exc}", err=True)
122
+ raise typer.Exit(1)
123
+
124
+ table = Table(title=f"Sheets in {files[0].name}")
125
+ table.add_column("Sheet", style="cyan")
126
+ table.add_column("Description")
127
+
128
+ for sheet_name, description in book.table_of_contents.items():
129
+ if not isinstance(sheet_name, str) or not sheet_name.strip():
130
+ continue
131
+ table.add_row(str(sheet_name), str(description))
132
+
133
+ console.print(table)
134
+
135
+
136
+ @archives_app.command("exec")
137
+ def exec_archive(
138
+ archive_id: int = typer.Argument(..., help="Archive ID (e.g. 34 for I90DIA)"),
139
+ sheet: str = typer.Option(..., "--sheet", help="Sheet name (e.g. I90DIA03, I90DIA26)"),
140
+ date: Optional[str] = typer.Option(None, "--date", "-d", help="Single date (YYYY-MM-DD)"),
141
+ start: Optional[str] = typer.Option(None, "--start", "-s", help="Start date (YYYY-MM-DD)"),
142
+ end: Optional[str] = typer.Option(None, "--end", "-e", help="End date (YYYY-MM-DD)"),
143
+ expr: str = typer.Option("df", "--expr", "-x", help="Python expression to evaluate (df, pd, np available)"),
144
+ format: str = typer.Option("table", "--format", "-f", help="Output format: table, csv, json"),
145
+ output: Optional[str] = typer.Option(None, "--output", "-o", help="Output file path"),
146
+ token: Optional[str] = typer.Option(None, "--token", "-t"),
147
+ ):
148
+ """Parse archive files and evaluate a Python expression on the data.
149
+
150
+ Downloads archive files (cached), parses the specified sheet from each
151
+ file using I90Book, concatenates the DataFrames, and evaluates the
152
+ expression. The parsed data is available as `df` (pandas DataFrame).
153
+ `pd` (pandas) and `np` (numpy) are also available.
154
+
155
+ \b
156
+ Examples:
157
+ # Show curtailment data for a single day
158
+ esios archives exec 34 --sheet I90DIA03 --date 2025-06-01
159
+
160
+ # Curtailment by technology over a month
161
+ esios archives exec 34 --sheet I90DIA03 -s 2025-05-05 -e 2025-06-08 \\
162
+ -x "df[df['Sentido']=='Bajar'].groupby('Tecnología')['value'].sum().sort_values()"
163
+
164
+ # Export PBF generation program to CSV
165
+ esios archives exec 34 --sheet I90DIA26 --date 2025-06-01 -f csv -o pbf.csv
166
+
167
+ # Descriptive statistics
168
+ esios archives exec 34 --sheet I90DIA03 --date 2025-06-01 -x "df.describe()"
169
+ """
170
+ import numpy as np
171
+ import pandas as pd
172
+
173
+ from esios.processing.i90 import I90Book
174
+
175
+ files = _download_files(archive_id, token, date, start, end)
176
+
177
+ # Parse all files and extract the requested sheet
178
+ all_dfs = []
179
+ for f in files:
180
+ try:
181
+ book = I90Book(f)
182
+ s = book[sheet]
183
+ if s.df is not None and not s.df.empty:
184
+ all_dfs.append(s.df.reset_index())
185
+ except KeyError:
186
+ # Sheet not found — show available sheets
187
+ try:
188
+ book = I90Book(f)
189
+ available = list(book.table_of_contents.keys())
190
+ except Exception:
191
+ available = []
192
+ typer.echo(f"Sheet '{sheet}' not found in {f.name}.", err=True)
193
+ if available:
194
+ typer.echo(f"Available sheets: {', '.join(str(s) for s in available)}", err=True)
195
+ raise typer.Exit(1)
196
+ except Exception as exc:
197
+ typer.echo(f"Warning: skipping {f.name}: {exc}", err=True)
198
+ continue
199
+
200
+ if not all_dfs:
201
+ typer.echo("No data extracted from the specified sheet.", err=True)
202
+ raise typer.Exit(1)
203
+
204
+ df = pd.concat(all_dfs, ignore_index=True)
205
+
206
+ # Evaluate expression
207
+ namespace = {"df": df, "pd": pd, "np": np}
208
+ try:
209
+ result = eval(expr, {"__builtins__": {}}, namespace) # noqa: S307
210
+ except Exception as exc:
211
+ typer.echo(f"Error evaluating expression: {exc}", err=True)
212
+ raise typer.Exit(1)
213
+
214
+ # Render output — reuse _render from exec_cmd
215
+ from esios.cli.exec_cmd import _render
216
+
217
+ _render(result, format, output)
esios/data/__init__.py ADDED
File without changes
File without changes
@@ -0,0 +1,5 @@
1
+ """Archives catalog — static reference of all known ESIOS archives."""
2
+
3
+ from esios.data.catalogs.archives.catalog import ARCHIVES_CATALOG
4
+
5
+ __all__ = ["ARCHIVES_CATALOG"]
@@ -0,0 +1,163 @@
1
+ """Static catalog of ESIOS archives.
2
+
3
+ Auto-generated by refresh.py — do not edit manually.
4
+ """
5
+
6
+ from __future__ import annotations
7
+
8
+ ARCHIVES_CATALOG: dict[int, dict[str, str]] = {
9
+ 2: {"name": "A1_liquicomun", "description": "Avance de liquidación A1", "horizon": "M", "archive_type": "zip"},
10
+ 3: {"name": "A2_liquicomun", "description": "Avance de liquidación A2", "horizon": "M", "archive_type": "zip"},
11
+ 8: {"name": "C2_liquicomun", "description": "Cierre de liquidación C2", "horizon": "M", "archive_type": "zip"},
12
+ 9: {"name": "C3_liquicomun", "description": "Cierre de liquidación C3", "horizon": "M", "archive_type": "zip"},
13
+ 10: {"name": "C4_liquicomun", "description": "Cierre de liquidación C4", "horizon": "M", "archive_type": "zip"},
14
+ 11: {"name": "C5_liquicomun", "description": "Cierre de liquidación C5", "horizon": "M", "archive_type": "zip"},
15
+ 12: {"name": "C6_liquicomun", "description": "Cierre de liquidación C6", "horizon": "M", "archive_type": "zip"},
16
+ 13: {"name": "C7_liquicomun", "description": "Cierre de liquidación C7", "horizon": "M", "archive_type": "zip"},
17
+ 14: {"name": "C8_liquicomun", "description": "Cierre de liquidación C8", "horizon": "M", "archive_type": "zip"},
18
+ 15: {"name": "hemeroteca_DD_ent", "description": "Fichero de hemeroteca de entrada.", "horizon": "D", "archive_type": "zip"},
19
+ 16: {"name": "hemeroteca_DD_sal", "description": "Fichero de hemeroteca de salida", "horizon": "D", "archive_type": "zip"},
20
+ 17: {"name": "liquicierre", "description": "Resultados de la Regulación Secundaria exigidos para la Liquidación", "horizon": "D", "archive_type": "xml"},
21
+ 18: {"name": "totalpdbf", "description": "Programa agregado resultante del PBF", "horizon": "D", "archive_type": "xml"},
22
+ 19: {"name": "totalrpdvpprec", "description": "Solución de Restricciones Técnicas", "horizon": "D", "archive_type": "xml"},
23
+ 20: {"name": "totalpdvp", "description": "PVP Agregado Horizonte Diario", "horizon": "D", "archive_type": "xml"},
24
+ 21: {"name": "totalasigsec", "description": "Asignación Regulación Secundaria", "horizon": "D", "archive_type": "xml"},
25
+ 22: {"name": "totalrpibcirest", "description": "Solución de Restricciones Técnicas MI", "horizon": "H", "archive_type": "xml"},
26
+ 23: {"name": "totalasigdesv", "description": "Mercado de Gestión de Desvíos", "horizon": "H", "archive_type": "xml"},
27
+ 24: {"name": "totalasigter", "description": "Mercado de Regulación Terciaria", "horizon": "D", "archive_type": "xml"},
28
+ 25: {"name": "totalenersec", "description": "Energía Secundaria utilizada", "horizon": "D", "archive_type": "xml"},
29
+ 26: {"name": "totalliquicierre", "description": "Cálculo Variación Coste Fijo", "horizon": "D", "archive_type": "xml"},
30
+ 27: {"name": "totalrp48prec", "description": "Redespachos Seguridad Tiempo Real (Horario)", "horizon": "H", "archive_type": "xml"},
31
+ 28: {"name": "totalrp48preccierre", "description": "Redespachos Seguridad Tiempo Real (Cierre)", "horizon": "D", "archive_type": "xml"},
32
+ 29: {"name": "totalp48", "description": "Programa Agregado P48 (Horario)", "horizon": "H", "archive_type": "xml"},
33
+ 30: {"name": "totalp48cierre", "description": "Programa Agregado P48 (Cierre)", "horizon": "D", "archive_type": "xml"},
34
+ 32: {"name": "I3DIA", "description": "Información agregada diaria por tecnología (I3DIA)", "horizon": "D", "archive_type": "zip"},
35
+ 33: {"name": "IMES", "description": "Cuotas mensuales por Sujetos de Liquidación (IMES)", "horizon": "M", "archive_type": "zip"},
36
+ 34: {"name": "I90DIA", "description": "Información de detalle por Unidad de Programación (I90DIA)", "horizon": "D", "archive_type": "zip"},
37
+ 35: {"name": "COEF_PERD_PEN_MM", "description": "Coeficientes horarios de pérdidas en el sistema peninsular", "horizon": "M", "archive_type": "xls"},
38
+ 36: {"name": "COEF_PERD_MEL_MM", "description": "Coeficientes horarios de pérdidas en el sistema de Melilla", "horizon": "M", "archive_type": "xls"},
39
+ 37: {"name": "COEF_PERD_CEU_MM", "description": "Coeficientes horarios de pérdidas en el sistema de Ceuta", "horizon": "M", "archive_type": "xls"},
40
+ 38: {"name": "COEF_PERD_CAN_MM", "description": "Coeficientes horarios de pérdidas en el sistema de Canarias", "horizon": "M", "archive_type": "xls"},
41
+ 39: {"name": "COEF_PERD_BAL_MM", "description": "Coeficientes horarios de pérdidas en el sistema de Baleares", "horizon": "M", "archive_type": "xls"},
42
+ 40: {"name": "COEF_KEST_MM", "description": "Coeficiente de ajuste horario estimado en el sistema peninsular y no peninsular", "horizon": "M", "archive_type": "xls"},
43
+ 42: {"name": "Espec_Subasta_Diaria_EF", "description": "Especificación de subastas explícitas Diarias de capacidad en la interconexión con Francia", "horizon": "D", "archive_type": "pdf"},
44
+ 43: {"name": "Espec_Subasta_Diaria_FE", "description": "Especificación de subastas explícitas Diarias de capacidad en la interconexión con Francia", "horizon": "D", "archive_type": "pdf"},
45
+ 44: {"name": "Espec_Subasta_Intra1_EF", "description": "Especificación de subastas explícitas Intradiarias 1 de capacidad en la interconexión con Francia", "horizon": "D", "archive_type": "pdf"},
46
+ 45: {"name": "Espec_Subasta_Intra1_FE", "description": "Especificación de subastas explícitas Intradiarias 1 de capacidad en la interconexión con Francia", "horizon": "D", "archive_type": "pdf"},
47
+ 46: {"name": "Espec_Subasta_Intra2_EF", "description": "Especificación de subastas explícitas Intradiarias 2 de capacidad en la interconexión con Francia", "horizon": "D", "archive_type": "pdf"},
48
+ 47: {"name": "Espec_Subasta_Intra2_FE", "description": "Especificación de subastas explícitas Intradiarias 2 de capacidad en la interconexión con Francia", "horizon": "D", "archive_type": "pdf"},
49
+ 48: {"name": "Espec_Subasta_Mensual_EF", "description": "Especificación de subastas explícitas Mensuales de capacidad en la interconexión con Francia", "horizon": "M", "archive_type": "pdf"},
50
+ 49: {"name": "Espec_Subasta_Mensual_FE", "description": "Especificación de subastas explícitas Mensuales de capacidad en la interconexión con Francia", "horizon": "M", "archive_type": "pdf"},
51
+ 50: {"name": "Espec_Subasta_Anual_EF", "description": "Especificación de subastas explícitas Anuales de capacidad en la interconexión con Francia", "horizon": "A", "archive_type": "pdf"},
52
+ 51: {"name": "Espec_Subasta_Anual_FE", "description": "Especificación de subastas explícitas Anuales de capacidad en la interconexión con Francia", "horizon": "A", "archive_type": "pdf"},
53
+ 56: {"name": "Espec_Subasta_Intra2_EF_Cancel", "description": "Especificación de subastas explícitas Intradiarias 2 de capacidad en la interconexión con Francia", "horizon": "D", "archive_type": "pdf"},
54
+ 57: {"name": "Espec_Subasta_Intra2_FE_Cancel", "description": "Especificación de subastas explícitas Intradiarias 2 de capacidad en la interconexión con Francia", "horizon": "D", "archive_type": "pdf"},
55
+ 58: {"name": "Espec_Subasta_Mensual_EF_Cancel", "description": "Especificación de subastas explícitas Mensuales de capacidad en la interconexión con Francia", "horizon": "M", "archive_type": "pdf"},
56
+ 59: {"name": "Espec_Subasta_Mensual_FE_Cancel", "description": "Especificación de subastas explícitas Mensuales de capacidad en la interconexión con Francia", "horizon": "M", "archive_type": "pdf"},
57
+ 62: {"name": "IND_DemandaInterrumpible", "description": "IND_DemandaInterrumpible", "horizon": "D", "archive_type": "json"},
58
+ 63: {"name": "IND_Interconexiones", "description": "IND_Interconexiones", "horizon": "D", "archive_type": "json"},
59
+ 64: {"name": "IND_PotenciaInstalada", "description": "IND_PotenciaInstalada", "horizon": "M", "archive_type": "json"},
60
+ 65: {"name": "IND_PrecioDesvios", "description": "IND_PrecioDesvios", "horizon": "D", "archive_type": "json"},
61
+ 66: {"name": "IND_PrecioFinal", "description": "IND_PrecioFinal", "horizon": "D", "archive_type": "json"},
62
+ 67: {"name": "IND_Umbrales", "description": "IND_Umbrales", "horizon": "D", "archive_type": "json"},
63
+ 68: {"name": "INF_SUBASTA_AA", "description": "Informe anual de subastas de capacidad con Francia", "horizon": "A", "archive_type": "pdf"},
64
+ 69: {"name": "INF_SUBASTA_MM", "description": "Informe mensual de subastas de capacidad con Francia", "horizon": "M", "archive_type": "pdf"},
65
+ 70: {"name": "PVPC_CURV_DD", "description": "Datos origen de curva PVPC", "horizon": "D", "archive_type": "json"},
66
+ 71: {"name": "PVPC_DETALLE_DD", "description": "PVPC Término de facturación energía activa – Desglose", "horizon": "D", "archive_type": "xls"},
67
+ 72: {"name": "PVPC_GEN_P1_DD", "description": "Datos origen de tarifa general P1", "horizon": "D", "archive_type": "json"},
68
+ 73: {"name": "PVPC_NOC_P1_DD", "description": "Datos origen de tarifa nocturna P1", "horizon": "D", "archive_type": "json"},
69
+ 74: {"name": "PVPC_NOC_P2_DD", "description": "Datos origen de tarifa nocturna P2", "horizon": "D", "archive_type": "json"},
70
+ 75: {"name": "PVPC_VHC_P1_DD", "description": "Datos origen de tarifa vehículo eléctrico P1", "horizon": "D", "archive_type": "json"},
71
+ 76: {"name": "PVPC_VHC_P2_DD", "description": "Datos origen de tarifa vehículo eléctrico P2", "horizon": "D", "archive_type": "json"},
72
+ 77: {"name": "PVPC_VHC_P3_DD", "description": "Datos origen de tarifa vehículo eléctrico P3", "horizon": "D", "archive_type": "json"},
73
+ 78: {"name": "perfilconsumo", "description": "PVPC Coeficientes de perfilado", "horizon": "S", "archive_type": "xml"},
74
+ 79: {"name": "preciovoluntariopconsumidor", "description": "PVPC por ciclo de lectura", "horizon": "D", "archive_type": "xml"},
75
+ 80: {"name": "pvpcdesglosehorario", "description": "PVPC Término de facturación energía activa", "horizon": "D", "archive_type": "xml"},
76
+ 81: {"name": "UnidadesFisicas", "description": "Datos Estructurales de unidades físicas", "horizon": "NA", "archive_type": "json"},
77
+ 82: {"name": "UnidadesProgramacion", "description": "Datos estructurales de unidades de programación", "horizon": "NA", "archive_type": "json"},
78
+ 83: {"name": "SujetosMercado", "description": "Datos estructurales de sujetos de mercado", "horizon": "NA", "archive_type": "json"},
79
+ 84: {"name": "ParticipantesSubasta", "description": "Datos estructurales de participantes de subasta", "horizon": "NA", "archive_type": "json"},
80
+ 87: {"name": "Resultado_Subasta_Mensual_FRA", "description": "Subastas mensuales del calendariode subastas de Francia", "horizon": "M", "archive_type": "xls"},
81
+ 88: {"name": "Resultado_Subasta_Anual_FRA", "description": "Subastas anuales del calendariode subastas de Francia", "horizon": "A", "archive_type": "xls"},
82
+ 89: {"name": "Descargos_AND_Planificados", "description": "Descargos Andorra", "horizon": "O", "archive_type": "xml"},
83
+ 91: {"name": "Descargos_MAR_Planificados", "description": "Descargos Marruecos", "horizon": "O", "archive_type": "xml"},
84
+ 93: {"name": "Descargos_POR_Planificados", "description": "Descargos Portugal", "horizon": "O", "archive_type": "xml"},
85
+ 95: {"name": "Descargos_FRA_Planificados", "description": "Descargos Francia", "horizon": "O", "archive_type": "xml"},
86
+ 101: {"name": "REE_InterChangeAvailab_FRA", "description": "Cambios en disponibilidad real de capacidad en interconexión con Francia", "horizon": "O", "archive_type": "xml"},
87
+ 102: {"name": "REE_InterChangeAvailab_POR", "description": "Cambios en disponibilidad real de capacidad en interconexión con Portugal", "horizon": "O", "archive_type": "xml"},
88
+ 105: {"name": "Indisponibilidades", "description": "Indisponibilidades de las unidades de generación y consumo", "horizon": "D", "archive_type": "xls"},
89
+ 106: {"name": "PlanesMantenimiento", "description": "Plan de mantenimiento", "horizon": "M", "archive_type": "xls"},
90
+ 107: {"name": "p48cierre", "description": "Cierre de programa Desagregado P48", "horizon": "D", "archive_type": "xml"},
91
+ 108: {"name": "REE_AggGenOutput", "description": "Generación real por tipo de producción", "horizon": "QM", "archive_type": "xml"},
92
+ 109: {"name": "REE_ActualGenOutput", "description": "Generación real por unidad física", "horizon": "D", "archive_type": "xml"},
93
+ 110: {"name": "GenerationUnits", "description": "Generation Units Structural Data", "horizon": "NA", "archive_type": "json"},
94
+ 111: {"name": "ProgrammingUnits", "description": "Programming Units Structural Data", "horizon": "NA", "archive_type": "json"},
95
+ 112: {"name": "BalanceResponsibleParties", "description": "Balance Responsible Parties Structural Data", "horizon": "NA", "archive_type": "json"},
96
+ 113: {"name": "EntitledParticipants", "description": "Entitled Participants Structural Data", "horizon": "NA", "archive_type": "json"},
97
+ 114: {"name": "IND_DemandaPrevProg", "description": "IND_DemandaPrevProg", "horizon": "D", "archive_type": "json"},
98
+ 115: {"name": "IND_DemandaRealGen", "description": "IND_DemandaRealGen", "horizon": "D", "archive_type": "json"},
99
+ 116: {"name": "IND_MaxMin", "description": "IND_MaxMin", "horizon": "D", "archive_type": "json"},
100
+ 117: {"name": "IND_MaxMinRenovEol", "description": "IND_MaxMinRenovEol", "horizon": "D", "archive_type": "json"},
101
+ 118: {"name": "ActividadesSubactividades", "description": "Informe de actividades y subactividades", "horizon": "NA", "archive_type": "pdf"},
102
+ 119: {"name": "AgregacionesValidas", "description": "Informe de agregaciones válidas", "horizon": "NA", "archive_type": "pdf"},
103
+ 120: {"name": "CNAEValidos", "description": "Informe de códigos CNAE válidos", "horizon": "NA", "archive_type": "pdf"},
104
+ 121: {"name": "Comercializadores", "description": "Informe de comercializadores", "horizon": "NA", "archive_type": "pdf"},
105
+ 122: {"name": "Concentradores", "description": "Informe de concentradores", "horizon": "NA", "archive_type": "pdf"},
106
+ 123: {"name": "DiscriminacionHoraria", "description": "Informe de discriminaciones horarias", "horizon": "NA", "archive_type": "pdf"},
107
+ 124: {"name": "Distribuidores", "description": "Informe de distribuidores", "horizon": "NA", "archive_type": "pdf"},
108
+ 125: {"name": "Fabricantes", "description": "Informe de fabricantes", "horizon": "NA", "archive_type": "pdf"},
109
+ 126: {"name": "Magnitudes", "description": "Informe de magnitudes", "horizon": "NA", "archive_type": "pdf"},
110
+ 127: {"name": "ModeloContadores", "description": "Informe de modelo de contadores", "horizon": "NA", "archive_type": "pdf"},
111
+ 128: {"name": "ModeloRegistradores", "description": "Informe de modelos de registradores", "horizon": "NA", "archive_type": "pdf"},
112
+ 129: {"name": "ModeloTransformadores", "description": "Informe de modelo de transformadores", "horizon": "NA", "archive_type": "pdf"},
113
+ 130: {"name": "NivelesTension", "description": "Informe de niveles de tensión", "horizon": "NA", "archive_type": "pdf"},
114
+ 131: {"name": "Provincias", "description": "Informe de provincias", "horizon": "NA", "archive_type": "pdf"},
115
+ 132: {"name": "Representantes", "description": "Informe de representantes", "horizon": "NA", "archive_type": "pdf"},
116
+ 133: {"name": "Subsistemas", "description": "Informe de subsistemas", "horizon": "NA", "archive_type": "pdf"},
117
+ 134: {"name": "ZonasGeograficas", "description": "Informe de zonas geográficas", "horizon": "NA", "archive_type": "pdf"},
118
+ 135: {"name": "TarifaAcceso", "description": "Informe de tarifas de acceso", "horizon": "NA", "archive_type": "pdf"},
119
+ 136: {"name": "A6_liquicomun", "description": "Avance de liquidación A6", "horizon": "M", "archive_type": "zip"},
120
+ 137: {"name": "A7_liquicomun", "description": "Avance de liquidación A7", "horizon": "M", "archive_type": "zip"},
121
+ 139: {"name": "IND_EnergiaMensual", "description": "IND_EnergiaMensual", "horizon": "M", "archive_type": "json"},
122
+ 140: {"name": "IND_EnergiaAnual", "description": "IND_EnergiaAnual", "horizon": "A", "archive_type": "json"},
123
+ 142: {"name": "ConsumidoresDirectos", "description": "Informe de Consumidores Directos", "horizon": "NA", "archive_type": "pdf"},
124
+ 143: {"name": "MotivosObjecionEdLD", "description": "Informe de Motivos Objeción EdLD", "horizon": "NA", "archive_type": "pdf"},
125
+ 144: {"name": "MotivosObjecionEdLOS", "description": "Informe de Motivos Objeción EdLOS", "horizon": "NA", "archive_type": "pdf"},
126
+ 145: {"name": "TiposMedidaPuntoMedida", "description": "Informe de tipos de medida de punto medida", "horizon": "NA", "archive_type": "pdf"},
127
+ 146: {"name": "TiposMedidaPuntoFrontera", "description": "Informe de tipos de medida de punto frontera", "horizon": "NA", "archive_type": "pdf"},
128
+ 147: {"name": "Espec_Subasta_Mensual_EP", "description": "Especificación de subastas explícitas Mensuales de capacidad en la interconexión con Portugal", "horizon": "M", "archive_type": "pdf"},
129
+ 148: {"name": "Espec_Subasta_Mensual_PE", "description": "Especificación de subastas explícitas Mensuales de capacidad en la interconexión con Portugal", "horizon": "M", "archive_type": "pdf"},
130
+ 149: {"name": "Espec_Subasta_Trimestral_EP", "description": "Especificación de subastas explícitas Trimestrales de capacidad en la interconexión con Portugal", "horizon": "T", "archive_type": "pdf"},
131
+ 150: {"name": "Espec_Subasta_Trimestral_PE", "description": "Especificación de subastas explícitas Trimestrales de capacidad en la interconexión con Portugal", "horizon": "T", "archive_type": "pdf"},
132
+ 151: {"name": "Espec_Subasta_Anual_EP", "description": "Especificación de subastas explícitas Trimestrales de capacidad en la interconexión con Portugal", "horizon": "A", "archive_type": "pdf"},
133
+ 152: {"name": "Espec_Subasta_Anual_PE", "description": "Especificación de subastas explícitas Trimestrales de capacidad en la interconexión con Portugal", "horizon": "A", "archive_type": "pdf"},
134
+ 159: {"name": "Resultado_Subasta_Mensual_POR", "description": "Subastas mensuales del calendario de subastas de Portugal", "horizon": "M", "archive_type": "xls"},
135
+ 160: {"name": "Resultado_Subasta_Anual_POR", "description": "Subastas anuales del calendario de subastas de Portugal", "horizon": "A", "archive_type": "xls"},
136
+ 161: {"name": "Resultado_Subasta_Trimestral_POR", "description": "Subastas trimestrales del calendario de subastas de Portugal", "horizon": "T", "archive_type": "xls"},
137
+ 164: {"name": "CodigosTensiones", "description": "Informe de códigos de tensiones de suministro", "horizon": "NA", "archive_type": "pdf"},
138
+ 165: {"name": "UnidadesConsumoGeneracion", "description": "Informe de unidades de consumo de generación", "horizon": "NA", "archive_type": "pdf"},
139
+ 166: {"name": "CodigosAutoconsumo", "description": "Informe de códigos de autoconsumo", "horizon": "NA", "archive_type": "pdf"},
140
+ 167: {"name": "IND_PotenciaInstaladaNacional", "description": "IND_PotenciaInstaladaNacional", "horizon": "M", "archive_type": "json"},
141
+ 168: {"name": "IND_EnergiaMensualNacional", "description": "IND_EnergiaMensualNacional", "horizon": "M", "archive_type": "json"},
142
+ 169: {"name": "IND_EnergiaAnualNacional", "description": "IND_EnergiaAnualNacional", "horizon": "A", "archive_type": "json"},
143
+ 170: {"name": "IND_CoeficientesCO2", "description": "IND_CoeficientesCO2", "horizon": "NA", "archive_type": "json"},
144
+ 171: {"name": "IND_InformacionApp", "description": "IND_InformacionApp", "horizon": "NA", "archive_type": "json"},
145
+ 172: {"name": "IND_PreguntasApp", "description": "IND_PreguntasApp", "horizon": "NA", "archive_type": "json"},
146
+ 173: {"name": "IND_VersionApp", "description": "IND_VersionApp", "horizon": "NA", "archive_type": "json"},
147
+ 174: {"name": "IND_PrecioMinorista", "description": "IND_PrecioMinorista", "horizon": "D", "archive_type": "json"},
148
+ 181: {"name": "REE_BalancingEnerBids", "description": "Mercado de Regulación Terciaria", "horizon": "QM", "archive_type": "csv"},
149
+ 183: {"name": "IND_EnergiaAnualAutoconsumoNacional", "description": "IND_EnergiaAnualAutoconsumoNacional", "horizon": "A", "archive_type": "json"},
150
+ 184: {"name": "IND_EnergiaMensualAutoconsumoNacional", "description": "IND_EnergiaMensualAutoconsumoNacional", "horizon": "M", "archive_type": "json"},
151
+ 185: {"name": "ActivacionesDelServicio", "description": "Activaciones del Servicio", "horizon": "NA", "archive_type": "json"},
152
+ 186: {"name": "ServiceActivation", "description": "Service Activation", "horizon": "NA", "archive_type": "json"},
153
+ 187: {"name": "C2_PrecioFinal", "description": "Precio final de la energía C2", "horizon": "M", "archive_type": "zip"},
154
+ 188: {"name": "C5_PrecioFinal", "description": "Precio final de la energía C5", "horizon": "M", "archive_type": "zip"},
155
+ 193: {"name": "IND_Novedades", "description": "IND_Novedades", "horizon": "NA", "archive_type": "json"},
156
+ 194: {"name": "IND_Novedades_Paso1", "description": "Json lottie Paso1", "horizon": "NA", "archive_type": "json"},
157
+ 195: {"name": "IND_Novedades_Paso2", "description": "Json lottie Paso2", "horizon": "NA", "archive_type": "json"},
158
+ 196: {"name": "IND_Novedades_Paso3", "description": "Json lottie Paso3", "horizon": "NA", "archive_type": "json"},
159
+ 197: {"name": "IND_Novedades_Paso4", "description": "Json lottie Paso3", "horizon": "NA", "archive_type": "json"},
160
+ 198: {"name": "IND_Novedades_Step1", "description": "Json lottie Step1", "horizon": "NA", "archive_type": "json"},
161
+ 199: {"name": "IND_Novedades_Step2", "description": "Json lottie Step2", "horizon": "NA", "archive_type": "json"},
162
+ }
163
+
@@ -0,0 +1,95 @@
1
+ """Refresh the static archives catalog by scanning the ESIOS API.
2
+
3
+ Usage:
4
+ uv run python -m esios.data.catalogs.archives.refresh
5
+
6
+ Scans archive IDs 1-200 against the live API and regenerates catalog.py.
7
+ """
8
+
9
+ from __future__ import annotations
10
+
11
+ import sys
12
+ from pathlib import Path
13
+
14
+
15
+ def scan_archives(max_id: int = 200) -> dict[int, dict[str, str]]:
16
+ """Scan ESIOS API for all accessible archives."""
17
+ from esios import ESIOSClient
18
+
19
+ client = ESIOSClient()
20
+ catalog: dict[int, dict[str, str]] = {}
21
+
22
+ for i in range(1, max_id + 1):
23
+ try:
24
+ data = client.get(f"archives/{i}")
25
+ a = data.get("archive", {})
26
+ catalog[i] = {
27
+ "name": a.get("name", ""),
28
+ "description": a.get("description", ""),
29
+ "horizon": a.get("horizon", ""),
30
+ "archive_type": a.get("archive_type", ""),
31
+ }
32
+ except Exception:
33
+ continue
34
+
35
+ return catalog
36
+
37
+
38
+ def generate_catalog_py(catalog: dict[int, dict[str, str]]) -> str:
39
+ """Generate the catalog.py source code."""
40
+ lines = [
41
+ '"""Static catalog of ESIOS archives.',
42
+ "",
43
+ "Auto-generated by refresh.py — do not edit manually.",
44
+ '"""',
45
+ "",
46
+ "from __future__ import annotations",
47
+ "",
48
+ "ARCHIVES_CATALOG: dict[int, dict[str, str]] = {",
49
+ ]
50
+
51
+ for id_ in sorted(catalog):
52
+ entry = catalog[id_]
53
+ name = entry["name"]
54
+ desc = entry["description"].replace('"', '\\"')
55
+ horizon = entry["horizon"]
56
+ atype = entry["archive_type"]
57
+ lines.append(
58
+ f' {id_}: {{"name": "{name}", "description": "{desc}", '
59
+ f'"horizon": "{horizon}", "archive_type": "{atype}"}},'
60
+ )
61
+
62
+ lines.append("}")
63
+ lines.append("")
64
+ return "\n".join(lines)
65
+
66
+
67
+ def main() -> None:
68
+ from esios.data.catalogs.archives.catalog import ARCHIVES_CATALOG
69
+
70
+ old_ids = set(ARCHIVES_CATALOG.keys())
71
+
72
+ print(f"Scanning ESIOS API for archives (IDs 1-200)...")
73
+ catalog = scan_archives()
74
+ new_ids = set(catalog.keys())
75
+
76
+ # Diff summary
77
+ added = new_ids - old_ids
78
+ removed = old_ids - new_ids
79
+
80
+ print(f"\nFound {len(catalog)} archives (was {len(old_ids)})")
81
+ if added:
82
+ print(f" Added: {sorted(added)}")
83
+ if removed:
84
+ print(f" Removed: {sorted(removed)}")
85
+ if not added and not removed:
86
+ print(" No changes in archive IDs.")
87
+
88
+ # Write catalog.py
89
+ catalog_path = Path(__file__).parent / "catalog.py"
90
+ catalog_path.write_text(generate_catalog_py(catalog))
91
+ print(f"\nWrote {catalog_path}")
92
+
93
+
94
+ if __name__ == "__main__":
95
+ main()
@@ -76,20 +76,20 @@ class ArchiveHandle:
76
76
  date: str | None = None,
77
77
  output_dir: str | Path | None = None,
78
78
  date_type: str = "datos",
79
- ) -> Path:
79
+ ) -> list[Path]:
80
80
  """Download archive files for a single date or date range.
81
81
 
82
82
  Files are always stored in the cache directory. If ``output_dir`` is
83
83
  provided, a copy is placed there as well.
84
84
 
85
- Returns the cache directory where files were stored.
85
+ Returns a sorted list of downloaded/cached file paths.
86
86
  """
87
87
  if date and not (start and end):
88
88
  self.configure(date=date, date_type=date_type)
89
89
  cache_folder = self._download_single()
90
90
  if output_dir:
91
91
  self._copy_to_output(cache_folder, Path(output_dir))
92
- return cache_folder
92
+ return sorted(f for f in cache_folder.iterdir() if f.is_file())
93
93
 
94
94
  if not (start and end):
95
95
  raise ValueError("Provide 'date' or both 'start' and 'end'.")
@@ -101,7 +101,7 @@ class ArchiveHandle:
101
101
  horizon = self.metadata.get("archive", {}).get("horizon", "D")
102
102
  archive_type = self.metadata.get("archive", {}).get("archive_type", "zip")
103
103
  current = start_date
104
- last_folder: Path | None = None
104
+ files: list[Path] = []
105
105
 
106
106
  while current <= end_date:
107
107
  if horizon == "M":
@@ -118,7 +118,7 @@ class ArchiveHandle:
118
118
  logger.info("Cache hit: %s", cache_folder)
119
119
  if output_dir:
120
120
  self._copy_to_output(cache_folder, Path(output_dir))
121
- last_folder = cache_folder
121
+ files.extend(f for f in cache_folder.iterdir() if f.is_file())
122
122
  current = chunk_end + timedelta(days=1)
123
123
  continue
124
124
 
@@ -136,7 +136,7 @@ class ArchiveHandle:
136
136
  # Write to cache
137
137
  cache_folder = self._cache.archive_dir(self.id, self.name, key)
138
138
  self._write_content(content, cache_folder, key, archive_type)
139
- last_folder = cache_folder
139
+ files.extend(f for f in cache_folder.iterdir() if f.is_file())
140
140
 
141
141
  # Copy to output if requested
142
142
  if output_dir:
@@ -144,7 +144,7 @@ class ArchiveHandle:
144
144
 
145
145
  current = chunk_end + timedelta(days=1)
146
146
 
147
- return last_folder or self._cache.archive_dir(self.id, self.name, "")
147
+ return sorted(files)
148
148
 
149
149
  # -- Internal helpers ------------------------------------------------------
150
150
 
@@ -199,12 +199,25 @@ class ArchiveHandle:
199
199
  class ArchivesManager(BaseManager):
200
200
  """Manager for ``/archives`` endpoints."""
201
201
 
202
- def list(self) -> pd.DataFrame:
203
- """List all available archives as a DataFrame."""
204
- data = self._get("archives")
205
- df = pd.DataFrame(data.get("archives", []))
206
- if "id" in df.columns:
207
- df = df.set_index("id")
202
+ def list(self, *, source: str = "local") -> pd.DataFrame:
203
+ """List all available archives as a DataFrame.
204
+
205
+ Args:
206
+ source: ``"local"`` (default) returns the full static catalog
207
+ (153 archives including I90, settlements, etc.).
208
+ ``"api"`` queries the ESIOS API which only returns ~24 archives.
209
+ """
210
+ if source == "api":
211
+ data = self._get("archives", params={"date_type": "publicacion"})
212
+ df = pd.DataFrame(data.get("archives", []))
213
+ if "id" in df.columns:
214
+ df = df.set_index("id")
215
+ return df
216
+
217
+ from esios.data.catalogs.archives import ARCHIVES_CATALOG
218
+
219
+ df = pd.DataFrame.from_dict(ARCHIVES_CATALOG, orient="index")
220
+ df.index.name = "id"
208
221
  return df
209
222
 
210
223
  def get(self, archive_id: int) -> ArchiveHandle:
@@ -225,10 +238,10 @@ class ArchivesManager(BaseManager):
225
238
  date: str | None = None,
226
239
  output_dir: str | Path | None = None,
227
240
  date_type: str = "datos",
228
- ) -> Path:
241
+ ) -> list[Path]:
229
242
  """Convenience method: get + download in one call.
230
243
 
231
- Returns the cache directory where files were stored.
244
+ Returns a sorted list of downloaded/cached file paths.
232
245
  """
233
246
  handle = self.get(archive_id)
234
247
  return handle.download(
@@ -385,6 +385,11 @@ class IndicatorsManager(BaseManager):
385
385
  indicator = Indicator.from_api(raw)
386
386
  handle = IndicatorHandle(self, indicator)
387
387
 
388
+ # Enrich geos from values returned in metadata response.
389
+ # Some indicators (e.g. province-level breakdown) have an empty
390
+ # "geos" field in the metadata but geo_id/geo_name in the values.
391
+ handle._enrich_geo_map(raw.get("values", []))
392
+
388
393
  # Persist metadata and geos
389
394
  if cache.config.enabled:
390
395
  handle._persist_meta()
esios/processing/i90.py CHANGED
@@ -8,11 +8,15 @@ from __future__ import annotations
8
8
 
9
9
  import logging
10
10
  from pathlib import Path
11
+ from typing import TYPE_CHECKING
11
12
 
12
13
  import numpy as np
13
14
  import pandas as pd
14
15
  import python_calamine
15
16
 
17
+ if TYPE_CHECKING:
18
+ from esios.managers.archives import ArchiveHandle
19
+
16
20
  logger = logging.getLogger("esios")
17
21
 
18
22
 
@@ -26,7 +30,7 @@ def _get_idx_column_start(columns: np.ndarray) -> int:
26
30
 
27
31
  def _any_value_greater_than_30(series: np.ndarray) -> bool:
28
32
  """Check if any numeric value exceeds 30 (quarter-hourly indicator)."""
29
- return any(v > 30 for v in series if isinstance(v, (int, float)) and not np.isnan(v))
33
+ return any(v > 30 for v in series if isinstance(v, (int, float, np.integer, np.floating)) and not np.isnan(v))
30
34
 
31
35
 
32
36
  class I90Book:
@@ -43,26 +47,34 @@ class I90Book:
43
47
  self.path = Path(path)
44
48
  self.metadata: dict = {}
45
49
  self.table_of_contents: dict[str, str] = {}
46
- self.sheets: dict[str, I90Sheet] = {}
50
+ self._sheets: dict[str, I90Sheet] = {}
51
+ self._workbook: python_calamine.CalamineWorkbook | None = None
52
+ self._sheet_names: list[str] = []
47
53
  self._extract_metadata_and_toc()
48
54
 
49
- def _read_excel(self) -> None:
50
- with open(self.path, "rb") as f:
51
- self.workbook = python_calamine.CalamineWorkbook.from_filelike(f)
52
- self.sheets = {
53
- name: I90Sheet(name, self.workbook, self.path, self.metadata)
54
- for name in self.workbook.sheet_names
55
- }
55
+ @property
56
+ def sheets(self) -> dict[str, I90Sheet]:
57
+ """Access already-loaded sheets."""
58
+ return self._sheets
59
+
60
+ def _open_workbook(self) -> python_calamine.CalamineWorkbook:
61
+ """Open the workbook (cached after first call)."""
62
+ if self._workbook is None:
63
+ with open(self.path, "rb") as f:
64
+ self._workbook = python_calamine.CalamineWorkbook.from_filelike(f)
65
+ self._sheet_names = self._workbook.sheet_names
66
+ return self._workbook
56
67
 
57
68
  def _extract_metadata_and_toc(self) -> None:
58
69
  """Extract dates and table of contents from the first sheet."""
59
- self._read_excel()
60
- first_sheet = self.sheets[next(iter(self.sheets))]
61
- rows = first_sheet.rows
70
+ wb = self._open_workbook()
71
+ first_name = self._sheet_names[0]
72
+ first_sheet = I90Sheet(first_name, wb, self.path, self.metadata)
73
+ self._sheets[first_name] = first_sheet
62
74
 
63
75
  # Dates from row 4
64
- self.metadata["date_data"] = pd.to_datetime(rows[3][0])
65
- self.metadata["date_publication"] = pd.to_datetime(rows[3][2])
76
+ self.metadata["date_data"] = pd.to_datetime(first_sheet.rows[3][0])
77
+ self.metadata["date_publication"] = pd.to_datetime(first_sheet.rows[3][2])
66
78
 
67
79
  # Table of contents from row 10 onwards
68
80
  df = pd.read_excel(self.path, sheet_name=0, header=None, skiprows=9, usecols="A,B", engine="calamine")
@@ -70,12 +82,13 @@ class I90Book:
70
82
  self.table_of_contents = df.set_index("sheet_name")["description"].to_dict()
71
83
 
72
84
  def get_sheet(self, sheet_name: str) -> I90Sheet:
73
- """Get and preprocess a specific sheet by name."""
74
- if sheet_name not in self.sheets:
75
- self._read_excel()
76
- sheet = self.sheets.get(sheet_name)
77
- if sheet is None:
78
- raise KeyError(f"Sheet '{sheet_name}' not found in {self.path.name}")
85
+ """Get and preprocess a specific sheet by name (lazy — only reads on demand)."""
86
+ if sheet_name not in self._sheets:
87
+ wb = self._open_workbook()
88
+ if sheet_name not in self._sheet_names:
89
+ raise KeyError(f"Sheet '{sheet_name}' not found in {self.path.name}")
90
+ self._sheets[sheet_name] = I90Sheet(sheet_name, wb, self.path, self.metadata)
91
+ sheet = self._sheets[sheet_name]
79
92
  if sheet.df is None:
80
93
  sheet.df = sheet._preprocess()
81
94
  return sheet
@@ -83,6 +96,36 @@ class I90Book:
83
96
  def __getitem__(self, sheet_name: str) -> I90Sheet:
84
97
  return self.get_sheet(sheet_name)
85
98
 
99
+ @classmethod
100
+ def from_archive(
101
+ cls,
102
+ archive: ArchiveHandle,
103
+ *,
104
+ start: str,
105
+ end: str,
106
+ ) -> list[I90Book]:
107
+ """Download I90 files and parse them into I90Book objects.
108
+
109
+ Calls ``archive.download()`` (cache-aware), then parses each file.
110
+ Files that fail to parse are logged and skipped.
111
+
112
+ Args:
113
+ archive: An :class:`ArchiveHandle` from ``client.archives.get(34)``.
114
+ start: Start date (``"YYYY-MM-DD"``).
115
+ end: End date (``"YYYY-MM-DD"``).
116
+
117
+ Returns:
118
+ A list of successfully parsed :class:`I90Book` objects, sorted by date.
119
+ """
120
+ files = archive.download(start=start, end=end)
121
+ books: list[I90Book] = []
122
+ for f in files:
123
+ try:
124
+ books.append(cls(f))
125
+ except Exception as e:
126
+ logger.warning("Failed to parse %s: %s", f.name, e)
127
+ return books
128
+
86
129
  def __repr__(self) -> str:
87
130
  return f"<I90Book {self.path.name} sheets={len(self.sheets)}>"
88
131
 
@@ -121,15 +164,36 @@ class I90Sheet:
121
164
  return arr
122
165
 
123
166
  def _normalize_datetime_columns(self, columns: np.ndarray) -> np.ndarray:
124
- """Normalize time column headers to integer period indices."""
167
+ """Normalize time column headers to integer period indices.
168
+
169
+ Handles three column formats found in I90 files:
170
+ - Sequential integers 1–24 (hourly) or 1–96 (quarterly)
171
+ - H-Q format with dash notation: "1-1", "1-2", "1-3", "1-4", "2-1", …
172
+ - NaN-filler format: [1, NaN, NaN, NaN, 2, …] (one label per hour,
173
+ three trailing NaNs for quarters 2–4)
174
+ """
125
175
  if any(pd.isna(columns)):
126
176
  self._n_columns_totals = 3
127
177
  else:
128
178
  self._n_columns_totals = 2
129
179
 
130
180
  series = pd.Series(columns, dtype=str).ffill()
131
- series = series.str.split("-").str[0]
132
- return series.astype(float).astype(int).values
181
+ parts = series.str.split("-")
182
+ hours = parts.str[0].astype(float).astype(int)
183
+
184
+ # H-Q format: any column carries an explicit quarter suffix (e.g. "1-2")
185
+ if (parts.str.len() > 1).any():
186
+ quarters = parts.apply(
187
+ lambda x: int(x[1]) if len(x) > 1 and str(x[1]).isdigit() else 1
188
+ )
189
+ return ((hours - 1) * 4 + quarters).values
190
+
191
+ # NaN-filler quarterly format: after ffill the same hour number repeats
192
+ # four times (quarters share the hour label). Assign sequential indices.
193
+ if hours.duplicated().any():
194
+ return np.arange(1, len(hours) + 1)
195
+
196
+ return hours.values
133
197
 
134
198
  def _preprocess_double_index(
135
199
  self, idx: int, columns: np.ndarray
@@ -192,7 +256,7 @@ class I90Sheet:
192
256
  columns_datetime = base_date + pd.to_timedelta(time_deltas, unit="m")
193
257
  columns_datetime = pd.DatetimeIndex(columns_datetime).tz_localize(
194
258
  "Europe/Madrid", ambiguous="infer"
195
- ).tz_convert("UTC")
259
+ )
196
260
 
197
261
  data = pd.DataFrame(self.rows[idx + 1 :], columns=columns)
198
262
 
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: python-esios
3
- Version: 2.0.0
3
+ Version: 2.0.2
4
4
  Summary: A Python wrapper for the ESIOS API (Spanish electricity market)
5
5
  Project-URL: Homepage, https://github.com/datons/python-esios
6
6
  Project-URL: Repository, https://github.com/datons/python-esios
@@ -4,18 +4,24 @@ esios/cache.py,sha256=GgbrL9Rc9aLrEWHvXtQOCGQRgq2T4m6VBJDvBJfWMTk,18920
4
4
  esios/client.py,sha256=rE06YD70oTFA7Tc9Okf5Rcsntnx9W_3U4A6rbZlfsGc,5695
5
5
  esios/constants.py,sha256=pwB2UlBI96zYBA8wAbcCSHcm_E-aIj2hBarDA8t1Vp8,474
6
6
  esios/exceptions.py,sha256=AiWLdRDWj50JEsld9CvVBsfLnZZKFmW62_bZmZ7Z_eA,899
7
+ esios/.agents/skills/esios/SKILL.md,sha256=Z7ltdPCtJPA4cMAI8_94jy_12laq-356PJ78TqfY2NQ,6984
7
8
  esios/cli/__init__.py,sha256=9gd5ZDIH1-yNP_xcd60ethOFXm9w6un0CJ9CX0Qvb2A,256
8
9
  esios/cli/app.py,sha256=JL-4QWlzu5UKd6mq3-bEmWn8YR463edGRNUcv6rv8L8,1370
9
- esios/cli/archives.py,sha256=gGlPGhVuRddWBWwiaKtL-tFTgNrQMbZdFIiresoEOHc,2100
10
+ esios/cli/archives.py,sha256=Re9ZMauTiJlHdmiE7F3ZlV2wfaEyShS0C7Z4M2X4Ra8,7715
10
11
  esios/cli/cache_cmd.py,sha256=TWOn1UrkxK1paq_bbDHfvBd_OrQYcmoYtUG65evohOA,3165
11
12
  esios/cli/config.py,sha256=uYnCEy6NPkzX5tCrwYia4L24J-G9qw-rgTrN8KLDBVQ,1676
12
13
  esios/cli/config_cmd.py,sha256=KWZ6Uc4VvmP4wHl0CukqMjZirTjf_FiwFEuP8XI-lNA,923
13
14
  esios/cli/exec_cmd.py,sha256=BlBzyVaTxg8rwgTfTsMCy5MGrAkeQJ6VB3IE2szOoyA,4485
14
15
  esios/cli/indicators.py,sha256=xkR7vZr376W9r43FGAxG1oeq-LyIUA6Iz88HvCYsxa8,6709
16
+ esios/data/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
17
+ esios/data/catalogs/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
18
+ esios/data/catalogs/archives/__init__.py,sha256=ewGnzX2uPoB4l_ktBSD0UeLPDqc1_tw_6hJ7ueZj8DM,172
19
+ esios/data/catalogs/archives/catalog.py,sha256=P7YpTJAsMKDHE05gjt4T7R5ZrXg-E61SvTMQbSlQatQ,21760
20
+ esios/data/catalogs/archives/refresh.py,sha256=VxuUHdtF4rQgN-jq5ewiOge-J3ok7FivalvYDGQej9A,2662
15
21
  esios/managers/__init__.py,sha256=-1AwL7arUf7WEZn1RSiK_DZhY3j6U4GE9_dqjbukCJc,268
16
- esios/managers/archives.py,sha256=QT16-DaV6IR8X1tCqlLBy2PSqldMeZxB0zYZSNyCCS4,8772
22
+ esios/managers/archives.py,sha256=7mK3BSYEPNygMO2pQCXYqNE90QTqfOGY4byBNn76ikQ,9386
17
23
  esios/managers/base.py,sha256=7XcdrUtUOPuqfHYlz4w562TD8o9cNdBWOgs4CHHonoo,835
18
- esios/managers/indicators.py,sha256=xnTdyvLsRi3DO-7_CckNQuxtq4ghA1ttzPnYQZUoOoA,16387
24
+ esios/managers/indicators.py,sha256=GEyHq09TCPnf3ARULS7olJYC6iiom2XrcyhDAf946po,16653
19
25
  esios/managers/offer_indicators.py,sha256=0MjEKkj77YC2fRSHVTEc7FW6E8AuwwciAXK-bOVEL5Q,4187
20
26
  esios/models/__init__.py,sha256=oppuTASpf0Dh2KbGMXInULT0F4sELjeo-9UhPiPOZiA,289
21
27
  esios/models/archive.py,sha256=P2LaT7_ff4ujwqVn_ofgQP3dbpf7jqON0R22dKwSJ_w,1062
@@ -23,10 +29,10 @@ esios/models/indicator.py,sha256=u1AJyEA3YeOqQFjV08_lzyMaofuCiMoLPjvosls9gfE,111
23
29
  esios/models/offer_indicator.py,sha256=nA80Y7Yp0utDaDOdZ-ObcWTsAdhvuXlfJjJBpdVQ7Lo,758
24
30
  esios/processing/__init__.py,sha256=1kLt_gO_wDhXM1BbY0zTyfAYo-CjYKW1ljgRRDZ7USM,278
25
31
  esios/processing/dataframes.py,sha256=OitzBvAerssGP2VXNC-sSO48XsHdIB2nKTUgByN5eYQ,2524
26
- esios/processing/i90.py,sha256=-RQh6ZIc01dlcqUjaxcRq9iUtQc7EO1GddRCUC_x0Kg,7884
32
+ esios/processing/i90.py,sha256=k4RH4lIwIm04ASYnubdQwJ3WM98iLj5l14zwxXBQEBo,10443
27
33
  esios/processing/zip.py,sha256=12LbFHJTdX_h3JG-clEgQ4Haj-kw0UjfopGLlCRXfGM,1913
28
- python_esios-2.0.0.dist-info/METADATA,sha256=1eDvpsLpuCWrb4rBX9JxS8yYPcuOVA04urG4HdCMvPU,2803
29
- python_esios-2.0.0.dist-info/WHEEL,sha256=QccIxa26bgl1E6uMy58deGWi-0aeIkkangHcxk2kWfw,87
30
- python_esios-2.0.0.dist-info/entry_points.txt,sha256=7ngseyIyvJ4buTHFL9htaZ4tTFHpG4zzJNkc8B5Jr8U,40
31
- python_esios-2.0.0.dist-info/licenses/LICENSE,sha256=LorLs1-VeBW70Wo9fLAtLJN7nNd6Poy0xzvqdWVqFlE,35128
32
- python_esios-2.0.0.dist-info/RECORD,,
34
+ python_esios-2.0.2.dist-info/METADATA,sha256=zGyHPDmU_YJLEry8ltNFkDzw07hpBqVJuvumQoxeKNM,2803
35
+ python_esios-2.0.2.dist-info/WHEEL,sha256=QccIxa26bgl1E6uMy58deGWi-0aeIkkangHcxk2kWfw,87
36
+ python_esios-2.0.2.dist-info/entry_points.txt,sha256=7ngseyIyvJ4buTHFL9htaZ4tTFHpG4zzJNkc8B5Jr8U,40
37
+ python_esios-2.0.2.dist-info/licenses/LICENSE,sha256=LorLs1-VeBW70Wo9fLAtLJN7nNd6Poy0xzvqdWVqFlE,35128
38
+ python_esios-2.0.2.dist-info/RECORD,,