apographe 0.1.0__tar.gz

apographe-0.1.0/.gitignore

@@ -0,0 +1,172 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/latest/usage/project/#working-with-version-control
+.pdm.toml
+.pdm-python
+.pdm-build/
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+
+# vscode settings
+.vscode/
+
+data/*
+.scratch/
+!data/.gitkeep
+.DS_Store
+
+artifacts/*

apographe-0.1.0/CHANGELOG.md

@@ -0,0 +1,27 @@
+# Changelog
+
+## 0.1.0 - 2026-10-25
+
+First public release. The project began as `fuel_prices_retrieval` (v0.2.0), a
+scraper for Cyprus fuel prices, and grew into a way to collect public data from
+many sources into dated plain files.
+
+- Source framework: a `Source` interface, a polite HTTP client, OAuth client
+  credentials, atomic CSV and JSON storage.
+- Use it from Python: `from apographe import run` collects a source into dated
+  files, and the main classes (`Source`, `Store`, `PoliteClient`, ...) are
+  exported.
+- `apographe list`, `apographe info SOURCE` and `apographe run`. `info` shows a
+  source's jobs and the environment variables and extras each needs; a job with
+  a missing variable stops early and names it. `run --env-file PATH` reads
+  `KEY=VALUE` lines for one run.
+- `cy.ekalathi`: Cyprus supermarket prices, every product in every store, daily;
+  cheapest-price history since September 2025.
+- `cy.fuel`: Cyprus fuel prices per station.
+  - `prices` (default): scrapes the Ministry's website with Chrome. Chrome's
+    sandbox stays on except in CI.
+  - `api`: the official CY Connect API (OAuth 2.0 client credentials). The
+    endpoints and credentials come from environment variables
+    (`CYCONNECT_TOKEN_URL`, `CYCONNECT_API_URL`, `CYCONNECT_CLIENT_ID`,
+    `CYCONNECT_CLIENT_SECRET`). Sandbox answers go to their own folder.
+  - `pdf`: import the observatory page saved as PDFs.

apographe-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025-2026 Aristides Zenonos
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

apographe-0.1.0/PKG-INFO

@@ -0,0 +1,117 @@
+Metadata-Version: 2.5
+Name: apographe
+Version: 0.1.0
+Summary: Collect public data, source by source and day by day, into plain files.
+Project-URL: Homepage, https://github.com/aristidesz/apographe
+Project-URL: Documentation, https://github.com/aristidesz/apographe/tree/main/docs
+Project-URL: Changelog, https://github.com/aristidesz/apographe/blob/main/CHANGELOG.md
+Project-URL: Issues, https://github.com/aristidesz/apographe/issues
+Author-email: Aristides Zenonos <10169524+aristidesz@users.noreply.github.com>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: data-collection,etl,open-data,time-series,web-scraping
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Requires-Dist: defusedxml>=0.7
+Requires-Dist: requests>=2.31
+Provides-Extra: cy-fuel
+Requires-Dist: beautifulsoup4>=4.12; extra == 'cy-fuel'
+Requires-Dist: pandas>=2.2; extra == 'cy-fuel'
+Requires-Dist: pdfplumber>=0.11; extra == 'cy-fuel'
+Requires-Dist: pyvirtualdisplay>=3.0; extra == 'cy-fuel'
+Requires-Dist: selenium>=4.36; extra == 'cy-fuel'
+Description-Content-Type: text/markdown
+
+# Apographe
+
+[![PyPI](https://img.shields.io/pypi/v/apographe)](https://pypi.org/project/apographe/)
+[![Python](https://img.shields.io/pypi/pyversions/apographe)](https://pypi.org/project/apographe/)
+[![CI](https://github.com/aristidesz/apographe/actions/workflows/ci.yml/badge.svg)](https://github.com/aristidesz/apographe/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](https://github.com/aristidesz/apographe/blob/main/LICENSE)
+
+**Collect public data, day by day, into plain CSV files.**
+
+Governments and public bodies publish data on websites and APIs, usually as
+today's snapshot with no history. Apographe collects it on a schedule and saves
+it as dated CSV files you can open in pandas, Excel or anything else. Run it
+every day and you build your own history.
+
+Prices are the first sources (supermarket products, petrol stations), not the
+limit: a source is one small class, so any public dataset can be added.
+
+*ἀπογραφή (apographḗ): a register, the official record of what was counted.*
+
+## Getting started
+
+```bash
+pip install apographe
+
+apographe list                                   # what you can collect
+apographe run cy.ekalathi --limit 5 --out data   # try it: 5 products, about a minute
+```
+
+Among the files it writes:
+
+```
+data/cy/ekalathi/
+├── products.csv                        what each product is
+├── stores.csv                          each store, its chain and location
+└── store_prices/2026/2026-10-12.csv.gz the price of each product in each store
+```
+
+Read it with pandas:
+
+```python
+import glob
+import pandas as pd
+
+files = glob.glob("data/cy/ekalathi/store_prices/*/*.csv.gz")
+prices = pd.concat(map(pd.read_csv, files))
+```
+
+Leave out `--limit 5` to collect every product (about 45 minutes, since
+Apographe waits a few seconds between requests).
+
+Or from Python:
+
+```python
+from apographe import run
+
+report = run("cy.ekalathi", out="data", limit=5)
+print(report.rows)  # rows written per table
+```
+
+Some jobs need credentials; `apographe info <source>` lists them.
+
+## Sources
+
+| Source | What it collects | Install |
+|---|---|---|
+| `cy.ekalathi` | Cyprus supermarket prices: every product in every store, from [e-kalathi.gov.cy](https://www.e-kalathi.gov.cy) | `pip install apographe` |
+| `cy.fuel` | Cyprus fuel prices at every petrol station. The default job scrapes the [Ministry's observatory](https://eforms.eservices.cyprus.gov.cy/MCIT/MCIT/PetroleumPrices) (needs Chrome, [caveats](https://github.com/aristidesz/apographe/blob/main/docs/sources/cy-fuel.md#caveats-of-the-default-prices-job)); the `api` job uses the official API (needs approval and credentials) | `pip install "apographe[cy-fuel]"` |
+
+## Learn more
+
+- [Commands and options](https://github.com/aristidesz/apographe/blob/main/docs/usage.md)
+- [Getting started notebook](https://github.com/aristidesz/apographe/blob/main/notebooks/00_getting_started.ipynb)
+- [Using it from Python](https://github.com/aristidesz/apographe/blob/main/docs/library.md)
+- [Collecting every day](https://github.com/aristidesz/apographe/blob/main/docs/scheduling.md)
+- [What each source collects](https://github.com/aristidesz/apographe/tree/main/docs/sources)
+- [Adding a source for your country](https://github.com/aristidesz/apographe/blob/main/docs/adding-a-source.md)
+- [Contributing](https://github.com/aristidesz/apographe/blob/main/CONTRIBUTING.md)
+
+Apographe makes one request at a time, waits between requests and identifies
+itself to the sites it reads. It is an independent project, not affiliated with
+any of them; check each site's terms before you collect from it.
+
+## License
+
+MIT. The data you collect belongs to its publishers.

apographe-0.1.0/README.md

@@ -0,0 +1,85 @@
+# Apographe
+
+[![PyPI](https://img.shields.io/pypi/v/apographe)](https://pypi.org/project/apographe/)
+[![Python](https://img.shields.io/pypi/pyversions/apographe)](https://pypi.org/project/apographe/)
+[![CI](https://github.com/aristidesz/apographe/actions/workflows/ci.yml/badge.svg)](https://github.com/aristidesz/apographe/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](https://github.com/aristidesz/apographe/blob/main/LICENSE)
+
+**Collect public data, day by day, into plain CSV files.**
+
+Governments and public bodies publish data on websites and APIs, usually as
+today's snapshot with no history. Apographe collects it on a schedule and saves
+it as dated CSV files you can open in pandas, Excel or anything else. Run it
+every day and you build your own history.
+
+Prices are the first sources (supermarket products, petrol stations), not the
+limit: a source is one small class, so any public dataset can be added.
+
+*ἀπογραφή (apographḗ): a register, the official record of what was counted.*
+
+## Getting started
+
+```bash
+pip install apographe
+
+apographe list                                   # what you can collect
+apographe run cy.ekalathi --limit 5 --out data   # try it: 5 products, about a minute
+```
+
+Among the files it writes:
+
+```
+data/cy/ekalathi/
+├── products.csv                        what each product is
+├── stores.csv                          each store, its chain and location
+└── store_prices/2026/2026-10-12.csv.gz the price of each product in each store
+```
+
+Read it with pandas:
+
+```python
+import glob
+import pandas as pd
+
+files = glob.glob("data/cy/ekalathi/store_prices/*/*.csv.gz")
+prices = pd.concat(map(pd.read_csv, files))
+```
+
+Leave out `--limit 5` to collect every product (about 45 minutes, since
+Apographe waits a few seconds between requests).
+
+Or from Python:
+
+```python
+from apographe import run
+
+report = run("cy.ekalathi", out="data", limit=5)
+print(report.rows)  # rows written per table
+```
+
+Some jobs need credentials; `apographe info <source>` lists them.
+
+## Sources
+
+| Source | What it collects | Install |
+|---|---|---|
+| `cy.ekalathi` | Cyprus supermarket prices: every product in every store, from [e-kalathi.gov.cy](https://www.e-kalathi.gov.cy) | `pip install apographe` |
+| `cy.fuel` | Cyprus fuel prices at every petrol station. The default job scrapes the [Ministry's observatory](https://eforms.eservices.cyprus.gov.cy/MCIT/MCIT/PetroleumPrices) (needs Chrome, [caveats](https://github.com/aristidesz/apographe/blob/main/docs/sources/cy-fuel.md#caveats-of-the-default-prices-job)); the `api` job uses the official API (needs approval and credentials) | `pip install "apographe[cy-fuel]"` |
+
+## Learn more
+
+- [Commands and options](https://github.com/aristidesz/apographe/blob/main/docs/usage.md)
+- [Getting started notebook](https://github.com/aristidesz/apographe/blob/main/notebooks/00_getting_started.ipynb)
+- [Using it from Python](https://github.com/aristidesz/apographe/blob/main/docs/library.md)
+- [Collecting every day](https://github.com/aristidesz/apographe/blob/main/docs/scheduling.md)
+- [What each source collects](https://github.com/aristidesz/apographe/tree/main/docs/sources)
+- [Adding a source for your country](https://github.com/aristidesz/apographe/blob/main/docs/adding-a-source.md)
+- [Contributing](https://github.com/aristidesz/apographe/blob/main/CONTRIBUTING.md)
+
+Apographe makes one request at a time, waits between requests and identifies
+itself to the sites it reads. It is an independent project, not affiliated with
+any of them; check each site's terms before you collect from it.
+
+## License
+
+MIT. The data you collect belongs to its publishers.

apographe-0.1.0/docs/adding-a-source.md

@@ -0,0 +1,59 @@
+# Adding a source
+
+A source is one class. It names its jobs, and each job writes tables through the
+`Store` it is given. The framework does the rest: the command line, the day the
+run belongs to, the output folder, the report.
+
+```python
+from apographe.core.http import PoliteClient
+from apographe.core.source import Options, Report, Source
+from apographe.core.storage import Store
+
+
+class ExampleFuelSource(Source):
+    id = "xx.fuel"  # country code, then a short name
+    title = "Example fuel prices"
+    country = "XX"  # ISO 3166-1 alpha-2
+    homepage = "https://fuel.example.gov"
+    timezone = "Europe/Athens"  # "today" is taken here
+    jobs = {"prices": "every station's prices, now"}
+    default_jobs = ("prices",)
+
+    def run(self, job: str, out: Store, options: Options) -> Report:
+        http = PoliteClient(
+            "https://fuel.example.gov/api", min_interval=options.delay or 3
+        )
+        stations = http.get_json("/stations")
+        day = options.day.isoformat()
+        rows = [
+            {"date": day, "station_id": s["id"], "fuel": s["fuel"], "price": s["price"]}
+            for s in stations
+        ]
+        report = Report()
+        path, count = out.write_csv(
+            f"prices/{day[:4]}/{day}.csv.gz",
+            rows,
+            ["date", "station_id", "fuel", "price"],
+        )
+        report.add("prices", count, path)
+        return report
+```
+
+## Shipping it
+
+Put the module under `src/apographe/sources/<country>/`, add it to `BUILT_IN`
+in `src/apographe/core/registry.py`, add tests with a few trimmed real responses
+under `tests/fixtures/`, and open a pull request.
+
+## Conventions
+
+- **One file per day** for anything collected daily: `<table>/YYYY/YYYY-MM-DD.csv.gz`.
+  Lookups that change slowly (stores, chains) are one CSV, rewritten.
+- **Ids in every row** so tables join: `product_id`, `store_id`, `chain_id`.
+- **Keep the raw response** when it is small, under `raw/`, so a parsing fix can
+  rebuild the tables.
+- **Be polite:** one request at a time, a few seconds apart; never parallelise
+  requests to a public site.
+- **Raise `LoginWall`** (the `PoliteClient` does this for you) when a site starts
+  asking for a login, so scheduled runs end cleanly instead of failing.
+- **Print counts, never data:** logs of scheduled runs are often public.

apographe-0.1.0/docs/library.md

@@ -0,0 +1,59 @@
+# Using Apographe from Python
+
+Everything the command line does is one function, `run`:
+
+```python
+from apographe import run
+
+report = run("cy.ekalathi", out="data", limit=5)
+print(report.rows)  # {"products": 5, ...}: rows written per table
+print(report.files)  # the files written
+```
+
+`run` takes the same options as `apographe run`: `jobs`, `day` (a date or
+`"2026-10-01"`), `limit`, `only`, `delay`, `inputs` and `force`. `help(run)`
+lists them all. It raises `ValueError` for a job the source does not have,
+`LoginWall` when the site now asks for a login, and `SourceError` for anything
+else that goes wrong. It prints nothing.
+
+## Your own source
+
+Pass a `Source` (an instance or the class) instead of an id:
+
+```python
+from apographe import Report, Source, run
+
+
+class Stations(Source):
+    id = "xx.fuel"
+    title = "Example fuel prices"
+    country = "XX"
+    homepage = "https://fuel.example.gov"
+    jobs = {"prices": "every station's prices, now"}
+    default_jobs = ("prices",)
+
+    def run(self, job, out, options):
+        rows = [{"station": "A", "price": 1.5}]  # fetch your data here
+        path, count = out.write_csv(
+            f"prices/{options.day}.csv", rows, ["station", "price"]
+        )
+        report = Report()
+        report.add("prices", count, path)
+        return report
+
+
+run(Stations(), out="data")
+```
+
+[Adding a source](adding-a-source.md) shows a full one and the conventions.
+
+## What you can import
+
+| Name | What it is |
+|---|---|
+| `run` | Run a source |
+| `Source`, `Options`, `Report` | Write a source: the class, its run settings, what it hands back |
+| `Store` | The folder a source writes into: `write_csv`, `write_json`, `read_csv` |
+| `PoliteClient` | A JSON client that waits between requests and retries |
+| `ClientCredentials` | OAuth sign-in for sources behind an API gateway |
+| `SourceError`, `LoginWall` | The errors above |

apographe-0.1.0/docs/scheduling.md

@@ -0,0 +1,49 @@
+# Collecting every day
+
+Public data only becomes a history if you collect them every day. Apographe is a
+command, so any scheduler works: cron on a server, a container on a timer, or
+GitHub Actions.
+
+## GitHub Actions
+
+This workflow collects e-kalathi every morning and commits the new files to the
+repository it runs in. Keep that repository private if the data should be.
+
+```yaml
+name: Collect data
+
+on:
+  schedule:
+    - cron: "30 4 * * *" # 07:30 in Cyprus in summer
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+jobs:
+  collect:
+    runs-on: ubuntu-latest
+    timeout-minutes: 120
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: pip install apographe
+      - run: apographe run cy.ekalathi --out .
+      - run: |
+          git config user.name "github-actions[bot]"
+          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
+          git add cy
+          git commit -m "e-kalathi $(date -u +%F)" || exit 0
+          git push
+```
+
+Pin the version you use (`pip install apographe==0.1.0`) so an upgrade never
+surprises a scheduled run.
+
+## cron
+
+```cron
+30 7 * * *  cd /path/to/data && apographe run cy.ekalathi --out . >> collect.log 2>&1
+```

apographe-0.1.0/docs/sources/cy-ekalathi.md

@@ -0,0 +1,72 @@
+# cy.ekalathi: Cyprus supermarket prices
+
+[e-kalathi.gov.cy](https://www.e-kalathi.gov.cy) is the Cypriot government's
+supermarket price comparison site. It lists about 476 everyday products and their
+price in each of more than 140 stores, listed under 48 chains.
+
+## Jobs
+
+| Job | Requests | Writes |
+|---|---|---|
+| `lookups` | 3 | `chains.csv`, `regions.csv`, `categories.csv` |
+| `catalogue` | ~5 | `catalogue/YYYY/DATE.csv.gz` (cheapest price per product), `products.csv`, the raw response |
+| `store-prices` | 1 per product (~476) | `store_prices/YYYY/DATE.csv.gz`, `stores.csv` |
+| `history` | 1 per product | `history/<product_id>.csv`: daily cheapest price since 3 Sep 2025 |
+
+`apographe run cy.ekalathi` runs the first three. A full day takes about 45
+minutes at the default 3 seconds between requests. If it stops part way, the
+next run on the same day carries on where it left off.
+
+## Tables
+
+| Table | Key | Columns |
+|---|---|---|
+| products | product_id | code, name, category, category_en, category_id |
+| catalogue | date, product_id | price (cheapest in Cyprus), previous_price, store_count |
+| store_prices | date, product_id, store_id | price, price_before_offer, on_offer |
+| stores | store_id | name, chain_id, chain, phone, address, postcode, region_id, lat, lon, last_seen |
+| chains | chain_id | name |
+| regions | region_id | name |
+| categories | category_id | code, name, name_en, parent_id |
+| history | date, product_id | price |
+
+Example, Kykkos water 500 ml on 9 Oct 2026:
+
+```
+catalogue:     2026-10-09, 2074, 0.25, 0.23, 115
+store_prices:  2026-10-09, 2074, 539, 0.25, 0.25, False
+stores:        539, ALPHA SIGMA SUPERMARKET Περιοχή Φασούρι, 452, ALPHA SIGMA, …, 4640, 3, 34.644167, 32.944778
+chains:        452, ALPHA SIGMA
+regions:       3, ΛΕΜΕΣΟΣ
+```
+
+## Things to know
+
+- **Ids are per table.** Chain 539 (ΣΥΠΑΛ - ΛΙΜΝΙΑ) and store 539 (ALPHA SIGMA
+  Φασούρι) are unrelated.
+- **Some "chains" are single shops,** such as each SMART DISCOUNT SHOP branch.
+- **`store_count` counts stores, not chains,** although the site calls it
+  `numberOfChains`.
+- **Region comes from the postcode** (1xxx-2xxx Nicosia, 3xxx-4xxx Limassol,
+  5xxx Famagusta, 6xxx-7xxx Larnaca, 8xxx Paphos), or from the town named in the
+  address when there is no postcode; store rows carry no region of their own.
+- **Past prices exist only as the cheapest per product.** Per-store prices can
+  be collected from the day you start.
+
+## The site's API
+
+The site's pages call a JSON backend at
+`https://www.e-kalathi.gov.cy/ekalathi-website-server/api`. It is not a
+published API and may change without notice. None of these calls needs a login.
+
+| Path | Arguments | Returns | Time |
+|---|---|---|---|
+| `/fetch-product-list` | `page`, `size`, `sort=DEFAULT,ASC`; optional `categoryIds`, `productName` | Paged products: `productMasterId`, `code`, `name`, `startPrice` (cheapest today), `previousPrice`, `productCategoryName(English)`, `numberOfChains` | ~1 s per page |
+| `/retail/fetch-retail-branch-list` | `productId`, `page`, `size`, `sort` | Paged store rows: `id`, `name`, `companyName`, `landPhone`, `postalAddress`, `branchLatitude/Longitude` (degrees-minutes-seconds), `retailerProductPrice`, `retailerInitialProductPrice`, `isInOfferOrDiscount`, and the chain logo as base64 (`companyPhotoFile`, ~100 KB per row) | ~4 s, ~11 MB |
+| `/fetch-product-price-diagram` | `id`, `from`, `to` (YYYY-MM-DD); optional `cId` (chain), `rId` (region) | `[{date, price}]`, the cheapest in Cyprus each day, or one chain's/region's | ~30 s; often times out, so retried |
+| `/fetch-product?id=` | product id | Product details, no prices | ~1 s |
+| `/fetch-companies`, `/fetch-regions`, `/fetch-product-categories` | none | Lookup lists | ~1 s |
+
+Paged calls answer `{"content": [...], "last": bool, "totalPages": n, ...}`.
+`/fetch-retail-branches`, `/fetch-basket-types` and `/fetch-company-offers` need
+a login (HTTP 401).