chellow 1742914638.0.0__tar.gz → 1767090201.0.0__tar.gz

{chellow-1742914638.0.0 → chellow-1767090201.0.0}/.github/workflows/chellow.yml

@@ -10,7 +10,7 @@ jobs:
   # Label of the container job
   test:
     runs-on: ubuntu-latest
-    container: python:3.9
+    container: python:3.11
 
     services:
       postgres:
@@ -38,7 +38,7 @@ jobs:
 
   build:
     runs-on: ubuntu-latest
-    container: python:3.9
+    container: python:3.11
     needs: test
     steps:
     - uses: actions/checkout@v4

{chellow-1742914638.0.0 → chellow-1767090201.0.0}/PKG-INFO

@@ -1,31 +1,30 @@
 Metadata-Version: 2.4
 Name: chellow
-Version: 1742914638.0.0
+Version: 1767090201.0.0
 Summary: Web Application for checking UK energy bills.
 Project-URL: Homepage, https://github.com/WessexWater/chellow
 Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
-Requires-Python: >=3.9
-Requires-Dist: flask-restx==1.3.0
-Requires-Dist: flask==3.1.0
-Requires-Dist: jsonschema==4.17.3
+Requires-Python: >=3.11
+Requires-Dist: flask-restx==1.3.2
+Requires-Dist: flask==3.1.2
+Requires-Dist: markdown-it-py==4.0.0
 Requires-Dist: odio==0.0.23
-Requires-Dist: openpyxl==3.1.2
-Requires-Dist: paramiko==3.4.1
-Requires-Dist: pep3143daemon==0.0.6
-Requires-Dist: pg8000==1.31.1
+Requires-Dist: openpyxl==3.1.5
+Requires-Dist: paramiko==4.0.0
+Requires-Dist: pg8000==1.31.5
 Requires-Dist: pip>=9.0.1
-Requires-Dist: psutil==5.9.5
-Requires-Dist: pympler==1.0.1
-Requires-Dist: pypdf==4.3.1
+Requires-Dist: psutil==7.1.1
+Requires-Dist: pympler==1.1
+Requires-Dist: pypdf==6.4.0
 Requires-Dist: python-dateutil==2.8.2
-Requires-Dist: pytz==2022.6
-Requires-Dist: requests==2.32.2
-Requires-Dist: sqlalchemy==2.0.30
-Requires-Dist: waitress==3.0.1
-Requires-Dist: xlrd==2.0.1
-Requires-Dist: zish==0.1.11
+Requires-Dist: pytz==2025.2
+Requires-Dist: requests==2.32.5
+Requires-Dist: sqlalchemy==2.0.44
+Requires-Dist: waitress==3.0.2
+Requires-Dist: xlrd==2.0.2
+Requires-Dist: zish==0.1.12
 Description-Content-Type: text/markdown
 
 # Chellow
@@ -47,7 +46,7 @@ Chellow is a web application for checking UK electricity and gas bills. It's des
 for organizations with high electricity consumption. The software is hosted at
 https://github.com/WessexWater/chellow.
 
-[![Build Status](https://github.com/WessexWater/chellow/workflows/chellow/badge.svg)](https://github.com/WessexWater/chellow/workflows/chellow)
+[![Build Status](https://github.com/WessexWater/chellow/actions/workflows/chellow.yml/badge.svg)](https://github.com/WessexWater/chellow/actions/workflows/chellow.yml)
 
 
 ## Installation
@@ -55,8 +54,7 @@ https://github.com/WessexWater/chellow.
 Chellow is a Python web application that uses the PostgreSQL database. To install
 Chellow, follow these steps:
 
-* Install [PostgreSQL](http://www.postgresql.org/) 12
-  * Set the PostgreSQL time zone to 'UTC'.
+* Install [PostgreSQL](http://www.postgresql.org/) 15
 * Create a PostgreSQL database: `createdb --encoding=UTF8 chellow`
 * Set up the following environment variables to configure Chellow:
 
@@ -76,7 +74,7 @@ in Windows an environment variable can be set by doing:
 
 `set PGUSER=postgres`
 
-* Install Python 3.9 (tested on the [CPython 3.9.8](http://www.python.org/) interpreter)
+* Install Python 3.11
 * Create a virtual environment: `python3 -m venv venv`
 * Activate the virtual environment: `source venv/bin/activate`
 * Install Chellow: `pip install chellow`
@@ -86,6 +84,19 @@ in Windows an environment variable can be set by doing:
   `admin@example.com` and the password `admin`, and then the home page should appear.
   Change the admin password from the `users` page.
 
+### Azure
+
+SSH into the machine and you'll be in ``/home/site/wwwroot``.
+
+* Create a virtual environment: ``python -m venv antenv``
+* Activate the virtual environment with: ``source antenv/bin/activate``
+*
+In ``/home/site/wwwroot`` create a file called ``app.py`` containing the following:
+
+    import chellow
+    
+    app = chellow.create_app()
+
 
 ### Manually Upgrading Chellow
 

{chellow-1742914638.0.0 → chellow-1767090201.0.0}/README.md

@@ -17,7 +17,7 @@ Chellow is a web application for checking UK electricity and gas bills. It's des
 for organizations with high electricity consumption. The software is hosted at
 https://github.com/WessexWater/chellow.
 
-[![Build Status](https://github.com/WessexWater/chellow/workflows/chellow/badge.svg)](https://github.com/WessexWater/chellow/workflows/chellow)
+[![Build Status](https://github.com/WessexWater/chellow/actions/workflows/chellow.yml/badge.svg)](https://github.com/WessexWater/chellow/actions/workflows/chellow.yml)
 
 
 ## Installation
@@ -25,8 +25,7 @@ https://github.com/WessexWater/chellow.
 Chellow is a Python web application that uses the PostgreSQL database. To install
 Chellow, follow these steps:
 
-* Install [PostgreSQL](http://www.postgresql.org/) 12
-  * Set the PostgreSQL time zone to 'UTC'.
+* Install [PostgreSQL](http://www.postgresql.org/) 15
 * Create a PostgreSQL database: `createdb --encoding=UTF8 chellow`
 * Set up the following environment variables to configure Chellow:
 
@@ -46,7 +45,7 @@ in Windows an environment variable can be set by doing:
 
 `set PGUSER=postgres`
 
-* Install Python 3.9 (tested on the [CPython 3.9.8](http://www.python.org/) interpreter)
+* Install Python 3.11
 * Create a virtual environment: `python3 -m venv venv`
 * Activate the virtual environment: `source venv/bin/activate`
 * Install Chellow: `pip install chellow`
@@ -56,6 +55,19 @@ in Windows an environment variable can be set by doing:
   `admin@example.com` and the password `admin`, and then the home page should appear.
   Change the admin password from the `users` page.
 
+### Azure
+
+SSH into the machine and you'll be in ``/home/site/wwwroot``.
+
+* Create a virtual environment: ``python -m venv antenv``
+* Activate the virtual environment with: ``source antenv/bin/activate``
+*
+In ``/home/site/wwwroot`` create a file called ``app.py`` containing the following:
+
+    import chellow
+    
+    app = chellow.create_app()
+
 
 ### Manually Upgrading Chellow
 

chellow-1767090201.0.0/VERSION

@@ -0,0 +1 @@
+__version__ = "1767090201.0.0"

chellow-1767090201.0.0/bin/chellow_upgrader.sh

@@ -0,0 +1,36 @@
+#!/usr/bin/env bash
+
+cd /var/www/chellow
+source venv/bin/activate
+
+if [[ $* == *--test* ]] ; then
+	if pip list --outdated --extra-index-url https://test.pypi.org/simple/ | grep chellow ; then
+		echo "Found a new Chellow version."
+
+		echo "Stopping Chellow"
+		/usr/bin/systemctl stop chellow
+
+		echo "Doing a pip upgrade of Chellow."
+		pip install chellow --upgrade --extra-index-url https://test.pypi.org/simple/
+
+		echo "Starting Chellow"
+		/usr/bin/systemctl start chellow
+	else
+		echo "No new version found."
+	fi
+else
+	if pip list --outdated | grep chellow ; then
+		echo "Found a new Chellow version."
+
+		echo "Stopping Chellow"
+		/usr/bin/systemctl stop chellow
+
+		echo "Doing a pip upgrade of Chellow."
+		pip install chellow --upgrade
+
+		echo "Starting Chellow"
+		/usr/bin/systemctl start chellow
+	else
+		echo "No new version found."
+	fi
+fi

{chellow-1742914638.0.0 → chellow-1767090201.0.0}/chellow/__init__.py

@@ -1,10 +1,15 @@
+import json
 import os
+from base64 import b64decode
 from datetime import datetime as Datetime
 from importlib.metadata import version
 from pathlib import Path
 
 from flask import Flask, Response, g, make_response, render_template, request
 
+from markdown_it import MarkdownIt
+
+from markupsafe import Markup
 
 from sqlalchemy import select
 
@@ -19,9 +24,11 @@ import chellow.e.bmarketidx
 import chellow.e.bsuos
 import chellow.e.elexon
 import chellow.e.hh_importer
+import chellow.e.isd
 import chellow.e.rcrc
 import chellow.e.system_price
 import chellow.e.views
+import chellow.fake_batch_updater
 import chellow.gas.cv
 import chellow.gas.views
 import chellow.national_grid
@@ -52,6 +59,7 @@ TEMPLATE_FORMATS = {
 
 def get_importer_modules():
     return (
+        chellow.e.isd,
         chellow.e.elexon,
         chellow.e.bsuos,
         chellow.e.lcc,
@@ -62,6 +70,7 @@ def get_importer_modules():
         chellow.national_grid,
         chellow.rate_server,
         chellow.rrun,
+        chellow.fake_batch_updater,
     )
 
 
@@ -125,17 +134,45 @@ def create_app(testing=False, instance_path=None):
             g.config = {}
         ad_props = g.config.get("ad_authentication", {})
         ad_auth_on = ad_props.get("on", False)
+        easy_auth_props = g.config.get("easy_auth", {})
+        easy_auth_on = easy_auth_props.get("on", False)
+        path = request.path
+        if path in ("/health", "/robots933456.txt"):
+            return
         if ad_auth_on:
             username = request.headers["X-Isrw-Proxy-Logon-User"].upper()
-            user = g.sess.query(User).filter(User.email_address == username).first()
+            user = g.sess.scalars(
+                select(User).where(User.email_address == username)
+            ).first()
             if user is None:
                 try:
                     username = ad_props["default_user"]
-                    user = (
-                        g.sess.query(User)
-                        .filter(User.email_address == username)
-                        .first()
-                    )
+                    user = g.sess.scalars(
+                        select(User).where(User.email_address == username)
+                    ).first()
+                except KeyError:
+                    user = None
+            if user is not None:
+                g.user = user
+        elif easy_auth_on:
+            jwt_enc = request.headers["X-MS-CLIENT-PRINCIPAL"]
+            jwt = json.loads(b64decode(jwt_enc))
+            for claim in jwt["claims"]:
+                if claim["typ"] == (
+                    "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/"
+                    "emailaddress"
+                ):
+                    username = claim["val"]
+                    user = g.sess.scalars(
+                        select(User).where(User.email_address == username)
+                    ).first()
+                    break
+            if user is None:
+                try:
+                    username = easy_auth_props["default_user"]
+                    user = g.sess.scalars(
+                        select(User).where(User.email_address == username)
+                    ).first()
                 except KeyError:
                     user = None
             if user is not None:
@@ -154,25 +191,20 @@ def create_app(testing=False, instance_path=None):
                         key = None
 
                     email = ips[key]
-                    g.user = (
-                        g.sess.query(User).filter(User.email_address == email).first()
-                    )
+                    g.user = g.sess.scalars(
+                        select(User).where(User.email_address == email)
+                    ).first()
                 except KeyError:
                     pass
             else:
-                user = (
-                    g.sess.query(User)
-                    .filter(User.email_address == auth.username)
-                    .first()
-                )
+                user = g.sess.scalars(
+                    select(User).where(User.email_address == auth.username)
+                ).first()
                 if user is not None and user.password_matches(auth.password):
                     g.user = user
 
         # Got our user
-        path = request.path
         method = request.method
-        if path in ("/health",):
-            return
 
         if g.user is not None:
             if "X-Isrw-Proxy-Logon-User" in request.headers:
@@ -225,7 +257,9 @@ def create_app(testing=False, instance_path=None):
             g.sess.commit()
             return
 
-        if g.user is None or (not ad_auth_on and auth is None):
+        if (not easy_auth_on) and (
+            (g.user is None) or (not ad_auth_on and auth is None)
+        ):
             return Response(
                 "Could not verify your access level for that URL.\n"
                 "You have to login with proper credentials",
@@ -299,6 +333,11 @@ def create_app(testing=False, instance_path=None):
     def dumps_filter(d):
         return dumps(d)
 
+    @app.template_filter("markdown")
+    def markdown_filter(txt):
+        md = MarkdownIt()
+        return Markup(md.render(txt))
+
     return app
 
 

{chellow-1742914638.0.0 → chellow-1767090201.0.0}/chellow/bank_holidays.py

@@ -12,7 +12,9 @@ from zish import loads
 from chellow.models import Contract, RateScript, Session
 from chellow.utils import (
     HH,
+    ct_datetime,
     hh_format,
+    to_utc,
     utc_datetime,
     utc_datetime_now,
     utc_datetime_parse,
@@ -24,11 +26,23 @@ bh_importer = None
 
 def _run(log_f, sess):
     log_f("Starting to check bank holidays")
-    contract = Contract.get_non_core_by_name(sess, "bank_holidays")
+    contract_name = "bank_holidays"
+    contract = Contract.find_non_core_by_name(sess, contract_name)
+    if contract is None:
+        contract = Contract.insert_non_core(
+            sess,
+            contract_name,
+            "",
+            {"enabled": True},
+            to_utc(ct_datetime(1996, 4, 1)),
+            None,
+            {},
+        )
+        sess.commit()
     contract_props = contract.make_properties()
 
-    if contract_props.get("enabled", False):
-        url_str = contract_props["url"]
+    if contract_props.get("enabled", True):
+        url_str = "https://www.gov.uk/bank-holidays/england-and-wales.ics"
 
         log_f(f"Downloading from {url_str}.")
         res = requests.get(url_str)

{chellow-1742914638.0.0 → chellow-1767090201.0.0}/chellow/dloads.py

@@ -91,7 +91,7 @@ class DloadFile:
 def open_file(base, user, mode="r", newline=None, is_zip=False):
     global download_id
 
-    base = base.replace("/", "").replace(" ", "")
+    base = "".join(x if (x.isalnum() or x in "_.") else "_" for x in base)
     try:
         lock.acquire()
         if len(list(download_path.iterdir())) == 0:
@@ -108,7 +108,7 @@ def open_file(base, user, mode="r", newline=None, is_zip=False):
             un = user.proxy_username
         else:
             un = user.email_address
-        uname = un.replace("@", "").replace(".", "").replace("\\", "")
+        uname = "".join(x if (x.isalnum() or x in "_") else "_" for x in un)
 
     names = tuple("_".join((serial, v, uname, base)) for v in ("RUNNING", "FINISHED"))
     running_name, finished_name = tuple(download_path / name for name in names)

chellow-1767090201.0.0/chellow/e/aahedc.py

@@ -0,0 +1,88 @@
+from decimal import Decimal
+
+from sqlalchemy import select
+
+from chellow.models import Contract, RateScript
+from chellow.national_grid import api_get
+from chellow.utils import ct_datetime, to_utc
+
+
+def hh(supply_source):
+    try:
+        aahedc_cache = supply_source.caches["aahedc"]
+    except KeyError:
+        aahedc_cache = supply_source.caches["aahedc"] = {}
+
+    for hh in supply_source.hh_data:
+        hh["aahedc-kwh"] = hh["gsp-kwh"]
+        try:
+            rate = aahedc_cache[hh["start-date"]]
+        except KeyError:
+            hh_start = hh["start-date"]
+            rates = supply_source.non_core_rate("aahedc", hh_start)
+            rate = aahedc_cache[hh_start] = float(rates["aahedc_gbp_per_gsp_kwh"])
+
+        hh["aahedc-rate"] = rate
+        hh["aahedc-gbp"] = hh["gsp-kwh"] * rate
+
+
+def national_grid_import(sess, log, set_progress, s):
+    log("Starting to check for new AAHEDC rates")
+
+    contract_name = "aahedc"
+    contract = Contract.find_non_core_by_name(sess, contract_name)
+    if contract is None:
+        contract = Contract.insert_non_core(
+            sess, contract_name, "", {}, to_utc(ct_datetime(1996, 4, 1)), None, {}
+        )
+
+    params = {"sql": """SELECT * FROM "ffd29cc8-3c55-4e83-aa0e-73212d4fedba" """}
+    res_j = api_get(s, "datastore_search_sql", params=params)
+    for record in res_j["result"]["records"]:
+        # {
+        #   "_full_text": "'-07':4 '-15':5 '0.012247':7 '0.028 '2026':2 'final':1",
+        #   "Published Date": "2025-07-15",
+        #   "AAHEDC tariff excluding the Shetland Assistance Amount in p/kwh":
+        #   "0.028737",
+        #   "Shetland Tariff in p/kwh": "0.012247",
+        #   "Year FY": 2026,
+        #   "Publication Type": "Final",
+        #   "_id": 7,
+        #   "Total Scheme Tariff in p/kwh": "0.040984"
+        # }
+
+        fy_year = int(record["Year FY"]) - 1
+        fy_start = to_utc(ct_datetime(fy_year, 4, 1))
+        if fy_start < contract.start_rate_script.start_date:
+            continue
+
+        rs = sess.execute(
+            select(RateScript).where(
+                RateScript.contract == contract,
+                RateScript.start_date == fy_start,
+            )
+        ).scalar_one_or_none()
+        if rs is None:
+            rs = contract.insert_rate_script(sess, fy_start, {})
+
+        rs_script = rs.make_script()
+        try:
+            rs_record = rs_script["record"]
+        except KeyError:
+            rs_record = rs_script["record"] = {}
+
+        record_published_date = record["Published Date"]
+
+        if (
+            "Published Date" not in rs_record
+            or rs_record["Published Date"] < record_published_date
+        ):
+            rs_script["record"] = record
+            rs_script["aahedc_gbp_per_gsp_kwh"] = Decimal(
+                record["Total Scheme Tariff in p/kwh"]
+            ) / Decimal("100")
+            rs.update(rs_script)
+            sess.commit()
+
+    log("Finished Checking for AAHEDC Tariffs")
+    sess.commit()