bcdata 0.17.2__tar.gz → 0.18.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {bcdata-0.17.2 → bcdata-0.18.0}/PKG-INFO +6 -4
- {bcdata-0.17.2 → bcdata-0.18.0}/README.md +3 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/docker-compose.yml +2 -2
- {bcdata-0.17.2 → bcdata-0.18.0}/pyproject.toml +2 -3
- {bcdata-0.17.2 → bcdata-0.18.0}/src/bcdata/_version.py +3 -3
- {bcdata-0.17.2 → bcdata-0.18.0}/src/bcdata/database.py +8 -2
- {bcdata-0.17.2 → bcdata-0.18.0}/src/bcdata/wfs.py +3 -3
- {bcdata-0.17.2 → bcdata-0.18.0}/src/bcdata.egg-info/PKG-INFO +6 -4
- {bcdata-0.17.2 → bcdata-0.18.0}/src/bcdata.egg-info/requires.txt +1 -1
- bcdata-0.18.0/src/bcdata.egg-info/scm_version.json +8 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/tests/test_bc2pg.py +1 -1
- {bcdata-0.17.2 → bcdata-0.18.0}/tests/test_cli.py +1 -1
- bcdata-0.18.0/uv.lock +1391 -0
- bcdata-0.17.2/src/bcdata.egg-info/scm_version.json +0 -8
- bcdata-0.17.2/uv.lock +0 -1606
- {bcdata-0.17.2 → bcdata-0.18.0}/.github/workflows/release.yml +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/.github/workflows/tests.yml +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/.gitignore +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/.pre-commit-config.yaml +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/CODE_OF_CONDUCT.md +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/LICENSE.txt +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/Manifest.in +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/data/README.md +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/data/primary_keys.json +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/data/validate.py +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/ogr2ogr.md +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/release_checklist.md +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/scripts/document_cli.sh +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/setup.cfg +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/src/bcdata/__init__.py +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/src/bcdata/bc2pg.py +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/src/bcdata/bcdc.py +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/src/bcdata/cli.py +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/src/bcdata.egg-info/SOURCES.txt +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/src/bcdata.egg-info/dependency_links.txt +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/src/bcdata.egg-info/entry_points.txt +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/src/bcdata.egg-info/scm_file_list.json +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/src/bcdata.egg-info/top_level.txt +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/tests/conftest.py +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/tests/test_bcdc.py +0 -0
- {bcdata-0.17.2 → bcdata-0.18.0}/tests/test_wfs.py +0 -0
|
@@ -1,26 +1,25 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: bcdata
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.18.0
|
|
4
4
|
Summary: Download open data, monitor and report on changes
|
|
5
5
|
Author-email: Simon Norris <snorris@hillcrestgeo.ca>
|
|
6
6
|
Project-URL: Homepage, https://github.com/smnorris/bcdata
|
|
7
7
|
Project-URL: Issues, https://github.com/smnorris/bcdata
|
|
8
8
|
Classifier: Development Status :: 4 - Beta
|
|
9
|
-
Classifier: Programming Language :: Python :: 3.10
|
|
10
9
|
Classifier: Programming Language :: Python :: 3.11
|
|
11
10
|
Classifier: Programming Language :: Python :: 3.12
|
|
12
11
|
Classifier: Programming Language :: Python :: 3.13
|
|
13
12
|
Classifier: License :: OSI Approved :: Apache Software License
|
|
14
13
|
Classifier: Topic :: Scientific/Engineering :: GIS
|
|
15
14
|
Classifier: Operating System :: OS Independent
|
|
16
|
-
Requires-Python: >=3.
|
|
15
|
+
Requires-Python: >=3.11
|
|
17
16
|
Description-Content-Type: text/markdown
|
|
18
17
|
License-File: LICENSE.txt
|
|
19
18
|
Requires-Dist: click
|
|
20
19
|
Requires-Dist: geoalchemy2
|
|
21
20
|
Requires-Dist: geopandas
|
|
22
21
|
Requires-Dist: owslib
|
|
23
|
-
Requires-Dist:
|
|
22
|
+
Requires-Dist: psycopg[binary]
|
|
24
23
|
Requires-Dist: requests
|
|
25
24
|
Requires-Dist: sqlalchemy
|
|
26
25
|
Requires-Dist: stamina
|
|
@@ -44,6 +43,9 @@ but most sources are available only via WFS - and the syntax to download WFS dat
|
|
|
44
43
|
|
|
45
44
|
This tool attempts to simplify downloads of BC geographic data and smoothly integrate with PostGIS and Python GIS tools like `geopandas` and `fiona`. The tool only accesses geographic data [avialable via WFS](https://bcgov.github.io/data-publication/pages/tips_tricks_webservices.html) - for other BC open data, download the files directly with `requests` / `curl` / `ogr2ogr` / etc (or the [bcdata R package](https://github.com/bcgov/bcdata)).
|
|
46
45
|
|
|
46
|
+
|
|
47
|
+
If downloads are failing, consult the [BCGov Data Systems and Services status monitor](https://catalogue-status.data.gov.bc.ca) - this tool depends on BC Open Geospatial Web Services API at [openmaps.gov.bc.ca](https://openmaps.gov.bc.ca/geo/pub/ows?service=WFS&request=Getcapabilities) and the [BC Data Catalogue API](https://catalogue.data.gov.bc.ca/dataset/bc-data-catalogue-api).
|
|
48
|
+
|
|
47
49
|
**Disclaimer**
|
|
48
50
|
*Data are generally licensed as [OGL-BC](http://www2.gov.bc.ca/gov/content/governments/about-the-bc-government/databc/open-data/open-government-license-bc), but it is the user's responsibility to check the licensing for any downloads.*
|
|
49
51
|
|
|
@@ -11,6 +11,9 @@ but most sources are available only via WFS - and the syntax to download WFS dat
|
|
|
11
11
|
|
|
12
12
|
This tool attempts to simplify downloads of BC geographic data and smoothly integrate with PostGIS and Python GIS tools like `geopandas` and `fiona`. The tool only accesses geographic data [avialable via WFS](https://bcgov.github.io/data-publication/pages/tips_tricks_webservices.html) - for other BC open data, download the files directly with `requests` / `curl` / `ogr2ogr` / etc (or the [bcdata R package](https://github.com/bcgov/bcdata)).
|
|
13
13
|
|
|
14
|
+
|
|
15
|
+
If downloads are failing, consult the [BCGov Data Systems and Services status monitor](https://catalogue-status.data.gov.bc.ca) - this tool depends on BC Open Geospatial Web Services API at [openmaps.gov.bc.ca](https://openmaps.gov.bc.ca/geo/pub/ows?service=WFS&request=Getcapabilities) and the [BC Data Catalogue API](https://catalogue.data.gov.bc.ca/dataset/bc-data-catalogue-api).
|
|
16
|
+
|
|
14
17
|
**Disclaimer**
|
|
15
18
|
*Data are generally licensed as [OGL-BC](http://www2.gov.bc.ca/gov/content/governments/about-the-bc-government/databc/open-data/open-government-license-bc), but it is the user's responsibility to check the licensing for any downloads.*
|
|
16
19
|
|
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
services:
|
|
2
2
|
postgres:
|
|
3
|
-
image: postgis/postgis:18-3.
|
|
3
|
+
image: postgis/postgis:18-3.7-alpine
|
|
4
4
|
pull_policy: never
|
|
5
5
|
build:
|
|
6
|
-
context: https://github.com/postgis/docker-postgis.git#master:18-3.
|
|
6
|
+
context: https://github.com/postgis/docker-postgis.git#master:18-3.7/alpine
|
|
7
7
|
environment:
|
|
8
8
|
POSTGRES_PASSWORD: postgres
|
|
9
9
|
ports:
|
|
@@ -11,10 +11,9 @@ authors = [
|
|
|
11
11
|
{name="Simon Norris", email="snorris@hillcrestgeo.ca"},
|
|
12
12
|
]
|
|
13
13
|
description = "Download open data, monitor and report on changes"
|
|
14
|
-
requires-python = ">=3.
|
|
14
|
+
requires-python = ">=3.11"
|
|
15
15
|
classifiers = [
|
|
16
16
|
"Development Status :: 4 - Beta",
|
|
17
|
-
"Programming Language :: Python :: 3.10",
|
|
18
17
|
"Programming Language :: Python :: 3.11",
|
|
19
18
|
"Programming Language :: Python :: 3.12",
|
|
20
19
|
"Programming Language :: Python :: 3.13",
|
|
@@ -27,7 +26,7 @@ dependencies = [
|
|
|
27
26
|
"geoalchemy2",
|
|
28
27
|
"geopandas",
|
|
29
28
|
"owslib",
|
|
30
|
-
"
|
|
29
|
+
"psycopg[binary]",
|
|
31
30
|
"requests",
|
|
32
31
|
"sqlalchemy",
|
|
33
32
|
"stamina"
|
|
@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
|
|
|
18
18
|
commit_id: str | None
|
|
19
19
|
__commit_id__: str | None
|
|
20
20
|
|
|
21
|
-
__version__ = version = '0.
|
|
22
|
-
__version_tuple__ = version_tuple = (0,
|
|
21
|
+
__version__ = version = '0.18.0'
|
|
22
|
+
__version_tuple__ = version_tuple = (0, 18, 0)
|
|
23
23
|
|
|
24
|
-
__commit_id__ = commit_id = '
|
|
24
|
+
__commit_id__ = commit_id = 'g97959038e'
|
|
@@ -2,9 +2,10 @@ import logging
|
|
|
2
2
|
import os
|
|
3
3
|
|
|
4
4
|
from geoalchemy2 import Geometry
|
|
5
|
-
from
|
|
5
|
+
from psycopg import errors, sql
|
|
6
6
|
from sqlalchemy import Column, MetaData, Table, create_engine
|
|
7
7
|
from sqlalchemy.dialects.postgresql import DATE, NUMERIC, VARCHAR
|
|
8
|
+
from sqlalchemy.engine import make_url
|
|
8
9
|
|
|
9
10
|
log = logging.getLogger(__name__)
|
|
10
11
|
|
|
@@ -14,7 +15,12 @@ class Database:
|
|
|
14
15
|
|
|
15
16
|
def __init__(self, url=None):
|
|
16
17
|
self.url = url or os.environ.get("DATABASE_URL")
|
|
17
|
-
|
|
18
|
+
# bcdata uses psycopg (v3) directly; request that driver explicitly
|
|
19
|
+
# (SQLAlchemy <2.1 defaults bare postgresql:// urls to psycopg2)
|
|
20
|
+
sa_url = make_url(self.url)
|
|
21
|
+
if sa_url.drivername in ("postgresql", "postgres", "postgresql+psycopg2"):
|
|
22
|
+
sa_url = sa_url.set(drivername="postgresql+psycopg")
|
|
23
|
+
self.engine = create_engine(sa_url)
|
|
18
24
|
# make sure postgis is available
|
|
19
25
|
try:
|
|
20
26
|
self.query("SELECT postgis_full_version()")
|
|
@@ -5,7 +5,7 @@ import os
|
|
|
5
5
|
import sys
|
|
6
6
|
import warnings
|
|
7
7
|
import xml.etree.ElementTree as ET
|
|
8
|
-
from datetime import datetime, timedelta
|
|
8
|
+
from datetime import UTC, datetime, timedelta
|
|
9
9
|
from pathlib import Path
|
|
10
10
|
from urllib.parse import urlencode
|
|
11
11
|
|
|
@@ -138,10 +138,10 @@ class BCWFS:
|
|
|
138
138
|
if not os.path.exists(os.path.join(cache_file)):
|
|
139
139
|
return True
|
|
140
140
|
else:
|
|
141
|
-
mod_date = datetime.fromtimestamp(os.path.getmtime(cache_file), tz=
|
|
141
|
+
mod_date = datetime.fromtimestamp(os.path.getmtime(cache_file), tz=UTC)
|
|
142
142
|
# if file older than specified days or empty, return true
|
|
143
143
|
return (
|
|
144
|
-
mod_date < (datetime.now(tz=
|
|
144
|
+
mod_date < (datetime.now(tz=UTC) - timedelta(days=self.cache_refresh_days))
|
|
145
145
|
or os.stat(cache_file).st_size == 0
|
|
146
146
|
)
|
|
147
147
|
|
|
@@ -1,26 +1,25 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: bcdata
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.18.0
|
|
4
4
|
Summary: Download open data, monitor and report on changes
|
|
5
5
|
Author-email: Simon Norris <snorris@hillcrestgeo.ca>
|
|
6
6
|
Project-URL: Homepage, https://github.com/smnorris/bcdata
|
|
7
7
|
Project-URL: Issues, https://github.com/smnorris/bcdata
|
|
8
8
|
Classifier: Development Status :: 4 - Beta
|
|
9
|
-
Classifier: Programming Language :: Python :: 3.10
|
|
10
9
|
Classifier: Programming Language :: Python :: 3.11
|
|
11
10
|
Classifier: Programming Language :: Python :: 3.12
|
|
12
11
|
Classifier: Programming Language :: Python :: 3.13
|
|
13
12
|
Classifier: License :: OSI Approved :: Apache Software License
|
|
14
13
|
Classifier: Topic :: Scientific/Engineering :: GIS
|
|
15
14
|
Classifier: Operating System :: OS Independent
|
|
16
|
-
Requires-Python: >=3.
|
|
15
|
+
Requires-Python: >=3.11
|
|
17
16
|
Description-Content-Type: text/markdown
|
|
18
17
|
License-File: LICENSE.txt
|
|
19
18
|
Requires-Dist: click
|
|
20
19
|
Requires-Dist: geoalchemy2
|
|
21
20
|
Requires-Dist: geopandas
|
|
22
21
|
Requires-Dist: owslib
|
|
23
|
-
Requires-Dist:
|
|
22
|
+
Requires-Dist: psycopg[binary]
|
|
24
23
|
Requires-Dist: requests
|
|
25
24
|
Requires-Dist: sqlalchemy
|
|
26
25
|
Requires-Dist: stamina
|
|
@@ -44,6 +43,9 @@ but most sources are available only via WFS - and the syntax to download WFS dat
|
|
|
44
43
|
|
|
45
44
|
This tool attempts to simplify downloads of BC geographic data and smoothly integrate with PostGIS and Python GIS tools like `geopandas` and `fiona`. The tool only accesses geographic data [avialable via WFS](https://bcgov.github.io/data-publication/pages/tips_tricks_webservices.html) - for other BC open data, download the files directly with `requests` / `curl` / `ogr2ogr` / etc (or the [bcdata R package](https://github.com/bcgov/bcdata)).
|
|
46
45
|
|
|
46
|
+
|
|
47
|
+
If downloads are failing, consult the [BCGov Data Systems and Services status monitor](https://catalogue-status.data.gov.bc.ca) - this tool depends on BC Open Geospatial Web Services API at [openmaps.gov.bc.ca](https://openmaps.gov.bc.ca/geo/pub/ows?service=WFS&request=Getcapabilities) and the [BC Data Catalogue API](https://catalogue.data.gov.bc.ca/dataset/bc-data-catalogue-api).
|
|
48
|
+
|
|
47
49
|
**Disclaimer**
|
|
48
50
|
*Data are generally licensed as [OGL-BC](http://www2.gov.bc.ca/gov/content/governments/about-the-bc-government/databc/open-data/open-government-license-bc), but it is the user's responsibility to check the licensing for any downloads.*
|
|
49
51
|
|
|
@@ -7,7 +7,7 @@ import bcdata
|
|
|
7
7
|
from bcdata.cli import cli
|
|
8
8
|
from bcdata.database import Database
|
|
9
9
|
|
|
10
|
-
DB_URL = os.environ.get("
|
|
10
|
+
DB_URL = os.environ.get("BCDATA_TEST_DATABASE_URL")
|
|
11
11
|
DB_CONNECTION = Database(url=DB_URL)
|
|
12
12
|
AIRPORTS_PACKAGE = "bc-airports"
|
|
13
13
|
AIRPORTS_TABLE = "WHSE_IMAGERY_AND_BASE_MAPS.GSR_AIRPORTS_SVW"
|