env-able 0.2.0__tar.gz → 0.3.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {env_able-0.2.0 → env_able-0.3.3}/PKG-INFO +73 -16
- env_able-0.3.3/README.md +142 -0
- {env_able-0.2.0 → env_able-0.3.3}/pyproject.toml +4 -1
- {env_able-0.2.0 → env_able-0.3.3}/src/env_able/__init__.py +2 -1
- env_able-0.3.3/src/env_able/pull.py +257 -0
- env_able-0.2.0/README.md +0 -87
- {env_able-0.2.0 → env_able-0.3.3}/CHANGELOG.md +0 -0
- {env_able-0.2.0 → env_able-0.3.3}/LICENSE +0 -0
- {env_able-0.2.0 → env_able-0.3.3}/src/env_able/morph.py +0 -0
- {env_able-0.2.0 → env_able-0.3.3}/src/env_able/vector.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: env-able
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.3.3
|
|
4
4
|
Summary: AI-ready GIS toolkit for energy and subsurface workflows.
|
|
5
5
|
License: MIT
|
|
6
6
|
License-File: LICENSE
|
|
@@ -9,6 +9,8 @@ Requires-Dist: fiona
|
|
|
9
9
|
Requires-Dist: geopandas
|
|
10
10
|
Requires-Dist: openpyxl
|
|
11
11
|
Requires-Dist: xlrd
|
|
12
|
+
Provides-Extra: databricks
|
|
13
|
+
Requires-Dist: databricks-sql-connector; extra == 'databricks'
|
|
12
14
|
Description-Content-Type: text/markdown
|
|
13
15
|
|
|
14
16
|
# env-able
|
|
@@ -19,24 +21,73 @@ An open source spatial analysis library built for AI-driven GIS workflows. Desig
|
|
|
19
21
|
|
|
20
22
|
```bash
|
|
21
23
|
pip install env-able
|
|
24
|
+
|
|
25
|
+
# with Databricks support
|
|
26
|
+
pip install env-able[databricks]
|
|
22
27
|
```
|
|
23
28
|
|
|
29
|
+
## Usage
|
|
30
|
+
|
|
31
|
+
```python
|
|
32
|
+
import env_able as env
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
---
|
|
36
|
+
|
|
24
37
|
## Functions
|
|
25
38
|
|
|
26
|
-
### `
|
|
39
|
+
### `env.pull(table, output=None, wkt_col=None, chunk_size=None, crs="EPSG:4326")`
|
|
40
|
+
|
|
41
|
+
Pull a Databricks table or query to a local file, chunking around the row/byte limit automatically.
|
|
42
|
+
|
|
43
|
+
Databricks caps result sets (~4 096 rows for narrow tables, fewer when WKT columns are present). `pull()` paginates with `LIMIT`/`OFFSET`, assembles the complete dataset in memory, and writes it to disk in the requested format.
|
|
44
|
+
|
|
45
|
+
**Parameters**
|
|
46
|
+
- `table` — fully-qualified table name (`catalog.schema.table`) or a complete `SELECT` query
|
|
47
|
+
- `output` — destination file path; format inferred from extension (`.gpkg`, `.geojson`, `.shp`, `.parquet`, `.csv`, `.xlsx`). Omit to return a GeoDataFrame or DataFrame.
|
|
48
|
+
- `wkt_col` — column containing WKT geometry strings. Auto-detected if omitted.
|
|
49
|
+
- `chunk_size` — rows per Databricks query. Defaults to 512 (WKT) or 4 096 (tabular). Reduce if you hit payload errors on wide tables.
|
|
50
|
+
- `crs` — CRS to assign geometry. Default `EPSG:4326`.
|
|
51
|
+
|
|
52
|
+
**Environment variables required**
|
|
53
|
+
```
|
|
54
|
+
DATABRICKS_HOST # https://adb-<workspace-id>.azuredatabricks.net
|
|
55
|
+
DATABRICKS_TOKEN # personal access token
|
|
56
|
+
DATABRICKS_HTTP_PATH # /sql/1.0/warehouses/<warehouse-id>
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
```python
|
|
60
|
+
import env_able as env
|
|
61
|
+
|
|
62
|
+
# pull a full table to GeoPackage
|
|
63
|
+
env.pull("catalog.schema.wells", "wells.gpkg")
|
|
64
|
+
|
|
65
|
+
# pull with a filter query
|
|
66
|
+
env.pull("SELECT * FROM catalog.schema.wells WHERE state = 'TX'", "wells_tx.gpkg")
|
|
67
|
+
|
|
68
|
+
# pull tabular (no geometry)
|
|
69
|
+
env.pull("catalog.schema.formations", "formations.csv")
|
|
70
|
+
|
|
71
|
+
# return in-memory without writing
|
|
72
|
+
gdf = env.pull("catalog.schema.leases", wkt_col="geom_wkt", crs="EPSG:4269")
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
### `env.Intersect(input_layer, intersect_layer, output=None)`
|
|
27
78
|
Computes geometric intersection of input features against a polygon boundary.
|
|
28
79
|
- `input_layer` — point, line, or polygon (file path or GeoDataFrame)
|
|
29
80
|
- `intersect_layer` — polygon layer defining the intersection boundary
|
|
30
81
|
- `output` — file path to save result (`.gpkg`, `.shp`, etc.) — omit to return a GeoDataFrame
|
|
31
82
|
|
|
32
83
|
```python
|
|
33
|
-
import env_able as
|
|
34
|
-
|
|
84
|
+
import env_able as env
|
|
85
|
+
env.Intersect("wells.shp", "boundary.shp", "result.gpkg")
|
|
35
86
|
```
|
|
36
87
|
|
|
37
88
|
---
|
|
38
89
|
|
|
39
|
-
### `
|
|
90
|
+
### `env.Buffer(input_layer, distance, unit="meters", output=None)`
|
|
40
91
|
Buffers input features by a given distance and unit.
|
|
41
92
|
- `input_layer` — point, line, or polygon (file path or GeoDataFrame)
|
|
42
93
|
- `distance` — numeric buffer distance
|
|
@@ -44,24 +95,24 @@ Buffers input features by a given distance and unit.
|
|
|
44
95
|
- `output` — file path to save result — omit to return a GeoDataFrame
|
|
45
96
|
|
|
46
97
|
```python
|
|
47
|
-
|
|
98
|
+
env.Buffer("wells.shp", 1, "miles", "wells_buffer.gpkg")
|
|
48
99
|
```
|
|
49
100
|
|
|
50
101
|
---
|
|
51
102
|
|
|
52
|
-
### `
|
|
103
|
+
### `env.Clip(input_layer, clip_layer, output=None)`
|
|
53
104
|
Clips input features to the extent of a polygon clip boundary.
|
|
54
105
|
- `input_layer` — point, line, or polygon (file path or GeoDataFrame)
|
|
55
106
|
- `clip_layer` — polygon layer defining the clip boundary
|
|
56
107
|
- `output` — file path to save result — omit to return a GeoDataFrame
|
|
57
108
|
|
|
58
109
|
```python
|
|
59
|
-
|
|
110
|
+
env.Clip("roads.shp", "county.shp", "roads_clipped.gpkg")
|
|
60
111
|
```
|
|
61
112
|
|
|
62
113
|
---
|
|
63
114
|
|
|
64
|
-
### `
|
|
115
|
+
### `env.morph(input_path, output_path, **kwargs)`
|
|
65
116
|
Universal format translation. Converts between shp, gpkg, gdb, csv, xlsx, xls, dbf, geojson, json with automatic CRS handling, field name fixes, and multi-layer support.
|
|
66
117
|
|
|
67
118
|
- `input_path` — source file or geodatabase
|
|
@@ -71,13 +122,13 @@ Universal format translation. Converts between shp, gpkg, gdb, csv, xlsx, xls, d
|
|
|
71
122
|
- `crs` — coordinate reference system e.g. `EPSG:4326` (required for tabular → spatial)
|
|
72
123
|
|
|
73
124
|
```python
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
125
|
+
env.morph("roads.shp", "roads.gpkg")
|
|
126
|
+
env.morph("county.gdb", "county.gpkg")
|
|
127
|
+
env.morph("county.gdb", "output_folder/")
|
|
128
|
+
env.morph("owners.csv", "owners.geojson", crs="EPSG:4269")
|
|
129
|
+
env.morph("owners.csv", "owners.shp", x_col="LONGITUDE", y_col="LATITUDE", crs="EPSG:4269")
|
|
130
|
+
env.morph("roads.gpkg", "roads.parcels.gpkg")
|
|
131
|
+
env.morph("data.json", "data.gpkg")
|
|
81
132
|
```
|
|
82
133
|
|
|
83
134
|
**Smart behavior:**
|
|
@@ -91,6 +142,12 @@ e.morph("data.json", "data.gpkg")
|
|
|
91
142
|
|
|
92
143
|
## Changelog
|
|
93
144
|
|
|
145
|
+
### v0.3.0 — 2026-07-14
|
|
146
|
+
- `pull()` — paginated Databricks table fetch with WKT auto-detection and chunked assembly
|
|
147
|
+
|
|
148
|
+
### v0.2.0 — 2026-07-14
|
|
149
|
+
- Renamed import alias convention from `e` to `env`
|
|
150
|
+
|
|
94
151
|
### v0.1.0 — 2026-07-14
|
|
95
152
|
- Initial release of env-able
|
|
96
153
|
- `Intersect()`, `Buffer()`, `Clip()` with full guardrails
|
env_able-0.3.3/README.md
ADDED
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
# env-able
|
|
2
|
+
|
|
3
|
+
An open source spatial analysis library built for AI-driven GIS workflows. Designed to give AI systems like Claude reliable, hallucination-free tools for spatial operations in energy and subsurface contexts.
|
|
4
|
+
|
|
5
|
+
## Install
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
pip install env-able
|
|
9
|
+
|
|
10
|
+
# with Databricks support
|
|
11
|
+
pip install env-able[databricks]
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
## Usage
|
|
15
|
+
|
|
16
|
+
```python
|
|
17
|
+
import env_able as env
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Functions
|
|
23
|
+
|
|
24
|
+
### `env.pull(table, output=None, wkt_col=None, chunk_size=None, crs="EPSG:4326")`
|
|
25
|
+
|
|
26
|
+
Pull a Databricks table or query to a local file, chunking around the row/byte limit automatically.
|
|
27
|
+
|
|
28
|
+
Databricks caps result sets (~4 096 rows for narrow tables, fewer when WKT columns are present). `pull()` paginates with `LIMIT`/`OFFSET`, assembles the complete dataset in memory, and writes it to disk in the requested format.
|
|
29
|
+
|
|
30
|
+
**Parameters**
|
|
31
|
+
- `table` — fully-qualified table name (`catalog.schema.table`) or a complete `SELECT` query
|
|
32
|
+
- `output` — destination file path; format inferred from extension (`.gpkg`, `.geojson`, `.shp`, `.parquet`, `.csv`, `.xlsx`). Omit to return a GeoDataFrame or DataFrame.
|
|
33
|
+
- `wkt_col` — column containing WKT geometry strings. Auto-detected if omitted.
|
|
34
|
+
- `chunk_size` — rows per Databricks query. Defaults to 512 (WKT) or 4 096 (tabular). Reduce if you hit payload errors on wide tables.
|
|
35
|
+
- `crs` — CRS to assign geometry. Default `EPSG:4326`.
|
|
36
|
+
|
|
37
|
+
**Environment variables required**
|
|
38
|
+
```
|
|
39
|
+
DATABRICKS_HOST # https://adb-<workspace-id>.azuredatabricks.net
|
|
40
|
+
DATABRICKS_TOKEN # personal access token
|
|
41
|
+
DATABRICKS_HTTP_PATH # /sql/1.0/warehouses/<warehouse-id>
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
```python
|
|
45
|
+
import env_able as env
|
|
46
|
+
|
|
47
|
+
# pull a full table to GeoPackage
|
|
48
|
+
env.pull("catalog.schema.wells", "wells.gpkg")
|
|
49
|
+
|
|
50
|
+
# pull with a filter query
|
|
51
|
+
env.pull("SELECT * FROM catalog.schema.wells WHERE state = 'TX'", "wells_tx.gpkg")
|
|
52
|
+
|
|
53
|
+
# pull tabular (no geometry)
|
|
54
|
+
env.pull("catalog.schema.formations", "formations.csv")
|
|
55
|
+
|
|
56
|
+
# return in-memory without writing
|
|
57
|
+
gdf = env.pull("catalog.schema.leases", wkt_col="geom_wkt", crs="EPSG:4269")
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
### `env.Intersect(input_layer, intersect_layer, output=None)`
|
|
63
|
+
Computes geometric intersection of input features against a polygon boundary.
|
|
64
|
+
- `input_layer` — point, line, or polygon (file path or GeoDataFrame)
|
|
65
|
+
- `intersect_layer` — polygon layer defining the intersection boundary
|
|
66
|
+
- `output` — file path to save result (`.gpkg`, `.shp`, etc.) — omit to return a GeoDataFrame
|
|
67
|
+
|
|
68
|
+
```python
|
|
69
|
+
import env_able as env
|
|
70
|
+
env.Intersect("wells.shp", "boundary.shp", "result.gpkg")
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
---
|
|
74
|
+
|
|
75
|
+
### `env.Buffer(input_layer, distance, unit="meters", output=None)`
|
|
76
|
+
Buffers input features by a given distance and unit.
|
|
77
|
+
- `input_layer` — point, line, or polygon (file path or GeoDataFrame)
|
|
78
|
+
- `distance` — numeric buffer distance
|
|
79
|
+
- `unit` — `meters`, `km`, `miles`, `feet`, `usfeet`, `nautical miles`
|
|
80
|
+
- `output` — file path to save result — omit to return a GeoDataFrame
|
|
81
|
+
|
|
82
|
+
```python
|
|
83
|
+
env.Buffer("wells.shp", 1, "miles", "wells_buffer.gpkg")
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
---
|
|
87
|
+
|
|
88
|
+
### `env.Clip(input_layer, clip_layer, output=None)`
|
|
89
|
+
Clips input features to the extent of a polygon clip boundary.
|
|
90
|
+
- `input_layer` — point, line, or polygon (file path or GeoDataFrame)
|
|
91
|
+
- `clip_layer` — polygon layer defining the clip boundary
|
|
92
|
+
- `output` — file path to save result — omit to return a GeoDataFrame
|
|
93
|
+
|
|
94
|
+
```python
|
|
95
|
+
env.Clip("roads.shp", "county.shp", "roads_clipped.gpkg")
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
---
|
|
99
|
+
|
|
100
|
+
### `env.morph(input_path, output_path, **kwargs)`
|
|
101
|
+
Universal format translation. Converts between shp, gpkg, gdb, csv, xlsx, xls, dbf, geojson, json with automatic CRS handling, field name fixes, and multi-layer support.
|
|
102
|
+
|
|
103
|
+
- `input_path` — source file or geodatabase
|
|
104
|
+
- `output_path` — destination file. Extension sets the format. Use trailing `/` for directory output (one file per layer). Use dot notation for named layers: `roads.parcels.gpkg`
|
|
105
|
+
- `x_col`, `y_col` — column names for X/Y coordinates (auto-detected if not provided)
|
|
106
|
+
- `wkt_col` — column containing WKT geometry (auto-detected if not provided)
|
|
107
|
+
- `crs` — coordinate reference system e.g. `EPSG:4326` (required for tabular → spatial)
|
|
108
|
+
|
|
109
|
+
```python
|
|
110
|
+
env.morph("roads.shp", "roads.gpkg")
|
|
111
|
+
env.morph("county.gdb", "county.gpkg")
|
|
112
|
+
env.morph("county.gdb", "output_folder/")
|
|
113
|
+
env.morph("owners.csv", "owners.geojson", crs="EPSG:4269")
|
|
114
|
+
env.morph("owners.csv", "owners.shp", x_col="LONGITUDE", y_col="LATITUDE", crs="EPSG:4269")
|
|
115
|
+
env.morph("roads.gpkg", "roads.parcels.gpkg")
|
|
116
|
+
env.morph("data.json", "data.gpkg")
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
**Smart behavior:**
|
|
120
|
+
- GDB / GPKG with multiple layers → detects all layers automatically
|
|
121
|
+
- CRS mismatch → auto-reprojects
|
|
122
|
+
- Shapefile field name limit (10 chars) → auto-truncates with warnings
|
|
123
|
+
- Invalid output path → plain English error
|
|
124
|
+
- Empty layers → skipped with a warning, not a crash
|
|
125
|
+
|
|
126
|
+
---
|
|
127
|
+
|
|
128
|
+
## Changelog
|
|
129
|
+
|
|
130
|
+
### v0.3.0 — 2026-07-14
|
|
131
|
+
- `pull()` — paginated Databricks table fetch with WKT auto-detection and chunked assembly
|
|
132
|
+
|
|
133
|
+
### v0.2.0 — 2026-07-14
|
|
134
|
+
- Renamed import alias convention from `e` to `env`
|
|
135
|
+
|
|
136
|
+
### v0.1.0 — 2026-07-14
|
|
137
|
+
- Initial release of env-able
|
|
138
|
+
- `Intersect()`, `Buffer()`, `Clip()` with full guardrails
|
|
139
|
+
- `morph()` — universal format translation
|
|
140
|
+
- Smart geometry detection for WKT and lat/lon columns
|
|
141
|
+
- GeoJSON, JSON support with auto WGS84 reprojection
|
|
142
|
+
- Multi-layer GDB/GPKG support
|
|
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "env-able"
|
|
7
|
-
version = "0.
|
|
7
|
+
version = "0.3.3"
|
|
8
8
|
description = "AI-ready GIS toolkit for energy and subsurface workflows."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
requires-python = ">=3.8"
|
|
@@ -16,6 +16,9 @@ dependencies = [
|
|
|
16
16
|
"xlrd",
|
|
17
17
|
]
|
|
18
18
|
|
|
19
|
+
[project.optional-dependencies]
|
|
20
|
+
databricks = ["databricks-sql-connector"]
|
|
21
|
+
|
|
19
22
|
[tool.hatch.build.targets.wheel]
|
|
20
23
|
packages = ["src/env_able"]
|
|
21
24
|
|
|
@@ -0,0 +1,257 @@
|
|
|
1
|
+
import os
|
|
2
|
+
import pandas as pd
|
|
3
|
+
import geopandas as gpd
|
|
4
|
+
from pathlib import Path
|
|
5
|
+
from shapely import wkt as shapely_wkt
|
|
6
|
+
|
|
7
|
+
_NARROW_CHUNK = 4096 # default rows per query for non-geometry tables
|
|
8
|
+
_WKT_CHUNK = 512 # rows per query when WKT column is present (strings are large)
|
|
9
|
+
|
|
10
|
+
_WKT_COL_NAMES = {
|
|
11
|
+
"geometry", "geom", "wkt", "shape", "geo",
|
|
12
|
+
"the_geom", "geom_wkt", "wkt_geometry", "shape_wkt",
|
|
13
|
+
}
|
|
14
|
+
|
|
15
|
+
_DRIVER_MAP = {
|
|
16
|
+
".gpkg": "GPKG",
|
|
17
|
+
".geojson": "GeoJSON",
|
|
18
|
+
".shp": "ESRI Shapefile",
|
|
19
|
+
".json": "GeoJSON",
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
|
|
23
|
+
def _connect():
|
|
24
|
+
try:
|
|
25
|
+
from databricks import sql as dbsql
|
|
26
|
+
except ImportError:
|
|
27
|
+
raise ImportError(
|
|
28
|
+
"env-able | pull: databricks-sql-connector is not installed.\n"
|
|
29
|
+
" Install it with: pip install databricks-sql-connector\n"
|
|
30
|
+
" Or: pip install env-able[databricks]"
|
|
31
|
+
)
|
|
32
|
+
|
|
33
|
+
host = os.environ.get("DATABRICKS_HOST") or os.environ.get("DATABRICKS_SERVER_HOSTNAME")
|
|
34
|
+
token = os.environ.get("DATABRICKS_TOKEN")
|
|
35
|
+
http_path = os.environ.get("DATABRICKS_HTTP_PATH")
|
|
36
|
+
|
|
37
|
+
if not host:
|
|
38
|
+
raise EnvironmentError(
|
|
39
|
+
"env-able | pull: DATABRICKS_HOST environment variable is not set.\n"
|
|
40
|
+
" Set it to your Databricks workspace URL, e.g.:\n"
|
|
41
|
+
" DATABRICKS_HOST=https://adb-<workspace-id>.azuredatabricks.net"
|
|
42
|
+
)
|
|
43
|
+
if not token:
|
|
44
|
+
raise EnvironmentError(
|
|
45
|
+
"env-able | pull: DATABRICKS_TOKEN environment variable is not set.\n"
|
|
46
|
+
" Set it to a personal access token from your Databricks workspace."
|
|
47
|
+
)
|
|
48
|
+
if not http_path:
|
|
49
|
+
raise EnvironmentError(
|
|
50
|
+
"env-able | pull: DATABRICKS_HTTP_PATH environment variable is not set.\n"
|
|
51
|
+
" Set it to the HTTP path of your SQL warehouse or cluster.\n"
|
|
52
|
+
" Example: /sql/1.0/warehouses/abc123def456"
|
|
53
|
+
)
|
|
54
|
+
|
|
55
|
+
return dbsql.connect(
|
|
56
|
+
server_hostname=host.replace("https://", "").replace("http://", ""),
|
|
57
|
+
http_path=http_path,
|
|
58
|
+
access_token=token,
|
|
59
|
+
)
|
|
60
|
+
|
|
61
|
+
|
|
62
|
+
def _detect_wkt_col(df):
|
|
63
|
+
for col in df.columns:
|
|
64
|
+
if col.lower() in _WKT_COL_NAMES:
|
|
65
|
+
sample = df[col].dropna().head(3)
|
|
66
|
+
if sample.empty:
|
|
67
|
+
continue
|
|
68
|
+
try:
|
|
69
|
+
shapely_wkt.loads(str(sample.iloc[0]))
|
|
70
|
+
return col
|
|
71
|
+
except Exception:
|
|
72
|
+
pass
|
|
73
|
+
# second pass: any column whose values look like WKT keywords
|
|
74
|
+
for col in df.columns:
|
|
75
|
+
sample = df[col].dropna().head(3)
|
|
76
|
+
if sample.empty:
|
|
77
|
+
continue
|
|
78
|
+
if all(str(v).upper().startswith(("POINT", "LINESTRING", "POLYGON", "MULTI", "GEOMETRY"))
|
|
79
|
+
for v in sample):
|
|
80
|
+
return col
|
|
81
|
+
return None
|
|
82
|
+
|
|
83
|
+
|
|
84
|
+
def _write(result, output):
|
|
85
|
+
out = Path(output)
|
|
86
|
+
out.parent.mkdir(parents=True, exist_ok=True)
|
|
87
|
+
ext = out.suffix.lower()
|
|
88
|
+
|
|
89
|
+
if isinstance(result, gpd.GeoDataFrame):
|
|
90
|
+
driver = _DRIVER_MAP.get(ext)
|
|
91
|
+
if driver:
|
|
92
|
+
result.to_file(str(out), driver=driver)
|
|
93
|
+
elif ext == ".parquet":
|
|
94
|
+
result.to_parquet(str(out))
|
|
95
|
+
else:
|
|
96
|
+
raise ValueError(
|
|
97
|
+
f"env-able | pull: unsupported spatial output format '{ext}'.\n"
|
|
98
|
+
f" Supported: .gpkg, .geojson, .shp, .parquet"
|
|
99
|
+
)
|
|
100
|
+
else:
|
|
101
|
+
if ext == ".csv":
|
|
102
|
+
result.to_csv(str(out), index=False)
|
|
103
|
+
elif ext in (".xlsx", ".xls"):
|
|
104
|
+
result.to_excel(str(out), index=False)
|
|
105
|
+
elif ext == ".parquet":
|
|
106
|
+
result.to_parquet(str(out), index=False)
|
|
107
|
+
else:
|
|
108
|
+
raise ValueError(
|
|
109
|
+
f"env-able | pull: unsupported tabular output format '{ext}'.\n"
|
|
110
|
+
f" Supported: .csv, .xlsx, .parquet"
|
|
111
|
+
)
|
|
112
|
+
|
|
113
|
+
print(f"env-able | pull: written → {output}")
|
|
114
|
+
|
|
115
|
+
|
|
116
|
+
def pull(table, output=None, wkt_col=None, chunk_size=None, crs="EPSG:4326"):
|
|
117
|
+
"""
|
|
118
|
+
Pull a Databricks table or query to a local file, chunking around the row/byte limit.
|
|
119
|
+
|
|
120
|
+
Databricks caps result sets (~4 096 rows for narrow tables, fewer when WKT columns
|
|
121
|
+
are present). pull() paginates with LIMIT/OFFSET automatically, assembles the full
|
|
122
|
+
dataset in memory, and writes it to disk in the requested format.
|
|
123
|
+
|
|
124
|
+
Parameters
|
|
125
|
+
----------
|
|
126
|
+
table : str
|
|
127
|
+
Fully-qualified table name (catalog.schema.table)
|
|
128
|
+
OR a complete SELECT query (e.g. "SELECT * FROM catalog.schema.wells WHERE ...")
|
|
129
|
+
output : str, optional
|
|
130
|
+
Destination file path. Format is inferred from the extension:
|
|
131
|
+
Spatial → .gpkg .geojson .shp .parquet
|
|
132
|
+
Tabular → .csv .xlsx .parquet
|
|
133
|
+
If None, returns a GeoDataFrame (when geometry is found) or DataFrame.
|
|
134
|
+
wkt_col : str, optional
|
|
135
|
+
Column name containing WKT geometry strings. Auto-detected if omitted.
|
|
136
|
+
chunk_size : int, optional
|
|
137
|
+
Rows per Databricks query. Defaults to 512 when a WKT column is detected,
|
|
138
|
+
4 096 otherwise. Reduce if you hit payload errors on wide tables.
|
|
139
|
+
crs : str
|
|
140
|
+
CRS to assign to the geometry column. Default EPSG:4326.
|
|
141
|
+
|
|
142
|
+
Returns
|
|
143
|
+
-------
|
|
144
|
+
GeoDataFrame — when a WKT column is detected or supplied
|
|
145
|
+
DataFrame — when no geometry is found
|
|
146
|
+
|
|
147
|
+
Environment variables required
|
|
148
|
+
------------------------------
|
|
149
|
+
DATABRICKS_HOST — workspace URL (https://adb-<id>.azuredatabricks.net)
|
|
150
|
+
DATABRICKS_TOKEN — personal access token
|
|
151
|
+
DATABRICKS_HTTP_PATH — SQL warehouse HTTP path (/sql/1.0/warehouses/...)
|
|
152
|
+
|
|
153
|
+
Examples
|
|
154
|
+
--------
|
|
155
|
+
import env_able as env
|
|
156
|
+
|
|
157
|
+
# pull a full table to GeoPackage
|
|
158
|
+
env.pull("catalog.schema.wells", "wells.gpkg")
|
|
159
|
+
|
|
160
|
+
# pull with a filter query
|
|
161
|
+
env.pull("SELECT * FROM catalog.schema.wells WHERE state = 'TX'", "wells_tx.gpkg")
|
|
162
|
+
|
|
163
|
+
# pull tabular (no geometry)
|
|
164
|
+
env.pull("catalog.schema.formations", "formations.csv")
|
|
165
|
+
|
|
166
|
+
# return in-memory without writing
|
|
167
|
+
gdf = env.pull("catalog.schema.leases", wkt_col="geom_wkt", crs="EPSG:4269")
|
|
168
|
+
"""
|
|
169
|
+
is_query = table.strip().upper().startswith("SELECT")
|
|
170
|
+
base_sql = table.strip() if is_query else f"SELECT * FROM {table}"
|
|
171
|
+
label = table if is_query else table.split(".")[-1]
|
|
172
|
+
|
|
173
|
+
con = _connect()
|
|
174
|
+
print(f"env-able | pull: connected to Databricks")
|
|
175
|
+
|
|
176
|
+
try:
|
|
177
|
+
# --- row count ---
|
|
178
|
+
count_sql = f"SELECT COUNT(*) AS _n FROM ({base_sql}) _count_wrap"
|
|
179
|
+
with con.cursor() as cur:
|
|
180
|
+
cur.execute(count_sql)
|
|
181
|
+
total_rows = cur.fetchone()[0]
|
|
182
|
+
|
|
183
|
+
if total_rows == 0:
|
|
184
|
+
print(f"env-able | warning | pull: '{label}' returned 0 rows — nothing to fetch.")
|
|
185
|
+
return gpd.GeoDataFrame() if wkt_col else pd.DataFrame()
|
|
186
|
+
|
|
187
|
+
print(f"env-able | pull: {total_rows:,} rows in '{label}'")
|
|
188
|
+
|
|
189
|
+
# --- probe first rows to detect WKT and columns ---
|
|
190
|
+
with con.cursor() as cur:
|
|
191
|
+
cur.execute(f"{base_sql} LIMIT 5")
|
|
192
|
+
cols = [d[0] for d in cur.description]
|
|
193
|
+
probe_df = pd.DataFrame(cur.fetchall(), columns=cols)
|
|
194
|
+
|
|
195
|
+
if wkt_col is None:
|
|
196
|
+
wkt_col = _detect_wkt_col(probe_df)
|
|
197
|
+
if wkt_col:
|
|
198
|
+
print(f"env-able | pull: WKT column auto-detected → '{wkt_col}'")
|
|
199
|
+
else:
|
|
200
|
+
print("env-able | pull: no WKT column detected — returning tabular data")
|
|
201
|
+
|
|
202
|
+
if chunk_size is None:
|
|
203
|
+
chunk_size = _WKT_CHUNK if wkt_col else _NARROW_CHUNK
|
|
204
|
+
mode_label = "WKT mode" if wkt_col else "narrow mode"
|
|
205
|
+
print(f"env-able | pull: chunk size = {chunk_size} rows ({mode_label})")
|
|
206
|
+
|
|
207
|
+
# --- paginated fetch ---
|
|
208
|
+
chunks = []
|
|
209
|
+
offset = 0
|
|
210
|
+
|
|
211
|
+
while offset < total_rows:
|
|
212
|
+
chunk_sql = f"{base_sql} LIMIT {chunk_size} OFFSET {offset}"
|
|
213
|
+
with con.cursor() as cur:
|
|
214
|
+
cur.execute(chunk_sql)
|
|
215
|
+
rows = cur.fetchall()
|
|
216
|
+
|
|
217
|
+
if not rows:
|
|
218
|
+
break
|
|
219
|
+
|
|
220
|
+
chunks.append(pd.DataFrame(rows, columns=cols))
|
|
221
|
+
fetched = offset + len(rows)
|
|
222
|
+
print(f"env-able | pull: {min(fetched, total_rows):,} / {total_rows:,} rows fetched")
|
|
223
|
+
offset += chunk_size
|
|
224
|
+
|
|
225
|
+
finally:
|
|
226
|
+
con.close()
|
|
227
|
+
|
|
228
|
+
if not chunks:
|
|
229
|
+
raise ValueError(
|
|
230
|
+
f"env-able | pull: no data returned from '{label}'.\n"
|
|
231
|
+
f" The row count was {total_rows} but all chunks came back empty.\n"
|
|
232
|
+
f" Check that your query or table name is correct."
|
|
233
|
+
)
|
|
234
|
+
|
|
235
|
+
df = pd.concat(chunks, ignore_index=True)
|
|
236
|
+
print(f"env-able | pull: assembly complete — {len(df):,} rows, {len(df.columns)} columns")
|
|
237
|
+
|
|
238
|
+
# --- build geometry ---
|
|
239
|
+
if wkt_col and wkt_col in df.columns:
|
|
240
|
+
try:
|
|
241
|
+
geometry = gpd.GeoSeries.from_wkt(df[wkt_col], crs=crs)
|
|
242
|
+
result = gpd.GeoDataFrame(df.drop(columns=[wkt_col]), geometry=geometry, crs=crs)
|
|
243
|
+
print(f"env-able | pull: geometry built from '{wkt_col}' ({crs})")
|
|
244
|
+
except Exception as exc:
|
|
245
|
+
print(
|
|
246
|
+
f"env-able | warning | pull: geometry build failed — returning plain DataFrame.\n"
|
|
247
|
+
f" Error: {exc}"
|
|
248
|
+
)
|
|
249
|
+
result = df
|
|
250
|
+
else:
|
|
251
|
+
result = df
|
|
252
|
+
|
|
253
|
+
if output is None:
|
|
254
|
+
return result
|
|
255
|
+
|
|
256
|
+
_write(result, output)
|
|
257
|
+
return result
|
env_able-0.2.0/README.md
DELETED
|
@@ -1,87 +0,0 @@
|
|
|
1
|
-
# env-able
|
|
2
|
-
|
|
3
|
-
An open source spatial analysis library built for AI-driven GIS workflows. Designed to give AI systems like Claude reliable, hallucination-free tools for spatial operations in energy and subsurface contexts.
|
|
4
|
-
|
|
5
|
-
## Install
|
|
6
|
-
|
|
7
|
-
```bash
|
|
8
|
-
pip install env-able
|
|
9
|
-
```
|
|
10
|
-
|
|
11
|
-
## Functions
|
|
12
|
-
|
|
13
|
-
### `e.Intersect(input_layer, intersect_layer, output=None)`
|
|
14
|
-
Computes geometric intersection of input features against a polygon boundary.
|
|
15
|
-
- `input_layer` — point, line, or polygon (file path or GeoDataFrame)
|
|
16
|
-
- `intersect_layer` — polygon layer defining the intersection boundary
|
|
17
|
-
- `output` — file path to save result (`.gpkg`, `.shp`, etc.) — omit to return a GeoDataFrame
|
|
18
|
-
|
|
19
|
-
```python
|
|
20
|
-
import env_able as e
|
|
21
|
-
e.Intersect("wells.shp", "boundary.shp", "result.gpkg")
|
|
22
|
-
```
|
|
23
|
-
|
|
24
|
-
---
|
|
25
|
-
|
|
26
|
-
### `e.Buffer(input_layer, distance, unit="meters", output=None)`
|
|
27
|
-
Buffers input features by a given distance and unit.
|
|
28
|
-
- `input_layer` — point, line, or polygon (file path or GeoDataFrame)
|
|
29
|
-
- `distance` — numeric buffer distance
|
|
30
|
-
- `unit` — `meters`, `km`, `miles`, `feet`, `usfeet`, `nautical miles`
|
|
31
|
-
- `output` — file path to save result — omit to return a GeoDataFrame
|
|
32
|
-
|
|
33
|
-
```python
|
|
34
|
-
e.Buffer("wells.shp", 1, "miles", "wells_buffer.gpkg")
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
---
|
|
38
|
-
|
|
39
|
-
### `e.Clip(input_layer, clip_layer, output=None)`
|
|
40
|
-
Clips input features to the extent of a polygon clip boundary.
|
|
41
|
-
- `input_layer` — point, line, or polygon (file path or GeoDataFrame)
|
|
42
|
-
- `clip_layer` — polygon layer defining the clip boundary
|
|
43
|
-
- `output` — file path to save result — omit to return a GeoDataFrame
|
|
44
|
-
|
|
45
|
-
```python
|
|
46
|
-
e.Clip("roads.shp", "county.shp", "roads_clipped.gpkg")
|
|
47
|
-
```
|
|
48
|
-
|
|
49
|
-
---
|
|
50
|
-
|
|
51
|
-
### `e.morph(input_path, output_path, **kwargs)`
|
|
52
|
-
Universal format translation. Converts between shp, gpkg, gdb, csv, xlsx, xls, dbf, geojson, json with automatic CRS handling, field name fixes, and multi-layer support.
|
|
53
|
-
|
|
54
|
-
- `input_path` — source file or geodatabase
|
|
55
|
-
- `output_path` — destination file. Extension sets the format. Use trailing `/` for directory output (one file per layer). Use dot notation for named layers: `roads.parcels.gpkg`
|
|
56
|
-
- `x_col`, `y_col` — column names for X/Y coordinates (auto-detected if not provided)
|
|
57
|
-
- `wkt_col` — column containing WKT geometry (auto-detected if not provided)
|
|
58
|
-
- `crs` — coordinate reference system e.g. `EPSG:4326` (required for tabular → spatial)
|
|
59
|
-
|
|
60
|
-
```python
|
|
61
|
-
e.morph("roads.shp", "roads.gpkg")
|
|
62
|
-
e.morph("county.gdb", "county.gpkg")
|
|
63
|
-
e.morph("county.gdb", "output_folder/")
|
|
64
|
-
e.morph("owners.csv", "owners.geojson", crs="EPSG:4269")
|
|
65
|
-
e.morph("owners.csv", "owners.shp", x_col="LONGITUDE", y_col="LATITUDE", crs="EPSG:4269")
|
|
66
|
-
e.morph("roads.gpkg", "roads.parcels.gpkg")
|
|
67
|
-
e.morph("data.json", "data.gpkg")
|
|
68
|
-
```
|
|
69
|
-
|
|
70
|
-
**Smart behavior:**
|
|
71
|
-
- GDB / GPKG with multiple layers → detects all layers automatically
|
|
72
|
-
- CRS mismatch → auto-reprojects
|
|
73
|
-
- Shapefile field name limit (10 chars) → auto-truncates with warnings
|
|
74
|
-
- Invalid output path → plain English error
|
|
75
|
-
- Empty layers → skipped with a warning, not a crash
|
|
76
|
-
|
|
77
|
-
---
|
|
78
|
-
|
|
79
|
-
## Changelog
|
|
80
|
-
|
|
81
|
-
### v0.1.0 — 2026-07-14
|
|
82
|
-
- Initial release of env-able
|
|
83
|
-
- `Intersect()`, `Buffer()`, `Clip()` with full guardrails
|
|
84
|
-
- `morph()` — universal format translation
|
|
85
|
-
- Smart geometry detection for WKT and lat/lon columns
|
|
86
|
-
- GeoJSON, JSON support with auto WGS84 reprojection
|
|
87
|
-
- Multi-layer GDB/GPKG support
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|