geocrime-stl 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 lweber89
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,85 @@
1
+ Metadata-Version: 2.4
2
+ Name: geocrime-stl
3
+ Version: 0.1.0
4
+ Summary: An ETL for fetching/cleaning St. Louis crime data sets.
5
+ Requires-Python: >=3.8
6
+ Description-Content-Type: text/markdown
7
+ License-File: LICENSE
8
+ Requires-Dist: geopandas>=1.1.3
9
+ Requires-Dist: matplotlib>=3.10.9
10
+ Requires-Dist: pandas>=3.0.3
11
+ Requires-Dist: requests>=2.34.2
12
+ Dynamic: license-file
13
+
14
+ # geocrime-stl
15
+
16
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
17
+ [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
18
+
19
+ A Python utility for cleaning and standardizing St. Louis Metropolitan Police Department (SLMPD) crime data for spatial analysis.
20
+
21
+ ## ✨ Features
22
+
23
+ * **End-to-End ETL:** Fetch, clean, and standardize monthly SLMPD crime data files automatically.
24
+ * **Spatial Formats:** Convert cleaned DataFrames seamlessly into GeoJSON, GeoPackage (GPKG), or CSV.
25
+ * **Analysis Ready:** Access built-in methods for minimal data summarization and visualization.
26
+
27
+ ## 🌐 Data Source & Disclaimer
28
+
29
+ This utility programmatically fetches publicly available crime data directly from the **[St. Louis Metropolitan Police Department (SLMPD) Stats Page](https://slmpd.org/stats/)**.
30
+
31
+ ⚠️ **Data Disclaimer & Limitations:**
32
+ * **Availability:** This tool relies entirely on the upstream availability and hosting structure of the SLMPD website. If their servers are down or their URL layout changes, the extraction pipeline may fail.
33
+ * **Data Integrity & Spatial Boundaries:** The data processed is strictly "as-is" from the published monthly files. To ensure high data quality for spatial analysis, this utility applies two strict filtering constraints:
34
+ 1. **Temporal Filtering:** Because the SLMPD frequently updates historical records retroactively, this utility isolates and preserves only the records belonging to the target month.
35
+ 2. **Spatial Clipping:** Records containing invalid, malformed, or erroneous latitude/longitude coordinates that plot completely outside the official St. Louis city boundaries are automatically dropped.
36
+ * **Project Status:** This is an independent, open-source utility. It is not affiliated with, endorsed by, or officially maintained by the SLMPD or the City of St. Louis.
37
+
38
+ ## 🚀 Quick Start
39
+
40
+ ### Install
41
+ ``` bash
42
+ pip install geocrime-stl
43
+ ```
44
+ ### Fetch/Clean/Visualize Data
45
+ ```python
46
+ import geocrime_stl as gc
47
+
48
+ data_pkg = gc.run_pipeline(4, 2026)
49
+
50
+ gc.generate_monthly_metrics(data_pkg)
51
+
52
+ gc.plot_monthly_maps(data_pkg)
53
+ ```
54
+
55
+ ## 🛠️ Local Setup (Alternative)
56
+
57
+ If you want to run the demo notebook locally or explore the source code, you can clone the repository directly:
58
+
59
+ ```bash
60
+ git clone https://github.com/lweber89/geocrime-stl.git
61
+ cd geocrime-stl
62
+ pip install -e .
63
+ ```
64
+
65
+ ## 📓 Demo & Tutorial
66
+ To see a step-by-step walkthrough of the entire ETL process, check out the interactive Jupyter notebook included in this package:
67
+
68
+ 👉 **[View the Demo Notebook](./notebooks/demo.ipynb)**
69
+
70
+ Note: If you have cloned the repository locally, you can open this file using Jupyter Lab or Jupyter Notebook to run the cells yourself.
71
+
72
+ ## 🤝 Contributing & Pull Requests
73
+ Thank you for your interest in the project!
74
+
75
+ ⚠️ Please Note: This is a personal project I developed and maintain in an effort to better understand the implementation of Python in geospatial data engineering / solutions architecture. I am not accepting pull requests or code contributions at this time.
76
+
77
+ If you find a bug or have an idea, feel free to open an Issue to let me know, or feel free to fork the repository and adapt it for your own personal needs!
78
+
79
+ ## 📜 License
80
+ This project is licensed under the MIT License - see the LICENSE file for details.
81
+
82
+ ## 🔗 Project Links
83
+ GitHub Repository: https://github.com/lweber89/geocrime-stl
84
+
85
+ PyPI Home: https://pypi.org/project/geocrime-stl/
@@ -0,0 +1,72 @@
1
+ # geocrime-stl
2
+
3
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
4
+ [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
5
+
6
+ A Python utility for cleaning and standardizing St. Louis Metropolitan Police Department (SLMPD) crime data for spatial analysis.
7
+
8
+ ## ✨ Features
9
+
10
+ * **End-to-End ETL:** Fetch, clean, and standardize monthly SLMPD crime data files automatically.
11
+ * **Spatial Formats:** Convert cleaned DataFrames seamlessly into GeoJSON, GeoPackage (GPKG), or CSV.
12
+ * **Analysis Ready:** Access built-in methods for minimal data summarization and visualization.
13
+
14
+ ## 🌐 Data Source & Disclaimer
15
+
16
+ This utility programmatically fetches publicly available crime data directly from the **[St. Louis Metropolitan Police Department (SLMPD) Stats Page](https://slmpd.org/stats/)**.
17
+
18
+ ⚠️ **Data Disclaimer & Limitations:**
19
+ * **Availability:** This tool relies entirely on the upstream availability and hosting structure of the SLMPD website. If their servers are down or their URL layout changes, the extraction pipeline may fail.
20
+ * **Data Integrity & Spatial Boundaries:** The data processed is strictly "as-is" from the published monthly files. To ensure high data quality for spatial analysis, this utility applies two strict filtering constraints:
21
+ 1. **Temporal Filtering:** Because the SLMPD frequently updates historical records retroactively, this utility isolates and preserves only the records belonging to the target month.
22
+ 2. **Spatial Clipping:** Records containing invalid, malformed, or erroneous latitude/longitude coordinates that plot completely outside the official St. Louis city boundaries are automatically dropped.
23
+ * **Project Status:** This is an independent, open-source utility. It is not affiliated with, endorsed by, or officially maintained by the SLMPD or the City of St. Louis.
24
+
25
+ ## 🚀 Quick Start
26
+
27
+ ### Install
28
+ ``` bash
29
+ pip install geocrime-stl
30
+ ```
31
+ ### Fetch/Clean/Visualize Data
32
+ ```python
33
+ import geocrime_stl as gc
34
+
35
+ data_pkg = gc.run_pipeline(4, 2026)
36
+
37
+ gc.generate_monthly_metrics(data_pkg)
38
+
39
+ gc.plot_monthly_maps(data_pkg)
40
+ ```
41
+
42
+ ## 🛠️ Local Setup (Alternative)
43
+
44
+ If you want to run the demo notebook locally or explore the source code, you can clone the repository directly:
45
+
46
+ ```bash
47
+ git clone https://github.com/lweber89/geocrime-stl.git
48
+ cd geocrime-stl
49
+ pip install -e .
50
+ ```
51
+
52
+ ## 📓 Demo & Tutorial
53
+ To see a step-by-step walkthrough of the entire ETL process, check out the interactive Jupyter notebook included in this package:
54
+
55
+ 👉 **[View the Demo Notebook](./notebooks/demo.ipynb)**
56
+
57
+ Note: If you have cloned the repository locally, you can open this file using Jupyter Lab or Jupyter Notebook to run the cells yourself.
58
+
59
+ ## 🤝 Contributing & Pull Requests
60
+ Thank you for your interest in the project!
61
+
62
+ ⚠️ Please Note: This is a personal project I developed and maintain in an effort to better understand the implementation of Python in geospatial data engineering / solutions architecture. I am not accepting pull requests or code contributions at this time.
63
+
64
+ If you find a bug or have an idea, feel free to open an Issue to let me know, or feel free to fork the repository and adapt it for your own personal needs!
65
+
66
+ ## 📜 License
67
+ This project is licensed under the MIT License - see the LICENSE file for details.
68
+
69
+ ## 🔗 Project Links
70
+ GitHub Repository: https://github.com/lweber89/geocrime-stl
71
+
72
+ PyPI Home: https://pypi.org/project/geocrime-stl/
@@ -0,0 +1,23 @@
1
+ [build-system]
2
+ requires = ["setuptools>=61.0"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "geocrime-stl"
7
+ version = "0.1.0"
8
+ description = "An ETL for fetching/cleaning St. Louis crime data sets."
9
+ readme = "README.md"
10
+ requires-python = ">=3.8"
11
+ dependencies = [
12
+ "geopandas>=1.1.3",
13
+ "matplotlib>=3.10.9",
14
+ "pandas>=3.0.3",
15
+ "requests>=2.34.2"
16
+ ]
17
+
18
+ [tool.setuptools.packages.find]
19
+ where = ["src"]
20
+ include = ["geocrime_stl"]
21
+
22
+ [tool.setuptools.package-data]
23
+ geocrime_stl = ["data/*.geojson"]
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,24 @@
1
+ from __future__ import annotations
2
+
3
+ from .config import CITY_BNDY, NBHD_BNDY, STL_MAP_CONFIG
4
+
5
+ # Expose the master pipeline orchestrator directly to the root namespace
6
+ from .etl.transform import run_pipeline
7
+
8
+ # Expose the monthly summary and plottin methods to the root namespace
9
+ from .insights.monthly_summary import generate_monthly_metrics, plot_monthly_maps
10
+
11
+ # Expose the utilities and exporters cleanly to the root namespace
12
+ from .utils.export import export_to_csv, export_to_geojson, export_to_gpkg
13
+
14
+ __all__ = [
15
+ "CITY_BNDY",
16
+ "NBHD_BNDY",
17
+ "STL_MAP_CONFIG",
18
+ "run_pipeline",
19
+ "export_to_csv",
20
+ "export_to_geojson",
21
+ "export_to_gpkg",
22
+ "plot_monthly_maps",
23
+ "generate_monthly_metrics"
24
+ ]
@@ -0,0 +1,75 @@
1
+ from pathlib import Path
2
+ from typing import NamedTuple
3
+
4
+ # --- Paths ---
5
+ # .parent takes us to 'geocrime_stl', the second .parent takes us out to 'src'
6
+ # The third .parent takes us to the root project directory where 'data' lives!
7
+ BASE_DIR = Path(__file__).resolve().parent.parent.parent
8
+ DATA_DIR = BASE_DIR / "data"
9
+
10
+ CITY_BNDY = DATA_DIR / "stl_boundary.geojson"
11
+ NBHD_BNDY = DATA_DIR / "stl_neighborhoods.geojson"
12
+ CITY_BNDY_WGS84 = DATA_DIR / "stl_boundary_wgs84.geojson"
13
+
14
+
15
+ # --- Map Configuration ---
16
+ class MapConfig(NamedTuple):
17
+ coords: tuple[float, float]
18
+ zoom: float
19
+ height: str
20
+ bbox: tuple[float, float, float, float]
21
+
22
+ STL_MAP_CONFIG = MapConfig(
23
+ coords=(38.65428167189044, -90.25320053100587),
24
+ zoom=11.5,
25
+ height="925px",
26
+ bbox=(38.5318, -90.3205, 38.7744, -90.1663),
27
+ )
28
+
29
+
30
+ # --- SLMPD ETL Settings ---
31
+ BASE_URL = "https://slmpd.org/wp-content/uploads"
32
+
33
+ COLUMN_RENAMES = {
34
+ "Latitude": "lat",
35
+ "Longitude": "lon",
36
+ "IncidentNum": "inc_#",
37
+ "NIBRS": "nibrs_code",
38
+ "NIBRSCategory": "nibrs_cat",
39
+ "CrimeAgainst": "off_type",
40
+ "IncidentLocation": "address",
41
+ "District": "district",
42
+ "Neighborhood": "nbhd",
43
+ "NbhdNum": "nbhd_num",
44
+ "FirearmUsed": "firearm",
45
+ "IncidentNature": "inc_desc",
46
+ "Offense": "offense",
47
+ }
48
+
49
+ COLUMNS_TO_DROP = (
50
+ "IncidentDate",
51
+ "OccurredFromTime",
52
+ "SRS_UCR",
53
+ "IncidentTopSRS_UCR",
54
+ "IntersectionOtherLoc",
55
+ "IncidentSupplemented",
56
+ "LastSuppDate",
57
+ "VictimNum",
58
+ "FelMisdCit",
59
+ )
60
+ FINAL_SCHEMA_COLUMNS = [
61
+ "inc_#",
62
+ "date_time",
63
+ "nibrs_cat",
64
+ "nibrs_code",
65
+ "offense",
66
+ "inc_desc",
67
+ "off_type",
68
+ "firearm",
69
+ "address",
70
+ "nbhd",
71
+ "nbhd_num",
72
+ "district",
73
+ "lat",
74
+ "lon",
75
+ ]
@@ -0,0 +1,93 @@
1
+ from __future__ import annotations
2
+
3
+ import logging
4
+ from datetime import datetime
5
+
6
+ import requests
7
+
8
+ from geocrime_stl.config import BASE_URL
9
+
10
+ logger = logging.getLogger(__name__)
11
+
12
+ class ExtractionError(Exception):
13
+ """Raised if the target URL cannot be formed or network operations fail."""
14
+ pass
15
+
16
+
17
+ def construct_url(
18
+ month_input: str | int | None = None,
19
+ year_input: str | int | None = None
20
+ ) -> tuple[str, int, int]:
21
+ """Parses flexible date inputs and builds the target SLMPD data asset URL."""
22
+ try:
23
+ now = datetime.now()
24
+ if now.month == 1:
25
+ default_month = 12
26
+ default_year = now.year - 1
27
+ else:
28
+ default_month = now.month - 1
29
+ default_year = now.year
30
+
31
+ # Parse year input into consistent types
32
+ if year_input is None:
33
+ year_int = default_year
34
+ year_str = str(year_int)
35
+ else:
36
+ year_str = str(year_input).strip()
37
+ year_int = int(year_str)
38
+
39
+ # Parse month input into consistent types
40
+ if month_input is None:
41
+ month_int = default_month
42
+ month_str = datetime.strptime(str(month_int), "%m").strftime("%B")
43
+ else:
44
+ clean_month = str(month_input).strip()
45
+ try:
46
+ month_int = int(clean_month)
47
+ month_str = datetime.strptime(str(month_int), "%m").strftime("%B")
48
+ except ValueError:
49
+ try:
50
+ dt = datetime.strptime(clean_month.capitalize(), "%B")
51
+ except ValueError:
52
+ dt = datetime.strptime(clean_month.capitalize(), "%b")
53
+ month_int = dt.month
54
+ month_str = dt.strftime("%B")
55
+
56
+ # Handle the city's specific upload folder schedule offset
57
+ folder_month_int = (month_int % 12) + 1
58
+ folder_month_str = f"{folder_month_int:02d}"
59
+ folder_year_str = str(year_int + 1) if month_int == 12 else year_str
60
+
61
+ url = f"{BASE_URL}/{folder_year_str}/{folder_month_str}/{month_str}{year_str}.csv"
62
+ return url, month_int, year_int
63
+
64
+ except Exception as e:
65
+ raise ExtractionError(f"Invalid month ('{month_input}') or year ('{year_input}') mapping: {e}") from e
66
+
67
+
68
+ def fetch_crime_data_csv(url: str, keep_raw_csv: bool = False) -> bytes:
69
+ """Downloads raw network bytes. Standardized on 'keep_raw_csv' to match project rules."""
70
+ headers = {"User-Agent": "Mozilla/5.0"}
71
+ filename = url.split("/")[-1] or "downloaded_data.csv"
72
+
73
+ logger.info(f"Attempting network download from: {url}")
74
+ try:
75
+ response = requests.get(url, headers=headers, timeout=10)
76
+ response.raise_for_status()
77
+ raw_bytes = response.content
78
+ except requests.exceptions.HTTPError as http_err:
79
+ raise ExtractionError(f"HTTP error downloading data asset: {http_err}") from http_err
80
+ except requests.exceptions.RequestException as net_err:
81
+ raise ExtractionError(f"Network transport error reaching server: {net_err}") from net_err
82
+ except Exception as e:
83
+ raise ExtractionError(f"Unexpected extraction execution failure: {e}") from e
84
+
85
+ if keep_raw_csv:
86
+ try:
87
+ with open(filename, "wb") as f:
88
+ f.write(raw_bytes)
89
+ logger.info(f"💾 Raw CSV backup cached locally: {filename}")
90
+ except IOError as io_err:
91
+ logger.warning(f"⚠️ Could not write raw data local cache: {io_err}")
92
+
93
+ return raw_bytes
@@ -0,0 +1,160 @@
1
+ from __future__ import annotations
2
+
3
+ import logging
4
+ from dataclasses import dataclass
5
+
6
+ import geopandas as gpd
7
+ import pandas as pd
8
+
9
+ from geocrime_stl.config import (
10
+ CITY_BNDY_WGS84,
11
+ COLUMN_RENAMES,
12
+ COLUMNS_TO_DROP,
13
+ FINAL_SCHEMA_COLUMNS,
14
+ )
15
+
16
+ logger = logging.getLogger(__name__)
17
+
18
+
19
+ @dataclass(frozen=True)
20
+ class CrimeDataPackage:
21
+ """Wrapper that bundles structured monthly crime data with its target temporal metadata."""
22
+
23
+ df: pd.DataFrame
24
+ month: int
25
+ year: int
26
+
27
+
28
+ def raw_csv_to_dataframe(csv_bytes: bytes, **kwargs) -> pd.DataFrame:
29
+ """Converts raw byte stream cleanly into an in-memory Pandas DataFrame.
30
+
31
+ Accepts optional kwargs to pass directly to pd.read_csv if needed down the road.
32
+ """
33
+ import io
34
+
35
+ try:
36
+ return pd.read_csv(io.BytesIO(csv_bytes), **kwargs)
37
+ except Exception as e:
38
+ raise ValueError(f"Failed to decode raw byte stream to DataFrame: {e}") from e
39
+
40
+
41
+ def clean_and_spatial_clip(
42
+ df: pd.DataFrame, month: int, year: int
43
+ ) -> CrimeDataPackage:
44
+ """Normalizes features, handles type enforcement, and spatially clips observations
45
+
46
+ to St. Louis City limits. Returns a populated CrimeDataPackage wrapper.
47
+ """
48
+ # Handle the original empty CSV dataset scenario gracefully
49
+ if df.empty:
50
+ logger.warning(
51
+ f"⚠️ Ingested DataFrame was empty. Skipping cleaning pipelines for {month}/{year}."
52
+ )
53
+ return CrimeDataPackage(df=df, month=month, year=year)
54
+
55
+ # Date Alignments & Conversions
56
+ df["IncidentDate"] = pd.to_datetime(
57
+ df["IncidentDate"], format="%m/%d/%Y", exact=False, errors="coerce"
58
+ )
59
+
60
+ # Filter by date alignment (keeps only requested month/year observations)
61
+ df = df[(df["IncidentDate"].dt.month == month) & (df["IncidentDate"].dt.year == year)].copy()
62
+
63
+ # String Timestamp Synthesis
64
+ df["OccurredFromTime"] = df["OccurredFromTime"].fillna("00:00")
65
+ combined_time_str = (
66
+ df["IncidentDate"].dt.strftime("%Y-%m-%d") + " " + df["OccurredFromTime"].astype(str)
67
+ )
68
+ df["date_time"] = pd.to_datetime(combined_time_str, errors="coerce")
69
+
70
+ # Structural Re-mapping & Column Dropping
71
+ df = df.rename(columns=COLUMN_RENAMES)
72
+ df = df.drop(columns=COLUMNS_TO_DROP, errors="ignore")
73
+
74
+ # Data Type Enforcement
75
+ df["district"] = pd.to_numeric(df["district"], errors="coerce")
76
+ df["district"] = df["district"].astype("Int64").astype("string")
77
+ df["inc_#"] = df["inc_#"].astype("string")
78
+
79
+ # Regex Character Stripping (e.g., "G-88" -> 88)
80
+ df["nbhd_num"] = df["nbhd_num"].astype(str).str.extract(r"(\d+)", expand=False)
81
+ df["nbhd_num"] = pd.to_numeric(df["nbhd_num"], errors="coerce").astype("Int64")
82
+ df["off_type"] = df["off_type"].replace("Unspecified", "Other")
83
+
84
+ # Coordinate Sanitization
85
+ df["lon"] = pd.to_numeric(df["lon"], errors="coerce")
86
+ df["lat"] = pd.to_numeric(df["lat"], errors="coerce")
87
+ df = df.dropna(subset=["lon", "lat"]).copy()
88
+
89
+ # Apply Final Schema Alignment from global configuration
90
+ existing_cols = [col for col in FINAL_SCHEMA_COLUMNS if col in df.columns]
91
+ df = df[existing_cols]
92
+
93
+ # Spatial Boundary Clipping Execution
94
+ try:
95
+ clipped_df = clip_df(df)
96
+ return CrimeDataPackage(df=clipped_df, month=month, year=year)
97
+ except Exception as e:
98
+ logger.warning(f"⚠️ Spatial fencing bypassed due to pipeline engine block: {e}")
99
+ logger.info("▶️ Flowing original full metropolitan boundary coordinates forward.")
100
+ return CrimeDataPackage(df=df, month=month, year=year)
101
+
102
+
103
+ def clip_df(df: pd.DataFrame) -> pd.DataFrame:
104
+ """Restricts point coordinates to the formal polygon limits of St. Louis City
105
+
106
+ using an inner spatial join.
107
+ """
108
+ if not CITY_BNDY_WGS84.exists():
109
+ raise FileNotFoundError(
110
+ f"Could not locate boundary file asset at designated config path: {CITY_BNDY_WGS84}"
111
+ )
112
+
113
+ # Temporarily switch structure to an explicit GeoDataFrame ('gdf')
114
+ gdf = gpd.GeoDataFrame(
115
+ df, geometry=gpd.points_from_xy(df.lon, df.lat), crs="EPSG:4326"
116
+ )
117
+
118
+ # Ingest the boundary file directly using your pathlib setup
119
+ stl_boundary = gpd.read_file(CITY_BNDY_WGS84).to_crs("EPSG:4326")
120
+
121
+ # Inner join drops any points falling outside the boundary polygon limits
122
+ gdf = gpd.sjoin(gdf, stl_boundary[["geometry"]], how="inner", predicate="within").copy()
123
+
124
+ if "index_right" in gdf.columns:
125
+ gdf = gdf.drop(columns=["index_right"])
126
+
127
+ # Revert back to standard DataFrame structure and throw away spatial metadata
128
+ return pd.DataFrame(gdf.drop(columns=["geometry"])).copy()
129
+
130
+ def run_pipeline(month: int, year: int) -> CrimeDataPackage:
131
+ """Runs the entire ingestion, cleaning, and spatial-clipping pipeline
132
+
133
+ for a given month and year. Safely catches missing assets gracefully.
134
+ """
135
+ # Change the import to target the module directly
136
+ from geocrime_stl.etl import extract
137
+
138
+ # Build the target URL
139
+ url, m_int, y_int = extract.construct_url(month_input=month, year_input=year)
140
+
141
+ # Fetch the data stream from SLMPD with built-in safety net
142
+ try:
143
+ raw_bytes = extract.fetch_crime_data_csv(url, keep_raw_csv=False)
144
+ except extract.ExtractionError as e:
145
+ # Catch 404s or network drops gracefully
146
+ logger.warning(
147
+ f"🛑 Unable to retrieve data for {month}/{year}. "
148
+ f"The asset might not be published yet. Details: {e}"
149
+ )
150
+ # Return an empty dataframe package so the user's notebook doesn't crash
151
+ import pandas as pd
152
+ return CrimeDataPackage(df=pd.DataFrame(), month=m_int, year=y_int)
153
+
154
+ # Decode bytes to DataFrame (Only runs if extraction succeeded!)
155
+ raw_df = raw_csv_to_dataframe(raw_bytes)
156
+
157
+ # Clean, type-enforce, and clip to city boundaries
158
+ data_package = clean_and_spatial_clip(raw_df, m_int, y_int)
159
+
160
+ return data_package
@@ -0,0 +1,85 @@
1
+ Metadata-Version: 2.4
2
+ Name: geocrime-stl
3
+ Version: 0.1.0
4
+ Summary: An ETL for fetching/cleaning St. Louis crime data sets.
5
+ Requires-Python: >=3.8
6
+ Description-Content-Type: text/markdown
7
+ License-File: LICENSE
8
+ Requires-Dist: geopandas>=1.1.3
9
+ Requires-Dist: matplotlib>=3.10.9
10
+ Requires-Dist: pandas>=3.0.3
11
+ Requires-Dist: requests>=2.34.2
12
+ Dynamic: license-file
13
+
14
+ # geocrime-stl
15
+
16
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
17
+ [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
18
+
19
+ A Python utility for cleaning and standardizing St. Louis Metropolitan Police Department (SLMPD) crime data for spatial analysis.
20
+
21
+ ## ✨ Features
22
+
23
+ * **End-to-End ETL:** Fetch, clean, and standardize monthly SLMPD crime data files automatically.
24
+ * **Spatial Formats:** Convert cleaned DataFrames seamlessly into GeoJSON, GeoPackage (GPKG), or CSV.
25
+ * **Analysis Ready:** Access built-in methods for minimal data summarization and visualization.
26
+
27
+ ## 🌐 Data Source & Disclaimer
28
+
29
+ This utility programmatically fetches publicly available crime data directly from the **[St. Louis Metropolitan Police Department (SLMPD) Stats Page](https://slmpd.org/stats/)**.
30
+
31
+ ⚠️ **Data Disclaimer & Limitations:**
32
+ * **Availability:** This tool relies entirely on the upstream availability and hosting structure of the SLMPD website. If their servers are down or their URL layout changes, the extraction pipeline may fail.
33
+ * **Data Integrity & Spatial Boundaries:** The data processed is strictly "as-is" from the published monthly files. To ensure high data quality for spatial analysis, this utility applies two strict filtering constraints:
34
+ 1. **Temporal Filtering:** Because the SLMPD frequently updates historical records retroactively, this utility isolates and preserves only the records belonging to the target month.
35
+ 2. **Spatial Clipping:** Records containing invalid, malformed, or erroneous latitude/longitude coordinates that plot completely outside the official St. Louis city boundaries are automatically dropped.
36
+ * **Project Status:** This is an independent, open-source utility. It is not affiliated with, endorsed by, or officially maintained by the SLMPD or the City of St. Louis.
37
+
38
+ ## 🚀 Quick Start
39
+
40
+ ### Install
41
+ ``` bash
42
+ pip install geocrime-stl
43
+ ```
44
+ ### Fetch/Clean/Visualize Data
45
+ ```python
46
+ import geocrime_stl as gc
47
+
48
+ data_pkg = gc.run_pipeline(4, 2026)
49
+
50
+ gc.generate_monthly_metrics(data_pkg)
51
+
52
+ gc.plot_monthly_maps(data_pkg)
53
+ ```
54
+
55
+ ## 🛠️ Local Setup (Alternative)
56
+
57
+ If you want to run the demo notebook locally or explore the source code, you can clone the repository directly:
58
+
59
+ ```bash
60
+ git clone https://github.com/lweber89/geocrime-stl.git
61
+ cd geocrime-stl
62
+ pip install -e .
63
+ ```
64
+
65
+ ## 📓 Demo & Tutorial
66
+ To see a step-by-step walkthrough of the entire ETL process, check out the interactive Jupyter notebook included in this package:
67
+
68
+ 👉 **[View the Demo Notebook](./notebooks/demo.ipynb)**
69
+
70
+ Note: If you have cloned the repository locally, you can open this file using Jupyter Lab or Jupyter Notebook to run the cells yourself.
71
+
72
+ ## 🤝 Contributing & Pull Requests
73
+ Thank you for your interest in the project!
74
+
75
+ ⚠️ Please Note: This is a personal project I developed and maintain in an effort to better understand the implementation of Python in geospatial data engineering / solutions architecture. I am not accepting pull requests or code contributions at this time.
76
+
77
+ If you find a bug or have an idea, feel free to open an Issue to let me know, or feel free to fork the repository and adapt it for your own personal needs!
78
+
79
+ ## 📜 License
80
+ This project is licensed under the MIT License - see the LICENSE file for details.
81
+
82
+ ## 🔗 Project Links
83
+ GitHub Repository: https://github.com/lweber89/geocrime-stl
84
+
85
+ PyPI Home: https://pypi.org/project/geocrime-stl/
@@ -0,0 +1,12 @@
1
+ LICENSE
2
+ README.md
3
+ pyproject.toml
4
+ src/geocrime_stl/__init__.py
5
+ src/geocrime_stl/config.py
6
+ src/geocrime_stl.egg-info/PKG-INFO
7
+ src/geocrime_stl.egg-info/SOURCES.txt
8
+ src/geocrime_stl.egg-info/dependency_links.txt
9
+ src/geocrime_stl.egg-info/requires.txt
10
+ src/geocrime_stl.egg-info/top_level.txt
11
+ src/geocrime_stl/etl/extract.py
12
+ src/geocrime_stl/etl/transform.py
@@ -0,0 +1,4 @@
1
+ geopandas>=1.1.3
2
+ matplotlib>=3.10.9
3
+ pandas>=3.0.3
4
+ requests>=2.34.2
@@ -0,0 +1 @@
1
+ geocrime_stl