sumry 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
sumry-0.1.0/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2025 Sparkgeo
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
sumry-0.1.0/PKG-INFO ADDED
@@ -0,0 +1,101 @@
1
+ Metadata-Version: 2.4
2
+ Name: sumry
3
+ Version: 0.1.0
4
+ Summary: A data summarization tool for analyzing various file formats
5
+ Author: Sparkgeo
6
+ License: MIT
7
+ Project-URL: Homepage, https://github.com/sparkgeo/sumry
8
+ Project-URL: Repository, https://github.com/sparkgeo/sumry
9
+ Project-URL: Issues, https://github.com/sparkgeo/sumry/issues
10
+ Keywords: data,summary,analysis,cli
11
+ Classifier: Development Status :: 3 - Alpha
12
+ Classifier: Intended Audience :: Developers
13
+ Classifier: License :: OSI Approved :: MIT License
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3.12
16
+ Classifier: Topic :: Software Development :: Libraries :: Python Modules
17
+ Requires-Python: >=3.12
18
+ Description-Content-Type: text/markdown
19
+ License-File: LICENSE
20
+ Requires-Dist: duckdb>=1.1.2
21
+ Requires-Dist: geopandas>=1.0.1
22
+ Requires-Dist: openpyxl>=3.1.5
23
+ Requires-Dist: python-calamine>=0.2.3
24
+ Requires-Dist: rich>=13.0.0
25
+ Requires-Dist: typer>=0.12.5
26
+ Dynamic: license-file
27
+
28
+ # sumry
29
+
30
+ A fast and user-friendly CLI tool for summarizing various data file formats with rich terminal output.
31
+
32
+ ## Features
33
+
34
+ - **Multiple Format Support**: CSV, Excel, GeoJSON, and Shapefiles
35
+ - **Rich Terminal Output**: Beautiful, colorful summaries using Rich library
36
+ - **Quick Insights**: Get file statistics, column types, and sample data
37
+ - **Geospatial Support**: Special handling for GeoJSON and Shapefile formats with geometry information
38
+
39
+ ## Installation
40
+
41
+ ```bash
42
+ pip install sumry
43
+ ```
44
+
45
+ ## Usage
46
+
47
+ ### Basic Summary
48
+
49
+ ```bash
50
+ sumry path/to/your/file.csv
51
+ ```
52
+
53
+ ### Show Sample Records
54
+
55
+ ```bash
56
+ # Show 5 sample records (default)
57
+ sumry path/to/your/file.xlsx --show-sample
58
+
59
+ # Show specific number of samples
60
+ sumry path/to/your/file.geojson --sample 10
61
+ ```
62
+
63
+ ### Verbose Output
64
+
65
+ ```bash
66
+ sumry path/to/your/file.shp --verbose
67
+ ```
68
+
69
+ ## Supported File Types
70
+
71
+ - **CSV** (`.csv`)
72
+ - **Excel** (`.xlsx`, `.xls`)
73
+ - **GeoJSON** (`.geojson`, `.json`)
74
+ - **Shapefile** (`.shp`)
75
+
76
+ ## Requirements
77
+
78
+ - Python >= 3.12
79
+ - Dependencies are automatically installed with the package
80
+
81
+ ## Development
82
+
83
+ ```bash
84
+ # Clone the repository
85
+ git clone https://github.com/yourusername/sumry.git
86
+ cd sumry
87
+
88
+ # Install with development dependencies
89
+ pip install -e ".[dev]"
90
+
91
+ # Run tests
92
+ pytest
93
+ ```
94
+
95
+ ## License
96
+
97
+ MIT License - see LICENSE file for details
98
+
99
+ ## Contributing
100
+
101
+ Contributions are welcome! Please feel free to submit a Pull Request.
sumry-0.1.0/README.md ADDED
@@ -0,0 +1,74 @@
1
+ # sumry
2
+
3
+ A fast and user-friendly CLI tool for summarizing various data file formats with rich terminal output.
4
+
5
+ ## Features
6
+
7
+ - **Multiple Format Support**: CSV, Excel, GeoJSON, and Shapefiles
8
+ - **Rich Terminal Output**: Beautiful, colorful summaries using Rich library
9
+ - **Quick Insights**: Get file statistics, column types, and sample data
10
+ - **Geospatial Support**: Special handling for GeoJSON and Shapefile formats with geometry information
11
+
12
+ ## Installation
13
+
14
+ ```bash
15
+ pip install sumry
16
+ ```
17
+
18
+ ## Usage
19
+
20
+ ### Basic Summary
21
+
22
+ ```bash
23
+ sumry path/to/your/file.csv
24
+ ```
25
+
26
+ ### Show Sample Records
27
+
28
+ ```bash
29
+ # Show 5 sample records (default)
30
+ sumry path/to/your/file.xlsx --show-sample
31
+
32
+ # Show specific number of samples
33
+ sumry path/to/your/file.geojson --sample 10
34
+ ```
35
+
36
+ ### Verbose Output
37
+
38
+ ```bash
39
+ sumry path/to/your/file.shp --verbose
40
+ ```
41
+
42
+ ## Supported File Types
43
+
44
+ - **CSV** (`.csv`)
45
+ - **Excel** (`.xlsx`, `.xls`)
46
+ - **GeoJSON** (`.geojson`, `.json`)
47
+ - **Shapefile** (`.shp`)
48
+
49
+ ## Requirements
50
+
51
+ - Python >= 3.12
52
+ - Dependencies are automatically installed with the package
53
+
54
+ ## Development
55
+
56
+ ```bash
57
+ # Clone the repository
58
+ git clone https://github.com/yourusername/sumry.git
59
+ cd sumry
60
+
61
+ # Install with development dependencies
62
+ pip install -e ".[dev]"
63
+
64
+ # Run tests
65
+ pytest
66
+ ```
67
+
68
+ ## License
69
+
70
+ MIT License - see LICENSE file for details
71
+
72
+ ## Contributing
73
+
74
+ Contributions are welcome! Please feel free to submit a Pull Request.
@@ -0,0 +1,48 @@
1
+ [build-system]
2
+ requires = ["setuptools>=61.0", "wheel"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "sumry"
7
+ version = "0.1.0"
8
+ description = "A data summarization tool for analyzing various file formats"
9
+ readme = "README.md"
10
+ requires-python = ">=3.12"
11
+ license = {text = "MIT"}
12
+ authors = [
13
+ {name = "Sparkgeo"}
14
+ ]
15
+ keywords = ["data", "summary", "analysis", "cli"]
16
+ classifiers = [
17
+ "Development Status :: 3 - Alpha",
18
+ "Intended Audience :: Developers",
19
+ "License :: OSI Approved :: MIT License",
20
+ "Programming Language :: Python :: 3",
21
+ "Programming Language :: Python :: 3.12",
22
+ "Topic :: Software Development :: Libraries :: Python Modules",
23
+ ]
24
+ dependencies = [
25
+ "duckdb>=1.1.2",
26
+ "geopandas>=1.0.1",
27
+ "openpyxl>=3.1.5",
28
+ "python-calamine>=0.2.3",
29
+ "rich>=13.0.0",
30
+ "typer>=0.12.5",
31
+ ]
32
+
33
+ [project.urls]
34
+ Homepage = "https://github.com/sparkgeo/sumry"
35
+ Repository = "https://github.com/sparkgeo/sumry"
36
+ Issues = "https://github.com/sparkgeo/sumry/issues"
37
+
38
+ [dependency-groups]
39
+ dev = [
40
+ "pre-commit>=4.0.1",
41
+ "pytest>=8.3.3",
42
+ ]
43
+
44
+ [project.scripts]
45
+ sumry = "sumry.cli:app"
46
+
47
+ [tool.setuptools.packages.find]
48
+ where = ["src"]
sumry-0.1.0/setup.cfg ADDED
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,22 @@
1
+ """
2
+ sumry - A data summarization tool for analyzing and processing various file formats.
3
+ """
4
+
5
+ __version__ = "0.1.0"
6
+
7
+ from sumry.readers import (
8
+ read_csv,
9
+ read_excel,
10
+ read_geojson,
11
+ read_shapefile,
12
+ detect_file_type,
13
+ )
14
+
15
+ __all__ = [
16
+ "read_csv",
17
+ "read_excel",
18
+ "read_geojson",
19
+ "read_shapefile",
20
+ "detect_file_type",
21
+ "__version__",
22
+ ]
@@ -0,0 +1,235 @@
1
+ import typer
2
+ from pathlib import Path
3
+ from typing import Optional
4
+ from rich.console import Console
5
+ from rich.table import Table
6
+ from rich.panel import Panel
7
+ from rich import print as rprint
8
+ import warnings
9
+ import os
10
+
11
+ from sumry.readers import (
12
+ read_csv,
13
+ read_excel,
14
+ read_geojson,
15
+ read_shapefile,
16
+ detect_file_type
17
+ )
18
+
19
+ app = typer.Typer(
20
+ name="sumry",
21
+ help="Summarize various data sources (CSV, Excel, GeoJSON, Shapefiles)",
22
+ add_completion=False
23
+ )
24
+
25
+ console = Console()
26
+
27
+
28
+ @app.command()
29
+ def main(
30
+ file_path: Path = typer.Argument(
31
+ ...,
32
+ help="Path to the file to summarize",
33
+ exists=True
34
+ ),
35
+ verbose: bool = typer.Option(
36
+ False,
37
+ "--verbose",
38
+ "-v",
39
+ help="Show detailed information"
40
+ ),
41
+ sample: Optional[int] = typer.Option(
42
+ None,
43
+ "--sample",
44
+ "-n",
45
+ help="Number of sample records to display (default: 5)"
46
+ ),
47
+ show_sample: bool = typer.Option(
48
+ False,
49
+ "--show-sample",
50
+ "-s",
51
+ help="Show sample records (displays 5 records)"
52
+ )
53
+ ):
54
+ """
55
+ Summarize a data file (CSV, Excel, GeoJSON, or Shapefile).
56
+ """
57
+
58
+ # Suppress warnings and verbose output in non-verbose mode
59
+ if not verbose:
60
+ warnings.filterwarnings('ignore')
61
+ os.environ['PYOGRIO_USE_ARROW'] = '0' # Suppress pyogrio Arrow warnings
62
+
63
+ if not file_path.exists():
64
+ console.print(f"[bold red]Error:[/bold red] File {file_path} does not exist")
65
+ raise typer.Exit(1)
66
+
67
+ file_type = detect_file_type(file_path)
68
+
69
+ if not file_type:
70
+ console.print(f"[bold red]Error:[/bold red] Unsupported file type for {file_path}")
71
+ raise typer.Exit(1)
72
+
73
+ try:
74
+ if file_type == "CSV":
75
+ summary = read_csv(file_path, verbose)
76
+ elif file_type == "Excel":
77
+ summary = read_excel(file_path, verbose)
78
+ elif file_type == "GeoJSON":
79
+ summary = read_geojson(file_path, verbose)
80
+ elif file_type == "Shapefile":
81
+ summary = read_shapefile(file_path, verbose)
82
+ else:
83
+ console.print(f"[bold red]Error:[/bold red] Handler not implemented for {file_type}")
84
+ raise typer.Exit(1)
85
+
86
+ display_summary(summary, file_type, verbose)
87
+
88
+ # Display sample records if requested
89
+ if sample is not None or show_sample:
90
+ sample_count = sample if sample and sample > 0 else 5
91
+ display_sample_records(file_path, file_type, sample_count)
92
+
93
+ except Exception as e:
94
+ console.print(f"[bold red]Error reading file:[/bold red] {str(e)}")
95
+ raise typer.Exit(1)
96
+
97
+
98
+ def display_sample_records(file_path: Path, file_type: str, sample_count: int):
99
+ """Display sample records from the file."""
100
+ import pandas as pd
101
+ import geopandas as gpd
102
+
103
+ console.print(f"\n[bold cyan]Sample Records (first {sample_count}):[/bold cyan]")
104
+
105
+ try:
106
+ # Read the data based on file type
107
+ if file_type == "CSV":
108
+ df = pd.read_csv(file_path, nrows=sample_count)
109
+ elif file_type == "Excel":
110
+ df = pd.read_excel(file_path, nrows=sample_count)
111
+ elif file_type in ["GeoJSON", "Shapefile"]:
112
+ # For spatial files, use geopandas
113
+ if file_type == "GeoJSON":
114
+ gdf = gpd.read_file(file_path, rows=sample_count)
115
+ else: # Shapefile
116
+ gdf = gpd.read_file(file_path, rows=sample_count)
117
+
118
+ # Replace geometry column with placeholder
119
+ df = gdf.copy()
120
+ if 'geometry' in df.columns:
121
+ df['geometry'] = '<geometry>'
122
+ else:
123
+ console.print(f"[yellow]Sample display not supported for {file_type}[/yellow]")
124
+ return
125
+
126
+ # Create a Rich table for displaying the sample
127
+ from rich.box import ROUNDED
128
+ table = Table(show_header=True, box=ROUNDED)
129
+
130
+ # Add columns
131
+ for col in df.columns:
132
+ table.add_column(str(col), style="white", overflow="fold")
133
+
134
+ # Add rows
135
+ for _, row in df.iterrows():
136
+ row_values = []
137
+ for val in row.values:
138
+ if pd.isna(val):
139
+ row_values.append("N/A")
140
+ elif isinstance(val, (int, float)):
141
+ if pd.isna(val):
142
+ row_values.append("N/A")
143
+ elif isinstance(val, float):
144
+ row_values.append(f"{val:.2f}" if val != int(val) else str(int(val)))
145
+ else:
146
+ row_values.append(str(val))
147
+ else:
148
+ # Truncate long strings
149
+ str_val = str(val)
150
+ if len(str_val) > 50:
151
+ str_val = str_val[:47] + "..."
152
+ row_values.append(str_val)
153
+ table.add_row(*row_values)
154
+
155
+ console.print(table)
156
+
157
+ except Exception as e:
158
+ console.print(f"[red]Error displaying sample records: {str(e)}[/red]")
159
+
160
+
161
+ def display_summary(summary: dict, file_type: str, verbose: bool):
162
+ """Display the file summary using Rich formatting."""
163
+
164
+ panel = Panel.fit(
165
+ f"[bold green]{file_type} File Summary[/bold green]",
166
+ border_style="cyan"
167
+ )
168
+ console.print(panel)
169
+
170
+ table = Table(show_header=False, box=None, padding=(0, 2))
171
+ table.add_column("Property", style="cyan", no_wrap=True)
172
+ table.add_column("Value", style="white")
173
+
174
+ for key, value in summary.get("basic_info", {}).items():
175
+ if isinstance(value, (list, dict)):
176
+ value_str = str(value)
177
+ else:
178
+ value_str = str(value)
179
+ table.add_row(key, value_str)
180
+
181
+ console.print(table)
182
+
183
+ if "columns" in summary and summary["columns"]:
184
+ console.print("\n[bold cyan]Columns/Fields:[/bold cyan]")
185
+ col_table = Table(show_header=True, box=None)
186
+ col_table.add_column("Name", style="green")
187
+ col_table.add_column("Type", style="yellow")
188
+
189
+ if verbose and "sample_values" in summary:
190
+ col_table.add_column("Sample Values", style="white")
191
+
192
+ for col_info in summary["columns"]:
193
+ if verbose and "sample_values" in summary:
194
+ sample = summary["sample_values"].get(col_info["name"], [])
195
+ sample_str = ", ".join(str(v) for v in sample[:3])
196
+ col_table.add_row(col_info["name"], col_info["type"], sample_str)
197
+ else:
198
+ col_table.add_row(col_info["name"], col_info["type"])
199
+
200
+ console.print(col_table)
201
+
202
+ if verbose and "statistics" in summary:
203
+ console.print("\n[bold cyan]Statistics:[/bold cyan]")
204
+ stats_table = Table(show_header=True, box=None)
205
+ stats_table.add_column("Column", style="green")
206
+ stats_table.add_column("Min", style="yellow")
207
+ stats_table.add_column("Max", style="yellow")
208
+ stats_table.add_column("Mean", style="yellow")
209
+ stats_table.add_column("Unique", style="yellow")
210
+
211
+ for col_name, stats in summary["statistics"].items():
212
+ stats_table.add_row(
213
+ col_name,
214
+ str(stats.get("min", "N/A")),
215
+ str(stats.get("max", "N/A")),
216
+ str(stats.get("mean", "N/A")),
217
+ str(stats.get("unique", "N/A"))
218
+ )
219
+
220
+ console.print(stats_table)
221
+
222
+ if "geometry_info" in summary:
223
+ console.print("\n[bold cyan]Geometry Information:[/bold cyan]")
224
+ geo_table = Table(show_header=False, box=None, padding=(0, 2))
225
+ geo_table.add_column("Property", style="cyan", no_wrap=True)
226
+ geo_table.add_column("Value", style="white")
227
+
228
+ for key, value in summary["geometry_info"].items():
229
+ geo_table.add_row(key, str(value))
230
+
231
+ console.print(geo_table)
232
+
233
+
234
+ if __name__ == "__main__":
235
+ app()
@@ -0,0 +1 @@
1
+ from .xlsx import XLSX
@@ -0,0 +1,20 @@
1
+
2
+ from typing import Final
3
+
4
+
5
+ class BaseFormat:
6
+ format: Final = None
7
+
8
+ @property
9
+ def name(self):
10
+ raise NotImplementedError("Method not implemented")
11
+
12
+ @property
13
+ def size(self):
14
+ raise NotImplementedError("Method not implemented")
15
+
16
+ def metadata(self, *args, **kwargs):
17
+ return {}
18
+
19
+ def sample_data(self, sample_size: int = 20, *args, **kwargs):
20
+ raise NotImplementedError("Method not implemented")
@@ -0,0 +1,149 @@
1
+ import os
2
+ from typing import Final
3
+ from pathlib import Path
4
+ from rich.console import Console
5
+ from rich.table import Table
6
+
7
+ import duckdb
8
+
9
+ from .base import BaseFormat
10
+ # from ..timing import timing, Timer
11
+
12
+
13
+ class XLSX(BaseFormat):
14
+ verbose_name: Final = "Excel Open XML Spreadsheet"
15
+ format: Final = "xlsx"
16
+ _wb = None
17
+
18
+
19
+ def __init__(self, file, layer=None, *args, **kwargs):
20
+ self.file = Path(file)
21
+ self.layer = layer
22
+
23
+ # def _load(self, sheet_name, nrows=None):
24
+ # if not self._wb:
25
+ # self._wb = CalamineWorkbook.from_path(self.file)
26
+
27
+ # return self._wb
28
+
29
+ def _sheet_names(self):
30
+ return self._wb.sheet_names
31
+
32
+ def _get_fields(self, sheet_name: str):
33
+ df = self._read_xlsx(self.file, num_rows=0, sheet_name=sheet_name)
34
+ return df.dtypes
35
+
36
+ @property
37
+ def name(self):
38
+ return self.file.name
39
+
40
+ @property
41
+ def size(self):
42
+ return os.path.getsize(self.file)
43
+
44
+ # def record_count(self, sheet):
45
+ # workbook = load_workbook(filename=self.file, read_only=True)
46
+ # sheet = workbook[sheet] if sheet else workbook.active
47
+ # return sheet.max_row
48
+
49
+ def info(self, *args, **kwargs):
50
+ # meta = super().metadata(*args, **kwargs)
51
+
52
+ duckdb.install_extension('spatial')
53
+ duckdb.load_extension('spatial')
54
+
55
+ sheet_names_query = f"""
56
+ SELECT * FROM st_read('{self.file}', layer='')
57
+ """
58
+
59
+ # Execute the query
60
+ result = duckdb.sql(sheet_names_query).df()
61
+
62
+ # Display sheet names
63
+ print(result)
64
+
65
+ def sample(self, count: int = 5, *args, **kwargs):
66
+ duckdb.install_extension('spatial')
67
+ duckdb.load_extension('spatial')
68
+
69
+ console = Console()
70
+
71
+ # If no layer specified, list available layers
72
+ if not self.layer:
73
+ sheet_names_query = f"""
74
+ SELECT * FROM st_read('{self.file}', layer='')
75
+ """
76
+ result = duckdb.sql(sheet_names_query).df()
77
+ console.print("\n[bold]Available layers:[/bold]")
78
+ for idx, row in result.iterrows():
79
+ console.print(f" {idx}: {row['layer_name']}")
80
+ console.print("\n[yellow]Please specify a layer using the layer argument[/yellow]")
81
+ return
82
+
83
+ # Read sample data from the specified layer
84
+ try:
85
+ sample_query = f"""
86
+ SELECT * FROM st_read('{self.file}', layer='{self.layer}')
87
+ LIMIT {count}
88
+ """
89
+ df = duckdb.sql(sample_query).df()
90
+
91
+ if df.empty:
92
+ console.print(f"[yellow]No data found in layer '{self.layer}'[/yellow]")
93
+ return
94
+
95
+ # Create a Rich table
96
+ table = Table(title=f"Sample data from '{self.layer}' (showing {min(count, len(df))} of {count} requested records)")
97
+
98
+ # Add columns to the table
99
+ for col in df.columns:
100
+ table.add_column(col, style="cyan", no_wrap=False)
101
+
102
+ # Add rows to the table
103
+ for _, row in df.iterrows():
104
+ row_data = []
105
+ for col in df.columns:
106
+ value = row[col]
107
+ # Check if column name suggests geometry or if value looks like geometry
108
+ if 'geom' in col.lower() or 'geometry' in col.lower():
109
+ row_data.append("<geometry>")
110
+ elif isinstance(value, (bytes, str)) and (
111
+ (isinstance(value, str) and (value.startswith('POLYGON') or value.startswith('POINT') or
112
+ value.startswith('LINE') or value.startswith('MULTI'))) or
113
+ (isinstance(value, bytes))
114
+ ):
115
+ row_data.append("<geometry>")
116
+ else:
117
+ row_data.append(str(value) if value is not None else "")
118
+ table.add_row(*row_data)
119
+
120
+ console.print()
121
+ console.print(table)
122
+ console.print()
123
+
124
+ except Exception as e:
125
+ console.print(f"[red]Error reading data from layer '{self.layer}': {e}[/red]")
126
+
127
+
128
+
129
+
130
+ # if not self.layer:
131
+ # print("Layers available:")
132
+ # for n, sheet_name in enumerate(self._sheet_names()):
133
+ # print(f"{n}: {sheet_name}")
134
+
135
+ # return
136
+
137
+ # print()
138
+ # print(f"Name: {self.file.name}")
139
+ # print(f"Size: {self.size}")
140
+ # print(f"\tSheet Name: {self.layer}")
141
+
142
+ # df = self._load(self.layer)
143
+
144
+ # print(f"\tRecord Count: {len(df.index)}")
145
+
146
+
147
+ # def sample_data(self, sample_size: int = 20, *args, **kwargs):
148
+ # self._load(nrows=sample_size)
149
+ # return self._pd
@@ -0,0 +1,28 @@
1
+ from typing import Optional
2
+ import typer
3
+ from sumry.formats import XLSX
4
+
5
+ app = typer.Typer(help="A CLI tool for XLSX files.")
6
+
7
+
8
+ @app.command()
9
+ def info(
10
+ file: str = typer.Argument(..., help="Source file."),
11
+ layer: str = typer.Argument(None, help="Layer name."),
12
+ ):
13
+ xlsx = XLSX(file, layer=layer)
14
+ xlsx.info()
15
+
16
+
17
+ @app.command()
18
+ def sample(
19
+ file: str = typer.Argument(..., help="Source file."),
20
+ layer: str = typer.Argument(None, help="Layer name."),
21
+ count: int = typer.Option(5, "--count", "-n", help="Number of sample records to display."),
22
+ ):
23
+ xlsx = XLSX(file, layer=layer)
24
+ xlsx.sample(count=count)
25
+
26
+
27
+ if __name__ == "__main__":
28
+ app()
@@ -0,0 +1,256 @@
1
+ from pathlib import Path
2
+ import pandas as pd
3
+ import geopandas as gpd
4
+ import json
5
+ import warnings
6
+ from typing import Dict, Any, Optional, List
7
+
8
+
9
+ def detect_file_type(file_path: Path) -> Optional[str]:
10
+ """Detect the type of file based on its extension."""
11
+ suffix = file_path.suffix.lower()
12
+
13
+ if suffix in ['.csv', '.tsv']:
14
+ return "CSV"
15
+ elif suffix in ['.xlsx', '.xls', '.xlsm']:
16
+ return "Excel"
17
+ elif suffix in ['.geojson', '.json']:
18
+ if suffix == '.json':
19
+ try:
20
+ with open(file_path, 'r') as f:
21
+ data = json.load(f)
22
+ if isinstance(data, dict) and data.get('type') in ['Feature', 'FeatureCollection']:
23
+ return "GeoJSON"
24
+ except:
25
+ pass
26
+ else:
27
+ return "GeoJSON"
28
+ elif suffix in ['.shp']:
29
+ return "Shapefile"
30
+
31
+ return None
32
+
33
+
34
+ def read_csv(file_path: Path, verbose: bool = False) -> Dict[str, Any]:
35
+ """Read and summarize a CSV file."""
36
+ df = pd.read_csv(file_path)
37
+
38
+ summary = {
39
+ "basic_info": {
40
+ "File": file_path.name,
41
+ "Rows": len(df),
42
+ "Columns": len(df.columns),
43
+ "Memory Usage": f"{df.memory_usage(deep=True).sum() / 1024:.2f} KB"
44
+ },
45
+ "columns": []
46
+ }
47
+
48
+ for col in df.columns:
49
+ col_info = {
50
+ "name": col,
51
+ "type": str(df[col].dtype)
52
+ }
53
+ summary["columns"].append(col_info)
54
+
55
+ if verbose:
56
+ summary["sample_values"] = {}
57
+ summary["statistics"] = {}
58
+
59
+ for col in df.columns:
60
+ summary["sample_values"][col] = df[col].dropna().head(5).tolist()
61
+
62
+ if pd.api.types.is_numeric_dtype(df[col]):
63
+ summary["statistics"][col] = {
64
+ "min": float(df[col].min()) if not pd.isna(df[col].min()) else None,
65
+ "max": float(df[col].max()) if not pd.isna(df[col].max()) else None,
66
+ "mean": float(df[col].mean()) if not pd.isna(df[col].mean()) else None,
67
+ "unique": df[col].nunique()
68
+ }
69
+ else:
70
+ summary["statistics"][col] = {
71
+ "unique": df[col].nunique(),
72
+ "most_common": df[col].value_counts().head(1).index[0] if not df[col].empty else None
73
+ }
74
+
75
+ return summary
76
+
77
+
78
+ def read_excel(file_path: Path, verbose: bool = False) -> Dict[str, Any]:
79
+ """Read and summarize an Excel file."""
80
+ xl_file = pd.ExcelFile(file_path)
81
+
82
+ summary = {
83
+ "basic_info": {
84
+ "File": file_path.name,
85
+ "Sheets": len(xl_file.sheet_names),
86
+ "Sheet Names": ", ".join(xl_file.sheet_names)
87
+ }
88
+ }
89
+
90
+ if xl_file.sheet_names:
91
+ df = pd.read_excel(file_path, sheet_name=xl_file.sheet_names[0])
92
+
93
+ summary["basic_info"]["Active Sheet"] = xl_file.sheet_names[0]
94
+ summary["basic_info"]["Rows"] = len(df)
95
+ summary["basic_info"]["Columns"] = len(df.columns)
96
+ summary["basic_info"]["Memory Usage"] = f"{df.memory_usage(deep=True).sum() / 1024:.2f} KB"
97
+
98
+ summary["columns"] = []
99
+ for col in df.columns:
100
+ col_info = {
101
+ "name": col,
102
+ "type": str(df[col].dtype)
103
+ }
104
+ summary["columns"].append(col_info)
105
+
106
+ if verbose:
107
+ summary["sample_values"] = {}
108
+ summary["statistics"] = {}
109
+
110
+ for col in df.columns:
111
+ summary["sample_values"][col] = df[col].dropna().head(5).tolist()
112
+
113
+ if pd.api.types.is_numeric_dtype(df[col]):
114
+ summary["statistics"][col] = {
115
+ "min": float(df[col].min()) if not pd.isna(df[col].min()) else None,
116
+ "max": float(df[col].max()) if not pd.isna(df[col].max()) else None,
117
+ "mean": float(df[col].mean()) if not pd.isna(df[col].mean()) else None,
118
+ "unique": df[col].nunique()
119
+ }
120
+ else:
121
+ summary["statistics"][col] = {
122
+ "unique": df[col].nunique(),
123
+ "most_common": df[col].value_counts().head(1).index[0] if not df[col].empty else None
124
+ }
125
+
126
+ return summary
127
+
128
+
129
+ def read_geojson(file_path: Path, verbose: bool = False) -> Dict[str, Any]:
130
+ """Read and summarize a GeoJSON file."""
131
+ if not verbose:
132
+ warnings.filterwarnings('ignore')
133
+
134
+ gdf = gpd.read_file(file_path)
135
+
136
+ summary = {
137
+ "basic_info": {
138
+ "File": file_path.name,
139
+ "Features": len(gdf),
140
+ "CRS": str(gdf.crs) if gdf.crs else "None",
141
+ "Memory Usage": f"{gdf.memory_usage(deep=True).sum() / 1024:.2f} KB"
142
+ },
143
+ "columns": [],
144
+ "geometry_info": {}
145
+ }
146
+
147
+ for col in gdf.columns:
148
+ if col != 'geometry':
149
+ col_info = {
150
+ "name": col,
151
+ "type": str(gdf[col].dtype)
152
+ }
153
+ summary["columns"].append(col_info)
154
+
155
+ if 'geometry' in gdf.columns:
156
+ geom_types = gdf.geometry.geom_type.value_counts()
157
+ bounds = gdf.total_bounds
158
+
159
+ # Calculate area and length with warning suppression
160
+ with warnings.catch_warnings():
161
+ warnings.simplefilter("ignore")
162
+ area_sum = gdf.geometry.area.sum() if gdf.geometry.geom_type.iloc[0] in ['Polygon', 'MultiPolygon'] else None
163
+ length_sum = gdf.geometry.length.sum() if gdf.geometry.geom_type.iloc[0] in ['LineString', 'MultiLineString'] else None
164
+
165
+ summary["geometry_info"] = {
166
+ "Geometry Types": ", ".join(f"{t}: {c}" for t, c in geom_types.items()),
167
+ "Bounds (minx, miny, maxx, maxy)": f"[{bounds[0]:.6f}, {bounds[1]:.6f}, {bounds[2]:.6f}, {bounds[3]:.6f}]",
168
+ "Total Area": f"{area_sum:.6f}" if area_sum is not None else "N/A",
169
+ "Total Length": f"{length_sum:.6f}" if length_sum is not None else "N/A"
170
+ }
171
+
172
+ if verbose:
173
+ summary["sample_values"] = {}
174
+ summary["statistics"] = {}
175
+
176
+ for col in gdf.columns:
177
+ if col != 'geometry':
178
+ summary["sample_values"][col] = gdf[col].dropna().head(5).tolist()
179
+
180
+ if pd.api.types.is_numeric_dtype(gdf[col]):
181
+ summary["statistics"][col] = {
182
+ "min": float(gdf[col].min()) if not pd.isna(gdf[col].min()) else None,
183
+ "max": float(gdf[col].max()) if not pd.isna(gdf[col].max()) else None,
184
+ "mean": float(gdf[col].mean()) if not pd.isna(gdf[col].mean()) else None,
185
+ "unique": gdf[col].nunique()
186
+ }
187
+ else:
188
+ summary["statistics"][col] = {
189
+ "unique": gdf[col].nunique()
190
+ }
191
+
192
+ return summary
193
+
194
+
195
+ def read_shapefile(file_path: Path, verbose: bool = False) -> Dict[str, Any]:
196
+ """Read and summarize a Shapefile."""
197
+ if not verbose:
198
+ warnings.filterwarnings('ignore')
199
+
200
+ gdf = gpd.read_file(file_path)
201
+
202
+ summary = {
203
+ "basic_info": {
204
+ "File": file_path.name,
205
+ "Features": len(gdf),
206
+ "CRS": str(gdf.crs) if gdf.crs else "None",
207
+ "Memory Usage": f"{gdf.memory_usage(deep=True).sum() / 1024:.2f} KB"
208
+ },
209
+ "columns": [],
210
+ "geometry_info": {}
211
+ }
212
+
213
+ for col in gdf.columns:
214
+ if col != 'geometry':
215
+ col_info = {
216
+ "name": col,
217
+ "type": str(gdf[col].dtype)
218
+ }
219
+ summary["columns"].append(col_info)
220
+
221
+ if 'geometry' in gdf.columns:
222
+ geom_types = gdf.geometry.geom_type.value_counts()
223
+ bounds = gdf.total_bounds
224
+
225
+ # Calculate area and length with warning suppression
226
+ with warnings.catch_warnings():
227
+ warnings.simplefilter("ignore")
228
+ area_sum = gdf.geometry.area.sum() if gdf.geometry.geom_type.iloc[0] in ['Polygon', 'MultiPolygon'] else None
229
+ length_sum = gdf.geometry.length.sum() if gdf.geometry.geom_type.iloc[0] in ['LineString', 'MultiLineString'] else None
230
+
231
+ summary["geometry_info"] = {
232
+ "Geometry Types": ", ".join(f"{t}: {c}" for t, c in geom_types.items()),
233
+ "Bounds (minx, miny, maxx, maxy)": f"[{bounds[0]:.6f}, {bounds[1]:.6f}, {bounds[2]:.6f}, {bounds[3]:.6f}]",
234
+ "Total Area": f"{area_sum:.6f}" if area_sum is not None else "N/A",
235
+ "Total Length": f"{length_sum:.6f}" if length_sum is not None else "N/A"
236
+ }
237
+
238
+ if verbose:
239
+ summary["sample_values"] = {}
240
+ summary["statistics"] = {}
241
+
242
+ for col in gdf.columns:
243
+ if col != 'geometry':
244
+ summary["sample_values"][col] = gdf[col].dropna().head(5).tolist()
245
+
246
+ if pd.api.types.is_numeric_dtype(gdf[col]):
247
+ summary["statistics"][col] = {
248
+ "min": float(gdf[col].min()) if not pd.isna(gdf[col].min()) else None,
249
+ "max": float(gdf[col].max()) if not pd.isna(gdf[col].max()) else None,
250
+ "mean": float(gdf[col].mean()) if not pd.isna(gdf[col].mean()) else None,
251
+ "unique": gdf[col].nunique()
252
+ }
253
+ else:
254
+ summary["statistics"][col] = {
255
+ "unique": gdf[col].nunique()
256
+ }
@@ -0,0 +1,35 @@
1
+ from functools import wraps
2
+ from time import time
3
+
4
+ class Timer:
5
+ def __init__(self, name):
6
+ self.name = name
7
+
8
+ def __enter__(self):
9
+ self.start_time = time()
10
+
11
+ def __exit__(self, type, value, traceback):
12
+ delta = time() - self.start_time
13
+ self.delta = delta
14
+ self.start_time = None
15
+
16
+ msg = f"{self.name}: {delta:.3f}s"
17
+ if traceback:
18
+ msg += " (with error)"
19
+ print(msg)
20
+
21
+
22
+ def timing(name=None):
23
+ def decorator(f):
24
+ @wraps(f)
25
+ def wrap(*args, **kw):
26
+ ts = time()
27
+ result = f(*args, **kw)
28
+ te = time()
29
+ func_name = name if name else f.__name__
30
+ print("func:%r took: %2.4f sec" % (func_name, te - ts))
31
+ return result
32
+
33
+ return wrap
34
+
35
+ return decorator
@@ -0,0 +1,101 @@
1
+ Metadata-Version: 2.4
2
+ Name: sumry
3
+ Version: 0.1.0
4
+ Summary: A data summarization tool for analyzing various file formats
5
+ Author: Sparkgeo
6
+ License: MIT
7
+ Project-URL: Homepage, https://github.com/sparkgeo/sumry
8
+ Project-URL: Repository, https://github.com/sparkgeo/sumry
9
+ Project-URL: Issues, https://github.com/sparkgeo/sumry/issues
10
+ Keywords: data,summary,analysis,cli
11
+ Classifier: Development Status :: 3 - Alpha
12
+ Classifier: Intended Audience :: Developers
13
+ Classifier: License :: OSI Approved :: MIT License
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3.12
16
+ Classifier: Topic :: Software Development :: Libraries :: Python Modules
17
+ Requires-Python: >=3.12
18
+ Description-Content-Type: text/markdown
19
+ License-File: LICENSE
20
+ Requires-Dist: duckdb>=1.1.2
21
+ Requires-Dist: geopandas>=1.0.1
22
+ Requires-Dist: openpyxl>=3.1.5
23
+ Requires-Dist: python-calamine>=0.2.3
24
+ Requires-Dist: rich>=13.0.0
25
+ Requires-Dist: typer>=0.12.5
26
+ Dynamic: license-file
27
+
28
+ # sumry
29
+
30
+ A fast and user-friendly CLI tool for summarizing various data file formats with rich terminal output.
31
+
32
+ ## Features
33
+
34
+ - **Multiple Format Support**: CSV, Excel, GeoJSON, and Shapefiles
35
+ - **Rich Terminal Output**: Beautiful, colorful summaries using Rich library
36
+ - **Quick Insights**: Get file statistics, column types, and sample data
37
+ - **Geospatial Support**: Special handling for GeoJSON and Shapefile formats with geometry information
38
+
39
+ ## Installation
40
+
41
+ ```bash
42
+ pip install sumry
43
+ ```
44
+
45
+ ## Usage
46
+
47
+ ### Basic Summary
48
+
49
+ ```bash
50
+ sumry path/to/your/file.csv
51
+ ```
52
+
53
+ ### Show Sample Records
54
+
55
+ ```bash
56
+ # Show 5 sample records (default)
57
+ sumry path/to/your/file.xlsx --show-sample
58
+
59
+ # Show specific number of samples
60
+ sumry path/to/your/file.geojson --sample 10
61
+ ```
62
+
63
+ ### Verbose Output
64
+
65
+ ```bash
66
+ sumry path/to/your/file.shp --verbose
67
+ ```
68
+
69
+ ## Supported File Types
70
+
71
+ - **CSV** (`.csv`)
72
+ - **Excel** (`.xlsx`, `.xls`)
73
+ - **GeoJSON** (`.geojson`, `.json`)
74
+ - **Shapefile** (`.shp`)
75
+
76
+ ## Requirements
77
+
78
+ - Python >= 3.12
79
+ - Dependencies are automatically installed with the package
80
+
81
+ ## Development
82
+
83
+ ```bash
84
+ # Clone the repository
85
+ git clone https://github.com/yourusername/sumry.git
86
+ cd sumry
87
+
88
+ # Install with development dependencies
89
+ pip install -e ".[dev]"
90
+
91
+ # Run tests
92
+ pytest
93
+ ```
94
+
95
+ ## License
96
+
97
+ MIT License - see LICENSE file for details
98
+
99
+ ## Contributing
100
+
101
+ Contributions are welcome! Please feel free to submit a Pull Request.
@@ -0,0 +1,17 @@
1
+ LICENSE
2
+ README.md
3
+ pyproject.toml
4
+ src/sumry/__init__.py
5
+ src/sumry/cli.py
6
+ src/sumry/main.py
7
+ src/sumry/readers.py
8
+ src/sumry/timing.py
9
+ src/sumry.egg-info/PKG-INFO
10
+ src/sumry.egg-info/SOURCES.txt
11
+ src/sumry.egg-info/dependency_links.txt
12
+ src/sumry.egg-info/entry_points.txt
13
+ src/sumry.egg-info/requires.txt
14
+ src/sumry.egg-info/top_level.txt
15
+ src/sumry/formats/__init__.py
16
+ src/sumry/formats/base.py
17
+ src/sumry/formats/xlsx.py
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ sumry = sumry.cli:app
@@ -0,0 +1,6 @@
1
+ duckdb>=1.1.2
2
+ geopandas>=1.0.1
3
+ openpyxl>=3.1.5
4
+ python-calamine>=0.2.3
5
+ rich>=13.0.0
6
+ typer>=0.12.5
@@ -0,0 +1 @@
1
+ sumry