las-inspector 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- las_inspector-0.1.0/LICENSE +21 -0
- las_inspector-0.1.0/PKG-INFO +250 -0
- las_inspector-0.1.0/README.md +200 -0
- las_inspector-0.1.0/las_inspector.egg-info/PKG-INFO +250 -0
- las_inspector-0.1.0/las_inspector.egg-info/SOURCES.txt +16 -0
- las_inspector-0.1.0/las_inspector.egg-info/dependency_links.txt +1 -0
- las_inspector-0.1.0/las_inspector.egg-info/entry_points.txt +2 -0
- las_inspector-0.1.0/las_inspector.egg-info/requires.txt +9 -0
- las_inspector-0.1.0/las_inspector.egg-info/top_level.txt +1 -0
- las_inspector-0.1.0/las_inspector.py +1164 -0
- las_inspector-0.1.0/pyproject.toml +46 -0
- las_inspector-0.1.0/setup.cfg +4 -0
- las_inspector-0.1.0/tests/test_cli.py +43 -0
- las_inspector-0.1.0/tests/test_header.py +31 -0
- las_inspector-0.1.0/tests/test_helpers.py +59 -0
- las_inspector-0.1.0/tests/test_reader.py +64 -0
- las_inspector-0.1.0/tests/test_report.py +42 -0
- las_inspector-0.1.0/tests/test_stats.py +82 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Gökser Pirik
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,250 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: las-inspector
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: ASPRS LAS/LAZ quality control inspector
|
|
5
|
+
Author: Gökser Pirik
|
|
6
|
+
License: MIT License
|
|
7
|
+
|
|
8
|
+
Copyright (c) 2026 Gökser Pirik
|
|
9
|
+
|
|
10
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
11
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
12
|
+
in the Software without restriction, including without limitation the rights
|
|
13
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
14
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
15
|
+
furnished to do so, subject to the following conditions:
|
|
16
|
+
|
|
17
|
+
The above copyright notice and this permission notice shall be included in all
|
|
18
|
+
copies or substantial portions of the Software.
|
|
19
|
+
|
|
20
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
21
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
22
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
23
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
24
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
25
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
26
|
+
SOFTWARE.
|
|
27
|
+
|
|
28
|
+
Project-URL: Homepage, https://github.com/gokserpirik/las-inspector
|
|
29
|
+
Project-URL: Repository, https://github.com/gokserpirik/las-inspector
|
|
30
|
+
Keywords: lidar,point-cloud,las,laz,quality-control
|
|
31
|
+
Classifier: Development Status :: 4 - Beta
|
|
32
|
+
Classifier: Intended Audience :: Science/Research
|
|
33
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
34
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
35
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
36
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
37
|
+
Classifier: Topic :: Scientific/Engineering :: GIS
|
|
38
|
+
Requires-Python: >=3.12
|
|
39
|
+
Description-Content-Type: text/markdown
|
|
40
|
+
License-File: LICENSE
|
|
41
|
+
Requires-Dist: numpy<3.0.0,>=1.20.0
|
|
42
|
+
Requires-Dist: matplotlib<4.0.0,>=3.4.0
|
|
43
|
+
Requires-Dist: lazrs<1.0.0,>=0.5.0
|
|
44
|
+
Requires-Dist: laspy<3.0.0,>=2.0.0
|
|
45
|
+
Provides-Extra: dev
|
|
46
|
+
Requires-Dist: pytest>=7.0; extra == "dev"
|
|
47
|
+
Requires-Dist: black; extra == "dev"
|
|
48
|
+
Requires-Dist: ruff; extra == "dev"
|
|
49
|
+
Dynamic: license-file
|
|
50
|
+
|
|
51
|
+
# Automated LiDAR Quality Control Inspector (las-inspector) 🔍
|
|
52
|
+
|
|
53
|
+
A lightweight Python CLI tool designed to ingest standard ASPRS LAS/LAZ point cloud files, run automated quality control checks, and generate interactive, client-ready HTML reports complete with spatial density heatmaps and elevation profiles.
|
|
54
|
+
|
|
55
|
+

|
|
56
|
+
|
|
57
|
+
> 📄 **[View a live example report](https://gokserpirik.github.io/las-inspector/example-report.html)** — or open [`example-report.html`](example-report.html) locally.
|
|
58
|
+
|
|
59
|
+
> ℹ️
|
|
60
|
+
> I started this as an experiment to understand low-level ASPRS binary formats and memory-efficient NumPy arrays. It ended up robust and useful enough to become a general-purpose QC CLI — a portfolio piece showing how AI multiplies a solo engineer's output, contrasting a basic prototype (`main.py`) with a custom binary-parsing tool (`las_inspector.py`).
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## 📊 Dataset Credit & Sourcing
|
|
65
|
+
|
|
66
|
+
The sample dataset (`las2018.laz`) used for development was sourced from the [USGS National Map Lidar Explorer](https://viewer.nationalmap.gov/basic/).
|
|
67
|
+
* **Format:** `.laz` (Compressed LiDAR)
|
|
68
|
+
* **Scale Precision:** Millimeter (0.001m precision)
|
|
69
|
+
|
|
70
|
+
To run the pipeline with your own data:
|
|
71
|
+
1. Download a tile in `.las` or `.laz` format from the USGS portal.
|
|
72
|
+
2. Save it to the working directory.
|
|
73
|
+
3. Pass the file path as an argument to `las_inspector.py`.
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## 🤔 Why Use This?
|
|
78
|
+
|
|
79
|
+
While professional civil engineering projects typically rely on feature-rich CAD suites (like Autodesk Civil 3D) or GIS software (like ArcGIS Pro) to inspect LiDAR files, this lightweight Python CLI workflow offers key practical advantages for specific use cases:
|
|
80
|
+
|
|
81
|
+
1. **Zero Licensing Cost:** It uses free, open-source libraries (`numpy`, `matplotlib`, `lazrs`). Anyone can inspect point clouds without purchasing expensive proprietary CAD/GIS software licenses.
|
|
82
|
+
2. **Low Overhead:** Heavy CAD programs can take minutes to load massive datasets and often freeze on standard laptops. This script starts instantly, parses headers with custom binary decoding, and runs QC checks with memory-efficient NumPy arrays — no GUI, no license server, no project setup.
|
|
83
|
+
3. **Easy Automation & CI/CD Pipelines:** Because it is a simple terminal command-line tool, it can easily be scheduled (e.g., cron jobs), integrated into automated file pipelines, or run as an automated check (pre-commit hooks or GitHub Actions) on newly received survey tiles.
|
|
84
|
+
4. **Transparency & Customizability:** All QC thresholds (like density limits and classification ratios) are explicitly written in code. This provides a clear, reproducible calculation logic rather than a "black-box" proprietary algorithm.
|
|
85
|
+
|
|
86
|
+
---
|
|
87
|
+
|
|
88
|
+
## 📋 Sample Scenario: Quality Control (QC) Delivery Auditing
|
|
89
|
+
|
|
90
|
+
### The Context
|
|
91
|
+
A civil engineering firm is working on a municipal subdivision project. Every week, a drone surveying subcontractor delivers raw LiDAR point cloud tiles (`.laz` format) covering different sections of the project site.
|
|
92
|
+
|
|
93
|
+
### The Problem
|
|
94
|
+
* The firm must verify that the point cloud files meet the contractually required specifications (e.g., density of at least 2.0 pts/m², sufficient ground classification for digital terrain modeling, and no corrupt headers).
|
|
95
|
+
* Manually loading each tile into heavy GIS software to inspect metadata and run statistics takes hours of manual click-work every week.
|
|
96
|
+
* If a contractor submits a tile with incorrect classification or low density, the error might not be discovered until weeks later during detailed modeling, causing costly project delays.
|
|
97
|
+
|
|
98
|
+
### The Solution with this Script
|
|
99
|
+
1. **Data Collection:** The subcontractor uploads the weekly survey tiles to a shared directory.
|
|
100
|
+
2. **Execution:** An automated script triggers `python las_inspector.py [filename].laz --density` for each incoming file.
|
|
101
|
+
3. **Result:** The script automatically parses the binary structure, validates the file against density and classification heuristics, and outputs a clean text summary along with a standalone `qc_report.html` featuring embedded density and elevation plots.
|
|
102
|
+
4. **Value:** The project engineer receives an automated notification with the QC verdict (PASSED/FAILED) and a single-file interactive report. Subcontractor deliveries can be audited and accepted or rejected instantly, ensuring only high-quality data enters the design pipeline.
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## 🛠️ How it Works
|
|
107
|
+
|
|
108
|
+
1. **Low-Level Header Decoding:** Reads the public header block of the LAS/LAZ file to extract scale factors, offsets, coordinate bounds, point records count, and coordinate reference system (CRS) WKT data.
|
|
109
|
+
2. **Binary Point Record Parsing:** Bypasses heavy wrapper libraries by decoding point records directly using Python's `struct` module and NumPy buffers to unpack X, Y, Z coordinates, intensities, return numbers, and classification bytes.
|
|
110
|
+
3. **Dynamic LAZ Decompression:** Transparently detects compressed `.laz` files and reads the LASzip VLR (Variable Length Record), using high-speed Rust-based `lazrs` bindings to decompress point data on the fly.
|
|
111
|
+
4. **Heuristic QC Warning Engine:** Scans point flags and statistics to trigger warnings/errors for low point density, lack of ground-classified points, withheld/synthetic flags, or potential flight-line overlap zones.
|
|
112
|
+
5. **Asset-Embedded Report Compilation:** Renders an interactive, responsive HTML5 report styled with a dark theme. It base64-encodes the generated density and elevation plots directly into the HTML file, creating a single, fully portable deliverable.
|
|
113
|
+
|
|
114
|
+
### 🧠 AI Co-Pilot Case Study: Baseline vs. Enhanced
|
|
115
|
+
|
|
116
|
+
This repository serves as a showcase of using AI as a force multiplier for software engineering, contrasting a basic prototyping script with a custom binary-parsing CLI tool.
|
|
117
|
+
|
|
118
|
+
| Feature | Baseline Script (`main.py`) | Enhanced CLI Tool (`las_inspector.py`) |
|
|
119
|
+
| :--- | :--- | :--- |
|
|
120
|
+
| **Dependencies** | Requires `laspy` high-level wrapper | Native binary decoding (uses Python `struct` & `numpy` buffers) |
|
|
121
|
+
| **Parsing Logic** | Wrapped abstraction | Low-level binary reading from file offsets (IEEE 754 doubles, packed bytes) |
|
|
122
|
+
| **Visualizations** | None | 2D Density Heatmap & Elevation profiles |
|
|
123
|
+
| **Outputs** | Console log + basic HTML snippet | Standalone, semantic HTML5 report styled with dark mode and modern typography |
|
|
124
|
+
| **Warnings** | Hardcoded checks | Configurable warning heuristics (low density, missing ground data) |
|
|
125
|
+
|
|
126
|
+
#### 📚 Technical Learnings & Code Evolution
|
|
127
|
+
|
|
128
|
+
Developing the enhanced CLI tool involved deep diving into the ASPRS LAS specification details:
|
|
129
|
+
|
|
130
|
+
1. **Bitwise Unpacking:**
|
|
131
|
+
LiDAR point formats compress spatial attributes using bitfields. The parser extracts packed properties using bitwise masks.
|
|
132
|
+
* For **LAS 1.2** (Formats 0-5), the classification and flags share a single byte at offset 15:
|
|
133
|
+
```python
|
|
134
|
+
flag_synthetic = bool(byte & 0x20)
|
|
135
|
+
flag_keypoint = bool(byte & 0x40)
|
|
136
|
+
flag_withheld = bool(byte & 0x80)
|
|
137
|
+
classification = byte & 0x1F # 5-bit code
|
|
138
|
+
```
|
|
139
|
+
* For **LAS 1.4** (Formats 6-10), attributes are expanded. Return Number and Number of Returns are packed together in byte 14, and Classification Flags are separated from the Classification byte:
|
|
140
|
+
```python
|
|
141
|
+
# Byte 14: Return number (bits 0-3) & Number of returns (bits 4-7)
|
|
142
|
+
return_number = byte_14 & 0x0F
|
|
143
|
+
number_of_returns = (byte_14 >> 4) & 0x0F
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
2. **Byte Alignment Debugging:**
|
|
147
|
+
During development, we resolved a critical coordinate misalignment where LAS 1.4 files (PDRF 6-10) were returning false-positive classification warnings. By analyzing byte sizes, we corrected the offsets (e.g. referencing byte 16 for the expanded 8-bit Classification code, and reading the 8-byte GPS Time double at offset 22 instead of 20).
|
|
148
|
+
|
|
149
|
+
3. **Memory Optimization Thoughts:**
|
|
150
|
+
While the current version loads the binary file into a byte buffer for speed, dealing with multi-gigabyte point clouds requires mapping the file via `mmap` or streaming point packets in chunks.
|
|
151
|
+
|
|
152
|
+
---
|
|
153
|
+
|
|
154
|
+
## ⏱️ Benchmarks
|
|
155
|
+
|
|
156
|
+
Honest numbers first: `laspy` (native C++/Rust parsing) beats this tool's pure-Python parser at raw point parsing. The win here is workflow overhead — no GUI, no license, instant startup, scriptable — not CPU throughput. Reproduce with `python scripts/benchmark.py <file> --repeat 3`:
|
|
157
|
+
|
|
158
|
+
| Workload (`las2018.laz`, 2.2M pts, LAS 1.4) | Mean | Best of 3 |
|
|
159
|
+
| :--- | :--- | :--- |
|
|
160
|
+
| `las_inspector.read_las` + `compute_stats` | 4.25 s | 4.13 s |
|
|
161
|
+
| `laspy.read` + NumPy stats | 0.24 s | 0.23 s |
|
|
162
|
+
|
|
163
|
+
Takeaway: if you need maximum parse throughput, use `laspy` (or PDAL). If you need a one-command QC verdict plus a portable client report with zero setup, this tool earns its keep. A header-only fast path (skip point records when only bounds/density estimates are needed) is planned future work.
|
|
164
|
+
|
|
165
|
+
---
|
|
166
|
+
|
|
167
|
+
## ⚙️ Configuration & Parameters
|
|
168
|
+
|
|
169
|
+
To customize the CLI run behavior and thresholds, adjust the command-line options or modify the warning heuristics within `las_inspector.py`:
|
|
170
|
+
|
|
171
|
+
### CLI Flags & Arguments
|
|
172
|
+
* **`input` (positional):** Path to the raw `.las` or `.laz` file.
|
|
173
|
+
* **`--output`, `-o`:** Name of the output HTML report (defaults to `qc_report.html`).
|
|
174
|
+
* **`--density`:** Generates a dual-plot PNG showing the 2D Point Density heatmap and terrain-colored elevation scatter.
|
|
175
|
+
* **`--max-points`:** Limits the number of points processed (useful for rapid testing on massive files).
|
|
176
|
+
* **Overlap detection:** runs automatically on every file — possible flight-line overlap zones are reported as info-level warnings.
|
|
177
|
+
|
|
178
|
+
### Modifying Heuristic Thresholds
|
|
179
|
+
Open `las_inspector.py` and modify the following conditional checks in `compute_stats()` to fit your project standards:
|
|
180
|
+
|
|
181
|
+
1. **Minimum Ground Classification:**
|
|
182
|
+
```python
|
|
183
|
+
if 2 not in classifications or classifications[2] < len(pts) * 0.01:
|
|
184
|
+
# Warns if Ground points (Class 2) are less than 1% of total points
|
|
185
|
+
```
|
|
186
|
+
2. **Point Density Warning:**
|
|
187
|
+
```python
|
|
188
|
+
if hdr.point_density < 10:
|
|
189
|
+
# Triggers warning if density drops below 10 points per square unit
|
|
190
|
+
```
|
|
191
|
+
3. **Withheld Point Tolerance:**
|
|
192
|
+
```python
|
|
193
|
+
withheld_pct = (withheld / len(pts)) * 100
|
|
194
|
+
if withheld_pct > 5:
|
|
195
|
+
# Triggers warning if more than 5% of points are flagged as withheld
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
---
|
|
199
|
+
|
|
200
|
+
## 🚀 Quick Start
|
|
201
|
+
|
|
202
|
+
### Prerequisites
|
|
203
|
+
* Python 3.12 or newer
|
|
204
|
+
* [uv](https://github.com/astral-sh/uv) (recommended) or `pip`/`pipx`
|
|
205
|
+
|
|
206
|
+
### Install & Run
|
|
207
|
+
1. Clone this repository.
|
|
208
|
+
2. Try it immediately on the bundled sample tile (`data/sample.laz`, ~100k points, 1.2 MB):
|
|
209
|
+
|
|
210
|
+
```bash
|
|
211
|
+
# Using uv:
|
|
212
|
+
uv sync
|
|
213
|
+
source .venv/bin/activate
|
|
214
|
+
las-inspector data/sample.laz --density --output qc_report.html
|
|
215
|
+
|
|
216
|
+
# Or install as a standalone tool with pipx (no clone needed):
|
|
217
|
+
pipx install git+https://github.com/gokserpirik/las-inspector.git
|
|
218
|
+
las-inspector your-tile.laz --density # point it at your own .las/.laz file
|
|
219
|
+
|
|
220
|
+
# Or using standard pip:
|
|
221
|
+
pip install -r requirements.txt
|
|
222
|
+
python las_inspector.py data/sample.laz --density --output qc_report.html
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
To build the sample yourself from a large tile: `python scripts/make_sample.py las2018.laz --output data/sample.laz --stride 22`.
|
|
226
|
+
|
|
227
|
+
### Output Files & Visualizations
|
|
228
|
+
|
|
229
|
+
The script generates three key deliverables (names derive from your input/output paths):
|
|
230
|
+
|
|
231
|
+
1. **`qc_report.html`**: A standalone, semantic HTML5 report with responsive dark-mode styling, Google Fonts typography, interactive cards, and a detailed summary of point count, CRS, and returns. If `--density` is enabled, the visualization is embedded directly into this file.
|
|
232
|
+
2. **`<input>_qc_summary.txt`** (e.g. `sample_qc_summary.txt`): A clean, lightweight text file summary saved to the working directory for quick inspection or piping to other tools.
|
|
233
|
+
3. **`<output>_density.png`** (only with `--density`): A dual-panel figure mapping point density per bin next to a terrain-colored elevation scatter plot.
|
|
234
|
+
|
|
235
|
+

|
|
236
|
+
|
|
237
|
+
---
|
|
238
|
+
|
|
239
|
+
## 🧪 Running Tests
|
|
240
|
+
|
|
241
|
+
```bash
|
|
242
|
+
uv sync
|
|
243
|
+
.venv/bin/python -m pytest tests/ -v
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
40 tests covering header math, bitfield unpacking, QC stats, binary LAS parsing, reports, and the CLI.
|
|
247
|
+
|
|
248
|
+
## 📄 License
|
|
249
|
+
|
|
250
|
+
MIT — see [LICENSE](LICENSE).
|
|
@@ -0,0 +1,200 @@
|
|
|
1
|
+
# Automated LiDAR Quality Control Inspector (las-inspector) 🔍
|
|
2
|
+
|
|
3
|
+
A lightweight Python CLI tool designed to ingest standard ASPRS LAS/LAZ point cloud files, run automated quality control checks, and generate interactive, client-ready HTML reports complete with spatial density heatmaps and elevation profiles.
|
|
4
|
+
|
|
5
|
+

|
|
6
|
+
|
|
7
|
+
> 📄 **[View a live example report](https://gokserpirik.github.io/las-inspector/example-report.html)** — or open [`example-report.html`](example-report.html) locally.
|
|
8
|
+
|
|
9
|
+
> ℹ️
|
|
10
|
+
> I started this as an experiment to understand low-level ASPRS binary formats and memory-efficient NumPy arrays. It ended up robust and useful enough to become a general-purpose QC CLI — a portfolio piece showing how AI multiplies a solo engineer's output, contrasting a basic prototype (`main.py`) with a custom binary-parsing tool (`las_inspector.py`).
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## 📊 Dataset Credit & Sourcing
|
|
15
|
+
|
|
16
|
+
The sample dataset (`las2018.laz`) used for development was sourced from the [USGS National Map Lidar Explorer](https://viewer.nationalmap.gov/basic/).
|
|
17
|
+
* **Format:** `.laz` (Compressed LiDAR)
|
|
18
|
+
* **Scale Precision:** Millimeter (0.001m precision)
|
|
19
|
+
|
|
20
|
+
To run the pipeline with your own data:
|
|
21
|
+
1. Download a tile in `.las` or `.laz` format from the USGS portal.
|
|
22
|
+
2. Save it to the working directory.
|
|
23
|
+
3. Pass the file path as an argument to `las_inspector.py`.
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## 🤔 Why Use This?
|
|
28
|
+
|
|
29
|
+
While professional civil engineering projects typically rely on feature-rich CAD suites (like Autodesk Civil 3D) or GIS software (like ArcGIS Pro) to inspect LiDAR files, this lightweight Python CLI workflow offers key practical advantages for specific use cases:
|
|
30
|
+
|
|
31
|
+
1. **Zero Licensing Cost:** It uses free, open-source libraries (`numpy`, `matplotlib`, `lazrs`). Anyone can inspect point clouds without purchasing expensive proprietary CAD/GIS software licenses.
|
|
32
|
+
2. **Low Overhead:** Heavy CAD programs can take minutes to load massive datasets and often freeze on standard laptops. This script starts instantly, parses headers with custom binary decoding, and runs QC checks with memory-efficient NumPy arrays — no GUI, no license server, no project setup.
|
|
33
|
+
3. **Easy Automation & CI/CD Pipelines:** Because it is a simple terminal command-line tool, it can easily be scheduled (e.g., cron jobs), integrated into automated file pipelines, or run as an automated check (pre-commit hooks or GitHub Actions) on newly received survey tiles.
|
|
34
|
+
4. **Transparency & Customizability:** All QC thresholds (like density limits and classification ratios) are explicitly written in code. This provides a clear, reproducible calculation logic rather than a "black-box" proprietary algorithm.
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## 📋 Sample Scenario: Quality Control (QC) Delivery Auditing
|
|
39
|
+
|
|
40
|
+
### The Context
|
|
41
|
+
A civil engineering firm is working on a municipal subdivision project. Every week, a drone surveying subcontractor delivers raw LiDAR point cloud tiles (`.laz` format) covering different sections of the project site.
|
|
42
|
+
|
|
43
|
+
### The Problem
|
|
44
|
+
* The firm must verify that the point cloud files meet the contractually required specifications (e.g., density of at least 2.0 pts/m², sufficient ground classification for digital terrain modeling, and no corrupt headers).
|
|
45
|
+
* Manually loading each tile into heavy GIS software to inspect metadata and run statistics takes hours of manual click-work every week.
|
|
46
|
+
* If a contractor submits a tile with incorrect classification or low density, the error might not be discovered until weeks later during detailed modeling, causing costly project delays.
|
|
47
|
+
|
|
48
|
+
### The Solution with this Script
|
|
49
|
+
1. **Data Collection:** The subcontractor uploads the weekly survey tiles to a shared directory.
|
|
50
|
+
2. **Execution:** An automated script triggers `python las_inspector.py [filename].laz --density` for each incoming file.
|
|
51
|
+
3. **Result:** The script automatically parses the binary structure, validates the file against density and classification heuristics, and outputs a clean text summary along with a standalone `qc_report.html` featuring embedded density and elevation plots.
|
|
52
|
+
4. **Value:** The project engineer receives an automated notification with the QC verdict (PASSED/FAILED) and a single-file interactive report. Subcontractor deliveries can be audited and accepted or rejected instantly, ensuring only high-quality data enters the design pipeline.
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## 🛠️ How it Works
|
|
57
|
+
|
|
58
|
+
1. **Low-Level Header Decoding:** Reads the public header block of the LAS/LAZ file to extract scale factors, offsets, coordinate bounds, point records count, and coordinate reference system (CRS) WKT data.
|
|
59
|
+
2. **Binary Point Record Parsing:** Bypasses heavy wrapper libraries by decoding point records directly using Python's `struct` module and NumPy buffers to unpack X, Y, Z coordinates, intensities, return numbers, and classification bytes.
|
|
60
|
+
3. **Dynamic LAZ Decompression:** Transparently detects compressed `.laz` files and reads the LASzip VLR (Variable Length Record), using high-speed Rust-based `lazrs` bindings to decompress point data on the fly.
|
|
61
|
+
4. **Heuristic QC Warning Engine:** Scans point flags and statistics to trigger warnings/errors for low point density, lack of ground-classified points, withheld/synthetic flags, or potential flight-line overlap zones.
|
|
62
|
+
5. **Asset-Embedded Report Compilation:** Renders an interactive, responsive HTML5 report styled with a dark theme. It base64-encodes the generated density and elevation plots directly into the HTML file, creating a single, fully portable deliverable.
|
|
63
|
+
|
|
64
|
+
### 🧠 AI Co-Pilot Case Study: Baseline vs. Enhanced
|
|
65
|
+
|
|
66
|
+
This repository serves as a showcase of using AI as a force multiplier for software engineering, contrasting a basic prototyping script with a custom binary-parsing CLI tool.
|
|
67
|
+
|
|
68
|
+
| Feature | Baseline Script (`main.py`) | Enhanced CLI Tool (`las_inspector.py`) |
|
|
69
|
+
| :--- | :--- | :--- |
|
|
70
|
+
| **Dependencies** | Requires `laspy` high-level wrapper | Native binary decoding (uses Python `struct` & `numpy` buffers) |
|
|
71
|
+
| **Parsing Logic** | Wrapped abstraction | Low-level binary reading from file offsets (IEEE 754 doubles, packed bytes) |
|
|
72
|
+
| **Visualizations** | None | 2D Density Heatmap & Elevation profiles |
|
|
73
|
+
| **Outputs** | Console log + basic HTML snippet | Standalone, semantic HTML5 report styled with dark mode and modern typography |
|
|
74
|
+
| **Warnings** | Hardcoded checks | Configurable warning heuristics (low density, missing ground data) |
|
|
75
|
+
|
|
76
|
+
#### 📚 Technical Learnings & Code Evolution
|
|
77
|
+
|
|
78
|
+
Developing the enhanced CLI tool involved deep diving into the ASPRS LAS specification details:
|
|
79
|
+
|
|
80
|
+
1. **Bitwise Unpacking:**
|
|
81
|
+
LiDAR point formats compress spatial attributes using bitfields. The parser extracts packed properties using bitwise masks.
|
|
82
|
+
* For **LAS 1.2** (Formats 0-5), the classification and flags share a single byte at offset 15:
|
|
83
|
+
```python
|
|
84
|
+
flag_synthetic = bool(byte & 0x20)
|
|
85
|
+
flag_keypoint = bool(byte & 0x40)
|
|
86
|
+
flag_withheld = bool(byte & 0x80)
|
|
87
|
+
classification = byte & 0x1F # 5-bit code
|
|
88
|
+
```
|
|
89
|
+
* For **LAS 1.4** (Formats 6-10), attributes are expanded. Return Number and Number of Returns are packed together in byte 14, and Classification Flags are separated from the Classification byte:
|
|
90
|
+
```python
|
|
91
|
+
# Byte 14: Return number (bits 0-3) & Number of returns (bits 4-7)
|
|
92
|
+
return_number = byte_14 & 0x0F
|
|
93
|
+
number_of_returns = (byte_14 >> 4) & 0x0F
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
2. **Byte Alignment Debugging:**
|
|
97
|
+
During development, we resolved a critical coordinate misalignment where LAS 1.4 files (PDRF 6-10) were returning false-positive classification warnings. By analyzing byte sizes, we corrected the offsets (e.g. referencing byte 16 for the expanded 8-bit Classification code, and reading the 8-byte GPS Time double at offset 22 instead of 20).
|
|
98
|
+
|
|
99
|
+
3. **Memory Optimization Thoughts:**
|
|
100
|
+
While the current version loads the binary file into a byte buffer for speed, dealing with multi-gigabyte point clouds requires mapping the file via `mmap` or streaming point packets in chunks.
|
|
101
|
+
|
|
102
|
+
---
|
|
103
|
+
|
|
104
|
+
## ⏱️ Benchmarks
|
|
105
|
+
|
|
106
|
+
Honest numbers first: `laspy` (native C++/Rust parsing) beats this tool's pure-Python parser at raw point parsing. The win here is workflow overhead — no GUI, no license, instant startup, scriptable — not CPU throughput. Reproduce with `python scripts/benchmark.py <file> --repeat 3`:
|
|
107
|
+
|
|
108
|
+
| Workload (`las2018.laz`, 2.2M pts, LAS 1.4) | Mean | Best of 3 |
|
|
109
|
+
| :--- | :--- | :--- |
|
|
110
|
+
| `las_inspector.read_las` + `compute_stats` | 4.25 s | 4.13 s |
|
|
111
|
+
| `laspy.read` + NumPy stats | 0.24 s | 0.23 s |
|
|
112
|
+
|
|
113
|
+
Takeaway: if you need maximum parse throughput, use `laspy` (or PDAL). If you need a one-command QC verdict plus a portable client report with zero setup, this tool earns its keep. A header-only fast path (skip point records when only bounds/density estimates are needed) is planned future work.
|
|
114
|
+
|
|
115
|
+
---
|
|
116
|
+
|
|
117
|
+
## ⚙️ Configuration & Parameters
|
|
118
|
+
|
|
119
|
+
To customize the CLI run behavior and thresholds, adjust the command-line options or modify the warning heuristics within `las_inspector.py`:
|
|
120
|
+
|
|
121
|
+
### CLI Flags & Arguments
|
|
122
|
+
* **`input` (positional):** Path to the raw `.las` or `.laz` file.
|
|
123
|
+
* **`--output`, `-o`:** Name of the output HTML report (defaults to `qc_report.html`).
|
|
124
|
+
* **`--density`:** Generates a dual-plot PNG showing the 2D Point Density heatmap and terrain-colored elevation scatter.
|
|
125
|
+
* **`--max-points`:** Limits the number of points processed (useful for rapid testing on massive files).
|
|
126
|
+
* **Overlap detection:** runs automatically on every file — possible flight-line overlap zones are reported as info-level warnings.
|
|
127
|
+
|
|
128
|
+
### Modifying Heuristic Thresholds
|
|
129
|
+
Open `las_inspector.py` and modify the following conditional checks in `compute_stats()` to fit your project standards:
|
|
130
|
+
|
|
131
|
+
1. **Minimum Ground Classification:**
|
|
132
|
+
```python
|
|
133
|
+
if 2 not in classifications or classifications[2] < len(pts) * 0.01:
|
|
134
|
+
# Warns if Ground points (Class 2) are less than 1% of total points
|
|
135
|
+
```
|
|
136
|
+
2. **Point Density Warning:**
|
|
137
|
+
```python
|
|
138
|
+
if hdr.point_density < 10:
|
|
139
|
+
# Triggers warning if density drops below 10 points per square unit
|
|
140
|
+
```
|
|
141
|
+
3. **Withheld Point Tolerance:**
|
|
142
|
+
```python
|
|
143
|
+
withheld_pct = (withheld / len(pts)) * 100
|
|
144
|
+
if withheld_pct > 5:
|
|
145
|
+
# Triggers warning if more than 5% of points are flagged as withheld
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
---
|
|
149
|
+
|
|
150
|
+
## 🚀 Quick Start
|
|
151
|
+
|
|
152
|
+
### Prerequisites
|
|
153
|
+
* Python 3.12 or newer
|
|
154
|
+
* [uv](https://github.com/astral-sh/uv) (recommended) or `pip`/`pipx`
|
|
155
|
+
|
|
156
|
+
### Install & Run
|
|
157
|
+
1. Clone this repository.
|
|
158
|
+
2. Try it immediately on the bundled sample tile (`data/sample.laz`, ~100k points, 1.2 MB):
|
|
159
|
+
|
|
160
|
+
```bash
|
|
161
|
+
# Using uv:
|
|
162
|
+
uv sync
|
|
163
|
+
source .venv/bin/activate
|
|
164
|
+
las-inspector data/sample.laz --density --output qc_report.html
|
|
165
|
+
|
|
166
|
+
# Or install as a standalone tool with pipx (no clone needed):
|
|
167
|
+
pipx install git+https://github.com/gokserpirik/las-inspector.git
|
|
168
|
+
las-inspector your-tile.laz --density # point it at your own .las/.laz file
|
|
169
|
+
|
|
170
|
+
# Or using standard pip:
|
|
171
|
+
pip install -r requirements.txt
|
|
172
|
+
python las_inspector.py data/sample.laz --density --output qc_report.html
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
To build the sample yourself from a large tile: `python scripts/make_sample.py las2018.laz --output data/sample.laz --stride 22`.
|
|
176
|
+
|
|
177
|
+
### Output Files & Visualizations
|
|
178
|
+
|
|
179
|
+
The script generates three key deliverables (names derive from your input/output paths):
|
|
180
|
+
|
|
181
|
+
1. **`qc_report.html`**: A standalone, semantic HTML5 report with responsive dark-mode styling, Google Fonts typography, interactive cards, and a detailed summary of point count, CRS, and returns. If `--density` is enabled, the visualization is embedded directly into this file.
|
|
182
|
+
2. **`<input>_qc_summary.txt`** (e.g. `sample_qc_summary.txt`): A clean, lightweight text file summary saved to the working directory for quick inspection or piping to other tools.
|
|
183
|
+
3. **`<output>_density.png`** (only with `--density`): A dual-panel figure mapping point density per bin next to a terrain-colored elevation scatter plot.
|
|
184
|
+
|
|
185
|
+

|
|
186
|
+
|
|
187
|
+
---
|
|
188
|
+
|
|
189
|
+
## 🧪 Running Tests
|
|
190
|
+
|
|
191
|
+
```bash
|
|
192
|
+
uv sync
|
|
193
|
+
.venv/bin/python -m pytest tests/ -v
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
40 tests covering header math, bitfield unpacking, QC stats, binary LAS parsing, reports, and the CLI.
|
|
197
|
+
|
|
198
|
+
## 📄 License
|
|
199
|
+
|
|
200
|
+
MIT — see [LICENSE](LICENSE).
|