trackclean 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. trackclean-0.1.0/CHANGELOG.md +36 -0
  2. trackclean-0.1.0/CITATION.cff +21 -0
  3. trackclean-0.1.0/LICENSE +21 -0
  4. trackclean-0.1.0/MANIFEST.in +10 -0
  5. trackclean-0.1.0/PKG-INFO +287 -0
  6. trackclean-0.1.0/README.md +248 -0
  7. trackclean-0.1.0/examples/benchmark_synthetic.py +112 -0
  8. trackclean-0.1.0/pyproject.toml +123 -0
  9. trackclean-0.1.0/setup.cfg +4 -0
  10. trackclean-0.1.0/src/trackclean/__init__.py +128 -0
  11. trackclean-0.1.0/src/trackclean/__main__.py +7 -0
  12. trackclean-0.1.0/src/trackclean/clean.py +399 -0
  13. trackclean-0.1.0/src/trackclean/cli.py +238 -0
  14. trackclean-0.1.0/src/trackclean/filters.py +329 -0
  15. trackclean-0.1.0/src/trackclean/geodesy.py +203 -0
  16. trackclean-0.1.0/src/trackclean/io/__init__.py +83 -0
  17. trackclean-0.1.0/src/trackclean/io/csv.py +224 -0
  18. trackclean-0.1.0/src/trackclean/io/geojson.py +268 -0
  19. trackclean-0.1.0/src/trackclean/io/gpx.py +325 -0
  20. trackclean-0.1.0/src/trackclean/io/tcx.py +240 -0
  21. trackclean-0.1.0/src/trackclean/metrics.py +301 -0
  22. trackclean-0.1.0/src/trackclean/modes.py +83 -0
  23. trackclean-0.1.0/src/trackclean/privacy.py +53 -0
  24. trackclean-0.1.0/src/trackclean/py.typed +0 -0
  25. trackclean-0.1.0/src/trackclean/resample.py +108 -0
  26. trackclean-0.1.0/src/trackclean/segment.py +300 -0
  27. trackclean-0.1.0/src/trackclean/simplify.py +172 -0
  28. trackclean-0.1.0/src/trackclean/synthetic.py +233 -0
  29. trackclean-0.1.0/src/trackclean/timeutil.py +155 -0
  30. trackclean-0.1.0/src/trackclean/track.py +338 -0
  31. trackclean-0.1.0/src/trackclean.egg-info/PKG-INFO +287 -0
  32. trackclean-0.1.0/src/trackclean.egg-info/SOURCES.txt +42 -0
  33. trackclean-0.1.0/src/trackclean.egg-info/dependency_links.txt +1 -0
  34. trackclean-0.1.0/src/trackclean.egg-info/entry_points.txt +2 -0
  35. trackclean-0.1.0/src/trackclean.egg-info/requires.txt +14 -0
  36. trackclean-0.1.0/src/trackclean.egg-info/top_level.txt +1 -0
  37. trackclean-0.1.0/tests/conftest.py +24 -0
  38. trackclean-0.1.0/tests/test_accuracy.py +90 -0
  39. trackclean-0.1.0/tests/test_cli.py +117 -0
  40. trackclean-0.1.0/tests/test_geodesy_time.py +150 -0
  41. trackclean-0.1.0/tests/test_io.py +289 -0
  42. trackclean-0.1.0/tests/test_metrics_segment.py +172 -0
  43. trackclean-0.1.0/tests/test_simplify_resample_privacy.py +115 -0
  44. trackclean-0.1.0/tests/test_track_clean.py +258 -0
@@ -0,0 +1,36 @@
1
+ # Changelog
2
+
3
+ All notable changes to this project will be documented in this file.
4
+ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project
5
+ uses [Semantic Versioning](https://semver.org/).
6
+
7
+ ## [Unreleased]
8
+
9
+ ## [0.1.0] - 2026-11-08
10
+ ### Added
11
+ - `Track`, `TrackSet` and `Waypoint` containers with per-point fields that follow the
12
+ points through every processing step; pandas import/export (`[pandas]` extra).
13
+ - Readers and writers for GPX 1.0/1.1 (tracks, segments, waypoints, routes; Garmin
14
+ TrackPointExtension, `<power>` and other numeric extensions preserved), TCX (activities,
15
+ courses, laps, HR, cadence, speed, watts), GeoJSON (LineString with `coordTimes`, Point
16
+ features) and CSV (column auto-detection, delimiter sniffing, epoch or ISO times).
17
+ - Time parsing for ISO 8601, epoch s/ms/us/ns and naive timestamps in any IANA time zone;
18
+ `shift_naive_time` for local clock times written as UTC.
19
+ - Cleaning: invalid/duplicate/out-of-order fixes, kinematic speed/acceleration gate per
20
+ travel mode (walk, run, bike, car, transit), spike and multipath-burst removal, teleport
21
+ splitting, Hampel filter, constant-velocity Kalman filter with RTS smoothing and
22
+ per-point accuracy, standstill drift collapse, elevation spike removal and
23
+ distance-window smoothing, and the `clean()` pipeline with a per-step report.
24
+ - Resampling to fixed time or distance steps; Douglas-Peucker, Visvalingam-Whyatt and
25
+ TD-TR (synchronized Euclidean distance) simplification.
26
+ - Stay points (Li et al. 2008), trip/stop segmentation, gap splitting and a heuristic
27
+ transport-mode guess.
28
+ - Metrics: haversine and Vincenty distance, elapsed and moving time, pace, speed profile,
29
+ elevation gain/loss with hysteresis, radius of gyration, stops summary, `summarize()`.
30
+ - Privacy: `trim_ends` by radius (default) or path length, with optional randomisation.
31
+ - `trackclean.synthetic`: seeded ground-truth tracks with realistic GPS corruption, plus
32
+ error and stay-point-recall helpers used by the tests and the README benchmark.
33
+ - Command line: `trackclean clean | stats | segment | convert`.
34
+
35
+ [Unreleased]: https://github.com/YusufEminoglu/trackclean_sdk/compare/v0.1.0...HEAD
36
+ [0.1.0]: https://github.com/YusufEminoglu/trackclean_sdk/releases/tag/v0.1.0
@@ -0,0 +1,21 @@
1
+ cff-version: 1.2.0
2
+ message: "If you use this software, please cite it as below."
3
+ authors:
4
+ - family-names: "Eminoğlu"
5
+ given-names: "Yusuf"
6
+ email: "yusufeminoglu@gmail.com"
7
+ orcid: "https://orcid.org/0009-0005-6000-2934"
8
+ title: "trackclean"
9
+ version: "0.1.0"
10
+ date-released: "2026-11-08"
11
+ license: "MIT"
12
+ repository-code: "https://github.com/YusufEminoglu/trackclean_sdk"
13
+ url: "https://pypi.org/project/trackclean/"
14
+ abstract: "Pure-Python cleaning, smoothing, segmentation and metrics for raw GPS trajectories: GPX/TCX/GeoJSON/CSV I/O, kinematic outlier removal, Kalman/RTS smoothing, standstill drift collapse, stay points and realistic elevation gain."
15
+ keywords:
16
+ - gps
17
+ - trajectory
18
+ - gpx
19
+ - mobility
20
+ - kalman-filter
21
+ - stay-points
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Yusuf Eminoğlu
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,10 @@
1
+ include LICENSE
2
+ include README.md
3
+ include CHANGELOG.md
4
+ include CITATION.cff
5
+ include pyproject.toml
6
+ include src/trackclean/py.typed
7
+ recursive-include src/trackclean *.py
8
+ recursive-include tests *.py
9
+ recursive-include examples *.py
10
+ global-exclude __pycache__ *.py[cod]
@@ -0,0 +1,287 @@
1
+ Metadata-Version: 2.4
2
+ Name: trackclean
3
+ Version: 0.1.0
4
+ Summary: Clean, smooth, segment and measure raw GPS trajectories: GPX/TCX/GeoJSON/CSV I/O, outlier and drift removal, Kalman/RTS smoothing, stay points, trips and honest metrics.
5
+ Author-email: Yusuf Eminoğlu <yusufeminoglu@gmail.com>
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/YusufEminoglu/trackclean_sdk
8
+ Project-URL: Repository, https://github.com/YusufEminoglu/trackclean_sdk
9
+ Project-URL: Issues, https://github.com/YusufEminoglu/trackclean_sdk/issues
10
+ Project-URL: Changelog, https://github.com/YusufEminoglu/trackclean_sdk/blob/main/CHANGELOG.md
11
+ Keywords: gps,gpx,tcx,geojson,trajectory,track,mobility,cleaning,outlier-detection,kalman-filter,rts-smoother,hampel-filter,stay-points,trip-segmentation,douglas-peucker,elevation-gain,telematics,fitness,cycling,running
12
+ Classifier: Development Status :: 4 - Beta
13
+ Classifier: Programming Language :: Python :: 3
14
+ Classifier: Programming Language :: Python :: 3.9
15
+ Classifier: Programming Language :: Python :: 3.10
16
+ Classifier: Programming Language :: Python :: 3.11
17
+ Classifier: Programming Language :: Python :: 3.12
18
+ Classifier: Programming Language :: Python :: 3.13
19
+ Classifier: Operating System :: OS Independent
20
+ Classifier: Topic :: Scientific/Engineering :: GIS
21
+ Classifier: Topic :: Scientific/Engineering :: Information Analysis
22
+ Classifier: Intended Audience :: Science/Research
23
+ Classifier: Intended Audience :: Developers
24
+ Classifier: Typing :: Typed
25
+ Requires-Python: >=3.9
26
+ Description-Content-Type: text/markdown
27
+ License-File: LICENSE
28
+ Requires-Dist: numpy>=1.20.0
29
+ Requires-Dist: tzdata; sys_platform == "win32"
30
+ Provides-Extra: pandas
31
+ Requires-Dist: pandas>=1.3.0; extra == "pandas"
32
+ Provides-Extra: dev
33
+ Requires-Dist: pytest>=7.0.0; extra == "dev"
34
+ Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
35
+ Requires-Dist: ruff>=0.1.0; extra == "dev"
36
+ Requires-Dist: mypy>=1.0.0; extra == "dev"
37
+ Requires-Dist: pandas>=1.3.0; extra == "dev"
38
+ Dynamic: license-file
39
+
40
+ # trackclean
41
+
42
+ **Clean, smooth, segment and measure raw GPS trajectories in pure Python + NumPy.**
43
+
44
+ [![CI](https://github.com/YusufEminoglu/trackclean_sdk/actions/workflows/ci.yml/badge.svg)](https://github.com/YusufEminoglu/trackclean_sdk/actions/workflows/ci.yml)
45
+ [![PyPI](https://img.shields.io/pypi/v/trackclean.svg)](https://pypi.org/project/trackclean/)
46
+ [![Python](https://img.shields.io/pypi/pyversions/trackclean.svg)](https://pypi.org/project/trackclean/)
47
+ [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
48
+
49
+ Raw fixes from phones, sports watches, loggers and fleet telematics are messy. They contain
50
+ spikes hundreds of metres off, teleports after tunnels, duplicated and out-of-order
51
+ timestamps, drift that adds kilometres while you stand still, and elevation noise that turns
52
+ a 130 m hike into "13 000 m of ascent". `trackclean` is written for mobility researchers,
53
+ sports and fitness developers and transport analysts. It reads the common formats, removes
54
+ these errors with documented algorithms and reports distance, moving time, gain and stops
55
+ that hold up against ground truth.
56
+
57
+ - **Formats:** GPX 1.0/1.1 (tracks, segments, waypoints, routes; heart rate, cadence,
58
+ power, temperature and other extensions are preserved), TCX, GeoJSON (LineString or
59
+ Points) and CSV with automatic column detection.
60
+ - **Cleaning:** time parsing, sorting and deduplication with time-zone handling;
61
+ speed and acceleration outlier removal tuned per travel mode; teleport detection;
62
+ collapse of standstill drift; Hampel filter; Kalman/RTS smoother that uses per-fix
63
+ accuracy when present; elevation smoothing and gain/loss with a hysteresis threshold;
64
+ resampling to fixed time or distance steps; simplification (Douglas-Peucker,
65
+ Visvalingam-Whyatt, time-aware TD-TR).
66
+ - **Segmentation:** stay points (Li et al. 2008), trip and stop segmentation, splitting at
67
+ signal gaps, and a transport-mode guess that is *clearly labelled as a heuristic*.
68
+ - **Metrics:** haversine and Vincenty distance, moving vs elapsed time, pace, speed
69
+ profile, elevation gain/loss, radius of gyration and a stops summary.
70
+ - **Privacy:** trim the first and last N metres around home or work.
71
+ - **CLI:** `trackclean clean | stats | segment | convert`.
72
+
73
+ The only dependency is NumPy (plus `tzdata` on Windows). pandas is optional.
74
+
75
+ ## Install
76
+
77
+ ```bash
78
+ pip install trackclean # core
79
+ pip install "trackclean[pandas]" # + DataFrame import/export
80
+ ```
81
+
82
+ ## Before and after: synthetic ground truth
83
+
84
+ The table below compares raw and cleaned tracks against their ground truth. Every track is
85
+ synthetic, generated from a seed by `trackclean.synthetic.make_synthetic`, so no real
86
+ person's location data is used. Each track has three moving legs separated by two
87
+ 25-minute stops, sampled at 1 Hz (about 5 600 fixes). The generator then adds:
88
+
89
+ - 4 m white noise and 2.5 m slowly varying (correlated) error per axis;
90
+ - 7 m wandering indoor drift while stopped;
91
+ - 1 % spikes of 100 to 600 m and one 6-fix multipath burst of about 400 m;
92
+ - one 2-minute signal gap;
93
+ - 1 % duplicated fixes and 5 out-of-order fixes;
94
+ - elevation noise of about 4 m, with occasional 40 m spikes.
95
+
96
+ "Raw" means the fixes were only sorted and deduplicated, so that times align with the
97
+ truth. Each value is the mean over 5 seeds
98
+ (`python examples/benchmark_synthetic.py`; the same checks run in `tests/test_accuracy.py`).
99
+
100
+ | metric | walk: raw → clean (truth) | bike: raw → clean (truth) | car: raw → clean (truth) |
101
+ |---|---|---|---|
102
+ | position RMSE, m | 41.6 → **3.3** | 41.6 → **3.0** | 41.6 → **3.0** |
103
+ | median error, m | 7.5 → **1.9** | 7.5 → **1.9** | 7.5 → **1.9** |
104
+ | max error, m | 595 → **30** | 595 → **24** | 595 → **26** |
105
+ | distance, km | 80.98 → **3.68** (3.77) | 85.05 → **14.23** (14.75) | 99.91 → **34.81** (34.87) |
106
+ | elevation gain, m | 13 130 → **127** (133) | 13 113 → **449** (443) | 13 195 → **1 102** (1 066) |
107
+ | moving time, min | 92.8 → **42.5** (44.9) | 91.8 → **42.9** (44.9) | 91.2 → **43.1** (44.9) |
108
+ | stay-point recall | 0.00 → **1.00** | 0.00 → **1.00** | 0.00 → **1.00** |
109
+
110
+ Cleaning takes about 0.1 s for 5 600 fixes. The cleaned distance is slightly short
111
+ because the 2-minute signal gap is not bridged: steps between segments are not counted
112
+ unless you ask for them with `across_segments=True`. On the raw track, the spikes break
113
+ every 25-minute stop into pieces shorter than the 20-minute threshold, so Li's detector
114
+ finds none of them.
115
+
116
+ ## Quick start
117
+
118
+ ```python
119
+ import trackclean as tc
120
+
121
+ track = tc.read_track("ride.gpx") # GPX / TCX / GeoJSON / CSV
122
+ result = tc.clean(track, mode="bike") # walk, run, bike, car, transit
123
+ print(result.report) # what every step removed or changed
124
+ stats = tc.summarize(result.track, mode="bike")
125
+ print(stats.format())
126
+
127
+ seg = tc.segment_trips(result.track) # stops (Li et al. 2008) + trips between them
128
+ for trip in seg.trips:
129
+ print(trip.start_time, len(trip.track), trip.mode.mode) # heuristic mode guess
130
+
131
+ safe = tc.trim_ends(result.track, 300, randomize=0.5) # hide home/work
132
+ tc.write(safe, "ride.clean.gpx") # extensions (hr, cad, power...) kept
133
+ ```
134
+
135
+ ### Command line
136
+
137
+ ```bash
138
+ trackclean clean in.gpx -o out.gpx --mode bike
139
+ trackclean clean phone.csv -o out.geojson --mode walk --resample 5s --simplify 3 --trim-ends 300
140
+ trackclean stats in.gpx --mode run # add --json for machine-readable output
141
+ trackclean stats in.gpx --clean --mode bike # clean first, then measure
142
+ trackclean segment in.csv -o trips.geojson --mode car --stay-distance 200 --stay-time 1200
143
+ trackclean convert in.tcx -o out.gpx
144
+ ```
145
+
146
+ Use `--tz Europe/Istanbul` when the input has timestamps without an offset.
147
+ Use `--local-time TZ` when a logger wrote local clock time but marked it as UTC.
148
+
149
+ ## How it works
150
+
151
+ `tc.clean(track, mode)` runs these steps in order. Each step can be switched off, and
152
+ each is also available as its own function.
153
+
154
+ 1. **Validity, order, duplicates** (`remove_invalid`, `sort_and_dedup`): drops fixes
155
+ with NaN or out-of-range coordinates and exact `0, 0` "null island" fixes. Sorts by
156
+ time and removes repeated timestamps, keeping either the first fix or the one with the
157
+ smallest `accuracy`.
158
+ 2. **Signal gaps** (`split_segments_on_gaps`): starts a new segment when the time step is
159
+ longer than the mode's `max_gap`. Smoothing and distance never bridge a segment break.
160
+ 3. **Speed/acceleration outliers and teleports** (`remove_outliers`, `detect_jumps`):
161
+ a step is impossible when it is longer than
162
+ `min(max_speed, v_ref + max_accel·dt)·dt + k·σ_pair`. Here `v_ref` is a robust local
163
+ median speed and `σ_pair` is the combined noise of the two fixes, which keeps normal
164
+ scatter on slow modes from being flagged. If the track jumps away and jumps back
165
+ within 30 fixes and 5 minutes, and the bridging step is plausible, the fixes in
166
+ between are removed (single spikes and multipath bursts). A jump that never returns
167
+ is a real discontinuity, such as reacquisition after a tunnel, so the track is split
168
+ there instead.
169
+ 4. **Hampel filter** (`hampel_filter`): applied to east and north metres. It replaces
170
+ moderate outliers that pass the kinematic gate with the local median.
171
+ 5. **Kalman filter + Rauch-Tung-Striebel smoother** (`kalman_smooth`, `kalman_rts`):
172
+ uses a constant-velocity model with white-acceleration process noise `q`, set by the
173
+ mode. Measurement noise is taken per fix from `accuracy` (phones report it as a 68 %
174
+ radius, so σ = accuracy / 1.515), otherwise from `hdop × 4 m`, otherwise from the
175
+ mode default. The covariance recursion runs once per segment, which keeps 10 000
176
+ fixes well under a second.
177
+ 6. **Standstill drift** (`collapse_stationary`): finds stays (for example 25 m and
178
+ 2 min for walking) and moves every fix of a stay to its median centre. Adjacent
179
+ pieces of the same stop are merged first. Timestamps are kept, so dwell time
180
+ survives while the phantom "jitter distance" disappears.
181
+ 7. **Elevation** (`smooth_elevation`): a Hampel filter removes spikes, then values are
182
+ averaged over a 100 m window of path length. Smoothing by distance instead of by
183
+ sample treats slow and fast travel alike. It also averages all the fixes taken while
184
+ standing still, which is where most phantom ascent comes from.
185
+
186
+ **Elevation gain/loss** (`elevation_gain_loss(ele, threshold=5)`) reduces the profile to
187
+ its peaks and valleys. A turn only counts once the elevation has moved `threshold`
188
+ metres back from the running extreme. This is the dead-band approach that sports devices
189
+ use to report realistic ascent. `threshold=0` gives the naive sum.
190
+
191
+ **Simplification** (`simplify(track, tol, method)`) offers three methods:
192
+
193
+ - `dp`: Ramer-Douglas-Peucker, with the perpendicular distance in metres.
194
+ - `vw`: Visvalingam-Whyatt, where the tolerance is an effective triangle area in m² and
195
+ areas are kept monotone.
196
+ - `tdtr`: top-down time-ratio (Meratnia and de By 2004). It uses the *synchronized
197
+ Euclidean distance*, so stops and speed changes survive.
198
+
199
+ **Stay points** (`detect_stay_points(track, distance=200, min_duration=1200)`) follow
200
+ Li et al. (2008). From an anchor fix, the scan continues while the following fixes stay
201
+ within `distance`. If they span at least `min_duration`, they form a stay point. A
202
+ signal gap inside a stay (no fixes indoors) is handled naturally. The centre is the
203
+ median by default; use `centroid="mean"` for the paper's mean.
204
+
205
+ **Mode inference** (`infer_mode`) is a **heuristic, not a classifier**. It applies rules
206
+ to the median, 85th and 95th percentile speeds and the 95th percentile of acceleration,
207
+ and returns `stationary`, `walk`, `bike`, `motorized` or `high_speed_rail_or_air`, with
208
+ the features it used. Expect slow cycling and running to be confused, and car, bus and
209
+ tram to be indistinguishable.
210
+
211
+ **Radius of gyration** is the RMS distance of all fixes from their centre of mass
212
+ (González, Hidalgo and Barabási 2008).
213
+
214
+ ### Mode profiles
215
+
216
+ | mode | max speed | max accel | GPS σ | Kalman q | drift radius / time | gap split |
217
+ |---|---|---|---|---|---|---|
218
+ | walk | 4 m/s | 3 m/s² | 6 m | 0.05 | 25 m / 120 s | 120 s |
219
+ | run | 8 m/s | 4 m/s² | 6 m | 0.1 | 25 m / 90 s | 120 s |
220
+ | bike | 22 m/s | 4 m/s² | 6 m | 0.1 | 25 m / 90 s | 120 s |
221
+ | car | 70 m/s | 8 m/s² | 8 m | 0.3 | 30 m / 90 s | 180 s |
222
+ | transit | 90 m/s | 4 m/s² | 10 m | 0.2 | 40 m / 60 s | 300 s |
223
+
224
+ To make your own profile, change a preset: `tc.MODES["bike"].with_(max_speed=30)`.
225
+
226
+ ## Data model
227
+
228
+ A `Track` holds parallel NumPy arrays: `lat`, `lon`, `time` (epoch seconds, UTC; `nan`
229
+ when unknown), `ele`, `segment` (integer ids) and `fields`. `fields` is a dict of extra
230
+ per-point channels such as `hr`, `cad`, `power`, `speed`, `accuracy` and `hdop`. Every
231
+ step carries the fields along, and every writer writes them back. A `TrackSet` is what a
232
+ file holds: tracks, waypoints and metadata. `tc.read()` returns a `TrackSet`, and
233
+ `tc.read_track()` merges it into one `Track`. `Track.to_dataframe()` and
234
+ `Track.from_dataframe()` need the `pandas` extra.
235
+
236
+ | format | read | write | notes |
237
+ |---|---|---|---|
238
+ | GPX 1.1 / 1.0 | ✓ | ✓ | Garmin TrackPointExtension (hr, cad, atemp, ...), `<power>`, hdop/vdop/pdop/sat; other numeric fields go in a `trackclean:` namespace |
239
+ | TCX | ✓ | ✓ | activities and courses, laps as segments, HR, cadence, ActivityExtension speed/watts |
240
+ | GeoJSON | ✓ | ✓ | LineString with `coordTimes`/`coordinateProperties` (togeojson convention) or Point features |
241
+ | CSV/TSV | ✓ | ✓ | delimiter sniffing, decimal comma, ISO or epoch s/ms/µs times, separate date+time columns, `track_id` splitting, km/h speeds |
242
+ | FIT | - | - | not supported; convert with GPSBabel or the vendor app |
243
+
244
+ Time zones: naive timestamps are read in `tz=` (an IANA name or `+hh:mm`; the default is
245
+ UTC). `shift_naive_time(track, tz)` fixes tracks whose local clock time was written as
246
+ UTC. DST transitions are handled.
247
+
248
+ ## Privacy
249
+
250
+ `trim_ends(track, 300)` removes **every** fix within 300 m of the first and last fix,
251
+ including fixes where the track passes home again later. A fixed radius still lets
252
+ someone guess the centre of the circle, so `randomize=0.5` enlarges each radius by a
253
+ random factor between 0 and 50 %. Treat any trajectory as personal data, and keep raw
254
+ tracks out of public repositories. This project's tests use synthetic tracks only.
255
+
256
+ ## Development
257
+
258
+ ```bash
259
+ pip install -e ".[dev]"
260
+ pytest --cov=trackclean
261
+ ruff check . && ruff format --check src tests && mypy src
262
+ python examples/benchmark_synthetic.py
263
+ ```
264
+
265
+ ## References
266
+
267
+ - Li, Q., Zheng, Y., Xie, X., Chen, Y., Liu, W., Ma, W.-Y. (2008). Mining user similarity
268
+ based on location history. *ACM SIGSPATIAL GIS '08*.
269
+ - Vincenty, T. (1975). Direct and inverse solutions of geodesics on the ellipsoid with
270
+ application of nested equations. *Survey Review* 23(176).
271
+ - Rauch, H. E., Tung, F., Striebel, C. T. (1965). Maximum likelihood estimates of linear
272
+ dynamic systems. *AIAA Journal* 3(8).
273
+ - Pearson, R. K. (2002). Outliers in process modeling and identification (Hampel
274
+ identifier). *IEEE Trans. Control Systems Technology* 10(1).
275
+ - Douglas, D., Peucker, T. (1973); Visvalingam, M., Whyatt, J. D. (1993);
276
+ Meratnia, N., de By, R. A. (2004). Spatiotemporal compression techniques for moving
277
+ point objects. *EDBT 2004*.
278
+ - González, M. C., Hidalgo, C. A., Barabási, A.-L. (2008). Understanding individual human
279
+ mobility patterns. *Nature* 453.
280
+
281
+ ## Citation
282
+
283
+ If you use trackclean in research, please cite it using [CITATION.cff](CITATION.cff).
284
+
285
+ ## License
286
+
287
+ MIT © Yusuf Eminoğlu
@@ -0,0 +1,248 @@
1
+ # trackclean
2
+
3
+ **Clean, smooth, segment and measure raw GPS trajectories in pure Python + NumPy.**
4
+
5
+ [![CI](https://github.com/YusufEminoglu/trackclean_sdk/actions/workflows/ci.yml/badge.svg)](https://github.com/YusufEminoglu/trackclean_sdk/actions/workflows/ci.yml)
6
+ [![PyPI](https://img.shields.io/pypi/v/trackclean.svg)](https://pypi.org/project/trackclean/)
7
+ [![Python](https://img.shields.io/pypi/pyversions/trackclean.svg)](https://pypi.org/project/trackclean/)
8
+ [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
9
+
10
+ Raw fixes from phones, sports watches, loggers and fleet telematics are messy. They contain
11
+ spikes hundreds of metres off, teleports after tunnels, duplicated and out-of-order
12
+ timestamps, drift that adds kilometres while you stand still, and elevation noise that turns
13
+ a 130 m hike into "13 000 m of ascent". `trackclean` is written for mobility researchers,
14
+ sports and fitness developers and transport analysts. It reads the common formats, removes
15
+ these errors with documented algorithms and reports distance, moving time, gain and stops
16
+ that hold up against ground truth.
17
+
18
+ - **Formats:** GPX 1.0/1.1 (tracks, segments, waypoints, routes; heart rate, cadence,
19
+ power, temperature and other extensions are preserved), TCX, GeoJSON (LineString or
20
+ Points) and CSV with automatic column detection.
21
+ - **Cleaning:** time parsing, sorting and deduplication with time-zone handling;
22
+ speed and acceleration outlier removal tuned per travel mode; teleport detection;
23
+ collapse of standstill drift; Hampel filter; Kalman/RTS smoother that uses per-fix
24
+ accuracy when present; elevation smoothing and gain/loss with a hysteresis threshold;
25
+ resampling to fixed time or distance steps; simplification (Douglas-Peucker,
26
+ Visvalingam-Whyatt, time-aware TD-TR).
27
+ - **Segmentation:** stay points (Li et al. 2008), trip and stop segmentation, splitting at
28
+ signal gaps, and a transport-mode guess that is *clearly labelled as a heuristic*.
29
+ - **Metrics:** haversine and Vincenty distance, moving vs elapsed time, pace, speed
30
+ profile, elevation gain/loss, radius of gyration and a stops summary.
31
+ - **Privacy:** trim the first and last N metres around home or work.
32
+ - **CLI:** `trackclean clean | stats | segment | convert`.
33
+
34
+ The only dependency is NumPy (plus `tzdata` on Windows). pandas is optional.
35
+
36
+ ## Install
37
+
38
+ ```bash
39
+ pip install trackclean # core
40
+ pip install "trackclean[pandas]" # + DataFrame import/export
41
+ ```
42
+
43
+ ## Before and after: synthetic ground truth
44
+
45
+ The table below compares raw and cleaned tracks against their ground truth. Every track is
46
+ synthetic, generated from a seed by `trackclean.synthetic.make_synthetic`, so no real
47
+ person's location data is used. Each track has three moving legs separated by two
48
+ 25-minute stops, sampled at 1 Hz (about 5 600 fixes). The generator then adds:
49
+
50
+ - 4 m white noise and 2.5 m slowly varying (correlated) error per axis;
51
+ - 7 m wandering indoor drift while stopped;
52
+ - 1 % spikes of 100 to 600 m and one 6-fix multipath burst of about 400 m;
53
+ - one 2-minute signal gap;
54
+ - 1 % duplicated fixes and 5 out-of-order fixes;
55
+ - elevation noise of about 4 m, with occasional 40 m spikes.
56
+
57
+ "Raw" means the fixes were only sorted and deduplicated, so that times align with the
58
+ truth. Each value is the mean over 5 seeds
59
+ (`python examples/benchmark_synthetic.py`; the same checks run in `tests/test_accuracy.py`).
60
+
61
+ | metric | walk: raw → clean (truth) | bike: raw → clean (truth) | car: raw → clean (truth) |
62
+ |---|---|---|---|
63
+ | position RMSE, m | 41.6 → **3.3** | 41.6 → **3.0** | 41.6 → **3.0** |
64
+ | median error, m | 7.5 → **1.9** | 7.5 → **1.9** | 7.5 → **1.9** |
65
+ | max error, m | 595 → **30** | 595 → **24** | 595 → **26** |
66
+ | distance, km | 80.98 → **3.68** (3.77) | 85.05 → **14.23** (14.75) | 99.91 → **34.81** (34.87) |
67
+ | elevation gain, m | 13 130 → **127** (133) | 13 113 → **449** (443) | 13 195 → **1 102** (1 066) |
68
+ | moving time, min | 92.8 → **42.5** (44.9) | 91.8 → **42.9** (44.9) | 91.2 → **43.1** (44.9) |
69
+ | stay-point recall | 0.00 → **1.00** | 0.00 → **1.00** | 0.00 → **1.00** |
70
+
71
+ Cleaning takes about 0.1 s for 5 600 fixes. The cleaned distance is slightly short
72
+ because the 2-minute signal gap is not bridged: steps between segments are not counted
73
+ unless you ask for them with `across_segments=True`. On the raw track, the spikes break
74
+ every 25-minute stop into pieces shorter than the 20-minute threshold, so Li's detector
75
+ finds none of them.
76
+
77
+ ## Quick start
78
+
79
+ ```python
80
+ import trackclean as tc
81
+
82
+ track = tc.read_track("ride.gpx") # GPX / TCX / GeoJSON / CSV
83
+ result = tc.clean(track, mode="bike") # walk, run, bike, car, transit
84
+ print(result.report) # what every step removed or changed
85
+ stats = tc.summarize(result.track, mode="bike")
86
+ print(stats.format())
87
+
88
+ seg = tc.segment_trips(result.track) # stops (Li et al. 2008) + trips between them
89
+ for trip in seg.trips:
90
+ print(trip.start_time, len(trip.track), trip.mode.mode) # heuristic mode guess
91
+
92
+ safe = tc.trim_ends(result.track, 300, randomize=0.5) # hide home/work
93
+ tc.write(safe, "ride.clean.gpx") # extensions (hr, cad, power...) kept
94
+ ```
95
+
96
+ ### Command line
97
+
98
+ ```bash
99
+ trackclean clean in.gpx -o out.gpx --mode bike
100
+ trackclean clean phone.csv -o out.geojson --mode walk --resample 5s --simplify 3 --trim-ends 300
101
+ trackclean stats in.gpx --mode run # add --json for machine-readable output
102
+ trackclean stats in.gpx --clean --mode bike # clean first, then measure
103
+ trackclean segment in.csv -o trips.geojson --mode car --stay-distance 200 --stay-time 1200
104
+ trackclean convert in.tcx -o out.gpx
105
+ ```
106
+
107
+ Use `--tz Europe/Istanbul` when the input has timestamps without an offset.
108
+ Use `--local-time TZ` when a logger wrote local clock time but marked it as UTC.
109
+
110
+ ## How it works
111
+
112
+ `tc.clean(track, mode)` runs these steps in order. Each step can be switched off, and
113
+ each is also available as its own function.
114
+
115
+ 1. **Validity, order, duplicates** (`remove_invalid`, `sort_and_dedup`): drops fixes
116
+ with NaN or out-of-range coordinates and exact `0, 0` "null island" fixes. Sorts by
117
+ time and removes repeated timestamps, keeping either the first fix or the one with the
118
+ smallest `accuracy`.
119
+ 2. **Signal gaps** (`split_segments_on_gaps`): starts a new segment when the time step is
120
+ longer than the mode's `max_gap`. Smoothing and distance never bridge a segment break.
121
+ 3. **Speed/acceleration outliers and teleports** (`remove_outliers`, `detect_jumps`):
122
+ a step is impossible when it is longer than
123
+ `min(max_speed, v_ref + max_accel·dt)·dt + k·σ_pair`. Here `v_ref` is a robust local
124
+ median speed and `σ_pair` is the combined noise of the two fixes, which keeps normal
125
+ scatter on slow modes from being flagged. If the track jumps away and jumps back
126
+ within 30 fixes and 5 minutes, and the bridging step is plausible, the fixes in
127
+ between are removed (single spikes and multipath bursts). A jump that never returns
128
+ is a real discontinuity, such as reacquisition after a tunnel, so the track is split
129
+ there instead.
130
+ 4. **Hampel filter** (`hampel_filter`): applied to east and north metres. It replaces
131
+ moderate outliers that pass the kinematic gate with the local median.
132
+ 5. **Kalman filter + Rauch-Tung-Striebel smoother** (`kalman_smooth`, `kalman_rts`):
133
+ uses a constant-velocity model with white-acceleration process noise `q`, set by the
134
+ mode. Measurement noise is taken per fix from `accuracy` (phones report it as a 68 %
135
+ radius, so σ = accuracy / 1.515), otherwise from `hdop × 4 m`, otherwise from the
136
+ mode default. The covariance recursion runs once per segment, which keeps 10 000
137
+ fixes well under a second.
138
+ 6. **Standstill drift** (`collapse_stationary`): finds stays (for example 25 m and
139
+ 2 min for walking) and moves every fix of a stay to its median centre. Adjacent
140
+ pieces of the same stop are merged first. Timestamps are kept, so dwell time
141
+ survives while the phantom "jitter distance" disappears.
142
+ 7. **Elevation** (`smooth_elevation`): a Hampel filter removes spikes, then values are
143
+ averaged over a 100 m window of path length. Smoothing by distance instead of by
144
+ sample treats slow and fast travel alike. It also averages all the fixes taken while
145
+ standing still, which is where most phantom ascent comes from.
146
+
147
+ **Elevation gain/loss** (`elevation_gain_loss(ele, threshold=5)`) reduces the profile to
148
+ its peaks and valleys. A turn only counts once the elevation has moved `threshold`
149
+ metres back from the running extreme. This is the dead-band approach that sports devices
150
+ use to report realistic ascent. `threshold=0` gives the naive sum.
151
+
152
+ **Simplification** (`simplify(track, tol, method)`) offers three methods:
153
+
154
+ - `dp`: Ramer-Douglas-Peucker, with the perpendicular distance in metres.
155
+ - `vw`: Visvalingam-Whyatt, where the tolerance is an effective triangle area in m² and
156
+ areas are kept monotone.
157
+ - `tdtr`: top-down time-ratio (Meratnia and de By 2004). It uses the *synchronized
158
+ Euclidean distance*, so stops and speed changes survive.
159
+
160
+ **Stay points** (`detect_stay_points(track, distance=200, min_duration=1200)`) follow
161
+ Li et al. (2008). From an anchor fix, the scan continues while the following fixes stay
162
+ within `distance`. If they span at least `min_duration`, they form a stay point. A
163
+ signal gap inside a stay (no fixes indoors) is handled naturally. The centre is the
164
+ median by default; use `centroid="mean"` for the paper's mean.
165
+
166
+ **Mode inference** (`infer_mode`) is a **heuristic, not a classifier**. It applies rules
167
+ to the median, 85th and 95th percentile speeds and the 95th percentile of acceleration,
168
+ and returns `stationary`, `walk`, `bike`, `motorized` or `high_speed_rail_or_air`, with
169
+ the features it used. Expect slow cycling and running to be confused, and car, bus and
170
+ tram to be indistinguishable.
171
+
172
+ **Radius of gyration** is the RMS distance of all fixes from their centre of mass
173
+ (González, Hidalgo and Barabási 2008).
174
+
175
+ ### Mode profiles
176
+
177
+ | mode | max speed | max accel | GPS σ | Kalman q | drift radius / time | gap split |
178
+ |---|---|---|---|---|---|---|
179
+ | walk | 4 m/s | 3 m/s² | 6 m | 0.05 | 25 m / 120 s | 120 s |
180
+ | run | 8 m/s | 4 m/s² | 6 m | 0.1 | 25 m / 90 s | 120 s |
181
+ | bike | 22 m/s | 4 m/s² | 6 m | 0.1 | 25 m / 90 s | 120 s |
182
+ | car | 70 m/s | 8 m/s² | 8 m | 0.3 | 30 m / 90 s | 180 s |
183
+ | transit | 90 m/s | 4 m/s² | 10 m | 0.2 | 40 m / 60 s | 300 s |
184
+
185
+ To make your own profile, change a preset: `tc.MODES["bike"].with_(max_speed=30)`.
186
+
187
+ ## Data model
188
+
189
+ A `Track` holds parallel NumPy arrays: `lat`, `lon`, `time` (epoch seconds, UTC; `nan`
190
+ when unknown), `ele`, `segment` (integer ids) and `fields`. `fields` is a dict of extra
191
+ per-point channels such as `hr`, `cad`, `power`, `speed`, `accuracy` and `hdop`. Every
192
+ step carries the fields along, and every writer writes them back. A `TrackSet` is what a
193
+ file holds: tracks, waypoints and metadata. `tc.read()` returns a `TrackSet`, and
194
+ `tc.read_track()` merges it into one `Track`. `Track.to_dataframe()` and
195
+ `Track.from_dataframe()` need the `pandas` extra.
196
+
197
+ | format | read | write | notes |
198
+ |---|---|---|---|
199
+ | GPX 1.1 / 1.0 | ✓ | ✓ | Garmin TrackPointExtension (hr, cad, atemp, ...), `<power>`, hdop/vdop/pdop/sat; other numeric fields go in a `trackclean:` namespace |
200
+ | TCX | ✓ | ✓ | activities and courses, laps as segments, HR, cadence, ActivityExtension speed/watts |
201
+ | GeoJSON | ✓ | ✓ | LineString with `coordTimes`/`coordinateProperties` (togeojson convention) or Point features |
202
+ | CSV/TSV | ✓ | ✓ | delimiter sniffing, decimal comma, ISO or epoch s/ms/µs times, separate date+time columns, `track_id` splitting, km/h speeds |
203
+ | FIT | - | - | not supported; convert with GPSBabel or the vendor app |
204
+
205
+ Time zones: naive timestamps are read in `tz=` (an IANA name or `+hh:mm`; the default is
206
+ UTC). `shift_naive_time(track, tz)` fixes tracks whose local clock time was written as
207
+ UTC. DST transitions are handled.
208
+
209
+ ## Privacy
210
+
211
+ `trim_ends(track, 300)` removes **every** fix within 300 m of the first and last fix,
212
+ including fixes where the track passes home again later. A fixed radius still lets
213
+ someone guess the centre of the circle, so `randomize=0.5` enlarges each radius by a
214
+ random factor between 0 and 50 %. Treat any trajectory as personal data, and keep raw
215
+ tracks out of public repositories. This project's tests use synthetic tracks only.
216
+
217
+ ## Development
218
+
219
+ ```bash
220
+ pip install -e ".[dev]"
221
+ pytest --cov=trackclean
222
+ ruff check . && ruff format --check src tests && mypy src
223
+ python examples/benchmark_synthetic.py
224
+ ```
225
+
226
+ ## References
227
+
228
+ - Li, Q., Zheng, Y., Xie, X., Chen, Y., Liu, W., Ma, W.-Y. (2008). Mining user similarity
229
+ based on location history. *ACM SIGSPATIAL GIS '08*.
230
+ - Vincenty, T. (1975). Direct and inverse solutions of geodesics on the ellipsoid with
231
+ application of nested equations. *Survey Review* 23(176).
232
+ - Rauch, H. E., Tung, F., Striebel, C. T. (1965). Maximum likelihood estimates of linear
233
+ dynamic systems. *AIAA Journal* 3(8).
234
+ - Pearson, R. K. (2002). Outliers in process modeling and identification (Hampel
235
+ identifier). *IEEE Trans. Control Systems Technology* 10(1).
236
+ - Douglas, D., Peucker, T. (1973); Visvalingam, M., Whyatt, J. D. (1993);
237
+ Meratnia, N., de By, R. A. (2004). Spatiotemporal compression techniques for moving
238
+ point objects. *EDBT 2004*.
239
+ - González, M. C., Hidalgo, C. A., Barabási, A.-L. (2008). Understanding individual human
240
+ mobility patterns. *Nature* 453.
241
+
242
+ ## Citation
243
+
244
+ If you use trackclean in research, please cite it using [CITATION.cff](CITATION.cff).
245
+
246
+ ## License
247
+
248
+ MIT © Yusuf Eminoğlu