fastgpx 0.1.0__tar.gz

fastgpx-0.1.0/.clang-format

@@ -0,0 +1,36 @@
+---
+AccessModifierOffset: -2
+# AlignConsecutiveAssignments: Consecutive
+AllowShortBlocksOnASingleLine: false
+# AllowShortCaseLabelsOnASingleLine: false
+AllowShortEnumsOnASingleLine: false
+AllowShortFunctionsOnASingleLine: Inline # InlineOnly # Empty
+AllowShortIfStatementsOnASingleLine: Never
+AllowShortLambdasOnASingleLine: All # Inline
+BasedOnStyle: Google
+BreakBeforeBraces: Custom
+BraceWrapping:
+  AfterCaseLabel: true
+  AfterClass: true
+  AfterControlStatement: Always
+  AfterEnum: true
+  AfterExternBlock: true
+  AfterFunction: true
+  AfterNamespace: false
+  AfterStruct: true
+  AfterUnion: true
+  BeforeCatch: true
+  BeforeElse: true
+  BeforeLambdaBody: false
+  BeforeWhile: true
+  SplitEmptyFunction: false
+ColumnLimit: 100
+DerivePointerAlignment: false
+IncludeBlocks: Preserve
+IndentCaseLabels: false
+IndentExternBlock: NoIndent
+IndentPPDirectives: BeforeHash
+PointerAlignment: Left
+ReferenceAlignment: Left
+SpaceAfterTemplateKeyword: false
+SpacesBeforeTrailingComments: 1

fastgpx-0.1.0/.gitignore

@@ -0,0 +1,24 @@
+__pycache__
+.venv
+.env
+env39/
+
+/dist/
+
+/build/
+/output/
+
+/.vs/
+/out/
+
+.mypy_cache/
+.pytest_cache/
+
+/*.png
+
+/profiling/*.prof
+
+/coverage/
+LastCoverageResults.log
+
+/docs/build/

fastgpx-0.1.0/.python-version

@@ -0,0 +1 @@
+3.11

fastgpx-0.1.0/CMakeLists.txt

@@ -0,0 +1,17 @@
+cmake_minimum_required(VERSION 3.30.2)
+
+project(fastgpx)
+
+# > The CTest module defines a BUILD_TESTING cache variable which defaults to true.
+# > It is used to decide whether the module calls enable_testing() or not, so the
+# > project does not have to make its own explicit call to enable_testing(). The
+# > project can also use this cache variable to perform certain processing only
+# > if testing is enabled. If the project has many tests that take a long time
+# > to build, this can be a useful way to avoid adding them to the build when
+# > they are not needed.
+#
+# The BUILD_TESTING is set to false in pyproject.toml, so the tests will not build
+# when building the Python package.
+include(CTest)
+
+add_subdirectory(src/cpp)

fastgpx-0.1.0/CMakeSettings.json

@@ -0,0 +1,29 @@
+{
+  "configurations": [
+    {
+      "name": "x64-Debug",
+      "generator": "Ninja",
+      "configurationType": "Debug",
+      "inheritEnvironments": [ "msvc_x64_x64" ],
+      "buildRoot": "${projectDir}\\out\\build\\${name}",
+      "installRoot": "${projectDir}\\out\\install\\${name}",
+      "cmakeCommandArgs": "-DPYTHON_EXECUTABLE=.venv/Scripts/python.exe -DCMAKE_PREFIX_PATH=${projectDir}/.venv/Lib/site-packages/pybind11/share/cmake/pybind11",
+      "buildCommandArgs": "",
+      "ctestCommandArgs": "",
+      "cmakeExecutable": "C:/Program Files/CMake/bin/cmake.exe"
+    },
+    {
+      "name": "x64-Release",
+      "generator": "Ninja",
+      "configurationType": "RelWithDebInfo",
+      "buildRoot": "${projectDir}\\out\\build\\${name}",
+      "installRoot": "${projectDir}\\out\\install\\${name}",
+      "cmakeExecutable": "C:/Program Files/CMake/bin/cmake.exe",
+      "cmakeCommandArgs": "-DPYTHON_EXECUTABLE=.venv/Scripts/python.exe -DCMAKE_PREFIX_PATH=${projectDir}/.venv/Lib/site-packages/pybind11/share/cmake/pybind11",
+      "buildCommandArgs": "",
+      "ctestCommandArgs": "",
+      "inheritEnvironments": [ "msvc_x64_x64" ],
+      "variables": []
+    }
+  ]
+}

fastgpx-0.1.0/Development.md

@@ -0,0 +1,152 @@
+# Development
+
+## Python
+
+### Import Order
+
+The typical organization of Python imports follows a structured convention to improve readability and maintainability of the code. This organization often adheres to the PEP 8 style guide, which is widely adopted in the Python community. Here is the recommended order and format:
+
+Python imports are generally organized into three main sections, each separated by a blank line:
+
+1. Standard Library Imports: These are modules that are part of Python's standard library, such as `os`, `sys`, `datetime`, etc.
+
+2. Third-Party Imports: These are external libraries that are not part of the standard library, such as `numpy`, `requests`, etc.
+
+3. Local Application or Project-Specific Imports: These are your own modules that are part of the project.
+
+```py
+# Standard library imports
+import os
+import sys
+from datetime import datetime
+
+# Third-party imports
+import requests
+import numpy as np
+
+# Local application imports
+from my_project.module import my_function
+from . import another_module
+```
+
+
+### Python C Extension
+
+Once set up, you can build the C++ extension with a simple `pip install .` or `pip install --editable .` for development builds.
+
+```sh
+pip install --editable .
+```
+
+```sh
+pybind11-stubgen fastgpx -o src
+
+pybind11-stubgen fastgpx --enum-class-locations "Precision:fastgpx.polyline" -o src
+```
+
+### Python Profiling
+
+https://learn.microsoft.com/en-us/visualstudio/python/profiling-python-code-in-visual-studio?view=vs-2022
+
+```sh
+snakeviz profiling/fastgpx_polyline_encode.prof
+```
+
+```sh
+snakeviz profiling/polyline_encode.prof
+```
+
+### pyproject.toml
+
+> Installing Dependencies with pyproject.toml
+>
+> You no longer need to use `pip install -r requirements.txt`. Instead, you can simply install dependencies directly using:
+>
+> ```sh
+> pip install .
+> ```
+
+> For Development Dependencies:
+>
+> To install development dependencies (like `pytest` and `pybind11-stubgen`), you can use the `--extra` option (assuming you defined them under dev):
+>
+> ```sh
+> pip install .[dev]
+> ```
+>
+> This will install both the main dependencies and the development dependencies defined in the dev section of `pyproject.toml`.
+
+### Locally test build wheel
+
+```sh
+pip install --upgrade build twine wheel
+```
+
+```sh
+python -m build
+twine check dist/*
+```
+
+## C++
+
+### Include order
+
+The organization of C++ includes follows certain conventions that are similar in spirit to Python import conventions. Well-structured includes can improve readability, reduce compile times, and minimize dependencies. Here are the typical guidelines and best practices for organizing C++ includes:
+
+C++ includes are generally organized in a specific order, often grouped and separated by blank lines. The typical order is as follows:
+
+1. Header File for the Current Implementation File (if applicable)
+2. Standard Library Headers (e.g., `<iostream>`, `<vector>`)
+3. Third-Party Library Headers (e.g., `boost`, or other external dependencies)
+4. Project-Specific Headers (e.g., your own modules or classes)
+
+This organization helps to ensure that your file includes what it needs directly, and minimizes the chance of accidentally relying on transitive includes from other files.
+
+```cpp
+// Current implementation file's corresponding header
+#include "my_class.h"
+
+// Standard library headers
+#include <iostream>
+#include <vector>
+#include <string>
+
+// Third-party library headers
+#include <boost/algorithm/string.hpp>
+
+// Project-specific headers
+#include "utils.h"
+#include "data_processing.h"
+```
+
+### Reformat all C++ sources
+
+```sh
+cd src\cpp
+for /R %f in (*.cpp *.hpp) do "C:\Program Files\LLVM\bin\clang-format.exe" -i "%f"
+```
+
+### Coverage (C++ OpenCppCoverage)
+
+```sh
+coverage.bat ~[real_world]
+```
+
+### Catch2 Tests
+
+If the output prints UTF-8 characters the terminal needs to be set to UTF-8 mode on Windows:
+
+```sh
+chcp 65001
+```
+#### Running Tests
+
+```sh
+build\src\cpp\RelWithDebInfo\fastgpx_test.exe
+```
+
+#### Running Benchmarks
+
+```sh
+build\src\cpp\RelWithDebInfo\fastgpx_test.exe [!benchmark]
+```

fastgpx-0.1.0/GPX.md

@@ -0,0 +1,50 @@
+
+# GPX
+
+## GPX for Developers
+
+https://www.topografix.com/gpx_for_developers.asp
+
+> The standard in the gpx files for the time zone format is not completely
+> clear. For instance some applications generate time formats in the track
+> points of the form
+>
+> <time>2008-07-18T16:07:50.000+02:00</time>
+>
+> this is slightly substandard because the standard as described in
+> http://www.topografix.com/gpx.asp says that
+>
+> "Date and time in are in Univeral Coordinated Time (UTC), not local time!
+> Conforms to ISO 8601 specification for date/time representation."
+
+https://web.archive.org/web/20130725164436/http://tech.groups.yahoo.com/group/gpsxml/message/1090?l=1
+
+## GPS Exchange Format (GPX): A Comprehensive Guide
+
+https://mapscaping.com/gps-exchange-format-gpx/
+
+## Assumed typical <time> representation in .GPX files
+
+To simplify logic and maximize performance, maybe one can assume a more narrow
+scope of ISO 8601.
+
+By using the length of the string one can make assumption to the variation of
+the format and directly extract the numeric components of the string. Probably
+worth validating the separator characters as a minimal validation safety check
+to eliminate pure junk input.
+
+Variations observed:
+* Only Extended Format.
+* With or without milliseconds. (3 fractional decimals)
+* Zulu hours are the norm, but timezone offsets might occur.
+* Some mention of missing timezone notation all together.
+  While ISO 8601 describe this as local time, one can probably assume this is
+  an omission by the author and it really should be Zulu hours.
+
+| Example                       | Length |                                     |
+|-------------------------------|--------|-------------------------------------|
+| 2008-07-18T16:07:50.000+02:00 |     29 | Not per GPX definition.             |
+| 2008-07-18T16:07:50.000Z      |     24 |                                     |
+| 2008-07-18T16:07:50+02:00     |     25 | Not per GPX definition.             |
+| 2008-07-18T16:07:50Z          |     20 |                                     |
+| 2008-07-18T16:07:50           |     19 | Assume Zulu time?                   |

fastgpx-0.1.0/LICENSE.md

@@ -0,0 +1,22 @@
+
+The MIT License (MIT)
+
+Copyright (c) 2024-2025 Thomas Thomassen
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

fastgpx-0.1.0/PKG-INFO

@@ -0,0 +1,148 @@
+Metadata-Version: 2.4
+Name: fastgpx
+Version: 0.1.0
+Summary: An experimental Python library for parsing GPX files fast.
+Keywords: gpx,parser,fast
+Author: Thomas Thomassen
+License-Expression: MIT
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: File Formats
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Programming Language :: Python :: 3.11
+Project-URL: Documentation, https://thomthom.github.io/fastgpx/
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+
+# fastgpx
+
+An experimental Python library for parsing GPX files fast.
+
+```py
+# Get the total length of the tracks in a GPX file:
+import fastgpx
+
+gpx = fastgpx.parse("example.gpx")
+print(f'{gpx.length_2d()} m')
+```
+
+```py
+# Iterate over GPX file:
+import fastgpx
+
+gpx = fastgpx.parse("example.gpx")
+for track in gpx.tracks:
+    print(f'Track: {track.name}')
+    print(f'Distance: {track.length_2d()} m')
+    if not track.time_bounds.is_empty():
+      print(f'Time: {track.time_bounds().start_time} - {track.time_bounds().end_time}')
+    for segment in track.segments:
+        print(f'Segment: {segment.name}')
+        for point in segment.points:
+            print(f'Point: {point.latitude}, {point.longitude}')
+```
+
+[Documentation](https://thomthom.github.io/fastgpx/)
+
+## GPX/XML Performance (Background)
+
+`gpxpy` appear to be the most popular GPX library for Python.
+
+`gpxpy` docs says that it uses `lxml` is available because it is faster than "`minidom`" (`etree`).
+When benchmarking that seemed not to be the case. It appear that the stdlib XML library has gotten
+much better since `gpxpy` was created.
+
+Reference: Open ticket on making `etree` default:
+https://github.com/tkrajina/gpxpy/issues/248
+
+## Benchmarks
+
+Test machine:
+
+* AMD Ryzen 7 5800 8-Core, 3.80 GHz
+* 32 GB memory
+* m2 SSD storage
+
+### gpxpy benchmarks
+
+Comparing getting the distance of a GPX file using `gpxpy` vs manually extracting
+the data using `xml_etree`, computing distance between points using `gpxpy`
+distance functions.
+
+#### gpxpy without `lxml`
+
+```
+Running benchmark with 3 iterations...
+gpxpy 5463041.784135511 meters
+gpxpy 5463041.784135511 meters
+gpxpy 5463041.784135511 meters
+gpxpy: 11.497863 seconds (Average: 3.832621 seconds)
+```
+
+#### gpxpy with `lxml`
+
+```
+Running benchmark with 3 iterations...
+gpxpy 5463041.784135511 meters
+gpxpy 5463041.784135511 meters
+gpxpy 5463041.784135511 meters
+gpxpy: 37.803625 seconds (Average: 12.601208 seconds)
+```
+
+#### xml_etree data extraction
+
+```
+Running benchmark with 3 iterations...
+xml_etree 5463043.740615641 meters
+xml_etree 5463043.740615641 meters
+xml_etree 5463043.740615641 meters
+xml_etree: 2.333200 seconds (Average: 0.777733 seconds)
+```
+
+Even with `gpxpy` using `etree` to parse the XML it is paster to parse it
+directly with `etree` and use `gpxpy.geo` distance functions to compute the
+distance of a GPX file. Unclear what the extra overhead is, possibly the cost
+of extraction additional data. (Some minor difference in how the total distance
+is computed in this example. Using different options for computing the distance.)
+
+### C++ benchmarks
+
+Since XML parsing itself appear to have a significant impact on performance some
+popular C++ XML libraries was tested:
+
+#### tinyxml2
+```
+Total Length: 5456930.710560566
+Elapsed time: 0.4980144 seconds
+```
+
+#### pugixml
+```
+Total Length: 5456930.710560566
+Elapsed time: 0.1890089 seconds
+```
+
+### C++ vs Python implementations
+
+
+```
+Running 5 benchmarks with 3 iterations...
+
+Running gpxpy ...
+gpxpy: 50.182288 seconds (Average: 16.727429 seconds)
+
+Running xml_etree ...
+xml_etree: 8.269050 seconds (Average: 2.756350 seconds)
+
+Running lxml ...
+lxml: 8.479702 seconds (Average: 2.826567 seconds)
+
+Running tinyxml (C++) ...
+tinyxml (C++): 2.699880 seconds (Average: 0.899960 seconds)
+
+Running pugixml (C++) ...
+pugixml (C++): 0.381095 seconds (Average: 0.127032 seconds)
+```
+
+For computing the length of a GPX file, `pugixml` in a Python C extension was ~140
+times faster than using `gpxpy`.

fastgpx-0.1.0/README.md

@@ -0,0 +1,132 @@
+# fastgpx
+
+An experimental Python library for parsing GPX files fast.
+
+```py
+# Get the total length of the tracks in a GPX file:
+import fastgpx
+
+gpx = fastgpx.parse("example.gpx")
+print(f'{gpx.length_2d()} m')
+```
+
+```py
+# Iterate over GPX file:
+import fastgpx
+
+gpx = fastgpx.parse("example.gpx")
+for track in gpx.tracks:
+    print(f'Track: {track.name}')
+    print(f'Distance: {track.length_2d()} m')
+    if not track.time_bounds.is_empty():
+      print(f'Time: {track.time_bounds().start_time} - {track.time_bounds().end_time}')
+    for segment in track.segments:
+        print(f'Segment: {segment.name}')
+        for point in segment.points:
+            print(f'Point: {point.latitude}, {point.longitude}')
+```
+
+[Documentation](https://thomthom.github.io/fastgpx/)
+
+## GPX/XML Performance (Background)
+
+`gpxpy` appear to be the most popular GPX library for Python.
+
+`gpxpy` docs says that it uses `lxml` is available because it is faster than "`minidom`" (`etree`).
+When benchmarking that seemed not to be the case. It appear that the stdlib XML library has gotten
+much better since `gpxpy` was created.
+
+Reference: Open ticket on making `etree` default:
+https://github.com/tkrajina/gpxpy/issues/248
+
+## Benchmarks
+
+Test machine:
+
+* AMD Ryzen 7 5800 8-Core, 3.80 GHz
+* 32 GB memory
+* m2 SSD storage
+
+### gpxpy benchmarks
+
+Comparing getting the distance of a GPX file using `gpxpy` vs manually extracting
+the data using `xml_etree`, computing distance between points using `gpxpy`
+distance functions.
+
+#### gpxpy without `lxml`
+
+```
+Running benchmark with 3 iterations...
+gpxpy 5463041.784135511 meters
+gpxpy 5463041.784135511 meters
+gpxpy 5463041.784135511 meters
+gpxpy: 11.497863 seconds (Average: 3.832621 seconds)
+```
+
+#### gpxpy with `lxml`
+
+```
+Running benchmark with 3 iterations...
+gpxpy 5463041.784135511 meters
+gpxpy 5463041.784135511 meters
+gpxpy 5463041.784135511 meters
+gpxpy: 37.803625 seconds (Average: 12.601208 seconds)
+```
+
+#### xml_etree data extraction
+
+```
+Running benchmark with 3 iterations...
+xml_etree 5463043.740615641 meters
+xml_etree 5463043.740615641 meters
+xml_etree 5463043.740615641 meters
+xml_etree: 2.333200 seconds (Average: 0.777733 seconds)
+```
+
+Even with `gpxpy` using `etree` to parse the XML it is paster to parse it
+directly with `etree` and use `gpxpy.geo` distance functions to compute the
+distance of a GPX file. Unclear what the extra overhead is, possibly the cost
+of extraction additional data. (Some minor difference in how the total distance
+is computed in this example. Using different options for computing the distance.)
+
+### C++ benchmarks
+
+Since XML parsing itself appear to have a significant impact on performance some
+popular C++ XML libraries was tested:
+
+#### tinyxml2
+```
+Total Length: 5456930.710560566
+Elapsed time: 0.4980144 seconds
+```
+
+#### pugixml
+```
+Total Length: 5456930.710560566
+Elapsed time: 0.1890089 seconds
+```
+
+### C++ vs Python implementations
+
+
+```
+Running 5 benchmarks with 3 iterations...
+
+Running gpxpy ...
+gpxpy: 50.182288 seconds (Average: 16.727429 seconds)
+
+Running xml_etree ...
+xml_etree: 8.269050 seconds (Average: 2.756350 seconds)
+
+Running lxml ...
+lxml: 8.479702 seconds (Average: 2.826567 seconds)
+
+Running tinyxml (C++) ...
+tinyxml (C++): 2.699880 seconds (Average: 0.899960 seconds)
+
+Running pugixml (C++) ...
+pugixml (C++): 0.381095 seconds (Average: 0.127032 seconds)
+```
+
+For computing the length of a GPX file, `pugixml` in a Python C extension was ~140
+times faster than using `gpxpy`.