ntpstats 2.5.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. ntpstats-2.5.0/LICENSE +21 -0
  2. ntpstats-2.5.0/NOTICE +15 -0
  3. ntpstats-2.5.0/PKG-INFO +332 -0
  4. ntpstats-2.5.0/README.md +295 -0
  5. ntpstats-2.5.0/pyproject.toml +65 -0
  6. ntpstats-2.5.0/setup.cfg +4 -0
  7. ntpstats-2.5.0/src/ntpstats/__init__.py +11 -0
  8. ntpstats-2.5.0/src/ntpstats/__main__.py +7 -0
  9. ntpstats-2.5.0/src/ntpstats/analysis.py +164 -0
  10. ntpstats-2.5.0/src/ntpstats/bench.py +157 -0
  11. ntpstats-2.5.0/src/ntpstats/cli.py +598 -0
  12. ntpstats-2.5.0/src/ntpstats/edf.py +118 -0
  13. ntpstats-2.5.0/src/ntpstats/estimators.py +363 -0
  14. ntpstats-2.5.0/src/ntpstats/filters.py +183 -0
  15. ntpstats-2.5.0/src/ntpstats/masks.py +112 -0
  16. ntpstats-2.5.0/src/ntpstats/monitor.py +142 -0
  17. ntpstats-2.5.0/src/ntpstats/network.py +126 -0
  18. ntpstats-2.5.0/src/ntpstats/nts.py +351 -0
  19. ntpstats-2.5.0/src/ntpstats/parsers.py +680 -0
  20. ntpstats-2.5.0/src/ntpstats/pcap.py +219 -0
  21. ntpstats-2.5.0/src/ntpstats/plotting.py +121 -0
  22. ntpstats-2.5.0/src/ntpstats/report.py +218 -0
  23. ntpstats-2.5.0/src/ntpstats/series.py +170 -0
  24. ntpstats-2.5.0/src/ntpstats/simulate.py +354 -0
  25. ntpstats-2.5.0/src/ntpstats/sntp.py +324 -0
  26. ntpstats-2.5.0/src/ntpstats/sources.py +195 -0
  27. ntpstats-2.5.0/src/ntpstats/stability.py +626 -0
  28. ntpstats-2.5.0/src/ntpstats/web/__init__.py +2 -0
  29. ntpstats-2.5.0/src/ntpstats/web/server.py +634 -0
  30. ntpstats-2.5.0/src/ntpstats/web/static/app.css +176 -0
  31. ntpstats-2.5.0/src/ntpstats/web/static/app.js +732 -0
  32. ntpstats-2.5.0/src/ntpstats/web/static/index.html +242 -0
  33. ntpstats-2.5.0/src/ntpstats/web/vendor/uPlot.LICENSE +21 -0
  34. ntpstats-2.5.0/src/ntpstats/web/vendor/uPlot.iife.min.js +2 -0
  35. ntpstats-2.5.0/src/ntpstats/web/vendor/uPlot.min.css +1 -0
  36. ntpstats-2.5.0/src/ntpstats.egg-info/PKG-INFO +332 -0
  37. ntpstats-2.5.0/src/ntpstats.egg-info/SOURCES.txt +50 -0
  38. ntpstats-2.5.0/src/ntpstats.egg-info/dependency_links.txt +1 -0
  39. ntpstats-2.5.0/src/ntpstats.egg-info/entry_points.txt +2 -0
  40. ntpstats-2.5.0/src/ntpstats.egg-info/requires.txt +24 -0
  41. ntpstats-2.5.0/src/ntpstats.egg-info/top_level.txt +1 -0
  42. ntpstats-2.5.0/tests/test_analysis.py +152 -0
  43. ntpstats-2.5.0/tests/test_bench.py +177 -0
  44. ntpstats-2.5.0/tests/test_cli_web.py +305 -0
  45. ntpstats-2.5.0/tests/test_edf.py +49 -0
  46. ntpstats-2.5.0/tests/test_nts.py +188 -0
  47. ntpstats-2.5.0/tests/test_parsers.py +182 -0
  48. ntpstats-2.5.0/tests/test_pcap.py +28 -0
  49. ntpstats-2.5.0/tests/test_sntp.py +219 -0
  50. ntpstats-2.5.0/tests/test_sources.py +62 -0
  51. ntpstats-2.5.0/tests/test_stability.py +296 -0
  52. ntpstats-2.5.0/tests/test_version.py +23 -0
ntpstats-2.5.0/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2012-2026 Thiago de Freitas <thiagodefreitas@gmail.com>
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
ntpstats-2.5.0/NOTICE ADDED
@@ -0,0 +1,15 @@
1
+ ntpstats — NTP clock-offset and time-stability analysis toolkit
2
+ Copyright (c) 2012-2026 Thiago de Freitas <thiagodefreitas@gmail.com>
3
+ Licensed under the MIT License (see LICENSE). Commercial use is welcome;
4
+ please keep the copyright notice and credit the author. If you use ntpstats
5
+ in academic work, please cite it (see CITATION.cff).
6
+
7
+ Originally developed during Google Summer of Code 2012 for the NTP Project.
8
+ Original credits: Judah Levine, Harlan Stenn, Antonio Lima.
9
+ The original 2012 prototype is kept unchanged under legacy/ and remains under
10
+ its original license (GPL, see legacy/gsoc2012/COPYING); none of it is used by
11
+ the new package.
12
+
13
+ Bundled third-party component:
14
+ * uPlot 1.6.32 (src/ntpstats/web/vendor/) — MIT License,
15
+ Copyright (c) 2022 Leon Sorokin. See src/ntpstats/web/vendor/uPlot.LICENSE.
@@ -0,0 +1,332 @@
1
+ Metadata-Version: 2.4
2
+ Name: ntpstats
3
+ Version: 2.5.0
4
+ Summary: NTP / network-time offset, delay and frequency-stability analysis (ADEV, MDEV, TDEV, HDEV, MTIE) with a lightweight web UI
5
+ Author-email: Thiago de Freitas <thiagodefreitas@gmail.com>
6
+ License: MIT
7
+ Project-URL: Homepage, https://github.com/thiagodefreitas/NetworkTime
8
+ Project-URL: Issues, https://github.com/thiagodefreitas/NetworkTime/issues
9
+ Keywords: ntp,chrony,ntpsec,allan deviation,tdev,mtie,time synchronization,clock
10
+ Classifier: Development Status :: 4 - Beta
11
+ Classifier: Intended Audience :: Science/Research
12
+ Classifier: Intended Audience :: System Administrators
13
+ Classifier: License :: OSI Approved :: MIT License
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Topic :: Scientific/Engineering
16
+ Classifier: Topic :: System :: Networking :: Time Synchronization
17
+ Requires-Python: >=3.9
18
+ Description-Content-Type: text/markdown
19
+ License-File: LICENSE
20
+ License-File: NOTICE
21
+ Requires-Dist: numpy>=1.22
22
+ Provides-Extra: plot
23
+ Requires-Dist: matplotlib>=3.6; extra == "plot"
24
+ Provides-Extra: nts
25
+ Requires-Dist: pyOpenSSL>=23.2; extra == "nts"
26
+ Requires-Dist: cryptography>=42; extra == "nts"
27
+ Provides-Extra: test
28
+ Requires-Dist: pytest>=7; extra == "test"
29
+ Requires-Dist: tomli>=2; python_version < "3.11" and extra == "test"
30
+ Provides-Extra: toml
31
+ Requires-Dist: tomli>=2; python_version < "3.11" and extra == "toml"
32
+ Provides-Extra: docs
33
+ Requires-Dist: mkdocs<2,>=1.5; extra == "docs"
34
+ Requires-Dist: mkdocs-material>=9.5; extra == "docs"
35
+ Requires-Dist: mkdocstrings[python]>=0.25; extra == "docs"
36
+ Dynamic: license-file
37
+
38
+ # ntpstats — NetworkTime analysis toolkit
39
+
40
+ **Validate, evaluate and study network time synchronisation.** `ntpstats` reads the logs written by
41
+ today's NTP implementations (**ntpd 4.2.8, NTPsec, chrony**), measures servers directly with a
42
+ privacy-preserving SNTP client, and computes the clock-offset, network-delay and
43
+ frequency-stability statistics used in timing research and in telecom standards (ADEV, MDEV,
44
+ TDEV, Hadamard, MTIE, floor packet percentage) with confidence intervals and noise
45
+ identification. A built-in simulator with **ground truth** turns it into a test bench for
46
+ synchronisation algorithms.
47
+
48
+ 📖 **Documentation:** [thiagodefreitas.github.io/NetworkTime](https://thiagodefreitas.github.io/NetworkTime/)
49
+ · [Wiki](https://github.com/thiagodefreitas/NetworkTime/wiki) · `pip install ntpstats`
50
+
51
+ It started as a Google Summer of Code 2012 project for the NTP Project (kept unchanged in
52
+ [`legacy/`](legacy/)); version 2 is a complete rewrite. See
53
+ [docs/STATE_OF_THE_ART.md](docs/STATE_OF_THE_ART.md) for what changed in NTP since 2012 and what
54
+ was wrong with the original code, [docs/INTEROP.md](docs/INTEROP.md) for live results against
55
+ public NTP/NTS servers, [CHANGELOG.md](CHANGELOG.md) for releases and
56
+ [ROADMAP.md](ROADMAP.md) for where it is going.
57
+
58
+ ![Offset view with RTS smoother and ground truth](docs/img/ui-offset-light.png)
59
+
60
+ | | |
61
+ |---|---|
62
+ | ![Stability with confidence bands and slope guides](docs/img/ui-stability.png) | ![Network wedge, delay floor and FPP](docs/img/ui-network.png) |
63
+ | ![Comparing two peers](docs/img/ui-stability-compare.png) | ![Dark mode](docs/img/ui-offset-dark.png) |
64
+
65
+ ## Highlights
66
+
67
+ - **Every common source, auto-detected**:
68
+ - ntpd/NTPsec `loopstats`, `peerstats` (per peer, `sys.peer` picked automatically) and
69
+ `rawstats` (offset/delay recomputed from the four on-wire timestamps);
70
+ - chrony `tracking.log`, `measurements.log`, `statistics.log` and `refclocks.log`
71
+ (GNSS/PPS reference clocks);
72
+ - **PTP**: linuxptp `ptp4l`, `phc2sys`, `ts2phc` output from stdout, syslog or journald
73
+ (monotonic stamps are mapped to UTC when the journal prefix is present);
74
+ - **packet captures** (pcap/pcapng, including nanosecond and hardware timestamps): NTP
75
+ exchanges are matched and measured from the capture host's clock;
76
+ - generic CSV and the 2012 `estimators.log`.
77
+
78
+ Every sign convention is normalised to *reference − local*, following each implementation's
79
+ documentation (table below).
80
+ - **Stability analysis done right**: non-overlapping and overlapping ADEV, MDEV, TDEV,
81
+ overlapping Hadamard, total and modified total deviation (TOTDEV, MTOT), Theo1, TheoBR, TheoH,
82
+ MTIE (O(N log N)) and TIErms, each with
83
+ - χ² confidence intervals from the **exact** equivalent degrees of freedom of the discrete
84
+ power-law model (Monte Carlo verified),
85
+ - per-τ power-law noise identification (lag-1 autocorrelation method),
86
+ - correct τ₀ from the data and **gap handling that never invents data** (irregular logs are
87
+ put on a grid; terms that would span a gap are dropped, not interpolated),
88
+ - **dynamic** (sliding-window) views to expose non-stationarity,
89
+ - **limit-mask** checks against user-supplied CSV masks (`tau,tdev`, `tau,mtie`, … e.g.
90
+ network TDEV/MTIE limits), with margins and PASS/FAIL,
91
+ - **compare** a source against a reference (PPS, GNSS or a better server) to get the error's
92
+ bias, RMS, TDEV and MTIE.
93
+ - **Network metrics**: delay floor and queueing distribution, Mills' offset-vs-delay *wedge*,
94
+ asymmetry indicator, floor packet percentage (ITU-T G.8260-style), NTP clock-filter
95
+ (minimum delay) selection.
96
+ - **Estimators**: two-state Kalman filter with irregular sampling, noise parameters fitted
97
+ from the data's ADEV, innovation gating, delay-aware measurement weighting and an
98
+ RTS smoother.
99
+ - **Research bench**: simulated clocks and networks with ground truth, and pluggable
100
+ estimators scored by `ntpstats bench`:
101
+ - simulator: power-law oscillator noise (white/flicker PM, white/flicker/random-walk FM),
102
+ frequency offset, drift, temperature wander, asymmetric queueing paths, route changes,
103
+ congestion and outages, and multi-server scenarios with falsetickers;
104
+ - reference algorithms: Kalman/RTS, NTP clock filter, chrony-style regression,
105
+ RADclock-style feed-forward, and RFC 5905 select/cluster/combine;
106
+ - bring your own estimator via a small API or a package entry point; scenarios can be
107
+ presets or TOML files;
108
+ - reproducible reports as tables, CSV/JSON or self-contained HTML.
109
+ - **Reports**: `ntpstats report` writes a single offline HTML file with charts, CI tables,
110
+ network analysis, parameters and input hashes (also a *Report* button in the UI).
111
+ - **Measurement clients** (they never set the clock):
112
+ - NTPv4 SNTP with a random transmit timestamp (data minimisation), origin check,
113
+ Kiss-o'-Death and 2036 era handling;
114
+ - **NTS** (RFC 8915): NTS-KE over TLS 1.3 with certificate and host-name checks, AES-SIV
115
+ authenticated exchanges and cookie renewal (optional extra `ntpstats[nts]`);
116
+ - experimental **NTPv5** (draft-ietf-ntp-ntpv5-09): v5 header with cookies, timescale and
117
+ era, plus the NTPv4→v5 upgrade probe;
118
+ - a polite `monitor` (RATE back-off, jitter), and `watch`, which samples the local
119
+ **chrony** (`chronyc -c tracking`) or **ntpd/NTPsec** (`ntpq -c rv`) without log files.
120
+ - **Lightweight UI**: `ntpstats ui` starts a local web app. It runs on the Python standard
121
+ library HTTP server with one static page and [uPlot](https://github.com/leeoniya/uPlot) (≈50 kB,
122
+ bundled, MIT), so there is no Qt, Electron, Node or CDN, and it works offline. Light and dark
123
+ themes, drag-and-drop, overlay comparison, zoom-to-analyse, PNG/CSV export and a live
124
+ monitor.
125
+ - **Small footprint**: the only runtime dependency is **numpy**. matplotlib is optional
126
+ (static/publication figures).
127
+
128
+ ## Install
129
+
130
+ ```bash
131
+ pip install ntpstats # core + UI (numpy only)
132
+ pip install "ntpstats[plot]" # + matplotlib figures
133
+ pip install "ntpstats[nts]" # + NTS client (pyOpenSSL, cryptography)
134
+ pip install git+https://github.com/thiagodefreitas/NetworkTime.git # latest master
135
+ # from a checkout, for development:
136
+ pip install -e ".[test,plot,nts,docs]" && pytest
137
+ ```
138
+
139
+ Python ≥ 3.9.
140
+
141
+ ## Quick start
142
+
143
+ ```bash
144
+ # Web UI (opens your browser at http://127.0.0.1:8123)
145
+ ntpstats ui /var/log/chrony/measurements.log /var/log/ntpstats/peerstats
146
+
147
+ # Summary statistics
148
+ ntpstats info /var/log/ntpstats/loopstats.20260928
149
+
150
+ # Stability table with 95 % confidence intervals and noise identification
151
+ ntpstats stability /var/log/chrony/tracking.log -k oadev,mdev,tdev,mtie --ci 0.95
152
+
153
+ # Check TDEV against a limit mask (CSV "tau,tdev"); exit code 3 on failure
154
+ ntpstats stability ptp.log -k tdev,mtie --mask my-tdev-mask.csv
155
+
156
+ # Validate a client against a reference (e.g. chrony vs a PPS refclock or GNSS host)
157
+ ntpstats compare /var/log/chrony/tracking.log /var/log/chrony/refclocks.log --ref-peer PPS0
158
+
159
+ # Sliding-window stability matrix (time x tau) as CSV
160
+ ntpstats dynamic peerstats --peer 192.0.2.10 -k mdev
161
+
162
+ # Network behaviour of one peer
163
+ ntpstats network /var/log/ntpstats/peerstats --peer 192.0.2.10
164
+
165
+ # Filters (Kalman / RTS smoother / min-delay) -> CSV
166
+ ntpstats filter peerstats --peer 192.0.2.10 -m rts -o smoothed.csv
167
+
168
+ # Publication figure (needs matplotlib)
169
+ ntpstats plot peerstats --all-peers -k oadev,tdev --detrend linear -o report.pdf
170
+
171
+ # Measure a server yourself (never sets the clock)
172
+ ntpstats query time.cloudflare.com -c 4
173
+ ntpstats monitor pool.ntp.org ptbtime1.ptb.de -i 64 -o mylog.csv
174
+
175
+ # Test bench: simulate NTP exchanges and score the built-in estimators
176
+ ntpstats simulate --preset internet --benchmark
177
+ ```
178
+
179
+ ```
180
+ Scenario 'internet': 1350 exchanges, poll 64 s, seed 1
181
+ estimator rms bias p95 |err| max |err|
182
+ raw measurements 1.65 ms -659 µs 3.61 ms 14.1 ms
183
+ Kalman 617 µs -569 µs 945 µs 1.84 ms
184
+ Kalman + delay weighting 183 µs -139 µs 355 µs 1.31 ms
185
+ RTS smoother + delay weighting 210 µs -202 µs 293 µs 315 µs
186
+ min-delay filter (8) 112 µs 761 ns 248 µs 752 µs
187
+ ```
188
+
189
+ More measurement commands:
190
+
191
+ ```bash
192
+ ntpstats query time.cloudflare.com --nts # NTS-authenticated (pip install 'ntpstats[nts]')
193
+ ntpstats query ntpd-rs.example.net --ntpv5 # experimental NTPv5 draft-09
194
+ ntpstats query pool.ntp.org --probe-v5 # does the server offer NTPv5?
195
+ ntpstats watch chrony -i 16 -o chrony-live.csv # local daemon, no log files needed
196
+ ntpstats info capture.pcapng # NTP exchanges from a packet capture
197
+ ntpstats stability /var/log/ptp4l.log -k tdev,mtie # PTP servo offsets
198
+ ```
199
+
200
+ ### Benchmarking synchronisation algorithms
201
+
202
+ ```bash
203
+ ntpstats bench --list # estimators and preset scenarios
204
+ ntpstats bench internet falseticker examples/scenarios/*.toml --seeds 1-10 \
205
+ --html bench.html --csv bench.csv # full comparison
206
+ python examples/04_custom_estimator.py # plug in your own algorithm
207
+ ```
208
+
209
+ ![Benchmark report](docs/img/benchmark-report.png)
210
+
211
+ Example reports: [benchmark](docs/examples/benchmark-report.html),
212
+ [peerstats analysis](docs/examples/peerstats-report.html) (download and open in a browser).
213
+
214
+ Some findings the bench makes visible:
215
+ - On a congested WAN, the delay-aware estimators (regression, feed-forward, min-delay,
216
+ delay-weighted Kalman) cut the error 10–30× compared with the raw measurements.
217
+ - An asymmetric route change (`route-change` preset) biases *every* estimator by half the
218
+ one-way step, and no amount of filtering can see it.
219
+ - RFC 5905 selection rejects falsetickers only when their error exceeds the root distance
220
+ (≈ delay/2). Smaller ones are indistinguishable from path asymmetry by design.
221
+
222
+ Common options: `--format` (override detection), `--peer`, `--all-peers`,
223
+ `--start/--end` (POSIX seconds or ISO 8601 UTC), `--outliers K` (drop > K·MAD after
224
+ detrending), `--json`/`--csv` output.
225
+
226
+ ### Sign conventions
227
+
228
+ | Source | Logged as | ntpstats |
229
+ |---|---|---|
230
+ | ntpd/NTPsec loopstats, peerstats, rawstats, `ntpq rv` | reference − local | kept |
231
+ | chrony `measurements.log` (θ), `refclocks.log` (cooked), `chronyc tracking` "System time" | reference − local ("positive = local slow") | kept |
232
+ | chrony `tracking.log`, `statistics.log`, `chronyc sourcestats` | local − reference ("positive = local fast") | negated |
233
+ | linuxptp `master offset` / `offset` (ns) | local − reference | negated, converted to s |
234
+ | pcap captures | computed from server T2/T3 and capture times | reference − capture host |
235
+
236
+ ### Enabling the logs
237
+
238
+ - **chrony** (`/etc/chrony/chrony.conf` or `/etc/chrony.conf`): `log tracking measurements statistics refclocks`
239
+ and `logdir /var/log/chrony`.
240
+ - **linuxptp**: run `ptp4l`/`phc2sys` with `-m` (stdout) or collect
241
+ `journalctl -u ptp4l -o short-iso-precise`, which gives UTC time stamps.
242
+ - **captures**: `tcpdump -i eth0 -j adapter_unsynced --time-stamp-precision=nano -w ntp.pcap udp port 123`.
243
+ - **ntpd / NTPsec** (`ntp.conf`): `statsdir /var/log/ntpstats/`, `statistics loopstats peerstats rawstats`,
244
+ `filegen peerstats file peerstats type day enable` (same for the others).
245
+
246
+ ### Docker
247
+
248
+ ```bash
249
+ docker build -t ntpstats .
250
+ docker run --rm -p 8123:8123 -v "$PWD:/data" ntpstats ui /data/peerstats --host 0.0.0.0 --no-browser
251
+ docker run --rm ntpstats query --nts time.cloudflare.com
252
+ ```
253
+
254
+ ### Performance
255
+
256
+ On a modest shared CPU: parsing takes about 1 s per million loopstats lines (2.4 s for
257
+ peerstats) using numpy's C tokenizer, and a full stability analysis (OADEV/MDEV/TDEV/HDEV with
258
+ exact-EDF confidence intervals) of **1 million samples** takes about 1.2 s. The UI only receives
259
+ decimated data (min/max per bucket), so it stays responsive with large files.
260
+
261
+ ## Python API
262
+
263
+ ```python
264
+ from ntpstats import load_one
265
+ from ntpstats.stability import series_stability
266
+ from ntpstats.analysis import summary, compare
267
+ from ntpstats.simulate import Scenario, simulate_ntp
268
+ from ntpstats.filters import kalman_series
269
+
270
+ s = load_one("/var/log/chrony/measurements.log", peer="192.0.2.10")
271
+ print(summary(s)["range_90"])
272
+ for r in series_stability(s, kinds=("oadev", "tdev"), ci=0.95):
273
+ print(r.kind, r.taus, r.dev, r.lo, r.hi, r.alpha)
274
+
275
+ meas, truth = simulate_ntp(Scenario(duration=86400, seed=1))
276
+ print(compare(kalman_series(meas, smooth=True), truth))
277
+ ```
278
+
279
+ More in [`examples/`](examples/): `01_quickstart.py`, `02_benchmark_filters.py` (a template for
280
+ evaluating your own algorithm), `03_report_figure.py`, and `make_example_logs.py`, which
281
+ regenerates the sample logs in `examples/data/` in each native format.
282
+
283
+ ## Validation
284
+
285
+ `pytest` runs about 170 tests (plus `ruff` lint and coverage in CI), and none of them need network access:
286
+
287
+ - every estimator is checked against a literal implementation of the NIST SP 1065 sums, against
288
+ the analytic log-log slopes of the five power-law noise types, and against a frozen table of
289
+ values from an independent implementation (no third-party stability library is needed);
290
+ - the confidence-interval coverage is checked by Monte Carlo;
291
+ - noise identification is checked for α = +2 … −2;
292
+ - parsers are checked on the line layouts from the ntpd and chrony documentation, and by
293
+ cross-checking that the same simulated peer reads identically from peerstats, rawstats and
294
+ chrony logs;
295
+ - the SNTP, NTPv5 and NTS clients are checked against local test servers (offset, delay, KoD,
296
+ spoofed replies, timeouts, 2036 rollover, TAI timescale, forged NTS responses, certificate
297
+ mismatch);
298
+ - the pcap/pcapng readers are checked on synthesised captures (VLAN tags, nanosecond
299
+ resolution, NTPv4 and v5 matching), and the linuxptp parser on the exact `pr_info` formats from
300
+ the linuxptp source;
301
+ - the web API is checked, including its CSRF, Host-header and path-traversal protections.
302
+
303
+ CI runs on Python 3.9, 3.11 and 3.13.
304
+
305
+ ## Security notes
306
+
307
+ The UI binds to `127.0.0.1`, rejects foreign `Host` headers (DNS rebinding), requires a custom
308
+ header on all state-changing requests (CSRF) and sends a strict Content-Security-Policy. It has no
309
+ authentication, so do not expose it with `--host 0.0.0.0` on untrusted networks. The SNTP client
310
+ is for measurement only: it does not authenticate time (for NTS use chrony or NTPsec and analyse
311
+ their logs).
312
+
313
+ ## Project layout
314
+
315
+ ```
316
+ src/ntpstats/ parsers, pcap, stability, edf, masks, analysis, network, filters, simulate,
317
+ estimators, bench, report, sntp (v4/v5), nts, sources (chronyc/ntpq),
318
+ monitor, cli, plotting
319
+ src/ntpstats/web stdlib HTTP server + static UI (uPlot vendored)
320
+ tests/ pytest suite (+ frozen reference data)
321
+ examples/ scripts and sample logs
322
+ docs/ state of the art, live interop results, example reports, screenshots
323
+ legacy/ the original 2012 GSoC code, untouched
324
+ ```
325
+
326
+ ## License and citation
327
+
328
+ MIT License. © 2012–2026 **Thiago de Freitas** <thiagodefreitas@gmail.com>. Free for commercial
329
+ and non-commercial use; please keep the copyright notice and credit the author (see
330
+ [NOTICE](NOTICE)). If you use it in research, please cite it via [CITATION.cff](CITATION.cff).
331
+ The bundled uPlot is MIT-licensed © Leon Sorokin. The code under `legacy/` keeps its original
332
+ 2012 license.
@@ -0,0 +1,295 @@
1
+ # ntpstats — NetworkTime analysis toolkit
2
+
3
+ **Validate, evaluate and study network time synchronisation.** `ntpstats` reads the logs written by
4
+ today's NTP implementations (**ntpd 4.2.8, NTPsec, chrony**), measures servers directly with a
5
+ privacy-preserving SNTP client, and computes the clock-offset, network-delay and
6
+ frequency-stability statistics used in timing research and in telecom standards (ADEV, MDEV,
7
+ TDEV, Hadamard, MTIE, floor packet percentage) with confidence intervals and noise
8
+ identification. A built-in simulator with **ground truth** turns it into a test bench for
9
+ synchronisation algorithms.
10
+
11
+ 📖 **Documentation:** [thiagodefreitas.github.io/NetworkTime](https://thiagodefreitas.github.io/NetworkTime/)
12
+ · [Wiki](https://github.com/thiagodefreitas/NetworkTime/wiki) · `pip install ntpstats`
13
+
14
+ It started as a Google Summer of Code 2012 project for the NTP Project (kept unchanged in
15
+ [`legacy/`](legacy/)); version 2 is a complete rewrite. See
16
+ [docs/STATE_OF_THE_ART.md](docs/STATE_OF_THE_ART.md) for what changed in NTP since 2012 and what
17
+ was wrong with the original code, [docs/INTEROP.md](docs/INTEROP.md) for live results against
18
+ public NTP/NTS servers, [CHANGELOG.md](CHANGELOG.md) for releases and
19
+ [ROADMAP.md](ROADMAP.md) for where it is going.
20
+
21
+ ![Offset view with RTS smoother and ground truth](docs/img/ui-offset-light.png)
22
+
23
+ | | |
24
+ |---|---|
25
+ | ![Stability with confidence bands and slope guides](docs/img/ui-stability.png) | ![Network wedge, delay floor and FPP](docs/img/ui-network.png) |
26
+ | ![Comparing two peers](docs/img/ui-stability-compare.png) | ![Dark mode](docs/img/ui-offset-dark.png) |
27
+
28
+ ## Highlights
29
+
30
+ - **Every common source, auto-detected**:
31
+ - ntpd/NTPsec `loopstats`, `peerstats` (per peer, `sys.peer` picked automatically) and
32
+ `rawstats` (offset/delay recomputed from the four on-wire timestamps);
33
+ - chrony `tracking.log`, `measurements.log`, `statistics.log` and `refclocks.log`
34
+ (GNSS/PPS reference clocks);
35
+ - **PTP**: linuxptp `ptp4l`, `phc2sys`, `ts2phc` output from stdout, syslog or journald
36
+ (monotonic stamps are mapped to UTC when the journal prefix is present);
37
+ - **packet captures** (pcap/pcapng, including nanosecond and hardware timestamps): NTP
38
+ exchanges are matched and measured from the capture host's clock;
39
+ - generic CSV and the 2012 `estimators.log`.
40
+
41
+ Every sign convention is normalised to *reference − local*, following each implementation's
42
+ documentation (table below).
43
+ - **Stability analysis done right**: non-overlapping and overlapping ADEV, MDEV, TDEV,
44
+ overlapping Hadamard, total and modified total deviation (TOTDEV, MTOT), Theo1, TheoBR, TheoH,
45
+ MTIE (O(N log N)) and TIErms, each with
46
+ - χ² confidence intervals from the **exact** equivalent degrees of freedom of the discrete
47
+ power-law model (Monte Carlo verified),
48
+ - per-τ power-law noise identification (lag-1 autocorrelation method),
49
+ - correct τ₀ from the data and **gap handling that never invents data** (irregular logs are
50
+ put on a grid; terms that would span a gap are dropped, not interpolated),
51
+ - **dynamic** (sliding-window) views to expose non-stationarity,
52
+ - **limit-mask** checks against user-supplied CSV masks (`tau,tdev`, `tau,mtie`, … e.g.
53
+ network TDEV/MTIE limits), with margins and PASS/FAIL,
54
+ - **compare** a source against a reference (PPS, GNSS or a better server) to get the error's
55
+ bias, RMS, TDEV and MTIE.
56
+ - **Network metrics**: delay floor and queueing distribution, Mills' offset-vs-delay *wedge*,
57
+ asymmetry indicator, floor packet percentage (ITU-T G.8260-style), NTP clock-filter
58
+ (minimum delay) selection.
59
+ - **Estimators**: two-state Kalman filter with irregular sampling, noise parameters fitted
60
+ from the data's ADEV, innovation gating, delay-aware measurement weighting and an
61
+ RTS smoother.
62
+ - **Research bench**: simulated clocks and networks with ground truth, and pluggable
63
+ estimators scored by `ntpstats bench`:
64
+ - simulator: power-law oscillator noise (white/flicker PM, white/flicker/random-walk FM),
65
+ frequency offset, drift, temperature wander, asymmetric queueing paths, route changes,
66
+ congestion and outages, and multi-server scenarios with falsetickers;
67
+ - reference algorithms: Kalman/RTS, NTP clock filter, chrony-style regression,
68
+ RADclock-style feed-forward, and RFC 5905 select/cluster/combine;
69
+ - bring your own estimator via a small API or a package entry point; scenarios can be
70
+ presets or TOML files;
71
+ - reproducible reports as tables, CSV/JSON or self-contained HTML.
72
+ - **Reports**: `ntpstats report` writes a single offline HTML file with charts, CI tables,
73
+ network analysis, parameters and input hashes (also a *Report* button in the UI).
74
+ - **Measurement clients** (they never set the clock):
75
+ - NTPv4 SNTP with a random transmit timestamp (data minimisation), origin check,
76
+ Kiss-o'-Death and 2036 era handling;
77
+ - **NTS** (RFC 8915): NTS-KE over TLS 1.3 with certificate and host-name checks, AES-SIV
78
+ authenticated exchanges and cookie renewal (optional extra `ntpstats[nts]`);
79
+ - experimental **NTPv5** (draft-ietf-ntp-ntpv5-09): v5 header with cookies, timescale and
80
+ era, plus the NTPv4→v5 upgrade probe;
81
+ - a polite `monitor` (RATE back-off, jitter), and `watch`, which samples the local
82
+ **chrony** (`chronyc -c tracking`) or **ntpd/NTPsec** (`ntpq -c rv`) without log files.
83
+ - **Lightweight UI**: `ntpstats ui` starts a local web app. It runs on the Python standard
84
+ library HTTP server with one static page and [uPlot](https://github.com/leeoniya/uPlot) (≈50 kB,
85
+ bundled, MIT), so there is no Qt, Electron, Node or CDN, and it works offline. Light and dark
86
+ themes, drag-and-drop, overlay comparison, zoom-to-analyse, PNG/CSV export and a live
87
+ monitor.
88
+ - **Small footprint**: the only runtime dependency is **numpy**. matplotlib is optional
89
+ (static/publication figures).
90
+
91
+ ## Install
92
+
93
+ ```bash
94
+ pip install ntpstats # core + UI (numpy only)
95
+ pip install "ntpstats[plot]" # + matplotlib figures
96
+ pip install "ntpstats[nts]" # + NTS client (pyOpenSSL, cryptography)
97
+ pip install git+https://github.com/thiagodefreitas/NetworkTime.git # latest master
98
+ # from a checkout, for development:
99
+ pip install -e ".[test,plot,nts,docs]" && pytest
100
+ ```
101
+
102
+ Python ≥ 3.9.
103
+
104
+ ## Quick start
105
+
106
+ ```bash
107
+ # Web UI (opens your browser at http://127.0.0.1:8123)
108
+ ntpstats ui /var/log/chrony/measurements.log /var/log/ntpstats/peerstats
109
+
110
+ # Summary statistics
111
+ ntpstats info /var/log/ntpstats/loopstats.20260928
112
+
113
+ # Stability table with 95 % confidence intervals and noise identification
114
+ ntpstats stability /var/log/chrony/tracking.log -k oadev,mdev,tdev,mtie --ci 0.95
115
+
116
+ # Check TDEV against a limit mask (CSV "tau,tdev"); exit code 3 on failure
117
+ ntpstats stability ptp.log -k tdev,mtie --mask my-tdev-mask.csv
118
+
119
+ # Validate a client against a reference (e.g. chrony vs a PPS refclock or GNSS host)
120
+ ntpstats compare /var/log/chrony/tracking.log /var/log/chrony/refclocks.log --ref-peer PPS0
121
+
122
+ # Sliding-window stability matrix (time x tau) as CSV
123
+ ntpstats dynamic peerstats --peer 192.0.2.10 -k mdev
124
+
125
+ # Network behaviour of one peer
126
+ ntpstats network /var/log/ntpstats/peerstats --peer 192.0.2.10
127
+
128
+ # Filters (Kalman / RTS smoother / min-delay) -> CSV
129
+ ntpstats filter peerstats --peer 192.0.2.10 -m rts -o smoothed.csv
130
+
131
+ # Publication figure (needs matplotlib)
132
+ ntpstats plot peerstats --all-peers -k oadev,tdev --detrend linear -o report.pdf
133
+
134
+ # Measure a server yourself (never sets the clock)
135
+ ntpstats query time.cloudflare.com -c 4
136
+ ntpstats monitor pool.ntp.org ptbtime1.ptb.de -i 64 -o mylog.csv
137
+
138
+ # Test bench: simulate NTP exchanges and score the built-in estimators
139
+ ntpstats simulate --preset internet --benchmark
140
+ ```
141
+
142
+ ```
143
+ Scenario 'internet': 1350 exchanges, poll 64 s, seed 1
144
+ estimator rms bias p95 |err| max |err|
145
+ raw measurements 1.65 ms -659 µs 3.61 ms 14.1 ms
146
+ Kalman 617 µs -569 µs 945 µs 1.84 ms
147
+ Kalman + delay weighting 183 µs -139 µs 355 µs 1.31 ms
148
+ RTS smoother + delay weighting 210 µs -202 µs 293 µs 315 µs
149
+ min-delay filter (8) 112 µs 761 ns 248 µs 752 µs
150
+ ```
151
+
152
+ More measurement commands:
153
+
154
+ ```bash
155
+ ntpstats query time.cloudflare.com --nts # NTS-authenticated (pip install 'ntpstats[nts]')
156
+ ntpstats query ntpd-rs.example.net --ntpv5 # experimental NTPv5 draft-09
157
+ ntpstats query pool.ntp.org --probe-v5 # does the server offer NTPv5?
158
+ ntpstats watch chrony -i 16 -o chrony-live.csv # local daemon, no log files needed
159
+ ntpstats info capture.pcapng # NTP exchanges from a packet capture
160
+ ntpstats stability /var/log/ptp4l.log -k tdev,mtie # PTP servo offsets
161
+ ```
162
+
163
+ ### Benchmarking synchronisation algorithms
164
+
165
+ ```bash
166
+ ntpstats bench --list # estimators and preset scenarios
167
+ ntpstats bench internet falseticker examples/scenarios/*.toml --seeds 1-10 \
168
+ --html bench.html --csv bench.csv # full comparison
169
+ python examples/04_custom_estimator.py # plug in your own algorithm
170
+ ```
171
+
172
+ ![Benchmark report](docs/img/benchmark-report.png)
173
+
174
+ Example reports: [benchmark](docs/examples/benchmark-report.html),
175
+ [peerstats analysis](docs/examples/peerstats-report.html) (download and open in a browser).
176
+
177
+ Some findings the bench makes visible:
178
+ - On a congested WAN, the delay-aware estimators (regression, feed-forward, min-delay,
179
+ delay-weighted Kalman) cut the error 10–30× compared with the raw measurements.
180
+ - An asymmetric route change (`route-change` preset) biases *every* estimator by half the
181
+ one-way step, and no amount of filtering can see it.
182
+ - RFC 5905 selection rejects falsetickers only when their error exceeds the root distance
183
+ (≈ delay/2). Smaller ones are indistinguishable from path asymmetry by design.
184
+
185
+ Common options: `--format` (override detection), `--peer`, `--all-peers`,
186
+ `--start/--end` (POSIX seconds or ISO 8601 UTC), `--outliers K` (drop > K·MAD after
187
+ detrending), `--json`/`--csv` output.
188
+
189
+ ### Sign conventions
190
+
191
+ | Source | Logged as | ntpstats |
192
+ |---|---|---|
193
+ | ntpd/NTPsec loopstats, peerstats, rawstats, `ntpq rv` | reference − local | kept |
194
+ | chrony `measurements.log` (θ), `refclocks.log` (cooked), `chronyc tracking` "System time" | reference − local ("positive = local slow") | kept |
195
+ | chrony `tracking.log`, `statistics.log`, `chronyc sourcestats` | local − reference ("positive = local fast") | negated |
196
+ | linuxptp `master offset` / `offset` (ns) | local − reference | negated, converted to s |
197
+ | pcap captures | computed from server T2/T3 and capture times | reference − capture host |
198
+
199
+ ### Enabling the logs
200
+
201
+ - **chrony** (`/etc/chrony/chrony.conf` or `/etc/chrony.conf`): `log tracking measurements statistics refclocks`
202
+ and `logdir /var/log/chrony`.
203
+ - **linuxptp**: run `ptp4l`/`phc2sys` with `-m` (stdout) or collect
204
+ `journalctl -u ptp4l -o short-iso-precise`, which gives UTC time stamps.
205
+ - **captures**: `tcpdump -i eth0 -j adapter_unsynced --time-stamp-precision=nano -w ntp.pcap udp port 123`.
206
+ - **ntpd / NTPsec** (`ntp.conf`): `statsdir /var/log/ntpstats/`, `statistics loopstats peerstats rawstats`,
207
+ `filegen peerstats file peerstats type day enable` (same for the others).
208
+
209
+ ### Docker
210
+
211
+ ```bash
212
+ docker build -t ntpstats .
213
+ docker run --rm -p 8123:8123 -v "$PWD:/data" ntpstats ui /data/peerstats --host 0.0.0.0 --no-browser
214
+ docker run --rm ntpstats query --nts time.cloudflare.com
215
+ ```
216
+
217
+ ### Performance
218
+
219
+ On a modest shared CPU: parsing takes about 1 s per million loopstats lines (2.4 s for
220
+ peerstats) using numpy's C tokenizer, and a full stability analysis (OADEV/MDEV/TDEV/HDEV with
221
+ exact-EDF confidence intervals) of **1 million samples** takes about 1.2 s. The UI only receives
222
+ decimated data (min/max per bucket), so it stays responsive with large files.
223
+
224
+ ## Python API
225
+
226
+ ```python
227
+ from ntpstats import load_one
228
+ from ntpstats.stability import series_stability
229
+ from ntpstats.analysis import summary, compare
230
+ from ntpstats.simulate import Scenario, simulate_ntp
231
+ from ntpstats.filters import kalman_series
232
+
233
+ s = load_one("/var/log/chrony/measurements.log", peer="192.0.2.10")
234
+ print(summary(s)["range_90"])
235
+ for r in series_stability(s, kinds=("oadev", "tdev"), ci=0.95):
236
+ print(r.kind, r.taus, r.dev, r.lo, r.hi, r.alpha)
237
+
238
+ meas, truth = simulate_ntp(Scenario(duration=86400, seed=1))
239
+ print(compare(kalman_series(meas, smooth=True), truth))
240
+ ```
241
+
242
+ More in [`examples/`](examples/): `01_quickstart.py`, `02_benchmark_filters.py` (a template for
243
+ evaluating your own algorithm), `03_report_figure.py`, and `make_example_logs.py`, which
244
+ regenerates the sample logs in `examples/data/` in each native format.
245
+
246
+ ## Validation
247
+
248
+ `pytest` runs about 170 tests (plus `ruff` lint and coverage in CI), and none of them need network access:
249
+
250
+ - every estimator is checked against a literal implementation of the NIST SP 1065 sums, against
251
+ the analytic log-log slopes of the five power-law noise types, and against a frozen table of
252
+ values from an independent implementation (no third-party stability library is needed);
253
+ - the confidence-interval coverage is checked by Monte Carlo;
254
+ - noise identification is checked for α = +2 … −2;
255
+ - parsers are checked on the line layouts from the ntpd and chrony documentation, and by
256
+ cross-checking that the same simulated peer reads identically from peerstats, rawstats and
257
+ chrony logs;
258
+ - the SNTP, NTPv5 and NTS clients are checked against local test servers (offset, delay, KoD,
259
+ spoofed replies, timeouts, 2036 rollover, TAI timescale, forged NTS responses, certificate
260
+ mismatch);
261
+ - the pcap/pcapng readers are checked on synthesised captures (VLAN tags, nanosecond
262
+ resolution, NTPv4 and v5 matching), and the linuxptp parser on the exact `pr_info` formats from
263
+ the linuxptp source;
264
+ - the web API is checked, including its CSRF, Host-header and path-traversal protections.
265
+
266
+ CI runs on Python 3.9, 3.11 and 3.13.
267
+
268
+ ## Security notes
269
+
270
+ The UI binds to `127.0.0.1`, rejects foreign `Host` headers (DNS rebinding), requires a custom
271
+ header on all state-changing requests (CSRF) and sends a strict Content-Security-Policy. It has no
272
+ authentication, so do not expose it with `--host 0.0.0.0` on untrusted networks. The SNTP client
273
+ is for measurement only: it does not authenticate time (for NTS use chrony or NTPsec and analyse
274
+ their logs).
275
+
276
+ ## Project layout
277
+
278
+ ```
279
+ src/ntpstats/ parsers, pcap, stability, edf, masks, analysis, network, filters, simulate,
280
+ estimators, bench, report, sntp (v4/v5), nts, sources (chronyc/ntpq),
281
+ monitor, cli, plotting
282
+ src/ntpstats/web stdlib HTTP server + static UI (uPlot vendored)
283
+ tests/ pytest suite (+ frozen reference data)
284
+ examples/ scripts and sample logs
285
+ docs/ state of the art, live interop results, example reports, screenshots
286
+ legacy/ the original 2012 GSoC code, untouched
287
+ ```
288
+
289
+ ## License and citation
290
+
291
+ MIT License. © 2012–2026 **Thiago de Freitas** <thiagodefreitas@gmail.com>. Free for commercial
292
+ and non-commercial use; please keep the copyright notice and credit the author (see
293
+ [NOTICE](NOTICE)). If you use it in research, please cite it via [CITATION.cff](CITATION.cff).
294
+ The bundled uPlot is MIT-licensed © Leon Sorokin. The code under `legacy/` keeps its original
295
+ 2012 license.