trace-digitiser 1.2.0__tar.gz → 2.0.0__tar.gz

trace_digitiser-2.0.0/CHANGELOG.md

@@ -0,0 +1,109 @@
+# Changelog
+
+All notable changes to trace-digitiser are recorded here.
+The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
+and the project uses [semantic versioning](https://semver.org/).
+
+## [2.0.0] - 2026-11-14
+
+Every call and command-line option of 1.2.0 still works, with the same
+meaning. The version number changes because the **results** change: figures
+are analysed in a new way, and the values that come out differ from those of
+1.2.0. By the package's own tests against known data they are much closer to
+what was drawn (55 of 57 test traces recovered to within 2 % with nothing
+supplied, against 20; see the README). Anything digitised with 1.x that
+matters should be digitised again.
+
+### Added
+
+- **Plot areas are found without being told the layout.** Axis lines are
+  located to a fraction of a pixel and assembled into plot areas: any number
+  of panels, boxed or L-shaped axes, with or without gridlines and tick
+  marks. `layout_mode`, `expected_rows` and the rest remain as hints and
+  checks.
+- **Tilted scans are straightened.** The tilt of the figure is measured and
+  undone first (`straighten="auto"`, `--straighten`), so a page that lay
+  askew on the scanner is digitised as accurately as one that did not.
+- **Each automatic step can be replaced by the answer.** `plot_area` for the
+  position of the axes, `y_ref` / `x_ref` for reference points on an axis,
+  `y_scale` / `x_scale` for logarithmic axes, `trace_colours` for the
+  colours to follow, `exclude` for legends and annotations, `achromatic` for
+  black and grey traces, `ocr=False` to do without Tesseract.
+- **The x axis can be calibrated** from `x_min` / `x_max`, from reference
+  points, or from numeric tick labels (`x_value` in the output).
+- **Axis scales read from tick labels come with a confidence rating**, and a
+  scale that too few labels agree on is declined rather than guessed. The
+  rating, and the labels the scale rests on, are in the panel metadata.
+- **Messages about anything that needs checking**: `result.warnings`,
+  `DigitiserWarning`, and `CHECK:` lines on the command line.
+- **An overlay image** of the result drawn over the figure (`save_overlay`,
+  `--overlay`, `result.overlay()`): the quickest way to see whether a
+  digitisation is right.
+- **A provenance file** for every run: version, SHA-256 of the image, all
+  settings, what was detected, and all messages.
+- **Validation tools** (`trace_digitiser.validation`): `round_trip` tests the
+  digitiser on your own data drawn at the size of your figure;
+  `run_validation` (`--validate`) and `run_axis_validation` test it on 44
+  figures and 750 axes drawn from known data.
+- New columns in the trace file: `n_pixels`, `interpolated`, `x_value`. New
+  columns at the end of the panel metadata: the sub-pixel plot area and how
+  each axis was calibrated. The 1.2.0 columns keep their names and order.
+- Image arrays are accepted as well as file names; transparent images are
+  placed on white, and photographs are turned according to their EXIF data.
+- `--version`; exit status 1 if any image could not be processed.
+- Examples: two scripts, a notebook, and a published figure (CC BY) that is
+  also used in the tests.
+- The repository itself: continuous testing on Python 3.10 to 3.14, and a
+  release workflow that publishes to PyPI through trusted publishing (see
+  `RELEASING.md`).
+
+### Changed
+
+- **Values differ from 1.2.0**, chiefly because:
+  - the plot area is now bounded by the centres of the axis lines; 1.2.0 put
+    its edges up to 13 pixels outside them when tick marks pointed outwards,
+    which shifted and rescaled every value read with `y_min` / `y_max`;
+  - traces are followed down onto the axis lines when their colour allows,
+    and thin, pale, grey and black traces are no longer missed;
+  - gridlines, text and compression fringes are no longer reported as
+    extra traces (1.2.0 did so in about half of single-trace test figures).
+- Axis labels are read in a new way (located geometrically, read by
+  Tesseract in several ways, and fitted on the principle that tick values
+  step by exactly the same amount). On the test axes it never gave a wrong
+  scale with high confidence; 1.2.0 misread decimal and logarithmic axes by
+  large factors without warning. It takes about half a second a figure
+  rather than twenty.
+- Tesseract is now optional: without it, give the axis limits.
+- Labels under 8 pixels high are read but never trusted outright; the
+  result carries a message asking for the values to be checked.
+- `mean_estimate` and `center_estimate` are documented for what they are
+  (statistics of the trace's pixels in a column), with the consequence for
+  dense records spelt out in the README.
+
+### Fixed
+
+- A one-pixel shift in the trace mask (from a 2×2 morphological opening)
+  that erased one-pixel lines and produced all-white "traces".
+- `tests/conftest.py` was missing from the 1.2.0 source distribution, so its
+  tests could not be run from it.
+- `use_lab=True` needed scikit-learn, and one helper needed SciPy, without
+  either being declared. scikit-learn is now the optional extra
+  `trace-digitiser[lab]`; SciPy is no longer used.
+- `eval` of text read back from a data frame replaced by `ast.literal_eval`.
+- The trace-accuracy test that failed in 1.2.0 (RMSE 102 against a limit of
+  50) passes, and a new one holds all eight example figures to 7.5.
+
+### Removed
+
+- The slow "exhaustive" reading of y labels is no longer tried by default
+  (`calibrate_y(..., exhaustive_fallback=True)` still does it). On axes that
+  the new method declined, it was right in none of twenty trials.
+
+## [1.2.0] - 2026-06-07
+
+First version tracked in this repository; identical to the 1.2.0 files on PyPI.
+Releases 0.1.0 to 1.0.0 (6-7 June 2026) were uploaded to PyPI directly and are
+not recorded here.
+
+[2.0.0]: https://github.com/comparativechrono/trace-digitiser/compare/1d34f75...v2.0.0
+[1.2.0]: https://pypi.org/project/trace-digitiser/1.2.0/

trace_digitiser-2.0.0/MANIFEST.in

@@ -0,0 +1,7 @@
+# Ship the complete test suite and the examples in the source distribution.
+# (Without this, setuptools includes tests/test_*.py but not conftest.py or
+# the reference data, so the tests in the sdist cannot run.)
+recursive-include tests *.py *.csv
+recursive-include examples *.py *.md *.jpg *.ipynb
+prune examples/output
+include CHANGELOG.md RELEASING.md

trace_digitiser-2.0.0/PKG-INFO

@@ -0,0 +1,296 @@
+Metadata-Version: 2.4
+Name: trace-digitiser
+Version: 2.0.0
+Summary: Recover the data from line plots in figure images, on your own computer: no uploads, no AI model.
+Author: Trace Digitiser Contributors
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/comparativechrono/trace-digitiser
+Project-URL: Repository, https://github.com/comparativechrono/trace-digitiser
+Project-URL: Issues, https://github.com/comparativechrono/trace-digitiser/issues
+Project-URL: Changelog, https://github.com/comparativechrono/trace-digitiser/blob/main/CHANGELOG.md
+Keywords: plot digitizer,graph digitiser,data extraction,data rescue,computer vision,time series,circadian,actigraphy
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Scientific/Engineering :: Image Processing
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: opencv-python-headless>=4.8
+Requires-Dist: pytesseract>=0.3.10
+Requires-Dist: Pillow>=10.0
+Requires-Dist: pandas>=2.0
+Requires-Dist: numpy>=1.24
+Requires-Dist: matplotlib>=3.7
+Provides-Extra: lab
+Requires-Dist: scikit-learn>=1.2; extra == "lab"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.4; extra == "dev"
+Requires-Dist: pytest-cov>=4.1; extra == "dev"
+Requires-Dist: ruff>=0.4; extra == "dev"
+Dynamic: license-file
+
+# trace-digitiser
+
+Get the numbers back out of a line plot when the figure is all that survives — on your own computer.
+
+`trace-digitiser` takes an image of a figure (a scan of an old paper, a figure saved from a PDF),
+finds the plot area, follows each trace, and writes the values to a CSV file. It is ordinary image
+processing, written to be read and checked.
+
+**Nothing is uploaded and no AI model is involved.** The only other program it runs is
+[Tesseract](https://github.com/tesseract-ocr/tesseract), also on your machine and optional, to read
+the numbers printed beside the axes. A figure that you are not free to pass to an online service
+can therefore still be digitised. (Whether the values in a published figure may be *reused* is a
+matter for the licence of the source and the law where you work; no tool can settle that for you.)
+
+A worked example with pictures is in
+[`examples/walkthrough.ipynb`](https://github.com/comparativechrono/trace-digitiser/blob/main/examples/walkthrough.ipynb).
+
+## Install
+
+```bash
+pip install trace-digitiser
+```
+
+Python 3.10 or later. To have axis values read from the figure, also install Tesseract
+(`sudo apt-get install tesseract-ocr`, `brew install tesseract`, or the
+[Windows installer](https://github.com/UB-Mannheim/tesseract/wiki)). Without it everything else
+works, and you type in the axis limits yourself — which is the better practice anyway.
+
+## Use
+
+From the command line:
+
+```bash
+trace-digitiser figure.png --y-min 0 --y-max 300 --x-min 0 --x-max 7 --overlay -o results/
+```
+
+or from Python:
+
+```python
+from trace_digitiser import digitise
+
+result = digitise("figure.png", y_min=0, y_max=300, x_min=0, x_max=7,
+                  output_dir="results", save_overlay=True)
+
+result.trace_data      # one row per pixel column of each trace
+result.warnings        # anything you should check
+result.overlay_path    # the digitised traces drawn over the figure
+```
+
+`y_min` and `y_max` are the values at the bottom and top axis lines, `x_min` and `x_max` those at
+the left and right. Nothing else is needed for most figures: the plot areas (any number of panels)
+and the traces (coloured, black or grey) are found automatically, and a scan that lies a few
+degrees askew is straightened first.
+
+Leave the limits out and the scale is read from the tick labels instead:
+
+```bash
+trace-digitiser figure.png --overlay -o results/
+```
+
+```
+  Panels: 2, Traces: 2
+    panel 1: y 0.31 to 299, read from tick labels (medium confidence); x not calibrated
+  CHECK: Panel 1: the y-axis scale was read from 7 of 7 tick labels (300, 250, 200, 150, 100, 50, 0);
+         the print is only 6 pixels high. Confidence in it is medium: check those values against
+         the figure, or give y_min and y_max.
+```
+
+A scale read from labels always comes with a statement of how far to trust it and, unless that is
+"high", of the labels it rests on. If too few labels can be read to agree on a scale, the axis is
+left uncalibrated (values from 0 to 1) and you are told so: the aim is never to pass off a wrong
+scale as a good one ([how well that is met](#how-accurate-is-it)).
+
+## Check the result
+
+1. **Look at the overlay** (`--overlay`). It shows the plot area that was found, the centre and
+   envelope of each digitised trace drawn over the original, and the positions of the axis values.
+   If the lines sit on the traces, the digitisation is right; if they do not, you can see where.
+2. **Read the messages.** Everything the program is unsure of is printed as a `CHECK:` line
+   (in Python: `result.warnings`, also raised as `DigitiserWarning`). A clean run prints none.
+3. **Keep the provenance file.** `<name>_automated_provenance.json` records the version, the
+   SHA-256 of the image, every setting, what was detected and all messages, so that a digitisation
+   can be reported and repeated exactly.
+
+## Output
+
+| File | Contents |
+|------|----------|
+| `<name>_automated_digitised_trace_by_column.csv` | One row per pixel column per trace (below). |
+| `<name>_automated_panel_metadata.csv` | Where each plot area is, its axis calibrations and how they were obtained. |
+| `<name>_automated_digitised_summary_by_label.csv` | Means per labelled interval, when the x axis carries interval names (`1N`, `1D`, …). |
+| `<name>_automated_provenance.json` | Everything needed to report and repeat the run. |
+| `<name>_automated_overlay.png` | With `--overlay`: the result drawn over the figure. |
+| `<name>_automated_straightened.png` | Only if the figure was tilted: the image turned upright, which is what pixel positions in the other files refer to. |
+
+Columns of the trace file:
+
+| Column | Meaning |
+|--------|---------|
+| `panel_id`, `trace_id` | Panels are numbered row by row from the top left; traces within a panel from 1. |
+| `x_pixel`, `x_norm_0_to_1` | The pixel column, and its position between the left (0) and right (1) axis lines. |
+| `x_value` | The x value, if the x axis is calibrated. |
+| `center_estimate` | Middle of the trace's ink in this column (median of its pixels). |
+| `mean_estimate` | Mean of the trace's pixels in this column. |
+| `upper_envelope`, `lower_envelope` | Top and bottom of the trace's ink in this column. |
+| `n_pixels`, `interpolated` | How many pixels the trace has in this column; `True` if it has none and the values were interpolated from its neighbours. |
+| `scale_type` | Where the y values come from: `user_supplied`, `ocr_value`, `normalised_0_to_1`, … |
+| `trace_median_rgb`, `trace_pixel_count` | The trace's colour and size, for identifying it. |
+
+`result.resample(...)` and `--sampling-interval` put the trace on an even grid in x.
+
+## What the values are — and are not
+
+A digitiser recovers what was **drawn**, not what was measured.
+
+* For an ordinary curve — a thin line with at most a few samples per pixel column —
+  `center_estimate` is the plotted value, to within a fraction of a pixel.
+* For a **dense record**, where many samples share each pixel column and the trace is drawn as a
+  solid band (an activity record of several days in a small figure, for instance), the drawing
+  shows only the *range* of the samples in each column. `upper_envelope` and `lower_envelope`
+  recover that range; `center_estimate` is its middle. **None of them is the mean of the samples**,
+  and for spiky data the middle of the range can be far from the mean. The timing of the rhythm
+  survives; the level does not.
+
+How much this matters for a particular kind of data and size of figure can be measured rather than
+guessed: see [Test it on data you know](#test-it-on-data-you-know).
+
+Positions cannot be better than the image. In a plot area 350 pixels wide showing seven days, one
+pixel is half an hour.
+
+## When something is not found
+
+Each automatic step can be replaced by telling the program the answer. Pixel positions count from
+the top-left corner of the image, starting at 0; any image viewer shows them.
+
+| Problem | Option | Python |
+|---------|--------|--------|
+| The figure is tilted and was not straightened, or was turned when it should not have been | `--straighten DEGREES` (its anticlockwise tilt), `--straighten never` | `straighten=1.3`, `straighten=False` |
+| The plot area is wrong or not found | `--plot-area LEFT,TOP,RIGHT,BOTTOM` (repeat per panel) | `plot_area=(l, t, r, b)` |
+| The wrong number of panels | `--layout`, `--rows`, `--cols`, `--panels` | `layout_mode`, `expected_rows`, … |
+| The axis continues beyond its last tick, or has no line at the top or right | `--y-ref ROW:VALUE` twice or more (also `--x-ref`) | `y_ref=[(row, value), …]` |
+| A logarithmic axis, with limits given | `--y-scale log` | `y_scale="log"` |
+| A trace is missed, or two are merged | `--trace-colour RRGGBB` (repeat per trace) | `trace_colours=["#578dd2", …]` |
+| A legend or annotation is taken for a trace | `--exclude LEFT,TOP,RIGHT,BOTTOM` | `exclude=(l, t, r, b)` |
+| A black trace is invented or ignored | `--achromatic never` / `always` | `achromatic="never"` |
+| No Tesseract, or it should not be used | `--no-ocr` | `ocr=False` |
+
+## How accurate is it?
+
+The package carries its own tests against known answers, and you can rerun them.
+
+**Traces.** `trace-digitiser --validate results/` draws 44 figures (57 traces) from known data —
+different axis styles, twelve colours including black and grey, 60 to 300 dpi, JPEG compression,
+noise and blur, line widths, multi-panel layouts, several traces per panel, a logarithmic axis,
+jagged and dense records, tilted scans — digitises them, and compares. Errors are in percent of the
+height of the plot. A trace counts as *recovered* if its panel and curve were found with no
+spurious extra trace, x positions are within 1 % and the RMS error in y is within 2 %.
+
+| | found | recovered | median error | time per figure¹ |
+|---|---|---|---|---|
+| **2.0, nothing supplied** | 57 of 57 | 55 | 0.13 % | 0.5 s |
+| **2.0, axis limits supplied** | 57 of 57 | 55 | 0.09 % | 0.3 s |
+| 1.2.0, panel layout supplied | 48 of 57 | 20 | 0.49 % | 20 s |
+| 1.2.0, panel layout and axis limits supplied | 48 of 57 | 13 | 1.39 % | 0.3 s |
+
+¹ On a two-core machine.
+
+The two traces that 2.0 does not recover are dense records in low-resolution figures (errors of
+2.3 % and 4.3 %), for the reason given [above](#what-the-values-are--and-are-not). Of the traces
+that 1.2.0 found but did not recover, most came with a spurious second trace.
+
+**Axis scales.** Reading a scale from tick labels is the one step that depends on OCR, and OCR of
+small print is unreliable, so it is tested separately:
+`python -m trace_digitiser.validation results/ --axes` reads 750 axes (whole numbers, decimals,
+negative values, thousands, clock and angle steps, logarithmic; print from 5 to 24 pixels high;
+three typefaces; JPEG compression and blur).
+
+| Height of the labels | axes | read correctly | declined | read wrongly, with a warning | read wrongly, no warning |
+|---|---|---|---|---|---|
+| 8 pixels or more | 477 | 464 | 13 | 0 | 0 |
+| under 8 pixels | 273 | 257 (all with a warning to check them) | 15 | 1 | 0 |
+
+*Declined* means that no scale was offered. All of those at 8 pixels or more are logarithmic axes
+whose raised exponents are too small to read, though the rest of the label is not. The outcome for small
+print depends on how the text was rasterised: with the test figures drawn by matplotlib 3.7, which
+renders small print more softly than current versions, 236 of 292 axes under 8 pixels were read
+correctly, 46 declined and 10 read wrongly with a warning — and still none without.
+
+Labels under 8 pixels high are never given high confidence, however well they agree, because print
+that small is misread in consistent ways — the decimal point dropped from every label, say. If
+your figure has such labels, enlarging the image does not help; typing in two numbers does.
+
+### Test it on data you know
+
+The best test of a digitisation is one on data like yours. `round_trip` draws a known series as a
+figure of whatever size and style you choose, digitises the drawing, and reports the errors:
+
+```python
+from trace_digitiser.validation import round_trip
+
+check = round_trip(t, y, "check/", xlim=(0, 7), ylim=(0, 300),
+                   panel_px=(347, 137), dpi=72, linewidth=0.8, fmt="jpg")
+check["metrics"]     # errors of centre and envelopes, correlation with the mean per column, …
+check["overlay"]     # the picture to look at
+```
+
+[`examples/validate_with_known_data.py`](https://github.com/comparativechrono/trace-digitiser/blob/main/examples/validate_with_known_data.py)
+does this for a week of minute-by-minute activity drawn at the size of a small journal figure.
+
+## Limits
+
+* The plot must have at least a left and a bottom axis line, drawn in black or grey. A figure
+  that is askew is straightened automatically if the tilt is under five degrees; beyond that,
+  or for a photograph taken at an angle (where the axes are no longer parallel), correct the
+  image first.
+* One trace per colour per panel. Two traces of the same colour, dashed or dotted lines, and
+  traces that are only markers are not separated or joined up.
+* Where traces cross or overlap, the one drawn underneath is hidden and its values there are
+  interpolated (`interpolated` is `True`).
+* Axis scales are read only from numeric labels on linear or logarithmic axes; dates, broken axes
+  and a second y axis are not understood. Limits typed in are assumed to be at the axis lines
+  unless `y_ref` / `x_ref` say otherwise.
+* Error bars, shaded bands, bars and scatter plots are not digitised.
+
+## Development
+
+```bash
+git clone https://github.com/comparativechrono/trace-digitiser.git
+cd trace-digitiser
+pip install -e ".[dev]"
+pytest                      # the tests that need Tesseract are skipped without it
+ruff check src tests
+```
+
+`src/trace_digitiser/` in the order the work is done:
+
+| Module | Does |
+|--------|------|
+| `straighten.py` | Measures the tilt of a scan and turns it upright. |
+| `frames.py`, `panel_detection.py` | Finds axis lines and assembles them into plot areas. |
+| `axis_labels.py`, `calibration.py`, `x_calibration.py` | Locates and reads tick labels, fits the scale, rates its confidence; or takes the limits given. |
+| `trace_detection.py` | Separates the traces from gridlines, axes and text. |
+| `digitise.py` | Converts each trace to values, column by column. |
+| `ocr.py`, `summarise.py` | Interval labels along the x axis, and means per interval. |
+| `diagnostics.py`, `provenance.py`, `io.py` | Overlay, record of the run, files. |
+| `validation.py`, `synthetic.py` | Figures drawn from known data, and the comparisons. |
+| `cli.py` | The command line. |
+
+Changes are listed in [CHANGELOG.md](https://github.com/comparativechrono/trace-digitiser/blob/main/CHANGELOG.md);
+making a release is described in
+[RELEASING.md](https://github.com/comparativechrono/trace-digitiser/blob/main/RELEASING.md).
+
+## Licence
+
+MIT; see `LICENSE`. The example figure has its own licence and attribution, given in
+[`examples/README.md`](https://github.com/comparativechrono/trace-digitiser/blob/main/examples/README.md).

trace_digitiser-2.0.0/README.md

@@ -0,0 +1,256 @@
+# trace-digitiser
+
+Get the numbers back out of a line plot when the figure is all that survives — on your own computer.
+
+`trace-digitiser` takes an image of a figure (a scan of an old paper, a figure saved from a PDF),
+finds the plot area, follows each trace, and writes the values to a CSV file. It is ordinary image
+processing, written to be read and checked.
+
+**Nothing is uploaded and no AI model is involved.** The only other program it runs is
+[Tesseract](https://github.com/tesseract-ocr/tesseract), also on your machine and optional, to read
+the numbers printed beside the axes. A figure that you are not free to pass to an online service
+can therefore still be digitised. (Whether the values in a published figure may be *reused* is a
+matter for the licence of the source and the law where you work; no tool can settle that for you.)
+
+A worked example with pictures is in
+[`examples/walkthrough.ipynb`](https://github.com/comparativechrono/trace-digitiser/blob/main/examples/walkthrough.ipynb).
+
+## Install
+
+```bash
+pip install trace-digitiser
+```
+
+Python 3.10 or later. To have axis values read from the figure, also install Tesseract
+(`sudo apt-get install tesseract-ocr`, `brew install tesseract`, or the
+[Windows installer](https://github.com/UB-Mannheim/tesseract/wiki)). Without it everything else
+works, and you type in the axis limits yourself — which is the better practice anyway.
+
+## Use
+
+From the command line:
+
+```bash
+trace-digitiser figure.png --y-min 0 --y-max 300 --x-min 0 --x-max 7 --overlay -o results/
+```
+
+or from Python:
+
+```python
+from trace_digitiser import digitise
+
+result = digitise("figure.png", y_min=0, y_max=300, x_min=0, x_max=7,
+                  output_dir="results", save_overlay=True)
+
+result.trace_data      # one row per pixel column of each trace
+result.warnings        # anything you should check
+result.overlay_path    # the digitised traces drawn over the figure
+```
+
+`y_min` and `y_max` are the values at the bottom and top axis lines, `x_min` and `x_max` those at
+the left and right. Nothing else is needed for most figures: the plot areas (any number of panels)
+and the traces (coloured, black or grey) are found automatically, and a scan that lies a few
+degrees askew is straightened first.
+
+Leave the limits out and the scale is read from the tick labels instead:
+
+```bash
+trace-digitiser figure.png --overlay -o results/
+```
+
+```
+  Panels: 2, Traces: 2
+    panel 1: y 0.31 to 299, read from tick labels (medium confidence); x not calibrated
+  CHECK: Panel 1: the y-axis scale was read from 7 of 7 tick labels (300, 250, 200, 150, 100, 50, 0);
+         the print is only 6 pixels high. Confidence in it is medium: check those values against
+         the figure, or give y_min and y_max.
+```
+
+A scale read from labels always comes with a statement of how far to trust it and, unless that is
+"high", of the labels it rests on. If too few labels can be read to agree on a scale, the axis is
+left uncalibrated (values from 0 to 1) and you are told so: the aim is never to pass off a wrong
+scale as a good one ([how well that is met](#how-accurate-is-it)).
+
+## Check the result
+
+1. **Look at the overlay** (`--overlay`). It shows the plot area that was found, the centre and
+   envelope of each digitised trace drawn over the original, and the positions of the axis values.
+   If the lines sit on the traces, the digitisation is right; if they do not, you can see where.
+2. **Read the messages.** Everything the program is unsure of is printed as a `CHECK:` line
+   (in Python: `result.warnings`, also raised as `DigitiserWarning`). A clean run prints none.
+3. **Keep the provenance file.** `<name>_automated_provenance.json` records the version, the
+   SHA-256 of the image, every setting, what was detected and all messages, so that a digitisation
+   can be reported and repeated exactly.
+
+## Output
+
+| File | Contents |
+|------|----------|
+| `<name>_automated_digitised_trace_by_column.csv` | One row per pixel column per trace (below). |
+| `<name>_automated_panel_metadata.csv` | Where each plot area is, its axis calibrations and how they were obtained. |
+| `<name>_automated_digitised_summary_by_label.csv` | Means per labelled interval, when the x axis carries interval names (`1N`, `1D`, …). |
+| `<name>_automated_provenance.json` | Everything needed to report and repeat the run. |
+| `<name>_automated_overlay.png` | With `--overlay`: the result drawn over the figure. |
+| `<name>_automated_straightened.png` | Only if the figure was tilted: the image turned upright, which is what pixel positions in the other files refer to. |
+
+Columns of the trace file:
+
+| Column | Meaning |
+|--------|---------|
+| `panel_id`, `trace_id` | Panels are numbered row by row from the top left; traces within a panel from 1. |
+| `x_pixel`, `x_norm_0_to_1` | The pixel column, and its position between the left (0) and right (1) axis lines. |
+| `x_value` | The x value, if the x axis is calibrated. |
+| `center_estimate` | Middle of the trace's ink in this column (median of its pixels). |
+| `mean_estimate` | Mean of the trace's pixels in this column. |
+| `upper_envelope`, `lower_envelope` | Top and bottom of the trace's ink in this column. |
+| `n_pixels`, `interpolated` | How many pixels the trace has in this column; `True` if it has none and the values were interpolated from its neighbours. |
+| `scale_type` | Where the y values come from: `user_supplied`, `ocr_value`, `normalised_0_to_1`, … |
+| `trace_median_rgb`, `trace_pixel_count` | The trace's colour and size, for identifying it. |
+
+`result.resample(...)` and `--sampling-interval` put the trace on an even grid in x.
+
+## What the values are — and are not
+
+A digitiser recovers what was **drawn**, not what was measured.
+
+* For an ordinary curve — a thin line with at most a few samples per pixel column —
+  `center_estimate` is the plotted value, to within a fraction of a pixel.
+* For a **dense record**, where many samples share each pixel column and the trace is drawn as a
+  solid band (an activity record of several days in a small figure, for instance), the drawing
+  shows only the *range* of the samples in each column. `upper_envelope` and `lower_envelope`
+  recover that range; `center_estimate` is its middle. **None of them is the mean of the samples**,
+  and for spiky data the middle of the range can be far from the mean. The timing of the rhythm
+  survives; the level does not.
+
+How much this matters for a particular kind of data and size of figure can be measured rather than
+guessed: see [Test it on data you know](#test-it-on-data-you-know).
+
+Positions cannot be better than the image. In a plot area 350 pixels wide showing seven days, one
+pixel is half an hour.
+
+## When something is not found
+
+Each automatic step can be replaced by telling the program the answer. Pixel positions count from
+the top-left corner of the image, starting at 0; any image viewer shows them.
+
+| Problem | Option | Python |
+|---------|--------|--------|
+| The figure is tilted and was not straightened, or was turned when it should not have been | `--straighten DEGREES` (its anticlockwise tilt), `--straighten never` | `straighten=1.3`, `straighten=False` |
+| The plot area is wrong or not found | `--plot-area LEFT,TOP,RIGHT,BOTTOM` (repeat per panel) | `plot_area=(l, t, r, b)` |
+| The wrong number of panels | `--layout`, `--rows`, `--cols`, `--panels` | `layout_mode`, `expected_rows`, … |
+| The axis continues beyond its last tick, or has no line at the top or right | `--y-ref ROW:VALUE` twice or more (also `--x-ref`) | `y_ref=[(row, value), …]` |
+| A logarithmic axis, with limits given | `--y-scale log` | `y_scale="log"` |
+| A trace is missed, or two are merged | `--trace-colour RRGGBB` (repeat per trace) | `trace_colours=["#578dd2", …]` |
+| A legend or annotation is taken for a trace | `--exclude LEFT,TOP,RIGHT,BOTTOM` | `exclude=(l, t, r, b)` |
+| A black trace is invented or ignored | `--achromatic never` / `always` | `achromatic="never"` |
+| No Tesseract, or it should not be used | `--no-ocr` | `ocr=False` |
+
+## How accurate is it?
+
+The package carries its own tests against known answers, and you can rerun them.
+
+**Traces.** `trace-digitiser --validate results/` draws 44 figures (57 traces) from known data —
+different axis styles, twelve colours including black and grey, 60 to 300 dpi, JPEG compression,
+noise and blur, line widths, multi-panel layouts, several traces per panel, a logarithmic axis,
+jagged and dense records, tilted scans — digitises them, and compares. Errors are in percent of the
+height of the plot. A trace counts as *recovered* if its panel and curve were found with no
+spurious extra trace, x positions are within 1 % and the RMS error in y is within 2 %.
+
+| | found | recovered | median error | time per figure¹ |
+|---|---|---|---|---|
+| **2.0, nothing supplied** | 57 of 57 | 55 | 0.13 % | 0.5 s |
+| **2.0, axis limits supplied** | 57 of 57 | 55 | 0.09 % | 0.3 s |
+| 1.2.0, panel layout supplied | 48 of 57 | 20 | 0.49 % | 20 s |
+| 1.2.0, panel layout and axis limits supplied | 48 of 57 | 13 | 1.39 % | 0.3 s |
+
+¹ On a two-core machine.
+
+The two traces that 2.0 does not recover are dense records in low-resolution figures (errors of
+2.3 % and 4.3 %), for the reason given [above](#what-the-values-are--and-are-not). Of the traces
+that 1.2.0 found but did not recover, most came with a spurious second trace.
+
+**Axis scales.** Reading a scale from tick labels is the one step that depends on OCR, and OCR of
+small print is unreliable, so it is tested separately:
+`python -m trace_digitiser.validation results/ --axes` reads 750 axes (whole numbers, decimals,
+negative values, thousands, clock and angle steps, logarithmic; print from 5 to 24 pixels high;
+three typefaces; JPEG compression and blur).
+
+| Height of the labels | axes | read correctly | declined | read wrongly, with a warning | read wrongly, no warning |
+|---|---|---|---|---|---|
+| 8 pixels or more | 477 | 464 | 13 | 0 | 0 |
+| under 8 pixels | 273 | 257 (all with a warning to check them) | 15 | 1 | 0 |
+
+*Declined* means that no scale was offered. All of those at 8 pixels or more are logarithmic axes
+whose raised exponents are too small to read, though the rest of the label is not. The outcome for small
+print depends on how the text was rasterised: with the test figures drawn by matplotlib 3.7, which
+renders small print more softly than current versions, 236 of 292 axes under 8 pixels were read
+correctly, 46 declined and 10 read wrongly with a warning — and still none without.
+
+Labels under 8 pixels high are never given high confidence, however well they agree, because print
+that small is misread in consistent ways — the decimal point dropped from every label, say. If
+your figure has such labels, enlarging the image does not help; typing in two numbers does.
+
+### Test it on data you know
+
+The best test of a digitisation is one on data like yours. `round_trip` draws a known series as a
+figure of whatever size and style you choose, digitises the drawing, and reports the errors:
+
+```python
+from trace_digitiser.validation import round_trip
+
+check = round_trip(t, y, "check/", xlim=(0, 7), ylim=(0, 300),
+                   panel_px=(347, 137), dpi=72, linewidth=0.8, fmt="jpg")
+check["metrics"]     # errors of centre and envelopes, correlation with the mean per column, …
+check["overlay"]     # the picture to look at
+```
+
+[`examples/validate_with_known_data.py`](https://github.com/comparativechrono/trace-digitiser/blob/main/examples/validate_with_known_data.py)
+does this for a week of minute-by-minute activity drawn at the size of a small journal figure.
+
+## Limits
+
+* The plot must have at least a left and a bottom axis line, drawn in black or grey. A figure
+  that is askew is straightened automatically if the tilt is under five degrees; beyond that,
+  or for a photograph taken at an angle (where the axes are no longer parallel), correct the
+  image first.
+* One trace per colour per panel. Two traces of the same colour, dashed or dotted lines, and
+  traces that are only markers are not separated or joined up.
+* Where traces cross or overlap, the one drawn underneath is hidden and its values there are
+  interpolated (`interpolated` is `True`).
+* Axis scales are read only from numeric labels on linear or logarithmic axes; dates, broken axes
+  and a second y axis are not understood. Limits typed in are assumed to be at the axis lines
+  unless `y_ref` / `x_ref` say otherwise.
+* Error bars, shaded bands, bars and scatter plots are not digitised.
+
+## Development
+
+```bash
+git clone https://github.com/comparativechrono/trace-digitiser.git
+cd trace-digitiser
+pip install -e ".[dev]"
+pytest                      # the tests that need Tesseract are skipped without it
+ruff check src tests
+```
+
+`src/trace_digitiser/` in the order the work is done:
+
+| Module | Does |
+|--------|------|
+| `straighten.py` | Measures the tilt of a scan and turns it upright. |
+| `frames.py`, `panel_detection.py` | Finds axis lines and assembles them into plot areas. |
+| `axis_labels.py`, `calibration.py`, `x_calibration.py` | Locates and reads tick labels, fits the scale, rates its confidence; or takes the limits given. |
+| `trace_detection.py` | Separates the traces from gridlines, axes and text. |
+| `digitise.py` | Converts each trace to values, column by column. |
+| `ocr.py`, `summarise.py` | Interval labels along the x axis, and means per interval. |
+| `diagnostics.py`, `provenance.py`, `io.py` | Overlay, record of the run, files. |
+| `validation.py`, `synthetic.py` | Figures drawn from known data, and the comparisons. |
+| `cli.py` | The command line. |
+
+Changes are listed in [CHANGELOG.md](https://github.com/comparativechrono/trace-digitiser/blob/main/CHANGELOG.md);
+making a release is described in
+[RELEASING.md](https://github.com/comparativechrono/trace-digitiser/blob/main/RELEASING.md).
+
+## Licence
+
+MIT; see `LICENSE`. The example figure has its own licence and attribution, given in
+[`examples/README.md`](https://github.com/comparativechrono/trace-digitiser/blob/main/examples/README.md).