trace-digitiser 2.0.0__tar.gz → 2.2.0__tar.gz

trace_digitiser-2.2.0/CHANGELOG.md

@@ -0,0 +1,228 @@
+# Changelog
+
+All notable changes to trace-digitiser are recorded here.
+The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
+and the project uses [semantic versioning](https://semver.org/).
+
+## [2.2.0] - 2026-11-15
+
+Tested on the figures of published papers whose underlying numbers are
+published too (Leypunskiy et al. 2017, eLife 6:e23539: markers with error
+bars, a line over shaded bands; Foley et al. 2019, eLife 8:e50063: bar charts;
+Pearl and Reed 1920, PNAS 6:275, a microfilm scan), 2.1.0 read what it found
+accurately but often found nothing. This release finds much more of it, there
+and on the figures of a paper held out until the changes were made (Andreani
+et al. 2022, eLife 11:e74327). Line plots digitise as before: the package's
+validation suite and the Hearn and Whitmore (2026) examples give the same
+results.
+
+### Added
+
+- **A test set of published figures**
+  ([`validation/published-figures`](https://github.com/comparativechrono/trace-digitiser/tree/main/validation/published-figures)):
+  seven figures from those four papers, the numbers behind them from their
+  authors' data, and a script that reads them with nothing given and with the
+  help a user can give, and compares. Its README has the results of 2.1.0 and
+  2.2.0, case by case, including the few that 2.2.0 reads worse. It is run
+  before each release and is not part of the package.
+
+### Fixed
+
+- **Bar charts in the style of GraphPad Prism**: black, grey, hatched and
+  outlined bars are found, on an axis that does not start at 0 (2.1.0 found 3
+  of the 35 bars of Foley et al.'s figure 2, and read those upside down; all
+  35 are now read, within 0.3 % of the axis). Bars are the columns filled from
+  the baseline, so an error bar or a significance bracket joined to a bar no
+  longer spoils it; a border of another colour (a black outline round a grey
+  bar) is part of the bar; a bar cut by the edge of the plot area is kept.
+- **Error bars drawn in the colour of their bar or marker** (the default in
+  Prism, MATLAB and matplotlib) are followed beyond the bar's end, or from the
+  marker, to their caps; one running across a fitted line goes on beyond it.
+  Where only the half beyond a bar's end shows, a note says so.
+- **Markers sitting on a line of their own colour** (a fitted line, their
+  error bars) are found: other sizes are tried where the lines' ridges
+  outnumber the markers, and such a colour counts as a colour of markers if
+  its markers are joined to its lines and much thicker than them. Round
+  letters of words written in the plot, and a legend's markers beside their
+  words, are left out; a marker found in two series of close colours is kept
+  in the nearer one.
+- **A plot area given by hand** (a box drawn in the workbench, or numbers read
+  off a picture) moves to the middle of the axis lines within a few pixels of
+  it, and their thickness is measured: with a thick axis line the tick labels
+  were looked for inside it, and the axis title was read instead.
+- Long tick marks of a small plot in a large picture are found, so that its
+  labels are anchored to them.
+- A colour that fills areas (a shaded band behind a plot) is not taken for a
+  trace of a line plot.
+- Tick labels that Tesseract misreads as a line of text (75 as 15, in large
+  Helvetica) are read once more as a single word when an axis is not settled.
+- Looking at a whole page of a scanned paper (`describe_figure`, the
+  workbench's first look at a page of a PDF) took a minute and a half, nearly
+  all of it in the search for polar axes; the circles are now proposed on a
+  smaller copy of the page (16 seconds in all).
+
+## [2.1.0] - 2026-11-14
+
+Everything in 2.0.0 works as before. What is new is readers for other kinds of
+figure, and helpers for graphical tools such as the
+[digitiser workbench](https://github.com/comparativechrono/digitiser-workbench).
+
+### Added
+
+- **Points**: `digitise_points` (`trace-digitiser points`) reads the markers of
+  a scatter or dot plot, or of a line drawn with markers, whatever their
+  shape, separating markers that overlap; and the error bars on them,
+  vertical or horizontal, to the middle of their caps.
+- **Bars**: `digitise_bars` (`trace-digitiser bars`) reads where each bar
+  ends, upright or lying down, rising or hanging from the baseline, and its
+  error bar; a legend's patch is left out.
+- **Rows**: `digitise_rows` (`trace-digitiser rows`) reads figures drawn as a
+  stack of rows - actograms, double-plotted records, birthograms - bin by bin,
+  and the dots that mark a peak in them.
+- **Heat maps**: `digitise_heatmap` (`trace-digitiser heatmap`) reads each
+  cell's value from its colour on the colour bar, whose labels are read like
+  an axis's (or from a matplotlib colour map).
+- **Polar plots**: `digitise_polar` (`trace-digitiser polar`) finds the polar
+  axes and their rings, and reads each point's angle and distance.
+- All of them return a `FigureResult` and write a CSV file, a provenance file
+  and, if asked, an overlay, as `digitise` does.
+- `describe_figure` (`trace-digitiser describe`): what the automatic steps
+  find in a figure - plot areas, each axis's tick labels and readings, the
+  colours drawn, a colour bar, polar axes - as plain values.
+- `pixels_to_values`: points picked by hand, in the figure's units.
+- Tick labels are read on the right-hand side of a plot too (a colour bar, a
+  second y axis): `calibrate_axis(..., "y2")`.
+- `y_axis="right"` (`--y-axis right`) reads a line's values against a second
+  y axis on the right of the plot (a twin axis); with `trace_colours`, the
+  lines drawn against each axis are digitised in a run of their own.
+- `describe_figure` gives the tick marks found on each plot's axes, for a
+  program that lets you click on a tick.
+- Colours can be given by name (`"black"`, `"steelblue"`, `"tab:blue"`).
+- Plots stacked on a shared x axis, of which only the lowest has labels, take
+  its x scale (with a message saying so).
+- A plot area hidden among others in a grid (a heat map whose dark colours
+  hide its axis lines) is found by its frame.
+
+### Changed
+
+- An axis scale is held to medium confidence when the tick labels support a
+  different scale just as well. Tesseract compiled to WebAssembly (as it runs
+  in a browser) read the superscript minus signs of a log axis's exponents
+  differently from the Tesseract program, and two scales then fitted equally;
+  between log scales that tie, the one that steps by a decade is chosen.
+- The sides of filled areas (bars) no longer make a plot area of their own
+  inside the real one, which bar charts with a legend could cause.
+- A long axis (ten or more labels) whose labels nearly all agree is not read
+  again in the slower ways; fitting the labels of a long axis (the 59 years of
+  a birthogram) takes a second or two rather than half a minute, with the same
+  result.
+- A heat map's colours are looked up on the colour bar once each, rather than
+  once for each pixel: Figure 4 of Hearn and Whitmore (2026) takes 4 seconds
+  rather than 45, with the same values.
+- pytesseract is imported only when it is first used.
+
+## [2.0.0] - 2026-11-14
+
+Every call and command-line option of 1.2.0 still works, with the same
+meaning. The version number changes because the **results** change: figures
+are analysed in a new way, and the values that come out differ from those of
+1.2.0. By the package's own tests against known data they are much closer to
+what was drawn (55 of 57 test traces recovered to within 2 % with nothing
+supplied, against 20; see the README). Anything digitised with 1.x that
+matters should be digitised again.
+
+### Added
+
+- **Plot areas are found without being told the layout.** Axis lines are
+  located to a fraction of a pixel and assembled into plot areas: any number
+  of panels, boxed or L-shaped axes, with or without gridlines and tick
+  marks. `layout_mode`, `expected_rows` and the rest remain as hints and
+  checks.
+- **Tilted scans are straightened.** The tilt of the figure is measured and
+  undone first (`straighten="auto"`, `--straighten`), so a page that lay
+  askew on the scanner is digitised as accurately as one that did not.
+- **Each automatic step can be replaced by the answer.** `plot_area` for the
+  position of the axes, `y_ref` / `x_ref` for reference points on an axis,
+  `y_scale` / `x_scale` for logarithmic axes, `trace_colours` for the
+  colours to follow, `exclude` for legends and annotations, `achromatic` for
+  black and grey traces, `ocr=False` to do without Tesseract.
+- **The x axis can be calibrated** from `x_min` / `x_max`, from reference
+  points, or from numeric tick labels (`x_value` in the output).
+- **Axis scales read from tick labels come with a confidence rating**, and a
+  scale that too few labels agree on is declined rather than guessed. The
+  rating, and the labels the scale rests on, are in the panel metadata.
+- **Messages about anything that needs checking**: `result.warnings`,
+  `DigitiserWarning`, and `CHECK:` lines on the command line.
+- **An overlay image** of the result drawn over the figure (`save_overlay`,
+  `--overlay`, `result.overlay()`): the quickest way to see whether a
+  digitisation is right.
+- **A provenance file** for every run: version, SHA-256 of the image, all
+  settings, what was detected, and all messages.
+- **Validation tools** (`trace_digitiser.validation`): `round_trip` tests the
+  digitiser on your own data drawn at the size of your figure;
+  `run_validation` (`--validate`) and `run_axis_validation` test it on 44
+  figures and 750 axes drawn from known data.
+- New columns in the trace file: `n_pixels`, `interpolated`, `x_value`. New
+  columns at the end of the panel metadata: the sub-pixel plot area and how
+  each axis was calibrated. The 1.2.0 columns keep their names and order.
+- Image arrays are accepted as well as file names; transparent images are
+  placed on white, and photographs are turned according to their EXIF data.
+- `--version`; exit status 1 if any image could not be processed.
+- Examples: two scripts, a notebook, and a published figure (CC BY) that is
+  also used in the tests.
+- The repository itself: continuous testing on Python 3.10 to 3.14, and a
+  release workflow that publishes to PyPI through trusted publishing (see
+  `RELEASING.md`).
+
+### Changed
+
+- **Values differ from 1.2.0**, chiefly because:
+  - the plot area is now bounded by the centres of the axis lines; 1.2.0 put
+    its edges up to 13 pixels outside them when tick marks pointed outwards,
+    which shifted and rescaled every value read with `y_min` / `y_max`;
+  - traces are followed down onto the axis lines when their colour allows,
+    and thin, pale, grey and black traces are no longer missed;
+  - gridlines, text and compression fringes are no longer reported as
+    extra traces (1.2.0 did so in about half of single-trace test figures).
+- Axis labels are read in a new way (located geometrically, read by
+  Tesseract in several ways, and fitted on the principle that tick values
+  step by exactly the same amount). On the test axes it never gave a wrong
+  scale with high confidence; 1.2.0 misread decimal and logarithmic axes by
+  large factors without warning. It takes about half a second a figure
+  rather than twenty.
+- Tesseract is now optional: without it, give the axis limits.
+- Labels under 8 pixels high are read but never trusted outright; the
+  result carries a message asking for the values to be checked.
+- `mean_estimate` and `center_estimate` are documented for what they are
+  (statistics of the trace's pixels in a column), with the consequence for
+  dense records spelt out in the README.
+
+### Fixed
+
+- A one-pixel shift in the trace mask (from a 2×2 morphological opening)
+  that erased one-pixel lines and produced all-white "traces".
+- `tests/conftest.py` was missing from the 1.2.0 source distribution, so its
+  tests could not be run from it.
+- `use_lab=True` needed scikit-learn, and one helper needed SciPy, without
+  either being declared. scikit-learn is now the optional extra
+  `trace-digitiser[lab]`; SciPy is no longer used.
+- `eval` of text read back from a data frame replaced by `ast.literal_eval`.
+- The trace-accuracy test that failed in 1.2.0 (RMSE 102 against a limit of
+  50) passes, and a new one holds all eight example figures to 7.5.
+
+### Removed
+
+- The slow "exhaustive" reading of y labels is no longer tried by default
+  (`calibrate_y(..., exhaustive_fallback=True)` still does it). On axes that
+  the new method declined, it was right in none of twenty trials.
+
+## [1.2.0] - 2026-06-07
+
+First version tracked in this repository; identical to the 1.2.0 files on PyPI.
+Releases 0.1.0 to 1.0.0 (6-7 June 2026) were uploaded to PyPI directly and are
+not recorded here.
+
+[2.2.0]: https://github.com/comparativechrono/trace-digitiser/compare/v2.1.0...v2.2.0
+[2.1.0]: https://github.com/comparativechrono/trace-digitiser/compare/v2.0.0...v2.1.0
+[2.0.0]: https://github.com/comparativechrono/trace-digitiser/compare/1d34f75...v2.0.0
+[1.2.0]: https://pypi.org/project/trace-digitiser/1.2.0/

{trace_digitiser-2.0.0 → trace_digitiser-2.2.0}/MANIFEST.in

@@ -5,3 +5,5 @@ recursive-include tests *.py *.csv
 recursive-include examples *.py *.md *.jpg *.ipynb
 prune examples/output
 include CHANGELOG.md RELEASING.md
+# The published-figure test set (validation/published-figures) is run from the repository, not shipped.
+prune validation

{trace_digitiser-2.0.0/src/trace_digitiser.egg-info → trace_digitiser-2.2.0}/PKG-INFO

@@ -1,14 +1,14 @@
 Metadata-Version: 2.4
 Name: trace-digitiser
-Version: 2.0.0
-Summary: Recover the data from line plots in figure images, on your own computer: no uploads, no AI model.
+Version: 2.2.0
+Summary: Recover the data from figure images - lines, points, bars, actograms, heat maps - on your own computer: no uploads, no AI model.
 Author: Trace Digitiser Contributors
 License-Expression: MIT
 Project-URL: Homepage, https://github.com/comparativechrono/trace-digitiser
 Project-URL: Repository, https://github.com/comparativechrono/trace-digitiser
 Project-URL: Issues, https://github.com/comparativechrono/trace-digitiser/issues
 Project-URL: Changelog, https://github.com/comparativechrono/trace-digitiser/blob/main/CHANGELOG.md
-Keywords: plot digitizer,graph digitiser,data extraction,data rescue,computer vision,time series,circadian,actigraphy
+Keywords: plot digitizer,graph digitiser,data extraction,data rescue,computer vision,time series,circadian,actigraphy,actogram,heat map,scatter plot,bar chart
 Classifier: Development Status :: 4 - Beta
 Classifier: Intended Audience :: Science/Research
 Classifier: Operating System :: OS Independent
@@ -40,11 +40,12 @@ Dynamic: license-file
 
 # trace-digitiser
 
-Get the numbers back out of a line plot when the figure is all that survives — on your own computer.
+Get the numbers back out of a figure when the figure is all that survives — on your own computer.
 
 `trace-digitiser` takes an image of a figure (a scan of an old paper, a figure saved from a PDF),
-finds the plot area, follows each trace, and writes the values to a CSV file. It is ordinary image
-processing, written to be read and checked.
+finds the plot area, follows each trace, and writes the values to a CSV file. It reads line plots,
+and also [points, bars, actograms, heat maps and polar plots](#other-kinds-of-figure). It is
+ordinary image processing, written to be read and checked.
 
 **Nothing is uploaded and no AI model is involved.** The only other program it runs is
 [Tesseract](https://github.com/tesseract-ocr/tesseract), also on your machine and optional, to read
@@ -111,6 +112,41 @@ A scale read from labels always comes with a statement of how far to trust it an
 left uncalibrated (values from 0 to 1) and you are told so: the aim is never to pass off a wrong
 scale as a good one ([how well that is met](#how-accurate-is-it)).
 
+## Other kinds of figure
+
+Points, bars, stacked rows, heat maps and polar plots each have a reader of their own. They find
+the plot areas and read the axes exactly as `digitise` does, take the same settings for them
+(`y_min`, `y_ref`, `plot_area`, `exclude` …), and give a table with a row for each thing found:
+
+| Figure | Python | Command line | One row for each |
+|---|---|---|---|
+| Scatter or dot plot, line with markers | `digitise_points` | `trace-digitiser points` | point, with its error bars |
+| Bar chart | `digitise_bars` | `trace-digitiser bars` | bar: where it ends, and its error bar |
+| Actogram, double-plotted record, any stack of rows | `digitise_rows` | `trace-digitiser rows` | bin (or column) of each row |
+| Heat map with a colour bar | `digitise_heatmap` | `trace-digitiser heatmap` | cell, from its colour on the bar |
+| Polar plot | `digitise_polar` | `trace-digitiser polar` | point: its angle and distance |
+
+```python
+from trace_digitiser import digitise_points, digitise_bars, digitise_rows, digitise_heatmap
+
+digitise_points("forest.png", save_overlay=True)        # markers, joined by a line or not; error bars too
+digitise_bars("bars.png", y_min=-25, y_max=0)           # the legend's patch is left out
+digitise_rows("birthogram.png", row_values=range(1955, 2014), row_spacing_value=1.1,
+              bins=24, dots="black")                    # each row's fill, month by month, and its dot
+digitise_heatmap("ccf.png", rows=21)                    # the colour bar's labels are read too
+```
+
+Each returns a `FigureResult`: `result.data` (the table), `result.warnings`, `result.overlay()`
+and the files written (`<image>_points.csv`, `<image>_points_provenance.json` …).
+
+For a program with a graphical interface — the
+[digitiser workbench](https://github.com/comparativechrono/digitiser-workbench), which runs all of
+this in a web browser, where you can mark plot areas, ticks and colours with the mouse —
+`describe_figure` reports what the automatic steps find (plot areas, the tick marks on each axis,
+each axis's tick labels and what was read from them, the colours drawn, a colour bar, polar axes)
+so that they can be shown and corrected, and `pixels_to_values` turns points picked by hand into
+the figure's units. `trace-digitiser describe figure.png` prints the same as JSON.
+
 ## Check the result
 
 1. **Look at the overlay** (`--overlay`). It shows the plot area that was found, the centre and
@@ -180,6 +216,7 @@ the top-left corner of the image, starting at 0; any image viewer shows them.
 | The wrong number of panels | `--layout`, `--rows`, `--cols`, `--panels` | `layout_mode`, `expected_rows`, … |
 | The axis continues beyond its last tick, or has no line at the top or right | `--y-ref ROW:VALUE` twice or more (also `--x-ref`) | `y_ref=[(row, value), …]` |
 | A logarithmic axis, with limits given | `--y-scale log` | `y_scale="log"` |
+| A line drawn against a second y axis, on the right | `--y-axis right` (with `--trace-colour`) | `y_axis="right"` |
 | A trace is missed, or two are merged | `--trace-colour RRGGBB` (repeat per trace) | `trace_colours=["#578dd2", …]` |
 | A legend or annotation is taken for a trace | `--exclude LEFT,TOP,RIGHT,BOTTOM` | `exclude=(l, t, r, b)` |
 | A black trace is invented or ignored | `--achromatic never` / `always` | `achromatic="never"` |
@@ -230,6 +267,18 @@ Labels under 8 pixels high are never given high confidence, however well they ag
 that small is misread in consistent ways — the decimal point dropped from every label, say. If
 your figure has such labels, enlarging the image does not help; typing in two numbers does.
 
+**Published figures.** The tests above draw their own figures.
+[`validation/published-figures`](https://github.com/comparativechrono/trace-digitiser/tree/main/validation/published-figures)
+holds seven figures from four papers whose authors published the numbers behind them — markers with error bars, a
+line over shaded bands, ten bar charts, dot plots, and a scanned page from 1920 — and reads them with nothing given
+and with the help a user can give. With nothing given, 2.2.0 reads all 35 bars of the bar charts to within 0.3 % of
+the axis, and the 29 points of a scatter plot of four series to within 0.5 % (2.1.0 read none of either); with two
+ticks clicked on each axis and the dots' colour, the census dots of the 1920 figure to within 0.9 %. On the figures
+of a paper held out until the changes were made, with the plot area drawn, it reads 24 bars to within 0.4 % once
+two ticks are clicked, and 33 of 37 points once the ticks of a slanted axis and the colours are. Stacked plots with no gap between them, dot
+plots whose dots overlap, and tick labels that are slanted or hand-lettered still need help or remain hard; that
+folder's README has the details.
+
 ### Test it on data you know
 
 The best test of a digitisation is one on data like yours. `round_trip` draws a known series as a
@@ -253,14 +302,19 @@ does this for a week of minute-by-minute activity drawn at the size of a small j
   that is askew is straightened automatically if the tilt is under five degrees; beyond that,
   or for a photograph taken at an angle (where the axes are no longer parallel), correct the
   image first.
-* One trace per colour per panel. Two traces of the same colour, dashed or dotted lines, and
-  traces that are only markers are not separated or joined up.
+* One trace per colour per panel. Two traces of the same colour and dashed or dotted lines are not
+  separated or joined up. Markers are read by `digitise_points`.
 * Where traces cross or overlap, the one drawn underneath is hidden and its values there are
-  interpolated (`interpolated` is `True`).
-* Axis scales are read only from numeric labels on linear or logarithmic axes; dates, broken axes
-  and a second y axis are not understood. Limits typed in are assumed to be at the axis lines
-  unless `y_ref` / `x_ref` say otherwise.
-* Error bars, shaded bands, bars and scatter plots are not digitised.
+  interpolated (`interpolated` is `True`). Markers that overlap heavily (the inner turns of a
+  spiral on a polar plot, say) are separated by their size, and some may be missed or doubled:
+  check the overlay.
+* Axis scales are read only from numeric labels on linear or logarithmic axes (on the left, below,
+  or for a colour bar or a second y axis, on the right); dates, month names and broken axes are
+  not understood: give `x_ref` / `y_ref` for those. Limits typed in are assumed to be at the axis
+  lines unless `y_ref` / `x_ref` say otherwise.
+* Shaded bands are not digitised. A fill drawn with an outline of its own colour (matplotlib's
+  `fill_between`) is measured to the middle of the outline; bars are measured to their edges, to
+  the nearest pixel where the bars' edges are not antialiased.
 
 ## Development
 
@@ -270,6 +324,7 @@ cd trace-digitiser
 pip install -e ".[dev]"
 pytest                      # the tests that need Tesseract are skipped without it
 ruff check src tests
+python validation/published-figures/run.py   # figures from papers against their authors' data (minutes)
 ```
 
 `src/trace_digitiser/` in the order the work is done:
@@ -284,6 +339,11 @@ ruff check src tests
 | `ocr.py`, `summarise.py` | Interval labels along the x axis, and means per interval. |
 | `diagnostics.py`, `provenance.py`, `io.py` | Overlay, record of the run, files. |
 | `validation.py`, `synthetic.py` | Figures drawn from known data, and the comparisons. |
+| `_figure.py` | What every reader does first: the image, straightening, plot areas and axes. |
+| `markers.py`, `points.py` | Markers and their error bars. |
+| `bars.py`, `rows.py`, `heatmap.py`, `polar.py` | Bar charts, stacks of rows, heat maps and polar plots. |
+| `explore.py` | `describe_figure` and `pixels_to_values`, for graphical tools. |
+| `_results.py` | What the readers of points, bars, rows, heat maps and polar plots give back. |
 | `cli.py` | The command line. |
 
 Changes are listed in [CHANGELOG.md](https://github.com/comparativechrono/trace-digitiser/blob/main/CHANGELOG.md);

{trace_digitiser-2.0.0 → trace_digitiser-2.2.0}/README.md

@@ -1,10 +1,11 @@
 # trace-digitiser
 
-Get the numbers back out of a line plot when the figure is all that survives — on your own computer.
+Get the numbers back out of a figure when the figure is all that survives — on your own computer.
 
 `trace-digitiser` takes an image of a figure (a scan of an old paper, a figure saved from a PDF),
-finds the plot area, follows each trace, and writes the values to a CSV file. It is ordinary image
-processing, written to be read and checked.
+finds the plot area, follows each trace, and writes the values to a CSV file. It reads line plots,
+and also [points, bars, actograms, heat maps and polar plots](#other-kinds-of-figure). It is
+ordinary image processing, written to be read and checked.
 
 **Nothing is uploaded and no AI model is involved.** The only other program it runs is
 [Tesseract](https://github.com/tesseract-ocr/tesseract), also on your machine and optional, to read
@@ -71,6 +72,41 @@ A scale read from labels always comes with a statement of how far to trust it an
 left uncalibrated (values from 0 to 1) and you are told so: the aim is never to pass off a wrong
 scale as a good one ([how well that is met](#how-accurate-is-it)).
 
+## Other kinds of figure
+
+Points, bars, stacked rows, heat maps and polar plots each have a reader of their own. They find
+the plot areas and read the axes exactly as `digitise` does, take the same settings for them
+(`y_min`, `y_ref`, `plot_area`, `exclude` …), and give a table with a row for each thing found:
+
+| Figure | Python | Command line | One row for each |
+|---|---|---|---|
+| Scatter or dot plot, line with markers | `digitise_points` | `trace-digitiser points` | point, with its error bars |
+| Bar chart | `digitise_bars` | `trace-digitiser bars` | bar: where it ends, and its error bar |
+| Actogram, double-plotted record, any stack of rows | `digitise_rows` | `trace-digitiser rows` | bin (or column) of each row |
+| Heat map with a colour bar | `digitise_heatmap` | `trace-digitiser heatmap` | cell, from its colour on the bar |
+| Polar plot | `digitise_polar` | `trace-digitiser polar` | point: its angle and distance |
+
+```python
+from trace_digitiser import digitise_points, digitise_bars, digitise_rows, digitise_heatmap
+
+digitise_points("forest.png", save_overlay=True)        # markers, joined by a line or not; error bars too
+digitise_bars("bars.png", y_min=-25, y_max=0)           # the legend's patch is left out
+digitise_rows("birthogram.png", row_values=range(1955, 2014), row_spacing_value=1.1,
+              bins=24, dots="black")                    # each row's fill, month by month, and its dot
+digitise_heatmap("ccf.png", rows=21)                    # the colour bar's labels are read too
+```
+
+Each returns a `FigureResult`: `result.data` (the table), `result.warnings`, `result.overlay()`
+and the files written (`<image>_points.csv`, `<image>_points_provenance.json` …).
+
+For a program with a graphical interface — the
+[digitiser workbench](https://github.com/comparativechrono/digitiser-workbench), which runs all of
+this in a web browser, where you can mark plot areas, ticks and colours with the mouse —
+`describe_figure` reports what the automatic steps find (plot areas, the tick marks on each axis,
+each axis's tick labels and what was read from them, the colours drawn, a colour bar, polar axes)
+so that they can be shown and corrected, and `pixels_to_values` turns points picked by hand into
+the figure's units. `trace-digitiser describe figure.png` prints the same as JSON.
+
 ## Check the result
 
 1. **Look at the overlay** (`--overlay`). It shows the plot area that was found, the centre and
@@ -140,6 +176,7 @@ the top-left corner of the image, starting at 0; any image viewer shows them.
 | The wrong number of panels | `--layout`, `--rows`, `--cols`, `--panels` | `layout_mode`, `expected_rows`, … |
 | The axis continues beyond its last tick, or has no line at the top or right | `--y-ref ROW:VALUE` twice or more (also `--x-ref`) | `y_ref=[(row, value), …]` |
 | A logarithmic axis, with limits given | `--y-scale log` | `y_scale="log"` |
+| A line drawn against a second y axis, on the right | `--y-axis right` (with `--trace-colour`) | `y_axis="right"` |
 | A trace is missed, or two are merged | `--trace-colour RRGGBB` (repeat per trace) | `trace_colours=["#578dd2", …]` |
 | A legend or annotation is taken for a trace | `--exclude LEFT,TOP,RIGHT,BOTTOM` | `exclude=(l, t, r, b)` |
 | A black trace is invented or ignored | `--achromatic never` / `always` | `achromatic="never"` |
@@ -190,6 +227,18 @@ Labels under 8 pixels high are never given high confidence, however well they ag
 that small is misread in consistent ways — the decimal point dropped from every label, say. If
 your figure has such labels, enlarging the image does not help; typing in two numbers does.
 
+**Published figures.** The tests above draw their own figures.
+[`validation/published-figures`](https://github.com/comparativechrono/trace-digitiser/tree/main/validation/published-figures)
+holds seven figures from four papers whose authors published the numbers behind them — markers with error bars, a
+line over shaded bands, ten bar charts, dot plots, and a scanned page from 1920 — and reads them with nothing given
+and with the help a user can give. With nothing given, 2.2.0 reads all 35 bars of the bar charts to within 0.3 % of
+the axis, and the 29 points of a scatter plot of four series to within 0.5 % (2.1.0 read none of either); with two
+ticks clicked on each axis and the dots' colour, the census dots of the 1920 figure to within 0.9 %. On the figures
+of a paper held out until the changes were made, with the plot area drawn, it reads 24 bars to within 0.4 % once
+two ticks are clicked, and 33 of 37 points once the ticks of a slanted axis and the colours are. Stacked plots with no gap between them, dot
+plots whose dots overlap, and tick labels that are slanted or hand-lettered still need help or remain hard; that
+folder's README has the details.
+
 ### Test it on data you know
 
 The best test of a digitisation is one on data like yours. `round_trip` draws a known series as a
@@ -213,14 +262,19 @@ does this for a week of minute-by-minute activity drawn at the size of a small j
   that is askew is straightened automatically if the tilt is under five degrees; beyond that,
   or for a photograph taken at an angle (where the axes are no longer parallel), correct the
   image first.
-* One trace per colour per panel. Two traces of the same colour, dashed or dotted lines, and
-  traces that are only markers are not separated or joined up.
+* One trace per colour per panel. Two traces of the same colour and dashed or dotted lines are not
+  separated or joined up. Markers are read by `digitise_points`.
 * Where traces cross or overlap, the one drawn underneath is hidden and its values there are
-  interpolated (`interpolated` is `True`).
-* Axis scales are read only from numeric labels on linear or logarithmic axes; dates, broken axes
-  and a second y axis are not understood. Limits typed in are assumed to be at the axis lines
-  unless `y_ref` / `x_ref` say otherwise.
-* Error bars, shaded bands, bars and scatter plots are not digitised.
+  interpolated (`interpolated` is `True`). Markers that overlap heavily (the inner turns of a
+  spiral on a polar plot, say) are separated by their size, and some may be missed or doubled:
+  check the overlay.
+* Axis scales are read only from numeric labels on linear or logarithmic axes (on the left, below,
+  or for a colour bar or a second y axis, on the right); dates, month names and broken axes are
+  not understood: give `x_ref` / `y_ref` for those. Limits typed in are assumed to be at the axis
+  lines unless `y_ref` / `x_ref` say otherwise.
+* Shaded bands are not digitised. A fill drawn with an outline of its own colour (matplotlib's
+  `fill_between`) is measured to the middle of the outline; bars are measured to their edges, to
+  the nearest pixel where the bars' edges are not antialiased.
 
 ## Development
 
@@ -230,6 +284,7 @@ cd trace-digitiser
 pip install -e ".[dev]"
 pytest                      # the tests that need Tesseract are skipped without it
 ruff check src tests
+python validation/published-figures/run.py   # figures from papers against their authors' data (minutes)
 ```
 
 `src/trace_digitiser/` in the order the work is done:
@@ -244,6 +299,11 @@ ruff check src tests
 | `ocr.py`, `summarise.py` | Interval labels along the x axis, and means per interval. |
 | `diagnostics.py`, `provenance.py`, `io.py` | Overlay, record of the run, files. |
 | `validation.py`, `synthetic.py` | Figures drawn from known data, and the comparisons. |
+| `_figure.py` | What every reader does first: the image, straightening, plot areas and axes. |
+| `markers.py`, `points.py` | Markers and their error bars. |
+| `bars.py`, `rows.py`, `heatmap.py`, `polar.py` | Bar charts, stacks of rows, heat maps and polar plots. |
+| `explore.py` | `describe_figure` and `pixels_to_values`, for graphical tools. |
+| `_results.py` | What the readers of points, bars, rows, heat maps and polar plots give back. |
 | `cli.py` | The command line. |
 
 Changes are listed in [CHANGELOG.md](https://github.com/comparativechrono/trace-digitiser/blob/main/CHANGELOG.md);

{trace_digitiser-2.0.0 → trace_digitiser-2.2.0}/RELEASING.md

@@ -23,14 +23,14 @@ This has to be done once by an owner of the `trace-digitiser` project on PyPI.
 
 3. Click **Add**.
 
-That is all. The link works while the repository is private.
+That is all.
 
 ### Optional hardening
 
 PyPI recommends tying the publisher to a GitHub *environment*, which lets you
-require a manual approval before each upload. GitHub only offers environments
-on private repositories for paid plans, so the workflow does not use one. Once
-the repository is public (or on a paid plan):
+require a manual approval before each upload. The workflow does not use one
+yet (when the repository was private, GitHub offered environments only on paid
+plans). Now that the repository is public, one can be added:
 
 1. Create an environment called `pypi` under *Settings -> Environments*
    (optionally with required reviewers).
@@ -39,12 +39,27 @@ the repository is public (or on a paid plan):
 
 ## Making a release
 
-1. Set `version` in `pyproject.toml` (PyPI never accepts the same version
-   twice; the last published version is 1.2.0).
+1. Set `version` in `pyproject.toml`. PyPI never accepts the same version
+   twice; the versions already published are listed at
+   <https://pypi.org/project/trace-digitiser/#history>.
 2. In `CHANGELOG.md`, give the section for the version its date in place of
-   "not yet released".
-3. Merge to `main` and wait for the *Tests* workflow to pass.
-4. On GitHub: *Releases -> Draft a new release*, create the tag `vX.Y.Z`
+   "unreleased", and end its link at the new tag (`...vX.Y.Z` in place of
+   `...HEAD`).
+3. Run the published-figure test set
+   ([`validation/published-figures`](validation/published-figures/README.md)),
+   which reads figures from papers whose data are published, and compare the
+   results with the last release's:
+
+   ```bash
+   python validation/published-figures/run.py \
+       --json validation/published-figures/results-X.Y.Z.json \
+       --compare validation/published-figures/results-<last release>.json
+   ```
+
+   Look into any case marked WORSE before going on. Commit the new results,
+   and update the table in that folder's README (`--markdown` prints it).
+4. Merge to `main` and wait for the *Tests* workflow to pass.
+5. On GitHub: *Releases -> Draft a new release*, create the tag `vX.Y.Z`
    (matching the version exactly, with a leading `v`) on `main`, and publish.
 
 Publishing the release starts the workflow. It checks that the tag matches the
@@ -53,8 +68,7 @@ uploads to PyPI. If the tag and version disagree, or a test fails, the run
 stops before anything is uploaded.
 
 The README is shown on PyPI as it is in the release. Its links to files in
-this repository are absolute, so they work there only once the repository is
-public.
+this repository are absolute, so they work there too.
 
 ## Checking a build without releasing
 

{trace_digitiser-2.0.0 → trace_digitiser-2.2.0}/pyproject.toml

@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "trace-digitiser"
-version = "2.0.0"
-description = "Recover the data from line plots in figure images, on your own computer: no uploads, no AI model."
+version = "2.2.0"
+description = "Recover the data from figure images - lines, points, bars, actograms, heat maps - on your own computer: no uploads, no AI model."
 readme = "README.md"
 license = "MIT"
 license-files = ["LICENSE"]
@@ -20,6 +20,10 @@ keywords = [
     "time series",
     "circadian",
     "actigraphy",
+    "actogram",
+    "heat map",
+    "scatter plot",
+    "bar chart",
 ]
 classifiers = [
     "Development Status :: 4 - Beta",