trace-digitiser 2.1.0__tar.gz → 2.3.0__tar.gz

{trace_digitiser-2.1.0 → trace_digitiser-2.3.0}/CHANGELOG.md

@@ -4,6 +4,153 @@ All notable changes to trace-digitiser are recorded here.
 The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
 and the project uses [semantic versioning](https://semver.org/).
 
+## [Unreleased]
+
+## [2.3.0] - 2026-11-15
+
+Read again with 2.2.0, some of the figures in the validation sets still came
+out wrong, or not at all: dot plots of translucent dots that overlap, legends
+and labels drawn inside plots, plots stacked with no gap between them, tick
+labels set close together, a line inside a pale band of its own colour. This
+release reads most of them: of the values in the figures of published papers
+(`validation/published-figures`), 1,184 of 1,601 are read (2.2.0: 614). The
+changes were made while looking at all of those figures, including the ones
+of Andreani et al. (2022) that were held out until 2.2.0 was made, so their
+results are no longer those of figures held out. Activity traces read as well
+as before or better (median correlation with the authors' series 0.95; 2.2.0:
+0.94), and dense traces give the same results as with 2.2.0.
+
+### Added
+
+- **Legends inside a plot are found and left out**
+  (`trace_digitiser.legend`; `digitise(..., legend="auto")`, the default, or
+  `legend="keep"`): two or more entries, each a symbol with its words to the
+  right, one above another or side by side in a row. Their symbols and words
+  are left out as user-given `exclude` boxes are, and a warning says where.
+- **Words written inside a plot are taken out of its traces**: a panel's
+  label in the trace's colour (such as "18:6") is found as a row of glyphs
+  standing on a baseline and left out of the trace.
+- **Stacked plots with no gap between them are separated**: lines as dark and
+  as thick as the frame's sides that run right across it are noted, and where
+  the frame's tick labels give no one scale and the plots between such lines
+  each read as an axis of their own, they are taken as plots of their own.
+  Leypunskiy et al. (2017) 2B with nothing given: all three plots, 0.2-0.6 %
+  (2.2.0: no axis read).
+- `validation/published-figures/run.py --digitised` keeps what each case read
+  (`validation/published-figures/digitised`), and `paper/figures.py` draws
+  the paper's supporting figures (S1-S6 Fig) from it and from the other
+  validation sets.
+- **A validation set of activity traces**
+  ([`validation/activity-traces`](https://github.com/comparativechrono/trace-digitiser/tree/main/validation/activity-traces)):
+  57 traces of mouse and rat activity over days, from the figures of 18
+  open-access papers found by a search of Europe PMC, read with 2.2.0 as a
+  workbench user would read them. For the 8 papers that published the numbers
+  behind their figures, the same rhythm analysis is run on the trace as read
+  and on the authors' series: the Lomb-Scargle periodogram and the 24-h
+  cosinor reach the same verdict for all 29 series, the share of activity in
+  the dark phase is within 0.05 for 27 of 29. For all 18, what the papers say
+  about their traces is checked against them. Not part of the package.
+- **A test of dense traces**
+  ([`validation/dense-traces`](https://github.com/comparativechrono/trace-digitiser/tree/main/validation/dense-traces)):
+  nine real activity records, kept every minute or every 5 minutes, drawn as
+  a week across a few hundred pixels (so that each pixel column is a stroke
+  from the lowest to the highest sample within it) and read back. Averaged
+  within each hour, `center_estimate` (the midpoint of each column) puts the
+  acrophase within 0.36 h of the record's (median) and reads the level 1.19
+  times too high; `upper_envelope` times the rhythm as well but reads the
+  level about twice too high. Every column statistic but the lower envelope
+  understates the share of activity in the dark phase. Not part of the
+  package.
+
+### Fixed
+
+- **Overlapping markers** are found by the edges of each that can be seen: a
+  Hough transform for circles of one size, in which each pixel on an edge -
+  of the blob, or, where the markers are translucent, between one layer and
+  two - votes for a centre a marker's radius inwards. Markers mostly hidden
+  are added where the blob is otherwise not covered; a marker partly hidden
+  under dark ink (a mean's bar, an error bar) is taken to go on under it; the
+  size of one marker is taken from the round blobs where most blobs are two
+  or three markers. Dot plots of Andreani et al. (2022) 3A-D: 386 of the 497
+  dots compared, with their colours given (2.2.0: 143), 348 with nothing
+  given (117); the polar plot of Hearn & Whitmore (2026): 55 of 60 years (50).
+- **Tick labels set close together, or run into an axis title**, are each
+  looked for under (or beside) their own tick, where the ticks are at a
+  regular step: Leypunskiy et al. 5A with nothing given, 9 of 9 points
+  (2.2.0: x axis not read); Andreani et al. 1E with its plot area, 24 of 24
+  bars (y axis not read).
+- **Ticks are found on plots shaded with pale bands**, by their contrast with
+  what they are drawn on.
+- **A line drawn inside a pale band of its own colour** (a mean within its
+  error, shaded) is followed by its strong colour, not by the band's middle:
+  Andreani et al. 1C, FD 1.0 % (4.2 %).
+- **Error bars** are judged against what lies round them, so that a pale band
+  is not taken for a bar; a line through a marker that runs on, uncapped, to
+  the edge of the plot, or aslant past where a bar would end, or on beyond a
+  gap (a broken gridline), is not a bar.
+- A label read as "04" counts as evidence that an axis's decimal points are
+  being lost in small print, or in larger print only if every reading of it
+  agrees.
+- A lump of more than fifty markers' area is a region of colour, not markers
+  (a figure with shaded bands took four minutes).
+
+## [2.2.0] - 2026-11-15
+
+Tested on the figures of published papers whose underlying numbers are
+published too (Leypunskiy et al. 2017, eLife 6:e23539: markers with error
+bars, a line over shaded bands; Foley et al. 2019, eLife 8:e50063: bar charts;
+Pearl and Reed 1920, PNAS 6:275, a microfilm scan), 2.1.0 read what it found
+accurately but often found nothing. This release finds much more of it, there
+and on the figures of a paper held out until the changes were made (Andreani
+et al. 2022, eLife 11:e74327). Line plots digitise as before: the package's
+validation suite and the Hearn and Whitmore (2026) examples give the same
+results.
+
+### Added
+
+- **A test set of published figures**
+  ([`validation/published-figures`](https://github.com/comparativechrono/trace-digitiser/tree/main/validation/published-figures)):
+  seven figures from those four papers, the numbers behind them from their
+  authors' data, and a script that reads them with nothing given and with the
+  help a user can give, and compares. Its README has the results of 2.1.0 and
+  2.2.0, case by case, including the few that 2.2.0 reads worse. It is run
+  before each release and is not part of the package.
+
+### Fixed
+
+- **Bar charts in the style of GraphPad Prism**: black, grey, hatched and
+  outlined bars are found, on an axis that does not start at 0 (2.1.0 found 3
+  of the 35 bars of Foley et al.'s figure 2, and read those upside down; all
+  35 are now read, within 0.3 % of the axis). Bars are the columns filled from
+  the baseline, so an error bar or a significance bracket joined to a bar no
+  longer spoils it; a border of another colour (a black outline round a grey
+  bar) is part of the bar; a bar cut by the edge of the plot area is kept.
+- **Error bars drawn in the colour of their bar or marker** (the default in
+  Prism, MATLAB and matplotlib) are followed beyond the bar's end, or from the
+  marker, to their caps; one running across a fitted line goes on beyond it.
+  Where only the half beyond a bar's end shows, a note says so.
+- **Markers sitting on a line of their own colour** (a fitted line, their
+  error bars) are found: other sizes are tried where the lines' ridges
+  outnumber the markers, and such a colour counts as a colour of markers if
+  its markers are joined to its lines and much thicker than them. Round
+  letters of words written in the plot, and a legend's markers beside their
+  words, are left out; a marker found in two series of close colours is kept
+  in the nearer one.
+- **A plot area given by hand** (a box drawn in the workbench, or numbers read
+  off a picture) moves to the middle of the axis lines within a few pixels of
+  it, and their thickness is measured: with a thick axis line the tick labels
+  were looked for inside it, and the axis title was read instead.
+- Long tick marks of a small plot in a large picture are found, so that its
+  labels are anchored to them.
+- A colour that fills areas (a shaded band behind a plot) is not taken for a
+  trace of a line plot.
+- Tick labels that Tesseract misreads as a line of text (75 as 15, in large
+  Helvetica) are read once more as a single word when an axis is not settled.
+- Looking at a whole page of a scanned paper (`describe_figure`, the
+  workbench's first look at a page of a PDF) took a minute and a half, nearly
+  all of it in the search for polar axes; the circles are now proposed on a
+  smaller copy of the page (16 seconds in all).
+
 ## [2.1.0] - 2026-11-14
 
 Everything in 2.0.0 works as before. What is new is readers for other kinds of
@@ -165,6 +312,9 @@ First version tracked in this repository; identical to the 1.2.0 files on PyPI.
 Releases 0.1.0 to 1.0.0 (6-7 June 2026) were uploaded to PyPI directly and are
 not recorded here.
 
+[Unreleased]: https://github.com/comparativechrono/trace-digitiser/compare/v2.3.0...HEAD
+[2.3.0]: https://github.com/comparativechrono/trace-digitiser/compare/v2.2.0...v2.3.0
+[2.2.0]: https://github.com/comparativechrono/trace-digitiser/compare/v2.1.0...v2.2.0
 [2.1.0]: https://github.com/comparativechrono/trace-digitiser/compare/v2.0.0...v2.1.0
 [2.0.0]: https://github.com/comparativechrono/trace-digitiser/compare/1d34f75...v2.0.0
 [1.2.0]: https://pypi.org/project/trace-digitiser/1.2.0/

{trace_digitiser-2.1.0 → trace_digitiser-2.3.0}/MANIFEST.in

@@ -5,3 +5,5 @@ recursive-include tests *.py *.csv
 recursive-include examples *.py *.md *.jpg *.ipynb
 prune examples/output
 include CHANGELOG.md RELEASING.md
+# The published-figure test set (validation/published-figures) is run from the repository, not shipped.
+prune validation

{trace_digitiser-2.1.0/src/trace_digitiser.egg-info → trace_digitiser-2.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: trace-digitiser
-Version: 2.1.0
+Version: 2.3.0
 Summary: Recover the data from figure images - lines, points, bars, actograms, heat maps - on your own computer: no uploads, no AI model.
 Author: Trace Digitiser Contributors
 License-Expression: MIT
@@ -255,7 +255,7 @@ three typefaces; JPEG compression and blur).
 | Height of the labels | axes | read correctly | declined | read wrongly, with a warning | read wrongly, no warning |
 |---|---|---|---|---|---|
 | 8 pixels or more | 477 | 464 | 13 | 0 | 0 |
-| under 8 pixels | 273 | 257 (all with a warning to check them) | 15 | 1 | 0 |
+| under 8 pixels | 273 | 260 (all with a warning to check them) | 12 | 1 | 0 |
 
 *Declined* means that no scale was offered. All of those at 8 pixels or more are logarithmic axes
 whose raised exponents are too small to read, though the rest of the label is not. The outcome for small
@@ -267,6 +267,39 @@ Labels under 8 pixels high are never given high confidence, however well they ag
 that small is misread in consistent ways — the decimal point dropped from every label, say. If
 your figure has such labels, enlarging the image does not help; typing in two numbers does.
 
+**Published figures.** The tests above draw their own figures.
+[`validation/published-figures`](https://github.com/comparativechrono/trace-digitiser/tree/main/validation/published-figures)
+holds seven figures from four papers whose authors published the numbers behind them — markers with error bars, a
+line over shaded bands, ten bar charts, dot plots, and a scanned page from 1920 — and reads them with nothing given
+and with the help a user can give. With nothing given, 2.3.0 reads all 35 bars of the bar charts to within 0.3 % of
+the axis, the 29 points of a scatter plot of four series to within 0.5 %, the 9 points of a phase-response curve whose
+tick labels are set close together, and each of three line plots stacked with no gap between them to within 0.6 %;
+with two ticks clicked on each axis and the dots' colour, it reads the census dots of the 1920 figure to within
+0.9 %. In four dot plots of translucent dots that overlap, it finds 386 of the 497 dots compared once their colours
+are clicked (2.2.0: 143). The figures of one of the papers were held out until 2.2.0 was made; 2.3.0 was made while
+looking at them too, so they are a held-out test of 2.2.0 only. Tick labels that are slanted or hand-lettered, and
+series of one colour told apart only by the shape of their markers, still need help; that folder's README has the
+details, case by case.
+
+**Activity traces.**
+[`validation/activity-traces`](https://github.com/comparativechrono/trace-digitiser/tree/main/validation/activity-traces)
+tests the use the package was written for: rhythm analysis of activity traces digitised from papers. It holds 57
+traces of mouse and rat activity over days from 18 open-access papers found by a literature search. For the 29
+series whose authors published the numbers behind them, the same rhythm analysis is run on the trace as read and on
+the authors' series: the Lomb–Scargle periodogram and the 24-h cosinor reach the same verdict on every series, the
+share of activity in the dark phase agrees to within 0.05 for 27, and the acrophase to within 1 h for 27. For all 18
+papers, what the authors say about their traces is checked against them: 46 of 53 statements are borne out, and
+where the authors' numbers can decide, the trace and the numbers agree on 21 of 22.
+
+**Dense traces.**
+[`validation/dense-traces`](https://github.com/comparativechrono/trace-digitiser/tree/main/validation/dense-traces)
+draws real activity records, kept every minute or every 5 minutes, as dense traces (a week across a few hundred
+pixels, so that each pixel column is a stroke from the lowest to the highest sample within it) and reads them back.
+Averaged within each hour, the midpoint of each column (`center_estimate`) puts the acrophase within 0.36 h of the
+record's (median) and reads the level 1.19 times too high; the top of each column (`upper_envelope`) times the rhythm
+as well but reads the level about twice too high. Every reading of a dense trace understates how much of the
+activity falls in the dark phase, by 0.03–0.04 for the means of three mice and more for single animals.
+
 ### Test it on data you know
 
 The best test of a digitisation is one on data like yours. `round_trip` draws a known series as a
@@ -312,6 +345,9 @@ cd trace-digitiser
 pip install -e ".[dev]"
 pytest                      # the tests that need Tesseract are skipped without it
 ruff check src tests
+python validation/published-figures/run.py   # figures from papers against their authors' data (minutes)
+python validation/activity-traces/run.py      # rhythm analysis of digitised activity traces (a minute)
+python validation/dense-traces/run.py         # real records drawn as dense traces and read back (seconds)
 ```
 
 `src/trace_digitiser/` in the order the work is done:

{trace_digitiser-2.1.0 → trace_digitiser-2.3.0}/README.md

@@ -215,7 +215,7 @@ three typefaces; JPEG compression and blur).
 | Height of the labels | axes | read correctly | declined | read wrongly, with a warning | read wrongly, no warning |
 |---|---|---|---|---|---|
 | 8 pixels or more | 477 | 464 | 13 | 0 | 0 |
-| under 8 pixels | 273 | 257 (all with a warning to check them) | 15 | 1 | 0 |
+| under 8 pixels | 273 | 260 (all with a warning to check them) | 12 | 1 | 0 |
 
 *Declined* means that no scale was offered. All of those at 8 pixels or more are logarithmic axes
 whose raised exponents are too small to read, though the rest of the label is not. The outcome for small
@@ -227,6 +227,39 @@ Labels under 8 pixels high are never given high confidence, however well they ag
 that small is misread in consistent ways — the decimal point dropped from every label, say. If
 your figure has such labels, enlarging the image does not help; typing in two numbers does.
 
+**Published figures.** The tests above draw their own figures.
+[`validation/published-figures`](https://github.com/comparativechrono/trace-digitiser/tree/main/validation/published-figures)
+holds seven figures from four papers whose authors published the numbers behind them — markers with error bars, a
+line over shaded bands, ten bar charts, dot plots, and a scanned page from 1920 — and reads them with nothing given
+and with the help a user can give. With nothing given, 2.3.0 reads all 35 bars of the bar charts to within 0.3 % of
+the axis, the 29 points of a scatter plot of four series to within 0.5 %, the 9 points of a phase-response curve whose
+tick labels are set close together, and each of three line plots stacked with no gap between them to within 0.6 %;
+with two ticks clicked on each axis and the dots' colour, it reads the census dots of the 1920 figure to within
+0.9 %. In four dot plots of translucent dots that overlap, it finds 386 of the 497 dots compared once their colours
+are clicked (2.2.0: 143). The figures of one of the papers were held out until 2.2.0 was made; 2.3.0 was made while
+looking at them too, so they are a held-out test of 2.2.0 only. Tick labels that are slanted or hand-lettered, and
+series of one colour told apart only by the shape of their markers, still need help; that folder's README has the
+details, case by case.
+
+**Activity traces.**
+[`validation/activity-traces`](https://github.com/comparativechrono/trace-digitiser/tree/main/validation/activity-traces)
+tests the use the package was written for: rhythm analysis of activity traces digitised from papers. It holds 57
+traces of mouse and rat activity over days from 18 open-access papers found by a literature search. For the 29
+series whose authors published the numbers behind them, the same rhythm analysis is run on the trace as read and on
+the authors' series: the Lomb–Scargle periodogram and the 24-h cosinor reach the same verdict on every series, the
+share of activity in the dark phase agrees to within 0.05 for 27, and the acrophase to within 1 h for 27. For all 18
+papers, what the authors say about their traces is checked against them: 46 of 53 statements are borne out, and
+where the authors' numbers can decide, the trace and the numbers agree on 21 of 22.
+
+**Dense traces.**
+[`validation/dense-traces`](https://github.com/comparativechrono/trace-digitiser/tree/main/validation/dense-traces)
+draws real activity records, kept every minute or every 5 minutes, as dense traces (a week across a few hundred
+pixels, so that each pixel column is a stroke from the lowest to the highest sample within it) and reads them back.
+Averaged within each hour, the midpoint of each column (`center_estimate`) puts the acrophase within 0.36 h of the
+record's (median) and reads the level 1.19 times too high; the top of each column (`upper_envelope`) times the rhythm
+as well but reads the level about twice too high. Every reading of a dense trace understates how much of the
+activity falls in the dark phase, by 0.03–0.04 for the means of three mice and more for single animals.
+
 ### Test it on data you know
 
 The best test of a digitisation is one on data like yours. `round_trip` draws a known series as a
@@ -272,6 +305,9 @@ cd trace-digitiser
 pip install -e ".[dev]"
 pytest                      # the tests that need Tesseract are skipped without it
 ruff check src tests
+python validation/published-figures/run.py   # figures from papers against their authors' data (minutes)
+python validation/activity-traces/run.py      # rhythm analysis of digitised activity traces (a minute)
+python validation/dense-traces/run.py         # real records drawn as dense traces and read back (seconds)
 ```
 
 `src/trace_digitiser/` in the order the work is done:

{trace_digitiser-2.1.0 → trace_digitiser-2.3.0}/RELEASING.md

@@ -45,8 +45,37 @@ plans). Now that the repository is public, one can be added:
 2. In `CHANGELOG.md`, give the section for the version its date in place of
    "unreleased", and end its link at the new tag (`...vX.Y.Z` in place of
    `...HEAD`).
-3. Merge to `main` and wait for the *Tests* workflow to pass.
-4. On GitHub: *Releases -> Draft a new release*, create the tag `vX.Y.Z`
+3. Run the published-figure test set
+   ([`validation/published-figures`](validation/published-figures/README.md)),
+   which reads figures from papers whose data are published, and compare the
+   results with the last release's:
+
+   ```bash
+   python validation/published-figures/run.py --digitised \
+       --json validation/published-figures/results-X.Y.Z.json \
+       --compare validation/published-figures/results-<last release>.json
+   ```
+
+   Look into any case marked WORSE before going on. Commit the new results
+   and what each case read (`digitised/`), and update the table in that
+   folder's README (`--markdown` prints it).
+
+   Then run the activity-trace set
+   ([`validation/activity-traces`](validation/activity-traces/README.md)),
+   which needs pandas, openpyxl, SciPy and matplotlib besides:
+
+   ```bash
+   python validation/activity-traces/run.py
+   git diff --stat validation/activity-traces
+   ```
+
+   Its readings and results are kept in git (the script records the
+   release in `digitised/VERSION`), so the diff shows what the release
+   changed. Look into any change in `results/agreement_*.csv` or in the
+   verdicts of `results/claims_checked.csv`, and update the numbers in that
+   folder's README (`tables.py` prints its tables).
+4. Merge to `main` and wait for the *Tests* workflow to pass.
+5. On GitHub: *Releases -> Draft a new release*, create the tag `vX.Y.Z`
    (matching the version exactly, with a leading `v`) on `main`, and publish.
 
 Publishing the release starts the workflow. It checks that the tag matches the

{trace_digitiser-2.1.0 → trace_digitiser-2.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "trace-digitiser"
-version = "2.1.0"
+version = "2.3.0"
 description = "Recover the data from figure images - lines, points, bars, actograms, heat maps - on your own computer: no uploads, no AI model."
 readme = "README.md"
 license = "MIT"

{trace_digitiser-2.1.0 → trace_digitiser-2.3.0}/src/trace_digitiser/__init__.py

@@ -61,6 +61,7 @@ from .diagnostics import (
 )
 from .digitise import digitise_trace_mask
 from .io import build_panel_metadata, load_image, save_outputs  # noqa: F401  (load_image: as before)
+from .legend import drop_text, find_text_rows
 from .models import Calibration, DigitiserResult, Panel, ResampledResult, Trace, XLabel
 from .ocr import detect_x_labels
 from .provenance import build_provenance, package_version, write_provenance
@@ -119,6 +120,7 @@ def digitise(
     write_outputs: bool = True,
     straighten: Union[str, bool, float] = "auto",
     y_axis: str = "left",
+    legend: Union[str, bool] = "auto",
 ) -> DigitiserResult:
     """Digitise the line traces in a figure image.
 
@@ -182,6 +184,11 @@ def digitise(
     exclude : (x_left, y_top, x_right, y_bottom), or a list of them, optional
         Regions to ignore when looking for traces, e.g. a legend or an
         annotation drawn inside the plot.
+    legend : {"auto", "keep"}
+        ``"auto"`` (default) finds a legend drawn inside a plot area - two
+        or more entries, each a symbol with its label to the right - and
+        leaves it out, as if it had been given in *exclude*; a warning
+        says where. ``"keep"`` does not look for one.
     achromatic : {"auto", "always", "never"}
         Whether to look for a black or grey trace.
     ocr : bool
@@ -214,7 +221,7 @@ def digitise(
         expected_panels=expected_panels, y_min=y_min, y_max=y_max, x_min=x_min, x_max=x_max, total_days=total_days,
         output_dir=output_dir, output_prefix=output_prefix, show_debug=show_debug, save_diagnostics=save_diagnostics,
         plot_area=plot_area, y_ref=y_ref, x_ref=x_ref, y_scale=y_scale, x_scale=x_scale, exclude=exclude, ocr=ocr,
-        write_outputs=write_outputs, straighten=straighten, y_axis=y_axis,
+        write_outputs=write_outputs, straighten=straighten, y_axis=y_axis, legend=legend,
         extra_parameters={"trace_colours": list(trace_colours) if trace_colours else None, "achromatic": achromatic},
     )
     rgb, dark, panels, notes, note = fig.rgb, fig.dark, fig.panels, fig.notes, fig.note
@@ -231,7 +238,17 @@ def digitise(
     for p in panels:
         masks = detect_trace_masks(
             rgb, p, colours=trace_colours, exclude=exclude_boxes, achromatic=achromatic, darkness=dark,
+            lines_only=True,
         )
+        # words written in the plot in a trace's colour (a panel's label, say) are not trace; words are found in
+        # black or grey print, so only a black or grey trace can hold them
+        grey = [tr for tr in masks if np.ptp(np.asarray(tr.get("median_rgb", (0, 0, 0)), float)) <= 60]
+        rows = find_text_rows(rgb, (p["x0"], p["y_top"], p["x1"], p["y_bottom"])) if grey else []
+        if rows:
+            for tr in grey:
+                tr["mask"] = drop_text(tr["mask"], (p["x0"], p["y_top"]), rows)
+                tr["pixel_count"] = int(tr["mask"].sum())
+            masks = [tr for tr in masks if tr["pixel_count"] > 0]
         if show_debug:
             print(f"Panel {p['panel_id']}: detected {len(masks)} trace(s)")
         if not masks:

{trace_digitiser-2.1.0 → trace_digitiser-2.3.0}/src/trace_digitiser/_figure.py

@@ -26,8 +26,9 @@ from .calibration import (
 from .diagnostics import draw_panel_overlay, print_panel_summary
 from .frames import darkness_map
 from .io import load_image
+from .legend import find_legend
 from .models import Calibration
-from .panel_detection import find_plot_panels
+from .panel_detection import find_plot_panels, split_stack
 from .straighten import MIN_TILT_DEGREES, estimate_tilt
 from .straighten import straighten as _turn_upright
 from .x_calibration import calibrate_x, x_calibration_from_points, x_calibration_from_range
@@ -209,6 +210,7 @@ def prepare_figure(
     write_outputs: bool = True,
     straighten: Union[str, bool, float] = "auto",
     y_axis: str = "left",
+    legend: Union[str, bool] = "auto",
     extra_parameters: Optional[dict] = None,
     stacklevel: int = 4,
 ) -> PreparedFigure:
@@ -227,6 +229,9 @@ def prepare_figure(
             raise ValueError(f"{name} must be 'auto', 'linear' or 'log'")
     if y_axis not in ("left", "right"):
         raise ValueError("y_axis must be 'left' or 'right'")
+    if legend not in ("auto", "keep", True, False):
+        raise ValueError("legend must be 'auto' (find legends drawn inside plots and leave them out) or 'keep'")
+    find_legends = legend in ("auto", True)
     if (y_min is None) != (y_max is None):
         raise ValueError("give both y_min and y_max, or neither")
     if (x_min is None) != (x_max is None):
@@ -276,6 +281,8 @@ def prepare_figure(
     }
     if y_axis != "left":
         parameters["y_axis"] = y_axis
+    if not find_legends:
+        parameters["legend"] = "keep"
     parameters.update(extra_parameters or {})
 
     notes: list[str] = []
@@ -389,6 +396,41 @@ def prepare_figure(
             cal = normalised_calibration(p)
         p["y_calibration"] = cal.to_dict()
 
+    # Plots stacked with no gap between them make one frame, whose tick labels then do not agree on a scale (they
+    # repeat, one set to a plot). Where lines as dark as the frame's sides run across it, and the plots between them
+    # each read as an axis of their own, they are taken as plots of their own.
+    if y_ref is None and y_min is None and ocr and plot_boxes is None:
+        split_any = False
+        result: list[dict] = []
+        for p in panels:
+            cal = p["y_calibration"]
+            weak = cal["scale_type"] == "normalised_0_to_1" or cal.get("confidence") == "low"
+            if p.get("stack_cuts") and weak:
+                subs = split_stack(rgb, p)
+                for k, sub in enumerate(subs):
+                    sub["panel_id"] = k + 1
+                    sub["y_calibration"] = calibrate_y(
+                        rgb, dark, sub, limit=limits[p["panel_id"]][0], scale=y_scale, verbose=show_debug,
+                        axis="y2" if y_axis == "right" else "y",
+                    ).to_dict()
+                if all(sub["y_calibration"]["scale_type"] != "normalised_0_to_1" for sub in subs):
+                    result.extend(subs)
+                    split_any = True
+                    note(
+                        f"Panel {p['panel_id']} is {len(subs)} plots stacked one above another, with no gap "
+                        "between them; each is read as a plot of its own."
+                    )
+                    continue
+            result.append(p)
+        if split_any:
+            result.sort(key=lambda d: (round(d["y_top"]), d["x0"]))
+            for k, sub in enumerate(result, start=1):
+                sub["panel_id"] = k
+                sub.setdefault("layout_mode", layout_mode)
+            panels = result
+            limits = _neighbour_limits(panels, right=(y_axis == "right"))
+            n_panels = len(panels)
+
     if y_ref is None and y_min is None:
         panels = propagate_y_calibration_across_rows(panels, verbose=show_debug)
 
@@ -464,6 +506,33 @@ def prepare_figure(
             "output has x_pixel and x_norm_0_to_1 but no x_value. Give x_min and x_max, or x_ref."
         )
 
+    # ------------------------------------------------------------------
+    # 4. A legend drawn inside a plot is left out, as if it had been excluded
+    # ------------------------------------------------------------------
+    if find_legends:
+        found_legends = []
+        for p in panels:
+            e = p["edges"]
+            half = int(p.get("line_thickness", 1)) // 2 + 1
+            box = (
+                int(np.ceil(e["left"])) + half,
+                int(np.ceil(e["top"])) + half,
+                int(np.floor(e["right"])) - half,
+                int(np.floor(e["bottom"])) - half,
+            )
+            boxes = [tuple(float(v) for v in b) for b in find_legend(rgb, box)]
+            if boxes:
+                found_legends.extend(boxes)
+                x0, y0 = min(b[0] for b in boxes), min(b[1] for b in boxes)
+                x1, y1 = max(b[2] for b in boxes), max(b[3] for b in boxes)
+                note(
+                    f"Panel {p['panel_id']}: a legend was found inside the plot area (x {x0:.0f}-{x1:.0f}, y "
+                    f"{y0:.0f}-{y1:.0f}) and left out. If that is not a legend, pass legend='keep'."
+                )
+        if found_legends:
+            exclude_boxes = list(exclude_boxes or []) + found_legends
+            parameters["legends_found"] = found_legends
+
     return PreparedFigure(
         rgb=rgb,
         dark=dark,

{trace_digitiser-2.1.0 → trace_digitiser-2.3.0}/src/trace_digitiser/axis_labels.py

@@ -176,13 +176,16 @@ class Preparation:
 # settled by the first (fast) attempt are therefore read again in other ways
 # and the readings pooled; the axis fit then keeps the readings that agree.
 # The settings and their order were chosen on labels 5 to 7 pixels high in
-# three typefaces: each one added reads labels that the ones before it miss.
+# three typefaces: each one added reads labels that the ones before it miss. The last was added for large print.
 PREPARATIONS = (
     Preparation(44, True, 2.0, True, True, 6),
     Preparation(64, False, 2.0, True, False, 7),
     Preparation(64, True, 0.0, False, False, 7, gamma=0.6),
     Preparation(32, False, 2.0, True, False, 7),
     Preparation(64, False, 0.0, False, False, 7, gamma=0.6),
+    # Large, clean print in some typefaces (MATLAB's Helvetica at 600 dpi) is misread as a line of text (75 as 15)
+    # and read right as a single word.
+    Preparation(64, False, 0.0, True, False, 8),
 )
 
 
@@ -658,6 +661,84 @@ def find_tick_labels(
             best = found
         if len(found) >= 2:
             break
+
+    # Labels set close together (or an axis title just below them, run into them) come out as one long piece. Where
+    # the axis has tick marks at a regular step, each label is looked for under its own tick instead: the first
+    # piece of print out from the axis in the stretch nearer that tick than any other.
+    origin = x_lo if axis == "x" else y_lo
+    key = {"x": "tick_cols", "y": "tick_rows", "y2": "tick_rows_right"}[axis]
+    ticks = sorted(t - origin for t in panel.get(key, []) if lo - 2 <= t - origin <= hi + 2)
+    lattice = _regular(ticks)
+    if len(lattice) >= 3:
+        step = float(np.median(np.diff(lattice)))
+        # (labels run together: one piece much longer than the rest, or one piece where there should be several -
+        # not merely labels longer than the step between minor ticks)
+        lengths = [((lab.box[2] - lab.box[0]) if not vertical else (lab.box[3] - lab.box[1])) + 1 for lab in best]
+        too_wide = bool(lengths) and (
+            (len(lengths) >= 2 and max(lengths) > 2.0 * float(np.median(lengths)) and max(lengths) > 1.1 * step)
+            or (len(lengths) == 1 and lengths[0] > 2.5 * step)
+        )
+        # (pieces of an axis title run in among the labels split them up: more pieces than ticks)
+        many = len(best) > 1.5 * (len(lattice) if len(lattice) >= 0.7 * len(ticks) else len(ticks))
+        if too_wide or len(best) < 0.6 * len(lattice) or many:
+            at_ticks = []
+            # under an x axis, the print of a label has no blank row across it, and a title below does; beside a
+            # y axis, the characters of a label stand a little apart
+            gap = max(1.0, 0.5 * glyph_h) if vertical else 1.0
+            for t in lattice:
+                a, b = int(math.ceil(t - 0.48 * step)), int(math.floor(t + 0.48 * step))
+                a, b = max(0, a), min(ink.shape[1] - 1, b)
+                if b - a < 2:
+                    continue
+                window = ink[:, a : b + 1]
+                rows = _merge_runs(_runs(window.any(axis=1)), max_gap=gap)
+                rows = [r for r in rows if window[r[0] : r[1] + 1].sum() >= 4]
+                if not rows or rows[0][0] > 4.0 * glyph_h + 0.12 * span:
+                    continue
+                da, db = rows[0]
+                cols = np.flatnonzero(window[da : db + 1].any(axis=0))
+                if cols[0] == 0 or cols[-1] == window.shape[1] - 1:
+                    continue  # it runs on past the stretch: a label of a tick further on (minor ticks between)
+                ca, cb = a + int(cols[0]), a + int(cols[-1])
+                if abs(0.5 * (ca + cb) - t) > 0.25 * step:
+                    continue  # not centred on this tick
+                if not 0.5 * glyph_h <= db - da + 1 <= 3.0 * glyph_h:
+                    continue
+                box = to_image_box(da, db, ca, cb)
+                at_ticks.append(
+                    TickLabel(
+                        pos=0.5 * (box[1] + box[3]) if vertical else 0.5 * (box[0] + box[2]),
+                        box=box,
+                        anchor=float(t + origin),
+                    )
+                )
+            if (
+                len(at_ticks) > len(best)
+                or (too_wide and len(at_ticks) >= 3)
+                or (many and len(at_ticks) >= max(3, 0.6 * len(lattice)))
+            ):
+                best = at_ticks
+    return best
+
+
+def _regular(positions: list[float]) -> list[float]:
+    """The longest run of *positions* at one regular step (to 4 %, or a pixel and a half): the tick marks along an
+    axis, without the ends of the axis or a gridline out of step."""
+    pos = sorted(positions)
+    best: list[float] = []
+    for i in range(len(pos)):
+        for j in range(i + 1, len(pos)):
+            step = pos[j] - pos[i]
+            if step < 4:
+                continue
+            run = [pos[i], pos[j]]
+            nxt = pos[j] + step
+            for q in pos[j + 1 :]:
+                if abs(q - nxt) <= max(1.5, 0.04 * step):
+                    run.append(q)
+                    nxt = q + step
+            if len(run) > len(best):
+                best = run
     return best
 
 
@@ -837,7 +918,10 @@ def _restore_lost_points(labels: list[TickLabel]) -> None:
     decimals = 0
     for lab in labels:
         whole = _whole_number(lab.text) if lab.text else None
-        if whole and len(whole[1]) > 1 and whole[1][0] == "0":
+        # (in print large enough to read, every reading of the label: "54" read once as "54" and twice as "04" is a
+        # misread 5, not a lost point; it is small print that loses its points)
+        small = lab.box[3] - lab.box[1] + 1 < 2 * SMALL_PRINT_PX
+        if whole and len(whole[1]) > 1 and whole[1][0] == "0" and (small or len(set(lab.readings)) == 1):
             decimals = max(decimals, len(whole[1]) - 1)
     if decimals == 0:
         return