cellpy 2.1.4__tar.gz → 2.1.5__tar.gz

cellpy-2.1.5/.issueflows/01-current-issues/issue1015_original.md

@@ -0,0 +1,7 @@
+# Issue #1015 — improve documentation
+
+Source: https://github.com/jepegit/cellpy/issues/1015
+
+I was looking at the summary_collector API docs and noticed that it is not any
+links that takes you to the code (on github). I have seen other docs where that
+is possible, and it is very helpful.

cellpy-2.1.5/.issueflows/01-current-issues/issue1015_plan.md

@@ -0,0 +1,21 @@
+# Issue #1015 — plan
+
+## Goal
+
+API reference pages let the reader see the code behind each documented
+object. Beyond the ask: iterate on the docs from a user's point of view
+(battery scientist with limited Python), one PR per iteration.
+
+## Iteration 1 (this PR): inline source
+
+Set mkdocstrings-python `show_source = true` in `zensical.toml`. Every class
+and function gets a collapsible "Source code in `<file>`" block with line
+numbers. A GitHub-link variant (template overrides) was prototyped and
+dropped: inline source answers the need without custom templates.
+
+## Later iterations (not started)
+
+Candidate topics, each its own branch/PR: troubleshooting page (common
+errors), "which loader for my tester" table, units/mass/nominal-capacity
+FAQ, glossary of columns (`c.schema`), batch quick-start for non-programmers,
+plotting cookbook, saving/exporting to Excel/CSV, configuration walkthrough.

cellpy-2.1.5/.issueflows/01-current-issues/issue1015_status.md

@@ -0,0 +1,15 @@
+# Issue #1015 — status
+
+- [ ] Done
+
+## What's done (iteration 1)
+
+- `zensical.toml`: `show_source = true` — collapsible source block under
+  every class and function in the API reference.
+- HISTORY entry.
+- Local build: `No issues found`.
+
+## Remaining work
+
+- Iterations 2–10 (user-perspective docs improvements) per plan file.
+- The issue's own ask is met after iteration 1 is merged.

cellpy-2.1.5/.issueflows/02-partly-solved-issues/issue_fix_cellpycore026_original.md

@@ -0,0 +1,14 @@
+# Iterative fixes: New cellpy-core release
+
+Source: local `/iflow-fix` session (GitHub issue create blocked in this Cloud
+Agent environment — `gh` is read-only). Session name: New cellpy-core release.
+
+## Original issue text
+
+Interactive `/iflow-fix` session whose individual fixes are recorded in the
+status markdown and landed together via `/iflow-close`.
+
+**First fix (from the invoking message):** Cellpy next: bump the `cellpycore`
+pin to the new PyPI release `0.2.6`, and drop the strict xfails on
+`test_empty_tail_is_noop` (core#147) and
+`test_gap_append_mid_step_equals_full_load` (core#148).

cellpy-2.1.5/.issueflows/02-partly-solved-issues/issue_fix_cellpycore026_status.md

@@ -0,0 +1,17 @@
+# Status — Iterative fixes: New cellpy-core release
+
+Interactive `/iflow-fix` session. GitHub issue was not created (`gh` write is
+blocked in this environment). Work lands on `cursor/cellpycore-pin-0-2-6-4828`.
+
+- [ ] Done
+
+## Iterative fixes log
+
+- **2026-09-08** — Pin `cellpycore` `0.2.5`→`0.2.6` in `pyproject.toml` /
+  `uv.lock`. Drop strict xfails on `test_empty_tail_is_noop` (core#147) and
+  `test_gap_append_mid_step_equals_full_load` (core#148). Update
+  `tests/README.md`, pin-gate doc, and `HISTORY.md`. Leave conda env YAMLs on
+  `0.2.4` (conda-forge latest).
+  - `uv sync --no-sources` → `cellpycore 0.2.6`
+  - `tests/test_incremental_update.py`: 7 passed (both former xfails now pass)
+  - `pytest -m essential`: 845 passed, 64 skipped, 0 xfailed

cellpy-2.1.5/.issueflows/03-solved-issues/issue1008_original.md

@@ -0,0 +1,7 @@
+# Issue #1008: nom cap specific lost
+
+Source: https://github.com/jepegit/cellpy/issues/1008
+
+## Original issue text
+
+When loading from xlsx database (batch), it did not pick up the nom cap specific value (gravimetric)

cellpy-2.1.5/.issueflows/03-solved-issues/issue1008_plan.md

@@ -0,0 +1,82 @@
+# Plan — issue #1008: nom cap specific lost (batch, xlsx db)
+
+Branch: `1008-batch-nom-cap-specifics`. Milestone `v2.1.4.post`.
+
+## Diagnosis (2026-09-08)
+
+Traced db → journal → spec → `cellpy.get` on master with the reporter's own
+config (`cellpy.toml`: `db_cols.nom_cap_specifics = "nom_cap_specifics"`) and db
+(`2025_Cell_Analysis_db_001.xlsx`, 976 gravimetric / 28 areal rows):
+
+- `Reader.get_nom_cap_specifics(id)` returns the sheet value.
+- `_dbengine._create_pages_dict` → `Journal.pages["nom_cap_specifics"]` →
+  `policy.resolve_specs` → `runner._get_kwargs` → `cellpy.get(nom_cap_specifics=…)`
+  keeps the value; journal JSON round-trip keeps it.
+- `batch.load()` end-to-end on the test db with an `areal` column: cells get
+  `nom_cap_specifics="areal"`, `cellpy_units.nominal_capacity="mAh/cm**2"` on
+  raw load, `.cellpy` AUTO reload and journal autoload.
+
+So the value is not lost by the pipeline itself. Two ways it shows up as `null`
+in `b.pages`, both silent today:
+
+1. **Journal autoload shadows the db.** `batch.load(name, project, …)` with
+   `allow_from_journal=True` (default) reuses `cwd/cellpy_batch_<name>.json`
+   when present and never opens the xlsx — logged at INFO only. A journal
+   written before the db column was filled (or by an older cellpy that did not
+   emit the column, e.g. `cellpy_batch_celf_mar.json` from July has no
+   `nom_cap_specifics` key) reads back as a `null` column. Even when the user
+   passes `db_reader=` / `batch_col=`, the stale journal still wins.
+2. **Configured column name ≠ sheet header.** `Reader._pick_info` swallows the
+   `KeyError` at `logging.debug` and returns `None` for every cell → `null`
+   column → default `gravimetric` at load. No warning.
+
+Side finding (out of scope, file follow-up): `journal_from_db(…, skip_file_search=True)`
+with the Excel reader crashes in `simple_db_engine` (`pd.DataFrame(pages_dict)`:
+`All arrays must be of the same length` — `raw_file_names` / `cellpy_file_name`
+stay `[]`).
+
+## Approach (KISS, no behaviour change on the happy path)
+
+1. `cellpy/readers/dbreader.py` — `_pick_info`: on missing column emit
+   `warnings.warn(...)` **once per column per reader instance** (keep the
+   `None` return so batches still build). Message names the configured key
+   (`config.db_cols.<field>`) and the sheet header it expected.
+2. `cellpy/batch/facade.py` — `_load_after_progress`: when the journal is
+   autoloaded **and** the caller signalled a db read (`db`, `db_reader`,
+   `reader`, `batch_col` or `reader_path` given), `warnings.warn` that the
+   database was not consulted and that `allow_from_journal=False` re-reads it.
+   Plain `batch.load(name, project)` keeps the INFO log (fast-path reopen).
+3. Tests (essential):
+   - `tests/test_batch_v3_facade.py`: autoload + `db_reader=` → `UserWarning`;
+     autoload without db args → no warning.
+   - `tests/test_dbreader.py` (or nearest existing dbreader test module):
+     `Reader(db_frame=…)` missing the configured column → one `UserWarning`,
+     second pick of same column silent; and a regression check that a present
+     `nominal_capacity_specifics` column flows into `_create_pages_dict`.
+4. Docs: `HISTORY.md` bullet (close step); one line in
+   `docs/getting_started/agents.md` batch bullet + `AGENTS.md` mirror:
+   "`batch.load` reuses `cellpy_batch_<name>.json` in cwd when present;
+   `allow_from_journal=False` forces a db read."
+5. Follow-up GitHub issue for the `skip_file_search=True` Excel crash.
+
+## Files
+
+- `cellpy/readers/dbreader.py`, `cellpy/batch/facade.py`
+- `tests/test_batch_v3_facade.py`, `tests/test_dbreader*.py`
+- `docs/getting_started/agents.md`, `AGENTS.md`, `HISTORY.md`
+
+## Constraints
+
+- No change to journal schema or defaults; `null` stays `null` (honest).
+- Warnings, not exceptions — batches must still build with a partial db.
+
+### Prior art
+
+- `_dbengine.find_files` filefinder-miss `UserWarning` (#964) — same style.
+- `#1000` journal `version` warning path in `journal.read_journal`.
+
+## Open questions
+
+- Should `db_reader=`/`batch_col=` given explicitly make the db **win** over the
+  autoloaded journal instead of just warning? Plan says warn only (smaller,
+  no surprise reload); flip to "db wins" if preferred.

cellpy-2.1.5/.issueflows/03-solved-issues/issue1008_status.md

@@ -0,0 +1,28 @@
+# Issue #1008 status — nom cap specific lost (batch, xlsx db)
+
+- [x] Done
+
+Branch: `1008-batch-nom-cap-specifics`. Plan accepted 2026-09-08 (warn-only variant).
+
+## What's done
+
+- Diagnosis: pipeline keeps the value (verified db → journal → spec →
+  `cellpy.get` → cell, incl. `.cellpy` reload and journal autoload). `null`
+  in `b.pages` comes from a cached `cellpy_batch_<name>.json` shadowing the
+  db (INFO-only before), or from a configured-vs-sheet column-name mismatch
+  swallowed in `Reader._pick_info`.
+- `cellpy/readers/dbreader.py`: `_warn_missing_column` — one `UserWarning`
+  per missing sheet header per reader, naming the `config.db_cols.<key>`.
+- `cellpy/batch/facade.py`: `_load_after_progress` warns when the journal is
+  autoloaded but `db` / `db_reader` / `batch_col` / `reader_path` / other db
+  kwargs were given.
+- Tests (essential): `test_dbreader.py::test_missing_column_warns_once`,
+  `::test_nom_cap_specifics_column_reaches_pages`;
+  `test_batch_v3_facade.py::test_load_warns_when_journal_autoload_shadows_db`,
+  `::test_load_autoload_without_db_args_is_quiet`.
+- Docs: `docs/getting_started/agents.md` batch recipe + `AGENTS.md` mirror.
+- Follow-up filed: #1017 (`skip_file_search=True` crashes with Excel reader).
+
+## Remaining work
+
+- None.

cellpy-2.1.5/.issueflows/03-solved-issues/issue1009_original.md

@@ -0,0 +1,7 @@
+# Issue #1009: summary collector plot family bug
+
+Source: https://github.com/jepegit/cellpy/issues/1009
+
+## Original issue text
+
+for the summary_collector family "fullcell_standard_gravimetric" we get too many sub-windows.

cellpy-2.1.5/.issueflows/03-solved-issues/issue1009_plan.md

@@ -0,0 +1,91 @@
+# Issue #1009 — plan (rev 2)
+
+## Diagnosis
+
+`summary_collector(b, family="fullcell_standard_gravimetric").plot()` draws 10
+facets instead of the 4 the family declares, and even the declared layout is
+one facet *per variable* where the user wants charge and discharge of the same
+quantity in **one** panel.
+
+Root causes:
+
+1. `PlotFamily.summary_options` (`cellpy/plotting/registry.py`) strips the
+   `_cv` / `_non_cv` suffix off declared columns and requests the *base*
+   column with `partition_by_cv=True` — "only the CV part" is lost.
+2. `collect_summaries` (`cellpy/collect/summary.py`) then keeps
+   `col`, `col_non_cv`, `col_cv` for **every** requested column, including
+   `coulombic_efficiency`.
+3. `summary_plotter` (`cellpy/plotting/collected.py`) facets on `variable`,
+   so `charge_x` and `discharge_x` always land in separate rows.
+
+Audit over all 20 `summary_plot` families (two `.res` cells, in-memory
+batch): the 13 `capacities*` / `voltages` families collect exactly their
+declared columns; all 7 `fullcell_standard*` families collect 10–12.
+Single-cell `summary_plot(y=...)` is unaffected by 1–2 (selects `y_cols`
+explicitly) and is out of scope for 3.
+
+## Approach
+
+### A. Collect exactly the declared columns
+
+1. `summary_options`: keep declared names literally in `columns` (no suffix
+   stripping); still set `partition_by_cv` when a `_cv` / `_non_cv` name (or
+   `supports_cv_split`) is present. `mod_01_*` handling unchanged.
+2. `collect_summaries`: expand `col → (col, col_non_cv, col_cv)` only when the
+   requested list names no CV variant itself; a list that spells out
+   `*_cv` / `*_non_cv` is taken literally. Existing
+   `columns=("charge_capacity",), partition_by_cv=True` still expands.
+
+### B. Combine charge/discharge into one panel (`summary_plotter`, Plotly)
+
+3. Panel key: split `variable` on `_`, drop the first token equal to
+   `charge` / `discharge` → panel; that token is the `direction`. Works for
+   prefix (`charge_capacity_gravimetric_cv`), suffix
+   (`potential_end_charge`) and mid-name
+   (`test_cumulated_discharge_capacity_loss_gravimetric`,
+   `mod_01_discharge_capacity_gravimetric`). No token → panel = variable,
+   direction none.
+4. Before the backend call, rewrite `variable` to the panel key and add a
+   `direction` column. Everything keyed on `variable` (facets, `y_ranges`,
+   `order_variables`, label mapper, `spread_plot`) keeps working on panel
+   keys; `y_ranges` / `order_variables` given as original variable names are
+   translated the same way.
+5. `px.line(..., line_dash="direction", line_dash_map={charge: solid,
+   discharge: dash})` when any direction is present. Trace names reduced to
+   the series (cell / group) with one legend entry per series; a second
+   Plotly legend (`legend2`, title "Direction") carries two style-only
+   entries: Charge (solid) / Discharge (dashed). `spread_plot` (grouped
+   mean/std) gets the same dash per direction.
+6. Panel labels: `_pretty_variable_label` must recognise `capacity_gravimetric`
+   (no direction prefix) so units still appear → "Capacity (mAh/g)".
+7. Opt-out: `combine_directions=False` on `summary_plotter` /
+   `Collection.plot` restores one facet per variable. Default **on** (this is
+   the requested layout; noted in HISTORY).
+8. Seaborn backend: facets follow the panel rewrite; `style="direction"` only
+   when style is free (not `group_cells`). Matplotlib/bokeh summary paths
+   untouched.
+
+Resulting facet counts: `fullcell_standard_*` 4 (CE, capacity, retention,
+CV part); `capacities_*` 1; `*_coulombic_efficiency` 2; `*_with_rate` 2;
+`*_split_constant_voltage` 3; `voltages` 1.
+
+### C. Tests (essential)
+
+- Registry/collect: tighten the family oracle in `tests/test_collect.py` to
+  reject *extra* columns; literal-list case for `collect_summaries`.
+- Plot: unit tests for the panel/direction split; Plotly facet count and
+  dash mapping on a synthetic frame; `legend2` present with two entries;
+  `combine_directions=False` gives per-variable facets; `y_ranges` keyed by
+  original variable still applies.
+- Update `tests/test_collected_summary_groups.py` fixtures that used
+  `cap_charge` / `cap_discharge` as separate facets (they now merge).
+
+### D. Docs
+
+`HISTORY.md` `[Unreleased]`, `AGENTS.md` / `docs/getting_started/agents.md`
+batch bullet (combined panels + `combine_directions`), `test-registry.md`.
+
+## Constraints
+
+- No new `SummaryOptions` fields; family declarations unchanged.
+- Legacy `plot_cycle_life_summary_plotly` (Batch `b.plot()`) untouched.

cellpy-2.1.5/.issueflows/03-solved-issues/issue1009_status.md

@@ -0,0 +1,56 @@
+# Issue #1009 — status
+
+- [x] Done
+
+## Diagnosis
+
+`summary_collector(b, family="fullcell_standard_gravimetric").plot()` drew 10
+facets instead of 4: `summary_options` stripped the `_cv` suffix and asked for
+the base column with `partition_by_cv=True`; `collect_summaries` then kept
+`col`, `col_non_cv`, `col_cv` for every requested column (CE included).
+All 7 `fullcell_standard*` families were affected (10–12 columns vs 4); the 13
+other summary families were exact. On top of that the plot faceted one row per
+variable, so charge and discharge of the same quantity never shared a panel.
+
+## Done
+
+- `cellpy/plotting/registry.py` — `summary_options` requests declared columns
+  literally (still turns `partition_by_cv` on for `*_cv` / `*_non_cv`).
+- `cellpy/collect/summary.py` — the `(col, col_non_cv, col_cv)` expansion only
+  applies when the requested list names no CV variant itself.
+  `SummaryOptions` docstring updated.
+- `cellpy/plotting/collected.py` (`summary_plotter`, Plotly):
+  - `split_direction()` / `_panel_mapping()`: variables differing only by a
+    `charge` / `discharge` token (prefix, suffix or mid-name) share a panel
+    named by the token-free key; a variable alone on its key keeps its name.
+  - `line_dash="direction"` (charge solid, discharge dashed); trace names
+    reduced to the series with one legend entry each; second legend
+    `legend2` ("Direction"). `spread_plot` gets the same dashes.
+  - `order_variables`, `y_ranges`, `y_label_mapper` accept original variable
+    names or panel keys; `_yaxis_key_for_variable` falls back to the panel key.
+  - `combine_directions=False` opt-out.
+  - Labels: `capacity_gravimetric` (direction-less panel key) gets units;
+    `*_non_cv` → "… non-CV" with correct mode/unit; `mod_01_*` →
+    "Normalized … (%)".
+- Tests: oracle in `tests/test_collect.py` now rejects extra columns; literal
+  CV list test; `summary_options` tuple test; new
+  `tests/test_collected_summary_directions.py` (8 essential tests);
+  `tests/test_collected_summary_groups.py` synthetic names neutralised and the
+  #947/#948 snippets updated for the merged capacity panel;
+  `test_collected_summary_axes.py` spread hover test skips `legend2` entries.
+- Docs: `HISTORY.md`, `AGENTS.md`, `docs/getting_started/agents.md`,
+  `.issueflows/04-designs-and-guides/test-registry.md`.
+
+## Resulting facet counts
+
+`fullcell_standard_*` 4 (CV part / capacity / CE / normalized); `capacities_*`
+1; `*_coulombic_efficiency` 2; `*_with_rate` 2; `*_split_constant_voltage` 3;
+`voltages` 1. Verified with an in-memory batch of the two Arbin `.res` test
+cells and rendered PNGs.
+
+## Out of scope / notes
+
+- Seaborn / matplotlib summary backends: facets follow the panel rewrite, no
+  dash styling.
+- Legacy `b.plot()` (`plot_cycle_life_summary_plotly`) and single-cell
+  `summary_plot(y=...)` untouched.

cellpy-2.1.5/.issueflows/03-solved-issues/issue1017_original.md

@@ -0,0 +1,17 @@
+# Issue #1017: journal_from_db(skip_file_search=True) crashes with the Excel reader
+
+Source: https://github.com/jepegit/cellpy/issues/1017
+
+## Original issue text
+
+Found while diagnosing #1008.
+
+`cellpy.batch.db.journal_from_db(name, project, db_reader="simple_excel_reader", skip_file_search=True)` raises
+
+```
+ValueError: All arrays must be of the same length
+```
+
+from `_dbengine.simple_db_engine` at `pd.DataFrame(pages_dict)`: `_create_pages_dict` seeds `raw_file_names` / `cellpy_file_name` as `[]` and only `find_files` fills them, so skipping the search leaves two zero-length columns next to the per-cell ones.
+
+`skip_file_search` is documented for JSON readers that already carry the file columns, but the Excel path should either fill those columns with `None` per cell or reject the flag with a clear message.

cellpy-2.1.5/.issueflows/03-solved-issues/issue1017_plan.md

@@ -0,0 +1,77 @@
+# Issue #1017 — plan
+
+## Goal
+
+`journal_from_db(..., skip_file_search=True)` must return a journal for every
+reader instead of raising `ValueError: All arrays must be of the same length`.
+Cells whose files were not searched get `None` in `raw_file_names` /
+`cellpy_file_name` (same shape as a filefinder miss).
+
+## Constraints
+
+- Toolchain: `uv run pytest` (see `this-project.md`); merge gate is `-m essential`.
+- No behaviour change for the default `skip_file_search=False` path.
+- Do not change the JSON readers' public `pages_dict` contract (they seed the
+  two file columns as `[]` too — `BatBaseJSONReader` and `CustomJSONReader`,
+  `cellpy/readers/json_dbreader.py`); fix once, at the seam both paths share.
+- Keep `_create_pages_dict` seeding as is (tests in `tests/test_dbreader.py`
+  and `tests/test_batch.py` read it directly); `find_files` is the only place
+  that knows whether the columns will be filled.
+- `HISTORY.md` `[Unreleased]` bullet at close (`v2.1.4.post` milestone).
+
+### Prior art
+
+- `find_files` (`cellpy/batch/_dbengine.py`) already seeds `[]` when the
+  columns are absent and appends per cell; the `skip_file_search` early return
+  is the only branch that returns without normalising lengths → mirror the
+  seeding there, padded to the cell count.
+- Filefinder miss convention: `raw_files = None`, `cellpyfile = None` per cell
+  (`find_files` loop) → reuse `None` as the "not searched" value so
+  `policy.py` / `facade.py` (`row.get(...)`, `_clean`, FAILED marking) keep
+  working unchanged.
+- `tests/test_batch.py::test_find_files_skip_file_search` — existing unit test
+  for the skip branch with pre-filled columns; extend alongside it.
+- Toolbox (`.issueflows/00-tools/`): nothing applicable. `graphify-out/` absent.
+
+## Approach
+
+1. In `find_files`, replace the bare `if skip_file_search: return info_dict`
+   with: compute `n = len(file_name_indicators)` (fallback `filename`), then for
+   each of `hdr_journal["raw_file_names"]` / `hdr_journal["cellpy_file_name"]`:
+   if the key is missing or its list is empty while `n > 0`, set it to
+   `[None] * n`. Pre-filled columns (JSON carrying paths) stay untouched.
+2. Docstrings: `find_files` (skip now pads instead of "returned unchanged") and
+   `journal_from_db` in `cellpy/batch/db.py` (`skip_file_search=True` also
+   valid for the Excel reader; file columns come back `None`, so `b.update()`
+   marks those cells `FAILED` until paths are filled or a search is run).
+3. No change to `_create_pages_dict` or the JSON readers.
+
+Alternative rejected: raising on `skip_file_search=True` for the Excel reader —
+users legitimately want a journal without a (slow/remote) file search and fill
+paths afterwards; `None` per cell is the existing "not found" shape.
+
+## Files to touch
+
+- `cellpy/batch/_dbengine.py` — `find_files` skip branch pads the two columns;
+  docstring.
+- `cellpy/batch/db.py` — `journal_from_db` docstring wording.
+- `tests/test_batch.py` — new `test_find_files_skip_file_search_pads_missing_columns`
+  (missing + empty columns → `[None] * n`; `n == 0` → `[]`).
+- `tests/test_dbreader.py` — `@pytest.mark.essential`
+  `test_simple_db_engine_skip_file_search_excel_reader`: `simple_db_engine(reader,
+  ids, skip_file_search=True)` on the Excel fixture returns one row per id with
+  `None` file columns (reproduces #1017 before the fix).
+- `HISTORY.md` — `[Unreleased]` bullet (at `/iflow-close`).
+
+## Test strategy
+
+- `uv run pytest tests/test_dbreader.py tests/test_batch.py -q`
+- `uv run pytest -m essential` (merge gate)
+- Repro before fix: the new engine test fails with `ValueError: All arrays
+  must be of the same length`.
+
+## Open questions
+
+- None. (Optional: also make `journal_from_db` log at INFO that file columns
+  are unset when `skip_file_search=True` and the reader is the Excel one —
+  skipped for KISS unless wanted.)

cellpy-2.1.5/.issueflows/03-solved-issues/issue1017_status.md

@@ -0,0 +1,31 @@
+# Issue #1017 status
+
+`journal_from_db(skip_file_search=True)` crashes with the Excel reader.
+
+- [x] Done
+
+PR: https://github.com/jepegit/cellpy/pull/1020 (#1020, draft)
+Branch: `cursor/1017-journal-from-db-skip-file-search-3975`
+
+## What's done
+
+- Issue captured, plan confirmed (`issue1017_plan.md`).
+- `_dbengine.find_files`: the `skip_file_search=True` branch pads missing or
+  empty `raw_file_names` / `cellpy_file_name` with `[None] * n_cells`
+  (pre-filled JSON paths untouched). Docstring updated.
+- `batch.db.journal_from_db` docstring: flag valid for any reader; unset file
+  columns are `None`, `update()` marks those cells `FAILED`.
+- Tests: `tests/test_dbreader.py::test_simple_db_engine_skip_file_search_excel_reader`
+  (essential; reproduced `ValueError: All arrays must be of the same length`
+  before the fix) and
+  `tests/test_batch.py::test_find_files_skip_file_search_pads_missing_columns`.
+- `uv run pytest tests/test_dbreader.py tests/test_batch.py`: 71 passed.
+- `uv run pytest -m essential`: 829 passed, 64 skipped
+  (`tests/test_arbin_variants_two_stage.py` ignored locally — `pyodbc` needs
+  `libodbc.so.2`, missing on this VM; unrelated to the change).
+
+- `HISTORY.md` `[Unreleased]` bullet; `test-registry.md` rows.
+
+## Remaining work
+
+- None. Merge PR #1020 when CI is green (no version bump requested).

cellpy-2.1.5/.issueflows/03-solved-issues/issue778_original.md

@@ -0,0 +1,9 @@
+# Issue #778: L6: golden equality test — incremental update() == full load
+
+Source: https://github.com/jepegit/cellpy/issues/778
+
+## Original issue text
+
+Epic L of **cellpy 2.2 (Stage 5)**. Design: [live-incremental](https://github.com/cellpy/architecture-plan/blob/main/cellpy2-live-incremental-design.md) §7 item 6. **Author first** — this is the correctness anchor for the whole epic.
+
+Load a truncated file, append the tail, and assert `update()` == a full load of the whole file (summary equality). *Incremental refresh of a split file must equal a full load of the whole file.*

cellpy-2.1.5/.issueflows/03-solved-issues/issue778_plan.md

@@ -0,0 +1,129 @@
+# Issue #778 plan — L6: golden equality test (incremental update == full load)
+
+## Goal
+
+Add the Stage 5 Epic L correctness anchor: a test proving that loading a
+**truncated** tester file and then feeding the **tail rows** through the
+incremental-update path yields the same `raw` / `steps` / `summary` as one
+**full load** of the whole file. Written first, against the shipped core
+primitive (`update_core_data`), so L1–L5 (#779/#780/#164/#781/#782) have an
+oracle to land against.
+
+## Constraints
+
+- Test-only issue: **no library code changes**. `CellpyCell.update()` and
+  `load_since()` do not exist yet (L3/L2); the test drives the seam that does
+  exist — `c.core.update_core_data(data, new_raw, …)` (cellpycore 0.2.5
+  `merge.update_data`). The helper that produces the "tail" is written so L3
+  can swap it for `c.update()` in one place.
+- Must run on the merge gate (`@pytest.mark.essential`) → fast, no mdbtools:
+  use a **text** fixture. `testdata/data/neware_uio.csv` (`instrument="neware_txt"`,
+  `model="UIO"`): 1 header line + 9064 rows, 4 cycles / 32 steps, `datapoint_num`
+  1…9065, loads in ~0.4 s. (`maccor_001.txt` is 5 s and has 1 cycle — not the
+  anchor; `custom_data_001.csv` has non-integer `datapoint_num` — unsuitable.)
+- Truncated files are written to `tmp_path`, never into `testdata/` (input data
+  read-only).
+- `update_data` contract (core): `new_raw` may **overlap** the tail; overlap is
+  trimmed on `source_datapoint_num` → falls back to `datapoint_num` (cellpy
+  loaders set no `source_datapoint_num`). Rejects `new_raw` starting at/before
+  the existing range. Both branches (`gap_append` vs overlap) exercised.
+- Per-cell frames are **pandas** in cellpy; `update_core_data` converts.
+  Comparisons use `pandas.testing` with a small float tolerance, columns
+  aligned by name, `test_id` normalised.
+- Summary scope **now**: `update_core_data(refresh_derived=True)` produces the
+  *core* summary (cycle-end columns + C-rates + IR). cellpy's `make_summary`
+  additionally runs `add_scaled_summary_columns` (gravimetric/areal, equivalent
+  cycles). Until L3 defines what `c.update()` does after the core step, the
+  summary assertion compares the **intersection of columns**; the test records
+  the missing columns so L3 flips it to full-column equality. `raw` and `steps`
+  are compared on the **full** column set.
+
+### Prior art
+
+- `cellpycore.merge.update_data` / `CellpyCellCore.update_core_data`
+  (`../cellpy-core/src/cellpycore/merge.py:173`, `cell_core.py:528`) — the
+  engine under test; core has its own unit tests on synthetic frames, none
+  through a real loader.
+- `CellpyCell.merge(cells, mode="campaign")` (`cellpy/readers/cellreader.py:1852`)
+  — the *campaign* merge (renumber cycles); not the incremental path. Coexist.
+- `tests/parity.py::assert_value_parity` — legacy↔native comparator with
+  named exceptions; heavier than needed here (same-schema frames). Mirror its
+  spirit (explicit tolerance, no silent widening) with `pandas.testing`.
+- `tests/loader_golden_support.py` / `tests/golden_support.py` — committed
+  golden snapshots. **Not** used: this oracle is *self-referential* (full load
+  is the reference), so no new golden files and no `regenerate_goldens.py`
+  suite.
+- `tests/test_cellpy_splitting.py` (`cell.split`, `drop_to`) — cycle-level
+  splitting of a loaded cell; different axis (cycles, not file rows). Coexist.
+- `.issueflows/00-tools/` — nothing applicable (AST scanners, prms migrator).
+
+## Approach
+
+1. **`tests/incremental_support.py`** (small, reused by L2/L3/L5 tests later):
+   - `truncate_text_file(src, dst, *, n_data_rows, header_lines=1)` → writes
+     header + first `n_data_rows` data lines. Returns `dst`.
+   - `tail_rows(full_raw, *, since, overlap, datapoint_col)` → pandas slice
+     `datapoint_num > since - overlap` — this is what a future `load_since`
+     returns for marker `since`; the only place L3 has to replace.
+   - `incremental_update(head_cell, new_raw)` → today:
+     `head_cell.core.update_core_data(head_cell.data, new_raw, nom_cap_abs=…,
+     current_conversion_factor=…)` with the same factors cellpy's
+     `make_summary` computes (`core_units.calculate_current_conversion_factor`,
+     nominal capacity from `head_cell.data.meta`). Returns the new `Data`.
+   - `assert_frames_equal(left, right, *, columns=None, rtol)` thin wrapper
+     over `pandas.testing.assert_frame_equal` (reset index, sort by
+     `datapoint_num` / `cycle_num`, `check_dtype=False`).
+2. **`tests/test_incremental_update.py`**:
+   - Fixtures: `full_cell` (module-scoped load of `neware_uio.csv`);
+     `head_cell(tmp_path, n)` via `truncate_text_file` + `cellpy.get`.
+   - `test_head_matches_full_prefix` — sanity: head `raw` == first `n` rows of
+     full `raw` (proves the truncation + `datapoint_num` assignment are
+     deterministic; without this the oracle proves nothing).
+   - `test_incremental_update_equals_full_load[cut]` — parametrised over cut
+     points: **mid-step** inside cycle 2, **on a step boundary**, **on a cycle
+     boundary**; `overlap=0` (gap-append branch). Assert `raw` full equality,
+     `steps` full equality, `summary` equality on shared columns.
+   - `test_incremental_update_with_overlap` — same cut, `overlap=50` rows
+     (trim branch).
+   - `test_incremental_update_noop_on_empty_tail` — empty `new_raw` returns an
+     equal copy (poll tick with nothing new).
+   - `test_summary_column_gap_is_documented` — asserts the set of full-summary
+     columns missing after the core-only update equals a **named** list
+     (`add_scaled_summary_columns` outputs). When L3 lands and closes the gap,
+     this test is deleted and the main test flips to full-column equality.
+   - Marker `@pytest.mark.essential` on the equality tests (golden-oracle rule
+     in `this-project.md`); whole file well under 5 s.
+3. **`tests/README.md`** — one short subsection "Incremental-update oracle
+   (Stage 5 L6)" stating the invariant, the fixture, and the L3 flip rule.
+4. **Status file** `issue778_status.md`; HISTORY entry at close (test-only —
+   a one-liner under Unreleased is enough).
+
+## Files to touch
+
+| Path | Change |
+|---|---|
+| `tests/incremental_support.py` | new — truncate / tail / update / compare helpers |
+| `tests/test_incremental_update.py` | new — the L6 oracle tests (essential) |
+| `tests/README.md` | add "Incremental-update oracle" subsection |
+| `.issueflows/01-current-issues/issue778_status.md` | new |
+
+No changes under `cellpy/`.
+
+## Test strategy
+
+- `uv run pytest tests/test_incremental_update.py -q` during build.
+- `uv run pytest -m essential` before close (merge gate).
+- If a cut point exposes a real `update_data` defect (e.g. step-table rows at
+  the cut not refreshed identically), **do not** widen tolerance or drop the
+  case: record it in the status file and open a `cellpy-core` issue — that is
+  precisely what the oracle is for.
+
+## Open questions
+
+1. Summary scope: OK to compare **shared columns only** now (with the
+   documented gap test) and defer full-column equality to L3 (#164)? The
+   alternative — calling cellpy `make_summary()` after the core update — would
+   make the summary assertion trivially true and is rejected.
+2. Second fixture: add `maccor_001.txt` as a non-essential parametrisation
+   (5 s, single cycle) for loader diversity, or keep neware-only? Proposal:
+   neware-only now; L2 (#780) adds per-loader cases when `load_since` exists.