rosetta-eclab 0.1.1__tar.gz → 0.1.2__tar.gz

{rosetta_eclab-0.1.1 → rosetta_eclab-0.1.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: rosetta-eclab
-Version: 0.1.1
+Version: 0.1.2
 Summary: Name the columns of BioLogic EC-Lab .mpr files that your parser does not know - no language model, evidence for every answer
 Author: Nataniel Ataseven
 License-Expression: GPL-3.0-or-later
@@ -39,7 +39,7 @@ drift.mpr
     evidence: relation: I*dt summed within half cycle: r2 1.0000, scale 0.0002778
     also considered: dQ, step time
 
-  self-check: hid each of 11 known columns in turn; named 11, 11 correctly, 0 wrong, the rest abstained
+  self-check: hid each of 11 known columns in turn; named 11, 11 correctly, 0 wrong, 0 abstained
 ```
 
 *`drift.mpr` is `gcpl.mpr` from the yadg test suite, with one column ID changed to 950, an ID no parser knows (see `tests/test_cli.py`).*
@@ -76,6 +76,28 @@ Each unknown column gets one of three answers:
 
 `--self-check` shows how well Rosetta knows *your* kind of file. It hides each known column in turn and tries to name it.
 
+## If galvani stops with "Column ID … is unknown"
+
+In files from EC-Lab 11.50 and later, galvani 0.5.0 reads only the low byte of each column ID. IDs of 256 and above therefore turn into other IDs:
+
+| Real ID | Column | galvani 0.5.0 sees |
+|---|---|---|
+| 471 | `<Ece>/V` | 215 |
+| 438 | `step time/s` | 182 |
+| 431–433 | `\|Zwe-ce\|`, `Re(Zwe-ce)`, `-Im(Zwe-ce)` | 175–177 |
+
+The column is often one that galvani already knows. Rosetta reads the full IDs and says so:
+
+```
+$ rosetta peis.issue_225.with_ece.mpr
+peis.issue_225.with_ece.mpr
+  technique PEIS, 32 rows, 112 bytes per record
+  galvani 0.5.0 reads only the low byte of each column ID in this file layout, so it misreads 471 as 215, 430 as 174, 431 as 175, 432 as 176, 433 as 177
+  every column ID is in galvani's column map; nothing to name
+```
+
+The fix was released in datalab-org-galvani 0.6.0 ([galvani-mirror#11](https://github.com/datalab-org/galvani-mirror/issues/11)). IDs that are missing from the map altogether, such as 880 or 883 and 884, are what Rosetta names.
+
 ## How it works
 
 1. **Read the file** (`namer/mprfile.py`). Columns the reference parser knows are typed from its column map. For unknown IDs the byte width is solved from the record size, and the storage type is chosen by plausibility.

{rosetta_eclab-0.1.1 → rosetta_eclab-0.1.2}/README.md

@@ -14,7 +14,7 @@ drift.mpr
     evidence: relation: I*dt summed within half cycle: r2 1.0000, scale 0.0002778
     also considered: dQ, step time
 
-  self-check: hid each of 11 known columns in turn; named 11, 11 correctly, 0 wrong, the rest abstained
+  self-check: hid each of 11 known columns in turn; named 11, 11 correctly, 0 wrong, 0 abstained
 ```
 
 *`drift.mpr` is `gcpl.mpr` from the yadg test suite, with one column ID changed to 950, an ID no parser knows (see `tests/test_cli.py`).*
@@ -51,6 +51,28 @@ Each unknown column gets one of three answers:
 
 `--self-check` shows how well Rosetta knows *your* kind of file. It hides each known column in turn and tries to name it.
 
+## If galvani stops with "Column ID … is unknown"
+
+In files from EC-Lab 11.50 and later, galvani 0.5.0 reads only the low byte of each column ID. IDs of 256 and above therefore turn into other IDs:
+
+| Real ID | Column | galvani 0.5.0 sees |
+|---|---|---|
+| 471 | `<Ece>/V` | 215 |
+| 438 | `step time/s` | 182 |
+| 431–433 | `\|Zwe-ce\|`, `Re(Zwe-ce)`, `-Im(Zwe-ce)` | 175–177 |
+
+The column is often one that galvani already knows. Rosetta reads the full IDs and says so:
+
+```
+$ rosetta peis.issue_225.with_ece.mpr
+peis.issue_225.with_ece.mpr
+  technique PEIS, 32 rows, 112 bytes per record
+  galvani 0.5.0 reads only the low byte of each column ID in this file layout, so it misreads 471 as 215, 430 as 174, 431 as 175, 432 as 176, 433 as 177
+  every column ID is in galvani's column map; nothing to name
+```
+
+The fix was released in datalab-org-galvani 0.6.0 ([galvani-mirror#11](https://github.com/datalab-org/galvani-mirror/issues/11)). IDs that are missing from the map altogether, such as 880 or 883 and 884, are what Rosetta names.
+
 ## How it works
 
 1. **Read the file** (`namer/mprfile.py`). Columns the reference parser knows are typed from its column map. For unknown IDs the byte width is solved from the record size, and the storage type is chosen by plausibility.

{rosetta_eclab-0.1.1 → rosetta_eclab-0.1.2}/namer/cli.py

@@ -88,7 +88,7 @@ def main(argv: list[str] | None = None) -> int:
     for n in m.notes:
         print(f"  {n}")
     if not result["unknown_columns"]:
-        print(f"  every column is known to {a.reference}; nothing to name")
+        print(f"  every column ID is in {a.reference}'s column map; nothing to name")
     for u in result["unknown_columns"]:
         if u["verdict"] == "NAME":
             head = f"{u['name']} [{u['unit'] or 'no unit'}]  (confidence {u['confidence']})"
@@ -103,7 +103,8 @@ def main(argv: list[str] | None = None) -> int:
     if a.self_check:
         s = result["self_check"]
         print(f"\n  self-check: hid each of {s['hidden_one_at_a_time']} known columns in turn; named {s['named']}, "
-              f"{s['named_correctly']} correctly, {len(s['wrong'])} wrong, the rest abstained")
+              f"{s['named_correctly']} correctly, {len(s['wrong'])} wrong, "
+              f"{s['hidden_one_at_a_time'] - s['named']} abstained")
         for w in s["wrong"]:
             print(f"    wrong: {w['column']} -> {w['answer']}")
     return 0

{rosetta_eclab-0.1.1 → rosetta_eclab-0.1.2}/namer/mprfile.py

@@ -79,6 +79,23 @@ def _plausibility(v: np.ndarray, dtype: str) -> float:
     return 0.9 if span < 2 ** (8 * width - 4) else 0.2
 
 
+def _galvani_low_byte_note(full_ids: list[int], gmap: dict) -> list[str]:
+    """galvani 0.5.0 reads only the low byte of each column ID in the 11.50+ layout.
+
+    IDs above 255 then become other IDs (471 -> 215), which galvani either does not
+    know (it stops) or knows as a different column. Its map repeats a few entries at
+    the low-byte ID (211 = 467), so only IDs whose low-byte entry differs are listed.
+    """
+    misread = [x for x in full_ids if x > 255 and gmap.get(x & 0xFF) != gmap.get(x)]
+    if not misread:
+        return []
+    from importlib.metadata import version
+
+    pairs = ", ".join(f"{x} as {x & 0xFF}" for x in misread)
+    return [f"galvani {version('galvani')} reads only the low byte of each column ID in this file layout, "
+            f"so it misreads {pairs}"]
+
+
 def read(path: str | Path, reference: str = "galvani") -> MprData:
     from galvani import BioLogic
     from yadg.extractors.eclab.mpr import parse_columns, process_settings
@@ -117,6 +134,7 @@ def read(path: str | Path, reference: str = "galvani") -> MprData:
 
     names, dtypes, units, _ = parse_columns(yids, technique)  # yadg's map: names for known columns
     gmap = {int(k): v for k, v in BioLogic.VMPdata_colID_dtype_map.items()}
+    notes = _galvani_low_byte_note(full, gmap) if version in (10, 11) and reference == "galvani" else []
     layout = []  # record layout in file order: (raw_id or None for the flags byte, ...)
     for yid, fid in zip(yids, full):
         if yid in FLAG_IDS:  # all flag IDs share one byte, at the position of the first one
@@ -146,7 +164,6 @@ def read(path: str | Path, reference: str = "galvani") -> MprData:
     known_width = sum(np.dtype(d).itemsize for _, _, _, d, kn in filled if kn)
     unknown_idx = [i for i, e in enumerate(filled) if not e[4]]
     rest = rec - known_width
-    notes = []
     best = None
     if unknown_idx:
         combos = [c for c in itertools.product((1, 2, 4, 8), repeat=len(unknown_idx)) if sum(c) == rest]

{rosetta_eclab-0.1.1 → rosetta_eclab-0.1.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "rosetta-eclab"
-version = "0.1.1"
+version = "0.1.2"
 description = "Name the columns of BioLogic EC-Lab .mpr files that your parser does not know - no language model, evidence for every answer"
 readme = "README.md"
 license = "GPL-3.0-or-later"

{rosetta_eclab-0.1.1 → rosetta_eclab-0.1.2}/rosetta_eclab.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: rosetta-eclab
-Version: 0.1.1
+Version: 0.1.2
 Summary: Name the columns of BioLogic EC-Lab .mpr files that your parser does not know - no language model, evidence for every answer
 Author: Nataniel Ataseven
 License-Expression: GPL-3.0-or-later
@@ -39,7 +39,7 @@ drift.mpr
     evidence: relation: I*dt summed within half cycle: r2 1.0000, scale 0.0002778
     also considered: dQ, step time
 
-  self-check: hid each of 11 known columns in turn; named 11, 11 correctly, 0 wrong, the rest abstained
+  self-check: hid each of 11 known columns in turn; named 11, 11 correctly, 0 wrong, 0 abstained
 ```
 
 *`drift.mpr` is `gcpl.mpr` from the yadg test suite, with one column ID changed to 950, an ID no parser knows (see `tests/test_cli.py`).*
@@ -76,6 +76,28 @@ Each unknown column gets one of three answers:
 
 `--self-check` shows how well Rosetta knows *your* kind of file. It hides each known column in turn and tries to name it.
 
+## If galvani stops with "Column ID … is unknown"
+
+In files from EC-Lab 11.50 and later, galvani 0.5.0 reads only the low byte of each column ID. IDs of 256 and above therefore turn into other IDs:
+
+| Real ID | Column | galvani 0.5.0 sees |
+|---|---|---|
+| 471 | `<Ece>/V` | 215 |
+| 438 | `step time/s` | 182 |
+| 431–433 | `\|Zwe-ce\|`, `Re(Zwe-ce)`, `-Im(Zwe-ce)` | 175–177 |
+
+The column is often one that galvani already knows. Rosetta reads the full IDs and says so:
+
+```
+$ rosetta peis.issue_225.with_ece.mpr
+peis.issue_225.with_ece.mpr
+  technique PEIS, 32 rows, 112 bytes per record
+  galvani 0.5.0 reads only the low byte of each column ID in this file layout, so it misreads 471 as 215, 430 as 174, 431 as 175, 432 as 176, 433 as 177
+  every column ID is in galvani's column map; nothing to name
+```
+
+The fix was released in datalab-org-galvani 0.6.0 ([galvani-mirror#11](https://github.com/datalab-org/galvani-mirror/issues/11)). IDs that are missing from the map altogether, such as 880 or 883 and 884, are what Rosetta names.
+
 ## How it works
 
 1. **Read the file** (`namer/mprfile.py`). Columns the reference parser knows are typed from its column map. For unknown IDs the byte width is solved from the record size, and the storage type is chosen by plausibility.

{rosetta_eclab-0.1.1 → rosetta_eclab-0.1.2}/tests/test_cli.py

@@ -51,6 +51,15 @@ def test_real_drift_duplicates_are_reported():
     assert [r["verdict"] for r in res] == ["DUPLICATE", "DUPLICATE"]
 
 
+def test_galvani_low_byte_misreads_are_reported():
+    """galvani 0.5.0 stops on this file only because it reads 471 (<Ece>/V) as 215; nothing is unknown."""
+    m = mprfile.read(EXT / "peis.issue_225.with_ece.mpr", reference="galvani")
+    assert m.unknown == []
+    assert any("471 as 215" in n for n in m.notes), m.notes
+    assert not any("low byte" in n for n in mprfile.read(EXT / "gcpl.mpr", reference="galvani").notes)
+    assert not any("low byte" in n for n in mprfile.read(EXT / "peis.issue_225.with_ece.mpr", reference="yadg").notes)
+
+
 @pytest.mark.parametrize("old_id,expected", [(467, "Q charge or discharge"), (7, "dq"), (4, "time")])
 def test_simulated_drift_is_named(tmp_path, old_id, expected):
     f = tmp_path / "drift.mpr"