evals-lab 0.2.0 → 0.4.0

package/CHANGELOG.md

@@ -0,0 +1,95 @@
+# Changelog
+
+What changed in each release of evals-lab, for someone who has it installed.
+Newest first. Read a release's **Upgrade notes** before installing it: an
+upgrade can rewrite what the lab keeps in your data directory, and an older
+version cannot always read it back.
+
+## 0.4.0
+
+### Upgrade notes
+
+- **Copy your data directory before upgrading** (see "Where your data is
+  kept" in the README). A dataset 0.4.0 saves is dataset version 7, which
+  0.3.x cannot read. To go back, reinstall 0.3.0 and restore the copy.
+- Stored datasets are not rewritten on upgrade: 0.4.0 reads a version-6
+  dataset as version 7, and writes version 7 the next time it is saved.
+- Export JSON writes dataset version 7, which 0.3.x refuses to import.
+  Import reads versions 1 to 7.
+
+### Added
+
+- Re-run, in a History row's menu and in Results' run menu: a new run of the
+  same pipeline over the same file revisions and the same dataset version.
+  It is refused while the run is still going, or when one of the files it
+  read is gone. The new run says which run it re-ran, and links back to it
+  in Results. Undo cancels it while it is still queued.
+- A dataset carries four new sections, `scoring`, `grader`, `every` and
+  `run`: how it scores, which model grades it, and metrics for every item
+  and for the whole run. Edit raw… shows them. Runs do not read them yet.
+  A dataset from 0.3.x has them as scored All, by the lab's grader, with no
+  metrics of its own.
+
+### Changed
+
+- Resume is in a History row's menu, beside Re-run.
+
+## 0.3.0
+
+### Upgrade notes
+
+- **Copy your data directory before upgrading** (see "Where your data is
+  kept" in the README). 0.3.0 changes what it keeps there in a form 0.2.x
+  cannot read. To go back, reinstall 0.2.x and restore the copy.
+- On its first start, 0.3.0 rewrites every stored dataset as dataset version
+  6. The originals are kept in the store as well.
+- Pipelines are read and saved as pipeline version 12.
+- Export JSON now writes dataset version 6, which 0.2.x refuses to import.
+  Import still reads versions 1 to 6.
+- A pipeline file that lists its evals under `tests:` still imports.
+- A plugin that registers `testTypes` still loads. The new name is
+  `evalTypes`.
+
+### Added
+
+- `discarded` metric: passes only when the job's rules discarded the reply.
+- `contains` takes `except`.
+- Add metric asks for the type first, from a list grouped by what the metric
+  checks. Change type… is in each metric's menu.
+- A header over Runs, Library › Datasets and a Source names what is open.
+  Tap the name to switch. Its menu holds the rest of the actions.
+- A loading indicator wherever the page waits, with Retry when a request
+  fails. A list no longer reads as empty while it loads.
+
+### Changed
+
+- Tests are called Evals: Runs › Evals, and the Evals columns in Results
+  and History.
+- A case is an item of its dataset's Source, plus the metrics a good reply
+  passes. A dataset names its Source.
+- Library › Datasets is one view. The table shows Item, Status (Graded, To do
+  or Missing), Metrics and Note. Add case picks from the Source's items that
+  have no case yet. Change item… is in the case's menu.
+- An eval that names a dataset grades each item by that case's metrics. The
+  separate `case` metric is gone, and is dropped from older pipelines.
+- Contains, Contains any and Contains all match a list reply item by item,
+  and say what they found, missed and invented.
+- Ignore case now applies item by item. A new Contains metric starts with it
+  on. A dataset's existing metrics are set to it, as they always matched
+  that way. A pipeline's keep what you set.
+- A metric of weight 0 is watched: listed under Watch in Results, never
+  scored. Results shows one verdict per eval.
+- A Source's Upload replaces Add files.
+- Rename is a dialog for datasets, as it is for pipelines.
+- Every tab has the dotted background, not Home alone.
+- History opens faster: the list loads without each run's full replies, and
+  the server compresses what it sends.
+
+### Fixed
+
+- History showed one row for two runs queued in the same second.
+
+### Removed
+
+- Map against Source, and the map run in Runs › Mapping. A case matches its
+  item by exact name.

package/README.md

@@ -128,16 +128,18 @@ dialog, does not carry over.
 
 ## Google Drive
 
-The lab can sign in to Google Drive, so a Source can read files from a Drive
-folder. The sign-in is held by the lab's server, on your machine, and it can
-reach only the files you pick: the lab asks for `drive.file` access, never
-your whole Drive.
+**Not usable yet.** The lab can sign in to Google Drive, but nothing reads
+from Drive yet: a Source made from a Drive folder is still to come. Until it
+arrives, there is no reason to set up a Google app, and the steps below are
+provisional. They haven't been tried against a real Google project.
 
-Open **Setup › Connections**, then use the Google Drive row's menu ›
-**Sign in**. If the row says *Not Configured*, give the lab a Google app
-first, once. These steps are provisional: they haven't yet been tried
-against a real Google project, and the Drive Source that uses the sign-in
-is still to come.
+When it arrives, the sign-in will be held by the lab's server, on your
+machine. It will reach only the files you pick: the lab asks for
+`drive.file` access, never your whole Drive.
+
+To sign in, open **Setup › Connections**, then use the Google Drive row's
+menu › **Sign in**. If the row says *Not Configured*, give the lab a Google
+app first, once:
 
 1. At <https://console.cloud.google.com>, make a project. Enable the
    **Google Drive API** and the **Google Picker API**.
@@ -173,6 +175,12 @@ lab holds your API keys.
 
 ## Upgrading and removing
 
+Read `CHANGELOG.md` in the package, or on
+<https://www.npmjs.com/package/evals-lab?activeTab=code>, before upgrading.
+Its **Upgrade notes** say when a release rewrites what is in your data
+directory in a form an older version cannot read; copy the directory first
+if you may want to go back.
+
 ```bash
 npm install -g evals-lab@latest   # upgrade
 npm uninstall -g evals-lab        # remove; your data directory stays

package/lab/VERSION

@@ -1 +1 @@
-0.2.0 (2026.10.02-323)
+0.4.0 (2026.10.02-361)