openprocess 0.14.0__tar.gz → 0.16.0__tar.gz

{openprocess-0.14.0/openprocess.egg-info → openprocess-0.16.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: openprocess
-Version: 0.14.0
+Version: 0.16.0
 Summary: Process mining, Petri nets and workflows in one open app: every algorithm visible, every result reproducible, every step teachable
 Author: Sam Struthers
 License: MIT
@@ -73,7 +73,7 @@ tools (and, until version 0.7, went by the name CPNpy):
 | **Petri nets & WF-nets** | Draw nets the way the lectures do. Get a soundness verdict with a counterexample for every violation, and replay it in the token game. Also: behavioural properties, P- and T-invariants, the footprint matrix, the reachability graph, PNML import and export. | WoPeD, ProM, pen and paper |
 | **Process mining** | Import XES or CSV logs, OCEL 2.0 object-centric logs, or type textbook logs like `[<a,b,c>^3, <a,c>^2]`. Filter them, explore variants, the dotted chart and the process map. Discover models (α-algorithm, Inductive Miner, Heuristics Miner, state-based regions). Check conformance (token replay, alignments, precision…) and compare logs. | ProM, Disco |
 | **Coloured Petri nets** | Open, edit and save CPN Tools models (`.cpn`), hierarchical ones included. Step through or simulate them, compute the state space, and export a simulation as an event log to mine. | CPN Tools / CPN IDE |
-| **Workflows** | Boxes on a canvas: a log, a miner, a fitness check, a comparison, a sweep over a setting, a prediction pipeline. Click a box to see its result, *how* it got there (the α-algorithm's eight steps, the replay per variant), its code and its settings. Change a setting and only what follows runs again. Saved as a `.cpnflow` file with everything needed to get the same numbers back. Your own algorithm is one Python function in a `boxes/` folder. | RapidProM |
+| **Workflows** | Boxes on a canvas: a log, a miner, a fitness check, a comparison, a sweep over a setting, a prediction pipeline. Click a box to see its result, *how* it got there (the α-algorithm's eight steps, the replay per variant), its code and its settings. Change a setting and only what follows runs again. Saved as a `.analysis` file with everything needed to get the same numbers back. Your own algorithm is one Python function in a `boxes/` folder. | RapidProM |
 | **Learn** | A mode of its own for worksheets, built on the app: answer in boxes on the sheet (sets, markings, matrices, cuts and trees, alignments, a net in the editor or a workflow on the canvas beside it) and press **Check**. Most answers are checked automatically, often against answers worked out from the given log or net. Packs are plain Markdown, can be exams with a clock and points, and a past exam imports as a skeleton pack. Demo exercises included. | Answer sheets, a notes app and guesswork |
 | **Folders** | Open a folder such as *Week 2*: every log and net in it is listed in the sidebar, with its subfolders. The folder and the app stay in step both ways: new nets and edits are saved into it as you go, and changes made in Finder show up by themselves. | Finder windows and *File ▸ Open* every time |
 
@@ -598,7 +598,7 @@ without distractions, and *Exit* brings your folder back exactly as it was.
   writes the marks as CSV. A pack can also give every student a **variant**
   of its own (a log played out from a net with a seed made from their name).
 - **Your work is saved as you go**, in the exercise's folder:
-  `my answers.json`, `my answer.pnml` for a net, `my workflow.cpnflow` for a
+  `my answers.json`, `my answer.pnml` for a net, `my workflow.analysis` for a
   workflow and `my notes.md`.
 
 **Writing a pack** (for a course or an exam) is plain Markdown: put an
@@ -791,14 +791,28 @@ Miner, a fitness check), *Compare discovery* (three miners side by side),
 confidence* (bootstrap intervals, a test and a plot; needs the `science`
 extra) and *Predict the next activity*. Or start empty.
 
-| The canvas: *Compare discovery*, three miners side by side | Summary: every box's key figure, and the process map |
+| The canvas: *Compare discovery*, three miners side by side | Report: every box's result, in run order, on one page |
 |---|---|
-| ![The canvas of a Compare discovery analysis, the Compare box's table beside it](docs/screenshots/workflow-canvas.png) | ![The Summary view: a tile per box, the process map under them](docs/screenshots/workflow-summary.png) |
-
-- **The page is the canvas.** Its header has **Canvas | Summary**, then
-  **+ Add box**, **Run ▶**, and **⋯** for Re-run, Record, Export experiment
-  and Save. The sidebar beside it lists the folder's analyses and, folded
-  under them, its logs.
+| ![The canvas of a Compare discovery analysis, the Compare box's table beside it](docs/screenshots/workflow-canvas.png) | ![The Report view: a section per box with its result, a Steps outline beside them](docs/screenshots/workflow-report.png) |
+
+- **An analysis opens on its Report.** Its header has **Report | Canvas |
+  Versions**, then **+ Add box**, **Run ▶**, and **⋯** for Run from scratch,
+  Keep as version, Record, Export experiment and Save. The sidebar beside it
+  lists the folder's analyses and, folded under them, its logs.
+- **Report** is every box's result, in the order the boxes run, on one
+  scrolling page: a section per box with the group it is in, its name, the
+  line the canvas shows under it, its status, the result as the side panel
+  shows it (a table scrolls sideways inside its section when it is wider; a
+  figure keeps its proportions; a score gets a key-figure tile), its
+  settings as chips, and *Open ›*, *How* and *Code*, which open the box in
+  the side panel on the canvas. A slim *Steps* outline on the left lists
+  the sections, grouped as the canvas groups the boxes, with each box's
+  key figure; a click scrolls to the section, and on a narrow window the
+  outline folds into a *Steps* menu. Nothing is computed there: while a
+  run goes, each section is redrawn as its box reports; a box that has not
+  run says so, a failed one shows its error. The process map is the
+  *Directly-follows graph* box's result; an analysis without that box is
+  offered one in one click.
 - **+ Add box** (or a double-click on the canvas) opens every box, one
   section per group (Input, Filter, Discover, Check, Measure, Compare, Output,
   then Science, Predict, Coloured nets, Sweep and your own boxes), in a list that
@@ -812,12 +826,6 @@ extra) and *Predict the next activity*. Or start empty.
   notation…* in the sidebar land as boxes on the current analysis (or a new
   one named after the file). In a box's **Settings**, *Choose…* picks the
   file and the list beside it offers only the files the box can read.
-- **Summary** is the key figure of every box, in one place: a tile per box,
-  in the order the boxes run (a score's first metric, a table's size, a
-  log's cases, a net's places and transitions). Nothing is computed there:
-  a tile is the box's result, as on the canvas, and a tile opens the box.
-  The process map under the tiles is the *Directly-follows graph* box's
-  result; an analysis without that box offers to add it in one click.
 - **Connect** by dragging from the dot on the right of a box: while you drag,
   only the inputs that take that kind of result light up, so a wrong
   connection cannot be made. Click a wire and press Delete to remove it.
@@ -838,9 +846,14 @@ extra) and *Predict the next activity*. Or start empty.
 - **Only what changed runs again.** Every box shows a status dot: waiting,
   running, done, failed (the error is on the Result tab; the rest keeps
   working), or waiting for your OK.
-- **Record** shows the workflow as Python and the file with its record;
-  **Re-run** says what changed since it was saved (an input file, a box file,
-  a seed) before running everything again.
+- **Versions** is where the numbers stand against the record and every
+  version kept. The first run with no failure is kept as v1; from then on the
+  badge beside the title says *v1 · reproducible*, *v1 · 2 changes* or
+  *Changed since v1*, and **Keep as version…** records the run as v2 with a
+  note. Any version can be compared with now (the boxes whose result differs,
+  first) or restored whole. **Record** shows the workflow as Python and the
+  file; **Run from scratch** says what changed since the version was kept
+  before computing everything again.
 - **Your own boxes**: a `.py` file in the folder's `boxes/` subfolder (see
   below). The app asks once per folder before running them, and reloads a
   box when its file is saved.
@@ -894,15 +907,15 @@ extra) and *Predict the next activity*. Or start empty.
   `~/Library/Application Support/OpenProcess/packages`), and the boxes that
   need the package come alive without a restart. PM4Py needs a Python
   install (`pip install pm4py`), as parts of it are built from source.
-- **Motion** is kept short: a space switch, the side panel and Canvas ↔
-  Summary settle in a fifth of a second, and a box pulses once when it
+- **Motion** is kept short: a space switch, the side panel and Report ↔
+  Canvas settle in a fifth of a second, and a box pulses once when it
   finishes. `OPENPROCESS_NO_MOTION=1` turns it off.
 
 ## Workflows from Python
 
 Every algorithm is also a **box**: a Python function with type hints that
 the app (from 0.7) draws on a canvas. A workflow of boxes runs from Python
-or from the command line, and its `.cpnflow` file records what is needed
+or from the command line, and its `.analysis` file records what is needed
 to get the same numbers again: fingerprints of the input files and of the
 box code, every seed, and the installed packages.
 
@@ -920,12 +933,12 @@ wf.connect(log, model); wf.connect(model, fit, "model"); wf.connect(log, fit, "l
 run = Runner().run(wf)
 print(run.value(fit).metrics)                  # fitness, precision, generalisation, simplicity
 print(run.result(model).explanation.steps)     # how the Inductive Miner got there
-save(wf, "orders.cpnflow", run)
+save(wf, "orders.analysis", run)
 ```
 
 ```bash
-openprocess run orders.cpnflow --check     # re-run; fails if any result differs from the record
-openprocess run orders.cpnflow --sweep noise=0..0.5 step 0.1
+openprocess run orders.analysis --check     # re-run; fails if any result differs from the record
+openprocess run orders.analysis --sweep noise=0..0.5 step 0.1
 openprocess boxes                          # every box (listed in docs/boxes.md), with a folder's boxes/
 openprocess datasets list                  # the BPI Challenge, Sepsis, ... logs by name
 ```

{openprocess-0.14.0 → openprocess-0.16.0}/README.md

@@ -31,7 +31,7 @@ tools (and, until version 0.7, went by the name CPNpy):
 | **Petri nets & WF-nets** | Draw nets the way the lectures do. Get a soundness verdict with a counterexample for every violation, and replay it in the token game. Also: behavioural properties, P- and T-invariants, the footprint matrix, the reachability graph, PNML import and export. | WoPeD, ProM, pen and paper |
 | **Process mining** | Import XES or CSV logs, OCEL 2.0 object-centric logs, or type textbook logs like `[<a,b,c>^3, <a,c>^2]`. Filter them, explore variants, the dotted chart and the process map. Discover models (α-algorithm, Inductive Miner, Heuristics Miner, state-based regions). Check conformance (token replay, alignments, precision…) and compare logs. | ProM, Disco |
 | **Coloured Petri nets** | Open, edit and save CPN Tools models (`.cpn`), hierarchical ones included. Step through or simulate them, compute the state space, and export a simulation as an event log to mine. | CPN Tools / CPN IDE |
-| **Workflows** | Boxes on a canvas: a log, a miner, a fitness check, a comparison, a sweep over a setting, a prediction pipeline. Click a box to see its result, *how* it got there (the α-algorithm's eight steps, the replay per variant), its code and its settings. Change a setting and only what follows runs again. Saved as a `.cpnflow` file with everything needed to get the same numbers back. Your own algorithm is one Python function in a `boxes/` folder. | RapidProM |
+| **Workflows** | Boxes on a canvas: a log, a miner, a fitness check, a comparison, a sweep over a setting, a prediction pipeline. Click a box to see its result, *how* it got there (the α-algorithm's eight steps, the replay per variant), its code and its settings. Change a setting and only what follows runs again. Saved as a `.analysis` file with everything needed to get the same numbers back. Your own algorithm is one Python function in a `boxes/` folder. | RapidProM |
 | **Learn** | A mode of its own for worksheets, built on the app: answer in boxes on the sheet (sets, markings, matrices, cuts and trees, alignments, a net in the editor or a workflow on the canvas beside it) and press **Check**. Most answers are checked automatically, often against answers worked out from the given log or net. Packs are plain Markdown, can be exams with a clock and points, and a past exam imports as a skeleton pack. Demo exercises included. | Answer sheets, a notes app and guesswork |
 | **Folders** | Open a folder such as *Week 2*: every log and net in it is listed in the sidebar, with its subfolders. The folder and the app stay in step both ways: new nets and edits are saved into it as you go, and changes made in Finder show up by themselves. | Finder windows and *File ▸ Open* every time |
 
@@ -556,7 +556,7 @@ without distractions, and *Exit* brings your folder back exactly as it was.
   writes the marks as CSV. A pack can also give every student a **variant**
   of its own (a log played out from a net with a seed made from their name).
 - **Your work is saved as you go**, in the exercise's folder:
-  `my answers.json`, `my answer.pnml` for a net, `my workflow.cpnflow` for a
+  `my answers.json`, `my answer.pnml` for a net, `my workflow.analysis` for a
   workflow and `my notes.md`.
 
 **Writing a pack** (for a course or an exam) is plain Markdown: put an
@@ -749,14 +749,28 @@ Miner, a fitness check), *Compare discovery* (three miners side by side),
 confidence* (bootstrap intervals, a test and a plot; needs the `science`
 extra) and *Predict the next activity*. Or start empty.
 
-| The canvas: *Compare discovery*, three miners side by side | Summary: every box's key figure, and the process map |
+| The canvas: *Compare discovery*, three miners side by side | Report: every box's result, in run order, on one page |
 |---|---|
-| ![The canvas of a Compare discovery analysis, the Compare box's table beside it](docs/screenshots/workflow-canvas.png) | ![The Summary view: a tile per box, the process map under them](docs/screenshots/workflow-summary.png) |
-
-- **The page is the canvas.** Its header has **Canvas | Summary**, then
-  **+ Add box**, **Run ▶**, and **⋯** for Re-run, Record, Export experiment
-  and Save. The sidebar beside it lists the folder's analyses and, folded
-  under them, its logs.
+| ![The canvas of a Compare discovery analysis, the Compare box's table beside it](docs/screenshots/workflow-canvas.png) | ![The Report view: a section per box with its result, a Steps outline beside them](docs/screenshots/workflow-report.png) |
+
+- **An analysis opens on its Report.** Its header has **Report | Canvas |
+  Versions**, then **+ Add box**, **Run ▶**, and **⋯** for Run from scratch,
+  Keep as version, Record, Export experiment and Save. The sidebar beside it
+  lists the folder's analyses and, folded under them, its logs.
+- **Report** is every box's result, in the order the boxes run, on one
+  scrolling page: a section per box with the group it is in, its name, the
+  line the canvas shows under it, its status, the result as the side panel
+  shows it (a table scrolls sideways inside its section when it is wider; a
+  figure keeps its proportions; a score gets a key-figure tile), its
+  settings as chips, and *Open ›*, *How* and *Code*, which open the box in
+  the side panel on the canvas. A slim *Steps* outline on the left lists
+  the sections, grouped as the canvas groups the boxes, with each box's
+  key figure; a click scrolls to the section, and on a narrow window the
+  outline folds into a *Steps* menu. Nothing is computed there: while a
+  run goes, each section is redrawn as its box reports; a box that has not
+  run says so, a failed one shows its error. The process map is the
+  *Directly-follows graph* box's result; an analysis without that box is
+  offered one in one click.
 - **+ Add box** (or a double-click on the canvas) opens every box, one
   section per group (Input, Filter, Discover, Check, Measure, Compare, Output,
   then Science, Predict, Coloured nets, Sweep and your own boxes), in a list that
@@ -770,12 +784,6 @@ extra) and *Predict the next activity*. Or start empty.
   notation…* in the sidebar land as boxes on the current analysis (or a new
   one named after the file). In a box's **Settings**, *Choose…* picks the
   file and the list beside it offers only the files the box can read.
-- **Summary** is the key figure of every box, in one place: a tile per box,
-  in the order the boxes run (a score's first metric, a table's size, a
-  log's cases, a net's places and transitions). Nothing is computed there:
-  a tile is the box's result, as on the canvas, and a tile opens the box.
-  The process map under the tiles is the *Directly-follows graph* box's
-  result; an analysis without that box offers to add it in one click.
 - **Connect** by dragging from the dot on the right of a box: while you drag,
   only the inputs that take that kind of result light up, so a wrong
   connection cannot be made. Click a wire and press Delete to remove it.
@@ -796,9 +804,14 @@ extra) and *Predict the next activity*. Or start empty.
 - **Only what changed runs again.** Every box shows a status dot: waiting,
   running, done, failed (the error is on the Result tab; the rest keeps
   working), or waiting for your OK.
-- **Record** shows the workflow as Python and the file with its record;
-  **Re-run** says what changed since it was saved (an input file, a box file,
-  a seed) before running everything again.
+- **Versions** is where the numbers stand against the record and every
+  version kept. The first run with no failure is kept as v1; from then on the
+  badge beside the title says *v1 · reproducible*, *v1 · 2 changes* or
+  *Changed since v1*, and **Keep as version…** records the run as v2 with a
+  note. Any version can be compared with now (the boxes whose result differs,
+  first) or restored whole. **Record** shows the workflow as Python and the
+  file; **Run from scratch** says what changed since the version was kept
+  before computing everything again.
 - **Your own boxes**: a `.py` file in the folder's `boxes/` subfolder (see
   below). The app asks once per folder before running them, and reloads a
   box when its file is saved.
@@ -852,15 +865,15 @@ extra) and *Predict the next activity*. Or start empty.
   `~/Library/Application Support/OpenProcess/packages`), and the boxes that
   need the package come alive without a restart. PM4Py needs a Python
   install (`pip install pm4py`), as parts of it are built from source.
-- **Motion** is kept short: a space switch, the side panel and Canvas ↔
-  Summary settle in a fifth of a second, and a box pulses once when it
+- **Motion** is kept short: a space switch, the side panel and Report ↔
+  Canvas settle in a fifth of a second, and a box pulses once when it
   finishes. `OPENPROCESS_NO_MOTION=1` turns it off.
 
 ## Workflows from Python
 
 Every algorithm is also a **box**: a Python function with type hints that
 the app (from 0.7) draws on a canvas. A workflow of boxes runs from Python
-or from the command line, and its `.cpnflow` file records what is needed
+or from the command line, and its `.analysis` file records what is needed
 to get the same numbers again: fingerprints of the input files and of the
 box code, every seed, and the installed packages.
 
@@ -878,12 +891,12 @@ wf.connect(log, model); wf.connect(model, fit, "model"); wf.connect(log, fit, "l
 run = Runner().run(wf)
 print(run.value(fit).metrics)                  # fitness, precision, generalisation, simplicity
 print(run.result(model).explanation.steps)     # how the Inductive Miner got there
-save(wf, "orders.cpnflow", run)
+save(wf, "orders.analysis", run)
 ```
 
 ```bash
-openprocess run orders.cpnflow --check     # re-run; fails if any result differs from the record
-openprocess run orders.cpnflow --sweep noise=0..0.5 step 0.1
+openprocess run orders.analysis --check     # re-run; fails if any result differs from the record
+openprocess run orders.analysis --sweep noise=0..0.5 step 0.1
 openprocess boxes                          # every box (listed in docs/boxes.md), with a folder's boxes/
 openprocess datasets list                  # the BPI Challenge, Sepsis, ... logs by name
 ```

{openprocess-0.14.0 → openprocess-0.16.0}/openprocess/__init__.py

@@ -48,7 +48,7 @@ from .sim.simulator import Simulator
 #: The one place the version number is written (pyproject.toml, the update
 #: check and packaging/build.py read it from here).  A release is tagged
 #: "v" + this, e.g. v0.2.0.
-__version__ = "0.14.0"
+__version__ = "0.16.0"
 
 __all__ = [
     "Arc", "BindingElement", "CPNet", "Marking", "Multiset", "Page", "Place",

{openprocess-0.14.0 → openprocess-0.16.0}/openprocess/cli.py

@@ -19,7 +19,7 @@ Commands
                 ``stats``, ``filter``, ``discover``, ``conform``,
                 ``soundness``, ``invariants``
 ``exercises``   ``check``, ``marks``, ``import``, ``computes``: exercise packs (OpenProcess Learn)
-``run``         run a ``.cpnflow`` workflow headless; ``--check`` re-runs and
+``run``         run a ``.analysis`` workflow headless; ``--check`` re-runs and
                 compares with the record, ``--sweep`` varies a setting,
                 ``--lock`` writes the requirements the record names,
                 ``--export`` and ``--report`` write a zip and an HTML page
@@ -447,12 +447,19 @@ def command_exercises_computes(arguments: argparse.Namespace) -> int:
 # ---------------------------------------------------------------------------
 def command_run(arguments: argparse.Namespace) -> int:
     """Run a workflow file without the app, and say what every box gave."""
-    from .flow import Runner, Sweep, check, differences, library_for, load, save
-    from .flow.record import requirements_lock
+    from .flow import Runner, Sweep, check, differences, library_for, save
+    from .flow.record import load_all, requirements_lock
     path = Path(arguments.workflow)
     folder = path.parent
     library = library_for(folder)
-    workflow, record = load(path, library)
+    workflow, record, versions = load_all(path, library)
+    if arguments.against is not None:
+        version = versions.get(arguments.against)
+        if version is None:
+            print(f"openprocess: {path.name} has no version {arguments.against} "
+                  f"(it has {', '.join(f'v{v.number}' for v in versions.items) or 'none'})", file=sys.stderr)
+            return 2
+        record = version.record
     if arguments.lock:
         target = folder / "requirements.lock"
         target.write_text(requirements_lock(record), encoding="utf-8")
@@ -504,6 +511,8 @@ def command_run(arguments: argparse.Namespace) -> int:
     if arguments.report:
         from .flow.report import write_report
         print(f"Wrote {write_report(arguments.report, workflow=workflow, run=run, folder=folder)}")
+    against = f"v{arguments.against}" if arguments.against is not None else (
+        f"v{versions.latest.number}" if versions.latest else "the record")
     if arguments.check:
         if not record.results:
             # Nothing to compare is not a pass: a CI job would claim a reproduction that never happened.
@@ -513,12 +522,18 @@ def command_run(arguments: argparse.Namespace) -> int:
         report = check(workflow, record, run, complete=True)
         if report:
             for line in report:
-                print(f"DIFFERS: {line}")
+                print(f"DIFFERS from {against}: {line}")
             return 1
-        print(f"Every recorded result was reproduced ({len(record.results)} boxes).")
+        print(f"Every result recorded in {against} was reproduced ({len(record.results)} boxes).")
     elif arguments.save:
-        save(workflow, path, run, folder)
-        print(f"Recorded the results in {path.name}")
+        if run.failed():
+            save(workflow, path, run, folder, previous=record, versions=versions)
+            print(f"A box failed, so {path.name} keeps " + (f"{against} as its record" if record.results
+                                                             else "no record yet"))
+        else:
+            save(workflow, path, run, folder, previous=record, versions=versions, keep=arguments.note or "")
+            print(f"Kept the results as v{versions.latest.number} in {path.name}"
+                  + (f": {arguments.note}" if arguments.note else ""))
     return 0 if run.ok else 1
 
 
@@ -710,11 +725,13 @@ def build_parser() -> argparse.ArgumentParser:
         "computes", help="list every compute: an answer block may use")
     exercise_computes.set_defaults(handler=command_exercises_computes)
 
-    run = subparsers.add_parser("run", help="run a .cpnflow workflow without the app")
-    run.add_argument("workflow", help="the .cpnflow file")
+    run = subparsers.add_parser("run", help="run a .analysis workflow without the app")
+    run.add_argument("workflow", help="the .analysis file")
     run.add_argument("--check", action="store_true",
-                     help="re-run and fail if any result differs from the recorded one")
-    run.add_argument("--save", action="store_true", help="record the results in the file")
+                     help="re-run and fail if any result differs from the recorded one (the latest version)")
+    run.add_argument("--against", type=int, metavar="N", help="check against version N instead of the latest")
+    run.add_argument("--save", action="store_true", help="keep the results in the file as the next version")
+    run.add_argument("--note", metavar="TEXT", help="with --save: the version's note (what changed, or why)")
     run.add_argument("--sweep", action="append", metavar="SETTING=VALUES",
                      help="vary a setting, e.g. noise=0..0.5 step 0.1 or n3.seed=1,2,3")
     run.add_argument("--lock", action="store_true",

{openprocess-0.14.0 → openprocess-0.16.0}/openprocess/exercises/pack.md

@@ -7,7 +7,7 @@ straight away, and the app never shows you a result you are meant to work out
 until you ask for it.
 
 Your answers are saved in each exercise's folder (*my answers.json*, and
-*my answer.pnml* for a net you draw, *my workflow.cpnflow* for a workflow
+*my answer.pnml* for a net you draw, *my workflow.analysis* for a workflow
 you build), so you can stop and carry on later.
 
 To write your own pack, copy this folder and change the `question.md` files:

{openprocess-0.14.0 → openprocess-0.16.0}/openprocess/flow/__init__.py

@@ -24,7 +24,7 @@ Python::
     wf.connect(log, model); wf.connect(model, fit, "model"); wf.connect(log, fit, "log")
     run = Runner().run(wf)
     print(run.value(fit).metrics)
-    save(wf, "demo.cpnflow", run)      # with the record that makes it reproducible
+    save(wf, "demo.analysis", run)      # with the record that makes it reproducible
 
 Work over many independent items (variants, cases, resamples) runs on all
 cores with :func:`parallel_map`, with exactly the result of a loop.

{openprocess-0.14.0 → openprocess-0.16.0}/openprocess/flow/figures.py

@@ -1,6 +1,6 @@
 """The key figure of a result: one number and a line, for a tile or a report.
 
-Shared by the Summary (:mod:`openprocess.gui.flow.summary`) and the HTML
+Shared by the Report (:mod:`openprocess.gui.flow.report`) and the HTML
 report (:mod:`openprocess.flow.report`), so both say the same thing about a
 result: a score's first metric, a table's size, a log's cases, a net's
 places and transitions, a map's activities and paths.

{openprocess-0.14.0 → openprocess-0.16.0}/openprocess/flow/record.py

@@ -1,7 +1,8 @@
-"""The ``.cpnflow`` file: a workflow, and the record that makes it reproducible.
+"""The ``.analysis`` file: a workflow, and the record that makes it reproducible.
 
-A workflow file is JSON: the boxes, their settings and positions, the
-connections, the groups, and a **record**:
+An analysis file is JSON: the boxes, their settings and positions, the
+connections, the groups, its **versions** (:mod:`.versions`: the record as
+it was each time the user kept it) and the **record** of the latest version:
 
 * the versions of OpenProcess, Python and the optional libraries;
 * the environment: every installed package and its version (the lock);
@@ -34,7 +35,44 @@ from .runner import DONE, Run
 from .types import Dataset, Figure, Predictions, Scores, Table, Text
 from .workflow import Workflow
 
-FORMAT = "cpnflow/1"
+FORMAT = "analysis/1"
+
+#: The extension of an analysis file, and the ones it had before (``.cpnflow``,
+#: from when the app was CPNpy: the content is the same, only the name changed).
+EXTENSION = ".analysis"
+OLD_EXTENSIONS = (".cpnflow",)
+
+
+def is_analysis_file(path) -> bool:
+    """Whether ``path`` is an analysis file by its name, under either extension."""
+    return str(path).lower().endswith((EXTENSION,) + OLD_EXTENSIONS)
+
+
+def analysis_name(path) -> Path:
+    """``path`` under the current extension (``orders.cpnflow`` → ``orders.analysis``)."""
+    path = Path(path)
+    return path.with_name(path.stem + EXTENSION) if path.suffix.lower() in OLD_EXTENSIONS else path
+
+
+def carry_over(folder, depth: int = 3) -> list[Path]:
+    """Rename every ``.cpnflow`` under ``folder`` to ``.analysis`` (the content is
+    read either way); a file whose new name is taken is left.  Returns the new paths."""
+    renamed = []
+    root = Path(folder)
+    for old in OLD_EXTENSIONS:
+        for source in sorted(root.rglob(f"*{old}")):
+            if len(source.relative_to(root).parts) - 1 > depth or any(
+                    part.startswith(".") for part in source.relative_to(root).parts):
+                continue
+            target = analysis_name(source)
+            if target.exists():
+                continue
+            try:
+                source.replace(target)
+            except OSError:
+                continue
+            renamed.append(target)
+    return renamed
 
 
 # ---------------------------------------------------------------------------
@@ -242,8 +280,11 @@ def canonical(value: Any, _active: set[int] | None = None) -> Any:
         if isinstance(value, (list, tuple)):
             return [canonical(v, _active) for v in value]
         if dataclasses.is_dataclass(value) and not isinstance(value, type):
+            # A field named with a leading underscore is a cache or a working
+            # value (a DFG's per-arc means and medians, say), not content.
             fields = {f.name: canonical(getattr(value, f.name, None), _active)
-                      for f in dataclasses.fields(value) if f.name not in _NOT_CONTENT}
+                      for f in dataclasses.fields(value)
+                      if not f.name.startswith("_") and f.name not in _NOT_CONTENT}
             return {"type": type(value).__name__, **fields}
         attributes = getattr(value, "__dict__", None)
         if isinstance(attributes, dict):
@@ -456,14 +497,46 @@ def differences(record: Record, workflow: Workflow, folder: str | Path | None =
 # Files
 # ---------------------------------------------------------------------------
 def save(workflow: Workflow, path: str | Path, run: Run | None = None,
-         folder: str | Path | None = None, lock: bool = True, history: list[dict] | None = None) -> Record:
-    """Write the workflow and its record to ``path``; returns the record.
-    ``history``: the analysis's snapshots (see :mod:`.snapshots`), kept in the file."""
+         folder: str | Path | None = None, lock: bool = True, history: list[dict] | None = None,
+         previous: Record | None = None, versions: "Versions | None" = None,
+         keep: str | None = None) -> Record:
+    """Write the workflow, its versions and its record to ``path``; returns the record.
+
+    ``history``: the analysis's runs (see :mod:`.snapshots`), kept in the file.
+    ``previous``: the file's record so far, and ``versions`` the versions it
+    holds; see :func:`carried`.  ``keep``: a note, to keep this run as a new
+    version (``""`` for a version without one): the record becomes this run's.
+    Without ``keep`` the record stays as it was, except that the first run in
+    which no box fails is kept as version 1 by itself, so a new analysis is
+    recorded without a word.
+
+    ``versions`` is updated in place when a version is kept."""
+    from .versions import Versions
     path = Path(path)
     folder = folder if folder is not None else path.parent
-    record = make_record(workflow, run, folder, lock)
+    versions = versions if versions is not None else Versions()
+    if keep is not None and (run is None or run.failed()):
+        raise ValueError("A version keeps a run in which no box failed")
+    if keep is None and previous is not None and previous.results:
+        record = carried(previous, workflow)
+    elif keep is None and (run is None or run.failed()):
+        record = make_record(workflow, None, folder, lock)      # nothing to keep yet
+    else:
+        record = make_record(workflow, run, folder, lock)
+        if previous is not None and previous.scheme == record.scheme:
+            # A box that could not run here (a library it needs is missing)
+            # keeps the fingerprint recorded where it could.
+            still = {k: v for k, v in previous.results.items() if k in workflow.nodes}
+            record.results = {**still, **record.results}
+        for entry in record.inputs:
+            if entry["sha256"] == "missing" and previous is not None:
+                then = next((i for i in previous.inputs if i["node"] == entry["node"]
+                             and i["setting"] == entry["setting"] and i["file"] == entry["file"]), None)
+                if then is not None:
+                    entry["sha256"] = then["sha256"]
+        versions.keep(workflow, run, record, keep or ("First run" if not versions.items else ""))
     record.history = list(history or [])
-    data = {"format": FORMAT, **workflow.to_dict(), "record": record.to_dict()}
+    data = {"format": FORMAT, **workflow.to_dict(), "versions": versions.to_list(), "record": record.to_dict()}
     text = json.dumps(data, indent=2, ensure_ascii=False, default=str)
     temporary = path.with_name(path.name + ".tmp")
     temporary.write_text(text, encoding="utf-8")
@@ -471,13 +544,34 @@ def save(workflow: Workflow, path: str | Path, run: Run | None = None,
     return record
 
 
+def carried(previous: Record, workflow: Workflow) -> Record:
+    """The record to write when no version is kept: the previous one as it
+    was, results of boxes since removed aside.  Saving as you work never
+    replaces the record; keeping a version does (:func:`save` with ``keep``)."""
+    still = {k: v for k, v in previous.results.items() if k in workflow.nodes}
+    return Record(**{**previous.__dict__, "results": still})
+
+
 def load(path: str | Path, library=None) -> tuple[Workflow, Record]:
-    """Read a ``.cpnflow`` file."""
+    """Read an analysis file (``.analysis``, or ``.cpnflow`` as it was called)."""
+    workflow, record, _versions = load_all(path, library)
+    return workflow, record
+
+
+def load_all(path: str | Path, library=None) -> "tuple[Workflow, Record, Versions]":
+    """Read an analysis file: its workflow, the record of its latest version, and
+    every version kept.  A file from before versions (a record, no list) gives
+    its record as version 1."""
+    from .versions import Versions
     data = json.loads(Path(path).read_text(encoding="utf-8"))
-    if data.get("format", FORMAT).split("/")[0] != "cpnflow":
-        raise ValueError(f"{path} is not a workflow file")
+    if data.get("format", FORMAT).split("/")[0] not in ("analysis", "cpnflow"):
+        raise ValueError(f"{path} is not an analysis file")
     workflow = Workflow.from_dict(data, library)
-    return workflow, Record.from_dict(data.get("record"))
+    record = Record.from_dict(data.get("record"))
+    versions = Versions.from_list(data.get("versions"))
+    if not versions.items and record.results:
+        versions = Versions.first_from(workflow, record)
+    return workflow, record, versions
 
 
 def check(workflow: Workflow, record: Record, run: Run, complete: bool = False) -> list[str]:
@@ -547,7 +641,7 @@ def export_experiment(workflow: Workflow, run: Run | None, target: str | Path,
     name = workflow.name or "workflow"
     lines = [f"# {name}", "", f"Exported {record.saved} with OpenProcess {record.versions.get('openprocess', '')}, "
              f"Python {record.versions.get('python', '')}.", "",
-             "Run it again with `openprocess run " + f"{name}.cpnflow --check`: it fails if any result differs "
+             "Run it again with `openprocess run " + f"{name}{EXTENSION} --check`: it fails if any result differs "
              "from the recorded one. `requirements.lock` is the environment it ran in. `run.py` is the same "
              "analysis as a plain Python script (`python run.py`). Or drop this zip on OpenProcess: it unpacks "
              "it into a folder and runs it.", "", "## Boxes", ""]
@@ -575,7 +669,7 @@ def export_experiment(workflow: Workflow, run: Run | None, target: str | Path,
     lines += ["", "## How to cite", "", citation.app_text(), "",
               "`CITATIONS.bib` has this entry and the papers the algorithms follow, as BibTeX."]
     with zipfile.ZipFile(target, "w", zipfile.ZIP_DEFLATED) as zipped:
-        zipped.writestr(f"{name}.cpnflow", json.dumps(data, indent=2, ensure_ascii=False, default=str))
+        zipped.writestr(f"{name}{EXTENSION}", json.dumps(data, indent=2, ensure_ascii=False, default=str))
         zipped.writestr("requirements.lock", requirements_lock(record))
         from .script import script_text
         zipped.writestr("run.py", script_text(packed_workflow, "run.py"))
@@ -611,11 +705,11 @@ def _describe(value) -> str:
 
 
 def is_experiment(path) -> bool:
-    """Whether ``path`` is a zip :func:`export_experiment` wrote: one ``.cpnflow`` at the top."""
+    """Whether ``path`` is a zip :func:`export_experiment` wrote: one analysis file at the top."""
     import zipfile
     try:
         with zipfile.ZipFile(path) as zipped:
-            return sum(1 for n in zipped.namelist() if n.endswith(".cpnflow") and "/" not in n) == 1
+            return sum(1 for n in zipped.namelist() if is_analysis_file(n) and "/" not in n) == 1
     except (OSError, zipfile.BadZipFile):
         return False
 
@@ -630,9 +724,9 @@ def import_experiment(archive, into) -> Path:
     archive = Path(archive)
     with zipfile.ZipFile(archive) as zipped:
         names = zipped.namelist()
-        flows = [n for n in names if n.endswith(".cpnflow") and "/" not in n]
+        flows = [n for n in names if is_analysis_file(n) and "/" not in n]
         if len(flows) != 1:
-            raise ValueError(f"{archive.name} is not an exported experiment (one .cpnflow at the top is needed)")
+            raise ValueError(f"{archive.name} is not an exported experiment (one {EXTENSION} file at the top is needed)")
         folder = Path(into) / archive.stem
         counter = 2
         while folder.exists():
@@ -648,6 +742,9 @@ def import_experiment(archive, into) -> Path:
             target.write_bytes(zipped.read(name))
     flow_file = folder / flows[0]
     data = json.loads(flow_file.read_text(encoding="utf-8"))
+    if flow_file.suffix.lower() in OLD_EXTENSIONS:          # an experiment exported before 0.16
+        flow_file.unlink()
+        flow_file = analysis_name(flow_file)
     moved: dict[tuple[str, str], str] = {}
     for item in data.get("record", {}).get("inputs", []):
         new = f"inputs/{Path(item['file']).name}"
@@ -697,6 +794,7 @@ def _as_file(value, stem: str):
 #: (name, bytes or text) for a result worth a file of its own; the script an experiment carries uses it.
 as_file = _as_file
 
-__all__ = ["FORMAT", "Record", "as_file", "check", "differences", "environment", "export_experiment",
-           "fingerprint", "import_experiment", "is_experiment", "load", "make_record", "requirements_lock",
-           "save", "sha256_file", "sha256_text", "versions"]
+__all__ = ["EXTENSION", "FORMAT", "OLD_EXTENSIONS", "Record", "analysis_name", "as_file", "carry_over", "check",
+           "differences", "environment", "export_experiment", "fingerprint", "import_experiment",
+           "is_analysis_file", "is_experiment", "load", "load_all", "make_record", "requirements_lock", "save",
+           "sha256_file", "sha256_text", "versions"]

{openprocess-0.14.0 → openprocess-0.16.0}/openprocess/flow/report.py

@@ -9,7 +9,7 @@ external assets, readable in any browser and printable.
 
 The page is built from the workflow and its run alone; drawings (SVG) are
 passed in by the app, which has the canvas to render them.  So the same
-report can be made without the app (``openprocess run analysis.cpnflow
+report can be made without the app (``openprocess run orders.analysis
 --report report.html``), minus the pictures.
 """