de-shell 0.2.0__tar.gz

de_shell-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Direct Electron
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

de_shell-0.2.0/PKG-INFO

@@ -0,0 +1,196 @@
+Metadata-Version: 2.4
+Name: de-shell
+Version: 0.2.0
+Summary: The Electron + Python desktop-app shell Direct Electron's apps (SpyDE, Ground Crew, Autopilot) are assembled from
+Author-email: Carter Francis <cartsfrancis@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/CSSFrancis/de-shell
+Project-URL: Repository, https://github.com/CSSFrancis/de-shell
+Project-URL: Issues, https://github.com/CSSFrancis/de-shell/issues
+Project-URL: Changelog, https://github.com/CSSFrancis/de-shell/blob/main/CHANGELOG.md
+Keywords: electron,desktop,anyplotlib,electron-microscopy,direct-electron
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: JavaScript
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.20.0
+Requires-Dist: anyplotlib>=0.7.3
+Requires-Dist: pyyaml
+Provides-Extra: tests
+Requires-Dist: pytest>=3.6; extra == "tests"
+Requires-Dist: pytest-timeout; extra == "tests"
+Dynamic: license-file
+
+# DE Shell
+
+[![PyPI](https://img.shields.io/pypi/v/de-shell.svg)](https://pypi.org/project/de-shell/)
+[![CI](https://github.com/CSSFrancis/de-shell/actions/workflows/ci.yml/badge.svg)](https://github.com/CSSFrancis/de-shell/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+
+The substrate Direct Electron's desktop apps are assembled from: **SpyDE**
+(offline analysis), **Ground Crew** (manual camera control) and **Autopilot**
+(automated acquisition). Each app is an Electron window over a Python
+sidecar, and everything the three have in common lives here — the
+Python↔JS message pipe, the window and its menus, the figure bridge over
+[anyplotlib](https://pypi.org/project/anyplotlib/), the sidecar process
+manager and its Python environment, the updater, the problem reporter, the
+Playwright harness.
+
+It contains **no domain logic**. No detectors, no microscopes, no signal
+types, no analysis. `tests/test_boundary.py` enforces that in a clean
+subprocess: the shell must stay installable without the science stack, so
+the live in-memory apps never acquire it transitively.
+
+```bash
+pip install de-shell
+```
+
+## One package
+
+The shell is **one pip package**, and the TypeScript rides inside the wheel:
+
+```
+pyproject.toml          the package: de-shell
+de_shell/               app loop, session base, actions, IPC, figures, compute, logging
+de_shell/js/            the TypeScript half, one folder per Electron target
+  main/                 Electron main: window, sidecar + stdout demuxer, python env, updater, reports
+  preload/              the contextBridge surface (exposeShellBridge)
+  renderer/             React: figure bridge, FigureFrame, the chrome slice of state
+  testing/              the Playwright harness (launchApp)
+tests/                  the Python suite (incl. the boundary test)
+package.json            DEV ONLY: typechecks and unit-tests de_shell/js; nothing is published to npm
+```
+
+The JavaScript that speaks the sidecar protocol ships in the same artifact
+as the Python that speaks it. One `pip install -U de-shell` moves both, and
+an app cannot end up with the two halves at different versions. The
+TypeScript is shipped as **source** and compiled by the consuming app's
+bundler, so there is no build step here and an editable install is
+live-editable from the app.
+
+## Consuming it
+
+**Python** — an ordinary dependency:
+
+```toml
+dependencies = ["de-shell>=0.2,<0.3"]
+```
+
+For hacking on the shell and an app at once, a sibling checkout as an
+editable path source (uv) or `pip install -e ../de-shell` overlays it.
+
+**Electron** — ask the installed package where its TypeScript is and link
+it into the project at a fixed path, then alias and `paths` through the link:
+
+```
+python -m de_shell.js        # prints …/de_shell/js
+```
+
+Autopilot's `electron/scripts/shell-link.mjs` is the reference: it makes
+`electron/shell` a junction (a symlink off Windows) to that folder, runs from
+npm's postinstall and from the vite config on every build, and re-points a
+stale link rather than trusting it. With it in place:
+
+```ts
+// electron.vite.config.ts
+const shell = ensureShellLink(__dirname)
+resolve: { alias: { '@de/shell-main': resolve(shell, 'main', 'index.ts') },
+           dedupe: ['react', 'react-dom'] }
+```
+
+```json
+// tsconfig.json
+"noEmit": true, "allowImportingTsExtensions": true,
+"paths": { "@de/shell-main": ["./shell/main/index.ts"], … }
+```
+
+`dedupe` matters: an editable checkout carries its own `node_modules` for
+its typecheck, and without it the renderer would bundle a second React. The
+peer dependencies — react, electron, electron-updater, @playwright/test —
+are the app's to declare; every app already does. The e2e specs take the
+harness from `shell/testing/harness.cjs`.
+
+Autopilot and SpyDE (on its `chore/de-shell-split` branch) are wired this
+way; Ground Crew still carries a copy under its `packages/` and is next.
+
+## Developing it
+
+```bash
+uv sync --extra tests && uv run pytest        # the Python suite
+ELECTRON_SKIP_BINARY_DOWNLOAD=1 npm install   # types only; drop the variable to run Electron
+npm run typecheck                             # every target, tests included, under tsconfig.json
+npm run test:unit                             # node --test over de_shell/js
+uv build                                      # the wheel — CI checks it carries de_shell/js
+```
+
+The unit tests run under Node's native type stripping, which resolves
+relative imports literally — so shell modules import their siblings with the
+`.ts` extension spelled out, and every tsconfig that compiles them (this one
+and each app's) sets `allowImportingTsExtensions`.
+
+CI (`.github/workflows/ci.yml`) runs the Python suite on Linux, Windows and
+macOS at the oldest and newest supported Python, the typecheck and unit
+tests, and builds the wheel and checks what it carries.
+
+## Releasing
+
+The version is written once, in `de_shell/__init__.py`. To release:
+
+1. Bump `__version__`, move the `CHANGELOG.md` entries under the new version.
+2. Commit, tag `vX.Y.Z`, push the tag.
+
+`.github/workflows/publish.yml` builds the distributions, refuses a tag that
+does not match `__version__`, and uploads to PyPI through trusted publishing
+— on pypi.org the project must list this repository, that workflow file and
+the `pypi` environment as a publisher (no token lives in the repo). Semver,
+with the 0.x caveat: a breaking change to the sidecar protocol bumps the
+minor, and the apps pin `>=0.x,<0.(x+1)`.
+
+## Provenance
+
+Merged 2026-09-02 from the three vendored copies, three-way against the
+SpyDE commit the app copies were taken from:
+
+* SpyDE `main` @ 1f3331d (v0.4.3): the problem reporter (`errorReport`,
+  `problemLog`, `sentryEnvelope`), `recentBackendOutput`, the workspace-member
+  wheels in `pythonEnv`, the update handoff that tree-kills the sidecar first,
+  `run_on_worker`'s in-flight count and `ComputeHandle` in `lifecycle.py`.
+* Autopilot @ 7f0651e: the sidecar's close handler forgets only ITS child, the
+  malformed-message report, the figure/stream fixes ported from the siblings,
+  the renderer state and FigureFrame changes.
+* Ground Crew `main` @ 26e853a: the spawn-error trap and 5 s tree-kill grace,
+  the resolved `uv` path, the open-directory dialog, `_pin_tile_band` (black
+  panes on large stills), JSON emit that never writes bare `NaN`, the harness
+  hardening, and the unit tests for all of it.
+
+Not yet included: Ground Crew's `dev/instrument-actions` shell deltas
+(`stdoutDemux`, `sizeReporter`, the `frameBytes` transport) — they ride a
+branch that has not merged.
+
+## Rules
+
+* **Nothing here mentions a detector, a signal type, or an analysis.** If
+  extracting something into the shell requires touching one, the boundary is
+  in the wrong place.
+* **The Python side stays tiny.** Every dependency added is one all three
+  apps install: numpy, anyplotlib, pyyaml, and that is the list.
+* **The protocol is the contract.** `PLOTAPP:` JSON lines and `PLOTBIN:`
+  binary frames over the sidecar's stdio. Both halves of it live in this one
+  package on purpose; keep it that way.
+* **LF line endings**, enforced by `.gitattributes`.
+
+## License
+
+MIT — see [LICENSE](LICENSE).

de_shell-0.2.0/README.md

@@ -0,0 +1,161 @@
+# DE Shell
+
+[![PyPI](https://img.shields.io/pypi/v/de-shell.svg)](https://pypi.org/project/de-shell/)
+[![CI](https://github.com/CSSFrancis/de-shell/actions/workflows/ci.yml/badge.svg)](https://github.com/CSSFrancis/de-shell/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+
+The substrate Direct Electron's desktop apps are assembled from: **SpyDE**
+(offline analysis), **Ground Crew** (manual camera control) and **Autopilot**
+(automated acquisition). Each app is an Electron window over a Python
+sidecar, and everything the three have in common lives here — the
+Python↔JS message pipe, the window and its menus, the figure bridge over
+[anyplotlib](https://pypi.org/project/anyplotlib/), the sidecar process
+manager and its Python environment, the updater, the problem reporter, the
+Playwright harness.
+
+It contains **no domain logic**. No detectors, no microscopes, no signal
+types, no analysis. `tests/test_boundary.py` enforces that in a clean
+subprocess: the shell must stay installable without the science stack, so
+the live in-memory apps never acquire it transitively.
+
+```bash
+pip install de-shell
+```
+
+## One package
+
+The shell is **one pip package**, and the TypeScript rides inside the wheel:
+
+```
+pyproject.toml          the package: de-shell
+de_shell/               app loop, session base, actions, IPC, figures, compute, logging
+de_shell/js/            the TypeScript half, one folder per Electron target
+  main/                 Electron main: window, sidecar + stdout demuxer, python env, updater, reports
+  preload/              the contextBridge surface (exposeShellBridge)
+  renderer/             React: figure bridge, FigureFrame, the chrome slice of state
+  testing/              the Playwright harness (launchApp)
+tests/                  the Python suite (incl. the boundary test)
+package.json            DEV ONLY: typechecks and unit-tests de_shell/js; nothing is published to npm
+```
+
+The JavaScript that speaks the sidecar protocol ships in the same artifact
+as the Python that speaks it. One `pip install -U de-shell` moves both, and
+an app cannot end up with the two halves at different versions. The
+TypeScript is shipped as **source** and compiled by the consuming app's
+bundler, so there is no build step here and an editable install is
+live-editable from the app.
+
+## Consuming it
+
+**Python** — an ordinary dependency:
+
+```toml
+dependencies = ["de-shell>=0.2,<0.3"]
+```
+
+For hacking on the shell and an app at once, a sibling checkout as an
+editable path source (uv) or `pip install -e ../de-shell` overlays it.
+
+**Electron** — ask the installed package where its TypeScript is and link
+it into the project at a fixed path, then alias and `paths` through the link:
+
+```
+python -m de_shell.js        # prints …/de_shell/js
+```
+
+Autopilot's `electron/scripts/shell-link.mjs` is the reference: it makes
+`electron/shell` a junction (a symlink off Windows) to that folder, runs from
+npm's postinstall and from the vite config on every build, and re-points a
+stale link rather than trusting it. With it in place:
+
+```ts
+// electron.vite.config.ts
+const shell = ensureShellLink(__dirname)
+resolve: { alias: { '@de/shell-main': resolve(shell, 'main', 'index.ts') },
+           dedupe: ['react', 'react-dom'] }
+```
+
+```json
+// tsconfig.json
+"noEmit": true, "allowImportingTsExtensions": true,
+"paths": { "@de/shell-main": ["./shell/main/index.ts"], … }
+```
+
+`dedupe` matters: an editable checkout carries its own `node_modules` for
+its typecheck, and without it the renderer would bundle a second React. The
+peer dependencies — react, electron, electron-updater, @playwright/test —
+are the app's to declare; every app already does. The e2e specs take the
+harness from `shell/testing/harness.cjs`.
+
+Autopilot and SpyDE (on its `chore/de-shell-split` branch) are wired this
+way; Ground Crew still carries a copy under its `packages/` and is next.
+
+## Developing it
+
+```bash
+uv sync --extra tests && uv run pytest        # the Python suite
+ELECTRON_SKIP_BINARY_DOWNLOAD=1 npm install   # types only; drop the variable to run Electron
+npm run typecheck                             # every target, tests included, under tsconfig.json
+npm run test:unit                             # node --test over de_shell/js
+uv build                                      # the wheel — CI checks it carries de_shell/js
+```
+
+The unit tests run under Node's native type stripping, which resolves
+relative imports literally — so shell modules import their siblings with the
+`.ts` extension spelled out, and every tsconfig that compiles them (this one
+and each app's) sets `allowImportingTsExtensions`.
+
+CI (`.github/workflows/ci.yml`) runs the Python suite on Linux, Windows and
+macOS at the oldest and newest supported Python, the typecheck and unit
+tests, and builds the wheel and checks what it carries.
+
+## Releasing
+
+The version is written once, in `de_shell/__init__.py`. To release:
+
+1. Bump `__version__`, move the `CHANGELOG.md` entries under the new version.
+2. Commit, tag `vX.Y.Z`, push the tag.
+
+`.github/workflows/publish.yml` builds the distributions, refuses a tag that
+does not match `__version__`, and uploads to PyPI through trusted publishing
+— on pypi.org the project must list this repository, that workflow file and
+the `pypi` environment as a publisher (no token lives in the repo). Semver,
+with the 0.x caveat: a breaking change to the sidecar protocol bumps the
+minor, and the apps pin `>=0.x,<0.(x+1)`.
+
+## Provenance
+
+Merged 2026-09-02 from the three vendored copies, three-way against the
+SpyDE commit the app copies were taken from:
+
+* SpyDE `main` @ 1f3331d (v0.4.3): the problem reporter (`errorReport`,
+  `problemLog`, `sentryEnvelope`), `recentBackendOutput`, the workspace-member
+  wheels in `pythonEnv`, the update handoff that tree-kills the sidecar first,
+  `run_on_worker`'s in-flight count and `ComputeHandle` in `lifecycle.py`.
+* Autopilot @ 7f0651e: the sidecar's close handler forgets only ITS child, the
+  malformed-message report, the figure/stream fixes ported from the siblings,
+  the renderer state and FigureFrame changes.
+* Ground Crew `main` @ 26e853a: the spawn-error trap and 5 s tree-kill grace,
+  the resolved `uv` path, the open-directory dialog, `_pin_tile_band` (black
+  panes on large stills), JSON emit that never writes bare `NaN`, the harness
+  hardening, and the unit tests for all of it.
+
+Not yet included: Ground Crew's `dev/instrument-actions` shell deltas
+(`stdoutDemux`, `sizeReporter`, the `frameBytes` transport) — they ride a
+branch that has not merged.
+
+## Rules
+
+* **Nothing here mentions a detector, a signal type, or an analysis.** If
+  extracting something into the shell requires touching one, the boundary is
+  in the wrong place.
+* **The Python side stays tiny.** Every dependency added is one all three
+  apps install: numpy, anyplotlib, pyyaml, and that is the list.
+* **The protocol is the contract.** `PLOTAPP:` JSON lines and `PLOTBIN:`
+  binary frames over the sidecar's stdio. Both halves of it live in this one
+  package on purpose; keep it that way.
+* **LF line endings**, enforced by `.gitattributes`.
+
+## License
+
+MIT — see [LICENSE](LICENSE).

de_shell-0.2.0/de_shell/__init__.py

@@ -0,0 +1,25 @@
+"""
+de_shell — the Python half of the DE desktop-app shell.
+
+The substrate three applications share: SpyDE (offline EM analysis),
+de-groundcrew (live camera/hardware control) and de-autopilot (automated
+acquisition). It answers "how do I be a desktop app with a Python brain and
+pictures in it?" — the asyncio stdin/stdout loop, the PLOTAPP IPC protocol, log
+streaming, the window/figure registry, the action + staged-wizard framework, and
+the anyplotlib plotting wrapper.
+
+It answers nothing about what the data IS. No HyperSpy, no Dask, no
+RosettaSciIO, no pyxem — de-groundcrew and de-autopilot are live, in-memory
+applications and must not acquire those dependencies transitively. That
+constraint is what fixes the boundary, and `tests/test_boundary.py` enforces
+it in a clean subprocess: anything that answers "what is the data and what do
+you do to it?" (the array-cache tiering, the signal tree, the navigator read
+path, the distributed compute branch, every action handler) stays in the app.
+"""
+from __future__ import annotations
+
+#: The package version — the one place it is written. pyproject.toml reads it
+#: (dynamic version) and the release workflow refuses a tag that disagrees.
+__version__ = "0.2.0"
+
+__all__ = ["ipc", "log_stream", "process_guard", "debug_flags", "compute"]

de_shell-0.2.0/de_shell/actions/context.py

@@ -0,0 +1,62 @@
+"""
+context.py — ActionContext: the adapter passed to action functions.
+
+An action function is handed one of these instead of reaching for the UI. It
+carries the clicked plot, the parameter values the frontend's panel collected
+(forwarded as kwargs), and a per-plot scratch dict for state that must outlive a
+single invocation (an FFT window, a toggle group, a widget the action added).
+
+Everything it touches is duck-typed — ``plot.plot_window``, ``plot.session`` —
+so it does not care what kind of plot or session an app has.
+"""
+from __future__ import annotations
+
+from typing import Any
+
+
+class ActionContext:
+    """The attribute surface an action function is written against."""
+
+    def __init__(self, plot, params: dict[str, Any] | None = None,
+                 action_name: str = ""):
+        self.plot = plot
+        self.params = params or {}
+        self.action_name = action_name
+
+        # Per-plot persistent action state (FFT windows, toggle groups, …).
+        # Stored ON THE PLOT so it survives across action invocations — an
+        # ActionContext is built fresh for each one.
+        if not hasattr(plot, "_action_widgets"):
+            plot._action_widgets = {}
+        self.action_widgets = plot._action_widgets
+
+    # ── Plot / session access ─────────────────────────────────────────────────
+
+    @property
+    def plot_window(self):
+        return self.plot.plot_window
+
+    @property
+    def session(self):
+        return self.plot.session
+
+    # ── Stateful action registration ──────────────────────────────────────────
+
+    def register_action_plot_item(self, action_name: str, item, key: str) -> None:
+        slot = self.action_widgets.setdefault(action_name, {})
+        slot.setdefault("plot_items", {})[key] = item
+
+    def register_action_plot_window(self, action_name: str, plot_window, key: str) -> None:
+        slot = self.action_widgets.setdefault(action_name, {})
+        slot.setdefault("plot_windows", {})[key] = plot_window
+
+    def add_action_widget(self, action_name: str, widget=None, layout=None) -> None:
+        slot = self.action_widgets.setdefault(action_name, {})
+        slot["widget"] = widget
+        slot["layout"] = layout
+
+    def actions(self) -> list:
+        """The toolbar lives in the frontend, so there are no host-side action
+        objects to return. Kept because action code written against the old Qt
+        toolbar still calls it."""
+        return []

de_shell-0.2.0/de_shell/actions/figure_registry.py

@@ -0,0 +1,53 @@
+"""
+figure_registry.py — per-window keep-alive for bare anyplotlib figures.
+
+Result windows that are NOT registered ``Plot``s emit raw ``figure`` messages
+whose Python-side figure objects must be kept referenced, or their widget
+callbacks are garbage-collected while the window is still open. Historically
+each module kept its own append-only ``_ALIVE`` list, which leaked every figure
+for the process lifetime.
+
+This registry keys the references by ``window_id`` and is evicted from the
+session's ``_forget_window``, so a figure lives exactly as long as its window.
+
+Apps hang their own per-window state off the same eviction via
+:func:`register_evictor`, rather than this module reaching into them — which is
+what it used to do (a hardcoded import of SpyDE's ``actions.views``).
+"""
+from __future__ import annotations
+
+from typing import Any, Callable
+
+_FIGS: dict[int, list[Any]] = {}
+
+#: App callbacks run when a window is forgotten. See `register_evictor`.
+_EVICTORS: list[Callable[[int], None]] = []
+
+
+def register_evictor(fn: Callable[[int], None]) -> None:
+    """Register ``fn(window_id)`` to run whenever a window is forgotten.
+
+    For app state keyed by window id that must die with the window — SpyDE's
+    per-window chip-view arrays, for instance. Registering the same function
+    twice is a no-op, so a module can call this at import without guarding.
+    """
+    if fn not in _EVICTORS:
+        _EVICTORS.append(fn)
+
+
+def keep_alive(window_id: int, fig: Any) -> None:
+    """Keep *fig* referenced until *window_id*'s window is forgotten."""
+    _FIGS.setdefault(int(window_id), []).append(fig)
+
+
+def forget_window(window_id: int) -> None:
+    """Drop every reference held for *window_id*, and run the app's evictors."""
+    wid = int(window_id)
+    _FIGS.pop(wid, None)
+    for fn in _EVICTORS:
+        try:
+            fn(wid)
+        except Exception:
+            # Teardown must not fail: this runs while a window is going away,
+            # and one app's bookkeeping error should not strand the rest.
+            pass