starbash 0.3.0__tar.gz → 0.3.2__tar.gz

{starbash-0.3.0 → starbash-0.3.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: starbash
-Version: 0.3.0
+Version: 0.3.2
 Summary: A tool for automating/standardizing/sharing astrophotography workflows.
 License-File: LICENSE
 Author: Kevin Hester
@@ -43,12 +43,14 @@ A tool for automating/standardizing/sharing astrophotography workflows.
 ## What is starbash?
 <img src="https://raw.githubusercontent.com/geeksville/starbash/refs/heads/main/doc/icon/generated/web/icon-192.png" alt="Starbash: Astrophotography workflows simplified" width="30%" align="right" style="margin-bottom: 20px;">
 
-* Automatic - with sensible defaults that you can change as needed.  Just a few keystrokes to preprocess an entire repo of raw images.
-* Easy - provides a 'seestar-like' autoprocessing all your sessions (by default).  You can optionally do manual postprocessing with all this 'grunt work' preprocessing automated.
+* Automatic - with sensible defaults that you can change as needed.  Just a few keystrokes to auto-process an **entire** repo of raw images.  It uses sensible customizable defaults - you can optionally tweak parameters and reprocess as needed.
+* Easy - provides a 'seestar-like' autoprocessing all your sessions (by default).  You can optionally do manual postprocessing with all this 'grunt work' default processing automated.
 * Fast<sup>1</sup> - even with large image repositories.  Automatic master bias and flat generation.  Automatic preprocessing/ stacking/background-elimination with reasonable defaults.
-* Multi-session - by default.  So your workflows can stack from multiple nights (and still use the correct flats etc...).
+* Multi-session - by default.  So your workflows can stack from multiple nights (and still use/auto-generate the correct flats, darks/biases etc...).
 * Shareable - you can share/use recipes for image preprocessing flows.  The eventual goal is a transition away from opaque burried scripts.  Recipes can come from multiple repos and you or other orgs can host recipes.
+* Automatic (optional) posting/creation of your generated images and workflows to a free ['github pages'](https://docs.github.com/en/pages) website.  [Sample](https://geeksville.github.io/starbash-public/targets/sh2-91/).
 * Attribution (by default) - a full set of contributions to final image can be tracked: instrument, raw images, processed-by, recipes-by/version etc...
+* Full session/FITS aware history - select targets by name, date-range, instrument etc... 
 
 (This project is currently 'alpha' and missing recipes for some workflows, but adding new recipes is easy and we're happy to help.  Please file a GitHub issue if your images are not auto-processed and we'll work out a fix.)
 
@@ -58,7 +60,7 @@ A tool for automating/standardizing/sharing astrophotography workflows.
 ## What is starbash NOT
 
 ### Not a new image processing tool (like Siril)
-It is important to understand that Starbash is primarily a tool for **repeatable**, **sharable**, **semi-automated** workflows.  It is tool agnostic and really just focuses on the relationship between tools, images and sessions.  It builds upon the great work of existing tools (currently mainly Siril and Graxpert) so we can grow an ecosystem of recipes.
+It is important to understand that Starbash is primarily a tool for **repeatable**, **sharable**, **semi-automated** workflows.  It is tool agnostic and really just focuses on the relationship between tools, images and sessions.  It builds upon the great work of existing tools (currently mainly [Siril](https://siril.org/), [GraXpert](https://graxpert.com/), [Blurxterminator](https://www.rc-astro.com/software/bxt/) and [Starnet2](https://starnetastro.com/cli-tools/)) so we can grow an ecosystem of recipes.
 
 ### Not a set of scripts
 
@@ -98,7 +100,12 @@ If you are interested in alpha-testing we ❤️ you.  This README should have e
 
 ### From the alpha 3 (0.3.0) release (2026/08/31)
 
-* Auto publish your finished images and workflow report to github-sites
+- Added end-to-end GitHub Pages publishing, including a friendlier GitHub setup flow and automatic token refresh.
+- Introduced the new processing layout with shareable per-target configuration and improved recipe/dependency handling.
+- Added Starnet, BlurXTerminator, and NoiseXTerminator support, with live Siril progress and automatic star recomposition.
+- Improved FITS/session reporting with focal-length and FWHM information, plus more reliable OSC stacking.
+- Added safer cleanup of temporary processing files and clearer warnings when external tools are missing.
+- Publishing is now about twice as fast thanks to concurrent uploads.
 
 ## Features coming soon
 

{starbash-0.3.0 → starbash-0.3.2}/README.md

@@ -9,12 +9,14 @@ A tool for automating/standardizing/sharing astrophotography workflows.
 ## What is starbash?
 <img src="https://raw.githubusercontent.com/geeksville/starbash/refs/heads/main/doc/icon/generated/web/icon-192.png" alt="Starbash: Astrophotography workflows simplified" width="30%" align="right" style="margin-bottom: 20px;">
 
-* Automatic - with sensible defaults that you can change as needed.  Just a few keystrokes to preprocess an entire repo of raw images.
-* Easy - provides a 'seestar-like' autoprocessing all your sessions (by default).  You can optionally do manual postprocessing with all this 'grunt work' preprocessing automated.
+* Automatic - with sensible defaults that you can change as needed.  Just a few keystrokes to auto-process an **entire** repo of raw images.  It uses sensible customizable defaults - you can optionally tweak parameters and reprocess as needed.
+* Easy - provides a 'seestar-like' autoprocessing all your sessions (by default).  You can optionally do manual postprocessing with all this 'grunt work' default processing automated.
 * Fast<sup>1</sup> - even with large image repositories.  Automatic master bias and flat generation.  Automatic preprocessing/ stacking/background-elimination with reasonable defaults.
-* Multi-session - by default.  So your workflows can stack from multiple nights (and still use the correct flats etc...).
+* Multi-session - by default.  So your workflows can stack from multiple nights (and still use/auto-generate the correct flats, darks/biases etc...).
 * Shareable - you can share/use recipes for image preprocessing flows.  The eventual goal is a transition away from opaque burried scripts.  Recipes can come from multiple repos and you or other orgs can host recipes.
+* Automatic (optional) posting/creation of your generated images and workflows to a free ['github pages'](https://docs.github.com/en/pages) website.  [Sample](https://geeksville.github.io/starbash-public/targets/sh2-91/).
 * Attribution (by default) - a full set of contributions to final image can be tracked: instrument, raw images, processed-by, recipes-by/version etc...
+* Full session/FITS aware history - select targets by name, date-range, instrument etc... 
 
 (This project is currently 'alpha' and missing recipes for some workflows, but adding new recipes is easy and we're happy to help.  Please file a GitHub issue if your images are not auto-processed and we'll work out a fix.)
 
@@ -24,7 +26,7 @@ A tool for automating/standardizing/sharing astrophotography workflows.
 ## What is starbash NOT
 
 ### Not a new image processing tool (like Siril)
-It is important to understand that Starbash is primarily a tool for **repeatable**, **sharable**, **semi-automated** workflows.  It is tool agnostic and really just focuses on the relationship between tools, images and sessions.  It builds upon the great work of existing tools (currently mainly Siril and Graxpert) so we can grow an ecosystem of recipes.
+It is important to understand that Starbash is primarily a tool for **repeatable**, **sharable**, **semi-automated** workflows.  It is tool agnostic and really just focuses on the relationship between tools, images and sessions.  It builds upon the great work of existing tools (currently mainly [Siril](https://siril.org/), [GraXpert](https://graxpert.com/), [Blurxterminator](https://www.rc-astro.com/software/bxt/) and [Starnet2](https://starnetastro.com/cli-tools/)) so we can grow an ecosystem of recipes.
 
 ### Not a set of scripts
 
@@ -64,7 +66,12 @@ If you are interested in alpha-testing we ❤️ you.  This README should have e
 
 ### From the alpha 3 (0.3.0) release (2026/08/31)
 
-* Auto publish your finished images and workflow report to github-sites
+- Added end-to-end GitHub Pages publishing, including a friendlier GitHub setup flow and automatic token refresh.
+- Introduced the new processing layout with shareable per-target configuration and improved recipe/dependency handling.
+- Added Starnet, BlurXTerminator, and NoiseXTerminator support, with live Siril progress and automatic star recomposition.
+- Improved FITS/session reporting with focal-length and FWHM information, plus more reliable OSC stacking.
+- Added safer cleanup of temporary processing files and clearer warnings when external tools are missing.
+- Publishing is now about twice as fast thanks to concurrent uploads.
 
 ## Features coming soon
 

{starbash-0.3.0 → starbash-0.3.2}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "starbash"
-version = "0.3.0"
+version = "0.3.2"
 description = "A tool for automating/standardizing/sharing astrophotography workflows."
 authors = ["Kevin Hester <kevinh@geeksville.com>"]
 readme = "README.md"

{starbash-0.3.0 → starbash-0.3.2}/src/starbash/app.py

@@ -15,6 +15,7 @@ from rich.progress import track
 from toml_repo import Repo, RepoManager, get_config_suffix
 
 import starbash
+from starbash import doit_types
 from starbash.aliases import (
     Aliases,
     get_aliases,
@@ -185,6 +186,9 @@ class Starbash:
 
         # Load app defaults and initialize the repository manager
         self._init_repos()
+        doit_types.configure_max_contexts(
+            self.user_repo.get("config.max_contexts", doit_types.DEFAULT_MAX_CONTEXTS)
+        )
         self._init_analytics(cmd)  # after init repos so we have user prefs
 
         app_version = version("starbash")

{starbash-0.3.0 → starbash-0.3.2}/src/starbash/commands/publish.py

@@ -27,10 +27,10 @@ UPLOAD_PATH_BLACKLIST = ("Gemfile",)
 MAX_BLOB_UPLOADS = 4
 
 
-def _rewrite() -> Path:
+def _rewrite(github_username: str | None = None) -> Path:
     site: Path | None = None
     with Starbash("publish.github.rewrite") as sb:
-        site = GitHubPublisher(sb).publish()
+        site = GitHubPublisher(sb, github_username=github_username).publish()
     assert site is not None
     console.print(f"Generated site: {site}")
     return site
@@ -248,7 +248,18 @@ def _publish_github(dry_run: bool, login: bool) -> None:
             if credential is None:
                 credential = _authenticate()
 
-    site = _rewrite()
+    service: GitHubService | None = None
+    owner: str | None = None
+    if not dry_run:
+        assert credential is not None
+        service = _credential_service(credential)
+        if credential.needs_refresh() and credential.refresh_token:
+            service.apply_token_response(
+                service.refresh_access_token(CLIENT_ID, credential.refresh_token)
+            )
+        owner = str(service.user()["login"])
+
+    site = _rewrite(owner) if owner else _rewrite()
     files = sorted(
         path
         for path in site.rglob("*")
@@ -271,13 +282,10 @@ def _publish_github(dry_run: bool, login: bool) -> None:
         console.print("No GitHub repository, branch, commit, or Pages configuration was changed.")
         return
     assert credential is not None
+    assert service is not None
+    assert owner is not None
     with analytics_start_span(name="github", op="upload"):
-        service = _credential_service(credential)
         try:
-            if credential.needs_refresh() and credential.refresh_token:
-                service.apply_token_response(
-                    service.refresh_access_token(CLIENT_ID, credential.refresh_token)
-                )
             with Progress(
                 SpinnerColumn(),
                 TextColumn("[progress.description]{task.description}"),
@@ -288,7 +296,6 @@ def _publish_github(dry_run: bool, login: bool) -> None:
             ) as progress:
                 operation = progress.add_task("Contacting GitHub", total=8 + len(files))
 
-                owner = str(service.user()["login"])
                 pages_url = f"https://{owner}.github.io/starbash-public/"
                 progress.update(operation, description=f"Authenticated as {owner}", advance=1)
                 _require_app_installation(service)

{starbash-0.3.0 → starbash-0.3.2}/src/starbash/defaults/starbash.toml

@@ -91,6 +91,14 @@ model.long = "Optolong 2-inch L-Extreme Dual Band 7nm Ha+Oiii Filter"
 url.info = "https://www.highpointscientific.com/optolong-2-inch-l-extreme-dual-band-7nm-ha-oiii-filter-lxt-200"
 subtype = "HaOiii"
 
+[[equipment]]
+type = "filter"
+vendor = "Agena"
+fits.filter = "Tri" # must match the FITS header FILTER field for this filter if you want auto matching.
+model.short = "ant-tri"
+model.long = "Agena Tri-Band RGB Ultra Filter"
+url.info = "https://agenaastro.com/antlia-triband-rgb-ultra-filter-2-mounted.html"
+
 [[equipment]]
 type = "mount"
 vendor = "Juwei"
@@ -138,7 +146,9 @@ HaOiii = ["HaOiii", "HaO3", "Duo"]
 # Astro is the name Dwarf3 uses for their IR cut filter
 # VIS is the name Dwarf3 uses for their 'clear' filter
 # IRCUT/LP are the names Seestar uses for their filters
-None = ["None", "Astro", "IRCUT", "LP", "VIS"]
+# Tri is a common name for a tri-band light pollution filter (LPF)
+
+None = ["None", "Astro", "IRCUT", "LP", "VIS", "Tri"]
 
 camera_osc = ["OSC", "ZWO ASI2600MC Duo"]
 camera_seestar = ["Seestar", "Seestar S50", "Seestar S30", "Seestar S30 Pro"]
@@ -157,7 +167,10 @@ camera_seestar = ["Seestar", "Seestar S50", "Seestar S30", "Seestar S30 Pro"]
 # http://www.antliafilter.com/pd.jsp?fromColId=2&id=160#_pp=2_671
 SiiHb = ["SiiHb", "S2Hb"]
 
+[config]
+# Set this lower if you need to reduce disk usage. If omitted, the default is 2 contexts.
+max_contexts = 2
 
 # List the initial search path to find other repos, when the app launches it always first loads this sb.toml file and this file should list all of the
 # standard default repo locations.  When searching repos, repos listed LAST have precedence, so target file can override the root processing defaults,
-# then the user prefs, then a live github URL or whatever
+# then the user prefs, then a live github URL or whatever

{starbash-0.3.0 → starbash-0.3.2}/src/starbash/doit.py

@@ -22,7 +22,7 @@ from toml_repo import Repo
 from starbash import InputDef
 from starbash.database import ImageRow
 from starbash.doit_types import TaskDict
-from starbash.exception import UserHandledError
+from starbash.exception import FilesystemUnavailableError, UserHandledError
 from starbash.os import symlink_or_copy
 from starbash.paths import get_user_cache_dir
 from starbash.tool.base import Tool
@@ -257,7 +257,9 @@ def merge_to(base_name: str, fi: FileInfo) -> None:
                             indexed_ids = []
                             break
                         indexed_ids.append(source_ids[source_index - 1])
-                    if len(indexed_ids) == len(matching_files) and len(set(indexed_ids)) == len(indexed_ids):
+                    if len(indexed_ids) == len(matching_files) and len(set(indexed_ids)) == len(
+                        indexed_ids
+                    ):
                         resolved_source_ids = indexed_ids
                     else:
                         logging.warning(
@@ -269,7 +271,9 @@ def merge_to(base_name: str, fi: FileInfo) -> None:
                         )
 
             if resolved_source_ids is not None:
-                for matching_file, source_id in zip(matching_files, resolved_source_ids, strict=True):
+                for matching_file, source_id in zip(
+                    matching_files, resolved_source_ids, strict=True
+                ):
                     source_path = Path(matching_file)
                     collected_files.append(source_path)
                     provenance[source_path.name] = source_id
@@ -288,7 +292,7 @@ def merge_to(base_name: str, fi: FileInfo) -> None:
     ):
         dest_name = f"{base_name}_{index:05d}.fits"
         dest_path = output_dir / dest_name
-        #logging.debug(f"Linking {source_file} to {dest_path}")
+        # logging.debug(f"Linking {source_file} to {dest_path}")
         symlink_or_copy(str(source_file), str(dest_path))
 
     merged_provenance = {
@@ -355,6 +359,17 @@ class ToolAction(BaseAction):
                 self.result = self.tool.run(
                     self.commands, context=context, cwd=self.cwd, log_out=logfile, **self.parameters
                 )
+        except OSError as e:
+            error = FilesystemUnavailableError(
+                f"running {self.tool.name} for task '{self.task.name}'", e
+            )
+            logging.error("%s (%s)", error, e)
+            self.task.meta["exception"] = error
+            return TaskFailed(str(error))
+        except FilesystemUnavailableError as e:
+            logging.error("%s", e)
+            self.task.meta["exception"] = e
+            return TaskFailed(str(e))
         except Exception as e:
             # We pass back any exceptions in task.meta - so that our ConsoleReporter can pick them up (doit normally strips exceptions)
             self.task.meta["exception"] = e
@@ -422,6 +437,12 @@ class ProcessingResult:
                 # General error from user misconfiguration or tools - not a bug in our code
                 logging.error(f"Skipping run due to: {e}")
                 self.notes = str(e)
+            elif isinstance(e, OSError):
+                self.notes = (
+                    "The filesystem became unavailable during processing. "
+                    "Check that the drive or network mount is connected, then retry."
+                )
+                logging.error("Filesystem unavailable during processing: %s", e)
             else:
                 # Unexpected exception - log it and re-raise
                 logging.exception("Unexpected error during processing:")
@@ -562,7 +583,7 @@ class StarbashDoit(TaskLoader2):
             "verbosity": 2,
             "dep_file": dep_file,
             "reporter": MyReporter,
-            "backend": "dbm", # the json backend is slow and buggy, use dbm instead
+            "backend": "dbm",  # the json backend is slow and buggy, use dbm instead
         }
 
     def load_tasks(self, cmd: Any, pos_args: Any) -> list[Task]:

starbash-0.3.2/src/starbash/doit_types.py

@@ -0,0 +1,85 @@
+"""Shared types and utilities for doit processing to avoid circular imports."""
+
+from __future__ import annotations
+
+import logging
+import os
+import shutil
+from pathlib import Path
+from typing import Any
+
+from starbash.paths import get_user_cache_dir
+
+type TaskDict = dict[str, Any]  # a doit task dictionary
+
+DEFAULT_MAX_CONTEXTS = 2
+max_contexts = DEFAULT_MAX_CONTEXTS
+
+
+def configure_max_contexts(value: Any) -> None:
+    """Set the maximum number of processing contexts from user configuration.
+
+    Invalid values are ignored so a malformed preference cannot prevent
+    Starbash from starting.
+    """
+    global max_contexts
+
+    if isinstance(value, bool) or not isinstance(value, int) or value < 1:
+        logging.warning(
+            "Ignoring invalid max_contexts preference %r; using %d.",
+            value,
+            DEFAULT_MAX_CONTEXTS,
+        )
+        max_contexts = DEFAULT_MAX_CONTEXTS
+        return
+
+    max_contexts = value
+
+
+def get_processing_dir() -> Path:
+    """Get the base directory for processing contexts."""
+    cache_dir = get_user_cache_dir()
+    processing_dir = cache_dir / "processing"
+    processing_dir.mkdir(parents=True, exist_ok=True)
+    return processing_dir
+
+
+def cleanup_old_contexts() -> None:
+    """Remove oldest context directories if we exceed max_contexts."""
+    processing_dir = get_processing_dir()
+    logging.debug(f"Removing old processing contexts in: {processing_dir}")
+    if not processing_dir.exists():
+        return
+
+    # Safety guard: if we're running under pytest but the processing dir is NOT a test
+    # override (no explicit cache override and no STARBASH_CACHE_DIR), refuse to delete
+    # anything. A leaked test global (e.g. a small max_contexts from a test config) must
+    # never prune the real user's cache.
+    from starbash import paths
+
+    if os.environ.get("PYTEST_CURRENT_TEST") is not None:
+        using_override = (
+            paths._override_cache_dir is not None or os.getenv("STARBASH_CACHE_DIR")
+        )
+        if not using_override:
+            logging.warning(
+                "Refusing to clean processing contexts during tests without a cache override: %s",
+                processing_dir,
+            )
+            return
+
+    # Get all subdirectories in processing_dir
+    contexts = [d for d in processing_dir.iterdir() if d.is_dir()]
+
+    # If we have more than max_contexts, delete the oldest ones
+    if len(contexts) > max_contexts:
+        # Sort by modification time (oldest first)
+        contexts.sort(key=lambda d: d.stat().st_mtime)
+
+        # Calculate how many to delete
+        num_to_delete = len(contexts) - max_contexts
+
+        # Delete the oldest directories
+        for context_dir in contexts[:num_to_delete]:
+            logging.debug(f"Removing old processing context: {context_dir}")
+            shutil.rmtree(context_dir, ignore_errors=True)

{starbash-0.3.0 → starbash-0.3.2}/src/starbash/exception.py

@@ -25,6 +25,17 @@ class UserHandledError(NonSoftwareError):
         return False
 
 
+class FilesystemUnavailableError(UserHandledError):
+    """Raised when a filesystem disappears while processing is in progress."""
+
+    def __init__(self, operation: str, cause: OSError) -> None:
+        super().__init__(
+            f"The filesystem became unavailable while {operation}. "
+            "Check that the drive or network mount is connected, then retry."
+        )
+        self.cause = cause
+
+
 def raise_missing_repo(kind: str) -> NoReturn:
     """Raise a UserHandledError indicating that a repository of the given kind is missing."""
     raise UserHandledError(
@@ -54,6 +65,7 @@ class NoSuitableMastersException(NonFatalException):
 
 __all__ = [
     "UserHandledError",
+    "FilesystemUnavailableError",
     "NonFatalException",
     "NotEnoughFilesError",
     "NoSuitableMastersException",

{starbash-0.3.0 → starbash-0.3.2}/src/starbash/filtering.py

@@ -30,6 +30,10 @@ def _apply_filter(requires: RequireDef, candidates: list[ImageRow]) -> list[Imag
 
     kind = get_safe(requires, "kind")
     value = requires.get("value")  # value is optional for some kinds
+    # Optional `invert` negates the boolean match so non-matching candidates are
+    # kept (e.g. select sessions whose filter is NOT narrowband). Only meaningful
+    # for boolean-match kinds (metadata/camera/unprocessed/filename), not min_count.
+    invert = bool(requires.get("invert", False))
 
     # Stage 1: Filter candidates using kind-specific filter functions
     def _filter_metadata(metadata: Metadata) -> bool:
@@ -95,7 +99,10 @@ def _apply_filter(requires: RequireDef, candidates: list[ImageRow]) -> list[Imag
         raise ValueError(f"Unknown requires kind: {kind}")
 
     # Apply the filter function to all candidates
-    filtered_candidates = [img for img in candidates if filter_func(img)]
+    if invert:
+        filtered_candidates = [img for img in candidates if not filter_func(img)]
+    else:
+        filtered_candidates = [img for img in candidates if filter_func(img)]
 
     # Stage 2: Handle min_count check after filtering
     if kind == "min_count":

{starbash-0.3.0 → starbash-0.3.2}/src/starbash/fits.py

@@ -8,11 +8,34 @@ from astropy.io import fits
 __all__ = ["read_dimensions", "read_fits_header"]
 
 
+def _json_safe(value: Any) -> Any:
+    """Convert an astropy header value into a plain JSON-serializable Python value.
+
+    Commentary cards (COMMENT/HISTORY/blank keyword) come back as
+    ``_HeaderCommentaryCards`` (a list-like of strings) and other cards can be
+    ``bool``, numpy scalars etc. None of those serialize via ``json.dumps``.
+    """
+    # Commentary cards: list-like container of strings; join into one string
+    if isinstance(value, fits.header._HeaderCommentaryCards):
+        return "\n".join(value)
+    if isinstance(value, (list, tuple)):
+        return [_json_safe(v) for v in value]
+    if isinstance(value, bool):  # must precede int check (bool is an int subclass)
+        return value
+    if isinstance(value, (int, float, str)) or value is None:
+        return value
+    # numpy scalars and anything else: fall back to str()
+    return str(value)
+
+
 def read_fits_header(file_path: str | Path) -> dict[str, Any]:
-    """Read and return the primary FITS header from ``file_path``."""
+    """Read and return the primary FITS header from ``file_path``.
+
+    All values are converted to plain JSON-serializable Python types.
+    """
     with fits.open(str(file_path), memmap=False) as hdul:
         hdu0: Any = hdul[0]
-        return dict(hdu0.header)
+        return {key: _json_safe(value) for key, value in hdu0.header.items()}
 
 
 def read_dimensions(file_path: str | Path) -> tuple[int, int]: