normalize-tabular-data 0.1.1__tar.gz → 0.1.3__tar.gz

{normalize_tabular_data-0.1.1 → normalize_tabular_data-0.1.3}/LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2026, Members of Service Employees International Union (SEIU)
+Copyright (c) 2026, Service Employees International Union (SEIU)
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

{normalize_tabular_data-0.1.1 → normalize_tabular_data-0.1.3}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: normalize-tabular-data
-Version: 0.1.1
-Summary: TUI for normalizing tabular data with polars
+Version: 0.1.3
+Summary: TUI for normalizing tabular data with Polars
 Author: David Mertz, Ph.D.
 Author-email: David Mertz, Ph.D. <mertz@gnosis.cx>
 License-Expression: BSD-2-Clause
@@ -11,6 +11,7 @@ Requires-Dist: textual>=8.2.8
 Requires-Dist: polars>=1.44,<2
 Requires-Dist: fastexcel>=0.10
 Requires-Dist: xlsxwriter>=0.9
+Requires-Dist: platformdirs>=3.0
 Requires-Python: >=3.14
 Description-Content-Type: text/markdown
 
@@ -26,6 +27,8 @@ Load a file, see a preview, build up a pipeline of normalization operations
 combine/split columns, drop columns), then save the
 cleaned result.
 
+![TUI preview](https://raw.githubusercontent.com/SEIU-Tech/normalize-tabular-data/main/docs/screenshot.png)
+
 ## Running
 
 ### Persistent install
@@ -43,15 +46,64 @@ uv tool upgrade normalize-tabular-data
 ```bash
 uvx normalize-tabular-data
 # pin a specific version:
-uvx normalize-tabular-data==0.1.0
+uvx normalize-tabular-data==0.1.1
 ```
 
-### Straight from a checkout
+### From a git checkout
+
+Within the directory of the cloned repository:
 
 ```bash
 uv run normalize-tabular-data
 ```
 
+## Command line
+
+An optional `FILE` argument names a table to open at startup, as if you
+had picked it from the in-app dialog.
+
+```bash
+normalize-tabular-data data/members.csv      # or with uvx/uv run
+uvx normalize-tabular-data data/members.tsv
+```
+
+If the file is missing or its format cannot be read, the TUI still
+starts — it shows an alert toast and you can open something else.
+
+### Open with a script
+
+With `--script/-s`, an `.ntd` script is played on `FILE` as soon as it
+loads (same flow as the `p` key, same stop-and-rollback on a step that
+does not fit the file):
+
+```bash
+uvx normalize-tabular-data data/members.csv -s data/members.ntd
+```
+
+### Headless run (`-x`, `--run`)
+
+`--run/-x` skips the TUI: it opens `FILE`, plays `SCRIPT` step by step,
+saves the result and exits. Each operation prints to stdout as it is
+applied; anything else — including a failing step, named and rolled
+back exactly as the interactive player would — goes to stderr with a
+nonzero exit code.
+
+```bash
+# saves data/members-normalized.csv next to the original
+uvx normalize-tabular-data members.csv -s members.ntd -x
+
+# choose the destination (format taken from its extension):
+uvx normalize-tabular-data members.csv -s members.ntd -x -o out/members.parquet
+
+# write the CSV to a pipe instead:
+uvx normalize-tabular-data members.csv -s members.ntd -x -o - | gzip > clean.csv.gz
+```
+
+With `-o -` (attached `-o-` works too), the transformed table is written
+to stdout itself and the step lines are suppressed, so stdout carries
+only the data. After a successful run the confirmation line goes to
+stderr.
+
 ## Usage
 
 Launch with `normalize-tabular-data`. Keys:
@@ -68,16 +120,22 @@ Launch with `normalize-tabular-data`. Keys:
 
 ## Operations
 
-- **Normalize dates** — parse a messy date column of *any* input format into
-  canonical UTC datetimes; unparseable values become null.
+- **Normalize date(t)imes** — parse a messy date column of *any* input
+  format into canonical UTC datetimes at millisecond resolution;
+  unparseable values become null. CSV/TSV/JSONLines export serializes
+  datetime columns as second-resolution `year-month-dayThh:mm:ss`
+  strings; Parquet and Excel keep the real datetime values.
+- **Normalize (d)ates** — the same match-anything parsing, condensed to
+  the UTC calendar date: the column becomes date-only ISO-8601
+  (`year-month-day`); unparseable values become null.
 - **Trim whitespace** — strip edges and collapse internal whitespace runs,
   per selected columns.
 - Rename a column — click its header in the preview and type the new name.
-- **Deduplicate rows** — on all or selected columns, keeping first or last.
 - **Combine columns** — concatenate two or more columns with a separator.
 - **Split column** — break one column into `{col}_1..{col}_k`; a blank
   delimiter splits on runs of whitespace.
 - **Remove columns** — drop selected columns entirely.
+- **Deduplicate rows** — on all or selected columns, keeping first or last.
 
 ## Formats
 
@@ -92,30 +150,34 @@ still runs on the full data.
 Every open file keeps an internal log of the operations performed on it. When
 saving, the dialog offers a **Save sequence of operations?** checkbox
 (unticked by default); with it ticked, a second dialog asks where to write
-the script — suggested `<table>.ntd`, any other extension you type is kept.
+the script — suggested `<table>.ntd`; scripts always use the `.ntd`
+extension, so the name you type is saved with that suffix whatever it
+ends in.
 
 The script is plain ASCII text, one operation per line, e.g.:
 
 ```
 # normalize-tabular-data script
-# source: employees.csv
-# table: /data/employees-normalized.csv
+# source: worksite.csv
+# table: /data/worksite-normalized.csv
 # saved: 2026-10-05T12:30:11
-trim_collapse(columns=["Dept", "Name"])
-date_normalize(column="Hired Date")
+date_normalize(column="Signed Up")
+trim_collapse(columns=["Full Name", "Worksite", "Job Class"])
+split_column(column="Full Name", delimiter="")
+rename_single(column="Full Name_1", new_name="First Name")
+rename_single(column="Full Name_2", new_name="Last Name")
 ```
 
 ## Playing a script back
 
 Press `p` (available while a file is loaded) to pick a script file: the
 dialog previews the highlighted `.ntd` file (syntax-highlighted, first 40
-lines) before you confirm. Its operations are applied, in order, to the table you have open. If any step
-cannot be performed against the currently loaded file — a column it
-renames, trims or splits is missing, the operation is unknown — playing
-stops with an alert naming the failing step, and every step the script had
-already applied is rolled back, so the table is left exactly as it was.
-
-![TUI preview](https://raw.githubusercontent.com/SEIU-Tech/normalize-tabular-data/main/docs/screenshot.png)
+lines) before you confirm. Its operations are applied, in order, to the table 
+you have open. If any step cannot be performed against the currently loaded 
+file — a column it renames, trims or splits is missing, the operation is 
+unknown — playing stops with an alert naming the failing step, and every step
+the script had already applied is rolled back, so the table is left exactly as 
+it was.
 
 ## Publishing
 

{normalize_tabular_data-0.1.1 → normalize_tabular_data-0.1.3}/README.md

@@ -10,6 +10,8 @@ Load a file, see a preview, build up a pipeline of normalization operations
 combine/split columns, drop columns), then save the
 cleaned result.
 
+![TUI preview](https://raw.githubusercontent.com/SEIU-Tech/normalize-tabular-data/main/docs/screenshot.png)
+
 ## Running
 
 ### Persistent install
@@ -27,15 +29,64 @@ uv tool upgrade normalize-tabular-data
 ```bash
 uvx normalize-tabular-data
 # pin a specific version:
-uvx normalize-tabular-data==0.1.0
+uvx normalize-tabular-data==0.1.1
 ```
 
-### Straight from a checkout
+### From a git checkout
+
+Within the directory of the cloned repository:
 
 ```bash
 uv run normalize-tabular-data
 ```
 
+## Command line
+
+An optional `FILE` argument names a table to open at startup, as if you
+had picked it from the in-app dialog.
+
+```bash
+normalize-tabular-data data/members.csv      # or with uvx/uv run
+uvx normalize-tabular-data data/members.tsv
+```
+
+If the file is missing or its format cannot be read, the TUI still
+starts — it shows an alert toast and you can open something else.
+
+### Open with a script
+
+With `--script/-s`, an `.ntd` script is played on `FILE` as soon as it
+loads (same flow as the `p` key, same stop-and-rollback on a step that
+does not fit the file):
+
+```bash
+uvx normalize-tabular-data data/members.csv -s data/members.ntd
+```
+
+### Headless run (`-x`, `--run`)
+
+`--run/-x` skips the TUI: it opens `FILE`, plays `SCRIPT` step by step,
+saves the result and exits. Each operation prints to stdout as it is
+applied; anything else — including a failing step, named and rolled
+back exactly as the interactive player would — goes to stderr with a
+nonzero exit code.
+
+```bash
+# saves data/members-normalized.csv next to the original
+uvx normalize-tabular-data members.csv -s members.ntd -x
+
+# choose the destination (format taken from its extension):
+uvx normalize-tabular-data members.csv -s members.ntd -x -o out/members.parquet
+
+# write the CSV to a pipe instead:
+uvx normalize-tabular-data members.csv -s members.ntd -x -o - | gzip > clean.csv.gz
+```
+
+With `-o -` (attached `-o-` works too), the transformed table is written
+to stdout itself and the step lines are suppressed, so stdout carries
+only the data. After a successful run the confirmation line goes to
+stderr.
+
 ## Usage
 
 Launch with `normalize-tabular-data`. Keys:
@@ -52,16 +103,22 @@ Launch with `normalize-tabular-data`. Keys:
 
 ## Operations
 
-- **Normalize dates** — parse a messy date column of *any* input format into
-  canonical UTC datetimes; unparseable values become null.
+- **Normalize date(t)imes** — parse a messy date column of *any* input
+  format into canonical UTC datetimes at millisecond resolution;
+  unparseable values become null. CSV/TSV/JSONLines export serializes
+  datetime columns as second-resolution `year-month-dayThh:mm:ss`
+  strings; Parquet and Excel keep the real datetime values.
+- **Normalize (d)ates** — the same match-anything parsing, condensed to
+  the UTC calendar date: the column becomes date-only ISO-8601
+  (`year-month-day`); unparseable values become null.
 - **Trim whitespace** — strip edges and collapse internal whitespace runs,
   per selected columns.
 - Rename a column — click its header in the preview and type the new name.
-- **Deduplicate rows** — on all or selected columns, keeping first or last.
 - **Combine columns** — concatenate two or more columns with a separator.
 - **Split column** — break one column into `{col}_1..{col}_k`; a blank
   delimiter splits on runs of whitespace.
 - **Remove columns** — drop selected columns entirely.
+- **Deduplicate rows** — on all or selected columns, keeping first or last.
 
 ## Formats
 
@@ -76,30 +133,34 @@ still runs on the full data.
 Every open file keeps an internal log of the operations performed on it. When
 saving, the dialog offers a **Save sequence of operations?** checkbox
 (unticked by default); with it ticked, a second dialog asks where to write
-the script — suggested `<table>.ntd`, any other extension you type is kept.
+the script — suggested `<table>.ntd`; scripts always use the `.ntd`
+extension, so the name you type is saved with that suffix whatever it
+ends in.
 
 The script is plain ASCII text, one operation per line, e.g.:
 
 ```
 # normalize-tabular-data script
-# source: employees.csv
-# table: /data/employees-normalized.csv
+# source: worksite.csv
+# table: /data/worksite-normalized.csv
 # saved: 2026-10-05T12:30:11
-trim_collapse(columns=["Dept", "Name"])
-date_normalize(column="Hired Date")
+date_normalize(column="Signed Up")
+trim_collapse(columns=["Full Name", "Worksite", "Job Class"])
+split_column(column="Full Name", delimiter="")
+rename_single(column="Full Name_1", new_name="First Name")
+rename_single(column="Full Name_2", new_name="Last Name")
 ```
 
 ## Playing a script back
 
 Press `p` (available while a file is loaded) to pick a script file: the
 dialog previews the highlighted `.ntd` file (syntax-highlighted, first 40
-lines) before you confirm. Its operations are applied, in order, to the table you have open. If any step
-cannot be performed against the currently loaded file — a column it
-renames, trims or splits is missing, the operation is unknown — playing
-stops with an alert naming the failing step, and every step the script had
-already applied is rolled back, so the table is left exactly as it was.
-
-![TUI preview](https://raw.githubusercontent.com/SEIU-Tech/normalize-tabular-data/main/docs/screenshot.png)
+lines) before you confirm. Its operations are applied, in order, to the table 
+you have open. If any step cannot be performed against the currently loaded 
+file — a column it renames, trims or splits is missing, the operation is 
+unknown — playing stops with an alert naming the failing step, and every step
+the script had already applied is rolled back, so the table is left exactly as 
+it was.
 
 ## Publishing
 

{normalize_tabular_data-0.1.1 → normalize_tabular_data-0.1.3}/pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "normalize-tabular-data"
-version = "0.1.1"
-description = "TUI for normalizing tabular data with polars"
+version = "0.1.3"
+description = "TUI for normalizing tabular data with Polars"
 readme = "README.md"
 license = "BSD-2-Clause"
 license-files = ["LICENSE"]
@@ -12,6 +12,7 @@ dependencies = [
     "polars>=1.44,<2",
     "fastexcel>=0.10",
     "xlsxwriter>=0.9",
+    "platformdirs>=3.0",
 ]
 
 [[project.authors]]
@@ -27,6 +28,7 @@ dev = [
     "pytest-asyncio>=0.25",
     "ruff>=0.16.10",
     "twine>=7.0.0",
+    "ty>=0.0.84",
 ]
 
 [build-system]

{normalize_tabular_data-0.1.1 → normalize_tabular_data-0.1.3}/pyproject.toml.orig

@@ -1,7 +1,7 @@
 [project]
 name = "normalize-tabular-data"
-version = "0.1.1"
-description = "TUI for normalizing tabular data with polars"
+version = "0.1.3"
+description = "TUI for normalizing tabular data with Polars"
 readme = "README.md"
 license = "BSD-2-Clause"
 license-files = ["LICENSE"]
@@ -15,6 +15,7 @@ dependencies = [
     "polars>=1.44,<2",
     "fastexcel>=0.10",
     "xlsxwriter>=0.9",
+    "platformdirs>=3.0",
 ]
 
 [dependency-groups]
@@ -23,6 +24,7 @@ dev = [
     "pytest-asyncio>=0.25",
     "ruff>=0.16.10",
     "twine>=7.0.0",
+    "ty>=0.0.84",
 ]
 
 [project.scripts]

normalize_tabular_data-0.1.3/src/normalize_tabular_data/__init__.py

@@ -0,0 +1,152 @@
+"""normalize-tabular-data: TUI for normalizing tabular data with polars."""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+
+__version__ = "0.1.3"
+
+
+def _run_script(path: Path, script_path: Path, output: Path | None) -> int:
+    """`--run` mode: play SCRIPT against FILE and save the result, no TUI.
+
+    Operation steps print to STDOUT as they are applied (suppressed when
+    STDOUT itself is the output, so the only thing there is the table);
+    every other message, including any step failure, goes to STDERR and
+    exits nonzero. A failed script leaves nothing written."""
+    from normalize_tabular_data import io
+    from normalize_tabular_data.ops import Pipeline, apply_script_steps
+
+    try:
+        steps = io.read_script(script_path)
+    except Exception as exc:
+        print(f"Could not read {script_path.name}: {exc}", file=sys.stderr)
+        return 1
+    if not steps:
+        print(f"{script_path.name} contains no operations", file=sys.stderr)
+        return 1
+    try:
+        fmt = io.detect_format(path)
+        table = io.read_table(path, fmt)
+    except Exception as exc:
+        print(f"Could not read {path.name}: {exc}", file=sys.stderr)
+        return 1
+
+    pipeline = Pipeline(source=table)
+    op_log: list[tuple[str, dict]] = []
+    to_stdout = output is not None and str(output) == "-"
+    if not to_stdout:
+        for key, params in steps:
+            print(io.step_line(key, params))
+
+    def fail(reason: str) -> None:
+        print(reason, file=sys.stderr)
+
+    if not apply_script_steps(steps, pipeline, op_log, fail):
+        return 1
+    # the transformed frame lives in the pipeline; the `table` variable
+    # still holds the raw source
+    table = pipeline.current()
+
+    out_path = (
+        None if to_stdout else (output or path.with_stem(path.stem + "-normalized"))
+    )
+    try:
+        if out_path is None:
+            if fmt not in ("csv", "tsv", "jsonl"):
+                print(
+                    f"{fmt!r} is not a text format: it cannot be written to "
+                    "STDOUT. Name an --output file with a csv/tsv/jsonl "
+                    "extension instead.",
+                    file=sys.stderr,
+                )
+                return 1
+            if fmt == "csv":
+                table.write_csv(sys.stdout)
+            elif fmt == "tsv":
+                table.write_csv(sys.stdout, separator="\t")
+            else:
+                table.write_ndjson(sys.stdout)
+        else:
+            out_fmt = io.detect_format(out_path)
+            if out_path.suffix.lower() == ".xls":
+                print(
+                    ".xls cannot be written; use --output <name>.xlsx", file=sys.stderr
+                )
+                return 1
+            io.write_table(table, out_path, out_fmt)
+    except Exception as exc:
+        print(f"Save failed: {exc}", file=sys.stderr)
+        return 1
+
+    count = len(steps)
+    landing = "STDOUT" if out_path is None else str(out_path)
+    print(
+        f"Applied {count} step{'s' if count != 1 else ''} from "
+        f"{script_path.name} -> {landing}",
+        file=sys.stderr,
+    )
+    return 0
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        prog="normalize-tabular-data",
+        description="A terminal UI for normalizing tabular data with polars.",
+    )
+    parser.add_argument(
+        "file",
+        nargs="?",
+        type=Path,
+        metavar="FILE",
+        help="table file to open at startup (csv, tsv, jsonl, parquet, xlsx)",
+    )
+    parser.add_argument(
+        "-s",
+        "--script",
+        type=Path,
+        metavar="SCRIPT",
+        help="an .ntd operation script to play on FILE after opening it",
+    )
+    parser.add_argument(
+        "-o",
+        "--output",
+        type=Path,
+        metavar="PATH",
+        help="where --run saves the transformed table: a path whose format "
+        "is its extension, or - for STDOUT (default: the FILE stem plus "
+        "-normalized, same directory and extension)",
+    )
+    parser.add_argument(
+        "-x",
+        "--run",
+        action="store_true",
+        help="open FILE, play SCRIPT step by step, save the result and "
+        "exit (no UI; requires FILE and --script)",
+    )
+    argv = sys.argv[1:]
+    args = parser.parse_args(argv)
+
+    if args.script is not None and args.file is None:
+        parser.error("--script needs a FILE to open")
+    if args.output is not None and not args.run:
+        parser.error("--output only applies with --run")
+    if args.run:
+        if args.file is None:
+            parser.error("--run needs a FILE to open")
+        if args.script is None:
+            parser.error("--run needs --script (there would be nothing to play)")
+        raise SystemExit(_run_script(args.file, args.script, args.output))
+
+    try:
+        from normalize_tabular_data.app import NormalizeApp
+    except ImportError as exc:  # helpful message if an optional engine is missing
+        print(
+            f"normalize-tabular-data is missing a dependency ({exc}).\n"
+            "Reinstall with: uv tool install --force --reinstall normalize-tabular-data",
+            file=sys.stderr,
+        )
+        raise SystemExit(1)
+    NormalizeApp(initial_file=args.file, initial_script=args.script).run()

{normalize_tabular_data-0.1.1 → normalize_tabular_data-0.1.3}/src/normalize_tabular_data/app.py

@@ -6,6 +6,7 @@ import bisect
 import datetime as _dt
 import random
 from pathlib import Path
+from platformdirs import user_config_dir
 from typing import Any, Iterable
 
 from textual import on
@@ -28,6 +29,7 @@ from normalize_tabular_data.ops import (
     Operation,
     Pipeline,
     analyze_column,
+    apply_script_steps as ops_apply_script_steps,
 )
 from normalize_tabular_data.screens import (
     ModalDialog,
@@ -42,6 +44,7 @@ from normalize_tabular_data.screens import (
 from normalize_tabular_data.widgets import ColumnSidebar, MenuFooter, StepsBar
 
 PREVIEW_ROWS = 250
+APP_CONFIG_NAME = "normalize-tabular-data"
 
 
 class CommandMenu(CommandPalette):
@@ -134,10 +137,19 @@ class OpChooserModal(ModalDialog):
 
     def _marked_title(self, op_key: str, title: str, hotkey: str = "") -> str:
         """Return "Normalize (d)ates"-style markup: the op's hotkey letter
-        highlighted where it sits in the title (keeping its case). Ops
-        without a designated hotkey claim the first unused letter in the
-        title instead; plain title if no letter can be claimed."""
+        highlighted where it sits in the title (keeping its case). A title
+        that already parenthesizes its letter ("Normalize date(t)imes")
+        highlights the letter between the parens; a plain title matches
+        its first occurrence and adds the parens. Ops without a designated
+        hotkey claim the first unused letter in the title instead; plain
+        title if no letter can be claimed."""
         if hotkey:
+            i = title.lower().find(f"({hotkey.lower()})")
+            if i >= 0:
+                # the letter to claim sits inside existing parentheses:
+                # highlight it without adding a second pair
+                self._hotkeys[title[i + 1].lower()] = op_key
+                return f"{title[: i + 1]}[cyan]{title[i + 1]}[/cyan]{title[i + 2:]}"
             i = title.lower().find(hotkey.lower())
         else:
             i = next(
@@ -270,8 +282,29 @@ class NormalizeApp(App[None]):
     }
     """
 
-    def __init__(self) -> None:
+    def __init__(
+        self,
+        initial_file: str | Path | None = None,
+        initial_script: str | Path | None = None,
+        config_dir: Path | None = None,
+    ) -> None:
         super().__init__()
+        # where the chosen theme (and any future settings) are stored;
+        # None means the per-user platform directory
+        self._config_dir = config_dir
+        # flips to True at on_mount: only theme changes made after are the
+        # user's and only those are persisted
+        self._app_ready = False
+        # filesystem path to open when the app starts (the optional file
+        # named on the command line); loaded like a file chosen in-app
+        self.initial_file: Path | None = (
+            Path(initial_file) if initial_file is not None else None
+        )
+        # .ntd script played automatically once that file is loaded (the
+        # optional --script named on the command line); requires a file
+        self.initial_script: Path | None = (
+            Path(initial_script) if initial_script is not None else None
+        )
         self.path: Path | None = None
         self.pipeline: Pipeline | None = None
         self.analyzed: dict[str, Any] = {}
@@ -339,8 +372,56 @@ class NormalizeApp(App[None]):
         return list(self.pipeline.current().columns)
 
     def on_mount(self) -> None:
+        # remember this point so theme changes from here on are the
+        # user's own choices and get persisted
+        self._apply_saved_theme()
         self.push_screen(MainScreen())
         self._refresh_steps()
+        # the command line's file, if any, is opened once the main screen
+        # is composed; load_path toasts a readable alert instead of
+        # crashing if it is unreadable or has an unsupported format
+        if self.initial_file is not None:
+            self.call_after_refresh(self.load_path, self.initial_file)
+        self._app_ready = True
+
+    # --- theme persistence ---------------------------------------------
+
+    def watch_theme(self, theme_name: str) -> None:
+        """Persist every theme the user chooses (menu or command), so the
+        next launch opens with it again. Skipped while headless, so the
+        pilot tests cannot touch the real configuration; skipped before
+        mount, where only the reactive's default applies."""
+        if self._app_ready and not self.is_headless:
+            try:
+                self._theme_config_path().parent.mkdir(parents=True, exist_ok=True)
+                self._theme_config_path().write_text(
+                    theme_name + "\n", encoding="utf-8"
+                )
+            except OSError:
+                pass  # unwritable config location: run without remembering
+
+    def _apply_saved_theme(self) -> None:
+        """Restore the theme chosen in a previous session, if it still exists.
+
+        Skipped in headless runs that use the real configuration location
+        (pilot tests): an eyeballing developer's own saved theme should
+        not leak into supposedly default-themed tests. A headless run
+        given an explicit config_dir is a deliberate fixture and loads it."""
+        if self.is_headless and self._config_dir is None:
+            return
+        try:
+            saved = self._theme_config_path().read_text(encoding="utf-8").strip()
+        except OSError:
+            return
+        if not saved:
+            return
+        try:
+            self.theme = saved
+        except Exception:
+            pass  # a theme name that no longer exists: keep the default
+
+    def _theme_config_path(self) -> Path:
+        return (self._config_dir or Path(user_config_dir(APP_CONFIG_NAME))) / "theme"
 
     # --- data plumbing ---------------------------------------------------
 
@@ -386,6 +467,11 @@ class NormalizeApp(App[None]):
         # pipeline is now set: re-evaluate the footer's enabled keys
         self.screen.refresh_bindings()
         self.notify(f"Loaded {path.name} ({df.height:,} rows x {df.width} columns)")
+        # --script named on the command line: play it as soon as its file
+        # is on screen (same flow as the P key, with the same rollback)
+        if self.initial_script is not None:
+            self.play_script_file(self.initial_script)
+            self.initial_script = None
 
     def analyze_columns(self) -> None:
         if self.pipeline is None:
@@ -499,50 +585,27 @@ class NormalizeApp(App[None]):
             return
         started_applied = len(self.pipeline.applied)
         started_log = len(self.op_log)
-        for index, (key, params) in enumerate(steps, 1):
-            op = PLAY_REGISTRY.get(key)
-            if op is None:
-                self._rollback_script(
-                    index,
-                    started_applied,
-                    started_log,
-                    f"Step {index}: {key!r} is not a known operation",
-                )
-                return
-            try:
-                self.pipeline.apply(op, params)
-                # the pipeline refolds lazily; fold now so an op-specific
-                # error (e.g. unparseable dates) rolls back this step too
-                self.pipeline.current()
-            except Exception as exc:
-                self._rollback_script(
-                    index,
-                    started_applied,
-                    started_log,
-                    f"Step {index} of {len(steps)} ({op.title}) failed: {exc}",
-                )
-                return
-            self.op_log.append((key, dict(params)))
+
+        def alert(reason: str) -> None:
+            # same alert the player has always shown: step description plus
+            # how much of the script had to be rolled back
+            undid = len(self.pipeline.applied) - started_applied
+            detail = (
+                f" — rolled back {undid} step{'s' if undid != 1 else ''}"
+                if undid
+                else ""
+            )
+            self.refresh_all()
+            self.notify(f"{reason}{detail}", severity="error")
+
+        if not ops_apply_script_steps(steps, self.pipeline, self.op_log, alert):
+            return
         self.refresh_all()
         count = len(steps)
         self.notify(
             f"Applied {count} step{'s' if count != 1 else ''} from {script_path.name}"
         )
 
-    def _rollback_script(
-        self, step: int, started_applied: int, started_log: int, reason: str
-    ) -> None:
-        """Undo everything a partially played script applied, then alert."""
-        undid = len(self.pipeline.applied) - started_applied
-        del self.pipeline.applied[started_applied:]
-        self.pipeline.redo_stack.clear()
-        del self.op_log[started_log:]
-        self.refresh_all()
-        detail = (
-            f" — rolled back {undid} step{'s' if undid != 1 else ''}" if undid else ""
-        )
-        self.notify(f"{reason}{detail}", severity="error")
-
     def _apply_now(self, op: Operation, params: dict[str, Any]) -> None:
         try:
             self.pipeline.apply(op, params)

{normalize_tabular_data-0.1.1 → normalize_tabular_data-0.1.3}/src/normalize_tabular_data/io.py

@@ -79,6 +79,24 @@ def excel_sheets(path: Path) -> list[str]:
     return list(fastexcel.read_excel(path).sheet_names)
 
 
+def _second_resolution(df: pl.DataFrame) -> pl.DataFrame:
+    """Datetime columns serialized as canonical UTC second-resolution text.
+
+    Used only for the text formats (CSV/TSV/JSONL): polars pads datetime
+    output with sub-second digits no matter the unit, and the canonical
+    text form carries no fractional part. Parquet and Excel keep the real
+    Datetime values (millisecond resolution)."""
+    datetime_cols = [
+        c for c, dtype in df.schema.items() if isinstance(dtype, pl.Datetime)
+    ]
+    if not datetime_cols:
+        return df
+    return df.with_columns(
+        pl.col(c).dt.truncate("1s").dt.to_string("%Y-%m-%dT%H:%M:%S")
+        for c in datetime_cols
+    )
+
+
 def write_table(
     df: pl.DataFrame,
     path: Path,
@@ -86,11 +104,11 @@ def write_table(
     sheet_name: str = "data",
 ) -> None:
     if fmt == "csv":
-        df.write_csv(path)
+        _second_resolution(df).write_csv(path)
     elif fmt == "tsv":
-        df.write_csv(path, separator="\t")
+        _second_resolution(df).write_csv(path, separator="\t")
     elif fmt == "jsonl":
-        df.write_ndjson(path)
+        _second_resolution(df).write_ndjson(path)
     elif fmt == "parquet":
         df.write_parquet(path)
     elif fmt == "xlsx":
@@ -104,21 +122,21 @@ def write_table(
 SCRIPT_SUFFIX = ".ntd"
 
 
+def step_line(key: str, params: dict) -> str:
+    """One operation description: `<key>(<param>=<json value>, ...)`."""
+    return f"{key}({', '.join(f'{p}={json.dumps(v)}' for p, v in params.items())})"
+
+
 def script_text(steps: list[tuple[str, dict]], header_fields: dict[str, str]) -> str:
     """Human-readable ASCII text, one operation description per line.
 
-    Every operation is `<key>(<param>=<json value>, ...)` so the lines stay
-    readable while remaining mechanically parseable for a future
-    "apply a script" capability. `header_fields` become leading `#`
+    Every operation is `step_line()`'s `<key>(<param>=<json value>, ...)`
+    so the lines stay readable while remaining mechanically parseable for a
+    future "apply a script" capability. `header_fields` become leading `#`
     comment lines (source file, save time, ...)"""
     lines = ["# normalize-tabular-data script"]
     lines += [f"# {text}" for text in header_fields.values() if text]
-    lines += [
-        f"{key}("
-        + ", ".join(f"{param}={json.dumps(value)}" for param, value in params.items())
-        + ")"
-        for key, params in steps
-    ]
+    lines += [step_line(key, params) for key, params in steps]
     return "\n".join(lines) + "\n"
 
 

{normalize_tabular_data-0.1.1 → normalize_tabular_data-0.1.3}/src/normalize_tabular_data/ops.py

@@ -48,8 +48,20 @@ def _apply_date_normalize(df: pl.DataFrame, p: dict[str, Any]) -> pl.DataFrame:
     series = df.get_column(col)
     if series.dtype != pl.String:
         series = series.cast(pl.String)
-    # Datetime("ns"), unparseable -> null
-    return df.with_columns(date_parser.parse_series(series).alias(col))
+    # canonical UTC Datetime at millisecond resolution; unparseable -> null
+    return df.with_columns(
+        date_parser.parse_series(series).dt.cast_time_unit("ms").alias(col)
+    )
+
+
+def _apply_date_only(df: pl.DataFrame, p: dict[str, Any]) -> pl.DataFrame:
+    col: str = p["column"]
+    series = df.get_column(col)
+    if series.dtype != pl.String:
+        series = series.cast(pl.String)
+    # date-only ISO-8601: any input format condensed to the UTC calendar
+    # date; unparseable values become null
+    return df.with_columns(date_parser.parse_series(series).dt.date().alias(col))
 
 
 def _apply_trim_collapse(df: pl.DataFrame, p: dict[str, Any]) -> pl.DataFrame:
@@ -117,8 +129,8 @@ def _apply_split_column(df: pl.DataFrame, p: dict[str, Any]) -> pl.DataFrame:
 OPS: tuple[Operation, ...] = (
     Operation(
         key="date_normalize",
-        title="Normalize dates",
-        hotkey="d",
+        title="Normalize date(t)imes",
+        hotkey="t",
         params=(
             ParamSpec(
                 "column",
@@ -129,6 +141,20 @@ OPS: tuple[Operation, ...] = (
         ),
         apply=_apply_date_normalize,
     ),
+    Operation(
+        key="date_only",
+        title="Normalize (d)ates",
+        hotkey="d",
+        params=(
+            ParamSpec(
+                "column",
+                "column",
+                "Date column",
+                help="Any input format; condenses to the UTC calendar date",
+            ),
+        ),
+        apply=_apply_date_only,
+    ),
     Operation(
         key="trim_collapse",
         title="Trim whitespace",
@@ -143,22 +169,6 @@ OPS: tuple[Operation, ...] = (
         ),
         apply=_apply_trim_collapse,
     ),
-    Operation(
-        key="dedup_rows",
-        title="Deduplicate rows",
-        hotkey="p",
-        params=(
-            ParamSpec("columns", "column_multi", "Key columns (empty = all)"),
-            ParamSpec(
-                "keep",
-                "choice",
-                "Keep",
-                default="first",
-                choices=("first", "last"),
-            ),
-        ),
-        apply=_apply_dedup_rows,
-    ),
     Operation(
         key="combine_columns",
         title="Combine columns",
@@ -203,6 +213,22 @@ OPS: tuple[Operation, ...] = (
         ),
         apply=_apply_drop_columns,
     ),
+    Operation(
+        key="dedup_rows",
+        title="Deduplicate rows",
+        hotkey="p",
+        params=(
+            ParamSpec("columns", "column_multi", "Key columns (empty = all)"),
+            ParamSpec(
+                "keep",
+                "choice",
+                "Keep",
+                default="first",
+                choices=("first", "last"),
+            ),
+        ),
+        apply=_apply_dedup_rows,
+    ),
 )
 
 OP_REGISTRY: dict[str, Operation] = {op.key: op for op in OPS}
@@ -224,6 +250,47 @@ RENAME_OP = Operation(
 PLAY_REGISTRY: dict[str, Operation] = {**OP_REGISTRY, RENAME_OP.key: RENAME_OP}
 
 
+def apply_script_steps(
+    steps: list[tuple[str, dict]],
+    pipeline: Pipeline,
+    op_log: list[tuple[str, Any]],
+    on_failure: Callable[[str], None],
+) -> bool:
+    """Apply a played script's steps to `pipeline`, all-or-nothing.
+
+    Each step is resolved against PLAY_REGISTRY, applied, and logged to
+    `op_log` (the same keys the interactive player records). On the first
+    step that is impossible for the currently loaded file the work done so
+    far is rolled back — `pipeline.applied` and `op_log` truncated to where
+    the script started and the redo stack cleared — and the failure reason
+    is passed to `on_failure`. Returns False when a step failed, in which
+    case `pipeline` is left exactly as it was before the call."""
+    started_applied = len(pipeline.applied)
+    started_log = len(op_log)
+
+    def give_up(reason: str) -> None:
+        del pipeline.applied[started_applied:]
+        pipeline.redo_stack.clear()
+        del op_log[started_log:]
+        on_failure(reason)
+
+    for index, (key, params) in enumerate(steps, 1):
+        op = PLAY_REGISTRY.get(key)
+        if op is None:
+            give_up(f"Step {index}: {key!r} is not a known operation")
+            return False
+        try:
+            pipeline.apply(op, params)
+            # the pipeline refolds lazily; fold now so an op-specific error
+            # (e.g. unparseable dates) rolls back this step too
+            pipeline.current()
+        except Exception as exc:
+            give_up(f"Step {index} of {len(steps)} ({op.title}) failed: {exc}")
+            return False
+        op_log.append((key, dict(params)))
+    return True
+
+
 # --- pipeline ---------------------------------------------------------------
 
 

{normalize_tabular_data-0.1.1 → normalize_tabular_data-0.1.3}/src/normalize_tabular_data/screens.py

@@ -363,7 +363,11 @@ class OpParamsModal(ModalDialog):
 
     def compose_body(self) -> ComposeResult:
         self.widgets: dict[str, object] = {}
-        candidates = self._date_candidates() if self.op.key == "date_normalize" else []
+        candidates = (
+            self._date_candidates()
+            if self.op.key in ("date_normalize", "date_only")
+            else []
+        )
         for spec in self.op.params:
             if spec.kind == "column_multi":
                 sel = SelectionList(
@@ -531,8 +535,8 @@ class SaveModal(ModalDialog):
 class ScriptNameModal(ModalDialog):
     """Name the '.ntd' script file when saving a sequence of operations.
 
-    The default carries the .ntd extension; typing a different one saves
-    under that extension instead."""
+    The extension is always .ntd: whatever the user types, the saved name
+    ends in the fixed script suffix."""
 
     dialog_title = "Save script"
 
@@ -547,17 +551,14 @@ class ScriptNameModal(ModalDialog):
             id="script_input",
         )
         yield self.path_input
-        yield Static(
-            f"Default extension: {SCRIPT_SUFFIX} — type another to override",
-            classes="help",
-        )
+        yield Static(f"Scripts always end in {SCRIPT_SUFFIX}", classes="help")
 
     def action_ok(self) -> None:
         raw = self.path_input.value.strip()
         if not raw:
             self.app.notify("Type a script file name or path", severity="error")
             return
-        target = Path(raw).expanduser()
+        target = Path(raw).expanduser().with_suffix(SCRIPT_SUFFIX)
         if target.exists():
             self.app.notify("Script file exists: pick another name", severity="warning")
             return

{normalize_tabular_data-0.1.1 → normalize_tabular_data-0.1.3}/src/normalize_tabular_data/widgets.py

@@ -92,13 +92,13 @@ class ColumnSidebar(Widget):
 
 
 def _dtype_mark(dtype: str) -> str:
-    """Compact 3-wide dtype mark for the sidebar: dt (datetime/date),
-    str (string), int (integer), dec (float/decimal), t/f (boolean);
-    everything else oth. Each mark is padded to 3 columns so column
-    names left-align."""
+    """Compact 3-wide dtype mark for the sidebar: dt (datetime),
+    day (date-only), str (string), int (integer), dec (float/decimal),
+    t/f (boolean); everything else oth. Each mark is padded to 3 columns
+    so column names left-align."""
     for prefix, mark in (
         ("Datetime", "dt "),
-        ("Date", "dt "),
+        ("Date", "day"),
         ("Time", "dt "),
         ("String", "str"),
         ("Categorical", "str"),

normalize_tabular_data-0.1.1/src/normalize_tabular_data/__init__.py

@@ -1,18 +0,0 @@
-"""normalize-tabular-data: TUI for normalizing tabular data with polars."""
-
-__version__ = "0.1.0"
-
-
-def main() -> None:
-    try:
-        from normalize_tabular_data.app import NormalizeApp
-    except ImportError as exc:  # helpful message if an optional engine is missing
-        import sys
-
-        print(
-            f"normalize-tabular-data is missing a dependency ({exc}).\n"
-            "Reinstall with: uv tool install --force --reinstall normalize-tabular-data",
-            file=sys.stderr,
-        )
-        raise SystemExit(1)
-    NormalizeApp().run()