jw-automator 4.0.1 → 6.0.0

package/docs/11a-macro-fix.md

@@ -0,0 +1,56 @@
+# Proposed Solution: Consistent Cascading Catch-up Macros
+
+This document outlines a plan to address inconsistencies and enhance the cascading behavior of `catchUpMode` macros within the `jw-automator` library.
+
+## Current State Analysis
+
+### Strengths
+*   **Existing Cascading Logic:** A basic cascading system is present where a task's `catchUpMode` can override the `Automator` instance's `defaultCatchUpMode`.
+*   **Pure Core Engine:** The `CoreEngine` correctly consumes only normalized `catchUpWindow` and `catchUpLimit` values, remaining isolated from macro interpretation.
+*   **Normalization-then-Deletion:** The `catchUpMode` is processed into concrete `catchUpWindow`/`catchUpLimit` values and then deleted from the task object before persistence. This ensures that the persisted state reflects the "ground truth" and prevents unintended changes if macro definitions evolve.
+
+### Weaknesses / Inconsistencies
+*   **Inconsistent Update Behavior:** The `updateTaskByID` and `updateTaskByName` methods do not fully apply the cascading logic. They only process `catchUpMode` if it's explicitly provided in the `updates` object, failing to fall back to the `Automator`'s `defaultCatchUpMode` if the task being updated previously used the default. This can lead to tasks losing their intended catch-up behavior upon update.
+*   **Limited Macro Support:** The current implementation only supports `'default'` and `'realtime'` `catchUpMode` values. The `'auto'` mode, which was discussed as calculating values based on recurrence intervals, is not yet implemented.
+
+## Proposed Solution
+
+The goal is to reinforce and consistently apply the existing "one-way normalization" philosophy across all task management operations (`addTask`, `updateTaskByID`, `updateTaskByName`), while also extending its capabilities.
+
+### 1. Centralize Normalization Logic
+
+*   **Action:** Create a new private helper method in `src/Automator.js`, e.g., `_normalizeCatchUpSettings(taskSpec, existingTask)`.
+*   **Purpose:** This method will be responsible for resolving the final `catchUpWindow` and `catchUpLimit` values based on a clear cascading order:
+    1.  **Explicit values on input:** `taskSpec.catchUpWindow` or `taskSpec.catchUpLimit` take highest precedence.
+    2.  **Input `catchUpMode`:** If `taskSpec.catchUpMode` is present, it will be used to derive values.
+    3.  **Existing task values:** If no `catchUpWindow`, `catchUpLimit`, or `catchUpMode` is provided in `taskSpec`, the values from `existingTask` (if provided for an update operation) will be retained.
+    4.  **Automator `defaultCatchUpMode`:** If none of the above are present, the `this.options.defaultCatchUpMode` will be used to derive the values.
+    5.  **Hardcoded System Default:** If all else fails, a sensible default (e.g., `realtime` equivalent) will be used.
+*   **Macro Deletion:** This helper will ensure `taskSpec.catchUpMode` is always deleted after normalization.
+
+### 2. Refactor `addTask`
+
+*   **Action:** Modify `addTask` to call `_normalizeCatchUpSettings` to process the incoming `taskSpec` before assigning task properties.
+*   **Benefit:** Simplifies `addTask` and ensures consistent application of the normalization rules.
+
+### 3. Refactor `updateTaskByID` and `updateTaskByName`
+
+*   **Action:**
+    *   For both methods, first retrieve the `existingTask` before applying any updates.
+    *   Pass the `updates` object (which acts as the `taskSpec` for normalization) and the `existingTask` to `_normalizeCatchUpSettings`.
+    *   Apply the resulting `catchUpWindow` and `catchUpLimit` to the task being updated.
+*   **Benefit:** Resolves the current inconsistency. Tasks will now correctly retain their existing `catchUp` settings during an update if no new `catchUpMode` or explicit values are provided, or correctly apply the `Automator`'s `defaultCatchUpMode` if applicable.
+
+### 4. Implement `'auto'` Macro (Recommended Enhancement)
+
+*   **Action:** Integrate the logic for an `'auto'` `catchUpMode` within the `_normalizeCatchUpSettings` helper.
+*   **Logic:** When `catchUpMode` is `'auto'`, calculate `catchUpWindow` and `catchUpLimit` dynamically, potentially based on the task's `repeat.interval` (e.g., `catchUpWindow = 25% of interval`, `catchUpLimit = 1`).
+*   **Benefit:** Aligns the implementation with the ideas in `dev/11-catchup_macros.md`, providing a more intelligent default based on task recurrence.
+
+## Summary of Benefits
+
+This plan will:
+*   Ensure **consistent and predictable** catch-up behavior across all task creation and modification operations.
+*   Maintain the project's philosophy of **pure core engine** and **persisted ground truth**.
+*   Improve **developer ergonomics** by centralizing complex logic.
+*   Provide a clear path for **future macro enhancements** like the `'auto'` mode.

package/docs/ARCHITECTURE.md

@@ -12,7 +12,7 @@ This document describes the internal architecture of jw-automator v3.
 2. **Local Time First**: Human-centric time semantics, not UTC-based
 3. **Deterministic Core**: Pure step function enables testing and simulation
 4. **Resilience**: Survive offline gaps, DST transitions, and event loop stalls
-5. **Separation of Concerns**: Spec vs. state, core vs. host, storage abstraction
+5. **Separation of Concerns**: Spec vs. state, core vs. host, persistence strategy
 
 ---
 
@@ -23,38 +23,38 @@ This document describes the internal architecture of jw-automator v3.
 │           Automator (API)               │
 │  - User-facing API                      │
 │  - Event management                     │
-│  - Action CRUD operations               │
-│  - Persistence coordination             │
+│  - Task CRUD operations                 │
+│  - Moratorium-based persistence         │
 └──────────────┬──────────────────────────┘
                │
-        ┌──────┴──────┐
+               │
+               │
+        ┌──────▼──────┐
+        │SchedulerHost│
         │             │
-┌───────▼──────┐  ┌──▼──────────────┐
-│ SchedulerHost│  │ Storage Adapter │
-│              │  │                 │
-│ - 1-sec tick │  │ - load()        │
-│ - Event emit │  │ - save()        │
-│ - Functions  │  └─────────────────┘
-└───────┬──────┘
-        │
-┌───────▼──────────────────────────┐
-│         CoreEngine               │
-│                                  │
-│  step(state, lastTick, now)      │
-│    → { newState, events }        │
-│                                  │
-│  - Deterministic                 │
-│  - Pure function                 │
-│  - Testable                      │
-└──────────┬───────────────────────┘
-           │
-    ┌──────▼────────┐
-    │RecurrenceEngine│
-    │                │
-    │ - Next occurrence calc │
-    │ - DST handling         │
-    │ - Local-time logic     │
-    └────────────────┘
+        │ - 1-sec tick│
+        │ - Event emit│
+        │ - Functions │
+        └──────┬──────┘
+               │
+┌──────────────▼──────────────────────┐
+│         CoreEngine                  │
+│                                     │
+│  step(state, lastTick, now)         │
+│    → { newState, events }           │
+│                                     │
+│  - Deterministic                    │
+│  - Pure function                    │
+│  - Testable                         │
+└──────────────┬──────────────────────┘
+               │
+        ┌──────▼────────┐
+        │RecurrenceEngine│
+        │                │
+        │ - Next occurrence calc │
+        │ - DST handling         │
+        │ - Local-time logic     │
+        └────────────────┘
 ```
 
 ---
@@ -68,7 +68,7 @@ This document describes the internal architecture of jw-automator v3.
 **Responsibilities**:
 - Process scheduling steps
 - Manage catch-up logic
-- Emit action events
+- Emit task events
 - Enforce safety invariants (max iterations, monotonic time)
 
 **Key Method**: `step(state, lastTick, now)`
@@ -119,28 +119,36 @@ setTimeout(() => tick(), wait);
 
 **Why?**
 - Prevents cumulative drift
-- Ensures all actions check against stable boundaries
+- Ensures all tasks check against stable boundaries
 - Enables deterministic simulation
 
 ---
 
-### 4. Storage Adapters
+### 4. Persistence (v5+)
 
-**Purpose**: Pluggable persistence
+**Purpose**: State persistence with minimal disk wear
 
-**Interface**:
-```javascript
-{
-  load: () => state,
-  save: (state) => void
-}
-```
+**Implementation**: Integrated directly into Automator class
 
-**Built-in Adapters**:
-- `FileStorage`: JSON file persistence
-- `MemoryStorage`: In-memory (no persistence)
+**Strategy**: Moratorium-based state machine
+- CRUD operations (add/update/remove) save immediately and start moratorium period
+- Task execution marks state dirty and saves if moratorium has expired
+- If moratorium is active, dirty state waits until moratorium ends, then saves automatically
+- Single entry point: `_requestSave(force)` with "tell vs ask" semantics
+- `saveInterval` defines the moratorium period (minimum cooling time, default: 15 seconds)
+- `stop()` forces save if dirty, ignoring any active moratorium
+- No periodic polling - one-shot timer only fires when state is dirty
 
-**Custom Adapters**: Users can provide their own (database, cloud, etc.)
+**State Machine**:
+- `stateDirty`: Boolean flag indicating unsaved changes
+- `moratoriumActive`: Boolean flag indicating cooling period is active
+- `moratoriumTimer`: One-shot setTimeout handle (not setInterval)
+
+**Files**:
+- File-based: `storageFile` option specifies JSON file path
+- Memory-only: No `storageFile` option
+
+**Custom Persistence**: Use `getTasks()` and event listeners for database/cloud storage
 
 ---
 
@@ -151,28 +159,28 @@ setTimeout(() => tick(), wait);
 **Responsibilities**:
 - Coordinate all components
 - Provide user-facing API
-- Manage action lifecycle
+- Manage task lifecycle
 - Handle auto-save
 - Event emission
 
 **Key APIs**:
-- Action management: `addAction`, `updateActionByID`, `updateActionByName`, `removeActionByID`, etc.
+- Task management: `addTask`, `updateTaskByID`, `updateTaskByName`, `removeTaskByID`, etc.
 - Function registration: `addFunction`, `removeFunction`
-- Introspection: `getActions`, `describeAction`, `getActionsInRange`
+- Introspection: `getTasks`, `describeTask`, `getTasksInRange`
 - Lifecycle: `start`, `stop`
 
 ---
 
 ## Data Flow
 
-### Adding an Action
+### Adding a Task
 
 ```
-User calls addAction()
+User calls addTask()
     ↓
 Automator validates spec
     ↓
-Creates action with ID and state
+Creates task with ID and state
     ↓
 Adds to SchedulerHost state
     ↓
@@ -188,7 +196,7 @@ SchedulerHost timer fires
     ↓
 Calls CoreEngine.step(state, lastTick, now)
     ↓
-CoreEngine processes each action:
+CoreEngine processes each task:
   - Is nextRun <= now?
   - Buffered/unbuffered logic
   - Execute or skip
@@ -199,9 +207,9 @@ Returns { newState, events }
     ↓
 SchedulerHost updates state
     ↓
-SchedulerHost executes action events:
+SchedulerHost executes task events:
   - Call registered function
-  - Emit 'action' event
+  - Emit 'task' event
     ↓
 Schedule next tick
 ```
@@ -209,13 +217,13 @@ Schedule next tick
 ### Simulation
 
 ```
-User calls getActionsInRange(start, end)
+User calls getTasksInRange(start, end)
     ↓
 CoreEngine.simulate() clones state
     ↓
 Steps through time second-by-second
     ↓
-Collects all action events
+Collects all task events
     ↓
 Returns event list (state unchanged)
 ```
@@ -246,9 +254,9 @@ Returns event list (state unchanged)
 
 ### Why `catchUpWindow` (and Legacy Buffered/UnBuffered)?
 
-The `catchUpWindow` property, supported by the CoreEngine, precisely defines the time window for recovering missed actions.
+The `catchUpWindow` property, supported by the CoreEngine, precisely defines the time window for recovering missed tasks.
 
-- **Smart Defaults**: For recurring actions, the `catchUpWindow` defaults to the action's interval (e.g., a 1-hour action has a 1-hour catch-up window). For one-time actions, it defaults to `0` (no catch-up). This prevents "thundering herd" issues by ensuring that actions too old are simply skipped.
+- **Smart Defaults**: For recurring tasks, the `catchUpWindow` defaults to the task's interval (e.g., a 1-hour task has a 1-hour catch-up window). For one-time tasks, it defaults to `0` (no catch-up). This prevents "thundering herd" issues by ensuring that tasks too old are simply skipped.
 - **Explicit Control**: Users can still set `catchUpWindow` to `0` (skip all missed), a specific millisecond value (tolerate N ms lag), or `"unlimited"` (catch up all, like old buffered behavior).
 - **Legacy `unBuffered`**: The `unBuffered` flag (`true` or `false`) is now a legacy alias for `catchUpWindow: 0` and `catchUpWindow: "unlimited"` respectively. The system transparently maps it.
 
@@ -276,7 +284,7 @@ The `catchUpWindow` property, supported by the CoreEngine, precisely defines the
 
 ### 3. State Integrity
 
-- Actions cloned before mutation
+- Tasks cloned before mutation
 - Spec vs. state separation
 - Deep copies returned from getters
 - Auto-save with configurable intervals
@@ -285,25 +293,32 @@ The `catchUpWindow` property, supported by the CoreEngine, precisely defines the
 
 ## Performance Characteristics
 
-- **Per-Tick Complexity**: O(n) where n = number of actions
+- **Per-Tick Complexity**: O(n) where n = number of tasks
 - **Recurrence Calculation**: O(1) per step
-- **Memory**: O(n) for action storage
-- **Disk I/O**: Configurable auto-save interval (default 5s)
+- **Memory**: O(n) for task storage
+- **Disk I/O**: Dirty-flag with cooling period (default: 15s minimum between saves)
 
-**Scalability**: Designed for 10-1000 actions, not 100,000s
+**Scalability**: Designed for 10-1000 tasks, not 100,000s
 
 ---
 
 ## Extension Points
 
-### Custom Storage
+### Custom Persistence
+
+Use `getTasks()` and event listeners for database/cloud storage:
 
 ```javascript
-new Automator({
-  storage: {
-    load: () => loadFromDatabase(),
-    save: (state) => saveToDatabase(state)
-  }
+const automator = new Automator(); // Memory-only
+
+automator.seed(async (auto) => {
+  const tasks = await loadFromDatabase();
+  tasks.forEach(task => auto.addTask(task));
+});
+
+automator.on('update', async () => {
+  const tasks = automator.getTasks();
+  await saveToDatabase(tasks);
 });
 ```
 
@@ -311,9 +326,9 @@ new Automator({
 
 Extend `RecurrenceEngine` with new types (requires code modification)
 
-### Meta-Actions
+### Meta-Tasks
 
-Actions can call `automator.addAction()` to create dynamic schedules
+Tasks can call `automator.addTask()` to create dynamic schedules
 
 ---
 
@@ -335,9 +350,9 @@ Potential improvements for future versions:
 - Timezone support (explicit tz parameter)
 - Cron expression compatibility layer
 - Web-based dashboard
-- Action priorities
+- Task priorities
 - Conditional execution (predicates)
-- Action dependencies (chains)
+- Task dependencies (chains)
 
 ---