@willwade/aac-processors 0.0.6 → 0.0.8

package/README.md

@@ -404,8 +404,8 @@ processor.saveFromTree(tree, "styled-board.spb");
 | **Grid3**         | ✅ Yes     | ✅ Yes       | ✅ Yes  | ✅ Style references             |
 | **Asterics Grid** | ✅ Yes     | ✅ Yes       | ✅ Yes  | ✅ Metadata-based               |
 | **Apple Panels**  | ✅ Yes     | ✅ Size only | ❌ No   | ✅ Display weight               |
-| **Dot**           |  ❌No     | ❌ Yes       | ❌ No   | ❌ Basic only                   |
-| **OPML**          |  ❌No     | ❌ Yes       | ❌ No   | ❌ Basic only                   |
+| **Dot**           | ❌No       | ❌ Yes       | ❌ No   | ❌ Basic only                   |
+| **OPML**          | ❌No       | ❌ Yes       | ❌ No   | ❌ Basic only                   |
 | **Excel**         | ✅ Yes     | ✅ Size only | ❌ No   | ✅ Display weight               |
 
 #### Cross-Format Styling Conversion
@@ -658,13 +658,8 @@ npm test
 
 - The repository keeps `package.json` at `0.0.0-development`; release tags control the published version.
 - Create a GitHub release with a semantic tag (e.g. `v2.1.0`). Publishing only runs for non-prerelease tags.
-- Add an `NPM_TOKEN` repository secret with publish rights. The release workflow uses it to authenticate and calls `npm publish`.
 - The workflow (`.github/workflows/publish.yml`) automatically installs dependencies, rewrites the package version from the tag, and runs the standard publish pipeline.
 
-**Private distribution options**
-- Unscoped packages on npm must be public. To keep this package private, re-scope it (e.g. `@your-org/aac-processors`) and configure `publishConfig.access: "restricted"`—this requires an npm org with paid seats.
-- Alternatively, publish to GitHub Packages by adjusting the workflow’s registry URL and using a `GITHUB_TOKEN` with the `packages: write` permission.
-
 ---
 
 ## 📄 License
@@ -715,16 +710,6 @@ Inspired by the Python AACProcessors project
 - [ ] **Add Symbol Tools coverage** (currently 0%) - Implement tests for PCS and ARASAAC symbol lookups to reach >70% coverage
 - [ ] **Fix property-based test failures** - Resolve TypeScript interface compatibility issues in edge case generators
 
-### Recently Completed ✅
-
-- [x] **Core utilities test coverage** - Complete implementation for analyze.ts and fileProcessor.ts (0% → 100% coverage, 45 comprehensive tests, src/core/ directory 30% → 83% coverage)
-- [x] **CLI test infrastructure** - Fixed DotProcessor parsing and test expectations (0 → 25 passing tests, 100% CLI functionality)
-- [x] **Critical linting fixes** - Resolved unsafe argument types and CLI type safety issues (177 → 123 errors, 32% improvement)
-- [x] **Audio test syntax fixes** - Fixed non-null assertion errors in audio tests (21 → 5 failing tests, 76% improvement)
-- [x] **Comprehensive styling support** - Complete implementation across all AAC formats with cross-format preservation (Added: AACStyle interface, enhanced all processors, 7 new test cases, complete documentation)
-- [x] **TouchChatProcessor save/load functionality** - Fixed button persistence and ID mapping (coverage improved from 57.62% to 86.44%)
-- [x] **Build integration** - Ensure `npm run build` is executed before CLI tests (Fixed: All test scripts now automatically build before running)
-
 ### Medium Priority
 
 - [ ] **Performance optimization** - Optimize memory usage for very large communication boards (1000+ buttons)
@@ -749,29 +734,6 @@ Inspired by the Python AACProcessors project
 - [ ] **CI/CD improvements** - Add automated releases and npm publishing
 - [ ] **Documentation improvements** - Add more real-world examples and tutorials
 
-### Known Issues
-
-- ⚠️ **Audio Persistence**: 5 functional tests failing due to audio recording not persisting through SnapProcessor save/load cycle
-- ⚠️ **Grid3 Layout**: Grid sizes not reliable and X,Y positions incorrect, particularly affecting Grid3 processor functionality
-- ⚠️ **Database Constraints**: UNIQUE constraint violations with real-world data files (Page.Id and buttons.id conflicts)
-- ⚠️ **Linting**: 123 ESLint issues remaining (mostly in test files, reduced from 177)
-- ⚠️ **SnapProcessor**: Lowest coverage at 48.32%, missing comprehensive audio handling tests
-- ⚠️ **Memory usage**: Large files (>50MB) may cause memory pressure
-- ⚠️ **Concurrent access**: Some processors not fully thread-safe for simultaneous writes
-
-### 🧪 Current Test Status & Immediate Follow-Up
-
-As of the latest run (`npm test`), **47 suites pass / 6 fail (10 individual tests)**. Remaining blockers are:
-
-1. **Edge-case loaders** – corrupted JSON/XML fixtures still expect explicit exceptions. Decide whether to restore the old throwing behaviour (Asterics/OPML/DOT) or update the tests to accept the softer error reporting.
-2. **Gridset exports & styling** – round-trip continues to lose a button and the styling suite cannot find `style.xml`. GridsetProcessor needs to preserve button arrays and emit the styling assets Grid 3 expects.
-3. **DOT navigation semantics** – the deterministic DOT test still falls back to `SPEAK`. Improve semantic reconstruction when loading navigation edges so navigation buttons survive round-trips.
-4. **Advanced workflow scenario** – the multi-format pipeline still loses Spanish translations (likely during the Gridset ⇄ Snap steps); trace text propagation and patch the conversion chain.
-5. **Styling suite** – Grid 3 export still lacks `style.xml`; ensure the styling assets are generated alongside the gridset payload.
-6. **Memory comparison suite** – memory delta expectations are still unmet (TouchChat GC + DOT vs others). Either recalibrate the harness or tune the processors before re-enabling the assertions.
-
-Clearing these items will put the test matrix back in the green and unblock the release.
-
 ## More enhancements
 
 - Much more effort put into fixing the layout issues. Grid sizes are not reliably and X, Y positions too. Particularly in the Grid3

package/dist/analytics/history.d.ts

@@ -0,0 +1,57 @@
+import { dotNetTicksToDate } from '../utils/dotnetTicks';
+import { Grid3UserPath } from '../processors/gridset/helpers';
+import { SnapUserInfo } from '../processors/snap/helpers';
+export type HistorySource = 'Grid' | 'Snap';
+export interface HistoryOccurrence {
+    timestamp: Date;
+    latitude?: number | null;
+    longitude?: number | null;
+    modeling?: boolean;
+    accessMethod?: number | null;
+    pageId?: string | null;
+}
+export interface HistoryPlatformExtras {
+    label?: string;
+    message?: string;
+    buttonId?: string;
+    contentXml?: string;
+}
+export interface HistoryEntry {
+    id: string;
+    source: HistorySource;
+    content: string;
+    occurrences: HistoryOccurrence[];
+    raw?: unknown;
+    platform?: HistoryPlatformExtras;
+}
+export { dotNetTicksToDate };
+/**
+ * Read Grid 3 phrase history from a history.sqlite database and tag entries with their source.
+ */
+export declare function readGrid3History(historyDbPath: string): HistoryEntry[];
+/**
+ * Read Grid 3 history for a specific user/language combination.
+ */
+export declare function readGrid3HistoryForUser(userName: string, langCode?: string): HistoryEntry[];
+/**
+ * Read every available Grid 3 history database on the machine.
+ */
+export declare function readAllGrid3History(): HistoryEntry[];
+/**
+ * Read Snap button usage from a pageset database and tag entries with source.
+ */
+export declare function readSnapUsage(pagesetPath: string): HistoryEntry[];
+/**
+ * Read Snap usage for a specific user across all discovered pagesets.
+ */
+export declare function readSnapUsageForUser(userId?: string, packageNamePattern?: string): HistoryEntry[];
+export declare function listSnapUsers(): SnapUserInfo[];
+/**
+ * List Grid 3 users on the current machine.
+ */
+export declare function listGrid3Users(): Grid3UserPath[];
+/**
+ * Convenience helper to gather all available history across Grid 3 and Snap.
+ * Returns an empty array if no history files are present.
+ */
+export declare function collectUnifiedHistory(): HistoryEntry[];

package/dist/analytics/history.js

@@ -0,0 +1,72 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.dotNetTicksToDate = void 0;
+exports.readGrid3History = readGrid3History;
+exports.readGrid3HistoryForUser = readGrid3HistoryForUser;
+exports.readAllGrid3History = readAllGrid3History;
+exports.readSnapUsage = readSnapUsage;
+exports.readSnapUsageForUser = readSnapUsageForUser;
+exports.listSnapUsers = listSnapUsers;
+exports.listGrid3Users = listGrid3Users;
+exports.collectUnifiedHistory = collectUnifiedHistory;
+const dotnetTicks_1 = require("../utils/dotnetTicks");
+Object.defineProperty(exports, "dotNetTicksToDate", { enumerable: true, get: function () { return dotnetTicks_1.dotNetTicksToDate; } });
+const helpers_1 = require("../processors/gridset/helpers");
+const helpers_2 = require("../processors/snap/helpers");
+/**
+ * Read Grid 3 phrase history from a history.sqlite database and tag entries with their source.
+ */
+function readGrid3History(historyDbPath) {
+    return (0, helpers_1.readGrid3History)(historyDbPath).map((e) => ({
+        ...e,
+        source: 'Grid',
+    }));
+}
+/**
+ * Read Grid 3 history for a specific user/language combination.
+ */
+function readGrid3HistoryForUser(userName, langCode) {
+    return (0, helpers_1.readGrid3HistoryForUser)(userName, langCode).map((e) => ({
+        ...e,
+        source: 'Grid',
+    }));
+}
+/**
+ * Read every available Grid 3 history database on the machine.
+ */
+function readAllGrid3History() {
+    return (0, helpers_1.readAllGrid3History)().map((e) => ({ ...e, source: 'Grid' }));
+}
+/**
+ * Read Snap button usage from a pageset database and tag entries with source.
+ */
+function readSnapUsage(pagesetPath) {
+    return (0, helpers_2.readSnapUsage)(pagesetPath).map((e) => ({ ...e, source: 'Snap' }));
+}
+/**
+ * Read Snap usage for a specific user across all discovered pagesets.
+ */
+function readSnapUsageForUser(userId, packageNamePattern = 'TobiiDynavox') {
+    return (0, helpers_2.readSnapUsageForUser)(userId, packageNamePattern).map((e) => ({
+        ...e,
+        source: 'Snap',
+    }));
+}
+function listSnapUsers() {
+    return (0, helpers_2.findSnapUsers)();
+}
+/**
+ * List Grid 3 users on the current machine.
+ */
+function listGrid3Users() {
+    return (0, helpers_1.findGrid3Users)();
+}
+/**
+ * Convenience helper to gather all available history across Grid 3 and Snap.
+ * Returns an empty array if no history files are present.
+ */
+function collectUnifiedHistory() {
+    const gridHistory = readAllGrid3History();
+    const snapHistory = (0, helpers_2.findSnapUsers)().flatMap((u) => readSnapUsageForUser(u.userId));
+    return [...gridHistory, ...snapHistory];
+}

package/dist/core/analyze.d.ts

@@ -1,6 +1,16 @@
 import { AACTree } from './treeStructure';
 import { BaseProcessor, ProcessorOptions } from './baseProcessor';
+/**
+ * Resolve a processor instance by friendly format name or common extension.
+ * @param format Format key or extension (e.g., 'snap', 'obf', 'xlsx')
+ * @param options Optional processor configuration
+ */
 export declare function getProcessor(format: string, options?: ProcessorOptions): BaseProcessor;
+/**
+ * Convenience helper to load a file into an AACTree using the inferred processor.
+ * @param file Path to the source file
+ * @param format Format key or extension (passed to getProcessor)
+ */
 export declare function analyze(file: string, format: string): {
     tree: AACTree;
 };

package/dist/core/analyze.js

@@ -11,6 +11,11 @@ const snapProcessor_1 = require("../processors/snapProcessor");
 const dotProcessor_1 = require("../processors/dotProcessor");
 const excelProcessor_1 = require("../processors/excelProcessor");
 const applePanelsProcessor_1 = require("../processors/applePanelsProcessor");
+/**
+ * Resolve a processor instance by friendly format name or common extension.
+ * @param format Format key or extension (e.g., 'snap', 'obf', 'xlsx')
+ * @param options Optional processor configuration
+ */
 function getProcessor(format, options) {
     const normalizedFormat = (format || '').toLowerCase();
     switch (normalizedFormat) {
@@ -42,6 +47,11 @@ function getProcessor(format, options) {
             throw new Error('Unknown format: ' + format);
     }
 }
+/**
+ * Convenience helper to load a file into an AACTree using the inferred processor.
+ * @param file Path to the source file
+ * @param format Format key or extension (passed to getProcessor)
+ */
 function analyze(file, format) {
     const processor = getProcessor(format);
     const tree = processor.loadIntoTree(file);

package/dist/core/stringCasing.d.ts

@@ -20,6 +20,9 @@ export declare enum StringCasing {
  * @param text - The text to analyze for casing pattern
  * @returns StringCasing enum value representing the detected casing
  */
+/**
+ * Infer the dominant casing style of a string (camel, pascal, snake, kebab, title, sentence, upper, lower).
+ */
 export declare function detectCasing(text: string): StringCasing;
 /**
  * Converts text to the specified casing
@@ -27,6 +30,11 @@ export declare function detectCasing(text: string): StringCasing;
  * @param targetCasing - The desired casing format
  * @returns The text converted to the target casing
  */
+/**
+ * Convert a string into the requested casing style.
+ * @param text Input string
+ * @param targetCasing Desired casing variant
+ */
 export declare function convertCasing(text: string, targetCasing: StringCasing): string;
 /**
  * Utility function to check if text is primarily numeric or empty
@@ -34,4 +42,7 @@ export declare function convertCasing(text: string, targetCasing: StringCasing):
  * @param text - The text to check
  * @returns True if the text should be considered non-meaningful
  */
+/**
+ * Check whether a string is empty or represents a numeric value.
+ */
 export declare function isNumericOrEmpty(text: string): boolean;

package/dist/core/stringCasing.js

@@ -27,6 +27,9 @@ var StringCasing;
  * @param text - The text to analyze for casing pattern
  * @returns StringCasing enum value representing the detected casing
  */
+/**
+ * Infer the dominant casing style of a string (camel, pascal, snake, kebab, title, sentence, upper, lower).
+ */
 function detectCasing(text) {
     if (!text || text.length === 0)
         return StringCasing.LOWER;
@@ -102,6 +105,11 @@ function detectCasing(text) {
  * @param targetCasing - The desired casing format
  * @returns The text converted to the target casing
  */
+/**
+ * Convert a string into the requested casing style.
+ * @param text Input string
+ * @param targetCasing Desired casing variant
+ */
 function convertCasing(text, targetCasing) {
     if (!text || text.length === 0)
         return text;
@@ -164,6 +172,9 @@ function convertCasing(text, targetCasing) {
  * @param text - The text to check
  * @returns True if the text should be considered non-meaningful
  */
+/**
+ * Check whether a string is empty or represents a numeric value.
+ */
 function isNumericOrEmpty(text) {
     const trimmed = text.trim();
     if (trimmed.length <= 1)

package/dist/index.d.ts

@@ -2,6 +2,7 @@ export * from './core/treeStructure';
 export * from './core/baseProcessor';
 export * from './core/stringCasing';
 export * from './processors';
+export { collectUnifiedHistory, listGrid3Users as listHistoryGrid3Users, listSnapUsers as listHistorySnapUsers, } from './analytics/history';
 import { BaseProcessor } from './core/baseProcessor';
 /**
  * Factory function to get the appropriate processor for a file extension

package/dist/index.js

@@ -14,6 +14,7 @@ var __exportStar = (this && this.__exportStar) || function(m, exports) {
     for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
 };
 Object.defineProperty(exports, "__esModule", { value: true });
+exports.listHistorySnapUsers = exports.listHistoryGrid3Users = exports.collectUnifiedHistory = void 0;
 exports.getProcessor = getProcessor;
 exports.getSupportedExtensions = getSupportedExtensions;
 exports.isExtensionSupported = isExtensionSupported;
@@ -22,6 +23,10 @@ __exportStar(require("./core/treeStructure"), exports);
 __exportStar(require("./core/baseProcessor"), exports);
 __exportStar(require("./core/stringCasing"), exports);
 __exportStar(require("./processors"), exports);
+var history_1 = require("./analytics/history");
+Object.defineProperty(exports, "collectUnifiedHistory", { enumerable: true, get: function () { return history_1.collectUnifiedHistory; } });
+Object.defineProperty(exports, "listHistoryGrid3Users", { enumerable: true, get: function () { return history_1.listGrid3Users; } });
+Object.defineProperty(exports, "listHistorySnapUsers", { enumerable: true, get: function () { return history_1.listSnapUsers; } });
 const dotProcessor_1 = require("./processors/dotProcessor");
 const excelProcessor_1 = require("./processors/excelProcessor");
 const opmlProcessor_1 = require("./processors/opmlProcessor");

package/dist/processors/dotProcessor.js

@@ -17,17 +17,14 @@ class DotProcessor extends baseProcessor_1.BaseProcessor {
         const edges = [];
         // Extract all edge statements using regex to handle single-line DOT files
         const edgeRegex = /"?([^"\s]+)"?\s*->\s*"?([^"\s]+)"?(?:\s*\[label="([^"]+)"\])?/g;
-        const nodeRegex = /"?([^"\s]+)"?\s*\[label="([^"]+)"\]/g;
-        // Find all explicit node definitions
-        let nodeMatch;
-        while ((nodeMatch = nodeRegex.exec(content)) !== null) {
-            const [, id, label] = nodeMatch;
-            nodes.set(id, { id, label });
-        }
-        // Find all edge definitions
+        // We need to find nodes, but avoid matching the target of an edge which might look like a node definition
+        // e.g. A -> B [label="L"]  -- "B [label="L"]" looks like a node def
+        // Strategy: Find all edges, record them, and then "mask" them in the content to avoid false positives for nodes
+        let maskedContent = content;
         let edgeMatch;
+        // Find all edge definitions
         while ((edgeMatch = edgeRegex.exec(content)) !== null) {
-            const [, from, to, label] = edgeMatch;
+            const [fullMatch, from, to, label] = edgeMatch;
             edges.push({ from, to, label });
             // Add nodes if they don't exist (implicit definition)
             if (!nodes.has(from)) {
@@ -36,6 +33,23 @@ class DotProcessor extends baseProcessor_1.BaseProcessor {
             if (!nodes.has(to)) {
                 nodes.set(to, { id: to, label: to });
             }
+            // Mask this edge in the content so we don't match it as a node
+            // We replace it with spaces to preserve indices if needed, but simple replacement is enough here
+            maskedContent = maskedContent.replace(fullMatch, ' '.repeat(fullMatch.length));
+        }
+        // Now find explicit node definitions in the masked content
+        // This regex matches: ID [label="LABEL"]
+        // We use a non-greedy match for the label content to handle escaped quotes if possible,
+        // but the previous regex `[^"]+` was too simple.
+        // Better regex for quoted string content: (?:[^"\\]|\\.)*
+        const nodeRegex = /"?([^"\s]+)"?\s*\[label="((?:[^"\\]|\\.)*)"\]/g;
+        let nodeMatch;
+        while ((nodeMatch = nodeRegex.exec(maskedContent)) !== null) {
+            const [, id, rawLabel] = nodeMatch;
+            // Unescape the label: replace \" with " and \\ with \
+            const label = rawLabel.replace(/\\"/g, '"').replace(/\\\\/g, '\\');
+            // Only update if not already defined or if we want to override the implicit label
+            nodes.set(id, { id, label });
         }
         return { nodes: Array.from(nodes.values()), edges };
     }
@@ -82,7 +96,8 @@ class DotProcessor extends baseProcessor_1.BaseProcessor {
         let hasControl = false;
         for (let i = 0; i < head.length; i++) {
             const code = head.charCodeAt(i);
-            if (code === 0 || (code >= 0 && code <= 8) || (code >= 14 && code <= 31) || code >= 127) {
+            // Allow UTF-8 characters (code >= 127)
+            if (code === 0 || (code >= 0 && code <= 8) || (code >= 14 && code <= 31)) {
                 hasControl = true;
                 break;
             }
@@ -150,10 +165,14 @@ class DotProcessor extends baseProcessor_1.BaseProcessor {
     }
     saveFromTree(tree, _outputPath) {
         let dotContent = 'digraph AACBoard {\n';
+        // Helper to escape DOT string
+        const escapeDotString = (str) => {
+            return str.replace(/\\/g, '\\\\').replace(/"/g, '\\"');
+        };
         // Add nodes
         for (const pageId in tree.pages) {
             const page = tree.pages[pageId];
-            dotContent += `  "${page.id}" [label="${page.name}"]\n`;
+            dotContent += `  "${page.id}" [label="${escapeDotString(page.name)}"]\n`;
         }
         // Add edges from navigation buttons (semantic intent or legacy targetPageId)
         for (const pageId in tree.pages) {
@@ -166,7 +185,7 @@ class DotProcessor extends baseProcessor_1.BaseProcessor {
                 .forEach((btn) => {
                 const target = btn.semanticAction?.targetId || btn.targetPageId;
                 if (target) {
-                    dotContent += `  "${page.id}" -> "${target}" [label="${btn.label}"]\n`;
+                    dotContent += `  "${page.id}" -> "${target}" [label="${escapeDotString(btn.label)}"]\n`;
                 }
             });
         }

package/dist/processors/gridset/colorUtils.d.ts

@@ -0,0 +1,69 @@
+/**
+ * Grid3 Color Utilities
+ *
+ * Comprehensive color handling for Grid3 format, including:
+ * - CSS color name lookup (147 named colors)
+ * - Color format conversion (hex, RGB, RGBA, named colors)
+ * - Color manipulation (darkening, normalization)
+ * - Grid3-specific color formatting (8-digit ARGB hex)
+ */
+/**
+ * Get RGB values for a CSS color name
+ * @param name - CSS color name (case-insensitive)
+ * @returns RGB tuple [r, g, b] or undefined if not found
+ */
+export declare function getNamedColor(name: string): [number, number, number] | undefined;
+/**
+ * Convert RGBA values to hex format
+ * @param r - Red channel (0-255)
+ * @param g - Green channel (0-255)
+ * @param b - Blue channel (0-255)
+ * @param a - Alpha channel (0-1)
+ * @returns Hex color string in format #RRGGBBAA
+ */
+export declare function rgbaToHex(r: number, g: number, b: number, a: number): string;
+/**
+ * Convert a single color channel value to hex
+ * @param value - Channel value (0-255)
+ * @returns Two-digit hex string
+ */
+export declare function channelToHex(value: number): string;
+/**
+ * Clamp RGB channel value to valid range
+ * @param value - Channel value
+ * @returns Clamped value (0-255)
+ */
+export declare function clampColorChannel(value: number): number;
+/**
+ * Clamp alpha value to valid range
+ * @param value - Alpha value
+ * @returns Clamped value (0-1)
+ */
+export declare function clampAlpha(value: number): number;
+/**
+ * Convert any color format to hex
+ * Supports: hex (#RGB, #RRGGBB, #RRGGBBAA), RGB/RGBA, and CSS color names
+ * @param value - Color string in any supported format
+ * @returns Hex color string (#RRGGBBAA) or undefined if invalid
+ */
+export declare function toHexColor(value: string): string | undefined;
+/**
+ * Darken a hex color by a specified amount
+ * @param hex - Hex color string
+ * @param amount - Amount to darken (0-255)
+ * @returns Darkened hex color
+ */
+export declare function darkenColor(hex: string, amount: number): string;
+/**
+ * Normalize any color format to Grid3's 8-digit hex format
+ * @param input - Color string in any supported format
+ * @param fallback - Fallback color if input is invalid (default: white)
+ * @returns Normalized color in format #AARRGGBBFF
+ */
+export declare function normalizeColor(input: string, fallback?: string): string;
+/**
+ * Ensure a color has an alpha channel (Grid3 format requires 8-digit ARGB)
+ * @param color - Color string (hex format)
+ * @returns Color with alpha channel in format #AARRGGBBFF
+ */
+export declare function ensureAlphaChannel(color: string | undefined): string;