@willwade/aac-processors 0.0.12 → 0.0.13

package/README.md

@@ -1,8 +1,8 @@
 # AACProcessors
 
-[![Coverage](https://img.shields.io/badge/coverage-77%25-green.svg)](./coverage)
+[![Coverage](https://img.shields.io/badge/coverage-74%25-green.svg)](./coverage)
 [![TypeScript](https://img.shields.io/badge/TypeScript-100%25-blue.svg)](https://www.typescriptlang.org/)
-[![Tests](https://img.shields.io/badge/tests-140%20tests-brightgreen.svg)](./test)
+[![Tests](https://img.shields.io/badge/tests-495%20tests-brightgreen.svg)](./test)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
 A comprehensive **TypeScript library** for processing AAC (Augmentative and Alternative Communication) file formats with advanced translation support, cross-format conversion, and robust error handling.
@@ -20,6 +20,7 @@ A comprehensive **TypeScript library** for processing AAC (Augmentative and Alte
 - **Apple Panels** (MacOS) - Plist format support
 - **Asterics Grid** - Native Asterics Grid format with audio
 - **Excel** - Export to Microsoft Excel for vocabulary analysis
+- **Analyics & Metrics** - High-parity AAC effort metrics and clinical analysis tools
 
 ### **Advanced Capabilities**
 
@@ -28,6 +29,7 @@ A comprehensive **TypeScript library** for processing AAC (Augmentative and Alte
 - 🎨 **Comprehensive styling support** - Preserve visual appearance across formats
 - 🧪 **Property-based testing** - Robust validation with 140+ tests
 - ✅ **Format validation** - Spec-based validation for all supported formats
+- 📊 **Clinical Metrics** - High-parity AAC effort algorithm (v0.2) and vocabulary coverage analysis
 - ⚡ **Performance optimized** - Memory-efficient processing of large files
 - 🛡️ **Error recovery** - Graceful handling of corrupted data
 - 🔒 **Thread-safe** - Concurrent processing support
@@ -192,6 +194,29 @@ const translatedBuffer = processor.processTexts(
 console.log("Translation complete!");
 ```
 
+### 📊 AAC Analytics & Clinical Metrics
+
+The library includes an optional high-performance analytics engine for evaluating AAC board sets based on the **AAC Effort Algorithm (v0.2)**.
+
+#### **Key Metrics Features**
+- **Effort Scores**: Calculate the physical/cognitive cost of any word (Distance, Field Size, Motor Planning).
+- **Vocabulary Coverage**: Compare board sets against core vocabulary lists (e.g., Anderson & Bitner).
+- **Sentence Analysis**: Measure the effort required to construct common test sentences.
+- **Comparative Analysis**: Identify gaps and improvements between two pageset versions.
+
+For detailed documentation, see the **[AAC Metrics Guide](./src/optional/analytics/docs/AAC_METRICS_GUIDE.md)** and **[Vocabulary Analysis Guide](./src/optional/analytics/docs/VOCABULARY_ANALYSIS_GUIDE.md)**.
+
+```typescript
+import { ObfsetProcessor, Analytics } from '@willwade/aac-processors';
+
+const processor = new ObfsetProcessor();
+const tree = processor.loadIntoTree('my_pageset.obfset');
+
+// Run clinical effort analysis
+const result = new Analytics.MetricsCalculator().analyze(tree);
+console.log(`Average Effort: ${result.total_words}`);
+```
+
 ### Format Validation
 
 Validate AAC files against format specifications to ensure data integrity:
@@ -710,6 +735,14 @@ npm run coverage:report
 
 **Note**: All test commands automatically run `npm run build` first to ensure CLI tests have the required `dist/` files. CLI integration tests require the compiled JavaScript files to test the command-line interface.
 
+### 🛠️ Utility Scripts
+
+A wide range of utility scripts for batch processing, audio generation, and advanced analysis are available in the **[scripts/](./scripts/README.md)** directory. These include:
+- **Analysis**: Pageset reporting and vocabulary extraction.
+- **Audio**: Automated TTS generation and audio-enhanced pageset creation.
+- **Conversion**: TSV-to-Gridset and other format-shifting tools.
+- **Translation**: Batch translation workflows using Google, Azure, and Gemini.
+
 ### Development Commands
 
 ```bash
@@ -798,57 +831,27 @@ Inspired by the Python AACProcessors project
 
 ### 🔥 Critical Priority (Immediate Action Required)
 
-- [ ] **Fix audio persistence issues** (5 tests failing) - Resolve functional audio recording persistence in SnapProcessor save/load cycle (reduced from 21 failing tests)
-- [ ] **Resolve database constraint violations** - Fix UNIQUE constraint failures in Page.Id and buttons.id when processing real-world data files (blocks production usage)
-- [ ] **Fix Grid3 layout issues** - Grid sizes not reliable and X,Y positions incorrect, particularly affecting Grid3 processor functionality
+- [ ] **Road Testing** - Perform comprehensive layout and formatting validation across diverse pagesets to verify conversion fidelity.
+- [ ] **Fix audio persistence issues** - Resolve functional audio recording persistence in `SnapProcessor` save/load cycle (5 failing tests remaining).
+- [x] **Access Method Modeling** - Support for switch scanning (linear, row-column, block) integrated into AAC metrics.
 
 ### 🚨 High Priority (Next Sprint)
 
-- [ ] **Complete SnapProcessor coverage** (currently 48.32% - lowest processor) - Add comprehensive audio handling, database corruption tests, and SQLite schema validation to reach >75% coverage
-- [ ] **Reduce remaining linting issues** (123 remaining) - Address TypeScript eslint errors and warnings in test files and less critical areas
-- [ ] **Improve GridsetProcessor coverage** (currently 71.69%) - Enhance ZIP handling and complex Grid3 archive support to reach >80% coverage
-
-### ⚠️ Medium Priority (Following Sprint)
+- [ ] **Complete SnapProcessor coverage** (currently ~60%) - Reach >75% coverage by adding comprehensive audio handling and database corruption tests.
+- [ ] **Symbol System Rethink** - Explore treating "Symbols" as a first-class entity (alongside Pages/Buttons) to support richer metadata (library IDs, synonyms, multi-lang names).
+- [ ] **Language & Locale Persistence** - Ensure current language and locale information is correctly preserved and bubbled up to the `AACTree` level.
 
-- [ ] **Improve GridsetProcessor coverage** (currently 71.69%) - Enhance ZIP handling and complex Grid3 archive support to reach >80% coverage
-- [ ] **Improve AstericsGridProcessor coverage** (currently 62.5%) - Add comprehensive tests for Asterics Grid format processing to reach >80% coverage
-- [ ] **Add Symbol Tools coverage** (currently 0%) - Implement tests for PCS and ARASAAC symbol lookups to reach >70% coverage
-- [ ] **Fix property-based test failures** - Resolve TypeScript interface compatibility issues in edge case generators
+### ⚠️ Medium Priority
 
-### Medium Priority
+- [ ] **Adaptive Metrics** - Expand scanning analysis to include dwell times and more complex switch logic configurations.
 
-- [ ] **Performance optimization** - Optimize memory usage for very large communication boards (1000+ buttons)
-- [ ] **Add GridsetProcessor ZIP handling** - Improve support for complex Grid3 ZIP archives
-- [ ] **Enhance error recovery** - Better handling of partially corrupted database files
-- [ ] **Add streaming support** - Process very large files without loading entirely into memory
-- [ ] **Improve translation workflows** - Add support for translation service integrations (Google Translate, Azure, etc.)
-- [ ] **Add symbol library integration** - Complete implementation of PCS, ARASAAC symbol lookups
 
 ### Low Priority
 
-- [ ] **Add more AAC formats** - Look for other common AAC formats to add
-- [ ] **Plugin system** - Allow third-party processors and extensions
-- [ ] **Batch processing CLI** - Process multiple files in parallel
-- [ ] **Configuration file support** - Allow .aacprocessors.json config files
 
-### Testing & Quality
+- [ ] **Batch processing CLI** - Process multiple files/directories in parallel.
 
-- [ ] **Reach 80%+ test coverage** - Current: 65.15% (target: 80%+)
-- [ ] **Add mutation testing** - Use Stryker.js for mutation testing
-- [ ] **Add benchmark suite** - Performance regression testing
-- [ ] **CI/CD improvements** - Add automated releases and npm publishing
-- [ ] **Documentation improvements** - Add more real-world examples and tutorials
-
-## More enhancements
-
-- Much more effort put into fixing the layout issues. Grid sizes are not reliably and X, Y positions too. Particularly in the Grid3
-- Action types on buttons. We have SPEAK, NAVIGATE, but we could also have things like "PLAY_AUDIO", "SEND_MESSAGE", "OPEN_URL", etc. This would allow us to have a more flexible structure for actions, especially if we want to support different types of actions or additional metadata in the future. This needs to be very flexible. eaach system has a very different set of actions. We could have a set of core actions that are supported by all systems, and then allow each system to define its own additional actions. This could be done by having a set of core action types that are defined in the AACSystem type, and then allowing each system to define its own additional action types in the AACPageSet or AACPage types.
-- Current language and locale information of aac pageset
-- Symbols and their associated data. Now we have an optional part but I think this might need rethinking. I wonder if "Symbols" should be a separate type - alongside AACPageSet and AACPage. that then AACPageSet and AACPage can reference. This would allow us to have a more flexible structure for symbols, especially if we want to support different types of symbols or additional metadata in the future. Symbols would have: Library name, ID, Text name, Reference (URL, DB ID, etc.), and an optional part for additional metadata. We could have synoynms, antonyms, also and for different languages somehow.
-- Somehow we need to deal with access. Switch scanning: Blocks probably being the main aspect. But also - we need to identify somewhere a set of core rules for each aac system. Like for example - what access methods are supported, what languages are supported, what symbols are supported, etc. This could be a set of rules that can be referenced by AACSystem - a different type altogether maybe? It could be in a JSON format maybe. (use case here being for a MCP).
 
 ### Contributing
 
 Want to help with any of these items? See our [Contributing Guidelines](#-contributing) and pick an issue that interests you!
-
----

package/dist/core/treeStructure.d.ts

@@ -29,6 +29,25 @@ export declare enum AACSemanticIntent {
     SCAN_SELECT = "SCAN_SELECT",
     PLATFORM_SPECIFIC = "PLATFORM_SPECIFIC"
 }
+/**
+ * Scanning types for accessibility
+ */
+export declare enum AACScanType {
+    LINEAR = "linear",// Left-to-right, top-to-bottom
+    ROW_COLUMN = "row-column",// Scan rows, then columns
+    COLUMN_ROW = "column-row",// Scan columns, then rows
+    BLOCK_ROW_COLUMN = "block-row-column",// Scan blocks, then rows, then columns
+    BLOCK_COLUMN_ROW = "block-column-row"
+}
+/**
+ * Configuration for a scan block
+ */
+export interface AACScanBlock {
+    id: number;
+    name?: string;
+    order?: number;
+    scanType?: AACScanType;
+}
 export interface AACSemanticAction {
     category?: AACSemanticCategory;
     intent: AACSemanticIntent | string;
@@ -110,7 +129,14 @@ export declare class AACButton implements IAACButton {
     y?: number;
     columnSpan?: number;
     rowSpan?: number;
+    /**
+     * @deprecated Use scanBlock instead (singular, not array)
+     */
     scanBlocks?: number[];
+    /**
+     * Scan block number (1-8) for block scanning
+     */
+    scanBlock?: number;
     visibility?: 'Visible' | 'Hidden' | 'Disabled' | 'PointerAndTouchOnly' | 'Empty';
     directActivate?: boolean;
     audioDescription?: string;
@@ -119,7 +145,7 @@ export declare class AACButton implements IAACButton {
     };
     semantic_id?: string;
     clone_id?: string;
-    constructor({ id, label, message, targetPageId, semanticAction, audioRecording, style, contentType, contentSubType, image, resolvedImageEntry, symbolLibrary, symbolPath, x, y, columnSpan, rowSpan, scanBlocks, visibility, directActivate, parameters, semantic_id, clone_id, type, action, }: {
+    constructor({ id, label, message, targetPageId, semanticAction, audioRecording, style, contentType, contentSubType, image, resolvedImageEntry, symbolLibrary, symbolPath, x, y, columnSpan, rowSpan, scanBlocks, scanBlock, visibility, directActivate, parameters, semantic_id, clone_id, type, action, }: {
         id: string;
         label?: string;
         message?: string;
@@ -143,6 +169,7 @@ export declare class AACButton implements IAACButton {
         columnSpan?: number;
         rowSpan?: number;
         scanBlocks?: number[];
+        scanBlock?: number;
         visibility?: 'Visible' | 'Hidden' | 'Disabled' | 'PointerAndTouchOnly' | 'Empty';
         directActivate?: boolean;
         parameters?: {
@@ -177,7 +204,10 @@ export declare class AACPage implements IAACPage {
     sounds?: any[];
     semantic_ids?: string[];
     clone_ids?: string[];
-    constructor({ id, name, grid, buttons, parentId, style, locale, descriptionHtml, images, sounds, semantic_ids, clone_ids, }: {
+    scanningConfig?: import('../types/aac').ScanningConfig;
+    scanType?: AACScanType;
+    scanBlocksConfig?: AACScanBlock[];
+    constructor({ id, name, grid, buttons, parentId, style, locale, descriptionHtml, images, sounds, semantic_ids, clone_ids, scanningConfig, scanBlocksConfig, scanType, }: {
         id: string;
         name?: string;
         grid?: Array<Array<AACButton | null>> | {
@@ -193,6 +223,9 @@ export declare class AACPage implements IAACPage {
         sounds?: any[];
         semantic_ids?: string[];
         clone_ids?: string[];
+        scanningConfig?: import('../types/aac').ScanningConfig;
+        scanBlocksConfig?: AACScanBlock[];
+        scanType?: AACScanType;
     });
     addButton(button: AACButton): void;
 }

package/dist/core/treeStructure.js

@@ -1,6 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.AACTree = exports.AACPage = exports.AACButton = exports.AACSemanticIntent = exports.AACSemanticCategory = void 0;
+exports.AACTree = exports.AACPage = exports.AACButton = exports.AACScanType = exports.AACSemanticIntent = exports.AACSemanticCategory = void 0;
 // Semantic action categories for cross-platform compatibility
 var AACSemanticCategory;
 (function (AACSemanticCategory) {
@@ -42,8 +42,19 @@ var AACSemanticIntent;
     // Custom
     AACSemanticIntent["PLATFORM_SPECIFIC"] = "PLATFORM_SPECIFIC";
 })(AACSemanticIntent || (exports.AACSemanticIntent = AACSemanticIntent = {}));
+/**
+ * Scanning types for accessibility
+ */
+var AACScanType;
+(function (AACScanType) {
+    AACScanType["LINEAR"] = "linear";
+    AACScanType["ROW_COLUMN"] = "row-column";
+    AACScanType["COLUMN_ROW"] = "column-row";
+    AACScanType["BLOCK_ROW_COLUMN"] = "block-row-column";
+    AACScanType["BLOCK_COLUMN_ROW"] = "block-column-row";
+})(AACScanType || (exports.AACScanType = AACScanType = {}));
 class AACButton {
-    constructor({ id, label = '', message = '', targetPageId, semanticAction, audioRecording, style, contentType, contentSubType, image, resolvedImageEntry, symbolLibrary, symbolPath, x, y, columnSpan, rowSpan, scanBlocks, visibility, directActivate, parameters, semantic_id, clone_id, 
+    constructor({ id, label = '', message = '', targetPageId, semanticAction, audioRecording, style, contentType, contentSubType, image, resolvedImageEntry, symbolLibrary, symbolPath, x, y, columnSpan, rowSpan, scanBlocks, scanBlock, visibility, directActivate, parameters, semantic_id, clone_id, 
     // Legacy input support
     type, action, }) {
         this.id = id;
@@ -64,6 +75,7 @@ class AACButton {
         this.columnSpan = columnSpan;
         this.rowSpan = rowSpan;
         this.scanBlocks = scanBlocks;
+        this.scanBlock = scanBlock;
         this.visibility = visibility;
         this.directActivate = directActivate;
         this.parameters = parameters;
@@ -156,7 +168,7 @@ class AACButton {
 }
 exports.AACButton = AACButton;
 class AACPage {
-    constructor({ id, name = '', grid = [], buttons = [], parentId = null, style, locale, descriptionHtml, images, sounds, semantic_ids, clone_ids, }) {
+    constructor({ id, name = '', grid = [], buttons = [], parentId = null, style, locale, descriptionHtml, images, sounds, semantic_ids, clone_ids, scanningConfig, scanBlocksConfig, scanType, }) {
         this.id = id;
         this.name = name;
         if (Array.isArray(grid)) {
@@ -179,6 +191,9 @@ class AACPage {
         this.sounds = sounds;
         this.semantic_ids = semantic_ids;
         this.clone_ids = clone_ids;
+        this.scanningConfig = scanningConfig;
+        this.scanBlocksConfig = scanBlocksConfig;
+        this.scanType = scanType;
     }
     addButton(button) {
         this.buttons.push(button);

package/dist/optional/analytics/history.d.ts

@@ -1,7 +1,8 @@
 import { dotNetTicksToDate } from '../../utils/dotnetTicks';
 import { Grid3UserPath } from '../../processors/gridset/helpers';
 import { SnapUserInfo } from '../../processors/snap/helpers';
-export type HistorySource = 'Grid' | 'Snap';
+import { AACSemanticCategory, AACSemanticIntent } from '../../core/treeStructure';
+export type HistorySource = 'Grid' | 'Snap' | 'OBL' | string;
 export interface HistoryOccurrence {
     timestamp: Date;
     latitude?: number | null;
@@ -9,12 +10,22 @@ export interface HistoryOccurrence {
     modeling?: boolean;
     accessMethod?: number | null;
     pageId?: string | null;
+    buttonId?: string | null;
+    boardId?: string | null;
+    spoken?: boolean;
+    vocalization?: string;
+    imageUrl?: string;
+    actions?: any[];
+    type?: 'button' | 'action' | 'utterance' | 'note' | 'other';
+    intent?: AACSemanticIntent | string;
+    category?: AACSemanticCategory;
 }
 export interface HistoryPlatformExtras {
     label?: string;
     message?: string;
     buttonId?: string;
     contentXml?: string;
+    [key: string]: any;
 }
 export interface HistoryEntry {
     id: string;

package/dist/optional/analytics/index.d.ts

@@ -13,6 +13,8 @@ export * from './metrics/types';
 export * from './metrics/effort';
 export * from './utils/idGenerator';
 export * from './history';
+export * from './metrics/obl-types';
+export { OblUtil, OblAnonymizer } from './metrics/obl';
 export { MetricsCalculator } from './metrics/core';
 export { VocabularyAnalyzer } from './metrics/vocabulary';
 export { SentenceAnalyzer } from './metrics/sentence';

package/dist/optional/analytics/index.js

@@ -28,7 +28,7 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
     return (mod && mod.__esModule) ? mod : { "default": mod };
 };
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.ReferenceLoader = exports.ComparisonAnalyzer = exports.SentenceAnalyzer = exports.VocabularyAnalyzer = exports.MetricsCalculator = void 0;
+exports.ReferenceLoader = exports.ComparisonAnalyzer = exports.SentenceAnalyzer = exports.VocabularyAnalyzer = exports.MetricsCalculator = exports.OblAnonymizer = exports.OblUtil = void 0;
 exports.getReferenceDataPath = getReferenceDataPath;
 exports.hasReferenceData = hasReferenceData;
 const path_1 = __importDefault(require("path"));
@@ -39,6 +39,11 @@ __exportStar(require("./metrics/effort"), exports);
 __exportStar(require("./utils/idGenerator"), exports);
 // Export history functionality
 __exportStar(require("./history"), exports);
+// Export OBL logging support
+__exportStar(require("./metrics/obl-types"), exports);
+var obl_1 = require("./metrics/obl");
+Object.defineProperty(exports, "OblUtil", { enumerable: true, get: function () { return obl_1.OblUtil; } });
+Object.defineProperty(exports, "OblAnonymizer", { enumerable: true, get: function () { return obl_1.OblAnonymizer; } });
 // Export core metrics calculator
 var core_1 = require("./metrics/core");
 Object.defineProperty(exports, "MetricsCalculator", { enumerable: true, get: function () { return core_1.MetricsCalculator; } });

package/dist/optional/analytics/metrics/comparison.js

@@ -87,13 +87,17 @@ class ComparisonAnalyzer {
             }
         });
         highEffortWords.sort((a, b) => {
-            const diffA = targetWords.get(a).effort - (compareWords.get(a)?.effort || 0);
-            const diffB = targetWords.get(b).effort - (compareWords.get(b)?.effort || 0);
+            const targetBtnA = targetWords.get(a);
+            const targetBtnB = targetWords.get(b);
+            const diffA = (targetBtnA?.effort || 0) - (compareWords.get(a)?.effort || 0);
+            const diffB = (targetBtnB?.effort || 0) - (compareWords.get(b)?.effort || 0);
             return diffB - diffA;
         });
         lowEffortWords.sort((a, b) => {
-            const diffA = (compareWords.get(a)?.effort || 0) - targetWords.get(a).effort;
-            const diffB = (compareWords.get(b)?.effort || 0) - targetWords.get(b).effort;
+            const targetBtnA = targetWords.get(a);
+            const targetBtnB = targetWords.get(b);
+            const diffA = (compareWords.get(a)?.effort || 0) - (targetBtnA?.effort || 0);
+            const diffB = (compareWords.get(b)?.effort || 0) - (targetBtnB?.effort || 0);
             return diffB - diffA;
         });
         // Sentence analysis

package/dist/optional/analytics/metrics/core.d.ts

@@ -21,6 +21,11 @@ export declare class MetricsCalculator {
      * Build reference maps for semantic_id and clone_id frequencies
      */
     private buildReferenceMaps;
+    /**
+     * Count scan items for visual scanning effort
+     * When block scanning is enabled, count unique scan blocks instead of individual buttons
+     */
+    private countScanItems;
     /**
      * Analyze starting from a specific board
      */
@@ -33,4 +38,8 @@ export declare class MetricsCalculator {
      * Calculate grid dimensions from the tree
      */
     private calculateGridDimensions;
+    /**
+     * Calculate scanning steps and selections for a button based on access method
+     */
+    private calculateScanSteps;
 }