baburchi 1.0.0

package/LICENSE.md

@@ -0,0 +1,9 @@
+MIT License
+
+Copyright (c) 2025 Ragaeeb Haq
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

package/README.md

@@ -0,0 +1,262 @@
+# baburchi
+
+[![wakatime](https://wakatime.com/badge/user/a0b906ce-b8e7-4463-8bce-383238df6d4b/project/84c1b69b-fd5f-4e84-9c10-545c723a0fa9.svg)](https://wakatime.com/badge/user/a0b906ce-b8e7-4463-8bce-383238df6d4b/project/84c1b69b-fd5f-4e84-9c10-545c723a0fa9)
+![Bun](https://img.shields.io/badge/Bun-%23000000.svg?style=for-the-badge&logo=bun&logoColor=white)
+[![Node.js CI](https://github.com/ragaeeb/baburchi/actions/workflows/build.yml/badge.svg)](https://github.com/ragaeeb/baburchi/actions/workflows/build.yml)
+![GitHub License](https://img.shields.io/github/license/ragaeeb/baburchi)
+![GitHub Release](https://img.shields.io/github/v/release/ragaeeb/baburchi)
+[![codecov](https://codecov.io/gh/ragaeeb/baburchi/graph/badge.svg?token=R3BOH5KVXM)](https://codecov.io/gh/ragaeeb/baburchi)
+[![Size](https://deno.bundlejs.com/badge?q=baburchi@latest&badge=detailed)](https://bundlejs.com/?q=baburchi%40latest)
+![typescript](https://badgen.net/badge/icon/typescript?icon=typescript&label&color=blue)
+![npm](https://img.shields.io/npm/dm/baburchi)
+![GitHub issues](https://img.shields.io/github/issues/ragaeeb/baburchi)
+![GitHub stars](https://img.shields.io/github/stars/ragaeeb/baburchi?style=social)
+
+A lightweight TypeScript library for intelligent OCR text post-processing, specializing in Arabic text with advanced typo correction using sequence alignment algorithms.
+
+## Features
+
+- 🧠 **Intelligent Text Alignment**: Uses the Needleman-Wunsch algorithm for optimal text sequence alignment
+- 🔤 **Arabic Text Specialization**: Advanced normalization and diacritics handling for Arabic text
+- 📝 **Footnote Management**: Smart handling of embedded and standalone footnotes
+- ⚡ **High Performance**: Space-optimized algorithms with O(min(m,n)) space complexity
+- 🎯 **Special Symbol Preservation**: Configurable preservation of religious symbols and honorifics
+- 🔧 **Flexible Configuration**: Customizable similarity thresholds and typo symbols
+- 📦 **Zero Dependencies**: Pure TypeScript implementation with no external dependencies
+- 🌐 **Universal Compatibility**: Works in Node.js, Bun, and modern browsers
+
+## Installation
+
+```bash
+# Using npm
+npm install baburchi
+
+# Using yarn
+yarn add baburchi
+
+# Using pnpm
+pnpm add baburchi
+
+# Using bun
+bun add baburchi
+```
+
+## Quick Start
+
+```typescript
+import { fixTypo } from 'baburchi';
+
+// Basic usage with Arabic text
+const originalText = 'محمد صلى الله عليه وسلم رسول الله';
+const correctedText = 'محمد ﷺ رسول الله';
+const typoSymbols = ['ﷺ', '﷽', 'ﷻ'];
+
+const result = fixTypo(originalText, correctedText, { typoSymbols });
+console.log(result); // 'محمد صلى الله عليه ﷺ رسول الله'
+```
+
+## API Reference
+
+### `fixTypo(original, correction, options)`
+
+The main function for correcting typos using text alignment.
+
+**Parameters:**
+
+- `original` (string): The original OCR text that may contain typos
+- `correction` (string): The reference text for comparison
+- `options` (object): Configuration options
+
+**Options:**
+
+- `typoSymbols` (string[], required): Array of special symbols to preserve
+- `similarityThreshold` (number, optional): Threshold for token alignment (default: 0.6)
+- `highSimilarityThreshold` (number, optional): Threshold for duplicate detection (default: 0.8)
+
+**Returns:** Corrected text string
+
+### `processTextAlignment(originalText, altText, options)`
+
+Low-level function for advanced text processing with full configuration control.
+
+**Parameters:**
+
+- `originalText` (string): Original text to process
+- `altText` (string): Reference text for alignment
+- `options` (FixTypoOptions): Complete configuration object
+
+## Usage Examples
+
+### Basic Arabic Text Correction
+
+```typescript
+import { fixTypo } from 'baburchi';
+
+const original = 'النص الأصلي مع أخطاء إملائية';
+const reference = 'النص الأصلي مع أخطاء إملائية';
+const typoSymbols = ['ﷺ', '﷽', 'ﷻ'];
+
+const corrected = fixTypo(original, reference, { typoSymbols });
+```
+
+### Handling Religious Symbols
+
+```typescript
+import { fixTypo } from 'baburchi';
+
+// OCR might split religious phrases
+const ocrText = 'محمد صلى الله عليه وسلم خير الأنام';
+const referenceText = 'محمد ﷺ خير الأنام';
+
+const result = fixTypo(ocrText, referenceText, {
+    typoSymbols: ['ﷺ', '﷽', 'ﷻ'],
+    similarityThreshold: 0.7,
+});
+
+console.log(result); // 'محمد صلى الله عليه ﷺ خير الأنام'
+```
+
+### Custom Similarity Thresholds
+
+```typescript
+import { fixTypo } from 'baburchi';
+
+const result = fixTypo(original, reference, {
+    typoSymbols: ['ﷺ'],
+    similarityThreshold: 0.8, // Stricter alignment
+    highSimilarityThreshold: 0.95, // Very strict duplicate detection
+});
+```
+
+### Advanced Usage with Full Configuration
+
+```typescript
+import { processTextAlignment } from 'baburchi';
+
+const options = {
+    typoSymbols: ['ﷺ', '﷽', 'ﷻ'],
+    similarityThreshold: 0.7,
+    highSimilarityThreshold: 0.9,
+};
+
+const result = processTextAlignment('Original text with typos', 'Reference text for correction', options);
+```
+
+### Footnote Handling
+
+```typescript
+import { fixTypo } from 'baburchi';
+
+// Handles embedded and standalone footnotes intelligently
+const textWithFootnotes = 'النص (١) مع الحواشي (٢)أخرجه البخاري';
+const reference = 'النص (١) مع الحواشي (٢)';
+
+const corrected = fixTypo(textWithFootnotes, reference, {
+    typoSymbols: [],
+});
+// Result preserves footnote formatting
+```
+
+## Algorithm Overview
+
+Baburchi uses the **Needleman-Wunsch global sequence alignment algorithm** to optimally align text tokens:
+
+1. **Tokenization**: Text is split into tokens while preserving special symbols
+2. **Normalization**: Arabic text is normalized by removing diacritics and tatweel marks
+3. **Alignment**: Tokens are aligned using dynamic programming with custom scoring
+4. **Selection**: Best tokens are selected based on similarity and special rules
+5. **Post-processing**: Duplicates are removed and footnotes are fused
+
+### Scoring System
+
+- **Perfect Match** (+2): Identical tokens after normalization
+- **Soft Match** (+1): High similarity or contains typo symbols
+- **Mismatch** (-2): Dissimilar tokens
+- **Gap Penalty** (-1): Insertion or deletion
+
+## Performance
+
+- **Time Complexity**: O(m×n) for alignment, where m and n are token sequence lengths
+- **Space Complexity**: O(min(m,n)) using space-optimized dynamic programming
+- **Memory Efficient**: Processes text in chunks without storing large matrices
+
+## Browser Support
+
+Baburchi works in all modern environments:
+
+- ✅ Node.js 18+
+- ✅ Bun 1.0+
+- ✅ Modern browsers (ES2020+)
+- ✅ Deno (with npm compatibility)
+
+## TypeScript Support
+
+Baburchi is written in TypeScript and provides full type definitions:
+
+```typescript
+import type { FixTypoOptions } from 'baburchi';
+
+const options: FixTypoOptions = {
+    typoSymbols: ['ﷺ'],
+    similarityThreshold: 0.7,
+    highSimilarityThreshold: 0.9,
+};
+```
+
+## Utilities
+
+The library also exports utility functions for advanced use cases:
+
+```typescript
+import { calculateSimilarity, normalizeArabicText, tokenizeText, alignTokenSequences } from 'baburchi';
+
+// Calculate similarity between two strings
+const similarity = calculateSimilarity('hello', 'helo'); // 0.8
+
+// Normalize Arabic text
+const normalized = normalizeArabicText('اَلسَّلَامُ'); // 'السلام'
+
+// Tokenize with symbol preservation
+const tokens = tokenizeText('محمد ﷺ رسول', ['ﷺ']); // ['محمد', 'ﷺ', 'رسول']
+```
+
+## Contributing
+
+Contributions are welcome. Please ensure your contributions adhere to the coding standards and include relevant tests.
+
+### Development Setup
+
+1. Fork the repository
+2. Install dependencies: `bun install` (requires [Bun](https://bun.sh/))
+3. Make your changes
+4. Run tests: `bun test`
+5. Run linting: `bun run lint`
+6. Submit a pull request
+
+### Running Tests
+
+```bash
+# Run tests with coverage
+bun test --coverage
+
+# Run tests in watch mode
+bun test --watch
+```
+
+## Changelog
+
+See [CHANGELOG.md](./CHANGELOG.md) for release history.
+
+## License
+
+`baburchi` is released under the MIT License. See the [LICENSE.md](./LICENSE.md) file for more details.
+
+## Author
+
+Ragaeeb Haq
+
+- GitHub: [@ragaeeb](https://github.com/ragaeeb)
+
+---
+
+Built with ❤️ using TypeScript and Bun. Optimized for Arabic text processing and OCR post-processing.

package/dist/index.d.ts

@@ -0,0 +1,216 @@
+/**
+ * Configuration options for fixing typos in OCR text using alignment algorithms.
+ * These options control how text tokens are compared, aligned, and merged during typo correction.
+ */
+type FixTypoOptions = {
+    /**
+     * High similarity threshold (0.0 to 1.0) for detecting and removing duplicate tokens.
+     * Used in post-processing to eliminate redundant tokens that are nearly identical.
+     * Should typically be higher than similarityThreshold to catch only very similar duplicates.
+     * @default 0.9
+     * @example 0.95 // Removes tokens that are 95% or more similar
+     */
+    readonly highSimilarityThreshold: number;
+    /**
+     * Similarity threshold (0.0 to 1.0) for determining if two tokens should be aligned.
+     * Higher values require closer matches, lower values are more permissive.
+     * Used in the Needleman-Wunsch alignment algorithm for token matching.
+     * @default 0.7
+     * @example 0.8 // Requires 80% similarity for token alignment
+     */
+    readonly similarityThreshold: number;
+    /**
+     * Array of special symbols that should be preserved during typo correction.
+     * These symbols (like honorifics or religious markers) take precedence in token selection.
+     * @example ['ﷺ', '﷽', 'ﷻ'] // Common Arabic religious symbols
+     */
+    readonly typoSymbols: string[];
+};
+
+/**
+ * Calculates Levenshtein distance between two strings using space-optimized dynamic programming.
+ * The Levenshtein distance is the minimum number of single-character edits (insertions,
+ * deletions, or substitutions) required to change one string into another.
+ *
+ * @param textA - First string to compare
+ * @param textB - Second string to compare
+ * @returns Minimum edit distance between the two strings
+ * @complexity Time: O(m*n), Space: O(min(m,n)) where m,n are string lengths
+ * @example
+ * calculateLevenshteinDistance('kitten', 'sitting') // Returns 3
+ * calculateLevenshteinDistance('', 'hello') // Returns 5
+ */
+declare const calculateLevenshteinDistance: (textA: string, textB: string) => number;
+/**
+ * Calculates similarity ratio between two strings as a value between 0.0 and 1.0.
+ * Uses Levenshtein distance normalized by the length of the longer string.
+ * A ratio of 1.0 indicates identical strings, 0.0 indicates completely different strings.
+ *
+ * @param textA - First string to compare
+ * @param textB - Second string to compare
+ * @returns Similarity ratio from 0.0 (completely different) to 1.0 (identical)
+ * @example
+ * calculateSimilarity('hello', 'hello') // Returns 1.0
+ * calculateSimilarity('hello', 'help') // Returns 0.6
+ */
+declare const calculateSimilarity: (textA: string, textB: string) => number;
+/**
+ * Checks if two texts are similar after Arabic normalization.
+ * Normalizes both texts by removing diacritics and decorative elements,
+ * then compares their similarity against the provided threshold.
+ *
+ * @param textA - First text to compare
+ * @param textB - Second text to compare
+ * @param threshold - Similarity threshold (0.0 to 1.0)
+ * @returns True if normalized texts meet the similarity threshold
+ * @example
+ * areSimilarAfterNormalization('السَّلام', 'السلام', 0.9) // Returns true
+ */
+declare const areSimilarAfterNormalization: (textA: string, textB: string, threshold?: number) => boolean;
+/**
+ * Calculates alignment score for two tokens in sequence alignment.
+ * Uses different scoring criteria: perfect match after normalization gets highest score,
+ * typo symbols or highly similar tokens get soft match score, mismatches get penalty.
+ *
+ * @param tokenA - First token to score
+ * @param tokenB - Second token to score
+ * @param typoSymbols - Array of special symbols that get preferential treatment
+ * @param similarityThreshold - Threshold for considering tokens highly similar
+ * @returns Alignment score (higher is better match)
+ * @example
+ * calculateAlignmentScore('hello', 'hello', [], 0.8) // Returns 2 (perfect match)
+ * calculateAlignmentScore('hello', 'help', [], 0.8) // Returns 1 or -2 based on similarity
+ */
+declare const calculateAlignmentScore: (tokenA: string, tokenB: string, typoSymbols: string[], similarityThreshold: number) => number;
+type AlignedTokenPair = [null | string, null | string];
+type AlignmentCell = {
+    direction: 'diagonal' | 'left' | 'up' | null;
+    score: number;
+};
+/**
+ * Backtracks through the scoring matrix to reconstruct optimal sequence alignment.
+ * Follows the directional indicators in the matrix to build the sequence of aligned
+ * token pairs from the Needleman-Wunsch algorithm.
+ *
+ * @param matrix - Scoring matrix with directional information from alignment
+ * @param tokensA - First sequence of tokens
+ * @param tokensB - Second sequence of tokens
+ * @returns Array of aligned token pairs, where null indicates a gap
+ * @throws Error if invalid alignment direction is encountered
+ */
+declare const backtrackAlignment: (matrix: AlignmentCell[][], tokensA: string[], tokensB: string[]) => AlignedTokenPair[];
+/**
+ * Performs global sequence alignment using the Needleman-Wunsch algorithm.
+ * Aligns two token sequences to find the optimal pairing that maximizes
+ * the total alignment score, handling insertions, deletions, and substitutions.
+ *
+ * @param tokensA - First sequence of tokens to align
+ * @param tokensB - Second sequence of tokens to align
+ * @param typoSymbols - Special symbols that affect scoring
+ * @param similarityThreshold - Threshold for high similarity scoring
+ * @returns Array of aligned token pairs, with null indicating gaps
+ * @example
+ * alignTokenSequences(['a', 'b'], ['a', 'c'], [], 0.8)
+ * // Returns [['a', 'a'], ['b', 'c']]
+ */
+declare const alignTokenSequences: (tokensA: string[], tokensB: string[], typoSymbols: string[], similarityThreshold: number) => AlignedTokenPair[];
+
+declare const PATTERNS: {
+    arabicDigits: RegExp;
+    arabicLettersAndDigits: RegExp;
+    arabicPunctuationAndWhitespace: RegExp;
+    diacritics: RegExp;
+    footnoteEmbedded: RegExp;
+    footnoteStandalone: RegExp;
+    tatweel: RegExp;
+    whitespace: RegExp;
+};
+/**
+ * Normalizes Arabic text by removing diacritics, and tatweel marks.
+ * This normalization enables better text comparison by focusing on core characters
+ * while ignoring decorative elements that don't affect meaning.
+ *
+ * @param text - Arabic text to normalize
+ * @returns Normalized text with diacritics, tatweel, and basic tags removed
+ * @example
+ * normalizeArabicText('اَلسَّلَامُ عَلَيْكُمْ') // Returns 'السلام عليكم'
+ */
+declare const normalizeArabicText: (text: string) => string;
+/**
+ * Extracts the first sequence of Arabic or Western digits from text.
+ * Used primarily for footnote number comparison to match related footnote elements.
+ *
+ * @param text - Text containing digits to extract
+ * @returns First digit sequence found, or empty string if none found
+ * @example
+ * extractDigits('(٥)أخرجه البخاري') // Returns '٥'
+ * extractDigits('See note (123)') // Returns '123'
+ */
+declare const extractDigits: (text: string) => string;
+/**
+ * Tokenizes text into individual words while preserving special symbols.
+ * Removes HTML tags, adds spacing around preserved symbols to ensure they
+ * are tokenized separately, then splits on whitespace.
+ *
+ * @param text - Text to tokenize
+ * @param preserveSymbols - Array of symbols that should be tokenized as separate tokens
+ * @returns Array of tokens, or empty array if input is empty/whitespace
+ * @example
+ * tokenizeText('Hello ﷺ world', ['ﷺ']) // Returns ['Hello', 'ﷺ', 'world']
+ */
+declare const tokenizeText: (text: string, preserveSymbols?: string[]) => string[];
+/**
+ * Handles fusion of standalone and embedded footnotes during token processing.
+ * Detects patterns where standalone footnotes should be merged with embedded ones
+ * or where trailing standalone footnotes should be skipped.
+ *
+ * @param result - Current result array being built
+ * @param previousToken - The previous token in the sequence
+ * @param currentToken - The current token being processed
+ * @returns True if the current token was handled (fused or skipped), false otherwise
+ * @example
+ * // (٥) + (٥)أخرجه → result gets (٥)أخرجه
+ * // (٥)أخرجه + (٥) → (٥) is skipped
+ */
+declare const handleFootnoteFusion: (result: string[], previousToken: string, currentToken: string) => boolean;
+/**
+ * Handles selection logic for tokens with embedded footnotes during alignment.
+ * Prefers tokens that contain embedded footnotes over plain text, and among
+ * tokens with embedded footnotes, prefers the shorter one.
+ *
+ * @param tokenA - First token to compare
+ * @param tokenB - Second token to compare
+ * @returns Array containing selected token(s), or null if no special handling needed
+ * @example
+ * handleFootnoteSelection('text', '(١)text') // Returns ['(١)text']
+ * handleFootnoteSelection('(١)longtext', '(١)text') // Returns ['(١)text']
+ */
+declare const handleFootnoteSelection: (tokenA: string, tokenB: string) => null | string[];
+/**
+ * Handles selection logic for standalone footnote tokens during alignment.
+ * Manages cases where one or both tokens are standalone footnotes, preserving
+ * both tokens when one is a footnote and the other is regular text.
+ *
+ * @param tokenA - First token to compare
+ * @param tokenB - Second token to compare
+ * @returns Array containing selected token(s), or null if no special handling needed
+ * @example
+ * handleStandaloneFootnotes('(١)', 'text') // Returns ['(١)', 'text']
+ * handleStandaloneFootnotes('(١)', '(٢)') // Returns ['(١)'] (shorter one)
+ */
+declare const handleStandaloneFootnotes: (tokenA: string, tokenB: string) => null | string[];
+
+/**
+ * Processes text alignment between original and alternate OCR results to fix typos.
+ * Uses the Needleman-Wunsch sequence alignment algorithm to align tokens,
+ * then selects the best tokens and performs post-processing.
+ *
+ * @param originalText - Original OCR text that may contain typos
+ * @param altText - Reference text from alternate OCR for comparison
+ * @param options - Configuration options for alignment and selection
+ * @returns Corrected text with typos fixed
+ */
+declare const processTextAlignment: (originalText: string, altText: string, options: FixTypoOptions) => string;
+declare const fixTypo: (original: string, correction: string, { highSimilarityThreshold, similarityThreshold, typoSymbols, }: Partial<FixTypoOptions> & Pick<FixTypoOptions, "typoSymbols">) => string;
+
+export { PATTERNS, alignTokenSequences, areSimilarAfterNormalization, backtrackAlignment, calculateAlignmentScore, calculateLevenshteinDistance, calculateSimilarity, extractDigits, fixTypo, handleFootnoteFusion, handleFootnoteSelection, handleStandaloneFootnotes, normalizeArabicText, processTextAlignment, tokenizeText };

package/dist/index.js

@@ -0,0 +1,2 @@
+var u={arabicDigits:/[0-9\u0660-\u0669]+/,arabicLettersAndDigits:/[0-9\u0621-\u063A\u0641-\u064A\u0660-\u0669]+/g,arabicPunctuationAndWhitespace:/[\s\u060C\u061B\u061F\u06D4]+/,diacritics:/[\u0610-\u061A\u064B-\u065F\u0670\u06D6-\u06ED]/g,footnoteEmbedded:/\([0-9\u0660-\u0669]+\)/,footnoteStandalone:/^\(?[0-9\u0660-\u0669]+\)?[،.]?$/,tatweel:/\u0640/g,whitespace:/\s+/},d=t=>t.replace(u.tatweel,"").replace(u.diacritics,"").trim(),x=t=>{let n=t.match(u.arabicDigits);return n?n[0]:""},A=(t,n=[])=>{let e=t;for(let r of n){let i=new RegExp(r,"g");e=e.replace(i,` ${r} `)}return e.trim().split(u.whitespace).filter(Boolean)},T=(t,n,e)=>{let r=u.footnoteStandalone.test(n),i=u.footnoteEmbedded.test(e),s=u.footnoteStandalone.test(e),l=u.footnoteEmbedded.test(n),o=x(n),c=x(e);return r&&i&&o===c?(t[t.length-1]=e,!0):!!(l&&s&&o===c)},y=(t,n)=>{let e=u.footnoteEmbedded.test(t),r=u.footnoteEmbedded.test(n);return e&&!r?[t]:r&&!e?[n]:e&&r?[t.length<=n.length?t:n]:null},E=(t,n)=>{let e=u.footnoteStandalone.test(t),r=u.footnoteStandalone.test(n);return e&&!r?[t,n]:r&&!e?[n,t]:e&&r?[t.length<=n.length?t:n]:null};var f={GAP_PENALTY:-1,MISMATCH_PENALTY:-2,PERFECT_MATCH:2,SOFT_MATCH:1},C=(t,n)=>{let e=t.length,r=n.length;if(e===0)return r;if(r===0)return e;let[i,s]=e<=r?[t,n]:[n,t],l=i.length,o=s.length,c=Array.from({length:l+1},(a,g)=>g);for(let a=1;a<=o;a++){let g=[a];for(let m=1;m<=l;m++){let b=s[a-1]===i[m-1]?0:1,h=Math.min(c[m]+1,g[m-1]+1,c[m-1]+b);g.push(h)}c=g}return c[l]},p=(t,n)=>{let e=Math.max(t.length,n.length)||1,r=C(t,n);return(e-r)/e},P=(t,n,e=.6)=>{let r=d(t),i=d(n);return p(r,i)>=e},M=(t,n,e,r)=>{let i=d(t),s=d(n);if(i===s)return f.PERFECT_MATCH;let l=e.includes(t)||e.includes(n),o=p(i,s)>=r;return l||o?f.SOFT_MATCH:f.MISMATCH_PENALTY},z=(t,n,e)=>{let r=[],i=n.length,s=e.length;for(;i>0||s>0;)switch(t[i][s].direction){case"diagonal":r.push([n[--i],e[--s]]);break;case"left":r.push([null,e[--s]]);break;case"up":r.push([n[--i],null]);break;default:throw new Error("Invalid alignment direction")}return r.reverse()},F=(t,n,e,r)=>{let i=t.length,s=n.length,l=Array.from({length:i+1},()=>Array.from({length:s+1},()=>({direction:null,score:0})));for(let o=1;o<=i;o++)l[o][0]={direction:"up",score:o*f.GAP_PENALTY};for(let o=1;o<=s;o++)l[0][o]={direction:"left",score:o*f.GAP_PENALTY};for(let o=1;o<=i;o++)for(let c=1;c<=s;c++){let a=M(t[o-1],n[c-1],e,r),g=l[o-1][c-1].score+a,m=l[o-1][c].score+f.GAP_PENALTY,b=l[o][c-1].score+f.GAP_PENALTY,h=Math.max(g,m,b),S="left";h===g?S="diagonal":h===m&&(S="up"),l[o][c]={direction:S,score:h}}return z(l,t,n)};var L=(t,n,{similarityThreshold:e,typoSymbols:r})=>{if(t===null)return[n];if(n===null)return[t];if(d(t)===d(n))return[t];let i=y(t,n);if(i)return i;let s=E(t,n);if(s)return s;if(r.includes(t)||r.includes(n)){let a=r.find(g=>g===t||g===n);return a?[a]:[t]}let l=d(t),o=d(n);return[p(l,o)>e?t:n]},_=(t,n)=>{if(t.length===0)return t;let e=[];for(let r of t){if(e.length===0){e.push(r);continue}let i=e.at(-1);if(P(i,r,n)){r.length<i.length&&(e[e.length-1]=r);continue}T(e,i,r)||e.push(r)}return e},D=(t,n,e)=>{let r=A(t,e.typoSymbols),i=A(n,e.typoSymbols),l=F(r,i,e.typoSymbols,e.similarityThreshold).flatMap(([c,a])=>L(c,a,e));return _(l,e.highSimilarityThreshold).join(" ")},I=(t,n,{highSimilarityThreshold:e=.8,similarityThreshold:r=.6,typoSymbols:i})=>D(t,n,{highSimilarityThreshold:e,similarityThreshold:r,typoSymbols:i});export{u as PATTERNS,F as alignTokenSequences,P as areSimilarAfterNormalization,z as backtrackAlignment,M as calculateAlignmentScore,C as calculateLevenshteinDistance,p as calculateSimilarity,x as extractDigits,I as fixTypo,T as handleFootnoteFusion,y as handleFootnoteSelection,E as handleStandaloneFootnotes,d as normalizeArabicText,D as processTextAlignment,A as tokenizeText};
+//# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -0,0 +1 @@
+{"version":3,"sources":["../src/textUtils.ts","../src/similarity.ts","../src/index.ts"],"sourcesContent":["export const PATTERNS = {\n    arabicDigits: /[0-9\\u0660-\\u0669]+/,\n    arabicLettersAndDigits: /[0-9\\u0621-\\u063A\\u0641-\\u064A\\u0660-\\u0669]+/g,\n    arabicPunctuationAndWhitespace: /[\\s\\u060C\\u061B\\u061F\\u06D4]+/,\n    diacritics: /[\\u0610-\\u061A\\u064B-\\u065F\\u0670\\u06D6-\\u06ED]/g,\n    footnoteEmbedded: /\\([0-9\\u0660-\\u0669]+\\)/,\n    footnoteStandalone: /^\\(?[0-9\\u0660-\\u0669]+\\)?[،.]?$/,\n    tatweel: /\\u0640/g,\n    whitespace: /\\s+/,\n};\n\n/**\n * Normalizes Arabic text by removing diacritics, and tatweel marks.\n * This normalization enables better text comparison by focusing on core characters\n * while ignoring decorative elements that don't affect meaning.\n *\n * @param text - Arabic text to normalize\n * @returns Normalized text with diacritics, tatweel, and basic tags removed\n * @example\n * normalizeArabicText('اَلسَّلَامُ عَلَيْكُمْ') // Returns 'السلام عليكم'\n */\nexport const normalizeArabicText = (text: string): string => {\n    return text.replace(PATTERNS.tatweel, '').replace(PATTERNS.diacritics, '').trim();\n};\n\n/**\n * Extracts the first sequence of Arabic or Western digits from text.\n * Used primarily for footnote number comparison to match related footnote elements.\n *\n * @param text - Text containing digits to extract\n * @returns First digit sequence found, or empty string if none found\n * @example\n * extractDigits('(٥)أخرجه البخاري') // Returns '٥'\n * extractDigits('See note (123)') // Returns '123'\n */\nexport const extractDigits = (text: string): string => {\n    const match = text.match(PATTERNS.arabicDigits);\n    return match ? match[0] : '';\n};\n\n/**\n * Tokenizes text into individual words while preserving special symbols.\n * Removes HTML tags, adds spacing around preserved symbols to ensure they\n * are tokenized separately, then splits on whitespace.\n *\n * @param text - Text to tokenize\n * @param preserveSymbols - Array of symbols that should be tokenized as separate tokens\n * @returns Array of tokens, or empty array if input is empty/whitespace\n * @example\n * tokenizeText('Hello ﷺ world', ['ﷺ']) // Returns ['Hello', 'ﷺ', 'world']\n */\nexport const tokenizeText = (text: string, preserveSymbols: string[] = []): string[] => {\n    let processedText = text;\n\n    // Add spaces around each preserve symbol to ensure they're tokenized separately\n    for (const symbol of preserveSymbols) {\n        const symbolRegex = new RegExp(symbol, 'g');\n        processedText = processedText.replace(symbolRegex, ` ${symbol} `);\n    }\n\n    return processedText.trim().split(PATTERNS.whitespace).filter(Boolean);\n};\n\n/**\n * Handles fusion of standalone and embedded footnotes during token processing.\n * Detects patterns where standalone footnotes should be merged with embedded ones\n * or where trailing standalone footnotes should be skipped.\n *\n * @param result - Current result array being built\n * @param previousToken - The previous token in the sequence\n * @param currentToken - The current token being processed\n * @returns True if the current token was handled (fused or skipped), false otherwise\n * @example\n * // (٥) + (٥)أخرجه → result gets (٥)أخرجه\n * // (٥)أخرجه + (٥) → (٥) is skipped\n */\nexport const handleFootnoteFusion = (result: string[], previousToken: string, currentToken: string): boolean => {\n    const prevIsStandalone = PATTERNS.footnoteStandalone.test(previousToken);\n    const currHasEmbedded = PATTERNS.footnoteEmbedded.test(currentToken);\n    const currIsStandalone = PATTERNS.footnoteStandalone.test(currentToken);\n    const prevHasEmbedded = PATTERNS.footnoteEmbedded.test(previousToken);\n\n    const prevDigits = extractDigits(previousToken);\n    const currDigits = extractDigits(currentToken);\n\n    // Replace standalone with fused version: (٥) + (٥)أخرجه → (٥)أخرجه\n    if (prevIsStandalone && currHasEmbedded && prevDigits === currDigits) {\n        result[result.length - 1] = currentToken;\n        return true;\n    }\n\n    // Skip trailing standalone: (٥)أخرجه + (٥) → (٥)أخرجه\n    if (prevHasEmbedded && currIsStandalone && prevDigits === currDigits) {\n        return true;\n    }\n\n    return false;\n};\n\n/**\n * Handles selection logic for tokens with embedded footnotes during alignment.\n * Prefers tokens that contain embedded footnotes over plain text, and among\n * tokens with embedded footnotes, prefers the shorter one.\n *\n * @param tokenA - First token to compare\n * @param tokenB - Second token to compare\n * @returns Array containing selected token(s), or null if no special handling needed\n * @example\n * handleFootnoteSelection('text', '(١)text') // Returns ['(١)text']\n * handleFootnoteSelection('(١)longtext', '(١)text') // Returns ['(١)text']\n */\nexport const handleFootnoteSelection = (tokenA: string, tokenB: string): null | string[] => {\n    const aHasEmbedded = PATTERNS.footnoteEmbedded.test(tokenA);\n    const bHasEmbedded = PATTERNS.footnoteEmbedded.test(tokenB);\n\n    if (aHasEmbedded && !bHasEmbedded) return [tokenA];\n    if (bHasEmbedded && !aHasEmbedded) return [tokenB];\n    if (aHasEmbedded && bHasEmbedded) {\n        return [tokenA.length <= tokenB.length ? tokenA : tokenB];\n    }\n\n    return null;\n};\n\n/**\n * Handles selection logic for standalone footnote tokens during alignment.\n * Manages cases where one or both tokens are standalone footnotes, preserving\n * both tokens when one is a footnote and the other is regular text.\n *\n * @param tokenA - First token to compare\n * @param tokenB - Second token to compare\n * @returns Array containing selected token(s), or null if no special handling needed\n * @example\n * handleStandaloneFootnotes('(١)', 'text') // Returns ['(١)', 'text']\n * handleStandaloneFootnotes('(١)', '(٢)') // Returns ['(١)'] (shorter one)\n */\nexport const handleStandaloneFootnotes = (tokenA: string, tokenB: string): null | string[] => {\n    const aIsFootnote = PATTERNS.footnoteStandalone.test(tokenA);\n    const bIsFootnote = PATTERNS.footnoteStandalone.test(tokenB);\n\n    if (aIsFootnote && !bIsFootnote) return [tokenA, tokenB];\n    if (bIsFootnote && !aIsFootnote) return [tokenB, tokenA];\n    if (aIsFootnote && bIsFootnote) {\n        return [tokenA.length <= tokenB.length ? tokenA : tokenB];\n    }\n\n    return null;\n};\n","import { normalizeArabicText } from './textUtils';\n\n// Alignment scoring constants\nconst ALIGNMENT_SCORES = {\n    GAP_PENALTY: -1,\n    MISMATCH_PENALTY: -2,\n    PERFECT_MATCH: 2,\n    SOFT_MATCH: 1,\n};\n\n/**\n * Calculates Levenshtein distance between two strings using space-optimized dynamic programming.\n * The Levenshtein distance is the minimum number of single-character edits (insertions,\n * deletions, or substitutions) required to change one string into another.\n *\n * @param textA - First string to compare\n * @param textB - Second string to compare\n * @returns Minimum edit distance between the two strings\n * @complexity Time: O(m*n), Space: O(min(m,n)) where m,n are string lengths\n * @example\n * calculateLevenshteinDistance('kitten', 'sitting') // Returns 3\n * calculateLevenshteinDistance('', 'hello') // Returns 5\n */\nexport const calculateLevenshteinDistance = (textA: string, textB: string): number => {\n    const lengthA = textA.length;\n    const lengthB = textB.length;\n\n    if (lengthA === 0) {\n        return lengthB;\n    }\n\n    if (lengthB === 0) {\n        return lengthA;\n    }\n\n    // Use shorter string for the array to optimize space\n    const [shorter, longer] = lengthA <= lengthB ? [textA, textB] : [textB, textA];\n    const shortLen = shorter.length;\n    const longLen = longer.length;\n\n    let previousRow = Array.from({ length: shortLen + 1 }, (_, index) => index);\n\n    for (let i = 1; i <= longLen; i++) {\n        const currentRow = [i];\n\n        for (let j = 1; j <= shortLen; j++) {\n            const substitutionCost = longer[i - 1] === shorter[j - 1] ? 0 : 1;\n            const minCost = Math.min(\n                previousRow[j] + 1, // deletion\n                currentRow[j - 1] + 1, // insertion\n                previousRow[j - 1] + substitutionCost, // substitution\n            );\n            currentRow.push(minCost);\n        }\n\n        previousRow = currentRow;\n    }\n\n    return previousRow[shortLen];\n};\n\n/**\n * Calculates similarity ratio between two strings as a value between 0.0 and 1.0.\n * Uses Levenshtein distance normalized by the length of the longer string.\n * A ratio of 1.0 indicates identical strings, 0.0 indicates completely different strings.\n *\n * @param textA - First string to compare\n * @param textB - Second string to compare\n * @returns Similarity ratio from 0.0 (completely different) to 1.0 (identical)\n * @example\n * calculateSimilarity('hello', 'hello') // Returns 1.0\n * calculateSimilarity('hello', 'help') // Returns 0.6\n */\nexport const calculateSimilarity = (textA: string, textB: string): number => {\n    const maxLength = Math.max(textA.length, textB.length) || 1;\n    const distance = calculateLevenshteinDistance(textA, textB);\n    return (maxLength - distance) / maxLength;\n};\n\n/**\n * Checks if two texts are similar after Arabic normalization.\n * Normalizes both texts by removing diacritics and decorative elements,\n * then compares their similarity against the provided threshold.\n *\n * @param textA - First text to compare\n * @param textB - Second text to compare\n * @param threshold - Similarity threshold (0.0 to 1.0)\n * @returns True if normalized texts meet the similarity threshold\n * @example\n * areSimilarAfterNormalization('السَّلام', 'السلام', 0.9) // Returns true\n */\nexport const areSimilarAfterNormalization = (textA: string, textB: string, threshold: number = 0.6): boolean => {\n    const normalizedA = normalizeArabicText(textA);\n    const normalizedB = normalizeArabicText(textB);\n    return calculateSimilarity(normalizedA, normalizedB) >= threshold;\n};\n\n/**\n * Calculates alignment score for two tokens in sequence alignment.\n * Uses different scoring criteria: perfect match after normalization gets highest score,\n * typo symbols or highly similar tokens get soft match score, mismatches get penalty.\n *\n * @param tokenA - First token to score\n * @param tokenB - Second token to score\n * @param typoSymbols - Array of special symbols that get preferential treatment\n * @param similarityThreshold - Threshold for considering tokens highly similar\n * @returns Alignment score (higher is better match)\n * @example\n * calculateAlignmentScore('hello', 'hello', [], 0.8) // Returns 2 (perfect match)\n * calculateAlignmentScore('hello', 'help', [], 0.8) // Returns 1 or -2 based on similarity\n */\nexport const calculateAlignmentScore = (\n    tokenA: string,\n    tokenB: string,\n    typoSymbols: string[],\n    similarityThreshold: number,\n): number => {\n    const normalizedA = normalizeArabicText(tokenA);\n    const normalizedB = normalizeArabicText(tokenB);\n\n    // Perfect match after normalization\n    if (normalizedA === normalizedB) {\n        return ALIGNMENT_SCORES.PERFECT_MATCH;\n    }\n\n    // Check if either token is a typo symbol or high similarity\n    const isTypoSymbol = typoSymbols.includes(tokenA) || typoSymbols.includes(tokenB);\n    const isHighlySimilar = calculateSimilarity(normalizedA, normalizedB) >= similarityThreshold;\n\n    if (isTypoSymbol || isHighlySimilar) {\n        return ALIGNMENT_SCORES.SOFT_MATCH;\n    }\n\n    return ALIGNMENT_SCORES.MISMATCH_PENALTY;\n};\n\ntype AlignedTokenPair = [null | string, null | string];\n\ntype AlignmentCell = {\n    direction: 'diagonal' | 'left' | 'up' | null;\n    score: number;\n};\n\n/**\n * Backtracks through the scoring matrix to reconstruct optimal sequence alignment.\n * Follows the directional indicators in the matrix to build the sequence of aligned\n * token pairs from the Needleman-Wunsch algorithm.\n *\n * @param matrix - Scoring matrix with directional information from alignment\n * @param tokensA - First sequence of tokens\n * @param tokensB - Second sequence of tokens\n * @returns Array of aligned token pairs, where null indicates a gap\n * @throws Error if invalid alignment direction is encountered\n */\nexport const backtrackAlignment = (\n    matrix: AlignmentCell[][],\n    tokensA: string[],\n    tokensB: string[],\n): AlignedTokenPair[] => {\n    const alignment: AlignedTokenPair[] = [];\n    let i = tokensA.length;\n    let j = tokensB.length;\n\n    while (i > 0 || j > 0) {\n        const currentCell = matrix[i][j];\n\n        switch (currentCell.direction) {\n            case 'diagonal':\n                alignment.push([tokensA[--i], tokensB[--j]]);\n                break;\n            case 'left':\n                alignment.push([null, tokensB[--j]]);\n                break;\n            case 'up':\n                alignment.push([tokensA[--i], null]);\n                break;\n            default:\n                throw new Error('Invalid alignment direction');\n        }\n    }\n\n    return alignment.reverse();\n};\n\n/**\n * Performs global sequence alignment using the Needleman-Wunsch algorithm.\n * Aligns two token sequences to find the optimal pairing that maximizes\n * the total alignment score, handling insertions, deletions, and substitutions.\n *\n * @param tokensA - First sequence of tokens to align\n * @param tokensB - Second sequence of tokens to align\n * @param typoSymbols - Special symbols that affect scoring\n * @param similarityThreshold - Threshold for high similarity scoring\n * @returns Array of aligned token pairs, with null indicating gaps\n * @example\n * alignTokenSequences(['a', 'b'], ['a', 'c'], [], 0.8)\n * // Returns [['a', 'a'], ['b', 'c']]\n */\nexport const alignTokenSequences = (\n    tokensA: string[],\n    tokensB: string[],\n    typoSymbols: string[],\n    similarityThreshold: number,\n): AlignedTokenPair[] => {\n    const lengthA = tokensA.length;\n    const lengthB = tokensB.length;\n\n    // Initialize scoring matrix\n    const scoringMatrix: AlignmentCell[][] = Array.from({ length: lengthA + 1 }, () =>\n        Array.from({ length: lengthB + 1 }, () => ({ direction: null, score: 0 })),\n    );\n\n    // Initialize first row and column\n    for (let i = 1; i <= lengthA; i++) {\n        scoringMatrix[i][0] = { direction: 'up', score: i * ALIGNMENT_SCORES.GAP_PENALTY };\n    }\n    for (let j = 1; j <= lengthB; j++) {\n        scoringMatrix[0][j] = { direction: 'left', score: j * ALIGNMENT_SCORES.GAP_PENALTY };\n    }\n\n    // Fill scoring matrix\n    for (let i = 1; i <= lengthA; i++) {\n        for (let j = 1; j <= lengthB; j++) {\n            const alignmentScore = calculateAlignmentScore(\n                tokensA[i - 1],\n                tokensB[j - 1],\n                typoSymbols,\n                similarityThreshold,\n            );\n\n            const diagonalScore = scoringMatrix[i - 1][j - 1].score + alignmentScore;\n            const upScore = scoringMatrix[i - 1][j].score + ALIGNMENT_SCORES.GAP_PENALTY;\n            const leftScore = scoringMatrix[i][j - 1].score + ALIGNMENT_SCORES.GAP_PENALTY;\n\n            const bestScore = Math.max(diagonalScore, upScore, leftScore);\n            let bestDirection: 'diagonal' | 'left' | 'up' = 'left';\n\n            if (bestScore === diagonalScore) {\n                bestDirection = 'diagonal';\n            } else if (bestScore === upScore) {\n                bestDirection = 'up';\n            }\n\n            scoringMatrix[i][j] = { direction: bestDirection, score: bestScore };\n        }\n    }\n\n    // Backtrack to build alignment\n    return backtrackAlignment(scoringMatrix, tokensA, tokensB);\n};\n","import type { FixTypoOptions } from './types';\n\nimport { alignTokenSequences, areSimilarAfterNormalization, calculateSimilarity } from './similarity';\nimport {\n    handleFootnoteFusion,\n    handleFootnoteSelection,\n    handleStandaloneFootnotes,\n    normalizeArabicText,\n    tokenizeText,\n} from './textUtils';\n\n/**\n * Selects the best token(s) from an aligned pair during typo correction.\n * Uses various heuristics including normalization, footnote handling, typo symbols,\n * and similarity scores to determine which token(s) to keep.\n *\n * @param originalToken - Token from the original OCR text (may be null)\n * @param altToken - Token from the alternative OCR text (may be null)\n * @param options - Configuration options including typo symbols and similarity threshold\n * @returns Array of selected tokens (usually contains one token, but may contain multiple)\n */\nconst selectBestTokens = (\n    originalToken: null | string,\n    altToken: null | string,\n    { similarityThreshold, typoSymbols }: FixTypoOptions,\n): string[] => {\n    // Handle missing tokens\n    if (originalToken === null) {\n        return [altToken!];\n    }\n    if (altToken === null) {\n        return [originalToken];\n    }\n\n    // Preserve original if same after normalization (keeps diacritics)\n    if (normalizeArabicText(originalToken) === normalizeArabicText(altToken)) {\n        return [originalToken];\n    }\n\n    // Handle embedded footnotes\n    const result = handleFootnoteSelection(originalToken, altToken);\n    if (result) return result;\n\n    // Handle standalone footnotes\n    const footnoteResult = handleStandaloneFootnotes(originalToken, altToken);\n    if (footnoteResult) return footnoteResult;\n\n    // Handle typo symbols - prefer the symbol itself\n    if (typoSymbols.includes(originalToken) || typoSymbols.includes(altToken)) {\n        const typoSymbol = typoSymbols.find((symbol) => symbol === originalToken || symbol === altToken);\n        return typoSymbol ? [typoSymbol] : [originalToken];\n    }\n\n    // Choose based on similarity\n    const normalizedOriginal = normalizeArabicText(originalToken);\n    const normalizedAlt = normalizeArabicText(altToken);\n    const similarity = calculateSimilarity(normalizedOriginal, normalizedAlt);\n\n    return [similarity > similarityThreshold ? originalToken : altToken];\n};\n\n/**\n * Removes duplicate tokens and handles footnote fusion in post-processing.\n * Identifies and removes tokens that are highly similar while preserving\n * important variations. Also handles special cases like footnote merging.\n *\n * @param tokens - Array of tokens to process\n * @param highSimilarityThreshold - Threshold for detecting duplicates (0.0 to 1.0)\n * @returns Array of tokens with duplicates removed and footnotes fused\n */\nconst removeDuplicateTokens = (tokens: string[], highSimilarityThreshold: number): string[] => {\n    if (tokens.length === 0) {\n        return tokens;\n    }\n\n    const result: string[] = [];\n\n    for (const currentToken of tokens) {\n        if (result.length === 0) {\n            result.push(currentToken);\n            continue;\n        }\n\n        const previousToken = result.at(-1)!;\n\n        // Handle ordinary echoes (similar tokens)\n        if (areSimilarAfterNormalization(previousToken, currentToken, highSimilarityThreshold)) {\n            // Keep the shorter version\n            if (currentToken.length < previousToken.length) {\n                result[result.length - 1] = currentToken;\n            }\n            continue;\n        }\n\n        // Handle footnote fusion cases\n        if (handleFootnoteFusion(result, previousToken, currentToken)) {\n            continue;\n        }\n\n        result.push(currentToken);\n    }\n\n    return result;\n};\n\n/**\n * Processes text alignment between original and alternate OCR results to fix typos.\n * Uses the Needleman-Wunsch sequence alignment algorithm to align tokens,\n * then selects the best tokens and performs post-processing.\n *\n * @param originalText - Original OCR text that may contain typos\n * @param altText - Reference text from alternate OCR for comparison\n * @param options - Configuration options for alignment and selection\n * @returns Corrected text with typos fixed\n */\nexport const processTextAlignment = (originalText: string, altText: string, options: FixTypoOptions): string => {\n    const originalTokens = tokenizeText(originalText, options.typoSymbols);\n    const altTokens = tokenizeText(altText, options.typoSymbols);\n\n    // Align token sequences\n    const alignedPairs = alignTokenSequences(\n        originalTokens,\n        altTokens,\n        options.typoSymbols,\n        options.similarityThreshold,\n    );\n\n    // Select best tokens from each aligned pair\n    const mergedTokens = alignedPairs.flatMap(([original, alt]) => selectBestTokens(original, alt, options));\n\n    // Remove duplicates and handle post-processing\n    const finalTokens = removeDuplicateTokens(mergedTokens, options.highSimilarityThreshold);\n\n    return finalTokens.join(' ');\n};\n\nexport const fixTypo = (\n    original: string,\n    correction: string,\n    {\n        highSimilarityThreshold = 0.8,\n        similarityThreshold = 0.6,\n        typoSymbols,\n    }: Partial<FixTypoOptions> & Pick<FixTypoOptions, 'typoSymbols'>,\n) => {\n    return processTextAlignment(original, correction, { highSimilarityThreshold, similarityThreshold, typoSymbols });\n};\n\nexport * from './similarity';\nexport * from './textUtils';\n"],"mappings":"AAAO,IAAMA,EAAW,CACpB,aAAc,sBACd,uBAAwB,iDACxB,+BAAgC,gCAChC,WAAY,mDACZ,iBAAkB,0BAClB,mBAAoB,mCACpB,QAAS,UACT,WAAY,KAChB,EAYaC,EAAuBC,GACzBA,EAAK,QAAQF,EAAS,QAAS,EAAE,EAAE,QAAQA,EAAS,WAAY,EAAE,EAAE,KAAK,EAavEG,EAAiBD,GAAyB,CACnD,IAAME,EAAQF,EAAK,MAAMF,EAAS,YAAY,EAC9C,OAAOI,EAAQA,EAAM,CAAC,EAAI,EAC9B,EAaaC,EAAe,CAACH,EAAcI,EAA4B,CAAC,IAAgB,CACpF,IAAIC,EAAgBL,EAGpB,QAAWM,KAAUF,EAAiB,CAClC,IAAMG,EAAc,IAAI,OAAOD,EAAQ,GAAG,EAC1CD,EAAgBA,EAAc,QAAQE,EAAa,IAAID,CAAM,GAAG,CACpE,CAEA,OAAOD,EAAc,KAAK,EAAE,MAAMP,EAAS,UAAU,EAAE,OAAO,OAAO,CACzE,EAeaU,EAAuB,CAACC,EAAkBC,EAAuBC,IAAkC,CAC5G,IAAMC,EAAmBd,EAAS,mBAAmB,KAAKY,CAAa,EACjEG,EAAkBf,EAAS,iBAAiB,KAAKa,CAAY,EAC7DG,EAAmBhB,EAAS,mBAAmB,KAAKa,CAAY,EAChEI,EAAkBjB,EAAS,iBAAiB,KAAKY,CAAa,EAE9DM,EAAaf,EAAcS,CAAa,EACxCO,EAAahB,EAAcU,CAAY,EAG7C,OAAIC,GAAoBC,GAAmBG,IAAeC,GACtDR,EAAOA,EAAO,OAAS,CAAC,EAAIE,EACrB,IAIP,GAAAI,GAAmBD,GAAoBE,IAAeC,EAK9D,EAcaC,EAA0B,CAACC,EAAgBC,IAAoC,CACxF,IAAMC,EAAevB,EAAS,iBAAiB,KAAKqB,CAAM,EACpDG,EAAexB,EAAS,iBAAiB,KAAKsB,CAAM,EAE1D,OAAIC,GAAgB,CAACC,EAAqB,CAACH,CAAM,EAC7CG,GAAgB,CAACD,EAAqB,CAACD,CAAM,EAC7CC,GAAgBC,EACT,CAACH,EAAO,QAAUC,EAAO,OAASD,EAASC,CAAM,EAGrD,IACX,EAcaG,EAA4B,CAACJ,EAAgBC,IAAoC,CAC1F,IAAMI,EAAc1B,EAAS,mBAAmB,KAAKqB,CAAM,EACrDM,EAAc3B,EAAS,mBAAmB,KAAKsB,CAAM,EAE3D,OAAII,GAAe,CAACC,EAAoB,CAACN,EAAQC,CAAM,EACnDK,GAAe,CAACD,EAAoB,CAACJ,EAAQD,CAAM,EACnDK,GAAeC,EACR,CAACN,EAAO,QAAUC,EAAO,OAASD,EAASC,CAAM,EAGrD,IACX,EChJA,IAAMM,EAAmB,CACrB,YAAa,GACb,iBAAkB,GAClB,cAAe,EACf,WAAY,CAChB,EAeaC,EAA+B,CAACC,EAAeC,IAA0B,CAClF,IAAMC,EAAUF,EAAM,OAChBG,EAAUF,EAAM,OAEtB,GAAIC,IAAY,EACZ,OAAOC,EAGX,GAAIA,IAAY,EACZ,OAAOD,EAIX,GAAM,CAACE,EAASC,CAAM,EAAIH,GAAWC,EAAU,CAACH,EAAOC,CAAK,EAAI,CAACA,EAAOD,CAAK,EACvEM,EAAWF,EAAQ,OACnBG,EAAUF,EAAO,OAEnBG,EAAc,MAAM,KAAK,CAAE,OAAQF,EAAW,CAAE,EAAG,CAACG,EAAGC,IAAUA,CAAK,EAE1E,QAASC,EAAI,EAAGA,GAAKJ,EAASI,IAAK,CAC/B,IAAMC,EAAa,CAACD,CAAC,EAErB,QAASE,EAAI,EAAGA,GAAKP,EAAUO,IAAK,CAChC,IAAMC,EAAmBT,EAAOM,EAAI,CAAC,IAAMP,EAAQS,EAAI,CAAC,EAAI,EAAI,EAC1DE,EAAU,KAAK,IACjBP,EAAYK,CAAC,EAAI,EACjBD,EAAWC,EAAI,CAAC,EAAI,EACpBL,EAAYK,EAAI,CAAC,EAAIC,CACzB,EACAF,EAAW,KAAKG,CAAO,CAC3B,CAEAP,EAAcI,CAClB,CAEA,OAAOJ,EAAYF,CAAQ,CAC/B,EAcaU,EAAsB,CAAChB,EAAeC,IAA0B,CACzE,IAAMgB,EAAY,KAAK,IAAIjB,EAAM,OAAQC,EAAM,MAAM,GAAK,EACpDiB,EAAWnB,EAA6BC,EAAOC,CAAK,EAC1D,OAAQgB,EAAYC,GAAYD,CACpC,EAcaE,EAA+B,CAACnB,EAAeC,EAAemB,EAAoB,KAAiB,CAC5G,IAAMC,EAAcC,EAAoBtB,CAAK,EACvCuB,EAAcD,EAAoBrB,CAAK,EAC7C,OAAOe,EAAoBK,EAAaE,CAAW,GAAKH,CAC5D,EAgBaI,EAA0B,CACnCC,EACAC,EACAC,EACAC,IACS,CACT,IAAMP,EAAcC,EAAoBG,CAAM,EACxCF,EAAcD,EAAoBI,CAAM,EAG9C,GAAIL,IAAgBE,EAChB,OAAOzB,EAAiB,cAI5B,IAAM+B,EAAeF,EAAY,SAASF,CAAM,GAAKE,EAAY,SAASD,CAAM,EAC1EI,EAAkBd,EAAoBK,EAAaE,CAAW,GAAKK,EAEzE,OAAIC,GAAgBC,EACThC,EAAiB,WAGrBA,EAAiB,gBAC5B,EAoBaiC,EAAqB,CAC9BC,EACAC,EACAC,IACqB,CACrB,IAAMC,EAAgC,CAAC,EACnC,EAAIF,EAAQ,OACZpB,EAAIqB,EAAQ,OAEhB,KAAO,EAAI,GAAKrB,EAAI,GAGhB,OAFoBmB,EAAO,CAAC,EAAEnB,CAAC,EAEX,UAAW,CAC3B,IAAK,WACDsB,EAAU,KAAK,CAACF,EAAQ,EAAE,CAAC,EAAGC,EAAQ,EAAErB,CAAC,CAAC,CAAC,EAC3C,MACJ,IAAK,OACDsB,EAAU,KAAK,CAAC,KAAMD,EAAQ,EAAErB,CAAC,CAAC,CAAC,EACnC,MACJ,IAAK,KACDsB,EAAU,KAAK,CAACF,EAAQ,EAAE,CAAC,EAAG,IAAI,CAAC,EACnC,MACJ,QACI,MAAM,IAAI,MAAM,6BAA6B,CACrD,CAGJ,OAAOE,EAAU,QAAQ,CAC7B,EAgBaC,EAAsB,CAC/BH,EACAC,EACAP,EACAC,IACqB,CACrB,IAAM1B,EAAU+B,EAAQ,OAClB9B,EAAU+B,EAAQ,OAGlBG,EAAmC,MAAM,KAAK,CAAE,OAAQnC,EAAU,CAAE,EAAG,IACzE,MAAM,KAAK,CAAE,OAAQC,EAAU,CAAE,EAAG,KAAO,CAAE,UAAW,KAAM,MAAO,CAAE,EAAE,CAC7E,EAGA,QAASQ,EAAI,EAAGA,GAAKT,EAASS,IAC1B0B,EAAc1B,CAAC,EAAE,CAAC,EAAI,CAAE,UAAW,KAAM,MAAOA,EAAIb,EAAiB,WAAY,EAErF,QAASe,EAAI,EAAGA,GAAKV,EAASU,IAC1BwB,EAAc,CAAC,EAAExB,CAAC,EAAI,CAAE,UAAW,OAAQ,MAAOA,EAAIf,EAAiB,WAAY,EAIvF,QAASa,EAAI,EAAGA,GAAKT,EAASS,IAC1B,QAASE,EAAI,EAAGA,GAAKV,EAASU,IAAK,CAC/B,IAAMyB,EAAiBd,EACnBS,EAAQtB,EAAI,CAAC,EACbuB,EAAQrB,EAAI,CAAC,EACbc,EACAC,CACJ,EAEMW,EAAgBF,EAAc1B,EAAI,CAAC,EAAEE,EAAI,CAAC,EAAE,MAAQyB,EACpDE,EAAUH,EAAc1B,EAAI,CAAC,EAAEE,CAAC,EAAE,MAAQf,EAAiB,YAC3D2C,EAAYJ,EAAc1B,CAAC,EAAEE,EAAI,CAAC,EAAE,MAAQf,EAAiB,YAE7D4C,EAAY,KAAK,IAAIH,EAAeC,EAASC,CAAS,EACxDE,EAA4C,OAE5CD,IAAcH,EACdI,EAAgB,WACTD,IAAcF,IACrBG,EAAgB,MAGpBN,EAAc1B,CAAC,EAAEE,CAAC,EAAI,CAAE,UAAW8B,EAAe,MAAOD,CAAU,CACvE,CAIJ,OAAOX,EAAmBM,EAAeJ,EAASC,CAAO,CAC7D,ECpOA,IAAMU,EAAmB,CACrBC,EACAC,EACA,CAAE,oBAAAC,EAAqB,YAAAC,CAAY,IACxB,CAEX,GAAIH,IAAkB,KAClB,MAAO,CAACC,CAAS,EAErB,GAAIA,IAAa,KACb,MAAO,CAACD,CAAa,EAIzB,GAAII,EAAoBJ,CAAa,IAAMI,EAAoBH,CAAQ,EACnE,MAAO,CAACD,CAAa,EAIzB,IAAMK,EAASC,EAAwBN,EAAeC,CAAQ,EAC9D,GAAII,EAAQ,OAAOA,EAGnB,IAAME,EAAiBC,EAA0BR,EAAeC,CAAQ,EACxE,GAAIM,EAAgB,OAAOA,EAG3B,GAAIJ,EAAY,SAASH,CAAa,GAAKG,EAAY,SAASF,CAAQ,EAAG,CACvE,IAAMQ,EAAaN,EAAY,KAAMO,GAAWA,IAAWV,GAAiBU,IAAWT,CAAQ,EAC/F,OAAOQ,EAAa,CAACA,CAAU,EAAI,CAACT,CAAa,CACrD,CAGA,IAAMW,EAAqBP,EAAoBJ,CAAa,EACtDY,EAAgBR,EAAoBH,CAAQ,EAGlD,MAAO,CAFYY,EAAoBF,EAAoBC,CAAa,EAEnDV,EAAsBF,EAAgBC,CAAQ,CACvE,EAWMa,EAAwB,CAACC,EAAkBC,IAA8C,CAC3F,GAAID,EAAO,SAAW,EAClB,OAAOA,EAGX,IAAMV,EAAmB,CAAC,EAE1B,QAAWY,KAAgBF,EAAQ,CAC/B,GAAIV,EAAO,SAAW,EAAG,CACrBA,EAAO,KAAKY,CAAY,EACxB,QACJ,CAEA,IAAMC,EAAgBb,EAAO,GAAG,EAAE,EAGlC,GAAIc,EAA6BD,EAAeD,EAAcD,CAAuB,EAAG,CAEhFC,EAAa,OAASC,EAAc,SACpCb,EAAOA,EAAO,OAAS,CAAC,EAAIY,GAEhC,QACJ,CAGIG,EAAqBf,EAAQa,EAAeD,CAAY,GAI5DZ,EAAO,KAAKY,CAAY,CAC5B,CAEA,OAAOZ,CACX,EAYagB,EAAuB,CAACC,EAAsBC,EAAiBC,IAAoC,CAC5G,IAAMC,EAAiBC,EAAaJ,EAAcE,EAAQ,WAAW,EAC/DG,EAAYD,EAAaH,EAASC,EAAQ,WAAW,EAWrDI,EAReC,EACjBJ,EACAE,EACAH,EAAQ,YACRA,EAAQ,mBACZ,EAGkC,QAAQ,CAAC,CAACM,EAAUC,CAAG,IAAMhC,EAAiB+B,EAAUC,EAAKP,CAAO,CAAC,EAKvG,OAFoBV,EAAsBc,EAAcJ,EAAQ,uBAAuB,EAEpE,KAAK,GAAG,CAC/B,EAEaQ,EAAU,CACnBF,EACAG,EACA,CACI,wBAAAjB,EAA0B,GAC1B,oBAAAd,EAAsB,GACtB,YAAAC,CACJ,IAEOkB,EAAqBS,EAAUG,EAAY,CAAE,wBAAAjB,EAAyB,oBAAAd,EAAqB,YAAAC,CAAY,CAAC","names":["PATTERNS","normalizeArabicText","text","extractDigits","match","tokenizeText","preserveSymbols","processedText","symbol","symbolRegex","handleFootnoteFusion","result","previousToken","currentToken","prevIsStandalone","currHasEmbedded","currIsStandalone","prevHasEmbedded","prevDigits","currDigits","handleFootnoteSelection","tokenA","tokenB","aHasEmbedded","bHasEmbedded","handleStandaloneFootnotes","aIsFootnote","bIsFootnote","ALIGNMENT_SCORES","calculateLevenshteinDistance","textA","textB","lengthA","lengthB","shorter","longer","shortLen","longLen","previousRow","_","index","i","currentRow","j","substitutionCost","minCost","calculateSimilarity","maxLength","distance","areSimilarAfterNormalization","threshold","normalizedA","normalizeArabicText","normalizedB","calculateAlignmentScore","tokenA","tokenB","typoSymbols","similarityThreshold","isTypoSymbol","isHighlySimilar","backtrackAlignment","matrix","tokensA","tokensB","alignment","alignTokenSequences","scoringMatrix","alignmentScore","diagonalScore","upScore","leftScore","bestScore","bestDirection","selectBestTokens","originalToken","altToken","similarityThreshold","typoSymbols","normalizeArabicText","result","handleFootnoteSelection","footnoteResult","handleStandaloneFootnotes","typoSymbol","symbol","normalizedOriginal","normalizedAlt","calculateSimilarity","removeDuplicateTokens","tokens","highSimilarityThreshold","currentToken","previousToken","areSimilarAfterNormalization","handleFootnoteFusion","processTextAlignment","originalText","altText","options","originalTokens","tokenizeText","altTokens","mergedTokens","alignTokenSequences","original","alt","fixTypo","correction"]}

package/package.json

@@ -0,0 +1,65 @@
+{
+    "name": "baburchi",
+    "version": "1.0.0",
+    "author": "Ragaeeb Haq",
+    "repository": {
+        "type": "git",
+        "url": "https://github.com/ragaeeb/baburchi.git"
+    },
+    "main": "dist/index.js",
+    "module": "dist/index.ts",
+    "devDependencies": {
+        "@eslint/js": "^9.29.0",
+        "@types/bun": "^1.2.16",
+        "eslint": "^9.29.0",
+        "eslint-config-prettier": "^10.1.5",
+        "eslint-plugin-perfectionist": "^4.15.0",
+        "eslint-plugin-prettier": "^5.5.0",
+        "globals": "^16.2.0",
+        "prettier": "^3.5.3",
+        "semantic-release": "^24.2.5",
+        "tsup": "^8.5.0",
+        "typescript-eslint": "^8.34.1"
+    },
+    "bugs": {
+        "url": "https://github.com/ragaeeb/baburchi/issues"
+    },
+    "description": "A lightweight TypeScript library designed to fix typos in OCR post-processing.",
+    "engines": {
+        "bun": ">=1.2.16",
+        "node": ">=22.0.0"
+    },
+    "exports": {
+        ".": {
+            "import": "./dist/index.js",
+            "types": "./dist/index.d.ts"
+        }
+    },
+    "sideEffects": false,
+    "files": [
+        "dist/index.js",
+        "dist/index.js.map",
+        "dist/*.d.ts",
+        "LICENSE.md",
+        "README.md"
+    ],
+    "homepage": "https://github.com/ragaeeb/baburchi",
+    "keywords": [
+        "nodejs",
+        "ocr",
+        "formatting",
+        "typos",
+        "correction",
+        "paragraphs",
+        "text-processing",
+        "typescript"
+    ],
+    "license": "MIT",
+    "scripts": {
+        "build": "tsup",
+        "test": "bun test --coverage --coverage-reporter=lcov"
+    },
+    "source": "src/index.ts",
+    "type": "module",
+    "types": "dist/index.d.ts"
+}