iraqi-dialect-bad-words 1.1.0 → 1.2.1

package/README.md

@@ -64,6 +64,7 @@ Creates a filter with the built-in word list.
 ```ts
 new IraqiBadWordsFilter();                               // profanity only
 new IraqiBadWordsFilter({ includeInsults: true });       // also milder insults
+new IraqiBadWordsFilter({ stripSuffixes: true });        // also unlisted suffix forms
 new IraqiBadWordsFilter({ customWords: ['word1', 'word2'] });
 new IraqiBadWordsFilter(['word1', 'word2']);             // same as customWords
 ```
@@ -72,6 +73,7 @@ new IraqiBadWordsFilter(['word1', 'word2']);             // same as customWords
 |---|---|---|
 | `customWords` | `[]` | Extra words to filter. |
 | `includeInsults` | `false` | Also filter milder insults like كلب, حمار, قندرة, نعال. Off by default because many of them are everyday words, so turning it on flags sentences like `عندي كلب` ("I have a dog"). |
+| `stripSuffixes` | `false` | Also match words with a pronoun suffix (ي ك ج ه ها نا كم هم) that the list doesn't spell out, e.g. `طيزهم` → `طيز`. Off by default because it widens matching: a clean word that ends like a bad word plus a suffix gets flagged too. |
 
 ### `isProfane(text: string): boolean`
 
@@ -81,6 +83,36 @@ Returns `true` if any word in the text is on the list.
 
 Replaces each bad word with the placeholder, repeated to the word's length. Spaces and punctuation are kept.
 
+### `inspect(text: string, options?): ProfanityMatch[]`
+
+Lists each bad word in the text and explains why it matched. Use it to highlight words, to treat lists differently (e.g. block `words` but only warn on `insults`), or to let someone review a match before your app blocks it.
+
+```ts
+filter.inspect('روح للكواد!');
+// [{ token: 'للكواد', start: 4, end: 10, normalized: 'لكواد',
+//    entry: 'كواد', source: 'words', prefix: 'ل' }]
+```
+
+| Field | Description |
+|---|---|
+| `token` | The word exactly as written. |
+| `start`, `end` | Offsets in the **original** text, so `text.slice(start, end) === token`. Counted in JavaScript string units (UTF-16); convert them if another platform counts characters differently. |
+| `normalized` | The word after normalization (Arabizi is folded too, so `Qa7ba` shows as `gahba`). |
+| `entry` | The list entry it matched, spelled as in the list. |
+| `source` | `'words'`, `'insults'` or `'custom'`. A word on more than one list keeps the first: built-in, then insults, then custom. |
+| `prefix` | The attached prefix stripped before matching (after normalization, so `لل` shows as `ل`), or `''` for a direct match. |
+| `suffix` | The suffix stripped before matching (only with `stripSuffixes`), or `''`. |
+| `suppressedBy` | Only with `includeSuppressed`: the safe stem that cleared this match. |
+
+Pass `{ includeSuppressed: true }` to also see words that matched but were cleared by a safe stem. They are not profane (`isProfane` and `clean` ignore them), but they show why a word *wasn't* flagged:
+
+```ts
+filter.inspect('وبعيرك', { includeSuppressed: true });
+// [{ token: 'وبعيرك', …, entry: 'عيرك', prefix: 'وب', suppressedBy: 'بعير' }]
+```
+
+Including the output of `inspect` in a [“Clean word flagged” issue](https://github.com/karam-fikre/iraqi-dialect-bad-words/issues/new?template=false-positive.yml) shows exactly which entry and prefix rule fired.
+
 ### `addWords(words: string[]): void`
 
 Adds words to this filter.
@@ -99,9 +131,13 @@ Matching is done per whole word, after normalizing both the text and the word li
 | Stretched letters | `عاااارة` |
 | Tatweel | `عـــارة` |
 | ة / ه, ى / ي, أ / إ / آ / ا | `عاره` |
+| Persian/Kurdish keyboard letters ی ک ە | `عیر`, `کس` |
+| Invisible characters hidden inside a word (zero-width, direction marks, soft hyphen) | `عا‌رة` |
 | Attached prefixes و ف ب ل ك ال | `والعارة`, `للعارة` |
+| Pronoun suffixes (with `stripSuffixes`) | `طيزهم`, `وطيزكم` |
 | Punctuation next to the word | `عارة!` `عارة،` |
-| Arabizi (Latin letters and numbers) | `ga7ba`, `3ayri` |
+| Digits glued to or inside an Arabic word | `عارة3`, `عا٣رة` |
+| Arabizi (Latin letters and numbers, Western or Arabic-Indic), including spelling variants | `ga7ba` = `ga٧ba` = `qahba` = `gahba`, `kosomak` = `kusumak`, `3ayri` = `ayri` |
 
 Because matching is whole-word, a clean word that merely contains a bad word is not flagged.
 
@@ -119,6 +155,8 @@ The word list is the heart of this package, and native speakers are the best jud
 - **A clean word is flagged:** open a [“Clean word flagged” issue](https://github.com/karam-fikre/iraqi-dialect-bad-words/issues/new?template=false-positive.yml).
   If it's flagged only because of a prefix (e.g. `بعير` "camel" = `ب` + a bad word), the fix is adding it to [`src/safe-stems.json`](src/safe-stems.json).
 
+- **A sentence is judged wrong:** add it to [`test/fixtures/sentences.json`](test/fixtures/sentences.json) with the verdict a native speaker expects (`"profane": true` or `false`, plus `"options"` if it needs `includeInsults` or `stripSuffixes`). Every sentence there runs as a test, so later word-list changes can't silently flip it.
+
 Words that also have a common, normal meaning (like كلب "dog") belong in [`src/insults.json`](src/insults.json), which is opt-in, not in `words.json`.
 
 ### Development

package/dist/index.d.mts

@@ -6,26 +6,98 @@ interface IraqiBadWordsFilterOptions {
      * many of them are everyday words ("عندي كلب", "جلب لي هدية").
      */
     includeInsults?: boolean;
+    /**
+     * Also match words with a pronoun suffix the list doesn't spell out
+     * (طيزهم -> طيز). Off by default: it widens matching, so it can flag
+     * clean words that happen to end like a bad word plus a suffix.
+     */
+    stripSuffixes?: boolean;
+}
+/** Which list a dictionary entry came from. */
+type WordSource = "words" | "insults" | "custom";
+interface InspectOptions {
+    /**
+     * Also return words that matched but were cleared by a safe stem
+     * (e.g. بعير "camel" = ب + عير). They carry `suppressedBy`.
+     */
+    includeSuppressed?: boolean;
+}
+/** A bad word found by inspect(). */
+interface ProfanityMatch {
+    /** The word exactly as written in the input. */
+    token: string;
+    /**
+     * Offset of the word in the input; text.slice(start, end) === token.
+     * Counted in JavaScript string units (UTF-16), like String.prototype.slice.
+     */
+    start: number;
+    /** Offset just past the end of the word in the input. */
+    end: number;
+    /** The word after normalization (diacritics, stretched letters, … removed). */
+    normalized: string;
+    /** The dictionary entry it matched, spelled as in its list. */
+    entry: string;
+    /** The list the entry came from. */
+    source: WordSource;
+    /** The prefix stripped before matching (e.g. "وال"), or "" for a direct match. */
+    prefix: string;
+    /** The suffix stripped before matching (only with stripSuffixes), or "". */
+    suffix: string;
+    /**
+     * The safe stem that cleared this match, spelled as in safe-stems.json.
+     * Only set on matches returned with `includeSuppressed`; these are not profane.
+     */
+    suppressedBy?: string;
 }
 declare class IraqiBadWordsFilter {
     private words;
     private safeStems;
+    private stripSuffixes;
     constructor(options?: string[] | IraqiBadWordsFilterOptions);
     /**
      * Normalizes Arabic text so spelling variants compare equal:
-     * removes diacritics (Tashkeel) and tatweel, unifies Alef/Teh Marbuta/Alef Maksura
-     * forms, and collapses stretched letters (عاااارة -> عارة).
+     * removes diacritics (Tashkeel), tatweel and invisible characters, unifies
+     * Alef/Teh Marbuta/Alef Maksura forms and the Persian/Kurdish keyboard letters
+     * ی ک ە, turns Arabic-Indic digits into 0-9, and collapses stretched letters
+     * (عاااارة -> عارة).
      */
     private normalizeArabic;
+    /**
+     * Folds Arabizi (Arabic written in Latin letters and digits) so spelling
+     * variants compare equal: ga7ba / qahba / gahba, kosomak / kusomak,
+     * teezak / tizak. Only words made purely of Latin letters and digits are
+     * folded; expects text already passed through normalizeArabic.
+     */
+    private foldArabizi;
+    /** Normalizes a word or token for comparison. */
+    private normalize;
+    /**
+     * Adds words from one list. A word already present keeps its first source.
+     */
+    private addEntries;
     /**
      * Checks a single word, also trying it with attached prefixes removed
-     * (والكحبة -> كحبة, بطيزك -> طيزك).
+     * (والكحبة -> كحبة, بطيزك -> طيزك), and suffixes removed when stripSuffixes is on.
+     * Returns the matched entry, or null.
+     * A match cleared by a safe stem is returned with `suppressedBy` set.
      */
-    private isProfaneWord;
+    private matchWord;
+    /**
+     * Finds every match in the text, including ones cleared by a safe stem.
+     * A token with invisible characters is matched whole first (عا‌رة -> عارة);
+     * if that finds nothing profane, each visible piece is tried on its own, so an
+     * invisible character between two words can't hide a bad word either.
+     */
+    private findMatches;
     /**
      * Checks if a given text contains profane words.
      */
     isProfane(text: string): boolean;
+    /**
+     * Lists every bad word in the text with its position in the original input
+     * and why it matched, e.g. to highlight words or review a match before blocking.
+     */
+    inspect(text: string, options?: InspectOptions): ProfanityMatch[];
     /**
      * Replaces profane words with a placeholder string.
      */
@@ -40,4 +112,4 @@ declare class IraqiBadWordsFilter {
     removeWords(wordsToRemove: string[]): void;
 }
 
-export { IraqiBadWordsFilter, type IraqiBadWordsFilterOptions, IraqiBadWordsFilter as default };
+export { type InspectOptions, IraqiBadWordsFilter, type IraqiBadWordsFilterOptions, type ProfanityMatch, type WordSource, IraqiBadWordsFilter as default };

package/dist/index.d.ts

@@ -6,26 +6,98 @@ interface IraqiBadWordsFilterOptions {
      * many of them are everyday words ("عندي كلب", "جلب لي هدية").
      */
     includeInsults?: boolean;
+    /**
+     * Also match words with a pronoun suffix the list doesn't spell out
+     * (طيزهم -> طيز). Off by default: it widens matching, so it can flag
+     * clean words that happen to end like a bad word plus a suffix.
+     */
+    stripSuffixes?: boolean;
+}
+/** Which list a dictionary entry came from. */
+type WordSource = "words" | "insults" | "custom";
+interface InspectOptions {
+    /**
+     * Also return words that matched but were cleared by a safe stem
+     * (e.g. بعير "camel" = ب + عير). They carry `suppressedBy`.
+     */
+    includeSuppressed?: boolean;
+}
+/** A bad word found by inspect(). */
+interface ProfanityMatch {
+    /** The word exactly as written in the input. */
+    token: string;
+    /**
+     * Offset of the word in the input; text.slice(start, end) === token.
+     * Counted in JavaScript string units (UTF-16), like String.prototype.slice.
+     */
+    start: number;
+    /** Offset just past the end of the word in the input. */
+    end: number;
+    /** The word after normalization (diacritics, stretched letters, … removed). */
+    normalized: string;
+    /** The dictionary entry it matched, spelled as in its list. */
+    entry: string;
+    /** The list the entry came from. */
+    source: WordSource;
+    /** The prefix stripped before matching (e.g. "وال"), or "" for a direct match. */
+    prefix: string;
+    /** The suffix stripped before matching (only with stripSuffixes), or "". */
+    suffix: string;
+    /**
+     * The safe stem that cleared this match, spelled as in safe-stems.json.
+     * Only set on matches returned with `includeSuppressed`; these are not profane.
+     */
+    suppressedBy?: string;
 }
 declare class IraqiBadWordsFilter {
     private words;
     private safeStems;
+    private stripSuffixes;
     constructor(options?: string[] | IraqiBadWordsFilterOptions);
     /**
      * Normalizes Arabic text so spelling variants compare equal:
-     * removes diacritics (Tashkeel) and tatweel, unifies Alef/Teh Marbuta/Alef Maksura
-     * forms, and collapses stretched letters (عاااارة -> عارة).
+     * removes diacritics (Tashkeel), tatweel and invisible characters, unifies
+     * Alef/Teh Marbuta/Alef Maksura forms and the Persian/Kurdish keyboard letters
+     * ی ک ە, turns Arabic-Indic digits into 0-9, and collapses stretched letters
+     * (عاااارة -> عارة).
      */
     private normalizeArabic;
+    /**
+     * Folds Arabizi (Arabic written in Latin letters and digits) so spelling
+     * variants compare equal: ga7ba / qahba / gahba, kosomak / kusomak,
+     * teezak / tizak. Only words made purely of Latin letters and digits are
+     * folded; expects text already passed through normalizeArabic.
+     */
+    private foldArabizi;
+    /** Normalizes a word or token for comparison. */
+    private normalize;
+    /**
+     * Adds words from one list. A word already present keeps its first source.
+     */
+    private addEntries;
     /**
      * Checks a single word, also trying it with attached prefixes removed
-     * (والكحبة -> كحبة, بطيزك -> طيزك).
+     * (والكحبة -> كحبة, بطيزك -> طيزك), and suffixes removed when stripSuffixes is on.
+     * Returns the matched entry, or null.
+     * A match cleared by a safe stem is returned with `suppressedBy` set.
      */
-    private isProfaneWord;
+    private matchWord;
+    /**
+     * Finds every match in the text, including ones cleared by a safe stem.
+     * A token with invisible characters is matched whole first (عا‌رة -> عارة);
+     * if that finds nothing profane, each visible piece is tried on its own, so an
+     * invisible character between two words can't hide a bad word either.
+     */
+    private findMatches;
     /**
      * Checks if a given text contains profane words.
      */
     isProfane(text: string): boolean;
+    /**
+     * Lists every bad word in the text with its position in the original input
+     * and why it matched, e.g. to highlight words or review a match before blocking.
+     */
+    inspect(text: string, options?: InspectOptions): ProfanityMatch[];
     /**
      * Replaces profane words with a placeholder string.
      */
@@ -40,4 +112,4 @@ declare class IraqiBadWordsFilter {
     removeWords(wordsToRemove: string[]): void;
 }
 
-export { IraqiBadWordsFilter, type IraqiBadWordsFilterOptions, IraqiBadWordsFilter as default };
+export { type InspectOptions, IraqiBadWordsFilter, type IraqiBadWordsFilterOptions, type ProfanityMatch, type WordSource, IraqiBadWordsFilter as default };

package/dist/index.js

@@ -41,6 +41,12 @@ var words_default = [
   "\u0639\u064A\u0631\u0647",
   "\u0639\u064A\u0631\u0647\u0627",
   "\u0639\u064A\u0631\u064A\u0646",
+  "\u0639\u064A\u063A",
+  "\u0639\u064A\u063A\u064A",
+  "\u0639\u064A\u063A\u0643",
+  "\u0639\u064A\u063A\u0647",
+  "\u0639\u064A\u063A\u0647\u0627",
+  "\u0639\u064A\u063A\u064A\u0646",
   "\u0632\u0628",
   "\u0632\u0628\u064A",
   "\u0632\u0628\u0643",
@@ -226,87 +232,193 @@ var insults_default = [
 // src/safe-stems.json
 var safe_stems_default = [
   "\u0628\u0639\u064A\u0631",
+  "\u0628\u0639\u064A\u063A",
   "\u0628\u0643\u0633",
   "\u0648\u0643\u0633",
   "\u0641\u0643\u0633"
 ];
 
 // src/index.ts
-var WORD_PATTERN = /[A-Za-z0-9_À-ɏ؀-؋؍-ؚ؜-؞ؠ-٩ٮ-ۓە-ۿݐ-ݿࢠ-ࣿﭐ-﷿ﹰ-ﻼ]+/g;
+var INVISIBLE = "\\u00AD\\u061C\\u200B-\\u200F\\u202A-\\u202E\\u2060-\\u2064\\u2066-\\u2069\\uFEFF";
+var INVISIBLE_CHARS = new RegExp(`[${INVISIBLE}]`, "g");
+var HAS_INVISIBLE = new RegExp(`[${INVISIBLE}]`);
+var VISIBLE_RUN = new RegExp(`[^${INVISIBLE}]+`, "g");
+var WORD_PATTERN = new RegExp(
+  `[A-Za-z0-9_\xC0-\u024F\u0600-\u060B\u060D-\u061A\u061C-\u061E\u0620-\u0669\u066E-\u06D3\u06D5-\u06FF\u0750-\u077F\u08A0-\u08FF\uFB50-\uFDFF\uFE70-\uFEFC${INVISIBLE}]+`,
+  "g"
+);
 var CONJUNCTIONS = ["", "\u0648", "\u0641"];
 var PREPOSITIONS = ["", "\u0628", "\u0644", "\u0643"];
 var ARTICLES = ["", "\u0627\u0644"];
+var SUFFIXES = ["\u064A", "\u0643", "\u062C", "\u0647", "\u0647\u0627", "\u0646\u0627", "\u0643\u0645", "\u0647\u0645"];
 var IraqiBadWordsFilter = class {
   constructor(options = {}) {
-    const { customWords = [], includeInsults = false } = Array.isArray(options) ? { customWords: options } : options;
-    this.words = /* @__PURE__ */ new Set();
-    this.safeStems = safe_stems_default.map((stem) => this.normalizeArabic(stem));
-    this.addWords([...words_default, ...includeInsults ? insults_default : [], ...customWords]);
+    const { customWords = [], includeInsults = false, stripSuffixes = false } = Array.isArray(options) ? { customWords: options } : options;
+    this.words = /* @__PURE__ */ new Map();
+    this.stripSuffixes = stripSuffixes;
+    this.safeStems = safe_stems_default.map((word) => ({ normalized: this.normalize(word), word }));
+    this.addEntries(words_default, "words");
+    if (includeInsults) this.addEntries(insults_default, "insults");
+    this.addEntries(customWords, "custom");
   }
   /**
    * Normalizes Arabic text so spelling variants compare equal:
-   * removes diacritics (Tashkeel) and tatweel, unifies Alef/Teh Marbuta/Alef Maksura
-   * forms, and collapses stretched letters (عاااارة -> عارة).
+   * removes diacritics (Tashkeel), tatweel and invisible characters, unifies
+   * Alef/Teh Marbuta/Alef Maksura forms and the Persian/Kurdish keyboard letters
+   * ی ک ە, turns Arabic-Indic digits into 0-9, and collapses stretched letters
+   * (عاااارة -> عارة).
    */
   normalizeArabic(text) {
-    return text.replace(/[ً-ٰٟـ]/g, "").replace(/[أإآٱ]/g, "\u0627").replace(/ة/g, "\u0647").replace(/ى/g, "\u064A").replace(/(.)\1+/gu, "$1").toLowerCase();
+    return text.replace(/[\u064B-\u065F\u0670\u0640]/g, "").replace(INVISIBLE_CHARS, "").replace(/[أإآٱ]/g, "\u0627").replace(/[ةەہۀ]/g, "\u0647").replace(/[ىی]/g, "\u064A").replace(/[کڪ]/g, "\u0643").replace(/[٠-٩]/g, (digit) => String(digit.charCodeAt(0) - 1632)).replace(/[۰-۹]/g, (digit) => String(digit.charCodeAt(0) - 1776)).replace(/(.)\1+/gu, "$1").toLowerCase();
+  }
+  /**
+   * Folds Arabizi (Arabic written in Latin letters and digits) so spelling
+   * variants compare equal: ga7ba / qahba / gahba, kosomak / kusomak,
+   * teezak / tizak. Only words made purely of Latin letters and digits are
+   * folded; expects text already passed through normalizeArabic.
+   */
+  foldArabizi(word) {
+    if (!/^[a-z0-9]+$/.test(word) || !/[a-z]/.test(word)) return word;
+    return word.replace(/7/g, "h").replace(/5/g, "kh").replace(/[23]/g, "").replace(/q/g, "g").replace(/ou|o/g, "u").replace(/e/g, "i").replace(/(.)\1+/g, "$1");
+  }
+  /** Normalizes a word or token for comparison. */
+  normalize(text) {
+    const word = this.normalizeArabic(text);
+    return /[\u0600-\uFEFF]/.test(word) ? word.replace(/[0-9]/g, "").replace(/(.)\1+/gu, "$1") : this.foldArabizi(word);
+  }
+  /**
+   * Adds words from one list. A word already present keeps its first source.
+   */
+  addEntries(newWords, source) {
+    for (const word of newWords) {
+      const normalized = this.normalize(word);
+      if (!this.words.has(normalized)) this.words.set(normalized, { word, source });
+    }
   }
   /**
    * Checks a single word, also trying it with attached prefixes removed
-   * (والكحبة -> كحبة, بطيزك -> طيزك).
+   * (والكحبة -> كحبة, بطيزك -> طيزك), and suffixes removed when stripSuffixes is on.
+   * Returns the matched entry, or null.
+   * A match cleared by a safe stem is returned with `suppressedBy` set.
    */
-  isProfaneWord(token) {
-    const word = this.normalizeArabic(token);
-    if (this.words.has(word)) return true;
-    const stems = [word];
+  matchWord(token) {
+    const word = this.normalize(token);
+    const direct = this.words.get(word);
+    if (direct) {
+      return { normalized: word, entry: direct.word, source: direct.source, prefix: "", suffix: "" };
+    }
+    const candidates = [{ prefix: "", stem: word, suffix: "" }];
     for (const conjunction of CONJUNCTIONS) {
       for (const preposition of PREPOSITIONS) {
         for (const article of ARTICLES) {
           const prefix = conjunction + preposition + article;
           if (prefix && word.startsWith(prefix) && word.length - prefix.length >= 2) {
-            stems.push(word.slice(prefix.length));
+            candidates.push({ prefix, stem: word.slice(prefix.length), suffix: "" });
+          }
+        }
+      }
+    }
+    if (this.stripSuffixes) {
+      for (const { prefix, stem } of [...candidates]) {
+        for (const suffix of SUFFIXES) {
+          if (stem.endsWith(suffix) && stem.length - suffix.length >= 2) {
+            candidates.push({ prefix, stem: stem.slice(0, -suffix.length), suffix });
           }
         }
       }
     }
-    if (stems.some((stem) => this.safeStems.some((safe) => stem.startsWith(safe)))) {
-      return false;
+    const match = candidates.find(({ stem }) => this.words.has(stem));
+    if (!match) return null;
+    const { word: entry, source } = this.words.get(match.stem);
+    const result = {
+      normalized: word,
+      entry,
+      source,
+      prefix: match.prefix,
+      suffix: match.suffix
+    };
+    for (const { stem } of candidates) {
+      const safe = this.safeStems.find(({ normalized }) => stem.startsWith(normalized));
+      if (safe) return { ...result, suppressedBy: safe.word };
+    }
+    return result;
+  }
+  /**
+   * Finds every match in the text, including ones cleared by a safe stem.
+   * A token with invisible characters is matched whole first (عا‌رة -> عارة);
+   * if that finds nothing profane, each visible piece is tried on its own, so an
+   * invisible character between two words can't hide a bad word either.
+   */
+  *findMatches(text) {
+    const pattern = new RegExp(WORD_PATTERN.source, "g");
+    let found;
+    while (found = pattern.exec(text)) {
+      const token = found[0];
+      const start = found.index;
+      const whole = this.matchWord(token);
+      const wholeMatch = whole && { token, start, end: start + token.length, ...whole };
+      if (whole && whole.suppressedBy === void 0 || !HAS_INVISIBLE.test(token)) {
+        if (wholeMatch) yield wholeMatch;
+        continue;
+      }
+      const pieces = [];
+      const piecePattern = new RegExp(VISIBLE_RUN.source, "g");
+      let piece;
+      while (piece = piecePattern.exec(token)) {
+        const match = this.matchWord(piece[0]);
+        const pieceStart = start + piece.index;
+        if (match) pieces.push({ token: piece[0], start: pieceStart, end: pieceStart + piece[0].length, ...match });
+      }
+      if (pieces.length) yield* pieces;
+      else if (wholeMatch) yield wholeMatch;
     }
-    return stems.some((stem) => this.words.has(stem));
   }
   /**
    * Checks if a given text contains profane words.
    */
   isProfane(text) {
     if (!text) return false;
-    for (const token of text.match(WORD_PATTERN) ?? []) {
-      if (this.isProfaneWord(token)) {
-        return true;
-      }
+    for (const match of this.findMatches(text)) {
+      if (match.suppressedBy === void 0) return true;
     }
     return false;
   }
+  /**
+   * Lists every bad word in the text with its position in the original input
+   * and why it matched, e.g. to highlight words or review a match before blocking.
+   */
+  inspect(text, options = {}) {
+    if (!text) return [];
+    const matches = [];
+    for (const match of this.findMatches(text)) {
+      if (match.suppressedBy === void 0 || options.includeSuppressed) matches.push(match);
+    }
+    return matches;
+  }
   /**
    * Replaces profane words with a placeholder string.
    */
   clean(text, placeholder = "*") {
     if (!text) return "";
-    return text.replace(
-      WORD_PATTERN,
-      (token) => this.isProfaneWord(token) ? placeholder.repeat(token.length) : token
-    );
+    let result = "";
+    let last = 0;
+    for (const { start, end } of this.inspect(text)) {
+      result += text.slice(last, start) + placeholder.repeat(end - start);
+      last = end;
+    }
+    return result + text.slice(last);
   }
   /**
    * Dynamically add words to the active dictionary.
    */
   addWords(newWords) {
-    newWords.forEach((word) => this.words.add(this.normalizeArabic(word)));
+    this.addEntries(newWords, "custom");
   }
   /**
    * Remove words from the active filter dictionary.
    */
   removeWords(wordsToRemove) {
-    wordsToRemove.forEach((word) => this.words.delete(this.normalizeArabic(word)));
+    wordsToRemove.forEach((word) => this.words.delete(this.normalize(word)));
   }
 };
 var index_default = IraqiBadWordsFilter;

package/dist/index.mjs

@@ -14,6 +14,12 @@ var words_default = [
   "\u0639\u064A\u0631\u0647",
   "\u0639\u064A\u0631\u0647\u0627",
   "\u0639\u064A\u0631\u064A\u0646",
+  "\u0639\u064A\u063A",
+  "\u0639\u064A\u063A\u064A",
+  "\u0639\u064A\u063A\u0643",
+  "\u0639\u064A\u063A\u0647",
+  "\u0639\u064A\u063A\u0647\u0627",
+  "\u0639\u064A\u063A\u064A\u0646",
   "\u0632\u0628",
   "\u0632\u0628\u064A",
   "\u0632\u0628\u0643",
@@ -199,87 +205,193 @@ var insults_default = [
 // src/safe-stems.json
 var safe_stems_default = [
   "\u0628\u0639\u064A\u0631",
+  "\u0628\u0639\u064A\u063A",
   "\u0628\u0643\u0633",
   "\u0648\u0643\u0633",
   "\u0641\u0643\u0633"
 ];
 
 // src/index.ts
-var WORD_PATTERN = /[A-Za-z0-9_À-ɏ؀-؋؍-ؚ؜-؞ؠ-٩ٮ-ۓە-ۿݐ-ݿࢠ-ࣿﭐ-﷿ﹰ-ﻼ]+/g;
+var INVISIBLE = "\\u00AD\\u061C\\u200B-\\u200F\\u202A-\\u202E\\u2060-\\u2064\\u2066-\\u2069\\uFEFF";
+var INVISIBLE_CHARS = new RegExp(`[${INVISIBLE}]`, "g");
+var HAS_INVISIBLE = new RegExp(`[${INVISIBLE}]`);
+var VISIBLE_RUN = new RegExp(`[^${INVISIBLE}]+`, "g");
+var WORD_PATTERN = new RegExp(
+  `[A-Za-z0-9_\xC0-\u024F\u0600-\u060B\u060D-\u061A\u061C-\u061E\u0620-\u0669\u066E-\u06D3\u06D5-\u06FF\u0750-\u077F\u08A0-\u08FF\uFB50-\uFDFF\uFE70-\uFEFC${INVISIBLE}]+`,
+  "g"
+);
 var CONJUNCTIONS = ["", "\u0648", "\u0641"];
 var PREPOSITIONS = ["", "\u0628", "\u0644", "\u0643"];
 var ARTICLES = ["", "\u0627\u0644"];
+var SUFFIXES = ["\u064A", "\u0643", "\u062C", "\u0647", "\u0647\u0627", "\u0646\u0627", "\u0643\u0645", "\u0647\u0645"];
 var IraqiBadWordsFilter = class {
   constructor(options = {}) {
-    const { customWords = [], includeInsults = false } = Array.isArray(options) ? { customWords: options } : options;
-    this.words = /* @__PURE__ */ new Set();
-    this.safeStems = safe_stems_default.map((stem) => this.normalizeArabic(stem));
-    this.addWords([...words_default, ...includeInsults ? insults_default : [], ...customWords]);
+    const { customWords = [], includeInsults = false, stripSuffixes = false } = Array.isArray(options) ? { customWords: options } : options;
+    this.words = /* @__PURE__ */ new Map();
+    this.stripSuffixes = stripSuffixes;
+    this.safeStems = safe_stems_default.map((word) => ({ normalized: this.normalize(word), word }));
+    this.addEntries(words_default, "words");
+    if (includeInsults) this.addEntries(insults_default, "insults");
+    this.addEntries(customWords, "custom");
   }
   /**
    * Normalizes Arabic text so spelling variants compare equal:
-   * removes diacritics (Tashkeel) and tatweel, unifies Alef/Teh Marbuta/Alef Maksura
-   * forms, and collapses stretched letters (عاااارة -> عارة).
+   * removes diacritics (Tashkeel), tatweel and invisible characters, unifies
+   * Alef/Teh Marbuta/Alef Maksura forms and the Persian/Kurdish keyboard letters
+   * ی ک ە, turns Arabic-Indic digits into 0-9, and collapses stretched letters
+   * (عاااارة -> عارة).
    */
   normalizeArabic(text) {
-    return text.replace(/[ً-ٰٟـ]/g, "").replace(/[أإآٱ]/g, "\u0627").replace(/ة/g, "\u0647").replace(/ى/g, "\u064A").replace(/(.)\1+/gu, "$1").toLowerCase();
+    return text.replace(/[\u064B-\u065F\u0670\u0640]/g, "").replace(INVISIBLE_CHARS, "").replace(/[أإآٱ]/g, "\u0627").replace(/[ةەہۀ]/g, "\u0647").replace(/[ىی]/g, "\u064A").replace(/[کڪ]/g, "\u0643").replace(/[٠-٩]/g, (digit) => String(digit.charCodeAt(0) - 1632)).replace(/[۰-۹]/g, (digit) => String(digit.charCodeAt(0) - 1776)).replace(/(.)\1+/gu, "$1").toLowerCase();
+  }
+  /**
+   * Folds Arabizi (Arabic written in Latin letters and digits) so spelling
+   * variants compare equal: ga7ba / qahba / gahba, kosomak / kusomak,
+   * teezak / tizak. Only words made purely of Latin letters and digits are
+   * folded; expects text already passed through normalizeArabic.
+   */
+  foldArabizi(word) {
+    if (!/^[a-z0-9]+$/.test(word) || !/[a-z]/.test(word)) return word;
+    return word.replace(/7/g, "h").replace(/5/g, "kh").replace(/[23]/g, "").replace(/q/g, "g").replace(/ou|o/g, "u").replace(/e/g, "i").replace(/(.)\1+/g, "$1");
+  }
+  /** Normalizes a word or token for comparison. */
+  normalize(text) {
+    const word = this.normalizeArabic(text);
+    return /[\u0600-\uFEFF]/.test(word) ? word.replace(/[0-9]/g, "").replace(/(.)\1+/gu, "$1") : this.foldArabizi(word);
+  }
+  /**
+   * Adds words from one list. A word already present keeps its first source.
+   */
+  addEntries(newWords, source) {
+    for (const word of newWords) {
+      const normalized = this.normalize(word);
+      if (!this.words.has(normalized)) this.words.set(normalized, { word, source });
+    }
   }
   /**
    * Checks a single word, also trying it with attached prefixes removed
-   * (والكحبة -> كحبة, بطيزك -> طيزك).
+   * (والكحبة -> كحبة, بطيزك -> طيزك), and suffixes removed when stripSuffixes is on.
+   * Returns the matched entry, or null.
+   * A match cleared by a safe stem is returned with `suppressedBy` set.
    */
-  isProfaneWord(token) {
-    const word = this.normalizeArabic(token);
-    if (this.words.has(word)) return true;
-    const stems = [word];
+  matchWord(token) {
+    const word = this.normalize(token);
+    const direct = this.words.get(word);
+    if (direct) {
+      return { normalized: word, entry: direct.word, source: direct.source, prefix: "", suffix: "" };
+    }
+    const candidates = [{ prefix: "", stem: word, suffix: "" }];
     for (const conjunction of CONJUNCTIONS) {
       for (const preposition of PREPOSITIONS) {
         for (const article of ARTICLES) {
           const prefix = conjunction + preposition + article;
           if (prefix && word.startsWith(prefix) && word.length - prefix.length >= 2) {
-            stems.push(word.slice(prefix.length));
+            candidates.push({ prefix, stem: word.slice(prefix.length), suffix: "" });
+          }
+        }
+      }
+    }
+    if (this.stripSuffixes) {
+      for (const { prefix, stem } of [...candidates]) {
+        for (const suffix of SUFFIXES) {
+          if (stem.endsWith(suffix) && stem.length - suffix.length >= 2) {
+            candidates.push({ prefix, stem: stem.slice(0, -suffix.length), suffix });
           }
         }
       }
     }
-    if (stems.some((stem) => this.safeStems.some((safe) => stem.startsWith(safe)))) {
-      return false;
+    const match = candidates.find(({ stem }) => this.words.has(stem));
+    if (!match) return null;
+    const { word: entry, source } = this.words.get(match.stem);
+    const result = {
+      normalized: word,
+      entry,
+      source,
+      prefix: match.prefix,
+      suffix: match.suffix
+    };
+    for (const { stem } of candidates) {
+      const safe = this.safeStems.find(({ normalized }) => stem.startsWith(normalized));
+      if (safe) return { ...result, suppressedBy: safe.word };
+    }
+    return result;
+  }
+  /**
+   * Finds every match in the text, including ones cleared by a safe stem.
+   * A token with invisible characters is matched whole first (عا‌رة -> عارة);
+   * if that finds nothing profane, each visible piece is tried on its own, so an
+   * invisible character between two words can't hide a bad word either.
+   */
+  *findMatches(text) {
+    const pattern = new RegExp(WORD_PATTERN.source, "g");
+    let found;
+    while (found = pattern.exec(text)) {
+      const token = found[0];
+      const start = found.index;
+      const whole = this.matchWord(token);
+      const wholeMatch = whole && { token, start, end: start + token.length, ...whole };
+      if (whole && whole.suppressedBy === void 0 || !HAS_INVISIBLE.test(token)) {
+        if (wholeMatch) yield wholeMatch;
+        continue;
+      }
+      const pieces = [];
+      const piecePattern = new RegExp(VISIBLE_RUN.source, "g");
+      let piece;
+      while (piece = piecePattern.exec(token)) {
+        const match = this.matchWord(piece[0]);
+        const pieceStart = start + piece.index;
+        if (match) pieces.push({ token: piece[0], start: pieceStart, end: pieceStart + piece[0].length, ...match });
+      }
+      if (pieces.length) yield* pieces;
+      else if (wholeMatch) yield wholeMatch;
     }
-    return stems.some((stem) => this.words.has(stem));
   }
   /**
    * Checks if a given text contains profane words.
    */
   isProfane(text) {
     if (!text) return false;
-    for (const token of text.match(WORD_PATTERN) ?? []) {
-      if (this.isProfaneWord(token)) {
-        return true;
-      }
+    for (const match of this.findMatches(text)) {
+      if (match.suppressedBy === void 0) return true;
     }
     return false;
   }
+  /**
+   * Lists every bad word in the text with its position in the original input
+   * and why it matched, e.g. to highlight words or review a match before blocking.
+   */
+  inspect(text, options = {}) {
+    if (!text) return [];
+    const matches = [];
+    for (const match of this.findMatches(text)) {
+      if (match.suppressedBy === void 0 || options.includeSuppressed) matches.push(match);
+    }
+    return matches;
+  }
   /**
    * Replaces profane words with a placeholder string.
    */
   clean(text, placeholder = "*") {
     if (!text) return "";
-    return text.replace(
-      WORD_PATTERN,
-      (token) => this.isProfaneWord(token) ? placeholder.repeat(token.length) : token
-    );
+    let result = "";
+    let last = 0;
+    for (const { start, end } of this.inspect(text)) {
+      result += text.slice(last, start) + placeholder.repeat(end - start);
+      last = end;
+    }
+    return result + text.slice(last);
   }
   /**
    * Dynamically add words to the active dictionary.
    */
   addWords(newWords) {
-    newWords.forEach((word) => this.words.add(this.normalizeArabic(word)));
+    this.addEntries(newWords, "custom");
   }
   /**
    * Remove words from the active filter dictionary.
    */
   removeWords(wordsToRemove) {
-    wordsToRemove.forEach((word) => this.words.delete(this.normalizeArabic(word)));
+    wordsToRemove.forEach((word) => this.words.delete(this.normalize(word)));
   }
 };
 var index_default = IraqiBadWordsFilter;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "iraqi-dialect-bad-words",
-  "version": "1.1.0",
+  "version": "1.2.1",
   "description": "An Iraqi dialect profanity filter for React, React Native Expo, and Node.js",
   "main": "./dist/index.js",
   "module": "./dist/index.mjs",