allprofanity 2.0.0 → 2.1.0

package/README.md

@@ -1,29 +1,28 @@
 # AllProfanity
 
-A comprehensive, zero-dependency, multi-language profanity filter for JavaScript/TypeScript applications with built-in support for English, Hindi, Hinglish, Bengali, Tamil, Telugu, French, German, and Spanish content.
+A blazing-fast, multi-language, enterprise-grade profanity filter for JavaScript/TypeScript with advanced leet-speak normalization, Unicode support, and a robust, modern API.
 
 [![npm version](https://img.shields.io/npm/v/allprofanity.svg)](https://www.npmjs.com/package/allprofanity)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
+---
+
 ## Features
 
-- **Multi-language Support**: Built-in dictionaries for English, Hindi/Hinglish, Bengali, Tamil, Telugu, French, German, and Spanish.
-- **Multiple Scripts**: Detects profanity in both Latin/Roman and native scripts (Devanagari, Bengali, Tamil, Telugu).
-- **Case Insensitive (configurable)**: By default, not case sensitive, but can be configured to be case sensitive.
-- **Leet Speak Detection**: Optionally detects leet speak and obfuscated profanities.
-- **Flexible Cleaning Options**:
-  - Character-level replacement (each character of a profane word becomes a placeholder).
-  - Word-level replacement (entire profane word becomes a single placeholder).
-- **Customizable & Extensible**:
-  - Dynamically add/remove words or whole dictionaries.
-  - Set custom placeholder characters or strings.
-  - Supports custom language packs.
-  - Whitelist words to avoid false positives.
-  - Strict mode and partial word detection options.
-- **Severity Levels**: Detects severity of profanities (MILD, MODERATE, SEVERE, EXTREME).
-- **Zero External Dependencies**: Fully built from scratch for maximum performance and control.
-- **TypeScript Support**: Full TypeScript type definitions included.
-- **Exportable Dictionaries**: Language word lists are exportable for direct use or extension.
+- **Ultra-Fast O(n) Detection:** TRIE-based, single-pass algorithm for massive performance gains over regex/set-based filters.
+- **Multi-Language Support:** Built-in dictionaries for English, Hindi, French, German, Spanish, Bengali, Tamil, Telugu. Load multiple at once.
+- **Multiple Scripts:** Detects profanity in Latin/Roman (Hinglish) and native scripts (e.g., Devanagari, Tamil, Telugu).
+- **Advanced Leet-Speak Normalization:** Detects obfuscated profanities (`f#ck`, `a55hole`, etc.) with context-aware mapping.
+- **Unicode & Punctuation Robustness:** Handles word boundaries and mixed language content with near-zero false positives.
+- **Flexible Cleaning:** Replace matches character-by-character or as whole words, with configurable placeholders.
+- **Custom Dictionaries:** Add/remove words or entire lists at runtime, including your own language packs.
+- **Whitelisting:** Exclude safe words or false positives from detection.
+- **Severity Levels:** Assess how offensive a string is (`MILD`, `MODERATE`, `SEVERE`, `EXTREME`).
+- **No Dictionary Exposure:** For security, the full list of loaded profanities is never exposed.
+- **TypeScript Support:** Typed, documented API and result objects.
+- **Zero 3rd-Party Dependencies:** Only internal code and data.
+
+---
 
 ## Installation
 
@@ -31,366 +30,373 @@ A comprehensive, zero-dependency, multi-language profanity filter for JavaScript
 npm install allprofanity
 # or
 yarn add allprofanity
-# or
-pnpm add allprofanity
 ```
 
-## Basic Usage
+---
 
-```javascript
-import profanity from 'allprofanity';
-
-// Check if a string contains profanity
-profanity.check('This is a clean sentence.');  // false
-profanity.check('This is a fucking test.');    // true
-profanity.check('यह एक चूतिया परीक्षण है।');    // true (Hindi example)
-profanity.check('Ye ek chutiya test hai.');    // true (Hinglish example)
+## Quick Start
 
-// Clean profanity (character by character replacement)
-profanity.clean('This is a fucking test.');
-// => "This is a ****ing test."
+```typescript
+import profanity from 'allprofanity';
 
-// Clean profanity (whole word replacement)
-profanity.cleanWithWord('This is a fucking test.');
-// => "This is a *** test."
+// Simple check
+profanity.check('This is a clean sentence.');        // false
+profanity.check('What the f#ck is this?');           // true (leet-speak detected)
+profanity.check('यह एक चूतिया परीक्षण है।');           // true (Hindi)
+profanity.check('Ye ek chutiya test hai.');          // true (Hinglish Roman script)
 ```
 
-## API Reference
-
-### `check(string: string): boolean`
+---
 
-Checks if a string contains any profanity from the loaded dictionaries.
-
-```javascript
-profanity.check('This contains bullshit.');  // true
-profanity.check('This is clean.');           // false
-```
+## API Reference & Examples
 
-### `detect(string: string): ProfanityDetectionResult`
+### `check(text: string): boolean`
 
-Advanced detection with details about profanities found, severity, cleaned text, and word positions.
+Returns `true` if the text contains any profanity.
 
-```javascript
-const result = profanity.detect('This contains bullshit.');
-// result: {
-//   hasProfanity: true,
-//   detectedWords: [...],
-//   cleanedText: ...,
-//   severity: ...,
-//   positions: [...]
-// }
+```typescript
+profanity.check('This is a clean sentence.');  // false
+profanity.check('This is a bullshit sentence.'); // true
+profanity.check('What the f#ck is this?'); // true (leet-speak)
+profanity.check('यह एक चूतिया परीक्षण है।'); // true (Hindi)
 ```
 
-### `clean(string: string, placeholder?: string): string`
+---
+
+### `detect(text: string): ProfanityDetectionResult`
 
-Cleans a string by replacing each character of profane words with a placeholder character.
+Returns a detailed result:
 
-```javascript
-// Default placeholder is "*"
-profanity.clean('This contains bullshit.');
-// => "This contains ********."
+- `hasProfanity: boolean`
+- `detectedWords: string[]` (actual matched words)
+- `cleanedText: string` (character-masked)
+- `severity: ProfanitySeverity` (`MILD`, `MODERATE`, `SEVERE`, `EXTREME`)
+- `positions: Array<{ word: string, start: number, end: number }>`
 
-// Custom placeholder character
-profanity.clean('This contains bullshit.', '#');
-// => "This contains ########."
+```typescript
+const result = profanity.detect('This is fucking bullshit and chutiya.');
+console.log(result.hasProfanity); // true
+console.log(result.detectedWords); // ['fucking', 'bullshit', 'chutiya']
+console.log(result.severity); // 3 (SEVERE)
+console.log(result.cleanedText); // "This is ******* ******** and ******."
+console.log(result.positions); // e.g. [{word: 'fucking', start: 8, end: 15}, ...]
 ```
 
-### `cleanWithWord(string: string, placeholder?: string): string`
+---
 
-Cleans a string by replacing entire profane words with a single placeholder string.
+### `clean(text: string, placeholder?: string): string`
 
-```javascript
-// Default placeholder is "***"
-profanity.cleanWithWord('This contains bullshit.');
-// => "This contains ***."
+Replace each character of profane words with a placeholder (default: `*`).
 
-// Custom placeholder string
-profanity.cleanWithWord('This contains bullshit.', '[CENSORED]');
-// => "This contains [CENSORED]."
+```typescript
+profanity.clean('This contains bullshit.'); // "This contains ********."
+profanity.clean('This contains bullshit.', '#'); // "This contains ########."
+profanity.clean('यह एक चूतिया परीक्षण है।'); // e.g. "यह एक ***** परीक्षण है।"
 ```
 
-### `list(): string[]`
+---
 
-Returns an array of all the profanity words currently in the filter.
+### `cleanWithPlaceholder(text: string, placeholder?: string): string`
 
-```javascript
-const words = profanity.list();
-console.log(words);  // ["fuck", "bullshit", "chutiya", ...]
+Replace each profane word with a single placeholder (default: `***`).  
+(If the placeholder is omitted, uses `***`.)
+
+```typescript
+profanity.cleanWithPlaceholder('This contains bullshit.'); // "This contains ***."
+profanity.cleanWithPlaceholder('This contains bullshit.', '[CENSORED]'); // "This contains [CENSORED]."
+profanity.cleanWithPlaceholder('यह एक चूतिया परीक्षण है।', '####'); // e.g. "यह एक #### परीक्षण है।"
 ```
 
+---
+
 ### `add(word: string | string[]): void`
 
-Adds one or more words to the profanity filter.
+Add a word or an array of words to the profanity filter.
 
-```javascript
-// Add a single word
-profanity.add('badword');
+```typescript
+profanity.add('badword123');
+profanity.check('This is badword123.'); // true
 
-// Add multiple words
-profanity.add(['badword1', 'badword2', 'बुराशब्द']);
+profanity.add(['mierda', 'puta']);
+profanity.check('Esto es mierda.'); // true (Spanish)
+profanity.check('Qué puta situación.'); // true
 ```
 
+---
+
 ### `remove(word: string | string[]): void`
 
-Removes one or more words from the profanity filter.
+Remove a word or an array of words from the profanity filter.
 
-```javascript
-// Remove a single word
+```typescript
 profanity.remove('bullshit');
+profanity.check('This is bullshit.'); // false
 
-// Remove multiple words
-profanity.remove(['fuck', 'chutiya']);
+profanity.remove(['mierda', 'puta']);
+profanity.check('Esto es mierda.'); // false
 ```
 
-### `clearList(): void`
+---
 
-Removes all words from the profanity filter, resetting it to an empty list.
+### `addToWhitelist(words: string[]): void`
 
-```javascript
-profanity.clearList();
+Whitelist words so they are never flagged as profane.
+
+```typescript
+profanity.addToWhitelist(['anal', 'ass']);
+profanity.check('He is an associate professor.'); // false
+profanity.check('I work as an analyst.'); // false
+// Remove from whitelist to restore detection
+profanity.removeFromWhitelist(['anal', 'ass']);
 ```
 
-### `setPlaceholder(placeholder: string): void`
+---
 
-Sets the default placeholder character for the `clean` method.
+### `removeFromWhitelist(words: string[]): void`
 
-```javascript
-// Set "#" as the default placeholder character
-profanity.setPlaceholder('#');
+Remove words from the whitelist so they can be detected again.
+
+```typescript
+profanity.removeFromWhitelist(['anal']);
 ```
 
-### `getLoadedLanguages(): string[]`
+---
+
+### `setPlaceholder(placeholder: string): void`
 
-Returns the list of currently loaded languages.
+Set the default placeholder character for `clean()`.
 
-```javascript
-const loaded = profanity.getLoadedLanguages();
-// => ['english', 'hindi', ...]
+```typescript
+profanity.setPlaceholder('#');
+profanity.clean('This is bullshit.'); // "This is ########."
+profanity.setPlaceholder('*'); // Reset to default
 ```
 
-### `getAvailableLanguages(): string[]`
+---
+
+### `updateConfig(options: Partial<AllProfanityOptions>): void`
 
-Returns the list of all available built-in languages.
+Change configuration at runtime.  
+Options include: `enableLeetSpeak`, `caseSensitive`, `strictMode`, `detectPartialWords`, `defaultPlaceholder`, `languages`, `whitelistWords`.
 
-```javascript
-const available = profanity.getAvailableLanguages();
-// => ['english', 'hindi', 'bengali', 'tamil', 'telugu', 'french', 'german', 'spanish']
+```typescript
+profanity.updateConfig({ caseSensitive: true, enableLeetSpeak: false });
+profanity.check('FUCK'); // false (if caseSensitive)
+profanity.updateConfig({ caseSensitive: false, enableLeetSpeak: true });
+profanity.check('f#ck'); // true
 ```
 
+---
+
 ### `loadLanguage(language: string): boolean`
 
-Loads a built-in language dictionary by name.
+Load a built-in language.
 
-```javascript
-profanity.loadLanguage('bengali');
+```typescript
 profanity.loadLanguage('french');
+profanity.check('Ce mot est merde.'); // true
 ```
 
+---
+
 ### `loadLanguages(languages: string[]): number`
 
-Loads multiple languages at once.
+Load multiple built-in languages at once.
 
-```javascript
-profanity.loadLanguages(['tamil', 'german', 'spanish']);
+```typescript
+profanity.loadLanguages(['english', 'french', 'german']);
+profanity.check('Das ist scheiße.'); // true (German)
 ```
 
+---
+
 ### `loadIndianLanguages(): number`
 
-Loads Hindi, Bengali, Tamil, and Telugu dictionaries.
+Convenience: Load all major Indian language packs.
 
-```javascript
+```typescript
 profanity.loadIndianLanguages();
+profanity.check('यह एक बेंगाली गाली है।'); // true (Bengali)
+profanity.check('This is a Tamil profanity: புண்டை'); // true
 ```
 
+---
+
 ### `loadCustomDictionary(name: string, words: string[]): void`
 
-Loads a custom dictionary under the given name.
+Add your own dictionary as an additional language.
 
-```javascript
-profanity.loadCustomDictionary('myLanguage', ['word1', 'word2']);
-profanity.loadLanguage('myLanguage');
+```typescript
+profanity.loadCustomDictionary('swedish', ['fan', 'jävla', 'skit']);
+profanity.loadLanguage('swedish');
+profanity.check('Det här är skit.'); // true
 ```
 
-### `addToWhitelist(words: string[]): void` / `removeFromWhitelist(words: string[]): void`
+---
 
-Add or remove words from the whitelist (words never flagged as profanity).
+### `getLoadedLanguages(): string[]`
 
-```javascript
-profanity.addToWhitelist(['anal', 'ass']);
-profanity.removeFromWhitelist(['anal']);
-```
+Returns the names of all currently loaded language packs.
 
-### `getConfig(): AllProfanityOptions`
+```typescript
+console.log(profanity.getLoadedLanguages()); // ['english', 'hindi', ...]
+```
 
-Get current configuration.
+---
 
-### `updateConfig(options: Partial<AllProfanityOptions>): void`
+### `getAvailableLanguages(): string[]`
 
-Update configuration (enable/disable leet speak, case sensitivity, etc.).
+Returns the names of all available built-in language packs.
 
-## Word Boundary Detection
+```typescript
+console.log(profanity.getAvailableLanguages());
+// ['english', 'hindi', 'french', 'german', 'spanish', 'bengali', 'tamil', 'telugu']
+```
 
-The library handles word boundaries and reduces false positives:
+---
 
-```javascript
-profanity.check('He is an associate professor.');  // false, even though 'ass' is a profane word
-profanity.check('I\'m an analyst at this company.'); // false, even though 'anal' is a profane word
-profanity.check('This is ass and that\'s bad.');     // true
-```
+### `clearList(): void`
 
-## Language Support
+Remove all loaded languages and dynamic words (start with a clean filter).
 
-### Current Languages
+```typescript
+profanity.clearList();
+profanity.check('fuck'); // false
+profanity.loadLanguage('english');
+profanity.check('fuck'); // true
+```
 
-- **English** (imported from `./languages/english-words.js`)
-- **Hindi/Hinglish** (`./languages/hindi-words.js`)
-- **Bengali** (`./languages/bengali-words.js`)
-- **Tamil** (`./languages/tamil-words.js`)
-- **Telugu** (`./languages/telugu-words.js`)
-- **French** (`./languages/french-words.js`)
-- **German** (`./languages/german-words.js`)
-- **Spanish** (`./languages/spanish-words.js`)
+---
 
-> **Note:** All dictionaries are exported for direct access/import.
+### `getConfig(): Partial<AllProfanityOptions>`
 
-#### Usage Examples
+Get the current configuration.
 
-```javascript
-profanity.check('इस वाक्य में लंड शब्द है।');  // true (Hindi)
-profanity.check('Is vakya mein lund shabd hai.');  // true (Hinglish)
-profanity.check('এই বাক্যে বাল শব্দ আছে।');  // true (Bengali)
-profanity.check('இந்த வாக்கியத்தில் கூதி உள்ளது.');  // true (Tamil)
-profanity.check('Cette phrase contient le mot merde.');  // true (French)
+```typescript
+console.log(profanity.getConfig());
+/*
+{
+  defaultPlaceholder: '*',
+  enableLeetSpeak: true,
+  caseSensitive: false,
+  strictMode: false,
+  detectPartialWords: false,
+  languages: [...],
+  whitelistWords: [...]
+}
+*/
 ```
 
-### Mixed Language Content
+---
 
-AllProfanity can detect profanities from multiple languages in a single string:
+## Severity Levels
 
-```javascript
-profanity.check('This English sentence has chutiya which is bad.');  // true
-profanity.check('I\'m saying मादरचोद and bullshit in one sentence.');  // true
-```
+Severity reflects the number and variety of detected profanities:
 
-### Loading Additional or Custom Languages
+| Level     | Enum Value | Description                             |
+|-----------|------------|-----------------------------------------|
+| MILD      | 1          | 1 unique/total word                     |
+| MODERATE  | 2          | 2 unique or total words                 |
+| SEVERE    | 3          | 3 unique/total words                    |
+| EXTREME   | 4          | 4+ unique or 5+ total profane words     |
 
-By default, only English and Hindi are loaded. You can load additional languages as needed:
+---
 
-```javascript
-profanity.loadLanguage('bengali');
-profanity.loadLanguages(['tamil', 'french']);
-```
+## Language Support
 
-You can also load custom dictionaries:
+- **Built-in:** English, Hindi, French, German, Spanish, Bengali, Tamil, Telugu
+- **Scripts:** Latin/Roman, Devanagari, Tamil, Telugu, Bengali, etc.
+- **Mixed Content:** Handles mixed-language and code-switched sentences.
 
-```javascript
-profanity.loadCustomDictionary('swedish', ['fulord1', 'fulord2']);
-profanity.loadLanguage('swedish');
+```typescript
+profanity.check('This is bullshit and चूतिया.'); // true (mixed English/Hindi)
+profanity.check('Ce mot est merde and पागल.');   // true (French/Hindi)
 ```
 
-## Customizing The Library
+---
 
-### Adding Custom Profanity Lists
+## Use Exported Wordlists
 
-You can add your own profanity words to extend support for other languages or add additional words to existing languages:
+For sample words in a language (for UIs, admin, etc):
 
-```javascript
-// Add custom profanity words
-profanity.add([
-  'customword1',
-  'customword2',
-  'customword3'
-]);
+```typescript
+import { englishBadWords, hindiBadWords } from 'allprofanity';
+console.log(englishBadWords.slice(0, 5)); // ["fuck", "shit", ...]
 ```
 
-## Creating a Custom-Configured Instance
-
-If you need multiple differently-configured filters, import the `AllProfanity` class directly:
+---
 
-```javascript
-import { AllProfanity } from 'allprofanity';
+## Security
 
-const kidSafeFilter = new AllProfanity({ enableLeetSpeak: true, strictMode: true });
-const adultFilter = new AllProfanity({ enableLeetSpeak: false, detectPartialWords: false });
-```
+- **No wordlist exposure:** There is no `.list()` function for security and encapsulation. Use exported word arrays for samples.
+- **TRIE-based:** Scales easily to 50,000+ words.
+- **Handles leet-speak:** Catches obfuscated variants like `f#ck`, `a55hole`.
 
-## Advanced Use Cases
+---
 
-### Performance Optimization
+## Full Example
 
-For high-throughput applications:
+```typescript
+import profanity, { ProfanitySeverity } from 'allprofanity';
 
-```javascript
-const badWordsList = profanity.list();
-const preCompiledRegex = new RegExp('\\b(' + badWordsList.join('|') + ')\\b', 'i');
+// Multi-language detection
+profanity.loadLanguages(['english', 'french', 'tamil']);
+console.log(profanity.check('Ce mot est merde.')); // true
 
-function quickCheck(text) {
-  return preCompiledRegex.test(text);
-}
-```
+// Leet-speak detection
+console.log(profanity.check('You a f#cking a55hole!')); // true
 
-### Content Moderation Systems
-
-```javascript
-function moderateContent(content) {
-  if (profanity.check(content)) {
-    return {
-      isApproved: false,
-      cleanedContent: profanity.cleanWithWord(content, '[INAPPROPRIATE]'),
-      reason: 'Contains profanity'
-    };
-  }
-  return { isApproved: true, cleanedContent: content };
-}
-```
+// Whitelisting
+profanity.addToWhitelist(['anal', 'ass']);
+console.log(profanity.check('He is an associate professor.')); // false
 
-## Use Cases
+// Severity
+const result = profanity.detect('This is fucking bullshit and chutiya.');
+console.log(ProfanitySeverity[result.severity]); // "SEVERE"
 
-- Content moderation systems
-- Chat applications
-- User-generated content platforms
-- Educational software
-- Forums and community platforms
-- Social media content filtering
-- Comment sections
-- Gaming chat filters
-- Email filtering
-- Document processing systems
+// Custom dictionary
+profanity.loadCustomDictionary('pirate', ['barnacle-head', 'landlubber']);
+profanity.loadLanguage('pirate');
+console.log(profanity.check('You barnacle-head!')); // true
 
-## Browser Support
+// Placeholder configuration
+profanity.setPlaceholder('#');
+console.log(profanity.clean('This is bullshit.')); // "This is ########."
+profanity.setPlaceholder('*'); // Reset
+```
 
-AllProfanity works in all modern browsers and Node.js environments.
+---
 
-## Roadmap
+## FAQ
 
-- Add support for more languages (Arabic, Chinese, Russian, etc.)
-- Contextual profanity detection
-- Severity levels for different categories of profanity
-- Phonetic matching for evasion attempts
-- Plugin system for custom detection strategies
+**Q: How do I see all loaded profanities?**  
+A: For security, the internal word list is not exposed. Use `englishBadWords` etc. for samples.
 
-## License
+**Q: How do I reset the filter?**  
+A: Use `clearList()` and reload languages/dictionaries.
 
-This project is licensed under the MIT License - see the [LICENSE](https://github.com/ayush-jadaun/allprofanity/blob/main/LICENSE) file for details.
+**Q: Is this safe for browser and Node.js?**  
+A: Yes! AllProfanity is universal.
 
-## Contributing
+---
 
-Contributions are welcome! Whether it's adding new language support, improving detection algorithms, or enhancing documentation.
+## Roadmap
 
-Please feel free to submit a Pull Request or open an Issue on GitHub.
+- More language packs (Arabic, Russian, etc.)
+- Contextual detection & severity scoring
+- Phonetic/typo/obfuscation resilience
+- Plugin system for custom detection
 
-### Adding a New Language
+---
 
-To add support for a new language:
+## License
 
-1. Create a new file in the `src/languages` directory (e.g., `french-words.ts`)
-2. Follow the format of the existing word lists
-3. Submit a pull request with your changes
+MIT — See [LICENSE](https://github.com/ayush-jadaun/allprofanity/blob/main/LICENSE)
 
-## Acknowledgements
+---
 
-- Inspired by [leo-profanity](https://github.com/jojoee/leo-profanity), but fully rebuilt for extensibility and multi-language support.
+## Contributing
 
-```diff
-- Note: As of v2+, AllProfanity is zero-dependency and does not use leo-profanity internally.
-```
+We welcome new language packs, detection improvements, and docs!  
+To add a new language, create a file in `src/languages/` and export a string array.  
+Open a PR or issue for bugs, features, or suggestions.