allprofanity 1.1.0 → 2.1.0

package/README.md

@@ -1,24 +1,28 @@
 # AllProfanity
 
-A comprehensive multi-language profanity filter for JavaScript/TypeScript applications with built-in support for English, Hindi, Hinglish, Bengali, Tamil, Telugu, French, German, and Spanish content.
+A blazing-fast, multi-language, enterprise-grade profanity filter for JavaScript/TypeScript with advanced leet-speak normalization, Unicode support, and a robust, modern API.
 
 [![npm version](https://img.shields.io/npm/v/allprofanity.svg)](https://www.npmjs.com/package/allprofanity)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
+---
+
 ## Features
 
-- **Multi-language Support**: Pre-loaded with English profanities (from leo-profanity) and extensive dictionaries for Hindi/Hinglish, Bengali, Tamil, Telugu, French, German, and Spanish
-- **Multiple Scripts**: Detects profanity in both Latin/Roman and native scripts (Devanagari, Bengali, Tamil, Telugu)
-- **Case Insensitive**: Works regardless of letter case
-- **Flexible Cleaning Options**:
-  - Character-level replacement (each character of a profane word becomes a placeholder)
-  - Word-level replacement (entire profane word becomes a single placeholder)
-- **Customizable**:
-  - Dynamically add/remove words from the filter
-  - Set custom placeholder characters or strings
-- **Zero Dependencies**: Only depends on leo-profanity as the base filter
-- **TypeScript Support**: Full TypeScript type definitions included
-- **Extensible**: Designed with multi-language support in mind, making it easy to add more languages in the future
+- **Ultra-Fast O(n) Detection:** TRIE-based, single-pass algorithm for massive performance gains over regex/set-based filters.
+- **Multi-Language Support:** Built-in dictionaries for English, Hindi, French, German, Spanish, Bengali, Tamil, Telugu. Load multiple at once.
+- **Multiple Scripts:** Detects profanity in Latin/Roman (Hinglish) and native scripts (e.g., Devanagari, Tamil, Telugu).
+- **Advanced Leet-Speak Normalization:** Detects obfuscated profanities (`f#ck`, `a55hole`, etc.) with context-aware mapping.
+- **Unicode & Punctuation Robustness:** Handles word boundaries and mixed language content with near-zero false positives.
+- **Flexible Cleaning:** Replace matches character-by-character or as whole words, with configurable placeholders.
+- **Custom Dictionaries:** Add/remove words or entire lists at runtime, including your own language packs.
+- **Whitelisting:** Exclude safe words or false positives from detection.
+- **Severity Levels:** Assess how offensive a string is (`MILD`, `MODERATE`, `SEVERE`, `EXTREME`).
+- **No Dictionary Exposure:** For security, the full list of loaded profanities is never exposed.
+- **TypeScript Support:** Typed, documented API and result objects.
+- **Zero 3rd-Party Dependencies:** Only internal code and data.
+
+---
 
 ## Installation
 
@@ -26,327 +30,373 @@ A comprehensive multi-language profanity filter for JavaScript/TypeScript applic
 npm install allprofanity
 # or
 yarn add allprofanity
-# or
-pnpm add allprofanity
 ```
 
-## Basic Usage
-
-```javascript
-import profanity from 'allprofanity';
+---
 
-// Check if a string contains profanity
-profanity.check('This is a clean sentence.');  // false
-profanity.check('This is a fucking test.');    // true
-profanity.check('यह एक चूतिया परीक्षण है।');    // true (Hindi example)
-profanity.check('Ye ek chutiya test hai.');    // true (Hinglish example)
+## Quick Start
 
-// Clean profanity (character by character replacement)
-profanity.clean('This is a fucking test.');  
-// => "This is a ****ing test."
+```typescript
+import profanity from 'allprofanity';
 
-// Clean profanity (whole word replacement)
-profanity.cleanWithWord('This is a fucking test.');  
-// => "This is a *** test."
+// Simple check
+profanity.check('This is a clean sentence.');        // false
+profanity.check('What the f#ck is this?');           // true (leet-speak detected)
+profanity.check('यह एक चूतिया परीक्षण है।');           // true (Hindi)
+profanity.check('Ye ek chutiya test hai.');          // true (Hinglish Roman script)
 ```
 
-## API Reference
+---
 
-### `check(string: string): boolean`
+## API Reference & Examples
 
-Checks if a string contains any profanity from the loaded dictionaries.
+### `check(text: string): boolean`
 
-```javascript
-profanity.check('This contains bullshit.');  // true
-profanity.check('This is clean.');           // false
+Returns `true` if the text contains any profanity.
+
+```typescript
+profanity.check('This is a clean sentence.');  // false
+profanity.check('This is a bullshit sentence.'); // true
+profanity.check('What the f#ck is this?'); // true (leet-speak)
+profanity.check('यह एक चूतिया परीक्षण है।'); // true (Hindi)
 ```
 
-### `clean(string: string, placeholder?: string): string`
+---
 
-Cleans a string by replacing each character of profane words with a placeholder character.
+### `detect(text: string): ProfanityDetectionResult`
 
-```javascript
-// Default placeholder is "*"
-profanity.clean('This contains bullshit.');
-// => "This contains ********."
+Returns a detailed result:
 
-// Custom placeholder character
-profanity.clean('This contains bullshit.', '#');
-// => "This contains ########."
+- `hasProfanity: boolean`
+- `detectedWords: string[]` (actual matched words)
+- `cleanedText: string` (character-masked)
+- `severity: ProfanitySeverity` (`MILD`, `MODERATE`, `SEVERE`, `EXTREME`)
+- `positions: Array<{ word: string, start: number, end: number }>`
+
+```typescript
+const result = profanity.detect('This is fucking bullshit and chutiya.');
+console.log(result.hasProfanity); // true
+console.log(result.detectedWords); // ['fucking', 'bullshit', 'chutiya']
+console.log(result.severity); // 3 (SEVERE)
+console.log(result.cleanedText); // "This is ******* ******** and ******."
+console.log(result.positions); // e.g. [{word: 'fucking', start: 8, end: 15}, ...]
 ```
 
-### `cleanWithWord(string: string, placeholder?: string): string`
+---
 
-Cleans a string by replacing entire profane words with a single placeholder string.
+### `clean(text: string, placeholder?: string): string`
 
-```javascript
-// Default placeholder is "***"
-profanity.cleanWithWord('This contains bullshit.');
-// => "This contains ***."
+Replace each character of profane words with a placeholder (default: `*`).
 
-// Custom placeholder string
-profanity.cleanWithWord('This contains bullshit.', '[CENSORED]');
-// => "This contains [CENSORED]."
+```typescript
+profanity.clean('This contains bullshit.'); // "This contains ********."
+profanity.clean('This contains bullshit.', '#'); // "This contains ########."
+profanity.clean('यह एक चूतिया परीक्षण है।'); // e.g. "यह एक ***** परीक्षण है।"
 ```
 
-### `list(): string[]`
+---
+
+### `cleanWithPlaceholder(text: string, placeholder?: string): string`
 
-Returns an array of all the profanity words currently in the filter.
+Replace each profane word with a single placeholder (default: `***`).  
+(If the placeholder is omitted, uses `***`.)
 
-```javascript
-const words = profanity.list();
-console.log(words);  // ["fuck", "bullshit", "chutiya", ...]
+```typescript
+profanity.cleanWithPlaceholder('This contains bullshit.'); // "This contains ***."
+profanity.cleanWithPlaceholder('This contains bullshit.', '[CENSORED]'); // "This contains [CENSORED]."
+profanity.cleanWithPlaceholder('यह एक चूतिया परीक्षण है।', '####'); // e.g. "यह एक #### परीक्षण है।"
 ```
 
+---
+
 ### `add(word: string | string[]): void`
 
-Adds one or more words to the profanity filter.
+Add a word or an array of words to the profanity filter.
 
-```javascript
-// Add a single word
-profanity.add('badword');
+```typescript
+profanity.add('badword123');
+profanity.check('This is badword123.'); // true
 
-// Add multiple words
-profanity.add(['badword1', 'badword2', 'बुराशब्द']);
+profanity.add(['mierda', 'puta']);
+profanity.check('Esto es mierda.'); // true (Spanish)
+profanity.check('Qué puta situación.'); // true
 ```
 
+---
+
 ### `remove(word: string | string[]): void`
 
-Removes one or more words from the profanity filter.
+Remove a word or an array of words from the profanity filter.
 
-```javascript
-// Remove a single word
+```typescript
 profanity.remove('bullshit');
+profanity.check('This is bullshit.'); // false
 
-// Remove multiple words
-profanity.remove(['fuck', 'chutiya']);
+profanity.remove(['mierda', 'puta']);
+profanity.check('Esto es mierda.'); // false
 ```
 
-### `clearList(): void`
+---
 
-Removes all words from the profanity filter, resetting it to an empty list.
+### `addToWhitelist(words: string[]): void`
 
-```javascript
-profanity.clearList();
+Whitelist words so they are never flagged as profane.
+
+```typescript
+profanity.addToWhitelist(['anal', 'ass']);
+profanity.check('He is an associate professor.'); // false
+profanity.check('I work as an analyst.'); // false
+// Remove from whitelist to restore detection
+profanity.removeFromWhitelist(['anal', 'ass']);
 ```
 
+---
+
+### `removeFromWhitelist(words: string[]): void`
+
+Remove words from the whitelist so they can be detected again.
+
+```typescript
+profanity.removeFromWhitelist(['anal']);
+```
+
+---
+
 ### `setPlaceholder(placeholder: string): void`
 
-Sets the default placeholder character for the `clean` method.
+Set the default placeholder character for `clean()`.
 
-```javascript
-// Set "#" as the default placeholder character
+```typescript
 profanity.setPlaceholder('#');
+profanity.clean('This is bullshit.'); // "This is ########."
+profanity.setPlaceholder('*'); // Reset to default
 ```
 
-## Word Boundary Detection
+---
 
-The library is designed to handle word boundaries correctly, reducing false positives:
+### `updateConfig(options: Partial<AllProfanityOptions>): void`
 
-```javascript
-profanity.check('He is an associate professor.');  // false, even though 'ass' is a profane word
-profanity.check('I'm an analyst at this company.'); // false, even though 'anal' is a profane word
-profanity.check('This is ass and that's bad.');     // true
-```
+Change configuration at runtime.  
+Options include: `enableLeetSpeak`, `caseSensitive`, `strictMode`, `detectPartialWords`, `defaultPlaceholder`, `languages`, `whitelistWords`.
 
-## Language Support
+```typescript
+profanity.updateConfig({ caseSensitive: true, enableLeetSpeak: false });
+profanity.check('FUCK'); // false (if caseSensitive)
+profanity.updateConfig({ caseSensitive: false, enableLeetSpeak: true });
+profanity.check('f#ck'); // true
+```
 
-### Current Languages
+---
 
-#### English
+### `loadLanguage(language: string): boolean`
 
-Built on top of the leo-profanity library, AllProfanity includes comprehensive English profanity detection.
+Load a built-in language.
 
-#### Hindi/Hinglish Support
+```typescript
+profanity.loadLanguage('french');
+profanity.check('Ce mot est merde.'); // true
+```
 
-The library comes pre-loaded with an extensive list of Hindi profanities in both Devanagari and Roman scripts, as well as common Hinglish abbreviations and variations.
+---
 
-```javascript
-// Hindi in Devanagari script
-profanity.check('इस वाक्य में लंड शब्द है।');  // true
+### `loadLanguages(languages: string[]): number`
 
-// Hindi in Roman script
-profanity.check('Is vakya mein lund shabd hai.');  // true
+Load multiple built-in languages at once.
 
-// Hinglish abbreviations
-profanity.check('Usne bc kaha.');  // true
+```typescript
+profanity.loadLanguages(['english', 'french', 'german']);
+profanity.check('Das ist scheiße.'); // true (German)
 ```
 
-#### Indian Languages
+---
 
-AllProfanity supports multiple Indian languages including Bengali, Tamil, and Telugu in both their native scripts and Roman transliterations.
+### `loadIndianLanguages(): number`
 
-```javascript
-// Bengali in Bengali script
-profanity.check('এই বাক্যে বাল শব্দ আছে।');  // true
+Convenience: Load all major Indian language packs.
 
-// Tamil in Tamil script
-profanity.check('இந்த வாக்கியத்தில் கூதி உள்ளது.');  // true
+```typescript
+profanity.loadIndianLanguages();
+profanity.check('यह एक बेंगाली गाली है।'); // true (Bengali)
+profanity.check('This is a Tamil profanity: புண்டை'); // true
+```
 
-// Telugu in Telugu script
-profanity.check('ఈ వాక్యంలో పూకు పదం ఉంది.');  // true
+---
 
-// Loading all Indian languages at once
-import { AllProfanity } from 'allprofanity';
-const filter = new AllProfanity();
-filter.loadIndianLanguages();  // Loads Hindi, Bengali, Tamil, and Telugu
-```
+### `loadCustomDictionary(name: string, words: string[]): void`
 
-#### European Languages
+Add your own dictionary as an additional language.
 
-AllProfanity also supports several European languages including French, German, and Spanish.
+```typescript
+profanity.loadCustomDictionary('swedish', ['fan', 'jävla', 'skit']);
+profanity.loadLanguage('swedish');
+profanity.check('Det här är skit.'); // true
+```
 
-```javascript
-// French example
-profanity.check('Cette phrase contient le mot merde.');  // true
+---
 
-// German example
-profanity.check('Dieser Satz enthält das Wort scheisse.');  // true
+### `getLoadedLanguages(): string[]`
 
-// Spanish example
-profanity.check('Esta frase contiene la palabra mierda.');  // true
+Returns the names of all currently loaded language packs.
+
+```typescript
+console.log(profanity.getLoadedLanguages()); // ['english', 'hindi', ...]
 ```
 
-### Loading Additional Languages
+---
 
-By default, only English and Hindi are loaded. You can load additional languages as needed:
+### `getAvailableLanguages(): string[]`
 
-```javascript
-// Load individual languages
-profanity.loadLanguage('bengali');
-profanity.loadLanguage('tamil');
-profanity.loadLanguage('french');
+Returns the names of all available built-in language packs.
 
-// Load multiple languages at once
-profanity.loadLanguages(['telugu', 'german', 'spanish']);
+```typescript
+console.log(profanity.getAvailableLanguages());
+// ['english', 'hindi', 'french', 'german', 'spanish', 'bengali', 'tamil', 'telugu']
+```
 
-// Get available languages
-const availableLanguages = profanity.getAvailableLanguages();
-// => ['hindi', 'bengali', 'tamil', 'telugu', 'french', 'german', 'spanish']
+---
 
-// Get currently loaded languages
-const loadedLanguages = profanity.getLoadedLanguages();
-// => ['english', 'hindi', ...]
-```
+### `clearList(): void`
+
+Remove all loaded languages and dynamic words (start with a clean filter).
 
-### Future Language Support
+```typescript
+profanity.clearList();
+profanity.check('fuck'); // false
+profanity.loadLanguage('english');
+profanity.check('fuck'); // true
+```
 
-AllProfanity is designed with extensibility in mind. If you'd like to contribute language packs, please see the Contributing section below.
+---
 
-## Mixed Language Content
+### `getConfig(): Partial<AllProfanityOptions>`
 
-AllProfanity effectively handles mixed-language content containing profanities from different languages:
+Get the current configuration.
 
-```javascript
-profanity.check('This English sentence has chutiya which is bad.');  // true
-profanity.check('I'm saying मादरचोद and bullshit in one sentence.');  // true
+```typescript
+console.log(profanity.getConfig());
+/*
+{
+  defaultPlaceholder: '*',
+  enableLeetSpeak: true,
+  caseSensitive: false,
+  strictMode: false,
+  detectPartialWords: false,
+  languages: [...],
+  whitelistWords: [...]
+}
+*/
 ```
 
-## Customizing The Library
+---
 
-### Adding Custom Profanity Lists
+## Severity Levels
 
-You can add your own profanity lists to extend support for other languages or add additional words to existing languages:
+Severity reflects the number and variety of detected profanities:
 
-```javascript
-// Add custom profanity words
-profanity.add([
-  'customword1',
-  'customword2',
-  'customword3'
-]);
+| Level     | Enum Value | Description                             |
+|-----------|------------|-----------------------------------------|
+| MILD      | 1          | 1 unique/total word                     |
+| MODERATE  | 2          | 2 unique or total words                 |
+| SEVERE    | 3          | 3 unique/total words                    |
+| EXTREME   | 4          | 4+ unique or 5+ total profane words     |
 
-// Now it will detect Spanish profanity
-profanity.check('Este es un ejemplo de mierda.');  // true
+---
+
+## Language Support
+
+- **Built-in:** English, Hindi, French, German, Spanish, Bengali, Tamil, Telugu
+- **Scripts:** Latin/Roman, Devanagari, Tamil, Telugu, Bengali, etc.
+- **Mixed Content:** Handles mixed-language and code-switched sentences.
+
+```typescript
+profanity.check('This is bullshit and चूतिया.'); // true (mixed English/Hindi)
+profanity.check('Ce mot est merde and पागल.');   // true (French/Hindi)
 ```
 
-### Creating a Custom-Configured Instance
+---
 
-If you need multiple differently-configured instances of the filter, you can import the AllProfanity class directly:
+## Use Exported Wordlists
 
-```javascript
-import { AllProfanity } from 'allprofanity';
+For sample words in a language (for UIs, admin, etc):
 
-// Create custom instances
-const kidSafeFilter = new AllProfanity({ includeModerate: true });
-const adultFilter = new AllProfanity({ includeModerate: false });
+```typescript
+import { englishBadWords, hindiBadWords } from 'allprofanity';
+console.log(englishBadWords.slice(0, 5)); // ["fuck", "shit", ...]
 ```
 
-## Advanced Use Cases
+---
 
-### Performance Optimization
+## Security
 
-For applications processing large volumes of text:
+- **No wordlist exposure:** There is no `.list()` function for security and encapsulation. Use exported word arrays for samples.
+- **TRIE-based:** Scales easily to 50,000+ words.
+- **Handles leet-speak:** Catches obfuscated variants like `f#ck`, `a55hole`.
 
-```javascript
-// Pre-compile your most used strings for faster checking
-const badWordsList = profanity.list();
-const preCompiledRegex = new RegExp('\\b(' + badWordsList.join('|') + ')\\b', 'i');
+---
 
-function quickCheck(text) {
-  return preCompiledRegex.test(text);
-}
-```
+## Full Example
 
-### Content Moderation Systems
-
-```javascript
-function moderateContent(content) {
-  if (profanity.check(content)) {
-    return {
-      isApproved: false,
-      cleanedContent: profanity.cleanWithWord(content, '[INAPPROPRIATE]'),
-      reason: 'Contains profanity'
-    };
-  }
-  return { isApproved: true, cleanedContent: content };
-}
-```
+```typescript
+import profanity, { ProfanitySeverity } from 'allprofanity';
 
-## Use Cases
+// Multi-language detection
+profanity.loadLanguages(['english', 'french', 'tamil']);
+console.log(profanity.check('Ce mot est merde.')); // true
 
-- Content moderation systems
-- Chat applications
-- User-generated content platforms
-- Educational software
-- Forums and community platforms
-- Social media content filtering
-- Comment sections
-- Gaming chat filters
-- Email filtering
-- Document processing systems
+// Leet-speak detection
+console.log(profanity.check('You a f#cking a55hole!')); // true
 
-## Browser Support
+// Whitelisting
+profanity.addToWhitelist(['anal', 'ass']);
+console.log(profanity.check('He is an associate professor.')); // false
 
-AllProfanity works in all modern browsers and Node.js environments.
+// Severity
+const result = profanity.detect('This is fucking bullshit and chutiya.');
+console.log(ProfanitySeverity[result.severity]); // "SEVERE"
 
-## Roadmap
+// Custom dictionary
+profanity.loadCustomDictionary('pirate', ['barnacle-head', 'landlubber']);
+profanity.loadLanguage('pirate');
+console.log(profanity.check('You barnacle-head!')); // true
 
-- Add support for more languages (Spanish, French, German, Arabic, etc.)
-- Contextual profanity detection
-- Severity levels for different categories of profanity
-- Phonetic matching for evasion attempts
-- Plugin system for custom detection strategies
+// Placeholder configuration
+profanity.setPlaceholder('#');
+console.log(profanity.clean('This is bullshit.')); // "This is ########."
+profanity.setPlaceholder('*'); // Reset
+```
 
-## License
+---
 
-This project is licensed under the MIT License - see the [LICENSE](https://github.com/ayush-jadaun/allprofanity/blob/main/LICENSE) file for details.
+## FAQ
 
-## Contributing
+**Q: How do I see all loaded profanities?**  
+A: For security, the internal word list is not exposed. Use `englishBadWords` etc. for samples.
+
+**Q: How do I reset the filter?**  
+A: Use `clearList()` and reload languages/dictionaries.
 
-Contributions are welcome! Whether it's adding new language support, improving detection algorithms, or enhancing documentation.
+**Q: Is this safe for browser and Node.js?**  
+A: Yes! AllProfanity is universal.
 
-Please feel free to submit a Pull Request or open an Issue on GitHub.
+---
 
-### Adding a New Language
+## Roadmap
 
-To add support for a new language:
+- More language packs (Arabic, Russian, etc.)
+- Contextual detection & severity scoring
+- Phonetic/typo/obfuscation resilience
+- Plugin system for custom detection
 
-1. Create a new file in the `src/languages` directory (e.g., `french-words.ts`)
-2. Follow the format of the existing word lists
-3. Submit a pull request with your changes
+---
 
-## Acknowledgements
+## License
 
-- Built on top of [leo-profanity](https://github.com/jojoee/leo-profanity)
+MIT — See [LICENSE](https://github.com/ayush-jadaun/allprofanity/blob/main/LICENSE)
+
+---
+
+## Contributing
 
+We welcome new language packs, detection improvements, and docs!  
+To add a new language, create a file in `src/languages/` and export a string array.  
+Open a PR or issue for bugs, features, or suggestions.