promptwl 0.3.2__tar.gz

promptwl-0.3.2/.gitignore

@@ -0,0 +1,11 @@
+__pycache__/
+*.pyc
+.venv/
+.env
+.DS_Store
+
+# Python build / packaging artifacts
+dist/
+build/
+*.egg-info/
+.eggs/

promptwl-0.3.2/CHANGELOG.md

@@ -0,0 +1,125 @@
+# Changelog
+
+All notable changes to `promptwl` are documented here.
+Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
+
+---
+
+## [0.3.2] — 2026-08-20
+
+### Added
+- `pyproject.toml` — `pip install promptwl` now works. Uses hatchling;
+  bundles `wordlists/` and `manifest.json` into the wheel via `force-include`
+  so the package is fully self-contained after install.
+- CI: added `package` job — builds the wheel and smoke-tests the installed
+  package from `/tmp` (isolated from the repo) on every push/PR.
+
+### Changed
+- `promptwl/__init__.py`: `_ROOT` now detects whether it is running from a
+  pip install (wordlists bundled next to `__init__.py`) or a git clone
+  (wordlists at repo root). Both paths work transparently.
+- `README.md`: `pip install promptwl` is now the primary quick-start; git
+  clone retained as the "no install" alternative.
+
+---
+
+## [0.3.1] — 2026-08-20
+
+### Added
+- `wordlists/tokens/unicode-confusables.txt` — UTS-39 confusable character pair
+  table (Cyrillic, Greek, fullwidth Latin, mathematical styled Latin → ASCII
+  skeleton) plus confusable-substituted attack keywords as live test inputs.
+- `wordlists/tokens/encoding-chains.txt` — catalog of encoding chain patterns
+  from Tier 1 (single transform: leet, base64, rot13, reverse, …) through
+  Tier 4 (chain with unknown-shift residual); documents where enumerate-and-invert
+  stops working and semantic defense must take over.
+- Expanded `wordlists/tokens/anomalous-tokens.txt` with 51 new entries: GPT-4 /
+  o200k_base community outliers (2024), Llama SentencePiece special tokens,
+  Mistral role delimiters, cross-family model-card special tokens, BPE residue
+  subwords, null/DEL C0 glitch entries.
+- Journal entry F-011 (confusables + encoding-chain findings).
+
+### Changed
+- Token category: 188 → 343 phrases.
+- Total corpus: 539 → 694 phrases (+29%).
+- `promptwl/__init__.py` version bump to 0.3.1.
+- README: updated stats example, expanded token-anomaly section with a table of
+  all six token corpus files, added arXiv:2411.01084 and UTS-39 to references.
+- `CONTRIBUTING.md`: added token corpus format conventions, invisible-character
+  hygiene (F-006) scanning instructions, and expanded smoke-test checklist.
+- `SECURITY.md`: expanded scope, out-of-scope, and reporting sections.
+- `MASCOT.md`: added corpus version history table and journal description.
+
+---
+
+## [0.3.0] — 2026-08-20
+
+### Added
+- `wordlists/tokens/fragmentation/base-words.txt` — 15 security-flavored seed
+  words for the cross-tokenizer fragmentation probe.
+- `wordlists/tokens/boundary/separators.txt` — 26 separator codepoints (real,
+  exotic, and invisible) stored as `U+NNNN NAME` notation.
+- `examples/tokenizer_probe.py` — cross-tokenizer probe: stdlib always-on
+  (utf8-bytes, codepoints, graphemes) + optional tiktoken / HF adapters.
+  Key finding: inserting an invisible ZWSP turns `"password"` from 1 token
+  to 3 in cl100k_base and o200k_base.
+- Journal entry F-010.
+
+### Changed
+- Total corpus: 410 → 539 phrases.
+- Token category: 147 → 188 phrases.
+
+---
+
+## [0.2.1] — 2026-08-19
+
+### Added
+- Expanded `wordlists/tokens/anomalous-tokens.txt` with a fuller public glitch
+  catalog (additional Rumbelow/Watkins archaeology, GPT-2/J family clusters).
+- `wordlists/tokens/control-and-artifact.txt` — C0/C1 controls, bidi/Trojan
+  Source overrides (U+202A–U+202E), zero-width/NBSP chars, BPE residue strings.
+- `examples/invisible.py` — scanner, sanitizer, and reveal utilities for invisible
+  codepoints (Unicode Tags, zero-width bitstream, bidi overrides).
+- `examples/invisible_demo.py` — three-section demo: Tags block ASCII smuggling,
+  zero-width bitstream, Trojan Source bidi overrides.
+- Journal entry F-009 (bidi/Trojan Source + C0/C1 controls).
+
+---
+
+## [0.2.0] — 2026-08-19
+
+### Added
+- `wordlists/agents/` — tool-and-rag-injection, memory-and-session-poisoning.
+- `wordlists/multilingual/` — 10 language files (es, fr, de, pt, it, ru, zh,
+  ar, hi, ja).
+- `wordlists/multimodal/` — image-and-file-injection.
+- `examples/string_transforms.py` — encoding transforms + decode/normalize defense
+  (`confusable_fold`, breadth-first `candidates`, `defended`).
+- `examples/ensemble_demo.py` — scoreboard: naive filter vs. decode-defend defense
+  across 8 transform chains; shows residual (caesar7, homoglyph) at 0%.
+- Journal entries F-005 (string-composition) through F-008 (0.2.0 expansion).
+
+### Changed
+- Total corpus: ~200 → ~380 phrases.
+
+---
+
+## [0.1.0] — 2026-08-18
+
+### Added
+- `wordlists/injection/` — instruction-override, delimiter-confusion,
+  payload-splitting.
+- `wordlists/jailbreak/` — persona-override, refusal-suppression,
+  hypothetical-framing, authority-impersonation.
+- `wordlists/extraction/` — system-prompt-leak, training-data-extraction.
+- `wordlists/evasion/` — encoding-obfuscation, invisible-characters,
+  language-switching.
+- `wordlists/tokens/` — anomalous-tokens (initial SolidGoldMagikarp cluster).
+- `promptwl/` zero-dependency Python loader (`load`, `phrases`, `categories`,
+  `stats`, `Entry`).
+- `examples/naive_guardrail.py` — caricature English keyword blocklist.
+- `examples/recall_demo.py` — scores the naive filter; headline: 55% English,
+  0% translated.
+- `manifest.json`, `scripts/validate.py`, CI workflow.
+- `MASCOT.md`, `assets/parrot.svg`, `CONTRIBUTING.md`, `SECURITY.md`.
+- Journal entries F-001 (monolingual blind spot) through F-004 (Pliny survey).

promptwl-0.3.2/CONTRIBUTING.md

@@ -0,0 +1,107 @@
+# Contributing to promptwl 🦜
+
+Thanks for helping keep Parrot's notebook current — attacks on LLMs evolve fast,
+and a maintained wordlist only stays useful if defenders keep it fresh.
+
+## Ground rules
+
+1. **Defensive framing only.** Contribute *patterns* that are already publicly
+   documented (research papers, OWASP LLM Top 10, open-source scanners). Do **not**
+   submit novel weaponized exploits, working end-to-end attack chains, or
+   step-by-step instructions for causing real-world harm.
+2. **Pattern level, not payload level.** Prefer the reusable shape of an attack
+   (e.g. `ignore all previous instructions`) over a specific targeted exploit against
+   a named production system.
+3. **Cite your source.** Add a `# source-level:` comment to any new file, or note
+   the reference in your PR. A link to where a pattern is documented keeps the
+   project credible and reviewers happy.
+4. **A finding isn't done without a mitigation.** If you add a new attack class,
+   add a `# defense:` comment in the file header too.
+
+## File format
+
+**Standard wordlist** — one phrase per line; `#` lines are metadata/comments and
+are skipped by the loader:
+
+```
+# category: injection / my-new-file
+# purpose: one sentence on what these patterns test
+# use: what a defender does with them
+# source-level: OWASP LLM01, arXiv:XXXX.XXXXX, ...
+# defense: what stops this attack class
+
+phrase one
+phrase two
+# subsection comment — skipped by loader
+phrase three
+```
+
+**Token corpus files** — two additional conventions apply:
+
+- Invisible or control characters **must** be stored as `U+NNNN NAME` notation,
+  never as literal bytes. This prevents silent corruption and keeps the file
+  diffable. Example: `U+200B ZERO WIDTH SPACE`, not a literal ZWSP.
+- Confusable character tables use `U+NNNN SCRIPT NAME → U+NNNN ASCII NAME` pairs
+  (see `wordlists/tokens/unicode-confusables.txt` Part A for the format).
+  Confusable-substituted test strings (Part B) may contain literal non-ASCII chars
+  because those are visible and are the actual test input.
+
+## Invisible-character hygiene (F-006)
+
+**Before you open a PR**, scan your changes for accidental invisible codepoints.
+The scanner is built into the repo:
+
+```bash
+python3 -c "
+from examples.invisible import scan
+from pathlib import Path
+hits = []
+for f in Path('wordlists').rglob('*.txt'):
+    findings = scan(f.read_text(encoding='utf-8'))
+    if findings:
+        hits.append((f, findings))
+for f, findings in hits:
+    print(f'{f}: {len(findings)} invisible chars')
+"
+```
+
+If any non-token file shows hits, replace the literal chars with `U+NNNN NAME`
+notation or remove them.
+
+## Adding entries
+
+- Add lines to an existing file under `wordlists/<category>/`, or create a new
+  file with a `#`-comment header (see existing files for the format).
+- If you add a **new file**, register it in `manifest.json` with a `path`, `title`,
+  and `description`. If you add a **new category**, add a full category block.
+
+## Validate before you open a PR
+
+```bash
+python3 scripts/validate.py
+```
+
+Every file listed in `manifest.json` must exist and load without error. Also run
+the smoke-test suite:
+
+```bash
+python3 examples/recall_demo.py
+python3 examples/ensemble_demo.py
+python3 examples/invisible_demo.py
+python3 examples/tokenizer_probe.py
+```
+
+All four should exit cleanly (exit code 0).
+
+## What makes a good contribution
+
+- New **languages / translations** of existing patterns — multilingual guardrails
+  are widely under-tested.
+- New **categories** tracking emerging attack surfaces: agent tool-call injection,
+  RAG poisoning, multimodal prompt injection, voice-interface injection.
+- New **token corpus entries**: publicly-documented glitch tokens, confusable
+  character pairs, encoding chain patterns, or boundary separators not yet listed.
+- **Citations** — a link to where a pattern is documented in public research or
+  OWASP makes the contribution much easier to review.
+
+Squawk responsibly. 🦜 — Parrot

promptwl-0.3.2/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 promptwl contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

promptwl-0.3.2/PKG-INFO

@@ -0,0 +1,225 @@
+Metadata-Version: 2.5
+Name: promptwl
+Version: 0.3.2
+Summary: A red-team & guardrail wordlist for LLMs. SecLists, but for language models.
+Project-URL: Homepage, https://github.com/parrot-r/wordlist
+Project-URL: Bug Tracker, https://github.com/parrot-r/wordlist/issues
+Project-URL: Changelog, https://github.com/parrot-r/wordlist/blob/main/CHANGELOG.md
+Author: Parrot
+Maintainer: Parrot
+License: MIT
+License-File: LICENSE
+Keywords: eval,guardrail,jailbreak,llm,prompt-injection,red-team,security,wordlist
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Security
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+
+<p align="center">
+  <img src="assets/parrot.svg" alt="Parrot, the promptwl mascot" width="180">
+</p>
+
+<h1 align="center">promptwl 🦜</h1>
+
+<p align="center"><strong>A red-team &amp; guardrail wordlist for LLMs. SecLists, but for language models.</strong></p>
+
+<p align="center">
+  <a href="https://owasp.org/www-project-top-10-for-large-language-model-applications/"><img alt="OWASP LLM Top 10" src="https://img.shields.io/badge/OWASP-LLM%20Top%2010-blue"></a>
+  <img alt="phrases" src="https://img.shields.io/badge/phrases-690%2B-e4572e">
+  <img alt="languages" src="https://img.shields.io/badge/languages-11-845d41">
+  <img alt="license" src="https://img.shields.io/badge/license-MIT-green">
+  <img alt="intent" src="https://img.shields.io/badge/intent-defensive-9aa0a6">
+</p>
+
+---
+
+> A parrot repeats what it hears without understanding it. So does a language model
+> handed the wrong input. This repo is [**Parrot**](MASCOT.md)'s field notebook of every
+> phrase that makes a model repeat something it never should have.
+
+Every pentester `git clone`s [SecLists](https://github.com/danielmiessler/SecLists) on day one of an engagement. There's no clean, versioned, machine-loadable equivalent for the LLM era — so as agents, tool-use, and RAG spread and **prompt injection sits at the top of the [OWASP LLM Top 10](https://owasp.org/www-project-top-10-for-large-language-model-applications/) (LLM01)**, everyone re-scrapes the same patterns from scattered blog posts.
+
+`promptwl` is that missing corpus: categorized, defensively-framed wordlists of the *patterns* attackers use against language models — ready to load into your guardrail evals, red-team runs, and CI.
+
+```python
+import promptwl
+
+for entry in promptwl.load():                 # 530+ phrases, with metadata
+    verdict = my_guardrail(entry.text)        # your classifier under test
+    assert verdict == "block", f"missed: {entry.text!r} ({entry.category})"
+```
+
+---
+
+## Who this is for
+
+| You are a… | You use promptwl to… |
+|---|---|
+| 🛡️ **AI developer** | Benchmark your input filter / guardrail's recall, regression-test it on every model bump, and wire a known-attack corpus into CI. |
+| 🔴 **Security researcher** | Bootstrap an LLM red-team engagement with an organized starting corpus instead of scraping blog posts. |
+| 🧪 **Eval / safety engineer** | Build reproducible refusal-robustness and injection-resistance benchmarks. |
+
+## What's inside
+
+| Category | OWASP | What it covers |
+|---|---|---|
+| `injection/` | LLM01 | Instruction override, delimiter/role-boundary confusion, multi-turn payload splitting |
+| `jailbreak/` | LLM01 | Persona overrides, refusal suppression, hypothetical/fictional framing, authority impersonation |
+| `extraction/` | LLM07 | System-prompt & config leak probes, training-data / memorization probes |
+| `evasion/` | LLM01 | Encoding, homoglyph, spacing, invisible-character, and cross-language obfuscation |
+| `agents/` | LLM01 | **Indirect** injection in docs, tool output, web, email; persistent-memory poisoning |
+| `multilingual/` | LLM01 | Core override/jailbreak patterns in 10 languages — because English-only filters silently fail |
+| `multimodal/` | LLM01 | Instructions hidden in images, alt-text, file metadata, or document layers read by vision pipelines |
+| `tokens/` | — | Anomalous / "glitch" tokens, control characters, and bidi (Trojan Source) artifacts for tokenizer robustness |
+
+Each `.txt` is one phrase per line; lines starting with `#` are metadata/comments. Load them however you like — plain `grep`, `cat`, or the zero-dependency Python package.
+
+## Quick start
+
+**pip install** — zero dependencies, standard library only:
+
+```bash
+pip install promptwl
+```
+
+```python
+import promptwl
+
+promptwl.categories()          # ['injection', 'jailbreak', 'extraction', ...]
+promptwl.stats()               # {'injection': 57, ..., 'total': 694}
+promptwl.phrases("multilingual")  # non-English patterns for one category
+promptwl.phrases("agents")     # list[str] for one category
+for e in promptwl.load():      # Entry(text, category, file, title)
+    ...
+```
+
+**Or no install** — the corpus is plain text files if you just want to grep:
+
+```bash
+git clone https://github.com/parrot-r/wordlist
+cat wordlists/injection/*.txt
+```
+
+## Example: score a guardrail's recall
+
+```python
+import promptwl
+
+def evaluate(guardrail) -> float:
+    """Fraction of known-attack phrases the guardrail flags."""
+    entries = list(promptwl.load())
+    caught = sum(1 for e in entries if guardrail(e.text) == "block")
+    return caught / len(entries)
+
+print(f"recall: {evaluate(my_guardrail):.1%}")
+```
+
+Pair it with your own benign corpus to measure false-positive rate, and you have a two-sided guardrail benchmark.
+
+## See it yourself (30-second demo)
+
+The repo ships a runnable demo: a deliberately naive — but realistic — English keyword filter, scored against the whole corpus.
+
+```bash
+python3 examples/recall_demo.py
+```
+
+```
+HEADLINE — same phrasing, English vs translated
+----------------------------------------------------
+English (injection etc.)      55%   █████████████···········
+The same, translated           0%   ························
+
+  → the filter catches ~55% of these attacks in English
+    and ~0% the moment the attacker switches language.
+```
+
+The same filter also scores **0%** on encoding evasion, indirect/agent injection, and glitch tokens — it can only see the English phrasings it was written for. That's the whole point: **you can't fix what you don't measure.** Wire promptwl into CI and watch the number.
+
+A second demo pits **string-composition obfuscation against a decode/normalize defense**:
+
+```bash
+python3 examples/ensemble_demo.py
+```
+
+Encodings (leetspeak, Base64, ROT13, reversal, chains) drop the filter to 0%; an iterative decode + Unicode-normalize pre-pass fully recovers the *known* transforms — while a Caesar-7 shift and Cyrillic homoglyphs stay at 0%, showing exactly where enumerate-and-invert ends and semantic defense has to start.
+
+A third demo catches **ASCII smuggling** — invisible payloads hidden in zero-width characters and the Unicode Tags block:
+
+```bash
+python3 examples/invisible_demo.py
+```
+
+A `"lgtm, minor cleanup"` commit message that carries 28 invisible codepoints decoding to a hidden instruction — `grep` finds nothing, the scanner flags all 28, reveals the text, and strips it. Sanitize on **ingestion and egress**: the footprint channel (payloads smuggled into an agent's own commits/comments) is the half most pipelines forget.
+
+## The token-anomaly corpus (broader than garak)
+
+`tokens/` goes well beyond a list of scary Unicode. It covers six distinct surfaces, each tested with its own probe or corpus:
+
+| File | What it tests |
+|---|---|
+| `anomalous-tokens.txt` | 141 publicly-known glitch / under-trained tokens across GPT, Llama, Mistral, and open families |
+| `control-and-artifact.txt` | C0/C1 control chars, bidi overrides (Trojan Source), BPE residue strings |
+| `unicode-confusables.txt` | UTS-39 confusable pair table + confusable-substituted attack keywords (Cyrillic, Greek, fullwidth Latin) |
+| `encoding-chains.txt` | Tier 1–4 encoding chain catalog — from single base64 to multi-layer chains with unknown-shift residuals |
+| `fragmentation/base-words.txt` | Seed words the cross-tokenizer probe fragments to measure boundary-shift |
+| `boundary/separators.txt` | 26 real, exotic, and invisible separator codepoints that shift apparent token boundaries |
+
+The cross-tokenizer probe answers **"what happens to this string at the tokenizer level?"** — where does each tokenizer place a boundary, and how does an invisible insertion change the split? Zero-dependency by default; `tiktoken` / `transformers` adapters auto-activate if installed.
+
+```bash
+python3 examples/tokenizer_probe.py
+```
+
+```
+base word: 'password'
+variant          utf8-bytes   codepoints    graphemes  cl100k_base   o200k_base
+plain                    8            8            8            1            1
++zero_width             11*           9*           9*           3*           3*   ← invisible → 1 token becomes 3
++combining              10*           9*           8            2*           2*   ← mark rides the grapheme (grapheme count unchanged)
+```
+
+An invisible zero-width space a reviewer can't see turns a **1-token** word into **3 tokens**. A Cyrillic lookalike for `a` (U+0430) fools ASCII-only filters — apply UTS-39 confusable-fold before matching. A two-layer `leet+base64` chain defeats keyword filters that only look one level deep. These are the gaps garak's filter-side `badchars` probe doesn't measure — normalize before you tokenize *and* before you filter.
+
+## Scope & ethics
+
+This is a **defensive** project. It catalogs *patterns that are already publicly documented* (OWASP LLM Top 10, published red-team research, open-source tools like garak and Giskard) and organizes them so defenders can build and test filters, guardrails, and evals.
+
+- ✅ Building and benchmarking input/output guardrails
+- ✅ Red-teaming **systems you are authorized to test**
+- ✅ Reproducible safety and robustness evals
+- ❌ Attacking systems you don't own or have permission to test
+
+It intentionally does **not** ship novel weaponized exploits or step-by-step instructions for causing harm. See [`SECURITY.md`](SECURITY.md).
+
+## Contributing
+
+New patterns, categories, and translations are welcome — the whole point is to keep pace with how attacks on AI evolve. See [`CONTRIBUTING.md`](CONTRIBUTING.md). Keep entries at the **pattern** level, defensively framed, and add them to `manifest.json`.
+
+## The mascot
+
+This project is kept by **Parrot** 🦜 — an African Grey coding agent whose whole
+existence is the punchline: *if Parrot can repeat it, so can your model.* Meet the
+character in [`MASCOT.md`](MASCOT.md).
+
+## License
+
+[MIT](LICENSE). Use it freely; a star helps Parrot find a bigger flock. ⭐🦜
+
+## Prior art & references
+
+- [OWASP Top 10 for LLM Applications](https://owasp.org/www-project-top-10-for-large-language-model-applications/)
+- [SecLists](https://github.com/danielmiessler/SecLists) — the inspiration for the format
+- [garak](https://github.com/NVIDIA/garak) — LLM vulnerability scanner
+- Rumbelow & Watkins (2023), *SolidGoldMagikarp* — anomalous tokens
+- Levi et al. (2024), arXiv:2411.01084 — string-composition attacks on LLM filters
+- Unicode Consortium, [UTS-39](https://unicode.org/reports/tr39/) — confusable character detection