lime-audit 0.1.0__tar.gz

lime_audit-0.1.0/.gitignore

@@ -0,0 +1,27 @@
+.env
+.env.local
+audit/__pycache__
+__pycache__/
+skills-lock.json
+*.pyc
+*.pyo
+REVIEW.md
+venv/
+.venv/
+backend/venv/
+backend/.venv/
+
+node_modules/
+.next/
+out/
+
+*.egg-info/
+dist/
+build/
+
+.cache/
+.DS_Store
+
+.agents
+Decisions.md
+Rules.md

lime_audit-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Parshvi Jain
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

lime_audit-0.1.0/Makefile

@@ -0,0 +1,20 @@
+.PHONY: audit analyse charts verify clean
+
+PYTHON ?= .venv/Scripts/python
+
+audit:
+	$(PYTHON) -m audit.runner
+	$(PYTHON) -m audit.analyse_results
+	$(PYTHON) -m audit.generate_charts
+
+analyse:
+	$(PYTHON) -m audit.analyse_results
+
+charts:
+	$(PYTHON) -m audit.generate_charts
+
+verify:
+	$(PYTHON) -c "from audit.verify import verify_reproducibility; verify_reproducibility()"
+
+clean:
+	$(PYTHON) -c "import glob, os; [os.remove(f) for f in glob.glob('audit/results/*.csv') + glob.glob('audit/results/*.json') + glob.glob('audit/results/*.png')]"

lime_audit-0.1.0/PKG-INFO

@@ -0,0 +1,248 @@
+Metadata-Version: 2.5
+Name: lime-audit
+Version: 0.1.0
+Summary: Audit LIME explanations for stability and faithfulness on any HuggingFace text classifier
+Project-URL: Homepage, https://github.com/parshvi1508/XAI_Forensic
+Project-URL: Documentation, https://github.com/parshvi1508/XAI_Forensic#readme
+Project-URL: Issues, https://github.com/parshvi1508/XAI_Forensic/issues
+Author-email: Parshvi Jain <parshvijain1508@gmail.com>
+License: MIT
+License-File: LICENSE
+Keywords: LIME,NLP,XAI,audit,explainability,transformers
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Requires-Dist: lime>=0.2
+Requires-Dist: matplotlib>=3.7
+Requires-Dist: numpy>=1.24
+Requires-Dist: scikit-learn>=1.2
+Requires-Dist: scipy>=1.10
+Requires-Dist: torch>=2.0
+Requires-Dist: transformers>=4.30
+Provides-Extra: dev
+Requires-Dist: build; extra == 'dev'
+Requires-Dist: pytest; extra == 'dev'
+Requires-Dist: twine; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# XAI Forensics
+
+A diagnostic tool for evaluating when LIME token attributions are trustworthy on transformer sentiment classifiers. Runs three independent forensic checks (attribution stability, counterfactual faithfulness, cross-model agreement) and validates results against a 30-input pre-registered audit.
+
+**Key finding:** high-confidence predictions (>95%) produce the least stable attributions (mean Jaccard 0.46 on strong baselines), while lexical-shortcut inputs are perfectly faithful (Jaccard 1.0, 100% flip rate). This tool shows you when LIME is signal vs. noise.
+
+Sentiment classification is the controlled test task. The project evaluates the explanation method, not the classifier output.
+
+
+
+## Live Demo
+
+| Component | Link |
+|-----------|------|
+| Frontend | [xai-forensic.vercel.app](https://xai-forensic.vercel.app/) |
+| Backend API | [jainparshvi-xai-forensics-backend.hf.space](https://jainparshvi-xai-forensics-backend.hf.space) |
+| API Docs (Swagger) | [/docs](https://jainparshvi-xai-forensics-backend.hf.space/docs) |
+
+**Demo input:** `I am not entirely unhappy with this result.`
+
+This sentence contains a double negation that causes the two models to genuinely disagree.
+
+> The backend runs on Hugging Face Spaces free tier. If the Space has been idle, the first request triggers a cold start (30-60 seconds) while models download. Subsequent requests are faster.
+
+
+
+![alt text](public/image.png)
+![alt text](public/image-1.png)
+![alt text](public/image-2.png)
+![alt text](public/image-3.png)
+![alt text](public/image-4.png)
+![alt text](public/image-5.png)
+
+## What This Does
+
+XAI Forensics runs three independent diagnostic checks on any short English text:
+
+1. **WHY (Attribution Stability)** - Tests whether LIME can produce a stable, reproducible token ranking for a given prediction. Seeds the random state for deterministic output.
+2. **FLIP (Counterfactual Faithfulness)** - Tests whether the tokens LIME identifies as important are actually causally influential. Removes the top-attributed word and measures the real confidence shift.
+3. **DISAGREE (Cross-Model Consistency)** - Tests whether the prediction itself is domain-stable enough to warrant attribution analysis. Compares DistilBERT-SST2 against Twitter-RoBERTa.
+
+Each check targets a different failure mode of LIME: instability under re-sampling, unfaithfulness to the model's actual reasoning, and domain sensitivity of the underlying prediction.
+
+### Audit Results
+
+A pre-registered 30-input audit (5 seeds each, 150 total LIME runs) found:
+- Overall mean Jaccard stability: **0.81** (passes 0.6 threshold)
+- Deletion faithfulness direction correct: **89.3%** (passes 70% threshold)
+- Strong baselines category (high confidence): **Jaccard 0.67** (lowest category, driven by redundant evidence)
+- Lexical shortcuts category: **100% label flip rate** (highest faithfulness)
+
+Full interactive results available at [/audit](https://xai-forensic.vercel.app/audit).
+
+## How It Works
+
+### WHY: LIME Token Attribution
+
+Uses [LIME](https://arxiv.org/abs/1602.04938) (Ribeiro et al., 2016) for model-agnostic local explanations. LIME generates perturbed versions of the input text, reruns the classifier on each perturbation, and fits a local linear model to estimate which tokens most influenced the prediction.
+
+- Runs 300 perturbation samples per explanation
+- Returns the top 10 tokens with signed weights (positive = pushes toward positive class)
+- SHAP was considered and rejected: slower for transformers, expensive on free-tier CPU
+- Attention weights were rejected as explanations per Jain and Wallace (2019)
+
+### FLIP: Counterfactual Word Removal
+
+Removes each word one at a time, reruns inference, and finds the word whose removal causes the largest confidence shift. Shows the full before-and-after comparison: original label, modified label, confidence delta, and whether the verdict changed.
+
+- Greedy O(n) search over words in the input
+- Deterministic, no generation model required
+- Word removal can create ungrammatical text (documented limitation)
+- Does not always flip the label on highly confident predictions
+
+### DISAGREE: Dual-Model Divergence
+
+Runs the same text through both models and computes the absolute difference in their positive-class confidence scores. A high divergence score means the models have different views on the same text, which reveals linguistic ambiguity across training domains.
+
+- Divergence = abs(positive_score_A - positive_score_B)
+- This is an interpretable confidence delta, not a formal divergence metric like KL divergence
+- KL divergence was considered and rejected: harder to interpret for non-technical audiences
+
+
+## Models Used
+
+| Model | Training Data | Strength |
+|-------|---------------|----------|
+| [distilbert-base-uncased-finetuned-sst-2-english](https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english) | SST-2 movie reviews | Formal, structured English |
+| [cardiffnlp/twitter-roberta-base-sentiment-latest](https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment-latest) | 124M tweets | Sarcasm, slang, informal tone |
+
+These two models are chosen because they genuinely disagree on informal or ambiguous language. DistilBERT expects clean, formal text. Twitter-RoBERTa handles internet language better. This domain mismatch makes the DISAGREE panel meaningful rather than artificial.
+
+## Architecture
+
+```
+Frontend (Vercel)              Backend (HF Spaces Docker)
+Next.js + Tailwind             FastAPI + PyTorch
+       |                              |
+       |--- POST /why   ------------->|--- LIME (300 perturbations)
+       |--- POST /flip  ------------->|--- Greedy word removal
+       |--- POST /disagree ---------->|--- Dual model inference
+       |                              |
+       |<---- JSON responses ---------|
+```
+
+- Frontend and backend are fully decoupled
+- All endpoints accept `{"text": "..."}` and return structured JSON
+- Frontend calls all three endpoints in parallel using `Promise.allSettled`
+- Models load once at container startup, not per request
+
+## Run it Locally
+
+### Backend
+
+```bash
+cd backend
+python -m venv venv
+source venv/bin/activate  # Windows: venv\Scripts\activate
+pip install -r requirements.txt
+uvicorn main:app --reload --port 8000
+```
+
+First run downloads models (~500MB total). Subsequent starts use cached weights.
+
+### Frontend
+
+```bash
+cd frontend
+npm install
+npm run dev
+```
+
+Frontend runs at `http://localhost:3000` and calls the backend at `http://localhost:8000` by default. To change the backend URL, set `NEXT_PUBLIC_API_URL` in `frontend/.env.local`.
+
+
+## API Endpoints
+
+All endpoints accept POST with `Content-Type: application/json` and a body of `{"text": "your input"}`.
+
+| Endpoint | Returns |
+|----------|---------|
+| `POST /why` | Predicted label, confidence, and top 10 token weights from LIME |
+| `POST /flip` | Original and modified labels, removed word, confidence delta, verdict changed status |
+| `POST /disagree` | Both model predictions, divergence score, agreement status |
+| `GET /` | Health check: `{"status": "ok"}` |
+| `GET /docs` | Interactive Swagger API documentation |
+
+Input is limited to 1000 characters. Requests with longer text return HTTP 400.
+
+
+## Latency Notes
+
+| Endpoint | Approx. CPU time | Why |
+|----------|-------------------|-----|
+| `/why` | 15-45 seconds | LIME runs 300 model inference calls per explanation |
+| `/flip` | 2-10 seconds | One model call per word in the input |
+| `/disagree` | Under 1 second | Two forward passes |
+
+LIME is the main latency source. CPU inference is used because deployment is on the Hugging Face free tier, which does not guarantee GPU availability. This is acceptable for a demo tool with short text inputs.
+
+
+## Latency Benchmark
+
+The backend runs on Hugging Face Spaces free-tier CPU. The table below reports median endpoint runtime across 3 runs after one warmup request. Timings are approximate because free-tier CPU performance varies.
+
+| Words | WHY median | FLIP median | DISAGREE median |
+|---:|---:|---:|---:|
+| 5 | 4.5s | 414ms | 98ms |
+| 10 | 5.8s | 270ms | 75ms |
+| 20 | 6.1s | 475ms | 72ms |
+| 50 | 8.3s | 4.2s | 104ms |
+
+WHY is slowest because LIME generates perturbed versions of the input and reruns model inference many times. FLIP scales with word count because it removes candidate words one at a time and reruns inference. DISAGREE is fastest because it only runs two model forward passes.
+
+The benchmark can be reproduced with `scripts/benchmark_latency.py`.
+
+
+## Security and Cost
+
+- No paid APIs. Both models are public Hugging Face models.
+- No database. Nothing is stored.
+- No authentication. This is a public demo tool.
+- No user data is collected, logged, or persisted.
+- CORS allows all origins (appropriate for a public demo with no sensitive operations).
+- Input capped at 1000 characters to prevent LIME timeouts on free-tier CPU.
+
+
+## Known Behavior and Limitations
+
+Single-word inputs are not ideal for this tool. LIME works by perturbing parts of the input and observing prediction changes. With only one word, there is very little structure to perturb, so the explanation can be unstable or uninformative. In testing, short inputs such as "Fine." can produce domain-sensitive behavior because different models interpret minimal context differently.
+
+Highly confident predictions may not flip after one-word removal. For example, strongly positive sentences such as "I absolutely love this, it is the best thing ever." often remain positive after removing one word. This does not mean every word is irrelevant. It means the model found enough evidence across the sentence that removing one token did not change the final verdict.
+
+Counterfactual removal can create ungrammatical text because the method deletes a word rather than rewriting the sentence. This is a deliberate MVP tradeoff. The FLIP panel is a fragility test, not a full natural-language counterfactual generator. For example, removing a key word from "I am not entirely unhappy with this result." can flip the verdict, but the modified sentence may not always be natural English.
+
+Additional limitations:
+
+- LIME explanation takes 15-45 seconds on CPU for short text
+- Input limited to 1000 characters
+- Hugging Face free tier sleeps after inactivity; first request after sleep has a 30-60 second cold start
+- Only tested on English text
+- Sentiment-specific; adapting to other tasks would require different models and possibly different XAI methods
+
+
+## Tech Stack
+
+| Layer | Technology |
+|-------|-----------|
+| Backend | Python, FastAPI, PyTorch, Hugging Face Transformers, LIME |
+| Frontend | Next.js, React, Tailwind CSS |
+| Backend deployment | Hugging Face Spaces (Docker) |
+| Frontend deployment | Vercel |
+| ML models | DistilBERT-SST2, Twitter-RoBERTa |
+| Infrastructure cost | Zero (free tier only) |
+
+
+## License
+
+MIT

lime_audit-0.1.0/README.md

@@ -0,0 +1,218 @@
+# XAI Forensics
+
+A diagnostic tool for evaluating when LIME token attributions are trustworthy on transformer sentiment classifiers. Runs three independent forensic checks (attribution stability, counterfactual faithfulness, cross-model agreement) and validates results against a 30-input pre-registered audit.
+
+**Key finding:** high-confidence predictions (>95%) produce the least stable attributions (mean Jaccard 0.46 on strong baselines), while lexical-shortcut inputs are perfectly faithful (Jaccard 1.0, 100% flip rate). This tool shows you when LIME is signal vs. noise.
+
+Sentiment classification is the controlled test task. The project evaluates the explanation method, not the classifier output.
+
+
+
+## Live Demo
+
+| Component | Link |
+|-----------|------|
+| Frontend | [xai-forensic.vercel.app](https://xai-forensic.vercel.app/) |
+| Backend API | [jainparshvi-xai-forensics-backend.hf.space](https://jainparshvi-xai-forensics-backend.hf.space) |
+| API Docs (Swagger) | [/docs](https://jainparshvi-xai-forensics-backend.hf.space/docs) |
+
+**Demo input:** `I am not entirely unhappy with this result.`
+
+This sentence contains a double negation that causes the two models to genuinely disagree.
+
+> The backend runs on Hugging Face Spaces free tier. If the Space has been idle, the first request triggers a cold start (30-60 seconds) while models download. Subsequent requests are faster.
+
+
+
+![alt text](public/image.png)
+![alt text](public/image-1.png)
+![alt text](public/image-2.png)
+![alt text](public/image-3.png)
+![alt text](public/image-4.png)
+![alt text](public/image-5.png)
+
+## What This Does
+
+XAI Forensics runs three independent diagnostic checks on any short English text:
+
+1. **WHY (Attribution Stability)** - Tests whether LIME can produce a stable, reproducible token ranking for a given prediction. Seeds the random state for deterministic output.
+2. **FLIP (Counterfactual Faithfulness)** - Tests whether the tokens LIME identifies as important are actually causally influential. Removes the top-attributed word and measures the real confidence shift.
+3. **DISAGREE (Cross-Model Consistency)** - Tests whether the prediction itself is domain-stable enough to warrant attribution analysis. Compares DistilBERT-SST2 against Twitter-RoBERTa.
+
+Each check targets a different failure mode of LIME: instability under re-sampling, unfaithfulness to the model's actual reasoning, and domain sensitivity of the underlying prediction.
+
+### Audit Results
+
+A pre-registered 30-input audit (5 seeds each, 150 total LIME runs) found:
+- Overall mean Jaccard stability: **0.81** (passes 0.6 threshold)
+- Deletion faithfulness direction correct: **89.3%** (passes 70% threshold)
+- Strong baselines category (high confidence): **Jaccard 0.67** (lowest category, driven by redundant evidence)
+- Lexical shortcuts category: **100% label flip rate** (highest faithfulness)
+
+Full interactive results available at [/audit](https://xai-forensic.vercel.app/audit).
+
+## How It Works
+
+### WHY: LIME Token Attribution
+
+Uses [LIME](https://arxiv.org/abs/1602.04938) (Ribeiro et al., 2016) for model-agnostic local explanations. LIME generates perturbed versions of the input text, reruns the classifier on each perturbation, and fits a local linear model to estimate which tokens most influenced the prediction.
+
+- Runs 300 perturbation samples per explanation
+- Returns the top 10 tokens with signed weights (positive = pushes toward positive class)
+- SHAP was considered and rejected: slower for transformers, expensive on free-tier CPU
+- Attention weights were rejected as explanations per Jain and Wallace (2019)
+
+### FLIP: Counterfactual Word Removal
+
+Removes each word one at a time, reruns inference, and finds the word whose removal causes the largest confidence shift. Shows the full before-and-after comparison: original label, modified label, confidence delta, and whether the verdict changed.
+
+- Greedy O(n) search over words in the input
+- Deterministic, no generation model required
+- Word removal can create ungrammatical text (documented limitation)
+- Does not always flip the label on highly confident predictions
+
+### DISAGREE: Dual-Model Divergence
+
+Runs the same text through both models and computes the absolute difference in their positive-class confidence scores. A high divergence score means the models have different views on the same text, which reveals linguistic ambiguity across training domains.
+
+- Divergence = abs(positive_score_A - positive_score_B)
+- This is an interpretable confidence delta, not a formal divergence metric like KL divergence
+- KL divergence was considered and rejected: harder to interpret for non-technical audiences
+
+
+## Models Used
+
+| Model | Training Data | Strength |
+|-------|---------------|----------|
+| [distilbert-base-uncased-finetuned-sst-2-english](https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english) | SST-2 movie reviews | Formal, structured English |
+| [cardiffnlp/twitter-roberta-base-sentiment-latest](https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment-latest) | 124M tweets | Sarcasm, slang, informal tone |
+
+These two models are chosen because they genuinely disagree on informal or ambiguous language. DistilBERT expects clean, formal text. Twitter-RoBERTa handles internet language better. This domain mismatch makes the DISAGREE panel meaningful rather than artificial.
+
+## Architecture
+
+```
+Frontend (Vercel)              Backend (HF Spaces Docker)
+Next.js + Tailwind             FastAPI + PyTorch
+       |                              |
+       |--- POST /why   ------------->|--- LIME (300 perturbations)
+       |--- POST /flip  ------------->|--- Greedy word removal
+       |--- POST /disagree ---------->|--- Dual model inference
+       |                              |
+       |<---- JSON responses ---------|
+```
+
+- Frontend and backend are fully decoupled
+- All endpoints accept `{"text": "..."}` and return structured JSON
+- Frontend calls all three endpoints in parallel using `Promise.allSettled`
+- Models load once at container startup, not per request
+
+## Run it Locally
+
+### Backend
+
+```bash
+cd backend
+python -m venv venv
+source venv/bin/activate  # Windows: venv\Scripts\activate
+pip install -r requirements.txt
+uvicorn main:app --reload --port 8000
+```
+
+First run downloads models (~500MB total). Subsequent starts use cached weights.
+
+### Frontend
+
+```bash
+cd frontend
+npm install
+npm run dev
+```
+
+Frontend runs at `http://localhost:3000` and calls the backend at `http://localhost:8000` by default. To change the backend URL, set `NEXT_PUBLIC_API_URL` in `frontend/.env.local`.
+
+
+## API Endpoints
+
+All endpoints accept POST with `Content-Type: application/json` and a body of `{"text": "your input"}`.
+
+| Endpoint | Returns |
+|----------|---------|
+| `POST /why` | Predicted label, confidence, and top 10 token weights from LIME |
+| `POST /flip` | Original and modified labels, removed word, confidence delta, verdict changed status |
+| `POST /disagree` | Both model predictions, divergence score, agreement status |
+| `GET /` | Health check: `{"status": "ok"}` |
+| `GET /docs` | Interactive Swagger API documentation |
+
+Input is limited to 1000 characters. Requests with longer text return HTTP 400.
+
+
+## Latency Notes
+
+| Endpoint | Approx. CPU time | Why |
+|----------|-------------------|-----|
+| `/why` | 15-45 seconds | LIME runs 300 model inference calls per explanation |
+| `/flip` | 2-10 seconds | One model call per word in the input |
+| `/disagree` | Under 1 second | Two forward passes |
+
+LIME is the main latency source. CPU inference is used because deployment is on the Hugging Face free tier, which does not guarantee GPU availability. This is acceptable for a demo tool with short text inputs.
+
+
+## Latency Benchmark
+
+The backend runs on Hugging Face Spaces free-tier CPU. The table below reports median endpoint runtime across 3 runs after one warmup request. Timings are approximate because free-tier CPU performance varies.
+
+| Words | WHY median | FLIP median | DISAGREE median |
+|---:|---:|---:|---:|
+| 5 | 4.5s | 414ms | 98ms |
+| 10 | 5.8s | 270ms | 75ms |
+| 20 | 6.1s | 475ms | 72ms |
+| 50 | 8.3s | 4.2s | 104ms |
+
+WHY is slowest because LIME generates perturbed versions of the input and reruns model inference many times. FLIP scales with word count because it removes candidate words one at a time and reruns inference. DISAGREE is fastest because it only runs two model forward passes.
+
+The benchmark can be reproduced with `scripts/benchmark_latency.py`.
+
+
+## Security and Cost
+
+- No paid APIs. Both models are public Hugging Face models.
+- No database. Nothing is stored.
+- No authentication. This is a public demo tool.
+- No user data is collected, logged, or persisted.
+- CORS allows all origins (appropriate for a public demo with no sensitive operations).
+- Input capped at 1000 characters to prevent LIME timeouts on free-tier CPU.
+
+
+## Known Behavior and Limitations
+
+Single-word inputs are not ideal for this tool. LIME works by perturbing parts of the input and observing prediction changes. With only one word, there is very little structure to perturb, so the explanation can be unstable or uninformative. In testing, short inputs such as "Fine." can produce domain-sensitive behavior because different models interpret minimal context differently.
+
+Highly confident predictions may not flip after one-word removal. For example, strongly positive sentences such as "I absolutely love this, it is the best thing ever." often remain positive after removing one word. This does not mean every word is irrelevant. It means the model found enough evidence across the sentence that removing one token did not change the final verdict.
+
+Counterfactual removal can create ungrammatical text because the method deletes a word rather than rewriting the sentence. This is a deliberate MVP tradeoff. The FLIP panel is a fragility test, not a full natural-language counterfactual generator. For example, removing a key word from "I am not entirely unhappy with this result." can flip the verdict, but the modified sentence may not always be natural English.
+
+Additional limitations:
+
+- LIME explanation takes 15-45 seconds on CPU for short text
+- Input limited to 1000 characters
+- Hugging Face free tier sleeps after inactivity; first request after sleep has a 30-60 second cold start
+- Only tested on English text
+- Sentiment-specific; adapting to other tasks would require different models and possibly different XAI methods
+
+
+## Tech Stack
+
+| Layer | Technology |
+|-------|-----------|
+| Backend | Python, FastAPI, PyTorch, Hugging Face Transformers, LIME |
+| Frontend | Next.js, React, Tailwind CSS |
+| Backend deployment | Hugging Face Spaces (Docker) |
+| Frontend deployment | Vercel |
+| ML models | DistilBERT-SST2, Twitter-RoBERTa |
+| Infrastructure cost | Zero (free tier only) |
+
+
+## License
+
+MIT

lime_audit-0.1.0/article.md

@@ -0,0 +1,103 @@
+# I Audited LIME on 30 Inputs and Found It Fails on the Easiest Ones
+
+LIME is one of the most cited explanation methods in machine learning. It tells you which words in a sentence drove a classifier's prediction. But does it give you the same answer twice?
+
+I ran LIME on 30 carefully chosen inputs across 5 random seeds, measured whether the explanations stayed stable, and tested whether removing the "most important" token actually changed the model's output. The input with the worst stability was not the hardest or most ambiguous one. It was "This is the best product I have ever purchased."
+
+## The Setup
+
+I froze a test set of 30 inputs before looking at any results. The inputs span six categories designed to stress-test LIME: negation and minimal pairs ("This is not good" vs. "This is good"), lexical shortcuts where surface cues contradict the true sentiment, ambiguous sentences near the decision boundary, distribution shifts from domains like clinical notes and financial reports, strong baselines that any competent model should handle easily, and edge cases like repeated tokens or alternating contradictions.
+
+The model is DistilBERT fine-tuned on SST-2 (the standard sentiment benchmark), pinned to an exact revision hash. LIME runs with 300 perturbation samples and returns the top 10 tokens ranked by attribution weight.
+
+I measure two things:
+
+**Stability**: For each input, I run LIME with five different random seeds and compute the Jaccard overlap of the top-5 attributed tokens across all 10 seed pairs. A Jaccard of 1.0 means every seed returned the same five tokens. A Jaccard of 0.25 means the explanations barely overlap.
+
+**Deletion faithfulness**: Using the canonical seed, I identify the token LIME considers most important and remove it from the input. If LIME says a token has positive weight, removing it should decrease the model's positive score. If the score barely moves, the "most important" token was not actually important.
+
+All thresholds, metrics, and the 30 inputs were frozen in a pre-registration protocol before seeing any results. The raw data, every CSV, and the runner code are public.
+
+## The Surprise: Easy Inputs Are the Least Stable
+
+The overall mean Jaccard is 0.81 [0.74, 0.87] (95% bootstrap CI), comfortably above the 0.60 threshold. That headline number looks fine. But the per-input distribution tells a different story.
+
+![LIME Attribution Stability Across 5 Random Seeds](audit/results/chart_jaccard_by_input.png)
+
+The leftmost bar in the chart, the input with the worst stability of all 29 tested inputs, is Input 21: "This is the best product I have ever purchased." Its mean Jaccard is 0.46, with a minimum of 0.25 across seed pairs. This is an unambiguously positive sentence that any sentiment model handles with near-perfect confidence (0.9999).
+
+Why does the easiest input have the worst explanation stability?
+
+The sentence has nine whitespace tokens: "This", "is", "the", "best", "product", "I", "have", "ever", "purchased". Only one or two of those carry real sentiment signal ("best", maybe "purchased"). The remaining seven are function words with near-zero LIME weights. Across seeds, LIME assigns slightly different random weights to "is", "the", "I", "have", and "ever", causing them to shuffle positions in the top-5 ranking. The top-1 token is not even consistent, because "best" receives a weight of only 0.008, barely above the noise floor.
+
+The instability is not about hard cases. It is about the signal-to-noise ratio between meaningful tokens and filler. Longer sentences with clear sentiment have more function words competing for top-5 slots with near-zero weights, making the ranking fragile.
+
+The per-category breakdown confirms this. Strong baselines, the easiest category, have the second-worst mean Jaccard (0.67). Ambiguity, a category you would expect to be unstable, actually scores 0.97 because ambiguous sentences are short and nearly every token carries some signal.
+
+| Category | Mean Jaccard | 95% CI |
+|----------|-------------|--------|
+| Ambiguity | 0.97 | [0.93, 1.00] |
+| Edge cases | 0.94 | [0.86, 1.00] |
+| Negation | 0.80 | [0.66, 0.93] |
+| Lexical shortcuts | 0.77 | [0.62, 0.92] |
+| Distribution shift | 0.72 | [0.61, 0.84] |
+| Strong baselines | 0.67 | [0.52, 0.82] |
+
+## LIME Says "This Token Matters." Deletion Says "Nothing Matters."
+
+The deletion faithfulness test reveals a second, independent problem. Overall, 89% of deletions are directionally correct: if LIME says a token is positive, removing it decreases the score. That passes the 0.70 threshold. But the magnitude tells a different story.
+
+![LIME Deletion Faithfulness: Confidence vs. Impact](audit/results/chart_faithfulness_scatter.png)
+
+The chart shows two distinct clusters. In the upper-left corner, lexical shortcuts: inputs like "The movie was terrible but I loved every minute of it." Removing the decisive token ("loved") causes a confidence swing of 0.987 and a full label flip. LIME works perfectly here because one token genuinely dominates the prediction.
+
+In the lower-right corner, strong baselines: inputs where the model is 0.9999 confident. Removing the "most important" token changes the score by 0.0002. Not a single strong baseline input flips its label when the top token is removed. LIME is directionally correct but the deletion test is powerless. When a model outputs 0.9999, removing one word drops it to 0.9995. The direction is right, but the magnitude is meaningless.
+
+The per-category faithfulness numbers make this concrete:
+
+| Category | Direction Correct | Flip Rate | Mean |Delta| |
+|----------|------------------|-----------|------------|
+| Lexical shortcuts | 100% | 100% | 0.995 |
+| Ambiguity | 100% | 40% | 0.408 |
+| Distribution shift | 100% | 20% | 0.230 |
+| Negation | 80% | 60% | 0.590 |
+| Strong baselines | 80% | 0% | 0.0002 |
+| Edge cases | 75% | 25% | 0.162 |
+
+LIME is faithful in direction but blind to magnitude. For high-confidence predictions, the standard one-token deletion test cannot distinguish a genuinely important token from a random one.
+
+## The Tokenizer Problem Nobody Talks About
+
+LIME splits text on whitespace. The model uses WordPiece tokenization. These are not the same thing.
+
+When LIME sees "I'm", it treats that as one token. When DistilBERT sees "I'm", it splits it into three subword pieces: "i", "'", "m". LIME's perturbation masks whole whitespace tokens, but the model processes subwords. This means LIME's attribution is to whitespace tokens, not to the features the model actually uses. The mismatch is fundamental and cannot be fixed without replacing LIME's perturbation strategy.
+
+Across the 29 tested inputs, the model's WordPiece tokenizer produces more tokens than LIME's whitespace splitter in the majority of cases. The worst mismatch occurs on longer sentences where compound words, contractions, and punctuation create divergent tokenizations. LIME attributes importance to a unit of text that does not correspond to how the model reads it.
+
+## When to Trust LIME
+
+LIME is not useless. It works well under specific conditions:
+
+**Trust LIME when one token dominates.** Lexical shortcuts score 100% direction-correct and 100% label flip. When a sentence has a single decisive word that, if removed, reverses the prediction, LIME finds it reliably.
+
+**Trust LIME on short inputs.** Shorter sentences have fewer near-zero-weight function words competing for top-5 slots. "This is good" (3 tokens) has perfect Jaccard across all seeds. "The quality exceeded all my expectations and I am thrilled" (11 tokens) drops to 0.68.
+
+**Do not trust LIME when model confidence exceeds 0.99.** The deletion test becomes powerless. LIME will tell you a token is important, but removing it changes nothing measurable.
+
+**Do not trust the ranking below rank 1.** The top-1 token is unanimous across seeds for 66% of inputs. Ranks 2 through 5 are where the instability lives. If you are reporting LIME results, report the top token and its weight. Do not treat the full ranking as meaningful.
+
+**Be skeptical of LIME on long sentences with clear sentiment.** Counterintuitively, these are where LIME is least stable, because function words create ranking noise.
+
+## Limitations
+
+This audit tests 30 inputs, giving 4 to 5 data points per category. Per-category statistics have wide confidence intervals, which is why I report bootstrap CIs rather than point estimates. Only one model (DistilBERT-SST2) is audited. The findings may not generalize to other architectures, though the tokenizer mismatch problem is architectural and affects any model that does not split on whitespace. The deletion test uses 300 perturbation samples, which is low. Higher sample counts would tighten attributions but would not fix the signal-to-noise problem on function-word-heavy sentences. Input 30 (a whitespace-only string) was intentionally included to test pipeline robustness. The pipeline correctly refused it, and it is logged as skipped rather than silently excluded.
+
+## Try It Yourself
+
+The live demo is at [xai-forensic.vercel.app](https://xai-forensic.vercel.app). Type a sentence and see the LIME attribution, counterfactual word removal, and dual-model divergence in real time.
+
+The full audit code, pre-registration protocol, raw CSVs, and this article's charts are at [github.com/parshvi1508/XAI_Forensic](https://github.com/parshvi1508/XAI_Forensic). Every number in this article can be reproduced with `make audit`.
+
+---
+
+*Parshvi Jain is a computer science student at ABES Engineering College and IIT Madras (BS Data Science), researching multimodal sentiment analysis and explainable AI. She deliberately chose LIME over attention weights as an explanation method (Jain and Wallace, 2019, showed attention is unreliable for explanation) and built XAI Forensics to make model explanations auditable.*