pytextad 0.1.0__tar.gz

pytextad-0.1.0/LICENSE

@@ -0,0 +1,24 @@
+BSD 2-Clause License
+
+Copyright (c) 2026, Yang Cao
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

pytextad-0.1.0/PKG-INFO

@@ -0,0 +1,108 @@
+Metadata-Version: 2.4
+Name: pytextad
+Version: 0.1.0
+Summary: A unified library for text anomaly detection (document- and token-level), in the style of PyOD
+Author: Yang Cao
+License-Expression: BSD-2-Clause
+Project-URL: Homepage, https://github.com/charles-cao/pytextad
+Project-URL: Documentation, https://pytextad.readthedocs.io
+Keywords: anomaly detection,outlier detection,text,NLP,token-level
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: THIRD_PARTY_NOTICES.md
+Requires-Dist: numpy>=1.21
+Requires-Dist: scikit-learn>=1.0
+Requires-Dist: torch>=1.13
+Requires-Dist: transformers>=4.30
+Provides-Extra: test
+Requires-Dist: pytest>=7; extra == "test"
+Provides-Extra: docs
+Requires-Dist: sphinx>=7; extra == "docs"
+Requires-Dist: furo; extra == "docs"
+Requires-Dist: myst-parser; extra == "docs"
+Dynamic: license-file
+
+# PyTextAD: Text Anomaly Detection in Python
+
+[![PyPI](https://img.shields.io/pypi/v/pytextad.svg)](https://pypi.org/project/pytextad/)
+[![Documentation](https://readthedocs.org/projects/pytextad/badge/?version=latest)](https://pytextad.readthedocs.io)
+[![Tests](https://github.com/charles-cao/pytextad/actions/workflows/tests.yml/badge.svg)](https://github.com/charles-cao/pytextad/actions/workflows/tests.yml)
+[![License](https://img.shields.io/badge/license-BSD--2--Clause-blue.svg)](LICENSE)
+
+**PyTextAD** is a Python library for detecting anomalies in text, at the document
+and at the token level. Every detector follows the PyOD interface
+(`fit`, `decision_function`, `predict`, `decision_scores_`, `labels_`), and every
+re-implemented method is checked numerically against its original code.
+
+## Installation
+
+```bash
+pip install pytextad
+```
+
+From source:
+
+```bash
+git clone https://github.com/charles-cao/pytextad.git
+cd pytextad
+pip install -e ".[test]"
+```
+
+Requires Python >= 3.9, PyTorch >= 1.13 and transformers >= 4.30. Install the
+PyTorch build that matches your CUDA version first (https://pytorch.org).
+
+## Quick start
+
+```python
+from pytextad import CVDD, DATE, FATE, RSRAE, TokenEmbedder, mean_pool
+
+# frozen token embeddings from any Hugging Face encoder
+emb = TokenEmbedder("bert-base-uncased")
+H_train, _ = emb.transform(train_texts)
+H_test, _ = emb.transform(test_texts)
+
+scores = CVDD().fit(H_train).decision_function(H_test)                       # higher = more anomalous
+scores = RSRAE().fit(mean_pool(H_train)).decision_function(mean_pool(H_test))
+scores = DATE().fit(train_texts).decision_function(test_texts)               # raw text in, trains its own model
+scores = FATE().fit(texts, y).decision_function(test_texts)                  # few-shot: y = 1 for labelled anomalies
+
+word_scores = DATE().fit(train_texts).token_scores([t.split() for t in test_texts])
+```
+
+A runnable example on AG News: `python examples/quickstart.py`.
+
+## Implemented methods
+
+| Method | Year | Input | Token scores | Reference |
+|---|---|---|---|---|
+| CVDD | 2019 | frozen token embeddings | yes | Ruff et al., *Self-Attentive, Multi-Context One-Class Classification for Unsupervised Anomaly Detection on Text*, ACL 2019 |
+| RSRAE | 2020 | document vectors | no | Lai et al., *Robust Subspace Recovery Layer for Unsupervised Anomaly Detection*, ICLR 2020 |
+| DATE | 2021 | raw text | yes | Manolache et al., *DATE: Detecting Anomalies in Text via Self-Supervision of Transformers*, NAACL 2021 |
+| FATE | 2023 | raw text (+ few labelled anomalies) | no | Das et al., *Few-shot Anomaly Detection in Text with Deviation Learning*, ICONIP 2023 |
+
+Default hyperparameters are those of the official code. Each module's docstring
+lists where the official code and the paper disagree and which one we follow.
+
+## Faithfulness to the original implementations
+
+`tests/verification/` runs each original implementation next to ours with the same
+weights, inputs and random seeds and compares the results (DATE against the original
+transformers 3.0.2 code, RSRAE against the original TensorFlow code). All checks
+pass; see [tests/verification/README.md](tests/verification/README.md).
+
+## Running the tests
+
+```bash
+pytest                               # fast API tests, a few seconds
+PYTEXTAD_DEVICE=cuda pytest            # same, on GPU (PowerShell: $env:PYTEXTAD_DEVICE="cuda"; pytest)
+```
+
+## License
+
+BSD 2-Clause. Third-party notices for the original implementations are in
+[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).

pytextad-0.1.0/README.md

@@ -0,0 +1,79 @@
+# PyTextAD: Text Anomaly Detection in Python
+
+[![PyPI](https://img.shields.io/pypi/v/pytextad.svg)](https://pypi.org/project/pytextad/)
+[![Documentation](https://readthedocs.org/projects/pytextad/badge/?version=latest)](https://pytextad.readthedocs.io)
+[![Tests](https://github.com/charles-cao/pytextad/actions/workflows/tests.yml/badge.svg)](https://github.com/charles-cao/pytextad/actions/workflows/tests.yml)
+[![License](https://img.shields.io/badge/license-BSD--2--Clause-blue.svg)](LICENSE)
+
+**PyTextAD** is a Python library for detecting anomalies in text, at the document
+and at the token level. Every detector follows the PyOD interface
+(`fit`, `decision_function`, `predict`, `decision_scores_`, `labels_`), and every
+re-implemented method is checked numerically against its original code.
+
+## Installation
+
+```bash
+pip install pytextad
+```
+
+From source:
+
+```bash
+git clone https://github.com/charles-cao/pytextad.git
+cd pytextad
+pip install -e ".[test]"
+```
+
+Requires Python >= 3.9, PyTorch >= 1.13 and transformers >= 4.30. Install the
+PyTorch build that matches your CUDA version first (https://pytorch.org).
+
+## Quick start
+
+```python
+from pytextad import CVDD, DATE, FATE, RSRAE, TokenEmbedder, mean_pool
+
+# frozen token embeddings from any Hugging Face encoder
+emb = TokenEmbedder("bert-base-uncased")
+H_train, _ = emb.transform(train_texts)
+H_test, _ = emb.transform(test_texts)
+
+scores = CVDD().fit(H_train).decision_function(H_test)                       # higher = more anomalous
+scores = RSRAE().fit(mean_pool(H_train)).decision_function(mean_pool(H_test))
+scores = DATE().fit(train_texts).decision_function(test_texts)               # raw text in, trains its own model
+scores = FATE().fit(texts, y).decision_function(test_texts)                  # few-shot: y = 1 for labelled anomalies
+
+word_scores = DATE().fit(train_texts).token_scores([t.split() for t in test_texts])
+```
+
+A runnable example on AG News: `python examples/quickstart.py`.
+
+## Implemented methods
+
+| Method | Year | Input | Token scores | Reference |
+|---|---|---|---|---|
+| CVDD | 2019 | frozen token embeddings | yes | Ruff et al., *Self-Attentive, Multi-Context One-Class Classification for Unsupervised Anomaly Detection on Text*, ACL 2019 |
+| RSRAE | 2020 | document vectors | no | Lai et al., *Robust Subspace Recovery Layer for Unsupervised Anomaly Detection*, ICLR 2020 |
+| DATE | 2021 | raw text | yes | Manolache et al., *DATE: Detecting Anomalies in Text via Self-Supervision of Transformers*, NAACL 2021 |
+| FATE | 2023 | raw text (+ few labelled anomalies) | no | Das et al., *Few-shot Anomaly Detection in Text with Deviation Learning*, ICONIP 2023 |
+
+Default hyperparameters are those of the official code. Each module's docstring
+lists where the official code and the paper disagree and which one we follow.
+
+## Faithfulness to the original implementations
+
+`tests/verification/` runs each original implementation next to ours with the same
+weights, inputs and random seeds and compares the results (DATE against the original
+transformers 3.0.2 code, RSRAE against the original TensorFlow code). All checks
+pass; see [tests/verification/README.md](tests/verification/README.md).
+
+## Running the tests
+
+```bash
+pytest                               # fast API tests, a few seconds
+PYTEXTAD_DEVICE=cuda pytest            # same, on GPU (PowerShell: $env:PYTEXTAD_DEVICE="cuda"; pytest)
+```
+
+## License
+
+BSD 2-Clause. Third-party notices for the original implementations are in
+[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).

pytextad-0.1.0/THIRD_PARTY_NOTICES.md

@@ -0,0 +1,36 @@
+# Third-party notices
+
+## CVDD (pytextad/cvdd.py)
+Re-implemented from https://github.com/lukasruff/CVDD-PyTorch
+
+MIT License
+
+Copyright (c) 2019 lukasruff
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+## RSRAE (pytextad/rsrae.py)
+Ported to PyTorch from https://github.com/dmzou/RSRAE
+
+MIT License
+
+Copyright (c) 2019-present Chieh-Hsin Lai, Dongmian Zou and Gilad Lerman
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+## DATE (pytextad/date.py)
+Re-implemented from https://github.com/bit-ml/date, whose model code lives in a
+modified copy of simpletransformers distributed under the Apache License 2.0.
+No source code is copied; the logic was re-written against modern transformers.
+
+## FATE (pytextad/fate.py)
+Re-implemented from https://github.com/arav1ndajay/fate, which has no licence file.
+No source code is copied; the logic was re-written and checked numerically against it.

pytextad-0.1.0/pyproject.toml

@@ -0,0 +1,44 @@
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "pytextad"
+dynamic = ["version"]
+description = "A unified library for text anomaly detection (document- and token-level), in the style of PyOD"
+readme = "README.md"
+license = "BSD-2-Clause"
+license-files = ["LICENSE", "THIRD_PARTY_NOTICES.md"]
+authors = [{name = "Yang Cao"}]
+requires-python = ">=3.9"
+keywords = ["anomaly detection", "outlier detection", "text", "NLP", "token-level"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Science/Research",
+    "Programming Language :: Python :: 3",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+dependencies = [
+    "numpy>=1.21",
+    "scikit-learn>=1.0",
+    "torch>=1.13",
+    "transformers>=4.30",
+]
+
+[project.optional-dependencies]
+test = ["pytest>=7"]
+docs = ["sphinx>=7", "furo", "myst-parser"]
+
+[project.urls]
+Homepage = "https://github.com/charles-cao/pytextad"
+Documentation = "https://pytextad.readthedocs.io"
+
+[tool.setuptools.dynamic]
+version = {attr = "pytextad.version.__version__"}
+
+[tool.setuptools.packages.find]
+include = ["pytextad*"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "--ignore=tests/verification"

pytextad-0.1.0/pytextad/__init__.py

@@ -0,0 +1,9 @@
+"""PyTextAD: a unified library for text anomaly detection, in the style of PyOD."""
+from .version import __version__
+from .models.cvdd import CVDD
+from .models.date import DATE
+from .models.fate import FATE
+from .models.rsrae import RSRAE
+from .utils.embeddings import TokenEmbedder, mean_pool, words_from_subwords
+
+__all__ = ["__version__", "CVDD", "DATE", "FATE", "RSRAE", "TokenEmbedder", "mean_pool", "words_from_subwords"]

pytextad-0.1.0/pytextad/models/__init__.py

@@ -0,0 +1,7 @@
+from .base import BaseTextDetector
+from .cvdd import CVDD
+from .date import DATE
+from .fate import FATE
+from .rsrae import RSRAE
+
+__all__ = ["BaseTextDetector", "CVDD", "DATE", "FATE", "RSRAE"]

pytextad-0.1.0/pytextad/models/base.py

@@ -0,0 +1,71 @@
+"""Common interface for all detectors, modelled on PyOD's BaseDetector.
+
+Conventions (identical to PyOD):
+  * fit(X, y=None) returns self and sets ``decision_scores_`` (scores of the training data),
+    ``threshold_`` and ``labels_``.
+  * decision_function(X) returns one score per sample; higher = more anomalous.
+  * predict(X) returns 0/1 using ``threshold_`` (the (1 - contamination) quantile of
+    the training scores).
+Detectors that can score tokens additionally implement ``token_scores``.
+"""
+
+import abc
+import random
+
+import numpy as np
+import torch
+
+
+class BaseTextDetector(abc.ABC):
+
+    def __init__(self, contamination=0.1, random_state=0, device=None, verbose=False):
+        if not 0.0 < contamination <= 0.5:
+            raise ValueError("contamination must be in (0, 0.5]")
+        self.contamination = contamination
+        self.random_state = random_state
+        self.device = device or ("cuda" if torch.cuda.is_available() else "cpu")
+        self.verbose = verbose
+
+    # ------------------------------------------------------------------ to implement
+    @abc.abstractmethod
+    def fit(self, X, y=None):
+        """Fit the detector on (mostly) normal data and set ``decision_scores_``,
+        ``threshold_`` and ``labels_``. ``y`` is ignored except by semi-supervised
+        detectors (FATE). Returns ``self``."""
+
+    @abc.abstractmethod
+    def decision_function(self, X):
+        """Anomaly score of every sample in ``X``; higher means more anomalous."""
+
+    # ------------------------------------------------------------------ shared
+    def _set_seed(self):
+        if self.random_state is None:
+            return
+        random.seed(self.random_state)
+        np.random.seed(self.random_state)
+        torch.manual_seed(self.random_state)
+        if torch.cuda.is_available():
+            torch.cuda.manual_seed_all(self.random_state)
+
+    def _process_decision_scores(self, scores):
+        self.decision_scores_ = np.asarray(scores, dtype=float)
+        self.threshold_ = float(np.percentile(self.decision_scores_, 100 * (1 - self.contamination)))
+        self.labels_ = (self.decision_scores_ > self.threshold_).astype(int)
+        return self
+
+    def predict(self, X):
+        """Binary labels (1 = anomaly) using ``threshold_`` from the training scores."""
+        self._check_fitted()
+        return (self.decision_function(X) > self.threshold_).astype(int)
+
+    def fit_predict(self, X, y=None):
+        """Fit on ``X`` and return the labels of the training data (``labels_``)."""
+        return self.fit(X, y).labels_
+
+    def _check_fitted(self):
+        if not hasattr(self, "decision_scores_"):
+            raise RuntimeError(f"{type(self).__name__} is not fitted yet; call fit() first.")
+
+    def _log(self, msg):
+        if self.verbose:
+            print(f"[{type(self).__name__}] {msg}")

pytextad-0.1.0/pytextad/models/cvdd.py

@@ -0,0 +1,221 @@
+"""CVDD: Context Vector Data Description (Ruff et al., ACL 2019).
+
+Re-implementation for modern PyTorch. The model and training logic follow the
+official code, https://github.com/lukasruff/CVDD-PyTorch (MIT License,
+Copyright (c) 2019 lukasruff); see THIRD_PARTY_NOTICES.
+
+Input: a list of frozen token-embedding matrices, one [n_tokens, dim] array per
+document (GloVe/fastText vectors, or hidden states of a frozen PLM, e.g. from
+``pytextad.embeddings.TokenEmbedder``). Only the self-attention layer and the context
+vectors are trained, exactly as in the official code.
+
+Faithful to the official code
+  * self-attention  A = softmax_over_tokens(W2 tanh(W1 H)), r heads, no biases
+  * M = A H; cosine distance d_k = 0.5 (1 - cos(M_k, c_k))
+  * context vectors initialised by k-means on L2-normalised mean token embeddings
+    of the training set, then L2-normalised
+  * loss = mean_n sum_k softmax_k(-alpha d) d_k + lambda_p * mean((C C^T - I)^2)
+  * temperature alpha annealed at 5 equidistant milestones
+    (soft / linear / logarithmic / hard, same values as the official code)
+  * Adam with weight decay, gradient-norm clipping at 0.5, MultiStepLR(gamma=0.1)
+    stepped at the start of each epoch as in the official trainer (so with
+    PyTorch >= 1.1 the LR drop happens one epoch before the nominal milestone)
+  * anomaly score = mean_k d_k ("context_dist_mean", the official default)
+  * defaults = the settings in the official README for Reuters / 20 Newsgroups
+    (3 heads, attention size 150, lambda_p 1, logarithmic, 100 epochs, lr 0.01,
+    lr milestone 40, batch 64, weight decay 5e-7)
+
+Deliberate deviations
+  * Padding is masked out of the attention softmax. The official code pads with index
+    0 and does not mask. With static word vectors (GloVe, fastText) the pad vector is
+    zero, which only rescales M, so the cosine distances, the loss and the training
+    are identical (verified numerically with variable-length documents). With PLM
+    hidden states the official "bert" option feeds non-zero pad vectors into M, which
+    makes a document's score depend on the length of the others in its batch; we do
+    not reproduce that.
+  * Mini-batches are drawn uniformly at random and every document is scored. The
+    official loaders group documents by length (BucketBatchSampler) with
+    drop_last=True for both training and testing, so up to batch_size - 1 test
+    documents are never scored in the official evaluation.
+  * The official "context_best" score is NOT provided: it selects, per run, the
+    head with the highest AUROC on the labelled test set, which uses test labels.
+    ``head_scores()`` returns per-head distances if you need them.
+
+Extension (not in the CVDD paper)
+  * token_scores(): min over heads of the cosine distance between each token vector
+    and the context vectors. Comparable across documents. Use with care: it is our
+    definition, not part of CVDD.
+"""
+
+import warnings
+
+import numpy as np
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from sklearn.cluster import KMeans
+
+from .base import BaseTextDetector
+
+
+def _alpha_schedule(name, n_epochs):
+    milestones = list(np.arange(1, 6) * int(n_epochs / 5))
+    if name == "soft":
+        alphas = [0.0] * 5
+    elif name == "linear":
+        alphas = list(np.linspace(0.2, 1, 5))
+    elif name == "logarithmic":
+        alphas = list(np.logspace(-4, 0, 5))
+    elif name == "hard":
+        alphas = [100.0] * 4            # official list has 4 entries
+    else:
+        raise ValueError(f"unknown alpha_scheduler {name}")
+    return milestones, alphas
+
+
+class _CVDDNet(nn.Module):
+    def __init__(self, dim, attention_size, n_heads):
+        super().__init__()
+        self.W1 = nn.Linear(dim, attention_size, bias=False)
+        self.W2 = nn.Linear(attention_size, n_heads, bias=False)
+        self.c = nn.Parameter((torch.rand(n_heads, dim) - 0.5) * 2)
+        self.alpha = 0.0
+
+    def attend(self, H, mask):
+        # H [B, L, d], mask [B, L] (True = real token)
+        logits = self.W2(torch.tanh(self.W1(H)))                    # [B, L, r]
+        logits = logits.masked_fill(~mask.unsqueeze(-1), float("-inf"))
+        A = F.softmax(logits, dim=1).transpose(1, 2)                # [B, r, L]
+        return A @ H, A                                             # M [B, r, d]
+
+    def forward(self, H, mask):
+        M, A = self.attend(H, mask)
+        d = 0.5 * (1 - F.cosine_similarity(M, self.c.unsqueeze(0), dim=2))   # [B, r]
+        w = F.softmax(-self.alpha * d, dim=1)
+        return d, w, A
+
+
+class CVDD(BaseTextDetector):
+
+    def __init__(self, n_heads=3, attention_size=150, lambda_p=1.0,
+                 alpha_scheduler="logarithmic", n_epochs=100, lr=0.01, lr_milestones=(40,),
+                 batch_size=64, weight_decay=0.5e-6, contamination=0.1, random_state=0,
+                 device=None, verbose=False):
+        super().__init__(contamination, random_state, device, verbose)
+        self.n_heads = n_heads
+        self.attention_size = attention_size
+        self.lambda_p = lambda_p
+        self.alpha_scheduler = alpha_scheduler
+        self.n_epochs = n_epochs
+        self.lr = lr
+        self.lr_milestones = tuple(lr_milestones)
+        self.batch_size = batch_size
+        self.weight_decay = weight_decay
+
+    # ------------------------------------------------------------------ batching
+    @staticmethod
+    def _check(X):
+        X = [np.asarray(x, dtype=np.float32) for x in X]
+        if any(x.ndim != 2 or len(x) == 0 for x in X):
+            raise ValueError("X must be a list of non-empty [n_tokens, dim] arrays")
+        if len({x.shape[1] for x in X}) != 1:
+            raise ValueError("all token embeddings must have the same dimension")
+        return X
+
+    def _batch(self, X, idx):
+        L = max(len(X[i]) for i in idx)
+        H = np.zeros((len(idx), L, X[idx[0]].shape[1]), dtype=np.float32)
+        mask = np.zeros((len(idx), L), dtype=bool)
+        for j, i in enumerate(idx):
+            H[j, :len(X[i])] = X[i]
+            mask[j, :len(X[i])] = True
+        return torch.from_numpy(H).to(self.device), torch.from_numpy(mask).to(self.device)
+
+    def _batches(self, n, shuffle):
+        order = np.random.permutation(n) if shuffle else np.arange(n)
+        for s in range(0, n, self.batch_size):
+            yield order[s:s + self.batch_size]
+
+    # ------------------------------------------------------------------ training
+    def fit(self, X, y=None):
+        self._set_seed()
+        X = self._check(X)
+        dim = X[0].shape[1]
+        self.net_ = _CVDDNet(dim, self.attention_size, self.n_heads).to(self.device)
+
+        # context vector initialisation (official initialize_context_vectors)
+        means = np.stack([x.mean(0) for x in X])
+        means = means / np.clip(np.linalg.norm(means, axis=1, keepdims=True), 1e-8, None)
+        km = KMeans(n_clusters=self.n_heads, n_init=10, random_state=self.random_state).fit(means)
+        centers = km.cluster_centers_ / np.linalg.norm(km.cluster_centers_, axis=1, keepdims=True)
+        self.net_.c.data = torch.from_numpy(centers.astype(np.float32)).to(self.device)
+
+        opt = torch.optim.Adam(self.net_.parameters(), lr=self.lr, weight_decay=self.weight_decay)
+        sched = torch.optim.lr_scheduler.MultiStepLR(opt, milestones=list(self.lr_milestones), gamma=0.1)
+        milestones, alphas = _alpha_schedule(self.alpha_scheduler, self.n_epochs)
+        alpha_i = 0
+        I = torch.eye(self.n_heads, device=self.device)
+
+        self.net_.alpha = 0.0
+        self.history_ = []
+        for epoch in range(self.n_epochs):
+            with warnings.catch_warnings():  # official order: step at the START of each epoch
+                warnings.simplefilter("ignore", UserWarning)
+                sched.step()
+            if epoch in milestones and alpha_i < len(alphas):     # official: one step per epoch
+                self.net_.alpha = float(alphas[alpha_i])
+                alpha_i += 1
+            self.net_.train()
+            tot, nb = 0.0, 0
+            for idx in self._batches(len(X), shuffle=True):
+                H, mask = self._batch(X, idx)
+                d, w, _ = self.net_(H, mask)
+                P = torch.mean((self.net_.c @ self.net_.c.t() - I) ** 2)
+                loss = torch.mean(torch.sum(w * d, dim=1)) + self.lambda_p * P
+                opt.zero_grad()
+                loss.backward()
+                torch.nn.utils.clip_grad_norm_(self.net_.parameters(), 0.5)   # as in the official trainer
+                opt.step()
+                tot += loss.item()
+                nb += 1
+            self.history_.append(tot / nb)
+            self._log(f"epoch {epoch + 1}/{self.n_epochs} loss {tot / nb:.6f} alpha {self.net_.alpha:g}")
+
+        return self._process_decision_scores(self.decision_function(X))
+
+    # ------------------------------------------------------------------ inference
+    @torch.no_grad()
+    def _forward_all(self, X):
+        X = self._check(X)
+        self.net_.eval()
+        D, A_all = [], []
+        for idx in self._batches(len(X), shuffle=False):
+            H, mask = self._batch(X, idx)
+            d, _, A = self.net_(H, mask)
+            D.append(d.cpu().numpy())
+            A = A.cpu().numpy()
+            A_all += [A[j, :, :len(X[i])] for j, i in enumerate(idx)]
+        return np.concatenate(D), A_all
+
+    def head_scores(self, X):
+        """Per-head cosine distances, shape [n_docs, n_heads]."""
+        return self._forward_all(X)[0]
+
+    def decision_function(self, X):
+        """Official 'context_dist_mean' score: mean cosine distance over heads."""
+        return self.head_scores(X).mean(1)
+
+    def attention(self, X):
+        """Attention weights per document, each [n_heads, n_tokens] (for inspection)."""
+        return self._forward_all(X)[1]
+
+    @torch.no_grad()
+    def token_scores(self, X):
+        """EXTENSION (not in the paper): per-token min_k 0.5 (1 - cos(h_t, c_k))."""
+        X = self._check(X)
+        C = F.normalize(self.net_.c.detach(), dim=1).cpu().numpy()
+        out = []
+        for x in X:
+            xn = x / np.clip(np.linalg.norm(x, axis=1, keepdims=True), 1e-8, None)
+            out.append((0.5 * (1 - xn @ C.T)).min(1))
+        return out