PsychiatryNLPKit 0.1.0__tar.gz

psychiatrynlpkit-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Rukun Dou, Tiana Wei, Alban Elias Voppel
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

psychiatrynlpkit-0.1.0/PKG-INFO

@@ -0,0 +1,570 @@
+Metadata-Version: 2.4
+Name: PsychiatryNLPKit
+Version: 0.1.0
+Summary: Computational linguistics toolkit for psychosis risk assessment and thought disorder analysis
+Author-email: Rukun Dou <rukun.dou2004@gmail.com>, Tiana Wei <hsi.wei@mail.mcgill.ca>, Alban Elias Voppel <alban.voppel@mail.mcgill.ca>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/rukun-dou/PsychiatryNLPKit
+Project-URL: Repository, https://github.com/rukun-dou/PsychiatryNLPKit
+Project-URL: Documentation, https://rukun-dou.github.io/PsychiatryNLPKit/
+Project-URL: Bug Tracker, https://github.com/rukun-dou/PsychiatryNLPKit/issues
+Keywords: nlp,psychiatry,clinical-nlp,thought-disorder,psychosis-risk
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
+Classifier: Topic :: Text Processing :: Linguistic
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: benepar>=0.2.0
+Requires-Dist: numpy>=2.0.0
+Requires-Dist: nltk>=3.9.0
+Requires-Dist: pandas>=2.2.0
+Requires-Dist: stanza>=1.8.0
+Requires-Dist: torch>=2.5.0
+Requires-Dist: transformers>=4.40.0
+Requires-Dist: sentence-transformers>=5.0.0
+Requires-Dist: scikit-learn>=1.3.0
+Requires-Dist: scikit-dimension>=0.3.2
+Requires-Dist: matplotlib>=3.8.0
+Requires-Dist: zstandard>=0.22.0
+Provides-Extra: graph
+Requires-Dist: networkx>=3.0; extra == "graph"
+Provides-Extra: image
+Requires-Dist: pillow>=10.0.0; extra == "image"
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: pyright>=1.1; extra == "dev"
+Requires-Dist: ruff>=0.4; extra == "dev"
+Requires-Dist: sphinx>=7.0; extra == "dev"
+Requires-Dist: sphinx-rtd-theme>=2.0; extra == "dev"
+Dynamic: license-file
+
+# PsychiatryNLPKit
+
+[![CI](https://github.com/rukun-dou/PsychiatryNLPKit/workflows/CI/badge.svg)](https://github.com/rukun-dou/PsychiatryNLPKit/actions)
+[![PyPI version](https://img.shields.io/pypi/v/PsychiatryNLPKit.svg)](https://pypi.org/project/PsychiatryNLPKit/)
+[![Python 3.12+](https://img.shields.io/pypi/pyversions/PsychiatryNLPKit.svg)](https://python.org/downloads/release/python-3120/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/rukun-dou/PsychiatryNLPKit/blob/main/LICENSE)
+[![Documentation](https://img.shields.io/badge/docs-latest-blue.svg)](https://rukun-dou.github.io/PsychiatryNLPKit/)
+
+A scientific Python package for computational linguistics analysis of clinical data in psychiatry. It provides a curated set of linguistic metrics across seven analytical categories, each grounded in peer-reviewed research on psychosis risk assessment and thought disorder characterization.
+
+---
+
+## Overview
+
+PsychiatryNLPKit is designed for researchers and clinicians who need to extract validated computational linguistics features from spoken or written clinical text. The package implements analysis functions derived from the scientific literature on language markers of psychosis, including formal thought disorder, disorganization, and cognitive impairment.
+
+The toolkit supports **English** and **French** text and provides metrics spanning:
+
+| Category | What it measures |
+|---|---|
+| **Syntax** | POS ratios, clause structure, syntax tree depth, sentence complexity |
+| **Similarity** |Semantic coherence via adjacent word/sentence cosine similarity |
+| **Perplexity** | Language model perplexity at paragraph and sentence levels (generative + masked LM) |
+| **Graph** | Network metrics from structural word-transition graphs (nodes, edges, diameter, z-scores) |
+| **Density** | Semantic space dimensionality via PCA explained variance and intrinsic dimension estimation |
+| **Lexicon** | Disfluency and filler word frequency |
+| **ImageSimilarity** | Cross-modal cosine similarity between images and text sections (ViT-based) |
+
+Each function accepts pre-computed linguistic data from the `TextData` container and returns a dictionary mapping section names to numeric metric values. This modular design lets you compose analyses flexibly or run them all at once via the batch API.
+
+### Design principles
+
+- **Scientific grounding** -- analysis functions are based on the psychiatry research literature. This package implements speech metrics that have been shown to correlate with clinical rating scales (PANSS, TLC, TLI).
+- **Batch efficiency** -- expensive computations (tokenization, embedding, constituency parsing) are lazy-loaded and cached on the `TextData` container. Running multiple analyses over the same text incurs no redundant work.
+- **Hardware acceleration** -- all deep learning pipelines run on CUDA, MPS (Apple Silicon), or Intel XPU when available, with automatic fallback to CPU.
+- **Composable architecture** -- individual analysis functions can be called standalone, or orchestrated together via `BatchAnalyzer`. The same `TextData` object serves all analyses.
+
+---
+
+## Quick Start
+
+### Batch API (recommended for most use cases)
+
+The high-level batch API processes an entire corpus in a few lines:
+
+```python
+import logging
+import PsychiatryNLPKit as pnlp
+from PsychiatryNLPKit.data import Section, TextData
+from PsychiatryNLPKit.model import HFEmbeddingLLM, HFGenerativeLLM
+
+# 1. Configure logging (optional but recommended)
+pnlp.configure_logging(level=logging.INFO)
+
+# 2. Load your text data from a file or database. Organize the dataset into different sections for separate analysis.
+sections = [
+    Section(text="I work in a factory that produces humanoid robots.", name="p1"),
+    Section(text="I just came back from a vacation in the mountains.", name="p2"),
+]
+data = TextData(sections=sections, lang="en")
+
+# 3. Attach models (lazy-loaded on first use)
+data.embedding_model = HFEmbeddingLLM("unsloth/embeddinggemma-300m")
+data.generative_model = HFGenerativeLLM("unsloth/Llama-3.2-1B")
+
+# 4. Run analyses (exclude analyses that require additional models)
+result = pnlp.BatchAnalyzer(
+    data,
+    excluded_analyses=[
+        "paragraph_level_pseudo_perplexity",  # requires mask-filling model
+        "sentence_level_pseudo_perplexity",
+        "structural_graph",  # requires networkx extra
+        "image_text_similarity",  # requires pillow extra and image paths
+    ],
+).run()
+
+# 5. Inspect results
+print(result.sections)          # ['p1', 'p2']
+print(result.analyses_run)      # list of successfully executed function names
+print(result.results["p1"])     # {'sentence_length': 12.5, 'adverb_ratio': 0.08, ...}
+
+# 6. Export to CSV for downstream statistical analysis
+import csv
+with open("results.csv", "w", newline="") as f:
+    writer = csv.writer(f)
+    writer.writerow(["section"] + list(result.results["p1"].keys()))
+    for section in result.sections:
+        writer.writerow([section] + [result.results[section].get(k, float("nan"))
+                                     for k in result.results["p1"].keys()])
+```
+
+### Low-level API (fine-grained control)
+
+For targeted analysis or custom pipelines, call individual functions directly:
+
+```python
+from PsychiatryNLPKit import analysis as pnlp_a
+from PsychiatryNLPKit.data import Section, TextData
+from PsychiatryNLPKit.model import HFEmbeddingLLM, HFGenerativeLLM
+
+sections = [Section(text="The quick brown fox jumps over the lazy dog.", name="p1")]
+data = TextData(sections=sections, lang="en")
+
+# Attach models (lazy-loaded on first use)
+data.embedding_model = HFEmbeddingLLM("unsloth/embeddinggemma-300m")
+data.generative_model = HFGenerativeLLM("unsloth/Llama-3.2-1B")
+
+# Syntax: sentence length
+lengths = pnlp_a.sentence_length(data.pos_tags)
+# → {"p1": {"sentence_length": 10.0}}
+
+# Semantic coherence: adjacent sentence similarity
+coherence = pnlp_a.sentence_level_cosine_similarity(data.sentence_embedding_vectors)
+# → {"p1": {"sentence_level_cosine_similarity": 0.73}}
+
+# Perplexity: generative model on paragraphs
+perplexity = pnlp_a.paragraph_level_perplexity(
+    data.paragraph_generative_tokens, data.generative_model
+)
+# → {"p1": {"paragraph_level_perplexity": 42.5}}
+
+# Graph: structural word-transition network metrics
+graph_metrics = pnlp_a.structural_graph(data.content_words)
+# → {"p1": {"nodes_count": 8.0, "edges_count": 9.0, ...}}
+
+# Density: PCA-based semantic space compression (uses token-level embeddings)
+density = pnlp_a.pca_density_metrics(data.token_embedding_vectors)
+# → {"p1": {"Ncomp_90": 3.0, "Pcomp_90": 0.375, "ExVar_2": 0.45}}
+
+# Lexicon: filler word disfluency count
+fillers = pnlp_a.filler_words_count(data.pos_tags, language="en")
+# → {"p1": {"filler_words_count": 0, "filler_words_ratio": 0.0}}
+```
+
+---
+
+## Installation
+
+### From PyPI
+
+```bash
+pip install PsychiatryNLPKit              # core package
+pip install PsychiatryNLPKit[graph]       # + networkx (for structural_graph)
+pip install PsychiatryNLPKit[image]       # + pillow (for image_text_similarity)
+pip install PsychiatryNLPKit[dev]         # + pytest, pyright, sphinx (for development)
+
+# All extras at once:
+pip install PsychiatryNLPKit[graph,image,dev]
+```
+
+### From source
+
+```bash
+git clone https://github.com/rukun-dou/PsychiatryNLPKit.git
+cd PsychiatryNLPKit
+pip install -e ".[graph,image,dev]"
+```
+
+**Requirements:** Python 3.12+, [Hugging Face token](https://huggingface.co/settings/tokens) (for gated models). Set the `HF_TOKEN` environment variable before loading any model that requires authentication.
+
+---
+
+## Configuration
+
+### Device detection
+
+PsychiatryNLPKit automatically detects and uses the fastest available hardware:
+
+```
+CUDA (NVIDIA) → MPS (Apple Silicon) → Intel XPU → CPU
+```
+
+```python
+from PsychiatryNLPKit import device, hf_token
+
+print(device)        # torch.device('cuda') or 'mps' or 'cpu'
+print(hf_token)      # str | None  (read from HF_TOKEN env var)
+```
+
+Models are loaded with `bfloat16` precision on GPUs for memory efficiency and faster inference. All deep learning computations (tokenization, embedding generation, perplexity scoring, image-text similarity) run on the detected device.
+
+### Logging
+
+```python
+import logging
+import PsychiatryNLPKit as pnlp
+
+pnlp.configure_logging(level=logging.DEBUG)  # default is INFO
+```
+
+The package uses a dedicated logger (`PsychiatryNLPKit`) with `propagate=False`, so messages won't interfere with your application's root logger. Calling `configure_logging` multiple times is safe (duplicate handlers are guarded against).
+
+---
+
+## Data Model
+
+### TextData
+
+The central container for all analysis. It holds raw text sections and computes linguistic properties lazily on first access, caching results to avoid redundant computation:
+
+```python
+from PsychiatryNLPKit.data import Section, TextData
+
+sections = [
+    Section(text="First paragraph.", name="p1"),
+    Section(text="Second paragraph.", name="p2"),
+]
+data = TextData(sections=sections, lang="en")
+```
+
+| Property | Requires model | Returns | Description |
+|---|---|---|---|
+| `data` | No | `dict[str, str]` | Raw section text keyed by name |
+| `sentences` | No | `dict[str, list[str]]` | Sentences per section (regex-split) |
+| `pos_tags` | No | `dict[str, list[list[tuple[str,str,str]]]]` | POS-tagged sentences: `(word, lemma, tag)` tuples |
+| `syntax_trees` | No | `dict[str, list[benepar.Tree]]` | Constituency parse trees per sentence |
+| `token_embedding_vectors` | `embedding_model` | `dict[str, torch.Tensor]` | Token-level embeddings, shape `(n_tokens, dim)` |
+| `attention_scores` | `embedding_model` | `dict[str, torch.Tensor]` | Aggregated attention scores, shape `(n_tokens, n_tokens)` |
+| `sentence_embedding_vectors` | `embedding_model` | `dict[str, torch.Tensor]` | Sentence-level embeddings, shape `(n_sentences, dim)` |
+| `content_words` | No | `dict[str, list[str]]` | Content words (nouns, verbs, adjectives, adverbs) per section |
+| `content_word_embedding_vectors` | No | `dict[str, torch.Tensor]` | Word2Vec embeddings for content words, shape `(n_words, dim)` |
+| `paragraph_generative_tokens` | `generative_model` | `dict[str, dict[str, torch.Tensor]]` | Tokenized paragraphs (input_ids + attention_mask) |
+| `sentence_generative_tokens` | `generative_model` | `dict[str, dict[str, torch.Tensor]]` | Tokenized sentences |
+
+### Section
+
+A named text segment:
+
+```python
+from PsychiatryNLPKit.data import Section
+
+section = Section(text="Clinical interview transcript...", name="session_01")
+```
+
+Each section has a `text` attribute (the raw string) and an optional `name` used as the dictionary key in all analysis outputs. If no name is provided, sections are auto-named `section_0`, `section_1`, etc.
+
+### ImageData
+
+For image-text similarity analysis:
+
+```python
+from PsychiatryNLPKit.data import ImageData
+
+image = ImageData(path="patient_response.jpg", name="p1")
+# Access via image.image → PIL.Image.Image (lazy-loaded)
+```
+
+---
+
+## Batch Analysis
+
+`BatchAnalyzer` is the high-level orchestrator. It runs selected analyses on a `TextData` object and collects all results into a single structured container:
+
+```python
+from PsychiatryNLPKit.analysis import BatchAnalyzer, AnalysisResult
+
+# Run every analysis (requires all models)
+result = BatchAnalyzer(
+    data,
+    mask_filling_model=mask_lm,      # for pseudo-perplexity
+    vit_model=vit,                    # for image-text similarity
+    image_paths={"p1": "img1.jpg", "p2": "img2.jpg"},  # per-section images
+).run()
+
+# Run a subset
+result = BatchAnalyzer(
+    data,
+    included_analyses=["sentence_length", "adverb_ratio", "filler_words_count"],
+).run()
+
+# Exclude specific analyses from the full set
+result = BatchAnalyzer(
+    data,
+    included_analyses="all",
+    excluded_analyses=[
+        "paragraph_level_pseudo_perplexity",  # requires mask-filling model
+        "sentence_level_pseudo_perplexity",
+        "structural_graph",  # requires networkx extra
+        "image_text_similarity",  # requires pillow extra and image paths
+    ],
+).run()
+```
+
+### Parameters
+
+| Parameter | Type | Description |
+|---|---|---|
+| `text_data` | `TextData` | The data container with sections and optional model references |
+| `included_analyses` | `"all"` or `list[str]` | Default `"all"` runs every registered analysis. Pass an explicit list for a subset |
+| `excluded_analyses` | `list[str] \| None` | Function names to remove from the inclusion set. Every excluded name must be in the resolved inclusion list; otherwise an `AssertionError` is raised |
+| `mask_filling_model` | `HFMaskFillingModel \| None` | Required if any pseudo-perplexity function remains after filtering |
+| `vit_model` | `HuggingFaceViTModel \| None` | Required if `"image_text_similarity"` is in the analysis set |
+| `image_paths` | `dict[str, str] \| None` | Maps section name → image file path. Must cover every section in `text_data.section_names` when image analysis is requested |
+
+### AnalysisResult
+
+| Attribute | Type | Description |
+|---|---|---|
+| `results` | `dict[str, dict[str, float]]` | Merged per-section metrics keyed by section name, then metric name |
+| `sections` | `list[str]` | Section names in order (from `TextData.section_names`) |
+| `analyses_run` | `list[str]` | Names of analyses that executed successfully |
+| `errors` | `dict[str, str]` | Per-function failures: `{function_name: error_message}` |
+
+---
+
+## Analysis Functions Reference
+
+All functions accept a `sections` parameter (`list[str] \| None`) to restrict processing to specific sections. When `None`, all available sections are processed. Empty or missing sections receive `float("nan")`. Language-dependent functions support `"en"` and `"fr"`.
+
+### Syntax (`pnlp.analysis.Syntax`) — 13 functions
+
+Measures of sentence structure, part-of-speech distributions, and syntax tree complexity derived from clinical research on thought disorder.
+
+| Function | Required input | Returns | Clinical basis |
+|---|---|---|---|
+| `sentence_length` | `pos_tags` | `{section: {"sentence_length": float}}` | Poverty of content (Bilgrami et al., 2022) |
+| `syntax_depth` | `syntax_trees` | `{section: {"syntax_depth": float}}` | Reduced complexity predicts psychosis onset (Morice & Ingram, 1983) |
+| `unique_pos_tags` | `pos_tags` | `{section: {"unique_pos_tags": int}}` | Linguistic diversity marker |
+| `adverb_ratio` | `pos_tags`, `lang` | `{section: {"adverb_ratio": float, "adverb_count": int}}` | Associated with negative symptoms (Haas et al., 2020) |
+| `coordinating_conjunction_ratio` | `pos_tags`, `lang` | `{section: {"coordinating_conjunction_ratio": float}}` | Negative symptom correlation (Haas et al., 2020) |
+| `adjective_ratio` | `pos_tags`, `lang` | `{section: {"adjective_ratio": float}}` | — |
+| `pronoun_ratio` | `pos_tags`, `lang` | `{section: {"pronoun_ratio": float}}` | Predicts semantic similarity patterns (He et al., 2024) |
+| `determiner_ratio` | `pos_tags`, `lang` | `{section: {"determiner_ratio": float}}` | Negative thought disorder marker (Bilgrami et al., 2022) |
+| `modal_auxiliary_verb_ratio` | `pos_tags`, `lang` | `{section: {"modal_auxiliary_verb_ratio": float}}` | — |
+| `stop_words_ratio` | `pos_tags`, `lang` | `{section: {"stop_words_ratio": float}}` | Baseline lexical measure |
+| `clause_count` | `syntax_trees` | `{section: {"clause_count": int}}` | Psychosis onset prediction (Bilgrami et al., 2022) |
+| `noun_group_count` | `syntax_trees` | `{section: {"noun_group_count": int}}` | — |
+| `adjective_sentence_length` | `syntax_trees` | `{section: {"adjective_sentence_length": float}}` | — |
+
+POS tag aliases support both Penn Treebank (`JJ`, `NN`, `RB`) and Universal Dependencies (`ADJ`, `NOUN`, `ADV`) schemes.
+
+### Similarity (`pnlp.analysis.Similarity`) — 2 functions
+
+Cosine similarity between adjacent embeddings to quantify semantic coherence:
+
+| Function | Required input | Returns | Clinical basis |
+|---|---|---|---|
+| `word_level_cosine_similarity` | `content_word_embedding_vectors` | `{section: {"word_level_cosine_similarity": float}}` | Correlated with tangentiality, circumstantiality, derailment (Bilgrami et al., 2022; Elvevag et al., 2007; He et al., 2024) |
+| `sentence_level_cosine_similarity` | `sentence_embedding_vectors` | `{section: {"sentence_level_cosine_similarity": float}}` | Detects incoherent speech in formal thought disorder (same references) |
+
+### Perplexity (`pnlp.analysis.Perplexity`) — 4 functions
+
+Language model perplexity at paragraph and sentence levels using both generative (causal LM) and masked language models:
+
+| Function | Model type | Required input | Returns | Clinical basis |
+|---|---|---|---|---|
+| `paragraph_level_perplexity` | Generative (causal LM) | `paragraph_generative_tokens`, `generative_model` | `{section: {"paragraph_level_perplexity": float}}` | High perplexity predicts delusion and unusual thought content (Alqahtani et al., 2022; He et al., 2024) |
+| `sentence_level_perplexity` | Generative (causal LM) | `sentence_generative_tokens`, `generative_model` | `{section: {"sentence_level_perplexity": float}}` | Same references |
+| `paragraph_level_pseudo_perplexity` | Masked LM | `data.data` (raw text), `mask_filling_model` | `{section: {"paragraph_level_pseudo_perplexity": float}}` | Same references |
+| `sentence_level_pseudo_perplexity` | Masked LM | `data.data` (raw text), `mask_filling_model` | `{section: {"sentence_level_pseudo_perplexity": float}}` | Same references |
+
+### Graph (`pnlp.analysis.Graph`) — 1 function
+
+Constructs a directed, weighted word-transition graph from lemmatized content words and computes network metrics:
+
+| Function | Required input | Returns | Clinical basis |
+|---|---|---|---|
+| `structural_graph` | `content_words` | `{section: {nodes_count, edges_count, average_degree, density, diameter, average_shortest_path_length, largest_connected_component, largest_strongly_connected_component, lcc/n, lsc/n, edge_weight_repetition_index, lcc_z_score, lsc_z_score, aspl_z_score, degree_distribution_z_score}}` | Schizophrenia patients show smaller connected components and lower ASPL (Nikzad et al., 2022) |
+
+Parameters: `directed=True`, `weighted=True`, `n_random_graphs=1000` (for z-score computation against a null distribution).
+
+### Density (`pnlp.analysis.Density`) — 2 functions
+
+Measures of semantic space dimensionality derived from token embeddings:
+
+| Function | Required input | Returns | Clinical basis |
+|---|---|---|---|
+| `pca_density_metrics` | `token_embedding_vectors` | `{section: {Ncomp_90, Pcomp_90, ExVar_2}}` | Schizophrenia patients show altered semantic compressibility (Palominos et al., 2025) |
+| `intrinsic_dimensionality_density` | `token_embedding_vectors` | `{section: {ID_MLE}}` | Lower intrinsic dimensionality indicates more redundant speech (same reference) |
+
+PCA is applied in **token space** (features = tokens, i.e. `X.T`) so metrics reflect how many token directions are needed to explain semantic variance. Total paragraph length must be controlled during statistical analysis.
+
+### Lexicon (`pnlp.analysis.Lexicon`) — 1 function
+
+| Function | Required input | Returns | Clinical basis |
+|---|---|---|---|
+| `filler_words_count` | `pos_tags`, `lang` | `{section: {filler_words_count, filler_words_ratio}}` | Disfluencies correlate with symptom severity and PANSS negative scores (Vail et al., 2018; Liebenthal et al., 2022) |
+
+Filler word lists are loaded from the packaged `resources/filler_words.json` resource. The `averaging_method` parameter controls computation: `"macro"` averages per-sentence filler ratios, `"micro"` computes total fillers / total words across the section (default: `"macro"`).
+
+### ImageSimilarity (`pnlp.analysis.ImageSimilarity`) — 1 function
+
+| Function | Required input | Returns | Clinical basis |
+|---|---|---|---|
+| `image_text_similarity` | `image`, `text`, `vit_model` | `{section: {"image_text_similarity": float}}` | Lower CLIP scores predict higher conceptual disorganization (He et al., 2024) |
+
+The image argument accepts a PIL Image object or a file path. Text is split into chunks that fit the model's context window, encoded at the token level alongside patch-level visual embeddings, and pairwise cosine similarity is aggregated with mean pooling.
+
+---
+
+## Model Wrappers
+
+All models use lazy loading: the underlying Hugging Face model is downloaded and loaded only when first accessed. Models are unloaded after each analysis pass to free GPU memory.
+
+| Class | Purpose | Example model ID |
+|---|---|---|
+| `HFEmbeddingLLM` | Sentence/word embedding models | `unsloth/embeddinggemma-300m` |
+| `HFGenerativeLLM` | Autoregressive causal LMs | `unsloth/Llama-3.2-1B` |
+| `HFMaskFillingModel` | Masked language models | `google-bert/bert-base-multilingual-uncased` |
+| `HuggingFaceViTModel` | Vision-language (SigLIP/CLIP-style) | `openai/clip-vit-base-patch16` |
+
+```python
+from PsychiatryNLPKit.model import HFEmbeddingLLM, HFGenerativeLLM, HFMaskFillingModel, HuggingFaceViTModel
+
+embedding = HFEmbeddingLLM("unsloth/embeddinggemma-300m")
+embedding.load()
+# embedding.model.encode(...)  # use the underlying model directly
+embedding.unload()  # frees GPU memory
+```
+
+---
+
+## Data Formats
+
+### Input: Text sections
+
+The primary input is a list of `Section` objects, each containing raw text and an optional name. Sections can represent individual interview responses, paragraphs from clinical notes, or any other text unit.
+
+For batch processing from files, a typical workflow looks like:
+
+```python
+import csv
+from PsychiatryNLPKit.data import Section, TextData
+
+# Read from CSV: columns "id", "text"
+sections = []
+with open("clinical_corpus.csv", newline="") as f:
+    reader = csv.DictReader(f)
+    for row in reader:
+        sections.append(Section(text=row["text"], name=row["id"]))
+
+data = TextData(sections=sections, lang="en")
+```
+
+### Output: AnalysisResult
+
+The `AnalysisResult` container provides three access patterns:
+
+```python
+# By section → metric
+result.results["p1"]["sentence_length"]  # 12.5
+
+# Export to pandas DataFrame for statistical analysis
+import pandas as pd
+records = []
+for section in result.sections:
+    record = {"section": section}
+    record.update({k: v for k, v in result.results[section].items()})
+    records.append(record)
+df = pd.DataFrame(records)
+
+# Check for failures
+if result.errors:
+    print("Failed analyses:", result.errors)
+```
+
+---
+
+## Project Structure
+
+```
+PsychiatryNLPKit/
+├── config.py                  # Device detection, logging setup, HF token
+├── data/
+│   ├── Text.py                # Section, TextData (lazy-computed properties)
+│   ├── Image.py               # ImageData container
+│   └── _resources.py          # Packaged resource paths
+├── model/
+│   ├── LLM.py                 # HFEmbeddingLLM, HFGenerativeLLM, HFMaskFillingModel
+│   └── ViT.py                 # HuggingFaceViTModel (SigLIP/CLIP)
+├── analysis/
+│   ├── Syntax.py              # 13 syntax functions
+│   ├── Similarity.py          # 2 similarity functions
+│   ├── Perplexity.py          # 4 perplexity functions
+│   ├── Graph.py               # structural_graph (network metrics + z-scores)
+│   ├── Density.py             # PCA density + intrinsic dimensionality
+│   ├── Lexicon.py             # Filler word disfluency analysis
+│   ├── ImageSimilarity.py     # ViT-based image-text similarity
+│   └── batch.py               # BatchAnalyzer, AnalysisResult
+├── resources/
+│   ├── filler_words.json      # EN/FR filler word lists
+│   └── word2vec.model         # Gensim KeyedVectors (pre-trained)
+└── tests/                     # Unit and integration tests
+```
+
+---
+
+## BibTeX Citation
+
+If you use PsychiatryNLPKit in your research, please cite:
+
+```bibtex
+@software{psychiatrynlpkit2026,
+  author = {Dou, Rukun and Wei, Tiana and Voppel, Alban Elias},
+  title = {PsychiatryNLPKit: Computational linguistics toolkit for psychosis risk assessment and thought disorder analysis},
+  year = {2026},
+  url = {https://github.com/rukun-dou/PsychiatryNLPKit},
+  version = {0.1.0},
+  license = {MIT}
+}
+```
+
+---
+
+## References
+
+All analysis functions include their theoretical basis and primary references in their docstrings. Key publications underpinning this toolkit:
+
+- Alqahtani, A., Kayi, E. S., Hamidian, S., Compton, M., & Diab, M. (2022). A quantitative and qualitative analysis of schizophrenia language. *Proceedings of the 13th International Workshop on Health Text Mining and Information Analysis (LOUHI)*, 173–183.
+- Bilgrami, Z. R., et al. (2022). Construct validity for computational linguistic metrics in individuals at clinical risk for psychosis. *Schizophrenia Research*, 245, 90–96.
+- Elvevag, B., Palmer, B. W., Gold, J. M., & Jeste, D. V. (2007). Semantic coherence of speech in schizophrenia and bipolar disorder. *Schizophrenia Research*, 93(1-3), 304–316.
+- Haas, S. S., et al. (2020). POS ratios and negative symptom correlation in psychosis.
+- He, R., Palominos, C., Zhang, H., Alonso-Sánchez, M. F., Palaniyappan, L., & Hinzen, W. (2024). Navigating the semantic space: Unraveling the structure of meaning in psychosis using different computational language models. *Psychiatry Research*, 333, 115752.
+- Liebenthal, E., et al. (2022). Linguistic and non-linguistic markers of disorganization in psychotic illness. *Schizophrenia Research*, 259, 111–120.
+- Morice, R., & Ingram, J. C. I. (1983). Verbal fluency as a screening test for schizophrenia. *Neuropsychobiology*, 10(3), 158–161.
+- Nikzad, A. H., et al. (2022). Who does what to whom? graph representations of action-predication in speech relate to psychopathological dimensions of psychosis. *Schizophrenia*, 8(1), 58.
+- Palominos, C., et al. (2025). Lexical meaning is lower dimensional in psychosis. *Scientific Reports*, 16(1), 859.
+- Vail, A. K., Liebson, E., Baker, J. T., & Morency, L.-P. (2018). Toward objective, multifaceted characterization of psychotic disorders: Lexical, structural, and disfluency markers of spoken language. *Proceedings of the 20th ACM International Conference on Multimodal Interaction*, 170–178.
+
+---
+
+## License
+
+[MIT License](LICENSE) -- Copyright (c) 2026 Rukun Dou, Tiana Wei, Alban Elias Voppel