dustclust 0.1.0__tar.gz

dustclust-0.1.0/PKG-INFO

@@ -0,0 +1,152 @@
+Metadata-Version: 2.4
+Name: dustclust
+Version: 0.1.0
+Summary: DustClust: interactive hardware inventory clustering with embeddings and a web visualization
+Author: DustClust contributors
+License: MIT
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: fastapi
+Requires-Dist: uvicorn[standard]
+Requires-Dist: python-multipart
+Requires-Dist: pandas
+Requires-Dist: openpyxl
+Requires-Dist: numpy
+Requires-Dist: sentence-transformers
+Requires-Dist: scikit-learn
+
+# DustClust
+
+**DustClust** (`import DustClust`) is an interactive web application that clusters dirty hardware inventory data using semantic embeddings and visualizes the results in a D3.js radial network graph. The backend runs live clustering with configurable thresholds and matches clusters to real-world devices from the iFixit catalog plus a custom device list.
+
+Install from PyPI as **`dustclust`** (`pip install dustclust`).
+
+## Features
+
+### GUI
+
+- **Interactive Network Graph**: Zoom, pan, and drag nodes. Hover for quick tooltips; click to open the detail panel.
+- **Search Clusters**: Real-time search by category, subcategory, or record content (e.g., Router, iPhone, 4331).
+- **Clustering Threshold**: Adjust the similarity threshold (0.10–0.90) and click **Recalculate Clusters** to re-run clustering.
+- **Min Cluster Size**: Filter out small clusters with a slider.
+- **Upload Dataset**: Upload your own `.csv` or `.xlsx` file. The server generates embeddings and clusters on the fly.
+- **Reset View**: Reset zoom and clear search/filters.
+- **Detail Panel**: Click any node to see:
+  - **Category / Subcategory** (e.g., Computer Hardware / Mouse)
+  - Sample records
+  - **Matched iFixit Device** (when a cluster matches a known device, with link to iFixit)
+
+### Backend
+
+- **Embedding-based clustering**: Uses sentence-transformers with `paraphrase-multilingual-MiniLM-L12-v2` (or another HuggingFace model ID) for semantic similarity.
+- **Hebrew & multilingual support**: The model and tokenization support Hebrew and other Unicode scripts. Category inference includes Hebrew keywords (e.g. מסך, מקלדת, עכבר).
+- **Dynamic category inference**: Token-based matching against iFixit categories and `custom_devices.py`.
+- **Subcategory inference**: Finer-grained labels (e.g., Keyboard, Monitor, Printer) derived from the device list.
+- **Device matching**: IDF-scored matching of cluster records to iFixit devices and custom entries.
+
+## Device List
+
+The app uses two device sources:
+
+1. **iFixit catalog** (`ifixit_devices.json`): Run `fetch_ifixit_devices.py` to populate.
+2. **Custom devices** (module `DustClust.custom_devices`): Generic hardware that supplements iFixit for category/subcategory inference and device matching.
+
+### Custom Device Categories
+
+| Category | Subcategories / Types |
+|---------|------------------------|
+| **SIM Card** | Nano, Micro, Standard, eSIM |
+| **Cable** | USB-C, Lightning, HDMI, DisplayPort, Ethernet, VGA, DVI, Thunderbolt, SATA, etc. |
+| **Adapter** | USB hubs, HDMI/DisplayPort adapters |
+| **Storage** | MicroSD, SD, Flash Drive, HDD, SSD, NAS, CD/DVD/Blu-ray drives |
+| **PC Component** | RAM, CPU, GPU, Motherboard, PSU, SSD, Cooling, Thermal paste |
+| **Computer Hardware** | Keyboard, Mouse, Laptop, Monitor, Webcam, Dashcam, Projector, Printer |
+| **Audio** | Headphones, Earbuds, Microphone |
+| **Telecom** | Router, Modem, Network Switch, Access Point, enterprise gear |
+
+Edit `src/DustClust/custom_devices.py` (or the installed package’s `custom_devices.py`) to add or adjust devices. Each entry has `name`, `category`, `subcategory`, and optional `url`.
+
+### Hebrew / Multilingual Datasets
+
+The app works with Hebrew and other Unicode datasets. The default model (`sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2`) supports 50+ languages. Category inference includes Hebrew boost keywords (מסך, מקלדת, עכבר, כבל, etc.), and `custom_devices.py` has Hebrew device entries for matching. Upload a Hebrew CSV and the clustering and categorization will work out of the box.
+
+## Getting Started
+
+### Install with pip
+
+From the repository root (editable install for development):
+
+```bash
+pip install -e .
+```
+
+Or from a sdist/wheel once published:
+
+```bash
+pip install dustclust
+```
+
+This installs the **`dustclust`** CLI and the importable package **`DustClust`**.
+
+### Prerequisites
+
+- Python 3.10+
+- Dependencies are declared in `pyproject.toml` (installed automatically with `pip install`).
+
+### Optional: conda and iFixit cache
+
+```bash
+conda create -n dustclust python=3.11
+conda activate dustclust
+pip install -e .
+```
+
+Fetch / refresh the iFixit device catalog into the current directory (optional; a copy may already be bundled in the package):
+
+```bash
+python fetch_ifixit_devices.py
+```
+
+### Running the App
+
+1. Start the server (any of these):
+
+```bash
+dustclust
+# or
+python -m DustClust
+# or, from a clone without installing
+python server.py
+```
+
+2. Open [http://localhost:8001](http://localhost:8001) in your browser.
+
+**Command-line options** (run `dustclust --help` for details):
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--model` | `sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2` | Path to local model folder or HuggingFace ID |
+| `--device` | `cpu` | Device for embeddings: `cpu` or `cuda` |
+| `--threshold` | `0.3` | Default clustering threshold (0.1–0.9) |
+| `--batch-size` | `512` | Batch size for embedding generation |
+| `--data` | *(bundled sample in the package)* | Path to CSV dataset; omit to use the bundled sample |
+| `--host` | `localhost` | Host to bind |
+| `--port` | `8001` | Port to run on |
+| `--no-reload` | — | Disable auto-reload on file changes |
+
+The server loads the default dataset (`data/dirty_hardware_data_40k.csv`) and **serves the GUI immediately**; initial embeddings run in the background (the graph appears when they finish—see `/api/status`). Use **Recalculate Clusters** or **Upload Dataset** to change the data or clustering.
+
+## Model
+
+The default embedding model is `sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2`. Use `--model` to pass a local model folder or another HuggingFace model ID.
+
+## Data
+
+CSV files live in the `data/` folder. The default dataset is `data/dirty_hardware_data_40k.csv`. Use `--data` or `--input`/`--output` to point scripts at other paths.
+
+## Data Processing Workflow
+
+- `dataset_generator.py`: Generates raw `data/dirty_hardware_data_40k.csv`.
+- `cluster_hardware` / CLI **`cluster-hardware`**: Produces `data/clustered_output.csv` from embeddings and clustering.
+- `prepare_viz_data.py`: Builds static `src/DustClust/cluster_viz/data.json` for the legacy static workflow.
+- `DustClust.server` / CLI **`dustclust`**: Serves the live app with on-demand clustering and device matching.

dustclust-0.1.0/README.md

@@ -0,0 +1,135 @@
+# DustClust
+
+**DustClust** (`import DustClust`) is an interactive web application that clusters dirty hardware inventory data using semantic embeddings and visualizes the results in a D3.js radial network graph. The backend runs live clustering with configurable thresholds and matches clusters to real-world devices from the iFixit catalog plus a custom device list.
+
+Install from PyPI as **`dustclust`** (`pip install dustclust`).
+
+## Features
+
+### GUI
+
+- **Interactive Network Graph**: Zoom, pan, and drag nodes. Hover for quick tooltips; click to open the detail panel.
+- **Search Clusters**: Real-time search by category, subcategory, or record content (e.g., Router, iPhone, 4331).
+- **Clustering Threshold**: Adjust the similarity threshold (0.10–0.90) and click **Recalculate Clusters** to re-run clustering.
+- **Min Cluster Size**: Filter out small clusters with a slider.
+- **Upload Dataset**: Upload your own `.csv` or `.xlsx` file. The server generates embeddings and clusters on the fly.
+- **Reset View**: Reset zoom and clear search/filters.
+- **Detail Panel**: Click any node to see:
+  - **Category / Subcategory** (e.g., Computer Hardware / Mouse)
+  - Sample records
+  - **Matched iFixit Device** (when a cluster matches a known device, with link to iFixit)
+
+### Backend
+
+- **Embedding-based clustering**: Uses sentence-transformers with `paraphrase-multilingual-MiniLM-L12-v2` (or another HuggingFace model ID) for semantic similarity.
+- **Hebrew & multilingual support**: The model and tokenization support Hebrew and other Unicode scripts. Category inference includes Hebrew keywords (e.g. מסך, מקלדת, עכבר).
+- **Dynamic category inference**: Token-based matching against iFixit categories and `custom_devices.py`.
+- **Subcategory inference**: Finer-grained labels (e.g., Keyboard, Monitor, Printer) derived from the device list.
+- **Device matching**: IDF-scored matching of cluster records to iFixit devices and custom entries.
+
+## Device List
+
+The app uses two device sources:
+
+1. **iFixit catalog** (`ifixit_devices.json`): Run `fetch_ifixit_devices.py` to populate.
+2. **Custom devices** (module `DustClust.custom_devices`): Generic hardware that supplements iFixit for category/subcategory inference and device matching.
+
+### Custom Device Categories
+
+| Category | Subcategories / Types |
+|---------|------------------------|
+| **SIM Card** | Nano, Micro, Standard, eSIM |
+| **Cable** | USB-C, Lightning, HDMI, DisplayPort, Ethernet, VGA, DVI, Thunderbolt, SATA, etc. |
+| **Adapter** | USB hubs, HDMI/DisplayPort adapters |
+| **Storage** | MicroSD, SD, Flash Drive, HDD, SSD, NAS, CD/DVD/Blu-ray drives |
+| **PC Component** | RAM, CPU, GPU, Motherboard, PSU, SSD, Cooling, Thermal paste |
+| **Computer Hardware** | Keyboard, Mouse, Laptop, Monitor, Webcam, Dashcam, Projector, Printer |
+| **Audio** | Headphones, Earbuds, Microphone |
+| **Telecom** | Router, Modem, Network Switch, Access Point, enterprise gear |
+
+Edit `src/DustClust/custom_devices.py` (or the installed package’s `custom_devices.py`) to add or adjust devices. Each entry has `name`, `category`, `subcategory`, and optional `url`.
+
+### Hebrew / Multilingual Datasets
+
+The app works with Hebrew and other Unicode datasets. The default model (`sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2`) supports 50+ languages. Category inference includes Hebrew boost keywords (מסך, מקלדת, עכבר, כבל, etc.), and `custom_devices.py` has Hebrew device entries for matching. Upload a Hebrew CSV and the clustering and categorization will work out of the box.
+
+## Getting Started
+
+### Install with pip
+
+From the repository root (editable install for development):
+
+```bash
+pip install -e .
+```
+
+Or from a sdist/wheel once published:
+
+```bash
+pip install dustclust
+```
+
+This installs the **`dustclust`** CLI and the importable package **`DustClust`**.
+
+### Prerequisites
+
+- Python 3.10+
+- Dependencies are declared in `pyproject.toml` (installed automatically with `pip install`).
+
+### Optional: conda and iFixit cache
+
+```bash
+conda create -n dustclust python=3.11
+conda activate dustclust
+pip install -e .
+```
+
+Fetch / refresh the iFixit device catalog into the current directory (optional; a copy may already be bundled in the package):
+
+```bash
+python fetch_ifixit_devices.py
+```
+
+### Running the App
+
+1. Start the server (any of these):
+
+```bash
+dustclust
+# or
+python -m DustClust
+# or, from a clone without installing
+python server.py
+```
+
+2. Open [http://localhost:8001](http://localhost:8001) in your browser.
+
+**Command-line options** (run `dustclust --help` for details):
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--model` | `sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2` | Path to local model folder or HuggingFace ID |
+| `--device` | `cpu` | Device for embeddings: `cpu` or `cuda` |
+| `--threshold` | `0.3` | Default clustering threshold (0.1–0.9) |
+| `--batch-size` | `512` | Batch size for embedding generation |
+| `--data` | *(bundled sample in the package)* | Path to CSV dataset; omit to use the bundled sample |
+| `--host` | `localhost` | Host to bind |
+| `--port` | `8001` | Port to run on |
+| `--no-reload` | — | Disable auto-reload on file changes |
+
+The server loads the default dataset (`data/dirty_hardware_data_40k.csv`) and **serves the GUI immediately**; initial embeddings run in the background (the graph appears when they finish—see `/api/status`). Use **Recalculate Clusters** or **Upload Dataset** to change the data or clustering.
+
+## Model
+
+The default embedding model is `sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2`. Use `--model` to pass a local model folder or another HuggingFace model ID.
+
+## Data
+
+CSV files live in the `data/` folder. The default dataset is `data/dirty_hardware_data_40k.csv`. Use `--data` or `--input`/`--output` to point scripts at other paths.
+
+## Data Processing Workflow
+
+- `dataset_generator.py`: Generates raw `data/dirty_hardware_data_40k.csv`.
+- `cluster_hardware` / CLI **`cluster-hardware`**: Produces `data/clustered_output.csv` from embeddings and clustering.
+- `prepare_viz_data.py`: Builds static `src/DustClust/cluster_viz/data.json` for the legacy static workflow.
+- `DustClust.server` / CLI **`dustclust`**: Serves the live app with on-demand clustering and device matching.

dustclust-0.1.0/pyproject.toml

@@ -0,0 +1,36 @@
+[build-system]
+requires = ["setuptools>=69", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "dustclust"
+version = "0.1.0"
+description = "DustClust: interactive hardware inventory clustering with embeddings and a web visualization"
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "MIT" }
+authors = [{ name = "DustClust contributors" }]
+dependencies = [
+    "fastapi",
+    "uvicorn[standard]",
+    "python-multipart",
+    "pandas",
+    "openpyxl",
+    "numpy",
+    "sentence-transformers",
+    "scikit-learn",
+]
+
+[project.scripts]
+dustclust = "DustClust.server:main"
+cluster-hardware = "DustClust.cluster_hardware:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+DustClust = [
+    "cluster_viz/**/*",
+    "data/**/*",
+    "ifixit_devices.json",
+]

dustclust-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

dustclust-0.1.0/src/DustClust/__init__.py

@@ -0,0 +1,3 @@
+"""DustClust: hardware dataset clustering with embeddings and web visualization."""
+
+__version__ = "0.1.0"

dustclust-0.1.0/src/DustClust/__main__.py

@@ -0,0 +1,4 @@
+from DustClust.server import main
+
+if __name__ == "__main__":
+    main()

dustclust-0.1.0/src/DustClust/cluster_hardware.py

@@ -0,0 +1,259 @@
+#!/usr/bin/env python3
+"""
+Cluster dirty hardware records using text embeddings + agglomerative hierarchical clustering.
+
+For large datasets (>10K rows), uses a two-phase approach:
+  Phase 1: Pre-group rows with MiniBatchKMeans into manageable chunks
+  Phase 2: Run agglomerative clustering within each chunk, then merge labels
+
+Usage:
+    python cluster_hardware.py --input data/dirty_hardware_data_40k.csv --threshold 0.3
+    python cluster_hardware.py --input data/dirty_hardware_data_40k.csv --threshold 0.2 --sample-size 5000
+"""
+
+import argparse
+import os
+import time
+import sys
+
+import pandas as pd
+import numpy as np
+from sentence_transformers import SentenceTransformer
+from sklearn.cluster import AgglomerativeClustering, MiniBatchKMeans
+
+
+# Maximum rows for direct agglomerative clustering (above this, use two-phase approach)
+DIRECT_CLUSTERING_LIMIT = 15_000
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="Cluster dirty hardware CSV records using embeddings + agglomerative clustering."
+    )
+    parser.add_argument(
+        "--input", default="data/dirty_hardware_data_40k.csv",
+        help="Path to the input CSV file (default: data/dirty_hardware_data_40k.csv)"
+    )
+    parser.add_argument(
+        "--output", default="data/clustered_output.csv",
+        help="Path to the output CSV file (default: data/clustered_output.csv)"
+    )
+    parser.add_argument(
+        "--threshold", type=float, default=0.3,
+        help="Distance threshold for clustering. Lower = tighter/more clusters, higher = looser/fewer clusters. "
+             "Range 0.0–2.0 for cosine distance (default: 0.3)"
+    )
+    parser.add_argument(
+        "--batch-size", type=int, default=512,
+        help="Batch size for embedding generation (default: 512)"
+    )
+    parser.add_argument(
+        "--sample-size", type=int, default=None,
+        help="Use only the first N rows for quick testing (default: use all rows)"
+    )
+    parser.add_argument(
+        "--model", default="sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2",
+        help="Path to local model folder or HuggingFace model ID (default: paraphrase-multilingual-MiniLM-L12-v2)"
+    )
+    parser.add_argument(
+        "--device", default="cpu",
+        help="Device for embedding model: 'cpu' or 'cuda' (default: cpu, avoids CUDA index errors with some models)"
+    )
+    parser.add_argument(
+        "--pre-clusters", type=int, default=None,
+        help="Number of KMeans pre-clusters for large datasets. Auto-calculated if not set."
+    )
+    return parser.parse_args()
+
+
+def load_data(path, sample_size=None):
+    """Load CSV and optionally sample rows."""
+    print(f"📂 Loading data from {path}...")
+    df = pd.read_csv(path, dtype=str).fillna("")
+    if sample_size is not None:
+        df = df.head(sample_size)
+    print(f"   Loaded {len(df):,} rows × {len(df.columns)} columns")
+    return df
+
+
+def build_text_representations(df):
+    """Concatenate all columns into a single text string per row."""
+    print("📝 Building text representations...")
+    texts = df.apply(lambda row: " | ".join(row.values), axis=1).tolist()
+    print(f"   Example: {texts[0]!r}")
+    return texts
+
+
+def generate_embeddings(texts, model_name, batch_size, device="cpu"):
+    """Generate embeddings using a sentence-transformer model."""
+    # Resolve local paths to absolute (avoids cwd issues)
+    if "/" not in model_name or os.path.exists(model_name):
+        model_path = os.path.abspath(model_name)
+    else:
+        model_path = model_name
+
+    print(f"🤖 Loading model '{model_path}' on {device}...")
+    model = SentenceTransformer(model_path, device=device)
+
+    print(f"⚡ Generating embeddings for {len(texts):,} texts (batch_size={batch_size})...")
+    t0 = time.time()
+    embeddings = model.encode(
+        texts,
+        batch_size=batch_size,
+        show_progress_bar=True,
+        normalize_embeddings=True,  # Pre-normalize for cosine similarity
+    )
+    elapsed = time.time() - t0
+    print(f"   Done in {elapsed:.1f}s — shape: {embeddings.shape}")
+    return embeddings
+
+
+def cluster_direct(embeddings, threshold):
+    """Run agglomerative clustering directly (for smaller datasets)."""
+    print(f"🔗 Clustering {len(embeddings):,} rows with distance_threshold={threshold}...")
+    t0 = time.time()
+    clustering = AgglomerativeClustering(
+        n_clusters=None,
+        distance_threshold=threshold,
+        metric="cosine",
+        linkage="average",
+    )
+    labels = clustering.fit_predict(embeddings)
+    elapsed = time.time() - t0
+    n_clusters = len(set(labels))
+    print(f"   Found {n_clusters:,} clusters in {elapsed:.1f}s")
+    return labels
+
+
+def cluster_twophase(embeddings, threshold, n_pre_clusters=None):
+    """
+    Two-phase clustering for large datasets:
+      Phase 1: MiniBatchKMeans to create manageable pre-groups
+      Phase 2: Agglomerative clustering within each pre-group
+    """
+    n = len(embeddings)
+    if n_pre_clusters is None:
+        # Aim for pre-groups of ~5000 rows each
+        n_pre_clusters = max(10, n // 5000)
+    n_pre_clusters = min(n_pre_clusters, n)
+
+    print(f"🔗 Phase 1: Pre-grouping {n:,} rows into {n_pre_clusters} groups with KMeans...")
+    t0 = time.time()
+    kmeans = MiniBatchKMeans(
+        n_clusters=n_pre_clusters,
+        batch_size=min(4096, n),
+        random_state=42,
+        n_init=3,
+    )
+    pre_labels = kmeans.fit_predict(embeddings)
+    elapsed = time.time() - t0
+    print(f"   Pre-grouping done in {elapsed:.1f}s")
+
+    print(f"🔗 Phase 2: Agglomerative clustering within each group (threshold={threshold})...")
+    t0 = time.time()
+    final_labels = np.zeros(n, dtype=int)
+    global_cluster_id = 0
+
+    for group_id in range(n_pre_clusters):
+        mask = pre_labels == group_id
+        group_embeddings = embeddings[mask]
+        group_size = len(group_embeddings)
+
+        if group_size <= 1:
+            final_labels[mask] = global_cluster_id
+            global_cluster_id += 1
+            continue
+
+        clustering = AgglomerativeClustering(
+            n_clusters=None,
+            distance_threshold=threshold,
+            metric="cosine",
+            linkage="average",
+        )
+        group_labels = clustering.fit_predict(group_embeddings)
+        n_group_clusters = len(set(group_labels))
+
+        # Map local cluster IDs to global IDs
+        final_labels[mask] = group_labels + global_cluster_id
+        global_cluster_id += n_group_clusters
+
+    elapsed = time.time() - t0
+    n_clusters = len(set(final_labels))
+    print(f"   Found {n_clusters:,} clusters in {elapsed:.1f}s")
+    return final_labels
+
+
+def cluster_embeddings(embeddings, threshold, n_pre_clusters=None):
+    """Choose clustering strategy based on dataset size."""
+    n = len(embeddings)
+    if n <= DIRECT_CLUSTERING_LIMIT:
+        return cluster_direct(embeddings, threshold)
+    else:
+        print(f"ℹ️  Dataset has {n:,} rows (>{DIRECT_CLUSTERING_LIMIT:,}), using two-phase clustering")
+        return cluster_twophase(embeddings, threshold, n_pre_clusters)
+
+
+def print_summary(df, max_clusters=30, samples_per_cluster=3):
+    """Print a summary of the clustering results."""
+    cluster_sizes = df.groupby("cluster_id").size().sort_values(ascending=False)
+    n_clusters = len(cluster_sizes)
+
+    print("\n" + "=" * 80)
+    print(f"📊 CLUSTERING SUMMARY")
+    print(f"   Total rows:     {len(df):,}")
+    print(f"   Total clusters: {n_clusters:,}")
+    print(f"   Largest:        {cluster_sizes.iloc[0]:,} rows")
+    print(f"   Smallest:       {cluster_sizes.iloc[-1]:,} rows")
+    print(f"   Median size:    {int(cluster_sizes.median()):,} rows")
+    print("=" * 80)
+
+    # Show the top clusters with sample rows
+    show_n = min(max_clusters, n_clusters)
+    print(f"\n🔍 Top {show_n} clusters by size:\n")
+
+    for i, (cluster_id, size) in enumerate(cluster_sizes.head(show_n).items()):
+        cluster_rows = df[df["cluster_id"] == cluster_id]
+        # Show a few sample rows (excluding the cluster_id column for readability)
+        display_cols = [c for c in df.columns if c != "cluster_id"]
+        samples = cluster_rows[display_cols].head(samples_per_cluster)
+
+        print(f"  Cluster {cluster_id} ({size:,} rows):")
+        for _, row in samples.iterrows():
+            print(f"    → {' | '.join(row.values)}")
+        print()
+
+
+def main():
+    args = parse_args()
+
+    print(f"\n{'=' * 80}")
+    print(f"  Hardware Record Clustering")
+    print(f"  Threshold: {args.threshold}  |  Model: {args.model}")
+    print(f"{'=' * 80}\n")
+
+    # 1. Load data
+    df = load_data(args.input, args.sample_size)
+
+    # 2. Build text representations
+    texts = build_text_representations(df)
+
+    # 3. Generate embeddings
+    embeddings = generate_embeddings(texts, args.model, args.batch_size, device=args.device)
+
+    # 4. Cluster
+    labels = cluster_embeddings(embeddings, args.threshold, args.pre_clusters)
+
+    # 5. Attach labels and sort by cluster
+    df["cluster_id"] = labels
+    df = df.sort_values("cluster_id").reset_index(drop=True)
+
+    # 6. Save output
+    df.to_csv(args.output, index=False)
+    print(f"\n💾 Saved clustered output to {args.output}")
+
+    # 7. Print summary
+    print_summary(df)
+
+
+if __name__ == "__main__":
+    main()