imageatlas 0.1.0__tar.gz

imageatlas-0.1.0/CHANGELOG.md

@@ -0,0 +1,16 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0] - 2024-01-22
+### Added
+- Core `ImageClusterer` API for high-level clustering workflows.
+- Feature extraction support for DINOv2, ResNet, ViT, CLIP, Swin, and more.
+- Clustering algorithms: K-Means, GMM, and HDBSCAN.
+- Dimensionality reduction wrappers for PCA, UMAP, and t-SNE.
+- Visualization tools (`GridVisualizer`) for creating image grids from clusters.
+- HDF5 caching system for efficient feature storage.
+- Export functionality to CSV, JSON, and Excel.

imageatlas-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,55 @@
+# Contributing to Image Clustering and Visualization Project
+
+
+Thank you for considering contributing to this project! Your contributions are vital in making this tool more effective and accessible. This document outlines the guidelines for contributing to the project.
+
+
+## Getting Started
+
+Before you begin contributing, make sure to follow the steps below to set up the project locally and familiarize yourself with the codebase.
+
+
+### Prerequisites
+
+- Python 3.8+
+- Pip or Conda for package management
+- GPU (optional, but recommended for faster feature extraction)
+
+
+### Installation
+
+1. Clone the repository:
+
+   ```bash
+   git clone https://github.com/ahmadjaved97/ImageClusterViz.git
+   cd ImageClusterViz
+2. Set up a virtual environment:
+
+    ```bash
+    python -m venv venv
+    source venv/bin/activate
+3. Install the required dependencies:
+    
+    ```bash
+    pip install -r requirements.txt
+## How to Contribute
+
+We encourage you to contribute to the project by reporting bugs, suggesting new features, improving documentation, or submitting code contributions.
+
+### Reporting Issues
+
+If you encounter any bugs or have suggestions for improvements, please create an issue on the [GitHub issue tracker](https://github.com/ahmadjaved97/ImageClusterViz/issues). Please provide a clear and concise description of the issue, including steps to reproduce it if it's a bug.
+
+### Suggesting Features
+
+Feature requests are also welcome! Please open an issue for feature suggestions and provide details on how the feature would enhance the project. When possible, describe the use case to help us understand your needs.
+
+
+### Community Guidelines
+- Be respectful and constructive in your communication.
+- Provide clear and concise commit messages and issue descriptions.
+- Ensure that any feedback provided to other contributors is thoughtful and - actionable.
+
+## License
+
+By contributing to this project, you agree that your contributions will be licensed under the MIT License.

imageatlas-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Ahmad Javed
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

imageatlas-0.1.0/MANIFEST.in

@@ -0,0 +1,62 @@
+# MANIFEST.in - Specifies additional files to include in the distribution
+
+# Include important documentation files
+include README.md
+include LICENSE
+include CONTRIBUTING.md
+include CHANGELOG.md
+include requirements.txt
+include pyproject.toml
+
+# Include all Python files in the package
+recursive-include imageatlas *.py
+
+# Include type hints marker
+include imageatlas/py.typed
+
+# Include test files
+recursive-include tests *.py
+
+# Include example files
+recursive-include examples *.py
+recursive-include examples *.ipynb
+
+# Exclude compiled Python files
+global-exclude __pycache__
+global-exclude *.py[cod]
+global-exclude *.so
+global-exclude .DS_Store
+
+# Exclude cache and build artifacts
+global-exclude *.egg-info
+global-exclude .pytest_cache
+global-exclude .tox
+global-exclude .coverage
+global-exclude htmlcov
+
+# Exclude version control files
+global-exclude .git*
+global-exclude .gitignore
+
+# Exclude IDE and editor files
+global-exclude .vscode
+global-exclude .idea
+global-exclude *.swp
+global-exclude *.swo
+global-exclude *~
+
+# Exclude output directories
+global-exclude output
+global-exclude output_grids
+global-exclude output_clusters
+
+# Exclude virtual environment directories
+global-exclude venv
+global-exclude .venv
+global-exclude env
+
+# Exclude HDF5 cache files and pickle files
+global-exclude *.h5
+global-exclude *.hdf5
+global-exclude *.pkl
+global-exclude *.pickle

imageatlas-0.1.0/PKG-INFO

@@ -0,0 +1,203 @@
+Metadata-Version: 2.4
+Name: imageatlas
+Version: 0.1.0
+Summary: ImageAtlas: A toolkit for organizing, cleaning and analysing your image datasets.
+Author-email: Ahmad Javed <ahmadjaved97@gmail.com>
+Maintainer-email: Ahmad Javed <ahmadjaved97@gmail.com>
+License: MIT License
+        
+        Copyright (c) 2024 Ahmad Javed
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+        
+Project-URL: Homepage, https://github.com/ahmadjaved97/imageatlas
+Project-URL: Documentation, https://github.com/ahmadjaved97/imageatlas
+Project-URL: Repository, https://github.com/ahmadjaved97/imageatlas
+Project-URL: Issues, https://github.com/ahmadjaved97/imageatlas/issues
+Project-URL: Changelog, https://github.com/ahmadjaved97/imageatlas/blob/main/CHANGELOG.md
+Keywords: machine-learning,computer-vision,clustering,embeddings,feature-extraction,dataset-visualization,deep-learning,image-processing,data-science,pytorch
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Scientific/Engineering :: Image Recognition
+Classifier: Topic :: Scientific/Engineering :: Visualization
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: torch>=2.0.1
+Requires-Dist: torchvision>=0.15.2
+Requires-Dist: numpy>=1.24.2
+Requires-Dist: Pillow>=9.5.0
+Requires-Dist: opencv-python>=4.8.0.74
+Requires-Dist: scikit-learn>=1.3.0
+Requires-Dist: tqdm>=4.67.1
+Requires-Dist: h5py>=3.15.1
+Requires-Dist: pandas>=2.3.3
+Provides-Extra: full
+Requires-Dist: umap-learn; extra == "full"
+Requires-Dist: hdbscan; extra == "full"
+Requires-Dist: openpyxl; extra == "full"
+Dynamic: license-file
+
+# ImageAtlas
+
+## Overview
+
+ImageAtlas is a comprehensive toolkit designed to organize, clean, and analyze image datasets.
+
+⚠️ Note: ImageAtlas is currently in active development. The current version focuses on clustering and visualization functionality, with additional features coming soon.
+
+Perfect for dataset curation, duplicate detection, quality control, and exploratory data analysis.
+
+## 📦 Installation
+
+**Basic Installation**
+
+```
+pip install imageatlas
+```
+
+**Full Installation**
+
+```
+pip install imageatlas[full]
+```
+
+**From Source**
+```
+git clone https://github.com/ahmadjaved97/ImageAtlas.git
+cd ImageAtlas
+pip install -e .
+```
+
+## 🚀 Quick Start
+
+### Minimal Working Example
+
+```python
+import os
+from imageatlas import ImageClusterer
+
+# Initialize clusterer
+clusterer = ImageClusterer(
+    model='dinov2',           # State-of-the-art features
+    clustering_method='kmeans',
+    n_clusters=10,
+    device='cuda'             # or 'cpu'
+)
+
+# Run clustering on your images
+results = clusterer.fit("./path/to/images")
+
+# Save results to JSON
+results.to_json("./output/clustering_results.json")
+
+# Create visual grids for each cluster
+results.create_grids(
+    image_dir="./path/to/images",
+    output_dir="./output/grids"
+)
+
+# Organize images into cluster folders
+results.create_cluster_folders(
+    image_dir="./path/to/images",
+    output_dir="./output/clusters"
+)
+```
+
+That's it! Your images are now clustered, visualized, and organized.
+
+## Available Models & Algorithms
+
+### Feature Extraction Models
+
+| Model            | Variants                                            |
+| ---------------- | --------------------------------------------------- |
+| **DINOv2**       | `vits14`, `vitb14`, `vitl14`, `vitg14`              |
+| **ViT**          | `b_16`, `b_32`, `l_16`, `l_32`, `h_14`              |
+| **ResNet**       | `18`, `34`, `50`, `101`, `152`                      |
+| **EfficientNet** | `s`, `m`, `l`                                       |
+| **CLIP**         | `RN50`, `RN101`, `ViT-B/32`, `ViT-B/16`, `ViT-L/14` |
+| **ConvNeXt**     | `tiny`, `small`, `base`, `large`                    |
+| **Swin**         | `t`, `s`, `b`, `v2_t`, `v2_s`, `v2_b`               |
+| **MobileNetV3**  | `small`, `large`                                    |
+| **VGG16**        | \-                                                  |
+
+### Clustering Algorithms
+
+| Algorithm   | Parameters                        |
+| ----------- | --------------------------------- |
+| **K-Means** | `n_clusters`                      |
+| **HDBSCAN** | `min_cluster_size`, `min_samples` |
+| **GMM**     | `n_components`, `covariance_type` |
+
+### Dimensionality Reduction
+
+| Method                    | Parameters                                |
+| --------------------------| ----------------------------------------- |
+| **PCA**                   | `n_components`, `whiten`                  |
+| **UMAP**                  | `n_components`, `n_neighbors`, `min_dist` |
+| **t-SNE(in development)** | `n_components`, `perplexity`              |
+
+
+## 📝 Citation
+
+If you use ImageAtlas in your research, please cite:
+
+```bibtex
+@software{imageatlas2024,
+  author = {Javed, Ahmad},
+  title = {ImageAtlas: A Toolkit for Organizing and Analyzing Image Datasets},
+  year = {2024},
+  url = {https://github.com/ahmadjaved97/ImageAtlas}
+}
+```
+##  Acknowledgments
+
+- [DINOv2](https://github.com/facebookresearch/dinov2): Facebook Research
+- [CLIP](https://github.com/openai/CLIP): OpenAI
+- [Vision Transformers](https://github.com/google-research/vision_transformer): Google Research
+- Built with [PyTorch](https://github.com/pytorch/pytorch), [scikit-learn](https://github.com/scikit-learn/scikit-learn), and [OpenCV](https://github.com/opencv/opencv)
+
+
+### Sample Output
+- Dataset Used: [Fruit and Vegetable Classification](https://www.kaggle.com/code/abdelrahman16/fruit-and-vegetable-classification/input)
+- Number of Clusters: 8
+- Model Used: ViT
+- Clustering Method: Kmeans
+- Output:
+    <p align="center">
+        <img src="./output_grids/cluster_0.jpg" alt="Image 1" width="250" height="250">
+        <img src="./output_grids/cluster_1.jpg" alt="Image 2" width="250" height="250">
+        <img src="./output_grids/cluster_2.jpg" alt="Image 3" width="250" height= "250">
+        <img src="./output_grids/cluster_3.jpg" alt="Image 3" width="250" height= "250">
+        <img src="./output_grids/cluster_4.jpg" alt="Image 3" width="250" height= "250">
+        <img src="./output_grids/cluster_5.jpg" alt="Image 3" width="250" height= "250">
+        <img src="./output_grids/cluster_6.jpg" alt="Image 3" width="250" height= "250">
+        <img src="./output_grids/cluster_7.jpg" alt="Image 3" width="250" height= "250">
+    </p>
+
+

imageatlas-0.1.0/README.md

@@ -0,0 +1,139 @@
+# ImageAtlas
+
+## Overview
+
+ImageAtlas is a comprehensive toolkit designed to organize, clean, and analyze image datasets.
+
+⚠️ Note: ImageAtlas is currently in active development. The current version focuses on clustering and visualization functionality, with additional features coming soon.
+
+Perfect for dataset curation, duplicate detection, quality control, and exploratory data analysis.
+
+## 📦 Installation
+
+**Basic Installation**
+
+```
+pip install imageatlas
+```
+
+**Full Installation**
+
+```
+pip install imageatlas[full]
+```
+
+**From Source**
+```
+git clone https://github.com/ahmadjaved97/ImageAtlas.git
+cd ImageAtlas
+pip install -e .
+```
+
+## 🚀 Quick Start
+
+### Minimal Working Example
+
+```python
+import os
+from imageatlas import ImageClusterer
+
+# Initialize clusterer
+clusterer = ImageClusterer(
+    model='dinov2',           # State-of-the-art features
+    clustering_method='kmeans',
+    n_clusters=10,
+    device='cuda'             # or 'cpu'
+)
+
+# Run clustering on your images
+results = clusterer.fit("./path/to/images")
+
+# Save results to JSON
+results.to_json("./output/clustering_results.json")
+
+# Create visual grids for each cluster
+results.create_grids(
+    image_dir="./path/to/images",
+    output_dir="./output/grids"
+)
+
+# Organize images into cluster folders
+results.create_cluster_folders(
+    image_dir="./path/to/images",
+    output_dir="./output/clusters"
+)
+```
+
+That's it! Your images are now clustered, visualized, and organized.
+
+## Available Models & Algorithms
+
+### Feature Extraction Models
+
+| Model            | Variants                                            |
+| ---------------- | --------------------------------------------------- |
+| **DINOv2**       | `vits14`, `vitb14`, `vitl14`, `vitg14`              |
+| **ViT**          | `b_16`, `b_32`, `l_16`, `l_32`, `h_14`              |
+| **ResNet**       | `18`, `34`, `50`, `101`, `152`                      |
+| **EfficientNet** | `s`, `m`, `l`                                       |
+| **CLIP**         | `RN50`, `RN101`, `ViT-B/32`, `ViT-B/16`, `ViT-L/14` |
+| **ConvNeXt**     | `tiny`, `small`, `base`, `large`                    |
+| **Swin**         | `t`, `s`, `b`, `v2_t`, `v2_s`, `v2_b`               |
+| **MobileNetV3**  | `small`, `large`                                    |
+| **VGG16**        | \-                                                  |
+
+### Clustering Algorithms
+
+| Algorithm   | Parameters                        |
+| ----------- | --------------------------------- |
+| **K-Means** | `n_clusters`                      |
+| **HDBSCAN** | `min_cluster_size`, `min_samples` |
+| **GMM**     | `n_components`, `covariance_type` |
+
+### Dimensionality Reduction
+
+| Method                    | Parameters                                |
+| --------------------------| ----------------------------------------- |
+| **PCA**                   | `n_components`, `whiten`                  |
+| **UMAP**                  | `n_components`, `n_neighbors`, `min_dist` |
+| **t-SNE(in development)** | `n_components`, `perplexity`              |
+
+
+## 📝 Citation
+
+If you use ImageAtlas in your research, please cite:
+
+```bibtex
+@software{imageatlas2024,
+  author = {Javed, Ahmad},
+  title = {ImageAtlas: A Toolkit for Organizing and Analyzing Image Datasets},
+  year = {2024},
+  url = {https://github.com/ahmadjaved97/ImageAtlas}
+}
+```
+##  Acknowledgments
+
+- [DINOv2](https://github.com/facebookresearch/dinov2): Facebook Research
+- [CLIP](https://github.com/openai/CLIP): OpenAI
+- [Vision Transformers](https://github.com/google-research/vision_transformer): Google Research
+- Built with [PyTorch](https://github.com/pytorch/pytorch), [scikit-learn](https://github.com/scikit-learn/scikit-learn), and [OpenCV](https://github.com/opencv/opencv)
+
+
+### Sample Output
+- Dataset Used: [Fruit and Vegetable Classification](https://www.kaggle.com/code/abdelrahman16/fruit-and-vegetable-classification/input)
+- Number of Clusters: 8
+- Model Used: ViT
+- Clustering Method: Kmeans
+- Output:
+    <p align="center">
+        <img src="./output_grids/cluster_0.jpg" alt="Image 1" width="250" height="250">
+        <img src="./output_grids/cluster_1.jpg" alt="Image 2" width="250" height="250">
+        <img src="./output_grids/cluster_2.jpg" alt="Image 3" width="250" height= "250">
+        <img src="./output_grids/cluster_3.jpg" alt="Image 3" width="250" height= "250">
+        <img src="./output_grids/cluster_4.jpg" alt="Image 3" width="250" height= "250">
+        <img src="./output_grids/cluster_5.jpg" alt="Image 3" width="250" height= "250">
+        <img src="./output_grids/cluster_6.jpg" alt="Image 3" width="250" height= "250">
+        <img src="./output_grids/cluster_7.jpg" alt="Image 3" width="250" height= "250">
+    </p>
+
+