tactik 0.1.3__tar.gz

tactik-0.1.3/PKG-INFO

@@ -0,0 +1,269 @@
+Metadata-Version: 2.4
+Name: tactik
+Version: 0.1.3
+Summary: A Python library for topic modeling, clustering, and NLP analysis.
+Author-email: Your Name <you@example.com>
+License: MIT
+Project-URL: Homepage, https://github.com/yourusername/your-package
+Project-URL: Documentation, https://yourusername.github.io/your-package
+Project-URL: Source, https://github.com/yourusername/your-package
+Project-URL: Issues, https://github.com/yourusername/your-package/issues
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: license.txt
+Requires-Dist: matplotlib==3.8.0
+Requires-Dist: numpy==1.24.4
+Requires-Dist: pandas==1.5.3
+Requires-Dist: scikit-learn==1.2.2
+Requires-Dist: scipy==1.10.1
+Requires-Dist: seaborn==0.12.2
+Requires-Dist: tqdm==4.65.0
+Requires-Dist: gensim==4.3.0
+Requires-Dist: hdbscan==0.8.40
+Requires-Dist: nltk
+Requires-Dist: sentence-transformers==4.1.0
+Requires-Dist: spacy==3.8.7
+Requires-Dist: torch==2.3.0+cpu
+Requires-Dist: transformers==4.52.3
+Requires-Dist: umap-learn==0.5.7
+Requires-Dist: yake==0.4.8
+Dynamic: license-file
+
+<img src="tactik_header.png" alt="TACTIK Header" width="800" style="display:block;margin:auto;"/>
+
+# TACTIK
+## Text Analysis, Clustering, Tuning, Information and Keyword Extraction
+
+Tactik started as a side project to streamline clustering of aviation-related reports. The pipeline initially faced long processing times and became a bottleneck for analysis. These issues were addressed, and further functionality was added to enable intuitive topic extraction. The pipeline was adapted to work domain-agnostically while keeping the core use case in mind. With this functionality, we decided to release the package publicly so other researchers can contribute to it, build on it, or benefit from the included tools. Thank you for using TACTIK — we hope you find it as useful as we did in our research!
+
+## Features
+
+- **End-to-End Clustering Pipeline**: Automated workflow from preprocessing to cluster analysis  
+- **Modular Design**: Use the different components as standalone modules or full pipelines  
+- **Layered Effective Methods**: UMAP dimensionality reduction + HDBSCAN clustering  
+- **Hyperparameter Tuning**: Automated parameter optimization using random search  
+- **Keyword Extraction**: Multiple methods including TF, TF-IDF, DF, and YAKE  
+- **Topic Modeling**: LDA-based topic discovery with BERT-powered semantic matching *(still in development)*  
+- **Rich Visualizations**: t-SNE plots with customizable styling and annotations  
+- **Memory Efficient**: Optimized for large datasets with lazy evaluation and caching  
+
+## Installation
+
+```bash
+# Install tactik
+pip install tactik
+
+# Or install from source
+git clone https://github.com/npsAub/tactik.git
+cd tactik
+pip install -e .
+```
+
+### Dependencies
+
+```bash
+# Core dependencies
+pip install pandas numpy matplotlib seaborn scikit-learn
+pip install umap-learn hdbscan gensim nltk yake
+pip install transformers torch
+```
+
+## Core Components
+
+### 1. ClusteringPipeline
+Main orchestrator class that coordinates the entire analysis workflow.
+
+**Key Methods:**
+- `preprocess_data()` - Text cleaning and stopword removal  
+- `cluster_data()` - Clustering with fixed parameters  
+- `tune_and_cluster()` - Clustering with hyperparameter tuning  
+- `cluster_and_extract_keywords()` - Integrated clustering + keyword extraction  
+- `cluster_and_analyze_topics()` - Integrated clustering + topic modeling  
+- `visualize_clusters()` - Create cluster visualizations  
+- `get_cluster_summary()` - Generate cluster statistics  
+
+### 2. Clustering & Tuning
+Low-level clustering functions with hyperparameter optimization.
+
+**Key Functions:**
+- `tune_clustering_hyperparameters()` - Random search optimization  
+- `apply_best_clustering()` - Apply optimized parameters  
+- `full_clustering_pipeline()` - Complete pipeline with tuning  
+- `full_clustering_pipeline_fixed_params()` - Pipeline with fixed parameters  
+
+**Supported Metrics:**
+- `davies_bouldin`: Lower is better (measures cluster separation)  
+- `calinski_harabasz`: Higher is better (ratio of between/within cluster dispersion)  
+
+### 3. Keyword Extraction
+Extract representative keywords from each cluster using multiple methods.
+
+**KeywordExtractor Class:**
+- `extract_keywords_per_cluster()` - Extract keywords using multiple methods  
+- `save_keywords()` - Save results to CSV  
+
+**Extraction Methods:**
+- **TF**: Term Frequency  
+- **TF-IDF**: Term Frequency–Inverse Document Frequency  
+- **TF-DF**: Term Frequency–Document Frequency  
+- **YAKE**: Yet Another Keyword Extractor (long and short narratives)  
+
+### 4. Topic Modeling
+Discover latent topics using LDA and match them to predefined designators.
+
+**TopicModeler Class:**
+- `train_lda()` - Train Latent Dirichlet Allocation model  
+- `get_cluster_topics()` - Get top topics per cluster  
+- `match_designators_to_topics()` - Match topics to designators using BERT embeddings  
+- `get_bert_embedding()` - Compute BERT embeddings for semantic matching  
+
+**Default Aviation Safety Designators:**
+- Inadequate or inaccurate knowledge  
+- Poor judgment and decision-making  
+- Failure to follow procedures  
+- Poor communication  
+- Inadequate monitoring or vigilance  
+- Task management and prioritization  
+- Stress and psychological factors  
+- Physical or physiological factors  
+- Technical or system failures  
+- Environmental factors  
+
+### 5. Visualization
+Create publication-ready visualizations of clustering results.
+
+**Visualization Functions:**
+- `plot_clusters()` - Basic cluster scatter plot  
+- `plot_clusters_with_annotations()` - Plot with category annotations  
+- `plot_cluster_comparison()` - Side-by-side comparison plots  
+- `set_visualization_style()` - Configure plot styling  
+- `get_cluster_palette()` - Generate color palettes  
+- `get_cluster_markers()` - Generate marker styles  
+
+## Pipeline Architecture
+
+```
+Input Data (DataFrame)
+    ↓
+Preprocessing
+    ├── Text cleaning
+    ├── Stopword removal
+    └── Tokenization
+    ↓
+Vectorization (TF-IDF)
+    ↓
+Dimensionality Reduction (UMAP)
+    ↓
+Clustering (HDBSCAN)
+    ↓
+Visualization (t-SNE)
+    ↓
+Analysis
+    ├── Keyword Extraction
+    └── Topic Modeling (LDA + BERT)
+```
+
+## Performance Considerations
+
+### Memory Optimization
+- DataFrame lazy copying  
+- Vectorization caching  
+- BERT embedding cache with `clear_cache()` method  
+- Incremental topic probability calculations  
+
+### GPU Acceleration
+GPU acceleration is available for BERT computations when initializing TopicModeler with `use_gpu=True`.
+
+### Large Datasets
+For large datasets, consider:
+- Disabling t-SNE computation with `compute_tsne=False`  
+- Using fixed parameters instead of hyperparameter tuning  
+- Clearing BERT embedding cache periodically  
+
+## Evaluation Metrics
+
+- **Davies-Bouldin Score**: Measures average similarity between clusters (lower is better)  
+- **Calinski-Harabasz Score**: Ratio of between-cluster to within-cluster variance (higher is better)  
+- **Cluster Count**: Number of discovered clusters  
+- **Noise Ratio**: Proportion of outlier points  
+
+## Output Formats
+
+### Cluster Summary
+DataFrame with columns: Cluster ID, Size, Percentage  
+
+### Keywords DataFrame
+DataFrame with columns: cluster, Yake Long, Yake Short, TF, TFIDF, TFDF  
+
+### Topic Analysis
+Dictionary containing:
+- `cluster_topics`: Mapping of clusters to top topics  
+- `topic_designators`: Matching of topics to designators  
+- `model`: TopicModeler instance  
+
+## Dependencies
+
+- **Core**: pandas, numpy, scikit-learn  
+- **Clustering**: umap-learn, hdbscan  
+- **Visualization**: matplotlib, seaborn  
+- **NLP**: nltk, gensim, yake  
+- **Deep Learning**: transformers, torch  
+
+## Contributing
+
+Contributions are welcome! We encourage you to:
+
+- **Report Issues**: Found a bug or have a feature request? Open an issue on GitHub  
+- **Submit Pull Requests**: Improvements to code, documentation, or tests are appreciated  
+- **Share Use Cases**: Let us know how you're using tactik  
+- **Improve Documentation**: Help us make TACTIK more accessible  
+
+### Development Setup
+
+```bash
+# Clone the repository
+git clone https://github.com/npsAub/tactik.git
+cd tactik
+
+# Install in development mode with dev dependencies
+pip install -e ".[dev]"
+
+# Run tests
+unittest discover
+
+# Run linting
+flake8 tactik/
+black tactik/
+```
+
+### Contribution Guidelines
+
+1. Fork the repository and create a feature branch  
+2. Write clear, documented code following the existing style  
+3. Add tests for new functionality  
+4. Update documentation as needed  
+5. Submit a pull request with a clear description  
+
+For major changes, please open an issue first to discuss your proposal.
+
+## Citation
+
+If you use TACTIK in your research, please cite:
+
+```bibtex
+@software{tactik,
+  title={tactik: Text Analysis, Clustering, Tuning, Information and Keyword Extraction},
+  author={Niklas P. Schulmeyer and Nicoletta Fala},
+  year={2025},
+  url={https://github.com/npsAub/tactik}
+}
+```
+
+## License
+
+MIT License — See `LICENSE` for details
+
+## Contact
+
+For questions or support, please open an issue on GitHub or contact [nps0027@auburn.edu].
+

tactik-0.1.3/README.md

@@ -0,0 +1,238 @@
+<img src="tactik_header.png" alt="TACTIK Header" width="800" style="display:block;margin:auto;"/>
+
+# TACTIK
+## Text Analysis, Clustering, Tuning, Information and Keyword Extraction
+
+Tactik started as a side project to streamline clustering of aviation-related reports. The pipeline initially faced long processing times and became a bottleneck for analysis. These issues were addressed, and further functionality was added to enable intuitive topic extraction. The pipeline was adapted to work domain-agnostically while keeping the core use case in mind. With this functionality, we decided to release the package publicly so other researchers can contribute to it, build on it, or benefit from the included tools. Thank you for using TACTIK — we hope you find it as useful as we did in our research!
+
+## Features
+
+- **End-to-End Clustering Pipeline**: Automated workflow from preprocessing to cluster analysis  
+- **Modular Design**: Use the different components as standalone modules or full pipelines  
+- **Layered Effective Methods**: UMAP dimensionality reduction + HDBSCAN clustering  
+- **Hyperparameter Tuning**: Automated parameter optimization using random search  
+- **Keyword Extraction**: Multiple methods including TF, TF-IDF, DF, and YAKE  
+- **Topic Modeling**: LDA-based topic discovery with BERT-powered semantic matching *(still in development)*  
+- **Rich Visualizations**: t-SNE plots with customizable styling and annotations  
+- **Memory Efficient**: Optimized for large datasets with lazy evaluation and caching  
+
+## Installation
+
+```bash
+# Install tactik
+pip install tactik
+
+# Or install from source
+git clone https://github.com/npsAub/tactik.git
+cd tactik
+pip install -e .
+```
+
+### Dependencies
+
+```bash
+# Core dependencies
+pip install pandas numpy matplotlib seaborn scikit-learn
+pip install umap-learn hdbscan gensim nltk yake
+pip install transformers torch
+```
+
+## Core Components
+
+### 1. ClusteringPipeline
+Main orchestrator class that coordinates the entire analysis workflow.
+
+**Key Methods:**
+- `preprocess_data()` - Text cleaning and stopword removal  
+- `cluster_data()` - Clustering with fixed parameters  
+- `tune_and_cluster()` - Clustering with hyperparameter tuning  
+- `cluster_and_extract_keywords()` - Integrated clustering + keyword extraction  
+- `cluster_and_analyze_topics()` - Integrated clustering + topic modeling  
+- `visualize_clusters()` - Create cluster visualizations  
+- `get_cluster_summary()` - Generate cluster statistics  
+
+### 2. Clustering & Tuning
+Low-level clustering functions with hyperparameter optimization.
+
+**Key Functions:**
+- `tune_clustering_hyperparameters()` - Random search optimization  
+- `apply_best_clustering()` - Apply optimized parameters  
+- `full_clustering_pipeline()` - Complete pipeline with tuning  
+- `full_clustering_pipeline_fixed_params()` - Pipeline with fixed parameters  
+
+**Supported Metrics:**
+- `davies_bouldin`: Lower is better (measures cluster separation)  
+- `calinski_harabasz`: Higher is better (ratio of between/within cluster dispersion)  
+
+### 3. Keyword Extraction
+Extract representative keywords from each cluster using multiple methods.
+
+**KeywordExtractor Class:**
+- `extract_keywords_per_cluster()` - Extract keywords using multiple methods  
+- `save_keywords()` - Save results to CSV  
+
+**Extraction Methods:**
+- **TF**: Term Frequency  
+- **TF-IDF**: Term Frequency–Inverse Document Frequency  
+- **TF-DF**: Term Frequency–Document Frequency  
+- **YAKE**: Yet Another Keyword Extractor (long and short narratives)  
+
+### 4. Topic Modeling
+Discover latent topics using LDA and match them to predefined designators.
+
+**TopicModeler Class:**
+- `train_lda()` - Train Latent Dirichlet Allocation model  
+- `get_cluster_topics()` - Get top topics per cluster  
+- `match_designators_to_topics()` - Match topics to designators using BERT embeddings  
+- `get_bert_embedding()` - Compute BERT embeddings for semantic matching  
+
+**Default Aviation Safety Designators:**
+- Inadequate or inaccurate knowledge  
+- Poor judgment and decision-making  
+- Failure to follow procedures  
+- Poor communication  
+- Inadequate monitoring or vigilance  
+- Task management and prioritization  
+- Stress and psychological factors  
+- Physical or physiological factors  
+- Technical or system failures  
+- Environmental factors  
+
+### 5. Visualization
+Create publication-ready visualizations of clustering results.
+
+**Visualization Functions:**
+- `plot_clusters()` - Basic cluster scatter plot  
+- `plot_clusters_with_annotations()` - Plot with category annotations  
+- `plot_cluster_comparison()` - Side-by-side comparison plots  
+- `set_visualization_style()` - Configure plot styling  
+- `get_cluster_palette()` - Generate color palettes  
+- `get_cluster_markers()` - Generate marker styles  
+
+## Pipeline Architecture
+
+```
+Input Data (DataFrame)
+    ↓
+Preprocessing
+    ├── Text cleaning
+    ├── Stopword removal
+    └── Tokenization
+    ↓
+Vectorization (TF-IDF)
+    ↓
+Dimensionality Reduction (UMAP)
+    ↓
+Clustering (HDBSCAN)
+    ↓
+Visualization (t-SNE)
+    ↓
+Analysis
+    ├── Keyword Extraction
+    └── Topic Modeling (LDA + BERT)
+```
+
+## Performance Considerations
+
+### Memory Optimization
+- DataFrame lazy copying  
+- Vectorization caching  
+- BERT embedding cache with `clear_cache()` method  
+- Incremental topic probability calculations  
+
+### GPU Acceleration
+GPU acceleration is available for BERT computations when initializing TopicModeler with `use_gpu=True`.
+
+### Large Datasets
+For large datasets, consider:
+- Disabling t-SNE computation with `compute_tsne=False`  
+- Using fixed parameters instead of hyperparameter tuning  
+- Clearing BERT embedding cache periodically  
+
+## Evaluation Metrics
+
+- **Davies-Bouldin Score**: Measures average similarity between clusters (lower is better)  
+- **Calinski-Harabasz Score**: Ratio of between-cluster to within-cluster variance (higher is better)  
+- **Cluster Count**: Number of discovered clusters  
+- **Noise Ratio**: Proportion of outlier points  
+
+## Output Formats
+
+### Cluster Summary
+DataFrame with columns: Cluster ID, Size, Percentage  
+
+### Keywords DataFrame
+DataFrame with columns: cluster, Yake Long, Yake Short, TF, TFIDF, TFDF  
+
+### Topic Analysis
+Dictionary containing:
+- `cluster_topics`: Mapping of clusters to top topics  
+- `topic_designators`: Matching of topics to designators  
+- `model`: TopicModeler instance  
+
+## Dependencies
+
+- **Core**: pandas, numpy, scikit-learn  
+- **Clustering**: umap-learn, hdbscan  
+- **Visualization**: matplotlib, seaborn  
+- **NLP**: nltk, gensim, yake  
+- **Deep Learning**: transformers, torch  
+
+## Contributing
+
+Contributions are welcome! We encourage you to:
+
+- **Report Issues**: Found a bug or have a feature request? Open an issue on GitHub  
+- **Submit Pull Requests**: Improvements to code, documentation, or tests are appreciated  
+- **Share Use Cases**: Let us know how you're using tactik  
+- **Improve Documentation**: Help us make TACTIK more accessible  
+
+### Development Setup
+
+```bash
+# Clone the repository
+git clone https://github.com/npsAub/tactik.git
+cd tactik
+
+# Install in development mode with dev dependencies
+pip install -e ".[dev]"
+
+# Run tests
+unittest discover
+
+# Run linting
+flake8 tactik/
+black tactik/
+```
+
+### Contribution Guidelines
+
+1. Fork the repository and create a feature branch  
+2. Write clear, documented code following the existing style  
+3. Add tests for new functionality  
+4. Update documentation as needed  
+5. Submit a pull request with a clear description  
+
+For major changes, please open an issue first to discuss your proposal.
+
+## Citation
+
+If you use TACTIK in your research, please cite:
+
+```bibtex
+@software{tactik,
+  title={tactik: Text Analysis, Clustering, Tuning, Information and Keyword Extraction},
+  author={Niklas P. Schulmeyer and Nicoletta Fala},
+  year={2025},
+  url={https://github.com/npsAub/tactik}
+}
+```
+
+## License
+
+MIT License — See `LICENSE` for details
+
+## Contact
+
+For questions or support, please open an issue on GitHub or contact [nps0027@auburn.edu].
+

tactik-0.1.3/license.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 TACTIC Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

tactik-0.1.3/pyproject.toml

@@ -0,0 +1,43 @@
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "tactik"
+version = "0.1.3"
+description = "A Python library for topic modeling, clustering, and NLP analysis."
+readme = "README.md"
+requires-python = ">=3.8"
+license = { text = "MIT" }
+
+authors = [
+    { name = "Your Name", email = "you@example.com" }
+]
+
+dependencies = [
+    "matplotlib==3.8.0",
+    "numpy==1.24.4",
+    "pandas==1.5.3",
+    "scikit-learn==1.2.2",
+    "scipy==1.10.1",
+    "seaborn==0.12.2",
+    "tqdm==4.65.0",
+    "gensim==4.3.0",
+    "hdbscan==0.8.40",
+    "nltk",
+    "sentence-transformers==4.1.0",
+    "spacy==3.8.7",
+    "torch==2.3.0+cpu",
+    "transformers==4.52.3",
+    "umap-learn==0.5.7",
+    "yake==0.4.8"
+]
+
+[project.urls]
+Homepage = "https://github.com/yourusername/your-package"
+Documentation = "https://yourusername.github.io/your-package"
+Source = "https://github.com/yourusername/your-package"
+Issues = "https://github.com/yourusername/your-package/issues"
+
+[tool.setuptools.packages.find]
+where = ["src"]

tactik-0.1.3/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

tactik-0.1.3/src/tactik/__init__.py

@@ -0,0 +1,19 @@
+
+"""
+TACTIK Package
+Text Analysis, Clustering, Tuning, Information and Keyword Extraction
+"""
+
+# Package version
+__version__ = "0.1.4"
+
+
+from . import clustering_pipeline
+from . import clustering_tuning
+from . import embeddings
+from . import preprocessing
+from . import topic_extraction
+from . import utilities
+from . import visualization
+
+