MDANCE 0.0.1__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of MDANCE might be problematic. Click here for more details.
- MDANCE-0.0.1.dist-info/LICENSE +21 -0
- MDANCE-0.0.1.dist-info/METADATA +80 -0
- MDANCE-0.0.1.dist-info/RECORD +21 -0
- MDANCE-0.0.1.dist-info/WHEEL +5 -0
- MDANCE-0.0.1.dist-info/top_level.txt +1 -0
- mdance/__init__.py +1 -0
- mdance/inputs/__init__.py +0 -0
- mdance/inputs/preprocess.py +239 -0
- mdance/modules/PRIME/__init__.py +0 -0
- mdance/modules/PRIME/rep_frames.py +132 -0
- mdance/modules/PRIME/rep_frames_cl.py +33 -0
- mdance/modules/PRIME/sim_calc.py +280 -0
- mdance/modules/PRIME/similarity_cl.py +65 -0
- mdance/modules/kmeansNANI/__init__.py +0 -0
- mdance/modules/kmeansNANI/nani.py +257 -0
- mdance/outputs/__init__.py +0 -0
- mdance/outputs/postprocess.py +41 -0
- mdance/tools/__init__.py +0 -0
- mdance/tools/bts.py +530 -0
- mdance/tools/esim_modules.py +486 -0
- mdance/tools/isim_modules.py +229 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2024 mqcomplab
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
Metadata-Version: 2.1
|
|
2
|
+
Name: MDANCE
|
|
3
|
+
Version: 0.0.1
|
|
4
|
+
Summary: flexible n-ary clustering package for all applications.
|
|
5
|
+
Author-email: Lexin Chen <lxchen017@gmail.com>
|
|
6
|
+
Project-URL: Homepage, https://github.com/mqcomplab/MDANCE
|
|
7
|
+
Project-URL: Issues, https://github.com/mqcomplab/MDANCE/issues
|
|
8
|
+
Classifier: Programming Language :: Python :: 3
|
|
9
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
10
|
+
Classifier: Operating System :: OS Independent
|
|
11
|
+
Requires-Python: >=3.8
|
|
12
|
+
Description-Content-Type: text/markdown
|
|
13
|
+
License-File: LICENSE
|
|
14
|
+
|
|
15
|
+
# Molecular Dynamics Analysis with *N*-ary Clustering Ensembles (MDANCE)
|
|
16
|
+
Molecular Dynamics Analysis with *N*-ary Clustering Ensembles (MDANCE) is a flexible *n*-ary clustering package that provides a set of tools for clustering Molecular Dynamics trajectories. The package is written in Python and an extension of the *n*-ary similarity framework. The package is designed to be modular and extensible, allowing for the addition of new clustering algorithms and similarity metrics.
|
|
17
|
+
|
|
18
|
+
## Table of Contents
|
|
19
|
+
- [Background](#background)
|
|
20
|
+
- [Clustering Algorithms](#clustering-algorithms)
|
|
21
|
+
- [NANI](#nani)
|
|
22
|
+
- [Clustering Postprocessing](#clustering-postprocessing)
|
|
23
|
+
- [PRIME](#prime)
|
|
24
|
+
- [Installation](#installation)
|
|
25
|
+
|
|
26
|
+
## Background
|
|
27
|
+
Molecular Dynamics (MD) simulations are a powerful tool for studying the dynamics of biomolecules. However, the analysis of MD trajectories is challenging due to the large amount of data generated. Clustering is an unsupervised machine learning approach to group similar frames into clusters. The clustering results can be used to reveal the structure of the data, identify the most representative structures, and to study the dynamics of the system.
|
|
28
|
+
|
|
29
|
+
## Installation
|
|
30
|
+
```
|
|
31
|
+
pip install mdance
|
|
32
|
+
```
|
|
33
|
+
To check for proper installation, run the following command:
|
|
34
|
+
```python
|
|
35
|
+
import mdance
|
|
36
|
+
print(mdance.__version__)
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
## Clustering Algorithms
|
|
40
|
+
### NANI
|
|
41
|
+
<p align="center">
|
|
42
|
+
<img src="img/nani-logo.PNG" width="200" height=auto align="center"></a></p>
|
|
43
|
+
|
|
44
|
+
<h3 align="center">
|
|
45
|
+
<p><b>🪄NANI🪄the first installment of MDANCE</b></p>
|
|
46
|
+
</h3>
|
|
47
|
+
|
|
48
|
+
*k*-Means *N*-Ary Natural Initiation (NANI) is an algorithm for selecting initial centroids for *k*-Means clustering. NANI is an extension of the *k*-Means++ algorithm. NANI stratifies the data to high density region and perform diversity selection on top of the it to select the initial centroids. This is a deterministic algorithm that will always select the same initial centroids for the same dataset and improve on *k*-means++ by reducing the number of iterations required to converge and improve the clustering quality.
|
|
49
|
+
|
|
50
|
+
**[A tutorial is available for NANI](https://github.com/mqcomplab/MDANCE/blob/main/tutorials/nani.md).**
|
|
51
|
+
|
|
52
|
+
For more information on the NANI algorithm, please refer to the [NANI paper](https://www.biorxiv.org/content/10.1101/2024.03.07.583975v1).
|
|
53
|
+
|
|
54
|
+
|
|
55
|
+
## Clustering Postprocessing
|
|
56
|
+
### PRIME
|
|
57
|
+
<h3 align="center">
|
|
58
|
+
<img src="img/logo.png" width="800" height=auto align="center"></a>
|
|
59
|
+
 
|
|
60
|
+
<p><b>🪄 Predict Protein Structure with Precision 🪄</b></p>
|
|
61
|
+
</h3>
|
|
62
|
+
|
|
63
|
+
<table>
|
|
64
|
+
<tr>
|
|
65
|
+
<td>
|
|
66
|
+
<p>Protein Retrieval via Integrative Molecular Ensembles (PRIME)</b> is a novel algorithm that predicts the native structure of a protein from simulation or clustering data. These methods perfectly mapped all the structural motifs in the studied systems and required unprecedented linear scaling.</p>
|
|
67
|
+
</td>
|
|
68
|
+
<td>
|
|
69
|
+
<figure>
|
|
70
|
+
<img src="img/2k2e.png" alt="2k2e" width="300" height="auto">
|
|
71
|
+
<figcaption><i>Fig 1. Superposition of the most representative structures found with extended indices (yellow) and experimental native structures (blue) of 2k2e.</i></figcaption>
|
|
72
|
+
</figure>
|
|
73
|
+
</td>
|
|
74
|
+
</tr>
|
|
75
|
+
</table>
|
|
76
|
+
|
|
77
|
+
**[A tutorial is available for PRIME](https://github.com/mqcomplab/MDANCE/blob/main/tutorials/prime.md).**
|
|
78
|
+
|
|
79
|
+
For more information on the PRIME algorithm, please refer to the [PRIME paper](https://www.biorxiv.org/content/10.1101/2024.03.19.585783v1).
|
|
80
|
+
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
mdance/__init__.py,sha256=frcCV1k9oG9oKj3dpUqdJg1PxRT2RSN_XKdLCPjaYaY,2
|
|
2
|
+
mdance/inputs/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
3
|
+
mdance/inputs/preprocess.py,sha256=ZuKLJI0z0SEozFP6fWcAsuRY2pnxoUOWHxFNAHIP9ao,10128
|
|
4
|
+
mdance/modules/PRIME/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
5
|
+
mdance/modules/PRIME/rep_frames.py,sha256=MoWypCulyIDcMBydKj34jxdelyuOv_LY35R0g8BQeKQ,5714
|
|
6
|
+
mdance/modules/PRIME/rep_frames_cl.py,sha256=kBKl5I84p-oQ1eJA6LlLiVvTKES5NlQt5QzvcnmIrBA,1495
|
|
7
|
+
mdance/modules/PRIME/sim_calc.py,sha256=rg54-2NfuLKXNyq_JiBtbXxIKyUQDKUKBYhGXu0-o4I,13406
|
|
8
|
+
mdance/modules/PRIME/similarity_cl.py,sha256=nw4Xe3z4-K5jbu1TPjLRlmftSrHikAIu2DZ-y7Gnv5Y,2761
|
|
9
|
+
mdance/modules/kmeansNANI/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
10
|
+
mdance/modules/kmeansNANI/nani.py,sha256=bkdBpMyQcWEaZA5HF_YOXF_0azvvdSYUanG7qSwJMgc,10676
|
|
11
|
+
mdance/outputs/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
12
|
+
mdance/outputs/postprocess.py,sha256=4Bq2U-nS30wRaKA91k-L09Fe6U2FS3bJ9xIMZBReXU4,1255
|
|
13
|
+
mdance/tools/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
14
|
+
mdance/tools/bts.py,sha256=ACaO2IoZwL9EmOFOcyvVdTlZCIWi72ly3Llj6RDo4mE,20690
|
|
15
|
+
mdance/tools/esim_modules.py,sha256=UmZv4kgT3-3vPUOEhbsUqCrOoooH9Koed5_OPnBaDQ8,17793
|
|
16
|
+
mdance/tools/isim_modules.py,sha256=rOKFjsCK6gy8Sf4c4gJ7k1cyFcpB1a2xBHS9mcY9M2w,8248
|
|
17
|
+
MDANCE-0.0.1.dist-info/LICENSE,sha256=V7J3Pwcll1WnrOoMh0PnnR1RPudUaioDSxQH7sM46po,1087
|
|
18
|
+
MDANCE-0.0.1.dist-info/METADATA,sha256=dGUcnvHzX1pf1BAvx2velxEJdkD-4r1UVjvk5RtRuOw,4071
|
|
19
|
+
MDANCE-0.0.1.dist-info/WHEEL,sha256=GJ7t_kWBFywbagK5eo9IoUwLW6oyOeTKmQ-9iHFVNxQ,92
|
|
20
|
+
MDANCE-0.0.1.dist-info/top_level.txt,sha256=9e0J6OaYuWt9e4P_kyxCnVl8syLDHZaU3_iXyQqeC4Q,7
|
|
21
|
+
MDANCE-0.0.1.dist-info/RECORD,,
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
mdance
|
mdance/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
File without changes
|
|
@@ -0,0 +1,239 @@
|
|
|
1
|
+
import glob
|
|
2
|
+
from itertools import chain
|
|
3
|
+
import MDAnalysis as mda
|
|
4
|
+
import numpy as np
|
|
5
|
+
import re
|
|
6
|
+
|
|
7
|
+
def gen_traj_numpy(prmtopFileName, trajFileName, atomSel):
|
|
8
|
+
"""Reads in a trajectory and returns a 2D numpy array of the coordinates
|
|
9
|
+
of the selected atoms.
|
|
10
|
+
|
|
11
|
+
Parameters
|
|
12
|
+
----------
|
|
13
|
+
prmtopFileName : str
|
|
14
|
+
The file path of the topology file.
|
|
15
|
+
trajFileName : str
|
|
16
|
+
The file path of the trajectory file.
|
|
17
|
+
atomSel : str
|
|
18
|
+
The atom selection string. For example, 'resid 3:12 and name N H CA C O'.
|
|
19
|
+
View details in the MDAnalysis documentation:
|
|
20
|
+
https://docs.mdanalysis.org/stable/documentation_pages/selections.html
|
|
21
|
+
|
|
22
|
+
Returns
|
|
23
|
+
-------
|
|
24
|
+
traj_numpy : np.ndarray
|
|
25
|
+
The 2D numpy array of the coordinates of the selected atoms.
|
|
26
|
+
|
|
27
|
+
Examples
|
|
28
|
+
--------
|
|
29
|
+
>>> traj_numpy = gen_traj_numpy('aligned_tau.pdb', 'aligned_tau.dcd',
|
|
30
|
+
'resid 3:12 and name N CA C')
|
|
31
|
+
"""
|
|
32
|
+
coord = mda.Universe(prmtopFileName,trajFileName)
|
|
33
|
+
print('Number of atoms in trajectory:', coord.atoms.n_atoms)
|
|
34
|
+
print('Number of frames in trajectory:', coord.trajectory.n_frames)
|
|
35
|
+
atomSel = coord.select_atoms(atomSel)
|
|
36
|
+
print('Number of atoms in selection:', atomSel.n_atoms)
|
|
37
|
+
# Create traj data of the atom selection
|
|
38
|
+
traj_numpy = np.empty((coord.trajectory.n_frames,atomSel.n_atoms, 3), dtype=float)
|
|
39
|
+
# Loop every frame and store the coordinates of the atom selection
|
|
40
|
+
for ts in coord.trajectory:
|
|
41
|
+
traj_numpy[ts.frame,:] = atomSel.positions
|
|
42
|
+
# Flatten 3D array to 2D array
|
|
43
|
+
traj_numpy = traj_numpy.reshape(traj_numpy.shape[0],-1)
|
|
44
|
+
return traj_numpy
|
|
45
|
+
|
|
46
|
+
class Normalizer:
|
|
47
|
+
"""Class for normalizing data from cpptraj CRD/MDCRD files.
|
|
48
|
+
|
|
49
|
+
Attributes
|
|
50
|
+
----------
|
|
51
|
+
file_path : str, optional
|
|
52
|
+
The file path of the input data. If provided, the data is read from
|
|
53
|
+
the file. Defaults to None.
|
|
54
|
+
data : array_like of shape (n_samples, n_features), optional
|
|
55
|
+
The input data as a numpy array. If provided, the file_path argument
|
|
56
|
+
is ignored. Defaults to None.
|
|
57
|
+
custom_min : float or None, optional
|
|
58
|
+
The minimum value to use for normalization. If not provided,
|
|
59
|
+
the minimum value of the input data is used. Defaults to None.
|
|
60
|
+
custom_max : float or None, optional
|
|
61
|
+
The maximum value to use for normalization. If not provided,
|
|
62
|
+
the maximum value of the input data is used. Defaults to None.
|
|
63
|
+
normed_data : np.ndarray
|
|
64
|
+
The normalized input data as a numpy array.
|
|
65
|
+
c_total : np.ndarray
|
|
66
|
+
The sum of columns of the normalized input data.
|
|
67
|
+
min : float
|
|
68
|
+
The minimum value of the input data.
|
|
69
|
+
max : float
|
|
70
|
+
The maximum value of the input data.
|
|
71
|
+
|
|
72
|
+
Methods
|
|
73
|
+
-------
|
|
74
|
+
get_min_max()
|
|
75
|
+
Returns the minimum and maximum values of the input data.
|
|
76
|
+
get_v2_norm()
|
|
77
|
+
Returns the v2 normalized data.
|
|
78
|
+
get_v3_norm()
|
|
79
|
+
Returns the v3 normalized data.
|
|
80
|
+
get_c_total()
|
|
81
|
+
Returns the c_total values.
|
|
82
|
+
"""
|
|
83
|
+
def __init__(self, file_path=None, data=None, custom_min=None, custom_max=None, custom_avg=None):
|
|
84
|
+
"""Initialize the Normalizer class.
|
|
85
|
+
|
|
86
|
+
Parameters
|
|
87
|
+
----------
|
|
88
|
+
file_path : str, optional
|
|
89
|
+
The file path of the input data. If provided, the data is read from
|
|
90
|
+
the file. Defaults to None.
|
|
91
|
+
data : array_like of shape (n_samples, n_features), optional
|
|
92
|
+
The input data as a numpy array. If provided, the file_path argument
|
|
93
|
+
is ignored. Defaults to None.
|
|
94
|
+
custom_min : {float, None}, optional
|
|
95
|
+
The minimum value to use for normalization. If not provided,
|
|
96
|
+
the minimum value of the input data is used. Defaults to None.
|
|
97
|
+
custom_max : {float, None}, optional
|
|
98
|
+
The maximum value to use for normalization. If not provided,
|
|
99
|
+
the maximum value of the input data is used. Defaults to None.
|
|
100
|
+
custom_avg : {float, None}, optional
|
|
101
|
+
The average value to use for normalization. If not provided,
|
|
102
|
+
the average value of the input data is used. Defaults to None.
|
|
103
|
+
"""
|
|
104
|
+
if file_path:
|
|
105
|
+
self.file_path = file_path
|
|
106
|
+
self.data = np.genfromtxt(self.file_path)
|
|
107
|
+
elif data is not None:
|
|
108
|
+
self.data = data
|
|
109
|
+
if custom_min and custom_max:
|
|
110
|
+
self.min = custom_min
|
|
111
|
+
self.max = custom_max
|
|
112
|
+
else:
|
|
113
|
+
self.min = np.min(self.data)
|
|
114
|
+
self.max = np.max(self.data)
|
|
115
|
+
|
|
116
|
+
self.v3_norm = (self.data - self.min) / (self.max - self.min)
|
|
117
|
+
if custom_avg is not None:
|
|
118
|
+
self.avg = custom_avg
|
|
119
|
+
else:
|
|
120
|
+
self.avg = np.mean(self.v3_norm, axis=0)
|
|
121
|
+
self.v2_norm = 1 - np.abs(self.v3_norm - self.avg)
|
|
122
|
+
self.c_total = np.sum(1 - np.abs(self.v3_norm - np.mean(self.v3_norm, axis=0)), axis=0)
|
|
123
|
+
|
|
124
|
+
def get_min_max(self):
|
|
125
|
+
"""Returns the minimum and maximum values of the input data."""
|
|
126
|
+
return self.min, self.max, self.avg
|
|
127
|
+
|
|
128
|
+
def get_v2_norm(self):
|
|
129
|
+
"""Returns the 'v2' normalized data."""
|
|
130
|
+
return self.v2_norm
|
|
131
|
+
|
|
132
|
+
def get_v3_norm(self):
|
|
133
|
+
"""Returns the 'v3' normalized data."""
|
|
134
|
+
return self.v3_norm
|
|
135
|
+
|
|
136
|
+
def get_c_total(self):
|
|
137
|
+
"""Returns the 'c_total' values."""
|
|
138
|
+
return self.c_total
|
|
139
|
+
|
|
140
|
+
def read_cpptraj(break_line=None, norm_type=None, min=None, max=None, avg=None, normalize=False):
|
|
141
|
+
"""Read multiple AMBER CRD files to convert to numpy ndarray formatting and normalize the data.
|
|
142
|
+
|
|
143
|
+
Parameters
|
|
144
|
+
----------
|
|
145
|
+
break_line : int
|
|
146
|
+
The number of columns per line of the input file. (have to n-1 because ignore first line)
|
|
147
|
+
norm_type : str
|
|
148
|
+
The type of normalization to use. Can be "v2" or "v3".
|
|
149
|
+
min : float or None, optional
|
|
150
|
+
The minimum value to use for normalization. If not provided,
|
|
151
|
+
the minimum value of the input data is used. Defaults to None.
|
|
152
|
+
max : float or None, optional
|
|
153
|
+
The maximum value to use for normalization. If not provided,
|
|
154
|
+
the maximum value of the input data is used. Defaults to None.
|
|
155
|
+
avg : float or None, optional
|
|
156
|
+
The average value to use for normalization. If not provided,
|
|
157
|
+
the average value of the input data is used. Defaults to None.
|
|
158
|
+
normalize : bool, optional
|
|
159
|
+
Whether to normalize the input data. If True, the data is
|
|
160
|
+
normalized to the range [0, 1]. Defaults to False.
|
|
161
|
+
|
|
162
|
+
Returns
|
|
163
|
+
-------
|
|
164
|
+
np.ndarray
|
|
165
|
+
The concatenated input data as a numpy array.
|
|
166
|
+
"""
|
|
167
|
+
input_files = sorted(glob.glob("clusttraj.c*"), key=lambda x: int(re.findall("\d+", x)[0]))
|
|
168
|
+
break_line = break_line
|
|
169
|
+
frames_list = []
|
|
170
|
+
count_frames = []
|
|
171
|
+
for file in input_files:
|
|
172
|
+
with open(file, 'r') as infile:
|
|
173
|
+
lines = [line.rstrip() for line in infile][1:]
|
|
174
|
+
sep_lines = [[line[i:i+8] for i in range(0, len(line), 8)] for line in lines]
|
|
175
|
+
chunks = [sep_lines[i:i+break_line] for i in range(0, len(sep_lines), break_line)]
|
|
176
|
+
str_frames = [list(chain.from_iterable(chunk)) for chunk in chunks]
|
|
177
|
+
str_frames = [' '.join(frame) for frame in str_frames]
|
|
178
|
+
frames = np.array([np.fromstring(frame, dtype='float32', sep=' ') for frame in str_frames])
|
|
179
|
+
if normalize:
|
|
180
|
+
norm = Normalizer(data=frames, custom_min=min, custom_max=max, custom_avg=avg)
|
|
181
|
+
if norm_type == "v2":
|
|
182
|
+
normed_frame = norm.get_v2_norm()
|
|
183
|
+
elif norm_type == "v3":
|
|
184
|
+
normed_frame = norm.get_v3_norm()
|
|
185
|
+
np.savetxt(f"normed_{file}", normed_frame)
|
|
186
|
+
else:
|
|
187
|
+
frames_list.append(frames)
|
|
188
|
+
count_frames.append(len(frames))
|
|
189
|
+
if not normalize:
|
|
190
|
+
data = np.concatenate(frames_list, axis=0)
|
|
191
|
+
return data
|
|
192
|
+
|
|
193
|
+
def normalize_file(file, break_line=None, norm_type=None):
|
|
194
|
+
"""Normalize a single file and output the normalized data to a new file.
|
|
195
|
+
|
|
196
|
+
Parameters
|
|
197
|
+
----------
|
|
198
|
+
file : str
|
|
199
|
+
The file path of the input data.
|
|
200
|
+
output : str
|
|
201
|
+
The file path of the output data.
|
|
202
|
+
break_line : int
|
|
203
|
+
The number of columns per line of the input file. (have to n-1 because ignore first line)
|
|
204
|
+
norm_type : str
|
|
205
|
+
The type of normalization to use. Can be "v2" or "v3".
|
|
206
|
+
min : float or None, optional
|
|
207
|
+
The minimum value to use for normalization. If not provided,
|
|
208
|
+
the minimum value of the input data is used. Defaults to None.
|
|
209
|
+
max : float or None, optional
|
|
210
|
+
The maximum value to use for normalization. If not provided,
|
|
211
|
+
the maximum value of the input data is used. Defaults to None.
|
|
212
|
+
avg : float or None, optional
|
|
213
|
+
The average value to use for normalization. If not provided,
|
|
214
|
+
the average value of the input data is used. Defaults to None.
|
|
215
|
+
|
|
216
|
+
Returns
|
|
217
|
+
-------
|
|
218
|
+
tuple
|
|
219
|
+
The minimum, maximum, and average values of the input data.
|
|
220
|
+
"""
|
|
221
|
+
if file is not isinstance(file, str):
|
|
222
|
+
frames = file
|
|
223
|
+
if break_line:
|
|
224
|
+
break_line = break_line
|
|
225
|
+
with open(file, 'r') as infile:
|
|
226
|
+
lines = [line.rstrip() for line in infile][1:]
|
|
227
|
+
sep_lines = [[line[i:i+8] for i in range(0, len(line), 8)] for line in lines]
|
|
228
|
+
chunks = [sep_lines[i:i+break_line] for i in range(0, len(sep_lines), break_line)]
|
|
229
|
+
str_frames = [list(chain.from_iterable(chunk)) for chunk in chunks]
|
|
230
|
+
str_frames = [' '.join(frame) for frame in str_frames]
|
|
231
|
+
frames = np.array([np.fromstring(frame, dtype='float32', sep=' ') for frame in str_frames])
|
|
232
|
+
if norm_type == "v2":
|
|
233
|
+
norm = Normalizer(data=frames)
|
|
234
|
+
normed_frame = norm.get_v2_norm()
|
|
235
|
+
elif norm_type == "v3":
|
|
236
|
+
norm = Normalizer(data=frames)
|
|
237
|
+
normed_frame = norm.get_v3_norm()
|
|
238
|
+
min, max, avg = norm.get_min_max()
|
|
239
|
+
return normed_frame, min, max, avg
|
|
File without changes
|
|
@@ -0,0 +1,132 @@
|
|
|
1
|
+
"This script aims to find the representative frame for each method below."""
|
|
2
|
+
from src.tools.esim_modules import calc_medoid
|
|
3
|
+
from src.modules.PRIME.sim_calc import _trim_outliers
|
|
4
|
+
import numpy as np
|
|
5
|
+
import json
|
|
6
|
+
import re
|
|
7
|
+
|
|
8
|
+
def calculate_max_key(dict):
|
|
9
|
+
"""Find the key with the max value"""
|
|
10
|
+
max_val = float('-inf')
|
|
11
|
+
max_key = None
|
|
12
|
+
|
|
13
|
+
for k, v in dict.items():
|
|
14
|
+
for num in v:
|
|
15
|
+
if num > max_val:
|
|
16
|
+
max_val = num
|
|
17
|
+
max_key = k
|
|
18
|
+
|
|
19
|
+
max_key = int(re.findall(r'\d+', max_key)[0])
|
|
20
|
+
return max_key
|
|
21
|
+
|
|
22
|
+
def gen_all_methods_max(sim_folder='nw', norm_folder='v3_norm', weighted_by_frames=True,
|
|
23
|
+
trim_frac=0.1, n_ary='RR', weight='nw', output_name='rep'):
|
|
24
|
+
"""Generate the representative frame for each method.
|
|
25
|
+
|
|
26
|
+
Args:
|
|
27
|
+
weighted_by_frames (bool, optional): Similarity is weighted by frames. Defaults to True.
|
|
28
|
+
trim_frac (float, optional): . Defaults to 0.1.
|
|
29
|
+
n_ary (str, optional): _description_. Defaults to "RR".
|
|
30
|
+
weight (str, optional): _description_. Defaults to 'nw'.
|
|
31
|
+
output_name (str, optional): _description_. Defaults to "rep".
|
|
32
|
+
"""
|
|
33
|
+
if weighted_by_frames is True:
|
|
34
|
+
w = "w_"
|
|
35
|
+
elif weighted_by_frames is False:
|
|
36
|
+
w = ""
|
|
37
|
+
if trim_frac:
|
|
38
|
+
t = f"_t{int(float(trim_frac) * 100)}"
|
|
39
|
+
elif not trim_frac:
|
|
40
|
+
t= ""
|
|
41
|
+
with open(f"{sim_folder}/{w}{output_name}_{n_ary}{t}.txt","w") as output:
|
|
42
|
+
output.write("# Frame number with max values by method: medoid_all, medoid_c0, medoid_c0(trimmed), pairwise, union, medoid, outlier\n")
|
|
43
|
+
|
|
44
|
+
# medoid_all
|
|
45
|
+
c_all = np.load(f"{norm_folder}/normed_data.npy")
|
|
46
|
+
output.write(f"{calc_medoid(c_all, n_ary=n_ary, weight=weight)}, ")
|
|
47
|
+
|
|
48
|
+
# medoid_c0 (untrimmed)
|
|
49
|
+
c0 = np.load(f"{norm_folder}/normed_clusttraj.c0.npy")
|
|
50
|
+
output.write(f"{calc_medoid(c0, n_ary=n_ary, weight=weight)}, ")
|
|
51
|
+
|
|
52
|
+
# medoid_c0 (trimmed)
|
|
53
|
+
if not trim_frac:
|
|
54
|
+
output.write(f"{calc_medoid(c0, n_ary=n_ary, weight=weight)}, ")
|
|
55
|
+
elif trim_frac:
|
|
56
|
+
trim_c0 = _trim_outliers(c0, trim_frac=trim_frac, n_ary=n_ary, weight=weight, removal='delete')
|
|
57
|
+
index = calc_medoid(trim_c0)
|
|
58
|
+
search = trim_c0[index]
|
|
59
|
+
new_index = np.where((c0 == search).all(axis=1))[0]
|
|
60
|
+
output.write(f"{new_index[0]}, ")
|
|
61
|
+
|
|
62
|
+
# pairwise
|
|
63
|
+
with open(f"{sim_folder}/{w}pairwise_{n_ary}{t}.txt", "r") as file:
|
|
64
|
+
pairwise = json.load(file)
|
|
65
|
+
output.write(f"{calculate_max_key(pairwise)}, ")
|
|
66
|
+
|
|
67
|
+
# union
|
|
68
|
+
with open(f"{sim_folder}/{w}union_{n_ary}{t}.txt", "r") as file:
|
|
69
|
+
union = json.load(file)
|
|
70
|
+
output.write(f"{calculate_max_key(union)}, ")
|
|
71
|
+
|
|
72
|
+
# medoid
|
|
73
|
+
with open(f"{sim_folder}/{w}medoid_{n_ary}{t}.txt", "r") as file:
|
|
74
|
+
medoid = json.load(file)
|
|
75
|
+
output.write(f"{calculate_max_key(medoid)}, ")
|
|
76
|
+
|
|
77
|
+
# outlier
|
|
78
|
+
with open(f"{sim_folder}/{w}outlier_{n_ary}{t}.txt", "r") as file:
|
|
79
|
+
outlier = json.load(file)
|
|
80
|
+
output.write(f"{calculate_max_key(outlier)}")
|
|
81
|
+
|
|
82
|
+
def gen_one_method_max(method, sim_folder='nw', norm_folder='v3_norm', weighted_by_frames=True,
|
|
83
|
+
trim_frac=0.1, n_ary='RR', weight='nw', output_name='rep'):
|
|
84
|
+
"""Generate the representative frame for each method.
|
|
85
|
+
|
|
86
|
+
Args:
|
|
87
|
+
weighted_by_frames (bool, optional): Similarity is weighted by frames. Defaults to True.
|
|
88
|
+
trim_frac (float, optional): . Defaults to 0.1.
|
|
89
|
+
n_ary (str, optional): _description_. Defaults to "RR".
|
|
90
|
+
weight (str, optional): _description_. Defaults to 'nw'.
|
|
91
|
+
output_name (str, optional): _description_. Defaults to "rep".
|
|
92
|
+
"""
|
|
93
|
+
if method not in ['medoid_all', 'medoid_c0', 'medoid_c0(trimmed)', 'pairwise', 'union',
|
|
94
|
+
'medoid', 'outlier']:
|
|
95
|
+
raise ValueError("Invalid method. Choose from 'medoid_all', 'medoid_c0', \
|
|
96
|
+
'medoid_c0(trimmed)', 'pairwise', 'union', 'medoid', 'outlier'")
|
|
97
|
+
if weighted_by_frames is True:
|
|
98
|
+
w = "w_"
|
|
99
|
+
elif weighted_by_frames is False:
|
|
100
|
+
w = ""
|
|
101
|
+
if trim_frac:
|
|
102
|
+
t = f"_t{int(float(trim_frac) * 100)}"
|
|
103
|
+
elif not trim_frac:
|
|
104
|
+
t= ""
|
|
105
|
+
|
|
106
|
+
with open(f"{sim_folder}/{w}{output_name}_{n_ary}{t}_{method}.txt","w") as output:
|
|
107
|
+
output.write(f"# Frame number with max values by method: {method}\n")
|
|
108
|
+
|
|
109
|
+
if method == 'medoid_all':
|
|
110
|
+
c_all = np.load(f"{norm_folder}/normed_data.npy")
|
|
111
|
+
output.write(f"{calc_medoid(c_all, n_ary=n_ary, weight=weight)}")
|
|
112
|
+
|
|
113
|
+
elif method == 'medoid_c0':
|
|
114
|
+
c0 = np.load(f"{norm_folder}/normed_clusttraj.c0")
|
|
115
|
+
output.write(f"{calc_medoid(c0, n_ary=n_ary, weight=weight)}")
|
|
116
|
+
|
|
117
|
+
elif method == 'medoid_c0(trimmed)':
|
|
118
|
+
if not trim_frac:
|
|
119
|
+
raise ValueError("No trimmed frac available")
|
|
120
|
+
elif trim_frac:
|
|
121
|
+
trim_c0 = _trim_outliers(c0, trim_frac=trim_frac, n_ary=n_ary, weight=weight, removal='delete')
|
|
122
|
+
index = calc_medoid(trim_c0)
|
|
123
|
+
search = trim_c0[index]
|
|
124
|
+
new_index = np.where((c0 == search).all(axis=1))[0]
|
|
125
|
+
output.write(f"{new_index[0]}")
|
|
126
|
+
|
|
127
|
+
prime_methods = ['pairwise', 'union', 'medoid', 'outlier']
|
|
128
|
+
|
|
129
|
+
if method in prime_methods:
|
|
130
|
+
with open(f"{sim_folder}/{w}{method}_{n_ary}{t}.txt", "r") as file:
|
|
131
|
+
data = json.load(file)
|
|
132
|
+
output.write(f"{calculate_max_key(data)}")
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
"""Call the `gen_one_method_max` to generate the method max for a single method
|
|
2
|
+
or `gen_all_methods_max` to generate the method max for all methods.
|
|
3
|
+
|
|
4
|
+
Example usage:
|
|
5
|
+
----------------
|
|
6
|
+
No trim, RR index
|
|
7
|
+
>>> python scripts/rep_frames.py
|
|
8
|
+
10% trim, RR index
|
|
9
|
+
>>> python scripts/rep_frames.py -t 0.1
|
|
10
|
+
20% trim, SM index
|
|
11
|
+
>>> python scripts/rep_frames.py -t 0.2 -i SM
|
|
12
|
+
"""
|
|
13
|
+
import sys
|
|
14
|
+
sys.path.insert(0, '../../')
|
|
15
|
+
from src.modules.PRIME.rep_frames import gen_one_method_max, gen_all_methods_max
|
|
16
|
+
import argparse
|
|
17
|
+
|
|
18
|
+
parser = argparse.ArgumentParser(description='Generate method max with optional trim and n_ary')
|
|
19
|
+
parser.add_argument('-m', '--method', type=str, help='method to use')
|
|
20
|
+
parser.add_argument('-s', '--sim_folder', type=str, help='folder to access')
|
|
21
|
+
parser.add_argument('-t', '--trim_frac', type=float, default=None,
|
|
22
|
+
help='Trim parameter for gen_method_max method')
|
|
23
|
+
parser.add_argument('-i', '--index', type=str, default='RR',
|
|
24
|
+
help='n_ary parameter for gen_method_max method')
|
|
25
|
+
parser.add_argument('-d', '--norm_folder', type=str, help='norm_folder to access')
|
|
26
|
+
|
|
27
|
+
args = parser.parse_args()
|
|
28
|
+
if args.method:
|
|
29
|
+
gen_one_method_max(method=args.method, sim_folder=args.sim_folder, norm_folder=args.norm_folder,
|
|
30
|
+
trim_frac=args.trim_frac, n_ary=args.index)
|
|
31
|
+
else:
|
|
32
|
+
gen_all_methods_max(sim_folder=args.sim_folder, norm_folder=args.norm_folder,
|
|
33
|
+
trim_frac=args.trim_frac, n_ary=args.index)
|