heavyedge-dataset 0.1.0__tar.gz → 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- heavyedge_dataset-0.2.0/PKG-INFO +90 -0
- heavyedge_dataset-0.2.0/README.md +52 -0
- {heavyedge_dataset-0.1.0 → heavyedge_dataset-0.2.0}/pyproject.toml +1 -1
- heavyedge_dataset-0.1.0/src/heavyedge_dataset/datasets.py → heavyedge_dataset-0.2.0/src/heavyedge_dataset/__init__.py +102 -123
- heavyedge_dataset-0.2.0/src/heavyedge_dataset.egg-info/PKG-INFO +90 -0
- {heavyedge_dataset-0.1.0 → heavyedge_dataset-0.2.0}/src/heavyedge_dataset.egg-info/SOURCES.txt +0 -2
- heavyedge_dataset-0.1.0/PKG-INFO +0 -39
- heavyedge_dataset-0.1.0/README.md +0 -1
- heavyedge_dataset-0.1.0/src/heavyedge_dataset/__init__.py +0 -15
- heavyedge_dataset-0.1.0/src/heavyedge_dataset/landmarks.py +0 -41
- heavyedge_dataset-0.1.0/src/heavyedge_dataset.egg-info/PKG-INFO +0 -39
- {heavyedge_dataset-0.1.0 → heavyedge_dataset-0.2.0}/LICENSE +0 -0
- {heavyedge_dataset-0.1.0 → heavyedge_dataset-0.2.0}/setup.cfg +0 -0
- {heavyedge_dataset-0.1.0 → heavyedge_dataset-0.2.0}/src/heavyedge_dataset.egg-info/dependency_links.txt +0 -0
- {heavyedge_dataset-0.1.0 → heavyedge_dataset-0.2.0}/src/heavyedge_dataset.egg-info/requires.txt +0 -0
- {heavyedge_dataset-0.1.0 → heavyedge_dataset-0.2.0}/src/heavyedge_dataset.egg-info/top_level.txt +0 -0
@@ -0,0 +1,90 @@
|
|
1
|
+
Metadata-Version: 2.4
|
2
|
+
Name: heavyedge-dataset
|
3
|
+
Version: 0.2.0
|
4
|
+
Summary: PyTorch-compatible edge profile dataset API
|
5
|
+
Author-email: Jisoo Song <jeesoo9595@snu.ac.kr>
|
6
|
+
License-Expression: MIT
|
7
|
+
Project-URL: homepage, https://pypi.python.org/pypi/heavyedge-dataset/
|
8
|
+
Project-URL: source, https://github.com/heavyedge/heavyedge-dataset
|
9
|
+
Project-URL: documentation, https://heavyedge-dataset.readthedocs.io
|
10
|
+
Classifier: Development Status :: 5 - Production/Stable
|
11
|
+
Classifier: Intended Audience :: Science/Research
|
12
|
+
Classifier: Programming Language :: Python
|
13
|
+
Classifier: Programming Language :: Python :: 3
|
14
|
+
Classifier: Programming Language :: Python :: 3.10
|
15
|
+
Classifier: Programming Language :: Python :: 3.11
|
16
|
+
Classifier: Programming Language :: Python :: 3.12
|
17
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
18
|
+
Classifier: Topic :: Scientific/Engineering
|
19
|
+
Classifier: Operating System :: OS Independent
|
20
|
+
Requires-Python: >=3.10
|
21
|
+
Description-Content-Type: text/markdown
|
22
|
+
License-File: LICENSE
|
23
|
+
Requires-Dist: heavyedge>=1.1.2
|
24
|
+
Requires-Dist: torch
|
25
|
+
Provides-Extra: test
|
26
|
+
Requires-Dist: pytest; extra == "test"
|
27
|
+
Provides-Extra: doc
|
28
|
+
Requires-Dist: sphinx; extra == "doc"
|
29
|
+
Requires-Dist: numpydoc; extra == "doc"
|
30
|
+
Requires-Dist: pydata_sphinx_theme; extra == "doc"
|
31
|
+
Requires-Dist: matplotlib; extra == "doc"
|
32
|
+
Provides-Extra: dev
|
33
|
+
Requires-Dist: flake8; extra == "dev"
|
34
|
+
Requires-Dist: black; extra == "dev"
|
35
|
+
Requires-Dist: isort; extra == "dev"
|
36
|
+
Requires-Dist: heavyedge-dataset[doc,test]; extra == "dev"
|
37
|
+
Dynamic: license-file
|
38
|
+
|
39
|
+
# HeavyEdge-Dataset
|
40
|
+
|
41
|
+
[](https://pypi.python.org/pypi/heavyedge-dataset/)
|
42
|
+
[](https://pypi.python.org/pypi/heavyedge-dataset/)
|
43
|
+
[](https://github.com/heavyedge/heavyedge-dataset/blob/master/LICENSE)
|
44
|
+
[](https://github.com/heavyedge/heavyedge-dataset/actions/workflows/ci.yml)
|
45
|
+
[](https://github.com/heavyedge/heavyedge-dataset/actions/workflows/cd.yml)
|
46
|
+
[](https://heavyedge-dataset.readthedocs.io/en/latest/?badge=latest)
|
47
|
+
|
48
|
+
Package to load edge profile data as PyTorch dataset.
|
49
|
+
|
50
|
+
## Usage
|
51
|
+
|
52
|
+
HeavyEdge-Dataset provides custom dataset classes which wraps profile data file.
|
53
|
+
|
54
|
+
A simple use case to load a list of profiles as two-dimensional coordinates:
|
55
|
+
|
56
|
+
```python
|
57
|
+
from heavyedge import get_sample_path, ProfileData
|
58
|
+
from heavyedge_dataset import ProfileDataset
|
59
|
+
with ProfileData(get_sample_path("Prep-Type2.h5")) as file: # Profile data file object
|
60
|
+
data = ProfileDataset(file, 2)[:]
|
61
|
+
```
|
62
|
+
|
63
|
+
Refer to the package documentation for more information.
|
64
|
+
|
65
|
+
## Documentation
|
66
|
+
|
67
|
+
The manual can be found online:
|
68
|
+
|
69
|
+
> https://heavyedge-dataset.readthedocs.io
|
70
|
+
|
71
|
+
If you want to build the document yourself, get the source code and install with `[doc]` dependency.
|
72
|
+
Then, go to `doc` directory and build the document:
|
73
|
+
|
74
|
+
```
|
75
|
+
$ pip install .[doc]
|
76
|
+
$ cd doc
|
77
|
+
$ make html
|
78
|
+
```
|
79
|
+
|
80
|
+
Document will be generated in `build/html` directory. Open `index.html` to see the central page.
|
81
|
+
|
82
|
+
## Developing
|
83
|
+
|
84
|
+
### Installation
|
85
|
+
|
86
|
+
For development features, you must install the package by `pip install -e .[dev]`.
|
87
|
+
|
88
|
+
### Testing
|
89
|
+
|
90
|
+
Run `pytest` command to perform unit test.
|
@@ -0,0 +1,52 @@
|
|
1
|
+
# HeavyEdge-Dataset
|
2
|
+
|
3
|
+
[](https://pypi.python.org/pypi/heavyedge-dataset/)
|
4
|
+
[](https://pypi.python.org/pypi/heavyedge-dataset/)
|
5
|
+
[](https://github.com/heavyedge/heavyedge-dataset/blob/master/LICENSE)
|
6
|
+
[](https://github.com/heavyedge/heavyedge-dataset/actions/workflows/ci.yml)
|
7
|
+
[](https://github.com/heavyedge/heavyedge-dataset/actions/workflows/cd.yml)
|
8
|
+
[](https://heavyedge-dataset.readthedocs.io/en/latest/?badge=latest)
|
9
|
+
|
10
|
+
Package to load edge profile data as PyTorch dataset.
|
11
|
+
|
12
|
+
## Usage
|
13
|
+
|
14
|
+
HeavyEdge-Dataset provides custom dataset classes which wraps profile data file.
|
15
|
+
|
16
|
+
A simple use case to load a list of profiles as two-dimensional coordinates:
|
17
|
+
|
18
|
+
```python
|
19
|
+
from heavyedge import get_sample_path, ProfileData
|
20
|
+
from heavyedge_dataset import ProfileDataset
|
21
|
+
with ProfileData(get_sample_path("Prep-Type2.h5")) as file: # Profile data file object
|
22
|
+
data = ProfileDataset(file, 2)[:]
|
23
|
+
```
|
24
|
+
|
25
|
+
Refer to the package documentation for more information.
|
26
|
+
|
27
|
+
## Documentation
|
28
|
+
|
29
|
+
The manual can be found online:
|
30
|
+
|
31
|
+
> https://heavyedge-dataset.readthedocs.io
|
32
|
+
|
33
|
+
If you want to build the document yourself, get the source code and install with `[doc]` dependency.
|
34
|
+
Then, go to `doc` directory and build the document:
|
35
|
+
|
36
|
+
```
|
37
|
+
$ pip install .[doc]
|
38
|
+
$ cd doc
|
39
|
+
$ make html
|
40
|
+
```
|
41
|
+
|
42
|
+
Document will be generated in `build/html` directory. Open `index.html` to see the central page.
|
43
|
+
|
44
|
+
## Developing
|
45
|
+
|
46
|
+
### Installation
|
47
|
+
|
48
|
+
For development features, you must install the package by `pip install -e .[dev]`.
|
49
|
+
|
50
|
+
### Testing
|
51
|
+
|
52
|
+
Run `pytest` command to perform unit test.
|
@@ -1,8 +1,8 @@
|
|
1
|
-
"""
|
2
|
-
|
3
|
-
|
1
|
+
"""Custom dataset classes for edge profiles.
|
2
|
+
|
3
|
+
Refer to `PyTorch tutorial <tutorial>`_ for information about custom PyTorch dataset.
|
4
4
|
|
5
|
-
|
5
|
+
.. _tutorial: https://docs.pytorch.org/tutorials/beginner/data_loading_tutorial.html
|
6
6
|
"""
|
7
7
|
|
8
8
|
import abc
|
@@ -13,13 +13,10 @@ import numpy as np
|
|
13
13
|
from heavyedge.api import landmarks_type3
|
14
14
|
from torch.utils.data import Dataset
|
15
15
|
|
16
|
-
from .landmarks import math_landmarks_1d, pseudo_landmarks_1d, pseudo_landmarks_2d
|
17
|
-
|
18
16
|
__all__ = [
|
19
17
|
"ProfileDataset",
|
20
|
-
"
|
21
|
-
"
|
22
|
-
"MathLm2dDataset",
|
18
|
+
"PseudoLandmarkDataset",
|
19
|
+
"MathematicalLandmarkDataset",
|
23
20
|
]
|
24
21
|
|
25
22
|
|
@@ -36,23 +33,10 @@ class ProfileDatasetBase(abc.ABC):
|
|
36
33
|
heavyedge.ProfileData
|
37
34
|
"""
|
38
35
|
|
39
|
-
@abc.abstractmethod
|
40
|
-
def default_transform(self, profiles, lengths):
|
41
|
-
"""Default transform by the dataset class.
|
42
|
-
|
43
|
-
Parameters
|
44
|
-
----------
|
45
|
-
profiles : (N, M) array
|
46
|
-
Profile data.
|
47
|
-
lengths : (N,) array
|
48
|
-
Length of each profile.
|
49
|
-
"""
|
50
|
-
pass
|
51
|
-
|
52
36
|
@property
|
53
37
|
@abc.abstractmethod
|
54
38
|
def transform(self):
|
55
|
-
"""Optional transformation
|
39
|
+
"""Optional transformation to be applied on samples.
|
56
40
|
|
57
41
|
Returns
|
58
42
|
-------
|
@@ -65,37 +49,49 @@ class ProfileDatasetBase(abc.ABC):
|
|
65
49
|
def __getitem__(self, idx):
|
66
50
|
if isinstance(idx, numbers.Integral):
|
67
51
|
Y, L, _ = self.file[idx]
|
68
|
-
|
69
|
-
if self.transform:
|
70
|
-
ret = self.transform(ret)
|
71
|
-
ret = ret[0]
|
52
|
+
Ys, Ls = [Y], [L]
|
72
53
|
else:
|
73
|
-
|
74
|
-
|
54
|
+
# Support multi-indexing
|
55
|
+
idxs = idx
|
56
|
+
needs_sort = isinstance(idx, (Sequence, np.ndarray))
|
57
|
+
if needs_sort:
|
58
|
+
# idxs must be sorted for h5py
|
59
|
+
idxs = np.array(idxs)
|
60
|
+
sort_idx = np.argsort(idxs)
|
61
|
+
idxs = idxs[sort_idx]
|
62
|
+
Ys, Ls, _ = self.file[idxs]
|
63
|
+
if needs_sort:
|
64
|
+
reverse_idx = np.argsort(sort_idx)
|
65
|
+
Ys = Ys[reverse_idx]
|
66
|
+
Ls = Ls[reverse_idx]
|
75
67
|
|
76
|
-
def __getitems__(self, idxs):
|
77
|
-
# PyTorch API
|
78
|
-
needs_sort = isinstance(idxs, (Sequence, np.ndarray))
|
79
|
-
if needs_sort:
|
80
|
-
# idxs must be sorted for h5py
|
81
|
-
idxs = np.array(idxs)
|
82
|
-
sort_idx = np.argsort(idxs)
|
83
|
-
idxs = idxs[sort_idx]
|
84
|
-
else:
|
85
|
-
pass
|
86
|
-
Ys, Ls, _ = self.file[idxs]
|
87
|
-
if needs_sort:
|
88
|
-
reverse_idx = np.argsort(sort_idx)
|
89
|
-
Ys = Ys[reverse_idx]
|
90
|
-
Ls = Ls[reverse_idx]
|
91
68
|
ret = self.default_transform(Ys, Ls)
|
92
69
|
if self.transform:
|
93
70
|
ret = self.transform(ret)
|
94
71
|
return ret
|
95
72
|
|
73
|
+
def __getitems__(self, idxs):
|
74
|
+
# PyTorch API
|
75
|
+
return self.__getitem__(idxs)
|
76
|
+
|
77
|
+
@abc.abstractmethod
|
78
|
+
def default_transform(self, profiles, lengths):
|
79
|
+
"""Default data transformation.
|
80
|
+
|
81
|
+
Subclass must implement this method to transform profile data into target data.
|
82
|
+
|
83
|
+
Parameters
|
84
|
+
----------
|
85
|
+
profiles : (N, M) array
|
86
|
+
Profile data.
|
87
|
+
lengths : (N,) array
|
88
|
+
Length of each profile in *profiles*.
|
89
|
+
"""
|
90
|
+
pass
|
91
|
+
|
96
92
|
|
97
93
|
class ProfileDataset(ProfileDatasetBase, Dataset):
|
98
|
-
"""Full profile dataset.
|
94
|
+
"""Full profile dataset in 1-D or 2-D.
|
99
95
|
|
100
96
|
Parameters
|
101
97
|
----------
|
@@ -105,7 +101,7 @@ class ProfileDataset(ProfileDatasetBase, Dataset):
|
|
105
101
|
Profile data dimension.
|
106
102
|
1 means only y coordinates, and 2 means both x and y coordinates.
|
107
103
|
transform : callable, optional
|
108
|
-
Optional
|
104
|
+
Optional transformation to be applied on samples.
|
109
105
|
|
110
106
|
Examples
|
111
107
|
--------
|
@@ -129,7 +125,20 @@ class ProfileDataset(ProfileDatasetBase, Dataset):
|
|
129
125
|
def file(self):
|
130
126
|
return self._file
|
131
127
|
|
128
|
+
@property
|
129
|
+
def transform(self):
|
130
|
+
return self._transform
|
131
|
+
|
132
132
|
def default_transform(self, profiles, lengths):
|
133
|
+
"""Crop profiles by their contact points.
|
134
|
+
|
135
|
+
Parameters
|
136
|
+
----------
|
137
|
+
profiles : (N, M) array
|
138
|
+
Profile data.
|
139
|
+
lengths : (N,) array
|
140
|
+
Length of each profile in *profiles*.
|
141
|
+
"""
|
133
142
|
if self.m == 1:
|
134
143
|
ret = [Y[:L].reshape(1, -1) for Y, L in zip(profiles, lengths)]
|
135
144
|
elif self.m == 2:
|
@@ -138,13 +147,11 @@ class ProfileDataset(ProfileDatasetBase, Dataset):
|
|
138
147
|
raise ValueError(f"Invalid dimension: {self.m}")
|
139
148
|
return ret
|
140
149
|
|
141
|
-
@property
|
142
|
-
def transform(self):
|
143
|
-
return self._transform
|
144
150
|
|
151
|
+
class PseudoLandmarkDataset(ProfileDatasetBase, Dataset):
|
152
|
+
"""Pseudo-landmark dataset in 1-D or 2-D.
|
145
153
|
|
146
|
-
|
147
|
-
"""Pseudo-landmark dataset.
|
154
|
+
Pseudo-landmarks are points that are equidistantly sampled.
|
148
155
|
|
149
156
|
Parameters
|
150
157
|
----------
|
@@ -156,14 +163,14 @@ class PseudoLmDataset(ProfileDatasetBase, Dataset):
|
|
156
163
|
Profile data dimension.
|
157
164
|
1 means only y coordinates, and 2 means both x and y coordinates.
|
158
165
|
transform : callable, optional
|
159
|
-
Optional
|
166
|
+
Optional transformation to be applied on samples.
|
160
167
|
|
161
168
|
Examples
|
162
169
|
--------
|
163
170
|
>>> from heavyedge import get_sample_path, ProfileData
|
164
|
-
>>> from heavyedge_dataset import
|
171
|
+
>>> from heavyedge_dataset import PseudoLandmarkDataset
|
165
172
|
>>> with ProfileData(get_sample_path("Prep-Type2.h5")) as file:
|
166
|
-
... data =
|
173
|
+
... data = PseudoLandmarkDataset(file, 10, 2)[:]
|
167
174
|
>>> import matplotlib.pyplot as plt # doctest: +SKIP
|
168
175
|
... plt.plot(*data.transpose(1, 2, 0), color="gray")
|
169
176
|
"""
|
@@ -180,22 +187,39 @@ class PseudoLmDataset(ProfileDatasetBase, Dataset):
|
|
180
187
|
def file(self):
|
181
188
|
return self._file
|
182
189
|
|
190
|
+
@property
|
191
|
+
def transform(self):
|
192
|
+
return self._transform
|
193
|
+
|
183
194
|
def default_transform(self, profiles, lengths):
|
195
|
+
"""Sample pseudo-landmarks from profiles.
|
196
|
+
|
197
|
+
Parameters
|
198
|
+
----------
|
199
|
+
profiles : (N, M) array
|
200
|
+
Profile data.
|
201
|
+
lengths : (N,) array
|
202
|
+
Length of each profile in *profiles*.
|
203
|
+
"""
|
204
|
+
ret = []
|
184
205
|
if self.m == 1:
|
185
|
-
|
206
|
+
for Y, L in zip(profiles, lengths):
|
207
|
+
idxs = np.linspace(0, L - 1, self.k, dtype=int)
|
208
|
+
ret.append(Y[idxs].reshape(1, -1))
|
186
209
|
elif self.m == 2:
|
187
|
-
|
210
|
+
for Y, L in zip(profiles, lengths):
|
211
|
+
idxs = np.linspace(0, L - 1, self.k, dtype=int)
|
212
|
+
ret.append(np.stack([self.x[idxs], Y[idxs]]))
|
188
213
|
else:
|
189
214
|
raise ValueError(f"Invalid dimension: {self.m}")
|
190
|
-
return ret
|
215
|
+
return np.array(ret)
|
191
216
|
|
192
|
-
@property
|
193
|
-
def transform(self):
|
194
|
-
return self._transform
|
195
217
|
|
218
|
+
class MathematicalLandmarkDataset(ProfileDatasetBase, Dataset):
|
219
|
+
"""Mathematical landmark dataset in 1-D.
|
196
220
|
|
197
|
-
|
198
|
-
|
221
|
+
Mathematical landmarks are points which are choosed by their
|
222
|
+
mathematical properties, i.e., slope or curvature.
|
199
223
|
|
200
224
|
Parameters
|
201
225
|
----------
|
@@ -204,14 +228,14 @@ class MathLm1dDataset(ProfileDatasetBase, Dataset):
|
|
204
228
|
sigma : scalar
|
205
229
|
Standard deviation of Gaussian kernel for landmark detection.
|
206
230
|
transform : callable, optional
|
207
|
-
Optional
|
231
|
+
Optional transformation to be applied on samples.
|
208
232
|
|
209
233
|
Examples
|
210
234
|
--------
|
211
235
|
>>> from heavyedge import get_sample_path, ProfileData
|
212
|
-
>>> from heavyedge_dataset import
|
236
|
+
>>> from heavyedge_dataset import MathematicalLandmarkDataset
|
213
237
|
>>> with ProfileData(get_sample_path("Prep-Type2.h5")) as file:
|
214
|
-
... data =
|
238
|
+
... data = MathematicalLandmarkDataset(file, 32)[:]
|
215
239
|
>>> import matplotlib.pyplot as plt # doctest: +SKIP
|
216
240
|
... plt.plot(*data.transpose(1, 2, 0), color="gray")
|
217
241
|
"""
|
@@ -225,69 +249,24 @@ class MathLm1dDataset(ProfileDatasetBase, Dataset):
|
|
225
249
|
def file(self):
|
226
250
|
return self._file
|
227
251
|
|
228
|
-
def default_transform(self, profiles, lengths):
|
229
|
-
return math_landmarks_1d(profiles, lengths, self.sigma)
|
230
|
-
|
231
252
|
@property
|
232
253
|
def transform(self):
|
233
254
|
return self._transform
|
234
255
|
|
235
|
-
|
236
|
-
class MathLm2dDataset(ProfileDatasetBase, Dataset):
|
237
|
-
"""2-D mathematical landmarks dataset.
|
238
|
-
|
239
|
-
Parameters
|
240
|
-
----------
|
241
|
-
file : heavyedge.ProfileData
|
242
|
-
Open hdf5 file.
|
243
|
-
sigma : scalar
|
244
|
-
Standard deviation of Gaussian kernel for landmark detection.
|
245
|
-
transform : callable, optional
|
246
|
-
Optional transform to be applied on a sample.
|
247
|
-
|
248
|
-
Examples
|
249
|
-
--------
|
250
|
-
>>> from heavyedge import get_sample_path, ProfileData
|
251
|
-
>>> from heavyedge_dataset import MathLm2dDataset
|
252
|
-
>>> with ProfileData(get_sample_path("Prep-Type2.h5")) as file:
|
253
|
-
... lm, _ = MathLm2dDataset(file, 32)[:]
|
254
|
-
>>> import matplotlib.pyplot as plt # doctest: +SKIP
|
255
|
-
... plt.plot(*lm.transpose(1, 2, 0), color="gray")
|
256
|
-
"""
|
257
|
-
|
258
|
-
def __init__(self, file, sigma, transform=None):
|
259
|
-
self._file = file
|
260
|
-
self.sigma = sigma
|
261
|
-
self._transform = transform
|
262
|
-
|
263
|
-
self.x = file.x()
|
264
|
-
|
265
|
-
@property
|
266
|
-
def file(self):
|
267
|
-
return self._file
|
268
|
-
|
269
256
|
def default_transform(self, profiles, lengths):
|
270
|
-
|
271
|
-
|
272
|
-
|
257
|
+
"""Detect mathematical landmarks from profiles.
|
258
|
+
|
259
|
+
Parameters
|
260
|
+
----------
|
261
|
+
profiles : (N, M) array
|
262
|
+
Profile data.
|
263
|
+
lengths : (N,) array
|
264
|
+
Length of each profile in *profiles*.
|
265
|
+
"""
|
266
|
+
ret = []
|
273
267
|
for Y, L in zip(profiles, lengths):
|
274
268
|
Y = Y[:L]
|
275
269
|
indices = np.flip(landmarks_type3(Y, self.sigma))
|
276
|
-
|
277
|
-
|
278
|
-
return np.array(
|
279
|
-
|
280
|
-
def __getitem__(self, idx):
|
281
|
-
if isinstance(idx, numbers.Integral):
|
282
|
-
Y, L, _ = self.file[idx]
|
283
|
-
lm, ch = self.default_transform([Y], [L])
|
284
|
-
if self.transform:
|
285
|
-
lm, ch = self.transform(lm, ch)
|
286
|
-
lm, ch = lm[0], ch[0]
|
287
|
-
else:
|
288
|
-
lm, ch = self.__getitems__(idx)
|
289
|
-
return (lm, ch)
|
290
|
-
|
291
|
-
@property
|
292
|
-
def transform(self):
|
293
|
-
return self._transform
|
270
|
+
y = np.concat([[np.mean(Y[: indices[0]])], Y[indices]])
|
271
|
+
ret.append(y.reshape(1, -1))
|
272
|
+
return np.array(ret)
|
@@ -0,0 +1,90 @@
|
|
1
|
+
Metadata-Version: 2.4
|
2
|
+
Name: heavyedge-dataset
|
3
|
+
Version: 0.2.0
|
4
|
+
Summary: PyTorch-compatible edge profile dataset API
|
5
|
+
Author-email: Jisoo Song <jeesoo9595@snu.ac.kr>
|
6
|
+
License-Expression: MIT
|
7
|
+
Project-URL: homepage, https://pypi.python.org/pypi/heavyedge-dataset/
|
8
|
+
Project-URL: source, https://github.com/heavyedge/heavyedge-dataset
|
9
|
+
Project-URL: documentation, https://heavyedge-dataset.readthedocs.io
|
10
|
+
Classifier: Development Status :: 5 - Production/Stable
|
11
|
+
Classifier: Intended Audience :: Science/Research
|
12
|
+
Classifier: Programming Language :: Python
|
13
|
+
Classifier: Programming Language :: Python :: 3
|
14
|
+
Classifier: Programming Language :: Python :: 3.10
|
15
|
+
Classifier: Programming Language :: Python :: 3.11
|
16
|
+
Classifier: Programming Language :: Python :: 3.12
|
17
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
18
|
+
Classifier: Topic :: Scientific/Engineering
|
19
|
+
Classifier: Operating System :: OS Independent
|
20
|
+
Requires-Python: >=3.10
|
21
|
+
Description-Content-Type: text/markdown
|
22
|
+
License-File: LICENSE
|
23
|
+
Requires-Dist: heavyedge>=1.1.2
|
24
|
+
Requires-Dist: torch
|
25
|
+
Provides-Extra: test
|
26
|
+
Requires-Dist: pytest; extra == "test"
|
27
|
+
Provides-Extra: doc
|
28
|
+
Requires-Dist: sphinx; extra == "doc"
|
29
|
+
Requires-Dist: numpydoc; extra == "doc"
|
30
|
+
Requires-Dist: pydata_sphinx_theme; extra == "doc"
|
31
|
+
Requires-Dist: matplotlib; extra == "doc"
|
32
|
+
Provides-Extra: dev
|
33
|
+
Requires-Dist: flake8; extra == "dev"
|
34
|
+
Requires-Dist: black; extra == "dev"
|
35
|
+
Requires-Dist: isort; extra == "dev"
|
36
|
+
Requires-Dist: heavyedge-dataset[doc,test]; extra == "dev"
|
37
|
+
Dynamic: license-file
|
38
|
+
|
39
|
+
# HeavyEdge-Dataset
|
40
|
+
|
41
|
+
[](https://pypi.python.org/pypi/heavyedge-dataset/)
|
42
|
+
[](https://pypi.python.org/pypi/heavyedge-dataset/)
|
43
|
+
[](https://github.com/heavyedge/heavyedge-dataset/blob/master/LICENSE)
|
44
|
+
[](https://github.com/heavyedge/heavyedge-dataset/actions/workflows/ci.yml)
|
45
|
+
[](https://github.com/heavyedge/heavyedge-dataset/actions/workflows/cd.yml)
|
46
|
+
[](https://heavyedge-dataset.readthedocs.io/en/latest/?badge=latest)
|
47
|
+
|
48
|
+
Package to load edge profile data as PyTorch dataset.
|
49
|
+
|
50
|
+
## Usage
|
51
|
+
|
52
|
+
HeavyEdge-Dataset provides custom dataset classes which wraps profile data file.
|
53
|
+
|
54
|
+
A simple use case to load a list of profiles as two-dimensional coordinates:
|
55
|
+
|
56
|
+
```python
|
57
|
+
from heavyedge import get_sample_path, ProfileData
|
58
|
+
from heavyedge_dataset import ProfileDataset
|
59
|
+
with ProfileData(get_sample_path("Prep-Type2.h5")) as file: # Profile data file object
|
60
|
+
data = ProfileDataset(file, 2)[:]
|
61
|
+
```
|
62
|
+
|
63
|
+
Refer to the package documentation for more information.
|
64
|
+
|
65
|
+
## Documentation
|
66
|
+
|
67
|
+
The manual can be found online:
|
68
|
+
|
69
|
+
> https://heavyedge-dataset.readthedocs.io
|
70
|
+
|
71
|
+
If you want to build the document yourself, get the source code and install with `[doc]` dependency.
|
72
|
+
Then, go to `doc` directory and build the document:
|
73
|
+
|
74
|
+
```
|
75
|
+
$ pip install .[doc]
|
76
|
+
$ cd doc
|
77
|
+
$ make html
|
78
|
+
```
|
79
|
+
|
80
|
+
Document will be generated in `build/html` directory. Open `index.html` to see the central page.
|
81
|
+
|
82
|
+
## Developing
|
83
|
+
|
84
|
+
### Installation
|
85
|
+
|
86
|
+
For development features, you must install the package by `pip install -e .[dev]`.
|
87
|
+
|
88
|
+
### Testing
|
89
|
+
|
90
|
+
Run `pytest` command to perform unit test.
|
{heavyedge_dataset-0.1.0 → heavyedge_dataset-0.2.0}/src/heavyedge_dataset.egg-info/SOURCES.txt
RENAMED
@@ -2,8 +2,6 @@ LICENSE
|
|
2
2
|
README.md
|
3
3
|
pyproject.toml
|
4
4
|
src/heavyedge_dataset/__init__.py
|
5
|
-
src/heavyedge_dataset/datasets.py
|
6
|
-
src/heavyedge_dataset/landmarks.py
|
7
5
|
src/heavyedge_dataset.egg-info/PKG-INFO
|
8
6
|
src/heavyedge_dataset.egg-info/SOURCES.txt
|
9
7
|
src/heavyedge_dataset.egg-info/dependency_links.txt
|
heavyedge_dataset-0.1.0/PKG-INFO
DELETED
@@ -1,39 +0,0 @@
|
|
1
|
-
Metadata-Version: 2.4
|
2
|
-
Name: heavyedge-dataset
|
3
|
-
Version: 0.1.0
|
4
|
-
Summary: PyTorch-compatible edge profile dataset API
|
5
|
-
Author-email: Jisoo Song <jeesoo9595@snu.ac.kr>
|
6
|
-
License-Expression: MIT
|
7
|
-
Project-URL: homepage, https://pypi.python.org/pypi/heavyedge-dataset/
|
8
|
-
Project-URL: source, https://github.com/heavyedge/heavyedge-dataset
|
9
|
-
Project-URL: documentation, https://heavyedge-dataset.readthedocs.io
|
10
|
-
Classifier: Development Status :: 5 - Production/Stable
|
11
|
-
Classifier: Intended Audience :: Science/Research
|
12
|
-
Classifier: Programming Language :: Python
|
13
|
-
Classifier: Programming Language :: Python :: 3
|
14
|
-
Classifier: Programming Language :: Python :: 3.10
|
15
|
-
Classifier: Programming Language :: Python :: 3.11
|
16
|
-
Classifier: Programming Language :: Python :: 3.12
|
17
|
-
Classifier: Programming Language :: Python :: 3 :: Only
|
18
|
-
Classifier: Topic :: Scientific/Engineering
|
19
|
-
Classifier: Operating System :: OS Independent
|
20
|
-
Requires-Python: >=3.10
|
21
|
-
Description-Content-Type: text/markdown
|
22
|
-
License-File: LICENSE
|
23
|
-
Requires-Dist: heavyedge>=1.1.2
|
24
|
-
Requires-Dist: torch
|
25
|
-
Provides-Extra: test
|
26
|
-
Requires-Dist: pytest; extra == "test"
|
27
|
-
Provides-Extra: doc
|
28
|
-
Requires-Dist: sphinx; extra == "doc"
|
29
|
-
Requires-Dist: numpydoc; extra == "doc"
|
30
|
-
Requires-Dist: pydata_sphinx_theme; extra == "doc"
|
31
|
-
Requires-Dist: matplotlib; extra == "doc"
|
32
|
-
Provides-Extra: dev
|
33
|
-
Requires-Dist: flake8; extra == "dev"
|
34
|
-
Requires-Dist: black; extra == "dev"
|
35
|
-
Requires-Dist: isort; extra == "dev"
|
36
|
-
Requires-Dist: heavyedge-dataset[doc,test]; extra == "dev"
|
37
|
-
Dynamic: license-file
|
38
|
-
|
39
|
-
# HeavyEdge-Dataset
|
@@ -1 +0,0 @@
|
|
1
|
-
# HeavyEdge-Dataset
|
@@ -1,15 +0,0 @@
|
|
1
|
-
"""PyTorch-compatiable dataset API for edge profiles."""
|
2
|
-
|
3
|
-
__all__ = [
|
4
|
-
"ProfileDataset",
|
5
|
-
"PseudoLmDataset",
|
6
|
-
"MathLm1dDataset",
|
7
|
-
"MathLm2dDataset",
|
8
|
-
]
|
9
|
-
|
10
|
-
from .datasets import (
|
11
|
-
MathLm1dDataset,
|
12
|
-
MathLm2dDataset,
|
13
|
-
ProfileDataset,
|
14
|
-
PseudoLmDataset,
|
15
|
-
)
|
@@ -1,41 +0,0 @@
|
|
1
|
-
"""
|
2
|
-
Landmark locators
|
3
|
-
-----------------
|
4
|
-
|
5
|
-
Functions to locate landmarks from edge profiles.
|
6
|
-
"""
|
7
|
-
|
8
|
-
import numpy as np
|
9
|
-
from heavyedge.api import landmarks_type3
|
10
|
-
|
11
|
-
__all__ = [
|
12
|
-
"pseudo_landmarks_1d",
|
13
|
-
"pseudo_landmarks_2d",
|
14
|
-
"math_landmarks_1d",
|
15
|
-
]
|
16
|
-
|
17
|
-
|
18
|
-
def pseudo_landmarks_1d(Ys, Ls, k):
|
19
|
-
ret = []
|
20
|
-
for Y, L in zip(Ys, Ls):
|
21
|
-
idxs = np.linspace(0, L - 1, k, dtype=int)
|
22
|
-
ret.append(Y[idxs].reshape(1, -1))
|
23
|
-
return np.array(ret)
|
24
|
-
|
25
|
-
|
26
|
-
def pseudo_landmarks_2d(x, Ys, Ls, k):
|
27
|
-
ret = []
|
28
|
-
for Y, L in zip(Ys, Ls):
|
29
|
-
idxs = np.linspace(0, L - 1, k, dtype=int)
|
30
|
-
ret.append(np.stack([x[idxs], Y[idxs]]))
|
31
|
-
return np.array(ret)
|
32
|
-
|
33
|
-
|
34
|
-
def math_landmarks_1d(Ys, Ls, sigma):
|
35
|
-
ret = []
|
36
|
-
for Y, L in zip(Ys, Ls):
|
37
|
-
Y = Y[:L]
|
38
|
-
indices = np.flip(landmarks_type3(Y, sigma))
|
39
|
-
y = np.concat([[np.mean(Y[: indices[0]])], Y[indices]])
|
40
|
-
ret.append(y.reshape(1, -1))
|
41
|
-
return np.array(ret)
|
@@ -1,39 +0,0 @@
|
|
1
|
-
Metadata-Version: 2.4
|
2
|
-
Name: heavyedge-dataset
|
3
|
-
Version: 0.1.0
|
4
|
-
Summary: PyTorch-compatible edge profile dataset API
|
5
|
-
Author-email: Jisoo Song <jeesoo9595@snu.ac.kr>
|
6
|
-
License-Expression: MIT
|
7
|
-
Project-URL: homepage, https://pypi.python.org/pypi/heavyedge-dataset/
|
8
|
-
Project-URL: source, https://github.com/heavyedge/heavyedge-dataset
|
9
|
-
Project-URL: documentation, https://heavyedge-dataset.readthedocs.io
|
10
|
-
Classifier: Development Status :: 5 - Production/Stable
|
11
|
-
Classifier: Intended Audience :: Science/Research
|
12
|
-
Classifier: Programming Language :: Python
|
13
|
-
Classifier: Programming Language :: Python :: 3
|
14
|
-
Classifier: Programming Language :: Python :: 3.10
|
15
|
-
Classifier: Programming Language :: Python :: 3.11
|
16
|
-
Classifier: Programming Language :: Python :: 3.12
|
17
|
-
Classifier: Programming Language :: Python :: 3 :: Only
|
18
|
-
Classifier: Topic :: Scientific/Engineering
|
19
|
-
Classifier: Operating System :: OS Independent
|
20
|
-
Requires-Python: >=3.10
|
21
|
-
Description-Content-Type: text/markdown
|
22
|
-
License-File: LICENSE
|
23
|
-
Requires-Dist: heavyedge>=1.1.2
|
24
|
-
Requires-Dist: torch
|
25
|
-
Provides-Extra: test
|
26
|
-
Requires-Dist: pytest; extra == "test"
|
27
|
-
Provides-Extra: doc
|
28
|
-
Requires-Dist: sphinx; extra == "doc"
|
29
|
-
Requires-Dist: numpydoc; extra == "doc"
|
30
|
-
Requires-Dist: pydata_sphinx_theme; extra == "doc"
|
31
|
-
Requires-Dist: matplotlib; extra == "doc"
|
32
|
-
Provides-Extra: dev
|
33
|
-
Requires-Dist: flake8; extra == "dev"
|
34
|
-
Requires-Dist: black; extra == "dev"
|
35
|
-
Requires-Dist: isort; extra == "dev"
|
36
|
-
Requires-Dist: heavyedge-dataset[doc,test]; extra == "dev"
|
37
|
-
Dynamic: license-file
|
38
|
-
|
39
|
-
# HeavyEdge-Dataset
|
File without changes
|
File without changes
|
File without changes
|
{heavyedge_dataset-0.1.0 → heavyedge_dataset-0.2.0}/src/heavyedge_dataset.egg-info/requires.txt
RENAMED
File without changes
|
{heavyedge_dataset-0.1.0 → heavyedge_dataset-0.2.0}/src/heavyedge_dataset.egg-info/top_level.txt
RENAMED
File without changes
|