halib 0.2.23__tar.gz → 0.2.28__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (58) hide show
  1. {halib-0.2.23 → halib-0.2.28}/PKG-INFO +40 -3
  2. {halib-0.2.23 → halib-0.2.28}/README.md +39 -3
  3. {halib-0.2.23 → halib-0.2.28}/halib/system/path.py +2 -2
  4. {halib-0.2.23 → halib-0.2.28}/halib/utils/dict.py +47 -1
  5. halib-0.2.28/halib/utils/list.py +17 -0
  6. halib-0.2.28/halib/utils/slack.py +84 -0
  7. {halib-0.2.23 → halib-0.2.28}/halib.egg-info/PKG-INFO +40 -3
  8. {halib-0.2.23 → halib-0.2.28}/halib.egg-info/SOURCES.txt +2 -1
  9. {halib-0.2.23 → halib-0.2.28}/halib.egg-info/requires.txt +1 -0
  10. {halib-0.2.23 → halib-0.2.28}/setup.py +1 -1
  11. halib-0.2.23/halib/utils/list.py +0 -12
  12. {halib-0.2.23 → halib-0.2.28}/.gitignore +0 -0
  13. {halib-0.2.23 → halib-0.2.28}/GDriveFolder.txt +0 -0
  14. {halib-0.2.23 → halib-0.2.28}/LICENSE.txt +0 -0
  15. {halib-0.2.23 → halib-0.2.28}/MANIFEST.in +0 -0
  16. {halib-0.2.23 → halib-0.2.28}/halib/__init__.py +0 -0
  17. {halib-0.2.23 → halib-0.2.28}/halib/common/__init__.py +0 -0
  18. {halib-0.2.23 → halib-0.2.28}/halib/common/common.py +0 -0
  19. {halib-0.2.23 → halib-0.2.28}/halib/common/rich_color.py +0 -0
  20. {halib-0.2.23 → halib-0.2.28}/halib/exp/__init__.py +0 -0
  21. {halib-0.2.23 → halib-0.2.28}/halib/exp/core/__init__.py +0 -0
  22. {halib-0.2.23 → halib-0.2.28}/halib/exp/core/base_config.py +0 -0
  23. {halib-0.2.23 → halib-0.2.28}/halib/exp/core/base_exp.py +0 -0
  24. {halib-0.2.23 → halib-0.2.28}/halib/exp/core/param_gen.py +0 -0
  25. {halib-0.2.23 → halib-0.2.28}/halib/exp/core/wandb_op.py +0 -0
  26. {halib-0.2.23 → halib-0.2.28}/halib/exp/data/__init__.py +0 -0
  27. {halib-0.2.23 → halib-0.2.28}/halib/exp/data/dataclass_util.py +0 -0
  28. {halib-0.2.23 → halib-0.2.28}/halib/exp/data/dataset.py +0 -0
  29. {halib-0.2.23 → halib-0.2.28}/halib/exp/data/torchloader.py +0 -0
  30. {halib-0.2.23 → halib-0.2.28}/halib/exp/perf/__init__.py +0 -0
  31. {halib-0.2.23 → halib-0.2.28}/halib/exp/perf/flop_calc.py +0 -0
  32. {halib-0.2.23 → halib-0.2.28}/halib/exp/perf/gpu_mon.py +0 -0
  33. {halib-0.2.23 → halib-0.2.28}/halib/exp/perf/perfcalc.py +0 -0
  34. {halib-0.2.23 → halib-0.2.28}/halib/exp/perf/perfmetrics.py +0 -0
  35. {halib-0.2.23 → halib-0.2.28}/halib/exp/perf/perftb.py +0 -0
  36. {halib-0.2.23 → halib-0.2.28}/halib/exp/perf/profiler.py +0 -0
  37. {halib-0.2.23 → halib-0.2.28}/halib/exp/viz/__init__.py +0 -0
  38. {halib-0.2.23 → halib-0.2.28}/halib/exp/viz/plot.py +0 -0
  39. {halib-0.2.23 → halib-0.2.28}/halib/filetype/__init__.py +0 -0
  40. {halib-0.2.23 → halib-0.2.28}/halib/filetype/csvfile.py +0 -0
  41. {halib-0.2.23 → halib-0.2.28}/halib/filetype/ipynb.py +0 -0
  42. {halib-0.2.23 → halib-0.2.28}/halib/filetype/jsonfile.py +0 -0
  43. {halib-0.2.23 → halib-0.2.28}/halib/filetype/textfile.py +0 -0
  44. {halib-0.2.23 → halib-0.2.28}/halib/filetype/videofile.py +0 -0
  45. {halib-0.2.23 → halib-0.2.28}/halib/filetype/yamlfile.py +0 -0
  46. {halib-0.2.23 → halib-0.2.28}/halib/online/__init__.py +0 -0
  47. {halib-0.2.23 → halib-0.2.28}/halib/online/gdrive.py +0 -0
  48. {halib-0.2.23 → halib-0.2.28}/halib/online/gdrive_mkdir.py +0 -0
  49. {halib-0.2.23 → halib-0.2.28}/halib/online/projectmake.py +0 -0
  50. {halib-0.2.23 → halib-0.2.28}/halib/online/tele_noti.py +0 -0
  51. {halib-0.2.23 → halib-0.2.28}/halib/system/__init__.py +0 -0
  52. {halib-0.2.23 → halib-0.2.28}/halib/system/_list_pc.csv +0 -0
  53. {halib-0.2.23 → halib-0.2.28}/halib/system/cmd.py +0 -0
  54. {halib-0.2.23 → halib-0.2.28}/halib/system/filesys.py +0 -0
  55. {halib-0.2.23 → halib-0.2.28}/halib/utils/__init__.py +0 -0
  56. {halib-0.2.23 → halib-0.2.28}/halib.egg-info/dependency_links.txt +0 -0
  57. {halib-0.2.23 → halib-0.2.28}/halib.egg-info/top_level.txt +0 -0
  58. {halib-0.2.23 → halib-0.2.28}/setup.cfg +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: halib
3
- Version: 0.2.23
3
+ Version: 0.2.28
4
4
  Summary: Small library for common tasks
5
5
  Author: Hoang Van Ha
6
6
  Author-email: hoangvanhauit@gmail.com
@@ -42,6 +42,7 @@ Requires-Dist: tube_dl
42
42
  Requires-Dist: wandb
43
43
  Requires-Dist: ipynbname
44
44
  Requires-Dist: typed-argument-parser
45
+ Requires-Dist: slack_sdk
45
46
  Dynamic: author
46
47
  Dynamic: author-email
47
48
  Dynamic: classifier
@@ -56,25 +57,32 @@ Dynamic: summary
56
57
 
57
58
  ## v0.2.x (Experiment & Core Updates)
58
59
 
59
- ### **v0.2.23**
60
+ ### **v0.2.28**
61
+
62
+ - ✨ **New Feature:** Implement `utils.slack.SlackUtils` class for managing Slack channel message deletion
63
+
60
64
  - ✨ **New Feature:** Added `utils.dict.DictUtils` for advanced dictionary manipulations (merging, filtering, transforming).
61
65
 
62
66
  - ✨ **New Feature:** Added `common.common.pprint_stack_trace` to print stack traces with optional custom messages and force stop capability.
63
67
 
64
- - 🚀 **Improvement:** `exp.perf.profiler` - allow to export *report dict* as csv files for further analysis
68
+ - 🚀 **Improvement:** `exp.perf.profiler` - allow to export _report dict_ as csv files for further analysis
65
69
 
66
70
  ### **v0.2.19**
71
+
67
72
  - ✨ **New Feature:** Added `exp.core.param_gen` to facilitate fast generation of parameter combination sweeps (grid search) using YAML configurations.
68
73
 
69
74
  ### **v0.2.17**
75
+
70
76
  - 🚀 **Improvement:** Updated `exp.perf.profiler` with an `enabled` flag for dynamic toggling.
71
77
  - 🚀 **Improvement:** Added a `measure` context manager to simplify performance measuring of code blocks.
72
78
 
73
79
  ### **v0.2.13**
80
+
74
81
  - ♻️ **Refactor:** Major reorganization of packages. Renamed `research` package to `exp` (Experiment Management).
75
82
  - 🚀 **Improvement:** Updated `exp/perfcalc.py` to allow saving computed performance metrics to CSV without explicitly calling `calc_perfs`.
76
83
 
77
84
  ### **v0.2.1**
85
+
78
86
  - ✨ **New Feature:** Added `eval_exp` method to `exp/base_exp` for running evaluations (e.g., model testing) after experiments conclude.
79
87
 
80
88
  ---
@@ -82,15 +90,19 @@ Dynamic: summary
82
90
  ## v0.1.9x (Visualization & Generators)
83
91
 
84
92
  ### **v0.1.99**
93
+
85
94
  - ✨ **New Feature:** Added `gen_ipynb_name` to `filetype/ipynb`. Generates filenames based on the current notebook name with optional timestamps.
86
95
 
87
96
  ### **v0.1.96**
97
+
88
98
  - ✨ **New Feature:** Added `PlotHelper` class in `research/plot` for plotting training history and image grids (dataset samples or model outputs).
89
99
 
90
100
  ### **v0.1.91**
101
+
91
102
  - ✨ **New Feature:** Added `ParamGen` class to `research/param_gen` for parsing YAML files into parameter lists for hyperparameter searches.
92
103
 
93
104
  ### **v0.1.90**
105
+
94
106
  - ✨ **New Feature:** Added `zProfiler` class to `research/profiler` for measuring context/step execution time, supporting dynamic color scales in plots.
95
107
 
96
108
  ---
@@ -98,26 +110,33 @@ Dynamic: summary
98
110
  ## v0.1.5x - v0.1.7x (Infrastructure & Utilities)
99
111
 
100
112
  ### **v0.1.77**
113
+
101
114
  - ✨ **New Feature:** Added `BaseExp` class in `research/base_exp` to handle common experiment tasks (performance calculation, result saving).
102
115
 
103
116
  ### **v0.1.67**
117
+
104
118
  - 🔧 **Maintenance:** Switched to **uv** for virtual environment management.
105
119
  - 🚀 **Improvement:** Updated `research/perfcalc` to support both `torchmetrics` and custom metrics.
106
120
 
107
121
  ### **v0.1.61**
122
+
108
123
  - ✨ **New Feature:** Added `VideoUtils` (`util/video`) for common video handling tasks.
109
124
  - ✨ **New Feature:** Added `GPUMonitor` (`util/gpu_mon`) for tracking GPU usage and performance.
110
125
 
111
126
  ### **v0.1.59**
127
+
112
128
  - 🔨 **Architecture:** Added `util/perfcalc` abstract base class. This requires implementation of specific performance calculation logic.
113
129
 
114
130
  ### **v0.1.55**
131
+
115
132
  - ✨ **New Feature:** Added `util/dataclass_util` for dynamic creation of `dataclass` objects from dictionaries or YAML (supports nested structures).
116
133
 
117
134
  ### **v0.1.52**
135
+
118
136
  - ✨ **New Feature:** Added `research/perftb` module for managing experiment performance tables (filtering by dataset, metric, etc.).
119
137
 
120
138
  ### **v0.1.50**
139
+
121
140
  - ✨ **New Feature:** Added `pprint_local_path` to print clickable file URIs for local paths.
122
141
  - ✨ **New Feature:** Added `research` package containing `benchquery` for dataframe benchmarking.
123
142
  - ✨ **New Feature:** Added `wandb` module for offline syncing and batch clearing of Weights & Biases runs.
@@ -127,19 +146,24 @@ Dynamic: summary
127
146
  ## v0.1.4x (Display & formatting)
128
147
 
129
148
  ### **v0.1.47**
149
+
130
150
  - ✨ **New Feature:** Added `pprint_box` to print objects or strings inside a decorative box frame.
131
151
 
132
152
  ### **v0.1.46**
153
+
133
154
  - 🐛 **Fix:** Filtered `UserWarning: Unable to import Axes3D`.
134
155
  - 🚀 **Improvement:** Added `auto_wrap_text` to `fn_display_df` to prevent long text overflow in tables.
135
156
 
136
157
  ### **v0.1.42**
158
+
137
159
  - ✨ **New Feature:** Added `rich_color.py` wrapper for basic color lists.
138
160
 
139
161
  ### **v0.1.41**
162
+
140
163
  - ✨ **New Feature:** Added `rich_color.py` to support rich color information (palettes, strings) using the `rich` library.
141
164
 
142
165
  ### **v0.1.40**
166
+
143
167
  - 🚀 **Improvement:** Updated `csvfile.py` to use `itables` and `pygwalker` for interactive dataframe display in Jupyter notebooks.
144
168
 
145
169
  ---
@@ -147,12 +171,15 @@ Dynamic: summary
147
171
  ## v0.1.3x (Data & Loading)
148
172
 
149
173
  ### **v0.1.38**
174
+
150
175
  - ✨ **New Feature:** Added `torchloader.py` to search for optimal `DataLoader` configurations (num_workers, batch_size, pin_memory).
151
176
 
152
177
  ### **v0.1.37**
178
+
153
179
  - ✨ **New Feature:** Added `dataset.py` for splitting classification datasets into train/val/test sets.
154
180
 
155
181
  ### **v0.1.33**
182
+
156
183
  - ✨ **New Feature:** Added `plot.py` for plotting Deep Learning training history (accuracy/loss) using `seaborn` and `matplotlib`.
157
184
 
158
185
  ---
@@ -160,35 +187,45 @@ Dynamic: summary
160
187
  ## v0.1.0 - v0.1.2x (Early Utilities)
161
188
 
162
189
  ### **v0.1.29**
190
+
163
191
  - 🐛 **Fix:** Pinned `kaleido==0.1.*` for `tele_noti` as version `0.2.*` caused image generation hangs.
164
192
 
165
193
  ### **v0.1.24**
194
+
166
195
  - ♻️ **Refactor:** Renamed `sys` module to `system` to avoid conflicts with Python's built-in `sys`.
167
196
  - ✨ **New Feature:** Added `tele_noti` module for Telegram notifications regarding training progress.
168
197
 
169
198
  ### **v0.1.22**
199
+
170
200
  - ✨ **New Feature:** Added `cuda.py` to check CUDA availability for both PyTorch and TensorFlow.
171
201
 
172
202
  ### **v0.1.21**
203
+
173
204
  - ✨ **New Feature:** Added YAML inheritance and overriding support using `networkx` and `omegaconf`.
174
205
 
175
206
  ### **v0.1.15**
207
+
176
208
  - ✨ **New Feature:** Added common logging library and `@console_log` decorator for function tracing.
177
209
 
178
210
  ### **v0.1.10**
211
+
179
212
  - 🐛 **Fix:** Fixed typo `is_exit` -> `is_exist` in `filesys`.
180
213
  - 🚀 **Improvement:** Updated `gdrive` to support uploading to specific folders and returning direct shareable links.
181
214
 
182
215
  ### **v0.1.9**
216
+
183
217
  - 🔧 **Maintenance:** Added `requirements.txt`.
184
218
 
185
219
  ### **v0.1.6 - v0.1.8**
220
+
186
221
  - 🚀 **Performance:** Optimized table insertion by using an in-memory `row_pool_dict` before committing to the DataFrame.
187
222
  - ✨ **New Feature:** Added `DFCreator` for manipulating DataFrames (create, insert, display, save).
188
223
 
189
224
  ### **v0.1.4 - v0.1.5**
225
+
190
226
  - ✨ **New Feature:** Added `cmd` module.
191
227
  - ✨ **New Feature:** Support for creating Bitbucket Projects from templates.
192
228
 
193
229
  ### **v0.1.2**
230
+
194
231
  - ✨ **New Feature:** Added support for uploading local files to Google Drive.
@@ -2,25 +2,32 @@
2
2
 
3
3
  ## v0.2.x (Experiment & Core Updates)
4
4
 
5
- ### **v0.2.23**
5
+ ### **v0.2.28**
6
+
7
+ - ✨ **New Feature:** Implement `utils.slack.SlackUtils` class for managing Slack channel message deletion
8
+
6
9
  - ✨ **New Feature:** Added `utils.dict.DictUtils` for advanced dictionary manipulations (merging, filtering, transforming).
7
10
 
8
11
  - ✨ **New Feature:** Added `common.common.pprint_stack_trace` to print stack traces with optional custom messages and force stop capability.
9
12
 
10
- - 🚀 **Improvement:** `exp.perf.profiler` - allow to export *report dict* as csv files for further analysis
13
+ - 🚀 **Improvement:** `exp.perf.profiler` - allow to export _report dict_ as csv files for further analysis
11
14
 
12
15
  ### **v0.2.19**
16
+
13
17
  - ✨ **New Feature:** Added `exp.core.param_gen` to facilitate fast generation of parameter combination sweeps (grid search) using YAML configurations.
14
18
 
15
19
  ### **v0.2.17**
20
+
16
21
  - 🚀 **Improvement:** Updated `exp.perf.profiler` with an `enabled` flag for dynamic toggling.
17
22
  - 🚀 **Improvement:** Added a `measure` context manager to simplify performance measuring of code blocks.
18
23
 
19
24
  ### **v0.2.13**
25
+
20
26
  - ♻️ **Refactor:** Major reorganization of packages. Renamed `research` package to `exp` (Experiment Management).
21
27
  - 🚀 **Improvement:** Updated `exp/perfcalc.py` to allow saving computed performance metrics to CSV without explicitly calling `calc_perfs`.
22
28
 
23
29
  ### **v0.2.1**
30
+
24
31
  - ✨ **New Feature:** Added `eval_exp` method to `exp/base_exp` for running evaluations (e.g., model testing) after experiments conclude.
25
32
 
26
33
  ---
@@ -28,15 +35,19 @@
28
35
  ## v0.1.9x (Visualization & Generators)
29
36
 
30
37
  ### **v0.1.99**
38
+
31
39
  - ✨ **New Feature:** Added `gen_ipynb_name` to `filetype/ipynb`. Generates filenames based on the current notebook name with optional timestamps.
32
40
 
33
41
  ### **v0.1.96**
42
+
34
43
  - ✨ **New Feature:** Added `PlotHelper` class in `research/plot` for plotting training history and image grids (dataset samples or model outputs).
35
44
 
36
45
  ### **v0.1.91**
46
+
37
47
  - ✨ **New Feature:** Added `ParamGen` class to `research/param_gen` for parsing YAML files into parameter lists for hyperparameter searches.
38
48
 
39
49
  ### **v0.1.90**
50
+
40
51
  - ✨ **New Feature:** Added `zProfiler` class to `research/profiler` for measuring context/step execution time, supporting dynamic color scales in plots.
41
52
 
42
53
  ---
@@ -44,26 +55,33 @@
44
55
  ## v0.1.5x - v0.1.7x (Infrastructure & Utilities)
45
56
 
46
57
  ### **v0.1.77**
58
+
47
59
  - ✨ **New Feature:** Added `BaseExp` class in `research/base_exp` to handle common experiment tasks (performance calculation, result saving).
48
60
 
49
61
  ### **v0.1.67**
62
+
50
63
  - 🔧 **Maintenance:** Switched to **uv** for virtual environment management.
51
64
  - 🚀 **Improvement:** Updated `research/perfcalc` to support both `torchmetrics` and custom metrics.
52
65
 
53
66
  ### **v0.1.61**
67
+
54
68
  - ✨ **New Feature:** Added `VideoUtils` (`util/video`) for common video handling tasks.
55
69
  - ✨ **New Feature:** Added `GPUMonitor` (`util/gpu_mon`) for tracking GPU usage and performance.
56
70
 
57
71
  ### **v0.1.59**
72
+
58
73
  - 🔨 **Architecture:** Added `util/perfcalc` abstract base class. This requires implementation of specific performance calculation logic.
59
74
 
60
75
  ### **v0.1.55**
76
+
61
77
  - ✨ **New Feature:** Added `util/dataclass_util` for dynamic creation of `dataclass` objects from dictionaries or YAML (supports nested structures).
62
78
 
63
79
  ### **v0.1.52**
80
+
64
81
  - ✨ **New Feature:** Added `research/perftb` module for managing experiment performance tables (filtering by dataset, metric, etc.).
65
82
 
66
83
  ### **v0.1.50**
84
+
67
85
  - ✨ **New Feature:** Added `pprint_local_path` to print clickable file URIs for local paths.
68
86
  - ✨ **New Feature:** Added `research` package containing `benchquery` for dataframe benchmarking.
69
87
  - ✨ **New Feature:** Added `wandb` module for offline syncing and batch clearing of Weights & Biases runs.
@@ -73,19 +91,24 @@
73
91
  ## v0.1.4x (Display & formatting)
74
92
 
75
93
  ### **v0.1.47**
94
+
76
95
  - ✨ **New Feature:** Added `pprint_box` to print objects or strings inside a decorative box frame.
77
96
 
78
97
  ### **v0.1.46**
98
+
79
99
  - 🐛 **Fix:** Filtered `UserWarning: Unable to import Axes3D`.
80
100
  - 🚀 **Improvement:** Added `auto_wrap_text` to `fn_display_df` to prevent long text overflow in tables.
81
101
 
82
102
  ### **v0.1.42**
103
+
83
104
  - ✨ **New Feature:** Added `rich_color.py` wrapper for basic color lists.
84
105
 
85
106
  ### **v0.1.41**
107
+
86
108
  - ✨ **New Feature:** Added `rich_color.py` to support rich color information (palettes, strings) using the `rich` library.
87
109
 
88
110
  ### **v0.1.40**
111
+
89
112
  - 🚀 **Improvement:** Updated `csvfile.py` to use `itables` and `pygwalker` for interactive dataframe display in Jupyter notebooks.
90
113
 
91
114
  ---
@@ -93,12 +116,15 @@
93
116
  ## v0.1.3x (Data & Loading)
94
117
 
95
118
  ### **v0.1.38**
119
+
96
120
  - ✨ **New Feature:** Added `torchloader.py` to search for optimal `DataLoader` configurations (num_workers, batch_size, pin_memory).
97
121
 
98
122
  ### **v0.1.37**
123
+
99
124
  - ✨ **New Feature:** Added `dataset.py` for splitting classification datasets into train/val/test sets.
100
125
 
101
126
  ### **v0.1.33**
127
+
102
128
  - ✨ **New Feature:** Added `plot.py` for plotting Deep Learning training history (accuracy/loss) using `seaborn` and `matplotlib`.
103
129
 
104
130
  ---
@@ -106,35 +132,45 @@
106
132
  ## v0.1.0 - v0.1.2x (Early Utilities)
107
133
 
108
134
  ### **v0.1.29**
135
+
109
136
  - 🐛 **Fix:** Pinned `kaleido==0.1.*` for `tele_noti` as version `0.2.*` caused image generation hangs.
110
137
 
111
138
  ### **v0.1.24**
139
+
112
140
  - ♻️ **Refactor:** Renamed `sys` module to `system` to avoid conflicts with Python's built-in `sys`.
113
141
  - ✨ **New Feature:** Added `tele_noti` module for Telegram notifications regarding training progress.
114
142
 
115
143
  ### **v0.1.22**
144
+
116
145
  - ✨ **New Feature:** Added `cuda.py` to check CUDA availability for both PyTorch and TensorFlow.
117
146
 
118
147
  ### **v0.1.21**
148
+
119
149
  - ✨ **New Feature:** Added YAML inheritance and overriding support using `networkx` and `omegaconf`.
120
150
 
121
151
  ### **v0.1.15**
152
+
122
153
  - ✨ **New Feature:** Added common logging library and `@console_log` decorator for function tracing.
123
154
 
124
155
  ### **v0.1.10**
156
+
125
157
  - 🐛 **Fix:** Fixed typo `is_exit` -> `is_exist` in `filesys`.
126
158
  - 🚀 **Improvement:** Updated `gdrive` to support uploading to specific folders and returning direct shareable links.
127
159
 
128
160
  ### **v0.1.9**
161
+
129
162
  - 🔧 **Maintenance:** Added `requirements.txt`.
130
163
 
131
164
  ### **v0.1.6 - v0.1.8**
165
+
132
166
  - 🚀 **Performance:** Optimized table insertion by using an in-memory `row_pool_dict` before committing to the DataFrame.
133
167
  - ✨ **New Feature:** Added `DFCreator` for manipulating DataFrames (create, insert, display, save).
134
168
 
135
169
  ### **v0.1.4 - v0.1.5**
170
+
136
171
  - ✨ **New Feature:** Added `cmd` module.
137
172
  - ✨ **New Feature:** Support for creating Bitbucket Projects from templates.
138
173
 
139
174
  ### **v0.1.2**
140
- - ✨ **New Feature:** Added support for uploading local files to Google Drive.
175
+
176
+ - ✨ **New Feature:** Added support for uploading local files to Google Drive.
@@ -21,7 +21,7 @@ def load_pc_meta_info():
21
21
  # 2. Locate the file
22
22
  csv_path = resources.files(package_name).joinpath(file_name)
23
23
  global PC_TO_ABBR, ABBR_DISK_MAP, pc_df
24
- pc_df = pd.read_csv(csv_path, sep=';', encoding='utf-8')
24
+ pc_df = pd.read_csv(csv_path, sep=';', encoding='utf-8') # ty:ignore[no-matching-overload]
25
25
  PC_TO_ABBR = dict(zip(pc_df['pc_name'], pc_df['abbr']))
26
26
  ABBR_DISK_MAP = dict(zip(pc_df['abbr'], pc_df['working_disk']))
27
27
  # pprint("Loaded PC meta info:")
@@ -57,7 +57,7 @@ def get_working_disk(abbr_disk_map=ABBR_DISK_MAP):
57
57
 
58
58
  cDisk = get_working_disk()
59
59
 
60
- # ! This funcction search for full paths in the obj and normalize them according to the current platform and working disk
60
+ # ! This function search for full paths in the obj and normalize them according to the current platform and working disk
61
61
  # ! E.g: "E:/zdataset/DFire", but working_disk: "D:", current_platform: "windows" => "D:/zdataset/DFire"
62
62
  # ! E.g: "E:/zdataset/DFire", but working_disk: "D:", current_platform: "linux" => "/mnt/d/zdataset/DFire"
63
63
  def normalize_paths(obj, working_disk=cDisk, current_platform=cPlatform):
@@ -1,5 +1,8 @@
1
1
  from typing import Dict, Any, Callable, Optional
2
2
  from rich.pretty import pprint
3
+ import json
4
+ import hashlib
5
+
3
6
 
4
7
  class DictUtils:
5
8
  """
@@ -112,8 +115,26 @@ class DictUtils:
112
115
  target = {}
113
116
  target[parts[-1]] = value
114
117
 
118
+ @staticmethod
119
+ def get_unique_hash(input_dict, length=12):
120
+ """
121
+ Returns a unique hash string for a dictionary.
115
122
 
116
- if __name__ == "__main__":
123
+ :param input_dict: The dictionary params
124
+ :param length: The desired length of the hash string (default 12)
125
+ """
126
+ assert length >= 12, "Hash length must be at least 12 to ensure uniqueness."
127
+ # 1. Sort keys to ensure {a:1, b:2} == {b:2, a:1}
128
+ config_str = json.dumps(input_dict, sort_keys=True)
129
+
130
+ # 2. Generate full SHA-256 hash (64 chars long)
131
+ full_hash = hashlib.sha256(config_str.encode("utf-8")).hexdigest()
132
+
133
+ # 3. Truncate to desired length
134
+ return full_hash[:length]
135
+
136
+
137
+ def test_update():
117
138
  # --- Setup ---
118
139
  base_config = {
119
140
  "model": {
@@ -154,3 +175,28 @@ if __name__ == "__main__":
154
175
  # --- Deep Update (The Solution) ---
155
176
  pprint("Deep Update Result:")
156
177
  pprint(DictUtils.deep_update(b2, n2))
178
+
179
+
180
+ def test_hash():
181
+ # --- Usage ---
182
+ cfg1 = {"learning_rate": 0.01, "batch_size": 32, "optimizer": "adam"}
183
+ cfg1_shuffle = {
184
+ "batch_size": 32,
185
+ "optimizer": "adam",
186
+ "learning_rate": 0.01,
187
+ }
188
+ cfg2 = {"learning_rate": 0.02, "batch_size": 32, "optimizer": "adam"}
189
+ hash1 = DictUtils.get_unique_hash(cfg1)
190
+ hash2 = DictUtils.get_unique_hash(cfg1_shuffle)
191
+ hash3 = DictUtils.get_unique_hash(cfg2)
192
+ pprint(f"Config 1 Hash: {hash1}")
193
+ pprint(f"Config 1_shuffle Hash: {hash2}")
194
+ pprint(f"Config 2 Hash: {hash3}")
195
+
196
+ assert hash1 == hash2, "Hashes should match for identical dicts."
197
+ assert hash1 != hash3, "Hashes should differ for different dicts."
198
+
199
+
200
+ if __name__ == "__main__":
201
+ test_update()
202
+ test_hash()
@@ -0,0 +1,17 @@
1
+ class ListUtils:
2
+ """Utility functions for list operations."""
3
+
4
+ @staticmethod
5
+ def subtract(list_a, list_b):
6
+ return [item for item in list_a if item not in list_b]
7
+
8
+ @staticmethod
9
+ def union(list_a, list_b, no_duplicate=False):
10
+ if no_duplicate:
11
+ return list(set(list_a) | set(list_b))
12
+ else:
13
+ return list_a + list_b
14
+
15
+ @staticmethod
16
+ def intersection(list_a, list_b):
17
+ return list(set(list_a) & set(list_b))
@@ -0,0 +1,84 @@
1
+ import time
2
+ from slack_sdk import WebClient
3
+ from slack_sdk.errors import SlackApiError
4
+ from rich.pretty import pprint
5
+
6
+
7
+ class SlackUtils:
8
+ _instance = None
9
+
10
+ def __new__(cls, token=None):
11
+ """
12
+ Singleton __new__ method.
13
+ Ensures only one instance of SlackUtils exists.
14
+ """
15
+ if cls._instance is None:
16
+ if token is None:
17
+ raise ValueError(
18
+ "A Slack Token is required for the first initialization."
19
+ )
20
+
21
+ # Create the instance
22
+ cls._instance = super(SlackUtils, cls).__new__(cls)
23
+
24
+ # Initialize the WebClient only once
25
+ cls._instance.client = WebClient(token=token)
26
+ cls._instance.token = token
27
+
28
+ return cls._instance
29
+
30
+ def clear_channel(self, channel_id, sleep_interval=1.0):
31
+ """
32
+ Fetches and deletes all messages in a specified channel.
33
+ """
34
+ cursor = None
35
+ deleted_count = 0
36
+
37
+ pprint(f"--- Starting cleanup for Channel ID: {channel_id} ---")
38
+
39
+ while True:
40
+ try:
41
+ # Fetch history in batches of 100
42
+ response = self.client.conversations_history( # ty:ignore[unresolved-attribute]
43
+ channel=channel_id, cursor=cursor, limit=100
44
+ )
45
+
46
+ messages = response.get("messages", [])
47
+
48
+ if not messages:
49
+ pprint("No more messages found to delete.")
50
+ break
51
+
52
+ for msg in messages:
53
+ ts = msg.get("ts")
54
+
55
+ try:
56
+ # Attempt delete
57
+ self.client.chat_delete( # ty:ignore[unresolved-attribute]
58
+ channel=channel_id, ts=ts
59
+ )
60
+ pprint(f"Deleted: {ts}")
61
+ deleted_count += 1
62
+
63
+ # Rate limit protection (Tier 3 limit)
64
+ time.sleep(sleep_interval)
65
+
66
+ except SlackApiError as e:
67
+ error_code = e.response["error"]
68
+ if error_code == "cant_delete_message":
69
+ pprint(f"Skipped (Permission denied): {ts}")
70
+ elif error_code == "message_not_found":
71
+ pprint(f"Skipped (Already deleted): {ts}")
72
+ else:
73
+ pprint(f"Error deleting {ts}: {error_code}")
74
+ # Check for pagination
75
+ if response["has_more"]:
76
+ cursor = response["response_metadata"]["next_cursor"]
77
+ else:
78
+ break
79
+
80
+ except SlackApiError as e:
81
+ print(f"Critical API Error fetching history: {e.response['error']}")
82
+ break
83
+
84
+ print(f"--- Completed. Total messages deleted: {deleted_count} ---")
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: halib
3
- Version: 0.2.23
3
+ Version: 0.2.28
4
4
  Summary: Small library for common tasks
5
5
  Author: Hoang Van Ha
6
6
  Author-email: hoangvanhauit@gmail.com
@@ -42,6 +42,7 @@ Requires-Dist: tube_dl
42
42
  Requires-Dist: wandb
43
43
  Requires-Dist: ipynbname
44
44
  Requires-Dist: typed-argument-parser
45
+ Requires-Dist: slack_sdk
45
46
  Dynamic: author
46
47
  Dynamic: author-email
47
48
  Dynamic: classifier
@@ -56,25 +57,32 @@ Dynamic: summary
56
57
 
57
58
  ## v0.2.x (Experiment & Core Updates)
58
59
 
59
- ### **v0.2.23**
60
+ ### **v0.2.28**
61
+
62
+ - ✨ **New Feature:** Implement `utils.slack.SlackUtils` class for managing Slack channel message deletion
63
+
60
64
  - ✨ **New Feature:** Added `utils.dict.DictUtils` for advanced dictionary manipulations (merging, filtering, transforming).
61
65
 
62
66
  - ✨ **New Feature:** Added `common.common.pprint_stack_trace` to print stack traces with optional custom messages and force stop capability.
63
67
 
64
- - 🚀 **Improvement:** `exp.perf.profiler` - allow to export *report dict* as csv files for further analysis
68
+ - 🚀 **Improvement:** `exp.perf.profiler` - allow to export _report dict_ as csv files for further analysis
65
69
 
66
70
  ### **v0.2.19**
71
+
67
72
  - ✨ **New Feature:** Added `exp.core.param_gen` to facilitate fast generation of parameter combination sweeps (grid search) using YAML configurations.
68
73
 
69
74
  ### **v0.2.17**
75
+
70
76
  - 🚀 **Improvement:** Updated `exp.perf.profiler` with an `enabled` flag for dynamic toggling.
71
77
  - 🚀 **Improvement:** Added a `measure` context manager to simplify performance measuring of code blocks.
72
78
 
73
79
  ### **v0.2.13**
80
+
74
81
  - ♻️ **Refactor:** Major reorganization of packages. Renamed `research` package to `exp` (Experiment Management).
75
82
  - 🚀 **Improvement:** Updated `exp/perfcalc.py` to allow saving computed performance metrics to CSV without explicitly calling `calc_perfs`.
76
83
 
77
84
  ### **v0.2.1**
85
+
78
86
  - ✨ **New Feature:** Added `eval_exp` method to `exp/base_exp` for running evaluations (e.g., model testing) after experiments conclude.
79
87
 
80
88
  ---
@@ -82,15 +90,19 @@ Dynamic: summary
82
90
  ## v0.1.9x (Visualization & Generators)
83
91
 
84
92
  ### **v0.1.99**
93
+
85
94
  - ✨ **New Feature:** Added `gen_ipynb_name` to `filetype/ipynb`. Generates filenames based on the current notebook name with optional timestamps.
86
95
 
87
96
  ### **v0.1.96**
97
+
88
98
  - ✨ **New Feature:** Added `PlotHelper` class in `research/plot` for plotting training history and image grids (dataset samples or model outputs).
89
99
 
90
100
  ### **v0.1.91**
101
+
91
102
  - ✨ **New Feature:** Added `ParamGen` class to `research/param_gen` for parsing YAML files into parameter lists for hyperparameter searches.
92
103
 
93
104
  ### **v0.1.90**
105
+
94
106
  - ✨ **New Feature:** Added `zProfiler` class to `research/profiler` for measuring context/step execution time, supporting dynamic color scales in plots.
95
107
 
96
108
  ---
@@ -98,26 +110,33 @@ Dynamic: summary
98
110
  ## v0.1.5x - v0.1.7x (Infrastructure & Utilities)
99
111
 
100
112
  ### **v0.1.77**
113
+
101
114
  - ✨ **New Feature:** Added `BaseExp` class in `research/base_exp` to handle common experiment tasks (performance calculation, result saving).
102
115
 
103
116
  ### **v0.1.67**
117
+
104
118
  - 🔧 **Maintenance:** Switched to **uv** for virtual environment management.
105
119
  - 🚀 **Improvement:** Updated `research/perfcalc` to support both `torchmetrics` and custom metrics.
106
120
 
107
121
  ### **v0.1.61**
122
+
108
123
  - ✨ **New Feature:** Added `VideoUtils` (`util/video`) for common video handling tasks.
109
124
  - ✨ **New Feature:** Added `GPUMonitor` (`util/gpu_mon`) for tracking GPU usage and performance.
110
125
 
111
126
  ### **v0.1.59**
127
+
112
128
  - 🔨 **Architecture:** Added `util/perfcalc` abstract base class. This requires implementation of specific performance calculation logic.
113
129
 
114
130
  ### **v0.1.55**
131
+
115
132
  - ✨ **New Feature:** Added `util/dataclass_util` for dynamic creation of `dataclass` objects from dictionaries or YAML (supports nested structures).
116
133
 
117
134
  ### **v0.1.52**
135
+
118
136
  - ✨ **New Feature:** Added `research/perftb` module for managing experiment performance tables (filtering by dataset, metric, etc.).
119
137
 
120
138
  ### **v0.1.50**
139
+
121
140
  - ✨ **New Feature:** Added `pprint_local_path` to print clickable file URIs for local paths.
122
141
  - ✨ **New Feature:** Added `research` package containing `benchquery` for dataframe benchmarking.
123
142
  - ✨ **New Feature:** Added `wandb` module for offline syncing and batch clearing of Weights & Biases runs.
@@ -127,19 +146,24 @@ Dynamic: summary
127
146
  ## v0.1.4x (Display & formatting)
128
147
 
129
148
  ### **v0.1.47**
149
+
130
150
  - ✨ **New Feature:** Added `pprint_box` to print objects or strings inside a decorative box frame.
131
151
 
132
152
  ### **v0.1.46**
153
+
133
154
  - 🐛 **Fix:** Filtered `UserWarning: Unable to import Axes3D`.
134
155
  - 🚀 **Improvement:** Added `auto_wrap_text` to `fn_display_df` to prevent long text overflow in tables.
135
156
 
136
157
  ### **v0.1.42**
158
+
137
159
  - ✨ **New Feature:** Added `rich_color.py` wrapper for basic color lists.
138
160
 
139
161
  ### **v0.1.41**
162
+
140
163
  - ✨ **New Feature:** Added `rich_color.py` to support rich color information (palettes, strings) using the `rich` library.
141
164
 
142
165
  ### **v0.1.40**
166
+
143
167
  - 🚀 **Improvement:** Updated `csvfile.py` to use `itables` and `pygwalker` for interactive dataframe display in Jupyter notebooks.
144
168
 
145
169
  ---
@@ -147,12 +171,15 @@ Dynamic: summary
147
171
  ## v0.1.3x (Data & Loading)
148
172
 
149
173
  ### **v0.1.38**
174
+
150
175
  - ✨ **New Feature:** Added `torchloader.py` to search for optimal `DataLoader` configurations (num_workers, batch_size, pin_memory).
151
176
 
152
177
  ### **v0.1.37**
178
+
153
179
  - ✨ **New Feature:** Added `dataset.py` for splitting classification datasets into train/val/test sets.
154
180
 
155
181
  ### **v0.1.33**
182
+
156
183
  - ✨ **New Feature:** Added `plot.py` for plotting Deep Learning training history (accuracy/loss) using `seaborn` and `matplotlib`.
157
184
 
158
185
  ---
@@ -160,35 +187,45 @@ Dynamic: summary
160
187
  ## v0.1.0 - v0.1.2x (Early Utilities)
161
188
 
162
189
  ### **v0.1.29**
190
+
163
191
  - 🐛 **Fix:** Pinned `kaleido==0.1.*` for `tele_noti` as version `0.2.*` caused image generation hangs.
164
192
 
165
193
  ### **v0.1.24**
194
+
166
195
  - ♻️ **Refactor:** Renamed `sys` module to `system` to avoid conflicts with Python's built-in `sys`.
167
196
  - ✨ **New Feature:** Added `tele_noti` module for Telegram notifications regarding training progress.
168
197
 
169
198
  ### **v0.1.22**
199
+
170
200
  - ✨ **New Feature:** Added `cuda.py` to check CUDA availability for both PyTorch and TensorFlow.
171
201
 
172
202
  ### **v0.1.21**
203
+
173
204
  - ✨ **New Feature:** Added YAML inheritance and overriding support using `networkx` and `omegaconf`.
174
205
 
175
206
  ### **v0.1.15**
207
+
176
208
  - ✨ **New Feature:** Added common logging library and `@console_log` decorator for function tracing.
177
209
 
178
210
  ### **v0.1.10**
211
+
179
212
  - 🐛 **Fix:** Fixed typo `is_exit` -> `is_exist` in `filesys`.
180
213
  - 🚀 **Improvement:** Updated `gdrive` to support uploading to specific folders and returning direct shareable links.
181
214
 
182
215
  ### **v0.1.9**
216
+
183
217
  - 🔧 **Maintenance:** Added `requirements.txt`.
184
218
 
185
219
  ### **v0.1.6 - v0.1.8**
220
+
186
221
  - 🚀 **Performance:** Optimized table insertion by using an in-memory `row_pool_dict` before committing to the DataFrame.
187
222
  - ✨ **New Feature:** Added `DFCreator` for manipulating DataFrames (create, insert, display, save).
188
223
 
189
224
  ### **v0.1.4 - v0.1.5**
225
+
190
226
  - ✨ **New Feature:** Added `cmd` module.
191
227
  - ✨ **New Feature:** Support for creating Bitbucket Projects from templates.
192
228
 
193
229
  ### **v0.1.2**
230
+
194
231
  - ✨ **New Feature:** Added support for uploading local files to Google Drive.
@@ -51,4 +51,5 @@ halib/system/filesys.py
51
51
  halib/system/path.py
52
52
  halib/utils/__init__.py
53
53
  halib/utils/dict.py
54
- halib/utils/list.py
54
+ halib/utils/list.py
55
+ halib/utils/slack.py
@@ -30,3 +30,4 @@ tube_dl
30
30
  wandb
31
31
  ipynbname
32
32
  typed-argument-parser
33
+ slack_sdk
@@ -8,7 +8,7 @@ with open("requirements.txt", "r", encoding="utf-8") as f:
8
8
 
9
9
  setuptools.setup(
10
10
  name="halib",
11
- version="0.2.23",
11
+ version="0.2.28",
12
12
  author="Hoang Van Ha",
13
13
  author_email="hoangvanhauit@gmail.com",
14
14
  description="Small library for common tasks",
@@ -1,12 +0,0 @@
1
- def subtract(list_a, list_b):
2
- return [item for item in list_a if item not in list_b]
3
-
4
-
5
- def union(list_a, list_b, no_duplicate=False):
6
- if no_duplicate:
7
- return list(set(list_a) | set(list_b))
8
- else:
9
- return list_a + list_b
10
-
11
- def intersection(list_a, list_b):
12
- return list(set(list_a) & set(list_b))
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes