sciveo 0.0.50__tar.gz → 0.1.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {sciveo-0.0.50 → sciveo-0.1.1}/PKG-INFO +12 -3
- {sciveo-0.0.50 → sciveo-0.1.1}/README.md +11 -2
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/monitoring/monitor.py +22 -17
- sciveo-0.1.1/sciveo/version.py +2 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo.egg-info/PKG-INFO +12 -3
- sciveo-0.0.50/sciveo/version.py +0 -2
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/__init__.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/api/__init__.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/api/base.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/api/upload.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/common/__init__.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/common/configuration.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/common/model.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/common/optimizers.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/common/sampling.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/common/tools/__init__.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/common/tools/daemon.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/common/tools/formating.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/common/tools/hardware.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/common/tools/logger.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/common/tools/synchronized.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/common/tools/timers.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/content/__init__.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/content/dataset.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/content/experiment.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/content/project.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/content/runner.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/monitoring/__init__.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/monitoring/network.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo/monitoring/start.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo.egg-info/SOURCES.txt +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo.egg-info/dependency_links.txt +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo.egg-info/requires.txt +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/sciveo.egg-info/top_level.txt +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/setup.cfg +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/setup.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/test/test_configuration.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/test/test_monitoring.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/test/test_runner.py +0 -0
- {sciveo-0.0.50 → sciveo-0.1.1}/test/test_sampling.py +0 -0
|
@@ -1,18 +1,21 @@
|
|
|
1
1
|
Metadata-Version: 2.1
|
|
2
2
|
Name: sciveo
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.1.1
|
|
4
4
|
Description-Content-Type: text/markdown
|
|
5
5
|
Provides-Extra: mon
|
|
6
6
|
Provides-Extra: net
|
|
7
7
|
Provides-Extra: all
|
|
8
8
|
|
|
9
|
-
# SCIVEO - ML/
|
|
9
|
+
# SCIVEO - ML/AI and Scientific tools
|
|
10
10
|
|
|
11
|
+
## Experiments Management Client
|
|
11
12
|
`sciveo` is a Python library that serves as a client for managing machine learning and scientific experiments on the sciveo.com platform. This library provides a convenient interface to interact with the sciveo.com API, enabling users to organize, track, and analyze their experiments efficiently.
|
|
12
13
|
There are few configuration params samplers, which allows easy parameter tuning. The "auto" sampler perhaps is the easiest to use, but also
|
|
13
14
|
"random" and "grid" ones are available.
|
|
14
15
|
|
|
15
|
-
|
|
16
|
+
## Monitoring client
|
|
17
|
+
There is also the sciveo.monitor() which will start monitoring machine CPU/RAM/GPU/NET/DISK/TEMP etc.
|
|
18
|
+
It is very easy to start and use.
|
|
16
19
|
|
|
17
20
|
|
|
18
21
|
## Features
|
|
@@ -21,6 +24,7 @@ There is also the sciveo.monitor() which will start monitoring machine CPU/RAM/G
|
|
|
21
24
|
- **Experiment Comparison:** Compare different experiments and their results.
|
|
22
25
|
- **Data Visualization:** Visualize experiment metrics and results.
|
|
23
26
|
- **Integration with sciveo.com:** Seamlessly connect and synchronize with the sciveo.com platform.
|
|
27
|
+
- **Monitoring machines (from HPC to jetson nano):** Visualisation and metrics collection in sciveo platform.
|
|
24
28
|
|
|
25
29
|
## Installation
|
|
26
30
|
|
|
@@ -29,6 +33,11 @@ pip install sciveo
|
|
|
29
33
|
|
|
30
34
|
- optional for sciveo monitoring
|
|
31
35
|
pip install sciveo[mon]
|
|
36
|
+
- optional for sciveo network tools
|
|
37
|
+
pip install sciveo[net]
|
|
38
|
+
|
|
39
|
+
- for full installation
|
|
40
|
+
pip install sciveo[all]
|
|
32
41
|
|
|
33
42
|
## Example usage
|
|
34
43
|
|
|
@@ -1,10 +1,13 @@
|
|
|
1
|
-
# SCIVEO - ML/
|
|
1
|
+
# SCIVEO - ML/AI and Scientific tools
|
|
2
2
|
|
|
3
|
+
## Experiments Management Client
|
|
3
4
|
`sciveo` is a Python library that serves as a client for managing machine learning and scientific experiments on the sciveo.com platform. This library provides a convenient interface to interact with the sciveo.com API, enabling users to organize, track, and analyze their experiments efficiently.
|
|
4
5
|
There are few configuration params samplers, which allows easy parameter tuning. The "auto" sampler perhaps is the easiest to use, but also
|
|
5
6
|
"random" and "grid" ones are available.
|
|
6
7
|
|
|
7
|
-
|
|
8
|
+
## Monitoring client
|
|
9
|
+
There is also the sciveo.monitor() which will start monitoring machine CPU/RAM/GPU/NET/DISK/TEMP etc.
|
|
10
|
+
It is very easy to start and use.
|
|
8
11
|
|
|
9
12
|
|
|
10
13
|
## Features
|
|
@@ -13,6 +16,7 @@ There is also the sciveo.monitor() which will start monitoring machine CPU/RAM/G
|
|
|
13
16
|
- **Experiment Comparison:** Compare different experiments and their results.
|
|
14
17
|
- **Data Visualization:** Visualize experiment metrics and results.
|
|
15
18
|
- **Integration with sciveo.com:** Seamlessly connect and synchronize with the sciveo.com platform.
|
|
19
|
+
- **Monitoring machines (from HPC to jetson nano):** Visualisation and metrics collection in sciveo platform.
|
|
16
20
|
|
|
17
21
|
## Installation
|
|
18
22
|
|
|
@@ -21,6 +25,11 @@ pip install sciveo
|
|
|
21
25
|
|
|
22
26
|
- optional for sciveo monitoring
|
|
23
27
|
pip install sciveo[mon]
|
|
28
|
+
- optional for sciveo network tools
|
|
29
|
+
pip install sciveo[net]
|
|
30
|
+
|
|
31
|
+
- for full installation
|
|
32
|
+
pip install sciveo[all]
|
|
24
33
|
|
|
25
34
|
## Example usage
|
|
26
35
|
|
|
@@ -15,6 +15,7 @@ import time
|
|
|
15
15
|
import datetime
|
|
16
16
|
import socket
|
|
17
17
|
import psutil
|
|
18
|
+
import platform
|
|
18
19
|
import uuid
|
|
19
20
|
import numpy as np
|
|
20
21
|
|
|
@@ -25,6 +26,21 @@ from sciveo.common.tools.formating import format_memory_size
|
|
|
25
26
|
from sciveo.api.base import APIRemoteClient
|
|
26
27
|
|
|
27
28
|
|
|
29
|
+
class MonitorTools:
|
|
30
|
+
@staticmethod
|
|
31
|
+
def serial():
|
|
32
|
+
machine_serial = ""
|
|
33
|
+
s = "-"
|
|
34
|
+
list_uid_calls = [socket.gethostname, psutil.cpu_count, platform.processor]
|
|
35
|
+
for uid_call in list_uid_calls:
|
|
36
|
+
try:
|
|
37
|
+
machine_serial += f"{uid_call()}{s}"
|
|
38
|
+
s = ""
|
|
39
|
+
except Exception:
|
|
40
|
+
pass
|
|
41
|
+
return machine_serial
|
|
42
|
+
|
|
43
|
+
|
|
28
44
|
class BaseMonitor(DaemonBase):
|
|
29
45
|
def __init__(self, period=5):
|
|
30
46
|
super().__init__(period=period)
|
|
@@ -50,7 +66,7 @@ class BaseMonitor(DaemonBase):
|
|
|
50
66
|
self.previous_time = {"DISK": time.time(), "NET": time.time()}
|
|
51
67
|
time.sleep(1)
|
|
52
68
|
|
|
53
|
-
machine_serial =
|
|
69
|
+
machine_serial = MonitorTools.serial()
|
|
54
70
|
|
|
55
71
|
debug(type(self).__name__, f"init monitor with period={period}", machine_serial, "initial_cpu_usage", initial_cpu_usage)
|
|
56
72
|
|
|
@@ -103,22 +119,6 @@ class BaseMonitor(DaemonBase):
|
|
|
103
119
|
except Exception:
|
|
104
120
|
pass
|
|
105
121
|
|
|
106
|
-
def getserial(self):
|
|
107
|
-
machine_serial = None
|
|
108
|
-
try:
|
|
109
|
-
machine_serial = f"{socket.gethostname()}-{uuid.getnode()}"
|
|
110
|
-
except Exception:
|
|
111
|
-
pass
|
|
112
|
-
if machine_serial is None:
|
|
113
|
-
try:
|
|
114
|
-
machine_serial = f"UUID-{uuid.getnode()}"
|
|
115
|
-
except Exception:
|
|
116
|
-
pass
|
|
117
|
-
if machine_serial is None:
|
|
118
|
-
machine_serial = f"RND-{random_token(8)}"
|
|
119
|
-
self.data["serial"] = machine_serial
|
|
120
|
-
return machine_serial
|
|
121
|
-
|
|
122
122
|
def tail_file(self, file_path, block_size=1024):
|
|
123
123
|
result = ["EMPTY"]
|
|
124
124
|
try:
|
|
@@ -153,6 +153,9 @@ class BaseMonitor(DaemonBase):
|
|
|
153
153
|
list_metrics = ["read bytes", "write bytes", "read count", "write count", "read time", "write time"]
|
|
154
154
|
disk_io_counters = psutil.disk_io_counters(perdisk=False)
|
|
155
155
|
self.get_io_metrics("DISK", list_metrics, disk_io_counters)
|
|
156
|
+
|
|
157
|
+
disk_usage = psutil.disk_usage('/')._asdict()
|
|
158
|
+
self.data["DISK"]["print"] = f"{disk_usage['percent']}% ({round(disk_usage['used'] / (1024 * 1024 * 1024), 1)} GB / {round(disk_usage['total'] / (1024 * 1024 * 1024), 1)} GB)"
|
|
156
159
|
except Exception as e:
|
|
157
160
|
pass
|
|
158
161
|
|
|
@@ -179,6 +182,8 @@ class BaseMonitor(DaemonBase):
|
|
|
179
182
|
self.previous_io_counters[name] = io_counters
|
|
180
183
|
|
|
181
184
|
if __name__ == "__main__":
|
|
185
|
+
print(MonitorTools.serial())
|
|
186
|
+
|
|
182
187
|
mon = BaseMonitor(period=10)
|
|
183
188
|
mon.start()
|
|
184
189
|
|
|
@@ -1,18 +1,21 @@
|
|
|
1
1
|
Metadata-Version: 2.1
|
|
2
2
|
Name: sciveo
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.1.1
|
|
4
4
|
Description-Content-Type: text/markdown
|
|
5
5
|
Provides-Extra: mon
|
|
6
6
|
Provides-Extra: net
|
|
7
7
|
Provides-Extra: all
|
|
8
8
|
|
|
9
|
-
# SCIVEO - ML/
|
|
9
|
+
# SCIVEO - ML/AI and Scientific tools
|
|
10
10
|
|
|
11
|
+
## Experiments Management Client
|
|
11
12
|
`sciveo` is a Python library that serves as a client for managing machine learning and scientific experiments on the sciveo.com platform. This library provides a convenient interface to interact with the sciveo.com API, enabling users to organize, track, and analyze their experiments efficiently.
|
|
12
13
|
There are few configuration params samplers, which allows easy parameter tuning. The "auto" sampler perhaps is the easiest to use, but also
|
|
13
14
|
"random" and "grid" ones are available.
|
|
14
15
|
|
|
15
|
-
|
|
16
|
+
## Monitoring client
|
|
17
|
+
There is also the sciveo.monitor() which will start monitoring machine CPU/RAM/GPU/NET/DISK/TEMP etc.
|
|
18
|
+
It is very easy to start and use.
|
|
16
19
|
|
|
17
20
|
|
|
18
21
|
## Features
|
|
@@ -21,6 +24,7 @@ There is also the sciveo.monitor() which will start monitoring machine CPU/RAM/G
|
|
|
21
24
|
- **Experiment Comparison:** Compare different experiments and their results.
|
|
22
25
|
- **Data Visualization:** Visualize experiment metrics and results.
|
|
23
26
|
- **Integration with sciveo.com:** Seamlessly connect and synchronize with the sciveo.com platform.
|
|
27
|
+
- **Monitoring machines (from HPC to jetson nano):** Visualisation and metrics collection in sciveo platform.
|
|
24
28
|
|
|
25
29
|
## Installation
|
|
26
30
|
|
|
@@ -29,6 +33,11 @@ pip install sciveo
|
|
|
29
33
|
|
|
30
34
|
- optional for sciveo monitoring
|
|
31
35
|
pip install sciveo[mon]
|
|
36
|
+
- optional for sciveo network tools
|
|
37
|
+
pip install sciveo[net]
|
|
38
|
+
|
|
39
|
+
- for full installation
|
|
40
|
+
pip install sciveo[all]
|
|
32
41
|
|
|
33
42
|
## Example usage
|
|
34
43
|
|
sciveo-0.0.50/sciveo/version.py
DELETED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|