matrice 1.0.977__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (34) hide show
  1. matrice-1.0.977/LICENSE.txt +19 -0
  2. matrice-1.0.977/PKG-INFO +381 -0
  3. matrice-1.0.977/README.md +355 -0
  4. matrice-1.0.977/matrice.egg-info/PKG-INFO +381 -0
  5. matrice-1.0.977/matrice.egg-info/SOURCES.txt +33 -0
  6. matrice-1.0.977/matrice.egg-info/dependency_links.txt +1 -0
  7. matrice-1.0.977/matrice.egg-info/requires.txt +1 -0
  8. matrice-1.0.977/matrice.egg-info/top_level.txt +1 -0
  9. matrice-1.0.977/setup.cfg +7 -0
  10. matrice-1.0.977/setup.py +49 -0
  11. matrice-1.0.977/src/matrice/__init__.py +0 -0
  12. matrice-1.0.977/src/matrice/action.py +144 -0
  13. matrice-1.0.977/src/matrice/actionTracker.py +1288 -0
  14. matrice-1.0.977/src/matrice/annotation.py +863 -0
  15. matrice-1.0.977/src/matrice/compute.py +514 -0
  16. matrice-1.0.977/src/matrice/dataset.py +1408 -0
  17. matrice-1.0.977/src/matrice/deploy.py +247 -0
  18. matrice-1.0.977/src/matrice/deployment.py +953 -0
  19. matrice-1.0.977/src/matrice/drift_monitor.py +184 -0
  20. matrice-1.0.977/src/matrice/experiment.py +529 -0
  21. matrice-1.0.977/src/matrice/exported_model.py +564 -0
  22. matrice-1.0.977/src/matrice/local_test.py +251 -0
  23. matrice-1.0.977/src/matrice/metrics_calculator.py +572 -0
  24. matrice-1.0.977/src/matrice/metrics_calculator_oop.py +610 -0
  25. matrice-1.0.977/src/matrice/model_store.py +1693 -0
  26. matrice-1.0.977/src/matrice/models.py +771 -0
  27. matrice-1.0.977/src/matrice/old_model_store.py +2003 -0
  28. matrice-1.0.977/src/matrice/projects.py +1856 -0
  29. matrice-1.0.977/src/matrice/rpc.py +103 -0
  30. matrice-1.0.977/src/matrice/scaling.py +251 -0
  31. matrice-1.0.977/src/matrice/session.py +340 -0
  32. matrice-1.0.977/src/matrice/testing.py +1277 -0
  33. matrice-1.0.977/src/matrice/token_auth.py +111 -0
  34. matrice-1.0.977/src/matrice/utils.py +183 -0
@@ -0,0 +1,19 @@
1
+ Copyright (c) 2018 The Python Packaging Authority
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining a copy
4
+ of this software and associated documentation files (the "Software"), to deal
5
+ in the Software without restriction, including without limitation the rights
6
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
7
+ copies of the Software, and to permit persons to whom the Software is
8
+ furnished to do so, subject to the following conditions:
9
+
10
+ The above copyright notice and this permission notice shall be included in all
11
+ copies or substantial portions of the Software.
12
+
13
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
14
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
15
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
16
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
17
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
18
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
19
+ SOFTWARE.
@@ -0,0 +1,381 @@
1
+ Metadata-Version: 2.1
2
+ Name: matrice
3
+ Version: 1.0.977
4
+ Summary: SDK for connecting to matrice.ai services
5
+ Home-page: https://github.com/matrice-ai/python-sdk
6
+ Author: Matrice.ai
7
+ Author-email: dipendra@matrice.ai
8
+ Project-URL: Bug Reports, https://github.com/matrice-ai/python-sdk/issues
9
+ Project-URL: Source, https://github.com/matrice-ai/python-sdk/
10
+ Keywords: matrice setuptools sdk development
11
+ Classifier: Development Status :: 4 - Beta
12
+ Classifier: Intended Audience :: Developers
13
+ Classifier: Operating System :: OS Independent
14
+ Classifier: Topic :: Software Development :: Build Tools
15
+ Classifier: License :: OSI Approved :: MIT License
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Python :: 3.7
18
+ Classifier: Programming Language :: Python :: 3.8
19
+ Classifier: Programming Language :: Python :: 3.9
20
+ Classifier: Programming Language :: Python :: 3.10
21
+ Classifier: Programming Language :: Python :: 3 :: Only
22
+ Requires-Python: >=3.7, <4
23
+ Description-Content-Type: text/markdown
24
+ License-File: LICENSE.txt
25
+ Requires-Dist: requests
26
+
27
+ ## Introduction
28
+
29
+ The matrice-datasets-sdk is a python package that has APIs defined in it to perform operations related to datasets on the matrice.ai platform.
30
+
31
+ ### Installation (will update it after we release this repo as a python package):
32
+
33
+ ```
34
+ git clone https://github.com/matrice-ai/matrice-dataset-sdk.git
35
+ cd matrice-database-sdk
36
+ ```
37
+
38
+ ## File structure
39
+ ```
40
+ python_sdk/
41
+ ├── docs/
42
+ │ ├── Be-Annotation.md
43
+ │ ├── Be-Dataset.md
44
+ │ ├── Be-Deployment.md
45
+ │ ├── Be-Model.md
46
+ │ ├── Be-Project.md
47
+ │ ├── Be-Resources.py
48
+ │ ├── Be-User.py
49
+ │ └── Inference-Optimization.md
50
+ ├── src/
51
+ │ ├── annotation.py
52
+ │ ├── dataset.py
53
+ │ ├── deployment.py
54
+ │ ├── inference_optim.py
55
+ │ ├── models.py
56
+ │ ├── resources.py
57
+ │ ├── rpc.py
58
+ │ └── token_auth.py
59
+ ├── test/
60
+ │ ├── annotation.ipynb
61
+ │ ├── config.py
62
+ │ ├── dataset.ipynb
63
+ │ ├── deployment.ipynb
64
+ │ ├── inference_optim.ipynb
65
+ │ ├── projects.ipynb
66
+ │ ├── models.ipynb
67
+ │ ├── resources.ipynb
68
+ │ └── user.ipynb
69
+ ├── setup.py
70
+ └── README.md
71
+ ```
72
+
73
+ ### Usage:
74
+
75
+ #### Class dataset.Dataset [[Source]](https://github.com/matrice-ai/matrice-dataset-sdk/blob/main/src/dataset.py)
76
+
77
+ All the operations on a dataset will be performed via the methods of the `Dataset` class. So, before performing any operation on the datasets, we need to instantiate an object of the `Dataset` class.
78
+
79
+ ```python
80
+ class dataset.Dataset(project_id, dataset_id=None, email="", password="")
81
+ ```
82
+
83
+ This constructor should be used to instantiate an object of the `Dataset` class.
84
+ Parameters in the constructor:
85
+
86
+ * `project_id`: string, the id of the project in which you want to create the dataset
87
+ * `dataset_id`: string, the id of the dataset on which you want to perform operations
88
+ * `email`: string, email associated with the matrice.ai platform account
89
+ * `password`: string, password for your account corresponding to `email` for the matrice.ai platform
90
+
91
+ If you want to create a new dataset and perform some operations on it, you can instantiate an object of the `Dataset` class without setting the value of `dataset_id`.
92
+
93
+ ```python3
94
+
95
+ # import Dataset class
96
+ from src.dataset import Dataset
97
+
98
+ # set the value of my_email to the email associated with the matrice.ai account
99
+ my_email = "example@gmail.com"
100
+
101
+ # set the value of my_password to the password corresponding to `my_email` for matrice.ai platform
102
+ my_password = "password"
103
+
104
+ # set the value of project_id to the id of the project in which you want to create the dataset
105
+ project_id = "abcc123facc"
106
+
107
+ # instantiate an object of the Dataset class
108
+ d1 = Dataset(project_id = project_id,email=email, password=password)
109
+
110
+ ```
111
+
112
+ If you want to perform any operation on an already created dataset, you will have to set the value of `dataset_id` to the id of the dataset on which you want to perform operations while instantiating the object of the `Datasets` class.
113
+
114
+ ```python3
115
+
116
+ # set the value of my_email to the email associated with the matrice.ai account
117
+ my_email = "example@gmail.com"
118
+
119
+ # set the value of my_password to the password corresponding to `my_email` for matrice.ai platform
120
+ my_password = "password"
121
+
122
+ # set the value of project_id to the id of the project in which the dataset you want to access exists
123
+ project_id = "abcc123facc"
124
+
125
+ # set the value of dataset_id to the id of the dataset on which you want to perform operations
126
+ dataset_id = "addfc123facc"
127
+
128
+ # instantiate an object of the Dataset class
129
+ d1 = Dataset(project_id = project_id, dataset_id = dataset_id ,email=email, password=password)
130
+
131
+ ```
132
+
133
+ Note:
134
+ All the methods defined in `Dataset` will return `response`, `error`, and `message`. The `response` will be the response given by the `matrice.ai` backend, `error` will represent the error message(if there is any error else `None`) and `message` will be a simple text message that provides information about the status of the task being requested to execute.
135
+
136
+
137
+ #### Create a dataset:
138
+ To create a dataset in the matrice.ai platform using the `matrice-datasets-sdk`, we can use the `create_dataset()` method defined in the `dataset.Dataset` class. Before, calling the `create_dataset` method, we instantiate an object of the `Dataset` class.
139
+
140
+ ```
141
+ dataset.Dataset.create_dataset(self, name, type, is_unlabeled, source, source_url, dataset_description="", version_description="")
142
+ ```
143
+
144
+ Parameters in the method:
145
+
146
+ * `name`: string, name of the dataset to be created
147
+ * `type`: string, type of the task of the dataset (for now we support `"classification"` and `"detection"` types)
148
+ * `is_unlabeled`: boolean (True/False), True if the dataset is unlabeled
149
+ * `source`: string, Source for dataset files (currently supported only "url")
150
+ * `source_url` : string, URL of the dataset zipped file
151
+ * `dataset_description`: string, Optional field that is used to store some information about the dataset we are creating
152
+ * `version_description`: string, Optional field that is used to store some information about the `v1.0` of the dataset
153
+
154
+
155
+ In the matrice.ai platform, we can create a new dataset either by uploading local `zip` files or by using a URL of the dataset files stored somewhere remotely.
156
+
157
+ To create a dataset using the URL of the dataset files, we pass source="url" and pass URL of files as the `source_url` argument.
158
+
159
+ ```python3
160
+ resp, err, msg = d1.create_dataset(name = "dataset-name",type="detection", is_unlabeled=False, source="url", source_url="https://example.com/sample2_data.tar.gz", dataset_description="Dataset created using matricesdk", version_description="Initial Version")
161
+ ```
162
+
163
+ #### Add new samples to the existing dataset:
164
+
165
+ To add new samples to the existing dataset, we can use the `add_new_samples()` method defined in the `dataset.Dataset` class.
166
+
167
+ ```
168
+ dataset.Dataset.add_new_samples(self,old_version, new_version, is_unlabeled, source, source_url, dataset_description="", version_description="")
169
+ ```
170
+
171
+ Parameters in the method:
172
+
173
+ * `old_version`: string, the version of the dataset to which you want to add samples
174
+ * `new_version`: string, the new version of the dataset with added samples (set new_version equal to old_version argument if you don't want to create a new version)
175
+ * `is_unlabeled`: boolean (True/False), True if the dataset is unlabeled
176
+ * `source`: string, Source for dataset files (currently supported only "url")
177
+ * `source_url`: string, URL of the dataset zipped file
178
+ * `dataset_description`: string, Optional field if set to a string updates existing description of the dataset
179
+ * `version_description`: string, Optional field. Used to set the description of a version if we are creating a new version or used to update the description of a version if we are updating an existing version
180
+
181
+
182
+ In the matrice.ai platform, we can add new samples to the existing dataset either by uploading local `zip` files or by using a URL of the dataset files stored somewhere remotely.
183
+
184
+ To add new samples to the existing dataset using the URL of the dataset files, we pass source="url" and pass URL of files as the `source_url` argument.
185
+
186
+ ```python3
187
+ resp, err, msg = d1.add_new_samples(old_version="v1.0", new_version="v1.0", is_unlabeled=False, source="url", source_url="https://myexample.com/sample2_data.tar.gz")
188
+ ```
189
+
190
+ The python statement above will add new samples to version `v1.0` of the `d1` dataset using the files in the `source_url`. It will not create a new version in this case.
191
+
192
+ ```python3
193
+ resp, err, msg = d1.add_new_samples(old_version="v1.0", new_version="v1.1", is_unlabeled=False, source="url", source_url="https://s3.us-west-2.amazonaws.com/temp.matrice/sample2_data.tar.gz",version_description="Adding samples")
194
+ ```
195
+
196
+ The python statement above will add new samples to version `v1.0` of the `d1` dataset using the files in the `source_url and store the final result as version `v1.1`.
197
+
198
+ #### Create dataset splits:
199
+
200
+ To move samples from one split to another split or create new random splits in the existing dataset, one can use the `split_dataset()` method defined in the `dataset.Dataset` class.
201
+
202
+ ```
203
+ dataset.Dataset.split_dataset(self,old_version, new_version, is_random_split, train_num=0, val_num=0, test_num=0, unassigned_num=0.dataset_description="", version_description="",)
204
+ ```
205
+
206
+ Parameters in the method:
207
+
208
+ * `old_version`: string, the version of the dataset in which you want to move samples from one split to another split or create new random splits
209
+ * `new_version`: string, the new version of the dataset with added samples (set new_version equal to old_version argument if you don't want to create a new version)
210
+ * `is_random_split`: boolean (True/False), set to True if you want to create fresh new random splits
211
+ * `train_num`: int, Number of samples you want to be in the training set
212
+ * `val_num`: int, Number of samples you want to be in the validation set
213
+ * `test_num`: int, Number of samples you want to be in the test set
214
+ * `unassigned_num`: int, Number of samples you want to be in the unassigned set
215
+ * `dataset_description`: string, Optional field if set to a string updates existing description of the dataset
216
+ * `version_description`: string, Optional field. Used to set the description of a version if we are creating a new version or used to update the description of a version if we are updating an existing version
217
+
218
+ #### Delete a specific version of a dataset:
219
+
220
+ To delete a specific version of a dataset, we can use the `delete_version()` method defined in the `dataset.Dataset` class.
221
+
222
+ ```
223
+ delete_version(self, version):
224
+ ```
225
+
226
+ Parameters in the method:
227
+
228
+ * `version`: string, version of the dataset to be deleted
229
+
230
+ ```python3
231
+ resp, err, msg = d1.delete_version("v1.2")
232
+ ```
233
+
234
+ The Python statement above will delete version `v1.2` of the `d1` dataset.
235
+
236
+
237
+ #### Delete a dataset:
238
+
239
+ To delete a dataset, we can use the `delete_dataset()` method defined in the `dataset.Dataset` class.
240
+
241
+ ```
242
+ delete_dataset(self)
243
+ ```
244
+
245
+ ```python3
246
+ resp, err, msg = d1.delete_dataset()
247
+ ```
248
+
249
+ The Python statement above will delete the `d1` dataset.
250
+
251
+
252
+ #### Rename a dataset:
253
+
254
+ To rename a dataset, one can use the `rename_dataset()` method defined in the `dataset.Dataset` class.
255
+
256
+ ```
257
+ rename_dataset(self, updated_name)
258
+ ```
259
+
260
+ Parameters in the method:
261
+
262
+ * `updated_name`: string, the name you want the dataset to rename to
263
+
264
+ ```python3
265
+ resp, err, message = d1.rename_dataset(updated_name="my_dataset")
266
+ ```
267
+
268
+ The Python statement above will rename the `d1` dataset to `my_dataset`.
269
+
270
+
271
+ #### Get a list of all the datasets:
272
+
273
+ To get a list of all the datasets inside a project, one can use the `get_all_datasets()` method defined in the `dataset.Dataset` class.
274
+
275
+ ```
276
+ get_all_datasets(self)
277
+ ```
278
+
279
+ ```python3
280
+ resp, err, message = d1.get_all_datasets()
281
+ ```
282
+
283
+ The Python statement above will list all the datasets inside the `project_id` of `d1`.
284
+
285
+
286
+ #### Get information(details) about a dataset:
287
+
288
+ To get information about a particular dataset inside a project, we can use the `get_dataset_info()` method defined in the `dataset.Dataset` class.
289
+
290
+ ```
291
+ get_dataset_info(self)
292
+ ```
293
+
294
+ ```python3
295
+ resp, err, message = d1.get_dataset_info()
296
+ ```
297
+
298
+ The Python statement above will provide us with the information about `d1`.
299
+
300
+
301
+ #### Get a summary of a particular version of a dataset:
302
+
303
+ To get a summary of a particular version of a dataset, one can use the `get_version_summary()` method defined in the `dataset.Dataset` class.
304
+
305
+ ```
306
+ get_version_summary(self, version)
307
+ ```
308
+
309
+ Parameters in the method:
310
+
311
+ * `version`: string, version of the dataset whose summary you want to get
312
+
313
+ ```python3
314
+ resp, err, msg = d1.get_version_summary("v1.0")
315
+ ```
316
+
317
+ The Python statement above will provide us with the summary of version `v1.0` of dataset `d1`.
318
+
319
+
320
+ #### List all versions of a dataset:
321
+
322
+ To list all the versions of a dataset, one can use the `list_versions()` method defined in the `dataset.Dataset` class.
323
+
324
+ ```
325
+ list_versions(self)
326
+ ```
327
+
328
+ ```python3
329
+ resp, err, msg = d1.list_versions()
330
+ ```
331
+
332
+ The Python statement above will provide us with the latest version and the list of all versions of the dataset `d1`.
333
+
334
+
335
+ #### List all the label categories of a dataset:
336
+
337
+ To list all the label categories of a dataset, one can use the `list_categories()` method defined in the `dataset.Dataset` class.
338
+
339
+ ```
340
+ list_categories(self)
341
+ ```
342
+
343
+ ```python3
344
+ resp, err, msg = d1.list_categories()
345
+ ```
346
+
347
+ The Python statement above will provide us with all the label categories of the dataset `d1`.
348
+
349
+
350
+ #### List all the logs for a dataset:
351
+
352
+ To list all the logs for a dataset, one can use the `get_logs()` method defined in the `dataset.Dataset` class.
353
+
354
+ ```
355
+ get_logs(self)
356
+ ```
357
+
358
+ ```python3
359
+ resp, err, msg = d1.get_logs()
360
+ ```
361
+
362
+ The Python statement above will give us a list of all the logs for the dataset `d1`.
363
+
364
+
365
+ #### List all the dataset items in a dataset:
366
+
367
+ To list all the dataset items in a dataset, one can use the `list_all_dataset_items()` method defined in the `dataset.Dataset` class.
368
+
369
+ ```
370
+ list_all_dataset_items(self, version)
371
+ ```
372
+
373
+ Parameters in the method:
374
+
375
+ * `version`: string, version of the dataset whose items you want to get
376
+
377
+ ```python3
378
+ resp, err, msg = d1.list_all_dataset_items(version="v1.0")
379
+ ```
380
+
381
+ The Python statement above will give us a list of all the dataset items in version `v1.0` of the dataset `d1`.