cometx 3.6.4__tar.gz → 3.6.6__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. {cometx-3.6.4 → cometx-3.6.6}/PKG-INFO +44 -5
  2. {cometx-3.6.4 → cometx-3.6.6}/README.md +43 -4
  3. {cometx-3.6.4 → cometx-3.6.6}/cometx/_version.py +1 -1
  4. {cometx-3.6.4 → cometx-3.6.6}/cometx/api.py +131 -3
  5. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/__init__.py +3 -0
  6. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/copy.py +40 -4
  7. cometx-3.6.6/cometx/cli/migrate_users.py +393 -0
  8. {cometx-3.6.4 → cometx-3.6.6}/cometx.egg-info/PKG-INFO +44 -5
  9. {cometx-3.6.4 → cometx-3.6.6}/cometx.egg-info/SOURCES.txt +1 -0
  10. {cometx-3.6.4 → cometx-3.6.6}/LICENSE +0 -0
  11. {cometx-3.6.4 → cometx-3.6.6}/cometx/__init__.py +0 -0
  12. {cometx-3.6.4 → cometx-3.6.6}/cometx/_typing.py +0 -0
  13. {cometx-3.6.4 → cometx-3.6.6}/cometx/_ui.py +0 -0
  14. {cometx-3.6.4 → cometx-3.6.6}/cometx/audio.py +0 -0
  15. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/admin.py +0 -0
  16. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/admin_app.py +0 -0
  17. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/admin_gpu_app.py +0 -0
  18. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/admin_gpu_report.py +0 -0
  19. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/admin_optimizer_report.py +0 -0
  20. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/admin_usage_report.py +0 -0
  21. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/admin_utils.py +0 -0
  22. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/config.py +0 -0
  23. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/copy_utils.py +0 -0
  24. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/count.py +0 -0
  25. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/delete_assets.py +0 -0
  26. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/download.py +0 -0
  27. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/list_command.py +0 -0
  28. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/log.py +0 -0
  29. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/rename_duplicates.py +0 -0
  30. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/reproduce.py +0 -0
  31. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/smoke_test.py +0 -0
  32. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/update.py +0 -0
  33. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/upload_optimizer_experiments.py +0 -0
  34. {cometx-3.6.4 → cometx-3.6.6}/cometx/cli/utils.py +0 -0
  35. {cometx-3.6.4 → cometx-3.6.6}/cometx/framework/__init__.py +0 -0
  36. {cometx-3.6.4 → cometx-3.6.6}/cometx/framework/comet/__init__.py +0 -0
  37. {cometx-3.6.4 → cometx-3.6.6}/cometx/framework/comet/download_manager.py +0 -0
  38. {cometx-3.6.4 → cometx-3.6.6}/cometx/framework/neptune.py +0 -0
  39. {cometx-3.6.4 → cometx-3.6.6}/cometx/framework/wandb.py +0 -0
  40. {cometx-3.6.4 → cometx-3.6.6}/cometx/generate_utils.py +0 -0
  41. {cometx-3.6.4 → cometx-3.6.6}/cometx/magics.py +0 -0
  42. {cometx-3.6.4 → cometx-3.6.6}/cometx/optimizer.py +0 -0
  43. {cometx-3.6.4 → cometx-3.6.6}/cometx/panel_utils.py +0 -0
  44. {cometx-3.6.4 → cometx-3.6.6}/cometx/tools/__init__.py +0 -0
  45. {cometx-3.6.4 → cometx-3.6.6}/cometx/tools/dataset.py +0 -0
  46. {cometx-3.6.4 → cometx-3.6.6}/cometx/tools/pointcloud.py +0 -0
  47. {cometx-3.6.4 → cometx-3.6.6}/cometx/utils.py +0 -0
  48. {cometx-3.6.4 → cometx-3.6.6}/cometx.egg-info/dependency_links.txt +0 -0
  49. {cometx-3.6.4 → cometx-3.6.6}/cometx.egg-info/entry_points.txt +0 -0
  50. {cometx-3.6.4 → cometx-3.6.6}/cometx.egg-info/requires.txt +0 -0
  51. {cometx-3.6.4 → cometx-3.6.6}/cometx.egg-info/top_level.txt +0 -0
  52. {cometx-3.6.4 → cometx-3.6.6}/pyproject.toml +0 -0
  53. {cometx-3.6.4 → cometx-3.6.6}/setup.cfg +0 -0
  54. {cometx-3.6.4 → cometx-3.6.6}/setup.py +0 -0
  55. {cometx-3.6.4 → cometx-3.6.6}/tests/testlib.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: cometx
3
- Version: 3.6.4
3
+ Version: 3.6.6
4
4
  Summary: Python tools for Comet
5
5
  Home-page: https://github.com/comet-ml/cometx/
6
6
  Author: cometx development team
@@ -102,6 +102,7 @@ for ML management.
102
102
  * [cometx download](#cometx-download)
103
103
  * [cometx list](#cometx-list)
104
104
  * [cometx log](#cometx-log)
105
+ * [cometx migrate-users](#cometx-migrate-users)
105
106
  * [cometx rename-duplicates](#cometx-rename-duplicates)
106
107
  * [cometx reproduce](#cometx-reproduce)
107
108
  * [cometx smoke-test](#cometx-smoke-test)
@@ -324,6 +325,7 @@ Not all combinations are possible:
324
325
  * `--symlink` - Instead of copying, create a link to an experiment in a project
325
326
  * `--sync` - Check to see if experiment name has been created first; if so, skip
326
327
  * `--path PATH` - Path to prepend to workspace_src when accessing files (supports ~ for home directory)
328
+ * `--create-workspaces` - Attempt to create the destination workspace if it does not exist (requires appropriate API permissions)
327
329
 
328
330
  ### Using --path
329
331
 
@@ -385,6 +387,47 @@ Where TYPE is one of the following names:
385
387
 
386
388
  For more information, `cometx log --help`
387
389
 
390
+ ## cometx migrate-users
391
+
392
+ This command migrates users into workspaces from a source Comet environment to a destination environment. It reads workspace membership from a chargeback report fetched from the source environment's admin API (or a pre-downloaded local file) and invites each user to the corresponding workspace in the destination environment by email. If a user with that email does not yet exist in the destination, they will be provisioned access once they sign up.
393
+
394
+ ```
395
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY [FLAGS ...]
396
+ cometx migrate-users --api-key DEST_KEY --chargeback-report /path/to/report.json [FLAGS ...]
397
+ ```
398
+
399
+ **Arguments:**
400
+ * `--api-key API_KEY` - API key for the destination environment. Falls back to `COMET_API_KEY` if not provided.
401
+ * `--url URL` - Base URL of the destination Comet environment (e.g. `https://comet.example.com`). Required for self-hosted instances when the API key does not encode the server URL.
402
+ * `--source-api-key SOURCE_API_KEY` - API key for the source environment. Used to fetch the chargeback report. Required unless `--chargeback-report` is given.
403
+ * `--source-url SOURCE_URL` - Base URL of the source Comet environment. Required for self-hosted source instances when the source API key does not encode the server URL.
404
+ * `--chargeback-report PATH` - Path to a local chargeback report JSON file. When provided, `--source-api-key` is not required.
405
+
406
+ ### Flags
407
+
408
+ * `--create-workspaces` - Create workspaces on the destination if they don't already exist (default: off)
409
+ * `--dry-run` - Print what would happen without making any changes
410
+ * `--failures-output PATH` - Path to write failed operations JSON (default: `bulk_add_failures_by_email.json`)
411
+
412
+ ### Examples
413
+
414
+ ```bash
415
+ # Dry run — preview what would happen
416
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY --dry-run
417
+
418
+ # Execute the migration
419
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY
420
+
421
+ # Use a local chargeback report file
422
+ cometx migrate-users --api-key DEST_KEY --chargeback-report /tmp/chargeback_reports.json
423
+
424
+ # Self-hosted environments — provide explicit URLs
425
+ cometx migrate-users --api-key DEST_KEY --url https://comet.dest.example.com \
426
+ --source-api-key SOURCE_KEY --source-url https://comet.src.example.com
427
+ ```
428
+
429
+ For more information, `cometx migrate-users --help`
430
+
388
431
  ## cometx rename-duplicates
389
432
 
390
433
  This command is used to rename duplicate experiments within projects. When multiple experiments share the same name in a project, this command renames the duplicates to NAME-1, NAME-2, etc. while avoiding conflicts with existing names.
@@ -535,12 +578,8 @@ cometx admin chargeback-report [YEAR-MONTH]
535
578
  **Examples:**
536
579
  ```
537
580
  cometx admin chargeback-report
538
- <<<<<<< Updated upstream
539
- cometx admin usage-report
540
- =======
541
581
  cometx admin chargeback-report 2024-09
542
582
  ```
543
- >>>>>>> Stashed changes
544
583
 
545
584
  #### usage-report
546
585
 
@@ -46,6 +46,7 @@ for ML management.
46
46
  * [cometx download](#cometx-download)
47
47
  * [cometx list](#cometx-list)
48
48
  * [cometx log](#cometx-log)
49
+ * [cometx migrate-users](#cometx-migrate-users)
49
50
  * [cometx rename-duplicates](#cometx-rename-duplicates)
50
51
  * [cometx reproduce](#cometx-reproduce)
51
52
  * [cometx smoke-test](#cometx-smoke-test)
@@ -268,6 +269,7 @@ Not all combinations are possible:
268
269
  * `--symlink` - Instead of copying, create a link to an experiment in a project
269
270
  * `--sync` - Check to see if experiment name has been created first; if so, skip
270
271
  * `--path PATH` - Path to prepend to workspace_src when accessing files (supports ~ for home directory)
272
+ * `--create-workspaces` - Attempt to create the destination workspace if it does not exist (requires appropriate API permissions)
271
273
 
272
274
  ### Using --path
273
275
 
@@ -329,6 +331,47 @@ Where TYPE is one of the following names:
329
331
 
330
332
  For more information, `cometx log --help`
331
333
 
334
+ ## cometx migrate-users
335
+
336
+ This command migrates users into workspaces from a source Comet environment to a destination environment. It reads workspace membership from a chargeback report fetched from the source environment's admin API (or a pre-downloaded local file) and invites each user to the corresponding workspace in the destination environment by email. If a user with that email does not yet exist in the destination, they will be provisioned access once they sign up.
337
+
338
+ ```
339
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY [FLAGS ...]
340
+ cometx migrate-users --api-key DEST_KEY --chargeback-report /path/to/report.json [FLAGS ...]
341
+ ```
342
+
343
+ **Arguments:**
344
+ * `--api-key API_KEY` - API key for the destination environment. Falls back to `COMET_API_KEY` if not provided.
345
+ * `--url URL` - Base URL of the destination Comet environment (e.g. `https://comet.example.com`). Required for self-hosted instances when the API key does not encode the server URL.
346
+ * `--source-api-key SOURCE_API_KEY` - API key for the source environment. Used to fetch the chargeback report. Required unless `--chargeback-report` is given.
347
+ * `--source-url SOURCE_URL` - Base URL of the source Comet environment. Required for self-hosted source instances when the source API key does not encode the server URL.
348
+ * `--chargeback-report PATH` - Path to a local chargeback report JSON file. When provided, `--source-api-key` is not required.
349
+
350
+ ### Flags
351
+
352
+ * `--create-workspaces` - Create workspaces on the destination if they don't already exist (default: off)
353
+ * `--dry-run` - Print what would happen without making any changes
354
+ * `--failures-output PATH` - Path to write failed operations JSON (default: `bulk_add_failures_by_email.json`)
355
+
356
+ ### Examples
357
+
358
+ ```bash
359
+ # Dry run — preview what would happen
360
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY --dry-run
361
+
362
+ # Execute the migration
363
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY
364
+
365
+ # Use a local chargeback report file
366
+ cometx migrate-users --api-key DEST_KEY --chargeback-report /tmp/chargeback_reports.json
367
+
368
+ # Self-hosted environments — provide explicit URLs
369
+ cometx migrate-users --api-key DEST_KEY --url https://comet.dest.example.com \
370
+ --source-api-key SOURCE_KEY --source-url https://comet.src.example.com
371
+ ```
372
+
373
+ For more information, `cometx migrate-users --help`
374
+
332
375
  ## cometx rename-duplicates
333
376
 
334
377
  This command is used to rename duplicate experiments within projects. When multiple experiments share the same name in a project, this command renames the duplicates to NAME-1, NAME-2, etc. while avoiding conflicts with existing names.
@@ -479,12 +522,8 @@ cometx admin chargeback-report [YEAR-MONTH]
479
522
  **Examples:**
480
523
  ```
481
524
  cometx admin chargeback-report
482
- <<<<<<< Updated upstream
483
- cometx admin usage-report
484
- =======
485
525
  cometx admin chargeback-report 2024-09
486
526
  ```
487
- >>>>>>> Stashed changes
488
527
 
489
528
  #### usage-report
490
529
 
@@ -12,5 +12,5 @@
12
12
  # the express permission of Comet ML Inc.
13
13
  # *******************************************************
14
14
 
15
- version_info = (3, 6, 4)
15
+ version_info = (3, 6, 6)
16
16
  __version__ = ".".join(map(str, version_info))
@@ -171,7 +171,13 @@ class API(API):
171
171
  else:
172
172
  raise Exception("I don't know what to do with %r" % parsed_url.path)
173
173
 
174
- def upload_panel_code(self, workspace: str, panel_name: str, code: str) -> None:
174
+ def upload_panel_code(
175
+ self,
176
+ workspace: str,
177
+ panel_name: str,
178
+ code: str,
179
+ template_id: str = None,
180
+ ) -> Dict[str, Any]:
175
181
  """
176
182
  Upload Python code as a panel in a workspace.
177
183
 
@@ -179,6 +185,8 @@ class API(API):
179
185
  workspace (str): the workspace to place the panel into
180
186
  panel_name (str): the name of the panel
181
187
  code (str): the code to turn into a panel
188
+ template_id (str, optional): if provided, overwrites the existing
189
+ panel with this ID instead of creating a new one
182
190
 
183
191
  Example:
184
192
  ```python linenums="1"
@@ -194,15 +202,19 @@ class API(API):
194
202
  ```
195
203
  """
196
204
  filename = create_panel_zip(panel_name, code)
197
- self.upload_panel_zip(workspace, filename)
205
+ return self.upload_panel_zip(workspace, filename, template_id=template_id)
198
206
 
199
- def upload_panel_zip(self, workspace: str, filename: str) -> Dict[str, str]:
207
+ def upload_panel_zip(
208
+ self, workspace: str, filename: str, template_id: str = None
209
+ ) -> Dict[str, str]:
200
210
  """
201
211
  Upload a panel zip file to a workspace.
202
212
 
203
213
  Args:
204
214
  workspace (str): the workspace to place the panel into
205
215
  filename (str): the name of the panel zip to upload
216
+ template_id (str, optional): if provided, overwrites the existing
217
+ panel with this ID instead of creating a new one
206
218
 
207
219
  Returns: dictionary of results
208
220
 
@@ -215,6 +227,8 @@ class API(API):
215
227
  ```
216
228
  """
217
229
  params = {"teamName": workspace}
230
+ if template_id is not None:
231
+ params["templateId"] = template_id
218
232
  payload = {}
219
233
  with open(filename, "rb") as fp:
220
234
  files = {"file": (filename, fp)}
@@ -226,6 +240,120 @@ class API(API):
226
240
  )
227
241
  return results.json()
228
242
 
243
+ def _get_project_id(self, workspace: str, project_name: str) -> str:
244
+ projects = self._client.get_from_endpoint(
245
+ "projects", params={"workspaceName": workspace}
246
+ )
247
+ project = next(
248
+ (p for p in projects["projects"] if p["projectName"] == project_name),
249
+ None,
250
+ )
251
+ if project is None:
252
+ raise Exception(
253
+ f"Project {project_name!r} not found in workspace {workspace!r}"
254
+ )
255
+ return project["projectId"]
256
+
257
+ def create_dashboard(
258
+ self,
259
+ workspace: str,
260
+ project_name: str,
261
+ template_name: str,
262
+ template_id: str = None,
263
+ panels: List[str] = None,
264
+ **kwargs,
265
+ ) -> Dict[str, Any]:
266
+ """
267
+ Create a new dashboard in a project.
268
+
269
+ Args:
270
+ workspace (str): the workspace name the project belongs to
271
+ project_name (str): the project name to create the dashboard in
272
+ template_name (str): name for the new dashboard
273
+ template_id (str, optional): source template ID to clone code
274
+ panel associations from an existing dashboard
275
+ panels (list, optional): list of panel template IDs to include
276
+ in the new dashboard
277
+ **kwargs: additional DashboardTemplate fields
278
+
279
+ Returns: dictionary representing the created DashboardTemplate
280
+
281
+ Example:
282
+ ```python linenums="1"
283
+ from cometx import API
284
+
285
+ api = API()
286
+ dashboard = api.create_dashboard(
287
+ workspace="my-workspace",
288
+ project_name="my-project",
289
+ template_name="My Dashboard",
290
+ panels=["panel-template-id-1", "panel-template-id-2"],
291
+ )
292
+ ```
293
+ """
294
+ project_id = self._get_project_id(workspace, project_name)
295
+
296
+ body = {"project_id": project_id, "template_name": template_name, **kwargs}
297
+ if template_id is not None:
298
+ body["template_id"] = template_id
299
+ if panels is not None:
300
+ body["codePanelTemplateIds"] = panels
301
+ results = self._client.post_from_endpoint(
302
+ "write/dashboard-template/create",
303
+ payload=body,
304
+ )
305
+ return results.json()
306
+
307
+ def update_dashboard(
308
+ self,
309
+ workspace: str,
310
+ project_name: str,
311
+ template_id: str,
312
+ template_name: str = None,
313
+ panels: List[str] = None,
314
+ **kwargs,
315
+ ) -> Dict[str, Any]:
316
+ """
317
+ Update an existing dashboard in a project.
318
+
319
+ Args:
320
+ workspace (str): the workspace name the project belongs to
321
+ project_name (str): the project name the dashboard belongs to
322
+ template_id (str): the ID of the dashboard to update
323
+ template_name (str, optional): new name for the dashboard
324
+ panels (list, optional): list of panel template IDs to set on
325
+ the dashboard
326
+ **kwargs: additional DashboardTemplate fields
327
+
328
+ Returns: dictionary representing the updated DashboardTemplate
329
+
330
+ Example:
331
+ ```python linenums="1"
332
+ from cometx import API
333
+
334
+ api = API()
335
+ dashboard = api.update_dashboard(
336
+ workspace="my-workspace",
337
+ project_name="my-project",
338
+ template_id="existing-dashboard-id",
339
+ template_name="Renamed Dashboard",
340
+ panels=["panel-template-id-1", "panel-template-id-2"],
341
+ )
342
+ ```
343
+ """
344
+ project_id = self._get_project_id(workspace, project_name)
345
+
346
+ body = {"project_id": project_id, "template_id": template_id, **kwargs}
347
+ if template_name is not None:
348
+ body["template_name"] = template_name
349
+ if panels is not None:
350
+ body["codePanelTemplateIds"] = panels
351
+ results = self._client.post_from_endpoint(
352
+ "dashboard-templates/project/upsert",
353
+ payload=body,
354
+ )
355
+ return results.json()
356
+
229
357
  def log_pr_curves(
230
358
  self, experiment, y_true, y_predicted, labels=None, overwrite=False, step=None
231
359
  ):
@@ -26,6 +26,7 @@ Examples:
26
26
  cometx smoke-test
27
27
  cometx update
28
28
  cometx admin
29
+ cometx migrate-users
29
30
 
30
31
  For more information:
31
32
  cometx COMMAND --help
@@ -47,6 +48,7 @@ from . import (
47
48
  download,
48
49
  list_command,
49
50
  log,
51
+ migrate_users,
50
52
  rename_duplicates,
51
53
  reproduce,
52
54
  smoke_test,
@@ -108,6 +110,7 @@ def main(raw_args=sys.argv[1:]):
108
110
  add_subparser(subparsers, config, "config")
109
111
  add_subparser(subparsers, rename_duplicates, "rename-duplicates")
110
112
  add_subparser(subparsers, smoke_test, "smoke-test")
113
+ add_subparser(subparsers, migrate_users, "migrate-users")
111
114
  add_subparser(
112
115
  subparsers, upload_optimizer_experiments, "upload-optimizer-experiments"
113
116
  )
@@ -78,6 +78,8 @@ import threading
78
78
  import time
79
79
  import urllib.parse
80
80
  import zipfile
81
+
82
+ import requests
81
83
  from datetime import datetime, timedelta
82
84
 
83
85
  from comet_ml import APIExperiment, Artifact, Experiment, OfflineExperiment
@@ -102,6 +104,7 @@ from ..api import API
102
104
  from ..framework.comet.download_manager import sanitize_filename
103
105
  from ..utils import remove_extra_slashes
104
106
  from .copy_utils import upload_single_offline_experiment
107
+ from .migrate_users import _create_workspace
105
108
 
106
109
  ADDITIONAL_ARGS = False
107
110
 
@@ -197,6 +200,12 @@ def get_parser_arguments(parser):
197
200
  type=str,
198
201
  default=None,
199
202
  )
203
+ parser.add_argument(
204
+ "--create-workspaces",
205
+ help="Attempt to create the destination workspace if it does not exist",
206
+ default=False,
207
+ action="store_true",
208
+ )
200
209
 
201
210
 
202
211
  def copy(parsed_args, remaining=None):
@@ -214,6 +223,7 @@ def copy(parsed_args, remaining=None):
214
223
  parsed_args.ignore,
215
224
  parsed_args.debug,
216
225
  parsed_args.sync,
226
+ parsed_args.create_workspaces,
217
227
  )
218
228
 
219
229
  # Wait for all uploads to complete
@@ -790,7 +800,7 @@ class CopyManager:
790
800
  # Restore default signal handler
791
801
  signal.signal(signal.SIGINT, signal.default_int_handler)
792
802
 
793
- def copy(self, source, destination, symlink, ignore, debug, sync):
803
+ def copy(self, source, destination, symlink, ignore, debug, sync, create_workspaces=False):
794
804
  """ """
795
805
  self.ignore = ignore
796
806
  self.debug = debug
@@ -823,9 +833,35 @@ class CopyManager:
823
833
  # First check to make sure workspace_dst exists:
824
834
  workspaces = self.api.get_workspaces()
825
835
  if workspace_dst not in workspaces:
826
- raise Exception(
827
- f"{workspace_dst} does not exist; use the Comet UI to create it"
828
- )
836
+ if create_workspaces:
837
+ print(
838
+ f"Workspace {workspace_dst!r} does not exist, attempting to create it..."
839
+ )
840
+ dest_url = self.api._get_url_server()
841
+ headers = {
842
+ "Authorization": self.api.api_key,
843
+ "Content-Type": "application/json",
844
+ }
845
+ try:
846
+ _create_workspace(dest_url, headers, workspace_dst)
847
+ print(f"Workspace {workspace_dst!r} created successfully.")
848
+ except requests.exceptions.HTTPError as exc:
849
+ raise Exception(
850
+ f"Workspace {workspace_dst!r} does not exist and could not be "
851
+ f"created automatically (HTTP {exc.response.status_code}). "
852
+ f"Please create it via the Comet UI and try again."
853
+ ) from exc
854
+ except requests.exceptions.RequestException as exc:
855
+ raise Exception(
856
+ f"Workspace {workspace_dst!r} does not exist and could not be "
857
+ f"created automatically: {exc}. "
858
+ f"Please create it via the Comet UI and try again."
859
+ ) from exc
860
+ else:
861
+ raise Exception(
862
+ f"{workspace_dst} does not exist; use --create-workspaces to "
863
+ f"create it automatically, or create it via the Comet UI"
864
+ )
829
865
 
830
866
  if project_src == "panels":
831
867
  # experiment_src may be "*" or filename
@@ -0,0 +1,393 @@
1
+ #!/usr/bin/env python
2
+ # -*- coding: utf-8 -*-
3
+ # ****************************************
4
+ # __
5
+ # _________ ____ ___ ___ / /__ __
6
+ # / ___/ __ \/ __ `__ \/ _ \/ __/ |/_/
7
+ # / /__/ /_/ / / / / / / __/ /__> <
8
+ # \___/\____/_/ /_/ /_/\___/\__/_/|_|
9
+ #
10
+ #
11
+ # Copyright (c) 2022 Cometx Development
12
+ # Team. All rights reserved.
13
+ # ****************************************
14
+ """
15
+ Migrate users into workspaces from a source environment to a destination
16
+ environment based on a chargeback report.
17
+
18
+ The chargeback report is fetched automatically from the source environment's
19
+ admin API, or you can provide a local JSON file with --chargeback-report.
20
+
21
+ Examples:
22
+
23
+ $ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY --dry-run
24
+ $ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY
25
+ $ cometx migrate-users --api-key DEST_KEY --chargeback-report /path/to/report.json
26
+ """
27
+
28
+ import argparse
29
+ import base64
30
+ import json
31
+ import os
32
+ import sys
33
+ import urllib.parse
34
+
35
+ import requests
36
+
37
+ ADDITIONAL_ARGS = False
38
+
39
+
40
+ def get_parser_arguments(parser):
41
+ parser.add_argument(
42
+ "--api-key",
43
+ help="API key for the destination environment (used to add workspace members). "
44
+ "Falls back to COMET_API_KEY environment variable if not provided.",
45
+ type=str,
46
+ default=None,
47
+ )
48
+ parser.add_argument(
49
+ "--url",
50
+ help="Base URL of the destination Comet environment (e.g. https://comet.example.com). "
51
+ "Required for self-hosted instances when the API key does not encode the server URL.",
52
+ type=str,
53
+ default=None,
54
+ )
55
+ parser.add_argument(
56
+ "--source-api-key",
57
+ help="API key for the source environment (used to fetch the chargeback report). "
58
+ "Required unless --chargeback-report is given.",
59
+ type=str,
60
+ default=None,
61
+ )
62
+ parser.add_argument(
63
+ "--source-url",
64
+ help="Base URL of the source Comet environment. "
65
+ "Required for self-hosted instances when the source API key does not encode the server URL.",
66
+ type=str,
67
+ default=None,
68
+ )
69
+ parser.add_argument(
70
+ "--chargeback-report",
71
+ help="Path to a local chargeback report JSON file (skips fetching from source environment)",
72
+ type=str,
73
+ default=None,
74
+ )
75
+ parser.add_argument(
76
+ "--create-workspaces",
77
+ help="Create workspaces on the destination environment if they don't already exist",
78
+ default=False,
79
+ action="store_true",
80
+ )
81
+ parser.add_argument(
82
+ "--dry-run",
83
+ help="Print what would happen without making any changes",
84
+ default=False,
85
+ action="store_true",
86
+ )
87
+ parser.add_argument(
88
+ "--failures-output",
89
+ help="Path to write failed operations JSON (default: bulk_add_failures_by_email.json)",
90
+ type=str,
91
+ default="bulk_add_failures_by_email.json",
92
+ )
93
+
94
+
95
+ def _resolve_server_url(api_key, explicit_url=None):
96
+ """Return the server base URL.
97
+
98
+ Priority:
99
+ 1. ``explicit_url`` (from --url / --source-url), if provided.
100
+ 2. URL encoded inside the API key (new-style keys contain ``*<base64>``).
101
+ 3. Error — no silent fallback to cloud URL.
102
+ """
103
+ if explicit_url:
104
+ parsed = urllib.parse.urlparse(explicit_url)
105
+ if parsed.scheme != "https":
106
+ print("[ERROR] --url/--source-url must use https://.")
107
+ sys.exit(1)
108
+ return explicit_url.rstrip("/")
109
+
110
+ if "*" in api_key:
111
+ try:
112
+ _, encoded = api_key.split("*", 1)
113
+ # base64 padding may be missing
114
+ padding = (4 - len(encoded) % 4) % 4
115
+ payload = json.loads(base64.b64decode(encoded + "=" * padding))
116
+ return payload["baseUrl"].rstrip("/")
117
+ except Exception:
118
+ pass # Fall through to error below
119
+
120
+ print(
121
+ "[ERROR] Cannot determine the server URL from the API key. "
122
+ "Pass --url (destination) or --source-url (source) explicitly, "
123
+ "e.g. --url https://comet.example.com"
124
+ )
125
+ sys.exit(1)
126
+
127
+
128
+ def _fetch_chargeback_report(server_url, source_api_key):
129
+ url = f"{server_url}/api/admin/chargeback/report"
130
+ print(f"Fetching chargeback report from {url}...")
131
+ resp = requests.get(
132
+ url,
133
+ headers={"Authorization": source_api_key},
134
+ timeout=30,
135
+ )
136
+ resp.raise_for_status()
137
+ return resp.json()
138
+
139
+
140
+ def _load_chargeback_report(path):
141
+ print(f"Loading chargeback report from {path}...")
142
+ with open(path, "r") as f:
143
+ return json.load(f)
144
+
145
+
146
+ def _get_existing_workspaces(dest_url, headers):
147
+ url = f"{dest_url}/api/rest/v2/workspaces"
148
+ resp = requests.get(url, headers=headers, timeout=15)
149
+ resp.raise_for_status()
150
+ data = resp.json()
151
+ # Handle {"workspaceNames": [...]} dict shape
152
+ if isinstance(data, dict):
153
+ names = data.get("workspaceNames", [])
154
+ if names and isinstance(names[0], dict):
155
+ return {ws["name"] for ws in names}
156
+ return set(names)
157
+ # Handle list of dicts with a "name" key
158
+ if data and isinstance(data[0], dict):
159
+ return {ws["name"] for ws in data}
160
+ # Handle flat list of strings
161
+ return set(data)
162
+
163
+
164
+ def _create_workspace(dest_url, headers, workspace_name):
165
+ url = f"{dest_url}/api/rest/v2/write/workspace/new"
166
+ resp = requests.post(
167
+ url,
168
+ headers=headers,
169
+ json={"name": workspace_name},
170
+ timeout=15,
171
+ )
172
+ resp.raise_for_status()
173
+
174
+
175
+ def _add_member(url, headers, email, workspace_name):
176
+ """Attempt to add a single member. Returns (status, error_info_or_None)."""
177
+ payload = {
178
+ "userEmail": email,
179
+ "workspaceName": workspace_name,
180
+ "admin": True,
181
+ }
182
+ try:
183
+ response = requests.post(
184
+ url,
185
+ headers=headers,
186
+ json=payload,
187
+ timeout=15,
188
+ )
189
+
190
+ if response.status_code == 200:
191
+ return "added", None
192
+
193
+ try:
194
+ error_msg = response.json()
195
+ except ValueError:
196
+ error_msg = response.text
197
+
198
+ is_already_member = (
199
+ isinstance(error_msg, dict)
200
+ and "already member of" in error_msg.get("msg", "")
201
+ )
202
+ if is_already_member:
203
+ return "already_member", None
204
+
205
+ return "failed", {
206
+ "status": response.status_code,
207
+ "error": error_msg,
208
+ }
209
+
210
+ except requests.exceptions.RequestException as e:
211
+ return "failed", {
212
+ "status": "exception",
213
+ "error": str(e),
214
+ }
215
+
216
+
217
+ def migrate_users(parsed_args):
218
+ api_key = parsed_args.api_key or os.environ.get("COMET_API_KEY")
219
+ if not api_key:
220
+ print("[ERROR] No API key found. Set COMET_API_KEY or pass --api-key.")
221
+ sys.exit(1)
222
+
223
+ source_api_key = parsed_args.source_api_key
224
+ create_workspaces = parsed_args.create_workspaces
225
+ dry_run = parsed_args.dry_run
226
+ failures_output = parsed_args.failures_output
227
+
228
+ if not parsed_args.chargeback_report and not source_api_key:
229
+ print(
230
+ "[ERROR] --source-api-key is required when --chargeback-report is not provided."
231
+ )
232
+ sys.exit(1)
233
+
234
+ dest_url = _resolve_server_url(api_key, parsed_args.url)
235
+ print(f"Destination URL: {dest_url}")
236
+
237
+ if parsed_args.chargeback_report:
238
+ data = _load_chargeback_report(parsed_args.chargeback_report)
239
+ else:
240
+ source_url = _resolve_server_url(source_api_key, parsed_args.source_url)
241
+ print(f"Source URL: {source_url}")
242
+ if source_url == dest_url and source_api_key == api_key:
243
+ print(
244
+ "[WARNING] Source and destination URL and API key are identical. "
245
+ "Are you sure you want to migrate users to the same environment?"
246
+ )
247
+ try:
248
+ data = _fetch_chargeback_report(source_url, source_api_key)
249
+ except requests.exceptions.RequestException as e:
250
+ print(f"[ERROR] Failed to fetch chargeback report: {e}")
251
+ sys.exit(1)
252
+
253
+ workspaces = data.get("workspaces", [])
254
+ print(f"Found {len(workspaces)} workspaces\n")
255
+
256
+ headers = {
257
+ "Authorization": api_key,
258
+ "Content-Type": "application/json",
259
+ }
260
+
261
+ if create_workspaces and not dry_run:
262
+ print("Checking existing workspaces on destination...")
263
+ try:
264
+ existing = _get_existing_workspaces(dest_url, headers)
265
+ except requests.exceptions.RequestException as e:
266
+ print(f"[ERROR] Failed to list destination workspaces: {e}")
267
+ sys.exit(1)
268
+
269
+ ws_created = 0
270
+ for workspace in workspaces:
271
+ ws_name = workspace["name"]
272
+ if ws_name not in existing:
273
+ try:
274
+ _create_workspace(dest_url, headers, ws_name)
275
+ print(f" [CREATED] Workspace '{ws_name}'")
276
+ ws_created += 1
277
+ except requests.exceptions.RequestException as e:
278
+ print(f" [ERROR] Failed to create workspace '{ws_name}': {e}")
279
+ print(f"Created {ws_created} new workspace(s)\n")
280
+ elif create_workspaces and dry_run:
281
+ print("[DRY RUN] Would check/create workspaces on destination\n")
282
+
283
+ add_member_url = f"{dest_url}/api/rest/v2/write/add-workspace-member"
284
+
285
+ total_added = 0
286
+ total_skipped = 0
287
+ total_no_email = 0
288
+ total_already_member = 0
289
+ total_failed = 0
290
+ failures = []
291
+
292
+ for workspace in workspaces:
293
+ ws_name = workspace["name"]
294
+ members = workspace.get("members", [])
295
+
296
+ if not members:
297
+ print(f"[SKIP] Workspace '{ws_name}' — no members, skipping")
298
+ total_skipped += 1
299
+ continue
300
+
301
+ print(f"\n{'=' * 60}")
302
+ print(f"Workspace: {ws_name} ({len(members)} members)")
303
+ print(f"{'=' * 60}")
304
+
305
+ ws_added = 0
306
+ ws_no_email = 0
307
+ ws_already_member = 0
308
+ ws_failed = 0
309
+
310
+ for member in members:
311
+ email = member.get("email")
312
+
313
+ if not email:
314
+ ws_no_email += 1
315
+ total_no_email += 1
316
+ print(f" [SKIP] '{member['userName']}' has no email, skipping")
317
+ continue
318
+
319
+ if dry_run:
320
+ print(f" [DRY RUN] Would add '{email}' to '{ws_name}'")
321
+ ws_added += 1
322
+ total_added += 1
323
+ continue
324
+
325
+ status, error_info = _add_member(add_member_url, headers, email, ws_name)
326
+
327
+ if status == "added":
328
+ ws_added += 1
329
+ total_added += 1
330
+ elif status == "already_member":
331
+ ws_already_member += 1
332
+ total_already_member += 1
333
+ else:
334
+ print(
335
+ f" [FAIL] '{email}' -> '{ws_name}' "
336
+ f"— {error_info['status']}: {error_info['error']}"
337
+ )
338
+ ws_failed += 1
339
+ total_failed += 1
340
+ failures.append(
341
+ {
342
+ "workspace": ws_name,
343
+ "email": email,
344
+ **error_info,
345
+ }
346
+ )
347
+
348
+ if dry_run:
349
+ print(f" [DRY RUN] Total: would add {ws_added}/{len(members)} users")
350
+ else:
351
+ parts = [f" Added {ws_added}/{len(members)} users successfully"]
352
+ if ws_already_member:
353
+ parts.append(f"{ws_already_member} already a member")
354
+ if ws_no_email:
355
+ parts.append(f"{ws_no_email} skipped (no email)")
356
+ if ws_failed:
357
+ parts.append(f"{ws_failed} failed")
358
+ print(
359
+ parts[0]
360
+ + (" (" + ", ".join(parts[1:]) + ")" if len(parts) > 1 else "")
361
+ )
362
+
363
+ print(f"\n{'=' * 60}")
364
+ print("SUMMARY")
365
+ print(f"{'=' * 60}")
366
+ print(f" Workspaces processed : {len(workspaces)}")
367
+ print(f" Workspaces skipped : {total_skipped} (no members)")
368
+ print(f" Users added : {total_added}")
369
+ print(f" Already a member : {total_already_member}")
370
+ print(f" Skipped (no email) : {total_no_email}")
371
+ print(f" Failures : {total_failed}")
372
+
373
+ if dry_run:
374
+ print(f"\n *** DRY RUN MODE — no API calls were made ***")
375
+ print(f" *** Remove --dry-run to execute for real ***")
376
+
377
+ if failures:
378
+ with open(failures_output, "w") as f:
379
+ json.dump(failures, f, indent=2)
380
+ print(f"\n Failed operations saved to {failures_output}")
381
+
382
+
383
+ def main(args):
384
+ parser = argparse.ArgumentParser(
385
+ description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
386
+ )
387
+ get_parser_arguments(parser)
388
+ parsed_args = parser.parse_args(args)
389
+ migrate_users(parsed_args)
390
+
391
+
392
+ if __name__ == "__main__":
393
+ main(sys.argv[1:])
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: cometx
3
- Version: 3.6.4
3
+ Version: 3.6.6
4
4
  Summary: Python tools for Comet
5
5
  Home-page: https://github.com/comet-ml/cometx/
6
6
  Author: cometx development team
@@ -102,6 +102,7 @@ for ML management.
102
102
  * [cometx download](#cometx-download)
103
103
  * [cometx list](#cometx-list)
104
104
  * [cometx log](#cometx-log)
105
+ * [cometx migrate-users](#cometx-migrate-users)
105
106
  * [cometx rename-duplicates](#cometx-rename-duplicates)
106
107
  * [cometx reproduce](#cometx-reproduce)
107
108
  * [cometx smoke-test](#cometx-smoke-test)
@@ -324,6 +325,7 @@ Not all combinations are possible:
324
325
  * `--symlink` - Instead of copying, create a link to an experiment in a project
325
326
  * `--sync` - Check to see if experiment name has been created first; if so, skip
326
327
  * `--path PATH` - Path to prepend to workspace_src when accessing files (supports ~ for home directory)
328
+ * `--create-workspaces` - Attempt to create the destination workspace if it does not exist (requires appropriate API permissions)
327
329
 
328
330
  ### Using --path
329
331
 
@@ -385,6 +387,47 @@ Where TYPE is one of the following names:
385
387
 
386
388
  For more information, `cometx log --help`
387
389
 
390
+ ## cometx migrate-users
391
+
392
+ This command migrates users into workspaces from a source Comet environment to a destination environment. It reads workspace membership from a chargeback report fetched from the source environment's admin API (or a pre-downloaded local file) and invites each user to the corresponding workspace in the destination environment by email. If a user with that email does not yet exist in the destination, they will be provisioned access once they sign up.
393
+
394
+ ```
395
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY [FLAGS ...]
396
+ cometx migrate-users --api-key DEST_KEY --chargeback-report /path/to/report.json [FLAGS ...]
397
+ ```
398
+
399
+ **Arguments:**
400
+ * `--api-key API_KEY` - API key for the destination environment. Falls back to `COMET_API_KEY` if not provided.
401
+ * `--url URL` - Base URL of the destination Comet environment (e.g. `https://comet.example.com`). Required for self-hosted instances when the API key does not encode the server URL.
402
+ * `--source-api-key SOURCE_API_KEY` - API key for the source environment. Used to fetch the chargeback report. Required unless `--chargeback-report` is given.
403
+ * `--source-url SOURCE_URL` - Base URL of the source Comet environment. Required for self-hosted source instances when the source API key does not encode the server URL.
404
+ * `--chargeback-report PATH` - Path to a local chargeback report JSON file. When provided, `--source-api-key` is not required.
405
+
406
+ ### Flags
407
+
408
+ * `--create-workspaces` - Create workspaces on the destination if they don't already exist (default: off)
409
+ * `--dry-run` - Print what would happen without making any changes
410
+ * `--failures-output PATH` - Path to write failed operations JSON (default: `bulk_add_failures_by_email.json`)
411
+
412
+ ### Examples
413
+
414
+ ```bash
415
+ # Dry run — preview what would happen
416
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY --dry-run
417
+
418
+ # Execute the migration
419
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY
420
+
421
+ # Use a local chargeback report file
422
+ cometx migrate-users --api-key DEST_KEY --chargeback-report /tmp/chargeback_reports.json
423
+
424
+ # Self-hosted environments — provide explicit URLs
425
+ cometx migrate-users --api-key DEST_KEY --url https://comet.dest.example.com \
426
+ --source-api-key SOURCE_KEY --source-url https://comet.src.example.com
427
+ ```
428
+
429
+ For more information, `cometx migrate-users --help`
430
+
388
431
  ## cometx rename-duplicates
389
432
 
390
433
  This command is used to rename duplicate experiments within projects. When multiple experiments share the same name in a project, this command renames the duplicates to NAME-1, NAME-2, etc. while avoiding conflicts with existing names.
@@ -535,12 +578,8 @@ cometx admin chargeback-report [YEAR-MONTH]
535
578
  **Examples:**
536
579
  ```
537
580
  cometx admin chargeback-report
538
- <<<<<<< Updated upstream
539
- cometx admin usage-report
540
- =======
541
581
  cometx admin chargeback-report 2024-09
542
582
  ```
543
- >>>>>>> Stashed changes
544
583
 
545
584
  #### usage-report
546
585
 
@@ -35,6 +35,7 @@ cometx/cli/delete_assets.py
35
35
  cometx/cli/download.py
36
36
  cometx/cli/list_command.py
37
37
  cometx/cli/log.py
38
+ cometx/cli/migrate_users.py
38
39
  cometx/cli/rename_duplicates.py
39
40
  cometx/cli/reproduce.py
40
41
  cometx/cli/smoke_test.py
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes