cometx 3.6.4__tar.gz → 3.6.5__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. {cometx-3.6.4 → cometx-3.6.5}/PKG-INFO +44 -5
  2. {cometx-3.6.4 → cometx-3.6.5}/README.md +43 -4
  3. {cometx-3.6.4 → cometx-3.6.5}/cometx/_version.py +1 -1
  4. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/__init__.py +3 -0
  5. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/copy.py +40 -4
  6. cometx-3.6.5/cometx/cli/migrate_users.py +393 -0
  7. {cometx-3.6.4 → cometx-3.6.5}/cometx.egg-info/PKG-INFO +44 -5
  8. {cometx-3.6.4 → cometx-3.6.5}/cometx.egg-info/SOURCES.txt +1 -0
  9. {cometx-3.6.4 → cometx-3.6.5}/LICENSE +0 -0
  10. {cometx-3.6.4 → cometx-3.6.5}/cometx/__init__.py +0 -0
  11. {cometx-3.6.4 → cometx-3.6.5}/cometx/_typing.py +0 -0
  12. {cometx-3.6.4 → cometx-3.6.5}/cometx/_ui.py +0 -0
  13. {cometx-3.6.4 → cometx-3.6.5}/cometx/api.py +0 -0
  14. {cometx-3.6.4 → cometx-3.6.5}/cometx/audio.py +0 -0
  15. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/admin.py +0 -0
  16. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/admin_app.py +0 -0
  17. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/admin_gpu_app.py +0 -0
  18. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/admin_gpu_report.py +0 -0
  19. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/admin_optimizer_report.py +0 -0
  20. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/admin_usage_report.py +0 -0
  21. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/admin_utils.py +0 -0
  22. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/config.py +0 -0
  23. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/copy_utils.py +0 -0
  24. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/count.py +0 -0
  25. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/delete_assets.py +0 -0
  26. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/download.py +0 -0
  27. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/list_command.py +0 -0
  28. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/log.py +0 -0
  29. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/rename_duplicates.py +0 -0
  30. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/reproduce.py +0 -0
  31. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/smoke_test.py +0 -0
  32. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/update.py +0 -0
  33. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/upload_optimizer_experiments.py +0 -0
  34. {cometx-3.6.4 → cometx-3.6.5}/cometx/cli/utils.py +0 -0
  35. {cometx-3.6.4 → cometx-3.6.5}/cometx/framework/__init__.py +0 -0
  36. {cometx-3.6.4 → cometx-3.6.5}/cometx/framework/comet/__init__.py +0 -0
  37. {cometx-3.6.4 → cometx-3.6.5}/cometx/framework/comet/download_manager.py +0 -0
  38. {cometx-3.6.4 → cometx-3.6.5}/cometx/framework/neptune.py +0 -0
  39. {cometx-3.6.4 → cometx-3.6.5}/cometx/framework/wandb.py +0 -0
  40. {cometx-3.6.4 → cometx-3.6.5}/cometx/generate_utils.py +0 -0
  41. {cometx-3.6.4 → cometx-3.6.5}/cometx/magics.py +0 -0
  42. {cometx-3.6.4 → cometx-3.6.5}/cometx/optimizer.py +0 -0
  43. {cometx-3.6.4 → cometx-3.6.5}/cometx/panel_utils.py +0 -0
  44. {cometx-3.6.4 → cometx-3.6.5}/cometx/tools/__init__.py +0 -0
  45. {cometx-3.6.4 → cometx-3.6.5}/cometx/tools/dataset.py +0 -0
  46. {cometx-3.6.4 → cometx-3.6.5}/cometx/tools/pointcloud.py +0 -0
  47. {cometx-3.6.4 → cometx-3.6.5}/cometx/utils.py +0 -0
  48. {cometx-3.6.4 → cometx-3.6.5}/cometx.egg-info/dependency_links.txt +0 -0
  49. {cometx-3.6.4 → cometx-3.6.5}/cometx.egg-info/entry_points.txt +0 -0
  50. {cometx-3.6.4 → cometx-3.6.5}/cometx.egg-info/requires.txt +0 -0
  51. {cometx-3.6.4 → cometx-3.6.5}/cometx.egg-info/top_level.txt +0 -0
  52. {cometx-3.6.4 → cometx-3.6.5}/pyproject.toml +0 -0
  53. {cometx-3.6.4 → cometx-3.6.5}/setup.cfg +0 -0
  54. {cometx-3.6.4 → cometx-3.6.5}/setup.py +0 -0
  55. {cometx-3.6.4 → cometx-3.6.5}/tests/testlib.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: cometx
3
- Version: 3.6.4
3
+ Version: 3.6.5
4
4
  Summary: Python tools for Comet
5
5
  Home-page: https://github.com/comet-ml/cometx/
6
6
  Author: cometx development team
@@ -102,6 +102,7 @@ for ML management.
102
102
  * [cometx download](#cometx-download)
103
103
  * [cometx list](#cometx-list)
104
104
  * [cometx log](#cometx-log)
105
+ * [cometx migrate-users](#cometx-migrate-users)
105
106
  * [cometx rename-duplicates](#cometx-rename-duplicates)
106
107
  * [cometx reproduce](#cometx-reproduce)
107
108
  * [cometx smoke-test](#cometx-smoke-test)
@@ -324,6 +325,7 @@ Not all combinations are possible:
324
325
  * `--symlink` - Instead of copying, create a link to an experiment in a project
325
326
  * `--sync` - Check to see if experiment name has been created first; if so, skip
326
327
  * `--path PATH` - Path to prepend to workspace_src when accessing files (supports ~ for home directory)
328
+ * `--create-workspaces` - Attempt to create the destination workspace if it does not exist (requires appropriate API permissions)
327
329
 
328
330
  ### Using --path
329
331
 
@@ -385,6 +387,47 @@ Where TYPE is one of the following names:
385
387
 
386
388
  For more information, `cometx log --help`
387
389
 
390
+ ## cometx migrate-users
391
+
392
+ This command migrates users into workspaces from a source Comet environment to a destination environment. It reads workspace membership from a chargeback report fetched from the source environment's admin API (or a pre-downloaded local file) and invites each user to the corresponding workspace in the destination environment by email. If a user with that email does not yet exist in the destination, they will be provisioned access once they sign up.
393
+
394
+ ```
395
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY [FLAGS ...]
396
+ cometx migrate-users --api-key DEST_KEY --chargeback-report /path/to/report.json [FLAGS ...]
397
+ ```
398
+
399
+ **Arguments:**
400
+ * `--api-key API_KEY` - API key for the destination environment. Falls back to `COMET_API_KEY` if not provided.
401
+ * `--url URL` - Base URL of the destination Comet environment (e.g. `https://comet.example.com`). Required for self-hosted instances when the API key does not encode the server URL.
402
+ * `--source-api-key SOURCE_API_KEY` - API key for the source environment. Used to fetch the chargeback report. Required unless `--chargeback-report` is given.
403
+ * `--source-url SOURCE_URL` - Base URL of the source Comet environment. Required for self-hosted source instances when the source API key does not encode the server URL.
404
+ * `--chargeback-report PATH` - Path to a local chargeback report JSON file. When provided, `--source-api-key` is not required.
405
+
406
+ ### Flags
407
+
408
+ * `--create-workspaces` - Create workspaces on the destination if they don't already exist (default: off)
409
+ * `--dry-run` - Print what would happen without making any changes
410
+ * `--failures-output PATH` - Path to write failed operations JSON (default: `bulk_add_failures_by_email.json`)
411
+
412
+ ### Examples
413
+
414
+ ```bash
415
+ # Dry run — preview what would happen
416
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY --dry-run
417
+
418
+ # Execute the migration
419
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY
420
+
421
+ # Use a local chargeback report file
422
+ cometx migrate-users --api-key DEST_KEY --chargeback-report /tmp/chargeback_reports.json
423
+
424
+ # Self-hosted environments — provide explicit URLs
425
+ cometx migrate-users --api-key DEST_KEY --url https://comet.dest.example.com \
426
+ --source-api-key SOURCE_KEY --source-url https://comet.src.example.com
427
+ ```
428
+
429
+ For more information, `cometx migrate-users --help`
430
+
388
431
  ## cometx rename-duplicates
389
432
 
390
433
  This command is used to rename duplicate experiments within projects. When multiple experiments share the same name in a project, this command renames the duplicates to NAME-1, NAME-2, etc. while avoiding conflicts with existing names.
@@ -535,12 +578,8 @@ cometx admin chargeback-report [YEAR-MONTH]
535
578
  **Examples:**
536
579
  ```
537
580
  cometx admin chargeback-report
538
- <<<<<<< Updated upstream
539
- cometx admin usage-report
540
- =======
541
581
  cometx admin chargeback-report 2024-09
542
582
  ```
543
- >>>>>>> Stashed changes
544
583
 
545
584
  #### usage-report
546
585
 
@@ -46,6 +46,7 @@ for ML management.
46
46
  * [cometx download](#cometx-download)
47
47
  * [cometx list](#cometx-list)
48
48
  * [cometx log](#cometx-log)
49
+ * [cometx migrate-users](#cometx-migrate-users)
49
50
  * [cometx rename-duplicates](#cometx-rename-duplicates)
50
51
  * [cometx reproduce](#cometx-reproduce)
51
52
  * [cometx smoke-test](#cometx-smoke-test)
@@ -268,6 +269,7 @@ Not all combinations are possible:
268
269
  * `--symlink` - Instead of copying, create a link to an experiment in a project
269
270
  * `--sync` - Check to see if experiment name has been created first; if so, skip
270
271
  * `--path PATH` - Path to prepend to workspace_src when accessing files (supports ~ for home directory)
272
+ * `--create-workspaces` - Attempt to create the destination workspace if it does not exist (requires appropriate API permissions)
271
273
 
272
274
  ### Using --path
273
275
 
@@ -329,6 +331,47 @@ Where TYPE is one of the following names:
329
331
 
330
332
  For more information, `cometx log --help`
331
333
 
334
+ ## cometx migrate-users
335
+
336
+ This command migrates users into workspaces from a source Comet environment to a destination environment. It reads workspace membership from a chargeback report fetched from the source environment's admin API (or a pre-downloaded local file) and invites each user to the corresponding workspace in the destination environment by email. If a user with that email does not yet exist in the destination, they will be provisioned access once they sign up.
337
+
338
+ ```
339
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY [FLAGS ...]
340
+ cometx migrate-users --api-key DEST_KEY --chargeback-report /path/to/report.json [FLAGS ...]
341
+ ```
342
+
343
+ **Arguments:**
344
+ * `--api-key API_KEY` - API key for the destination environment. Falls back to `COMET_API_KEY` if not provided.
345
+ * `--url URL` - Base URL of the destination Comet environment (e.g. `https://comet.example.com`). Required for self-hosted instances when the API key does not encode the server URL.
346
+ * `--source-api-key SOURCE_API_KEY` - API key for the source environment. Used to fetch the chargeback report. Required unless `--chargeback-report` is given.
347
+ * `--source-url SOURCE_URL` - Base URL of the source Comet environment. Required for self-hosted source instances when the source API key does not encode the server URL.
348
+ * `--chargeback-report PATH` - Path to a local chargeback report JSON file. When provided, `--source-api-key` is not required.
349
+
350
+ ### Flags
351
+
352
+ * `--create-workspaces` - Create workspaces on the destination if they don't already exist (default: off)
353
+ * `--dry-run` - Print what would happen without making any changes
354
+ * `--failures-output PATH` - Path to write failed operations JSON (default: `bulk_add_failures_by_email.json`)
355
+
356
+ ### Examples
357
+
358
+ ```bash
359
+ # Dry run — preview what would happen
360
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY --dry-run
361
+
362
+ # Execute the migration
363
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY
364
+
365
+ # Use a local chargeback report file
366
+ cometx migrate-users --api-key DEST_KEY --chargeback-report /tmp/chargeback_reports.json
367
+
368
+ # Self-hosted environments — provide explicit URLs
369
+ cometx migrate-users --api-key DEST_KEY --url https://comet.dest.example.com \
370
+ --source-api-key SOURCE_KEY --source-url https://comet.src.example.com
371
+ ```
372
+
373
+ For more information, `cometx migrate-users --help`
374
+
332
375
  ## cometx rename-duplicates
333
376
 
334
377
  This command is used to rename duplicate experiments within projects. When multiple experiments share the same name in a project, this command renames the duplicates to NAME-1, NAME-2, etc. while avoiding conflicts with existing names.
@@ -479,12 +522,8 @@ cometx admin chargeback-report [YEAR-MONTH]
479
522
  **Examples:**
480
523
  ```
481
524
  cometx admin chargeback-report
482
- <<<<<<< Updated upstream
483
- cometx admin usage-report
484
- =======
485
525
  cometx admin chargeback-report 2024-09
486
526
  ```
487
- >>>>>>> Stashed changes
488
527
 
489
528
  #### usage-report
490
529
 
@@ -12,5 +12,5 @@
12
12
  # the express permission of Comet ML Inc.
13
13
  # *******************************************************
14
14
 
15
- version_info = (3, 6, 4)
15
+ version_info = (3, 6, 5)
16
16
  __version__ = ".".join(map(str, version_info))
@@ -26,6 +26,7 @@ Examples:
26
26
  cometx smoke-test
27
27
  cometx update
28
28
  cometx admin
29
+ cometx migrate-users
29
30
 
30
31
  For more information:
31
32
  cometx COMMAND --help
@@ -47,6 +48,7 @@ from . import (
47
48
  download,
48
49
  list_command,
49
50
  log,
51
+ migrate_users,
50
52
  rename_duplicates,
51
53
  reproduce,
52
54
  smoke_test,
@@ -108,6 +110,7 @@ def main(raw_args=sys.argv[1:]):
108
110
  add_subparser(subparsers, config, "config")
109
111
  add_subparser(subparsers, rename_duplicates, "rename-duplicates")
110
112
  add_subparser(subparsers, smoke_test, "smoke-test")
113
+ add_subparser(subparsers, migrate_users, "migrate-users")
111
114
  add_subparser(
112
115
  subparsers, upload_optimizer_experiments, "upload-optimizer-experiments"
113
116
  )
@@ -78,6 +78,8 @@ import threading
78
78
  import time
79
79
  import urllib.parse
80
80
  import zipfile
81
+
82
+ import requests
81
83
  from datetime import datetime, timedelta
82
84
 
83
85
  from comet_ml import APIExperiment, Artifact, Experiment, OfflineExperiment
@@ -102,6 +104,7 @@ from ..api import API
102
104
  from ..framework.comet.download_manager import sanitize_filename
103
105
  from ..utils import remove_extra_slashes
104
106
  from .copy_utils import upload_single_offline_experiment
107
+ from .migrate_users import _create_workspace
105
108
 
106
109
  ADDITIONAL_ARGS = False
107
110
 
@@ -197,6 +200,12 @@ def get_parser_arguments(parser):
197
200
  type=str,
198
201
  default=None,
199
202
  )
203
+ parser.add_argument(
204
+ "--create-workspaces",
205
+ help="Attempt to create the destination workspace if it does not exist",
206
+ default=False,
207
+ action="store_true",
208
+ )
200
209
 
201
210
 
202
211
  def copy(parsed_args, remaining=None):
@@ -214,6 +223,7 @@ def copy(parsed_args, remaining=None):
214
223
  parsed_args.ignore,
215
224
  parsed_args.debug,
216
225
  parsed_args.sync,
226
+ parsed_args.create_workspaces,
217
227
  )
218
228
 
219
229
  # Wait for all uploads to complete
@@ -790,7 +800,7 @@ class CopyManager:
790
800
  # Restore default signal handler
791
801
  signal.signal(signal.SIGINT, signal.default_int_handler)
792
802
 
793
- def copy(self, source, destination, symlink, ignore, debug, sync):
803
+ def copy(self, source, destination, symlink, ignore, debug, sync, create_workspaces=False):
794
804
  """ """
795
805
  self.ignore = ignore
796
806
  self.debug = debug
@@ -823,9 +833,35 @@ class CopyManager:
823
833
  # First check to make sure workspace_dst exists:
824
834
  workspaces = self.api.get_workspaces()
825
835
  if workspace_dst not in workspaces:
826
- raise Exception(
827
- f"{workspace_dst} does not exist; use the Comet UI to create it"
828
- )
836
+ if create_workspaces:
837
+ print(
838
+ f"Workspace {workspace_dst!r} does not exist, attempting to create it..."
839
+ )
840
+ dest_url = self.api._get_url_server()
841
+ headers = {
842
+ "Authorization": self.api.api_key,
843
+ "Content-Type": "application/json",
844
+ }
845
+ try:
846
+ _create_workspace(dest_url, headers, workspace_dst)
847
+ print(f"Workspace {workspace_dst!r} created successfully.")
848
+ except requests.exceptions.HTTPError as exc:
849
+ raise Exception(
850
+ f"Workspace {workspace_dst!r} does not exist and could not be "
851
+ f"created automatically (HTTP {exc.response.status_code}). "
852
+ f"Please create it via the Comet UI and try again."
853
+ ) from exc
854
+ except requests.exceptions.RequestException as exc:
855
+ raise Exception(
856
+ f"Workspace {workspace_dst!r} does not exist and could not be "
857
+ f"created automatically: {exc}. "
858
+ f"Please create it via the Comet UI and try again."
859
+ ) from exc
860
+ else:
861
+ raise Exception(
862
+ f"{workspace_dst} does not exist; use --create-workspaces to "
863
+ f"create it automatically, or create it via the Comet UI"
864
+ )
829
865
 
830
866
  if project_src == "panels":
831
867
  # experiment_src may be "*" or filename
@@ -0,0 +1,393 @@
1
+ #!/usr/bin/env python
2
+ # -*- coding: utf-8 -*-
3
+ # ****************************************
4
+ # __
5
+ # _________ ____ ___ ___ / /__ __
6
+ # / ___/ __ \/ __ `__ \/ _ \/ __/ |/_/
7
+ # / /__/ /_/ / / / / / / __/ /__> <
8
+ # \___/\____/_/ /_/ /_/\___/\__/_/|_|
9
+ #
10
+ #
11
+ # Copyright (c) 2022 Cometx Development
12
+ # Team. All rights reserved.
13
+ # ****************************************
14
+ """
15
+ Migrate users into workspaces from a source environment to a destination
16
+ environment based on a chargeback report.
17
+
18
+ The chargeback report is fetched automatically from the source environment's
19
+ admin API, or you can provide a local JSON file with --chargeback-report.
20
+
21
+ Examples:
22
+
23
+ $ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY --dry-run
24
+ $ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY
25
+ $ cometx migrate-users --api-key DEST_KEY --chargeback-report /path/to/report.json
26
+ """
27
+
28
+ import argparse
29
+ import base64
30
+ import json
31
+ import os
32
+ import sys
33
+ import urllib.parse
34
+
35
+ import requests
36
+
37
+ ADDITIONAL_ARGS = False
38
+
39
+
40
+ def get_parser_arguments(parser):
41
+ parser.add_argument(
42
+ "--api-key",
43
+ help="API key for the destination environment (used to add workspace members). "
44
+ "Falls back to COMET_API_KEY environment variable if not provided.",
45
+ type=str,
46
+ default=None,
47
+ )
48
+ parser.add_argument(
49
+ "--url",
50
+ help="Base URL of the destination Comet environment (e.g. https://comet.example.com). "
51
+ "Required for self-hosted instances when the API key does not encode the server URL.",
52
+ type=str,
53
+ default=None,
54
+ )
55
+ parser.add_argument(
56
+ "--source-api-key",
57
+ help="API key for the source environment (used to fetch the chargeback report). "
58
+ "Required unless --chargeback-report is given.",
59
+ type=str,
60
+ default=None,
61
+ )
62
+ parser.add_argument(
63
+ "--source-url",
64
+ help="Base URL of the source Comet environment. "
65
+ "Required for self-hosted instances when the source API key does not encode the server URL.",
66
+ type=str,
67
+ default=None,
68
+ )
69
+ parser.add_argument(
70
+ "--chargeback-report",
71
+ help="Path to a local chargeback report JSON file (skips fetching from source environment)",
72
+ type=str,
73
+ default=None,
74
+ )
75
+ parser.add_argument(
76
+ "--create-workspaces",
77
+ help="Create workspaces on the destination environment if they don't already exist",
78
+ default=False,
79
+ action="store_true",
80
+ )
81
+ parser.add_argument(
82
+ "--dry-run",
83
+ help="Print what would happen without making any changes",
84
+ default=False,
85
+ action="store_true",
86
+ )
87
+ parser.add_argument(
88
+ "--failures-output",
89
+ help="Path to write failed operations JSON (default: bulk_add_failures_by_email.json)",
90
+ type=str,
91
+ default="bulk_add_failures_by_email.json",
92
+ )
93
+
94
+
95
+ def _resolve_server_url(api_key, explicit_url=None):
96
+ """Return the server base URL.
97
+
98
+ Priority:
99
+ 1. ``explicit_url`` (from --url / --source-url), if provided.
100
+ 2. URL encoded inside the API key (new-style keys contain ``*<base64>``).
101
+ 3. Error — no silent fallback to cloud URL.
102
+ """
103
+ if explicit_url:
104
+ parsed = urllib.parse.urlparse(explicit_url)
105
+ if parsed.scheme != "https":
106
+ print("[ERROR] --url/--source-url must use https://.")
107
+ sys.exit(1)
108
+ return explicit_url.rstrip("/")
109
+
110
+ if "*" in api_key:
111
+ try:
112
+ _, encoded = api_key.split("*", 1)
113
+ # base64 padding may be missing
114
+ padding = (4 - len(encoded) % 4) % 4
115
+ payload = json.loads(base64.b64decode(encoded + "=" * padding))
116
+ return payload["baseUrl"].rstrip("/")
117
+ except Exception:
118
+ pass # Fall through to error below
119
+
120
+ print(
121
+ "[ERROR] Cannot determine the server URL from the API key. "
122
+ "Pass --url (destination) or --source-url (source) explicitly, "
123
+ "e.g. --url https://comet.example.com"
124
+ )
125
+ sys.exit(1)
126
+
127
+
128
+ def _fetch_chargeback_report(server_url, source_api_key):
129
+ url = f"{server_url}/api/admin/chargeback/report"
130
+ print(f"Fetching chargeback report from {url}...")
131
+ resp = requests.get(
132
+ url,
133
+ headers={"Authorization": source_api_key},
134
+ timeout=30,
135
+ )
136
+ resp.raise_for_status()
137
+ return resp.json()
138
+
139
+
140
+ def _load_chargeback_report(path):
141
+ print(f"Loading chargeback report from {path}...")
142
+ with open(path, "r") as f:
143
+ return json.load(f)
144
+
145
+
146
+ def _get_existing_workspaces(dest_url, headers):
147
+ url = f"{dest_url}/api/rest/v2/workspaces"
148
+ resp = requests.get(url, headers=headers, timeout=15)
149
+ resp.raise_for_status()
150
+ data = resp.json()
151
+ # Handle {"workspaceNames": [...]} dict shape
152
+ if isinstance(data, dict):
153
+ names = data.get("workspaceNames", [])
154
+ if names and isinstance(names[0], dict):
155
+ return {ws["name"] for ws in names}
156
+ return set(names)
157
+ # Handle list of dicts with a "name" key
158
+ if data and isinstance(data[0], dict):
159
+ return {ws["name"] for ws in data}
160
+ # Handle flat list of strings
161
+ return set(data)
162
+
163
+
164
+ def _create_workspace(dest_url, headers, workspace_name):
165
+ url = f"{dest_url}/api/rest/v2/write/workspace/new"
166
+ resp = requests.post(
167
+ url,
168
+ headers=headers,
169
+ json={"name": workspace_name},
170
+ timeout=15,
171
+ )
172
+ resp.raise_for_status()
173
+
174
+
175
+ def _add_member(url, headers, email, workspace_name):
176
+ """Attempt to add a single member. Returns (status, error_info_or_None)."""
177
+ payload = {
178
+ "userEmail": email,
179
+ "workspaceName": workspace_name,
180
+ "admin": True,
181
+ }
182
+ try:
183
+ response = requests.post(
184
+ url,
185
+ headers=headers,
186
+ json=payload,
187
+ timeout=15,
188
+ )
189
+
190
+ if response.status_code == 200:
191
+ return "added", None
192
+
193
+ try:
194
+ error_msg = response.json()
195
+ except ValueError:
196
+ error_msg = response.text
197
+
198
+ is_already_member = (
199
+ isinstance(error_msg, dict)
200
+ and "already member of" in error_msg.get("msg", "")
201
+ )
202
+ if is_already_member:
203
+ return "already_member", None
204
+
205
+ return "failed", {
206
+ "status": response.status_code,
207
+ "error": error_msg,
208
+ }
209
+
210
+ except requests.exceptions.RequestException as e:
211
+ return "failed", {
212
+ "status": "exception",
213
+ "error": str(e),
214
+ }
215
+
216
+
217
+ def migrate_users(parsed_args):
218
+ api_key = parsed_args.api_key or os.environ.get("COMET_API_KEY")
219
+ if not api_key:
220
+ print("[ERROR] No API key found. Set COMET_API_KEY or pass --api-key.")
221
+ sys.exit(1)
222
+
223
+ source_api_key = parsed_args.source_api_key
224
+ create_workspaces = parsed_args.create_workspaces
225
+ dry_run = parsed_args.dry_run
226
+ failures_output = parsed_args.failures_output
227
+
228
+ if not parsed_args.chargeback_report and not source_api_key:
229
+ print(
230
+ "[ERROR] --source-api-key is required when --chargeback-report is not provided."
231
+ )
232
+ sys.exit(1)
233
+
234
+ dest_url = _resolve_server_url(api_key, parsed_args.url)
235
+ print(f"Destination URL: {dest_url}")
236
+
237
+ if parsed_args.chargeback_report:
238
+ data = _load_chargeback_report(parsed_args.chargeback_report)
239
+ else:
240
+ source_url = _resolve_server_url(source_api_key, parsed_args.source_url)
241
+ print(f"Source URL: {source_url}")
242
+ if source_url == dest_url and source_api_key == api_key:
243
+ print(
244
+ "[WARNING] Source and destination URL and API key are identical. "
245
+ "Are you sure you want to migrate users to the same environment?"
246
+ )
247
+ try:
248
+ data = _fetch_chargeback_report(source_url, source_api_key)
249
+ except requests.exceptions.RequestException as e:
250
+ print(f"[ERROR] Failed to fetch chargeback report: {e}")
251
+ sys.exit(1)
252
+
253
+ workspaces = data.get("workspaces", [])
254
+ print(f"Found {len(workspaces)} workspaces\n")
255
+
256
+ headers = {
257
+ "Authorization": api_key,
258
+ "Content-Type": "application/json",
259
+ }
260
+
261
+ if create_workspaces and not dry_run:
262
+ print("Checking existing workspaces on destination...")
263
+ try:
264
+ existing = _get_existing_workspaces(dest_url, headers)
265
+ except requests.exceptions.RequestException as e:
266
+ print(f"[ERROR] Failed to list destination workspaces: {e}")
267
+ sys.exit(1)
268
+
269
+ ws_created = 0
270
+ for workspace in workspaces:
271
+ ws_name = workspace["name"]
272
+ if ws_name not in existing:
273
+ try:
274
+ _create_workspace(dest_url, headers, ws_name)
275
+ print(f" [CREATED] Workspace '{ws_name}'")
276
+ ws_created += 1
277
+ except requests.exceptions.RequestException as e:
278
+ print(f" [ERROR] Failed to create workspace '{ws_name}': {e}")
279
+ print(f"Created {ws_created} new workspace(s)\n")
280
+ elif create_workspaces and dry_run:
281
+ print("[DRY RUN] Would check/create workspaces on destination\n")
282
+
283
+ add_member_url = f"{dest_url}/api/rest/v2/write/add-workspace-member"
284
+
285
+ total_added = 0
286
+ total_skipped = 0
287
+ total_no_email = 0
288
+ total_already_member = 0
289
+ total_failed = 0
290
+ failures = []
291
+
292
+ for workspace in workspaces:
293
+ ws_name = workspace["name"]
294
+ members = workspace.get("members", [])
295
+
296
+ if not members:
297
+ print(f"[SKIP] Workspace '{ws_name}' — no members, skipping")
298
+ total_skipped += 1
299
+ continue
300
+
301
+ print(f"\n{'=' * 60}")
302
+ print(f"Workspace: {ws_name} ({len(members)} members)")
303
+ print(f"{'=' * 60}")
304
+
305
+ ws_added = 0
306
+ ws_no_email = 0
307
+ ws_already_member = 0
308
+ ws_failed = 0
309
+
310
+ for member in members:
311
+ email = member.get("email")
312
+
313
+ if not email:
314
+ ws_no_email += 1
315
+ total_no_email += 1
316
+ print(f" [SKIP] '{member['userName']}' has no email, skipping")
317
+ continue
318
+
319
+ if dry_run:
320
+ print(f" [DRY RUN] Would add '{email}' to '{ws_name}'")
321
+ ws_added += 1
322
+ total_added += 1
323
+ continue
324
+
325
+ status, error_info = _add_member(add_member_url, headers, email, ws_name)
326
+
327
+ if status == "added":
328
+ ws_added += 1
329
+ total_added += 1
330
+ elif status == "already_member":
331
+ ws_already_member += 1
332
+ total_already_member += 1
333
+ else:
334
+ print(
335
+ f" [FAIL] '{email}' -> '{ws_name}' "
336
+ f"— {error_info['status']}: {error_info['error']}"
337
+ )
338
+ ws_failed += 1
339
+ total_failed += 1
340
+ failures.append(
341
+ {
342
+ "workspace": ws_name,
343
+ "email": email,
344
+ **error_info,
345
+ }
346
+ )
347
+
348
+ if dry_run:
349
+ print(f" [DRY RUN] Total: would add {ws_added}/{len(members)} users")
350
+ else:
351
+ parts = [f" Added {ws_added}/{len(members)} users successfully"]
352
+ if ws_already_member:
353
+ parts.append(f"{ws_already_member} already a member")
354
+ if ws_no_email:
355
+ parts.append(f"{ws_no_email} skipped (no email)")
356
+ if ws_failed:
357
+ parts.append(f"{ws_failed} failed")
358
+ print(
359
+ parts[0]
360
+ + (" (" + ", ".join(parts[1:]) + ")" if len(parts) > 1 else "")
361
+ )
362
+
363
+ print(f"\n{'=' * 60}")
364
+ print("SUMMARY")
365
+ print(f"{'=' * 60}")
366
+ print(f" Workspaces processed : {len(workspaces)}")
367
+ print(f" Workspaces skipped : {total_skipped} (no members)")
368
+ print(f" Users added : {total_added}")
369
+ print(f" Already a member : {total_already_member}")
370
+ print(f" Skipped (no email) : {total_no_email}")
371
+ print(f" Failures : {total_failed}")
372
+
373
+ if dry_run:
374
+ print(f"\n *** DRY RUN MODE — no API calls were made ***")
375
+ print(f" *** Remove --dry-run to execute for real ***")
376
+
377
+ if failures:
378
+ with open(failures_output, "w") as f:
379
+ json.dump(failures, f, indent=2)
380
+ print(f"\n Failed operations saved to {failures_output}")
381
+
382
+
383
+ def main(args):
384
+ parser = argparse.ArgumentParser(
385
+ description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
386
+ )
387
+ get_parser_arguments(parser)
388
+ parsed_args = parser.parse_args(args)
389
+ migrate_users(parsed_args)
390
+
391
+
392
+ if __name__ == "__main__":
393
+ main(sys.argv[1:])
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: cometx
3
- Version: 3.6.4
3
+ Version: 3.6.5
4
4
  Summary: Python tools for Comet
5
5
  Home-page: https://github.com/comet-ml/cometx/
6
6
  Author: cometx development team
@@ -102,6 +102,7 @@ for ML management.
102
102
  * [cometx download](#cometx-download)
103
103
  * [cometx list](#cometx-list)
104
104
  * [cometx log](#cometx-log)
105
+ * [cometx migrate-users](#cometx-migrate-users)
105
106
  * [cometx rename-duplicates](#cometx-rename-duplicates)
106
107
  * [cometx reproduce](#cometx-reproduce)
107
108
  * [cometx smoke-test](#cometx-smoke-test)
@@ -324,6 +325,7 @@ Not all combinations are possible:
324
325
  * `--symlink` - Instead of copying, create a link to an experiment in a project
325
326
  * `--sync` - Check to see if experiment name has been created first; if so, skip
326
327
  * `--path PATH` - Path to prepend to workspace_src when accessing files (supports ~ for home directory)
328
+ * `--create-workspaces` - Attempt to create the destination workspace if it does not exist (requires appropriate API permissions)
327
329
 
328
330
  ### Using --path
329
331
 
@@ -385,6 +387,47 @@ Where TYPE is one of the following names:
385
387
 
386
388
  For more information, `cometx log --help`
387
389
 
390
+ ## cometx migrate-users
391
+
392
+ This command migrates users into workspaces from a source Comet environment to a destination environment. It reads workspace membership from a chargeback report fetched from the source environment's admin API (or a pre-downloaded local file) and invites each user to the corresponding workspace in the destination environment by email. If a user with that email does not yet exist in the destination, they will be provisioned access once they sign up.
393
+
394
+ ```
395
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY [FLAGS ...]
396
+ cometx migrate-users --api-key DEST_KEY --chargeback-report /path/to/report.json [FLAGS ...]
397
+ ```
398
+
399
+ **Arguments:**
400
+ * `--api-key API_KEY` - API key for the destination environment. Falls back to `COMET_API_KEY` if not provided.
401
+ * `--url URL` - Base URL of the destination Comet environment (e.g. `https://comet.example.com`). Required for self-hosted instances when the API key does not encode the server URL.
402
+ * `--source-api-key SOURCE_API_KEY` - API key for the source environment. Used to fetch the chargeback report. Required unless `--chargeback-report` is given.
403
+ * `--source-url SOURCE_URL` - Base URL of the source Comet environment. Required for self-hosted source instances when the source API key does not encode the server URL.
404
+ * `--chargeback-report PATH` - Path to a local chargeback report JSON file. When provided, `--source-api-key` is not required.
405
+
406
+ ### Flags
407
+
408
+ * `--create-workspaces` - Create workspaces on the destination if they don't already exist (default: off)
409
+ * `--dry-run` - Print what would happen without making any changes
410
+ * `--failures-output PATH` - Path to write failed operations JSON (default: `bulk_add_failures_by_email.json`)
411
+
412
+ ### Examples
413
+
414
+ ```bash
415
+ # Dry run — preview what would happen
416
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY --dry-run
417
+
418
+ # Execute the migration
419
+ cometx migrate-users --api-key DEST_KEY --source-api-key SOURCE_KEY
420
+
421
+ # Use a local chargeback report file
422
+ cometx migrate-users --api-key DEST_KEY --chargeback-report /tmp/chargeback_reports.json
423
+
424
+ # Self-hosted environments — provide explicit URLs
425
+ cometx migrate-users --api-key DEST_KEY --url https://comet.dest.example.com \
426
+ --source-api-key SOURCE_KEY --source-url https://comet.src.example.com
427
+ ```
428
+
429
+ For more information, `cometx migrate-users --help`
430
+
388
431
  ## cometx rename-duplicates
389
432
 
390
433
  This command is used to rename duplicate experiments within projects. When multiple experiments share the same name in a project, this command renames the duplicates to NAME-1, NAME-2, etc. while avoiding conflicts with existing names.
@@ -535,12 +578,8 @@ cometx admin chargeback-report [YEAR-MONTH]
535
578
  **Examples:**
536
579
  ```
537
580
  cometx admin chargeback-report
538
- <<<<<<< Updated upstream
539
- cometx admin usage-report
540
- =======
541
581
  cometx admin chargeback-report 2024-09
542
582
  ```
543
- >>>>>>> Stashed changes
544
583
 
545
584
  #### usage-report
546
585
 
@@ -35,6 +35,7 @@ cometx/cli/delete_assets.py
35
35
  cometx/cli/download.py
36
36
  cometx/cli/list_command.py
37
37
  cometx/cli/log.py
38
+ cometx/cli/migrate_users.py
38
39
  cometx/cli/rename_duplicates.py
39
40
  cometx/cli/reproduce.py
40
41
  cometx/cli/smoke_test.py
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes