diaspora-event-sdk 0.3.0__tar.gz → 0.3.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (47) hide show
  1. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/LICENSE +1 -1
  2. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/MANIFEST.in +1 -0
  3. diaspora-event-sdk-0.3.1/PKG-INFO +37 -0
  4. diaspora-event-sdk-0.3.1/README.md +26 -0
  5. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/__init__.py +1 -2
  6. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/_environments.py +2 -0
  7. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/aws_iam_msk.py +14 -9
  8. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/client.py +26 -12
  9. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/kafka_client.py +3 -3
  10. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/login_manager/client_login.py +1 -0
  11. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/login_manager/login_flow.py +1 -1
  12. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/login_manager/protocol.py +4 -8
  13. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/login_manager/tokenstore.py +0 -2
  14. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/web_client.py +47 -29
  15. diaspora-event-sdk-0.3.1/diaspora_event_sdk/version.py +1 -0
  16. diaspora-event-sdk-0.3.1/diaspora_event_sdk.egg-info/PKG-INFO +37 -0
  17. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk.egg-info/SOURCES.txt +3 -1
  18. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk.egg-info/requires.txt +1 -0
  19. diaspora-event-sdk-0.3.1/mypy.ini +4 -0
  20. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/setup.py +10 -2
  21. diaspora-event-sdk-0.3.1/tests/unit/apis_test.py +132 -0
  22. diaspora-event-sdk-0.3.1/tox.ini +32 -0
  23. diaspora-event-sdk-0.3.0/PKG-INFO +0 -43
  24. diaspora-event-sdk-0.3.0/README.md +0 -29
  25. diaspora-event-sdk-0.3.0/diaspora_event_sdk/version.py +0 -1
  26. diaspora-event-sdk-0.3.0/diaspora_event_sdk.egg-info/PKG-INFO +0 -43
  27. diaspora-event-sdk-0.3.0/tox.ini +0 -14
  28. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/__init__.py +0 -0
  29. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/botocore/__init__.py +0 -0
  30. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/botocore/auth.py +0 -0
  31. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/botocore/awsrequest.py +0 -0
  32. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/botocore/compat.py +0 -0
  33. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/botocore/credentials.py +0 -0
  34. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/botocore/exceptions.py +0 -0
  35. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/botocore/utils.py +0 -0
  36. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/decorators.py +0 -0
  37. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/login_manager/__init__.py +0 -0
  38. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/login_manager/decorators.py +0 -0
  39. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/login_manager/globus_auth.py +0 -0
  40. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/login_manager/manager.py +0 -0
  41. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/utils/__init__.py +0 -0
  42. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk/sdk/utils/uuid_like.py +0 -0
  43. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk.egg-info/dependency_links.txt +0 -0
  44. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/diaspora_event_sdk.egg-info/top_level.txt +0 -0
  45. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/setup.cfg +0 -0
  46. {diaspora-event-sdk-0.3.0 → diaspora-event-sdk-0.3.1}/tests/__init__.py +0 -0
  47. /diaspora-event-sdk-0.3.0/tests/unit/test_client.py → /diaspora-event-sdk-0.3.1/tests/unit/client_test.py +0 -0
@@ -199,4 +199,4 @@
199
199
  distributed under the License is distributed on an "AS IS" BASIS,
200
200
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201
201
  See the License for the specific language governing permissions and
202
- limitations under the License.
202
+ limitations under the License.
@@ -1,3 +1,4 @@
1
1
  include *.md
2
2
  include tox.ini
3
+ include mypy.ini
3
4
  recursive-include tests *.py
@@ -0,0 +1,37 @@
1
+ Metadata-Version: 2.1
2
+ Name: diaspora-event-sdk
3
+ Version: 0.3.1
4
+ Summary: SDK of Diaspora Event Fabric: Resilience-enabling services for science from HPC to edge
5
+ Home-page: https://github.com/globus-labs/diaspora-event-sdk
6
+ License: Apache 2.0
7
+ Description-Content-Type: text/markdown
8
+ Provides-Extra: kafka-python
9
+ Provides-Extra: test
10
+ License-File: LICENSE
11
+
12
+ # Diaspora Event Fabric SDK
13
+
14
+ ## Installation Guide
15
+ ### Recommended Method: With `kafka-python`
16
+ To integrate with Diaspora Event Fabric using `KafkaProducer` and `KafkaConsumer`, install the SDK with `kafka-python`:
17
+ ```bash
18
+ pip install "diaspora-event-sdk[kafka-python]"
19
+ ```
20
+
21
+ ### Alternative Installation: Without Kafka Client Library
22
+ For other Kafka client libraries (e.g., `confluent-kafka-python`, `aiokafka`), install the SDK without `kafka-python`:
23
+ ```bash
24
+ pip install diaspora-event-sdk
25
+ ```
26
+ Note: This does not include `KafkaProducer` and `KafkaConsumer` dependencies.
27
+
28
+ ## Using Diaspora Event Fabric SDK
29
+ Check our [Notebook](DiasporaDemo.ipynb) for a quickstart and demonstration.
30
+
31
+ <!-- **Getting Started**: Visit our [QuickStart Guide](docs/quickstart.md) for details on using the SDK with the kafka-python library and instructions for other Kafka clients.
32
+
33
+ **Troubleshooting and Credential Management**: Consult our [TrobleShooting Guide](docs/troubleshooting.md) for solving common issues and tips on managing keys effectively. -->
34
+
35
+ <!-- **Advanced Consumers with Faust**: Explore the [Faust Streaming Guide](docs/faust_weather_app.md) for advanced event streaming with Faust. -->
36
+
37
+ <!-- **Advanced Consumer Functions**: See our [Colab example](https://colab.research.google.com/drive/1tPKfxU2qPsLvNTreF6nKINU62k7pQWxa?usp=sharing) for demonstration. -->
@@ -0,0 +1,26 @@
1
+ # Diaspora Event Fabric SDK
2
+
3
+ ## Installation Guide
4
+ ### Recommended Method: With `kafka-python`
5
+ To integrate with Diaspora Event Fabric using `KafkaProducer` and `KafkaConsumer`, install the SDK with `kafka-python`:
6
+ ```bash
7
+ pip install "diaspora-event-sdk[kafka-python]"
8
+ ```
9
+
10
+ ### Alternative Installation: Without Kafka Client Library
11
+ For other Kafka client libraries (e.g., `confluent-kafka-python`, `aiokafka`), install the SDK without `kafka-python`:
12
+ ```bash
13
+ pip install diaspora-event-sdk
14
+ ```
15
+ Note: This does not include `KafkaProducer` and `KafkaConsumer` dependencies.
16
+
17
+ ## Using Diaspora Event Fabric SDK
18
+ Check our [Notebook](DiasporaDemo.ipynb) for a quickstart and demonstration.
19
+
20
+ <!-- **Getting Started**: Visit our [QuickStart Guide](docs/quickstart.md) for details on using the SDK with the kafka-python library and instructions for other Kafka clients.
21
+
22
+ **Troubleshooting and Credential Management**: Consult our [TrobleShooting Guide](docs/troubleshooting.md) for solving common issues and tips on managing keys effectively. -->
23
+
24
+ <!-- **Advanced Consumers with Faust**: Explore the [Faust Streaming Guide](docs/faust_weather_app.md) for advanced event streaming with Faust. -->
25
+
26
+ <!-- **Advanced Consumer Functions**: See our [Colab example](https://colab.research.google.com/drive/1tPKfxU2qPsLvNTreF6nKINU62k7pQWxa?usp=sharing) for demonstration. -->
@@ -1,6 +1,5 @@
1
- """ Diaspora Event Fabric: Resilience-enabling services for science from HPC to edge.
1
+ """Diaspora Event Fabric: Resilience-enabling services for science from HPC to edge."""
2
2
 
3
- """
4
3
  from diaspora_event_sdk.version import __version__ as _version
5
4
 
6
5
  __author__ = "The Diaspora Event Team"
@@ -9,12 +9,14 @@ DIASPORA_RESOURCE_SERVER = "2b9d2f5c-fa32-45b5-875b-b24cd343b917"
9
9
  def _get_envname():
10
10
  return os.getenv("DIASPORA_SDK_ENVIRONMENT", "production")
11
11
 
12
+
12
13
  def get_web_service_url(envname: Union[str, None] = None) -> str:
13
14
  env = envname or _get_envname()
14
15
  urls = {
15
16
  "production": "https://diaspora-web-service.ml22sevubfnks.us-east-1.cs.amazonlightsail.com",
16
17
  "dev": "https://diaspora-web-service-dev.ml22sevubfnks.us-east-1.cs.amazonlightsail.com",
17
18
  "local": "http://localhost:8000",
19
+ "legacy": "http://3.220.110.101/",
18
20
  }
19
21
 
20
22
  return urls.get(env, urls["production"])
@@ -2,6 +2,7 @@
2
2
  # SPDX-License-Identifier: Apache-2.0
3
3
 
4
4
  import base64
5
+
5
6
  # import logging
6
7
  from datetime import datetime, timezone
7
8
  from urllib.parse import parse_qs, urlparse
@@ -11,6 +12,7 @@ from urllib.parse import parse_qs, urlparse
11
12
  # import pkg_resources
12
13
  from .botocore.auth import SigV4QueryAuth
13
14
  from .botocore.awsrequest import AWSRequest
15
+
14
16
  # from botocore.config import Config
15
17
  from .botocore.credentials import Credentials
16
18
 
@@ -25,7 +27,7 @@ LIB_NAME = "aws-msk-iam-sasl-signer-python"
25
27
 
26
28
 
27
29
  def __get_user_agent__():
28
- return (f"{LIB_NAME}/1.0.1")
30
+ return f"{LIB_NAME}/1.0.1"
29
31
 
30
32
 
31
33
  def __get_expiration_time_ms(request):
@@ -37,15 +39,17 @@ def __get_expiration_time_ms(request):
37
39
  # Parse the signed request
38
40
  parsed_url = urlparse(request.url)
39
41
  parsed_ul_params = parse_qs(parsed_url.query)
40
- parsed_signing_time = datetime.strptime(parsed_ul_params['X-Amz-Date'][0],
41
- "%Y%m%dT%H%M%SZ")
42
+ parsed_signing_time = datetime.strptime(
43
+ parsed_ul_params["X-Amz-Date"][0], "%Y%m%dT%H%M%SZ"
44
+ )
42
45
 
43
46
  # Make the datetime object timezone-aware
44
47
  signing_time = parsed_signing_time.replace(tzinfo=timezone.utc)
45
48
 
46
49
  # Convert the Unix timestamp to milliseconds
47
- expiration_timestamp_seconds = int(
48
- signing_time.timestamp()) + DEFAULT_TOKEN_EXPIRY_SECONDS
50
+ expiration_timestamp_seconds = (
51
+ int(signing_time.timestamp()) + DEFAULT_TOKEN_EXPIRY_SECONDS
52
+ )
49
53
 
50
54
  # Get lifetime of token
51
55
  expiration_timestamp_ms = expiration_timestamp_seconds * 1000
@@ -74,8 +78,7 @@ def __construct_auth_token(region, aws_credentials):
74
78
 
75
79
  # Create SigV4 instance
76
80
  sig_v4 = SigV4QueryAuth(
77
- aws_credentials, SIGNING_NAME, region,
78
- expires=DEFAULT_TOKEN_EXPIRY_SECONDS
81
+ aws_credentials, SIGNING_NAME, region, expires=DEFAULT_TOKEN_EXPIRY_SECONDS
79
82
  )
80
83
 
81
84
  # Create request with url and parameters
@@ -110,10 +113,12 @@ def generate_auth_token(region, aws_debug_creds=False):
110
113
 
111
114
  # Load credentials
112
115
  import os
116
+
113
117
  assert os.environ["AWS_ACCESS_KEY_ID"]
114
118
  assert os.environ["AWS_SECRET_ACCESS_KEY"]
115
119
 
116
- aws_credentials = Credentials(os.environ["AWS_ACCESS_KEY_ID"],
117
- os.environ["AWS_SECRET_ACCESS_KEY"])
120
+ aws_credentials = Credentials(
121
+ os.environ["AWS_ACCESS_KEY_ID"], os.environ["AWS_SECRET_ACCESS_KEY"]
122
+ )
118
123
 
119
124
  return __construct_auth_token(region, aws_credentials)
@@ -28,7 +28,8 @@ class Client:
28
28
  self.login_manager.ensure_logged_in()
29
29
 
30
30
  self.web_client = self.login_manager.get_web_client(
31
- base_url=self.web_service_address)
31
+ base_url=self.web_service_address
32
+ )
32
33
  self.auth_client = self.login_manager.get_auth_client()
33
34
  self.subject_openid = self.auth_client.oauth2_userinfo()["sub"]
34
35
 
@@ -153,7 +154,9 @@ class Client:
153
154
  """
154
155
  Increases the number of partitions for a given topic to the specified new partition count.
155
156
  """
156
- return self.web_client.update_topic_partitions(self.subject_openid, topic, new_partitions)
157
+ return self.web_client.update_topic_partitions(
158
+ self.subject_openid, topic, new_partitions
159
+ )
157
160
 
158
161
  @requires_login
159
162
  def reset_topic(self, topic):
@@ -167,14 +170,18 @@ class Client:
167
170
  """
168
171
  Authorizes another user to access a registered topic under the invoker's account.
169
172
  """
170
- return self.web_client.grant_user_access(self.subject_openid, topic, user, "grant")
173
+ return self.web_client.grant_user_access(
174
+ self.subject_openid, topic, user, "grant"
175
+ )
171
176
 
172
177
  @requires_login
173
178
  def revoke_user_access(self, topic, user):
174
179
  """
175
180
  Removes access permissions for another user from a registered topic under the invoker's account.
176
181
  """
177
- return self.web_client.grant_user_access(self.subject_openid, topic, user, "revoke")
182
+ return self.web_client.grant_user_access(
183
+ self.subject_openid, topic, user, "revoke"
184
+ )
178
185
 
179
186
  @requires_login
180
187
  def list_topic_users(self, topic):
@@ -193,10 +200,16 @@ class Client:
193
200
  @requires_login
194
201
  def create_trigger(self, topic, function, function_configs, trigger_configs):
195
202
  """
196
- Creates a new trigger under the user's account with specific function and invocation configurations.
203
+ Creates a new trigger under the user's account with specific function and invocation configurations.
197
204
  """
198
205
  return self.web_client.create_trigger(
199
- self.subject_openid, topic, function, "create", function_configs, trigger_configs)
206
+ self.subject_openid,
207
+ topic,
208
+ function,
209
+ "create",
210
+ function_configs,
211
+ trigger_configs,
212
+ )
200
213
 
201
214
  @requires_login
202
215
  def delete_trigger(self, topic, function):
@@ -204,27 +217,28 @@ class Client:
204
217
  Deletes a trigger and related AWS resources, while the associated topic remains unaffected.
205
218
  """
206
219
  return self.web_client.create_trigger(
207
- self.subject_openid, topic, function, "delete", {}, {})
220
+ self.subject_openid, topic, function, "delete", {}, {}
221
+ )
208
222
 
209
223
  @requires_login
210
224
  def update_trigger(self, trigger_uuid, trigger_configs):
211
225
  """
212
226
  Updates invocation configurations of an existing trigger, identified by its unique trigger UUID.
213
227
  """
214
- return self.web_client.update_trigger(self.subject_openid, trigger_uuid, trigger_configs)
228
+ return self.web_client.update_trigger(
229
+ self.subject_openid, trigger_uuid, trigger_configs
230
+ )
215
231
 
216
232
  @requires_login
217
233
  def list_log_streams(self, trigger):
218
234
  """
219
235
  List log streams of a trigger under the user's account
220
236
  """
221
- return self.web_client.list_log_streams(
222
- self.subject_openid, trigger)
237
+ return self.web_client.list_log_streams(self.subject_openid, trigger)
223
238
 
224
239
  @requires_login
225
240
  def get_log_events(self, trigger, stream):
226
241
  """
227
242
  Get events in a particular log stream of a trigger under the user's account
228
243
  """
229
- return self.web_client.get_log_events(
230
- self.subject_openid, trigger, stream)
244
+ return self.web_client.get_log_events(self.subject_openid, trigger, stream)
@@ -17,7 +17,7 @@ try:
17
17
  def token(self):
18
18
  token, _ = generate_auth_token("us-east-1")
19
19
  return token
20
- except Exception as e:
20
+ except Exception:
21
21
  kafka_available = False
22
22
 
23
23
 
@@ -90,7 +90,7 @@ def block_until_ready(max_minutes=5):
90
90
  value={"message": "Synchronous message from Diaspora SDK"},
91
91
  )
92
92
  result["producer_connection_test"] = future.get(timeout=10)
93
- except Exception as e:
93
+ except Exception:
94
94
  pass
95
95
 
96
96
  def consumer_connection_test(result):
@@ -103,7 +103,7 @@ def block_until_ready(max_minutes=5):
103
103
  for msg in consumer:
104
104
  result["consumer_connection_test"] = msg
105
105
  break
106
- except Exception as e:
106
+ except Exception:
107
107
  pass
108
108
 
109
109
  result, retry_count = {}, 0
@@ -4,6 +4,7 @@ Logic for using client identities with the Diaspora SDK
4
4
  The design is based on the Globus CLI client login:
5
5
  https://github.com/globus/globus-cli/blob/main/src/globus_cli/login_manager/client_login.py
6
6
  """
7
+
7
8
  from __future__ import annotations
8
9
 
9
10
  import logging
@@ -32,4 +32,4 @@ def do_link_auth_flow(scopes: List[str]):
32
32
  auth_code = input("Enter the resulting Authorization Code here: ").strip()
33
33
 
34
34
  # finish auth flow
35
- return auth_client.oauth2_exchange_code_for_tokens(auth_code)
35
+ return auth_client.oauth2_exchange_code_for_tokens(auth_code)
@@ -16,14 +16,10 @@ else:
16
16
 
17
17
  @runtime_checkable
18
18
  class LoginManagerProtocol(Protocol):
19
- def ensure_logged_in(self) -> None:
20
- ...
19
+ def ensure_logged_in(self) -> None: ...
21
20
 
22
- def logout(self) -> bool:
23
- ...
21
+ def logout(self) -> bool: ...
24
22
 
25
- def get_auth_client(self) -> globus_sdk.AuthClient:
26
- ...
23
+ def get_auth_client(self) -> globus_sdk.AuthClient: ...
27
24
 
28
- def get_web_client(self, *, base_url: str | None = None) -> WebClient:
29
- ...
25
+ def get_web_client(self, *, base_url: str | None = None) -> WebClient: ...
@@ -1,6 +1,5 @@
1
1
  from __future__ import annotations
2
2
 
3
- import json
4
3
  import os
5
4
  import pathlib
6
5
 
@@ -8,7 +7,6 @@ from globus_sdk.tokenstorage import SQLiteAdapter
8
7
 
9
8
  from .._environments import _get_envname
10
9
  from .client_login import get_client_login, is_client_login
11
- from .globus_auth import internal_auth_client
12
10
 
13
11
 
14
12
  def _home() -> pathlib.Path:
@@ -36,15 +36,14 @@ class WebClient(globus_sdk.BaseClient):
36
36
  ) -> globus_sdk.GlobusHTTPResponse:
37
37
  return self.put(
38
38
  f"/api/v2/topic/{topic}",
39
- headers={"Subject": str(subject), "Action": action}
39
+ headers={"Subject": str(subject), "Action": action},
40
40
  )
41
41
 
42
42
  def get_topic_configs(
43
43
  self, subject: UUID_LIKE_T, topic: str
44
44
  ) -> globus_sdk.GlobusHTTPResponse:
45
45
  return self.get(
46
- f"/api/v2/topic/{topic}",
47
- headers={"Subject": str(subject), "Topic": topic}
46
+ f"/api/v2/topic/{topic}", headers={"Subject": str(subject), "Topic": topic}
48
47
  )
49
48
 
50
49
  def update_topic_configs(
@@ -52,9 +51,12 @@ class WebClient(globus_sdk.BaseClient):
52
51
  ) -> globus_sdk.GlobusHTTPResponse:
53
52
  return self.post(
54
53
  f"/api/v2/topic/{topic}",
55
- headers={"Subject": str(subject), "Topic": topic,
56
- "Content-Type": "text/plain"},
57
- data=json.dumps(configs).encode("utf-8")
54
+ headers={
55
+ "Subject": str(subject),
56
+ "Topic": topic,
57
+ "Content-Type": "text/plain",
58
+ },
59
+ data=json.dumps(configs).encode("utf-8"),
58
60
  )
59
61
 
60
62
  def update_topic_partitions(
@@ -62,8 +64,11 @@ class WebClient(globus_sdk.BaseClient):
62
64
  ) -> globus_sdk.GlobusHTTPResponse:
63
65
  return self.post(
64
66
  f"/api/v2/topic/{topic}/partitions",
65
- headers={"Subject": str(subject), "Topic": topic,
66
- "NewPartitions": str(new_partitions)}
67
+ headers={
68
+ "Subject": str(subject),
69
+ "Topic": topic,
70
+ "NewPartitions": str(new_partitions),
71
+ },
67
72
  )
68
73
 
69
74
  def reset_topic(
@@ -71,8 +76,7 @@ class WebClient(globus_sdk.BaseClient):
71
76
  ) -> globus_sdk.GlobusHTTPResponse:
72
77
  return self.post(
73
78
  f"/api/v2/topic/{topic}/reset",
74
- headers={"Subject": str(subject),
75
- "Topic": topic}
79
+ headers={"Subject": str(subject), "Topic": topic},
76
80
  )
77
81
 
78
82
  def grant_user_access(
@@ -80,8 +84,12 @@ class WebClient(globus_sdk.BaseClient):
80
84
  ) -> globus_sdk.GlobusHTTPResponse:
81
85
  return self.post(
82
86
  f"/api/v2/topic/{topic}/user",
83
- headers={"Subject": str(subject), "Action": action,
84
- "Topic": topic, "User": str(user)}
87
+ headers={
88
+ "Subject": str(subject),
89
+ "Action": action,
90
+ "Topic": topic,
91
+ "User": str(user),
92
+ },
85
93
  )
86
94
 
87
95
  def list_topic_users(
@@ -89,24 +97,33 @@ class WebClient(globus_sdk.BaseClient):
89
97
  ) -> globus_sdk.GlobusHTTPResponse:
90
98
  return self.get(
91
99
  f"/api/v2/topic/{topic}/users",
92
- headers={"Subject": str(subject),
93
- "Topic": topic}
100
+ headers={"Subject": str(subject), "Topic": topic},
94
101
  )
95
102
 
96
103
  def list_triggers(self, subject: UUID_LIKE_T) -> globus_sdk.GlobusHTTPResponse:
97
104
  return self.get("/api/v2/triggers", headers={"Subject": str(subject)})
98
105
 
99
106
  def create_trigger(
100
- self, subject: UUID_LIKE_T, topic: str, function: str, action: str,
101
- function_configs: dict, trigger_configs: dict
107
+ self,
108
+ subject: UUID_LIKE_T,
109
+ topic: str,
110
+ function: str,
111
+ action: str,
112
+ function_configs: dict,
113
+ trigger_configs: dict,
102
114
  ) -> globus_sdk.GlobusHTTPResponse:
103
115
  return self.put(
104
116
  "/api/v2/trigger",
105
- headers={"Subject": str(subject), "Topic": topic,
106
- "Trigger": function, "Action": action,
107
- "Content-Type": "text/plain"},
108
- data=json.dumps({"function": function_configs,
109
- "trigger": trigger_configs}).encode("utf-8")
117
+ headers={
118
+ "Subject": str(subject),
119
+ "Topic": topic,
120
+ "Trigger": function,
121
+ "Action": action,
122
+ "Content-Type": "text/plain",
123
+ },
124
+ data=json.dumps(
125
+ {"function": function_configs, "trigger": trigger_configs}
126
+ ).encode("utf-8"),
110
127
  )
111
128
 
112
129
  def update_trigger(
@@ -114,24 +131,25 @@ class WebClient(globus_sdk.BaseClient):
114
131
  ) -> globus_sdk.GlobusHTTPResponse:
115
132
  return self.post(
116
133
  f"/api/v2/triggers/{trigger_uuid}",
117
- headers={"Subject": str(subject), "Trigger_id": str(trigger_uuid),
118
- "Content-Type": "text/plain"},
119
- data=json.dumps(trigger_configs).encode("utf-8")
134
+ headers={
135
+ "Subject": str(subject),
136
+ "Trigger_id": str(trigger_uuid),
137
+ "Content-Type": "text/plain",
138
+ },
139
+ data=json.dumps(trigger_configs).encode("utf-8"),
120
140
  )
121
141
 
122
142
  def list_log_streams(
123
143
  self, subject: UUID_LIKE_T, trigger: str
124
144
  ) -> globus_sdk.GlobusHTTPResponse:
125
145
  return self.get(
126
- f"/api/v2/logs",
127
- headers={"Subject": str(subject), "Trigger": trigger}
146
+ "/api/v2/logs", headers={"Subject": str(subject), "Trigger": trigger}
128
147
  )
129
148
 
130
149
  def get_log_events(
131
150
  self, subject: UUID_LIKE_T, trigger: str, stream: str
132
151
  ) -> globus_sdk.GlobusHTTPResponse:
133
152
  return self.get(
134
- f"/api/v2/log",
135
- headers={"Subject": str(subject), "Trigger": trigger,
136
- "Stream": stream}
153
+ "/api/v2/log",
154
+ headers={"Subject": str(subject), "Trigger": trigger, "Stream": stream},
137
155
  )
@@ -0,0 +1 @@
1
+ __version__ = "0.3.1"
@@ -0,0 +1,37 @@
1
+ Metadata-Version: 2.1
2
+ Name: diaspora-event-sdk
3
+ Version: 0.3.1
4
+ Summary: SDK of Diaspora Event Fabric: Resilience-enabling services for science from HPC to edge
5
+ Home-page: https://github.com/globus-labs/diaspora-event-sdk
6
+ License: Apache 2.0
7
+ Description-Content-Type: text/markdown
8
+ Provides-Extra: kafka-python
9
+ Provides-Extra: test
10
+ License-File: LICENSE
11
+
12
+ # Diaspora Event Fabric SDK
13
+
14
+ ## Installation Guide
15
+ ### Recommended Method: With `kafka-python`
16
+ To integrate with Diaspora Event Fabric using `KafkaProducer` and `KafkaConsumer`, install the SDK with `kafka-python`:
17
+ ```bash
18
+ pip install "diaspora-event-sdk[kafka-python]"
19
+ ```
20
+
21
+ ### Alternative Installation: Without Kafka Client Library
22
+ For other Kafka client libraries (e.g., `confluent-kafka-python`, `aiokafka`), install the SDK without `kafka-python`:
23
+ ```bash
24
+ pip install diaspora-event-sdk
25
+ ```
26
+ Note: This does not include `KafkaProducer` and `KafkaConsumer` dependencies.
27
+
28
+ ## Using Diaspora Event Fabric SDK
29
+ Check our [Notebook](DiasporaDemo.ipynb) for a quickstart and demonstration.
30
+
31
+ <!-- **Getting Started**: Visit our [QuickStart Guide](docs/quickstart.md) for details on using the SDK with the kafka-python library and instructions for other Kafka clients.
32
+
33
+ **Troubleshooting and Credential Management**: Consult our [TrobleShooting Guide](docs/troubleshooting.md) for solving common issues and tips on managing keys effectively. -->
34
+
35
+ <!-- **Advanced Consumers with Faust**: Explore the [Faust Streaming Guide](docs/faust_weather_app.md) for advanced event streaming with Faust. -->
36
+
37
+ <!-- **Advanced Consumer Functions**: See our [Colab example](https://colab.research.google.com/drive/1tPKfxU2qPsLvNTreF6nKINU62k7pQWxa?usp=sharing) for demonstration. -->
@@ -1,6 +1,7 @@
1
1
  LICENSE
2
2
  MANIFEST.in
3
3
  README.md
4
+ mypy.ini
4
5
  setup.py
5
6
  tox.ini
6
7
  diaspora_event_sdk/__init__.py
@@ -35,4 +36,5 @@ diaspora_event_sdk/sdk/login_manager/tokenstore.py
35
36
  diaspora_event_sdk/sdk/utils/__init__.py
36
37
  diaspora_event_sdk/sdk/utils/uuid_like.py
37
38
  tests/__init__.py
38
- tests/unit/test_client.py
39
+ tests/unit/apis_test.py
40
+ tests/unit/client_test.py
@@ -10,3 +10,4 @@ coverage
10
10
  mypy
11
11
  tox
12
12
  check-manifest
13
+ pre-commit
@@ -0,0 +1,4 @@
1
+ [mypy]
2
+
3
+ [mypy-diaspora_event_sdk.sdk.botocore.*]
4
+ ignore_errors = True
@@ -2,9 +2,17 @@ import os
2
2
  import re
3
3
  from pathlib import Path
4
4
 
5
- from setuptools import setup, find_packages
5
+ from setuptools import find_packages, setup
6
6
 
7
- TEST_REQUIRES = ["pytest", "pytest-cov", "coverage", "mypy", "tox", "check-manifest"]
7
+ TEST_REQUIRES = [
8
+ "pytest",
9
+ "pytest-cov",
10
+ "coverage",
11
+ "mypy",
12
+ "tox",
13
+ "check-manifest",
14
+ "pre-commit",
15
+ ]
8
16
 
9
17
 
10
18
  def parse_version():
@@ -0,0 +1,132 @@
1
+ import pytest
2
+ import os
3
+ import logging
4
+ from globus_sdk import ConfidentialAppAuthClient
5
+ from diaspora_event_sdk import Client
6
+ from diaspora_event_sdk.sdk.login_manager import tokenstore
7
+
8
+ # Configure module-level logger
9
+ logging.basicConfig(
10
+ level=logging.INFO,
11
+ format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
12
+ )
13
+ logger = logging.getLogger(__name__)
14
+
15
+
16
+ @pytest.fixture(scope="module")
17
+ def setup():
18
+ client_id = os.environ["DIASPORA_SDK_CLIENT_ID"]
19
+ client_secret = os.environ["DIASPORA_SDK_CLIENT_SECRET"]
20
+ requested_scopes = os.environ["CLIENT_SCOPE"]
21
+
22
+ ca = ConfidentialAppAuthClient(
23
+ client_id=client_id,
24
+ client_secret=client_secret,
25
+ )
26
+ token_response = ca.oauth2_client_credentials_tokens(
27
+ requested_scopes=requested_scopes,
28
+ )
29
+ token = token_response.by_resource_server[client_id]
30
+
31
+ storage = tokenstore.get_token_storage_adapter()
32
+ storage.store(token_response)
33
+ storage.get_by_resource_server()
34
+
35
+ return {
36
+ "client_id": client_id,
37
+ "client_secret": client_secret,
38
+ "requested_scopes": requested_scopes,
39
+ "token_response": token_response,
40
+ "token": token,
41
+ }
42
+
43
+
44
+ @pytest.fixture(scope="module")
45
+ def client():
46
+ return Client()
47
+
48
+
49
+ def test_create_key(setup, client):
50
+ key_response = client.create_key()
51
+ assert isinstance(key_response, dict)
52
+ assert "access_key" in key_response
53
+ assert "secret_key" in key_response
54
+ assert "endpoint" in key_response
55
+
56
+
57
+ def test_register_topic(setup, client):
58
+ topic = "topic" + client.subject_openid[-12:]
59
+ register_response = client.register_topic(topic)
60
+ assert register_response["status"] in ["success", "no-op"]
61
+ assert "message" in register_response
62
+
63
+
64
+ def test_list_topics(setup, client):
65
+ topics = client.list_topics()
66
+ assert topics["status"] == "success"
67
+ assert isinstance(topics["topics"], list)
68
+ assert len(topics["topics"]) > 0
69
+ expected_topics = ["diaspora-cicd"]
70
+ assert set(expected_topics).issubset(set(topics["topics"]))
71
+
72
+
73
+ def test_get_topic_configs(setup, client):
74
+ topic = "topic" + client.subject_openid[-12:]
75
+ client.register_topic(topic)
76
+ configs_response = client.get_topic_configs(topic)
77
+ assert configs_response["status"] == "success"
78
+ assert "configs" in configs_response
79
+ assert isinstance(configs_response["configs"], dict)
80
+
81
+
82
+ def test_update_topic_configs(setup, client):
83
+ topic = "topic" + client.subject_openid[-12:]
84
+ client.register_topic(topic)
85
+ configs = {"min.insync.replicas": 1}
86
+ update_response = client.update_topic_configs(topic, configs)
87
+ assert update_response["status"] == "success"
88
+ assert "before" in update_response
89
+ assert "after" in update_response
90
+ assert isinstance(update_response["before"], dict)
91
+ assert isinstance(update_response["after"], dict)
92
+
93
+
94
+ def test_update_topic_partitions(setup, client):
95
+ topic = "topic" + client.subject_openid[-12:]
96
+ client.register_topic(topic)
97
+ new_partitions = 2
98
+ partitions_response = client.update_topic_partitions(topic, new_partitions)
99
+ assert partitions_response["status"] in ["success", "error"]
100
+ if partitions_response["status"] == "error":
101
+ assert "message" in partitions_response
102
+
103
+
104
+ def test_reset_topic(setup, client):
105
+ topic = "topic" + client.subject_openid[-12:]
106
+ client.register_topic(topic)
107
+ reset_response = client.reset_topic(topic)
108
+ assert reset_response["status"] in ["success", "error"]
109
+ if reset_response["status"] == "error":
110
+ assert "message" in reset_response
111
+
112
+
113
+ def test_user_access_management(setup, client):
114
+ topic = "topic" + client.subject_openid[-12:]
115
+ client.register_topic(topic)
116
+ user_id = "e2a8169b-feef-4d56-8eba-ab12747bee04"
117
+ grant_response = client.grant_user_access(topic, user_id)
118
+ assert grant_response["status"] in ["success", "no-op"]
119
+ assert "message" in grant_response
120
+
121
+ list_users_response = client.list_topic_users(topic)
122
+ assert list_users_response["status"] == "success"
123
+ assert "users" in list_users_response
124
+ assert isinstance(list_users_response["users"], list)
125
+
126
+ revoke_response = client.revoke_user_access(topic, user_id)
127
+ assert revoke_response["status"] in ["success", "no-op"]
128
+ assert "message" in revoke_response
129
+
130
+
131
+ if __name__ == "__main__":
132
+ pytest.main(["-s", "tests/unit/test_apis.py"])
@@ -0,0 +1,32 @@
1
+ [tox]
2
+ envlist = py{37, 38, 39, 310, 311, 312}, pre-commit, mypy
3
+
4
+ [testenv]
5
+ deps =
6
+ pytest
7
+ pytest-cov
8
+ coverage
9
+ setenv =
10
+ DIASPORA_SDK_CLIENT_ID={env:DIASPORA_SDK_CLIENT_ID}
11
+ DIASPORA_SDK_CLIENT_SECRET={env:DIASPORA_SDK_CLIENT_SECRET}
12
+ CLIENT_SCOPE={env:CLIENT_SCOPE}
13
+ passenv =
14
+ DIASPORA_SDK_CLIENT_ID
15
+ DIASPORA_SDK_CLIENT_SECRET
16
+ CLIENT_SCOPE
17
+ commands =
18
+ ; pytest --cov=diaspora_event_sdk --cov-report=term-missing tests {posargs}
19
+ coverage erase
20
+ coverage run -m pytest {posargs}
21
+ coverage report
22
+
23
+ [testenv:pre-commit]
24
+ skip_install = true
25
+ deps = pre-commit
26
+ commands = pre-commit run --all-files --show-diff-on-failure
27
+
28
+
29
+ [testenv:mypy]
30
+ deps =
31
+ mypy
32
+ commands = mypy -p diaspora_event_sdk {posargs}
@@ -1,43 +0,0 @@
1
- Metadata-Version: 2.1
2
- Name: diaspora-event-sdk
3
- Version: 0.3.0
4
- Summary: SDK of Diaspora Event Fabric: Resilience-enabling services for science from HPC to edge
5
- Home-page: https://github.com/globus-labs/diaspora-event-sdk
6
- License: Apache 2.0
7
- Platform: UNKNOWN
8
- Description-Content-Type: text/markdown
9
- Provides-Extra: kafka-python
10
- Provides-Extra: test
11
- License-File: LICENSE
12
-
13
- # Diaspora: Hybrid Event-Driven Architecture for Distributed Scientific Computing
14
-
15
- ## Event Fabric SDK Installation Guide
16
- ### Recommended Method: Use with `kafka-python`
17
- For easy integration with Diaspora Event Fabric, use the `KafkaProducer` and `KafkaConsumer` classes from our SDK. This requires the `kafka-python` library.
18
-
19
- To install the Event Fabric SDK and `kafka-python,` with the following command:
20
- ```bash
21
- pip install "diaspora-event-sdk[kafka-python]"
22
- ```
23
-
24
- ### Alternative Installation: Without Kafka Client Library
25
- To use alternative Kafka client libraries (e.g., `confluent-kafka-python`, `aiokafka`, and libraries for other programming laguages), you can install the SDK without the `kafka-python` dependency. This option still provides topic-level access control (authorization) and login credential management features.
26
-
27
- To install the SDK without `kafka-python`, use:
28
- ```bash
29
- pip install diaspora-event-sdk
30
- ```
31
- Note: This method does not include dependencies for `KafkaProducer` and `KafkaConsumer` classes mentioned in the QuickStart
32
-
33
- ## Use Diaspora Event Fabric SDK
34
-
35
- **Getting Started**: Visit our [QuickStart Guide](docs/quickstart.md) for details on using the SDK with the kafka-python library and instructions for other Kafka clients.
36
-
37
- **Troubleshooting and Credential Management**: Consult our [TrobleShooting Guide](docs/troubleshooting.md) for solving common issues and tips on managing keys effectively.
38
-
39
- **Advanced Consumers with Faust**: Explore the [Faust Streaming Guide](docs/faust_weather_app.md) for advanced event streaming with Faust.
40
-
41
- **Advanced Consumer Functions**: See our [Colab example](https://colab.research.google.com/drive/1tPKfxU2qPsLvNTreF6nKINU62k7pQWxa?usp=sharing) for demonstration.
42
-
43
-
@@ -1,29 +0,0 @@
1
- # Diaspora: Hybrid Event-Driven Architecture for Distributed Scientific Computing
2
-
3
- ## Event Fabric SDK Installation Guide
4
- ### Recommended Method: Use with `kafka-python`
5
- For easy integration with Diaspora Event Fabric, use the `KafkaProducer` and `KafkaConsumer` classes from our SDK. This requires the `kafka-python` library.
6
-
7
- To install the Event Fabric SDK and `kafka-python,` with the following command:
8
- ```bash
9
- pip install "diaspora-event-sdk[kafka-python]"
10
- ```
11
-
12
- ### Alternative Installation: Without Kafka Client Library
13
- To use alternative Kafka client libraries (e.g., `confluent-kafka-python`, `aiokafka`, and libraries for other programming laguages), you can install the SDK without the `kafka-python` dependency. This option still provides topic-level access control (authorization) and login credential management features.
14
-
15
- To install the SDK without `kafka-python`, use:
16
- ```bash
17
- pip install diaspora-event-sdk
18
- ```
19
- Note: This method does not include dependencies for `KafkaProducer` and `KafkaConsumer` classes mentioned in the QuickStart
20
-
21
- ## Use Diaspora Event Fabric SDK
22
-
23
- **Getting Started**: Visit our [QuickStart Guide](docs/quickstart.md) for details on using the SDK with the kafka-python library and instructions for other Kafka clients.
24
-
25
- **Troubleshooting and Credential Management**: Consult our [TrobleShooting Guide](docs/troubleshooting.md) for solving common issues and tips on managing keys effectively.
26
-
27
- **Advanced Consumers with Faust**: Explore the [Faust Streaming Guide](docs/faust_weather_app.md) for advanced event streaming with Faust.
28
-
29
- **Advanced Consumer Functions**: See our [Colab example](https://colab.research.google.com/drive/1tPKfxU2qPsLvNTreF6nKINU62k7pQWxa?usp=sharing) for demonstration.
@@ -1 +0,0 @@
1
- __version__ = "0.3.0"
@@ -1,43 +0,0 @@
1
- Metadata-Version: 2.1
2
- Name: diaspora-event-sdk
3
- Version: 0.3.0
4
- Summary: SDK of Diaspora Event Fabric: Resilience-enabling services for science from HPC to edge
5
- Home-page: https://github.com/globus-labs/diaspora-event-sdk
6
- License: Apache 2.0
7
- Platform: UNKNOWN
8
- Description-Content-Type: text/markdown
9
- Provides-Extra: kafka-python
10
- Provides-Extra: test
11
- License-File: LICENSE
12
-
13
- # Diaspora: Hybrid Event-Driven Architecture for Distributed Scientific Computing
14
-
15
- ## Event Fabric SDK Installation Guide
16
- ### Recommended Method: Use with `kafka-python`
17
- For easy integration with Diaspora Event Fabric, use the `KafkaProducer` and `KafkaConsumer` classes from our SDK. This requires the `kafka-python` library.
18
-
19
- To install the Event Fabric SDK and `kafka-python,` with the following command:
20
- ```bash
21
- pip install "diaspora-event-sdk[kafka-python]"
22
- ```
23
-
24
- ### Alternative Installation: Without Kafka Client Library
25
- To use alternative Kafka client libraries (e.g., `confluent-kafka-python`, `aiokafka`, and libraries for other programming laguages), you can install the SDK without the `kafka-python` dependency. This option still provides topic-level access control (authorization) and login credential management features.
26
-
27
- To install the SDK without `kafka-python`, use:
28
- ```bash
29
- pip install diaspora-event-sdk
30
- ```
31
- Note: This method does not include dependencies for `KafkaProducer` and `KafkaConsumer` classes mentioned in the QuickStart
32
-
33
- ## Use Diaspora Event Fabric SDK
34
-
35
- **Getting Started**: Visit our [QuickStart Guide](docs/quickstart.md) for details on using the SDK with the kafka-python library and instructions for other Kafka clients.
36
-
37
- **Troubleshooting and Credential Management**: Consult our [TrobleShooting Guide](docs/troubleshooting.md) for solving common issues and tips on managing keys effectively.
38
-
39
- **Advanced Consumers with Faust**: Explore the [Faust Streaming Guide](docs/faust_weather_app.md) for advanced event streaming with Faust.
40
-
41
- **Advanced Consumer Functions**: See our [Colab example](https://colab.research.google.com/drive/1tPKfxU2qPsLvNTreF6nKINU62k7pQWxa?usp=sharing) for demonstration.
42
-
43
-
@@ -1,14 +0,0 @@
1
- [tox]
2
- envlist = py{37, 38, 39, 310, 311, 312}, mypy
3
-
4
- [testenv]
5
- deps =
6
- pytest
7
- pytest-cov
8
- commands =
9
- pytest --cov=diaspora_event_sdk ./tests {posargs}
10
-
11
- [testenv:mypy]
12
- deps =
13
- mypy
14
- commands = mypy -p diaspora_event_sdk {posargs}