tug-provider 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,436 @@
1
+ Metadata-Version: 2.4
2
+ Name: tug-provider
3
+ Version: 0.1.0
4
+ Summary: Python SDK for tug provider plugins (protocol provider/v1alpha1)
5
+ Project-URL: Homepage, https://github.com/paingha/tugcli
6
+ Project-URL: Documentation, https://github.com/paingha/tugcli/blob/main/docs/plugins.md
7
+ Project-URL: Source, https://github.com/paingha/tugcli/tree/main/sdk/python
8
+ Keywords: tug,kubernetes,kubectl,plugin
9
+ Classifier: Development Status :: 3 - Alpha
10
+ Classifier: Intended Audience :: Developers
11
+ Classifier: Operating System :: MacOS
12
+ Classifier: Operating System :: POSIX :: Linux
13
+ Classifier: Programming Language :: Python :: 3
14
+ Classifier: Programming Language :: Python :: 3 :: Only
15
+ Classifier: Topic :: System :: Systems Administration
16
+ Classifier: Typing :: Typed
17
+ Requires-Python: >=3.9
18
+ Description-Content-Type: text/markdown
19
+
20
+ # tug-provider
21
+
22
+ The Python SDK for [tug](https://github.com/paingha/tugcli) provider plugins.
23
+ It speaks protocol `provider/v1alpha1`, mirrors the Go SDK,
24
+ [`pkg/provider`](https://github.com/paingha/tugcli/tree/main/pkg/provider),
25
+ and writes the same JSON. It needs Python 3.9 or later and nothing outside
26
+ the standard library.
27
+
28
+ The [plugin guide](https://github.com/paingha/tugcli/blob/main/docs/plugins.md)
29
+ explains what tug expects from a provider; this README shows how to do it in
30
+ Python.
31
+
32
+ ## Install
33
+
34
+ The package is to be published on PyPI as `tug-provider`; the import name is
35
+ `tug_provider`. Until then, install it from a checkout of tugcli:
36
+
37
+ ```bash
38
+ python3 -m pip install ./sdk/python
39
+ ```
40
+
41
+ ## A plugin
42
+
43
+ A plugin subclasses `Provider`, implements `info`, `list` and `credential`,
44
+ and hands an instance to `main`:
45
+
46
+ ```python
47
+ #!/usr/bin/env python3
48
+ """tug-provider-acme: Acme Kubernetes clusters for tug."""
49
+
50
+ import json
51
+ import sys
52
+ import urllib.error
53
+ import urllib.request
54
+ from datetime import datetime, timedelta, timezone
55
+
56
+ from tug_provider import (
57
+ UNAUTHENTICATED,
58
+ UNAVAILABLE,
59
+ Cluster,
60
+ Endpoints,
61
+ Info,
62
+ Provider,
63
+ ProviderError,
64
+ SecretSpec,
65
+ main,
66
+ status_error,
67
+ token,
68
+ )
69
+
70
+
71
+ class AcmeProvider(Provider):
72
+ def info(self):
73
+ return Info(
74
+ name="acme",
75
+ version="0.1.0",
76
+ description="Acme Kubernetes Engine",
77
+ secrets=[SecretSpec("api_token", prompt="acme API token")],
78
+ )
79
+
80
+ def list(self, request):
81
+ return [
82
+ Cluster(
83
+ id=c["id"], # stable for the life of the cluster
84
+ name=c["name"],
85
+ region=c["region"],
86
+ endpoints=Endpoints(public=c["endpoint"]),
87
+ labels={"env": c["environment"]},
88
+ )
89
+ for c in self.get(request, "/v1/clusters")
90
+ if c["endpoint"] # skip clusters that are not ready yet
91
+ ]
92
+
93
+ def credential(self, request):
94
+ t = self.get(request, "/v1/clusters/%s/token" % request.cluster_id)
95
+ return token(t["token"], datetime.now(timezone.utc) + timedelta(seconds=t["ttl"]))
96
+
97
+ def get(self, request, path):
98
+ api_token = request.secrets.get("api_token")
99
+ if not api_token:
100
+ raise ProviderError(UNAUTHENTICATED, "no api_token", hint="run tug import acme again to set one")
101
+ base = request.account.config_string("apiURL") or "https://api.acme.example"
102
+ req = urllib.request.Request(base + path, headers={"Authorization": "Bearer " + api_token})
103
+ try:
104
+ with urllib.request.urlopen(req, timeout=20) as resp:
105
+ return json.load(resp)
106
+ except urllib.error.HTTPError as e:
107
+ raise status_error(e.code, "GET %s: %s" % (path, e.reason), e.headers) from e
108
+ except urllib.error.URLError as e:
109
+ raise ProviderError(UNAVAILABLE, "GET %s: %s" % (path, e.reason)) from e
110
+
111
+
112
+ if __name__ == "__main__":
113
+ sys.exit(main(AcmeProvider()))
114
+ ```
115
+
116
+ Save it as `tug-provider-acme`, make it executable, and write a `plugin.yaml`
117
+ beside it whose platforms all point at the script, as the
118
+ [example's](https://github.com/paingha/tugcli/blob/main/sdk/python/examples/tug-provider-pyexample/plugin.yaml)
119
+ do. Then:
120
+
121
+ ```bash
122
+ tug plugin link .
123
+ TUG_SECRET_API_TOKEN=... tug plugin test acme --account staging
124
+ ```
125
+
126
+ `main` reads the operation from `sys.argv` and one request from stdin, calls
127
+ your method, writes exactly one JSON response to stdout, and returns the exit
128
+ code: 0 for a response, 1 for an error, 2 for a usage error such as an
129
+ unknown operation. It refuses a request for another protocol version or
130
+ operation with `InvalidConfig`, accepts requests that leave out `apiVersion`
131
+ and `kind`, and rejects an empty request for the operations that need one.
132
+ With the operation `serve`, `main` answers many requests in one process
133
+ instead; see [Serve mode](#serve-mode).
134
+
135
+ ## The API
136
+
137
+ The names follow the Go SDK, in Python's spelling. Fields are snake_case
138
+ (`cluster_id`, `ca_data`, `expires_at`); on the wire they are the protocol's
139
+ camelCase.
140
+
141
+ | Go SDK | Python SDK |
142
+ | --- | --- |
143
+ | `provider.Provider` | `Provider`, with `info()`, `list(request)` and `credential(request)` |
144
+ | `AccountLister`, `Rotator`, `AccessChecker` | override `accounts()`, `rotate(request)` or `check_access(request)` |
145
+ | `provider.Capabilities(p)` | `capabilities(p)`, a list of names |
146
+ | `provider.Main(p)`, `provider.Run(...)` | `main(p)`, `run(p, args, stdin, stdout, stderr)`, which also serve |
147
+ | `ctx.Done()`, `ctx.Err()` | `sleep(seconds)`, `canceled()` and the `Canceled` exception |
148
+ | `Info`, `SecretSpec`, `Account`, `Cluster`, `Endpoints`, `ExecConfig`, `Credential`, `Finding`, `Description` | dataclasses of the same names |
149
+ | `ListRequest`, `CredentialRequest`, `AccessRequest` | dataclasses of the same names |
150
+ | `Token`, `ClientCert`, `OIDC`, `Delegate` | `token`, `client_cert`, `oidc`, `delegate` |
151
+ | `NoExpiry` | `NO_EXPIRY`, which is `None` |
152
+ | `Credential.Validate`, `Credential.Expired` | `Credential.validate()`, which raises `ValueError`, and `Credential.expired()` |
153
+ | `Account.ConfigString` | `Account.config_string` |
154
+ | `*provider.Error`, `Errorf`, `WithHint` | `ProviderError(code, message, hint=..., retry_after_seconds=...)`, `with_hint` |
155
+ | `Unauthenticated`, `RateLimited`, ... | `UNAUTHENTICATED`, `RATE_LIMITED`, ... |
156
+ | `CodeOf`, `AsError`, `StatusError` | `code_of`, `as_error`, `status_error(status, message, headers)` |
157
+ | `EncodeRequest`, `DecodeList`, ... | `encode_request`, `decode_list`, ... |
158
+ | `OpList`, `CapRotate`, `SeverityError`, `ExitError`, ... | `OP_LIST`, `CAP_ROTATE`, `SEVERITY_ERROR`, `EXIT_ERROR`, ... |
159
+
160
+ Each type has `to_dict()`, which returns the JSON object the Go SDK writes,
161
+ and `from_dict()`, which reads one as the Go SDK does.
162
+
163
+ ### Optional operations
164
+
165
+ Override `accounts`, `rotate` or `check_access` to support it, and `describe`
166
+ lists its capability. Also add the capability to `plugin.yaml`: tug reads
167
+ capabilities from the manifest, so it never calls an operation the manifest
168
+ does not declare. An operation you do not override answers `Unimplemented`.
169
+
170
+ ```python
171
+ class AcmeProvider(Provider):
172
+ ...
173
+
174
+ def accounts(self):
175
+ # For example, read profiles from the vendor CLI's config file.
176
+ return [Account("work", description="acme CLI profile work", config={"profile": "work"})]
177
+
178
+ def check_access(self, request):
179
+ # request.cluster is the cluster as tug knows it; request.egress_ip
180
+ # is the caller's public address, or "".
181
+ return [
182
+ Finding(
183
+ SEVERITY_ERROR,
184
+ "the control plane allow list does not include " + request.egress_ip,
185
+ fix="add %s/32 to the allow list in the acme console" % request.egress_ip,
186
+ )
187
+ ]
188
+ ```
189
+
190
+ ### Credentials
191
+
192
+ `token(token, expires_at)`, `client_cert(cert_pem, key_pem, expires_at)`,
193
+ `oidc(issuer, client_id, *scopes, client_secret="")` and
194
+ `delegate(command, *args, env=None, install_hint="", ...)` build the four
195
+ shapes. Give anything short-lived its expiry as an aware `datetime`, which is
196
+ converted to UTC; pass `NO_EXPIRY` only for credentials that really do not
197
+ expire. A naive `datetime` is refused, because it could be in any time zone.
198
+ PEM may be `bytes` or `str`.
199
+
200
+ For an exec plugin that reads its cluster from `KUBERNETES_EXEC_INFO`, as
201
+ kubeconfig's `provideClusterInfo: true` arranges, pass
202
+ `provide_cluster_info=True`: tug then gives it the cluster's server, CA data,
203
+ TLS server name, insecure flag and proxy URL in `spec.cluster`.
204
+ `cluster_config`, any JSON value, becomes `spec.cluster.config`, which a
205
+ kubeconfig keeps in the cluster's `client.authentication.k8s.io/exec`
206
+ extension; it needs `provide_cluster_info`:
207
+
208
+ ```python
209
+ return delegate("acme-auth", "token", provide_cluster_info=True, cluster_config={"audience": request.cluster_id})
210
+ ```
211
+
212
+ ### Errors
213
+
214
+ Raise `ProviderError` with one of the codes; tug prints the code and message,
215
+ and the hint on a line of its own. Any other exception reaches tug as
216
+ `Internal`, with its traceback in tug's log (`tug --debug`). For HTTP APIs,
217
+ `status_error(status, message, headers)` maps 401 to `Unauthenticated`, 403
218
+ to `PermissionDenied`, 404 to `NotFound`, 429 to `RateLimited` (with the
219
+ `Retry-After` seconds), 408 and 5xx to `Unavailable`, 400 and 422 to
220
+ `InvalidConfig`, and anything else to `Internal`. `headers` may be urllib's
221
+ `HTTPMessage` or any mapping. Never put a secret in a message or hint: tug
222
+ prints and logs them.
223
+
224
+ `code_of(err)` and `as_error(err)` follow `raise ... from` chains, as Go's
225
+ `errors.As` follows wrapped errors:
226
+
227
+ ```python
228
+ try:
229
+ clusters = self.get(request, "/v1/clusters")
230
+ except Exception as e:
231
+ if code_of(e) == PERMISSION_DENIED:
232
+ raise as_error(e).with_hint("the token needs the clusters:read scope") from e
233
+ raise
234
+ ```
235
+
236
+ ### Timeouts, cancellation and output
237
+
238
+ tug gives `list` 30 seconds, `credential` 10 and `rotate` 60. In one-shot
239
+ mode it then sends SIGTERM, and `main` turns SIGTERM and SIGINT into a
240
+ `Canceled` exception, raised in the main thread wherever the provider is.
241
+ The SDK answers `Unavailable` and exits 1, so `finally` blocks and `with`
242
+ statements run on the way out. `Canceled` derives from `BaseException`,
243
+ like `KeyboardInterrupt`, so `except Exception` does not swallow it. Still
244
+ set a timeout on every network call: a call blocked inside C code may not
245
+ see the signal until it returns.
246
+
247
+ In serve mode a request is cancelled without a signal, and the SDK does not
248
+ interrupt the thread answering it. Instead `canceled()` turns true and
249
+ `sleep(seconds)`, a `time.sleep` that wakes when the request is cancelled,
250
+ raises `Canceled`. So wait with `sleep`, and check `canceled()` between the
251
+ steps of long work; both work in one-shot mode too.
252
+
253
+ ```python
254
+ from tug_provider import NO_EXPIRY, UNAVAILABLE, ProviderError, sleep, token
255
+
256
+
257
+ class AcmeProvider(Provider):
258
+ ...
259
+
260
+ def rotate(self, request):
261
+ old = self.get(request, "/v1/clusters/%s/token" % request.cluster_id)
262
+ self.post(request, "/v1/tokens/%s/revoke" % old["id"])
263
+ for _ in range(20):
264
+ new = self.get(request, "/v1/clusters/%s/token" % request.cluster_id)
265
+ if new["id"] != old["id"]:
266
+ return token(new["token"], NO_EXPIRY)
267
+ sleep(2) # raises Canceled when tug gives up on the request
268
+ raise ProviderError(UNAVAILABLE, "no new token yet", hint="run tug auth token again in a minute")
269
+ ```
270
+
271
+ stdout carries exactly one JSON response, or in serve mode one per line. While the provider runs, `main`
272
+ points `sys.stdout` at stderr, so a stray `print()` ends up in tug's log
273
+ rather than breaking the protocol. Log with `logging` or
274
+ `print(..., file=sys.stderr)`; users see it with `tug --debug`. Child
275
+ processes still inherit the real stdout, so capture their output.
276
+
277
+ ## Serve mode
278
+
279
+ Every plugin built with the SDK can serve: tug starts it once as
280
+ `tug-provider-acme serve` and sends many requests, which saves a process
281
+ start per call. tug does so only when `plugin.yaml` lists `serve` in its
282
+ capabilities, so add it there; `describe` always lists it. There is nothing
283
+ else to implement.
284
+
285
+ In serve mode each line of stdin is one frame, `{"id", "op", "request"}`,
286
+ and the SDK answers each request in a thread of its own with one line,
287
+ `{"id", "response"}`, whose response is exactly what one-shot mode would
288
+ write. Answers may come out of order, and lines never interleave. A frame
289
+ `{"id", "cancel": true}` cancels that request. An unknown operation gets an
290
+ `InvalidConfig` error with the request's id, and a line that is not JSON or
291
+ has no id gets one with an empty id; serving carries on. When stdin closes,
292
+ the plugin finishes the requests in flight and exits 0. On SIGTERM or SIGINT
293
+ it cancels them all, gives them a second to answer, and exits 0 without the
294
+ rest.
295
+
296
+ Because requests run concurrently, a provider's methods must be safe to run
297
+ at the same time: keep per-request state in local variables, and guard
298
+ anything shared, such as a cache, with a lock.
299
+
300
+ ## Testing
301
+
302
+ `run` is `main` with explicit arguments and streams, and `encode_request`
303
+ and the `decode_*` functions build and read messages as tug does, so a test
304
+ calls an operation exactly as tug would, in process:
305
+
306
+ ```python
307
+ import io
308
+ import unittest
309
+
310
+ from tug_provider import OP_LIST, UNAUTHENTICATED, Account, ListRequest, ProviderError, decode_list, encode_request, run
311
+
312
+
313
+ def call(op, payload=None):
314
+ out, err = io.BytesIO(), io.StringIO()
315
+ run(AcmeProvider(), [op], io.BytesIO(encode_request(op, payload)), out, err)
316
+ return out.getvalue()
317
+
318
+
319
+ class ListTest(unittest.TestCase):
320
+ def test_no_token(self):
321
+ out = call(OP_LIST, ListRequest(account=Account("demo")))
322
+ with self.assertRaises(ProviderError) as cm:
323
+ decode_list(out)
324
+ self.assertEqual(cm.exception.code, UNAUTHENTICATED)
325
+ ```
326
+
327
+ Point the plugin at a fake API (`http.server` on a loopback port, through an
328
+ `apiURL` account setting), never the real one. Then run the conformance suite
329
+ with `tug plugin test`, as the plugin guide's
330
+ [testing section](https://github.com/paingha/tugcli/blob/main/docs/plugins.md#9-testing)
331
+ describes.
332
+
333
+ ## Shipping a plugin
334
+
335
+ `tug plugin package` archives `plugin.yaml` and the files its platforms name,
336
+ nothing else, so an installed plugin cannot import an SDK that sat beside it.
337
+ Either ask users to install `tug-provider` for the `python3` on their `PATH`,
338
+ or bundle the SDK and your script into one executable file with the standard
339
+ library's `zipapp`:
340
+
341
+ ```bash
342
+ mkdir -p build/app bin
343
+ cp tug-provider-acme build/app/__main__.py
344
+ python3 -m pip install --target build/app tug-provider # or copy sdk/python/tug_provider into build/app
345
+ python3 -m zipapp build/app -c -p '/usr/bin/env python3' -o bin/tug-provider-acme
346
+ ```
347
+
348
+ Point every platform in `plugin.yaml` at `bin/tug-provider-acme`, then run
349
+ `tug plugin package . -o dist`. Windows cannot run the file directly, so leave
350
+ it out of the platforms.
351
+
352
+ ## Differences from the Go SDK
353
+
354
+ - There is no context. In one-shot mode cancellation arrives as `Canceled`,
355
+ raised by the signal handlers that `main` installs; `run` installs none.
356
+ In serve mode a cancelled request is flagged rather than interrupted, and
357
+ only `sleep` raises `Canceled`, as a Go provider that ignores its context
358
+ keeps running. Raising an exception in another thread needs a
359
+ CPython-only call, can land in the SDK's own code after the provider has
360
+ returned, and still cannot interrupt a blocking call.
361
+ - After SIGTERM in serve mode, requests in flight get one second to answer;
362
+ the Go SDK waits for them all. tug kills a plugin 2 seconds after
363
+ SIGTERM.
364
+ - `ServeRequest` and `ServeResponse`, which Go exports for hosts, have no
365
+ Python counterpart, since a plugin never builds frames itself.
366
+ - `main` sends stray `print()` output to stderr; the Go SDK leaves stdout to
367
+ the plugin.
368
+ - Times are `datetime`, which holds microseconds, so finer fractions in a
369
+ request are truncated. Naive times are refused.
370
+ - Exceptions other than `ProviderError` become `Internal` with their type in
371
+ the message (`KeyError: 'api_token'`) and their traceback on stderr.
372
+ - A response that cannot be encoded, such as one with `NaN` in an account's
373
+ config, is sent as an `Internal` error rather than leaving stdout empty.
374
+ - Request field names must match exactly. Go also accepts them in another
375
+ case, which tug never sends.
376
+ - Numbers in free-form account config keep Python's formatting, so `1.0`
377
+ stays `1.0` where Go writes `1`; both are the same JSON number.
378
+ - `cluster_config` is a decoded value, written with its object keys sorted
379
+ as Go writes maps. Go's `ClusterConfig` is raw JSON, written as the
380
+ plugin gave it. A `null` cluster config is left out.
381
+
382
+ ## Developing the SDK
383
+
384
+ ```bash
385
+ cd sdk/python && python3 -m unittest discover # or, from the top: make python-sdk-test
386
+ go test ./sdk/python # conformance and byte-for-byte checks against the Go SDK
387
+ ```
388
+
389
+ [`examples/tug-provider-pyexample`](https://github.com/paingha/tugcli/tree/main/sdk/python/examples/tug-provider-pyexample)
390
+ is a complete plugin. From a checkout it finds the SDK two directories up, so
391
+ `tug plugin link sdk/python/examples/tug-provider-pyexample` works without
392
+ installing anything.
393
+
394
+ ## Releasing
395
+
396
+ [`publish-python-sdk.yml`](https://github.com/paingha/tugcli/blob/main/.github/workflows/publish-python-sdk.yml)
397
+ publishes the SDK when a tag `sdk/python/v<version>` is pushed, where
398
+ `<version>` is `__version__` in `tug_provider/__init__.py`, which
399
+ `pyproject.toml` reads. It checks the tag against that version, runs the
400
+ unit tests on Python 3.9 and the newest Python, builds the sdist and wheel
401
+ with `python -m build`, publishes them to PyPI, and attaches them to a
402
+ GitHub release for the tag. That release is not marked latest, so tug's own
403
+ releases stay the latest.
404
+
405
+ ```bash
406
+ # after the commit that sets __version__ = "0.1.0" is on main
407
+ git tag sdk/python/v0.1.0
408
+ git push origin sdk/python/v0.1.0
409
+ ```
410
+
411
+ PyPI accepts the upload through trusted publishing: it trusts the
412
+ workflow's OpenID Connect token, so no API token is stored anywhere. Before
413
+ the first release, set it up once:
414
+
415
+ 1. On PyPI, sign in as the account that is to own the project, open
416
+ [Publishing](https://pypi.org/manage/account/publishing/) and add a
417
+ pending publisher of type GitHub. It is pending because the project does
418
+ not exist until the first upload creates it; from then on it is listed
419
+ under the project's own Publishing settings.
420
+
421
+ | Field | Value |
422
+ | --- | --- |
423
+ | PyPI Project Name | `tug-provider` |
424
+ | Owner | `paingha` |
425
+ | Repository name | `tugcli` |
426
+ | Workflow name | `publish-python-sdk.yml` |
427
+ | Environment name | `pypi` |
428
+
429
+ 2. On GitHub, add an environment named `pypi` under the repository's
430
+ Settings, Environments. Under deployment branches and tags, choose
431
+ selected branches and tags and add the tag rule `sdk/python/v*`, so that
432
+ only a release tag can publish. Add required reviewers too if you want a
433
+ person to approve each upload and the plan offers them.
434
+
435
+ Until both are in place, the publish job fails with an `invalid-publisher`
436
+ error, and nothing reaches PyPI or the releases page.