@colony2/jobdb 0.0.16 → 0.0.18
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -100
- package/package.json +13 -13
package/README.md
CHANGED
|
@@ -23,61 +23,9 @@ jobdb --help
|
|
|
23
23
|
|
|
24
24
|
## Container Image
|
|
25
25
|
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
ghcr.io/colony-2/jobdb:<release-tag>
|
|
30
|
-
```
|
|
31
|
-
|
|
32
|
-
The image has a fixed `jobdb serve` entrypoint for stateless production use. It
|
|
33
|
-
always stores runtime records in Postgres and artifact bytes larger than the
|
|
34
|
-
inline threshold in S3, Google Cloud Storage, or Azure Blob Storage. The
|
|
35
|
-
supported container interface cannot select SQLite, the toy runtime, local
|
|
36
|
-
filesystem storage, or memory storage. Those modes remain available when using
|
|
37
|
-
the installed `jobdb` binary outside the container.
|
|
38
|
-
|
|
39
|
-
Required configuration:
|
|
40
|
-
|
|
41
|
-
| Variable | Default | Description |
|
|
42
|
-
| --- | --- | --- |
|
|
43
|
-
| `JOBDB_POSTGRES_DSN` | none | Postgres connection string. |
|
|
44
|
-
| `JOBDB_BLOB_STORE_URI` | none | Remote `s3://`, `gs://`, or `azblob://` bucket/container URI. |
|
|
45
|
-
| `JOBDB_LEASE_TOKEN_SIGNING_KEY` | none | Base64-encoded signing key containing at least 32 random bytes. |
|
|
46
|
-
| `JOBDB_LEASE_TOKEN_SIGNING_KEY_FILE` | none | Mounted file containing the signing key; use instead of the direct variable. |
|
|
47
|
-
| `JOBDB_LISTEN` | `0.0.0.0:8080` | HTTP listen address. |
|
|
48
|
-
| `JOBDB_MAX_INLINE_ARTIFACT_BYTES` | `4096` | Largest artifact size retained inline in Postgres. |
|
|
49
|
-
|
|
50
|
-
All replicas in one deployment must use the same signing key. Generate one and
|
|
51
|
-
store it in the deployment's secret manager rather than in an image or manifest:
|
|
52
|
-
|
|
53
|
-
```bash
|
|
54
|
-
openssl rand -base64 32
|
|
55
|
-
```
|
|
56
|
-
|
|
57
|
-
For example, with Postgres and the S3 bucket already available:
|
|
58
|
-
|
|
59
|
-
```bash
|
|
60
|
-
docker run --rm \
|
|
61
|
-
--read-only \
|
|
62
|
-
--tmpfs /tmp:rw,noexec,nosuid,size=64m \
|
|
63
|
-
--publish 8080:8080 \
|
|
64
|
-
--env JOBDB_POSTGRES_DSN \
|
|
65
|
-
--env JOBDB_BLOB_STORE_URI='s3://jobdb-artifacts?region=us-east-1' \
|
|
66
|
-
--env JOBDB_LEASE_TOKEN_SIGNING_KEY \
|
|
67
|
-
ghcr.io/colony-2/jobdb:<release-tag>
|
|
68
|
-
```
|
|
69
|
-
|
|
70
|
-
The image runs as UID/GID `65532:65532`, declares no volume, and supports a
|
|
71
|
-
read-only root filesystem. It exposes `GET /healthz` and includes an automatic
|
|
72
|
-
container health check. Operators can also check any JobDB server directly:
|
|
73
|
-
|
|
74
|
-
```bash
|
|
75
|
-
jobdb healthcheck http://jobdb.example:8080
|
|
76
|
-
```
|
|
77
|
-
|
|
78
|
-
Use an exact release tag or image digest in production. Provider credentials
|
|
79
|
-
are resolved by the cloud SDKs; prefer workload identity, instance/task roles,
|
|
80
|
-
or mounted secrets over credentials embedded in the blob URI.
|
|
26
|
+
The Postgres container is released from the
|
|
27
|
+
[`pgjobdb` repository](https://github.com/colony-2/pgjobdb) as
|
|
28
|
+
`ghcr.io/colony-2/pgjobdb:<release-tag>`.
|
|
81
29
|
|
|
82
30
|
## Quick Start
|
|
83
31
|
|
|
@@ -137,57 +85,23 @@ for durable execution.
|
|
|
137
85
|
jobdb toy --listen 127.0.0.1:9047
|
|
138
86
|
```
|
|
139
87
|
|
|
140
|
-
###
|
|
88
|
+
### Postgres
|
|
141
89
|
|
|
142
|
-
The
|
|
143
|
-
|
|
144
|
-
|
|
90
|
+
The base `jobdb` CLI provides SQLite and toy backends. The Postgres runtime,
|
|
91
|
+
`direct` and `serve` commands, and container image are provided by
|
|
92
|
+
[`pgjobdb`](https://github.com/colony-2/pgjobdb). It composes JobDB's public
|
|
93
|
+
runtime core with the typed Postgres scheduler.
|
|
145
94
|
|
|
146
|
-
|
|
147
|
-
JOBDB_POSTGRES_DSN='postgres://user:pass@localhost:5432/jobdb?sslmode=disable' \
|
|
148
|
-
jobdb direct --blob-store-uri 's3://jobdb-artifacts?region=us-east-1' --listen 127.0.0.1:9047
|
|
149
|
-
```
|
|
150
|
-
|
|
151
|
-
Flags:
|
|
152
|
-
|
|
153
|
-
- `--postgres-dsn`: Postgres DSN for `pgwf` state.
|
|
154
|
-
- `--blob-store-uri`: blob bucket URL for large artifacts. The `jobdb`
|
|
155
|
-
executable includes Go CDK providers, so it supports `file://`, `gs://`,
|
|
156
|
-
`s3://`, and `azblob://`; defaults to local `blobfs://`.
|
|
157
|
-
- `--listen`: HTTP listen address. Defaults to `127.0.0.1:9047`.
|
|
158
|
-
|
|
159
|
-
Environment:
|
|
160
|
-
|
|
161
|
-
- `JOBDB_POSTGRES_DSN`: Postgres DSN used when `--postgres-dsn` is not set.
|
|
162
|
-
|
|
163
|
-
Blob URL examples:
|
|
164
|
-
|
|
165
|
-
- Local filesystem: `file:///var/lib/jobdb/blobs` or legacy
|
|
166
|
-
`blobfs:///var/lib/jobdb/blobs`.
|
|
167
|
-
- Google Cloud Storage: `gs://jobdb-artifacts?prefix=prod/`.
|
|
168
|
-
- Amazon S3: `s3://jobdb-artifacts?region=us-east-1&prefix=prod/`.
|
|
169
|
-
- Azure Blob Storage: `azblob://jobdb-artifacts?prefix=prod/`.
|
|
170
|
-
|
|
171
|
-
Credential resolution is handled by the Go CDK provider drivers, so `jobdb`
|
|
172
|
-
does not need separate credential flags:
|
|
173
|
-
|
|
174
|
-
- GCS uses Application Default Credentials. Use
|
|
175
|
-
`GOOGLE_APPLICATION_CREDENTIALS`, `gcloud auth application-default login`, or
|
|
176
|
-
attached Google Cloud service account credentials in VM/container
|
|
177
|
-
environments.
|
|
178
|
-
- S3 uses the AWS SDK for Go v2 configuration chain. Provide `AWS_REGION` and
|
|
179
|
-
credentials through environment variables, shared `~/.aws/config` and
|
|
180
|
-
`~/.aws/credentials` profiles, or attached instance/task roles.
|
|
181
|
-
- Azure Blob Storage uses Go CDK's Azure driver. Provide
|
|
182
|
-
`AZURE_STORAGE_ACCOUNT` with `AZURE_STORAGE_KEY`, a connection string, a SAS
|
|
183
|
-
token, or Azure default credentials such as environment credentials, Azure
|
|
184
|
-
CLI credentials, or managed identity.
|
|
185
|
-
|
|
186
|
-
Library embedders of `runtime/sqlite` or `runtime/direct` only get `blobfs://`
|
|
95
|
+
Library embedders of `runtime/sqlite` only gets `blobfs://`
|
|
187
96
|
support by default. Import
|
|
188
97
|
`github.com/colony-2/jobdb/pkg/jobdb/blobstore/gocdk` from executable/server
|
|
189
98
|
code to enable Go CDK provider URI registration.
|
|
190
99
|
|
|
100
|
+
Library users can import `github.com/colony-2/pgjobdb/pkg/pgjobdb/runtime`
|
|
101
|
+
for Postgres or import another runtime implementation. The public
|
|
102
|
+
`github.com/colony-2/jobdb/pkg/jobdb` and `runtime/core` packages do not
|
|
103
|
+
import pgjobdb.
|
|
104
|
+
|
|
191
105
|
References:
|
|
192
106
|
|
|
193
107
|
- [Go CDK blob storage guide](https://gocloud.dev/howto/blob/)
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@colony2/jobdb",
|
|
3
3
|
"type": "module",
|
|
4
|
-
"version": "0.0.
|
|
4
|
+
"version": "0.0.18",
|
|
5
5
|
"description": "jobdb job system",
|
|
6
6
|
"scripts": {
|
|
7
7
|
"postinstall": "node install.js",
|
|
@@ -30,51 +30,51 @@
|
|
|
30
30
|
},
|
|
31
31
|
"archives": {
|
|
32
32
|
"darwin-arm64": {
|
|
33
|
-
"name": "jobdb_0.0.
|
|
34
|
-
"url": "https://github.com/colony-2/jobdb/releases/download/v0.0.
|
|
33
|
+
"name": "jobdb_0.0.18_Darwin_arm64.tar.gz",
|
|
34
|
+
"url": "https://github.com/colony-2/jobdb/releases/download/v0.0.18/jobdb_0.0.18_Darwin_arm64.tar.gz",
|
|
35
35
|
"bins": [
|
|
36
36
|
"jobdb"
|
|
37
37
|
],
|
|
38
38
|
"format": "tar.gz",
|
|
39
39
|
"checksum": {
|
|
40
40
|
"algorithm": "sha256",
|
|
41
|
-
"digest": "
|
|
41
|
+
"digest": "ffcd9de3b492a7b8ae4d6967e8f56fb8b7d35b58b6aa5b7c58a9f4a164e313cd"
|
|
42
42
|
}
|
|
43
43
|
},
|
|
44
44
|
"darwin-x64": {
|
|
45
|
-
"name": "jobdb_0.0.
|
|
46
|
-
"url": "https://github.com/colony-2/jobdb/releases/download/v0.0.
|
|
45
|
+
"name": "jobdb_0.0.18_Darwin_x86_64.tar.gz",
|
|
46
|
+
"url": "https://github.com/colony-2/jobdb/releases/download/v0.0.18/jobdb_0.0.18_Darwin_x86_64.tar.gz",
|
|
47
47
|
"bins": [
|
|
48
48
|
"jobdb"
|
|
49
49
|
],
|
|
50
50
|
"format": "tar.gz",
|
|
51
51
|
"checksum": {
|
|
52
52
|
"algorithm": "sha256",
|
|
53
|
-
"digest": "
|
|
53
|
+
"digest": "c708a3f41021da03271074415642d8c02a070a9f6766d6e2d7f854a52cb3798c"
|
|
54
54
|
}
|
|
55
55
|
},
|
|
56
56
|
"linux-arm64": {
|
|
57
|
-
"name": "jobdb_0.0.
|
|
58
|
-
"url": "https://github.com/colony-2/jobdb/releases/download/v0.0.
|
|
57
|
+
"name": "jobdb_0.0.18_Linux_arm64.tar.gz",
|
|
58
|
+
"url": "https://github.com/colony-2/jobdb/releases/download/v0.0.18/jobdb_0.0.18_Linux_arm64.tar.gz",
|
|
59
59
|
"bins": [
|
|
60
60
|
"jobdb"
|
|
61
61
|
],
|
|
62
62
|
"format": "tar.gz",
|
|
63
63
|
"checksum": {
|
|
64
64
|
"algorithm": "sha256",
|
|
65
|
-
"digest": "
|
|
65
|
+
"digest": "044be0eed4842c3243137eac77727084501258e4276caebe86393fcd72a4d650"
|
|
66
66
|
}
|
|
67
67
|
},
|
|
68
68
|
"linux-x64": {
|
|
69
|
-
"name": "jobdb_0.0.
|
|
70
|
-
"url": "https://github.com/colony-2/jobdb/releases/download/v0.0.
|
|
69
|
+
"name": "jobdb_0.0.18_Linux_x86_64.tar.gz",
|
|
70
|
+
"url": "https://github.com/colony-2/jobdb/releases/download/v0.0.18/jobdb_0.0.18_Linux_x86_64.tar.gz",
|
|
71
71
|
"bins": [
|
|
72
72
|
"jobdb"
|
|
73
73
|
],
|
|
74
74
|
"format": "tar.gz",
|
|
75
75
|
"checksum": {
|
|
76
76
|
"algorithm": "sha256",
|
|
77
|
-
"digest": "
|
|
77
|
+
"digest": "82d89205d808157635f28d8fa6aaa56eebcc5cf989238263d1a3a07cb9bbf88c"
|
|
78
78
|
}
|
|
79
79
|
}
|
|
80
80
|
},
|