@skillcdn/console 0.1.6 → 0.1.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +50 -2
- package/SKILLCDN.md +35 -0
- package/dist/cli/cli.d.ts +3 -0
- package/dist/cli/cli.d.ts.map +1 -1
- package/dist/cli/cli.js +61 -0
- package/dist/cli/cli.js.map +1 -1
- package/dist/cli/help.d.ts.map +1 -1
- package/dist/cli/help.js +5 -0
- package/dist/cli/help.js.map +1 -1
- package/dist/cli/main.js +2 -0
- package/dist/cli/main.js.map +1 -1
- package/dist/cli/serve.d.ts +37 -0
- package/dist/cli/serve.d.ts.map +1 -0
- package/dist/cli/serve.js +218 -0
- package/dist/cli/serve.js.map +1 -0
- package/dist/console.d.ts +6 -0
- package/dist/console.d.ts.map +1 -1
- package/dist/console.js +78 -13
- package/dist/console.js.map +1 -1
- package/dist/data.d.ts.map +1 -1
- package/dist/data.js +5 -1
- package/dist/data.js.map +1 -1
- package/dist/i18n/en.d.ts +4 -0
- package/dist/i18n/en.d.ts.map +1 -1
- package/dist/i18n/en.js +4 -0
- package/dist/i18n/en.js.map +1 -1
- package/dist/i18n/ko.d.ts.map +1 -1
- package/dist/i18n/ko.js +4 -0
- package/dist/i18n/ko.js.map +1 -1
- package/dist/schemas.d.ts +1 -0
- package/dist/schemas.d.ts.map +1 -1
- package/dist/schemas.js +5 -0
- package/dist/schemas.js.map +1 -1
- package/dist/web.d.ts +16 -0
- package/dist/web.d.ts.map +1 -0
- package/dist/web.js +149 -0
- package/dist/web.js.map +1 -0
- package/docs/specs/cli.md +92 -0
- package/docs/specs/rest.md +221 -0
- package/package.json +14 -3
- package/skills/building-a-console/SKILL.md +102 -0
- package/skills/building-a-console/references/the-default-console.md +140 -0
- package/skills/deploying-a-console/SKILL.md +130 -0
- package/skills/deploying-a-console/assets/deployer-policy.json +118 -0
- package/skills/deploying-a-console/assets/secrets.example.json +11 -0
- package/skills/deploying-a-console/references/access.md +78 -0
- package/skills/deploying-a-console/references/resources.md +516 -0
- package/skills/deploying-a-console/references/sizing.md +90 -0
- package/skills/deploying-a-console/scripts/import-aws-key.mjs +35 -0
- package/skills/deploying-a-console/scripts/store-secrets.mjs +114 -0
- package/skills/working-the-board/SKILL.md +86 -0
- package/src/api.ts +235 -0
- package/src/cli/cli.ts +1114 -0
- package/src/cli/credentials.ts +68 -0
- package/src/cli/directory.ts +59 -0
- package/src/cli/format.ts +284 -0
- package/src/cli/help.ts +326 -0
- package/src/cli/main.ts +39 -0
- package/src/cli/serve.ts +276 -0
- package/src/cli/terminal.ts +28 -0
- package/src/client.ts +533 -0
- package/src/components/board.tsx +112 -0
- package/src/components/connect.tsx +126 -0
- package/src/components/decision-form.tsx +136 -0
- package/src/components/decision-list.tsx +278 -0
- package/src/components/documents.tsx +569 -0
- package/src/components/event-feed.tsx +129 -0
- package/src/components/markdown.tsx +72 -0
- package/src/components/people.tsx +59 -0
- package/src/components/projects.tsx +364 -0
- package/src/components/runs.tsx +175 -0
- package/src/components/shell.tsx +284 -0
- package/src/components/sign-in.tsx +61 -0
- package/src/components/skills.tsx +78 -0
- package/src/components/task-form.tsx +229 -0
- package/src/components/task-view.tsx +272 -0
- package/src/components/tokens.tsx +223 -0
- package/src/components/ui.tsx +219 -0
- package/src/connect-page.tsx +148 -0
- package/src/connect.ts +27 -0
- package/src/console.tsx +1166 -0
- package/src/data.ts +493 -0
- package/src/docs-page.tsx +334 -0
- package/src/documents.ts +76 -0
- package/src/i18n/en.ts +537 -0
- package/src/i18n/index.ts +76 -0
- package/src/i18n/ko.ts +511 -0
- package/src/index.ts +142 -0
- package/src/limits.ts +61 -0
- package/src/router.ts +295 -0
- package/src/routes.ts +106 -0
- package/src/schemas.ts +776 -0
- package/src/styles/console.css +2432 -0
- package/src/text.ts +33 -0
- package/src/use-action.ts +37 -0
- package/src/vocabulary.ts +109 -0
- package/src/web.ts +170 -0
|
@@ -0,0 +1,516 @@
|
|
|
1
|
+
# The phases in detail: procedure, checks, failures
|
|
2
|
+
|
|
3
|
+
What each phase of the skill does, command by command, with its checks and what to do when a step fails. Placeholders are in angle brackets. The run keeps its notes (ids, names, endpoints) in `scratch/notes.md` of the run's folder; `DEPLOY.md` is written from them at the end. Facts that move carry the month they were checked; the vendor's reference is the source when one has moved.
|
|
4
|
+
|
|
5
|
+
## Conventions
|
|
6
|
+
|
|
7
|
+
The run's folder is the deployment repository to be; `scratch/` in it holds the scripts, the request bodies and the notes, and never enters git. Every AWS command runs as the deployer, in the region of the plan; every Cloudflare call carries the token from its file. Set once per shell, in the run's folder:
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
export MSYS_NO_PATHCONV=1 # Git Bash on Windows: keep /console and the like as written
|
|
11
|
+
export AWS_PROFILE=console AWS_REGION=<region>
|
|
12
|
+
export CLOUDFLARE_API_TOKEN=$(tr -d '\r\n ' < .cf-token)
|
|
13
|
+
cf() { curl -sS "https://api.cloudflare.com/client/v4$1" -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" -H "Content-Type: application/json" "${@:2}"; }
|
|
14
|
+
ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
|
|
15
|
+
SECRETS=../console-secrets.json # outside the repository, in the shape of assets/secrets.example.json
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
A value that must not be seen goes through a file or a pipe, never a command line: `aws ... --cli-input-json file://scratch/<file>` for a request that carries one, and the two scripts of this skill ([import-aws-key.mjs](../scripts/import-aws-key.mjs), [store-secrets.mjs](../scripts/store-secrets.mjs)), copied to `scratch/`, for the key and the parameters. A response that carries one is piped straight into the secrets file:
|
|
19
|
+
|
|
20
|
+
```sh
|
|
21
|
+
set_secret() { node scratch/store-secrets.mjs set "$SECRETS" "$1"; } # reads the value from standard input
|
|
22
|
+
node -e "process.stdout.write(require('node:crypto').randomBytes(36).toString('base64url'))" | set_secret AUTH_SECRET
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
## Phase 1: Access
|
|
26
|
+
|
|
27
|
+
1. The folder: `mkdir -p <folder>/scratch && cd <folder> && git init -b main`, and `.gitignore` with `.cf-token`, `.aws-key.csv`, `scratch/`, `*.pem`, `console-secrets.json`, `node_modules/`, `console-ui/dist/`, `.env`, `.env.*`, before anything else is written.
|
|
28
|
+
2. The command and the scripts: `npm install -g @skillcdn/console`, then copy `$(npm root -g)/@skillcdn/console/skills/deploying-a-console/scripts/*.mjs` into `scratch/` (or write them there from the connection's copy of the skill).
|
|
29
|
+
3. The key: the person saved `.aws-key.csv` in the folder ([access.md](access.md)). `node scratch/import-aws-key.mjs .aws-key.csv console` writes the profile and deletes the file. Then `aws configure set region <region> --profile console`.
|
|
30
|
+
4. `aws sts get-caller-identity`: the account id and `user/console-deployer`. A refusal means the key is wrong or not yet active (a minute after creation).
|
|
31
|
+
5. The network: `aws ec2 describe-vpcs --filters Name=is-default,Values=true --query 'Vpcs[0].VpcId' --output text`. `None`: `aws ec2 create-default-vpc`. Then the subnets: `aws ec2 describe-subnets --filters Name=vpc-id,Values=<vpc> Name=default-for-az,Values=true --query 'Subnets[].SubnetId' --output text`.
|
|
32
|
+
6. The token: `cf /user/tokens/verify` answers `"status": "active"`; `cf /accounts` the account and its id (`CF_ACCOUNT`); `cf "/zones?name=<domain>"` the zone and its `status`, or an empty list when the domain is not on Cloudflare yet.
|
|
33
|
+
7. The git host: `gh auth status`; the owner the person named is theirs or an organization they are in (`gh api user`, `gh api user/orgs`).
|
|
34
|
+
8. The tools: `node --version` (24 or newer), `git --version`, `jq --version`, `curl --version`.
|
|
35
|
+
|
|
36
|
+
Report what is missing in one message, with the row of access.md to add, before the plan.
|
|
37
|
+
|
|
38
|
+
## Phase 2: Plan and estimate
|
|
39
|
+
|
|
40
|
+
Nothing is created. The size from the answer and its numbers ([sizing.md](sizing.md) "The sizes"); the prices read and the estimate computed ("Reading today's prices"); the names:
|
|
41
|
+
|
|
42
|
+
| Resource | Name |
|
|
43
|
+
|---|---|
|
|
44
|
+
| Cluster; services; task families | `console`; `api`, `worker`; `console-api`, `console-worker`, `console-migrate` |
|
|
45
|
+
| Database instance, subnet group, user, database | `console`, `console`, `console`, `console` |
|
|
46
|
+
| Bucket, files user | `console-files-<account id>` (bucket names are global; the account id makes it unique), `console-files` |
|
|
47
|
+
| Image repository, log group | `console`, `/console` |
|
|
48
|
+
| Roles | `console-task-execution`, `console-task`, `console-github-deploy` |
|
|
49
|
+
| Security groups | `console-tasks`, `console-db` |
|
|
50
|
+
| Parameters | `/console/<NAME>`, one per secret of the secrets file |
|
|
51
|
+
| Tunnel, hostname | `console`, `<hostname>` |
|
|
52
|
+
| Deployment repository | `<owner>/<the folder's name>`, private |
|
|
53
|
+
|
|
54
|
+
The plan message is the estimate message of sizing.md followed by the line in the skill's phase 2.
|
|
55
|
+
|
|
56
|
+
## Phase 3: Foundation
|
|
57
|
+
|
|
58
|
+
### The zone, first
|
|
59
|
+
|
|
60
|
+
Not on Cloudflare yet: `cf /zones -X POST --data '{"name":"<domain>","account":{"id":"<CF_ACCOUNT>"},"type":"full"}'`; the answer's `name_servers` go to the person with the steps for their registrar (access.md "The domain"), the run goes on, and the hostname answers once `cf "/zones?name=<domain>"` says `"status": "active"`. A domain to buy: `cf "/accounts/<CF_ACCOUNT>/registrar/domain-search?q=<words>"` to find one, `cf "/accounts/<CF_ACCOUNT>/registrar/domain-check" -X POST --data '{"domains":["<domain>"]}'` for the live price, the checkpoint with the price and the renewal, then `cf "/accounts/<CF_ACCOUNT>/registrar/registrations" -X POST --data '{"domain_name":"<domain>"}'`; a `202` is polled at `.../registrations/<domain>/registration-status`; `extension_not_supported_via_api` means the dashboard (Domain Registration, Register Domains). A bought domain's zone is active at once (the Registrar API was in beta when checked, 2026-10). Either way: `ZONE=$(cf "/zones?name=<domain>" | jq -r '.result[0].id')`.
|
|
61
|
+
|
|
62
|
+
### The network
|
|
63
|
+
|
|
64
|
+
```sh
|
|
65
|
+
VPC=<vpc>; SUBNETS=<subnet ids, comma-separated>
|
|
66
|
+
TASKS_SG=$(aws ec2 create-security-group --group-name console-tasks --description "console tasks: no inbound" --vpc-id $VPC --query GroupId --output text)
|
|
67
|
+
DB_SG=$(aws ec2 create-security-group --group-name console-db --description "console database: from the tasks only" --vpc-id $VPC --query GroupId --output text)
|
|
68
|
+
aws ec2 authorize-security-group-ingress --group-id $DB_SG --protocol tcp --port 5432 --source-group $TASKS_SG
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
A new group allows all egress and no ingress; the tasks' group stays so. Check: `aws ec2 describe-security-groups --group-ids $TASKS_SG --query 'SecurityGroups[0].IpPermissions'` is `[]`.
|
|
72
|
+
|
|
73
|
+
### The database
|
|
74
|
+
|
|
75
|
+
Start it now; it takes five to fifteen minutes, during which the rest below is made.
|
|
76
|
+
|
|
77
|
+
```sh
|
|
78
|
+
aws rds create-db-subnet-group --db-subnet-group-name console --db-subnet-group-description console --subnet-ids <the subnet ids, space-separated>
|
|
79
|
+
node -e "process.stdout.write(require('node:crypto').randomBytes(24).toString('base64url'))" | set_secret DATABASE_PASSWORD
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
Write `scratch/db.template.json` from this template, with the size's class, and render it with the password from the secrets file (`node scratch/store-secrets.mjs render "$SECRETS" scratch/db.template.json scratch/db.json`, which fills `{{DATABASE_PASSWORD}}`):
|
|
83
|
+
|
|
84
|
+
```json
|
|
85
|
+
{
|
|
86
|
+
"DBInstanceIdentifier": "console",
|
|
87
|
+
"Engine": "postgres",
|
|
88
|
+
"EngineVersion": "18",
|
|
89
|
+
"DBInstanceClass": "<db.t4g.micro | db.t4g.small | db.t4g.medium>",
|
|
90
|
+
"AllocatedStorage": 20,
|
|
91
|
+
"MaxAllocatedStorage": <100 | 100 | 200>,
|
|
92
|
+
"StorageType": "gp3",
|
|
93
|
+
"StorageEncrypted": true,
|
|
94
|
+
"MasterUsername": "console",
|
|
95
|
+
"MasterUserPassword": "{{DATABASE_PASSWORD}}",
|
|
96
|
+
"DBName": "console",
|
|
97
|
+
"DBSubnetGroupName": "console",
|
|
98
|
+
"VpcSecurityGroupIds": ["<db security group>"],
|
|
99
|
+
"PubliclyAccessible": false,
|
|
100
|
+
"MultiAZ": <false, or true for two zones>,
|
|
101
|
+
"BackupRetentionPeriod": 7,
|
|
102
|
+
"DeletionProtection": true,
|
|
103
|
+
"AutoMinorVersionUpgrade": true,
|
|
104
|
+
"CopyTagsToSnapshot": true,
|
|
105
|
+
"Tags": [{ "Key": "app", "Value": "console" }]
|
|
106
|
+
}
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
The major version alone picks the region's current minor of PostgreSQL 18 (checked 2026-10; `aws rds describe-db-engine-versions --engine postgres --engine-version 18` lists what that is).
|
|
110
|
+
|
|
111
|
+
```sh
|
|
112
|
+
aws rds create-db-instance --cli-input-json file://scratch/db.json && rm scratch/db.json
|
|
113
|
+
# ... the bucket, the roles, the cluster and Cloudflare meanwhile ...
|
|
114
|
+
aws rds wait db-instance-available --db-instance-identifier console
|
|
115
|
+
ENDPOINT=$(aws rds describe-db-instances --db-instance-identifier console --query 'DBInstances[0].Endpoint.Address' --output text)
|
|
116
|
+
curl -sSf "https://truststore.pki.rds.amazonaws.com/$AWS_REGION/$AWS_REGION-bundle.pem" | set_secret DATABASE_CA
|
|
117
|
+
node scratch/store-secrets.mjs derive "$SECRETS" DATABASE_URL "postgres://console:{{DATABASE_PASSWORD}}@$ENDPOINT:5432/console"
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
The regional bundle holds the three authorities a database in the region may be signed by (about 4.5 KB, read 2026-10), so it covers the instance whatever authority it got; it goes in as `DATABASE_CA`, and the URL carries no `ssl` parameter of its own (the console refuses both together). The password is base64url, so the URL needs no encoding.
|
|
121
|
+
|
|
122
|
+
### The bucket and its keys
|
|
123
|
+
|
|
124
|
+
```sh
|
|
125
|
+
BUCKET=console-files-$ACCOUNT
|
|
126
|
+
aws s3api create-bucket --bucket $BUCKET --region $AWS_REGION --create-bucket-configuration LocationConstraint=$AWS_REGION # us-east-1: without the configuration
|
|
127
|
+
aws s3api put-public-access-block --bucket $BUCKET --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
|
|
128
|
+
aws iam create-user --user-name console-files --tags Key=app,Value=console
|
|
129
|
+
aws iam put-user-policy --user-name console-files --policy-name console-files --policy-document file://scratch/files-policy.json
|
|
130
|
+
node scratch/store-secrets.mjs key "$SECRETS" console-files # makes the key once, stores both halves, prints neither
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
`scratch/files-policy.json`:
|
|
134
|
+
|
|
135
|
+
```json
|
|
136
|
+
{
|
|
137
|
+
"Version": "2012-10-17",
|
|
138
|
+
"Statement": [
|
|
139
|
+
{ "Effect": "Allow", "Action": ["s3:ListBucket"], "Resource": "arn:aws:s3:::<bucket>" },
|
|
140
|
+
{ "Effect": "Allow", "Action": ["s3:GetObject", "s3:PutObject"], "Resource": "arn:aws:s3:::<bucket>/*" }
|
|
141
|
+
]
|
|
142
|
+
}
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
`ListBucket` on the bucket is what the console's probe at start (`HEAD` on the bucket) needs; the objects are read and written under their hash.
|
|
146
|
+
|
|
147
|
+
### The image repository, the logs, the roles, the cluster
|
|
148
|
+
|
|
149
|
+
```sh
|
|
150
|
+
aws ecr create-repository --repository-name console --image-scanning-configuration scanOnPush=true --image-tag-mutability IMMUTABLE
|
|
151
|
+
aws logs create-log-group --log-group-name /console && aws logs put-retention-policy --log-group-name /console --retention-in-days 30
|
|
152
|
+
aws iam create-role --role-name console-task-execution --assume-role-policy-document file://scratch/ecs-trust.json
|
|
153
|
+
aws iam attach-role-policy --role-name console-task-execution --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
|
|
154
|
+
aws iam put-role-policy --role-name console-task-execution --policy-name console-parameters --policy-document file://scratch/parameters-policy.json
|
|
155
|
+
aws iam create-role --role-name console-task --assume-role-policy-document file://scratch/ecs-trust.json
|
|
156
|
+
aws iam create-service-linked-role --aws-service-name ecs.amazonaws.com # "has been taken" means it exists: fine
|
|
157
|
+
aws ecs create-cluster --cluster-name console --capacity-providers FARGATE --default-capacity-provider-strategy capacityProvider=FARGATE,weight=1
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
`scratch/ecs-trust.json` lets ECS tasks assume the role: `{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"ecs-tasks.amazonaws.com"},"Action":"sts:AssumeRole"}]}`. `scratch/parameters-policy.json` lets the execution role read the console's parameters: `{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["ssm:GetParameters"],"Resource":"arn:aws:ssm:<region>:<account>:parameter/console/*"}]}` (the parameters use the account's default key, which needs no `kms:Decrypt` of its own; checked 2026-10). The task role carries nothing: the console reaches the bucket with its own keys.
|
|
161
|
+
|
|
162
|
+
### The tunnel, the hostname and the rules at Cloudflare
|
|
163
|
+
|
|
164
|
+
1. **The tunnel.** `cf "/accounts/<CF_ACCOUNT>/cfd_tunnel" -X POST --data '{"name":"console","config_src":"cloudflare"}' > scratch/tunnel.json`; `TUNNEL=$(jq -r .result.id scratch/tunnel.json)`; `jq -r .result.token scratch/tunnel.json | set_secret TUNNEL_TOKEN`; `rm scratch/tunnel.json`. Then its routing: `cf "/accounts/<CF_ACCOUNT>/cfd_tunnel/$TUNNEL/configurations" -X PUT --data '{"config":{"ingress":[{"hostname":"<hostname>","service":"http://localhost:11199"},{"service":"http_status:404"}]}}'`. The connector runs beside the `api` container, so the service is the loopback; the last rule, without a hostname, is required.
|
|
165
|
+
2. **The hostname.** `cf "/zones/$ZONE/dns_records" -X POST --data '{"type":"CNAME","name":"<hostname>","content":"'$TUNNEL'.cfargotunnel.com","proxied":true,"ttl":1}'`. Proxied: the traffic goes through Cloudflare, which terminates TLS with its own certificate for the zone.
|
|
166
|
+
3. **Settings.** `cf "/zones/$ZONE/settings/always_use_https" -X PATCH --data '{"value":"on"}'`, so that `http://` turns into `https://`, which `PUBLIC_URL` is.
|
|
167
|
+
4. **The rate limiting rule**, on the sign-in and connect paths, which is what the console asks of a proxy; the board's own traffic is not limited. `RULESET=$(cf "/zones/$ZONE/rulesets/phases/http_ratelimit/entrypoint" | jq -r '.result.id')`: the entry point ruleset's id, or `null` when the zone has none yet. With one, `cf "/zones/$ZONE/rulesets/$RULESET/rules" -X POST --data @scratch/rule.json`; with none, `cf "/zones/$ZONE/rulesets" -X POST --data '{"name":"console","kind":"zone","phase":"http_ratelimit","rules":[<the rule>]}'`. `scratch/rule.json`:
|
|
168
|
+
|
|
169
|
+
```json
|
|
170
|
+
{
|
|
171
|
+
"description": "console: sign-in and connect, per client",
|
|
172
|
+
"expression": "(starts_with(http.request.uri.path, \"/auth/\") or starts_with(http.request.uri.path, \"/api/v1/connect\"))",
|
|
173
|
+
"action": "block",
|
|
174
|
+
"enabled": true,
|
|
175
|
+
"ratelimit": { "characteristics": ["ip.src", "cf.colo.id"], "period": 10, "requests_per_period": 30, "mitigation_timeout": 10 }
|
|
176
|
+
}
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
The Free plan allows one rate limiting rule, with a ten-second window and a ten-second block (checked 2026-10); a paid plan allows more, with longer windows.
|
|
180
|
+
|
|
181
|
+
### The parameters
|
|
182
|
+
|
|
183
|
+
With the database's URL and authority, the bucket's keys and the tunnel's token in the secrets file, and the generated `AUTH_SECRET`: `node scratch/store-secrets.mjs store "$SECRETS" /console` writes each as a `SecureString` parameter `/console/<NAME>` (the certificate bundle in the advanced tier, which takes up to 8 KB; the rest standard), skipping `later` and the password that is already inside the URL. Check: `aws ssm describe-parameters --parameter-filters Key=Name,Option=BeginsWith,Values=/console/ --query 'Parameters[].Name'`.
|
|
184
|
+
|
|
185
|
+
### The secrets the person holds
|
|
186
|
+
|
|
187
|
+
The hostname being known, one message per provider chosen with the steps of access.md "The OAuth app" and "The OAuth client", the exact URLs filled in, the line to add to the secrets file (`"GITHUB_CLIENT_SECRET": "<the secret>"`), and `later` as the other answer. On an answer, store it (`store` again adds what is new) and note the client id for the task definition. A provider answered `later` is left out of the task definition entirely, since the console wants both halves or neither; a provider not chosen has no line in the secrets file and nothing in the record.
|
|
188
|
+
|
|
189
|
+
## Phase 4: The deployment repository
|
|
190
|
+
|
|
191
|
+
In the run's folder, write:
|
|
192
|
+
|
|
193
|
+
### `deploy/console.env`
|
|
194
|
+
|
|
195
|
+
```sh
|
|
196
|
+
CONSOLE_REPOSITORY=https://github.com/skillcdn/console.git
|
|
197
|
+
CONSOLE_REF=<the commit of main pinned at the plan>
|
|
198
|
+
AWS_ACCOUNT_ID=<account id>
|
|
199
|
+
AWS_REGION=<region>
|
|
200
|
+
ECR_REPOSITORY=console
|
|
201
|
+
ECS_CLUSTER=console
|
|
202
|
+
SUBNETS=<subnet ids, comma-separated>
|
|
203
|
+
TASKS_SECURITY_GROUP=<the tasks' security group id>
|
|
204
|
+
API_COUNT=<1 | 1 | 2>
|
|
205
|
+
WORKER_COUNT=<0 | 0 | 1>
|
|
206
|
+
PLATFORM=linux/amd64
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
`PLATFORM=linux/arm64` with `cpuArchitecture` `ARM64` in the task definitions when the organization's builds run on ARM runners (sizing.md "Options").
|
|
210
|
+
|
|
211
|
+
### `deploy/api.task.json`
|
|
212
|
+
|
|
213
|
+
The task definition of the `api` role with the tunnel's connector beside it, in the shape `register-task-definition` takes; the workflow fills `image`. At `small` and `medium`, `WORKER_IN_PROCESS` is `true` and there is no `worker.task.json`; at `large`, it is left out and the worker is a service of its own.
|
|
214
|
+
|
|
215
|
+
```json
|
|
216
|
+
{
|
|
217
|
+
"family": "console-api",
|
|
218
|
+
"networkMode": "awsvpc",
|
|
219
|
+
"requiresCompatibilities": ["FARGATE"],
|
|
220
|
+
"cpu": "<256 | 512 | 512>",
|
|
221
|
+
"memory": "<512 | 1024 | 1024>",
|
|
222
|
+
"runtimePlatform": { "operatingSystemFamily": "LINUX", "cpuArchitecture": "X86_64" },
|
|
223
|
+
"executionRoleArn": "arn:aws:iam::<account>:role/console-task-execution",
|
|
224
|
+
"taskRoleArn": "arn:aws:iam::<account>:role/console-task",
|
|
225
|
+
"containerDefinitions": [
|
|
226
|
+
{
|
|
227
|
+
"name": "api",
|
|
228
|
+
"image": "<image>",
|
|
229
|
+
"essential": true,
|
|
230
|
+
"command": ["api"],
|
|
231
|
+
"portMappings": [{ "containerPort": 11199, "protocol": "tcp" }],
|
|
232
|
+
"environment": [
|
|
233
|
+
{ "name": "PUBLIC_URL", "value": "https://<hostname>" },
|
|
234
|
+
{ "name": "WORKSPACE_NAME", "value": "<the board's name>" },
|
|
235
|
+
{ "name": "WORKER_IN_PROCESS", "value": "true" },
|
|
236
|
+
{ "name": "TRUSTED_PROXIES", "value": "127.0.0.1" },
|
|
237
|
+
{ "name": "CLIENT_IP_HEADER", "value": "cf-connecting-ip" },
|
|
238
|
+
{ "name": "HTTP_KEEP_ALIVE_SECONDS", "value": "100" },
|
|
239
|
+
{ "name": "MEMBERS", "value": "<logins and addresses, comma-separated>" },
|
|
240
|
+
{ "name": "ADMINS", "value": "<logins among them>" },
|
|
241
|
+
{ "name": "GITHUB_CLIENT_ID", "value": "<the OAuth app's client id>" },
|
|
242
|
+
{ "name": "SKILLS_ADDRESS", "value": "<the organization's skills address, or drop the line>" },
|
|
243
|
+
{ "name": "S3_ENDPOINT", "value": "https://s3.<region>.amazonaws.com" },
|
|
244
|
+
{ "name": "S3_BUCKET", "value": "console-files-<account>" },
|
|
245
|
+
{ "name": "S3_REGION", "value": "<region>" }
|
|
246
|
+
],
|
|
247
|
+
"secrets": [
|
|
248
|
+
{ "name": "DATABASE_URL", "valueFrom": "arn:aws:ssm:<region>:<account>:parameter/console/DATABASE_URL" },
|
|
249
|
+
{ "name": "DATABASE_CA", "valueFrom": "arn:aws:ssm:<region>:<account>:parameter/console/DATABASE_CA" },
|
|
250
|
+
{ "name": "AUTH_SECRET", "valueFrom": "arn:aws:ssm:<region>:<account>:parameter/console/AUTH_SECRET" },
|
|
251
|
+
{ "name": "GITHUB_CLIENT_SECRET", "valueFrom": "arn:aws:ssm:<region>:<account>:parameter/console/GITHUB_CLIENT_SECRET" },
|
|
252
|
+
{ "name": "S3_ACCESS_KEY_ID", "valueFrom": "arn:aws:ssm:<region>:<account>:parameter/console/S3_ACCESS_KEY_ID" },
|
|
253
|
+
{ "name": "S3_SECRET_ACCESS_KEY", "valueFrom": "arn:aws:ssm:<region>:<account>:parameter/console/S3_SECRET_ACCESS_KEY" }
|
|
254
|
+
],
|
|
255
|
+
"healthCheck": {
|
|
256
|
+
"command": ["CMD-SHELL", "node -e \"fetch('http://127.0.0.1:11199/readyz').then((r) => process.exit(r.ok ? 0 : 1), () => process.exit(1))\""],
|
|
257
|
+
"interval": 15,
|
|
258
|
+
"timeout": 5,
|
|
259
|
+
"retries": 3,
|
|
260
|
+
"startPeriod": 60
|
|
261
|
+
},
|
|
262
|
+
"stopTimeout": 30,
|
|
263
|
+
"logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "/console", "awslogs-region": "<region>", "awslogs-stream-prefix": "api" } }
|
|
264
|
+
},
|
|
265
|
+
{
|
|
266
|
+
"name": "tunnel",
|
|
267
|
+
"image": "cloudflare/cloudflared:<the latest release, from github.com/cloudflare/cloudflared/releases>",
|
|
268
|
+
"essential": true,
|
|
269
|
+
"command": ["tunnel", "--no-autoupdate", "run"],
|
|
270
|
+
"secrets": [{ "name": "TUNNEL_TOKEN", "valueFrom": "arn:aws:ssm:<region>:<account>:parameter/console/TUNNEL_TOKEN" }],
|
|
271
|
+
"dependsOn": [{ "containerName": "api", "condition": "HEALTHY" }],
|
|
272
|
+
"stopTimeout": 10,
|
|
273
|
+
"logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "/console", "awslogs-region": "<region>", "awslogs-stream-prefix": "tunnel" } }
|
|
274
|
+
}
|
|
275
|
+
]
|
|
276
|
+
}
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
With Google: `GOOGLE_CLIENT_ID` and `GOOGLE_WORKSPACE_DOMAIN` in `environment`, `GOOGLE_CLIENT_SECRET` in `secrets`; without GitHub, the two GitHub entries are dropped. The connector starts only once the console is ready (`dependsOn` on the health check, which is `/readyz`), and stops first when the task stops, so a rollout never routes to a task that is not serving; the console's `stopTimeout` of 30 seconds is above its grace period of 20. The console sees every request from the loopback with the client's address in `cf-connecting-ip`, which is why the trusted proxy is `127.0.0.1`; its keep-alive of 100 seconds outlasts the connector's idle timeout of 90 (checked 2026-10). The image is the Node.js base, which has no `curl`, so the health check uses Node itself.
|
|
280
|
+
|
|
281
|
+
### `deploy/worker.task.json` (at `large`)
|
|
282
|
+
|
|
283
|
+
The same with `"family": "console-worker"`, `"cpu": "256"`, `"memory": "512"`, one container `worker` with `"command": ["worker"]`, the `environment` reduced to `WORKSPACE_NAME`, the `secrets` to `DATABASE_URL` and `DATABASE_CA`, no port, no health check, `"stopTimeout": 30`, the log stream prefix `worker`, and no tunnel.
|
|
284
|
+
|
|
285
|
+
### `deploy/migrate.task.json`
|
|
286
|
+
|
|
287
|
+
The same with `"family": "console-migrate"`, `"cpu": "256"`, `"memory": "512"`, one container `migrate` with `"command": ["migrate"]`, `secrets` `DATABASE_URL` and `DATABASE_CA`, no environment, no port, no health check, the log stream prefix `migrate`, and no tunnel: a connector must never start for a one-off task.
|
|
288
|
+
|
|
289
|
+
### `.github/workflows/deploy.yml`
|
|
290
|
+
|
|
291
|
+
```yaml
|
|
292
|
+
name: Deploy the console
|
|
293
|
+
on:
|
|
294
|
+
push:
|
|
295
|
+
branches: [main]
|
|
296
|
+
paths: ["deploy/**", "console-ui/**", ".github/workflows/deploy.yml"]
|
|
297
|
+
workflow_dispatch:
|
|
298
|
+
permissions:
|
|
299
|
+
id-token: write
|
|
300
|
+
contents: read
|
|
301
|
+
concurrency: deploy
|
|
302
|
+
jobs:
|
|
303
|
+
deploy:
|
|
304
|
+
runs-on: ubuntu-latest
|
|
305
|
+
steps:
|
|
306
|
+
- uses: actions/checkout@v4
|
|
307
|
+
- name: Read the deployment's settings
|
|
308
|
+
run: cat deploy/console.env >> "$GITHUB_ENV"
|
|
309
|
+
- uses: aws-actions/configure-aws-credentials@v4
|
|
310
|
+
with:
|
|
311
|
+
role-to-assume: arn:aws:iam::${{ env.AWS_ACCOUNT_ID }}:role/console-github-deploy
|
|
312
|
+
aws-region: ${{ env.AWS_REGION }}
|
|
313
|
+
- uses: aws-actions/amazon-ecr-login@v2
|
|
314
|
+
id: ecr
|
|
315
|
+
- name: Get the console at the pinned commit
|
|
316
|
+
run: |
|
|
317
|
+
git clone --no-checkout "$CONSOLE_REPOSITORY" console
|
|
318
|
+
git -C console checkout --detach "$CONSOLE_REF"
|
|
319
|
+
- name: Build and push the image
|
|
320
|
+
run: |
|
|
321
|
+
IMAGE="${{ steps.ecr.outputs.registry }}/$ECR_REPOSITORY:${CONSOLE_REF::12}-${GITHUB_SHA::7}-$GITHUB_RUN_NUMBER"
|
|
322
|
+
docker build --platform "$PLATFORM" -f console/deploy/Dockerfile -t "$IMAGE" console
|
|
323
|
+
if [ -d console-ui ]; then
|
|
324
|
+
(cd console-ui && npm ci && npm run build)
|
|
325
|
+
printf 'FROM %s\nCOPY --chown=node:node console-ui/dist/ /app/web/\n' "$IMAGE" | docker build --platform "$PLATFORM" -t "$IMAGE" -f - .
|
|
326
|
+
fi
|
|
327
|
+
docker push "$IMAGE"
|
|
328
|
+
echo "IMAGE=$IMAGE" >> "$GITHUB_ENV"
|
|
329
|
+
- name: Register the task definitions
|
|
330
|
+
run: |
|
|
331
|
+
for role in migrate api worker; do
|
|
332
|
+
[ -f "deploy/$role.task.json" ] || continue
|
|
333
|
+
jq --arg image "$IMAGE" '(.containerDefinitions[] | select(.name != "tunnel") | .image) = $image' "deploy/$role.task.json" > "$RUNNER_TEMP/$role.json"
|
|
334
|
+
aws ecs register-task-definition --cli-input-json "file://$RUNNER_TEMP/$role.json" --query 'taskDefinition.taskDefinitionArn' --output text
|
|
335
|
+
done
|
|
336
|
+
- name: Migrate, before anything new starts
|
|
337
|
+
run: |
|
|
338
|
+
NETWORK="awsvpcConfiguration={subnets=[$SUBNETS],securityGroups=[$TASKS_SECURITY_GROUP],assignPublicIp=ENABLED}"
|
|
339
|
+
echo "NETWORK=$NETWORK" >> "$GITHUB_ENV"
|
|
340
|
+
TASK=$(aws ecs run-task --cluster "$ECS_CLUSTER" --task-definition console-migrate --launch-type FARGATE --network-configuration "$NETWORK" --query 'tasks[0].taskArn' --output text)
|
|
341
|
+
aws ecs wait tasks-stopped --cluster "$ECS_CLUSTER" --tasks "$TASK"
|
|
342
|
+
CODE=$(aws ecs describe-tasks --cluster "$ECS_CLUSTER" --tasks "$TASK" --query 'tasks[0].containers[0].exitCode' --output text)
|
|
343
|
+
aws logs tail /console --since 15m --format short --log-stream-name-prefix migrate || true
|
|
344
|
+
[ "$CODE" = "0" ] || { echo "migrate exited with $CODE"; exit 1; }
|
|
345
|
+
- name: Roll the services
|
|
346
|
+
run: |
|
|
347
|
+
for role in api worker; do
|
|
348
|
+
[ -f "deploy/$role.task.json" ] || continue
|
|
349
|
+
COUNT=$([ "$role" = api ] && echo "$API_COUNT" || echo "$WORKER_COUNT")
|
|
350
|
+
if aws ecs describe-services --cluster "$ECS_CLUSTER" --services "$role" --query 'services[?status==`ACTIVE`].serviceName' --output text | grep -q "$role"; then
|
|
351
|
+
aws ecs update-service --cluster "$ECS_CLUSTER" --service "$role" --task-definition "console-$role" --desired-count "$COUNT" --force-new-deployment > /dev/null
|
|
352
|
+
else
|
|
353
|
+
aws ecs create-service --cluster "$ECS_CLUSTER" --service-name "$role" --task-definition "console-$role" --desired-count "$COUNT" --launch-type FARGATE --network-configuration "$NETWORK" --deployment-configuration "minimumHealthyPercent=100,maximumPercent=200,deploymentCircuitBreaker={enable=true,rollback=true}" > /dev/null
|
|
354
|
+
fi
|
|
355
|
+
done
|
|
356
|
+
aws ecs wait services-stable --cluster "$ECS_CLUSTER" --services api
|
|
357
|
+
```
|
|
358
|
+
|
|
359
|
+
What it does on every push that touches `deploy/` or `console-ui/`: builds the image from the console's pinned commit (and, when `console-ui/` exists, the organization's own console on top of it, as `deploy/README.md` "A custom console in the image" says), pushes it under a tag of its own (the console's commit, the repository's commit and the run's number, so that a run on the same commit still gets a new tag), registers the task definitions with that image, runs `migrate` as a one-off task and stops if it did not exit `0`, then rolls `api` (and `worker`) with a circuit breaker that rolls back a deployment whose tasks never become healthy, and waits for `api` to be stable. The first push creates the services. Pin the actions' majors to the current ones (`checkout@v4`, `configure-aws-credentials@v4`, `amazon-ecr-login@v2`, checked 2026-10; the actions' pages say when a newer major exists).
|
|
360
|
+
|
|
361
|
+
### The role the workflow assumes
|
|
362
|
+
|
|
363
|
+
```sh
|
|
364
|
+
aws iam create-open-id-connect-provider --url https://token.actions.githubusercontent.com --client-id-list sts.amazonaws.com # "EntityAlreadyExists" is fine
|
|
365
|
+
aws iam create-role --role-name console-github-deploy --assume-role-policy-document file://scratch/github-trust.json
|
|
366
|
+
aws iam put-role-policy --role-name console-github-deploy --policy-name console-deploy --policy-document file://scratch/deploy-policy.json
|
|
367
|
+
```
|
|
368
|
+
|
|
369
|
+
`scratch/github-trust.json`, trusting this repository's `main` and nothing else:
|
|
370
|
+
|
|
371
|
+
```json
|
|
372
|
+
{
|
|
373
|
+
"Version": "2012-10-17",
|
|
374
|
+
"Statement": [{
|
|
375
|
+
"Effect": "Allow",
|
|
376
|
+
"Principal": { "Federated": "arn:aws:iam::<account>:oidc-provider/token.actions.githubusercontent.com" },
|
|
377
|
+
"Action": "sts:AssumeRoleWithWebIdentity",
|
|
378
|
+
"Condition": { "StringEquals": {
|
|
379
|
+
"token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
|
|
380
|
+
"token.actions.githubusercontent.com:sub": "repo:<owner>/<repository>:ref:refs/heads/main"
|
|
381
|
+
} }
|
|
382
|
+
}]
|
|
383
|
+
}
|
|
384
|
+
```
|
|
385
|
+
|
|
386
|
+
`scratch/deploy-policy.json`: `ecr:GetAuthorizationToken` on `*`; `ecr:BatchCheckLayerAvailability`, `ecr:InitiateLayerUpload`, `ecr:UploadLayerPart`, `ecr:CompleteLayerUpload`, `ecr:PutImage`, `ecr:BatchGetImage`, `ecr:DescribeImages` on `arn:aws:ecr:<region>:<account>:repository/console`; `ecs:RegisterTaskDefinition`, `ecs:DescribeTaskDefinition`, `ecs:RunTask`, `ecs:DescribeTasks`, `ecs:DescribeServices`, `ecs:CreateService`, `ecs:UpdateService`, `ecs:ListTasks` on `*`; `iam:PassRole` on the two task roles; `logs:DescribeLogStreams`, `logs:GetLogEvents`, `logs:FilterLogEvents` on `arn:aws:logs:<region>:<account>:log-group:/console:*`.
|
|
387
|
+
|
|
388
|
+
### The rest of the repository
|
|
389
|
+
|
|
390
|
+
- `README.md`, in the organization's language: what this repository is (the deployment of the organization's console), that a push to `main` deploys, where the record is, and where the skill that made it lives (`skillcdn.ai/gh/skillcdn/console`).
|
|
391
|
+
- `DEPLOY.md`: the draft from the template (phase 8), filled as far as the run has gone.
|
|
392
|
+
- `console-ui/`: only when the organization has its own console (building-a-console makes it); otherwise absent and the image's default UI serves.
|
|
393
|
+
|
|
394
|
+
### Commit, create, push, watch
|
|
395
|
+
|
|
396
|
+
`git add -A && git commit -m "The console's deployment"` (`.gitignore` keeps `scratch/`, the key file and the token file out; check with `git status --short` that nothing of them is staged); show the files to the person; `gh repo create <owner>/<name> --private --source . --remote origin` (without `--push`), or `git remote add origin <the address the person gave>`. Checkpoint, then `git push -u origin main`; a few seconds later `RUN=$(gh run list --limit 1 --json databaseId -q '.[0].databaseId')` and `gh run watch "$RUN" --exit-status` (`gh run view "$RUN" --log` when it fails). The first run takes about ten minutes: the build, the push, the migration, the services.
|
|
397
|
+
|
|
398
|
+
## Phase 5: Live
|
|
399
|
+
|
|
400
|
+
```sh
|
|
401
|
+
cf "/zones?name=<domain>" | jq -r '.result[0].status' # active; else the wait of the skill
|
|
402
|
+
curl -sS -o /dev/null -w '%{http_code}\n' https://<hostname>/readyz # 200
|
|
403
|
+
curl -sS https://<hostname>/api/v1/me | jq .signIn # the providers set, asked as nobody
|
|
404
|
+
cf "/accounts/<CF_ACCOUNT>/cfd_tunnel/$TUNNEL/connections" | jq '.result | length' # 1 or more
|
|
405
|
+
aws ecs describe-services --cluster console --services api --query 'services[0].[runningCount,deployments[0].rolloutState]'
|
|
406
|
+
aws logs tail /console --since 15m --format short | grep -E 'migrations applied|api listening|files go to the bucket'
|
|
407
|
+
```
|
|
408
|
+
|
|
409
|
+
Then the budget alert, at one and a half times the estimate, to the address of the plan, which emails when the month's bill passes it or is forecast to:
|
|
410
|
+
|
|
411
|
+
```sh
|
|
412
|
+
aws budgets create-budget --account-id $ACCOUNT --budget file://scratch/budget.json --notifications-with-subscribers file://scratch/notifications.json
|
|
413
|
+
```
|
|
414
|
+
|
|
415
|
+
`scratch/budget.json`: `{"BudgetName":"console","BudgetType":"COST","BudgetLimit":{"Amount":"<estimate x 1.5>","Unit":"USD"},"TimeUnit":"MONTHLY"}`; `scratch/notifications.json`: `[{"Notification":{"NotificationType":"ACTUAL","ComparisonOperator":"GREATER_THAN","Threshold":100,"ThresholdType":"PERCENTAGE"},"Subscribers":[{"SubscriptionType":"EMAIL","Address":"<email>"}]},{"Notification":{"NotificationType":"FORECASTED","ComparisonOperator":"GREATER_THAN","Threshold":100,"ThresholdType":"PERCENTAGE"},"Subscribers":[{"SubscriptionType":"EMAIL","Address":"<email>"}]}]`. The person confirms the subscription from the email AWS sends.
|
|
416
|
+
|
|
417
|
+
## Phases 6 and 7: the first people, and the proof
|
|
418
|
+
|
|
419
|
+
The person signs in and makes the first project on the pages (the skill's phase 6). Then, in the run's folder:
|
|
420
|
+
|
|
421
|
+
```sh
|
|
422
|
+
console login --url https://<hostname> --agent deploying-a-console # relay the address and the code; the person approves
|
|
423
|
+
console use <key>
|
|
424
|
+
printf '# Deployment check\n\nHanded in by the deploying agent on <date> to prove the bucket.\n' > scratch/deployment-check.md
|
|
425
|
+
NUMBER=$(console task new "Deployment check" --body "What this run deployed, in a few lines: the size, the address, what is owed." --json | jq -r .number)
|
|
426
|
+
RUN_ID=$(console take "$NUMBER" --json | jq -r .id)
|
|
427
|
+
console hand-in scratch/deployment-check.md --label "the check"
|
|
428
|
+
HASH=$(console run "$RUN_ID" --json | jq -r '.artifacts[0].file.sha256')
|
|
429
|
+
aws s3api head-object --bucket $BUCKET --key "$HASH" # the object, with its size
|
|
430
|
+
console finish --summary "Deployed <size> at <address>; owed: <what the delivery names>."
|
|
431
|
+
```
|
|
432
|
+
|
|
433
|
+
The hash `console run` shows is the object's key in the bucket (no prefix is set); `head-object` finding it proves the file went to the bucket and that the console reads it back from there. `console help <command>` is right where this differs from it.
|
|
434
|
+
|
|
435
|
+
## Phase 8: the record
|
|
436
|
+
|
|
437
|
+
`DEPLOY.md`, in the organization's language, from this template:
|
|
438
|
+
|
|
439
|
+
```markdown
|
|
440
|
+
# The console's deployment
|
|
441
|
+
|
|
442
|
+
Made on <date> with the skill `deploying-a-console` of the SkillCDN Console (skillcdn.ai/gh/skillcdn/console), from commit `<CONSOLE_REF>` of `skillcdn/console`.
|
|
443
|
+
|
|
444
|
+
## What exists
|
|
445
|
+
|
|
446
|
+
| What | Where |
|
|
447
|
+
|---|---|
|
|
448
|
+
| The console | https://<hostname>, through Cloudflare (zone <domain>, tunnel `console`, id `<tunnel id>`) |
|
|
449
|
+
| AWS account and region | <account id>, <region> |
|
|
450
|
+
| Containers | ECS cluster `console`; services `api` (<n> tasks of <cpu>/<memory>)<, `worker`>; task families `console-api`, `console-migrate`<, `console-worker`> |
|
|
451
|
+
| Database | RDS PostgreSQL <version>, `console`, <class>, <storage>, <one zone | two zones>, backups 7 days, deletion protection on |
|
|
452
|
+
| Files | S3 bucket `console-files-<account>`, read and written by the IAM user `console-files` |
|
|
453
|
+
| Secrets | SSM Parameter Store, `/console/*`: <the names> |
|
|
454
|
+
| Logs | CloudWatch Logs `/console`, kept 30 days |
|
|
455
|
+
| Images | ECR `console`, one tag per deployment |
|
|
456
|
+
| Roles | `console-task-execution`, `console-task`, `console-github-deploy` (assumed by this repository's `main`) |
|
|
457
|
+
| Size and cost | `<size>`; about <estimate> a month at <date>'s prices; a budget alert at <amount> emails <address> |
|
|
458
|
+
|
|
459
|
+
## How it is run
|
|
460
|
+
|
|
461
|
+
- **A deployment** is a push to `main` that touches `deploy/` or `console-ui/`: the workflow builds the image, runs the migrations, and rolls the services. `gh run list` shows the runs.
|
|
462
|
+
- **A new version of the console:** set `CONSOLE_REF` in `deploy/console.env` to the commit, push.
|
|
463
|
+
- **A person added or removed:** `MEMBERS` and `ADMINS` in `deploy/api.task.json`, push. Administrators also change roles on the People page.
|
|
464
|
+
- **The next size:** the numbers in `deploy/api.task.json` (and `deploy/worker.task.json` at `large`), `API_COUNT` and `WORKER_COUNT` in `deploy/console.env`, push; the database: `aws rds modify-db-instance --db-instance-identifier console --db-instance-class <class> --apply-immediately` (a few minutes of unavailability).
|
|
465
|
+
- **A secret rotated:** `aws ssm put-parameter --name /console/<NAME> --type SecureString --overwrite --cli-input-json file://<file>`, then `aws ecs update-service --cluster console --service api --force-new-deployment`.
|
|
466
|
+
- **The organization's own console:** its source in `console-ui/` (building-a-console makes it); a push builds it into the image.
|
|
467
|
+
- **Paused:** `aws ecs update-service --cluster console --service api --desired-count 0` (and `worker`), `aws rds stop-db-instance --db-instance-identifier console` (seven days at most); the reverse resumes.
|
|
468
|
+
- **Removed:** the skill's "Removing", resource by resource, confirmed.
|
|
469
|
+
|
|
470
|
+
## What is owed
|
|
471
|
+
|
|
472
|
+
<a provider's secret said later, with its steps; nameservers; a secret pasted into the chat, to be made anew; nothing>
|
|
473
|
+
|
|
474
|
+
## Verified on <date>
|
|
475
|
+
|
|
476
|
+
`/readyz` 200 at the address; <who> signed in as administrator; project `<key>` made; the agent connected with a code; the deployment check handed in and found in the bucket under its hash; workflow run <id> green.
|
|
477
|
+
```
|
|
478
|
+
|
|
479
|
+
A change to `DEPLOY.md` or `README.md` alone does not run the workflow (its `paths`), so the record is pushed without a deployment.
|
|
480
|
+
|
|
481
|
+
## Removing
|
|
482
|
+
|
|
483
|
+
Only on request, each step confirmed, in this order, after a last snapshot of the database: the services (`aws ecs update-service --desired-count 0`, then `delete-service`), the cluster; the hostname (`DELETE /zones/<zone>/dns_records/<id>`) and the tunnel (`DELETE /accounts/<account>/cfd_tunnel/<id>`, after `.../connections` is empty); the database (`modify-db-instance --no-deletion-protection`, then `delete-db-instance --final-db-snapshot-identifier console-final-<date>`); the bucket, emptied first (`aws s3 rm s3://<bucket> --recursive`, then `delete-bucket`); the parameters; the users' keys and the users; the roles' policies and the roles; the log group; the image repository (`--force`); the security groups; the budget. The deployment repository stays, with the record of what was removed and the snapshot's name.
|
|
484
|
+
|
|
485
|
+
## When a step fails
|
|
486
|
+
|
|
487
|
+
| Symptom | Cause | What to do |
|
|
488
|
+
|---|---|---|
|
|
489
|
+
| `AccessDenied` or `UnauthorizedOperation` naming an action | The deployer's policy lacks it | Add the action to the policy (access.md) and run the step again |
|
|
490
|
+
| A log group or a parameter gets a name like `C:/Program Files/Git/console` | Git Bash rewrote the `/console` argument | `export MSYS_NO_PATHCONV=1`, delete the misnamed resource, run the step again |
|
|
491
|
+
| `aws rds create-db-instance` refuses the engine version | The region has no PostgreSQL 18 yet | `describe-db-engine-versions --engine postgres` and take the newest major it lists |
|
|
492
|
+
| `create-bucket` says the name is taken | Bucket names are global | Add a short suffix and update the task definition and the files policy |
|
|
493
|
+
| `POST /zones` is refused with a permission error | The token lacks Zone Edit | Add the row (access.md) and run the step again |
|
|
494
|
+
| The migration task exits `78` | A variable refused; the log names it, never its value | Fix the parameter or the environment entry, push again |
|
|
495
|
+
| The migration task exits `1` with a connection error | The tasks' group is not allowed into the database's, or the URL's host is wrong | Check the `authorize-security-group-ingress` and the endpoint in `DATABASE_URL` |
|
|
496
|
+
| `ResourceInitializationError` on a task: unable to pull secrets | The execution role cannot read `/console/*` | Check `console-parameters` on `console-task-execution` and the parameter ARNs |
|
|
497
|
+
| A task stops before it is healthy; the log says `api listening` and then nothing | The health check fails: wrong port or `/readyz` not `200` (database unreachable, schema behind) | `aws logs tail /console --since 10m`; `/readyz` says which of the three |
|
|
498
|
+
| The service never stabilizes and rolls back | The circuit breaker tripped on the above | Fix the cause; `--force-new-deployment` |
|
|
499
|
+
| The tunnel shows no connector while `api` is healthy | The token is wrong, or the task has no public IP for its egress | Check `TUNNEL_TOKEN` and `assignPublicIp=ENABLED` in the network configuration |
|
|
500
|
+
| `https://<hostname>` answers 530 or 502 from Cloudflare | No connector yet, or the hostname's ingress rule points elsewhere | Wait for the service; check the tunnel's configuration and the DNS record |
|
|
501
|
+
| `https://<hostname>` does not resolve | The zone is not active yet | The nameservers at the registrar; `cf "/zones?name=<domain>"` says when |
|
|
502
|
+
| The sign-in button is missing | The provider's id or secret is not in the task definition | Both halves or neither; check `environment` and `secrets` |
|
|
503
|
+
| The sign-in is refused after the provider | The login is not in `MEMBERS` (or the domain not in `GOOGLE_WORKSPACE_DOMAIN`) | Add it, push |
|
|
504
|
+
| The image pull of `cloudflared` is refused | Docker Hub's anonymous pull limit | Try again; when it keeps failing, a step in the workflow mirrors the image into ECR (`docker pull`, `docker tag`, `docker push`) and the task definition points there |
|
|
505
|
+
| `gh run watch` shows the build failing on `npm ci` in `console-ui` | The UI's lockfile is out of step | Fix in `console-ui/` (building-a-console), push |
|
|
506
|
+
| A 524 at Cloudflare on a long read | A request held longer than the proxy's 100 seconds | The console holds a decision's read for 50 at most; look for a slow database |
|
|
507
|
+
|
|
508
|
+
## Facts that move
|
|
509
|
+
|
|
510
|
+
- AWS App Runner stopped taking new customers on 2026-04-30 and is in maintenance; ECS Express Mode (2025-11) is AWS's successor and provisions a load balancer (checked 2026-10).
|
|
511
|
+
- Fargate: `stopTimeout` from 2 to 120 seconds, 30 when unset; containers stop in the reverse order of their start dependencies (checked 2026-10).
|
|
512
|
+
- SSM Parameter Store: 4 KB per standard parameter, 8 KB per advanced one at about $0.05 a month; the RDS regional certificate bundle is about 4.5 KB with three authorities (checked 2026-10).
|
|
513
|
+
- RDS for PostgreSQL has had major version 18 since 2025-11 (checked 2026-10).
|
|
514
|
+
- Cloudflare: a proxied request's idle limit is 100 seconds on every plan but Enterprise; the Free plan allows one rate limiting rule with a ten-second window; a tunnel costs nothing and its connector token is read from `TUNNEL_TOKEN`; a remotely managed tunnel's routing is the `configurations` document, whose last rule has no hostname; creating a zone needs Zone Edit on the token (checked 2026-10).
|
|
515
|
+
- A new AWS account made after 2025-07-15 starts with $100 of credits and a choice between a free plan that ends after six months and a paid one (checked 2026-10).
|
|
516
|
+
- Git Bash on Windows rewrites an argument that starts with `/` into a Windows path for a native program unless `MSYS_NO_PATHCONV=1` is set (checked 2026-10).
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
# Sizing and cost: three sizes, and what each costs
|
|
2
|
+
|
|
3
|
+
How the organization's size becomes a size of the deployment, what each size is made of, how its monthly cost is estimated before anything is made, and what changes when the organization grows. Every price here is a snapshot that illustrates the shape, marked with the month it was read; the run reads today's prices where it can, shows those, and says which lines are still the snapshot.
|
|
4
|
+
|
|
5
|
+
## The sizes
|
|
6
|
+
|
|
7
|
+
The question is how many people will use the console within a year; an answer stands for the run and is written in the record. Growing later is a change of numbers (below).
|
|
8
|
+
|
|
9
|
+
| Size | For | The `api` service | The `worker` | The database | What else |
|
|
10
|
+
|---|---|---|---|---|---|
|
|
11
|
+
| `small` | Up to 10 people | One task, 0.25 vCPU, 0.5 GB, carrying the worker's work (`WORKER_IN_PROCESS=true`) | In the `api` task | One instance, the smallest class with 1 GB of memory (`db.t4g.micro`), 20 GB growing to 100, one zone | A bucket, the parameter store, one public address for the task's egress |
|
|
12
|
+
| `medium` | Up to 30 people | One task, 0.5 vCPU, 1 GB, carrying the worker's work | In the `api` task | One instance with 2 GB (`db.t4g.small`), 20 GB growing to 100, one zone | The same |
|
|
13
|
+
| `large` | Up to 100 people | Two tasks, 0.5 vCPU, 1 GB each | A service of its own, one task, 0.25 vCPU, 0.5 GB | One instance with 4 GB (`db.t4g.medium`), 20 GB growing to 200; in two zones when the organization wants the database to survive a zone | The same, with three public addresses |
|
|
14
|
+
|
|
15
|
+
The board is a light load: a page, a feed held open per browser tab with a heartbeat every 25 seconds, a command now and then from each agent. What grows with people is the number of open feeds (memory and connections) and the database's working set, not the CPU. These sizes leave room; the delivery says how to read the gauges (below) and the record says which size was chosen.
|
|
16
|
+
|
|
17
|
+
## What is billed
|
|
18
|
+
|
|
19
|
+
| Item | How it is billed | Illustrative price, N. Virginia (`us-east-1`), read 2026-10 |
|
|
20
|
+
|---|---|---|
|
|
21
|
+
| Fargate tasks | Per vCPU-hour and per GB-hour of each task, per second, a minute at least; x86 by default, ARM about a fifth cheaper when the image is built for it | x86 $0.04048 per vCPU-hour, $0.004445 per GB-hour; ARM $0.03238 and $0.00356 |
|
|
22
|
+
| Public IPv4 address, one per task | Per hour | $0.005 per hour, about $3.65 a month |
|
|
23
|
+
| RDS PostgreSQL instance | Per hour by class; two zones doubles it | `db.t4g.micro` $0.016 per hour, `db.t4g.small` $0.032 per hour, `db.t4g.medium` about twice the small |
|
|
24
|
+
| RDS storage, `gp3` | Per GB-month of what is allocated; backups up to the instance's size are free | $0.115 per GB-month |
|
|
25
|
+
| S3 | Per GB-month and per request; a board's files are small | $0.023 per GB-month |
|
|
26
|
+
| ECR | Per GB-month of images kept | $0.10 per GB-month |
|
|
27
|
+
| CloudWatch Logs | Per GB ingested and per GB-month kept, 30 days here | $0.50 per GB ingested, $0.03 per GB-month |
|
|
28
|
+
| SSM Parameter Store | Standard parameters free; the certificate bundle is an advanced parameter | $0.05 per advanced parameter a month |
|
|
29
|
+
| Cloudflare | The Free plan: DNS, the proxy, the tunnel, one rate limiting rule, certificates | $0 |
|
|
30
|
+
| A domain | Per year, at cost through the Registrar | About $10 a year for a `.com` |
|
|
31
|
+
| GitHub Actions | Minutes of the deployment repository's builds; a build takes about ten | Within the minutes a plan includes |
|
|
32
|
+
|
|
33
|
+
Other regions are dearer by a tenth to a third for the same items (Tokyo's ARM task rates were about a quarter above N. Virginia's, read 2026-10); Seoul's rates are read from the price list in the run. A new AWS account made after 2025-07-15 starts with credits ($100 at sign-up, up to $200 with the first activities), which the paid plan spends on these bills over the first months; the free plan ends after six months and is not for a console that stays.
|
|
34
|
+
|
|
35
|
+
## The estimate, size by size
|
|
36
|
+
|
|
37
|
+
Illustrative, from the prices above, x86 tasks, one zone, 730 hours a month, read 2026-10. The run recomputes each line with the prices of the day and the region, and shows the chosen size with the next one beside it.
|
|
38
|
+
|
|
39
|
+
| Size | Tasks | Addresses | Database | Storage, logs, images | About |
|
|
40
|
+
|---|---|---|---|---|---|
|
|
41
|
+
| `small` | $9 | $4 | $12 | $3 | $28 a month |
|
|
42
|
+
| `medium` | $18 | $4 | $23 | $4 | $49 a month |
|
|
43
|
+
| `large` | $45 | $11 | $47 (one zone) or $95 (two) | $6 | $109 a month, $157 in two zones |
|
|
44
|
+
|
|
45
|
+
A domain is on top, per year. Cloudflare costs nothing on the Free plan.
|
|
46
|
+
|
|
47
|
+
## Reading today's prices
|
|
48
|
+
|
|
49
|
+
The two lines that matter, the tasks and the database, come from the Price List API, which answers from `us-east-1` whatever the region priced (the deployer's policy allows `pricing:GetProducts`); the small lines (the address, storage, logs, images) from the pricing pages, or the snapshot with the region's premium stated. The database's instance, per hour:
|
|
50
|
+
|
|
51
|
+
```sh
|
|
52
|
+
aws pricing get-products --region us-east-1 --service-code AmazonRDS \
|
|
53
|
+
--filters Type=TERM_MATCH,Field=instanceType,Value=<db.t4g.micro | db.t4g.small | db.t4g.medium> \
|
|
54
|
+
Type=TERM_MATCH,Field=databaseEngine,Value=PostgreSQL \
|
|
55
|
+
Type=TERM_MATCH,Field=deploymentOption,Value=<Single-AZ | Multi-AZ> \
|
|
56
|
+
Type=TERM_MATCH,Field=regionCode,Value=<region> \
|
|
57
|
+
--query 'PriceList[0]' --output text | jq -r '.terms.OnDemand[] | .priceDimensions[] | .pricePerUnit.USD'
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
The tasks, per vCPU-hour and per GB-hour, by the usage type of the region (`APN2` for Seoul, `APN1` for Tokyo, `EUC1` for Frankfurt, `EUW1` for Ireland, `USE2` for Ohio, no prefix for N. Virginia; `ARM-` before `vCPU` and `GB` for ARM tasks):
|
|
61
|
+
|
|
62
|
+
```sh
|
|
63
|
+
for usage in "<prefix>-Fargate-vCPU-Hours:perCPU" "<prefix>-Fargate-GB-Hours"; do
|
|
64
|
+
aws pricing get-products --region us-east-1 --service-code AmazonECS \
|
|
65
|
+
--filters Type=TERM_MATCH,Field=usagetype,Value="$usage" Type=TERM_MATCH,Field=regionCode,Value=<region> \
|
|
66
|
+
--query 'PriceList[0]' --output text | jq -r '.terms.OnDemand[] | .priceDimensions[] | .pricePerUnit.USD'
|
|
67
|
+
done
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
The pages, each with the region selected, for the rest: the public IPv4 charge on the EC2 pricing page (`aws.amazon.com/ec2/pricing/on-demand`, "IPv4 address"), S3 (`aws.amazon.com/s3/pricing`), ECR, CloudWatch; or the AWS Pricing Calculator. A lookup that fails is said so in the plan, and that line is the snapshot with the premium of "What is billed" applied, named as an estimate. State in the plan where each number came from and the month.
|
|
71
|
+
|
|
72
|
+
## The estimate message
|
|
73
|
+
|
|
74
|
+
Part of the plan, in plain words, one line per item, the chosen size first:
|
|
75
|
+
|
|
76
|
+
> For <n> people I would set it up as **<size>**: <what it is made of, in words>. That is about **<total> a month** in <region> (<tasks>, <database>, <addresses>, <the rest>; <which lines are today's prices and which the snapshot>)<, plus the domain at <price> a year when one is bought>. The next size up, **<size>**, would be about <total> a month and is a change of numbers later, not a new deployment. A budget alert at <total × 1.5> will email <address> when the month's bill runs past it, or is forecast to.
|
|
77
|
+
|
|
78
|
+
## Options the plan offers
|
|
79
|
+
|
|
80
|
+
- **The database as serverless capacity.** Aurora Serverless v2 for PostgreSQL scales its capacity with load, down to nothing when idle (0 ACU with automatic pause, since 2024-11). The console holds a listening connection open and sweeps every fifteen minutes, so it never idles and the database never pauses: the floor is 0.5 ACU, about $44 a month at $0.12 per ACU-hour (read 2026-10), more than the `small` and `medium` instances. Offer it when the organization wants capacity that follows load without resizing, and say the floor.
|
|
81
|
+
- **The bucket on Cloudflare R2** instead of S3: the same S3 API, keys made in the Cloudflare dashboard, `S3_ENDPOINT=https://<account id>.r2.cloudflarestorage.com`, `S3_REGION=auto`, path style; no egress charges. Offer it when the organization keeps its files at Cloudflare already; S3 stays the default because the keys, the budget and the audit are then in one account.
|
|
82
|
+
- **A load balancer instead of the tunnel.** ECS Express Mode creates an Application Load Balancer with HTTPS on an AWS domain in one command, at about $16 to $23 a month for the balancer (read 2026-10); the custom domain is a manual step on the balancer, and Cloudflare then proxies to it. Offer it only when the organization does not want its traffic through the tunnel; the tunnel is the default because it costs nothing, exposes nothing, and brings the client's address to the console cleanly.
|
|
83
|
+
- **ARM tasks** when the organization has ARM runners for its builds (building for ARM on an x86 runner takes many times longer): a fifth off the task line.
|
|
84
|
+
|
|
85
|
+
## Growing
|
|
86
|
+
|
|
87
|
+
- **More people:** change the task sizes and counts in `deploy/api.task.json` and `deploy/worker.task.json` of the deployment repository and push; the services roll. At `large`, the `worker` becomes a service of its own and `WORKER_IN_PROCESS` is dropped from the `api` task; `deploy.yml` deploys both.
|
|
88
|
+
- **A bigger database:** `aws rds modify-db-instance --db-instance-identifier <name> --db-instance-class <class> --apply-immediately` (a few minutes of unavailability; do it when nobody works) and, for two zones, `--multi-az`. Storage grows on its own up to the maximum set.
|
|
89
|
+
- **Reading the gauges:** the ECS service's CPU and memory in CloudWatch (`AWS/ECS`, by cluster and service), the database's `FreeableMemory` and `DatabaseConnections` (`AWS/RDS`). Memory above 70% or connections near the instance's limit (87 on a `t4g.micro`) says the next size is due.
|
|
90
|
+
- **Fewer people, or a trial that ended:** the opposite; or `aws ecs update-service --desired-count 0` for both services and `aws rds stop-db-instance` (seven days at most, then it starts itself) to pause the deployment at the cost of storage alone.
|