spiderly 19.8.6 → 19.8.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -49,6 +49,34 @@ public class Achievement : BusinessObject<long>
|
|
|
49
49
|
|
|
50
50
|
The string is appended directly to the generated `.NewConfig<Entity, EntityDTO>()` chain. Use this for simple field mappings that the generator doesn't produce automatically.
|
|
51
51
|
|
|
52
|
+
### `[ProjectToDTO]` only fills the value — the field must exist on the DTO
|
|
53
|
+
|
|
54
|
+
`[ProjectToDTO]` adds a Mapster *mapping*; it does **not** create the property. If `dest.ProductId`
|
|
55
|
+
is not a property on the DTO, the field **never appears in the generated Angular type**
|
|
56
|
+
(`entities.generated.ts`) — `[ProjectToDTO]` fills a value that has nowhere to land. (A common
|
|
57
|
+
false alarm: the value maps fine at runtime, so it looks like the frontend "didn't regenerate" —
|
|
58
|
+
but a normal `dotnet build` *does* regenerate the Angular files; the property was just never on
|
|
59
|
+
the DTO for the generator to emit.)
|
|
60
|
+
|
|
61
|
+
To add a computed/projected field end-to-end, declare the property on a `partial class {Entity}DTO`
|
|
62
|
+
extension — it merges into the generated DTO automatically, no attribute needed — **then** map its value:
|
|
63
|
+
|
|
64
|
+
```csharp
|
|
65
|
+
// 1. The property — a partial extension of the generated OrderItemDTO. No [SpiderlyDTO]
|
|
66
|
+
// needed: a partial that extends a generated DTO is merged in by name.
|
|
67
|
+
public partial class OrderItemDTO
|
|
68
|
+
{
|
|
69
|
+
public int? ProductId { get; set; }
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
// 2. The value — [ProjectToDTO] on the entity fills it during projection.
|
|
73
|
+
[ProjectToDTO(".Map(dest => dest.ProductId, src => src.ProductVariant.ProductId)")]
|
|
74
|
+
public class OrderItem : BusinessObject<long> { /* ... */ }
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
Then `dotnet build` the backend — the source generators run on build and the field appears
|
|
78
|
+
in `entities.generated.ts`. There is no separate "regenerate" command; the build is it.
|
|
79
|
+
|
|
52
80
|
## Partial Method Override — Full Control
|
|
53
81
|
|
|
54
82
|
For complex mapping logic, override the entire generated method. The generator **skips generation** for any method that already exists in the user's partial `Mapper` class (detected by method name match).
|
package/agent/manifest.json
CHANGED
|
@@ -5,10 +5,15 @@
|
|
|
5
5
|
"surface": "skill",
|
|
6
6
|
"description": "Scaffold a new Spiderly entity end-to-end (entity class, Angular pages, routes, menu, migration)"
|
|
7
7
|
},
|
|
8
|
+
{
|
|
9
|
+
"name": "backups",
|
|
10
|
+
"surface": "skill",
|
|
11
|
+
"description": "Back up and restore a Spiderly app's Postgres database, and archive logs off the VPS. Use when setting up automated database backups, scheduling pg_dump to object storage (Cloudflare R2), restoring from a backup, running a restore drill, configuring backup retention, or archiving Serilog file logs. Covers the backup and restore scripts, cron scheduling, and the R2 bucket Terraform."
|
|
12
|
+
},
|
|
8
13
|
{
|
|
9
14
|
"name": "deployment",
|
|
10
15
|
"surface": "skill",
|
|
11
|
-
"description": "Deploy a Spiderly project to
|
|
16
|
+
"description": "Deploy a Spiderly project to production and keep it running. Use when deploying, redeploying, shipping, releasing, or rolling out the .NET backend or Angular admin — first-time setup or an ongoing deploy — and when diagnosing a down or erroring production origin (502/521, container crash-loop, failed deploy workflow). Covers the recommended VPS + Docker Compose + Caddy + Cloudflare + Terraform stack, CI/CD pipelines, TLS with Cloudflare origin certificates, and infrastructure-as-code layout."
|
|
12
17
|
},
|
|
13
18
|
{
|
|
14
19
|
"name": "ef-migrations",
|
|
@@ -0,0 +1,148 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: backups
|
|
3
|
+
description: Back up and restore a Spiderly app's Postgres database, and archive logs off the VPS. Use when setting up automated database backups, scheduling pg_dump to object storage (Cloudflare R2), restoring from a backup, running a restore drill, configuring backup retention, or archiving Serilog file logs. Covers the backup and restore scripts, cron scheduling, and the R2 bucket Terraform.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Backups
|
|
7
|
+
|
|
8
|
+
If you ship this stack to prod, you ship a backup with it. Postgres data lives in a Docker volume on a single VPS — disk failure, accidental `docker volume rm`, or `terraform destroy` all wipe it without a snapshot to restore from.
|
|
9
|
+
|
|
10
|
+
## Database backups (required)
|
|
11
|
+
|
|
12
|
+
Daily `pg_dump` to a Cloudflare R2 bucket, 7-day retention both local and remote. ~24h RPO; if you need tighter, layer WAL archiving on top (separate skill).
|
|
13
|
+
|
|
14
|
+
**1. R2 bucket — Terraform-managed.** No chicken-and-egg here (only the *state* bucket has to be bootstrapped manually); manage data buckets like any other resource:
|
|
15
|
+
|
|
16
|
+
```hcl
|
|
17
|
+
# infrastructure/cloudflare-r2.tf
|
|
18
|
+
resource "cloudflare_r2_bucket" "db_backups" {
|
|
19
|
+
account_id = var.cloudflare_account_id
|
|
20
|
+
name = "<your-app>-db-backups"
|
|
21
|
+
location = "EEUR"
|
|
22
|
+
|
|
23
|
+
lifecycle {
|
|
24
|
+
prevent_destroy = true
|
|
25
|
+
}
|
|
26
|
+
}
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
**2. Backup script — `infrastructure/scripts/pg_backup_s3.sh`** (deployed to `/usr/local/bin/` on the VPS). Note the `trap` cleanup, the `aws s3api list-objects-v2 --query` for retention (structured + locale-safe vs parsing `aws s3 ls` with awk), and the per-iteration warn-on-failure inside the `while` subshell (where `set -e` does NOT propagate):
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
#!/usr/bin/env bash
|
|
33
|
+
set -euo pipefail
|
|
34
|
+
|
|
35
|
+
CONTAINER_NAME="<your-app>-postgres-1"
|
|
36
|
+
DB_NAME="<your-db>"
|
|
37
|
+
DB_USER="postgres"
|
|
38
|
+
CLOUDFLARE_ACCOUNT_ID="<account-id>"
|
|
39
|
+
S3_BUCKET="s3://<your-app>-db-backups"
|
|
40
|
+
R2_ENDPOINT="https://${CLOUDFLARE_ACCOUNT_ID}.r2.cloudflarestorage.com"
|
|
41
|
+
RETENTION_DAYS=7
|
|
42
|
+
LOCAL_BACKUP_DIR="/var/backups/postgresql"
|
|
43
|
+
LOG_FILE="/var/log/<your-app>_pg_backup.log"
|
|
44
|
+
|
|
45
|
+
TIMESTAMP=$(date +%Y-%m-%d_%H%M%S)
|
|
46
|
+
BACKUP_FILE="<your-app>_${TIMESTAMP}.sql.gz"
|
|
47
|
+
LOCAL_PATH="${LOCAL_BACKUP_DIR}/${BACKUP_FILE}"
|
|
48
|
+
|
|
49
|
+
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"; }
|
|
50
|
+
|
|
51
|
+
trap 'rc=$?; [[ $rc -ne 0 ]] && rm -f "$LOCAL_PATH" && log "Aborted, removed partial $LOCAL_PATH"; exit $rc' ERR INT TERM
|
|
52
|
+
|
|
53
|
+
mkdir -p "$LOCAL_BACKUP_DIR"
|
|
54
|
+
|
|
55
|
+
if ! docker exec "$CONTAINER_NAME" pg_dump -U "$DB_USER" "$DB_NAME" | gzip > "$LOCAL_PATH"; then
|
|
56
|
+
log "ERROR: pg_dump failed"; exit 1
|
|
57
|
+
fi
|
|
58
|
+
log "Backup created: ${LOCAL_PATH} ($(du -h "$LOCAL_PATH" | cut -f1))"
|
|
59
|
+
|
|
60
|
+
if ! aws s3 --endpoint-url "$R2_ENDPOINT" cp "$LOCAL_PATH" "${S3_BUCKET}/${BACKUP_FILE}"; then
|
|
61
|
+
log "ERROR: S3 upload failed"; exit 1
|
|
62
|
+
fi
|
|
63
|
+
log "Uploaded to ${S3_BUCKET}/${BACKUP_FILE}"
|
|
64
|
+
|
|
65
|
+
find "$LOCAL_BACKUP_DIR" -name "<your-app>_*.sql.gz" -mtime +"$RETENTION_DAYS" -delete
|
|
66
|
+
|
|
67
|
+
CUTOFF_ISO=$(date -u -d "-${RETENTION_DAYS} days" +%Y-%m-%dT%H:%M:%SZ)
|
|
68
|
+
BUCKET_NAME="${S3_BUCKET#s3://}"
|
|
69
|
+
OLD_KEYS=$(aws s3api --endpoint-url "$R2_ENDPOINT" list-objects-v2 \
|
|
70
|
+
--bucket "$BUCKET_NAME" \
|
|
71
|
+
--query "Contents[?LastModified<'${CUTOFF_ISO}'].Key" \
|
|
72
|
+
--output text 2>/dev/null || echo "")
|
|
73
|
+
if [[ -n "$OLD_KEYS" && "$OLD_KEYS" != "None" ]]; then
|
|
74
|
+
for KEY in $OLD_KEYS; do
|
|
75
|
+
aws s3 --endpoint-url "$R2_ENDPOINT" rm "${S3_BUCKET}/${KEY}" \
|
|
76
|
+
&& log "Deleted remote: ${KEY}" \
|
|
77
|
+
|| log "WARN: failed to delete remote ${KEY}"
|
|
78
|
+
done
|
|
79
|
+
fi
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
**3. VPS prerequisites.** Add `awscli` and `cron` to `cloud-init` `packages:` so future server replacements have them. Configure R2 credentials in `/root/.aws/credentials` (same keys as the Terraform state backend — they're account-scoped):
|
|
83
|
+
|
|
84
|
+
```ini
|
|
85
|
+
[default]
|
|
86
|
+
aws_access_key_id = <R2 access key>
|
|
87
|
+
aws_secret_access_key = <R2 secret>
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
**4. Cron entry** (`/etc/cron.d/<your-app>-pg-backup`). Wrap with `flock -n` so a hung run doesn't get a second instance started 24h later:
|
|
91
|
+
|
|
92
|
+
```
|
|
93
|
+
0 2 * * * root /usr/bin/flock -n /var/lock/<your-app>-pg-backup.lock /usr/local/bin/pg_backup_s3.sh >> /var/log/<your-app>_pg_backup.log 2>&1
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
**5. Restore — `scripts/db-restore.sh`** (run from local). Lists server-side backups via SSH, prompts for selection, takes a safety dump first (with size check — refuses to proceed if pg_dump silently produced an empty file), then restores. Note `set -euo pipefail` inside the SSH heredocs (without it, a `pg_dump` failure inside a pipeline is masked by a successful `gzip`) and the `OVERWRITE <db>` confirmation phrase (a bare DB name is too easy to typo into):
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
#!/usr/bin/env bash
|
|
100
|
+
set -euo pipefail
|
|
101
|
+
SSH_ALIAS="<your-app>"
|
|
102
|
+
CONTAINER="<your-app>-postgres-1"
|
|
103
|
+
DB="<your-db>"
|
|
104
|
+
REMOTE_DIR="/var/backups/postgresql"
|
|
105
|
+
MIN_SAFETY_BYTES=1024
|
|
106
|
+
|
|
107
|
+
trap 'echo "Interrupted — DB may be inconsistent. Latest safety snapshot is in $SSH_ALIAS:$REMOTE_DIR."' INT TERM
|
|
108
|
+
|
|
109
|
+
ssh -o ConnectTimeout=5 "$SSH_ALIAS" "docker exec $CONTAINER pg_isready -U postgres -d $DB" >/dev/null \
|
|
110
|
+
|| { echo "Postgres not ready"; exit 1; }
|
|
111
|
+
|
|
112
|
+
mapfile -t BACKUPS < <(ssh "$SSH_ALIAS" "ls -1t $REMOTE_DIR/<your-app>_*.sql.gz 2>/dev/null | xargs -n1 basename")
|
|
113
|
+
[[ ${#BACKUPS[@]} -gt 0 ]] || { echo "No backups found"; exit 1; }
|
|
114
|
+
for i in "${!BACKUPS[@]}"; do printf " %2d) %s\n" "$((i+1))" "${BACKUPS[$i]}"; done
|
|
115
|
+
read -rp "Pick: " PICK
|
|
116
|
+
[[ "$PICK" =~ ^[0-9]+$ ]] && (( PICK >= 1 && PICK <= ${#BACKUPS[@]} )) || { echo "Invalid"; exit 1; }
|
|
117
|
+
CHOSEN="${BACKUPS[$((PICK-1))]}"
|
|
118
|
+
|
|
119
|
+
read -rp "Type 'OVERWRITE $DB' to confirm: " CONFIRM
|
|
120
|
+
[[ "$CONFIRM" == "OVERWRITE $DB" ]] || { echo "aborted"; exit 1; }
|
|
121
|
+
|
|
122
|
+
SAFETY="<your-app>_pre_restore_$(date +%Y-%m-%d_%H%M%S).sql.gz"
|
|
123
|
+
ssh "$SSH_ALIAS" "set -euo pipefail; docker exec $CONTAINER pg_dump -U postgres $DB | gzip > $REMOTE_DIR/$SAFETY"
|
|
124
|
+
SIZE=$(ssh "$SSH_ALIAS" "stat -c%s $REMOTE_DIR/$SAFETY")
|
|
125
|
+
(( SIZE >= MIN_SAFETY_BYTES )) || { echo "Safety snapshot suspiciously small ($SIZE B) — aborting"; exit 1; }
|
|
126
|
+
|
|
127
|
+
ssh "$SSH_ALIAS" "
|
|
128
|
+
set -euo pipefail
|
|
129
|
+
docker exec $CONTAINER psql -U postgres -d postgres -c \"DROP DATABASE IF EXISTS \\\"$DB\\\" WITH (FORCE);\"
|
|
130
|
+
docker exec $CONTAINER psql -U postgres -d postgres -c \"CREATE DATABASE \\\"$DB\\\";\"
|
|
131
|
+
gunzip -c $REMOTE_DIR/$CHOSEN | docker exec -i $CONTAINER psql -U postgres -d $DB
|
|
132
|
+
"
|
|
133
|
+
echo "Restored. Safety snapshot at $SSH_ALIAS:$REMOTE_DIR/$SAFETY."
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
**6. Restore drill — quarterly.** A backup you've never restored from is not a backup. At least once a quarter, restore the latest dump into a throwaway local Postgres and verify schema + row counts. Calendar reminder.
|
|
137
|
+
|
|
138
|
+
## Log archival (optional)
|
|
139
|
+
|
|
140
|
+
When to use it: you need long-term log retention beyond Docker's `json-file` rotation buffer (default ~150 MB rolling per container, configured in compose).
|
|
141
|
+
|
|
142
|
+
Pattern: a Hangfire recurring job watches `/app/logs/`, ships files older than N days to R2 once total > threshold, deletes locally.
|
|
143
|
+
|
|
144
|
+
To make `/app/logs` writable under `USER app`:
|
|
145
|
+
|
|
146
|
+
- **Bind mount with chowned host dir** (recommended): `volumes: - /var/log/<your-app>:/app/logs` in compose, one-time `mkdir -p /var/log/<your-app> && chown 1654:1654 /var/log/<your-app>` on the VPS. Bind mounts respect host ownership; named volumes overlay the path with root-owned storage which `app` can't write to.
|
|
147
|
+
- **No File sink at all** (simpler): rely on Console + Docker rotation. Drop the `/app/logs` volume entirely. Right answer for greenfield deploys.
|
|
148
|
+
|
|
@@ -1,18 +1,18 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: deployment
|
|
3
|
-
description: Deploy a Spiderly project to
|
|
3
|
+
description: Deploy a Spiderly project to production and keep it running. Use when deploying, redeploying, shipping, releasing, or rolling out the .NET backend or Angular admin — first-time setup or an ongoing deploy — and when diagnosing a down or erroring production origin (502/521, container crash-loop, failed deploy workflow). Covers the recommended VPS + Docker Compose + Caddy + Cloudflare + Terraform stack, CI/CD pipelines, TLS with Cloudflare origin certificates, and infrastructure-as-code layout.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Deployment
|
|
7
7
|
|
|
8
|
-
Spiderly is
|
|
8
|
+
Spiderly runs anywhere a .NET app does — there is no required host, orchestrator, or reverse proxy. The stack below is the one we **recommend** and test end to end: a single VPS running Docker Compose, fronted by Caddy and Cloudflare, with all infrastructure declared in Terraform. Swap any tier for your own — the only genuinely Spiderly-specific deploy concerns are flagged inline (EF migrations before app start, forwarded-header/proxy trust, cookie domain across subdomains, the CORS frontend URL, and the api + admin subdomain split).
|
|
9
9
|
|
|
10
10
|
## Recommended stack
|
|
11
11
|
|
|
12
12
|
| Tier | Choice | Why |
|
|
13
13
|
|---|---|---|
|
|
14
14
|
| Compute | **Hetzner Cloud** (or any VPS) | Predictable monthly cost, full control, no PaaS lock-in |
|
|
15
|
-
| Orchestration | **Docker Compose** | Single-host simplicity;
|
|
15
|
+
| Orchestration | **Docker Compose** | Single-host simplicity; reproducible, agent-friendly deploys |
|
|
16
16
|
| Reverse proxy / TLS | **Caddy v2** | Auto-config from Cloudflare origin certs, simple Caddyfile |
|
|
17
17
|
| DNS / WAF / CDN | **Cloudflare** (orange-cloud) | DDoS protection, origin certs, Turnstile |
|
|
18
18
|
| IaC | **Terraform** | Declarative; one source of truth for VPS + DNS + certs |
|
|
@@ -154,7 +154,7 @@ ENTRYPOINT ["dotnet", "<YourApp>.WebAPI.dll"]
|
|
|
154
154
|
|
|
155
155
|
**Run as non-root.** The aspnet image ships an `app` user (UID 1654). Add `USER app` after `COPY` for defense-in-depth — limits the blast radius of a container escape and matches the platform's sandbox model.
|
|
156
156
|
|
|
157
|
-
**Log volumes:** if you're using a Serilog File sink (not just Console) with a Hangfire log-archival job, see
|
|
157
|
+
**Log volumes:** if you're using a Serilog File sink (not just Console) with a Hangfire log-archival job, see **Log archival** in the `backups` skill for the full bind-mount-vs-Console-only trade-off. Greenfield deploys can stick with Console sink + Docker's `json-file` rotation and skip the volume entirely.
|
|
158
158
|
|
|
159
159
|
## Angular admin Dockerfile
|
|
160
160
|
|
|
@@ -343,146 +343,7 @@ For migration mechanics (creating migrations, the dedicated `*.Migrations` start
|
|
|
343
343
|
|
|
344
344
|
## Backups
|
|
345
345
|
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
### Database backups (required)
|
|
349
|
-
|
|
350
|
-
Daily `pg_dump` to a Cloudflare R2 bucket, 7-day retention both local and remote. ~24h RPO; if you need tighter, layer WAL archiving on top (separate skill).
|
|
351
|
-
|
|
352
|
-
**1. R2 bucket — Terraform-managed.** No chicken-and-egg here (only the *state* bucket has to be bootstrapped manually); manage data buckets like any other resource:
|
|
353
|
-
|
|
354
|
-
```hcl
|
|
355
|
-
# infrastructure/cloudflare-r2.tf
|
|
356
|
-
resource "cloudflare_r2_bucket" "db_backups" {
|
|
357
|
-
account_id = var.cloudflare_account_id
|
|
358
|
-
name = "<your-app>-db-backups"
|
|
359
|
-
location = "EEUR"
|
|
360
|
-
|
|
361
|
-
lifecycle {
|
|
362
|
-
prevent_destroy = true
|
|
363
|
-
}
|
|
364
|
-
}
|
|
365
|
-
```
|
|
366
|
-
|
|
367
|
-
**2. Backup script — `infrastructure/scripts/pg_backup_s3.sh`** (deployed to `/usr/local/bin/` on the VPS). Note the `trap` cleanup, the `aws s3api list-objects-v2 --query` for retention (structured + locale-safe vs parsing `aws s3 ls` with awk), and the per-iteration warn-on-failure inside the `while` subshell (where `set -e` does NOT propagate):
|
|
368
|
-
|
|
369
|
-
```bash
|
|
370
|
-
#!/usr/bin/env bash
|
|
371
|
-
set -euo pipefail
|
|
372
|
-
|
|
373
|
-
CONTAINER_NAME="<your-app>-postgres-1"
|
|
374
|
-
DB_NAME="<your-db>"
|
|
375
|
-
DB_USER="postgres"
|
|
376
|
-
CLOUDFLARE_ACCOUNT_ID="<account-id>"
|
|
377
|
-
S3_BUCKET="s3://<your-app>-db-backups"
|
|
378
|
-
R2_ENDPOINT="https://${CLOUDFLARE_ACCOUNT_ID}.r2.cloudflarestorage.com"
|
|
379
|
-
RETENTION_DAYS=7
|
|
380
|
-
LOCAL_BACKUP_DIR="/var/backups/postgresql"
|
|
381
|
-
LOG_FILE="/var/log/<your-app>_pg_backup.log"
|
|
382
|
-
|
|
383
|
-
TIMESTAMP=$(date +%Y-%m-%d_%H%M%S)
|
|
384
|
-
BACKUP_FILE="<your-app>_${TIMESTAMP}.sql.gz"
|
|
385
|
-
LOCAL_PATH="${LOCAL_BACKUP_DIR}/${BACKUP_FILE}"
|
|
386
|
-
|
|
387
|
-
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"; }
|
|
388
|
-
|
|
389
|
-
trap 'rc=$?; [[ $rc -ne 0 ]] && rm -f "$LOCAL_PATH" && log "Aborted, removed partial $LOCAL_PATH"; exit $rc' ERR INT TERM
|
|
390
|
-
|
|
391
|
-
mkdir -p "$LOCAL_BACKUP_DIR"
|
|
392
|
-
|
|
393
|
-
if ! docker exec "$CONTAINER_NAME" pg_dump -U "$DB_USER" "$DB_NAME" | gzip > "$LOCAL_PATH"; then
|
|
394
|
-
log "ERROR: pg_dump failed"; exit 1
|
|
395
|
-
fi
|
|
396
|
-
log "Backup created: ${LOCAL_PATH} ($(du -h "$LOCAL_PATH" | cut -f1))"
|
|
397
|
-
|
|
398
|
-
if ! aws s3 --endpoint-url "$R2_ENDPOINT" cp "$LOCAL_PATH" "${S3_BUCKET}/${BACKUP_FILE}"; then
|
|
399
|
-
log "ERROR: S3 upload failed"; exit 1
|
|
400
|
-
fi
|
|
401
|
-
log "Uploaded to ${S3_BUCKET}/${BACKUP_FILE}"
|
|
402
|
-
|
|
403
|
-
find "$LOCAL_BACKUP_DIR" -name "<your-app>_*.sql.gz" -mtime +"$RETENTION_DAYS" -delete
|
|
404
|
-
|
|
405
|
-
CUTOFF_ISO=$(date -u -d "-${RETENTION_DAYS} days" +%Y-%m-%dT%H:%M:%SZ)
|
|
406
|
-
BUCKET_NAME="${S3_BUCKET#s3://}"
|
|
407
|
-
OLD_KEYS=$(aws s3api --endpoint-url "$R2_ENDPOINT" list-objects-v2 \
|
|
408
|
-
--bucket "$BUCKET_NAME" \
|
|
409
|
-
--query "Contents[?LastModified<'${CUTOFF_ISO}'].Key" \
|
|
410
|
-
--output text 2>/dev/null || echo "")
|
|
411
|
-
if [[ -n "$OLD_KEYS" && "$OLD_KEYS" != "None" ]]; then
|
|
412
|
-
for KEY in $OLD_KEYS; do
|
|
413
|
-
aws s3 --endpoint-url "$R2_ENDPOINT" rm "${S3_BUCKET}/${KEY}" \
|
|
414
|
-
&& log "Deleted remote: ${KEY}" \
|
|
415
|
-
|| log "WARN: failed to delete remote ${KEY}"
|
|
416
|
-
done
|
|
417
|
-
fi
|
|
418
|
-
```
|
|
419
|
-
|
|
420
|
-
**3. VPS prerequisites.** Add `awscli` and `cron` to `cloud-init` `packages:` so future server replacements have them. Configure R2 credentials in `/root/.aws/credentials` (same keys as the Terraform state backend — they're account-scoped):
|
|
421
|
-
|
|
422
|
-
```ini
|
|
423
|
-
[default]
|
|
424
|
-
aws_access_key_id = <R2 access key>
|
|
425
|
-
aws_secret_access_key = <R2 secret>
|
|
426
|
-
```
|
|
427
|
-
|
|
428
|
-
**4. Cron entry** (`/etc/cron.d/<your-app>-pg-backup`). Wrap with `flock -n` so a hung run doesn't get a second instance started 24h later:
|
|
429
|
-
|
|
430
|
-
```
|
|
431
|
-
0 2 * * * root /usr/bin/flock -n /var/lock/<your-app>-pg-backup.lock /usr/local/bin/pg_backup_s3.sh >> /var/log/<your-app>_pg_backup.log 2>&1
|
|
432
|
-
```
|
|
433
|
-
|
|
434
|
-
**5. Restore — `scripts/db-restore.sh`** (run from local). Lists server-side backups via SSH, prompts for selection, takes a safety dump first (with size check — refuses to proceed if pg_dump silently produced an empty file), then restores. Note `set -euo pipefail` inside the SSH heredocs (without it, a `pg_dump` failure inside a pipeline is masked by a successful `gzip`) and the `OVERWRITE <db>` confirmation phrase (a bare DB name is too easy to typo into):
|
|
435
|
-
|
|
436
|
-
```bash
|
|
437
|
-
#!/usr/bin/env bash
|
|
438
|
-
set -euo pipefail
|
|
439
|
-
SSH_ALIAS="<your-app>"
|
|
440
|
-
CONTAINER="<your-app>-postgres-1"
|
|
441
|
-
DB="<your-db>"
|
|
442
|
-
REMOTE_DIR="/var/backups/postgresql"
|
|
443
|
-
MIN_SAFETY_BYTES=1024
|
|
444
|
-
|
|
445
|
-
trap 'echo "Interrupted — DB may be inconsistent. Latest safety snapshot is in $SSH_ALIAS:$REMOTE_DIR."' INT TERM
|
|
446
|
-
|
|
447
|
-
ssh -o ConnectTimeout=5 "$SSH_ALIAS" "docker exec $CONTAINER pg_isready -U postgres -d $DB" >/dev/null \
|
|
448
|
-
|| { echo "Postgres not ready"; exit 1; }
|
|
449
|
-
|
|
450
|
-
mapfile -t BACKUPS < <(ssh "$SSH_ALIAS" "ls -1t $REMOTE_DIR/<your-app>_*.sql.gz 2>/dev/null | xargs -n1 basename")
|
|
451
|
-
[[ ${#BACKUPS[@]} -gt 0 ]] || { echo "No backups found"; exit 1; }
|
|
452
|
-
for i in "${!BACKUPS[@]}"; do printf " %2d) %s\n" "$((i+1))" "${BACKUPS[$i]}"; done
|
|
453
|
-
read -rp "Pick: " PICK
|
|
454
|
-
[[ "$PICK" =~ ^[0-9]+$ ]] && (( PICK >= 1 && PICK <= ${#BACKUPS[@]} )) || { echo "Invalid"; exit 1; }
|
|
455
|
-
CHOSEN="${BACKUPS[$((PICK-1))]}"
|
|
456
|
-
|
|
457
|
-
read -rp "Type 'OVERWRITE $DB' to confirm: " CONFIRM
|
|
458
|
-
[[ "$CONFIRM" == "OVERWRITE $DB" ]] || { echo "aborted"; exit 1; }
|
|
459
|
-
|
|
460
|
-
SAFETY="<your-app>_pre_restore_$(date +%Y-%m-%d_%H%M%S).sql.gz"
|
|
461
|
-
ssh "$SSH_ALIAS" "set -euo pipefail; docker exec $CONTAINER pg_dump -U postgres $DB | gzip > $REMOTE_DIR/$SAFETY"
|
|
462
|
-
SIZE=$(ssh "$SSH_ALIAS" "stat -c%s $REMOTE_DIR/$SAFETY")
|
|
463
|
-
(( SIZE >= MIN_SAFETY_BYTES )) || { echo "Safety snapshot suspiciously small ($SIZE B) — aborting"; exit 1; }
|
|
464
|
-
|
|
465
|
-
ssh "$SSH_ALIAS" "
|
|
466
|
-
set -euo pipefail
|
|
467
|
-
docker exec $CONTAINER psql -U postgres -d postgres -c \"DROP DATABASE IF EXISTS \\\"$DB\\\" WITH (FORCE);\"
|
|
468
|
-
docker exec $CONTAINER psql -U postgres -d postgres -c \"CREATE DATABASE \\\"$DB\\\";\"
|
|
469
|
-
gunzip -c $REMOTE_DIR/$CHOSEN | docker exec -i $CONTAINER psql -U postgres -d $DB
|
|
470
|
-
"
|
|
471
|
-
echo "Restored. Safety snapshot at $SSH_ALIAS:$REMOTE_DIR/$SAFETY."
|
|
472
|
-
```
|
|
473
|
-
|
|
474
|
-
**6. Restore drill — quarterly.** A backup you've never restored from is not a backup. At least once a quarter, restore the latest dump into a throwaway local Postgres and verify schema + row counts. Calendar reminder.
|
|
475
|
-
|
|
476
|
-
### Log archival (optional)
|
|
477
|
-
|
|
478
|
-
When to use it: you need long-term log retention beyond Docker's `json-file` rotation buffer (default ~150 MB rolling per container, configured in compose).
|
|
479
|
-
|
|
480
|
-
Pattern: a Hangfire recurring job watches `/app/logs/`, ships files older than N days to R2 once total > threshold, deletes locally.
|
|
481
|
-
|
|
482
|
-
To make `/app/logs` writable under `USER app`:
|
|
483
|
-
|
|
484
|
-
- **Bind mount with chowned host dir** (recommended): `volumes: - /var/log/<your-app>:/app/logs` in compose, one-time `mkdir -p /var/log/<your-app> && chown 1654:1654 /var/log/<your-app>` on the VPS. Bind mounts respect host ownership; named volumes overlay the path with root-owned storage which `app` can't write to.
|
|
485
|
-
- **No File sink at all** (simpler): rely on Console + Docker rotation. Drop the `/app/logs` volume entirely. Right answer for greenfield deploys.
|
|
346
|
+
Postgres data lives in a Docker volume on a single VPS — disk failure, accidental `docker volume rm`, or `terraform destroy` all wipe it without a snapshot to restore from. **If you ship this stack to prod, ship a backup with it:** daily `pg_dump` to Cloudflare R2 with local + remote retention, tested restore + restore-drill scripts, and optional log archival. See the **`backups`** skill for the full setup (R2 bucket Terraform, backup and restore scripts, cron, log archival).
|
|
486
347
|
|
|
487
348
|
## Pitfalls
|
|
488
349
|
|