sandcms 0.7.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -4
- package/dist/admin/Admin.svelte +7 -5
- package/dist/admin/AssetAttachment.svelte +8 -3
- package/dist/admin/AssetAttachment.svelte.d.ts +3 -0
- package/dist/admin/Editor.svelte +5 -9
- package/dist/admin/Fields.svelte +2 -1
- package/dist/admin/Gallery.svelte +221 -76
- package/dist/admin/Inbox.svelte +3 -2
- package/dist/admin/MediaPicker.svelte +36 -20
- package/dist/admin/Navigation.svelte +13 -2
- package/dist/admin/Pages.svelte +195 -0
- package/dist/admin/Pages.svelte.d.ts +7 -0
- package/dist/admin/Shell.svelte +27 -5
- package/dist/admin/SortableList.svelte +82 -61
- package/dist/admin/SortableList.svelte.d.ts +3 -1
- package/dist/admin/Users.svelte +8 -3
- package/dist/admin/generated/admin.css +300 -20
- package/dist/admin/navigation.d.ts +9 -0
- package/dist/admin/navigation.js +20 -1
- package/dist/admin/types.d.ts +1 -0
- package/dist/blueprints/labels.d.ts +6 -0
- package/dist/blueprints/labels.js +14 -0
- package/dist/blueprints/load.js +11 -2
- package/dist/blueprints/types.d.ts +2 -2
- package/dist/cli/admin.d.ts +39 -0
- package/dist/cli/admin.js +95 -0
- package/dist/cli/backup.js +4 -4
- package/dist/cli/doctor.d.ts +8 -1
- package/dist/cli/doctor.js +19 -9
- package/dist/cli/email-test.d.ts +3 -0
- package/dist/cli/email-test.js +20 -0
- package/dist/cli/files.js +2 -1
- package/dist/cli/index.js +373 -135
- package/dist/cli/rebaseline.d.ts +10 -0
- package/dist/cli/rebaseline.js +117 -0
- package/dist/cli/upgrade.js +17 -9
- package/dist/env.d.ts +0 -8
- package/dist/env.js +0 -8
- package/dist/notification-fields.js +2 -1
- package/dist/server/admin-load.d.ts +72 -0
- package/dist/server/admin-load.js +20 -0
- package/dist/server/api-keys.d.ts +96 -0
- package/dist/server/api-keys.js +115 -0
- package/dist/server/api.js +16 -61
- package/dist/server/cms.d.ts +0 -5
- package/dist/server/cms.js +4 -3
- package/dist/server/db/auto-migrate.d.ts +0 -4
- package/dist/server/db/auto-migrate.js +12 -3
- package/dist/server/db/legacy.d.ts +13 -0
- package/dist/server/db/legacy.js +60 -0
- package/dist/server/db/migration-data.js +2 -38
- package/dist/server/db/migrations/{0000_foundation.sql → 0000_baseline.sql} +120 -4
- package/dist/server/db/migrations/meta/0000_snapshot.json +578 -3
- package/dist/server/db/migrations/meta/_journal.json +2 -65
- package/dist/server/db/schema.js +8 -6
- package/dist/server/db/store.d.ts +25 -2
- package/dist/server/db/store.js +90 -26
- package/dist/server/forms.js +2 -1
- package/dist/server/media/runner.d.ts +1 -1
- package/dist/server/media/runner.js +1 -1
- package/dist/server/media/service.d.ts +0 -4
- package/dist/server/media/service.js +0 -2
- package/dist/server/media/verify.d.ts +28 -0
- package/dist/server/media/verify.js +94 -2
- package/dist/server/platform/snapshots.js +3 -3
- package/docs/acceptance.md +2 -2
- package/docs/api.md +15 -1
- package/docs/cloudflare.md +9 -41
- package/docs/content.md +18 -3
- package/docs/deployment.md +97 -34
- package/docs/email.md +6 -0
- package/docs/forms.md +1 -1
- package/docs/media.md +10 -1
- package/docs/package.md +4 -4
- package/docs/releases.md +21 -1
- package/docs/upgrades.md +41 -7
- package/package.json +4 -10
- package/templates/Dockerfile +6 -2
- package/templates/integration.ts +10 -5
- package/templates/openship.json +9 -0
- package/templates/sandcms/config.yaml +1 -0
- package/dist/server/db/migrations/0001_independent_password_resets.sql +0 -3
- package/dist/server/db/migrations/0002_email_failure_reason.sql +0 -1
- package/dist/server/db/migrations/0003_direct_r2_uploads.sql +0 -10
- package/dist/server/db/migrations/0004_agent_api_keys.sql +0 -38
- package/dist/server/db/migrations/0005_api_retention_and_pagination.sql +0 -3
- package/dist/server/db/migrations/0006_durable_media_deletion.sql +0 -7
- package/dist/server/db/migrations/0007_content_and_job_indexes.sql +0 -2
- package/dist/server/db/migrations/0008_search_text_cache.sql +0 -45
- package/dist/server/db/migrations/0009_local_media.sql +0 -9
- package/dist/server/db/migrations/meta/0001_snapshot.json +0 -1197
- package/dist/server/db/migrations/meta/0002_snapshot.json +0 -1204
- package/dist/server/db/migrations/meta/0003_snapshot.json +0 -1281
- package/dist/server/db/migrations/meta/0004_snapshot.json +0 -1546
- package/dist/server/db/migrations/meta/0005_snapshot.json +0 -1572
- package/dist/server/db/migrations/meta/0006_snapshot.json +0 -1627
- package/dist/server/db/migrations/meta/0007_snapshot.json +0 -1643
- package/dist/server/db/migrations/meta/0008_snapshot.json +0 -1711
- package/dist/server/db/migrations/meta/0009_snapshot.json +0 -1763
- package/dist/server/media/import-r2.d.ts +0 -56
- package/dist/server/media/import-r2.js +0 -342
- package/dist/worker/index.d.ts +0 -10
- package/dist/worker/index.js +0 -460
- package/dist/worker/local.d.ts +0 -5
- package/dist/worker/local.js +0 -57
- package/dist/worker/source.d.ts +0 -2
- package/dist/worker/source.js +0 -31
- package/dist/worker/svg.d.ts +0 -1
- package/dist/worker/svg.js +0 -2
- package/dist/worker/uploads.d.ts +0 -22
- package/dist/worker/uploads.js +0 -48
|
@@ -5,71 +5,8 @@
|
|
|
5
5
|
{
|
|
6
6
|
"idx": 0,
|
|
7
7
|
"version": "6",
|
|
8
|
-
"when":
|
|
9
|
-
"tag": "
|
|
10
|
-
"breakpoints": true
|
|
11
|
-
},
|
|
12
|
-
{
|
|
13
|
-
"idx": 1,
|
|
14
|
-
"version": "6",
|
|
15
|
-
"when": 1790941625583,
|
|
16
|
-
"tag": "0001_independent_password_resets",
|
|
17
|
-
"breakpoints": true
|
|
18
|
-
},
|
|
19
|
-
{
|
|
20
|
-
"idx": 2,
|
|
21
|
-
"version": "6",
|
|
22
|
-
"when": 1790943374287,
|
|
23
|
-
"tag": "0002_email_failure_reason",
|
|
24
|
-
"breakpoints": true
|
|
25
|
-
},
|
|
26
|
-
{
|
|
27
|
-
"idx": 3,
|
|
28
|
-
"version": "6",
|
|
29
|
-
"when": 1790943435700,
|
|
30
|
-
"tag": "0003_direct_r2_uploads",
|
|
31
|
-
"breakpoints": true
|
|
32
|
-
},
|
|
33
|
-
{
|
|
34
|
-
"idx": 4,
|
|
35
|
-
"version": "6",
|
|
36
|
-
"when": 1790948837850,
|
|
37
|
-
"tag": "0004_agent_api_keys",
|
|
38
|
-
"breakpoints": true
|
|
39
|
-
},
|
|
40
|
-
{
|
|
41
|
-
"idx": 5,
|
|
42
|
-
"version": "6",
|
|
43
|
-
"when": 1790950792615,
|
|
44
|
-
"tag": "0005_api_retention_and_pagination",
|
|
45
|
-
"breakpoints": true
|
|
46
|
-
},
|
|
47
|
-
{
|
|
48
|
-
"idx": 6,
|
|
49
|
-
"version": "6",
|
|
50
|
-
"when": 1791075641694,
|
|
51
|
-
"tag": "0006_durable_media_deletion",
|
|
52
|
-
"breakpoints": true
|
|
53
|
-
},
|
|
54
|
-
{
|
|
55
|
-
"idx": 7,
|
|
56
|
-
"version": "6",
|
|
57
|
-
"when": 1791405303147,
|
|
58
|
-
"tag": "0007_content_and_job_indexes",
|
|
59
|
-
"breakpoints": true
|
|
60
|
-
},
|
|
61
|
-
{
|
|
62
|
-
"idx": 8,
|
|
63
|
-
"version": "6",
|
|
64
|
-
"when": 1791407294932,
|
|
65
|
-
"tag": "0008_search_text_cache",
|
|
66
|
-
"breakpoints": true
|
|
67
|
-
},
|
|
68
|
-
{
|
|
69
|
-
"idx": 9,
|
|
70
|
-
"version": "6",
|
|
71
|
-
"when": 1794886948774,
|
|
72
|
-
"tag": "0009_local_media",
|
|
8
|
+
"when": 1795294799888,
|
|
9
|
+
"tag": "0000_baseline",
|
|
73
10
|
"breakpoints": true
|
|
74
11
|
}
|
|
75
12
|
]
|
package/dist/server/db/schema.js
CHANGED
|
@@ -4,7 +4,7 @@ const timestamp = (name) => integer(name, { mode: 'timestamp_ms' });
|
|
|
4
4
|
export const user = sqliteTable('user', {
|
|
5
5
|
id: text().primaryKey(),
|
|
6
6
|
name: text().notNull(),
|
|
7
|
-
email: text().notNull()
|
|
7
|
+
email: text().notNull(),
|
|
8
8
|
emailVerified: integer({ mode: 'boolean' }).notNull().default(false),
|
|
9
9
|
image: text(),
|
|
10
10
|
createdAt: timestamp('createdAt').notNull(),
|
|
@@ -14,11 +14,11 @@ export const user = sqliteTable('user', {
|
|
|
14
14
|
.default('editor'),
|
|
15
15
|
mustChangePassword: integer({ mode: 'boolean' }).notNull().default(true),
|
|
16
16
|
twoFactorEnabled: integer({ mode: 'boolean' }).default(false)
|
|
17
|
-
});
|
|
17
|
+
}, (t) => [uniqueIndex('user_email_unique').on(t.email)]);
|
|
18
18
|
export const session = sqliteTable('session', {
|
|
19
19
|
id: text().primaryKey(),
|
|
20
20
|
expiresAt: timestamp('expiresAt').notNull(),
|
|
21
|
-
token: text().notNull()
|
|
21
|
+
token: text().notNull(),
|
|
22
22
|
createdAt: timestamp('createdAt').notNull(),
|
|
23
23
|
updatedAt: timestamp('updatedAt').notNull(),
|
|
24
24
|
ipAddress: text(),
|
|
@@ -26,7 +26,7 @@ export const session = sqliteTable('session', {
|
|
|
26
26
|
userId: text()
|
|
27
27
|
.notNull()
|
|
28
28
|
.references(() => user.id, { onDelete: 'cascade' })
|
|
29
|
-
}, (t) => [index('session_user').on(t.userId)]);
|
|
29
|
+
}, (t) => [uniqueIndex('session_token_unique').on(t.token), index('session_user').on(t.userId)]);
|
|
30
30
|
export const account = sqliteTable('account', {
|
|
31
31
|
id: text().primaryKey(),
|
|
32
32
|
accountId: text().notNull(),
|
|
@@ -107,7 +107,7 @@ export const apiRequests = sqliteTable('api_requests', {
|
|
|
107
107
|
export const documents = sqliteTable('documents', {
|
|
108
108
|
id: text().primaryKey(),
|
|
109
109
|
blueprint: text().notNull(),
|
|
110
|
-
singleton: text()
|
|
110
|
+
singleton: text(),
|
|
111
111
|
draft: text({ mode: 'json' }).$type().notNull(),
|
|
112
112
|
published: text({ mode: 'json' }).$type(),
|
|
113
113
|
draftSlug: text(),
|
|
@@ -119,6 +119,7 @@ export const documents = sqliteTable('documents', {
|
|
|
119
119
|
// Normalized search text cache; NULL means stale or not yet computed. Maintained by triggers.
|
|
120
120
|
searchText: text()
|
|
121
121
|
}, (t) => [
|
|
122
|
+
uniqueIndex('documents_singleton_unique').on(t.singleton),
|
|
122
123
|
uniqueIndex('draft_slug').on(t.blueprint, t.draftSlug),
|
|
123
124
|
uniqueIndex('published_slug').on(t.blueprint, t.publishedSlug),
|
|
124
125
|
index('documents_blueprint').on(t.blueprint),
|
|
@@ -214,13 +215,14 @@ export const accessLinks = sqliteTable('access_links', {
|
|
|
214
215
|
role: text({ enum: ['admin', 'editor'] })
|
|
215
216
|
.notNull()
|
|
216
217
|
.default('editor'),
|
|
217
|
-
tokenHash: text().notNull()
|
|
218
|
+
tokenHash: text().notNull(),
|
|
218
219
|
credentialHash: text(),
|
|
219
220
|
expiresAt: integer().notNull(),
|
|
220
221
|
status: text({ enum: ['sending', 'accepted', 'failed'] }).notNull(),
|
|
221
222
|
failureReason: text().$type(),
|
|
222
223
|
createdAt: integer().notNull()
|
|
223
224
|
}, (t) => [
|
|
225
|
+
uniqueIndex('access_links_tokenHash_unique').on(t.tokenHash),
|
|
224
226
|
index('access_link_recipient').on(t.kind, t.email),
|
|
225
227
|
uniqueIndex('access_invite_recipient')
|
|
226
228
|
.on(t.kind, t.email)
|
|
@@ -2,11 +2,18 @@ import { Database } from 'bun:sqlite';
|
|
|
2
2
|
import { type BunSQLiteDatabase } from 'drizzle-orm/bun-sqlite';
|
|
3
3
|
import * as schema from './schema.js';
|
|
4
4
|
export declare const SCHEMA_VERSION: number;
|
|
5
|
+
/**
|
|
6
|
+
* A migration whose SQL contains this line cannot be read by code built before it. Every other migration
|
|
7
|
+
* must keep the previous release working on the migrated database (overlapping deploys and rollbacks).
|
|
8
|
+
*/
|
|
9
|
+
export declare const BREAKING_MARKER: RegExp;
|
|
5
10
|
export declare function migrations(): {
|
|
6
11
|
hash: string;
|
|
12
|
+
breaking: boolean;
|
|
7
13
|
name: string;
|
|
8
14
|
sql: string;
|
|
9
15
|
}[];
|
|
16
|
+
export type Migration = ReturnType<typeof migrations>[number];
|
|
10
17
|
export declare function databasePath(path: string): string;
|
|
11
18
|
export declare function volumeGuard(path: string, exclusive?: boolean): Database | undefined;
|
|
12
19
|
export interface Store {
|
|
@@ -19,15 +26,31 @@ export declare function openStore(path: string, { migrate, guard }?: {
|
|
|
19
26
|
migrate?: boolean | undefined;
|
|
20
27
|
guard?: boolean | undefined;
|
|
21
28
|
}): Store;
|
|
22
|
-
export
|
|
29
|
+
export interface SchemaHistory {
|
|
30
|
+
/** Migrations this package has and the database lacks. */
|
|
31
|
+
pending: Migration[];
|
|
32
|
+
/** Non-breaking migrations a newer release applied that this package does not know. */
|
|
33
|
+
ahead: {
|
|
34
|
+
name: string;
|
|
35
|
+
hash: string;
|
|
36
|
+
}[];
|
|
37
|
+
}
|
|
38
|
+
/**
|
|
39
|
+
* Compare the database's migration history with this package. Throws for a foreign, modified or
|
|
40
|
+
* inconsistent history, a pre-0.8.0 database, or a newer release's breaking migration.
|
|
41
|
+
*/
|
|
42
|
+
export declare function schemaHistory(sqlite: Database): SchemaHistory;
|
|
43
|
+
export declare function pendingMigrations(store: Pick<Store, 'sqlite'>): {
|
|
23
44
|
hash: string;
|
|
45
|
+
breaking: boolean;
|
|
24
46
|
name: string;
|
|
25
47
|
sql: string;
|
|
26
48
|
}[];
|
|
27
|
-
export declare function assertSchema(store: Store):
|
|
49
|
+
export declare function assertSchema(store: Store): SchemaHistory;
|
|
28
50
|
export declare function applyMigrations(store: Store): string[];
|
|
29
51
|
export declare function inspectMigrations(path: string): {
|
|
30
52
|
hash: string;
|
|
53
|
+
breaking: boolean;
|
|
31
54
|
name: string;
|
|
32
55
|
sql: string;
|
|
33
56
|
}[];
|
package/dist/server/db/store.js
CHANGED
|
@@ -5,12 +5,19 @@ import { dirname, resolve, basename, join } from 'node:path';
|
|
|
5
5
|
import { createHash } from 'node:crypto';
|
|
6
6
|
import * as schema from './schema.js';
|
|
7
7
|
import { migrationData } from './migration-data.js';
|
|
8
|
+
import { assertNotLegacy } from './legacy.js';
|
|
8
9
|
export const SCHEMA_VERSION = migrationData.length;
|
|
9
|
-
const resetGuidance = 'Use the matching package version for this database.
|
|
10
|
+
const resetGuidance = 'Use the matching package version for this database. Databases from before the 0.8.0 baseline must be converted with sandcms rebaseline. See docs/upgrades.md.';
|
|
11
|
+
/**
|
|
12
|
+
* A migration whose SQL contains this line cannot be read by code built before it. Every other migration
|
|
13
|
+
* must keep the previous release working on the migrated database (overlapping deploys and rollbacks).
|
|
14
|
+
*/
|
|
15
|
+
export const BREAKING_MARKER = /^--\s*sandcms:breaking\s*$/m;
|
|
10
16
|
export function migrations() {
|
|
11
17
|
return migrationData.map((m) => ({
|
|
12
18
|
...m,
|
|
13
|
-
hash: createHash('sha256').update(m.sql).digest('hex')
|
|
19
|
+
hash: createHash('sha256').update(m.sql).digest('hex'),
|
|
20
|
+
breaking: BREAKING_MARKER.test(m.sql)
|
|
14
21
|
}));
|
|
15
22
|
}
|
|
16
23
|
// A separate DELETE-mode database holds shared locks for the lifetime of every connection.
|
|
@@ -54,7 +61,7 @@ export function openStore(path, { migrate = false, guard = true } = {}) {
|
|
|
54
61
|
const store = {
|
|
55
62
|
path,
|
|
56
63
|
sqlite,
|
|
57
|
-
db: drizzle(sqlite,
|
|
64
|
+
db: drizzle({ client: sqlite, schema }),
|
|
58
65
|
close() {
|
|
59
66
|
sqlite.close();
|
|
60
67
|
lock?.close();
|
|
@@ -70,41 +77,91 @@ export function openStore(path, { migrate = false, guard = true } = {}) {
|
|
|
70
77
|
throw e;
|
|
71
78
|
}
|
|
72
79
|
}
|
|
73
|
-
|
|
80
|
+
const MIGRATIONS_TABLE = 'CREATE TABLE IF NOT EXISTS schema_migrations (name TEXT PRIMARY KEY, hash TEXT NOT NULL, applied_at INTEGER NOT NULL, breaking INTEGER NOT NULL DEFAULT 0)';
|
|
81
|
+
/**
|
|
82
|
+
* Compare the database's migration history with this package. Throws for a foreign, modified or
|
|
83
|
+
* inconsistent history, a pre-0.8.0 database, or a newer release's breaking migration.
|
|
84
|
+
*/
|
|
85
|
+
export function schemaHistory(sqlite) {
|
|
86
|
+
const tracked = sqlite
|
|
87
|
+
.query("SELECT name FROM pragma_table_info('schema_migrations')")
|
|
88
|
+
.all()
|
|
89
|
+
.map((c) => c.name);
|
|
90
|
+
if (tracked.length && !tracked.includes('breaking'))
|
|
91
|
+
assertNotLegacy(sqlite);
|
|
74
92
|
// Each statement reads one snapshot, so a concurrent migration cannot mix version and history.
|
|
75
|
-
const
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
const rows = state.tracked
|
|
79
|
-
? store.sqlite
|
|
80
|
-
.query('SELECT NULL AS name, NULL AS hash, user_version FROM pragma_user_version UNION ALL SELECT name, hash, NULL FROM schema_migrations ORDER BY name')
|
|
93
|
+
const rows = tracked.length
|
|
94
|
+
? sqlite
|
|
95
|
+
.query('SELECT NULL AS name, NULL AS hash, NULL AS breaking, user_version FROM pragma_user_version UNION ALL SELECT name, hash, breaking, NULL FROM schema_migrations ORDER BY name')
|
|
81
96
|
.all()
|
|
82
|
-
:
|
|
83
|
-
const version =
|
|
97
|
+
: sqlite.query('SELECT user_version FROM pragma_user_version').all();
|
|
98
|
+
const version = rows[0].user_version;
|
|
84
99
|
const applied = rows.slice(1);
|
|
85
|
-
if (version.user_version > SCHEMA_VERSION)
|
|
86
|
-
throw new Error(`Database schema is incompatible with this package. ${resetGuidance}`);
|
|
87
100
|
const available = migrations();
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
101
|
+
if (version !== applied.length)
|
|
102
|
+
throw new Error(`Database migration history is inconsistent. ${resetGuidance}`);
|
|
103
|
+
for (const [index, old] of applied.entries()) {
|
|
104
|
+
const m = available[index];
|
|
105
|
+
if (!m)
|
|
106
|
+
break;
|
|
107
|
+
if (m.name !== old.name || m.hash !== old.hash)
|
|
108
|
+
throw new Error(available.some((item) => item.name === old.name)
|
|
109
|
+
? `Database migration history is inconsistent. ${resetGuidance}`
|
|
110
|
+
: `Incompatible or modified migration ${old.name}. ${resetGuidance}`);
|
|
92
111
|
}
|
|
93
|
-
|
|
94
|
-
|
|
112
|
+
const ahead = applied.slice(available.length);
|
|
113
|
+
if (ahead.length && ahead[0].name <= available[available.length - 1].name)
|
|
95
114
|
throw new Error(`Database migration history is inconsistent. ${resetGuidance}`);
|
|
96
|
-
|
|
115
|
+
const breaking = ahead.find((m) => m.breaking);
|
|
116
|
+
if (breaking)
|
|
117
|
+
throw new Error(`Database was migrated by a newer SandCMS release with the breaking migration ${breaking.name}; this package cannot use it. Deploy that release or a newer one.`);
|
|
118
|
+
return {
|
|
119
|
+
pending: available.slice(applied.length),
|
|
120
|
+
ahead: ahead.map(({ name, hash }) => ({ name, hash }))
|
|
121
|
+
};
|
|
122
|
+
}
|
|
123
|
+
export function pendingMigrations(store) {
|
|
124
|
+
return schemaHistory(store.sqlite).pending;
|
|
97
125
|
}
|
|
98
126
|
export function assertSchema(store) {
|
|
99
|
-
|
|
127
|
+
const history = schemaHistory(store.sqlite);
|
|
128
|
+
if (history.pending.length)
|
|
100
129
|
throw new Error('Database migrations are pending. Run sandcms migrate against this database.');
|
|
130
|
+
return history;
|
|
131
|
+
}
|
|
132
|
+
/**
|
|
133
|
+
* Table rebuilds (drizzle-kit's `PRAGMA foreign_keys=OFF` ... `__new_<table>`) need foreign keys off for
|
|
134
|
+
* the whole transaction, and SQLite ignores that pragma inside one. Triggers are dropped first and
|
|
135
|
+
* recreated afterwards: DROP TABLE removes a rebuilt table's triggers, and RENAME fails while a trigger
|
|
136
|
+
* on another table refers to the dropped name.
|
|
137
|
+
*/
|
|
138
|
+
function rebuilds(sql) {
|
|
139
|
+
return /PRAGMA\s+foreign_keys\s*=\s*(OFF|0|false)/i.test(sql);
|
|
140
|
+
}
|
|
141
|
+
function runMigration(sqlite, m) {
|
|
142
|
+
const triggers = sqlite
|
|
143
|
+
.query("SELECT name, sql FROM sqlite_master WHERE type = 'trigger'")
|
|
144
|
+
.all();
|
|
145
|
+
for (const t of triggers)
|
|
146
|
+
sqlite.exec(`DROP TRIGGER \`${t.name}\``);
|
|
147
|
+
sqlite.exec(m.sql);
|
|
148
|
+
const dropped = (name) => new RegExp(`DROP\\s+TRIGGER\\s+(IF\\s+EXISTS\\s+)?[\`"]?${name}[\`"]?`, 'i').test(m.sql);
|
|
149
|
+
for (const t of triggers)
|
|
150
|
+
if (!dropped(t.name) && !sqlite.query('SELECT 1 FROM sqlite_master WHERE name = ?').get(t.name))
|
|
151
|
+
sqlite.exec(t.sql);
|
|
152
|
+
const violations = sqlite.query('PRAGMA foreign_key_check').all();
|
|
153
|
+
if (violations.length)
|
|
154
|
+
throw new Error(`Table rebuild left invalid references: ${JSON.stringify(violations.slice(0, 5))}`);
|
|
101
155
|
}
|
|
102
156
|
export function applyMigrations(store) {
|
|
103
157
|
const pending = pendingMigrations(store);
|
|
104
|
-
store.sqlite.exec(
|
|
158
|
+
store.sqlite.exec(MIGRATIONS_TABLE);
|
|
105
159
|
const completed = [];
|
|
106
160
|
for (const m of pending) {
|
|
161
|
+
const rebuild = rebuilds(m.sql);
|
|
107
162
|
try {
|
|
163
|
+
if (rebuild)
|
|
164
|
+
store.sqlite.exec('PRAGMA foreign_keys=OFF');
|
|
108
165
|
const applied = store.sqlite
|
|
109
166
|
.transaction(() => {
|
|
110
167
|
// Another process starting concurrently may have applied it after the pending check.
|
|
@@ -116,10 +173,13 @@ export function applyMigrations(store) {
|
|
|
116
173
|
throw new Error(`Incompatible or modified migration ${m.name}. ${resetGuidance}`);
|
|
117
174
|
return false;
|
|
118
175
|
}
|
|
119
|
-
|
|
176
|
+
if (rebuild)
|
|
177
|
+
runMigration(store.sqlite, m);
|
|
178
|
+
else
|
|
179
|
+
store.sqlite.exec(m.sql);
|
|
120
180
|
store.sqlite
|
|
121
|
-
.query('INSERT INTO schema_migrations VALUES (?, ?, ?)')
|
|
122
|
-
.run(m.name, m.hash, Date.now());
|
|
181
|
+
.query('INSERT INTO schema_migrations (name, hash, applied_at, breaking) VALUES (?, ?, ?, ?)')
|
|
182
|
+
.run(m.name, m.hash, Date.now(), m.breaking ? 1 : 0);
|
|
123
183
|
store.sqlite.exec(`PRAGMA user_version=${migrations().findIndex((item) => item.name === m.name) + 1}`);
|
|
124
184
|
return true;
|
|
125
185
|
})
|
|
@@ -130,6 +190,10 @@ export function applyMigrations(store) {
|
|
|
130
190
|
catch (cause) {
|
|
131
191
|
throw new Error(`Migration ${m.name} failed. Applied in this run: ${completed.join(', ') || 'none'}. ${String(cause)}`, { cause });
|
|
132
192
|
}
|
|
193
|
+
finally {
|
|
194
|
+
if (rebuild)
|
|
195
|
+
store.sqlite.exec('PRAGMA foreign_keys=ON');
|
|
196
|
+
}
|
|
133
197
|
}
|
|
134
198
|
return completed;
|
|
135
199
|
}
|
package/dist/server/forms.js
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
import { createHash } from 'node:crypto';
|
|
2
2
|
import { desc, eq, lt } from 'drizzle-orm';
|
|
3
3
|
import { jsonPayload } from '../blueprints/validate.js';
|
|
4
|
+
import { fieldLabel } from '../blueprints/labels.js';
|
|
4
5
|
import { submissions, rateLimits } from './db/schema.js';
|
|
5
6
|
export const honeypotName = 'website_url';
|
|
6
7
|
export function rateLimit(store, key, max = 5, window = 60_000) {
|
|
@@ -69,7 +70,7 @@ export function createForms(store, definition, notify) {
|
|
|
69
70
|
const fields = new Map();
|
|
70
71
|
for (const row of rows)
|
|
71
72
|
for (const key of row.schema.export ?? Object.keys(row.schema.fields))
|
|
72
|
-
fields.set(key, row.schema.fields[key]
|
|
73
|
+
fields.set(key, fieldLabel(key, row.schema.fields[key]));
|
|
73
74
|
const lines = [
|
|
74
75
|
['Submitted', ...fields.values()],
|
|
75
76
|
...rows.map((r) => [
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import type { Store } from '../db/store.js';
|
|
2
2
|
import { type MediaProcessor, type MediaStorage, type VariantSpec } from './contracts.js';
|
|
3
3
|
/**
|
|
4
|
-
* Upload sessions that can still write or validate files: they reserve disk space
|
|
4
|
+
* Upload sessions that can still write or validate files: they reserve disk space.
|
|
5
5
|
* Expired, idle-past-expiry and permanently failed sessions are dead rows kept only for their error.
|
|
6
6
|
*/
|
|
7
7
|
export declare const liveUploadSession: (now?: number) => import("drizzle-orm").SQL<unknown> | undefined;
|
|
@@ -7,7 +7,7 @@ const uploadTimeout = 120_000, jobTimeout = 120_000, processorLease = 120_000, h
|
|
|
7
7
|
// Covers a typical preset set (sizes x formats plus the editor preview) in one decode.
|
|
8
8
|
batchSize = 16;
|
|
9
9
|
/**
|
|
10
|
-
* Upload sessions that can still write or validate files: they reserve disk space
|
|
10
|
+
* Upload sessions that can still write or validate files: they reserve disk space.
|
|
11
11
|
* Expired, idle-past-expiry and permanently failed sessions are dead rows kept only for their error.
|
|
12
12
|
*/
|
|
13
13
|
export const liveUploadSession = (now = Date.now()) => or(and(inArray(uploadSessions.status, ['uploading', 'receiving']), or(gt(uploadSessions.expiresAt, now), gt(uploadSessions.leaseUntil, now))), and(eq(uploadSessions.status, 'pending'), lt(uploadSessions.attempts, 4)));
|
|
@@ -9,8 +9,6 @@ export interface MediaOptions {
|
|
|
9
9
|
mediaDir: string;
|
|
10
10
|
processor?: MediaProcessor;
|
|
11
11
|
storage?: MediaStorage;
|
|
12
|
-
/** @deprecated 0.7.0: only read by `sandcms media import-r2`. */
|
|
13
|
-
workerUrl?: string;
|
|
14
12
|
/** Internal timings in ms, shortened by tests. */
|
|
15
13
|
timing?: {
|
|
16
14
|
heartbeat?: number;
|
|
@@ -111,8 +109,6 @@ export declare function createMedia(store: Store, definition: SiteDefinition, co
|
|
|
111
109
|
uploadAttempt: string | null;
|
|
112
110
|
} | undefined>>) => string;
|
|
113
111
|
storage: MediaStorage;
|
|
114
|
-
/** @deprecated 0.7.0: media no longer uses the Worker; kept for `sandcms media import-r2`. */
|
|
115
|
-
workerUrl: string;
|
|
116
112
|
list: () => {
|
|
117
113
|
blueprint: string;
|
|
118
114
|
metadata: Payload;
|
|
@@ -479,8 +479,6 @@ export function createMedia(store, definition, content, options) {
|
|
|
479
479
|
version,
|
|
480
480
|
versionForAsset: assetVersion,
|
|
481
481
|
storage,
|
|
482
|
-
/** @deprecated 0.7.0: media no longer uses the Worker; kept for `sandcms media import-r2`. */
|
|
483
|
-
workerUrl: options.workerUrl ?? settings.workerUrl ?? '',
|
|
484
482
|
list: () => db.select().from(media).orderBy(desc(media.createdAt)).all(),
|
|
485
483
|
updateMetadata,
|
|
486
484
|
async health() {
|
|
@@ -34,3 +34,31 @@ export declare function verifyMedia(cms: {
|
|
|
34
34
|
export declare function forceRebuild(cms: {
|
|
35
35
|
store: Store;
|
|
36
36
|
}): number;
|
|
37
|
+
export interface PruneItem {
|
|
38
|
+
/** Path relative to the media root, such as `originals/{id}` or `variants/{id}/{version}`. */
|
|
39
|
+
path: string;
|
|
40
|
+
bytes: number;
|
|
41
|
+
reason: 'no-asset' | 'old-version';
|
|
42
|
+
}
|
|
43
|
+
export interface PruneReport {
|
|
44
|
+
items: PruneItem[];
|
|
45
|
+
bytes: number;
|
|
46
|
+
removed: boolean;
|
|
47
|
+
}
|
|
48
|
+
/**
|
|
49
|
+
* Find originals and variant folders that no media row needs: files of assets the database no longer knows,
|
|
50
|
+
* and variant versions that are neither active, targeted, nor still being rendered (left behind by rebuilds).
|
|
51
|
+
* Entries changed within `minAge` ms are skipped so a concurrent upload or render is never reported.
|
|
52
|
+
* With `remove`, each candidate is checked against the database again just before it is deleted.
|
|
53
|
+
* The upload staging folder (tmp/) is swept by the running website and is not touched.
|
|
54
|
+
*/
|
|
55
|
+
export declare function pruneMedia(cms: {
|
|
56
|
+
store: Store;
|
|
57
|
+
media: {
|
|
58
|
+
storage: MediaStorage;
|
|
59
|
+
};
|
|
60
|
+
}, { remove, minAge, now }?: {
|
|
61
|
+
remove?: boolean | undefined;
|
|
62
|
+
minAge?: number | undefined;
|
|
63
|
+
now?: number | undefined;
|
|
64
|
+
}): PruneReport;
|
|
@@ -1,5 +1,8 @@
|
|
|
1
|
-
// Operator checks for local media: `sandcms media verify [--repair]
|
|
2
|
-
|
|
1
|
+
// Operator checks for local media: `sandcms media verify [--repair]`, `sandcms media rebuild --force` and
|
|
2
|
+
// `sandcms media prune [--delete]`.
|
|
3
|
+
import { readdirSync, rmSync, statSync } from 'node:fs';
|
|
4
|
+
import { join } from 'node:path';
|
|
5
|
+
import { and, eq, inArray, notExists } from 'drizzle-orm';
|
|
3
6
|
import { jobs, media, mediaDeletions, uploads, uploadSessions } from '../db/schema.js';
|
|
4
7
|
import { variantKey } from './runner.js';
|
|
5
8
|
const missingOriginal = 'Original file is missing. Upload this image again.';
|
|
@@ -107,3 +110,92 @@ export function forceRebuild(cms) {
|
|
|
107
110
|
});
|
|
108
111
|
return queued;
|
|
109
112
|
}
|
|
113
|
+
const assetId = /^[a-f0-9-]{36}$/;
|
|
114
|
+
function entries(dir) {
|
|
115
|
+
try {
|
|
116
|
+
return readdirSync(dir);
|
|
117
|
+
}
|
|
118
|
+
catch (error) {
|
|
119
|
+
if (error.code === 'ENOENT')
|
|
120
|
+
return [];
|
|
121
|
+
throw error;
|
|
122
|
+
}
|
|
123
|
+
}
|
|
124
|
+
function bytesOf(path) {
|
|
125
|
+
const info = statSync(path, { throwIfNoEntry: false });
|
|
126
|
+
if (!info)
|
|
127
|
+
return 0;
|
|
128
|
+
if (!info.isDirectory())
|
|
129
|
+
return info.size;
|
|
130
|
+
return entries(path).reduce((total, name) => total + bytesOf(join(path, name)), 0);
|
|
131
|
+
}
|
|
132
|
+
/**
|
|
133
|
+
* Find originals and variant folders that no media row needs: files of assets the database no longer knows,
|
|
134
|
+
* and variant versions that are neither active, targeted, nor still being rendered (left behind by rebuilds).
|
|
135
|
+
* Entries changed within `minAge` ms are skipped so a concurrent upload or render is never reported.
|
|
136
|
+
* With `remove`, each candidate is checked against the database again just before it is deleted.
|
|
137
|
+
* The upload staging folder (tmp/) is swept by the running website and is not touched.
|
|
138
|
+
*/
|
|
139
|
+
export function pruneMedia(cms, { remove = false, minAge = 60 * 60 * 1000, now = Date.now() } = {}) {
|
|
140
|
+
const { db } = cms.store, root = cms.media.storage.root;
|
|
141
|
+
const needed = (id, version) => {
|
|
142
|
+
const row = db
|
|
143
|
+
.select({ active: media.activeVersion, target: media.targetVersion })
|
|
144
|
+
.from(media)
|
|
145
|
+
.where(eq(media.id, id))
|
|
146
|
+
.get();
|
|
147
|
+
if (!row)
|
|
148
|
+
return false;
|
|
149
|
+
if (!version || version === row.active || version === row.target)
|
|
150
|
+
return true;
|
|
151
|
+
return Boolean(db
|
|
152
|
+
.select({ id: jobs.id })
|
|
153
|
+
.from(jobs)
|
|
154
|
+
.where(and(eq(jobs.mediaId, id), eq(jobs.version, version), inArray(jobs.status, ['pending', 'running'])))
|
|
155
|
+
.get());
|
|
156
|
+
};
|
|
157
|
+
const settled = (path) => {
|
|
158
|
+
const info = statSync(path, { throwIfNoEntry: false });
|
|
159
|
+
return Boolean(info && now - info.mtimeMs >= minAge);
|
|
160
|
+
};
|
|
161
|
+
const candidates = [];
|
|
162
|
+
for (const id of entries(join(root, 'originals'))) {
|
|
163
|
+
const path = join(root, 'originals', id);
|
|
164
|
+
if (!assetId.test(id) || needed(id) || !settled(path))
|
|
165
|
+
continue;
|
|
166
|
+
candidates.push({ id, path: `originals/${id}`, bytes: bytesOf(path), reason: 'no-asset' });
|
|
167
|
+
}
|
|
168
|
+
for (const id of entries(join(root, 'variants'))) {
|
|
169
|
+
if (!assetId.test(id))
|
|
170
|
+
continue;
|
|
171
|
+
const dir = join(root, 'variants', id);
|
|
172
|
+
if (!needed(id)) {
|
|
173
|
+
if (settled(dir))
|
|
174
|
+
candidates.push({ id, path: `variants/${id}`, bytes: bytesOf(dir), reason: 'no-asset' });
|
|
175
|
+
continue;
|
|
176
|
+
}
|
|
177
|
+
for (const version of entries(dir)) {
|
|
178
|
+
const path = join(dir, version);
|
|
179
|
+
if (!/^[a-f0-9]{16}$/.test(version) || needed(id, version) || !settled(path))
|
|
180
|
+
continue;
|
|
181
|
+
candidates.push({
|
|
182
|
+
id,
|
|
183
|
+
version,
|
|
184
|
+
path: `variants/${id}/${version}`,
|
|
185
|
+
bytes: bytesOf(path),
|
|
186
|
+
reason: 'old-version'
|
|
187
|
+
});
|
|
188
|
+
}
|
|
189
|
+
}
|
|
190
|
+
const items = [];
|
|
191
|
+
for (const { id, version, ...item } of candidates) {
|
|
192
|
+
if (remove) {
|
|
193
|
+
// The website may have started using the asset or version since the scan.
|
|
194
|
+
if (needed(id, version))
|
|
195
|
+
continue;
|
|
196
|
+
rmSync(join(root, item.path), { recursive: true, force: true });
|
|
197
|
+
}
|
|
198
|
+
items.push(item);
|
|
199
|
+
}
|
|
200
|
+
return { items, bytes: items.reduce((total, item) => total + item.bytes, 0), removed: remove };
|
|
201
|
+
}
|
|
@@ -4,7 +4,7 @@ import { Database } from 'bun:sqlite';
|
|
|
4
4
|
import { existsSync, mkdirSync, readdirSync, renameSync, rmSync, statSync } from 'node:fs';
|
|
5
5
|
import { basename, dirname, join } from 'node:path';
|
|
6
6
|
import { backup } from '../../cli/backup.js';
|
|
7
|
-
import {
|
|
7
|
+
import { schemaHistory } from '../db/store.js';
|
|
8
8
|
export const SNAPSHOT_INTERVAL = 60 * 60 * 1000;
|
|
9
9
|
export const KEEP_MIGRATION_SNAPSHOTS = 3;
|
|
10
10
|
export function snapshotPaths(dataDir) {
|
|
@@ -77,9 +77,9 @@ export async function takeSnapshotAsync(database, dataDir) {
|
|
|
77
77
|
const copy = new Database(temp, { readonly: true });
|
|
78
78
|
try {
|
|
79
79
|
const { user_version } = copy.query('PRAGMA user_version').get();
|
|
80
|
-
if (user_version < 1
|
|
80
|
+
if (user_version < 1)
|
|
81
81
|
throw new Error('Incompatible snapshot schema');
|
|
82
|
-
|
|
82
|
+
schemaHistory(copy); // migration names and hashes; a newer release's non-breaking ones are fine
|
|
83
83
|
}
|
|
84
84
|
finally {
|
|
85
85
|
copy.close();
|
package/docs/acceptance.md
CHANGED
|
@@ -9,7 +9,7 @@ Verification date: **26 September 2026** for type checking, packed installation
|
|
|
9
9
|
```sh
|
|
10
10
|
bun install --frozen-lockfile
|
|
11
11
|
bun run check
|
|
12
|
-
bun test
|
|
12
|
+
bun run test
|
|
13
13
|
bun run build
|
|
14
14
|
bun x playwright install chromium
|
|
15
15
|
bun run test:e2e
|
|
@@ -54,7 +54,7 @@ The drill owns and removes its temporary containers and volume. It uses localhos
|
|
|
54
54
|
## External release checks still required
|
|
55
55
|
|
|
56
56
|
1. **Dokploy:** deploy the Railpack image with HTTPS, the production volume, runtime secrets, and the actual reverse proxy configuration. Bootstrap against that mounted database, publish content, replace the container, verify session/cookie and proxy behavior, and perform the documented backup/offline-restore drill. The local Docker proof covers the application mechanics; Dokploy's operator access, mounting, restart policy, and routing remain unverified.
|
|
57
|
-
2. **
|
|
57
|
+
2. **CDN:** where a CDN is used, verify the [cache rules](cloudflare.md#optional-cdn-caching) on a staging hostname. Local tests do not prove deployed cache behavior. The R2 import was removed in 0.8.0 and is checked with 0.7.x.
|
|
58
58
|
3. **Private npm:** publish `sandcms` with restricted access from an authorized organization account, and prove an authenticated pinned install in a clean CI environment. Validate secret handling in the real build host. Local archive installation is complete; registry authentication and publication remain unverified.
|
|
59
59
|
|
|
60
60
|
Live acceptance on a deployed host is pending. These checks are release gates, not prerequisites for continuing local development.
|
package/docs/api.md
CHANGED
|
@@ -6,7 +6,21 @@ Coding agents map each site's content to its repository-owned YAML blueprints. S
|
|
|
6
6
|
|
|
7
7
|
Sign in as an administrator, finish the temporary password change (and two-factor sign-in if enabled), and open **Account & security → API keys** and choose **Create API key**. Name a **Read** or **Read & write** key and copy it immediately. The secret is shown once; Better Auth stores its hash. Write keys expire after 7 days by default; read keys expire after 30 days. The form offers 7, 30, 90, or 365 days; `keys/create` accepts any whole-number lifetime from 1 to 365 days. Keys belong to the issuing account. Administrators can list and revoke keys for every account, including another administrator; the list shows owner, access, expiration, and last use. Revocations are audited.
|
|
8
8
|
|
|
9
|
-
Use `x-api-key` on `/api/cms/` requests. No browser cookie or `Origin` header is needed. Read keys discover schemas and read drafts/media; write keys also edit, publish, unpublish, and delete content/media. Keys cannot access admin pages, browser preview delivery, auth endpoints, user management, invitations, or form submissions. Browser sessions retain the same-origin write requirement. Supplying an invalid key never falls back to a cookie. Keys are limited to 120 requests per minute; exceeding the limit returns 429. Revoked, disabled, or expired keys return 401. Missing permissions and an owner who still needs to change their password return 403. Every successful password change, emailed password reset, and operator password/two-factor recovery revokes the account's keys. Password changes and emailed resets audit the number of keys revoked. Deleting a user cascades to their keys. There is no role-change API; current owner permissions are checked on every key request.
|
|
9
|
+
Use `x-api-key` on `/api/cms/` requests. No browser cookie or `Origin` header is needed. Read keys discover schemas and read drafts/media; write keys also edit, publish, unpublish, and delete content/media. Keys cannot access admin pages, browser preview delivery, auth endpoints, user management, invitations, or form submissions. Browser sessions retain the same-origin write requirement. Supplying an invalid key never falls back to a cookie. Keys are limited to 120 requests per minute; exceeding the limit returns 429. Revoked, disabled, or expired keys return 401. Missing permissions and an owner who still needs to change their password return 403. Every successful password change, emailed password reset, and operator password/two-factor recovery revokes the account's keys. Password changes and emailed resets audit the number of keys revoked. Deleting a user cascades to their keys. There is no role-change API; operators change roles with `sandcms user set-role`, and current owner permissions are checked on every key request.
|
|
10
|
+
|
|
11
|
+
### From the CLI
|
|
12
|
+
|
|
13
|
+
An operator, or an agent working in the site's project directory, can create a key without a browser session:
|
|
14
|
+
|
|
15
|
+
```sh
|
|
16
|
+
export SANDCMS_API_KEY=$(bun x --no-install sandcms key create \
|
|
17
|
+
--email editor@example.com --name "Migration agent" --scope write --days 7)
|
|
18
|
+
bun x --no-install sandcms key list
|
|
19
|
+
bun x --no-install sandcms key revoke --id KEY_ID
|
|
20
|
+
bun x --no-install sandcms key revoke --email editor@example.com # all of that user's keys
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
`key list --json` prints the list for scripts. `key create` prints only the secret on stdout; the key ID, owner, expiry and revoke command go to stderr. `--scope` is `read` or `write`, and `--days` (1–365) defaults to 7 for write keys and 30 for read keys. The owner must be an existing admin or editor who has already replaced their temporary password, since the API refuses keys of an owner with a temporary password. An editor-owned key is a good fit for an agent: it gets the same content and media access without admin rights. The command needs the site's `SANDCMS_SECRET` and `SANDCMS_ORIGIN` (read from `.env`, as the website does) and a migrated database, selected with `--database PATH` or `SANDCMS_DATABASE`. CLI creation and revocation are audited with the actor `operator`. Running the CLI requires write access to the database, which already allows resetting any password, so this grants nothing new; keep shell access to the site private.
|
|
10
24
|
|
|
11
25
|
Key management also has session-only endpoints: `POST /api/cms/keys/create` with `{ "name": "Migration agent", "scope": "write", "expiresIn": 604800 }`, `GET /api/cms/keys/list`, and `POST /api/cms/keys/delete` with `{ "id": "key-id" }`. Write requests require the site's `Origin`. Direct Better Auth key-management HTTP endpoints remain disabled; server integrations use the configured `cms.auth.api` methods.
|
|
12
26
|
|