@inixiative/archive 0.10.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +18 -6
- package/compose.yaml +5 -3
- package/package.json +2 -2
- package/src/cli.ts +10 -3
- package/src/config.ts +48 -22
- package/src/kingdom.ts +59 -4
- package/src/remote.ts +22 -0
- package/src/server.ts +66 -2
- package/src/store.ts +9 -0
package/README.md
CHANGED
|
@@ -21,7 +21,7 @@ archive up # creates ~/.local/share/archive/server.token, then docker compo
|
|
|
21
21
|
archive down
|
|
22
22
|
```
|
|
23
23
|
|
|
24
|
-
`up` runs the `compose.yaml` that ships in the package (project `archive`) with `ARCHIVE_DATA_DIR` set to the archive home and the server token from its `server.token`. Postgres data lives in `$ARCHIVE_DATA_DIR/postgres` and the server's destinations in `$ARCHIVE_DATA_DIR/config`, so both survive rebuilds and volume prunes. Pin a version with `ARCHIVE_VERSION`; build from a checkout with `docker compose -f compose.yaml -f compose.build.yaml up -d --build`.
|
|
24
|
+
`up` runs the `compose.yaml` that ships in the package (project `archive`) with `ARCHIVE_DATA_DIR` set to the archive home and the server token from its `server.token`. Postgres data lives in `$ARCHIVE_DATA_DIR/postgres` and the server's configuration (`destinations.json`, destination token files, the Kingdom pairing) in `$ARCHIVE_DATA_DIR/config`, so both survive rebuilds and volume prunes. Compose mounts the config directory at the same absolute path, so the CLI and the server read one `destinations.json` and every path in it resolves for both. Running Compose directly requires an absolute `ARCHIVE_DATA_DIR`. Pin a version with `ARCHIVE_VERSION`; build from a checkout with `docker compose -f compose.yaml -f compose.build.yaml up -d --build`.
|
|
25
25
|
|
|
26
26
|
The server applies its schema migrations on start and requires bearer authentication for data endpoints; `/health` has no session data. Every other command talks to it over HTTP at `--url` (default `http://127.0.0.1:4700`) with the token in `--token-file` (default `~/.local/share/archive/server.token`) or `ARCHIVE_TOKEN`:
|
|
27
27
|
|
|
@@ -44,11 +44,11 @@ Every deployment runs the published image `ghcr.io/inixiative/archive` (tags: ve
|
|
|
44
44
|
The local Archive publishes onward to destinations. Place the destination's token in a private file under the server's config directory (`chmod 600`, owned by you) or an environment variable, then run:
|
|
45
45
|
|
|
46
46
|
```sh
|
|
47
|
-
archive connect --
|
|
48
|
-
archive search --remote --
|
|
47
|
+
archive connect --url https://your-archive.example --project-id inixiative --token-file ~/.local/share/archive/config/credentials/inixiative.token
|
|
48
|
+
archive search --remote --query 'migration'
|
|
49
49
|
```
|
|
50
50
|
|
|
51
|
-
`setup` is an alias for `connect`. Setup verifies access before saving
|
|
51
|
+
`setup` is an alias for `connect`. Setup verifies access before saving `<home>/config/destinations.json` and does not upload; the Compose server (`serve --sync`) reads it and publishes every 30 seconds. `sync` and `routes` run the same publication, or preview it, once against `DATABASE_URL`. The URL locates the server; the token grants access. Configuration stores only the token file path (`tokenFile`) or environment variable name (`--token-env`, `tokenEnv`), never the token. A token file is read on every request, so rotating it needs no restart; symlinks and files readable by group or others are refused. Redirects are refused; only HTTPS or loopback HTTP is allowed.
|
|
52
52
|
|
|
53
53
|
## Always-on local archiving
|
|
54
54
|
|
|
@@ -76,14 +76,14 @@ First, pair this Archive with Kingdom:
|
|
|
76
76
|
bun run archive pair --kingdom https://api.your-kingdom.example --name "Work laptop"
|
|
77
77
|
```
|
|
78
78
|
|
|
79
|
-
`pair` registers this Archive with Kingdom as an **Installation**, named by a key kept in `<home>/kingdom/<kingdom host>/`. The same key is reused for every owner it registers with. It then asks to be registered as an integration and prints a review code with a link.
|
|
79
|
+
`pair` registers this Archive with Kingdom as an **Installation**, named by a key kept in `<home>/config/kingdom/<kingdom host>/`. The same key is reused for every owner it registers with. It then asks to be registered as an integration and prints a review code with a link.
|
|
80
80
|
|
|
81
81
|
Open the link, choose the owner and the hosted Archives this machine may write to, and approve. The command waits for approval, then shows which owner it was registered with and asks you to accept. Pass `--yes` to accept without the prompt; without a terminal, `--yes` is required. Only then does it collect the Signet (0600) and list the libraries it may write to. Confirming the owner protects you if someone else claimed your code into their own owner.
|
|
82
82
|
|
|
83
83
|
Next, route a project to one of those libraries:
|
|
84
84
|
|
|
85
85
|
```sh
|
|
86
|
-
bun run archive connect --kind kingdom --url https://api.your-kingdom.example --credential-file ~/.archive/kingdom/signet-SIGNET.json --integration-id HOSTED_ARCHIVE_INTEGRATION --resource-id LIBRARY --project-id inixiative
|
|
86
|
+
bun run archive connect --kind kingdom --url https://api.your-kingdom.example --credential-file ~/.local/share/archive/config/kingdom/KINGDOM_HOST/signet-SIGNET.json --integration-id HOSTED_ARCHIVE_INTEGRATION --resource-id LIBRARY --project-id inixiative
|
|
87
87
|
```
|
|
88
88
|
|
|
89
89
|
`connect` checks that the Signet grants `sessions.write` on that library before it saves anything. `serve --sync` then sends each changed session through `POST /api/v1/access/execute`. Every call carries the Signet's access token and a fresh DPoP proof (`@inixiative/signet` renews the token).
|
|
@@ -104,6 +104,17 @@ Foundry writes its captures to the local Archive server through `@inixiative/arc
|
|
|
104
104
|
|
|
105
105
|
Only explicit `projectId` matches authorize publication. An archive with no matching destination stays local. Multiple matches create deliberate separate copies. `routes` previews these destinations before upload. Tags, references and actors never change ownership or routing.
|
|
106
106
|
|
|
107
|
+
The server manages routes over HTTP too, so Foundry can read and set them. Only Kingdom destinations are set this way, through the Signets `pair` collected; direct token destinations stay CLI-only.
|
|
108
|
+
|
|
109
|
+
| Action | Body | Returns |
|
|
110
|
+
|---|---|---|
|
|
111
|
+
| `destinations/list` | `projectId?` | `destinations`: each as `connect` reports it, with `delivered` (archives acknowledged at their latest digest) and `pending` |
|
|
112
|
+
| `destinations/libraries` | none | `paired`, and the `libraries` (`integrationId`, `resourceId`, `name`) the held Signets may write to |
|
|
113
|
+
| `destinations/connect` | `projectId`, `integrationId`, `resourceId` | the saved destination, after Kingdom confirms `sessions.write` on the library (403 otherwise) |
|
|
114
|
+
| `destinations/remove` | `projectId`, `integrationId`, `resourceId` | `removed` |
|
|
115
|
+
|
|
116
|
+
`@inixiative/archive/remote`'s `ArchiveClient` wraps these as `destinations()`, `libraries()`, `connectDestination()` and `removeDestination()`. The sync loop picks up changes within 30 seconds.
|
|
117
|
+
|
|
107
118
|
## Tags, references, actors and retention
|
|
108
119
|
|
|
109
120
|
The archive owns its tag system. It keeps two different things apart:
|
|
@@ -133,6 +144,7 @@ All data endpoints are `POST /api/v1/archive/<action>` with a JSON body and `Aut
|
|
|
133
144
|
| `delete` | `archiveId` | deleted |
|
|
134
145
|
| `settings/read`, `settings/update` | `integrations?`, `retentionDays?` | settings |
|
|
135
146
|
| `tags/list`, `tags/define`, `tags/remove` | `tag`, `actorId?`, `description?` | offered tags with usage counts |
|
|
147
|
+
| `destinations/list`, `destinations/libraries`, `destinations/connect`, `destinations/remove` | see [Routing](#routing) | routes |
|
|
136
148
|
|
|
137
149
|
Filters are `projectId`, `source`, `tag`, `actorId`, `model`, `effort` and `reference` (`{integration, ref}`). Jev integration is not enabled. Session content must not be sent to Jev without selecting that service for the relevant ownership boundary.
|
|
138
150
|
|
package/compose.yaml
CHANGED
|
@@ -1,4 +1,6 @@
|
|
|
1
1
|
# Ports are the archive block of @inixiative/config ports: HTTP 4700, Postgres 6132.
|
|
2
|
+
# ARCHIVE_DATA_DIR is the absolute archive home (`archive up` sets it). Its config directory is
|
|
3
|
+
# mounted at the same path, so destinations.json and the files it names resolve for the CLI too.
|
|
2
4
|
services:
|
|
3
5
|
postgres:
|
|
4
6
|
image: pgvector/pgvector:pg17
|
|
@@ -9,7 +11,7 @@ services:
|
|
|
9
11
|
ports:
|
|
10
12
|
- "127.0.0.1:${ARCHIVE_POSTGRES_PORT:-6132}:5432"
|
|
11
13
|
volumes:
|
|
12
|
-
- ${ARCHIVE_DATA_DIR
|
|
14
|
+
- ${ARCHIVE_DATA_DIR:?Set ARCHIVE_DATA_DIR to the absolute archive home}/postgres:/var/lib/postgresql/data
|
|
13
15
|
restart: unless-stopped
|
|
14
16
|
healthcheck:
|
|
15
17
|
test: ["CMD", "pg_isready", "-U", "archive"]
|
|
@@ -18,7 +20,7 @@ services:
|
|
|
18
20
|
retries: 20
|
|
19
21
|
archive:
|
|
20
22
|
image: ghcr.io/inixiative/archive:${ARCHIVE_VERSION:-latest}
|
|
21
|
-
command: ["bun", "run", "src/cli.ts", "serve", "--home", "
|
|
23
|
+
command: ["bun", "run", "src/cli.ts", "serve", "--home", "${ARCHIVE_DATA_DIR}", "--hostname", "0.0.0.0", "--sync"]
|
|
22
24
|
depends_on:
|
|
23
25
|
postgres:
|
|
24
26
|
condition: service_healthy
|
|
@@ -28,7 +30,7 @@ services:
|
|
|
28
30
|
ARCHIVE_SERVER_TOKEN: ${ARCHIVE_SERVER_TOKEN:?Set a unique random token of at least 32 characters}
|
|
29
31
|
DATABASE_URL: postgresql://archive:${ARCHIVE_POSTGRES_PASSWORD:-archive}@postgres:5432/archive
|
|
30
32
|
volumes:
|
|
31
|
-
- ${ARCHIVE_DATA_DIR
|
|
33
|
+
- ${ARCHIVE_DATA_DIR}/config:${ARCHIVE_DATA_DIR}/config
|
|
32
34
|
restart: unless-stopped
|
|
33
35
|
healthcheck:
|
|
34
36
|
test: ["CMD", "bun", "-e", "fetch('http://127.0.0.1:4700/health').then(r=>{if(!r.ok)process.exit(1)})"]
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@inixiative/archive",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.11.0",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"exports": {
|
|
6
6
|
".": "./src/index.ts",
|
|
@@ -43,7 +43,7 @@
|
|
|
43
43
|
},
|
|
44
44
|
"devDependencies": {
|
|
45
45
|
"@biomejs/biome": "2.5.2",
|
|
46
|
-
"@inixiative/config": "^0.5.
|
|
46
|
+
"@inixiative/config": "^0.5.64",
|
|
47
47
|
"@types/bun": "1.4.2",
|
|
48
48
|
"lefthook": "2.1.9",
|
|
49
49
|
"typescript": "6.0.3"
|
package/src/cli.ts
CHANGED
|
@@ -18,7 +18,13 @@ import {
|
|
|
18
18
|
import { importChatGPTThread } from './chatgpt';
|
|
19
19
|
import { archiveRequest, routingPreview, searchRemotes, syncArchives } from './client';
|
|
20
20
|
import { collectSessions } from './collector';
|
|
21
|
-
import {
|
|
21
|
+
import {
|
|
22
|
+
archiveDestinationSchema,
|
|
23
|
+
connectDestination,
|
|
24
|
+
destinationsFile,
|
|
25
|
+
kingdomDirectory,
|
|
26
|
+
readDestinations,
|
|
27
|
+
} from './config';
|
|
22
28
|
import { importTranscriptFile } from './import-file';
|
|
23
29
|
import { pairWithKingdom, verifyKingdomDestination } from './kingdom';
|
|
24
30
|
import { previewImports } from './preview';
|
|
@@ -76,7 +82,7 @@ export async function runCli(args = Bun.argv.slice(2)) {
|
|
|
76
82
|
},
|
|
77
83
|
});
|
|
78
84
|
const home = resolve(v.home!),
|
|
79
|
-
config = v.config ??
|
|
85
|
+
config = v.config ?? destinationsFile(home),
|
|
80
86
|
tokenFile = v['token-file'] ? resolve(v['token-file']) : join(home, 'server.token');
|
|
81
87
|
const output = (value: unknown) => console.log(JSON.stringify(value, null, 2));
|
|
82
88
|
const command = positionals[0];
|
|
@@ -153,6 +159,7 @@ export async function runCli(args = Bun.argv.slice(2)) {
|
|
|
153
159
|
token: process.env.ARCHIVE_SERVER_TOKEN ?? readToken() ?? '',
|
|
154
160
|
port: Number(v.port ?? process.env.PORT ?? 4700),
|
|
155
161
|
hostname: v.hostname,
|
|
162
|
+
config: { destinationsFile: config, kingdomDirectory: kingdomDirectory(home) },
|
|
156
163
|
});
|
|
157
164
|
output({ listening: instance.server.url.href, sync: v.sync });
|
|
158
165
|
let stopped = false;
|
|
@@ -190,7 +197,7 @@ export async function runCli(args = Bun.argv.slice(2)) {
|
|
|
190
197
|
kingdom: v.kingdom,
|
|
191
198
|
name: v.name ?? `${hostname()} Archive`,
|
|
192
199
|
sourceId: await local.sourceId(),
|
|
193
|
-
directory:
|
|
200
|
+
directory: kingdomDirectory(home),
|
|
194
201
|
onReview: (review) => console.error(JSON.stringify(review)),
|
|
195
202
|
onError: (error) =>
|
|
196
203
|
console.error(`Still waiting: ${error instanceof Error ? error.message : String(error)}`),
|
package/src/config.ts
CHANGED
|
@@ -11,7 +11,7 @@ import {
|
|
|
11
11
|
renameSync,
|
|
12
12
|
writeFileSync,
|
|
13
13
|
} from 'node:fs';
|
|
14
|
-
import { dirname, isAbsolute } from 'node:path';
|
|
14
|
+
import { dirname, isAbsolute, join } from 'node:path';
|
|
15
15
|
import { z } from 'zod';
|
|
16
16
|
|
|
17
17
|
const projectId = z.string().min(1).max(256);
|
|
@@ -35,7 +35,7 @@ const archiveDestination = z
|
|
|
35
35
|
* A hosted Archive reached through Kingdom: this Archive presents the Signet it was paired with,
|
|
36
36
|
* granting sessions.write on that Archive's library.
|
|
37
37
|
*/
|
|
38
|
-
const
|
|
38
|
+
export const kingdomDestinationSchema = z.strictObject({
|
|
39
39
|
kind: z.literal('kingdom'),
|
|
40
40
|
projectId,
|
|
41
41
|
url: z.url(),
|
|
@@ -43,7 +43,7 @@ const kingdomDestination = z.strictObject({
|
|
|
43
43
|
integrationId: z.uuid(),
|
|
44
44
|
resourceId: z.uuid(),
|
|
45
45
|
});
|
|
46
|
-
export const archiveDestinationSchema = z.union([archiveDestination,
|
|
46
|
+
export const archiveDestinationSchema = z.union([archiveDestination, kingdomDestinationSchema]);
|
|
47
47
|
export type ArchiveDestination = z.infer<typeof archiveDestinationSchema>;
|
|
48
48
|
|
|
49
49
|
/** Private regular file owned by this user; a symlink or group/world access is refused. */
|
|
@@ -94,30 +94,56 @@ export function destinationIdentity(destination: ArchiveDestination) {
|
|
|
94
94
|
: ['kingdom', destination.integrationId, destination.resourceId],
|
|
95
95
|
]);
|
|
96
96
|
}
|
|
97
|
+
/**
|
|
98
|
+
* The server's configuration: destinations and the Kingdom pairing. Compose mounts it at the same
|
|
99
|
+
* absolute path, so the CLI and the server read one file and the paths in it resolve for both.
|
|
100
|
+
*/
|
|
101
|
+
export const configDirectory = (home: string) => join(home, 'config');
|
|
102
|
+
export const destinationsFile = (home: string) => join(configDirectory(home), 'destinations.json');
|
|
103
|
+
export const kingdomDirectory = (home: string) => join(configDirectory(home), 'kingdom');
|
|
104
|
+
|
|
105
|
+
/** A destination as reported: never a token, and no credential file for Kingdom entries. */
|
|
106
|
+
export const describeDestination = (destination: ArchiveDestination) => ({
|
|
107
|
+
configured: true as const,
|
|
108
|
+
kind: destination.kind,
|
|
109
|
+
projectId: destination.projectId,
|
|
110
|
+
url: destination.url,
|
|
111
|
+
...(destination.kind === 'kingdom'
|
|
112
|
+
? { integrationId: destination.integrationId, resourceId: destination.resourceId }
|
|
113
|
+
: destination.tokenFile
|
|
114
|
+
? { tokenFile: destination.tokenFile }
|
|
115
|
+
: { tokenEnv: destination.tokenEnv }),
|
|
116
|
+
});
|
|
117
|
+
export type DescribedDestination = ReturnType<typeof describeDestination>;
|
|
118
|
+
|
|
119
|
+
function writeDestinations(file: string, destinations: ArchiveDestination[]) {
|
|
120
|
+
mkdirSync(dirname(file), { recursive: true, mode: 0o700 });
|
|
121
|
+
const temp = `${file}.${randomUUID()}.tmp`;
|
|
122
|
+
writeFileSync(temp, `${JSON.stringify(destinations, null, 2)}\n`, { mode: 0o600, flag: 'wx' });
|
|
123
|
+
renameSync(temp, file);
|
|
124
|
+
chmodSync(file, 0o600);
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
const sameRoute = (a: ArchiveDestination, projectId: string, identity: string) =>
|
|
128
|
+
a.projectId === projectId && destinationIdentity(a) === identity;
|
|
129
|
+
|
|
97
130
|
export function connectDestination(file: string, input: unknown) {
|
|
98
131
|
const destination = archiveDestinationSchema.parse(input);
|
|
99
132
|
destination.url = destinationUrl(destination.url).href;
|
|
100
133
|
const destinations = readDestinations(file);
|
|
101
134
|
const identity = destinationIdentity(destination);
|
|
102
|
-
const index = destinations.findIndex(
|
|
103
|
-
(item) => item.projectId === destination.projectId && destinationIdentity(item) === identity,
|
|
104
|
-
);
|
|
135
|
+
const index = destinations.findIndex((item) => sameRoute(item, destination.projectId, identity));
|
|
105
136
|
if (index < 0) destinations.push(destination);
|
|
106
137
|
else destinations[index] = destination;
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
? { integrationId: destination.integrationId, resourceId: destination.resourceId }
|
|
119
|
-
: destination.tokenFile
|
|
120
|
-
? { tokenFile: destination.tokenFile }
|
|
121
|
-
: { tokenEnv: destination.tokenEnv }),
|
|
122
|
-
};
|
|
138
|
+
writeDestinations(file, destinations);
|
|
139
|
+
return describeDestination(destination);
|
|
140
|
+
}
|
|
141
|
+
|
|
142
|
+
/** Stops routing a project to a destination (by `destinationIdentity`); its receipts remain. */
|
|
143
|
+
export function removeDestination(file: string, route: { projectId: string; identity: string }) {
|
|
144
|
+
const destinations = readDestinations(file);
|
|
145
|
+
const kept = destinations.filter((item) => !sameRoute(item, route.projectId, route.identity));
|
|
146
|
+
if (kept.length === destinations.length) return false;
|
|
147
|
+
writeDestinations(file, kept);
|
|
148
|
+
return true;
|
|
123
149
|
}
|
package/src/kingdom.ts
CHANGED
|
@@ -1,6 +1,18 @@
|
|
|
1
|
+
import { existsSync, readdirSync } from 'node:fs';
|
|
2
|
+
import { join } from 'node:path';
|
|
1
3
|
import { pairInstallation, SignetClient } from '@inixiative/signet';
|
|
2
4
|
import type { ArchiveDestination } from './config';
|
|
3
5
|
|
|
6
|
+
type KingdomDestination = Extract<ArchiveDestination, { kind: 'kingdom' }>;
|
|
7
|
+
type Library = { integrationId: string; resourceId: string; name: string };
|
|
8
|
+
|
|
9
|
+
/** The Signet does not grant sessions.write on the library. */
|
|
10
|
+
export class KingdomGrantMissing extends Error {
|
|
11
|
+
constructor() {
|
|
12
|
+
super("This Archive's Signet does not grant sessions.write on that library");
|
|
13
|
+
}
|
|
14
|
+
}
|
|
15
|
+
|
|
4
16
|
/** Libraries this Archive's Signet may write to, as Kingdom describes them. */
|
|
5
17
|
export async function writableLibraries(credentialFile: string) {
|
|
6
18
|
const description = await (await SignetClient.fromFile(credentialFile)).describe();
|
|
@@ -60,9 +72,7 @@ export async function pairWithKingdom(input: Pairing) {
|
|
|
60
72
|
};
|
|
61
73
|
}
|
|
62
74
|
|
|
63
|
-
export async function verifyKingdomDestination(
|
|
64
|
-
destination: Extract<ArchiveDestination, { kind: 'kingdom' }>,
|
|
65
|
-
) {
|
|
75
|
+
export async function verifyKingdomDestination(destination: KingdomDestination) {
|
|
66
76
|
const libraries = await writableLibraries(destination.credentialFile);
|
|
67
77
|
if (
|
|
68
78
|
!libraries.some(
|
|
@@ -71,5 +81,50 @@ export async function verifyKingdomDestination(
|
|
|
71
81
|
library.resourceId === destination.resourceId,
|
|
72
82
|
)
|
|
73
83
|
)
|
|
74
|
-
throw new
|
|
84
|
+
throw new KingdomGrantMissing();
|
|
85
|
+
}
|
|
86
|
+
|
|
87
|
+
/** Credential files of the Signets `pair` collected into the directory, one per owner. */
|
|
88
|
+
export function heldSignets(directory: string) {
|
|
89
|
+
if (!existsSync(directory)) return [];
|
|
90
|
+
return readdirSync(directory, { withFileTypes: true })
|
|
91
|
+
.filter((entry) => entry.isDirectory())
|
|
92
|
+
.flatMap((kingdom) =>
|
|
93
|
+
readdirSync(join(directory, kingdom.name))
|
|
94
|
+
.filter((name) => /^signet-[^/]+\.json$/.test(name))
|
|
95
|
+
.map((name) => join(directory, kingdom.name, name)),
|
|
96
|
+
)
|
|
97
|
+
.sort();
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
/** Every library the held Signets may write to; `paired` is false until `pair` collected one. */
|
|
101
|
+
export async function pairedLibraries(directory: string) {
|
|
102
|
+
const credentialFiles = heldSignets(directory);
|
|
103
|
+
const libraries = new Map<string, Library>();
|
|
104
|
+
for (const credentialFile of credentialFiles)
|
|
105
|
+
for (const library of await writableLibraries(credentialFile))
|
|
106
|
+
libraries.set(JSON.stringify([library.integrationId, library.resourceId]), library);
|
|
107
|
+
return { paired: credentialFiles.length > 0, libraries: [...libraries.values()] };
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
/** The Kingdom destination for a library, through whichever held Signet grants writing to it. */
|
|
111
|
+
export async function pairedDestination(
|
|
112
|
+
directory: string,
|
|
113
|
+
route: { projectId: string; integrationId: string; resourceId: string },
|
|
114
|
+
): Promise<KingdomDestination> {
|
|
115
|
+
for (const credentialFile of heldSignets(directory)) {
|
|
116
|
+
const destination: KingdomDestination = {
|
|
117
|
+
kind: 'kingdom',
|
|
118
|
+
...route,
|
|
119
|
+
url: (await SignetClient.fromFile(credentialFile)).url,
|
|
120
|
+
credentialFile,
|
|
121
|
+
};
|
|
122
|
+
try {
|
|
123
|
+
await verifyKingdomDestination(destination);
|
|
124
|
+
return destination;
|
|
125
|
+
} catch (error) {
|
|
126
|
+
if (!(error instanceof KingdomGrantMissing)) throw error;
|
|
127
|
+
}
|
|
128
|
+
}
|
|
129
|
+
throw new KingdomGrantMissing();
|
|
75
130
|
}
|
package/src/remote.ts
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
import { existsSync, readFileSync } from 'node:fs';
|
|
2
2
|
import { join } from 'node:path';
|
|
3
3
|
import { defaultHome } from './agents';
|
|
4
|
+
import type { DescribedDestination } from './config';
|
|
4
5
|
import type { ArchiveChunk, ArchiveSnapshot } from './index';
|
|
5
6
|
import { archiveKey, archiveSnapshotSchema } from './index';
|
|
6
7
|
import type { ArchiveFilter, ArchiveSettings, TagDefinition } from './schemas';
|
|
@@ -16,6 +17,14 @@ export type ArchiveListing = Omit<ArchiveSnapshot, 'entries'> & {
|
|
|
16
17
|
references: ArchiveReferenceCount[];
|
|
17
18
|
};
|
|
18
19
|
|
|
20
|
+
/** A configured destination and how many of its project's archives it holds at their latest. */
|
|
21
|
+
export type DestinationStatus = DescribedDestination & { delivered: number; pending: number };
|
|
22
|
+
/** A hosted Archive library a held Signet may write to. */
|
|
23
|
+
export type ArchiveLibrary = { integrationId: string; resourceId: string; name: string };
|
|
24
|
+
export type ArchiveLibraries = { paired: boolean; libraries: ArchiveLibrary[] };
|
|
25
|
+
/** A project routed to a hosted Archive library through Kingdom. */
|
|
26
|
+
export type KingdomRoute = { projectId: string; integrationId: string; resourceId: string };
|
|
27
|
+
|
|
19
28
|
export class ArchiveRequestError extends Error {
|
|
20
29
|
constructor(
|
|
21
30
|
readonly status: number,
|
|
@@ -147,6 +156,19 @@ export class ArchiveClient {
|
|
|
147
156
|
actorId ? { actorId } : {},
|
|
148
157
|
);
|
|
149
158
|
}
|
|
159
|
+
destinations(filter: { projectId?: string } = {}) {
|
|
160
|
+
return this.request<{ destinations: DestinationStatus[] }>('destinations/list', filter);
|
|
161
|
+
}
|
|
162
|
+
libraries() {
|
|
163
|
+
return this.request<ArchiveLibraries>('destinations/libraries');
|
|
164
|
+
}
|
|
165
|
+
/** Routes a project to a library; the server verifies its Signet grants sessions.write first. */
|
|
166
|
+
connectDestination(route: KingdomRoute) {
|
|
167
|
+
return this.request<DescribedDestination>('destinations/connect', route);
|
|
168
|
+
}
|
|
169
|
+
removeDestination(route: KingdomRoute) {
|
|
170
|
+
return this.request<{ removed: boolean }>('destinations/remove', route);
|
|
171
|
+
}
|
|
150
172
|
}
|
|
151
173
|
|
|
152
174
|
/** The machine's local Archive server (the `archive` block in @inixiative/config ports). */
|
package/src/server.ts
CHANGED
|
@@ -1,6 +1,15 @@
|
|
|
1
1
|
import { timingSafeEqual } from 'node:crypto';
|
|
2
2
|
import { z } from 'zod';
|
|
3
|
+
import {
|
|
4
|
+
connectDestination,
|
|
5
|
+
describeDestination,
|
|
6
|
+
destinationIdentity,
|
|
7
|
+
kingdomDestinationSchema,
|
|
8
|
+
readDestinations,
|
|
9
|
+
removeDestination,
|
|
10
|
+
} from './config';
|
|
3
11
|
import { archiveReferenceSchema, archiveSnapshotSchema, selectChunks } from './index';
|
|
12
|
+
import { KingdomGrantMissing, pairedDestination, pairedLibraries } from './kingdom';
|
|
4
13
|
import { ArchiveConflict, ArchiveStore, archiveSettingsSchema, tagDefinitionSchema } from './store';
|
|
5
14
|
|
|
6
15
|
const ingestSchema = z.strictObject({
|
|
@@ -36,9 +45,22 @@ const tagSchema = z.strictObject({
|
|
|
36
45
|
});
|
|
37
46
|
const tagListSchema = z.strictObject({ actorId: z.string().min(1).max(256).optional() });
|
|
38
47
|
const tagRemoveSchema = tagDefinitionSchema.pick({ tag: true, actorId: true });
|
|
48
|
+
const destinationListSchema = z.strictObject({ projectId: filterFields.projectId });
|
|
49
|
+
const kingdomRouteSchema = kingdomDestinationSchema.pick({
|
|
50
|
+
projectId: true,
|
|
51
|
+
integrationId: true,
|
|
52
|
+
resourceId: true,
|
|
53
|
+
});
|
|
54
|
+
|
|
55
|
+
/** Where the server finds its destinations and the Signets `pair` collected. */
|
|
56
|
+
export type ArchiveServerConfig = { destinationsFile: string; kingdomDirectory: string };
|
|
39
57
|
|
|
40
58
|
/** One deployment, one ownership boundary. No shared cross-tenant database or admin API. */
|
|
41
|
-
export function createArchiveHandler(
|
|
59
|
+
export function createArchiveHandler(
|
|
60
|
+
store: ArchiveStore,
|
|
61
|
+
token: string,
|
|
62
|
+
config?: ArchiveServerConfig,
|
|
63
|
+
) {
|
|
42
64
|
if (token.length < 32)
|
|
43
65
|
throw new Error('ARCHIVE_SERVER_TOKEN must contain at least 32 characters');
|
|
44
66
|
const expected = Buffer.from(`Bearer ${token}`);
|
|
@@ -162,9 +184,50 @@ export function createArchiveHandler(store: ArchiveStore, token: string) {
|
|
|
162
184
|
return json({ data: await store.defineTag(tagDefinitionSchema.parse(body)) });
|
|
163
185
|
if (path === '/api/v1/archive/tags/remove')
|
|
164
186
|
return json({ data: { removed: await store.undefineTag(tagRemoveSchema.parse(body)) } });
|
|
187
|
+
if (config && path === '/api/v1/archive/destinations/list') {
|
|
188
|
+
const { projectId } = destinationListSchema.parse(body);
|
|
189
|
+
const destinations = [];
|
|
190
|
+
for (const destination of readDestinations(config.destinationsFile))
|
|
191
|
+
if (!projectId || destination.projectId === projectId)
|
|
192
|
+
destinations.push({
|
|
193
|
+
...describeDestination(destination),
|
|
194
|
+
...(await store.delivery(destination.projectId, destinationIdentity(destination))),
|
|
195
|
+
});
|
|
196
|
+
return json({ data: { destinations } });
|
|
197
|
+
}
|
|
198
|
+
if (config && path === '/api/v1/archive/destinations/libraries') {
|
|
199
|
+
z.strictObject({}).parse(body);
|
|
200
|
+
return json({ data: await pairedLibraries(config.kingdomDirectory) });
|
|
201
|
+
}
|
|
202
|
+
if (config && path === '/api/v1/archive/destinations/connect') {
|
|
203
|
+
const route = kingdomRouteSchema.parse(body);
|
|
204
|
+
const destination = await pairedDestination(config.kingdomDirectory, route);
|
|
205
|
+
return json({ data: connectDestination(config.destinationsFile, destination) });
|
|
206
|
+
}
|
|
207
|
+
if (config && path === '/api/v1/archive/destinations/remove') {
|
|
208
|
+
const route = kingdomRouteSchema.parse(body);
|
|
209
|
+
const configured = readDestinations(config.destinationsFile).find(
|
|
210
|
+
(d) =>
|
|
211
|
+
d.kind === 'kingdom' &&
|
|
212
|
+
d.projectId === route.projectId &&
|
|
213
|
+
d.integrationId === route.integrationId &&
|
|
214
|
+
d.resourceId === route.resourceId,
|
|
215
|
+
);
|
|
216
|
+
return json({
|
|
217
|
+
data: {
|
|
218
|
+
removed: configured
|
|
219
|
+
? removeDestination(config.destinationsFile, {
|
|
220
|
+
projectId: route.projectId,
|
|
221
|
+
identity: destinationIdentity(configured),
|
|
222
|
+
})
|
|
223
|
+
: false,
|
|
224
|
+
},
|
|
225
|
+
});
|
|
226
|
+
}
|
|
165
227
|
return json({ error: 'Not found' }, 404);
|
|
166
228
|
} catch (error) {
|
|
167
229
|
if (error instanceof ArchiveConflict) return json({ error: error.message }, 409);
|
|
230
|
+
if (error instanceof KingdomGrantMissing) return json({ error: error.message }, 403);
|
|
168
231
|
if (error instanceof z.ZodError || error instanceof SyntaxError)
|
|
169
232
|
return json({ error: 'Invalid archive request' }, 400);
|
|
170
233
|
return json({ error: 'Archive operation failed' }, 500);
|
|
@@ -177,6 +240,7 @@ export async function startArchiveServer(options: {
|
|
|
177
240
|
token: string;
|
|
178
241
|
port?: number;
|
|
179
242
|
hostname?: string;
|
|
243
|
+
config?: ArchiveServerConfig;
|
|
180
244
|
}) {
|
|
181
245
|
const store = new ArchiveStore(options.databaseUrl);
|
|
182
246
|
try {
|
|
@@ -188,7 +252,7 @@ export async function startArchiveServer(options: {
|
|
|
188
252
|
port: options.port ?? 4700,
|
|
189
253
|
hostname: options.hostname ?? '127.0.0.1',
|
|
190
254
|
maxRequestBodySize: 65_000_000,
|
|
191
|
-
fetch: createArchiveHandler(store, options.token),
|
|
255
|
+
fetch: createArchiveHandler(store, options.token, options.config),
|
|
192
256
|
});
|
|
193
257
|
return {
|
|
194
258
|
server,
|
package/src/store.ts
CHANGED
|
@@ -429,6 +429,15 @@ export class ArchiveStore {
|
|
|
429
429
|
this.db.outbox.deleteMany({ where: { archiveId: id, destination } }),
|
|
430
430
|
]);
|
|
431
431
|
}
|
|
432
|
+
/** A project's archives at one destination: delivered at their latest digest, or pending. */
|
|
433
|
+
async delivery(projectId: string, destination: string) {
|
|
434
|
+
const archives = await this.db.archive.findMany({
|
|
435
|
+
where: { projectId },
|
|
436
|
+
select: { digest: true, receipts: { where: { destination }, select: { digest: true } } },
|
|
437
|
+
});
|
|
438
|
+
const delivered = archives.filter((a) => a.receipts[0]?.digest === a.digest).length;
|
|
439
|
+
return { delivered, pending: archives.length - delivered };
|
|
440
|
+
}
|
|
432
441
|
async close() {
|
|
433
442
|
await this.db.$disconnect();
|
|
434
443
|
}
|