@undefineds.co/xpod 0.1.7 → 0.2.0

package/README.md

@@ -12,13 +12,31 @@ This is achieved through **Pods** (Personal Online Data stores)—portable web c
 
 ## Installation
 
-Ensure you have [Node.js](https://nodejs.org/) and [Yarn](https://yarnpkg.com/) installed:
+Ensure you have [Node.js 22.x LTS](https://nodejs.org/) and [Yarn](https://yarnpkg.com/) installed:
 
 ```bash
 yarn install
 yarn build
 ```
 
+> **Node runtime note**
+> Xpod currently targets Node 22 (`>=22 <23`). `.nvmrc` only declares the version; it does not force non-interactive runners to switch automatically.
+> AI/Codex runtimes, IDE background tasks, CI shells, or tmux panes may still start with the wrong `node` on `PATH`.
+>
+> Before running installs, local startup, or integration tests, run:
+>
+> ```bash
+> nvm use
+> yarn check:abi
+> ```
+>
+> If you hit `better-sqlite3` / `NODE_MODULE_VERSION` errors, reinstall native dependencies under the same Node major:
+>
+> ```bash
+> nvm use
+> yarn install --force --ignore-engines
+> ```
+
 
 ## Single-File Build & Local npm Release
 
@@ -66,6 +84,102 @@ Visit [http://localhost:3000/](http://localhost:3000/) after startup.
 
 See [docs/deployment-modes.md](docs/deployment-modes.md) for detailed profile comparison and cloud-edge coordination.
 
+## Library Mode for Tests
+
+If you want to start the full Xpod stack inside your test process, you can import it as a library instead of spawning the CLI.
+
+```ts
+import { startXpodRuntime } from '@undefineds.co/xpod/runtime';
+
+const runtime = await startXpodRuntime({
+  mode: 'local',
+  open: true,
+  transport: 'socket',
+});
+
+const res = await runtime.fetch('/service/status');
+console.log(await res.json());
+
+await runtime.stop();
+```
+
+For the lightest test setup, use the test helper export:
+
+```ts
+import { startNoAuthXpod } from '@undefineds.co/xpod/test-utils';
+
+const xpod = await startNoAuthXpod();
+console.log(xpod.baseUrl);
+await xpod.stop();
+```
+
+Notes:
+- In socket mode, the logical base URL is `http://localhost/`, but real traffic goes through a Unix socket.
+- This mode is ideal for CI and integration tests that should avoid external port conflicts.
+- Docker/full integration tests should still use real service startup.
+
+### Using Xpod from a downstream project
+
+Install Xpod as a dev dependency in your test project:
+
+```bash
+yarn add -D @undefineds.co/xpod
+```
+
+Important for downstream CI / agent runners:
+- Run the parent toolchain with Node 22 as well. If Codex, IDE tasks, or other AI runtimes are launched under Node 23, they may ignore your directory-level `nvm` hook and still load the wrong native ABI.
+- The package ships `yarn check:abi`; use it before startup when debugging local environment issues.
+
+Recommended entry points:
+- `@undefineds.co/xpod` — CSS/components main entry
+- `@undefineds.co/xpod/runtime` — in-process runtime API (`startXpodRuntime`)
+- `@undefineds.co/xpod/test-utils` — lightest no-auth helper (`startNoAuthXpod`)
+
+A minimal `vitest` example in a downstream repo:
+
+```ts
+import { afterAll, beforeAll, describe, expect, it } from 'vitest';
+import { startNoAuthXpod } from '@undefineds.co/xpod/test-utils';
+
+let xpod: Awaited<ReturnType<typeof startNoAuthXpod>>;
+
+afterAll(async() => {
+  await xpod?.stop();
+});
+
+beforeAll(async() => {
+  xpod = await startNoAuthXpod();
+}, 60_000);
+
+describe('xpod integration', () => {
+  it('starts in-process', async() => {
+    const response = await fetch(new URL('/service/status', xpod.baseUrl));
+    expect(response.ok).toBe(true);
+  });
+});
+```
+
+If you need more control in a downstream repo, use `startXpodRuntime` directly:
+
+```ts
+import { startXpodRuntime } from '@undefineds.co/xpod/runtime';
+
+const runtime = await startXpodRuntime({
+  mode: 'local',
+  open: true,
+  transport: 'socket',
+  runtimeRoot: './.test-data/xpod-runtime',
+});
+
+// use runtime.baseUrl / runtime.fetch / runtime.sockets here
+await runtime.stop();
+```
+
+Typical downstream use cases:
+- Use `startNoAuthXpod()` for CI-friendly integration tests that only need an open local stack.
+- Use `startXpodRuntime()` when you need auth mode, custom storage paths, or explicit runtime options.
+- Keep Docker/cluster tests on real services; use library mode for local/lite test paths.
+
 ## Single-File Packaging
 
 Build a self-extracting single-file launcher:
@@ -132,6 +246,45 @@ Xpod introduces a robust distributed architecture on top of standard Solid proto
 - **Control Plane (The Cloud)**: The Xpod Server acts as a coordinator, providing DNS, TLS certificates, and Tunneling services. It ensures your Agent is discoverable and reachable from the global internet.
 - **Data Plane (The Edge)**: Your Local Xpod holds the actual data and runs the Agents. It connects to the Cloud for reachability but executes Skills locally, ensuring data never leaves your physical control.
 
+### 7. Dual-Process Architecture (CSS + API Server)
+
+Xpod runs as two cooperating processes behind a single gateway:
+
+| Process | Port | Responsibility |
+|---------|------|----------------|
+| **CSS** (Solid Server) | 3000 | LDP resource access, OIDC auth, SPARQL, WebSocket notifications |
+| **API Server** | 3001 | Node management, heartbeat/DNS sync, quota, API Keys, AI chat |
+
+- **CSS** handles everything related to the Solid protocol — it's the kernel of the Pod file system.
+- **API Server** handles management APIs that don't belong in the Solid request chain.
+- Both share the same PostgreSQL database (server mode) or SQLite (local mode).
+
+**Auth methods**: Solid Token (DPoP/Bearer), API Key (`sk-*`), Node Token (`XpodNode nodeId:token`).
+
+**API Server routes**:
+
+| Route | Description | Mode |
+|-------|-------------|------|
+| `POST /v1/signal` | Edge node heartbeat → health check → DNS sync | Shared |
+| `/v1/nodes/*` | Node CRUD | Shared |
+| `/v1/keys/*` | API Key management | Shared |
+| `/v1/chat/completions` | OpenAI-compatible chat (SSE streaming) | Shared |
+| `/v1/responses` | OpenAI Responses API | Shared |
+| `/v1/messages` | Anthropic/OpenAI Threads compatible | Shared |
+| `/v1/models` | List available models | Shared |
+| `POST /v1/chatkit` | ChatKit protocol endpoint (streaming) | Shared |
+| `/v1/chatkit/threads/*` | ChatKit REST API (threads CRUD, items) | Shared |
+| `/v1/subdomains/*` | Subdomain allocation | Cloud |
+| `/v1/ddns/*` | Dynamic DNS management | Cloud |
+| `/v1/webid-profile/*` | WebID Profile hosting | Cloud |
+| `/provision/nodes` | SP registration | Cloud |
+| `/provision/pods` | Pod creation (SP callback) | Local |
+| `/admin/*` | Local config & restart | Local |
+
+Key components:
+- `EdgeNodeSignalHandler` (API Server) — receives heartbeats, triggers health checks and DNS synchronization
+- `EdgeNodeSignalClient` (local/edge) — sends periodic heartbeats with system metrics, network info, and tunnel status
+
 ## Privacy & Compliance (GDPR)
 
 Xpod is designed for the post-GDPR world, where **Data Sovereignty** is paramount.
@@ -149,18 +302,26 @@ Xpod is designed for the post-GDPR world, where **Data Sovereignty** is paramoun
 - [x] Sidecar API: SPARQL (`/-/sparql`)
 - [x] Sidecar API: Vector (`/-/vector`)
 - [x] Cloud-Edge Cluster Architecture
-- [ ] Sidecar API: Chat Completions (`/-/chat/completions`)
-- [ ] Sidecar API: Responses (`/-/responses`)
+- [x] API Server (dual-process architecture, separated from CSS)
+- [x] Multi-Auth: Solid Token / API Key / Node Token
+- [x] Edge Node Heartbeat + Health Check + DNS Sync
+- [x] SP Provision & Pod Management
+- [x] AI API: Chat Completions, Responses, Messages, Models (OpenAI-compatible, SSE streaming)
+- [x] ChatKit API (conversation management)
+- [x] Subdomain & DDNS Management
 - [ ] Sidecar API: Terminal (`/-/terminal`)
+- [ ] Rate Limiting
 - [ ] Attribute-Based Access Control (ABAC)
 - [ ] Feature Store (Federated Learning)
 
 ## Documentation
 
 - [docs/COMPONENTS.md](docs/COMPONENTS.md) - Components.js reference and configuration patterns
+- [docs/api-service-design.md](docs/api-service-design.md) - API Server architecture, routes, and auth design
 - [docs/deployment-modes.md](docs/deployment-modes.md) - Deployment profiles (local, cloud)
 - [docs/admin-guide.md](docs/admin-guide.md) - Admin initialization, roles, and reserved names
 - [docs/edge-cluster-architecture.md](docs/edge-cluster-architecture.md) - Cloud-edge coordination and routing
+- [docs/edge-node-deployment-modes.md](docs/edge-node-deployment-modes.md) - Edge node deployment scenarios
 - [docs/sidecar-api.md](docs/sidecar-api.md) - Sidecar API pattern (`/-/{service}`)
 - [docs/sparql-support.md](docs/sparql-support.md) - SPARQL 1.1 support details
 - [docs/vector-api-server-guide.md](docs/vector-api-server-guide.md) - Vector search API guide

package/config/cli.json

@@ -10,7 +10,7 @@
       "@type": "YargsCliExtractor",
       "parameters": [
         {
-          "comment": "=== Visible CLI parameters ===",
+          "comment": "=== Visible CLI parameters (核心模式开关) ===",
           "@type": "YargsParameter",
           "name": "edition",
           "options": {
@@ -20,18 +20,19 @@
         },
         {
           "@type": "YargsParameter",
-          "name": "edgeNodesEnabled",
+          "name": "authMode",
           "options": {
-            "type": "boolean",
-            "describe": "Enable edge node registry and routing."
+            "type": "string",
+            "describe": "Authorization mode: acp, acl, or allow-all",
+            "hidden": true
           }
         },
         {
           "@type": "YargsParameter",
-          "name": "edgeAgentEnabled",
+          "name": "edgeNodesEnabled",
           "options": {
             "type": "boolean",
-            "describe": "Enable local edge agent functionality."
+            "describe": "Enable edge node registry and routing."
           }
         },
         {
@@ -51,14 +52,6 @@
             "hidden": true
           }
         },
-        {
-          "@type": "YargsParameter",
-          "name": "identityTablePrefix",
-          "options": {
-            "type": "string",
-            "hidden": true
-          }
-        },
         {
           "@type": "YargsParameter",
           "name": "usageDbUrl",
@@ -99,22 +92,6 @@
             "hidden": true
           }
         },
-        {
-          "@type": "YargsParameter",
-          "name": "cachePath",
-          "options": {
-            "type": "string",
-            "hidden": true
-          }
-        },
-        {
-          "@type": "YargsParameter",
-          "name": "cacheMaxSize",
-          "options": {
-            "type": "string",
-            "hidden": true
-          }
-        },
         {
           "@type": "YargsParameter",
           "name": "redisClient",
@@ -173,14 +150,6 @@
         },
         {
           "comment": "=== Hidden: Business Features ===",
-          "@type": "YargsParameter",
-          "name": "signalEndpoint",
-          "options": {
-            "type": "string",
-            "hidden": true
-          }
-        },
-        {
           "@type": "YargsParameter",
           "name": "clusterIngressDomain",
           "options": {
@@ -196,14 +165,6 @@
             "hidden": true
           }
         },
-        {
-          "@type": "YargsParameter",
-          "name": "dnsRootDomain",
-          "options": {
-            "type": "string",
-            "hidden": true
-          }
-        },
         {
           "@type": "YargsParameter",
           "name": "dnsRecordTtl",
@@ -244,14 +205,6 @@
             "hidden": true
           }
         },
-        {
-          "@type": "YargsParameter",
-          "name": "tunnelEntrypoints",
-          "options": {
-            "type": "string",
-            "hidden": true
-          }
-        },
         {
           "@type": "YargsParameter",
           "name": "edgeHealthProbesEnabled",
@@ -308,22 +261,6 @@
             "hidden": true
           }
         },
-        {
-          "@type": "YargsParameter",
-          "name": "nodeToken",
-          "options": {
-            "type": "string",
-            "hidden": true
-          }
-        },
-        {
-          "@type": "YargsParameter",
-          "name": "nodePublicAddress",
-          "options": {
-            "type": "string",
-            "hidden": true
-          }
-        },
         {
           "@type": "YargsParameter",
           "name": "nodeHeartbeatInterval",
@@ -344,9 +281,10 @@
         {
           "comment": "=== Hidden: IdP/SP Separation ===",
           "@type": "YargsParameter",
-          "name": "baseDomain",
+          "name": "baseStorageDomain",
           "options": {
             "type": "string",
+            "describe": "Root domain for edge/SP node DNS records (e.g. undefineds.site)",
             "hidden": true
           }
         },

package/config/cloud.json

@@ -18,7 +18,7 @@
       "overrideParameters": {
         "@type": "WrappedExpiringReadWriteLocker",
         "locker": {
-          "@type": "RedisLocker",
+          "@type": "UrlAwareRedisLocker",
           "redisClient": {
             "@id": "urn:solid-server:default:variable:redisClient",
             "@type": "Variable"
@@ -83,12 +83,11 @@
       ]
     },
     {
-      "comment": "Default quota limits (in-memory overrides)",
+      "comment": "Quota service backed by identity DB, falls back to env defaults then unlimited",
       "@id": "urn:undefineds:xpod:QuotaService",
-      "@type": "DefaultQuotaService",
-      "DefaultQuotaService:_options_defaultAccountQuotaBytes": {
-        "@type": "Literal",
-        "value": 10737418240
+      "@type": "DrizzleQuotaService",
+      "DrizzleQuotaService:_options_identityDbUrl": {
+        "@id": "urn:solid-server:default:variable:identityDbUrl"
       }
     },
     {
@@ -126,7 +125,8 @@
       },
       "unstructuredDataAccessor": {
         "@id": "urn:undefineds:xpod:RemoteDataAccessor"
-      }
+      },
+      "presignedRedirectEnabled": true
     },
     {
       "comment": "PgQuintStore instance for QuintStoreSparqlDataAccessor (PostgreSQL backend).",
@@ -511,6 +511,33 @@
           }
         ]
       }
+    },
+    {
+      "comment": "Override PodCreator with ProvisionPodCreator to support remote SP pod creation via provisionCode",
+      "@type": "Override",
+      "overrideInstance": {
+        "@id": "urn:solid-server:default:PodCreator"
+      },
+      "overrideParameters": {
+        "@type": "ProvisionPodCreator",
+        "baseUrl": { "@id": "urn:solid-server:default:variable:baseUrl" },
+        "identifierGenerator": { "@id": "urn:solid-server:default:IdentifierGenerator" },
+        "relativeWebIdPath": "/profile/card#me",
+        "webIdStore": { "@id": "urn:solid-server:default:WebIdStore" },
+        "podStore": { "@id": "urn:solid-server:default:PodStore" }
+      }
+    },
+    {
+      "comment": "Override create-pod template to pass provisionCode from sessionStorage into POST settings",
+      "@type": "Override",
+      "overrideInstance": {
+        "@id": "urn:solid-server:default:CreatePodHtml"
+      },
+      "overrideParameters": {
+        "@type": "HtmlViewEntry",
+        "filePath": "./templates/identity/account/create-pod.html.ejs",
+        "route": { "@id": "urn:solid-server:default:AccountPodRoute" }
+      }
     }
   ]
 }

package/config/local.json

@@ -12,9 +12,13 @@
   ],
   "@graph": [
     {
-      "comment": "Disable quota for local profile",
+      "comment": "Quota service backed by identity DB, no env defaults = unlimited",
       "@id": "urn:undefineds:xpod:QuotaService",
-      "@type": "NoopQuotaService"
+      "@type": "DrizzleQuotaService",
+      "DrizzleQuotaService:_options_identityDbUrl": {
+        "@id": "urn:solid-server:default:variable:identityDbUrl",
+        "@type": "Variable"
+      }
     },
     {
       "comment": "定义 idpUrl 变量（外部 IdP 基础 URL），设置后启用 SP 模式",

package/config/resolver.json

@@ -10,7 +10,7 @@
       "@type": "CombinedShorthandResolver",
       "resolvers": [
         {
-          "comment": "=== CLI Parameters (KeyExtractor, from Yargs args dict) ===",
+          "comment": "=== CLI Parameters (核心模式开关) ===",
           "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:edition",
           "CombinedShorthandResolver:_resolvers_value": {
             "@type": "KeyExtractor",
@@ -19,18 +19,18 @@
           }
         },
         {
-          "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:edgeNodesEnabled",
+          "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:authMode",
           "CombinedShorthandResolver:_resolvers_value": {
             "@type": "KeyExtractor",
-            "key": "edgeNodesEnabled",
-            "defaultValue": "false"
+            "key": "authMode",
+            "defaultValue": "acp"
           }
         },
         {
-          "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:edgeAgentEnabled",
+          "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:edgeNodesEnabled",
           "CombinedShorthandResolver:_resolvers_value": {
             "@type": "KeyExtractor",
-            "key": "edgeAgentEnabled",
+            "key": "edgeNodesEnabled",
             "defaultValue": "false"
           }
         },
@@ -44,7 +44,7 @@
         },
 
         {
-          "comment": "=== Data Layer (KeyExtractor, hidden CLI params with CSS_ prefix) ===",
+          "comment": "=== Data Layer ===",
           "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:identityDbUrl",
           "CombinedShorthandResolver:_resolvers_value": {
             "@type": "KeyExtractor",
@@ -147,15 +147,7 @@
         },
 
         {
-          "comment": "=== Xpod Business Features (KeyExtractor, CSS_ prefix via Yargs) ===",
-          "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:signalEndpoint",
-          "CombinedShorthandResolver:_resolvers_value": {
-            "@type": "KeyExtractor",
-            "key": "signalEndpoint",
-            "defaultValue": ""
-          }
-        },
-        {
+          "comment": "=== Business Features ===",
           "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:clusterIngressDomain",
           "CombinedShorthandResolver:_resolvers_value": {
             "@type": "KeyExtractor",
@@ -171,13 +163,6 @@
             "defaultValue": ""
           }
         },
-        {
-          "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:dnsRootDomain",
-          "CombinedShorthandResolver:_resolvers_value": {
-            "@type": "KeyExtractor",
-            "key": "dnsRootDomain"
-          }
-        },
         {
           "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:dnsRecordTtl",
           "CombinedShorthandResolver:_resolvers_value": {
@@ -218,14 +203,6 @@
             "defaultValue": "0"
           }
         },
-        {
-          "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:tunnelEntrypoints",
-          "CombinedShorthandResolver:_resolvers_value": {
-            "@type": "KeyExtractor",
-            "key": "tunnelEntrypoints",
-            "defaultValue": ""
-          }
-        },
         {
           "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:edgeHealthProbesEnabled",
           "CombinedShorthandResolver:_resolvers_value": {
@@ -284,22 +261,6 @@
             "defaultValue": ""
           }
         },
-        {
-          "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:nodeToken",
-          "CombinedShorthandResolver:_resolvers_value": {
-            "@type": "KeyExtractor",
-            "key": "nodeToken",
-            "defaultValue": ""
-          }
-        },
-        {
-          "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:nodePublicAddress",
-          "CombinedShorthandResolver:_resolvers_value": {
-            "@type": "KeyExtractor",
-            "key": "nodePublicAddress",
-            "defaultValue": ""
-          }
-        },
         {
           "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:nodeHeartbeatInterval",
           "CombinedShorthandResolver:_resolvers_value": {
@@ -316,12 +277,13 @@
             "defaultValue": "false"
           }
         },
+
         {
           "comment": "=== IdP/SP Separation Parameters ===",
-          "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:baseDomain",
+          "CombinedShorthandResolver:_resolvers_key": "urn:solid-server:default:variable:baseStorageDomain",
           "CombinedShorthandResolver:_resolvers_value": {
             "@type": "KeyExtractor",
-            "key": "baseDomain",
+            "key": "baseStorageDomain",
             "defaultValue": ""
           }
         },

package/config/runtime-open.json

@@ -0,0 +1,22 @@
+{
+  "@context": [
+    "https://linkedsoftwaredependencies.org/bundles/npm/@solid/community-server/^8.0.0/components/context.jsonld",
+    "https://linkedsoftwaredependencies.org/bundles/npm/asynchronous-handlers/^1.0.0/components/context.jsonld"
+  ],
+  "@graph": [
+    {
+      "comment": "Runtime-only override: disable auth resource handler in allow-all mode.",
+      "@type": "Override",
+      "overrideInstance": {
+        "@id": "urn:solid-server:default:AuthResourceHttpHandler"
+      },
+      "overrideParameters": {
+        "@type": "StaticThrowHandler",
+        "onlyHandle": false,
+        "factory": {
+          "@id": "urn:solid-server:default:UnsupportedErrorFactory"
+        }
+      }
+    }
+  ]
+}

package/config/xpod.base.json

@@ -19,6 +19,38 @@
       "@id": "urn:solid-server:default:variable:edgeNodesEnabled",
       "@type": "Variable"
     },
+    {
+      "comment": "ACP permission reader clone for auth mode switching.",
+      "@id": "urn:undefineds:xpod:AcpPermissionReader",
+      "@type": "CachedHandler",
+      "fields": [
+        "credentials",
+        "requestedModes"
+      ],
+      "source": {
+        "@id": "urn:solid-server:default:AuxiliaryReader"
+      }
+    },
+    {
+      "comment": "Override default permission reader so tests can switch auth mode with a parameter.",
+      "@type": "Override",
+      "overrideInstance": {
+        "@id": "urn:solid-server:default:PermissionReader"
+      },
+      "overrideParameters": {
+        "@type": "AuthModeSelector",
+        "authMode": {
+          "@id": "urn:solid-server:default:variable:authMode",
+          "@type": "Variable"
+        },
+        "acpReader": {
+          "@id": "urn:undefineds:xpod:AcpPermissionReader"
+        },
+        "aclReader": {
+          "@id": "urn:undefineds:xpod:AcpPermissionReader"
+        }
+      }
+    },
     {
       "comment": "An accessor for a Quadstore",
       "@type": "QuadstoreSparqlDataAccessor",