@rubytech/create-maxy-code 0.1.321 → 0.1.323
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/index.js +134 -1
- package/package.json +1 -1
- package/payload/platform/lib/embed-client/dist/index.d.ts +2 -0
- package/payload/platform/lib/embed-client/dist/index.d.ts.map +1 -1
- package/payload/platform/lib/embed-client/dist/index.js +50 -10
- package/payload/platform/lib/embed-client/dist/index.js.map +1 -1
- package/payload/platform/lib/embed-client/src/index.ts +57 -9
- package/payload/platform/plugins/admin/skills/insight/SKILL.md +1 -1
- package/payload/platform/plugins/admin/skills/platform-architecture/SKILL.md +39 -5
- package/payload/platform/plugins/docs/references/admin-ui.md +12 -0
- package/payload/platform/plugins/docs/references/deployment.md +20 -0
- package/payload/platform/plugins/docs/references/internals.md +2 -0
- package/payload/platform/plugins/docs/references/plugins-guide.md +4 -4
- package/payload/platform/plugins/memory/PLUGIN.md +0 -1
- package/payload/platform/plugins/memory/mcp/dist/index.js +0 -45
- package/payload/platform/plugins/memory/mcp/dist/index.js.map +1 -1
- package/payload/platform/plugins/memory/mcp/dist/lib/typed-edge-schema.js +6 -6
- package/payload/platform/plugins/memory/mcp/dist/lib/typed-edge-schema.js.map +1 -1
- package/payload/platform/plugins/memory/mcp/dist/tools/__tests__/memory-ingest-body-server-sliced.test.js +41 -2
- package/payload/platform/plugins/memory/mcp/dist/tools/__tests__/memory-ingest-body-server-sliced.test.js.map +1 -1
- package/payload/platform/plugins/memory/mcp/dist/tools/memory-ingest.d.ts.map +1 -1
- package/payload/platform/plugins/memory/mcp/dist/tools/memory-ingest.js +34 -4
- package/payload/platform/plugins/memory/mcp/dist/tools/memory-ingest.js.map +1 -1
- package/payload/platform/plugins/notion-import/.claude-plugin/plugin.json +1 -1
- package/payload/platform/plugins/notion-import/PLUGIN.md +6 -6
- package/payload/platform/plugins/notion-import/skills/notion-import/SKILL.md +1 -1
- package/payload/platform/plugins/notion-import/skills/notion-import/references/databases.md +1 -1
- package/payload/platform/plugins/obsidian-import/.claude-plugin/plugin.json +1 -1
- package/payload/platform/plugins/obsidian-import/PLUGIN.md +5 -5
- package/payload/platform/plugins/obsidian-import/skills/obsidian-import/SKILL.md +1 -1
- package/payload/platform/plugins/substack-import/.claude-plugin/plugin.json +1 -1
- package/payload/platform/plugins/substack-import/PLUGIN.md +8 -8
- package/payload/platform/plugins/substack-import/skills/substack-import/SKILL.md +2 -2
- package/payload/platform/plugins/substack-import/skills/substack-import/references/posts.md +1 -1
- package/payload/platform/plugins/x-import/.claude-plugin/plugin.json +1 -1
- package/payload/platform/plugins/x-import/PLUGIN.md +5 -5
- package/payload/platform/plugins/x-import/skills/x-import/SKILL.md +1 -1
- package/payload/platform/scripts/check-no-esm-require.mjs +3 -0
- package/payload/platform/scripts/identity-forbidden-token-check.mjs +0 -1
- package/payload/platform/services/claude-session-manager/dist/canonical-tool-names.generated.d.ts.map +1 -1
- package/payload/platform/services/claude-session-manager/dist/canonical-tool-names.generated.js +0 -1
- package/payload/platform/services/claude-session-manager/dist/canonical-tool-names.generated.js.map +1 -1
- package/payload/platform/services/claude-session-manager/dist/http-server.d.ts.map +1 -1
- package/payload/platform/services/claude-session-manager/dist/http-server.js +75 -2
- package/payload/platform/services/claude-session-manager/dist/http-server.js.map +1 -1
- package/payload/platform/services/claude-session-manager/dist/index.js +3 -3
- package/payload/platform/services/claude-session-manager/dist/index.js.map +1 -1
- package/payload/platform/services/claude-session-manager/dist/jsonl-tail.d.ts.map +1 -1
- package/payload/platform/services/claude-session-manager/dist/jsonl-tail.js +8 -4
- package/payload/platform/services/claude-session-manager/dist/jsonl-tail.js.map +1 -1
- package/payload/platform/services/claude-session-manager/dist/pty-spawner.d.ts +16 -3
- package/payload/platform/services/claude-session-manager/dist/pty-spawner.d.ts.map +1 -1
- package/payload/platform/services/claude-session-manager/dist/pty-spawner.js +29 -12
- package/payload/platform/services/claude-session-manager/dist/pty-spawner.js.map +1 -1
- package/payload/platform/services/claude-session-manager/dist/rc-daemon.d.ts +5 -4
- package/payload/platform/services/claude-session-manager/dist/rc-daemon.d.ts.map +1 -1
- package/payload/platform/services/claude-session-manager/dist/rc-daemon.js.map +1 -1
- package/payload/platform/services/claude-session-manager/dist/reaper.js +6 -6
- package/payload/platform/services/claude-session-manager/dist/reaper.js.map +1 -1
- package/payload/platform/services/claude-session-manager/dist/system-prompt.d.ts.map +1 -1
- package/payload/platform/services/claude-session-manager/dist/system-prompt.js +0 -1
- package/payload/platform/services/claude-session-manager/dist/system-prompt.js.map +1 -1
- package/payload/platform/templates/specialists/agents/coding-assistant.md +1 -1
- package/payload/platform/templates/specialists/agents/database-operator.md +2 -6
- package/payload/platform/templates/specialists/agents/librarian.md +20 -7
- package/payload/server/power-health.cjs +129 -0
- package/payload/server/public/assets/{AdminLoginScreens-DKwWudsP.js → AdminLoginScreens-BBtA0mi8.js} +1 -1
- package/payload/server/public/assets/AdminShell-CZW9x0-A.js +1 -0
- package/payload/server/public/assets/{Checkbox-V5ubVZG1.js → Checkbox-CezYRnKL.js} +1 -1
- package/payload/server/public/assets/{OperatorConversations-DOGCHZhv.css → OperatorConversations-Bhyf22Ct.css} +1 -1
- package/payload/server/public/assets/OperatorConversations-D6BawEz3.js +9 -0
- package/payload/server/public/assets/{admin-Cfqj5N4P.js → admin-BVpCUS4j.js} +1 -1
- package/payload/server/public/assets/{browser-CtMwlpyW.js → browser-ByLzuh0X.js} +1 -1
- package/payload/server/public/assets/chat-DTvzNb92.js +1 -0
- package/payload/server/public/assets/data-Da1oTowD.js +1 -0
- package/payload/server/public/assets/{graph-Bez5QF37.js → graph-iYrodF_0.js} +3 -3
- package/payload/server/public/assets/{graph-labels-BkbaBPIP.js → graph-labels-DM9u5VTf.js} +1 -1
- package/payload/server/public/assets/operator-BPplyxq-.js +1 -0
- package/payload/server/public/assets/page-oogapuCz.js +30 -0
- package/payload/server/public/assets/{public-wy93nfmL.js → public-wBFC1mQO.js} +1 -1
- package/payload/server/public/browser.html +4 -4
- package/payload/server/public/chat.html +5 -5
- package/payload/server/public/data.html +4 -4
- package/payload/server/public/graph.html +6 -6
- package/payload/server/public/index.html +6 -6
- package/payload/server/public/operator.html +7 -7
- package/payload/server/public/public.html +5 -5
- package/payload/server/server-init.cjs +40 -0
- package/payload/server/server.js +212 -12
- package/payload/platform/plugins/memory/mcp/scripts/backfill-typed-edges.ts +0 -72
- package/payload/platform/templates/specialists/agents/archive-ingest-operator.md +0 -45
- package/payload/server/public/assets/AdminShell-BsPsXvJv.js +0 -1
- package/payload/server/public/assets/OperatorConversations-DPRRlS_z.js +0 -9
- package/payload/server/public/assets/chat-Bm11OGsU.js +0 -1
- package/payload/server/public/assets/data-7_dN5vN5.js +0 -1
- package/payload/server/public/assets/operator-F5KHg5WG.js +0 -1
- package/payload/server/public/assets/page-CExyEcvh.js +0 -30
package/dist/index.js
CHANGED
|
@@ -20,7 +20,7 @@ import { classifyPortHolder } from "./preflight-port-classifier.js";
|
|
|
20
20
|
import { parsePluginList, computeInstallActions, parseExternalPlugins, findUnregisteredResyncs, } from "./lib/plugin-install.js";
|
|
21
21
|
import { findPremiumMcpDirs } from "./lib/premium-mcp-discover.js";
|
|
22
22
|
import { pickBindDecision, mergeSmbConf, formatSambaMarker, SAMBA_ENABLE_UNITS, } from "./samba-provision.js";
|
|
23
|
-
import { networkInterfaces, userInfo } from "node:os";
|
|
23
|
+
import { networkInterfaces, userInfo, cpus } from "node:os";
|
|
24
24
|
const PAYLOAD_DIR = resolve(import.meta.dirname, "../payload");
|
|
25
25
|
// Brand manifest — read from payload to derive all brand-specific installation values.
|
|
26
26
|
// The bundler stamps brand.json into the payload at build time.
|
|
@@ -608,6 +608,55 @@ function writeChromiumBinaryPathFile() {
|
|
|
608
608
|
console.log(` Wrote ${target} → ${RESOLVED_CHROMIUM_BIN}`);
|
|
609
609
|
logFile(` [snap-chromium] wrote ${target} contents=${RESOLVED_CHROMIUM_BIN}`);
|
|
610
610
|
}
|
|
611
|
+
// Arm the systemd system-manager hardware watchdog so a *kernel hang* (the
|
|
612
|
+
// freeze mode: board powered but stuck, /dev/watchdog never petted) reboots the
|
|
613
|
+
// board without a manual power cycle. Distinct from the brand user-service's
|
|
614
|
+
// software watchdog (Type=notify WATCHDOG_USEC), which only restarts the UI
|
|
615
|
+
// process — this resets the whole SoC. Idempotent host-wide drop-in; gated on
|
|
616
|
+
// /dev/watchdog existing (driver loaded) and Linux. Best-effort: a failure
|
|
617
|
+
// warns, never aborts the install. Cannot recover a latched-off brownout (no
|
|
618
|
+
// power = no watchdog) — that needs the external power-cycler documented in
|
|
619
|
+
// deployment.md.
|
|
620
|
+
function configureHardwareWatchdog() {
|
|
621
|
+
try {
|
|
622
|
+
if (!isLinux()) {
|
|
623
|
+
logFile(" hardware watchdog: skipped (not Linux)");
|
|
624
|
+
return;
|
|
625
|
+
}
|
|
626
|
+
if (!existsSync("/dev/watchdog")) {
|
|
627
|
+
logFile(" hardware watchdog: skipped (/dev/watchdog absent)");
|
|
628
|
+
return;
|
|
629
|
+
}
|
|
630
|
+
const desired = "[Manager]\nRuntimeWatchdogSec=20s\nRebootWatchdogSec=2min\n";
|
|
631
|
+
const dropinDir = "/etc/systemd/system.conf.d";
|
|
632
|
+
const dropinPath = `${dropinDir}/10-maxy-watchdog.conf`;
|
|
633
|
+
let current = "";
|
|
634
|
+
if (existsSync(dropinPath)) {
|
|
635
|
+
try {
|
|
636
|
+
current = readFileSync(dropinPath, "utf-8");
|
|
637
|
+
}
|
|
638
|
+
catch {
|
|
639
|
+
// Unreadable existing drop-in — treat as changed so we rewrite.
|
|
640
|
+
current = "";
|
|
641
|
+
}
|
|
642
|
+
}
|
|
643
|
+
if (current === desired) {
|
|
644
|
+
logFile(` hardware watchdog: already active (${dropinPath})`);
|
|
645
|
+
return;
|
|
646
|
+
}
|
|
647
|
+
const tmpPath = "/tmp/10-maxy-watchdog.conf";
|
|
648
|
+
writeFileSync(tmpPath, desired);
|
|
649
|
+
console.log(" [privileged] install systemd hardware watchdog drop-in (RuntimeWatchdogSec=20s RebootWatchdogSec=2min)");
|
|
650
|
+
shell("mkdir", ["-p", dropinDir], { sudo: true });
|
|
651
|
+
shell("cp", [tmpPath, dropinPath], { sudo: true });
|
|
652
|
+
spawnSync("rm", ["-f", tmpPath]);
|
|
653
|
+
spawnSync("sudo", ["systemctl", "daemon-reload"], { stdio: "inherit" });
|
|
654
|
+
logFile(` hardware watchdog: enabled (wrote ${dropinPath}; arms on next boot or systemd re-exec)`);
|
|
655
|
+
}
|
|
656
|
+
catch (err) {
|
|
657
|
+
console.error(` WARNING: failed to configure hardware watchdog: ${err instanceof Error ? err.message : String(err)}`);
|
|
658
|
+
}
|
|
659
|
+
}
|
|
611
660
|
function installSystemDeps() {
|
|
612
661
|
log("1", TOTAL, "System dependencies and network...");
|
|
613
662
|
const platform = requireSupportedPlatform(process.platform);
|
|
@@ -1557,6 +1606,86 @@ function ensureOllamaServing() {
|
|
|
1557
1606
|
logFile(` Ollama serve timed out after 30s. Log tail:\n${serveLog}`);
|
|
1558
1607
|
throw new Error(`Ollama server did not respond within 30s. Log: ${ollamaLog}\n${serveLog}`);
|
|
1559
1608
|
}
|
|
1609
|
+
// Cap ollama's CPU so an embedding burst cannot thermally hard-reset a
|
|
1610
|
+
// passively-cooled Pi. The platform embeds a document's summary plus every
|
|
1611
|
+
// section body via one `/api/embed` POST; ollama's llama-server then runs
|
|
1612
|
+
// flat-out across every core for the whole batch with no idle gaps, and on a
|
|
1613
|
+
// passively-cooled Pi 5 the SoC climbs past its soft-temp limit until the board
|
|
1614
|
+
// reaches its thermal ceiling and hard-resets. A systemd `CPUQuota` drop-in on
|
|
1615
|
+
// `ollama.service` bounds llama-server to a fraction of the cores so peak
|
|
1616
|
+
// temperature plateaus below the limit regardless of batch shape or which
|
|
1617
|
+
// platform path issues the embeds — the caller-agnostic fix, inherited by every
|
|
1618
|
+
// install. Quota is one core (100%): an embedding burst then draws roughly a
|
|
1619
|
+
// quarter of a 4-core Pi's compute current instead of half, which both keeps
|
|
1620
|
+
// peak SoC temperature down and shaves the current spike that browns out a
|
|
1621
|
+
// marginal PSU. Ingest is slower in exchange. Applied only when the ollama
|
|
1622
|
+
// system service exists (the normal Linux path, where ollama's install.sh
|
|
1623
|
+
// registers `ollama.service`); a detached `ollama serve` has no unit to cap.
|
|
1624
|
+
// ollama is one shared host service across co-resident brands, so the drop-in is
|
|
1625
|
+
// host-wide with a neutral filename — idempotent across brands and reinstalls,
|
|
1626
|
+
// and ollama restarts only when the drop-in content actually changes. Best-
|
|
1627
|
+
// effort: a failure warns, never aborts the install.
|
|
1628
|
+
function capOllamaServiceCpu() {
|
|
1629
|
+
try {
|
|
1630
|
+
const svcCheck = spawnSync("systemctl", ["cat", "ollama"], { stdio: "pipe", timeout: 5_000 });
|
|
1631
|
+
if (svcCheck.status !== 0) {
|
|
1632
|
+
logFile(" ollama CPU cap: skipped (no ollama system service to cap)");
|
|
1633
|
+
return;
|
|
1634
|
+
}
|
|
1635
|
+
const coreCount = Math.max(1, cpus().length);
|
|
1636
|
+
const quotaCores = 1;
|
|
1637
|
+
const quotaPct = quotaCores * 100;
|
|
1638
|
+
const desired = `[Service]\nCPUQuota=${quotaPct}%\n`;
|
|
1639
|
+
const dropinDir = "/etc/systemd/system/ollama.service.d";
|
|
1640
|
+
const dropinPath = `${dropinDir}/10-maxy-cpuquota.conf`;
|
|
1641
|
+
let current = "";
|
|
1642
|
+
if (existsSync(dropinPath)) {
|
|
1643
|
+
try {
|
|
1644
|
+
current = readFileSync(dropinPath, "utf-8");
|
|
1645
|
+
}
|
|
1646
|
+
catch {
|
|
1647
|
+
// Unreadable existing drop-in — treat as changed so we rewrite + restart.
|
|
1648
|
+
current = "";
|
|
1649
|
+
}
|
|
1650
|
+
}
|
|
1651
|
+
if (current === desired) {
|
|
1652
|
+
logFile(` ollama CPU cap: already active (${dropinPath} matches CPUQuota=${quotaPct}%)`);
|
|
1653
|
+
return;
|
|
1654
|
+
}
|
|
1655
|
+
const tmpPath = "/tmp/10-maxy-ollama-cpuquota.conf";
|
|
1656
|
+
writeFileSync(tmpPath, desired);
|
|
1657
|
+
console.log(` [privileged] install ollama CPUQuota=${quotaPct}% drop-in (${quotaCores}/${coreCount} cores)`);
|
|
1658
|
+
shell("mkdir", ["-p", dropinDir], { sudo: true });
|
|
1659
|
+
shell("cp", [tmpPath, dropinPath], { sudo: true });
|
|
1660
|
+
spawnSync("rm", ["-f", tmpPath]);
|
|
1661
|
+
spawnSync("sudo", ["systemctl", "daemon-reload"], { stdio: "inherit" });
|
|
1662
|
+
console.log(" [privileged] systemctl restart ollama");
|
|
1663
|
+
spawnSync("sudo", ["systemctl", "restart", "ollama"], { stdio: "inherit", timeout: 30_000 });
|
|
1664
|
+
// The restart briefly drops the API. Wait for ollama to serve again before
|
|
1665
|
+
// returning so the immediately-following ensureOllamaServing() probe finds
|
|
1666
|
+
// it up and does not race the restart by spawning a second detached
|
|
1667
|
+
// `ollama serve` that would contend for port 11434. Bounded poll; the drop-in
|
|
1668
|
+
// is already on disk either way, so a timeout still applies the cap on the
|
|
1669
|
+
// next ollama start.
|
|
1670
|
+
let served = false;
|
|
1671
|
+
for (let elapsed = 0; elapsed < 15; elapsed += 3) {
|
|
1672
|
+
if (spawnSync("curl", ["-sf", "http://localhost:11434/api/tags"], { stdio: "pipe", timeout: 5_000 }).status === 0) {
|
|
1673
|
+
served = true;
|
|
1674
|
+
break;
|
|
1675
|
+
}
|
|
1676
|
+
spawnSync("sleep", ["3"]);
|
|
1677
|
+
}
|
|
1678
|
+
if (served) {
|
|
1679
|
+
logFile(` ollama CPU cap: enabled (wrote ${dropinPath} CPUQuota=${quotaPct}%, ollama serving again under the cap)`);
|
|
1680
|
+
}
|
|
1681
|
+
else {
|
|
1682
|
+
logFile(` ollama CPU cap: drop-in written (${dropinPath} CPUQuota=${quotaPct}%); ollama not serving yet after restart — cap applies on its next start`);
|
|
1683
|
+
}
|
|
1684
|
+
}
|
|
1685
|
+
catch (err) {
|
|
1686
|
+
console.error(` WARNING: failed to cap ollama CPU: ${err instanceof Error ? err.message : String(err)}`);
|
|
1687
|
+
}
|
|
1688
|
+
}
|
|
1560
1689
|
function installOllama(embedModel) {
|
|
1561
1690
|
if (!commandExists("ollama")) {
|
|
1562
1691
|
log("5", TOTAL, "Installing Ollama...");
|
|
@@ -1565,6 +1694,9 @@ function installOllama(embedModel) {
|
|
|
1565
1694
|
else {
|
|
1566
1695
|
log("5", TOTAL, "Ollama already installed.");
|
|
1567
1696
|
}
|
|
1697
|
+
// Bound ollama's CPU before bringing the service up so the capped unit is what
|
|
1698
|
+
// serves embeds (and the restart below lands the new quota).
|
|
1699
|
+
capOllamaServiceCpu();
|
|
1568
1700
|
ensureOllamaServing();
|
|
1569
1701
|
console.log(` Pulling ${embedModel} embedding model...`);
|
|
1570
1702
|
logFile(` Pulling embedding model: ${embedModel}`);
|
|
@@ -4107,6 +4239,7 @@ logDiagnostics("pre-flight");
|
|
|
4107
4239
|
logFile(` Neo4j instance: ${NEO4J_DEDICATED ? "dedicated" : "shared"} on bolt://localhost:${NEO4J_PORT}`);
|
|
4108
4240
|
try {
|
|
4109
4241
|
installSystemDeps();
|
|
4242
|
+
configureHardwareWatchdog();
|
|
4110
4243
|
installNodejs();
|
|
4111
4244
|
installClaudeCode();
|
|
4112
4245
|
installNeo4j();
|
package/package.json
CHANGED
|
@@ -1,4 +1,6 @@
|
|
|
1
1
|
export declare const EMBED_INPUT_MAX_CHARS = 8000;
|
|
2
|
+
export declare const EMBED_SUBBATCH_SIZE: number;
|
|
3
|
+
export declare const EMBED_SUBBATCH_COOLDOWN_MS: number;
|
|
2
4
|
export declare function embed(text: string): Promise<number[]>;
|
|
3
5
|
export declare function embedBatch(texts: string[]): Promise<number[][]>;
|
|
4
6
|
//# sourceMappingURL=index.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAcA,eAAO,MAAM,qBAAqB,OAAO,CAAC;
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAcA,eAAO,MAAM,qBAAqB,OAAO,CAAC;AAmB1C,eAAO,MAAM,mBAAmB,QAA6D,CAAC;AAC9F,eAAO,MAAM,0BAA0B,QAGtC,CAAC;AA4BF,wBAAsB,KAAK,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC,CAa3D;AAED,wBAAsB,UAAU,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,CAoBrE"}
|
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
// and one POST shape across the platform (Task 636). fetch-only — no node:fs —
|
|
7
7
|
// so it is safe to bundle into the ESM payload.
|
|
8
8
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
9
|
-
exports.EMBED_INPUT_MAX_CHARS = void 0;
|
|
9
|
+
exports.EMBED_SUBBATCH_COOLDOWN_MS = exports.EMBED_SUBBATCH_SIZE = exports.EMBED_INPUT_MAX_CHARS = void 0;
|
|
10
10
|
exports.embed = embed;
|
|
11
11
|
exports.embedBatch = embedBatch;
|
|
12
12
|
const OLLAMA_URL = process.env.OLLAMA_URL ?? "http://localhost:11434";
|
|
@@ -16,28 +16,49 @@ const EMBED_MODEL = process.env.EMBED_MODEL ?? "nomic-embed-text";
|
|
|
16
16
|
// an oversized body embeds (truncated) instead of throwing and aborting the
|
|
17
17
|
// caller.
|
|
18
18
|
exports.EMBED_INPUT_MAX_CHARS = 8000;
|
|
19
|
+
// Batch pacing (Task 960). On a passively-cooled Pi, embedding a whole
|
|
20
|
+
// document's sections in one `/api/embed` POST pins ollama's llama-server
|
|
21
|
+
// across every core for the entire batch with no idle gaps — the SoC climbs
|
|
22
|
+
// past its soft-temp limit and the board thermally hard-resets. embedBatch
|
|
23
|
+
// splits an oversized input array into sub-batches and waits a cooldown between
|
|
24
|
+
// them so the SoC gets cooling gaps. Centralising here means every batch caller
|
|
25
|
+
// (memory-ingest, memory-archive-write, memory-reindex) inherits the pacing.
|
|
26
|
+
// The systemd CPUQuota cap on ollama.service is the load-bearing thermal fix;
|
|
27
|
+
// this pacing is defence-in-depth and faster recovery. Both knobs are
|
|
28
|
+
// env-overridable so an operator can tune them from on-device thermal
|
|
29
|
+
// measurement without a code change.
|
|
30
|
+
function envInt(name, def) {
|
|
31
|
+
const raw = process.env[name];
|
|
32
|
+
if (raw === undefined || raw === "")
|
|
33
|
+
return def;
|
|
34
|
+
const n = Number(raw);
|
|
35
|
+
return Number.isFinite(n) ? n : def;
|
|
36
|
+
}
|
|
37
|
+
exports.EMBED_SUBBATCH_SIZE = Math.max(1, Math.floor(envInt("EMBED_SUBBATCH_SIZE", 32)));
|
|
38
|
+
exports.EMBED_SUBBATCH_COOLDOWN_MS = Math.max(0, envInt("EMBED_SUBBATCH_COOLDOWN_MS", 1000));
|
|
19
39
|
function capInput(text) {
|
|
20
40
|
if (text.length <= exports.EMBED_INPUT_MAX_CHARS)
|
|
21
41
|
return text;
|
|
22
42
|
process.stderr.write(`[embed] op=truncate origBytes=${text.length} cappedBytes=${exports.EMBED_INPUT_MAX_CHARS}\n`);
|
|
23
43
|
return text.slice(0, exports.EMBED_INPUT_MAX_CHARS);
|
|
24
44
|
}
|
|
25
|
-
|
|
26
|
-
|
|
45
|
+
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
|
46
|
+
// POST one already-capped sub-batch of inputs to the embed endpoint.
|
|
47
|
+
async function postBatch(inputs) {
|
|
27
48
|
const res = await fetch(`${OLLAMA_URL}/api/embed`, {
|
|
28
49
|
method: "POST",
|
|
29
50
|
headers: { "Content-Type": "application/json" },
|
|
30
|
-
body: JSON.stringify({ model: EMBED_MODEL, input }),
|
|
51
|
+
body: JSON.stringify({ model: EMBED_MODEL, input: inputs }),
|
|
31
52
|
});
|
|
32
53
|
if (!res.ok) {
|
|
33
54
|
const body = await res.text();
|
|
34
|
-
throw new Error(`Ollama embedding failed (${res.status}): ${body}`);
|
|
55
|
+
throw new Error(`Ollama batch embedding failed (${res.status}): ${body}`);
|
|
35
56
|
}
|
|
36
57
|
const data = (await res.json());
|
|
37
|
-
return data.embeddings
|
|
58
|
+
return data.embeddings;
|
|
38
59
|
}
|
|
39
|
-
async function
|
|
40
|
-
const input =
|
|
60
|
+
async function embed(text) {
|
|
61
|
+
const input = capInput(text);
|
|
41
62
|
const res = await fetch(`${OLLAMA_URL}/api/embed`, {
|
|
42
63
|
method: "POST",
|
|
43
64
|
headers: { "Content-Type": "application/json" },
|
|
@@ -45,9 +66,28 @@ async function embedBatch(texts) {
|
|
|
45
66
|
});
|
|
46
67
|
if (!res.ok) {
|
|
47
68
|
const body = await res.text();
|
|
48
|
-
throw new Error(`Ollama
|
|
69
|
+
throw new Error(`Ollama embedding failed (${res.status}): ${body}`);
|
|
49
70
|
}
|
|
50
71
|
const data = (await res.json());
|
|
51
|
-
return data.embeddings;
|
|
72
|
+
return data.embeddings[0];
|
|
73
|
+
}
|
|
74
|
+
async function embedBatch(texts) {
|
|
75
|
+
const inputs = texts.map(capInput);
|
|
76
|
+
// Common case — an array within the sub-batch size is one POST, unchanged.
|
|
77
|
+
if (inputs.length <= exports.EMBED_SUBBATCH_SIZE) {
|
|
78
|
+
return postBatch(inputs);
|
|
79
|
+
}
|
|
80
|
+
// Oversized: split into sub-batches, cooling between each (not after the
|
|
81
|
+
// last). Vectors are concatenated in input order so callers can keep their
|
|
82
|
+
// index-aligned `embeddings[i]` ↔ `texts[i]` contract.
|
|
83
|
+
const out = [];
|
|
84
|
+
for (let i = 0; i < inputs.length; i += exports.EMBED_SUBBATCH_SIZE) {
|
|
85
|
+
if (i > 0 && exports.EMBED_SUBBATCH_COOLDOWN_MS > 0) {
|
|
86
|
+
await sleep(exports.EMBED_SUBBATCH_COOLDOWN_MS);
|
|
87
|
+
}
|
|
88
|
+
const chunk = inputs.slice(i, i + exports.EMBED_SUBBATCH_SIZE);
|
|
89
|
+
out.push(...(await postBatch(chunk)));
|
|
90
|
+
}
|
|
91
|
+
return out;
|
|
52
92
|
}
|
|
53
93
|
//# sourceMappingURL=index.js.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AAAA,yEAAyE;AACzE,6EAA6E;AAC7E,2EAA2E;AAC3E,6EAA6E;AAC7E,+EAA+E;AAC/E,gDAAgD;;;
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":";AAAA,yEAAyE;AACzE,6EAA6E;AAC7E,2EAA2E;AAC3E,6EAA6E;AAC7E,+EAA+E;AAC/E,gDAAgD;;;AA4DhD,sBAaC;AAED,gCAoBC;AA7FD,MAAM,UAAU,GAAG,OAAO,CAAC,GAAG,CAAC,UAAU,IAAI,wBAAwB,CAAC;AACtE,MAAM,WAAW,GAAG,OAAO,CAAC,GAAG,CAAC,WAAW,IAAI,kBAAkB,CAAC;AAElE,2EAA2E;AAC3E,gFAAgF;AAChF,4EAA4E;AAC5E,UAAU;AACG,QAAA,qBAAqB,GAAG,IAAI,CAAC;AAE1C,uEAAuE;AACvE,0EAA0E;AAC1E,4EAA4E;AAC5E,2EAA2E;AAC3E,gFAAgF;AAChF,gFAAgF;AAChF,6EAA6E;AAC7E,8EAA8E;AAC9E,sEAAsE;AACtE,sEAAsE;AACtE,qCAAqC;AACrC,SAAS,MAAM,CAAC,IAAY,EAAE,GAAW;IACvC,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;IAC9B,IAAI,GAAG,KAAK,SAAS,IAAI,GAAG,KAAK,EAAE;QAAE,OAAO,GAAG,CAAC;IAChD,MAAM,CAAC,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;IACtB,OAAO,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC;AACtC,CAAC;AACY,QAAA,mBAAmB,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,qBAAqB,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC;AACjF,QAAA,0BAA0B,GAAG,IAAI,CAAC,GAAG,CAChD,CAAC,EACD,MAAM,CAAC,4BAA4B,EAAE,IAAI,CAAC,CAC3C,CAAC;AAEF,SAAS,QAAQ,CAAC,IAAY;IAC5B,IAAI,IAAI,CAAC,MAAM,IAAI,6BAAqB;QAAE,OAAO,IAAI,CAAC;IACtD,OAAO,CAAC,MAAM,CAAC,KAAK,CAClB,iCAAiC,IAAI,CAAC,MAAM,gBAAgB,6BAAqB,IAAI,CACtF,CAAC;IACF,OAAO,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,6BAAqB,CAAC,CAAC;AAC9C,CAAC;AAED,MAAM,KAAK,GAAG,CAAC,EAAU,EAAiB,EAAE,CAC1C,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,UAAU,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC,CAAC;AAEpD,qEAAqE;AACrE,KAAK,UAAU,SAAS,CAAC,MAAgB;IACvC,MAAM,GAAG,GAAG,MAAM,KAAK,CAAC,GAAG,UAAU,YAAY,EAAE;QACjD,MAAM,EAAE,MAAM;QACd,OAAO,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE;QAC/C,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,WAAW,EAAE,KAAK,EAAE,MAAM,EAAE,CAAC;KAC5D,CAAC,CAAC;IACH,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,CAAC;QACZ,MAAM,IAAI,GAAG,MAAM,GAAG,CAAC,IAAI,EAAE,CAAC;QAC9B,MAAM,IAAI,KAAK,CAAC,kCAAkC,GAAG,CAAC,MAAM,MAAM,IAAI,EAAE,CAAC,CAAC;IAC5E,CAAC;IACD,MAAM,IAAI,GAAG,CAAC,MAAM,GAAG,CAAC,IAAI,EAAE,CAA+B,CAAC;IAC9D,OAAO,IAAI,CAAC,UAAU,CAAC;AACzB,CAAC;AAEM,KAAK,UAAU,KAAK,CAAC,IAAY;IACtC,MAAM,KAAK,GAAG,QAAQ,CAAC,IAAI,CAAC,CAAC;IAC7B,MAAM,GAAG,GAAG,MAAM,KAAK,CAAC,GAAG,UAAU,YAAY,EAAE;QACjD,MAAM,EAAE,MAAM;QACd,OAAO,EAAE,EAAE,cAAc,EAAE,kBAAkB,EAAE;QAC/C,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,EAAE,KAAK,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;KACpD,CAAC,CAAC;IACH,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,CAAC;QACZ,MAAM,IAAI,GAAG,MAAM,GAAG,CAAC,IAAI,EAAE,CAAC;QAC9B,MAAM,IAAI,KAAK,CAAC,4BAA4B,GAAG,CAAC,MAAM,MAAM,IAAI,EAAE,CAAC,CAAC;IACtE,CAAC;IACD,MAAM,IAAI,GAAG,CAAC,MAAM,GAAG,CAAC,IAAI,EAAE,CAA+B,CAAC;IAC9D,OAAO,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC;AAC5B,CAAC;AAEM,KAAK,UAAU,UAAU,CAAC,KAAe;IAC9C,MAAM,MAAM,GAAG,KAAK,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;IAEnC,2EAA2E;IAC3E,IAAI,MAAM,CAAC,MAAM,IAAI,2BAAmB,EAAE,CAAC;QACzC,OAAO,SAAS,CAAC,MAAM,CAAC,CAAC;IAC3B,CAAC;IAED,yEAAyE;IACzE,2EAA2E;IAC3E,uDAAuD;IACvD,MAAM,GAAG,GAAe,EAAE,CAAC;IAC3B,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,MAAM,CAAC,MAAM,EAAE,CAAC,IAAI,2BAAmB,EAAE,CAAC;QAC5D,IAAI,CAAC,GAAG,CAAC,IAAI,kCAA0B,GAAG,CAAC,EAAE,CAAC;YAC5C,MAAM,KAAK,CAAC,kCAA0B,CAAC,CAAC;QAC1C,CAAC;QACD,MAAM,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,GAAG,2BAAmB,CAAC,CAAC;QACvD,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,SAAS,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC;IACxC,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC"}
|
|
@@ -14,6 +14,29 @@ const EMBED_MODEL = process.env.EMBED_MODEL ?? "nomic-embed-text";
|
|
|
14
14
|
// caller.
|
|
15
15
|
export const EMBED_INPUT_MAX_CHARS = 8000;
|
|
16
16
|
|
|
17
|
+
// Batch pacing (Task 960). On a passively-cooled Pi, embedding a whole
|
|
18
|
+
// document's sections in one `/api/embed` POST pins ollama's llama-server
|
|
19
|
+
// across every core for the entire batch with no idle gaps — the SoC climbs
|
|
20
|
+
// past its soft-temp limit and the board thermally hard-resets. embedBatch
|
|
21
|
+
// splits an oversized input array into sub-batches and waits a cooldown between
|
|
22
|
+
// them so the SoC gets cooling gaps. Centralising here means every batch caller
|
|
23
|
+
// (memory-ingest, memory-archive-write, memory-reindex) inherits the pacing.
|
|
24
|
+
// The systemd CPUQuota cap on ollama.service is the load-bearing thermal fix;
|
|
25
|
+
// this pacing is defence-in-depth and faster recovery. Both knobs are
|
|
26
|
+
// env-overridable so an operator can tune them from on-device thermal
|
|
27
|
+
// measurement without a code change.
|
|
28
|
+
function envInt(name: string, def: number): number {
|
|
29
|
+
const raw = process.env[name];
|
|
30
|
+
if (raw === undefined || raw === "") return def;
|
|
31
|
+
const n = Number(raw);
|
|
32
|
+
return Number.isFinite(n) ? n : def;
|
|
33
|
+
}
|
|
34
|
+
export const EMBED_SUBBATCH_SIZE = Math.max(1, Math.floor(envInt("EMBED_SUBBATCH_SIZE", 32)));
|
|
35
|
+
export const EMBED_SUBBATCH_COOLDOWN_MS = Math.max(
|
|
36
|
+
0,
|
|
37
|
+
envInt("EMBED_SUBBATCH_COOLDOWN_MS", 1000),
|
|
38
|
+
);
|
|
39
|
+
|
|
17
40
|
function capInput(text: string): string {
|
|
18
41
|
if (text.length <= EMBED_INPUT_MAX_CHARS) return text;
|
|
19
42
|
process.stderr.write(
|
|
@@ -22,23 +45,26 @@ function capInput(text: string): string {
|
|
|
22
45
|
return text.slice(0, EMBED_INPUT_MAX_CHARS);
|
|
23
46
|
}
|
|
24
47
|
|
|
25
|
-
|
|
26
|
-
|
|
48
|
+
const sleep = (ms: number): Promise<void> =>
|
|
49
|
+
new Promise((resolve) => setTimeout(resolve, ms));
|
|
50
|
+
|
|
51
|
+
// POST one already-capped sub-batch of inputs to the embed endpoint.
|
|
52
|
+
async function postBatch(inputs: string[]): Promise<number[][]> {
|
|
27
53
|
const res = await fetch(`${OLLAMA_URL}/api/embed`, {
|
|
28
54
|
method: "POST",
|
|
29
55
|
headers: { "Content-Type": "application/json" },
|
|
30
|
-
body: JSON.stringify({ model: EMBED_MODEL, input }),
|
|
56
|
+
body: JSON.stringify({ model: EMBED_MODEL, input: inputs }),
|
|
31
57
|
});
|
|
32
58
|
if (!res.ok) {
|
|
33
59
|
const body = await res.text();
|
|
34
|
-
throw new Error(`Ollama embedding failed (${res.status}): ${body}`);
|
|
60
|
+
throw new Error(`Ollama batch embedding failed (${res.status}): ${body}`);
|
|
35
61
|
}
|
|
36
62
|
const data = (await res.json()) as { embeddings: number[][] };
|
|
37
|
-
return data.embeddings
|
|
63
|
+
return data.embeddings;
|
|
38
64
|
}
|
|
39
65
|
|
|
40
|
-
export async function
|
|
41
|
-
const input =
|
|
66
|
+
export async function embed(text: string): Promise<number[]> {
|
|
67
|
+
const input = capInput(text);
|
|
42
68
|
const res = await fetch(`${OLLAMA_URL}/api/embed`, {
|
|
43
69
|
method: "POST",
|
|
44
70
|
headers: { "Content-Type": "application/json" },
|
|
@@ -46,8 +72,30 @@ export async function embedBatch(texts: string[]): Promise<number[][]> {
|
|
|
46
72
|
});
|
|
47
73
|
if (!res.ok) {
|
|
48
74
|
const body = await res.text();
|
|
49
|
-
throw new Error(`Ollama
|
|
75
|
+
throw new Error(`Ollama embedding failed (${res.status}): ${body}`);
|
|
50
76
|
}
|
|
51
77
|
const data = (await res.json()) as { embeddings: number[][] };
|
|
52
|
-
return data.embeddings;
|
|
78
|
+
return data.embeddings[0];
|
|
79
|
+
}
|
|
80
|
+
|
|
81
|
+
export async function embedBatch(texts: string[]): Promise<number[][]> {
|
|
82
|
+
const inputs = texts.map(capInput);
|
|
83
|
+
|
|
84
|
+
// Common case — an array within the sub-batch size is one POST, unchanged.
|
|
85
|
+
if (inputs.length <= EMBED_SUBBATCH_SIZE) {
|
|
86
|
+
return postBatch(inputs);
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
// Oversized: split into sub-batches, cooling between each (not after the
|
|
90
|
+
// last). Vectors are concatenated in input order so callers can keep their
|
|
91
|
+
// index-aligned `embeddings[i]` ↔ `texts[i]` contract.
|
|
92
|
+
const out: number[][] = [];
|
|
93
|
+
for (let i = 0; i < inputs.length; i += EMBED_SUBBATCH_SIZE) {
|
|
94
|
+
if (i > 0 && EMBED_SUBBATCH_COOLDOWN_MS > 0) {
|
|
95
|
+
await sleep(EMBED_SUBBATCH_COOLDOWN_MS);
|
|
96
|
+
}
|
|
97
|
+
const chunk = inputs.slice(i, i + EMBED_SUBBATCH_SIZE);
|
|
98
|
+
out.push(...(await postBatch(chunk)));
|
|
99
|
+
}
|
|
100
|
+
return out;
|
|
53
101
|
}
|
|
@@ -19,6 +19,6 @@ Pass 2 — Operator-relationship updates. Walk this session for tonal signals, r
|
|
|
19
19
|
|
|
20
20
|
Pass 3 — Graph completeness sweep. Walk this session for any node, edge, or commitment that was discussed but not written to the graph in-flight. For each missing write, delegate to `database-operator` via the Task tool. Count what you write.
|
|
21
21
|
|
|
22
|
-
Pass 4 — Typed-edge auto-extraction. Delegate to `
|
|
22
|
+
Pass 4 — Typed-edge auto-extraction. Delegate to `typed-edge-classifier` via the Task tool with a prompt supplying the `accountId` and the session-start timestamp as `sinceIso`. The specialist reads prose nodes written since that timestamp, writes typed edges from the closed allowlist, and returns one line `typed-edge-classifier: nodesProcessed=… proposed=… accepted=… rejected=… ms=…`. Capture `nodesProcessed` and `accepted` from that line.
|
|
23
23
|
|
|
24
24
|
Then reply with one short paragraph in your house voice summarising what you wrote — naming the five counts (learnings, preferences, graph-writes, typed-edges, nodesProcessed) — and carry on with the operator's work.
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: platform-architecture
|
|
3
3
|
description: Use when grounding any documented-surface claim about what Maxy ships — plugins, skills, specialists, install/deploy flows, internals. This is the install catalogue, not evidence of what is enabled on the current account. For install state on this account, call `capabilities-here`; for documented surface, cite the `Source:` URL inline.
|
|
4
|
-
content-hash: sha256:
|
|
4
|
+
content-hash: sha256:479e04a1e25ace04e44415db3c536ed28faac7ee30b9171524d38b7a667716d1
|
|
5
5
|
brand: maxy-code
|
|
6
6
|
product-name: Maxy
|
|
7
7
|
---
|
|
@@ -365,10 +365,10 @@ These are enabled during onboarding and can be added or removed at any time. Som
|
|
|
365
365
|
| `whatsapp` | WhatsApp messaging, pairing, and conversation browsing | Personal assistant |
|
|
366
366
|
| `replicate` | Image generation — three models for photorealistic, design, and fast draft images | Content producer, Research assistant |
|
|
367
367
|
| `linkedin-import` | Import a LinkedIn Basic Data Export — Profile and Connections today, more CSVs as references land | Database operator |
|
|
368
|
-
| `notion-import` | Import a Notion workspace export (markdown + CSV) — pages, databases, hierarchy, attachments, schema-bounded relations, `@person` mentions account-filtered |
|
|
369
|
-
| `obsidian-import` | Import an extracted Obsidian vault — pages map to `:KnowledgeDocument`, wikilinks resolve to intra-vault pages or existing entities, tags become `:DefinedTerm`, embedded images become `:DigitalDocument`. Two-phase tool (dry-run → operator disambiguation → commit). |
|
|
370
|
-
| `x-import` | Import an X (Twitter) Basic Data Export — tweet stream renders as one chronological transcript and ingests as a single `:KnowledgeDocument` (`source='x'`); each DM `sessionId` ingests as one `:ConversationArchive` (`source='x-dm'`, keyed on `conversationIdentity`) via `conversation-archive-ingest.sh`. Mentions, replies, and quote-tweet authors resolve to `:Person` on lowercased `xHandle`; every handle and DM senderId confirms against existing nodes (no auto-create). Per-thread KD granularity and `:Post` / `:DirectMessage` labels are explicitly rejected. |
|
|
371
|
-
| `substack-import` | Import a Substack "Export your data" archive — per-essay `:KnowledgeDocument {kind:'substack-post'}` via librarian/document-ingest with synthetic stable `attachmentId = "substack-post-${substackPostId}"` (survives Substack edits); one `:KnowledgeDocument {kind:'substack-subscriber-roster'}` per import run with `:MENTIONS {mentionContext:'substack-subscription', tier, totalOpens, totalClicks, lastOpenedAt, lastClickedAt, engagementWindowDays}` to each subscriber `:Person` MERGEd on `(accountId, email)`. Engagement aggregates parsed from `email_activity.csv` (or `subscriber_activity.csv` / `emails.csv`); overwrite-on-reimport. No new label, no new edge type, no new graph writer. Images attach via canonical `:HAS_ENCLOSURE` (or `:MENTIONS` fallback). Bulk-gate at >200 posts or >2000 subscribers. |
|
|
368
|
+
| `notion-import` | Import a Notion workspace export (markdown + CSV) — pages, databases, hierarchy, attachments, schema-bounded relations, `@person` mentions account-filtered | Librarian |
|
|
369
|
+
| `obsidian-import` | Import an extracted Obsidian vault — pages map to `:KnowledgeDocument`, wikilinks resolve to intra-vault pages or existing entities, tags become `:DefinedTerm`, embedded images become `:DigitalDocument`. Two-phase tool (dry-run → operator disambiguation → commit). | Librarian |
|
|
370
|
+
| `x-import` | Import an X (Twitter) Basic Data Export — tweet stream renders as one chronological transcript and ingests as a single `:KnowledgeDocument` (`source='x'`); each DM `sessionId` ingests as one `:ConversationArchive` (`source='x-dm'`, keyed on `conversationIdentity`) via `conversation-archive-ingest.sh`. Mentions, replies, and quote-tweet authors resolve to `:Person` on lowercased `xHandle`; every handle and DM senderId confirms against existing nodes (no auto-create). Per-thread KD granularity and `:Post` / `:DirectMessage` labels are explicitly rejected. | Librarian |
|
|
371
|
+
| `substack-import` | Import a Substack "Export your data" archive — per-essay `:KnowledgeDocument {kind:'substack-post'}` via librarian/document-ingest with synthetic stable `attachmentId = "substack-post-${substackPostId}"` (survives Substack edits); one `:KnowledgeDocument {kind:'substack-subscriber-roster'}` per import run with `:MENTIONS {mentionContext:'substack-subscription', tier, totalOpens, totalClicks, lastOpenedAt, lastClickedAt, engagementWindowDays}` to each subscriber `:Person` MERGEd on `(accountId, email)`. Engagement aggregates parsed from `email_activity.csv` (or `subscriber_activity.csv` / `emails.csv`); overwrite-on-reimport. No new label, no new edge type, no new graph writer. Images attach via canonical `:HAS_ENCLOSURE` (or `:MENTIONS` fallback). Bulk-gate at >200 posts or >2000 subscribers. | Librarian |
|
|
372
372
|
| `memory/skills/conversation-archive` | Source-agnostic conversation transcript ingest. One skill for WhatsApp `_chat.txt`, Telegram, Signal, LinkedIn DMs, Zoom transcript, meeting minutes, iMessage, Slack, X DMs — `--source <enum>` selects the per-source normaliser. Single Bash entry — `bash platform/plugins/memory/bin/conversation-archive-ingest.sh <archive> --source <enum> --participant-person-ids <csv> --scope <admin\|public>` — runs normalise → operator-confirms owner + every distinct sender (owner derived from env via Cypher, no flag) → sessionize at the fixed 8h gap → emit one JSON line carrying prepared sessions (turn-attributed text + per-session cursor). The dispatched specialist iterates the sessions in-turn, produces a typed-section JSON chunking for each, and calls the `memory-ingest` MCP tool with `conversationIdentity` set (writes `:ConversationArchive`, source=<enum>) once per session — chunks + cursor advance commit atomically inside one Cypher transaction, so a kill mid-archive resumes from the next session on re-issue without re-classifying anything already written. Re-imports are delta-append. Auto-creating participants is forbidden — any sender outside the operator-confirmed closed set LOUD-FAILs with `parser-miss`. Distinct from the live `whatsapp` plugin (Baileys). | Database operator |
|
|
373
373
|
| `memory/skills/conversation-archive-enrich` | Phase 2 for any named `:ConversationArchive` — source-agnostic per-row insight derivation. Operator-triggered (never auto-fires on Phase 1 completion). Walks the parent's `:Section` chunks in pages via the read-only MCP tool `mcp__plugin_memory_memory__conversation-archive-list-chunks`; the dispatched specialist reads each chunk in-turn and emits claims under the four-kind contract (`mention`, `task`, `preference`, `observed-relationship`); the skill hands those claims to `mcp__plugin_memory_memory__conversation-archive-derive-insights` for per-kind cypher emission, then runs the per-row operator gate (`wire / skip / reject`). Idempotent on `(elementId(chunk), kind, contentHash)` — re-runs collapse identical claims. Confidence floor is a hedging-avoidance instruction the skill embeds in the specialist's per-chunk prompt, not a numeric post-filter; the LLM step runs in-turn from the dispatched specialist rather than as a server-side OAuth round-trip. | Database operator |
|
|
374
374
|
|
|
@@ -2290,6 +2290,18 @@ reserved for vertical / integration notes (LinkedIn extension,
|
|
|
2290
2290
|
PropertyData, Real Agent standalone, MCP server inventory) — not for
|
|
2291
2291
|
core-platform docs.
|
|
2292
2292
|
|
|
2293
|
+
## UI conventions
|
|
2294
|
+
|
|
2295
|
+
**Icons are Lucide components only — no emoji.** Every glyph rendered in
|
|
2296
|
+
`platform/ui` JSX (the `/chat` web chat, the WhatsApp/channel readers, the
|
|
2297
|
+
admin shell, the `/data` grid) is a `lucide-react` component (e.g.
|
|
2298
|
+
`<Paperclip>`, `<FileText>`), never an emoji/emoticon character in the rendered
|
|
2299
|
+
string. The rule is enforced at build time by
|
|
2300
|
+
[`scripts/check-no-emoji-in-jsx.mjs`](../../../ui/scripts/check-no-emoji-in-jsx.mjs),
|
|
2301
|
+
which fails `prebuild` if any emoji codepoint appears in a `.ts`/`.tsx` file
|
|
2302
|
+
under `app/` (tests excluded). Inline an icon at the surrounding text size and
|
|
2303
|
+
colour (`size={14}` matches body text; Lucide inherits `currentColor`).
|
|
2304
|
+
|
|
2293
2305
|
## Admin Hono routes
|
|
2294
2306
|
|
|
2295
2307
|
Every admin sub-app is mounted by
|
|
@@ -2995,6 +3007,8 @@ If Ollama is unavailable at write time, the node is stored without an embedding
|
|
|
2995
3007
|
|
|
2996
3008
|
The `memory-reindex` tool backfills missing embeddings by iterating nodes where `embedding IS NULL`. It uses the bounded representation (the `summary` for document/archive labels, never the raw body), embeds in batches with a per-node fallback on batch error, and **skips and continues** past a failing node — keeping skipped nodes out of the re-fetch via an exclude set — instead of aborting the whole run. `embed()`/`embedBatch()` cap oversized input (`EMBED_INPUT_MAX_CHARS`) before the Ollama POST, so a large body embeds truncated rather than throwing. Per-node `[reindex] op=embedded|skip-error` lines and a terminal `[reindex] op=done processed=N skipped=M`.
|
|
2997
3009
|
|
|
3010
|
+
`embedBatch()` also paces large batches. A document's summary plus every section body in one `/api/embed` POST pins ollama's `llama-server` across all cores for the whole batch with no idle gaps; on a passively-cooled Pi the SoC climbs past its soft-temp limit and the board thermally hard-resets. `embedBatch` splits an input array longer than `EMBED_SUBBATCH_SIZE` (default 32) into sub-batches, waiting `EMBED_SUBBATCH_COOLDOWN_MS` (default 1000) between them so the SoC gets cooling gaps; vectors are concatenated in input order. Both knobs are env-overridable for on-device tuning. The load-bearing thermal protection is the installer's `CPUQuota` cap on `ollama.service` (every embed caller is bounded there regardless of batch shape); this pacing is defence-in-depth. The UI's file-index path uses its own single-input embed client in `platform/ui/app/lib/neo4j-store.ts` (one embed per file, not a batch), so it is covered by the CPU cap, not by `embedBatch` pacing.
|
|
3011
|
+
|
|
2998
3012
|
---
|
|
2999
3013
|
|
|
3000
3014
|
## Knowledge Document Hierarchy
|
|
@@ -3456,6 +3470,25 @@ Source: https://docs.getmaxy.com/deployment.md
|
|
|
3456
3470
|
- Mac with macOS 14 (Sonoma) or newer — both Apple Silicon and Intel
|
|
3457
3471
|
- 256GB storage minimum
|
|
3458
3472
|
- Always-on power and network connection
|
|
3473
|
+
- **On a Pi 5: the official 27W USB-C power supply (5.1V / 5A) and a good cable.** This is not optional — see Power and recovery below.
|
|
3474
|
+
|
|
3475
|
+
## Power and recovery (Raspberry Pi)
|
|
3476
|
+
|
|
3477
|
+
A Pi 5 running Maxy draws hard during document ingest: the embedding model runs the CPU flat-out in bursts. An underpowered supply or a thin/long USB-C cable cannot hold 5V under that current spike, the board *browns out*, and you get one of two failures:
|
|
3478
|
+
|
|
3479
|
+
- **A reset or freeze** — the board drops out mid-task and either reboots or hangs.
|
|
3480
|
+
- **A latched-off board** — the board goes dark and stays dark until you physically unplug and replug the power. No software can recover this, because there is no power to run any recovery.
|
|
3481
|
+
|
|
3482
|
+
**The cure is hardware.** Use the official Raspberry Pi 5 27W PSU (5.1V / 5A) and a short, good-quality USB-C cable. A marginal phone charger or a cheap cable is the usual cause. This is the only thing that *prevents* a brownout.
|
|
3483
|
+
|
|
3484
|
+
**Recovery seatbelts the installer sets up for you:**
|
|
3485
|
+
|
|
3486
|
+
- **A hardware watchdog.** If the board *hangs* (powered but stuck), it reboots itself automatically within about two minutes — no manual power cycle. This only helps a hung board; it cannot help a board that has gone dark.
|
|
3487
|
+
- **Power-health logging.** The admin server writes a `[power-health]` line to `server.log` at boot and every 15 minutes, so a brownout shows up without waiting for a crash. A healthy line reads `throttled=0x0 ... undervolt_occurred=false`. Any `undervolt`-flag set, or `reboots_unclean_24h` above zero, means the board is browning out — fix the power before trusting it with heavy work.
|
|
3488
|
+
|
|
3489
|
+
**For a board that latches off (the dark-board case):** put the Pi on a smart plug that you can power-cycle remotely, ideally one driven by a heartbeat (cut power and restore it when the box stops responding). This is the robust recovery for the dark-board failure and is an operator hardware choice, not something the installer can do.
|
|
3490
|
+
|
|
3491
|
+
**Keep one address.** After a reboot the Pi can pick up a different DHCP address and look like it vanished. Add a **DHCP reservation** on your router that pins the Pi's MAC to a fixed LAN IP, so the box is always reachable at the same address. The boot `[power-health] op=boot lan_ip=<ip>` line records the address each boot — if it changes across reboots, the reservation is not in place.
|
|
3459
3492
|
|
|
3460
3493
|
## macOS install
|
|
3461
3494
|
|
|
@@ -3703,6 +3736,7 @@ A single Pi or laptop can host more than one brand (for example Maxy and Real Ag
|
|
|
3703
3736
|
- **Brand-isolated Neo4j:** when a brand provisions a dedicated Neo4j instance (any port other than 7687), the installer stops and disables the apt-package's system `neo4j.service` after enabling the brand-dedicated unit, so only one Neo4j process holds the shared `/var/lib/neo4j/run/` PID file. The seed step receives the brand-correct `NEO4J_URI` and `NEO4J_PASSWORD` as explicit environment variables — the seed script no longer carries a `bolt://localhost:7687` default. A failed dedicated start aborts the install loudly with a journalctl tail; there is no silent fallback to the system instance. Stop/disable targets the literal `neo4j.service` only, so peer brands running their own `neo4j-{brand}.service` are unaffected.
|
|
3704
3737
|
- **Peer-aware system-unit guard:** before stopping the system `neo4j.service`, the installer checks whether any other brand on the device still depends on it — that is, has `NEO4J_URI=bolt://localhost:7687` in its `~/.<peer>/.env`. If so, the system unit is left enabled and active, and the install log shows `[neo4j] system unit kept active — peer brand <name> depends on port 7687` instead of the usual `[neo4j] disabling system unit` line. This prevents a `create-realagent` install from disabling Maxy's database on a host where Maxy still uses the shared system instance (the earlier platform fixes reproducer on Neo's laptop, 2026-04-28). On single-brand hosts and on multi-brand hosts where every peer runs a dedicated port, behaviour is unchanged. The dedicated unit exports `NEO4J_HOME=<per-brand-data-dir>` alongside `NEO4J_CONF`, so `server.directories.run`, `server.directories.plugins`, and `server.directories.import` resolve per-brand — no collision with `/var/lib/neo4j/run/neo4j.pid`. The conf sed-overrides, mkdir-p, chown, and unit-write are idempotent and re-run on every install, so a host whose prior install left a broken unit recovers on retry.
|
|
3705
3738
|
- **Shared:** both brands share the system Chromium/VNC stack, the Ollama model server, and the `cloudflared` command itself. Browser automation is serialised — one admin session at a time across both brands.
|
|
3739
|
+
- **Thermal cap on shared Ollama:** the installer writes a host-wide systemd `CPUQuota` drop-in (`/etc/systemd/system/ollama.service.d/10-maxy-cpuquota.conf`) bounding `ollama.service` to half the cores (200% on a 4-core Pi). An embedding burst — a document's summary plus every section body in one POST — otherwise pins `llama-server` across all cores with no idle gaps until a passively-cooled Pi crosses its soft-temp limit and hard-resets. The cap keeps peak temperature below the limit regardless of which brand or platform path issues the embeds. The drop-in has a neutral filename and is rewritten only when its content changes, so it is idempotent across brands and reinstalls. Applied only where the `ollama.service` system unit exists; a detached `ollama serve` has no unit to cap.
|
|
3706
3740
|
|
|
3707
3741
|
To install a second brand on a device that already runs the first, just run the other installer. No flags needed for isolation:
|
|
3708
3742
|
|
|
@@ -25,6 +25,18 @@ reserved for vertical / integration notes (LinkedIn extension,
|
|
|
25
25
|
PropertyData, Real Agent standalone, MCP server inventory) — not for
|
|
26
26
|
core-platform docs.
|
|
27
27
|
|
|
28
|
+
## UI conventions
|
|
29
|
+
|
|
30
|
+
**Icons are Lucide components only — no emoji.** Every glyph rendered in
|
|
31
|
+
`platform/ui` JSX (the `/chat` web chat, the WhatsApp/channel readers, the
|
|
32
|
+
admin shell, the `/data` grid) is a `lucide-react` component (e.g.
|
|
33
|
+
`<Paperclip>`, `<FileText>`), never an emoji/emoticon character in the rendered
|
|
34
|
+
string. The rule is enforced at build time by
|
|
35
|
+
[`scripts/check-no-emoji-in-jsx.mjs`](../../../ui/scripts/check-no-emoji-in-jsx.mjs),
|
|
36
|
+
which fails `prebuild` if any emoji codepoint appears in a `.ts`/`.tsx` file
|
|
37
|
+
under `app/` (tests excluded). Inline an icon at the surrounding text size and
|
|
38
|
+
colour (`size={14}` matches body text; Lucide inherits `currentColor`).
|
|
39
|
+
|
|
28
40
|
## Admin Hono routes
|
|
29
41
|
|
|
30
42
|
Every admin sub-app is mounted by
|
|
@@ -6,6 +6,25 @@
|
|
|
6
6
|
- Mac with macOS 14 (Sonoma) or newer — both Apple Silicon and Intel
|
|
7
7
|
- 256GB storage minimum
|
|
8
8
|
- Always-on power and network connection
|
|
9
|
+
- **On a Pi 5: the official 27W USB-C power supply (5.1V / 5A) and a good cable.** This is not optional — see Power and recovery below.
|
|
10
|
+
|
|
11
|
+
## Power and recovery (Raspberry Pi)
|
|
12
|
+
|
|
13
|
+
A Pi 5 running {{productName}} draws hard during document ingest: the embedding model runs the CPU flat-out in bursts. An underpowered supply or a thin/long USB-C cable cannot hold 5V under that current spike, the board *browns out*, and you get one of two failures:
|
|
14
|
+
|
|
15
|
+
- **A reset or freeze** — the board drops out mid-task and either reboots or hangs.
|
|
16
|
+
- **A latched-off board** — the board goes dark and stays dark until you physically unplug and replug the power. No software can recover this, because there is no power to run any recovery.
|
|
17
|
+
|
|
18
|
+
**The cure is hardware.** Use the official Raspberry Pi 5 27W PSU (5.1V / 5A) and a short, good-quality USB-C cable. A marginal phone charger or a cheap cable is the usual cause. This is the only thing that *prevents* a brownout.
|
|
19
|
+
|
|
20
|
+
**Recovery seatbelts the installer sets up for you:**
|
|
21
|
+
|
|
22
|
+
- **A hardware watchdog.** If the board *hangs* (powered but stuck), it reboots itself automatically within about two minutes — no manual power cycle. This only helps a hung board; it cannot help a board that has gone dark.
|
|
23
|
+
- **Power-health logging.** The admin server writes a `[power-health]` line to `server.log` at boot and every 15 minutes, so a brownout shows up without waiting for a crash. A healthy line reads `throttled=0x0 ... undervolt_occurred=false`. Any `undervolt`-flag set, or `reboots_unclean_24h` above zero, means the board is browning out — fix the power before trusting it with heavy work.
|
|
24
|
+
|
|
25
|
+
**For a board that latches off (the dark-board case):** put the Pi on a smart plug that you can power-cycle remotely, ideally one driven by a heartbeat (cut power and restore it when the box stops responding). This is the robust recovery for the dark-board failure and is an operator hardware choice, not something the installer can do.
|
|
26
|
+
|
|
27
|
+
**Keep one address.** After a reboot the Pi can pick up a different DHCP address and look like it vanished. Add a **DHCP reservation** on your router that pins the Pi's MAC to a fixed LAN IP, so the box is always reachable at the same address. The boot `[power-health] op=boot lan_ip=<ip>` line records the address each boot — if it changes across reboots, the reservation is not in place.
|
|
9
28
|
|
|
10
29
|
## macOS install
|
|
11
30
|
|
|
@@ -253,6 +272,7 @@ A single Pi or laptop can host more than one brand (for example Maxy and Real Ag
|
|
|
253
272
|
- **Brand-isolated Neo4j:** when a brand provisions a dedicated Neo4j instance (any port other than 7687), the installer stops and disables the apt-package's system `neo4j.service` after enabling the brand-dedicated unit, so only one Neo4j process holds the shared `/var/lib/neo4j/run/` PID file. The seed step receives the brand-correct `NEO4J_URI` and `NEO4J_PASSWORD` as explicit environment variables — the seed script no longer carries a `bolt://localhost:7687` default. A failed dedicated start aborts the install loudly with a journalctl tail; there is no silent fallback to the system instance. Stop/disable targets the literal `neo4j.service` only, so peer brands running their own `neo4j-{brand}.service` are unaffected.
|
|
254
273
|
- **Peer-aware system-unit guard:** before stopping the system `neo4j.service`, the installer checks whether any other brand on the device still depends on it — that is, has `NEO4J_URI=bolt://localhost:7687` in its `~/.<peer>/.env`. If so, the system unit is left enabled and active, and the install log shows `[neo4j] system unit kept active — peer brand <name> depends on port 7687` instead of the usual `[neo4j] disabling system unit` line. This prevents a `create-realagent` install from disabling Maxy's database on a host where Maxy still uses the shared system instance (the earlier platform fixes reproducer on Neo's laptop, 2026-04-28). On single-brand hosts and on multi-brand hosts where every peer runs a dedicated port, behaviour is unchanged. The dedicated unit exports `NEO4J_HOME=<per-brand-data-dir>` alongside `NEO4J_CONF`, so `server.directories.run`, `server.directories.plugins`, and `server.directories.import` resolve per-brand — no collision with `/var/lib/neo4j/run/neo4j.pid`. The conf sed-overrides, mkdir-p, chown, and unit-write are idempotent and re-run on every install, so a host whose prior install left a broken unit recovers on retry.
|
|
255
274
|
- **Shared:** both brands share the system Chromium/VNC stack, the Ollama model server, and the `cloudflared` command itself. Browser automation is serialised — one admin session at a time across both brands.
|
|
275
|
+
- **Thermal cap on shared Ollama:** the installer writes a host-wide systemd `CPUQuota` drop-in (`/etc/systemd/system/ollama.service.d/10-maxy-cpuquota.conf`) bounding `ollama.service` to half the cores (200% on a 4-core Pi). An embedding burst — a document's summary plus every section body in one POST — otherwise pins `llama-server` across all cores with no idle gaps until a passively-cooled Pi crosses its soft-temp limit and hard-resets. The cap keeps peak temperature below the limit regardless of which brand or platform path issues the embeds. The drop-in has a neutral filename and is rewritten only when its content changes, so it is idempotent across brands and reinstalls. Applied only where the `ollama.service` system unit exists; a detached `ollama serve` has no unit to cap.
|
|
256
276
|
|
|
257
277
|
To install a second brand on a device that already runs the first, just run the other installer. No flags needed for isolation:
|
|
258
278
|
|
|
@@ -95,6 +95,8 @@ If Ollama is unavailable at write time, the node is stored without an embedding
|
|
|
95
95
|
|
|
96
96
|
The `memory-reindex` tool backfills missing embeddings by iterating nodes where `embedding IS NULL`. It uses the bounded representation (the `summary` for document/archive labels, never the raw body), embeds in batches with a per-node fallback on batch error, and **skips and continues** past a failing node — keeping skipped nodes out of the re-fetch via an exclude set — instead of aborting the whole run. `embed()`/`embedBatch()` cap oversized input (`EMBED_INPUT_MAX_CHARS`) before the Ollama POST, so a large body embeds truncated rather than throwing. Per-node `[reindex] op=embedded|skip-error` lines and a terminal `[reindex] op=done processed=N skipped=M`.
|
|
97
97
|
|
|
98
|
+
`embedBatch()` also paces large batches. A document's summary plus every section body in one `/api/embed` POST pins ollama's `llama-server` across all cores for the whole batch with no idle gaps; on a passively-cooled Pi the SoC climbs past its soft-temp limit and the board thermally hard-resets. `embedBatch` splits an input array longer than `EMBED_SUBBATCH_SIZE` (default 32) into sub-batches, waiting `EMBED_SUBBATCH_COOLDOWN_MS` (default 1000) between them so the SoC gets cooling gaps; vectors are concatenated in input order. Both knobs are env-overridable for on-device tuning. The load-bearing thermal protection is the installer's `CPUQuota` cap on `ollama.service` (every embed caller is bounded there regardless of batch shape); this pacing is defence-in-depth. The UI's file-index path uses its own single-input embed client in `platform/ui/app/lib/neo4j-store.ts` (one embed per file, not a batch), so it is covered by the CPU cap, not by `embedBatch` pacing.
|
|
99
|
+
|
|
98
100
|
---
|
|
99
101
|
|
|
100
102
|
## Knowledge Document Hierarchy
|
|
@@ -43,10 +43,10 @@ These are enabled during onboarding and can be added or removed at any time. Som
|
|
|
43
43
|
| `whatsapp` | WhatsApp messaging, pairing, and conversation browsing | Personal assistant |
|
|
44
44
|
| `replicate` | Image generation — three models for photorealistic, design, and fast draft images | Content producer, Research assistant |
|
|
45
45
|
| `linkedin-import` | Import a LinkedIn Basic Data Export — Profile and Connections today, more CSVs as references land | Database operator |
|
|
46
|
-
| `notion-import` | Import a Notion workspace export (markdown + CSV) — pages, databases, hierarchy, attachments, schema-bounded relations, `@person` mentions account-filtered |
|
|
47
|
-
| `obsidian-import` | Import an extracted Obsidian vault — pages map to `:KnowledgeDocument`, wikilinks resolve to intra-vault pages or existing entities, tags become `:DefinedTerm`, embedded images become `:DigitalDocument`. Two-phase tool (dry-run → operator disambiguation → commit). |
|
|
48
|
-
| `x-import` | Import an X (Twitter) Basic Data Export — tweet stream renders as one chronological transcript and ingests as a single `:KnowledgeDocument` (`source='x'`); each DM `sessionId` ingests as one `:ConversationArchive` (`source='x-dm'`, keyed on `conversationIdentity`) via `conversation-archive-ingest.sh`. Mentions, replies, and quote-tweet authors resolve to `:Person` on lowercased `xHandle`; every handle and DM senderId confirms against existing nodes (no auto-create). Per-thread KD granularity and `:Post` / `:DirectMessage` labels are explicitly rejected. |
|
|
49
|
-
| `substack-import` | Import a Substack "Export your data" archive — per-essay `:KnowledgeDocument {kind:'substack-post'}` via librarian/document-ingest with synthetic stable `attachmentId = "substack-post-${substackPostId}"` (survives Substack edits); one `:KnowledgeDocument {kind:'substack-subscriber-roster'}` per import run with `:MENTIONS {mentionContext:'substack-subscription', tier, totalOpens, totalClicks, lastOpenedAt, lastClickedAt, engagementWindowDays}` to each subscriber `:Person` MERGEd on `(accountId, email)`. Engagement aggregates parsed from `email_activity.csv` (or `subscriber_activity.csv` / `emails.csv`); overwrite-on-reimport. No new label, no new edge type, no new graph writer. Images attach via canonical `:HAS_ENCLOSURE` (or `:MENTIONS` fallback). Bulk-gate at >200 posts or >2000 subscribers. |
|
|
46
|
+
| `notion-import` | Import a Notion workspace export (markdown + CSV) — pages, databases, hierarchy, attachments, schema-bounded relations, `@person` mentions account-filtered | Librarian |
|
|
47
|
+
| `obsidian-import` | Import an extracted Obsidian vault — pages map to `:KnowledgeDocument`, wikilinks resolve to intra-vault pages or existing entities, tags become `:DefinedTerm`, embedded images become `:DigitalDocument`. Two-phase tool (dry-run → operator disambiguation → commit). | Librarian |
|
|
48
|
+
| `x-import` | Import an X (Twitter) Basic Data Export — tweet stream renders as one chronological transcript and ingests as a single `:KnowledgeDocument` (`source='x'`); each DM `sessionId` ingests as one `:ConversationArchive` (`source='x-dm'`, keyed on `conversationIdentity`) via `conversation-archive-ingest.sh`. Mentions, replies, and quote-tweet authors resolve to `:Person` on lowercased `xHandle`; every handle and DM senderId confirms against existing nodes (no auto-create). Per-thread KD granularity and `:Post` / `:DirectMessage` labels are explicitly rejected. | Librarian |
|
|
49
|
+
| `substack-import` | Import a Substack "Export your data" archive — per-essay `:KnowledgeDocument {kind:'substack-post'}` via librarian/document-ingest with synthetic stable `attachmentId = "substack-post-${substackPostId}"` (survives Substack edits); one `:KnowledgeDocument {kind:'substack-subscriber-roster'}` per import run with `:MENTIONS {mentionContext:'substack-subscription', tier, totalOpens, totalClicks, lastOpenedAt, lastClickedAt, engagementWindowDays}` to each subscriber `:Person` MERGEd on `(accountId, email)`. Engagement aggregates parsed from `email_activity.csv` (or `subscriber_activity.csv` / `emails.csv`); overwrite-on-reimport. No new label, no new edge type, no new graph writer. Images attach via canonical `:HAS_ENCLOSURE` (or `:MENTIONS` fallback). Bulk-gate at >200 posts or >2000 subscribers. | Librarian |
|
|
50
50
|
| `memory/skills/conversation-archive` | Source-agnostic conversation transcript ingest. One skill for WhatsApp `_chat.txt`, Telegram, Signal, LinkedIn DMs, Zoom transcript, meeting minutes, iMessage, Slack, X DMs — `--source <enum>` selects the per-source normaliser. Single Bash entry — `bash platform/plugins/memory/bin/conversation-archive-ingest.sh <archive> --source <enum> --participant-person-ids <csv> --scope <admin\|public>` — runs normalise → operator-confirms owner + every distinct sender (owner derived from env via Cypher, no flag) → sessionize at the fixed 8h gap → emit one JSON line carrying prepared sessions (turn-attributed text + per-session cursor). The dispatched specialist iterates the sessions in-turn, produces a typed-section JSON chunking for each, and calls the `memory-ingest` MCP tool with `conversationIdentity` set (writes `:ConversationArchive`, source=<enum>) once per session — chunks + cursor advance commit atomically inside one Cypher transaction, so a kill mid-archive resumes from the next session on re-issue without re-classifying anything already written. Re-imports are delta-append. Auto-creating participants is forbidden — any sender outside the operator-confirmed closed set LOUD-FAILs with `parser-miss`. Distinct from the live `whatsapp` plugin (Baileys). | Database operator |
|
|
51
51
|
| `memory/skills/conversation-archive-enrich` | Phase 2 for any named `:ConversationArchive` — source-agnostic per-row insight derivation. Operator-triggered (never auto-fires on Phase 1 completion). Walks the parent's `:Section` chunks in pages via the read-only MCP tool `mcp__plugin_memory_memory__conversation-archive-list-chunks`; the dispatched specialist reads each chunk in-turn and emits claims under the four-kind contract (`mention`, `task`, `preference`, `observed-relationship`); the skill hands those claims to `mcp__plugin_memory_memory__conversation-archive-derive-insights` for per-kind cypher emission, then runs the per-row operator gate (`wire / skip / reject`). Idempotent on `(elementId(chunk), kind, contentHash)` — re-runs collapse identical claims. Confidence floor is a hedging-avoidance instruction the skill embeds in the specialist's per-chunk prompt, not a numeric post-filter; the LLM step runs in-turn from the dispatched specialist rather than as a server-side OAuth round-trip. | Database operator |
|
|
52
52
|
|