getmnemo-migrate 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Proofly, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,111 @@
+# Mnemo Migrate
+
+Resumable CLI for importing historical documents and memories into [Mnemo](https://mnemohq.com). It accepts common customer exports without giving Mnemo credentials to the source system.
+
+## Install
+
+```bash
+npm install -g getmnemo-migrate
+```
+
+## Quickstart
+
+```bash
+export GETMNEMO_API_KEY=mn_...
+export GETMNEMO_WORKSPACE_ID=...
+
+# Inspect a HubSpot-style CSV export. This makes no API calls.
+getmnemo-migrate plan from raw-csv \
+  --file ./contacts.csv \
+  --map 'content=*,id=Record ID'
+
+# Submit it to the isolated pilot container and wait for ingestion.
+getmnemo-migrate from raw-csv \
+  --file ./contacts.csv \
+  --map 'content=*,id=Record ID' \
+  --container pilot:customer-name
+```
+
+The `content=*` mapping serializes each complete CSV or JSON record. For cleaner source data, map a specific field instead, such as `content=notes,id=record_id,company=company`.
+
+## Import Lanes
+
+| Lane | Default sources | Behavior |
+| --- | --- | --- |
+| `document` | CSV, JSON, JSONL, Markdown, text, directories | Sends source material through Mnemo document ingestion so facts, provenance, entities, and time can be extracted. |
+| `memory` | Mem0, Zep, Supermemory, Letta | Preserves already-extracted memory text as atomic memories. |
+
+Override the default only when the source semantics require it with `--lane memory` or `--lane document`.
+
+## File Sources
+
+| Source | Required option | Notes |
+| --- | --- | --- |
+| `raw-csv` | `--file export.csv` | RFC-compatible CSV parsing, including quoted multiline fields. |
+| `raw-json` | `--file export.json` | Top-level arrays up to 50 MB. Convert larger files to JSONL. |
+| `raw-jsonl` | `--file export.jsonl` | Streams one JSON object per line. |
+| `text-file` | `--file notes.md` | Imports one Markdown or text document. |
+| `directory` | `--file ./export` | Recursively imports supported files in deterministic order; ignores symlinks and unsupported files. |
+
+Mappings use `target=source` pairs and support dotted source paths:
+
+```bash
+getmnemo-migrate from raw-json \
+  --file ./history.json \
+  --map 'content=payload.body,id=record.id,customer=account.id' \
+  --container pilot:customer-name
+```
+
+## Provider Sources
+
+| Source | Source credentials | Useful options |
+| --- | --- | --- |
+| `mem0` | `MEM0_API_KEY` | `--user`, `--base-url` |
+| `zep` | `ZEP_API_KEY` | `--user` is required, `--base-url` |
+| `supermemory` | `SUPERMEMORY_API_KEY` | `--base-url` |
+| `letta` | `LETTA_API_KEY`, `LETTA_BASE_URL` | Imports core and archival memory |
+
+## Safe Operating Flow
+
+1. Export data from the source system.
+2. Run `plan from ...` and inspect the count plus sample. No Mnemo credentials are needed for planning.
+3. Import into a customer-specific container with conservative concurrency.
+4. Keep the printed migration job ID.
+5. Run `status <jobId>` or `reconcile <jobId>` to obtain the final completion report.
+6. Validate retrieval against a small set of customer-approved questions before connecting an agent.
+
+```bash
+getmnemo-migrate status
+getmnemo-migrate cancel mig_...
+getmnemo-migrate resume mig_...
+getmnemo-migrate reconcile mig_...
+```
+
+Cancellation pauses after the current bounded batch. Resume skips successful source records and retries unresolved ones. Mnemo custom IDs and idempotency keys also protect against duplicate writes if local state is lost.
+
+Job state, processed IDs, remote job IDs, and failure journals are stored under `~/.getmnemo/migrations` with owner-only permissions. Set `GETMNEMO_MIGRATION_DIR` to use another state directory.
+
+## Reliability Limits
+
+- Concurrency is capped at 25 even if a larger value is requested.
+- HTTP 429 and server/network failures retry up to five times with backoff.
+- Documents larger than the API limit are split deterministically near line boundaries.
+- Atomic memories longer than the memory endpoint limit are split with part metadata.
+- JSON arrays are capped at 50 MB to avoid loading unbounded files into memory; JSONL and CSV stream.
+- The completion report distinguishes accepted writes from completed and failed ingestion jobs.
+
+## Environment
+
+| Variable | Purpose |
+| --- | --- |
+| `GETMNEMO_API_KEY` | Destination API key. Required for writes and reconciliation. |
+| `GETMNEMO_WORKSPACE_ID` | Destination workspace. Required for writes and reconciliation. |
+| `GETMNEMO_API_URL` | API base URL. Defaults to `https://api.mnemohq.com`. |
+| `GETMNEMO_CONTAINER_TAG` | Optional default destination container. |
+| `GETMNEMO_MIGRATION_DIR` | Optional local checkpoint directory. |
+
+See [Pilot integration contract](./docs/pilot-integration.md) for the boundary between source exports, Mnemo, governance, and agents.
+
+## License
+
+[MIT](./LICENSE)

package/dist/adapters/base.d.ts

@@ -0,0 +1,34 @@
+export type ImportLane = "memory" | "document";
+export interface SourceRecord {
+    /** Provider-native id; carried into Mnemo metadata for traceability. */
+    sourceId?: string;
+    /** Free-form text content to be stored as a memory. */
+    content: string;
+    /** Optional metadata to merge into the Mnemo memory. */
+    metadata?: Record<string, unknown>;
+    /** Optional source-provided content classification. */
+    contentType?: string;
+}
+export interface AdapterConfig {
+    /** Optional path to a JSONL file (used by raw-jsonl adapter). */
+    file?: string;
+    /** Optional mapping like `content=text,id=mem_id`. */
+    map?: string;
+    /** Optional explicit user/agent id (some providers require it). */
+    user?: string;
+    /** Optional override for the provider base URL. */
+    baseUrl?: string;
+    /** Page size hint for paginated providers. */
+    pageSize?: number;
+}
+export declare abstract class Adapter {
+    protected readonly cfg: AdapterConfig;
+    abstract readonly name: string;
+    abstract readonly defaultLane: ImportLane;
+    constructor(cfg?: AdapterConfig);
+    /** Best-effort total record count. Return null if unknown. */
+    abstract count(): Promise<number | null>;
+    /** Async iterator yielding source records. */
+    abstract iterate(): AsyncIterable<SourceRecord>;
+}
+//# sourceMappingURL=base.d.ts.map

package/dist/adapters/base.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"base.d.ts","sourceRoot":"","sources":["../../src/adapters/base.ts"],"names":[],"mappings":"AAAA,MAAM,MAAM,UAAU,GAAG,QAAQ,GAAG,UAAU,CAAC;AAE/C,MAAM,WAAW,YAAY;IAC3B,wEAAwE;IACxE,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,uDAAuD;IACvD,OAAO,EAAE,MAAM,CAAC;IAChB,wDAAwD;IACxD,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACnC,uDAAuD;IACvD,WAAW,CAAC,EAAE,MAAM,CAAC;CACtB;AAED,MAAM,WAAW,aAAa;IAC5B,iEAAiE;IACjE,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,sDAAsD;IACtD,GAAG,CAAC,EAAE,MAAM,CAAC;IACb,mEAAmE;IACnE,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,mDAAmD;IACnD,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,8CAA8C;IAC9C,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB;AAED,8BAAsB,OAAO;IAIf,SAAS,CAAC,QAAQ,CAAC,GAAG,EAAE,aAAa;IAHjD,QAAQ,CAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAC/B,QAAQ,CAAC,QAAQ,CAAC,WAAW,EAAE,UAAU,CAAC;gBAEX,GAAG,GAAE,aAAkB;IAEtD,8DAA8D;IAC9D,QAAQ,CAAC,KAAK,IAAI,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC;IAExC,8CAA8C;IAC9C,QAAQ,CAAC,OAAO,IAAI,aAAa,CAAC,YAAY,CAAC;CAChD"}

package/dist/adapters/base.js

@@ -0,0 +1,7 @@
+export class Adapter {
+    cfg;
+    constructor(cfg = {}) {
+        this.cfg = cfg;
+    }
+}
+//# sourceMappingURL=base.js.map

package/dist/adapters/base.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"base.js","sourceRoot":"","sources":["../../src/adapters/base.ts"],"names":[],"mappings":"AA0BA,MAAM,OAAgB,OAAO;IAII;IAA/B,YAA+B,MAAqB,EAAE;QAAvB,QAAG,GAAH,GAAG,CAAoB;IAAG,CAAC;CAO3D"}

package/dist/adapters/directory.d.ts

@@ -0,0 +1,9 @@
+import { Adapter, AdapterConfig, SourceRecord } from "./base.js";
+export declare class DirectoryAdapter extends Adapter {
+    readonly name = "directory";
+    readonly defaultLane: "document";
+    constructor(cfg?: AdapterConfig);
+    count(): Promise<number | null>;
+    iterate(): AsyncIterable<SourceRecord>;
+}
+//# sourceMappingURL=directory.d.ts.map

package/dist/adapters/directory.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"directory.d.ts","sourceRoot":"","sources":["../../src/adapters/directory.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,MAAM,WAAW,CAAC;AAoCjE,qBAAa,gBAAiB,SAAQ,OAAO;IAC3C,QAAQ,CAAC,IAAI,eAAe;IAC5B,QAAQ,CAAC,WAAW,EAAG,UAAU,CAAU;gBAE/B,GAAG,GAAE,aAAkB;IAK7B,KAAK,IAAI,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC;IAU9B,OAAO,IAAI,aAAa,CAAC,YAAY,CAAC;CAa9C"}

package/dist/adapters/directory.js

@@ -0,0 +1,69 @@
+import { promises as fs } from "node:fs";
+import { extname, relative, resolve } from "node:path";
+import { Adapter } from "./base.js";
+import { RawCsvAdapter } from "./raw-csv.js";
+import { RawJsonAdapter } from "./raw-json.js";
+import { RawJsonlAdapter } from "./raw-jsonl.js";
+import { TextFileAdapter } from "./text-file.js";
+const SUPPORTED_EXTENSIONS = new Set([".csv", ".json", ".jsonl", ".md", ".txt"]);
+async function listFiles(root) {
+    const entries = await fs.readdir(root, { withFileTypes: true });
+    const files = [];
+    for (const entry of entries.sort((a, b) => a.name.localeCompare(b.name))) {
+        const path = resolve(root, entry.name);
+        if (entry.isDirectory())
+            files.push(...(await listFiles(path)));
+        if (entry.isFile() && SUPPORTED_EXTENSIONS.has(extname(entry.name).toLowerCase()))
+            files.push(path);
+    }
+    return files;
+}
+function adapterFor(file, cfg) {
+    const childConfig = { ...cfg, file };
+    switch (extname(file).toLowerCase()) {
+        case ".csv":
+            return new RawCsvAdapter(childConfig);
+        case ".json":
+            return new RawJsonAdapter(childConfig);
+        case ".jsonl":
+            return new RawJsonlAdapter(childConfig);
+        case ".md":
+        case ".txt":
+            return new TextFileAdapter(childConfig);
+        default:
+            throw new Error(`Unsupported import file: ${file}`);
+    }
+}
+export class DirectoryAdapter extends Adapter {
+    name = "directory";
+    defaultLane = "document";
+    constructor(cfg = {}) {
+        super(cfg);
+        if (!cfg.file)
+            throw new Error("--file <directory> is required for the directory adapter");
+    }
+    async count() {
+        let total = 0;
+        for (const file of await listFiles(this.cfg.file)) {
+            const count = await adapterFor(file, this.cfg).count();
+            if (count === null)
+                return null;
+            total += count;
+        }
+        return total;
+    }
+    async *iterate() {
+        const root = resolve(this.cfg.file);
+        for (const file of await listFiles(root)) {
+            const relativePath = relative(root, file);
+            for await (const record of adapterFor(file, this.cfg).iterate()) {
+                yield {
+                    ...record,
+                    sourceId: `${relativePath}:${record.sourceId ?? "document"}`,
+                    metadata: { ...record.metadata, importPath: relativePath },
+                };
+            }
+        }
+    }
+}
+//# sourceMappingURL=directory.js.map

package/dist/adapters/directory.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"directory.js","sourceRoot":"","sources":["../../src/adapters/directory.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,IAAI,EAAE,EAAE,MAAM,SAAS,CAAC;AACzC,OAAO,EAAE,OAAO,EAAE,QAAQ,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AACvD,OAAO,EAAE,OAAO,EAA+B,MAAM,WAAW,CAAC;AACjE,OAAO,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AAC7C,OAAO,EAAE,cAAc,EAAE,MAAM,eAAe,CAAC;AAC/C,OAAO,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAC;AACjD,OAAO,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAC;AAEjD,MAAM,oBAAoB,GAAG,IAAI,GAAG,CAAC,CAAC,MAAM,EAAE,OAAO,EAAE,QAAQ,EAAE,KAAK,EAAE,MAAM,CAAC,CAAC,CAAC;AAEjF,KAAK,UAAU,SAAS,CAAC,IAAY;IACnC,MAAM,OAAO,GAAG,MAAM,EAAE,CAAC,OAAO,CAAC,IAAI,EAAE,EAAE,aAAa,EAAE,IAAI,EAAE,CAAC,CAAC;IAChE,MAAM,KAAK,GAAa,EAAE,CAAC;IAC3B,KAAK,MAAM,KAAK,IAAI,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,EAAE,CAAC;QACzE,MAAM,IAAI,GAAG,OAAO,CAAC,IAAI,EAAE,KAAK,CAAC,IAAI,CAAC,CAAC;QACvC,IAAI,KAAK,CAAC,WAAW,EAAE;YAAE,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,MAAM,SAAS,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QAChE,IAAI,KAAK,CAAC,MAAM,EAAE,IAAI,oBAAoB,CAAC,GAAG,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,WAAW,EAAE,CAAC;YAAE,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACtG,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAS,UAAU,CAAC,IAAY,EAAE,GAAkB;IAClD,MAAM,WAAW,GAAG,EAAE,GAAG,GAAG,EAAE,IAAI,EAAE,CAAC;IACrC,QAAQ,OAAO,CAAC,IAAI,CAAC,CAAC,WAAW,EAAE,EAAE,CAAC;QACpC,KAAK,MAAM;YACT,OAAO,IAAI,aAAa,CAAC,WAAW,CAAC,CAAC;QACxC,KAAK,OAAO;YACV,OAAO,IAAI,cAAc,CAAC,WAAW,CAAC,CAAC;QACzC,KAAK,QAAQ;YACX,OAAO,IAAI,eAAe,CAAC,WAAW,CAAC,CAAC;QAC1C,KAAK,KAAK,CAAC;QACX,KAAK,MAAM;YACT,OAAO,IAAI,eAAe,CAAC,WAAW,CAAC,CAAC;QAC1C;YACE,MAAM,IAAI,KAAK,CAAC,4BAA4B,IAAI,EAAE,CAAC,CAAC;IACxD,CAAC;AACH,CAAC;AAED,MAAM,OAAO,gBAAiB,SAAQ,OAAO;IAClC,IAAI,GAAG,WAAW,CAAC;IACnB,WAAW,GAAG,UAAmB,CAAC;IAE3C,YAAY,MAAqB,EAAE;QACjC,KAAK,CAAC,GAAG,CAAC,CAAC;QACX,IAAI,CAAC,GAAG,CAAC,IAAI;YAAE,MAAM,IAAI,KAAK,CAAC,0DAA0D,CAAC,CAAC;IAC7F,CAAC;IAED,KAAK,CAAC,KAAK;QACT,IAAI,KAAK,GAAG,CAAC,CAAC;QACd,KAAK,MAAM,IAAI,IAAI,MAAM,SAAS,CAAC,IAAI,CAAC,GAAG,CAAC,IAAK,CAAC,EAAE,CAAC;YACnD,MAAM,KAAK,GAAG,MAAM,UAAU,CAAC,IAAI,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,CAAC;YACvD,IAAI,KAAK,KAAK,IAAI;gBAAE,OAAO,IAAI,CAAC;YAChC,KAAK,IAAI,KAAK,CAAC;QACjB,CAAC;QACD,OAAO,KAAK,CAAC;IACf,CAAC;IAED,KAAK,CAAC,CAAC,OAAO;QACZ,MAAM,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,GAAG,CAAC,IAAK,CAAC,CAAC;QACrC,KAAK,MAAM,IAAI,IAAI,MAAM,SAAS,CAAC,IAAI,CAAC,EAAE,CAAC;YACzC,MAAM,YAAY,GAAG,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;YAC1C,IAAI,KAAK,EAAE,MAAM,MAAM,IAAI,UAAU,CAAC,IAAI,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC;gBAChE,MAAM;oBACJ,GAAG,MAAM;oBACT,QAAQ,EAAE,GAAG,YAAY,IAAI,MAAM,CAAC,QAAQ,IAAI,UAAU,EAAE;oBAC5D,QAAQ,EAAE,EAAE,GAAG,MAAM,CAAC,QAAQ,EAAE,UAAU,EAAE,YAAY,EAAE;iBAC3D,CAAC;YACJ,CAAC;QACH,CAAC;IACH,CAAC;CACF"}

package/dist/adapters/index.d.ts

@@ -0,0 +1,7 @@
+import { Adapter, AdapterConfig } from "./base.js";
+export type AdapterName = "mem0" | "zep" | "supermemory" | "letta" | "raw-jsonl" | "raw-json" | "raw-csv" | "text-file" | "directory";
+export declare const ADAPTER_NAMES: AdapterName[];
+export declare function createAdapter(name: string, cfg?: AdapterConfig): Adapter;
+export { Adapter };
+export type { AdapterConfig, ImportLane, SourceRecord } from "./base.js";
+//# sourceMappingURL=index.d.ts.map

package/dist/adapters/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/adapters/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,aAAa,EAAE,MAAM,WAAW,CAAC;AAWnD,MAAM,MAAM,WAAW,GACnB,MAAM,GACN,KAAK,GACL,aAAa,GACb,OAAO,GACP,WAAW,GACX,UAAU,GACV,SAAS,GACT,WAAW,GACX,WAAW,CAAC;AAEhB,eAAO,MAAM,aAAa,EAAE,WAAW,EAUtC,CAAC;AAEF,wBAAgB,aAAa,CAAC,IAAI,EAAE,MAAM,EAAE,GAAG,GAAE,aAAkB,GAAG,OAAO,CAuB5E;AAED,OAAO,EAAE,OAAO,EAAE,CAAC;AACnB,YAAY,EAAE,aAAa,EAAE,UAAU,EAAE,YAAY,EAAE,MAAM,WAAW,CAAC"}

package/dist/adapters/index.js

@@ -0,0 +1,47 @@
+import { Adapter } from "./base.js";
+import { Mem0Adapter } from "./mem0.js";
+import { ZepAdapter } from "./zep.js";
+import { SupermemoryAdapter } from "./supermemory.js";
+import { LettaAdapter } from "./letta.js";
+import { RawJsonlAdapter } from "./raw-jsonl.js";
+import { RawJsonAdapter } from "./raw-json.js";
+import { RawCsvAdapter } from "./raw-csv.js";
+import { TextFileAdapter } from "./text-file.js";
+import { DirectoryAdapter } from "./directory.js";
+export const ADAPTER_NAMES = [
+    "mem0",
+    "zep",
+    "supermemory",
+    "letta",
+    "raw-jsonl",
+    "raw-json",
+    "raw-csv",
+    "text-file",
+    "directory",
+];
+export function createAdapter(name, cfg = {}) {
+    switch (name) {
+        case "mem0":
+            return new Mem0Adapter(cfg);
+        case "zep":
+            return new ZepAdapter(cfg);
+        case "supermemory":
+            return new SupermemoryAdapter(cfg);
+        case "letta":
+            return new LettaAdapter(cfg);
+        case "raw-jsonl":
+            return new RawJsonlAdapter(cfg);
+        case "raw-json":
+            return new RawJsonAdapter(cfg);
+        case "raw-csv":
+            return new RawCsvAdapter(cfg);
+        case "text-file":
+            return new TextFileAdapter(cfg);
+        case "directory":
+            return new DirectoryAdapter(cfg);
+        default:
+            throw new Error(`Unknown adapter "${name}". Valid: ${ADAPTER_NAMES.join(", ")}`);
+    }
+}
+export { Adapter };
+//# sourceMappingURL=index.js.map

package/dist/adapters/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/adapters/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAiB,MAAM,WAAW,CAAC;AACnD,OAAO,EAAE,WAAW,EAAE,MAAM,WAAW,CAAC;AACxC,OAAO,EAAE,UAAU,EAAE,MAAM,UAAU,CAAC;AACtC,OAAO,EAAE,kBAAkB,EAAE,MAAM,kBAAkB,CAAC;AACtD,OAAO,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAC1C,OAAO,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAC;AACjD,OAAO,EAAE,cAAc,EAAE,MAAM,eAAe,CAAC;AAC/C,OAAO,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AAC7C,OAAO,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAC;AACjD,OAAO,EAAE,gBAAgB,EAAE,MAAM,gBAAgB,CAAC;AAalD,MAAM,CAAC,MAAM,aAAa,GAAkB;IAC1C,MAAM;IACN,KAAK;IACL,aAAa;IACb,OAAO;IACP,WAAW;IACX,UAAU;IACV,SAAS;IACT,WAAW;IACX,WAAW;CACZ,CAAC;AAEF,MAAM,UAAU,aAAa,CAAC,IAAY,EAAE,MAAqB,EAAE;IACjE,QAAQ,IAAI,EAAE,CAAC;QACb,KAAK,MAAM;YACT,OAAO,IAAI,WAAW,CAAC,GAAG,CAAC,CAAC;QAC9B,KAAK,KAAK;YACR,OAAO,IAAI,UAAU,CAAC,GAAG,CAAC,CAAC;QAC7B,KAAK,aAAa;YAChB,OAAO,IAAI,kBAAkB,CAAC,GAAG,CAAC,CAAC;QACrC,KAAK,OAAO;YACV,OAAO,IAAI,YAAY,CAAC,GAAG,CAAC,CAAC;QAC/B,KAAK,WAAW;YACd,OAAO,IAAI,eAAe,CAAC,GAAG,CAAC,CAAC;QAClC,KAAK,UAAU;YACb,OAAO,IAAI,cAAc,CAAC,GAAG,CAAC,CAAC;QACjC,KAAK,SAAS;YACZ,OAAO,IAAI,aAAa,CAAC,GAAG,CAAC,CAAC;QAChC,KAAK,WAAW;YACd,OAAO,IAAI,eAAe,CAAC,GAAG,CAAC,CAAC;QAClC,KAAK,WAAW;YACd,OAAO,IAAI,gBAAgB,CAAC,GAAG,CAAC,CAAC;QACnC;YACE,MAAM,IAAI,KAAK,CAAC,oBAAoB,IAAI,aAAa,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACrF,CAAC;AACH,CAAC;AAED,OAAO,EAAE,OAAO,EAAE,CAAC"}

package/dist/adapters/letta.d.ts

@@ -0,0 +1,12 @@
+import { Adapter, AdapterConfig, SourceRecord } from "./base.js";
+export declare class LettaAdapter extends Adapter {
+    readonly name = "letta";
+    readonly defaultLane: "memory";
+    private apiKey;
+    private baseUrl;
+    constructor(cfg?: AdapterConfig);
+    private request;
+    count(): Promise<number | null>;
+    iterate(): AsyncIterable<SourceRecord>;
+}
+//# sourceMappingURL=letta.d.ts.map

package/dist/adapters/letta.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"letta.d.ts","sourceRoot":"","sources":["../../src/adapters/letta.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,MAAM,WAAW,CAAC;AAuBjE,qBAAa,YAAa,SAAQ,OAAO;IACvC,QAAQ,CAAC,IAAI,WAAW;IACxB,QAAQ,CAAC,WAAW,EAAG,QAAQ,CAAU;IAEzC,OAAO,CAAC,MAAM,CAAS;IACvB,OAAO,CAAC,OAAO,CAAS;gBAEZ,GAAG,GAAE,aAAkB;YAcrB,OAAO;IAaf,KAAK,IAAI,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC;IAI9B,OAAO,IAAI,aAAa,CAAC,YAAY,CAAC;CAsD9C"}

package/dist/adapters/letta.js

@@ -0,0 +1,91 @@
+import { Adapter } from "./base.js";
+export class LettaAdapter extends Adapter {
+    name = "letta";
+    defaultLane = "memory";
+    apiKey;
+    baseUrl;
+    constructor(cfg = {}) {
+        super(cfg);
+        const key = process.env.LETTA_API_KEY;
+        const baseUrl = process.env.LETTA_BASE_URL ?? cfg.baseUrl;
+        if (!key) {
+            throw new Error("LETTA_API_KEY is required for the letta adapter");
+        }
+        if (!baseUrl) {
+            throw new Error("LETTA_BASE_URL is required for the letta adapter");
+        }
+        this.apiKey = key;
+        this.baseUrl = baseUrl.replace(/\/$/, "");
+    }
+    async request(path) {
+        const res = await fetch(new URL(path, this.baseUrl + "/"), {
+            headers: {
+                Authorization: `Bearer ${this.apiKey}`,
+                Accept: "application/json",
+            },
+        });
+        if (!res.ok) {
+            throw new Error(`letta request failed: ${res.status} ${res.statusText}`);
+        }
+        return (await res.json());
+    }
+    async count() {
+        return null;
+    }
+    async *iterate() {
+        const agents = await this.request("/v1/agents/");
+        for (const agent of agents) {
+            const agentId = agent.id;
+            if (!agentId)
+                continue;
+            // Core memory blocks
+            for (const block of agent.memory?.blocks ?? []) {
+                if (!block.value)
+                    continue;
+                yield {
+                    sourceId: block.id,
+                    content: block.value,
+                    metadata: {
+                        source: "letta",
+                        kind: "core_block",
+                        agentId,
+                        ...(agent.name ? { agentName: agent.name } : {}),
+                        ...(block.label ? { label: block.label } : {}),
+                        ...(block.created_at ? { sourceCreatedAt: block.created_at } : {}),
+                        ...(block.metadata ?? {}),
+                    },
+                };
+            }
+            // Archival memory (paginated by `after` cursor)
+            const pageSize = this.cfg.pageSize ?? 100;
+            let after;
+            while (true) {
+                const path = `/v1/agents/${encodeURIComponent(agentId)}/archival-memory?limit=${pageSize}${after ? `&after=${encodeURIComponent(after)}` : ""}`;
+                const page = await this.request(path);
+                if (!page || page.length === 0)
+                    break;
+                for (const m of page) {
+                    if (!m.text)
+                        continue;
+                    yield {
+                        sourceId: m.id,
+                        content: m.text,
+                        metadata: {
+                            source: "letta",
+                            kind: "archival",
+                            agentId,
+                            ...(agent.name ? { agentName: agent.name } : {}),
+                            ...(m.created_at ? { sourceCreatedAt: m.created_at } : {}),
+                            ...(m.metadata ?? {}),
+                        },
+                    };
+                }
+                const last = page[page.length - 1];
+                if (!last?.id || page.length < pageSize)
+                    break;
+                after = last.id;
+            }
+        }
+    }
+}
+//# sourceMappingURL=letta.js.map

package/dist/adapters/letta.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"letta.js","sourceRoot":"","sources":["../../src/adapters/letta.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAA+B,MAAM,WAAW,CAAC;AAuBjE,MAAM,OAAO,YAAa,SAAQ,OAAO;IAC9B,IAAI,GAAG,OAAO,CAAC;IACf,WAAW,GAAG,QAAiB,CAAC;IAEjC,MAAM,CAAS;IACf,OAAO,CAAS;IAExB,YAAY,MAAqB,EAAE;QACjC,KAAK,CAAC,GAAG,CAAC,CAAC;QACX,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,CAAC,aAAa,CAAC;QACtC,MAAM,OAAO,GAAG,OAAO,CAAC,GAAG,CAAC,cAAc,IAAI,GAAG,CAAC,OAAO,CAAC;QAC1D,IAAI,CAAC,GAAG,EAAE,CAAC;YACT,MAAM,IAAI,KAAK,CAAC,iDAAiD,CAAC,CAAC;QACrE,CAAC;QACD,IAAI,CAAC,OAAO,EAAE,CAAC;YACb,MAAM,IAAI,KAAK,CAAC,kDAAkD,CAAC,CAAC;QACtE,CAAC;QACD,IAAI,CAAC,MAAM,GAAG,GAAG,CAAC;QAClB,IAAI,CAAC,OAAO,GAAG,OAAO,CAAC,OAAO,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC;IAC5C,CAAC;IAEO,KAAK,CAAC,OAAO,CAAI,IAAY;QACnC,MAAM,GAAG,GAAG,MAAM,KAAK,CAAC,IAAI,GAAG,CAAC,IAAI,EAAE,IAAI,CAAC,OAAO,GAAG,GAAG,CAAC,EAAE;YACzD,OAAO,EAAE;gBACP,aAAa,EAAE,UAAU,IAAI,CAAC,MAAM,EAAE;gBACtC,MAAM,EAAE,kBAAkB;aAC3B;SACF,CAAC,CAAC;QACH,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,CAAC;YACZ,MAAM,IAAI,KAAK,CAAC,yBAAyB,GAAG,CAAC,MAAM,IAAI,GAAG,CAAC,UAAU,EAAE,CAAC,CAAC;QAC3E,CAAC;QACD,OAAO,CAAC,MAAM,GAAG,CAAC,IAAI,EAAE,CAAM,CAAC;IACjC,CAAC;IAED,KAAK,CAAC,KAAK;QACT,OAAO,IAAI,CAAC;IACd,CAAC;IAED,KAAK,CAAC,CAAC,OAAO;QACZ,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,OAAO,CAAe,aAAa,CAAC,CAAC;QAC/D,KAAK,MAAM,KAAK,IAAI,MAAM,EAAE,CAAC;YAC3B,MAAM,OAAO,GAAG,KAAK,CAAC,EAAE,CAAC;YACzB,IAAI,CAAC,OAAO;gBAAE,SAAS;YAEvB,qBAAqB;YACrB,KAAK,MAAM,KAAK,IAAI,KAAK,CAAC,MAAM,EAAE,MAAM,IAAI,EAAE,EAAE,CAAC;gBAC/C,IAAI,CAAC,KAAK,CAAC,KAAK;oBAAE,SAAS;gBAC3B,MAAM;oBACJ,QAAQ,EAAE,KAAK,CAAC,EAAE;oBAClB,OAAO,EAAE,KAAK,CAAC,KAAK;oBACpB,QAAQ,EAAE;wBACR,MAAM,EAAE,OAAO;wBACf,IAAI,EAAE,YAAY;wBAClB,OAAO;wBACP,GAAG,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,SAAS,EAAE,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;wBAChD,GAAG,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,KAAK,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;wBAC9C,GAAG,CAAC,KAAK,CAAC,UAAU,CAAC,CAAC,CAAC,EAAE,eAAe,EAAE,KAAK,CAAC,UAAU,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;wBAClE,GAAG,CAAC,KAAK,CAAC,QAAQ,IAAI,EAAE,CAAC;qBAC1B;iBACF,CAAC;YACJ,CAAC;YAED,gDAAgD;YAChD,MAAM,QAAQ,GAAG,IAAI,CAAC,GAAG,CAAC,QAAQ,IAAI,GAAG,CAAC;YAC1C,IAAI,KAAyB,CAAC;YAC9B,OAAO,IAAI,EAAE,CAAC;gBACZ,MAAM,IAAI,GAAG,cAAc,kBAAkB,CAAC,OAAO,CAAC,0BAA0B,QAAQ,GACtF,KAAK,CAAC,CAAC,CAAC,UAAU,kBAAkB,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC,CAAC,EAClD,EAAE,CAAC;gBACH,MAAM,IAAI,GAAG,MAAM,IAAI,CAAC,OAAO,CAAwB,IAAI,CAAC,CAAC;gBAC7D,IAAI,CAAC,IAAI,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC;oBAAE,MAAM;gBACtC,KAAK,MAAM,CAAC,IAAI,IAAI,EAAE,CAAC;oBACrB,IAAI,CAAC,CAAC,CAAC,IAAI;wBAAE,SAAS;oBACtB,MAAM;wBACJ,QAAQ,EAAE,CAAC,CAAC,EAAE;wBACd,OAAO,EAAE,CAAC,CAAC,IAAI;wBACf,QAAQ,EAAE;4BACR,MAAM,EAAE,OAAO;4BACf,IAAI,EAAE,UAAU;4BAChB,OAAO;4BACP,GAAG,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,SAAS,EAAE,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;4BAChD,GAAG,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,EAAE,eAAe,EAAE,CAAC,CAAC,UAAU,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;4BAC1D,GAAG,CAAC,CAAC,CAAC,QAAQ,IAAI,EAAE,CAAC;yBACtB;qBACF,CAAC;gBACJ,CAAC;gBACD,MAAM,IAAI,GAAG,IAAI,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;gBACnC,IAAI,CAAC,IAAI,EAAE,EAAE,IAAI,IAAI,CAAC,MAAM,GAAG,QAAQ;oBAAE,MAAM;gBAC/C,KAAK,GAAG,IAAI,CAAC,EAAE,CAAC;YAClB,CAAC;QACH,CAAC;IACH,CAAC;CACF"}

package/dist/adapters/mem0.d.ts

@@ -0,0 +1,12 @@
+import { Adapter, AdapterConfig, SourceRecord } from "./base.js";
+export declare class Mem0Adapter extends Adapter {
+    readonly name = "mem0";
+    readonly defaultLane: "memory";
+    private apiKey;
+    private baseUrl;
+    constructor(cfg?: AdapterConfig);
+    private request;
+    count(): Promise<number | null>;
+    iterate(): AsyncIterable<SourceRecord>;
+}
+//# sourceMappingURL=mem0.d.ts.map

package/dist/adapters/mem0.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"mem0.d.ts","sourceRoot":"","sources":["../../src/adapters/mem0.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,MAAM,WAAW,CAAC;AAoBjE,qBAAa,WAAY,SAAQ,OAAO;IACtC,QAAQ,CAAC,IAAI,UAAU;IACvB,QAAQ,CAAC,WAAW,EAAG,QAAQ,CAAU;IAEzC,OAAO,CAAC,MAAM,CAAS;IACvB,OAAO,CAAC,OAAO,CAAS;gBAEZ,GAAG,GAAE,aAAkB;YAUrB,OAAO;IAaf,KAAK,IAAI,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC;IAO9B,OAAO,IAAI,aAAa,CAAC,YAAY,CAAC;CA0B9C"}

package/dist/adapters/mem0.js

@@ -0,0 +1,62 @@
+import { Adapter } from "./base.js";
+const DEFAULT_BASE_URL = "https://api.mem0.ai";
+export class Mem0Adapter extends Adapter {
+    name = "mem0";
+    defaultLane = "memory";
+    apiKey;
+    baseUrl;
+    constructor(cfg = {}) {
+        super(cfg);
+        const key = process.env.MEM0_API_KEY;
+        if (!key) {
+            throw new Error("MEM0_API_KEY is required for the mem0 adapter");
+        }
+        this.apiKey = key;
+        this.baseUrl = (cfg.baseUrl ?? DEFAULT_BASE_URL).replace(/\/$/, "");
+    }
+    async request(path) {
+        const res = await fetch(new URL(path, this.baseUrl + "/"), {
+            headers: {
+                Authorization: `Token ${this.apiKey}`,
+                Accept: "application/json",
+            },
+        });
+        if (!res.ok) {
+            throw new Error(`mem0 request failed: ${res.status} ${res.statusText}`);
+        }
+        return (await res.json());
+    }
+    async count() {
+        const params = new URLSearchParams({ page: "1", page_size: "1" });
+        if (this.cfg.user)
+            params.set("user_id", this.cfg.user);
+        const page = await this.request(`/v1/memories/?${params.toString()}`);
+        return typeof page.count === "number" ? page.count : null;
+    }
+    async *iterate() {
+        const pageSize = this.cfg.pageSize ?? 100;
+        let nextUrl = `/v1/memories/?page_size=${pageSize}${this.cfg.user ? `&user_id=${encodeURIComponent(this.cfg.user)}` : ""}`;
+        while (nextUrl) {
+            const page = await this.request(nextUrl);
+            const results = page.results ?? [];
+            for (const m of results) {
+                const content = m.memory ?? m.text;
+                if (!content)
+                    continue;
+                yield {
+                    sourceId: m.id,
+                    content,
+                    metadata: {
+                        source: "mem0",
+                        ...(m.user_id ? { userId: m.user_id } : {}),
+                        ...(m.agent_id ? { agentId: m.agent_id } : {}),
+                        ...(m.created_at ? { sourceCreatedAt: m.created_at } : {}),
+                        ...(m.metadata ?? {}),
+                    },
+                };
+            }
+            nextUrl = page.next ? page.next.replace(this.baseUrl, "") : null;
+        }
+    }
+}
+//# sourceMappingURL=mem0.js.map

package/dist/adapters/mem0.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"mem0.js","sourceRoot":"","sources":["../../src/adapters/mem0.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAA+B,MAAM,WAAW,CAAC;AAkBjE,MAAM,gBAAgB,GAAG,qBAAqB,CAAC;AAE/C,MAAM,OAAO,WAAY,SAAQ,OAAO;IAC7B,IAAI,GAAG,MAAM,CAAC;IACd,WAAW,GAAG,QAAiB,CAAC;IAEjC,MAAM,CAAS;IACf,OAAO,CAAS;IAExB,YAAY,MAAqB,EAAE;QACjC,KAAK,CAAC,GAAG,CAAC,CAAC;QACX,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,CAAC,YAAY,CAAC;QACrC,IAAI,CAAC,GAAG,EAAE,CAAC;YACT,MAAM,IAAI,KAAK,CAAC,+CAA+C,CAAC,CAAC;QACnE,CAAC;QACD,IAAI,CAAC,MAAM,GAAG,GAAG,CAAC;QAClB,IAAI,CAAC,OAAO,GAAG,CAAC,GAAG,CAAC,OAAO,IAAI,gBAAgB,CAAC,CAAC,OAAO,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC;IACtE,CAAC;IAEO,KAAK,CAAC,OAAO,CAAC,IAAY;QAChC,MAAM,GAAG,GAAG,MAAM,KAAK,CAAC,IAAI,GAAG,CAAC,IAAI,EAAE,IAAI,CAAC,OAAO,GAAG,GAAG,CAAC,EAAE;YACzD,OAAO,EAAE;gBACP,aAAa,EAAE,SAAS,IAAI,CAAC,MAAM,EAAE;gBACrC,MAAM,EAAE,kBAAkB;aAC3B;SACF,CAAC,CAAC;QACH,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,CAAC;YACZ,MAAM,IAAI,KAAK,CAAC,wBAAwB,GAAG,CAAC,MAAM,IAAI,GAAG,CAAC,UAAU,EAAE,CAAC,CAAC;QAC1E,CAAC;QACD,OAAO,CAAC,MAAM,GAAG,CAAC,IAAI,EAAE,CAAa,CAAC;IACxC,CAAC;IAED,KAAK,CAAC,KAAK;QACT,MAAM,MAAM,GAAG,IAAI,eAAe,CAAC,EAAE,IAAI,EAAE,GAAG,EAAE,SAAS,EAAE,GAAG,EAAE,CAAC,CAAC;QAClE,IAAI,IAAI,CAAC,GAAG,CAAC,IAAI;YAAE,MAAM,CAAC,GAAG,CAAC,SAAS,EAAE,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QACxD,MAAM,IAAI,GAAG,MAAM,IAAI,CAAC,OAAO,CAAC,iBAAiB,MAAM,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC;QACtE,OAAO,OAAO,IAAI,CAAC,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC;IAC5D,CAAC;IAED,KAAK,CAAC,CAAC,OAAO;QACZ,MAAM,QAAQ,GAAG,IAAI,CAAC,GAAG,CAAC,QAAQ,IAAI,GAAG,CAAC;QAC1C,IAAI,OAAO,GAAkB,2BAA2B,QAAQ,GAC9D,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,YAAY,kBAAkB,CAAC,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,EACpE,EAAE,CAAC;QACH,OAAO,OAAO,EAAE,CAAC;YACf,MAAM,IAAI,GAAG,MAAM,IAAI,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC;YACzC,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,IAAI,EAAE,CAAC;YACnC,KAAK,MAAM,CAAC,IAAI,OAAO,EAAE,CAAC;gBACxB,MAAM,OAAO,GAAG,CAAC,CAAC,MAAM,IAAI,CAAC,CAAC,IAAI,CAAC;gBACnC,IAAI,CAAC,OAAO;oBAAE,SAAS;gBACvB,MAAM;oBACJ,QAAQ,EAAE,CAAC,CAAC,EAAE;oBACd,OAAO;oBACP,QAAQ,EAAE;wBACR,MAAM,EAAE,MAAM;wBACd,GAAG,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;wBAC3C,GAAG,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC,CAAC,QAAQ,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;wBAC9C,GAAG,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,EAAE,eAAe,EAAE,CAAC,CAAC,UAAU,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;wBAC1D,GAAG,CAAC,CAAC,CAAC,QAAQ,IAAI,EAAE,CAAC;qBACtB;iBACF,CAAC;YACJ,CAAC;YACD,OAAO,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;QACnE,CAAC;IACH,CAAC;CACF"}

package/dist/adapters/raw-csv.d.ts

@@ -0,0 +1,9 @@
+import { Adapter, AdapterConfig, SourceRecord } from "./base.js";
+export declare class RawCsvAdapter extends Adapter {
+    readonly name = "raw-csv";
+    readonly defaultLane: "document";
+    constructor(cfg?: AdapterConfig);
+    count(): Promise<number>;
+    iterate(): AsyncIterable<SourceRecord>;
+}
+//# sourceMappingURL=raw-csv.d.ts.map

package/dist/adapters/raw-csv.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"raw-csv.d.ts","sourceRoot":"","sources":["../../src/adapters/raw-csv.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,MAAM,WAAW,CAAC;AAGjE,qBAAa,aAAc,SAAQ,OAAO;IACxC,QAAQ,CAAC,IAAI,aAAa;IAC1B,QAAQ,CAAC,WAAW,EAAG,UAAU,CAAU;gBAE/B,GAAG,GAAE,aAAkB;IAK7B,KAAK,IAAI,OAAO,CAAC,MAAM,CAAC;IAMvB,OAAO,IAAI,aAAa,CAAC,YAAY,CAAC;CAc9C"}

package/dist/adapters/raw-csv.js

@@ -0,0 +1,34 @@
+import { createReadStream } from "node:fs";
+import { parse } from "csv-parse";
+import { Adapter } from "./base.js";
+import { parseMap, recordFromObject } from "./record-mapping.js";
+export class RawCsvAdapter extends Adapter {
+    name = "raw-csv";
+    defaultLane = "document";
+    constructor(cfg = {}) {
+        super(cfg);
+        if (!cfg.file)
+            throw new Error("--file <path.csv> is required for the raw-csv adapter");
+    }
+    async count() {
+        let total = 0;
+        for await (const _record of this.iterate())
+            total += 1;
+        return total;
+    }
+    async *iterate() {
+        const parser = createReadStream(this.cfg.file).pipe(parse({ bom: true, columns: true, skip_empty_lines: true, trim: true }));
+        const map = parseMap(this.cfg.map);
+        for await (const value of parser) {
+            if (!value || typeof value !== "object" || Array.isArray(value))
+                continue;
+            const record = recordFromObject(value, map, {
+                source: "raw-csv",
+                file: this.cfg.file,
+            });
+            if (record)
+                yield record;
+        }
+    }
+}
+//# sourceMappingURL=raw-csv.js.map

package/dist/adapters/raw-csv.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"raw-csv.js","sourceRoot":"","sources":["../../src/adapters/raw-csv.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,gBAAgB,EAAE,MAAM,SAAS,CAAC;AAC3C,OAAO,EAAE,KAAK,EAAE,MAAM,WAAW,CAAC;AAClC,OAAO,EAAE,OAAO,EAA+B,MAAM,WAAW,CAAC;AACjE,OAAO,EAAE,QAAQ,EAAE,gBAAgB,EAAE,MAAM,qBAAqB,CAAC;AAEjE,MAAM,OAAO,aAAc,SAAQ,OAAO;IAC/B,IAAI,GAAG,SAAS,CAAC;IACjB,WAAW,GAAG,UAAmB,CAAC;IAE3C,YAAY,MAAqB,EAAE;QACjC,KAAK,CAAC,GAAG,CAAC,CAAC;QACX,IAAI,CAAC,GAAG,CAAC,IAAI;YAAE,MAAM,IAAI,KAAK,CAAC,uDAAuD,CAAC,CAAC;IAC1F,CAAC;IAED,KAAK,CAAC,KAAK;QACT,IAAI,KAAK,GAAG,CAAC,CAAC;QACd,IAAI,KAAK,EAAE,MAAM,OAAO,IAAI,IAAI,CAAC,OAAO,EAAE;YAAE,KAAK,IAAI,CAAC,CAAC;QACvD,OAAO,KAAK,CAAC;IACf,CAAC;IAED,KAAK,CAAC,CAAC,OAAO;QACZ,MAAM,MAAM,GAAG,gBAAgB,CAAC,IAAI,CAAC,GAAG,CAAC,IAAK,CAAC,CAAC,IAAI,CAClD,KAAK,CAAC,EAAE,GAAG,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,gBAAgB,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC,CACxE,CAAC;QACF,MAAM,GAAG,GAAG,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;QACnC,IAAI,KAAK,EAAE,MAAM,KAAK,IAAI,MAAM,EAAE,CAAC;YACjC,IAAI,CAAC,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC;gBAAE,SAAS;YAC1E,MAAM,MAAM,GAAG,gBAAgB,CAAC,KAAgC,EAAE,GAAG,EAAE;gBACrE,MAAM,EAAE,SAAS;gBACjB,IAAI,EAAE,IAAI,CAAC,GAAG,CAAC,IAAK;aACrB,CAAC,CAAC;YACH,IAAI,MAAM;gBAAE,MAAM,MAAM,CAAC;QAC3B,CAAC;IACH,CAAC;CACF"}

package/dist/adapters/raw-json.d.ts

@@ -0,0 +1,10 @@
+import { Adapter, AdapterConfig, SourceRecord } from "./base.js";
+export declare class RawJsonAdapter extends Adapter {
+    readonly name = "raw-json";
+    readonly defaultLane: "document";
+    constructor(cfg?: AdapterConfig);
+    private load;
+    count(): Promise<number>;
+    iterate(): AsyncIterable<SourceRecord>;
+}
+//# sourceMappingURL=raw-json.d.ts.map

package/dist/adapters/raw-json.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"raw-json.d.ts","sourceRoot":"","sources":["../../src/adapters/raw-json.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,MAAM,WAAW,CAAC;AAKjE,qBAAa,cAAe,SAAQ,OAAO;IACzC,QAAQ,CAAC,IAAI,cAAc;IAC3B,QAAQ,CAAC,WAAW,EAAG,UAAU,CAAU;gBAE/B,GAAG,GAAE,aAAkB;YAKrB,IAAI;IAYZ,KAAK,IAAI,OAAO,CAAC,MAAM,CAAC;IAIvB,OAAO,IAAI,aAAa,CAAC,YAAY,CAAC;CAO9C"}