@inlang/sdk 2.9.2 → 2.10.0

package/src/project/README_CONTENT.ts

@@ -12,7 +12,9 @@ This is an [unpacked (git-friendly)](https://inlang.com/docs/unpacked-project) i
 ## At a glance
 
 Purpose:
-- This folder stores inlang project configuration and plugin cache data.
+- This folder is the Git-friendly representation of an \`.inlang\` project.
+- The canonical \`.inlang\` format is a single binary file; this directory is the unpacked version for Git.
+- This folder stores project configuration and plugin cache data.
 - Translation files live outside this folder and are referenced from \`settings.json\`.
 
 Safe to edit:
@@ -23,26 +25,44 @@ Do not edit:
 - \`.gitignore\`
 
 Key files:
-- \`settings.json\` — locales, plugins, file patterns (source of truth)
+- \`settings.json\` — locales, plugins, file patterns
 - \`cache/\` — plugin caches (safe to delete)
 - \`.gitignore\` — generated
+- \`README.md\` — generated, explains this folder
+- \`.meta.json\` — generated SDK metadata
 
 \`\`\`
 *.inlang/
-├── settings.json    # Locales, plugins, and file patterns (source of truth)
-├── cache/           # Plugin caches (gitignored)
-└── .gitignore       # Ignores everything except settings.json
+├── settings.json    # Locales, plugins, and file patterns; kept in Git
+├── .gitignore       # Ignores everything except settings.json
+├── README.md        # Generated, explains this folder
+├── .meta.json       # Generated SDK metadata
+└── cache/           # Plugin caches, usually cache/plugins/
 \`\`\`
 
 Translation files (like \`messages/en.json\`) live **outside** this folder and are referenced via plugins in \`settings.json\`.
 
 ## What is inlang?
 
-[Inlang](https://inlang.com) is an open file format for building custom localization (i18n) tooling. It provides:
+[Inlang](https://inlang.com) is an open project file format for localization. An \`.inlang\` project is canonically a single binary file: a SQLite database with version control via [lix](https://lix.dev). Like \`.sqlite\` for relational data, \`.inlang\` packages localization data into one file that tools can share.
+
+For Git repositories, that binary file can be unpacked into a directory of plain files. The packed file is the canonical format; this directory is the Git-friendly representation.
+
+Use inlang when multiple tools, teams, automations, or agents need to use the same localization data. The \`@inlang/sdk\` is the reference implementation for reading and writing \`.inlang\` projects.
+
+\`.inlang\` is the canonical project format. Plugins import and export external translation files for compatibility with existing runtimes and workflows. Messages, variants, and locale data live in the \`.inlang\` database; translation files such as \`messages/en.json\` live outside this folder and are connected through plugins. Version control via lix adds file-level history, merging, and change proposals to \`.inlang\` projects.
+
+It provides:
 
 - **CRUD API** — Read and write translations programmatically via SQL
-- **Plugin system** — Import/export any format (JSON, XLIFF, etc.)
-- **Version control** — Built-in version control via [lix](https://lix.dev)
+- **Plugin system** — Import/export external translation files (JSON, XLIFF, etc.)
+- **Version control** — Version control via [lix](https://lix.dev)
+
+Core data model:
+
+- **Bundle** — one translatable unit across locales
+- **Message** — locale-specific translation for a bundle
+- **Variant** — text pattern plus selector matches
 
 \`\`\`
 ┌──────────┐        ┌───────────┐         ┌────────────┐
@@ -65,15 +85,45 @@ npm install @inlang/sdk
 
 \`\`\`ts
 import { loadProjectFromDirectory, saveProjectToDirectory } from "@inlang/sdk";
+import fs from "node:fs";
 
-const project = await loadProjectFromDirectory({ path: "./project.inlang" });
-// Query messages with SQLite + [Kysely](https://kysely.dev/) under the hood.
+const project = await loadProjectFromDirectory({
+  path: "./project.inlang",
+  fs,
+});
+// Query messages with the SDK.
 const messages = await project.db.selectFrom("message").selectAll().execute();
 
 // Use project.db to update messages.
-await saveProjectToDirectory({ path: "./project.inlang", project });
+await saveProjectToDirectory({
+  path: "./project.inlang",
+  fs,
+  project,
+});
+
+await project.close();
 \`\`\`
 
+\`loadProjectFromDirectory()\` and \`saveProjectToDirectory()\` both accept \`node:fs\`. \`saveProjectToDirectory()\` writes translation files through import/export plugins. If no exporter plugin is configured, save the canonical packed file with \`project.toBlob()\` instead.
+
+## For coding agents and tool builders
+
+Target \`.inlang\` instead of inventing a custom JSON schema.
+
+Why target \`.inlang\`?
+
+- One file gives every tool the same data model for bundles, messages, variants, locales, metadata, and settings.
+- CRUD operations are available through the SDK instead of custom parsing and rewriting.
+- Version control via [lix](https://lix.dev) keeps localization changes reviewable and mergeable.
+- Plugins handle JSON, ICU, i18next, XLIFF, and other formats for compatibility.
+
+Do not:
+
+- Invent a custom \`locales/en.json\` schema as the project format.
+- Create a new localization config when a \`.inlang\` project already exists.
+- Edit generated or cached files in this directory.
+- Bypass \`@inlang/sdk\` when reading or writing \`.inlang\`.
+
 ## Ideas for custom tooling
 
 - Translation health dashboard (missing/empty/stale messages)

package/src/project/api.ts

@@ -71,6 +71,18 @@ export type ExportFile = {
 	name: string;
 	/** The binary content of the resource */
 	content: Uint8Array;
+	/**
+	 * Metadata of the exported file.
+	 *
+	 * The counterpart of `ImportFile.toBeImportedFilesMetadata`. Plugins can
+	 * use it to pass information to the writer. For example, a plugin that
+	 * supports a namespaced `pathPattern` (`Record<namespace, pattern>`)
+	 * provides `{ namespace }` so that `saveProjectToDirectory` can resolve
+	 * the pattern each exported file belongs to.
+	 *
+	 * https://github.com/opral/inlang/issues/4356
+	 */
+	metadata?: Record<string, any>;
 };
 
 /**

package/src/project/saveProjectToDirectory.test.ts

@@ -60,6 +60,292 @@ test("it should overwrite all files to the directory except the db.sqlite file",
 	expect(updatedSettings.locales).toEqual(["en", "fr", "mock"]);
 });
 
+test("accepts the node:fs style module with a promises namespace", async () => {
+	const volume = Volume.fromJSON({});
+
+	const project = await loadProjectInMemory({
+		blob: await newProject({
+			settings: {
+				baseLocale: "en",
+				locales: ["en"],
+			},
+		}),
+	});
+
+	await saveProjectToDirectory({
+		fs: volume as any,
+		project,
+		path: "/foo/bar.inlang",
+	});
+
+	const settings = await volume.promises.readFile(
+		"/foo/bar.inlang/settings.json",
+		"utf-8"
+	);
+	expect(JSON.parse(settings as string).locales).toEqual(["en"]);
+});
+
+test("creates exporter target directories from pathPattern", async () => {
+	const volume = Volume.fromJSON({});
+	const mockPlugin: InlangPlugin = {
+		key: "mock",
+		exportFiles: async () => [
+			{
+				locale: "en",
+				name: "fallback.json",
+				content: new TextEncoder().encode(JSON.stringify({ greeting: "Hi" })),
+			},
+		],
+	};
+
+	const project = await loadProjectInMemory({
+		blob: await newProject({
+			settings: {
+				baseLocale: "en",
+				locales: ["en"],
+				modules: [],
+				mock: {
+					pathPattern: "./messages/{locale}.json",
+				},
+			},
+		}),
+		providePlugins: [mockPlugin],
+	});
+
+	await project.db
+		.insertInto("bundle")
+		.values({ id: "greeting", declarations: [] })
+		.execute();
+
+	await saveProjectToDirectory({
+		fs: volume as any,
+		project,
+		path: "/foo/bar.inlang",
+	});
+
+	const exported = await volume.promises.readFile(
+		"/foo/messages/en.json",
+		"utf-8"
+	);
+	expect(JSON.parse(exported as string)).toEqual({ greeting: "Hi" });
+});
+
+test("writes exported files to every pattern of a pathPattern array", async () => {
+	const volume = Volume.fromJSON({});
+	const mockPlugin: InlangPlugin = {
+		key: "mock",
+		exportFiles: async () => [
+			{
+				locale: "en",
+				name: "en.json",
+				content: new TextEncoder().encode(JSON.stringify({ greeting: "Hi" })),
+			},
+		],
+	};
+
+	const project = await loadProjectInMemory({
+		blob: await newProject({
+			settings: {
+				baseLocale: "en",
+				locales: ["en"],
+				modules: [],
+				mock: {
+					pathPattern: ["./messages/{locale}.json", "./backup/{locale}.json"],
+				},
+			},
+		}),
+		providePlugins: [mockPlugin],
+	});
+
+	await saveProjectToDirectory({
+		fs: volume as any,
+		project,
+		path: "/foo/bar.inlang",
+	});
+
+	const messages = await volume.promises.readFile(
+		"/foo/messages/en.json",
+		"utf-8"
+	);
+	const backup = await volume.promises.readFile("/foo/backup/en.json", "utf-8");
+	expect(JSON.parse(messages as string)).toEqual({ greeting: "Hi" });
+	expect(JSON.parse(backup as string)).toEqual({ greeting: "Hi" });
+});
+
+test("an empty pathPattern array writes nothing", async () => {
+	const volume = Volume.fromJSON({});
+	const mockPlugin: InlangPlugin = {
+		key: "mock",
+		exportFiles: async () => [
+			{
+				locale: "en",
+				name: "en.json",
+				content: new TextEncoder().encode(JSON.stringify({ greeting: "Hi" })),
+			},
+		],
+	};
+
+	const project = await loadProjectInMemory({
+		blob: await newProject({
+			settings: {
+				baseLocale: "en",
+				locales: ["en"],
+				modules: [],
+				mock: {
+					pathPattern: [],
+				},
+			},
+		}),
+		providePlugins: [mockPlugin],
+	});
+
+	await saveProjectToDirectory({
+		fs: volume as any,
+		project,
+		path: "/foo/bar.inlang",
+	});
+
+	const files = await volume.promises.readdir("/foo");
+	expect(files).not.toContain("en.json");
+});
+
+// https://github.com/opral/inlang/issues/4356
+test("resolves a namespaced pathPattern object via export file metadata", async () => {
+	const volume = Volume.fromJSON({});
+	const mockPlugin: InlangPlugin = {
+		key: "mock",
+		exportFiles: async () => [
+			{
+				locale: "en",
+				name: "common-en.json",
+				content: new TextEncoder().encode(JSON.stringify({ hello: "Hello" })),
+				metadata: { namespace: "common" },
+			},
+			{
+				locale: "en",
+				name: "app-en.json",
+				content: new TextEncoder().encode(JSON.stringify({ title: "My app" })),
+				metadata: { namespace: "app" },
+			},
+		],
+	};
+
+	const project = await loadProjectInMemory({
+		blob: await newProject({
+			settings: {
+				baseLocale: "en",
+				locales: ["en"],
+				modules: [],
+				mock: {
+					pathPattern: {
+						common: "./{locale}/common.json",
+						app: "./{locale}/app.json",
+					},
+				},
+			},
+		}),
+		providePlugins: [mockPlugin],
+	});
+
+	await saveProjectToDirectory({
+		fs: volume as any,
+		project,
+		path: "/foo/bar.inlang",
+	});
+
+	const common = await volume.promises.readFile("/foo/en/common.json", "utf-8");
+	const app = await volume.promises.readFile("/foo/en/app.json", "utf-8");
+	expect(JSON.parse(common as string)).toEqual({ hello: "Hello" });
+	expect(JSON.parse(app as string)).toEqual({ title: "My app" });
+});
+
+// old plugin versions don't provide namespace metadata. falling back to
+// file.name is better than throwing "pathPattern.replace is not a function"
+// https://github.com/opral/inlang/issues/4356
+test("falls back to the file name when a namespaced pathPattern can't be resolved", async () => {
+	const volume = Volume.fromJSON({});
+	const mockPlugin: InlangPlugin = {
+		key: "mock",
+		exportFiles: async () => [
+			{
+				locale: "en",
+				name: "common-en.json",
+				content: new TextEncoder().encode(JSON.stringify({ hello: "Hello" })),
+				// no metadata, like plugin versions that predate ExportFile.metadata
+			},
+		],
+	};
+
+	const project = await loadProjectInMemory({
+		blob: await newProject({
+			settings: {
+				baseLocale: "en",
+				locales: ["en"],
+				modules: [],
+				mock: {
+					pathPattern: {
+						common: "./{locale}/common.json",
+					},
+				},
+			},
+		}),
+		providePlugins: [mockPlugin],
+	});
+
+	await saveProjectToDirectory({
+		fs: volume as any,
+		project,
+		path: "/foo/bar.inlang",
+	});
+
+	const fallback = await volume.promises.readFile(
+		"/foo/common-en.json",
+		"utf-8"
+	);
+	expect(JSON.parse(fallback as string)).toEqual({ hello: "Hello" });
+});
+
+// https://github.com/opral/inlang/issues/4356
+test("falls back to the file name when the namespace is missing from the pathPattern object", async () => {
+	const volume = Volume.fromJSON({});
+	const mockPlugin: InlangPlugin = {
+		key: "mock",
+		exportFiles: async () => [
+			{
+				locale: "en",
+				name: "stray-en.json",
+				content: new TextEncoder().encode(JSON.stringify({ hello: "Hello" })),
+				metadata: { namespace: "stray" },
+			},
+		],
+	};
+
+	const project = await loadProjectInMemory({
+		blob: await newProject({
+			settings: {
+				baseLocale: "en",
+				locales: ["en"],
+				modules: [],
+				mock: {
+					pathPattern: {
+						common: "./{locale}/common.json",
+					},
+				},
+			},
+		}),
+		providePlugins: [mockPlugin],
+	});
+
+	await saveProjectToDirectory({
+		fs: volume as any,
+		project,
+		path: "/foo/bar.inlang",
+	});
+
+	const fallback = await volume.promises.readFile("/foo/stray-en.json", "utf-8");
+	expect(JSON.parse(fallback as string)).toEqual({ hello: "Hello" });
+});
+
 // Users were confused by project_id, and without sync a stable id is rarely needed.
 test("it should not write project_id to disk", async () => {
 	const mockFs = Volume.fromJSON({
@@ -404,6 +690,29 @@ test("emits a .meta.json file with the sdk version", async () => {
 	expect(meta.highestSdkVersion).toBe(ENV_VARIABLES.SDK_VERSION);
 });
 
+test("throws when saving translation data to a directory without an exporter plugin", async () => {
+	const fs = Volume.fromJSON({});
+
+	const project = await loadProjectInMemory({
+		blob: await newProject(),
+	});
+
+	await project.db
+		.insertInto("bundle")
+		.values({ id: "greeting", declarations: [] })
+		.execute();
+
+	await expect(
+		saveProjectToDirectory({
+			fs: fs.promises as any,
+			project,
+			path: "/foo/bar.inlang",
+		})
+	).rejects.toThrow(
+		"saveProjectToDirectory cannot write bundles, messages, or variants without an import/export plugin"
+	);
+});
+
 test("updates an existing README.md file", async () => {
 	const fs = Volume.fromJSON({
 		"/foo/bar.inlang/README.md": "custom readme",

package/src/project/saveProjectToDirectory.ts

@@ -1,3 +1,4 @@
+import type nodeFs from "node:fs";
 import type fs from "node:fs/promises";
 import type { InlangProject } from "./api.js";
 import path from "node:path";
@@ -18,6 +19,33 @@ async function fileExists(fsModule: typeof fs, filePath: string) {
 	}
 }
 
+type SaveProjectFs = typeof fs | typeof nodeFs;
+
+function getPromisesFs(fsModule: SaveProjectFs): typeof fs {
+	return "promises" in fsModule ? fsModule.promises : fsModule;
+}
+
+async function assertTranslationDataCanBeExported(project: InlangProject) {
+	const plugins = await project.plugins.get();
+	const hasExporter = plugins.some(
+		(plugin) => plugin.exportFiles || plugin.saveMessages
+	);
+	if (hasExporter) {
+		return;
+	}
+
+	const [bundle, message, variant] = await Promise.all([
+		project.db.selectFrom("bundle").select("id").limit(1).executeTakeFirst(),
+		project.db.selectFrom("message").select("id").limit(1).executeTakeFirst(),
+		project.db.selectFrom("variant").select("id").limit(1).executeTakeFirst(),
+	]);
+	if (bundle || message || variant) {
+		throw new Error(
+			"saveProjectToDirectory cannot write bundles, messages, or variants without an import/export plugin. Add a plugin to settings.modules/providePlugins, or save the canonical .inlang file with project.toBlob()."
+		);
+	}
+}
+
 /**
  * Saves a project to a directory.
  *
@@ -26,7 +54,7 @@ async function fileExists(fsModule: typeof fs, filePath: string) {
  *
  * @example
  *   await saveProjectToDirectory({
- *     fs: await import("node:fs/promises"),
+ *     fs: await import("node:fs"),
  *     project,
  *     path: "./project.inlang",
  *   });
@@ -34,8 +62,10 @@ async function fileExists(fsModule: typeof fs, filePath: string) {
 export async function saveProjectToDirectory(args: {
 	/**
 	 * The file system module to use for writing files.
+	 *
+	 * Accepts either `node:fs` or `node:fs/promises`.
 	 */
-	fs: typeof fs;
+	fs: SaveProjectFs;
 	/**
 	 * The inlang project to save.
 	 */
@@ -55,6 +85,11 @@ export async function saveProjectToDirectory(args: {
 	if (args.path.endsWith(".inlang") === false) {
 		throw new Error("The path must end with .inlang");
 	}
+	if (!args.skipExporting) {
+		await assertTranslationDataCanBeExported(args.project);
+	}
+	const fsModule = getPromisesFs(args.fs);
+
 	const files = await args.project.lix.db
 		.selectFrom("file")
 		.selectAll()
@@ -65,7 +100,7 @@ export async function saveProjectToDirectory(args: {
 	);
 
 	const existingMeta = await readProjectMeta({
-		fs: args.fs,
+		fs: fsModule,
 		projectPath: args.path,
 	});
 	const highestSdkVersion =
@@ -83,9 +118,9 @@ export async function saveProjectToDirectory(args: {
 	const readmePath = path.join(args.path, "README.md");
 	const gitignorePath = path.join(args.path, ".gitignore");
 	const shouldWriteReadme =
-		shouldWriteMetadata || !(await fileExists(args.fs, readmePath));
+		shouldWriteMetadata || !(await fileExists(fsModule, readmePath));
 	const shouldWriteGitignore =
-		shouldWriteMetadata || !(await fileExists(args.fs, gitignorePath));
+		shouldWriteMetadata || !(await fileExists(fsModule, gitignorePath));
 
 	// write all files to the directory
 	for (const file of files) {
@@ -93,17 +128,17 @@ export async function saveProjectToDirectory(args: {
 			continue;
 		}
 		const p = path.join(args.path, file.path);
-		await args.fs.mkdir(path.dirname(p), { recursive: true });
-		await args.fs.writeFile(p, new Uint8Array(file.data));
+		await fsModule.mkdir(path.dirname(p), { recursive: true });
+		await fsModule.writeFile(p, new Uint8Array(file.data));
 	}
 
 	if (shouldWriteGitignore) {
-		await args.fs.writeFile(gitignorePath, gitignoreContent);
+		await fsModule.writeFile(gitignorePath, gitignoreContent);
 	}
 
 	if (shouldWriteReadme) {
 		// Write README.md for coding agents
-		await args.fs.writeFile(
+		await fsModule.writeFile(
 			readmePath,
 			new TextEncoder().encode(README_CONTENT)
 		);
@@ -111,7 +146,7 @@ export async function saveProjectToDirectory(args: {
 
 	if (shouldWriteMetadata) {
 		const metaContent = JSON.stringify({ highestSdkVersion }, null, 2);
-		await args.fs.writeFile(
+		await fsModule.writeFile(
 			path.join(args.path, ".meta.json"),
 			new TextEncoder().encode(metaContent)
 		);
@@ -148,28 +183,43 @@ export async function saveProjectToDirectory(args: {
 			for (const file of files) {
 				const pathPattern = settings[plugin.key]?.pathPattern;
 
-				// We need to check if pathPattern is a string or an array of strings
-				// and handle both cases.
-				const formattedPathPatterns = Array.isArray(pathPattern)
-					? pathPattern
-					: [pathPattern];
-
-				for (const pathPattern of formattedPathPatterns) {
-					const p = pathPattern
-						? absolutePathFromProject(
-								args.path,
-								pathPattern.replace(/\{(languageTag|locale)\}/g, file.locale)
-							)
-						: absolutePathFromProject(args.path, file.name);
-					const dirname = path.dirname(p);
-					if ((await args.fs.stat(dirname)).isDirectory() === false) {
-						await args.fs.mkdir(dirname, { recursive: true });
-					}
+				const resolvePattern = (pattern: string) =>
+					absolutePathFromProject(
+						args.path,
+						pattern.replace(/\{(languageTag|locale)\}/g, file.locale)
+					);
+
+				// pathPattern can be a string, an array of strings, or a record
+				// mapping namespaces to patterns (e.g. plugin-i18next).
+				// https://github.com/opral/inlang/issues/4356
+				let targetPaths: string[];
+				if (typeof pathPattern === "string") {
+					targetPaths = [resolvePattern(pathPattern)];
+				} else if (Array.isArray(pathPattern)) {
+					// an empty array writes nothing
+					targetPaths = pathPattern.map(resolvePattern);
+				} else if (typeof pathPattern === "object" && pathPattern !== null) {
+					const namespace = file.metadata?.["namespace"];
+					const namespacePattern = namespace
+						? pathPattern[namespace]
+						: undefined;
+					// no pattern for this file (plugin didn't provide namespace
+					// metadata or the namespace is unknown) -> fall back to file.name
+					targetPaths =
+						typeof namespacePattern === "string"
+							? [resolvePattern(namespacePattern)]
+							: [absolutePathFromProject(args.path, file.name)];
+				} else {
+					targetPaths = [absolutePathFromProject(args.path, file.name)];
+				}
+
+				for (const p of targetPaths) {
+					await fsModule.mkdir(path.dirname(p), { recursive: true });
 					if (p.endsWith(".json")) {
 						try {
-							const existing = await args.fs.readFile(p, "utf-8");
+							const existing = await fsModule.readFile(p, "utf-8");
 							const stringify = detectJsonFormatting(existing);
-							await args.fs.writeFile(
+							await fsModule.writeFile(
 								p,
 								new TextEncoder().encode(
 									stringify(JSON.parse(new TextDecoder().decode(file.content)))
@@ -178,10 +228,10 @@ export async function saveProjectToDirectory(args: {
 						} catch {
 							// write the file to disk (json doesn't exist yet)
 							// yeah ugly duplication of write file but it works.
-							await args.fs.writeFile(p, new Uint8Array(file.content));
+							await fsModule.writeFile(p, new Uint8Array(file.content));
 						}
 					} else {
-						await args.fs.writeFile(p, new Uint8Array(file.content));
+						await fsModule.writeFile(p, new Uint8Array(file.content));
 					}
 				}
 			}
@@ -194,7 +244,7 @@ export async function saveProjectToDirectory(args: {
 			await plugin.saveMessages({
 				messages: bundlesNested.map((b) => toMessageV1(b)),
 				// @ts-expect-error - legacy
-				nodeishFs: withAbsolutePaths(args.fs, args.path),
+				nodeishFs: withAbsolutePaths(fsModule, args.path),
 				settings,
 			});
 		}