rainbo 0.1.13 → 0.2.1

package/README.md

@@ -14,16 +14,55 @@ Those platform packages are published under the `@singdata` scope.
 
 ## Install
 
+First-time install:
+
 ```bash
 npm install -g rainbo
 rainbo version
 ```
 
+The npm package automatically attempts to sync the official Rainbo skills after
+install. If the postinstall sync cannot reach GitLab or `npx skills`, it does
+not fail the npm install; run the same sync manually:
+
+```bash
+rainbo update --force
+```
+
+Set `RAINBO_NPM_NO_SKILLS_SYNC=1` to skip the automatic postinstall sync in
+scripted environments. If GitLab is unavailable, use the offline package from
+数据校验宝典:
+
+```bash
+rainbo update --skills-package /path/to/rainbo-skills.zip
+```
+
+Maintainers can regenerate the offline package from the repository root with:
+
+```bash
+cd packages/rainbo
+npm run pack:skills
+```
+
+This writes `../../skills.zip`, a standard `npx skills` source archive that
+contains `skills/index.json` plus all official Rainbo skill directories.
+
+If `rainbo` is already installed, prefer the built-in upgrade flow instead of
+running bare `npm install -g rainbo`:
+
+```bash
+rainbo update
+```
+
+`rainbo update` upgrades the npm package and syncs official Rainbo skills in
+one step, which avoids `_notice.skills` drift.
+
 ## Development
 
 ```bash
 node packaging/npm/run.js version
 node packaging/npm/run.js help
+npm run check:npm
 npm run check:skills
 npm run build:npm:local
 ```
@@ -55,6 +94,7 @@ Errors are written to stderr:
 rainbo update --check
 rainbo update
 rainbo update --force
+rainbo update --skills-only
 rainbo update --skills-package /path/to/rainbo-skills.zip
 ```
 
@@ -64,12 +104,16 @@ and `_notice.skills` when locally synced skills are out of sync with the
 running binary. The update cache refreshes at most every 2 hours. Both notices
 recommend `rainbo update`.
 
+`update --skills-only` syncs Rainbo skills without checking npm for newer
+versions or attempting a self-update. It is intended for npm postinstall and
+other bootstrap flows that only need skill synchronization.
+
 `update` detects npm installations, runs `npm install -g rainbo@<latest>`
 when auto-update is available, verifies the new binary, and syncs repository
 skills with:
 
 ```bash
-npx skills add git@gitlab.clickzetta-inc.com:ee/rainbo.git -y -g
+npx skills add git@gitlab.clickzetta-inc.com:ee/rainbo.git -y -g --agent '*'
 ```
 
 Skill sync records detailed state in `skills-state.json` and updates installed
@@ -79,7 +123,10 @@ available, it falls back to a full install.
 When users cannot access the GitLab skills repository, they can use the offline
 skills package from 数据校验宝典 and run `rainbo update --skills-package
 /path/to/rainbo-skills.zip`. The CLI will update Rainbo as usual and sync skills
-from the local package path instead of the GitLab source.
+from the local package path instead of the GitLab source. The offline package is
+installed through the same `npx skills` protocol after Rainbo extracts the zip,
+and `skills-state.json` records the package path for future `_notice.skills`
+version drift checks.
 
 If automatic update is unavailable, the JSON action is `manual_required` and
 the response includes release and changelog URLs.
@@ -92,8 +139,8 @@ rainbo auth status
 rainbo auth logout
 ```
 
-Generate the connection URL from the logged-in Rainbo web UI user menu. `auth
-connect` exchanges the one-time connection code for a revocable CLI bearer token
+Generate the Rainbo connection URL from the logged-in Rainbo web UI user menu. `auth
+connect` exchanges the one-time connection code for a revocable bearer token
 and stores it under `~/.config/rainbo/auth.json` on Unix-like systems or
 `%APPDATA%\rainbo\auth.json` on Windows. Unix-like systems use `0600`
 permissions for this file.
@@ -102,10 +149,12 @@ permissions for this file.
 
 Authenticated API commands reuse the session created by `rainbo auth connect`.
 Complex request bodies can be passed with `--data-json`, `--data-file`, or
-`--data @payload.json`.
+`--data @payload.json`. Add `--dry-run` to print the request method, path,
+query, and body without reading auth state or calling Rainbo APIs.
 
 ```bash
 rainbo data-sync cmt-jobs --data-file cmt-jobs.json
+rainbo data-sync cmt-jobs --data-file cmt-jobs.json --dry-run
 rainbo data-verify lineage-verify-table --data-file verify-table.json
 rainbo toolbox sql-debug --datasource-id 1 --sql "select 1"
 rainbo toolbox lineage-spark --data-file lineage-spark.json
@@ -145,8 +194,10 @@ npm run publish:npm:local
 ```
 
 The publish script refuses to continue if `package.json` and `rust/Cargo.toml`
-versions differ. After building the current-platform binary, it also runs the
-bundled binary and verifies `rainbo version` matches `package.json`.
+versions differ. It also runs `npm run check:npm` to verify npm metadata,
+generated meta-package contents, and platform optional dependencies before
+publishing. After building the current-platform binary, it runs the bundled
+binary and verifies `rainbo version` matches `package.json`.
 The publish script is cross-platform and can be run from macOS shells,
 Windows PowerShell, or Windows CMD.
 
@@ -167,7 +218,7 @@ To build multiple platforms from a machine with the required Rust
 targets/toolchains installed, set:
 
 ```bash
-RAINBO_NPM_BUILD_TARGETS=darwin-arm64,darwin-x64,win32-x64,win32-arm64 npm run publish:npm:local
+RAINBO_NPM_BUILD_TARGETS=linux-x64,linux-arm64,darwin-arm64,darwin-x64,win32-x64,win32-arm64 npm run publish:npm:local
 ```
 
 The package source lives under the Rainbo GitLab repository as
@@ -181,18 +232,27 @@ top-level `rainbo` meta package that depends on those platform packages.
 Repository skills live in `skills/`. They can be installed from the repository:
 
 ```bash
-npx skills add git@gitlab.clickzetta-inc.com:ee/rainbo.git -y -g
+npx skills add git@gitlab.clickzetta-inc.com:ee/rainbo.git -y -g --agent '*'
 ```
 
 `git@gitlab.clickzetta-inc.com:ee/rainbo.git` is the skills repository
 source used by `rainbo update`. The old aggregate `rainbo` skill has
-been removed; current Rainbo skills use focused names such as `rainbo-auth`.
+been removed; current Rainbo skills use focused names such as
+`rainbo-shared`, `rainbo-auth`, `rainbo-data-sync`, `rainbo-data-verify`,
+`rainbo-studio`, `rainbo-lineage`, and `rainbo-toolbox`.
+
+The repository root `skills/index.json` is the official skill list consumed by
+the standard `npx skills` installer. Keep it in sync with `skills/*/SKILL.md`
+and run `npm run check:skills` before publishing. Do not add agent-specific
+install paths such as Claude or Codex directories to the Rainbo CLI; the
+installer owns host compatibility.
 
 New Rainbo skills should follow the style in the repository-level
-`skill-template/`: the top-level
-`SKILL.md` acts as a routing and safety rulebook, while detailed command
-families live in `references/*.md`. Authenticated workflow skills should point
-to `rainbo-auth` first instead of duplicating session rules.
+`skill-template/`: the top-level `SKILL.md` acts as a routing and safety
+rulebook, while detailed command families live in `references/*.md`.
+Business workflow skills should point to `rainbo-shared` first and then
+`rainbo-auth` when they call authenticated APIs instead of duplicating shared
+output, risk, dry-run, session, or secret-handling rules.
 
 ## Project layout
 

package/package.json

@@ -1,7 +1,24 @@
 {
   "name": "rainbo",
-  "version": "0.1.13",
+  "version": "0.2.1",
   "description": "Rainbo command line tools for data migration workflows",
+  "keywords": [
+    "rainbo",
+    "cli",
+    "data-migration",
+    "data-validation",
+    "lineage",
+    "ai-agent"
+  ],
+  "homepage": "https://gitlab.clickzetta-inc.com/ee/rainbo#readme",
+  "repository": {
+    "type": "git",
+    "url": "git+ssh://git@gitlab.clickzetta-inc.com/ee/rainbo.git",
+    "directory": "packages/rainbo"
+  },
+  "bugs": {
+    "url": "https://gitlab.clickzetta-inc.com/ee/rainbo/-/issues"
+  },
   "bin": {
     "rainbo": "packaging/npm/run.js"
   },
@@ -9,14 +26,20 @@
     "node": ">=16"
   },
   "optionalDependencies": {
-    "@singdata/rainbo-cli-darwin-x64": "0.1.13",
-    "@singdata/rainbo-cli-darwin-arm64": "0.1.13",
-    "@singdata/rainbo-cli-win32-x64": "0.1.13",
-    "@singdata/rainbo-cli-win32-arm64": "0.1.13"
+    "@singdata/rainbo-cli-linux-x64": "0.2.1",
+    "@singdata/rainbo-cli-linux-arm64": "0.2.1",
+    "@singdata/rainbo-cli-darwin-x64": "0.2.1",
+    "@singdata/rainbo-cli-darwin-arm64": "0.2.1",
+    "@singdata/rainbo-cli-win32-x64": "0.2.1",
+    "@singdata/rainbo-cli-win32-arm64": "0.2.1"
   },
   "license": "MIT",
+  "scripts": {
+    "postinstall": "node packaging/npm/postinstall.js"
+  },
   "files": [
     "packaging/npm/platform.js",
+    "packaging/npm/postinstall.js",
     "packaging/npm/run.js",
     "README.md",
     "LICENSE"

package/packaging/npm/platform.js

@@ -1,4 +1,20 @@
 const PLATFORM_PACKAGES = {
+  "linux-x64": {
+    packageName: "@singdata/rainbo-cli-linux-x64",
+    target: "x86_64-unknown-linux-gnu",
+    binary: "rainbo",
+    os: "linux",
+    cpu: "x64",
+    archiveExt: ".tar.xz",
+  },
+  "linux-arm64": {
+    packageName: "@singdata/rainbo-cli-linux-arm64",
+    target: "aarch64-unknown-linux-gnu",
+    binary: "rainbo",
+    os: "linux",
+    cpu: "arm64",
+    archiveExt: ".tar.xz",
+  },
   "darwin-x64": {
     packageName: "@singdata/rainbo-cli-darwin-x64",
     target: "x86_64-apple-darwin",

package/packaging/npm/postinstall.js

@@ -0,0 +1,111 @@
+#!/usr/bin/env node
+
+const { spawn } = require("child_process");
+const fs = require("fs");
+const path = require("path");
+
+const isTruthy = (value) => value && value !== "0" && value.toLowerCase() !== "false";
+
+const sourceManifest = path.join(__dirname, "..", "..", "rust", "Cargo.toml");
+const launcher = path.join(__dirname, "run.js");
+
+if (isTruthy(process.env.RAINBO_NPM_NO_INSTALL_HINT) || isTruthy(process.env.RAINBO_NPM_NO_SKILLS_SYNC)) {
+  process.exit(0);
+}
+
+function writeHint(reason) {
+  const reasonLine = reason ? [`Rainbo skills were not synced automatically: ${reason}`] : [];
+  const lines = [
+    "",
+    ...reasonLine,
+    "To sync the official Rainbo skills, run:",
+    "  rainbo update --force",
+    "If GitLab is unavailable, use an offline skills package:",
+    "  rainbo update --skills-package <path-to-rainbo-skills.zip>",
+    "Set RAINBO_NPM_NO_SKILLS_SYNC=1 to skip automatic postinstall sync.",
+    "",
+  ];
+  process.stderr.write(`${lines.join("\n")}\n`);
+}
+
+if (fs.existsSync(sourceManifest)) {
+  writeHint("source checkout detected");
+  process.exit(0);
+}
+
+if (!fs.existsSync(launcher)) {
+  writeHint("rainbo npm launcher is missing");
+  process.exit(0);
+}
+
+process.stderr.write("\nrainbo installed. Syncing official Rainbo skills...\n");
+const child = spawn(
+  process.execPath,
+  [launcher, "update", "--skills-only", "--force", "--format", "json", "--progress", "pretty"],
+  {
+    stdio: ["ignore", "pipe", "pipe"],
+    encoding: "utf8",
+    env: {
+      ...process.env,
+      RAINBO_NPM_POSTINSTALL: "1",
+      RAINBO_CLI_NO_UPDATE_NOTIFIER: "1",
+      RAINBO_CLI_NO_SKILLS_NOTIFIER: "1",
+    },
+  },
+);
+
+let stdout = "";
+let stderr = "";
+
+child.stdout.setEncoding("utf8");
+child.stdout.on("data", (chunk) => {
+  stdout += chunk;
+});
+
+child.stderr.setEncoding("utf8");
+child.stderr.on("data", (chunk) => {
+  stderr += chunk;
+  process.stderr.write(chunk);
+});
+
+child.on("error", (err) => {
+  writeHint(err.message);
+  process.exit(0);
+});
+
+child.on("close", (status) => {
+  if (status !== 0) {
+    writeHint(`sync command exited with status ${status}`);
+    process.exit(0);
+  }
+
+  let output;
+  try {
+    output = JSON.parse(stdout.trim());
+  } catch (err) {
+    const trimmedStdout = stdout.trim();
+    const trimmedStderr = stderr.trim();
+    if (trimmedStdout) process.stderr.write(`${trimmedStdout}\n`);
+    if (trimmedStderr) process.stderr.write(`${trimmedStderr}\n`);
+    writeHint(`sync command returned non-JSON output: ${err.message}`);
+    process.exit(0);
+  }
+
+  const action = output && output.data && output.data.skills_action;
+  if (!action) {
+    writeHint("sync command did not report skills_action");
+    process.exit(0);
+  }
+  if (action === "failed" || action === "fallback_failed") {
+    const warning = output.data.skills_warning || output.data.skills_detail || action;
+    writeHint(warning);
+    process.exit(0);
+  }
+
+  const lines = [
+    "",
+    "Rainbo skills synced.",
+    "",
+  ];
+  process.stderr.write(`${lines.join("\n")}\n`);
+});

package/packaging/npm/run.js

@@ -9,6 +9,19 @@ const isWindows = process.platform === "win32";
 const repoRoot = path.join(__dirname, "..", "..");
 const sourceManifest = path.join(repoRoot, "rust", "Cargo.toml");
 
+function writeError(type, subtype, message, hint) {
+  const error = {
+    ok: false,
+    error: {
+      type,
+      subtype,
+      message,
+    },
+  };
+  if (hint) error.error.hint = hint;
+  process.stderr.write(`${JSON.stringify(error)}\n`);
+}
+
 function resolveBundledLauncher() {
   const platformPackage = currentPlatformPackage();
   if (!platformPackage) return null;
@@ -37,9 +50,12 @@ function installIfMissing() {
   const packageHint = platformPackage
     ? `${platformPackage.packageName} (${currentPlatformKey()})`
     : `unsupported platform ${currentPlatformKey()}`;
-  console.error(`rainbo binary package is missing for ${currentPlatformKey()}.`);
-  console.error(`Expected installed npm binary package: ${packageHint}`);
-  console.error("Reinstall the rainbo npm package so the matching optional dependency is installed.");
+  writeError(
+    "internal",
+    platformPackage ? "binary_package_missing" : "unsupported_platform",
+    `rainbo binary package is missing for ${currentPlatformKey()}`,
+    `expected npm binary package: ${packageHint}; reinstall rainbo so npm installs the matching optional dependency`,
+  );
   process.exit(1);
 }
 
@@ -55,11 +71,24 @@ const command = useSource ? "cargo" : resolveLauncher();
 const args = useSource
   ? ["run", "--quiet", "--manifest-path", sourceManifest, "--", ...process.argv.slice(2)]
   : process.argv.slice(2);
-if (!command) process.exit(1);
+if (!command) {
+  writeError(
+    "internal",
+    "launcher_missing",
+    "rainbo launcher could not resolve a source or bundled binary",
+    "reinstall the rainbo npm package",
+  );
+  process.exit(1);
+}
 const result = spawnSync(command, args, { stdio: "inherit", shell: isWindows });
 
 if (result.error) {
-  console.error(result.error.message);
+  writeError(
+    "internal",
+    "launcher_spawn_failed",
+    result.error.message,
+    `failed to execute ${command}`,
+  );
   process.exit(1);
 }
 process.exit(result.status || 0);