global-app-skill 0.0.0-stage → 0.1.1

package/README.md

@@ -1,3 +1,62 @@
-# Temporary Holding Version
+# global-app-skill
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+海外 Android App 调研 skill 的薄入口、资源更新器和脚本 CLI。保持 `overseas-app-research` 的下载、业务调研与 UED 图册三个分支。
+
+## 安装
+
+要求 Node.js >=22.12；执行调研脚本需要 `python3` >=3.9，Android 设备、SDK 和其他工具由工作流按需检查。
+
+```sh
+npx -y --prefer-online global-app-skill@latest skill install
+```
+
+默认只安装 `~/.agents/skills/overseas-app-research/SKILL.md`。也可用 `--dir /absolute/skills/overseas-app-research` 指定当前客户端实际使用的 skill 目录。已存在的 SKILL.md 会被原子替换，目录中的其他文件保留，符号链接入口拒绝覆盖。安装不下载 references，也不修改 MCP 或客户端配置。安装后让客户端重新加载 skills。
+
+## 执行
+
+```sh
+npx -y --prefer-online global-app-skill@latest context resolve --mode research
+```
+
+`--mode` 为 `download`、`research` 或 `ued`。CLI 每次查询 GitHub 的 `releases/latest.json`，验证清单、压缩包及逐文件 SHA-256，返回绝对 `resourceRoot` 和有序 `mustRead`。agent 读取这些文件后按原工作流编排任务。
+
+```sh
+npx -y --prefer-online global-app-skill@latest run preflight --mode research --resource-root /absolute/resolved/hash -- --report /absolute/task/environment.json
+```
+
+`run` 支持 `preflight`、`download`、`export_device`、`verify`、`attempts`、`inspect_package`、`long_screenshot`，`--` 后参数原样传递。它保留当前工作目录、Python 输出和退出码；下载模式进度使用 `/4`，业务/UED 使用 `/10`。
+
+同一任务固定在已读取的快照上执行，新任务重新 resolve。省略 `--resource-root` 的 `run` 会先获取当前资源。设备环境门槛、原包与证据规则保持在动态工作流内；CLI 不自动完成反编译、页面操作、Mock 或业务验收。
+
+## 结构与更新机制
+
+| 路径 | 职责 |
+|---|---|
+| `skills/overseas-app-research/SKILL.md` | 安装到客户端的薄入口 |
+| `src/` | npm CLI：安装、资源解析、校验、脚本运行 |
+| `resources/overseas-app-research/references/` | 主流程和各阶段操作文档 |
+| `resources/overseas-app-research/scripts/` | 原有 Python 脚本及回归测试 |
+| `resources/overseas-app-research/examples/` | 业务报告模板 |
+| `scripts/build-resources.mjs` | 生成确定性的资源包与清单 |
+| `releases/` | GitHub 托管的最新清单与按哈希命名的资源包 |
+
+分发源为 `https://raw.githubusercontent.com/shiguang-lab/global-app-skill/main/releases/latest.json`。npm 包仅包含 CLI 与薄 skill，工作流资源独立从 GitHub 更新，无需为文档或 Python 脚本修改发布 npm 新版本。
+
+缓存默认位于 `~/.cache/global-app-skill/releases/<resourceHash>`，可用 `GLOBAL_APP_SKILL_CACHE` 或 `--cache-dir` 指定。下载先在临时目录完整校验，再发布快照。网络或完整性失败会报错，不静默使用旧资源；损坏的固定快照拒绝执行，重新 resolve 可修复。旧快照保留供现有任务使用。运行产物写入工作区，不写回资源缓存。
+
+## 开发与发布
+
+在当前仓库编辑资源后：
+
+```sh
+npm run resources:build
+npm test
+npm run test:python
+npm run resources:check
+```
+
+Python 回归测试中的长图部分需要 Pillow。CLI 无第三方 Node 依赖。`--feed` 可指定 HTTPS 清单，测试允许 loopback HTTP，用于在发布前验证同一协议。
+
+将资源源文件、`releases/latest.json` 和新资源包一起提交并 push 到 GitHub 的 `main`。保留旧资源包，避免读到旧清单的客户端遇到 404。只修改资源时到此即可，后续任务自动获取最新快照；每个快照完整包含 references、scripts 和 examples，删除的源文件不会残留到新快照。
+
+修改 CLI 或薄入口时更新 `package.json` 的版本，完成检查并推送代码后执行 `npm publish`。`prepublishOnly` 会执行 Node/Python 测试和资源一致性检查。初次发布需要 npm 登录和账号所要求的发布认证；GitHub 仓库需公开可读，才能使用默认免认证资源源。当前项目未授予开源许可证，npm 元数据使用 `UNLICENSED`。

package/package.json

@@ -1,6 +1,22 @@
 {
   "name": "global-app-skill",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.1.1",
+  "description": "Thin skill bootstrap and GitHub resource runner for overseas Android app research",
+  "type": "module",
+  "license": "UNLICENSED",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/shiguang-lab/global-app-skill.git"
+  },
+  "engines": { "node": ">=22.12.0" },
+  "bin": { "global-app-skill": "src/cli.mjs" },
+  "files": ["src", "skills"],
+  "publishConfig": { "registry": "https://registry.npmjs.org/", "access": "public" },
+  "scripts": {
+    "test": "node --test tests/*.test.mjs",
+    "test:python": "python3 -m unittest discover -s resources/overseas-app-research/scripts -p 'test_*.py'",
+    "resources:build": "node scripts/build-resources.mjs",
+    "resources:check": "node scripts/build-resources.mjs --check",
+    "prepublishOnly": "npm test && npm run test:python && npm run resources:check"
+  }
+}

package/skills/overseas-app-research/SKILL.md

@@ -0,0 +1,18 @@
+---
+name: overseas-app-research
+description: 调研海外 Android App 的业务流程、功能入口、字段权限，或为 UED 提取整款 App/指定范围的原始界面图册；也支持仅下载/导出 APK。
+---
+
+# 海外 App 调研
+
+需要 Node.js >=22.12、`python3` >=3.9；设备/AVD 门槛由解析后的工作流检查。
+
+按目标选择 `download`（仅取包）、`research`（业务调研，含同时需要图册）或 `ued`（仅界面图册），执行：
+
+```sh
+npx -y --prefer-online global-app-skill@latest context resolve --mode MODE
+```
+
+依次读取返回的全部 `mustRead`，按其中流程执行；以返回的 `resourceRoot` 解析文档、模板和脚本。新任务重新 resolve，同一任务沿用已读快照。更新或校验失败时停止依赖工作并报告原因，不静默使用旧资源。
+
+脚本通过 `npx -y --prefer-online global-app-skill@latest run SCRIPT --mode MODE --resource-root RESOURCE_ROOT -- 参数` 执行；可用脚本及参数见解析后的 CLI 文档。业务判断与页面操作由 agent 按文档编排，CLI 成功不代表调研完成；关键进度仍须输出到聊天正文。

package/src/cli.mjs

@@ -0,0 +1,52 @@
+#!/usr/bin/env node
+import { parseArgs } from 'node:util';
+import { installSkill } from './skill.mjs';
+import { resolveContext } from './resources.mjs';
+import { runScript } from './run.mjs';
+import { validateMode, VERSION } from './protocol.mjs';
+
+const usage = `global-app-skill ${VERSION}
+  skill install [--dir ABSOLUTE_SKILL_DIRECTORY]
+  context resolve --mode download|research|ued [--cache-dir PATH] [--feed HTTPS_MANIFEST_URL]
+  run SCRIPT [--mode download|research|ued] [--resource-root PATH] [--cache-dir PATH] [--feed URL] -- SCRIPT_ARGS
+  --version
+
+skill install writes only SKILL.md; default: ~/.agents/skills/overseas-app-research.
+run uses a verified pinned snapshot when --resource-root is supplied; otherwise it resolves current resources.
+Script stdout/stderr and exit code are preserved. JSON command results go to stdout; progress goes to stderr.
+`;
+
+export async function main(argv) {
+  const separator = argv.indexOf('--');
+  const { values, positionals } = parseArgs({
+    args: separator < 0 ? argv : argv.slice(0, separator), allowPositionals: true,
+    options: { dir: { type: 'string' }, mode: { type: 'string' }, 'resource-root': { type: 'string' },
+      'cache-dir': { type: 'string' }, feed: { type: 'string' }, help: { type: 'boolean' }, version: { type: 'boolean' } },
+  });
+  if (values.version) { console.log(VERSION); return; }
+  if (values.help || positionals.length === 0) { console.log(usage); return; }
+  const [command, action] = positionals;
+  if (positionals.length !== 2 || (separator >= 0 && command !== 'run')) throw new Error('Invalid command; use --help');
+  const allowed = command === 'skill' && action === 'install' ? ['dir'] :
+    command === 'context' && action === 'resolve' ? ['mode', 'cache-dir', 'feed'] :
+    command === 'run' ? ['mode', 'resource-root', 'cache-dir', 'feed'] : null;
+  if (!allowed) throw new Error('Unknown command; use --help');
+  for (const key of Object.keys(values)) {
+    if (!allowed.includes(key)) throw new Error(`--${key} is not supported by ${command} ${action}`);
+  }
+  const options = { mode: values.mode || 'research', resourceRoot: values['resource-root'], cacheDir: values['cache-dir'],
+    feed: values.feed, progress: message => process.stderr.write(`[global-app-skill] ${message}\n`) };
+  if (command !== 'skill') validateMode(command === 'context' ? values.mode : options.mode);
+  if (command === 'run') {
+    if (options.resourceRoot && (values.feed || values['cache-dir'])) throw new Error('--resource-root cannot be combined with --feed or --cache-dir');
+    process.exitCode = await runScript(action, separator < 0 ? [] : argv.slice(separator + 1), options);
+    return;
+  }
+  const output = command === 'skill' ? await installSkill(values.dir) : await resolveContext(options);
+  console.log(JSON.stringify(output, null, 2));
+}
+
+main(process.argv.slice(2)).catch(error => {
+  process.stderr.write(JSON.stringify({ ok: false, error: error.message }) + '\n');
+  process.exitCode = 1;
+});

package/src/protocol.mjs

@@ -0,0 +1,52 @@
+import { createHash } from 'node:crypto';
+import { readFileSync } from 'node:fs';
+
+export const VERSION = JSON.parse(readFileSync(new URL('../package.json', import.meta.url))).version;
+export const FEED = 'https://raw.githubusercontent.com/shiguang-lab/global-app-skill/main/releases/latest.json';
+export const MAX_BYTES = 16 * 1024 * 1024;
+export const MODES = {
+  download: 'references/download-workflow.md',
+  research: 'references/research-workflow.md',
+  ued: 'references/ued-workflow.md',
+};
+export const SCRIPTS = ['preflight', 'download', 'export_device', 'verify', 'attempts', 'inspect_package', 'long_screenshot'];
+export const sha256 = (bytes) => createHash('sha256').update(bytes).digest('hex');
+export const identity = (files) => sha256(JSON.stringify(files));
+
+export function validateMode(mode) {
+  if (!Object.hasOwn(MODES, mode)) throw new Error('--mode must be download, research or ued');
+  return mode;
+}
+
+export function validateManifest(manifest) {
+  const hash = /^[a-f0-9]{64}$/;
+  if (manifest?.schemaVersion !== 1 || !hash.test(manifest.resourceHash) ||
+      !/^\d+\.\d+\.\d+$/.test(manifest.minCliVersion) ||
+      !Array.isArray(manifest.files) || !manifest.files.length || manifest.files.length > 1024 ||
+      manifest.bundle !== `${manifest.resourceHash}.json.gz` || !hash.test(manifest.sha256) ||
+      !Number.isSafeInteger(manifest.size) || manifest.size <= 0 || manifest.size > MAX_BYTES) {
+    throw new Error('Invalid resource manifest');
+  }
+  const seen = new Set();
+  let total = 0;
+  for (const file of manifest.files) {
+    if (typeof file.path !== 'string' || !/^(references|scripts|examples)\//.test(file.path) ||
+        file.path.split('/').some(part => !/^[a-zA-Z0-9_.-]+$/.test(part) || part === '.' || part === '..') ||
+        seen.has(file.path) || !hash.test(file.sha256) ||
+        !Number.isSafeInteger(file.size) || file.size < 0 || (total += file.size) > MAX_BYTES) {
+      throw new Error('Invalid resource file entry');
+    }
+    seen.add(file.path);
+  }
+  if (identity(manifest.files) !== manifest.resourceHash) throw new Error('Resource manifest hash mismatch');
+  const required = ['references/guide.md', 'references/cli.md', 'references/orchestration.md',
+    ...Object.values(MODES), ...SCRIPTS.map(name => `scripts/${name}.py`), 'scripts/apk_utils.py', 'scripts/progress.py'];
+  if (required.some(name => !seen.has(name))) throw new Error('Resource manifest is missing workflow files');
+  const installed = VERSION.split('.').map(Number);
+  const minimum = manifest.minCliVersion.split('.').map(Number);
+  for (let i = 0; i < 3; i++) {
+    if (installed[i] > minimum[i]) break;
+    if (installed[i] < minimum[i]) throw new Error(`Resources require CLI >=${manifest.minCliVersion}; rerun with global-app-skill@latest`);
+  }
+  return manifest;
+}

package/src/resources.mjs

@@ -0,0 +1,115 @@
+import { lstat, mkdir, mkdtemp, readFile, rename, rm, writeFile } from 'node:fs/promises';
+import { homedir } from 'node:os';
+import path from 'node:path';
+import { setTimeout as delay } from 'node:timers/promises';
+import { gunzipSync } from 'node:zlib';
+import { FEED, MAX_BYTES, MODES, sha256, validateManifest, validateMode } from './protocol.mjs';
+
+async function download(url, limit) {
+  const response = await fetch(url, { cache: 'no-store', signal: AbortSignal.timeout(30000) });
+  if (!response.ok) throw new Error(`Cannot download ${url}: HTTP ${response.status}`);
+  const chunks = [];
+  let size = 0;
+  for await (const chunk of response.body) {
+    size += chunk.length;
+    if (size > limit) {
+      throw new Error('Resource download exceeds limit');
+    }
+    chunks.push(chunk);
+  }
+  return Buffer.concat(chunks);
+}
+
+export async function readSnapshot(resourceRoot) {
+  resourceRoot = path.resolve(resourceRoot);
+  if (!(await lstat(resourceRoot)).isDirectory()) throw new Error('Resource root must be a real directory');
+  const manifestPath = path.join(resourceRoot, 'manifest.json');
+  if (!(await lstat(manifestPath)).isFile()) throw new Error('Resource manifest must be a regular file');
+  const manifest = validateManifest(JSON.parse(await readFile(manifestPath, 'utf8')));
+  if (path.basename(resourceRoot) !== manifest.resourceHash) throw new Error('Resource root does not match its snapshot hash');
+  for (const file of manifest.files) {
+    let directory = resourceRoot;
+    for (const segment of file.path.split('/').slice(0, -1)) {
+      directory = path.join(directory, segment);
+      if (!(await lstat(directory)).isDirectory()) throw new Error(`Invalid resource directory: ${directory}`);
+    }
+    const filename = path.join(resourceRoot, file.path);
+    const stat = await lstat(filename);
+    if (!stat.isFile() || stat.size !== file.size || sha256(await readFile(filename)) !== file.sha256) {
+      throw new Error(`Resource integrity check failed: ${file.path}`);
+    }
+  }
+  return manifest;
+}
+
+async function validSnapshot(root, resourceHash) {
+  try { return (await readSnapshot(root)).resourceHash === resourceHash; }
+  catch { return false; }
+}
+
+export async function resolveContext({ mode, feed = FEED, cacheDir, progress = () => {} } = {}) {
+  validateMode(mode);
+  const url = new URL(feed);
+  if ((url.protocol !== 'https:' && !(url.protocol === 'http:' && ['localhost', '127.0.0.1', '[::1]'].includes(url.hostname))) || url.username || url.password) {
+    throw new Error('--feed must be HTTPS (HTTP is allowed only for loopback testing)');
+  }
+  progress('Checking GitHub resource manifest');
+  const manifest = validateManifest(JSON.parse((await download(url, 1024 * 1024)).toString('utf8')));
+  const root = path.resolve(cacheDir || process.env.GLOBAL_APP_SKILL_CACHE || path.join(homedir(), '.cache/global-app-skill'));
+  const releases = path.join(root, 'releases');
+  const resourceRoot = path.join(releases, manifest.resourceHash);
+  let updated = false;
+  if (!(await validSnapshot(resourceRoot, manifest.resourceHash))) {
+    progress(`Downloading and verifying ${manifest.files.length} resource files`);
+    const bytes = await download(new URL(manifest.bundle, url), MAX_BYTES);
+    if (bytes.length !== manifest.size || sha256(bytes) !== manifest.sha256) throw new Error('Resource bundle integrity check failed');
+    const bundle = JSON.parse(gunzipSync(bytes, { maxOutputLength: MAX_BYTES * 2 }).toString('utf8'));
+    if (!Array.isArray(bundle.files) || bundle.files.length !== manifest.files.length) throw new Error('Invalid resource bundle');
+    // Check the complete payload before writing anything into the snapshot.
+    const files = bundle.files.map((file, index) => {
+      const expected = manifest.files[index];
+      if (file.path !== expected.path || typeof file.content !== 'string') throw new Error('Resource bundle file mismatch');
+      const content = Buffer.from(file.content, 'base64');
+      if (content.length !== expected.size || sha256(content) !== expected.sha256) throw new Error(`Resource bundle integrity check failed: ${file.path}`);
+      return { path: expected.path, content };
+    });
+    await mkdir(releases, { recursive: true });
+    const staging = await mkdtemp(path.join(releases, '.prepare-'));
+    try {
+      for (const file of files) {
+        const target = path.join(staging, file.path);
+        await mkdir(path.dirname(target), { recursive: true });
+        await writeFile(target, file.content, { flag: 'wx' });
+      }
+      await writeFile(path.join(staging, 'manifest.json'), JSON.stringify(manifest));
+      const lock = path.join(releases, `.${manifest.resourceHash}.lock`);
+      const deadline = Date.now() + 5000;
+      for (;;) {
+        try { await mkdir(lock); break; }
+        catch (error) {
+          if (error.code !== 'EEXIST') throw error;
+          if (Date.now() >= deadline) throw new Error(`Snapshot publication is locked: ${lock}; retry after the other updater finishes. Remove a stale lock only when no updater is running.`);
+          await delay(50);
+        }
+      }
+      try {
+        // Recheck under the lock so a concurrent resolver's completed snapshot
+        // cannot be deleted after an earlier ENOENT integrity check.
+        if (!(await validSnapshot(resourceRoot, manifest.resourceHash))) {
+          await rm(resourceRoot, { recursive: true, force: true });
+          await rename(staging, resourceRoot);
+        }
+      } finally {
+        await rm(lock, { recursive: true, force: true });
+      }
+      updated = true;
+    } finally { await rm(staging, { recursive: true, force: true }); }
+  }
+  await readSnapshot(resourceRoot);
+  progress(updated ? 'Resource snapshot ready' : 'Verified cached snapshot is current');
+  const mustRead = ['references/cli.md', 'references/guide.md', 'references/orchestration.md', MODES[mode]]
+    .map(name => path.join(resourceRoot, name));
+  return { ok: true, source: url.hostname === 'raw.githubusercontent.com' ? 'github' : 'custom', feed: url.href,
+    mode, resourceHash: manifest.resourceHash, resourceRoot,
+    referencePath: path.join(resourceRoot, 'references/guide.md'), mustRead, updated };
+}

package/src/run.mjs

@@ -0,0 +1,21 @@
+import { execFile, spawn } from 'node:child_process';
+import path from 'node:path';
+import { promisify } from 'node:util';
+import { SCRIPTS } from './protocol.mjs';
+import { readSnapshot, resolveContext } from './resources.mjs';
+
+export async function runScript(script, args, options = {}) {
+  if (!SCRIPTS.includes(script)) throw new Error(`Unknown script; choose ${SCRIPTS.join(', ')}`);
+  const resourceRoot = options.resourceRoot ? path.resolve(options.resourceRoot) : (await resolveContext(options)).resourceRoot;
+  await readSnapshot(resourceRoot);
+  await promisify(execFile)('python3', ['-c', 'import sys; sys.exit(0 if sys.version_info >= (3, 9) else 1)'])
+    .catch(() => { throw new Error('Python >=3.9 must be available as python3'); });
+  return new Promise((resolve, reject) => {
+    const child = spawn('python3', [path.join(resourceRoot, 'scripts', `${script}.py`), ...args], {
+      stdio: 'inherit',
+      env: { ...process.env, PYTHONDONTWRITEBYTECODE: '1', APP_RESEARCH_TOTAL_STAGES: options.mode === 'download' ? '4' : '10' },
+    });
+    child.once('error', reject);
+    child.once('exit', (code, signal) => resolve(code ?? (signal === 'SIGINT' ? 130 : 1)));
+  });
+}

package/src/skill.mjs

@@ -0,0 +1,22 @@
+import { lstat, mkdir, readFile, rename, rm, writeFile } from 'node:fs/promises';
+import { randomUUID } from 'node:crypto';
+import { homedir } from 'node:os';
+import path from 'node:path';
+
+export async function installSkill(directory) {
+  const target = path.resolve(directory || path.join(homedir(), '.agents/skills/overseas-app-research'));
+  const skillPath = path.join(target, 'SKILL.md');
+  for (const filename of [target, skillPath]) {
+    try {
+      if ((await lstat(filename)).isSymbolicLink()) throw new Error(`Refusing to replace a symlinked skill: ${filename}`);
+    } catch (error) { if (error.code !== 'ENOENT') throw error; }
+  }
+  const content = await readFile(new URL('../skills/overseas-app-research/SKILL.md', import.meta.url), 'utf8');
+  await mkdir(target, { recursive: true });
+  const temporary = path.join(target, `.SKILL-${randomUUID()}.tmp`);
+  try {
+    await writeFile(temporary, content, { flag: 'wx' });
+    await rename(temporary, skillPath);
+  } finally { await rm(temporary, { force: true }); }
+  return { ok: true, skillPath };
+}