com.jimuwd.xian.registry-proxy 1.0.133 → 1.0.134

package/README.MD

@@ -6,7 +6,7 @@
 
 ## 概述
 
-本项目提供了一个代理服务器（`registry-proxy`），允许 Yarn 从多个注册表获取包，并支持身份验证令牌。项目还包括一个脚本（`scripts/install-from-proxy-registries.sh`），用于自动化启动代理服务器、安装依赖和清理资源的过程。该设置确保与 Yarn 无缝集成，开发者只需使用标准的 `yarn` 命令即可通过代理安装依赖。
+本项目提供了一个本地代理服务器（`registry-proxy`），允许 Yarn 从多个注册表获取包，并支持身份验证令牌。项目还包括一个脚本（`src/client/yarn-install`），用于自动化启动代理服务器、安装依赖和清理资源的过程。该设置确保与 Yarn 无缝集成，开发者只需使用标准的 `yarn` 命令即可通过代理安装依赖。
 
 ## 功能
 
@@ -63,31 +63,22 @@ registries:
 
 ```yaml
 unsafeHttpWhitelist:
-  - "localhost"
+  - "[::1]"
 ```
 
-### 4. 创建安装脚本
-
-在 `scripts/` 目录下创建 `install-from-proxy-registries.sh` 脚本，用于自动化代理设置和依赖安装：
-
-### 5. 设置脚本权限
-
-确保脚本具有可执行权限，并将权限状态提交到版本控制：
-
-```bash
-chmod +x scripts/install-from-proxy-registries.sh
-git add scripts/install-from-proxy-registries.sh
-git commit -m "Add install script with executable permission"
+### 4. 安装命令
+```shell
+yarn dlx -p com.jimuwd.xian.registry-proxy yarn-install
 ```
 
-### 6. 与 Yarn 集成
+### 5. 与 Yarn 集成
 
 更新 `package.json`，使 `yarn` 命令自动运行脚本：
 
 ```json
 {
   "scripts": {
-    "preinstall": "bash scripts/install-from-proxy-registries.sh",
+    "preinstall": "yarn dlx -p com.jimuwd.xian.registry-proxy yarn-install",
     "install": "echo 'Custom install script is running via preinstall, skipping default install.'"
   }
 }
@@ -276,7 +267,7 @@ A lightweight proxy server for Yarn to fetch packages from multiple registries w
 
 ## Overview
 
-This project provides a proxy server (`registry-proxy`) that allows Yarn to fetch packages from multiple registries, with support for authentication tokens. It also includes a script (`scripts/install-from-proxy-registries.sh`) to automate the process of starting the proxy server, installing dependencies, and cleaning up resources. The setup ensures seamless integration with Yarn, allowing developers to use the standard `yarn` command to install dependencies via the proxy.
+This project provides a proxy server (`registry-proxy`) that allows Yarn to fetch packages from multiple registries, with support for authentication tokens. It also includes a script (`src/client/yarn-install`) to automate the process of starting the proxy server, installing dependencies, and cleaning up resources. The setup ensures seamless integration with Yarn, allowing developers to use the standard `yarn` command to install dependencies via the proxy.
 
 ## Features
 
@@ -333,167 +324,24 @@ Create a `.yarnrc.yml` file in your project root to allow Yarn to use the local
 
 ```yaml
 unsafeHttpWhitelist:
-  - "localhost"
-```
-
-### 4. Create the Install Script
-
-Create a script at `scripts/install-from-proxy-registries.sh` to automate the proxy setup and dependency installation:
-
-```bash
-#!/bin/bash
-
-# 启用严格模式，但移除 set -e，手动处理错误
-set -u  # 未定义变量时退出
-set -o pipefail  # 管道中任一命令失败时退出
-
-# 动态确定项目根目录（假设 package.json 所在目录为根目录）
-find_project_root() {
-  local dir="$PWD"
-  while [ "$dir" != "/" ]; do
-    if [ -f "$dir/package.json" ]; then
-      echo "$dir"
-      return 0
-    fi
-    dir=$(dirname "$dir")
-  done
-  echo "Error: Could not find project root (package.json not found)" >&2
-  exit 1
-}
-
-PROJECT_ROOT=$(find_project_root)
-
-# 定义锁文件和端口文件路径（固定在项目根目录）
-LOCK_FILE="$PROJECT_ROOT/.registry-proxy-install.lock"
-
-# 检查是否已经在运行（通过锁文件）
-if [ -f "$LOCK_FILE" ]; then
-  echo "Custom install script is already running (lock file $LOCK_FILE exists)."
-  echo "If this is unexpected, please remove $LOCK_FILE and try again."
-  exit 0  # 内层脚本直接退出，表示任务已由外层脚本完成
-fi
-
-# 创建锁文件
-touch "$LOCK_FILE"
-
-# 清理函数，支持不同的退出状态
-# 参数 $1：退出状态（0 表示正常退出，1 表示异常退出）
-cleanup() {
-  local exit_code=${1:-1}  # 默认退出码为 1（异常退出）
-
-  # 显式清除 EXIT 信号的 trap，避免潜在的误解
-  trap - EXIT
-
-  if [ "$exit_code" -eq 0 ]; then
-    echo "Cleaning up after successful execution..."
-  else
-    echo "Caught interrupt signal or error, cleaning up..."
-  fi
-
-  # 清理临时文件
-  rm -f "$LOCK_FILE" 2>/dev/null
-  # PORT_FILE端口临时文件是registry-proxy服务器管理的文件这里不负责清理，服务器退出时会自动清理
-  #rm -f "$PORT_FILE" 2>/dev/null
-
-  # 停止代理服务器
-  if [ -n "${PROXY_PID:-}" ]; then
-    echo "Stopping proxy server (PID: $PROXY_PID)..."
-    kill -TERM "$PROXY_PID" 2>/dev/null
-    wait "$PROXY_PID" 2>/dev/null || true
-    echo "Proxy server stopped."
-  fi
-
-  # 切换到项目根目录
-  # shellcheck disable=SC2164
-  cd "$PROJECT_ROOT"
-
-  # 清理 npmRegistryServer 配置
-  yarn config unset npmRegistryServer 2>/dev/null || true
-  echo "Cleared npmRegistryServer configuration"
-
-  # 根据退出状态退出
-  exit "$exit_code"
-}
-
-# 注册信号处理
-trap 'cleanup 1' SIGINT SIGTERM EXIT  # 异常退出时调用 cleanup，退出码为 1
-
-# 切换到项目根目录
-# shellcheck disable=SC2164
-cd "$PROJECT_ROOT"
-
-# 使用 yarn dlx 直接运行 registry-proxy，可通过环境变量指定registry-proxy版本号，默认是latest，registry-proxy将会被放入后台运行，并在安装结束后自动退出。
-REGISTRY_PROXY_VERSION="${REGISTRY_PROXY_VERSION:-latest}"
-echo "Starting registry-proxy@$REGISTRY_PROXY_VERSION in the background (logs will be displayed below)..."
-# 下载registry-proxy临时可执行程序并运行  因yarn可能会缓存tarball url的缘故（yarn.lock内<package>.resolution值），这里不得已只能写死本地代理端口地址，以便无论是从缓存获取tarball url还是从代理服务提供的元数据获取tarball url地址都能成功下载tarball文件
-# 但是注意 这个端口不能暴露到外部使用，只允许本地使用，避免不必要的安全隐患 事实上registry-proxy server也是只监听着::1本机端口的。
-yarn dlx com.jimuwd.xian.registry-proxy@"$REGISTRY_PROXY_VERSION" .registry-proxy.yml .yarnrc.yml ~/.yarnrc.yml 40061 &
-PROXY_PID=$!
-
-# 等待代理服务器启动并写入端口，最多 30 秒
-echo "Waiting for proxy server to start (up to 30 seconds)..."
-PORT_FILE="$PROJECT_ROOT/.registry-proxy-port"
-# shellcheck disable=SC2034
-for i in {1..300}; do  # 300 次循环，每次 0.1 秒，总共 30 秒
-  if [ -f "$PORT_FILE" ]; then
-    PROXY_PORT=$(cat "$PORT_FILE")
-    if [ -z "$PROXY_PORT" ]; then
-      echo "Error: Port file $PORT_FILE is empty"
-      cleanup 1
-    fi
-    if nc -z localhost "$PROXY_PORT" 2>/dev/null; then
-      echo "Proxy server is ready on port $PROXY_PORT!"
-      break
-    else
-      # 检查端口是否被占用
-      if netstat -tuln 2>/dev/null | grep -q ":$PROXY_PORT "; then
-        echo "Error: Port $PROXY_PORT is already in use by another process"
-        cleanup 1
-      fi
-    fi
-  fi
-  sleep 0.1
-done
-
-# 检查是否成功启动
-if [ -z "${PROXY_PORT:-}" ] || ! nc -z localhost "$PROXY_PORT" 2>/dev/null; then
-  echo "Error: Proxy server failed to start after 30 seconds"
-  echo "Please check the registry-proxy logs above for more details."
-  cleanup 1
-fi
-
-# 动态设置 npmRegistryServer 为代理地址 注意：yarn对“localhost”域名不友好，请直接使用 [::1]
-yarn config set npmRegistryServer "http://[::1]:$PROXY_PORT"
-echo "Set npmRegistryServer to http://[::1]:$PROXY_PORT"
-
-# 使用动态代理端口运行 yarn install，并捕获错误
-if ! yarn install; then
-  echo "Error: yarn install failed"
-  cleanup 1
-fi
-
-# 正常执行完成，调用 cleanup 并传入退出码 0
-cleanup 0
+  - "[::1]"
 ```
 
-### 5. Set Script Permissions
+### 4. install Script
 
-Ensure the script is executable and commit the permission to version control:
-
-```bash
-chmod +x scripts/install-from-proxy-registries.sh
-git add scripts/install-from-proxy-registries.sh
-git commit -m "Add install script with executable permission"
+use below cmd to automate the proxy setup and dependency installation.
+```shell
+yarn dlx -p com.jimuwd.xian.registry-proxy yarn-install
 ```
 
-### 6. Integrate with Yarn
+### 5. Integrate with Yarn
 
 Update your `package.json` to run the script automatically when `yarn` is executed:
 
 ```json
 {
   "scripts": {
-    "preinstall": "bash scripts/install-from-proxy-registries.sh",
+    "preinstall": "yarn dlx -p com.jimuwd.xian.registry-proxy yarn-install",
     "install": "echo 'Custom install script is running via preinstall, skipping default install.'"
   }
 }

package/dist/server/index.js

@@ -160,7 +160,7 @@ async function fetchFromRegistry(registry, targetUrl, reqFromDownstreamClient, l
         limiter.release();
     }
 }
-async function writeResponseToDownstreamClient(registryInfo, targetUrl, resToDownstreamClient, upstreamResponse, reqFromDownstreamClient, proxyInfo, proxyPort, registryInfos) {
+async function writeResponseToDownstreamClient(registryInfo, targetUrl, resToDownstreamClient, upstreamResponse, reqFromDownstreamClient, proxyInfo, _proxyPort, registryInfos) {
     logger.info(`Writing upstream registry server ${registryInfo.normalizedRegistryUrl}'s ${upstreamResponse.status}${upstreamResponse.statusText ? (' "' + upstreamResponse.statusText + '"') : ''} response to downstream client.`);
     if (!upstreamResponse.ok)
         throw new Error("Only 2xx upstream response is supported");
@@ -174,7 +174,7 @@ async function writeResponseToDownstreamClient(registryInfo, targetUrl, resToDow
             const data = await upstreamResponse.json();
             if (data.versions) { // 处理node依赖包元数据
                 logger.info("Write package meta data application/json response from upstream to downstream", targetUrl);
-                const host = reqFromDownstreamClient.headers.host /*|| `[::1]:${proxyPort}`*/;
+                const host = reqFromDownstreamClient.headers.host /*|| `[::1]:${_proxyPort}`*/;
                 const baseUrl = `${proxyInfo.https ? 'https' : 'http'}://${host}${proxyInfo.basePath === '/' ? '' : proxyInfo.basePath}`;
                 for (const versionKey in data.versions) {
                     const packageVersion = data.versions[versionKey];
@@ -190,10 +190,11 @@ async function writeResponseToDownstreamClient(registryInfo, targetUrl, resToDow
                 logger.info("Write none meta data application/json response from upstream to downstream", targetUrl);
             }
             const bodyData = JSON.stringify(data);
-            resToDownstreamClient.removeHeader('Transfer-Encoding');
-            // 默认是 connection: keep-alive 和 keep-alive: timeout=5，这里直接给它咔嚓掉
-            resToDownstreamClient.setHeader('Connection', 'close');
-            resToDownstreamClient.removeHeader('Keep-Alive');
+            // remove transfer-encoding = chunked header if present, because the content-length is fixed.
+            resToDownstreamClient.removeHeader('transfer-encoding');
+            /* 默认是 connection: keep-alive 和 keep-alive: timeout=5
+            resToDownstreamClient.setHeader('connection', 'close');
+            resToDownstreamClient.removeHeader('Keep-Alive');*/
             resToDownstreamClient.setHeader('content-type', contentType);
             resToDownstreamClient.setHeader('content-length', Buffer.byteLength(bodyData));
             logger.info(`Response to downstream client headers`, JSON.stringify(resToDownstreamClient.getHeaders()), targetUrl);
@@ -207,41 +208,33 @@ async function writeResponseToDownstreamClient(registryInfo, targetUrl, resToDow
             }
             else {
                 // write back to client
-                // 准备通用响应头信息
                 const safeHeaders = new Map();
-                safeHeaders.set("Content-Type", contentType);
                 // 复制所有可能需要的头信息（不包含安全相关的敏感头信息，如access-control-allow-origin、set-cookie、server、strict-transport-security等，这意味着代理服务器向下游客户端屏蔽了这些认证等安全数据）
                 // 也不能包含cf-cache-status、cf-ray（Cloudflare 特有字段）可能干扰客户端解析。
-                const headersToCopy = ['cache-control', 'connection', 'content-encoding', 'content-length', /*'date',*/ 'etag', 'last-modified', 'transfer-encoding', 'vary',];
+                const headersToCopy = ['cache-control', 'etag', 'last-modified', 'vary', 'connection', 'keep-alive', 'content-encoding'];
                 headersToCopy.forEach(header => {
                     const value = upstreamResponse.headers.get(header);
                     if (value)
                         safeHeaders.set(header, value);
                 });
-                // 必须使用 ServerResponse.setHeaders(safeHeaders)来覆盖现有headers而不是ServerResponse.writeHead(status,headers)来合并headers！
-                // 这个坑害我浪费很久事件来调试！
+                // 强制设置二进制流传输的响应头，需要使用 ServerResponse.setHeaders(safeHeaders)来覆盖现有headers而不是ServerResponse.writeHead(status,headers)来合并headers，避免不必要的麻烦
                 resToDownstreamClient.setHeaders(safeHeaders);
-                // 调试代码
-                resToDownstreamClient.removeHeader('content-encoding');
-                resToDownstreamClient.removeHeader('Transfer-Encoding');
+                resToDownstreamClient.setHeader("content-type", contentType);
                 resToDownstreamClient.removeHeader('content-length');
                 resToDownstreamClient.setHeader('transfer-encoding', 'chunked');
-                // 默认是 connection: keep-alive 和 keep-alive: timeout=5，这里直接给它咔嚓掉
-                resToDownstreamClient.removeHeader('connection');
+                /* 这两个header使用upstream透传过来的header即可
                 resToDownstreamClient.setHeader('connection', 'close');
-                resToDownstreamClient.removeHeader('Keep-Alive');
-                resToDownstreamClient.removeHeader('content-type');
-                resToDownstreamClient.setHeader('content-type', contentType);
+                resToDownstreamClient.removeHeader('Keep-Alive');*/
                 logger.info(`Response to downstream client headers`, JSON.stringify(resToDownstreamClient.getHeaders()), targetUrl);
                 // 不再writeHead()而是设置状态码，然后执行pipe操作
                 resToDownstreamClient.statusCode = upstreamResponse.status;
                 resToDownstreamClient.statusMessage = upstreamResponse.statusText;
-                // resToDownstreamClient.writeHead(upstreamResponse.status);
                 // stop pipe when req from client is closed accidentally.
+                // this is good when proxying big stream from upstream to downstream.
                 const cleanup = () => {
                     reqFromDownstreamClient.off('close', cleanup);
                     logger.info(`Req from downstream client is closed, stop pipe from upstream ${targetUrl} to downstream client.`);
-                    // upstreamResponse.body?.unpipe();
+                    upstreamResponse.body?.unpipe();
                 };
                 reqFromDownstreamClient.on('close', cleanup);
                 reqFromDownstreamClient.on('end', () => logger.info("Req from downstream client ends."));
@@ -253,20 +246,15 @@ async function writeResponseToDownstreamClient(registryInfo, targetUrl, resToDow
                 };
                 resToDownstreamClient.on('close', cleanup0);
                 // write back body data (chunked probably)
-                // pipe upstream body to downstream client
+                // pipe upstream body-stream to downstream stream and automatically ends the stream to downstream when upstream stream is ended.
                 upstreamResponse.body.pipe(resToDownstreamClient, { end: true });
                 upstreamResponse.body
                     .on('data', (chunk) => logger.debug(`Chunk transferred from ${targetUrl} to downstream client size=${chunk.length}`))
-                    .on('end', () => {
-                    logger.info(`Upstream server ${targetUrl} response.body ended.`);
-                    // resToDownstreamClient.end();
-                })
+                    .on('end', () => logger.info(`Upstream server ${targetUrl} response.body ended.`))
                     // connection will be closed automatically when all chunk data is transferred (after stream ends).
-                    .on('close', () => {
-                    logger.info(`Upstream server ${targetUrl} closed connection.`);
-                })
+                    .on('close', () => logger.info(`Upstream server ${targetUrl} closed connection.`))
                     .on('error', (err) => {
-                    const errMsg = `Stream error: ${err.message}`;
+                    const errMsg = `Stream error between upstream and registry-proxy server: ${err.message}. Upstream url is ${targetUrl}`;
                     logger.error(errMsg);
                     resToDownstreamClient.destroy(new Error(errMsg, { cause: err, }));
                     reqFromDownstreamClient.destroy(new Error(errMsg, { cause: err, }));
@@ -274,15 +262,15 @@ async function writeResponseToDownstreamClient(registryInfo, targetUrl, resToDow
             }
         }
         else {
-            logger.warn(`Write unsupported content-type=${contentType} response from upstream to downstream ${targetUrl}`);
+            logger.warn(`Write unsupported content-type=${contentType} response from upstream to downstream. Upstream url is ${targetUrl}`);
             const bodyData = await upstreamResponse.text();
-            resToDownstreamClient.removeHeader('Transfer-Encoding');
+            resToDownstreamClient.removeHeader('transfer-encoding');
             // 默认是 connection: keep-alive 和 keep-alive: timeout=5，这里直接给它咔嚓掉
-            resToDownstreamClient.setHeader('Connection', 'close');
+            resToDownstreamClient.setHeader('connection', 'close');
             resToDownstreamClient.removeHeader('Keep-Alive');
             resToDownstreamClient.setHeader('content-type', contentType);
             resToDownstreamClient.setHeader('content-length', Buffer.byteLength(bodyData));
-            logger.info(`Response to downstream client headers`, JSON.stringify(resToDownstreamClient.getHeaders()), targetUrl);
+            logger.info(`Response to downstream client headers`, JSON.stringify(resToDownstreamClient.getHeaders()), `Upstream url is ${targetUrl}`);
             resToDownstreamClient.writeHead(upstreamResponse.status).end(bodyData);
         }
     }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "com.jimuwd.xian.registry-proxy",
-  "version": "1.0.133",
+  "version": "1.0.134",
   "description": "A lightweight npm registry proxy with fallback support",
   "type": "module",
   "main": "dist/server/index.js",