@kne/fastify-file-manager 1.2.4 → 2.0.0-alpha.1

package/README.md

@@ -13,6 +13,24 @@
 npm i --save @kne/fastify-file-manager
 ```
 
+
+### 概述
+
+### 项目概述
+
+Fastify File Manager 是一个基于 Fastify 框架构建的高性能文件管理服务，提供文件上传、存储、检索和管理功能。该项目采用模块化设计，支持多种存储后端，包括本地文件系统和云存储服务（如阿里云OSS、AWS S3等）。
+
+#### 核心特性
+
+- 🚀 **高性能文件处理**：基于 Fastify 的轻量级架构
+- 🔒 **安全认证**：集成 JWT 认证和权限控制
+- 📁 **多存储支持**：可配置本地存储或云存储
+- 🏷️ **文件分类**：通过命名空间(namespace)组织文件
+- 📊 **元数据管理**：记录文件大小、类型、上传时间等信息
+- 🔍 **文件检索**：支持按命名空间查询文件列表
+
+
+
 ### 示例
 
 #### 示例代码
@@ -21,7 +39,272 @@ npm i --save @kne/fastify-file-manager
 
 ### API
 
-| 属性名 | 说明 | 类型 | 默认值 |
-|-----|----|----|-----|
-|     |    |    |     |
+| 参数                          | 类型       | 默认值                                  | 必填 | 描述                                                                  |
+|-----------------------------|----------|--------------------------------------|----|---------------------------------------------------------------------|
+| **基础配置**                    |          |                                      |    |                                                                     |
+| `root`                      | string   | `path.join(process.cwd(), 'static')` | 否  | 文件存储根目录路径                                                           |
+| `namespace`                 | string   | `'default'`                          | 否  | 默认文件分类命名空间                                                          |
+| `prefix`                    | string   | `/api/v${majorVersion}/static`       | 否  | API路由前缀，自动从package.json提取主版本号                                       |
+| `dbTableNamePrefix`         | string   | `'t_file_manager_'`                  | 否  | 数据库表名前缀                                                             |
+| **文件上传**                    |          |                                      |    |                                                                     |
+| `multipart.limits.fileSize` | number   | `524288000` (500MB)                  | 否  | 单个文件最大上传大小                                                          |
+| **静态文件服务**                  |          |                                      |    |                                                                     |
+| `static`                    | object   | `{}`                                 | 否  | 透传[@fastify/static](https://github.com/fastify/fastify-static)的所有配置 |
+| **适配器配置**                   |          |                                      |    |                                                                     |
+| `ossAdapter`                | function | `() => {}`                           | 否  | OSS适配器工厂函数，需返回OSS配置对象                                               |
+| `createAuthenticate`        | function | `() => []`                           | 否  | 认证中间件工厂函数                                                           |
+
+### 配置示例
+
+```javascript
+const options = {
+    root: '/data/uploads',  // 自定义存储目录
+    namespace: 'user_files', // 业务隔离命名空间
+    multipart: {
+        limits: {
+            fileSize: 100 * 1024 * 1024 // 调整为100MB
+        }
+    },
+    ossAdapter: () => {
+        /** 需要注册 '@kne/fastify-aliyun' 插件
+         * fastify.register(require('@kne/fastify-aliyun'), {
+         prefix: `${apiPrefix}/aliyun`,
+         oss: {
+         baseDir: 'leapin-setting',
+         region: fastify.config.OSS_REGION,
+         accessKeyId: fastify.config.OSS_ACCESS_KEY_ID,
+         accessKeySecret: fastify.config.OSS_ACCESS_KEY_SECRET,
+         bucket: fastify.config.OSS_BUCKET
+         }
+         });
+         * */
+        return fastify.aliyun.services.oss;
+    },
+    createAuthenticate: (requiredPermission) => [
+        fastify.jwtVerify,
+        checkPermission(requiredPermission)
+    ]
+}
+```
+
+### 配置说明
+
+1. **版本兼容性**  
+   `prefix` 自动从 `package.json` 提取主版本号（如 `1.2.3` → `/api/v1/static`）
+
+2. **存储目录**
+    - 默认会在项目根目录创建 `static` 文件夹
+    - 生产环境建议设置为绝对路径（如 `/var/www/uploads`）
+
+3. **权限控制**  
+   `createAuthenticate` 应返回 Fastify 钩子数组，典型实现：
+   ```javascript
+   createAuthenticate: (perm) => [
+     fastify.authenticate,
+     (req, reply, done) => {
+       if(!req.user.permissions.includes(perm)) {
+         return reply.code(403).send()
+       }
+       done()
+     }
+   ]
+   ```
+
+### 文件上传接口
+
+#### `POST /api/v1/static/upload`
+
+上传文件到服务器或配置的存储服务
+
+##### 认证要求
+
+- 需要 `file:write` 权限
+- 通过 JWT 认证
+
+##### 请求参数
+
+| 参数        | 位置    | 类型     | 必填 | 描述       | 示例             |
+|-----------|-------|--------|----|----------|----------------|
+| namespace | query | string | 否  | 文件分类命名空间 | `user-avatars` |
+| file      | body  | file   | 是  | 要上传的文件   | -              |
+
+##### 请求示例
+
+```bash
+curl -X POST \
+  -H "Authorization: Bearer <JWT_TOKEN>" \
+  -F "file=@test.jpg" \
+  "http://localhost:3000/api/v1/static/upload?namespace=user-avatars"
+```
+
+##### 响应状态码
+
+| 状态码 | 描述        |
+|-----|-----------|
+| 200 | 文件上传成功    |
+| 400 | 无效请求或缺少文件 |
+| 401 | 未授权访问     |
+| 413 | 文件大小超过限制  |
+| 500 | 服务器错误     |
+
+---
+
+### 文件列表查询
+
+#### `POST /api/v1/static/file-list`
+
+查询指定条件下的文件列表（支持分页和筛选）
+
+#### 认证要求
+
+- 需要 `file:mange` 权限
+- 通过 JWT 认证
+
+#### 请求参数
+
+##### Body 参数 (application/json)
+
+| 参数                 | 类型       | 必填 | 描述                  | 默认值 | 示例                |
+|--------------------|----------|----|---------------------|-----|-------------------|
+| `perPage`          | number   | 否  | 每页显示数量              | 20  | `10`              |
+| `currentPage`      | number   | 否  | 当前页码                | 1   | `2`               |
+| `filter`           | object   | 否  | 筛选条件                | -   | -                 |
+| `filter.namespace` | string   | 否  | 文件分类命名空间            | -   | `"user-docs"`     |
+| `filter.size`      | number[] | 否  | 文件大小范围[min,max]（字节） | -   | `[1024, 1048576]` |
+| `filter.filename`  | string   | 否  | 文件名模糊匹配             | -   | `"report.pdf"`    |
+
+##### 请求示例
+
+```bash
+curl -X POST \
+  -H "Authorization: Bearer <JWT_TOKEN>" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "perPage": 10,
+    "currentPage": 2,
+    "filter": {
+      "namespace": "project-files",
+      "size": [1024, 1048576],
+      "filename": "report"
+    }
+  }' \
+  "http://localhost:3000/api/v1/static/file-list"
+```
+
+##### 响应状态码
+
+| 状态码 | 描述      |
+|-----|---------|
+| 200 | 查询成功    |
+| 400 | 参数验证失败  |
+| 401 | 未授权访问   |
+| 500 | 服务器内部错误 |
+
+##### 筛选逻辑说明
+
+1. **命名空间筛选**：精确匹配传入的 `namespace` 值
+2. **文件大小筛选**：
+    - 数组第一个元素为最小值
+    - 数组第二个元素为最大值
+3. **文件名筛选**：使用 `LIKE %value%` 模糊匹配
+
+##### 分页说明
+
+- 分页计算：`offset = (currentPage - 1) * perPage`
+- 建议每页数量不超过 100 条
+- 页码从 1 开始计数
+
+---
+
+### 文件删除接口
+
+#### `DELETE /api/v1/static/:fileId`
+
+删除指定文件
+
+##### 认证要求
+
+- 需要 `file:write` 权限
+
+##### 请求参数
+
+| 参数     | 位置   | 类型     | 必填 | 描述       | 示例       |
+|--------|------|--------|----|----------|----------|
+| fileId | path | string | 是  | 要删除的文件ID | `abc123` |
+
+##### 请求示例
+
+```bash
+curl -X DELETE \
+  -H "Authorization: Bearer <JWT_TOKEN>" \
+  "http://localhost:3000/api/v1/static/abc123"
+```
+
+##### 响应状态码
+
+| 状态码 | 描述     |
+|-----|--------|
+| 204 | 文件删除成功 |
+| 404 | 文件不存在  |
+| 500 | 服务器错误  |
+
+---
+
+### 文件信息接口
+
+#### `GET /api/v1/static/:fileId`
+
+获取文件详细信息
+
+##### 认证要求
+
+- 需要 `file:read` 权限
+
+##### 请求参数
+
+| 参数     | 位置   | 类型     | 必填 | 描述       | 示例       |
+|--------|------|--------|----|----------|----------|
+| fileId | path | string | 是  | 要查询的文件ID | `abc123` |
+
+##### 请求示例
+
+```bash
+curl -X GET \
+  -H "Authorization: Bearer <JWT_TOKEN>" \
+  "http://localhost:3000/api/v1/static/abc123"
+```
+
+##### 响应示例
+
+```json
+{
+  "id": "abc123",
+  "name": "document.pdf",
+  "size": 102400,
+  "mimeType": "application/pdf",
+  "createdAt": "2023-01-01T00:00:00Z",
+  "url": "/api/v1/static/file/documents/abc123.pdf"
+}
+```
+
+---
+
+### 接口使用说明
+
+1. **认证方式**：
+    - 所有接口都需要在 Header 中添加 `Authorization: Bearer <JWT_TOKEN>`
+    - JWT 需要包含相应的权限声明
+
+2. **命名空间规则**：
+    - 命名空间支持字母、数字和下划线组合
+    - 未指定时使用默认命名空间
+
+3. **文件大小限制**：
+    - 默认最大 500MB
+    - 可在服务端配置中调整
+
+4. **错误处理**：
+    - 所有错误响应都包含标准格式的 error 字段
+    - 客户端应检查状态码而非仅依赖响应体
 
+> 注意：实际 API 路径前缀会根据配置的 `prefix` 参数变化

package/index.js

@@ -1,5 +1,5 @@
 const fp = require('fastify-plugin');
-const path = require('path');
+const path = require('node:path');
 const fs = require('fs-extra');
 const packageJson = require('./package.json');
 
@@ -31,6 +31,7 @@ module.exports = fp(
     fastify.register(require('@kne/fastify-namespace'), {
       name: 'fileManager',
       options,
+      singleton: true,
       modules: [
         [
           'models',

package/libs/controllers/index.js

@@ -2,186 +2,147 @@ const fp = require('fastify-plugin');
 
 module.exports = fp(async (fastify, options) => {
   const { services } = fastify.fileManager;
-  fastify.post(
-    `${options.prefix}/upload`,
-    {
-      onRequest: options.createAuthenticate('file:write'),
-      schema: {
-        query: {
-          type: 'object',
-          properties: {
-            namespace: { type: 'string' }
-          }
+  fastify.post(`${options.prefix}/upload`, {
+    onRequest: options.createAuthenticate('file:write'), schema: {
+      summary: '上传文件', description: '上传单个文件到服务器或配置的存储服务', query: {
+        type: 'object', properties: {
+          namespace: { type: 'string', description: '文件分类命名空间' }
         }
       }
-    },
-    async request => {
-      const file = await request.file();
-      if (!file) {
-        throw new Error('不能获取到上传文件');
-      }
-      //1. 保存到服务器目录 2.对接oss
-      return await services.fileRecord.uploadToFileSystem({
-        file,
-        namespace: request.query.namespace || options.namespace
-      });
     }
-  );
+  }, async request => {
+    const file = await request.file();
+    if (!file) {
+      throw new Error('不能获取到上传文件');
+    }
+    return await services.fileRecord.uploadToFileSystem({
+      file, namespace: request.query.namespace || options.namespace
+    });
+  });
 
-  fastify.get(
-    `${options.prefix}/file-url/:id`,
-    {
-      onRequest: options.createAuthenticate('file:read'),
-      schema: {
-        params: {
-          type: 'object',
-          required: ['id'],
-          properties: {
-            id: { type: 'string' }
-          }
+  fastify.post(`${options.prefix}/uploadUrl`, {
+    onRequest: options.createAuthenticate('file:write'), schema: {
+      summary: '上传URL文件', description: '上传单个文件到服务器或配置的存储服务', body: {
+        type: 'object', properties: {
+          url: { type: 'string', description: '文件url' },
+          namespace: { type: 'string', description: '文件分类命名空间' }
         }
       }
-    },
-    async request => {
-      const { id } = request.params;
-      return await services.fileRecord.getFileUrl({ id });
     }
-  );
+  }, async request => {
+    return await services.fileRecord.uploadFromUrl({
+      url: request.body.url, namespace: request.body.namespace || options.namespace
+    });
+  });
 
-  fastify.get(
-    `${options.prefix}/file-id/:id`,
-    {
-      onRequest: options.createAuthenticate('file:read'),
-      schema: {
-        query: {
-          type: 'object',
-          properties: {
-            attachment: { type: 'boolean' },
-            filename: { type: 'string' }
-          }
-        },
-        params: {
-          type: 'object',
-          required: ['id'],
-          properties: {
-            id: { type: 'string' }
-          }
+  fastify.get(`${options.prefix}/file-url/:id`, {
+    onRequest: options.createAuthenticate('file:read'), schema: {
+      summary: '获取文件url', description: '获取文件url', params: {
+        type: 'object', required: ['id'], properties: {
+          id: { type: 'string', description: '文件id' }
         }
       }
-    },
-    async (request, reply) => {
-      const { id } = request.params;
-      const { attachment, filename: targetFilename } = request.query;
-      const { filePath, targetFile, filename, mimetype, ...props } = await services.fileRecord.getFileInfo({
-        id
-      });
-      if (targetFile) {
-        const outputFilename = encodeURIComponent(targetFilename || filename);
-        reply.header('Content-Type', mimetype);
-        reply.header('Content-Disposition', attachment ? `attachment; filename="${outputFilename}"` : `filename="${outputFilename}"`);
-        return reply.send(targetFile);
+    }
+  }, async request => {
+    const { id } = request.params;
+    return await services.fileRecord.getFileUrl({ id });
+  });
+
+  fastify.get(`${options.prefix}/file-id/:id`, {
+    onRequest: options.createAuthenticate('file:read'), schema: {
+      summary: '获取文件信息', description: '获取文件信息', query: {
+        type: 'object', properties: {
+          attachment: { type: 'boolean', description: '是否下载' },
+          filename: { type: 'string', description: '下载文件名' }
+        }
+      }, params: {
+        type: 'object', required: ['id'], properties: {
+          id: { type: 'string', description: '文件id' }
+        }
       }
-      return attachment ? reply.download(filePath, targetFilename || filename) : reply.sendFile(filePath);
     }
-  );
+  }, async (request, reply) => {
+    const { id } = request.params;
+    const { attachment, filename: targetFilename } = request.query;
+    const { filePath, targetFile, filename, mimetype, ...props } = await services.fileRecord.getFileInfo({
+      id
+    });
+    if (targetFile) {
+      const outputFilename = encodeURIComponent(targetFilename || filename);
+      reply.header('Content-Type', mimetype);
+      reply.header('Content-Disposition', attachment ? `attachment; filename="${outputFilename}"` : `filename="${outputFilename}"`);
+      return reply.send(targetFile);
+    }
+    return attachment ? reply.download(filePath, targetFilename || filename) : reply.sendFile(filePath);
+  });
 
-  fastify.post(
-    `${options.prefix}/file-list`,
-    {
-      onRequest: options.createAuthenticate('file:mange'),
-      schema: {
-        body: {
-          type: 'object',
-          properties: {
-            perPage: { type: 'number' },
-            currentPage: { type: 'number' },
-            filter: {
-              type: 'object',
-              properties: {
-                namespace: { type: 'string' },
-                size: { type: 'array', items: { type: 'number' } },
-                filename: { type: 'string' }
-              }
+  fastify.post(`${options.prefix}/file-list`, {
+    onRequest: options.createAuthenticate('file:mange'), schema: {
+      summary: '获取文件列表', description: '查询指定命名空间下的文件列表', body: {
+        type: 'object', properties: {
+          perPage: { type: 'number', description: '每页数量' },
+          currentPage: { type: 'number', description: '当前页数' },
+          filter: {
+            type: 'object', properties: {
+              namespace: { type: 'string', description: '文件分类命名空间' },
+              size: { type: 'array', items: { type: 'number' }, description: '文件大小' },
+              filename: { type: 'string', description: '文件名' }
             }
           }
         }
       }
-    },
-    async request => {
-      const { filter, perPage, currentPage } = Object.assign(
-        {},
-        {
-          perPage: 20,
-          currentPage: 1
-        },
-        request.body
-      );
-      return await services.fileRecord.getFileList({
-        filter,
-        perPage,
-        currentPage
-      });
     }
-  );
+  }, async request => {
+    const { filter, perPage, currentPage } = Object.assign({}, {
+      perPage: 20, currentPage: 1
+    }, request.body);
+    return await services.fileRecord.getFileList({
+      filter, perPage, currentPage
+    });
+  });
 
   // Replace file
 
-  fastify.post(
-    `${options.prefix}/replace-file`,
-    {
-      onRequest: options.createAuthenticate('file:mange'),
-      schema: {
-        type: 'object',
-        properties: {
-          id: { type: 'string' }
+  fastify.post(`${options.prefix}/replace-file`, {
+    onRequest: options.createAuthenticate('file:mange'), schema: {
+      summary: '替换文件', description: '替换文件', query: {
+        type: 'object', properties: {
+          id: { type: 'string', description: '文件id' }
         }
       }
-    },
-    async request => {
-      const file = await request.file();
-      if (!file) {
-        throw new Error('不能获取到上传文件');
-      }
-      return await services.fileRecord.uploadToFileSystem({ id: request.query.id, file });
     }
-  );
+  }, async request => {
+    const file = await request.file();
+    if (!file) {
+      throw new Error('不能获取到上传文件');
+    }
+    return await services.fileRecord.uploadToFileSystem({ id: request.query.id, file });
+  });
 
-  fastify.post(
-    `${options.prefix}/rename-file`,
-    {
-      onRequest: options.createAuthenticate('file:mange'),
-      schema: {
-        type: 'object',
-        properties: {
-          id: { type: 'string' },
-          filename: { type: 'string' }
+  fastify.post(`${options.prefix}/rename-file`, {
+    onRequest: options.createAuthenticate('file:mange'), schema: {
+      summary: '重命名文件', description: '重命名文件', body: {
+        type: 'object', properties: {
+          id: { type: 'string', description: '文件id' }, filename: { type: 'string', description: '新文件名' }
         }
       }
-    },
-    async request => {
-      await services.fileRecord.renameFile(request.body);
-      return {};
     }
-  );
+  }, async request => {
+    await services.fileRecord.renameFile(request.body);
+    return {};
+  });
 
-  fastify.post(
-    `${options.prefix}/delete-files`,
-    {
-      onRequest: options.createAuthenticate('file:mange'),
-      schema: {
-        body: {
-          type: 'object',
-          required: ['ids'],
-          properties: {
-            ids: { type: 'array', items: { type: 'string' } }
-          }
+  fastify.post(`${options.prefix}/delete-files`, {
+    onRequest: options.createAuthenticate('file:mange'), schema: {
+      summary: '删除文件', description: '删除文件', body: {
+        type: 'object', required: ['ids'], properties: {
+          ids: { type: 'array', items: { type: 'string' }, description: '文件id列表' }
         }
       }
-    },
-    async request => {
-      const { ids } = request.body;
-      await services.fileRecord.deleteFiles({ ids });
-      return {};
     }
-  );
+  }, async request => {
+    const { ids } = request.body;
+    await services.fileRecord.deleteFiles({ ids });
+    return {};
+  });
 });

package/libs/services/file-record.js

@@ -9,20 +9,35 @@ module.exports = fp(async (fastify, options) => {
   const { Op } = fastify.sequelize.Sequelize;
   const uploadToFileSystem = async ({ id, file, namespace }) => {
     const { filename, encoding, mimetype } = file;
-    const buffer = await file.toBuffer();
     const hash = crypto.createHash('md5');
-    hash.update(buffer);
-    const digest = hash.digest('hex');
     const extension = path.extname(filename);
+    let buffer = Buffer.alloc(0);
+
+    // 使用流处理文件数据
+    const stream = file.createReadStream();
+    for await (const chunk of stream) {
+      hash.update(chunk);
+      buffer = Buffer.concat([buffer, chunk]);
+    }
+    const digest = hash.digest('hex');
 
     let storageType;
     const ossServices = options.ossAdapter();
     if (typeof ossServices.uploadFile === 'function') {
-      await ossServices.uploadFile({ file: buffer, filename: `${digest}${extension}` });
+      // 使用流上传到OSS
+      const uploadStream = file.createReadStream();
+      await ossServices.uploadFileStream({ stream: uploadStream, filename: `${digest}${extension}` });
       storageType = 'oss';
     } else {
+      // 使用流写入本地文件
       const filepath = path.resolve(options.root, `${digest}${extension}`);
-      await fs.writeFile(filepath, buffer);
+      const writeStream = fs.createWriteStream(filepath);
+      const readStream = file.createReadStream();
+      await new Promise((resolve, reject) => {
+        readStream.pipe(writeStream)
+          .on('finish', resolve)
+          .on('error', reject);
+      });
       storageType = 'local';
     }
 
@@ -56,6 +71,30 @@ module.exports = fp(async (fastify, options) => {
     return Object.assign({}, outputFile.get({ plain: true }), { id: outputFile.uuid });
   };
 
+  const uploadFromUrl = async ({ id, url, namespace }) => {
+    const response = await fetch(url);
+    if (!response.ok) {
+      throw new Error('下载文件失败');
+    }
+    const chunks = [];
+    for await (const chunk of response.body) {
+      chunks.push(chunk);
+    }
+    const buffer = Buffer.concat(chunks);
+    const tempFile = {
+      filename: path.basename(url).split('?')[0],
+      mimetype: response.headers.get('content-type'),
+      encoding: 'binary',
+      createReadStream: () => {
+        const readable = new require('stream').Readable();
+        readable.push(buffer);
+        readable.push(null);
+        return readable;
+      }
+    };
+    return await uploadToFileSystem({ id, file: tempFile, namespace });
+  };
+
   const getFileUrl = async ({ id, namespace }) => {
     const file = await models.fileRecord.findOne({
       where: { uuid: id }
@@ -158,5 +197,9 @@ module.exports = fp(async (fastify, options) => {
     await file.save();
   };
 
-  services.fileRecord = { uploadToFileSystem, getFileUrl, getFileInfo, getFileList, deleteFiles, renameFile };
+  Object.assign(services, {
+    uploadToFileSystem, uploadFromUrl, getFileUrl, getFileInfo, getFileList, deleteFiles, renameFile,
+    // 兼容之前api，后面可能会删掉
+    fileRecord: { uploadToFileSystem, uploadFromUrl, getFileUrl, getFileInfo, getFileList, deleteFiles, renameFile }
+  });
 });

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@kne/fastify-file-manager",
-  "version": "1.2.4",
+  "version": "2.0.0-alpha.1",
   "description": "用于管理静态文件上传查看等",
   "main": "index.js",
   "scripts": {
@@ -32,20 +32,23 @@
   },
   "homepage": "https://github.com/kne-union/fastify-file-manager#readme",
   "devDependencies": {
-    "@fastify/env": "^4.4.0",
+    "@fastify/env": "^5.0.2",
     "@kne/fastify-aliyun": "^1.1.1",
     "@kne/fastify-sequelize": "^2.0.1",
-    "fastify": "^4.27.0",
+    "fastify": "^5.3.2",
     "husky": "^9.0.11",
     "prettier": "^3.2.5",
     "qs": "^6.12.3",
     "sqlite3": "^5.1.7"
   },
+  "peerDependencies": {
+    "@kne/fastify-namespace": "*",
+    "@kne/fastify-sequelize": "*",
+    "fastify-plugin": ">=5"
+  },
   "dependencies": {
-    "@fastify/multipart": "^8.2.0",
-    "@fastify/static": "^7.0.4",
-    "@kne/fastify-namespace": "^0.1.0",
-    "fastify-plugin": "^4.5.1",
+    "@fastify/multipart": "^9.0.3",
+    "@fastify/static": "^8.1.1",
     "fs-extra": "^11.2.0",
     "http-errors": "^2.0.0"
   }