@dan-uni/dan-any 1.4.7 → 2.0.0

package/README.md

@@ -1,67 +1,111 @@
-# Dan-Any 弹幕随心转
+# @dan-uni/dan-any
 
-[![Publish @dan-uni/dan-any](https://github.com/ani-uni/danuni/actions/workflows/npm-dan-any.yml/badge.svg)](https://github.com/ani-uni/danuni/actions/workflows/npm-dan-any.yml)
+`@dan-uni/dan-any` (V2) 是一个弹幕转换与处理库，用于在不同平台格式之间导入、导出与统一处理弹幕数据。
 
-## Features
+它的核心思路是：
+- 用 `Adapter` 把外部格式导入为统一数据
+- 用 `Transformer` 把统一数据导出为目标格式
+- 用 `Plugin` 在处理中间态（`UniChunk`）上做增强/清洗/统计
 
-- 多种弹幕格式之间互相转换
-- 对多平台弹幕进行统一格式化
-- 获取一组弹幕中的共通值
-- 对弹幕按一定key值进行分组
-- 合并较短时间内出现的重复弹幕
-- 管道式插件系统
+## 功能概览
 
-## Formats 支持转换的格式
+### Formats 支持转换的格式
 
 `pb`指`protobuf`格式(grpc协议下的默认传输格式)
 
-- [x] DanUni(json,pb)
-- [x] bili(普通+高级弹幕,xml) `双向`
-- [x] bili(普通+高级弹幕,pb) `正向`
-- [x] bili(指令弹幕,pb) `正向`
-- [x] dplayer
-- [x] artplayer
-- [x] 弹弹Play
-- [x] ASS `双向(部分支持，见下)`
+- [x] DanUni(json,pb): `DanuniJsonAdapter` `DanuniJsonTransformerConfigurator` `DanuniPbAdapter` `DanuniPbTransformer`
+- [x] bili(普通+高级弹幕,xml) `双向`: `BiliXmlAdapter` `BiliXmlTransformerConfigurator`
+- [x] bili(普通+高级弹幕,pb) `正向`: `BiliGrpcAdapter`
+- [x] bili(指令弹幕,pb) `正向`: `BiliCommandGrpcAdapter`
+- [x] dplayer: `DplayerAdapter` `DplayerTransformer`
+- [x] artplayer: `ArtplayerAdapter` `ArtplayerTransformer`
+- [x] 弹弹Play: `DdplayAdapter` `DdplayTransformer`
+- [x] [ASS(`@dan-uni/dan-any-ext-ass`)](https://github.com/ani-uni/dan-any-ext-ass) `双向(部分支持，仅该库生成的ass文件支持还原)`
 
-## Plguins 插件
+### Plugins 常用插件
 
-插件的使用方法参见对应路径下README.md  
+- `MergePluginConfigurator`（合并重复弹幕）
+- `DetaoluPluginConfigurator` (基于pakku.js的弹幕过滤器): 由 [@dan-uni/dan-any-plugin-detaolu](https://github.com/ani-uni/dan-any-plugin-detaolu) 提供
+- `DowngradeAdvancedPluginConfigurator`（高级弹幕降级）
+- `GetStatsTransformerConfigurator`（统计输出）
+- `CountTransformer`（数量统计）
 
-- [dan-any-plugin-detaolu](https://github.com/ani-uni/danuni/tree/master/packages/dan-any-plugin-detaolu): 基于pakku.js的弹幕过滤器
-- `src/plugins`下的插件 `bili`(B站弹幕相关) `stats`(统计相关)，内部附有使用文档
+## 安装
 
-## 转换提示
+```bash
+vp add @dan-uni/dan-any
+bun add @dan-uni/dan-any
+pnpm add @dan-uni/dan-any
+```
 
-使用该库可以不失去任何关键信息地导入常见格式的弹幕，并已附加字段的形式保留部分弹幕格式专有的信息；但将弹幕转换为第三方格式时，由于其设计上无法封装附加信息，会导致不可避免的信息损失。  
-如果需要以最为可靠的方式传送弹幕，请尽量使用 `danuni.json` / `danuni.binpb` 进行导出分发。  
+## 快速开始
 
-由于设计上想尽可能多地保留原始信息，将弹幕转换为第三方格式的弹幕时，可能在非常用字段上与官方格式不完全相同，若有极强的兼容性需求，请手动开启对应转换函数的兼容性选项。  
+详细使用方法可以参考库中的测试文件。
 
-*致开发者： 若导入后检测到 `UniPool.info.fromConverted` 为true ，建议提示用户不要再对该弹幕文件再次进行转换，否则可能导致解析错误*  
+### 1) 从 Bilibili XML 导入，再导出为 Danuni JSON
 
-## 特殊字段提示
+```ts
+import { UniDB } from '@dan-uni/dan-any/core'
+import {
+	BiliXmlAdapter,
+	DanuniJsonTransformerConfigurator,
+} from '@dan-uni/dan-any/adapters'
 
-### ASS Raw 字段
+const xml = `...bili xml弹幕文本...`
 
-为便于由ASS格式还原，由本工具生成的ASS弹幕格式中，包含以下字段：`RawCompressType` `RawBaseType` `Raw`  
-根据其内容可以还原原始转换数据。
-同时本工具兼容了`justorez/biliy`[^1]生成的ASS，并支持了其暂未实现的还原功能。  
+const udb = await new UniDB().init()
+const chunk = await udb.import(BiliXmlAdapter(xml))
+const json = await chunk.export(DanuniJsonTransformerConfigurator({ minify: true }))
 
-## TODO
+console.log(json)
+await udb.close()
+```
 
-- [ ] 完善弹幕降级显示功能(使高级弹幕有损转换为普通弹幕)
-- [ ] 完善使用文档
-- [ ] 支持chpt弹幕转换为Artplayer章节(artplayer-plugin-chapter)
-- [ ] 使用zod重写类型系统
+### 2) 在处理链中使用插件
 
-## Awesome Projects
+```ts
+import { mergePluginConfigurator } from '@dan-uni/dan-any/plugins'
+import { DanuniJsonTransformerConfigurator } from '@dan-uni/dan-any/adapters'
 
-- [HengXin666/BiLiBiLi_DanMu_Crawling](https://github.com/HengXin666/BiLiBiLi_DanMu_Crawling)
+const merged = await chunk.plugin(mergePluginConfigurator(10))
+const result = await merged.export(DanuniJsonTransformerConfigurator({ minify: true }))
 
-## License 许可证
+console.log(result)
+```
 
-Released under the GNU LESSER GENERAL PUBLIC LICENSE (LGPL) 3.0.  
-本库采用 LGPL 3.0 or later 许可证发布。  
+### 3) PB 双向示例（导出再导入）
 
-[^1]: 对于`src/ass-gen`内的文件，使用了[justorez/biliy](https://github.com/justorez/biliy)的代码(MIT LICENSE)，并使用了AI进行修改与修复，同时支持了更优的Raw还原功能。  
+```ts
+import { DanuniPbTransformer, DanuniPbAdapter } from '@dan-uni/dan-any/adapters'
+
+const pb = await chunk.export(DanuniPbTransformer)
+const reimported = await udb.import(DanuniPbAdapter(pb))
+```
+
+### 高级: 提供兼容的drizzle+pglite实例接入自己的数据库
+
+详见 测试文件 `tests/db.test.ts`.
+
+## 模块入口
+
+包已提供以下子路径导出：
+- `@dan-uni/dan-any`（聚合导出）
+- `@dan-uni/dan-any/adapters`
+- `@dan-uni/dan-any/core`
+- `@dan-uni/dan-any/plugins`
+- `@dan-uni/dan-any/utils`
+- `@dan-uni/dan-any/core/db/schema`
+- `@dan-uni/dan-any/core/db/utils`
+
+## 开发
+
+```bash
+vp install
+vp check
+vp test
+vp pack
+```
+
+## 许可
+
+`LGPL-3.0-or-later`

package/dist/adapters.d.mts

@@ -0,0 +1,2 @@
+import { A as enumModeCodec, C as BiliGrpcAdapter, D as DanuniJsonAdapter, E as DanuniPbTransformer, O as DanuniJsonTransformerConfigurator, S as BiliCommonParser, T as DanuniPbAdapter, _ as BiliXmlAdapter, a as UDanmaku, b as BiliCommonBuilder, c as defineTransformer, d as DplayerAdapter, f as DplayerTransformer, g as ArtplayerTransformer, h as ArtplayerAdapter, i as TransformerInput, j as enumPoolCodec, k as enumAttrsCodec, m as DdplayTransformer, n as Plugin, o as defineAdapter, p as DdplayAdapter, r as Transformer, s as definePlugin, t as Adapter, u as transformerInput2Danmakus, v as BiliXmlTransformerConfigurator, w as BiliCommandGrpcAdapter, x as BiliCommonModeRec, y as BiliUpAdapter } from "./index-vAs4qV9A.mjs";
+export { Adapter, ArtplayerAdapter, ArtplayerTransformer, BiliCommandGrpcAdapter, BiliCommonBuilder, BiliCommonModeRec, BiliCommonParser, BiliGrpcAdapter, BiliUpAdapter, BiliXmlAdapter, BiliXmlTransformerConfigurator, DanuniJsonAdapter, DanuniJsonTransformerConfigurator, DanuniPbAdapter, DanuniPbTransformer, DdplayAdapter, DdplayTransformer, DplayerAdapter, DplayerTransformer, Plugin, Transformer, TransformerInput, UDanmaku, defineAdapter, definePlugin, defineTransformer, enumAttrsCodec, enumModeCodec, enumPoolCodec, transformerInput2Danmakus };

package/dist/adapters.mjs

@@ -0,0 +1 @@
+import{A as e,C as t,D as n,E as r,O as i,S as a,T as o,_ as s,b as c,c as l,d as u,f as d,g as f,h as p,j as m,k as h,l as g,m as _,p as v,s as y,u as b,v as x,w as S,x as C,y as w}from"./core-D7LMAB5h.mjs";export{p as ArtplayerAdapter,f as ArtplayerTransformer,S as BiliCommandGrpcAdapter,c as BiliCommonBuilder,C as BiliCommonModeRec,a as BiliCommonParser,t as BiliGrpcAdapter,w as BiliUpAdapter,s as BiliXmlAdapter,x as BiliXmlTransformerConfigurator,n as DanuniJsonAdapter,i as DanuniJsonTransformerConfigurator,o as DanuniPbAdapter,r as DanuniPbTransformer,v as DdplayAdapter,_ as DdplayTransformer,u as DplayerAdapter,d as DplayerTransformer,y as defineAdapter,l as definePlugin,g as defineTransformer,h as enumAttrsCodec,e as enumModeCodec,m as enumPoolCodec,b as transformerInput2Danmakus};

package/dist/chunk-0Lt9GpW0.mjs

@@ -0,0 +1 @@
+var e=Object.defineProperty,t=(t,n)=>{let r={};for(var i in t)e(r,i,{get:t[i],enumerable:!0});return n||e(r,Symbol.toStringTag,{value:`Module`}),r};export{t};

package/dist/core/db/schema.d.mts

@@ -0,0 +1,2 @@
+import { _ as onConflictDoUpdateSet, a as PoolSchema, c as chunk2danmakusZod, d as danmakus, f as danmakusInsertZod, g as onConflictDoUpdate, h as modeEnum, i as ModeSchema, l as chunks, m as dmAttrEnum, n as DanmakusInsert, o as baseRelations, p as danmakusSelectZod, r as DanmakusSelect, s as chunk2danmakus, t as DMAttrSchema, u as chunksZod, v as poolEnum, y as relations } from "../../schema-BuenbDx9.mjs";
+export { DMAttrSchema, DanmakusInsert, DanmakusSelect, ModeSchema, PoolSchema, baseRelations, chunk2danmakus, chunk2danmakusZod, chunks, chunksZod, danmakus, danmakusInsertZod, danmakusSelectZod, dmAttrEnum, modeEnum, onConflictDoUpdate, onConflictDoUpdateSet, poolEnum, relations };

package/dist/core/db/schema.mjs

@@ -0,0 +1,2 @@
+import{defineRelations as e,defineRelationsPart as t,sql as n}from"drizzle-orm";import*as r from"drizzle-orm/pg-core";import{z as i}from"zod";import{createInsertSchema as a}from"drizzle-orm/zod";const o=r.pgEnum(`mode`,[`Normal`,`Bottom`,`Top`,`Reverse`,`Ext`]),s=r.pgEnum(`pool`,[`Def`,`Sub`,`Adv`,`Ix`]),c=r.pgEnum(`dm_attr`,[`Protect`,`FromLive`,`HighLike`,`Compatible`,`Reported`,`Unchecked`,`HasEvent`,`Hide`]),l=r.pgTable(`danmakus`,{SOID:r.text().notNull(),DMID:r.text().primaryKey(),progress:r.integer().notNull(),mode:o().notNull(),fontsize:r.smallint().notNull(),color:r.integer().notNull(),senderID:r.text().notNull(),content:r.text().notNull(),ctime:r.timestamp().notNull(),weight:r.smallint().notNull(),pool:s().notNull(),attr:c().array().notNull(),platform:r.text(),extra:r.jsonb().$type()}),u=a(l),d=a(l),f=r.pgTable(`chunks`,{id:r.serial().primaryKey(),fromConverted:r.boolean().notNull().default(!1),tmp:r.boolean().notNull().default(!1)}),p=a(f),m=r.pgTable(`chunk_danmakus`,{id:r.bigserial({mode:`bigint`}).primaryKey(),chunkID:r.integer().references(()=>f.id).notNull(),DMID:r.text().references(()=>l.DMID).notNull()}),h=a(m),g=e({danmakus:l,chunks:f,chunk2danmakus:m}),_=t({danmakus:l,chunks:f,chunk2danmakus:m},e=>({chunks:{danmakus:e.many.danmakus({from:e.chunks.id.through(e.chunk2danmakus.chunkID),to:e.danmakus.DMID.through(e.chunk2danmakus.DMID)})}})),v={danmakus:Object.fromEntries(Object.keys(l).map(e=>[e,n.raw(`excluded."${e}"`)]))},y={danmakus:{target:[l.DMID],set:v.danmakus}},b=i.enum(l.mode.enumValues),x=i.enum(l.pool.enumValues),S=i.enum(l.attr.enumValues);export{S as DMAttrSchema,b as ModeSchema,x as PoolSchema,g as baseRelations,m as chunk2danmakus,h as chunk2danmakusZod,f as chunks,p as chunksZod,l as danmakus,u as danmakusInsertZod,d as danmakusSelectZod,c as dmAttrEnum,o as modeEnum,y as onConflictDoUpdate,v as onConflictDoUpdateSet,s as poolEnum,_ as relations};
+//# sourceMappingURL=schema.mjs.map

package/dist/core/db/schema.mjs.map

@@ -0,0 +1 @@
+{"version":3,"file":"schema.mjs","names":[],"sources":["../../../src/core/db/schema.ts"],"sourcesContent":["import { defineRelations, defineRelationsPart, sql } from \"drizzle-orm\";\nimport * as t from \"drizzle-orm/pg-core\";\nimport type { Extra as UniDanExtra } from \"@/core/dm-extra.ts\";\nimport { z } from \"zod\";\nimport { createInsertSchema } from \"drizzle-orm/zod\";\nimport type { Simplify } from \"type-fest\";\n\nexport const modeEnum = t.pgEnum(\"mode\", [\"Normal\", \"Bottom\", \"Top\", \"Reverse\", \"Ext\"]);\nexport const poolEnum = t.pgEnum(\"pool\", [\"Def\", \"Sub\", \"Adv\", \"Ix\"]);\nexport const dmAttrEnum = t.pgEnum(\"dm_attr\", [\n  \"Protect\",\n  \"FromLive\",\n  \"HighLike\",\n  \"Compatible\",\n  \"Reported\",\n  \"Unchecked\",\n  \"HasEvent\",\n  \"Hide\",\n]);\n\n// bsnapshot的弹幕不设置默认值，尽量遵循上传组的原始数据(保证所有重要值均有确定)\nexport const danmakus = t.pgTable(\"danmakus\", {\n  SOID: t.text().notNull(),\n  // .references(() => pools.SOID),\n  DMID: t.text().primaryKey(), // 该值在此直接map到dan-any的DMID，为唯一值\n  progress: t.integer().notNull(), // 毫秒\n  mode: modeEnum().notNull(),\n  fontsize: t.smallint().notNull(),\n  color: t.integer().notNull(),\n  senderID: t.text().notNull(),\n  content: t.text().notNull(),\n  ctime: t.timestamp().notNull(),\n  weight: t.smallint().notNull(),\n  pool: poolEnum().notNull(),\n  attr: dmAttrEnum().array().notNull(),\n  platform: t.text(),\n  extra: t.jsonb().$type<UniDanExtra>(),\n});\nexport const danmakusInsertZod = createInsertSchema(danmakus);\nexport const danmakusSelectZod = createInsertSchema(danmakus);\nexport type DanmakusInsert = Simplify<\n  Omit<z.infer<typeof danmakusInsertZod>, \"extra\"> & { extra: UniDanExtra | null }\n>;\nexport type DanmakusSelect = Simplify<\n  Omit<z.infer<typeof danmakusSelectZod>, \"extra\"> & { extra: UniDanExtra | null }\n>;\n// export const pools = t.pgTable(\"pools\", {\n//   SOID: t.text().primaryKey(),\n// });\n\nexport const chunks = t.pgTable(\"chunks\", {\n  id: t.serial().primaryKey(),\n  // 信息区\n  fromConverted: t.boolean().notNull().default(false), // 是否包含有损转换过的弹幕，导入后不建议再次导出(应用层决定：导出按钮检测到后提示/禁用)\n  tmp: t.boolean().notNull().default(false), // 是否为临时chunk，临时chunk会在完成导出行为后自动删除/shrink时自动批量删除\n});\nexport const chunksZod = createInsertSchema(chunks);\n\n// many-to-many 映射表：chunk <-> danmakus\nexport const chunk2danmakus = t.pgTable(\"chunk_danmakus\", {\n  id: t.bigserial({ mode: \"bigint\" }).primaryKey(),\n  chunkID: t\n    .integer()\n    .references(() => chunks.id)\n    .notNull(),\n  DMID: t\n    .text()\n    .references(() => danmakus.DMID)\n    .notNull(),\n});\nexport const chunk2danmakusZod = createInsertSchema(chunk2danmakus);\n\nexport const baseRelations = defineRelations({ danmakus, chunks, chunk2danmakus });\nexport const relations = defineRelationsPart({ danmakus, chunks, chunk2danmakus }, (r) => ({\n  // pools: {\n  //   danmakus: r.many.danmakus({\n  //     from: r.pools.SOID,\n  //     to: r.danmakus.SOID,\n  //   }),\n  // },\n  chunks: {\n    danmakus: r.many.danmakus({\n      from: r.chunks.id.through(r.chunk2danmakus.chunkID),\n      to: r.danmakus.DMID.through(r.chunk2danmakus.DMID),\n    }),\n  },\n}));\n\nexport const onConflictDoUpdateSet = {\n  danmakus: Object.fromEntries(\n    Object.keys(danmakus).map((key) => [key, sql.raw(`excluded.\"${key}\"`)]),\n  ),\n} as const;\nexport const onConflictDoUpdate = {\n  danmakus: {\n    target: [danmakus.DMID],\n    set: onConflictDoUpdateSet.danmakus,\n  },\n};\n\nexport const ModeSchema = z.enum(danmakus.mode.enumValues);\nexport const PoolSchema = z.enum(danmakus.pool.enumValues);\nexport const DMAttrSchema = z.enum(danmakus.attr.enumValues);\n"],"mappings":"mMAOA,MAAa,EAAW,EAAE,OAAO,OAAQ,CAAC,SAAU,SAAU,MAAO,UAAW,MAAM,CAAC,CAC1E,EAAW,EAAE,OAAO,OAAQ,CAAC,MAAO,MAAO,MAAO,KAAK,CAAC,CACxD,EAAa,EAAE,OAAO,UAAW,CAC5C,UACA,WACA,WACA,aACA,WACA,YACA,WACA,OACD,CAAC,CAGW,EAAW,EAAE,QAAQ,WAAY,CAC5C,KAAM,EAAE,MAAM,CAAC,SAAS,CAExB,KAAM,EAAE,MAAM,CAAC,YAAY,CAC3B,SAAU,EAAE,SAAS,CAAC,SAAS,CAC/B,KAAM,GAAU,CAAC,SAAS,CAC1B,SAAU,EAAE,UAAU,CAAC,SAAS,CAChC,MAAO,EAAE,SAAS,CAAC,SAAS,CAC5B,SAAU,EAAE,MAAM,CAAC,SAAS,CAC5B,QAAS,EAAE,MAAM,CAAC,SAAS,CAC3B,MAAO,EAAE,WAAW,CAAC,SAAS,CAC9B,OAAQ,EAAE,UAAU,CAAC,SAAS,CAC9B,KAAM,GAAU,CAAC,SAAS,CAC1B,KAAM,GAAY,CAAC,OAAO,CAAC,SAAS,CACpC,SAAU,EAAE,MAAM,CAClB,MAAO,EAAE,OAAO,CAAC,OAAoB,CACtC,CAAC,CACW,EAAoB,EAAmB,EAAS,CAChD,EAAoB,EAAmB,EAAS,CAWhD,EAAS,EAAE,QAAQ,SAAU,CACxC,GAAI,EAAE,QAAQ,CAAC,YAAY,CAE3B,cAAe,EAAE,SAAS,CAAC,SAAS,CAAC,QAAQ,GAAM,CACnD,IAAK,EAAE,SAAS,CAAC,SAAS,CAAC,QAAQ,GAAM,CAC1C,CAAC,CACW,EAAY,EAAmB,EAAO,CAGtC,EAAiB,EAAE,QAAQ,iBAAkB,CACxD,GAAI,EAAE,UAAU,CAAE,KAAM,SAAU,CAAC,CAAC,YAAY,CAChD,QAAS,EACN,SAAS,CACT,eAAiB,EAAO,GAAG,CAC3B,SAAS,CACZ,KAAM,EACH,MAAM,CACN,eAAiB,EAAS,KAAK,CAC/B,SAAS,CACb,CAAC,CACW,EAAoB,EAAmB,EAAe,CAEtD,EAAgB,EAAgB,CAAE,WAAU,SAAQ,iBAAgB,CAAC,CACrE,EAAY,EAAoB,CAAE,WAAU,SAAQ,iBAAgB,CAAG,IAAO,CAOzF,OAAQ,CACN,SAAU,EAAE,KAAK,SAAS,CACxB,KAAM,EAAE,OAAO,GAAG,QAAQ,EAAE,eAAe,QAAQ,CACnD,GAAI,EAAE,SAAS,KAAK,QAAQ,EAAE,eAAe,KAAK,CACnD,CAAC,CACH,CACF,EAAE,CAEU,EAAwB,CACnC,SAAU,OAAO,YACf,OAAO,KAAK,EAAS,CAAC,IAAK,GAAQ,CAAC,EAAK,EAAI,IAAI,aAAa,EAAI,GAAG,CAAC,CAAC,CACxE,CACF,CACY,EAAqB,CAChC,SAAU,CACR,OAAQ,CAAC,EAAS,KAAK,CACvB,IAAK,EAAsB,SAC5B,CACF,CAEY,EAAa,EAAE,KAAK,EAAS,KAAK,WAAW,CAC7C,EAAa,EAAE,KAAK,EAAS,KAAK,WAAW,CAC7C,EAAe,EAAE,KAAK,EAAS,KAAK,WAAW"}