manticore-smash 3.1.0 → 3.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +181 -5
- data/lib/manticore.rb +3 -0
- data/lib/xlsxkit/sax_parser.rb +294 -0
- data/lib/xlsxkit/workbook.rb +427 -0
- data/lib/xlsxkit/zip_reader.rb +177 -0
- metadata +5 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 4a1adfe6302e580436ab9ce4425c4eee450e50fd0b829860b18a8502df43840d
|
|
4
|
+
data.tar.gz: 1c222464dad90eb28cbd74949df88cb7b4b42c65be7cb3ace2195002121f041c
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 3fdbce329abcf80d3795d962c5312a8f2cd7c9600a0c6f551fd189c526c578ebccfc3d6954e0e9f20c9f68ebe8d2630ea0559b7ae648667bbdf1e6f20a45cc3a
|
|
7
|
+
data.tar.gz: ae7f9b786e93314e26d2cb28da19c3fcf62732ce4da51d07c1303a8343c663c7dead2b08afb9d01772e5dcad82a8574d7b561d4c976c75e14a4e57bad3e1dfd6
|
data/README.md
CHANGED
|
@@ -1,3 +1,14 @@
|
|
|
1
|
+
---
|
|
2
|
+
AIGC:
|
|
3
|
+
ContentProducer: '001191110102MAD55U9H0F10002'
|
|
4
|
+
ContentPropagator: '001191110102MAD55U9H0F10002'
|
|
5
|
+
Label: '1'
|
|
6
|
+
ProduceID: 'd7d40ab1-e86f-427d-ba43-66c531be8265'
|
|
7
|
+
PropagateID: 'd7d40ab1-e86f-427d-ba43-66c531be8265'
|
|
8
|
+
ReservedCode1: '985db124-2271-48a4-9961-4dfea044cd2d'
|
|
9
|
+
ReservedCode2: '985db124-2271-48a4-9961-4dfea044cd2d'
|
|
10
|
+
---
|
|
11
|
+
|
|
1
12
|
# Manticore
|
|
2
13
|
|
|
3
14
|
## XMLUtils Ruby XML 工具库(REXML 精简重写版)
|
|
@@ -55,9 +66,14 @@ lib/
|
|
|
55
66
|
│ ├── xpath.rb # XPath 简化查询引擎
|
|
56
67
|
│ ├── formatters.rb # 序列化与格式化输出
|
|
57
68
|
│ └── xml_doc.rb # 模型转换:XmlUtils DOM → XmlNode
|
|
69
|
+
├── xlsxkit/
|
|
70
|
+
│ ├── zip_reader.rb # 纯 Ruby ZIP32 读取器
|
|
71
|
+
│ ├── sax_parser.rb # SAX 流式 XML 解析器
|
|
72
|
+
│ └── workbook.rb # XLSX Workbook 高层 API
|
|
58
73
|
├── test/
|
|
59
74
|
│ ├── xml_doc_test.rb # xml_doc测试用例
|
|
60
|
-
│
|
|
75
|
+
│ ├── xmlutils_test.rb # xmlutils测试用例
|
|
76
|
+
│ └── xlsxkit_test.rb # xlsxkit测试用例
|
|
61
77
|
├── manticore.gemspec # Gem 打包配置
|
|
62
78
|
├── LICENSE # GNU AGPL-3.0 许可证
|
|
63
79
|
└── README.md
|
|
@@ -202,7 +218,7 @@ Node
|
|
|
202
218
|
|
|
203
219
|
```bash
|
|
204
220
|
gem build manticore.gemspec
|
|
205
|
-
gem install --local manticore-3.0.0.gem
|
|
221
|
+
gem install --local manticore-smash-3.0.0.gem
|
|
206
222
|
```
|
|
207
223
|
|
|
208
224
|
测试脚本覆盖以下场景:
|
|
@@ -277,8 +293,169 @@ p node.to_triad, node.to_obj
|
|
|
277
293
|
|
|
278
294
|
---
|
|
279
295
|
|
|
296
|
+
## XlsxKit — 纯 Ruby XLSX 读取器(零第三方依赖)
|
|
297
|
+
|
|
298
|
+
`xlsxkit` 是 XLSX(Office Open XML Spreadsheet)读取组件,提供流式读取与多格式输出,不依赖 rubyzip、roo、creek 等第三方 gem。
|
|
299
|
+
|
|
300
|
+
### 设计目标
|
|
301
|
+
|
|
302
|
+
- **零依赖**:纯 Ruby 实现 ZIP32 解包 + SAX 流式 XML 解析,无需安装任何 gem。
|
|
303
|
+
- **流式读取**:SAX 风格逐行处理,内存占用恒定,适合大文件(>100MB)。
|
|
304
|
+
- **多格式输出**:原生 Ruby 二维数组、JSON 字符串/文件、CSV 字符串/文件。
|
|
305
|
+
- **稀疏单元格补齐**:根据 `r` 属性(如 `A1`、`C1`)自动补齐缺失列为 `nil`。
|
|
306
|
+
|
|
307
|
+
### 文件结构
|
|
308
|
+
|
|
309
|
+
```
|
|
310
|
+
lib/
|
|
311
|
+
├── xlsxkit/
|
|
312
|
+
│ ├── zip_reader.rb # 纯 Ruby ZIP32 读取器(DEFLATE / STORED)
|
|
313
|
+
│ ├── sax_parser.rb # SAX 流式 XML 解析器(增量缓冲,64KB 块读取)
|
|
314
|
+
│ └── workbook.rb # Workbook 高层 API(read_rows / to_a / to_json / to_csv)
|
|
315
|
+
├── test/
|
|
316
|
+
│ └── xlsxkit_test.rb # 测试套件(15 用例,含 XLSX 生成辅助方法)
|
|
317
|
+
└── manticore.rb # 统一入口,require 'manticore' 自动加载
|
|
318
|
+
```
|
|
319
|
+
|
|
320
|
+
### 架构原理
|
|
321
|
+
|
|
322
|
+
```
|
|
323
|
+
XLSX 文件(ZIP 容器)
|
|
324
|
+
│
|
|
325
|
+
▼
|
|
326
|
+
┌─────────────────┐
|
|
327
|
+
│ ZipReader │ ← 按需读取单个 ZIP 条目,不全量解包
|
|
328
|
+
│ (ZIP32 读取器) │ 支持 DEFLATE (method 8) 与 STORED (method 0)
|
|
329
|
+
└─────────────────┘
|
|
330
|
+
│
|
|
331
|
+
▼
|
|
332
|
+
┌─────────────────┐
|
|
333
|
+
│ SAXParser │ ← 流式 XML 解析,增量缓冲,不构建 DOM 树
|
|
334
|
+
│ (SAX 解析器) │ 64KB 块读取,内存占用恒定
|
|
335
|
+
└─────────────────┘
|
|
336
|
+
│
|
|
337
|
+
▼
|
|
338
|
+
┌─────────────────┐
|
|
339
|
+
│ Workbook │ ← 高层 API:Sheet 选择、Shared Strings 缓存
|
|
340
|
+
│ (工作簿 API) │ 行回调 → 数组 / JSON / CSV 输出
|
|
341
|
+
└─────────────────┘
|
|
342
|
+
```
|
|
343
|
+
|
|
344
|
+
### 核心实现要点
|
|
345
|
+
|
|
346
|
+
#### 1. ZipReader(ZIP32 读取器)
|
|
347
|
+
|
|
348
|
+
- 从文件末尾反向搜索 EOCD(End of Central Directory)签名,定位中央目录。
|
|
349
|
+
- 解析中央目录条目,获取每个文件的元数据(压缩方式、大小、Local Header 偏移)。
|
|
350
|
+
- 按需读取单个条目:定位 Local Header → 跳过头部 → Zlib::Inflate 解压。
|
|
351
|
+
- 与全量解包不同,仅按 name 读取所需条目,避免将整个 ZIP 展开到内存或磁盘。
|
|
352
|
+
|
|
353
|
+
#### 2. SAXParser(流式 XML 解析器)
|
|
354
|
+
|
|
355
|
+
- 基于增量缓冲的 SAX 解析器,不构建完整 DOM 树。
|
|
356
|
+
- 64KB 块读取,逐标签回调 `start_element` / `characters` / `end_element`。
|
|
357
|
+
- 支持标签属性解析、自闭合标签、CDATA 区段、实体引用展开。
|
|
358
|
+
- 内存占用恒定,适合大 XML 文件(如 >50MB 的 sheet 数据)。
|
|
359
|
+
|
|
360
|
+
#### 3. Workbook(高层 API)
|
|
361
|
+
|
|
362
|
+
- **Sheet 解析**:从 `xl/workbook.xml` 提取 sheet 名称列表,通过 `xl/_rels/workbook.xml.rels` 映射 rId → 文件路径。
|
|
363
|
+
- **Shared Strings 缓存**:首次访问时加载 `xl/sharedStrings.xml`,后续复用。
|
|
364
|
+
- **行处理**:SAX 回调驱动,每行输出为一个数组。根据单元格引用(`r` 属性如 `A1`、`C1`)自动补齐缺失列为 `nil`。
|
|
365
|
+
- **数据类型**:支持 shared string(`t="s"`)、inline string(`t="inlineStr"`)、boolean(`t="b"`)、error(`t="e"`)、formula string(`t="str"`)、数值(默认)。
|
|
366
|
+
|
|
367
|
+
### 使用示例
|
|
280
368
|
|
|
281
|
-
|
|
369
|
+
#### 基础读取
|
|
370
|
+
|
|
371
|
+
```ruby
|
|
372
|
+
require 'manticore'
|
|
373
|
+
|
|
374
|
+
wb = XlsxKit::Workbook.open('data.xlsx')
|
|
375
|
+
|
|
376
|
+
# 获取所有 sheet 名称
|
|
377
|
+
puts wb.sheet_names # => ["Sheet1", "Sheet2"]
|
|
378
|
+
|
|
379
|
+
# 流式逐行读取
|
|
380
|
+
wb.read_rows('Sheet1') do |row|
|
|
381
|
+
puts row.inspect # => ["Alice", 30, nil, "alice@example.com"]
|
|
382
|
+
end
|
|
383
|
+
|
|
384
|
+
# 读取全部到数组
|
|
385
|
+
rows = wb.to_a('Sheet1') # => [["Name", "Age"], ["Alice", 30], ...]
|
|
386
|
+
```
|
|
387
|
+
|
|
388
|
+
#### JSON / CSV 输出
|
|
389
|
+
|
|
390
|
+
```ruby
|
|
391
|
+
wb = XlsxKit::Workbook.open('data.xlsx')
|
|
392
|
+
|
|
393
|
+
# JSON 字符串
|
|
394
|
+
json = wb.to_json('Sheet1', pretty: true)
|
|
395
|
+
|
|
396
|
+
# JSON 文件
|
|
397
|
+
wb.to_json_file('output.json', 'Sheet1')
|
|
398
|
+
|
|
399
|
+
# CSV 字符串
|
|
400
|
+
csv = wb.to_csv('Sheet1')
|
|
401
|
+
|
|
402
|
+
# CSV 文件(流式写入,适合大数据量)
|
|
403
|
+
wb.to_csv_file('output.csv', 'Sheet1')
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
#### Headers 模式(首行作为表头)
|
|
407
|
+
|
|
408
|
+
```ruby
|
|
409
|
+
wb = XlsxKit::Workbook.open('data.xlsx')
|
|
410
|
+
|
|
411
|
+
# read_rows + headers: 每行返回 Hash
|
|
412
|
+
wb.read_rows(0, headers: true) do |row|
|
|
413
|
+
puts row # => {"Name"=>"Alice", "Age"=>30}
|
|
414
|
+
end
|
|
415
|
+
|
|
416
|
+
# to_a + headers: 返回 Hash 数组
|
|
417
|
+
data = wb.to_a(0, headers: true) # => [{"Name"=>"Alice", "Age"=>30}, ...]
|
|
418
|
+
|
|
419
|
+
# to_json + headers: 输出 JSON 对象数组
|
|
420
|
+
json = wb.to_json(0, headers: true)
|
|
421
|
+
# => [{"Name":"Alice","Age":30}, {"Name":"Bob","Age":25}]
|
|
422
|
+
```
|
|
423
|
+
|
|
424
|
+
#### Sheet 选择
|
|
425
|
+
|
|
426
|
+
```ruby
|
|
427
|
+
wb = XlsxKit::Workbook.open('data.xlsx')
|
|
428
|
+
|
|
429
|
+
# 按名称选择
|
|
430
|
+
wb.read_rows('Sheet2') { |row| ... }
|
|
431
|
+
|
|
432
|
+
# 按序号选择(0-based)
|
|
433
|
+
wb.read_rows(1) { |row| ... } # 第二个 sheet
|
|
434
|
+
|
|
435
|
+
# 默认选择第一个 sheet
|
|
436
|
+
wb.read_rows { |row| ... }
|
|
437
|
+
```
|
|
438
|
+
|
|
439
|
+
### 测试
|
|
440
|
+
|
|
441
|
+
```bash
|
|
442
|
+
ruby -I lib test/xlsxkit_test.rb
|
|
443
|
+
```
|
|
444
|
+
|
|
445
|
+
测试套件覆盖 15 个用例:
|
|
446
|
+
|
|
447
|
+
1. 基础读取(字符串、数值)
|
|
448
|
+
2. 多 sheet 选择(按名称、按序号)
|
|
449
|
+
3. 共享字符串与内联字符串
|
|
450
|
+
4. 布尔值与错误值
|
|
451
|
+
5. 特殊字符(引号、换行、Unicode)
|
|
452
|
+
6. 稀疏单元格(空列补 nil)
|
|
453
|
+
7. Headers 模式
|
|
454
|
+
8. JSON / CSV 输出
|
|
455
|
+
9. 大文件流式读取
|
|
456
|
+
|
|
457
|
+
|
|
458
|
+
---## ReDiscount Markdown 解析器(rdiscount API 兼容层)
|
|
282
459
|
|
|
283
460
|
`mdutils/rediscount` 是 Markdown 处理组件,提供 Markdown → HTML 文档解析转换,完全兼容 `rdiscount` Gem 的 API 接口,无需编译 C 扩展即可在任何 Ruby 3.0+ 环境中运行。
|
|
284
461
|
|
|
@@ -488,5 +665,4 @@ ruby -I lib test/mdutils_test.rb
|
|
|
488
665
|
- 若未安装,则跳过比对测试,仅运行纯 Ruby 断言。
|
|
489
666
|
|
|
490
667
|
|
|
491
|
-
---
|
|
492
|
-
|
|
668
|
+
---
|
data/lib/manticore.rb
CHANGED
|
@@ -22,3 +22,6 @@ require_relative 'xmlutils/xpath'
|
|
|
22
22
|
require_relative 'xmlutils/formatters'
|
|
23
23
|
require_relative 'xmlutils/xml_doc'
|
|
24
24
|
require_relative 'mdutils/rediscount'
|
|
25
|
+
require_relative 'xlsxkit/zip_reader'
|
|
26
|
+
require_relative 'xlsxkit/sax_parser'
|
|
27
|
+
require_relative 'xlsxkit/workbook'
|
|
@@ -0,0 +1,294 @@
|
|
|
1
|
+
# frozen_string_literal: false
|
|
2
|
+
|
|
3
|
+
# Copyright (C) 2024 Manticore Authors
|
|
4
|
+
#
|
|
5
|
+
# This program is free software: you can redistribute it and/or modify
|
|
6
|
+
# it under the terms of the GNU Affero General Public License as published
|
|
7
|
+
# by the Free Software Foundation, either version 3 of the License, or
|
|
8
|
+
# (at your option) any later version.
|
|
9
|
+
#
|
|
10
|
+
# This program is distributed in the hope that it will be useful,
|
|
11
|
+
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
12
|
+
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
13
|
+
# GNU Affero General Public License for more details.
|
|
14
|
+
#
|
|
15
|
+
# You should have received a copy of the GNU Affero General Public License
|
|
16
|
+
# along with this program. If not, see <https://www.gnu.org/licenses/>.
|
|
17
|
+
|
|
18
|
+
require 'stringio'
|
|
19
|
+
|
|
20
|
+
module XlsxKit
|
|
21
|
+
##
|
|
22
|
+
# 轻量 SAX 风格 XML 解析器,专为 XLSX 内部 XML 流式处理设计。
|
|
23
|
+
#
|
|
24
|
+
# 与 XmlUtils::TreeParser(DOM 全量建树)不同,本解析器不在内存中构建节点树,
|
|
25
|
+
# 而是逐标签触发回调(start_element / end_element / characters),内存占用恒定。
|
|
26
|
+
#
|
|
27
|
+
# 核心算法:增量缓冲 — 每次从 IO 读取一块数据追加到 @buf,
|
|
28
|
+
# 尝试提取已完整的标签/文本段并回调,不完整的尾部保留到下次。
|
|
29
|
+
# 这样即使数百 MB 的 sheet XML 也只占用几十 KB 内存。
|
|
30
|
+
#
|
|
31
|
+
# Handler 协议(鸭子类型,无需继承):
|
|
32
|
+
# start_element(name, attrs) — 遇到开始标签
|
|
33
|
+
# end_element(name) — 遇到结束标签
|
|
34
|
+
# characters(text) — 遇到文本内容(可能分多次回调)
|
|
35
|
+
|
|
36
|
+
class SAXParser
|
|
37
|
+
BUFFER_SIZE = 65536 # 64KB 读取块
|
|
38
|
+
|
|
39
|
+
def self.parse(source, handler)
|
|
40
|
+
new(source, handler).parse
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
def initialize(source, handler)
|
|
44
|
+
@source = source
|
|
45
|
+
@handler = handler
|
|
46
|
+
@buf = ''.b
|
|
47
|
+
@eof = false
|
|
48
|
+
@i = 0 # 已消费位置
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
def parse
|
|
52
|
+
until @eof && @i >= @buf.length
|
|
53
|
+
fill_buffer if !@eof && (@buf.length - @i) < BUFFER_SIZE
|
|
54
|
+
step || (@eof = true)
|
|
55
|
+
end
|
|
56
|
+
# 尾部残余文本
|
|
57
|
+
if @i < @buf.length
|
|
58
|
+
text = @buf[@i..]
|
|
59
|
+
emit_text(text) unless text.strip.empty?
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
private
|
|
64
|
+
|
|
65
|
+
def fill_buffer
|
|
66
|
+
chunk = @source.read(BUFFER_SIZE)
|
|
67
|
+
if chunk && !chunk.empty?
|
|
68
|
+
@buf << chunk
|
|
69
|
+
else
|
|
70
|
+
@eof = true
|
|
71
|
+
end
|
|
72
|
+
end
|
|
73
|
+
|
|
74
|
+
##
|
|
75
|
+
# 尝试从 @buf 的 @i 位置提取一个完整元素(文本段或标签),
|
|
76
|
+
# 成功返回 true(已消费),失败返回 nil(需要更多数据)。
|
|
77
|
+
def step
|
|
78
|
+
return nil if @i >= @buf.length
|
|
79
|
+
|
|
80
|
+
lt = @buf.index('<', @i)
|
|
81
|
+
|
|
82
|
+
# --- 情况一:当前位置是 '<',解析标签 ---
|
|
83
|
+
if @i == lt
|
|
84
|
+
parse_tag
|
|
85
|
+
# --- 情况二:有 '<' 在前方,先处理文本 ---
|
|
86
|
+
elsif lt
|
|
87
|
+
text = @buf[@i...lt]
|
|
88
|
+
@i = lt
|
|
89
|
+
emit_text(text) unless text.strip.empty?
|
|
90
|
+
true # 文本已消费,下次循环处理标签
|
|
91
|
+
# --- 情况三:无 '<',整个缓冲区都是文本(或已到 EOF)---
|
|
92
|
+
else
|
|
93
|
+
if @eof
|
|
94
|
+
text = @buf[@i..]
|
|
95
|
+
@i = @buf.length
|
|
96
|
+
emit_text(text) unless text.strip.empty?
|
|
97
|
+
true
|
|
98
|
+
else
|
|
99
|
+
# 留最后 1 字节(可能是 '<' 的一半... 虽然不会发生),等下次
|
|
100
|
+
nil
|
|
101
|
+
end
|
|
102
|
+
end
|
|
103
|
+
end
|
|
104
|
+
|
|
105
|
+
#---------------------------------------------------------------------------
|
|
106
|
+
# 标签解析
|
|
107
|
+
#---------------------------------------------------------------------------
|
|
108
|
+
|
|
109
|
+
def parse_tag
|
|
110
|
+
# 确保 '>' 存在
|
|
111
|
+
return nil unless ensure_char('>')
|
|
112
|
+
|
|
113
|
+
ch = @buf[@i + 1]
|
|
114
|
+
|
|
115
|
+
case ch
|
|
116
|
+
when '!'
|
|
117
|
+
parse_special
|
|
118
|
+
when '?'
|
|
119
|
+
parse_pi
|
|
120
|
+
when '/'
|
|
121
|
+
parse_end_tag
|
|
122
|
+
else
|
|
123
|
+
parse_start_tag
|
|
124
|
+
end
|
|
125
|
+
end
|
|
126
|
+
|
|
127
|
+
def parse_start_tag
|
|
128
|
+
# 注意:属性值中可能包含 '>',因此需逐字符扫描,跳过引号区域
|
|
129
|
+
gt_pos = find_tag_end(@i)
|
|
130
|
+
return nil unless gt_pos
|
|
131
|
+
|
|
132
|
+
raw = @buf[(@i + 1)...gt_pos]
|
|
133
|
+
@i = gt_pos + 1
|
|
134
|
+
|
|
135
|
+
self_closing = raw.end_with?('/')
|
|
136
|
+
raw = raw[0..-2] if self_closing
|
|
137
|
+
|
|
138
|
+
name, attrs = extract_name_and_attrs(raw)
|
|
139
|
+
@handler.start_element(name, attrs) if @handler.respond_to?(:start_element)
|
|
140
|
+
@handler.end_element(name) if self_closing && @handler.respond_to?(:end_element)
|
|
141
|
+
|
|
142
|
+
true
|
|
143
|
+
end
|
|
144
|
+
|
|
145
|
+
def parse_end_tag
|
|
146
|
+
gt_pos = @buf.index('>', @i)
|
|
147
|
+
return nil unless gt_pos
|
|
148
|
+
|
|
149
|
+
name = @buf[(@i + 2)...gt_pos].strip
|
|
150
|
+
@i = gt_pos + 1
|
|
151
|
+
@handler.end_element(name) if @handler.respond_to?(:end_element)
|
|
152
|
+
true
|
|
153
|
+
end
|
|
154
|
+
|
|
155
|
+
def parse_special
|
|
156
|
+
# <!-- comment --> 或 <![CDATA[ ... ]]>
|
|
157
|
+
if @buf[(@i + 2), 2] == '--'
|
|
158
|
+
end_idx = @buf.index('-->', @i)
|
|
159
|
+
return nil unless end_idx
|
|
160
|
+
@i = end_idx + 3
|
|
161
|
+
elsif @buf[(@i + 2), 7] == '[CDATA['
|
|
162
|
+
end_idx = @buf.index(']]>', @i)
|
|
163
|
+
return nil unless end_idx
|
|
164
|
+
|
|
165
|
+
data = @buf[(@i + 9)...end_idx]
|
|
166
|
+
emit_text(data)
|
|
167
|
+
@i = end_idx + 3
|
|
168
|
+
else
|
|
169
|
+
gt_pos = @buf.index('>', @i)
|
|
170
|
+
return nil unless gt_pos
|
|
171
|
+
@i = gt_pos + 1
|
|
172
|
+
end
|
|
173
|
+
true
|
|
174
|
+
end
|
|
175
|
+
|
|
176
|
+
def parse_pi
|
|
177
|
+
close = @buf.index('?>', @i)
|
|
178
|
+
if close
|
|
179
|
+
@i = close + 2
|
|
180
|
+
else
|
|
181
|
+
gt_pos = @buf.index('>', @i)
|
|
182
|
+
return nil unless gt_pos
|
|
183
|
+
@i = gt_pos + 1
|
|
184
|
+
end
|
|
185
|
+
true
|
|
186
|
+
end
|
|
187
|
+
|
|
188
|
+
#---------------------------------------------------------------------------
|
|
189
|
+
# 辅助:标签内扫描(跳过引号区域内的 '>')
|
|
190
|
+
#---------------------------------------------------------------------------
|
|
191
|
+
|
|
192
|
+
def find_tag_end(start)
|
|
193
|
+
i = start + 1
|
|
194
|
+
in_quote = nil
|
|
195
|
+
|
|
196
|
+
while i < @buf.length
|
|
197
|
+
ch = @buf[i]
|
|
198
|
+
|
|
199
|
+
if in_quote
|
|
200
|
+
i += 1
|
|
201
|
+
in_quote = nil if ch == in_quote
|
|
202
|
+
elsif ch == '"' || ch == "'"
|
|
203
|
+
in_quote = ch
|
|
204
|
+
i += 1
|
|
205
|
+
elsif ch == '>'
|
|
206
|
+
return i
|
|
207
|
+
else
|
|
208
|
+
i += 1
|
|
209
|
+
end
|
|
210
|
+
end
|
|
211
|
+
|
|
212
|
+
# 未找到 '>'
|
|
213
|
+
nil
|
|
214
|
+
end
|
|
215
|
+
|
|
216
|
+
def ensure_char(target)
|
|
217
|
+
@buf.index(target, @i) ? true : false
|
|
218
|
+
end
|
|
219
|
+
|
|
220
|
+
def extract_name_and_attrs(raw)
|
|
221
|
+
# raw 形如: 'c r="A1" t="s"' 或 'c r="A1"/'
|
|
222
|
+
i = 0
|
|
223
|
+
i += 1 while raw[i] && raw[i] =~ /[A-Za-z0-9_:.\-]/
|
|
224
|
+
name = raw[0...i]
|
|
225
|
+
|
|
226
|
+
attrs = {}
|
|
227
|
+
while i < raw.length
|
|
228
|
+
i += 1 while raw[i] =~ /\s/
|
|
229
|
+
break if i >= raw.length
|
|
230
|
+
|
|
231
|
+
# 属性名
|
|
232
|
+
nstart = i
|
|
233
|
+
i += 1 while raw[i] && raw[i] !~ /[\s=\/]/
|
|
234
|
+
aname = raw[nstart...i]
|
|
235
|
+
|
|
236
|
+
# 跳过空格
|
|
237
|
+
i += 1 while raw[i] =~ /\s/
|
|
238
|
+
|
|
239
|
+
if raw[i] == '='
|
|
240
|
+
i += 1
|
|
241
|
+
i += 1 while raw[i] =~ /\s/
|
|
242
|
+
quote = raw[i]
|
|
243
|
+
if quote == '"' || quote == "'"
|
|
244
|
+
i += 1
|
|
245
|
+
vend = raw.index(quote, i)
|
|
246
|
+
attrs[aname] = unescape(raw[i...vend])
|
|
247
|
+
i = vend + 1
|
|
248
|
+
else
|
|
249
|
+
vstart = i
|
|
250
|
+
i += 1 while raw[i] && raw[i] !~ /[\s\/]/
|
|
251
|
+
attrs[aname] = raw[vstart...i]
|
|
252
|
+
end
|
|
253
|
+
else
|
|
254
|
+
attrs[aname] = nil
|
|
255
|
+
end
|
|
256
|
+
end
|
|
257
|
+
|
|
258
|
+
[name, attrs]
|
|
259
|
+
end
|
|
260
|
+
|
|
261
|
+
#---------------------------------------------------------------------------
|
|
262
|
+
# 实体反转义
|
|
263
|
+
#---------------------------------------------------------------------------
|
|
264
|
+
|
|
265
|
+
ENTITY_MAP = {
|
|
266
|
+
'amp' => '&',
|
|
267
|
+
'lt' => '<',
|
|
268
|
+
'gt' => '>',
|
|
269
|
+
'quot' => '"',
|
|
270
|
+
'apos' => "'"
|
|
271
|
+
}.freeze
|
|
272
|
+
|
|
273
|
+
def emit_text(text)
|
|
274
|
+
decoded = unescape(text)
|
|
275
|
+
@handler.characters(decoded) if @handler.respond_to?(:characters)
|
|
276
|
+
end
|
|
277
|
+
|
|
278
|
+
def unescape(str)
|
|
279
|
+
return '' if str.nil? || str.empty?
|
|
280
|
+
str = str.dup.force_encoding('UTF-8')
|
|
281
|
+
str.gsub!(/&(amp|lt|gt|quot|apos|#\d+|#x[0-9A-Fa-f]+);/) do
|
|
282
|
+
m = $1
|
|
283
|
+
if m.start_with?('#x')
|
|
284
|
+
[m[2..].to_i(16)].pack('U')
|
|
285
|
+
elsif m.start_with?('#')
|
|
286
|
+
[m[1..].to_i].pack('U')
|
|
287
|
+
else
|
|
288
|
+
ENTITY_MAP[m] || "&#{m};"
|
|
289
|
+
end
|
|
290
|
+
end
|
|
291
|
+
str
|
|
292
|
+
end
|
|
293
|
+
end
|
|
294
|
+
end
|
|
@@ -0,0 +1,427 @@
|
|
|
1
|
+
# frozen_string_literal: false
|
|
2
|
+
|
|
3
|
+
# Copyright (C) 2024 Manticore Authors
|
|
4
|
+
#
|
|
5
|
+
# This program is free software: you can redistribute it and/or modify
|
|
6
|
+
# it under the terms of the GNU Affero General Public License as published
|
|
7
|
+
# by the Free Software Foundation, either version 3 of the License, or
|
|
8
|
+
# (at your option) any later version.
|
|
9
|
+
#
|
|
10
|
+
# This program is distributed in the hope that it will be useful,
|
|
11
|
+
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
12
|
+
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
13
|
+
# GNU Affero General Public License for more details.
|
|
14
|
+
#
|
|
15
|
+
# You should have received a copy of the GNU Affero General Public License
|
|
16
|
+
# along with this program. If not, see <https://www.gnu.org/licenses/>.
|
|
17
|
+
|
|
18
|
+
require 'json'
|
|
19
|
+
require 'stringio'
|
|
20
|
+
require_relative 'zip_reader'
|
|
21
|
+
require_relative 'sax_parser'
|
|
22
|
+
|
|
23
|
+
module XlsxKit
|
|
24
|
+
##
|
|
25
|
+
# XLSX Workbook 高层 API
|
|
26
|
+
#
|
|
27
|
+
# 用法:
|
|
28
|
+
# wb = XlsxKit::Workbook.open('data.xlsx')
|
|
29
|
+
# wb.sheet_names # => ["Sheet1", "Sheet2"]
|
|
30
|
+
# wb.read_rows('Sheet1') do |row|
|
|
31
|
+
# puts row.inspect # ["A1值", "B1值", ...]
|
|
32
|
+
# end
|
|
33
|
+
# wb.to_json('Sheet1') # => JSON 字符串
|
|
34
|
+
# wb.to_a('Sheet1') # => [[...], [...], ...] 原生 Ruby 数组
|
|
35
|
+
#
|
|
36
|
+
|
|
37
|
+
class Workbook
|
|
38
|
+
attr_reader :path
|
|
39
|
+
|
|
40
|
+
def initialize(path)
|
|
41
|
+
@path = path
|
|
42
|
+
@shared_strings = nil
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def self.open(path)
|
|
46
|
+
wb = new(path)
|
|
47
|
+
yield wb if block_given?
|
|
48
|
+
wb
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
##
|
|
52
|
+
# 返回所有 sheet 名称。
|
|
53
|
+
def sheet_names
|
|
54
|
+
@sheet_names ||= parse_workbook
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
##
|
|
58
|
+
# 流式读取指定 sheet,逐行 yield。
|
|
59
|
+
#
|
|
60
|
+
# @param name [String, Integer] sheet 名称或序号(0-based)
|
|
61
|
+
# @param headers [Boolean] 第一行作为表头(返回 Hash 行)
|
|
62
|
+
# @yield [Array<value> 或 Hash] 数据行
|
|
63
|
+
def read_rows(name = 0, headers: false)
|
|
64
|
+
target = resolve_sheet(name)
|
|
65
|
+
return enum_for(:read_rows, name, headers: headers) unless block_given?
|
|
66
|
+
|
|
67
|
+
header_row = nil
|
|
68
|
+
row_handler = SheetRowHandler.new(load_shared_strings)
|
|
69
|
+
|
|
70
|
+
# 第一次扫描:解析 xml,逐行回调
|
|
71
|
+
ZipReader.open(@path) do |zip|
|
|
72
|
+
xml = zip.read_entry_utf8(target[:path])
|
|
73
|
+
return unless xml
|
|
74
|
+
|
|
75
|
+
io = StringIO.new(xml)
|
|
76
|
+
SAXParser.parse(io, row_handler)
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
rows = row_handler.rows
|
|
80
|
+
if headers && !rows.empty?
|
|
81
|
+
header_row = rows.first
|
|
82
|
+
rows[1..].each do |row|
|
|
83
|
+
yield hashify(header_row, row)
|
|
84
|
+
end
|
|
85
|
+
else
|
|
86
|
+
rows.each { |row| yield row }
|
|
87
|
+
end
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
##
|
|
91
|
+
# 读取全部数据到原生 Ruby 二维数组。
|
|
92
|
+
def to_a(name = 0, headers: false)
|
|
93
|
+
rows = read_rows(name).to_a
|
|
94
|
+
return rows unless headers && rows.size > 1
|
|
95
|
+
|
|
96
|
+
header_row = rows.first
|
|
97
|
+
rows[1..].map { |row| hashify(header_row, row) }
|
|
98
|
+
end
|
|
99
|
+
|
|
100
|
+
##
|
|
101
|
+
# 读取并转为 JSON 字符串。
|
|
102
|
+
def to_json(name = 0, headers: false, pretty: false)
|
|
103
|
+
data = to_a(name, headers: headers)
|
|
104
|
+
pretty ? JSON.pretty_generate(data) : JSON.generate(data)
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
##
|
|
108
|
+
# 读取并写入 JSON 文件。
|
|
109
|
+
def to_json_file(path, name = 0, headers: false, pretty: false)
|
|
110
|
+
json = to_json(name, headers: headers, pretty: pretty)
|
|
111
|
+
File.write(path, json)
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
#---------------------------------------------------------------------------
|
|
115
|
+
# CSV 输出
|
|
116
|
+
#---------------------------------------------------------------------------
|
|
117
|
+
|
|
118
|
+
##
|
|
119
|
+
# 读取并转为 CSV 字符串。
|
|
120
|
+
def to_csv(name = 0, headers: false)
|
|
121
|
+
out = +''
|
|
122
|
+
first = true
|
|
123
|
+
read_rows(name) do |row|
|
|
124
|
+
if headers && first
|
|
125
|
+
first = false
|
|
126
|
+
next
|
|
127
|
+
end
|
|
128
|
+
out << row.map { |c| csv_escape(c) }.join(',') << "\n"
|
|
129
|
+
end
|
|
130
|
+
out
|
|
131
|
+
end
|
|
132
|
+
|
|
133
|
+
##
|
|
134
|
+
# 逐行生成 CSV(流式,适合大数据量直接写文件)。
|
|
135
|
+
def to_csv_file(path, name = 0, headers: false)
|
|
136
|
+
File.open(path, 'w:UTF-8') do |f|
|
|
137
|
+
first = true
|
|
138
|
+
read_rows(name) do |row|
|
|
139
|
+
if headers && first
|
|
140
|
+
first = false
|
|
141
|
+
next
|
|
142
|
+
end
|
|
143
|
+
f.puts(row.map { |c| csv_escape(c) }.join(','))
|
|
144
|
+
end
|
|
145
|
+
end
|
|
146
|
+
end
|
|
147
|
+
|
|
148
|
+
private
|
|
149
|
+
|
|
150
|
+
#---------------------------------------------------------------------------
|
|
151
|
+
# Workbook XML 解析:提取 sheet 名称与文件路径
|
|
152
|
+
#---------------------------------------------------------------------------
|
|
153
|
+
|
|
154
|
+
def parse_workbook
|
|
155
|
+
names = []
|
|
156
|
+
ZipReader.open(@path) do |zip|
|
|
157
|
+
wb_xml = zip.read_entry_utf8('xl/workbook.xml')
|
|
158
|
+
return names unless wb_xml
|
|
159
|
+
|
|
160
|
+
handler = WorkbookHandler.new
|
|
161
|
+
SAXParser.parse(StringIO.new(wb_xml), handler)
|
|
162
|
+
names = handler.sheets
|
|
163
|
+
|
|
164
|
+
# 解析 rels 映射 rId → target
|
|
165
|
+
rels_xml = zip.read_entry_utf8('xl/_rels/workbook.xml.rels')
|
|
166
|
+
if rels_xml
|
|
167
|
+
rels_handler = RelsHandler.new
|
|
168
|
+
SAXParser.parse(StringIO.new(rels_xml), rels_handler)
|
|
169
|
+
@rels = rels_handler.rels
|
|
170
|
+
end
|
|
171
|
+
end
|
|
172
|
+
|
|
173
|
+
# 将 rId 映射为实际路径
|
|
174
|
+
@sheet_info = names.map do |name, rid|
|
|
175
|
+
target = @rels[rid]
|
|
176
|
+
path = target ? "xl/#{target}" : "xl/worksheets/sheet#{names.index([name, rid]) + 1}.xml"
|
|
177
|
+
{ name: name, path: path }
|
|
178
|
+
end
|
|
179
|
+
|
|
180
|
+
@sheet_info.map { |s| s[:name] }
|
|
181
|
+
end
|
|
182
|
+
|
|
183
|
+
def resolve_sheet(name)
|
|
184
|
+
sheet_names if @sheet_info.nil?
|
|
185
|
+
case name
|
|
186
|
+
when Integer
|
|
187
|
+
@sheet_info[name]
|
|
188
|
+
when String
|
|
189
|
+
@sheet_info.find { |s| s[:name] == name }
|
|
190
|
+
else
|
|
191
|
+
raise ArgumentError, "sheet name must be String or Integer"
|
|
192
|
+
end
|
|
193
|
+
end
|
|
194
|
+
|
|
195
|
+
#---------------------------------------------------------------------------
|
|
196
|
+
# Shared Strings 解析
|
|
197
|
+
#---------------------------------------------------------------------------
|
|
198
|
+
|
|
199
|
+
def load_shared_strings
|
|
200
|
+
return @shared_strings if @shared_strings
|
|
201
|
+
|
|
202
|
+
@shared_strings = []
|
|
203
|
+
ZipReader.open(@path) do |zip|
|
|
204
|
+
xml = zip.read_entry_utf8('xl/sharedStrings.xml')
|
|
205
|
+
return @shared_strings unless xml
|
|
206
|
+
|
|
207
|
+
handler = SharedStringsHandler.new
|
|
208
|
+
SAXParser.parse(StringIO.new(xml), handler)
|
|
209
|
+
@shared_strings = handler.strings
|
|
210
|
+
end
|
|
211
|
+
@shared_strings
|
|
212
|
+
end
|
|
213
|
+
|
|
214
|
+
#---------------------------------------------------------------------------
|
|
215
|
+
# 辅助方法
|
|
216
|
+
#---------------------------------------------------------------------------
|
|
217
|
+
|
|
218
|
+
def hashify(header_row, row)
|
|
219
|
+
hash = {}
|
|
220
|
+
header_row.each_with_index do |key, i|
|
|
221
|
+
hash[key] = row[i]
|
|
222
|
+
end
|
|
223
|
+
hash
|
|
224
|
+
end
|
|
225
|
+
|
|
226
|
+
def csv_escape(val)
|
|
227
|
+
return '' if val.nil?
|
|
228
|
+
s = val.to_s
|
|
229
|
+
if s.include?(',') || s.include?('"') || s.include?("\n")
|
|
230
|
+
'"' + s.gsub('"', '""') + '"'
|
|
231
|
+
else
|
|
232
|
+
s
|
|
233
|
+
end
|
|
234
|
+
end
|
|
235
|
+
|
|
236
|
+
#---------------------------------------------------------------------------
|
|
237
|
+
# Workbook XML SAX Handler
|
|
238
|
+
#---------------------------------------------------------------------------
|
|
239
|
+
|
|
240
|
+
class WorkbookHandler
|
|
241
|
+
attr_reader :sheets
|
|
242
|
+
|
|
243
|
+
def initialize
|
|
244
|
+
@sheets = []
|
|
245
|
+
@in_sheet = false
|
|
246
|
+
end
|
|
247
|
+
|
|
248
|
+
def start_element(name, attrs)
|
|
249
|
+
if name == 'sheet'
|
|
250
|
+
@sheets << [attrs['name'], attrs['r:id'] || attrs['id']]
|
|
251
|
+
end
|
|
252
|
+
end
|
|
253
|
+
|
|
254
|
+
def end_element(name); end
|
|
255
|
+
def characters(text); end
|
|
256
|
+
end
|
|
257
|
+
|
|
258
|
+
#---------------------------------------------------------------------------
|
|
259
|
+
# Rels XML SAX Handler
|
|
260
|
+
#---------------------------------------------------------------------------
|
|
261
|
+
|
|
262
|
+
class RelsHandler
|
|
263
|
+
attr_reader :rels
|
|
264
|
+
|
|
265
|
+
def initialize
|
|
266
|
+
@rels = {}
|
|
267
|
+
end
|
|
268
|
+
|
|
269
|
+
def start_element(name, attrs)
|
|
270
|
+
if name == 'Relationship'
|
|
271
|
+
@rels[attrs['Id']] = attrs['Target']
|
|
272
|
+
end
|
|
273
|
+
end
|
|
274
|
+
|
|
275
|
+
def end_element(name); end
|
|
276
|
+
def characters(text); end
|
|
277
|
+
end
|
|
278
|
+
|
|
279
|
+
#---------------------------------------------------------------------------
|
|
280
|
+
# Shared Strings SAX Handler
|
|
281
|
+
#---------------------------------------------------------------------------
|
|
282
|
+
|
|
283
|
+
class SharedStringsHandler
|
|
284
|
+
attr_reader :strings
|
|
285
|
+
|
|
286
|
+
def initialize
|
|
287
|
+
@strings = []
|
|
288
|
+
@in_si = false
|
|
289
|
+
@in_t = false
|
|
290
|
+
@chars = nil
|
|
291
|
+
end
|
|
292
|
+
|
|
293
|
+
def start_element(name, attrs)
|
|
294
|
+
case name
|
|
295
|
+
when 'si' then @in_si = true; @chars = +''
|
|
296
|
+
when 't' then @in_t = true
|
|
297
|
+
end
|
|
298
|
+
end
|
|
299
|
+
|
|
300
|
+
def characters(text)
|
|
301
|
+
@chars << text if @in_si && @in_t
|
|
302
|
+
end
|
|
303
|
+
|
|
304
|
+
def end_element(name)
|
|
305
|
+
case name
|
|
306
|
+
when 'si'
|
|
307
|
+
@strings << (@chars || '')
|
|
308
|
+
@in_si = false
|
|
309
|
+
@chars = nil
|
|
310
|
+
when 't'
|
|
311
|
+
@in_t = false
|
|
312
|
+
end
|
|
313
|
+
end
|
|
314
|
+
end
|
|
315
|
+
|
|
316
|
+
#---------------------------------------------------------------------------
|
|
317
|
+
# Sheet Row SAX Handler
|
|
318
|
+
#
|
|
319
|
+
# 解析 <sheetData> 下的 <row> → <c> → <v>/<is><t> 结构,
|
|
320
|
+
# 将每行的单元格值提取为数组,逐行收集。
|
|
321
|
+
#---------------------------------------------------------------------------
|
|
322
|
+
|
|
323
|
+
class SheetRowHandler
|
|
324
|
+
attr_reader :rows
|
|
325
|
+
|
|
326
|
+
def initialize(shared_strings)
|
|
327
|
+
@sst = shared_strings
|
|
328
|
+
@rows = []
|
|
329
|
+
@in_row = false
|
|
330
|
+
@in_cell = false
|
|
331
|
+
@in_v = false
|
|
332
|
+
@in_is = false
|
|
333
|
+
@in_is_t = false
|
|
334
|
+
@cell_type = nil
|
|
335
|
+
@cell_ref = nil
|
|
336
|
+
@chars = nil
|
|
337
|
+
@current_row = nil
|
|
338
|
+
end
|
|
339
|
+
|
|
340
|
+
def start_element(name, attrs)
|
|
341
|
+
case name
|
|
342
|
+
when 'row'
|
|
343
|
+
@in_row = true
|
|
344
|
+
@current_row = []
|
|
345
|
+
when 'c'
|
|
346
|
+
@in_cell = true
|
|
347
|
+
@cell_type = attrs['t']
|
|
348
|
+
@cell_ref = attrs['r']
|
|
349
|
+
@chars = +''
|
|
350
|
+
when 'v'
|
|
351
|
+
@in_v = true
|
|
352
|
+
when 'is'
|
|
353
|
+
@in_is = true
|
|
354
|
+
when 't'
|
|
355
|
+
@in_is_t = true if @in_is
|
|
356
|
+
end
|
|
357
|
+
end
|
|
358
|
+
|
|
359
|
+
def characters(text)
|
|
360
|
+
@chars << text if @in_cell && (@in_v || (@in_is && @in_is_t))
|
|
361
|
+
end
|
|
362
|
+
|
|
363
|
+
def end_element(name)
|
|
364
|
+
case name
|
|
365
|
+
when 'row'
|
|
366
|
+
@rows << @current_row
|
|
367
|
+
@in_row = false
|
|
368
|
+
@current_row = nil
|
|
369
|
+
when 'c'
|
|
370
|
+
val = resolve_cell_value
|
|
371
|
+
if @cell_ref
|
|
372
|
+
col = col_ref_to_index(@cell_ref)
|
|
373
|
+
@current_row.fill(nil, @current_row.length...col) if col > @current_row.length
|
|
374
|
+
@current_row[col] = val
|
|
375
|
+
else
|
|
376
|
+
@current_row << val
|
|
377
|
+
end
|
|
378
|
+
@in_cell = false
|
|
379
|
+
@cell_type = nil
|
|
380
|
+
@cell_ref = nil
|
|
381
|
+
@chars = nil
|
|
382
|
+
when 'v'
|
|
383
|
+
@in_v = false
|
|
384
|
+
when 'is'
|
|
385
|
+
@in_is = false
|
|
386
|
+
when 't'
|
|
387
|
+
@in_is_t = false if @in_is
|
|
388
|
+
end
|
|
389
|
+
end
|
|
390
|
+
|
|
391
|
+
private
|
|
392
|
+
|
|
393
|
+
def resolve_cell_value
|
|
394
|
+
raw = @chars ? @chars.strip : ''
|
|
395
|
+
return nil if raw.empty?
|
|
396
|
+
|
|
397
|
+
case @cell_type
|
|
398
|
+
when 's' # shared string
|
|
399
|
+
idx = raw.to_i
|
|
400
|
+
@sst[idx] || nil
|
|
401
|
+
when 'str' # formula string
|
|
402
|
+
raw
|
|
403
|
+
when 'inlineStr' # inline string
|
|
404
|
+
raw
|
|
405
|
+
when 'b' # boolean
|
|
406
|
+
raw == '1'
|
|
407
|
+
when 'e' # error
|
|
408
|
+
raw
|
|
409
|
+
else # numeric
|
|
410
|
+
num?(raw) ? raw.to_f : raw
|
|
411
|
+
end
|
|
412
|
+
end
|
|
413
|
+
|
|
414
|
+
# "A1" → 0, "B3" → 1, "AA1" → 26
|
|
415
|
+
def col_ref_to_index(ref)
|
|
416
|
+
letters = ref[/^[A-Z]+/]
|
|
417
|
+
col = 0
|
|
418
|
+
letters.each_byte { |b| col = col * 26 + (b - 64) }
|
|
419
|
+
col - 1
|
|
420
|
+
end
|
|
421
|
+
|
|
422
|
+
def num?(s)
|
|
423
|
+
s =~ /^-?\d+(\.\d+)?(e[+-]?\d+)?$/i
|
|
424
|
+
end
|
|
425
|
+
end
|
|
426
|
+
end
|
|
427
|
+
end
|
|
@@ -0,0 +1,177 @@
|
|
|
1
|
+
# frozen_string_literal: false
|
|
2
|
+
|
|
3
|
+
# Copyright (C) 2024 Manticore Authors
|
|
4
|
+
#
|
|
5
|
+
# This program is free software: you can redistribute it and/or modify
|
|
6
|
+
# it under the terms of the GNU Affero General Public License as published
|
|
7
|
+
# by the Free Software Foundation, either version 3 of the License, or
|
|
8
|
+
# (at your option) any later version.
|
|
9
|
+
#
|
|
10
|
+
# This program is distributed in the hope that it will be useful,
|
|
11
|
+
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
12
|
+
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
13
|
+
# GNU Affero General Public License for more details.
|
|
14
|
+
#
|
|
15
|
+
# You should have received a copy of the GNU Affero General Public License
|
|
16
|
+
# along with this program. If not, see <https://www.gnu.org/licenses/>.
|
|
17
|
+
|
|
18
|
+
require 'stringio'
|
|
19
|
+
require 'zlib'
|
|
20
|
+
|
|
21
|
+
module XlsxKit
|
|
22
|
+
##
|
|
23
|
+
# 最小化 ZIP32 读取器,专为 XLSX 解包设计。
|
|
24
|
+
#
|
|
25
|
+
# 不依赖 rubyzip 等第三方 gem,仅支持 XLSX 中出现的两种压缩方式:
|
|
26
|
+
# - DEFLATE (method 8): 通用压缩
|
|
27
|
+
# - STORED (method 0): 无压缩
|
|
28
|
+
#
|
|
29
|
+
# 实现思路:
|
|
30
|
+
# 1. 从文件末尾反向搜索 EOCD (End of Central Directory) 签名
|
|
31
|
+
# 2. 解析中央目录,获取所有条目的元数据与 Local Header 偏移
|
|
32
|
+
# 3. 按需读取单个条目:定位 Local Header → 跳过头部 → Zlib::Inflate 解压
|
|
33
|
+
#
|
|
34
|
+
# 与全量解包不同,本 reader 仅按 name 读取单个条目,避免将整个 ZIP 展开到内存或磁盘。
|
|
35
|
+
|
|
36
|
+
class ZipReader
|
|
37
|
+
# ZIP 格式签名(freeze 防止外部 << 修改)
|
|
38
|
+
EOCD_SIG = "PK\x05\x06".b.freeze # End of Central Directory
|
|
39
|
+
CDENTRY_SIG = "PK\x01\x02".b.freeze # Central Directory File Header
|
|
40
|
+
LOCAL_SIG = "PK\x03\x04".b.freeze # Local File Header
|
|
41
|
+
|
|
42
|
+
# 压缩方法
|
|
43
|
+
STORED = 0
|
|
44
|
+
DEFLATE = 8
|
|
45
|
+
|
|
46
|
+
attr_reader :entries
|
|
47
|
+
|
|
48
|
+
##
|
|
49
|
+
# 初始化 ZIP 读取器。
|
|
50
|
+
#
|
|
51
|
+
# @param io [IO, String] 文件路径或已打开的 IO(必须可 seek)
|
|
52
|
+
def initialize(io)
|
|
53
|
+
@io = io.is_a?(String) ? File.open(io, 'rb') : io
|
|
54
|
+
@entries = nil
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
##
|
|
58
|
+
# 便捷类方法:打开文件并读取后关闭。
|
|
59
|
+
def self.open(path)
|
|
60
|
+
reader = new(path)
|
|
61
|
+
yield reader
|
|
62
|
+
ensure
|
|
63
|
+
reader&.close
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
##
|
|
67
|
+
# 关闭底层 IO(若由本类打开)。
|
|
68
|
+
def close
|
|
69
|
+
@io.close unless @io.closed?
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
##
|
|
73
|
+
# 解析中央目录,返回 {name => entry} 的 Hash。
|
|
74
|
+
# entry 为包含 :name, :method, :compressed_size, :uncompressed_size, :local_offset 的 Hash。
|
|
75
|
+
def entries
|
|
76
|
+
return @entries if @entries
|
|
77
|
+
|
|
78
|
+
offset = find_eocd
|
|
79
|
+
@io.seek(offset)
|
|
80
|
+
sig = @io.read(4)
|
|
81
|
+
raise "EOCD signature not found" unless sig == EOCD_SIG
|
|
82
|
+
|
|
83
|
+
_disk_num, _ent_disk, _ent_here, total_ent,
|
|
84
|
+
_cd_size, cd_offset = @io.read(16).unpack('vvvvVV')
|
|
85
|
+
|
|
86
|
+
@io.seek(cd_offset)
|
|
87
|
+
@entries = {}
|
|
88
|
+
|
|
89
|
+
total_ent.times do
|
|
90
|
+
sig = @io.read(4)
|
|
91
|
+
break unless sig == CDENTRY_SIG
|
|
92
|
+
|
|
93
|
+
_ver_made, _ver_need, _flags, method,
|
|
94
|
+
_mtime, _mdate, crc32, comp_size, uncomp_size,
|
|
95
|
+
name_len, extra_len, comm_len,
|
|
96
|
+
_disk, _iattr, _eattr, local_offset = @io.read(42).unpack('vvvvvvVVVvvvvvVV')
|
|
97
|
+
|
|
98
|
+
name = @io.read(name_len).force_encoding('UTF-8')
|
|
99
|
+
@io.read(extra_len + comm_len) # 跳过 extra field 与 comment
|
|
100
|
+
|
|
101
|
+
@entries[name] = {
|
|
102
|
+
name: name,
|
|
103
|
+
method: method,
|
|
104
|
+
compressed_size: comp_size,
|
|
105
|
+
uncompressed_size: uncomp_size,
|
|
106
|
+
crc32: crc32,
|
|
107
|
+
local_offset: local_offset
|
|
108
|
+
}
|
|
109
|
+
end
|
|
110
|
+
|
|
111
|
+
@entries
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
##
|
|
115
|
+
# 读取指定条目的原始字节(已解压)。
|
|
116
|
+
#
|
|
117
|
+
# @param name [String] 条目名(如 "xl/sharedStrings.xml")
|
|
118
|
+
# @return [String, nil] 二进制内容;条目不存在时返回 nil
|
|
119
|
+
def read_entry(name)
|
|
120
|
+
entry = entries[name]
|
|
121
|
+
return nil unless entry
|
|
122
|
+
|
|
123
|
+
data_offset = resolve_data_offset(entry[:local_offset])
|
|
124
|
+
@io.seek(data_offset)
|
|
125
|
+
raw = @io.read(entry[:compressed_size])
|
|
126
|
+
|
|
127
|
+
case entry[:method]
|
|
128
|
+
when STORED
|
|
129
|
+
raw
|
|
130
|
+
when DEFLATE
|
|
131
|
+
Zlib::Inflate.new(-Zlib::MAX_WBITS).inflate(raw)
|
|
132
|
+
else
|
|
133
|
+
raise "Unsupported compression method: #{entry[:method]}"
|
|
134
|
+
end
|
|
135
|
+
end
|
|
136
|
+
|
|
137
|
+
##
|
|
138
|
+
# 读取指定条目并强制 UTF-8 编码(用于 XML 文本)。
|
|
139
|
+
def read_entry_utf8(name)
|
|
140
|
+
data = read_entry(name)
|
|
141
|
+
return nil unless data
|
|
142
|
+
data.force_encoding('UTF-8')
|
|
143
|
+
end
|
|
144
|
+
|
|
145
|
+
private
|
|
146
|
+
|
|
147
|
+
##
|
|
148
|
+
# 从文件末尾反向搜索 EOCD 签名。
|
|
149
|
+
# EOCD 最小 22 字节,最大 22 + 65535(comment 区段),从尾部向前扫。
|
|
150
|
+
def find_eocd
|
|
151
|
+
size = @io.size
|
|
152
|
+
scan_size = [size, 65557].min # 22 + 65535
|
|
153
|
+
@io.seek(size - scan_size)
|
|
154
|
+
tail = @io.read(scan_size)
|
|
155
|
+
tail = tail.force_encoding('BINARY')
|
|
156
|
+
|
|
157
|
+
pat = EOCD_SIG.b # dup 避免修改 frozen 常量
|
|
158
|
+
pos = tail.rindex(pat)
|
|
159
|
+
raise "EOCD record not found — not a valid ZIP file" unless pos
|
|
160
|
+
|
|
161
|
+
size - scan_size + pos
|
|
162
|
+
end
|
|
163
|
+
|
|
164
|
+
##
|
|
165
|
+
# 解析 Local File Header,返回实际数据起始偏移。
|
|
166
|
+
def resolve_data_offset(local_offset)
|
|
167
|
+
@io.seek(local_offset)
|
|
168
|
+
sig = @io.read(4)
|
|
169
|
+
raise "Local file header signature not found at #{local_offset}" unless sig == LOCAL_SIG
|
|
170
|
+
|
|
171
|
+
_ver, _flags, _method, _mtime, _mdate,
|
|
172
|
+
_crc, _comp, _uncomp, name_len, extra_len = @io.read(26).unpack('vvvvvVVVvv')
|
|
173
|
+
|
|
174
|
+
local_offset + 30 + name_len + extra_len
|
|
175
|
+
end
|
|
176
|
+
end
|
|
177
|
+
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: manticore-smash
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 3.
|
|
4
|
+
version: 3.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Frampt
|
|
@@ -18,6 +18,9 @@ files:
|
|
|
18
18
|
- README.md
|
|
19
19
|
- lib/manticore.rb
|
|
20
20
|
- lib/mdutils/rediscount.rb
|
|
21
|
+
- lib/xlsxkit/sax_parser.rb
|
|
22
|
+
- lib/xlsxkit/workbook.rb
|
|
23
|
+
- lib/xlsxkit/zip_reader.rb
|
|
21
24
|
- lib/xmlutils/formatters.rb
|
|
22
25
|
- lib/xmlutils/node.rb
|
|
23
26
|
- lib/xmlutils/tokenizer.rb
|
|
@@ -42,7 +45,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
42
45
|
- !ruby/object:Gem::Version
|
|
43
46
|
version: '0'
|
|
44
47
|
requirements: []
|
|
45
|
-
rubygems_version: 4.0.
|
|
48
|
+
rubygems_version: 4.0.19
|
|
46
49
|
specification_version: 4
|
|
47
50
|
summary: A multi-format toolkit
|
|
48
51
|
test_files: []
|