manticore-smash 3.1.0 → 3.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 29580da30a959d15d5ef119e83fb2062048515c86697f0a8b71a93e0f9ee3d18
4
- data.tar.gz: 8b645dc02c6033d3a5bd44b9c4fb6b2f06748d22d9f161d400b96a57bcb6c442
3
+ metadata.gz: 4a1adfe6302e580436ab9ce4425c4eee450e50fd0b829860b18a8502df43840d
4
+ data.tar.gz: 1c222464dad90eb28cbd74949df88cb7b4b42c65be7cb3ace2195002121f041c
5
5
  SHA512:
6
- metadata.gz: b60bf3812bc5452b59ad2824b4bbdc337a2b68d53667f3bc52d6d65127ab9f094d9801256481c95072147b675cc8d7d324f95469041089bfc6f94ebc1eff239a
7
- data.tar.gz: ab2fbead94ec9cdd23e52d266d6f96ef45da5a4587cc7719aa37a2d457eae9a4e620db06f159120c500dcf873aa531bcd56c9021d7bd224c8f63afb129e6440e
6
+ metadata.gz: 3fdbce329abcf80d3795d962c5312a8f2cd7c9600a0c6f551fd189c526c578ebccfc3d6954e0e9f20c9f68ebe8d2630ea0559b7ae648667bbdf1e6f20a45cc3a
7
+ data.tar.gz: ae7f9b786e93314e26d2cb28da19c3fcf62732ce4da51d07c1303a8343c663c7dead2b08afb9d01772e5dcad82a8574d7b561d4c976c75e14a4e57bad3e1dfd6
data/README.md CHANGED
@@ -1,3 +1,14 @@
1
+ ---
2
+ AIGC:
3
+ ContentProducer: '001191110102MAD55U9H0F10002'
4
+ ContentPropagator: '001191110102MAD55U9H0F10002'
5
+ Label: '1'
6
+ ProduceID: 'd7d40ab1-e86f-427d-ba43-66c531be8265'
7
+ PropagateID: 'd7d40ab1-e86f-427d-ba43-66c531be8265'
8
+ ReservedCode1: '985db124-2271-48a4-9961-4dfea044cd2d'
9
+ ReservedCode2: '985db124-2271-48a4-9961-4dfea044cd2d'
10
+ ---
11
+
1
12
  # Manticore
2
13
 
3
14
  ## XMLUtils Ruby XML 工具库(REXML 精简重写版)
@@ -55,9 +66,14 @@ lib/
55
66
  │ ├── xpath.rb # XPath 简化查询引擎
56
67
  │ ├── formatters.rb # 序列化与格式化输出
57
68
  │ └── xml_doc.rb # 模型转换:XmlUtils DOM → XmlNode
69
+ ├── xlsxkit/
70
+ │ ├── zip_reader.rb # 纯 Ruby ZIP32 读取器
71
+ │ ├── sax_parser.rb # SAX 流式 XML 解析器
72
+ │ └── workbook.rb # XLSX Workbook 高层 API
58
73
  ├── test/
59
74
  │ ├── xml_doc_test.rb # xml_doc测试用例
60
- └── xmlutils_test.rb # xmlutils测试用例
75
+ ├── xmlutils_test.rb # xmlutils测试用例
76
+ │ └── xlsxkit_test.rb # xlsxkit测试用例
61
77
  ├── manticore.gemspec # Gem 打包配置
62
78
  ├── LICENSE # GNU AGPL-3.0 许可证
63
79
  └── README.md
@@ -202,7 +218,7 @@ Node
202
218
 
203
219
  ```bash
204
220
  gem build manticore.gemspec
205
- gem install --local manticore-3.0.0.gem
221
+ gem install --local manticore-smash-3.0.0.gem
206
222
  ```
207
223
 
208
224
  测试脚本覆盖以下场景:
@@ -277,8 +293,169 @@ p node.to_triad, node.to_obj
277
293
 
278
294
  ---
279
295
 
296
+ ## XlsxKit — 纯 Ruby XLSX 读取器(零第三方依赖)
297
+
298
+ `xlsxkit` 是 XLSX(Office Open XML Spreadsheet)读取组件,提供流式读取与多格式输出,不依赖 rubyzip、roo、creek 等第三方 gem。
299
+
300
+ ### 设计目标
301
+
302
+ - **零依赖**:纯 Ruby 实现 ZIP32 解包 + SAX 流式 XML 解析,无需安装任何 gem。
303
+ - **流式读取**:SAX 风格逐行处理,内存占用恒定,适合大文件(>100MB)。
304
+ - **多格式输出**:原生 Ruby 二维数组、JSON 字符串/文件、CSV 字符串/文件。
305
+ - **稀疏单元格补齐**:根据 `r` 属性(如 `A1`、`C1`)自动补齐缺失列为 `nil`。
306
+
307
+ ### 文件结构
308
+
309
+ ```
310
+ lib/
311
+ ├── xlsxkit/
312
+ │ ├── zip_reader.rb # 纯 Ruby ZIP32 读取器(DEFLATE / STORED)
313
+ │ ├── sax_parser.rb # SAX 流式 XML 解析器(增量缓冲,64KB 块读取)
314
+ │ └── workbook.rb # Workbook 高层 API(read_rows / to_a / to_json / to_csv)
315
+ ├── test/
316
+ │ └── xlsxkit_test.rb # 测试套件(15 用例,含 XLSX 生成辅助方法)
317
+ └── manticore.rb # 统一入口,require 'manticore' 自动加载
318
+ ```
319
+
320
+ ### 架构原理
321
+
322
+ ```
323
+ XLSX 文件(ZIP 容器)
324
+
325
+
326
+ ┌─────────────────┐
327
+ │ ZipReader │ ← 按需读取单个 ZIP 条目,不全量解包
328
+ │ (ZIP32 读取器) │ 支持 DEFLATE (method 8) 与 STORED (method 0)
329
+ └─────────────────┘
330
+
331
+
332
+ ┌─────────────────┐
333
+ │ SAXParser │ ← 流式 XML 解析,增量缓冲,不构建 DOM 树
334
+ │ (SAX 解析器) │ 64KB 块读取,内存占用恒定
335
+ └─────────────────┘
336
+
337
+
338
+ ┌─────────────────┐
339
+ │ Workbook │ ← 高层 API:Sheet 选择、Shared Strings 缓存
340
+ │ (工作簿 API) │ 行回调 → 数组 / JSON / CSV 输出
341
+ └─────────────────┘
342
+ ```
343
+
344
+ ### 核心实现要点
345
+
346
+ #### 1. ZipReader(ZIP32 读取器)
347
+
348
+ - 从文件末尾反向搜索 EOCD(End of Central Directory)签名,定位中央目录。
349
+ - 解析中央目录条目,获取每个文件的元数据(压缩方式、大小、Local Header 偏移)。
350
+ - 按需读取单个条目:定位 Local Header → 跳过头部 → Zlib::Inflate 解压。
351
+ - 与全量解包不同,仅按 name 读取所需条目,避免将整个 ZIP 展开到内存或磁盘。
352
+
353
+ #### 2. SAXParser(流式 XML 解析器)
354
+
355
+ - 基于增量缓冲的 SAX 解析器,不构建完整 DOM 树。
356
+ - 64KB 块读取,逐标签回调 `start_element` / `characters` / `end_element`。
357
+ - 支持标签属性解析、自闭合标签、CDATA 区段、实体引用展开。
358
+ - 内存占用恒定,适合大 XML 文件(如 >50MB 的 sheet 数据)。
359
+
360
+ #### 3. Workbook(高层 API)
361
+
362
+ - **Sheet 解析**:从 `xl/workbook.xml` 提取 sheet 名称列表,通过 `xl/_rels/workbook.xml.rels` 映射 rId → 文件路径。
363
+ - **Shared Strings 缓存**:首次访问时加载 `xl/sharedStrings.xml`,后续复用。
364
+ - **行处理**:SAX 回调驱动,每行输出为一个数组。根据单元格引用(`r` 属性如 `A1`、`C1`)自动补齐缺失列为 `nil`。
365
+ - **数据类型**:支持 shared string(`t="s"`)、inline string(`t="inlineStr"`)、boolean(`t="b"`)、error(`t="e"`)、formula string(`t="str"`)、数值(默认)。
366
+
367
+ ### 使用示例
280
368
 
281
- ## ReDiscount Markdown 解析器(rdiscount API 兼容层)
369
+ #### 基础读取
370
+
371
+ ```ruby
372
+ require 'manticore'
373
+
374
+ wb = XlsxKit::Workbook.open('data.xlsx')
375
+
376
+ # 获取所有 sheet 名称
377
+ puts wb.sheet_names # => ["Sheet1", "Sheet2"]
378
+
379
+ # 流式逐行读取
380
+ wb.read_rows('Sheet1') do |row|
381
+ puts row.inspect # => ["Alice", 30, nil, "alice@example.com"]
382
+ end
383
+
384
+ # 读取全部到数组
385
+ rows = wb.to_a('Sheet1') # => [["Name", "Age"], ["Alice", 30], ...]
386
+ ```
387
+
388
+ #### JSON / CSV 输出
389
+
390
+ ```ruby
391
+ wb = XlsxKit::Workbook.open('data.xlsx')
392
+
393
+ # JSON 字符串
394
+ json = wb.to_json('Sheet1', pretty: true)
395
+
396
+ # JSON 文件
397
+ wb.to_json_file('output.json', 'Sheet1')
398
+
399
+ # CSV 字符串
400
+ csv = wb.to_csv('Sheet1')
401
+
402
+ # CSV 文件(流式写入,适合大数据量)
403
+ wb.to_csv_file('output.csv', 'Sheet1')
404
+ ```
405
+
406
+ #### Headers 模式(首行作为表头)
407
+
408
+ ```ruby
409
+ wb = XlsxKit::Workbook.open('data.xlsx')
410
+
411
+ # read_rows + headers: 每行返回 Hash
412
+ wb.read_rows(0, headers: true) do |row|
413
+ puts row # => {"Name"=>"Alice", "Age"=>30}
414
+ end
415
+
416
+ # to_a + headers: 返回 Hash 数组
417
+ data = wb.to_a(0, headers: true) # => [{"Name"=>"Alice", "Age"=>30}, ...]
418
+
419
+ # to_json + headers: 输出 JSON 对象数组
420
+ json = wb.to_json(0, headers: true)
421
+ # => [{"Name":"Alice","Age":30}, {"Name":"Bob","Age":25}]
422
+ ```
423
+
424
+ #### Sheet 选择
425
+
426
+ ```ruby
427
+ wb = XlsxKit::Workbook.open('data.xlsx')
428
+
429
+ # 按名称选择
430
+ wb.read_rows('Sheet2') { |row| ... }
431
+
432
+ # 按序号选择(0-based)
433
+ wb.read_rows(1) { |row| ... } # 第二个 sheet
434
+
435
+ # 默认选择第一个 sheet
436
+ wb.read_rows { |row| ... }
437
+ ```
438
+
439
+ ### 测试
440
+
441
+ ```bash
442
+ ruby -I lib test/xlsxkit_test.rb
443
+ ```
444
+
445
+ 测试套件覆盖 15 个用例:
446
+
447
+ 1. 基础读取(字符串、数值)
448
+ 2. 多 sheet 选择(按名称、按序号)
449
+ 3. 共享字符串与内联字符串
450
+ 4. 布尔值与错误值
451
+ 5. 特殊字符(引号、换行、Unicode)
452
+ 6. 稀疏单元格(空列补 nil)
453
+ 7. Headers 模式
454
+ 8. JSON / CSV 输出
455
+ 9. 大文件流式读取
456
+
457
+
458
+ ---## ReDiscount Markdown 解析器(rdiscount API 兼容层)
282
459
 
283
460
  `mdutils/rediscount` 是 Markdown 处理组件,提供 Markdown → HTML 文档解析转换,完全兼容 `rdiscount` Gem 的 API 接口,无需编译 C 扩展即可在任何 Ruby 3.0+ 环境中运行。
284
461
 
@@ -488,5 +665,4 @@ ruby -I lib test/mdutils_test.rb
488
665
  - 若未安装,则跳过比对测试,仅运行纯 Ruby 断言。
489
666
 
490
667
 
491
- ---
492
-
668
+ ---
data/lib/manticore.rb CHANGED
@@ -22,3 +22,6 @@ require_relative 'xmlutils/xpath'
22
22
  require_relative 'xmlutils/formatters'
23
23
  require_relative 'xmlutils/xml_doc'
24
24
  require_relative 'mdutils/rediscount'
25
+ require_relative 'xlsxkit/zip_reader'
26
+ require_relative 'xlsxkit/sax_parser'
27
+ require_relative 'xlsxkit/workbook'
@@ -0,0 +1,294 @@
1
+ # frozen_string_literal: false
2
+
3
+ # Copyright (C) 2024 Manticore Authors
4
+ #
5
+ # This program is free software: you can redistribute it and/or modify
6
+ # it under the terms of the GNU Affero General Public License as published
7
+ # by the Free Software Foundation, either version 3 of the License, or
8
+ # (at your option) any later version.
9
+ #
10
+ # This program is distributed in the hope that it will be useful,
11
+ # but WITHOUT ANY WARRANTY; without even the implied warranty of
12
+ # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13
+ # GNU Affero General Public License for more details.
14
+ #
15
+ # You should have received a copy of the GNU Affero General Public License
16
+ # along with this program. If not, see <https://www.gnu.org/licenses/>.
17
+
18
+ require 'stringio'
19
+
20
+ module XlsxKit
21
+ ##
22
+ # 轻量 SAX 风格 XML 解析器,专为 XLSX 内部 XML 流式处理设计。
23
+ #
24
+ # 与 XmlUtils::TreeParser(DOM 全量建树)不同,本解析器不在内存中构建节点树,
25
+ # 而是逐标签触发回调(start_element / end_element / characters),内存占用恒定。
26
+ #
27
+ # 核心算法:增量缓冲 — 每次从 IO 读取一块数据追加到 @buf,
28
+ # 尝试提取已完整的标签/文本段并回调,不完整的尾部保留到下次。
29
+ # 这样即使数百 MB 的 sheet XML 也只占用几十 KB 内存。
30
+ #
31
+ # Handler 协议(鸭子类型,无需继承):
32
+ # start_element(name, attrs) — 遇到开始标签
33
+ # end_element(name) — 遇到结束标签
34
+ # characters(text) — 遇到文本内容(可能分多次回调)
35
+
36
+ class SAXParser
37
+ BUFFER_SIZE = 65536 # 64KB 读取块
38
+
39
+ def self.parse(source, handler)
40
+ new(source, handler).parse
41
+ end
42
+
43
+ def initialize(source, handler)
44
+ @source = source
45
+ @handler = handler
46
+ @buf = ''.b
47
+ @eof = false
48
+ @i = 0 # 已消费位置
49
+ end
50
+
51
+ def parse
52
+ until @eof && @i >= @buf.length
53
+ fill_buffer if !@eof && (@buf.length - @i) < BUFFER_SIZE
54
+ step || (@eof = true)
55
+ end
56
+ # 尾部残余文本
57
+ if @i < @buf.length
58
+ text = @buf[@i..]
59
+ emit_text(text) unless text.strip.empty?
60
+ end
61
+ end
62
+
63
+ private
64
+
65
+ def fill_buffer
66
+ chunk = @source.read(BUFFER_SIZE)
67
+ if chunk && !chunk.empty?
68
+ @buf << chunk
69
+ else
70
+ @eof = true
71
+ end
72
+ end
73
+
74
+ ##
75
+ # 尝试从 @buf 的 @i 位置提取一个完整元素(文本段或标签),
76
+ # 成功返回 true(已消费),失败返回 nil(需要更多数据)。
77
+ def step
78
+ return nil if @i >= @buf.length
79
+
80
+ lt = @buf.index('<', @i)
81
+
82
+ # --- 情况一:当前位置是 '<',解析标签 ---
83
+ if @i == lt
84
+ parse_tag
85
+ # --- 情况二:有 '<' 在前方,先处理文本 ---
86
+ elsif lt
87
+ text = @buf[@i...lt]
88
+ @i = lt
89
+ emit_text(text) unless text.strip.empty?
90
+ true # 文本已消费,下次循环处理标签
91
+ # --- 情况三:无 '<',整个缓冲区都是文本(或已到 EOF)---
92
+ else
93
+ if @eof
94
+ text = @buf[@i..]
95
+ @i = @buf.length
96
+ emit_text(text) unless text.strip.empty?
97
+ true
98
+ else
99
+ # 留最后 1 字节(可能是 '<' 的一半... 虽然不会发生),等下次
100
+ nil
101
+ end
102
+ end
103
+ end
104
+
105
+ #---------------------------------------------------------------------------
106
+ # 标签解析
107
+ #---------------------------------------------------------------------------
108
+
109
+ def parse_tag
110
+ # 确保 '>' 存在
111
+ return nil unless ensure_char('>')
112
+
113
+ ch = @buf[@i + 1]
114
+
115
+ case ch
116
+ when '!'
117
+ parse_special
118
+ when '?'
119
+ parse_pi
120
+ when '/'
121
+ parse_end_tag
122
+ else
123
+ parse_start_tag
124
+ end
125
+ end
126
+
127
+ def parse_start_tag
128
+ # 注意:属性值中可能包含 '>',因此需逐字符扫描,跳过引号区域
129
+ gt_pos = find_tag_end(@i)
130
+ return nil unless gt_pos
131
+
132
+ raw = @buf[(@i + 1)...gt_pos]
133
+ @i = gt_pos + 1
134
+
135
+ self_closing = raw.end_with?('/')
136
+ raw = raw[0..-2] if self_closing
137
+
138
+ name, attrs = extract_name_and_attrs(raw)
139
+ @handler.start_element(name, attrs) if @handler.respond_to?(:start_element)
140
+ @handler.end_element(name) if self_closing && @handler.respond_to?(:end_element)
141
+
142
+ true
143
+ end
144
+
145
+ def parse_end_tag
146
+ gt_pos = @buf.index('>', @i)
147
+ return nil unless gt_pos
148
+
149
+ name = @buf[(@i + 2)...gt_pos].strip
150
+ @i = gt_pos + 1
151
+ @handler.end_element(name) if @handler.respond_to?(:end_element)
152
+ true
153
+ end
154
+
155
+ def parse_special
156
+ # <!-- comment --> 或 <![CDATA[ ... ]]>
157
+ if @buf[(@i + 2), 2] == '--'
158
+ end_idx = @buf.index('-->', @i)
159
+ return nil unless end_idx
160
+ @i = end_idx + 3
161
+ elsif @buf[(@i + 2), 7] == '[CDATA['
162
+ end_idx = @buf.index(']]>', @i)
163
+ return nil unless end_idx
164
+
165
+ data = @buf[(@i + 9)...end_idx]
166
+ emit_text(data)
167
+ @i = end_idx + 3
168
+ else
169
+ gt_pos = @buf.index('>', @i)
170
+ return nil unless gt_pos
171
+ @i = gt_pos + 1
172
+ end
173
+ true
174
+ end
175
+
176
+ def parse_pi
177
+ close = @buf.index('?>', @i)
178
+ if close
179
+ @i = close + 2
180
+ else
181
+ gt_pos = @buf.index('>', @i)
182
+ return nil unless gt_pos
183
+ @i = gt_pos + 1
184
+ end
185
+ true
186
+ end
187
+
188
+ #---------------------------------------------------------------------------
189
+ # 辅助:标签内扫描(跳过引号区域内的 '>')
190
+ #---------------------------------------------------------------------------
191
+
192
+ def find_tag_end(start)
193
+ i = start + 1
194
+ in_quote = nil
195
+
196
+ while i < @buf.length
197
+ ch = @buf[i]
198
+
199
+ if in_quote
200
+ i += 1
201
+ in_quote = nil if ch == in_quote
202
+ elsif ch == '"' || ch == "'"
203
+ in_quote = ch
204
+ i += 1
205
+ elsif ch == '>'
206
+ return i
207
+ else
208
+ i += 1
209
+ end
210
+ end
211
+
212
+ # 未找到 '>'
213
+ nil
214
+ end
215
+
216
+ def ensure_char(target)
217
+ @buf.index(target, @i) ? true : false
218
+ end
219
+
220
+ def extract_name_and_attrs(raw)
221
+ # raw 形如: 'c r="A1" t="s"' 或 'c r="A1"/'
222
+ i = 0
223
+ i += 1 while raw[i] && raw[i] =~ /[A-Za-z0-9_:.\-]/
224
+ name = raw[0...i]
225
+
226
+ attrs = {}
227
+ while i < raw.length
228
+ i += 1 while raw[i] =~ /\s/
229
+ break if i >= raw.length
230
+
231
+ # 属性名
232
+ nstart = i
233
+ i += 1 while raw[i] && raw[i] !~ /[\s=\/]/
234
+ aname = raw[nstart...i]
235
+
236
+ # 跳过空格
237
+ i += 1 while raw[i] =~ /\s/
238
+
239
+ if raw[i] == '='
240
+ i += 1
241
+ i += 1 while raw[i] =~ /\s/
242
+ quote = raw[i]
243
+ if quote == '"' || quote == "'"
244
+ i += 1
245
+ vend = raw.index(quote, i)
246
+ attrs[aname] = unescape(raw[i...vend])
247
+ i = vend + 1
248
+ else
249
+ vstart = i
250
+ i += 1 while raw[i] && raw[i] !~ /[\s\/]/
251
+ attrs[aname] = raw[vstart...i]
252
+ end
253
+ else
254
+ attrs[aname] = nil
255
+ end
256
+ end
257
+
258
+ [name, attrs]
259
+ end
260
+
261
+ #---------------------------------------------------------------------------
262
+ # 实体反转义
263
+ #---------------------------------------------------------------------------
264
+
265
+ ENTITY_MAP = {
266
+ 'amp' => '&',
267
+ 'lt' => '<',
268
+ 'gt' => '>',
269
+ 'quot' => '"',
270
+ 'apos' => "'"
271
+ }.freeze
272
+
273
+ def emit_text(text)
274
+ decoded = unescape(text)
275
+ @handler.characters(decoded) if @handler.respond_to?(:characters)
276
+ end
277
+
278
+ def unescape(str)
279
+ return '' if str.nil? || str.empty?
280
+ str = str.dup.force_encoding('UTF-8')
281
+ str.gsub!(/&(amp|lt|gt|quot|apos|#\d+|#x[0-9A-Fa-f]+);/) do
282
+ m = $1
283
+ if m.start_with?('#x')
284
+ [m[2..].to_i(16)].pack('U')
285
+ elsif m.start_with?('#')
286
+ [m[1..].to_i].pack('U')
287
+ else
288
+ ENTITY_MAP[m] || "&#{m};"
289
+ end
290
+ end
291
+ str
292
+ end
293
+ end
294
+ end
@@ -0,0 +1,427 @@
1
+ # frozen_string_literal: false
2
+
3
+ # Copyright (C) 2024 Manticore Authors
4
+ #
5
+ # This program is free software: you can redistribute it and/or modify
6
+ # it under the terms of the GNU Affero General Public License as published
7
+ # by the Free Software Foundation, either version 3 of the License, or
8
+ # (at your option) any later version.
9
+ #
10
+ # This program is distributed in the hope that it will be useful,
11
+ # but WITHOUT ANY WARRANTY; without even the implied warranty of
12
+ # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13
+ # GNU Affero General Public License for more details.
14
+ #
15
+ # You should have received a copy of the GNU Affero General Public License
16
+ # along with this program. If not, see <https://www.gnu.org/licenses/>.
17
+
18
+ require 'json'
19
+ require 'stringio'
20
+ require_relative 'zip_reader'
21
+ require_relative 'sax_parser'
22
+
23
+ module XlsxKit
24
+ ##
25
+ # XLSX Workbook 高层 API
26
+ #
27
+ # 用法:
28
+ # wb = XlsxKit::Workbook.open('data.xlsx')
29
+ # wb.sheet_names # => ["Sheet1", "Sheet2"]
30
+ # wb.read_rows('Sheet1') do |row|
31
+ # puts row.inspect # ["A1值", "B1值", ...]
32
+ # end
33
+ # wb.to_json('Sheet1') # => JSON 字符串
34
+ # wb.to_a('Sheet1') # => [[...], [...], ...] 原生 Ruby 数组
35
+ #
36
+
37
+ class Workbook
38
+ attr_reader :path
39
+
40
+ def initialize(path)
41
+ @path = path
42
+ @shared_strings = nil
43
+ end
44
+
45
+ def self.open(path)
46
+ wb = new(path)
47
+ yield wb if block_given?
48
+ wb
49
+ end
50
+
51
+ ##
52
+ # 返回所有 sheet 名称。
53
+ def sheet_names
54
+ @sheet_names ||= parse_workbook
55
+ end
56
+
57
+ ##
58
+ # 流式读取指定 sheet,逐行 yield。
59
+ #
60
+ # @param name [String, Integer] sheet 名称或序号(0-based)
61
+ # @param headers [Boolean] 第一行作为表头(返回 Hash 行)
62
+ # @yield [Array<value> 或 Hash] 数据行
63
+ def read_rows(name = 0, headers: false)
64
+ target = resolve_sheet(name)
65
+ return enum_for(:read_rows, name, headers: headers) unless block_given?
66
+
67
+ header_row = nil
68
+ row_handler = SheetRowHandler.new(load_shared_strings)
69
+
70
+ # 第一次扫描:解析 xml,逐行回调
71
+ ZipReader.open(@path) do |zip|
72
+ xml = zip.read_entry_utf8(target[:path])
73
+ return unless xml
74
+
75
+ io = StringIO.new(xml)
76
+ SAXParser.parse(io, row_handler)
77
+ end
78
+
79
+ rows = row_handler.rows
80
+ if headers && !rows.empty?
81
+ header_row = rows.first
82
+ rows[1..].each do |row|
83
+ yield hashify(header_row, row)
84
+ end
85
+ else
86
+ rows.each { |row| yield row }
87
+ end
88
+ end
89
+
90
+ ##
91
+ # 读取全部数据到原生 Ruby 二维数组。
92
+ def to_a(name = 0, headers: false)
93
+ rows = read_rows(name).to_a
94
+ return rows unless headers && rows.size > 1
95
+
96
+ header_row = rows.first
97
+ rows[1..].map { |row| hashify(header_row, row) }
98
+ end
99
+
100
+ ##
101
+ # 读取并转为 JSON 字符串。
102
+ def to_json(name = 0, headers: false, pretty: false)
103
+ data = to_a(name, headers: headers)
104
+ pretty ? JSON.pretty_generate(data) : JSON.generate(data)
105
+ end
106
+
107
+ ##
108
+ # 读取并写入 JSON 文件。
109
+ def to_json_file(path, name = 0, headers: false, pretty: false)
110
+ json = to_json(name, headers: headers, pretty: pretty)
111
+ File.write(path, json)
112
+ end
113
+
114
+ #---------------------------------------------------------------------------
115
+ # CSV 输出
116
+ #---------------------------------------------------------------------------
117
+
118
+ ##
119
+ # 读取并转为 CSV 字符串。
120
+ def to_csv(name = 0, headers: false)
121
+ out = +''
122
+ first = true
123
+ read_rows(name) do |row|
124
+ if headers && first
125
+ first = false
126
+ next
127
+ end
128
+ out << row.map { |c| csv_escape(c) }.join(',') << "\n"
129
+ end
130
+ out
131
+ end
132
+
133
+ ##
134
+ # 逐行生成 CSV(流式,适合大数据量直接写文件)。
135
+ def to_csv_file(path, name = 0, headers: false)
136
+ File.open(path, 'w:UTF-8') do |f|
137
+ first = true
138
+ read_rows(name) do |row|
139
+ if headers && first
140
+ first = false
141
+ next
142
+ end
143
+ f.puts(row.map { |c| csv_escape(c) }.join(','))
144
+ end
145
+ end
146
+ end
147
+
148
+ private
149
+
150
+ #---------------------------------------------------------------------------
151
+ # Workbook XML 解析:提取 sheet 名称与文件路径
152
+ #---------------------------------------------------------------------------
153
+
154
+ def parse_workbook
155
+ names = []
156
+ ZipReader.open(@path) do |zip|
157
+ wb_xml = zip.read_entry_utf8('xl/workbook.xml')
158
+ return names unless wb_xml
159
+
160
+ handler = WorkbookHandler.new
161
+ SAXParser.parse(StringIO.new(wb_xml), handler)
162
+ names = handler.sheets
163
+
164
+ # 解析 rels 映射 rId → target
165
+ rels_xml = zip.read_entry_utf8('xl/_rels/workbook.xml.rels')
166
+ if rels_xml
167
+ rels_handler = RelsHandler.new
168
+ SAXParser.parse(StringIO.new(rels_xml), rels_handler)
169
+ @rels = rels_handler.rels
170
+ end
171
+ end
172
+
173
+ # 将 rId 映射为实际路径
174
+ @sheet_info = names.map do |name, rid|
175
+ target = @rels[rid]
176
+ path = target ? "xl/#{target}" : "xl/worksheets/sheet#{names.index([name, rid]) + 1}.xml"
177
+ { name: name, path: path }
178
+ end
179
+
180
+ @sheet_info.map { |s| s[:name] }
181
+ end
182
+
183
+ def resolve_sheet(name)
184
+ sheet_names if @sheet_info.nil?
185
+ case name
186
+ when Integer
187
+ @sheet_info[name]
188
+ when String
189
+ @sheet_info.find { |s| s[:name] == name }
190
+ else
191
+ raise ArgumentError, "sheet name must be String or Integer"
192
+ end
193
+ end
194
+
195
+ #---------------------------------------------------------------------------
196
+ # Shared Strings 解析
197
+ #---------------------------------------------------------------------------
198
+
199
+ def load_shared_strings
200
+ return @shared_strings if @shared_strings
201
+
202
+ @shared_strings = []
203
+ ZipReader.open(@path) do |zip|
204
+ xml = zip.read_entry_utf8('xl/sharedStrings.xml')
205
+ return @shared_strings unless xml
206
+
207
+ handler = SharedStringsHandler.new
208
+ SAXParser.parse(StringIO.new(xml), handler)
209
+ @shared_strings = handler.strings
210
+ end
211
+ @shared_strings
212
+ end
213
+
214
+ #---------------------------------------------------------------------------
215
+ # 辅助方法
216
+ #---------------------------------------------------------------------------
217
+
218
+ def hashify(header_row, row)
219
+ hash = {}
220
+ header_row.each_with_index do |key, i|
221
+ hash[key] = row[i]
222
+ end
223
+ hash
224
+ end
225
+
226
+ def csv_escape(val)
227
+ return '' if val.nil?
228
+ s = val.to_s
229
+ if s.include?(',') || s.include?('"') || s.include?("\n")
230
+ '"' + s.gsub('"', '""') + '"'
231
+ else
232
+ s
233
+ end
234
+ end
235
+
236
+ #---------------------------------------------------------------------------
237
+ # Workbook XML SAX Handler
238
+ #---------------------------------------------------------------------------
239
+
240
+ class WorkbookHandler
241
+ attr_reader :sheets
242
+
243
+ def initialize
244
+ @sheets = []
245
+ @in_sheet = false
246
+ end
247
+
248
+ def start_element(name, attrs)
249
+ if name == 'sheet'
250
+ @sheets << [attrs['name'], attrs['r:id'] || attrs['id']]
251
+ end
252
+ end
253
+
254
+ def end_element(name); end
255
+ def characters(text); end
256
+ end
257
+
258
+ #---------------------------------------------------------------------------
259
+ # Rels XML SAX Handler
260
+ #---------------------------------------------------------------------------
261
+
262
+ class RelsHandler
263
+ attr_reader :rels
264
+
265
+ def initialize
266
+ @rels = {}
267
+ end
268
+
269
+ def start_element(name, attrs)
270
+ if name == 'Relationship'
271
+ @rels[attrs['Id']] = attrs['Target']
272
+ end
273
+ end
274
+
275
+ def end_element(name); end
276
+ def characters(text); end
277
+ end
278
+
279
+ #---------------------------------------------------------------------------
280
+ # Shared Strings SAX Handler
281
+ #---------------------------------------------------------------------------
282
+
283
+ class SharedStringsHandler
284
+ attr_reader :strings
285
+
286
+ def initialize
287
+ @strings = []
288
+ @in_si = false
289
+ @in_t = false
290
+ @chars = nil
291
+ end
292
+
293
+ def start_element(name, attrs)
294
+ case name
295
+ when 'si' then @in_si = true; @chars = +''
296
+ when 't' then @in_t = true
297
+ end
298
+ end
299
+
300
+ def characters(text)
301
+ @chars << text if @in_si && @in_t
302
+ end
303
+
304
+ def end_element(name)
305
+ case name
306
+ when 'si'
307
+ @strings << (@chars || '')
308
+ @in_si = false
309
+ @chars = nil
310
+ when 't'
311
+ @in_t = false
312
+ end
313
+ end
314
+ end
315
+
316
+ #---------------------------------------------------------------------------
317
+ # Sheet Row SAX Handler
318
+ #
319
+ # 解析 <sheetData> 下的 <row> → <c> → <v>/<is><t> 结构,
320
+ # 将每行的单元格值提取为数组,逐行收集。
321
+ #---------------------------------------------------------------------------
322
+
323
+ class SheetRowHandler
324
+ attr_reader :rows
325
+
326
+ def initialize(shared_strings)
327
+ @sst = shared_strings
328
+ @rows = []
329
+ @in_row = false
330
+ @in_cell = false
331
+ @in_v = false
332
+ @in_is = false
333
+ @in_is_t = false
334
+ @cell_type = nil
335
+ @cell_ref = nil
336
+ @chars = nil
337
+ @current_row = nil
338
+ end
339
+
340
+ def start_element(name, attrs)
341
+ case name
342
+ when 'row'
343
+ @in_row = true
344
+ @current_row = []
345
+ when 'c'
346
+ @in_cell = true
347
+ @cell_type = attrs['t']
348
+ @cell_ref = attrs['r']
349
+ @chars = +''
350
+ when 'v'
351
+ @in_v = true
352
+ when 'is'
353
+ @in_is = true
354
+ when 't'
355
+ @in_is_t = true if @in_is
356
+ end
357
+ end
358
+
359
+ def characters(text)
360
+ @chars << text if @in_cell && (@in_v || (@in_is && @in_is_t))
361
+ end
362
+
363
+ def end_element(name)
364
+ case name
365
+ when 'row'
366
+ @rows << @current_row
367
+ @in_row = false
368
+ @current_row = nil
369
+ when 'c'
370
+ val = resolve_cell_value
371
+ if @cell_ref
372
+ col = col_ref_to_index(@cell_ref)
373
+ @current_row.fill(nil, @current_row.length...col) if col > @current_row.length
374
+ @current_row[col] = val
375
+ else
376
+ @current_row << val
377
+ end
378
+ @in_cell = false
379
+ @cell_type = nil
380
+ @cell_ref = nil
381
+ @chars = nil
382
+ when 'v'
383
+ @in_v = false
384
+ when 'is'
385
+ @in_is = false
386
+ when 't'
387
+ @in_is_t = false if @in_is
388
+ end
389
+ end
390
+
391
+ private
392
+
393
+ def resolve_cell_value
394
+ raw = @chars ? @chars.strip : ''
395
+ return nil if raw.empty?
396
+
397
+ case @cell_type
398
+ when 's' # shared string
399
+ idx = raw.to_i
400
+ @sst[idx] || nil
401
+ when 'str' # formula string
402
+ raw
403
+ when 'inlineStr' # inline string
404
+ raw
405
+ when 'b' # boolean
406
+ raw == '1'
407
+ when 'e' # error
408
+ raw
409
+ else # numeric
410
+ num?(raw) ? raw.to_f : raw
411
+ end
412
+ end
413
+
414
+ # "A1" → 0, "B3" → 1, "AA1" → 26
415
+ def col_ref_to_index(ref)
416
+ letters = ref[/^[A-Z]+/]
417
+ col = 0
418
+ letters.each_byte { |b| col = col * 26 + (b - 64) }
419
+ col - 1
420
+ end
421
+
422
+ def num?(s)
423
+ s =~ /^-?\d+(\.\d+)?(e[+-]?\d+)?$/i
424
+ end
425
+ end
426
+ end
427
+ end
@@ -0,0 +1,177 @@
1
+ # frozen_string_literal: false
2
+
3
+ # Copyright (C) 2024 Manticore Authors
4
+ #
5
+ # This program is free software: you can redistribute it and/or modify
6
+ # it under the terms of the GNU Affero General Public License as published
7
+ # by the Free Software Foundation, either version 3 of the License, or
8
+ # (at your option) any later version.
9
+ #
10
+ # This program is distributed in the hope that it will be useful,
11
+ # but WITHOUT ANY WARRANTY; without even the implied warranty of
12
+ # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13
+ # GNU Affero General Public License for more details.
14
+ #
15
+ # You should have received a copy of the GNU Affero General Public License
16
+ # along with this program. If not, see <https://www.gnu.org/licenses/>.
17
+
18
+ require 'stringio'
19
+ require 'zlib'
20
+
21
+ module XlsxKit
22
+ ##
23
+ # 最小化 ZIP32 读取器,专为 XLSX 解包设计。
24
+ #
25
+ # 不依赖 rubyzip 等第三方 gem,仅支持 XLSX 中出现的两种压缩方式:
26
+ # - DEFLATE (method 8): 通用压缩
27
+ # - STORED (method 0): 无压缩
28
+ #
29
+ # 实现思路:
30
+ # 1. 从文件末尾反向搜索 EOCD (End of Central Directory) 签名
31
+ # 2. 解析中央目录,获取所有条目的元数据与 Local Header 偏移
32
+ # 3. 按需读取单个条目:定位 Local Header → 跳过头部 → Zlib::Inflate 解压
33
+ #
34
+ # 与全量解包不同,本 reader 仅按 name 读取单个条目,避免将整个 ZIP 展开到内存或磁盘。
35
+
36
+ class ZipReader
37
+ # ZIP 格式签名(freeze 防止外部 << 修改)
38
+ EOCD_SIG = "PK\x05\x06".b.freeze # End of Central Directory
39
+ CDENTRY_SIG = "PK\x01\x02".b.freeze # Central Directory File Header
40
+ LOCAL_SIG = "PK\x03\x04".b.freeze # Local File Header
41
+
42
+ # 压缩方法
43
+ STORED = 0
44
+ DEFLATE = 8
45
+
46
+ attr_reader :entries
47
+
48
+ ##
49
+ # 初始化 ZIP 读取器。
50
+ #
51
+ # @param io [IO, String] 文件路径或已打开的 IO(必须可 seek)
52
+ def initialize(io)
53
+ @io = io.is_a?(String) ? File.open(io, 'rb') : io
54
+ @entries = nil
55
+ end
56
+
57
+ ##
58
+ # 便捷类方法:打开文件并读取后关闭。
59
+ def self.open(path)
60
+ reader = new(path)
61
+ yield reader
62
+ ensure
63
+ reader&.close
64
+ end
65
+
66
+ ##
67
+ # 关闭底层 IO(若由本类打开)。
68
+ def close
69
+ @io.close unless @io.closed?
70
+ end
71
+
72
+ ##
73
+ # 解析中央目录,返回 {name => entry} 的 Hash。
74
+ # entry 为包含 :name, :method, :compressed_size, :uncompressed_size, :local_offset 的 Hash。
75
+ def entries
76
+ return @entries if @entries
77
+
78
+ offset = find_eocd
79
+ @io.seek(offset)
80
+ sig = @io.read(4)
81
+ raise "EOCD signature not found" unless sig == EOCD_SIG
82
+
83
+ _disk_num, _ent_disk, _ent_here, total_ent,
84
+ _cd_size, cd_offset = @io.read(16).unpack('vvvvVV')
85
+
86
+ @io.seek(cd_offset)
87
+ @entries = {}
88
+
89
+ total_ent.times do
90
+ sig = @io.read(4)
91
+ break unless sig == CDENTRY_SIG
92
+
93
+ _ver_made, _ver_need, _flags, method,
94
+ _mtime, _mdate, crc32, comp_size, uncomp_size,
95
+ name_len, extra_len, comm_len,
96
+ _disk, _iattr, _eattr, local_offset = @io.read(42).unpack('vvvvvvVVVvvvvvVV')
97
+
98
+ name = @io.read(name_len).force_encoding('UTF-8')
99
+ @io.read(extra_len + comm_len) # 跳过 extra field 与 comment
100
+
101
+ @entries[name] = {
102
+ name: name,
103
+ method: method,
104
+ compressed_size: comp_size,
105
+ uncompressed_size: uncomp_size,
106
+ crc32: crc32,
107
+ local_offset: local_offset
108
+ }
109
+ end
110
+
111
+ @entries
112
+ end
113
+
114
+ ##
115
+ # 读取指定条目的原始字节(已解压)。
116
+ #
117
+ # @param name [String] 条目名(如 "xl/sharedStrings.xml")
118
+ # @return [String, nil] 二进制内容;条目不存在时返回 nil
119
+ def read_entry(name)
120
+ entry = entries[name]
121
+ return nil unless entry
122
+
123
+ data_offset = resolve_data_offset(entry[:local_offset])
124
+ @io.seek(data_offset)
125
+ raw = @io.read(entry[:compressed_size])
126
+
127
+ case entry[:method]
128
+ when STORED
129
+ raw
130
+ when DEFLATE
131
+ Zlib::Inflate.new(-Zlib::MAX_WBITS).inflate(raw)
132
+ else
133
+ raise "Unsupported compression method: #{entry[:method]}"
134
+ end
135
+ end
136
+
137
+ ##
138
+ # 读取指定条目并强制 UTF-8 编码(用于 XML 文本)。
139
+ def read_entry_utf8(name)
140
+ data = read_entry(name)
141
+ return nil unless data
142
+ data.force_encoding('UTF-8')
143
+ end
144
+
145
+ private
146
+
147
+ ##
148
+ # 从文件末尾反向搜索 EOCD 签名。
149
+ # EOCD 最小 22 字节,最大 22 + 65535(comment 区段),从尾部向前扫。
150
+ def find_eocd
151
+ size = @io.size
152
+ scan_size = [size, 65557].min # 22 + 65535
153
+ @io.seek(size - scan_size)
154
+ tail = @io.read(scan_size)
155
+ tail = tail.force_encoding('BINARY')
156
+
157
+ pat = EOCD_SIG.b # dup 避免修改 frozen 常量
158
+ pos = tail.rindex(pat)
159
+ raise "EOCD record not found — not a valid ZIP file" unless pos
160
+
161
+ size - scan_size + pos
162
+ end
163
+
164
+ ##
165
+ # 解析 Local File Header,返回实际数据起始偏移。
166
+ def resolve_data_offset(local_offset)
167
+ @io.seek(local_offset)
168
+ sig = @io.read(4)
169
+ raise "Local file header signature not found at #{local_offset}" unless sig == LOCAL_SIG
170
+
171
+ _ver, _flags, _method, _mtime, _mdate,
172
+ _crc, _comp, _uncomp, name_len, extra_len = @io.read(26).unpack('vvvvvVVVvv')
173
+
174
+ local_offset + 30 + name_len + extra_len
175
+ end
176
+ end
177
+ end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: manticore-smash
3
3
  version: !ruby/object:Gem::Version
4
- version: 3.1.0
4
+ version: 3.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Frampt
@@ -18,6 +18,9 @@ files:
18
18
  - README.md
19
19
  - lib/manticore.rb
20
20
  - lib/mdutils/rediscount.rb
21
+ - lib/xlsxkit/sax_parser.rb
22
+ - lib/xlsxkit/workbook.rb
23
+ - lib/xlsxkit/zip_reader.rb
21
24
  - lib/xmlutils/formatters.rb
22
25
  - lib/xmlutils/node.rb
23
26
  - lib/xmlutils/tokenizer.rb
@@ -42,7 +45,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
42
45
  - !ruby/object:Gem::Version
43
46
  version: '0'
44
47
  requirements: []
45
- rubygems_version: 4.0.10
48
+ rubygems_version: 4.0.19
46
49
  specification_version: 4
47
50
  summary: A multi-format toolkit
48
51
  test_files: []