bibframe_ruby 0.1.2 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +24 -2
- data/docs/superpowers/plans/2026-09-09-marc-conversion.md +537 -0
- data/docs/superpowers/specs/2026-09-09-marc-conversion-design.md +110 -0
- data/lib/bibframe_ruby/marc_converter.rb +67 -0
- data/lib/bibframe_ruby/parser.rb +4 -2
- data/lib/bibframe_ruby/version.rb +1 -1
- data/lib/bibframe_ruby.rb +7 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-001-007.xsl +2276 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-006,008.xsl +1295 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-010-048.xsl +1712 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-050-088.xsl +713 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-1XX,7XX,8XX-names.xsl +1141 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-200-247not240-Titles.xsl +771 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-240andX30-UnifTitle.xsl +535 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-250-270.xsl +283 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-3XX.xsl +2245 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-460-468-SeriesTreat.xsl +135 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-490-510-Links.xsl +145 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-5XX.xsl +1448 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-600-662.xsl +1417 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-720+740to755.xsl +333 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-758.xsl +102 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-760-788-Links.xsl +771 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-841-887.xsl +328 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-880.xsl +119 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-ControlSubfields.xsl +495 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-LDR.xsl +183 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-Preprocess0-Splitting.xsl +765 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-Process6-Series.xsl +627 -0
- data/vendor/marc2bibframe2/xsl/ConvSpec-Process8-ProvAct.xsl +1021 -0
- data/vendor/marc2bibframe2/xsl/conf/abbreviations.xml +13 -0
- data/vendor/marc2bibframe2/xsl/conf/code-to-script.xml +399 -0
- data/vendor/marc2bibframe2/xsl/conf/codeMaps.xml +723 -0
- data/vendor/marc2bibframe2/xsl/conf/exclusions.xml +4 -0
- data/vendor/marc2bibframe2/xsl/conf/iso6392-to-1.xml +615 -0
- data/vendor/marc2bibframe2/xsl/conf/languageCrosswalk.xml +1503 -0
- data/vendor/marc2bibframe2/xsl/conf/map880.xml +175 -0
- data/vendor/marc2bibframe2/xsl/conf/scriptCrosswalk.xml +210 -0
- data/vendor/marc2bibframe2/xsl/conf/subjectThesaurus.xml +29 -0
- data/vendor/marc2bibframe2/xsl/marc2bibframe2.xsl +610 -0
- data/vendor/marc2bibframe2/xsl/naco-normalize.xsl +205 -0
- data/vendor/marc2bibframe2/xsl/utils.xsl +963 -0
- data/vendor/marc2bibframe2/xsl/variables.xsl +108 -0
- metadata +68 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 656c3e0e3d78aa14eca5596e153c7aada8a5a89fb6645bed67ed0aba0b9a49b8
|
|
4
|
+
data.tar.gz: 64dead66a959b38cbba57831c898a00807039266af64bcb09f4a2928c1a27fda
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: '0957c3c819f85e626b24225efd65bdee191b884b91bcb9147e7b21bb1de25eee54076fa1017e6911af1d192f6c6f0fc231a8e55795aa3bb610443dab8658cdf0'
|
|
7
|
+
data.tar.gz: d8f596aa28c8873cfa68d24340f350bbdb0846b7e210c23fe1c92c59049ed026dcf34ffe0f736fea7e7715f422750ff42caf08d85f973716a24126d00fc8ee96
|
data/README.md
CHANGED
|
@@ -193,6 +193,26 @@ result.works.first.instances.first.extent
|
|
|
193
193
|
# => "532 pages"
|
|
194
194
|
```
|
|
195
195
|
|
|
196
|
+
### Converting from MARC
|
|
197
|
+
|
|
198
|
+
Convert MARC records (binary MARC21 or MARCXML) to BIBFRAME using the Library of Congress [marc2bibframe2](https://github.com/lcnetdev/marc2bibframe2) XSLT stylesheets:
|
|
199
|
+
|
|
200
|
+
```ruby
|
|
201
|
+
# Convert a binary MARC file
|
|
202
|
+
graph = BibframeRuby.convert_marc("record.mrc")
|
|
203
|
+
|
|
204
|
+
# With a custom base URI for minted resource URIs
|
|
205
|
+
graph = BibframeRuby.convert_marc("record.mrc", baseuri: "https://mylib.org/catalog/")
|
|
206
|
+
|
|
207
|
+
# MARCXML is also accepted
|
|
208
|
+
graph = BibframeRuby.convert_marc("record.xml", baseuri: "https://mylib.org/catalog/")
|
|
209
|
+
|
|
210
|
+
graph.works.first.title.main_title
|
|
211
|
+
# => "The Title"
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
The `baseuri` parameter controls the base URI for generated resource identifiers (default: `http://example.org/`). The optional `idsource` parameter identifies the converting agent.
|
|
215
|
+
|
|
196
216
|
### Stub Resources
|
|
197
217
|
|
|
198
218
|
When a parsed document references an external resource by URI (e.g., an agent in the Library of Congress), a stub `Resource` is created with just the `id` set:
|
|
@@ -229,8 +249,10 @@ All models inherit from `Resource` and support hash-style access via `[]` for an
|
|
|
229
249
|
| Format | Status | File Extension |
|
|
230
250
|
|--------|--------|----------------|
|
|
231
251
|
| JSON-LD | Supported | `.jsonld` |
|
|
232
|
-
| Turtle |
|
|
233
|
-
| RDF/XML |
|
|
252
|
+
| Turtle | Supported | `.ttl` |
|
|
253
|
+
| RDF/XML | Supported | `.rdf` |
|
|
254
|
+
| MARC21 (binary) | Supported (convert) | `.mrc` |
|
|
255
|
+
| MARCXML | Supported (convert) | `.xml` |
|
|
234
256
|
|
|
235
257
|
## Development
|
|
236
258
|
|
|
@@ -0,0 +1,537 @@
|
|
|
1
|
+
# MARC Conversion Implementation Plan
|
|
2
|
+
|
|
3
|
+
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
4
|
+
|
|
5
|
+
**Goal:** Add `BibframeRuby.convert_marc` to convert MARC records to BIBFRAME Ruby objects via the LC marc2bibframe2 XSLT stylesheets.
|
|
6
|
+
|
|
7
|
+
**Architecture:** MarcConverter detects MARC format (binary vs XML), converts to MARCXML if needed, applies vendored XSLT to produce RDF/XML, then feeds into the existing Parser/GraphBuilder pipeline. The XSLT is vendored from lcnetdev/marc2bibframe2 v3.1.0.
|
|
8
|
+
|
|
9
|
+
**Tech Stack:** Ruby 3.2+, RSpec, `nokogiri` (XSLT), `marc` (binary MARC reading), `rdf-rdfxml` (RDF/XML parsing)
|
|
10
|
+
|
|
11
|
+
**Spec:** `docs/superpowers/specs/2026-09-09-marc-conversion-design.md`
|
|
12
|
+
|
|
13
|
+
## Global Constraints
|
|
14
|
+
|
|
15
|
+
- Ruby >= 3.2.0
|
|
16
|
+
- All files use `# frozen_string_literal: true`
|
|
17
|
+
- BIBFRAME namespace: `http://id.loc.gov/ontologies/bibframe/`
|
|
18
|
+
- Conventional commits
|
|
19
|
+
- TDD: write failing tests first, then implement
|
|
20
|
+
- marc2bibframe2 XSLT vendored at tag v3.1.0
|
|
21
|
+
- The XSLT requires `Dir.chdir` into the xsl directory (or equivalent) because it uses relative `xsl:include` paths
|
|
22
|
+
- Binary MARC detection: content starts with 5 digits (the MARC leader record length); MARCXML starts with `<`
|
|
23
|
+
|
|
24
|
+
---
|
|
25
|
+
|
|
26
|
+
### Task 1: Vendor XSLT, Add Dependencies, Add RDF/XML Parser Support
|
|
27
|
+
|
|
28
|
+
Vendor the marc2bibframe2 stylesheets, add `nokogiri` and `marc` to the gemspec, and add `:rdfxml` to Parser's READER_MAP.
|
|
29
|
+
|
|
30
|
+
**Files:**
|
|
31
|
+
- Create: `vendor/marc2bibframe2/xsl/` (copy from lcnetdev/marc2bibframe2 v3.1.0)
|
|
32
|
+
- Modify: `bibframe_ruby.gemspec` (add nokogiri, marc dependencies)
|
|
33
|
+
- Modify: `lib/bibframe_ruby/parser.rb` (add rdfxml reader)
|
|
34
|
+
- Create: `spec/bibframe_ruby/parser_rdfxml_spec.rb`
|
|
35
|
+
- Create: `spec/fixtures/work.rdf` (RDF/XML fixture for parser test)
|
|
36
|
+
|
|
37
|
+
**Interfaces:**
|
|
38
|
+
- Consumes: nothing new
|
|
39
|
+
- Produces: `Parser` supports `format: :rdfxml`, vendored XSL at `vendor/marc2bibframe2/xsl/marc2bibframe2.xsl`
|
|
40
|
+
|
|
41
|
+
- [ ] **Step 1: Clone and vendor the XSLT stylesheets**
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
cd /tmp && rm -rf marc2bibframe2
|
|
45
|
+
git clone --depth 1 --branch v3.1.0 https://github.com/lcnetdev/marc2bibframe2.git
|
|
46
|
+
mkdir -p vendor/marc2bibframe2
|
|
47
|
+
cp -r /tmp/marc2bibframe2/xsl vendor/marc2bibframe2/
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Verify the vendored files are present:
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
ls vendor/marc2bibframe2/xsl/marc2bibframe2.xsl
|
|
54
|
+
ls vendor/marc2bibframe2/xsl/conf/codeMaps.xml
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
- [ ] **Step 2: Create an RDF/XML fixture**
|
|
58
|
+
|
|
59
|
+
Generate one by running the XSLT on a minimal MARCXML input:
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
bundle exec ruby -rnokogiri -e '
|
|
63
|
+
Dir.chdir("vendor/marc2bibframe2/xsl") do
|
|
64
|
+
xsl = Nokogiri::XSLT(File.read("marc2bibframe2.xsl"))
|
|
65
|
+
marcxml = %q(<?xml version="1.0" encoding="UTF-8"?>
|
|
66
|
+
<collection xmlns="http://www.loc.gov/MARC21/slim">
|
|
67
|
+
<record>
|
|
68
|
+
<leader>00000nam a2200000 a 4500</leader>
|
|
69
|
+
<controlfield tag="001">test123</controlfield>
|
|
70
|
+
<controlfield tag="008">240101s2024 nyu 000 0 eng d</controlfield>
|
|
71
|
+
<datafield tag="245" ind1="1" ind2="0">
|
|
72
|
+
<subfield code="a">Test Title</subfield>
|
|
73
|
+
</datafield>
|
|
74
|
+
</record>
|
|
75
|
+
</collection>)
|
|
76
|
+
doc = Nokogiri::XML(marcxml)
|
|
77
|
+
result = xsl.transform(doc, ["baseuri", "\"http://example.org/\"", "idsource", "\"test\""])
|
|
78
|
+
File.write("../../../spec/fixtures/work.rdf", result.to_xml)
|
|
79
|
+
end
|
|
80
|
+
'
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
- [ ] **Step 3: Add dependencies to gemspec**
|
|
84
|
+
|
|
85
|
+
Add after the existing `rdf-turtle` dependency:
|
|
86
|
+
|
|
87
|
+
```ruby
|
|
88
|
+
spec.add_dependency "marc"
|
|
89
|
+
spec.add_dependency "nokogiri"
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
Run `bundle install` to update the lock file.
|
|
93
|
+
|
|
94
|
+
- [ ] **Step 4: Write failing RDF/XML parser spec**
|
|
95
|
+
|
|
96
|
+
```ruby
|
|
97
|
+
# spec/bibframe_ruby/parser_rdfxml_spec.rb
|
|
98
|
+
# frozen_string_literal: true
|
|
99
|
+
|
|
100
|
+
RSpec.describe BibframeRuby::Parser do
|
|
101
|
+
let(:work_rdf) { read_fixture("work.rdf") }
|
|
102
|
+
|
|
103
|
+
describe "RDF/XML parsing" do
|
|
104
|
+
it "returns an RDF::Graph from RDF/XML input" do
|
|
105
|
+
parser = described_class.new(work_rdf, format: :rdfxml)
|
|
106
|
+
result = parser.parse
|
|
107
|
+
expect(result).to be_a(RDF::Graph)
|
|
108
|
+
end
|
|
109
|
+
|
|
110
|
+
it "parses triples from RDF/XML input" do
|
|
111
|
+
parser = described_class.new(work_rdf, format: :rdfxml)
|
|
112
|
+
graph = parser.parse
|
|
113
|
+
expect(graph.count).to be > 0
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
it "contains the work subject" do
|
|
117
|
+
parser = described_class.new(work_rdf, format: :rdfxml)
|
|
118
|
+
graph = parser.parse
|
|
119
|
+
subjects = graph.subjects.map(&:to_s)
|
|
120
|
+
expect(subjects).to include("http://example.org/test123#Work")
|
|
121
|
+
end
|
|
122
|
+
end
|
|
123
|
+
end
|
|
124
|
+
|
|
125
|
+
RSpec.describe BibframeRuby do
|
|
126
|
+
describe "RDF/XML integration" do
|
|
127
|
+
it "parses an RDF/XML file via parse_file" do
|
|
128
|
+
result = described_class.parse_file(fixture_path("work.rdf"))
|
|
129
|
+
expect(result).to be_a(BibframeRuby::Graph)
|
|
130
|
+
expect(result.works.length).to eq(1)
|
|
131
|
+
end
|
|
132
|
+
|
|
133
|
+
it "produces a Work with a title from RDF/XML" do
|
|
134
|
+
result = described_class.parse_file(fixture_path("work.rdf"))
|
|
135
|
+
work = result.works.first
|
|
136
|
+
expect(work.title).to be_a(BibframeRuby::Title)
|
|
137
|
+
expect(work.title.main_title).to eq("Test Title")
|
|
138
|
+
end
|
|
139
|
+
end
|
|
140
|
+
end
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
- [ ] **Step 5: Run test to verify it fails**
|
|
144
|
+
|
|
145
|
+
Run: `bundle exec rspec spec/bibframe_ruby/parser_rdfxml_spec.rb`
|
|
146
|
+
Expected: FAIL with `Unsupported format: rdfxml`
|
|
147
|
+
|
|
148
|
+
- [ ] **Step 6: Add RDF/XML support to Parser**
|
|
149
|
+
|
|
150
|
+
In `lib/bibframe_ruby/parser.rb`, add the require and reader:
|
|
151
|
+
|
|
152
|
+
Add after `require "rdf/turtle"`:
|
|
153
|
+
|
|
154
|
+
```ruby
|
|
155
|
+
require "rdf/rdfxml"
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
Update READER_MAP:
|
|
159
|
+
|
|
160
|
+
```ruby
|
|
161
|
+
READER_MAP = {
|
|
162
|
+
jsonld: JSON::LD::Reader,
|
|
163
|
+
turtle: RDF::Turtle::Reader,
|
|
164
|
+
rdfxml: RDF::RDFXML::Reader
|
|
165
|
+
}.freeze
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
- [ ] **Step 7: Run tests and verify they pass**
|
|
169
|
+
|
|
170
|
+
Run: `bundle exec rspec spec/bibframe_ruby/parser_rdfxml_spec.rb`
|
|
171
|
+
Expected: all pass
|
|
172
|
+
|
|
173
|
+
- [ ] **Step 8: Run full test suite**
|
|
174
|
+
|
|
175
|
+
Run: `bundle exec rspec`
|
|
176
|
+
Expected: all pass
|
|
177
|
+
|
|
178
|
+
- [ ] **Step 9: Commit**
|
|
179
|
+
|
|
180
|
+
```bash
|
|
181
|
+
git add vendor/marc2bibframe2/ spec/fixtures/work.rdf spec/bibframe_ruby/parser_rdfxml_spec.rb lib/bibframe_ruby/parser.rb bibframe_ruby.gemspec Gemfile.lock
|
|
182
|
+
git commit -m "feat: vendor marc2bibframe2 XSLT, add RDF/XML parser support, add nokogiri and marc dependencies"
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
---
|
|
186
|
+
|
|
187
|
+
### Task 2: MarcConverter
|
|
188
|
+
|
|
189
|
+
Create the MarcConverter class that converts MARC input to RDF/XML via XSLT.
|
|
190
|
+
|
|
191
|
+
**Files:**
|
|
192
|
+
- Create: `lib/bibframe_ruby/marc_converter.rb`
|
|
193
|
+
- Create: `spec/bibframe_ruby/marc_converter_spec.rb`
|
|
194
|
+
- Create: `spec/fixtures/record.mrc` (generated programmatically in spec setup)
|
|
195
|
+
|
|
196
|
+
**Interfaces:**
|
|
197
|
+
- Consumes: vendored XSLT at `vendor/marc2bibframe2/xsl/marc2bibframe2.xsl`, `nokogiri`, `marc` gems
|
|
198
|
+
- Produces: `BibframeRuby::MarcConverter.new(input, baseuri:, idsource:).convert` returns RDF/XML string
|
|
199
|
+
|
|
200
|
+
- [ ] **Step 1: Write failing MarcConverter spec**
|
|
201
|
+
|
|
202
|
+
```ruby
|
|
203
|
+
# spec/bibframe_ruby/marc_converter_spec.rb
|
|
204
|
+
# frozen_string_literal: true
|
|
205
|
+
|
|
206
|
+
require "marc"
|
|
207
|
+
|
|
208
|
+
RSpec.describe BibframeRuby::MarcConverter do
|
|
209
|
+
let(:marcxml) do
|
|
210
|
+
<<~XML
|
|
211
|
+
<?xml version="1.0" encoding="UTF-8"?>
|
|
212
|
+
<collection xmlns="http://www.loc.gov/MARC21/slim">
|
|
213
|
+
<record>
|
|
214
|
+
<leader>00000nam a2200000 a 4500</leader>
|
|
215
|
+
<controlfield tag="001">conv123</controlfield>
|
|
216
|
+
<controlfield tag="008">240101s2024 nyu 000 0 eng d</controlfield>
|
|
217
|
+
<datafield tag="245" ind1="1" ind2="0">
|
|
218
|
+
<subfield code="a">MARC Conversion Test</subfield>
|
|
219
|
+
</datafield>
|
|
220
|
+
</record>
|
|
221
|
+
</collection>
|
|
222
|
+
XML
|
|
223
|
+
end
|
|
224
|
+
|
|
225
|
+
let(:binary_marc) do
|
|
226
|
+
record = MARC::Record.new
|
|
227
|
+
record.leader = "00000nam a2200000 a 4500"
|
|
228
|
+
record.append(MARC::ControlField.new("001", "conv456"))
|
|
229
|
+
record.append(MARC::ControlField.new("008", "240101s2024 nyu 000 0 eng d"))
|
|
230
|
+
record.append(MARC::DataField.new("245", "1", "0", ["a", "Binary MARC Test"]))
|
|
231
|
+
writer = StringIO.new
|
|
232
|
+
marc_writer = MARC::Writer.new(writer)
|
|
233
|
+
marc_writer.write(record)
|
|
234
|
+
marc_writer.close
|
|
235
|
+
writer.string
|
|
236
|
+
end
|
|
237
|
+
|
|
238
|
+
describe "#convert" do
|
|
239
|
+
it "converts MARCXML to RDF/XML" do
|
|
240
|
+
converter = described_class.new(marcxml, baseuri: "http://example.org/", idsource: "test")
|
|
241
|
+
result = converter.convert
|
|
242
|
+
expect(result).to include("rdf:RDF")
|
|
243
|
+
expect(result).to include("bf:Work")
|
|
244
|
+
expect(result).to include("MARC Conversion Test")
|
|
245
|
+
end
|
|
246
|
+
|
|
247
|
+
it "converts binary MARC to RDF/XML" do
|
|
248
|
+
converter = described_class.new(binary_marc, baseuri: "http://example.org/", idsource: "test")
|
|
249
|
+
result = converter.convert
|
|
250
|
+
expect(result).to include("rdf:RDF")
|
|
251
|
+
expect(result).to include("bf:Work")
|
|
252
|
+
expect(result).to include("Binary MARC Test")
|
|
253
|
+
end
|
|
254
|
+
|
|
255
|
+
it "passes baseuri to the XSLT" do
|
|
256
|
+
converter = described_class.new(marcxml, baseuri: "http://mylib.org/catalog/", idsource: "test")
|
|
257
|
+
result = converter.convert
|
|
258
|
+
expect(result).to include("http://mylib.org/catalog/")
|
|
259
|
+
end
|
|
260
|
+
|
|
261
|
+
it "uses default baseuri when none provided" do
|
|
262
|
+
converter = described_class.new(marcxml, idsource: "test")
|
|
263
|
+
result = converter.convert
|
|
264
|
+
expect(result).to include("http://example.org/")
|
|
265
|
+
end
|
|
266
|
+
end
|
|
267
|
+
|
|
268
|
+
describe ".marcxml?" do
|
|
269
|
+
it "returns true for XML content" do
|
|
270
|
+
expect(described_class.marcxml?(marcxml)).to be true
|
|
271
|
+
end
|
|
272
|
+
|
|
273
|
+
it "returns false for binary MARC content" do
|
|
274
|
+
expect(described_class.marcxml?(binary_marc)).to be false
|
|
275
|
+
end
|
|
276
|
+
end
|
|
277
|
+
end
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
- [ ] **Step 2: Run test to verify it fails**
|
|
281
|
+
|
|
282
|
+
Run: `bundle exec rspec spec/bibframe_ruby/marc_converter_spec.rb`
|
|
283
|
+
Expected: FAIL with `uninitialized constant BibframeRuby::MarcConverter`
|
|
284
|
+
|
|
285
|
+
- [ ] **Step 3: Implement MarcConverter**
|
|
286
|
+
|
|
287
|
+
```ruby
|
|
288
|
+
# lib/bibframe_ruby/marc_converter.rb
|
|
289
|
+
# frozen_string_literal: true
|
|
290
|
+
|
|
291
|
+
# BibframeRuby::MarcConverter: Converts MARC records to BIBFRAME RDF/XML via XSLT
|
|
292
|
+
|
|
293
|
+
require "nokogiri"
|
|
294
|
+
require "marc"
|
|
295
|
+
require "stringio"
|
|
296
|
+
|
|
297
|
+
module BibframeRuby
|
|
298
|
+
# Converts MARC (binary or MARCXML) to BIBFRAME RDF/XML using the vendored marc2bibframe2 XSLT.
|
|
299
|
+
class MarcConverter
|
|
300
|
+
XSL_DIR = File.expand_path("../../vendor/marc2bibframe2/xsl", __dir__)
|
|
301
|
+
XSL_PATH = File.join(XSL_DIR, "marc2bibframe2.xsl")
|
|
302
|
+
|
|
303
|
+
def initialize(input, baseuri: "http://example.org/", idsource: nil)
|
|
304
|
+
@input = input
|
|
305
|
+
@baseuri = baseuri
|
|
306
|
+
@idsource = idsource
|
|
307
|
+
end
|
|
308
|
+
|
|
309
|
+
def convert
|
|
310
|
+
xml = self.class.marcxml?(@input) ? @input : binary_to_marcxml(@input)
|
|
311
|
+
doc = Nokogiri::XML(xml)
|
|
312
|
+
rdfxml = transform(doc)
|
|
313
|
+
rdfxml.to_xml
|
|
314
|
+
end
|
|
315
|
+
|
|
316
|
+
def self.marcxml?(input)
|
|
317
|
+
input.lstrip.start_with?("<")
|
|
318
|
+
end
|
|
319
|
+
|
|
320
|
+
private
|
|
321
|
+
|
|
322
|
+
def binary_to_marcxml(input)
|
|
323
|
+
reader = MARC::Reader.new(StringIO.new(input))
|
|
324
|
+
output = StringIO.new
|
|
325
|
+
writer = MARC::XMLWriter.new(output)
|
|
326
|
+
reader.each { |record| writer.write(record) }
|
|
327
|
+
writer.close
|
|
328
|
+
output.string
|
|
329
|
+
end
|
|
330
|
+
|
|
331
|
+
def transform(doc)
|
|
332
|
+
params = ["baseuri", "\"#{@baseuri}\""]
|
|
333
|
+
params += ["idsource", "\"#{@idsource}\""] if @idsource
|
|
334
|
+
|
|
335
|
+
Dir.chdir(XSL_DIR) do
|
|
336
|
+
self.class.stylesheet.transform(doc, params)
|
|
337
|
+
end
|
|
338
|
+
end
|
|
339
|
+
|
|
340
|
+
def self.stylesheet
|
|
341
|
+
@stylesheet ||= Dir.chdir(XSL_DIR) do
|
|
342
|
+
Nokogiri::XSLT(File.read(XSL_PATH))
|
|
343
|
+
end
|
|
344
|
+
end
|
|
345
|
+
end
|
|
346
|
+
end
|
|
347
|
+
```
|
|
348
|
+
|
|
349
|
+
- [ ] **Step 4: Add require to bibframe_ruby.rb**
|
|
350
|
+
|
|
351
|
+
Add after `require_relative "bibframe_ruby/graph"`:
|
|
352
|
+
|
|
353
|
+
```ruby
|
|
354
|
+
require_relative "bibframe_ruby/marc_converter"
|
|
355
|
+
```
|
|
356
|
+
|
|
357
|
+
- [ ] **Step 5: Run tests and verify they pass**
|
|
358
|
+
|
|
359
|
+
Run: `bundle exec rspec spec/bibframe_ruby/marc_converter_spec.rb`
|
|
360
|
+
Expected: all pass
|
|
361
|
+
|
|
362
|
+
- [ ] **Step 6: Commit**
|
|
363
|
+
|
|
364
|
+
```bash
|
|
365
|
+
git add lib/bibframe_ruby/marc_converter.rb spec/bibframe_ruby/marc_converter_spec.rb lib/bibframe_ruby.rb
|
|
366
|
+
git commit -m "feat: add MarcConverter for MARC to RDF/XML conversion via XSLT"
|
|
367
|
+
```
|
|
368
|
+
|
|
369
|
+
---
|
|
370
|
+
|
|
371
|
+
### Task 3: Public API and Integration Tests
|
|
372
|
+
|
|
373
|
+
Add `BibframeRuby.convert_marc` and integration tests that verify the full pipeline.
|
|
374
|
+
|
|
375
|
+
**Files:**
|
|
376
|
+
- Modify: `lib/bibframe_ruby.rb` (add `self.convert_marc`)
|
|
377
|
+
- Create: `spec/convert_marc_spec.rb`
|
|
378
|
+
|
|
379
|
+
**Interfaces:**
|
|
380
|
+
- Consumes: `BibframeRuby::MarcConverter.new(input, baseuri:, idsource:).convert` returns RDF/XML string, `BibframeRuby.parse(input, format: :rdfxml)` returns Graph
|
|
381
|
+
- Produces: `BibframeRuby.convert_marc(path, baseuri:, idsource:)` returns `BibframeRuby::Graph`
|
|
382
|
+
|
|
383
|
+
- [ ] **Step 1: Create a binary MARC fixture file**
|
|
384
|
+
|
|
385
|
+
```ruby
|
|
386
|
+
# Run this to create the fixture:
|
|
387
|
+
bundle exec ruby -e '
|
|
388
|
+
require "marc"
|
|
389
|
+
record = MARC::Record.new
|
|
390
|
+
record.leader = "00000nam a2200000 a 4500"
|
|
391
|
+
record.append(MARC::ControlField.new("001", "fixture789"))
|
|
392
|
+
record.append(MARC::ControlField.new("008", "240101s2024 nyu 000 0 eng d"))
|
|
393
|
+
record.append(MARC::DataField.new("245", "1", "0", ["a", "Fixture MARC Record"]))
|
|
394
|
+
record.append(MARC::DataField.new("260", " ", " ", ["a", "New York :"], ["b", "Publisher,"], ["c", "2024."]))
|
|
395
|
+
writer = MARC::Writer.new("spec/fixtures/record.mrc")
|
|
396
|
+
writer.write(record)
|
|
397
|
+
writer.close
|
|
398
|
+
'
|
|
399
|
+
```
|
|
400
|
+
|
|
401
|
+
- [ ] **Step 2: Write failing integration spec**
|
|
402
|
+
|
|
403
|
+
```ruby
|
|
404
|
+
# spec/convert_marc_spec.rb
|
|
405
|
+
# frozen_string_literal: true
|
|
406
|
+
|
|
407
|
+
RSpec.describe BibframeRuby do
|
|
408
|
+
describe ".convert_marc" do
|
|
409
|
+
let(:marc_path) { fixture_path("record.mrc") }
|
|
410
|
+
|
|
411
|
+
it "returns a Graph" do
|
|
412
|
+
result = described_class.convert_marc(marc_path)
|
|
413
|
+
expect(result).to be_a(BibframeRuby::Graph)
|
|
414
|
+
end
|
|
415
|
+
|
|
416
|
+
it "produces Works from MARC" do
|
|
417
|
+
result = described_class.convert_marc(marc_path)
|
|
418
|
+
expect(result.works.length).to eq(1)
|
|
419
|
+
end
|
|
420
|
+
|
|
421
|
+
it "produces Instances from MARC" do
|
|
422
|
+
result = described_class.convert_marc(marc_path)
|
|
423
|
+
expect(result.instances.length).to eq(1)
|
|
424
|
+
end
|
|
425
|
+
|
|
426
|
+
it "populates the Work title" do
|
|
427
|
+
result = described_class.convert_marc(marc_path)
|
|
428
|
+
work = result.works.first
|
|
429
|
+
expect(work.title).to be_a(BibframeRuby::Title)
|
|
430
|
+
expect(work.title.main_title).to eq("Fixture MARC Record")
|
|
431
|
+
end
|
|
432
|
+
|
|
433
|
+
it "passes baseuri through to the transform" do
|
|
434
|
+
result = described_class.convert_marc(marc_path, baseuri: "http://mylib.org/")
|
|
435
|
+
work = result.works.first
|
|
436
|
+
expect(work.id).to start_with("http://mylib.org/")
|
|
437
|
+
end
|
|
438
|
+
|
|
439
|
+
it "populates the Work language" do
|
|
440
|
+
result = described_class.convert_marc(marc_path)
|
|
441
|
+
work = result.works.first
|
|
442
|
+
expect(work.language).to eq("http://id.loc.gov/vocabulary/languages/eng")
|
|
443
|
+
end
|
|
444
|
+
end
|
|
445
|
+
end
|
|
446
|
+
```
|
|
447
|
+
|
|
448
|
+
- [ ] **Step 3: Run test to verify it fails**
|
|
449
|
+
|
|
450
|
+
Run: `bundle exec rspec spec/convert_marc_spec.rb`
|
|
451
|
+
Expected: FAIL with `NoMethodError: undefined method 'convert_marc'`
|
|
452
|
+
|
|
453
|
+
- [ ] **Step 4: Implement convert_marc**
|
|
454
|
+
|
|
455
|
+
Add to `lib/bibframe_ruby.rb`, after the `parse_uri` method:
|
|
456
|
+
|
|
457
|
+
```ruby
|
|
458
|
+
def self.convert_marc(path, baseuri: "http://example.org/", idsource: nil)
|
|
459
|
+
input = File.read(path)
|
|
460
|
+
rdfxml = MarcConverter.new(input, baseuri: baseuri, idsource: idsource).convert
|
|
461
|
+
parse(rdfxml, format: :rdfxml)
|
|
462
|
+
end
|
|
463
|
+
```
|
|
464
|
+
|
|
465
|
+
- [ ] **Step 5: Run tests and verify they pass**
|
|
466
|
+
|
|
467
|
+
Run: `bundle exec rspec spec/convert_marc_spec.rb`
|
|
468
|
+
Expected: all pass
|
|
469
|
+
|
|
470
|
+
- [ ] **Step 6: Run full test suite**
|
|
471
|
+
|
|
472
|
+
Run: `bundle exec rspec`
|
|
473
|
+
Expected: all pass
|
|
474
|
+
|
|
475
|
+
- [ ] **Step 7: Commit**
|
|
476
|
+
|
|
477
|
+
```bash
|
|
478
|
+
git add lib/bibframe_ruby.rb spec/convert_marc_spec.rb spec/fixtures/record.mrc
|
|
479
|
+
git commit -m "feat: add convert_marc public API for MARC to BIBFRAME conversion"
|
|
480
|
+
```
|
|
481
|
+
|
|
482
|
+
---
|
|
483
|
+
|
|
484
|
+
### Task 4: Update README
|
|
485
|
+
|
|
486
|
+
Add MARC conversion documentation to the README.
|
|
487
|
+
|
|
488
|
+
**Files:**
|
|
489
|
+
- Modify: `README.md`
|
|
490
|
+
|
|
491
|
+
**Interfaces:**
|
|
492
|
+
- Consumes: everything from Tasks 1-3
|
|
493
|
+
- Produces: updated documentation
|
|
494
|
+
|
|
495
|
+
- [ ] **Step 1: Read the current README**
|
|
496
|
+
|
|
497
|
+
Read `README.md` to understand the existing structure.
|
|
498
|
+
|
|
499
|
+
- [ ] **Step 2: Add MARC conversion section**
|
|
500
|
+
|
|
501
|
+
After the "Combining Multiple Documents" section and before "Stub Resources", add:
|
|
502
|
+
|
|
503
|
+
```markdown
|
|
504
|
+
### Converting from MARC
|
|
505
|
+
|
|
506
|
+
Convert MARC records (binary MARC21 or MARCXML) to BIBFRAME using the Library of Congress [marc2bibframe2](https://github.com/lcnetdev/marc2bibframe2) XSLT stylesheets:
|
|
507
|
+
|
|
508
|
+
```ruby
|
|
509
|
+
# Convert a binary MARC file
|
|
510
|
+
graph = BibframeRuby.convert_marc("record.mrc")
|
|
511
|
+
|
|
512
|
+
# With a custom base URI for minted resource URIs
|
|
513
|
+
graph = BibframeRuby.convert_marc("record.mrc", baseuri: "https://mylib.org/catalog/")
|
|
514
|
+
|
|
515
|
+
# MARCXML is also accepted
|
|
516
|
+
graph = BibframeRuby.convert_marc("record.xml", baseuri: "https://mylib.org/catalog/")
|
|
517
|
+
|
|
518
|
+
graph.works.first.title.main_title
|
|
519
|
+
# => "The Title"
|
|
520
|
+
```
|
|
521
|
+
|
|
522
|
+
The `baseuri` parameter controls the base URI for generated resource identifiers (default: `http://example.org/`). The optional `idsource` parameter identifies the converting agent.
|
|
523
|
+
```
|
|
524
|
+
|
|
525
|
+
Also add to the Supported Formats table:
|
|
526
|
+
|
|
527
|
+
```
|
|
528
|
+
| MARC21 (binary) | Supported (convert) | `.mrc` |
|
|
529
|
+
| MARCXML | Supported (convert) | `.xml` |
|
|
530
|
+
```
|
|
531
|
+
|
|
532
|
+
- [ ] **Step 3: Commit**
|
|
533
|
+
|
|
534
|
+
```bash
|
|
535
|
+
git add README.md
|
|
536
|
+
git commit -m "docs: add MARC conversion to README"
|
|
537
|
+
```
|
|
@@ -0,0 +1,110 @@
|
|
|
1
|
+
# MARC to BIBFRAME Conversion Design
|
|
2
|
+
|
|
3
|
+
## Overview
|
|
4
|
+
|
|
5
|
+
Add `BibframeRuby.convert_marc` to convert MARC records (binary MARC21 or MARCXML) into BIBFRAME Ruby objects. Uses the Library of Congress marc2bibframe2 XSLT stylesheets vendored into the gem, with Nokogiri for the XSLT transform and the `marc` gem for reading binary MARC.
|
|
6
|
+
|
|
7
|
+
## Public API
|
|
8
|
+
|
|
9
|
+
```ruby
|
|
10
|
+
# Convert from a MARC file path (binary .mrc or MARCXML .xml)
|
|
11
|
+
graph = BibframeRuby.convert_marc("record.mrc", baseuri: "http://example.org/")
|
|
12
|
+
|
|
13
|
+
# Same Graph object as parse/parse_file/parse_uri
|
|
14
|
+
graph.works # => [Work, ...]
|
|
15
|
+
graph.instances # => [Instance, ...]
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
### Parameters
|
|
19
|
+
|
|
20
|
+
- `path` -- file path to a binary MARC (.mrc) or MARCXML (.xml) file
|
|
21
|
+
- `baseuri:` -- base URI for minted resource URIs (default: `"http://example.org/"`)
|
|
22
|
+
- `idsource:` -- identifier for the converting agent (default: `nil`)
|
|
23
|
+
|
|
24
|
+
## Pipeline
|
|
25
|
+
|
|
26
|
+
1. `MarcConverter` reads the file and detects format (binary MARC vs MARCXML) by checking if content starts with digits (MARC leader) or `<` (XML)
|
|
27
|
+
2. If binary MARC, converts to MARCXML using the `marc` gem's `MARC::XMLWriter`
|
|
28
|
+
3. Applies the vendored marc2bibframe2 XSLT via Nokogiri, producing RDF/XML
|
|
29
|
+
4. Parses the RDF/XML into an `RDF::Graph` via the Parser (`:rdfxml` format)
|
|
30
|
+
5. Feeds into the existing `Graph.from_rdf` pipeline
|
|
31
|
+
|
|
32
|
+
## Components
|
|
33
|
+
|
|
34
|
+
### MarcConverter (`lib/bibframe_ruby/marc_converter.rb`)
|
|
35
|
+
|
|
36
|
+
Owns the MARC-to-RDF/XML conversion:
|
|
37
|
+
|
|
38
|
+
- `MarcConverter.new(input, baseuri:, idsource:)` -- accepts file content as a string
|
|
39
|
+
- `#convert` -- returns RDF/XML string
|
|
40
|
+
- Detects binary MARC vs MARCXML by inspecting content
|
|
41
|
+
- Converts binary MARC to MARCXML via `marc` gem
|
|
42
|
+
- Loads and caches the vendored XSLT stylesheet (compiled once per process via class-level memoization)
|
|
43
|
+
- Transforms MARCXML to RDF/XML via Nokogiri XSLT
|
|
44
|
+
- Passes `baseuri` and `idsource` as XSLT parameters
|
|
45
|
+
|
|
46
|
+
### Parser changes
|
|
47
|
+
|
|
48
|
+
Add `:rdfxml` to `READER_MAP` with `RDF::RDFXML::Reader`, and `require "rdf/rdfxml"`. This follows the same pattern as the Turtle addition and also enables `parse_file("something.rdf")`.
|
|
49
|
+
|
|
50
|
+
### Entry point
|
|
51
|
+
|
|
52
|
+
`BibframeRuby.convert_marc(path, baseuri:, idsource:)` in `lib/bibframe_ruby.rb`:
|
|
53
|
+
|
|
54
|
+
```ruby
|
|
55
|
+
def self.convert_marc(path, baseuri: "http://example.org/", idsource: nil)
|
|
56
|
+
input = File.read(path)
|
|
57
|
+
rdfxml = MarcConverter.new(input, baseuri: baseuri, idsource: idsource).convert
|
|
58
|
+
parse(rdfxml, format: :rdfxml)
|
|
59
|
+
end
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
## Vendored Assets
|
|
63
|
+
|
|
64
|
+
`vendor/marc2bibframe2/xsl/` -- the full `xsl/` directory from lcnetdev/marc2bibframe2 at a specific release tag. Includes:
|
|
65
|
+
|
|
66
|
+
- `marc2bibframe2.xsl` -- main entry stylesheet
|
|
67
|
+
- `variables.xsl`, `utils.xsl` -- shared utilities
|
|
68
|
+
- `ConvSpec-*.xsl` -- ~20 conversion spec files
|
|
69
|
+
- `conf/` -- lookup tables (subjectThesaurus.xml, codeMaps.xml, etc.)
|
|
70
|
+
|
|
71
|
+
The stylesheets are CC0 licensed.
|
|
72
|
+
|
|
73
|
+
## New Dependencies
|
|
74
|
+
|
|
75
|
+
- `nokogiri` -- XSLT 1.0 transformation via libxslt
|
|
76
|
+
- `marc` -- reading binary MARC21 records and converting to MARCXML
|
|
77
|
+
|
|
78
|
+
Both added to the gemspec as runtime dependencies.
|
|
79
|
+
|
|
80
|
+
## File Structure
|
|
81
|
+
|
|
82
|
+
```
|
|
83
|
+
vendor/
|
|
84
|
+
marc2bibframe2/
|
|
85
|
+
xsl/ # Vendored from lcnetdev/marc2bibframe2
|
|
86
|
+
marc2bibframe2.xsl
|
|
87
|
+
variables.xsl
|
|
88
|
+
utils.xsl
|
|
89
|
+
ConvSpec-*.xsl
|
|
90
|
+
conf/
|
|
91
|
+
lib/
|
|
92
|
+
bibframe_ruby.rb # Add convert_marc, require marc_converter
|
|
93
|
+
bibframe_ruby/
|
|
94
|
+
parser.rb # Add rdfxml to READER_MAP
|
|
95
|
+
marc_converter.rb # New: MARC to RDF/XML conversion
|
|
96
|
+
spec/
|
|
97
|
+
fixtures/
|
|
98
|
+
record.mrc # Binary MARC fixture
|
|
99
|
+
bibframe_ruby/
|
|
100
|
+
marc_converter_spec.rb # MarcConverter unit tests
|
|
101
|
+
parser_rdfxml_spec.rb # RDF/XML reader tests
|
|
102
|
+
convert_marc_spec.rb # Integration tests
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
## Testing Strategy
|
|
106
|
+
|
|
107
|
+
- **MarcConverter spec** -- detects binary MARC vs MARCXML, converts binary to MARCXML, applies XSLT to produce valid RDF/XML, passes baseuri/idsource parameters
|
|
108
|
+
- **Parser RDF/XML spec** -- verifies the new :rdfxml reader works (same pattern as Turtle spec)
|
|
109
|
+
- **Integration spec** -- `BibframeRuby.convert_marc(path)` returns a Graph with Works, Instances, titles
|
|
110
|
+
- All tests use local fixtures and vendored stylesheets, no network calls
|