traject 3.4.0 → 3.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/workflows/ruby.yml +26 -0
- data/CHANGES.md +11 -0
- data/README.md +1 -1
- data/doc/settings.md +4 -2
- data/lib/traject/command_line.rb +34 -43
- data/lib/traject/macros/marc21.rb +1 -1
- data/lib/traject/marc_extractor.rb +3 -3
- data/lib/traject/solr_json_writer.rb +28 -10
- data/lib/traject/version.rb +1 -1
- data/test/command_line_test.rb +51 -0
- data/test/solr_json_writer_test.rb +45 -10
- data/traject.gemspec +1 -1
- metadata +7 -11
- data/.travis.yml +0 -17
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 01ca968682bb3fc2a8313131ef6344bfc9e5418b767b2900c3d799caa356d016
|
4
|
+
data.tar.gz: a3fd6c9a3bec88c6ba592500ea170357b059533f28c5ba3fb2fe72de39702a2a
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 93547927e90b7947588c1983bd37de4651722bebaff7aaad2d3965ec46eed8c647971f1ce093beb86a5c47c2efa999516d00fb6956682c98913cd54ea5a1a2b8
|
7
|
+
data.tar.gz: 128ea6e2517711f2324541215f814430f963b0042ae321ebc3f29646398990dc09d2a307acc32fb89a70142d74763fdfee6a61d220043622b5b8b11fbcd645d8
|
@@ -0,0 +1,26 @@
|
|
1
|
+
name: CI
|
2
|
+
|
3
|
+
on:
|
4
|
+
push:
|
5
|
+
branches: [ master ]
|
6
|
+
pull_request:
|
7
|
+
branches: ['**']
|
8
|
+
|
9
|
+
jobs:
|
10
|
+
tests:
|
11
|
+
runs-on: ubuntu-latest
|
12
|
+
strategy:
|
13
|
+
fail-fast: false
|
14
|
+
matrix:
|
15
|
+
ruby: [ '2.4', '2.5', '2.6', '2.7', 'jruby-9.1', 'jruby-9.2' ]
|
16
|
+
name: Ruby ${{ matrix.ruby }}
|
17
|
+
steps:
|
18
|
+
- uses: actions/checkout@v2
|
19
|
+
- name: Set up Ruby
|
20
|
+
uses: ruby/setup-ruby@v1
|
21
|
+
with:
|
22
|
+
ruby-version: ${{ matrix.ruby }}
|
23
|
+
- name: Install dependencies
|
24
|
+
run: bundle install --jobs 4 --retry 3
|
25
|
+
- name: Run tests
|
26
|
+
run: bundle exec rake
|
data/CHANGES.md
CHANGED
@@ -6,6 +6,17 @@
|
|
6
6
|
|
7
7
|
*
|
8
8
|
|
9
|
+
*
|
10
|
+
|
11
|
+
## 3.5.0
|
12
|
+
|
13
|
+
* `traject -v` and `traject -h` correctly return 0 exit code indicating success.
|
14
|
+
|
15
|
+
* upgrade to slop gem 4.x, which carries with it a slightly different format of human-readable command-line arg errors, should be otherwise invisible.
|
16
|
+
|
17
|
+
* the SolrJsonWriter now supports HTTP basic auth credentials embedded in `solr.url` or `solr.update_url`, eg `http://user:pass@example.org/solr` https://github.com/traject/traject/pull/262
|
18
|
+
|
19
|
+
|
9
20
|
## 3.4.0
|
10
21
|
|
11
22
|
* XML-mode `extract_xpath` now supports extracting attribute values with xpath @attr syntax.
|
data/README.md
CHANGED
@@ -9,7 +9,7 @@ Traject can also be generalized to a set of tools for getting structured data fr
|
|
9
9
|
**Traject is stable, mature software, that is already being used in production by its authors and several other institutions.**
|
10
10
|
|
11
11
|
[](http://badge.fury.io/rb/traject)
|
12
|
-
[](https://github.com/traject/traject/actions?query=workflow%3ACI+branch%3Amaster)
|
13
13
|
|
14
14
|
|
15
15
|
## Background/Goals
|
data/doc/settings.md
CHANGED
@@ -83,7 +83,8 @@ settings are applied first of all. It's recommended you use `provide`.
|
|
83
83
|
### Writing to solr
|
84
84
|
|
85
85
|
* `json_writer.pretty_print`: used by the JsonWriter, if set to true, will output pretty printed json (with added whitespace) for easier human readability. Default false.
|
86
|
-
|
86
|
+
|
87
|
+
* `solr.url`: URL to connect to a solr instance for indexing, eg http://example.org:8983/solr . Command-line short-cut `-u`. (Can include embedded HTTP basic auth as eg `http://user:pass@example.org/solr`)
|
87
88
|
|
88
89
|
* `solr.version`: Set to eg "1.4.0", "4.3.0"; currently un-used, but in the future will control some default settings, and/or sanity check and warn you if you're doing something that might not work with that version of solr. Set now for help in the future.
|
89
90
|
|
@@ -93,7 +94,8 @@ settings are applied first of all. It's recommended you use `provide`.
|
|
93
94
|
|
94
95
|
* `solr_writer.thread_pool`: defaults to 1 (single bg thread). A thread pool is used for submitting docs to solr. Set to 0 or nil to disable threading. Set to 1, there will still be a single bg thread doing the adds. May make sense to set higher than number of cores on your indexing machine, as these threads will mostly be waiting on Solr. Speed/capacity of your solr might be more relevant. Note that processing_thread_pool threads can end up submitting to solr too, if solr_json_writer.thread_pool is full.
|
95
96
|
|
96
|
-
* `solr_writer.basic_auth_user`, `solr_writer.basic_auth_password`: Not set by default but when both are set the default writer is configured with basic auth.
|
97
|
+
* `solr_writer.basic_auth_user`, `solr_writer.basic_auth_password`: Not set by default but when both are set the default writer is configured with basic auth. You can also just embed basic
|
98
|
+
auth credentials in `solr.url` using standard URI syntax.
|
97
99
|
|
98
100
|
|
99
101
|
### Dealing with MARC data
|
data/lib/traject/command_line.rb
CHANGED
@@ -29,10 +29,10 @@ module Traject
|
|
29
29
|
self.console = $stderr
|
30
30
|
|
31
31
|
self.orig_argv = argv.dup
|
32
|
-
self.remaining_argv = argv
|
33
32
|
|
34
|
-
self.slop = create_slop!
|
35
|
-
self.options =
|
33
|
+
self.slop = create_slop!(argv)
|
34
|
+
self.options = self.slop
|
35
|
+
self.remaining_argv = self.slop.arguments
|
36
36
|
end
|
37
37
|
|
38
38
|
# Returns true on success or false on failure; may also raise exceptions;
|
@@ -40,11 +40,11 @@ module Traject
|
|
40
40
|
def execute
|
41
41
|
if options[:version]
|
42
42
|
self.console.puts "traject version #{Traject::VERSION}"
|
43
|
-
return
|
43
|
+
return true
|
44
44
|
end
|
45
45
|
if options[:help]
|
46
|
-
self.console.puts slop.
|
47
|
-
return
|
46
|
+
self.console.puts slop.to_s
|
47
|
+
return true
|
48
48
|
end
|
49
49
|
|
50
50
|
|
@@ -179,11 +179,11 @@ module Traject
|
|
179
179
|
end
|
180
180
|
|
181
181
|
def arg_check!
|
182
|
-
if options[:command] == "process" && (options[:conf]
|
182
|
+
if options[:command] == "process" && (!options[:conf] || options[:conf].length == 0)
|
183
183
|
self.console.puts "Error: Missing required configuration file"
|
184
184
|
self.console.puts "Exiting..."
|
185
185
|
self.console.puts
|
186
|
-
self.console.puts self.slop.
|
186
|
+
self.console.puts self.slop.to_s
|
187
187
|
exit 2
|
188
188
|
end
|
189
189
|
end
|
@@ -234,28 +234,36 @@ module Traject
|
|
234
234
|
end
|
235
235
|
|
236
236
|
|
237
|
-
def create_slop!
|
238
|
-
|
239
|
-
banner "traject [options] -c configuration.rb [-c config2.rb] file.mrc"
|
237
|
+
def create_slop!(argv)
|
238
|
+
options = Slop::Options.new do |o|
|
239
|
+
o.banner = "traject [options] -c configuration.rb [-c config2.rb] file.mrc"
|
240
240
|
|
241
|
-
on 'v', 'version', "print version information to stderr"
|
242
|
-
on 'd', 'debug', "Include debug log, -s log.level=debug"
|
243
|
-
on 'h', 'help', "print usage information to stderr"
|
244
|
-
|
245
|
-
|
246
|
-
|
247
|
-
|
248
|
-
|
249
|
-
|
250
|
-
|
251
|
-
|
252
|
-
|
241
|
+
o.on '-v', '--version', "print version information to stderr"
|
242
|
+
o.on '-d', '--debug', "Include debug log, -s log.level=debug"
|
243
|
+
o.on '-h', '--help', "print usage information to stderr"
|
244
|
+
o.array '-c', '--conf', 'configuration file path (repeatable)', :delimiter => nil
|
245
|
+
o.string "-i", '--indexer', "Traject indexer class name or shortcut", :default => "marc"
|
246
|
+
o.array "-s", "--setting", "settings: `-s key=value` (repeatable)", :delimiter => nil
|
247
|
+
o.string "-r", "--reader", "Set reader class, shortcut for -s reader_class_name="
|
248
|
+
o.string "-o", "--output_file", "output file for Writer classes that write to files"
|
249
|
+
o.string "-w", "--writer", "Set writer class, shortcut for -s writer_class_name="
|
250
|
+
o.string "-u", "--solr", "Set solr url, shortcut for -s solr.url="
|
251
|
+
o.string "-t", "--marc_type", "xml, json or binary. shortcut for -s marc_source.type="
|
252
|
+
o.array "-I", "--load_path", "append paths to ruby $LOAD_PATH", :delimiter => ":"
|
253
253
|
|
254
|
-
|
254
|
+
o.string "-x", "--command", "alternate traject command: process (default); marcout; commit", :default => "process"
|
255
255
|
|
256
|
-
on "stdin", "read input from stdin"
|
257
|
-
on "debug-mode", "debug logging, single threaded, output human readable hashes"
|
256
|
+
o.on "--stdin", "read input from stdin"
|
257
|
+
o.on "--debug-mode", "debug logging, single threaded, output human readable hashes"
|
258
258
|
end
|
259
|
+
|
260
|
+
options.parse(argv)
|
261
|
+
rescue Slop::Error => e
|
262
|
+
self.console.puts "Error: #{e.message}"
|
263
|
+
self.console.puts "Exiting..."
|
264
|
+
self.console.puts
|
265
|
+
self.console.puts options.to_s
|
266
|
+
exit 1
|
259
267
|
end
|
260
268
|
|
261
269
|
def initialize_indexer!
|
@@ -267,22 +275,5 @@ module Traject
|
|
267
275
|
|
268
276
|
return indexer
|
269
277
|
end
|
270
|
-
|
271
|
-
def parse_options(argv)
|
272
|
-
|
273
|
-
begin
|
274
|
-
self.slop.parse!(argv)
|
275
|
-
rescue Slop::Error => e
|
276
|
-
self.console.puts "Error: #{e.message}"
|
277
|
-
self.console.puts "Exiting..."
|
278
|
-
self.console.puts
|
279
|
-
self.console.puts slop.help
|
280
|
-
exit 1
|
281
|
-
end
|
282
|
-
|
283
|
-
return self.slop.to_hash
|
284
|
-
end
|
285
|
-
|
286
|
-
|
287
278
|
end
|
288
279
|
end
|
@@ -15,7 +15,7 @@ module Traject::Macros
|
|
15
15
|
# field/substring specification.
|
16
16
|
#
|
17
17
|
# First argument is a string spec suitable for the MarcExtractor, see
|
18
|
-
# MarcExtractor::
|
18
|
+
# Traject::MarcExtractor::Spec.
|
19
19
|
#
|
20
20
|
# Second arg is optional options, including options valid on MarcExtractor.new,
|
21
21
|
# and others. By default, will de-duplicate results, but see :allow_duplicates
|
@@ -2,9 +2,9 @@ require 'traject/marc_extractor_spec'
|
|
2
2
|
|
3
3
|
module Traject
|
4
4
|
# MarcExtractor is a class for extracting lists of strings from a MARC::Record,
|
5
|
-
# according to specifications. See
|
6
|
-
# string arguments used to specify extraction. See #initialize for
|
7
|
-
# that can be set controlling extraction.
|
5
|
+
# according to specifications. See Traject::MarcExtractor::Spec for description
|
6
|
+
# of string string arguments used to specify extraction. See #initialize for
|
7
|
+
# options that can be set controlling extraction.
|
8
8
|
#
|
9
9
|
# Examples:
|
10
10
|
#
|
@@ -41,10 +41,12 @@ require 'concurrent' # for atomic_fixnum
|
|
41
41
|
#
|
42
42
|
# ## Relevant settings
|
43
43
|
#
|
44
|
-
# * solr.url (optional if solr.update_url is set) The URL to the solr core to index into
|
44
|
+
# * solr.url (optional if solr.update_url is set) The URL to the solr core to index into.
|
45
|
+
# (Can include embedded HTTP basic auth as eg `http://user:pass@host/solr`)
|
45
46
|
#
|
46
47
|
# * solr.update_url: The actual update url. If unset, we'll first see if
|
47
|
-
# "#{solr.url}/update/json" exists, and if not use "#{solr.url}/update"
|
48
|
+
# "#{solr.url}/update/json" exists, and if not use "#{solr.url}/update". (Can include
|
49
|
+
# embedded HTTP basic auth as eg `http://user:pass@host/solr)
|
48
50
|
#
|
49
51
|
# * solr_writer.batch_size: How big a batch to send to solr. Default is 100.
|
50
52
|
# My tests indicate that this setting doesn't change overall index speed by a ton.
|
@@ -101,12 +103,17 @@ class Traject::SolrJsonWriter
|
|
101
103
|
def initialize(argSettings)
|
102
104
|
@settings = Traject::Indexer::Settings.new(argSettings)
|
103
105
|
|
106
|
+
|
104
107
|
# Set max errors
|
105
108
|
@max_skipped = (@settings['solr_writer.max_skipped'] || DEFAULT_MAX_SKIPPED).to_i
|
106
109
|
if @max_skipped < 0
|
107
110
|
@max_skipped = nil
|
108
111
|
end
|
109
112
|
|
113
|
+
|
114
|
+
# Figure out where to send updates, and if with basic auth
|
115
|
+
@solr_update_url, basic_auth_user, basic_auth_password = self.determine_solr_update_url
|
116
|
+
|
110
117
|
@http_client = if @settings["solr_json_writer.http_client"]
|
111
118
|
@settings["solr_json_writer.http_client"]
|
112
119
|
else
|
@@ -115,9 +122,8 @@ class Traject::SolrJsonWriter
|
|
115
122
|
client.connect_timeout = client.receive_timeout = client.send_timeout = @settings["solr_writer.http_timeout"]
|
116
123
|
end
|
117
124
|
|
118
|
-
if
|
119
|
-
|
120
|
-
client.set_auth(@settings["solr.url"], @settings["solr_writer.basic_auth_user"], @settings["solr_writer.basic_auth_password"])
|
125
|
+
if basic_auth_user || basic_auth_password
|
126
|
+
client.set_auth(@solr_update_url, basic_auth_user, basic_auth_password)
|
121
127
|
end
|
122
128
|
|
123
129
|
client
|
@@ -143,13 +149,11 @@ class Traject::SolrJsonWriter
|
|
143
149
|
# this the new default writer.
|
144
150
|
@commit_on_close = (settings["solr_writer.commit_on_close"] || settings["solrj_writer.commit_on_close"]).to_s == "true"
|
145
151
|
|
146
|
-
# Figure out where to send updates
|
147
|
-
@solr_update_url = self.determine_solr_update_url
|
148
152
|
|
149
153
|
@solr_update_args = settings["solr_writer.solr_update_args"]
|
150
154
|
@commit_solr_update_args = settings["solr_writer.commit_solr_update_args"]
|
151
155
|
|
152
|
-
logger.info(" #{self.class.name} writing to '#{@solr_update_url}' in batches of #{@batch_size} with #{@thread_pool_size} bg threads")
|
156
|
+
logger.info(" #{self.class.name} writing to '#{@solr_update_url}' #{"(with HTTP basic auth)" if basic_auth_user || basic_auth_password}in batches of #{@batch_size} with #{@thread_pool_size} bg threads")
|
153
157
|
end
|
154
158
|
|
155
159
|
|
@@ -368,13 +372,27 @@ class Traject::SolrJsonWriter
|
|
368
372
|
end
|
369
373
|
|
370
374
|
|
371
|
-
# Relatively complex logic to determine if we have a valid URL and what it is
|
375
|
+
# Relatively complex logic to determine if we have a valid URL and what it is,
|
376
|
+
# and if we have basic_auth info
|
377
|
+
#
|
378
|
+
# Empties out user and password embedded in URI returned, to help avoid logging it.
|
379
|
+
#
|
380
|
+
# @returns [update_url, basic_auth_user, basic_auth_password]
|
372
381
|
def determine_solr_update_url
|
373
|
-
if settings['solr.update_url']
|
382
|
+
url = if settings['solr.update_url']
|
374
383
|
check_solr_update_url(settings['solr.update_url'])
|
375
384
|
else
|
376
385
|
derive_solr_update_url_from_solr_url(settings['solr.url'])
|
377
386
|
end
|
387
|
+
|
388
|
+
parsed_uri = URI.parse(url)
|
389
|
+
user_from_uri, password_from_uri = parsed_uri.user, parsed_uri.password
|
390
|
+
parsed_uri.user, parsed_uri.password = nil, nil
|
391
|
+
|
392
|
+
basic_auth_user = @settings["solr_writer.basic_auth_user"] || user_from_uri
|
393
|
+
basic_auth_password = @settings["solr_writer.basic_auth_password"] || password_from_uri
|
394
|
+
|
395
|
+
return [parsed_uri.to_s, basic_auth_user, basic_auth_password]
|
378
396
|
end
|
379
397
|
|
380
398
|
|
data/lib/traject/version.rb
CHANGED
@@ -0,0 +1,51 @@
|
|
1
|
+
# we mostly unit test with a Traject::Indexer itself and lower-level, but
|
2
|
+
# we need at least some basic top-level integration actually command line tests,
|
3
|
+
# this is a start, we can add more.
|
4
|
+
#
|
5
|
+
# Should we be testing Traject::CommandLine as an object instead of/in addition to
|
6
|
+
# actually testing shell-out to command line call? Maybe.
|
7
|
+
|
8
|
+
require 'test_helper'
|
9
|
+
|
10
|
+
describe "Shell out to command line" do
|
11
|
+
# just encapsuluate using the minitest capture helper, but also
|
12
|
+
# getting and returning exit code
|
13
|
+
#
|
14
|
+
# out, err, result = execute_with_args("-c configuration")
|
15
|
+
def execute_with_args(args)
|
16
|
+
out, err = capture_subprocess_io do
|
17
|
+
system("./bin/traject #{args}")
|
18
|
+
end
|
19
|
+
|
20
|
+
return out, err, $?
|
21
|
+
end
|
22
|
+
|
23
|
+
it "can display version" do
|
24
|
+
out, err, result = execute_with_args("-v")
|
25
|
+
assert_equal err, "traject version #{Traject::VERSION}\n"
|
26
|
+
assert result.success?
|
27
|
+
end
|
28
|
+
|
29
|
+
it "can display help text" do
|
30
|
+
out, err, result = execute_with_args("-h")
|
31
|
+
|
32
|
+
assert err.start_with?("traject [options] -c configuration.rb [-c config2.rb] file.mrc")
|
33
|
+
assert result.success?
|
34
|
+
end
|
35
|
+
|
36
|
+
it "handles bad argument" do
|
37
|
+
out, err, result = execute_with_args("--no-such-arg")
|
38
|
+
refute result.success?
|
39
|
+
|
40
|
+
assert err.start_with?("Error: unknown option `--no-such-arg'\nExiting...\n")
|
41
|
+
end
|
42
|
+
|
43
|
+
it "does basic dry run" do
|
44
|
+
out, err, result = execute_with_args("--debug-mode -s one=two -s three=four -c test/test_support/demo_config.rb test/test_support/emptyish_record.marc")
|
45
|
+
|
46
|
+
assert result.success?
|
47
|
+
assert_includes err, "executing with: `--debug-mode -s one=two -s three=four"
|
48
|
+
assert_match /bib_1000165 +author_sort +Collection la/, out
|
49
|
+
end
|
50
|
+
end
|
51
|
+
|
@@ -178,17 +178,52 @@ describe "Traject::SolrJsonWriter" do
|
|
178
178
|
assert(auth.empty?)
|
179
179
|
end
|
180
180
|
|
181
|
-
|
182
|
-
settings = {
|
183
|
-
"solr.url" => "http://example.com/solr/foo",
|
184
|
-
"solr_writer.basic_auth_user" => "foo",
|
185
|
-
"solr_writer.basic_auth_password" => "bar",
|
186
|
-
}
|
181
|
+
describe "HTTP basic auth" do
|
187
182
|
|
188
|
-
|
189
|
-
|
190
|
-
|
191
|
-
|
183
|
+
it "supports basic authentication settings" do
|
184
|
+
settings = {
|
185
|
+
"solr.url" => "http://example.com/solr/foo",
|
186
|
+
"solr_writer.basic_auth_user" => "foo",
|
187
|
+
"solr_writer.basic_auth_password" => "bar",
|
188
|
+
}
|
189
|
+
|
190
|
+
# testing with some internal implementation of HTTPClient sorry
|
191
|
+
|
192
|
+
writer = Traject::SolrJsonWriter.new(settings)
|
193
|
+
|
194
|
+
auth = writer.instance_variable_get("@http_client")
|
195
|
+
.www_auth.basic_auth.instance_variable_get("@auth")
|
196
|
+
assert(!auth.empty?)
|
197
|
+
assert_equal(auth.values.first, Base64.encode64("foo:bar").chomp)
|
198
|
+
end
|
199
|
+
|
200
|
+
it "supports basic auth from solr.url" do
|
201
|
+
settings = {
|
202
|
+
"solr.url" => "http://foo:bar@example.com/solr/foo",
|
203
|
+
}
|
204
|
+
|
205
|
+
# testing with some internal implementation of HTTPClient sorry
|
206
|
+
|
207
|
+
writer = Traject::SolrJsonWriter.new(settings)
|
208
|
+
auth = writer.instance_variable_get("@http_client")
|
209
|
+
.www_auth.basic_auth.instance_variable_get("@auth")
|
210
|
+
assert(!auth.empty?)
|
211
|
+
assert_equal(auth.values.first, Base64.encode64("foo:bar").chomp)
|
212
|
+
end
|
213
|
+
|
214
|
+
it "does not log basic auth from solr.url" do
|
215
|
+
string_io = StringIO.new
|
216
|
+
settings = {
|
217
|
+
"solr.url" => "http://secret_username:secret_password@example.com/solr/foo",
|
218
|
+
"logger" => Logger.new(string_io)
|
219
|
+
}
|
220
|
+
|
221
|
+
|
222
|
+
writer = Traject::SolrJsonWriter.new(settings)
|
223
|
+
|
224
|
+
refute_includes string_io.string, "secret_username:secret_password"
|
225
|
+
assert_includes string_io.string, "(with HTTP basic auth)"
|
226
|
+
end
|
192
227
|
end
|
193
228
|
|
194
229
|
describe "commit" do
|
data/traject.gemspec
CHANGED
@@ -25,7 +25,7 @@ Gem::Specification.new do |spec|
|
|
25
25
|
spec.add_dependency "marc", "~> 1.0"
|
26
26
|
|
27
27
|
spec.add_dependency "hashie", ">= 3.1", "< 5" # used for Indexer#settings
|
28
|
-
spec.add_dependency "slop", "
|
28
|
+
spec.add_dependency "slop", "~> 4.0" # command line parsing
|
29
29
|
spec.add_dependency "yell" # logging
|
30
30
|
spec.add_dependency "dot-properties", ">= 0.1.1" # reading java style .properties
|
31
31
|
spec.add_dependency "httpclient", "~> 2.5"
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: traject
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 3.
|
4
|
+
version: 3.5.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Jonathan Rochkind
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2020-
|
12
|
+
date: 2020-12-14 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: concurrent-ruby
|
@@ -63,20 +63,14 @@ dependencies:
|
|
63
63
|
name: slop
|
64
64
|
requirement: !ruby/object:Gem::Requirement
|
65
65
|
requirements:
|
66
|
-
- - "
|
67
|
-
- !ruby/object:Gem::Version
|
68
|
-
version: 3.4.5
|
69
|
-
- - "<"
|
66
|
+
- - "~>"
|
70
67
|
- !ruby/object:Gem::Version
|
71
68
|
version: '4.0'
|
72
69
|
type: :runtime
|
73
70
|
prerelease: false
|
74
71
|
version_requirements: !ruby/object:Gem::Requirement
|
75
72
|
requirements:
|
76
|
-
- - "
|
77
|
-
- !ruby/object:Gem::Version
|
78
|
-
version: 3.4.5
|
79
|
-
- - "<"
|
73
|
+
- - "~>"
|
80
74
|
- !ruby/object:Gem::Version
|
81
75
|
version: '4.0'
|
82
76
|
- !ruby/object:Gem::Dependency
|
@@ -246,8 +240,8 @@ extra_rdoc_files:
|
|
246
240
|
- doc/settings.md
|
247
241
|
- doc/xml.md
|
248
242
|
files:
|
243
|
+
- ".github/workflows/ruby.yml"
|
249
244
|
- ".gitignore"
|
250
|
-
- ".travis.yml"
|
251
245
|
- ".yardopts"
|
252
246
|
- CHANGES.md
|
253
247
|
- Gemfile
|
@@ -307,6 +301,7 @@ files:
|
|
307
301
|
- lib/translation_maps/marc_geographic.yaml
|
308
302
|
- lib/translation_maps/marc_instruments.yaml
|
309
303
|
- lib/translation_maps/marc_languages.yaml
|
304
|
+
- test/command_line_test.rb
|
310
305
|
- test/debug_writer_test.rb
|
311
306
|
- test/delimited_writer_test.rb
|
312
307
|
- test/experimental_nokogiri_streaming_reader_test.rb
|
@@ -411,6 +406,7 @@ specification_version: 4
|
|
411
406
|
summary: An easy to use, high-performance, flexible and extensible metadata transformation
|
412
407
|
system, focused on library-archives-museums input, and indexing to Solr as output.
|
413
408
|
test_files:
|
409
|
+
- test/command_line_test.rb
|
414
410
|
- test/debug_writer_test.rb
|
415
411
|
- test/delimited_writer_test.rb
|
416
412
|
- test/experimental_nokogiri_streaming_reader_test.rb
|
data/.travis.yml
DELETED
@@ -1,17 +0,0 @@
|
|
1
|
-
language: ruby
|
2
|
-
cache: bundler
|
3
|
-
# we don't really need `sudo: true`, but for some reason travis docker-based systems are unreliable
|
4
|
-
# at downloading jruby, and
|
5
|
-
sudo: true
|
6
|
-
rvm:
|
7
|
-
- 2.4.4
|
8
|
-
- 2.5.1
|
9
|
-
- 2.6.1
|
10
|
-
- 2.7.0
|
11
|
-
# avoid having travis install jdk on MRI builds where we don't need it.
|
12
|
-
matrix:
|
13
|
-
include:
|
14
|
-
- jdk: openjdk8
|
15
|
-
rvm: jruby-9.1.17.0
|
16
|
-
- jdk: openjdk8
|
17
|
-
rvm: jruby-9.2.6.0
|