sip2-ruby 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 35f09dc0ff1cc1cc6d033faf06eb5ec6108d37c4f9cd89d3e63ecf66614292ab
+  data.tar.gz: 6e21ab95f53d364cc215eeb66db6f540d0315cc203a355476d172d13bb7aaccd
+SHA512:
+  metadata.gz: f695891e29b8ada0c7ec5fa68b606931fa5f0f4b866bd0e4cf02c1914dbe5a29013313384af73f76eae04f9b18d4078df03ddfbb9e558f48124a3c22b9edcc5b
+  data.tar.gz: 7ca77c9993bef861c547ef2405be6da9d0383cd3065be6464b582acfc50b9a13709e6227e288474e696c6481e3adddaf312b7bc814a15ce1328d6624fa76efa4

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2023 University of Borås
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,136 @@
+# Sip2 (sip2-ruby)
+
+Sip2 is a library for parsing and encoding messages in the [Sip2 standard][sip2]
+used for communication between library automation devices and automated
+circulation systems. It is also two executables, `sip2-to-json` and
+`json-to-sip2`, for parsing and encoding respectively.
+
+[sip2]: https://developers.exlibrisgroup.com/wp-content/uploads/2020/01/3M-Standard-Interchange-Protocol-Version-2.00.pdf
+
+Sip2 messages are parsed into a Hash/JSON representation, and the
+Hash/JSON representation can be converted back into Sip2 format.
+
+The library is only concerned with parsing and encoding and does not provide
+anything related to the connection between the devices and the library system.
+
+The parser is built on the excellent Parslet library.
+
+## Basic usage
+
+### As a script
+
+Given that `sip2.log` contains only Sip2 messages, separated by `\r` (CR) and
+optionally `\n` (LF).
+
+```
+sip2-to-json < sip2.log > sip2-log.json
+
+json-to-sip2 < sip2-log.json > sip2.txt
+```
+
+Normally `sip2.log` and `sip2.txt` should now be semantically identical. (Field
+order might differ.)
+
+Typically `sip2-to-json` can be used to view, query and filter logs, while
+`json-to-sip2` is best used in combination with `sip2-to-json` to rewrite
+messages, operating on the JSON representation rather than on the raw strings:
+
+```
+sip2-to-json < sip2.log | your-script-here | json-to-sip2
+```
+
+### As a library
+
+The main interfaces are two methods on the `Sip2` module: `Sip2.parse` and
+`Sip2.encode`.
+
+```ruby
+require 'sip2'
+message = "9900802.00\r"
+
+parsed_message = Sip2.parse(message).first
+#=> {:message_code=>"99", :message_name=>"SC Status", :status_code=>0,
+#   :max_print_width=>80, :protocol_version=>"2.00"}
+
+parsed_message[:max_print_width] = 100
+
+Sip2.encode(parsed_message)
+#=> "9901002.00\r"
+```
+
+## Validation
+
+`Sip2.parse` only accepts well formed messages, but handles invalid dates in a
+possibly unexpected way. It enforces *date-like* values, but would accept e.g.
+"2023-02-30" *and convert it* to 2023-03-02. Messages that are not well formed
+will cause a `Parslet::ParseFailed` error to be raised. See [Parslet
+documentation][parslet-docs] for how to get more information from the error.
+
+[parslet-docs]: https://kschiess.github.io/parslet/documentation.html
+
+`Sip2.encode` enforces the type of all values and outputs valid Sip2
+messages. On most type errors, a `Dry::Struct::Error` will be raised.
+
+Unrecognized message types and unrecognized fields for recognized messages are
+accepted, in accordance with the Sip2 standard. The data is captured and can
+round trip between parsing and encoding.
+
+## Error detection and checksums
+
+The Sip2 standard allows for some basic error detection using checksums. This
+library does not validate checksums, but can create and update them.
+When integrating with a system that validates and enforces checksums the way
+checksums are calculated must match.
+
+The standard is a bit vague on the topic of encodings, and especially for
+checksums, where it is important since characters are "binary summed".
+`Sip2#encode` accepts an optional `checksum_encoder`, a simple lambda that
+accepts a string and returns a string. If supplied, the checksum will be
+calculated on the return value of this lambda applied to the message. (The
+lambda should not alter the original message.) In `Sip2::ChecksumEncoder` a
+couple of encoders are provided (accessible both through their constant names
+and through `#[]`):
+
+    * `ALMA`: Encodes the message as ASCII, using replacement characters for
+    non-ascii values. Matches the (undocumented) behaviour of Ex Libris Alma.
+
+    * `IDENTITY`: Pass through of the original message. Thus, a message in UTF-8
+    will be checksummed as UTF-8, while a message in Latin-1 will be checksummed
+    as Latin1. (This is the same as not specifying a `checksum_encoder`.)
+
+The pre defined encoders are also available for `json-to-sip2` through the
+command line option `--checksum-encoder`.
+
+## Caveats
+
+### Documentation
+
+*Note that the ruby code is currently entirely undocumented.* This is due to
+much of it being dynamically generated from a code representation of the Sip2
+standard, and I simply do not know how to let rdoc or yard pick this up. The
+parts that are static can of course be documented, but as they read or return
+the dynamically generated message classes the documentation would still be
+incomplete.
+
+Running `script/sip2-messages-info` will output a JSON representation of all
+known messages and the fields they include. There is also a JSON representation
+of all fields in `doc/sip2_fields.json`. These two sources can be used to some
+extent to see how the mapping between the Sip2 format and the Hash/JSON
+representation looks like.
+
+### Time Zones
+
+Time zones are not fully implemented and do not round trip. The Sip2
+specification declares that time zones should be expressed according to ancient
+ANSI standard X3.43, which in its turn refers to X3.51. This standard is
+mentioned in RFC822, which specifies a few two or three letter codes, "military"
+one letter time zone codes, and explicit offset using four digits prefixed with
+`+` or `-`. The prefixed digits will not fit in the designated four character
+segment.
+
+In this library only `Z` for "UTC" and `    ` (four spaces, no time zone) are
+round tripped. When parsing, `Z` and `UTC` will be parsed as times with UTC time
+zone, four blanks will be parsed as local time, and military one letter time
+zone codes will be parsed into times with the correct offset but with no
+`Time#zone` set. When encoding, times in UTC will be encoded as `Z`, and all
+other times, including local time, will be encoded without time zone data.

data/Rakefile

@@ -0,0 +1,10 @@
+require 'bundler/gem_tasks'
+require 'rake/testtask'
+
+task default: "test"
+
+Rake::TestTask.new do |t|
+   t.pattern = 'test/**/*_spec.rb'
+   t.libs << "test"
+   t.libs << "lib"
+end

data/bin/json-to-sip2

@@ -0,0 +1,109 @@
+#!/usr/bin/env ruby
+
+require_relative '../environment'
+
+require 'yajl'
+require 'optparse'
+require 'sip2'
+require 'sip2/checksum_encoder'
+
+options = {
+  checksum_encoder: nil,
+  continue_on_error: false,
+  debug: false,
+  error_detection: false,
+  line_by_line: true,
+  strip_extra_fields: false,
+}
+OptionParser.new do |opts|
+  opts.banner = "Usage: #{$0} [options]"
+  opts.separator ""
+  opts.separator "Options:"
+
+  opts.on("-a", "--array", "Read messages from a JSON array.") do
+    options[:line_by_line] = false
+  end
+
+  opts.on("-c", "--continue-on-error",
+          "When parsing an object fails, continue with the next.",
+          "The exit code will still be non-zero.",
+          'This option does not apply when using `--array`.'
+         ) do
+    options[:continue_on_error] = true
+  end
+
+  opts.on("-C", "--checksum-encoder=<NAME>",
+          "Reencode the message with Sip2::ChecksumEncoder::<NAME>",
+          "before calculating the checksum, where <NAME> is one of:",
+          *Sip2::ChecksumEncoder.constants.map { |c| sprintf("    %s", c) }
+         ) do |name|
+           options[:checksum_encoder] = name
+  end
+
+  opts.on("-E", "--error-detection",
+          "Append error detection fields to the message",
+         ) do |name|
+           options[:error_detection] = true
+  end
+
+  opts.on("--strip-extra-fields",
+          "Remove extra fields that are not part of the specification.",
+         ) do
+           options[:strip_extra_fields] = true
+         end
+
+  opts.on("-v", "--version", "Print version and exit.") do
+    puts Sip2::VERSION
+    exit
+  end
+
+  opts.on("-h", "--help", "Display this message.") do
+    puts opts
+    exit
+  end
+
+end.parse!
+
+UnknownError = Class.new(StandardError)
+
+checksum_encoder_name = options[:checksum_encoder] || ENV['SIP2_CHECKSUM_ENCODER']
+checksum_encoder = Sip2::ChecksumEncoder[checksum_encoder_name] if checksum_encoder_name
+
+encode = ->(obj) {
+  Sip2.encode(
+    obj,
+    add_error_detection: options[:error_detection],
+    checksum_encoder: checksum_encoder,
+    strip_extra_fields: options[:strip_extra_fields]
+  )
+}
+
+if options[:line_by_line]
+  error_count = 0
+  Yajl.load($stdin) do |obj|
+    begin
+      puts encode.call(obj)
+    rescue Dry::Struct::Error => error
+      warn "Invalid object: #{Yajl.dump(obj)}"
+      if options[:continue_on_error]
+        warn error.message
+        error_count += 1
+      else
+        raise error
+      end
+    end
+  end
+  if error_count > 0
+    warn sprintf(
+      "WARNING: %d object%s could not be parsed",
+      error_count,
+      error_count == 1 ? "" : "s"
+    )
+    exit 1
+  end
+else
+  data = Yajl.load($stdin)
+  data.each do |obj|
+    puts encode.call(obj)
+  end
+end

data/bin/sip2-to-json

@@ -0,0 +1,81 @@
+#!/usr/bin/env ruby
+
+require_relative '../environment'
+
+require 'json'
+require 'optparse'
+require 'sip2'
+
+options = {
+  continue_on_error: false,
+  debug: false,
+  line_by_line: true,
+}
+OptionParser.new do |opts|
+  opts.banner = "Usage: #{$0} [options]"
+  opts.separator ""
+  opts.separator "Options:"
+
+  opts.on("-a", "--all", "Parse all messages into a JSON array.") do
+    options[:line_by_line] = false
+  end
+
+  opts.on("-c", "--continue-on-error",
+          "When parsing a line fails, continue with the next.",
+          "The exit code will still be non-zero.",
+          'This option does not apply when using `--all`.'
+         ) do
+    options[:continue_on_error] = true
+  end
+
+  opts.on("-D", "--debug", "Report errors with a parse tree.") do
+    options[:debug] = true
+  end
+
+  opts.on("-v", "--version", "Print version and exit.") do
+    puts Sip2::VERSION
+    exit
+  end
+
+  opts.on("-h", "--help", "Display this message.") do
+    puts opts
+    exit
+  end
+
+end.parse!
+
+if options[:line_by_line]
+  begin
+    error_count = 0
+    ARGF.each_line.with_index(1) do |data,i|
+      begin
+        puts Sip2.parse(data).first.to_json
+      rescue Parslet::ParseFailed => error
+        warn "Error parsing line #{i}: #{data.dump}"
+        warn error.parse_failure_cause.ascii_tree if options[:debug]
+        if options[:continue_on_error]
+          error_count += 1
+        else
+          raise error
+        end
+      end
+    end
+    if error_count > 0
+      warn sprintf(
+        "WARNING: %d line%s could not be parsed",
+        error_count,
+        error_count == 1 ? "" : "s"
+      )
+      exit 1
+    end
+  rescue Errno::EPIPE
+    exit 0
+  end
+else
+  begin
+    puts Sip2.parse(ARGF.read).to_json
+  rescue Parslet::ParseFailed => error
+    warn error.parse_failure_cause.ascii_tree if options[:debug]
+    raise error
+  end
+end

data/lib/sip2/checksum_encoder.rb

@@ -0,0 +1,11 @@
+module Sip2
+  module ChecksumEncoder
+
+    def self.[](name)
+      const_get(name.to_s.upcase)
+    end
+
+    ALMA = ->(str) { str.encode(Encoding::ASCII, undef: :replace) }
+    IDENTITY = ->(x) { x }
+  end
+end