stupidedi 1.1.0 → 1.2.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 1dc73ca111e43b38ae8715ff61a968a98a11fb37
+  data.tar.gz: f65b0a26440dc252b2dc63d47eb986650009d1a0
+SHA512:
+  metadata.gz: ae4fdaee26ab0872ad5b998b108690850eb709754a22d89cca2104d137d31c0de7beb8b599e03deff9c92fe8a697c069499a3203a27b8a53c484badee183405e
+  data.tar.gz: 236d121428f8a7bc1f11a8d426e5335917b5d44cae3cb688a00c98f0e7f04e5e27f587608410656f89940137c9730a850ae38734dcf9323baf9047be534f16b8

data/README.md

@@ -1,9 +1,12 @@
 # Stupidedi
+[![Build Status](https://secure.travis-ci.org/kputnam/stupidedi.png?branch=master)](http://travis-ci.org/kputnam/stupidedi) [![Dependency Status](https://gemnasium.com/irobayna/stupidedi.svg)](https://gemnasium.com/irobayna/stupidedi) [![GitHub version](https://badge.fury.io/gh/kputnam%2Fstupidedi.svg)](http://badge.fury.io/gh/kputnam%2Fstupidedi) [![Code Climate](https://codeclimate.com/github/kputnam/stupidedi.png)](https://codeclimate.com/github/kputnam/stupidedi)
 
 ![Screenshot](https://raw.github.com/kputnam/stupidedi/master/doc/images/edi-pp.png)
 
+
 * [GitHub project](http://github.com/kputnam/stupidedi)
-* [Documentation](http://rubydoc.info/github/kputnam/stupidedi/master/frames)
+* [Human Documentation](https://github.com/kputnam/stupidedi/tree/master/doc)
+* [API Documentation](http://rubydoc.info/github/kputnam/stupidedi/master/frames)
 
 Stupidedi is a high-quality library for parsing, generating, validating,
 and manipulating ASC X12 EDI documents. Very roughly, it's jQuery for
@@ -26,7 +29,7 @@ and require flexibility to parse them.
 Stupidedi enables you to encode these transaction set specifications
 directly in Ruby. From these specifications, it will generate a parser
 to read incoming messages and a DSL to generate outgoing messages. This
-approach has a huge advantage of writing a parser from scratch, which
+approach has a huge advantage over writing a parser from scratch, which
 can be error-prone and difficult to change.
 
 Significant thought was put into the design of the library. Some of
@@ -50,22 +53,105 @@ specification, an exception is thrown with a meaningful stack trace.
 Other libraries only perform validation after the entire document has
 been generated, while some don't perform validation at all.
 
+Stupidedi performs extensive validation and ensures well-formedness.
+See the human readable documentation in doc/Generating.md for more
+details.
+
 ### Encourages readable client code
 
+Unlike other libraries, generating documents doesn't involve naming
+obscure identifiers from the specification (like C001, DE522 or LOOP2000),
+for elements of the grammar that don't actually appear in the output.
+
+Like HAML or Builder::XmlMarkup, the DSL for generating documents closely
+matches terminology from the problem domain. You can see in the example
+below that code looks very similar to an EDI document. This makes it easy
+to understand, assuming a reasonable familiarity with EDI.
+
 ### Efficient parsing and traversing
 
+The parser is designed using immutable data structures, making it thread-safe
+for runtimes that can utilize multiple cores. While in certain cases,
+immutability places higher demand on garbage collection, this has been
+mitigated with careful optimization. Input can be streamed incrementally, so
+very large files aren't read into memory all at once.
+
+![Benchmark](https://raw.github.com/kputnam/stupidedi/master/notes/benchmark/throughput.png)
+
+<table>
+  <tr>
+    <th>segments</th>
+    <th>1.9.3</th>
+    <th>1.9.2</th>
+    <th>rbx-head</th>
+    <th>jruby-1.6.6</th>
+  </tr>
+  <tr>
+    <td>1680</td>
+    <td>2107.90</td>
+    <td>2007.17</td>
+    <td>503.14</td>
+    <td>317.52</td>
+  </tr>
+  <tr>
+    <td>3360</td>
+    <td>2461.54</td>
+    <td>2420.75</td>
+    <td>731.71</td>
+    <td>477.07</td>
+  </tr>
+  <tr>
+    <td>6720</td>
+    <td>2677.29</td>
+    <td>2620.90</td>
+    <td>950.63</td>
+    <td>685.15</td>
+  </tr>
+  <tr>
+    <td>13440</td>
+    <td>2699.88</td>
+    <td>2663.50</td>
+    <td>1071.00</td>
+    <td>897.50</td>
+  </tr>
+  <tr>
+    <td>26880</td>
+    <td>2558.54</td>
+    <td>2510.51</td>
+    <td>1124.50</td>
+    <td>1112.67</td>
+  </tr>
+  <tr>
+    <td>53760</td>
+    <td>2254.94</td>
+    <td>2164.16</td>
+    <td>1039.81</td>
+    <td>1292.62</td>
+  </tr>
+</table>
+
+These benchmarks aren't scientific by any means. They were performed on a
+MacBook Pro, 2.2GHz Core i7 with 8GB RAM by reading the X222-HC837 fixture data
+files, in serial. The results show the parser runtime is O(n), linear in the
+size of the input, but the drop in throughput at 13K+ segments is likely due to
+memory allocation. The steady increase in throughput on JRuby and Rubinus is
+probably attributable optimizations performed by the JIT compiler.
+
+Lastly, these results should approximate the performance of document generation
+with BuilderDSL, except BuilderDSL API should have less overhead, as it skips
+the tokenizer. On the other hand, BuilderDSL frequently queries the call stack
+to track provenance of elements in the parse tree. In common real-world use,
+custom application logic and database access are going to bottleneck performance,
+rather than Stupidedi.
+
 ### Helps developers gain familiarity
 
 ## Why not a commercial translator?
 
-Because enterprise software is garbage. The costs include not only
-licensing, but support, maintenance, and training.
-
-Most importantly, commercial EDI translators solve a different set
-of problems. Many focus on translating between EDI and another data
-format, like XML, CSV, or a relational database. This isn't particularly
-productive, as you still have to unserialize the data to do
-anything with it.
+Commercial EDI translators solve a different set of problems. Many focus
+on translating between EDI and another data format, like XML, CSV, or a
+relational database. This isn't particularly productive, as you still have
+to unserialize the data to do anything with it.
 
 ## What doesn't it solve?
 
@@ -76,24 +162,35 @@ commercial EDI translators have, so it...
 * Doesn't transmit or receive files
 * Doesn't do encryption
 * Doesn't connect to your database
+* Doesn't transmit over a dial-up modem
 * Doesn't queue messages for delivery or receipt
 * Doesn't generate acknowledgements
 * Doesn't have a graphical interface
 
-These can be accomplished using other libraries or your own code.
+These features are orthogonal to the problem Stupidedi aims to solve, but they
+can certainly be implemented with other code taking advantage of Stupidedi.
 
 ## Alternative libraries
 
 Stupidedi is an opinionated library, and maybe you don't agree with
 it. Here are a few alternative libraries:
 
+* https://github.com/rjackson/X12
+* https://github.com/dlabare/ruby-edi
+* https://github.com/pstuteville/x12
 * http://www.appdesign.com/x12parser/
+  * https://github.com/swalberg/x12
+  * https://github.com/mjpete3/x12
+* https://github.com/mrcsparker/x12_parser
 * http://edi4r.rubyforge.org/edi4r/
 * http://edival.sourceforge.net/
 * http://www.edidev.com/
 
 ## Examples
 
+In addition to these brief examples, see sample code in the `notes` directory
+and the human-readable Markdown documentation in `doc`.
+
 ### Utilities
 
 Pretty print the syntax tree
@@ -131,85 +228,115 @@ Perform validation on a file
 
 ### Generating, Writing
 
-    require "stupidedi"
-
-    b = Stupidedi::Builder::BuilderDsl.build(Stupidedi::Config.default)
-
-    b.ISA("00", "", "00", "",
-          "ZZ", "SUBMITTER ID",
-          "ZZ", "RECEIVER ID",
-          "990531", "1230", nil, "00501", "123456789", "1", "T", nil)
-    b. GS("HC", "SENDER ID", "RECEIVER ID", "19990531", "1230", "1", "X", "005010X222")
-    b. ST("837", "1234", b.default)
-    b.BHT("0019", "00", "X"*30, "19990531", Time.now.utc, "CH")
-    b.NM1(b.default, "1", "PREMIER BILLING SERVICE", "", "", "", "", "46", "12EEER000TY")
-    b.PER("IC", "JERRY THE CLOWN", "TE", "3056660000")
-    b.NM1("40", "2", "REPRICER JONES", "", "", "", "", "46", "66783JJT")
-    b. HL("1", "", "20", "1")
-    b.NM1("85", "2", "PREMIER BILLING SERVICE", "", "", "", "", "XX", "123234560")
-    b. N3("1234 SEAWAY ST")
-    b. N4("MIAMI", "FL", "331111234")
-    b.REF("EI", "123667894")
-    b.PER("IC", b.blank, "TE", "3056661111")
-    b.NM1("87", "2")
-    b. N3("2345 OCEAN BLVD")
-    b. N4("MIAMI", "FL", "33111")
-    b. HL("2", "1", "22", "0")
-    b.SBR("S", "18", "", "", "12", "", "", "", "MB")
-    b.NM1("IL", "1", "BACON", "KEVIN", "", "", "", "MI", "222334444")
-    b. N3("236 N MAIN ST")
-    b. N4("MIAMI", "FL", "33413")
-    b.DMG("D8", "19431022", "F")
-
-    b.machine.zipper.tap do |z|
-      separators =
-        Stupidedi::Reader::Separators.build(:segment    => "~\n",
-                                            :element    => "*",
-                                            :component  => ":",
-                                            :repetition => "^")
-
-      w = Stupidedi::Writer::Default.new(z.root, separators)
-      w.write($stdout)
-    end
+```ruby
+require "stupidedi"
+
+# You can customize this to delegate to your own grammar definitions, if needed.
+config = Stupidedi::Config.default
+
+b = Stupidedi::Builder::BuilderDsl.build(config)
+
+# These methods perform error checking: number of elements, element types, min/max
+# length requirements, conditionally required elements, valid segments, number of
+# segment occurrences, number of loop occurrences, etc.
+b.ISA "00", nil, "00", nil,
+      "ZZ", "SUBMITTER ID",
+      "ZZ", "RECEIVER ID",
+      "990531", "1230", nil, "00501", "123456789", "1", "T", nil
+
+# The API tracks the current position in the specification (e.g., the current loop,
+# table, etc) to ensure well-formedness as each segment is generated.
+b.GS "HC", "SENDER ID", "RECEIVER ID", "19990531", "1230", "1", "X", "005010X222"
+
+# The `b.default` value can be used to generate the appropriate value if it can
+# be unambigously inferred from the grammar.
+b.ST "837", "1234", b.default
+  # You can use string representations of data or standard Ruby data types, like Time.
+  b.BHT "0019", "00", "X"*30, "19990531", Time.now.utc, "CH"
+  b.NM1 b.default, "1", "PREMIER BILLING SERVICE", nil, nil, nil, nil, "46", "12EEER000TY"
+  b.PER "IC", "JERRY THE CLOWN", "TE", "3056660000"
+
+  b.NM1 "40", "2", "REPRICER JONES", nil, nil, nil, nil, "46", "66783JJT"
+    b.HL "1", nil, "20", "1"
+
+  b.NM1 "85", "2", "PREMIER BILLING SERVICE", nil, nil, nil, nil, "XX", "123234560"
+    b.N3  "1234 SEAWAY ST"
+    b.N4  "MIAMI", "FL", "331111234"
+    b.REF "EI", "123667894"
+    b.PER "IC", b.blank, "TE", "3056661111"
+
+  b.NM1 "87", "2"
+    b.N3 "2345 OCEAN BLVD"
+    b.N4 "MIAMI", "FL", "33111"
+
+b.HL "2", "1", "22", "0"
+  b.SBR "S", "18", nil, nil, "12", nil, nil, nil, "MB"
+
+  b.NM1 "IL", "1", "BACON", "KEVIN", nil, nil, nil, "MI", "222334444"
+    b.N3  "236 N MAIN ST"
+    b.N4  "MIAMI", "FL", "33413"
+    b.DMG "D8", "19431022", "F"
+
+b.machine.zipper.tap do |z|
+  # The :component, and :repetition parameters can also be specified as elements
+  # of the ISA segment, at `b.ISA(...)` above. When generating a document from
+  # scratch, :segment and :element must be specified -- if you've parsed the doc
+  # from a file, these params will default to whatever was used in the file, or
+  # you can override them here.
+  separators =
+    Stupidedi::Reader::Separators.build :segment    => "~\n",
+                                        :element    => "*",
+                                        :component  => ":",
+                                        :repetition => "^"
+
+  # You can also serialize any subtree within the document (e.g., everything inside
+  # some ST..SE transaction set, or a single loop. Here, z.root is the entire tree.
+  w = Stupidedi::Writer::Default.new(z.root, separators)
+  w.write($stdout)
+end
+```
 
 ### Reading, Traversing
 
-    require "stupidedi"
-
-    config = Stupidedi::Config.default
-    parser = Stupidedi::Builder::StateMachine.build(config)
-
-    input  = if RUBY_VERSION > "1.8"
-               File.open("spec/fixtures/X221-HP835/1-good.txt", :encoding => "ISO-8859-1")
-             else
-               File.open("spec/fixtures/X221-HP835/1-good.txt")
-             end
-
-    # Reader.build accepts IO (File), String, and DelegateInput
-    parser, result = parser.read(Stupidedi::Reader.build(input))
-
-    # Report fatal tokenizer failures
-    if result.fatal?
-      result.explain{|reason| raise reason + " at #{result.position.inspect}" }
-    end
-
-    def el(m, *ns, &block)
-      if Stupidedi::Either === m
-        m.tap{|m| el(m, *ns, &block) }
-      else
-        yield(*ns.map{|n| m.element(*n).map(&:node).map(&:value).fetch(nil) })
-      end
-    end
-
-    # Print some information
-    parser.first
-      .flatmap{|m| m.find(:GS) }
-      .flatmap{|m| m.find(:ST) }
-      .tap do |m|
-        el(m.find(:N1, "PR"), 2){|e| puts "Payer: #{e}" }
-        el(m.find(:N1, "PE"), 2){|e| puts "Payee: #{e}" }
-      end
-      .flatmap{|m| m.find(:LX) }
-      .flatmap{|m| m.find(:CLP) }
-      .flatmap{|m| m.find(:NM1, "QC") }
-      .tap{|m| el(m, 3, 4){|l,f| puts "Patient: #{l}, #{f}" }}
+```ruby
+require "stupidedi"
+
+config = Stupidedi::Config.default
+parser = Stupidedi::Builder::StateMachine.build(config)
+
+input  = if RUBY_VERSION > "1.8"
+           File.open("spec/fixtures/X221-HP835/1-good.txt", :encoding => "ISO-8859-1")
+         else
+           File.open("spec/fixtures/X221-HP835/1-good.txt")
+         end
+
+# Reader.build accepts IO (File), String, and DelegateInput
+parser, result = parser.read(Stupidedi::Reader.build(input))
+
+# Report fatal tokenizer failures
+if result.fatal?
+  result.explain{|reason| raise reason + " at #{result.position.inspect}" }
+end
+
+# Helper function: fetch an element from the current segment
+def el(m, *ns, &block)
+  if Stupidedi::Either === m
+    m.tap{|m| el(m, *ns, &block) }
+  else
+    yield(*ns.map{|n| m.elementn(n).map(&:value).fetch })
+  end
+end
+
+# Print some information
+parser.first
+  .flatmap{|m| m.find(:GS) }
+  .flatmap{|m| m.find(:ST) }
+  .tap do |m|
+    el(m.find(:N1, "PR"), 2){|e| puts "Payer: #{e}" }
+    el(m.find(:N1, "PE"), 2){|e| puts "Payee: #{e}" }
+  end
+  .flatmap{|m| m.find(:LX) }
+  .flatmap{|m| m.find(:CLP) }
+  .flatmap{|m| m.find(:NM1, "QC") }
+  .tap{|m| el(m, 3, 4){|l,f| puts "Patient: #{l}, #{f}" }}
+```

data/Rakefile

@@ -3,8 +3,8 @@ abspath = Pathname.new(File.dirname(__FILE__)).expand_path
 relpath = abspath.relative_path_from(Pathname.pwd)
 
 begin
-  require "rubygems"
-  require "bundler/setup"
+# require "rubygems"
+# require "bundler/setup"
 rescue LoadError
   warn "couldn't load bundler:"
   warn "  #{$!}"
@@ -13,7 +13,7 @@ end
 task :default => :spec
 
 task :console do
-  exec *%w(irb -I lib -r stupidedi)
+  exec(*%w(irb -I lib -r stupidedi))
 end
 
 begin
@@ -80,12 +80,18 @@ begin
       end
     end
   end
-rescue LoadError
+rescue LoadError => e
   task :rcov do
     warn "couldn't load rcov:"
-    warn "  #{$!}"
+    warn "  #{e}"
     exit 1
   end
+end if RUBY_VERSION <= "1.9"
+
+if RUBY_VERSION >= "1.9"
+  # spec/spec_helper.rb will load SimpleCov
+  task :rcov => :spec do
+  end
 end
 
 begin
@@ -99,10 +105,10 @@ begin
     mkdir_p "#{relpath}/doc/generated/images"
   end
 
-rescue LoadError
+rescue LoadError => e
   task :yard do
     warn "couldn't load yard:"
-    warn "  #{$!}"
+    warn "  #{e}"
     exit 1
   end
 end

data/bin/edi-ed

@@ -2,41 +2,18 @@
 require "pathname"
 $:.unshift(File.expand_path("../../lib", Pathname.new(__FILE__).realpath))
 
+# Short-lived processes should win some peformance gains here
+GC.disable
+
 require "stupidedi"
 require "pp"
 
 # This will be auto-enabled when $stdout.tty?, but -C forces color output
 require "term/ansicolor" if ARGV.delete("-C")
 
-config = Stupidedi::Config.new
-config.interchange.tap do |c|
-  c.register("00401") { Stupidedi::Versions::Interchanges::FourOhOne::InterchangeDef }
-  c.register("00501") { Stupidedi::Versions::Interchanges::FiveOhOne::InterchangeDef }
-end
-
-config.functional_group.tap do |c|
-# c.register("004010") { Stupidedi::Versions::FunctionalGroups::FortyTen::FunctionalGroupDef }
-  c.register("004010") { Stupidedi::Versions::FunctionalGroups::FiftyTen::FunctionalGroupDef }
-  c.register("005010") { Stupidedi::Versions::FunctionalGroups::FiftyTen::FunctionalGroupDef }
-end
-
-config.transaction_set.tap do |c|
-  c.register("005010", "HN", "277") { Stupidedi::Versions::FunctionalGroups::FiftyTen::TransactionSetDefs::HN277 }
-  c.register("005010", "HP", "835") { Stupidedi::Versions::FunctionalGroups::FiftyTen::TransactionSetDefs::HP835 }
-  c.register("005010", "HC", "837") { Stupidedi::Versions::FunctionalGroups::FiftyTen::TransactionSetDefs::HC837 }
-  c.register("005010", "FA", "999") { Stupidedi::Versions::FunctionalGroups::FiftyTen::TransactionSetDefs::FA999 }
-
-  c.register("005010X214", "HN", "277") { Stupidedi::Guides::FiftyTen::X214::HN277  }
-  c.register("005010X221", "HP", "835") { Stupidedi::Guides::FiftyTen::X221::HP835  }
-  c.register("005010X222", "HC", "837") { Stupidedi::Guides::FiftyTen::X222::HC837P }
-  c.register("005010X231", "FA", "999") { Stupidedi::Guides::FiftyTen::X231::FA999  }
-
-  c.register("005010X221A1", "HP", "835") { Stupidedi::Guides::FiftyTen::X221A1::HP835  }
-  c.register("004010X098A1", "HC", "837") { Stupidedi::Guides::FiftyTen::X222A1::HC837P }
-  c.register("005010X222A1", "HC", "837") { Stupidedi::Guides::FiftyTen::X222A1::HC837P }
-  c.register("005010X231A1", "FA", "999") { Stupidedi::Guides::FiftyTen::X231A1::FA999  }
-end
-
+# config = Stupidedi::Config.default
+# config = Stupidedi::Config.contrib
+config = Stupidedi::Config.hipaa
 config.editor.tap do |c|
   c.register(Stupidedi::Versions::Interchanges::FourOhOne::InterchangeDef) { Stupidedi::Editor::FiveOhOneEd }
   c.register(Stupidedi::Versions::Interchanges::FiveOhOne::InterchangeDef) { Stupidedi::Editor::FiveOhOneEd }
@@ -51,7 +28,7 @@ start  = Time.now
 
 ARGV.each do |path|
   File.open(path) do |io|
-    parser, result = parser.read(Stupidedi::Reader.build(io))
+    parser, = parser.read(Stupidedi::Reader.build(io))
   end
 end
 
@@ -64,7 +41,7 @@ begin
   a = parser.first
   b = parser.last
 
-  a.flatmap{|a| b.flatmap{|b| a.distance(b) }}.
+  a.flatmap{|_a| b.flatmap{|_b| _a.distance(_b) }}.
     tap{|d| puts "#{d + 1} segments" }
 end