filter_io 0.1.1 → 0.1.2

data/.gitignore

@@ -1,2 +1,3 @@
 *.gemspec
 pkg
+coverage

data/README.markdown

@@ -1,7 +1,7 @@
 # `filter_io`
 ## Filter IO streams with a block. Ruby's FilterInputStream.
 
-`filter_io` is analogous to Java's `FilterIOStream` in that it allows you to intercept and process data in an IO stream. This is particularly useful when used to clean up bad input data when using a CSV or XML parser.
+`filter_io` is analogous to Java's `FilterIOStream` in that it allows you to intercept and process data in an IO stream. This is particularly useful when cleaning up bad input data for a CSV or XML parser.
 
 `filter_io` provides a one-pass approach to filtering data which can be much faster and memory efficient than doing two passes (cleaning the source file into a buffer and then calling the original parser).
 
@@ -9,9 +9,9 @@
 
 ### Installation
 
-You can install from Gemcutter by running:
+You can install the gem by running:
 
-    sudo gem install filter_io
+    gem install filter_io
 
 ### Example Usage
 
@@ -45,7 +45,7 @@ A common usage of `filter_io` is to normalise line endings before parsing CSV da
 
 ### Reference
 
-Call `FilterIO.new` with the original IO stream, any options and the filtering block. The returned object pretends like a normal read-only non-seekable IO stream.
+Call `FilterIO.new` with the original IO stream, any options and the filtering block. The returned `filter_io` object acts like a normal read-only forward-only IO stream.
 
 #### Block `state` parameter
 
@@ -54,7 +54,7 @@ An optional second parameter to the block is the `state` parameter which contain
 * `bof?`: Returns true if this is the *first* chuck of the stream.
 * `eof?`: Returns true if this is the *last* chunk of the stream.
 
-#### `NeedMoreData`
+#### Requesting Additional Data
 
 If the filtering block needs more data to be able to return anything, you can raise a `FilterIO::NeedMoreData` exception and `filter_io` will read another block and pass the additional data to you. This can be repeated as necessary until enough data is retrieved.
 
@@ -62,7 +62,7 @@ For example usage of `NeedMoreData`, see the line ending normalisation example a
 
 #### Re-buffering Unprocessed Data
 
-If your block is unable to process the whole chunk of data immediately, it can return both the processed chuck and the remainder to be processed later. This is done by returning a 2 element array: `[processed, unprocessed]`. If output is empty and unprocessed data is returned, `filter_io` will grab some more data and call the block again.
+If your block is unable to process the whole chunk of data immediately, it can return both the processed chuck and the remainder to be processed later. This is done by returning a 2 element array: `[processed, unprocessed]`. If `processed` is empty and there is `unprocessed` data, `filter_io` will grab another block of data from the source stream and call the block again.
 
 Here's an example which processes whole lines and prepends the line length to the beginning of each line.
 
@@ -88,15 +88,13 @@ Ruby 1.9 has character encoding support can convert between UTF-8, ISO-8859-1, A
 As per the core `IO` object, if `read` is called with a length (in bytes), the data will be returned in the external encoding.
 In summary, everything should Just Work™
 
-### Note on Patches/Pull Requests
+### Notes on Patches/Pull Requests
  
-* Fork the project.
-* Make your feature addition or bug fix.
-* Add tests for it. This is important so I don't break it in a
-  future version unintentionally.
-* Commit, do not mess with rakefile, version, or history.
-  (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
-* Send me a pull request. Bonus points for topic branches.
+1. Fork the project.
+1. Make your feature addition or bug fix.
+1. Add tests for it. This is important so I don't break it in a future version unintentionally.
+1. Commit, do not mess with Rakefile, VERSION, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
+1. Send me a pull request. Bonus points for topic branches.
 
 ### Copyright
 

data/Rakefile

@@ -30,3 +30,25 @@ begin
 rescue LoadError
   puts "Jeweler (or a dependency) not available. Install it with: gem install jeweler"
 end
+
+begin
+  require 'rcov/rcovtask'
+  Rcov::RcovTask.new do |t|
+    t.libs << "test"
+    t.rcov_opts = [
+      "--exclude '^(?!lib)'"
+    ]
+    t.test_files = FileList[
+      'test/**/*_test.rb'
+    ]
+    t.output_dir = 'coverage'
+    t.verbose = true
+  end
+  task :rcov do
+    system "open coverage/index.html"
+  end
+rescue LoadError
+  task :rcov do
+    raise "You must install the 'rcov' gem"
+  end
+end

data/VERSION

@@ -1 +1 @@
-0.1.1
+0.1.2

data/lib/filter_io.rb

@@ -167,7 +167,7 @@ class FilterIO
     
     # fill the buffer until it contains the separator sequence
     until source_eof? or @buffer.index(sep_string)
-      buffer_data
+      buffer_data @options[:block_size]
     end
     
     # calculate how much of the buffer to return

data/test/filter_io_test.rb

@@ -317,7 +317,7 @@ class FilterIOTest < ActiveSupport::TestCase
     assert_equal expected, io.read
   end
   
-  test "block size" do
+  test "block size for read(nil)" do
     [1,4,7,9,13,30].each do |block_size|
       input = ('A'..'Z').to_a.join
       expected = input.chars.enum_for(:each_slice, block_size).to_a.map(&:join).map { |x| "[#{x}]" }.join
@@ -328,6 +328,18 @@ class FilterIOTest < ActiveSupport::TestCase
     end
   end
   
+  test "block size for gets/readline" do
+    [1,4,7,9,13,30].each do |block_size|
+      input = "ABCDEFG\nHJIKLMNOP\n"
+      expected = input.chars.enum_for(:each_slice, block_size).to_a.map(&:join).map { |x| "[#{x}]" }.join.lines.to_a
+      io = FilterIO.new(StringIO.new(input), :block_size => block_size) do |data|
+        "[#{data}]"
+      end
+      actual = io.readlines
+      assert_equal expected, actual
+    end
+  end
+  
   test "block size different to read size" do
     (1..5).each do |block_size|
       input_str = ('A'..'Z').to_a.join

metadata

@@ -1,12 +1,13 @@
 --- !ruby/object:Gem::Specification 
 name: filter_io
 version: !ruby/object:Gem::Version 
+  hash: 31
   prerelease: false
   segments: 
   - 0
   - 1
-  - 1
-  version: 0.1.1
+  - 2
+  version: 0.1.2
 platform: ruby
 authors: 
 - Jason Weathered
@@ -14,7 +15,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2010-06-14 00:00:00 +10:00
+date: 2010-06-27 00:00:00 +10:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency 
@@ -25,6 +26,7 @@ dependencies:
     requirements: 
     - - ">="
       - !ruby/object:Gem::Version 
+        hash: 3
         segments: 
         - 0
         version: "0"
@@ -62,6 +64,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements: 
   - - ">="
     - !ruby/object:Gem::Version 
+      hash: 3
       segments: 
       - 0
       version: "0"
@@ -70,6 +73,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
   requirements: 
   - - ">="
     - !ruby/object:Gem::Version 
+      hash: 3
       segments: 
       - 0
       version: "0"