sinew 2.0.1 → 2.0.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fde4bbaa95fce45f3a7ae7aeacab1672615ea1ace852845b0395ce9cce32f861
-  data.tar.gz: 5743800570722443f704c5fc7bc421346cc4f2fb116b8fe9f615bf84fb95f826
+  metadata.gz: 33506a03f47a88cae5bf7e0f4675d7cf83d86ba3c96f0880f5c473a7b23b167b
+  data.tar.gz: 990bd4690f9fe799774c349314a32ab2c08979d555f03891316c5e0be8a4ad3d
 SHA512:
-  metadata.gz: 94009061e7f4e36cc23528be3866c6a372df51a83e096144cafbd923259439e6d44a7d656fbdcfe09c2e059b48deb553caca3ec5d332b33845afd1e91550371a
-  data.tar.gz: 5a8baf7fbdba371065c796c9fdce4312039558b27b4a16676b3df16d5138916ce84db0677dce6ede1831be8040df9112a0491e421813af5e5fd0b0b747d49239
+  metadata.gz: 9644097a2e11d8cba59a7985dfe770f27b00d5d18b676d0cacdee3e73a21f1b6c237b3bb58d68489d2a67fc981f7a7f8bb27a6e6fb23781f318cde78b392d7cd
+  data.tar.gz: 667c301e7896b27162a77cff5165f264a0c2b73afbe5c35f541181709118185a13241d187ff8f8d5964e302537064ff927d9fa64ece2cb10ca65ba7dd89ce807

data/LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2012 Adam Doppelt
+Copyright (c) 2012-2018 Adam Doppelt
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the

data/README.md

@@ -1,11 +1,13 @@
+![Travis](https://travis-ci.org/gurgeous/sinew.svg?branch=master)
+
 ## Welcome to Sinew
 
 Sinew collects structured data from web sites (screen scraping). It provides a Ruby DSL built for crawling, a robust caching system, and integration with [Nokogiri](http://nokogiri.org). Though small, this project is the culmination of years of effort based on crawling systems built at several different companies.
 
 Sinew is distributed as a ruby gem:
 
-```ruby
-gem install sinew
+```sh
+$ gem install sinew
 ```
 
 or in your Gemfile:
@@ -16,17 +18,16 @@ gem 'sinew'
 
 ## Table of Contents
 
-<!---
-markdown-toc --no-firsth1 --maxdepth 1 readme.md
--->
+<!--- markdown-toc --no-firsth1 --maxdepth 1 readme.md -->
 
-* [Sinew 2 (May 2018)](#sinew-2-may-2018)
+* [Sinew 2](#sinew-2-may-2018)
 * [Quick Example](#quick-example)
 * [How it Works](#how-it-works)
 * [DSL Reference](#dsl-reference)
 * [Hints](#hints)
 * [Limitations](#limitations)
 * [Changelog](#changelog)
+* [License](#license)
 
 ## Sinew 2 (May 2018)
 
@@ -34,7 +35,7 @@ I am pleased to announce the release of Sinew 2.0, a complete rewrite of Sinew f
 
 * Remove dependencies on active_support, curl and tidy. We use HTTParty now.
 * Much easier to customize requests in `.sinew` files. For example, setting User-Agent or Bearer tokens.
-* More operations like `post_json` or the generic `http`. These methods are thing wrappers around HTTParty.
+* More operations like `post_json` or the generic `http`. These methods are thin wrappers around HTTParty.
 * New end-of-run report.
 * Tests, rubocop, vscode settings, travis, etc.
 
@@ -124,15 +125,18 @@ Because all requests are cached, you can run Sinew repeatedly with confidence. R
 #### Making requests
 
 * `get(url, query = {})` - fetch a url with HTTP GET. URL parameters can be added using `query.
-* `post(url, form = {})` - fetch a url with HTTP POST, using `form` as the POST body.
+* `post(url, form = {})` - fetch a url with HTTP POST, using `form` as the URL encoded POST body.
 * `post_json(url, json = {})` - fetch a url with HTTP POST, using `json` as the POST body.
 * `http(method, url, options = {})` - use this for more complex requests
 
 #### Parsing the response
 
+These variables are set after each HTTP request.
+
 * `raw` - the raw response from the last request
 * `html` - like `raw`, but with a handful of HTML-specific whitespace cleanups
-* `noko` - a [Nokogiri](http://nokogiri.org) document built from the tidied HTML
+* `noko` - parse the response as HTML and return a [Nokogiri](http://nokogiri.org) document
+* `xml` - parse the response as XML and return a [Nokogiri](http://nokogiri.org) document
 * `json` - parse the response as JSON, with symbolized keys
 * `url` - the url of the last request. If the request goes through a redirect, `url` will reflect the final url.
 * `uri` - the URI of the last request. This is useful for resolving relative URLs.
@@ -169,19 +173,24 @@ noko.css("table")[4].css("td").select { |i| i[:width].to_i > 80 }.map(&:text)
 
 ## Changelog
 
-#### 2.0.0 (May 2018)
+#### 2.0.2 (May 2018)
 
-* Complete rewrite. See above.
+* Support for `--limit`, `--proxy` and the `xml` variable
+* Dedup - warn and ignore if row[:url] has already been emitted
+* Auto gunzip if contents are compressed
 
-#### 1.0.3
+#### 2.0.1 (May 2018)
 
-* Friendlier message if curl or tidy are missing.
+* Support for legacy cached `head` files from Sinew 1
+
+#### 2.0.0 (May 2018)
+
+* Complete rewrite. See above.
 
-#### 1.0.2
+#### 1.0.3 (June 2012)
 
-* Remove entity options from tidy, which didn't work on MacOS (thanks Rex!)
+...
 
-#### 1.0.1
+## License
 
-* Trying to run on 1.8 produces a fatal error. Onward!
-* Added first batch of unit tests
+This extension is [licensed under the MIT License](LICENSE).

data/bin/sinew

@@ -11,11 +11,13 @@ require 'slop'
 
 options = Slop.parse do |o|
   o.banner = 'Usage: sinew [options] <gub.sinew>'
-  o.bool '-v', '--verbose', 'dump every row'
-  o.bool '--version', 'show version'
+  o.bool '-v', '--verbose', 'dump emitted rows while running'
   o.bool '-q', '--quiet', 'suppress some output'
-  o.string '--cache', 'Set the cache directory (defaults to ~/.sinew)', default: "#{ENV['HOME']}/.sinew"
-  o.on '--help' do
+  o.integer '-l', '--limit', 'quit after emitting this many rows'
+  o.string '-c', '--cache', 'set custom cache directory', default: "#{ENV['HOME']}/.sinew"
+  o.string '--proxy', 'use host[:port] as HTTP proxy'
+  o.bool '--version', 'show version and exit'
+  o.on('--help', 'show this help') do
     puts o
     exit
   end

data/lib/sinew/dsl.rb

@@ -7,6 +7,9 @@ require 'cgi'
 
 module Sinew
   class DSL
+    # this is used to break out of --limit
+    class LimitError < StandardError; end
+
     attr_reader :sinew, :raw, :uri, :elapsed
 
     def initialize(sinew)
@@ -15,8 +18,12 @@ module Sinew
 
     def run
       tm = Time.now
-      recipe = sinew.options[:recipe]
-      instance_eval(File.read(recipe, mode: 'rb'), recipe)
+      begin
+        recipe = sinew.options[:recipe]
+        instance_eval(File.read(recipe, mode: 'rb'), recipe)
+      rescue LimitError
+        # ignore - this is flow control for --limit
+      end
       @elapsed = Time.now - tm
     end
 
@@ -46,14 +53,13 @@ module Sinew
 
     def http(method, url, options = {})
       # reset
-      @html = @noko = @json = @url = nil
+      instance_variables.each do |i|
+        instance_variable_set(i, nil) if i != :@sinew
+      end
 
-      # fetch
+      # fetch and make response available to callers
       response = sinew.http(method, url, options)
-
-      # respond
-      @uri = response.uri
-      @raw = response.body
+      @uri, @raw = response.uri, response.body
     end
 
     #
@@ -75,6 +81,10 @@ module Sinew
       @noko ||= Nokogiri::HTML(html)
     end
 
+    def xml
+      @xml ||= Nokogiri::XML(html)
+    end
+
     def json
       @json ||= JSON.parse(raw, symbolize_names: true)
     end
@@ -93,6 +103,9 @@ module Sinew
 
     def csv_emit(row)
       sinew.output.emit(row)
+      if sinew.output.count == sinew.options[:limit]
+        raise LimitError.new
+      end
     end
   end
 end

data/lib/sinew/main.rb

@@ -15,6 +15,12 @@ module Sinew
       @runtime_options = RuntimeOptions.new
       @request_tm = Time.at(0)
       @request_count = 0
+
+      if options[:proxy]
+        addr, port = options[:proxy].split(':')
+        runtime_options.httparty_options[:http_proxyaddr] = addr
+        runtime_options.httparty_options[:http_proxyport] = port || 80
+      end
     end
 
     def run

data/lib/sinew/output.rb

@@ -1,4 +1,5 @@
 require 'csv'
+require 'set'
 require 'stringex'
 
 #
@@ -7,11 +8,12 @@ require 'stringex'
 
 module Sinew
   class Output
-    attr_reader :sinew, :columns, :rows, :csv
+    attr_reader :sinew, :columns, :rows, :urls, :csv
 
     def initialize(sinew)
       @sinew = sinew
       @rows = []
+      @urls = Set.new
     end
 
     def filename
@@ -41,6 +43,8 @@ module Sinew
       # implicit header if necessary
       header(row.keys) if !csv
 
+      # don't allow duplicate urls
+      return if dup_url?(row)
       rows << row.dup
 
       # map columns to row, and normalize along the way
@@ -94,6 +98,9 @@ module Sinew
         s.to_s
       end
 
+      # strip html tags. Note that we replace tags with spaces
+      s = s.gsub(/<[^>]+>/, ' ')
+
       #
       # Below uses stringex
       #
@@ -101,9 +108,6 @@ module Sinew
       # github.com/rsl/stringex/blob/master/lib/stringex/localization/conversion_expressions.rb
       #
 
-      # <a>b</a> => b
-      s = s.strip_html_tags
-
       # Converts MS Word 'smart punctuation' to ASCII
       s = s.convert_smart_punctuation
 
@@ -122,5 +126,17 @@ module Sinew
       s
     end
     protected :normalize
+
+    def dup_url?(row)
+      if url = row[:url]
+        if urls.include?(url)
+          sinew.warning("duplicate url: #{url}") if !sinew.quiet?
+          return true
+        end
+        urls << url
+      end
+      false
+    end
+    protected :dup_url?
   end
 end

data/lib/sinew/request.rb

@@ -28,7 +28,10 @@ module Sinew
     def perform
       validate!
 
-      # merge global/options headers
+      # merge optons
+      options = self.options.merge(sinew.runtime_options.httparty_options)
+
+      # merge headers
       headers = sinew.runtime_options.headers
       headers = headers.merge(options[:headers]) if options[:headers]
       options[:headers] = headers

data/lib/sinew/response.rb

@@ -1,3 +1,6 @@
+require 'stringio'
+require 'zlib'
+
 #
 # An HTTP response. Mostly a wrapper around HTTParty.
 #
@@ -16,13 +19,7 @@ module Sinew
         response.uri = party_response.request.last_uri
         response.code = party_response.code
         response.headers = party_response.headers.to_h
-
-        # force to utf-8 as best we can
-        body = party_response.body
-        if body.encoding != Encoding::UTF_8
-          body = body.encode('UTF-8', invalid: :replace, undef: :replace, replace: '?')
-        end
-        response.body = body
+        response.body = process_body(party_response)
       end
     end
 
@@ -60,21 +57,43 @@ module Sinew
     end
 
     def self.from_legacy_head(response, head)
-      response.tap do |response|
+      response.tap do |r|
         case head
         when /\ACURLER_ERROR/
           # error
-          response.code = 999
+          r.code = 999
         when /\AHTTP/
           # redirect
           location = head.scan(/Location: ([^\r\n]+)/).flatten.last
-          response.uri += location
+          r.uri += location
         else
-          $stderr.puts "unknown cached /head for #{response.uri}"
+          $stderr.puts "unknown cached /head for #{r.uri}"
         end
       end
     end
 
+    # helper for decoding bodies before parsing
+    def self.process_body(response)
+      body = response.body
+
+      # inflate if necessary
+      bits = body[0, 10].force_encoding('BINARY')
+      if bits =~ /\A\x1f\x8b/n
+        body = Zlib::GzipReader.new(StringIO.new(body)).read
+      end
+
+      # force to utf-8 if we think this could be text
+      if body.encoding != Encoding::UTF_8
+        if content_type = response.headers['content-type']
+          if content_type =~ /\b(html|javascript|json|text|xml)\b/
+            body = body.encode('UTF-8', invalid: :replace, undef: :replace, replace: '?')
+          end
+        end
+      end
+
+      body
+    end
+
     #
     # accessors
     #

data/lib/sinew/runtime_options.rb

@@ -7,6 +7,7 @@ module Sinew
     attr_accessor :retries
     attr_accessor :rate_limit
     attr_accessor :headers
+    attr_accessor :httparty_options
     attr_accessor :before_generate_cache_key
 
     def initialize
@@ -15,6 +16,7 @@ module Sinew
       self.headers = {
         'User-Agent' => "sinew/#{VERSION}",
       }
+      self.httparty_options = {}
       self.before_generate_cache_key = ->(i) { i }
 
       # for testing

data/lib/sinew/version.rb

@@ -1,4 +1,4 @@
 module Sinew
   # Gem version
-  VERSION = '2.0.1'.freeze
+  VERSION = '2.0.2'.freeze
 end

data/test/recipes/array_header.sinew

@@ -0,0 +1,6 @@
+csv_header(%i[n a p])
+csv_emit(n: 'n1', a: 'a1')
+
+# OUTPUT
+# n,a,p
+# n1,a1,""

data/test/recipes/basic.sinew

@@ -0,0 +1,8 @@
+get 'http://httpbin.org/html'
+raw.scan(/<h1>([^<]+)/) do
+  csv_emit(h1: $1)
+end
+
+# OUTPUT
+# h1
+# Herman Melville - Moby-Dick

data/test/recipes/dups.sinew

@@ -0,0 +1,7 @@
+5.times do
+  csv_emit(url: 'https://gub')
+end
+
+# OUTPUT
+# url
+# https://gub

data/test/recipes/implicit_header.sinew

@@ -0,0 +1,5 @@
+csv_emit(name: 'bob', address: 'main')
+
+# OUTPUT
+# name,address
+# bob,main

data/test/recipes/limit.sinew

@@ -0,0 +1,11 @@
+# OPTIONS { limit: 3 }
+
+(1..5).each do |i|
+  csv_emit(i: i)
+end
+
+# OUTPUT
+# i
+# 1
+# 2
+# 3

data/test/recipes/noko.sinew

@@ -0,0 +1,9 @@
+get 'http://httpbin.org/xml'
+noko.css('slide title').each do |title|
+  csv_emit(title: title.text)
+end
+
+# OUTPUT
+# title
+# Wake up to WonderWidgets!
+# Overview

data/test/recipes/uri.sinew

@@ -0,0 +1,11 @@
+# This tests get by URI, URI math, and csv_emit with uri
+get(URI.parse('http://httpbin.org/html'))
+csv_emit(url: uri)
+
+get(uri + '../get')
+csv_emit(url: uri)
+
+# OUTPUT
+# url
+# http://httpbin.org/html
+# http://httpbin.org/get

data/test/recipes/xml.sinew

@@ -0,0 +1,8 @@
+get 'http://httpbin.org/html'
+noko.css('h1').each do |h1|
+  csv_emit(h1: h1.text)
+end
+
+# OUTPUT
+# h1
+# Herman Melville - Moby-Dick

data/test/test_helper.rb

@@ -12,8 +12,6 @@ require 'sinew'
 
 class MiniTest::Test
   TMP = '/tmp/_test_sinew'.freeze
-  RECIPE = "#{TMP}/test.sinew".freeze
-  CSV = "#{TMP}/test.csv".freeze
   HTML = File.read("#{__dir__}/test.html")
 
   def setup
@@ -27,16 +25,10 @@ class MiniTest::Test
   end
 
   def sinew
-    @sinew ||= Sinew::Main.new(cache: TMP, quiet: true, recipe: RECIPE)
+    @sinew ||= Sinew::Main.new(cache: TMP, quiet: true, recipe: "#{TMP}/ignore.sinew")
   end
   protected :sinew
 
-  def run_recipe(recipe)
-    File.write(RECIPE, recipe)
-    sinew.run
-  end
-  protected :run_recipe
-
   def test_network?
     !!ENV['SINEW_TEST_NETWORK']
   end
@@ -50,6 +42,7 @@ class MiniTest::Test
     stub_request(:get, %r{http://[^/]+/status/\d+}).to_return(method(:respond_status))
     stub_request(:get, %r{http://[^/]+/(relative-)?redirect/\d+}).to_return(method(:respond_redirect))
     stub_request(:get, %r{http://[^/]+/delay/\d+}).to_timeout
+    stub_request(:get, %r{http://[^/]+/xml}).to_return(method(:respond_xml))
   end
   protected :stub_network
 
@@ -58,7 +51,7 @@ class MiniTest::Test
   #
 
   def respond_html(_request)
-    # this html was carefully chosen to match httpbin.org/html
+    # this html was carefully chosen to somewhat match httpbin.org/html
     html = <<~EOF
       <body>
         <h1>Herman Melville - Moby-Dick</h1>
@@ -68,20 +61,37 @@ class MiniTest::Test
   end
   protected :respond_html
 
+  def respond_xml(_request)
+    # this xml was carefully chosen to somewhat match httpbin.org/xml
+    xml = <<~EOF
+      <!--   A SAMPLE set of slides   -->
+      <slideshow>
+        <slide type="all">
+          <title>Wake up to WonderWidgets!</title>
+        </slide>
+        <slide type="all">
+          <title>Overview</title>
+        </slide>
+      </slideshow>
+    EOF
+    { body: xml }
+  end
+  protected :respond_xml
+
   def respond_echo(request)
     response = {}
     response[:headers] = request.headers
 
     # args
     response[:args] = if request.uri.query
-      CGI.parse(request.uri.query).map { |k, v| [k, v.first] }.to_h
+      CGI.parse(request.uri.query).map { |k, v| [ k, v.first ] }.to_h
     else
       {}
     end
 
     # form
     if request.headers['Content-Type'] == 'application/x-www-form-urlencoded'
-      response[:form] = CGI.parse(request.body).map { |k, v| [k, v.first] }.to_h
+      response[:form] = CGI.parse(request.body).map { |k, v| [ k, v.first ] }.to_h
     end
 
     # json

data/test/test_legacy.rb

@@ -12,8 +12,10 @@ class TestLegacy < MiniTest::Test
   end
 
   def test_legacy
-    sinew.dsl.get('http://eu.httpbin.org/status/500')
-    assert_equal "\n", sinew.dsl.raw
+    assert_output(/failed with 999/) do
+      sinew.dsl.get('http://eu.httpbin.org/status/500')
+      assert_equal "\n", sinew.dsl.raw
+    end
 
     sinew.dsl.get('http://eu.httpbin.org/redirect/3')
     assert_equal 'http://eu.httpbin.org/get', sinew.dsl.url

data/test/test_main.rb

@@ -1,26 +1,8 @@
 require_relative 'test_helper'
 
-class TestMain < MiniTest::Test
-  def test_noko
-    run_recipe <<~'EOF'
-      get 'http://httpbin.org/html'
-      noko.css("h1").each do |h1|
-        csv_emit(h1: h1.text)
-      end
-    EOF
-    assert_equal("h1\nHerman Melville - Moby-Dick\n", File.read(CSV))
-  end
-
-  def test_raw
-    run_recipe <<~'EOF'
-      get "http://httpbin.org/html"
-      raw.scan(/<h1>([^<]+)/) do
-        csv_emit(h1: $1)
-      end
-    EOF
-    assert_equal("h1\nHerman Melville - Moby-Dick\n", File.read(CSV))
-  end
+require 'base64'
 
+class TestMain < MiniTest::Test
   def test_rate_limit
     # true network requests call sleep for timeouts, which interferes with our
     # instrumentation of Kernel#sleep
@@ -43,4 +25,10 @@ class TestMain < MiniTest::Test
     Kernel.send(:alias_method, :sleep, :old_sleep)
     Kernel.send(:undef_method, :old_sleep)
   end
+
+  def test_gunzip
+    body = Base64.decode64('H4sICBRI61oAA2d1Yi50eHQASy9N4gIAJlqRYgQAAAA=')
+    body = Sinew::Response.process_body(OpenStruct.new(body: body))
+    assert_equal 'gub', body.strip
+  end
 end

data/test/test_output.rb

@@ -1,25 +1,6 @@
 require_relative 'test_helper'
 
 class TestOutput < MiniTest::Test
-  def test_output
-    sinew.dsl.csv_header(:n, :a, :p)
-    sinew.dsl.csv_emit(n: 'n1', a: 'a1')
-    sinew.dsl.csv_emit(n: 'n2', a: 'a2')
-    assert_equal 2, sinew.output.count
-    assert_equal "n,a,p\nn1,a1,\"\"\nn2,a2,\"\"\n", File.read(CSV)
-  end
-
-  def test_implicit_header
-    sinew.dsl.csv_emit(name: 'bob', address: 'main')
-    assert_equal "name,address\nbob,main\n", File.read(CSV)
-  end
-
-  def test_array_header
-    sinew.dsl.csv_header(%i[n a p])
-    sinew.dsl.csv_emit(n: 'n1', a: 'a1')
-    assert_equal "n,a,p\nn1,a1,\"\"\n", File.read(CSV)
-  end
-
   def test_filenames
     sinew = Sinew::Main.new(recipe: 'gub.sinew')
     assert_equal 'gub.csv', sinew.output.filename
@@ -59,6 +40,8 @@ class TestOutput < MiniTest::Test
 
     # strip_html_tags
     assert_equal('gub', output.send(:normalize, '<tag>gub</tag>'))
+    # strip_html_tags and replace with spaces
+    assert_equal('hello world', output.send(:normalize, '<tag>hello<br>world</tag>'))
     # convert_smart_punctuation
     assert_equal('"gub"', output.send(:normalize, "\302\223gub\302\224"))
     # convert_accented_html_entities

data/test/test_recipes.rb

@@ -0,0 +1,60 @@
+require_relative 'test_helper'
+
+class TestRecipe < MiniTest::Test
+  DIR = File.expand_path('recipes', __dir__)
+  TEST_SINEW = "#{TMP}/test.sinew".freeze
+  TEST_CSV = "#{TMP}/test.csv".freeze
+
+  def test_recipes
+    Dir.chdir(DIR) do
+      Dir['*.sinew'].sort.each do |filename|
+        recipe = IO.read(filename)
+
+        # get ready
+        IO.write(TEST_SINEW, recipe)
+        sinew = Sinew::Main.new(cache: TMP, quiet: true, recipe: TEST_SINEW)
+
+        # read OPTIONS
+        if options = options_from(recipe)
+          options.each do |key, value|
+            sinew.options[key] = value
+          end
+        end
+
+        # read OUTPUT
+        output = output_from(recipe, filename)
+
+        # run
+        sinew.run
+
+        # assert
+        csv = IO.read(TEST_CSV)
+        assert_equal(output, csv, "Output didn't match for recipes/#{filename}")
+      end
+    end
+  end
+
+  def options_from(recipe)
+    if options = recipe[/^#\s*OPTIONS\s*(\{.*\})/, 1]
+      # rubocop:disable Security/Eval
+      eval(options)
+      # rubocop:enable Security/Eval
+    end
+  end
+  protected :options_from
+
+  def output_from(recipe, filename)
+    lines = recipe.split("\n")
+    first_line = lines.index { |i| i =~ /^# OUTPUT/ }
+    if !first_line
+      raise "# OUTPUT not found in recipes/#{filename}"
+    end
+
+    output = lines[first_line + 1..-1]
+    output = output.map { |i| i.gsub(/^# /, '') }
+    output = output.join("\n")
+    output += "\n"
+    output
+  end
+  protected :output_from
+end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: sinew
 version: !ruby/object:Gem::Version
-  version: 2.0.1
+  version: 2.0.2
 platform: ruby
 authors:
 - Adam Doppelt
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-05-02 00:00:00.000000000 Z
+date: 2018-05-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: awesome_print
@@ -186,6 +186,14 @@ files:
 - test/legacy/eu.httpbin.org/redirect,3
 - test/legacy/eu.httpbin.org/status,500
 - test/legacy/legacy.sinew
+- test/recipes/array_header.sinew
+- test/recipes/basic.sinew
+- test/recipes/dups.sinew
+- test/recipes/implicit_header.sinew
+- test/recipes/limit.sinew
+- test/recipes/noko.sinew
+- test/recipes/uri.sinew
+- test/recipes/xml.sinew
 - test/test.html
 - test/test_cache.rb
 - test/test_helper.rb
@@ -193,6 +201,7 @@ files:
 - test/test_main.rb
 - test/test_nokogiri_ext.rb
 - test/test_output.rb
+- test/test_recipes.rb
 - test/test_requests.rb
 - test/test_utf8.rb
 homepage: http://github.com/gurgeous/sinew
@@ -225,6 +234,14 @@ test_files:
 - test/legacy/eu.httpbin.org/redirect,3
 - test/legacy/eu.httpbin.org/status,500
 - test/legacy/legacy.sinew
+- test/recipes/array_header.sinew
+- test/recipes/basic.sinew
+- test/recipes/dups.sinew
+- test/recipes/implicit_header.sinew
+- test/recipes/limit.sinew
+- test/recipes/noko.sinew
+- test/recipes/uri.sinew
+- test/recipes/xml.sinew
 - test/test.html
 - test/test_cache.rb
 - test/test_helper.rb
@@ -232,5 +249,6 @@ test_files:
 - test/test_main.rb
 - test/test_nokogiri_ext.rb
 - test/test_output.rb
+- test/test_recipes.rb
 - test/test_requests.rb
 - test/test_utf8.rb