sec_query 1.2.0 → 1.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 7fcc071c9c2a20d9785eb3e18f08bafb8aa5d76a
-  data.tar.gz: 28e85cbe4e92ca568bdd66477bb34b345ec0a3b9
+  metadata.gz: e0dcb3dc699b2d6b83a79f026dec445d99c3edd5
+  data.tar.gz: 6192703420f6bcfd6ebbac230852ae4d1e6341a6
 SHA512:
-  metadata.gz: da3bc6d656bebf0c4248a2c476caa5c9fec28ad2e1cf9f2c2cd06b16519a802045255257d525bc3afd85c52e00689258fbc87fc7724621fdda65ad19f5bdf07e
-  data.tar.gz: f04996d7a20f500ee31e83798efa1cd1930f6d909fb86b5e94b3f7f5f9c29591cba74d701a18e742d8849d62ff414491f3e8c4c2a9b1f39f888440539b71f057
+  metadata.gz: 11a23324f82965969a807e269ed1e810253d85a890a504424026201835fae08eff389d9a9f1d3ec32f02cd7c179934bc394fd19b667b8388de55f7be5aa98eec
+  data.tar.gz: 621457f23173bbd5659477ff7232dd0b2b63f60d25bd43383bfa5d0ebccdae756ca143757d67b52bd45eadf7389580d02b85e9272a231d462d5ca89be0062a26

data/.gitignore

@@ -4,6 +4,8 @@
 .bundle
 .config
 .yardoc
+.ruby-gemset
+.ruby-version
 Gemfile.lock
 InstalledFiles
 _yardoc

data/README.md

@@ -8,7 +8,7 @@ Additionally retrieve some, or all, Relationships, Transactions and Filings as r
 
 ## Note: 9/13/16, SEC.GOV embraces SSL!
 
-On or before Septmember 13th, 2016, the SEC.gov updated their site to use SSL (Huzzah!). Version 1.2.0 addresses this change. All versions less than 1.2.0, will cease to function as expected. Update immeditiately.
+On or before Septmember 13th, 2016, the SEC.gov updated their site to use SSL (Huzzah!). Version 1.2.0 addresses this change. All versions less than 1.2.0, will cease to function as expected. Update immediately.
 
 ## Installation
 
@@ -28,9 +28,29 @@ If running 'sec_query' from the command prompt in irb:
 
 ## Functionality
 
-### FIND COMPANY:
+### Entity:
 
-#### By Stock Symbol:
+An Sec::Entity instance contains the following attributes:
+
+* cik
+* name
+* mailing_address
+* business_adddress
+* assigned_sic
+* assigned_sic_desc
+* assigned_sic_href
+* assitant_director
+* cik_href
+* formerly_name
+* state_location
+* state_location_href
+* state_of_incorporation
+
+#### Class Methods
+
+##### .find 
+
+###### By Stock Symbol:
 
 `SecQuery::Entity.find("aapl")`
 
@@ -38,7 +58,7 @@ Or:
 
 `SecQuery::Entity.find({:symbol=> "aapl"})`
 
-#### By Name:
+###### By Name:
 
 `SecQuery::Entity.find("Apple, Inc.")`
 
@@ -46,7 +66,7 @@ Or:
 
 `SecQuery::Entity.find({:name=> "Apple, Inc."})`
 
-#### By Central Index Key, CIK:
+######  Central Index Key, CIK
 
 `SecQuery::Entity.find( "0000320193")`
 
@@ -54,7 +74,7 @@ Or:
 
 `SecQuery::Entity.find({:cik=> "0000320193"})`
 
-#### FIND PERSON:
+###### By First, Middle and Last Name:
 
 By First, Middle and Last Name.
 
@@ -62,62 +82,41 @@ By First, Middle and Last Name.
 
 Middle initial or name is optional, but helps when there are multiple results for First and Last Name.
 
-### RELATIONSHIPS, TRANSACTIONS, FILINGS
-
-To return everything - All Relationships, Transactions and Filings - that the SEC Edgar system has stored on a company or person, do any of the following commands (They all do the same thing.):
-
-`SecQuery::Entity.find("AAPL",  true)`
-
-`SecQuery::Entity.find("AAPL",  true, true, true)`
-
-`SecQuery::Entity.find("AAPL", {:relationships=> true, :transactions=> true, :filings=>true})`
-
-`SecQuery::Entity.find("AAPL", :relationships=> true, :transactions=> true, :filings=>true)`
-
-`SecQuery::Entity.find("AAPL", :relationships=> true, :transactions=> {:start=> 0, :count=> 80}, :filings=>{:start=> 0, :count=> 80})`
-
-You may also limit either the transactions or filings by adding the :limit to the transaction or filing arguements.
-
-For example,
-
-`SecQuery::Entity.find("AAPL", :relationships=> true, :transactions=> {:start=> 0, :count=>20, :limit=> 20}, :filings=>{:start=> 0, :count=> 20, :limit=> 20})`
+#### Instance Methods
 
-The above query will only return the last 20 transactions and filings.  This is helpful when querying companies that may have thousands or tens of thousands of transactions or filings.
+##### .filings
 
-## Classes
+Returns a list of Sec::Filing instances for an Sec::Entity
 
-This gem contains four classes - Entity, Relationship, Transaction and Filing.  Each Class contains the listed fields. (Everything I could parse out of the query results.)
+### Sec::Filing
 
-* Entity
+SecQuery::Filing instance may contains the following attributes:
 
-`:first, :middle, :last, :name, :symbol, :cik, :url, :type, :sic, :location, :state_of_inc, :formerly, :mailing_address, :business_address, :relationships, :transactions, :filings`
+* cik
+* title
+* symmary
+* link
+* term
+* date
+* file_id
 
-* Relationship
+#### Class Methods
 
-`:name, :position, :date, :cik`
+##### .recent
 
-* Transaction
+Find filings by a specific Date:
 
-`:filing_number, :code, :date, :reporting_owner, :form, :type, :modes, :shares, :price, :owned, :number, :owner_cik, :security_name, :deemed, :exercise, :nature, :derivative, :underlying_1, :exercised,	:underlying_2, :expires, :underlying_3`
+`SecQuery::Filing.recent(start: 0, count: 10, limit: 10)`
 
-* Filing
+Returns the most recent filings. Use start, count and limit to iterate through recent filings.
 
-`:cik, :title, :summary, :link, :term, :date, :file_id`
+#### Instance Methods
 
-Filings can are fetched a few different ways. Here are some of the supported
-methods:
+##### .content
 
-```rb
-# prints the links for the most recent filings
-SecQuery::Filing.recent do |filing|
-  p filing.link
-end
+`SecQuery::Filing.recent(start: 0, count: 1, limit: 1).first.content`
 
-# prints all of the links for cik 0000704051 (LEGG MASON, INC.)
-SecQuery::Filing.for_cik('0000704051') do |filing|
-  p filing.link
-end
-```
+Returns the actual text content of any Sec::Filing instance.
 
 ## To Whom It May Concern at the SEC
 

data/lib/sec_query.rb

@@ -4,7 +4,6 @@
 require 'active_support/all'
 require 'addressable/uri'
 require 'open-uri'
-require 'net/ftp'
 require 'rest-client'
 require 'rss'
 require 'nokogiri'
@@ -13,7 +12,5 @@ require 'rubygems'
 # internal
 require 'sec_query/entity'
 require 'sec_query/filing'
-require 'sec_query/relationship'
 require 'sec_query/sec_uri'
-require 'sec_query/transaction'
 require 'sec_query/version'

data/lib/sec_query/entity.rb

@@ -15,12 +15,8 @@ module SecQuery
       end
     end
 
-    def filings
-      Filing.find(@cik)
-    end
-
-    def transactions
-      Transaction.find(@cik)
+    def filings(args={})
+      Filing.find(@cik, 0, 80, args)
     end
 
     def self.query(url)

data/lib/sec_query/filing.rb

@@ -46,20 +46,11 @@ module SecQuery
     end
 
     def self.for_date(date, &blk)
-      ftp = Net::FTP.new('ftp.sec.gov')
-      ftp.login
-      file_name = ftp.nlst("edgar/daily-index/#{ date.to_sec_uri_format }*")[0]
-      ftp.close
-      open("ftp://ftp.sec.gov/#{ file_name }") do |file|
-        if file_name[-2..-1] == 'gz'
-          gz_reader = Zlib::GzipReader.new(file)
-          gz_reader.rewind
-          filings_for_index(gz_reader).each(&blk)
-        else
-          filings_for_index(file).each(&blk)
-        end
+      open(SecURI.for_date(date).to_s) do |file|
+        filings_for_index(file.read).each(&blk)
       end
-    rescue Net::FTPTempError
+    rescue OpenURI::HTTPError
+      return
     end
 
     def self.filings_for_index(index)
@@ -134,12 +125,14 @@ module SecQuery
       end
     end
 
-    def self.find(cik, start = 0, count = 80)
+    def self.find(cik, start = 0, count = 80, args={})
       temp = {}
       temp[:url] = SecURI.browse_edgar_uri({cik: cik})
       temp[:url][:action] = :getcompany
       temp[:url][:start] = start
       temp[:url][:count] = count
+      args.each {|k,v| temp[:url][k]=v }
+      
       response = Entity.query(temp[:url].output_atom.to_s)
       document = Nokogiri::HTML(response)
       parse(cik, document)

data/lib/sec_query/sec_uri.rb

@@ -25,9 +25,9 @@ module SecQuery
 
     def self.for_date(date)
       instance = SecURI.new
-      instance.scheme = 'ftp'
-      instance.host = 'ftp.sec.gov'
-      instance.path = "edgar/daily-index/#{ date.to_sec_uri_format }"
+      instance.host = 'www.sec.gov'
+      instance.scheme = 'https'
+      instance.path = "Archives/edgar/daily-index/#{ date.to_sec_uri_format }"
       instance
     end
 

data/lib/sec_query/version.rb

@@ -1,5 +1,5 @@
 # encoding: UTF-8
 # => SecQuery::VERSION
 module SecQuery
-  VERSION = "1.2.0"
+  VERSION = "1.3.0"
 end

data/sec_query.gemspec

@@ -12,7 +12,7 @@ Gem::Specification.new do |s|
   s.license       = 'MIT'
   s.homepage    = 'https://github.com/tyrauber/sec_query'
   s.summary     = 'A ruby gem for querying the United States Securities and Exchange Commission Edgar System.'
-  s.description = 'Search for company or person, by name, symbol or Central Index Key (CIK), and retrieve relationships, transactions and filings.'
+  s.description = 'Search for company or person, by name, symbol or Central Index Key (CIK), and retrieve filings.'
 
   s.rubyforge_project = 'sec_query'
   s.files         = `git ls-files`.split("\n")
@@ -20,12 +20,12 @@ Gem::Specification.new do |s|
   s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
   s.require_paths = ['lib']
 
-  s.add_development_dependency 'rspec', '~> 3.4'
-  s.add_development_dependency 'vcr', '~> 3.0.1'
-  s.add_development_dependency 'webmock', '~> 1.24.6'
-  s.add_development_dependency 'rubocop', '~> 0.40'
-  s.add_development_dependency 'byebug', '~> 9.0.5'
-  s.add_runtime_dependency 'rest-client', '~> 1.8'
-  s.add_runtime_dependency 'addressable', '~> 2.3'
-  s.add_runtime_dependency 'nokogiri', '~> 1.6'
+  s.add_development_dependency 'rspec', '~> 3.5'
+  s.add_development_dependency 'vcr', '~> 3.0'
+  s.add_development_dependency 'webmock', '~> 2.3'
+  s.add_development_dependency 'rubocop', '~> 0.47'
+  s.add_development_dependency 'byebug', '~> 9.0'
+  s.add_runtime_dependency 'rest-client', '~> 2.0'
+  s.add_runtime_dependency 'addressable', '~> 2.5'
+  s.add_runtime_dependency 'nokogiri', '~> 1.7'
 end

data/spec/sec_query/filing_spec.rb

@@ -13,49 +13,24 @@ describe SecQuery::Filing do
       .to eq('https://www.sec.gov/cgi-bin/browse-edgar?CIK=testing&action=getcompany&company&count=100&output=atom&owner=include&start=0')
   end
 
-  describe '::filings_for_index' do
-    let(:index) { File.read('./spec/support/idx/test.idx') }
-    let(:filing1) { SecQuery::Filing.filings_for_index(index).first }
+  describe '::for_date', vcr: { cassette_name: 'idx' } do
+    let(:index) { SecQuery::Filing.for_date(Date.parse('20161230')) }
 
     it 'parses all of the filings' do
-      expect(SecQuery::Filing.filings_for_index(index).count).to eq(4628)
+      expect(index.count).to eq(2554)
     end
 
     it 'correctly parses out the link' do
-      expect(filing1.link)
-        .to eq('https://www.sec.gov/Archives/edgar/data/38723/0000038723-14-000001.txt')
+      expect(index.first.link)
+        .to match(/https:\/\/www.sec.gov\/Archives\/edgar\/data\//)
     end
 
     it 'correctly parses out the cik' do
-      expect(filing1.cik).to eq('38723')
+      expect(index.first.cik).to eq('1605941')
     end
 
     it 'correctly parses out the term' do
-      expect(filing1.term).to eq('424B3')
-    end
-  end
-
-  describe '::for_date' do
-    let(:filings) do
-      [].tap do |filings|
-        SecQuery::Filing.for_date(Date.parse('20121123')) do |f|
-          filings << f
-        end
-      end
-    end
-
-    let(:filing1) { filings.first }
-
-    it 'correctly parses a filing from a zipped company index' do
-      expect(filing1.term).to eq('4')
-      expect(filing1.cik).to eq('1551138')
-      expect(filing1.date).to eq(Date.parse('20121123'))
-      expect(filing1.link)
-        .to eq('https://www.sec.gov/Archives/edgar/data/1551138/0001144204-12-064668.txt')
-    end
-
-    it 'returns nil if for_date is run on a market close day' do
-      expect(SecQuery::Filing.for_date(Date.parse('20120101'))).to eq(nil)
+      expect(index.first.term).to eq('N-CSR')
     end
   end
 
@@ -86,60 +61,44 @@ describe SecQuery::Filing do
   end
 
   describe "::find" do
-    let(:query){{
-     name: "JOBS STEVEN P", :cik => "0001007844",
-      relationships:[
-        {cik: "0000320193", name: "APPLE INC"},
-        {cik: "0001001039", name: "WALT DISNEY CO/"},
-        {cik: "0001002114", name: "PIXAR \\CA\\"}
-      ],
-      transactions:[
-        {filing_number: "0001181431-07-052839", reporting_owner: "APPLE INC", shares:120000.0},
-        {filing_number: "0001181431-07-052839", reporting_owner: "APPLE INC", shares: 40000.0},
-        {filing_number: "0001181431-07-052839", reporting_owner: "APPLE INC", shares: 40000.0},
-        {filing_number: "0001181431-07-052839", reporting_owner: "APPLE INC", shares: 40000.0},
-        {filing_number: "0001181431-06-028746", reporting_owner: "WALT DISNEY CO/", shares: 138000004.0},
-        {filing_number: "0001356184-06-000008", reporting_owner: "PIXAR \\CA\\", shares: 60000002.0},
-        {filing_number: "0001181431-06-019230", reporting_owner: "APPLE COMPUTER INC", shares: 4573553.0},
-        {filing_number: "0001181431-06-028747", reporting_owner: "WALT DISNEY CO/", shares: 0.0}
-      ],
-      filings:[
-        {cik: "0001007844", file_id: "0001181431-07-052839"},
-        {cik: "0001007844", file_id: "0001356184-06-000008"},
-        {cik: "0001007844", file_id: "0001193125-06-103741"},
-        {cik: "0001007844", file_id: "0001181431-06-028747"},
-        {cik: "0001007844", file_id: "0001181431-06-028746"},
-        {cik: "0001007844", file_id: "0001181431-06-019230"},
-        {cik: "0001007844", file_id: "0001193125-06-019727"},
-        {cik: "0001007844", file_id: "0001104659-03-004723"}
-      ]
-    }}
-
-    let(:entity) {SecQuery::Entity.find(query[:cik])}
-
-    describe "Filings", vcr: { cassette_name: "Steve Jobs"} do
-      it "should respond to filings" do
-        entity.should respond_to(:filings)
-        entity.filings.should be_kind_of(Array)
+    shared_examples_for "it found filings" do
+      it "should return an array of filings" do
+        filings.should be_kind_of(Array)
+      end
+
+      it "the filings should be valid" do
+        is_valid_filing?(filings.first)
       end
+    end
+
+    let(:cik){"0000320193"}
+    
+    context "when querying by cik" do
+      let(:filings){ SecQuery::Filing.find(cik) }
 
-      it "should be valid filing" do
-        is_valid_filing?(entity.filings.first)
+      describe "Filings", vcr: { cassette_name: "Steve Jobs"} do
+        it_behaves_like "it found filings"
       end
     end
+    
+    context "when querying cik and by type param" do
+      let(:filings){ SecQuery::Filing.find(cik, 0, 40, { type: "10-K" }) }
 
+      describe "Filings", vcr: { cassette_name: "Steve Jobs"} do
+        it_behaves_like "it found filings"
+
+        it "should only return filings of type" do
+          filings.first.term.should == "10-K"
+        end
+      end
+    end
+    
     describe '#content', vcr: { cassette_name: 'content' } do
+      let(:index) { SecQuery::Filing.for_date(Date.parse('20161230')) }
+      
       it 'returns content of the filing by requesting the link' do
-        f = Filing.new(
-          cik: 123,
-          title: 'test filing title',
-          summary: 'test filing',
-          link: 'https://www.sec.gov/Archives/edgar/data/1572871/000114036114019536/0001140361-14-019536.txt',
-          term: '4',
-          date: Date.today,
-          file_id: 1
-        )
-        expect(f.content).to eq(File.read('./spec/support/filings/filing.txt'))
+        f = Filing.new(link: index.first.link)
+        expect(f.content).to match(/^(<SEC-DOCUMENT>)/)
       end
     end
   end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: sec_query
 version: !ruby/object:Gem::Version
-  version: 1.2.0
+  version: 1.3.0
 platform: ruby
 authors:
 - Ty Rauber
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-09-13 00:00:00.000000000 Z
+date: 2017-01-31 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rspec
@@ -16,114 +16,114 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '3.4'
+        version: '3.5'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '3.4'
+        version: '3.5'
 - !ruby/object:Gem::Dependency
   name: vcr
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 3.0.1
+        version: '3.0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 3.0.1
+        version: '3.0'
 - !ruby/object:Gem::Dependency
   name: webmock
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 1.24.6
+        version: '2.3'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 1.24.6
+        version: '2.3'
 - !ruby/object:Gem::Dependency
   name: rubocop
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.40'
+        version: '0.47'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.40'
+        version: '0.47'
 - !ruby/object:Gem::Dependency
   name: byebug
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 9.0.5
+        version: '9.0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 9.0.5
+        version: '9.0'
 - !ruby/object:Gem::Dependency
   name: rest-client
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.8'
+        version: '2.0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.8'
+        version: '2.0'
 - !ruby/object:Gem::Dependency
   name: addressable
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.3'
+        version: '2.5'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.3'
+        version: '2.5'
 - !ruby/object:Gem::Dependency
   name: nokogiri
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.6'
+        version: '1.7'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.6'
+        version: '1.7'
 description: Search for company or person, by name, symbol or Central Index Key (CIK),
-  and retrieve relationships, transactions and filings.
+  and retrieve filings.
 email:
 - tyrauber@mac.com
 executables: []
@@ -138,18 +138,13 @@ files:
 - lib/sec_query.rb
 - lib/sec_query/entity.rb
 - lib/sec_query/filing.rb
-- lib/sec_query/relationship.rb
 - lib/sec_query/sec_uri.rb
-- lib/sec_query/transaction.rb
 - lib/sec_query/version.rb
 - sec_query.gemspec
 - spec/sec_query/entity_spec.rb
 - spec/sec_query/filing_spec.rb
 - spec/sec_query/sec_uri_spec.rb
-- spec/sec_query/transaction_spec.rb
 - spec/spec_helper.rb
-- spec/support/filings/filing.txt
-- spec/support/idx/test.idx
 - spec/support/vcr.rb
 homepage: https://github.com/tyrauber/sec_query
 licenses:
@@ -180,8 +175,5 @@ test_files:
 - spec/sec_query/entity_spec.rb
 - spec/sec_query/filing_spec.rb
 - spec/sec_query/sec_uri_spec.rb
-- spec/sec_query/transaction_spec.rb
 - spec/spec_helper.rb
-- spec/support/filings/filing.txt
-- spec/support/idx/test.idx
 - spec/support/vcr.rb