ib-ruby 0.8.5 → 0.9.0

data/.gitignore

@@ -20,6 +20,7 @@ tmtags
 *.iml
 
 ## PROJECT::GENERAL
+config/flex.yml
 coverage
 data
 rdoc

data/HISTORY

@@ -201,3 +201,7 @@
 == 0.8.5 / 2012-08-11
 
 * Complete Rails Engine, TWS 929 support
+
+== 0.9.0 / 2012-10-31
+
+* Support for API 967 final

data/README.md

@@ -70,7 +70,8 @@ other API implementations. The choice is yours.
     | 0.5.21      |    918-920  |    965       |
     | 0.6.1       |    921-923  |    966       |
     | 0.7.1       |    924-925  |    966       |
-    | 0.8.1+      |    926-929  |    967       |
+    | 0.8.1       |    926-930  |    967 beta  |
+    | 0.9.0+      |    931-932  |    967 final |
 
 4. Start Interactive Broker's Trader Work Station or Gateway before your code
    attempts to connect to it. Note that TWS and Gateway listen to different ports,
@@ -119,23 +120,23 @@ See `lib/ib/messages` for a full list of supported incoming/outgoing messages
 and their attributes. The original TWS docs and code samples can also be found
 in `misc` directory.
 
-Sample scripts in `example` directory demonstrate common ib-ruby use cases. Examples 
-show you how to access account info, print real time quotes, retrieve historic or 
+Sample scripts in `example` directory demonstrate common ib-ruby use cases. Examples
+show you how to access account info, print real time quotes, retrieve historic or
 fundamental data, request options calculations, place, list, and cancel orders.
-You may also want to look into `spec/integration` directory for more scenarios, 
+You may also want to look into `spec/integration` directory for more scenarios,
 use cases and examples of handling IB messages.
 
 ## RAILS INTEGRATION:
 
 This gem has two operating modes: standalone and Rails-engine. If you require it in a
 Rails environment, it loads Rails engine automatically. Otherwise, it does not load any
-Rails integration. 
+Rails integration.
 
 To add ib-ruby to your Rails 3 project, follow these steps:
 
 Add to your Gemfile:
 ``` ruby
-gem 'ib-ruby', '~>0.8'
+gem 'ib-ruby', '~>0.9'
 ```
 Add the require to your config/application.rb:
 ``` ruby
@@ -152,12 +153,12 @@ Now run:
 
 This will install ib-ruby gem and copy its migrations into your Rails apps migrations.
 
-You can now use or modify IB models, develop controllers and views for them in your Rails app. 
+You can now use or modify IB models, develop controllers and views for them in your Rails app.
 
 ## DB BACKEND:
 
-Even if you don't use Rails, you can still take advantage of its data persistance layer 
-(ActiveRecord ORM). In order to use data persistance, you have to set up the database 
+Even if you don't use Rails, you can still take advantage of its data persistance layer
+(ActiveRecord ORM). In order to use data persistance, you have to set up the database
 (SQLite recommended for simplicity) and run migrations located at gems 'db/migrate' folder.
 It is recommended that you use a gem like [standalone_migrations](https://github.com/thuss/standalone-migrations) for this.
 
@@ -169,25 +170,25 @@ You further need to:
 ```
 Only require 'ib' AFTER you've connected to DB, otherwise your Models will not
 inherit from ActiveRecord::Base and won't be persistent. If you are using Rails,
-you don't need IB::DB.connect part, Rails will take care of it for you.  
+you don't need IB::DB.connect part, Rails will take care of it for you.
 
 Now, all your IB Models are just ActiveRecords and you can save them to DB just
 like you would with Rails models.
 
 ## RUNNING TESTS:
 
-The gem comes with a spec suit that may be used to test ib-ruby compatibility with your 
+The gem comes with a spec suit that may be used to test ib-ruby compatibility with your
 specific TWS/Gateway installation. Please read 'spec/Readme.md' for more details about
 running specs.
 
 ## RUBY VERSION COMPATIBILITY:
 
-The library is continuously tested with JRuby 1.6.7 (ruby-1.8.7-p357-compatible mode) and 
-JRuby head (ruby-1.9.3-p203-compatible mode). It is not JRuby-specific though, as it is currently used in a some MRI Ruby based projects. If there are any problems in any mode 
+The library is continuously tested with JRuby 1.6.7 (ruby-1.8.7-p357-compatible mode) and
+JRuby head (ruby-1.9.3-p203-compatible mode). It is not JRuby-specific though, as it is currently used in a some MRI Ruby based projects. If there are any problems in any mode
 for either JRuby or MRI, please report an [issue](https://github.com/ib-ruby/ib-ruby/issues/new)
 and we will work on it.
 
-Please keep in mind that when using Ruby 1.8.7, you need to either explicitly: 
+Please keep in mind that when using Ruby 1.8.7, you need to either explicitly:
 ``` ruby
     require 'rubygems'
     require 'ib'

data/Rakefile

@@ -39,7 +39,6 @@ rescue LoadError => e
   puts "gem install standalone_migrations to get db:migrate:* tasks! (Error: #{e})"
 end
 
-
 # rake db:redo DB=test"
 namespace :db do
   desc "Remake db from scratch: $ rake db:redo DB=test"

data/VERSION

@@ -1 +1 @@
-0.8.5
+0.9.0

data/bin/scaffold.rb

@@ -1,6 +1,6 @@
 #!/usr/bin/env ruby
 #
-# This script converts given migration file into rails scaffold command
+# This script converts given migration file into a Rails scaffold command
 
 output = STDOUT
 ARGV.each do |file_name|

data/db/migrate/101_add_ib_executions.rb

@@ -4,18 +4,20 @@ class AddIbExecutions < ActiveRecord::Migration
     create_table(:ib_executions) do |t|
       # TWS orders have fixed local_id of 0 AND client id of 0
       t.references :order
-      t.integer :local_id #    TWS orders have a fixed order id of 0
-      t.integer :client_id #   Id of the client that placed the order
-      t.integer :perm_id #     Permanent order id, remains the same over TWS sessions
-      t.string :order_ref #    Order reference
-      t.string :exec_id #      Unique order execution id
+      t.integer :local_id #     TWS orders have a fixed order id of 0
+      t.integer :client_id #    Id of the client that placed the order
+      t.integer :perm_id #      Permanent order id, remains the same over TWS sessions
+      t.string :order_ref #     Order reference
+      t.string :ev_rule #       Australian products only
+      t.float :ev_multiplier #  Australian products only
+      t.string :exec_id #       Unique order execution id
       t.string :side, :limit => 1 # Was the transaction a buy or a sale: BOT|SLD
       t.integer :quantity #              The number of shares filled
       t.integer :cumulative_quantity # Cumulative quantity
-      t.float :price #         double: The order execution price
-      t.float :average_price # double: Average price (for all executions?)
-      t.string :exchange #     Exchange that executed the order
-      t.string :account_name # The customer account number
+      t.float :price #           double: The order execution price
+      t.float :average_price #   double: Average price (for all executions?)
+      t.string :exchange #       Exchange that executed the order
+      t.string :account_name #   The customer account number
       t.boolean :liquidation, :limit => 1 # This position to be liquidated last should the need arise
       t.string :time, :limit => 18 # String! The order execution time
       t.timestamps
@@ -24,4 +26,7 @@ class AddIbExecutions < ActiveRecord::Migration
 end
 
 __END__
-rails generate scaffold execution order_id:integer local_id:integer client_id:integer perm_id:integer order_ref:string exec_id:string side:string quantity:integer cumulative_quantity:integer price:float average_price:float exchange:string account_name:string liquidation:boolean time:string 
+rails generate scaffold execution order_id:integer local_id:integer client_id:integer 
+perm_id:integer order_ref:string ev_rule:string ev_multiplier:float exec_id:string 
+side:string quantity:integer cumulative_quantity:integer price:float average_price:float 
+exchange:string account_name:string liquidation:boolean time:string 

data/db/migrate/161_add_ib_contract_details.rb

@@ -34,18 +34,25 @@ class AddIbContractDetails < ActiveRecord::Migration
       t.boolean :puttable, :limit => 1 # Can be sold back to the issuer under certain conditions
       t.boolean :convertible, :limit => 1 # Can be converted to stock under certain conditions.
       t.boolean :next_option_partial, :limit => 1 # # only if bond has embedded options.
+      t.string :valid_next_option_date # Bonds only
+      t.string :valid_next_option_type # Bonds only
+      t.string :valid_next_option_partial # Bonds only
+      t.string :ev_rule # Australian non-currency products only
+      t.float :ev_multiplier # Australian non-currency products only
+      t.text :sec_id_list
       t.timestamps
     end
   end
 end
 
 __END__
-rails generate scaffold contract_detail contract_id:integer market_name:string 
- trading_class:string min_tick:float price_magnifier:integer order_types:string 
- valid_exchanges:string under_con_id:integer long_name:string contract_month:string 
- industry:string category:string subcategory:string time_zone:string trading_hours:string 
- liquid_hours:string cusip:string ratings:string desc_append:string bond_type:string 
- coupon_type:string coupon:float maturity:string issue_date:string next_option_date:string 
- next_option_type:string notes:string callable:boolean puttable:boolean convertible:boolean 
- next_option_partial:boolean
+rails generate scaffold contract_detail contract_id:integer market_name:string
+ trading_class:string min_tick:float price_magnifier:integer order_types:string
+ valid_exchanges:string under_con_id:integer long_name:string contract_month:string
+ industry:string category:string subcategory:string time_zone:string trading_hours:string
+ liquid_hours:string cusip:string ratings:string desc_append:string bond_type:string
+ coupon_type:string coupon:float maturity:string issue_date:string next_option_date:string
+ next_option_type:string notes:string callable:boolean puttable:boolean convertible:boolean
+ next_option_partial:boolean valid_next_option_date:string valid_next_option_type:string
+ valid_next_option_partial:string ev_rule:string ev_multiplier:float sec_id_list:text
 

data/db/schema.rb

@@ -70,12 +70,18 @@ ActiveRecord::Schema.define(:version => 171) do
     t.string   "next_option_date"
     t.string   "next_option_type"
     t.string   "notes"
-    t.boolean  "callable",            :limit => 1
-    t.boolean  "puttable",            :limit => 1
-    t.boolean  "convertible",         :limit => 1
-    t.boolean  "next_option_partial", :limit => 1
-    t.datetime "created_at",                       :null => false
-    t.datetime "updated_at",                       :null => false
+    t.boolean  "callable",                  :limit => 1
+    t.boolean  "puttable",                  :limit => 1
+    t.boolean  "convertible",               :limit => 1
+    t.boolean  "next_option_partial",       :limit => 1
+    t.string   "valid_next_option_date"
+    t.string   "valid_next_option_type"
+    t.string   "valid_next_option_partial"
+    t.string   "ev_rule"
+    t.float    "ev_multiplier"
+    t.text     "sec_id_list"
+    t.datetime "created_at",                             :null => false
+    t.datetime "updated_at",                             :null => false
   end
 
   create_table "ib_contracts", :force => true do |t|
@@ -105,6 +111,8 @@ ActiveRecord::Schema.define(:version => 171) do
     t.integer  "client_id"
     t.integer  "perm_id"
     t.string   "order_ref"
+    t.string   "ev_rule"
+    t.float    "ev_multiplier"
     t.string   "exec_id"
     t.string   "side",                :limit => 1
     t.integer  "quantity"

data/lib/ib/connection.rb

@@ -1,6 +1,7 @@
 require 'thread'
 require 'ib/socket'
 require 'ib/logger'
+require 'ib/messages'
 
 module IB
   # Encapsulates API connection to TWS or Gateway
@@ -18,8 +19,8 @@ module IB
                        :received => true, # Keep all received messages in a @received Hash
                        :logger => nil,
                        :client_id => nil, # Will be randomly assigned
-                       :client_version => 57, # 48, # 57 = can receive commissionReport message
-                       :server_version => 60 # 53? Minimal server version required
+                       :client_version => IB::Messages::CLIENT_VERSION,
+                       :server_version => IB::Messages::SERVER_VERSION
     }
 
     # Singleton to make active Connection universally accessible as IB::Connection.current
@@ -68,7 +69,7 @@ module IB
       server[:client_version] = options[:client_version]
       server[:server_version] = socket.read_int
       if server[:server_version] < options[:server_version]
-        error "TWS version #{server[:server_version]}, #{options[:server_version]} required."
+        error "Server version #{server[:server_version]}, #{options[:server_version]} required."
       end
       server[:remote_connect_time] = socket.read_string
       server[:local_connect_time] = Time.now()
@@ -80,7 +81,7 @@ module IB
       socket.write_data server[:client_id]
 
       @connected = true
-      log.info "Connected to server, version: #{server[:server_version]}, connection time: " +
+      log.info "Connected to server, ver: #{server[:server_version]}, connection time: " +
                    "#{server[:local_connect_time]} local, " +
                    "#{server[:remote_connect_time]} remote."
 

data/lib/ib/errors.rb

@@ -13,6 +13,9 @@ module IB
   class LoadError < LoadError
   end
 
+  class FlexError < RuntimeError
+  end
+
 end # module IB
 
 # Patching Object with universally accessible top level error method. 
@@ -29,6 +32,8 @@ def error message, type=:standard, backtrace=nil
     IB::SymbolError.new message
   when :load
     IB::LoadError.new message
+  when :flex
+    IB::FlexError.new message
   end
   e.set_backtrace(backtrace) if backtrace
   raise e

data/lib/ib/flex.rb

@@ -0,0 +1,105 @@
+require 'net/http'
+require 'net/https'
+require 'xmlsimple'
+
+module IB
+
+  # FLEX is a web-based service from IB that helps you to retrieve your activity,
+  # trades and positions. It is working independently from TWS or Gateway, using your
+  # internet connection directly. See /misc/flex for extended FLEX documentation.
+  #
+  # In order to use this service, activate it and configure your token first.
+  # Your Token is located at Account Management->Reports->Delivery Settings->Flex Web Service.
+  # You need to activate Flex Web Service and generate new token(s) there.
+  # Your Flex Query Ids are in Account Management->Reports->Activity->Flex Queries.
+  # Create new Flex query and make sure to set its output format to XML.
+  #
+  # IB::Flex object incapsulates a single pre-defined Flex query.
+  class Flex
+    class << self
+      attr_accessor :token, :uri
+
+      # By default, uri is a well known FLEX Web Service URI
+      def uri
+        @uri || 'https://www.interactivebrokers.com/Universal/servlet/FlexStatementService.SendRequest'
+      end
+    end
+
+    #  Create new Flex query with options:  
+    #    :token => 1111111111111111111111111111111111  # CHANGE to your actual token!
+    #    :query_id => 11111                            # CHANGE to actual query id!
+    #    :format => :xml (default) / :csv
+    #    :verbose => true / false (default)
+    def initialize opts
+      @query_id = opts[:query_id]
+      @token = opts[:token] || Flex.token
+      @format = opts[:format] || :xml
+      @verbose = !!opts[:verbose]
+      yield self if block_given?
+    end
+
+    # Run a pre-defined Flex query against IB Flex Web Service
+    # Returns a (parsed) report or raises FlexError in case of problems
+    def run
+      # Initiate FLEX request at a known FLEX Web Service URI
+      resp = get_content Flex.uri, :t => @token, :q => @query_id, :v => 3
+      error("#{resp['ErrorCode']}: #{resp['ErrorMessage']}", :flex) if resp['Status'] == 'Fail'
+
+      reference_code = resp['ReferenceCode']
+      report_uri = resp['Url']
+
+      # Retrieve the FLEX report
+      report = nil
+      until report do
+        report = get_content(report_uri, :t => @token, :q => reference_code, :v => 3,
+                               :text_ok => @format != :xml)
+
+          # If Status is specified, returned xml contains only error message, not actual report
+          if report.is_a?(Hash) && report['Status'] == 'Fail'
+            error_code = report['ErrorCode'].to_i
+            error_message = "#{error_code}: #{report['ErrorMessage']}"
+
+            case error_code
+            when 1001..1009, 1018, 1019, 1021
+              # Report is just not ready yet, wait and retry
+              puts error_message if @verbose
+              report = nil
+              sleep 1
+            else # Fatal error
+              error error_message, :flex
+            end
+          end
+      end
+      report
+    end
+
+    # Helper method to get (and parse XML) responses from IB Flex Web Service
+    def get_content address, fields
+      text_ok = fields.delete(:text_ok)
+      resp = get address, fields
+      if resp.content_type == 'text/xml'
+        XmlSimple.xml_in(resp.body, :ForceArray => false)
+      else
+        error("Expected xml, got #{resp.content_type}", :flex) unless text_ok
+        resp.body
+      end
+    end
+
+    # Helper method to get raw responses from IB Flex Web Service
+    def get address, fields
+      uri = URI("#{address}?" + fields.map { |k, v| "#{k}=#{URI.encode(v.to_s)}" }.join('&'))
+
+      server = Net::HTTP.new(uri.host, uri.port)
+      server.use_ssl = (uri.scheme == 'https')
+      server.verify_mode = OpenSSL::SSL::VERIFY_NONE if server.use_ssl? # Avoid OpenSSL failures
+
+      resp = server.start do |http|
+        req = Net::HTTP::Get.new(uri.request_uri)
+        http.request(req)
+      end
+      error("URI responded with #{resp.code}", :flex) unless resp.code.to_i == 200
+      resp
+    end
+
+  end
+end

data/lib/ib/messages.rb

@@ -1,8 +1,88 @@
 module IB
   module Messages
+  	# This gem supports incoming/outgoing IB messages compatible with the following
+  	# IB client/server versions:
+    CLIENT_VERSION = 59 # 59? Maximal client version implemented
+    SERVER_VERSION = 60 # 38? 53? 62? Minimal server version required
   end
 end
 
 require 'ib/messages/outgoing'
 require 'ib/messages/incoming'
 
+__END__
+    // Client version history
+    //
+    //  6 = Added parentId to orderStatus
+    //  7 = The new execDetails event returned for an order filled status and reqExecDetails
+    //     Also market depth is available.
+    //  8 = Added lastFillPrice to orderStatus() event and permId to execution details
+    //  9 = Added 'averageCost', 'unrealizedPNL', and 'unrealizedPNL' to updatePortfolio event
+    // 10 = Added 'serverId' to the 'open order' & 'order status' events.
+    //      We send back all the API open orders upon connection.
+    //      Added new methods reqAllOpenOrders, reqAutoOpenOrders()
+    //      Added FA support - reqExecution has filter.
+    //                       - reqAccountUpdates takes acct code.
+    // 11 = Added permId to openOrder event.
+    // 12 = requsting open order attributes ignoreRth, hidden, and discretionary
+    // 13 = added goodAfterTime
+    // 14 = always send size on bid/ask/last tick
+    // 15 = send allocation description string on openOrder
+    // 16 = can receive account name in account and portfolio updates, and fa params in openOrder
+    // 17 = can receive liquidation field in exec reports, and notAutoAvailable field in mkt data
+    // 18 = can receive good till date field in open order messages, and request intraday backfill
+    // 19 = can receive rthOnly flag in ORDER_STATUS
+    // 20 = expects TWS time string on connection after server version >= 20.
+    // 21 = can receive bond contract details.
+    // 22 = can receive price magnifier in version 2 contract details message
+    // 23 = support for scanner
+    // 24 = can receive volatility order parameters in open order messages
+    // 25 = can receive HMDS query start and end times
+    // 26 = can receive option vols in option market data messages
+    // 27 = can receive delta neutral order type and delta neutral aux price in place order version 20: API 8.85
+    // 28 = can receive option model computation ticks: API 8.9
+    // 29 = can receive trail stop limit price in open order and can place them: API 8.91
+    // 30 = can receive extended bond contract def, new ticks, and trade count in bars
+    // 31 = can receive EFP extensions to scanner and market data, and combo legs on open orders
+    //    ; can receive RT bars 
+    // 32 = can receive TickType.LAST_TIMESTAMP
+    //    ; can receive "whyHeld" in order status messages 
+    // 33 = can receive ScaleNumComponents and ScaleComponentSize is open order messages 
+    // 34 = can receive whatIf orders / order state
+    // 35 = can receive contId field for Contract objects
+    // 36 = can receive outsideRth field for Order objects
+    // 37 = can receive clearingAccount and clearingIntent for Order objects
+    // 38 = can receive multiplier and primaryExchange in portfolio updates
+    //    ; can receive cumQty and avgPrice in execution
+    //    ; can receive fundamental data
+    //    ; can receive underComp for Contract objects
+    //    ; can receive reqId and end marker in contractDetails/bondContractDetails
+    //    ; can receive ScaleInitComponentSize and ScaleSubsComponentSize for Order objects
+    // 39 = can receive underConId in contractDetails
+    // 40 = can receive algoStrategy/algoParams in openOrder
+    // 41 = can receive end marker for openOrder
+    //    ; can receive end marker for account download
+    //    ; can receive end marker for executions download
+    // 42 = can receive deltaNeutralValidation
+    // 43 = can receive longName(companyName)
+    //    ; can receive listingExchange
+    //    ; can receive RTVolume tick
+    // 44 = can receive end market for ticker snapshot
+    // 45 = can receive notHeld field in openOrder
+    // 46 = can receive contractMonth, industry, category, subcategory fields in contractDetails
+    //    ; can receive timeZoneId, tradingHours, liquidHours fields in contractDetails
+    // 47 = can receive gamma, vega, theta, undPrice fields in TICK_OPTION_COMPUTATION
+    // 48 = can receive exemptCode in openOrder
+    // 49 = can receive hedgeType and hedgeParam in openOrder
+    // 50 = can receive optOutSmartRouting field in openOrder
+    // 51 = can receive smartComboRoutingParams in openOrder
+    // 52 = can receive deltaNeutralConId, deltaNeutralSettlingFirm, deltaNeutralClearingAccount and deltaNeutralClearingIntent in openOrder
+    // 53 = can receive orderRef in execution
+    // 54 = can receive scale order fields (PriceAdjustValue, PriceAdjustInterval, ProfitOffset, AutoReset, 
+    //      InitPosition, InitFillQty and RandomPercent) in openOrder
+    // 55 = can receive orderComboLegs (price) in openOrder
+    // 56 = can receive trailingPercent in openOrder
+    // 57 = can receive commissionReport message
+    // 58 = can receive CUSIP/ISIN/etc. in contractDescription/bondContractDescription
+    // 59 = can receive evRule, evMultiplier in contractDescription/bondContractDescription/executionDetails
+    //      can receive multiplier in executionDetails