bankscrap 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 7567c87bd47ae9505b4abaa485cae8442dd41b40
+  data.tar.gz: 9e5a1bdc25a930b5e8340bd65be5f1302d0a7da0
+SHA512:
+  metadata.gz: 63da5ae6218c554d0ce6d745cdf252f18ecd2bfa137bd8a06d24018d108802ba02907d97ae9e986ea89de9274e683a5f15b54fdf9696e4258307746db87edfab
+  data.tar.gz: 542a627dfcc461df1ad682ffed3af835345e8b85b8c521dede4ce03690fba24f204c97f671057f023b1897a2ad5a0b5b1004c828c07853c5c377b1dfe48ad91a

data/README.md

@@ -0,0 +1,147 @@
+# Bankscrap
+
+[![](http://188.166.39.57:3000/badge.svg)](http://188.166.39.57:3000)
+
+Ruby gem to extract account balance and transactions from banks. You can use it either as command line tool or as a library.
+
+Feel free to contribute and add your bank if it isn't supported.
+
+## Supported banks
+
+* **BBVA**: [bankscrap-bbva](https://github.com/bankscrap/bankscrap-bbva)
+* **ING Direct**: [bankscrap-ing](https://github.com/bankscrap/bankscrap-ing)
+* **Banc Sabadell** (WIP): [bankscrap-banc-sabadell](https://github.com/bankscrap/bankscrap-banc-sabadell)
+
+Interested in any other bank? Open a new Issue and we'll try to help.
+
+## Background and motivation
+
+Most banks don't offer public APIs and the only way to access your data (balance and transactions) is through their websites... and most bank websites are a f*cking nightmare.
+
+We are developers and we don't want to waste time doing things we are able to automate. Having to perform 20 clicks in an awful website just to check how much money we have is not something we like.
+
+There are two approaches to solve this problem:
+- Web scraping on the bank's site.
+- Reverse engineering the bank's mobile app or the bank's single page web app (if they have one) to use the same API.
+
+BankScrap uses both methods depending on the bank.
+
+## Requirements
+
+Some banks needs a JavaScript runtime in order to work. So if you find an error like "Could not find JavaScript runtime" try to install one. It has been tested with nodejs.
+
+## Installation
+
+Installation from RubyGems:
+
+    # BBVA
+    gem install bankscrap-bbva
+
+    # ING
+    gem install bankscrap-ing
+
+Or, if you're using Bundler, just add the following to your Gemfile:
+
+    # BBVA
+    gem 'bankscrap-bbva'
+
+    # ING
+    gem 'bankscrap-ing'
+
+Note that you only need to install the gem for your selected bank – the main gem (`bankscrap`) will be installed as a dependency.
+
+## Usage
+
+### From terminal
+#### Bank account balance
+
+###### BBVA
+
+    $ bankscrap balance YourBank --user YOUR_BANK_USER --password YOUR_BANK_PASSWORD
+
+###### ING Direct
+ING needs one more argument: your birthday.
+
+    $ bankscrap balance ING --user YOUR_DNI --password YOUR_PASSWORD --extra=birthday:01/01/1980
+
+Replace 01/01/1980 with your actual birthday.
+
+#### Transactions for last 30 days
+###### BBVA
+
+    $ bankscrap transactions BBVA --user YOUR_BBVA_USER --password YOUR_BBVA_PASSWORD
+
+###### ING Direct
+
+    $ bankscrap transactions ING --user YOUR_DNI --password YOUR_PASSWORD --extra=birthday:01/01/1980
+
+#### Transactions with date range
+
+    $ bankscrap transactions YourBank --user YOUR_BANK_USER --password YOUR_BANK_PASSWORD --extra=from:01-01-2015 to:01-02-2015
+
+---
+
+By default it will use your first bank account, if you want to fetch transactions for a different account you can use this syntax:
+
+    $ bankscrap transactions YourBank your_iban --user YOUR_DNI --password YOUR_PASSWORD
+
+If you don't want to pass your user and password everytime you can define them in your .bash_profile by adding:
+
+    export BANK_SCRAP_USER=YOUR_BANK_USER
+    export BANK_SCRAP_PASSWORD=YOUR_BANK_PASSWORD
+
+### From Ruby code
+
+You can also use this gem from your own app as library. To do so first you must initialize a BankScrap::Bank object
+
+
+```ruby
+require 'bankscrap-bbva'
+# BBVA
+bbva = Bankscrap::BBVA::Bank.new(YOUR_BBVA_USER, YOUR_BBVA_PASSWORD)
+# ING
+ing = Bankscrap::ING::Bank.new(YOUR_DNI, YOUR_ING_PASSWORD, extra_args: {"birthday" => "dd/mm/yyyy"})
+```
+
+
+The initialize method will automatically login and fetch your bank accounts
+
+You can now explore your bank accounts accounts:
+
+```ruby
+bank.accounts
+```
+
+And get its balance:
+```ruby
+bank.accounts.first.balance
+```
+
+Get last month transactions for a particular account:
+
+```ruby
+account = bank.accounts.first
+account.transactions
+```
+
+Get transactions for last year (from now):
+
+```ruby
+account = bank.accounts.first
+account.transactions = account.fetch_transactions(start_date: Date.today - 1.year, end_date: Date.today)
+account.transactions
+```
+
+
+
+## Contributing
+
+1. Fork it ( https://github.com/bank-scrap/bank_scrap/fork )
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create a new Pull Request
+
+## Thanks
+
+Thanks to Javier Cuevas (@javiercr) for his [BBVA](https://github.com/javiercr/bbva) gem.

data/bin/bankscrap

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+$LOAD_PATH.unshift(File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib')))
+
+require 'bankscrap'
+Bankscrap::CLI.start(ARGV)

data/lib/bankscrap.rb

@@ -0,0 +1,11 @@
+require 'active_support/all'
+require 'money'
+require 'rubygems'
+require_relative 'bankscrap/utils/inspectable'
+require_relative 'bankscrap/version'
+require_relative 'bankscrap/config'
+require_relative 'bankscrap/cli'
+require_relative 'bankscrap/bank'
+require_relative 'bankscrap/account'
+require_relative 'bankscrap/investment'
+require_relative 'bankscrap/transaction'

data/lib/bankscrap/account.rb

@@ -0,0 +1,25 @@
+module Bankscrap
+  class Account
+    include Utils::Inspectable
+
+    attr_accessor :bank, :id, :name, :balance, :currency, :available_balance, :description, :transactions, :iban, :bic
+
+    def initialize(params = {})
+      params.each { |key, value| send "#{key}=", value }
+    end
+
+    def transactions
+      @transactions ||= bank.fetch_transactions_for(self)
+    end
+
+    def fetch_transactions(start_date: Date.today - 2.years, end_date: Date.today)
+      bank.fetch_transactions_for(self, start_date: start_date, end_date: end_date)
+    end
+
+    private
+
+    def inspect_attributes
+      %i(id name balance currency available_balance description iban bic)
+    end
+  end
+end

data/lib/bankscrap/bank.rb

@@ -0,0 +1,82 @@
+require 'mechanize'
+require 'logger'
+
+module Bankscrap
+  class Bank
+    WEB_USER_AGENT = 'Mozilla/5.0 (Linux; Android 4.2.1; en-us; Nexus 4 Build/JOP40D) AppleWebKit/535.19 (KHTML, ' \
+                     'like Gecko) Chrome/18.0.1025.166 Mobile Safari/535.19'.freeze
+    attr_accessor :headers, :accounts, :investments
+
+    def initialize(_user, _password, log: false, debug: false, extra_args: nil)
+      @accounts = fetch_accounts
+    end
+
+    # Interface method placeholders
+
+    def fetch_accounts
+      raise "#{self.class} should implement a fetch_account method"
+    end
+
+    def fetch_investments
+      raise "#{self.class} should implement a fetch_investment method"
+    end
+
+    def fetch_transactions_for(*)
+      raise "#{self.class} should implement a fetch_transactions method"
+    end
+
+    def account_with_iban(iban)
+      accounts.find do |account|
+        account.iban.delete(' ') == iban.delete(' ')
+      end
+    end
+
+    private
+
+    def get(url, params: [], referer: nil)
+      @http.get(url, params, referer, @headers).body
+    end
+
+    def post(url, fields: {})
+      @http.post(url, fields, @headers).body
+    end
+
+    def put(url, fields: {})
+      @http.put(url, fields, @headers).body
+    end
+
+    # Sets temporary HTTP headers, execute a code block
+    # and resets the headers
+    def with_headers(tmp_headers)
+      current_headers = @headers
+      add_headers(tmp_headers)
+      yield
+    ensure
+      add_headers(current_headers)
+    end
+
+    def add_headers(headers)
+      @headers.merge! headers
+      @http.request_headers = @headers
+    end
+
+    def initialize_cookie(url)
+      @http.get(url).body
+    end
+
+    def initialize_connection
+      @http = Mechanize.new do |mechanize|
+        mechanize.user_agent = WEB_USER_AGENT
+        mechanize.agent.http.verify_mode = OpenSSL::SSL::VERIFY_NONE
+        mechanize.log = Logger.new(STDOUT) if @debug
+        # mechanize.set_proxy 'localhost', 8888
+      end
+
+      @headers = {}
+    end
+
+    def log(msg)
+      puts msg if @log
+    end
+  end
+end

data/lib/bankscrap/cli.rb

@@ -0,0 +1,88 @@
+require 'thor'
+require 'active_support/core_ext/string'
+
+module Bankscrap
+  class CLI < Thor
+    def self.shared_options
+      option :user,     default: ENV['BANKSCRAP_USER']
+      option :password, default: ENV['BANKSCRAP_PASSWORD']
+      option :log,      default: false
+      option :debug,    default: false
+
+      # Some bank needs more input, like birthday, this would go here
+      # Usage:
+      # bankscrap balance BANK_NAME --extra=birthday:01/12/1980
+      option :extra, type: :hash, default: {}
+    end
+
+    desc 'balance BANK', "get accounts' balance"
+    shared_options
+    def balance(bank)
+      assign_shared_options
+      initialize_client_for(bank)
+
+      @client.accounts.each do |account|
+        say "Account: #{account.description} (#{account.iban})", :cyan
+        say "Balance: #{account.balance}", :green
+      end
+    end
+
+    desc 'transactions BANK', "get account's transactions"
+    shared_options
+    def transactions(bank, iban = nil)
+      assign_shared_options
+
+      begin
+        start_date = @extra_args.key?('from') ? Date.strptime(@extra_args['from'], '%d-%m-%Y') : nil
+        end_date = @extra_args.key?('to') ? Date.strptime(@extra_args['to'], '%d-%m-%Y') : nil
+      rescue ArgumentError
+        say 'Invalid date format. Correct format d-m-Y', :red
+      end
+
+      initialize_client_for(bank)
+
+      account = iban ? @client.account_with_iban(iban) : @client.accounts.first
+
+      if !start_date.nil? && !end_date.nil?
+        if start_date > end_date
+          say 'From date must be lower than to date', :red
+          exit
+        end
+
+        transactions = account.fetch_transactions(start_date: start_date, end_date: end_date)
+      else
+        transactions = account.transactions
+      end
+
+      say "Transactions for: #{account.description} (#{account.iban})", :cyan
+
+      transactions.each do |transaction|
+        say transaction.to_s, (transaction.amount > Money.new(0) ? :green : :red)
+      end
+    end
+
+    private
+
+    def assign_shared_options
+      @user       = options[:user]
+      @password   = options[:password]
+      @log        = options[:log]
+      @debug      = options[:debug]
+      @extra_args = options[:extra]
+    end
+
+    def initialize_client_for(bank_name)
+      bank_class = find_bank_class_for(bank_name)
+      @client = bank_class.new(@user, @password, log: @log, debug: @debug, extra_args: @extra_args)
+    end
+
+    def find_bank_class_for(bank_name)
+      require "bankscrap-#{bank_name.downcase}"
+      Object.const_get("Bankscrap::#{bank_name}::Bank")
+    rescue LoadError
+      raise ArgumentError.new('Invalid bank name.')
+    rescue NameError
+      raise ArgumentError.new("Invalid bank name. Did you mean \"#{bank_name.upcase}\"?")
+    end
+  end
+end

data/lib/bankscrap/config.rb

@@ -0,0 +1,5 @@
+module Bankscrap
+  # Default format for money: 1.000,00 €
+  Money.default_formatting_rules = { symbol_position: :after }
+  I18n.load_path += Dir.glob(File.expand_path('../locale/*.yml', __FILE__))
+end

data/lib/bankscrap/investment.rb

@@ -0,0 +1,19 @@
+module Bankscrap
+  class Investment
+    include Utils::Inspectable
+
+    attr_accessor :bank, :id, :name, :balance, :currency, :investment
+
+    def initialize(params = {})
+      params.each { |key, value| send "#{key}=", value }
+    end
+
+    private
+
+    def inspect_attributes
+      [
+        :id, :name, :balance, :currency, :investment
+      ]
+    end
+  end
+end

data/lib/bankscrap/locale/en.yml

@@ -0,0 +1,12 @@
+en:
+  number:
+    currency:
+      format:
+        delimiter: '.'
+        format: '%n %u'
+        negative_format: '-%n %u'
+        precision: 2
+        separator: ','
+        significant: false
+        strip_insignificant_zeros: false
+        unit: '€'

data/lib/bankscrap/transaction.rb

@@ -0,0 +1,27 @@
+module Bankscrap
+  class Transaction
+    include Utils::Inspectable
+
+    attr_accessor :id, :amount, :currency,
+                  :effective_date, :description,
+                  :balance, :account
+
+    def initialize(params = {})
+      params.each { |key, value| send "#{key}=", value }
+    end
+
+    def to_s
+      "#{effective_date.strftime('%d/%m/%Y')}   #{description.ljust(45)} #{amount.format.rjust(20)}"
+    end
+
+    private
+
+    def inspect_attributes
+      [
+        :id, :amount, :currency,
+        :effective_date, :description,
+        :balance
+      ]
+    end
+  end
+end

data/lib/bankscrap/utils/inspectable.rb

@@ -0,0 +1,19 @@
+module Bankscrap
+  module Utils
+    module Inspectable
+      def inspect
+        attributes = inspect_attributes.reject do |x|
+          begin
+            attribute = send x
+            !attribute || (attribute.respond_to?(:empty?) && attribute.empty?)
+          rescue NoMethodError
+            true
+          end
+        end.map do |attribute|
+          "#{attribute}: #{send(attribute).inspect}"
+        end.join ' '
+        "#<#{self.class.name}:#{sprintf('0x%x', object_id)} #{attributes}>"
+      end
+    end
+  end
+end

data/lib/bankscrap/version.rb

@@ -0,0 +1,3 @@
+module Bankscrap
+  VERSION = '1.0.0'.freeze
+end

metadata

@@ -0,0 +1,178 @@
+--- !ruby/object:Gem::Specification
+name: bankscrap
+version: !ruby/object:Gem::Version
+  version: 1.0.0
+platform: ruby
+authors:
+- Javier Cuevas
+- Raúl Marcos
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2016-04-30 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.7'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.7'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+- !ruby/object:Gem::Dependency
+  name: byebug
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.5'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 3.5.1
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.5'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 3.5.1
+- !ruby/object:Gem::Dependency
+  name: rubocop
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.39.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.39.0
+- !ruby/object:Gem::Dependency
+  name: thor
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.19'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.19'
+- !ruby/object:Gem::Dependency
+  name: mechanize
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 2.7.4
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 2.7.4
+- !ruby/object:Gem::Dependency
+  name: activesupport
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.1'
+- !ruby/object:Gem::Dependency
+  name: money
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 6.5.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 6.5.0
+description: Command line tools to get bank account details from some banks.
+email:
+- javi@diacode.com
+- raulmarcosl@gmail.com
+executables:
+- bankscrap
+extensions: []
+extra_rdoc_files: []
+files:
+- README.md
+- bin/bankscrap
+- lib/.DS_Store
+- lib/bankscrap.rb
+- lib/bankscrap/account.rb
+- lib/bankscrap/bank.rb
+- lib/bankscrap/cli.rb
+- lib/bankscrap/config.rb
+- lib/bankscrap/investment.rb
+- lib/bankscrap/locale/en.yml
+- lib/bankscrap/transaction.rb
+- lib/bankscrap/utils/inspectable.rb
+- lib/bankscrap/version.rb
+homepage: https://github.com/bank-scrap/bank_scrap
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - "~>"
+    - !ruby/object:Gem::Version
+      version: '2.1'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.6.4
+signing_key: 
+specification_version: 4
+summary: Get your bank account details.
+test_files: []