bankjob 0.5.0
Sign up to get free protection for your applications and to get access to all the features.
- data/History.txt +4 -0
- data/PostInstall.txt +4 -0
- data/README.rdoc +77 -0
- data/bin/bankjob +10 -0
- data/lib/bankjob.rb +12 -0
- data/lib/bankjob/bankjob_runner.rb +184 -0
- data/lib/bankjob/cli.rb +258 -0
- data/lib/bankjob/payee.rb +114 -0
- data/lib/bankjob/scraper.rb +495 -0
- data/lib/bankjob/statement.rb +355 -0
- data/lib/bankjob/support.rb +217 -0
- data/lib/bankjob/transaction.rb +400 -0
- data/scrapers/base_scraper.rb +133 -0
- data/scrapers/bpi_scraper.rb +190 -0
- data/spec/bankjob_cli_spec.rb +15 -0
- data/spec/spec.opts +1 -0
- data/spec/spec_helper.rb +10 -0
- data/spec/statement_spec.rb +121 -0
- data/spec/transaction_spec.rb +81 -0
- metadata +114 -0
data/History.txt
ADDED
data/PostInstall.txt
ADDED
data/README.rdoc
ADDED
@@ -0,0 +1,77 @@
|
|
1
|
+
= bankjob
|
2
|
+
|
3
|
+
http://bankjob.rubyforge.org/
|
4
|
+
|
5
|
+
== DESCRIPTION:
|
6
|
+
|
7
|
+
Bankjob is a command-line ruby program for scraping online banking sites and producing statements in OFX (Open Fincancial Exchange) or CSV (Comma Separated Values) formats.
|
8
|
+
|
9
|
+
Bankjob was created for people like me who want to get their bank data into a 3rd party application but whose bank does not support downloads in OFX format.
|
10
|
+
It's also useful for keeping a permanent store of bank statements on your computer for reading in Excel (vs filing paper statements)
|
11
|
+
|
12
|
+
== FEATURES:
|
13
|
+
|
14
|
+
* Scrapes an online banking website to produce a bank statement
|
15
|
+
* Stores bank statements locally in CSV files, which can be loaded directly in spreadsheets like Microsoft Excel
|
16
|
+
* Stores bank statements locally in OFX files, which can be imported by many programs such as Quicken, MS Money, Gnu Cash and uploaded to some web applications
|
17
|
+
* Built-in support for uploading to your Wesabe account (www.wesabe.com)
|
18
|
+
* Supports coding of simple rules in ruby for modifying transaction details. E.g. automatically change "pment inst 3245003" to "paid home loan interest"
|
19
|
+
|
20
|
+
== SYNOPSIS:
|
21
|
+
|
22
|
+
bankjob --csv c:\bank\csv --scraper c:\bank\my_bpi_scraper.rb
|
23
|
+
--scraper-args "<my_username> <my_password>"
|
24
|
+
--wesabe "<wesabe_user> <wesabe_pass> <wesabe_acct>
|
25
|
+
--log c:\bank\bankjob.log --debug
|
26
|
+
|
27
|
+
I have this command in a .bat file which is launched daily by a scheduled task on my windows Media Center PC (which, since it's always on and connected to the internet, makes a useful home server)
|
28
|
+
|
29
|
+
This one command will:
|
30
|
+
* scrape my online banking website after logging in as me and navigating to the page with recent transactions
|
31
|
+
* apply some rules, coded in the my_bpi_scraper.rb file that make the descriptions more readable
|
32
|
+
* produce a statement in comma-separated-value format, keeping the original raw data as well as the new descriptions,
|
33
|
+
storing that in a file with a name like "20090327-20090406.csv" in my local directory c:\bank\csv (a permanent record)
|
34
|
+
* produce an OFX document with the same statement information
|
35
|
+
* upload the OFX statement to my wesabe account
|
36
|
+
* log debug-level details in bankjob.log
|
37
|
+
|
38
|
+
== REQUIREMENTS:
|
39
|
+
|
40
|
+
* Runs in ruby so you need to have ruby installed
|
41
|
+
* Requires a scraper for your online bank site
|
42
|
+
Some examples come packaged with Bankjob but you will probably need to write your own scraper in ruby.
|
43
|
+
For help go to http://groups.google.com/group/bankjob, but read http://bankjob.rubyforge.org first.
|
44
|
+
|
45
|
+
== INSTALL:
|
46
|
+
|
47
|
+
Mac OSX (linux):
|
48
|
+
|
49
|
+
<tt>sudo gem install bankjob</tt>
|
50
|
+
|
51
|
+
Windows:
|
52
|
+
<tt>gem install bankjob</tt>
|
53
|
+
|
54
|
+
== LICENSE:
|
55
|
+
|
56
|
+
(The MIT License)
|
57
|
+
|
58
|
+
Copyright (c) 2009 rubarb.bankjob@gmail.com
|
59
|
+
|
60
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
61
|
+
a copy of this software and associated documentation files (the
|
62
|
+
'Software'), to deal in the Software without restriction, including
|
63
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
64
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
65
|
+
permit persons to whom the Software is furnished to do so, subject to
|
66
|
+
the following conditions:
|
67
|
+
|
68
|
+
The above copyright notice and this permission notice shall be
|
69
|
+
included in all copies or substantial portions of the Software.
|
70
|
+
|
71
|
+
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
|
72
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
73
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
74
|
+
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
75
|
+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
|
76
|
+
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
|
77
|
+
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/bin/bankjob
ADDED
data/lib/bankjob.rb
ADDED
@@ -0,0 +1,12 @@
|
|
1
|
+
$:.unshift(File.dirname(__FILE__)) unless
|
2
|
+
$:.include?(File.dirname(__FILE__)) || $:.include?(File.expand_path(File.dirname(__FILE__)))
|
3
|
+
|
4
|
+
require 'bankjob/support.rb'
|
5
|
+
require 'bankjob/statement.rb'
|
6
|
+
require 'bankjob/transaction.rb'
|
7
|
+
require 'bankjob/scraper.rb'
|
8
|
+
require 'bankjob/payee.rb'
|
9
|
+
|
10
|
+
module Bankjob
|
11
|
+
BANKJOB_VERSION = '0.5.0' unless defined?(BANKJOB_VERSION)
|
12
|
+
end
|
@@ -0,0 +1,184 @@
|
|
1
|
+
require 'rubygems'
|
2
|
+
require 'logger'
|
3
|
+
require 'bankjob.rb'
|
4
|
+
|
5
|
+
module Bankjob
|
6
|
+
class BankjobRunner
|
7
|
+
|
8
|
+
# Runs the bankjob application, loading and running the
|
9
|
+
# scraper specified in the command line args and generating
|
10
|
+
# the output file.
|
11
|
+
def run(options, stdout)
|
12
|
+
logger = options.logger
|
13
|
+
|
14
|
+
if options.wesabe_help
|
15
|
+
Bankjob.wesabe_help(options.wesabe_args)
|
16
|
+
exit(0) # Wesabe help describes to the user how to use the wesabe options then quits
|
17
|
+
end
|
18
|
+
|
19
|
+
# Load the scraper object dynamically, then scrape the web
|
20
|
+
# to get a new bank statement
|
21
|
+
scraper = Scraper.load_scraper(options.scraper, options, logger)
|
22
|
+
|
23
|
+
begin
|
24
|
+
statement = scraper.scrape_statement(options.scraper_args)
|
25
|
+
statement = Scraper.post_process_transactions(statement)
|
26
|
+
rescue Exception => e
|
27
|
+
logger.fatal(e)
|
28
|
+
puts "Failed to scrape a statement successfully with #{options.scraper} due to: #{e.message}\n"
|
29
|
+
puts "Use --debug --log bankjob.log then check the log for more details"
|
30
|
+
exit (1)
|
31
|
+
end
|
32
|
+
|
33
|
+
# a lot of if's here but we allow for the user to generate ofx
|
34
|
+
# and csv to files while simultaneously uploading to wesabe
|
35
|
+
|
36
|
+
if options.csv
|
37
|
+
if options.csv_out.nil?
|
38
|
+
puts write_csv_doc([statement], true) # dump to console with header, no file specified
|
39
|
+
else
|
40
|
+
csv_file = file_name_from_option(options.csv_out, statement, "csv")
|
41
|
+
|
42
|
+
# Output data as comma separated values possibly merging
|
43
|
+
if File.file?(csv_file)
|
44
|
+
# TODO until we fix merging csv files are appended
|
45
|
+
open(csv_file, "a") do |f|
|
46
|
+
f.puts(write_csv_doc([statement]))
|
47
|
+
end
|
48
|
+
logger.info("Statement is being appended as csv to #{csv_file}")
|
49
|
+
#
|
50
|
+
# TODO fix the merging then uncomment this
|
51
|
+
# old_file_path = csv_file
|
52
|
+
# # The file already exists, lets load it and merge with the new data
|
53
|
+
# old_statement = scraper.create_statement()
|
54
|
+
# old_statement.from_csv(old_file_path, scraper.decimal)
|
55
|
+
# begin
|
56
|
+
# old_statement.merge!(statement)
|
57
|
+
# statement = old_statement
|
58
|
+
# rescue Exception => e
|
59
|
+
# # the merge failed, so leave the statement as the original and store it separately
|
60
|
+
# output_file = output_file + "_#{date_range}_merge_failed"
|
61
|
+
# logger.warn("Merge failed, storing new data in #{output_file} instead of appending it to #{old_file_path}")
|
62
|
+
# logger.debug("Merge failed due to: #{e.message}")
|
63
|
+
# end
|
64
|
+
else
|
65
|
+
open(csv_file, "w") do |f|
|
66
|
+
f.puts(write_csv_doc([statement], true)) # true = write with header
|
67
|
+
end
|
68
|
+
logger.info("Statement is being written as csv to #{csv_file}")
|
69
|
+
end
|
70
|
+
end
|
71
|
+
end # if csv
|
72
|
+
|
73
|
+
# Create an ofx document and write it if necessary
|
74
|
+
if (options.ofx or options.wesabe_upload)
|
75
|
+
ofx_doc = write_ofx_doc([statement])
|
76
|
+
end
|
77
|
+
|
78
|
+
# Output ofx file
|
79
|
+
if options.ofx
|
80
|
+
if options.ofx_out.nil?
|
81
|
+
puts ofx_doc # dump to console, no file specified
|
82
|
+
else
|
83
|
+
ofx_file = file_name_from_option(options.ofx_out, statement, "ofx")
|
84
|
+
open(ofx_file, "w") do |f|
|
85
|
+
f.puts(ofx_doc)
|
86
|
+
end
|
87
|
+
logger.info("Statement is being output as ofx to #{ofx_file}")
|
88
|
+
end
|
89
|
+
end
|
90
|
+
|
91
|
+
# Upload to wesabe if requested
|
92
|
+
if options.wesabe_upload
|
93
|
+
begin
|
94
|
+
Bankjob.wesabe_upload(options.wesabe_args, ofx_doc, logger)
|
95
|
+
rescue Exception => e
|
96
|
+
logger.fatal("Failed to upload to Wesabe")
|
97
|
+
logger.fatal(e)
|
98
|
+
puts "Failed to upload to Wesabe: #{e.message}\n"
|
99
|
+
puts "Try bankjob --wesabe-help for help on this feature."
|
100
|
+
exit(1)
|
101
|
+
end
|
102
|
+
end
|
103
|
+
end # run
|
104
|
+
|
105
|
+
##
|
106
|
+
# Generates an OFX document to a string that starts with the stanadard
|
107
|
+
# OFX header and contains the XML for the specified +statements+
|
108
|
+
#
|
109
|
+
def write_ofx_doc(statements)
|
110
|
+
ofx = generate_ofx2_header
|
111
|
+
statements.each do |statement|
|
112
|
+
ofx << statement.to_ofx
|
113
|
+
end
|
114
|
+
return ofx
|
115
|
+
end
|
116
|
+
|
117
|
+
##
|
118
|
+
# Generates a CSV document to a string containing the transactions in
|
119
|
+
# all of the specified +statements+
|
120
|
+
#
|
121
|
+
def write_csv_doc(statements, header = false)
|
122
|
+
csv = ""
|
123
|
+
csv << Statement.csv_header if header
|
124
|
+
statements.each do |statement|
|
125
|
+
csv << statement.to_csv
|
126
|
+
end
|
127
|
+
return csv
|
128
|
+
end
|
129
|
+
|
130
|
+
##
|
131
|
+
# Generates the (XML) OFX2 header lines that allow the OFX 2.0 document
|
132
|
+
# to be recognized.
|
133
|
+
#
|
134
|
+
# <em>(Note that this is crucial for www.wesabe.com to accept the OFX
|
135
|
+
# document in an upload)</em>
|
136
|
+
#
|
137
|
+
def generate_ofx2_header
|
138
|
+
return <<-EOF
|
139
|
+
<?xml version="1.0" encoding="UTF-8"?>
|
140
|
+
<?OFX OFXHEADER="200" SECURITY="NONE" OLDFILEUID="NONE" NEWFILEUID="NONE" VERSION="200"?>
|
141
|
+
EOF
|
142
|
+
end
|
143
|
+
|
144
|
+
##
|
145
|
+
# Generates the (non-XML) OFX header lines that allow the OFX 1.0 document
|
146
|
+
# to be recognized.
|
147
|
+
#
|
148
|
+
# <em>(Note that this is crucial for www.wesabe.com to accept the OFX
|
149
|
+
# document in an upload)</em>
|
150
|
+
#
|
151
|
+
def generate_ofx_header
|
152
|
+
return <<-EOF
|
153
|
+
OFXHEADER:100
|
154
|
+
DATA:OFXSGML
|
155
|
+
VERSION:102
|
156
|
+
SECURITY:NONE
|
157
|
+
ENCODING:USASCII
|
158
|
+
CHARSET:1252
|
159
|
+
COMPRESSION:NONE
|
160
|
+
OLDFILEUID:NONE
|
161
|
+
NEWFILEUID:NONE
|
162
|
+
EOF
|
163
|
+
end
|
164
|
+
|
165
|
+
##
|
166
|
+
# Takes a name or path for an output file and a Statement and if the file
|
167
|
+
# path is a directory, creates a new file name based on the date range
|
168
|
+
# of the statement and returns a path to that file.
|
169
|
+
# If +output_file+ is not a directory it is returned as-is.
|
170
|
+
#
|
171
|
+
def file_name_from_option(output_file, statement, type)
|
172
|
+
# if the output_file is a directory, we create a new file name
|
173
|
+
if (output_file and File.directory?(output_file))
|
174
|
+
# Create a date range string for the first and last transactions in the statement
|
175
|
+
# This will looks something like: 20090130000000-20090214000000
|
176
|
+
date_range = "#{Bankjob.date_time_to_ofx(statement.from_date)[0..7]}-#{Bankjob.date_time_to_ofx(statement.to_date)[0..7]}"
|
177
|
+
filename = "#{date_range}.#{type}"
|
178
|
+
output_file = File.join(output_file, filename)
|
179
|
+
end
|
180
|
+
# else we assume output_file is a file name/path already
|
181
|
+
return output_file
|
182
|
+
end
|
183
|
+
end # class BankjobRunner
|
184
|
+
end # module Bankjob
|
data/lib/bankjob/cli.rb
ADDED
@@ -0,0 +1,258 @@
|
|
1
|
+
require 'rubygems'
|
2
|
+
require 'ostruct'
|
3
|
+
require 'optparse'
|
4
|
+
require 'logger'
|
5
|
+
|
6
|
+
$:.unshift(File.dirname(__FILE__)) unless
|
7
|
+
$:.include?(File.dirname(__FILE__)) || $:.include?(File.expand_path(File.dirname(__FILE__)))
|
8
|
+
|
9
|
+
require 'bankjob_runner.rb'
|
10
|
+
|
11
|
+
module Bankjob
|
12
|
+
class CLI
|
13
|
+
|
14
|
+
NEEDED = "Needed" # constant to indicate compulsory options
|
15
|
+
NOT_NEEDED = "Not Needed" # constant to indicate no-longer compulsory options
|
16
|
+
|
17
|
+
def self.execute(stdout, argv)
|
18
|
+
# The BanjobOptions module above, through the magic of OptiFlags
|
19
|
+
# has augmented ARGV with the command line options accessible through
|
20
|
+
# ARGV.flags.
|
21
|
+
runner = BankjobRunner.new()
|
22
|
+
runner.run(parse(argv), stdout)
|
23
|
+
end # execute
|
24
|
+
|
25
|
+
##
|
26
|
+
# Parses the command line arguments using OptionParser and returns
|
27
|
+
# an open struct with an attribute for each option
|
28
|
+
#
|
29
|
+
def self.parse(args)
|
30
|
+
options = OpenStruct.new
|
31
|
+
|
32
|
+
# Set the default options
|
33
|
+
options.scraper = NEEDED
|
34
|
+
options.scraper_args = []
|
35
|
+
options.log_level = Logger::WARN
|
36
|
+
options.log_file = nil
|
37
|
+
options.debug = false
|
38
|
+
options.input = nil
|
39
|
+
options.ofx = false # ofx is the default but only if csv is false
|
40
|
+
options.ofx_out = false
|
41
|
+
options.csv = false
|
42
|
+
options.csv_out = nil # allow for separate csv and ofx output files
|
43
|
+
options.wesabe_help = false
|
44
|
+
options.wesabe_upload = false
|
45
|
+
options.wesabe_args = nil
|
46
|
+
options.logger = nil
|
47
|
+
|
48
|
+
opt = OptionParser.new do |opt|
|
49
|
+
|
50
|
+
opt.banner = "Bankjob - scrapes your online banking website and produces an OFX or CSV document.\n" +
|
51
|
+
"Usage: bankjob [options]\n"
|
52
|
+
|
53
|
+
opt.version = Bankjob::BANKJOB_VERSION
|
54
|
+
|
55
|
+
opt.on('-s', '--scraper SCRAPER',
|
56
|
+
"The name of the ruby file that scrapes the website.\n") do |file|
|
57
|
+
options.scraper = file
|
58
|
+
end
|
59
|
+
|
60
|
+
opt.on('--scraper-args ARGS',
|
61
|
+
"Any arguments you want to pass on to your scraper.",
|
62
|
+
"The entire set of arguments must be quoted and separated by spaces",
|
63
|
+
"but you can use single quotes to specify multi-word arguments for",
|
64
|
+
"your scraper. E.g.",
|
65
|
+
" -scraper-args \"-user Joe -password Joe123 -arg3 'two words'\""," ",
|
66
|
+
"This assumes your scraper accepts an array of args and knows what",
|
67
|
+
"to do with them, it will vary from scraper to scraper.\n") do |sargs|
|
68
|
+
options.scraper_args = sub_args_to_array(sargs)
|
69
|
+
end
|
70
|
+
|
71
|
+
opt.on('-i', '--input INPUT_HTML_FILE',
|
72
|
+
"An html file used as the input instead of scraping the website -",
|
73
|
+
"useful for debugging.\n") do |file|
|
74
|
+
options.input = file
|
75
|
+
end
|
76
|
+
|
77
|
+
opt.on('-l', '--log LOG_FILE',
|
78
|
+
"Specify a file to log information and debug messages.",
|
79
|
+
"If --debug is used, log info will go to the console, but if neither",
|
80
|
+
"this nor --debug is specfied, there will be no log.",
|
81
|
+
"Note that the log is rolled over once per week\n") do |log_file|
|
82
|
+
options.log_file = log_file
|
83
|
+
end
|
84
|
+
|
85
|
+
opt.on('q', '--quiet', "Suppress all messages, warnings and errors.",
|
86
|
+
"Only fatal errors will go in the log") do
|
87
|
+
options.log_level = Logger::FATAL
|
88
|
+
end
|
89
|
+
|
90
|
+
opt.on( '--verbose', "Log detailed informational messages.\n") do
|
91
|
+
options.log_level = Logger::INFO
|
92
|
+
end
|
93
|
+
|
94
|
+
opt.on('--debug',
|
95
|
+
"Log debug-level information to the log",
|
96
|
+
"if here is one and put debug info in log\n") do
|
97
|
+
options.log_level = Logger::DEBUG
|
98
|
+
options.debug = true
|
99
|
+
end
|
100
|
+
|
101
|
+
opt.on('--ofx [FILE]',
|
102
|
+
"Write out the statement as an OFX2 compliant XML document."," ",
|
103
|
+
"If FILE is not specified, the XML is dumped to the console.",
|
104
|
+
"If FILE specifies a directory then a new file will be created with a",
|
105
|
+
"name generated from the dates of the first and last transactions.",
|
106
|
+
"If FILE specifies a file that already exists it will be overwritten."," ",
|
107
|
+
"(Note that ofx is the default format unless --csv is specified,",
|
108
|
+
"and that both CSV and OFX documents can be produced by specifying",
|
109
|
+
"both options.)\n") do |file|
|
110
|
+
options.ofx = true
|
111
|
+
options.ofx_out = file
|
112
|
+
end
|
113
|
+
|
114
|
+
opt.on('--csv [FILE]',
|
115
|
+
"Writes out the statement as a CSV (comma separated values) document.",
|
116
|
+
"All of the information available including numeric values for amount,",
|
117
|
+
"raw and rule-generated descriptions, etc, are produced in the CSV document.", " ",
|
118
|
+
"The document produced is suitable for loading into a spreadsheet like",
|
119
|
+
"Microsoft Excel with the dates formatted to allow for auto recognition.",
|
120
|
+
"This option can be used in conjunction with --ofx or --wesabe to produce",
|
121
|
+
"a local permanent log of all the data scraped over time.", " ",
|
122
|
+
"If FILE is not specified, the CSV is dumped to the console.",
|
123
|
+
"If FILE specifies a directory then a new file will be created with a",
|
124
|
+
"name generated from the dates of the first and last transactions.",
|
125
|
+
"If FILE specifies a file that already exists then the new statement",
|
126
|
+
"will be appended to the existing one in that file with care taken to",
|
127
|
+
"merge removing duplicate entries.\n",
|
128
|
+
"[WARNING - this merging does not yet function properly - its best to specify a directory for now.]\n"
|
129
|
+
) do |file|
|
130
|
+
# TODO update this warning when we have merging working
|
131
|
+
options.csv = true
|
132
|
+
options.csv_out = file
|
133
|
+
end
|
134
|
+
|
135
|
+
opt.on('--wesabe-help [WESABE_ARGS]',
|
136
|
+
"Show help information on how to use Bankjob to upload to Wesabe.",
|
137
|
+
"Optionally use with \"wesabe-user password\" to get Wesabe account info.",
|
138
|
+
"Note that the quotes around the WESABE_ARGS to send both username",
|
139
|
+
"and password are necessary.", " ",
|
140
|
+
"Use --wesabe-help with no args for more details.\n") do |wargs|
|
141
|
+
options.wesabe_args = sub_args_to_array(wargs)
|
142
|
+
options.wesabe_help = true
|
143
|
+
options.scraper = NOT_NEEDED # scraper is not NEEDED when this option is set
|
144
|
+
end
|
145
|
+
|
146
|
+
opt.on('--wesabe WESABE_ARGS',
|
147
|
+
"Produce an OFX document from the statement and upload it to a Wesabe account.",
|
148
|
+
"WESABE_ARGS must be quoted and space-separated, specifying the wesabe account",
|
149
|
+
"username, password and - if there is more than one - the wesabe account number.", " ",
|
150
|
+
"Before trying this, use bankjob --wesabe-help to get more information.\n"
|
151
|
+
) do |wargs|
|
152
|
+
options.wesabe_args = sub_args_to_array(wargs)
|
153
|
+
options.wesabe_upload = true
|
154
|
+
end
|
155
|
+
|
156
|
+
opt.on('--version', "Display program version and exit.\n" ) do
|
157
|
+
puts opt.version
|
158
|
+
exit
|
159
|
+
end
|
160
|
+
|
161
|
+
opt.on_tail('-h', '--help', "Display this usage message and exit.\n" ) do
|
162
|
+
puts opt
|
163
|
+
puts <<-EOF
|
164
|
+
|
165
|
+
Some common options:
|
166
|
+
|
167
|
+
o Debugging:
|
168
|
+
--debug --scraper bpi_scraper.rb --input /tmp/DownloadedPage.html --ofx
|
169
|
+
|
170
|
+
o Regular use: (output ofx documents to a directory called 'bank')
|
171
|
+
--scraper /bank/mybank_scraper.rb --scraper-args "me mypass123" --ofx /bank --log /bank/bankjob.log --verbose
|
172
|
+
|
173
|
+
o Abbreviated options with CSV output: (output csv appended continuously to a file)
|
174
|
+
-s /bank/otherbank_scraper.rb --csv /bank/statements.csv -l /bank/bankjob.log -q
|
175
|
+
|
176
|
+
o Get help on using Wesabe:
|
177
|
+
--wesabe-help
|
178
|
+
|
179
|
+
o Upload to Wesabe: (I have 4 Wesabe accounts and am uploading to the 3rd)
|
180
|
+
-s /bank/mybank_scraper.rb --wesabe "mywesabeuser password 3" -l /bank/bankjob.log --debug
|
181
|
+
EOF
|
182
|
+
exit!
|
183
|
+
end
|
184
|
+
|
185
|
+
end #OptionParser.new
|
186
|
+
|
187
|
+
begin
|
188
|
+
opt.parse!(args)
|
189
|
+
_validate_options(options) # will raise exceptions if options are invalid
|
190
|
+
_init_logger(options) # sets the logger
|
191
|
+
rescue Exception => e
|
192
|
+
puts e, "", opt
|
193
|
+
exit
|
194
|
+
end
|
195
|
+
|
196
|
+
return options
|
197
|
+
end #self.parse
|
198
|
+
|
199
|
+
private
|
200
|
+
|
201
|
+
# Checks if the options are valid, raising exceptiosn if they are not.
|
202
|
+
# If the --debug option is true, then messages are dumped but flow continues
|
203
|
+
def self._validate_options(options)
|
204
|
+
begin
|
205
|
+
#Note that OptionParser doesn't really handle compulsory arguments so we use
|
206
|
+
#our own mechanism
|
207
|
+
if options.scraper == NEEDED
|
208
|
+
raise "Incomplete arguments: You must specify a scaper ruby script with --scraper"
|
209
|
+
end
|
210
|
+
|
211
|
+
# Add in the --ofx option if it is not already specified and if --csv is not specified either
|
212
|
+
options.ofx = true unless options.csv or options.wesabe_upload
|
213
|
+
rescue Exception => e
|
214
|
+
if options.debug
|
215
|
+
# just dump the message and eat the exception -
|
216
|
+
# we may be using dummy values for debugging
|
217
|
+
puts "Ignoring error in options due to --debug flag: #{e}"
|
218
|
+
else
|
219
|
+
raise e
|
220
|
+
end
|
221
|
+
end #begin/rescue
|
222
|
+
|
223
|
+
end #_validate_options
|
224
|
+
|
225
|
+
##
|
226
|
+
# Initializes the logger taking the log-level and the log
|
227
|
+
# file name from the command line +options+ and setting the logger back on
|
228
|
+
# the options struct as +options.logger+
|
229
|
+
#
|
230
|
+
# Note that the level is not set explicitly in options but derived from
|
231
|
+
# flag options like --verbose (INFO), --quiet (FATAL) and --debug (DEBUG)
|
232
|
+
#
|
233
|
+
def self._init_logger(options)
|
234
|
+
# the log log should roll over weekly
|
235
|
+
if options.log_file.nil?
|
236
|
+
if options.debug
|
237
|
+
# if debug is on but no logfile is specified then log to console
|
238
|
+
options.log_file = STDOUT
|
239
|
+
else
|
240
|
+
# Setting the log level to UNKNOWN effectively turns logging off
|
241
|
+
options.log_level = Logger::UNKNOWN
|
242
|
+
end
|
243
|
+
end
|
244
|
+
options.logger = Logger.new(options.log_file, 'weekly') # roll over weekly
|
245
|
+
options.logger.level = options.log_level
|
246
|
+
end
|
247
|
+
|
248
|
+
# Takes a string of arguments and splits it into an array, allowing for 'single quotes'
|
249
|
+
# to join words into a single argument.
|
250
|
+
# (Note that parentheses are used to group to exclude the single quotes themselves, but grouping
|
251
|
+
# results in scan creating an array of arrays with some nil elements hence flatten and delete)
|
252
|
+
def self.sub_args_to_array(subargs)
|
253
|
+
return nil if subargs.nil?
|
254
|
+
return subargs.scan(/([^\s']+)|'([^']*)'/).flatten.delete_if { |x| x.nil?}
|
255
|
+
end
|
256
|
+
|
257
|
+
end #class CLI
|
258
|
+
end
|