quick-etl 0.1.0

data/README

@@ -0,0 +1,126 @@
+= QuickETL = Quicken Extract Transform and Load
+
+Version: 0.1.0 
+Date:    2008-02-10
+
+---
+
+QuickETL = Quicken Extract Transform and Load
+QuickETL - Copyright 2008 by Chris Joakim.
+QuickETL is available under GNU General Public License (GPL) license.
+
+QuickETL is used to parse your Quicken *.qif files into csv format, then
+load this csv data into a sqlite database, where it can be subsequently
+queried and used for reporting.
+
+QuickETL functionality is invoked from a command line with the Ruby Rake program.
+File 'rakefile.rb' is included in the gem file.
+
+Future versions of QuickETL will add a User Interface for running queries
+and generating reports.
+
+
+= General
+
+Author::   Chris Joakim <chris@joakim-systems.com>
+Requires:: Ruby 1.8.4 or later
+License::  Copyright 2008 by Chris Joakim.
+           GNU General Public License (GPL) license.
+           See http://www.gnu.org/copyleft/gpl.html
+		
+
+= License
+
+QuickETL is available under GNU General Public License (GPL) license.
+
+
+= Download
+
+The latest version of QuickETL can be found at http://rubyforge.org/projects/quick-etl/
+
+
+= Installation
+
+Please perform the following steps to install QuickETL.
+
+1.  Create a directory on your computer which will be considered the "home" directory
+    for QuickETL.  This directory will contain your Quicken *.qif files, as well as
+    the following files:
+      - QuickenData.qif       (YOUR qif file, created by using the Quicken File -> Export... function)
+      - rakefile.rb           (The Ruby Rake "rakefile"; your interface to QuickETL functionality)      
+      - quick-etl.properties  (required but unused in this release)
+      - quick-etl.csv         (csv file created by parsing your *.qif file).
+      - quick-etl.db          (sqlite database file).
+      - quick-etl.load        (generated file used to load the csv file)
+
+2.  Create the environment variable QUICK_ETL_HOME, and set its value to the directory name
+    you chose in step 1.  This environment variable MUST be set for QuickETL to function properly.
+  
+    Restart your shell program after setting this environment variable.
+        
+3.  Verify that you have the sqlite3 program installed on your computer.
+    See http://www.sqlite.org/ for sqlite3 installation and usage instructions.
+
+    For Max OS/X 10.4+ users, the sqlite3 is already installed with the OS.
+               
+4.  QuickETL is packaged and installed as a ruby 'gem'.  Download the latest gem to 
+    your computer, then run the following command from your download directory.
+ 
+    gem install quick-etl-1.0.0.gem
+    
+5.  Unpack the 'quick-etl-0.1.0.gem' file in your download directory with the following command:
+
+	gem unpack quick-etl-0.1.0.gem   
+	
+	Then copy the 'rakefile.rb' and 'quick-etl.properties' files, that were unpacked from the gem,
+	to your QUICK_ETL_HOME directory.
+
+
+= Usage	
+
+1.  Use your Quicken program to export its data to the QUICK_ETL_HOME directory.
+    Specify output file 'QuickenData.qif'.
+
+2.  Open a shell window in your QUICK_ETL_HOME directory.
+
+    Execute command 'rake -T' to see the list of functions available.
+    
+    Execute command 'rake parse' to parse the qif file into csv format.
+    
+    Execute command 'rake load' to load the csv file data into sqlite3 database 'quick-etl.db'.
+
+    Execute command 'rake parse_and_load' to run both the parsing and loading function.
+    
+    Execute command 'rake show_ddl' to see the structure of the database table and its column names.
+        
+3.  Use the sqlite3 CLP (Command Line Program) to query the database.
+    From the QUICK_ETL_HOME directory, execute command 'sqlite3 quick-etl.db'.
+    
+    You can then enter and execute queries like the following:
+    
+    sqlite> select * from transactions;
+    sqlite> select * from transactions where tran_date == '2008-02-05';
+    
+    Note: A User Interface will be created for QuickETL its next release.    
+            
+    
+= Road Map / TODO List
+
+1.  Develop a UI for the quick-etl.db sqlite database.
+
+2.  Develop reporting functionality.
+
+3.  Support the "merging" of multiple *qif files into one database.
+
+
+= Support
+
+Please see http://rubyforge.org/projects/quick-etl/ to submit a request or report
+a bug, on the Tracker page.
+
+
+= Warranty
+
+This software is provided "as is" and without any express or implied warranties, 
+including, without limitation, the implied warranties of merchantibility and 
+fitness for a particular purpose.

data/doc/qifFormatDoc.txt

@@ -0,0 +1,231 @@
+
+http://www.intuit.com/quicken/technical-support/quicken/old-faqs/dosfaqs/60006.html
+
+Q: What is the Quicken interchange format (QIF)?
+
+A: The Quicken interchange format (QIF) is a specially formatted text (ASCII)
+   file that lets you to move Quicken transactions: 
+
+From one Quicken account register into another Quicken account register, or 
+To/From another application that supports the QIF format. 
+Note: For Quicken to translate data from a text file into the Quicken register
+as transactions, the text file must be in the QIF format. 
+
+
+Required File Formatting: 
+
+
+Each transaction must end with a symbol, indicating the end of entry. 
+Each item in the transaction must display on a separate line. 
+When Quicken exports an account register or list, it adds a line to the top 
+of the file that identifies the type of account or list.  Listed below are 
+the header lines Quicken adds to the exported files: 
+
+Header           Type of data  
+!Type:Bank       Bank account transactions  
+!Type:Cash       Cash account transactions  
+!Type:CCard      Credit card account transactions  
+!Type:Invst      Investment account transactions  
+!Type:Oth A      Asset account transactions  
+!Type:Oth L      Liability account transactions  
+!Account         Account list or which account follows  
+!Type:Cat        Category list 
+!Type:Class      Class list 
+!Type:Memorized  Memorized transaction list  
+
+
+You can force Quicken to import all transfers, regardless of whether 
+Ignore Transfers is selected when the file is imported.  You must add a 
+line to the file being imported into a Quicken account.  Use a text editor
+or word processor to put the following line right after the header line 
+at the top of the file:
+
+!Option:AllXfr
+
+Items for Non-Investment Accounts 
+
+Each item in a bank, cash, credit card, other liability, or other asset 
+account must begin with a letter that indicates the field in the Quicken 
+register.  The non-split items can be in any sequence: 
+
+Field Indicator Explanation  
+D Date 
+T Amount 
+C Cleared status 
+N Num (check or reference number)  
+P Payee 
+M Memo 
+A Address (up to five lines; the sixth line is an optional message)  
+L Category (Category/Subcategory/Transfer/Class)  
+S Category in split (Category/Transfer/Class)  
+E Memo in split 
+$ Dollar amount of split 
+^ End of the entry 
+
+Note: Repeat the S, E, and $ lines as many times as needed for additional
+items in a split.  If an item is omitted from the transaction in the QIF 
+file, Quicken treats it as a blank item.
+
+Items for Investment Accounts 
+
+Field Indicator Explanation  
+D Date 
+N Action 
+Y Security 
+I Price 
+Q Quantity (number of shares or split ratio)  
+T Transaction amount 
+C Cleared status 
+P Text in the first line for transfers and reminders  
+M Memo 
+O Commission 
+L Account for the transfer  
+$ Amount transferred 
+^ End of the entry 
+
+Items for Account Information 
+
+The account header !Account is used in two places-at the start of an account
+list and the start of a list of transactions to specify to which account they
+belong. 
+
+Field Indicator Explanation  
+N Name 
+T Type of account 
+D Description 
+L Credit limit (only for credit card accounts)  
+/ Statement balance date 
+$ Statement balance amount  
+^ End of entry 
+
+Items for a Category List
+
+Field Indicator Explanation  
+N Category name:subcategory name  
+D Description 
+T Tax related if included, not tax related if omitted  
+I Income category 
+E Expense category 
+  (if category type is unspecified, quicken assumes expense type)  
+B Budget amount (only in a Budget Amounts QIF file)  
+R Tax schedule information  
+^ End of entry 
+
+Items for a Class List
+
+Field Indicator Explanation  
+N Class name 
+D Description 
+^ End of entry  
+
+Items for a Memorized Transaction List
+
+Immediately preceding the ^ character, each entry must end with one of the 
+following file indicators to specify the transaction type. 
+
+KC 
+KD 
+KP 
+KI 
+KE 
+With that exception, memorized transaction entries have the same format as 
+regular transaction entries (non-investment accounts). However, the Date 
+or Num field is included. All items are optional, but if an amortization 
+record is included, all seven amortization lines must also be included.
+
+Field Indicator Explanation  
+KC Check transaction 
+KD Deposit transaction 
+KP Payment transaction 
+KI Investment transaction 
+KE Electronic payee transaction  
+T  Amount 
+C  Cleared status 
+P  Payee 
+M  Memo 
+A  Address 
+L  Category or Transfer/Class  
+S  Category/class in split 
+E  Memo in split 
+$  Dollar amount of split 
+1  Amortization: First payment date  
+2  Amortization: Total years for loan  
+3  Amortization: Number of payments already made  
+4  Amortization: Number of periods per year  
+5  Amortization: Interest rate  
+6  Amortization: Current loan balance  
+7  Amortization: Original loan amount  
+^  End of entry 
+
+Examples of QIF files
+
+Normal Transactions Example
+
+Transaction Item Comment (not in file)  
+!Type:Bank Header 
+D6/ 1/94 Date 
+T-1,000.00 Amount 
+N1005 Check number 
+PBank Of Mortgage Payee  
+L[linda] Category 
+S[linda] First category in split  
+$-253.64 First amount in split  
+SMort Int Second category in split  
+$-746.36 Second amount in split  
+^ End of the transaction 
+D6/ 2/94 Date 
+T75.00 Amount 
+PDeposit Payee 
+^ End of the transaction 
+D6/ 3/94 Date 
+T-10.00 Amount 
+PJoBob Biggs Payee 
+MJ.B. gets bucks Memo 
+LEntertain Category 
+A1010 Rodeo Dr. Address (line 1)  
+AWaco, Tx Address (line 2)  
+A80505 Address (line 3) 
+A Address (line 4) 
+A Address (line 5) 
+A Address (line 6) 
+^ End of the transaction 
+
+Investment Example
+
+Transaction Item Comment (not in file)  
+!Type:Invst Header line 
+D8/25/93 Date 
+NShrsIn Action (optional)  
+Yibm4 Security 
+I11.260 Price 
+Q88.81 Quantity 
+CX Cleared Status 
+T1,000.00 Amount 
+MOpening Balance Memo 
+^ End of the transaction 
+D8/25/93 Date 
+NBuyX Action 
+Yibm4 Security 
+I11.030 Price 
+Q9.066 Quantity 
+T100.00 Amount 
+MEst. price as of 8/25/93 Memo  
+L[CHECKING] Account for transfer  
+$100.00 Amount transferred  
+^ End of the transaction 
+
+Memorized List Example
+
+Transaction Item Comment (not in file)  
+!Type:Memorized Header line  
+T-50.00 Amount 
+PJoe Hayes Payee 
+MRent Memo 
+KC Check transaction 
+^ End of the transaction 
+T-25.00 Amount 
+T-25.00 Company Payee 
+LTelephone Category 
+KP Payment transaction 
+^ End of the transaction 
+

data/lib/quick_etl.rb

@@ -0,0 +1,28 @@
+=begin rdoc
+
+QuickETL = Quicken Extract Transform and Load
+QuickETL - Copyright 2008 by Chris Joakim.
+QuickETL is available under GNU General Public License (GPL) license.
+
+-
+  
+This is the main ruby file for this project.  It contains require statements
+for all of the necessary ruby standard libraries, and quick_etl custom files.
+
+=end
+
+# System libraries and gems:
+require 'rubygems'
+require 'rbconfig'
+require 'date'
+require 'time'
+require 'sqlite3'
+
+# Custom libraries:
+require 'quick_etl_kernel'
+require 'quick_etl_object'
+require 'quick_etl_dbo'
+require 'quick_etl_money'
+require 'quick_etl_qif_date'
+require 'quick_etl_qif_file_reader'
+require 'quick_etl_qif_transaction'

data/lib/quick_etl_dbo.rb

@@ -0,0 +1,88 @@
+=begin rdoc
+
+QuickETL = Quicken Extract Transform and Load
+QuickETL - Copyright 2008 by Chris Joakim.
+QuickETL is available under GNU General Public License (GPL) license.
+
+-
+  
+This class is used to interface with the sqlite3 database.
+
+=end
+
+module QuickETL
+
+  class DBO < QuickETLObject
+   
+    attr_accessor :db , :db_filename
+    
+    def initialize()
+      @db_filename = "#{home_dir}/#{QuickETL::DATABASE_FILE}"
+    end
+    
+    def open_database
+      @db = SQLite3::Database.new(@db_filename)
+    end
+
+    def show_ddl
+      puts "\nThe following DDL (SQL) is used to create database #{@db_filename}"
+      puts ""
+      puts drop_transactions_ddl
+      puts create_transactions_ddl  
+      puts ""
+    end
+    
+    def drop_and_create_database
+      open_database
+      puts @db.execute(drop_transactions_ddl);
+      puts @db.execute(create_transactions_ddl);  
+      puts "\nDatabase has been created - #{@db_filename}"
+      puts "Database schema is shown below:"
+      puts ""
+      puts `sqlite3 #{@db_filename} VACUUM`      
+      puts `sqlite3 #{@db_filename} .schema`
+      puts ""
+    end
+
+    def drop_transactions_ddl   
+      "drop table if exists transactions;"
+    end
+    
+    def create_transactions_ddl      
+      s =  "create table transactions ( \n" 
+      s << "  id              INTEGER PRIMARY KEY, \n" 
+      s << "  account_owner   TEXT, \n"       
+      s << "  account_name    TEXT, \n"  
+      s << "  account_type    TEXT, \n"  
+      s << "  tran_date       TEXT, \n"  
+      s << "  cleared_status  TEXT, \n"  
+      s << "  item_number     INTEGER, \n"  
+      s << "  amount          REAL, \n" 
+      s << "  payee           TEXT, \n"  
+      s << "  category        TEXT, \n"         
+      s << "  memo            TEXT, \n"  
+      s << "  line_number     INTEGER \n" 
+      s << ");" 
+      s
+    end
+    
+    def load_database
+      command_file = "#{home_dir}/#{LOAD_COMMAND_FILE}"
+      lines =  Array.new
+      lines << ".import #{home_dir}/quick-etl.csv transactions"
+      write_lines(command_file, lines)
+      puts `sqlite3 #{@db_filename} < #{command_file}`
+    end
+    
+    def query_all_transactions
+      command_file = "#{home_dir}/#{QUERY_ALL_FILE}"
+      results_file = "#{home_dir}/output.txt"
+      lines =  Array.new
+      lines << "select * from transactions;"
+      write_lines(command_file, lines)
+      puts `sqlite3 #{@db_filename} < #{command_file} > #{results_file}`
+    end
+    
+  end
+  
+end 

data/lib/quick_etl_kernel.rb

@@ -0,0 +1,295 @@
+=begin rdoc
+
+QuickETL = Quicken Extract Transform and Load
+QuickETL - Copyright 2008 by Chris Joakim.
+QuickETL is available under GNU General Public License (GPL) license.
+
+-
+  
+This is the core module for this project, and is included in base class
+QuickETLObject.
+
+=end
+
+module QuickETL
+    
+  DATABASE_FILE      = 'quick-etl.db'  
+  PROPERTIES_FILE    = 'quick-etl.properties'
+  LOAD_COMMAND_FILE  = 'quick-etl.load' 
+  QUERY_ALL_FILE     = 'query_all.sql'    
+  DEFAULT_QIF_FILE   = 'QuickenData.qif'
+  DAY_NAMES          = %w(Sun Mon Tue Wed Thu Fri Sat)
+  QUICK_ETL_HOME     = 'QUICK_ETL_HOME'
+  QUICK_ETL_DEV_HOME = 'QUICK_ETL_DEV_HOME'
+
+  @@properties = Hash.new
+  @@debug      = false
+    
+  module QuickETLKernel
+
+    def project_name
+      'QuickETL'
+    end
+    
+    def project_version_number   
+      '0.1.0'
+    end
+
+    def project_date       
+      '2008/02/10'
+    end
+
+    def project_author
+      'Chris Joakim'
+    end
+  
+    def project_year       
+      project_date[0...4]  # start, length
+    end
+ 
+    def project_copyright
+      "Copyright (C) #{project_year} #{project_author}"
+    end
+
+    def project_embedded_comment
+      "#{project_name} #{project_version_number}"
+    end
+
+    def project_license    
+      'GNU General Public License (GPL).  See http://www.gnu.org/copyleft/gpl.html'
+    end
+
+    def display_general_info
+      puts ""
+      puts "This is '#{project_name}'. " 
+      puts "#{project_copyright} " 
+      puts "#{project_license} "   
+      puts ""
+      puts "QuickETL home pages: "
+      puts "  -- http://rubyforge.org/projects/quick-etl/ "  
+      puts "  -- http://www.joakim-systems.com/ "    
+      puts ""
+      puts "QuickETL uses the SQLite3 database; see the following URLs: "
+      puts "  -- http://www.sqlite.org/"
+      puts "  -- http://sqlite-ruby.rubyforge.org/ "
+      puts "  -- http://sqlite-ruby.rubyforge.org/sqlite3/faq.html "
+      puts ""
+      puts "Enter 'rake help' for QuickETL usage instructions. "
+      puts "Enter 'rake -T' for the list of available QuickETL rake commands. "  
+      puts ""
+      
+      home = home_dir
+      if (home_dir)
+        puts "Your QuickETL home directory is currently set to: #{home_dir}"  
+      else
+        puts "Warning: You have not yet set your 'QUICK_ETL_HOME' environment variable."
+      end
+      puts ""
+    end
+    
+    def display_usage_instructions
+      puts "usage"
+      puts ""
+    end
+    
+    def home_dir
+      is_dev = ENV['dev']
+      if ((is_dev) && (is_dev == 'y'))
+        ENV[QUICK_ETL_DEV_HOME]
+      else
+        ENV[QUICK_ETL_HOME]
+      end
+    end
+    
+    def command_line_arg(name, default)
+      value = default
+      if name
+        value = ENV[name]
+      value = default  if value == nil
+      end
+      value 
+    end
+      
+    def command_line_boolean_arg(name, default)
+      v = command_line_arg(name, "#{default}")
+      return true  if ((v.downcase == 'true') || (v.downcase == 't'))
+      return true  if ((v.downcase == 'yes')  || (v.downcase == 'y'))
+      false
+    end
+      
+    def command_line_array_arg(name, default)
+      s = command_line_arg(name, nil)
+      return default  if s == nil
+      return s.split(',')
+    end
+    
+    def get_property(name, default='')
+      if @@properties.has_key? "#{name}"
+        return @@properties["#{name}"]
+      end
+      default
+    end
+
+    def get_array_property(name, default=Array.new)
+      s = get_property(name)
+      if s
+        return s.split(',')
+      end
+      default
+    end
+ 
+    def startup
+      load_properties("#{home_dir}/#{QuickETL::PROPERTIES_FILE}") 
+    end
+
+    def load_properties(properties_filename)
+      props = Hash.new
+      if File.exist? properties_filename
+        File.open(properties_filename, 'r') do | properties_file |
+          properties_file.read.each_line do |line|
+            line.strip!
+            if (line[0] != ?# and line[0] != ?=)
+              i = line.index('=')
+              if (i)
+                props[line[0..i - 1].strip] = line[i + 1..-1].strip
+              else
+                props[line] = ''
+              end
+            end
+          end
+          puts "properties file read - #{properties_filename} , entry count=#{props.size}"  if true
+        end
+      else
+        puts "properties file does not exist - #{properties_filename}"  if false
+      end
+      @@properties = props
+    end
+    
+    def parse_qif_files
+      qif_file = command_line_arg('qif', QuickETL::DEFAULT_QIF_FILE)
+      qif_dir  = get_property('qif_dir', home_dir)
+      qif_path = "#{qif_dir}/#{qif_file}"
+      owner  = command_line_arg('owner', 'owner')      
+      reader = QuickETL::QifFileReader.new(owner, qif_path)
+      array  = Array.new
+      reader.transactions.each_with_index { | tran, idx| 
+        array << "#{idx+1}#{tran.to_s}"  
+      }  
+      write_lines("#{home_dir}/quick-etl.csv", array)
+    end
+      
+    def read_lines(filename, strip=false)  
+      array = IO.readlines(filename)
+      array = strip_lines(array)  if strip
+      array
+    end
+  
+    def write_file(out_name, content)
+      out = File.new out_name, "w+"
+      out.write content
+      out.flush
+      out.close  
+      puts "file written: #{out_name}"  
+    end
+
+    def write_lines(out_name, lines)
+      s = ''
+      lines.each { | line | s << "#{line}\n" }
+      write_file(out_name, s)
+    end
+
+    def read_as_ascii_lines(filename, delim=10, strip=false)  
+      array = Array.new  
+      file  = File.new(filename)
+      currLine = ''
+      bytesRead = 0
+      linesRead = 0
+      
+      file.each_byte { |b| 
+        bytesRead = bytesRead + 1
+        if (b == delim) # delim is 13 for quicken, 10 for address book xml
+          array << currLine
+          currLine = ''
+          linesRead = linesRead + 1
+        else
+          if (b < 127)
+            currLine << "#{b.chr}"
+          end
+        end  
+      }
+      if currLine.size > 0
+        array << currLine
+      end
+      if strip
+        array = strip_lines(array)
+      end
+      array    
+    end
+
+    def strip_lines(array)  
+      newArray = Array.new
+      if (array != nil)
+        array.each { |line| line.strip! ; newArray << line }
+      end
+      newArray
+    end
+  
+    def line_value(s)
+      if ((s == nil) || (s.size < 1))
+        s
+      else
+        s[1, s.size]
+      end
+    end
+    
+    def tokenize(string, delim=nil, strip=false)  
+      if string
+        tokens = string.split(delim)
+        if strip
+          tokens.each { |tok| tok.strip! }
+        end
+        tokens
+      else
+        Array.new
+      end
+    end
+    
+    def default_delimiter
+      '|'
+    end
+    
+    def usage_instructions
+      s = <<HERE
+      
+1.  Use your Quicken program to export its data to the QUICK_ETL_HOME directory.
+    Specify output file 'QuickenData.qif'.
+
+2.  Open a shell window in your QUICK_ETL_HOME directory.
+
+    Execute command 'rake -T' to see the list of functions available.
+    
+    Execute command 'rake parse' to parse the qif file into csv format.
+    
+    Execute command 'rake load' to load the csv file data into sqlite3 database 'quick-etl.db'.
+
+    Execute command 'rake parse_and_load' to run both the parsing and loading function.
+    
+    Execute command 'rake show_ddl' to see the structure of the database table and its column names.
+        
+3.  Use the sqlite3 CLP (Command Line Program) to query the database.
+    From the QUICK_ETL_HOME directory, execute command 'sqlite3 quick-etl.db'.
+    
+    You can then enter and execute queries like the following:
+    
+    sqlite> select * from transactions;
+    sqlite> select * from transactions where tran_date == '2008-02-05';
+    
+    Note: A User Interface will be created for QuickETL its next release.    
+        
+HERE
+      s
+    end
+    
+  end
+
+end 

data/lib/quick_etl_money.rb

@@ -0,0 +1,48 @@
+=begin rdoc
+
+QuickETL = Quicken Extract Transform and Load
+QuickETL - Copyright 2008 by Chris Joakim.
+QuickETL is available under GNU General Public License (GPL) license.
+
+-
+  
+Instances of this class represent a monetary amount from within a Quicken qif file,
+such as 'T-38.09' (i.e. - $38.09 US Dollars).
+
+=end
+
+module QuickETL
+
+  class Money < QuickETLObject
+    
+    attr_accessor :string_value, :numeric_value
+
+    def initialize(s='0.00')
+      s ? @string_value = s : @string_value = '0.00'
+      @string_value.strip!
+      @numeric_value = numeric(@string_value).to_f
+    end
+
+    def to_s
+      "#{@numeric_value}"
+    end
+    
+    private 
+    
+    def numeric(s)
+      n = ''
+      if s
+        s.each_byte { | b |
+          if ((b == 44) || (b == 32))
+            # bypass commas (44) and spaces (32)
+          else
+            n << "#{b.chr}"
+          end
+        }
+      end
+      n
+    end
+    
+  end
+  
+end 

data/lib/quick_etl_object.rb

@@ -0,0 +1,20 @@
+=begin rdoc
+
+QuickETL = Quicken Extract Transform and Load
+QuickETL - Copyright 2008 by Chris Joakim.
+QuickETL is available under GNU General Public License (GPL) license.
+
+-
+  
+This is the abstract superclass of several QuickETL classes.
+It includes module QuickETLKernel.
+
+=end
+
+module QuickETL
+  
+  class QuickETLObject  
+    include QuickETL::QuickETLKernel
+  end
+  
+end

data/lib/quick_etl_qif_date.rb

@@ -0,0 +1,44 @@
+=begin rdoc
+
+QuickETL = Quicken Extract Transform and Load
+QuickETL - Copyright 2008 by Chris Joakim.
+QuickETL is available under GNU General Public License (GPL) license.
+
+-
+  
+Instances of this class represent a date from within a Quicken qif file,
+such as 'D5/24/94'.
+
+=end
+
+module QuickETL
+
+  class QifDate < QuickETLObject
+
+    attr_accessor :raw_line, :ccyymmdd, :year, :year_mm, :yy, :mm, :dd
+    
+    def initialize(raw_line)
+      @raw_line = raw_line
+      @y, @m, @d  =  0, 0, 0
+      @cc, @yy, @mm, @dd = '00', '00', '00', '00'
+      
+      if @raw_line
+        tokens = @raw_line.split('/')
+        if (tokens && tokens.size > 2)
+          m   = tokens[0].to_i
+          d   = tokens[1].to_i
+          y   = tokens[2].to_i
+          @yy = tokens[2]
+          y < 50 ? @cc = "20" : @cc = "19"
+          m < 10 ? @mm = "0#{m}" : @mm = "#{m}"
+          d < 10 ? @dd = "0#{d}" : @dd = "#{d}"
+        end
+      end
+      @ccyymmdd = "#{@cc}#{@yy}-#{@mm}-#{@dd}"
+      @year     = "#{@cc}#{@yy}"
+      @year_mm  = "#{@cc}#{@yy}-#{@mm}"
+    end
+    
+  end
+
+end 

data/lib/quick_etl_qif_file_reader.rb

@@ -0,0 +1,90 @@
+=begin rdoc
+
+QuickETL = Quicken Extract Transform and Load
+QuickETL - Copyright 2008 by Chris Joakim.
+QuickETL is available under GNU General Public License (GPL) license.
+
+-
+  
+Instances of this class are used to read and parse a Quicken qif file.
+
+=end
+
+module QuickETL
+
+  class QifFileReader < QuickETLObject
+  
+    attr_accessor :filename, :transactions, :debug
+  
+    def initialize(owner, filename, debug=false)
+      @owner    = owner
+      @filename = filename
+      @debug    = debug
+      @transactions = Array.new
+      read
+    end
+  
+    def read
+      @lines = read_as_ascii_lines(@filename, 13, false)
+      in_tran_zone    = false
+      in_acct_header  = false    
+      @acct_name   = ''
+      @acct_type   = '' 
+      line_num     = 0
+      tran         = QifTran.new(@owner, @acct_name, @acct_type, 0)   
+      start_token  = '!Type:Class'
+    
+      @lines.each { | line |
+        line_num = line_num + 1
+      
+        if (line == start_token) 
+          in_tran_zone = true
+          next
+        end
+      
+        if (line.match(/^!Account/))
+          in_acct_header = true
+          next
+        end
+        
+        if in_acct_header
+          if (line.match(/^N/))
+            @acct_name = (line_value(line))
+            tran.acct_name = @acct_name
+          elsif (line.match(/^T/))
+            @acct_type = (line_value(line)) 
+            tran.acct_type = @acct_type        
+          elsif (line == '^')
+            in_acct_header = false
+          end
+          next
+        end
+            
+        if in_tran_zone       
+          if (line == '^')
+            add_transaction(tran)
+            tran = QifTran.new(@owner, @acct_name, @acct_type, line_num)
+          else     
+            tran.add_line(line)
+          end
+        end
+      }   
+    end
+  
+    def add_transaction(t)
+      t.parse
+      if t.valid
+        @transactions.push(t)
+      else
+        puts "invalid trans"
+      end
+    end
+    
+    public 
+  
+    def to_s
+      "QifFileReader #{@filename} #{@lines.size} #{@transactions.size}"
+    end  
+  end
+
+end 

data/lib/quick_etl_qif_transaction.rb

@@ -0,0 +1,91 @@
+=begin rdoc
+
+QuickETL = Quicken Extract Transform and Load
+QuickETL - Copyright 2008 by Chris Joakim.
+QuickETL is available under GNU General Public License (GPL) license.
+
+-
+  
+Instances of this class represent one transaction within a Quicken qif file.
+
+=end
+
+module QuickETL
+
+  class QifTran < QuickETLObject
+  
+    attr_accessor :raw_lines, :acct_owner, :acct_name, :acct_type, :date, :amount
+    attr_accessor :valid, :balance, :cleared, :category, :number, :payee, :memo
+  
+    def initialize(anOwner, aName, aType, line_num, delim='|')
+      @acct_owner = "#{anOwner}"
+      @acct_name  = "#{aName}"
+      @acct_type  = "#{aType}"
+      @line_num   = line_num
+      @delim      = delim
+      @valid      = false
+      @cleared    = ' '
+      @number     = ' '
+      @memo       = ' '
+      @raw_lines  = Array.new 
+      @splits     = Array.new    
+    end
+  
+    public    
+  
+    def add_line(line)    
+      @raw_lines.push(line) 
+    end
+
+=begin
+    Field Indicator Explanations: 
+    D Date 
+    T Amount 
+    C Cleared status 
+    N Num (check or reference number)  
+    P Payee 
+    M Memo 
+    A Address (up to five lines; the sixth line is an optional message)     NOT IMPLEMENTED IN QUICK-ETL
+    L Category (Category/Subcategory/Transfer/Class)  
+    S Category in split (Category/Transfer/Class)  
+    E Memo in split 
+    $ Dollar amount of split 
+    ^ End of the entry
+=end
+  
+    def parse
+      @raw_lines.each { | line |
+        if (line.match(/^D/))
+          @date = QifDate.new(line_value(line))
+        elsif (line.match(/^T/))
+          s = line_value(line)
+          @amount = Money.new(s)
+        elsif (line.match(/^P/))
+          @payee = line_value(line)
+        elsif (line.match(/^C/))
+          @cleared = line_value(line)
+        elsif (line.match(/^N/))
+          @number = line_value(line)
+        elsif (line.match(/^M/))
+          @memo = line_value(line)
+        elsif (line.match(/^L/))
+          @category = line_value(line)
+        elsif (line.match(/^S/))
+          @split_category = line_value(line)
+        elsif (line.match(/^E/))
+          @split_memo = line_value(line)
+        elsif (line.match(/^$/))
+          @split_amount = Money.new(line_value(line))
+        end                     
+      }
+      if (@date && @amount && @payee)
+        @valid = true
+      end    
+    end 
+  
+    def to_s
+      "#{@delim}#{@acct_owner}#{@delim}#{@acct_name}#{@delim}#{@acct_type}#{@delim}#{@date.ccyymmdd}#{@delim}#{@cleared}#{@delim}#{@number}#{@delim}#{@amount.to_s}#{@delim}#{@payee}#{@delim}#{@category}#{@delim}#{@memo}#{@delim}#{@line_num}"
+    end
+  end
+
+end 

data/quick-etl.properties

@@ -0,0 +1,3 @@
+qif_files=c
+qif_file_c=QuickenData.qif
+qif_file_t=TQuickenData.qif

data/rakefile.rb

@@ -0,0 +1,79 @@
+=begin
+
+Ruby Rakefile for Quick-ETL.
+
+Quick-ETL - Copyright 2008 by Chris Joakim.
+Quick-ETL is available under GNU General Public License (GPL) license.
+
+http://sqlite-ruby.rubyforge.org/
+http://sqlite-ruby.rubyforge.org/sqlite3/faq.html
+=end
+
+require 'rubygems'
+require 'rake'
+require 'rbconfig'
+require 'date'
+require 'time'
+require 'sqlite3'
+
+# The following prefixes /lib to the active ruby load path
+#$:.unshift File.join(File.dirname(__FILE__), "", "lib")
+require 'quick_etl.rb'
+
+desc "Display information about this open-source project."
+task :default do
+  obj = QuickETL::QuickETLObject.new
+  obj.display_general_info
+end
+
+desc "Display QuickETL usage instructions."
+task :help do
+  obj = QuickETL::QuickETLObject.new
+  puts obj.usage_instructions
+end
+
+desc "Show the DDL (SQL) used to create the sqlite3 'quick-etl.db' database."
+task :show_ddl do
+  dbo = QuickETL::DBO.new
+  dbo.show_ddl
+end
+
+desc "Drop and (re)create the sqlite3 'quick-etl.db' database."
+task :drop_create_db do
+  startup
+  dbo = QuickETL::DBO.new  
+  dbo.drop_and_create_database
+end
+
+desc "Parse *qif files into csv per properties file."
+task :parse do
+  startup
+  obj = QuickETL::QuickETLObject.new  
+  obj.parse_qif_files
+end
+
+desc "Parse *qif files into csv per properties file."
+task :load do
+  startup
+  dbo = QuickETL::DBO.new    
+  dbo.drop_and_create_database
+  dbo.load_database
+end
+
+desc "Parse *qif files into csv per properties file."
+task :parse_and_load do
+  startup
+
+  obj = QuickETL::QuickETLObject.new  
+  obj.parse_qif_files
+  
+  dbo = QuickETL::DBO.new    
+  dbo.drop_and_create_database
+  dbo.load_database
+end
+
+# -----------------------------------------------------------------------------
+
+def startup
+  QuickETL::QuickETLObject.new.startup
+end

metadata

@@ -0,0 +1,59 @@
+--- !ruby/object:Gem::Specification 
+rubygems_version: 0.9.4
+specification_version: 1
+name: quick-etl
+version: !ruby/object:Gem::Version 
+  version: 0.1.0
+date: 2008-02-10 00:00:00 -05:00
+summary: Quicken Extract Transform and Load
+require_paths: 
+- lib
+email: chris@joakim-systems.com
+homepage: http://rubyforge.org/projects/quick-etl/
+rubyforge_project: quick-etl
+description: 
+autorequire: 
+default_executable: 
+bindir: bin
+has_rdoc: false
+required_ruby_version: !ruby/object:Gem::Version::Requirement 
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      version: 1.8.4
+  version: 
+platform: ruby
+signing_key: 
+cert_chain: 
+post_install_message: 
+authors: 
+- Chris Joakim
+files: 
+- doc/qifFormatDoc.txt
+- lib/quick_etl.rb
+- lib/quick_etl_dbo.rb
+- lib/quick_etl_kernel.rb
+- lib/quick_etl_money.rb
+- lib/quick_etl_object.rb
+- lib/quick_etl_qif_date.rb
+- lib/quick_etl_qif_file_reader.rb
+- lib/quick_etl_qif_transaction.rb
+- README
+- rakefile.rb
+- quick-etl.properties
+test_files: []
+
+rdoc_options: []
+
+extra_rdoc_files: 
+- README
+- rakefile.rb
+- quick-etl.properties
+executables: []
+
+extensions: []
+
+requirements: []
+
+dependencies: []
+