ofxtools 0.5.1__tar.gz → 0.5.2__tar.gz

{ofxtools-0.5.1/ofxtools.egg-info → ofxtools-0.5.2}/PKG-INFO

@@ -1,13 +1,12 @@
 Metadata-Version: 1.1
 Name: ofxtools
-Version: 0.5.1
+Version: 0.5.2
 Summary: Library for working with Open Financial Exchange (OFX) formatted data used by financial institutions
 Home-page: https://github.com/csingley/ofxtools
 Author: Christopher Singley
 Author-email: csingley@gmail.com
 License: MIT
-Download-URL: https://github.com/csingley/ofxtools/tarball/0.5.1
-Description-Content-Type: UNKNOWN
+Download-URL: https://github.com/csingley/ofxtools/tarball/0.5.2
 Description: ====================
         OFX tools for Python
         ====================
@@ -27,7 +26,7 @@ Description: ====================
            ElementTree structure, then converts the parsed data into normal Python
            types (e.g. datetime.datetime, list) and exposes them through more Pythonic
            attribute access (e.g. ``OFX.statements[0].ledgerbal``); and
-        -  An (optional) ``ofxalchemy`` object model that persists OFX data in an
+        -  A (deprecated) ``ofxalchemy`` object model that persists OFX data in an
            SQL database where they can be queried.
         
         Installation
@@ -39,18 +38,20 @@ Description: ====================
             pip install ofxtools
         
         To install the most recent prerelease (which is where the magic happens, and
-        also the bugs), you can download the `current master`_ and unzip it.  Since
-        dependencies are minimal, it's recommended to use the Python user installation
-        scheme:
+        also the bugs), you can download the `current master`_, unzip it, and install
+        via the included setup file:
         
         ::
         
-            python setup.py install --user
+            python setup.py install
         
         In addition to the Python package, this will also install a script ``ofxget``
         in ``~/.local/bin``, and its sample configuration file in
         ``~/.config/ofxtools``.
         
+        For any installation method, it's recommended to install ``ofxtools`` in a
+        `virtual environment`_.
+        
         OFX Client
         ==========
         
@@ -181,6 +182,129 @@ Description: ====================
             In [23]: ET.tostring(tree)[:512]  # Back to str
             Out[23]: b'<OFX><SIGNONMSGSRSV1><SONRS><STATUS><CODE>0</CODE><SEVERITY>INFO</SEVERITY><MESSAGE>Success</MESSAGE></STATUS><DTSERVER>20170421170513</DTSERVER><LANGUAGE>ENG</LANGUAGE><FI><ORG>Ameritrade Technology Group</ORG><FID>AIS</FID></FI></SONRS></SIGNONMSGSRSV1><INVSTMTMSGSRSV1><INVSTMTTRNRS><TRNUID>2a656f1c-5f86-4265-84f1-6c7f0dc8c370</TRNUID><STATUS><CODE>0</CODE><SEVERITY>INFO</SEVERITY><MESSAGE>pr-ctlvofx-pp03-clientsys Success</MESSAGE></STATUS><INVSTMTRS><DTASOF>20170401030609</DTASOF><CURDEF>USD</CURDEF><IN'
         
+        ofxtools and SQL
+        ================
+        
+        ``ofxtools`` does include the ``ofxalchemy`` subpackage, but you probably don't
+        want to use it.  The implementation is pretty nasty, and the SQL tables it
+        generates are cumbersome and inefficient.
+        
+        Frankly, OFX is a poor data format for serious use, as is obvious to anyone
+        who's tried to work with its handling of online bill payees or securities
+        reorganizations.  A better approach is to decouple your SQL data model from
+        OFX, which will also allow you to better handle other financial data formats.
+        
+        It's recommended to define your own ORM models based on your needs.  Import
+        OFX into Python using the main ``ofxtools.Parser.OFXTree`` parser, extract
+        the relevant data, and feed it to your model classes.  Something like this:
+        
+        .. code:: python
+        
+            from sqlalchemy.ext.declarative import declarative_base
+            from sqlalchemy import (
+                Column, Integer, String, Text, DateTime, Numeric, ForeignKey, Enum,
+            )
+            from sqlalchemy.orm import (relationship, sessionmaker, )
+            from sqlalchemy import create_engine
+        
+            from ofxtools.models.i18n import CURRENCY_CODES
+            from ofxtools.Client import (OFXClient, InvStmtRq, )
+            from ofxtools.Parser import OFXTree
+            from ofxtools.models.investment import (BUYSTOCK, SELLSTOCK)
+        
+        
+            # Data model
+            Base = declarative_base()
+        
+        
+            class Account(Base):
+                id = Column(Integer, primary_key=True)
+                brokerid = Column(String, nullable=False, unique=True)
+                number = Column(String, nullable=False)
+                name = Column(String)
+        
+        
+            class Security(Base):
+                id = Column(Integer, primary_key=True)
+                name = Column(String)
+                ticker = Column(String)
+                uniqueidtype = Column(String, nullable=False)
+                uniqueid = Column(String, nullable=False)
+        
+        
+            class Transaction(Base):
+                id = Column(Integer, primary_key=True)
+                uniqueid = Column(String, nullable=False, unique=True)
+                datetime = Column(DateTime, nullable=False)
+                dtsettle = Column(DateTime)
+                type = Column(Enum('returnofcapital', 'split', 'spinoff', 'transfer',
+                                   'trade', 'exercise', name='transaction_type'),
+                              nullable=False)
+                memo = Column(Text)
+                currency = Column(Enum(*CURRENCY_CODES, name='transaction_currency'))
+                cash = Column(Numeric)
+                account_id = Column(Integer,
+                                    ForeignKey('account.id', onupdate='CASCADE'),
+                                    nullable=False)
+                account = relationship('Account', foreign_keys=[account_id],
+                                       backref='transactions')
+                security_id = Column(Integer,
+                                     ForeignKey('security.id', onupdate='CASCADE'),
+                                     nullable=False)
+                security = relationship('Security', foreign_keys=[security_id],
+                                        backref='transactions')
+                units = Column(Numeric)
+        
+        
+            # Import
+            client = OFXClient('https://ofxs.ameritrade.com/cgi-bin/apps/OFX',
+                               org='Ameritrade Technology Group', fid='AIS',
+                               brokerid='ameritrade.com')
+            stmtrq = InvStmtRq(acctid='999999999')
+            response = client.request_statements(user='elmerfudd',
+                                                 password='T0PS3CR3T', invstmtrqs=[stmtrq])
+            parser = OFXTree()
+            parser.parse(response)
+            ofx = parser.convert()
+        
+        
+            # Extract
+            def make_security(secinfo):
+                return Security(
+                    name=secinfo.secname, ticker=secinfo.ticker,
+                    uniqueidtype=secinfo.uniqueidtype, uniqueid=secinfo.uniqueid)
+        
+        
+            securities = {(sec.uniqueidtype, sec.uniqueid): make_security(sec)
+                          for sec in ofx.securities}
+        
+        
+            stmt = ofx.statements[0]
+            account = Account(brokerid=stmt.brokerid, number=stmt.acctid)
+        
+        
+            def make_trade(invtran):
+                security = securities[(invtran.uniqueidtype, invtran.uniqueid)]
+                return Transaction(
+                    uniqueid=invtran.fitid, datetime=invtran.dttrade,
+                    dtsettle=invtran.dtsettle, type='trade', memo=invtran.memo,
+                    currency=invtran.currency, cash=invtran.total, account=account,
+                    security=security, units=invtran.units)
+        
+        
+            trades = [make_trade(tx) for tx in stmt.transactions
+                      if isinstance(tx, (BUYSTOCK, SELLSTOCK))]  # dispatch by model class
+        
+        
+            # Persist
+            engine = create_engine('')
+            Session = sessionmaker(bind=engine)
+            session = Session()
+            session.add(account)
+            session.add_all(securities.values())
+            session.add_all(trades)
+            session.commit()
+        
         Contributing
         ============
         
@@ -191,16 +315,20 @@ Description: ====================
         
             git clone https://github.com/csingley/ofxtools.git
         
-        Feel free to `create pull requests`_ on `ofxtools repository on GitHub`_.
+        Set up a `virtual environment`_, and install the package in development mode
+        so you're working on live code:
         
-        Again, the minimal dependencies make it simple to install with the Python user
-        installation scheme:
+        ::
+        
+            python setup.py develop
+        
+        Install all development requirements:
         
         ::
         
-            python setup.py develop --user
+            pip install -r requirements-development.txt
         
-        However, you'll want to run the tests, either with ``make``:
+        Run the tests, either with ``make``:
         
         ::
         
@@ -210,18 +338,10 @@ Description: ====================
         
         ::
         
-            nosetests -dsv --with-yanc --with-coverage --cover-package ofxtools
-        
-        
-        if you don't already have `nose`, `coverage`, etc. installed, and you don't
-        want to clutter your system libraries just for this work, you can create a
-        `virtual environment`_ and install all development requirements:
+            nosetests -dsv  --with-coverage --cover-package ofxtools
         
-        ::
+        Feel free to `create pull requests`_ on `ofxtools repository on GitHub`_.
         
-            virtualenv .venv
-            source .venv/bin/activate
-            pip install -r requirements-development.txt
         
         Resources
         =========
@@ -236,14 +356,15 @@ Description: ====================
         .. _Requests: http://docs.python-requests.org/en/master/
         .. _PyPI: https://pypi.python.org/pypi/ofxtools
         .. _current master: https://github.com/csingley/ofxtools/archive/master.zip
+        .. _virtual environment: https://packaging.python.org/tutorials/installing-packages/#creating-virtual-environments
         .. _OFX spec: http://www.ofx.net/downloads.html
         .. _Git: https://git-scm.com/
         .. _create pull requests: https://help.github.com/articles/using-pull-requests/
         .. _ofxtools repository on GitHub: https://github.com/csingley/ofxtools
-        .. _virtual environment: http://docs.python-guide.org/en/latest/dev/virtualenvs/
         .. _Quicken data mapping guide: https://fi.intuit.com/downloads/QW_DataMappingGuide.pdf
         .. _OFX Home: http://www.ofxhome.com/
         
+        
 Keywords: ofx,Open Financial Exchange
 Platform: UNKNOWN
 Classifier: Development Status :: 3 - Alpha

{ofxtools-0.5.1 → ofxtools-0.5.2}/README.rst

@@ -17,7 +17,7 @@ The primary facilities provided include:
    ElementTree structure, then converts the parsed data into normal Python
    types (e.g. datetime.datetime, list) and exposes them through more Pythonic
    attribute access (e.g. ``OFX.statements[0].ledgerbal``); and
--  An (optional) ``ofxalchemy`` object model that persists OFX data in an
+-  A (deprecated) ``ofxalchemy`` object model that persists OFX data in an
    SQL database where they can be queried.
 
 Installation
@@ -29,18 +29,20 @@ Installation
     pip install ofxtools
 
 To install the most recent prerelease (which is where the magic happens, and
-also the bugs), you can download the `current master`_ and unzip it.  Since
-dependencies are minimal, it's recommended to use the Python user installation
-scheme:
+also the bugs), you can download the `current master`_, unzip it, and install
+via the included setup file:
 
 ::
 
-    python setup.py install --user
+    python setup.py install
 
 In addition to the Python package, this will also install a script ``ofxget``
 in ``~/.local/bin``, and its sample configuration file in
 ``~/.config/ofxtools``.
 
+For any installation method, it's recommended to install ``ofxtools`` in a
+`virtual environment`_.
+
 OFX Client
 ==========
 
@@ -171,6 +173,129 @@ Usage Example
     In [23]: ET.tostring(tree)[:512]  # Back to str
     Out[23]: b'<OFX><SIGNONMSGSRSV1><SONRS><STATUS><CODE>0</CODE><SEVERITY>INFO</SEVERITY><MESSAGE>Success</MESSAGE></STATUS><DTSERVER>20170421170513</DTSERVER><LANGUAGE>ENG</LANGUAGE><FI><ORG>Ameritrade Technology Group</ORG><FID>AIS</FID></FI></SONRS></SIGNONMSGSRSV1><INVSTMTMSGSRSV1><INVSTMTTRNRS><TRNUID>2a656f1c-5f86-4265-84f1-6c7f0dc8c370</TRNUID><STATUS><CODE>0</CODE><SEVERITY>INFO</SEVERITY><MESSAGE>pr-ctlvofx-pp03-clientsys Success</MESSAGE></STATUS><INVSTMTRS><DTASOF>20170401030609</DTASOF><CURDEF>USD</CURDEF><IN'
 
+ofxtools and SQL
+================
+
+``ofxtools`` does include the ``ofxalchemy`` subpackage, but you probably don't
+want to use it.  The implementation is pretty nasty, and the SQL tables it
+generates are cumbersome and inefficient.
+
+Frankly, OFX is a poor data format for serious use, as is obvious to anyone
+who's tried to work with its handling of online bill payees or securities
+reorganizations.  A better approach is to decouple your SQL data model from
+OFX, which will also allow you to better handle other financial data formats.
+
+It's recommended to define your own ORM models based on your needs.  Import
+OFX into Python using the main ``ofxtools.Parser.OFXTree`` parser, extract
+the relevant data, and feed it to your model classes.  Something like this:
+
+.. code:: python
+
+    from sqlalchemy.ext.declarative import declarative_base
+    from sqlalchemy import (
+        Column, Integer, String, Text, DateTime, Numeric, ForeignKey, Enum,
+    )
+    from sqlalchemy.orm import (relationship, sessionmaker, )
+    from sqlalchemy import create_engine
+
+    from ofxtools.models.i18n import CURRENCY_CODES
+    from ofxtools.Client import (OFXClient, InvStmtRq, )
+    from ofxtools.Parser import OFXTree
+    from ofxtools.models.investment import (BUYSTOCK, SELLSTOCK)
+
+
+    # Data model
+    Base = declarative_base()
+
+
+    class Account(Base):
+        id = Column(Integer, primary_key=True)
+        brokerid = Column(String, nullable=False, unique=True)
+        number = Column(String, nullable=False)
+        name = Column(String)
+
+
+    class Security(Base):
+        id = Column(Integer, primary_key=True)
+        name = Column(String)
+        ticker = Column(String)
+        uniqueidtype = Column(String, nullable=False)
+        uniqueid = Column(String, nullable=False)
+
+
+    class Transaction(Base):
+        id = Column(Integer, primary_key=True)
+        uniqueid = Column(String, nullable=False, unique=True)
+        datetime = Column(DateTime, nullable=False)
+        dtsettle = Column(DateTime)
+        type = Column(Enum('returnofcapital', 'split', 'spinoff', 'transfer',
+                           'trade', 'exercise', name='transaction_type'),
+                      nullable=False)
+        memo = Column(Text)
+        currency = Column(Enum(*CURRENCY_CODES, name='transaction_currency'))
+        cash = Column(Numeric)
+        account_id = Column(Integer,
+                            ForeignKey('account.id', onupdate='CASCADE'),
+                            nullable=False)
+        account = relationship('Account', foreign_keys=[account_id],
+                               backref='transactions')
+        security_id = Column(Integer,
+                             ForeignKey('security.id', onupdate='CASCADE'),
+                             nullable=False)
+        security = relationship('Security', foreign_keys=[security_id],
+                                backref='transactions')
+        units = Column(Numeric)
+
+
+    # Import
+    client = OFXClient('https://ofxs.ameritrade.com/cgi-bin/apps/OFX',
+                       org='Ameritrade Technology Group', fid='AIS',
+                       brokerid='ameritrade.com')
+    stmtrq = InvStmtRq(acctid='999999999')
+    response = client.request_statements(user='elmerfudd',
+                                         password='T0PS3CR3T', invstmtrqs=[stmtrq])
+    parser = OFXTree()
+    parser.parse(response)
+    ofx = parser.convert()
+
+
+    # Extract
+    def make_security(secinfo):
+        return Security(
+            name=secinfo.secname, ticker=secinfo.ticker,
+            uniqueidtype=secinfo.uniqueidtype, uniqueid=secinfo.uniqueid)
+
+
+    securities = {(sec.uniqueidtype, sec.uniqueid): make_security(sec)
+                  for sec in ofx.securities}
+
+
+    stmt = ofx.statements[0]
+    account = Account(brokerid=stmt.brokerid, number=stmt.acctid)
+
+
+    def make_trade(invtran):
+        security = securities[(invtran.uniqueidtype, invtran.uniqueid)]
+        return Transaction(
+            uniqueid=invtran.fitid, datetime=invtran.dttrade,
+            dtsettle=invtran.dtsettle, type='trade', memo=invtran.memo,
+            currency=invtran.currency, cash=invtran.total, account=account,
+            security=security, units=invtran.units)
+
+
+    trades = [make_trade(tx) for tx in stmt.transactions
+              if isinstance(tx, (BUYSTOCK, SELLSTOCK))]  # dispatch by model class
+
+
+    # Persist
+    engine = create_engine('')
+    Session = sessionmaker(bind=engine)
+    session = Session()
+    session.add(account)
+    session.add_all(securities.values())
+    session.add_all(trades)
+    session.commit()
+
 Contributing
 ============
 
@@ -181,16 +306,20 @@ clone the repository:
 
     git clone https://github.com/csingley/ofxtools.git
 
-Feel free to `create pull requests`_ on `ofxtools repository on GitHub`_.
+Set up a `virtual environment`_, and install the package in development mode
+so you're working on live code:
 
-Again, the minimal dependencies make it simple to install with the Python user
-installation scheme:
+::
+
+    python setup.py develop
+
+Install all development requirements:
 
 ::
 
-    python setup.py develop --user
+    pip install -r requirements-development.txt
 
-However, you'll want to run the tests, either with ``make``:
+Run the tests, either with ``make``:
 
 ::
 
@@ -200,18 +329,10 @@ or directly with ``nosetests``:
 
 ::
 
-    nosetests -dsv --with-yanc --with-coverage --cover-package ofxtools
-
-
-if you don't already have `nose`, `coverage`, etc. installed, and you don't
-want to clutter your system libraries just for this work, you can create a
-`virtual environment`_ and install all development requirements:
+    nosetests -dsv  --with-coverage --cover-package ofxtools
 
-::
+Feel free to `create pull requests`_ on `ofxtools repository on GitHub`_.
 
-    virtualenv .venv
-    source .venv/bin/activate
-    pip install -r requirements-development.txt
 
 Resources
 =========
@@ -226,10 +347,11 @@ Resources
 .. _Requests: http://docs.python-requests.org/en/master/
 .. _PyPI: https://pypi.python.org/pypi/ofxtools
 .. _current master: https://github.com/csingley/ofxtools/archive/master.zip
+.. _virtual environment: https://packaging.python.org/tutorials/installing-packages/#creating-virtual-environments
 .. _OFX spec: http://www.ofx.net/downloads.html
 .. _Git: https://git-scm.com/
 .. _create pull requests: https://help.github.com/articles/using-pull-requests/
 .. _ofxtools repository on GitHub: https://github.com/csingley/ofxtools
-.. _virtual environment: http://docs.python-guide.org/en/latest/dev/virtualenvs/
 .. _Quicken data mapping guide: https://fi.intuit.com/downloads/QW_DataMappingGuide.pdf
 .. _OFX Home: http://www.ofxhome.com/
+