casparser_isin 2023.9.10__tar.gz

casparser_isin-2023.9.10/CHANGELOG.md

@@ -0,0 +1,43 @@
+# Changelog
+
+## 2023.9.10
+- Fallback to old lookup when direct isin search fails
+- update database
+
+## 2023.9.3
+- Lookup scheme via isin
+- update database
+
+## 2023.8.18
+- fix issues with hdfc mutual fund lookups
+- update database
+
+## 2023.1.16
+- DB updates
+
+## 2021.7.21 - 2021-07-21
+- better support for Franklin Templeton funds
+- support new CAS pdf files after migration of funds from FTAMIL RTA to CAMS
+
+## 2021.7.1 - 2021-07-01
+- add scheme type (`EQUITY`/`DEBT`) to `SchemeData`
+- add nav table for looking up scheme nav for 31-Jan-2018
+
+## 2021.6.1 - 2021-06-01
+- support for using custom isin database via `CASPARSER_ISIN_DB` environment variable.
+- updated isin.db
+- packaging fixes
+
+## 2021.5.1 - 2021-03-02
+- DB updates
+  - Essel mutual funds have been renamed to NAVI
+  - Dividend options of funds renamed as IDCW
+
+## 2021.4.1 - 2021-04-01
+- updated isin.db
+- updated dependent package versions
+
+## 2021.3.1 - 2021-03-02
+- Switch to calendar versioning
+- Fix bugs with version comparison in cli update tool
+- DB files are hosted in CDN for more frequent updates via CLI. [pypi releases will be limited to major changes in codebase]

casparser_isin-2023.9.10/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2021 Sandeep Somasekharan
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

casparser_isin-2023.9.10/PKG-INFO

@@ -0,0 +1,79 @@
+Metadata-Version: 2.1
+Name: casparser_isin
+Version: 2023.9.10
+Summary: ISIN database for casparser
+Home-page: https://github.com/codereverser/casparser-isin
+License: MIT
+Author: Sandeep Somasekharan
+Author-email: codereverser@gmail.com
+Requires-Python: >=3.8,<4.0
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Requires-Dist: packaging (>=20.9)
+Requires-Dist: rapidfuzz (>=3.2.0,<4.0.0)
+Description-Content-Type: text/markdown
+
+# CASParser-ISIN
+
+[![code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+[![GitHub](https://img.shields.io/github/license/codereverser/casparser)](https://github.com/codereverser/casparser/blob/main/LICENSE)
+![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/codereverser/casparser-isin/run-pytest.yml?branch=main)
+[![codecov](https://codecov.io/gh/codereverser/casparser-isin/branch/main/graph/badge.svg?token=MQ8ZEVTG1B)](https://codecov.io/gh/codereverser/casparser-isin)
+![PyPI - Python Version](https://img.shields.io/pypi/pyversions/casparser-isin)
+
+ISIN Database for [casparser](https://github.com/codereverser/casparser).
+
+## Installation
+```bash
+pip install -U casparser-isin
+```
+
+## Usage
+
+
+```python
+from casparser_isin import MFISINDb
+with MFISINDb() as db:
+    scheme_data = db.isin_lookup("Axis Long Term Equity Fund - Growth",  # scheme name
+                                 "KFINTECH", # RTA
+                                 "128TSDGG", # Scheme RTA code
+                                 )
+print(scheme_data)
+```
+```
+SchemeData(name="axis long term equity fund - direct growth",
+           isin="INF846K01EW2",
+           amfi_code="120503",
+           score=100.0)
+```
+
+The database also contains NAV values on 31-Jan-2018 for all funds, which can be used for
+taxable LTCG computation for units purchased before the same date.
+
+```
+from casparser_isin import MFISINDb
+with MFISINDb() as db:
+    nav = db.nav_lookup("INF846K01EW2")
+print(nav)
+```
+```
+Decimal('44.8938')
+```
+
+
+## Notes
+
+- casparser-isin is shipped with a local database which may get obsolete over time. The local
+database can be updated via the cli tool
+
+```shell
+casparser-isin --update
+```
+
+- casparser-isin will try to use the file provided by `CASPARSER_ISIN_DB` environment variable; if present, and the file exists
+

casparser_isin-2023.9.10/README.md

@@ -0,0 +1,58 @@
+# CASParser-ISIN
+
+[![code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+[![GitHub](https://img.shields.io/github/license/codereverser/casparser)](https://github.com/codereverser/casparser/blob/main/LICENSE)
+![GitHub Workflow Status](https://img.shields.io/github/actions/workflow/status/codereverser/casparser-isin/run-pytest.yml?branch=main)
+[![codecov](https://codecov.io/gh/codereverser/casparser-isin/branch/main/graph/badge.svg?token=MQ8ZEVTG1B)](https://codecov.io/gh/codereverser/casparser-isin)
+![PyPI - Python Version](https://img.shields.io/pypi/pyversions/casparser-isin)
+
+ISIN Database for [casparser](https://github.com/codereverser/casparser).
+
+## Installation
+```bash
+pip install -U casparser-isin
+```
+
+## Usage
+
+
+```python
+from casparser_isin import MFISINDb
+with MFISINDb() as db:
+    scheme_data = db.isin_lookup("Axis Long Term Equity Fund - Growth",  # scheme name
+                                 "KFINTECH", # RTA
+                                 "128TSDGG", # Scheme RTA code
+                                 )
+print(scheme_data)
+```
+```
+SchemeData(name="axis long term equity fund - direct growth",
+           isin="INF846K01EW2",
+           amfi_code="120503",
+           score=100.0)
+```
+
+The database also contains NAV values on 31-Jan-2018 for all funds, which can be used for
+taxable LTCG computation for units purchased before the same date.
+
+```
+from casparser_isin import MFISINDb
+with MFISINDb() as db:
+    nav = db.nav_lookup("INF846K01EW2")
+print(nav)
+```
+```
+Decimal('44.8938')
+```
+
+
+## Notes
+
+- casparser-isin is shipped with a local database which may get obsolete over time. The local
+database can be updated via the cli tool
+
+```shell
+casparser-isin --update
+```
+
+- casparser-isin will try to use the file provided by `CASPARSER_ISIN_DB` environment variable; if present, and the file exists

casparser_isin-2023.9.10/casparser_isin/__init__.py

@@ -0,0 +1,8 @@
+from .mf_isin import MFISINDb
+
+__all__ = [
+    "MFISINDb",
+    "__version__",
+]
+
+__version__ = "2023.9.10"

casparser_isin-2023.9.10/casparser_isin/cli.py

@@ -0,0 +1,131 @@
+import argparse
+import logging
+from packaging import version
+import sqlite3
+import sys
+from urllib.error import HTTPError
+from urllib import request
+
+from . import __version__
+from .utils import get_isin_db_path
+
+META_URL = "https://casparser.atomcoder.com/isin.db.meta"
+DB_URL = "https://casparser.atomcoder.com/isin.db"
+
+
+def get_metadata():
+    conn = sqlite3.connect(get_isin_db_path())
+    cursor = conn.cursor()
+    try:
+        with conn:
+            cursor.execute("SELECT key, value from meta")
+            metadata = {}
+            for key, value in cursor.fetchall():
+                if key in ("dbformat", "version"):
+                    value = version.parse(value)
+                metadata[key] = value
+            metadata["cli-version"] = version.parse(__version__)
+            return metadata
+    finally:
+        cursor.close()
+        conn.close()
+
+
+def print_version():
+    metadata = get_metadata()
+    print(f"cli-version : {metadata['cli-version']}")
+    print(f"db-version  : {metadata['version']}")
+    print(f"db-format   : {metadata['dbformat']}")
+
+
+def build_request(url):
+    hdr = {
+        "User Agent": f"casparser-isin {__version__}",
+        "X-origin-casparser": "true",
+    }
+    return request.Request(url, headers=hdr)
+
+
+def get_isin_db_details():
+    local_meta = get_metadata()
+    remote_meta = None
+    logging.info("Fetching remote isin db metadata")
+    try:
+        with request.urlopen(build_request(META_URL)) as response:
+            data = response.read().decode()
+    except HTTPError as e:
+        logging.error("Received error from remote server :: %s", e.reason)
+    else:
+        remote_meta = {}
+        for line in data.splitlines():
+            split = line.split("=")
+            if len(split) == 2:
+                key, value = [x.strip() for x in split]
+                if key in ("dbformat", "version"):
+                    value = version.parse(value)
+                remote_meta[key] = value
+        logging.info("Local db version  : %s", local_meta.get("version"))
+        logging.info("Remote db version : %s", remote_meta.get("version"))
+    return remote_meta, local_meta
+
+
+def check_isin_db():
+    """Compare remote and local db versions
+    Return code:
+    0 - no new database available
+    1 - new database available
+    """
+    remote_meta, local_meta = get_isin_db_details()
+    if (
+        remote_meta is not None
+        and remote_meta["version"] > local_meta["version"]
+        and remote_meta["dbformat"] == local_meta["dbformat"]
+    ):
+        logging.info("To update the database, re-run the command with --update flag.")
+        sys.exit(1)
+    else:
+        logging.info("Local database is up to date.")
+        sys.exit(0)
+
+
+def update_isin_db():
+    remote_meta, local_meta = get_isin_db_details()
+    if remote_meta is None:
+        return
+    elif (
+        remote_meta["version"] > local_meta["version"]
+        and remote_meta["dbformat"] == local_meta["dbformat"]
+    ):
+        logging.info("Fetching database version :: %s", remote_meta["version"])
+        try:
+            with request.urlopen(build_request(DB_URL)) as response:
+                data = response.read()
+        except HTTPError as e:
+            logging.error("Error fetching isin database :: %s", e.reason)
+            return
+        with open(get_isin_db_path(), "wb") as f:
+            f.write(data)
+        logging.info("Updated casparser-isin database.")
+    else:
+        logging.info("casparser-isin database is already upto date")
+
+
+def main():
+    parser = argparse.ArgumentParser("casparser-isin", description="casparser-isin cli")
+    parser.add_argument("-v", "--version", help="Print version information", action="store_true")
+    parser.add_argument("--update", help="Update isin database", action="store_true")
+    parser.add_argument("--check", help="Check remote isin database version", action="store_true")
+    args = parser.parse_args()
+    logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s")
+    if args.version:
+        print_version()
+    elif args.update:
+        update_isin_db()
+    elif args.check:
+        check_isin_db()
+    else:
+        parser.print_help()
+
+
+if __name__ == "__main__":
+    main()

casparser_isin-2023.9.10/casparser_isin/mf_isin.py

@@ -0,0 +1,193 @@
+from collections import namedtuple
+from decimal import Decimal
+import re
+import sqlite3
+from typing import Optional
+
+from rapidfuzz import process, utils
+
+from .utils import get_isin_db_path
+
+RTA_MAP = {
+    "CAMS": "CAMS",
+    "FTAMIL": "FRANKLIN",
+    "FRANKLIN": "FRANKLIN",
+    "KFINTECH": "KARVY",
+    "KARVY": "KARVY",
+}
+
+SchemeData = namedtuple("SchemeData", "name isin amfi_code type score")
+
+
+def dict_factory(cursor, row):
+    d = {}
+    for idx, col in enumerate(cursor.description):
+        d[col[0]] = row[idx]
+    return d
+
+
+class MFISINDb:
+    """ISIN database for (Indian) Mutual Funds."""
+
+    connection = None
+    cursor = None
+
+    def __enter__(self):
+        self.initialize()
+        return self
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        self.close()
+
+    def initialize(self):
+        """Initialize database."""
+        self.connection = sqlite3.connect(get_isin_db_path())
+        self.connection.row_factory = dict_factory
+        self.cursor = self.connection.cursor()
+
+    def close(self):
+        """Close database connection."""
+        if self.cursor is not None:
+            self.cursor.close()
+            self.cursor = None
+        if self.connection is not None:
+            self.connection.close()
+            self.connection = None
+
+    def run_query(self, sql, arguments, fetchone=False):
+        self_initialized = False
+        if self.connection is None:
+            self.initialize()
+            self_initialized = True
+        try:
+            self.cursor.execute(sql, arguments)
+            if fetchone:
+                return self.cursor.fetchone()
+            return self.cursor.fetchall()
+        finally:
+            if self_initialized:
+                self.close()
+
+    def direct_isin_lookup(self, isin: str):
+        """
+        Lookup scheme data via ISIN code
+        :param isin: Fund ISIN
+        :return:
+        """
+        sql = """SELECT name, isin, amfi_code, type from scheme WHERE isin = :isin"""
+        return self.run_query(sql, {"isin": isin})
+
+    def scheme_lookup(self, rta: str, scheme_name: str, rta_code: str):
+        """
+        Lookup scheme details from the database
+        :param rta: RTA (CAMS, KARVY, FTAMIL)
+        :param scheme_name: scheme name
+        :param rta_code: RTA code for the scheme
+        :return:
+        """
+        if rta_code is not None:
+            rta_code = re.sub(r"\s+", "", rta_code)
+
+        sql = """SELECT name, isin, amfi_code, type from scheme"""
+        where = ["rta = :rta"]
+
+        if re.search(r"fti(\d+)", rta_code, re.I) and rta.upper() in ("CAMS", "FRANKLIN", "FTAMIL"):
+            # Try searching db for Franklin schemes
+            where_ = ["rta = :rta", "rta_code = :rta_code"]
+            args = {"rta": "FRANKLIN", "rta_code": rta_code}
+            sql_statement = "{} WHERE {}".format(sql, " AND ".join(where_))
+            results = self.run_query(sql_statement, args)
+            if len(results) != 0:
+                return results
+
+        args = {"rta": RTA_MAP.get(str(rta).upper(), ""), "rta_code": rta_code}
+
+        if "hdfc" in scheme_name.lower():
+            if re.search("direct", scheme_name, re.I):
+                where.append("name LIKE '%direct%'")
+            else:
+                where.append("name NOT LIKE '%direct%'")
+
+            if re.search("dividend|idcw", scheme_name, re.I):
+                if re.search("re-*invest", scheme_name, re.I):
+                    where.append("name LIKE '%reinvest%'")
+                else:
+                    where.append("name LIKE '%payout%'")
+            where.append("rta_code like :rta_code_d")
+            args.update(rta_code_d=f"{rta_code}%")
+        else:
+            where.append("rta_code = :rta_code")
+
+        sql_statement = "{} WHERE {} GROUP BY isin".format(sql, " AND ".join(where))
+        results = self.run_query(sql_statement, args)
+        if len(results) == 0 and "rta_code" in args:
+            args["rta_code"] = args["rta_code"][:-1]
+            results = self.run_query(sql_statement, args)
+        return results
+
+    def isin_lookup(
+        self,
+        scheme_name: str,
+        rta: str,
+        rta_code: str,
+        isin: Optional[str] = None,
+        min_score: int = 60,
+    ) -> SchemeData:
+        """
+        Return the closest matching scheme from MF isin database.
+
+        :param scheme_name: Scheme Name
+        :param rta: RTA (CAMS, KARVY, KFINTECH)
+        :param rta_code: Scheme RTA code
+        :param isin: Fund ISIN
+        :param min_score: Minimum score (out of 100) required from the fuzzy match algorithm
+
+        :return: isin and amfi_code code for matching scheme.
+        :rtype: SchemeData
+        :raises: ValueError if no scheme is found in the database.
+        """
+
+        if not (
+            isinstance(scheme_name, str) and isinstance(rta, str) and isinstance(rta_code, str)
+        ):
+            raise TypeError("Invalid input")
+        if rta.upper() not in RTA_MAP:
+            raise ValueError(f"Invalid RTA : {rta}")
+        results = []
+        if isin is not None:
+            results = self.direct_isin_lookup(isin)
+        if len(results) == 0:
+            results = self.scheme_lookup(rta, scheme_name, rta_code)
+        if len(results) == 1:
+            result = results[0]
+            return SchemeData(
+                name=result["name"],
+                isin=result["isin"],
+                amfi_code=result["amfi_code"],
+                type=result["type"],
+                score=100,
+            )
+        elif len(results) > 1:
+            schemes = {
+                x["name"]: (x["name"], x["isin"], x["amfi_code"], x["type"]) for x in results
+            }
+            key, score, _ = process.extractOne(
+                scheme_name, schemes.keys(), processor=utils.default_process
+            )
+            if score >= min_score:
+                name, isin, amfi_code, scheme_type = schemes[key]
+                return SchemeData(
+                    name=name, isin=isin, amfi_code=amfi_code, type=scheme_type, score=score
+                )
+        raise ValueError("No schemes found")
+
+    def nav_lookup(self, isin: str) -> Optional[Decimal]:
+        """
+        Return the NAV of the fund on 31st Jan 2018. used for LTCG computations
+        :param isin: Fund ISIN
+        :return: nav value as a Decimal if available, else return None
+        """
+        sql = """SELECT nav FROM nav20180131 where isin = :isin"""
+        result = self.run_query(sql, {"isin": isin}, fetchone=True)
+        if result is not None:
+            return Decimal(result["nav"])

casparser_isin-2023.9.10/casparser_isin/utils.py

@@ -0,0 +1,15 @@
+import os
+import pathlib
+
+BASE_DIR = pathlib.Path(__file__).resolve().parent
+INTERNAL_ISIN_DB_PATH = BASE_DIR / "isin.db"
+
+
+def get_isin_db_path():
+    env_isin_path = os.getenv("CASPARSER_ISIN_DB")
+    try:
+        if os.path.exists(env_isin_path) and os.path.isfile(env_isin_path):
+            return pathlib.Path(env_isin_path)
+    except TypeError:
+        pass
+    return INTERNAL_ISIN_DB_PATH

casparser_isin-2023.9.10/pyproject.toml

@@ -0,0 +1,58 @@
+[tool.poetry]
+name = "casparser_isin"
+version = "0"
+description = "ISIN database for casparser"
+authors = ["Sandeep Somasekharan <codereverser@gmail.com>"]
+homepage = "https://github.com/codereverser/casparser-isin"
+license = "MIT License"
+readme = "README.md"
+classifiers = [
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Operating System :: OS Independent"
+]
+include = [ "CHANGELOG.md" ]
+
+[tool.poetry.dependencies]
+python = "^3.8"
+packaging = ">=20.9"
+rapidfuzz = "^3.2.0"
+
+[tool.poetry.dev-dependencies]
+coverage = {version = "^7.3.0", extras=["toml"]}
+pytest = "^7.4.0"
+pytest-cov = "^4.1.0"
+apsw = "^3.43.0"
+b2sdk = "^1.24.0"
+lxml = "^4.9.0"
+python-dotenv = "^1.0.0"
+requests = "^2.31.0"
+requests-cache = "^1.1.0"
+pre-commit = "^3.4.0"
+
+[tool.poetry.scripts]
+casparser-isin = "casparser_isin.cli:main"
+
+[build-system]
+requires = ["poetry-core >= 1.0.0"]
+build-backend = "poetry.core.masonry.api"
+
+[tool.black]
+line-length = 100
+target-version = ["py38"]
+
+[tool.pytest.ini_options]
+minversion = "6.0"
+addopts = "--cov=casparser_isin --cov-config=tox.ini --cov-report=xml --cov-report=html --exitfirst"
+testpaths = [
+    "tests",
+]
+
+[tool.ruff]
+line-length = 100
+target-version = "py38"
+
+[tool.poetry-version-plugin]
+source = "init"