db-toolkit 0.0.2__tar.gz → 0.0.4__tar.gz

db_toolkit-0.0.4/PKG-INFO

@@ -0,0 +1,112 @@
+Metadata-Version: 2.4
+Name: db-toolkit
+Version: 0.0.4
+Summary: utility tools for db access and data extraction
+Home-page: https://github.com/Analytics/db-toolkit
+Author: frangs
+Author-email: giordanofrancisco@duck.com
+License: Apache Software License 2.0
+Keywords: nbdev jupyter notebook python
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Natural Language :: English
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: License :: OSI Approved :: Apache Software License
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: fastcore
+Requires-Dist: pandas
+Requires-Dist: sqlalchemy
+Provides-Extra: dev
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: keywords
+Dynamic: license
+Dynamic: license-file
+Dynamic: provides-extra
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# db-toolkit
+
+
+<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
+
+``` python
+from db_toolkit.db_extract import *
+from db_toolkit.db_connect import *
+from pathlib import Path
+```
+
+## Developer Guide
+
+### Install db_toolkit in Development mode
+
+``` sh
+# make sure db_toolkit package is installed in development mode
+$ pip install -e .
+
+# make changes under nbs/ directory
+# ...
+
+# compile to have changes apply to db_toolkit
+$ nbdev_prepare
+```
+
+## Usage
+
+### Installation
+
+To use this package in another project, install as below
+
+Install latest from the AzureDevOps \[repository\]\[repo\]:
+
+``` sh
+$ pip install db-toolkit
+```
+
+## How to use
+
+### 1. Set up your config file
+
+Create an `.ini` file with your database connection details:
+
+``` ini
+[CONN_ORC]
+user:TEST_USER
+pass:<your_password>
+dsn:<...>
+port:<...>
+dbname:<...>
+
+[CONN_MS]
+server:<...>
+dbname:<...>
+```
+
+### 2. Create your SQL file
+
+Save your query in `data/sql/` with the naming convention
+`<CONNECTION>__<tablename>.sql`:
+
+    data/sql/ODS__address_ctax.sql
+
+### 3. Pull the data
+
+``` python
+from db_toolkit.db_extract import DBExtract
+
+dbe = DBExtract(cfg_fpath='path/to/your/config.ini')
+dbe.pull_table('BIODS__address_ctax')
+```
+
+The resulting CSV will be saved to `data/landing/`.

db_toolkit-0.0.4/README.md

@@ -0,0 +1,74 @@
+# db-toolkit
+
+
+<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
+
+``` python
+from db_toolkit.db_extract import *
+from db_toolkit.db_connect import *
+from pathlib import Path
+```
+
+## Developer Guide
+
+### Install db_toolkit in Development mode
+
+``` sh
+# make sure db_toolkit package is installed in development mode
+$ pip install -e .
+
+# make changes under nbs/ directory
+# ...
+
+# compile to have changes apply to db_toolkit
+$ nbdev_prepare
+```
+
+## Usage
+
+### Installation
+
+To use this package in another project, install as below
+
+Install latest from the AzureDevOps \[repository\]\[repo\]:
+
+``` sh
+$ pip install db-toolkit
+```
+
+## How to use
+
+### 1. Set up your config file
+
+Create an `.ini` file with your database connection details:
+
+``` ini
+[CONN_ORC]
+user:TEST_USER
+pass:<your_password>
+dsn:<...>
+port:<...>
+dbname:<...>
+
+[CONN_MS]
+server:<...>
+dbname:<...>
+```
+
+### 2. Create your SQL file
+
+Save your query in `data/sql/` with the naming convention
+`<CONNECTION>__<tablename>.sql`:
+
+    data/sql/ODS__address_ctax.sql
+
+### 3. Pull the data
+
+``` python
+from db_toolkit.db_extract import DBExtract
+
+dbe = DBExtract(cfg_fpath='path/to/your/config.ini')
+dbe.pull_table('BIODS__address_ctax')
+```
+
+The resulting CSV will be saved to `data/landing/`.

db_toolkit-0.0.4/db_toolkit/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.0.4"

{db_toolkit-0.0.2 → db_toolkit-0.0.4}/db_toolkit/_modidx.py

@@ -34,12 +34,14 @@ d = { 'settings': { 'branch': 'main',
                                                                                        'db_toolkit/db_extract.py'),
                                        'db_toolkit.db_extract.DBExtract._get_sql': ( 'db_extract.html#dbextract._get_sql',
                                                                                      'db_toolkit/db_extract.py'),
-                                       'db_toolkit.db_extract.DBExtract._process_all_sql': ( 'db_extract.html#dbextract._process_all_sql',
-                                                                                             'db_toolkit/db_extract.py'),
-                                       'db_toolkit.db_extract.DBExtract._save_data_landing': ( 'db_extract.html#dbextract._save_data_landing',
+                                       'db_toolkit.db_extract.DBExtract.pull_table': ( 'db_extract.html#dbextract.pull_table',
+                                                                                       'db_toolkit/db_extract.py'),
+                                       'db_toolkit.db_extract.DBExtract.pull_tables': ( 'db_extract.html#dbextract.pull_tables',
+                                                                                        'db_toolkit/db_extract.py'),
+                                       'db_toolkit.db_extract.DBExtract.pull_with_manifest': ( 'db_extract.html#dbextract.pull_with_manifest',
                                                                                                'db_toolkit/db_extract.py'),
-                                       'db_toolkit.db_extract.DBExtract.process_save_landing': ( 'db_extract.html#dbextract.process_save_landing',
-                                                                                                 'db_toolkit/db_extract.py'),
+                                       'db_toolkit.db_extract.DBExtract.sample': ( 'db_extract.html#dbextract.sample',
+                                                                                   'db_toolkit/db_extract.py'),
                                        'db_toolkit.db_extract.PullManifest': ('db_extract.html#pullmanifest', 'db_toolkit/db_extract.py'),
                                        'db_toolkit.db_extract.PullManifest.__init__': ( 'db_extract.html#pullmanifest.__init__',
                                                                                         'db_toolkit/db_extract.py'),

{db_toolkit-0.0.2 → db_toolkit-0.0.4}/db_toolkit/db_extract.py

@@ -6,27 +6,65 @@ from pathlib import Path
 import json, time
 from datetime import datetime, timedelta
 from fastcore.basics import patch
+from concurrent.futures import ThreadPoolExecutor
 import pandas as pd
 from .db_connect import *
 
 
 # %% auto #0
-__all__ = ['DBExtract', 'PullManifest']
+__all__ = ['PullManifest', 'DBExtract']
+
+# %% ../nbs/02_db_extract.ipynb #bf53f340
+class PullManifest:
+    def __init__(self, manifest_path, pull_interval_days=14):
+        self.path = Path(manifest_path)
+        self.pull_interval_days = pull_interval_days
+        self._load()
+    
+    def _load(self):
+        """Load manifest from disk, or create empty one"""
+        if self.path.exists(): self.data = json.loads(self.path.read_text())
+        else: self.data = {"pulls": {}}
+    
+    def _save(self):
+        """Persist manifest to disk"""
+        self.path.write_text(json.dumps(self.data, indent=2))
+    
+    def should_pull(self, table_name):
+        """Check if table needs refresh based on TTL"""
+        if table_name not in self.data["pulls"]: return True
+        last = datetime.fromisoformat(self.data["pulls"][table_name]["last_pull"])
+        return datetime.now() - last > timedelta(days=self.pull_interval_days)
+    
+    def record_pull(self, table_name, row_count, status="success"):
+        """Record a completed pull"""
+        self.data["pulls"][table_name] = {
+            "last_pull": datetime.now().isoformat(),
+            "row_count": row_count,
+            "status": status
+
+        }
+        self._save()
+
 
 # %% ../nbs/02_db_extract.ipynb #4ebf670b-04ae-4d44-9eac-aa4caf54be99
 class DBExtract:
-    def __init__(self, cfg_fpath):
+    def __init__(self, cfg_fpath, manifest=None):
         """
         Constructor for the DBExtract class.
         Reads parameters from the specified configuration file
-        Pairs SQL scrit to connection engine
+        Pairs SQL script to connection engine
         Pulls and saves data to landing folder
+        
+        SQL files must follow naming convention: <DB_NAME>__<TABLE_NAME>.sql
+        where DB_NAME matches a key in the config connections.
         """
         DBConfig.file_exists(cfg_fpath)
         self.data_dir = Path("../data")
         self.sql_dir = self.data_dir / 'sql'    
         self.cfg = DBConfig(cfg_fpath)
         self.cfg.set_conn()
+        self.manifest = manifest
 
     def _get_pairs(self):
         """Using SQL files name convention <CONN>__<DF NAME>, create list collection type (df name, connection engine, sql script)"""
@@ -54,53 +92,43 @@ class DBExtract:
             retries += 1
             wait_time = 2**retries
             time.sleep(wait_time)
-            return self._get_dataframe(stream, retries=retries)
+            return self._get_dataframe(conn, sql, chunksize, retries=retries)
         return pd.concat([df.dropna(how='all', axis=1) for df in dataframes], ignore_index=True)
-
-    def _process_all_sql(self, chunksize=10_000):
-        self._get_pairs()
-        self.dataframes = {}
-        for name, conn, sql in self.pairs: self.dataframes[name] = self._get_dataframe(conn, sql, chunksize=chunksize)
-
-    def _save_data_landing(self):
-         for key, df in self.dataframes.items(): df.to_csv(self.data_dir/f"landing/{key}.csv", index=False)
-    
-    def process_save_landing(self, chunksize=10_000):
-        """pull and saves all db dataframes as csv to data/landing"""
-        self._process_all_sql(chunksize=chunksize)
-        self._save_data_landing()
        
  
 
-# %% ../nbs/02_db_extract.ipynb #b6b66498-f021-4fe5-9706-43f27bad1fdb
-class PullManifest:
-    def __init__(self, manifest_path, pull_interval_days=14):
-        self.path = Path(manifest_path)
-        self.pull_interval_days = pull_interval_days
-        self._load()
-    
-    def _load(self):
-        """Load manifest from disk, or create empty one"""
-        if self.path.exists(): self.data = json.loads(self.path.read_text())
-        else: self.data = {"pulls": {}}
-    
-    def _save(self):
-        """Persist manifest to disk"""
-        self.path.write_text(json.dumps(self.data, indent=2))
-    
-    def should_pull(self, table_name):
-        """Check if table needs refresh based on TTL"""
-        if table_name not in self.data["pulls"]: return True
-        last = datetime.fromisoformat(self.data["pulls"][table_name]["last_pull"])
-        return datetime.now() - last > timedelta(days=self.pull_interval_days)
-    
-    def record_pull(self, table_name, row_count, status="success"):
-        """Record a completed pull"""
-        self.data["pulls"][table_name] = {
-            "last_pull": datetime.now().isoformat(),
-            "row_count": row_count,
-            "status": status
+# %% ../nbs/02_db_extract.ipynb #bc75d61c-94cb-4322-a1bf-d3be6d73110f
+@patch
+def sample(self: DBExtract, name, n=100):
+    """Pull first n rows from a table for testing"""
+    self._get_pairs()
+    _, conn, sql = next((p for p in self.pairs if p[0] == name))
+    return next(self._get_data_stream(conn, sql, chunksize=n))
 
-        }
-        self._save()
+# %% ../nbs/02_db_extract.ipynb #e2934306
+@patch
+def pull_table(self:DBExtract, name, chunksize=10_000):
+    """Pull full table from a given name save it to landing"""
+    self._get_pairs()
+    _, conn, sql = next((p for p in self.pairs if p[0] == name))
+    df = self._get_dataframe(conn, sql, chunksize=chunksize)
+    df.to_csv(self.data_dir/f"landing/{name}.csv", index=False)
+    if self.manifest: self.manifest.record_pull(name, len(df))
+    return df
+
+# %% ../nbs/02_db_extract.ipynb #44b8f7be
+@patch
+def pull_tables(self:DBExtract, names, chunksize=10_000, max_workers=4):
+    """Pull multiple tables in parallel"""
+    with ThreadPoolExecutor(max_workers=max_workers) as executor:
+        results = list(executor.map(lambda name: self.pull_table(name, chunksize), names))
+    return results
 
+# %% ../nbs/02_db_extract.ipynb #112fff76
+@patch
+def pull_with_manifest(self:DBExtract, chunksize=10_000, max_workers=4):
+    """Pull full table from a given collection of table names"""
+    if self.manifest is None: return  # No manifest, skip pulling
+    self._get_pairs()
+    tables_to_pull = [name for name, _, _ in self.pairs if self.manifest.should_pull(name)]
+    if tables_to_pull: self.pull_tables(tables_to_pull, chunksize, max_workers)

db_toolkit-0.0.4/db_toolkit.egg-info/PKG-INFO

@@ -0,0 +1,112 @@
+Metadata-Version: 2.4
+Name: db-toolkit
+Version: 0.0.4
+Summary: utility tools for db access and data extraction
+Home-page: https://github.com/Analytics/db-toolkit
+Author: frangs
+Author-email: giordanofrancisco@duck.com
+License: Apache Software License 2.0
+Keywords: nbdev jupyter notebook python
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Natural Language :: English
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: License :: OSI Approved :: Apache Software License
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: fastcore
+Requires-Dist: pandas
+Requires-Dist: sqlalchemy
+Provides-Extra: dev
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: keywords
+Dynamic: license
+Dynamic: license-file
+Dynamic: provides-extra
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# db-toolkit
+
+
+<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
+
+``` python
+from db_toolkit.db_extract import *
+from db_toolkit.db_connect import *
+from pathlib import Path
+```
+
+## Developer Guide
+
+### Install db_toolkit in Development mode
+
+``` sh
+# make sure db_toolkit package is installed in development mode
+$ pip install -e .
+
+# make changes under nbs/ directory
+# ...
+
+# compile to have changes apply to db_toolkit
+$ nbdev_prepare
+```
+
+## Usage
+
+### Installation
+
+To use this package in another project, install as below
+
+Install latest from the AzureDevOps \[repository\]\[repo\]:
+
+``` sh
+$ pip install db-toolkit
+```
+
+## How to use
+
+### 1. Set up your config file
+
+Create an `.ini` file with your database connection details:
+
+``` ini
+[CONN_ORC]
+user:TEST_USER
+pass:<your_password>
+dsn:<...>
+port:<...>
+dbname:<...>
+
+[CONN_MS]
+server:<...>
+dbname:<...>
+```
+
+### 2. Create your SQL file
+
+Save your query in `data/sql/` with the naming convention
+`<CONNECTION>__<tablename>.sql`:
+
+    data/sql/ODS__address_ctax.sql
+
+### 3. Pull the data
+
+``` python
+from db_toolkit.db_extract import DBExtract
+
+dbe = DBExtract(cfg_fpath='path/to/your/config.ini')
+dbe.pull_table('BIODS__address_ctax')
+```
+
+The resulting CSV will be saved to `data/landing/`.

{db_toolkit-0.0.2 → db_toolkit-0.0.4}/settings.ini

@@ -1,16 +1,10 @@
 [DEFAULT]
-# All sections below are required unless otherwise specified.
-# See https://github.com/AnswerDotAI/nbdev/blob/main/settings.ini for examples.
-
-### Python library ###
 repo = db-toolkit
-lib_name = %(repo)s
-version = 0.0.2
+lib_name = db-toolkit
+version = 0.0.4
 min_python = 3.9
 license = apache2
 black_formatting = False
-
-### nbdev ###
 doc_path = _docs
 lib_path = db_toolkit
 nbs_path = nbs
@@ -18,29 +12,27 @@ recursive = True
 tst_flags = notest
 put_version_in_init = True
 update_pyproject = True
-
-### Docs ###
 branch = main
 custom_sidebar = False
-doc_host = https://%(user)s.github.io
-doc_baseurl = /%(repo)s
-git_url = https://github.com/%(user)s/%(repo)s
-title = %(lib_name)s
-
-### PyPI ###
+doc_host = https://Analytics.github.io
+doc_baseurl = /db-toolkit
+git_url = https://github.com/Analytics/db-toolkit
+title = db-toolkit
 audience = Developers
 author = frangs
 author_email = giordanofrancisco@duck.com
-copyright = 2025 onwards, %(author)s
+copyright = 2025 onwards, frangs
 description = utility tools for db access and data extraction
 keywords = nbdev jupyter notebook python
 language = English
 status = 3
 user = Analytics
-
-### Optional ###
 requirements = fastcore pandas sqlalchemy
-# dev_requirements = 
-# console_scripts =
-# conda_user = 
-# package_data =
+readme_nb = index.ipynb
+allowed_metadata_keys = 
+allowed_cell_metadata_keys = 
+jupyter_hooks = False
+clean_ids = True
+clear_all = False
+skip_procs = 
+

db_toolkit-0.0.2/PKG-INFO

@@ -1,167 +0,0 @@
-Metadata-Version: 2.4
-Name: db-toolkit
-Version: 0.0.2
-Summary: utility tools for db access and data extraction
-Home-page: https://github.com/Analytics/db-toolkit
-Author: frangs
-Author-email: giordanofrancisco@duck.com
-License: Apache Software License 2.0
-Keywords: nbdev jupyter notebook python
-Classifier: Development Status :: 4 - Beta
-Classifier: Intended Audience :: Developers
-Classifier: Natural Language :: English
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: License :: OSI Approved :: Apache Software License
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: fastcore
-Requires-Dist: pandas
-Requires-Dist: sqlalchemy
-Provides-Extra: dev
-Dynamic: author
-Dynamic: author-email
-Dynamic: classifier
-Dynamic: description
-Dynamic: description-content-type
-Dynamic: home-page
-Dynamic: keywords
-Dynamic: license
-Dynamic: license-file
-Dynamic: provides-extra
-Dynamic: requires-dist
-Dynamic: requires-python
-Dynamic: summary
-
-# db-toolkit
-
-
-<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
-
-``` python
-from db_toolkit.db_extract import *
-from db_toolkit.db_connect import *
-from pathlib import Path
-```
-
-This file will become your README and also the index of your
-documentation.
-
-## Developer Guide
-
-If you are new to using `nbdev` here are some useful pointers to get you
-started.
-
-### Install db_toolkit in Development mode
-
-``` sh
-# make sure db_toolkit package is installed in development mode
-$ pip install -e .
-
-# make changes under nbs/ directory
-# ...
-
-# compile to have changes apply to db_toolkit
-$ nbdev_prepare
-```
-
-## Usage
-
-### Installation
-
-To use this package in another project, install as below
-
-Install latest from the AzureDevOps
-[repository](https://lambeth@dev.azure.com/lambeth/Analytics/_git/db-toolkit):
-
-``` sh
-$ pip install git+https://lambeth@dev.azure.com/lambeth/Analytics/_git/db-toolkit
-```
-
-### Dependencies
-
-The YAML file for this package dependencies can be found in the `envs\`
-DIR. If using toolkit on another project, it is advisable to define
-`db-toolkit` as the below format YAML.
-
-``` yaml
-name: er
-channels:
-  - conda-forge
-dependencies:
-  - jupyterlab
-  - pandas
-  - oracledb
-  - pyodbc
-  - pip
-  - pip:
-    - nbdev
-    - git+https://lambeth@dev.azure.com/lambeth/Analytics/_git/db-toolkit
-    - sqlalchemy
-```
-
-### Documentation
-
-TODO potentially when github is available.
-
-## How to use
-
-## Set ini file location path
-
-- TODO init file struct details show example
-
-``` python
-fpath2 = '../../../data/db_connections/cfg_address.ini'
-dbe = DBExtract(cfg_fpath=fpath2)
-```
-
-## SQL files
-
-- Saved at ../data/sql
-- TODO add details on path \<dbname\>\_\_\<tablename\>
-
-------------------------------------------------------------------------
-
-<a
-href="https://github.com/Analytics/db-toolkit/blob/main/db_toolkit/db_connect.py#LNone"
-target="_blank" style="float:right; font-size:smaller">source</a>
-
-### DBConfig
-
->  DBConfig (file_path)
-
-*Constructor for the DBConfig class. Reads parameters from the specified
-configuration file and presents them appropriately to the application.*
-
-------------------------------------------------------------------------
-
-<a
-href="https://github.com/Analytics/db-toolkit/blob/main/db_toolkit/db_extract.py#LNone"
-target="_blank" style="float:right; font-size:smaller">source</a>
-
-### DBExtract
-
->  DBExtract (cfg_fpath)
-
-*Constructor for the DBExtract class. Reads parameters from the
-specified configuration file Pairs SQL scrit to connection engine Pulls
-and saves data to landing folder*
-
-``` python
-data_dir = Path("../data")
-sql_dir = data_dir / 'sql'
-sql_files = list(sql_dir.glob("*.sql"))
-sql_files
-```
-
-    [Path('../data/sql/BIODS__address_ctax.sql')]
-
-## Process files
-
-``` python
-# files wil be saved at ../data/landing
-# dbe.process_save_landing()
-```

db_toolkit-0.0.2/README.md

@@ -1,129 +0,0 @@
-# db-toolkit
-
-
-<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
-
-``` python
-from db_toolkit.db_extract import *
-from db_toolkit.db_connect import *
-from pathlib import Path
-```
-
-This file will become your README and also the index of your
-documentation.
-
-## Developer Guide
-
-If you are new to using `nbdev` here are some useful pointers to get you
-started.
-
-### Install db_toolkit in Development mode
-
-``` sh
-# make sure db_toolkit package is installed in development mode
-$ pip install -e .
-
-# make changes under nbs/ directory
-# ...
-
-# compile to have changes apply to db_toolkit
-$ nbdev_prepare
-```
-
-## Usage
-
-### Installation
-
-To use this package in another project, install as below
-
-Install latest from the AzureDevOps
-[repository](https://lambeth@dev.azure.com/lambeth/Analytics/_git/db-toolkit):
-
-``` sh
-$ pip install git+https://lambeth@dev.azure.com/lambeth/Analytics/_git/db-toolkit
-```
-
-### Dependencies
-
-The YAML file for this package dependencies can be found in the `envs\`
-DIR. If using toolkit on another project, it is advisable to define
-`db-toolkit` as the below format YAML.
-
-``` yaml
-name: er
-channels:
-  - conda-forge
-dependencies:
-  - jupyterlab
-  - pandas
-  - oracledb
-  - pyodbc
-  - pip
-  - pip:
-    - nbdev
-    - git+https://lambeth@dev.azure.com/lambeth/Analytics/_git/db-toolkit
-    - sqlalchemy
-```
-
-### Documentation
-
-TODO potentially when github is available.
-
-## How to use
-
-## Set ini file location path
-
-- TODO init file struct details show example
-
-``` python
-fpath2 = '../../../data/db_connections/cfg_address.ini'
-dbe = DBExtract(cfg_fpath=fpath2)
-```
-
-## SQL files
-
-- Saved at ../data/sql
-- TODO add details on path \<dbname\>\_\_\<tablename\>
-
-------------------------------------------------------------------------
-
-<a
-href="https://github.com/Analytics/db-toolkit/blob/main/db_toolkit/db_connect.py#LNone"
-target="_blank" style="float:right; font-size:smaller">source</a>
-
-### DBConfig
-
->  DBConfig (file_path)
-
-*Constructor for the DBConfig class. Reads parameters from the specified
-configuration file and presents them appropriately to the application.*
-
-------------------------------------------------------------------------
-
-<a
-href="https://github.com/Analytics/db-toolkit/blob/main/db_toolkit/db_extract.py#LNone"
-target="_blank" style="float:right; font-size:smaller">source</a>
-
-### DBExtract
-
->  DBExtract (cfg_fpath)
-
-*Constructor for the DBExtract class. Reads parameters from the
-specified configuration file Pairs SQL scrit to connection engine Pulls
-and saves data to landing folder*
-
-``` python
-data_dir = Path("../data")
-sql_dir = data_dir / 'sql'
-sql_files = list(sql_dir.glob("*.sql"))
-sql_files
-```
-
-    [Path('../data/sql/BIODS__address_ctax.sql')]
-
-## Process files
-
-``` python
-# files wil be saved at ../data/landing
-# dbe.process_save_landing()
-```

db_toolkit-0.0.2/db_toolkit/__init__.py

@@ -1 +0,0 @@
-__version__ = "0.0.2"

db_toolkit-0.0.2/db_toolkit.egg-info/PKG-INFO

@@ -1,167 +0,0 @@
-Metadata-Version: 2.4
-Name: db-toolkit
-Version: 0.0.2
-Summary: utility tools for db access and data extraction
-Home-page: https://github.com/Analytics/db-toolkit
-Author: frangs
-Author-email: giordanofrancisco@duck.com
-License: Apache Software License 2.0
-Keywords: nbdev jupyter notebook python
-Classifier: Development Status :: 4 - Beta
-Classifier: Intended Audience :: Developers
-Classifier: Natural Language :: English
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: License :: OSI Approved :: Apache Software License
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: fastcore
-Requires-Dist: pandas
-Requires-Dist: sqlalchemy
-Provides-Extra: dev
-Dynamic: author
-Dynamic: author-email
-Dynamic: classifier
-Dynamic: description
-Dynamic: description-content-type
-Dynamic: home-page
-Dynamic: keywords
-Dynamic: license
-Dynamic: license-file
-Dynamic: provides-extra
-Dynamic: requires-dist
-Dynamic: requires-python
-Dynamic: summary
-
-# db-toolkit
-
-
-<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
-
-``` python
-from db_toolkit.db_extract import *
-from db_toolkit.db_connect import *
-from pathlib import Path
-```
-
-This file will become your README and also the index of your
-documentation.
-
-## Developer Guide
-
-If you are new to using `nbdev` here are some useful pointers to get you
-started.
-
-### Install db_toolkit in Development mode
-
-``` sh
-# make sure db_toolkit package is installed in development mode
-$ pip install -e .
-
-# make changes under nbs/ directory
-# ...
-
-# compile to have changes apply to db_toolkit
-$ nbdev_prepare
-```
-
-## Usage
-
-### Installation
-
-To use this package in another project, install as below
-
-Install latest from the AzureDevOps
-[repository](https://lambeth@dev.azure.com/lambeth/Analytics/_git/db-toolkit):
-
-``` sh
-$ pip install git+https://lambeth@dev.azure.com/lambeth/Analytics/_git/db-toolkit
-```
-
-### Dependencies
-
-The YAML file for this package dependencies can be found in the `envs\`
-DIR. If using toolkit on another project, it is advisable to define
-`db-toolkit` as the below format YAML.
-
-``` yaml
-name: er
-channels:
-  - conda-forge
-dependencies:
-  - jupyterlab
-  - pandas
-  - oracledb
-  - pyodbc
-  - pip
-  - pip:
-    - nbdev
-    - git+https://lambeth@dev.azure.com/lambeth/Analytics/_git/db-toolkit
-    - sqlalchemy
-```
-
-### Documentation
-
-TODO potentially when github is available.
-
-## How to use
-
-## Set ini file location path
-
-- TODO init file struct details show example
-
-``` python
-fpath2 = '../../../data/db_connections/cfg_address.ini'
-dbe = DBExtract(cfg_fpath=fpath2)
-```
-
-## SQL files
-
-- Saved at ../data/sql
-- TODO add details on path \<dbname\>\_\_\<tablename\>
-
-------------------------------------------------------------------------
-
-<a
-href="https://github.com/Analytics/db-toolkit/blob/main/db_toolkit/db_connect.py#LNone"
-target="_blank" style="float:right; font-size:smaller">source</a>
-
-### DBConfig
-
->  DBConfig (file_path)
-
-*Constructor for the DBConfig class. Reads parameters from the specified
-configuration file and presents them appropriately to the application.*
-
-------------------------------------------------------------------------
-
-<a
-href="https://github.com/Analytics/db-toolkit/blob/main/db_toolkit/db_extract.py#LNone"
-target="_blank" style="float:right; font-size:smaller">source</a>
-
-### DBExtract
-
->  DBExtract (cfg_fpath)
-
-*Constructor for the DBExtract class. Reads parameters from the
-specified configuration file Pairs SQL scrit to connection engine Pulls
-and saves data to landing folder*
-
-``` python
-data_dir = Path("../data")
-sql_dir = data_dir / 'sql'
-sql_files = list(sql_dir.glob("*.sql"))
-sql_files
-```
-
-    [Path('../data/sql/BIODS__address_ctax.sql')]
-
-## Process files
-
-``` python
-# files wil be saved at ../data/landing
-# dbe.process_save_landing()
-```