data-manipulation 0.48__tar.gz → 0.50__tar.gz

data_manipulation-0.50/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,38 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. iOS]
+ - Browser [e.g. chrome, safari]
+ - Version [e.g. 22]
+
+**Smartphone (please complete the following information):**
+ - Device: [e.g. iPhone6]
+ - OS: [e.g. iOS8.1]
+ - Browser [e.g. stock browser, safari]
+ - Version [e.g. 22]
+
+**Additional context**
+Add any other context about the problem here.

data_manipulation-0.50/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

data_manipulation-0.50/.gitignore

@@ -0,0 +1,99 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+env/
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+*.egg-info/
+data_manipulation/_version.py
+.installed.cfg
+*.egg
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+
+# Sphinx documentation
+# docs/_build/
+!docs/build
+
+# PyBuilder
+target/
+
+# DotEnv configuration
+.env
+
+# Database
+*.db
+*.rdb
+
+# Pycharm
+.idea
+
+# VS Code
+.vscode/
+
+# Spyder
+.spyproject/
+
+# Jupyter NB Checkpoints
+.ipynb_checkpoints/
+
+# exclude data from source control by default
+/data/
+*.csv
+*.xlsx
+
+# Mac OS-specific storage files
+.DS_Store
+
+# vim
+*.swp
+*.swo
+
+# Mypy cache
+.mypy_cache/
+
+# custom
+build.sh
+install_packages.sh
+docs/README*
+.opencode

data_manipulation-0.50/CODE_OF_CONDUCT.md

@@ -0,0 +1,76 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, sex characteristics, gender identity and expression,
+level of experience, education, socio-economic status, nationality, personal
+appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+ advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+ address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+ professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at Shawn.coding.acc@gmail.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see
+https://www.contributor-covenant.org/faq

data_manipulation-0.50/CONTRIBUTING.md

@@ -0,0 +1,19 @@
+# Contributing to data manipulation
+
+Whether you are a novice or experienced software developer, all contributions and suggestions are welcome!
+
+## Getting Started
+
+If you are looking to contribute to the *data manipulation* codebase, the best place to start is the [GitHub "issues" tab](https://github.com/shawnngtq/data-manipulation/issues). This is also a great place for filing bug reports and making suggestions for ways in which we can improve the code and documentation.
+
+## Filing Issues
+
+If you notice a bug in the code or documentation, or have suggestions for how we can improve either, feel free to create an issue on the [GitHub "issues" tab](https://github.com/shawnngtq/data-manipulation/issues) using [GitHub's "issue" form](https://github.com/shawnngtq/data-manipulation/issues/new). The form contains some questions that will help us best address your issue.
+
+## Contributing to the Codebase
+
+The code is hosted on [GitHub](https://github.com/shawnngtq/data-manipulation), so you will need to use [Git](https://git-scm.com/) to clone the project and make changes to the codebase. Once you have obtained a copy of the code, you should create a development environment that is separate from your existing Python environment so that you can make and test changes without compromising your own work environment.
+
+Before submitting your changes for review, make sure to check that your changes do not break any tests.
+
+Once your changes are ready to be submitted, make sure to push your changes to GitHub before creating a pull request. We will review your changes, and you will most likely be asked to make additional changes before it is finally ready to merge. However, once it's ready, we will merge it, and you will have successfully contributed to the codebase!

data_manipulation-0.50/Makefile

@@ -0,0 +1,144 @@
+.PHONY: clean data lint requirements sync_data_to_s3 sync_data_from_s3
+
+#################################################################################
+# GLOBALS                                                                       #
+#################################################################################
+
+PROJECT_DIR := $(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))
+BUCKET = [OPTIONAL] your-bucket-for-syncing-data (do not include 's3://')
+PROFILE = default
+PROJECT_NAME = data_manipulation
+PYTHON_INTERPRETER = python3
+
+ifeq (,$(shell which conda))
+HAS_CONDA=False
+else
+HAS_CONDA=True
+endif
+
+#################################################################################
+# COMMANDS                                                                      #
+#################################################################################
+
+## Install Python Dependencies
+requirements: test_environment
+	$(PYTHON_INTERPRETER) -m pip install -U pip setuptools wheel
+	$(PYTHON_INTERPRETER) -m pip install -r requirements.txt
+
+## Make Dataset
+data: requirements
+	$(PYTHON_INTERPRETER) data_manipulation/data/make_dataset.py data/raw data/processed
+
+## Delete all compiled Python files
+clean:
+	find . -type f -name "*.py[co]" -delete
+	find . -type d -name "__pycache__" -delete
+
+## Lint using flake8
+lint:
+	flake8 data_manipulation
+
+## Upload Data to S3
+sync_data_to_s3:
+ifeq (default,$(PROFILE))
+	aws s3 sync data/ s3://$(BUCKET)/data/
+else
+	aws s3 sync data/ s3://$(BUCKET)/data/ --profile $(PROFILE)
+endif
+
+## Download Data from S3
+sync_data_from_s3:
+ifeq (default,$(PROFILE))
+	aws s3 sync s3://$(BUCKET)/data/ data/
+else
+	aws s3 sync s3://$(BUCKET)/data/ data/ --profile $(PROFILE)
+endif
+
+## Set up python interpreter environment
+create_environment:
+ifeq (True,$(HAS_CONDA))
+		@echo ">>> Detected conda, creating conda environment."
+ifeq (3,$(findstring 3,$(PYTHON_INTERPRETER)))
+	conda create --name $(PROJECT_NAME) python=3
+else
+	conda create --name $(PROJECT_NAME) python=2.7
+endif
+		@echo ">>> New conda env created. Activate with:\nsource activate $(PROJECT_NAME)"
+else
+	$(PYTHON_INTERPRETER) -m pip install -q virtualenv virtualenvwrapper
+	@echo ">>> Installing virtualenvwrapper if not already installed.\nMake sure the following lines are in shell startup file\n\
+	export WORKON_HOME=$$HOME/.virtualenvs\nexport PROJECT_HOME=$$HOME/Devel\nsource /usr/local/bin/virtualenvwrapper.sh\n"
+	@bash -c "source `which virtualenvwrapper.sh`;mkvirtualenv $(PROJECT_NAME) --python=$(PYTHON_INTERPRETER)"
+	@echo ">>> New virtualenv created. Activate with:\nworkon $(PROJECT_NAME)"
+endif
+
+## Test python environment is setup correctly
+test_environment:
+	$(PYTHON_INTERPRETER) test_environment.py
+
+#################################################################################
+# PROJECT RULES                                                                 #
+#################################################################################
+
+
+
+#################################################################################
+# Self Documenting Commands                                                     #
+#################################################################################
+
+.DEFAULT_GOAL := help
+
+# Inspired by <http://marmelab.com/blog/2016/02/29/auto-documented-makefile.html>
+# sed script explained:
+# /^##/:
+# 	* save line in hold space
+# 	* purge line
+# 	* Loop:
+# 		* append newline + line to hold space
+# 		* go to next line
+# 		* if line starts with doc comment, strip comment character off and loop
+# 	* remove target prerequisites
+# 	* append hold space (+ newline) to line
+# 	* replace newline plus comments by `---`
+# 	* print line
+# Separate expressions are necessary because labels cannot be delimited by
+# semicolon; see <http://stackoverflow.com/a/11799865/1968>
+.PHONY: help
+help:
+	@echo "$$(tput bold)Available rules:$$(tput sgr0)"
+	@echo
+	@sed -n -e "/^## / { \
+		h; \
+		s/.*//; \
+		:doc" \
+		-e "H; \
+		n; \
+		s/^## //; \
+		t doc" \
+		-e "s/:.*//; \
+		G; \
+		s/\\n## /---/; \
+		s/\\n/ /g; \
+		p; \
+	}" ${MAKEFILE_LIST} \
+	| LC_ALL='C' sort --ignore-case \
+	| awk -F '---' \
+		-v ncol=$$(tput cols) \
+		-v indent=19 \
+		-v col_on="$$(tput setaf 6)" \
+		-v col_off="$$(tput sgr0)" \
+	'{ \
+		printf "%s%*s%s ", col_on, -indent, $$1, col_off; \
+		n = split($$2, words, " "); \
+		line_length = ncol - indent; \
+		for (i = 1; i <= n; i++) { \
+			line_length -= length(words[i]) + 1; \
+			if (line_length <= 0) { \
+				line_length = ncol - indent - length(words[i]) - 1; \
+				printf "\n%*s ", -indent, " "; \
+			} \
+			printf "%s ", words[i]; \
+		} \
+		printf "\n"; \
+	}' \
+	| more $(shell test $(shell uname) = Darwin && echo '--no-init --raw-control-chars')

data_manipulation-0.50/PKG-INFO

@@ -0,0 +1,84 @@
+Metadata-Version: 2.4
+Name: data_manipulation
+Version: 0.50
+Summary: Powerful data manipulation
+Author-email: Shawn Ng <shawn.coding.acc@gmail.com>
+License-Expression: BSD-3-Clause
+Project-URL: Homepage, https://github.com/shawnngtq/data-manipulation
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Cython
+Classifier: Topic :: Scientific/Engineering
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: beautifulsoup4
+Requires-Dist: boto3
+Requires-Dist: cryptography
+Requires-Dist: django
+Requires-Dist: flask
+Requires-Dist: geopandas
+Requires-Dist: ipython
+Requires-Dist: loguru
+Requires-Dist: mysql-connector-python
+Requires-Dist: numpy
+Requires-Dist: pandas
+Requires-Dist: psycopg
+Requires-Dist: pyspark
+Requires-Dist: python-dotenv
+Requires-Dist: sqlalchemy
+Dynamic: license-file
+
+# Data Manipulation
+
+- [Data Manipulation](#data-manipulation)
+  - [What is it?](#what-is-it)
+  - [Where to get it](#where-to-get-it)
+  - [Dependencies](#dependencies)
+  - [License](#license)
+  - [Getting Help](#getting-help)
+  - [Discussion and Development](#discussion-and-development)
+  - [Contributing to data manipulation](#contributing-to-data-manipulation)
+
+My data manipulation library includes functions build on top of popular Python libraries such as Pandas, PySpark and more.
+
+## What is it?
+
+Data Manipulation is a Python package providing powerful utility functions. It contains many subpackages with utility functions built for popular packages such as Pandas, PySpark and many more.
+
+## Where to get it
+
+The source code is currently hosted on GitHub at: https://github.com/shawnngtq/data-manipulation
+
+```bash
+pip install data-manipulation
+```
+
+## Dependencies
+
+The dependencies will be installed automatically along with this package, using metadata in `pyproject.toml`.
+
+## License
+
+[BSD 3](LICENSE)
+
+## Getting Help
+
+For usage questions, the best place to go to is StackOverflow.
+
+## Discussion and Development
+
+Most development discussion is taking place on github in this repo.
+
+## Contributing to data manipulation
+
+All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome.
+
+As contributors and maintainers to this project, you are expected to abide by our code of conduct. More information can be found at: [Contributor Code of Conduct](https://github.com/shawnngtq/data-manipulation/blob/master/CODE_OF_CONDUCT.md)

{data_manipulation-0.48 → data_manipulation-0.50}/README.md

@@ -25,7 +25,7 @@ pip install data-manipulation
 
 ## Dependencies
 
-The dependencies will be installed automatically along with this package, in `setup.py`
+The dependencies will be installed automatically along with this package, using metadata in `pyproject.toml`.
 
 ## License
 

{data_manipulation-0.48 → data_manipulation-0.50}/data_manipulation/__init__.py

@@ -17,6 +17,12 @@ from data_manipulation import (
     sqlalchemy_,
 )
 
-from . import _version
+try:
+    from ._version import __version__
+except ImportError:
+    try:
+        from importlib.metadata import version
 
-__version__ = _version.get_versions()["version"]
+        __version__ = version("data_manipulation")
+    except Exception:
+        __version__ = "0.0.0"

data_manipulation-0.50/data_manipulation/_version.py

@@ -0,0 +1,24 @@
+# file generated by vcs-versioning
+# don't change, don't track in version control
+from __future__ import annotations
+
+__all__ = [
+    "__version__",
+    "__version_tuple__",
+    "version",
+    "version_tuple",
+    "__commit_id__",
+    "commit_id",
+]
+
+version: str
+__version__: str
+__version_tuple__: tuple[int | str, ...]
+version_tuple: tuple[int | str, ...]
+commit_id: str | None
+__commit_id__: str | None
+
+__version__ = version = '0.50'
+__version_tuple__ = version_tuple = (0, 50)
+
+__commit_id__ = commit_id = 'g3a0e189a0'

{data_manipulation-0.48 → data_manipulation-0.50}/data_manipulation/sqlalchemy_.py

@@ -1,9 +1,10 @@
+from typing import Any, Dict, Optional
+
 import sqlalchemy
-from sqlalchemy.engine.url import URL
+from loguru import logger
 from sqlalchemy.engine.base import Engine
+from sqlalchemy.engine.url import URL
 from sqlalchemy.exc import SQLAlchemyError
-from typing import Optional, Dict, Any
-from loguru import logger
 
 
 class DatabaseConnectionError(Exception):
@@ -27,7 +28,7 @@ def create_sqlalchemy_url(
         drivername (str): Database driver name. Supported options include:
             - 'mysql+mysqlconnector'
             - 'mysql+pymysql'
-            - 'postgresql+psycopg2'
+            - 'postgresql+psycopg'
             - 'mssql+pyodbc'
             - 'oracle+cx_oracle'
             - 'sqlite3'
@@ -44,7 +45,7 @@ def create_sqlalchemy_url(
 
     Examples:
         >>> url = create_sqlalchemy_url(
-        ...     drivername='postgresql+psycopg2',
+        ...     drivername='postgresql+psycopg',
         ...     host='localhost',
         ...     dbname='mydb',
         ...     user='admin',
@@ -52,7 +53,7 @@ def create_sqlalchemy_url(
         ...     port=5432
         ... )
         >>> str(url)
-        'postgresql+psycopg2://admin:secret@localhost:5432/mydb'
+        'postgresql+psycopg://admin:secret@localhost:5432/mydb'
     """
     return URL.create(
         drivername=drivername,
@@ -84,7 +85,7 @@ def create_sqlalchemy_engine(
         drivername (str): Database driver name. Supported options include:
             - 'mysql+mysqlconnector'
             - 'mysql+pymysql'
-            - 'postgresql+psycopg2'
+            - 'postgresql+psycopg'
             - 'mssql+pyodbc'
             - 'oracle+cx_oracle'
             - 'sqlite3'
@@ -107,7 +108,7 @@ def create_sqlalchemy_engine(
 
     Examples:
         >>> engine = create_sqlalchemy_engine(
-        ...     drivername='postgresql+psycopg2',
+        ...     drivername='postgresql+psycopg',
         ...     host='localhost',
         ...     dbname='mydb',
         ...     user='admin',

data_manipulation-0.50/data_manipulation.egg-info/PKG-INFO

@@ -0,0 +1,84 @@
+Metadata-Version: 2.4
+Name: data_manipulation
+Version: 0.50
+Summary: Powerful data manipulation
+Author-email: Shawn Ng <shawn.coding.acc@gmail.com>
+License-Expression: BSD-3-Clause
+Project-URL: Homepage, https://github.com/shawnngtq/data-manipulation
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Cython
+Classifier: Topic :: Scientific/Engineering
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: beautifulsoup4
+Requires-Dist: boto3
+Requires-Dist: cryptography
+Requires-Dist: django
+Requires-Dist: flask
+Requires-Dist: geopandas
+Requires-Dist: ipython
+Requires-Dist: loguru
+Requires-Dist: mysql-connector-python
+Requires-Dist: numpy
+Requires-Dist: pandas
+Requires-Dist: psycopg
+Requires-Dist: pyspark
+Requires-Dist: python-dotenv
+Requires-Dist: sqlalchemy
+Dynamic: license-file
+
+# Data Manipulation
+
+- [Data Manipulation](#data-manipulation)
+  - [What is it?](#what-is-it)
+  - [Where to get it](#where-to-get-it)
+  - [Dependencies](#dependencies)
+  - [License](#license)
+  - [Getting Help](#getting-help)
+  - [Discussion and Development](#discussion-and-development)
+  - [Contributing to data manipulation](#contributing-to-data-manipulation)
+
+My data manipulation library includes functions build on top of popular Python libraries such as Pandas, PySpark and more.
+
+## What is it?
+
+Data Manipulation is a Python package providing powerful utility functions. It contains many subpackages with utility functions built for popular packages such as Pandas, PySpark and many more.
+
+## Where to get it
+
+The source code is currently hosted on GitHub at: https://github.com/shawnngtq/data-manipulation
+
+```bash
+pip install data-manipulation
+```
+
+## Dependencies
+
+The dependencies will be installed automatically along with this package, using metadata in `pyproject.toml`.
+
+## License
+
+[BSD 3](LICENSE)
+
+## Getting Help
+
+For usage questions, the best place to go to is StackOverflow.
+
+## Discussion and Development
+
+Most development discussion is taking place on github in this repo.
+
+## Contributing to data manipulation
+
+All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome.
+
+As contributors and maintainers to this project, you are expected to abide by our code of conduct. More information can be found at: [Contributor Code of Conduct](https://github.com/shawnngtq/data-manipulation/blob/master/CODE_OF_CONDUCT.md)