project-toolkit 1.0.2__tar.gz

project_toolkit-1.0.2/LICENSE

@@ -0,0 +1,20 @@
+MIT License
+
+Copyright (c) 2024 m3air
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

project_toolkit-1.0.2/PKG-INFO

@@ -0,0 +1,419 @@
+Metadata-Version: 2.4
+Name: project-toolkit
+Version: 1.0.2
+Summary: A modular toolkit for managing project paths, cloud services, and configuration in Python projects
+Author: guocity
+License: MIT
+Project-URL: Homepage, https://github.com/guocity/project-toolkit
+Project-URL: Repository, https://github.com/guocity/project-toolkit
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pydantic>=2.0
+Requires-Dist: pydantic-settings>=2.0
+Provides-Extra: google
+Requires-Dist: gspread; extra == "google"
+Requires-Dist: gspread-dataframe; extra == "google"
+Requires-Dist: ratelimit; extra == "google"
+Requires-Dist: google-api-python-client; extra == "google"
+Requires-Dist: google-auth; extra == "google"
+Requires-Dist: google-cloud-bigquery; extra == "google"
+Requires-Dist: pandas; extra == "google"
+Requires-Dist: pandas_gbq; extra == "google"
+Provides-Extra: cloudflare
+Requires-Dist: boto3; extra == "cloudflare"
+Requires-Dist: requests; extra == "cloudflare"
+Provides-Extra: vault
+Requires-Dist: hvac; extra == "vault"
+Provides-Extra: wan-ip
+Requires-Dist: requests; extra == "wan-ip"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: requests; extra == "dev"
+Provides-Extra: all
+Requires-Dist: project-toolkit[cloudflare,dev,google,vault,wan-ip]; extra == "all"
+Dynamic: license-file
+
+# project-toolkit
+
+A modular Python toolkit for managing project paths, cloud services (Google Cloud, Cloudflare R2), HashiCorp Vault, and configuration — powered by **pydantic-settings**.
+
+## Features
+
+- ⚙️ **Auto-Initialized Settings** — Just import and use, no setup needed
+- 🔑 **Dynamic Env Access** — Read any env var or `.env` value with `settings.env("KEY")`
+- 🔐 **Vault Integration** — Read secrets like `vault kv get` / `vault kv get -field=`
+- 📁 **Project Path Management** — Create and navigate project directory structures
+- ☁️ **Cloudflare R2** — Upload/download files with `boto3` and `requests` clients
+- 📊 **Google Services** — BigQuery, Drive, and Sheets integrations
+- 🌐 **WAN IP Tracker** — Detect and log public IP address changes
+
+## Installation
+
+```bash
+# Core only
+pip install project-toolkit
+
+# With specific extras
+pip install "project-toolkit[google]"
+pip install "project-toolkit[cloudflare]"
+pip install "project-toolkit[vault]"
+pip install "project-toolkit[wan-ip]"
+
+# Everything
+pip install "project-toolkit[all]"
+```
+
+---
+
+## API Usage Guide
+
+### 1. Auto-Initialized Settings (No Setup Needed)
+
+Settings auto-initialize on first use — no boilerplate required:
+
+```python
+from project_toolkit import settings
+
+# Access any module setting directly
+print(settings.cloudflare.r2_access_token)
+print(settings.google.google_service_account_json)
+print(settings.vault.is_configured)
+```
+
+**Environment detection** is automatic:
+- If `AIRFLOW_HOME` is set → loads `config/.env.airflow`
+- Otherwise → loads `config/.env.dev`
+
+---
+
+### 2. Dynamic Environment Variable Access
+
+Read **any** env var or `.env` value — no need to pre-define it in code:
+
+```python
+from project_toolkit import settings
+
+# Read API keys (from env vars or .env file)
+openai_key = settings.env("OPENAI_API_KEY")
+gemini_key = settings.env("GEMINI_API_KEY")
+claude_key = settings.env("CLAUDE_API_KEY")
+
+# With a default fallback
+db_host = settings.env("DATABASE_HOST", "localhost")
+debug = settings.env("DEBUG_MODE", "false")
+```
+
+**Priority**: Environment variable > `.env` file value > default
+
+Your `config/.env.dev` can contain any key:
+
+```env
+OPENAI_API_KEY=sk-xxxxx
+GEMINI_API_KEY=AIza-xxxxx
+CLAUDE_API_KEY=sk-ant-xxxxx
+DATABASE_HOST=db.example.com
+```
+
+---
+
+### 3. HashiCorp Vault
+
+Read secrets from Vault — same env vars as the `vault` CLI.
+
+**Setup** — Add to `config/.env.dev`:
+
+```env
+VAULT_ADDR=http://192.168.12.2:8200
+VAULT_TOKEN=hvs.your-vault-token
+```
+
+**Read all fields** (like `vault kv get secret/my-credentials`):
+
+```python
+from project_toolkit import settings
+
+secret = settings.vault.read_secret("secret/my-credentials")
+print(secret)
+# {'username': 'admin', 'password': 'secret-password-123'}
+```
+
+**Read a single field** (like `vault kv get -field=OPENAI_API_KEY secret/api`):
+
+```python
+from project_toolkit import settings
+
+# Single field reads
+api_key = settings.vault.read_field("secret/api", "OPENAI_API_KEY")
+password = settings.vault.read_field("secret/my-credentials", "password")
+
+# With fallback default
+db_pass = settings.vault.read_field("secret/db", "password", "default-pw")
+```
+
+**Use the hvac client directly** for advanced operations:
+
+```python
+from project_toolkit import settings
+
+client = settings.vault.get_vault_client()
+if client:
+    # Write a secret
+    client.secrets.kv.v2.create_or_update_secret(
+        path="my-new-secret",
+        secret={"api_key": "abc123"},
+    )
+
+    # List secrets
+    secrets = client.secrets.kv.v2.list_secrets(path="", mount_point="secret")
+    print(secrets["data"]["keys"])
+```
+
+---
+
+### 4. Manual Settings Override
+
+For testing or custom configurations, construct `AppSettings` directly:
+
+```python
+from project_toolkit.settings.base import AppSettings
+from project_toolkit.settings.cloudflare import CloudflareSettings
+from project_toolkit.settings.google import GoogleSettings
+from project_toolkit.settings.path import PathSettings
+from project_toolkit.settings.vault import VaultSettings
+from project_toolkit.settings.wan_ip import WanIpSettings
+
+custom_settings = AppSettings(
+    path_config=PathSettings(
+        data_dir="/custom/data",
+        project_name="my-project",
+    ),
+    cloudflare=CloudflareSettings(
+        r2_access_token="custom-token",
+        r2_account_id="custom-id",
+    ),
+    google=GoogleSettings(),
+    vault=VaultSettings(),
+    wan_ip=WanIpSettings(),
+)
+
+# Use the custom settings
+print(custom_settings.cloudflare.r2_access_token)  # "custom-token"
+print(custom_settings.env("R2_ACCESS_TOKEN"))       # reads from env/dotenv
+```
+
+To reset the cached auto-initialized singleton:
+
+```python
+from project_toolkit import settings, get_settings
+
+# Reset cached settings (e.g., after changing env vars)
+get_settings.cache_clear()
+settings._reset()
+
+# Next access will re-initialize
+print(settings.cloudflare.r2_access_token)
+```
+
+---
+
+### 5. Project Path Management
+
+Manage project directory structures with automatic creation:
+
+```python
+from project_toolkit import DataPathConfig
+
+# Direct usage
+dpc = DataPathConfig(
+    project_name="my-project",
+    subproject="etl",
+    data_dir="/data",
+)
+print(dpc.data_dir())        # /data
+print(dpc.project_dir())     # /data/my-project
+print(dpc.sub_project_dir()) # /data/my-project/etl
+
+# Get a timestamped file path
+filepath = dpc.get_filepath("output", "csv")
+# /data/my-project/etl/output_20260209.csv
+```
+
+**From auto-initialized settings** (recommended):
+
+```python
+from project_toolkit import settings, DataPathConfig
+
+dpc = DataPathConfig.from_settings(
+    settings.path_config,
+    project_name="my-project",
+)
+```
+
+---
+
+### 6. Cloudflare R2
+
+```python
+from project_toolkit import settings
+from project_toolkit.cloudflare.boto3_client import CloudflareBoto3Client
+
+# From auto-initialized settings
+client = CloudflareBoto3Client.from_settings(settings.cloudflare)
+
+# Upload a file
+client.upload_file("my-bucket", "file.csv", open("/path/to/file.csv", "rb").read())
+
+# List buckets
+buckets = client.list_buckets()
+```
+
+---
+
+### 7. Google Sheets
+
+```python
+from project_toolkit import settings
+from project_toolkit.google.sheet import GoogleSheetsManager
+
+sheets = GoogleSheetsManager.from_settings(settings.google)
+
+# Read a sheet as a DataFrame
+df = sheets.sheet_df(sheet_id="your-sheet-id")
+print(df.head())
+```
+
+---
+
+### 8. Google BigQuery
+
+```python
+from project_toolkit import settings
+from project_toolkit.google.bigquery import BigQueryManager
+
+bq = BigQueryManager.from_settings(settings.google)
+
+# Run a query
+df = bq.query_table("SELECT * FROM my_dataset.my_table LIMIT 10")
+```
+
+---
+
+### 9. Google Drive
+
+```python
+from project_toolkit import settings
+from project_toolkit.google.drive import GoogleDriveManager
+
+drive = GoogleDriveManager.from_settings(settings.google)
+
+# List files in a folder
+files = drive.list_files(folder_id="your-folder-id")
+```
+
+---
+
+### 10. WAN IP Tracker
+
+```python
+from project_toolkit.wan_ip import WanIpTracker
+
+tracker = WanIpTracker(log_dir="/tmp/ip-logs")
+
+# Get current IPs (includes datetime)
+print(tracker)
+# Date: 2026-02-09 22:31:45
+# IPv4: 1.2.3.4
+# IPv6: 2001:db8::1
+
+# Log only if changed
+entry = tracker.log_if_changed()
+```
+
+---
+
+## Configuration
+
+### Environment Variables
+
+| Variable | Module | Description |
+|---|---|---|
+| `R2_ACCESS_TOKEN` | Cloudflare | Cloudflare R2 API token |
+| `R2_ACCOUNT_ID` | Cloudflare | Cloudflare account ID |
+| `R2_DOMAIN` | Cloudflare | R2 custom domain URL |
+| `GOOGLE_SERVICE_ACCOUNT_JSON` | Google | Path to service account JSON |
+| `TEST_GOOGLE_SHEET_ID` | Google | Test sheet ID (dev only) |
+| `VAULT_ADDR` | Vault | Vault server address (e.g., `http://192.168.12.2:8200`) |
+| `VAULT_TOKEN` | Vault | Vault authentication token |
+| `WAN_IP_LOG_DIR` | WAN IP | Directory for IP change logs |
+| `AIRFLOW_HOME` | Core | Auto-detected for env switching |
+
+> **Any key** in your `.env` file or environment can be read dynamically
+> via `settings.env("KEY")` — no need to define it in Python code.
+
+### `.env` Files
+
+Place config files in the `config/` directory:
+
+```
+config/
+├── .env.dev        # Local development (loaded when no AIRFLOW_HOME)
+└── .env.airflow    # Airflow production (loaded when AIRFLOW_HOME is set)
+```
+
+### Priority Order
+
+Environment variables > `.env` files > Vault secrets > defaults
+
+---
+
+## Development
+
+```bash
+# Install with all dependencies
+pip install -e ".[all]"
+
+# Run tests
+pytest
+
+# Run specific tests
+pytest tests/test_settings.py -s -vv
+pytest tests/test_vault.py -s -vv
+pytest tests/test_auto_settings.py -s -vv
+```
+
+## Project Structure
+
+```
+project_toolkit/
+├── __init__.py           # Package exports + auto-initialized settings singleton
+├── project_path.py       # Project path management (DataPathConfig)
+├── settings/
+│   ├── __init__.py       # Settings exports
+│   ├── base.py           # AppSettings + get_settings() + env()
+│   ├── path.py           # PathSettings
+│   ├── cloudflare.py     # CloudflareSettings
+│   ├── google.py         # GoogleSettings
+│   ├── vault.py          # VaultSettings + read_secret() + read_field()
+│   └── wan_ip.py         # WanIpSettings
+├── cloudflare/
+│   ├── boto3_client.py   # R2 via boto3
+│   └── requests_client.py # R2 via requests
+├── google/
+│   ├── bigquery.py       # BigQuery operations
+│   ├── drive.py          # Google Drive operations
+│   └── sheet.py          # Google Sheets operations
+└── wan_ip/
+    └── tracker.py        # WAN IP detection + logging
+```
+
+## License
+
+MIT