aio-sf 0.1.0b1__tar.gz

aio_sf-0.1.0b1/.cursor/rules/async-patterns.mdc

@@ -0,0 +1,37 @@
+---
+globs: *.py
+description: "Async programming patterns and conventions"
+---
+
+# Async Programming Patterns
+
+This project uses async/await patterns throughout. Follow these conventions:
+
+## Connection Management
+- Always use `async with SalesforceConnection(auth_strategy=auth_strategy) as sf:` for connection handling
+- Connection objects should be passed to async functions, not created within them
+- Use `await sf.ensure_authenticated()` before making API calls
+
+## Bulk Query Patterns
+```python
+# Correct async bulk query usage
+query_result = await bulk_query(
+    sf=sf,
+    soql_query="SELECT Id, Name FROM Contact"
+)
+
+# Async iteration over results
+async for record in query_result:
+    # process record
+    pass
+```
+
+## Authentication Strategies
+- Use `ClientCredentialsAuth` for OAuth client credentials flow
+- Use `StaticTokenAuth` for existing access tokens
+- Use `RefreshTokenAuth` for refresh token flow
+
+## Error Handling
+- Catch `SalesforceAuthError` for authentication issues
+- Use proper async context managers for resource cleanup
+- Always handle connection timeouts and retries appropriately

aio_sf-0.1.0b1/.cursor/rules/data-export.mdc

@@ -0,0 +1,39 @@
+---
+description: "Data export patterns and best practices"
+---
+
+# Data Export Patterns
+
+## Parquet Export Workflow
+1. **Get metadata** (optional but recommended): `get_bulk_fields(sf, object_name)`
+2. **Create schema**: `create_schema_from_metadata(fields_metadata)`
+3. **Execute query**: `bulk_query(sf, soql_query)`
+4. **Export data**: `write_query_to_parquet_async(query_result, file_path, schema=schema)`
+
+## Export Options
+- **Inferred schema**: Let pandas/pyarrow infer types from data
+- **Metadata schema**: Use Salesforce field metadata for proper typing
+- **Batch processing**: Configure batch sizes for memory efficiency
+- **Empty value handling**: Convert empty strings to null values
+
+## Memory Management
+```python
+# For large datasets, use smaller batch sizes
+writer = ParquetWriter(
+    file_path="large_dataset.parquet",
+    schema=schema,
+    batch_size=1000,  # Smaller batches for memory efficiency
+    convert_empty_to_null=True
+)
+writer.write_query_result(query_result)
+```
+
+## Resume Capability
+```python
+# Resume from job locator for interrupted queries
+query_result = resume_from_locator(
+    sf=sf,
+    job_id="job_id_from_previous_run",
+    locator="locator_from_previous_run"
+)
+```

aio_sf-0.1.0b1/.cursor/rules/import-conventions.mdc

@@ -0,0 +1,38 @@
+---
+globs: *.py
+description: "Import conventions and module organization"
+---
+
+# Import Conventions
+
+## Main Package Imports
+```python
+# Core connection functionality
+from aio_salesforce import SalesforceConnection, ClientCredentialsAuth
+
+# Bulk query and export
+from aio_salesforce import bulk_query, ParquetWriter
+
+# Direct exporter imports
+from aio_salesforce.exporter import (
+    bulk_query, 
+    get_bulk_fields,
+    create_schema_from_metadata,
+    write_query_to_parquet_async
+)
+```
+
+## Internal Module Imports
+- Use relative imports within the package: `from ..api.types import FieldInfo`
+- Import from parent modules: `from ..connection import SalesforceConnection`
+- Keep imports organized: connection, then API, then utilities
+
+## Export Patterns
+- All public functions must be listed in `__all__`
+- Use `# noqa: F401` for re-exported imports
+- Maintain consistent export structure across modules
+
+## API Module Structure
+- `bulk_v2/` - Bulk API 2.0 client and types
+- `metadata/` - Metadata API client and types  
+- `query/` - SOQL query client and types

aio_sf-0.1.0b1/.cursor/rules/packaging.mdc

@@ -0,0 +1,40 @@
+---
+globs: pyproject.toml,setup.py,setup.cfg
+description: "Package configuration and distribution"
+---
+
+# Package Configuration
+
+## Build System
+- Uses `hatchling` as build backend
+- Version dynamically read from `src/aio_salesforce/__init__.py`
+- Built with `uv build` command
+
+## Dependencies
+- **Core**: httpx, requests, pandas, pyarrow, pydantic, boto3
+- **Auth**: python-dotenv for environment variables
+- **CLI**: typer (for future CLI implementation)
+- **Dev**: pytest, black, ruff, mypy, pre-commit
+
+## Distribution
+- **Package name**: `aio-salesforce` (PyPI)
+- **Import name**: `aio_salesforce` (Python)
+- **Version**: Semantic versioning (currently 0.1.0)
+
+## Building and Testing
+```bash
+# Build package
+uv build
+
+# Test installation in clean environment
+uv init test-project && cd test-project
+uv add /path/to/dist/aio_salesforce-0.1.0-py3-none-any.whl
+uv run python -c "import aio_salesforce; print('Success!')"
+```
+
+## Release Checklist
+1. Update version in `__init__.py`
+2. Update CHANGELOG/README if needed
+3. Run `uv build`
+4. Test wheel installation
+5. Publish to PyPI: `uv publish`

aio_sf-0.1.0b1/.cursor/rules/project-structure.mdc

@@ -0,0 +1,26 @@
+---
+alwaysApply: true
+---
+
+# aio-salesforce Project Structure
+
+This is an async Salesforce library for Python with Bulk API 2.0 support.
+
+## Core Structure
+- **Main package**: [src/aio_salesforce/__init__.py](mdc:src/aio_salesforce/__init__.py) - Main package exports and version info
+- **Connection handling**: [src/aio_salesforce/connection.py](mdc:src/aio_salesforce/connection.py) - Authentication and connection management
+- **API modules**: [src/aio_salesforce/api/](mdc:src/aio_salesforce/api/) - Salesforce API clients (bulk_v2, metadata, query)
+- **Export functionality**: [src/aio_salesforce/exporter/](mdc:src/aio_salesforce/exporter/) - Data export utilities
+
+## Key Modules
+- **Bulk Export**: [src/aio_salesforce/exporter/bulk_export.py](mdc:src/aio_salesforce/exporter/bulk_export.py) - Bulk API 2.0 query functionality
+- **Parquet Writer**: [src/aio_salesforce/exporter/parquet_writer.py](mdc:src/aio_salesforce/exporter/parquet_writer.py) - Parquet file export with schema mapping
+- **Configuration**: [pyproject.toml](mdc:pyproject.toml) - Package configuration and dependencies
+- **Documentation**: [README.md](mdc:README.md) - Usage examples and API reference
+
+## Package Name
+- **PyPI name**: `aio-salesforce` (with hyphen)
+- **Import name**: `aio_salesforce` (with underscore)
+
+## Testing
+- **Test file**: [test.py](mdc:test.py) - Example usage and testing

aio_sf-0.1.0b1/.cursor/rules/salesforce-api.mdc

@@ -0,0 +1,37 @@
+---
+description: "Salesforce API integration patterns and best practices"
+---
+
+# Salesforce API Integration Patterns
+
+## Authentication Flow
+1. **Client Credentials** (Recommended): Use OAuth client credentials flow
+2. **Static Token**: Use existing access token
+3. **Refresh Token**: Use refresh token to obtain access tokens
+
+## Bulk API 2.0 Usage
+- Use for large data queries (>2000 records)
+- Supports both sync and async query execution
+- Handles pagination automatically via locators
+- Provides resume capability for interrupted queries
+
+## Schema Handling
+```python
+# Get field metadata for proper type mapping
+fields_metadata = await get_bulk_fields(sf, 'Contact')
+schema = create_schema_from_metadata(fields_metadata)
+
+# Schema automatically filters to match query fields
+query_result = await bulk_query(sf, "SELECT Id, Name FROM Contact")
+await write_query_to_parquet_async(query_result, "contacts.parquet", schema=schema)
+```
+
+## Type Mapping
+- Salesforce types → PyArrow types automatically handled
+- Support for: string, boolean, int, double, currency, percent, date, datetime, reference, picklist, id
+- Empty strings converted to null when `convert_empty_to_null=True`
+
+## API Versioning
+- Default: v60.0
+- Configurable per connection
+- Consistent across all API calls within a connection

aio_sf-0.1.0b1/.github/workflows/publish.yml

@@ -0,0 +1,72 @@
+name: Publish Python 🐍 distribution 📦 to PyPI and TestPyPI
+
+on: push
+
+jobs:
+  build:
+    name: Build distribution 📦
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: "3.x"
+    - name: Install uv
+      uses: astral-sh/setup-uv@v3
+      with:
+        version: "latest"
+    - name: Build a binary wheel and a source tarball
+      run: uv build
+    - name: Store the distribution packages
+      uses: actions/upload-artifact@v4
+      with:
+        name: python-package-distributions
+        path: dist/
+
+  publish-to-testpypi:
+    name: Publish Python 🐍 distribution 📦 to TestPyPI
+    needs:
+    - build
+    runs-on: ubuntu-latest
+
+    environment:
+      name: testpypi
+      url: https://test.pypi.org/p/aio-sf
+
+    permissions:
+      id-token: write  # IMPORTANT: mandatory for trusted publishing
+
+    steps:
+    - name: Download all the dists
+      uses: actions/download-artifact@v4
+      with:
+        name: python-package-distributions
+        path: dist/
+    - name: Publish distribution 📦 to TestPyPI
+      uses: pypa/gh-action-pypi-publish@release/v1
+      with:
+        repository-url: https://test.pypi.org/legacy/
+
+  publish-to-pypi:
+    name: >-
+      Publish Python 🐍 distribution 📦 to PyPI
+    if: startsWith(github.ref, 'refs/tags/')  # only publish to PyPI on tag pushes
+    needs:
+    - build
+    runs-on: ubuntu-latest
+    environment:
+      name: pypi
+      url: https://pypi.org/p/aio-sf
+    permissions:
+      id-token: write  # IMPORTANT: mandatory for trusted publishing
+
+    steps:
+    - name: Download all the dists
+      uses: actions/download-artifact@v4
+      with:
+        name: python-package-distributions
+        path: dist/
+    - name: Publish distribution 📦 to PyPI
+      uses: pypa/gh-action-pypi-publish@release/v1

aio_sf-0.1.0b1/.github/workflows/test.yml

@@ -0,0 +1,51 @@
+name: Test
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.9", "3.10", "3.11", "3.12"]
+    
+    steps:
+    - uses: actions/checkout@v4
+    
+    - name: Install uv
+      uses: astral-sh/setup-uv@v3
+      with:
+        version: "latest"
+    
+    - name: Set up Python ${{ matrix.python-version }}
+      run: uv python install ${{ matrix.python-version }}
+    
+    - name: Install dependencies
+      run: uv sync --all-extras --dev
+    
+    - name: Test core installation (no exporter)
+      run: |
+        uv run --isolated python -c "
+        from aio_salesforce import SalesforceConnection, ClientCredentialsAuth
+        print('✅ Core installation works')
+        "
+    
+    - name: Test full installation (with exporter)
+      run: |
+        uv run python -c "
+        from aio_salesforce import SalesforceConnection
+        from aio_salesforce.exporter import bulk_query, ParquetWriter
+        print('✅ Full installation works')
+        "
+    
+    - name: Build package
+      run: uv build
+    
+    - name: Test wheel installation
+      run: |
+        uv run --isolated pip install dist/*.whl
+        uv run --isolated python -c "import aio_salesforce; print('✅ Wheel installation works')"

aio_sf-0.1.0b1/.gitignore

@@ -0,0 +1,207 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[codz]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py.cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# UV
+#   Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#uv.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+#poetry.toml
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#   pdm recommends including project-wide configuration in pdm.toml, but excluding .pdm-python.
+#   https://pdm-project.org/en/latest/usage/project/#working-with-version-control
+#pdm.lock
+#pdm.toml
+.pdm-python
+.pdm-build/
+
+# pixi
+#   Similar to Pipfile.lock, it is generally recommended to include pixi.lock in version control.
+#pixi.lock
+#   Pixi creates a virtual environment in the .pixi directory, just like venv module creates one
+#   in the .venv directory. It is recommended not to include this directory in version control.
+.pixi
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.envrc
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+
+# Abstra
+# Abstra is an AI-powered process automation framework.
+# Ignore directories containing user credentials, local state, and settings.
+# Learn more at https://abstra.io/docs
+.abstra/
+
+# Visual Studio Code
+#  Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore 
+#  that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
+#  and can be added to the global gitignore or merged into this file. However, if you prefer, 
+#  you could uncomment the following to ignore the entire vscode folder
+# .vscode/
+
+# Ruff stuff:
+.ruff_cache/
+
+# PyPI configuration file
+.pypirc
+
+# Cursor
+#  Cursor is an AI-powered code editor. `.cursorignore` specifies files/directories to
+#  exclude from AI features like autocomplete and code analysis. Recommended for sensitive data
+#  refer to https://docs.cursor.com/context/ignore-files
+.cursorignore
+.cursorindexingignore
+
+# Marimo
+marimo/_static/
+marimo/_lsp/
+__marimo__/

aio_sf-0.1.0b1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Callaway Cloud
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.