datus-clickzetta 0.1.0__tar.gz

datus_clickzetta-0.1.0/.gitignore

@@ -0,0 +1,137 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+Pipfile.lock
+
+# uv
+uv.lock
+
+# PEP 582
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# IDEs
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+
+# OS
+.DS_Store
+Thumbs.db

datus_clickzetta-0.1.0/PKG-INFO

@@ -0,0 +1,409 @@
+Metadata-Version: 2.4
+Name: datus-clickzetta
+Version: 0.1.0
+Summary: ClickZetta database adapter for Datus
+Project-URL: Homepage, https://github.com/Datus-ai/datus-db-adapters
+Project-URL: Repository, https://github.com/Datus-ai/datus-db-adapters
+Project-URL: Issues, https://github.com/Datus-ai/datus-db-adapters/issues
+Author-email: DatusAI <support@datus.ai>
+License: Apache-2.0
+Keywords: adapter,clickzetta,database,datus,lakehouse
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.12
+Requires-Dist: clickzetta-connector-python
+Requires-Dist: clickzetta-zettapark-python
+Requires-Dist: datus-agent>=0.2.1
+Requires-Dist: pandas>=2.0.0
+Requires-Dist: pyarrow>=10.0.0
+Description-Content-Type: text/markdown
+
+# Datus ClickZetta Adapter
+
+This package provides a [ClickZetta](https://www.singdata.com/) Lakehouse adapter for [Datus](https://github.com/Datus-ai/datus-agent), enabling seamless integration with ClickZetta analytics platform.
+
+[ClickZetta](https://www.singdata.com/) is developed by [Singdata](https://www.singdata.com/) and [Yunqi](https://www.yunqi.tech/).
+
+## Installation
+
+```bash
+pip install datus-clickzetta
+```
+
+## Dependencies
+
+This adapter requires the following ClickZetta Python packages:
+- `clickzetta-connector-python`
+- `clickzetta-zettapark-python`
+
+## Configuration
+
+Configure ClickZetta connection in your Datus configuration. A complete example is available at [`examples/agent.clickzetta.yml.example`](examples/agent.clickzetta.yml.example).
+
+```yaml
+namespace:
+  clickzetta_prod:
+    type: clickzetta
+    service: "your-service-endpoint.clickzetta.com"
+    username: "your-username"
+    password: "your-password"
+    instance: "your-instance-id"
+    workspace: "your-workspace"
+    schema: "PUBLIC"  # optional, defaults to PUBLIC
+    vcluster: "DEFAULT_AP"  # optional, defaults to DEFAULT_AP
+    secure: false  # optional
+```
+
+### Configuration Parameters
+
+| Parameter | Type | Required | Default | Description |
+|-----------|------|----------|---------|-------------|
+| `service` | string | Yes | - | ClickZetta service endpoint |
+| `username` | string | Yes | - | ClickZetta username |
+| `password` | string | Yes | - | ClickZetta password |
+| `instance` | string | Yes | - | ClickZetta instance identifier |
+| `workspace` | string | Yes | - | ClickZetta workspace name |
+| `schema` | string | No | "PUBLIC" | Default schema name |
+| `vcluster` | string | No | "DEFAULT_AP" | Virtual cluster name |
+| `secure` | boolean | No | null | Enable secure connection |
+| `hints` | object | No | {} | Additional connection hints |
+| `extra` | object | No | {} | Extra connection parameters |
+
+## Features
+
+- **Full SQL Support**: Execute queries, DDL, DML operations
+- **Metadata Discovery**: Automatic discovery of databases, schemas, tables, and views
+- **Volume Integration**: Read files from ClickZetta volumes
+- **Sample Data**: Extract sample rows for data profiling
+- **Connection Management**: Automatic connection pooling and session management
+
+## Usage
+
+Once installed and configured, use the ClickZetta adapter with Datus:
+
+```python
+# Execute queries
+result = agent.query("SELECT * FROM my_table LIMIT 10")
+
+# Get table information
+tables = agent.get_tables("my_schema")
+```
+
+## Volume Operations
+
+The adapter supports reading files from ClickZetta volumes:
+
+```python
+# Read a file from a volume
+content = connector.read_volume_file("volume:user://my_volume", "path/to/file.yaml")
+
+# List files in a volume directory
+files = connector.list_volume_files("volume:user://my_volume", "config/", suffixes=(".yaml", ".yml"))
+```
+
+## Connection Hints
+
+You can customize ClickZetta connection behavior using hints:
+
+```yaml
+namespace:
+  clickzetta_prod:
+    type: clickzetta
+    # ... other connection parameters
+    hints:
+      sdk.job.timeout: 600
+      query_tag: "Datus Analytics Query"
+      cz.storage.parquet.vector.index.read.memory.cache: "true"
+```
+
+## Error Handling
+
+The adapter provides comprehensive error handling with detailed error messages for common issues:
+
+- Connection failures
+- Authentication errors
+- Query execution errors
+- Schema/workspace switching limitations
+
+## Development
+
+### Development Mode Setup (Complete Guide)
+
+This guide covers the complete setup from Datus agent installation to ClickZetta adapter development and testing.
+
+#### Prerequisites
+- Python 3.11+ recommended
+- Git
+
+#### Step 1: Setup Datus Agent Development Environment
+
+```bash
+# Clone the Datus agent repository
+git clone https://github.com/Datus-ai/datus-agent.git
+cd datus-agent
+
+# Create and activate virtual environment
+python -m venv .venv
+source .venv/bin/activate  # On Windows: .venv\Scripts\activate
+
+# Install Datus agent in editable mode
+pip install -e .
+```
+
+#### Step 2: Clone and Install ClickZetta Adapter
+
+```bash
+# From your development directory
+git clone https://github.com/Datus-ai/datus-db-adapters.git
+cd datus-db-adapters/datus-clickzetta
+
+# Install ClickZetta adapter in editable mode (using the same virtual environment)
+pip install -e .
+
+# Verify installation
+pip show datus-clickzetta
+```
+
+#### Step 3: Configure Environment Variables
+
+Create a `.env` file or set environment variables:
+
+```bash
+# ClickZetta connection settings
+export CLICKZETTA_SERVICE="your-service.clickzetta.com"
+export CLICKZETTA_USERNAME="your-username"
+export CLICKZETTA_PASSWORD="your-password"
+export CLICKZETTA_INSTANCE="your-instance-id"
+export CLICKZETTA_WORKSPACE="your-workspace"
+export CLICKZETTA_SCHEMA="your-schema"
+export CLICKZETTA_VCLUSTER="default_ap"
+
+# LLM API keys (optional for testing)
+export DASHSCOPE_API_KEY="your-dashscope-key"
+export DEEPSEEK_API_KEY="your-deepseek-key"
+```
+
+#### Step 4: Create ClickZetta Configuration
+
+In your Datus agent directory, create a ClickZetta configuration file using the provided example:
+
+```bash
+# In datus-agent directory
+cp ../datus-db-adapters/datus-clickzetta/examples/agent.clickzetta.yml.example conf/agent.clickzetta.yml
+```
+
+Update `conf/agent.clickzetta.yml` with ClickZetta settings:
+
+```yaml
+agent:
+  target: qwen_main  # or your preferred model
+  home: .datus_home
+
+  models:
+    # Your model configurations here
+
+  namespace:
+    clickzetta:
+      type: clickzetta
+      service: ${CLICKZETTA_SERVICE}
+      username: ${CLICKZETTA_USERNAME}
+      password: ${CLICKZETTA_PASSWORD}
+      instance: ${CLICKZETTA_INSTANCE}
+      workspace: ${CLICKZETTA_WORKSPACE}
+      schema: ${CLICKZETTA_SCHEMA}
+      vcluster: ${CLICKZETTA_VCLUSTER}
+      secure: false
+```
+
+#### Step 5: Start Development and Testing
+
+**Test the adapter directly:**
+```bash
+# From datus-clickzetta directory
+python -c "
+from datus_clickzetta.connector import ClickZettaConnector
+import os
+
+connector = ClickZettaConnector(
+    service=os.getenv('CLICKZETTA_SERVICE'),
+    username=os.getenv('CLICKZETTA_USERNAME'),
+    password=os.getenv('CLICKZETTA_PASSWORD'),
+    instance=os.getenv('CLICKZETTA_INSTANCE'),
+    workspace=os.getenv('CLICKZETTA_WORKSPACE'),
+    schema=os.getenv('CLICKZETTA_SCHEMA'),
+    vcluster=os.getenv('CLICKZETTA_VCLUSTER'),
+    secure=False
+)
+
+result = connector.execute('SHOW SCHEMAS')
+print(f'Connected! Found {result.row_count} schemas')
+"
+```
+
+**Start Datus CLI with ClickZetta:**
+```bash
+# From datus-agent directory
+python -m datus.cli.main --config conf/agent.clickzetta.yml --namespace clickzetta
+```
+
+#### Step 6: Development Workflow
+
+**Making Changes:**
+1. Edit code in `datus-clickzetta/datus_clickzetta/connector.py`
+2. Changes are immediately available (editable install)
+3. No need to reinstall the package
+
+**Testing Changes:**
+```bash
+# Run adapter tests
+cd datus-clickzetta
+python test.py
+
+# Test with Datus CLI
+cd ../datus-agent
+python -m datus.cli.main --config conf/agent.clickzetta.yml --namespace clickzetta
+```
+
+**Commit and Push:**
+```bash
+# From adapter directory
+git add .
+git commit -m "Your changes"
+git push origin your-branch
+
+# From agent directory (if you made agent changes)
+git add .
+git commit -m "Your agent changes"
+git push origin your-branch
+```
+
+#### Directory Structure
+
+```text
+your-dev-folder/
+├── datus-agent/                    # Datus agent repository
+│   ├── .venv/                     # Shared virtual environment
+│   ├── conf/agent.clickzetta.yml  # ClickZetta configuration
+│   └── ...
+└── datus-db-adapters/             # Adapters repository
+    └── datus-clickzetta/          # ClickZetta adapter
+        ├── datus_clickzetta/
+        │   └── connector.py       # Main connector code
+        └── ...
+```
+
+#### Tips for Development
+
+- **Editable Installs**: Both packages are installed in editable mode, so code changes are immediate
+- **Environment Variables**: Use `.env` files for local development, environment variables for production
+- **Testing**: Always test both the adapter directly and through the Datus CLI
+- **Debugging**: Use `logger.debug()` statements; enable with `DATUS_LOG_LEVEL=DEBUG`
+
+#### Contributing Guidelines
+
+1. Clone the repository
+2. Create a feature branch
+3. Make your changes
+4. Run tests: `python test.py`
+5. Ensure code style compliance
+6. Submit a pull request
+
+#### Common Development Issues
+
+**Import Errors:**
+- Ensure both packages are installed in editable mode
+- Check virtual environment is activated
+
+**Connection Issues:**
+- Verify environment variables are set
+- Test connection with the direct connector test above
+
+**CLI Issues:**
+- Check configuration file syntax
+- Verify namespace configuration matches your environment
+
+## Testing
+
+This adapter includes comprehensive test coverage with multiple test types and execution modes.
+
+### Test Structure
+
+```text
+tests/
+├── unit/                     # Unit tests for individual components
+├── integration/              # Integration tests with mocked dependencies
+├── run_tests.py             # Main test runner with multiple modes
+├── comprehensive_test.py     # Real connection testing script
+└── conftest.py              # Shared test fixtures and configuration
+```
+
+### Running Tests
+
+**Quick Start (from project root):**
+```bash
+# Run all tests
+python test.py
+
+# Run specific test types
+python test.py --mode unit          # Unit tests only (fastest)
+python test.py --mode integration   # Integration tests only
+python test.py --mode all          # All tests
+python test.py --mode coverage     # Tests with coverage report
+```
+
+**Advanced Usage (from tests/ directory):**
+```bash
+cd tests
+
+# Basic test execution
+python run_tests.py --mode unit
+python run_tests.py --mode integration -v
+
+# Real connection testing (requires credentials)
+python comprehensive_test.py
+
+# Direct pytest usage
+pytest unit/                    # Unit tests
+pytest integration/             # Integration tests
+pytest -k "test_config"        # Specific test patterns
+```
+
+### Test Requirements
+
+- **Unit Tests**: No external dependencies, run with mocked components
+- **Integration Tests**: Mocked ClickZetta SDK, test connector logic
+- **Real Connection Tests**: Require actual ClickZetta credentials
+
+Set environment variables for real connection testing:
+```bash
+export CLICKZETTA_SERVICE="your-service.clickzetta.com"
+export CLICKZETTA_USERNAME="your-username"
+export CLICKZETTA_PASSWORD="your-password"
+export CLICKZETTA_INSTANCE="your-instance"
+export CLICKZETTA_WORKSPACE="your-workspace"
+export CLICKZETTA_SCHEMA="your-schema"
+export CLICKZETTA_VCLUSTER="your-vcluster"
+```
+
+### Test Coverage
+
+- ✅ Configuration validation and error handling
+- ✅ SQL query execution and result processing
+- ✅ Metadata discovery (tables, views, schemas)
+- ✅ Connection management and lifecycle
+- ✅ Volume operations and file listing
+- ✅ Error handling and exception cases
+
+## License
+
+This project is licensed under the Apache 2.0 License - see the [LICENSE](../LICENSE) file for details.
+
+## Support
+
+For issues and questions:
+- [GitHub Issues](https://github.com/Datus-ai/datus-db-adapters/issues)
+- [Datus Documentation](https://docs.datus.ai/)
+- [ClickZetta Documentation](https://www.yunqi.tech/documents)