synapse-sdk 1.0.0a98__tar.gz → 1.0.0b2__tar.gz

{synapse_sdk-1.0.0a98 → synapse_sdk-1.0.0b2}/AGENT.md

@@ -360,4 +360,191 @@ pytest --cov=synapse_sdk
 - Ask for clarification if feedback is unclear
 - Make requested changes promptly
 - Re-run formatting and tests after changes
-- Update documentation as needed
+- Update documentation as needed
+
+## 🌐 Synapse Backend API Reference
+
+This section documents the Synapse Backend API endpoints and data structures discovered through exploration and implementation.
+
+### API Root Endpoints
+
+The API root (`https://api.test.synapse.sh/`) returns a comprehensive list of available endpoints:
+
+```json
+{
+    "agents": "https://api.test.synapse.sh/agents/",
+    "jobs": "https://api.test.synapse.sh/jobs/",
+    "serve_applications": "https://api.test.synapse.sh/serve_applications/",
+    "plugins": "https://api.test.synapse.sh/plugins/",
+    "plugin_releases": "https://api.test.synapse.sh/plugin_releases/",
+    "storages": "https://api.test.synapse.sh/storages/",
+    "data_collections": "https://api.test.synapse.sh/data_collections/",
+    // ... and many more
+}
+```
+
+### Authentication
+
+All API requests require authentication using the `SYNAPSE-Access-Token` header:
+
+```bash
+curl -H "SYNAPSE-Access-Token: Token YOUR_TOKEN_HERE" \
+     -H "Accept: application/json" \
+     https://api.test.synapse.sh/endpoint/
+```
+
+### Key API Endpoints
+
+#### 1. Agents API (`/agents/`)
+
+**GET /agents/{id}/** - Retrieve agent details
+
+Response structure:
+```json
+{
+    "id": 1,
+    "name": "기본 에이전트",
+    "url": "http://10.0.22.1:8000",
+    "status": "connected",
+    "status_display": "Connected",
+    "is_connected": true,
+    "is_active": true,
+    "service_provider": "datamaker",
+    "service_provider_display": "datamaker cloud",
+    "last_connected": "2025-08-04T13:29:08.106889+09:00",
+    "nodes": [
+        {
+            "id": "node_id",
+            "status": "ALIVE",
+            "host_name": "synapse-agent-test",
+            "ip": "10.0.22.1",
+            "cpu": {...},
+            "memory": {...},
+            "disk": {...},
+            "gpus": [...]
+        }
+    ]
+}
+```
+
+#### 2. Plugin Releases API (`/plugin_releases/`)
+
+**GET /plugin_releases/{id}/** - Retrieve plugin release details
+
+Response structure:
+```json
+{
+    "id": 26,
+    "plugin": 20,  // References the plugin ID
+    "version": "0.1.3",
+    "config": {
+        "code": "yolov8",
+        "name": "yolov8",
+        "tasks": ["image.object_detection"],
+        "actions": {...},
+        "category": "neural_net",
+        // ... other config fields
+    },
+    "readme": "# Plugin documentation...",
+    "created": "2025-06-10T10:51:52.814743+09:00"
+}
+```
+
+#### 3. Plugins API (`/plugins/`)
+
+**GET /plugins/** - List all plugins
+
+Response structure:
+```json
+{
+    "count": 33,
+    "next": "https://api.test.synapse.sh/plugins/?page=2",
+    "previous": null,
+    "results": [
+        {
+            "id": 1,
+            "name": "익스포트 기본 테스트",
+            "code": "test_export2",
+            "category": "export",
+            "category_display": "내보내기",
+            "latest_version": "0.3.0",
+            "is_public": true,
+            "is_active": true,
+            "maintainers": [...],
+            "created": "2025-06-10T10:51:52.814743+09:00"
+        }
+    ]
+}
+```
+
+**GET /plugins/{id}/** - Retrieve specific plugin details
+
+#### 4. Jobs API (`/jobs/`)
+
+**GET /jobs/** - List jobs with optional filtering
+
+Query parameters:
+- `agent`: Filter by agent ID
+
+Response includes job details with plugin_release reference.
+
+#### 5. Serve Applications API (`/serve_applications/`)
+
+**GET /serve_applications/** - List serve applications
+
+Query parameters:
+- `agent`: Filter by agent ID
+
+Response includes deployment details with plugin_release and agent references.
+
+### Data Enrichment Pattern
+
+When displaying plugin and agent information in the UI, follow this enrichment pattern:
+
+1. **Fetch primary data** (jobs, serve applications, etc.)
+2. **For each plugin_release ID**:
+   - Fetch `/plugin_releases/{id}/` to get version and plugin ID
+   - Fetch `/plugins/{plugin_id}/` to get plugin name and code
+3. **For each agent ID**:
+   - Fetch `/agents/{id}/` to get agent name and URL
+
+### Configuration Storage
+
+The SDK stores configuration in `~/.config/synapse/devtools.json`:
+
+```json
+{
+    "backend": {
+        "host": "https://api.test.synapse.sh",
+        "token": "syn_xxx"
+    },
+    "agent": {
+        "id": 1,
+        "name": "기본 에이전트",
+        "url": "http://10.0.22.1:8000",
+        "token": "xxx"
+    },
+    "plugin_storage": "s3://..."
+}
+```
+
+### Implementation Notes
+
+1. **API Response Handling**:
+   - Most endpoints return paginated responses with `count`, `next`, `previous`, and `results` fields
+   - Handle both direct array responses and paginated responses
+   - Korean text is common in responses (내보내기, 기본 에이전트, etc.)
+
+2. **Error Handling**:
+   - 404 responses return `{"detail": "찾을 수 없습니다."}` (Not found in Korean)
+   - Always wrap API calls in try-except blocks for resilience
+
+3. **Performance Optimization**:
+   - Cache frequently accessed data (plugins, agents) to reduce API calls
+   - Use batch fetching where possible
+   - Consider implementing local caching for plugin and agent metadata
+
+4. **Display Patterns**:
+   - Plugins: Show `{name} (v{version})` when available
+   - Agents: Show `{name} @ {url}` when available
+   - Always provide fallbacks when names aren't available

{synapse_sdk-1.0.0a98/synapse_sdk.egg-info → synapse_sdk-1.0.0b2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: synapse-sdk
-Version: 1.0.0a98
+Version: 1.0.0b2
 Summary: synapse sdk
 Author-email: datamaker <developer@datamaker.io>
 License: MIT
@@ -27,11 +27,9 @@ Requires-Dist: ray[all]==2.44.1; extra == "all"
 Requires-Dist: python-nmap; extra == "all"
 Requires-Dist: hyperopt; extra == "all"
 Requires-Dist: bayesian-optimization==1.4.3; extra == "all"
-Provides-Extra: dashboard
-Requires-Dist: fastapi>=0.115.13; extra == "dashboard"
-Requires-Dist: uvicorn[standard]>=0.34.3; extra == "dashboard"
-Requires-Dist: websockets>=15.0.1; extra == "dashboard"
-Requires-Dist: aiofiles>=24.1.0; extra == "dashboard"
+Provides-Extra: devtools
+Requires-Dist: streamlit>=1.29.0; extra == "devtools"
+Requires-Dist: streamlit-ace>=0.1.1; extra == "devtools"
 Provides-Extra: test
 Requires-Dist: pytest>=7.0.0; extra == "test"
 Requires-Dist: pytest-cov>=4.0.0; extra == "test"

{synapse_sdk-1.0.0a98 → synapse_sdk-1.0.0b2}/docs/api/clients/agent.md

@@ -10,7 +10,7 @@ Client for agent-specific operations and distributed execution.
 
 ## Overview
 
-The `AgentClient` provides access to agent operations, including plugin execution, job management, and Ray cluster integration.
+The `AgentClient` provides access to agent operations, including plugin execution, job management, and distributed computing integration.
 
 ## Constructor
 

synapse_sdk-1.0.0b2/docs/configuration.md

@@ -0,0 +1,83 @@
+---
+id: configuration
+title: Configuration
+sidebar_position: 8
+---
+
+# Configuration
+
+Configure Synapse SDK for your environment and use cases.
+
+## Configuration Methods
+
+There are three ways to configure Synapse SDK:
+
+1. **Synapse CLI**
+2. **Environment variables**
+3. **Configuration file**
+
+## CLI
+
+The interactive configuration menu provides an easy way to configure your Synapse SDK:
+
+```bash
+$ synapse config
+```
+
+This will open an interactive menu where you can:
+- Configure backend host and API token
+- Select or manually configure an agent
+- View current configuration
+
+### Key Features:
+- **Plain text token display**: Tokens are shown in plain text for easy verification
+- **Connection testing**: After configuring backend or agent, the connection is automatically tested
+- **No startup delays**: Connection checks only happen when you configure settings, not on CLI startup
+
+## Configuration File
+
+Create a configuration file at `~/.synapse/config.json`:
+
+```json
+{
+  "backend": {
+    "host": "https://api.synapse.sh",
+    "token": "your-api-token"
+  },
+  "agent": {
+    "id": "agent-uuid-123",
+    "name": "My Development Agent",
+    "token": "your-agent-token"
+  }
+}
+```
+
+## Plugin Management
+
+The CLI provides an interactive plugin management interface:
+
+```bash
+$ synapse
+# Select "Plugin Management"
+```
+
+### Available Options:
+
+1. **Create new plugin**: Creates a new plugin from templates using cookiecutter
+2. **Run plugin locally**: Interactive interface to run plugins with configurable parameters
+3. **Publish plugin**: Publishes plugins to the configured backend with debug mode option
+
+### Plugin Publishing
+
+When publishing plugins:
+- **Debug mode**: Enabled by default for development
+- **Backend integration**: Uses your configured backend settings
+- **Connection testing**: Verifies backend connection before publishing
+- **Error handling**: Stops on errors and waits for user acknowledgment
+
+### Plugin Development Workflow
+
+1. **Create**: Use the CLI to create a new plugin template
+2. **Develop**: Write your plugin code following the generated structure
+3. **Test**: Run plugins locally through the interactive interface
+4. **Publish**: Deploy to your configured backend with debug options

{synapse_sdk-1.0.0a98 → synapse_sdk-1.0.0b2}/docs/features/plugins/index.md

@@ -16,7 +16,7 @@ The plugin system is built around the concept of **actions** - discrete operatio
 
 - **🔌 Modular Architecture**: Plugins are self-contained with their own dependencies and configuration
 - **⚡ Multiple Execution Methods**: Support for Jobs, Tasks, and REST API endpoints
-- **📦 Distributed Execution**: Built on Ray for scalable, distributed computing
+- **📦 Distributed Execution**: Built for scalable, distributed computing
 - **🛠️ Template System**: Cookiecutter-based scaffolding for rapid plugin development
 - **📊 Progress Tracking**: Built-in logging, metrics, and progress monitoring
 - **🔄 Dynamic Loading**: Runtime plugin discovery and registration
@@ -122,7 +122,7 @@ Plugins support three different execution methods depending on the use case:
 
 ### Job Execution
 
-**Ray Job-based execution** for long-running, distributed processing tasks.
+**Job-based execution** for long-running, distributed processing tasks.
 
 - Best for: Training models, processing large datasets
 - Features: Distributed execution, resource management, fault tolerance
@@ -130,7 +130,7 @@ Plugins support three different execution methods depending on the use case:
 
 ### Task Execution
 
-**Ray Task-based execution** for simple, short-running operations.
+**Task-based execution** for simple, short-running operations.
 
 - Best for: Quick data processing, validation checks
 - Features: Lightweight execution, fast startup
@@ -138,7 +138,7 @@ Plugins support three different execution methods depending on the use case:
 
 ### REST API Execution
 
-**Ray Serve-based execution** for web API endpoints.
+**Serve-based execution** for web API endpoints.
 
 - Best for: Real-time inference, interactive applications
 - Features: HTTP endpoints, auto-scaling, load balancing
@@ -351,7 +351,7 @@ synapse plugin run serve --debug
 # Run plugin tests
 pytest plugin/test_*.py
 
-# Integration testing with Ray
+# Integration testing with distributed computing
 synapse plugin run action-name --debug --job-id test-job
 ```
 
@@ -361,7 +361,7 @@ synapse plugin run action-name --debug --job-id test-job
 # Package plugin
 synapse plugin publish
 
-# Deploy to Ray cluster
+# Deploy to cluster
 synapse plugin run action-name --job-id production-job
 ```
 
@@ -447,7 +447,7 @@ class TrainParams(BaseModel):
 ### 3. Testing
 
 - **Unit Tests**: Test individual action logic
-- **Integration Tests**: Test with Ray execution environment
+- **Integration Tests**: Test with distributed execution environment
 - **Parameter Validation**: Test edge cases and error conditions
 - **Performance Tests**: Validate execution time and resource usage
 

{synapse_sdk-1.0.0a98 → synapse_sdk-1.0.0b2}/docs/installation.md

@@ -29,7 +29,7 @@ pip install synapse-sdk
 For additional features, install with extras:
 
 ```bash
-# Install with all dependencies (Ray, optimization libraries)
+# Install with all dependencies (distributed computing, optimization libraries)
 pip install synapse-sdk[all]
 
 # Install with dashboard dependencies (FastAPI, Uvicorn)

{synapse_sdk-1.0.0a98 → synapse_sdk-1.0.0b2}/docs/introduction.md

@@ -15,7 +15,7 @@ Synapse SDK is..
 ### Key Features
 
 - **🚀 Plugin Development and Testing**: Develop modular, reusable components organized by categories for Synapse
-- **⚡ Distributed Computing**: Built on Ray for scalable, distributed execution
+- **⚡ Distributed Computing**: Built for scalable, distributed execution
 - **🔧 Multiple Execution Modes**: Support for Jobs, Tasks, and REST APIs
 - **📦 Isolated Runtime Environments**: Each plugin runs in its own environment with dependency management
 - **📊 Progress Tracking**: Built-in progress monitoring and metrics reporting

{synapse_sdk-1.0.0a98 → synapse_sdk-1.0.0b2}/docs/troubleshooting.md

@@ -19,7 +19,15 @@ Common issues and their solutions when using Synapse SDK.
 **Diagnosis Steps**:
 
 
-3. **Verify token in code**:
+3. **Verify token via CLI**:
+   ```bash
+   # Use the interactive config to check your token
+   synapse config
+   # Select "Show current configuration"
+   # Tokens are now displayed in plain text for easy verification
+   ```
+
+4. **Verify token in code**:
    ```python
    from synapse_sdk.clients.backend import BackendClient
    
@@ -48,7 +56,9 @@ Common issues and their solutions when using Synapse SDK.
 
 1. **Check agent configuration**:
    ```bash
-   synapse config show
+   synapse config
+   # Select "Show current configuration"
+   # Both backend and agent tokens are displayed in plain text
    ```
 
 2. **Verify agent token**:
@@ -162,7 +172,16 @@ python -c "import plugin; print('OK')"
 
 **Debugging Steps**:
 
-1. **Enable debug mode**:
+1. **Enable debug mode via CLI**:
+   ```bash
+   synapse
+   # Select "Plugin Management"
+   # Select "Run plugin locally"
+   # Enter your action and parameters
+   # Debug mode is enabled by default for local runs
+   ```
+
+2. **Enable debug mode manually**:
    ```bash
    export SYNAPSE_DEBUG=true
    synapse plugin run --path ./my-plugin --action test
@@ -236,78 +255,67 @@ python -c "import plugin; print('OK')"
                process_chunk(chunk)
    ```
 
-## Ray Integration Issues
+## Distributed Computing Issues
 
-### Ray Cluster Connection
+### Cluster Connection
 
-**Symptoms**: Cannot connect to Ray cluster.
+**Symptoms**: Cannot connect to compute cluster.
 
 **Solutions**:
 
-1. **Check Ray status**:
+1. **Check cluster status**:
    ```bash
-   ray status
+   synapse cluster status
    # Should show cluster information
    ```
 
-2. **Start local Ray cluster**:
+2. **Start local cluster**:
    ```bash
-   ray start --head --dashboard-host=0.0.0.0
+   synapse cluster start --dashboard-host=0.0.0.0
    ```
 
 3. **Connect to remote cluster**:
    ```bash
-   export RAY_ADDRESS="ray://remote-cluster:10001"
-   ray status  # Should connect to remote cluster
+   export SYNAPSE_CLUSTER_ADDRESS="cluster://remote-cluster:10001"
+   synapse cluster status  # Should connect to remote cluster
    ```
 
-### Ray Memory Issues
+### Memory Issues
 
-**Symptoms**: Out of memory errors in Ray.
+**Symptoms**: Out of memory errors during execution.
 
 **Solutions**:
 
-1. **Increase object store memory**:
+1. **Increase memory allocation**:
    ```bash
-   ray start --head --object-store-memory=2000000000  # 2GB
+   synapse cluster start --memory=2000000000  # 2GB
    ```
 
 2. **Configure in code**:
    ```python
-   import ray
-   ray.init(object_store_memory=2000000000)
+   from synapse_sdk.compute import init
+   init(memory=2000000000)
    ```
 
 3. **Process data in smaller chunks**:
    ```python
-   @ray.remote
    def process_chunk(data_chunk):
        return process(data_chunk)
    
    # Split large data into chunks
    chunks = split_data(large_data)
-   results = ray.get([process_chunk.remote(chunk) for chunk in chunks])
+   results = [process_chunk(chunk) for chunk in chunks]
    ```
 
-### Ray Job Failures
+### Job Failures
 
-**Symptoms**: Ray jobs failing to start or complete.
-
-**Diagnosis**:
-```bash
-# Check Ray logs
-ray logs --follow
-
-# Check specific job
-ray job logs <job-id>
-```
+**Symptoms**: Jobs failing to start or complete.
 
 **Solutions**:
 
 1. **Check resource requirements**:
    ```python
-   @ray.remote(num_cpus=2, memory=1000*1024*1024)  # 1GB
-   def my_task():
+   def my_task(resources={'cpus': 2, 'memory': '1GB'}):
        pass
    ```
 
@@ -391,18 +399,17 @@ ray job logs <job-id>
        self.run.log(f"Execution took {time.time() - start_time:.2f}s")
    ```
 
-2. **Use Ray for parallelization**:
+2. **Use parallel processing**:
    ```python
-   import ray
+   from concurrent.futures import ThreadPoolExecutor
    
-   @ray.remote
    def parallel_task(item):
        return process_item(item)
    
    def start(self):
        # Process items in parallel
-       futures = [parallel_task.remote(item) for item in items]
-       results = ray.get(futures)
+       with ThreadPoolExecutor() as executor:
+           results = list(executor.map(parallel_task, items))
    ```
 
 3. **Optimize data loading**:

{synapse_sdk-1.0.0a98 → synapse_sdk-1.0.0b2}/pyproject.toml

@@ -37,11 +37,10 @@ all = [
     "hyperopt",
     "bayesian-optimization==1.4.3"
 ]
-dashboard = [
-    "fastapi>=0.115.13",
-    "uvicorn[standard]>=0.34.3",
-    "websockets>=15.0.1",
-    "aiofiles>=24.1.0"
+
+devtools = [
+    "streamlit>=1.29.0",
+    "streamlit-ace>=0.1.1"
 ]
 test = [
     "pytest>=7.0.0",