synapse-sdk 1.0.0b12__py3-none-any.whl → 1.0.0b14__py3-none-any.whl

synapse_sdk/devtools/docs/docs/examples/index.md

@@ -0,0 +1,29 @@
+---
+id: index
+title: Examples
+sidebar_position: 1
+---
+
+# Examples
+
+Practical examples and tutorials for using Synapse SDK.
+
+## Getting Started Examples
+
+Basic examples to help you get started quickly.
+
+## Plugin Examples
+
+Real-world plugin examples for different use cases.
+
+## Integration Examples
+
+Examples of integrating Synapse SDK with other tools and services.
+
+## Advanced Examples
+
+Complex workflows and advanced usage patterns.
+
+## Code Samples
+
+Ready-to-use code snippets for common operations.

synapse_sdk/devtools/docs/docs/faq.md

@@ -0,0 +1,179 @@
+---
+id: faq
+title: Frequently Asked Questions
+sidebar_position: 9
+---
+
+# Frequently Asked Questions
+
+Common questions and answers about Synapse SDK.
+
+## Installation & Setup
+
+### Q: What Python versions are supported?
+
+Synapse SDK requires **Python 3.10 or higher**.
+
+```bash
+pip install "synapse-sdk[all,devtools]"
+```
+
+### Q: How do I install code-server for local development?
+
+You have several options:
+
+```bash
+# Recommended: Install script
+curl -fsSL https://code-server.dev/install.sh | sh
+
+# Using npm
+npm install -g code-server
+
+# Using yarn
+yarn global add code-server
+```
+
+For more installation methods, visit: [code-server installation guide](https://coder.com/docs/code-server/latest/install)
+
+## CLI Usage
+
+### Q: How do I start the Synapse CLI?
+
+Simply run:
+
+```bash
+synapse
+```
+
+This opens the interactive menu where you can access all Synapse features.
+
+### Q: What's the difference between agent and local code-server?
+
+- **Agent Code-Server**: Runs on a remote agent with your project files synced. Includes plugin encryption and secure transfer.
+- **Local Code-Server**: Runs on your local machine. Faster startup, uses your local environment and settings.
+
+### Q: How do I configure the code-server port?
+
+Code-server port is automatically detected from `~/.config/code-server/config.yaml`. If no config exists, it defaults to port 8070.
+
+Example config:
+
+```yaml
+bind-addr: 127.0.0.1:8070
+auth: password
+password: your-password-here
+cert: false
+```
+
+### Q: Why does the agent workspace path differ from my local path?
+
+Agents run in containerized environments where your local project is mounted to `/home/coder/workspace`. This is normal and ensures consistent paths across different development environments.
+
+## Code-Server Troubleshooting
+
+### Q: Code-server shows "not available" error
+
+This usually means:
+
+1. The agent doesn't have code-server support enabled
+2. Network connectivity issues
+3. Agent is not properly configured
+
+**Solution**: Reinstall the agent with code-server support, or check agent configuration.
+
+### Q: Browser doesn't open automatically
+
+This happens in headless environments or when display is not available.
+
+**Solution**: Manually copy the provided URL (including the `?folder=` parameter) to your browser.
+
+### Q: Plugin not detected in workspace
+
+**Solution**: Ensure your directory contains a valid `config.yaml` file with plugin metadata:
+
+```yaml
+name: my-plugin
+version: 1.0.0
+description: My awesome plugin
+entry_point: main.py
+```
+
+### Q: How does plugin encryption work?
+
+When opening code-server with an agent, Synapse:
+
+1. Detects if your workspace contains a plugin
+2. Creates a ZIP archive of the plugin files
+3. Encrypts the archive using AES-256 encryption
+4. Securely transfers it to the agent
+5. Decrypts and extracts it in the agent workspace
+
+This ensures your plugin code is protected during transfer.
+
+## Configuration
+
+### Q: Where are configuration files stored?
+
+- **Synapse Config**: `~/.synapse/devtools.yaml`
+- **Code-Server Config**: `~/.config/code-server/config.yaml`
+
+### Q: How do I reset my configuration?
+
+```bash
+# Remove configuration files
+rm ~/.synapse/devtools.yaml
+rm ~/.config/code-server/config.yaml
+
+# Run configuration wizard
+synapse config
+```
+
+### Q: What if I get "Invalid token (401)" error?
+
+This means your API token is expired or invalid.
+
+**Solution**:
+
+1. Generate a new token from your Synapse backend
+2. Run `synapse config` to update the token
+3. Test connection with `synapse --dev-tools`
+
+## Plugin Development
+
+### Q: How do I create a new plugin?
+
+Use the interactive plugin creator:
+
+```bash
+synapse
+# Select "🔌 Plugin Management" → "Create new plugin"
+```
+
+This creates a complete plugin structure with examples and documentation.
+
+### Q: How do I test plugins locally?
+
+```bash
+# Test with local script execution
+synapse plugin run my_action '{"param": "value"}' --run-by script
+
+# Test with agent execution
+synapse plugin run my_action '{"param": "value"}' --run-by agent
+```
+
+Always test locally before publishing to ensure your plugin works correctly.
+
+### Q: Plugin publishing fails with errors
+
+Common issues:
+
+1. **Missing dependencies**: Ensure `requirements.txt` includes all needed packages
+2. **Syntax errors**: Test locally first with `--run-by script`
+3. **Configuration errors**: Check `config.yaml` format and required fields
+4. **Backend connectivity**: Ensure backend is accessible and token is valid
+
+**Solution**: Use debug mode for detailed error information:
+
+```bash
+synapse plugin publish --debug
+```

synapse_sdk/devtools/docs/docs/features/converters/index.md

@@ -0,0 +1,455 @@
+---
+id: converters
+title: Converters
+sidebar_position: 1
+---
+
+# Converters
+
+The Synapse SDK provides comprehensive data format conversion utilities for computer vision datasets. These converters enable seamless transformation between different annotation formats commonly used in machine learning workflows.
+
+## Overview
+
+The converter system supports bidirectional conversion between:
+
+- **DM Format** - Synapse Data Manager's native annotation format (supports v1 ⟷ v2 migration)
+- **COCO Format** - Microsoft Common Objects in Context format  
+- **Pascal VOC Format** - Visual Object Classes XML format
+- **YOLO Format** - You Only Look Once text-based format
+
+All converters support both categorized datasets (with train/valid/test splits) and non-categorized datasets. Additionally, all converters now support single file conversion mode for processing individual files.
+
+## Supported Annotation Types
+
+| Annotation Type | DM | COCO | Pascal VOC | YOLO |
+|----------------|----|----|-----------|------|
+| Bounding Boxes | ✅ | ✅ | ✅ | ✅ |
+| Polygons | ✅ | ✅ | ❌ | ✅ |
+| Segmentation | ✅ | ✅ | ✅ | ❌ |
+| Keypoints | ✅ | ✅ | ❌ | ✅ |
+| Classifications | ✅ | ✅ | ❌ | ❌ |
+
+## Pascal VOC Converters
+
+### FromDMToPascalConverter
+
+Converts DM format annotations to Pascal VOC XML format.
+
+**Features:**
+- Converts bounding box annotations and segmentation masks
+- Creates standard Pascal VOC directory structure
+- Generates `classes.txt` file automatically
+- Supports both categorized and non-categorized datasets
+
+**Usage:**
+```python
+from synapse_sdk.utils.converters.pascal.from_dm import FromDMToPascalConverter
+
+# Convert categorized dataset
+converter = FromDMToPascalConverter(
+    root_dir='/path/to/dm/dataset',
+    is_categorized_dataset=True
+)
+converted_data = converter.convert()
+converter.save_to_folder('/output/pascal/dataset')
+
+# Convert non-categorized dataset
+converter = FromDMToPascalConverter(
+    root_dir='/path/to/dm/dataset',
+    is_categorized_dataset=False
+)
+converted_data = converter.convert()
+converter.save_to_folder('/output/pascal/dataset')
+
+# Single file conversion
+converter = FromDMToPascalConverter(is_single_conversion=True)
+with open('data.json') as f:
+    dm_data = json.load(f)
+with open('image.jpg', 'rb') as img_file:
+    pascal_xml = converter.convert_single_file(dm_data, img_file)
+```
+
+**Input Structure (Categorized):**
+```
+dm_dataset/
+├── train/
+│   ├── json/
+│   │   ├── image1.json
+│   │   └── image2.json
+│   └── original_files/
+│       ├── image1.jpg
+│       └── image2.jpg
+├── valid/
+│   ├── json/
+│   └── original_files/
+└── test/ (optional)
+    ├── json/
+    └── original_files/
+```
+
+**Output Structure:**
+```
+pascal_dataset/
+├── train/
+│   ├── Annotations/
+│   │   ├── image1.xml
+│   │   └── image2.xml
+│   └── Images/
+│       ├── image1.jpg
+│       └── image2.jpg
+├── valid/
+├── test/ (if present)
+└── classes.txt
+```
+
+### PascalToDMConverter
+
+Converts Pascal VOC XML annotations to DM format.
+
+**Features:**
+- Parses Pascal VOC XML files
+- Flexible directory naming (supports Annotations/annotations, Images/images/JPEGImages)
+- Extracts bounding box annotations and segmentation information
+- Maintains class information
+
+**Usage:**
+```python
+from synapse_sdk.utils.converters.pascal.to_dm import PascalToDMConverter
+
+# Convert Pascal VOC dataset
+converter = PascalToDMConverter(
+    root_dir='/path/to/pascal/dataset',
+    is_categorized_dataset=True
+)
+converted_data = converter.convert()
+converter.save_to_folder('/output/dm/dataset')
+```
+
+## YOLO Converters
+
+### FromDMToYOLOConverter
+
+Converts DM format annotations to YOLO format with comprehensive annotation support.
+
+**Features:**
+- Supports bounding boxes, polygons, and keypoints
+- Creates `dataset.yaml` configuration file
+- Normalizes coordinates automatically
+- Handles keypoint visibility flags
+
+**Usage:**
+```python
+from synapse_sdk.utils.converters.yolo.from_dm import FromDMToYOLOConverter
+
+# Convert with all annotation types
+converter = FromDMToYOLOConverter(
+    root_dir='/path/to/dm/dataset',
+    is_categorized_dataset=True
+)
+converted_data = converter.convert()
+converter.save_to_folder('/output/yolo/dataset')
+
+# Single file conversion
+converter = FromDMToYOLOConverter(is_single_conversion=True)
+with open('data.json') as f:
+    dm_data = json.load(f)
+with open('image.jpg', 'rb') as img_file:
+    yolo_labels = converter.convert_single_file(dm_data, img_file)
+```
+
+**Output Structure:**
+```
+yolo_dataset/
+├── train/
+│   ├── images/
+│   │   ├── image1.jpg
+│   │   └── image2.jpg
+│   └── labels/
+│       ├── image1.txt
+│       └── image2.txt
+├── valid/
+├── test/ (if present)
+├── dataset.yaml
+└── classes.txt
+```
+
+**YOLO Label Format Examples:**
+```
+# Bounding box: class_id center_x center_y width height
+0 0.5 0.5 0.3 0.4
+
+# Polygon: class_id x1 y1 x2 y2 x3 y3 x4 y4 ...
+0 0.1 0.1 0.9 0.1 0.9 0.9 0.1 0.9
+
+# Keypoints: class_id center_x center_y width height kp1_x kp1_y kp1_v kp2_x kp2_y kp2_v ...
+0 0.5 0.5 0.3 0.4 0.45 0.3 2 0.55 0.3 2 0.5 0.7 1
+```
+
+### YOLOToDMConverter
+
+Converts YOLO format annotations back to DM format.
+
+**Features:**
+- Intelligent parsing of different YOLO annotation types
+- Requires `dataset.yaml` for class name mapping
+- Handles bounding boxes, polygons, and keypoints
+- Automatically detects image dimensions
+
+**Usage:**
+```python
+from synapse_sdk.utils.converters.yolo.to_dm import YOLOToDMConverter
+
+converter = YOLOToDMConverter(
+    root_dir='/path/to/yolo/dataset',
+    is_categorized_dataset=True
+)
+converted_data = converter.convert()
+converter.save_to_folder('/output/dm/dataset')
+```
+
+## COCO Converters
+
+### FromDMToCOCOConverter
+
+Converts DM format to COCO format with full metadata support.
+
+**Features:**
+- Comprehensive COCO metadata (info, licenses, categories)
+- Supports bounding boxes, polygons, segmentation, and keypoints
+- Dynamic category management
+- Extensible for different data types
+
+**Usage:**
+```python
+from synapse_sdk.utils.converters.coco.from_dm import FromDMToCOCOConverter
+
+# Basic conversion
+converter = FromDMToCOCOConverter(
+    root_dir='/path/to/dm/dataset',
+    is_categorized_dataset=True
+)
+converted_data = converter.convert()
+converter.save_to_folder('/output/coco/dataset')
+
+# With custom metadata
+info_dict = {
+    "description": "My Custom Dataset",
+    "version": "1.0",
+    "contributor": "My Organization"
+}
+
+licenses_list = [{
+    "id": 1,
+    "name": "Custom License",
+    "url": "https://example.com/license"
+}]
+
+converter = FromDMToCOCOConverter(
+    root_dir='/path/to/dm/dataset',
+    info_dict=info_dict,
+    licenses_list=licenses_list,
+    is_categorized_dataset=True
+)
+
+# Single file conversion
+converter = FromDMToCOCOConverter(
+    data_type='img',
+    is_single_conversion=True
+)
+with open('data.json') as f:
+    dm_data = json.load(f)
+with open('image.jpg', 'rb') as img_file:
+    coco_annotation = converter.convert_single_file(dm_data, img_file)
+```
+
+**Output Structure:**
+```
+coco_dataset/
+├── train/
+│   ├── annotations.json
+│   ├── image1.jpg
+│   └── image2.jpg
+├── valid/
+│   ├── annotations.json
+│   └── images...
+└── test/ (if present)
+```
+
+### COCOToDMConverter
+
+Converts COCO format annotations to DM format.
+
+**Features:**
+- Parses COCO JSON annotations
+- Handles image datasets
+- Maintains keypoint groupings through DM groups
+- Supports bounding boxes and keypoints
+
+**Usage:**
+```python
+from synapse_sdk.utils.converters.coco.to_dm import COCOToDMConverter
+
+converter = COCOToDMConverter(
+    root_dir='/path/to/coco/dataset',
+    is_categorized_dataset=True
+)
+converted_data = converter.convert()
+converter.save_to_folder('/output/dm/dataset')
+```
+
+## DM Version Converter
+
+### DMV1ToV2Converter
+
+Migrates legacy DM v1 datasets to the current v2 format.
+
+**Features:**
+- Comprehensive migration for all annotation types
+- Handles data structure changes between versions
+- Supports images and videos
+- Maintains annotation tool integrity
+
+**Supported Tools:**
+- Bounding boxes
+- Named entities
+- Classifications with attributes
+- Polylines and polygons
+- Keypoints
+- 3D bounding boxes
+- Segmentation
+- Relations and groups
+
+**Usage:**
+```python
+from synapse_sdk.utils.converters.dm.from_v1 import DMV1ToV2Converter
+
+converter = DMV1ToV2Converter(
+    root_dir='/path/to/dm/v1/dataset',
+    is_categorized_dataset=True
+)
+converted_data = converter.convert()
+converter.save_to_folder('/output/dm/v2/dataset')
+```
+
+### DMV2ToV1Converter
+
+Converts DM v2 datasets back to the legacy v1 format for compatibility with older systems.
+
+**Features:**
+- Reverse migration from DM v2 to v1 format
+- Preserves all annotation types and metadata
+- Maintains coordinate integrity across formats
+- Generates appropriate v1 structure with annotations and annotationsData
+
+**Supported Tools:**
+- Bounding boxes
+- Named entities  
+- Classifications with attributes
+- Polylines and polygons
+- Keypoints
+- 3D bounding boxes
+- Segmentation
+- Relations and groups
+
+**Usage:**
+```python
+from synapse_sdk.utils.converters.dm.to_v1 import DMV2ToV1Converter
+
+# Load v2 data and convert to v1
+with open('dm_v2_data.json', 'r') as f:
+    v2_data = json.load(f)
+
+converter = DMV2ToV1Converter(v2_data)
+v1_data = converter.convert()
+
+# Save or use the converted v1 data
+with open('dm_v1_data.json', 'w') as f:
+    json.dump(v1_data, f, indent=2)
+```
+
+## Common Parameters
+
+All converters share these common parameters:
+
+### `root_dir` (str)
+Path to the root directory containing the dataset. Not required when using single file conversion mode.
+
+### `is_categorized_dataset` (bool)
+- `True`: Dataset has train/valid/test splits in separate subdirectories
+- `False`: Dataset is in a single directory without splits
+
+### `is_single_conversion` (bool)
+- `True`: Enable single file conversion mode for processing individual files
+- `False`: Process entire dataset directories (default behavior)
+
+### Common Methods
+
+#### `convert()`
+Performs in-memory conversion and returns the converted data structure.
+
+#### `convert_single_file(data, original_file, **kwargs)`
+Available when `is_single_conversion=True`. Converts a single data object and corresponding original file.
+
+#### `save_to_folder(output_dir)`
+Saves the converted data to the specified output directory, creating the appropriate file structure for the target format.
+
+## Error Handling
+
+All converters include robust error handling:
+
+- **File Validation**: Checks for required files and directories
+- **Format Validation**: Validates annotation format correctness
+- **Graceful Degradation**: Warns about unsupported annotations instead of failing
+- **Progress Tracking**: Shows progress for large dataset conversions
+
+## Best Practices
+
+1. **Backup Original Data**: Always keep backups before conversion
+2. **Validate Results**: Check converted annotations for accuracy
+3. **Test on Small Datasets**: Test conversion on small samples first
+4. **Check Requirements**: Ensure all required files (dataset.yaml, annotations.json, etc.) are present
+5. **Monitor Warnings**: Pay attention to conversion warnings for data quality issues
+
+## Examples
+
+### Converting a Complete Workflow
+
+```python
+# 1. Convert Pascal VOC to DM
+pascal_converter = PascalToDMConverter('/data/pascal', True)
+dm_data = pascal_converter.convert()
+pascal_converter.save_to_folder('/data/dm_intermediate')
+
+# 2. Convert DM to YOLO
+yolo_converter = FromDMToYOLOConverter('/data/dm_intermediate', True)
+yolo_data = yolo_converter.convert()
+yolo_converter.save_to_folder('/data/yolo_final')
+
+# 3. Convert DM to COCO for evaluation
+coco_converter = FromDMToCOCOConverter('/data/dm_intermediate', True)
+coco_data = coco_converter.convert()
+coco_converter.save_to_folder('/data/coco_eval')
+```
+
+### Single File Conversion Example
+
+```python
+import json
+from synapse_sdk.utils.converters.yolo.from_dm import FromDMToYOLOConverter
+
+# Initialize converter for single file processing
+converter = FromDMToYOLOConverter(is_single_conversion=True)
+
+# Load DM format data
+with open('annotation.json', 'r') as f:
+    dm_data = json.load(f)
+
+# Convert single file
+with open('image.jpg', 'rb') as img_file:
+    yolo_labels = converter.convert_single_file(dm_data, img_file)
+
+# yolo_labels contains the converted YOLO format labels
+print(yolo_labels)
+```
+
+This converter system provides a complete solution for dataset format transformation, enabling seamless integration between different machine learning workflows and annotation tools.

synapse_sdk/devtools/docs/docs/features/index.md

@@ -0,0 +1,24 @@
+---
+id: features
+title: Features
+sidebar_position: 5
+---
+
+# Features
+
+This section covers the key features and functionality provided by the Synapse SDK.
+
+## [Plugin System](./plugins/index.md)
+
+Comprehensive plugin framework for building and managing ML workflows.
+
+- **[Plugin Categories](./plugins/index.md#plugin-categories)** - Neural networks, export, upload, smart tools, and validation plugins
+- **[Execution Methods](./plugins/index.md#execution-methods)** - Job, Task, and REST API execution modes
+- **[Development Guide](./plugins/index.md#creating-plugins)** - Create, test, and deploy custom plugins
+
+## [Data Converters](./converters/index.md)
+
+Comprehensive data format conversion utilities for computer vision datasets.
+
+- **[Format Converters](./converters/index.md)** - Convert between DM, COCO, Pascal VOC, and YOLO formats
+- **[Version Migration](./converters/index.md#dm-version-converter)** - Migrate DM datasets between versions