opencue-cuecmd 1.13.6__tar.gz

opencue_cuecmd-1.13.6/.gitignore

@@ -0,0 +1,20 @@
+*.py[cod]
+__pycache__/
+*.so
+*.egg
+*.egg-info/
+dist/
+build/
+.eggs/
+*.log
+.pytest_cache/
+.coverage
+htmlcov/
+.tox/
+*.swp
+*.swo
+*~
+.DS_Store
+.vscode/
+.idea/
+*.iml

opencue_cuecmd-1.13.6/PKG-INFO

@@ -0,0 +1,403 @@
+Metadata-Version: 2.1
+Name: opencue_cuecmd
+Version: 1.13.6
+Summary: Cuecmd is a command-line tool for OpenCue that executes a list of commands as frames on the render farm. It enables batch processing by chunking commands and submitting them as OpenCue jobs.
+Requires-Python: >3.7
+Requires-Dist: opencue-pycue
+Requires-Dist: opencue-pyoutline
+Provides-Extra: dev
+Requires-Dist: black; extra == 'dev'
+Requires-Dist: isort; extra == 'dev'
+Requires-Dist: mock==2.0.0; extra == 'dev'
+Requires-Dist: pyfakefs==5.2.3; extra == 'dev'
+Requires-Dist: pylint; extra == 'dev'
+Requires-Dist: pytest; extra == 'dev'
+Requires-Dist: pytest-cov; extra == 'dev'
+Requires-Dist: pytest-mock; extra == 'dev'
+Provides-Extra: test
+Requires-Dist: mock==2.0.0; extra == 'test'
+Requires-Dist: pyfakefs==5.2.3; extra == 'test'
+Requires-Dist: pytest; extra == 'test'
+Requires-Dist: pytest-cov; extra == 'test'
+Requires-Dist: pytest-mock; extra == 'test'
+Description-Content-Type: text/markdown
+
+# OpenCue Cuecmd
+
+Cuecmd is a command-line tool for OpenCue that executes a list of commands as frames on the render farm. It enables batch processing by chunking commands and submitting them as OpenCue jobs.
+
+## Table of Contents
+
+1. [Overview](#overview)
+2. [Features](#features)
+3. [Installation](#installation)
+4. [Quick Start](#quick-start)
+5. [Usage](#usage)
+6. [Options](#options)
+7. [Examples](#examples)
+8. [Environment Variables](#environment-variables)
+9. [How It Works](#how-it-works)
+10. [Running Tests](#running-tests)
+11. [Docker Support](#docker-support)
+12. [Contributing](#contributing)
+
+## Overview
+
+`cuecmd` is a tool for executing arbitrary shell commands on the OpenCue render farm. It takes a text file containing a list of commands (one per line) and distributes them across multiple frames for parallel execution.
+
+### About Cuecmd
+
+The name "cuecmd" combines:
+- **"Cue"** - From OpenCue, the render farm management system
+- **"cmd"** - Short for command, representing its purpose of executing commands
+
+The `cuecmd` provides a simple way to parallelize any batch of shell commands using the OpenCue infrastructure.
+
+## Features
+
+- **Command batching**: Execute any list of shell commands on the render farm
+- **Chunking**: Group multiple commands per frame for efficient resource usage
+- **Resource control**: Specify cores and memory requirements per frame
+- **Job management**: Launch jobs paused or test with pretend mode
+- **Flexible configuration**: Override show, shot, and user from command line
+- **Simple input format**: Plain text file with one command per line
+
+## Installation
+
+### Prerequisites
+- OpenCue server running and accessible
+- Python 3.7 or higher
+- OpenCue Python packages installed
+
+### Install Methods
+
+Install as part of the OpenCue Python packages:
+
+```bash
+pip install opencue-cuecmd
+```
+
+Or install from source:
+
+```bash
+cd OpenCue/cuecmd
+pip install .
+```
+
+### Configuration
+Set up your environment variables:
+```bash
+export OPENCUE_HOSTS="your-cuebot-server:8443"
+export OPENCUE_FACILITY="your-facility-code"
+```
+
+## Quick Start
+
+1. **Create a command file:**
+   ```bash
+   cat > commands.txt << EOF
+   echo "Processing file 1"
+   echo "Processing file 2"
+   echo "Processing file 3"
+   EOF
+   ```
+
+2. **Submit to OpenCue:**
+   ```bash
+   cuecmd commands.txt
+   ```
+
+3. **Monitor the job:**
+   ```bash
+   cueadmin -lj  # List jobs
+   cueman -lf <job_name>  # List frames
+   ```
+
+## Usage
+
+```bash
+cuecmd [options] <command_file>
+```
+
+### Basic Usage
+
+```bash
+# Submit a simple command list
+cuecmd commands.txt
+
+# Chunk 5 commands per frame
+cuecmd commands.txt --chunk 5
+
+# Specify resources
+cuecmd commands.txt --cores 4 --memory 8
+
+# Launch paused for review
+cuecmd commands.txt --pause
+
+# Test without submitting
+cuecmd commands.txt --pretend
+```
+
+## Options
+
+### Required Arguments
+- `command_file` - A text file with a list of commands to run (one per line)
+
+### Optional Arguments
+
+#### Execution Options
+- `-c, --chunk <N>` - Number of commands to chunk per frame (Default: 1)
+- `--cores <N>` - Number of cores required per frame (Default: 1.0)
+- `--memory <N>` - Amount of RAM in GB required per frame (Default: 1.0)
+
+#### Job Control
+- `-p, --pause` - Launch the job in the paused state
+- `--pretend` - Generate the outline and print info without submitting
+
+#### Job Metadata
+- `--show <name>` - Show name for the job (Default: from $SHOW or "default")
+- `-s, --shot <name>` - Shot name for the job (Default: from $SHOT or "default")
+- `--user <name>` - User name for the job (Default: from $USER or "unknown")
+- `--job-name <name>` - Custom job name (Default: auto-generated)
+
+## Examples
+
+### Example 1: Simple Command Execution
+```bash
+# Create command file
+cat > render_frames.txt << EOF
+blender -b scene.blend -o /tmp/frame_#### -f 1
+blender -b scene.blend -o /tmp/frame_#### -f 2
+blender -b scene.blend -o /tmp/frame_#### -f 3
+EOF
+
+# Submit to OpenCue
+cuecmd render_frames.txt
+```
+
+### Example 2: Chunked Execution
+```bash
+# Process 1000 images, 10 per frame
+for i in {1..1000}; do
+  echo "convert input_$i.jpg -resize 50% output_$i.jpg"
+done > convert_images.txt
+
+cuecmd convert_images.txt --chunk 10
+# This creates 100 frames, each processing 10 images
+```
+
+### Example 3: Resource-Intensive Jobs
+```bash
+# Heavy computation requiring more resources
+cat > simulations.txt << EOF
+python simulate.py --scene 1
+python simulate.py --scene 2
+python simulate.py --scene 3
+EOF
+
+cuecmd simulations.txt --cores 8 --memory 32
+```
+
+### Example 4: Testing Before Launch
+```bash
+# Test the job configuration without submitting
+cuecmd commands.txt --chunk 5 --cores 2 --memory 4 --pretend
+
+# Output shows:
+# - Job name
+# - Frame range
+# - Resource requirements
+# - Command file location
+```
+
+### Example 5: Custom Job Configuration
+```bash
+cuecmd commands.txt \
+  --show "my_project" \
+  --shot "sequence_010" \
+  --user "artist" \
+  --job-name "batch_processing" \
+  --chunk 10 \
+  --cores 4 \
+  --memory 8 \
+  --pause
+```
+
+### Example 6: Data Processing Pipeline
+```bash
+# Generate processing commands
+cat > data_pipeline.txt << EOF
+python extract.py data1.csv
+python extract.py data2.csv
+python extract.py data3.csv
+python transform.py data1.json
+python transform.py data2.json
+python transform.py data3.json
+EOF
+
+cuecmd data_pipeline.txt --chunk 2 --cores 2 --memory 4
+```
+
+## Environment Variables
+
+Cuecmd respects the following environment variables:
+
+- `OPENCUE_HOSTS` - Comma-separated list of OpenCue servers
+- `OPENCUE_FACILITY` - Default facility code
+- `SHOW` - Default show name
+- `SHOT` - Default shot name
+- `USER` - Default user name
+
+These can be overridden using command-line arguments.
+
+## How It Works
+
+1. **Command File Reading**: Cuecmd reads your text file containing commands
+2. **Chunking**: Commands are divided into chunks based on the `--chunk` parameter
+3. **Frame Range Calculation**: A frame range is calculated (1 to ceil(commands/chunk))
+4. **Outline Creation**: An OpenCue outline is created with a Shell layer
+5. **Job Submission**: The job is submitted to OpenCue for execution
+6. **Frame Execution**: Each frame executes its assigned chunk of commands
+
+### Technical Details
+
+- Commands are copied to a temporary location for access during execution
+- Each frame executes a helper script (`execute_commands.py`) that:
+  - Reads the command file
+  - Calculates which commands to run based on frame number and chunk size
+  - Executes each command in sequence
+  - Reports success/failure
+
+## Running Tests
+
+The `cuecmd` includes a comprehensive test suite.
+
+### Quick Start
+
+```bash
+# Install with test dependencies
+pip install -e ".[test]"
+
+# Run all tests
+pytest
+
+# Run with coverage
+pytest --cov=cuecmd --cov-report=term-missing
+```
+
+### Test Types
+
+- **Unit tests** - Test individual functions (`tests/test_main.py`, `tests/test_execute_commands.py`)
+- **Integration tests** - Test complete workflows (`tests/integration_tests.py`, `tests/test_integration_workflows.py`)
+
+### Running Tests
+
+```bash
+# Basic test run
+pytest tests/
+
+# Verbose output
+pytest -v
+
+# Run specific test file
+pytest tests/test_main.py
+
+# Run with coverage and HTML report
+pytest --cov=cuecmd --cov-report=html --cov-report=term-missing
+```
+
+### CI/CD Integration
+
+Cuecmd is integrated into OpenCue's CI/CD pipeline:
+
+```bash
+# In OpenCue root directory
+./ci/run_python_tests.sh     # Includes cuecmd tests
+./ci/run_python_lint.sh      # Includes cuecmd linting
+```
+
+## Docker Support
+
+### Building the Docker Image
+
+From the OpenCue root directory:
+```bash
+docker build -f cuecmd/Dockerfile -t cuecmd .
+```
+
+### Running the Docker Container
+
+**Show help:**
+```bash
+docker run --rm cuecmd cuecmd --help
+```
+
+**Test with commands:**
+```bash
+# Create test commands
+echo "echo 'test 1'" > /tmp/commands.txt
+echo "echo 'test 2'" >> /tmp/commands.txt
+
+# Run in pretend mode
+docker run --rm -v /tmp:/data cuecmd cuecmd /data/commands.txt --pretend
+```
+
+## Contributing
+
+### Development Setup
+
+```bash
+# Clone and setup
+git clone https://github.com/<username>/OpenCue.git
+cd OpenCue/cuecmd
+
+# Install with development dependencies
+pip install -e ".[dev]"
+```
+
+### Testing and Quality
+
+```bash
+# Run tests
+pytest --cov=cuecmd --cov-report=term-missing
+
+# Code formatting and linting
+black cuecmd tests
+isort cuecmd tests
+pylint cuecmd tests
+```
+
+### Guidelines
+
+- Add tests for new features
+- Update documentation for API changes
+- Ensure all tests pass before submitting PRs
+
+## Troubleshooting
+
+### Common Issues
+
+**Command file not found:**
+```bash
+# Make sure file exists and path is correct
+ls -l commands.txt
+cuecmd $(pwd)/commands.txt  # Use absolute path
+```
+
+**No commands found:**
+```bash
+# Check file has content and is not all blank lines
+cat commands.txt
+```
+
+**Job fails to launch:**
+```bash
+# Check OpenCue connection
+echo $OPENCUE_HOSTS
+cueadmin -lj  # Test connection
+```
+
+**Commands fail during execution:**
+- Check command syntax is correct
+- Verify commands have access to required files/paths
+- Ensure sufficient resources (cores/memory) are allocated