agent-portal-2 0.1.0

package/README.md

@@ -0,0 +1,385 @@
+<p align="center">
+  <img src="assets/branding/agent-portal-logo.png" alt="Agent Portal Logo" width="420" />
+</p>
+
+# Agent Portal
+
+[![npm](https://img.shields.io/npm/v/agent-portal-2)](https://www.npmjs.com/package/agent-portal-2)
+
+Agent Portal is a desktop-native operating environment for AI agents.
+
+Instead of limiting an LLM to code generation, Agent Portal gives it a controlled visual workspace where it can understand, navigate, test, and interact with real user interfaces, local applications, browser sessions, development environments, and project ecosystems.
+
+## Vision
+
+Give AI agents eyes, hands, memory, context, and permissions.
+
+That means combining:
+
+- visual understanding
+- browser and desktop control
+- long-lived workspace memory
+- multi-agent orchestration
+- test and reporting workflows
+- secure execution boundaries
+
+## What Exists Today
+
+The project now has a real local runtime foundation rather than just a concept scaffold. The current repository includes:
+
+- a Python-first local runtime in `python/agent_portal`
+- Playwright-backed browser control for open, click, type, hover, scroll, wait, inspect, screenshot, execute, and text reading
+- a local HTTP runtime server for health, status, control, browser, and report routes
+- an agent steering and policy layer with pause, resume, stop, queue approval, blocked actions, and risk-aware behavior
+- report generation with runtime state, actions, risk events, screenshots, and reproduction steps
+- plugin manifest discovery and validation
+- a VS Code extension control center that talks directly to the Python runtime
+- a TypeScript SDK surface that is being shifted into a client of the Python runtime
+- connector scaffolds for ChatGPT-style tools, Claude MCP, Gemini, and generic REST/WebSocket integration
+- a desktop demo surface for local runtime verification
+- documentation for installation, quickstart, runtime, CLI, safety, testing, plugins, and architecture
+
+## Core Direction
+
+The strongest current architectural direction is:
+
+```text
+Agent Portal
+│
+├── agent-portal Python package
+│   ├── local runtime owner
+│   ├── Playwright browser/session owner
+│   ├── control and safety policy engine
+│   ├── HTTP API server
+│   └── reporting and plugin validation
+│
+├── VS Code extension
+│   └── developer-facing control panel
+│
+├── TypeScript SDK
+│   └── client wrapper for the Python runtime
+│
+└── Agent connectors
+    ├── ChatGPT tools
+    ├── Claude MCP server
+    ├── Gemini connector
+    └── REST/WebSocket API
+```
+
+The key principle is that runtime and browser session ownership should live in the Python runtime server, while editor tooling, SDKs, and connectors should act as clients of that runtime.
+
+## Repo Layout
+
+```text
+agent-portal.config.json      Runtime configuration
+assets/
+  branding/                   Shared logo and visual brand assets
+apps/
+  desktop/                    Desktop runtime demo and proving ground
+  vscode-extension/           Developer control panel
+connectors/
+  chatgpt-tools/              ChatGPT-facing connector direction
+  claude-mcp-server/          Claude MCP integration direction
+  gemini-connector/           Gemini integration direction
+  rest-websocket-api/         Generic transport direction
+docs/
+  architecture.md             System design and boundaries
+  roadmap.md                  Suggested phased delivery plan
+packages/
+  core/                       Shared TypeScript contracts and intelligence helpers
+  sdk/                        Runtime client SDK
+  mcp-server/                 Tool-facing MCP bridge surface
+plugins/
+  */plugin.json               Plugin manifests and examples
+python/
+  agent_portal/               Local runtime, browser control, CLI, doctor, server
+  tests/                      Python runtime test suite
+tests/
+  *.test.mjs                  Workspace-level Node tests
+```
+
+## Runtime Capabilities
+
+The local Python runtime currently covers these areas:
+
+- startup validation and single-instance locking
+- runtime health checks and doctor diagnostics
+- browser launch and cleanup
+- browser action execution with structured errors
+- policy-aware action approval and blocking
+- screenshot evidence capture
+- report generation
+- plugin manifest validation
+- localhost-first serving with optional bearer token auth
+
+### Runtime HTTP Routes
+
+- `GET /health`
+- `GET /status`
+- `GET /report/latest`
+- `POST /control/start`
+- `POST /control/stop`
+- `POST /control/pause`
+- `POST /control/resume`
+- `POST /control/restart`
+- `POST /control/goal`
+- `POST /control/approve-next`
+- `POST /control/reject-next`
+- `POST /browser/start`
+- `POST /browser/open`
+- `POST /browser/click`
+- `POST /browser/type`
+- `POST /browser/scroll`
+- `POST /browser/hover`
+- `POST /browser/wait`
+- `POST /browser/screenshot`
+- `POST /browser/capture`
+- `POST /browser/inspect`
+- `POST /browser/read-text`
+- `POST /browser/execute`
+- `POST /report/generate`
+
+## Agent Steering
+
+The current steering model is focused on keeping the runtime usable and safe while still allowing automation:
+
+- pause agent execution
+- resume execution
+- stop execution
+- inspect pending actions
+- approve next pending action
+- reject next pending action
+- assign or redirect current goal
+- risk-score actions
+- block password typing
+- block billing and payment-style actions
+- escalate destructive actions
+
+The longer-term target is a fuller steering layer with richer action editing, manual override states, and live queue streaming.
+
+## Getting Started
+
+1. Install Node dependencies:
+
+```bash
+npm install
+```
+
+2. Install Playwright browser binaries:
+
+```bash
+npx playwright install chromium
+```
+
+3. Install the Python runtime package:
+
+```bash
+pip install -e ./python
+```
+
+4. Run a health check:
+
+```bash
+agent-portal doctor
+```
+
+5. Start the Python runtime:
+
+```bash
+agent-portal start
+```
+
+6. In another terminal, run the desktop demo:
+
+```bash
+npm run dev --workspace @agent-portal/desktop
+```
+
+7. Open the VS Code extension sidebar and connect to the runtime.
+
+## Developer Workflow
+
+Typical local workflow:
+
+1. Start your local app on something like `localhost:3000` or `localhost:5173`.
+2. Start Agent Portal with `agent-portal start`.
+3. Connect the VS Code extension to `http://127.0.0.1:8765`.
+4. Use the runtime or SDK to open the app, inspect the page, and drive actions.
+5. Capture screenshots and reports for QA or debugging.
+6. Review blocked or pending actions through the control surface.
+
+## VS Code Extension
+
+The VS Code extension lives in `apps/vscode-extension` and is designed to be the developer-facing control panel.
+
+Official extension link:
+
+- [Agent Portal on the Visual Studio Marketplace](https://marketplace.visualstudio.com/manage/publishers/magnificent-language/extensions/agent-portal/hub)
+
+It currently provides:
+
+- a branded sidebar view
+- runtime start, stop, and restart commands
+- runtime polling and status display
+- pending action queue display
+- approve/reject controls
+- current goal display
+- local dev server detection
+- quick access to reports and docs
+
+Important settings:
+
+- `agentPortal.runtimeUrl`
+- `agentPortal.preferredLocalDevPort`
+
+Official Python Package link:
+
+-[Agent Portal on Python Package Index](https://pypi.org/project/agent-portal/)
+
+
+## SDK And Connectors
+
+The TypeScript SDK is shifting toward a thin client model that targets the Python runtime instead of owning browser sessions itself.
+
+Connector direction:
+
+- ChatGPT tools should translate tool calls into runtime API actions
+- Claude MCP should expose runtime tools and reports over MCP
+- Gemini connector should mirror the same runtime contract
+- REST/WebSocket connector should become the stable integration boundary for all external clients
+
+## Using Agent Portal With MCP
+
+Agent Portal now includes a Python MCP bridge package under `packages/agent-portal-mcp`.
+
+Basic flow:
+
+1. Start the runtime:
+
+```bash
+agent-portal start
+```
+
+2. Start the MCP server:
+
+```bash
+agent-portal mcp start
+```
+
+3. Connect an AI client that supports MCP to the local Agent Portal MCP server.
+
+The MCP server exposes browser, navigation, inspection, steering, and report tools while routing risky actions through the Agent Portal approval system.
+
+## Plugins
+
+The plugin model is manifest-driven.
+
+Each plugin can declare:
+
+- name
+- version
+- type
+- permissions
+- entry point
+- commands
+- settings
+- panels
+- lifecycle hooks
+
+The repository includes example and product-surface plugin manifests under `plugins/`.
+
+## Testing
+
+Current verification commands:
+
+```bash
+python -m compileall python
+python -m unittest discover -s python/tests -v
+npm run check
+npm test
+```
+
+These cover:
+
+- Python runtime compilation
+- runtime unit tests
+- doctor/config/plugin validation
+- HTTP server behavior
+- workspace TypeScript builds
+- extension manifest expectations
+- local browser/runtime workflow coverage in Node tests
+
+## Release Outputs
+
+The repo now includes first-pass release packaging flows for both Python and the desktop runtime.
+
+### Python Build Artifacts
+
+Build Python release artifacts into `releases/python` with:
+
+```bash
+npm run release:python
+```
+
+Expected outputs:
+
+- source distribution (`.tar.gz`)
+- wheel (`.whl`)
+
+If the `build` module is missing, the script exits with a clear install command.
+
+### Desktop Release Package
+
+Build a desktop runtime release zip into `releases/desktop` with:
+
+```bash
+npm run release:desktop
+```
+
+That package includes:
+
+- compiled desktop runtime output
+- local workflow fixtures
+- branding assets
+- a Windows launch script
+- release notes
+
+## Safety And Reliability
+
+Current hardening themes include:
+
+- localhost binding by default
+- optional local bearer token auth
+- structured runtime errors
+- duplicate-instance prevention
+- graceful shutdown paths
+- blocked high-risk categories
+- better user-facing diagnostics through `agent-portal doctor`
+- report-based traceability for actions and failures
+
+## Documentation
+
+Top-level docs available in this repo:
+
+- `INSTALLATION.md`
+- `QUICKSTART.md`
+- `CLI.md`
+- `RUNTIME.md`
+- `PYTHON_SDK.md`
+- `VS_CODE_EXTENSION.md`
+- `AGENT_STEERING.md`
+- `PLUGIN_SYSTEM.md`
+- `SAFETY_MODEL.md`
+- `TESTING.md`
+- `TROUBLESHOOTING.md`
+- `ARCHITECTURE.md`
+- `ROADMAP.md`
+
+## Near-Term Priorities
+
+1. Complete the migration of live session ownership from TypeScript runtime surfaces into the Python runtime server.
+2. Add a broadcast channel for live runtime events so the extension and connectors can stream state instead of polling.
+3. Expand approval flow from "approve next pending action" into richer queued-action execution control.
+4. Expand `VisionCore` from heuristics into a stronger multimodal understanding engine.
+5. Add durable memory retrieval, comparison, and project-aware context reuse.
+6. Extend runtime understanding beyond the browser into desktop applications and developer tools.

package/RELEASE_NOTES_v0.1.0.md

@@ -0,0 +1,281 @@
+# Agent Portal v0.1.0 - Release Summary
+
+## Overview
+
+Version 0.1.0 represents a major security and observability upgrade for Agent Portal. This release introduces enterprise-grade features including input validation, rate limiting, and comprehensive metrics collection, while maintaining full backward compatibility.
+
+## Key Statistics
+
+- **New Files Added**: 5 core modules + 3 test suites + 3 documentation files
+- **Lines of Code Added**: ~1,500 lines of production code
+- **Lines of Test Code**: ~800 lines of comprehensive tests
+- **Test Coverage**: 40+ new tests, all passing
+- **Security Improvements**: 15+ new security checks
+- **New Metrics**: 12 built-in runtime metrics
+
+## New Features
+
+### 1. Input Validation Module (`agent_portal/validation.py`)
+
+**Purpose**: Prevent malicious inputs and ensure data integrity
+
+**Functions**:
+- `validate_url()` - Validates URLs and blocks dangerous protocols
+- `validate_selector()` - Checks CSS selectors for XSS patterns
+- `validate_script()` - Validates JavaScript for unsafe operations
+- `validate_action_type()` - Ensures valid action types
+- `validate_risk_level()` - Validates risk level values
+- `validate_text_input()` - General text validation
+- `sanitize_text()` - Removes control characters and harmful content
+- `validate_config()` - Validates runtime configuration
+
+**Security Checks**:
+- Blocks dangerous URL schemes (javascript:, data:, file:, vbscript:)
+- Detects XSS patterns in selectors
+- Identifies unsafe JavaScript operations
+- Validates configuration security settings
+- Sanitizes user input
+
+### 2. Rate Limiting System (`agent_portal/rate_limit.py`)
+
+**Purpose**: Prevent abuse and ensure fair usage
+
+**Classes**:
+- `RateLimiter` - Sliding window rate limiting
+- `ActionThrottler` - Per-action throttling
+- `RateLimitConfig` - Configurable limits
+
+**Features**:
+- Per-minute, per-hour, and burst limits
+- Client-based tracking
+- Automatic blocking and timeout
+- Memory-efficient cleanup
+- Thread-safe implementation
+
+**Default Limits**:
+- 60 requests per minute
+- 1,000 requests per hour
+- 10 burst requests per second
+
+**Action-Specific Limits**:
+- Execute: 5 per minute, 50 per hour
+- Open URL: 20 per minute, 200 per hour
+- Type: 30 per minute, 300 per hour
+- Click: 60 per minute, 600 per hour
+- Screenshot: 10 per minute, 100 per hour
+
+### 3. Metrics & Telemetry (`agent_portal/metrics.py`)
+
+**Purpose**: Provide observability and performance insights
+
+**Classes**:
+- `MetricsCollector` - Thread-safe metrics collection
+- `TimerContext` - Context manager for timing operations
+- `MetricType` - Enum of metric types
+
+**Metric Types**:
+- **Counters** - Monotonically increasing values
+- **Gauges** - Point-in-time values
+- **Histograms** - Value distributions
+- **Timers** - Duration measurements with percentiles
+
+**Built-in Metrics**:
+- Runtime: uptime, active sessions, browser connected
+- Actions: total, completed, failed, blocked, approved, rejected
+- Browser: navigations, screenshots, errors
+- Network: requests, failures, console errors
+
+**Features**:
+- Thread-safe operations
+- Automatic sample limiting (prevents memory leaks)
+- Export to JSON
+- Percentile calculations (p50, p95, p99)
+- Histogram statistics (min, max, avg, sum, count)
+
+## Testing Infrastructure
+
+### Test Modules
+
+1. **`tests/test_validation.py`** (29 tests)
+   - URL validation (safe/unsafe protocols)
+   - Selector validation (XSS patterns)
+   - Script validation (dangerous operations)
+   - Configuration validation
+   - Text sanitization
+
+2. **`tests/test_rate_limit.py`** (13 tests)
+   - Basic rate limiting
+   - Burst limit enforcement
+   - Per-minute/per-hour limits
+   - Client independence
+   - Block timeout behavior
+   - Cleanup functionality
+   - Action throttling
+
+3. **`tests/test_metrics.py`** (23 tests)
+   - Counter operations
+   - Gauge operations
+   - Histogram recording
+   - Timer context manager
+   - Percentile calculations
+   - Export functionality
+   - Global metrics singleton
+   - Built-in metrics initialization
+
+**Total Test Count**: 65+ tests, all passing
+
+## Documentation
+
+### New Documentation Files
+
+1. **`CHANGELOG.md`**
+   - Comprehensive version history
+   - Categorized changes (Added, Security, Performance)
+   - Follows Keep a Changelog format
+
+2. **`UPGRADE_GUIDE.md`**
+   - Step-by-step upgrade instructions
+   - Migration examples
+   - Configuration options
+   - Troubleshooting guide
+
+3. **`RELEASE_NOTES_v0.1.0.md`** (this file)
+   - Executive summary of changes
+   - Key statistics
+   - Feature highlights
+
+## Security Improvements
+
+### Before v0.1.0
+- Basic input type checking
+- No rate limiting
+- No metrics for security events
+- Manual policy enforcement
+
+### After v0.1.0
+- Comprehensive input validation
+- Automatic rate limiting
+- Security event tracking
+- Action throttling
+- Dangerous pattern detection
+- Configuration validation
+
+### Threats Mitigated
+
+1. **Injection Attacks**
+   - XSS via selector manipulation
+   - JavaScript injection in page context
+   - URL scheme injection
+
+2. **Denial of Service**
+   - Request flooding (rate limiting)
+   - Resource exhaustion (action throttling)
+   - Memory leaks (sample limiting)
+
+3. **Data Integrity**
+   - Invalid configuration enforcement
+   - Input sanitization
+   - Type validation
+
+## Performance Impact
+
+### Memory
+- +2-5MB base memory for metrics collection
+- Configurable max samples (default: 10,000)
+- Automatic cleanup prevents unbounded growth
+
+### CPU
+- Negligible impact from validation (<1ms per request)
+- Sliding window algorithm is O(1) per request
+- Timer overhead: ~0.01ms
+
+### Network
+- No additional network calls
+- Optional metric export to filesystem
+
+## Backward Compatibility
+
+✅ **Fully Compatible**
+- All existing workflows work unchanged
+- No breaking API changes
+- Optional features (can be disabled if needed)
+- Default behavior is safe
+
+## Installation
+
+```bash
+# Upgrade existing installation
+pip install --upgrade agent-portal
+
+# Or install fresh
+pip install agent-portal
+```
+
+## Quick Start
+
+### Enable Validation
+```python
+from agent_portal.validation import validate_url
+
+result = validate_url("https://example.com")
+if not result.is_valid:
+    print(f"Error: {result.errors}")
+```
+
+### Monitor Metrics
+```python
+from agent_portal import get_metrics
+
+metrics = get_metrics()
+print(f"Actions completed: {metrics.get_counter('actions.completed')}")
+```
+
+### Configure Rate Limiting
+```python
+from agent_portal import RateLimiter, RateLimitConfig
+
+config = RateLimitConfig(
+    requests_per_minute=100,
+    burst_limit=15
+)
+limiter = RateLimiter(config)
+```
+
+## Release Artifacts
+
+- **Python Package**: `agent_portal-0.0.2-py3-none-any.whl`
+- **Source Distribution**: `agent_portal-0.0.2.tar.gz`
+- **Desktop Release**: `agent-portal-desktop.zip`
+- **GitHub Release**: https://github.com/magnexis/agent-portal/releases/tag/v0.1.0
+
+## Contributing
+
+This release includes contributions from the core team. We welcome community contributions!
+
+## Future Roadmap
+
+See [docs/roadmap.md](docs/roadmap.md) for upcoming features including:
+- Advanced vision core
+- Multi-agent coordination
+- Desktop application control
+- Enhanced reporting
+
+## Acknowledgments
+
+- Built with Python 3.10+
+- Powered by Playwright for browser automation
+- Uses ThreadingHTTPServer for concurrent requests
+- Metrics export compatible with common observability tools
+
+## Support
+
+- **Issues**: https://github.com/magnexis/agent-portal/issues
+- **Discussions**: https://github.com/magnexis/agent-portal/discussions
+- **Documentation**: https://github.com/magnexis/agent-portal/tree/main/docs
+
+---
+
+**Release Date**: January 2025
+**Version**: 0.1.0
+**Status**: Stable ✅
+**Backward Compatible**: Yes ✅

package/ROADMAP.md

@@ -0,0 +1,3 @@
+# Roadmap
+
+See [docs/roadmap.md](docs/roadmap.md) for the canonical roadmap.

package/RUNTIME.md

@@ -0,0 +1,44 @@
+# Runtime
+
+The local runtime lives in `python/agent_portal`.
+
+## Responsibilities
+
+- own the local browser process
+- enforce agent steering and risk policy
+- expose runtime control endpoints
+- generate session reports
+- validate plugin manifests
+
+## Startup
+
+```bash
+agent-portal doctor
+agent-portal start
+```
+
+By default the runtime binds to `127.0.0.1:8765`.
+
+## Safety Defaults
+
+- localhost-only binding by default
+- optional bearer token support through `api_token`
+- blocked password typing
+- blocked billing and payment actions
+- destructive actions escalated to high risk
+- screenshot capture disabled for sensitive flows unless enabled
+
+## Reports
+
+Generated reports include:
+
+- project name
+- session id
+- current url
+- goals
+- approved, rejected, blocked, completed, and failed actions
+- console and network errors
+- screenshots
+- reproduction steps
+
+Reports are written to the configured report directory.