portacode 1.3.26__tar.gz → 1.3.27__tar.gz

{portacode-1.3.26 → portacode-1.3.27}/.claude/settings.local.json

@@ -17,7 +17,6 @@
       "Bash(git -C /home/menas/testing_folder status --porcelain)",
       "Bash(docker-compose restart:*)"
     ],
-    "deny": [],
-    "defaultMode": "acceptEdits"
+    "deny": []
   }
 }

portacode-1.3.27/MANIFEST.in

@@ -0,0 +1,3 @@
+prune server
+exclude .pypirc
+recursive-include portacode/static *.js *.html

{portacode-1.3.26 → portacode-1.3.27}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: portacode
-Version: 1.3.26
+Version: 1.3.27
 Summary: Portacode CLI client and SDK
 Home-page: https://github.com/portacode/portacode
 Author: Meena Erian
@@ -23,6 +23,7 @@ Requires-Dist: GitPython>=3.1.45
 Requires-Dist: watchdog>=3.0
 Requires-Dist: diff-match-patch>=20230430
 Requires-Dist: Pygments>=2.14.0
+Requires-Dist: ntplib>=0.4.0
 Provides-Extra: dev
 Requires-Dist: black; extra == "dev"
 Requires-Dist: flake8; extra == "dev"

{portacode-1.3.26 → portacode-1.3.27}/portacode/_version.py

@@ -28,7 +28,7 @@ version_tuple: VERSION_TUPLE
 commit_id: COMMIT_ID
 __commit_id__: COMMIT_ID
 
-__version__ = version = '1.3.26'
-__version_tuple__ = version_tuple = (1, 3, 26)
+__version__ = version = '1.3.27'
+__version_tuple__ = version_tuple = (1, 3, 27)
 
 __commit_id__ = commit_id = None

{portacode-1.3.26 → portacode-1.3.27}/portacode/connection/handlers/WEBSOCKET_PROTOCOL.md

@@ -1,10 +1,38 @@
 # WebSocket Communication Protocol
 
-This document outlines the WebSocket communication protocol between the Portacode server and the connected client devices.
+This document outlines the WebSocket communication protocol used in Portacode. The protocol involves three main participants: client sessions, the Portacode server, and devices.
+
+## Architecture Overview
+
+```
+┌─────────────┐          ┌──────────────────┐          ┌─────────────────────────┐
+│   Client    │          │   Portacode      │          │        Device           │
+│   Session   │◄────────►│    Server        │◄────────►│ (Portacode CLI or       │
+│             │          │                  │          │  Python package)        │
+└─────────────┘          └──────────────────┘          └─────────────────────────┘
+     │                           │                                  │
+     │                           │                                  │
+  Client-Side              Acts as middleman                  Device-Side
+  Protocol                 - Routes messages                  Protocol
+                          - Manages sessions
+```
+
+The Portacode server acts as a **routing middleman** between client sessions and devices. It manages routing fields that are included in messages to specify routing destinations but are removed or transformed before reaching the final recipient:
+
+**Routing Fields Behavior:**
+
+- **`device_id`** (Client → Server): Client includes this to specify which device to route to. Server uses it for routing, then **removes it** before forwarding to the device (the device knows the message is for them). Server **adds it** when routing device responses back to clients (so clients know which device the message came from).
+
+- **`client_sessions`** (Device → Server): Device includes this to specify which client session(s) to route to. Server uses it for routing, then **removes it** before forwarding to clients (clients just receive the message without seeing routing metadata).
+
+- **`source_client_session`** (Server → Device): Server **adds this** when forwarding client commands to devices (so device knows which client sent the command and can target responses back). Clients never include this field.
+
+This document describes the complete protocol for communicating with devices through the server, guiding app developers on how to get their client sessions to communicate with devices.
 
 ## Table of Contents
 
-- [Raw Message Format](#raw-message-format)
+- [Raw Message Format On Device Side](#raw-message-format-on-device-side)
+- [Raw Message Format On Client Side](#raw-message-format-on-client-side)
 - [Actions](#actions)
   - [Terminal Actions](#terminal-actions)
     - [`terminal_start`](#terminal_start)
@@ -82,11 +110,11 @@ This document outlines the WebSocket communication protocol between the Portacod
     - [`device_status`](#device_status)
     - [`devices`](#devices)
 
-## Raw Message Format
+## Raw Message Format On Device Side
 
-All communication over the WebSocket is managed by a [multiplexer](./multiplex.py) that wraps every message in a JSON object with a `channel` and a `payload`. This allows for multiple virtual communication channels over a single connection.
+Communication between the server and devices uses a [multiplexer](./multiplex.py) that wraps every message in a JSON object with a `channel` and a `payload`. This allows for multiple virtual communication channels over a single WebSocket connection.
 
-**Raw Message Structure:**
+**Device-Side Message Structure:**
 
 ```json
 {
@@ -97,9 +125,99 @@ All communication over the WebSocket is managed by a [multiplexer](./multiplex.p
 }
 ```
 
-*   `channel` (string|integer, mandatory): Identifies the virtual channel the message is for. When sending control commands to the device, they should be sent to channel 0 and when the device responsed to such control commands or sends system events, they will also be send on the zero channel. When a terminal session is created in the device, it is assigned a uuid, the uuid becomes the channel for communicating to that specific terminal.
+**Field Descriptions:**
+
+*   `channel` (string|integer, mandatory): Identifies the virtual channel the message is for. When sending control commands to the device, they should be sent to channel 0 and when the device responds to such control commands or sends system events, they will also be sent on the zero channel. When a terminal session is created in the device, it is assigned a uuid, the uuid becomes the channel for communicating to that specific terminal.
 *   `payload` (object, mandatory): The content of the message, which will be either an [Action](#actions) or an [Event](#events) object.
 
+**Channel Types:**
+- **Channel 0** (control channel): Used for system commands, terminal management, file operations, and project state management
+- **Channel UUID** (terminal channel): Used for terminal I/O to a specific terminal session
+
+---
+
+## Raw Message Format On Client Side
+
+Client sessions communicate with the server using a unified message format with the same field names as the device protocol, plus routing information.
+
+**Client-Side Message Structure (Client → Server):**
+
+```json
+{
+  "device_id": <number>,
+  "channel": <number|string>,
+  "payload": {
+    "cmd": "<command_name>",
+    ...command-specific fields
+  }
+}
+```
+
+**Field Descriptions:**
+
+*   `device_id` (number, mandatory): Routing field - specifies which device to send the message to. The server validates that the client has access to this device before forwarding.
+*   `channel` (number|string, mandatory): Same as device protocol - the target channel (0 for control, UUID for terminal). Uses the same field name for consistency.
+*   `payload` (object, mandatory): Same as device protocol - the command payload. Uses the same field name for consistency.
+
+**Server Transformation (Client → Device):**
+
+When the server receives a client message, it:
+1. Validates client has access to the specified `device_id`
+2. **Removes** `device_id` from the message (device doesn't need to be told "this is for you")
+3. **Adds** `source_client_session` to the payload (so device knows which client to respond to)
+4. Forwards to device: `{channel, payload: {...payload, source_client_session}}`
+
+**Server Transformation (Device → Client):**
+
+When the server receives a device response, it:
+1. **Adds** `device_id` to the message (so client knows which device it came from, based on authenticated device connection)
+2. **Removes** `client_sessions` routing metadata (clients don't need to see routing info)
+3. Routes to appropriate client session(s)
+
+**Server Response Format (Server → Client):**
+
+```json
+{
+  "event": "<event_name>",
+  "device_id": <number>,
+  ...event-specific fields
+}
+```
+
+**Field Descriptions:**
+
+*   `event` (string, mandatory): The name of the event being sent.
+*   `device_id` (number, mandatory): Authenticated field - identifies which device the event came from (added by server based on authenticated device connection).
+*   Additional fields depend on the specific event type.
+
+**Example Client Message:**
+```json
+{
+  "device_id": 42,
+  "channel": 0,
+  "payload": {
+    "cmd": "terminal_start",
+    "shell": "bash",
+    "cwd": "/home/user/project"
+  }
+}
+```
+
+**Example Server Response:**
+```json
+{
+  "event": "terminal_started",
+  "device_id": 42,
+  "terminal_id": "uuid-1234-5678",
+  "channel": "uuid-1234-5678",
+  "pid": 12345
+}
+```
+
+**Note:** The server acts as a translator between the client-side and device-side protocols:
+- When a client sends a command, the server transforms it from the client format to the device format
+- When a device sends an event, the server adds the `device_id` and routes it to the appropriate client sessions
+
 ---
 
 ## Actions
@@ -110,18 +228,18 @@ Actions are messages sent from the server to the device, placed within the `payl
 
 ```json
 {
-  "command": "<command_name>",
-  "payload": {
-    "arg1": "value1",
-    "...": "..."
-  },
+  "cmd": "<command_name>",
+  "arg1": "value1",
+  "arg2": "value2",
   "source_client_session": "channel.abc123"
 }
 ```
 
-*   `command` (string, mandatory): The name of the action to be executed (e.g., `terminal_start`).
-*   `payload` (object, mandatory): An object containing the specific arguments for the action.
+**Field Descriptions:**
+
+*   `cmd` (string, mandatory): The name of the action to be executed (e.g., `terminal_start`, `file_read`, `system_info`).
 *   `source_client_session` (string, mandatory): The channel name of the client session that originated this command. This field is automatically added by the server and allows devices to identify which specific client sent the command.
+*   Additional fields depend on the specific command (see individual command documentation below).
 
 **Note**: Actions do not require targeting information - responses are automatically routed using the client session management system.
 

{portacode-1.3.26 → portacode-1.3.27}/portacode/connection/handlers/base.py

@@ -4,6 +4,7 @@ import asyncio
 import logging
 from abc import ABC, abstractmethod
 from typing import Any, Dict, Optional, TYPE_CHECKING
+from portacode.utils.ntp_clock import ntp_clock
 
 if TYPE_CHECKING:
     from ..multiplex import Channel
@@ -42,35 +43,45 @@ class BaseHandler(ABC):
     
     async def send_response(self, payload: Dict[str, Any], reply_channel: Optional[str] = None, project_id: str = None) -> None:
         """Send a response back to the gateway with client session awareness.
-        
+
         Args:
             payload: Response payload
             reply_channel: Optional reply channel for backward compatibility
             project_id: Optional project filter for targeting specific sessions
         """
+        # Add device_send timestamp if trace present
+        if "trace" in payload and "request_id" in payload:
+            device_send_time = ntp_clock.now_ms()
+            if device_send_time is not None:
+                payload["trace"]["device_send"] = device_send_time
+                # Update ping to show total time from client_send
+                if "client_send" in payload["trace"]:
+                    payload["trace"]["ping"] = device_send_time - payload["trace"]["client_send"]
+                logger.info(f"📤 Device sending traced response: {payload['request_id']}")
+
         # Get client session manager from context
         client_session_manager = self.context.get("client_session_manager")
-        
+
         if client_session_manager and client_session_manager.has_interested_clients():
             # Get target sessions
             target_sessions = client_session_manager.get_target_sessions(project_id)
             if not target_sessions:
                 logger.debug("handler: No target sessions found, skipping response send")
                 return
-            
+
             # Add session targeting information
             enhanced_payload = dict(payload)
             enhanced_payload["client_sessions"] = target_sessions
-            
+
             # Add backward compatibility reply_channel (first session if not provided)
             if not reply_channel:
                 reply_channel = client_session_manager.get_reply_channel_for_compatibility()
             if reply_channel:
                 enhanced_payload["reply_channel"] = reply_channel
-            
-            logger.debug("handler: Sending response to %d client sessions: %s", 
+
+            logger.debug("handler: Sending response to %d client sessions: %s",
                         len(target_sessions), target_sessions)
-            
+
             await self.control_channel.send(enhanced_payload)
         else:
             # Fallback to original behavior if no client session manager or no clients
@@ -107,18 +118,43 @@ class AsyncHandler(BaseHandler):
     
     async def handle(self, message: Dict[str, Any], reply_channel: Optional[str] = None) -> None:
         """Handle the command by executing it and sending the response."""
-        logger.info("handler: Processing command %s with reply_channel=%s", 
+        logger.info("handler: Processing command %s with reply_channel=%s",
                    self.command_name, reply_channel)
-        
+
+        # Add handler_dispatch timestamp if trace present
+        if "trace" in message and "request_id" in message:
+            handler_dispatch_time = ntp_clock.now_ms()
+            if handler_dispatch_time is not None:
+                message["trace"]["handler_dispatch"] = handler_dispatch_time
+                # Update ping to show total time from client_send
+                if "client_send" in message["trace"]:
+                    message["trace"]["ping"] = handler_dispatch_time - message["trace"]["client_send"]
+                logger.info(f"🔧 Handler dispatching: {message['request_id']} ({self.command_name})")
+
         try:
             response = await self.execute(message)
             logger.info("handler: Command %s executed successfully", self.command_name)
-            
+
             # Handle cases where execute() sends responses directly and returns None
             if response is not None:
+                # Automatically copy request_id if present in the incoming message
+                if "request_id" in message and "request_id" not in response:
+                    response["request_id"] = message["request_id"]
+
+                # Pass through trace from request to response (add to existing trace, don't create new one)
+                if "trace" in message and "request_id" in message:
+                    response["trace"] = dict(message["trace"])
+                    handler_complete_time = ntp_clock.now_ms()
+                    if handler_complete_time is not None:
+                        response["trace"]["handler_complete"] = handler_complete_time
+                        # Update ping to show total time from client_send
+                        if "client_send" in response["trace"]:
+                            response["trace"]["ping"] = handler_complete_time - response["trace"]["client_send"]
+                        logger.info(f"✅ Handler completed: {message['request_id']} ({self.command_name})")
+
                 # Extract project_id from response for session targeting
                 project_id = response.get("project_id")
-                logger.info("handler: %s response project_id=%s, response=%s", 
+                logger.info("handler: %s response project_id=%s, response=%s",
                            self.command_name, project_id, response)
                 await self.send_response(response, reply_channel, project_id)
             else:
@@ -147,10 +183,32 @@ class SyncHandler(BaseHandler):
     
     async def handle(self, message: Dict[str, Any], reply_channel: Optional[str] = None) -> None:
         """Handle the command by executing it in an executor and sending the response."""
+        # Add handler_dispatch timestamp if trace present
+        if "trace" in message and "request_id" in message:
+            handler_dispatch_time = ntp_clock.now_ms()
+            if handler_dispatch_time is not None:
+                message["trace"]["handler_dispatch"] = handler_dispatch_time
+                # Update ping to show total time from client_send
+                if "client_send" in message["trace"]:
+                    message["trace"]["ping"] = handler_dispatch_time - message["trace"]["client_send"]
+                logger.info(f"🔧 Handler dispatching: {message['request_id']} ({self.command_name})")
+
         try:
             loop = asyncio.get_running_loop()
             response = await loop.run_in_executor(None, self.execute, message)
-            
+
+            # Automatically copy request_id if present in the incoming message
+            if "request_id" in message and "request_id" not in response:
+                response["request_id"] = message["request_id"]
+
+            # Pass through trace from request to response (add to existing trace, don't create new one)
+            if "trace" in message and "request_id" in message:
+                response["trace"] = dict(message["trace"])
+                handler_complete_time = ntp_clock.now_ms()
+                if handler_complete_time is not None:
+                    response["trace"]["handler_complete"] = handler_complete_time
+                    logger.info(f"✅ Handler completed: {message['request_id']} ({self.command_name})")
+
             # Extract project_id from response for session targeting
             project_id = response.get("project_id")
             await self.send_response(response, reply_channel, project_id)

{portacode-1.3.26 → portacode-1.3.27}/portacode/connection/handlers/file_handlers.py

@@ -382,22 +382,18 @@ class ContentRequestHandler(AsyncHandler):
     async def execute(self, message: Dict[str, Any]) -> None:
         """Return content by hash if available, chunked for large content."""
         content_hash = message.get("content_hash")
-        request_id = message.get("request_id")
         source_client_session = message.get("source_client_session")
-        
+
         if not content_hash:
             raise ValueError("content_hash parameter is required")
-        if not request_id:
-            raise ValueError("request_id parameter is required")
-        
+
         # Check if content is in cache
         content = _content_cache.get(content_hash)
-        
+
         if content is not None:
-            # Create base response
+            # Create base response (request_id will be added automatically by base class)
             base_response = {
                 "event": "content_response",
-                "request_id": request_id,
                 "content_hash": content_hash,
                 "success": True,
             }
@@ -411,10 +407,9 @@ class ContentRequestHandler(AsyncHandler):
             
             logger.info(f"Sent content response in {len(responses)} chunk(s) for hash: {content_hash[:16]}...")
         else:
-            # Content not found in cache
+            # Content not found in cache (request_id will be added automatically by base class)
             response = {
                 "event": "content_response",
-                "request_id": request_id,
                 "content_hash": content_hash,
                 "content": None,
                 "success": False,

{portacode-1.3.26 → portacode-1.3.27}/portacode/connection/handlers/project_state/handlers.py

@@ -683,9 +683,8 @@ class ProjectStateDiffContentHandler(AsyncHandler):
         from_hash = message.get("from_hash")
         to_hash = message.get("to_hash")
         content_type = message.get("content_type")  # 'original', 'modified', 'html_diff'
-        request_id = message.get("request_id")
         source_client_session = message.get("source_client_session")
-        
+
         # Validate required fields
         if not server_project_id:
             raise ValueError("project_id is required")
@@ -697,8 +696,6 @@ class ProjectStateDiffContentHandler(AsyncHandler):
             raise ValueError("to_ref is required")
         if not content_type:
             raise ValueError("content_type is required")
-        if not request_id:
-            raise ValueError("request_id is required")
         if not source_client_session:
             raise ValueError("source_client_session is required")
         
@@ -816,9 +813,12 @@ class ProjectStateDiffContentHandler(AsyncHandler):
                 "from_ref": from_ref,
                 "to_ref": to_ref,
                 "content_type": content_type,
-                "request_id": request_id,
                 "success": success
             }
+
+            # Add request_id if present in original message
+            if "request_id" in message:
+                base_response["request_id"] = message["request_id"]
             
             if from_hash:
                 base_response["from_hash"] = from_hash
@@ -850,9 +850,12 @@ class ProjectStateDiffContentHandler(AsyncHandler):
                 "from_ref": from_ref,
                 "to_ref": to_ref,
                 "content_type": content_type,
-                "request_id": request_id,
                 "success": False,
                 "error": str(e),
                 "chunked": False
             }
+
+            # Add request_id if present in original message
+            if "request_id" in message:
+                error_response["request_id"] = message["request_id"]
             await self.send_response(error_response, project_id=server_project_id)

{portacode-1.3.26 → portacode-1.3.27}/portacode/connection/handlers/registry.py

@@ -2,6 +2,7 @@
 
 import logging
 from typing import Dict, Type, Any, Optional, List, TYPE_CHECKING
+from portacode.utils.ntp_clock import ntp_clock
 
 if TYPE_CHECKING:
     from ..multiplex import Channel
@@ -72,22 +73,32 @@ class CommandRegistry:
     
     async def dispatch(self, command_name: str, message: Dict[str, Any], reply_channel: Optional[str] = None) -> bool:
         """Dispatch a command to its handler.
-        
+
         Args:
             command_name: The command name
             message: The command message
             reply_channel: Optional reply channel
-            
+
         Returns:
             True if handler was found and executed, False otherwise
         """
         logger.info("registry: Dispatching command '%s' with reply_channel=%s", command_name, reply_channel)
-        
+
+        # Add device_receive timestamp if trace present
+        if "trace" in message and "request_id" in message:
+            device_receive_time = ntp_clock.now_ms()
+            if device_receive_time is not None:
+                message["trace"]["device_receive"] = device_receive_time
+                # Update ping to show total time from client_send
+                if "client_send" in message["trace"]:
+                    message["trace"]["ping"] = device_receive_time - message["trace"]["client_send"]
+                logger.info(f"📨 Device received traced message: {message['request_id']}")
+
         handler = self.get_handler(command_name)
         if handler is None:
             logger.warning("registry: No handler found for command: %s", command_name)
             return False
-        
+
         try:
             await handler.handle(message, reply_channel)
             logger.info("registry: Successfully dispatched command '%s'", command_name)

portacode-1.3.27/portacode/static/js/test-ntp-clock.html

@@ -0,0 +1,63 @@
+<!DOCTYPE html>
+<html>
+<head>
+    <title>NTP Clock Test</title>
+    <style>
+        body {
+            font-family: monospace;
+            padding: 20px;
+            background: #1e1e1e;
+            color: #d4d4d4;
+        }
+        h1 {
+            color: #4ec9b0;
+        }
+        #status {
+            background: #252526;
+            padding: 15px;
+            border-radius: 5px;
+            border: 1px solid #3e3e42;
+        }
+        .synced {
+            color: #4ec9b0;
+        }
+        .not-synced {
+            color: #f48771;
+        }
+        .label {
+            color: #9cdcfe;
+            font-weight: bold;
+        }
+    </style>
+</head>
+<body>
+    <h1>NTP Clock Test - time.cloudflare.com</h1>
+    <div id="status"></div>
+    <script type="module">
+        import ntpClock from './utils/ntp-clock.js';
+
+        function updateDisplay() {
+            const status = ntpClock.getStatus();
+            const syncClass = status.isSynced ? 'synced' : 'not-synced';
+
+            document.getElementById('status').innerHTML = `
+                <p><span class="label">Sync Status:</span> <span class="${syncClass}">${status.isSynced ? '✅ SYNCED' : '❌ NOT SYNCED'}</span></p>
+                <p><span class="label">NTP Server:</span> ${status.server}</p>
+                <p><span class="label">NTP Time:</span> ${ntpClock.nowISO() || 'null (not synced)'}</p>
+                <p><span class="label">Local Time:</span> ${new Date().toISOString()}</p>
+                <p><span class="label">Offset:</span> ${status.offset !== null ? status.offset.toFixed(2) + 'ms' : 'null'}</p>
+                <p><span class="label">Last Sync:</span> ${status.lastSync || 'Never'}</p>
+                <p><span class="label">Time Since Sync:</span> ${status.timeSinceSync ? (status.timeSinceSync / 1000).toFixed(0) + 's' : 'N/A'}</p>
+            `;
+        }
+
+        // Update display every 100ms
+        setInterval(updateDisplay, 100);
+
+        // Log status to console every 5 seconds
+        setInterval(() => {
+            console.log('NTP Clock Status:', ntpClock.getStatus());
+        }, 5000);
+    </script>
+</body>
+</html>