mcp-server-macos-use 0.1.0

package/LICENSE

@@ -0,0 +1,37 @@
+Business Source License 1.1
+
+TERMS
+
+The Licensor hereby grants you the right to copy, modify, create derivative works, redistribute, and make non-production use of the Licensed Work. The Licensor may make an Additional Use Grant, above, permitting limited production use.
+
+Effective on the Change Date, or the fourth anniversary of the first publicly available distribution of a specific version of the Licensed Work under this License, whichever comes first, the Licensor hereby grants you rights under the terms of the Change License, and the rights granted in the paragraph above terminate.
+
+If your use of the Licensed Work does not comply with the requirements currently in effect as described in this License, you must purchase a commercial license from the Licensor, its affiliated entities, or authorized resellers, or you must refrain from using the Licensed Work.
+
+All copies of the original and modified Licensed Work, and derivative works of the Licensed Work, are subject to this License. This License applies separately for each version of the Licensed Work and the Change Date may vary for each version of the Licensed Work released by Licensor.
+
+You must conspicuously display this License on each original or modified copy of the Licensed Work. If you receive the Licensed Work in original or modified form from a third party, the terms and conditions set forth in this License apply to your use of that work.
+
+Any use of the Licensed Work in violation of this License will automatically terminate your rights under this License for the current and all other versions of the Licensed Work.
+
+This License does not grant you any right in any trademark or logo of Licensor or its affiliates (provided that you may use a trademark or logo of Licensor as expressly required by this License).
+
+TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED ON AN "AS IS" BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS, EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND TITLE.
+
+MariaDB hereby grants you permission to use this License's text to license your works, and to refer to it using the trademark "Business Source License", as long as you comply with the Covenants of Licensor below.
+
+COVENANTS OF LICENSOR
+
+In consideration of the right to use this License's text and the "Business Source License" name and trademark, Licensor covenants to MariaDB, and to all other recipients of the licensed work to be provided by Licensor:
+
+1. To specify as the Change License the GPL Version 2.0 or any later version, or a license that is compatible with GPL Version 2.0 or a later version, where "compatible" means that software provided under the Change License can be included in a program with software provided under GPL Version 2.0 or a later version. Licensor may specify additional Change Licenses without limitation.
+2. To either: (a) specify an additional grant of rights to use that does not impose any additional restriction on the right granted in this License, as the Additional Use Grant; or (b) insert the text "None".
+3. Not to modify this License in any other way.
+
+-------------------------
+
+Licensed Work: screenpipe Computer Agent
+Licensor: Mediar, Inc.
+Additional Use Grant: Production use is permitted for non-commercial, educational purposes only
+Change Date: April 9, 2028
+Change License: MIT License 

package/Package.resolved

@@ -0,0 +1,86 @@
+{
+  "pins" : [
+    {
+      "identity" : "eventsource",
+      "kind" : "remoteSourceControl",
+      "location" : "https://github.com/mattt/eventsource.git",
+      "state" : {
+        "revision" : "07957602bb99a5355c810187e66e6ce378a1057d",
+        "version" : "1.1.1"
+      }
+    },
+    {
+      "identity" : "macosusesdk",
+      "kind" : "remoteSourceControl",
+      "location" : "https://github.com/mediar-ai/MacosUseSDK.git",
+      "state" : {
+        "branch" : "main",
+        "revision" : "5d810b557dc73f8fb930767671b0cfcd989a23e7"
+      }
+    },
+    {
+      "identity" : "swift-async-algorithms",
+      "kind" : "remoteSourceControl",
+      "location" : "https://github.com/apple/swift-async-algorithms.git",
+      "state" : {
+        "revision" : "2971dd5d9f6e0515664b01044826bcea16e59fac",
+        "version" : "1.1.2"
+      }
+    },
+    {
+      "identity" : "swift-atomics",
+      "kind" : "remoteSourceControl",
+      "location" : "https://github.com/apple/swift-atomics.git",
+      "state" : {
+        "revision" : "b601256eab081c0f92f059e12818ac1d4f178ff7",
+        "version" : "1.3.0"
+      }
+    },
+    {
+      "identity" : "swift-collections",
+      "kind" : "remoteSourceControl",
+      "location" : "https://github.com/apple/swift-collections.git",
+      "state" : {
+        "revision" : "7b847a3b7008b2dc2f47ca3110d8c782fb2e5c7e",
+        "version" : "1.3.0"
+      }
+    },
+    {
+      "identity" : "swift-log",
+      "kind" : "remoteSourceControl",
+      "location" : "https://github.com/apple/swift-log.git",
+      "state" : {
+        "revision" : "ce592ae52f982c847a4efc0dd881cc9eb32d29f2",
+        "version" : "1.6.4"
+      }
+    },
+    {
+      "identity" : "swift-nio",
+      "kind" : "remoteSourceControl",
+      "location" : "https://github.com/apple/swift-nio.git",
+      "state" : {
+        "revision" : "e932d3c4d8f77433c8f7093b5ebcbf91463948a0",
+        "version" : "2.95.0"
+      }
+    },
+    {
+      "identity" : "swift-sdk",
+      "kind" : "remoteSourceControl",
+      "location" : "https://github.com/modelcontextprotocol/swift-sdk.git",
+      "state" : {
+        "revision" : "6112a3995a992d159ad0e82c2d62a008ce932666",
+        "version" : "0.11.0"
+      }
+    },
+    {
+      "identity" : "swift-system",
+      "kind" : "remoteSourceControl",
+      "location" : "https://github.com/apple/swift-system.git",
+      "state" : {
+        "revision" : "890830fff1a577dc83134890c7984020c5f6b43b",
+        "version" : "1.6.2"
+      }
+    }
+  ],
+  "version" : 2
+}

package/Package.swift

@@ -0,0 +1,26 @@
+// swift-tools-version: 5.9
+// The swift-tools-version declares the minimum version of Swift required to build this package.
+
+import PackageDescription
+
+let package = Package(
+    name: "mcp-server-macos-use",
+    platforms: [
+        .macOS(.v13)
+    ],
+    dependencies: [
+        .package(url: "https://github.com/modelcontextprotocol/swift-sdk.git", from: "0.11.0"),
+        .package(url: "https://github.com/mediar-ai/MacosUseSDK.git", branch: "main")
+    ],
+    targets: [
+        // Targets are the basic building blocks of a package, defining a module or a test suite.
+        // Targets can depend on other targets in this package and products from dependencies.
+        .executableTarget(
+            name: "mcp-server-macos-use",
+            dependencies: [
+                .product(name: "MCP", package: "swift-sdk"),
+                .product(name: "MacosUseSDK", package: "MacosUseSDK")
+            ]
+        ),
+    ]
+)

package/README.md

@@ -0,0 +1,96 @@
+# mcp-server-macos-use
+
+Model Context Protocol (MCP) server in Swift. It allows controlling macOS applications by leveraging the accessibility APIs, primarily through the `MacosUseSDK`.
+
+You can use it in Claude Desktop or other compatible MCP-client.
+
+The server listens for MCP commands over standard input/output (`stdio`) and exposes several tools to interact with applications.
+
+
+https://github.com/user-attachments/assets/b43622a3-3d20-4026-b02f-e9add06afe2b
+
+## Available Tools
+
+The server exposes the following tools via the `CallTool` MCP method:
+
+1.  **`macos-use_open_application_and_traverse`**
+    *   **Description:** Opens or activates a specified application and then traverses its accessibility tree.
+    *   **Parameters:**
+        *   `identifier` (String, Required): The application's name, bundle ID, or file path.
+
+2.  **`macos-use_click_and_traverse`**
+    *   **Description:** Simulates a mouse click at specific coordinates within the window of the target application (identified by PID) and then traverses its accessibility tree.
+    *   **Parameters:**
+        *   `pid` (Number, Required): The Process ID (PID) of the target application.
+        *   `x` (Number, Required): The X-coordinate for the click (relative to the window/screen, depending on SDK behavior).
+        *   `y` (Number, Required): The Y-coordinate for the click.
+
+3.  **`macos-use_type_and_traverse`**
+    *   **Description:** Simulates typing text into the target application (identified by PID) and then traverses its accessibility tree.
+    *   **Parameters:**
+        *   `pid` (Number, Required): The Process ID (PID) of the target application.
+        *   `text` (String, Required): The text to be typed.
+
+4.  **`macos-use_press_key_and_traverse`**
+    *   **Description:** Simulates pressing a specific keyboard key (e.g., 'Enter', 'Tab', 'a', 'B') with optional modifier keys held down, targeting the application specified by PID, and then traverses its accessibility tree.
+    *   **Parameters:**
+        *   `pid` (Number, Required): The Process ID (PID) of the target application.
+        *   `keyName` (String, Required): The name of the key (e.g., `Return`, `Escape`, `ArrowUp`, `Delete`, `a`, `B`). Case-sensitive for letters if no modifiers are active.
+        *   `modifierFlags` (Array<String>, Optional): An array of modifier keys to hold during the press. Valid values: `CapsLock` (or `Caps`), `Shift`, `Control` (or `Ctrl`), `Option` (or `Opt`, `Alt`), `Command` (or `Cmd`), `Function` (or `Fn`), `NumericPad` (or `Numpad`), `Help`.
+
+5.  **`macos-use_refresh_traversal`**
+    *   **Description:** Only performs the accessibility tree traversal for the specified application (identified by PID). Useful for getting the current UI state without performing an action.
+    *   **Parameters:**
+        *   `pid` (Number, Required): The Process ID (PID) of the application to traverse.
+
+**Common Optional Parameters (for `CallTool`)**
+
+These can potentially be passed in the `arguments` object for any tool call to override default `MacosUseSDK` behavior (refer to `ActionOptions` in the code):
+
+*   `traverseBefore` (Boolean, Optional): Traverse accessibility tree before the primary action.
+*   `traverseAfter` (Boolean, Optional): Traverse accessibility tree after the primary action (usually defaults to true).
+*   `showDiff` (Boolean, Optional): Include a diff between traversals (if applicable).
+*   `onlyVisibleElements` (Boolean, Optional): Limit traversal to visible elements.
+*   `showAnimation` (Boolean, Optional): Show visual feedback animation for actions.
+*   `animationDuration` (Number, Optional): Duration of the feedback animation.
+*   `delayAfterAction` (Number, Optional): Add a delay after performing the action.
+
+## Dependencies
+
+*   `MacosUseSDK` (Assumed local or external Swift package providing macOS control functionality)
+
+## Building and Running
+
+```bash
+# Example build command (adjust as needed, use 'debug' for development)
+swift build -c debug # Or 'release' for production
+
+# Running the server (it communicates via stdin/stdout)
+./.build/debug/mcp-server-macos-use
+```
+
+**Integrating with Clients (Example: Claude Desktop)**
+
+Once built, you need to tell your client application where to find the server executable. For example, to configure Claude Desktop, you might add the following to its configuration:
+
+```json
+{
+    "mcpServers": {
+        "mcp-server-macos-use": {
+            "command": "/path/to/your/project/mcp-server-macos-use/.build/debug/mcp-server-macos-use"
+        }
+    }
+}
+```
+
+*Replace `/path/to/your/project/` with the actual absolute path to your `mcp-server-macos-use` directory.*
+
+## Help
+
+Reach out to matt@mediar.ai
+Discord: m13v_
+
+
+## Plans
+
+Happy to tailor the server for your needs, feel free to open an issue or reach out