fletchshot-mcp 0.1.2 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +99 -12
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -67,6 +67,10 @@ No other permission, entitlement, or FletchShot setting is required — and
|
|
|
67
67
|
nothing is configured inside FletchShot's own Settings window for this (see
|
|
68
68
|
FletchShot's help page for why).
|
|
69
69
|
|
|
70
|
+
If the Screen Recording permission is missing, `capture` fails with a message
|
|
71
|
+
telling you to grant it (System Settings → Privacy & Security → Screen
|
|
72
|
+
Recording) to the process that launched this server and relaunch it.
|
|
73
|
+
|
|
70
74
|
## Tools
|
|
71
75
|
|
|
72
76
|
1. **`capture`** — take a screenshot. `mode: "window"` matches a window by
|
|
@@ -75,19 +79,63 @@ FletchShot's help page for why).
|
|
|
75
79
|
the whole screen; `mode: "region"` captures a screen-coordinate rectangle
|
|
76
80
|
after a delay (for menus/popovers, which can't be captured as a standalone
|
|
77
81
|
window — ask the human to open the menu within the delay). Returns the
|
|
78
|
-
file path
|
|
79
|
-
|
|
82
|
+
file path and pixel size of the **original** image (that's the coordinate
|
|
83
|
+
space `open_draft` always uses), plus a downscaled preview (long edge
|
|
84
|
+
capped at 1568px — see "Preview images" below) embedded as the tool's
|
|
85
|
+
image content.
|
|
80
86
|
2. **`open_draft`** — take the path `capture` returned plus a list of simple
|
|
81
|
-
annotations (`arrow`, `text`, `rect`, `ellipse`, `mosaic`, `number
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
(
|
|
87
|
+
annotations (`arrow`, `text`, `rect`, `ellipse`, `mosaic`, `number`) and
|
|
88
|
+
open it in FletchShot as an editable draft. The human adjusts it, then
|
|
89
|
+
presses **Return to agent** (⌘-Return). Coordinates can be given either as
|
|
90
|
+
numeric image pixels (`{x,y}` or `[x,y]`, top-left origin) or as a **text
|
|
91
|
+
anchor** `{text: "some label"}` — see "Text anchors" below. If a text
|
|
92
|
+
anchor doesn't resolve to exactly one match, the call fails with the list
|
|
93
|
+
of strings it did recognize (with `occurrence` numbers) so the agent can
|
|
94
|
+
retry.
|
|
85
95
|
3. **`wait_for_return`** — wait until the human returns the draft (or a
|
|
86
|
-
timeout), then return the finished image
|
|
96
|
+
timeout), then return the finished image (same preview-downscaling as
|
|
97
|
+
`capture`, unless `includeImage: false` is passed, in which case only the
|
|
98
|
+
path and pixel size are returned).
|
|
87
99
|
|
|
88
100
|
Typical flow: `capture` → `open_draft` → (human edits, presses ⌘-Return) →
|
|
89
101
|
`wait_for_return`.
|
|
90
102
|
|
|
103
|
+
### Text anchors
|
|
104
|
+
|
|
105
|
+
Any point (`arrow.from`/`arrow.to`, `text.at`, `number.at`) or region
|
|
106
|
+
(`rect`/`ellipse`/`mosaic`) can be given as `{text: "label", occurrence?: n}`
|
|
107
|
+
instead of numeric coordinates. `open_draft` runs on-device OCR (macOS
|
|
108
|
+
Vision, `VNRecognizeTextRequest`, Japanese + English) over the image to find
|
|
109
|
+
that string and computes the coordinates itself — so if the target has
|
|
110
|
+
visible text, there's no need to zoom into the screenshot, measure pixels, or
|
|
111
|
+
write a script to find its position; just pass the string you can read.
|
|
112
|
+
Targets without text (icons, shapes, …) still need an approximate numeric
|
|
113
|
+
position — that's fine, the human fine-tunes it in FletchShot anyway.
|
|
114
|
+
|
|
115
|
+
- `arrow.to` as a text anchor docks the arrowhead just outside the nearest
|
|
116
|
+
edge of the text (not centered on it, so it doesn't cover the label); if
|
|
117
|
+
`arrow.from` is omitted, it's placed automatically outside the text in
|
|
118
|
+
whichever diagonal direction stays on-screen.
|
|
119
|
+
- `text.at` / `number.at` as a text anchor are placed just next to the target
|
|
120
|
+
(to the right by default) instead of on top of it.
|
|
121
|
+
- `rect` / `ellipse` / `mosaic` as a text anchor become the text's bounding
|
|
122
|
+
box plus a small margin (`margin`, default 12px).
|
|
123
|
+
- If the string matches more than once, the call fails and returns every
|
|
124
|
+
match found with its `occurrence` number; pass that number to disambiguate.
|
|
125
|
+
- For scale: at the default annotation size, text is drawn about 55px tall
|
|
126
|
+
and an arrowhead is about 120px long × 100px wide on a 2x (Retina) capture.
|
|
127
|
+
|
|
128
|
+
### Preview images
|
|
129
|
+
|
|
130
|
+
The images embedded in `capture` and `wait_for_return` are downscaled copies
|
|
131
|
+
(long edge capped at 1568px by default, via macOS `sips`; the original file
|
|
132
|
+
on disk is untouched) — this is what keeps a single screenshot from costing
|
|
133
|
+
tens of thousands of tokens. `open_draft` coordinates are always in the
|
|
134
|
+
**original** image's pixel space (`capture`'s `widthPx`/`heightPx`), never
|
|
135
|
+
the preview's; the tool results also return `previewScaleFactor` (preview
|
|
136
|
+
pixels × `previewScaleFactor` = original pixels) in case you need to reason
|
|
137
|
+
about the preview image itself.
|
|
138
|
+
|
|
91
139
|
## Privacy
|
|
92
140
|
|
|
93
141
|
This server does not use the network at all (stdio transport only). Captured
|
|
@@ -165,21 +213,60 @@ args = ["-y", "fletchshot-mcp"]
|
|
|
165
213
|
これ以外の許可・entitlement・FletchShot側の設定は不要です。FletchShotの設定画面にも、この
|
|
166
214
|
機能のための項目は置いていません(理由はFletchShotのヘルプページを参照してください)。
|
|
167
215
|
|
|
216
|
+
「画面収録」の許可が無い場合、`capture` は許可の付与場所(システム設定 > プライバシーと
|
|
217
|
+
セキュリティ > 画面収録)と、起動元プロセスを再起動する必要がある旨を案内するメッセージで
|
|
218
|
+
失敗します。
|
|
219
|
+
|
|
168
220
|
## ツール
|
|
169
221
|
|
|
170
222
|
1. **`capture`** — 画面を撮る。`mode: "window"` はアプリ名・ウィンドウタイトルの部分一致で
|
|
171
223
|
ウィンドウ単体を撮る(一致なし・複数一致なら撮らずに候補一覧を返す)。`mode: "screen"` は
|
|
172
224
|
画面全体。`mode: "region"` は画面座標の矩形を遅延つきで撮る(メニュー・ポップオーバー用。
|
|
173
|
-
|
|
174
|
-
|
|
225
|
+
ウィンドウ単体では撮れないため、遅延の間に人へメニューを開いてもらう)。戻り値のファイルパス・
|
|
226
|
+
幅高さは**元画像**のもの(open_draftの座標は常にこちらが基準)。埋め込む画像は長辺最大1568px
|
|
227
|
+
に縮小したプレビュー(後述「プレビュー画像」)。
|
|
175
228
|
2. **`open_draft`** — `capture` が返したパスと、簡易な注釈(`arrow`・`text`・`rect`・
|
|
176
|
-
`ellipse`・`mosaic`・`number
|
|
177
|
-
|
|
229
|
+
`ellipse`・`mosaic`・`number`)を受け取り、FletchShotで掴める下書きとして開く。人が直して
|
|
230
|
+
「エージェントに返す」(⌘↩)を押す。座標は数値({x,y}・[x,y]・画像ピクセル・左上原点)か、
|
|
231
|
+
**文字アンカー** `{text: "対象の文字列"}`(後述)のいずれかで指定する。文字アンカーが一意に
|
|
232
|
+
解決できないときは、認識できた候補一覧(occurrence番号つき)とともに失敗するので、
|
|
233
|
+
occurrenceを指定して呼び直せる。
|
|
178
234
|
3. **`wait_for_return`** — 人が下書きを返す(または時間切れになる)まで待ち、仕上がり画像を
|
|
179
|
-
|
|
235
|
+
返す(captureと同様、埋め込みは縮小プレビュー。`includeImage: false` ならパスと幅高さだけ
|
|
236
|
+
返す)。
|
|
180
237
|
|
|
181
238
|
典型的な流れ: `capture` → `open_draft` →(人が直して⌘↩)→ `wait_for_return`。
|
|
182
239
|
|
|
240
|
+
### 文字アンカー
|
|
241
|
+
|
|
242
|
+
点座標(`arrow.from`/`arrow.to`・`text.at`・`number.at`)や範囲(`rect`・`ellipse`・`mosaic`)は、
|
|
243
|
+
数値座標の代わりに `{text: "対象の文字列", occurrence?: n}` で指定できる。`open_draft` は
|
|
244
|
+
画像に対してmacOS Vision(`VNRecognizeTextRequest`、日本語・英語)でオンデバイスの文字認識を
|
|
245
|
+
行い、その文字列を探して座標を自動算出する——つまり対象に文字が写っていれば、画像を拡大して
|
|
246
|
+
画素を確認したり座標を測るスクリプトを書いたりせずに、見えている文字列をそのまま渡せばよい。
|
|
247
|
+
文字の無い対象(アイコンや図形など)はおおまかな数値座標で構わない(人がFletchShot上で微調整
|
|
248
|
+
する前提)。
|
|
249
|
+
|
|
250
|
+
- `arrow.to` を文字アンカーにすると、矢頭は文字の中心ではなく、文字の縁から少し外側(文字を
|
|
251
|
+
隠さない位置)に自動配置する。`arrow.from` を省略すると、画像の端に出ない範囲で対象の外側・
|
|
252
|
+
斜め方向から自動で引く。
|
|
253
|
+
- `text.at`・`number.at` を文字アンカーにすると、対象に重ねず近く(既定で右側)に自動で置く。
|
|
254
|
+
- `rect`・`ellipse`・`mosaic` を文字アンカーにすると、文字の枠に余白(`margin`、既定12px)を
|
|
255
|
+
足した範囲になる。
|
|
256
|
+
- 同じ文字列が複数見つかった場合は失敗し、見つかった候補すべてをoccurrence番号つきで返す。
|
|
257
|
+
その番号を指定して呼び直せばよい。
|
|
258
|
+
- 目安: 既定の注釈サイズでは、文字の高さは約55px、矢頭は長さ約120px・幅約100px(2倍=Retina
|
|
259
|
+
撮影時)。
|
|
260
|
+
|
|
261
|
+
### プレビュー画像
|
|
262
|
+
|
|
263
|
+
`capture`・`wait_for_return` が埋め込む画像は、長辺最大1568px(既定値)に縮小したコピー
|
|
264
|
+
(macOS標準の`sips`を使用。ディスク上の元ファイルは書き換えない)——1枚のスクリーンショットだけで
|
|
265
|
+
数万トークンを消費してしまうのを防ぐための縮小。`open_draft`の座標は常に**元画像**のピクセル
|
|
266
|
+
座標(`capture`が返す`widthPx`/`heightPx`基準)であり、プレビューの座標ではない。戻り値の
|
|
267
|
+
`previewScaleFactor`(プレビュー座標×`previewScaleFactor`=元画像ピクセル)は、プレビュー画像
|
|
268
|
+
そのものについて考える必要があるときのために返している。
|
|
269
|
+
|
|
183
270
|
## プライバシー
|
|
184
271
|
|
|
185
272
|
本サーバーは通信を一切行いません(stdioのみ)。撮影した画像・下書きはディスクの一時ディレクトリへ
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "fletchshot-mcp",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.3",
|
|
4
4
|
"description": "FletchShot (macOS screenshot annotation app) agent draft MCP server: capture a screenshot, open it in FletchShot as an editable draft, and wait for a human to return the finished image.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|