autogui 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +20 -0
- data/LICENSE +29 -0
- data/README.md +431 -0
- data/bin/autogui +19 -0
- data/docs/api.md +186 -0
- data/docs/guide.md +243 -0
- data/docs/publishing.md +90 -0
- data/examples/hello.rb +20 -0
- data/examples/locate_demo.rb +16 -0
- data/lib/autogui/exceptions.rb +9 -0
- data/lib/autogui/geometry.rb +98 -0
- data/lib/autogui/image.rb +344 -0
- data/lib/autogui/keys.rb +50 -0
- data/lib/autogui/message_box.rb +125 -0
- data/lib/autogui/platform/darwin.rb +185 -0
- data/lib/autogui/platform/linux.rb +172 -0
- data/lib/autogui/platform/windows.rb +400 -0
- data/lib/autogui/platform.rb +41 -0
- data/lib/autogui/run.rb +168 -0
- data/lib/autogui/screenshot.rb +95 -0
- data/lib/autogui/tween.rb +254 -0
- data/lib/autogui/version.rb +5 -0
- data/lib/autogui/window.rb +170 -0
- data/lib/autogui.rb +738 -0
- metadata +90 -0
data/docs/guide.md
ADDED
|
@@ -0,0 +1,243 @@
|
|
|
1
|
+
# AutoGUI guide
|
|
2
|
+
|
|
3
|
+
This guide expands the README with behavior notes, recipes, and the command mini-language. Method signatures are in [api.md](api.md).
|
|
4
|
+
|
|
5
|
+
## Coordinates
|
|
6
|
+
|
|
7
|
+
The origin `(0, 0)` is the **top-left pixel of the primary monitor**. `x` grows right, `y` grows down. The bottom-right pixel of a 1920×1080 screen is `(1919, 1079)` — `on_screen(1920, 1080)` is false.
|
|
8
|
+
|
|
9
|
+
`position` and `size` return objects that unpack:
|
|
10
|
+
|
|
11
|
+
```ruby
|
|
12
|
+
x, y = AutoGUI.position
|
|
13
|
+
width, height = AutoGUI.size
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
Multi-monitor setups: `size` / `on_screen` / fail-safe corners refer to the **primary** display. Mouse moves may still land on other screens if you pass coordinates that the OS accepts.
|
|
17
|
+
|
|
18
|
+
### Flexible `x, y` arguments
|
|
19
|
+
|
|
20
|
+
These are equivalent:
|
|
21
|
+
|
|
22
|
+
```ruby
|
|
23
|
+
AutoGUI.click(100, 200)
|
|
24
|
+
AutoGUI.click([100, 200])
|
|
25
|
+
AutoGUI.click(AutoGUI::Point.new(100, 200))
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
A four-number sequence or a `Box` clicks the **center**:
|
|
29
|
+
|
|
30
|
+
```ruby
|
|
31
|
+
AutoGUI.click([100, 200, 80, 40])
|
|
32
|
+
AutoGUI.click(AutoGUI.locate_on_screen("ok.png"))
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
A string is treated as an image path:
|
|
36
|
+
|
|
37
|
+
```ruby
|
|
38
|
+
AutoGUI.click("submit.png") # locate on screen, click center
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Pass `nil` to keep one axis:
|
|
42
|
+
|
|
43
|
+
```ruby
|
|
44
|
+
AutoGUI.move_to(nil, 0) # same x, y = 0
|
|
45
|
+
AutoGUI.move_to(0, nil) # x = 0, same y
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
## Fail-safe in practice
|
|
49
|
+
|
|
50
|
+
Fail-safe points are filled in when the library loads:
|
|
51
|
+
|
|
52
|
+
```ruby
|
|
53
|
+
AutoGUI.FAILSAFE_POINTS
|
|
54
|
+
# [Point(0,0), Point(0, height-1), Point(width-1, 0), Point(width-1, height-1)]
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
If you change resolution later, update that list yourself. During a tweened `move_to`, AutoGUI will not trip fail-safe merely because a **step** of the tween sits on a corner — but if **you** slam the real cursor to a corner between steps, the next check raises.
|
|
58
|
+
|
|
59
|
+
`PAUSE` exists so you have time to do that. For debugging, raise it:
|
|
60
|
+
|
|
61
|
+
```ruby
|
|
62
|
+
AutoGUI.PAUSE = 0.5
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
For tight loops, lower it — never to the point you cannot abort:
|
|
66
|
+
|
|
67
|
+
```ruby
|
|
68
|
+
AutoGUI.PAUSE = 0.01
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
## Mouse recipes
|
|
72
|
+
|
|
73
|
+
Smooth move with an easing curve:
|
|
74
|
+
|
|
75
|
+
```ruby
|
|
76
|
+
AutoGUI.move_to(800, 400, duration: 1.0, tween: AutoGUI.method(:easeOutCubic))
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Drag a selection:
|
|
80
|
+
|
|
81
|
+
```ruby
|
|
82
|
+
AutoGUI.move_to(100, 100)
|
|
83
|
+
AutoGUI.drag_to(400, 300, duration: 0.4, button: "left")
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Split a drag into segments (`mouseDownUp: false` skips the extra down/up):
|
|
87
|
+
|
|
88
|
+
```ruby
|
|
89
|
+
AutoGUI.mouse_down
|
|
90
|
+
AutoGUI.drag_to(200, 200, mouseDownUp: false)
|
|
91
|
+
AutoGUI.drag_to(300, 150, mouseDownUp: false)
|
|
92
|
+
AutoGUI.mouse_up
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
Scroll at a point:
|
|
96
|
+
|
|
97
|
+
```ruby
|
|
98
|
+
AutoGUI.scroll(-8, 500, 400) # down, over (500, 400)
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
## Keyboard recipes
|
|
102
|
+
|
|
103
|
+
Type into the focused field, then submit:
|
|
104
|
+
|
|
105
|
+
```ruby
|
|
106
|
+
AutoGUI.click(field_box)
|
|
107
|
+
AutoGUI.hotkey("ctrl", "a")
|
|
108
|
+
AutoGUI.write("new value")
|
|
109
|
+
AutoGUI.press("enter")
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
Select a range with Shift held:
|
|
113
|
+
|
|
114
|
+
```ruby
|
|
115
|
+
AutoGUI.hold("shift") do
|
|
116
|
+
AutoGUI.press("end")
|
|
117
|
+
end
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
`write` of a String sends **characters**. `write` of an Array sends **key names** (so `"left"` is the arrow, not the letters l-e-f-t):
|
|
121
|
+
|
|
122
|
+
```ruby
|
|
123
|
+
AutoGUI.write("Hi!") # H, i, !
|
|
124
|
+
AutoGUI.write(["H", "i", "enter"]) # H, i, Enter
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
`hotkey` presses keys down left-to-right, then up right-to-left, which is what OS shortcuts expect.
|
|
128
|
+
|
|
129
|
+
## Screenshot recipes
|
|
130
|
+
|
|
131
|
+
Full screen to disk and to memory:
|
|
132
|
+
|
|
133
|
+
```ruby
|
|
134
|
+
image = AutoGUI.screenshot("full.png")
|
|
135
|
+
image.width
|
|
136
|
+
image.getpixel(0, 0) # [r, g, b]
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
Grab a region, search inside it:
|
|
140
|
+
|
|
141
|
+
```ruby
|
|
142
|
+
region = [0, 0, 800, 600]
|
|
143
|
+
hay = AutoGUI.screenshot(region: region)
|
|
144
|
+
box = AutoGUI.locate("ok.png", hay)
|
|
145
|
+
# box is relative to the screenshot; offset if you need screen coords:
|
|
146
|
+
if box
|
|
147
|
+
AutoGUI.click(box.left + region[0] + box.width / 2,
|
|
148
|
+
box.top + region[1] + box.height / 2)
|
|
149
|
+
end
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
Wait up to five seconds for a button to appear:
|
|
153
|
+
|
|
154
|
+
```ruby
|
|
155
|
+
box = AutoGUI.locate_on_screen("ready.png", minSearchTime: 5)
|
|
156
|
+
raise "UI never appeared" unless box
|
|
157
|
+
AutoGUI.click(box)
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
Fuzzy match (anti-aliased UI):
|
|
161
|
+
|
|
162
|
+
```ruby
|
|
163
|
+
AutoGUI.locate_on_screen("icon.png", grayscale: true, confidence: 0.92)
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
`confidence` is `1 - mean(|pixel difference|) / 255`. Exact matching (`confidence: nil`) is much faster because whole rows can be compared as strings.
|
|
167
|
+
|
|
168
|
+
Locating is CPU-bound Ruby. Keep needles small and pass `region:` when you can.
|
|
169
|
+
|
|
170
|
+
## Window recipes (Windows)
|
|
171
|
+
|
|
172
|
+
```ruby
|
|
173
|
+
notepad = AutoGUI.get_windows_with_title("Untitled").first
|
|
174
|
+
raise "open Notepad first" unless notepad
|
|
175
|
+
|
|
176
|
+
notepad.activate
|
|
177
|
+
AutoGUI.sleep(0.2)
|
|
178
|
+
notepad.move_to(40, 40)
|
|
179
|
+
notepad.resize_to(640, 480)
|
|
180
|
+
AutoGUI.write("typed into Notepad")
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
`get_windows_with_title` is a case-insensitive substring match. Pass a `Regexp` for tighter control.
|
|
184
|
+
|
|
185
|
+
## Dialogs
|
|
186
|
+
|
|
187
|
+
Use dialogs as breakpoints in unattended scripts:
|
|
188
|
+
|
|
189
|
+
```ruby
|
|
190
|
+
next unless AutoGUI.confirm("Click OK to start clicking") == "OK"
|
|
191
|
+
AutoGUI.click(100, 100)
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
`prompt` / `password` return `nil` when the user cancels.
|
|
195
|
+
|
|
196
|
+
## Mini-language
|
|
197
|
+
|
|
198
|
+
`AutoGUI.run` is a compact way to write short macros. Commands are lowercase. Whitespace is ignored. Strings use **single** quotes.
|
|
199
|
+
|
|
200
|
+
```ruby
|
|
201
|
+
# two clicks, move to 100,200, click, type hi, press enter
|
|
202
|
+
AutoGUI.run("c c g 100, 200 c w'hi' k'enter'")
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
Relative vs absolute: a leading `+` or `-` on **both** numbers means relative.
|
|
206
|
+
|
|
207
|
+
```ruby
|
|
208
|
+
AutoGUI.run("g+20,-10") # move 20 right, 10 up
|
|
209
|
+
AutoGUI.run("g 20, 10") # move to pixel (20, 10)
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
Loops:
|
|
213
|
+
|
|
214
|
+
```ruby
|
|
215
|
+
AutoGUI.run("f5(c s0.2)") # click, sleep 0.2, five times
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
`p` changes `PAUSE` only until `run` returns.
|
|
219
|
+
|
|
220
|
+
Invalid tokens raise `AutoGUI::AutoGUIException` with an index into the string.
|
|
221
|
+
|
|
222
|
+
## Logging screenshots
|
|
223
|
+
|
|
224
|
+
```ruby
|
|
225
|
+
AutoGUI.LOG_SCREENSHOTS = true
|
|
226
|
+
AutoGUI.LOG_SCREENSHOTS_LIMIT = 20
|
|
227
|
+
AutoGUI.click(10, 10)
|
|
228
|
+
# writes a timestamped PNG in the working directory
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
## Mapping from PyAutoGUI
|
|
232
|
+
|
|
233
|
+
| Python | Ruby |
|
|
234
|
+
| --- | --- |
|
|
235
|
+
| `import pyautogui` | `require "autogui"` |
|
|
236
|
+
| `pyautogui.moveTo` | `AutoGUI.move_to` (or `moveTo`) |
|
|
237
|
+
| `with pyautogui.hold("shift"):` | `AutoGUI.hold("shift") { ... }` |
|
|
238
|
+
| `None` | `nil` |
|
|
239
|
+
| `True` / `False` | `true` / `false` |
|
|
240
|
+
| `pyautogui.PAUSE = 0.5` | `AutoGUI.PAUSE = 0.5` |
|
|
241
|
+
| keyword args `duration=1` | `duration: 1` |
|
|
242
|
+
|
|
243
|
+
Return tuples become `Point` / `Size` / `Box` but still unpack.
|
data/docs/publishing.md
ADDED
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
# Publishing AutoGUI to RubyGems
|
|
2
|
+
|
|
3
|
+
The gem name is **`autogui`**. It was unused on [rubygems.org](https://rubygems.org/gems/autogui) at first publication.
|
|
4
|
+
|
|
5
|
+
## One-time setup
|
|
6
|
+
|
|
7
|
+
1. Create an account at [https://rubygems.org](https://rubygems.org) (enable MFA).
|
|
8
|
+
2. Sign in, open **Edit profile → API keys**, create a key with **push** permission.
|
|
9
|
+
3. Store it locally (do not commit this file):
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
mkdir $HOME/.gem
|
|
13
|
+
# Unix
|
|
14
|
+
echo ":rubygems_api_key: YOUR_KEY" > $HOME/.gem/credentials
|
|
15
|
+
chmod 0600 $HOME/.gem/credentials
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
Windows (PowerShell):
|
|
19
|
+
|
|
20
|
+
```
|
|
21
|
+
New-Item -ItemType Directory -Force $env:USERPROFILE\.gem | Out-Null
|
|
22
|
+
Set-Content $env:USERPROFILE\.gem\credentials ":rubygems_api_key: YOUR_KEY"
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
Or paste the key when `gem push` asks.
|
|
26
|
+
|
|
27
|
+
4. Confirm:
|
|
28
|
+
|
|
29
|
+
```
|
|
30
|
+
gem signin
|
|
31
|
+
gem owner autogui
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
(The last command works only after the first successful push.)
|
|
35
|
+
|
|
36
|
+
## Release checklist
|
|
37
|
+
|
|
38
|
+
1. Bump `AutoGUI::VERSION` in `lib/autogui/version.rb`.
|
|
39
|
+
2. Add a section to `CHANGELOG.md`.
|
|
40
|
+
3. Run tests:
|
|
41
|
+
|
|
42
|
+
```
|
|
43
|
+
ruby -Ilib:test test/test_autogui.rb
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
4. Build and inspect:
|
|
47
|
+
|
|
48
|
+
```
|
|
49
|
+
gem build autogui.gemspec
|
|
50
|
+
gem unpack autogui-VERSION.gem --target /tmp/autogui-unpack
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
5. Push:
|
|
54
|
+
|
|
55
|
+
```
|
|
56
|
+
gem push autogui-VERSION.gem
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
6. Tag and push git:
|
|
60
|
+
|
|
61
|
+
```
|
|
62
|
+
git tag -a vVERSION -m "vVERSION"
|
|
63
|
+
git push origin main --tags
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
After that, anyone can install with:
|
|
67
|
+
|
|
68
|
+
```
|
|
69
|
+
gem install autogui
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
```ruby
|
|
73
|
+
# Gemfile
|
|
74
|
+
gem "autogui"
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
## Metadata this gemspec already sets
|
|
78
|
+
|
|
79
|
+
- `homepage`, `source_code_uri`, `bug_tracker_uri`, `changelog_uri`, `documentation_uri`
|
|
80
|
+
- `allowed_push_host` → `https://rubygems.org`
|
|
81
|
+
- `rubygems_mfa_required` → `true` (new versions must be pushed with MFA)
|
|
82
|
+
- `extra_rdoc_files` → README, CHANGELOG, LICENSE, docs
|
|
83
|
+
|
|
84
|
+
## Yanking a bad release
|
|
85
|
+
|
|
86
|
+
```
|
|
87
|
+
gem yank autogui -v VERSION
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
Use yank only for secrets or broken installs; prefer a new patch version for ordinary bugs.
|
data/examples/hello.rb
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
$LOAD_PATH.unshift File.expand_path("../lib", __dir__)
|
|
4
|
+
require "autogui"
|
|
5
|
+
|
|
6
|
+
puts AutoGUI.print_info
|
|
7
|
+
|
|
8
|
+
puts "Screen: #{AutoGUI.size}"
|
|
9
|
+
puts "Mouse: #{AutoGUI.position}"
|
|
10
|
+
puts "On screen (0,0): #{AutoGUI.on_screen(0, 0)}"
|
|
11
|
+
|
|
12
|
+
puts
|
|
13
|
+
puts "In 3 seconds the mouse will move to (200, 200) and back."
|
|
14
|
+
puts "Slam it into a corner to abort."
|
|
15
|
+
AutoGUI.countdown(3)
|
|
16
|
+
|
|
17
|
+
here = AutoGUI.position
|
|
18
|
+
AutoGUI.move_to(200, 200, duration: 0.4)
|
|
19
|
+
AutoGUI.move_to(here.x, here.y, duration: 0.4)
|
|
20
|
+
puts "back at #{AutoGUI.position}"
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
$LOAD_PATH.unshift File.expand_path("../lib", __dir__)
|
|
4
|
+
require "autogui"
|
|
5
|
+
|
|
6
|
+
needle_path = ARGV[0]
|
|
7
|
+
abort "usage: ruby examples/locate_demo.rb path/to/needle.png" unless needle_path
|
|
8
|
+
|
|
9
|
+
puts "Searching for #{needle_path} on screen…"
|
|
10
|
+
box = AutoGUI.locate_on_screen(needle_path, confidence: 0.95)
|
|
11
|
+
if box
|
|
12
|
+
puts "found #{box}"
|
|
13
|
+
puts "center #{AutoGUI.center(box)}"
|
|
14
|
+
else
|
|
15
|
+
puts "not found"
|
|
16
|
+
end
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module AutoGUI
|
|
4
|
+
class Point
|
|
5
|
+
attr_reader :x, :y
|
|
6
|
+
|
|
7
|
+
def initialize(x, y)
|
|
8
|
+
@x = x.to_i
|
|
9
|
+
@y = y.to_i
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
def to_a
|
|
13
|
+
[@x, @y]
|
|
14
|
+
end
|
|
15
|
+
alias to_ary to_a
|
|
16
|
+
|
|
17
|
+
def [](index)
|
|
18
|
+
to_a[index]
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
def ==(other)
|
|
22
|
+
case other
|
|
23
|
+
when Point then @x == other.x && @y == other.y
|
|
24
|
+
when Array then to_a == other.to_a
|
|
25
|
+
else false
|
|
26
|
+
end
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
def to_s
|
|
30
|
+
"Point(x=#{@x}, y=#{@y})"
|
|
31
|
+
end
|
|
32
|
+
alias inspect to_s
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
class Size
|
|
36
|
+
attr_reader :width, :height
|
|
37
|
+
|
|
38
|
+
def initialize(width, height)
|
|
39
|
+
@width = width.to_i
|
|
40
|
+
@height = height.to_i
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
def to_a
|
|
44
|
+
[@width, @height]
|
|
45
|
+
end
|
|
46
|
+
alias to_ary to_a
|
|
47
|
+
|
|
48
|
+
def [](index)
|
|
49
|
+
to_a[index]
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
def ==(other)
|
|
53
|
+
case other
|
|
54
|
+
when Size then @width == other.width && @height == other.height
|
|
55
|
+
when Array then to_a == other.to_a
|
|
56
|
+
else false
|
|
57
|
+
end
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
def to_s
|
|
61
|
+
"Size(width=#{@width}, height=#{@height})"
|
|
62
|
+
end
|
|
63
|
+
alias inspect to_s
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
class Box
|
|
67
|
+
attr_reader :left, :top, :width, :height
|
|
68
|
+
|
|
69
|
+
def initialize(left, top, width, height)
|
|
70
|
+
@left = left.to_i
|
|
71
|
+
@top = top.to_i
|
|
72
|
+
@width = width.to_i
|
|
73
|
+
@height = height.to_i
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
def to_a
|
|
77
|
+
[@left, @top, @width, @height]
|
|
78
|
+
end
|
|
79
|
+
alias to_ary to_a
|
|
80
|
+
|
|
81
|
+
def [](index)
|
|
82
|
+
to_a[index]
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
def ==(other)
|
|
86
|
+
case other
|
|
87
|
+
when Box then to_a == other.to_a
|
|
88
|
+
when Array then to_a == other.to_a
|
|
89
|
+
else false
|
|
90
|
+
end
|
|
91
|
+
end
|
|
92
|
+
|
|
93
|
+
def to_s
|
|
94
|
+
"Box(left=#{@left}, top=#{@top}, width=#{@width}, height=#{@height})"
|
|
95
|
+
end
|
|
96
|
+
alias inspect to_s
|
|
97
|
+
end
|
|
98
|
+
end
|