AXElements 0.7.8 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.yardopts +1 -10
- data/README.markdown +7 -14
- data/ext/accessibility/key_coder/key_coder.c +7 -0
- data/lib/AXElements.rb +0 -2
- data/lib/accessibility/core.rb +180 -123
- data/lib/accessibility/dsl.rb +310 -191
- data/lib/accessibility/enumerators.rb +9 -8
- data/lib/accessibility/errors.rb +7 -8
- data/lib/accessibility/factory.rb +16 -9
- data/lib/accessibility/graph.rb +68 -22
- data/lib/accessibility/highlighter.rb +86 -0
- data/lib/accessibility/pp_inspector.rb +4 -4
- data/lib/accessibility/qualifier.rb +11 -9
- data/lib/accessibility/string.rb +12 -4
- data/lib/accessibility/translator.rb +19 -10
- data/lib/accessibility/version.rb +3 -1
- data/lib/accessibility.rb +42 -17
- data/lib/ax/application.rb +90 -30
- data/lib/ax/button.rb +5 -2
- data/lib/ax/element.rb +133 -149
- data/lib/ax/pop_up_button.rb +12 -0
- data/lib/ax/radio_button.rb +5 -2
- data/lib/ax/row.rb +2 -2
- data/lib/ax/static_text.rb +5 -2
- data/lib/ax/systemwide.rb +24 -12
- data/lib/ax_elements/awesome_print.rb +13 -0
- data/lib/ax_elements/exception_workaround.rb +5 -0
- data/lib/ax_elements/nsarray_compat.rb +1 -0
- data/lib/ax_elements.rb +2 -1
- data/lib/minitest/ax_elements.rb +60 -4
- data/lib/mouse.rb +47 -20
- data/lib/rspec/expectations/ax_elements.rb +180 -88
- data/rakelib/doc.rake +7 -0
- data/test/helper.rb +2 -1
- data/test/integration/accessibility/test_dsl.rb +126 -18
- data/test/integration/accessibility/test_errors.rb +1 -1
- data/test/integration/ax/test_element.rb +17 -0
- data/test/integration/minitest/test_ax_elements.rb +33 -38
- data/test/integration/rspec/expectations/test_ax_elements.rb +68 -19
- data/test/sanity/accessibility/test_core.rb +45 -37
- data/test/sanity/accessibility/test_highlighter.rb +56 -0
- data/test/sanity/ax/test_application.rb +8 -0
- data/test/sanity/ax/test_element.rb +7 -3
- data/test/sanity/minitest/test_ax_elements.rb +2 -0
- data/test/sanity/rspec/expectations/test_ax_elements.rb +3 -0
- data/test/sanity/test_accessibility.rb +9 -0
- data/test/sanity/test_mouse.rb +2 -2
- metadata +11 -38
- data/docs/AccessibilityTips.markdown +0 -119
- data/docs/Acting.markdown +0 -340
- data/docs/Debugging.markdown +0 -165
- data/docs/Inspecting.markdown +0 -261
- data/docs/KeyboardEvents.markdown +0 -122
- data/docs/NewBehaviour.markdown +0 -151
- data/docs/Notifications.markdown +0 -271
- data/docs/Searching.markdown +0 -250
- data/docs/TestingExtensions.markdown +0 -52
- data/docs/images/all_the_buttons.jpg +0 -0
- data/docs/images/next_version.png +0 -0
- data/docs/images/ui_hierarchy.dot +0 -34
- data/docs/images/ui_hierarchy.png +0 -0
- data/lib/accessibility/debug.rb +0 -164
- data/test/integration/accessibility/test_debug.rb +0 -44
- data/test/sanity/accessibility/test_debug.rb +0 -63
data/docs/Debugging.markdown
DELETED
@@ -1,165 +0,0 @@
|
|
1
|
-
# Debugging
|
2
|
-
|
3
|
-
This document includes instructions on using AXElements' built in
|
4
|
-
tools to help you debug issues with your scripts, or in cases where
|
5
|
-
you might find a bug in AXElements itself or MacRuby.
|
6
|
-
|
7
|
-
## Visibility
|
8
|
-
|
9
|
-
Sometimes an object that should be visible to accessibility does not
|
10
|
-
show up. Sometimes it will be invisible to the Accessibility Inspector
|
11
|
-
and sometimes it will be invisible to AXElements. This is because the
|
12
|
-
inspector uses hit testing to find objects and AXElements uses the
|
13
|
-
`children` and `parent` attributes (usually). Depending on which part
|
14
|
-
of the accessibility protocol is not being implemented correctly, one
|
15
|
-
or both tools might fail to work.
|
16
|
-
|
17
|
-
Fortunately, failures in the accessibility API are few and far
|
18
|
-
between. The one big,
|
19
|
-
[known issue](http://openradar.appspot.com/6832098) is with menu bar
|
20
|
-
items; you cannot work around this issue without hacking into private
|
21
|
-
Apple APIs. Specifically, you would need to override built in
|
22
|
-
accessibility methods in the class that implement the user interface
|
23
|
-
for NSStatusItem, which is a private class; or you could build your
|
24
|
-
status bar item as an NSMenuExtra, which is another private class. You
|
25
|
-
can find more tips on augmenting accessibility for apps in the
|
26
|
-
{file:docs/AccessibilityTips.markdown Accessibility Tips} document.
|
27
|
-
|
28
|
-
## Trees
|
29
|
-
|
30
|
-
Sometimes you need to see the big picture, the whole UI tree at
|
31
|
-
once or at least be able to see the root of the hierarchy from where
|
32
|
-
you are. For these troubling cases AXElements provides a few tools.
|
33
|
-
|
34
|
-
### Text Tree
|
35
|
-
|
36
|
-
Printing a text tree is similar to how a UI dump works with
|
37
|
-
accessibility on iOS. AXElements does improve a bit on its iOS
|
38
|
-
equivalent. Simply using the Accessibility Inspector does not give you
|
39
|
-
a good picture of the whole tree and with complicated structures it is
|
40
|
-
easy to make mistakes navigating the UI tree.
|
41
|
-
|
42
|
-
The text tree comes formatted for you, and you can simply print it out
|
43
|
-
to the console. The output uses indentation to indicate how far down
|
44
|
-
the tree each element is, and the first element up the tree with one
|
45
|
-
indentation level less will always be the parent for an element.
|
46
|
-
|
47
|
-
Printing out a text tree in the console is very easy. First you generate
|
48
|
-
the text dump and then you print it out:
|
49
|
-
|
50
|
-
puts Accessibility.dump(app)
|
51
|
-
|
52
|
-
This method is useful when you are having difficulties with
|
53
|
-
search. Sometimes when searching you will end up finding the wrong
|
54
|
-
element because your search query was ambiguous and you didn't
|
55
|
-
know at the time. Printing out a dump can often be all the help you
|
56
|
-
need in order to identify your problem.
|
57
|
-
|
58
|
-
However, if the standard `#inspect` isn't doing it for you, you can
|
59
|
-
perform your own inspection every element in a tree yourself using the
|
60
|
-
built in enumerators. Searches use the {Accessibility::BFEnumerator},
|
61
|
-
but the text tree dump method uses {Accessibility::DFEnumerator}.
|
62
|
-
|
63
|
-
### Dot Graph
|
64
|
-
|
65
|
-
For super fancy text trees, AXElements can generate dot graphs for
|
66
|
-
consumption by [Graphviz](http://www.graphviz.org/). In this case, you
|
67
|
-
want to call {Accessibility.graph} and pass the root of the tree you
|
68
|
-
want to have turned into a dot graph; you will get a string back that
|
69
|
-
you will then need to give to Graphviz in order to generate the visual
|
70
|
-
graph.
|
71
|
-
|
72
|
-
File.open('graph.dot', 'w') do |file|
|
73
|
-
app = Accessibility.application_with_name 'Terminal'
|
74
|
-
file.write Accessibility.graph(app.window)
|
75
|
-
end
|
76
|
-
`dot graph.dot -Tpng > graph.png`
|
77
|
-
`open graph.png`
|
78
|
-
|
79
|
-
### Text Tree Won't Print?
|
80
|
-
|
81
|
-
AXElements isn't perfect and it is possible that an edge case slipped
|
82
|
-
between the cracks. However, it's also possible that the problem is
|
83
|
-
actually how accessibility information is being vended out.
|
84
|
-
|
85
|
-
As an example, Radar #10040865 is a ticket that I filed with Apple
|
86
|
-
where they broke accessibility with search fields. The effect to
|
87
|
-
AXElements was that you could not search through the menu bar causing
|
88
|
-
a text dump to fail in a very difficult to trace manner.
|
89
|
-
|
90
|
-
In these cases you will need to use your deductive reasoning to figure
|
91
|
-
out where the problem is coming from. Fortunately, I have provided
|
92
|
-
some tools to help you along the way.
|
93
|
-
|
94
|
-
## All The Way Up
|
95
|
-
|
96
|
-
Sometimes you don't need a whole tree to be printed out. Sometimes
|
97
|
-
just seeing the path from an element to the top level element is
|
98
|
-
enough. {Accessibility.path} is your friend in this case, it will
|
99
|
-
provide you with an array of UI element objects, each successive
|
100
|
-
element will be the parent of the previous element. This is almost
|
101
|
-
like the view that the Accessibility Inspector provides.
|
102
|
-
|
103
|
-
## Custom Exceptions
|
104
|
-
|
105
|
-
AXElements provides some customized exceptions in the OO layer that
|
106
|
-
should help give you much better hints at what went wrong when you
|
107
|
-
have a problem.
|
108
|
-
|
109
|
-
Custom exceptions have been created to help identify the point of
|
110
|
-
failure that would have caused a more cryptic exception to have been
|
111
|
-
raised instead. These custom exceptions also capture more metadata
|
112
|
-
about the problem that occurred which should give good hints as to what
|
113
|
-
went wrong.
|
114
|
-
|
115
|
-
### Search Failures
|
116
|
-
|
117
|
-
An {AX::Element::SearchFailure `SearchFailure`} will occur when you
|
118
|
-
perform an implicit search that fails to find anything.
|
119
|
-
|
120
|
-
In cases where implicit searches are chained, which happens frequently
|
121
|
-
with deep UI hierarchies, if one of the searches were to fail then you
|
122
|
-
would receive a `NoMethodError` about something having
|
123
|
-
failed. Sometimes the failure would happen because the search returned
|
124
|
-
`nil`, and of course `nil` would not respond to another search, though
|
125
|
-
this problem was easy to identify if you are familiar with the
|
126
|
-
AXElements source code; in other cases the failure would occur
|
127
|
-
somewhere in the search {Accessibility::Qualifier qualifier} or in the
|
128
|
-
{Accessibility::BFEnumerator enumerator}, and it was not always clear why.
|
129
|
-
|
130
|
-
The other feature of a search failure is that the exception message
|
131
|
-
will include an element back trace using {Accessibility.path}. This is
|
132
|
-
meant to give a hint about why the search failed.
|
133
|
-
|
134
|
-
### Attribute Not Writable
|
135
|
-
|
136
|
-
You will receive {AX::Element::ReadOnlyAttribute `ReadOnlyAttribute`}
|
137
|
-
exceptions only when you try to set an attribute that is not
|
138
|
-
writable. Again, this was originally designed to more easily identify the
|
139
|
-
point of failure when you try to write to an attribute that you should
|
140
|
-
not write to.
|
141
|
-
|
142
|
-
Specifically, `set_focus` is called by methods internally by
|
143
|
-
AXElements and at one time it was causing some problems when elements
|
144
|
-
were unexpectedly not allowing their `focused` attribute to be
|
145
|
-
written.
|
146
|
-
|
147
|
-
## Logging
|
148
|
-
|
149
|
-
The core level of AXElements has logging in every case that an error
|
150
|
-
code is returned. Though, it can show false positives because of
|
151
|
-
hiccups in the accessibility API or implementation that an app
|
152
|
-
provides; so it turned off by default. This feature is also going away
|
153
|
-
in favour of more intelligent error handling in the core wrapper.
|
154
|
-
|
155
|
-
When weird bugs are occuring, possibly even crashing MacRuby, you
|
156
|
-
should try turning on the logs and then trying the script again. You
|
157
|
-
might see some tell tale logs printed out right before the crash. You
|
158
|
-
can turn on logging right after you load AXElements, like so:
|
159
|
-
|
160
|
-
require 'ax_elements'
|
161
|
-
Accessibility.log.level = Logger::DEBUG
|
162
|
-
|
163
|
-
The standard log levels are available, the full set is available
|
164
|
-
[here](http://rdoc.info/stdlib/logger/1.9.2/Logger/Severity). `Logger::DEBUG`
|
165
|
-
will turn on all logs.
|
data/docs/Inspecting.markdown
DELETED
@@ -1,261 +0,0 @@
|
|
1
|
-
# Inspecting The User Interface
|
2
|
-
|
3
|
-
When it comes to inspecting the user interface there are _many_ nooks
|
4
|
-
and crannies that are interesting to talk about. This document covers
|
5
|
-
the core concepts that you will need to understand before you can
|
6
|
-
begin discovering them yourself.
|
7
|
-
|
8
|
-
## The UI Tree
|
9
|
-
|
10
|
-
The first most important thing to understand is that accessibility
|
11
|
-
exposes the user interface as a hierarchy of user interface
|
12
|
-
tokens. Each token references a GUI element on screen, either
|
13
|
-
something literal like a button, or something structural like a group
|
14
|
-
of buttons. The organization of the hierarchy should make sense to
|
15
|
-
human beings since it is designed to be used by the disabled, so it
|
16
|
-
does not nescesarrily match the underlying organization of views and
|
17
|
-
can be completely arbitrary. That being said, most apps will use the
|
18
|
-
defaults provided by AppKit and "Just Work"™ the way you would
|
19
|
-
expect.
|
20
|
-
|
21
|
-
Throughout this documentation I will refer to tokens as if they were
|
22
|
-
the object themselves unless otherwise noted. It's much easier Each
|
23
|
-
token object knows who its parent object is, and knows about any
|
24
|
-
children objects that it may have. thus creating a tree structure. At
|
25
|
-
least this is the theory, but there are, on occasion, some hiccups
|
26
|
-
since accessibility is a protocol to which multiple parties have to
|
27
|
-
conform.
|
28
|
-
|
29
|
-
A sample hierarchy might start with the application, which has a menu
|
30
|
-
bar as one of its children and the menu bar has menu bar items, each
|
31
|
-
menu bar item has a menu and each menu has menu items; some menu items
|
32
|
-
have another menu as its child which then leads to more menu items and
|
33
|
-
so on. This hierarchy is much easier to understand once visualized:
|
34
|
-
|
35
|
-

|
36
|
-
|
37
|
-
This example is meant to be instructive, the menu bar is one of the
|
38
|
-
more complicated hierarchies to navigate and many other nodes in the
|
39
|
-
hierarchy have been left out. The good news is that AXElements has
|
40
|
-
techniques and patterns for simplifying navigation, so don't get
|
41
|
-
scared off just yet. The point here is that each menu item knows which
|
42
|
-
menu is its parent, and each menu knows which menu item or menu bar
|
43
|
-
item is its parent, and the menu bar knows that which application is
|
44
|
-
its parent. But who is the parent of the application? It turns out
|
45
|
-
that that is a trick question, an application does not have a
|
46
|
-
parent. An application is the entry point for the UI hierarchy, it
|
47
|
-
will be the place where a script usually starts and application
|
48
|
-
objects can be created using the {Accessibility} singleton
|
49
|
-
methods. You can create the object for an application that is already
|
50
|
-
running using {Accessibility.application_with_name} like so:
|
51
|
-
|
52
|
-
app = Accessibility.application_with_name = 'Finder'
|
53
|
-
|
54
|
-
### Accessibility Inspector
|
55
|
-
|
56
|
-
To quickly navigate through the UI tree, Apple has provided a tool,
|
57
|
-
the Accessibility Inspector, as part of the Developer Tools. The
|
58
|
-
inspector will come in handy when you are writing scripts using
|
59
|
-
AXElements, though there are some potential pitfalls that will be
|
60
|
-
discussed later.
|
61
|
-
|
62
|
-
Once you install the Developer Tools, the inspector can be found in
|
63
|
-
`/Developer/Applications/Utilities/Accessibility Tools/`. It is worth
|
64
|
-
playing around with the inspector to get a feel for what the
|
65
|
-
accessibility APIs offer; but keep in mind that the inspector is a
|
66
|
-
dumb interface to the accessibility APIs.
|
67
|
-
|
68
|
-
## Attributes
|
69
|
-
|
70
|
-
Each item in the GUI tree has attributes; buttons have a title,
|
71
|
-
sliders have a value, etc.. Pointers to the parent and chilrden nodes
|
72
|
-
are also attributes. Programmatically, you can get a list of
|
73
|
-
attributes that an object has by asking nicely. AXElements exposes
|
74
|
-
this API via {AX::Element#attributes}. {AX::Element} actually acts as
|
75
|
-
the abstract base class for all objects, encapsulating everything that
|
76
|
-
the accessibility APIs offer.
|
77
|
-
|
78
|
-
### Accessing Attributes
|
79
|
-
|
80
|
-
Every attribute can be accessed as a method of the UI object. The
|
81
|
-
method name will always be the
|
82
|
-
[snake_case](http://en.wikipedia.org/wiki/Letter_case) version of the
|
83
|
-
attribute name without the prefix.
|
84
|
-
|
85
|
-
Some examples:
|
86
|
-
|
87
|
-
- `AXChildren` would become `children`
|
88
|
-
- `AXMainWindow` would become `main_window`
|
89
|
-
- `AXIsApplicationRunning` would become `application_running?`
|
90
|
-
|
91
|
-
The last case is special because we consider "`Is`" to be part of the
|
92
|
-
prefix and the method name has a "`?`" at the end. This is to follow
|
93
|
-
Ruby conventions of putting "`?`" at the end of the method name if the
|
94
|
-
method is a predicate. There will be more details about these rules in
|
95
|
-
other tutorial documents, but this is really something that should be
|
96
|
-
abstracted away. This detail is not hidden right now becaues the
|
97
|
-
Accessibility Inspector does not hide the information and you still
|
98
|
-
need to understand it in order to use the inspector with AXElements.
|
99
|
-
|
100
|
-
#### Example
|
101
|
-
|
102
|
-
We can demonstrate how this all comes together with a small
|
103
|
-
example. In the terminal, you can start up a console session of
|
104
|
-
AXElements by loading `ax_elements` in `macirb` or by navigating to
|
105
|
-
the AXElements repository and running the `console` task if you have a
|
106
|
-
clone. Then you can try the following code, one line at a time:
|
107
|
-
|
108
|
-
app = Accessibility.application_with_name 'Terminal'
|
109
|
-
window = app.main_window
|
110
|
-
title = window.title
|
111
|
-
puts "The window's title is '#{title}'"
|
112
|
-
|
113
|
-
In the first line, we are creating the object for the application. As
|
114
|
-
mentioned earlier, you will usually start navigating the UI tree from
|
115
|
-
the application object. Giving the name of the application is the
|
116
|
-
easiest way to create an application object but requires the
|
117
|
-
application to already be running.
|
118
|
-
|
119
|
-
On the second line we use the application object and ask it for the
|
120
|
-
value of the `main_window` attribute. This will return to us another
|
121
|
-
UI object, this time for the window. You will also notice that the
|
122
|
-
console printed out some extra information about the window, such as
|
123
|
-
the title of the window and its position (in flipped
|
124
|
-
coordinates). Each UI element has implemented the `#inspect` method in
|
125
|
-
such a way as to provide users with a succinct but useful way to
|
126
|
-
identify the UI element on the screen, and `macirb` is designed to
|
127
|
-
happily print that information out for each statement that you enter.
|
128
|
-
|
129
|
-
On the third line, we ask the window for it's `title`, and it gives
|
130
|
-
us back a string which we then print out on the fourth line. Notice
|
131
|
-
that the title of the window was also printed by the console as part
|
132
|
-
of the `#inspect` output that `macirb` asks to print out.
|
133
|
-
|
134
|
-
### Inspect Output
|
135
|
-
|
136
|
-
Using `#inspect` is a great way to see the important details of a UI
|
137
|
-
element, it shows the values of the most important attributes so that
|
138
|
-
you can quickly identify which element it really is on screen, but not
|
139
|
-
so many details that it becomes a pain. A typical example of
|
140
|
-
`#inspect` output looks like this:
|
141
|
-
|
142
|
-
#<AX::StandardWindow "AXElementsTester" (1584.0, 184.0) 17 children focused[✘]>
|
143
|
-
|
144
|
-
That output includes all the pieces that you will normally see from
|
145
|
-
`#inspect`, but you may see less or more depending on the UI element
|
146
|
-
that is being inspected. As is the norm in Ruby, you will always at
|
147
|
-
least get the name of the class; then AXElements will try to include a
|
148
|
-
piece of identifying information such as the `title`, then you have
|
149
|
-
numbers in parentheses which are the screen coordinates for the
|
150
|
-
object, then you have the number of children, and then check boxes for
|
151
|
-
boolean attributes. Aside from the class name, the other pieces of
|
152
|
-
information will only be included if they are relevant, and certain
|
153
|
-
objects will also include other information.
|
154
|
-
|
155
|
-
The values shown by `#inspect` are pieced together using helper
|
156
|
-
methods from the {Accessibility::PPInspector} module. {AX::Element}
|
157
|
-
implements a generic implementation with
|
158
|
-
{AX::Element#inspect}. However, the generic `#inspect` may not always
|
159
|
-
choose the best attributes to show. An example would be
|
160
|
-
{AX::Application#inspect}, which overrides the generic `inspect` so
|
161
|
-
that the process identifier for the application is also included. In
|
162
|
-
other cases, the screen co-ordinates or whether the element is enabled
|
163
|
-
may not be relevant, so you can override the method in the specific
|
164
|
-
subclass to not include those attributes and/or include other
|
165
|
-
attributes. The key idea is to make `#inspect` helpful when exploring
|
166
|
-
a UI through the console or when debugging a script.
|
167
|
-
|
168
|
-
## Accessing Children
|
169
|
-
|
170
|
-
Following first principles shown in the example from above you might
|
171
|
-
be led to believe that in order to navigate around the UI tree you
|
172
|
-
will have to write code that looks something like this:
|
173
|
-
|
174
|
-
app.main_window.children.find do |child|
|
175
|
-
child.class == AX::Button && child.title == 'Add'
|
176
|
-
end
|
177
|
-
|
178
|
-
However, AXElements provides a way to specify what you want that is
|
179
|
-
much more convenient to use. Behold:
|
180
|
-
|
181
|
-
app.main_window.button(title: 'Add')
|
182
|
-
|
183
|
-
This is quite the simplification! If we break it down, you see that
|
184
|
-
the method name is the class of the object you want, and then if you
|
185
|
-
need to be more specific you can pass key-value pairs where the key
|
186
|
-
is an attribute and the value is the expected value. The above example
|
187
|
-
says "find a button with the title of 'Add'".
|
188
|
-
|
189
|
-
You can use as many or as few key-value pairs as you need in order to
|
190
|
-
find the element that you are looking for. If you do not specify any
|
191
|
-
key-value pairs, then the first object with the correct class will be
|
192
|
-
chosen. The {file:docs/Searching.markdown Searching Tutorial} goes
|
193
|
-
into more depth on how key-value pairs are used to specify which
|
194
|
-
object you want.
|
195
|
-
|
196
|
-
## Parameterized Attributes
|
197
|
-
|
198
|
-
There is a special type of attribute that is called the parameterized
|
199
|
-
attribute. The difference from a regular attribute is that you need to
|
200
|
-
supply a parameter. An example of this would look like this:
|
201
|
-
|
202
|
-
static_text.string_for_range CFRange.new(0,5)
|
203
|
-
|
204
|
-
The method name suggests that you need to provide a range and in
|
205
|
-
return you will be given part of the string that corresponds to the
|
206
|
-
range. Of course, this example is quite contrived since string slicing
|
207
|
-
is so trivial in Ruby (but this parameterized attribute actually exists).
|
208
|
-
|
209
|
-
Parameterized attributes are different enough from regular attributes
|
210
|
-
that Apple does not want them mixing together and producing
|
211
|
-
offspring. AXElements is a bit progressive, but still keeps the list
|
212
|
-
of parameterized attributes separate from attributes; you can get a
|
213
|
-
list of parameterized attributes for an object with
|
214
|
-
{AX::Element#param_attributes}. Similarly, you have probably already
|
215
|
-
noticed that parameterized attributes have their own section in the
|
216
|
-
Accessibility Inspector and that many objectss do not have any
|
217
|
-
parameterized attributes.
|
218
|
-
|
219
|
-
In my experience, parameterized attributes have not been very useful,
|
220
|
-
but I haven't looked hard enough and am still looking for a good
|
221
|
-
example to put in this section of the tutorial.
|
222
|
-
|
223
|
-
## Explicit Attribute Access
|
224
|
-
|
225
|
-
In cases where you know what you want is going to be an attribute, you
|
226
|
-
can get better performance from accessing attributes by calling
|
227
|
-
{AX::Element#attribute} and passing the attribute name.
|
228
|
-
|
229
|
-
app.attribute(:main_window)
|
230
|
-
|
231
|
-
Similarly, for parameterized attributes, you need to call
|
232
|
-
{AX::Element#param_attribute} and pass the attribute name and
|
233
|
-
parameter as parameters to that method.
|
234
|
-
|
235
|
-
app.param_attribute(:string_for_range, CFRange.new(0,5))
|
236
|
-
|
237
|
-
These methods are exposed so that other library classes can achieve
|
238
|
-
better performance; but you should avoid using them regularly. These
|
239
|
-
APIs may be hidden in the future in order to enforce the DSL usage.
|
240
|
-
|
241
|
-
## Adding Accessibility Attributes
|
242
|
-
|
243
|
-
You can add custom attributes to objects, or even inject or hide
|
244
|
-
objects from the UI hierarchy. It is simply a matter of
|
245
|
-
overriding/implementing methods from the
|
246
|
-
[NSAccessibility](http://developer.apple.com/library/mac/#documentation/Cocoa/Reference/ApplicationKit/Protocols/NSAccessibility_Protocol/Reference/Reference.html)
|
247
|
-
protocol where needed.
|
248
|
-
|
249
|
-
You should peruse the {file:docs/AccessibilityTips.markdown Accessibility Tips}
|
250
|
-
documentation before making nontrivial changes. There are a couple of
|
251
|
-
guidelines you need to be aware of in order to make sure things remain
|
252
|
-
compatible with AXElements.
|
253
|
-
|
254
|
-
## Next Steps
|
255
|
-
|
256
|
-
You may want to play with what you have learnt so far, see if you can
|
257
|
-
find bugs and then fix them, or perhaps add missing features. ;)
|
258
|
-
|
259
|
-
From here the next logical step would be to figure out how to trigger
|
260
|
-
some sort of action and then inspect the UI for changes; for that
|
261
|
-
topic you should read the {file:docs/Acting.markdown Acting Tutorial}.
|
@@ -1,122 +0,0 @@
|
|
1
|
-
# Keyboard Events
|
2
|
-
|
3
|
-
Keyboard events are a system provided by Apple that allows you to
|
4
|
-
simulate keyboard input. The API for this in the `ApplicationServices`
|
5
|
-
framework, but there is an analogue in the `Acessibility` APIs which
|
6
|
-
has the additional option of directing the input to a specific application.
|
7
|
-
|
8
|
-
Using accessibility actions and setting attributes you can already
|
9
|
-
perform most of the interactions that would be possible with the
|
10
|
-
keyboard simulation. However, there are some things that you will need
|
11
|
-
to, or it will just make more sense to, simulate keyboard input. For
|
12
|
-
example, to make use of hot keys you would have to add extra actions
|
13
|
-
or attributes to a control in the application; that would be more
|
14
|
-
work, possibly prone to error, than simply simulating the hot key from
|
15
|
-
outside the application. In other situations you may specifically want
|
16
|
-
to test out keyboard navigation and so actions would not be a good
|
17
|
-
substitute. It may be that the APIs that AXElements provides for
|
18
|
-
typing just make more sense when writing tests or scripts.
|
19
|
-
|
20
|
-
## Typing with the DSL
|
21
|
-
|
22
|
-
The {Accessibility::DSL} mix-in exposes keyboard events through the
|
23
|
-
`type` method. A simple example would look like this:
|
24
|
-
|
25
|
-
type "Hello, #{ENV['USER']}! How are you today?\n"
|
26
|
-
|
27
|
-
And watch your computer come to life! The `type` command takes an
|
28
|
-
additional optional parameter that we'll get to later. The first
|
29
|
-
parameter is just a string that you want AXElements to type out. How
|
30
|
-
to format the string should be obvious for the most part, but some
|
31
|
-
things like the command key and arrows might not be so obvious.
|
32
|
-
|
33
|
-
## Formatting Strings
|
34
|
-
|
35
|
-
Letters and numbers should be written just as you would for any other
|
36
|
-
string. Any of the standard symbols can also be plainly added to a
|
37
|
-
string that you want to have typed. Here are some examples:
|
38
|
-
|
39
|
-
type "UPPER CASE LETTERS"
|
40
|
-
type "lower case letters"
|
41
|
-
type "1337 message @/\/|) 57|_||=|="
|
42
|
-
type "A proper sentence can be typed out (all at once)."
|
43
|
-
|
44
|
-
### Regular Escape Sequences
|
45
|
-
|
46
|
-
Things like newlines and tabs should be formatted just like they would
|
47
|
-
in a regular string. That is, normal string escape sequences should
|
48
|
-
"just work" with AXElements. Here are some more examples:
|
49
|
-
|
50
|
-
type "Have a bad \b\b\b\b\b good day!"
|
51
|
-
type "First line.\nSecond line."
|
52
|
-
type "I \t like \t to \t use \t tabs \t a \t lot."
|
53
|
-
type "Explicit\sSpaces."
|
54
|
-
|
55
|
-
### Custom Escape Sequences
|
56
|
-
|
57
|
-
Unfortunately, there is no built in escape sequence for deleting to
|
58
|
-
the right or pressing command keys like `F1`. AXElements defines some
|
59
|
-
extra escape sequences in order to easily represent the remaining
|
60
|
-
keys.
|
61
|
-
|
62
|
-
These custom escape sequences __shoud start with two `\` characters__,
|
63
|
-
as in this example:
|
64
|
-
|
65
|
-
type "\\F1"
|
66
|
-
|
67
|
-
A custom escape sequence __should terminate with a space or the end of
|
68
|
-
the string__, as in this example:
|
69
|
-
|
70
|
-
type "\\PAGEDOWN notice the space afterwards\\PAGEUP but not before"
|
71
|
-
|
72
|
-
The full list of supported custom escape sequences is listed in
|
73
|
-
{Accessibility::StringParser::CUSTOM}. Some escapes have an alias,
|
74
|
-
such as the right arrow key which can be escaped as `"\\RIGHT"` or as
|
75
|
-
`"\\->"`.
|
76
|
-
|
77
|
-
### Hot Keys
|
78
|
-
|
79
|
-
To support pressing multiple keys at the same time (i.e. hot keys), you
|
80
|
-
must start with the custom escape sequence for the combination and
|
81
|
-
instead of ending with a space you should put a `+` character to chain
|
82
|
-
the next key. The entire sequence should be ended with a space or
|
83
|
-
nil. Some common examples are opening a file or quitting an
|
84
|
-
application:
|
85
|
-
|
86
|
-
type "\\COMMAND+o"
|
87
|
-
type "\\CONTROL+a Typing at the start of the line"
|
88
|
-
type "\\COMMAND+\\SHIFT+s"
|
89
|
-
|
90
|
-
You might also note that `CMD+SHIFT+s` could also be:
|
91
|
-
|
92
|
-
type "\\COMMAND+S"
|
93
|
-
|
94
|
-
Since a capital `S` will cause the shift key to be held down.
|
95
|
-
|
96
|
-
One caveat with hot keys is that you cannot use `"\\COMMAND+ "` to
|
97
|
-
represent command and space combination, you will need to use
|
98
|
-
`"\\COMMAND+\s"` instead.
|
99
|
-
|
100
|
-
## Protips
|
101
|
-
|
102
|
-
In order make sure that certain sequences of characters are properly
|
103
|
-
escaped, it is recommended to simply always use double quoted
|
104
|
-
strings.
|
105
|
-
|
106
|
-
### Posting To A Specific Application
|
107
|
-
|
108
|
-
The second argument to the `type` command can be an {AX::Application}
|
109
|
-
object. If you do not include the argument, the events will be posted
|
110
|
-
to the system, which usually means the application that currently is
|
111
|
-
active. Note that you cannot be more specific than the application
|
112
|
-
that you want to send the events to, within the application, the
|
113
|
-
control that has keyboard focus will receive the events.
|
114
|
-
|
115
|
-
### Changing Typing Speed
|
116
|
-
|
117
|
-
You can set the typing speed at load time by setting the environment
|
118
|
-
variable `KEY_RATE`. See {Accessibility::Core::KEY\_RATE} for details on
|
119
|
-
possible values. An example of using it would be:
|
120
|
-
|
121
|
-
KEY_RATE=SLOW irb -rubygems -rax_elements
|
122
|
-
KEY_RATE=0.25 rspec gui_spec.rb
|