AXElements 0.6.0beta1

data/docs/Inspecting.markdown

@@ -0,0 +1,255 @@
+# Inspecting The User Interface
+
+When it comes to inspecting the user interface there are _many_ nooks
+and crannies that are interesting to talk about. This document covers
+the core concepts that you will need to understand before you can
+begin discovering them yourself.
+
+## The UI Tree
+
+The first most important thing to understand is that accessibility
+exposes the user interface as a hierarchy of user interface
+tokens. Each token references a GUI element on screen, either
+something literal like a button, or something more structural like a
+group of buttons. For simplicity, I will refer to tokens as if they were the
+objects themselves.
+
+Each object knows who its parent object is, and knows about any
+children objects that it may have. thus creating a tree structure. At
+least this is the theory, but there are, on occasion, some hiccups
+since accessibility is a protocol to which multiple parties have to
+conform.
+
+A sample hierarchy might start with the application, which has a menu
+bar as one of its children and the menu bar has menu bar items, each
+menu bar item has a menu and each menu has menu items; some menu items
+have another menu as its child which then leads to more menu items and
+so on. This hierarchy is much easier to understand once visualized:
+
+![Example GUI Hierarchy](images/ui_hierarchy.png)
+
+This example is meant to be instructive, the menu bar is one of the
+more complicated hierarchies to navigate and many other nodes in the
+hierarchy have been left out. The good news is that AXElements has
+techniques and patterns for simplifying navigation, so don't get
+scared off just yet. The point here is that each menu item knows which
+menu is its parent, and each menu knows which menu item or menu bar
+item is its parent, and the menu bar knows that which application is
+its parent. But who is the parent of the application? It turns out
+that that is a trick question, an application does not have a
+parent. An application is the entry point for the UI hierarchy, it
+will be the place where a script usually starts and application
+objects can be created using the {Accessibility} singleton
+methods. You can create the object for an application that is already
+running using {Accessibility.application_with_name} like so:
+
+    app = Accessibility.application_with_name = 'Finder'
+
+### Accessibility Inspector
+
+To quickly navigate through the UI tree, Apple has provided a tool,
+the Accessibility Inspector, as part of the Developer Tools. The
+inspector will come in handy when you are writing scripts using
+AXElements, though there are some potential pitfalls that will be
+discussed later.
+
+Once you install the Developer Tools, the inspector can be found in
+`/Developer/Applications/Utilities/Accessibility Tools/`. It is worth
+playing around with the inspector to get a feel for what the
+accessibility APIs offer; but keep in mind that the inspector is a
+dumb interface to the accessibility APIs.
+
+## Attributes
+
+Each item in the GUI tree has attributes; buttons have a title,
+sliders have a value, etc.. Pointers to the parent and chilrden nodes
+are also attributes. Programmatically, you can get a list of
+attributes that an object has by asking nicely. AXElements exposes
+this API via {AX::Element#attributes}. {AX::Element} actually acts as
+the abstract base class for all objects, encapsulating everything that
+the accessibility APIs offer.
+
+### Accessing Attributes
+
+Every attribute can be accessed as a method of the UI object. The
+method name will always be the
+[snake_case](http://en.wikipedia.org/wiki/Letter_case) version of the
+attribute name without the prefix.
+
+Some examples:
+
+- `AXChildren` would become `children`
+- `AXMainWindow` would become `main_window`
+- `AXIsApplicationRunning` would become `application_running?`
+
+The last case is special because we consider "`Is`" to be part of the
+prefix and the method name has a "`?`" at the end. This is to follow
+Ruby conventions of putting "`?`" at the end of the method name if the
+method is a predicate. There will be more details about these rules in
+other tutorial documents, but this is really something that should be
+abstracted away. This detail is not hidden right now becaues the
+Accessibility Inspector does not hide the information and you still
+need to understand it in order to use the inspector with AXElements.
+
+#### Example
+
+We can demonstrate how this all comes together with a small
+example. In the terminal, you can start up a console session of
+AXElements by loading `ax_elements` in `macirb` or by navigating to
+the AXElements repository and running the `console` task if you have a
+clone. Then you can try the following code, one line at a time:
+
+    app    = Accessibility.application_with_name 'Terminal'
+    window = app.main_window
+    title  = window.title
+    puts "The window's title is '#{title}'"
+
+In the first line, we are creating the object for the application. As
+mentioned earlier, you will usually start navigating the UI tree from
+the application object. Giving the name of the application is the
+easiest way to create an application object but requires the
+application to already be running.
+
+On the second line we use the application object and ask it for the
+value of the `main_window` attribute. This will return to us another
+UI object, this time for the window. You will also notice that the
+console printed out some extra information about the window, such as
+the title of the window and its position (in flipped
+coordinates). Each UI element has implemented the `#inspect` method in
+such a way as to provide users with a succinct but useful way to
+identify the UI element on the screen, and `macirb` is designed to
+happily print that information out for each statement that you enter.
+
+On the third line, we ask the window for it's `title`, and it gives
+us back a string which we then print out on the fourth line. Notice
+that the title of the window was also printed by the console as part
+of the `#inspect` output that `macirb` asks to print out.
+
+### Inspect Output
+
+Using `#inspect` is a great way to see the important details of a UI
+element, it shows the values of the most important attributes so that
+you can quickly identify which element it really is on screen, but not
+so many details that it becomes a pain. A typical example of
+`#inspect` output looks like this:
+
+    #<AX::StandardWindow "AXElementsTester" (1584.0, 184.0) 17 children focused[✘]>
+
+That output includes all the pieces that you will normally see from
+`#inspect`, but you may see less or more depending on the UI element
+that is being inspected. As is the norm in Ruby, you will always at
+least get the name of the class; then AXElements will try to include a
+piece of identifying information such as the `title`, then you have
+numbers in parentheses which are the screen coordinates for the
+object, then you have the number of children, and then check boxes for
+boolean attributes. Aside from the class name, the other pieces of
+information will only be included if they are relevant, and certain
+objects will also include other information.
+
+The values shown by `#inspect` are pieced together using helper
+methods from the {Accessibility::PPInspector} module. {AX::Element}
+implements a generic implementation with
+{AX::Element#inspect}. However, the generic `#inspect` may not always
+choose the best attributes to show. An example would be
+{AX::Application#inspect}, which overrides the generic `inspect` so
+that the process identifier for the application is also included. In
+other cases, the screen co-ordinates or whether the element is enabled
+may not be relevant, so you can override the method in the specific
+subclass to not include those attributes and/or include other
+attributes. The key idea is to make `#inspect` helpful when exploring
+a UI through the console or when debugging a script.
+
+## Accessing Children
+
+Following first principles shown in the example from above you might
+be led to believe that in order to navigate around the UI tree you
+will have to write code that looks something like this:
+
+    app.main_window.children.find do |child|
+      child.class == AX::Button && child.title == 'Add'
+    end
+
+However, AXElements provides a way to specify what you want that is
+much more convenient to use. Behold:
+
+    app.main_window.button(title: 'Add')
+
+This is quite the simplification! If we break it down, you see that
+the method name is the class of the object you want, and then if you
+need to be more specific you can pass key-value pairs where the key
+is an attribute and the value is the expected value. The above example
+says "find a button with the title of 'Add'".
+
+You can use as many or as few key-value pairs as you need in order to
+find the element that you are looking for. If you do not specify any
+key-value pairs, then the first object with the correct class will be
+chosen. The {file:docs/Searching.markdown Searching Tutorial} goes
+into more depth on how key-value pairs are used to specify which
+object you want.
+
+## Parameterized Attributes
+
+There is a special type of attribute that is called the parameterized
+attribute. The difference from a regular attribute is that you need to
+supply a parameter. An example of this would look like this:
+
+     static_text.string_for_range CFRange.new(0,5)
+
+The method name suggests that you need to provide a range and in
+return you will be given part of the string that corresponds to the
+range. Of course, this example is quite contrived since string slicing
+is so trivial in Ruby (but this parameterized attribute actually exists).
+
+Parameterized attributes are different enough from regular attributes
+that Apple does not want them mixing together and producing
+offspring. AXElements is a bit progressive, but still keeps the list
+of parameterized attributes separate from attributes; you can get a
+list of parameterized attributes for an object with
+{AX::Element#param_attributes}. Similarly, you have probably already
+noticed that parameterized attributes have their own section in the
+Accessibility Inspector and that many objectss do not have any
+parameterized attributes.
+
+In my experience, parameterized attributes have not been very useful,
+but I haven't looked hard enough and am still looking for a good
+example to put in this section of the tutorial.
+
+## Explicit Attribute Access
+
+In cases where you know what you want is going to be an attribute, you
+can get better performance from accessing attributes by calling
+{AX::Element#attribute} and passing the attribute name.
+
+    app.attribute(:main_window)
+
+Similarly, for parameterized attributes, you need to call
+{AX::Element#param_attribute} and pass the attribute name and
+parameter as parameters to that method.
+
+    app.param_attribute(:string_for_range, CFRange.new(0,5))
+
+These methods are exposed so that other library classes can achieve
+better performance; but you should avoid using them regularly. These
+APIs may be hidden in the future in order to enforce the DSL usage.
+
+## Adding Accessibility Attributes
+
+You can add custom attributes to objects, or even inject or hide
+objects from the UI hierarchy. It is simply a matter of
+overriding/implementing methods from the
+[NSAccessibility](http://developer.apple.com/library/mac/#documentation/Cocoa/Reference/ApplicationKit/Protocols/NSAccessibility_Protocol/Reference/Reference.html)
+protocol where needed.
+
+You should peruse the {file:docs/AccessibilityTips.markdown Accessibility Tips}
+documentation before making nontrivial changes. There are a couple of
+guidelines you need to be aware of in order to make sure things remain
+compatible with AXElements.
+
+## Next Steps
+
+You may want to play with what you have learnt so far, see if you can
+find bugs and then fix them, or perhaps add missing features. ;)
+
+From here the next logical step would be to figure out how to trigger
+some sort of action and then inspect the UI for changes; for that
+topic you should read the {file:docs/Acting.markdown Acting Tutorial}.

data/docs/KeyboardEvents.markdown

@@ -0,0 +1,57 @@
+# Keyboard Events
+
+@todo Still a little bit to do in this doc
+
+In some cases you cannot, or do not want to,  set the value of a field
+directly by assigning to the accessibility object's value
+attribute. In these cases you can post keyboard events to an
+application object.
+
+Other cases for using keyboard input simulation might include keyboard
+navigation or hotkey activation.
+
+The DSL exposes Keyboard events through the `type` keyword. An example
+would look like this:
+
+```ruby
+type "hello, world!", app
+```
+
+In order make sure that certain sequences of characters are properly
+transcribed to the screen you should be consistent about using double
+quotes, `"`,  for literal strings. An escape sequence should begin
+`\\`.
+
+First the keyword `type`, and then the string you want to type
+out. Finally, there is a second argument, which is optional, that
+should be the application that you wish to send the keyboard input
+to; if you do not include the argument, then the input will go to the
+currently activate application.
+
+## Behaviour
+
+The method is asynchronous. It appears to type information in at the
+keyboard repeat rate, but I have yet to check that out.
+
+## Escape Sequences
+
+A number of custom escape sequences have been added in order to allow easy
+encoding of all key sequences/combinations.
+
+<table style="1px solid black">
+<tr><td>Key</td><td>Escape Sequence</td></tr>
+<tr><td>Delete Backwards (Backspace)</td><td>`\b`</td></tr>
+<tr><td>Enter</td><td>`\n` or `\r`</td></tr>
+<tr><td>Control</td><td>`\^`</td></tr>
+<tr><td>Command</td><td>`\C`</td></tr>
+<tr><td>Option/Alt</td><td>`\O`</td></tr>
+<tr><td>Tab</td><td>`\t`</td></tr>
+<tr><td>Escape</td><td>`\e`</td></tr>
+<tr><td>Space</td><td>` `</td></tr>
+<tr><td>Scroll To Top</td><td>`\x01`</td></tr>
+<tr><td>Scroll To Bottom</td><td>`\x04`</td></tr>
+<tr><td>Page Down</td><td>`\f`</td></tr>
+<tr><td>Left Arrow</td><td>`\\<->`</td></tr>
+<tr><td>Right Arrow</td><td>`\\->`</td></tr>
+<tr><td>Down Arrow</td><td>`\DOWN`</td></tr>
+</table>

data/docs/NewBehaviour.markdown

@@ -0,0 +1,151 @@
+# Adding Behaviour
+
+Sometimes it is necessary to add extra methods to a UI
+element. There are a few cases of this in the AXElements source code
+itself, but many more opportunities exist. Unfortunately, extending UI
+element classes in AXElements is not totally straightforward. Some of
+the implementation details need to be understood before you can
+successfully extend AXElements.
+
+## Laziness
+
+In a laziness contest between AXElements and Garfield, AXElements
+wins (I assume). A lot of data that AXElements needs to be processed,
+such as name translation, and the work for this is delayed until it
+needs to be done in order to avoid a very large amount of overhead at
+boot time. Not all parts of AXElements are lazy, at least not yet.
+
+## Class Hierarchy
+
+At run time you will have noticed that you were returned objects which
+have a class like `AX::StandardWindow`, but you can never find the
+definition of the class in the source code. This is because the class
+hierarchy is lazily defined.
+
+### Deciding The Class Name
+
+The first thing to understand is the way that AXElements decides what
+class to instantiate for a UI element. This is actually pretty
+simple. Each UI element has a `role` attribute that is supposed to be
+used by assistive applications (read: AXElements) to understand which
+attributes will likely be available. This is Apple's hint as to what
+kind of class structure to choose.
+
+However, some UI elements also have a `subrole` attribute. For
+instance, `AXStandardWindow` is a subrole attribute that a UI element
+can have if it has a `role` of `AXWindow`. Once again, this is a hint
+that has been built into the system, and AXElements follows these
+hints. Put into object-oriented terms, if an object has a `subrole`,
+then that `subrole` becomes the class for the object and the `role`
+becomes the superclass; if the object does not have a `subrole`, then
+the class of the object will be decided by the `role`.
+
+### Abstract Base
+
+In either case, the {AX::Element} class will be an ancestor for the
+class that is chosen. A class that is its `subrole` will always have a
+superclass that is its `role`, and a class that is a `role` will
+always have {AX::Element} as its superclass.
+
+The advantage to creating this hierarchy is that it becomes much
+easier to implement searches that can find a "kind of" object. For
+instance, you can search for `text_fields` and find `AX::TextField`
+objects as well as `AX::SecureTextField` objects. This is one of the
+more powerful features outlined in the
+{file:docs/Searching.markdown Searching tutorial}.
+
+### Why Lazy?
+
+Laziness was chosen as it makes the library more resilient to custom
+roles and subroles. However, it also allows the MacRuby run time to
+boot faster since it avoids having to load all the different classes
+that would need to be defined.
+
+## Explicitly Defining Classes
+
+Now that you understand how classes are structured it should be very
+obvious how you should name your classes and choose your
+superclass. However, you could also use this technique to customize
+the hierarchy from what Apple has defined. For instance, you could force a
+`AXPopUpButton` to be a subclass of `AXButton` even though Apple has
+declared them to be separate roles. This may or may not be convenient
+depending on what custom methods you wish to add.
+
+## Reasons To Add A Custom Method
+
+For this topic I can only lead by example. Fortunately AXElements has
+a few examples to show off. The full list is in
+`lib/ax_elements/elements`, but I'll go over a few here.
+
+### Application Objects
+
+{AX::Application} has been extended in a couple of ways. First off, in
+order to provide an object oriented interface to sending keyboard
+events I added {AX::Application#type_string}.
+
+However, the big change with {AX::Application} is the merging of
+functionality from `NSRunningApplication`. In order to provide methods
+to set focus to an application I had to cache the
+`NSRunningApplication` instance at initialization and forward some
+method calls to that object. For instance, when you call `#set_focus`
+and pass an application object, AXElements does not actually using
+accessibility to set focus to the application, it uses the
+`NSRunningApplication` class internally still support the
+functionality in a transparent way.
+
+### Overriding `#==`
+
+The most popular customization to make is to overload `#==` for an
+object class that provides a more natural interface, and also one that
+makes search much more flexible. There is more than one example in
+this case, you could look at {AX::StaticText#==} which allows you
+check equality against a string that equal to the `value` attribute
+for the static text object. Similarly, {AX::Button#==} was added to
+check equality with buttons.
+
+### Table Rows
+
+When working with `AX::Table` objects you may have issues identifying
+children that belong to a specific column. Children of table rows, in
+my experience, do not have descriptions identifying which column they
+belong to. In cases where you have no unique identifier for a child,
+such as if you have multiple check boxes, there is no good way to find
+a specific check box using the built in search mechanics.
+
+You could hope that the column order never changes and just use the
+index of the children array but that is fragile; or perhaps you actually
+know what the order of the columns is to begin with and were able to
+keep track of how they changed.
+
+A much more sane way to identify the child is by identifying the
+column that the child belongs to. For instance, a the column for a
+table is an `AX::Column` object that usually has a `header` or `title`
+attribute which will be unique for the table. For this case,
+AXElements includes the {AX::Row#child_in_column} method which
+provides something similar to a search but with the few extra steps
+that would be necessary to correlate the child to the column and then
+return the child that you wanted.
+
+### More
+
+There are likely other cases that I have not come across yet which
+would be significantly simplified by a helper method or the merging of
+functionality from another class. Don't be afraid to share your
+extensions.
+
+## Tests
+
+If you are adding new features to AXElements then you should add tests
+for the new features and also make sure that you don't break existing
+features without realizing it.
+
+Running the test suite is covered in the {file:README.markdown}.
+
+Figuring out the test suite internals may not be easy, there is a bit
+of duplication, and something things need better organization. The
+test suite isn't well documented (on purpose) so you will have to read
+some of the other code to understand how things should work before
+writing your own tests. Be careful not to introduce state dependencies
+between tests or else you will not have a fun time tracking down why a
+certain test seems fail occassionally (which is a problem I had with
+notifications tests).