How to automate windows applications with AutoGui

A quick peek at stack overflow shows that there is a lack of simple tools for windows automation.

Some common tools for windows automation are Sikuli, CodedUI, PywinAuto, and PyAutoGUI. Each of these have their own issues, whether that be complicated syntax for CodedUI, speed for PywinAuto, image recognition for Sikuli, or flexibility for PyAutoGUI.

Introduce AutoGui. An open-source Python and C# package for windows automation that automates using window properties.

Installation

You can install AutoGui via pip.

pip install autogui

Demo

For this example I am going to use the tool RecorderSpy available here. To use the tool, clone the repo and launch the executable.

Getting started with AutoGui is easy. Lets automate the windows calculator.

Launch RecorderSpy and start the recording with Ctrl-R. Record a click on a control with Ctrl-W. When you are finished, end the recording with Ctrl-E.

Generating and running a simple script using RecorderSpy and AutoGui

Note that RecorderSpy is just a generic windows spy tool. You can use any other tools you like. Another good option is inspect.exe that comes bundled with visual studio.

Lets take a look at the generated Python script.

from autogui import *
setWindow("Calculator")
click("id:=num1Button")
click("id:=num2Button")
click("id:=num3Button")

Here we set the active window using the title of the application. Then we select what control to click.

The syntax for the commands is simple, and you can easily write and edit scripts without the recorder.

Lets do another example to show how to write to a control using notepad.

Writing to a control using AutoGui

The script used to generate the automation:

from autogui import *
open("notepad")
click("File")
click("Text Editor")
write("This is AutoGui ","Text Editor")
append(" writing, appending, and sending keys is easy!","Text Editor")
sendkey("{ENTER}")
read("Text Editor")
close()

We opened notepad with autogui’s open command which automatically sets the active window to notepad. Write() sets the value of the control to the entered text. Append() manually sends each key to the control. A list of keys for sendkey is available here. Read() returns the text in a control and can be useful for assertions in GUI testing.

This script was very fast. As is common with GUI automation, you may have to add sleeps to get the desired functionality.

Lets check out the control properties using RecorderSpy.

RecorderSpy shows all the properties that can be used to select an element and can record scripts of actions

Using the control properties, lets make a python script to automate both the calculator and notepad.

from autogui import * 
open("calc")
open("notepad") 
write("this is autogui","Text Editor")
setWindow("calculator")
click("One")
close("Untitled - Notepad")
close()

Lets run the script

Running multiple applications with AutoGui. To switch to a new window use setWindow().

Running multiple applications can be done easily so long as you set which window you want to operate on using setWindow().

FAQ

I can’t find a locator to select the control.

This will happen sometimes. You will have 20 dialog boxes all with the same characteristics. In this case you can match on all 20 dialog boxes and select which one using the child parameter which is set to 0 by default.

autogui.click(“generic dialog box”,5)

The above call selects the fifth dialog box with the title “generic dialog box”

How do I select based on multiple properties?

You can select based on multiple parameters by passing in parameters in the format: param1:=value1,param2:=value2,param3:=value3.

autogui.click(“Editor,class:=EditorClass,id:=EditorID”)

In the above, we match for a control with title of Editor, class of EditorClass, and id of EditorID. We do not need to specify the title property. This is assumed when you do not list the property type.

All windows automation properties are available for selection, although the most useful and common are title/name, class, id, value, and controltype.

I get an exception after opening or closing a program.

This can be the case if you have more than one instance of the program running. You can either remove the second instance, or call open() with the setactive parameter equal to false, then set the active window manually.

The command times out before finding the control

Each method has an option to adjust the timeout of each call which is set to three seconds by default.

How does it work under the hood?

AutoGui is a python port of a c# wrapper for CodedUI which is available as a nuget package as well. AutoGui uses windows automation properties such as title, id, class, value, ect to select controls to act upon. AutoGui searches the active window set using setWindow() or open() for a match on the entered properties.

Using AutoGui for GUI Testing

AutoGui is fully able to be used to test your GUI’s. Each time RecorderSpy runs, it also generates a robot-framework script which is a fantastic test acceptance framework tool.

Source and Further Reading

The project source with additional documentation is available here on my github.