Designing usable software

One of the fundamental needs when designing usable software is to know your user. If you don't know your user, then you don't know what you should be building -- you can only make educated guesses.

Commercial software companies employ numerous methods to understand their users and their needs. They run focus groups, conduct surveys, run user studies, look at marketing data, instrument their software to collect information about how the software is used... There are many methods to understand users' needs, but they all hinge on collecting data.

Usability in the Open Source Community

For volunteer-driven open source projects, real-world user data is much harder to come by -- there simply aren't easy ways to collect data about how users use the software in practice. To be sure, there are some avenues for collecting this data: Bug tracking software, mailing lists, and forums all provide outlets for users to describe usability problems and to indicate desired features. But only a tiny fraction of actual users take the time to contribute via these channels. As a result, information about how the community regularly uses the software is generally unknown for all open source projects.

Enter ingimp and StatsJam

ingimp is an instrumented version of the GIMP designed to gather usage data, or data characterizing how the software is used on a day-to-day basis, by real-world users. ingimp is not a plug-in, but a complete, standalone application that looks and operates just like the regular version of GIMP, except that it logs how you use the software. Among other things, ingimp logs the commands you use, characteristics of your documents (number of layers, image width/height, etc.), and activity tags -- short descriptions that you can provide to describe how you will use the software. When the application quits, the usage data is automatically sent to this website.

Data collected by ingimp

ingimp automatically logs three types of information: command use, document characteristics, and interaction data.

ingimp records the names of commands used (e.g., "Paintbrush"), but not their parameters. For example, ingimp records when you use the text tool, but not the text you use. Similarly, it records that you use the Paintbrush, but not the colors used nor where you paint. It also records the names of scripts, so if you have custom scripts you have created, their names will be recorded for anyone to see.

Commands provide a picture of what you are doing, but don't tell us anything about the nature of documents on which you are working. Thus, ingimp logs general descriptions of the documents edited: Their size, number of layers, image histograms, and so on. These data give an indication of whether people generally work on large or small images, complex or simple images, and photographic images or graphic illustrations.

Finally, ingimp logs interaction data, or information regarding how you use the application in general. ingimp records keyboard and mouse activity, but not the actual keys or mouse location. That's worth repeating: ingimp does not record what keys are pressed nor where clicks, drags, or pointer movements happen when using a pointing device like a mouse or stylus. ingimp also records the sizes and locations of windows, when they gain "focus" (i.e., when they are in the front), the platform you use it on (e.g., Windows or Linux), the size(s) of your monitor(s), among other data.

For a more complete list of data collected, please see the consent agreement.

What the data can tell us

Collecting data is only half the story (and is actually the easy part). Making sense of that data is how a project like this truly demonstrates its value. Accordingly, we have a number data summaries available that, for the first time for any open source project, richly characterize how hundreds of users regularly use an end-user, open source application (specifically, GIMP).

In the spirit of open source software, anyone can comment on, edit, or create their own data summaries using our StatsJam software, a modified version of MediaWiki that enables anyone to participate in data analysis.

ingimp can provide answers to a number of questions, including:

• When it is used during the day, how frequently, and for how long
• How many images a user has open at once, and the number of layers, height, width and overall "character" of those images (where the "character" is provided by familiar image histograms)
• What commands people use and which commands are most frequently undone
• The operating system of users, the number of monitors attached, and the monitors' resolutions
• How often document windows lose focus to other GIMP windows

There are many more ways one can characterize ingimp's use. Go to ingimp's StatsJam to see more analyses, comment on analyses, or contribute your own.

Privacy matters

ingimp automatically collects usability data, but only after getting your permission to do so. Because all data collected by ingimp is made publicly available on this website, you should be aware of what data are collected and implications for your privacy. Additionally, please make sure to read the consent agreement when you run ingimp for the first time.

Protecting privacy

We have designed ingimp to collect data useful to usability analyses, without intentionally collecting any personal information. For example, ingimp does not record the actual keys pressed, just when they are pressed, along with any modifier keys used (i.e., Shift, Alt, and Control). Furthermore, ingimp does not record command parameters (for example, it doesn't record the text you enter into the text tool) nor does it transmit the actual images on which you work.

We have taken a number of measures to make data collection process transparent. First, you can temporarily disable logging for any individual run of ingimp by checking the "Disable application usage logging for this run" checkbox at start-up. Note that you must do this every time you wish to disable logging.

Check the above box every time you wish to disable logging

Second, ingimp logs its data to human-readable XML files. For example, here are the log snippets corresponding to someone typing "ingimp" into the text tool's text editor:

<LogEntry entry_num="581" event_date="2007_05_03_11_52_55"

elapsed_time_usec="1140515625.000000">
<WindowKeyEvent window_name="GIMP Text Editor" window_role="gimp-text-editor"
event_type="9" modifier_state="0"/>

</LogEntry>

<LogEntry entry_num="583" event_date="2007_05_03_11_52_55"
elapsed_time_usec="1140656250.000000">
<WindowKeyEvent window_name="GIMP Text Editor" window_role="gimp-text-editor"

event_type="9" modifier_state="0"/>
</LogEntry>

<LogEntry entry_num="584" event_date="2007_05_03_11_52_55"

elapsed_time_usec="1140812500.000000">
<WindowKeyEvent window_name="GIMP Text Editor" window_role="gimp-text-editor"
event_type="9" modifier_state="0"/>

</LogEntry>

<LogEntry entry_num="585" event_date="2007_05_03_11_52_56"
elapsed_time_usec="1141234375.000000">
<WindowKeyEvent window_name="GIMP Text Editor" window_role="gimp-text-editor"

event_type="9" modifier_state="0"/>
</LogEntry>

<LogEntry entry_num="586" event_date="2007_05_03_11_52_56"

elapsed_time_usec="1141375000.000000">
<WindowKeyEvent window_name="GIMP Text Editor" window_role="gimp-text-editor"
event_type="9" modifier_state="0"/>

</LogEntry>

<LogEntry entry_num="587" event_date="2007_05_03_11_52_56"
elapsed_time_usec="1141531250.000000">
<WindowKeyEvent window_name="GIMP Text Editor" window_role="gimp-text-editor"

event_type="9" modifier_state="0"/>
</LogEntry>

Note that the log shows 6 keyboard events, but none of the actual keys are logged. (Entry number 582, the undo event, is not shown above.) A more complete log file can be found here: sample_log.xml.

Finally, since it is open source, you have the additional peace of mind that you can directly inspect all of the changes we have made to the GIMP within ingimp. For example, check out the patch in Downloads to see exactly how ingimp differs from the stock distribution.

What you should be aware of

At present, there are a few ways that potentially personal information could inadvertently be collected by ingimp:

• If you create your own custom scripts, your script names will be recorded because all command names are recorded. Thus, if you named your script "john_smiths_acme_script", that script name would be recorded whenever you used it
• When you first start ingimp, you have the opportunity to enter "activity tags" (free-form descriptions of how you plan on using the software -- see figure below). These are purely optional. You are free to enter any text in the activity tag field, but should remember that these tags will be available for anyone to see in your log file. Thus, if you enter, "Removing Aunt Edna from John Smith's family photo" in the Activity tags field, this text will be available for anyone to read in the uploaded log files

Any text added to the "Activity tags" field is recorded in the log file

If you have any additional questions about what data is collected, feel free to send us an email at ingimp@cs.uwaterloo.ca.

Who maintains ingimp?

ingimp is part of human-computer interaction (HCI) research at the University of Waterloo investigating new forms of sustainable open usability. In particular, our goal is to research new tools that assist open source projects in their efforts to make more usable software, without creating significant new overhead to end-users, developers, or other project members. It is being run by Professor Michael Terry.

If you have any additional questions about what data is collected, feel free to send us an email at ingimp@cs.uwaterloo.ca.