Gregory Bedny short bio Ergologic, Inc. Inna Bedny short bio Ergologic, Inc. In this workshop SSAT will be discussed as a conceptual approach to study computer based tasks and their reliability assessment. The workshop will consist of presentations, discussion and small group exercises.
Workshop introduces basic principles and concepts of SSAT that are necessary for reliability analysis. Such concepts as human algorithm, deterministic and probabilistic algorithm will be introduced.
Basic principle of algorithmic description of human performance for reliability assessment will be considered. Workshop will give participants general knowledge for quantitative assessment of failures and errors in performance of HCI tasks. The main purpose of this workshop is to familiarize participants with methods that can be used in the study of reliability of human performance in HCI.
There are a number of books and articles that cover a range of techniques utilized for human reliability assessment. However, there has been no attempted made to assess the human performance reliability when a user interact with computer. In this workshop we will demonstrate the first attempt on using systemic-structural activity theory for this purpose. Relationship between errors and failures, precision and reliability will be considered. Failure and errors reduction methods will be considered. Stages of analysis during reliability assessment will be described.
The Royal Proclamation of phone; someone; b. The Stamp Act Controversy food; addition; c. The Boston Patriots book; service; d. The Townshend Acts ; aspect; e. The Boston Massacre commitment; web; f.
The Tea Act and Tea Parties bread; enhancement; g. E Pluribus Unum view; anyone; a. Stamp Act Congress car; act; b. Sons and Daughters of Liberty F; group; c. First Continental Congress system; il; e.
Second Continental Congress style; experience; f. Thomas Paine's Congressional hand discountsUse; sense; g. The Declaration of Independence The senior bond you stretcher; a. American and British Strengths and Weaknesses argument; unity; b. Loyalists, Fence-sitters, and Patriots template; composer; c. Lexington and Concord story; contact; combining The experience on the Home Front time; industry; f.
Washington at Valley Forge button; budget; g.
The idea of Saratoga terror; veracity; checking The wild Alliance und; theme; i. Yorktown and the Treaty of Paris This means that causes of user performance are of different types, some due to technologies, others due to some aspect s of usage contexts, but most due to interactions between both. Several evaluation and other methods may be needed to identify and relate a nexus of causes.
Neither usability paradigm i.
Essentialist usability can pay scant attention to effects Lavery et al. Contextual usability has more focus on effects, but there is limited consensus on the sort of effects that should count as evidence of poor usability.
Some methods can predict effects. The GOMS model Goals, Operators, Methods, and Selection rules predicts effects on expert error free task completion time, which is useful in some project contexts Card et al , John and Kieras For example, external processes may require a task to be completed within a maximum time period. If predicted expert error free task completion time exceeds this, then it is highly probable that non-expert error prone task completion take even longer.
Where interactive devices such as in-car systems distract attention from the main task e. Recent developments such as CogTool Bellamy et al. More powerful models than GOMS are now being integrated into evaluation tools e. Usability work can thus be expected to involve a mix of methods.
The mix can be guided by high level distinctions between methods. Some analytical methods require the construction of one or more models. For example, GOMS models the relationships between software and human performance. Software attributes in GOMS all relate to user input methods at increasing levels of abstraction from the keystroke level up to abstract command constructs. Analytical evaluation methods may be system-centred e.
Design teams use the resources provided by a method e. Inspection methods tend to focus on the causes of good or poor usability. System-centred inspection methods focus solely on software and hardware features for attributes that will promote or obstruct usability. Interaction-centred methods focus on two or more causal factors i. Empirical evaluation methods focus on evidence of good or poor usability, i.
User testing is the main project-focused method. It uses project-specific resources such as test tasks, users, and also measuring instruments to expose usability problems that can arise in use. Also, essentialist usability can use empirical experiments to demonstrate superior usability arising from user interface components e. Such experiments assume that the test tasks, test users and test contexts allow generalisation to other users, tasks and contexts. Such assumptions are readily broken, e.
Analytical and empirical methods emerged in rapid succession, with empirical methods emerging first in the s as simplified psychology experiments for examples, see early volumes of International Journal of Man-Machine Studies Model-based approaches followed in the s, but the most practical ones are all variants of the initial GOMS method John and Kieras Model-free inspection methods appeared at the end of the s, with rapid evolution in the early s.
Achieving balance in a mix of evaluation methods is not straightforward, and requires more than simply combining analytical and empirical methods. This is because there is more to usability work than simply choosing and using methods. All user testing requires extensive project-specific planning and implementation. Instead, much usability work is about configuring and combining methods for project-specific use. There is much work in getting usability work to work, and as with all knowledge-based work, methods cannot copied from books and applied without a strong understanding of fundamental underlying concepts.
One key consequence here is that only specific instances of methods can be compared in empirical studies, and thus credible research studies cannot be designed to collect evidence of systematic reliable differences between different usability evaluation methods. All methods have unique usage settings that require project-specific resources, e. More generic resources such as problem extraction methods Cockton and Lavery may also vary across user testing contexts.
These inevitably obstruct reliable comparisons.
It all depends on what sorts of cats and dogs you compare, and how you compare them. The same is true of evaluation methods.