Human Computer Interaction:
Cognition

Mick McQuaid

2024-02-01

Week THREE

Today

• Q and A from last time
• Cognition
• Discussion leading (Ellis Brown)
• Design Critique (Brie Warren)
• Article Presentation (Mick McQuaid on Norman (2023), Part II, Meaningful)
• Figma tutorial (Rachel Sung)

Q and A from last time

AI and portfolio websites

• implications?
• useful?
• unethical?
• equitable?

Kahneman

Why can’t his experiments be replicated? Falsifying data? Unique sample? Changes in society, e.g., social media, computer, technology, cultural change?

Optical illusion

What is the meaning of the red Ys in the optical illusion image?

Limitations of humans

In the context of the limitations of humans, how to address any cultural and religious beliefs that have shaped individuals’ ethical frameworks?

Boundary between AI and Human Designers

UX designers work as a bridge of aesthetics, usability, and usefulness between software engineers and users, and set the whole design concept (UX Book). I’m getting more concerned on the boundary between human designers and AI.

Opinion on Attitudes

“I think people’s attitudes are beginning to move towards two extremes, with some people increasingly seeking inner peace and spiritual abundance, but the other group becoming more utilitarian.”

Reminds me of a NY Times article from this past weekend on intrinsic vs extrinsic values as explaining the Trump phenomenon. Intrinsincally motivated people are immune to Trump, but extrinsically motivated people value eveything about Trump, according to the article.

Opinion on Cognitive Ease

“I’m not sure I agree that the model of cognitive ease applies only to legacy applications. They apply to today’s social media apps as they still require a clear display in terms of design and familiarity, ease of use (for good mood), and in terms of experience. In my opinion, we are still designing with the same input and output feedback in mind.”

Cognition

We talked a bit about cognitive psychology last week, particularly about Tversky and Kahneman’s work. This week we continue with some historical information.

Human cognition and emotion, from Norman (2013) pp 49–55

Levels of emotional design

Visceral level

basic, similar in all people, recoil from hot stove; input is immediate present, output is an affective state; not emotions but precursors to emotions; dismissed by people who don’t believe they are influenced by it;

Behavioral level

learned skills, subconscious response to patterns; overall awareness but no conscious awareness of details, e.g., speaking, sports; conscious of goals while behavioral level handles details; actions are associated with expectations as well as outcomes and lead to affect, both before and after;

Reflective level

conscious cognition, deep understanding, reasoning, slow, guilt, pride, blame, admiration; design takes place at all three levels: high-level cognition can trigger low-level emotion just as low-level emotion can trigger high-level cognition;

Levels of human processing

More on levels of human processing

Another way to think of these levels is illustrated in the previous frame: hardwired or prewired, short-term, and abstract or contemplative. All three levels play a role in our reactions to our environment, including designed artifacts.

Model human processor

Baby bubblehead, aka model human processor

Treat the user like a computer and the user’s work like a program. Break both into cognitive and motor processing components and assess performance time until completion under ideal conditions. That was one of the early approaches to human computer interaction, illustrated in this frame.

A model human processor schema from wikipedia

This frame shows a typical schema used in this early model of human processing, dividing everything into three subsystems. Notice that these three subsystems are a bit less sophisticated than the three levels of emotional design explored in Norman (2005).

Keystroke level model

Keystroke level model includes

• operators, such as key presses, mouse pointing, choosing
• encoding, lists of operators and operands for calculating time
• heuristics, rules to apply to cognitive operators (e.g., choosing)

The model human processor isn’t granular enough, so the same people thought of another model, consisting of operators, encoding methods, and heuristics, illustrated in this frame.

Operators include key presses, mouse pointing, waiting, mentally preparing to operate, and a very few others; extended version includes saccades, retrieve from memory, choose among methods, and a few others.

Encoding includes using the operators with operands to record actions and calculate the length of time it takes to perform those actions.

Heuristics include rules that apply to the cognitive operators, such as “mentally prepare”.

Keystroke model limitations

• error
• learning
• functionality
• recall
• concentration
• fatigue
• acceptability

GOMS

GOMS stands for

• Goals
• Operators (elementary actions)
• Methods (groups of operators)
• Selection rules (to choose methods)

Another major theory discussed in textbooks is the Goals, Operators, Methods, and Selection Rules Model, usually pronounced as a one-syllable word, GOMS, illustrated in this frame.

This model was widely studied from the mide eighties to the mid nineties. The basic idea is that people form goals and subgoals and think in terms of elementary actions called operators. People group these operators together into methods and form selection rules to determine which methods will best achieve goals. Here’s an example to clarify GOMS. Suppose I’m using a word processor and realize I’ve been incorrectly citing Norman’s Emotional Design book when I meant to cite Design of Everyday Things, my goal is to correct the same mistake in several places. Several methods might work, such as using a find and replace function, or looking through the stack of recent changes. I employ selection rules to these methods, such as my comfort level with the find and replace function, or my awareness that every mistake is near the top of the stack of recent changes. Having chosen one of these methods as the best, I employ the operators aggregated by the chosen method. The operators are the specific keystrokes, mouse movements, or menu choices my fingers do automatically.

GOMS fits in well with the notions of bounded rationality and satisficing that were pioneered by Herbert Simon. Like those ideas, it lost some favor as researchers became more interested in studying non expert users and less interested in the idealized human described with GOMS and its variants. As the attention of researchers shifted to topics like emotion and learning, the number of studies related to GOMS waned. Nevertheless, GOMS and its descendants still occupy an important place, especially in large systems where great economies of scale can be achieved, systems that rely on many skilled users, and critical systems like space vehicle launching and nuclear power plant operation.

protocol & verbal analysis

protocol analysis \(\rightarrow\) think-aloud process

Protocol analysis was an early hci tool

The main idea in Ericsson and Simon (1984) is to elicit information from a person they accomplish a task by asking them to think aloud about the information they attend to while solving problems, but not to describe or explain.

Verbal analysis differs in goals from protocol analysis

verbal analysis \(\rightarrow\) knowledge representation

Chi (1997) contrasts protocol analysis with an approach designed to elicit the structure of knowledge of the problem solver, rather than the problem solving process. This leads to a knowledge representation, as illustrated in this frame.

Both analyses lead to maps

protocol analysis \(\rightarrow\) process map

verbal analysis \(\rightarrow\) knowledge map

Both processes result in maps, as seen in this frame, one of a process and one of a knowledge structure.

Hick’s Law

Hick’s law predicts the time it will take for a user to make a choice, given the number of choices.

Hick’s law can be expressed as

\[t = b \log_2 (n + 1)\]

Here, \(t\) is reaction time, \(b\) is a constant to be found empirically, and \(n\) is the number of choices with which the user is confronted. The extra 1 represents the concept none of the above.

This law has influenced the development of menus in computers and the number of choices that are offered in each submenu. The shape of a menu tree is a reflection of Hick’s law.

Fitts’s Law

Fitts’s law was actually discovered by Paul Fitts in the 1950s, but has been applied to the use of mice and other pointing devices as well as screen layouts since. It is perhaps the most widely invoked theory in the world of human computer interaction, and is depicted in the next frame.

Fitts’s law formulation

\[t = a + b \log_2\left(\frac{D}{W} + 1\right)\]

where \(t\) is time, \(a\) is start / stop time in seconds for a given device and \(b\) is the inherent time for a device, \(D\) is distance, and \(W\) is the size of a target. This law, one of the most robust models in HCI, predicts how long (in seconds) it takes an adult to point to an object with a given device. I believe that Fitts’s law is the most important theory in HCI.

Fitts’s law is the basis for the original Apple menu bar and Dock, among other influences. By expanding the target, \(W\), to be the entire width of the display instead of the current window, the developers of the Dock and menu bar hoped to reduce the time it would take for people to point to them. Contrast this with the situation in Microsoft Windows, where the menu bar is particular to a given window. In this situation, a person must exhibit much finer motor control to point to the menu bar target than to point to a menu bar at the edge of the screen, where \(D\) can be any amount without missing the target. In other words, you can push the mouse as hard as you want against the top of the screen: it can’t go any further than the top of the screen.

Readings

Readings last week include Johnson (2020): Ch 1–5

Readings this week include Johnson (2020): Ch 7–9, Norman (2013): Ch 2, 4

Let’s look at the Johnson book together, then the Norman book.

I once worked with a group of junior Naval officers who were required by their commander to be responsible for all the information in their email and military message traffic, some 5,000 items per day. The group got together and devised a strategy of dividing up the material and briefing each other on the points of importance, so that no one person had to read 5,000 items, which would have been physically impossible anyway. You could use a similar divide and conquer strategy on our reading material.

Assignment

Milestone 0: Topic Idea

Can one person from each group report on theirs?

References

Chi, Michelene T. H. 1997. “Quantifying Qualitative Analyses of Verbal Data: A Practical Guide.” The Journal of the Learning Sciences 6 (3): 271–315.

Ericsson, K. Anders, and Herbert A. Simon. 1984. Protocol Analysis: Verbal Reports as Data. Cambridge, MA: MIT Press.

Johnson, Jeff. 2020. Designing with the Mind in Mind, 3rd Edition. Cambridge, MA: Morgan Kaufman.

Norman, Donald A. 2005. Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books.

———. 2013. The Design of Everyday Things, 2nd Edition. Basic Books.

———. 2023. Design for a Better World. Cambridge, MA: MIT Press.

END

Colophon

This slideshow was produced using quarto

Fonts are League Gothic, Lato, and STIX2