Seminar on Cognitive Modeling
in Human-Computer Interaction

A research seminar with Prof. Anthony J. Hornof

Computer and Information Science - CIS 607
Spring 2019 - 2 Credits
Thursdays, 10AM-11:20AM
200 Deschutes, University of Oregon

Prof. Hornof's office hours for the term will be Thursdays noon to 1PM, or by appointment.

Overview

This research seminar will discuss current research on computational cognitive modeling specifically as it applies to human-computer interaction (HCI), and background research that explains the theoretical foundation of this area of study. Computational cognitive models, in the context of this seminar, are computer programs that simulate the perceptual, motor, cognitive, and memory processes that a person uses to accomplish a task using a computer. The models are important to the field of HCI because they provide a framework for understanding and ultimately quantitatively predicting aspects of usability including: the time to learn an interface, task execution time, error rates, working memory load during task execution, and eye movements needed to accomplish a task. Though relatively few HCI researchers pursue this sort of research, it is very important within the field of HCI because it develops fundamental scientific theory that is needed within the field.

The class will emphasize the theoretical problem that human performance results from the execution of a changeable cognitive strategy within a set of largely unchangeable perceptual, memory, and motor abilities, which we call the human "architecture". It is difficult to determine whether aspects of observed behavior result from invariants in the architecture or from strategic decisions. This problem is referred to as the "architecture–strategy credit assignment problem." The class will also briefly discuss the practical challenge of making cognitive modeling theory accessible to interface designers in the form of tools that designers can easily and accurately use to evaluate and predict user performance and thus the usability of proposed interface designs. A number of such tools have been developed, and there is generally a trade-off in between ease-of-use and the psychological validity of the models created by such tools.

One highlight of the class is that Andrew Howes from the University of Birmingham will likely visit the class during Weeks 5 and 6. Dr. Howes is the co-author of two of the papers that we will be reading for the class, most notably Howes, Lewis, & Vera (2009), which discusses an important recent advance in the theoritical foundation of cognitive modeling research.

This class relates to Dr. Hornof's research on Multimodal Multitasking.

Evaluation

Each student will pass this course if he or she attends all sessions and submits each week's homework at the start of class. A student will not pass the course if any combination of four classes or homeworks are missed. Medical and approved-in-advance absences are not penalized.

Weekly Homework

Unless otherwise specified, your weekly homework is to actively read each week's paper(s) or chapter(s), taking notes on and responding to what you are reading. This is a great practice to develop as a researcher. You can take your notes in whatever structure and format feels best to you, and your notes can be handwritten or typed, but each summary should include:

Here two good examples: Carswell (1992) and Dourish (2006). Note that they are each very different, but that both satisfy all of the above criteria.

Readings

Week One:
Topics: How a unified theory of cognition can integrate cognitive psychology and provide practical value for HCI.
Reading: Chapters 1 and 2 from Card, S. K., Moran, T. P., & Newell, A. (1983). The Psychology of Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum Associates.

Week Two:
Topics: Mental chronometry, reaction time studies, stage models, speed-accuracy tradeoff.
Reading: Chapter 4 and Chapter 5 from Lachman, R., Lachman, J. L., & Butterfield, E. C. (1979). Cognitive Psychology and Information Processing: An Introduction. Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Week Three:
Topics: A contemporary computational instantiation of a candidate unified theory of cognition and how it is applied to HCI.
Kieras, D. E., & Meyer, D. E. (1997). An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Human-Computer Interaction, 12(4), 391-438.

Week Four:
Topics: Computational cognitive modeling applied to contemporary questions of executive processing in multitasking.
Reading: Meyer, D. E., & Kieras, D. E. (1997). A computational theory of executive cognitive processes and multiple-task performance: Part 1. Basic mechanisms. Psychological Review, 104(1), 3-65.

Week Five:
Topics: A tour de force of theoretical analysis that emphasizes the need to analyze and understand the role of cognitive strategies in computational models of human behavior.
Reading: Howes, A., Lewis, R. L., & Vera, A. (2009). Rational adaptation under task and processing constraints: Implications for testing theories of cognition and action. Psychological Review, 116(4), 717-751.

Week Six:
Topics: A means of having the equivalent of a cognitive strategy emerge in an HCI task by creating a model that uses a Markov Decision Process and an optimization policy.
Reading: Chen, X., Bailly, G., Brumby, D. P., Oulasvirta, A., & Howes, A. (2015, to appear). The emergence of interactive behaviour: A model of rational menu search. To appear in Proceedings of ACM CHI 2015: Conference on Human Factors in Computing Systems, New York: ACM.

Week Seven:
Topics: The role of microstrategies—small interrelated assemblies of perceptual, cognitive, memory, and motor operators—in human performance.
Reading: Gray, W. D., & Boehm-Davis, D. A. (2000). Milliseconds matter: An introduction to microstrategies and to their use in describing and predicting interactive behavior. Journal of Experimental Psychology: Applied, 6(4), 322-335

Week Eight:
Topics: How humans make cognitive-strategic decisions on whether to use perception-and-action versus memory to accomplish a task.
Reading: Gray, W. D., Sims, C. R., Fu, W. -T., & Schoelles, M. J. (2006). The soft constraints hypothesis: A rational analysis approach to resource allocation for interactive behavior. Psychological Review, 113(3), 461-482.

Week Nine:
Topics: An approach to identifying plausible human task strategies for a task by broadly searching through a large space of candidate strategies.
Reading: Zhang, Y., & Hornof, A. J. (2014). Understanding multitasking through strategy exploration and individualized cognitive modeling. Proceedings of ACM CHI 2014: Conference on Human Factors in Computing Systems, 3885-3894.

Week Ten:
Topics: User interface design tools that use cognitive models to predict user performance and usability
Reading: John, B., & Suzuki, S. (2009). Toward cognitive modeling for predicting usability. In J. Jacko (Ed.), Lecture Notes in Computer Science: Vol. 5610. Human-Computer interaction. New trends. Springer Berlin Heidelberg.