A.Hornof -- 10/17/07
CIS 443/543, Fall 2007

Project #4
Design and Evaluate a System

This project is due at the start of class on 10/31/07.

The purpose of this assignment is for you to learn the process of designing a user interface that is useful, effective, and efficient, and easy and satisfying to use. We will follow a scenario-based design process. This design phase will be followed by implementation in Project #5 and evaluation in Project #6.

There are three main activities in this design project: Generate design alternatives. Evaluate the designs. Decide on a final design.

You should read Chapters 4 - 7 of the Rosson and Carroll textbook as part of working on this assignment. You may wish to consult Rosson and Carroll's online case study library for ideas. Note that some of the case studies are more complete than others. Use any of the ideas you find at the site, provided that you don't design the same system, and provided that you do give credit where appropriate.

Identify Two Activity Scenarios

Start by reading all of the Project 3 assignments generated for Project 3. Identify two activity scenarios that your design will build on. These can come from any student's Project 3. Clarify which ideas you are starting with. You may need to write one or two new activity scenarios.

Design

Design at least two different systems to do the same set of tasks for the same set of users. These will be your two design alternatives. Use your observations and analyses from the requirements analysis, including your stakeholder profiles and problem scenarios, to motivate the designs. You will design activities, information, and interaction.

The activity design may be start out the same for the two alternative systems, or you may develop two different activity designs. Either way, the activity design will split for the two systems after the second step. The information and interaction design, however, will need to be substantially different between the two systems.

Each of the two alternative system designs does not need to be practical to implement, but it is probably good if at least one of them is because you will have to build something. But try to think abstractly about the problems and solutions independent of the technology. Don't jump from the problem straight to talking about GUI components. The point of this exercise is to stop in between, and to identify the activities, information, and interactions independent of the technological details.

Information Design

Determine the information that the user will need at each point in the two activity scenarios. First discuss the information in the context of the scenarios in paragraph form, as is done on pp.145-146. Then provide different representations of how this information might be conveyed to the user. If there are visual components, make different hand-drawn sketches that show how the information might be presented in each alternative design. Each sketch should be clearly annotated and discussed in the body of the report.

Interaction Design

Describe the interaction of each of the two designs by creating a storyboard, paper-based prototype, annotated series of hand-drawn screenshots, or other artifact that shows how a user would interact with the system in the context of the two activity scenarios. Note that the artifact does not necessarily confine you to a GUI system, but could capture the interaction between a user and many different types of systems. A storyboard can be written in a "cartoon strip" style provided that it successfully captures the interaction between the user and the system, including all of the user actions and system responses. Whatever artifact you create should successfully convey the user's goals and actions, and the system's responses, at every step of the exchange.

Fourth, conduct a claims analysis in which you hypothesize the pros (+) and cons (-) of each of the design features of the features proposed in any two of the activity scenarios. Follow Rosson and Carroll's example on p.186, and find at least one pro and one con for each feature.

Formative Evaluation

Build a prototype of each of the two proposed designs. A paper-based prototype is probably best. Recruit four real-world participants and have them test your two proposed systems by using the "think aloud" protocol while they try to do a few specific tasks. Present the test users with a set of tasks and ask them to "use" your system to do the tasks. You will have to be organized and ready with the system's response after every user input. If it is a standard GUI, for example, you could write a little number next to each button, and have a set of numbered screenshots that you pull out of a folder and present to the user after each input.

Follow the Guidelines for Conducting User Observations (by Apple Computer) except that by necessity there will have to be substantial interaction as you play the role of the system. Most importantly, remind the participants that (a) they can quit any time they want and (b) "we are testing the system, not you." When running the study, try to avoid discussions about the design of the system, but focus instead on how the participant would use the system to do the task. To compensate for any practice effects, vary the order in which you present each system to the different users.

One group member should "run" the meeting and do most of the talking with the user. Another might play the role of the computer, running the simulation. The other members should silently observe and take notes about what works, and what does not work. Optionally, videotape the study so that you can study it in more detail later. Write a Formative Evaluation Report that documents what did and did not work in each design, which design seemed to be better overall, and why. Propose modifications to each design based on the report.

Design Decisions

As a group, discuss the results of the formative evaluation. Identify a set of activities that will be the focus of the project. Identify a single design that you will build. Rather than just selecting one of the two alternative designs, perhaps incorporate the best parts from the two designs into one.

Deliverables

You should have a section for each boldface and italicized phrase that follows. The number of pages listed for each section is meant as a rough guideline for a single-spaced manuscript.

Root concept: (1 page): Identify a vision statement and rationale for the project. This section should include a Problem Statement of the human to be solved, independent of the system that you will build, and then a Proposed Solution--a very general description of the system you will build.
Information Design
Interaction Design
Formative Evaluation Study: (2 pages) This should discuss how the study was conducted, including a description of the participants. This section should also include the prototype used in the formative evaluation.
Formative Evaluation Report (2-3 pages)
Design Decisions (1 page)
Work Distribution (1 page)

[New for 2008] The document that you turn in must have (a) page numbers that run continuously from 1 to n with no interruptions or overlap in page numbers and (b) a table of contents that points the reader to the sections listed above. The reason for this important requirement is to specifically encourage groups to assemble all of the components a day in advance, and to have everyone in the group read the document before it is turned in. Everyone in the group is responsible for everything discussed in the syllabus "Good Writing" section.

Work Distribution

You can split up the assignment among the various group members, or you can take turns revising drafts, or whatever you like. It's up to you. But briefly note who did what, and how the work was distributed, at the end of the assignment.

Each of you should separately turn in a Group Member Evaluation in which you evaluate how you and your teammate contributed to the project. These should be turned in separately.

Your entire submission should be stapled together, and the top of the first page should include the student(s) names, the date, and the name of the class.

Grading criteria

Is there a clear motivation for the project? Does the design process generate and evaluate interesting alternatives? Is the information and interaction of the proposed system clearly defined? Does the formative evaluation study inform the design? Does the proposed system solve an interesting problem in an interesting way? Is the paper complete and well-written? [New for 2008] Are page numbers and a table of contents provided as specified?

Acknowledgments

This assignment is derived in part from a project designed by Susan Palmiter at Portland State University.