A student participates in an eye tracking experiment at the
University of Oregon Cognitive Modeling and Eye Tracking Lab.
Scroll down on this web page for more photos that capture the scenario.
A student participates in an eye tracking experiment at the
University of Oregon Cognitive Modeling and Eye Tracking Lab.
Scroll down on this web page for more photos that capture the scenario.
You can download a (5MB) PDF with instructions for how to build the chinrest shown in the photo above.
By Ishwinder Kaur
Friday, October 3rd 2003
In the beautiful coastal state of Oregon, in the quaint, quiet, university town of Eugene, there are a group of individuals who are intensely interested in finding out exactly how a human's visual perception and cognition processes work. These people are busily engaged in a cycle of reading, research and experimentation by which they hope to understand and consequently predict where you, as a human, would next focus your eyes, how you would interpret what you see and how you would react to visual stimuli in the world around you. The task is momentous, and so is the passion to accomplish it.
This is a story about the very same people. We take but a glimpse at but a part of their amazing work. We talk about a researcher who is conducting an experiment, in which a camera tracks or records the eye movement of a person engaged in a predetermined visual interaction with a computer. The experiment is about trying to figure out and verify cognitive theories about the human visual and oculomotor processes and is a reasonably complex and a very intriguing process. The eye-tracking experiment takes place in the Cognitive Modeling and Eye-Tracking (CMET) Lab. at the Deschutes Hall, which houses the Computer and Information Science (CIS) Department at the University of Oregon .
Tim Halverson: Tim is the person who carries out the experiment. He is a student at the CIS Department, has been involved with his graduate studies here for the past two years and has been working in the areas of cognitive-modeling and eye tracking for quite some time. Tim sets the goals, procedure and expectations of this experiment. He then assembles all the resources he needs and carries out his observational study to completion.
Anthony Hornof: A faculty at the CIS Department, Dr Hornof is Tim's adviser, guide and mentor. It is under his direction and supervision that Tim is carrying out his study and experimentation. He is deeply involved with the cognitive modeling and eye tracking areas and the outcome of the experiments matters a lot to him, and it is his energy and enthusiasm that paints the experiment in vibrant colors.
The Participant: The participant (henceforth referred to as P) has vested interests in the outcome of the experiment, as you shall subsequently see. For the purposes of capturing this scenario, Smita K-------, flatmate of the author, and also a computer science student, was cajoled and recruited, in spite of the fact that the previously mentioned vested interests didn't apply in her case. This was done because the actual participants are not to be photographed so as not to divulge their identity.
This is a technology-intensive experiment. A lot of high-tech and low-tech devices, widgets and gizmos form an integral part of the entire procedure, such as paper, the eye-tracker, computers, displays, chin-rest, eye testing equipment, video mixer etcetera. These are described in detail as and when the technology is encountered in the scenario.
The hierarchical task analysis (HTA) is largely derived from an artifact in the scenario that Tim prepared. In the list format, it is as follows:
1. To do before participant arrives:
a. Write the participant number and date on every document.
b. Setup "user environment"
i. Place that the proper chair is in front of the participant station.
ii. Mount the chin-rest.
iii. Remove all distractions from the participant station.
iv. Block the windows.
v. Unplug phone and turn answering machine volume down.
c. PC Initial Setup
i. Set the two monitors to "cloning" mode
ii. Ensure screen resolution is 1280x1024 @ 75Hz
iii. Disable Norton Firewall
iv. Disable Norton Anti-Virus
v. Ensure that the Power Options control panel is set to "Always On" power scheme.
vi. Connect Ethernet to Mac
vii. Set TCP/IP address to 1.0.0.2
d. Mac Setup
i. Quit all applications.
ii. Hide and minimize dock.
iii. Ensure mouse tracking speed is in the middle position (fourth from the right).
iv. Ensure that there is no tension of the mouse cable.
v. Set monitor switch-box to Mac
vi. Turn on mirroring.
vii. Ensure screen resolution is 1280x1024 @ 75Hz
viii. Ensure that Apple Menu > Location is set to "Eyegaze."
ix. Input data file
x. Open the practice software.
e. Switcher Setup
f. VTR Setup
2. To do when participant arrives
a. Place "Quiet" sign in the hallway.
b. Present consent form.
c. Present pre-experiment questionnaire.
d. Brief walk through of the lab and equipment.
e. Eye exams
i. Stereovision test (just the butterfly)
ii. Color test
iii. Acuity test - Using 16" card, ensure that acuity is 20/32 or better at 24"
iv. Dominant eye test
f. Present experiment instructions
g. Fine-tune "user environment"
i. Adjust chair.
ii. Adjust monitor/chin-rest height using electric switch
iii. Adjust the monitor height so that the participant's eyes are level with the location of the experiment precue .
iv. Adjust the camera
v. Adjust keyboard shelf
3. Run Experiment
a. Practice
i. Allow participant to perform as many trials in practice software
ii. Quit practice software
b. First experiment
i. Launch the first experiment application, as determined by the counterbalance document.
ii. Enable eye tracking
iii. Run the experiment (cmd-R)
iv. Monitor the initial eye tracker calibration.
v. Quit first experiment software on Mac.
c. Second experiment
i. Launch the second experiment application, as determined by the counterbalance document.
ii. Enable eye tracking
iii. Run the experiment (cmd-R)
iv. Monitor the initial eye tracker calibration.
v. Quit second experiment software on Mac.
4. Do during experiment
a. Occasionally ensure that the video is still good.
b. Monitor any automated re-calibration that take place.
5. Do after experiment
a. Administer post-experiment questionnaire
b. Debrief
c. Pay and have participant sign pay form
d. Wrap-up
i. Take down "Quit" sound.
ii. Input data file
iii. Cleaning
1. Stereovision glasses
2. Chinrest
iv. Quit EgServer on PC
The Hierarchical Task Analysis can be depicted in the following diagrams:
Tim and Anthony hope that the experiment would go well, without any hitches and that it would in some way contribute to a better understanding of the human cognitive process which would help them in modeling the same. But they know that problems do happen: sometimes participants don't show up, some of them don't calibrate well in coordination with the eye tracker, some wear contact lens which might be a problem, and so on. But in spite of these, the hope and expectation of a successful experiment is strong.
The P has much different expectations from the experiment. He or She expects to be monetarily remunerated for the same (participants usually make around $20 for one sitting) as well as, maybe, to get to know a bit more about the research at Anthony Hornof's CMET Lab. Although in this specific case, Smita is doing it purely out of goodwill for her room mate who is a PhD student and has just joined the CMET Lab and doubles as the photojournalist of this essay.
This is the best part. Now we get to know how Tim actually conducts his experiment.
As people who have had some experience with setting up and conducting experiments would realize, an experiment is not just an experiment. It starts off with a massive amount of background work including reading the existing research, identifying areas to experiment on, deciding on parameters, designing the experiment, coding for it, troubleshooting it and, once you're done with all this, other things like the Human Subject Compliance regulations. So following this winding path uphill, Tim and Anthony finally decide that they are at a stage where the user/participant needs to come in the picture. To announce the nature of this cause and to get a different variety of participant sample for his study, Tim goes ahead and places an advertisement on the notice board outside the CS departmental office (Fig. 1). But before that he makes it a point to take Cheri's permission.
Note: All photos are clickable to enlarge.
Figure 1. One of the many discussions between Tim & Anthony (left),
and putting up the advertisement to scouting for participants (right).
The newly displayed advertisement generates a student's interest. Smita is intrigued by the idea of figuring out what kind of experiments are carried out in Dr Hornof's Lab as well as making some money while satisfying her curiosity. She calls Tim to find out more about the idea and agrees upon a time to meet (Fig. 2). The identity of the participant in non-documented actual experiments is kept anonymous. (But since this is a documented photo-essay of the experimental procedure, whose results will not be used, the anonymity condition is not strictly adhered to.)
Figure 2. L to R: Smita reads the advertisement and considers participating (left),
and Tim subsequently receives her phone call and they arrange to meet (right).
On the day of the rendezvous, Tim makes it a point to get to the lab at least half an hour before the arrival of the P. In case it's the first experiment of the series he arrives a whole hour earlier to make preparations for the experiment. He needs to do so because the paper work and preparation involved in this process is lengthy and detailed. To avoid confusion at a later date, the first thing he does is to put the participant number and date on all the documents. These documents are procedural requirements. The Office of Human Subject Compliance lays some strict guidelines for these procedures and documentation. Anyone doing research and experimentation involving human subjects needs to submit their proposal to them. They require the consent form, a script of what the Ps will be told, including stuff like what their rights are etcetera. For example, in Tim's experiment, he has listed mental fatigue of looking at the computer screen for extended periods of time as a possible effect on the participants. (Fig. 3)
Figure 3. Tim takes out documents from his neatly-filed filing cabinet (left),
the preliminary questionnaire (center), and a part of the required paperwork (right).
Tim then sets up the user or P environment by cleaning and placing an ergonomically well designed chair in front of the testing station, and mounting the chinrest. He then removes all distractions for the participant as well as for himself including unplugging the phone and turning down the volume of the answering machine. He does this because he doesn't want to take the slightest of chance of anything going out of plan. Even the slightest disturbance during the experiment can possibly lead to unnatural variations in the data set collected. His next step is to block the windows. The eye tracker uses infra red (IR) rays. To reduce interference with the eye tracker and for optimal performance, stray IR radiation from ambient light needs to be removed by blocking as much of sunlight as possible as sunlight is a source of IR radiation. Figure 4 shows Tim set up the room for the experiment.
Figure 4. Tim is putting up the chin rest (left), and blocking the sunlight (right).
The next step is to setup the Eye gaze system and the workstation to be used by the P. Tim sets the monitor in cloning mode to monitor the video stream of the Macintosh from where he will run the experiment. This means that the Mac and the P's workstation's monitors show the exact same display. Visual aspects being very important in this experiment, he ensures a minimum screen resolution. Because the two computers are connected via the Ethernet (Fig. 5), he then disables the possible communication blocking software such as Norton Firewall and Anti virus and sets the power options to 'Always On' schema. He then connects the Ethernet to Mac and sets the appropriate TCP/IP address so that the two computers can communicate with each other during the experiment execution.
Figure 5. Simple Connection Schema.
He then sets up the client computer. After quitting all applications, he hides and minimizes the dock, which is the always-visible-unless-specified-otherwise bar containing program icons at the bottom of the screen. He does this because the P will be using this computer and he wants only experiment-related stimuli on the screen. He then sets the mouse tracking speed to medium and ensures that there is no tension in the mouse cable. He sets the monitor switchbox to Mac (cloning on PC means mirroring on Mac). He turns on the mirroring and sets the resolution of his own machine. Tim then inputs the test data which has been pseudo-randomized to avoid any order effects in the observed data.Tim then opens the practice software.
After the Mac setup, the next thing Tim does is to set up the switcher and video tape recorder (VTR) (Fig. 6) to appropriate modes. Setting up the video recording is quite a process in itself. It works as follows: The eye-tracker shows cross-hairs of where the P is looking. This image is converted into a video output. There is a second video of what the P is looking at. the two videos are mixed by using a video mixer (Fig. 7). The mixing requires some careful setting up of the two videos so that they line up on top of each other correctly.
Figure 6. The VTR and mixer.
Figure 7. Eye-gaze to video.
He finally arranges all relevant documents in the right order and sits down to wait for the meeting. Fig. 8 shows Tim updating his procedural checklist in which he keeps a track of all the tasks he needs to do which he does routinely, a very commendable practice. The procedural checklist is Tim's lifeline in the experiment- the silken thread which holds all the pearls of his experiment in order and together, and he maintains it diligently with all the due concern. The hour of the arrival of the P is near. Tim is constantly alert towards any untoward happenings and ready to take on any circumstance.
Figure 8. Tim updating the checklist.
Figure 9 shows the arrival of smita, the P. Clearly, she does not have much idea just how important and carefully planned everything in the CMET lab is at that moment. Before he begins any paperwork, Tim places a 'Quiet' sign in the hallway to inform people that an experiment is in progress in the lab, another one of his measures to ensure a no-distraction environment during the experiment. He then presents the consent form to the P and asks him/her to fill the pre-experiment questionnaire (Fig. 9). This is followed by a brief walk through of the lab and the equipment to the participant. Tim is eager to begin but he has the patience to make the P comfortable and at ease.
Figure 9. Introductions and greetings following Smita's arrival (left),
Tim placing the 'Quiet' sign outside the door (center),
and
the preliminary questionnaire (right).
For the purpose of his experiment, Tim needs to test the vision of the P so he gets to "play optometrist", as he says, for a short while. Figure 10 shows the administration of the vision exams. The first test is the stereovision test which is more of a game in which you spot an object in a confusing mass of lines and shapes and colors. Then he administers the color test to check for color differentiation as it is used in one of the experiments. This is followed by the acuity test which ensures that the P has sufficient acuity to see the computer screen clearly. Finally we have the dominant eye test in which Tim simply hands the P a kaleidoscope for her to play with and notes the eye with which she looks through it. He needs to do so to set the eye tracker on the P's dominant eye.
Figure 10. Optometry (from L to R: stereovision test, acuity test, dominant eye test).
The eye tests, if successful, are followed by Tim presenting the P with the experiment instructions. In this case, the tests were a success and Tim was relieved. But like all conscientious researchers, did he let go of his tension? No, he did not. He still had the worry whether the eye tracker would be able to calibrate in spite of the P's contact lenses, which create calibration problems in some cases. Figure 11 shows Tim explaining the apparatus to Smita and fitting the eye-tracker to her. Finally to begin the experiment, he fine tuned the user environment by adjusting the table height, chin rest, armrest, monitor angle, camera focus, key board shelf and mouse.
Figure 11. Preliminary stuff (from L to R: experiment instructions, adjusting user environment).
Tim calibrates the eye tracker to track the P's dominant eye accurately. He is hoping that the calibration would turn out to be alright so that the effort and preparation that has gone into the setup would not be a complete waste. It is finally when everything, including calibration, does turn out to be alright that Tim heaves a sigh of relief. It is all downhill from there on.
Tim presents the P with a practice run in which he explains to the P how to do the experiment. He shows him/her various examples, techniques for the usage of the mouse, and allows the practice run to run for as long as the P wants. She can practices and becomes comfortable with the apparatus and the procedure.
Then he launches the first experiment and enables the eye tracker. After running through the first experiment, Tim launches the second one, which is the color experiment, i.e., basically the same procedure as experiment 1 but now the visual stimuli have pertinent color information also present. All the while he is constantly monitoring the calibration, the video and other details. In addition to the flat rate paid to the P for participating, Tim has worked in a pay-off matrix of bonus and penalties (for correct and incorrect responses respectively) in the experimental procedure. The experimental software also penalizes the P for excess time spent per question (by subtracting money form the total bonus). In the experiment, there is a speed-accuracy tradeoff and Tim has set the financial constraints to motivate the Ps to go as fast as possible while maintaining an error rate of roughly 3 to 5%. (Fig. 12)
Figure 12. From L to R: Tim monitoring the experiment, the P during the experiment,
the color experiment, total bonus at the end of Experiment 1.
Tim also does not allow a P to take the test twice. This is because if a participant takes the test twice, he/she will be better at quickly selecting the response because they have had more practice at it. So some of the data will not only show the effects of human visual oculomotor and cognition processes but also a factor of training. Too much of practice effect is something which Tim wants to avoid because he wants to maintain the evenness or parity of test data.
At the end of the experiment, he administers the post questionnaire to P. Then he debriefs the P about policies and procedures. This is followed by a key event of the experiment, at least as far as the P is concerned, that is, paying the P the total bonus calculated and having him/her sign the payment form. (refer to Fig. 13)
Figure 13. Post Experiment Stuff (from L to R: Another Questionnaire, Tim signing pay form, money changes hands).
After the P leaves, Tim wraps up the experimental setup. He takes down the 'Quiet' sign; stores the data file using the participant number; cleans the stereovision glasses and the chinrest; and takes down the poster boards used to block the windows. (Fig. 14)
Figure 14. Wrapping up (From L to R: the P leaving, cleaning chinrest for extra hygiene).
At the end of it, Tim discusses the result of the experiment with Dr. Hornof. One of Anthony's first questions is "How did it go?". Every time, he half expects to hear that it went well, and half expects to hear that the P didn't show up or couldn't calibrate. Both of them are satisfied with Tim's organization of the experimental procedure (Fig. 15) and happy that things worked out well and the experiment was conducted successfully. They hope that the good trend will last for the entire series of experiments after which the most crucial and hardest parts of experimentation will begin, e.g., the data analysis.
Figure 15. The satisfied Tim.