Protocol Analysis: Verbal Reports as Data

"Well, that’s actually one of the other books on my list, Protocol Analysis (1993) . So one way that we can understand how people think is by conducting interviews for what are called ‘think aloud’ studies. You give somebody a problem—let’s say like a test item or a task—and say, ‘I want you now to think out loud as you solve this problem, and as you’re doing so, tell me everything that’s going through your mind as you try and figure it out. Now, that’s just one way of understanding because there are certain limitations with the approach. First of all, if the problem is too easy or too difficult, there won’t be anything to report. So the problem has to be of moderate difficulty for the individual to actually be able to report something. That assumes, as well, that they’re making use of their short-term working memory for the thoughts or ideas, that they’re then able to report. Another way is to use eye-tracking devices. This is done especially with reading comprehension problems. You give somebody a script to read or some text, and then you ask them some questions, and you follow their eye movements as they try and respond to the questions. So you track the way that they read the text and then you track the way that they go back to the text and look at certain words, look at certain parts, as they’re trying to respond to the question. Those kinds of behavioural indicators give clues as to the way that people might actually go about discriminating, selecting information, combining information ultimately to generate a response to a test item."