| |
Experiment 1,
2003:
This study was performed as part of Nadine Wirschum's diploma
thesis (see Wirschum,
2003).
Two groups of 42 subjects were presented
with a search results page that listed 25 web pages retrieved
for a query about “assessment center building blocks”.
For the second group, the summaries had been enhanced with
headings. The task of all subjects was to answer a question
about assessment centers by visiting at least some of the
listed pages. The statistical analysis of the results showed
that, with the enhanced summaries, a significantly higher
percentage of the visited pages (54% vs. 37%) were ones that
the experimenter had previously classified as being relevant
to the to-be-answered question. This 44% relative increase
in precision was not achieved at the expense of recall: In
both conditions just about 67% of the relevant pages were
visited. (Only pages whose title, URL, or summary the subject
had looked at in the result list were counted in this analysis.)
Subjects’ ratings and debriefing comments confirmed
that the enhanced summaries gave them a clearer impression
of the context in which the keywords appeared. But the presence
of the headings did not increase the time that the subjects
required to read or scan the summaries.
Further analyses of the eye tracking and
mouse-click data yielded additional results that have implications
for the design of search result pages. For example, about
one-third of the subjects scanned part or all of the search
list before going back to visit individual pages–a breadth-first
strategy that is not very well supported by the leading search
engines.
The study was performed using the ASL
Model 504 remote eye tracker; see the
videos recorded during
the study.
Experiment 2, 2004:
Strictly Depth-First Search |
Partly Breadth-First Search |
Strictly Breadth-First Search |
|
|
|
| |
|
|
In Experiment 2, each of 27 subjects was
asked to perform two tasks similar to those of Experiment
1, with 5 minutes allowed for each task. To create a situation
in which breadth- first processing seemed relatively attractive,
we allowed the subjects to open at most 10 of the 25 documents
listed, rewarding them for each relevant document found (about
half of all documents were relevant). Here again, contrasting
strategies were identified: 52% of the subjects showed virtually
no tendency to look ahead in the list. A minority of 11% used
the extreme breadth-first strategy, scanning the entire list
before opening any document; the remaining 37% applied a mixed
strategy, looking ahead at an average of 2 to 6 documents
within each list.
Publications on this subject: see Klöckner,
Wirschum, and Jameson (2004).
|
|
-Ausschnitt.jpg)
