Ratvinder Grewal, Mike Jackson, Jon Wallis, Peter Burden

School of Computing & Information Technology, University of Wolverhampton

Wulfruna Street, Wolverhampton, WV1 1SB, UK

{r.s.grewal, m.s.jackson, j.wallis, p.burden}@wlv.ac.uk

This paper addresses the problem of the interpretation of a results set from a World Wide Web search engine. Typically the results of a multiple-term query are presented as a one-dimensional list in which the contribution of the individual terms are hidden in a single "relevancy" score. The paper proposes the use of visual representations to overcome this problem. The results from an experiment comparing two related, but alternative, depictions are presented.

It is estimated that the World Wide Web contains over 350 million pages of data [Kowalski, 1997]. However, there is no widely accepted cataloguing mechanism, which makes it extremely difficult to locate information resources. A number tools for searching the Web (known as search-engines) have been implemented, all of which have been developed on an essentially ad hoc basis with respect to both indexing and query support, with a consequent lack of interface consistency and behaviour [Jenkins et al, 1998].

Search-engines compile their own indexes of Web-pages. These indexes normally identify all the pages containing a given word. Users can then submit a word, or set of words, (i.e., a query) to the search engine, which will respond with a set of links to pages. Each individual link in the set is referred to as a ‘hit’.

A review of the major current search engines, carried out as part of this research, has shown that all of them present the user with the same type of interface: a text entry box. In addition, all of the search engines reviewed presented the results of searches as a list of links. Some search engines list the links in order of relevance, although the way in which ‘relevance’ is determined is usually undefined. Irrespective of the definition of relevance for a single-term query, which is an area that has been researched in the field of information retrieval, none of the search-engines reviewed indicated the respective contribution of each term of a multiple-term query to the overall relevance of a hit.

This paper investigates the potential contribution of data visualisation to interpreting the results of search-engine queries. The benefits of data visualisation are discussed and two alternative visualisations of search engine results are compared in a two-part experiment. The eventual aim of the research is to develop a visual interface to the WWLib search-engine [Burden et al, 1995].

Data visualisation continues to find an increasing number of uses in science related fields [Schaller, 1991]. It can be defined as the application of computer imaging technology to data analysis [Haber et al, 1990]. However, the term data visualisation also encompasses the fields of computer graphics, image processing, computer vision, signal processing, computer aided design, human-machine interaction and user interface studies [Kaufman, 1990; McCormick et al, 1987].

Visualisation of experimental data has been used in scientific research for many years [Rosenblum, 1994; Arbarbanel et al, 1993]. An example is the use of graphs to illustrate a trend.

In principle, computer imaging to aid visualisation has been possible since the first computers were available. However, data visualisation has only emerged since the late 1980s, in response to the need to interpret ever increasing volumes and complexity of data [McCormick et al, 1987; Trenish, 1989; Helman et al, 1989; Hibbard et al, 1989]. There was a consequent need to develop more sophisticated visualisation techniques, for two principal reasons. Firstly, the human brain has difficulty in comprehending immense arrays of numbers from multiple sources of data [McCormick et al, 1987] and secondly, more complex data sets necessitate more advanced visualisation techniques [Rosenblum, 1994; Haber et al, 1990].

The broad goal of data visualisation is to assist researchers in understanding data by enabling them to see patterns and recognise any underlying relationships in the data [Marchak et al, 1993]. In short, the goal of visualisation is realisation [Rosenblum and Brown, 1992]. It acts to complement and amplify existing scientific methods for data comprehension and analysis [Kaufman, 1990]. It is estimated that fifty percent of the brains neurones are associated with vision [McCormick et al, 1987]. Data visualisation operates by using the extensive pattern-recognition capabilities that result from this neurological machinery in the eye-brain system [McCormick et al, 1987; Defanti et al, 1989].

When presented with data, humans will naturally try to use their inherent capacity for visualisation. Such mental imagery can be supported by the system only if the user interface is specifically designed to do so.

Information retrieval is an ideal application for visualisation, because there is an overwhelming need to reduce the cost and effort of receiving and understanding information and information structures [Card et al, 1991; Robertson et al, 1993]. The results provided by search-engines in response to user queries have all the properties normally associated with complex data and should thus be amenable to the application of visualisation techniques.

A typical search engine, in this case Excite, will return results to the query "energy saving bulb" in the following format:

The first line of the result indicates the relevance of the hit. The second line gives the URL and is followed by the first few lines of text from the web page. Using the information provided the user must judge whether the hit is relevant to their query.

The user is, however, only given partial information. It is unclear how each term in the query (i.e., "energy", "saving" and "bulb") contributes to the overall relevance of 95%. Providing the user with information about the relevance of each term in a multiple-term query is important, as they may want to view results in which one term is more relevant than the others. For example, in the query shown above, bulb could mean ‘light-bulb’ or bulb in the horticultural sense. A hit that owed the majority of its relevance to the word ‘bulb’ on its own might therefore be of lesser interest than one which was more equally relevant to all three terms.

Consider a three-term query. Figure 1 shows a three-sided pyramid, which looks a bit like a tepee. Each side of the tepee is used to represent one term of the three-term query.

In the initial scheme a pendulum was used to indicate which term was most relevant. The pendulum was shown as swinging to the side representing the most relevant term, the length of the pendulum indicating the overall relevance of the hit.

If two of the terms (for example x and y) are more relevant than the third (z), the pendulum would swing towards the edge joining the sides x and y.

One problem with this representation becomes apparent if a hit is relevant to all three terms. The pendulum cannot swing towards all three sides at the same time; it would therefore hang vertically, its length still indicating the overall relevance, but the respective contribution of each term is now hidden. Using three pendulums, one for each term, was briefly considered, but this idea was dismissed as too complex.

Another problem is that increased relevance of one term serves to attract the pendulum, thus diminishing the apparent relevance of the others.

A solution to these problems was to shade a percentage of each side of the tepee in proportion to the relevance of the term represented by that side; the more relevant the term, the greater the proportion of the side would be shaded. In this solution the pendulum no longer swings to a particular side to indicate which one is more relevant.

If a four-term query is represented, the tepee has a four-sided base. A five-term query results in a five-sided base. Thus a generalisation can be made; for an n-term query the tepee has a n-sided base. As n approaches infinity the base becomes circular, theoretically permitting the representation of as many terms as required.

Figure 5 illustrates this process of generalisation. The surface of the cone is partitioned according to how many terms are in the query.

The second solution was developed as a result of imagining standing inside the cone, looking up at the shaded surface. This 3-D perspective can then easily be mapped to a 2-D representation in which a circle is divided into segments, each segment representing a term in the query. The result is a "wheel" depicting the relevance of query terms in a result page. This symbol is referred to as a Results Wheel, or R-Wheel for short.

Figure 6 shows a result for a three-term query, represented using an R-Wheel. Areas of segments are shaded in proportion to the relevance of the term in the result, beginning from the centre of the circle. If a term that appears in the query is 100% relevant to the document retrieved, 100% of the area of the segment representing that term is shaded. If a term were only 50% relevant, only 50% of the area of its segment would be shaded.

A two-part experiment was conducted to assess user understanding of both the tepee and R-Wheel techniques of visually representing hits.

Thirty four subjects were used, their computing experience ranging from novice to professional, in approximately equal proportions. The subjects were only given the briefest of explanation about the problems of multiple-term queries and the two proposed solutions, as the aim of the experiment was to assess their intuitive interpretation of the visual representations.

The subjects were presented with both R-Wheel and tepee representations of simulated result sets for multiple-term queries (see appendices A and B). They were then asked to order the representations in each set of diagrams in terms of their overall relevance, using a pro-forma table (figure 7). After they had completed the table they were also asked to discuss how they had arrived at their ordering.

In the second part of the experiment the subjects were given sample figures for term relevancy and were asked to shade segments which represented the given percentages. The subjects were presented with the diagrams in figure 8, and were asked to shade 10% relevancy for fresh, 45% for baked and 80% for bread.

Analysis of the results and feedback showed that the subjects found it easier to interpret the R-Wheel representations than the tepee ones. A summary of the results is given in Table 1. Detailed results are given in Appendix C.

It was noted that the subjects experienced difficulty when comparing two hits of equal overall relevance. The difficulty was deciding whether a hit in which one term in the query was significantly less relevant than the other two (e.g., Appendix A, R-Wheel ‘A’) should be ranked higher or lower than one in which all three terms were approximately equally relevant (e.g., Appendix A, R-Wheel ‘H’).

More detailed analysis revealed that in the case of the tepee diagrams some subjects ignored the pendulum completely, but those who did not tended to rely on it exclusively to establish overall relevance.

The tepee symbol caused some problems, as the subjects found it slightly difficult to shade in the correct areas. The subjects became confused about which term represented which section and correctly shaded the two sides for fresh and bread, but shaded the base of the tepee for baked. They did not fully understand that the front side that they could see through was the one that had to be shaded for baked.

Most subjects completed the R-Wheels without any difficulty whatsoever. However, although they shaded the correct proportion of area, some subjects did not begin at the centre of the circle, rather they altered the angle of the segment in relation to the relevance - an example is shown in figure 9.

In fact, the approach of varying the angle of a segment to represent the relevance of a given term had been considered prior to the experiment, but was found to be confusing as it appears to double the number of segments which implies a greater number of query terms.

In this part the performance of the subjects was 82% correctly drawn for the R-Wheel and 64% for the tepee. Further details are in Appendix C.

This paper investigates the use of visual representations in helping users discriminate between similarly ranked search-engine results. The underlying assumption was that the results set from a search-engine hides the contribution of individual terms in a query to the overall relevance of a results ‘hit’.

The experimental results demonstrate that visual representations enable users to correctly assess the ranking of simulated search-engine results. From the two parts of the experiment, it is clear that the R-Wheel representation is more easily understood by novice users and more easily understood overall.

The initial hypothesis that n-dimensional representations would be appropriate for n-term queries proved to be incorrect as users found it difficult to form mental pictures of the meanings of such representations. Mapping n-dimensional images to two-dimensional pictures proved to be far more successful.

Abarbanel, R. Friedhoff, R.M., Langridge, R., Pearlman, J., Star, J. Is Visualization Really Necessary?, in Nielson, G.M., Bergeron, D. (Eds.) Proceedings Visualization, IEEE Computer Society Press, California, 1993.

Card, S. K., Robertson, G. G., Mackinlay, J. D. The Information Visualisation, an Information Workspace, Proceedings of CHI ’91, ACM, 1991, pp. 281-188.

Defanti, T., Brown, M. and McCormick, B. Visualisation: expanding scientific and Engineering Research opportunities, Computer, 1989, 22(8), 12-25.

Haber, B., McNabb, D.A. Visualization Idioms: A Conceptual Model for Scientific Visualization Systems, in Neilson. G. M., Shriver. B. (Eds.) Visualization in Scientific Computing, IEEE Computer Society Press, California, 1990.

Helman, J., Hesselink, L. Representation and Display of Vector Field Topology in Fluid Flow Data Sets, in Neilson. G. M., Shriver.B. (Eds.) Visualization in Scientific Computing, IEEE Computer Society Press, California, 1990.

Hibbard, W., Santek, D. Visualizing large data sets in the Earth Sciences, in Neilson. G. M., Shriver. B. (Eds.) Visualization in Scientific Computing, IEEE Computer Society Press, California, 1990.

Jenkins, C., Jackson, M., Burden, P., Wallis, J. Searching the World Wide Web: An Evaluation of Available Tools and Methodologies. Information & Software Technology, Vol. 39, No. 14-15, 1998.

Kaufman, A. "Preface", in Kaufman, A., (Ed.) Proceedings of the First IEEE Conference on Visualization, IEEE Computer Society Press, California, 1990.

Kowalski, G. Information Retrieval Systems: Theory and Implementation, Kluwer Academic Publishers, Boston, 1997.

Marchak, F., Cleveland, W., Rogowitz, B., Wickens, C. The Psychology of Visualization., in Nielson, G. M., Bergeron, D. (Eds.) Proceedings Visualization, IEEE Computer Society Press, California, 1993.

McCormick, B.H., DeFanti, T. A., Brown, M. D. Visualization in Scientific Computing. Computer Graphics, 21(6), 1987.

Robertson, G. G., Card, S. K., Mackinlay, J. D. Information Visualisation using 3D Interactive Animation, Communications of the ACM, Vol. 36, No. 4, April 1993, pp. 57-71.

Rosenblum, L., Brown, B. Guest Editors` introduction: Visualisation. IEEE Computer Graphics and Applications, 12(4), 1992, pp. 18-19.

Rosenblum, L.J. "Research Issues in Scientific Visualization", IEEE Computer Graphics and Applications, 14(2), 1994, pp. 61-63.

Scahller, N.C. Experiments with Interdisciplinary Projects and Scientific Visualization Applications at the Undergraduate Level, in Neilson, G.M., Bergeron, D. (Eds.) Proceedings Visualization, IEEE Computer Society Press, California, 1993.

Trenish, L. A. "Correlative Data Analysis in the Earth Sciences" IEEE Computer Graphics and Applications, 12, 1992, pp. 10-12.

R-Wheel representations of simulated result sets for multiple-term queries.

Tepee representations of simulated result sets for multiple-term queries.

Results from the experiment.

X means missed out by subject

1 means correct position

0 means incorrect position

A means partially correct (no mark given)

R means R-Wheel shaded radially