Author: EDS954
Last saved by: Academic Information Services
Company: university os salford

Extract from:
Education in a Changing Environment 17^th-18^th September 2003
Conference Proceedings

Disentangling the Web:
A Principled Guide to Information Retrieval

Martin Eayrs, m.eayrs@salford.ac.uk

Critical thinking and the World Wide Web
The ability to think critically is what allows the teacher to see beyond the literal meaning of any discourse and give balanced, reasoned and well-grounded responses. The same evaluative criteria should be brought to bear on the selection of suitable material for teaching or research purposes. So what does critical thinking consist of? Too big a question to tackle here, perhaps, but any working definition would probably want to include some or all of the following:

•	The ability to analyse;
•	The ability to evaluate;
•	The ability to see a subject from different angles;
•	The ability to perceive relationships such as cause/effect, same/different;
•	The ability to infer, imply and analogise;
•	The ability to distinguish between fact and opinion;
•	The ability to distinguish between sincerity and irony;
•	The ability to perceive any lapse in logic, such as the non sequitur;
•	The ability to decipher who is saying what - the writer or others whose ideas the writer is representing.

This is not an exhaustive list and readers will doubtless be able to add their own criteria.
Critical thinking is, in short, the ability to perceive and respond appropriately to a great deal that is - and is not - immediately apparent to the casual reader or viewer. Critical thinking takes the learner beyond the literal, surface aspects of any given piece of material. The purpose of this short article is to suggest some principles of critical thinking that can be applied to the evaluation of material found on the World Wide Web (WWW).

Reading in the Offline World
When we pick up something to read we already have certain expectations. To use examples from UK newspapers, we approach The Sunday Telegraph rather differently from The News of the World. Our previous reading experience has led us to identify each as a separate ‘type’ and we react to each on the basis of this experience.
If we think for a moment of ‘Periodicals’ we could perhaps divide these into four categories:
•	Scholarly - ‘concerned with academic study, especially research’;
•	Substantive News / General Interest - ‘having a solid base, being substantial’;
•	Popular - ‘fit for, or reflecting the taste and intelligence of, the people at large’;
•	Sensational - ‘arousing or intending to arouse strong curiosity, interest or reaction’.
(definitions from Webster's Third Edition, online version)

It is our experience and familiarity with these broad genres that enable us to assign a publication to one of these categories, and although there will be borderline cases we feel on the whole confident about so assigning them. What is interesting, however, is how we are able to do this, and I would suggest that critical thinking plays an important part.

This ‘grading’ system works well for printed periodicals, and when we open the publication we are largely unsurprised by what we find.  However, when we follow the results of a search engine or click on a link on a web page we can't always know what to expect and all may not always be what it seems. For example, we don't usually find advertising in academic journals, and we certainly don't find flashing sprites or animated gifs, to the accompaniment of midi or similar sound files. And whereas academic journals tend to be produced by distinguished scholars in the ivory towers of academe, all the world and his brother seems to be producing web-pages, some built and maintained by children as young as seven or eight. 

Put at its simplest, when you open a book or magazine you have a good idea where your reading will take you. When you click on a hypertext link on the World Wide Web you can't really be sure exactly where you are going, and even when you get there you can’t always be sure exactly where you are. But there are clues, as I shall show.

The Dangers of Unprincipled Surfing
As we have seen, as academics we have all had to learn how to distinguish 'scholarly' journals from other periodicals. In fact we might divide these into such areas as: 'academically respectable', 'professionally solid', 'popular' and 'sensationalist' - as I did above - but how do we do this? What criteria do we bring to bear?

When evaluating traditional printed material there are certain points one would do well bear in mind. These are well known and generally taught as part of an academic skills programme. I present them here in the form of questions one might wish to ask, drawing heavily on Susan Beck (1997):
Accuracy
•	Is the information reliable?
•	Has the information been checked for errors of fact ? By whom?
•	Is there an editor who assumes authority for this? 
•	Are there people entrusted with the specific duty of checking all the facts?
•	Do we know who these people are?

Authority
•	Is the author really ‘qualified’ to write on this subject?
•	Is the author a recognised authority in this field?
•	Is the publisher a ‘reputable’ publisher?
•	Is the book published privately, perhaps by the author himself?

Objectivity
•	Is bias kept to a minimum?
•	Is the information trying to influence the opinion of the audience?
•	Is theory supported by data or coherent argument?
•	Is unsupported theory built on to produce subsequent argument?

Currency
•	Is the content of the work up-to-date?
•	Is the date of publication clearly marked?
•	Are you satisfied that the given date is genuine?

Coverage
•	Is the range of topics included in the work adequate?
•	Is the range of topics included in the work relevant?
•	Is the range of topics included in the work explored in reasonable depth?

These criteria, remember, are used for evaluating printed material. Let us now apply these same points to publishing on the World Wide Web.

Accuracy
The sad truth is that anyone with the right software who knows how to type and navigate can publish on the Web. Statistically, few web resources are verified by editors or fact-checkers and there are as yet no internationally accepted Web Standards to ensure the accuracy of information displayed. In the worst of cases, there is nothing whatsoever to prevent the malicious posting of totally false (or, perhaps worse, partially false) information.

Authority
It is usually difficult and often impossible to determine real authorship of material posted on the Web. In cases where an author’s name is listed his/her qualifications are usually not, and responsibility for accuracy or truthfulness of content is very rarely stated. The fact that an article is attributed to, say, Noam Chomsky, is no guarantee that Chomsky himself wrote it or that it even reflects his views.

Objectivity
Beyond established institutions, the aims of persons or groups who post material on the web are often unclear. The Web has a tendency to act as a ‘virtual soapbox’ where anybody can get up and say what they want - indeed this is one of its strengths - but it certainly attracts a fair share of eccentrics along with the altruistically motivated. Which of these is which is not always clear.

Currency
Any active Internet user is aware of the number of dead pages littering the Web, e.g. sites saying ‘last modified August 1995’. There has been no obligation to include dates on Web pages and in cases where a date is included it may variously be the date the information was first written, the date the information was posted or the date the information was last modified. Again the fact that a page bears a date does not mean that the date is genuine, nor is there any way of authenticating it.

Coverage
It is hard to determine the extent of Web coverage because of the nature of hypertext links which join web pages and web sites together. Indeed, it is not technically possible to prevent one site linking to another, and with a little skill it is possible in most cases to link to any part of any site, thus implying an authoritative and authorised link, when in fact no such tacit connection was intended by the author(s) of the site so targeted. For this reason, any one web page should in principle be evaluated independently of any other, especially when leaving the ‘home’ site.

Further Issues
Advertising
In printed material the distinction between advertising and information is usually made explicit. On the web it is not always easy to discern, especially when advertising on so many sites nowadays – as an intricate and intrinsic part of web site economics – is wholly incorporated into the page and site design. Furthermore, while it is clear in a printed publication what is advertising and what is editorial copy, on the Web it is usually impossible to know whether the advertising and informational content are being supplied by the same person or organisation. If they are, the advertising is likely to bias the informational content, and the reader needs to be aware of this.

Customisation
What each user can see on their screen and how they see it, and what they can print or download to their computer will often vary from user to user, according to the user’s platform, system and browser configuration and the software and plug-ins they have installed.  Indeed the server may serve different page content according to the country of access, modem bandwidth or user-defined preferences. Furthermore many sites are dynamic, with constantly changing content, some being randomised on user access. And even respected sites change their content regularly – for example the concept of a ‘daily’ newspaper doesn’t exist on the Web, where  ‘breaking news’ breaks at any time of day or night.

All these factors can mean in essence that two people can go to the same website at the more or less the same time and see very different pages, with different content, which is not helpful when you want to cite a fixed reference and lead others to it.

URL Structure
Today’s search engines and indexes are extremely powerful for those that have learned how to use them. Still, they are far from perfect and can retrieve Web Pages totally out of context, frequently directing the user to somewhere wholly inappropriate. To quote Huw Jarvis, ‘Search engine ‘hits’ are no indication of ‘quality’, nor is the numerical list any true indication of the importance of the site’ (Jarvis 2003).

Sometimes you can work back through the URL, stripping off sub-directories backslash by backslash, to get to a relevant home page, but equally frequently users will be bewildered by the plethora of pop-up windows and aliased web addresses that ambush them from all sides, with the result that they end up not knowing where they are.  Site managers are frequently to blame for the inability to find previously visited and bookmarked files, or files cited in another document, and it would certainly be helpful if web managers would leave files in the same place on the site and not restructure web content every few months.

An ability to understand the way URLs are constructed and the principles of directory systems is helpful here, but even then a savvy hacker can dump a directory on some one else’s system which may go undetected for a while, leaving the user believing he is on one site while in truth he is somewhere else.

Accessibility
Web pages are notoriously unstable and not only can they become temporarily unavailable (web maintenance, too much traffic, server down, etc.) but they are likely to move or disappear without notice. Far worse, they can be altered without their owners knowing, either accidentally or intentionally. Many sites are not protected by firewalls and those that are may still be penetrated by a competent hacker. After all, for a skilled and dedicated hacker who can work his way into the Pentagon a University web site is hardly likely to present much of a challenge.

Online Publications
Many online journals are available nowadays. Some of these are online versions of printed journals, carrying much the same content. Others, however, are wholly electronic. In this latter case, articles which are ‘refereed’ are far more likely to be reliable sources that those which have just arrived from nowhere to fulfil the editor’s publication schedules. Check carefully to see if there is a review board and what the publishing policy of the journal is.

Caution should also be taken when quoting from electronic lists and discussion forums. Just because someone posts a message citing the words or opinions of a third party does not automatically mean that the ‘quote’ is accurate – it should, as in all such cases, be independently verified.

Data Entry
It is also useful to reflect on how information gets into digital form in the first place. Web resources are usually either keyboarded or scanned. Keyboarding of long texts is often farmed out as piecework to people who may be unfamiliar with technical texts – or even in extreme cases, the language. Like humans, OCR (optical character recognition) engines have an unfortunate tendency to make sense of unfamiliar words by changing them to something within their repertoire. Spell checkers obviously don't always help here, merely checking that each word typed corresponds to a word in the language, not necessarily the right one. Given this potential for error, along with the high costs of professional proofreading, it is hardly surprising that web sites have so many inaccurate texts and data.

Possible Solutions
One solution to the problem of site integrity has arrived in the shape of the portal – a web site which purports to guarantee the integrity of its content and that any outgoing links on its pages are acceptable to the site administrators. No site administrator can stop incoming links but once users arrive at the portal they know (in theory at least) where they are and what they can expect.

The down-side for the user is that many such sites are passworded. There are many reasons (legal, commercial, protection of data, etc.) why academic or commercial institutions are unable or unwilling to share all their content with the world at large. Most reach a compromise and make some of their information generally available and limit other areas to their own Intranet, available only to their own staff and students, and accessible only through their own onsite terminals and with a personalised password.

Another emerging solution is the new Extensible Markup Language (XML), which provides flexible and customisable identification of information through standardised, embedded databases. One expert in the field writes that XML ‘… provides a robust, non-proprietary, persistent, and verifiable file format for the storage and transmission of text and data both on and off the Web’ (Flynn 2002). 

XML is actually a metalanguage and, put simply, using XML means that in addition to page content a web page can also contain identification and categorisation data – in the form of data base fields - that will define its content and provenance more precisely. These data can include, for example, classification information (such as the Library of Congress subject areas) and institution identification (verifying an institution as a member of one of a number of professional bodies), but can also include user-defined fields for whatever purposes may serve the user(s). The system is still in development, but when fully implemented will be a great boon to academics keen to separate the wheat from the chaff. 

In addition to these technological solutions, as we have said earlier, it is of course vital that users demanding data integrity apply rigorous criteria in evaluating what they stumble upon on their wanderings around the Web.

Conclusion
My advice is that the Web is only one source of information, not always reliable, which can be very useful for researching certain topics on certain sites, and totally useless – possibly dangerous - for others. Judicious use of the critical faculty will improve the chances that the data you access will be genuine and useful.

Students should be taught that to research a topic thoroughly they should use a variety of sources, both Web and non-Web. As a work-around, CDs (if printed by responsible institutions and reasonably up-to-date,) may be more reliable than the Web in some respects but there is still very much a place for the old-fashioned dead-tree library.

Many of the points I have made are perhaps obvious to a mature, adult user of the Web. However, as the Web is increasingly being recommended for educational research projects and may - in the United States at least – be in the process of replacing the use of printed resources, younger users need to be warned of the limitations too. A new module in a study skills programme to this effect might not go amiss.

References
Beck, Susan(1997) “Evaluation Criteria” in The Good, The Bad & The Ugly: or, Why It’s a Good Idea to Evaluate Web Sources, accessed 15/09/03 at http://lib.nmsu.edu/instruction/evalcrit.html
Flynn, Peter (ed.) (2002) The XML faq v. 2.1.,accessed 15/09/03 at at http://www.ucc.ie/xml/faq.xml
Jarvis H (2001), ‘Internet usage of English for Academic Purposes courses’ in Recall 13, Cambridge University Press.