Learning Through E-Reflection: Desires and Designs

Dr John Rae, j.m.rae@salford.ac.uk
Gary Taylor, g.d.taylor@salford.ac.uk

Abstract
In the rush to e-learning there are generally two casualties, they are: learning process and self-reflection. They are the collateral damage from a technology-led as opposed to a pedagogy-led development. We correctly place much emphasis in higher education on the importance of a self-evaluative approach to learning, although this can frequently fall on deaf ears as the detritus of many abortive student learning logs demonstrate. Supporting a learner to develop a self-reflective approach can be challenging. This can simply be because the prompts, induction and handouts provided in preparation, are seen by the learner as worthy and possibly useful but just how and when they can be used and applied is not clear. A pedagogically supportive virtual learning environment should have at its heart technological processes to encourage, enable and support appropriate self-reflective learning. Many use discussion group technology for the group reflection and interaction, however, not all learners can be or want to be part of a discussion group. This paper examines the application of technology to promote and enable self-refection within a virtual learning environment.

Introduction
‘What do virtual learning environments know about learning?’ In a literal sense, absolutely nothing, just as a spreadsheet knows nothing of accountancy. However, both should have functionality that is supportive of the task they were designed for, and herein lies the rub. Are learning environments really designed for learning? Or, are they mostly the result of a set of historical and administratively convenient bits? 

The answer is probably somewhere between the two all mixed in with a poor separation of sub-systems within the overall system. In most learning environments there appears to be a mix’n’match of content management, learner management, and administrative processes. Within this mix there is also confusion over the functionality required to support learning, what it is and where it is. They are containers, more or less structured, for content which may or may not be tracked for access to some level. Linked to this is email. Add a collection of utilities like notepads and depositories, and probably a mechanism for synchronous and/or asynchronous discussion. These functional capabilities represent a set of useful tools under certain circumstances, but were they specifically designed for supporting learning, of course not. So how and when should we use them and what is missing? This is not a new question, Simpson (2001) argues:

...like many others I am keen to promote their (learning environment) use within my institution. But surely what I have described is a virtual environment - where is the learning? These systems support course management and access to a variety of resources but to promote them as 'learning environments' is a little misleading. Virtual learning is not a set of tools that provide access to information and space for discussion, just as learning is not a library and a lecture theatre.
(http://www.warwick.ac.uk/ETS/interactions/vol5no1/Simpson.htm)

Simpson argues it is the interaction with the academic staff that puts the ‘L’ into VLE (Virtual Learning Environment). We would not dispute this. However, is this the only way of earning the ‘L’ kite mark? This paper does not try to address the overall issue of the range of functionality required to support learning, it is too big a topic for one paper. However, we do focus on a part of this domain, that of reflective learning. The importance of reflection and reflective practice has permeated individual learning and classroom practice for many years (see Schön (1983), Dewey (1910), and for a critique Bleakey (1999)), not all proponents agree about the detail but most agree that reflection is an important aspect of learning. In this paper, we shall look at learning and reflection in the context of a learning environment and see if its functionality can be extended in ways that put the ‘L’ back into VLE.

Reflective Learning
What do we mean by being reflective? Osterman (1990) suggested: 
Reflective practice is the mindful consideration of one’s actions, specifically one’s professional actions and is a challenging, focused, and critical assessment of one’s behavior as a means towards developing one’s craftsmanship (p.134).

Reflective practice has also been described as a form of higher level cognitive activity. Valli (1992) states: 
Reflection is the capacity to ‘notice oneself noticing’ that is, to step back and see one’s mind working in relation to its projects (p.99).

The first is related to teaching and professional development, the second could apply to any activity, they both equally apply to learning of any description.

Most definitions involve the personal exploration of the experience of learning to better develop an understanding of how to learn and so, hopefully, improve their future learning. Such characteristics as increased self-awareness and objectivity, a greater openness of approach, increased motivation, and greater independence are eschewed on the successful reflective learner.

In recent years the growth in the emphasis on reflection has been encouraged by several influences, Continuing Professional Development (CPD) in professions such as teaching and health, as an opposing influence to the technically rational view of education that can permeate government policy, and as a means of achieving a progression from a surface to deep approach (Marton and Saljo, 1976) to learning linked to the ‘reflective’ step in Kolb’s Experiential Learning Cycle (Kolb, 1984). Of course, there are a number of strategies traditionally employed in attempting to promote self reflection, including: 

•	Diaries	•	Self and peer assessment	•	Development planning	
•	Learning logs	•	Reflective commentaries	•	Group learning	
•	Integrative Assignments	•	Discussion groups	•	Action learning	
They meet with varying success depending on such factors as induction and continuing support, it is also the case that some learners are more adept at reflection than others. In the current climate with student loans, debt, and a large number of students working part-time, the continued rise of the ‘strategic student’ (Kneale, 1997) who seeks the maximum output for the minimum input and so focuses almost relentlessly upon the assessed work, does not sit comfortably with the hoped for benefits of a supportive reflective journal.

There are, of course, developments that employ a VLE or web based learning and reflective elements such as discussion groups. Learners are able to share their views and receive constructive feedback from fellow discussants. Some projects have gone further and utilise ‘ePortfolios’, see for example ePortfolios at ‘Managed Environments for Portfolio-based Reflective Learning’ (http://www.eportfolios.ac.uk/FDTL4) that have been developed for use in medical training as an additional means to evidence the achievement of learning outcomes for summative assessment. This development is composed of a consortium of  the University of Newcastle (lead site), the University of Leeds, the University of Sheffield and the University of Dundee. Combined with the learning environment facilities learners can choose to make a range of evidence available to others. Such ePortfolios are linked to the Personal Development Planning (PDP) requirements that arose from the Dearing Review. PDPs are intended to provide a formative means to support students to monitor, build and reflect upon their personal and academic development.

In recent years there has been a rapid growth in experiments with Computer Supported Collaborative Learning (CSCL), that is, how collaborative learning can be supported by technology. This has been prompted by the growth of the Internet as much as anything else. CSCL aims to enhance peer interaction, group work, and the sharing, growing, and distribution of knowledge within a group. A rather complex and subtle process. Koschmann (1996), argued that this was an emerging paradigm in educational technology and since then there has been many attempts at collaborative learning from different pedagogical perspectives and with mixed results (for an interesting review of the area see Lipponen (2002)). The eighties saw many attempts at intelligent tutoring systems (ITS), that sought to diagnose and remediate the errors made by learners. They were usually based upon some domain knowledge combined with a ‘knowledge’ of buggy behaviour.

More recently, agent technology, including ‘pedagogical agents’, have offered support for the learner by, for example, getting to know what the learner may want and searching the Internet and filtering results. In such settings learning and adaptation are essential elements of agent technology. Their possible functions include (Johnson, 1998):

Pedagogical agents are autonomous agents that support human learning, by interacting with students in the context of interactive learning environments. They extend and improve upon previous work on intelligent tutoring systems in a number of ways. They adapt their behaviour to the dynamic state of the learning environment, taking advantage of learning opportunities as they arise. They can support collaborative learning as well as individualised learning, because multiple students and agents can interact in a shared environment (p. 13).
These pedagogical agents are potentially very useful behind the scenes operators, although from our perspective it is not clear if they are particularly supportive of reflective learning. Our particular goal is less ambitious than that of the above pedagogical agent description. It is simply to promote reflection, a student’s personal reflection upon their learning. This is, of course, not simple to do but it is a specific and constrained objective that can be tackled by stages and it may contribute significantly to the learning experience. In some respects what is utilised is an agent, but quite a limited one, and it will rely upon linguistic ‘tricks’ as much as clever adaptation.

Kolb, Constructivism, Eliza and Affective Learning
Kolb (1984) formulated what is now a well-known and long established educational model, called experiential learning. Learning, according to Kolb, is based upon a four stage cycle as shown in Figure 1. The important thing is to go around all four steps. Others, such as Honey and Mumford (1986) have developed from this the idea that learners have a preference for starting their learning at different points in the cycle depending on a person’s learning style. They have a more sophisticated model based around the Kolb cycle which identifies four learning styles: activists; reflectors; theorists; and pragmatists, each of which has characteristics. They essentially provide a way of thinking about how people differ in how they learn.

Kolb’s cycle appears in many guises and variations (see Atherton, 2002 ) and some relate it to the constructivist viewpoint. In essence, constructivists see that meaning is achieved through observation and through thinking about how these new experiences fit with what is already known. Reflection, integration, and generalisation are key processes where each learner creates his or her own mental model.

‘Experience plus Reflection equals Learning’, originally a phrase attributed to Dewey, tends to summarise the core of many approaches, whether it is reflection in learning or on learning. The approach taken in this paper is the latter; the learning experience, or a stage of the experience, is assumed to have happened and the general proposition is that open ended questions and prompts relating to progress, blockages and feelings will stimulate the learner to reflect on that experience. This is not specifically about the particular knowledge per se but about the progress to embedding new experiences and learning within a mental model. For our initial trials we adapt a classic artificial intelligence programme (Eliza) and combine it with Microsoft Agent. This approach can be further developed via process trace techniques (Ericsson, K.A. and Simon, H., 1996) to map the problem space and a grammar related to a reflective discourse.
Eliza (Weizenbaum, 1966) is an early artificial intelligence programme with an interesting history. She (it) was named after Eliza Doolittle from Bernard Shaw’s Pygmalion, in which Professor Henry Higgins (a linguist), wagers that he can transform the humble flower girl into the toast of London by teaching her how to speak proper. Eliza, the programme, like the flower girl can learn to ‘speak’ increasingly well by the upgrading of her scripts, that is, her linguistic rules (a kind of grammar) for manipulating input. Weizenbaum used a psychotherapist in the original script and many responses are simply re-phrasing, or reflecting, what was entered.

A typical Eliza session would be something like:
Person : Men are all alike.
ELIZA : In what way?
Person : They're always bugging us about something or other.
ELIZA : Can you think of a specific example?
Person : Well, my boyfriend made me come here.
ELIZA : Your boyfriend made you come here?
Person : He says I'm depressed much of the time.
ELIZA : I am sorry to hear that you are depressed.
Person : It's true. I am unhappy.
ELIZA : Do you think coming here will help you not to be unhappy?
Person : I need some help, that much seems certain.
ELIZA : What would it mean to you if you got some help?
Person : Perhaps I could learn to get along with my mother.
ELIZA : Tell me more about your family

Eliza uses simple linguistic tricks to make conversation, her script contains such rules as:
Person: (X me Y) ---> Eliza: (X you Y)
Person: (I remember X) ---> Eliza: (Why do you remember X just now?)
Person: (My {family-member} is Y) ---> Eliza: (Who else in your family is Y?)
Person: (X {family-member} Y) ---> Eliza: (Tell me more about your family)
If the left-hand-side is matched then the input is transformed to the output of the right-hand-side. Eliza also has several ways of responding to keep the input coming when the input is not recognised, this includes phrases such as:

-	Can you elaborate on that for me?
-	That is very interesting. Why do you say that?
-	Tell me more...

Although Eliza is a very simple programme, it has a history of successfully drawing people into quite detailed conversations. In essence, Eliza is a good listener. And a good listener, with appropriate prompts and supporting information, could be just the kind of approach to encourage some genuine reflection.

The original Eliza interface was an electromechanical typewriter, things have moved on somewhat in terms of interface since 1966 but many current day Elizas, or ‘talkbotts’,  (see for example (http://www.manifestation.com/neurotoys/eliza.php3) are still simple line input screens. The question is: personification of the interface or not? This is a complex area, the visualisation of ‘Eliza’ and the representation of ‘character’ is problematic. The difficulty is the potential for mismatch between how the representation is perceived and its actual capabilities. In other words, if you put up a representation of Einstein but it displays the mental capacity of my microwave oven it can have a credibility problem (see Masterton, 1998): 

A common problem with AI programs that interact with humans is that they must present themselves in a way that reflects their ability. Where there is a conflict between the ability of the system and the users' perception of that ability a breakdown occurs and users may either fail to exploit its full potential or become frustrated with its shortcomings.
(Masterton 1998, p. 215)

The representation ideally needs to display some emotion, this can add believability and it importantly recognises that learning and emotions are closely connected  (Frasson, C., 2000). This approach, which we endorse, is concerned with affective learning, that is the learner’s feelings and emotional state can affect learning progress. The ‘Affective Learning Companion’ project at MIT aims to build a ‘computerised learning companion that facilitates the child's own efforts at learning’ (http://affect.media.mit.edu/AC_research/lc/nsf1.html)

This particular development utilises six emotional axes and relates them to their affect on learning. The endpoints of the emotional axes are: 

•	Anxiety - Confidence
•	Boredom - Fascination
•	Frustration - Euphoria 
•	Dispirited - Encouraged
•	Terror - Enchantment 
•	Humiliation - Pride

Figure 2 shows where a child might be at some point in time with respect to some learning activity, a setback may place him/her in the III quadrant whereas a new idea may prompt a move to IV.

Peedy the Learning Coach and Affective Learning
For our own approach we have focused on prompting the learner to evaluate where they are on two axes: ‘progress’ and ‘feelings’. The system, employing a learning coach, is intended to be used by adult learners in whom we wish to develop evaluative behavioural skills based around a self-evaluation of progress and an affective emotional scale. The coach is incorporated within a learning environment (called Dimensions and constructed by the authors) in the ‘Learning to Learn’ support section, it can also work in a free standing format. The environment can also monitor approaches to learning based on the Kolb cycle. However, this element has not been integrated into the reflective mechanism at this point.

The two axes consist of:
Progress: 
Well				Fair				Poorly

Feelings: Happy		Pleased	Frustrated	Stressed	Annoyed

Learners start by selecting a point on each axis to begin with. According to this selection the MS Agent follows a particular script. At this point the actual scripts are based around ‘Eliza’ type reflections of meaning and some initial prompts based upon the experience of the authors in promoting constructive reflection in students. This particular aspect, that is the scripts, is the subject of further work involving real student-tutor interactions and a process trace analysis to help determine an appropriate grammar of responses. The initial work is limited to testing the format and approach.

Microsoft’s Peedy agent (Figure 3) is used to represent the learning coach. Peedy was chosen as a non-threatening character with a good range of emotional responses and actions, including:

•	Confused 	•	Explain	•	Read	
•	Congratulate	•	Get Attention 	•	Sad	
•	Decline	•	Greet	•	Start Listening	
•	Surprised	•	Think	•	Uncertain	

A sample dialogue following a selection of ‘fair’ and ‘frustrated’ in the initial self-evaluation would run as below:

PEEDY: I see from your notes you seem rather frustrated with your progress today. Tell me why that is.
YOUR RESPONSE:  i wish i was achieving higher.
PEEDY: What have you achieved today? What stands out in your memory?
YOUR RESPONSE:  i have a good understanding of the basics.
PEEDY: How could you improve your understanding?
YOUR RESPONSE:  maybe if i came at it from another angle.
PEEDY: Can't you be more positive?
YOUR RESPONSE:  i just seem to have trouble after the basics.
PEEDY: Why do you think it is causing a problem?
YOUR RESPONSE:  because i get myself very stressed with it.
PEEDY: If these things make you feel stressed, how could you approach them differently?
YOUR RESPONSE:  perhaps i could look at my approach to learning.
PEEDY: Have you considered altering your approach to learning?
YOUR RESPONSE:  yes. i start with theory at the moment.
PEEDY: Have you considered theory carefully? Have you examined an observation or experimental style of learning?
YOUR RESPONSE:  yes, but I could do with some help.
PEEDY: What do you need help with? Perhaps you should speak to a tutor?
YOUR RESPONSE:  thank you.
PEEDY: No problem.

The dialogue is relatively primitive but this is the area for further work. Our focus is reflection on learning with a view to generating reflective thoughts and identifying both useful advice and supportive actions. Cognitive psychology supports the validity of dealing both with emotions as well as approach or technique. It turns out that we process perceptions for emotional content first and we have all experienced the difficulty of learning while we are stressed. Therefore the research must progress recognising these factors and moving towards generating scripts based upon the analysis of real life reflective dialogues.

Conclusions
This is exploratory research and development, that is part of an ongoing project to develop a learning environment that has some connection to the way in which we learn. With a reflective tool, like the learning coach, we can begin to explore the aspects of support that learners may find useful. It would seem important that future developments utilise the affective state of a learner as well as their perceived progress. The author’s feel that this, and potentially other approaches in encouraging reflection, will increase in importance as the adoption and fuller usage of virtual learning environments grows and their capabilities increase. The need to match the growth of learning content delivery with equally available learning support, within the same learning context, is the underlying rationale for pursuing this research theme. The early experiments with an Eliza style system seemed to invoke useful dialogue, we shall therefore continue to refine and adapt the approach.

References
Atherton, J. S. (2002) Learning and Teaching: Learning from experience [On-line]: UK: Available: http://www.dmu.ac.uk/~jamesa/learning/experien.htm
Bleakey, A. (1999) From Reflective Practice to Holistic Reflexivity. Studies in Higher Education, 24 (3), 315-330
Dewey, J. (1910). How we think. Boston: D.C. Heath and Co.
Ericsson, K.A. and Simon, H. (1996) Protocol Analysis: Verbal Reports as Data (Revised Edition), MIT Press.
Frasson, C. (2000). The role of emotional agents in intelligent tutoring systems. In Proceedings of the Eighth International Conference on Computers in Education 
(ICCE 2000), Taipei, Taiwan, 21-24 November, p. 60.
Honey P. and Mumford A. (1986): Using your learning styles. Maidenhead, Berkshire.
Johnson, W.L. (1998). Pedagogical agents. Proceedings of the Sixth International Conference on Computers in Education (ICCE '98), Beijing, China, October 14-17, pp.13-22. 
Kneale, P. (1997) 'The rise of the "strategic student": how can we cope?', in M Armstrong, G Thompson and S Brown, (eds) Facing up to Radical Changes in Universities and Colleges SEDA / Kogan Page, London
Kolb, DA (1984): Experiential Learning, Englewood, Cliffs, NJ.: Prentice Hall.
Koschmann, T. (1996). Paradigm shifts and instructional technology: An introduction. In T. Koschmann, (Ed.), CSCL: Theory and practice of an emerging paradigm, (pp. 1-23). Mahwah, NJ: Lawrence Erlbaum Associates.
Lipponen, L. (2002) Exploring foundations for computer-supported collaborative learning, http://newmedia.colorado.edu/cscl/31.pdf
Marton, F. and Saljo, R. (1976). “On Qualitative Differences in Learning”. British Journal of Educational Psychology. 46, 4-11
Masterton, S. (1998). Computer support for learners using intelligent educational agents: The way forward. Proceedings of the Sixth International Conference on Computers in Education (ICCE'98), Beijing, China, 14-17 October, pp. 211-219.
Osterman, K. (1990). Reflective practice: A new agenda for education. 
Education and Urban Society, 22(2), 133-152.
Schön, Donald A. (1983): The Reflective Practitioner, Basic Books, reprinted 1995 
Simpson, V (2001): VE or VLE Don't forget the L! Volume 5, No. 1 Spring term 2001, Teaching and Learning Support Unit, University of Surrey, also in Association for Learning Technology Newsletter, Issue 31, October 2000.
Valli, L. (Ed.). (1992). Reflective teacher education: Cases and critiques. 
Albany: State University of New York Press
Weizenbaum, J. (1966) ELIZA - A computer program for the study of natural language communication between man and machine. Communications of the ACM 9(1): 36-45.