Caroline Elbra-Ramsay, Faculty of Education and Theology, York St John University

Abstract

Providing written feedback to learners within Higher Education can be a time-consuming and frequently ineffective process, with comments often undervalued or not acted upon. The same concerns exist within the primary sector; marking systems do not always effectively support the principles of Assessment for Learning. This frustration resulted in the development of the 'Quality Marking' model for primary schools (Clarke 1998) which follows an explicit process within which the type of comments, or prompts, used are key. This small scale study has investigated whether 'quality marking' – and in particular the use of scaffold prompts – can be applied to the Higher Education context. A group of final year student teachers within initial teacher training received and acted upon the feedback, as well as completing a series of questionnaires to record their perceptions of the process and its formative value. It is clear from the data gathered that 'quality marking' was viewed as an effective approach. Key identified reasons for this were the likelihood of the feedback to feed forward into future learning, the level of engagement from the learner, the clarity of how to bridge the learning gap, and the beginnings of self-monitoring.

Introduction

The publication of Black and Wiliam's seminal paper on assessment for learning in 1998 had a profound influence on understanding, policy and practice within the primary sector. However, the impact within Higher Education is less clear, with the key aim of empowering learners through a constructive and discursive approach appearing to be at odds with the often modular bound, summative based systems.

Within assessment for learning, feedback has been recognised (Black and Wiliam 1998, Gibbs and Simpson 2004, Cree 2000, Ramsden 2003) as a key formative strategy likely to lead to an increase in standards. Nevertheless, studies have repeatedly shown that within Higher Education, written comments are not perceived to 'close the gap' in learning to allow this to happen, or are not acted upon (Brown and Glover, 2005, Maclellan 2001). Given the hours devoted to providing written feedback, its apparent lack of use by students is frustrating and an opportunity lost. It seems pertinent to look to the successful models of feedback developed within primary schools over the last ten years in an attempt to identify possible areas of development. This small scale pilot project examines whether one such strategy can be modified for use in Higher Education and whether it provides an alternative and more effective form of feedback or feed forward.

Effective Feedback

What makes written feedback either effective or ineffective is complex to unravel. Not all written feedback informs future learning positively; in fact, poor feedback can cause further confusion and disengagement from the learner. Students are often puzzled about the value of the written comments they receive, perceiving that they are not necessarily helpful to their learning and understanding (Defeyter and McPartlin 2007, Gibbs and Simpson 2003, Maclellan 2001).

It is when we seek to define feedback that its core purpose is revealed; the fulfilment of this purpose can be used to determine whether it is effective or not. Ramaprasad (1983) states: '...feedback is information about the gap between the actual level and the reference level of a system parameter which is used to alter the gap in some way' (1983:4). As such, written feedback needs to close the gap between present and potential learning. This may appear obvious, and it could be assumed that this concept underpins any written feedback, but in fact this is not the case. Merely stating that a gap exists does not necessarily enable learning to move forward. It is only if there is a resulting positive outcome in learning/understanding that the 'feedback loop' (Sadler 1989) is completed and the underlying purpose is met.

Several studies have explored both the features and processes inherent in effective and ineffective written feedback. Chanock (2000) found that the inclusion of comments by the marker was not necessarily an indicator of developed understanding, as learners often misunderstood or misinterpreted their meaning, or were unsure how to rectify the problem. They did not enable 'students to make connections with their own work and so able to close the gap between their current and desired performance' (Brown and Glover 2006 p. 89). This is supported by Nicol and MacFarlane-Dick (2006) who, when identifying seven key principles of effective feedback, included the clarification of good performance and the ability to close the gap between current and desired performance (Nicol and MacFarlane-Dick 2006). In order to close the gap, students need to re-construct their own understanding, and as such feedback should be based upon a constructivist (or dialogic) approach (Gibbs and Simpson 2004). Feedback must be accessible to the learner, implying the need for some form of scaffolding or explanation. Stating what needs to be improved may not be enough; explaining how they can improve is much more likely to inform future performance (Walker 2009). Indeed without this detailing of how as well as what, most intended written feedback will in fact remain at a summative level (Glover and Brown 2006) i.e. it will provide learners with a measure of their current ability but will not inform future learning. The nature of the written comments received is therefore critical. Indeed Ramsden states: '...it is impossible to overstate the role of effective comments on students' progress in any discussion of effective teaching and assessment' (Ramsden 2003 p. 187). In an attempt to explore the nature of comments further, Brown and Glover (2004) analysed and then classified written feedback comments according to whether they: indicated a problem, offered a correction to the problem, or offered an explanation of the problem alongside a correction. It was the third of these that was perceived to be the most effective in that it did not merely identify the difficulty but explored how the difficulty could be remedied. This notion of comments enabling understanding of how to close the gap correlates closely with the long term aim of formative assessment to develop the learner's own ability to monitor, track and direct their own learning (Sadler 1989). The ownership of the process, according to Sadler (1989) relies on three things: an understanding of the goal, an ability to compare against current learning, and the ability to action the closure of this gap. Written feedback needs to encourage these self-regulatory abilities by modelling all three of these.

If feedback describes not only what is successful, what needs improving and crucially how it can be improved, learners also need to use this information to make improvements soon after if they are to become active in their own construction of meaning (Gibbs and Simpson 2004). The immediacy of the improvement is a feature particularly pertinent to Higher Education when modules tend to be organised around a final assessment that is often marked once the module is complete. Sadler identifies this when he states: '...the student must engage in appropriate action which leads to some closure of the gap' (Sadler, 1989, p. 121). Effective feedback needs to work as dialogue between learner and teacher, and students need to engage with the feedback. In Higher Education, the reality of the nature and timing of written feedback means that it can often act as an obstacle to this engagement rather than facilitate it.

Written feedback within the primary sector

In the primary school setting, the work of Black and Wiliam and the Assessment Reform Group had a huge influence on assessment policy and practice due to its potential to 'produce significant, and often substantial learning gains' (Black, 2000 p. 408). In fact Hargreaves (2001, cited by Kirton et al. 2007) refers to assessment for learning as 'revolutionary' in that it has impacted not only on the way primary school teachers view assessment but its ongoing cyclical relationship with teaching and learning. Key strategies have been developed and are now embedded within teaching and learning, including the use of peer/self evaluation, effective questioning and feedback. However in some settings, practice has become so focused on strategies that assessment for learning has become rather tokenistic, operating merely at a surface/procedural level rather than underlying pedagogy (Ward 2008).

Black and Wiliam emphasised the key role of feedback within assessment for learning: '...when anyone is trying to learn, feedback about the effort has three elements: recognition of the desired goal, evidence about present position, and some understanding of a way to close the gap between the two' (Black and Wiliam 1998 p.6). This emphasis on constructive feedback caused many primary practitioners to reconsider written feedback. Clarke (1998) made direct links between the work of Sadler (1989) discussed earlier and written feedback in schools, noting that it is the third of his key aspects, 'the ability to action the closure of this gap' that was not always evident in schools. As a result, she and her learning teams developed an approach called 'quality marking', a practical written feedback strategy focused on closing the learning gap for primary aged children. The approach consists of four stages:

• Recognise success.
• Identify an area for improvement.
• Add a comment (prompt) that attempts to closes the gap between present and future understanding. These prompts differ from standard feedback comments in that they serve not only to state what is needed but clearly support the learner in understanding how to achieve this. There are two main forms of prompt: scaffold prompts (give structured advice on how to make the improvement) and example prompts (provide a direct model of an improvement)
• Allow time for the comment (prompt) to be acted upon. (Clarke 2003)

As the prompts either illustrate or scaffold towards the desired next stage, learners are more likely to understand and construct future learning, effectively bridging the learner's zone of proximal development (Vygotsky 1978). This notion of demonstrating how the gap can be closed correlates closely with Nicol and MacFarlane-Dick's (2006) aforementioned principles of good feedback practice i.e. it can 'clarify what good performance is, delivers high quality information to the students about their learning and provides opportunities to close the gap between current and desired performance' (Nicol and MacFarlane - Dick 2006 p.7). Indeed, these prompts could also be applied to Brown and Glover's (2004) work on categorising written feedback comments; there are clear links between scaffold and example prompts, and the proposed effective category of 'explanation alongside a correction to the problem' (Brown and Glover 2006). The intention of prompts is to explain and engage the learner on how the gap can be closed, rather than correcting or indicating where the gap is. Their value is further supported by Cowie's 2005 research which found that pupils preferred feedback that included suggestions: '...suggestions supported their active engagement with ideas, both their own and those proposed by the teacher.' (Cowie 2005 p. 143) It is also worth noting that an immediacy of response is expected as part of the quality marking process, thereby further encouraging a more dialogic process and ensuring that the learner engages 'in appropriate action which leads to some closure of the gap' (Sadler, 1989, p. 121).

Although 'quality marking' is a strategy aimed at primary aged children, it reflects the generic principles of effective feedback and also correlates closely with some of the associated difficulties of feedback within the HE sector. These identified parallels led to the question and subsequent project: is it possible to apply quality marking prompts to a different context?

The project itself

The project took place within the final year of a BA degree (carrying QTS status) as part of a module equipping trainees with the skills required to be an English subject leader in school. There were 20 trainee teachers within the group, the majority of whom were young students. All had recently completed their final school experience and were in their final semester of the programme. The module had two assessments, both assignments of 2,000 words. Quality marking was used for the first of these, a piece of work focused on planning staff development on reading comprehension.

Alongside the standard University feedback for this assignment, three additional improvement prompts (either scaffold or example) were identified for each student to action. A timetabled session was arranged for students to both receive and act upon these prompts. The students worked with a partner to read through the prompt and then use the suggestions to re-draft the original work. The tutor was also available for support. All of the prompts given avoided simply stating the improvement needed but either gave a direct example or model of what could be improved or scaffold the students' understanding towards the next steps. It is worth noting that the researcher was experienced in writing feedback prompts, albeit with young children, and was able to construct similar comments with relative ease. Academics without this experience could find the art of writing feedback prompts initially more challenging.

Table 1

Questionnaires were used at three points during the project to elicit comparable quantitative and qualitative data. Each used open questioning alongside fixed alternative questions (including Likert scales). The intention of each questionnaire was to gain an overview of the group's attitudes and perceptions of both standard written feedback and quality marking prompts, their ease of understanding, usefulness and formative nature.

The first of these was administered at the start of the module, the second after the first assignment had been completed, marked and the feedback prompts received and acted upon, and then the third after the second assignment had been submitted. All responses were confidential and trainees were able to opt out at any time if they so wished.

Results/Discussion

a. Pre-Feedback Questionnaire (a)

In response to the question 'Do you read written feedback?' 75 per cent of respondents indicated that they were more likely to read it than not, but only 20 per cent indicated that they read it thoroughly. 25 per cent indicated that they did not read written feedback at all. When asked whether they found written feedback useful, a very varied response was received, with 55 per cent of responses indicating that it was of some use, but 10 per cent found written feedback 'not at all useful', seemingly reflecting the research of others (Brown and Glover, 2005, Crisp 2007, Maclellan 2001).

The third question sought to find out which type of feedback was the more useful. Students had to rank in order of preference: a. overall numerical score, b. overall summary comments at the front of the work, c. written comments made within or alongside the assignment itself.

80 per cent of trainees stated that despite its summative nature, the overall numerical score was the feedback they were most interested in. Feedback made alongside/within the assignment was seen as the least notable with 80 per cent placing it in third place although it is within this ongoing feedback that improvement prompts would be used. Students felt these written comments were '...sometime too specific to that essay', and '...do not help me with future work'. The point of specificity is an interesting one. Crooks (2001) stated that '...feedback should be specific and related to need' and that benefits come from focusing feedback on 'specific ways in which the student's work could be improved', but if comments are too specific to the context they are less likely to be transferable and formative. Specificity should therefore be in relation to the learner's needs.

The next questions explored the formative nature of the written comments received. Results were very varied, with only 10 per cent of those asked indicating that written feedback was always used formatively. Even when action was taken based on feedback, it was never immediate and often only within the year. As stated earlier, if the learning gap is to be closed, feedback needs to be received and acted upon by the learners (Gibbs and Simpson 2003, Sadler 1989). The data indicates that despite the intention of the marker, the pre-project written feedback received was not generally feeding forward as it was neither engaged with nor acted upon.

Table 3

Further questions provided qualitative information on the students' perceptions of effective feedback. Identified features included 'give specific ways to improve', 'organised and constructive', 'highlights areas within the assignment itself so it is clear to identify and rectify', 'to not only say what needs improving but how to improve it' and 'identify improvements that can be applied to other pieces of work'. These remarks allude to the earlier comment on specificity of improvement but not necessarily to context. It is also worth noting that the learners were seeking further advice on how to action the improvements, an aim of the scaffold/example prompts used within the project.

Using the Feedback Prompts

Each student worked with peers to respond to the prompts supplied, redrafting their original work. The clear expectations of the session ensured immediacy to the response and it was also apparent from their re-workings that the learning gap had, in the majority of cases, been closed to some extent.

Table 5

 Once all the prompts had been acted upon, trainees were given a copy of their improvements for future reference.

Post-Feedback Questionnaire (b)

This second questionnaire sought to ascertain whether the students felt the prompts were more effective than traditional feedback. In response to the question 'Do you prefer this form of feedback to the traditional form?' 95 per cent of students gave a positive response with the same number indicating that they were useful.

The level of usefulness was explored further in the next questions which were concerned with improvements made to both quality and understanding. All students felt that the prompts had resulted in an improvement in both. The quality of their redrafted work was perceived to be higher by all students with 59 per cent indicating i was considerably higher. Certainly in the short-term, the prompts appeared to have had had a consequence in learning. It would be interesting to further explore the longer term-benefits of this. Within the primary setting, learners receive prompts to action on a daily/weekly basis but in Higher Education learners submit far fewer pieces of work although they are longer and more significant. To ensure the actioned feedback endures, shorter formative tasks within sessions, possibly working with peers, may provide another context for their use.

Table 7

When asked to identify the effective features of this feedback a number of common points were made and these were categorised into six broad areas (see table 8.) The specific nature of the feedback and the enhanced understanding were the most common of these. These link to the themes already discussed i.e. the specificity but also transferrable nature of an improvement, the need for a focus on how to improve rather than what and also the value of models (examples).

Table 8

It was also interesting how formative the trainees considered the feedback to be. When asked, 94 per cent indicated that the feedback would inform future work and when identifying how it would do so, three broad areas were acknowledged: aiding clarity of explanation, developing individual specific weaknesses and increasing overall depth and criticality.

Post – Feedback Questionnaire (c)

The third part of the project took place at the end of the module after the second and final assignment had been completed. The purpose of this questionnaire was to establish whether or not the feedback had in fact been formative. 100 per cent of respondents felt that the additional feedback had assisted them with their second assignment. They were also able to specify how the feedback had informed the work. These responses can be categorised into four broad areas: clarity of writing, use of sources, criticality and self-study skills.

Table 9

The final question asked trainees to compare this and previous forms of feedback in terms of its ability to feed forward into future work. Responding on a scale, all trainees indicated that the prompts were more successful.

Table 10

What is not clear from the results is whether the apparently positive response was due solely to the prompts themselves and not the way in which they were acted upon. It is worth asking, could the process of working alongside a peer actually be more influential than the prompts alone? If so, would peer discussion ensure the usefulness and effectiveness of any written feedback in that learners themselves would provide the scaffolding and examples?

Conclusion

Providing constructive feedback to any learner is recognised as hugely influential in the learning/assessment cycle, but written feedback has limitations which can in fact render the process worthless. Difficulties associated with the immediacy, scaffolding and response to the feedback apply to both primary and Higher Education contexts, so it seems pertinent to use the practice of one to enhance the other. In this study it has become apparent that the use of scaffold and example prompts do go some way to ensure that feedback feeds forward into future learning as they, and the associated process, demand a more immediate engagement from the learner, bridge the learning gap by illustrating not only the next step but how to achieve it, and in some cases appear to develop the beginnings of self-monitoring. However, this was a small scale pilot study that only involved a very small group within a very specific context. In order to assess the potential further a larger group would be needed, ideally from across disciplines. The long-term benefits of this approach would also merit exploration as well as further consideration of the influence of peer work within the process, and how quality marking could be adapted to suit the specific constraints of the HE context.

References

• Black P (2000) Research and the Development of Educational Assessment. Oxford Review of Education Vol 26, Nos 3 and 4 2000 p. 407-419.
• Black P and Wiliam D (1998) Inside the Black Box: raising standards through classroom assessment. King's College London.
• Black P, McCormick R, James M and Pedder D (2006) Learning How to Learn and Assessment for Learning: a theoretical inquiry. Research Papers in Education Volume 21, Issue 2, 2006.
• Brown E, Glover C, Freake S and Stevens V (2004) Evaluating the effectiveness of written feedback as an element of formative assessment in science. 12th Improving Student Learning Symposium. Open University (Sheffield Hallam University).
• Brown E and Glover C (2005) Refocusing written feedback. 13th Improving Student Learning Symposium. Sheffield Hallam University.
• Brown E and Glover C (2006) Evaluating written feedback. In: Bryan C and Clegg K (2006) Innovative Assessment in Higher Education p.81-91 London: Routledge.
• Chanock (2000) Comments on essays: Do students understand what tutors write? Teaching in Higher Education Vol 5 No 1 2000.
• Clarke S (1998) Targeting Assessment in the Primary Classroom: Strategies for Planning, Assessment, Pupil Feedback and Target Setting. Hodder Education
• Clarke S (2003) Enriching Feedback in the Primary Classroom. Hodder Education
• Cowie (2005) Pupil Commentary on Assessment for Learning . The Curriculum Journal Vol 16 no 2 June 2005 p. 137-151.
• Cree (2000) The challenge of assessment. In: Cree (2000) Transfer of Learning in Professional and Vocational Education. London: Routledge.
• Crisp B (2007) Is it worth the effort? How feedback influences students' subsequent submission of assessable work. Assessment and Evaluation in Higher Education Vol 32 No 5 October 2007 p. 571-581
• Crooks T (2001) The Validity of Formative Assessments [Internet] Paper presented to the British Educational Research Association Annual Conference, University of Leeds, 13-15 September 2001 Available from www.leeds.ac.uk/educol/documents/00001862.htm [accessed 20^th July 2011].
• Defeyter and McPartlin 2007 Helping students understand essay marking criteria and feedback. Psychology Teaching Review 13 (1) 23-33.
• Gibbs G and Simpson C (2003) Does your assessment support your students' learning? Centre for Higher Education Practice, Open University BEE-j Volume 2: November 2003.
• Gibbs G and Simpson C (2004) Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education Issue1 2004-5 p. 3- 31
• Kirton, Hallam Peffers, Robertson and Stobart (b) (2007) Revolution, Evolution or a Trojan horse? Piloting assessment for learning in some Scottish primary schools. British Educational Research Journal Vol 33 No 4 August 2007 p. 605-627.
• Maclellan, E. (2001) Assessment for learning: the differing perceptions of tutors and students. Assessment and Evaluation in Higher Education, 26(4), 307–318.
• Nicol and MacFarlane-Dick (2006) Rethinking formative assessment in HE: a theoretical model and seven principles of good feedback practice. Studies in Higher Education Vol. 31, No. 2, April 2006, pp. 199–218
• Ramaprasad (1983) On the definition of feedback. Behavioural Science 28 4-13
• Ramsden, P. (2003) Learning to Teach in Higher Education (2nd edn) London: Routledge Falmer.
• Sadler, D.R. (1989) Formative assessment and the design of instructional systems. Instructional Science, 18, pp119-144.
• Vygotsky, L.S. (1978) Mind and society: The development of higher psychological processes. Cambridge, MA: Harvard University Press
• Walker (2009) An Investigation into written comments on assignments: do students find them useable? Assessment and Evaluation in Higher Education Vol 34, No1 February 2009 67-78
• Ward (2008) Assessment for Learning has fallen prey to gimmicks, says critic. Times Educational Supplement 17 October, 2008.