‘So, you want us to do the marking?!’ – peer review and feedback to promote assessment as learning
By Caroline Elbra-Ramsay & Anita Backhouse, Faculty of Education and Theology, York St John ÌìÃÀ´«Ã½
Contacts:c.elbraramsay@yorksj.ac.uk/ A.Backhouse@yorksj.ac.uk
Abstract
The aim of this small scale research study was to investigate whether a primary school formative assessment strategy could be transferred to Higher Education. Visualisers are an increasingly used teaching and learning resource in primary classrooms. Using examples, work is analysed and improved allowing for explicit scaffolding of feedback strategies as well as the unpicking of success criteria. This study sought to use a similar approach with part-time PGCE students, many of which were returning to education after a break. Using two curriculum modules (science and English) example assignments were analysed and improved using the visualiser to prompt discussion as a group and with partners. This then fed into peer and self- marking/ evaluation with the opportunity to apply some of the strategies explored earlier.
Data (questionnaires, interviews and assignment feedback) was collected before, during and after these scaffolded sessions and indicated that the students found it to be a valuable strategy in terms of exploring some of the tacit aspects of academic writing. The immediacy of the feedback, dialogic nature of the sessions and common shared understanding that the visualiser allowed all seemed to have impacted positively.
Key words: feedback, assessment, academic writing, visualiser, student teacher, modelling, analysis, science, English
Background to the study
The part-time primary postgraduate programme at York St John ÌìÃÀ´«Ã½ spans an 18 month period and includes academic study focussing on professional skills and attributes as well as curriculum-focussed modules. Students are encouraged throughout to make links between school-based experience and there is a strong underpinning philosophy of reflective practice and enquiry-based learning.
This small-scale study focussed on a cohort of 80 student teachers in the third month of their programme. The cohort comprised of many mature students who had been out of an academic environment for an extended period. Based on evidence and feedback from previous cohorts, the consequence of this was that confidence, in terms of reflective and critical thinking and subsequent transfer of these skills to academic writing, was likely to be low.
Two non-Masters level curriculum modules were chosen for the study based on helping to develop the students' skills in analysing evidence and linking theory and practice. The aim was to help students to engage in a formative process using peer review and feedback in order to facilitate assessment as learning (Black and Wiliam 1998) in order to become critical reflectors of their own work.
The investigation made use of visualisers which are essentially document cameras that connect to a computer. Images of objects or documents can be displayed, in real-time, via the screen to large audiences. Clarke (2013) suggests that the use of visualisers can be used effectively in the classroom to help 'signpost' those aspects of written text that learners need to consider when comparing and contrasting the quality of each other's work. Visualisers were an important tool in this project to help the students to develop a shared understanding of high quality academic writing through discussion and peer review, mediated and scaffolded by the tutors.
The project consisted of:
- a base line questionnaire to establish students' confidence in academic writing;
- two hour-long workshops focused on specific assessments;
- an analysis of completed written work;
- recordings of student discussions;
- post project questionnaire.
Supporting Literature
Research indicates (Brown and Glover, 2006, Crisp, 2007, Maclellen, 2001) that students are more interested in summative grades than formative commentaries and it is often the case that feedback is unread or not fully embraced and acted upon. Even with the best intentions, feedback does not effectively close the 'learning gap' (Sadler, 1989, p.121). This difficulty is further compounded within Higher Education when learning can be quite specific to modules, leaving little opportunity for feedback to inform future submissions.
A feature of effective formative feedback is that it is based on a socio- constructivist (or dialogic) approach (Gibbs and Simpson, 2004). This assessment dialogue (Clarke, 2007) enables learners to construct their own meaning by translating a judgement into a 'next step' by scaffolding or explaining both the gap and how to close it. Peer feedback strategies result in learners acting as co-responders (Van den Berg, Admirall and Pilot 2006) in collaborative tasks and are a useful mechanism for encouraging a more dialogic approach to the receiving, understanding of and acting upon feedback. Through it, learners articulate their understanding to others, assisting both the giver and receiver in actively constructing knowledge (Liu and Carless 2007.) Higher Education includes aspects of complex tacit knowledge that can be difficult for tutors to define and students to decipher; working alongside a peer allows this knowledge to be actively engaged with using a common understanding and language. Indeed, Nicol (2010) identifies that feedback should be expressed in a language that students understand if it is to be successful.
Actively engaging with the work of others has a consequential impact on the ability to monitor self-performance which Sadler (1989) recognises as the ultimate goal of feedback. Giving effective feedback to others is dependent on making critical connections between prior knowledge (Liu and Carless 2007) and desired understanding/performance, encouraging the development towards self-regulated learning (Butler and Winne 1995) or 'learnacy.' As such, it moves from 'assessment for learning to assessment as learning' in that it is 'an active process of cognitive restructuring that occurs when individuals interact with new ideas' (Western and Northern Canadian Protocol for Collaboration in Education, 2006 p.41).
However, working with a peer does not in itself develop learner self-regulation. Nicol and Macfarlane- Dick (2006) identify several necessary elements including the need to clarify good performance or the desired standard. Seeing an example of the desired goal goes some way towards this but relies on the learner having the necessary skills of evaluation and analysis to identify why and how it is the target. Learners need opportunities to develop these skills through modelling, supporting and scaffolding. Increasingly primary school settings do this by making use of a visualiser or document viewer enabling learners to not only see a desired model but, in collaboration, scrutinise its successful features and possible improvements (Clarke, 2013).
An additional benefit of peer feedback is its immediacy. The context of a workshop also means that there is time to not only receive but act upon feedback, which Sadler identifies when he states 'the student must engage in appropriate action which leads to some closure of the gap' (Sadler, 1989, p. 121). Receiving feedback from tutors can frustratingly be several weeks after the work was completed which compromises the perceived need to act upon it. It could be argued that tutor feedback is worth waiting for but, as Gibbs and Simpson (2004 p.19) state, 'there may be a trade-off between the rapidity and quality of feedback so that, for example, imperfect feedback from a fellow student provided almost immediately may have much more impact than more perfect feedback from a tutor four weeks later.'
Focus of the enquiry
Clarke (2008) advocates the use of a visualiser as a key tool in the modelling, identification and explanation of success criteria. In this way, learners are explicitly taught 'learnacy' or 'learn to learn' an underlying aim of assessment for learning (Sadler, 1989; Black et al, 2006). This project investigated the effectiveness of this approach in an HE setting.
Research methods
According to Cohen, Manion and Morrison (2010, p.289) 'a case study provides a unique example of real people in real situations'. This case study focussed on two cohorts of part-time student teachers and two course tutors working within two curriculum-focussed modules.
Data was collected in three ways: through surveys; audio recordings and document analysis. Informed consent was given by the participants for the surveys, audio recording of sessions and scrutiny and analysis of their written work.
The first set of data was collected via a survey given to an outgoing cohort of students to establish what issues they felt there were in terms of their academic writing (27 responses received). Outcomes from this survey were used to set the focus for subsequent work with the incoming cohort. A second survey was given to the incoming cohort to establish their levels of confidence in terms of the technicalities of academic writing (punctuation and referencing, for example) and their level of understanding in terms of linking theory and practice and in the analysis of writing. This second survey was administered prior to the workshops and before any written assignments were embarked upon (53 responses received).
Two workshops were delivered and attended by all the students in the cohort. They were designed to follow a similar theme with one focussing on the link between theory and practice and the other focussing on analysis of children's work.
Prior to each session, the student teachers were given an initial writing task. Using the model of the assessment itself, each was given a selection of reading for the science task and evidence relating to an individual child's reading for the English task.
Each workshop began by displaying a sample report on the visualiser as a prompt for the group. This was followed by a second sample of a higher quality. Students were asked to compare the two, identifying which was the most successful and why. The tutor highlighted each sample accordingly and scaffolded the discussion by focusing the analysis on key academic areas and by modelling alternative improvements. This comparison and exploration enabled examples of tacit academic knowledge to be explored, for example, depth of analysis.
Students then applied this process to their own work. Working in pairs, the students identified strengths and opportunities for improvement using some of the ideas modelled by the tutor earlier. Mini-plenaries were held periodically during which the tutor displayed randomly selected samples of this work on the visualiser and encouraged discussion about the quality of improvements and other potential possibilities. Students were then encouraged to revisit their own work and apply similar improvements, if appropriate. The annotated drafts were copied and returned to the students at the end of the session in order to inform their future written submission.
Observations were carried out during the workshop sessions and audio recordings taken of the student discussions. Further data was collected by scrutinising the copies of the students' annotated written work from the workshops and subsequent written assignments (85 participants). A final survey was given to the second cohort at the end of their programme, one year later, to establish how their feelings about academic writing had changed over time.
Theoretical frameworks
Through in-depth qualitative analysis of this wide range of data a rich and informative picture was gained in terms of the students' levels of confidence and understanding of the demands of academic writing. Their construction and re-construction of ideas through peer review and dialogue was monitored. Qualitative analysis provided an insight into social constructivist and cognitive apprenticeship models (Dennen and Burner, 2008).
The overall findings of the research were:
- a high level of student engagement in the tutor-led visualiser sessions;
- increased levels of student perception and awareness with regard to assessment criteria and expectations in terms of academic writing;
- increased student understanding of academic writing convention and tacit knowledge.
First cohort end of programme data collection
To aid the design of the workshops, an initial questionnaire was conducted with the cohort who were reaching the end of their programme. There were 29 responses to the questionnaire. The questionnaire was analysed to provide information about levels of confidence in academic writing. Key areas of concern included: integrating sources, analysis / criticality and constructing an argument.
As a further indicator, the completed assignments from this cohort were marked according to ten specific areas related to academic writing:,
It was felt that while there were issues with regard to the technicalities of writing, in terms of sentence structure and punctuation, the main areas of significance were those involving linking ideas and concepts, depth of analysis and linking theory and practice. These three key areas would provide a focus for the workshops.
By the time this cohort had reached the end of their programme there were still a significant number who lacked confidence and competence in terms of the quality of academic writing required at this level.
Second cohort pre-workshop survey
Based on the outcomes of this initial data collection from the previous cohort, the same baseline survey was given to the cohort who would attend the workshops.
Cohort 2 appeared to be overall more confident in their abilities at the start of their programme compared to Cohort 1 at the end of theirs.
Again, Cohort 2 has a higher degree of confidence in their understanding of criticality than Cohort 1.
The outcomes here are similar; however, it is significant that a large number of students are unclear about what the meaning of 'analysis' is.
Workshop Pre- tasks
The English task was focused on the analysis of children's word level reading. Varied assessment samples from one child were provided (phonic assessments, common word recognition, miscue analysis) and the students were asked to analyse strengths and weaknesses and then use these to inform future targets and teaching activities. Wordage was limited to 200 words for this report.
In preparation for the science workshop the students were given two pieces of reading to do – one linked to theory and the other to effective classroom practice. They were asked to write 300 words about the role of questioning in science. Analysis of the writing revealed the following outcomes in terms of the quality of the academic writing:
Despite the students' perceived levels of confidence in academic writing there were clearly some students for whom there were problems. This then provided the focus for the modelling, discussion and peer review during the subsequent workshop session using the visualiser.
Student feedback on the workshops
Overall, student responses were very positive. Many of them welcomed the tutors taking the time to explain what was meant by 'academic writing' and said they wished someone had explained it to them in their undergraduate degree. There were a number of 'light bulb' moments in the workshops when students realised the need to support their writing with references to literature or evidence in terms of children's work. Many students said that using a highlighter pen to emphasise the specific aspect of linking theory and practice on the visualiser was useful and it helped them to appreciate what was required. They seemed to engage with this well when analysing each other's writing and used the highlighters effectively. In the pre-workshop writing task the quality of writing was very variable but post-workshop analysis of the peer-corrections revealed a good level of understanding. The students were provided with a summary guidance sheet following the science workshop in order to address some of the 'technicalities' of effective writing and their on-going concerns after the workshop. This was in response to tutor review of the peer-marked task and post-it note questions that the students attached to their work.
Difficulties that were experienced during the first science workshop included an over-focus on referencing conventions by the students. This was clearly an area where they felt insecure but it did detract from the purpose of the workshop. This was addressed in the two remaining workshops and the students were directed to referencing support on the library website. This suggests that referencing input is probably a priority at the beginning of the programme to help reassure the students from the outset.
The English workshop was also well received by the students and many comments were made about how useful the process had been. Key learning that arose was the need to support findings with evidence, examine why particular patterns may be emerging in the children's learning and make targets and activities very specific and focused on the findings of the analysis. A slightly longer session meant that three model assignments could be analysed, two of which were considered to be 'good' but in different ways. The comparison of these two samples in particular really developed the students' understanding. Annotated drafts indicate a good awareness of what they will need to do next. To further support their understanding of referencing the students were provided with hard copies of the library guidance during the English sessions and use to address queries as they arose.
Cohort 2 post-workshop assignment analysis
Once the modules had come to an end, both the English and science assignments were analysed based on the three key areas for consideration. Responses indicated that the students who took part in the workshop were less likely to have issues with: linking ideas/concepts, linking theory/practice and analysing in sufficient depth.
In addition, a simple questionnaire was completed by the students at this time. The questionnaire consisted of two part questions: firstly a Likert scale to evaluate student perceptions of the usefulness of the workshops and also text responses to explain further.
1.In response to the question 'How useful do you think the workshops were in helping you to develop your academic writing?'
Useful in terms of explaining what is required of academic writing and clearing up misconceptions.
Useful because they gave me a chance to discuss success criteria
Useful but I needed more support from writing support tutors with regards to critical thinking.
Useful because it enabled me to focus my answers in an academic style.
They were useful to an extent however they could have been more beneficial if we had the opportunity to discuss with someone how to improve our writing in this context.
2. In response to the question 'Did the workshops support you in writing your module assignments?' the following responses were received:
Could draw on some of the success criteria I had a clearer understanding of 'the rules'.
I knew what aspects of essay I was expected to cover.
They allowed me to identify links between theory and practice.
The workshops helped me to understand what style of academic writing was required and gave me the opportunity to discuss my writing with the tutor.
It helped because I was able to utilise the advice I got from the tutor in terms of strengths and weaknesses, especially with regards to writing in a constructive and analytical way.
3. Overall, at the end of your programme, how do you feel the quality of your academic writing has improved in terms of your ability to link theory with practice and analyse critically?
Achieving a 74 in my research project was confirmation for me that my writing had improved.
Definite improvement, but could still improve further
Not much. I still have a long way to go in terms of explaining my ideas critically.
I think that I was competent in this regard, but the programme has definitely enabled me to nurture my writing skills.
It has vastly improved. I hadn't previously done any academic writing and the help and guidance I received enabled me to pass all my assignments. Thank you for your valuable input!
Amazingly well I could not have achieved had it not been for tutor support.
M level writing stayed very much in the C boundary so I don't think that it helped me to improve a great deal.
Discussion and conclusion
Evidence seems to suggest that the workshops developed both the student teachers' perceptions of their abilities to construct an academic and also their actual capabilities. The data collected from the Cohort 2 assignment implies that students were generally more capable in the three areas considered than they had been during the pre-workshop task. The overall level of competence was also better than that of Cohort 1. Feedback during the workshops indicated that the specific focus on those aspects of academic writing that the students felt less confident about, was effective. Tutor modelling using the visualiser, discussion and peer review in small groups / whole class all contributed to a raising of awareness about what constitutes quality academic writing. The high level of collaborative engagement and subsequent impact on understanding indicates that the learners were engaged as co-responders (Van den Berg, Admirall and Pilot 2006) within an assessment dialogue (Clarke 2007).
The structure and position of the workshops meant that feedback was both timely and purposeful (given the similar summative assessment due later in the term) and was immediate, a key indicator of effective feedback. The overall approach was felt by the tutors to be effective for this reason and would support Gibbs and Simpson's suggestions about the timeliness of feedback (2004).
The need for a shared understanding of the vocabulary for academic writing between students and tutors was also highlighted; a sharing of the criteria for success. As indicated by some of the student responses, it is important that students know what the tutors are looking for and to this extent dedicated time spent on this appeared to be productive. Students needed to experience the explicit unpicking of some aspects of (tacit) academic knowledge and associated conventions to help them construct their own meaning and engage fully with it. Bloxham (2010) identifies the crucial role dialogue plays in this process; the use of the visualiser encouraged this.
There is also some indication of the workshops as a form of 'feed forward' in that students appear to have recognised how they were useful with their later submissions. It appears this consequence is related to their developed understanding of tacit academic knowledge rather than their ability to self-regulate or self-efficacy which is understandable given the small number of workshops that took place. It would be worth exploring this feature in more depth in future longitudinal studies.
There are some limitations with regard to any firm conclusions that can be drawn, however, and it is worth bearing in mind that questionnaires and assignments were collected from different cohorts. As such, the groups studied were distinct in size, experience and understanding. The assignment data collected and analysed was very much dependant on the specific learning context. For example, the English assignment did not allow opportunity to both effectively measure a student's ability to link theory and practice and explore this aspect during the workshop. Consequently it is clear that such workshops need to be carefully constructed around specific assessment criteria relevant to the module.
The tutor modelling and visualiser approach has now been deployed across a number of modules in order to help students and tutors share the vocabulary and develop and construct a shared understanding of what quality academic writing looks like.
References
Black P and Wiliam D. (1998) Inside the Black Box: Raising Standards through Classroom Assessment. London, King's College
Black P, McCormick R, James M and Pedder D (2006) Learning How to Learn and Assessment for Learning: a theoretical inquiry Research Papers in Education Volume 21, Issue 2, 2006
Bloxham S and Campbell L (2010) Generating dialogue in assessment feedback: exploring the use of interactive cover sheets Assessment & Evaluation in Higher Education Vol. 35, No. 3, May 2010, 291–300
Brown and Glover ( 2006) Refocusing written feedback 13th Improving Student Learning Symposium, Improving Students Learning: Improving Student Learning Through Assessment, London 5-7 September 2005, Alden Press, pp 348-357
Butler, D. L. & Winne, P. H. (1995) Feedback and self-regulated learning: a theoretical synthesis Review of Educational Research, 65(3), 245–281.
Clarke S (2007) Tracking Significant Achievement Education 3-13 Education 3-13, Vol 25 No 3, p. 52-58 Clarke (2008) Active Learning through Formative Assessment London, Hodder
Clarke (accessed 23rd July 2013)
Cohen, Manion and Morrison (2010) Research Methods in Education (6th Edition) Oxon, Routledge
Crisp B (2007) Is it worth the effort? How feedback influences students' subsequent submission of assessable work Assessment and Evaluation in Higher Education Vol 32 No 5 October 2007 p. 571-581
Dennen, V. P., & Burner, K. J. (2008). The cognitive apprenticeship model in educational practice. In D. S. Jonassen (Ed.), Handbook of research on educational communications and technology (pp. 425-440). Routledge
Gibbs and Simpson (2004) Conditions Under Which Assessment Supports Students' Learning Learning and Teaching in Higher Education, Issue 1, 2004-05
Liu and Carless (2007) Peer feedback: the learning element of peer assessment Teaching in Higher Education 11:3, 279-290
Maclellan, E. (2001) Assessment for Learning: the differing perceptions of tutors and students, Assessment and Evaluation in Higher Education, 26(4), 307–318.
Nicol and MacFarlane-Dick (2006) Rethinking formative assessment in HE: A theoretical model and seven principles of good feedback practice Studies in Higher Education Vol. 31, No. 2, April 2006, 199–218
Nicol, D (2010) From monologue to dialogue: improving written feedback in mass higher education, Assessment and Evaluation in Higher Education, 35:5, 501-517
Sadler, D.R. (1989) Formative assessment and the design of instructional systems. Instructional Science, 18, 119-144 Van den Berg, I., Admirall, W., & Pilot, A. (2006). Designing student peer assessment in higher education: analysis of
written and oral feedback. Teaching in Higher Education, 11(2), 135-147.
Western and Northern Canadian Protocol for Collaboration in Education (2006) Rethinking Classroom Assessment with Purpose in Mind Manitoba, Manitoba Education
address
Academy for Learning and Teaching Excellence
ÌìÃÀ´«Ã½
ÌìÃÀ´«Ã½ Square
Luton, Bedfordshire
LU1 3JU