'But that's your job!': peer assessment in collaborative learning projects

Christine Crowe and Alec Pemberton
Department of Social Work, Social Policy and Sociology
University of Sydney

One of the assessment strategies used in a large undergraduate unit of study in Sociology at the University of Sydney involves the use of a group research project. In The Social Construction of Difference students are encouraged to engage in critical analysis of the relationships between various theoretical perspectives and case studies which highlight theoretical strengths and shortcomings. Collaborative group work encourages students to rely on their peers as another intellectual resource for their learning, and promotes social relationships which may continue throughout students' university life. Moreover, setting a collaborative assessment task encourages the development of crucial generic attributes or transferable skills, such as the ability to work with others and the development of written and oral communication skills, all of which relate to key aspects of real-life work situations after graduation.

This paper firstly outlines the method used to derive an individual mark from a group project using peer assessment. The paper then discusses students' initial and subsequent responses to the use of peer assessment, and gives examples from students' own experiences of group work as they relate to the assessment strategy. There is a general congruence between the individual mark received in the group project and subsequent mark for an individual piece of work. In cases where this does not occur, an examination of students' comments made in the peer assessment form reveals possible reasons for such non-congruence. The aim of this paper is to examine the relationships among group tasks, group rewards and individual accountability. We show that collaborative group learning enhances individual learning not only about the issue at hand, but may also contribute to the acquisition of establish generic attributes, or transferable skills, which benefit students in subsequent group endeavours, both within the academy and later on in their work life.

Peer assessment: rewarding individual effort in collaborative group projects

The assessment technique used in The Social Construction of Difference has three main goals: to enhance the quality of student learning, to discourage 'free-riding', and to reward individual input and effort. The challenge was to derive an individual mark from a group project mark. In particular, we wanted to allocate marks to reflect and reward individual input and effort. This is relatively straightforward for cooperative learning, in which individual students take responsibility for, and are assessed on, one section of a project. It is much more complex in the case of collaborative learning, in which students engage collectively in the entire enterprise. Collaborative learning is more suitable for the purposes of this unit of study, as it is recognised as more successful in bringing about conceptual change and the promotion of metacognitive skills.

The following technique has been developed over the past two years and has proven to be very successful. Its focus is on process, rewarding input as well as the final product. Our approach is firstly to award students a group mark for the group project. An individual mark is then derived from the group mark by using peer assessment. Students have been fully informed of this process in their induction into the unit of study and the course outline. Students may use the ten-point criteria suggested in the course outline, and are also encouraged to offer their own suggestions for alternative criteria. The criteria in the course outline criteria identify issues such as attendance, preparation, initiation, achievement of tasks, contribution to group discussion (in light of the learning outcomes rather than 'off the cuff' speculations), evidence of contribution to the learning of the group (summarising articles, distributing photocopies, book reviews, research initiatives etc).

On the day of submitting their project, students are asked to give all members of their group a mark out of ten for all ten criteria. To ensure as much confidentiality as possible, we ask students to sit apart from other members of their group while they complete their peer assessment. We find that students take their obligations very seriously, and the process takes at least one hour to complete. The judgment is made according to how much they think that a particular member of their group has contributed to the learning of the group and the submission of the group project. If all students contribute equally, then all students are entitled to 100 per cent of the group mark. If, however, some students in the group had poor attendance, or the quality of their input was poor for some reason (lack of preparation, failure to fulfil task obligations, for example) then they may receive a proportion of 10 for each criteria, such as 5, for example.

The overall mark for an individual student consists in the total percentage that students give each other in relation to the input of other students in the group. The judgment for the distribution of marks is for the students to make. This must relate directly to the number of people in the group. If the group consists of five students, then the sum of the each criterion must add up to a total of 40 (ie students do not give themselves a mark). To demonstrate how this example might work, imagine one student in the group receives 7 out of 10 for each criteria. This means that another student, who made more of a contribution, can be rewarded for his or her effort by receiving an assessment of 13 out of 10. Another scenario in this example could entail three other students receiving 11 out of 10. Students invariably give a range of marks according to the criteria and are asked to provide evidence for their assessment. This is one or two sentences, relating to the particular criteria, as to why they gave the specific mark. Some students initially baulk at the idea of providing written evidence. Once they begin, however, the majority usually provide much more than the required two sentences.

Using a ten point criterion, with marks out of ten for each criterion means that an overall mark out of 100 is achieved. If the group project received a mark of 70, and all students contributed equally, then this would also be their individual mark. If, on the other hand, the individual student's mark, as derived from the above method, is that of 80 out of a possible 100, they have not performed as well as others in the group. In this case the individual student receives a mark which is 80 per cent of the group mark of 70, or 56%. It is this mark, in this case 56%, which is recorded next to the student's name as their particular mark for the group project.

At the same time, and according to the distribution of the other marks in the group, some or all others, who have worked and contributed more, will receive an individual mark which is more than the original group project mark of 70. A student who receives an overall peer assessment mark of 110 out of 100 will receive 77% for their group project.

Staff remain the ultimate arbiter should there be disagreement. Initial concerns relating to the number and quality of disagreements proved to be unfounded. Over the last two years, we have encountered only two cases, from forty-eight groups, where staff intervention was called for. Both these cases were satisfactorily resolved. The lack of disputes is possibly the result of the clarity of the assessment strategy and the high degree of congruence between the individual mark received in the group project and the subsequent mark (teacher-assessed) in individual essays which comprise the other major piece of assessment. The comments given by students as evidence for marks provide insight into the process of collaborative learning in general, and can indicate possible reasons for non-congruence.

Benefits of the peer assessment strategy

Using this technique has many benefits. It promotes students' confidence in their ability to assess the work of others and provides the opportunity to develop skills for working in a team. The feedback from students has been positive. They felt that this technique, in comparison with a 'blanket mark' where all students in the group received the same mark, was fair. Students also pointed to the benefits of 'finding out about how they work with others', some, admittedly, with a greater degree of approbation than others. The next step in the implementation of this technique is to reduce the amount of manual calculation done by staff in the formulation of the individual mark from the group mark. Here the use of information technology is vital. Our ideal scenario involves students' having access to a web page which is confidential, accessed by user name and individual password1.

Allowing students to take significant control and responsibility for their assessment may initially be viewed as challenging (by teachers and students). The success of this technique depends upon students having an appreciation and understanding of how this process will be accomplished and seeing the benefits for their own professional development. The students' induction to the assessment strategies during the introduction to the unit of study, the clarity and of the strategies in the written course outline, and the availability of teachers for consultation are essential ingredients. From our experience, the clear criteria and the request for 'evidence' assists students to make informed judgments about the contribution of others, and works against students 'colluding' to give each other full marks2. The benefit for teachers is evident; the validity of assessment strategies is enhanced when more than one assessor is used. Who better to include in the assessment of students' work than those students who have engaged in collaborative projects? This technique reduces the amount of staff marking whilst at the same time delivers an individual mark and enhances student learning.

Student responses to peer assessment:

Many students initially expressed concerns about peer assessment. Some students felt that teachers were shirking their responsibility by asking students to assess each other. They initially believed that teachers should be the only assessors: 'But that's your job!', a comment by one student, encapsulates the general tone. A second and related issue was that students felt that they were unable to make judgments about other students, and that they did not want to be responsible for another student possibly receiving a low grade. Students commented that what we asked involved 'too much responsibility' for them; they 'couldn't possibly do it', because they are 'not objective'. A number of students stated that because they felt unable to assess the contribution of members of their group they would have no option but to allocate all members 10 out of 10 for all criteria. Interestingly, these responses were given before the commencement of the group project.

An analysis of the peer assessment marks, and the reasons given by students for allocating marks, shows clearly that during the course of collaborative learning tasks students become very aware of the quality and quantity of other students' input to the project, and are willing to formalise their judgment. The majority of students wrote at least two pages of 'evidence' to justify their mark, and overall there was a great degree of consistency in the judgments made by members of the group; quality input was recognised and rewarded. Many students showed a clear understanding of individual members' attributes for group work, in terms of both group process and output. One student wrote in their summation:

Anna: a lovely person who was very accommodating to the group's conflicts, worked hard at keeping the group working without blowing up with all the stress flying around; had good ideas
Amy: was very imaginative with ideas on how to approach the topic, worked at keeping the group together, a good mediator who helped us to reach decisions we were all comfortable with
Nick: heaps of reading to add to the group at meetings, piled the information on
Hannah: comfortable questioning the way we were headed to get us back on track, stopped us babbling on too much
Phillipa: hardworking, punctual and eager to do a good job

Importantly, those who did not 'pull their weight' were allocated marks which reflected their input. Some students also included comments which showed their unease about allocating low scores. In one case a student wrote extensively about how the five members of the group worked together. One member received a total group assessment of 40 out of a possible 100. The assessing student writes: '.....so I feel bad, I don't know what to think or do, but she never showed up when she said she would, and we had to do extra work'.

Two other themes related to the group processes emerged in students' assessment of the work of others. Firstly, students consistently made reference to group process in terms of democratic principles. They stated that their group was 'very democratic', that there were 'no leaders', 'no-one dominated', 'everyone was heard' and that there was 'lots of cooperation'. Those whose group processes were not as easy to manage used terms such as 'domination', 'inflexibility' and 'taking over', and 'not listening to others'.

The second theme to emerge was reflection about whether or not they would work in future with the same members of the group. Students offered brief summations: 'I would love to be [a] team member with these people in the future', 'would be happy to work with this group again'. In groups which contained individual members with different input, comments included: 'Smart but we had to take up the slack'; 'working with 'Emily' was a nightmare! Never listened, only imposed her views' and 'Never again, too damn (sic) lazy!'. Students also made comments which suggested that knowledge gained about group work will be carried or transferred to similar future tasks. These insights related to both process work and task management. Students commented that in the future they would 'make it clear that there has to be communication especially about not showing up', and that 'there has to be rules right from the start' about fulfilling responsibilities to the group. Other students indicated that they saw the value of group determined deadlines, and would in future highlight the importance of meeting those deadlines. One student writes:

Headway was slow but once we got organised with primary sources, we found that it all fell into place and everything flowed pretty much. It was hastily approaching the due date, but this was when we all gelled a bit - due dates helps one to dismiss irrelevant and superfluous information more easily ... we should have made ours 'stick' more - much less stressful!

Students also gained insight and confidence with regards to their ability to appraise the work of others. Several students from different groups approached us at the end of semester and wished to talk about why they gave one or two members of their group 'such a low mark'. They felt that they had 'no choice' but to judge those students' performance as poorer than that of others. Although no details were given by teachers, we pointed out that the individual mark received by students was a culmination of the entire group's criteria-based judgments. These students realised that their own assessment of the other students' contributions had been validated by other members of their group.

The relationship of group to individual assessment: insights from peer assessment

There was a great deal of congruence between the individual mark achieved in the group project and that received in a subsequent (teacher-assessed) essay. An examination of the 'evidence' given by team members can give insight into possible reasons for non-congruence. In one case, a group received a group mark of 70%. One member received 62% as their individual mark, and a subsequent mark of 75% for their individual essay. Upon reading the comments given by peers, it seems that attendance, initiative, input and 'flexibility' (a student-initiated theme) from this student were poorer than that of others. Comments about 'Paul' from team members included:

Paul came to meetings extremely unprepared. He showed no evidence of any research until the end of the project ... As a result of his lack of research/reading we all had to allocated extra tasks ... Also arranging meeting times with Paul was difficult as he was inflexible, often leaving meetings early.

Paul attended our meetings clearly unprepared ... Whilst the rest of the group was accommodating and flexible when it came to meetings and writing the actual project, Paul was inflexible and often difficult in arranging to meet ... When it came to task management and project outcome Paul's lack of contribution until the very end created an extra burden upon the other members ... We had to not only complete our own tasks but divide up his uncompleted (sic) work between us, picking up his slack

Paul did not show any evidence of doing any readings until the last day of writing the report ... The other group members tried to be accommodating in arranging meetings but Paul never seemed able/willing to make time

Although team members acknowledged 'Paul's' intelligence, they did not want to work with him in the future: 'Whilst he is intelligent ... he does not work in a group'. Another member stated: 'The only good thing I can say is that he is really smart so this helped us in the clarification of our ideas ... but he didn't do any of his tasks'. Although this student apparently contributed to writing the final draft, other members of the group perceived this input as 'at the last minute', 'running off with their work'; they felt that the student was 'acting high-handed' and 'taking over and imposing views' on the others.

Since one of the learning outcomes highlighted in this unit of study involved demonstration of the ability to work with others in a team, it is clear that 'Paul' did not successfully achieve this learning outcome; the overall grade subsequently reflected this. Although it could be suggested that the group project may not have received the initial 70% had it not been for this student's working on the final draft, it could be argued that had the student engaged in collaborative group learning from the beginning, the learning of all members of the group may have been enhanced, and thus an even higher group mark may have been achieved. It is impossible to speculate further on this issue. As it turned out, the subsequent individual essay marks of the other members of the group did not fall below 70 per cent. This suggests that the group project mark was achieved without 'Paul's' last minute 'intervention'.

Discussion:

Collaborative learning for a group project using peer assessment presented challenges for students. They were forced to reconsider their role in higher education in general and their responsibilities to others in their group in particular. Some groups worked very well together in terms of process and production of the project. Whereas many groups found it challenging to work collaboratively, they did not necessarily see the experience of group work as negative. Indeed, students' comments reveal that not only was learning accomplished for the task at hand, but many gained insight into group process, the management of conflicts (at least by witnessing others' management) and how to improve oral and written communication. There is evidence to suggest that knowledge of group work developed from this experience will be carried or transferred to group tasks in the future, both in any subsequent university assignments and at work following graduation.

It should be clear by now that we certainly think that assessment of students' work is our job. But equally we think that a student-centred approach to such assessment takes new and fruitful directions - including actively involving students in the assessment of their own learning. Our experience in the unit The Social Construction of Difference strengthened our conviction that strategies such as students' evaluation of their peers' contribution to a group project must be geared where possible to enhancing the development of generic attributes or transferable skills that enhance students' effectiveness both in the classroom itself and later on in their working lives.

We will pursue two particular themes. In the first place we wish to develop the unit of study (The Social Construction of Difference) by incorporating the student-initiated themes of 'democracy' and the 'ability to be a team player' - both of which are key aspects of group process in the learning situation - within the formal criteria for peer assessment. In that sense we are widening our understanding of student-centred learning to include criteria with an authenticity for students' experiences of collaborative learning and group tasks. In so doing we hope top enhance the validity of the criteria used in relation to the peer assessment task.

In the second place we are strengthened in our view that formalisation of the criteria highlights their significance as part of group work. Significant effort must always be put in to the induction phase and the spelling out of formal criteria and expectations in the course documents (outlines, website etc). Bringing these to the attention of students in future courses may curtail some of the behaviour which was reported as 'problematic' in collaborative learning and the achievement of group tasks. There is of course no 'end point' to the ongoing process of evaluating, challenging and changing in the curriculum cycle. We do have expectations, however, of refining the process of peer assessment within the framework of student-centred learning.

Footnotes:

1. The web page will contain the assessment grid, that is, the 10 criteria, columns for marks, and spaces for justifications for marks. Ideally, the spreadsheet will automatically calculate the overall peer assessment mark for each student. (back)
   
   
2. Approximately 15 per cent of groups indicated equal participation in group learning and production of the project. Members of these groups provided as much 'evidence' as other groups. (back)