Conducting a Marking Meeting with New Teaching Staff

By Margaret Buckridge, Griffith University

Introduction: Why is marking an important dimension of sessional teaching?

For both full-time and casual staff new to teaching, assessing student work can generate anxiety and consume a great deal of time and effort. This is true of both aspects of assessment: making a judgment about the quality of the work and providing feedback that will help the student to improve the quality.

There is much at stake if assessment is done badly, since information about how well you have performed is an integral part of learning. If work that is poor is given high marks, or if work that is good is given low marks, the process of learning will be quite damagingly de-railed. This is true of feedback also. If the quality of students’ work has been judged accurately, but they cannot understand the explanation as to why their marker came to that conclusion, their future attempts to improve its quality will probably not work.

What kinds of difficulties are faced by staff new to marking?

The following comments from sessional teachers suggest some of the ways in which staff have experienced problems in relation to marking students’ work:

• “It is often very hard to work out what the right mark should be. It seems like you have to juggle some pretty good things, which the assignment has done with other stuff that isn’t so good or that isn’t there at all.”
• “I have a real problem with how to balance out the form and content thing - what do you do with a report that pretty much covers what it has to and you can see where it is going, but is really not very well written at all, lots of grammar and punctuation mistakes.”
• “I just hate it when you get something that is kind of pretty good in its own way, even develops a sort of position and makes points relate to one another, but doesn’t really answer the question that they started out with.”
• “I’ve had a couple of assignments recently where you would have to say they were really nicely written - flowed, no grammatical mistakes, good vocab - but I kept looking back over them and they really kind of didn’t bring anything into focus. Like, there were mentions of all the right things, but they didn’t really say anything.”
• “I sometimes have trouble splicing together the criteria and, like, the content. So the criteria say, for example, that they must provide evidence for their points and develop an argument, but, like really, it’s still got to bring in certain kinds of content.”

For the staff making these comments, there is a problem both about what grade to give a particular performance (the summative aspect of assessment), and also about how to formulate feedback that will enable the student to understand why the grade is an appropriate one for the quality of the performance and what they have to do to improve (the formative aspect).

Marking Meetings

To assist with these problems, many course convenors and/or supervisors are recognising the usefulness of meetings which bring prospective markers together to discuss such issues and receive guidance.

• Such meetings are generally focused on a particular assignment or a particular examination
• Markers may be asked to do ‘homework’ in the form of marking 3 papers or this may be encompassed within the meeting time.
• The meeting time would normally be between 90 minutes (where marking has been done previously) and 2 hours (where marking is included)
• Sessional markers should be paid for their involvement.

A possible process for such a meeting is outlined below. It is abstracted from descriptions of marking meetings occurring in different schools at Griffith University. While the process can be adapted readily to the needs of different courses, its usefulness is lost, particularly for new markers, if it becomes too brief or too trivial. Quick agreement on ‘which paper should be awarded what grade’ should not be taken to signify that there has been a deep, cross-team understanding established which will reliably inform all the subsequent marking.

The process that is outlined below is fairly elaborate and would have its most appropriate application in relation to substantial pieces of work - often end-of-semester discursive work such as assignments, project reports, etc - which challenge students to demonstrate the full range of cognitive capacities. For shorter, more limited, or more "componentialised" pieces of work, a much briefer touching base, or a written marking guide, may be sufficient.

The Process

Before the meeting:

• The process will normally be convened and chaired by the course convenor.
• Participants will be asked to be familiar with:
  • The assignment topics, examination or task
  • Directives issued to students
  • Criteria information issued to students
  • Any information about standards issued to students
    (e.g. ‘To meet the criterion of ‘reference to secondary literature’ at Credit level, students must make appropriate reference to two additional scholarly sources beyond the text-book and the course dossier’.)
• The convenor will choose three examples for copying to the whole group, either circulating them before the meeting or providing time within the meeting for staff to read and grade them.
• First marking cycle: Staff are asked to give a mark to each of these sample papers in the terms in which the course reports marks to its students (i.e., A, or HD, or 18/20, or 90%, or Excellent, etc.)

The meeting activities:

1. Convenor opens meeting, welcomes attendees and indicates the nature of the exercise they will go through
2. As a first step, markers are asked to look again at the specifications for the task they are marking and the published criteria
3. They are then asked, on the basis of this refresher, to look again at their three essays and to categorise them on a different basis, this time putting them into one of 4 piles: ‘confused’, ‘minimal’, ‘adequate’, or ‘excellent’, (categories are from Angelo and Cross, 1993, cited in Prosser and Trigwell, 1999) without thinking about (and without knowing) how these categories might map on to their usual spectrum of grades. (That is, without knowing whether something categorised ‘confused’ is going to end up as an irredeemable failure or as a borderline passing grade.)
4. Participants should now, in twos, talk about how they have categorised each of the three essays, discussing this in terms of this particular set of categories and not making reference to their original mark.
5. Second marking cycle: In the light of the categorisation, and also in the light of the discussion with their partner, participants should decide whether they want to change the original mark given to any of the three papers (as denoted using the course marking scheme).
6. A third taxonomy is now introduced: The course convenor (or somebody invited for the purpose) does a brief induction of the participants into the SOLO taxonomy (Biggs & Collis, 1982).
   This is a system of categorisation by which the quality of knowledge or understanding is determined on the basis of its structural character. This can range from ‘pre-structural’ (no sense of what the topic is demanding) through uni-structural, multi-structural, relational, to extended abstract (a highly coherent understanding such that the student can point beyond the topic to broader connections, critique, etc).
   There are a number of websites which provide a good description of this taxonomy: use ‘SOLO Taxonomy’ as the search term.
7. Third marking cycle: Participants return to the three example papers and now assess them using the SOLO categories.
8.   Again, in the light of this exercise, participants should return to the paper’s current mark (as denoted using the course marking scheme) and determine whether they want to change that mark
   
9. Each participant will now be in a position to produce a small table
   

10. Participants should discuss their tables in pairs and make any further adjustments which are suggested by this discussion.
11. Each pair should then report to the whole group, reporting the sequence of marks which they each produced, the extent of their agreement or otherwise, and their rationale for the ultimate mark.
12. Convenor should facilitate a concluding plenary discussion within which any substantial disagreements which remain are analysed by the whole group in terms of the taxonomies which have been introduced. Agreement will not always be reached, but there may be important implications for the future as to the ways in which assessment tasks have been constructed, the kinds of directives given to students and the ways in which the criteria for marking have been framed and communicated.

Workshop Rationale

• Markers are compelled to look hard at the nature of the task and the criteria which they are using to make a judgment
• Markers are inducted into the process of making a qualitative, holistic judgment in the light of the criteria, and are provided with terms and perspectives which make this possible
• Because the focus is on the quality of the work, rather than on its position within a rank ordering process, there is an excellent basis for offering students meaningful feedback
• The disagreements which frequently arise point the way to improved assessment practice.

Acknowledgements: My thanks to convenors and tutors in the Schools of Applied Psychology - Health, Applied Psychology - Business, Marketing and Management and Humanities at Griffith University. The suggested workshop structure brings together components from processes which have been utilised in these (and no doubt other) schools.

References:

Angelo, T.A. and Cross, K.P. (1993). Classroom Assessment Techniques: A Handbook for College Teachers, 2^nd edition. San Francisco: Jossey-Bass.

Biggs, J.B. and Collis, K.F. (1982). Evaluating the Quality of Learning: the SOLO Taxonomy. New York: Academic Press.

Prosser, Michael and Trigwell, Keith. (1999). Understanding Learning and Teaching: The Experience in Higher Education. Buckingham: SRHE and OU Press.