Sharing the Load:
A Collaborative Approach to Developing and Implementing
Criterion-Referenced Assessment in Professional Degree Courses

by

Merrilyn GOOS
and
Karen MONI

Graduate School of Education
The University of Queensland

 

Introduction

This paper focuses on our approach to implementing criterion referenced assessment, which we refer to as "sharing the load". Our particular goals have been to develop coherent and specific sets of criteria and standards for preservice teacher education subjects, and to offer students assessment tasks which are both authentic and purposeful. We discuss five main themes:

  1. collaboration between lecturers from different disciplines
  2. collaboration between students on assessment tasks
  3. the use of formal and informal peer assessment
  4. the design of assessment tasks with multiple purposes and a range of outcomes
  5. criteria and standards drawn from professional education practice.

 

Background

As former secondary school teachers who are now university Education lecturers, we find it satisfying to observe the move away from examination-based, norm-referenced assessment of university subjects towards a greater variety of assessment modes and the use of criteria and standards to judge the quality of students' performance. This is an important area in which university practice has, in general, lagged behind professional practice in other educational sectors; for example, Queensland secondary schools have been required to use criteria and standards based assessment since the mid 1980s (see Board of Secondary School Studies, 1986-1988 for details). The knowledge and skills developed within secondary schools over this time have been influential in shaping our own practice as teachers and teacher educators, and have helped us to design assessment programs which are consistent with the policies and guidelines recently adopted by The University of Queensland (The University of Queensland Academic Board, nd., available at http://www.uq.edu.au/hupp/index.html?page=25109&pid=0).

The aim of this paper is to illustrate some strategies for putting into practice assessment principles endorsed by the university's Academic Board. These principles include the following:

  1. The primary focus of assessment is to encourage, direct and reinforce learning.

  2. The assessment methods employed should reflect the variety of subject and course goals.

  3. Well-constructed self assessment and peer assessment exercises have the potential to provide valuable learning experiences and encourage lifelong learning.

  4. Staff should seek to lessen the threat posed by assessment. This may be accomplished by giving students practice in the particular assessment methods being employed.

  5. Feedback is fundamental to the learning process and some form of feedback should be offered on all items of assessment.

  6. Each piece of assessment is to be accompanied by clear assessment criteria which are effectively communicated to students and markers.

  7. Marks and grades are awarded by reference to predetermined standards rather than by reference to the performance of other students in the subject.

While these principles are equally relevant to school and university settings, a direct transfer of school assessment methods to the university environment is inappropriate, since differences in students' maturity, interests and needs need to be taken into account. As our current students are adults undertaking a postgraduate teacher education course, we believe it is important to offer assessment tasks which mirror the authentic practices of the teaching profession. In this paper we demonstrate how we have made connections between these practices and the principles listed above, by referring to assessment tasks we have developed for The University of Queensland's Postgraduate Diploma in Education course.

The principle aim of the Diploma course is to provide preservice education for intending secondary school teachers. Students undertake compulsory studies in subjects dealing with professional issues and practices in education, and choose two curriculum areas which represent their teaching interests. Our own curriculum subjects prepare students to teach Mathematics and English across the broad range of courses offered in the Junior and Senior secondary school. These subjects introduce current theories of learning and teaching Mathematics and English, and give attention to curriculum development, assessment and evaluation, social justice issues, and the impact of technology on language/mathematics teaching and learning. To emphasise the link between our courses and professional practice, where possible the teaching and learning approaches used in workshops and tutorials are intended to model strategies that can be used in secondary classrooms. We encourage collegiality, collaboration and cooperative learning practices in these sessions as they are fundamental professional attributes.

In the remainder of the paper we explain how two lecturers from different disciplines can work together to design effective assessment strategies, and we describe some of the ways in which these strategies have enhanced students' learning.

 

Assessment Strategies

Collaboration between lecturers from different disciplines
It may seem unusual for lecturers from different disciplines-Mathematics and English-to collaborate in planning assessment programs. However, we have found that a collaborative approach offers personal and practical rewards. First, the sharing of ideas and expertise reduces the combined workload and prevents wasteful duplication of effort and resources. When face to face interaction is not possible, questions and information can easily be exchanged via email, and entire documents can be sent back and forth for comment and amendment. In addition, developing a coherent, consistent, cross-disciplinary approach to assessment is a worthwhile venture in its own right, as it forces us to articulate and justify our own values and principles in the process of reaching common ground. Students should also benefit from receiving consistent messages about assessment in different subjects, since this may reduce the perception that assessment is merely a game which can be won by learning each subject lecturer's idiosyncratic rules.

One of the products of our collaboration is illustrated in Figure 1, which shows how we have identified common themes in our subjects, and how assessment tasks in Mathematics and English were planned to meet students' changing needs and concerns in the different phases of the course. Assessment tasks have been designed to complement each part of the course and to give appropriately timed feedback on the development of students' knowledge and skills (The University of Queensland Academic Board, nd.).

Because assessment tasks are closely tied to students' professional duties as teachers, we recognise our responsibility to model task performance in workshops and to provide exemplars of good practice. In addition, the close relationship between assessment and learning is reinforced by using class time to provide students with the skills they will need to complete assessment tasks. For example, students are prepared to tackle the Mathematics textbook evaluation task in a series of tutorials which deal with mathematics and language, analysis of textbooks, and designing reading activities which compensate for a text's weaknesses and capitalise on its strengths. In English, as an outcome of a workshop on planning, the students were encouraged to develop a one page outline of their proposed unit plan for formative assessment. In addition, since the materials produced are intended to be useful resources for their future practice, we believe that this approach is effective in eliciting students' genuine effort, motivation and commitment (Maxwell, van Kraayenoord, Fields & Herschell, 1995).

Criteria and standards drawn from professional education practice
A further aspect of our collaboration has involved developing assessment criteria which are grounded in professional educational practice, and realistic standards for describing students' performance. We began with our course objectives, from which we formulated general criteria to identify the qualities we expected students to develop throughout the year. While the general criteria for Mathematics and English were not identical, their underlying similarities are clearly seen in the examples below. (There were four general criteria for Mathematics, and six for English.)

Theme Phase Mathematics Assessment Tasks English Assessment Tasks
Teaching Mathematics/ English: models, strategies, classroom skills      
Understanding Mathematics learning processes, language and literacy Phase 1: Understanding curriculum and teaching (9 weeks)    
Familiarisation with syllabus documents   Review of an article from a professional journal A set of lesson plans and accompanying resources
Practicum (5 weeks)
Curriculum development Phase 2: Consolidation and extension (4 weeks) Textbook evaluation
Technology seminar		 Annotated resource file for a Junior syllabus topic
Practicum (7 weeks)
Issues in Mathematics/ English education Phase 3: Preparation for the first year of teaching
(4 weeks)		 Annotated resource file for a Junior syllabus topic
Unit plan		 Seminar on an issue in English education
Statement of philosophy
Unit plan

Figure 1. Course structure and assessment tasks for Mathematics and English curriculum studies

 

.
Mathematics Criteria English Criteria
1. Demonstrated understanding of concepts, contexts, and issues in mathematics education1. Demonstrated understanding of theories and principles of language and learning in English education.
2. Application of theoretical ideas about teaching and learning to practical problems and tasks. 2. Demonstrated understanding and application of a range of approaches and strategies in English teaching relevant to classroom situations.

For each assessment task we then prepared detailed criteria and explicitly stated standards, which mapped out in subject-specific terms the range of performance levels we anticipated. Once again we shared ideas about task criteria, but these were then individually adapted to incorporate the vocabulary and values of our separate disciplines. For example, in both Mathematics and English curriculum studies, students were required to work in pairs to present a seminar dealing with an issue (English) or technology-based activity (Mathematics) of their choice. The task-specific assessment criteria for Mathematics and English seminars are provided in Figure 2, together with descriptions of the highest quality standard of performance. The major difference between these two tasks was the intended audience: although the purpose in both subjects was to simulate a professional development seminar, in English the aim was to develop teachers' understanding of the chosen issue, while in Mathematics the goal was for participants to try out an activity designed for school students.

Figure 2 also shows how specific assessment tasks allow students to demonstrate the qualities described by the general course criteria; for example, the first task criterion, which refers to understanding of the issue or topic, is a specific instance of the more general criterion dealing with understanding of theories and principles (English), and concepts, contexts and issues (Mathematics).

MATHEMATICS ENGLISH
Criterion Top Standard Criterion Top Standard
Understanding of the use of technology in teaching mathematics

CC1

 Seminar handout deals comprehensively with relevant aspects of implementing the activity, giving balanced attention to mathematics, technology, and teaching approach. Understanding of issue or topic selected

CC1, CC5

 Seminar deals comprehensively with relevant aspects of issue/topic giving balanced attention to different perspectives.
Relevance of selected technology based activity

CC3

 Activity is imaginatively developed, selected, or adapted, and highly appropriate for developing students' understanding of the mathematical topic. Rationale clearly explains relevance and purpose of the activity. Relevance of selected topic/ issue to English education

CC1, CC5

 Implications for English teaching are highly relevant, imaginatively developed, selected, or analysed, and highly appropriate for developing teachers' understanding of the issue/topic.
Structure and organisation of seminar presentation

CC2

 Logically structured; activity stimulates student interest and actively engages them with the technology and the mathematics; resources are used with flair and imagination; pacing makes best use of the time available. Structure and organisation of seminar presentation

CC2, CC3

 Seminar is sequenced to provide an engaging introduction, showed logical development and variety, and a relevant conclusion. Resources were used with imagination; pacing makes best use of the time available.
Quality of oral communication

CC4

 Communicates with clarity; good use of variation; demonstrates well developed questioning and explanation skills with individual students; uses students' questions and comments to orchestrate whole class discussion of activity. Quality of oral communication

CC6

 Communicates with clarity; good use of variation; demonstrates well developed questioning and explanation skills with individual students; orchestrates class participation very effectively.
Quality of written communication

CC4

 Writing is concise, well-structured and error-free. Format and structure of handout make it easy to follow and practical to use. Quality of written communication

CC6

 Writing is concise, well-structured and error-free. Format and structure of handout make it easy to follow and practical to use.

Figure 2. Task-specific criteria and top standard for Mathematics and English seminars

Collaboration between students on assessment tasks
While we encourage students to collaborate informally on all assessment tasks as well as during workshops, the previous section implies that we also see value in designing formal assessment tasks in which students are required to work together and prepare materials, ideas or activities for presentation to peers. The example outlined above involves the use of professional development seminars. These seminars have both an assessment and a professional focus, since their purposes are not only to develop students' planning, communication and organisational skills, but also to prepare them to work collegially in sharing ideas with fellow teachers. To illustrate these benefits we describe some features of the Technology Seminars presented by Mathematics students.

The students were required to prepare and present a technology based activity designed to teach some aspect of a topic of their choice from the Senior (i.e. Years 11 and 12) Mathematics Syllabuses. The activity was to make use of either computer software or graphing calculators, and sufficient time was allowed for the class to try the activity and offer feedback on its usefulness. To accompany their oral presentation, students provided class members with a handout containing information which would enable another teacher to implement the activity. Both the written handout and oral presentation were assessed, as shown in Figure 2, and students were assessed as a pair rather than individuals. Students had ample opportunities to develop the requisite technology skills during workshops leading up to the assessment task, and additional class time was set aside to allow them to choose a partner, decide on a topic, visit the library to search for print or Internet resources, try out activities, and consult with the lecturer. A full day was set aside for the seminars, in order to provide a realistic simulation of a teacher inservice course or conference. The level of expertise displayed by the presenters, the intensity of the class's engagement with the activities, and the stimulating discussion afterwards suggested that students found the task relevant and interesting. Collaboration was also observed to extend beyond the pairs of presenters, as students were encouraged to work together on the activities. Indeed, the whole enterprise was collaborative in the sense that the tangible product which resulted was a set of technology-based activities produced and shared by all members of the class. The task also permitted students to share their work with the wider professional community, as the next section describes.

The design of assessment tasks with multiple purposes and a range of outcomes Our assessment programs are intended to be both process and outcome oriented; that is, our goal is to offer authentic tasks which result in a useful product. This goal is achieved most convincingly when students' work reaches a professional audience beyond their course lecturers and peers. For example, one pair of Mathematics students has presented their Technology Seminar at the Annual Conference of the Queensland Association of Mathematics Teachers, and a further three pairs have submitted articles based on their technology-based activities to the journal published by this professional association. In addition, students have tendered their unit plans (the final assessment task, see Figure 1) as evidence of their competence when applying for teaching positions in secondary schools. This evaluation of students' work by practising teachers reinforces the authenticity of the original assessment tasks, and legitimates the criteria and standards we use to judge the quality of students' performance.

The use of formal and informal peer assessment
The University of Queensland's assessment policy acknowledges the value of well-constructed peer assessment exercises in providing valuable learning experiences and lifelong learning (The University of Queensland Academic Board, nd). In both our classes we include informal and formal peer assessment. Informal peer teaching takes place throughout the year: in microteaching situations, for example, we ask students to observe the teaching of their colleagues providing oral, and sometimes written, feedback on their performance. Reflection after role playing activities in workshops also allows the students to give constructive feedback in a supportive atmosphere. The example of sharing the Annotated Resource File used in the English curriculum subject demonstrates how more formally structured peer assessment can be used to augment the lecturer's judgments.

English students were required to develop a collection of resources which could be expanded during subsequent years of teaching (assessment focus). Each collection consisted of ten to fifteen resources suitable for teaching in a junior secondary English classroom. The aim of the task was to develop critical skills in selecting and evaluating resources, and so students were asked to provide critical evaluations of the resources focusing on language challenges they posed for students with learning difficulties (professional focus). The resources file was assessed by the lecturer and by peers. The peer assessment component of this task focused on three goals, all of which have a professional component: first, to share and swap resources, second, to provide peer feedback on resources, and third, to provide experience in writing evaluative comments. The peer assessment workshop lasted three hours and included the following steps:

  1. A review of feedback techniques led by the lecturer. This introduction addressed issues of how to write comments positively and provided a synthesis of current research on effective feedback.

  2. "One minute of fame" for students to share their topics. This enabled the students to identify Resource Files that were relevant to their teaching interests and needs for the forthcoming practicum.

  3. Reading and evaluation. Students were required to read and evaluate a minimum of two folders. Students used the same criteria as the lecturer for their peer evaluations.

  4. Whole group reflection.

Lively discussions were generated during the reading and evaluation phase and in the reflections. The students found talking about their resource files and sharing their collections to be the most important element of the workshop. They commented that they felt validated in their judgment as teachers and supported in their professional development. The students were also made aware of the amount of time needed to generate written comments and the difficulties of writing constructive comments on the performance of their friends This was a significant exercise in making them sensitive to the issues they would face when performing similar tasks with their own pupils. A very practical outcome of the task was the opportunity to exchange professional materials that would be useful in their early years of teaching.

So far we have presented a very positive picture of our implementation of criteria-based assessment. However, as we refine our assessment programs and implement them in our courses we are increasingly aware of issues that still need to be worked through in using this form of assessment. In the next section we highlight some of those issues.

 

Some Issues

The first issue concerns the continuing requirement to assign a single grade to sum up performance across a wide range of tasks over the course of a full year. Such a requirement seems at variance with the goal of providing detailed feedback on the many qualities/criteria that characterise competent teaching performance. Rich information is lost when achievement is reported in this way. We have shown that the task of generating suitable criteria is one which can be tackled collaboratively by lecturers within a broadly based discipline such as Education, and we suggest that staff in other departments will benefit from this type of professional development activity. A possible model could involve lecturers working as a team to formulate generic criteria that apply to learning within their discipline, followed by individuals identifying the unique knowledge and abilities to be developed in specific subjects.

Turning to assessment strategies, we are aware that student collaboration on assessment tasks may not meet with universal approval, since there are those who protest that students forced to work in pairs or groups will not necessarily make an equal contribution to the task. However, this assessment strategy has our full support, since one of our central goals is for students to develop the interpersonal skills needed to work in a collegial environment. Similar doubts may be expressed about peer assessment, particularly on the question of students' competence in making judgments about their fellow class members. We also acknowledge that there are difficulties in deciding how to incorporate students' evaluations into the final grade for the task.

Nevertheless, we maintain that this is a worthwhile approach if the right balance can be struck between making the task as realistic as possible and avoiding placing too much pressure on students who are still relatively inexperienced in assessment.

 

Conclusion

In our discussion of five themes

we have drawn on examples from our own practice to illustrate ways of implementing The University of Queensland's new assessment policies. Using criteria and standards allows us to map students' strengths and weaknesses in a wide variety of activities relevant to their development as teachers, and provides valuable feedback which shows them where and how they can improve on subsequent tasks. We have also benefited from examining in detail our own expectations and assumptions about learning, made explicit in the process of planning our assessment programs.

In this paper we have illustrated some of the assessment techniques that work for us. However, it is important to remember that all assessment has social, economic and political dimensions (Johnston, 1992), so that particular techniques are merely an expression of underlying principles. Teachers working within different academic disciplines must decide for themselves how best to apply those principles to their own contexts.

 

References

Board of Secondary School Studies (1986-1988). Discussion papers on school based assessment (Numbers 1-21). Brisbane: BSSS.

Johnston, P. (1992). Constructive evaluation of literacy. New York: Longman.

Maxwell, G., van Kraayenoord, C.E., Fields, C., & Herschell, P. (1995). Developing an assessment program: A planning guide for schools (A Professional Development Program). Brisbane: Department of Education, Queensland, Southern Vale School Support Centre.

The University of Queensland Academic Board (nd). Policy and guidelines on assessment: Advice to examiners from the assessment subcommittee. Available at http://www.uq.edu.au/hupp/index.html?page=25109&pid=0

 

Conference Entry Page About the Conference Abstracts Conference Program

TEDI Home About Assessment UQ Home