Flexible Assessment for Flexible Delivery: Preliminary Results and Tentative Conclusions
by
Dr Jeremy B Williams
Brisbane Graduate School of Business, Queensland University of Technology
This paper argues that a commitment to flexible delivery necessarily requires a commitment to flexible assessment. It describes how flexible delivery was understood to have been successfully applied in a large undergraduate unit until it became apparent that the assessment system was quite inconsistent with the notion of flexibility. A flexible assessment model has now been trialed, and survey results and analysis are presented.
Introduction
The increasing market orientation of higher education has brought sweeping changes in universities throughout Australia and elsewhere in the world. Amongst other things, changes to government funding have forced universities to become more innovative in their resourcing arrangements. Partnerships with professional bodies and the private sector are becoming more widespread, universities have expanded geographically to tap into non-traditional markets, and for course developments to proceed, tangible evidence of student demand and cost efficiency are of the utmost importance. The product of these changes (and others), is that universities have developed a much stronger 'customer focus'. In short, the hard reality of life in the higher education sector at the end of the 1990s is that failure to recognise students as 'clients' is to run the risk of anonymity in the marketp
One way in which some institutions have sought to enhance their international competitiveness is through the flexible delivery of their programs. Flexible delivery is, by definition, a client-oriented approach because it is a commitment, on the part of the education provider, to tailor courses to meet the various individual needs of its students. Furthermore, it is tacit recognition of the 'massification' of higher education, whereupon the student profile has changed quite dramatically - socially, culturally, economically - and that, pedagogically, there is a need to cater for this increasingly diverse student body.
As this paper will highlight, flexible delivery provides students with a number of different options for study. It is not prescriptive in the sense that one approach to study is identified as being superior to another. A student can chart a route through a degree that is most compatible with their budget, their social, family and working lives, and their preferred learning style. In short, flexible delivery is non-discriminatory, catering equally for an international student, a single parent working part-time, a business executive travelling regularly overseas and interstate, or a school leaver.
The main aim of this paper will be to focus on one important and frequently overlooked aspect of flexible delivery, the assessment component. It will demonstrate, by reference to the case of two flexibly delivered units within the Faculty of Business at Queensland University of Technology (QUT)1, that what was thought to flexible assessment was not, and how subsequent changes to the assessment system have proved to be enormously popular with the student body.
The discussion will concentrate, first of all, on the defining characteristics of a flexibly delivered program, and the salient features of the case studies referred to above. This is followed by a section on the flexible assessment model (FAM) that has been trialed, together with the results of a student survey on the FAM. The final section draws together the strands of the discussion and puts forward some tentative conclusions.
What is flexible delivery?
There has been a commitment to the principle of flexible delivery at QUT since the mid-1990s when it became a central feature of the University's Teaching and Learning strategic plan. Care has been taken not to define flexible delivery too rigidly so as not to erect any obstacles to innovative ways of delivering programs. However, to assist staff develop some notion of flexible delivery, the following internal QUT memorandum was circulated in April 1999:
'A flexibly delivered unit is one in which the options for delivery include alternatives to the traditional ways of on-campus in-classroom lectures, seminars, tutorials and practical classes. Such a unit will be designed with the aim of meeting students' diverse learning needs by incorporating one or more aspects of flexibility in time, place and/or technology, such as:
- delivery in the workplace or remote from the campuses of the university
- delivery in block mode, other intensive mode, or other non-standard delivery time format either on or off campus
- delivery with non-standard beginning and completion dates for the units (insofar as the Student Information System can cope with this kind of flexibility)
- the use of technology and resources for learning support to provide options for any student to access and use materials at his or her own time and place (eg Web-based teaching materials and exercises), or to be assessed without having to attend examinations at a specific place and time.
These aspects of flexibility will form the majority of the delivery mechanisms used in any flexibly delivered unit.'
Flexible delivery in the Faculty of Business
Within a relatively short period of time, flexible delivery has become a fairly prominent feature within courses offered by the Faculty of Business. It has been adopted, to varying degrees, in individual units, specialisations (groupings of units) and whole courses.
The Brisbane Graduate School of Business (BGSB) for example, offers a range of customised certificate courses in management for the defence forces, government departments, professional institutes and individual companies in the private sector. In addition, all schools in the Faculty have embraced the idea of intensive block mode and short sequential block mode to meet the needs of students visiting the campus for short periods or over the shortened summer semester.
A further initiative has been the development of on-line teaching (OLT) sites2. There are currently 103 OLT sites University-wide, with the Faculty of Business housing the largest number. Commencing with two sites in 1997, there are currently 24 OLT sites in Business with many more planned for 2000. The framework for OLT sites varies from unit to unit but, typically, there is a download facility where students can access PowerPoint lecture slides, tutorial solutions, past examination papers and the like, discussion forums (electronic bulletin boards), chat space, (internet relay chat [IRC]), and discipline-relevant links to the world wide web.
The first unit in the Faculty of Business to have an OLT site was large first year undergraduate unit in economics3. Redesigned in 1996, it was also one of the first units in the Faculty to make a firm commitment to flexible delivery4. A study manual was produced with weekly, issues-based modules, and considerable attention was given to pedagogical issues. In short, students were not left to 'sink or swim'. The response from students was overwhelmingly positive and the centrally administered student evaluation of the unit (SEU) scores increased dramatically. However, successive SEUs consistently showed that students' views on assessment practices were only slightly positive.
Flexible delivery and rigid assessment
When the new economics unit was introduced, flexible assessment was defined in terms of student choice with respect to the assignment topic and the corresponding submission date (one from a choice of six). Students had an element of flexibility, but not sufficient flexibility that a person could elect not to undertake the continuous assessment if it was incompatible with their preferred learning style, or their work patterns. Furthermore, it was an 'unforgiving' assessment system in that a student was not permitted to learn from their mistakes.
Conscientious students scoring below average marks in an assignment with an early submission date, for example, regularly requested that they be allowed to have a 'second bite at the cherry' by submitting one of the assignments with a later handing-in date. These requests were denied owing to the large student numbers and the constraints of the staffing budget. Furthermore, every student was compelled to sit a mid-semester test and a final exam. In summary, therefore, the assessment system was quite rigid, and at odds with the publicly stated commitment to flexible delivery.
The flexible assessment model
5Commencing in 1998, the new flexible assessment model (FAM) was introduced comprising four assessment items: six optional class tests, altogether worth a maximum of 30 marks (assessment item 1), an optional essay assignment worth a maximum of 10 marks (assessment item 2), an optional data response assignment worth a maximum of 10 marks (assessment item 3), and a compulsory final examination worth a maximum of 100 marks (assessment item 4). As a result students were given the choice of completing all, or some combination of the optional assessment items, such that it was possible to have between 50 and 100 per cent riding on the final exam. Importantly, a computer spread sheet identifies which combination (from a possible six combinations) maximises a student's mark (see Table 1). That is, it is not up to students to nominate their preferred assessment combination in advance. The spreadsheet performs this task for them. Any assessment item which yields a lower final mark is simply disregarded.
An important consideration at this time was that, for the sake of consistency, the same assessment instruments used in the continuous assessment items should appear in the final exam; that is, the final examination would contain a combination of multiple-choice questions, data response questions and essay work. In this way, by completing the continuous assessment. students were effectively getting some 'exam practice'. By electing not to complete the continuous assessment, a student foregos this opportunity - but this is their choice.
Table 1
Preliminary results
Early feedback from students was very favourable. In a survey of 118 students6, 38 per cent stated the FAM was the best they had encountered, with 52 per cent saying it compared very well with other systems (see Figure 1). Asked about its transferability, 40 per cent thought the FAM should be used in all the Faculty's units, while 31 per cent felt it could be used in most units (see Figure 2).
Figure 1: 'How does the FAM compare to other assessment systems you have encountered?'
Figure 2: 'Could the FAM be used more widely in the Faculty of Business?'
Following on from the introduction of the FAM in first year economics, the system was subsequently trialed in an MBA unit in managerial economics. In this case, the students' endorsement of the FAM was even stronger reflecting, perhaps, the higher proportion of part-time students in the unit who often struggle to complete items of continuous assessment given the large number of competing interests on their time.
Future plans
In the case of introductory economics and managerial economics, their 'modularisation' and 'electronicisation', together with the FAM, genuinely allows the possibility of self-paced learning. Furthermore, from 2000 there are plans for all students to be able to view their on-going performance when the Curwen Results program is integrated with QUT Virtual7; that is, students will be able to monitor which assessment option is giving them the highest grade.
But much work remains to be done. To date, only the assessment system has been determined. Within this framework, attention must be given, not just to outcomes, but also the student experiences that lead to these outcomes. The next objective is to encourage academic staff to give greater attention to these experiences with a view to improving the whole of student learning. This must be a collaborative activity, involving people from across the university community, including students themselves, and from outside the university, specifically employers8.
To this end, there are plans in a number of undergraduate and postgraduate units for multiple choice tests (previously conducted in class) to be put on-line. These tests, marked by computer instantaneously, are primarily designed to give students continuous feedback on their progress. Importantly, a student gets a mark for participation (5 per cent for the completion of all tests) rather than a mark for performance. There are two reasons for this (i) students have the opportunity to collude if the tests are unsupervised; and (ii) it is believed that the ability to perform in multiple-choice tests, while useful in that it tests content knowledge, is not the most important skill that students need to acquire. This brings us back to the question of the quality of the student's experience of assessment. By getting multiple-choice out of the classroom (and relegating it in terms of its contribution to final marks), it is possible to concentrate efforts on higher level skills; for example, the development of critical thinking skills9.
Tentative conclusions (and unresolved questions)
The FAM has a number of benefits:
- It provides students with numerous assessment options. In so doing, it accommodates student diversity and caters for different learning styles.
- . It is a very forgiving system in that it is possible for summative continuous assessment items to be treated as formative assessment. In short, students can learn from their mistakes. If a student performs poorly in an item of continuous assessment, the mark for this item need not necessarily contribute to their final mark. The student can take advantage of the feedback he or she receives and avoid making the same mistake in the final examination.
- The fear is taken out of assessment. Students find it comforting knowing that, unlike conventional assessment systems, they can complete an assessment item under the FAM and not worry that a poor performance will have an irreversible effect on their final grade.
A number of criticisms have been made of the FAM. The following quotations are among those received from students:
- 'It makes life easy for lazy students'.
- 'I would have passed the course if I'd been compelled to do the continuous assessment'.
- 'Too much choice is confusing'.
And from staff:
- "'Why is it necessary to have the same assessment instruments for the final examination and continuous assessment?'
- 'If a student elects to do only the final examination, they will have undertaken no formative assessment - doesn't this contravene University policy?'
- 'Why have a compulsory examination?'
- 'I think a student must pass the final examination to pass the unit overall'.
- 'Won't the system encourage grade inflation?'
It is clear from these comments that much research remains to be done in what, to date, has been a very under-researched field.
Endnotes