Quality Assurance Through a Continuous Curriculum Review (CCR) Strategy: Reflections on a Pilot Project

by

Calvin Smith, The Teaching and Development Institute

and

Wayne Robinson, Deputy President of the Academic Board

The University of Queensland

 

Abstract

Woodward (1993:113) defines institutional research as "the activity in which the research effort of an academic institution is directed at the solution of its own problems and to the enhancement of its own performance". In this paper we report on one such attempt at the University of Queensland in Brisbane, Australia. We piloted a method for gathering data on whole courses (as opposed to teaching or subject evaluation data) for the purposes of targeting improvement strategies. The project was funded out of university teaching quality funding and involved the cooperation of the University's centrally funded Teaching and Educational Development Institute (TEDI). In this paper we describe the strategy and the reporting protocols that were developed, and reflect on the costs and benefits of engaging in this kind of data gathering exercise for quality assurance and quality enhancement purposes.

 

Quality Assurance in Teaching and Learning in Higher Education

Performance measures are used increasingly by governments in Australia, as elsewhere, to compare universities with each other. Performance comparisons may be linked to recurrent funding in future. Consequently, measures of performance have increasing salience to the higher education community. Regardless how one views of the use of performance indicators in this way, it is becoming more and more likely that the future will see as much or more of this kind of application rather than less of it.

A recent review of higher education policy in Australia (West, 1998:50) concluded that the promotion of high quality learning experiences for students was one of the principles underpinning the committee's proposed higher education financing and policy framework. They reinforced the point thus:

In an increasingly competitive market for the provision of educational services, universities will need to seek continually to improve the range and quality of their offerings...(West, 1998:51)

and later made the recommendation that Government "encourage institutions to adopt a code of quality for scholarly teaching" (West, 1998:Recommendation 24, page 147).

The Course Experience Questionnaire (CEQ) (Ramsden, 1991) has become a central performance indicator of the quality of courses in the higher education sector in Australia because of its use by government to compare institutions' performances in teaching and course characteristics. The CEQ measures course characteristics using five multiple-item scales (Good Teaching Scale, Appropriate Workload Scale, Clear Goals and Standards Scale, Appropriate Assessment Scale and the Generic Skills Scale).

The CEQ is administered to all graduates early in the year following their graduation. It is mailed out to all graduates with another instrument called the Graduate Destinations Survey each year. These surveys are administered by the Graduate Careers Council of Australia (GCCA). Its use to compare universities has been the basis for much debate and some consternation. In 1999 DETYA funded a project to "enhance" or "extend" the CEQ as an instrument. Note that the project's brief was not to seek new replacement measures that all concerned would endorse; the instrument itself and the government's intended usage of it are in all likelihood not up for negotiation.

Much of the debate has hinged on the quality of the instrument itself, such as the validity and reliability of scales, the methods used to recode and report the data collected. But putting those things aside and looking at the use of the CEQ from a practitioner's perspective, it has two major flaws. First, it seeks from students a totalising and summary view of an entire course. This requires the "averaging" of each student's evaluation of 84-140 weeks of university experiences (depending on whether a three, four, or five year program), or approximately 24-40 different units or subjects. Students must either "smooth" out the contours of their experience in estimating this totalising average, or they will express a biased view based on the impact of a limited number of very salient (highly positive or negative) experiences. Either way the information is less valid or useful because of this. Second, the analyses are received by the universities late in the year that the survey is conducted. This means that, assuming that practitioners will try and respond to the information contained in the analyses, they will be forced to use information relating to an experience that started four (in the case of the three year program) to seven (in the case of the five year program) years earlier. There may have been changes to various parts of the curriculum over that time, rendering some of this information out-of-date. Because of these two problems, practitioners are left not knowing what areas of the curriculum are in need of examination or "fixing" and what are the most salient pleasures and problems for students.

Apart from generating debate, another impact of the CEQ has been to focus academics' attention to the kinds of things that might be done in these circumstances to improve their relative ranking based on the CEQ measures. However, the problems with the CEQ make it very difficult for academics to focus their efforts and attention in effective ways that might improve the curriculum and students' experiences of the course as a whole. The current project, called the Continuous Curriculum Review (CCR) Pilot Project was conceived as a method gather information about curricula that would help teaching staff overcome these problems and target specific areas of the course for improvement and to do so in a timely manner.

Another basis for the criticism of the use of the CEQ is that it is unfair not to take into account differences in the character and composition of individual schools and departments when comparing courses at different institutions. At the 1999 Conference of the Higher Education Research and Development Society of Australasia (HERDSA), David Curtis outlined a compelling argument about the effects of institutional differences on CEQ scale means (Curtis, 1999). This raises the problem of controlling for those differences which make a difference, or comparing like with like. This is in keeping with the view espoused by Johnes and Taylor (1990:50):

One approach to this task of evaluating universities is to compare their performance across a range of indicators. But if this comparative approach is to be adopted, it is essential to compare like with like. This means that any observed differences in performance must take account of any corresponding differences in the particular circumstances of each institution.

Even where comparisons between institutions are based on highly standardised performance indicators, such as in the Nordic countries, the need for adjustments for unique institutional characteristics is recognised (Smeby & Stensaker, 1999).

Though we may be a long way off having a wide-spread set of standardised indicators of course quality, sensitive to relevant institutional differences, and acceptable to all stakeholders, nonetheless universities are using performance indicators to evaluate course quality.

Evaluation however should be put to some use, and not simply done and the effort then let go to waste. In the present case we were advocating a strictly developmental use for the data we collected - it was to be used to aid decision making about curriculum development.

As Johnes and Taylor (1990:50) put it:

The purpose of evaluation is to discover whether there is a better way of doing things. As far as the university sector is concerned, evaluation involves discovering whether universities could achieve their objectives more efficiently and more effectively.

As Adelman and Alexander (1982) (cited in Stringer & Finlay, 1993:110) note:

...evaluation should be something that rocks the boat and institutions should be suspicious of evaluation programmes whose outcomes can be painlessly accommodated...

Stringer and Finlay (1993:110) go on to make the point that that the information gathered by seeking structured feedback from students should not be used solely for judgemental purposes by administrators who should give equal weight to the developmental advantages to be gained by the gathering of such information.

Woodward (1993:113) defines institutional research as "the activity in which the research effort of an academic institution is directed at the solution of its own problems and to the enhancement of its own performance". In this paper we describe and reflect on an attempt at the University of Queensland to use institutional resources to help to address the joint needs of course quality appraisal for improvement and engagement with the issues raised about the CEQ given its use as a nationally standardised performance indicator. The strategy, Continuous Curriculum Review (CCR) is simply an attempt to trial and promote regular comprehensive data collection for developing "snapshot" views of whole curricula so that decisions about what to change and what to change first can be made in an empirically defensible and timely manner.

 

Origins of the CCR Pilot Project

The enhancement of teaching and learning is now central to every university's strategic plan and is becoming an important part of scholars' lives. The current emphasis on enhancing teaching and learning has a number of origins, from the impact of educational theory at one end of the spectrum, to the push for accountability by funding bodies such as the federal government. Previously, it was assumed that each scholar was the best judge of the level, the depth and the assessment of their particular subject or subjects in which they were expert. Times have changed. There is now a strong drive toward the enhancement of the quality of courses using the system of policy, process (implementation) and outcome (evaluation). In most universities, the policy is in place, the processes are part of the way there, and the measurement of outcomes is embryonic.

One of the major impacts on universities in the area of teaching and learning has been the nationally administered CEQ. As already mentioned, the CEQ, originally devised to explore aspects of deeper learning, is being revised. A parallel development is the graduate attributes / generic skills objective tests, currently being devised by Australian Centre for Educational Research (ACER). Both research projects are funded by DETYA.

All place major emphasis on outcome. It is implicit that universities will focus on policies and procedures that will provide outcomes measurable on the national questionnaires. As each are essentially exit surveys, there is necessarily a long lead-time between the initiation of change and the ability to assess the effect of change. We described this in more detail above. It is therefore an appropriate time to pursue policies and processes that circumvent this long lead-time to foster change over a shorter time frame.

The responses by universities to provide the impetus for rapid change include:

There is another prime factor that needs to be taken into consideration and that is the linkage between policy, process and outcome. In a recent report, the GCCA quoted a UTS view:

It is a myth that all you have to do is to send back the results of a survey to those concerned and action, improvement and innovation will automatically occur. Such an assumption ignores all the research and motivation and change management in universities (Graduate Careers Council of Australia, 1999:20).

If we are looking for a quick turnaround as exemplified by the CCR, then the above maxim must be addressed. Surveys are of particular interest to the designers, it is in many senses "their baby", but it may be a chore for students to fill out, may be studiously ignored by staff, and more particularly by curriculum committees. Much good work and insight into the quality of courses is often consigned to someone's filing cabinet.

That is where the orientation toward outcome becomes central. One might ask what are the mechanisms that need to be put in place to ensure serious discussion and adoption of the messages contained in the CCR. This is where the system of policy and process determine outcome. When the concept of the CCR was being developed, one of the first questions asked was, whose support and commitment was required to give the best opportunity for successful adoption of the CCR findings?

At the University of Queensland, the Executive Deans have overall responsibility for the courses within Faculties. The Deans were therefore the first group that needed to be convinced of the value of CCR. The Directors of Studies, through the Deans, are responsible for each Faculty's course structures and quality. They too were integral to the success of the venture. There is finally the student body. Students, in many senses, are a captive audience and are asked on numerous occasions to complete questionnaires for many and various purposes. It is not often, however, that their collective view is fed back to them. In most circumstances it is a one way street. A decision was made that ensured that the results of the CCR surveys would be freely available to all, including students. It is clearly then, public document, publicly owned and therefore freely discussed throughout the university. This fosters collective ownership of any deficiencies; thus problems must be collectively solved. The successes are also collectively owned, providing a collective pride in that success.

The remaining major aspect relates to gaining formal executive and academic support, both for the concept and for the funding of the initiative. In the University of Queensland's case this was achieved through discussions with the Teaching and Learning committee, the Director of the Teaching and Educational Development Institute and the Vice-chancellor Teaching quality funds were available for the pilot study allowing the immediate commencement of the project

. The pathway from the initial idea to commencement of the pilot study required commitment from the

The ultimate success of the project will depend very much on the continuing support and encouragement of these pivotal groups. The short turnaround time will help cement that support. The short-term changes will also impact on the long-term national measures such as the CEQ and its respective clones.

 

The project strategy

One department or school from each faculty was chosen for this pilot. Table 1 shows the departments and schools involved in this pilot.

With the help of consultants from the Teaching and Educational Development Institute (TEDI), the members of each school's or department's curriculum committee, in consultation with other school/department staff, produced a questionnaire for each year of their degree program. Students were surveyed using these questionnaires early in the academic year (during Semester 1) in the case of the first participating school, and later in the year (during Semester 2) in the case of the remaining schools and departments.

The strategy was to solicit from leaders in these groups interest and support for the pilot. This involved reassuring them of the purposes to which the information we ultimately acquired would be put, the levels of commitment and cost involved from their point of view and the kinds of information that it might be reasonable to expect to acquire form the process. The main selling point of this approach is that it allows a school or department to tailor an inquiry to their specific needs at the time. The specificity of needs is both disciplinary and historical: distinct disciplines very often have different teaching and learning goals and methods; and departments and schools have different priorities and concerns at different times. A central feature of this pilot was that only a minimal time and financial commitment was required from participating schools and departments. That commitment consisted mainly in the promotion of the pilot to staff and the giving up of up to one hour of lecture time in one or more lectures for each year cohort, judiciously chosen because the largest number of a cohort would be present therein.

Survey instruments were constructed in consultation with members of teaching and learning committees or other appropriate persons. Two staff from The Teaching and Educational Development Institute prepared and reproduced the instruments and conducted the surveys. The surveys were conducted in class time. This allowed the schools and departments to select classes that would contain the maximum proportion of members of a year cohort (in most cases this was the full complement of students bar absentees). Where this strategy did not work (for example, some lecturers would or could not give up a lecture in their subjects) the surveys were conducted in the supplementary tutorial program. In one case the instruments were mailed out to students who were on clinical placements.

Structure of the instruments

The School of Veterinary Science was the first to participate in the pilot, and their broad questionnaire design was used to help others construct their own. The features of the design are as follows:

 

Examples of Data and Reporting

Reporting protocol

In order to make the results informative without creating statistical "fog" we devised a reporting protocol that minimised the cognitive and interpretive load required to make sense of the information. This approach may sound to some as patronising, but was based on the idea that the information should be presented in such a way that it was clear to the users just what they should do on the basis of the information. The reporting protocol is described in the following terms:

  • If 70% or more of students responded in a favourable way to an item, that item is reported as "A" (no action required).
  • If 30% or more of students responded in an unfavourable way, that item is reported as "C" (action recommended).
  • All other cases are reported as "B" (ambiguous distribution; further action may be required). A bipolar distribution of 70% favourable and 30% unfavourable is also reported as "B".

Some items are difficult to interpret when presented using this rating method (A, B or C). Items that will give the least trouble are those that are simple, stated in positive terms and which unambiguously specify something that is done in the school (e.g. The organisation of placements in semester two is fair )

Consider an item such as Computer assisted learning (CAL) programs are effective learning tools. Assume it received a C rating. According to the ratings protocol used herein, a C means that this item is a negative result for the school and picks out an area where the school should seriously consider early interventions designed to improve matters. But what does a "negative result for the school" mean with an item such as this. First off, the authors will have had to assume that the question was asked because the school implements some CAL (we probably would not be all that interested in the students' philosophical opinions on theoretical pedagogy, but the school would be interested in the students' opinions of pedagogical processes they had experienced). Thus a larger than 30% proportion of students disagreeing with this item means that in respect of the CAL implemented by the school, the students do not find it effective as a learning tool. It would get a C rating under those circumstances.

There are other circumstances in which items may be difficult to interpret because of other kinds of problems with their wording. For instance, complex questions are those that can be seen to contain more than one question (e.g. The teaching staff mark fairly and promptly). In this case one does not know whether students who agreed with the item were agreeing with the conjunction (marking is both fair and prompt) or have been tempted into agreeing with the item if only one of the conjuncts is true. Or, again, they may have disagreed with the item since one of the conjuncts is false even though the other conjunct is true. Items such as these make it difficult to determine what action to take on the basis of the data collected. A C rating here because more than 30% of the sample disagrees with the proposition does not give clear guidance as to what to do because it is not clear what the data are telling us.

Ambiguously worded questions are those that could be interpreted by the respondent in more than one way (e.g. The teaching staff work hard). This may be true, but it might not be true that the hard work the staff do is put into teaching - it might for instance be put into research. Thus an A rating on an item such as this (where 70% or more of the sample agrees with the statement) leaves open the latter interpretation about working at things other than those the question was designed (poorly) to tap.

Negatively worded items also provide some challenge (e.g. The small group work is not of high quality). These items will receive C rating when 30% or more of the sample agree with them.

Finally, positively worded items that depict negative attributes (e.g. The canteen is the worst in the world) will generate a C rating when over 30% of the sample agree with them.

Items by year and items by subject

The reporting protocol allowed us to report the rating, as opposed to the distribution of responses, for each item in these categories. An example is given in Table 2.

In an attempt to further reduce the information to more immediately interpretable snapshot of the performance of subjects on any item, we found it useful to report a summary of performance on each of the items in terms of the ratio of subjects rating 'B' or 'C' to the total number of subjects to which each item was applicable, as in Table 3.

Graduate Attributes

For the graduate attributes measures we asked students to rate their levels of competence in the graduate attributes at the start and end of the year in question and the pair t-tests were performed to test the hypothesis that there were no differences between the before and after means. A similar method of displaying the results was employed, again using ratings (A,B,C), with slightly different interpretation rules (see Table 4 and Table 5).

Table 4: Interpretation Guidelines for Graduate Attributes Interpretation Guidelines

 

 

  • Students were asked to (retrospectively) rate themselves on 20 attributes at the beginning of the first year of studying law, and at the end of the first year. These two scores were subtracted to obtain a Difference score.
  • Graduate attributes were assigned an A rating if students recorded a statistically significant positive increase in that item over the course of the year.
  • A C rating was assigned to graduate attributes where a statistically significant decrease was recorded by students over the course of the year
  • B ratings were assigned to graduate attributes in three cases:
    • no difference was observed
    • a positive difference was observed over the course of the year, but this difference was not significant
    • a negative difference was observed, but was not significant

 

 

CEQ

The CEQ items were dealt with in the same manner that they are dealt with by the GCCA. Scale means were calculated using the same re-scaling methods used by GCCA. We then reported for each year their scale means against the scale means of the "real" CEQ based on surveys of the graduating cohort of the previous year. As well we used t-tests to test the probability of differences between these pairs of scores on each scale (see Table 6).

Note: An Asterisk (*) is used to denote a statistically significant difference at the p<.05 level. Similarly, two asterisks (**) are used to denote a difference significant at the p<.01 level. Where three asterisks are used (***) a significance level of p<.001 applies. We also reported the CEQ results graphically, facilitating a scale-by-scale comparison of the year cohort data and the previous year's graduate cohort data, as in Table 7.

Table 7: Graphical Representation of CCR compared with CEQ Results

 

Discussion and Conclusions

The pilot project has taught us a great deal about the costs and benefits of this cyclical and wholesale approach to evaluating curricula.

Costs

Lecture time to do the surveys
The amount of time taken to conduct the surveys may be resented by both staff and students. Staff, who rightly have the expectation that their lecture time is available to them for teaching, may not have the flexibility in the design of their subject(s) to give up even half an hour without suffering considerable disruption.

Students have two good reasons for resisting these incursions into their subject time. The first is "over surveying". Students are already regularly surveyed for evaluation of teaching and subjects at this university (indeed at many, if not most universities). Second, as the cost of education is passed increasingly from the State onto individuals, students may perceive that they are not getting their money's worth if one of their timetabled lectures is given over to an evaluation survey.

A third reason why students might be cynical or resistive, one which can do something about, is the belief that either nothing is done with the information collected or, if something is done, it is some future cohort of students that benefits from what is done. We must work to address this concern by ensuring that when we officially receive feedback and design change on the basis of it, we ought to tell students about it. If this is a regular occurrence then a 'culture' of responsiveness begins to manifest itself.

Evaluation and evaluation feedback must become not only aspects of our professional practice but they must be seen by students to be effective and of meaningful benefit to them in order that they be seen as legitimate activities. The best way to address all these issues is to make it clear to students the relationship between their feedback and subject, course and teaching and learning design.

Commitment
This approach to curriculum-level evaluation therefore takes commitment from faculties/schools & departments. It takes a commitment of time and resources and both these cost money. This is not a cost free exercise. To be done properly, to be effective, comprehensive regular curriculum evaluation must be adequately resourced.

But there is a further requirement for success and that is a commitment to act on the information generated in the process. Without this, students and staff alike will soon see that the process has been reduced to a meaningless data gathering exercise. Actions must ensue upon the collection and analysis of the data.

Benefits

The benefits of committing to such comprehensive data collection and analysis strategies include:

Note however that the realisation of these benefits is parasitic on the conditions of commitment outlined above being met. None of the benefits will accrue without adequate funding, commitment, action and closure of the feedback loop to students.

Do the benefits out-weigh the costs?

Realistically it is too early to tell at this stage whether the benefits of doing this kind of evaluation out-ways the costs. Prima facie we think they will, but the costs are not unsubstantial or inconsequential. And since commitment to act on the information is a crucial ingredient, the benefits do not simply accrue in some mathematical relation to the cost. What you get out of the process is determined by what you are willing to put into it.

From a staff perspective if rational and empirically defensible improvements to the curriculum result, and students do not object to the process, and there are no inappropriate or non-consensual uses of the data, then the outcomes probably will outweigh the inputs.

From a student perspective provided they can see changes to the curriculum result that are beneficial to their learning and to their experience of the curriculum, then we think that they will see that the benefits outweigh the costs.

 

Caveats and Conclusions

The following points have emerged from reflection on the pilot projects as important caveats for any future implementation. We discuss some of these below.

Timing

The surveys ought to be timed so that the students have no trouble recalling their experiences of the year in question. The best timing from a student recall perspective is towards to end of the year to which the questions refer. Unfortunately such a timing could not be worse from students' learning perspective coming as it does near to the end of the teaching period when many are stressed over impending exams and struggling to submit final assessment pieces. These facts about students at that time of semester may even bias their responses. One alternative is to conduct the surveys early the next year this alternative suffers from the problem of requiring students to recall the previous years characteristics after a lengthy vacation period.

Size of instrument

The survey instrument must be a fairly large one by most standards if it is to comprehensively cover the kinds of information one might gather about students' perspectives on a whole year of a curriculum. However, especially given the concerns already raised about over surveying students, care should be taken to keep the survey instruments size down to manageable proportions. Early feedback from those teaching staff who administered the survey indicates that this is also a staff concern.

Use and meaning of these data

It is imperative that the information collected not be used or treated as a summative account of the state of a curriculum. Rather, these data ought to be reported as a starting point for reflection, further inquiry and date of collection, or the allocation of resources to facilitate development, change or innovation that is motived or prioritised by the data already gathered.

Contextual interpretation and use

In this regard it is also very important that these data be used contextually and given, at all times, a contextual interpretation. By 'contextual use' we mean that these data must be used only in the context of the department's of school's attempts to diagnose and ultimately to improve their courses or subjects within their curriculum. This implies that the school or department might rightly claim some privileged determination over the uses to which these data are put. By 'contextual interpretation' we mean that the data that are collected by schools and departments must be given meaning by members of the schools or departments. Only those persons who have an intimate knowledge of the origins of the curriculum and of the conditions surrounding the teaching can reasonably and fairly interpret the data generated with this kind of approach.

Kinds of data

In relation to the aforementioned point about using these data as an impetus to further inquiry, one should bear in mind the limitations of quantitative data generally for illuminating the experiences of respondents. One can state, on the basis of such data as these, that x% of students rates something very badly, but this does not illuminate what aspects of the thing being rated lead respondents to so rate it, nor what things might best be done to improve matters. Thus 'further inquiry' here implies any and all methods of enquiry and kinds of data that can and should be brought to bear on an question in order to answer it. This may include interviews and focus groups with students or the collection of more or other kinds of quantitative or qualitative data before committing to particular changes. At best the information generated by this study allows us to know what areas need our attention most, or first, but not exactly what should be done to address the problems in those areas.

Identification, confidentiality, consultation

One important concern from a staff perspective is the capacity under certain conditions, for an individual staff member to be identified by association with particular subjects in the curriculum. This can occur where the design of the questionnaire picks out specific subjects either explicitly or implicitly. It is next to impossible to review a curriculum without asking some questions that are addressed with respect to individual subjects. Consequently where this is a design feature, it is important that all staff are consulted prior to the survey being conducted and their consent obtained. Again what is crucial here is the intended use of the information and the explicit commitment from all concerned to protect the information from misuse or unintended uses.

Universities such as the University of Queensland may presently enjoy the benefits of reputation, in having a competitive edge in the enrolment stakes, but this will not necessarily stand forever as a proxy for quality. As the West committee opined "...we doubt that any institution will be able to trade on reputation unless that reputation is underpinned by actual performance" (West, 1998:126). In that case, we must look for ways to address the issue of quality improvement and assurance from an institutionally specific perspective. CCR can be one of those processes that the University supports in order to enhance the quality of the teaching and learning experiences for students.

 

References

Adelman, C., & Alexander, R. (1982). The Self-Evaluating Institution. Lonodn: Methuen.

Curtis, D. (1999). The Course Experience Questionnaire as an Institutional Performance Indicator. Paper presented at the Cornerstones: The 1999 International HERDSA Conference, Melbourne, Australia.

Graduate Careers Council of Australia. (1999). Institutional Arrangements for Student Feedback . Canberra: Graduate Careers Council of Australia.

Johnes, J., & Taylor, J. (1990). Performance Indicators in Higher Education. Buckingham: SRHE and OUP.

Ramsden, P. (1991). A Performance Indicator of Teaching Quality in Higher Education: The Course Experience Questionnaire. Studies in Higher Education, 16(2), 129-150.

Smeby, J., & Stensaker, B. (1999). National Quality Assessment Systems in the Nordic Countries: developing a balance between external and internal needs? Higher Education Policy, 12(1999), 3-14.

Stringer, M., & Finlay, C. (1993). Assuring Quality Through Student Evaluation. In R. Ellis (Ed.), Quality Assurance for University Teaching (pp. 92-112). Buckingham: SRHE and OUP.

West, R. (1998). Learning for Life: Final Report of the West Higher Education Review Committee . Canberra: DEETYA. W

oodward, R. (1993). Institutional Research and Quality Assurance. In R. Ellis (Ed.), Quality Assurance for University Teaching (pp. 113-132). Buckingham: OUP.

 

Conference home page About the conference Conference program Conference papers

TEDI Home UQ Home

 

Teaching and Educational Development Institute,
The University of Queensland
Brisbane, Queensland 4072 Australia
    Phone: +61 (7) 3365 2788
Authorised by: ACE Group, Teaching and Educational Development Institute
Modified: 8 March 2002
© 2002 The University of Queensland