Abstracts

 

Giving feedback from a non-graded, integrated, criterion-referenced examination?
H.G. Alexander, S.E. Chippindall, J.R. Jass, & C.B. Campbell
Graduate School of Medicine, The University of Queensland

At the Graduate School of Medicine at the University of Queensland, we faced the problem of how to give students feedback from a new style of assessment. Examination papers have been exempted from release for the first two cohorts of students in Years 1 and 2 as the types of assessment used were new to the School. The summative assessment for the first year of the Graduate Medical Course (GMC) is an integrated examination consisting of written papers and an OSCE-style examination, after 30 weeks of problem-based learning. The course is structured on 4 domains, each of which the students must pass, to progress after each summative assessment. The summative assessments are integrated, with questions written and marked by multi-disciplinary teams. Non-graded passes are used in the summative assessments in Years 1 and 2, with a graded pass in Year 4.

In this session, the form we have designed for feedback from summative assessment to the Year 1 students in 1997 will be presented for discussion. The same form has been used during 1998 for feedback from formative assessment. Several issues will be raised for discussion, including: the release of marks from a non-graded examination; the place of discipline-based information in feedback from an integrated examination; and the inclusion of the average result for the whole year in feedback from a criterion-referenced assessment.

 

What they know, what they don't know and what they think they know
H.G. Alexander, A.E. Dugdale, & D.A. Price
Graduate School of Medicine, The University of Queensland

Traditionally, we have assessed knowledge by measuring the number of correct responses that students make in examinations. All else has been classed as 'ignorance'. However, all graduates have false knowledge, which can lead to errors in practice. The prevalence and effects of false knowledge have been largely ignored in education.

This paper will present data from the assessment of 658 medical students who completed their degree at the University of Queensland from 1993 to 1995. The study showed that students graduate with a mixture of true knowledge, false knowledge and admitted ignorance. Some of the high ranking students had considerable false knowledge. The clinical and educational relevance of measuring true knowledge, false knowledge and ignorance in assessment, will be discussed.

 

Planning integrated assessment across 12 departments in a PBL course
H.G. Alexander, J.R. Jass, & C.B. Campbell
Graduate School of Medicine, The University of Queensland

The Graduate Medical Course (GMC) at the University of Queensland commenced in 1997 and is based on Problem-Based Learning (PBL). There is a strong link between curriculum development and assessment in the GMC. The course is structured on 4 domains, each of which the students must pass, to progress after each summative assessment. Twelve departments are involved in teaching the course. The summative assessment is integrated across all of these departments, is criterion-referenced, and is supported by frequent formative assessments. All assessment material is developed and marked by multi-disciplinary teams. This paper will describe the problems we faced in developing and marking assessment material across 12 departments in two different faculties and will describe strategies we have developed to overcome some of the difficulties.

Click here to read full paper

 

Brief round table: Standard setting in a criterion-referenced examination
H.G. Alexander, J.R. Jass, & C.B. Campbell
Graduate School of Medicine, The University of Queensland

Criterion-referenced assessment was introduced with the new Graduate Medical Course (GMC) in 1997. The summative assessment in the GMC is integrated and represents a major change from norm-referenced, department-based examinations. The assessment in Year 1 of the GMC includes written papers and an OSCE-style examination and covers material from the 12 departments involved in teaching the course. We have included 'minimum responses' with all summative and formative assessment questions and have also scored all questions numerically. The purpose of this session is to briefly present the issues we have faced in standard setting in this criterion-referenced assessment, to present results from our dual marking systems and to generate discussion and debate about standard setting, relevant to all faculties.

 

Computer managed testing on an in-house network
Trevor Appleton
Department of Chemistry, The University of Queensland

The Chemistry Department at the University of Queensland has been using a Computer-Managed Testing System (RLT) since 1996, predominantly in first level subjects, but with some upper-level subjects. The system is deliberately set up on an in-house network, rather than on www, so students have to physically come to the Chemistry Building to log on. A student logs on, prints out a test, unique to each student, takes it away to work out the answers, and returns within a preset deadline period to enter answers, which are marked for instant feedback. This enables large numbers of students (> 1500) to be accommodated with a small number of terminals (12). Procedures adopted to provide assistance to students using the system while keeping staff costs within bounds will be discussed, as well as policies relating to the proportion of overall assessment that is appropriate from Computer-Managed Testing.

 

BrainZone use in Physiology and Pharmacology
Alison Bailey
Department of Physiology and Pharmacology, The University of Queensland

The online assessment program BrainZone has been used over four semesters and in almost all first and second year Physiology subjects. It has been shown to improve students' performance significantly but was used by less than half of enrolled students when only formative tests were available. Introducing summative tests, worth 10-15% of final assessment, encouraged use very effectively with 75 -95% of students doing related formative tests. Surveys show that students like BrainZone and find it very helpful. A questionnaire (400 respondents) indicated that although 71% of respondents used it within the department, 42% accessed it from a library or college and 37.5% from off campus, reducing pressure on departmental facilities considerably.

Click here to read full paper

 

Assessment and learning –unlearning bad habits of assessment
David Boud
University of Technology, Sydney

Click here to read full paper

 

Assessing individual achievement on group-based work
Nola Caffin
Food Science and Technology Group, School of Land and Food, The University of Queensland

Problem based learning (PBL) requires assessment of the outcomes of group-based work. This paper reports on the assessment developed for a 1st year PBL subject called QF117 Nutrition and Food Science. The core activities of the subject were a number of problems and debates. Each student's performance in the debates was assessed and their activity portfolio, made up of their individual research on each problem, was marked on a regular basis. This enabled a student's individual performance to be assessed in these group-based activities. The final exam was based on the type of problems encountered through the semester giving students an opportunity to demonstrate skills gained in the subject.

Click here to read full paper

 

How can assessment be relevant for students who are full time professionals?
Glenys Carlson
Department of Occupational Therapy, The University of Queensland

A subject titled "Advanced Professional Issues" has recently been developed as a component of new postgraduate coursework options available through the Department of Occupational Therapy. The students enrolling in the subject are usually full time working professionals. The aims of the subject include increasing the students' personal strategies for and confidence in dealing with current issues within the human services sector. The assessment is comprised of a seminar and written assignment both of which involve the application of relevant theory to the students' work situation. The students develop a seminar about a specific issue encountered at their workplace. The written assignment is an analysis of the organisation within which they work. Both assessment items involve formative approaches and criterion based marking. The students' reactions and degree to which the aims of these assessments were achieved will be discussed.

 

Developing criterion referenced assessment: mistakes, successes and things to think about
Terry Carlson, Trish Gorely, Doune Macdonald, Stephanie Hanrahan, & Robin Burgess-Limerick
Department of Human Movement Studies, The University of Queensland.

This paper reports the findings of a year long project examining the implementation of criterion referenced assessment (CRA) within the Department of Human Movement Studies. Data were gathered through interviews and focus groups with staff and students. In addition, successive drafts of subjects' criteria and standards, and samples of students' responses were collected. Results suggested that CRA was perceived as useful by staff and students to clarify assessment expectations and enhance learning thus confirming its alignment to authentic assessment processes. There is no right way of doing CRA providing time is spent developing a shared understanding between all participants (i.e., staff, students, markers, practicum supervisors). Details of the difficulties encountered along the way, the successes we had and recommendations for the future will be discussed.

 

Round table: Assessment of personal development in professional courses
S.E. Chippindall, M. Parker, H.G. Alexander, R. Brown, R. Kerr, J. Harlen, C. Rosie, C.B. Campbell & M. Shapiro
Graduate School of Medicine & Department of Social Work and Social Policy, The University of Queensland.

The traditional discipline based approach of medical education within the University of Queensland has been replaced with a model which is horizontally and vertically integrated, and places greater importance on the personal and professional development of students. Noncognitive traits are of equal importance to cognitive traits in medical practice. Medical education has traditionally neglected these non-cognitive aspects. Poor performance in non-cognitive aspects of medicine has been identified as a small but significant problem among physicians (Herman, Veloski & Hojat 1983). This poor performance may affect later offers of further training (Herman, Veloski & Hojat 1983) and may also lead to malpractice action (Miller et al 1989).

The need to assess personal and professional development has been identified within the GMC. By assessing the non-cognitive traits of medical students, and thus encouraging students to develop desirable qualities, high quality care provided for patients will be facilitated, as will enhancement for graduates' later career prospects.

An instrument has been developed to address these needs. The GMC aims to recognise students who have difficulties, provide a clear process of handling problems, develop preventive measures to avoid problems in professional practice, and identify students needing remediation, before, and possibly after, graduation. The instrument will be presented briefly, and this session aims to generate discussion across faculties about this area of assessment.

References

 

Assessing graduate capability for nursing practice
Diane Collins1, Deanne Gaskill1 and Gail Hart2
1School of Nursing and 2Teaching and Learning Support Services, Queensland University of Technology

Current nursing students are being prepared for a largely unknown and unpredictable global workplace. As graduates they will be required to tackle new challenges in a health care sector marked by change. They will need confidence in their ability to accurately assess unfamiliar situations, select appropriate and ethical interventions and evaluate the impact of their actions. Such confidence suggests a sound understanding of their individual approach to learning, a commitment to peer consultation and collaborative practice, and an orientation to flexibility and adaptation. To ensure the ongoing relevance of their professional skills they will they will need to be able to reflect upon, and learn from, their experience of practice. The assessment of these requisite graduate capabilities is challenging. Ideally the assessment process should offer a learning experience that reflects the assessment of performance in the workplace and emphasises both professional and personal development. The assessment process should also model a professional commitment to accountability. A case study in nursing at Queensland University of Technology (QUT) is used to illustrate a student learning experience that focuses on professional, ethical and global issues and the development of skills to support life long learning. The assessment is designed as an authentic learning experience that acknowledges individual differences and fosters skills in self and peer assessment.

Click here to read full paper

 

Using role-play to assess and enhance the integration of learning in first year physiotherapy students
Robyn Cupit and Rosemary Isles
Department of Physiotherapy, The University of Queensland

A first year physiotherapy subject which had as its themes the existence of rights, the nature of caring and effective communication methods was assessed using student generated role plays. Various scenarios, designed to encourage integration of learning and involving a physiotherapist and patient, were used as triggers. The performances were videotaped but assessed immediately using criterion referenced assessment.

Role play presentations were generally of a high standard with maximised verbal participation of students. Students demonstrated commitment to the exercise, understanding of the key issues and ability to integrate knowledge and skills in a realistic setting.

Role-play provided an effective opportunity to assess not only the acquisition of knowledge and skills but the confidence in the application of them.

Click here to read full paper

 

Towards an understanding of the role of feedback in instructional design
Shani Doig
Faculty of Education, Griffith University

The important role of `feedback' in improving instructional design in general and student performance in particular has long been recognised by educational researchers (Mory, 1992). There is still considerable variation however, in educational theory as to the exact nature and function of instructive feedback. Specifically, there would appear to be some disparity in thought regarding how the provision of purposeful feedback to students should occur. While the term `feedback' remains problematic in education, scientific definitions of the term are considerably clearer. The models of `feedback' provided in science through discussions of process-control systems and cybernetics, while not providing all the answers, have much to offer by way of understanding how educational feedback can be used in the classroom context. This paper aims to explore scientific definitions of the term `feedback', and to compare and contrast these definitions with those provided for educational feedback. It is argued that `feedback' is an essential part of the educational process but one which cannot be approached in an ad hoc or random manner. Rather, feedback needs to be considered within the wider schema of educational objectives and outcomes if it is to improve student performance.

Click here to read full paper

 

"Plan? What plan?" - how law students approach assessment
J. Giddings
Faculty of Law, Griffith University

The Law Faculty at Griffith University is developing a student resource with suggestions regarding how students can best approach various forms of assessment. The catalyst for this project was a concern that some students were not making the most of their efforts. In particular, some students 'freeze' when they encounter hypothetical problems in exams. Information for this assessment resource is coming directly from Griffith law students through a series of focus groups. The presentation will outline student responses, which to date have been very interesting and, in some respects, surprising. While students have talked about their techniques for various forms of assessment, they have also been vocal in relation to how academics can improve the assessment process.

Click here to read full paper

 

Sharing the load: a collaborative approach to developing and implementing criterion-referenced assessment in professional degree courses
Merrilyn Goos and Karen Moni
Graduate School of Education, The University of Queensland

This paper focuses on our approach to implementing criterion-referenced assessment, which we refer to as "sharing the load". Our particular goals have been to develop coherent and specific sets of criteria and standards, and to offer students assessment tasks that are both authentic and purposeful. We discuss five main themes:
  1. collaboration between lecturers from different disciplines
  2. collaboration between students on assessment tasks
  3. the use of formal and informal peer assessment
  4. the design of assessment tasks with multiple purposes and a range of outcomes
  5. criteria and standards drawn from professional education practice.

Click here to read full paper

 

Lowering anxiety through take home exams and raising it through videotaped clinical skills practice
Kathryn M. Gow
Queensland University of Technology

The new QUT Clinical Hypnosis Graduate Diploma is the first university course of its kind in Queensland and only the second in Australasia. Geared towards busy health professionals, it must be flexible in all aspects of delivery and assessment. Combining the individual needs of these full fee paying students (who may or may not be technologically literate) with national practice standards means that sooner or later, either the anxiety of the students or the coordinator is raised. This paper describes the different assessment procedures required by a successful privately conducted professional practice course and a successful graduate university academic course.

Click here to read full paper

 

Engendering student learning by use of specific assessment strategies
D J Hargreaves
School of Mechanical, Manufacturing and Medical Engineering, Queensland University of Technology

All assessment practices reflect a number of assumptions relating to the nature of learning and the purpose of assessment. If educational institutions intend to equip graduates with attributes necessary for professional practice, for example critical thinking and lifelong learning capabilities, then current assessment practices must be examined. Teaching, learning and assessment are inextricably linked and assessment is the most significant motivator for learning. A number of innovative approaches to assessment have been implemented at Queensland University of Technology. This paper describes initiatives such as student-generated examinations and active participation in tutorials sessions.

It is a well known dictum that the best way to establish whether you know something is to try to teach it to someone else. On a similar basis, students were asked to demonstrate their knowledge of a particular topic by setting an equivalent examination question with its accompanying solution as an assignment instead of a mid-semester test. It was optional and about 75% of the class of 130 undertook this exercise.

Students tend not to attend tutorial sessions for a range of reasons, so as an incentive, a special approach was implemented. Students were awarded a few marks for attendance but they were required to work in groups of about ten to solve a problem given by the tutor. The groups were selected on a random basis from a class of 140 students. Each group is required to solve the problem and after about 30 minutes, one of the group is required to present both the problem and its solution to the whole tutorial group (about 40 students). The improvement in grades in the end-of-semester examinations was 8% when this approach as first adopted and 5% in the second instance.

Click here to read full paper

 

Brief briefing: peer and self assessment
Geoff Isaacs
Teaching and Educational Development Institute, The University of Queensland

Brief briefings consist of a 20 minute talk with ample time for questions and discussion. They are supported by brief documents on the Web outlining the main features of the topic and giving some key references to the literature.

When students assess other students studying the same subject, that is called peer assessment; when students assess their own work, that is called self assessment. Both peer and self assessment are fairly commonly used to assess individual contributions to group tasks. However they are also very useful for assessing tasks carried out by students individually and as a means of providing students with feedback on their work. Their application in large classes looks especially promising.

Click here to read full paper

 

Comparison of 'traditional' and 'criterion-referenced' assessment of an essay question in a neurological physiotherapy module
Rosemary Isles and Nancy Low Choy
Department of Physiotherapy, The University of Queensland

Two experienced teachers each marked fifty essay questions on Neurological Physiotherapy by both 'traditional' and 'criterion-referenced' methods. Scores and grades were compared between methods and between examiners.

Results showed no significant difference between methods for the whole group for scores or grades and no significant difference between examiners for grades using each method. There was a significant difference between examiners for scores by the traditional method and also between methods for scores by each examiner.

Criterion referenced assessment was easier and quicker to use, but it was difficult to set standards for criteria to be used by more than one examiner when the question was complex.

(This study was supported by Quality Funding (Assessment) to the School of Health and Rehabilitation Science.)

Click here to read full paper

 

Dimensions of oral assessment and student approaches to learning
Gordon Joughin
Queensland University of Technology and Griffith University

This paper will:
  1. outline some of the key qualities of oral assessment by suggesting six dimensions of oral assessment that can be used to describe and analyse any type of oral assessment;
  2. suggest how these dimensions might be expected to influence students' approaches to studying, and
  3. describe students' perceptions of oral assessment in two disciplines (law and theology) and how these perceptions appear to be related to their approaches to studying.
Participants will be invited to consider these dimensions, and perhaps add to them, in the light of their own experience of oral assessment, and to contribute to the discussion their perceptions of how oral assessment could be used to improve your own students' learning.

Click here to read full paper

 

On the road to flexible assessment
Marie Kavanagh
Department of Hospitality, Tourism and Property Management, The University of Queensland

One of the major problems confronted by most academics in providing a high educational standard is ensuring that one of the measures of this standard, student success, actually does reflect the ability of the majority of the students within the group. This paper investigates some of the factors which affect the results of students in a large first year financial accounting class. It examines the impact of different assessment techniques on the results of students from different backgrounds (cultural, educational, intellectual) and relates students attitudes to different assessment techniques to results achieved. It highlights the need for flexibility in assessment techniques and makes practical recommendations as to how this might be accomplished.

 

Implementation of an integrated, technology-based, discovery mode assessment item involving an "incubation period" to enhance learning outcomes for engineering maths students
Neil Kelson & Anand Tularam
School of Mathematical Sciences, Queensland University of Technology

In this paper we discuss our current efforts to develop and implement an exploratory, discovery mode assessment item into the total learning and assessment profile for a target group of about 100 second level engineering mathematics students.

The assessment item under development is composed of 2 parts, namely, a set of "pre-lab" homework problems (which focus on relevant prior mathematical knowledge, concepts and skills), and complementary computing laboratory exercises which are undertaken within a fixed (1 hour) time frame. In particular, the computing exercises exploit the algebraic manipulation and visualisation capabilities of the symbolic algebra package MAPLE, with the aim of promoting understanding of certain mathematical concepts and skills via visual and intuitive reasoning, rather than a formal or rigorous approach.

The assessment task we are developing is aimed at providing students with a significant learning experience, in addition to providing feedback on their individual knowledge and skills. To this end, a noteworthy feature of the scheme is that marks awarded for the laboratory work are primarily based on the extent to which reflective, critical thinking is demonstrated, rather than the amount of CBE-style tasks completed by the student within the allowed time.

With regard to student learning outcomes, a novel and potentially critical feature of our scheme is that the assessment task is designed to be intimately linked to the overall course content, in that it aims to introduce important concepts and skills (via individual student exploration) which will be revisited somewhat later in the pedagogically more restrictive formal lecture component of the course (typically a large group plenary format). Furthermore, the time delay involved, or "incubation period", is also a deliberate design feature: it is intended to allow students the opportunity to undergo potentially important internal re-adjustments in their understanding, before being exposed to lectures on related course content which are invariably delivered in a more condensed, formal and mathematically rigorous manner.

In our presentation, we will discuss in more detail our motivation and rationale for trailing such a scheme for the targeted student group. Some of the advantages and disadvantages of our approach (as we perceived them at the initial stages) will also be enumerated.

In a companion paper, the theoretical framework for our approach will be more fully elaborated, and measures of student learning outcomes (as obtained from eg. student provided feedback) will be discussed.

Click here to read full paper

 

Opening Keynote Address: Changing a culture
Professor Andrew Lister
Executive Dean, Faculty of Engineering, Physical Sciences and Architecture, The University of Queensland

In 1996 the University of Queensland adopted radically new assessment policies. Key changes were a move from norm-referenced to criterion-referenced assessment, the abandonment of a recommended distribution of grades, and a view of assessment as primarily a contributor to learning outcomes rather than an instrument for ranking students. Professor Lister will describe why these changes were made, how they were effected, and the extent to which an embedded culture can really be changed.

 

Assessment using electronic discussion groups: some strategies, problems and issues
David Neil
Dept of Geographical Sciences & Planning, The University of Queensland

Electronic discussion groups have been included as an integral part of teaching, learning and assessment in a second year subject (Sem. 2 of 1996, 1997 & 1998) and a third year subject (Sem. 1 of 1997) in Geographical Sciences & Planning. Some of the advantages and disadvantages of assessing students by grading their contributions to electronic discussion groups are outlined. Approaches to assessment of these contributions also warrants consideration because the structure, quantity and style of submitted material differs from that in essay type assessment items. Strategies which have been considered and/or implemented to address these differences are discussed (eg. targeted, random, total, peer and tiered assessment strategies). Problems and issues associated with this approach to assessment will be considered.

 

Development of generic criterion referenced assessment procedures in the Department of Geographical Sciences and Planning
David Neil, David Wadley & Stuart Phinn
Dept of Geographical Sciences & Planning, The University of Queensland

The paper outlines an attempt to develop a generic approach to criterion referencing for written assessment items in one department of the University. This approach is largely a response to the lack of guidance provided by the University and the desire to avoid each staff member having to "reinvent the wheel" of specific criteria and standards. We outline the procedure we used, the response of colleagues, issues raised by the process and we describe the structure and content of the final outcome.

 

Three ways to pursue academic quality?
Duncan D. Nulty
Teaching and Learning Development Unit, Queensland University of Technology

When, as academics, we assess our students' written work we presumably give marks for those features or qualities which we value. If so, it seems clear that our marking behaviours largely reflect our academic values. Research (Nulty, 1997) has pursued this simple line of reasoning and proposed six dimensions of academic quality. While these manifest differently in different disciplines, they are of sufficient generality to have wide applicability.

This paper will briefly re-visit what those dimensions are and then spend some time exploring three possible uses which can be derived from a conscious awareness of these dimensions. Specifically, knowledge of the academic values which underpin our marking behaviours can be used to improve:

Click here to read full paper

 

Is it different in Business? - A review of the assessment policy and practices in the Faculty of Business at the QUT
Jennifer Radbourne
Faculty of Business, Queensland University of Technology

The purpose of this presentation is to report on the process and initial findings of a review of the assessment policy and practices in the Faculty of Business at QUT. The review has embraced a broad range of issues including norm and competency based assessment, the place of individual and group assessment, the role of peer assessment, the use of industry and classroom assessment, the impact of assessment policies on international students, assessment and flexible delivery, the relationship of assessment to learning objectives and outcomes, and the perception of difference in assessment within business education when compared to other disciplines, studies and sciences within a university.

Click here to read full paper

 

Keynote Address: Formative assessment: The role of criteria and standards in improving learning
Professor D. Royce Sadler
Faculty of Education, Griffith University

Professor Sadler elected not to produce a written paper for the conference, but rather to talk about work in progress. However he has agreed to make his speaking notes available on request, with the warning that they may not be very comprehensible to those who did not hear the presentation.

 

Assessing 2 million words per semester: Managing essay assessment for a large first year Psychology subject
Virginia Slaughter
School of Psychology, The University of Queensland

This paper presentation will outline the development and use of essay assessment in a large (1800 students) first year Psychology class. The paper will provide a case study of essay assessment in a large group teaching context, and will highlight issues specifically related to assessment of large groups. The following topics will be addressed in the talk:

 

"What's the problem?" Teaching Business Law students to identify and solve legal issues in exam hypotheticals by means of corrected or augmented student answers
Barbara Smith
Faculty of Law, Griffith University

Business Law (in non-Law schools), a subject with large enrolments, uses typical Law assessment methods but does not have time to teach them. Many students need to see written examples of legal reasoning to learn it. By selecting, arranging, and commenting on samples of student answers to typical problem-type or hypothetical questions, teachers can help students learn how such legal problems are analysed and answered, and thus better prepare students for assessment.

Ms Smith will also be showing what she has done and will be available to discuss it over lunch on the first day of the conference.

 

Balancing a variety of learning needs in assessments in an education subject
Ruth Smith
Education, James Cook University

This paper describes the assessment in a second year university subject and how it aims to develop a number of skills relevant both to a general university education and teacher development.

All assessments aim to develop knowledge in the subject as well as other skills. Student tutorial participation is assessed on participation and a presentation in pairs, which develops teaching skills. The major assignment develops literacy, research and analytic skills. The final open-book examination tests breadth and depth of subject knowledge.

The paper discusses the strengths and weaknesses of each component for students and teaching staff, strategies for student preparation of each component, minor changes to the assessment practices over two years, as well as some general observations about students' and teaching staffs' perceptions of fairness and validity in grading.

 

Introduction of criterion referenced assessment in biological science laboratory subjects
Dianne Stomfay1 and Hock Ng2
1Department of Microbiology and 2Department of Botany, The University of Queensland

In response to the University's change in assessment policy, criterion referenced assessment was incorporated into various Science laboratory subjects in 1998. The introduction of criteria and standards for assessment items is a new teaching and learning experience for the whole teaching team (Subject co-ordinators and bench tutors). Students receive their assessment information at the start of semester; this enables students to clarify any matters well in advance of the assessment date. This paper describes examples of criterion referenced assessment trialed in Botany and Microbiology laboratory subjects. We outline some preliminary feedback from tutors and students, and expose some of the unexpected hiccups. Finally, we focus on ways to improve students' understanding of assessment, which we hope will enhance tutor-student interactions, and increase the effectiveness of learning in a laboratory class setting.

Click here to read full paper

 

Lecturing and tutoring critically in tertiary mathematics
Anand Tularam and Neil Kelson, School of Mathematical Sciences, Queensland University of Technology

In the past few years computer based teaching appears to have become a common feature in education generally, particularly in primary and secondary education areas (Nason & Kidman, 1997). The computer based method (CBM) of presenting domain specific knowledge appears to have a number of positive features. For example, students generally seem to be more motivated to work on computers. There is also some evidence that suggest that students learn to be reflective in their thinking after using the CBM (Patterson & Smith, 1986). Moreover, the CBM appears to be pedagogically more effective in that it seems to allow for various learning styles. Indeed, students who are kinesthetic oriented, reflective, and visual learners can use their dominant learning styles. The literature student learning is more effective when they use their preferred learning styles. Further, while working on computers students have the opportunity to observe their peers (screens) and thus effectively engaging in group discussion or cooperative learning (Smith, 1996; Webb, 1989). It can be argued that CBM also allows students to be more independent and thus take greater responsibility for their learning generally.

While there is some research evidence to suggest that students are motivated and also engaged in higher-order thinking when working on computers, there appears to be room for further work in this area, particularly in the tertiary education arena. Indeed, the computer based way of presenting learning materials has provided the impetus for tertiary educators to examine the effectiveness of this type of instruction in terms of whether the students are in fact acquiring the necessary skills that enhances their ability to problem solve and transfer knowledge. In their report on the use of technology in language learning, the National Board of Employment, Education and Training argued that, while there is a need for change in practice due to the influence of technology, the change is only warranted if there is evidence that learning is more effective (National Board of Employment Education and Training, NEET, 1996).

In following their view, we investigated the effectiveness of a novel way of teaching Fourier series to second year engineering mathematics students. However, we approached the teaching of Fourier series in a different manner. We incorporated the view presented by Rubenstein (1975) with regard to expert problem solving and creativity. After examining a number of experts' knowledge development, creative abilities and achievements, Rubenstein suggested that creativity involves the linking and "connectiveness" of knowledge (that is, cognitive connection building). He argued that such a process usually occurs after an "incubation" period. It seems that applying this incubation period method to the teaching of difficult higher level concepts of mathematics may be a useful teaching method. For example, the learners may need to learn concretely, visually, and symbolically first and then after an incubation period re-examine the content and/or procedures in a more formal manner (Piaget, 1968, Tall, 1991). In this way, the learners can reflect on their work and examine the content more critically. According to constructivist educational theorists such a process may facilitate the connectiveness of knowledge through the linking of concepts and procedures into a logical/coherent cognitive structure (Clements & Battista, 1990; English, 1997; Neiss,1993; Sternberg, 1988).

Fourier series has a number of applications in mathematics. It is a topic that all mathematics majors need to learn. However, anecdotal evidence suggests that students find the complexity the Fourier series difficult to deal with. Indeed, there is empirical and anecdotal evidence that indicate the majority of the tertiary students fail to adequately grasp such concepts. This in turn causes much distress, and negative feelings and attitudes toward mathematics as a subject at the tertiary level. Such a student view of mathematics is damaging in the long term given the growing importance of mathematical methods in non-mathematical fields. Importantly, the applicability of mathematics has quickly grown to areas that were traditionally regarded as non-mathematical. Areas such as biology, business and economics are a few where mathematical modelling has been used extensively. These areas are important new fields of mathematical research in the modern times. This suggests that students in various disciplines would also be required to learn higher level mathematical concepts and procedures such as Fourier series. Thus, it would seem that any suggestions for improvements in the teaching of Fourier series would be appreciated by many tertiary mathematics educators.

Method
In this laboratory program, a computer based on-line method was used to provide students with pre-lecture laboratory experience. The laboratory was not simply an on-line visual based tutorial but rather a combination of hands-on, visual and written/research work. In this manner the authors were able to examine, attitudes, beliefs and patterns in thinking to decipher the nature of thinking undergone during the tutorial (Schoenfeld, 1985; Tularam, 1998). The Fourier series program was designed to examine whether, given the opportunity, students self-engage their higher mental faculties of metacognitive and critical thinking while working on-line. The questions posed in the tutorials may have created some tension and conflict in students' minds and to resolve such conflicts, students needed to move away from the computers and engaging in written, reflective and/or critical thinking. Indeed, this "movement away" from the computer to written or other reflective research work, was an important aspect of the self-learning laboratory program. In such actions students' thinking was explored to determine whether they were indeed engaging their higher faculties (Flavell, 1987).

In this paper, we will present some analysis and results of the surveys and interviews conducted at this early stage of the program. The surveys and interviews were conducted after the initial laboratory program and after the lectures on the Fourier series which was presented several weeks later.

Nonetheless, there are a few areas that need further investigation. For example, do the students who experience technology teaching gain metacognitive/reflective and critical thinking skills while working on computers in an "on-line" manner. While there is some evidence to suggest that students do gain the necessary higher-order thinking skills there needs to be more work done in this area. Indeed, the new way presenting learning materials has provided the impetus for educators the examination of the effectiveness of computer based instruction in terms of whether the students are in fact acquiring the necessary skills that enhances students' ability to transfer knowledge.

Click here to read full paper

 

Assessment of general practice consulting skills using standardised patient interviews
Michael Yelland
University General Practice, University of Queensland

This paper describes the use of standardised patient interviews as a means of assessing the consulting skills of Fifth Year Medical students studying general practice. Volunteers play standardised roles of common clinical problems. These interviews allow a more valid assessment of consulting skills than written assessments. They offer greater control of the choice of clinical problems and better reliability in assessment than real patient interviews. A detailed mark sheet has been developed to increase inter-examiner reliability, discourage global marking and incorporate a patient assessment of the student. Problems with this form of assessment are discussed and solutions suggested.

Click here to read full paper

 

Conference Entry Page About the Conference Conference Program Abstracts Copyright information

TEDI Home About Assessment UQ Home


This site built on a Macintosh using Dreamweaver.. Last modified 20/1/99; 10:14:11 AM.