Published with permission from the Australian Veterinary Journal. Complete text in: Aust Vet J (1997);75:120-125.

Abbreviations: Problem-based learning (PBL), electrocardiogram (ECG), Teaching and Educational Development Institute (TEDI)

The aim of this paper is to report a subject-based form of PBL that was introduced into the fourth year of a five year veterinary course at the University Queensland, and to report the results of student evaluation of this form of teaching.

Materials and methods

In 1995, PBL was introduced into parts of two fourth year veterinary subjects at the University of Queensland, which has a five year veterinary course with approximately 85 students per year. Endocrinology, which previously consisted of five 50 minute lectures in General Medicine (containing 52 lectures), and feline medicine which previously consisted of 11 lectures in Small Animal Medicine (containing 24 lectures), were converted to PBL. These sections were taught by the same lecturer from 1992.

In 1995, when endocrinology and feline medicine were changed to PBL, lectures were changed to class discussion sessions, and material was learnt through solving the problem. In the week prior to the class session, students were given a problem to solve and worked in groups of three to four in 1995, and in pairs in 1996. The problem consisted of a detailed history and results of a physical examination based on a real patient, and required that the students be able to answer the question: What is your diagnosis and treatment for this animal?. The students were directed to work through the problem during the week, and to submit an assignment from the group at the beginning of the class session. The assignments were a summary of their problem-solving process and learning outcomes, and contained a series of questions to guide them through problem-based diagnosis.

Groups were given a limited amount of money they could spend working up the case, which was generally between $250 and $500, depending on the complexity of the case. A list of approximately 180 available tests was provided, and each test was given a code-number and price. Tests which required anaesthesia and/or surgery had these costs included. Prices were approximately those currently charged by the Companion Animal Practice at the University of Queensland.

Results of tests selected by the students were provided using the Scanvet program (Dr Jill McClure, Louisiana State University, Baton Rouge, Louisiana, USA). The computer program allowed laboratory data from the patient to be entered by the teacher, and accessed by the students as required. To obtain results of laboratory tests, students marked an optical-mark reader scan-sheet in the position which corresponded to the code-number of the test. This was fed through an optical-mark reader (Scantron 1300, OMR Date Terminal, Tustin CA, 92680 USA) which was connected to a DOS-based computer and printer. Either the result of the test requested was printed, or if X-rays or ECGs were requested, the print-out directed students to look in a coded envelope containing the X-rays or ECG tracing. If it was anticipated that students had insufficient prior knowledge of that area, and that the learning issues were outside the objectives of the course, some interpretation of X-rays or ECGs was provided.

The class session, which was previously a lecture, was used as a facilitated interactive feedback session with the whole class (85 students). Student groups were asked questions by the teacher that covered the important learning issues from the problem. Approximately two-third of the class would receive questions at each session. The purpose of the questions was to ascertain whether the learning issues were correctly understood. Students were also encouraged to ask questions. Where there appeared to be insufficient or incorrect understanding of an area, the important issues were clarified by the teacher.

Assessment of students
In the PBL sections, assessment was from assignments (30%) and an end of semester exam (70%). The assignment mark represented the work of the student group. Students were given a detailed feedback summary based on the assignment, and asked to mark their own assignments based on this summary to obtain feedback on their performance. The mark for assignments used in their final mark was based on a quality component (30% of assignment mark) and a completion component (70% of assignment mark). To obtain full marks for the completion component, students had to submit both the original assignment on the due date, and the marked assignment by the end of semester. The quality mark was given by the teacher, and was based on one assignment in the endocrine section and two in the feline section. These assignments were chosen at random for the whole class from the submitted assignments. The students self assessed mark was not used in the quality mark. Questions in the end of semester examination had a similar format to the assignments, but laboratory data were provided as part of the question instead of by the Scanvet program, and diagnostic tests were not limited by cost. Textbooks and notes were not permitted in the examination.

Student evaluation of the subject
Student perception of the learning outcomes, the course and teacher were obtained from the teaching feedback report from TEDI (University of Queensland). This report was based on results of a questionnaire with 10 mandatory questions and 12 questions chosen by the staff member from a bank of 183 questions. Questionnaires were given to students at the end of the last interactive session for the subject, and were directly returned to TEDI for analysis.

Statistics
The overall distributions of student responses for feline medicine were compared between years using c2 analysis. This analysis was also used to compare the distribution of student responses for feline medicine with the university distributions for the corresponding year. Where available, results for feline medicine were also compared with the distributions for the veterinary faculty for the corresponding year. Distributions of responses for 1993 (traditional lectures) provided the expected values for distributions obtained for 1995 when PBL was used. Specific goodness-of-fit analyses were conducted to identify where changes in the distributions were occurring. Given the number of correlated tests, it was necessary to use Bonferronis correction7 to maintain an experiment-wise a-level of 0.05.

Results

Results of student evaluation
The distribution of student responses for PBL in 1995 differed significantly from the distributions obtained university wide for all questions (see Table 1 and Table 2 for c2 results). Distributions for both PBL and the lecture-based subject were also significantly different from distributions obtained from the veterinary faculty for the corresponding year (Table 2). Most of the distributions for PBL also differed from those obtained for the lecture-based subject (1993). There were no differences in the students evaluation between the different methods of teaching for the lecturers effectiveness, the subject, and for how well the material was structured. Where the overall distributions differed significantly, goodness-of-fit analyses indicated that in every case, the responses to PBL was in the direction of increased approval for the method. For example, for the statement I have developed the ability to solve problems in this field, in 1995, 69% of the respondees agreed strongly and 29% agreed, compared with 37% agreeing strongly and 50% agreeing in 1993. Examination of these two responses alone yields a c2 of 24.71 with the shift to strong agreement responsible for 77% of the c2 value. Similar results were obtained for the other statements.

On the back of the student evaluation forms were the questions What are this staff members strengths in teaching? and What improvements would you suggest?. The overall response was that it was an enjoyable and effective way to learn. Many students suggested that more teaching should be done by this method. One student commented that the learning curve was maintained throughout the semester, rather than just a pre-exam cram. When students were asked if they wanted the same PBL approach in feline medicine (second semester) as in endocrinology (first semester), 33 of the 49 students who answered the question were positive, and 16 said they would like a mix of lectures and PBL. None of the students requested lectures alone.

The most frequent criticism was the amount of time involved in the assignments.

Cost of course development
The cost to set up the course included the computer program ($AUD1,300), an optical-mark reader ($AUD2,400) and computer. The major cost to the department was in staff time, and consequent loss of research productivity.

Discussion

Quality of learning
This study suggests that students believe PBL results in better understanding and problem-solving ability than traditional lectures. In traditional lecture-based veterinary courses, students only have limited opportunity to practise problem-solving using information given in lectures. It is very important for students to practise the problem-solving skills of making a diagnosis, if they are to be competent veterinarians when they graduate. In the present study, there was a significant increase in the percentage of students strongly agreeing that they had developed the ability to solve problems in the same field after the PBL subject. Although no formal assessment of long-term recall was made in the present study, when students were doing clinical practical work three to four months later, they were noticeably better at interpreting laboratory data, making a diagnosis and recalling the appropriate treatment for similar problems, compared to students in previous years who had done the lecture-based subject.

Group discussion
Because of the limitations of time and facilities, the small group discussions were tutorless. There was no time scheduled in the curriculum for the students to meet, and some students indicated difficulty in arranging meetings. It was for this reason that the group size was reduced from three to four students in 1995 to pairs in 1996. The smaller group size may reduce the quality of learning if there is less learning from other students, and less activation of prior knowledge. The class discussion session in this study functioned more as a feedback session than a typical PBL tutorial. Facilitators in PBL discussion groups usually guide students, but do not give answers to questions. In the present study, the facilitator for the class session was an expert in the field. Where misunderstanding of the learning issues was apparent, expert information was given to students. Students were not directed to search for more information, because further expenditure of student time could not be justified for the credit point value of the course. Receiving feedback about the correct solution has been shown to be important in PBL, if the concept learnt through problem-solving is to be successfully transferred to a new problem.¹

Student perceptions
Although the results suggest that students believe PBL results in better learning than traditional lectures followed by problem-solving exercises, the rating for the teacher and subject did not change substantially. One possible explanation is that students can differentiate teacher and subject effects from learning outcome. Alternatively, it may be because learning outcome had greater room for improvement. In 1993, students could only compare the subject with lecture-based subjects, as they had not been previously exposed to problem-based learning in the veterinary course.

Assessment
The major differences between the examination and assignments were the time available to work through the problem, access to textbooks, and individual rather group work being represented. In some ways, the examination more closely resembled how veterinarians have to perform in the consulting room with a client. The conditions for the assignment resembled the situation when an animal is hospitalized, because there were opportunities for consultation with colleagues (other students) and the literature.

Disadvantages of PBL
The only consistent criticism by students was the work-load relative to the credit points, and this largely stemmed from the requirement to complete a written assignment. Although a written report on each problem is not an inherent requirement of PBL, it was felt to aid student learning of the essential issues for the subject.

Some students commented that PBL was good to have in one section of the course, but it was too time consuming for the whole course. However, most problems covered many areas outside the learning objectives of the course, and other subjects could be covered with minimal extra work if they were integrated into the problems.

Students are reported to find the PBL approach more satisfying and enjoyable.² Results of the student evaluation were consistent with this. The interactive sessions were also more enjoyable for the teacher than traditional lectures.

PBL is reported to be a more interesting learning environment than traditional curriculum,³ and results of this study support this. In the present study, students comments that they willingly put more time into the PBL sections because they were interesting, suggests that they were not merely seeking to maximize their overall grades, but were seeking to understand. When interest is strong, and students feel a genuine need to increase their understanding, comprehension improves.⁴ Interest is greater when problem-solving is used to gain knowledge, rather than when knowledge is gained first and the problem-solving process follows.⁴

Changing a course structure usually takes time, effort and financial resources. With the current government funding of universities, it may be uneconomic for veterinary schools to consider PBL, and teachers may be better rewarded by increasing research productivity. However, the time involved in setting up PBL for a subject is probably no greater than for setting up a traditional lecture-based subject, and coordinators of new courses should seriously consider the benefits of subject-based PBL for both students and staff. Subject-based PBL overcomes some of the disadvantages of classical vertically-integrated PBL, which requires total reorganisation of the curriculum, and involvement of virtually all teaching staff.

The present study shows that subject-based PBL can be successfully introduced into a veterinary science course, and that students perception of the learning outcome is significantly better than for the traditional lecture-based subject.

Acknowledgments

We thank Barbara Rigsby of TEDI of The University of Queensland for providing the TEVAL assessments and Geoff Isaacs (TEDI) for providing advice on implementing problem-based learning and on the manuscript. Further development of PBL in 1996 was aided by a Teaching Development Grant from TEDI.

References

1. Needham DR, Begg IM. Problem-oriented training promotes spontaneous analogical transfer. Memory oriented training promotes memory for training. Mem Cognit1991;19:543-557.
2. Schmidt HG. Foundations of problem-based learning: some explanatory notes. Med Educ 1993;27:244-432.
3. Norman GR, Schmidt HG. The psychological basis of problem-based learning: a review of the evidence. Acad Med 1992;67:557-565.
4. Sadlo G. Problem-based learning. Tertiary Education News 1995;5:8-10.

TABLE 1 student responses to questions on perceptions of learning outcome.

Year	Strongly agree (5)	Agree (4)	Uncertain (3)	Disagree (2)	Strongly disagree (1)	c2(df=4)	Number
'I have developed the ability to solve problems in this field'
1995a	69%	29%	1%	0%	0%		68
1993b	37%	50%	12%	2%	0%	37.75d	60
1995c	14%	52%	25%	7%	2%	175.41d	8,622
'I have gained a good (1993) / better (1995) understanding of the field'
1995a	75%	22%	3%	0%	0%		68
1993b	33%	60%	5%	2%	0%	54.60d	60
1995c	21%	55%	17%	6%	2%	130.3d	20,881
'I have learned to apply principles from this class in new situations'
1995a	76%	18%	6%	0%	0%		68
1993b	42%	52%	5%	0%	0%	36.14d	60
1995c	21%	47%	22%	8%	2%	121.50d	4,443
'The teacher stimulated my interest in the subject'
1995a	71%	28%	1%	0%	0%		68
1993b	42%	52%	5%	0%	0%	21.71d	60
1995c	22%	43%	19%	10%	5%	97.76d	28,103
'The assignments tied in with the course objectives'
1995a	90%	7%	3%	0%	0%		68
1993b	63%	27%	8%	2%	0%	20.96d	60
1995c	30%	52%	12%	4%	1%	114.92d	6,495
'The lecturer structures the material well'
1995a	85%	13%	1%	0%	0%		68
1993b	82%	18%	0%	2%	0%	0.95e	60
1995c	34%	49%	10%	5%	2%	81.93d	26,410
'The lecturer emphasised thinking rather than just memorising'
1995a	87%	13%	0%	0%	0%		68
1995c	32%	43%	19%	6%	2%	96.10d	70,525
The lecturer helped me improve my learning skills'
1995a	69%	25%	6%	0%	0%		68
1995c	16%	38%	33%	10%	3%	146.96d	67,416
'The lecturer presented the material in an interesting way'
1995a	85%	15%	0%	0%	0%		68
1995b	25%	46%	15%	11%	3%	133.08d	71,263

1. Responses for subject taught using PBL
2. Responses for subject taught via traditional methods by the same teacher; c2 are for comparison with PBL
3. Responses for entire university in 1995; c2 are for comparison with PBL
4. P < 0.001
5. Not significant

The number of questionnaires used to calculate the university means varied depending on whether the questions were optional or mandatory for all staff.

TABLE 2 student evaluation of teacher and subject.

Year	7 Outstanding	6	5	4 Satisfactory	3	2	1 Very poor	c2(df=6)	Number
How would you rate this subject
1995a	51%	43%	4%	0%	1%	0%	0%		67
1993b	47%	45%	7%	2%	0%	0%	0%	2.19	60
1993c	8%	37%	28%	15%	7%	4%	1%	137.60	1,174
1995d	6%	29%	31%	20%	8%	3%	1%	264.93	32,477
1995e	15%	41%	32%	10%	2%	0%	0%	86.38	475
How would you rate this staff members effectiveness as a university teacher
1995a	70%	27%	3%	0%	0%	0%	0%		66
1993b	57%	40%	3%	0%	0%	0%	0%	4.54	58
1993c	17%	42%	20%	13%	7%	2%	0%	77.57	1,161
1995d	16%	41%	25%	12%	4%	1%	1%	146.59	70,210
1995e	22%	41%	25%	10%	2%	1%	0%	95.32	1,722

1. responses for part of subject taught using PBL
2. Responses for part of subject taught via traditional methods by same teacher; difference not significant when compared to PBL
3. Responses for veterinary faculty in 1993 (excluding Rand); P < 0 001 when compared to responses for teacher using traditional methods in 1993.
4. Responses for entire university in 1995; P < 0 001 when compared with responses for PBL for same year
5. Responses for veterinary faculty in 1995 (excluding Rand); P < 0 001 when compared to responses for PBL for same year.