Lowering Anxiety through Take Home Exams and Raising it through Videotaped Clinical Skills Assessment by Kathryn M Gow
Queensland University of Technology
Preamble
The paper commences with a background to the course, moves into a description of the nature of the flexible delivery and outlines the assessment processes. It then links the design of the teaching, learning and assessment in the program to certain aspects of learning theories and summarises some of the issues relating to flexible delivery of the program.
Background
The Queensland University of Technology commenced a radically different kind of full fee paying postgraduate course in 1998. Not only did the course content focus solely on hypnosis, but it was taught out of normal teaching hours in block mode and was supported by Internet driven materials and learning processes, as well as take home exams. The program was to be conducted with the assistance from the Queensland branch of the Australian Society of Hypnosis which would supply teaching expertise in the practical subjects.
The caveat on the program was that because of the QLD government's law restricting the practising of hypnosis to doctors, psychologists and dentists that only those health professionals could be admitted to the course. Thus all the trainees would be qualified practitioners who had registration or who were eligible for registration. Therefore most of the trainees had already completed between four and six years rigorous university academic training. Graduates of the ASH course who held a Diploma of Clinical Hypnosis were eligible to enter the second year of the QUT program, having been granted full credit for the first year by dint of their previous ASH study.
Before the Hypnosis diploma course was approved for commencement by the QUT Council, external comment was sought from clinical and academic experts in the field on all matters relating to teaching, learning and assessment. One of the major concerns was the advocacy that the assessment not take the form of the traditional examination processes, as the professional courses in this area had demonstrated that trainees stopped short of completing their professional diplomas because of fear of facing further examination processes. At a recent conference, Professor Kevin McConkey (1998) commented that examinations were a high source of anxiety for students generally. [Looking back, because nobody had attempted competency based assessment in a hypnosis course before, there was no mention of pressure arising from such an assessment.]
Thus as the designer of the course, I sought to find ways in which the academic standards of the program could be maintained, while at the same time providing assessment that would not raise the anxiety of the enrolled students too much. At this early stage, I had some concerns and a little anxiety about matching the needs of the students with the needs of the normal academic assessment processes, but it was not till mid-year that my anxiety levels were unexpectedly raised. But first an overview of the course structure and assessment details.
The QUT Hypnosis Course
From 1999, the program will be offered full time and thus students can complete the content and assessment for both years in just one year; however, in 1998, the program has been offered as a two-year part time course; the first year was practice based, while the second year was research based. Students were accepted into both years of the course in its first year because of the credit system available for ASH members who had completed the professional diploma. Fifteen students commenced after a late advertising campaign and across the year over 20 guest lecturers were employed to teach, in addition to the coordinator.
[The organising of, and the explanations to, so many different lectures in just one course and the pay arrangements for local, interstate and international visitors means extra work for the finance clerk as well as the coordinator. The range of guest lecturers means that the coordinator has to attend every session to check what is being taught and to ensure that no problems arise with the hypnotising of students, as this is always a possibility].
In first year, students attend a number of Saturday and weekend seminars, as well as competency assessment days (Saturdays or public holidays) and meet off-campus with two (in sequence) practice supervisors who have different specialist orientations. Prior experience with the ASH course indicated that this type of attendance was popular. Eighty percent attendance was set for the first year as one of the criteria for meeting the professional standards. For those students who could not attend for genuine reasons, video tapes of the lectures were made available. The video taping of the lectures was undertaken primarily to accumulate teaching material for future externally conducted programs, as well as being a study revision resource to be held in the Limited Access section of the library. These tapes have proven to be an important part of the resource material for current students, but have had to be handed to the students and the loans recorded by the coordinator, as most cannot access the library during the week.
In the second year, in order to complete the 12 credit point research unit, students were to meet for six wednesday nights only and complete the remainder of the course work through structured home study programs and to engage in Internet Discussion Groups (which they accessed through the QUT Hypnosis Home page). The idea behind minimal attendance was to suit the needs of busy practitioners, especially those who were travelling long distances to attend the Carseldine Campus. [Feedback from this cohort of students is that they prefer to drive 100 kilometres to attend weekly research and theory lectures, rather than do the home study program].
As the second year was research based, no set attendance was required. It was intended that students would meet with their supervisors fortnightly in first semester to design and structure the dissertation and to meet weekly in second semester to monitor the progress and completion of the dissertation which could lead to the award of a Research Masters if students chose to undertake a further 6 months full time, or 12 months part time, study.
There are 4 units to be completed in both years, although the 4 units in first year require a great deal more active participation in seminars than the second year. There are also more assessment items in the first year than the second, but there is a 15000 word dissertation worth 36 credit points in the second year.
Assessment Components
There are two pieces of assessment for each of the four units. Across all the four units, there are four take home exams, one literature review, two audio taped demonstrations of clinical sessions, and three full days of competency assessment. Only one of the units is graded as pass fail and the other three units are graded on a 1- 7 achievement scale. The pass/fail assessment (utilised in three of the assessments) initially engendered great anxiety in the students, and then later after they had completed the first assessment day and passed their first semester, it generated much less anxiety. As a bonus of formative assessment, they realised that if they failed the first time that they could re-attempt the assessment until they passed. [This pass-fail system also was a satisfactory arrangement for the coordinator who was then free to act more as a coach than an examiner.]
In contrast to this assessment, the previous professional program required students to complete one three hour exam (in formal mode), three written case studies and a 30 minute viva examination.
While the ASH program (leading to the professional award of Diploma in Clinical Hypnosis and admitting graduates to membership of the Australian Society of Hypnosis) was conducted over two years part time, it did not include a dissertation, nor was there a specific theory and research unit that had to be completed. The practical learning occurred over the two years in contrast to the one year concentration of the QUT program.
Anxiety and Performance
Considering the Yerkes-Dodson "U" shaped law (see Morris, 1991) in relation to performance and anxiety/arousal, it would be expected that there is a level of arousal/anxiety at which students would perform well and another level at which the over anxiety would lead to performing poorly. Taking note of the warnings from the external evaluators, it would seem that deleting formal time-limited exams would keep the students' anxiety levels at a reasonable level. Moreover being allowed to have nominated text books at hand while doing the exam and having the luxury of seven days in which to complete the exam ought also to lower the anxiety of those students who are prepared. In addition to these aids, the students were also advised to undertake the structured home study program, and that 50% of the exam questions would be taken from the home study questions. [Fortunately my hypothesis about the home study program being beneficial was held up, as all of the students who undertook the home study questions achieved substantially higher marks than the other students on the take home exam.]
Reverting back to the issue of maintaining the fine balance between the required academic rigour, and the lowering of students' anxiety, in relation to university level performance, the coordinator's anxiety levels rose considerably when the exam results were collated and all except one person in each case, achieved results of 80% and over on all three exams in first semester. In a discipline which prides itself on reinforcing the normal curve in examination returns, these positively skewed results were not exactly welcome. Even though I had changed the pass mark to 65, the credit to a 70, the distinction to a 90 and a high distinction to a 95, there was still an over preponderance of high grades. Fortunately the class size was small enough for the coordinator to escape penury in this matter. However, the seed of doubt was sufficient to raise the question of whether the take-home exam was too easy a process and inequitable across the university's programs.
However as a consequence of formative assessment, not content to receive high marks, each student asked to see the results of their exams to see where they had gone wrong. If we see education as stimulating learning, this form of take-home assessment, based on a textbook, certainly stimulated goal achievement in these mature age students. According to Sweller (1993), it may indicate that in the absence of excessive cognitive load, the students' problem solving capacities had been improved to the extent that they asked about how they could have done better. On the other hand, it may simply have been that when undertaking a subject which is inherently about learning at the unconscious level, having a consciously structured study and assessment process decreased the anxiety, because much of the students' time in the first semester was spent in trance states.
Bona fide exams. It seems remarkable that academic monitors worry that take-home exams might be fraudulently completed (that is, some one else writes the answers), but they do not have the same concerns about assignments. The disadvantage of cheating in such a subject area as hypnosis, is that you are very liable to be caught and no one else would know the subject matter, even if you asked them for help.
Learning through reflection and practice
The different approaches in the course and the upgrade programs served to make students learn and conceptualise in many different ways, both at a conscious and unconscious level.
The first year students were introduced to theory and practice concurrently. At the same time as they were learning how to induce hypnosis, they were studying what famous researchers in hypnosis had to say about what hypnosis was and who was hypnotisable under what conditions, etcetera. Before the students engaged in any practice, they were given verbal input about the techniques or process, the lecturer demonstrated the technique, they practised it and then in a plenary group they discussed their findings. Later as the semester progressed, they also learned to discuss the process with their "clients" as an introduction to EAT (The Experiential Analysis Technique) (Sheehan and McConkey, 1996). This technique is based on Kagan's (1975) process recall, in which the person's learning is facilitated by watching their performance in a video play back process and feeling safe enough to comment on how they were thinking, feeling and processing information at the time. Thus the students followed the circular process in Kolb's and Fry's (1975) model of reflective observation, concrete experience, abstract conceptualisation and active experimentation, or any variations of this.
On the other hand, the second year students who had been practising hypnosis for some years were able to reflect of their practice by the study of, and applications of theory and research to practice and commented that this made them think about what they were doing in the practice sessions and that it had sharpened their practice skills. Thus in Kolb's terms, they were able to reflect on their practice and theorise themselves about what was happening with their clients in their therapy rooms.
The second year was theory and research based and included no practice component, except that all the theory and research was related to real practice by the two lecturers.
Taken across the two years, one could demonstrate that a series of cognitive processes, such as those described by Haynes and van Dijk (1988) were needed for students to engage in the program. These were: identifying, organising, composing, analysing, judging critically and predicting. Not only were these skills required to complete the assessment of the program, they were essential skills to be transferred to the workplace, and in their case, generally a health practice.
While the majority of the students could be classified as adopting the learning strategy of reproducing and organising (see Ramsden, 1985), there was no evidence of their recognising a meaningful orientation in the first year - this was reinforced with a mainly practice based schedule; however this was almost forced on the students in the second year in their dissertation process when they had to read, reflect, conceptualise and link their concepts with practice and research.
If we think of the teaching and learning process from the perspective of a model of skilled action (Evans & Butler, 1992), while the students were able to tap into what a large number of experts had to offer, their coordinator was always there to thread the pearls of wisdom together and to act as the one consistent staff member in their flexibly delivered out-of-hours university study program. Evans and Butler (1992) speak about the necessity of task feedback and assisted reflection as part of training: this was built into this course through three full day feedback sessions, the supply of practice supervisors and through some items of assessment. The students learned also from sharing information and experiences with each other and by conducting a feedback session with their "clients".
While Fraser, Tobin and Kahle (1992) speak of the importance of understanding the effect of metaphors that lecturers use in their classes, the concept of metaphors was introduced in the first week of teaching, as part of a discussion about the links between "our" own metaphors and the effects on, and reactions to, hypnosis in "our" contact hours and there were substantial handouts given out in the lectures which formed the basis of the final exam in that unit.
In the second semester, having removed the set texts and focused on the lecture notes for the final exam in one unit, some of the students felt that they were directionless, without the set home study program and the textbooks to follow. Although they were being given up to 30 pages of notes at seminars to study, and were attending twice the number of sessions, they were worried that they were not learning as much. These were the students who successfully undertook the home study program and were therefore self starters and liked the feedback on their answers to questions. Reading the home study answers was very much like an enormous never ending set of assignments, but allowed me to interact and engage in a deeper dialogue with the students and this was a very satisfying aspect of the role.
THE PROBLEMS WITH FLEXIBLE DELIVERY
According to Wells (1989), we would be errant if we thought we were inventing any thing new here with flexible delivery and new technology. Wells reminds us that B.F. Skinner began experimenting with teaching machines and the concept of programmed learning in the 1940's. I certainly hope we have progressed since then in terms of understanding of teaching and technology. However we still have seem to have problems with new modes of delivery.
The issue of making IT skills an important part of a practice skills training course is interesting and not one that is self evident; a Hypnotist uses physical tools in which to conduct their trade. While they may utilised audio taping as part of the session for use by the client later as backup and occasionally use videotaping as a legal safeguard, they by and large do not use computers, or surf the Internet while the client is in the room; however, all professionals now are required to be IT literate if they are to compete in this technology driven economy.
Videotaping of seminars.
In this course, videotaping of the seminars meant that all participants had to give written permission to be videotaped throughout the year, as they would be seen on screen practising hypnosis on others and being hypnotised individually, or in a group. When one person refused to be on video, this meant that certain callisthenics had to undertaken by the camera person in order to avoid taking frames of this person. Videotaping also raised the anxiety level of most of the presenters, many of whom had never been in front of a video camera before for teaching purposes. However those, who requested it, received a copy of their videotaped session and appreciated the chance to study their presentation and skill demonstrations on tape.Monitoring the state of the camera persons was also required, as some of the sessions involved some "difficult" topics and the camera persons on occasions were seen to go into a trance state [no danger to them, but it did interfere with the recording in one case].
Once the camera rolled, it restricted the access of the coordinator to the students and restricted the interaction of the students as well. From the students' points of view, they felt that the camera was an interloper and only those who missed sessions and had to borrow videotapes to catch up on the missed seminars realised the full benefit of the videotaping. This appreciation grew across the year, but still did not make the actual learning setting any more comfortable because of the restriction of movement and sounds). Students still asked questions, but this was observed to be less than normal. Students would not use the microphone to ensure their voices were heard on the videotapes, so the coordinator worried about the consequent extensive editing required in terms of cost of time and money, if we were to produce really top quality tapes for distance education purposes.
Videotaping of competency-based assessment sessions
While there has been some debate about competency based training in Australia (Stevenson, 1992), it was evident to the designer of this flexibly delivered hypnosis training program that up till now, no training body in Australia had determined and then subsequently assessed the competencies that hypnotist needed to do their work effectively."Competency amount to a demonstration, through performance in a workplace, of an individuals' underlying personal competence" (National Training Board, 1991, p. 36), as we already had competent professionals coming into the program. In this course, we were after engendering competence defined by Kinsman (1994) as "an individual's capacity for "adequate' performance in a particular occupation" (p. 40), in this particular case, the practice of hypnosis. Thus the students in two units were rated as pass/fail. This did not in fact lower their skill output, as if they did not come up to standard, they had to repeat the assessment until they did. With practical skills, you can either do these or not, and the polish comes later after much practise and further study by attendance at workshops, or by reading, or watching videos, or listening to tapes to improve one's expertise.
Rather than just give a pass/fail tick on the competencies listed each time for assessment, Ivey's levels of competence were used (1 = recognition of skills, 2 = basic mastery, 3 = active mastery and 4 = teaching mastery). While the students had to obtain a level 2 on a 3 level scale, they were passed if they obtained 2, but were given feedback as to how to obtain a 3. In semester two, the level 4 was dropped from the competencies check list.
As none of the competencies had been delineated previously, the coordinator had to devise these as the course unfolded, an interesting task leading to a further research project in the area.
A complicated process allowed the coordinator to set up five rooms complete with video cameras and the students, armed with the list of competencies to be demonstrated on video, formed pairs for each segment of the assessment sessions. On the first assessment day, I was able to give them feedback by moving between the rooms and giving immediate or delayed feedback. Those students, whose sessions I had not viewed, then left me with the video recordings of their other sessions which I reviewed during the week. However during the first session there were only 10 competencies to be demonstrated; but by assessment day two, in second semester, there were over 30 competencies to be demonstrated and this could only be reviewed by marking the videotapes at home. Suddenly I did not have the extra 36 hours it would take to view and comment on these tapes; and at the same time I had 10 hours of audio-taped hypnosis sessions to listen to and make comments on as well. Nevertheless, I must admit that listening to the tapes was very relaxing!
Creation of Web pages
The use of the Internet meant that web pages had to be designed and content supplied. This was an another exciting aspect of the course, and it took several weeks of time in preparation, as all the materials, including the home study programs had to be prepared for Internet use, and an access bar had to be created for non-enrolled students. At times I wondered if all the effort, which was over and above an already very heavy workload, was worth it. Indeed, if it had not been for the feedback from overseas access points, and the fact that I knew that the students had seen the web pages, I would not have persisted in developing further materials for the web.Internet Discussion Groups
The Internet discussion groups had been trialed in the previous year and some of the problems of external servers addressed. The most disappointing outcome was that only a few of the students joined the discussion groups, with the avoiders giving reasons such as not having Internet access, or being too busy, to do so. By second semester, only three people were joining the discussion groups. Previous pilots with over 1500 students in our social science classes indicated that students will not become involved in discussion groups unless that involvement is a part of the assessment. So the answer is to make the use of the Internet compulsory in some way for assessment purposes. However in flexible delivery courses such as this, it was not equitable or possible to make this leaning activity compulsory because several students could not readily access the Internet off campus.Internet Chat Groups
Having been very inventive in proposing that some practice supervision sessions be conducted by Internet under conditions of restricted access, the coordinator was very disappointed to find after nearly a year of pilots, that the university system could not cope with access by external private servers. This caused more disappointment than anxiety, but added to the question of the efficacy of flexible delivery and the uncounted extra workload on the coordinator.Internet and Library systems training
Two training sessions were conducted for first and second year students, one in the special Orientation session and another at the beginning of second semester. These sessions demonstrated initially the use of the Internet for e-mail, discussions and chat group purposes and secondly as a reinforcer for accessing library systems through the Internet for research purposes. In addition to this, several students contacted the Library liaison person who was an expert in this area for assistance with locating articles and data bases.Mail outs galore
Because the students were not regularly attending the campus as most students do, it meant that all communications had to be written down and mailed to the students. E-mail distribution was prevented because not all students had access to e-mail and of those who did half of them experienced difficulties with their providers and had to change addresses etcetera. So an additional work load fell to the coordinator for a course that was not considered to be a distance education program.Administrative Guru
Being the only person whom the students had face to face contact with, meant that the coordinator was considered to be able to solve all the enrolments problems and administrative/library inquiries.Access to Services
Being on campus on Saturdays and Sundays meant that library access was restricted; on Saturdays there was only 30 minutes of the lunch break when students could access library materials. The Bookshop was never open nor the canteen. Rooms were locked and photocopy machines were turned off. The coordinator had to arrange for tea, coffee, biscuits, and milk. juice, and cups etc each time and had to clean up personally at the end of each day. As the Assistant Dean of the Faculty, I found it incongruous that I had to spend an hour at the photocopy machine, wash up, clean the benches after the students and empty the refrigerator. Likewise with security, the coordinator had to arrange for master keys to be left with her on Friday evenings and to be returned on the Monday morning, as external doors had to be kept closed. Each time that the taping of practice sessions was scheduled, the five videorecording cameras and rooms had to be set up with microphones and arranged in a suitable manner. At the end of the day, everything had to be packed away and tided up. These were 11 hour days - very flexible!Flexible Deadlines
If a course is to be really flexible, then the assessment deadlines have to be flexible as well; changing the deadlines for students in their favour obviously lowers the anxiety of the students, but often raises the anxiety of the coordinator who then has to make sure that all the varying pieces of assessment are completed and recorded at times outside of the normal marking time. It also means that students miss their normal results notice and even miss their graduation ceremony. [Metaphors that come to mind that explain the cognitive reactions and feelings that go with odd pieces of assessment being delivered in varied forms and then at staggered times are: "like Brown's cows" or "like a can of live worms - all over the place"].Flexible enrolment
It is always difficult explaining that mid-year intakes cannot work with such a program, as every unit would have to be offered every semester and there is no economic advantage in that. However, the Graduate Diploma is set up in such a way that students can enrol in either year 1 or year 2 and exit with a graduate certificate in practice or research and even take longer than two years to complete it, if necessary.Pioneers as unsupported expendable Resources?
Dr Michael Arnold (1998) recently queried whether enough support (time and resources) was being given by universities to lecturers who were developing their courses within an innovative techno-framework. I think that all of us know the answer to that question!John Hird quoted in Open Training (1998, p.1) asks "will multimedia applications lead to replacement of teachers or to enhancement /enlargement of their roles?". I can answer that, and the answer is yes to both aspects of the questions. That happens to suit me, as I like to move on to new challenges, after the role has been enlarged by the application of technology created by myself.
Conclusion
Probably those of us involved in flexible delivery and Internet-based programs would not turn back the clock to the days when such enhanced delivery of courses was not possible. Anxiety is probably inherent in Internet both for students and staff; however I'd like to finish with the quotation: "Man cannot explore new horizons, unless he consents to lose sight of the shore" which I think sums up the dilemma of staying put or moving forward - with flexibly delivery in all its forms.
References
Arnold, M. (1998). Using high-technology in support of traditional pedagogy. Seminar presented at the Queensland University of Technology, Brisbane, 23rd September.
Evans, G. & Butler, J. (1992). Expert models and feedback processes in developing competence in industrial trade areas. Australian Journal of TAFE Research and Development, 8 (1) , 13-32.
Fraser, B.J., Tobin, K. & Kahle, J. B. (1992). Factors which militate against learning science with understanding. The Australian Science Teachers Journal,38 (3), 63-66.
Haynes, A. & van Dijk, D. (1988). Framework for planning through concepts and processes. Unicorn, 14 (3), 142 -146.
Ivey, A. (1983). International Interviewing and counselling. Monterey, California: Brooks Cole.
Kagan, N. (1975). Influencing human interaction. East Lansing, Michigan: Instructional Media Centre, Michigan State University.
Kinsman, M. (1994). Conceptual analysis of competency - based training in Australia. Business Council Bulletin, June, 40-44
Kolb, D.A. & Fry, R. (1975). Towards an applied theory of experiential learning, In C.L. Cooper (Ed.). Theories of group process (pp. 33-58). London: Wiley.
Morris, C.G. (1991). Understanding psychology. New Jersey: Prentice Hall.
National Training Board (1991). National competency standards, policy and guidelines, National Capital Printing,
Open Training. (July, 1998) "Flexible delivery like beauty is in the eye of the beholder", 16, 1-2.
Race, P. (1998). Changing assessment to improve learning. Seminar presented at the Queensland University of Technology, Brisbane, 24th August.
Ramsden, P. (1985). Student learning research: Retrospect and prospect. Higher Education Research and Development, 4, 1, 51-69.
Sheehan, P.W. & McConkey, K.M. (1996). Hypnosis and Experience: The exploration of phenomena and process. New York: Brunner Mazel.
Stevenson, J. (1992). Competency-based training in Australia: an analysis of assumptions. Australian Journal of TAFE Research and Development, 8 (1), 87-104.
Sweller, J. (1993). Some cognitive processes and their consequences for the organisation and presentation of information. Australian Journal of Psychology, 45 (1), 1-8.
Dr Kathryn Gow
PO Box 268, RED HILL
Brisbane, Queensland, Australia 4059
Phone: (617) 3864 4525
Fax: (617) 3864 4711
Conference Entry Page About the Conference Abstracts Conference Program
TEDI Home About Assessment UQ Home
This site built on a Macintosh using Dreamweaver.. Last modified 20/1/99; 10:14:18 AM.