TEDI conferences: Dimensions of oral assessment and student approaches to learning

Dimensions of oral assessment and student approaches to learning
by
Gordon JOUGHIN
Queensland University of Technology and Griffith University

Introduction
Research into students' approaches to learning and teachers' everyday experience of assessment provide two propositions on which this paper is based:

that how students approach learning is determined to a significant extent by how they perceive the context of learning and what that context requires of them; and

that how students perceive the context of learning and what is expected of them is strongly influenced by how they perceive their assessment requirements.

There are four key questions that we might ask about oral assessment and student approaches to learning:

How can we describe oral forms of assessment? In particular, can we identify 'dimensions' of oral assessment that can help us come to grips with this form of assessment?

How do students perceive oral assessment?

Do students perceptions of oral assessment influence how they approach learning?

What dimensions of oral assessment seem to be particularly influential?

This paper addresses the first question directly, and considers the other three questions in terms of students' experience of oral assessment.

Can we identify 'dimensions' of oral assessment that can help us come to grips with this form of assessment?
I have described elsewhere six dimensions of oral assessment that were identified by analysing a large sample of examples of oral assessment reported in educational literature as well as by reviewing more general literature on oral assessment (Joughin in press a; in press b). These dimensions are as follows:
1. Content
In the oral assessment literature, there is surprising agreement regarding what can appropriately be tested by this oral assessment, viz

Knowledge and understanding. Here 'knowledge' is used in Bloom's sense to refer to the "recall of specifics and universals, the recall of methods and processes, or the recall of a pattern, structure or setting" (Bloom, 1956, p.201) while 'understanding' refers to the comprehension of the underlying meaning of what is known (OED).

Applied problem solving ability. The category of 'applied problem solving' has been variously described as "the ability to think on one's feet" (Muzzin & Hart, 1985, p.73), "the cognitive processes which constitute professional thinking" (Erhaut & Cole, 1993, p.10), "the ability to think quickly and diagnose problems in novel situations" (Habeshaw, Gibbs & Habeshaw, 1993, p.75), "clinical competence" (Solomon et al, 1990), and "problem-solving skills, application skills, interpretive skills" (Glowacki & Steele, 1992, p.13).

Interpersonal competence. 'Interpersonal competence' refers to a set of factors which Erhaut and Cole (1993, p.10) describe as "the personal skills ... required for a professional approach to the conduct of one's work". These typically include communication or interview skills, though it must be emphasized that it is not skills per se but rather skills exhibited in relation to a clinical situation or problem solving exercise that are the focus of assessment in the oral examination.

Personal qualities. Oral examinations, particularly those designed for certification purposes to test candidates' suitability to enter a particular profession, are sometimes to used to measure a candidate's intrapersonal qualities. Muzzin & Hart (1985, p.72) cite a number of personal attributes that oral examinations have been claimed to measure, including personality, alertness, reactions to stress, appearance, confidence and self-awareness.

2. Interaction
Interaction refers to reciprocity between examiner and candidate, with each acting on, responding to, and being influenced by the other. Most non-oral forms of assessment involve the student responding to a task which is presented at the beginning of the assessment process: the examiner sets the task, the student responds, and the response is then assessed by the examiner. In contrast to this, oral assessment creates the opportunity for a more complex set of interactions between examiner and student. It is this capacity that allows oral assessment to probe a student's understanding, widely recognized as one of the principal advantages of oral assessment (Brown & Knight, 1994, p.75).
3. Authenticity
'Authenticity' refers to the extent to which the assessment replicates the context of professional practice or 'real life'.
4. Structure
The dimension of 'structure' refers to the extent to which assessment is based on a pre-determined, organised body of questions or sequence of events.
5. Assessors
The 'assessor' dimension concerns who judges the worth of the student's responses. Unlike written forms of assessment, the 'product' of oral assessment is relatively public and is accessible to whoever is present at the time of the assessment. Thus oral assessment lends itself to assessment by multiple examiners, including faculty-based panels, or, if the assessment is held in a class setting, by peers. The possibility of self-assessment is probably present in most assessment formats, including oral assessment.
6. Orality
'Orality' as a dimension of oral assessment refers to the extent to which the assessment is conducted orally, ranging from the exclusively oral format of, for example, the clinical examination in medicine, to assessment in which the oral component is secondary to another component, for example, the oral presentation of a written paper or the oral explanation and defense of a physical product such as an architectural design.
Some of the six dimensions are continua. For example, the 'interaction' in an examination may be highly structured, relatively unstructured, or at any of number of points in between. A piece of assessment may be highly 'authentic' when it is based on an actual incident of professional practice or inauthentic when it is a more intellectual exercise such as the oral defence of a doctoral thesis. 'Structure' can vary considerably, and in the 'orality' dimension many combinations can be found between the purely 'oral' assessment and the assessment where the oral component forms only a small part of the exercise. The 'content' and 'assessor' dimensions are not continua but consist of more-or-less discrete categories.
The dimensions of oral assessment might be useful in several ways. For example:

They might assist or prompt teachers and researchers to provide fuller descriptions of oral assessment practices.

They may help to focus research into oral assessment.

They might suggest particular configurations of dimensions that may be expected to influence learning in particular ways. We might speculate, for example, that when students perceive that their understanding is being tested through a highly interactive process in relation to a relatively authentic task, they may have a greater tendency to adopt deep approaches to learning than would be the case in other circumstances.

Deep approaches to learning
Deep approaches to learning are characterised primarily by the intention to understand, in contrast to the intention to simply cope with course demands which characterizes surface approaches, or the intention to achieve high grades which characterizes strategic approaches (Entwistle 1997; Marton & Saljo 1997; Ramsden 1992). In fulfilling this intention to understand, students adopting deep approaches to learning seek to transform rather than merely to reproduce what they are learning and consequently adopt strategies that include

relating ideas to previous knowledge and experience;

looking for patterns and underlying principles;

checking evidence and relating it to conclusions;

exercising logic and argument cautiously and critically; and

becoming actively involved in course content (Entwistle 1997, p.19).

Is there any reason to believe that oral forms of assessment might encourage deep approaches to learning, that is, that the dimensions of oral assessment might be associated with the intentions and strategies of deep approaches to learning?

How students experience oral assessment?
A small qualitative study involving a total of eight students – four in a certificate level theology program and four in an undergraduate law program – has explored students' experiences of oral assessment and how they prepared for it. The theology students were studying a course in which assessment alternated between written papers marked by the program coordinators and oral presentations (referred to as 'practicums') to their peers. The law students had opted to complete a 'viva' before a panel of two lecturers as an alternative to a written paper. In both cases the students were able to comment on the experience of oral assessment in itself as well as in comparison to written forms of assessment.
The students were asked to describe what the viva or practicum was like for them, what they thought was expected of them as they prepared for it, and how they prepared for it. While the interviews generated information on a range of aspects of oral assessment, two dominant themes emerged: the students' intention to understand, and their perception of the personal nature of the assessment format, including an interesting sense of relationship to the spoken word.

On the intention to understand
Each student perceived a greater need to understand the material they were studying when preparing for oral rather than written forms of assessment. Seven of the students related this to an expectation that their understanding would be probed through questioning, whether by their peers (in theology) or by knowledgeable lecturers (in law), while the eighth student (law) referred to the need for knowledge to be better structured because the order in which questions would proceed could not be predicted.

I really think .... "Do I really understand what it is I've written and have I written it?" I mean I do that with a written assignment but I think there is more responsibility to really be clear about that when you know that somebody may in fact ask you a question on the spot. (Theology)
While all eight students saw a greater need to understand in the context of oral assessment, what is perhaps equally noteworthy is the degree of difference expressed by four of the students (three in theology, one in law). Not only was greater understanding required by the oral assessment – in written assessment they were prepared to express ideas which they knew they did not understand.

In the written work I would just put it down and think "I really don't understand this anyway but I'll just quote from the book and put it in." I wouldn't do that in (the oral assessment) because I'd be worried someone would ask me, "Well what do you mean by that" and "What do you think about it? Do you really agree with that?" (Theology)
....you think "I'll just read this book and I'll go there and do it like I do an assignment and just write some kind of bull, whether it relates to the topic or not". But (in the oral assessment) you know ... you're going to look a fool ... so you make sure you know what you are saying. (Law)

On the students' sense of personal involvement
Three of the law students and all of the theology students described the oral assessment as more personal than their written assessment.

In an exam you're just a number ... but the (viva's) personalized and you're in direct contact with the people who assess you. (Law)
Yes, you prepare yourself better because you've got the added stress of like you're in front of someone. So because you know you've got to do that you try to make sure. You're not just sitting in an exam room anonymously. (Law)
With a written assignment you submit that and you're just a number whereas when you're giving a presentation you're standing in front of ..other people. (Theology)
It's directly associated, well it's directly associated with you. (Theology)
Three students expressed this involvement in terms of the different relationships that they experience in relation to the words that they speak and write:
In a written assignment you can remain quite remote from what you write (Theology)
You've got to take ownership of the words that you speak. (Theology)

Conclusion
The studies reported in this paper are not definitive, but they are suggestive. In light of the 'dimensions' of oral assessment' and the responses of students in the short study, teachers might like to consider the following questions in relation to their own experience of using oral assessment:

Do the dimensions provide a satisfactory framework for describing your practice?

Are some dimensions more important than others in different contexts?

Do students prepare for oral assessment in ways that are different to how they prepare for other forms of assessment? If so, why?

Is the knowledge students express in oral assessment qualitatively different to that which they express in other forms of assessment? If so, why?

References
Bloom, BS (ed) (1956) Taxonomy of Educational Objectives: Cognitive Domain, New York, McKay.
Brown, S & Knight, P (1994), Assessing Learners in Higher Education, London, Kogan Page.
Entwistle, N (1997) 'Contrasting perspectives on learning' in The Experience of Learning, eds F Marton, D, Hounsell & N Entwistle, Edinburgh, Scottish Academic Press, pp. 3-22.
Erhaut, M & Cole, G (1993) 'Assessment of competence in higher level occupations', Competence and Assessment, 21, 10-14.
Glowacki, ML & Steele, DJ (1992) "A synthesis of the research on alternative assessment methods in teacher education', paper presented at the Annual Meeting of the Mid-South Educational Research Association, Knoxville, Tennessee, 11-13 November, 1992 (ED 355 257).
Habeshaw, S, Gibbs, G, & Habeshaw, T (1993) 53 Interesting Ways to Assess Your Students, Bristol, Technical and Educational Services.
Joughin, G (in press a) 'Dimensions of oral assessment', Assessment and Evaluation in Higher Education.
Joughin, G (in press b) 'Dimensions of oral assessment and student approaches to learning' in Assessment Matters in Higher Education, eds S. Brown & A. Glasner, Buckingham, Open University Press.
Marton & S(lj( (1997) 'Approaches to Learning' in The Experience of Learning, eds F Marton, D Hounsell & N Entwistle, Edinburgh, Scottish Academic Press, pp. 3-22.
Muzzin, LJ & Hart, L (1985) 'Oral examinations' in Neufeld, VR & Norman, GR (eds) Assessing Clinical Competence, New York, Springer, pp. 71-93. Ramsden, P (1992) Learning to Teach in Higher Education, London, Routledge.
Solomon, DJ, Reinhart, RC, Bridgham, RG, Munger, BS & Starnaman, S (1990) 'Free-response formats for evaluating clinical judgment', Academic Medicine, 65, 9, 543-544.

Conference Entry Page About the Conference Abstracts Conference Program

TEDI Home About Assessment UQ Home

This site built on a Macintosh using Dreamweaver.. Last modified 20/1/99; 10:14:22 AM.