This paper presents a case study of a professional development model of
student support as an illustration of the role of multi-faceted, team-based
reflective practice in devising solutions to an educational problem.
The problem was a low and falling pass-rate among students enrolled in
Accounting Decisions and Accountability, a large (1600 students) first-year
core subject. Pass rates had dropped steadily from 78% in semester 2, 1994
to 52% in semester 1, 1998. Early in Semester 2 1998, the subject coordinator
recognised that he was faced with a problem that was not solvable within
the realms of his professional knowledge.
A multi-disciplinary team was formed to design and plan the implementation
of a student centred learning environment to address the problem of the
falling pass rate. All members were challenged to make explicit the tacit
knowing that underpins their professional practice in order to communicate
it to others and solve the problem.
This paper discusses the stages of this model of multi-faceted reflective
practice in action, the resultant modifications to teaching practice and
their effects on outcomes for graduates.
Introduction
A professional development model of student support has been used successfully
at the University of South Australia (UniSA) for the past two years. It was
developed as part of the implementation of seven graduate qualities and as
such was a strategic response to a range of issues within the University and
the higher education sector (George and Hicks 1997). The approach involves
staff from a number of different areas of the University bringing together
their particular expertise to address specific student learning support issues
within a given subject. This paper presents a case study of the model as an
example of reflective practice (Schön 1997).
Multi-faceted reflective practice
(Schön (1987, pp5-6) describes reflective practice as a strategy for
dealing with a 'problematic situation' - one which 'presents itself as a unique
case', which cannot be solved by applying any one rule of professional knowledge.
The case must therefore be solved 'by a kind of improvisation' in which a
range of strategies are invented and tested until the right strategies are
found. However, before practitioners can even begin the search for the right
set of strategies to solve the problem, they must 'choose among conflicting
appreciations of a situation so as to construct a coherent problem worth solving'.
The problematic situation at UniSA was a low and falling pass-rate among
students enrolled in a first-year core subject. Approximately 1600 students
enrol in this subject each year. Pass rates dropped steadily from 78% in semester
2, 1994 to 52% in semester 1, 1998. During this time the content, assessment
and teaching methods had not altered significantly. Early in Semester 2 1998,
the subject coordinator recognised that there was a problem that was, in (Schön's
terms, not solvable within the realms of his or the subject team's professional
knowledge. Discussions with the Divisional Director of Teaching and Learning
resulted in the professional development model of student support being implemented.
This model had been developed in 1997 in another first-year subject in the
Division, Economic Environment, and is presented here as a six-stage
process of multi-faceted reflective practice.
The Case Study
Stage 1: Formation of the Team
The first stage in this process involved the formation of a multi-disciplinary
team to consider the problem. The team consisted of two study advisers, a
staff developer, the Divisional Director and Deputy Director of Teaching and
Learning, the subject coordinator and a subject lecturer. The team met together
every two weeks for a semester. In this time the members of the team worked
collaboratively to design and plan the implementation of a student-support
centred learning environment.
Stage 2: Constructing the problem coherently and developing a respect for
the artistry of other professionals
Schön (1996) identifies the need for competent practitioners to reconcile,
integrate or choose among conflicting appreciations of a situation in order
to construct a coherent problem worth solving (p6). He also says that often
situations are problematic in several ways at once, making the construction
of a coherent problem quite difficult.
Constructing a problem coherently in a group consisting of professionals
from diverse backgrounds can be difficult. Discussion during the first few
meetings of the group was largely focussed on identifying and exploring, in
some detail, many possible causes of the problem. All team members had different
views on whether the factors were internal and subject specific (i.e. related
to subject design, or teaching and assessment practices) or external and more
generalised (i.e. related to the students and their prior learning experiences,
university policy etc). As nothing within the subject content, organisation
or assessment had changed significantly in the period in which the pass rates
had dropped so dramatically it proved very difficult to move away from a discussion
of the external factors that might be causing the problem.
Possible causes explored included the increasing proportion of students
proceeding to University education, transition issues in moving from secondary
to tertiary education, changes in the South Australian Certificate in Education
and lower Tertiary Entrance Scores.
Most of these possible causes were beyond the control of anyone in the group,
and there were times when this resulted in feelings of intense frustration.
However, the discussion eventually led to a better understanding and definition
of the problem. Clearly there had been some significant changes in the nature
of the student population. The conclusion drawn was that it was likely that
the falling pass rate was caused by a mis-match between the structure and
organisation of the subject and the learning needs of the majority of the
student body. Thus the solution lay in restructuring and reorganising the
subject.
An important aspect of this second stage was the composition of the team.
The team was multi-faceted. It brought together competent professionals from
several different backgrounds (study advisers with a background in student
learning; a staff developer and a Director in Teaching and Learning with a
background in teaching and learning; subject specialists from the discipline
of Accounting) each of whom had a different perspective on the problem. Schön
identifies a 'special kind of knowing' inherent in professional practice.
Each person in the team brought to the problem their own special kind of knowing.
As the team was recognising, exploring and reconciling conflicting appreciations
of the situation (Schön, p6) individual team members were coming to know
each other better, to listen to and to trust each other and to develop mutual
respect for each other's artistry. A working professional relationship, the
foundation for solving the problem, was being established.
Now that the problem had been constructed coherently and the team was operating
effectively and efficiently, the process was ready to move into the third
stage - developing strategies to solve the problem.
Stage 3: Developing strategies for dealing with the problem
This relationship enabled the team, in further meetings, to move on, and
to work together to develop a range of strategies to solve the problem.
In order to achieve a better 'match' between the teaching and learning arrangements
and the needs of the students, the team decided to review the objectives of
the subject, the assessment tasks, their weighting and timing, and the teaching
and learning arrangements. Meetings were sharply focussed on the development
of strategies which would:
- better communicate the expectations of the subject to students
- more actively engage students with the content of the subjec
- assist more students to meet the course objectives
- make the thinking processes (the 'special kind of knowing') of the discipline
of accounting more explicit
- teach students to think more like accountants.
Significantly, these features characterise a professional approach to teaching
(Clark and Peterson, 1986; Schön, 1987). Schön says that all professional
practice is best learned by doing and that skillful practitioners conduct
'reflective conversations' in non-routine situations. Thus they are able to
detect consequences and implications. In the process of developing strategies
for dealing with the educational problem a range of 'reflective conversations'
were conducted within the team, during which solutions to the problem were
suggested and evaluated by all members. This multi-faceted reflective practice
ensured that the consequences and implications of any change were explored
in considerable detail from the different perspectives of team members. Thus
team members learned from each other and were empowered to see the problem
and possible solutions from multiple perspectives.
As a result of the 'reflective conversations' twenty-six changes were made
to the subject. All changes were carefully considered and agreed to by the
team. All were designed to improve the quality of the learning experience
by engaging students with the subject matter of accounting, increasing the
amount of time they spent 'on task' and encouraging a deep approach to learning
(Gibbs, 1992: Ramsden, 1992). All took place within the framework of the Graduate
Qualities, a central component of the University's Teaching and Learning Framework.
Graduate Qualities - informing planning and decision-making
The Graduate Qualities were introduced to UniSA in 1996, after a twelve-month
period of consultation with stakeholders. They indicated a significant shift
in the way the university defined 'quality' - a shift from an emphasis on
inputs (such as funding, academic staff qualifications and student
entry scores) to an emphasis on educational outputs. They also indicated
a significant shift in what we valued about the education we provide at UniSA.
The Graduate Qualities focus is very clearly on the student and what
the graduating student will have achieved during a course of study.
They provide a focus for curriculum development and course evaluation in all
undergraduate courses as well as for quality assurance and improvement processes
and by their very nature they focus these processes on students. In the subject
redevelopment process described here they provided a student-centred framework
for decision-making about subject objectives, teaching and learning arrangements
and assessment as shown in Figure 1. They assisted a diverse group of professionals
to make significant decisions and to respect each other's professional 'artistry'.
In meetings for Accounting Decisions and Accountability (ADA) the
team reviewed the Graduate Quality profile of the subject using a grid (see
Table 1). At the top of the grid are listed the seven Graduate Qualities of
a graduate of UniSA. The grid enabled the subject coordinator to map the objectives,
the teaching and learning arrangements and the assessment tasks onto the Graduate
Quality profile and to review the alignment between them. As a result of this
process the Graduate Quality profile of the subject was modified slightly
and therefore the changes that needed to be made to the objectives of the
subject, to the teaching and learning arrangements and to the assessment became
much clearer. The Graduate Qualities assisted the decision-making processes
of the group, by providing a framework that demanded that the objectives,
the teaching and learning arrangements and the assessment arrangements be
aligned. It also enabled the team to reach agreement relatively easily because
decisions were always made with reference to a shared and agreed framework.
Figure 1: A model of Student Centred Learning
Table 1: Graduate Qualities Grid - Accounting, Decisions and Accountability
The seven graduate qualities for the University of South Australia are:
(Subjective Objectives 3 to 7 have been omitted from this table)
Stage 4: Implementing the strategies
The changes were implemented in the semester following the development process,
although some were 'trialed' in the semester during which the meetings were
taking place. The most significant strategies included:
- Introduction of unsupervised workshops for students to attend immediately
prior to their tutorials. During this time students work in predesignated
groups to solve a continuing case study that links all sections of the subject
together and places them in a real world context.
- The team recommended that the workshops be followed with 'staffed' tutorials.
In these sessions tutors deal first with the tutorial questions that students
have found most difficult rather than focussing on going through the answers
to all the tutorial questions. This strategy was designed to make the tutorial
student driven, rather than question order driven.
- An ADA web site was developed and used to encourage students to
engage with the subject on a regular basis. Answers to the previous week's
tutorial and workshop were made available on the site for one week only
so that tutorials could deal with more difficult questions in depth. If
students did not check the answers each week they could not access them.
- Partial answers to tutorial questions were provided to students at the
beginning of the semester so that they would be more reflective in their
tutorial preparation and attempt to solve problems for themselves.
- In previous years students had been assessed for tutorial attendance
and participation. This was modified and workshop and tutorial preparation
are now marked for completeness, but not accuracy. Students check the accuracy
of their answers during tutorial discussions and by referring to the web
site the following week.
- A Help Desk was introduced to enable students to ask any question on
any topic and to provide a problem sharing forum for students with a tutor
present to facilitate the process and to direct the solution to the difficulties
raised.
- A weekly suggested activities page was provided in the Study Guide for
ADA to illustrate to students the amount of time they should spend
studying the subject and the activities they should engage in to succeed
in their studies. A detailed marksheet was also provided for the major assignment
to show students how a professional report should be presented and to make
the lecturer's expectations explicit.
- Hint Boxes were introduced for tutorial and assignment questions. These
are designed to make the expectations of the lecturer explicit and to encourage
students to reflect on the question before attempting to answer them.
- The final lecture of the semester concentrated on exam techniques specific
to the type of questions asked in the ADA exam. It was based on feedback
from the previous year's students about the exam. It also highlighted the
links between the Graduate Qualities, the subject objectives, the teaching
and learning arrangements and the assessment arrangements.
The strategies implemented have not been outlined in detail, as they are
not the main focus of this paper. More detailed explanations of the strategies
can be found in Feast, Kokkin, Medlin and Frangiosa (1999) and Leask, Medlin
and Feast (1999).
Stage 5: Evaluating the effectiveness of the strategies
According to a tutor focus group the most significant changes appear to
be the introduction of the workshops and the changes to the way tutorials
are assessed. Working in groups has provided both peer support and peer pressure
to complete the case study and has highlighted to students what they don't
know about each topic. Tutors report that when groups can't solve a case study
problem they will usually ask the tutor as soon as they enter the class room
and this has provided a focus for the tutorial. Tutors also report being asked
more 'why?' type questions, rather than the 'how to?' questions they have
been asked in the past. It seems that once a group of students have collectively
mastered how to answer a question in a workshop they move to another stage.
They start to ask why certain procedures are carried out in accounting. This
reflects a deeper level of learning than has been evident in the majority
of students studying the subject in first year in the past. The other frequent
comment from tutors is that having the tutorial immediately after the workshop
seems to 'warm the students up for discussion' as they have spent the previous
50 minutes talking within their groups. This has led to better tutorial participation
and greater engagement with the subject.
The change in the way tutorials were assessed had a surprising effect. Paradoxically,
the move from assessing attendance and participation to marking preparation
has led to improved levels of attendance and participation. Perhaps in prior
years, when no marks were awarded for tutorial preparation, students who didn't
do the preparation didn't make the effort to attend the class for fear of
showing their 'ignorance'. Those students who in the past did prepare and
attend did not participate because the activity in the tutorial focussed on
the how (which they were not interested in as they had mastered that
aspect), not the why, (which is what they were interested in). Now
that preparation is assessed students are coming to the tutorials to see how
well they have done and, furthermore, they are ready to engage with the subject
on a deeper level.
The reaction to the web site has been positive with many students commenting
that the fact that the tutorial answers are only available for one week following
the tutorial has encouraged them to download the answers each week. Often,
having down loaded the answers, and because the tutorial is still relatively
fresh in their minds, many students report having gone back over the tutorial
questions they found difficult. This means that students are looking at a
topic three weeks in a row. Firstly in the lecture, the following week in
workshops and tutorials and then in the third week when they review the answers
from the web site.
Assessment results for the first semester the changes were introduced
The academic staff are now well placed to measure possible outcomes that
have flowed from the multi-disciplinary team approach to staff training and
subject review.
For analysis purposes, the assessment tasks for ADA have been divided
into supervised (executed under closed book conditions in a limited time examination
environment) and unsupervised (attempted at the student's own time and at
their discretion using whichever resources the student sees fit). Traditionally,
supervised tasks have been seen by educational institutions and authorities
as giving a valid assessment of retained (as opposed to transitory) knowledge
and understanding Gagn (1985).
The changes made in ADA for 1999 seem to have made little difference to
unsupervised continuous assessment scores (Graph 1). However, the strategies
implemented as part of the review have been accompanied by a marked improvement
in performance in the tasks that are held under closed book supervised conditions.
Graph 1: Mean percentage scores for various components of assessment
(all students)
* A tutorial preparation / participation mark was not a part
of the assessment package until 1996
Graph 2 outlines the enormous improvement in the performance of the population
in 1999, giving the subject a record overall pass rate.
Graph 2: Percentage of students achieving a pass grade (P2 or better)
Graph 3, along with graph 2, illustrates how the results of both accounting
majors and non-majors have not only improved relative to their own groups'
previous performances, but they reveal another significant effect of the new
strategies, namely the reduction in the performance gap between the two groups.
Graph 3: The means of supervised task performance grouped by major
The reduction in the relative performance gap between accounting majors
and non-majors is an encouraging result particularly given the proportion
of non-accounting students enrolled in ADA has increased from 67 percent to
80 percent of the subject's enrolment. The results suggest that the graduate
qualities approach implemented by the multi-disciplinary team is having the
effect of changing the perception that success in the discipline of accounting
is basically the domain of those who are training to be accountants.
Stage 6: Reviewing, reflecting and modifying
There is a sense in which it is impossible to separate this stage from the
previous one for it evolves naturally from it. With the evaluation of the
changes completed and analysed it has been necessary to reflect on the effectiveness
of individual changes and strategies as well as their effectiveness as a whole.
As a result of further modifications and adjustments to the teaching, learning
and assessment arrangements have been made, particularly in providing students
with continuous feedback on their tutorial preparation and more timely and
detailed feedback on the test and assignment.
The success of this model for staff development and subject evaluation has
led to the use of multi-disciplinary teams in other subjects in the Division
of Business and Enterprise at UniSA. These groups have included some members
of the ADA team as well as academics from the particular discipline areas
being reviewed. The success with ADA has also led to subject coordinators
requesting the use of a similar multi-disciplinary team to review their subjects.
Importantly, the academics that have been involved in these teams are now
being asked by other staff within their Schools for advice on how to improve
the teaching and learning environment for the subjects they coordinate. The
multi-disciplinary teams approach to staff development has empowered the academic
staff to reflect on their practice far more effectively than the traditional
workshop method of staff development.
Implications for staff development
Subject development is traditionally seen as the sole responsibility of
the academics who teach the subject. Professional development and student
support are viewed as separate, although possibly related. The process described
here challenges that view, by integrating professional development, subject
development and student support.
The strategies that were developed are broad in scope and significant in
nature, yet they were conceived and implemented within six months. Given the
nature of the group involved in the redevelopment process perhaps the scope
of the changes is to be expected. However, what was surprising, given the
diversity of the group, was the relative ease and speed with which agreement
was reached regarding changes of such magnitude. This was largely due to the
framework provided by the Graduate Qualities.
The success of the approach was also due in large part to the diversity
of the team. It brought together people who would not normally work closely;
it gave them a common purpose and the opportunity to see a problem from a
range of different perspectives. The result was the rapid emergence of a much
broader and richer range of strategies to deal with the problem than would
otherwise have been possible.
Early in the process of review and redevelopment described above it became
clear that the discipline-based academic staff were particularly interested
in the perspective provided by those staff who had worked with students needing
assistance with various aspects of the subject ADA. These 'Learning Advisers'
were able to provide particular insights into the teaching and learning and
assessment arrangements because of their close contact with students. It was
also clear that the Graduate Qualities framework required some interpretation
and translation. This was the principal role of the 'Staff Developer' within
the team. Although there was some overlap between these roles, it was important
that both were strongly represented.
Traditionally, staff development in higher education is offered primarily
by staff developers through workshops on professional topics or issues. This
is changing at UniSA. It is now much more common to find 'Learning Advisers'
and 'Staff Developers' working intensively with teams of academics within
the framework of the Graduate Qualities as described in this case study. The
main reason for this move is that the demand for what is described here -
multi-faceted reflective practice - is growing rapidly. Furthermore, it is
becoming clear that this approach has a value beyond the development of individual
staff members. It is assisting the organisation to achieve student-centred
goals at the same time as providing opportunities for the professional development
of staff from a number of different professional backgrounds and has been
shown to improve outcomes for graduates.
Conclusion
The success of multi-faceted reflective practice in modifying teaching practices
and improving outcomes for graduates described in this paper is dependant
on a number of 'human' factors. These are the diversity of the team involved,
their individual and collective commitment to achieving improved outcomes
for graduates, their ability to work together as a team and their willingness
to trust each other and learn from each other - to appreciate and respect
each others 'artistry'. However, its success is also due in large part to
the existence of a shared framework within which professional staff from a
range of backgrounds can make informed decisions about teaching and learning.
Decisions arrived at using this process have the potential to achieve much
more than a solution to an immediate problem. They are a method by which we
can achieve long-term educational change and a rapid improvement in the quality
of teaching and learning in large and complex institutions. For the knowledge
and skills that each person gains from the 'reflective conversation' within
the team, are taken with them into their next educational encounter.
Bibliography
Bell, J. and Harrison, B.T. 1998. Leading People: Learning from people
lessons from education professionals. OUP: Buckingham.
Boyle, P. and Trevitt, C. 1997. 'Enhancing the quality of student learning
through the use of Subject Learning Plans', Higher Education Research and
Development, Vol. 16, No. 3, 1997. p293-308.
Cherryholmes, C.H. 1988. Power and Criticism - Poststructural Investigations
in Education. Teacher's College Press: New York.
Dow, G. (ed) 1982. Teacher Learning. Routledge and Kegan Paul Ltd:
London.
Feast, F., Kokkinn, B., Medlin, J. and Frangiosa, R. 1999 `Accounting for
student diversity'. HERDSA papers.
George, R. and Hicks, M. 1998. 'A strategic perspective on approaches to
student learning support at the University of South Australia'. HERDSA
papers.
Gibbs, G. 1992. Improving the Quality of Student Learning. Oxford
Centre for Staff Development: Oxford.
Gagné, R. M. 1985. The conditions of learning and theory of instruction,
4th Ed. New York: Holt, Rinehart & Winston.
Johnston, S. 1997. 'Educational Development Units: Aiming for a balanced
approach to supporting teaching'. Higher Education Research and Development,
Vol. 16, No. 3, 1997. p331-342.
Leask, B., Medlin, J. and Feast, V. 1999 ` Improving outcomes for graduates
through multi-faceted reflective practice in staff development'. HERDSA
papers.
Ramsden, P. 1992. Learning to Teach in Higher Education. Routledge:
London.
Schön, D. A. 1987 Educating the Reflective Practitioner. Jossey
Bass Publishers: California.
Smyth, W.J. 1986. Reflection-in-Action. Deakin University: Victoria.
Webb, G. 1996. Understanding Staff Development. Society for Research
into Higher Education & Open University Press: Buckingham.
Wittrock, M.C. (ed) 1986. Handbook of Research on Teaching. Third
Edition. Macmillan Publishing Company: New York.