Designing an assessment program
What is an assessment program?
An assessment program is a plan of how you are going to find out how well students have achieved the learning goals of your course and how you are going to provide feedback to students to help them to achieve these goals. The program may be as minimal as a timetable listing when and how students are to be assessed, or it may consist of a detailed portfolio, including the actual texts of assessment items; analyses of the ways in which goals are tested by these items; categorisation of the items as strictly for feedback, strictly for grading (not recommended), or an amalgam of feedback and grading; feedback pro formas; and other items. For further discussion of the nature of assessment programs and of the constraints on them generally and at UQ read the PDF document titled Assessment programs: Characteristics and constraints.
There is no simple recipe for designing an assessment program. Even if you restrict yourself only to a program aimed at giving you enough information to assign valid grades to students' achievement, you will still find the task a complex balancing act. You will be trying to balance what might be desirable were you and the students not human and fallible against what is realistic given various constraints.
In these pages
we will try to provide tools and ideas to help in this planning and
balancing process. The assessment portfolio is
a key tool
in documenting your program.
The assessment portfolio template
Download an assessment
portfolio template. This template contains a set of tables and other documents
which you may complete in order
to
compile
a useful record of your assessment program. A fairly complete
portfolio would include the tables as well as the actual texts of the
assessment
tasks, course level and task level assessment criteria, marking
schemes, and a means of assigning grades to students' work in
the course.
This portfolio is a private document which you will use to plan
and document
your assessment program for a course. Unlike the Course profile
it is not a published public document.
The assessment portfolio
A completed assessment portfolio will contain:
- a draft assessment program
- the learning goals for the course
- assessment criteria for these learning goals
- a set of assessment tasks which make up the substance of the assessment program
- for each assessment task, details of the
criteria which will be used in assessing students' attempts
at completing
the task
for each assessment task, details of the marking scheme which will be used in assigning grades to students' attempts at completing the task and/or giving students feedback on their attempts - a timetable showing week by week what assessment tasks are to be attempted by students
- a table showing the relationship between the assessment tasks and the course learning goals
- a table showing the relationship between the course level assessment criteria and the criteria used for the individual assessment tasks
- a scheme for translating a student's results on the various assessment tasks into a grade for the course
- a final assessment program.
Completing the assessment portfolio for a course you've already taught
You will need to use the template to create a new assessment portfolio document for each course you plan to work on. If you have already taught the course then you may already have an assessment plan for it. Your goal in working with this portfolio will then be to complete and to fine tune that plan.
- Gather together all the assessment tasks you used in the most recent offering of the course (examinations; other assessment tasks, both during and outside semester; tasks which you or others marked solely for feedback purposes).
- Classify each according to its main purpose: to help assign a grade to a student; to provide feedback to students on their learning; both of these.
- Describe what each task is designed to assess.
- Where the task is at least in part to help assign grades, write a brief description of how it feeds into a student's grade.
- For all tasks describe briefly how, if at all, students are to receive feedback on their attempts.
- Enter all this information into the table in the assessment portfolio for this course.
Completing the assessment portfolio for a new course
If the course is completely new then your goal in completing this portfolio will be to define and to set out in detail an assessment plan. You should use the same table as that for an existing course.
- Start by trying to formulate at least some possible
assessment tasks for the course. Possible sources
of inspiration are:
- tasks others have used in teaching previous versions of this or of somewhat similar courses
- tasks you encountered in this area while yourself a student
- tasks in text or reference books
- common practice in your School
- consultation with others (in other universities,
for example) teaching similar courses.
- You
will be seeking to include in your assessment program tasks serving
the dual purposes of
grading students' achievements
in
the course and
helping them learn, as well as tasks aimed
solely at helping students to learn. Tasks aimed solely
at assigning
grades
are not recommended;
where the task is a final examination, feedback
is still desirable, even though the resultant
learning may not
take place until
after grades have
been decided.
- Your assessment program should
include a variety of tasks to be completed under
a variety of conditions.
For example,
it might
include
tasks to
be attempted:
- during semester, after semester, and during mid-semester breaks
- in class and out of class
- under examination conditions (or not)
- by students individually or in groups.
- Try to classify each according
to the main purpose you see for it: to help assign
a grade
to a student;
to provide
feedback
to
students on their learning; both of these.
- Describe what each task is designed to assess.
- For all tasks describe
briefly how, if at all, students might receive feedback
on
their
attempts.
- Where a task is at least
in part to help assign grades, write a brief
description
of how it feeds
into a student's
grade. This
may
not be possible
until you have reduced your array
of possible tasks to a program.
- Enter all of this information into the table in the assessment portfolio for this course.
You
are now in the same situation as a teacher refining the assessment
program for an existing
course. Your
goals now
are to refine
and to fine tune the draft program you have
just developed in your
assessment portfolio.
Completing the assessment program
Goals and criteria
You need a set of learning goals for the course (statements of what the students ought to achieve as a result of studying the course). If you are already working on a course profile then probably you will have written the learning goals already. They then simply need to be copied from the course profile to the assessment portfolio. You also need a set of criteria against which the extent of achievement of the learning goals can be judged for each student. Assessment criteria might be thought of as dimensions along which students' achievement of the course learning goals can be judged, or as yardsticks against which such achievements can be measured. The possible extents of a student's achievements on each criterion are called the standards which might be achieved. These might be thought of as the lengths marked on the yardsticks.
Ideally the criteria are associated with the learning goals, and the relevant criteria are stated more specifically for each assessment item. In reality assessment criteria are often specified only in relation to specific assessment tasks.
Assessment tasks, marking criteria, feedback and marking schemes
Before adding these to your assessment portfolio please browse through the PDF document on Assessment programs: Characteristics and constraints. This document includes information on commonly used assessment methods, together with their generally claimed strengths and weaknesses.
In your portfolio you should try to include for each task:
- the text of the task. Where the task is to be given to students as a handout, ideally that handout will be written now and included in the portfolio. Where the task is to complete exercises from another source (textbook questions, for example) only the relevant references need to be included
- the criteria against which students' attempts at the task are to be judged
- a feedback scheme and, if some kind of mark or set of marks is to be provided, a marking scheme, where marks and feedback are linked clearly to the specified criteria.
Assessment timetable
At this stage you almost have a complete draft of the assessment program for the course. All that remains is to timetable that program over the period in which the course is to be attempted and to design a scheme for allocating grades to students. In defining the timetable you probably will consider:
- the teaching and learning program in the course - assessment is part of this program
- the demands on the students, especially their likely assessment loads from other courses. In unit programs (Arts and Science, for example) there may be no common pattern of course choice among students studying your course, so this may not be possible
- the demands on your own time over the semester (and the demands on the time of others likely to be involved in marking and in giving feedback)
- common practice in your School (always remembering that 'common' practice is unlikely to be mandatory).
Grading scheme
For many courses this is the most difficult step in the design of an assessment program. It is also an important one. The grade a student gains in your course may help to determine or even to constrain the future possibilities for that student. It may influence the courses the student studies in the future and even the nature of the student's future career. However, it is possible to overstate that importance; one of your main tasks is to help students to learn your course - given their inclinations and interests, to help them to achieve to the best of their abilities the learning goals for the course. A final grade is not the only, nor necessarily the main outcome, of a student studying your course.
Grades are an attempt to reduce the complex process of evaluating what a student has achieved in studying your course to the awarding of a single label. When seen this way, it is clear that the grading process is bound to be somewhat unsatisfactory and simplistic. A good grading method will reflect as accurately as is feasible the student's achievements.
The grade
will be linked to the extent to which
the student
has
achieved
the learning goals
in the course.
The student's
results
on the various
assessment tasks will
be the input to any
grading method.
Assessment
criteria and
standards
at
both course and
assessment task level
will be used to ensure
that these results
and the ultimate
grade
will be as valid as
is feasible.
Refining the assessment program
In refining the draft assessment program several questions need to be asked. They may even need to be re-asked several times in successive cycles of refinement. The questions are:
Is the timetable feasible and reasonable for
both staff and
students?
This question needs
to be revisited repeatedly
as other
aspects
of the program are
brought
closer to
finality.
If the time
and work
demands either on
yourself or the students
are
markedly excessive
then tasks
simply will not be
done or, if done,
will not
be done
on time.
It is especially
important
that feedback
on
students'
assessment
attempts
be
provided at appropriate
times. See the PDF
document Grades
and feedback for
further information
on
feedback.
Do the assessment
tasks assess all
of the course
learning
goals that
you have
decided
to assess?
There is a table
in the assessment
portfolio
designed to help
you to check this.
Remember
that it is
not necessary or
even necessarily
desirable
to assess all course
learning goals
for grading purposes.
Some
may
be assessed only
for feedback purposes;
some may
be completely
unassessed,
but included to
show
students what they
might aspire to
gain from the course.
Are the course level assessment
criteria
consistent with
the task criteria
- for example,
do the task level
criteria
reveal
that
there should
be other course
level criteria,
or even
another
course
learning goal?
There is a table
in the assessment
portfolio
designed to help
you check this.
One criterion
often included
at the
task level,
but
omitted at
the course level,
is the quality
of students'
expression.
If
quality of
expression
is used to determine
a student's marks
on various
tasks and, therefore,
to determine
grades,
perhaps
it is worth
including as
an assessment
criterion
at
the course level.
Doing this may
then lead
you to include
being able to
write clearly
in the course
area
as
a
course learning
goal. In turn,
this will
require that
you provide
students
with
the opportunity
to learn to do
so.
Does the grading
scheme provide
at the very
least a defensible
evaluation
of
each student's
achievements
in the course?
This is a crucial
question. If
you have written
grade descriptors
(descriptions
of
what a
student must
achieve to
be eligible
for
each grade)
then you can
check to
see whether
your grading
method awards
grades
appropriately
according
to the
descriptors.
The final assessment program
When you have answered all of these questions to your satisfaction, you are then in a position to complete in the assessment portfolio the table which sets out your final assessment program. Now you should also modify the timetable to reflect the final version and modify the grading scheme to reflect your final decision. All tasks included in the program should be documented in as much detail as possible, including the marking criteria for each task, and, where possible, marking schemes. When they become available, insert copies of the actual handouts given to students at the various stages of each assessment task.
If, in the course of finalising the assessment program, you have discarded some assessment tasks, do not throw these away. They may be a useful source of ideas and inspiration when you redesign the program in future years.
