Is it Different in Business?

A paper presenting the preliminary findings of a review of the assessment policy and practices
in the Faculty of Business at QUT

by

Dr Jennifer Radbourne

Director of Graduate Studies, Faculty of Business, Queensland University of Technology

 

The University and the External Environment

As universities seek to reposition themselves in response to reduced government funding and increased competition for students, the formerly untouchable aspects at the core of a university such as teaching and learning are being reviewed for quality assurance, efficiency and effectiveness. Questions of cost, variety in delivery, quality of staff, adherence to the principles of education, relationships of knowledge and work, access and equity, internationalisation, technological developments and the place of research, form the basis of many committee discussions. For "a university for the real world" as QUT brands itself, this repositioning and review of educational policies and practices is reflected in accountability structures right down the line.

The role of a university in society has changed. There is a philosophical shift from investment in supply to investment in demand. Course developments are market driven. Societal changes are reflected in the demography of the university through the student population, the areas of excellence, the leadership, resourcing, partnerships and geographical expansions. This changing role is particularly evident in accountability to professions and disciplines served by the university. QUT's Faculty of Business is an excellent example of this. A client orientation; responsiveness to business, industry and the professions; and a commitment to producing graduates and research that 'make a difference' in the business environment, are key outcomes in the Faculty's planning. This accountability to the external environment presents a new set of variables influencing the policies and practices for assessment of students' work:

 

Factors Influencing Assessment in QUT's Faculty of Business

  1. External pressures for change in practice

    To maintain a competitive edge in the development and delivery of courses, the Business Faculty seeks collaboration and accreditation with professional business institutes, such as the Public Relations Institute of Australia, The Australian Human Relations Institute, Institute of Chartered Accountants, Institute of Certified Management Accountants and The Australian Society of Certified Practising Accountants. Graduating students add this accreditation to their C.V. and use the membership as leverage in attaining employment. Accreditation and membership with such Institutes directly influences curriculum content and assessment.

  2. Trends to more diverse methods of assessment

    Flexibility, innovation and negotiation are encouraged in teaching and learning. Great value is placed in the Faculty on ideas and proposals that address innovation and flexibility. As academics continue to design assessment pieces that relate to the workplace and in some instances involve industry partners as assessors; new questions of accountability arise. Technological developments are forcing changes in assessment methodologies. Peer group assessment is increasing, as teamwork tasks become core to learning and work simulation. It is accepted that there is more beyond the traditional three pieces of assessment: the report, essay and exam. In particular there is a trend to student self-determination and negotiated learning contracts as a means of achieving both skills and knowledge development.

  3. Quality assurance of graduates

    These trends in diversity place great emphasis on the need for quality assurance methodologies in the Faculty. By adopting a whole of Faculty approach to course design, course reporting and course planning, the Faculty has set in place a means of measuring and monitoring quality. Learning outcomes, and thereby assessment practices, are now part of an accountability chain from senior administrators, course directors right through to unit coordinators and lecturers and tutors. The Faculty's Strategic Plan is in reality delivered in the content and assessment of each unit.

  4. Internal challenges to traditional methods and outcomes

    This quality assurance process encourages individual responsibility. In fact, while it enforces a rigorous reporting schedule, it also encourages internal challenges to traditional assessment methods and outcomes. Individuals, through School Teaching and Learning Committees, are stretching the boundaries of assessment rationale and methods. There is comfort in a group of colleagues as peer review team, investigating more effective and efficient means of assessing students in units developed and delivered by the School. Problems are explored where the impact is experienced. Academics accept responsibility for their teaching and assessment practices, not just for their research and knowledge, as being core to effective performance.

  5. Growing interest in school based assessment

    In this environment of accountability and performance measurement and in an administrative environment where more and more centrally based university systems are being devolved to Faculties and Schools, there is a growing demand for control over assessment. While QUT's norm based assessment system is accepted as policy and practice, school based assessment allows staff to measure competency in the unit's learning outcomes in relation to the expected norms. There is a strong case for a School managed assessment regime that feeds into the Faculty and university system.

Each one of these factors influencing the Faculty's assessment practices informs the Faculty's repositioning in the external environment, particularly the academic environment and the corporate environment. There is a growing corporatisation of the Faculty. Terminology such as "clients" and "customers" for students'; measures of cost efficiency and effectiveness for course development and delivery; and increasing use of marketing strategies and marketing officers, are some of the contemporary corporate practices. Indeed, the Faculty's Strategic Plan is the force that drives course design, delivery and assessment.

 

Strategic Planning Objectives

1 To provide a wide range of undergraduate and postgraduate business courses that are based on the principles of academic rigour, flexible delivery, ethical practices, social responsibility and professional relevance.
2 To produce business graduates who have generic skills, specialist knowledge and problem solving abilities.
3 To further develop collaborative links with business, industry, the professions and the community both on-shore and off-shore.
4 To develop the international dimensions of the Faculty's teaching, research and service activities.
5 To build a stronger research culture within the Faculty supported by appropriate research infrastructure.
6 To pursue a national and international research reputation in selected business and management areas.
7 To contribute to the training of qualified researchers in business and management disciplines.
8 Contribute to the intellectual, social, environmental, economic and cultural leadership and well-being of Australia, other nations and the community.
9 Foster mutual understanding, exchange of knowledge and cooperation between nations in the region.
10 Contribute to the development of and provide leadership to the professions.
11 Develop an awareness in students and staff of their social and cultural responsibility to the wider community.
12 To value and support the contributions of all staff of the Faculty.

These objectives are aimed at the Faculty of Business achieving leadership in the business community. Graduates with particular skills and knowledge are the primary measure of achievement of that position.

The title of this paper "Is it different in Business?" begins then with the question: Are assessment practices influenced by the external environment and the repositioning of the Business Faculty? To answer these questions the Faculty of Business at QUT has established a Working Party to review the Faculty's current assessment policies and practices.

 

Terms of Reference for the Assessment

The Terms of Reference for the Faculty of Business Assessment Review are:

  1. To review and evaluate the assessment policy and practices of the Faculty of Business in regard to each one of its courses and their constituent units. Issues raised in the literature review will direct the specific factors to be targeted in this review. Some of the factors to be considered include the methods of assessment and their weightings, amount of and percent of marks assigned to group work, and the balance of formative and summative assessment.

  2. To make recommendations to the Faculty Education Committee for changes to the assessment policy and practices that the Working Party deems appropriate.

  3. To have particular regard to the impact of assessment polices and practices on international students.

To address the above Terms of Reference the following strategies are being undertaken:

 

Trends in Literature

Two areas of interest to this review have been identified: contemporary issues regarding assessment at the tertiary level, and current assessment practices.

  1. Contemporary assessment issues

    Common themes discussed in the literature concentrating on assessment at the tertiary level include the roles of assessment, issues related to reliability and validity of assessment, and the limitations and benefits of particular assessment methodologies.

    • Roles of assessment

      There is argument that the focal role of assessment needs to move away from grading or categorising students (summative assessment), to include more assessment that assists in the learning process (formative assessment) (Piper, Nulty & O'Grady, 1996; Nightingale, 1994). This assumes reflection of assessment philosophies by education practitioners and School, Faculty and University hierarchy.

    • Reliability and validity issues
    • A number of recommendations to optimise reliability and validity of assessment are suggested in the literature:
      • define the criteria to be used to assess students' work (Piper et al, 1996; Crooks, 1988)
      • use of continuous assessment (Piper et al, 1996)
      • use of a variety of methods of assessment (Race and Brown 1998; Piper et al, 1996); and
      • match assessment to learning outcomes (Piper et al, 1996)

    • Assessment methods

      Numerous assessment methods are suggested in the literature, such as written exams, open-book exams, structured exams (multiple choice questions), essays, reviews, reports, practical work, portfolios, presentations and vivas, (Race & Brown, 1998), learning contracts, journals, and self and peer assessment (Academic Services Division, University of Canberra, 1994)

      Coinciding with this rise in methods of assessment is research into the impact different methods have on student learning. Research suggests that student learning is influenced by the type of assessment. The main arguments relating to this area are the most effective methods to encourage deep contextual understanding, life long learning, and the application of concepts and knowledge (Piper et al, 1996, Nightingale, 1994). Related themes are the benefits of continuous assessment, the balance between summative and formative assessment and the benefits and limitations of normative based and criterion based assessment (Piper et al, 1996).

  2. Current assessment practices

    The following discussion is a brief overview of preliminary findings based on the Assessment Guide from the University of Canberra (Academic Services Division, 1994), and reports on assessment from Griffith University (Gilmore, 1994) and the University of Queensland (Task Force, 1996). Information from other universities is forthcoming and will be analysed as part of the review.

    Both the University of Queensland and Griffith University reflected on the purposes of assessment and developed an assessment philosophy for their university.

    Matching assessment to a unit's learning outcomes, providing students with feedback and developing assessment criteria were seen to be important issues.

    All three of these universities reviewed the literature on the merits and limitations of normative and criterion based assessment. The trend by all three is towards a combination of both.

    The Griffith University Report found that most faculties had assessment items which were formative in nature in as much as assessment was continuous. However, comment was made that the duration of the semester limits the application of 'true' formative assessment.

    Another source of information about current assessment practice in universities is found in Piper, Nulty and O'Grady (1996). In their research of the assessment practices in the discipline areas of Humanities, Social Studies, Health, Science, and Maths and Computing, they found that examinations are the preferred means to assess students despite related reliability and validity concerns. Reasons provided for the popularity of the use of exams include issues related to their comparability, utility, cost effectiveness and the protection they offer against cheating and plagiarism (Piper et al, 1996).

    Other findings of interest in this study are:

    • The most popular type of univigilated assessment was essay.

    • Assessment by oral presentation was used in almost 20% of Humanities subjects, but rarely included as assessment in the other disciplines.

    • Differences were found in the balance of the use of the various methods of assessment between the five discipline areas.

 

Preliminary Findings of an Audit of Assessment in FOB Units

The assessment practices of QUT's Faculty of Business have been audited by the Assessment Review Working Party. In addition, some student focus groups have been conducted. The following reports on the preliminary analysis of the survey and focus groups' data. The analysis is in its early stage and at this point findings are limited.

The Faculty of Business at QUT offers 358 units in six Schools: the School of Accountancy, the Graduate School of Business, the School of Economics and Finance, the School of Communication, the School of Marketing and International Business and the School of Management.

Information on only 61% of these units is presented.

Table 1 shows the most used methods of assessment. End of year exams were the most widely used mode of assessment. 73.8% of undergraduate units included an end of semester exam and 56.6% of postgraduate units. The data also reveals that the proportion of units that include an end of year exam decreases from first year through to postgraduate level. Mid semester exams also appeared to be popular in undergraduate units, most notably in first year units (68.2% of units), decreasing in use in postgraduate units (21.1%).

However, collectively, there was great diversity in the types of assessment offered across all units. Furthermore, there appears to be significant variation in the modes of assessment included in units at different year levels, and it is hypothesised that variation will also be found between units offered by different Schools.

Table 1: Use of Various Assessment Modes

  First Year Units Second Year Units Third Year Units Undergraduate Units Postgraduate Units
End of Semester Exams 81.8% 78.1% 73.8% 76.5% 56.6%
Mid-Semester Exams 68.3% 37.5% 38.5% 43.7% 21.2%
Essays <2000 words 31.8% 37.5% 27.7% 31.1% 20%
Essays >2000 words 0 15.6% 20% 15.1% 35.3%
Reports 22.7% 31.2% 23.1% 25.2% 21.1%
Project 13.6% 18.7% 32.3% 25.2% 21.1%
Tutorial attendance/ preparation 4.5% 15.6% 7.7% 9.2% 13.3%
Weekly Assignments 22.7% 15.6% 1.5% 9.2% 7.8%
Oral Presentations 22.7% 25% 37% 31.1% 50%

Oral presentations were more likely to be offered in postgraduate units (50% of units) than in undergraduate units (31.1%). Oral presentations were least often included in first year units (22.7%).

Essays of 2,000 words or less were included more often in first and second year units than in third year or postgraduate units. The opposite was true for essays of more than 2000 words.

Reports appeared to be used equally despite the year level of the unit and projects were more commonly assigned in third year or postgraduate units.

Weekly assignments were more likely to be set in first year and second year units. Interestingly, marks for tutorial attendance and/or preparation were more likely to be included in second year units and postgraduate units.

Nearly all units included continuous assessment and the average number of pieces of assessment is shown in Table 2.

Table 2: Average Number of Assessment Pieces per Unit

Year Level Average Number of
Assessment Pieces		 Range of number of
assessment Pieces
1 7.6 2 - 28
2 4.47 1 - 15
3 4.45 1 - 16
PG 4.1 1 - 15

Group assessment was most likely to be included in the assessment in second year units (50%) and least likely in postgraduate units (26.7%). There was no consistency in the percentage of marks assigned to group work, ranging from 100% to 0%. Table 3 shows the average percent of marks assigned to group work according to unit year level. A smaller component of the total marks is assigned to group work in first year units (18.6% of total marks) than in third year (56.8% of marks) or postgraduate units (42.3% of total marks).

Table 3: The average percent of marks assigned to group work

Year Level Average Weighting Variance
1 18.6% 40% - 0%
2 35.6% 70% - 20%
3 56.8% 100% - 10%
PG 42.3% 100% - 10%

Students involved in the focus groups were asked their opinion on the proportion of marks that should be assigned to group work. The preference for the undergraduate students was for group work to only account for 20% or less of total marks. Although consensus was not reached with the postgraduate students, their preference was for group work to account for 25 - 40% of the marks for a unit.

Of the 219 units included in this analysis only 2 included a 100% end of semester exam. The average weightings of marks assigned to final exams is shown in Table 4.

Table 4: The average weighting given to final exams

Year Level Average % worth Range of Marks
1 54.7% 100% - 30%
2 50% 75% - 30%
3 52.1% 75% - 20%
PG 51.1% 100% - 10%

In summation, the Faculty of Business at QUT uses a variety of methods to assess students, as recommended in the literature. The most commonly used method is through examination. This reflects the practice of other Australian universities.

Although difficult to draw direct comparisons to Piper et al's (1996) findings, due to differences in data collection, it appears that the pattern of assessment of QUT's Faculty of Business QUT does not readily mirror the patterns of assessment of any of the five discipline areas. This reinforces Piper et al's (1996) finding that assessment methods differ between discipline areas.

 

Informal Practices in Assessment

One of the concerns in this review was the quantity and quality of informal practices used by individual staff members or by Schools in their assessment of students' work. There was sufficient evidence to suggest that informal practices were widespread, sufficiently entrenched to be viewed as formal policy, and possibly discriminatory against particular students. The audit of units included a set of questions aimed at identifying information on unit outlines that may be interpreted as policy or may indeed assist the development of assessment policy.

The following questions were asked:

  1. Are instructions included in the unit outline on the required methodology and content of the piece of assessment?

  2. Are instructions included on what is expected by the lecturer for a student to receive a grade of high distinction, distinction, credit, pass, fail for the assessment items in this unit?

  3. Is there any flexibility in assessment (ie. the student may negotiate alternative items of assessment)?

  4. Which scale is normally used to communicate the marks to the student?
    Two point scale Pass/Fail
    Five point scale A - E
    Seven point scale 7 - 1
    Percent %
    Number of marks  

  5. What is the turn-around time between due date and return of assessment?

  6. What evidence is there that a particular student group (eg. international students can access additional support (tutorials, submission of drafts of assignments, additional tests) to achieve assessment standards of the unit.

While many units included instructions on the required methodology and content for the pieces of assessment, very few provided information on the remaining five questions. In some instances this information was included in other written communication to students (books of readings, assignment guidelines).

The second means of eliciting data on informal practices involved group discussion of a set of questions by School Teaching and Learning Committees. The responses were recorded by the School representative on the Assessment Review Working Party. Groups were asked to discuss what they believed Faculty Policy to be in regard to:

The University publishes its policy on supplementary assessment, deferred exams, feedback on progressive assessment, cheating and plagiarism, grading scale and unsatisfactory performance. The Manual of Policy and Procedures (MOPP) also indicates that unit outlines include "where appropriate, a statement of any item/s for which a pass is required in order to gain an overall pass in the unit". The Faculty of Business has no discrete assessment policies but the Academic Board has discussed and resolved a "policy" on assignment grading; grade distribution guidelines for undergraduate, postgraduate, large and small classes; and the proportion of group and individual assessment at postgraduate level.

However, these policies do not have universal understanding or adherence. Individuals and Schools set penalties for late and resubmissions. Supplementary assessment is offered by some lecturers on a "case by case" basis. Schools have different methods of validating submission of assignments. A variety of assessment scales (marks, percentages, grades - numerical and alphabetical, are used for marking assignments. Assignment workload is not planned across a course. Turnaround on assignments varies from one School's prescribed "7 days" to up to five to six weeks. Grading distribution guidelines are challenged at every Faculty Examination Board meeting. No control is placed on the proportion of group and individual assessment. Many staff award marks for attendance and participation, while others believe the Faculty prohibits this.

Students in focus groups strongly believed that marginally failing a final exam but passing the unit overall must be given credit as a unit pass. They did not support a policy of a pass requirement in all aspects of assessment, especially the final piece of summative assessment. Students do not focus on the relationship of assessment to the learning objectives of the unit. Feedback on assessment was not rated highly because of the ongoing focus on the next piece of work. Very few students negotiated their own assessment.

The question that arises out of an examination of informal practices relates to the most effective level at which quality assurance can be measured. Would Faculty wide assessment policies ensure regulation, equity, achievement of desired learning outcomes, or is quality assurance best measured by the individual staff member who designs the unit, sets the learning outcomes, and works with the student to assess the development of skills and a knowledge base.

 

Summary

While literature in the field has not isolated assessment methodologies in business faculties and so provided a context for this review, it has provided a foundation of contemporary issues that the Faculty of Business must address. The preliminary primary research has presented a limited but predictable set of data that demonstrate the Faculty's assessment practices are both traditional and inconsistent. The question "is it different in Business?" is not able to be answered at this stage, and may well require a more lengthy study over three years once new policies and practices are set in place.

 

References

  1. Academic Services Division. (1994) Guide to Assessment Practices at the University of Canberra. Canberra: University of Canberra.

  2. Crooks, T. (1988) Assessing Student Performance, Green Guide No 8. Sydney: HERSDA.

  3. 3. Gilmore, AM (1994) Review of Assessment Policies and Practices at Griffith University. Brisbane: Griffith University.

  4. Nightingale, P. Issues in assessment in higher education. In Jones, G. (ed). Proceedings of Teaching and Learning Forum 1994: Effective teaching in tough times. Armidale: University of New England Academic Staff Development Unit.

  5. Piper, D., Nulty, D. D. and O'Grady, G. (1996) Examination Practices and Procedures in Australian Universities: Summary report. Canberra: AGPS.

  6. Race, P. & Brown, S. (1998) The Lecturer's Toolkit. London: Kogan Page.

  7. Task Force on Assessment Policies and Practices. (1996) Report of the Task Force on Assessment Policies and Practices. Brisbane: University of Queensland.

 

 

The Assessment Review Working Party would like to thank Ms Wendy Gilmour (Senior Research Assistant) and Dr Duncan Nulty (Higher Education Program Ealuator) for their assistance on this paper.

 

Conference Entry Page About the Conference Abstracts Conference Program

TEDI Home About Assessment UQ Home

This site built on a Macintosh using Dreamweaver.. Last modified 20/1/99; 10:14:25 AM.