Bias in today’s society is no longer limited to the loud and proud variety, but is fortified over long periods of time by layers upon layers of historically maintained injustices. According to a 2019 study, this is especially present in universities through student feedback to their teachers.
Over 16,000 universities across the globe run student evaluations of teaching (SETs) at the end of each teaching period. When numerous studies have concluded that these surveys are unreliable and biased, why is the Queensland University of Technology still running the program?
How does the Student Voice Survey work, and how did it start?
After reviewing their student feedback survey in 2021, a problematic numbers game, QUT Professor Abby Cathcart suggested screening the quantitative data against students’ written submissions.
This technique gave us the Student Voice Survey we know today, the only university-wide opportunity for students to give written feedback instead of reducing tutor service, attitude, and performance to a vague number.
According to their website, QUT’s Evaluation Policy began back in 2004 and since the Student Voice Survey’s introduction in 2007, it has changed into a Likert-scale and anecdotal machine learning-led survey.
This machine learning program was developed at QUT with the ability to screen harmful feedback before it ever reaches a tutor’s eyes. With the university’s duty of care being not only to students but also to their staff, this process is the best of both worlds.
But it’s not always 100 per cent effective.
Professor Cathcart did not have a ballpark estimate for how effective the screening program is but reiterated its usefulness and continual development as time has passed.
“The machine learning system we use at QUT can perceive passive aggression, as well as overt offences like racism, sexism, ageism, and ableism. The more answers screened, the more intuitive it becomes,” she said.
While profanity is “easy” to spot in feedback, it is harder to monitor baseless criticisms about a staff member’s race, gender, sexual orientation, accent, or appearance.
When the screening process flags a discriminatory response, QUT will notify the student that their comment has broken the university’s code of conduct, and will give them a second chance to edit their response and resubmit.
“We want students to learn that this is professional feedback we are seeking. A lot of negative comments are made in the heat of the moment, and we give them a second chance to keep that off their permanent record and to make their responses data we can use,” Professor Cathcart said.
According to the Professor, there are around 50,000 comments received from the survey each semester, and only 100 of those had to be removed last semester. That’s 0.002 per cent.
This number does not account for the edited comments that QUT students amend or the number of comments that slip through and are then flagged by staff.
In an email sent to all students at the end of Semester One this year, Professor Robina Xavier, QUT Deputy Vice-Chancellor and Vice-President said that submissions to the survey would not fall on deaf ears.
“Survey responses are reported directly to your educators, unit coordinators, and their supervisors,” she wrote.
But what do survey responses actually do?
In an exclusive interview with Glass earlier in 2023, Professor Cathcart advocated for the survey’s positives, clearing up myths about its intended use, function, and benefits, and mentioning past changes that the survey has supported.
These changes include identifying the growing need for mental health services for students, the professional development of staff to be able to identify at-risk students, and policy changes to standardise confusing language.
But does the Student Voice Survey affect curriculum delivery, tutor status, chances of promotion, and student experience overall? And if so, how can QUT’s Board justify it?
Professor Cathcart said that the Student Voice Survey has done some real behind-the-scenes work with unit structure and delivery in the past, taking a “contextual” view of how students’ experiences can be actioned.
“I remember when I was teaching first-year management in the Business School here at QUT, and there were a lot of students in that class who weren’t happy with the feedback on their assessments.
“I undertook a major project with my team and, using the submissions from the following semester, saw that it worked. Students were happier.”
She also said that in terms of staff promotions, student evaluations are definitely a part of the equation for a successful application, but won’t ever be used as the single deciding factor.
“The evaluation policy very clearly says that supervisors and educators alike should not rely on a single source of data to reach conclusions about what may be happening within a unit or with teaching.
“I’ve been on promotion committees in and outside of QUT, and we’ll look at student evaluations of teaching as one of the things being considered. But they do so by looking at other indicators alongside it, which I think is really important,” she said.
Interested in hearing more about QUT’s Student Voice Survey? See our full interview here.
Listening time: ~24 mins
So, when this is paired up with research across the globe displaying that English-speaking white male teachers will often get more positive feedback from students than their female, Indigenous, or non-English speaking background counterparts – it shouldn’t completely get rid of your scepticism.
Research from the University of Iceland, a country known for its progressive gender equality, revealed that male students rate female teachers lower than their male counterparts based on their adherence, or lack thereof, to traditional gender roles.
A Southern Cross University-led study reiterates that if unchecked, SETs are a breeding ground for heavily gendered comments that are severely damaging to educators’ well-being, with an alarming emphasis on female staff.
This is not an isolated phenomenon. Students have an inherent bias against teaching that differs from the prevailing establishment of white male academia, and QUT is, unintentionally, supporting it through using these evaluations.
What do QUT students think?
So, what do QUT students think about the Student Voice Survey? What do students think of universities peddling a process that inadvertently uses their biased opinions to maintain the substandard female benchmark of academic success?
To gather some preliminary data straight from the source, fifteen QUT students were surveyed for their opinions and experiences of the survey.
The survey results showed that students think rather apathetically about the Student Voice Survey, with almost 87 per cent saying that they believe their opinion on their tutor’s performance doesn’t matter and only say what they think about half of the time.
When asked if students thought that their submissions should have more sway over staff promotions, there was a resounding yes; 100 per cent of those surveyed think that their opinions of their teacher’s performance should be considered more in the process of seeking promotions.
That being said, if student evaluations were held in higher regard than other mitigating factors like tenure, referees, previous employment, and excellence in their field, this would only worsen the statistics of successful female academic promotion.
These survey results in part explain the low numbers of participation in the Student Voice Survey, despite QUT doing their utmost to attract students to give their honest opinion for the supposed better of the university.
Studies show that a contributing factor behind these apathetic student results might be partly due to the rise in survey fatigue.
According to HubSpot, survey fatigue is when people lose interest in surveys due to the large number of requests they receive, or the number of questions and effort required to complete them.
The idea of survey fatigue became more mainstream during the COVID-19 pandemic, with research from the National Institute of Health showing that the younger demographic was becoming harder to connect with.
From the students in the sample, those who had done the survey had complaints about their course experience but saw no change or further inquiry from tutors, unit coordinators, or supervisors.
Those were asked if they had any teacher horror stories while at QUT and took to the Student Voice Survey to anonymously report them, and 60 per cent had instances they could immediately recall – most featuring male teachers.
The students requested to remain anonymous as their referenced submissions were made in semesters past, and they feel like they still do not matter today.
“Yeah, I’ve complained a few times about unreasonable assessment guidelines enforced by my tutor, and time allowances for peer-review-based work – no dice. And the people that I know who do submit to the survey don’t really say what they feel – why would we, it makes no difference.” – Student A
“I would sometimes proofread a friend’s not-so-good work… and one of my male tutors would just pass them. Not an isolated thing either, it was a bunch of students. It didn’t matter if the standard of work was actually good… so I complained on the survey and heard nothing back. But I think him passing those students gave him great reviews… who knows?” – Student B
“I currently have a tutor horror story unfolding now in one of my classes, he’s a tyrant – but I don’t see the actual point in speaking my mind. QUT is just going to do what makes them the most money. At the end of the day, it’s a business and I think the current lack of students taking the survey keeps that business-oriented status quo consistent.” – Student C
Here’s the problem: this apathy from students is compounded by the mentality that even if they did support the survey, their opinions simply do not matter to those who have the power to action them.
And if white male teachers can receive positive reviews, despite questionable performance, while their female counterparts bear the brunt of harmful comments, how can any progress be made?
What do QUT staff think?
Director of Indigenous Health and an Associate Professor in the School of Public Health and Social Work Deb Duthie raised an interesting point that university students “need their voices heard” in a way that doesn’t endanger staff.
“I think it’s a damned if you do, damned if you don’t situation – because if you don’t provide an evaluation space, teachers suffer, education suffers, and students need that voice in a university setting,” she said.
As an academic who has faced negative feedback from numerous different surveys in and outside of QUT, Associate Professor Duthie has some horror stories herself.
“From the Student Voice Survey, I’ve had students say ‘I don’t pay for this course to hear about people’s personal lives and reflect’, which comes off as a personal attack when that is the way I teach in this unit – it’s sharing myself, my experiences as an Indigenous person.”
It’s not just her either, she said, “One tutor I know was having a tough time last year with one of her students who wrote some terrible things in her evaluation, so I rang her up the next morning and she’d been crying ever since.”
But when asked if she’d ever requested for a comment to be removed by QUT, Associate Professor Duthie said she was always “in two minds” about erasing those original comments.
“If there are going to be racist or misogynistic comments thrown into student feedback, sometimes I think they should stay in there as it shows what teaching staff are having to deal with.”
She also noted how common it was in her experience for male teachers to receive more positive feedback from SETs than their female counterparts.
“You learn to expect it. I think I probably almost expect that I’m going to get particular comments from the survey. And male staff tend to get better feedback than female staff across the board,” she said.
Some teaching staff from faculties across the university agreed that the survey’s format has improved but question its place in tertiary education with its reputation of harming teachers’ well-being.
“I get plenty of feedback from my students face to face – most of the time the students who do submit to the survey don’t have much to comment on… I think that student feedback is important as an academic, but the Student Voice Survey doesn’t do a good job of gauging real feedback.” – Lecturer X
“In my field, student feedback is important – I can never deny that, but I don’t see a lot of student’s more helpful feedback being actioned… I think that the Student Voice Survey season creates this air of competition and dread (amongst staff), and I wish it didn’t. I know a few staff who refuse to read student feedback – they can’t handle it.” – Professor Y
“I think the survey definitely has a better format now, but it’s no wonder students don’t engage with it. And it’s a breeding ground for hateful comments – even though the survey screens them, sometimes I swear I can feel them looming.” – Professor Z
An air of competition, hateful comments, and the feeling of being stuck in academic feedback purgatory – that’s the effect on QUT staff.
As discernible from answers by both students and staff, the Student Voice Survey is leaving a sour taste in people’s mouths. So, why is it still kicking?
Why is the Student Voice Survey still around?
According to an email sent to all students last semester by the Deputy Vice-Chancellor and Vice-President Professor Robina Xavier, the Student Voice Survey was undertaken by “20.2 per cent of students”.
In 2022, QUT had 50,191 students enrolled. That’s 10,139 students who submitted feedback and over 40,000 who didn’t.
QUT has tried everything to increase engagement in the survey, shy of making it compulsory; making the survey anonymous, incentivising participation with the chance to win money (a precious commodity for university students), and appealing to charity through giving money to the university-run Learning Potential Fund or the Guild’s Foodbank with each submission. But no dice.
So, why is it still around? When most parties agree that the negatives outweigh the potential positives and research points to getting rid of SETs in their entirety, why are numerous universities still opting to follow this format?
This question leads to another – is there a better way to gather feedback from students that is beneficial for everyone? Is this problem based on a lack of creativity on the university’s end?
Professor Cathcart said the survey has a place within the university as a widespread canvassing tool for student feedback, as long as it’s used “carefully”.
“I do think the survey has a place, and providing institutions use it carefully and transparently so that everybody understands what the response numbers are, what the response rates are, and how it should be used within context as a single form of data.
QUT is one of the few universities in Australia that even bother to screen their survey responses. This raises a bigger question of why universities are placing the onus on teachers to read and report harmful comments after the damage is done.
Below is the composed list of Australian universities that do and do not screen their student evaluations for harmful comments. Their rank in Australia is also shown, according to Top Universities.
Something is clearly amiss in the majority of Australian university’s SET policies. If QUT staff dread SET season, even when comments are screened, imagine how the rest of Australia’s tertiary educators feel at the end of each semester.
This issue is only made worse by students’ unconscious bias towards the female, indigenous, and ethnic minority of educators in universities across the board. Associate Professor Duthie said it best.
“If there’s someone who’s got beef, unfortunately, you’re gonna hear about it. I only ask myself, when is it going to stop?”
Emma Radford is a second-year Comms and Journalism student who’s interested in all things writing, pop culture, and storytelling. She has written stuff about Aussie cops, female empowerment through pole dance, and bikie-affiliated corner stores – really anything she can sink my teeth into. Got a story? She is most definitely interested in it – and she loves a chat. If you end up reading this – have a great day!
