Started in 2005, the National Student Survey (NSS) is a yearly national study aimed at final year undergraduate students with the aim to gather students’ views on their courses. Whilst it remains a key source of data on student opinion, to say it is without flaw would be misguided. From its links to the controversial Teaching Excellence Framework (TEF), student boycotts and rigged results, there’s much to keep in mind when discussing the NSS’ data.
The TEF (suspended as of writing) is a framework introduced in England in 2017 to rate higher education providers on a scale of Gold, Silver and Bronze. Unlike traditional league tables which tend to rank universities against each other, the TEF rates them against each university’s own benchmarks, calculated considering the makeup of their student body and
what courses are provided. The ratings given by students are compared against the calculated benchmarks and the school is then awarded one of 5 flags (double-positive flag, a positive flag, no flag, a negative flag or a double-negative flag), which then goes on to determine the institution’s rank. These ratings expire after 3 years. This means that a university with an ‘Academic Support’ benchmark rating of 80% and achieves 80% could be ranked lower than one with a 65% benchmark that achieves a 70%.
The NSS’ results feed into three of the six categories considered when rating universities, namely; ‘Teaching on my course’, ‘Academic support’, and ‘Assessment and feedback’.
In the TEF’s first year, the methods they employed for their ratings was criticised heavily by the then Russell Group acting director and the vice-chancellor of the University of Southampton, who claimed that the differing benchmarks for each institution removed equity between the assessments of said institutions. Another criticism aimed at the TEF rating
system is that some outcomes are manually altered, moving a university from a Silver to a Gold, A Bronze to Silver etc. In the 2017 rankings, it was noted that many institutions were ranked higher than they should have by the TEF flagging system, and that those who achieved negative flags in categories informed by NSS data tended to be overlooked and moved to a higher rating. Alongside this, many institutions with the same flags in the same categories were rated differently, bringing into question the subjectivity of the system.
Whilst the issues within the TEF are not the fault of the NSS itself, its connections with the system have been heavily criticised especially when taking into account its links to the pricing of tuition. Publicly funded higher education providers in England typically have a cap of £9000/year on their tuition fees. However, those with a TEF rating are enabled to increase their price tag to £9250/year regardless of what rating they achieve. Due to this, many universities have boycotted the NSS including Oxford, Cambridge, Bath and Kings College London. These boycotts take advantage of the NSS’ response threshold of 50%, meaning that institutions that do not achieve a 50% response rate to the study will not be included in the published data.
In 2022, the Cambridge Student Union announced that they will no longer be boycotting the NSS after a successful campaign. However, they have continued to share their concerns surrounding the NSS allowing students to make an informed decision on their participation.
In exchange for the ending of this boycott, the University of Cambridge have pledged to ‘publicly oppose any attempts by the Government or the Office for Students to link undergraduate home student tuition fees to the Teaching Excellence Framework’, implement a ‘student consultation framework’ and consider the inclusion of a reading week. Despite
this, Cambridge does not appear in the 2022 NSS data, indicating that students have independently continued the boycott.
In contrast, the Oxford Student Union have continued their boycott – despite the current suspension of the TEF – due to the other rankings that the NSS informs which they claim creates a competitive market for education and encourages band-aid solutions to student satisfaction issues rather than working to fix deeper issues. They also highlight the role NSS
data plays in disadvantaging minority students and innovative teaching as the study has been connected to funding cuts and courses being dropped. Oxford SU’s boycotts notably have been successful in the past, with four years’ data sets not containing the institution, including the recently released 2022 data.
In 2008, students at the University of Surrey were informed that if 80% of the eligible student body responded to the survey, the student union would be allocated the funds to hire the band ‘Scouting for Girls’. The university clarified that it was an attempt to encourage a higher student response, however it does highlight a temptation to direct money towards efforts to
inflate the survey, rather than to fix deeper issues.
The NSS’ methodology is not airtight. One Guardian reader claimed that due to it being an optional survey, not everyone will respond meaning that universities that don’t promote it as much as others are likely to have a low response rate, with only those with complaints are provoked to respond.
The competitiveness fostered by the study has also led to some serious allegations being made against a lecturer at Kingston University in 2008 in which the staff member in concern was recorded encouraging students to inflate their responses and that “If Kingston comes down the bottom, the bottom line is that nobody is going to want to employ you because they’ll think your degree is sh*t”. The recording went on to show the staff member instructing students what aspects of the survey to focus on, saying that the university was looking to improve its image in those areas. A second member of staff encourages students to refrainfrom responding negatively within the survey, that “all that garbage you’ve been spewing out
about us” is not to be included in the NSS data. The university confirmed that the recording was legit, and the incident was condemned. The Kingston University’s Department of Psychology was subsequently removed from the 2008-9 NSS data release.
Comments made under BBC articles on the scandal echo the allegations made within the pieces, claiming that their own universities and institutions also encouraged them to inflate their responses, and that they were warned that negative responses would negatively impact themselves. Other responses criticise the league table culture, reflecting Oxford SU’s concerns about commercialisation and public perception of institutions.
Whilst this specific incident took place 14 years ago, it highlights the pressure that the league table system puts on the legitimacy of the NSS’ results and on the students providing their responses.
In the aftermath of the Kingston incident, it was also alleged that staff from Anglia Ruskin University described the survey as an avenue for ‘reputation management’. However, the same email states that the staff are to only attempt to maximise the student response rate.
However, it is important to note that the NSS’ questionnaire is adjusted each year to account for the evolving student experience and to account for situations such as the Kingston one discussed above. The set of questions used in 2022 do not feature any mention of the COVID-19 pandemic and new guidelines were issued in 2008 to ensure that incidents such as those discussed would not be repeated.
Many universities, including the University of Edinburgh and the University of Oxford have praised the study, maintaining that the responses help them to improve the student experience. In 2009 – the year following the Kingston incident – Edinburgh stated that the 2008 NSS feedback encouraged the university to begin working with EUSA in order to improve the student experience in lacking areas, namely in regard to assessments and feedback.
At the end of the day being informed on any study you partake in is crucial. Before consulting the data provided by the NSS, it is important to remember that the study only covers the student experience of an institution and course, not the provision of them and so are subject to bias and meddling. League tables are not an accurate portrayal of the quality of a university and should be taken with several grains of salt.
Image courtesy of Jon Tyson, licensed under the Unsplash licence