Statistics watchdog urged to investigate exam regulator over secrecy and data handling

The statistics watchdog has been urged to investigate the exam regulator over its secrecy and data handling amid concerns that its approach risks undermining public trust in statistics.

The UK Statistics Authority should launch an urgent review into the English and Scottish exam authorities, according to The Royal Statistical Society (RSS).

Sharon Witherspoon, vice president for education and statistical literacy at the RSS, said that Ofqual and the Scottish Qualifications Authority should both be probed over the algorithms they used to predict students’ exam results.

Ofqual said it intends to publish the algorithm on results’ day, but the RSS believe this decision undermines the transparency and accountability of the process.

“Our understanding is that all of the qualification bodies have used very similar mechanisms,” Ms Witherspoon said.

“We think the Office for Statistical Regulation could look at the statistical models. Were there alternatives if more data had been asked for? Or given the data they had, did they use the best model?”

She went on: “I certainly think that more transparency at an earlier stage would help bolster public faith in statistics.

“The principal is that more transparency leads to more trustworthy statistics because it means you can have a better discussion. We have been calling since April for Ofqual to be more transparent about the types of model they are considering.”

An Ofqual spokesman said they convened an independent expert advisory group which has met regularly since April and has provided insight and guidance as the algorithm was developed.

It comes as an analysis suggests that nearly 40 per cent of A-level results could be downgraded.

This would mean that when pupils are awarded their A-levels next week, almost 300,000 grades will be lower than teachers’ predictions.

GCSEs are expected to have a similar downgrade rate according to the statistical analysis, which means that close to two millions teachers’ predicted grades will be adjusted downwards.

When students receive their grades this month, the majority will be predicted by a statistical model which takes into account their past performance as well as the last three years’ of exam results at their school.

The analysis was conducted by Huy Duong, the parent of an A-level student and a former medical statistician, who published his workings on a blog.

He explained that he examined Ofqual’s published data to calculate that 39 per cent of grades between A* and D will be lower than the teachers’ predictions.

Ofqual said that results this year will improve overall by two per cent compared to last year, and that if teachers’ predicted grades had been used this would have resulted in grades being i12 per cent higher than last year.

An Ofqual spokesman said: “The arrangements in place this summer are the fairest possible to enable the majority of students to move on to further study or employment as planned.

“Final grades will be calculated using both centre assessment grades and a rank order of students provided by the centre. Centre judgements will then be standardised by exam boards, just as coursework marking is routinely adjusted through moderation, to make sure the standard is consistent between schools and colleges – with adjustments only where they are needed to ensure fairness for students.

“The awarding process has been designed to allow leniency, where this can be done without undermining fairness, and from an early review of the data we expect the majority of grades students receive will be the same as their centre assessment grades, or within one grade.

“Results for students will therefore almost always be broadly in line with centres’ and teachers’ expectations.”