Algorithmic Bias and Fairness in School Assessment

The use of algorithms and AI to make judgements about students, from algorithmic grading to AI-writing detection, raises ethical, social and professional considerations about bias, fairness for different groups of students, and the role of teacher judgement in assessment.

Zhikai Huang and Jiahe Huangfu. Teachmeet research poster, Teaching and Learning with Digital Technologies, University of Technology Sydney, 2026.

The 2020 A-level grading algorithm

Bar chart. Change in the share of A* and A grades at A level in England, 2019 to 2020, using algorithm-calculated grades: independent schools up 4.7 percentage points; comprehensives and academies up 2.0; sixth-form, FE and tertiary colleges up 0.3.
Tap the chart to enlarge it.

What the chart shows

When exams in England were cancelled in 2020, an algorithm calculated A-level grades. Compared with 2019, the share of A* and A grades rose by 4.7 percentage points at independent schools, 2.0 at comprehensives and academies, and 0.3 at sixth-form, FE and tertiary colleges.

Why this may have happened

Larger classes were standardised against each school's results in previous years, so a student's grade depended heavily on how their school had done before. Classes of 15 or fewer relied more on teachers' grades. Small classes are more common in independent schools, and the Education Policy Institute noted the larger gains there may be linked to this rule.

Read with care

There were no exams in 2020, so no one can say which grades were "right". The chart shows who gained most, not proof that any group was treated unfairly. Ofqual's 2021 evaluation found no systematic disadvantage by students' socio-economic background or ethnicity, but its analysis left out classes of 15 or fewer.

Read the data source (Schools Week, 2020) Read Ofqual's 2021 evaluation Read Ofqual's 2020 results guide

AI-writing detectors

Dot chart. Average share of human-written essays wrongly flagged as AI across 7 GPT detectors. Non-native English writers, 91 TOEFL essays: 61 percent; after ChatGPT enriched only the vocabulary: 12 percent. US eighth-grade students, 88 essays: 5 percent; after ChatGPT simplified only the vocabulary: 57 percent.
Tap the chart to enlarge it.

What the chart shows

Researchers ran seven AI-writing detectors on essays written by people: 91 TOEFL essays by non-native English writers and 88 essays by US eighth-grade students. On average, the detectors flagged 61% of the TOEFL essays as AI-written, compared with 5% of the US essays.

The researchers then used ChatGPT to change only the vocabulary. With richer vocabulary, the TOEFL essays were flagged 12% of the time. With simpler vocabulary, the US essays were flagged 57% of the time.

Why this may happen

Many detectors judge how predictable the word choices are, a measure called perplexity. AI tends to choose the most likely words, but so do writers with a narrower vocabulary. The detectors were responding to the words used, not to who wrote them.

Read with care

The samples were small, the TOEFL essays were short, and the detectors were those available in 2023; Turnitin's detector was not tested. Turnitin's own testing found no significant difference between English language learners and native speakers for texts of 300 words or more, but a larger difference for shorter texts.

Read the study (Liang et al., 2023, open access) Read Turnitin's own testing (2023)

References

  1. Education Policy Institute. (2020). Analysis: A level results day 2020. https://dera.ioe.ac.uk/id/eprint/36185/
  2. Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(7), Article 100779. https://doi.org/10.1016/j.patter.2023.100779
  3. Ofqual. (2020, August 13). Guide to AS and A level results for England, 2020. GOV.UK. https://www.gov.uk/government/news/guide-to-as-and-a-level-results-for-england-2020
  4. Ofqual. (2021). Grading gaps in summer 2020: Who was affected by differences between centre assessment grades and calculated grades? GOV.UK. https://www.gov.uk/government/publications/evaluation-of-centre-assessment-grades-and-grading-gaps-in-summer-2020
  5. Schools Week. (2020, August 13). A-level results 2020: Private schools see biggest boost in top grades. https://schoolsweek.co.uk/a-level-results-2020-private-schools-see-biggest-boost-in-top-grades
  6. Turnitin. (2023, October 26). New research: Turnitin's AI detector shows no statistically significant bias against English language learners. https://www.turnitin.ca/blog/new-research-turnitin-s-ai-detector-shows-no-statistically-significant-bias-against-english-language-learners