Why heads should roll at Education Scotland over AI ‘hallucinations’ in new report – Cameron Wyllie
Former head teacher Cameron Wyllie is incensed after it emerges AI was used to compile a report about Scottish education with some deeply troubling consequences
What is ‘Education Scotland’ for? How many people work for it? How much does it cost? And finally… does it actually achieve anything worthwhile, anything that might ease the burden of Scotland’s put-upon teaching profession or improve the success and potential of its young people?
Well, you may remember that, after the publication of Scotland’s poor Pisa results in the late summer, we were assured by Education Secretary Màiri McAllan that Scotland was in the midst of a huge review of the curriculum, a tacit admission that something radical needed to be done about ‘Curriculum for Excellence’ which – with its discredited teaching methodologies and failure to make clear what it is that our young people actually need to be taught (say reading, say maths) – has reached the end of its shelf life.
Now, the process for reviewing a national curriculum is, inevitably, a laborious one, and, perfectly reasonably, it needs to be done in stages. The SNP government, of course, excels at committees, and working parties, and preliminary reports, usually stuffed with the language of fanciful theory, and involving caveats about ‘inclusion’, ‘equality’ and ‘well-being’.
Some very odd mistakes
So, the teaching profession, sick of the lack of action, probably took little interest in the first set of reports to emanate from Education Scotland a few months ago, the so-called ‘Curriculum Background and Evidence Papers’, produced over the past two years by groups of teachers seconded to Education Scotland for this very task.
But, fortunately for Education Scotland, Brodie Runciman, a lecturer in teacher education at the University of Edinburgh, did take an interest. Runciman is a PE specialist so he started with the ‘Health and Well-Being’ paper. Remember, readers, these papers were two years in the making, and were to provide evidence for curriculum changes crucial to the education of the next generations of young Scots in our schools.
Mr Runciman is a scrupulous fellow and, being fairly young, has a handle on the development of artificial intelligence (AI) – old fellas like me aren’t clear whether that’s for curing cancer or inventing robots which will enslave us all.
He immediately noticed that the Health and Well-Being report had been generated with AI, and he also noticed that it contained some very odd mistakes. And then he started to look at the papers about all the other subjects and, with the shining exception of the maths document, found that it seemed all the documents had huge AI inputs.
READ MORE: AI is a super-intelligent psychopath that could kill us all. Here's what Scotland needs to do
Ghosts and hallucinations
Then he found something altogether more troubling. Obviously, to support their ideas for our new curriculum, these teams of teachers, under the watchful eye of Education Scotland, had done masses of research – and, indeed, the papers were littered with references to academic publications.
So Runciman, tenacious to a fault, started investigating these links, and discovered that many of them led to pieces of research, to scholarly articles and books, readily quoted in the papers, which simply did not exist.
Now, as I say, I’ve never even done a PowerPoint presentation, so the vagaries of AI are a new volume in the Wyllie library, previously unopened. Apparently, however, it is not uncommon for less-developed AI programs to produce what are called ‘hallucinated’ or ‘ghost’ references to fake academic papers and books, and this, it would seem, is exactly what has happened here. These papers – supposedly the vital cornerstones of major curricular change in Scotland – contain hosts of stuff made up by machines.
READ MORE: Why I won't let Scotland become so afraid of AI that it loses £23 billion boost to GDP – Alyn Smith
Who published these papers?
So Runciman pointed all this out to Education Scotland who seemed at first to take a rather dismissive view of his concerns, a view which changed significantly when the press took an interest. The papers, still online, have been – to some extent – improved and now come with a caveat (excluding the maths one) that says that AI was used to edit them. Well, it looks to my unpractised eye, that AI was used to pretty much create them.
This all begs some fairly obvious questions. Firstly, what exactly were all these teachers – likely to be, I imagine, fairly senior teachers, seconded away from actual teaching (like in classrooms with young people) – doing for two years, and how in Heaven’s name could they have allowed these flimsy, erroneous but supposedly very important documents to be made public?
But much more importantly, who was scrutinising this work? Which particular apparatchiks in the cosy world of Education Scotland – safe from the day-to-day, increasingly stressful and sometimes dangerous world of today’s Scottish schools – were in charge of this process? And who actually decided to press the button that released these papers online?
A silver lining to a scandal
Yes, they can say they were ‘draft discussion documents’ but who at Education Scotland knew that they contained all these made-up references? And what is going to happen to these senior figures now?
In almost any other area of private or public policy-making, they would be out of a job. Their heads would roll or, at least, they would have been sent back to do a real job.
But this is Scottish education in 2026, and this monolith will look after its own, the folk of the quango and the committee, of the document and the working group. It will be seen as a mere detail as the old elephant of Scottish curricular reform stumbles on.
It’s a disgrace. The only silver lining to this ridiculous, costly story is that it shows how one clever man who actually cares about Scottish education, can make a difference. Well done, Brodie Runciman!
Cameron Wyllie is a retired headmaster. He writes a blog called A House in Joppa and hosts the podcast, Two Heads are Better than One, with Rod Grant
Comments
Want to join the conversation? Please or to comment on this article.