William Hardie: It’s hard to get a clear picture of the Scottish school system’s performance

The performance of ­Scotland’s school education system has recently been under intense scrutiny. In December the OECD published its Programme for International ­Student Assessment (PISA) results and the Scottish Government its Achievement of Curriculum for Excellence (CfE) Levels’ statistics.

William Hardie, Royal Society of Edinburgh Policy Advice Manager and Secretary to its Education Committee

But what do they tell us about the performance of our school system?

Undertaken every three years, PISA evaluates the performance of randomly sampled 15-year-old students from around the world in reading, mathematics and science. It therefore enables a comparative assessment to be made of Scotland’s ­performance relative to other countries. The Scottish Government decided in 2010 to withdraw Scotland from the other two widely regarded international comparator studies in which we used to participate – TIMSS and PIRLS.

The Government may come to rue this decision since it serves to increase the attention on PISA, now the only international comparator survey of education in which ­Scotland participates.

The recently published results make for some uncomfortable reading as far as Scotland is concerned, showing a continuing decline in Scotland’s performance in science and to a lesser extent, mathematics. The upturn in the reading results is welcome; however, it merely regains ground lost over the previous decade. PISA is undoubtedly a prominent and influential benchmark of the performance of Scotland’s school system. It is, however, not without its critics. They are ­concerned that it encourages assessment-driven approaches and the ­creation of league tables. PISA continues to evolve. The OECD is planning changes that will place more emphasis on 21st century transferable skills, including creativity.

The Royal Society of Edinburgh (RSE), Scotland’s National Academy, supports this direction of travel. We are taking forward activities to support interdisciplinary learning, an underdeveloped yet crucial and distinctive aspect of CfE that encourages these skills. PISA is of limited value when it comes to evaluating the impact of CfE since it deliberately does not examine the ­mastery of specific school curricula and ­samples only 15-year-olds, whereas the CfE covers the curriculum from age three to 18.

So, how do we assess the impact of CfE across the school system?

Since 2016, the Scottish Government has published annual reports on the extent to which school pupils have achieved the expected CfE ­levels relevant to their stage in reading, writing, listening and talking and numeracy at P1, P4, P7 and S3. The statistics are based on teachers’ ­professional judgment and have up until this year been labelled as experimental. However, there continue to be question marks over the reliability of the data since evidence has shown that teachers can be too optimistic about their own pupils’ attainment.

In 2017, the Scottish National Standardised Assessment (SNSA) was introduced. It provides diagnostic information at the level of the individual learner that can support teachers’ professional judgment. It replaced the Scottish Survey of ­Literacy and Numeracy (SSLN), a sample ­survey which collected data from both ­learners and teachers.

The SSLN provided a system level analysis of attainment and was therefore able to help evaluate the performance of the school system. The respective strengths of the SNSA and the SSLN mean that they could complement one another very well. They served entirely different ­purposes and the RSE has called for a sample survey akin to the SSLN to be ­reinstated. The most robust statistical research in education requires a series of ­surveys over time, and ­surveys that follow pupils through education ­longitudinally.

The discontinuation of, and withdrawal from, several previous surveys coupled with the introduction of newer ones means that it has been very difficult to develop a coherent picture of the performance of ­Scotland’s school system over time. Having ­contributed to the development of Scotland’s Education Research Strategy, the RSE would be pleased to work with Scottish Government in support of the reinstatement of a survey akin to the SSLN.

William Hardie, Royal Society of Edinburgh policy advice manager and secretary to its Education Committee