Education Scotland: All you need to know as the nation awaits its Pisa test scores

Influential international rankings will be unveiled by the OECD on Tuesday

Scottish education will be under the spotlight again this week when international Pisa scores are published.

The results of tests taken in more than 80 countries last year will be announced tomorrow, following delays linked to the pandemic.

Significant further declines in Scotland’s scores will be seized upon by opposition politicians, amid growing concerns over a stalled programme of reforms.

Pisa test results are due to be published on Tuesday

However, critics have long questioned the influence of the tests and the way they shape discourse and policy.

Ahead of the announcement, The Scotsman has taken a look at the key questions on Pisa.

What is Pisa?

Pisa is the “programme for international student assessment”. It was conceived in 1997 by the Organisation for Economic Co-operation and Development (OECD), and started in 2000.

Every three years, it tests the skills and knowledge of 15-year-old students in reading, mathematics and science. In 2022, about 690,000 students took part from 81 countries.

Which schools and pupils take part?

Schools are grouped based on previous exam performance and other characteristics, such as whether they are urban or rural.

They are then randomly selected within these groups to take part, producing a sample thought to be representative.

In 2018, a total of 107 schools in Scotland responded, a rate of 90 per cent, alongside 170 in England, 107 in Wales and 75 in Northern Ireland.

Within each school, 40 pupils are randomly selected to take part, although they can refuse, or be excluded by the school.

What do the tests involve?

The last PISA in 2018 focussed on reading, while mathematics, science and “global competence” were more minor areas of assessment.

Computer-based tests were used in most countries. There was a mixture of multiple-choice questions and questions requiring students to construct their own responses.

In 2022, the results of which will be published on Tuesday, the focus was on mathematics.

Most students spent one hour answering questions on maths, with the other hour used to assess reading, science or creative thinking.

They also spent 35 minutes answering a questionnaire to learn about their attitudes to school, their home life and other issues.

Is it important?

Pisa is seen as the key measure of the effectiveness of the secondary school systems countries’ that participate.

The first results, released in 2000, triggered a public outcry in Germany, which was found to be below the OECD average. Germany virtually doubled its spending on education following this “Pisa shock”.

John Jerrim, a professor of education and social statistics at University College London, said: “I think its main use now, particularly thinking about Scotland, is looking at trends over time – so going up or down – in comparison to the rest of the UK.”

Dr Marina Shapira, an associate professor at Stirling University, also said the Pisa data was important.

She said in Scotland there was a focus on judging performance based on attainment levels, which was “unhealthy, counter-educational, and contradicts the goals of Curriculum for Excellence”.

Dr Shapira added: “Using the Pisa data in conjunction with other data about national education system can offer additional invaluable insights about national education, that can be used for informing national educational policies and evaluating education system

performance.”

She said the figures include family and school-level characteristics that are absent from administrative education data, providing “valuable insights into how individual, family, and school-level attributes impact various outcomes for young people.”

How does Scotland perform?

From 2006 to 2012, Scotland scored comfortably above the OECD average in maths, reading and science.

However, scores dropped in 2015 and were closer to the average in the 2018 results, triggering concern that standards were falling.

England has not seen a similar decline. Its pupils recorded the highest scores among the UK nations in each of the subjects in 2018.

A recent study by the Institute for Fiscal Studies (IFS) research group highlighted a “particularly sharp” decline of 23 points in science scores for Scotland after 2012, with maths falling by 17 points. Reading results declined after 2009 but have since returned to 2006 levels.

On overall performance, the IFS said: “Between 2006 and 2018, Scotland goes from being the best-performing UK nation and 17 points above the OECD average, to being behind England, roughly in line with Northern Ireland and 6 points above the OECD average.”

Its report added that “the trends do suggest that something, somewhere is going wrong in Scottish education”.

What will experts be looking for this week?

The most recent Pisa tests were supposed to be taken in 2021 but were pushed back to 2022 because of the pandemic. Experts will be keen to see any impact to scores caused by the disruptions to education during the coronavirus crisis.

Prof Jerrim said: “It is a bit of a free hit this time in some respects, because any country that does badly is just going to turn around and say ‘Covid’.

"So it is not only about whether it goes up or down for Scotland, but how it compares to what other countries are doing, because I would be expecting a lot of countries probably will go down a little bit. It depends about whether it is more or less than elsewhere, I think.”.

He added that the UK conducts the tests around six months later than other northern hemisphere countries, meaning teenagers here had a bit longer to recover from the pandemic.

Are there any problems with Pisa?

There are many criticisms, including its focus on reading, maths and science at the expense of other areas, as well as the way results are interpreted, and the fairness of standardised testing between nations with very different cultures.

There is particular concern about the way it ranks countries, amid fears it can influence Governments into changing their systems in order to obtain a higher ranking.