A motion calling for the tests for four and five-year-olds to be axed was narrowly voted through by 63 to 61.
An amendment from the Scottish Government, describing the assessments as a “key tool” that could be “especially valuable” in closing the attainment gap, was defeated by 61 votes to 63.
Tory MSP Liz Smith had earlier led calls in Holyrood for the Scottish Government to scrap national assessments for P1 pupils.
The Conservative education spokeswoman said while her party had previously backed the use of the tests, they had been wrong to do so.
The Scottish Government appears to be facing defeat in today’s vote on the use of national assessments for P1 pupils.
MSPs at the Scottish Parliament are expected to back the Tory motion calling on ministers to halt the use of tests for the youngest students.
“We made a mistake about P1,” Ms Smith said at today’s debate.
She told education secretary John Swinney: “I think the time has come because of the evidence that has been piling up over the last two years, to call a halt, reconsider the evidence that is very much before us and evaluate what is the best way of progressing P1 pupils.”
Ms Smith went on to say she had been contacted by a primary teacher who said “there have been some very good questions in the new test, but others have undoubtedly created problems”, adding these had been the “catalyst for the current complaints”.
The teacher also complained that “some questions are too long, taking up too much time and there is an overemphasis on data handling within each assessment”.
Mr Swinney explained the tests had been introduced after the OECD reviewed the Scottish education system in 2015.
He told MSPs: “We sought external independent opinion, which said that we did not have enough information about learning outcomes and progress. So we have put in place the standardised assessments.”
The education secretary continued: “Assessment is an essential part of a good education system, it is an integral part of effective teaching and learning.”
He stressed there was “nothing new” about testing youngsters in their first year of school.
Mr Swinney said: “The vast majority – 29 out of 32 local authorities – were using some form of standardised assessment before the national scheme was introduced and, crucially, the majority were not just assessing P1 children, they were assessing P1 children twice during the year.”
He went on: “It is absolutely vital to get as much information as possible on children’s achievement as early as possible.”
Holyrood’s four opposition parties are all against national standardised assessments for four and five-year-olds.
But while the minority SNP administration is facing defeat on the issue, the result of the vote is not binding on the government.
Mr Swinney continues to insist the tests are a “valuable tool for teachers to identify the next steps in a child’s learning” and that the “additional information they provide is particularly useful in the early years if we are to continue to close the attainment gap”.
He demonstrated the tests to both MSPs and the media on Tuesday in a bid to ensure an “accurate and informed debate”.
But opposition leaders say if the Scottish Government is defeated, Mr Swinney should respect the wishes of the Parliament and call a halt to the tests.
Conservatives submitted the motion, which questions if the assessments are “in line with the play-based learning philosophy” of early years education, after some teachers reported P1 pupils had been reduced to tears.