The move will come as a significant blow to Nicola Sturgeon, who had set out plans to hold a second vote on independence on October 19 next year.
The First Minister previously said the SNP would fight the next general election as a “de facto” referendum if the Supreme Court ruled against them.
However, it is not clear how this would work in practice.
A panel of five justices delivered its decision at 9.45am on Wednesday.
The Scottish Government’s top law officer, the Lord Advocate, asked the court to rule on whether Holyrood has competence to legislate for the vote.
During a two-day hearing in October, Dorothy Bain KC said resolving the legality of the proposed Scottish Independence Referendum Bill is a “critically important question”.
She said she would not be able to “clear” the introduction of the Bill herself without the court’s ruling. Ms Bain said the issue had been “festering” since the early days of devolution.
The UK Government’s representative, Sir James Eadie KC, argued the Bill “squarely and directly” relates to a matter reserved to Westminster – the union between Scotland and England.
He also argued the Bill is at too early a stage for the court to issue a ruling on, saying the question of the Bill’s competence should not be “farmed out” to the Supreme Court.
The court addressed whether it would determine the “reference” from the Lord Advocate, before turning to the question of whether the Bill relates to a reserved issue.
Supreme Court president Lord Reed said the court decided it would make a ruling and that this was in the public interest.
He said the court had unanimously concluded the proposed Bill “does relate to reserved matters”, adding: “Accordingly, in the absence of any modification of the definition of reserved matters, by an order in council under section 30 of the Scotland Act or otherwise, the Scottish Parliament does not have the power to legislate for a referendum on Scottish independence.”
Scottish Labour leader Anas Sarwar said the focus must now turn to “the problems facing our country”.
He said: "It was right for the Scottish Government to seek legal clarity on this question. The Supreme Court's answer was clear and I thank them for their speedy work in this case.
"We must now focus on the problems facing our country, from rising bills to the crisis in our NHS.
“There is not a majority in Scotland for a referendum or independence, neither is there a majority for the status quo. One thing is clear, there is a majority in Scotland and across the UK for change.
"A Labour government will deliver the change that Scotland and the whole of the UK needs. Every part of the UK deserves better than more misery and decline under the Tories.”
Scottish Liberal Democrat leader Alex Cole-Hamilton said: “This is an embarrassing defeat for the First Minister. Scores of legal experts warned that the law is clear and that this case was a complete waste of Supreme Court time and taxpayers’ money, but the Scottish Government would not listen.
“It has been a terrible use of funds at a time when every penny should be squeezed to help people through the cost of living crisis. Breaking up the UK simply isn’t a priority for people opening their bills with dread or struggling to get the treatment they need.
“I have lost count of how many times the First Minister has launched independence campaigns, each with less energy and momentum than the last. While Nicola Sturgeon goes through the motions people wait days for an ambulance, months for NHS treatment and years for lifeline ferries.
“It is time the SNP Government finally focus on what really matters. What Scotland needs now is new hope not old divisions.”