The First Minister said his personal favourite Scottish tune was Highland Cathedral, but he did not believe it was right as a national anthem.
He said Flower of Scotland worked well at Murrayfield but would not be as good for individual medal ceremonies, while the reverse was true for Scotland the Brave.
Mr McConnell intervened in the controversy over Scotland's national anthem following increasing public disquiet over Scotland the Brave, which has been played to a worldwide audience at medal ceremonies at the Commonwealth Games in Melbourne.
Scotland's success at the games has reopened the argument about a national anthem, and yesterday Mr McConnell said there should be a proper national debate about it, although he stressed that it was not for politicians to lead it.
However, he added: "I think it's a very interesting debate. There are those who say Auld Lang Syne, but Auld Lang Syne is a song for the end of an evening rather than the beginning of the evening, so I'm not sure that would work.
"If you're trying to choose a national anthem then it has to work in a whole variety of different settings, and I think that's the challenge.
"It has to work when the flag is going up the pole at the Games, it has to work when the crowd is singing it at a big sporting event, it has to work in the school, perhaps, if kids are going to sing it at morning assembly.
"It has to work in a whole variety of settings and I think all of the various options that are put forward in Scotland - Highland Cathedral, Auld Lang Syne, Scots Wha Hae, Scotland the Brave, Flower of Scotland - all work in different circumstances.
"But I can't think of any of them that actually work in all of those circumstances.
"So I think there should be a debate about this, but I think politicians should be hesitant about having firm views."
The SNP has a party policy of Scots Wha Hae as a national anthem, and this is sung at each party conference, but Nicola Sturgeon, the SNP leader, was reluctant to ally herself to any one song and instead called for parliament to lead the debate and take a decision.
She said: "Many people are confused as to why organisations like the Scottish Football Association and the Scottish Rugby Union use Flower of Scotland as our national anthem, while the Commonwealth Games committee has chosen Scotland the Brave.
"Both are great Scottish songs, but a growing number of people think it's time to adopt an official national anthem for all Scots."
She added: "The Scottish Parliament has the power to decide on the Scottish national anthem, and so should take the lead now in the national debate on the issue. Our national song should be an anthem of the people ...
it should be a symbol of pride."