Mr McDonnell said he hoped the deal with the EU could still be re-negotiated, but failing that or a general election, a second public vote would be needed to break the impasse.
Prime Minister Theresa May is expected to lose a vote on the withdrawal deal in the House of Commons on Tuesday.
Speaking to journalists in Glasgow, Mr McDonnell said he and Labour leader Jeremy Corbyn had met First Minister Nicola Sturgeon to discuss their joint opposition to Mrs May’s deal.
But he said his party remained opposed to the “distraction” of a second independence referendum and would consider including a specific commitment in its next election manifesto.
Asked if he agreed with Unite general secretary Len McCluskey that a second referendum may be seen as a betrayal by voters, Mr McDonnell said: “No, if we get to a situation where we’ve tried everything.
“If we can’t get [an agreement] we need a general election because we can then change the team that will do the negotiations.
“If we can’t get [a General Election] I think people will recognise that we have no other option but to consider another public vote.
“People will respect us for doing our best to implement the spirit of the referendum, but we’ve got to resolve this issue – we can’t go on like this.”
Asked if there was likely to be a remain option on the ballot paper in the event of a second vote, he said: “I think it’s inevitable and if it was, I would vote remain.”
He cited the example of the Lisbon Treaty signed by EU states in 2007 as an example of how a re-negotiated EU withdrawal deal could be done quickly.
Mr McDonnell said he believed Parliament could re-open negotiations and reach a “speedy conclusion” with the EU based on a deal which would see the UK remain in a permanent customs union and retain a “close and collaborative” relationship with the single market.
Asked if Labour would come to a supply and demand agreement with the SNP should there be a minority government after the next general election, Mr McDonnell said: “Jeremy and I met with Nicola Sturgeon two weeks ago. It was purely about the Brexit issue and we had a discussion where we shared our view about opposing a no-deal and opposing Theresa May’s deal.
“If we didn’t get a significant majority, we would govern as a minority government. You know our position around future referendums, which we oppose, and that’s what we will continue to do.
“We want a united country. Our view is that it’s a complete distraction.”