David Mundell said leaving the EU will fundamentally change the devolved settlement in Scotland and elsewhere in the UK.
That change could be equivalent in scale to those brought about by the Calman and Smith commissions in Scotland, he signalled.
The Conservative minister said he now wants to encourage “debate and discussion” on the issue.
“One of the most significant differences that we could feel in Scotland post-Brexit is in the changes to the devolution settlement,” he said.
Mr Mundell successfully steered the 2016 Scotland Act, granting Holyrood new powers over tax and welfare, through the UK Parliament.
He has now indicated, in a Sunday Times Scotland interview, that a raft of further powers could be transferred to Holyrood as a result of the Brexit vote, and stressed that no functions would be “re-reserved to Westminster”.
He told the newspaper: “The devolution settlement could be affected to a similar extent as that proposed by Calman and Smith. This is a highly complex area and a huge body of work needs to be done to look at this to determine what might come back and how this should happen. So far people have not engaged in it to any great extent.
“We need to work out how the UK will work best after powers are repatriated to the United Kingdom, and which powers will be repatriated to Scotland. Whatever the circumstances, no powers will be re-reserved to Westminster.”
He later told BBC Scotland “a number of powers and responsibilities” would have to return to the UK or devolved administrations.
Speaking on the Sunday Politics Scotland programme, he said: “What I think hasn’t been fully understood and is only just beginning to be debated - which is what I want to encourage - is that by leaving the EU that will have a fundamental change on the devolved settlement here in Scotland and indeed elsewhere in the United Kingdom.
“These settlements were predicated on the basis that the United Kingdom was in the EU. Therefore there are a number of powers and responsibilities which are currently exercised by the EU which will have to return to the United Kingdom or to Scotland and the other devolved administrations.”
Asked for examples of powers which would return, he told the broadcaster: “For example, and I think self-evidently, agriculture and fisheries are two of the issues currently exercised at European level.
“Both the NFU in Scotland and the Scottish Fishermen’s Federation are coming forward with their views as to how these sort of powers should be taken forward post leaving the EU. But there will be significant powers in the area of the environment and there are also powers in relation to the criminal justice system as well.”