While the Prime Minister and others have highlighted opportunities that could come from leaving the European Union (EU), Mr Russell insisted that "there is no pot of gold" that will come as a result of this.
Concerns are growing about the "inability" of Theresa May and her government to "measure up to the task they have" in negotiating the country's exit deal, he said.
And with a prolonged standoff between Holyrood and Westminster over where powers should go when they come back from Brussels, Mr Russell insisted there needs to be "less rhetoric from the Prime Minister, more work on making sure everybody sits around the table and agrees".
If no agreement is reached between the UK and the devolved administrations he warned the "constitutional crisis simply gets worse, gets deeper".
With a year to go until the UK formally leaves the EU the Scottish Government published a document highlighting the impact leaving could have on a variety of people across the county.
The Protecting What Matters report concludes leaving the EU will have "huge implications" with Scottish ministers pledging to "give a voice to individuals all of whom have so much at stake" from Brexit.To launch the document Mr Russell met with some of those who spoke out about their fears, including Piotr Tlok from Poland, who said unanswered questions about what will happen to EU citizens had sparked concerns that people could be deported.
The 29-year-old recruitment consultant said: "What we're experiencing at the moment is a huge, huge question mark, I don't know what my status is, what my status will be in a year, I don't know what I should do, where I should go, what authorities I should approach to ask. That's the biggest problem at the moment."
He added: "I would like to know what my status is and what I need to do to stay in Scotland and to have my rights provided, that is the biggest question at the moment."
Asked about his biggest concern, he said: "If I will be deported or not. The amount of unanswered questions we have, some people actually consider deportation might happen. I believe it's quite extreme but you never know."
Mr Russell said: "What we're doing is listening to people who are affected by Brexit and we're publishing their views today to mark the first anniversary of the Article 50 letter and a year to go before it takes effect."
Worries have "simply increased" since the referendum in 2016, the Brexit Minister said, adding that "people are very unsure and uncertain about what is taking place".
He hit out "Far from things clarifying in the last 12 months we've got even more uncertainty. If you are EU citizen who is living here then you are unsure about what lies ahead, if you are somebody who is involved in farming or fishing you are unsure about what lies ahead, if you work in a university you are unsure about what lies ahead.
"The issue is people don't know what is happening and with six months left of negotiating time, there are huge issues to be addressed that are simply not being addressed.
"So I think you have to look at this government and see it is letting everybody down, but it is also talking often nonsense about Brexit."
He also spoke to Ana Rodriguez, 55, and originally from Spain, but who has lived in the UK for 35 years.
She has been in Scotland for 25 years, but said that after Brexit for the first time she had been told "to go to my own country".
Speaking about the referendum she said: "In Scotland I've always felt very much accepted and very integrated into life here so I felt betrayed by the result."