The 66-year-old left Central Park earlier this week after just 16 games in charge.
The former Hearts and Hibs assistant took over from Gary Locke after the former boss left to rejoin Hearts in a non-coaching role.
He has since admitted his managerial stint with the League Two club was the most difficult challenge he has encountered in his coaching career.
He said: “I wasn’t enjoying it. How can you enjoy it when you aren’t winning? I have been brought up on the fact that football is about winning and that is where the enjoyment comes from.
“It was becoming something that I wasn’t looking forward to. It was the right time to go as it gives the club plenty time to find someone else to come in.
“We have only played 10 league matches, so there is plenty games left yet. They need someone different, maybe someone with a bit of luck.
“They deserve a bit of luck, do Cowdenbeath. The last four seasons for them have been a challenge so I hope they get the bit of fortune they deserve.
“There is no doubt about it, this is the hardest job I have ever had.
“I have got to say, however, that there are a lot of good people there – I received a lot of help and support.
“I have nothing against the people against there at all. The board, particularly Donald Findlay and David Allan were excellent. I have absolutely no complaints.”
The Central Park side were playing in the second tier, rubbing shoulders alongside the likes of Hearts, Hibs and Rangers, as recently as 2015. However, they had to survive a play-off with East Kilbride to avoid three successive relegations.
It will be a big challenge for Brown’s predecessor, with the successful candidate becoming the Blue Brazil’s fifth manager in just over two seasons – and their fourth in 2017!
Brown added: “It is certainly a big challenge. People asked me when I took over in the summer if I was on a suicide mission!
“I think the more experience the squad gets, it will only go from strength to strength. I am sure that the oldest outfield player we had playing on Saturday was Josh Morris who is 26.
“There are definitely one or two good players there so the more exposure these boys get, then their confidence will hopefully rise and then they can get a run going.
“It must be a difficult time to be a supporter of Cowdenbeath considering what has happened. There is nothing else for it – if that is your team, you just need to keep going.
“I appreciate it must be a hard shift for them at the moment but I am positive it will turn round.”