Mr MacDougall, 60, lost his battle with cancer at the Victoria Hospital in Kirkcaldy, Fife.
A Labour Party spokesman said: "All our thoughts are with his family at this time."
The Prime Minister's position came under intense pressure from inside the party after Labour lost the previously safe seat of Glasgow East to the SNP last month.
The constituency of Glenrothes is also seen as a safe Labour seat. Mr MacDougall won with a 10,664 majority at the 2005 election.
But, given the SNP's astonishing victory in Glasgow East, Labour will face an incredibly tough challenge to hang on to Glenrothes when this by-election is called when the Commons returns in September.
Another loss of a safe Scottish Labour seat could bring such pressure to bear on the Prime Minister that he has no option but to stand down, particularly as the constituency is in Fife, Mr Brown's home territory and is close to the Prime Minister's own constituency of Kirkcaldy and Cowdenbeath.
Mr MacDougall was a close friend and political ally of the Prime Minister, who represents the neighbouring Kirkcaldy and Cowdenbeath constituency. Mr Brown visited him in hospital on Monday, and spent time chatting to him.
The SNP would need a 14 per cent swing to take the seat and inflict another hugely damaging blow to Labour.
The last by-election fought in Fife, in February 2006, saw the Liberal Democrats seize power in a strong Labour seat.
Former oil rig construction yard worker Mr MacDougall underwent surgery at Guy's Hospital in London last year to try to treat the asbestos-related cancer mesothelioma.
He had served Fife for more than 20 years, after being elected to Fife Regional Council in 1982.
Mr MacDougall later became the leader of the council and also served as the director of Glenrothes Development Corporation and Fife Enterprise before being elected to the House of Commons.
He was elected as the MP for Central Fife in 2001.
Boundary changes meant his constituency later became Glenrothes, a seat which he won with a majority of 10,664 at the last General Election in 2005.
He is survived by his mother Barbara, his wife Cathy and two grown-up children, Scott and Julie.
Helen Eadie, the Labour MSP for Dunfermline East, paid tribute to Mr MacDougall.
She said: "It's a very sad loss and to learn of that this morning makes me deeply saddened for John's family.
"His death means the loss of a good friend and a good colleague, and it is very upsetting for all of us here in Fife that we have lost someone who has dedicated all of his adult life to serving the Labour Party."
Ms Eadie had known Mr MacDougall since 1986, when she served with him on the local council, where she was the deputy leader and he was the leader.
She went on: "He was someone who I grew to both like and admire over the years, because he was so committed to what he was doing."
And she added: "John had above all a caring attitude, but he also had a very good sense of humour and that sense of humour always helped to cut through many difficult political situations.
"I always admire people who have got the ability for humour, and John certainly did have that in abundance."
She described him as a dedicated and caring politician, whose door was always open to people.
"If he could possibly be helpful, he would," Ms Eadie said.