Speaking yesterday, he also appeared to signal his opposition to those campaigning against the construction of an Islamic centre near the site of the 11 September 2001 attacks in New York.
Mr Bush, who initially kept a low-profile after leaving the White House, has been giving a series of high-profile interviews to promote his newly released memoirs, Decision Points.
In a live interview with Matt Lauer on NBC's Today show yesterday, Mr Bush said his administration did recognise a looming problem with Fannie Mae and Freddie Mac and that the US mortgage giants - which were controlled by the Federal government - "were making risky investments".
"Therefore, I called for the regulation of those two entities but was thwarted at every turn by powerful forces in Capitol Hill," Bush said.
"No question the housing bubble was fuelled by government policy and that is the result of people in Congress refusing to regulate Fannie and Freddie. So my conscience is clear when it came time to recognise an impending problem."
In 2008, Mr Bush signed into law a sweeping rescue package in part aimed at stabilising the two large mortgage finance companies and increasing oversight of them. His administration for years had advocated a smaller role for the firms, saying their management of trillions in assets placed too much risk in the US financial system.
Speaking of the furore over what its opponents have termed the "Ground Zero Mosque" in New York, Mr Bush said: "I think most Americans welcome freedom of religion and honour religions. I truly do."
He refused to be pressed further on the topic, answering: "The problem with the arena today is a few loud voices can dominate the discussion and I don't intend to be one of the voices in the discussion."
Mr Bush is thought to disapprove of former Alaska governor Sarah Palin and the wider Tea Party movement within the Republican party.Ms Palin and other right-wingers such as Rush Limbaugh have been vocal in their opposition to the Islamic centre plans.
The former president also discussed his angst at being called a racist by hip-hop superstar Kanye West.
In his book, Mr Bush wrote that it was a low point in his presidency when West declared at a fundraiser to benefit Hurricane Katrina victims that "George Bush doesn't care about black people."
Mr Bush was shown an interview taped this week with West, in which the musician said he would like to tell Mr Bush that he was speaking in a moment of frustration.
"I didn't have the grounds to call him a racist," West said.
"I believe that in a situation of high emotion like that we as human beings don't always choose the right words."
Mr Bush said he appreciated the remarks and forgave West.
"I'm not a hater," he said. "I don't hate Kanye West. I was talking about an environment in which people were willing to say things that hurt.
"Nobody wants to be called a racist if in your heart you believe in equality of races."
Mr Bush also joked tax cuts now being tackled on Capitol Hill would probably have a better chance of passing if they were not referred to by his name.
"It's too bad they call them the Bush tax cuts. They might have a better chance of being extended if they were the Lauer tax cuts," he said, using the interviewer's last name.
During the interview, Mr Bush also said that he does not plan to make a habit of staying in the limelight.
"After selling this book I'm heading back underground," he said.