The Lib Dem MP for Orkney and Shetland made his comments during a Westminster Hall debate he called on the future of Human Rights in the UK.
The Conservative government has made it clear that it wants to replace the Human Rights Act with a British Bill of Rights after the legislation has been used to prevent the deportation of suspected terrorists and so called hate preachers as well as test the rights of prisoners to vote.
Justice Secretary Michael Gove has suggested that Scotland could be exempt from the change because of Scottish legal system being devolved to Holyrood.
But in the debate Mr Carmichael noted: “The Human Rights Act is hard-wired into the devolution settlements of Scotland, Wales and Northern Ireland.”
He said that those who had campaigned to keep Scotland as part of the UK in the referendum last year “had not envisaged separate human rights laws” for different parts of the UK.
He added that this “will undermine the UK. Human rights need to be universal across the whole country.”
He pointed out that the Human Right Act was “at the heart of the Good Friday Agreement” which brought peace in Northern Ireland.
He said that parties involved “took a leap of faith” because of the protection of their human rights.
He added: “The peace process remains a very delicate animal. We should never takes its continuation for granted.”