John Angus, 66, died in hospital after being admitted to the Royal Infirmary on Tuesday, it was confirmed today.
He was an inmate from Hermiston block at HMP Edinburgh, on the western edge of the Capital, and is thought to have suffered from underlying health conditions.
Mr Angus was placed on a lifelong restriction order after being convicted of the indecent assault on a 21-year-old woman in 2010, and breaching a ban which prohibited him from travelling without notifying the police.
He had previously been given an 11-year jail term for tying up and gagging a primary school teacher in Easter Ross and abducting a school cleaner at the same school at knifepoint in 1994.
The Scottish Prison Service (SPS) said it did not commnt on individual prisoners, but said that 36 inmates across eight jails were currently self-isolating after either showing signs of Covid-19 or sharing a cell with someone that had.
Human rights groups have previously called for justice secretary Humza Yousaf to release inmates from Scottish prisons to reduce the risk of coronavirus quickly spreading.
A letter sent last week to the Holyrood minister warned the impossibility of social distancing in prisons put both staff and inmates at increased risk of infection.
Mr Yousaf previously told MSPs there were “robust” plans in place across the prison system to deal with the pandemic, with any prisoner who displays symptoms told to self-isolate.
A Scottish Government spokeswoman said: “Our priority and that of the Scottish Prison Service will always be the health and wellbeing of every person living and working in Scotland’s prisons, including inmates and the prison officers and healthcare workers who care for them
“Clearly the loss of any life in prison custody is regrettable and our thoughts are with those affected by this death.
“As with any death of a prisoner, the matter has been reported to the Procurator Fiscal and a Fatal Accident Inquiry will be held in due course.”