A court was told that William Melvin's 36-year-old victim had had cerebral palsy since birth. She has a low IQ and cannot walk unaided.
However, said advocate depute Alison Di Rollo, prosecuting, she knew the difference between rape and consent and had resisted Melvin when he forced himself on her, telling him "No".
Melvin, 50, of Watson Crescent, Polwarth, was remanded in custody after he admitted rape and faces sentence next month at the High Court in Edinburgh.
In court yesterday, judge Lord Doherty was told that after the attack on 28 June last year the woman had to leave her specially adapted home in Edinburgh – but liked her new house better.
Ms Di Rollo said that Melvin and his wife had befriended the woman, regularly checking that she was OK. Melvin would also watch TV in her house – particularly football matches.
During the evening of 28 June he told her he was going through to the bedroom because there was a larger television set there.
The woman followed him and sat on the bed.
"He told her to remove her top to which she replied 'No thank you,'" said Ms Di Rollo.
She continued to say no as Melvin took off her top, pushed her back on the bed and pulled off her trousers, leaving her naked.
Melvin then took off some of his own clothes and had sex with her, ignoring her protests and attempts to push him off.
The court heard that the offence came to light some days later because the woman phoned her mother, asking her to tell Melvin's wife that she didn't want him in her house because he had been "touching her".
When questioned by police, Melvin tried to deny having sex with the woman but changed his story when told about the results of a medical examination. He then said she had been a willing partner, but finally admitted he had "carried on regardless".
The court also heard that when a psychiatrist interviewed the woman, she told him: "I saw Billy on the bus about a month ago and he made me feel very dirty. What he did was wrong.
"I said no and he did it anyway."