Ms Rayner’s attack, delivered at a reception for activists at the Labour Party conference, has outraged members of the Government, with one minister accusing her of “talking crap”.
But Ms Rayner said she would only apologise if Boris Johnson said sorry for past comments he made “that are homophobic, that are racist, that are misogynistic”.
Labour leader Sir Keir said he would speak to his deputy but it was a matter for her whether she apologised.
“Angela and I take different approaches and that’s not language that I would use,” he told BBC One’s The Andrew Marr Show.
Ms Rayner said her comments, made in a “post-watershed” reception, were an attempt to get across the “anger and frustration” felt about Mr Johnson and the Cabinet.
“Anyone who leaves children hungry during a pandemic and can give billions of pounds to their mates on WhatsApp, I think that was pretty scummy,” she told Sky News.
The Ashton-under-Lyne MP said her attack was made in the “street language” of her northern working class roots.
In an apparent reference to Mr Johnson’s history of outspoken comments – including comparing burka-wearing Muslim women to “letter boxes” and describing gay men as “tank-topped bum boys” – Ms Rayner said she would only apologise if the Prime Minister also said sorry.
“I’m saying the Prime Minister has said those things and acted in that way. If the Prime Minister wants to apologise and remove himself from those comments he has made that are homophobic, that are racist, that are misogynistic, I will apologise for calling him scummy.”
The Daily Mirror reported that at the event in Brighton on Saturday night, Ms Rayner said: “We cannot get any worse than a bunch of scum, homophobic, racist, misogynistic, absolute vile… banana republic, vile, nasty, Etonian… piece of scum.”
To applause, she added she had “held back a little”.
Conservative chairman Oliver Dowden said: “At a time when the country is trying to pull together to recover from Covid, the last thing we need is the deputy leader of the Labour Party calling people ‘scum’ and yelling insults.
“We need to make politics better, not drag it into the gutter. Let’s see if we get an apology.”
Foreign Office minister James Cleverly said voters would see a Tory Party which has had two female prime ministers and the “most diverse government” and “they’ll know she’s talking crap”.
Left-wing Labour veteran John McDonnell defended Ms Rayner, suggesting she should not have used the language she did but that “deep down she’s expressing the anger many of us feel”.
The former shadow chancellor told Sky’s Trevor Phillips On Sunday: “We’ve all been there, late at night, getting very angry about what’s going on. What I like about Angie Rayner is that she’s human.”
Current Labour frontbencher Lisa Nandy said it is “completely up to” Ms Rayner whether she apologises over the remarks but conceded it is not the kind of language she herself would use.
The shadow foreign secretary told Times Radio: “It’s not my preferred choice of words. I’m not very interested in insulting the Tories. I just want to get rid of them.”