Lisa Willis left Philpott 10 years after she moved into the home he shared with his wife and their children, the court heard.
Miss Willis had four of her own children with Philpott during that time and said she and his wife Mairead took it in turns to alternate the nights they would share a bed with him.
The family achieved notoriety in 2006 and 2007 when they appeared on the Jeremy Kyle Show and in a documentary with MP Ann Widdecombe.
Philpott, 56, his 31-year-old wife and a third defendant, 46-year-old Paul Mosley, are on trial for the manslaughter of the Philpotts’ six children.
Jade, 10, and her brothers John, nine, Jack, eight, Jesse, six, Jayden, five, and Duwayne, 13, all perished after a fire which engulfed their home in Victory Road, Allenton, Derby, as they slept in their beds in the early hours of May 11 last year.
All three defendants have denied the charges.
Giving evidence from behind a screen at Nottingham Crown Court, Miss Willis, 29, agreed with Philpott’s barrister, Anthony Orchard QC, that the result of the television appearances was not positive.
He said: “Mr Philpott was being lambasted as a benefits scrounger. You were seen as a shameless family. It was a horrible time, wasn’t it?”
“Yes it was,” Miss Willis said.
He went on: “Michael Philpott is a man who raises in people quite a lot of emotion. People either love him or hate him.
“He could be arrogant. He could be a bit of a loud mouth. He would shout his business to anyone who would listen. He could rub people up the wrong way.
“There are some people who like him but many who do not.”
Miss Willis replied: “Yes, that’s true.”
The prosecution case, which was opened yesterday by Richard Latham QC, alleges that the Philpotts and Mosley started the fire in a botched plan to set Miss Willis up after she and her five children, four of whom were fathered by Philpott, left the relationship.
Mr Orchard said: “In relation to the children, you would agree that he was a good father?”
“Yes, he was a good father,” Miss Willis said.
He did not let them play in the street because he cared about their safety, Mr Orchard said.
He asked Miss Willis: “He would do nothing to endanger them?”
“No,” she replied.
Earlier, Miss Willis told the jury of seven men and five women that she met Philpott aged around 17 and moved in with him and Mrs Philpott at their council home in Victory Road, though the couple were not married at the time.
Miss Willis said that when they decided to marry she was a bridesmaid at their wedding and she and Mrs Philpott were good friends.
She treated their children as her own, she said, and regularly shared childcare responsibilities with Mrs Philpott.
Asked how she felt when she found out the six Philpott children had died in the house fire she said: “I was devastated.”
Her relationship with Philpott became sexual shortly after she moved in, something she claimed Mrs Philpott was not upset by and the three of them embarked upon an unconventional life together.
“We took it in turns, we never had all three of us together,” she said.
“We would take it in turns at night so one night it would be me, one night it would be Mairead.”
Her relationship with Philpott was not an easy one, she said, and was peppered with violence and his controlling behaviour.
She told jurors how in one incident Philpott repeatedly hit her on the legs, back and arms with a piece of wood after an argument about who was the father of her child.
“He just kept saying ‘no, it’s not true’,” she said.
She added she was “shocked and disappointed” after the incident and that “if we had a fall out he would bring it up again”.
The court heard Philpott controlled her finances - her wages as a canteen assistant, then as a cleaner, and her benefits went into his bank account - and her comings and goings were monitored by him.
She did not have her own personal key to open the door of the house, the court heard.
Mr Latham asked her: “Did you have free rein to leave the house if you wanted, or not?”
She replied: “I could leave if I wanted to but I did not go out because I was so sick of all the questions and answers of when I’ll get back, so I did not bother.”
Miss Willis told the court that unemployed Philpott, who she said did nothing round the house except watch television, drove her to work and picked her up, and constantly accused her of having affairs with colleagues.
His personal hygiene also left a lot to be desired. She told the court he bathed once a week or once a month and that he did not change his clothes often.
Miss Willis left the house on February 11 last year, having said she was taking the children swimming, jurors heard.
At the beginning of May she received a call from the police in relation to threats she was alleged to have made against Philpott.
“They said apparently I had made a phone call to Michael Philpott threatening him and his family,” she said.
Mr Latham asked her: “Have you ever made any threats to him or his family?”
Miss Willis replied: “No, none whatsoever.”
The prosecutor asked Miss Willis where she was on the night of the fire.
She told the court her sister and children were at her house until 8pm on May 10.
“Did you go anywhere near number 18 Victory Road that night?,” Mr Latham asked.
“No, I never”, she replied.
“Did you have any discussion with your sister about going to number 18 that night?,” he continued.
“No, we didn’t”, Miss Willis replied.
The trial continues.