Mr Hancock, the former health secretary, was suspended from the parliamentary Conservative Party on Tuesday after he was revealed as a surprise extra name added to the list of contenders for the ITV reality show.
Mr Balls told viewers on ITV’S Good Morning Britain he is worried it could be a misjudgement by the Tory MP.
He said: “Personally, I think good luck to him. But I think he is totally crackers to do this.
“Because I think it’s the wrong place to do it from, the wrong time and the wrong programme. To do it as a sitting MP just brings all these questions.
“He’s lost the whip, [Rishi] Sunak the Prime Minister is clearly furious, his constituents are going to be up in arms. He’s been paid to do this while he’s also going to the jungle.
“I mean, look, to be fair, Boris Johnson was on a Caribbean beach two weeks ago and he’s a sitting MP. But you know, this is a different thing.
“Secondly, this is the guy who was the health secretary during the pandemic, we have not had the inquiry yet.
“And therefore, it just brings back for so many people so much pain and suffering and it’s not something that you can make light of, until we’ve gone through that inquiry process.”
Mr Balls, who took part in Strictly Come Dancing in 2016, compared the BBC dancing contest to the ITV reality TV show, which starts on Sunday.
“And then finally, Strictly, nice, warm, fun programme,” he said. “I’m A Celebrity is tough. And you know, the other contestants will be, you know, I think supportive.
“And if he gets the tokens and brings in the food, then he’ll be popular. But, you know, I mean, to eat an ostrich anus, live on television, while still a sitting MP and then ask the public ‘do you want to see me do it again?’
“I think they might want to see him do it … I just worry for him that, wrong programme, wrong timing, wrong place to do it from, shouldn’t be doing it, but, you know, it’s gonna be fun, isn’t it?”
Tony Blackburn, who won the first season of I’m A Celebrity in 2002, said he understood why people were finding it “difficult” to comprehend Mr Hancock’s entry into the jungle.
He told BBC Breakfast: “I understand people who lost their relatives, I myself lost my sister to Covid when I was being told I couldn’t go and see her, I didn’t manage to get to her funeral so I can understand people being angry about that.
“On the other hand I can understand why he’s going in there. He’s now a backbencher, a lot of MPs have gone into reality shows and, of course, a lot of people will say this is our chance to get our own back because they can vote him to do the most appalling bush tucker trials.
“Of course he is going in there to talk about dyslexia as well and some of the money he’s being paid is going to go to charity. It’s a very, very tricky one.”
The 79-year-old broadcaster said he believed Mr Hancock was probably going on to show a different side to him, but he insisted the reality show was “tough.”
Mr Hancock was forced to quit as health secretary in June 2021 after breaking coronavirus social distancing rules by having an affair in his ministerial office with aide Gina Coladangelo.
The House of Commons is in recess from November 10-14, but celebrities could spend up to three weeks in the jungle – meaning Mr Hancock would miss significant Commons business, including the Autumn Statement on November 17, if he remains in the contest.