The animal was being walked around Helix Park in Falkirk when it fell into the Forth and Clyde canal near the Kelpies statues at about 1.30pm on Friday.
Local resident Gordon Clark was out walking in the area when he witnessed the dog fall through the ice.
The situation then escalated, according to the passerby, who claims the rescue mission put the lives of two people at risk.
"It all started with a dog wandering on to ice and then falling through,” he said.
"Its owner tried to call it back, but to no avail as it kept struggling to get out.
"A chap standing next to me went for a life ring and a pole to throw across to the owner and other people on the other side.
"When he went to throw it, he stood partly on ice and went under as well.
"He struggled to get out so I jumped the barrier with his partner and we struggled to get him out as he was soaking.
"Luckily we got him out and took off his wet jacket.”
A woman, believed to be the dog's owner, then tried to rescue the dog by going into the water herself, according to Mr Clark.
"This dog almost cost two lives,” he added.
Meanwhile, barge owner Robert Fettes, 54, who has been living on the Forth and Clyde Canal for the last three years started up his engine in a bid to help break the ice during the frantic rescue mission.
“I was woken up by someone going into the water, I had been on a night shift,” Mr Fettes said.
"I looked out the window and saw this little Poodle-like dog struggling in the water so I started up my engine with a view to breaking the ice, but the propeller I think sucked the dog under the ice.
"I really thought the animal was going to drown.
"I carried on though, moving the barge, breaking up the ice around it where I could which was helping move the dog closer to the edge of the canal.
"A woman, the owner I believe, was in the water and she then managed to pull the dog to safety through the cracks.”
Mr Fettes said if he hadn’t managed to break up the ice, the dog would have likely drowned.
"It was such a horrible moment," he added.
"People need to be a lot more careful with their pets in the area when there’s ice on the canal."
Mr Fettes said the Scottish Fire and Rescue Service arrived shortly after and provided oxygen for the distressed animal.
It is understood the dog suffered no further injuries in the incident.
A spokeswoman from Scottish Fire and Rescue Service said: “We were called at 1.35pm on Friday with a reports of a dog in the water at the Kelpies.
“Two appliances from Falkirk were mobilised to the scene but the dog was recovered from the water prior to our arrival.”
Fire crews have now left the scene.
I have just came from a walk at the Kelpies this afternoon and witness and pulled someone from the ice.