The four-time Tour champion was taken to hospital on Wednesday following a crash before stage four of the Criterium du Dauphine.
Team Ineos confirmed in a statement that the Briton had come off his bike during a reconnaissance of the day's ride and had withdrawn from the race.
And a spokesperson confirmed to Press Association Sport that he had suffered a suspected broken femur and was out of the Tour.
Froome's wife Michelle said he was involved in a "high-speed crash" while scoping out the route for Wednesday's time trial around Roanne.
Writing on her husband's Twitter account, she said: "Chris is on his way to hospital after a high speed crash during the route recon earlier this afternoon, injuries are yet to be confirmed. I am awaiting information and transport to get to where he is and as soon as we have any information we will update you."
Froome was eighth in the overall classification after three stages of the eight-day race, just 24 seconds behind leader Dylan Teuns.
The 34-year-old was using the race as part of his build-up to this year's Tour de France, which starts on July 6.
The Tour is only three and a half weeks away and Froome's preparations this season had been built around peaking for the race as he looked to join an elite group of just four riders to have won five Tour titles.