The giant balloon, operated by Virgin Balloon Flights, came within a few metres of the protected site near Forfar, Angus. Virgin has apologised.
Pictures show the four chicks fleeing the nest as the balloon approaches their tree.
Disturbing ospreys, which are endangered, is illegal under the Wildlife and Countryside Act. Offenders can be fined up to £5,000 and jailed for up to six months.
Mike Fenton, who saw the incident at Balgavies Loch on Wednesday, said he had “never seen anything so reckless”.
Mr Fenton, 52, of Letham, near Forfar, said: “I watched [the balloon] going down the loch, lower and lower, cameras came out for people to take photos. The juveniles took off in a panic. They had not returned half an hour later. The pilot couldn’t have got any closer to the nest. The nest is well publicised so the pilot cannot claim ignorance.”
Ian Thomson, of RSPB Scotland, said: “We became aware of this incident late last night. The osprey is listed on Schedule 1 of the Wildlife and Countryside Act, they are strictly protected from disturbance.
“Whilst it would be inappropriate to comment without knowing the specific details, I would urge witnesses to pass details to the Police Wildlife Crime Officer to investigate.”
Rab Potter, of the Scottish Wildlife Trust, said: “Human disturbance is a real threat to the conservation of ospreys. If this had happened earlier in the season, when the chicks are highly dependent on their parents, there could have been serious consequences.
An osprey expert, who asked not to be named, said: “If they were a couple of weeks younger and were unable to fly they would have jumped out of the nest and may well have died.”
A Virgin Balloon Flights spokeswoman said: “We are truly sorry one of our balloons appears to have caused distress to nesting ospreys. Hot air balloons travel with the wind and can’t be steered in the traditional sense.
“Had we known this area was home to a protected species our pilot would have marked it on his maps as such and may even have moved the flight to an alternative launch site to avoid the wind taking him over it.”