Hearts closed part of the stadium for the final two home games of last season after pyrotechnics, coins and a coconut were thrown onto the pitch after a defeat to Hibernian. As well as closing the section to away fans, around 200 season ticket holders were also affected by the move.
In an interview with the BBC, Budge said that although the corner sections will be closed to away fans, home supporters will be allowed access to their seats.
"We have closed corner sections - that will remain in place because it helped.
"We closed some of our own sections where we'd trouble. These are open again. But if we have the same sort of thing I will have to address it."
Budge also said that a helpline for supporters to alert the club when they witness misbehaviour has been a success.
"That is actually getting used more and more and more.
"As a consequence of that, if somebody sees another supporter or other supporters misbehaving to the point that they think 'this is unacceptable', we can do something about it, and we have done.
"If the supporters themselves are saying, "this is not what we want going on at our club, that's good. That's what we need to get to."