McInally was given the armband ahead of the World Cup but has struggled to produce his best form so far.
Brown, who started at flanker last Wednesday as the Scots thrashed Russia 61-0, is among three players who start for the second time in four days, with wings Darcy Graham and Tommy Seymour also retained.
Greig Laidlaw will now lead out the team at the International Stadium while Sean Maitland, who scored a vital double against Samoa in the Scots' first Pool A win, misses out with a groin injury.
The Scots need to beat the hosts to seal their place in the quarter-finals, but there remains serious concern the game will not go ahead as Super Typhoon Hagibis prepares to wreak havoc across Japan's eastern coast this weekend.
A decision on whether the Dark Blues' final Pool A game goes ahead will be made on the morning of the match.
Townsend said: "The opportunity to face the hosts in such a decisive Pool match will be a unique occasion and should be a great spectacle.
"Games of this magnitude don't come around very often in a playing career so we will be giving it everything to make sure it is a memorable match.
"It's taken a real squad effort from our players here in Japan to put us in a position to play for a place in the quarter-finals, having risen to the challenge of what's effectively been knockout rugby since the second round of our Pool.
"The entire group has featured in our past two Tests, both of which have been clinical and professional performances.
"It was great to see players get the opportunity to play in this World Cup on Wednesday and it looked like they enjoyed the experience.
"The effort invested in defence, support play and clearing has been encouraging and has put a lot of pressure on the opposition.
"A few players have pushed hard for selection with the form they've shown in training and in the last two games, and it's now down to this group of 23 men to build on this momentum and squad togetherness to deliver a winning performance against Japan."
Starting XV: Stuart Hogg; Tommy Seymour, Chris Harris, Sam Johnson, Darcy Graham; Finn Russell, Greig Laidlaw (captain); Allan Dell, Fraser Brown, WP Nel; Grant Gilchrist, Jonny Gray; Magnus Bradbury, Jamie Ritchie, Blade Thomson.
Replacements: Stuart McInally, Gordon Reid, Zander Fagerson, Scott Cummings, Ryan Wilson, George Horne, Pete Horne, Blair Kinghorn