The Scotland prop was dismissed in the 53rd minute when his shoulder connected with Wyn Jones’ head at a breakdown.
Referee Matthew Carley reviewed the incident with the television match official before deciding Fagerson had committed an act of dangerous play.
Townsend felt the review process with the TMO was inadequate.
“I didn’t think they had much of a discussion and I didn’t think they showed enough of the angles,” said the Scotland coach.
“They showed one slow motion angle to begin with then took ages to find another one. The TMO did say that, because of the player’s late movement, you should make sure there’s no mitigation there, but I thought the whole process could have been much better.
“It’s obviously very serious when someone gets a red card and it just felt that there wasn’t the right angles or a proper discussion. There was a discussion between the team of three when they were waiting for the angles, and they seemed to make their mind up then.
“We get on with whatever decision the referee has made and there’s nothing we can do about it now.”
Scotland were 17-15 ahead at that point but had to play out the remaining 27 minutes short-handed. Wales went on to win 25-24.
The initial conversation between the match officials seemed to indicate that the TMO was leaning towards a yellow card and Townsend felt the refereeing team would have benefited from more views of the incident.
“It looked like the referee discussed that with his assistants when they were waiting for more angles,” said the coach. “That was the disappointing thing: did we see it live? It looked like it was played in slow motion on one angle rather than actually going, ‘was there late movement, was there a clear strike of the head?’ One of the angles I saw, I don’t know if there was.
“But they’re part of the game, red cards, and you have to deal with them. Apart from the moment straight after the red card, I thought our players did very well in that period and got back ahead on the scoreboard.”
Scotland conceded shortly after Fagerson’s dismissal but fought back through Stuart Hogg’s second try to go 24-20 ahead with 16 minutes remaining. Wales, however, found an extra gear and Louis Rees-Zammit’s brilliant second try edged them into a 25-24 lead they would go on to hold.
Townsend felt Scotland’s discipline overall was not good enough and was in marked contrast to last week’s performance at Twickenham where the visitors kept the penalty count to a miserly level.
“When you go to 14 men for 25 minutes, it’s going to be more difficult,” said Townsend. “When you give away penalties that lead to points, that’s obviously not helping your chances of winning either.
“We had three penalties in a row from lineout mauls that ended up with a try at the end of the first half. We’ve got to be better there.”