The star - who returns in an updated version of the show - said male viewers would describe their sexual fantasies, prompted by her pinny-clad portrayal of parlourmaid Rose.
The actress, 76, who was also one of the creators of the show along with Eileen Atkins, is the only original cast member to return to the show, which was set at upmarket Belgravia.
She said: "I got more rude mail than the beautiful girls on the show. I'd open something thinking it was a fan letter to Rose and find some man's fantasy, which had nothing to do with Rose herself and everything to do with what she wore.
In the new series, set in the second half of the 1930s, Rose has moved up to the position of housekeeper and has set up her own servants' agency.
Upstairs Downstairs was a phenomenal TV hit the first time around, portraying the lives of an upper crust family, but also examining the lot of the serving staff.
Marsh said the issues of class divide were still as relevant today: "It's not so much about servants and masters, but people are still disenfranchised."