The six-year programme of work will support 5,000 jobs in the construction sector, according to the publicly-owned company.
The Scottish Government is making up to £720 million available to support the project, which is starting in 2015.
Deputy First Minister Nicola Sturgeon insisted that water charges would not increase by more than the rate of inflation.
An exemption scheme is to be extended to more of the charitable sector to help pay for their water and sewerage services.
Ms Sturgeon said: “Water and sewerage services are vital to our everyday lives.
“It is important that we continue to invest to ensure that these keep on meeting the needs of customers, the environment and economic development.
“I recognise that households and businesses are under financial pressure. That is why I expect Scottish Water to continue to deliver this £3.5 billion investment programme without the need for inflation-busting charge increases.”
Douglas Millican, Scottish Water’s chief executive, said: “I am delighted that we are announcing the start of this major investment programme which will enable Scottish Water to continue to provide first-class customer service while supporting the Scottish economy and creating jobs in the construction industry.”
Projects in the investment plan include more than £100 million for a Glasgow waste water tunnel to improve water quality in the River Clyde and reduce flooding in Mount Florida and Giffnock.
More than £280 million will be spent to improve water mains and water treatments works across Scotland, with £60 million allocated to reduce the impact of discharges to the River Clyde from Daldowie and Dalmarnock waste water treatment works.
Millions of pounds of drinking water improvements will also be made at Glenlatterach in Aberdeenshire, Fair Isle and South Uist.