The lender – which is 80 per cent owned by taxpayers – is expected to be handed the penalty by the Financial Conduct Authority (FCA), Sky News reported yesterday.
It said RBS was facing a penalty of around £15 million, although other reports suggested it would be higher.
The fine was reported to relate to inadequate advice given to consumers who took out mortgages with the bank.
Both RBS and the FCA both declined to comment.
The fine will represent the latest in a string of financial hits the bank has faced, including fines and compensation pay-outs in the wake of a series of scandals.
These include £3.25 billion to cover payment protection insurance (PPI) mis-selling and £1.3bn for interest-rate swaps – complex financial products which were sold to small firms.
It has also faced hundreds of millions of pounds in fines as part of the Libor rate-rigging scandal and paid out to settle sanctions-busting allegations with US authorities.
RBS made a loss of £8.2bn last year, which included making provisions for past scandals as well as the cost of setting up its “bad bank” to dispose of unwanted toxic assets. But it swung to a pre-tax profit of £2.65bn for the first half of this year.
Iain Gray, Labour MSP for East Lothian and Labour’s finance spokesman, said customer confidence in the banks had yet to be restored. He said: “The consequences of years of bad practice and worse by our major banks continue to rumble on through their balance sheets.
“Yet the taxpayers who ultimately paid the price, and continue to do so still see bankers paying themselves eye-watering bonuses. No wonder they ask themselves if the banks have really learned their lesson.”
David Bell, professor of economics at the University of Stirling, said the regulators were taking a more focused interest in how the banks were operating.
He added: “On the scale of fines, a fine of £15m is not big. But any organisation whose job it is to police anything will look at giving some kind of sanction which is commensurate with what they have found.”
Reports of the fine come after a catalogue of complaints from customers and businesses about poor advice.
One of the most notorious mis-selling scandals involved the issuing of endowment mortgages. This involved customers taking out a monthly savings plan, which was usually invested in share and property and designed to pay off their home loan at the end of the term.
However, many people found themselves with a shortfall and have had to make urgent arrangements to pay this off or risk losing their homes.