Unison said the Scottish Government "needs to understand how angry we are".
A survey for the RCN found 80 per cent of the public believe there are not enough nursing staff to provide safe and effective care in Scotland’s health and care services.
Meanwhile, 70 per cent said the Scottish Government's current pay offer is "less than adequate" to recognise the value of nursing staff.
The poll, designed by the Diffley Partnership on behalf of RCN Scotland, surveyed 1,043 respondents between June 27 and 30.
Julie Lamberth, chair of the RCN Scotland board, said: “Nursing pay has been held below inflation for years and the spiralling cost of living has only worsened the impact of long-standing low pay.
"With staffing levels at crisis point and nursing vacancies at a record high, there has never been a more important time to fight for the pay that nursing staff deserve.
“This poll shows that the public recognise that fair pay is vital for improving staffing levels and providing safe care and that they agree that the Scottish Government’s pay offer is inadequate.
"Our members are exhausted, feel undervalued and are leaving the profession in large numbers.
"To recruit and retain enough staff to deliver safe and effective care to patients, the Scottish government must pay nurses what they deserve. This is a political choice.
“Fair pay is vital to protect patient safety, address staff shortages and demonstrate that the nursing workforce is valued as a safety critical profession.”
The RCN pay ballot opens today and will run until August 4, while the consultative Unison ballot will close on August 8.
Unison said problems were building long before Covid and the pandemic had only exacerbated the issues.
Wilma Brown, chair of the Unison Scotland health committee said: “NHS staff have been taken for granted, staff have endured over 10 years of real terms pay cuts only to be told by the Scottish Government that, yet again, they will have to accept a below inflation pay rise.
“NHS staff have family bills to pay, food, energy and petrol prices are rocketing.
"NHS staff are struggling to afford the price of fuel to get them to work.
"They need more than praise and platitudes from Government, they need a decent pay rise to support their families.
“A 5 per cent pay increase across the board just doesn’t cut it and the Scottish Government need to understand how angry we are.
"Unison are urging Unison members to vote to reject this pay offer and indicate that they will take the very difficult decision to take industrial action, unless of course the health minister improves the offer on the table.”
A Scottish Government spokesperson said: “We are disappointed that Unison and the RCN are recommending members reject this offer which, if accepted, will be the biggest single year pay rise NHS Agenda for Staff have seen since devolution, and will ensure that our staff continue to remain the best paid in the UK.
“Under this pay offer experienced porters will receive more than £1,000 extra, while a healthcare support worker will see more than £1,200 extra. Experienced nurses will see their pay rise by more than £1,600 and an experienced advanced nurse practitioner will receive almost £2,400 more.
“The record 5% pay offer builds on NHS Scotland staff being the best paid in the four nations. The UK Government would need to deliver pay uplifts of between 6% to 14% to front line NHS England Agenda for Change staff to catch up with pay levels in Scotland.”
“Growing our workforce is crucial and that’s why we have steadily increased places on undergraduate nursing and midwifery courses over 10 consecutive years – doubling the number of new intake students in the last decade. We have recruited more than 1,000 additional healthcare support staff and almost 200 registered nurses from overseas to help address the unprecedented challenges facing the NHS.
“The Cabinet Secretary meets regularly with NHS trade unions, most recently with the RCN on 11 July.”