External relations secretary Fiona Hyslop also warned that Scotland has a "duty to protect the interests" of the Scottish fishing industry, with Scots marine authorities continuing to monitor the situation on the long-disputed territory 240 miles from the mainland.
But one MSP called on Scotland to renounce any claim on Rockall - the seizure of which was branded a "last act of colonialism."
The Irish Government has slammed the approach of the Scottish Government in the row.
Mrs Hyslop told MSPs today that "intensified engagement" is now underway between officials in both Governments to resolve the row.
But she revealed activity by Irish vessels has increased from 15 incursions into Rockall waters in 2015 to 33 in 2016 and then 94 in 2017.
This fell sightly last year and there has been a slowdown since the row surfaced at the weekend.
"It is our duty to protect the interests of the Scottish fishing industry and our territorial seas and we have an obligation to uphold the law and exercise our rights under international law," Mrs Hyslop said.
"Dialogue is continuing between the Irish and Scottish Governments.
"There have been close contacts at an official level in recent days and it has now been agreed that a process of intensified engagement will take place led by senior officials from both administrations."
"While that discussion takes place Marine Scotland will continue to monitor the area using aerial and satellite capability."
All EU boats, including Irish trawlers, are free to fish in the 200 nautical miles off Scotland, but not within 12 miles of land.
Ms Hyslop said the issue first emerged in 2017 when the Scottish Government became aware of a "significant increase" in fishing by Irish vessels in the territorial seas and fishing grounds within 12 miles of Rockall.
The Irish Foreign ministry sought talks with the Scottish Government in April of that year about the situation as "potential enforcement" by Marine Scotland loomed.
"Since 2017 we have had regular ministerial meetings and calls where this issue has been discussed alongside official level meetings," Mrs Hyslop added.
"We have made various political and diplomatic efforts to resolve the issue without the need for enforcement action."
No resolution had been reached by September last year and Scottish ministers told the Irish Government that it would be preparing to take action.
But Green MSP Andy Wightman, an Irish citizen, called on Scotland to renounce its claim on Rockall.
He said: "Rockall was annexed by the British Crown on the advice of the colonial office when in September 1955
Lieutenant Commander Scott landed on the rock, placed the Union flag and announced: "In the name of Her majesty I hereby take possession of the island of Rockall.'
"Does the minister agree with me that we reject complicity in Britain's last act of colonialism and make it clear we have nothing to do with such land grabs and instead renounce any Scottish claim on Rockall."