Scientists at Aberdeen University say the solution could be as simple as moving young raptor chicks from grouse moorland when their numbers become too big a threat.
The research is published to coincide with the Glorious Twelfth today, which marks the new season of grouse shooting in Scotland.
The hen harrier is a natural predator of the red grouse but the study suggests that, under certain conditions, the birds could co-exist with profitable grouse shooting.
Led by Professor Steve Redpath, the study involving grouse managers, conservationists and ecologists used science as a way to try to find a solution.
The ecologists developed a model and found a possible compromise. Prof Redpath said: “The model suggested that across the grouse moors of England, there was room for 70 pairs of hen harriers at relatively low cost for grouse shooting.”
When harriers breed at levels that have a significant economic impact on grouse shoots, the excess chicks would be removed from the grouse moors, reared in captivity and then released into the wild elsewhere.
Similar schemes are used in continental Europe where harriers breeding in crops are threatened by harvesting.
The next step is for grouse managers and conservationists to use the results of the model to agree on an acceptable number of harriers in moorlands in Scotland, and then test the idea in a field trial.
The Scottish Gamekeepers Association and Scottish Land and Estates welcomed the findings.
A spokesman for the gamekeepers said: “This study identifies and cements what was found during a joint raptor study at Langholm Moor in the Scottish Borders where a build-up of protected raptors, including hen harrier, made grouse shooting commercially unviable and led to the loss of gamekeepers’ jobs.
“The paper’s central premise is likely to be supported by those within our industry because we all seek workable resolutions to conflicts which will benefit both rural businesses and all wildlife, including hen harriers. Such models have been successful in other European countries, particularly France and Spain.”
He added that such an approach was similar to that being advocated with a government “hen harrier action plan” in England, which includes a brood management scheme.
Tim Baynes, of Scottish Land and Estates, said: “It is possible to have a management system which enables viable driven grouse shooting to continue, with all the benefits that brings to the rural economy and to declining species such as curlew, golden plover, lapwing and black grouse, while at the same time maintaining a sustainable number of hen harriers.”
An RSPB spokeswoman said the study “could play a major role in helping to resolve the hen harrier-grouse conflict”.
She added: “A brood management scheme may merit experimental investigation in the future, but only once hen harrier numbers have recovered to a pre-agreed level.
“The authors suggest that there is room on English grouse moors to support 70 pairs of hen harriers with only minimal costs to landowners..”