But research into Colonel Roderick MacNeil, last chief of Clan MacNeil, has revealed the desperate moves of the man who evicted his tenants and embarked on what is believed to be Scotland's only case of forced labour.
Author Ben Buxton said MacNeil, an officer in the British Army who lived mostly in Liverpool when not overseas, felt 'no sense of compassion or responsibility' to his tenants.
He inherited vast debts from his father, with the chief choosing a 'much more commercial - and tyrannical - policy' towards his estate and tenants who lived on Barra and the neighbouring Bishop's Isles of Mingulay, Sanday, Pabbay and Berneray.
When the market for kelp collapsed in the 1820s and crofters could no longer pay their rents, MacNeil raised their rents anyway.
He evicted crofters from fertile land, such as on the west coast of Barra, to make way for sheep grazing and settled the tenants on the east coast, where the land was far more barren.
By 1835, MacNeil faced financial ruin and he borrowed heavily to build a new kelp-processing factory on the shore at Northbay, Barra with a view to produce high quality alkaline to sell to the soap manufacturing industry.
Buxton added: "There is a suspicion that he was badly advised and that he hired what we could call a consultant to look into the business. It turns out this consultant was really quite crooked, a bit of a conman."
The heavily-financed enterprise opened in 1835
Buxton said: "MacNeil then hatched a monstrous plan: he evicted the people from the southern islands, and sent some of them, the people of Berneray anyway, to Northbay to work in his factory."
As Berneray people were pushed to Barra, their old land was populated with sheep. Soon, the neighbouring people of Mingulay were also threatened with eviction.
By March 1835, the threat had been carried out, with nine families - chiefly headed by fishermen - sent to Barra.
Up to 500 people, some involved in the supply of kelp to the factory, were employed at the factory with an unknown number of workers likely living in the plant's 'barracks room'.
The boundary wall to the factory still stands, with the outline of doorways still seen where boats would deliver the kelp for processing.
Indeed, MacNeil instructed all his crofting tenants in Barra to supply the factory with kelp - even forbidding them from using seaweed as fertiliser on their crofts - and set many of them to work at the factory.
Buxton added: "Although the crofters of all five southern islands were evicted, for some of the year 1835 at least, the people of Berneray bore the brunt of MacNeil's tyranny.
"Since it is unlikely that he gave them a choice in the matter, this looks like a case of forced labour, the only such case known in Scotland."
The factory, as it turns out, was a financial disaster and ultimately wrote his downfall.
It had barely got up and running when one of MacNeil's creditors, John Gordon of Cluny of Aberdeenshire demanded repayment of a loan.
Gordon of Cluny was to cast a long, dark shadow over the lives of these islanders.
He later went on to buy the Barra Estate, as well as South Uist and Benbecula, and is remembered as perhaps 'one of the worst' landowners ever to get his hands on a piece of the Highlands and Islands who went on to forcibly evict up to 3,000 tenants to Canada.
MacNeil was declared bankrupt in September 1836 with the factory and the estate was put in the hands of a trust.
Some of the workers returned to their old homes, with reports of islanders hunting for food among the cliffs as the famine struck the islands.
"There were plenty of instances of brutality and inhumanity carried out by landlords on their tenants during the clearances, but none as extreme as this," Buxton said.