Operation Heathyard, led by CID in West Lothian and supported by West Lothian Council, the National Crime Agency and Greater Manchester Police, saw a number of warrants carried out on Friday.
The men were detained at addresses in Livingston, Bathgate and Salford and have now been arrested and charged in connection with human trafficking offences, involvement in serious and organised crime and a sexual offence.
All four men, aged 35, 26 and two aged 28, are expected to appear in Livingston Sheriff Court next week.
The activity under the operation has been centred on the Latvian communities in West Lothian, police said.
Detective Chief Inspector Stephen Healy said: “In total, 25 people have been interviewed as part of this operation and inquiries are ongoing to establish the circumstances surrounding their travel to and from Scotland.
“In the meantime, they are being offered support and assistance in terms of health and welfare, housing and victim support.
“This operation is the result of a complex and lengthy investigation into serious and organised crime involving foreign nationals centred on West Lothian.
“A large number of officers and colleagues from partner agencies have been involved in both the planning and delivery of the action taken and I would like to thank them for their work so far.
“Serious and organised crime can have a significant impact on communities and is a top priority for Police Scotland.
“Those involved in such groups use violence and intimidation to target vulnerable people and we are constantly working to gather information about their activities and disrupt them, putting those responsible before the courts and removing their assets.
“Human trafficking is one method that these groups use to run their criminal endeavours, and it is important for us to ensure that when we dismantle these networks and we work with our partners to protect and safeguard anyone who has been exploited or abused.”
Anyone with information or concerns about human trafficking or exploitation in their community is being urged to contact police via 101.