A study by NHS Quality Improvement Scotland also found 15 per cent of the psychiatric patients had used alcohol during their stay in secure units.
The findings raise concerns about the recovery of patients treated in the units, who often have conditions such as bipolar disorder and schizophrenia.
Yesterday's report made a number of recommendations to improve the care of patients in Scotland's 14 intensive psychiatric care units (IPCUs), which have a total of 147 beds, but concluded that, overall, services were adequate.
But concerns were raised about potentially violent offenders being placed in units with other patients, which the report said was "unsatisfactory".
Out of 112 patients questioned about their stay at an IPCU, 19 per cent were recorded as having used non-prescription drugs and 15 per cent had instances of alcohol use during their stay.
Kevin Hodgson, an IPCU nurse, said: "These are technically low security areas and there is no perimeter security. I can talk about places where someone could walk up to the window and pass things through – they are generally on the ground floor. And visitors do come in and we don't automatically search them."
Where drink and drugs were found, they were removed from patients, he said.
The report's authors said alcohol could impact on people's behaviour in the ward, and make diagnosis more difficult.
Another concern was where patients were sent to units through the criminal justice system, leading to concerns about others' safety. "Particular difficulties can arise when some offenders are patients along with vulnerable adults, for example sex offenders with younger people and women," they said.
The report sought the views of patients using IPCUs, and several raised concerns about their care. Most patients agreed physical restraint was appropriate when used to deal with challenging behaviour, but a small number said they felt it had been used inappropriately and some said they had suffered pain.
Out of 50 patients interviewed in detail, 29 said they felt safe in the IPCU but 18 were afraid of threats from other patients. Three said they were afraid of staff.
The report highlighted concerns about patients not getting enough time to spend in the fresh air, a lack of access to talking therapies rather than drug treatment and the inappropriate admission of patients with conditions such as dementia and learning disabilities. Only one IPCU had dedicated input from a clinical psychologist; others relied on outside staff.
Liberal Democrat health spokesman Ross Finnie said:
"Mixing violent offenders with other mental health patients is simply wrong. It is shocking that the most vulnerable in society are being treated in this way."
A Scottish Government spokeswoman said: "We expect NHS boards to ensure that the mix of patients admitted to IPCUs is appropriate."
On the issue of alcohol and drugs getting on to wards, he said: "We expect all IPCUs to have policies in place for the safe care and management of those with substance misuse problems in mental health care settings, and to provide safe, supportive environments and care regimes."