In the last year the CAS consumer helpline has seen a 42% increase in reports of rogue credit brokers, whose role is to put customers in touch with payday lenders but who often charge unexpected fees.
The middlemen sometimes pass on personal details to other companies and refuse to provide refunds.
The average fee is around £70 and in around a fifth of the 254 cases reported in 2013/14, the consumer was denied a refund despite asking for one within the 14 days allowed under law.
A review of the main credit broker websites by CAS found four out of 10 did not make it sufficiently clear to consumers that they were a broker service who did not offer direct loans.
Head of policy Susan McPhee said: “This trend is leading people into financial hardship before they even take out the payday loan.
“A lot of credit brokers provide a legitimate service, searching the market on your behalf to find a lender who suits your circumstances. However it’s clear from our evidence that many brokers are in fact exploiting consumers, taking money off them without their knowledge.
“Our advice to anyone who is searching online for a payday loan is to be very cautious and read the small print of any website you deal with.
“Sometimes you are not dealing with a lender but with a broker, who will charge you a fee before they even pass you on to the lender, so you will then have to pay this fee in addition to whatever interest you get on your loan.
“The only way to avoid this is to read the small print very carefully, and never give your bank details until you know exactly who you are dealing with and what you are getting into.
“If you think you have unfairly lost money to a broker, then get in touch with us and we will see if we can help you get your money back.”
The CAS consumer helpline number is 03454 040506.