Some car hire companies used potentially "unfair" terms in their contracts, while others provided little or no information on terms and conditions on their websites, the magazine said.
It said three companies – Enterprise, Europcar and Hertz – had clauses which stated if customers could not produce the keys after their car was stolen then they would be liable for the full value of the vehicle.
Which? Holiday said this meant that even if the keys were stolen at the same time, or the vehicle was involved in a carjacking, the customer would still be liable for the full cost of the car.
Researchers also found that two car hire companies – Avis and Enterprise – had no information about the terms and conditions of hire on their websites, leaving customers unable to read all contractual information before booking online. Alamo, Budget and National provided some but not all of their terms and conditions online.
The researchers also found it difficult to compare the cost of car hire as a number of the companies investigated did not include the full costs of waivers and extras on their websites.
Rochelle Turner, head of research for Which? Holiday, said: "Hiring a car for your holiday may seem like a great idea, but our rigorous research shows that it can be a minefield.
"Trying to work out the real price you'll pay, the insurance you need and what is or isn't included can be so confusing that most of us just sign the contract, pay for the car and hope for the best.
"But it is really important that you read and understand everything before you sign on the dotted line.
"Failing to do so could leave you with a hefty excess or seriously out of pocket if the car is stolen."