The Tory MP for Ribble Valley said the two alleged victims, both men in their twenties, had been friends of his “until yesterday”, and he expressed incredulity at the allegations.
Mr Evans, 55, was arrested at his home in Pendleton, Lancashire, on Saturday on suspicion of raping one man and sexually assaulting another between July 2009 and March 2013.
The Swansea-born MP was questioned by police for several hours before being released on bail until 19 June.
Speaking in the beer garden of a pub next door to his cottage, Mr Evans said: “Yesterday, I was interviewed by the police concerning two complaints, one of which dates back four years, made by two people who are well known to each other and until yesterday, I regarded as friends.
“The complaints are completely false, and I cannot understand why they have been made, especially as I have continued to socialise with one as recently as last week.
“I appreciate the way the police have handled this in such a sensitive mannerm and I would like to thank my colleagues, friends and members of the public who have expressed their support and, like me, a sense of incredulity at these events. Thank you very much.”
Mr Evans, wearing glasses, read from a prepared statement and did not answer questions from waiting journalists.
He appeared to have attempted to cover up a scar on his forehead using make-up.
A source close to the MP later said: “He’s obviously very shaken up by this.”
He does not intend to stand down, either as deputy speaker or MP, according to his solicitor, Adrian Yalland.
However, Defence Secretary Philip Hammond warned it could be “very difficult” for Mr Evans to carry on as deputy speaker while the matter was being investigated.
He said. “I stick rigidly to the view that we should treat people as innocent until they are proven guilty, but it is quite difficult to carry out a sensitive and high-profile role while being under this kind of scrutiny.”
Mr Hammond added of the allegations: “I know Nigel well, I have known him for years. I’m obviously as shocked as everybody else is.”
Foreign Secretary William Hague said that Mr Evans was a “long-standing friend”.
Mr Hague said: “It’s right to point out, and for me as a long-standing friend of his, to point out that he is a very popular and well-respected Member of Parliament and deputy speaker. I think that is true across the House of Commons actually for MPs of all parties, so we will all be very sorry to see this situation.”
Former Metropolitan Police commissioner Lord Blair said that had the deputy speaker not been a public figure he would probably not have been named.
Mr Evans has been MP for the Lancashire constituency since 1992 and in June 2010 he was elected as one of the three Commons deputy speakers.
Later that year, he came out as gay, saying he was “tired of living a lie”.
Michael Ranson, chairman of the Ribble Valley Conservative Association, said people in the constituency were “shellshocked” at the news.
“He is a very popular MP and a very good constituency MP,” Mr Ranson said. “Everybody’s completely shellshocked.”
Profile: Steady rise through the ranks of MP who kept his Welsh links
Deputy Speaker Nigel Evans has been Conservative MP for the Ribble Valley for more than 20 years.
Born and raised in Swansea, he became a councillor on West Glamorgan County Council from 1985 to 1991, before being elected MP for the Lancashire constituency the following year.
Mr Evans kept his links to his home city, however, and owned family convenience store Evans the News in Townhill, set up by his grandfather in the 1930s, until he sold it last year.
He was parliamentary private secretary to several cabinet members in the 1990s, including employment secretary David Hunt, agriculture minister Tony Baldry and William Hague, when he was Welsh secretary.
In June 1997, Mr Evans was appointed opposition front-bench spokesman for Welsh affairs and he went on to lead the Conservative Party’s general election campaign in Wales three years later.
Iain Duncan-Smith gave Mr Evans the position of shadow secretary of state for Wales in 2001, which he held for two years, and he was appointed to the Welsh affairs select committee in 2003. He was vice-chairman of the Conservative Party from 2004 to 2005.
A popular figure at Westminster, in June 2010 Mr Evans was elected as one of the three Commons deputy speakers.