But sitting clean-shaven and sans shades in a corner of a London hotel bar, the 53-year-old iconoclast is unrecognisable. Which may not be a bad thing, given that his new satirical documentary, Religulous, tackles one of the most volatile subjects of our time: religious fundamentalism.
Fronted by the American comedian Bill Maher, the film puts Christians, Jews, Mormons and Muslims, among others, under scrutiny in an attempt to understand why they believe what they do. Maher, who for transparency's sake reveals he is the son of a Jewish mother and a Catholic father, in whose faith he was raised, believes religious certainty is leading the world towards destruction. Doubt is the safer, and arguably saner option. At the end of the movie he exhorts "anti-religionists" to "come out of the closet and assert themselves" in the face of religious extremism. "Grow up or die," he says provocatively.
This is incendiary stuff, and very funny to boot. But Maher and Charles, who grew up Jewish in Brooklyn, are playing with fire. Some of those perceived to have caused offence to Islam, for instance, have either been murdered, as in the case of Dutch film-maker Theo van Gogh, or, like Salman Rushdie, author of The Satanic Verses, had fatwas issued against them. "What I say is, if you get murdered, it will make a great DVD extra. That's the way I look at it," Charles says blithely.
Anyway, about that beard ... "I just shaved it," he says. "It was time. I think other people's reaction is much more radical than my own. They look at me like the Elephant Man now. They're like, 'What happened to you?'" He once said he wore sunglasses and facial hair so that people could not see where he was looking or gauge his emotions. "That's right ... and it's ironic that I had to shave my beard to hide myself. I took those things off and people don't know who I am."
Partly he was wary of becoming a self-parody. On the other hand, he learned while collaborating with Sacha Baron Cohen on the wildly successful Borat movie how image and influence can go hand in hand. Before that he often looked like "a dirty, hippy-like homeless person", which, he admits, sometimes had its drawbacks.
One day while directing an episode of Curb Your Enthusiasm, he went to grab some food and was accosted by a guard. "He was like throwing me off the set," he says, laughing. "The AD (assistant director] had to come over and go, 'Wait a minute, that's the director!'"
Cohen wanted Charles to present himself as a more authoritative figure when they met potential interviewees for Borat, and encouraged the director to wear a suit. It worked. "It's amazing," says Charles. "When I used to go to the airport the security would just hover near me. Now they actually let me bypass the lines." Thus he can be seen in Religulous, pre-shave, wearing a black two-piece.
I tell him this reminds me of Stanley Milgram's experiments where subjects willingly inflicted pain on people (or thought that's what they were doing) because they were told to by characters wearing lab coats. "Sacha and I talked about that," says Charles excitedly, "because it is exactly about that: how people react to authority and tend to fall into line behind authority."
Someone might realise in the middle of an interview that something is up, he recalls, but "what are you going to do? Pull the microphone off and cause a scene, or behave? Most people conform under those conditions and behave." In this sense, he says, films like Borat and Religulous are psychological, anthropological and philosophical experiments. "That's one of the things that really intrigues me about these kinds of projects."
And it is not just the urge to conform that works in the film-maker's favour. Something else learned on Borat, he says, was just how powerful "vanity, ego and hubris" are in our lives. Consequently, he dismisses the oft-made charge that he targets subjects who lack media savvy, particularly when applied to Religulous. "I feel like it's just the opposite," he argues. "They are media whores who can't wait to get on camera and then once they're on camera they get caught by surprise. They want to tell you what they believe. They want to tell you what they think and feel. What they're not ready for is to be asked questions about it and to have to defend those beliefs; they're not used to that."
Indeed, a sharp-suited minister (formerly a member of Harold Melvin and the Blue Notes) who preaches to thousands is unable even to quote accurately the New Testament passage about it being easier for a camel to pass through the eye of a needle than for a rich man to enter heaven. In fact, he inverts its meaning to justify his affluence.
Elsewhere, an American senator says he believes in the story of Adam and Eve and the talking snake from Genesis. When pressed by an incredulous Maher to concede the irrationality of his position, he says you don't need an IQ to be in the Senate. Too late, he realises his gaffe. Charles's sharp editing snaps the scene shut like a mousetrap.
While the fundamentalists stick to every word of the Bible, or the Koran for that matter, Maher also meets high-ranking clerics within the Vatican who offer a more enlightened point of view. Stories like Genesis and the virgin birth are merely myths, they say, from a benighted, pre-scientific age. To hold every word as the truth today is absurd.
Charles says that one of the most profound revelations for him while making the movie was the discovery that the story of Jesus is an archetype that had already existed for thousands of years before his (supposed) birth.
Unquestioning fundamentalist thinking becomes most disturbing, however, when it embraces the apocalyptic dimension inherent in most western religions. When religion and politics become entwined, as they did during the Bush administration, the situation becomes potentially explosive. Nuclear weapons, and our ability to destroy ourselves several times over, now mean that Armageddon is just the push of a button away.
"It's very simple for it to happen," says Charles. "I have read a lot about apocalyptic thinking and, from a Jungian point of view, it is almost like an internal thing: we are all living an apocalypse, we're all going to die so, in a sense, you want that to have significance, even if it's destructive significance on some level."
This is why he feels it is necessary for us to talk about these issues openly and honestly, and why he felt an urgent need to make Religulous. "I thought it could be healthy to explore that in a dialogue," he says. "But to be told, 'No, this is the answer. And if you don't believe it, by the way, I'm going to kill you,' that's what the problem is now."
Religulous is released on 3 April.