And then, for a second, he seems genuinely concerned that he's offended me. He becomes conciliatory. Sure, he still hates it, but it's not my fault.
"Please, it's not like that. It's just that I don't have anything new or positive to say about myself, that's all. I can't pretend, I'm not very good at that."
That's Sadowitz, a mess of contradictions. He says he has nothing to say, but get him on a subject he loves and he becomes almost invigorated. He's misanthropic, yet his job is to talk. He goes out of his way to make you dislike him but you come away liking him in spite of yourself.
For most people the glass is either half-full or half-empty. For Sadowitz, it's always completely empty, showing only the dregs of someone else's good time. Anger drives his creativity. He's angry that he's not more successful; angry that others are and have got there (he says) by stealing his material; angry about his difficult childhood, his frequent ill health. Angry, yes, at the sun streaming into his agent's office. He stands up and peels off his jumper, tossing his trademark top hat on an empty desk. "The only way I can be sure the weather's going to be nice is if I wear a jumper and a big overcoat," he grumbles.
Sure, life hasn't dealt Sadowitz the best of hands. He was born in New Jersey, and his parents split when he was seven. His Scottish mother took him back to Glasgow. That separation looms large over his life, as does the constant presence of ulcerative colitis: "It doesn't kill you, it just makes you really ill and constantly weak. What's the point?" But for all his hates, he doesn't hate life - he's just angry that he doesn't enjoy it.
He burst on to the stand-up scene in the late 1980s, sweeping safe, cosy alternative comedy out of his furious path. He was the riskiest and the rudest; the pioneer of comedy without borders. He was called "the most gifted young comedian of the decade" and, in 1992, got his own BBC TV series, The Pallbearer's Review.
But then everyone started to learn what comedy without borders really meant. It meant gags about Aids and cancer, the holocaust and child abuse. He was labelled racist and misogynist: venues banned him; promoters dropped him; the BBC (and later Channel 5) took fright and canned his series.
He ended up living in a bedsit in Camden, north London, doing occasional gigs and working in a magic shop to pay the rent. "Life," he says, "is a series of f****** disappointments."
But it wasn't just that life did the dirty on Sadowitz. He didn't help himself, and he alienated people who might have helped him. At times, he undermines his own hard work. Take this interview, for instance: he does it, under duress, to promote his forthcoming Scottish tour. Then, the next day, he sends me an e-mail asking if I can persuade my editor not to publish it. (Sorry, Jerry.)
Plenty of comedians before him have toned down their material to suit TV executives' sensibilities, but he doesn't. When I say a lot of people respect him for that, he brushes it off: "Anyone who thinks I'm brave is wrong. I do what I do because I have no choice."
But let's tune out the world according to Sadowitz for a minute. Things are looking up. Earlier this year his magic show broke all box-office records at London's Soho Theatre, and is now much in demand. At the Glasgow Comedy Festival he will perform magic at the Tron and stand-up at the ABC, both major venues. Then there's a Scottish tour, his first for two years, moving from Peebles to Inverness, Kilmarnock to Aberdeen, culminating in three dates at the Bloomsbury Theatre, London.
He has a strong team behind him, including Bill Burdett-Coutts, of Riverside Studios and the Assembly Rooms, and The Stand's Tommy Shepherd. PR guru Mark Borkowski is interested in coming on board. They aim to have him back on television within 18 months.
Shepherd says: "It's coming together in 2006. His stage performance is stronger than ever. His confidence is back up, the writing is flowing. I think, if anything, his relevance is greater than ever in the way he courts controversy, exploring exactly where the line should be drawn between humour and social commentary."
Sadowitz himself, however, would never be so upbeat. He's disappointed that he got a graveyard slot at the Soho, "because I have to fit myself in after all the other comedians who have got big promoters behind them," and that he's going on tour with essentially untried material.
"I still feel like the last beggar in the queue for the soup kitchen. Any other bloody comedian would get about 40 dates before their tour." Sadowitz, you see, is a perfectionist. He is, perhaps, a true artist. Like many such he is difficult to the point of being misanthropic, loves what he does and hates it at the same time, can't explain it or discuss it, but just does it because he "has no choice". His agent, Jay Lempereur, says: "He's a force of nature. He's the closest I've ever met to a proper artist, struggling with his ability and what to do with it."
Shepherd says: "More than any other performer I've ever worked with, he is exacting and demanding of himself and those he works with. He is constantly striving for perfection. The reason why the shows are so good is that they are preceded by months of anxiety and concern to get it right."
But he doesn't get any pleasure from making people laugh. He doesn't even taking a perverse pleasure in offending people. The one thing that gives him pleasure is magic.
He spends hours every day practising card tricks, and is regarded as one of the best close-up magicians of his generation. "Magic is an escape for me," he says. Can he say what he's escaping from? "Yeah. Life."
So let's talk about magic, I say - and, briefly, he is transformed. Sadowitz fell in love with card tricks aged 11, watching David Nixon and Johnny Hart on TV. "I told my mum that I wanted to do that, so she sent me off to the library, which was probably the least expensive route". He hung around Tam Shepherd's Magic Shop in Glasgow, watching the proprietor, the man whom the magic community agree is "the single greatest living card magician".
These days he loves magic, but hates magicians. "I think I've fallen out with the entire international magic community over the question of ethics and plagiarism. I don't think secrets should be sold on the internet from magician to magician.
"Magicians are their own worst enemies. All the things you're meant to do as a magician - like practising, or studying the history and theory of magic, or keeping secrets - magicians don't seem to want to do."
Magic shapes the way he sees the world. God, he says, is a magician. "I do a trick and the punter sees the effect of that trick. You don't see the method, because my job's to hide the method.
"If you think of the universe as a creation of God, we are living and seeing the effects of the creation, but we don't see the method because he doesn't choose to show us.
"Basically I believe that if even you don't believe in God directly, or love God directly, you do indirectly, because if there's anything you find in life you do love, that is indirectly loving God."
So what does Sadowitz love? "I love card magic, although magic is also a great source of pain. I love women, but I absolutely despise them because they have too much control, too many rights." [His agent laughs, nervously.] "I like music, but I think music is an addiction for me. There's nothing I haven't got some ambivalence to."
I'm starting to think Sadowitz is not at all what he seems. He is unexpectedly thoughtful, actually quite wise. But before I get too far down that route, the ranter is back.
"And here's another thing," he storms, "I really hate being taken seriously!"
Jerry Sadowitz will appear at the Glasgow Comedy Festival in More Card Tricks and Close-up Magic at the Tron, 10-11 March and as a stand-up at the ABC, 24 March. His Scottish tour starts on 25 March, see www.standoutcomedy.com