The real Byng

"It's Kahlua and vodka and milk.

"We've gone mad on them because we're going to publish a book about The Big Lebowski. Try it. You can have one if you like."

I try the cocktail favoured by The Dude, the lead character from the Coen Brothers' cult movie. It is delicious.

"I've got to pop out. Make yourself at home. Put some music on. Have a cigarette," he says.

And off Byng goes, sliding down the wooden outdoor stairs that lead to the basement, where the London arm of his successful Edinburgh-based publishing company, Canongate, resides.

So here I am, sitting in the living room of the son of an earl - a man whose mother ran off with one of his father's best friends - a self-proclaimed bad boy turned family man, and currently one of the most formidable independent publishers around.

I look at his bookshelves, all the usual suspects are there, along with Roberto Calasso's The Marriage of Cadmus and Harmony, Salman Rushdie's Shalimar the Clown and a few Byng publishes himself: Michel Faber's The Crimson Petal and the White, Louise Welsh's crime novel The Cutting Room, James Meek's The People's Act of Love - a remarkable love story set in Siberia during the Russian revolution which last week won The Royal Society of Literature Ondaatje prize. "I am so happy for him," says Byng. "It is a wonderful book. When I first read it I thought how original and ambitious it was."

And, of course, there is Yann Martel's 2002 Booker Prize-winning novel, The Life of Pi. It was this last novel that really propelled Canongate and Byng into the limelight. Pretty soon after that, Canongate won book publisher of the year and suddenly everyone was talking about Byng. He was described as a madman, an egotist, a lounge lizard, a perpetual party-going cokehead. He fostered this image by giving spicy quotes to newspapers such as: "[Coke's] the drug I like most at the moment. Alcohol, nicotine, coke."

I saw him at a party once - tall, round, tons of long brown hair, glasses, talking nineteen-to-the-dozen in a maniacal way to anyone who would listen. I'm rather relieved when Byng turns out to be a smaller and more enjoyable version of his former self. "I certainly know how to party," admits Byng, who has now reappeared back in his sitting room which, oddly for a publisher, is not chock-full of manuscripts. His house is stripped pine, almost minimalist, with a huge oil painting on the wall and a Voldemort-esque sculpture of a head on the table. There seems to be only one photograph, of himself and his second wife, Elizabeth Sheinkman, a literary agent, getting married. "I think Elizabeth's been a calming influence," he says. "Not that she doesn't like to have fun. But I'm older. In fact, I'm giving up smoking next week for Elizabeth's birthday so I'm going mad on it now. You know, I work hard, I play hard." So what about the drugs? "Today I certainly lead a much quieter life," he says.

Byng is renowned for the enthusiasm and loyalty he shows his authors. He wooed Martel from Faber by writing a long letter explaining why Martel should let Canongate publish Life of Pi. "I've probably still got the letter," he says. What did he say in it? "Oh that I thought the book was original, that I felt fervently passionate about it."

A love letter of sorts then?

"Yes."

Before Life of Pi, which has sold more than two million copies, Byng says he used to wake up paranoid that everything was going to crash and burn. "I used to look at the books and think, 'F***, this is all going horribly wrong.' I knew I had a great catalogue of books. I knew we had talented, original writers, but was anyone going to buy them?" That was when he realised that he himself had to promote the books and so the myth of Byng was born. "I don't care what people think of me," he says. "I don't care if they hate me. I'll do anything to publicise our books."

So who is Jamie Byng? "I am technically the Honourable Jamie Byng," he says, "because my father is the eighth Earl of Strafford but he is the most un-earl-like man you could ever come across." His father, Tom, has spent his life working in a local garden nursery and now looks after a stretch of the River Itchen where Byng grew up as a child. He and his three older siblings, one of whom is Georgina Byng who writes the Molly Moon children's books, spent their childhood in a rambling house, Abbotsworthy, in Hampshire.

"My parents were very relaxed about us," he says. "The back door was always open. We'd go off for six hours on the trot scrambling through woods and swimming in the river and we'd come back exhausted and dirty and no-one seemed to mind."

When he was 12, his parents divorced after his mother, Jennie, went off with Sir Christopher Bland, a former chairman of both LWT and the BBC governors. There was a great scandal. The word was that Bland "stole" Jennie from under the nose of Byng's father, who was a friend of his; that Tom Strafford was forced out of the family home and left to reside in a farmworker's cottage; that Bland and Strafford no longer speak.

"I have only positive things to say about my parents' divorce," says Byng, surprisingly. "I lost my parents' marriage, but I gained two new other important people in my life, my stepfather and stepmother [Strafford later remarried a local girl, Julia Howard].

"My father has never had those trappings of being an earl. None of us buy into that. But seeing my parents change their lives in such a fundamental way was living proof that change is always possible. They have always continued to be kind, gentle and loving towards each other."

Byng's father now lives in Easton, Hampshire, while the Blands live in Blissamore Hall, also in Hampshire. "I am very close to my stepfather," says Byng. "He has been like a second father and he has always been very good at giving me advice and helping out. In fact, he is the chairman of Canongate."

Byng is a passionate man. He came into Canongate 12 years ago as a 24-year-old. Before then he and his girlfriend, Whitney McVeigh - whom he later married and who is the daughter of Charles McVeigh, the wealthy co-chairman of the multinational investment bank Salomon Smith Barney - were running a successful nightclub called Chocolate City in Edinburgh. "I had done my degree there in English literature and I loved the city. Whitney and I wanted to stay there and a friend of mine suggested I try my hand at publishing." He wrote to Stephanie Wolfe Murray, the then doyenne of Scottish publishing and supremo at Canongate, and begged her for a job. He got unpaid work in the office. Within a few years, in an audacious move thanks to his excellent connections, he had bought the company along with his business partner, Hugh Andrew - who later left the company to found the similarly successful Birlinn publishing. They were helped financially by both Bland and McVeigh's father.

Canongate largely became all about Byng, though. So why do people focus so much on him?

"I don't know," he says. "I think having the company out of London surprised people. There are loads of small independent companies and it is hard to be successful." He is down in London now, although the main part of the office remains just off Edinburgh's Royal Mile. "Yes, I had to move here really. I was here anyway all the time having meetings and promoting the company, so I moved here. My children are here and London is the centre of everything."

Byng has two children by McVeigh. They were together for 12 years and, as he puts it, "had their children young".

"We were in our early to mid-twenties," he says. McVeigh, from whom he separated in 2001, gave up her career as an artist to concentrate on Marley, now ten, and Leo, eight.

"It's a job having children," he says. "Whitney was happy to do that and we are both devoted to our children. I know lots of people work so hard that they don't seem to have time for their kids and that is their decision but it could never have worked for me. I can't imagine not being with them."

He and McVeigh still live close to each other in London. Do they get on? "Absolutely," he says. "I think she is a wonderful person." He and Sheinkman have the children every other weekend and he sees them most days.

"I cannot imagine it any other way," he says. "Why be acrimonious? Of course the kids weren't delighted when we told them we were to divorce, but I think they have learned positive things from it. Whitney and I get on well and she also gets on well with Elizabeth."

But, surely, nothing is ever as simple as that. In an article written when they were together, Byng and McVeigh described each other as soulmates. I ask, rather cynically, how can he now have a different soulmate?

"Sometimes, when you are with someone, everything seems full of possibilities and that's how I feel about Elizabeth.

"I met her in 2001 briefly. I was DJing at a party at the Frankfurt Book Fair. She came and introduced herself but I then re-met her a while later and I was smitten. Absolutely rooted-to-the-spot smitten."

Sheinkman is a literary agent from New York who now works for Curtis Brown. In the photograph she looks very Manhattan, a bit Carolyn Bessette Kennedy - lucky him!

"Very lucky me," he says.

Does he hope they will have children soon? "Yes, absolutely," he says. Then again, he doesn't see his life calming down at all. "I hope I'll be a publisher for many years yet," he says.

"We have grown quicker and further than I ever expected." So, it's publishing that is shot through the heart of him is it? "Yes!" he says but then he says that actually, no, his children are the most important thing to him. "They are so exciting," he says. "Marley is very competitive, like me. She hates not winning.

"But having children has taught me many things: humility, the realisation of what is important; the ability to care less about what people think of me; the chance to think about what the future means.

"In fact, in real terms, if Canongate crashed and burned, I would be sad; but if something happened to my children I'd be devastated."