The £25,000 Walter Scott Prize will bring the cream of fiction writers to the Borders

• Picture: Jane Barlow

He was going to meet the Duke and Duchess, and his hopes were high. If things went as well as he hoped, within a few months he might be able to announce the launch of Britain's first literary prize for historical fiction. Unlike the Man Booker, the Costa, the Samuel Johnson and the Orange Prizes – all of which it would mirror in terms of prestige and quality of writing – this new prize would be decided and presented not in London, but just a dozen miles away, in Abbotsford. For if you're going to set up a prize for historical fiction, what better place than the home of the man who wrote the world's first best-selling historical novel? What better name for it than the Walter Scott Prize?

Moffat had already outlined much of this in an e-mail he'd sent to the Duchess of Buccleuch a fortnight before Christmas. He'd never met her, but he knew of her love of literature, just as he knew of the long and deep friendship between Sir Walter Scott and his clan chiefs – the third, fourth and fifth Dukes of Buccleuch.

• Shortlist for the Walter Scott Prize

She was intrigued by Moffat's proposal. He should, she suggested, drop round to Bowhill and talk about it further with her and her husband on the first day after the Christmas holidays.

All of this was why Moffat was driving his slightly battered Range Rover down the Ettrick valley on a picture-postcard Borders winter morning, trying and failing not to skid on the narrow lanes, then up the long tree-lined drive. "Sweet Bowhill" Scott called the house in front of him in The Lay of the Last Minstrel.

He opened the driver's door to get out, but the icy cold had locked it shut. So too were all the others. The only way he could get out was by climbing over the back seats and shoving open the tailgate. Hardly an auspicious beginning.

The Duke and Duchess – Richard and Elizabeth – met him and showed him into the main drawing room. Under the gaze of portraits by Raeburn, Reynolds and Gainsborough, he outlined his plans for a prize for historical fiction that would honour Sir Walter Scott.

They discussed it for two hours. "They'd clearly done their research," says Moffat. "I was very impressed. When it came to how much the annual prize would be, Richard said, 'We've got to commit 25,000 to this.' I said, 'That gets us there in one bound. A prize as big as that automatically belongs in the major league.'"

The benefits for Abbotsford – where, they decided, the prize would be presented – were obvious. Not only would it highlight the importance of Scott's former home in the nation's cultural history, but it would boost the fundraising campaign to restore it to its former glory. The Borders Book Festival would also benefit – all the shortlisted authors would be invited, adding further to the lustre of the programme Moffat will be unveiling next week.

But will it work? Can the ferociously London-centric media be lured out of the metropolis to the Borders? Can they, the London publishers and the wider literary world be made to realise that just because a prize is decided within Scotland it isn't just for Scottish writers?

That's the gamble the Duke and Duchess of Buccleuch decided to take at that meeting with Moffat. So far, though, all the omens are good.

Take, for example, the sheer literary quality of the Walter Scott Prize shortlist unveiled today (see right). The seven books on it range in subject matter from Rome to the Second World War, in each case burrowing beyond the mere facts of the historical record to bring the past alive. The great 19th-century critic Leslie Stephen pointed out that, at his best, Scott's fictional characters "are as good flesh and blood as ever walked in the Salt-market of Glasgow" – and that's what we can find too in the seven books shortlisted for the prize in his name.

Among them there is Thomas Cromwell rising to power in the court of a wildly wilful king, Cicero trying and failing to understand Caesar, a Jewish industrialist chasing his dreams in the shadow of the Holocaust, and a psychopathic medieval outlaw with scant resemblance to the Robin Hood we think we know. History might tell us what will happen to all these characters, but as long as we are turning the pages, we do not know: the past's sequentiality dissolves and we can join them in their own, engrossing, unpredictable present.

And here lies the importance of historical fiction in its own right – the reason it is worth having a prize for in the first place. The great 19th century historian Thomas Carlyle once pointed out that "Scott understood that the bygone ages of the world were actually filled by living men, not by protocols, state papers, controversies, and abstractions of men." Many of today's historians, focusing on analysis not narrative, concentrating on ever narrower periods and ever smaller topics, forget this basic lesson. Historical novelists don't.

"There's lots of instances where fiction enables you to get into the past in a way that the history books just can't," says novelist James Robertson. "Historical fiction not only joins up gaps in our understanding of the past but its stories give us a sense of structure. It's also very strong right now – you've only got to look at the books that win prizes."

That, in turn, takes us onto yet another reason that a prize for historical fiction will be worth following: historical novels don't only allow us to make sense of the past, they also have the sheer quality, variety and range that's worth celebrating in the present. Manchester academic Jerome de Groot, whose book The Historical Novel was published last year, says: "You could easily choose the best ones from each decade for the last 50 years, and they would also be the best novels. In fact, you could go right back to the 19th century, to Stendhal, Tolstoy and Scott himself, and it would be the same. The best novels are always historical novels."

In which case, you might think: why not have a UK-wide prize that celebrates them? Why not call it after the novelist who, from the Borders, gave the world its first truly global literature? Why not hold it in Abbotsford and welcome its winner to the Borders Book Festival at Melrose on 19 and 20 June?

On a bitterly cold, blue-skied day at the end of last year, in a house that Sir Walter Scott used to love to visit, three people sat down and decided that this will be exactly what will happen.