Scotland's first fairytale

Here’s a riddle. What do an American novelist, a children’s writer from Hemel Hempstead, a 1970s folk band and a Hollywood film studio have in common? All have found inspiration in the same ancient Scottish ballad: the story of Tam Lin.

The Border ballad, thought to be many hundreds of years old, has been cropping up in all sorts of places of late: an award-winning British children’s book; a best-selling debut novel in the US, and now Sony Picture Animation is considering making it into a major movie.

Tam Lin has all the ingredients of a good yarn: romance, adventure, a brush with the supernatural. Donald Smith, director of the Netherbow Scottish Storytelling Centre, says it is a very important part of Scotland’s literary history. "It’s one of the big classic Scottish ballads, a master work handed down on the page in high quality early versions. It is one of the great defining ballads. It encompasses a sort of crucial central myth which is very ancient."

The tormented figure of Tam Lin resides in Carterhaugh, a wooded stretch between the Ettrick and Yarrow rivers near Selkirk. Young women are warned away from the area. If they pass through, they will have to leave a pledge for Tam Lin, a ring or a mantle, "or else," says the ballad, as recorded in Roderick Watson’s The Poetry of Scotland, "their maidenhead". But it would take more than this to scare Janet, the laird’s daughter. She walks boldly to Carterhaugh and encounters the mysterious stranger. In some versions they fall in love, in others he rapes her. Either way, she returns to her father’s house pregnant. When she seeks Tam Lin out again, he explains to her that he is a knight, the grandson of a local nobleman, who was captured by the fairies when he fell from his horse while hunting.

Janet, if she is brave, can rescue him and return him to the mortal world by seizing him from his horse when the fairies ride past on Halloween. They will bewitch him and turn him into a variety of shapes - a snake, a bear, a hot iron bar - but if she holds fast to him he will be restored to the mortal world. The feisty Janet stands her ground and gets her man.

Of course, as with all ancient stories, there are variations: Tam Lin is sometimes Tamlane, Janet sometimes Lady Margaret. However, there is remarkably little variation in the basic elements of the story. First mentioned as a popular ballad in The Complaynt of Scotland in 1549, it was first written down by Francis James Child in his English and Scottish Popular Ballads. Great writers who collected ballads and songs - Robert Burns, James Hogg, Sir Walter Scott - all considered it important.

Smith says: "The ballads, folktales and poems of the 16th and 17th century are Scotland’s Shakespeare. We didn’t have the theatre, the defining dramatic corpus, but it was part of the same great fertile period for language and literature, and the distinct Scots contribution was the oral literature.

Part of Tam Lin’s fascination may lie in the fact that it has never been cleaned up and prettified as a children’s fairy tale. Its dark origins and adult themes are still clear: there is a pregnancy and (in some versions) an argument over an abortion. It ends not with a "happily ever after" but with the furious Queen of the Fairies cursing Janet for stealing away her "bonniest knight".

It is still firmly anchored in the Celtic world where, it was believed, the visible coexisted with the invisible, the natural with the supernatural, separated only by a thin veil. These fairies are not little critters who sit dreamily on flowers, they are malevolent powers who look exactly like humans, echoes of the ancient pagan gods who think nothing of paying a tithe to the devil every seven years with a human sacrifice.

Janet, on the other hand, is no princess waiting in a tower to be rescued. She is her own woman, a girl with an attitude who, in some versions of the story, strides to Carterhaugh as a dare. When questioned by Tam Lin about why she strays on his territory without his permission, she replies archly: "Carterhaugh, it is my ain/ My daddie gave it me;/ I’ll come and gang by Carterhaugh,/ And ask nae leave at thee."

When her father confronts her about her pregnancy, her stance is unusually independent. "If that I gae wi child, father,/ Mysel maun bear the blame - /There’s neer a laird about your ha/ Shall get the bairn’s name." She is no shrinking violet.

In fact, the story of Tam Lin is about two strong women who love the same man, one motivated by love and courage, the other by hate and destruction. Smith says: "The Fairy Queen is the femme fatale writ large, the dark image of the Earth Mother. She says that if she had known what Tam Lin was going to do she would have taken out his heart’s blood and put in a heart of clay.

"A lot of the women in 16th and 17th century oral tradition folk tales and ballads do not in any way cow-tow to the men. There is plenty of violence and oppression going on, but these stories are full of strong, feisty women who hold their end up and win through. There may be a connection between that and the fact that many of the singers and narrators who passed on these stories were women."

This reversal of stereotype, calling to mind films like Disney’s Mulan (about a woman rescuing her lover) may explain why the idea appealed to Sony Picture Animation as a 21st-century fairy tale.

The beauty of myth lies in its versatility. Tam Lin can work as a story in its own right or a potent piece of symbolism, as was discovered by American author Carolyn Parkhurst, who interwove it through her best-selling debut novel, Lorelei’s Secret. It is a contemporary story about a linguist, Paul Iverson, whose wife Lexy dies, falling from the apple tree in their garden - the only witness to the event being Lorelei, their dog. Intent on finding out if her death was really an accident, he decides to teach Lorelei to talk. Parkhurst says: "I first heard the story of Tam Lin when I was a little girl. I had a recording of it - a children’s version - which I used to listen to while I was going to sleep. It fascinated me. I loved the idea that Janet had to go and win her beloved back from the Queen of the Fairies, and that the Queen is so furious about losing him to Janet.

"Because the story had such a powerful effect on me as a child, I’d been thinking for a while that I’d like to use it somehow in my fiction. While I was working on Lorelei’s Secret, it struck me that it might work well as a metaphor for the way Paul tries to rescue Lexy from the darker side of her personality. In her worst moments, she transforms herself into all manner of unpleasant things in order to push Paul away, but what she really needs is for him to hold her fast."

In a completely different way, the story inspired Sally Prue, who won the Smarties Prize for her novel for young people, Cold Tom. Tom, part-elf, part-human, is cast out of his home with the elves, who are cold-blooded and hate all things human, especially the "vines" of emotional attachment which bind people to one another. Running for his life, Tom is rescued by Anna, a young girl trying to learn to live with her step-brother. Prue discovered the story through a song by folk band Fairport Convention.

Tom’s slow journey from one culture to another, and eventual acceptance of human relationships, also becomes a metaphor for the process of growing up. "Elves with no love have a clear and free independence, they stay away from any human bonds. Teenagers are like that to some extent."

Cold Tom has also been optioned for a film and Jeanette Winterson approached to write the screenplay. Prue believes that only a live-action film could really convey the power of the Tam Lin story.

"It seems a pity to animate it, it’s too easy in a way, because I think it’s important that the elves are very like us."

Elements of Tam Lin seem profoundly rooted in the real world. Scots writer Liz Lochhead confirmed that when she wrote a poem, Tam Lin’s Lady, exploring the human side of a not-so fairytale relationship. Her heroine is a young girl seduced by an older man.

"Tam Lin" persuades her to go through with the pregnancy, "[which] he was honest enough to admit would be hell and highwater for you... He’d been talking in symbols... and as usual the plain unmythical truth was worse." Happily ever after? Fat chance.

•Sally Prue is at the Book Festival Gala Day for Schools tomorrow.