Data Capital: New ways to explore creativity

From life-saving applications, like those spotting signs of breast cancer on mammograms, through to more mundane examples, such as predictive text on smartphones, few parts of our lives have been left untouched by the expansion of machine learning and artificial intelligence (AI). Even the world of art hasn’t escaped.

AI software such as Midjourney, Stable Diffusion, and Dall-E – the name of which is a mash-up of artist Salvador Dali and Wall-E, the robot from the Pixar animated film of the same name – gives members of the public the chance to type in a string of text, which the programs then convert into pieces of art.

The results range from the jaw-droppingly sublime to the nightmare-inducing ridiculous.

Drew Hemment of Edinburgh College of Art. Image: Andrew Perry

Now, The New Real (TNR) – an AI research hub that brings together the Alan Turing Institute, based at the British Library in London, Edinburgh’s festivals, and the University of Edinburgh – is using art to explore the social implications of AI advancement through its art programme, “Uncanny Machines”.

Back in March, five AI artists were awarded development grants, and last month a winner was selected to embark on a full commission.

“This project is of the greatest personal importance to me,” says Kasia Molga, the artist selected to undertake the commission. During her project, entitled How To Find The Soul Of A Sailor, Molga will feed text from her father’s merchant navy diaries into the TNR’s AI platform.

By combining her father’s diaries with data from ships’ logbooks and British Library maps of the Mediterranean Sea, Molga aims to create stories in his voice. Her project will also explore how the locations he visited are now being affected by climate change.

As a child, Molga sailed with her father on his voyages, and the pair built a very close bond – the artist has described feeling like she could read her father’s thoughts. He died unexpectedly 15 years ago, and now Molga aims to explore the emotional impact of recreating her father’s voice through AI.

“My dad’s diaries are extremely precious, and this is my way of showing my love for my father, and my shared concern for the ocean, his life’s passion,” says Molga, who originally hails from Poland and now divides her time between the UK and the Netherlands.

She adds: “I’m excited to go to places I could not have imagined before using AI, and I can think of no-one better to share this journey with than TNR.

“On one hand, this new body of work is about the transition of marine ecosystems. Yet it also probes the emotional implications of giving away data – almost fragments of personality – of departed loved ones and the ethics of AI in the context of digital afterlives.

“The work will use AI wisely to reveal unexpected interconnections between the words of those no longer with us and such large subjects as oceans and climate change.

“It is a huge pleasure and delight to work with TNR – they are the leading group spanning the arts and AI in this way.”

Over the coming months, Molga will use AI to create a piece of art that will go on display in March next year. While she has refused to be labelled – with a range of words and phrases including “artist”, “creative coder”, “design fusionist”, “environmentalist”, and “technologist” used to describe her – Molga’s previous works have featured a mixture of objects, music, sound, and interactive components, including installations at Tate Modern in London.

At the heart of the project sits the New Real Observatory (NRO), the AI platform created by the Edinburgh-based hub to allow artists and researchers to explore a field of study known as “experiential AI”.

The NRO can handle images, numbers, and words, and is built on a number of algorithms, including Word2vec, a natural language processing technology that handles text.

“I’ll say right up front, it’s not as shiny or polished as OpenAI,” smiles Drew Hemment, professor of data arts and society at Edinburgh College of Art, and director of festival futures at the university’s Edinburgh Futures Institute.

“What we’ve done in our platform is experimented with ways to give artists greater creative control.”

Most common AI platforms work using text prompts – an artist will type a sentence and the AI will produce an image.

The intermediate steps take place inside a “black box” – neither the artist nor the programmers who created the AI understand what the AI is doing to produce the work of art, and so the artist has little control over the process.

In contrast, the NRO gives artists more granular control over the process through the use of a “slider”, which can be used to control each specific variable or dimension in turn. Artists can also “train” their model by choosing which information is fed into it or “retrain” an existing model that they have inherited.

“This is a means of giving the artist the chance to work with the material or medium,” says Hemment. “Artists using AI work with data and code and software as their material or medium.”

He draws comparisons with paint and a paintbrush on a canvas, or with words on a page or screen – here, ones and zeros are used instead of yellow ochre and midnight blue.

“Giving people access to the model and ways to directly probe and explore and push it gives that creative control, which is more important than the actual mechanism because it gives the artists more scope, more creative freedom, to create works,” says Hemment.

“We’ve had some real success and the artists really like it. They describe it as being in the middle instead of being on the outside – they’re right inside and they’re able to explore for themselves.”

While very large AI models are complex and opaque, the NRO is more transparent, giving artists the opportunity to look critically at some of the inherent problems with AI, such as bias within the data that is fed into the models.

In the same way that artists working with traditional media can challenge society through their art, AI artists can also ask questions of their audience.

“Some people like the dazzle and shininess of AI, but other artists want to challenge it – they’re troubled by all this, and they want to ask questions and push back,” Hemment says.

“That’s actually another really big goal for us in all of this.”