Big data has a key place in planning

Smart data usage can help firms shape the way they work and get more out of their businesses. Picture: Getty

Our society is becoming increasingly dependent on information and communication technologies, with many activities generating data that is stored, communicated and processed one way or another.

More and more, the words “big data” and “data science” are starting to appear in the headlines and are associated with many activities such as social networking and health, and a range of industries such as oil and gas.

While there is increasing alarm over the amount of personal data that we as individuals generate and share online, there is much value to be gained from analysing data sets that are produced by other activities and could open the door to more efficient ways of working.

From the beginnings of our use of technology, data and data processing have been important concerns, but big data is usually associated with data sets that are so large and complex that our usual tools and technologies can not cope.

Increasingly, the computing science community – both academia and industry – has responded by developing new hardware, software, tools and techniques aimed at overcoming some of the restrictions on informatics that these data sets can impose.

It means that we are moving into a rapidly developing area of data science which provides novel opportunities to analyse data and provide new insights, for example in commerce, industry or health, through what is often called predictive analytics. This allows the discovery of novel trends and relationships in data, which help predict what might happen in the future.

Many techniques are based on statistical methods while others take their inspiration from biology, such as genetic algorithms based on the behaviour of genes.

The scientific importance of big data and data science is somewhat obvious but the economic implications are equally important. It is estimated that the big data market is worth at least £5 billion and growth in demand for solutions and data scientists is exponential.

The economic benefits of this have already been recognised by the government, with the Scottish Funding Council investing more than £10 million in the establishment of a big data facility under the Innovation Centres (ICs) initiative.

This new facility is called the Data Lab and has hubs in Edinburgh, Glasgow and Robert Gordon University (RGU) in Aberdeen, which will have a particular focus on energy, oil and gas.

In parallel to the Funding Council initiative, RGU has also established a new Centre for Smart Data Technologies that aims to directly bring the benefits of analysing and mining vast quantities of data to industry and commerce.

It focuses on big data analytics using smart approaches. This is a response to the oil and gas industry’s identification of a strong business need to make use of the vast data sets generated by exploration and production operations – offshore installations provide terabytes of data daily.

It is anticipated that the industry will secure huge efficiencies from the use of smart analytics.

The impact of this in terms of operational costs is not to be under-estimated, particularly given the current challenges the oil and gas industry is facing now and in the future, from low prices and dwindling reserves to increasing difficulties in extracting what still remains.

Big data will form the basis of new solutions that are likely to extend the life of North Sea operations but also assist in globalising the oil and gas expertise that has been accumulated in Scotland since the start of North Sea operations. In parallel to the application of smart data analytics within the oil and gas industry, the new centre will continue to carry out research in healthcare which involves areas such as treatment optimisation and diagnosis.

It is clear that the potential of big data to change the way we operate right across a range of industries is huge. We are still only beginning to see the possibilities big data could present – by embracing this age of information and continuing to invest in the future of data science, who knows where it could take us?

• Patrik O’Brian Holt is a research professor (cognitive engineering) at Robert Gordon University and contributes to the Smart Data Technologies Research Centre. www.rgu.ac.uk