Big data and data-intensive science

The government has selected eight great technologies, for which the UK has:

  • a combination of science strengths and business capabilities
  • world-leading research, with a wide range of applications across a spectrum of industries
  • the potential to be at the forefront of commercialisation.

The eight great technologies are: satellites, robotics and autonomous systems, synthetic biology, regenerative medicine, agri-science, advanced materials, energy storage, and big data.

Ninety per cent of the all of the data in existence has been created in the last two years. The birth of the World Wide Web has given rise to social media, ubiquitous smart devices and the internet of ‘things’ - it’s clear that we live in an information age. The case studies shown below are just a sample of the significant social and economic impacts of STFC’s data-intensive science, but showcase the strength of UK expertise in this area. You can read more in our big data impact brochure.

(Credit: Tim Smith)


Panasas aisle
(Credit: STFC)

In general it is not public demand that drives computing advances, but the requirements of researchers to collect, store and manipulate increasingly large and complex datasets. Big science projects such as those supported by STFC have consistently pushed the boundaries of data volumes and complexity, serving as ‘stretch goals’ that drive technical innovation.

JASMIN is a super-data-cluster that delivers the high-tech infrastructure required for data analysis for UK and European environmental science research. Housed at the Rutherford Appleton Laboratory, JASMIN is half supercomputer and half data centre. It has a high bandwidth networks linking it to satellite installations at the Universities of Bristol, Leeds, Reading and Edinburgh as well as Plymouth Marine Labs and the Met Office. This unique computational environment was funded by the Natural Environment Research Council (NERC) and the UK Space Agency (UKSA) and delivered by STFC.

Professor Duncan Wingham, NERC’s chief executive, said that “JASMIN is a unique hybrid of high-performance computing, storage and networking, coupled with cloud hosting capabilities, and will make a significant contribution to one of NERC’s most strategically important challenges: the improvement of predictive environmental science.” Professor Peter Jan van Leeuwen, acting director of the National Centre for Earth Observation, added “With JASMIN, the UK will be well placed to exploit the wealth of earth observation data coming from the European Space Agency and elsewhere over the next decade.”


Artist's impression of the SKA dishes.
(Credit: SPDO/Swinburne Astronomy Productions)

The Square Kilometre Array (SKA) will be the world’s largest and most sensitive radio telescope. Thousands of radio wave receptors (antennae) will combine to allow the SKA to see back into the early universe, before the stars were formed and there was only gas. It will allow researchers to investigate a wide range of fundamental questions in physics, astronomy, cosmology and astrobiology, exploring distant parts of the Universe for the first time.

The SKA presents unprecedented technology challenges. It digitally combines the signals from each antenna – using powerful supercomputers – to provide a virtual telescope with a collecting area of a square kilometre. It will be 50 times more powerful than any existing radio telescope. The data it will collect in just one day will be enough to fill 15 million 64 Gb mp3 players; it would take nearly two million years to play back on an iPod. The SKA central computer will need to have the processing power of over a 100 million PCs.

The UK is leading two consortiums – Signal and Data Transport (SaDT) and Science Data Processor (SDP). The SDP consortium focusing on the technology that’s needed to turn the data collected into useable science products, whilst the SaDT consortium is responsible for the design of the data transport networks that will have to handle the volume of data. Dr Keith Grainge, Deputy Lead of the SaDT consortium, says that “the SKA will be an extraordinary project. The flow of data we need to transport from the antennae to the processors is equivalent to the entire world's internet traffic rate in 2011. In addition, we will need to synchronize the clocks at each antenna to a thousand-billionth (0.000,000,000,001) of a second. With the team of experts we have round the world, we are confident that we can meet these challenges and we are all looking forward to exploring some fascinating new areas of science with the telescope."

Hartree Centre

STFC Hartree Centre
(Credit: STFC)

STFC’s Hartree Centre is an industrial gateway to world-class high performance computing (HPC) and simulation technology. Home to the UK’s most powerful supercomputer dedicated to the development, deployment and demonstration of new software, it enables new HPC collaborations that promote UK economic growth. Hartree’s visualisation suite allows industrial users to integrate the use of virtual models into their product development process, improving designs at an early stage, when changes are far less costly.

Working together, Bentley and the Virtual Engineering Centre (University of Liverpool) at the Hartree Centre developed a unique framework to evaluate the use of virtual reality technologies and immersive environments in product design and development. Due to the success of the project, Bentley engineers have adopted this approach for the development of their next-generation products. Using the state-of-the-art visualisation suite at the Hartree Centre allows Bentley to create new vehicle models virtually. This speeds up product development times through better understanding of design data and reduces the number of physical prototypes required, leading to lower costs and eliminating the need for late-stage modifications.

Peter Allan, director of the Hartree Centre, said “STFC is well known for its computational science expertise in modelling and simulation and one of our aims is to bridge the gap between science and industry to the benefit of UK companies competing on an international scale. We hope to pave the way for further advances which will benefit consumers and the wider economy, through significantly faster design processes and cost reduction.”

Resources for teachers on this topic are available from our public engagement big data and computing page, and RCUK have a Big Data timeline you can download as a PDF file.

Science and Technology Facilities Council Switchboard: 01793 442000