Extended abstract of invited talk at the 2nd European Semantic Web Conference. See www.semanticgrid.org for information about the Semantic Grid initiative.
In 2001, John Taylor (then the Director General of Research Councils at the Office of Science and Technology) recognised that many areas of science could benefit from a common IT infrastructure to support multidisciplinary and distributed collaborations. The UK initiated a £250M, 5-year program to develop the necessary tools, technologies and infrastructure - called e-Science it was described by Taylor as being about "global collaboration in key areas of science, and the next generation of infrastructure that will enable it."
The predominant technology proposed for the e-Science infrastructure was Grid computing, which had emerged from the efforts in the 1990s to combine computing resources in pursuit of new levels of computational power and very large scale data processing, by using high speed wide area networking of supercomputers and large clusters of commodity PCs. This new power enabled researchers to address exciting problems that would previously have taken lifetimes, and it encouraged collaborative scientific endeavours. The term Grid was chosen to draw an analogy with the way in which the electricity power grid brought about a revolutionary change from the use of local electricity generators. In this view, computational resources, data and expensive scientific instruments can be regarded as utilities to be delivered over the network.
In 2001, a number of researchers working at the intersection of the Semantic Web, Grid and software agent research and development communities were increasingly conscious of the gap between aspiration and practice in Grid computing - we felt that Grid alone would not meet the e-Science requirements. The potential benefit - indeed, necessity - of applying Semantic Web technologies to Grid infrastructure and applications was immediately apparent to us. Hence the Semantic Grid was born, articulated in the 2001 report Research agenda for the Semantic Grid: A future e-science infrastructure [1] and at a joint US-UK workshop in mid-2001 when UK researchers proposed the use of Semantic Web technologies, Web Services and Agents as steps towards achieving the full richness of the Grid and e-Science vision.
The Semantic Grid enables scientists to answer questions which involve integration of scientific data and automatic execution of computations, providing important functionality at the datagrid and scientific applications level (an aspect sometimes known as knowledge grid). Significantly, it also facilitates automation within the grid middleware - helping to discover and compose a variety of Grid resources and services in order to meet the dynamic requirements of multiple Grid applications. Hence the Semantic Grid is about the use of Semantic Web technologies both on and in the Grid [3].
In the years that have followed, many of these ideas have been put into practice and the Semantic Grid research and development community continues to grow. A series of events, starting with the International World Wide Web Conference in May 2002, has helped bridge the Semantic Web and Grid communities, and the Semantic Grid approach is increasingly a feature of international Grid activities. The Global Grid Forum, which leads the pervasive adoption of grid computing in research and industry by defining grid specifications and building an international community, has chartered a Research Group on Semantic Grid and run two Semantic Grid workshops, while other workshops have been held in the Web and Agents communities; the first Semantic Grid conference is planned for later this year. In addition to the flagship Semantic Grid projects in the e-Science programme (notably myGrid and CombeChem), other funding programmes are supporting an increasing number of projects; for example, the European Commission's FP6 Complex Problem Solving on the Grid programme lists at least five significant projects with 'semantic' components. A Web survey reveals Semantic Grid projects in every continent, plus workshops, conferences, journal special issues and industry briefing days. Google gives over 16000 hits for "Semantic Grid"
In 2004 we revisited the e-Science program three years on from the original analysis to examine if our expectations have been realised [3]. We see considerable progress, with five key technologies that are being used to address these requirements: Web Services, Metadata, Ontologies and Reasoning, Semantic Web Services and Software Agents. We also identified remaining challenges, which include for example the intersection between the grid and the physical world through pervasive computing devices, and the self-management, self-optimisation and self-healing (so called autonomic behaviour) necessary for large scale distributed computing.
The UK e-Science projects have provided a valuable test-bed for Semantic Web technologies, and best practice is emerging through these and a spectrum of international activities. There are still many challenges ahead, which will require the engagement of multiple research communities [4]. As we build these bridges and the community pioneers the application of these technologies, we move further towards the vision of a large scale, self-managing, collaborative Grid which can be used for flexible collaborations and computations on a global scale.
D. De Roure, N. R. Jennings, and N. R. Shadbolt. Research agenda for the Semantic Grid: A future e-science infrastructure. Technical Report UKeS-2002-02, National e-Science Centre, December 2001.
C.A. Goble, D. De Roure, N.R. Shadbolt, and A.A.A. Fernandes. Enhancing services and applications with knowledge and semantics. In I. Foster and C Kesselman, editors, The Grid 2: Blueprint for a New Computing Infrastructure, pages 431-458. Morgan-Kaufmann, 2004.
De Roure, D. Jennings, N.R. Shadbolt, N.R. The Semantic Grid: Past, Present, and Future, Proceedings of the IEEE, Volume 93, Issue 3, March 2005, Pages 669-681.
CA Goble and D. De Roure. The Semantic Grid: Myth busting and bridge building. In 16th European Conference on Artificial Intelligence (ECAI-2004), pages 1129-1135, Valencia, Spain, 2004.