Agents and the Grid: a personal view of the opportunity before us

It's not always easy to figure out exactly what is grid computing and the Grid. It was originally about joining computing resources together in order to do new things that demanded substantial scale of computation and data - using middleware to hide the heterogeneity of the underlying systems. As things have evolved its still about joining things up and about middleware that meets the needs of grid applications, but the Grid is now fundamentally service-oriented and the emphasis has shifted to the notion of the Grid as being about virtual organisations (VOs).

In 2001 we published the Semantic Grid report, which observed the gap between the ambitions of Grid computing and the practice in the Grid and e-Science world at that time. We advocated not only a service-oriented approach but also the adoption of techniques from the world of software agents and the application of knowledge technologies. We felt strongly that these things were necessary to achieve the full richness of the Grid vision - the essential, automated joining up of resources.

At that time grid practitioners were increasingly adopting a service-oriented approach. Grid Services - an enhancement of Web Services to address Grid requirements such as lifetime management, inspection and monitoring - came along, first in Globus Toolkit 3 and now in the form of WSRF, the WS-Resource Framework. Grid practitioners were also increasingly using metadata and have begun turning to Semantic Web technologies for this, as the Resource Description Framework tools have become available off-the-shelf in a timely manner. We are seeing the gap being filled along the lines we anticipated.

Well, almost. So what happened to Agents and the Grid? A key part of their appeal back in 2001 was the service-orientation: agents are producers, consumers and indeed brokers of services. Web and Grid Services have gone some way to provide this, and bring also a promise of take-up in the enterprise that perhaps was not evident through agent deployments. But agents also brought some other things, and it could well be that the Grid community is beginning to discover it needs these. The reason this has not happened sooner is that the Grid community has been very focused on delivering the technology for the large scale service-oriented Grid world but has focused less on what will happen when this world exists - when there are large numbers of services, with variable availability, qualities of service and cost. This is when the other aspects of agency become incredibly useful.

The second is autonomy - a behaviour not provided by Web Services but absolutely a given when working with agents. Automation is fundamental to the Grid but currently is not handled very flexibly. In contrast to Grid Services, autonomous agents work to achieve their individual objectives and they interact to meet their objectives in their common environment - techniques which directly address the Grid requirements.

The third point is architectural. In agent applications the agents often work with content which is not itself communicated in the agent communication language, as exemplified by information agents on the Web. This separation of the coordination architecture is also a Grid characteristic - in Grid applications, the high throughput requirements are still met by established Grid technologies while the Web services are used to control them rather than replace them.

In my view the autonomy argument is compelling but Agents might not yet have all the answers. A large scale distributed system like the Grid demands configuration and its components inevitably exhibit failures. People talk of the need for the grid to be self-healing, self-managing or self-organising - to exhibit autonomic behaviour. The need for this increases as grids become interconnected and decision-making becomes decentralised. Autonomy is part of this picture but there is more - it's also about giving up control, about self-organisation. It's fair to say that the agents community has yet to establish best practice in large scale decentralised self-organising multi-agent systems. The Autonomic Grid is perhaps a job of work for both communities and beyond, drawing on biologically-inspired computing, complex adaptive systems and cybernetics.

Some existing work is immediately applicable. The Semantic Grid vision promotes the application of Semantic Web technologies both on and in the Grid. The current research and development activities with agents and the Semantic Web fit the first category. Many real Grid applications - for example those in e-Science - have been shown to benefit from Semantic Web technologies at the content and applications levels, and they also stand to gain from applications of agency at this level. Pushing down a level, Grid applications work with very large volumes of data and there is a significant data management and curation task which can be automated through agents.

Further down inside the Grid middleware, these ideas are coming together in the world of Semantic Web Services - or Semantic Grid Services. Researchers are now able to look at emerging solutions - such as OWL-S and WSMO - and investigate them in the Grid context, with Grid use-cases. For example, will these approaches already accommodate the enhanced Web Services developed for Grid applications? The relationship between agency and Semantic Web Services is already a subject of research and this needs to extend to Semantic Grid Services.

So agents have much to offer the grid and in my view the timing is better than ever. What does the Grid offer agents researchers and developers? The key thing is real applications and real deployments, and a robust infrastructure for achieving this. I would not wish to imply that all agents researchers should be interested in this, but for those who believe that agents can make the world a better place there is definitely an opportunity here to try this out and inform future agents research. Significantly, the Grid community is actually quite open to new technologies that help them with the many challenges they have - they are motivated by real applications. They need to know what's off the shelf and what is research, but they're willing to put some effort into trying new solutions if they offer results, as evidenced by the spectrum of activities in the Global Grid Forum. For agents researchers, the results of this engagement may well transcend Grid computing per se - it is often observed that, in the fullness of time, Grid requirements might not be so different to those of other Web services application domains such as e-Business.

One way of engaging with the Grid community is via the Global Grid Forum. The mission of GGF is to lead the pervasive adoption of grid for research and industry. It achieves this through engagement with a broad international community for the exchange of ideas, experiences, requirements and best practices, coupled with defining grid specifications that lead to broadly adopted standards and interoperable software. The Semantic Grid Research Group is one of the community-oriented groups, tracking the development of relevant technologies and reporting these to the GGF membership, and bringing the communities together to do work that will lead to new Grid solutions.

At GGF13 in March we had presentations on Semantic Web Services and on Agents and the Grid. Meanwhile last year saw two calls to arms for the agents audience: the 'Brain meets Brawn' paper at AAMAS04 and the 'Bridge-Building' paper at ECAI04. The time has come for some serious - and mutually beneficial - engagement between these communities.

Further reading

The Semantic Grid: Past, Present, and Future by David De Roure, Nicholas R. Jennings and Nigel.R. Shadbolt, Proceedings of the IEEE, Volume 93, Issue 3, March 2005.

"Brain Meets Brawn: Why Grid and Agents Need Each Other" by Ian Foster, Nicholas R. Jennings and Carl Kesselman, in Proceedings of 3rd Int. Conf. on Autonomous Agents and Multi-Agent Systems (AAMAS 2004), New York, USA, 2004.

"The Semantic Grid: Myth Busting and Bridge Building" by Carole Goble and David De Roure, in Proceedings of the 16th European Conference on Artificial Intelligence (ECAI-2004), Valencia, Spain, 2004.