A Survey on Context-Aware Multi-Agent Systems: Techniques, Challenges and Future Directions
Research interest in agents is rising, particularly in Artificial Intelligence (AI) techniques such as deep multi-modal representation learning, deep graph learning, and deep reinforcement learning, which suggest potential for human-like task performance within a multi-agent framework. However, challenges exist in enabling these agents to coordinate, learn, reason, predict, and navigate uncertainties in dynamic environments. Context awareness is crucial for enhancing multi-agent systems in such situations. Current research often addresses context-awareness or multi-agent system issues separately, leaving a gap in comprehensive surveys that explore both fields, especially regarding the five essential agent capabilities (Sense-Learn-Reason-Predict-Act). This survey fills that gap by presenting a unified architecture and taxonomy for developing Context-Aware Multi-Agent Systems (CA-MAS), which are vital for improving agent robustness and adaptability in real-world environments. We provide a comprehensive overview of state-of-the-art context-awareness and multi-agent systems, followed by a framework for CA-MAS development. We detail the properties of context-awareness and multi-agent systems, and a general process for context-aware systems, highlighting approaches from various domains such as ambient intelligence, autonomous navigation, digital assistance, disaster relief management, education, energy efficiency & sustainability, IoT, and supply chain management. Finally, we discuss existing CA-MAS challenges and propose future research directions.
An agent can preceive contextual information: The development of Context-Aware Systems (CAS) illustrates the agent’s ability to sense and model contextual information and utilizes such data for learning and reasoning purposes [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14] (see also Section 3).
An agent can learn any sensory data: Advancements in deep learning techniques show that multiple types of sensory data, including images, audio, text, and signals, can be transformed into machine-readable formats [15, 16]. These formats are subsequently encoded into learnable tokens through the use of Transformer architectures [17, 18]. Frameworks, such as joint representation, coordinated representation, and Encoder- Decoder facilitate the conversion between tokens of different data modalities [19, 20, 21]. These innovations enable an agent to represent and work with various type of sensory data.
An agent can reason: The semantic relationships of sensory data are encapsulated in the form of an ontology [22, 23, 24]. An agent can utilize these relationships for reasoning using rule-based, case-based, or graph-based approaches (refer to Section 4.2.3). Moreover, advancements in Deep Reinforcement Learning (DRL) techniques enable the agent to plan and reason with goals (refer to Section 4.2.3). Additionally, the representation of the ontology, generated by Graph Neural Networks (GNN) or their variants [25, 26], can be integrated with DRL to further enhance the agent’s reasoning capabilities.
An agent can predict and take action: The agent estimates and predicts future states based on its model of historical data and observations. This prediction process can be carried out using weight schemes, probabilistic models, or reward-based models (see also Section 4.2.4). Additionally, DRL techniques demonstrate that the agent can choose optimal actions in various scenarios by interactively imitating the behaviors of other agents and humans [27, 28, 29, 30, 31, 32].
An agent can coordinate with other agents: The development of Multi-Agent Systems (MAS) demonstrates the agent’s capability of communicating and coordinating with other agents to achieve a goal within some set of constraints [33, 34, 35, 36, 37, 38, 39, 40, 41] (see also Section 2).
To understand dynamic and complex environments, an autonomous agent must possess a combination of five capabilities: Sense, Learn, Reason, Predict, and Act. For example, the agent learns and reasons about the behaviors, goals, and beliefs of other agents or humans. This understanding enables the agent to predict and act within the dynamic environment through self-evaluation [42, 43, 44]. Note that the limitations of a single autonomous agent such as inefficiency, high cost, and unreliability [37, 39] become apparent when addressing distributed complex tasks such as microgrid control [45], resource allocation in cloud computing [46], as well as in the realms of computer networking and security, and other complex tasks [35]. Overcoming these limitations requires the implementation of a system that allocates task responsibilities among autonomous agents, enabling effective coordination and communication during task resolution. Additionally, in a specific task, a collective of autonomous agents needs to adapt to changes of the environment by constantly learning and updating their knowledge over time. This results in the growing attention to the field of context-aware multi-agent systems where agents comprehend their knowledge according to perceived contextual information to adapt to any situation and optimally solve allocated tasks and achieve the global goal. Application domains of such systems include autonomous navigation, ambient intelligence, supply chain management, Internet of Things (IoT), disaster relief management, energy efficiency & sustainability, digital assistance and education, and other complex problems (see also Section 4).
Multi-Agent Systems (MAS) comprise multiple autonomous agents that interact within a shared environment, autonomously making decisions to accomplish tasks or address complex problems. An autonomous agent in MAS is endowed with initial knowledge about a given task and possesses its own set of goals. While engaged in task-solving, the agent interacts with other agents or the environment to perceive and comprehend information. It independently makes decisions based on its objectives, existing knowledge, and observations, subsequently executing actions [41, 47]. Depending on the task’s characteristics, agents can collaborate or compete strategically to outperform opponents. These attributes confer flexibility and adaptability on agents in dynamic environments, making MAS suited for addressing complex problems. Furthermore, an autonomous agent forms its beliefs based on its knowledge and observations of the environment and other agents. Moreover, motivations for actions can vary among agents. Consequently, determining the optimal solution for a specific task necessitates effective communication, coordination, or competition in MAS. In cooperative settings, achieving consensus, and a shared agreement on a particular interest is imperative for autonomous agents. Conversely, in competitive scenarios, agents must analyze the behavior of opponents, anticipate negative outcomes, and devise strategies to address such challenges. It is worth noting that attaining consensus or understanding the behavior of other agents requires an autonomous agent to be aware of the context, including roles, organizational structure, situations, and location [33, 39]. Therefore, the integration of context-aware systems into multi-agent systems, as known as context-aware multi-agent systems, becomes essential.
Context-Aware Systems (CAS) pertain to systems that dynamically adapt to the environment by leveraging context to retrieve prominent information for tasks or problems. Context encompasses various perceptual elements such as people, location, physical or virtual objects, events, and other information delineating the situation of an autonomous agent in a specific environment [48]. The CAS process involves three primary stages: context acquisition, context abstraction and comprehension, and context utilization. [6, 7, 9]. Through this continuous process, autonomous agents acquire contextual cues to comprehend the current environmental state and undertake actions relevant to the situation. In the context of autonomous agents, contextual information encompasses task objectives, organizational structure, agent roles, and temporal aspects. Such context assists agents precisely retrieving relevant information for task accomplishment [42, 49, 50].
3 Context-Aware Systems
Context encompasses various types of information, such as people, location, physical or virtual objects, events, time, and other data that can be employed to introduce different dimensions of a situation or conceptual information regarding specific circumstances [48]. Additionally, five key properties characterize context: type, value, time when sensed, source where the information was gathered, and confidence in information accuracy [4]. Furthermore, context can be categorized into two groups: intrinsic context and extrinsic context. In the context of MAS, the former specifies the internal factors of an agent (e.g., goal, task, behavior, belief, knowledge, etc.), while the latter focuses on external factors, such as the environment, situation, social influence, and more. Existing context modeling techniques in the literature fall into six categories: (1) key-value models, (2) markup schema models, (3) graphical models, (4) object-oriented models, (5) logical-based models, and (6) ontology-based models [117]. The selection of a context modeling technique depends on four factors, such as complexity, scalability, interoperability and reasoning assistance (see also Table 3) [117, 118]. For instance, key-value models are suitable for situations where simplicity is crucial, although they lack two following capabilities: capturing the relationships within context and reasoning. To address these challenges, other context modeling techniques are employed. It is worth noting that ontology-based context models are widely utilized due to their capability for semantic reasoning and representation of context relationships through knowledge graphs [119, 4, 120, 121]. However, challenges associated with ontology-based context modeling include lack of generality [120], the difficulty in understanding ontology complexity and the substantial cost of maintaining ontologies [5]. To address the challenge of generality, Gu et al. [120] proposed a context modeling technique that leverages the Web Ontology Language (OWL) to enhance semantic context representation. Additionally, Horrocks et al. [122] introduced the InstanceStore system, which employs relational databases for semantic indexing and optimizes the performance of ontology-based reasoning. While modeling or reasoning about context, it is crucial to consider three aspects: (1) the quality of context, including accuracy and completeness of information; (2) relationships between context elements; and (3) the flow of context, such as time-invariant context, time-variant context, and consistency in context switching [117, 5, 123].