As automation and reliable, high-bandwidth communication networks become more common, humans are increasingly responsible for monitoring and controlling the activity of distributed teams of cooperating agents, both human and machine. Such control decisions in many realistic domains are complex, and require human experience and judgment. Our vision is that human decision makers will be able to perform more important tasks than continuously monitoring incoming information by relying on an automated execution aid to alert them when significant new information warrants their attention. We are primarily interested in domains requiring human control and will describe two such domains. However, the majority of our techniques and analysis also apply to completely automated execution monitoring. In fact, in one of our domains we both interact with a human controller and autonomously adjust robot behavior and plans.
To rapidly make effective control decisions for distributed agent teams, the human needs automated support, for several reasons. First, inexpensive sensors and reliable, high-bandwidth communication networks provide large volumes of pertinent data arriving from sensors, team members, and other sources. Without automated support, the human cannot cope with the volume of incoming information. Second, plans that coordinate the activity of several team members, as many as several hundred in our first domain, can become too complex to monitor without automated help. Third, we are addressing domains that are dynamic, sometimes requiring responses in a few seconds or less. Fourth, the automated team members (robots) are complex, with different failure modes and recovery procedures, and automated support for controlling them is often essential. All these challenges are magnified as the tempo of the decision cycle increases or the user becomes stressed. Thus, domains with the above properties require an interactive, automated assistant to support humans in monitoring incoming information and controlling agent teams.
We will concentrate on dynamic, data-rich domains where humans are ultimately responsible for team behavior. Realistic domains often have adversaries to overcome. These may range from fairly benign forces of nature that introduce uncertainty, to intelligent adversaries that are trying to actively thwart plans. An automated execution assistant should interactively support effective and timely decision making by the human, and interact with the human to take advantage of knowledge the human possesses that is not explicitly modeled in the machine. Ideally, an execution assistant would allow its human user to, among other things:
One key idea is that rich plan representations allow the execution aid to share context with users, so both understand the semantics of plans and requests. Understanding the plan is the key to helping the user deal with the possible information glut created by advanced information systems. The execution aid uses the plan to filter, interpret, and react to the large volume of incoming information, and can alert the user appropriately when events threaten the plan or the user's physical existence.
Once the user develops trust in the execution aid, there will be a reduction of the need for human monitoring of the display of the information system, while simultaneously increasing the amount of relevant information monitored because the aid analyzes every piece of incoming data. Relying on alerts from an automated aid allows the human to pay attention to more important tasks than monitoring incoming data, attending to the display only when alerted by the execution aid.
In the next section, we characterize the domain-independent challenges posed by this problem, concentrating those that are unique to interactive execution aids in dynamic domains with distributed teams of cooperating agents. Then, we describe how properties of various domains influence these challenges and their solutions. In Section 4, we present a domain-independent categorization of the types of alerts a plan-based monitoring system might issue to a user. Next, we describe the concept of value of information and alerts that is key to reducing unwanted alerts (alarms). Sections 6 and 7 describe the Execution Assistants we implemented in the small unit operations and robotics domains, respectively. Sections 6.8 and 7.5 contain the results of evaluations performed in each domain. Finally, we discuss related work and present our conclusions.