Detecting and anticipating activities in wide-area surveillance missions, as for example in the NATO counter-piracy operations in the Mediterranean Sea, is a task literally comparable to finding a needle in the haystack: even with the most advanced sensing technologies at hand, sensor coverage is still limited to a fraction of the area of interest. Under such conditions, a sensor management making maximum use of available surveillance assets, taking into account the adversary modus operandi, is vital for mission success.
To improve sensor management for wide-area surveillance, we present innovative tools for the implementation of a task-oriented approach on sensor management, grounded on a detailed analysis of the activities to be detected and anticipated. Object-event graphs, a generic graphical representation for templates of multi-agent activities allows explicit modeling of relevant activities and their temporal structure by domain experts. Permanent sensor deployment and routine surveillance missions for initial detection of suspicious activities can be optimized for the detection of the activity templates and evaluated using a scenario simulation environment for heterogeneous surveillance systems. During operation, inexact graph matching of activity templates with object and event detections from smart sensor systems can provide early detection of suspicious developing activities. From the analysis of partial activity template matches, the management of mobile and configurable sensor systems can be focused to confirm or reject potential critical activities.