Back to top

Composite Maritime Event Recognition


Maritime Monitoring


According to the International Maritime Organization IMO, 90% of the global trade is handled by the shipping industry. Maritime monitoring systems support safe shipping by detecting, in real-time, dangerous, suspicious and illegal vessel activities. Such systems typically use the Automatic Identification System AIS, a tracking technology for locating vessels at sea through data exchange. We have been developing a complex event recognition system for maritime monitoring in the Event Calculus that uses AIS as its main source of information, along with contextual geographical information (e.g, Fishing areas, Natura 2000 areas, depth information etc.).


Steps prior to Composite Event Recognition


Composite event recognition (CER) on vessel position signals requires two online tasks/steps: (a) computing a set of spatial relations among vessels, such as proximity, and among vessels and areas of interest (e.g., fishing areas), and (b) labelling position signals of interest as 'critical'—such as when a vessel changes its speed, turns, stops, moves slowly or stops transmitting its position. The Figure below illustrates these steps. AIS position signals are streamed into the system, and go through a spatial preprocessing step, for the computation of the spatial relations required by the composite event patterns. The output is then fed into the trajectory synopsis generator which produces two streams, the Critical Point Stream and the Enriched AIS Stream. The Critical Point Stream contains only the 'critical' positions signals and their annotation (e.g., the annotation stop_start denotes that a vessel stopped moving) while the Enriched AIS Stream contains all the position signals along with the annotation of the 'critical' positions.


Recognition method


We perform complex event recognition using the 'Event Calculus for Run-Time Reasoning' (RTEC), an open-source Prolog implementation of the Event Calculus, designed to compute continuous temporal projection queries for pattern matching on data streams. RTEC has a formal, declarative semantics—composite patterns are (locally) stratified logic programs. Moreover, RTEC includes optimisation techniques for efficient pattern matching, such as 'windowing', whereby all input events that took place prior to the current window are discarded/'forgotten'.


Composite Maritime Events



Highspeed near coast

Several countries have regulated maritime zones. In French territorial waters, for example, there is a 5 knots speed limit for vessels or watercrafts within 300 meters from the coast. One of the causes of marine accidents near the coast is vessels sailing with high speed, thus the early detection of violators ensures safety by improving the efficiency of law enforcement. The code sample below defines a pattern in the language of RTEC that can be used for the detection of intervals at which a vessel has speed greater than a user specified threshold within 300m from the coast.


Some examples of detected speed violations, near the port of Brest in France are illustrated in the videos below.


Tugging

A vessel that should not move by itself—e.g. a ship in a crowded harbour or a narrow canal—or a vessel that cannot move by itself is typically pulled or towed by a tug boat. It is expected that during tugging the two vessels are close and their speed is lower than normal, for safety and manoeuvrability reasons. A formalisation of the tugging activity is the following:


According to the above definition two vessels are said to be engaged in tugging if one of them is a tug boat, neither of them is a pilot boat, and, for at least 'TuggingTime' time-points, they are close to each other and sail at tugging speed. The video below illustrates a detected tugging activity between two vessels.


Pilot Boarding

During piloting, a highly experienced sailor in navigation in specific areas—a maritime pilot—approaches with a pilot boat, boards and manoeuvres another vessel through dangerous or congested areas. Maritime pilots are navigational experts with knowledge of a particular area such as its depth, currents and hazards. Piloting, therefore, is of major importance for maritime safety. A formalisation of pilot boarding may be found below:


According to rule above, pilotBoarding(Vessel1,Vessel2) holds when one of the two vessels Vessel1, Vessel2 is a pilot vessel, neither of them is a tug boat, Vessel1, Vessel2 are close to each other, and they are stopped or sail at low speed far from the coast. According to domain experts, the boarding procedure in pilot operations takes place far from the coast for safety reasons. A detection of a pilot boarding operation is illustrated in the video below.


Trawling

Fishing is an activity that exploits natural resources, and thus needs to be regulated to safeguard fair access and sustainability. Maritime monitoring enables better regulation and monitoring of fishing activities. A common fishing method is trawling, involving a boat—trawler—pulling a fishing net through the water behind it. The trawler has steady—trawling—speed and a wide heading angle distribution.

trawlingMovement (left rule-set) is subject to the 'deadlines' mechanism of RTEC, i.e. this fluent is automatically terminated after a designated period of time—10 minutes for the detections illustrated below—has elapsed since its last initiation. (We omit the corresponding RTEC declarations to simplify the presentation.) As shown in the above rule-set trawlingMovement is also terminated when the vessel in question leaves the fishing area. (In other applications, it may be desirable to relax the constraint of restricting attention to designated fishing areas.) Consequently, trawlingMovement(Vessel) is true as long as the fishing Vessel performs a sequence of heading changes, each taking place at the latest 10 minutes after the previous one, while sailing in a fishing area.

According to the rule on the right, a vessel is said to be trawling if it is a fishing vessel, has trawling movement and sails in trawling speed for a period of time greater than Vtrawl (1 hour in our experiments). The videos below illustrate a trawling activity in isolation (left) and compared to the global traffic in the area (right).


Anchored or moored

A vessel lowers its anchor in specific areas—e.g. waiting to enter into a port, or taking on cargo or passengers where insufficient port facilities exist. Furthermore, vessels may be moored, i.e. when a vessel is secured with ropes in any kind of permanent fixture such as a quay or a dock. Consider the specification below:


anchoredOrMoored(Vessel)=true, holds when the Vessel is stopped in an anchorage area or near some port, for a time period greater than some threshold (see Vaorm in the rule above). In our experiments, this threshold was set to 30 minutes. The video below illustrates a detected anchoredOrMoored activity for a vessel near the port of Douarnenez, France.


Datasets & Code


  • The detected instances of all composite maritime activities mentioned above, over a period of 6 months (October 2015-March 2016), in the area of Brest, France, is available here. The AIS signals of the dataset are available here, while a semantic annotation of the dataset is available here.

  • The RTEC source is available here.