Acoustic Sensors Enhance Emergency Rescue Drones

In future, drones in the air that are equipped with microphone arrays will be able to locate cries for help and any acoustic signals given by those needing help, and to provide rescue teams with the location of injured people. This will significantly increase the chances of the quick rescue of people who are trapped or buried and cannot be seen by the aerial camera alone.

Floods in Libya, Greece and Slovenia, fires in Hawaii and Tenerife, earthquakes in Turkey and Morocco – every minute saved is valuable in rescuing the injured when a region is hit by a natural disaster. But searching for survivors is a complex task; buildings and roads can be damaged, and large areas can be inaccessible.

That is why drones with daylight and heat-sensitive cameras are increasingly being used to fly over large areas where the infrastructure has been damaged, to locate those that need help, and to raise the response ability of rescue teams.

One problem, however, is that trapped victims are not visible to optical sensors. The cameras also have their limits when there is smoke, fog, and darkness.

Researchers at the Fraunhofer FKIE have been working on a solution to enhance the cameras with acoustic sensors for these situations. Their LUCY solution – short for ‘Listening system Using a Crow’s nest arraY’ – has been developed by the Fraunhofer FKIE scientist Macarena Varela together with her colleagues and the head of the research group, Dr Marc Oispuu, with technology that can save the lives of people trapped under buildings or by fire.

Microphone Array Captures Signals from All Directions

LUCY consists of an array of MEMS microphones, a so-called crow’s nest array that is fitted to the drone to determine the direction from which noises, such as calls for help, clapping or knocking noises, are coming.

The tiny, robust MEMS microphones are cheap and are used in smart phones, for example. The special aspect of this system is that the microphones are arranged in a specific geometric pattern on the underside of the drone and can ‘hear’ noises from all directions.

“The highest point on sailing ships is known as the crow’s nest, from where you can see in all directions. It is the same for LUCY – our system can effectively hear in all directions,” explains the researcher.

LUCY works in a similar way to the human ear, which receives sound information and sends it to the brain where it can be analyzed. The ears are replaced by microphones in this array system, and the brain by a signal processing unit that estimates the source direction. The direction of the sound source can be precisely determined because LUCY currently has not just two but 48 microphones.

“Spatial hearing naturally works better with 48 or more microphones than with two acoustic sensors, and also active listening in one direction as well as actively not listening works better too,” says Dr Oispuu. The system is also sensitive to frequencies that the human ear cannot hear. The number of sensors will be increased to 256 in future, to process signals in real time.

Ambient Noise Interference Filtered Out

Noise disturbances, such as rescue equipment, wind, or birds, but also the humming of the drone’s own rotors blades, can be filtered out by the system. Using artificial intelligence methods (AI) and with the help of adaptive filters, such signals can be filtered out but also patterns such as screams, knocking or clapping noises that people make when they are trying to attract attention can be learnt.

The system refers to a database of various noises or noise signatures that is used to train AI in advance. In combination with signal processing techniques, such as coherent beam forming, noises can be detected, classified, and their precise incoming direction determined.

A compact processing unit makes it possible to process the signals quickly. The locational data of the emergency is transmitted to the rescue teams, who can then see the exact position of the victims, for example on a tablet.

Lightweight

The sensor modules and their microphone arrays can be fitted to many commercially available drones thanks to their scalability. It makes sense to employ multiple drones to search the affected area effectively because both the MEMS technology and the drones are cheap to buy.

Rescue responders can also easily carry the LUCY system because it is lightweight. Alternatively, it can be fitted to land vehicles or used while stationary. The Fraunhofer FKIE researchers are currently working on improvements to the experimental system.