03.06.2024 • Newsport securityAI

AI Project to Keep Danish Ports Safe

Despite their beauty and functionality, Danish harbours have unfortunately witnessed numerous drownings over recent years. From 2001 to 2015, a staggering 1,647 people died in Danish waters, with a quarter of these tragedies occurring within the ports themselves. An innovative solution was needed to combat this alarming trend.

© Milestone Systems

That’s when a remarkable team of innovators in video technology, including Actas, a Milestone technology partner, and Aalborg University (AAU) in northern Denmark, came together.

With sponsorship from the Danish Tryg Fonden, they embarked on a ground-breaking project to combine artificial intelligence (AI) with video technology to enhance safety significantly in the Port of Aalborg – one of Denmark’s busiest ports.

Transforming Tragedy into Progress

At the heart of the project was the development of the AAU/Milestone thermal dataset, comprising the largest published collection of annotated thermal images. By training AI models on this dataset, the goal is to create an advanced video analytics solution that will be able to promptly detect and alert rescue teams when a person falls into the harbour. This article explores the ground-breaking collaboration that has the potential to redefine safety standards in ports, ultimately saving countless lives in years to come.

The primary challenge faced by the team was training the AI system to identify instances when a person falls into the harbour. Training an AI system is akin to teaching a child to recognise objects or comprehend language. We show the child numerous pictures of dogs, enabling them to learn the defining characteristics of a dog.

Similarly, when training AI, we provide it with abundant data, in this case videos of people falling, enabling it to identify patterns associated with falls. The more data we provide, the more accurate the AI becomes in recognising falls. Unlike a child, an AI system necessitates thousands or even millions of data points and images to learn effectively.

Volunteers Take a Dip into Danger

To train the AI in detecting individuals falling into the harbour, volunteers initially simulated falls for analysis. However, due to safety concerns, they could not ask volunteers to actually fall. Instead, they were asked to jump, which is a different pattern of motion. It is important to note that genuine falls involve sudden and involuntary movements that were not captured in this data, rendering it less effective for AI training.

An AI model requires a diverse dataset to detect different types of falls consistently across various conditions and seasons.

The team set out to cut the 1,650 deaths in 15 years in Danish waters. Their...
The team set out to cut the 1,650 deaths in 15 years in Danish waters. Their biggest challenge was training the AI system to detect when a person falls into the harbour using test dummies
© Milestone Systems

For example, a fall in winter may present unique challenges due to the individual wearing bulky clothing. However, the potential risks associated with volunteers jumping into the freezing water of winter were too great, so an alternative approach was needed.

Test Dummies, a Safer Way to Train AI

Using a test dummy proved to be a safer and more effective method of collecting data. Instead of putting human lives at risk, researchers used a test dummy filled with warm water, making it detectable by thermal cameras, to simulate multiple falls into the harbour. This allowed the team to simulate many different types of falls, resulting in a more comprehensive dataset.

However, this approach also had its limitations. The dummy could only imitate human movements to a certain extent and lacked the complexity of a real person falling into the harbour. Additionally, gathering a large amount of data was unfeasible as simulating a person falling into the water many times with slight variations was not a viable option.

Scaling up with Synthetic Data

Initially, the team used thermal cameras to capture real images over eight months to create a thermal dataset. The resulting dataset is the largest collection of annotated thermal images ever published, with approximately 8 million bounding box annotations.

It covers four classes: person, bicycle, motorcycle and vehicle; however, luckily it does not include many examples of abnormal behaviours – including falls. Using synthetic data, the goal was to expand the thermal dataset by including many abnormal behaviours and incidents.

The AAU/Milestone research team took real images with thermal imaging cameras...
The AAU/Milestone research team took real images with thermal imaging cameras over an eight-month period in the port to create the largest collection of annotated thermal images ever published
© Milestone Systems

Synthetic data is a technique to provide additional data for an AI’s comprehension and identification of real-world events, such as when an individual falls into a harbour.

At AAU, it originated as a student project aimed at generating an artificial depiction of a person falling. However, the initial simulation was simplistic, and it soon became evident that these examples were inadequate.

Innovative Partnership to Save Lives

Recognising the need for an advanced dataset, Milestone Systems’ research department joined forces with AAU to extend the synthetic dataset. The drive for better and more precise synthetic data was not solely focused on improving an AI algorithm; it was about saving lives.

The training data was enhanced to simulate scenarios with more intricate behaviours, such as falls from wheelchairs, skating and biking. Each enhancement to the data had the potential to increase the system’s ability to detect real-life falls swiftly and accurately.

The innovative partnership between Milestone Systems and AAU has harnessed the power of synthetic data and AI to tackle the significant issue of drowning in Danish harbours. The team’s objective was to develop a comprehensive dataset capable of training an AI system to detect a wide range of hazardous situations.

The development of the AAU/Milestone thermal dataset represents a ground-breaking step forward in video technology. Ultimately, the goal is to help rescue teams respond faster and potentially reduce future drowning incidents.

Business Partner

Milestone Systems A/S

Banemarksvej 50 G
2605 Broendby

Business Partner contact

most read

12.04.2024 • NewsSecurity

Secutech Thailand 2024 Confirms Dates

Organisers of Secutech Thailand have confirmed that the 2024 edition will return to the Bangkok International Trade & Exhibition Centre (BITEC), Bangkok, from 30 October to 1 November 2024.