Camera-based incident detection has become an established technology to support tunnel operators in keeping their roads safe and in organizing fast response. After the introduction of thermal cameras as a solution for 24/7 automatic incident detection, systems supported by Artificial Intelligence (AI) may well be the next wave.
Today, AI is already bringing incident detection to an even higher performance level. sensors and cameras
In February 2023, the European Commission published preliminary figures on road fatalities for 2022. Around 20,600 people were killed in road crashes last year, a 3% increase on 2021 as traffic levels recovered after the pandemic. Although this is 10% below the fatality number of 2019, one year before the pandemic, the EU still has an important target to achieve, as it intends to halve the number of road deaths by 2030.
All this to say that the fight against traffic deaths is far from over. Today’s tunnel operators are fully aware of this. Although research tells us that there are typically less incidents in tunnels compared to open roads, it is also known that the severity of the incident is higher. Prevention of these incidents is one thing, fast response in case of an incident is another one. In 2004 already, a European directive on road tunnel safety prompted the roll-out of incident detection systems in tunnels, and since then the technology evolution has not stopped.
Technology leaders like Teledyne FLIR, who has been in the game for 30 years, have been crucial in this evolution. With the ongoing advancement of video analytics based systems, tunnels have indeed become safer. In case of an incident, first response teams can now be deployed minutes, even seconds after an incident or an irregularity (a fallen object, a pedestrian, a car slowing down) has been detected.
The thermal revolution
Ten years ago, thermal incident detection cameras in tunnels were the new kid on the block. Today, thermal cameras have become an established technology, even a critical asset for operators to guarantee accurate detection throughout the tunnel infrastructure. The use of thermal imaging cameras has especially proven valuable for tunnel entrances and exits. There, shadows or direct sunlight can obstruct the view of visible-light cameras and therefore disturb traffic detection. Because they detect heat, not light, thermal cameras have no issues with these phenomena and as a result, they can detect traffic 24/7 and in all weather conditions.
One of the biggest advantages of thermal imaging cameras in the field of tunnel safety is that thermal cameras can effectively see through many types of smoke. This makes them the ideal technology for tunnel safety operators or emergency response teams to find their way through a smoke-filled tunnel or for incident detection systems to spot incidents in time.
Thermal cameras are also not affected by headlights. Conventional CCTV systems typically have trouble with this. Headlights can generate false or missed calls, and make accurate observation of highway traffic at night impossible. At night, a road can look like an indistinct row of lights to a video camera, making meaningful data collection and incident assessment impossible. But thermal cameras see the heat signatures of vehicles clearly from miles away. Thermal cameras are also an ideal solution to detect fires in tunnels in an early stage, even within seconds of the appearance of visible flames. This allows traffic operators to immediately close the tunnel and take action to quickly extinguish the fire.
Both visual and thermal cameras have their merits. A visual camera may provide operators with more detail to assess the nature of an incident, while thermal cameras have proven to be unbeatable in detecting incidents in more challenging weather conditions. Today, both detection technologies are often combined into one system, hereby offering operators the best of both worlds. Teledyne FLIR’s ITS-Series Dual AID camera is an example of such a system.
The next wave: artificial intelligence
With AI being so prominently in the news lately, it’s almost impossible to ignore the huge contribution this technology is making to incident detection. Even more, it’s safe to say that AI-powered detection systems are the next big wave in tunnel safety. It’s not hard to see where it’s coming from. Over the past decades, computing speeds have increased, prices dropped, and the exponential growth of data has worked as fuel for making AI better and more efficient. Instead of conventional, rule-based detection systems – if x happens, then y – data-based systems are now in the lead. These systems can be trained on large datasets of images, and learn how to identify and classify objects in an image. They use this knowledge to make decisions based on new images that they have never seen before.
Previous generations of traffic detectors have always been looking at subtle color differences in the image on pixel level. Today’s AI-powered detectors are looking at the entire camera image to make ‘intelligent’ decisions. This has major repercussions. In short, AI systems can handle more complex traffic situations, and they are much better at making smart predictions.
AI-powered systems outperform non-AI detectors in at least three ways. First, the accuracy of detection is much higher with AI detectors. For tunnel operators, this is a big deal. Nothing is a bigger nuisance for control room operators than having to pay attention to continuous unwanted alarms. Even though conventional systems already managed to achieve good detection rates, AI-powered detectors can take away even more of the frustration caused by unwanted alarms. And then we haven’t even mentioned response teams that no longer have to be sent out erroneously because of unwanted alarms.
AI-based systems are more successful in detecting more different vehicle classes. Detectors from Teledyne FLIR will easily distinguish between a car and a van, or between a small and a large truck. The system can even be taught to recognize customized classes. It only needs to be fed with new data and trained. AI-based systems will also distinguish more easily between a fallen object and materials used in roadworks. With cameras so smart, installers nowadays are more flexible in installing their equipment. The position of camera units is no longer limited to a certain height. Even in less ideal camera positions, the detection performance of AI-based systems is high.
Secondly, AI detectors are better in predicting trajectories. Based on vehicle parameters such as speed and direction, they can easily see where a car is going, even if for part of that trajectory the view on that car is occluded by a passing truck. This makes detection much faster and more accurate. Operators can even be warned by so-called pre-alarms for cars that are slowing down and likely to cause a collision. Traffic detectors from Teledyne FLIR can also keep an alarm active until the situation is cleared, for example until a stopped car has left, a fallen object has been removed, or a pedestrian has left.
Another promising application of AI-based systems is that they enable setting up a digital twin of the tunnel environment. The digital twin is a concept that has already been popular in manufacturing and production industries for many years. But instead of the virtual, real-time representation of business processes, tunnel operators can use the digital twin to get a complete overview of all traffic operations and activities in the tunnel. A tunnel is a complex collection and collaboration of different systems and technologies, including safety management, ventilation, VMS system, communications, and many more. A digital twin establishes the exchange of data between these physical tunnel systems and their virtual representation. Based on this wealth of data, a digital twin can generate a real-time bird’s eye view of the traffic inside the tunnel, offering operators an invaluable source for decision-making.
Teledyne FLIR: making an impact with AI
Data-based detection systems will become the norm very fast. So, it’s easy to understand that a system’s detection performance will soon be determined by the quality of the data it is trained with. High-performance systems will need a lot of data for training – in our case video images of traffic – but not just any data that has been plucked from the internet.
The problem with publicly available internet data is just that: it’s often of low quality and quite one-sided. For example, internet data sets hardly have any thermal images, they are mostly collected during sunny weather or friendly daytime conditions, and they are often taken from useless angles. Which brings us to the strength of Teledyne FLIR, a company with a heritage of 30 years’ in-house data collection. If data is gold, then Teledyne FLIR is sitting on a big pile of it. For Teledyne FLIR, data is the true quality mark which stands for high-performance, accurate traffic detection.
AI-based camera products like Teledyne FLIR’s TrafiBot series also have their AI intelligence embedded into the camera. This means that it’s not necessary to send camera data over the network to a central server or cloud service for processing. As a result, the network is not overloaded when there is no detection, and detection can happen with much less latency. AI solutions from Teledyne FLIR also have redundant connections to the tunnel’s PLC system, which means they will continue to detect even when the network is down.
Today, we’re just at the beginning of a new wave of AI-based systems. No doubt, detection performance will improve even more in the future, especially in the difficult corner cases, where conventional systems often reach their limits. And what to expect when traffic operators decide to join forces by connecting their tunnel and highway data with sensors from city traffic? Will we see an even more complete and robust image of our traffic? In any case, there’s a great future ahead for AI in ITS.
www.teledyneflir.com