March 8, 2020
By: Lyat Avidor Peleg
How to Address Alert Fatigue in the Process Industry Without Turning off Alerts
Over the past few years, a number of studies have revealed that alert fatigue is a real and concerning issue for many industries. With hundreds of sensors producing millions of alerts every day in every plant, the process industry is no exception. Let’s explore the issue of alert fatigue and how to avoid it.
What is alert fatigue?
Alert fatigue is a recognized condition. It’s when people become inured to the seriousness of alerts to the extent that they stop responding to them with the right level of concern, and even stop noticing them at all.
What are the causes of alert fatigue?
Alert fatigue has two main causes:
- When there are too many alerts, it overwhelms the human ability to react
- When there are too many false alarms, it undermines the instinctive reaction that an alert is serious
Alert fatigue is essentially a form of desensitization, like the famous story about the boy who cried wolf. The first time you hear an alert, you probably run to check it out, but if it turns out to be false, then the next time you hear an alert, you might run a little more slowly. By the time that you’re responded to 5 false alarms and the alert goes for the 6th time, the chances are good that you’ll decide to finish off your task before you respond, or perhaps to leave it for someone else to check out.
Alerts for issues of low importance can have the same effect. If you hear an alarm beep or see it flash repeatedly, but 9 out of 10 times it’s for an issue that is neither urgent nor important, you’re a lot more likely to ignore it entirely. Eventually, the perpetual beeps and flashes become part of the background noise of your job.
Bear in mind that it takes time and resources to investigate every alert. 78% of security team leaders across enterprises say that it takes them at least 10 minutes to investigate every alert. The average security responder spends anything from 2.5 to 5 hours each day just responding to alerts. If even half of them turn out to be false, that’s still a significant waste of employee time and a powerful incentive to ignore alarms in general.
Who is affected by alert fatigue?
Alert fatigue is an issue that affects employees in a broad range of industries. Healthcare workers are greatly affected, due to the massive and growing number of IoT healthcare devices. In 2014, a study of a 66-bed ICU unit noted an average of 187 alerts per patient, per day, and the number of IoT devices has only increased since then. The problem is compounded by the fact that most of these alerts are inaccurate, irrelevant, meaningless, or non-urgent. In 2019, researchers found that 80-99% of hospital alerts do not need intervention.
Cybersecurity employees also suffer from alert fatigue. Security systems can generate hundreds of alerts, the vast majority about anomalies that are not serious security breaches. A Cisco study revealed that most security teams can only investigate a little over half of the alerts they receive, and of those, only one-third are found to be valid.
Alert fatigue is a serious concern in process plants, too. Overactive predictive analytics solutions can generate far too many alerts and alarms about anomalies in the system, most of which turn out to be false positive. The graph below from Chemical Engineering shows how many alerts were produced per hour in a chemical engineering plant back in 2009.
The numbers can only have risen in the last decade.
Alert fatigue is a serious concern
Alert fatigue isn’t something to be dismissed lightly. When employees experience alert fatigue, they stop reading alerts, no longer hear them, or even turn them off to get a break from the noise. Alert fatigue is a form of fatigue, first and foremost, leading to increased stress levels, heightened feelings of overwhelm, burnout and high churn rates. 80% of security centers have churn rates of 10-50% for analysts within the past year.
In a hospital setting, alert fatigue can be fatal. The Boston Globe reported cases where patients died because alerts about their situation were ignored due to alert fatigue. Between 2005 and 2008, 560 deaths in the US were due to alert fatigue.
In cybersecurity, alert fatigue can result in hacking and data breaches; breaches at Target and Sony occurred because security teams ignored alerts.
And in process plants, alert fatigue can lead to serious equipment failure or safety issues. One serious incident reported on in E&E News may have occurred because oil drill workers disabled crucial alarms, out of frustration at the number of false alerts they received. It’s vital to battle alert fatigue, but without doing away with alerts entirely.
How Precognize combats alert fatigue
There’s plenty of advice about how to avoid or reduce alert fatigue, but the best approach is to decrease the number of alerts, and find ways to make them consistently accurate and important so that employees feel that every alert does matter.
Precognize strives to reduce unnecessary alerts and decrease noise. The first step in implementing our predictive analytics solution is to work together with an expert in the plant to create a domain model, dividing up the plant into smaller regions. In this way, Precognize is able to focus on only the most important groups of sensors, aggregating anomalies, defining a more accurate standard of “normal,” and producing only relevant alerts relating to real and serious issues.
The end result is that Precognize produces fewer alerts, with an average of 2-3 per day. One client received 212 in a full year, fewer than one each day. The effort of investigating just 2-3 alerts every few days is not severe, and with fewer alerts, a higher percentage is relevant to the plant.
The graph below shows a screenshot from within the Precognize application, which breaks down the 212 alerts by their category.
Precognize reduces alert fatigue without switching off alerts
As you can see, alert fatigue is a serious condition that affects analysts, responders, and employees in all industries. The impact of alert fatigue can be immense and even life-threatening at times, including in process plant situations. It’s worth it to use a solution like Precognize that ensures that alerts are kept relevant, and reduces the number of alerts to more manageable levels.
To learn more about how SAM GUARD prevents alert fatigue: Request a Demo