Do too many warnings cause problems?

May 2017

Earnsy Liu, TechCommNZ member and GDID student, looks for evidence (not just opinions) to help you manage the daily conundrums we face in our profession. If you have a question for Earnsy to tackle, please email comms@techcomm.nz. This month, Earnsy investigates whether too many written warnings in manuals cause problems.

There seems to be enough concern about excessive warnings, but it was hard finding evidence.

Concern aplenty

These are signs of concern:

• The US Food and Drug Administration (FDA) planned a study to see if the risks of prescription drugs in TV ads could be shortened. The FDA feared the lists of risks were often long and could hinder consumers’ understanding and sensitivity to dangers (Warren, 2014). The study was announced in 2014 but sadly I haven’t been able to find out if it took place.
• Investment companies in New Zealand have been told to publish only the likely risks in their investment statements, instead of all possible risks (Mulligan, 2017).
• Doctors are concerned patients may stop taking potentially life-saving drugs because of the risk of side effects. Patients ask, ‘This drug can cause 100 things. Why did you prescribe that to me?’ (Mann, 2016).
• Someone asked on a TechWhirl forum, ‘At what point do cautions and warnings become counterproductive because people start dismissing them?’ (Downing, 2008).

Limited evidence

Despite the concern, there is no conclusive evidence of harm caused by excessive warning lists. Consumer New Zealand says the problem is getting people to read manuals, not the warnings themselves. The lady I spoke to had dealt with 20,000 complaints but none about excessive warnings in manuals (personal communication, Mar 09, 2017).

Research tends to be about product and drug labels, or the design of warnings, instead of the number of warnings per se. There is, however, pertinent feedback from space station crew, research on how we read lists, and opinions in court rulings.

Feedback from space station crew

International space station crew have complained of excessive warnings in procedures: repetitive warnings, and even warnings about things that were ‘intuitive and well understood by the crew, e.g. scissors are sharp’ (Rando, 2007, p. 1436). This desensitised the crew to warnings, leading to situations where important warnings have been overlooked.

To determine if the five most commonly identified warnings were indeed ‘common’ and ‘intuitive’, NASA evaluated their procedures for usability. Twenty-four crewmembers were given a procedure minus the five common warnings and asked which steps should have had warnings. Answers were correct at least 70% of the time, validating crewmembers’ belief that the warnings were indeed common and intuitive (the 70% cut-off was based on industry standards and other factors).

Although the article does not provide details, the overlooked warnings and NASA’s desire to ‘help prevent future incidents of human error’ (p. 1436) by conducting research make me wonder if there had been near misses or mishaps.

Research on lists

Results from food tasting tests show that longer lists are not read as well, which might mean we are less likely to heed long lists of warnings. Researchers in Uruguay gave five samples of crackers to 124 participants, telling them to tick the terms that described the samples (called a CATA task: check all that apply). Twelve or twenty terms such as hard, thin, tasteless, crunchy, and greasy were presented in a two-column list. List order varied for each participant and each sample. Participants were aged 18-65 years and were evenly split between genders.

The eye-tracking study found that participants:

• looked at individual terms more often and for longer when lists were shorter.
• looked at terms in top-left quadrant more often and for longer than terms in other quadrants, especially bottom-right quadrant (which makes sense to me, as it fits with the F-pattern of reading). This was true for both short and long lists.

As the test progressed, participants also spent less time looking at terms when lists were short, but not when lists were long, suggesting that they became more familiar with shorter lists but not longer ones (Ares, 2015).

Opinions in court rulings

US courts have ruled on excessive warnings. For example, California’s Supreme Court has mentioned the risks of overwarning for medical products several times, and large pharmaceutical companies have been accused of hiding warnings amongst others (Bexis 2007, 2014). Concerns include (quotes are citations from court rulings in Bexis, 2014):

• Increased time and effort to understand warnings

The ‘primary cost [of overwarning] is . . . the increase in time and effort required for the user to grasp the message. . . ’

• Risk of users ignoring crucial warnings

‘Overwarning effectively amounts to the manufacturer “crying wolf.” The more that product manufacturers warn of risks that never materialize, the less likely product users are to heed those warnings. If product users merely ignored the excessive warnings, the problem might be minimal − the only superfluous costs would be those of providing the warnings. However, product users might begin to ignore not only the excessive warnings, but also those that are crucial to safe product use.’

• Reduced effectiveness of warnings

‘Inclusion of the unnecessary and the inane can reduce the efficacy of warnings by diminishing the significance of warnings about non-obvious, not-generally-known risks…’

• Sensory overload

‘Excessive warnings on product labels may be counterproductive, causing “sensory overload”…’

As an aside: a study of 5,600 drug labels found 70 potential side effects on average – one label carried 525 (cited in Mann, 2011)! If each side effect could be written on a single line, the side effects would stretch for nearly twelve A-4 pages. Not a read for the faint-hearted.

• Poor decision making

‘To allow a warning based on inconclusive evidence or scientific hunches results in doctors not prescribing effective drugs to a patient because of the erroneous belief that a side-effect might occur. Alternatively, physicians may tune out if overwhelmed with risk information. In either case, the result may be suboptimal treatment choices.’

As far as I could tell – and I admit I struggled to understand the cases – none of the above was based on what actually happened, just what the judges thought was likely to happen.

Conclusion and recommendations

Is there evidence that excessive warnings cause harm? Not that I’ve found. We may want to think they do, and we may well be right, but the evidence isn’t there. Perhaps the answer is assumed to be obvious enough that it doesn’t need researching?

It’ll be interesting to watch developments on the health and safety front. The tougher Health and Safety at Work Act 2015 seems to have companies worried. Will that worry be manifested in lengthier manuals or more detailed policies and procedures? Will we hear cries of ‘there’s too much to read!’ and stories of things going wrong as a result?

Whatever it is, the concern alone warrants paying attention to how we write warnings. These strategies might make them more palatable:

• Don’t warn about the obvious, only what’s ‘reasonably foreseeable’ (Robinson, 2004; Wiens & Bluff, n.d.).
• Make safety part of product design so there’s no need for warnings (Robinson, 2004; Hart, 2008).
• Group similar warnings, for example, electrical hazards vs flammable vapour hazards. Grouping creates shorter individual lists, which are more likely to be read. Put headings on those groups and you’ve got signposts that help understanding and navigation. Separate product-specific from general warnings so users can decide which to read; or they might skip everything, thinking they are already familiar with the warnings (Robinson, 2006).
• Consider usability tests to determine what is needed.

References

Ares, G., Antúnez, L., Giménez, A., & Jaeger, S. R. (2015). List length has little impact on consumers’ visual attention to CATA questions. Food Quality and Preference, 42, 100-109. Retrieved from Academic OneFile database.

Bexis. (Jan 09, 2007). The apocalypse is upon us (overwarnings). Retrieved Feb 18, 2017, from Drug & Device Law website: https://www.druganddevicelawblog.com/2007/01/apocalypse-is-upon-us-overwarnings.html

Bexis. (Jan 09, 2014). On overwarning. Retrieved Feb 18, 2017, from Drug & Device Law website: https://www.druganddevicelawblog.com/2014/01/on-overwarning.html

Downing, D. (Dec 30, 2008). How many warnings are too many? [Online forum]. Retrieved Feb 02, 2017, from http://www.techwr-l.com/archives/0812/techwhirl-0812-00560.html

Hart, G. (Dec 30, 2008). How many warnings are too many? [Online forum]. Retrieved Feb 02, 2017, from http://www.techwr-l.com/archives/0812/techwhirl-0812-00561.html

Mann, D. (2011). Does your medication’s label lead to information overload? Retrieved Mar 13, from WebMD website: http://www.webmd.com/drug-medication/news/20110527/drug-label-overload

Mulligan, J. (Presenter). (2017, Mar 02). Money with Mary Holm. Jesse Mulligan [Radio interview]. Retrieved from RNZ website: http://www.radionz.co.nz/national/programmes/afternoons/audio/201835187/money-with-mary-holm

Rando, C. M., Patel, D. G., Duvall, L. E., & Martin, L. (2007, October). Use of cautions and warnings within International Space Station procedures: When too much information becomes risky. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 51, No. 20, pp. 1435-1438). Sage CA: Los Angeles, CA: Sage Publications.

Robinson, P. A. (2004). Warnings that work: four keys to effective warnings. Retrieved Feb 02, 2017, from https://www.jurispro.com/uploadArticles/Robinson%20-%20Warnings.pdf

Robinson, P. A. (2006). Beyond labels: Product safety information. Retrieved Feb 02, 2017, from http://www.coronadoconsulting.com/PDFs/Beyond_Labels.pdf

Warren, J. (2014). FDA looks into limiting TV commercials’ rambling lists of drug risks. Retrieved Feb 04, 2017, from In New York Daily News website: http://www.nydailynews.com/news/national/fda-studies-limiting-tv-ads-drug-risk-lists-article-1.1617940

Wiens, K., & Bluff, J. (n.d.). Chapter 9: Legal requirements. In Tech Writing Handbook. Retrieved Feb 04, 2017, from http://www.dozuki.com/Tech_Writing/chapter/9