How Sloppy Revision Control Cost WIPP Half a Billion Dollars

In 2014 a nuclear Waste Isolation Pilot Plant (WIPP) in Carlsbad, NM was the victim of poor revision control practices that lead to a half-billion dollar mistake; shutting down the facility for three years and sending six workers to the hospital.

That’s right, billion with a ‘B’.

While tragic, this accident presents an important opportunity to assess why good documentation practices matter and how we can proactively work to prevent similar mistakes in the future.

Radioactive Cat Litter

The root of the accident was a poorly described sorbent material used to pack the drums of nuclear waste.

The original packing procedure describes the material as “kitty litter/zeolite clay”, however through a collection of “internal communications,” “kitty litter/zeolite clay” was transformed into “kitty litter (clay).” The result was the improper use a of an organic sorbent (cat litter) in lieu of the intended inorganic sorbent (zeolite clay).

At first glance the substitution seems innocuous, but in the end the mixture proved to be reactive. Pressure and heat built up within the drum until it erupted, exposing half a dozen workers to hazardous radioactive chemicals and costing WIPP half a billion dollars to clean up.

Human Control vs. Process Control

It’s easy to point fingers after an accident, but in order to learn from our mistakes, it’s best to understand the underlying process that allowed the error to occur in the first place.

Human error is inevitable, but effective process control can mitigate our mistake-prone nature.

Sloppy Revision Control

When you hear a phrase like “internal communications,” mental alarms should sound. What exactly does “internal communications” mean? Vague terms like this are professional euphemisms that mask unstructured or sloppy processes.

If we unveil the phrase “internal communications,” we will likely find that it more aptly means — a long, convoluted email chain between technical writers, engineers, managers, and perhaps more. Revision "control" acts more like revision bureaucracy; which over-complicates a simple process.

Email chains have a tendency to get messy. John in HR forgets to select “reply all” and before you know it, there are four different conversations happening simultaneously. Standardizing your communication process will eliminate this ambiguity and allow you to focus on the things that matter.

What it Really Means to Standardize

Most standardization efforts are focused on SOPs, training, or safety procedures.The processes themselves aren’t what cost WIPP half a billion dollars, sloppy revision practices did.

Only Include Relevant Personnel

Co-workers with good intentions offer to provide feedback on document revisions, but adding unnecessary people to the approval workflow makes revising documents more time-consuming than it has to be. Only include people who have relevant background knowledge and interest in the revised procedure; anyone else is unessential. It’s ok to be ruthless here, your co-workers won’t hold a grudge over less work.

Automate Edit History

Have you ever seen one of those puzzles that presents you with two pictures and asks, “how many differences can you spot?” It’s a stimulating game, but not the one you want to play when identifying revisions in a technical document. Changes should be tracked as they are made. Many programs allow you to manually markup or highlight changes, but manually tracking these edits is prone to human error.

Copious annotations on a document also look more like technical graffiti instead of helpful information. Find a way to automate revision tracking in a standard format and link edits to specific document versions for quick reference.

Justify the Changes

Merely documenting the changes isn’t enough. Knowing why certain edits were made is helpful for everyone involved. For those included in the approval process, this can proactively answer questions about edits, thus reducing lengthy email correspondences to explain changes. For operators on the floor, justification provides a reason to implement the changes, increasing the probability that the changes will take effect. Include a justification by writing a note or comment on the document when the new version is released.

Control Document Versions

Changes to any document should be tracked and standardized across all current copies. Keeping documents in a digital format allows you to control versions far more effectively than paper documents. Physical copies are easy to lose and there is no way to verify that outdated documents aren’t still floating around.

Key Takeaway

WIPP was the victim of poor documentation practices, but you don’t have to be. Human errors can be prevented through automation and digital document control. The strategies outlined above will help you avoid documentation that’s littered with mistakes (pun intended) that could cause accidents or reduce efficiency.