Ammonia Control System Incident

Last week the Chemical Safety Board posted a safety video about hydraulic shock in an ammonia refrigeration system. On Saturday Jake Brodsky posted a very interesting comment about that video over on the SCADESEC mailing list. He noted that part of the cause of the accident as reported by CSB was a control system issue; he suggests in this short comment that this industrial control system (ICS) issue could be used as a method for attacking this type of cooling system.

Background

I will not try to go into a detailed discussion about how this control system works. The CSB did a great job is describing the system in their report on the report on the Millard Refrigeration Services accident that happened on August 23^rd, 2010. I’ll just do a very quick summary here.

Anhydrous ammonia refrigeration systems use circulate liquid ammonia into the heat exchangers inside of the refrigerator/freezer (Reefer) box. As the liquid circulates it picks up heat and turns into a gas (boils). The gas then goes through a condenser and is returned to the liquid state to repeat the process.

Water vapor in the air in the Reefer box condenses on the outside of the heat exchanger (frost) which acts as an insulator and adversely impacts the efficiency of the refrigeration system. Periodically the cooling system is shut down and hot gas is circulated through the heat exchanger to melt the frost. The line is then drained, cleared and then refilled with liquid anhydrous ammonia.

It is very important that the process of draining, clearing and refilling the lines is done properly, otherwise you get the hydraulic shock action that can break open the piping and cause a large anhydrous ammonia release. The Millard release was about 32,000 pounds of anhydrous ammonia that injured almost 150 people out in the open a ¼ mile away.

The Incident

According to the CSB report there had been a 7 hour power outage at the facility. When power was restored there were a number of alarms on the control system letting the operators know what was going on. Presumably because the alarms were obnoxious, someone (not a trained operator) turned them off without noting the problems being reported. One of the problems was there was still hot gas in one of the cooling systems.

With the operators not knowing that there was hot gas still in the system they restarted the filling process without having gone through the required draining and clearing process. The result was a large hydraulic shock to the system that broke a 12” ammonia line. (Again it is a tad more involved than that; see the report for two pages of details).

A Cyber Attack

THIS WAS NOT A CYBER ATTACK. But, a cyber-attack probably could have been designed to accomplish what happened here. Now a lot of the details are going to depend on exactly what type of control system is used in the facility so there is no way to provide a detailed outline of an attack scenario (and I wouldn’t want to in any case). With that in mind here are some techniques that might be used.

First and simplest, replicate the 7 hour power outage. Okay, you probably don’t have to do the whole 7 hours; you would just have to shut the system down during the middle of a defrost cycle. The 7 hour shutdown added command pressure and made things more hectic because management was concerned about food starting to go bad. The key point is that the shutdown has to be done during a defrost cycle. Some sort of denial of service attack to shut down the refrigeration control system or the facility power system may be adequate for this purpose.

The problem from an attackers point of view with this method is that you are relying on operators to do the wrong thing during the re-start and that is an iffy proposition. Of course, if at first you don’t succeed….. At some point though the WC Fields addendum kicks in: “Then stop; there is no use being a damn fool about it.” This is especially true in this type of set-up; the more times you require folks to practice their emergency response plan, the better they will get at performing it.

The more complex, but surer way of pulling off this attack would be to attack the control system and interfere with the ability of the operator to drain, clear and refill the line at the end of the defrost cycle. This means that the attacker would have to reprogram some PLCs so that they operate in manner other than that which was intended and futz some sensor outputs to make it look like everything was operating normally.

This is, of course, why attacking an industrial control system is so difficult. First off you have to understand that you can cause hydraulic shock ruptures of ammonia lines, then:

∙ You have to understand how to optimize the conditions for that shock;

∙ You have to understand what controls work together to get that optimized condition;

∙ You have to understand what the operator is going to expect to see to do their part;

∙ You have to be able to provide that expected information to the operator;

∙ You have to know what safety systems are in operation and how to futz them; and

∙ You have to do all of this while hacking the system.

The one thing that should not escape anyone’s attention here is that the easiest place to do this hack is from the engineering workstation routinely used to work on the control system. And I don’t mean hacking into this from some off site hacker shop, but actually sitting at the terminal, sipping on a cup of coffee and smiling at the people around you. Yep, the infamous insider attack.