Chemical Mixing Incident

Earlier this week the Chemical Safety Board (CSB) announced that it was sending an investigation team to the site of a major chemical release in Atchison, KS that occurred on October 21^st. The release was caused by the inadvertent mixing of two common industrial chemicals and resulted in a large chemical cloud sending hundreds to local hospitals with complaints of difficulty breathing.

The Incident

According to news reports (for example see here, here and here) the incident started at 8:00 am when a bulk chemical delivery was put into the wrong storage tank. The two chemicals involved were industrial strength bleach and sulfuric acid, both apparently being used in the facilities waste treatment plant. The chemical reaction between the two produced a large cloud of steam that also included chlorine gas, a byproduct of the reaction between the two chemicals.

There is no publicly available information about which chemical was being unloaded, but due to the odor of chlorine bleach being involved, I would guess that the delivery was sulfuric acid. Adding sulfuric acid to a bleach tank actually produces two separate reactions that would have contributed to the cloud.

First, since bleach is mainly water (only 6 to 12% sodium hypochlorite) the addition of sulfuric acid (which is typically shipped and stored at concentrations above 95% for safety reasons) produced a large amount of heat due to the ‘heat of dilution’. That heat and the lack of mixing would quickly raise the surface temperature of the bleach above the boiling point of water producing a large steam cloud. That steam cloud would be expected to contain trace amounts of unreacted bleach and sulfuric acid.

The chemical reaction between sodium hypochlorite and sulfuric acid produces chlorine gas and even more heat. The reaction is virtually instantaneous and consumes essentially all of which ever chemical is least available (typically the chemical being added to the tank because addition is usually stopped as soon as the steam cloud is observed). That is why I suspect that the sulfuric acid was being added to the bleach tank.

How Could This Happen?

This type of accident is way too common, especially at waste water treatment facilities. Such facilities typically rely on delivery drivers to unload bulk chemical shipments instead of facility personnel who would be more familiar with which tank contains which chemical. Hose connections are made from the delivery truck to piping that leads to the chemical storage tank. A single bulk truck unloading station typically has separate connections for each of the storage tanks at these facilities. Inadequate marking of the pipe connections, and/or inexperienced (for that facility) drivers results in the truck being hooked up to the wrong piping connection.

Larger chemical facilities avoid these types of incidents through a combination of personnel and design activities vetted through a chemical safety program under either the EPA’s Risk Management Program (RMP) and/or OSHA’s Process Safety Management (PSM) program. Typically, there are only a limited number of personnel on-site who are authorized to unload bulk deliveries of chemicals. They are specifically trained on the hazards associated with the bulk chemicals they will be handling, including the risks associated with mixing of chemicals in storage tanks, bulk unloading lines or hoses. Non-facility delivery drivers are never allowed to unload bulk chemicals without specific facility supervision.

Where there is a specific hazard from the mixing of chemicals being stored at that site (for example bleach and sulfuric acid) engineering measures are taken to prevent that mixing. The tanks may be located in separate tank farms, the bulk unloading lines may be physically separated at different unloading stations, or different types of hose connections are used with the unloading lines to make it more difficult to inadvertently mix those chemicals. Depending on the potential consequences involved (and this particular incident was nowhere near a worst-case incident) combinations of these and other engineering controls could be used.

The CSB Investigation

The CSB usually limits its investigations to larger more severe events that kill people or result in large scale damage. This is mainly due to their Congressional mandate, limited funds and limited personnel. Taking up this incident is almost certainly due to the amount of level of publicity related to the large cloud and how common this type of incident is.

Compared to other investigations this one should consume much less in the way of CSB resources. That does not mean that the report will be completed and published any sooner; the CSB will likely place a low priority on the completion of this investigation.

Public ICS Vulnerability Disclosures – 10-29-16

This week saw a public disclosure of a control system security vulnerability at the 2016 Industrial Control Systems (ICS) Cyber Security Conference (the old Joe Weiss conference under new management). Indegy CTO Mille Gandelsman presented a talk, “Ghost in the Machine: SCADA Vulnerability Enables Remote Control of ICS Networks”, about a vulnerability in the Schneider UnityPro software platform. This was a coordinated disclosure with Schneider publishing a Security Notification concerning the vulnerability.

Reading the Indegy blog post about this vulnerability and then looking at the Schneider notification, it almost looks like the two organizations are looking at two separate vulnerabilities. Indegy describes the vulnerability consequences this way:

“The vulnerability in Unity Pro allows any user to remotely execute code directly on any computer on which this product is installed, in debug privileges. The vulnerable software tool is present in every control network in the world that uses Schneider-Electric controllers. Regardless of the SCADA/DCS applications in use, if Schneider Electric controllers are deployed, this software will be used on the engineering workstations. This makes this attack relevant across virtually any process controlled by these PLCs. Since Schneider Electric is one of the largest industrial control equipment providers, this vulnerability is a major concern.”

Schneider simply notes: “This vulnerability is made possible when no application program has been loaded in the simulator or when the application program loaded in the simulator is not password protected.”

Schneider has produced a new version of the software that mitigates the vulnerability. They still note that: “It is up to user responsibility to protect his application by a proper password.”

Schneider published their notification on October 14^th and the Indegy presentation was made on October 25^th. ICS-CERT has not yet reported on this vulnerability, though it has been widely reported in the press (see for example here and here).

DHS Publishes New CFATS Fact Sheet

Today without any specific notice the DHS Infrastructure Security Compliance Division (ISCD) published a link on the CFATS Knowledge Center for a new Chemical Facility Anti-Terrorism Standards (CFATS) Fact Sheet ‘Documentation’ section. This is not the latest version of the statistics on the implementation of the CFATS program (each also called ‘CFATS Fact Sheet’ by the way) that I routinely report on. Rather, it looks like a replacement for the 2012 ‘CFATS Trifold Brochure’. The link to that brochure is still active as of 22:30 EDT. The tri-fold brochure link has been removed from the ‘Documentation’ section of the CFATS Knowledge Center.

The CFATS Fact Sheet has also been printed as an article in the frequently asked questions section of the CFATS Knowledge Center. It can be found as Article 1775.

Nothing really new here. The new documents briefly describe the CFATS program. There is a brief new mention of the 2014 CFATS authorization legislation, but no discussion of the changes brought about by the law. Neither is there any mention of the on-going CSAT 2.0 implementation process.

ICS-CERT Publishes Honeywell Advisory and ICS DDOS Warning

Today the DHS ICS-CERT published a control system security advisory for the Honeywell Process Knowledge System (PKS). They also issued a warning about the potential for distributed denial of service (DDOS) attacks on internet facing industrial control system products.

Honeywell Advisory

This advisory describes an improper input validation vulnerability in the Honeywell Experion Process Knowledge System (PKS) platform. This is apparently a self-reported vulnerability. Honeywell has produced patches to mitigate the vulnerability.

ICS-CERT reports that a moderately skilled attacker could remotely exploit this vulnerability to prevent the Experion PKS client tools from uploading firmware to Series-C devices.

ICS DDOS Warning

ICS-CERT posted a very short and very generic warning about the potential for DDOS attacks on internet facing control systems or components thereof. This is based upon the US-CERT report about recent very large DDOS attacks. There is no information provided that indicates a specific threat against ICS.

ICSJWG Spring Meeting

ICS-CERT recently published a notice concerning the date of the 2017 Spring meeting of the ICSJWG in Minneapolis, MN over April 11^th thru 13^th, 2017.

ICS-CERT Publishes Siemens SICAM Advisory

Today the DHS ICS-CERT published a control system security advisory describing a denial-of-service vulnerability in Siemens SICAM products. The vulnerability was reported by Adam Crain of Automatak LLC. Siemens has produced a firmware update to mitigate the vulnerability. There is no indication that Adam has been provided an opportunity to verify the efficacy of the fix.

ICS-CERT reports that a relatively unskilled attacker could remotely exploit this vulnerability to cause a denial of service. The Siemens Security Advisory reports that the vulnerability exist in the SM-2558 and SM-2556 IEC 60870-5-104 COM Modules used in the SICAM products.

Siemens announced their advisory on TWITTER® last Friday.

TSA Publishes 60-Day ICR Renewal Notice

Yesterday the DHS Transportation Security Administration (TSA) published a 60-day information collection request (ICR) renewal notice in the Federal Register (80 FR 73126-73127) to support the TSA’s Transportation Worker’s Identification Credential (TWIC) application and security assessment program. The request includes changes in the burden estimate and description of the affected individuals covered by the ICR.

Burden Estimate

The notice provides the numbers that will be included in the renewal that will be submitted to the OMB’s Office of Information and Regulatory Affairs (OIRA) after the public notice process is complete. There is no comparison provided with the currently approved ICR. The table below provides that comparison.

	Current	Proposed
Burden Hours	824,877	736,670
Burden Cost	$118,700,668	$90,276,808

There is no specific explanation in the notice for the changes in the burden hours or burden cost. When no explanation (or even identification) of a change is made it is extremely difficult for the public to comment on the appropriateness (of either the magnitude or the necessity) of that change.

Other Changes

The notice changes the description of the information collection requirement. The new description now specifically mentions the Chemical Facility Anti-Terrorism Standards (CFATS) program:

“Also, individuals in the field of transportation who are required to undergo a security threat assessment in certain other programs, such as the Chemical Facility Anti-Terrorism (CFATS) program, may apply for a TWIC® and the associated security threat assessment to satisfy CFATS requirements.”

This addition reflects the recent notice the TSA’s description of the term ‘field of transportation’ that I have previously discussed. This had been alluded to in the justification memo that TSA submitted in supporting the currently approved ICR:

“There are also some worker populations in the non-maritime environment who may be authorized/required by TSA to obtain a TWIC given the nature of their work and required access to controlled areas/facilities.  These individuals would be required to complete the same enrollment process as the TWIC-maritime population.”

It will be interesting to see what estimates (if any) TSA provides for the number of CFATS related TWIC submissions. Again without TSA providing an explanation of how this change reflects the burden estimate, it is difficult to comment on this change in the burden.

The notice also briefly notes the following changes in the ICR:

• To expand enrollment options and the potential use of biographic and biometric (e.g., fingerprints, iris scans, and/or photo) information;

• To remove the requirement to collect information about the Extended Expiration Date (EED) TWIC; and

• To revise the fee collection for the TWIC® Program in light of changes to the fee the FBI charges for fingerprint processing (reduction of $2.75 per TWIC submission).

There is no explanation of how these changes will affect the burden estimates. TSA is not alone in their failure to provide detailed explanations for how changes in an ICR will affect the burden estimate; many (most) of the ICR notices that I review also do a poor job of explaining what they are doing. It makes providing effective comments on such notices very difficult. I do not really think that the agencies are trying to avoid having to respond to comments. It is just simpler to create short, boiler-plate ICR notices. And most of the public does not pay attention to ICR notices in any case.

Public Comments

The TSA is soliciting public comments on this ICR. Comments may be submitted via the Federal eRulemaking Portal (www.Regulations.gov; Docket # TSA-2006-24191). I will be submitting a copy of this blog post as comment.

CSAT 2.0 Update – 10-21-16

Yesterday the DHS ICS-CERT made some additional changes to the CFATS Knowledge Center page in support of their on-going implementation of revisions to the Chemical Facility Anti-Terrorism Standards (CFATS) program’s Chemical Security Assessment Tool (CSAT) known as CSAT 2.0. Those changes centered on removing links to the older CSAT documents in the ‘Documentation’ section at the bottom of the page.

Documents Removed

The documents removed included:

• CSAT SSP Edit Process User Guide;

• CSAT Account Management User Guide;

• CSAT Security Vulnerability Assessment Questions;

• CSAT User Registration User Guide;

• CSAT Security Vulnerability Assessment Application Instructions;

• CSAT Site Security Plan Screenshots;

• CSAT Site Security Plan Questions; and

• CSAT Site Security Plan Instructions

This links were still good as of 7:30 EDT this morning. If you need copies of these documents for historical purposes you need to get them as soon as possible. There is no telling how long these documents will remain.

Old Documents Still Remaining

The ‘Documentation’ section of the CFATS Knowledge Center has been getting kind of bloated over time. There are a number of useful documents listed here that cannot be found anywhere else on the site without the links provided here. There are, however, a number of documents that could still be removed. Those include:

• CFATS Quarterly, Nov. 2015;

• CFATS Fact Sheet (July 2015);

• 2015 Chemical Sector Security Summit;

• CFATS Fact Sheet (June 2015);

• CFATS Personnel Surety Program 30-Day Notice Fact Sheet (PDF, 2 pages - 141 KB);

• Personnel Surety 60-Day Notice Fact Sheet (PDF, 2 pages - 283 KB);

• CFATS Personnel Surety Program Information Collection Request 60-Day Notice; and

• Chemical Facility Anti-Terrorism Standards

The CFATS Quarterly is not the most recent (the April 2016 issue is also listed and I have not seen anything more recent). It might be nice to have a CFATS Quarterly web page where links to all of the issues might be found, but only if these are going to be released on a routine (quarterly?) basis. The two CFATS fact sheets are from somewhere in the middle of this series of documents; a similar historical web page might be of limited use. The personnel surety program (PSP) documents are dated, they could be better listed on PSP website under an historical documents listing. And the last is a link to a website not a ‘document’ and that page is already listed on the CFATS landing page.