FRA PTC Final Rule to OMB

According to the Office of Management and Budget (OMB) web site, the Federal Railroad Administration (FRA) has submitted their final rule for their amendments to the Positive Train Control (PTC) regulations. The FRA is moving fairly quickly with this rule as the comment period on the NPRM closed in late October of last year.

Readers will remember that I expressed some concerns regarding the amendments relating to the specific wording of the revisions to the regulations governing the removal of track from the PTC regulations based upon the cessation or expected cessation of TIH traffic on a particular line prior to January 1^st, 2016. It will be interesting to see if that wording gets changed.

S 2151 Published – SECURE IT

The GPO now has the official version of S 2151, the Strengthening and Enhancing Cybersecurity by Using Research, Education, Information, and Technology (SECURE IT) Act of 2012, available on its web site. No significant changes from the version I reported on earlier.

Replacing PLCs

There’s an interesting blog post over at TofinoSecurity.com that looks at the cost of SCADA security. The author, Frank Williams, identifies the need for risk assessment as a key starting point for making decisions on security upgrades, but doesn’t go into a great deal of detail on how that is done. I have an interesting thought experiment here for chemical facilities that addresses that issue in a little more detail.

ACME Control Systems PLCs

Our friends at ACME Control Systems have just introduced a new line of high-security PLCs (HSPLC). These PLCs have a number of interesting design features that help to ensure that when installed in a properly designed control system with a secure communications system that the operations of the PLCs are protected against unauthorized changes to their operation and provide security system alerts when such unauthorized changes are attempted.

Along with the HSPLCs ACME has an upgraded high-security HMI and a secure Ethernet communications protocol that they are offering as an upgrade to their standard control system. The upgrades on the HMI and Ethernet systems are modestly priced, but each new PLC costs about four times the current installed cost of existing PLCs.

As the head control systems engineer and designated control system security manager for Tetramethyl Death Chemicals you are excited about the possibility of actually having a secure control system until you realize that your current equipment budget will only allow the purchase of the system upgrades and a limited number of HSPLCs. How do you determine which PLCs to replace?

Ranking PLC Vulnerabilities

With any kind of luck this type of scenario will be played out in any number of chemical facilities in the not too distant future. Wholesale replacement of entire control systems will just not be practical for large chemical facilities. With the long expected life-times of control system components I would expect that high-security PLCS and other control system components will probably be replaced piecemeal. A methodology will need to be developed to determine which components need to be replaced with high-security drop-ins first. Each PLC will have to be ranked based upon its risk ranking.

While changing out the oldest least secure PLC’s seems to be an obvious choice, I think that it is more important to evaluate each PLC on the basis of the consequences of a successful cyber-attack on that particular device; what could happen if a sophisticated cyber-terrorist were able to take control of the device.

We can start by assigning each consequence to one of four categories:

• Nothing costly can go wrong (a really limited category);

• Quality consequences (un-shippable product for instance);

• Local safety consequences (on-site personnel or equipment injured or damaged); and

• Off-site safety consequences (off-site personnel or equipment injured or damaged).

The last category is the one that society and politicians want us to pay the most attention to followed by the third category. Some management teams may switch the second and third in their priority rankings depending on the unit cost of the bad product. The corporate cost of minor injuries or equipment damage can quickly dwarfed by the cost of unsellable products.

Assuming that the above list is appropriately risk ordered (highest risk at the bottom of the list) one would assume that replacing the PLCs in the final category would have the highest priority. Priority ranking within each category would then be necessary. That becomes a tad bit more complicated. If only one dangerous chemical is involved then it is just a matter of ranking the PLCs by the amount that could be released by a successful attack. As the number of dangerous chemicals becomes larger it quickly becomes obvious that not all releases are equally dangerous so some system of qualitative risk analysis will have to be employed. Attention to chemical interactions will also have to be included in this analysis.

Don’t Replace All PLCs

As one goes through this exercise it becomes readily apparent that there is no necessity to replace all of the PLCs in a facility. If the HMI is replaced with a high security version most attack modes are dealt with. If secure communications exist between the HMI, the main control software and the PLCs, then the attack possibilities are further restricted.

While it is still possible to attack vulnerable PLCs it becomes much more difficult and will require a great deal of effort and planning on the part of an attacker. This is no longer in the realm of script kiddies. Appropriately skilled attackers are only going to be interested attacking high-consequence targets; thus only those PLCs that have large consequences, mostly off-site consequences, associated with them will be targets. Replacing them with HSPLCs will essentially take care of the control system security problem.

CSB and Employee Participation

Yesterday the Chemical Safety Board published on their web site the Board’s new policy about employee participation in the investigations that it undertakes. They note that the legislation that established the CSB provides employees and their representatives the same rights to participation in the investigation as those enjoyed when OSHA conducts similar investigations {42 USC § 7412(r)(6)(L)}.

Anyone that has conducted or participated in an incident investigation knows how important it is that the investigators have full access to all of the information concerning the incident. Anything less will almost ensure that the investigation will not discover the true root cause of the incident and will thus not be able to prevent re-occurrences. Since the whole point of a CSB accident investigation is to establish the root cause and propose preventive safety measures it is obviously imperative that they have access to all of the facts related to the incident. That means that they must have full and unfettered access to everyone involved.

The Chemical Safety Board is not a regulatory agency. They do not have the authority to issue citations, impose fines or require companies (or agencies) to make changes in policies or procedures. I would suppose that means that there is no requirement for them to issue regulations about the conduct of their investigations; that being the reason that this policy is being announced on their web site and not published in the Federal Register. That may be legally correct (I’m not sure that is the case, but I’m not really a legal scholar or a lawyer; anyone that is may feel free to chime in on this point), but I think that the Board is setting itself up for some interesting legal fights by doing things this way.

The CSB has already had problems with getting adequate cooperation in the conduct of two recent high-profile accident investigations; the MIC explosion at Bayer CropScience and the BP blowout in the Gulf. In the first incident Bayer tried unsuccessfully to deny CSB access to certain information by invoking protections under the Maritime Transportation Security Act (MTSA). In the second instance various parties tried to deny CSB participation in the investigation citing various jurisdictional issues.

Anything that impedes the Chemical Safety Board from conducting a complete and thorough investigation of chemical related accidents puts the entire chemical community at risk. To help avoid delays in future investigations while the legal status of employee participation is wrangled through the courts I would urge the Board to submit this policy to a notice and public comment process, preferably through the Federal Register – www.Regulations.gov process, or even via the slightly less formal process the Board used in formulating their request for a National Academy of Sciences review of the MIC manufacturing process.

Full and complete participation by facility employees and contractors in any CSB accident investigation process is a prerequisite for obtaining all of the information necessary for the CSB to determine to root cause of the incident under investigation and to formulate their recommendations for measures other facilities can take to prevent future occurrences of similar accidents. This policy will go a long way to encouraging that participation, but the CSB needs to ensure that the community has a chance to buy into this policy. That can only be achieved by a public notice and comment process whether or not it is legally required.

ISCD Hearing – Part Two

Yesterday the Subcommittee on Cybersecurity, Infrastructure Protection, and Security Technologies of the House Homeland Security Committee held their first oversight hearing looking at the problems in the DHS Infrastructure Security Compliance Division (ISCD) that deal with the implementation of the CFATS program. Neither the general public nor the regulated community has yet seen the internal ISCD report on the challenges ISCD is facing in completing the final stages of the CFATS program, but it was evident that this subcommittee was better read into the nuances of the report than was the previous panel that looked at this problem.

Management Questions

It is evident that the Anderson-Wulf report (Penny Anderson, the current ISCD Director and David Wulf, her Deputy Director) has not identified any actual criminal malfeasance involved in the problems identified in the program; that is the good news (though one would like to assume that a DHS IG investigation would be in order to assure that that is the case). So it would seem that the problems identified in the report (and I would like to suggest that at least a summary of the problems and proposed solutions should be made available to the public) were mismanagement issues which, unfortunately the DHS witnesses yesterday were unable to really address.

To be fair to Anderson and Wulf, they came into their current jobs and identified the problems existing within the program. Whether or not they have done anything of substance to clean up the problems remains to be seen (publicly at least), but they can hardly be held to account for the problems of their predecessors. As Ms. Anderson noted she wasn’t there when the problems started and thus couldn’t comment on why they occurred, suggesting in passing that the appropriate people should be questioned directly.

Under Secretary Beers was responsible for ISCD for almost two years before the Anderson-Wulf report was issued, but his consistent response in the two hearings to date was that nobody told him about the problems, with three exceptions that have been pretty well glossed over in both of the hearings to date. He does continue to note that he had commissioned two other reports on the CFATS implementation during his tenure, but they did not turn up any of the problems noted by Anderson and Wulf. It would be interesting to know if Congress has been given copies of those reports.

There continue to be suggestions that Beers had mislead Congress in his various hearing appearances over the 2+ years he has been at the helm of NPPD, failing to tell Congress about the delays in implementing the CFATS Site Security Plans. As best I can tell he has accurately reported the number of inspections that had been completed to-date at each hearing and he was never pressed for an explanation as to why the delays were occurring. And he was never held to account, from one hearing to the next, as to why his assurances of future improvement were never actually seen.

And, once again, he received a pass on the same issue at this hearing. His written testimony continues to report 55 authorized or conditionally authorized Tier 1 SSPs, the same number that was reported a month ago. Under the improved and accelerated program touted as resulting from the Anderson-Wulf reports one would have assumed that there should have been at least one or two additional facilities added to that list in that time frame. Otherwise it would seem that completion of the Tier 1 SSP authorizations in this fiscal year will again not be achieved.

Union Activities

As I had noted earlier I would have like to have seen this subcommittee question at least one current or former employee of the Division, but lacking that it was reasonable to include David Wright the President of the American Federation of Government Employees Local 918 that represents the chemical facility inspection force. Since his union was reportedly included in the Anderson-Wulf report as a contributing cause to the problems at ISCD and as one of the original group of inspectors brought into the program, his input should have been invaluable.

Once again, however, he is one of a long list of people that has never seen the Anderson-Wulf report, so he was unable to tell the Subcommittee much about the problems identified in the report. He did state that he has personally assured Anderson and Beers that he and his union are fully willing and committed to work with ISCD management in helping to resolve the issues involving the chemical inspection force.

Wright did receive a sympathetic hearing of his testimony. By the time he got a chance to testify, the only Republican asking questions was Chairman Lungren (R,CA) while all four Subcommittee Democrats were actively involved in questioning the industry-labor panel.

Industry Comments

Bill Almond from SOCMA and Timothy Scott from Dow Chemical were in the dark about the actual scope of the problems in ISCD as anyone else outside of the Department, not having seen the Anderson-Wulf report either. Both made it clear in their testimony and response to questions that they did not think the management issues apparently outlined in the report reflected any inherent problems with the CFATS program. They both indicated that they thought that an important part of any resolution to the management problems was a long-term re-authorization of the program.

Their written testimony includes a number of other recommendations for moving the program forward. I’ll review those in more detail in a later blog post.

Less Politics

I was impressed by the lack of overt politics involved in the questioning by this Subcommittee. Even Ranking Member Clarke’s (D,NY) questions about reauthorization were phrased to address how the current problems might impact Congress’ decision on how to go about reauthorizing the program; legitimately asking about the role Congress should take in addressing the identified problems.

The rhetoric was not fiery or accusatory in any of the questioning by this panel. It seemed that everyone was interested in getting to the root of the problems and all of questions indicated that these congress critters, at least, had done their homework about the program. Unfortunately, the five minute question-response format of this type hearing does not really lend itself to an investigation.

For Congress to effectively get to the bottom of these problems and adequately review the potential solutions they are going to need and independent report by the GAO or the DHS Inspectors General Office. The sooner that investigation is started the better.

More Changes to CFATS Web Pages

Today the folks at DHS ISCD made some changes to three of the web pages associated with the CFATS program. Two of the pages had inconsequential changes and the third had some more substantial but hardly earth shattering changes.

Date Changes

The two pages with inconsequential changes were the page for the CSAT Tool and the general CFATS page. In fact, the only change that I can note (and the easiest to catch) is the change in the ‘last reviewed/modified’ date on the bottom of each of these two pages. It is a shame that if the webmaster was just going through an reviewing the current status of the page (and this is done periodically or sometimes it is done before major modifications are made to a page) that necessary housekeeping changes were not made to these pages.

For example the CSAT Tool page still lists an out of date version of their CSAT User Registration User Guide (July 2008 vs October 2011) though the link does go to the most recent version of the manual. This could cause problems for people that use this page to ensure that all of their CSAT manuals are up to date.

The CFATS page still has links to an out-of-date site for the Chemical Facility Anti-Terrorism Standards Interim Final Rule. As I noted in a recent blog post that link will be useless in the near future and should be updated for the newer link provided by recent changes to the GPO web site.

More Consequential Changes

Two separate changes were made to the CFATS Knowledge Center web page. The one series of changes was due to the recent acting appointment of William Flynn as the Assistant Secretary for Infrastructure Protection to replace the recently resigned Todd Keil. A frequently asked question (FAQ; #1557) had to be revised to replace Keil’s name with Flynn’s. This is reflected in a brief note about the change in the “Latest News”. I believe that there are a couple of other FAQ’s where the ASIP is listed as an action officer for other appeals processes within CFATS.

The second change is odd. The new information tool that I wrote about yesterday has been removed from the page just as quietly as it was initially added. I hope that this wasn’t removed as a sign of a change in the information sharing policy at ISCD. I thought that the tool could have been used to provide some solid information to the CFATS community. Oh well, on the plus side, I don’t have to review the lengthy presentation for the readers of this blog.

Another Luigi HMI Alert

This afternoon the DHS ICS-CERT published an alert for multiple SCADA-HMI vulnerabilities reported by Luigi in the xArrow SCADA-HMI application. The four reported vulnerabilities are:

• Decompression NULL Pointer;

• Heap Corruption;

• Invalid Read Access; and

• Memory Corruption

The alert notes that the vulnerabilities are all remotely exploitable and proof-of-concept code is available on the Luigi web site (NOTE: ICS-CERT does not provide a link to that site).

Interestingly there is another SCADA vulnerability listed on the Luigi web site that was published on the same day (March 2^nd, 2012) as the xArrow vulnerability. That vulnerability is reported in the Beckhoff TwinCAT system; it is an integer overflow vulnerability in the TCatScopeView application. I wonder why there isn’t an ICS-CERT alert for that vulnerability, unless, of course, there is already another, coordinated disclosure, in the works for that vulnerability.