CFATS 2.0 Results Webinar

I just completed watching the DHS “CFATS Tiering Update - April 2017” webinar. This webinar provided information on the preliminary outcomes of the DHS Infrastructure Security Compliance Division’s (ISCD) review of CSAT 2.0 Top Screen submission that were started last fall. I say ‘preliminary’ because ISCD is still reviewing a number of the submitted Top Screens and is presumably still sending out Top Screen submission letters.

It was an interesting presentation and I recommend that interested parties that missed this webinar sign up for the next session that will be held early next month.

There have been a number of questions about the potential effects of the new risk-assessment methodology that is part of CSAT 2.0. The main question that folks have been asking is how that new methodology would end up affecting the Risk Tiering within the Chemical Facility Anti-Terrorism Standards (CFATS) program. The presentation today provides at least a partial answer.

Changed Risk Assessment Methodology

ISCD took a sample of 8,000 new Top Screen submissions and specifically looked at the new tiering results. Here are the results that ISCD reported today (Note: there was no mention of the missing 4% of the facilities):

• 5% moved from untiered to tiered;

• 5% moved from tiered to untiered;

• 51% moved between the four tier rankings; and

• 35% remained within their existing tier rankings

Remember, untiered facilities are not covered facilities under the CFATS program, and thus do not have to submit an SVA/SSP or have an approved site security plan (SSP) or alternative security plan (ASP).

The presenters also described two specific trends that they saw in tier changes. First, facilities that had just weapon of mass effect (WME) security issues tended to see a decrease in tier ranking because the new ‘physics-based modeling’ tended to see a lower risk for the same situation for these chemicals as compared to the old risk modeling process. Second, a counter-trend was seen with two specific chemicals (triethanolamine and methyldiethanolamine); the same ‘physics-based modeling’ tended to see an increased risk for these chemicals as compared to the previous methodology.

The presenters also noted that there were 235 facilities (not clear if they were part of the 8,000 used for the above analysis) that were facilities with only theft/diversion security issues that now had added release security issues. The presenters did not make it clear whether this was due to the risk modeling or if it was due to changes in the reported DHS chemicals of interest on site.

Commentary

I missed the early portion of the webinar, so I almost missed the information that ISCD probably presented on the number of letters sent out and the number of Top Screens that have been submitted. I should have more information on that in the near future.

I have some serious questions about the reported analysis of the risk assessment results presented in the webinar. Now this is probably due to my nitpicking of statistical analysis in general. I have a little more training (not that much though) in statistical analysis than most people, so I generally cringe whenever I see the word ‘analysis’ used in a presentation.

First, let’s look at that missing 4% I mentioned earlier. There is one category that is specifically missing from those reported, the untiered facilities that remained untiered facilities. For the sake of discussion, let us assume that those unreported 4% were those untiered facilities that did not change. That would mean that only 9% of the facilities in the 8,000-facility sample were untiered or facilities that were not covered by the CFATS program.

That is a problem because that would mean that 81% of the 8,000 facilities in the sample were currently covered facilities. That would be 6,480 facilities. But, as of the last reporting by ISCD, there were less than 3,000 covered facilities in the program. That means that the reported percentages cannot be of the whole 8,000 facility sample.

Let’s assume for the sake of argument that the all 2,948 facilities reported in the last CFATS Fact Sheet from October 1^st of last year were included in the 8,000-facility sample. That would mean that there were 5,052 initially untiered facilities in the sample. Plugging these numbers into the previously reported percentages we get:

• 252 moved from untiered to tiered;

• 147 moved from tiered to untiered;

• 1503 moved between the four tier rankings; and

• 1031 remained within their existing tier rankings

This still leaves 716 facilities for which no data was provided, or 24% of the covered facilities. So, any way we look at it we have internally inconsistent information provided. I will try to get clarification from ISCD.