Reader Comment – 03-07-09

Saturday we had an anonymous reader comment left on a previous response to another reader comment about an earlier blog on the DHS IG report on the effectiveness of Transportation Security Inspectors – Surface (TSIs). The current reader comment, in its entirety is shown below. “PJ, did DHS make any report data available for scrutiny? Do we know what methods the IG used to assess risk? We need a transparent government, especially if private corporate entities seem determined to keep large populations at high risk (e.g. railroads with TIH routing, food companies with salmonella sales).” IG Data and Methodology There is very little hard data in the IG report. In fact, there is no indication that the IG has done more than report the data and conclusions that the TSA has already reported. There are no claims that the IG has independently verified any of the data provided by TSA. From the IG reports that I have seen (admittedly a limited data universe for making claims) this is not unusual. Concrete data and data verification are much more common in GSA reports to Congress or reports by the Congressional Research Service. The IG report does briefly explain the data collection methodology that was used by TSA to collect their data (page 8).

“To evaluate freight rail risks, TSA created a five-phase SAI [Security Action Items] assessment program. In Phases I-III, TSIs collect data to assess freight railroads’ implementation of 24 recommended security practices (see appendix C). TSIs evaluate a freight company’s security systems and plans, employee security and identification procedures, and other practices. In Phases IV-V, TSIs visit freight railroad locations to assess employee TIH awareness, verify the location of loaded TIH cars, and determine whether loaded cars are attended. TSA uses the data to calculate the risk posed by TIH in a given High Threat Urban Area.”

The report then goes on to explain (page 9) that, as of May 2008: “TSIs have performed more than 3,000 SAI reviews of 500 freight rail facilities in 60 High Threat Urban Areas. The results of the reviews showed that TIH cars were left unattended 13% fewer hours and overall risk was reduced by 54%.” Now, a careful reading of that last sentence does not say that 13% fewer unattended hours directly led to the 54% risk reduction, but it certainly leaves that as an easy conclusion to draw. Admittedly, risk reduction is a difficult thing to accurately measure. Fred Millar in the earlier reader comment made reference to the general formula of “risk = probability x consequence”, but that is more of a philosophical statement than a real mathematical formula. Typically politicians and government agencies use surrogate equivalencies instead of actual calculations to define risk reductions; 13% fewer hours unattended = 13% risk reduction. This is a scientifically questionable evaluation, but it does have the advantage of being readily understood by politicians and the general public. In this case, unless there were additional, unreported, measures that contributed to the risk reduction TSA would have had to use a fudge factor of 4.15 to get from 13% to 54%. Politicians seldom use a fudge factor that large, it would be too noticeable. A major factor that could have contributed to that larger risk reduction number is the reduction in hours TIH shipments spent transiting the HTUA. That claim has been reported in other venues, but it is not discussed here in this report; a disturbing omission. Transparency vs Security Anonymous states that we “need a transparent government”, but transparency and security frequently run into conflict. Too much transparency and security is fatally compromised. Too little transparency and security lapses are missed. It is frequently difficult to find a safe middle ground. In the Federal government we have a system that is supposed to take care of this problem. The executive branch executes laws and regulations and the legislative branch is supposed to oversee that execution. This was part of the reason that this DHS IG report was produced; the 9/11 Implementation Act required the report to Congress. Now almost all of the data that TSA used to compile their risk reduction data would have been labeled Sensitive Security Information (SSI) which is treated as classified under Federal rules. Much of the data compiled from that SSI information would still be classified, but at some level the data could no longer be traced back to individual facilities and would no longer deserve the SSI classification. This is why the 13% figure mentioned above was releasable. Since the Homeland Security Committee members and staffers would be expected to have the requisite security clearances, the way around producing a publicly viewable document while providing the necessary supporting information to justify the conclusions would be to include a classified annex containing that data. That annex would be made available to Congress. It does not appear that such an annex was made available with this report. Congressional Response Since the production of this report was a requirement laid upon DHS by Congress, it is only Congress, or the House Committee on Homeland Security in this case, that can judge whether or not this report met their requirements. While neither Anonymous nor I are comfortable with the lack of supporting documentation, it is not up to us. Only Chairman Thompson and his staffers may judge the adequacy of this effort. One final interesting point; this report was prepared by the Bush DHS. It was written before November 26th, 2008 even though it was dated February 2009. You can discern this by the references to the ‘yet to be finalized’ TSA freight security rule. The delay between the writing and publishing of the report is no doubt due to the staff reviews that were required before it was released. In this case we must assume that it was reviewed by Secretary Napolitano’s office because of the release date.

TSA Inspectors

The DHS Inspector General’s Office recently released a report on the Effectiveness of TSA's Surface Transportation Security Inspector. What has been almost as interesting as the report itself has been the press reaction. The Washington Post complains about the understaffed agency and it inability to police rail and mass transit security. The Journal of Commerce boasts how much the inspectors have done to reduce the threat of PIH chemicals transiting rail corridors through urban areas. Both publications read the same report; sort of like the blind men describing the elephant. Transportation Security Inspectors - Surface The IG report notes that there are only about 100 Transportation Security Inspectors – Surface (TSIs) in a program that was established in 2005. The IG report notes that these TSIs have been assigned to a wide range of duties. They include:

Conducting Baseline Assessment for Security Enhancement reviews for passenger rail and bus lines; Conducting Security Action Item reviews for TIH freight rail shipments; Producing station profiles and by acting as liaisons between the Transportation Security Operations Center and transportation systems; and Participating in Visible Intermodal Prevention and Response exercises.

The report notes that an additional 75 inspectors have been authorized to provide additional coverage for the VIPR program where a variety Federal personnel make random security appearances at rail and bus passenger stations. They note that the lack of law enforcement training and authority limit the effectiveness of the TSIs in this program. Future Workload The report goes on to note that as the freight rail security rules come completely into effect on April 1st these same inspectors will also be tasked with enforcement of those regulations. Additional surface transportation security rules under consideration will further increase the workload on those TSIs. Previous IG recommendations have included using TSIs to verify and validate work done under a variety of TSA surface transportation grant programs. IG Recommendations The IG report suggests that TSA do an in depth evaluation of the use of the TSIs to determine how many are actually needed. They also suggest that TSA re-evaluate the use of TSIs in the VIPR program so that there expertise may be better utilized. Finally, the IG recommended that TSA work to make the chain of command for the TSIs more transparent. It is interesting to note that there is apparently a interest in the Obama Administration in increasing the scope of the duties of the TSIs as well as increasing their numbers. A complete reading of the IG report shows that increasing the scope of their duties will require a significant increase in their numbers. Hopefully the politicians will realize that increasing the numbers of inspectors should precede the increase in the required duties.