Cybersecurity Framework Update – Conformity Assessment

This is part of a detailed look at the recently published Discussion Draft of the Preliminary Cybersecurity Framework (PCF). The NIST Information Technology Laboratory (ITL) published this and supporting documentation to their web site during the week of August 28^th in order to allow for public comments and preparatory work for the 4^th Cybersecurity Framework Workshop in Dallas, TX next week. The other posts in the series are:

Cybersecurity Framework Update – 8-31-13

Cybersecurity Framework Update – Framework Core

Cybersecurity Framework Update – Adequate Specificity?

Langner Criticizes Cybersecurity Framework

Section 4 of the Discussion Draft addresses a number of issues for future action in support of the requirements set forth in §7(b) of Executive Order 13656. The Draft identifies the following areas “that should be addressed through future collaboration with particular sectors and 353 standards-developing organizations” (pg 11):

• Authentication

• Automated Indicator Sharing;

• Conformity Assessment;

• Data Analytics;

• International Aspects, Impacts, and Alignment;

• Privacy; and

• Supply Chains and Interdependencies

Since the third item appears to be related to something akin to regulations, a major concern of industry, I would like to use this post to address ‘Conformity Assessment’.

Private Sector Assessments

The discussion in §4.3 (pg 12) makes it clear that NIST is not referring to government assessments as part of a regulatory scheme. It starts the discussion off with the following statement:

“Industry has a long history of developing conformity assessment programs to meet society’s needs.”

And it closes the discussion with the following statement:

“Critical infrastructure’s evolving implementation of Framework profiles should drive the identification of private sector conformity assessment activities that address the confidence and information needs of stakeholders.”

Programs like the ISO 9000 quality assessment or The American Chemistry Council’s Responsible Care chemical safety program are the types of assessments to which the Discussion Draft is referring as examples of private sector assessment programs.

Conformance vs Security Debate

There has been a long standing debate in the security community about the differences between conformance to standards and actually instituting adequate security. There is a belief common in security professionals that conformance standards promote a culture of check lists and doing just what is necessary to get ‘approval’. I don’t know that anyone has ever done a real study on the relationship between standards conformance and whether or not an organization takes additional security (safety, quality, whatever) measures not required by the standards organization.

The other side of that is that even if there is a tendency to resort to check list security as a result of establishing a cybersecurity assessment program will that improve the general level of cybersecurity in an industry? Will organizations that do not currently have an effective (or any) cybersecurity program make real improvements (if perhaps less than optimal improvements) to their cybersecurity posture as a result of trying to achieve a minimum level of cybersecurity certification?

Right now the answers to these types of questions will largely be apocryphal. The only real user level cybersecurity standard (with assessment) that I know of is the NERC CIP program and I haven’t heard of anyone doing any real studies of the efficacy of that assessment process. It would be interesting to have either NIST or NSF conduct such a study.

Assessment as an Incentive

The reason that most organizations take part in voluntary assessment programs is that it is good for business. Initial organizations that join these standards assessments do so to differentiate themselves from their competition. As more organizations join the programs it becomes a defensive measure as customers begin to ask why a vendor doesn’t take part in the programs with the unasked question; what are you hiding?

Since the Cybersecurity Framework is specifically targeted at high-risk critical infrastructure organizations and facilities people are, over time, going to expect some sort of statement of compliance from such organizations. This will, of course be limited to some extent since DHS is not expected to announce what organizations are going to be targeted by this program. Even so, there will be some obvious facilities and organizations that will be assumed to be on the List (even if they may not be) so there will be an expectation of a need to comply with the Framework.

As cybersecurity becomes more of an obvious need (almost certainly after a publicly successful attack on an industrial control system) more organizations will be expected by the business community to need to comply with the Framework. The best way to gain recognition for compliance will be through one of these compliance assessment programs.

Moving Forward

As the Framework advances (and I still have some doubts about the completion of this process) I think that we will start to see industry organizations developing assessment programs to support facilities that will be expected to comply. I would not be surprised to see the American Petroleum Institute or ACC be among the early implementers. And NERC might be expected to adapt their CIP assessment process to more nearly include the Framework.