Sunday, December 15, 2013

Reader Comment – 12-13-13 – DNP3 Fuzzing

Andrew West, the Chair of the DNP3 Technical Committee, left a very nice comment about my blog post about their latest technical note helping people correct the improper input validation vulnerabilities that had been reported by Crain-Sistrunk.

In his comment he responded to my comment about the failure of the technical note to specifically mention Adam’s fuzz tester. He made a very good point about not being able to specifically plug one vendor’s device over another; I knew that and it really wasn’t fair for me to make that comment.

Andrew did go on to make another important point about fuzz testing. He noted that each tester had its own peculiar ‘directed randomness’ that it employed. This means that two different fuzz testers may detect faults not found by the other. Andrew commented that it “may be beneficial to use multiple different tools in order to increase test coverage”.

This does not mean that we will forever be responding to new vulnerabilities discovered by new fuzz testers. As vendors get better about their coding practices and internal testing before putting their devices out into the wild there will be fewer and fewer vulnerabilities that will be discoverable by this type tool.

Of course, that just means that someone will come up with another type of tool to look for new families of vulnerabilities that the coders had never considered. The competition between the coders and the hackers will be never ending. The improvement in the skills of one side will drive improvements in the skills of the others. That’s just the way of the world.

1 comment:

Adam Crain said...

Andrew's absolutely right. Different fuzzers may find different things. I'll be publishing data comparing the Aegis fuzzer to other commercial tools at SANS Scada Summit using code coverage as a metric.

Some fuzzers will prove your software has *fewer* bugs than another, but sadly there is no silver bullet that will guarantee your software is free of bugs.

The DNP UG rightfully doesn't promote vendors. The challenge I raise to the UG is to list different kinds of testing besides conformance. This will take time, but I cam confident it will happen eventually.

/* Use this with templates/template-twocol.html */