Andrew West, the Chair of the DNP3 Technical Committee, left
a very
nice comment about my
blog post about their latest technical note helping people correct the
improper input validation vulnerabilities that had been reported by
Crain-Sistrunk.
In his comment he responded to my comment about the failure
of the technical note to specifically mention Adam’s fuzz tester. He made a
very good point about not being able to specifically plug one vendor’s device
over another; I knew that and it really wasn’t fair for me to make that comment.
Andrew did go on to make another important point about fuzz
testing. He noted that each tester had its own peculiar ‘directed randomness’
that it employed. This means that two different fuzz testers may detect faults not
found by the other. Andrew commented that it “may be beneficial to use multiple
different tools in order to increase test coverage”.
This does not mean that we will forever be responding to new
vulnerabilities discovered by new fuzz testers. As vendors get better about
their coding practices and internal testing before putting their devices out
into the wild there will be fewer and fewer vulnerabilities that will be
discoverable by this type tool.
Of course, that just means that someone will come up with
another type of tool to look for new families of vulnerabilities that the
coders had never considered. The competition between the coders and the hackers
will be never ending. The improvement in the skills of one side will drive
improvements in the skills of the others. That’s just the way of the world.
1 comment:
Andrew's absolutely right. Different fuzzers may find different things. I'll be publishing data comparing the Aegis fuzzer to other commercial tools at SANS Scada Summit using code coverage as a metric.
Some fuzzers will prove your software has *fewer* bugs than another, but sadly there is no silver bullet that will guarantee your software is free of bugs.
The DNP UG rightfully doesn't promote vendors. The challenge I raise to the UG is to list different kinds of testing besides conformance. This will take time, but I cam confident it will happen eventually.
Post a Comment