#fuzzing

Lecture finishing covering advanced fuzzing topics. Lecture offers discussion of taint analysis, dynamic taint analysis, symbolic … source

TL;DR: Fuzz testing is really cool.

I've been interested in test automation for a long time, and one area in particular that I've been interested to learn about is fuzz testing. I'm now able to finally dig into this area of software engineering.

Fuzzing has been around for a while now, starting on a dark and stormy night in Wisconsin. Fuzzing is a technique for sending semi-random and possibly semi-invalid inputs to a piece of software (a function, a program, etc) to find bugs with this software. Over time, fuzzing - also known as fuzz testing - has been used to find security vulnerabilities in software in addition to purely functional software bugs. Fuzzing has also evolved from generating totally random data to becoming more intelligent about data generation and inputs. Fuzzing falls into the category of easy problems with difficult solutions that I enjoy.

One thing I find fascinating about fuzzing is that it is a classic case of test automation: after deciding what to fuzz, a developer can use a fuzzer to create test inputs and then execute tests using those inputs automatically. Effectively, automation is being used to do lots of work that a human would find tedious or even impossible to complete. Absolutely classic in the sense of using automation in testing and software quality assurance.

Another thing I find fascinating is how effective fuzz testing can be. Fuzzing is an exploratory technique since the developer using a fuzzer can control how many iterations of inputs are being sent to the software under test. In practice, this means a single fuzzer execution can take minutes, hours or even days depending on how many iterations the developer wishes to execute. In practice, however, many bugs can be detected within a minute. Such bugs may include overflow errors (heap, stack and so on), out-of-memory errors and null pointer exceptions. All of these kinds of bugs are simple to fix but can be surprisingly difficult to find manually even when using debuggers.

Lastly, and I cannot stress this enough: fuzzing is a well-established technique and subfield of computer science in its own right. The earliest fuzzers were literal homework assignments, but over the past few decades fuzzing has become quite sophisticated. There are techniques that can "guide" fuzzing inputs to find more "interesting" results (see feedback-based fuzzing), approaches to leverage multiple fuzzers at once (see ensemble fuzzing) and efforts to fuzz open source software effectively at scale. As well, specialized fuzzers have been created for fuzzing code in particular languages (C and C++ are the most common), fuzzing browsers, fuzzing web APIs are more.