Increasing Code Coverage Using Fuzzing Lessons

In a previous post, we introduced Diligence Fuzzing, a new fuzzing service for Ethereum smart contracts that is powered by our Harvey fuzzer.

The main goal of a fuzzer is to cover your code as well as possible. However, occasionally a fuzzer may struggle to reach certain regions of your code. In such cases, you may want to help the fuzzer by “teaching” it how to reach a line of code that it could not cover on its own.

To make this easy, Diligence Fuzzing allows you to record so-called fuzzing lessons. In this post, we will explain how to use fuzzing lessons to help the fuzzer out.

Fuzzing lessons

On a high level, a fuzzing lesson is recorded by observing how a human user would invoke the code to cover a given line of code. During the next fuzzing campaign, the fuzzer can use previously recorded lessons by replaying the observed invocations. This will allow the fuzzer to reach the given line of code but also other code that is reachable from there.

The user can record another lesson and repeat this process if other code regions are still not covered during the next campaign. Eventually, the user can help the fuzzer explore all the critical code regions by recording a small number of lessons.

Let us illustrate this process with a concrete example.

Example

Imagine that we want to test the following contract (only intended for illustration purposes and not for production use):

pragma solidity 0.8.17;

contract GaslessDestroy {
  bool isDestroyable;
  address owner;
  constructor (address o) {
    owner = o;
  }
  function destroy() external {
    require(isDestroyable);
    selfdestruct(payable(msg.sender));
  }
  function permitDestroy(uint8 v, bytes32 r, bytes32 s) external payable {
    require(!isDestroyable);
    bytes32 hash = keccak256(abi.encode("permit-destroy", address(this), block.chainid));
    address signer = ecrecover(hash, v, r, s);
    require(signer != address(0x0));
    require(signer == owner);
    isDestroyable = true;
  }
}

The contract allows any user to destroy it (by invoking the destroy function that, in addition, transfers any leftover Ether to the sender). However, this is only possible once the owner has shared their permission with at least one user by signing a hash value with their private key. The signature (represented by the inputs v, r, and s of function permitDestroy) is validated using Solidity’s ecrecover primitive; for a valid signature, it returns the signer’s address.

When we start a fuzzing campaign for this contract, the fuzzer will not be able to cover the last line of function permitDestroy (see the above screenshot). This is not surprising since the fuzzer would have to come up with a valid signature of the owner for a fixed hash value. This would require the fuzzer to guess the owner’s private key and to sign a specific hash value to obtain valid inputs for v, r, and s.

Luckily, we can use a fuzzing lesson to help the fuzzer overcome this obstacle. We can create a short script (see our documentation for an example script) that invokes the permitDestroy function with valid inputs. Alternatively, we could also use a local web frontend to manually interact with the contract.

On a high level, our script first generates the hash value and then signs it with the owner’s private key. Finally, it invokes the permitDestroy function with the corresponding valid inputs.

Once we have written our script, we can start recording a lesson using our fuzzing CLI:

$ fuzz lesson start --description "invoke 'permitDestroy' function"

The description is optional but helpful in documenting what individual lessons are supposed to “teach”.

Now, we are ready to run the script and save the recorded lesson:

$ fuzz lesson stop

The CLI saves the recorded lesson in a .fuzzing_lessons.json file, and we can start a new campaign to verify that the lesson achieved the desired effect.

The new campaign can now completely cover the function permitDestroy (see the above screenshot). On top of this, it can now also fully cover the function destroy. This demonstrates how a small hint can sometimes unlock new regions of the code that were not originally targeted by the lesson.

In this post, we have illustrated how to use fuzzing lessons to increase code coverage for your fuzzing campaigns.

Sign up for Diligence Fuzzing if you want to try it out yourself!

Thanks to Heiko Fisch for feedback on drafts of this article.

All posts