MythX and Continuous Integration (Part 1): CircleCI

In the first post of a new series, we discuss integrating security considerations to a continuous integration pipeline, starting with CircleCI.

Conveyor belts. Because continuity. Get it?

Conveyor belts. Because continuity. Get it?

Continuous testing of applications can be hard to figure out. While it is difficult to measure CI/CD adoption, the blockchain ecosystem offers a great opportunity to adopt in-depth testing and continuous delivery pipelines where they make sense: right from the start.

At MythX we don’t mind which technologies you are using to get things done. Choosing the right tool for the right job means that service providers must not make assumptions about their client’s technologies. Security testing especially should be widely available and easy to use.

Behind the scenes the MythX team is working on the tech behind the smart contract analysis to make adding security to your workflow as simple as possible. For instance, we continuously run constructed samples against MythX in our QA process to keep our result quality as high as possible. This has several advantages:

  • Avoid breaking changes. If things go wrong we will notice quickly and can iterate until the change is production-ready.
  • Granular insights. Tests run on small change sets, one step at a time. This makes breaking changes obvious and allows for concise fixes.
  • Save time on code reviews. We improve and review our internal test suites regularly. If they pass, confidence in the code’s correctness rises. That in turn keeps reviews focused on the business logic that matters.

Why CI?

There are plenty of reasons to use Continuous Integration (CI), but the above points have a nice side-effect: they also apply to smart contract development, or the software development lifecycle (SDLC) in general.

There are countless CI solutions out there and, as mentioned previously, we don’t mind which ones teams settle on.

That is why in this three-part series we will take a look at two of the most popular Continuous Integration solutions for open-source projects: CircleCI (this post), and Travis. Afterwards, we will generalize our approach to apply to other CI systems. Using the official MythX CLI it is fairly easy to get a basic test setup going.

Let’s jump right in!

CircleCI

CircleCI is configured using a YAML file located in .circleci/config.yml to describe the build steps. It defines workflows, which in turn consist of jobs. Each of the defined jobs contain a number of steps that are run. (CircleCI offers great tutorials if you need a quick refresher.)

In our case, this will be downloading MythX CLI, and running it against a set of given smart contracts. First we define our basic config:

version: 2
jobs:
  build:
    docker:
      - image: circleci/python:3.8
    steps:
      - checkout

This will define a job called build. All the steps inside this job will be executed in a Docker container running Python 3.8. The MythX CLI also runs on Python 3.6+ and PyPy.

Now for the meaty stuff. Let’s open up a virtual environment and install the MythX CLI into it. This makes sure that our Python application does not collide with any other Python code running in CI, and it avoids a range of CircleCI permission issues.

In our first step we checkout the code at the commit the CI job was given.

- run:
  name: Setup virtual environment
  command: python -m virtualenv venv
- run:
  name: Install MythX CLI
  command: |
    . venv/bin/activate
    pip install mythx-cli

There is nothing spectacular here. We define two steps, three commands, and we are set up. Now, submitting a contract to MythX can be done in a single command.

- run:
  name: Analyze with MythX
  command: |
    . venv/bin/activate
    mythx --ci --yes analyze --solc-version 0.5.0 contracts/

Well, actually two commands: Jump into the virtual environment, and then analyze the code.

There are a few things in the mythx call to note:

  • --ci tells the MythX CLI to return 1 if a medium or high severity issue has been found.
  • --yes disables the confirmation prompt asking the user whether they really want to submit X Solidity files.
  • analyze is the subcommand that tells the CLI we want to send an analysis job to the MythX API.
  • --solc-version defines, you guessed it, the Solidity compiler version to compile the given files with.
  • contracts/ is the directory the smart contracts live in. The MythX CLI will walk through this directory recursively and prepare analysis requests for each .sol file that has been found.

By default the CLI submits scans in quick mode. These return after roughly 120 seconds. Now our full CircleCI config looks like this:

version: 2
jobs:
  build:
    docker:
      - image: circleci/python:3.8
    steps:
      - checkout
      - run:
          name: Setup virtual environment
          command: python -m virtualenv venv
      - run:
          name: Install MythX CLI
          command: |
            . venv/bin/activate
            pip install mythx-cli
      - run:
          name: Analyze with MythX
          command: |
            . venv/bin/activate
            mythx --ci --yes analyze --solc-version 0.5.0 contracts/

Run it!

Below I have prepared a test repository with a vulnerable smart contract and connected it to CircleCI with the above configuration. As soon as the CI job is finished, the following output appears:

The CI check has failed and notified me of a vulnerability. Now I want to note exactly how to trigger it so I can write a test case and validate my fix.

So I have signed up for a free account on the MythX website and added my API token to the CI’s environment as MYTHX_ACCESS_TOKEN. That way all my CI analyses are directly attached to my account and I can retrieve them later. In the dashboard’s analysis listing I find my submission:

Diving into it, I see the vulnerability overview that was displayed in my CI output again:

Scrolling down, I see that my code has been highlighted with the vulnerability locations:

For each vulnerability I also have the test case data at my fingertips.

Nice!

Nice!

With the click of a button I can copy the raw transaction data and easily bootstrap my Truffle test cases. This helps me to validate the fix in my code and avoid the issue from occurring in the future.

What’s next?

The MythX CLI is continuously improving (CI jokes, hah!) but I have written exhaustive documentation and a usage guide.

In addition to what I showed here, MythX CLI allows for more advanced patterns such as blacklisting vulnerabilities, opening custom analysis groups, delivering results in various output formats, and much more.

Do you want to store your reports as artifact files attached to the CI job? Just add the -o parameter and configure the path in CircleCI. The possibilities are endless!

Feedback is a gift and a comment here or directly in the GitHub repository is greatly appreciated. In the next part, we will be talking about the same integration in Travis CI in part two. Stay tuned!

Have you signed up for a free MythX account yet? Get started today and integrate with your own CI pipeline.

Thinking about smart contract security? We can provide training, ongoing advice, and smart contract auditing. Contact us.

All posts chevronRight icon

`