Syzkaller is an unsupervised, grammar based, coverage guided fuzzer used for fuzzing operating system kernels. It primarily performs system call fuzzing, but it can also be used for fuzzing USB and network packets. It is currently used for continuous fuzzing of Linux, Android and other BSD kernels.

  • Automated: Syzkaller can automatically restart crashed virtual machines and also create a reproducer for the crashes.
  • Coverage guided: Syzkaller gets coverage information using the KCOV infrastructure, which is built into the kernel. After executing a mutated program from the corpus, Syzkaller checks if the program has increased the coverage, i.e. it checks whether any new code paths in the kernel have been reached during the execution of the program. If so, it adds the program to the corpus for further mutation.
  • Grammar based: Syzkaller understands the structure and API of system calls. It uses system calls descriptions written in Syzlang to generate valid system calls with randomized values in arguments. This allows it to execute deeper code paths that would not be possible with blind random fuzzing. Syzlang can describe structs that are passed as arguments to syscalls, the possible values for flags and resources (like file descriptors) which are passed across multiple syscalls and have to be created and destroyed using specific system calls.

Syzkaller overview

The above image, taken from Syzkaller documentation shows the overall working of Syzkaller. Syzkaller consists of different components like syz-manager, syz-fuzzer and syz-executor. syz-manager is the process that manages virtual machines, maintains and updates the corpus of input program based on coverage information obtained through syz-fuzzer. It also serves the web interface that displays VM information, bugs found and their respective crash logs. Inside the virtual machine, syz-executor is the program that executes the system calls based on the input from syz-fuzzer.

Setting up Syzkaller

1. Compile the Linux kernel for fuzzing

  • Download source

      git clone https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git linux
  cd linux
  make defconfig;        # Generate default .config
  make kvm_guest.config; # Enable some configs for virtualization

  • Ensure that the following configurations are enabled in the .config file.

      CONFIG_KCOV=y          # Coverage collection
  CONFIG_DEBUG_INFO=y    # Debug info

  CONFIG_KASAN=y         # Memory bug detector
  CONFIG_KASAN_INLINE=y

  CONFIG_CONFIGFS_FS=y   # Required for debian rootfs
  CONFIG_SECURITYFS=y

  • Compile

      make -j16

2. Setup Qemu

  • Create rootfs with SSH access

    We need a root filesystem for booting the kernel in a QEMU virtual machine. Syzkaller provides a helper script tools/create-image.sh for creating a debian image. The script also creates ssh keys for logging into the VM through SSH.

      git clone https://github.com/google/syzkaller
  cd syzkaller
  ./tools/create-image.sh -d buster             # Create a debian buster image
  # Now you should find buster.img, buster.id_rsa, buster.id_rsa.pub created

  • Test Qemu

    Syzkaller accesses VM through SSH and so we need to ensure that the kernel boots and that we are able to SSH into the VM using the generated SSH keys. If the VM boots fine and we are able to SSH into it, then we are all set for fuzzing.

      # Start qemu with the compiled kernel and debian image
  KERNEL=/home/nihaal/linux     # Path to linux source
  IMAGE=/home/nihaal/syzkaller  # Path to directory containing debian image and SSH keys
  qemu-system-x86_64 \
      -m 2G \
      -smp 2 \
      -kernel $KERNEL/arch/x86/boot/bzImage \
      -append "console=ttyS0 root=/dev/sda earlyprintk=serial net.ifnames=0" \
      -drive file=$IMAGE/buster.img,format=raw \
      -net user,host=10.0.2.10,hostfwd=tcp:127.0.0.1:10021-:22 \
      -net nic,model=e1000 \
      -enable-kvm \
      -nographic \
      -pidfile vm.pid \
      2>&1 | tee vm.log

  # From a different console, test SSH access
  ssh -i $IMAGE/buster.id_rsa -p 10021 -o "StrictHostKeyChecking no" root@localhost

3. Setup and run Syzkaller

  • Make Syzkaller

    Syzkaller is written in Go. Install golang on your system and compile Syzkaller with make.

      # Install golang
  cd syzkaller
  make

  • Create syz-manager config

    We need to pass a configuration file for Syzkaller that contains information about the location of the kernel and root filesystem, number of processes, number of VM and the number of virtual CPUs and memory in each VM. Description of different configs can be found here

      {
      "target": "linux/amd64",          # System architecture
      "http": "127.0.0.1:54321",        # URL for web interface
      "workdir": "$WORKDIR",            # Used for storing crash logs and corpus
      "kernel_obj": "$KERNEL",          # Kernel source directory
      "image": "$IMAGE/buster.img",     # Generated root filesystem image
      "sshkey": "$IMAGE/buster.id_rsa", # Generated SSH key
      "syzkaller": "$SYZKALLER",        # Syzkaller directory
      "procs": 2,                       # Number of processes to use in each VM
      "type": "qemu",
      "reproduce": false,               # Disable reproducer generation
      "vm": {
          "count": 4,                   # Number of VMs to use
          "kernel": "$KERNEL/arch/x86/boot/bzImage",  # Path to compiled kernel
          "cpu": 2,                     # Number of CPUs in each VM
          "mem": 2048                   # Memory used by each VM
      }
  }

  • Start fuzzing

      ./bin/syz-manager -config my.cfg

Syzbot

Syzbot is an automated continuous fuzzing infrastructure that performs continous fuzzing of different kernels using Syzkaller and reports any crashes and associated reproducers to the respective mailing lists. It can even test patches to test if the patch actually fixes the bug. The Syzbot infrastructure also has a public dashboard that shows the currently open bugs and crash logs.

TODO

  • Syzlang
  • Testing patches with syzbot
  • Adding custom syscall description
  • USB and network fuzzing
  • Fixing syzbot bugs

References