Cuckoo com Vmware Esxi

Cuckoo is an open-source malware analysis platform using sandboxing technology. The tool allows people like us to analyze malicious binaries in an isolated environment.
Since Cuckoo is commonly used with Oracle VirtualBox as its virtualization platform, a majority of online documentation is focused on configuration using VirtualBox. PlantainStan and I decided to test running Cuckoo on ESXi and document our success.
This guide will help with the basic configuration of ensuring Cuckoo properly interacts with ESXi. We will continue to update this post as we make continue to make an even more baller Cuckoo environment!
Note: In order to successfully interact with vSphere's API, you will need the VMWare ESX Standard license. API functionality is required for Cuckoo to work with ESX.

Configure ESX

Since this guide is not a "how to" on installing ESXi, we will assume that you have successfully installed the hypervisor on your system. There are plenty of online resources available to assist with this task.
In order for Cuckoo to properly perform network captures, you need to configure ESX to allow for traffic monitoring.

Virtual Switches

You will need to set up an addition vSwitch for the monitor interface allowing Cuckoo to acquire PCAPs during analysis. Make sure that Cuckoo's second interface is a member of this port group. Each victim VM will use the Monitor port group on its primary interface.

The second vSwitch (vSwitch1) will have settings identified below:

Configure Cuckoo Host VM

For our testing, we used Ubuntu Server 14.04.1 LTS as the Host OS.
The important configuration that needs to be done when creating the VM in esx is to add a second network adpter to the Monitor network.

Install Core Dependencies

First, update the OS and install the many dependencies required to run Cuckoo on ESX.

$ sudo apt-get update
$ sudo apt-get upgrade –y

$ sudo apt-get install python python-pip mongodb python-sqlalchemy python-bson python-dpkt python-jinja2 python-magic python-pymongo python-gridfs python-bottle python-pefile python-chardet python-django libffi-dev libssl-dev -y

Install and Compile libvirt with ESX Drivers

libvirt is an open source API, daemon and management tool for managing ESXi. The tool is the key for allowing Cuckoo to control the VMs. We will install and compile libvirt with the ESX driver. Note that the default version of the libvirt package that comes with Ubuntu does not have ESX support. A lot of write-ups out there fail to mention this.

$ sudo apt-get install gcc make pkg-config libxml2-dev libgnutls-dev libdevmapper-dev libcurl4-gnutls-dev python-dev libpciaccess-dev libxen-dev libnl-dev uuid-dev xsltproc -y

$ wget http://libvirt.org/sources/libvirt-1.3.1.tar.gz
$ tar –zxvf libvirt
$ cd libvirt
$ ./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc --with-esx=yes
$ make
$ sudo make install
$ sudo pip install libvirt-python

Test to make sure that libvirt was properly installed by issuing the following commands. The first command will output the current version installed. The second will prompt for the ESX root password and from there you can power the VMs to make sure libvirt is working correctly.

$ virsh –v
1.3.1  
$ virsh –c esx://root@<esx ip>?no_verify=1
Enter root’s password for <esx ip>:  
virsh: start [vm name] #if you have a spare vm built, this will turn it on (dont use the cuckoo vm)  

Configure Networking

We need to set up the interfaces on your host VM to allow for traffic monitoring. Edit the /etc/network/interfaces file using your favorite text editor. Obviously, change 'x.x.x.x' to match your networking configuration.

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo  
iface lo inet loopback

# The primary network interface
auto eth0  
iface eth0 inet static  
    address x.x.x.x
    netmask x.x.x.x
    gateway x.x.x.x
    dns-nameservers x.x.x.x

# The Monitor Network interface
auto eth1  
iface eth1 inet manual  
    up ip address add 0/0 dev $IFACE
    up ip link set $IFACE up
    up ip link set $IFACE promisc on
down ip link set $IFACE promisc off  
down ip link set $IFACE down

The final step is to add a few lines to /etc/rc.local to ensure our NIC enters promisc mode on reboot.

ifconfig eth1 up  
ifconfig eth1 promisc  
exit 0  

Install additional dependencies

There are some additional useful tools that really make Cuckoo awesome.

Install TCPDump

This needs to be installed and configured to create pcaps.

$ sudo apt-get install tcpdump libcap2-bin -y
$ sudo setcap cap_net_raw,cap_net_admin=eip /usr/sbin/tcpdump

Install pydeep

$ sudo apt-get install ssdeep python-pyrex libfuzzy-dev -y
$ wget https://github.com/kbandla/pydeep/archive/0.4.tar.gz
$ tar –zxvf 0.4.tar.gz
$ cd 0.4/
$ python setup.py build
$ sudo python setup.py install

Install Yara

$ sudo apt-get install libpcre3 libpcre3-dev autoconf libtool -y

$ git clone https://github.com/plusvic/yara.git
$ cd yara
$ ./bootstrap.sh
$ ./configure
$ make
$ sudo make install

Now build yara-python

$ cd yara-python
$ python setup.py build
$ sudo python setup.py install

INSTALL CUCKOO!!

Finally ready to install cuckoo onto your host VM. Follow the commands below and you are almost there (not really).

$ sudo mkdir /opt/cuckoo
$ sudo chown user:group /opt/cuckoo
$ git clone https://github.com/cuckoosandbox/cuckoo.git
$ cd cuckoo
$ sudo pip install -r requirements.txt

$ sudo adduser cuckoo

Configure Guest VM(s)

Cuckoo's documentation does a great job detailing the steps required for Guest VM configuration. For more information visit their documentation site.

vSphere Configuration

The important configuration item that is needed when initially setting up your VM is to ensure that the VM's NIC is connected to the "Monitor" VM Port Group.

Install Python

Since we set this up using Windows VMs and Python is not installed by default, we will need to download the Windows installer from http://www.python.org/getit/. Also, the Cuckoo developers recommend Python 2.7. In addition, we need to install the Python Image Library for taking screenshots of the Windows desktop during analysis. It can be downloaded from http://www.pythonware.com/products/pil/.

Network Configuration

First, disable Windows Firewall and Automatic Updates as it can affect the behavior of the malware and can pollute the network analysis performed by Cuckoo.
Change the network adapter configuration via control panel to match the following. Per their documentation, Cuckoo is unstable when using DHCP.
IP Address: 192.168.56.X
Subnet: 255.255.255.0
Gateway: 192.168.56.1
Primary DNS: 192.168.56.1

Install the Agent

In the agent/ directory of the Cuckoo folder you will find the agent.py file. Copy it to the Guest OS and run it. This will launch the XMLRPC server which will be listening for connections.
Note: On Windows, simply launching the script will spawn a python window. If you want to hide it, rename agent.py to agent.pyw. If you want it to be launched on Windows boot, place it in the Startup folder.

Additional Software

Depending on what kind of files you want to analyze, you may want to install additional software like browsers, PDF readers, Office, etc. Don't forget to disable "auto update" and "check for updates" on these programs.

Save a Snapshot

With the VM turned on and the agent running, save a snapshot of the VM and record the name of the snapshot (this will be needed when configuring Cuckoo).

Configure Cuckoo

Good news. You are nearly done! Trust us, it's totally worth the time and effort. On the host VM, go into each configuration file listed below in ../cuckoo/conf/ and make the recommended changes.

Cuckoo.conf

[cuckoo]
machinery = esx  
memory_dump = on

[result server]
ip = [Cuckoo vm IP]  
port = 2042  

Auxiliary.conf

[sniffer]
enabled = yes  
tcpdump = /usr/sbin/tcpdump  

Esx.conf

dsn = esx://[esx management ip]/?no_verify=1  
username = [esx username] #requires admin to control vms  
password = [esx password] #i know, its clear text :(

machines = [list vm of names cuckoo will use to analyze]  

Each VM will have its own section within the file. We used a VM named "winxp". Example configuration below.

[winxp]
# Specify the label name of the current machine as specified in your
# libvirt configuration.
label = winxp

# Specify the operating system platform used by current machine
# [windows/darwin/linux].
platform = windows

# Please specify the name of the base snapshot. This snapshot should be taken
# with agent in startup and the machine shut down.
snapshot = cuckooxp

# Specify the IP address of the current virtual machine. Make sure that the
# IP address is valid and that the host machine is able to reach it. If not,
# the analysis will fail. You may want to configure your network settings in
# /etc/libvirt/<hypervisor>/networks/
ip = 192.168.1.110

# (Optional) Specify the snapshot name to use. If you do not specify a snapshot
# name, the KVM MachineManager will use the current snapshot.
# Example (Snapshot1 is the snapshot name):
snapshot = cuckooxp

Reporting.conf

[jsondump]
enabled = yes  
indent = 4  
encoding = latin-1  
calls = yes

[reporthtml]
enabled = no

[mongodb]
enabled = yes  
host = 127.0.0.1  
port = 27017  
db = cuckoo  
store_memdump = yes  
paginate = 100

[moloch]
enabled = no  

Processing.conf

[analysisinfo]
enabled = yes

[analysisinfo]
enabled = yes

[baseline]
enabled = no

[behavior]
enabled = yes

[buffer]
enabled = yes

[debug]
enabled = yes

[droidmon]
enabled = no

[dropped]
enabled = yes

[dumptls]
enabled = yes

[memory]
# Create a memory dump of the entire Virtual Machine. This memory dump will
# then be analyzed using Volatility to locate interesting events that can be
# extracted from memory.
enabled = no

[network]
enabled = yes

[screenshots]
enabled = yes  
tesseract = /usr/bin/tesseract

[static]
enabled = yes

[strings]
enabled = yes

[virustotal]
enabled = no

Start Cuckoo

Finally, we can start analyzing malicious files! Go ahead and start the python script.

$ python cuckoo.py

Since we want to make it a little more user friendly, we previously configured the web gui. Now, start the web interface from the web folder. Both of these scripts can be set up to run on boot and as a service.

../cuckoo/web$ python manage.py runserver 0.0.0.0:8000