Starting and Stopping Controller Services

The following five services are required for controller operation:

  • vermilion-core — Orchestrates core deployment workflows
  • cobalt — Manages environment services
  • rabbitmq-server — Messaging broker
  • mongod — Database server
  • monit — Monitoring and recovery daemon

Service operations are handled using scripts inside the /etc/init.d/ directory. Each service supports the following actions:

  • start
  • stop
  • restart
  • status

You can confirm whether or not a service is alive with the ‘status’ action.

/etc/init.d/cobalt status

Backup and Restore

Backup the Controller State


This step requires root privileges.

The controller uses a local MongoDB database to persist information between restarts. In order to restore the controller state, you must backup the following files:

  • Original controller distribution
  • Zone settings
  • MongoDB database’s contents

To create a consistent MongoDB database, use the mongodump command bundled with MongoDB client. Before using mongodump, you must stop vermillion-core and cobalt. The following example demonstrates how to create a backup archive in the /tmp folder:

# Stop services

/etc/init.d/vermilion-core stop
/etc/init.d/cobalt stop

# Change working directory to /tmp and create backup

cd /tmp; rm -rf dump
tar -czf mongodb.`date +%F%.H%M%S`.tar.gz ./dump
rm -rf dump

# Start services

/etc/init.d/vermilion-core start
/etc/init.d/cobalt start

Restore the Controller State on a Clean Server


This step requires root privileges.

To restore the controller on a clean server, the following files must be available:

  • Original controller distribution
  • Zone settings
  • Contents of the MongoDB database


Before re-installing the controller on a server where it has been already installed, remove the following packages:

  • cobalt
  • vermilion-core
  • mongo-10gen-server

Additionally, clean out the /var/lib/mongo directory.

  1. Before installing an controller, you need to restore the MongoDB database (which requires manually installing MongoDB from the files provided by the controller archive):

    # Unpack the provided archive into temporary folder
    tar -xzf %%PKG_NAME%%.tar.gz -C /tmp
    # Navigate to folder containing bundled MongoDB
    cd /tmp/%%PKG_NAME%%/deps/21-mongo
    # Install MongoDB
    yum -y --nogpgcheck --disablerepo=* install *.rpm
    /etc/init.d/mongod restart
  2. Restore the MongoDB database from the backup:

    # Unpack archive into temp folder
    tar -xzf backup.tar.gz -C /tmp
    # Restore the databases
    cd /tmp
  3. Proceed with the controller installation.

Regular Maintenance

MongoDB Database Size

The MongoDB database (used to persist the controller state) will grow in size and should be occasionally compacted. The recommended periodicity is once per month; however, you may want to compact it more often if you observe fast growth caused by intensive use.


Stop the``cobalt`` and vermilion-core services before performing database maintenance.

The following commands can be used to compact the database:

mongo cobalt --eval 'db.repairDatabase()'
mongo worker --eval 'db.repairDatabase()'


repairDatabase requires free disk space that is equivalent to the size of your current data set, plus 2 Gigabytes. More information on this command could be found in the MongoDB documentation: http://docs.mongodb.org/manual/reference/command/repairDatabase/#dbcmd.repairDatabase