Peer Setup

Subutai is based on Snappy, and has its own snap package in the Ubuntu One Store. Installing the Subutai Snap package on a snapd supported (see the support matrix) operating system enables the host as a Subutai Resource Host (RH) RHs have shareable resources (RAM, CPU, DISK) and are added to peers to increase their capacity. An unlimited number of RHs added may be added to a peer. RHs may be physical or virtual machines using any hypervisor. . Importing a special container, named “management”, automatically installs the Subutai Management Console into the RH turning it into a Subutai Peer A Subutai Peer is a set of managed RHs on the same LAN. The Subutai Management Software orchestrates the creation and destruction of cloud segments running in a peer. It’s what swarms with other peers to create virtual private cloud environments. At least one RH in the peer MUST run a management container. .

  • Recommended Hardware Configuration

    Unlimited hardware configuration options are possible. However, if setting up a trial peer with a single RH we recommend running it on a virtual machine with at least:

    • 2 or more CPU cores
    • 2G of RAM (8G+ best)
    • 2 disks or partitions, one for the operating system >10G, and one for container storage >100G
    • 1 network adapter 1Gbit (bridged)

    Any hypervisor or cloud provider instance may be used so long as these virtual hardware resources are available to the guest operating system. Peers can run on Hyper-V, GCE, AWS, KVM, VirtualBox, VMWare, and Parellels.

    A virtual machine is not absolutely necessary, but a convenient However, before login, make sure the client browser has the Subutai E2E plugin installed with and contains PGP keys. approach. A peer can be installed on an existing physical host with full security and isolation without a VM. In this case, similar hardware requirements apply to the existing physical host.

  • Operating System Requirements

    Some general caveats and requirements for the operating system on which the snap package is being installed are:

    1. Using a freshly installed operating system is preferable,
    2. The operating system should have snapd installed and updated,
    3. All system components should be updated to the latest available version including system libraries and the kernel,
    4. Avoid installing any additional components on the system,
    5. Avoid desktop distributions due to NetworkManager’s dnsmask running on port 53
    6. The system should have network connectivity (via a bridge if virtual)
    7. A DHCP service should be present on the LAN to acquire an IP address
    8. Check that the required network ports listed below are not used

      Use this script to check:


      (i.e. netstat -lnp | grep ":53 " or lsof -i :53) before installing the snap:
      • udp/53 - DNS service port
      • udp/67 - DHCP service port
      • tcp/80 - Web service port
      • tcp/443 - Secure web service port
      • udp/1900 - Service discovery service port
      • udp/6881 - P2P service port
      • tcp/8086 - Metrics service port
      • tcp/8443 - Subutai Management Console web service port
      • tcp/8444 - Subutai Management LAN communication service port

    Using a guest virtual machine allows for maximum flexibility to install a stock unmodified guest VM operating system and reduces complications. Ubuntu Server 16.04.3 or 17.04 operating systems are best: they come pre-installed with snapd. Debian is also a good candidate, but the snapd package needs to be installed:

    sudo apt-get install snapd

    The snap has been tested on the following operating systems:

    • Ubuntu Core 16
    • Ubuntu Server: 16.04.3 and 17.04
    • Ubuntu Desktop: 16.04.3 and 17.04
    • Debian 9.1.0 - Stretch
    • Debian 9.1.0 - Stretch w/ MATE
    • Mint 18.2

    NOTE: All the desktop editions and other headed configurations (in bold red) have the NetworkManager problem, and the non-Ubuntu distributions need snapd installed. Debian Stretch (headless) needs snapd installed. See the warnings below about NetworkManager if you’re installing the snap on a desktop system with NetworkManager (most likely) installed.

  • WARNING: NetworkManager on desktop editions will give you a bad day!

    Virtual or not, desktop editions of all distributions should be avoided, since they unnecessarily waste resources and have serious issues because of NetworkManager which installs dnsmasq on port 53. If left without a choice you’ll have to disable NetworkManager’s dnsmasq if netstat -lnp | grep ":53 " produces any output. Here’s how:

    sudo systemctl disable systemd-resolved.service
    sudo service systemd-resolved stop

    Edit /etc/NetworkManager/NetworkManager.conf and change the dns property in the [main] section so it is set to default: dns=default.

    sudo rm /etc/resolv.conf
    sudo service network-manager restart
    sudo killall dnsmasq
    sudo netstat -lnp | grep “:53 “
    sudo lsof -i :53

    The last netstat or lsof command should not produce any output now. NOTE: if NetworkManager was not configured properly, after restart, you may need to add your nameserver into the /etc/resolv.conf file it generates. It’s best to add this to the NetworkManager connection configuration if you don’t want the resolv.conf file to be overwritten.

Install the Snap Package

  • Installing snapd on various operating systems

    Before installing the Subutai Snap you may need to install snapd. Here are some commands for installing and setting up snapd on non-Ubuntu Linux distributions:

    • Mint, Debian: sudo apt-get install snapd
    • Fedora 24: sudo dnf install snapd && sudo systemctl enable --now snapd.socket

If snapd is properly installed, and up to date, the following command downloads and installs the Subutai Snap package:

sudo snap install subutai --beta --devmode
snap list

The last command after installation checks that the package is installed. You should see similar output from it:

ubuntu@testbox:~$ snap list
Name Version Rev Developer Notes
core 16-2.27.5 2774 canonical core
subutai 5.0.2 183 devmode
ubuntu-image 0.12+real1 44 canonical devmode
  • Snap Installation Troubleshooting

    On non-Ubuntu Core distributions this might cause the core snap to be downloaded as well as the Subutai Snap which depends on it. Ubuntu Core come with the core snap already installed. Don’t be surprised to see this other core snap getting downloaded and installed.

    There are known problems with certain hypervisors and bridged networking over WiFi connections. If you’re having connectivity issues after snap installation please see the VM Wifi workaround.

    The snap package comes with the p2p daemon which uses UDP port 6881. If you previously installed the p2p daemon package to connect to your cloud environments, then you’ll have a port conflict. You’ll need to stop and remove the p2p daemon package.

    Some distributions do not immediately update paths immediately on snap package installation. If you don’t see the command subutai on the path just logout and log back in. It should now be visible.

Setup Container Storage

Containers storage is created on a block device (partition or disk). By default, Subutai uses BTRFS but can use other copy-on-write filesystems like ZFS. Use the following command to add a BTRFS storage device to your RH:

sudo /snap/subutai/current/bin/btrfsinit /dev/vdb
  • Block Device

    Do not forget to use your appropriate block device in place of /dev/vdb. It can be a disk, a partition, iSCSI target LUN, or an LVM logical volume. It just needs to be a block device.

Now you should see the block device mounted:

df -h /dev/vdb
subutai@ud1704s:~$ df -h /dev/ud1704/btrfs
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/ud1704-btrfs 94G 17M 92G 1% /var/snap/subutai-master/common/lxc

Congratulations you now have a Subutai Resource Host. You can start importing and running containers on it.

  • Adding RHs to Existing Peers

    You can use this same process to create more RHs and add them to peers. RHs must be on the same LAN. The process to add a RH uses the Simple Service Discovery Protocol. RHs that are not bound to a peer on the network announce their availability. The peer console will show the RH and the admin can use an authorization process to approve adding the RH to the peer.

To convert this RH into a peer the management container must be imported.

Import the Management Container

Now that the Subutai Snap package is installed, and you have an active RH, you can import the management software and its console to convert this host into a peer:

sudo subutai import management
Troubleshooting a CDN unreachable error
When installing directly on a desktop edition, the NetworkManager configuration may not configure the /etc/resolv.conf file properly to resolve CDN addresses causing “CDN unreachable errors” with the import command. If this happens, you must configure NetworkManager to use the right nameserver.

The command will output the Subutai Console’s access URL. See the red rectangle in the screen capture below:

The peer can now be managed from this URL via the Subutai Management Console. The default admin user password is secret. Admins will be asked to change this password on their first login. However, before the logging in make sure the client browser has the Subutai E2E plugin installed with your PGP keys.

Post Installation

With a peer up, some post installation steps are needed to use it.

Creating and Installing PGP Keys

Subutai uses PGP key pairs to identify all entities including users like cloud owners and peer administrators. The default administrator account on the new peer needs to be associated with the global identity of the peer owner to share or rent resources to others.

  • Manually managing PGP keys

    Users knowing how to manage their own keys can use other tools to do so. Various platform specific PGP tools like GPG on Linux can be used on the command line interface to create and use PGP keys. Although possible it’s a major hassle, especially for those that are not PGP savvy worse yet it can actually be detrimental for those who don’t understand PKI or how to secure their keys. Even PGP pros will be annoyed when having to manually deal with frequent challenge and response authorization operations.

    This is why we created browser plugins. Additionally we’re working on FIDO and hardware key fob integration to always improve security. The Subutai E2E browser plugins have these benefits while still leaving the manual approach open.

As mentioned, the Subutai E2E Browser Plugin / Extension should be installed in the browser with either an existing key or a newly generated PGP key before logging into the management console. The plugin will generate a new key pair for you if you don’t already have one. You can easily import and export keys into the plugin.

  • Watch video about "Creating and Installing PGP Keys"
  • Supported Browsers

    At this point we’ve stopped supporting Firefox because of changes to their extensions API, and have yet to provide support for Edge. Chrome and Safari are the two supported browsers.

Although the P2P executable can be used to connect to environments and talk to peer swarms via CLI commands, we recommend installing the OS Tray application for convenient use on a desktop system. The tray application allows you to swarm into environments and even control your peers while showing you your Goodwill balance as a wallet. So for a tight desktop client setup we recommend installing:

  • the browser plugin
  • the p2p package (IFF the snap is not installed on the same system)
  • the OS tray application

Packages for Windows, Linux, and Mac OS operating systems are available for the OS tray application. For Mac and Windows the installers are trivial and standard MSI and Mac packages. For Linux:

sudo dpkg -i subutai-tray-6.0.1.deb
sudo apt-get install -f
  • Watch video about how to "Install the Subutai OS Tray Application"