Using Mellanox Infiniband 40gb adapters with Proxmox v5.x

dependencies

apt-get -y install debhelper autotools-dev dkms zlib1g-dev python wget quilt python-libxml2 swig dpatch graphviz chrpath

download and install

wget http://www.mellanox.com/downloads/ofed/MLNX_EN-4.2-1.0.1.0/mlnx-en-4.2-1.0.1.0-debian9.0-x86_64.tgz
tar xzvf mlnx-en-4.2-1.0.1.0-debian9.0-x86_64.tgz
cd mlnx-en-4.2-1.0.1.0-debian9.0-x86_64
./install –skip-distro-check

then run this:

wget http://download.proxmox.com/debian/pve/dists/stretch/pvetest/binary-amd64/pve-headers-4.13.4-1-pve_4.13.4-25_amd64.deb
dpkg -i pve-headers-4.13.4-1-pve_4.13.4-25_amd64.deb

and reboot!

THEN… do this:

From the software side

sudo apt-get install opensm infiniband-diags librdmacm-dev libibverbs-dev libmlx4-dev

Then add these to /etc/modules via nano /etc/modules and pasting these lines in.

mlx4_ib
ip_ipoib
rdma_ucm

Then reboot.