Linux Networking: Proxmox VXLAN Setup

Oct 9, 2025

To deepen my understanding of Linux networking, I set up a small Proxmox cluster with VXLAN tunneling and an Envoy proxy in front of the nodes. It started as a simple learning project, but quickly turned into one of those rare experiences where the concepts finally click instead of just being memorized.

TLDR / Key Takeaways

What made this project memorable wasn’t just getting everything to work. It was the feeling that things finally connected.
When I read the explanation about the Diffie-Hellman key exchange failing because the packet size exceeded the MTU, it immediately made sense. Not because I memorized anything, but because everything I’d learned on my journey so far suddenly aligned. And that felt incredibly rewarding.

I wasn’t just applying a fix , I really understood why it was the fix.

And once again, I noticed something I keep rediscovering: whenever I take the time to visualize something, it stops being complicated and becomes intuitive. I had this experience before, and this project confirmed it again.

Key takeaways from this project:

topology.png Topology drawn by me

Challenges

SSH Connection problems from vm100 to vm101

The SSH Connection from VM100 (Frontend) to vm101 (Backend) wouldn’t work. I tried tinkering with the sshd config.

To debug I had to run the sshd service manually on vm101 with -o you can enable specific options without changing the config file permanently.

VM101$ sudo /usr/sbin/sshd -Dd -p 2222 -o PasswordAuthentication=yes

VM100$ ssh -vv -p 2222 user@172.20.0.101

sshd-debug

In the last line we can cleary see that the error occurs during the key exchange. After some googling i found an answer to the problem. (https://unix.stackexchange.com/a/786463) . And it made perfect sense.

I configured the MTU on the VXLAN to 1450. Since an Eliptic-Curve Diffie-Hellman (with a maximum key size of about 512 bits) shouldn’t exceed that MTU, the SSH2_KEXINIT message was likely being fragmanted in a way that prevented the connection from being established.

After setting the MTU on vm100 to 1450 it worked fine.

sudo ip link set dev ens18 down
sudo ip link set dev ens18 mtu 1450
sudo ip link set dev ens18 up

mtu-fix

Also this seems to be a bug in Proxmox. The normal behavior should be, that the virtual NIC inherits the MTU from the bridge. Which in our case is the VXLAN with MTU set to 1450. (https://forum.proxmox.com/threads/bug-vxlan-and-mtu.161412/ )

DNS

I also had some issues with DNS, so I set the DNS globally through /etc/systemd/resolved.conf

Setup and Configuration

Netplan

Configure network using netplan https://netplan.readthedocs.io/en/stable/examples/#how-to-configure-a-static-ip-address-on-an-interface

configure static ip (node01):

network:
  version: 2
  ethernets:
    ens2:
      addresses:
        - 192.168.250.2/24
      nameservers:
        addresses: [ "1.1.1.1", "8.8.8.8" ]
      routes:
        - to: default
          via: 192.168.250.1

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:97:29:b4 brd ff:ff:ff:ff:ff:ff
    altname enp0s2
    altname enx5254009729b4
    inet 192.168.250.2/24 brd 192.168.250.255 scope global ens2
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe97:29b4/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
3: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 52:54:00:80:cc:ec brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname enx52540080ccec
    inet6 fe80::5054:ff:fe80:ccec/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever

Network

VXLAN Source: https://forum.proxmox.com/threads/vxlan-multitenant.50686/#post-256806

apt install ifupdown2

ifreload -a

After rebooting the GW and node01 I had to restart the nftables service and I choose to enable it by default:

systemctl enable nftables
systemctl start nftables

Create VM 101 with cloud-init

qemu-img resize noble-server-cloudimg-amd64.img 16G

qm create 8001 --name "ubuntu-2404-cloudinit-template" --ostype l26 \
    --memory 4096 \
    --agent 1 \
    --bios ovmf --machine q35 --efidisk0 local-lvm:0,pre-enrolled-keys=0 \
    --cpu host --sockets 1 --cores 2 \
    --vga serial0 --serial0 socket  \
    --net0 virtio,bridge=vmbr0

qm importdisk 8001 jammy-server-cloudimg-amd64.img local-lvm
qm set 8001 --scsihw virtio-scsi-pci --virtio0 local-lvm:vm-8001-disk-1,discard=on
qm set 8001 --boot order=virtio0
qm set 8001 --scsi1 local-lvm:cloudinit


mkdir /var/lib/vz/snippets
cat << EOF | tee /var/lib/vz/snippets/vendor.yaml
#cloud-config
runcmd:
    - apt update
    - apt install -y qemu-guest-agent
    - systemctl start qemu-guest-agent
    - reboot
# Taken from https://forum.proxmox.com/threads/combining-custom-cloud-init-with-auto-generated.59008/page-3#post-428772
EOF

qm set 8001 --cicustom "vendor=local:snippets/vendor.yaml"
qm set 8001 --tags ubuntu-template,24.04,cloudinit
qm set 8001 --ciuser ins
qm set 8001 --cipassword ins@lab 
qm set 8001 --sshkeys ~/.ssh/authorized_keys
qm set 8001 --ipconfig0 ip=dhcp

sudo qm template 8001

Deploy Application on VM

Add port forwarding rule to nftables:

sudo sed -i 's/.*100:22.*/&\n    tcp dport 1337 dnat to 192.168.250.100:1337/' \
/etc/nftables.conf
sudo service nftables restart

Frontend (VM100)

#!/bin/bash
# /usr/local/bin/frontend-listener.sh
# Frontend proxy using socat – forwards traffic to backend VM

BACKEND_IP="172.20.0.101"
BACKEND_PORT="1337"
FRONTEND_PORT="1337"
LOGFILE="/var/log/frontend-listener.log"

echo "$(date): Starting frontend listener, forwarding ${FRONTEND_PORT} -> ${BACKEND_IP}:${BACKEND_PORT}" >> "$LOGFILE"

# loop forever to keep it running
while true; do
  socat TCP-LISTEN:${FRONTEND_PORT},fork TCP:${BACKEND_IP}:${BACKEND_PORT} >> "$LOGFILE" 2>&1
  echo "$(date): socat exited, restarting..." >> "$LOGFILE"
  sleep 2
done

[Unit]
Description=Frontend Listener (socat forwarder to backend)
After=network-online.target
Wants=network-online.target

[Service]
ExecStart=/usr/local/bin/frontend-listener.sh
Restart=always
RestartSec=3
User=root
StandardOutput=append:/var/log/frontend-listener.log
StandardError=append:/var/log/frontend-listener.log

[Install]
WantedBy=multi-user.target

Backend (VM101)

#!/bin/bash
# /usr/local/bin/backend-listener.sh
# Simple backend listener using named pipe + nc

LOGFILE="/var/log/backend-listener.log"
FIFO="/tmp/f"

# cleanup previous fifo if needed
rm -f "$FIFO"
mkfifo "$FIFO"

# endless loop
while true; do
  cat "$FIFO" | printf "backend @ $(date)\n\n(╯°□°）╯︵ ┻━┻\nCtrl-C to exit\n" \
  | nc -l -p 1337 >> "$LOGFILE" 2>&1 > "$FIFO"
done

[Unit]
Description=Custom Backend Listener on port 1337
After=network-online.target
Wants=network-online.target

[Service]
ExecStart=/usr/local/bin/backend-listener.sh
Restart=always
RestartSec=3
User=root
StandardOutput=append:/var/log/backend-listener.log
StandardError=append:/var/log/backend-listener.log

[Install]
WantedBy=multi-user.target

VXLAN Setup

Source: https://git.proxmox.com/?p=pve-docs.git;a=blob_plain;f=vxlan-and-evpn.adoc;hb=HEAD

auto lo
iface lo inet loopback

iface ens2 inet manual

auto ens3
iface ens3 inet static
        address 192.168.100.2/24

auto vxlan20000
iface vxlan20000 inet manual
        vxlan-id 20000
        vxlan-local-tunnelip 192.168.100.2
        vxlan-remoteip 192.168.200.2
        mtu 1450

auto vmbr0
iface vmbr0 inet static
        address 172.20.0.10/24
        bridge_ports vxlan20000
        bridge-stp off
        bridge-fd 0
        mtu 1450

auto vmbr1
iface vmbr1 inet static
        address 192.168.250.2/24
        gateway 192.168.250.1
#       dns-nameservers 1.1.1.1 8.8.8.8 globally defined in /etc/systemd/resolved.conf
        bridge-ports ens2
        bridge-stp off
        bridge-fd 0

auto lo
iface lo inet loopback

iface ens2 inet manual

auto ens3
iface ens3 inet static
        address 192.168.200.2/24

auto vxlan20000
iface vxlan20000 inet manual
        vxlan-id 20000
        vxlan-local-tunnelip 192.168.200.2
        vxlan-remoteip 192.168.100.2
        mtu 1450

auto vmbr0
iface vmbr0 inet static
        address 172.20.0.20/24
        bridge_ports vxlan20000
        bridge-stp off
        bridge-fd 0
        mtu 1450

auto vmbr1
iface vmbr1 inet static
        address 192.168.250.3/24
        gateway 192.168.250.1
#       dns-nameservers 1.1.1.1 8.8.8.8 globally configured in /etc/systemd/resolved.conf
        bridge-ports ens2
        bridge-stp off

# This file is generated from information provided by the datasource.  Changes
# to it will not persist across an instance reboot.  To disable cloud-init's
# network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    bridges:
        lan:
            addresses:
            - 192.168.250.1/24
            interfaces:
            - switchports
    ethernets:
        ens2:
            accept-ra: true
            dhcp4: true
            optional: false
        ens4:
            addresses:
              - 192.168.100.1/24
            dhcp4: false
            optional: true
        ens6:
            addresses:
              - 192.168.200.1/24
            dhcp4: false
            optional: true
        switchports:
            accept-ra: false
            dhcp4: false
            dhcp6: false
            link-local: []
            match:
                name: en*
            optional: true
    version: 2
Powered by Hugo with Console Theme.