Solving a Proxmox Networking Mystery: USB Ethernet Dongle Filtering Traffic
I recently encountered a strange networking issue while running Proxmox on a MacBook Air using a USB-to-Ethernet dongle. Although Proxmox could successfully ping devices on my local network, attempts to connect using certain other protocols (e.g., HTTP, netcat) would hang indefinitely.
To troubleshoot, I compared the behavior of my macOS laptop (connected via Wi-Fi) with Proxmox running on the same MacBook Air (connected via Ethernet through a USB dongle). Despite being on the same subnet, Proxmox devices could ping a specific device (EnvisaLink DUO) but failed to connect using curl or netcat.
Initial Observations
On macOS (Connected via Wi-Fi)
- Ping:
ping 192.168.86.150→ Success - HTTP Request:
curl http://192.168.86.150→ Success (retrieved the expected webpage) - Netcat:
nc -zv 192.168.86.150 4025→ Success - MTU Size:
ifconfigreturned an MTU size of 1500 - Port Scan:
nmap -p 80 192.168.86.150showed port 80 as open.
On Proxmox (Host and VMs Connected via USB Dongle)
- Ping:
ping 192.168.86.150→ Success - HTTP Request:
curl http://192.168.86.150→ Hangs with no response - Netcat:
nc -zv 192.168.86.150 4025→ Hangs - MTU Size:
ip link showreturned an MTU size of 1500 - Port Scan:
nmap -p 80 192.168.86.150showed port 80 as filtered.
This was puzzling: even though Proxmox was on the same subnet and could ping the target device, it couldn’t establish connections using other protocols.
Troubleshooting Steps
1. Verified Network Configuration
I confirmed that both the Proxmox host and its VMs had static IP addresses in the correct subnet.
2. Checked ARP Resolution
The ARP table showed that the target device was reachable. I flushed the ARP table to rule out any stale entries, but the issue persisted.
3. Ran tcpdump
Capturing outgoing packets with tcpdump revealed that SYN packets were being sent to the target device, but no responses were received.
4. Tested DNS Resolution
I added the target device’s IP to /etc/hosts to ensure hostname resolution wasn’t an issue. However, this didn’t solve the problem.
5. Checked MTU Size
Both macOS and Proxmox reported an MTU size of 1500, ruling out MTU mismatches as a potential cause.
6. Compared nmap Results
This was the key clue:
- On macOS,
nmapreported port 80 as open. - On Proxmox,
nmapreported port 80 as filtered.
Since no firewall was running on Proxmox, I suspected that the USB Ethernet dongle might be interfering with traffic.
Honing in on the USB Ethernet Dongle
Given that the issue only occurred when using the USB-to-Ethernet dongle, I started to suspect that the dongle might be filtering or altering traffic. To investigate further, I checked the dongle’s offloading features using ethtool:
ethtool -k enx803f5d096607
The output showed that RX and TX checksumming were enabled:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
Offloading generally improves performance by allowing the network interface to handle certain tasks. However, in some USB Ethernet dongles, enabling offloading can cause issues with packet integrity or filtering.
The Fix
To test whether disabling RX/TX checksumming would resolve the issue, I ran:
sudo ethtool -K enx803f5d096607 rx off tx off
I then re-ran the nmap scan:
nmap -p 80 192.168.86.150
This time, the port was reported as open:
PORT STATE SERVICE
80/tcp open http
A quick test with curl also worked:
curl http://192.168.86.150
Result:
<HTML><BODY><H1>Server Requires Authentication</H1></BODY></HTML>
Finally, the issue was resolved.
Automatically Disabling RX/TX Checksumming at Boot
The command sudo ethtool -K enx803f5d096607 rx off tx off is temporary and will be reset on reboot. To make it persistent, I created a systemd service:
Enable the service:
sudo systemctl enable disable-offload.service
Add the following content:
[Unit]
Description=Disable RX/TX checksumming on Ethernet USB dongle
After=network.target
[Service]
ExecStart=/sbin/ethtool -K enx803f5d096607 rx off tx off
Type=oneshot
RemainAfterExit=true
[Install]
WantedBy=multi-user.target
Create a new systemd service file:
sudo nano /etc/systemd/system/disable-offload.service
This ensures that the ethtool command runs automatically at boot.
Why Disabling Checksumming Helped
USB Ethernet dongles can have limited hardware capabilities. Enabling checksum offloading can lead to packet corruption or filtering by the dongle itself. Disabling RX/TX checksumming forces the kernel to handle checksumming in software, which resolved the issue in this case.
Final Thoughts and Lessons Learned
If you’re running Proxmox on a machine with a USB-to-Ethernet dongle and encounter strange network behavior where ping works but other protocols hang, try disabling RX/TX checksumming using ethtool. This simple fix saved me hours of frustration and could help others avoid similar issues.
Sometimes, the problem isn’t with your network configuration or the target device but with the hardware in between.
Commands Recap
Check offloading status:
ethtool -k <interface>
Disable RX/TX checksumming:
sudo ethtool -K <interface> rx off tx off
Re-enable checksumming (if needed):
sudo ethtool -K <interface> rx on tx on