Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Bullseye Kernel breaks my Intel ethernet

Need help with peripherals or devices?
Post Reply
Message
Author
Crowley
Posts: 7
Joined: 2021-08-17 04:59

Bullseye Kernel breaks my Intel ethernet

#1 Post by Crowley »

Hi,

I also asked about this on reddit but have not found a solution. Google finds little help.

I upgraded my home server running Debian Buster to Bullseye and the new kernel causes my Intel ethernet adapter to work really slow with high unsteady latency. Booting back to Kernel 4.19.0-17-amd64 fixes the issue.

My card is a "Kalea Informatique Network Card PCIe 2.0 1x 4 Ports Quad Gigabit Ethernet - Chipset Intel 82576"

lspci:
04:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
04:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
05:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01

Ethtool gives this info on the adapter:

Code: Select all

        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: on (auto)
        Supports Wake-on: pumbg
        Wake-on: g
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes
When I compare the results of ethtool -k on both kernels, these lines are different:

Code: Select all

tx-udp-segmentation: on <- This is off on the working old kernel.
These lines are missing from the old kernel:

Code: Select all

tx-tunnel-remcsum-segmentation: off [fixed]
tx-gso-list: off [fixed]
rx-gro-list: off
macsec-hw-offload: off [fixed]
Not sure if these cause the problems and haven't found a away to get the settings to match. I'm somewhat new to linux so bare with me. Right now I just use the old kernel without any problems but of course would be nice to get new one to work.

Crowley
Posts: 7
Joined: 2021-08-17 04:59

Re: Bullseye Kernel breaks my Intel ethernet

#2 Post by Crowley »

Just did some more testing, booting the same machine with live Debian 11, the ethernet card doesn't work any better, and that is expected.
Next I tried booting live Ubuntu 21 and with that it works perfectly. The kernel version here is 5.11. So it seems like a Debian only issue?

Aki
Global Moderator
Global Moderator
Posts: 2823
Joined: 2014-07-20 18:12
Location: Europe
Has thanked: 69 times
Been thanked: 385 times

Re: Bullseye Kernel breaks my Intel ethernet

#3 Post by Aki »

Hello,

I apologize in advance for the triviality of the question: have you checked whether a new firmware specific to the network card is required with the new Debian 11 kernel version? You should be able to check, for example, running the following command as root user in Debian 11:

Code: Select all

journalctl --no-pager -b -0 | grep -i firmware
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org
⠈⠳⣄⠀

mm3100
Posts: 336
Joined: 2020-10-21 21:39
Has thanked: 8 times
Been thanked: 13 times

Re: Bullseye Kernel breaks my Intel ethernet

#4 Post by mm3100 »

You can to use isenkram-cli and just run


isenkram-autoinstall-firmware


It would automatically install needed packages if it can find them, and note you if some kernel modules ask for missing files binaries.

Crowley
Posts: 7
Joined: 2021-08-17 04:59

Re: Bullseye Kernel breaks my Intel ethernet

#5 Post by Crowley »

It's not a firmware issue, no firmware is missing as reported by those tools/commands.

Ethool reports it uses the following with the bullseye kernel:
driver: igb
version: 5.10.0-8-amd64

With the older one it's igb, 5.4.0-k. With the working Ubuntu it was 5.11.someting-generic I believe. The only one that does not work right is the bullseye kernel.

Edit:

And just to be clear, it "works" it just doesn't work well.

Kernel 4.19.194-3:

Code: Select all

PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.

64 bytes from 8.8.8.8: icmp_seq=1 ttl=57 time=19.3 ms

64 bytes from 8.8.8.8: icmp_seq=2 ttl=57 time=18.7 ms

64 bytes from 8.8.8.8: icmp_seq=3 ttl=57 time=18.9 ms

64 bytes from 8.8.8.8: icmp_seq=4 ttl=57 time=19.9 ms

64 bytes from 8.8.8.8: icmp_seq=5 ttl=57 time=19.4 ms

64 bytes from 8.8.8.8: icmp_seq=6 ttl=57 time=19.7 ms

64 bytes from 8.8.8.8: icmp_seq=7 ttl=57 time=19.8 ms
Kernel 5.10.0-8:

Code: Select all

PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.

64 bytes from 8.8.8.8: icmp_seq=1 ttl=57 time=37.8 ms

64 bytes from 8.8.8.8: icmp_seq=2 ttl=57 time=1032 ms

64 bytes from 8.8.8.8: icmp_seq=3 ttl=57 time=108 ms

64 bytes from 8.8.8.8: icmp_seq=4 ttl=57 time=841 ms

64 bytes from 8.8.8.8: icmp_seq=5 ttl=57 time=43.6 ms

64 bytes from 8.8.8.8: icmp_seq=6 ttl=57 time=95.7 ms

Aki
Global Moderator
Global Moderator
Posts: 2823
Joined: 2014-07-20 18:12
Location: Europe
Has thanked: 69 times
Been thanked: 385 times

Re: Bullseye Kernel breaks my Intel ethernet

#6 Post by Aki »

@Crowley:
Crowley wrote: 2021-08-18 17:19 It's not a firmware issue, no firmware is missing as reported by those tools/commands
[..]
And just to be clear, it "works" it just doesn't work well.
Good! So it's up to you to fine tuning the ethernet configuration.
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org
⠈⠳⣄⠀

Crowley
Posts: 7
Joined: 2021-08-17 04:59

Re: Bullseye Kernel breaks my Intel ethernet

#7 Post by Crowley »

Aki wrote: 2021-08-18 18:34 Good! So it's up to you to fine tuning the ethernet configuration.
Could you elaborate? How can I fine tune this? The exact same setup works right while booting the older kernel but not with the new. Also works with other Distros like Ubuntu with slightly newer Kernel. With the bullseye kernel the latency is really erratic and nothing seems to help.

To me this seems like a problem with Debian's igb driver...

mm3100
Posts: 336
Joined: 2020-10-21 21:39
Has thanked: 8 times
Been thanked: 13 times

Re: Bullseye Kernel breaks my Intel ethernet

#8 Post by mm3100 »

Copy your network configuration here, maybe someone will know more.

Crowley
Posts: 7
Joined: 2021-08-17 04:59

Re: Bullseye Kernel breaks my Intel ethernet

#9 Post by Crowley »

I'm not trying to be rude, but the configuration makes no difference, like I stated above the issue persist with live booting Debian 11 from usb also, I'd imagine the configuration doesn't get more basic than that? But of course if it will help I will paste anything that is asked but I need specifics.

The issue seems to be with this specific hardware and Debian 11, I suspect the igb driver implementation. I have tried to give all the information on the Ethernet card I know how to get.

scelfo
Posts: 2
Joined: 2021-09-30 17:49

Re: Bullseye Kernel breaks my Intel ethernet

#10 Post by scelfo »

I found this thread because I upgraded from buster to bullseye and I see the same problem--the network connection feels laggy a ping test shows results like what's in comment #5 (occasional ping times > 1000ms).

I'm running debian inside vmware ESXi 7. My network adapter in set in ESXi as "VMXNET 3" but I have also tried "E1000e" with no change. The physical hardware is an HP ProLiant DL380 Gen9 with an HPE Ethernet 1Gb 4-port 331i Adapter - NIC.

Something that's interesting... I have another much simpler virtual machine (less packages installed) that I upgraded from buster to bullseye that doesn't seem to have the problem. Both virtual machines have very similar configurations in ESXi.

I have a snapshot of my virtual machine before the upgrade and I'm just going to roll it back to before buster. I'll try to reproduce the behavior on a simple vanilla bullseye installation that isn't a machine I need to have working.

Crowley
Posts: 7
Joined: 2021-08-17 04:59

Re: Bullseye Kernel breaks my Intel ethernet

#11 Post by Crowley »

It's interesting that you have a 4-port NIC as well. Perhaps there's something there. The bug has also been reported earlier here: https://bugs.debian.org/cgi-bin/bugrepo ... bug=939200 , perhaps everyone who comes across this should add their findings there in the hopes of someday getting it fixed.


I was not able to fix the problem and migrated my server to Ubuntu. You can get it quite close to Debian if you remove the bloat, snap, netplan etc.

scelfo
Posts: 2
Joined: 2021-09-30 17:49

Re: Bullseye Kernel breaks my Intel ethernet

#12 Post by scelfo »

I fixed my problem... the fix is weird. I had ESXi configured to pass a USB device through to the virtual machine. It happens to be an APC UPS device. Deleting the USB device in the ESXi config fixed my problem. On boot I had noticed an error message about a USB device so I wonder if something about the USB device kept triggering errors in the kernel which caused the seemingly random network latency spikes.

cynwulf

Re: Bullseye Kernel breaks my Intel ethernet

#13 Post by cynwulf »

It's likely that the latest 5.14 stable kernel would have brought about a resolution to this, as you found that 5.11 in Ubuntu was ok.

This is why I personally dislike the trend of some distributions to use LTS kernels as standard, as they store up these kind of bugs.

If you're not averse to building your own kernel, you could give Debian another go and build 5.14.

Crowley
Posts: 7
Joined: 2021-08-17 04:59

Re: Bullseye Kernel breaks my Intel ethernet

#14 Post by Crowley »

I have no problems with Ubuntu so It will probably be a while Until I have much motivation to test Debian again. Setting it up was a bit of a pain though, a lot of stuff to remove to get it to a "Debian" like state.

I'm not against compiling a kernel, just pretty new to linux world and never done that before. Might give it a try sometime, or just wait until Debian ships with a newer kernel again and test with live usb.

cynwulf

Re: Bullseye Kernel breaks my Intel ethernet

#15 Post by cynwulf »

Fair enough. Eventually there will be a newer kernel in backports - and those are not difficult to install. But if 'buntu is doing the job then yes it's wise to stick with that for now.

Post Reply