Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Configured Multiple NICS all traffic goes through only 1

Linux Kernel, Network, and Services configuration.
Post Reply
Message
Author
mark021689
Posts: 2
Joined: 2018-02-15 15:35

Configured Multiple NICS all traffic goes through only 1

#1 Post by mark021689 »

I have a system with 4 embedded NICS (HP Proliant DL360 G8)
I would like to have all 4 nics available for use.
I have configured all 4 nics in the interfaces file. I am able to ping all nics and everything *seems* to be working properly.
However ALL traffic for the server is only passing though the first NIC. I am verifying this by reviewing the ARP table on the switch. I ping all 4 nics from the switch and then review the results. In this case all IP addresses assigned to the 4 nics are only replying from the 1st nic.
I have specified a mac address for Nic 2 and tested. No change in behavior.
I do not have physical access to the server. It is in Utah, I am in California.

Any ideas or help would be greatly appreciated.

############################## My Hardware ###############################################
bond0: flags=5122<BROADCAST,MASTER,MULTICAST> mtu 1500
ether 7a:70:c7:fc:d3:19 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.98.50.36 netmask 255.255.255.0 broadcast 10.0.0.255
inet6 fe80::da9d:67ff:fe79:3ca4 prefixlen 64 scopeid 0x20<link>
ether d8:9d:67:79:3c:a4 txqueuelen 1000 (Ethernet)
RX packets 5575821 bytes 7970479635 (7.4 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2830867 bytes 155372064 (148.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xf7d00000-f7dfffff

eno2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.98.50.37 netmask 255.255.255.0 broadcast 10.0.0.255
inet6 fe80::da9d:67ff:fe79:3ca5 prefixlen 64 scopeid 0x20<link>
ether d8:9d:67:79:3c:a5 txqueuelen 1000 (Ethernet)
RX packets 26 bytes 8397 (8.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 203 bytes 9262 (9.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xf7b00000-f7bfffff

eno3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.98.50.38 netmask 255.255.255.0 broadcast 10.98.50.255
inet6 fe80::da9d:67ff:fe79:3ca6 prefixlen 64 scopeid 0x20<link>
ether d8:9d:67:79:3c:a6 txqueuelen 1000 (Ethernet)
RX packets 17223 bytes 5030634 (4.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 49 bytes 3518 (3.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xf7900000-f79fffff

eno4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.98.50.39 netmask 255.255.255.0 broadcast 10.98.50.255
inet6 fe80::da9d:67ff:fe79:3ca7 prefixlen 64 scopeid 0x20<link>
ether d8:9d:67:79:3c:a7 txqueuelen 1000 (Ethernet)
RX packets 17223 bytes 5030634 (4.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 48 bytes 3448 (3.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xf7700000-f77fffff

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 148601 bytes 11126679 (10.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 148601 bytes 11126679 (10.6 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0


################### My Interfaces File #######################################
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug eno1
iface eno1 inet static
address 10.98.50.36
netmask 255.255.255.0
network 10.98.50.0
broadcast 10.0.0.255
gateway 10.98.50.2
dns-nameservers 74.82.42.42 8.8.8.8
dns-domain sub.domain.com
dns-search sub.domain.com

# The secondary network interface
allow-hotplug eno2
iface eno2 inet static
address 10.98.50.37
netmask 255.255.255.0
network 10.98.50.0
broadcast 10.0.0.255
dns-nameservers 74.82.42.42 8.8.8.8
dns-domain sub.domain.com
dns-search sub.domain.com
hwaddress ether d8:9d:67:79:3c:a5


# The tertiary network interface
allow-hotplug eno3
iface eno3 inet static
address 10.98.50.38
netmask 255.255.255.0
network 10.98.50.0
broadcast 10.98.50.255
dns-nameservers 74.82.42.42 8.8.8.8
dns-domain sub.domain.com
dns-search sub.domain.com

# The quaternary network interface
allow-hotplug eno4
iface eno4 inet static
address 10.98.50.39
netmask 255.255.255.0
network 10.98.50.0
broadcast 10.98.50.255
dns-nameservers 74.82.42.42 8.8.8.8
dns-domain sub.domain.com
dns-search sub.domain.com


######################## Arp Table from Switch ###############################

IP ARP table

IP Address MAC Address Type Port
--------------- ----------------- ------- ----
10.98.50.36 d89d67-793ca4 dynamic 33
10.98.50.37 d89d67-793ca4 dynamic 33
10.98.50.38 d89d67-793ca4 dynamic 33
10.98.50.39 d89d67-793ca4 dynamic 33

User avatar
acewiza
Posts: 357
Joined: 2013-05-28 12:38
Location: Out West

Re: Configured Multiple NICS all traffic goes through only 1

#2 Post by acewiza »

How does a packet know where to go when they are all on the same network?
Nobody would ever ask questions If everyone possessed encyclopedic knowledge of the man pages.

mark021689
Posts: 2
Joined: 2018-02-15 15:35

Re: Configured Multiple NICS all traffic goes through only 1

#3 Post by mark021689 »

The goal is to have certain systems that connect to the server use different nics.

So system a will connect to nic1, system b will connect to nic 2 system c will connect to nic 3 etc.

Remedial load balancing.

Dai_trying
Posts: 1100
Joined: 2016-01-07 12:25
Has thanked: 5 times
Been thanked: 16 times

Re: Configured Multiple NICS all traffic goes through only 1

#4 Post by Dai_trying »

I'm not an expert (or even very experienced) with this sort of networking, but one thing I noticed was that device eno1 (and no other) has a gateway specified and would therefore be used for access to the other nic's wouldn't it??? like I say I'm not an expert and this is a bit of a guess really, but I would imagine you would need to specify all ports as gateways for them to be accessed directly.

User avatar
acewiza
Posts: 357
Joined: 2013-05-28 12:38
Location: Out West

Re: Configured Multiple NICS all traffic goes through only 1

#5 Post by acewiza »

mark021689 wrote:The goal is to have certain systems that connect to the server use different nics.
Then they need to be on different networks, unless using some sort of bonding facility to aggregate bandwidth.
Nobody would ever ask questions If everyone possessed encyclopedic knowledge of the man pages.

Post Reply