Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

[Solved]Multiple Ip's, different networks, one interface

Linux Kernel, Network, and Services configuration.
Post Reply
Message
Author
gring
Posts: 6
Joined: 2013-03-28 03:50

[Solved]Multiple Ip's, different networks, one interface

#1 Post by gring »

Hi,

I've been looking for this for the last 8 hours, nobody seems to have an answer anywhere, and I'm starting to lose my sanity.
note: Network manager is not installed
Sorry if this has already been answered, I didn't find it.

Short story : What is the syntax to use in /etc/network/interfaces to bind several virtual IP's to eth0 that are not in the same network than eth0's main IP ?



Long story : I'm probably stupidly wrong, but it may also be a bug, hence the verbosity

I'm migrating an old web server on Ubuntu to a new server in a xen virtual machine running on Wheezy. (From OVH, a big hosting company, no physical access to the servers)

On the old server, I had several ip adresses FROM DIFFERENT NETWORKS on the same interface.
It worked with the following good old syntax in /etc/network/interfaces (I changed the ip adresses)

Code: Select all

auto eth0
iface eth0 inet static
        address 95.125.209.64
        netmask 255.255.255.0
        network 95.125.209.0
        broadcast 95.125.209.255
        gateway 95.125.209.254

auto eth0:0
iface eth0:0 inet static
...

auto eth0:1
iface eth0:1 inet static
        address 84.96.133.38
        netmask 255.255.255.0
        network 84.96.133.0
        broadcast 84.96.133.255
        gateway 84.96.133.254
Now, trying to use the same on Wheezy doesn't work. (by the way Network-manager is NOT installed.)

Code: Select all

root@xxx:/# /etc/init.d/networking stop && /etc/init.d/networking start
[....] Deconfiguring network interfaces...RTNETLINK answers: No such process
done.
[....] Configuring network interfaces...RTNETLINK answers: File exists
Failed to bring up eth0:1.
done.

root@xxx:/# cat /etc/network/run/ifstate
lo=lo
eth0=eth0

root@xxx:/# ifconfig -a
eth0      Link encap:Ethernet  HWaddr ...

eth0:1    Link encap:Ethernet  HWaddr 4c:72:b9:7b:b3:4d
          inet addr:84.96.133.38  Bcast:84.96.133.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:20 Memory:fe500000-fe520000

lo        Link encap:Local Loopback ...
Why does eth0:1 appear anyway ?
and when I ping eth0:1 adress (84.96.133.38), it works.

I read somewhere that ifconfig is to be replaced by ip.
So, I try with this /etc/network/interfaces :

Code: Select all

auto eth0
allow-hotplug eth0
iface eth0 inet static
        address 6.134.187.173
        netmask 255.255.255.0
        network 6.134.187.0
        broadcast 6.134.187.255
        gateway 6.134.187.254
        up   ip addr add 84.96.133.38/24 dev eth0 label eth0:1
        down ip addr del 84.96.133.38/24 dev eth0 label eth0:1

iface eth0 inet6 static ...

Code: Select all

root@xxx:~# /etc/init.d/networking stop && /etc/init.d/networking start
[....] Deconfiguring network interfaces...RTNETLINK answers: No such process
done.
[ ok ] Configuring network interfaces...done.
Hurray, it seems to work, but, wait, these ip's are on different networks, I need to specify their gateways.

I try to add this

Code: Select all

auto eth0
allow-hotplug eth0
iface eth0 inet static
        address 6.134.187.173
        netmask 255.255.255.0
        network 6.134.187.0
        broadcast 6.134.187.255
        gateway 6.134.187.254
        up   ip addr add 84.96.133.38/24 dev eth0 label eth0:1
        down ip addr del 84.96.133.38/24 dev eth0 label eth0:1
        up route add -net 87.98.133.0 netmask 255.255.255.0 gw 87.98.133.254 dev eth0:1
        down route del -net 87.98.133.0 netmask 255.255.255.0 gw 87.98.133.254 dev eth0:1

iface eth0 inet6 static ...

Code: Select all

root@xxx:~# /etc/init.d/networking stop && /etc/init.d/networking start
[....] Deconfiguring network interfaces...SIOCDELRT: No such device
done.
[....] Configuring network interfaces...RTNETLINK answers: File exists
Failed to bring up eth0.
RTNETLINK answers: File exists
Failed to bring up eth0.
done.

root@xxx:~# cat /etc/network/run/ifstate
lo=lo
Ouch. There's something that seems to keep eth0 up anyway, otherwise I would be locked out
Let's rollback

Code: Select all

auto eth0
allow-hotplug eth0
iface eth0 inet static
...
        #up   ip addr add 87.98.133.38/24 dev eth0 label eth0:1
        #down ip addr del 87.98.133.38/24 dev eth0 label eth0:1
        #up route add -net 87.98.133.0 netmask 255.255.255.0 gw 87.98.133.254 dev eth0:1
        #down route del -net 87.98.133.0 netmask 255.255.255.0 gw 87.98.133.254 dev eth0:1

Code: Select all

root@xxx:~# /etc/init.d/networking stop && /etc/init.d/networking start
[ ok ] Deconfiguring network interfaces...done.
[....] Configuring network interfaces...RTNETLINK answers: File exists
Failed to bring up eth0.
RTNETLINK answers: File exists
Failed to bring up eth0.
done.
Wut ? Now I can try to change anything (like putting back the down commands), I'll always have the same answer until reboot.

Is that a bug ? Does the network script read its configuration from elsewhere ?
Is there a cleaner way to do it ?
Last edited by gring on 2013-12-10 11:08, edited 2 times in total.

gring
Posts: 6
Joined: 2013-03-28 03:50

Re: Multiple Ip's from different networks on one eth interfa

#2 Post by gring »

Ok, I understand why the route command doesn't work ... It's used to add the gateway to reach a network, not to go out of that network.

Still, I have no clue how to do this, while it's quite basic and nearly mandatory on production servers.

Once we solve this, we should add an example to the wiki : http://wiki.debian.org/NetworkConfigura ... _Interface

gring
Posts: 6
Joined: 2013-03-28 03:50

Re: Multiple Ip's from different networks on one eth interfa

#3 Post by gring »

I progressed a bit, now my /etc/network/interfaces file is as follows :

Code: Select all

auto eth0
allow-hotplug eth0
iface eth0 inet static
        address x.x.x.x
        netmask 255.255.255.0
        network x.x.x.x
        broadcast x.x.x.x
        gateway x.x.x.x
        up ip addr add dev eth0 local 84.96.133.38/24 broadcast 84.96.133.255 label eth0:1
        up ip route add 84.96.133.38 dev eth0:1 scope link
        #up ip route add 84.96.133.0/24 proto kernel scope link dev eth0:1 src84.96.133.38
        #up ip route add 0.0.0.0/0 proto kernel scope link dev eth0:1 via 84.96.133.254 src 84.96.133.38
        down ip addr del dev eth0 local 84.96.133.38/24 broadcast 84.96.133.255 label eth0:1
        down ip route del 84.96.133.38 dev eth0:1 scope link
        #down ip route del 84.96.133.0/24 proto kernel scope link dev eth0:1 src 84.96.133.38
        #down ip route del 0.0.0.0/0 proto kernel scope link dev eth0:1 via 84.96.133.254 src 84.96.133.38

This seems to work, but if I try the commented route commands, I get the "RTNETLINK answers: File exists" problem again whilre restarting network, even if I rollback the changes.
Maybe it's a problem with the protocol, or the scope ? I don't find explicit documentation about that .

Any ideas ?

gring
Posts: 6
Joined: 2013-03-28 03:50

Re: Multiple Ip's from different networks on one eth interfa

#4 Post by gring »

Latest try :

Code: Select all

auto eth0
allow-hotplug eth0
iface eth0 inet static
        address x.x.x.x
        netmask 255.255.255.0
        network x.x.x.x
        broadcast x.x.x.x
        gateway x.x.x.x
        up ip addr add dev eth0 local 84.96.133.38/24 broadcast 84.96.133.255 label eth0:1
        up ip route add 84.96.133.38 proto static scope host dev eth0:1
        up ip route add 84.96.133.0/24 proto static scope host dev eth0:1 src 84.96.133.38
        up ip route add 0.0.0.0/0 proto static scope link dev eth0:1 via 84.96.133.254 src 84.96.133.38
        down ip addr del dev eth0 local 84.96.133.38/24 broadcast 84.96.133.255 label eth0:1
        down ip route del 84.96.133.38 scope link dev eth0:1
        down ip route del 84.96.133.0/24 proto kernel scope link dev eth0:1 src 84.96.133.38
        down ip route del 0.0.0.0/0 proto kernel scope link dev eth0:1 via 84.96.133.254 src 84.96.133.38
still not working, and I still have to reboot the entire machine each time I make a change.

gring
Posts: 6
Joined: 2013-03-28 03:50

Re: Multiple Ip's from different networks on one eth interfa

#5 Post by gring »

New progress :

Code: Select all

auto eth0
allow-hotplug eth0
iface eth0 inet static
        address x.x.x.x
        netmask 255.255.255.0
        network x.x.x.x
        broadcast x.x.x.x
        gateway x.x.x.x
        up ip addr add dev eth0 local 84.96.133.38/24 broadcast 84.96.133.255 label eth0:1
        up ip route add 84.96.133.38 proto static scope host dev eth0:1
        up ip route add default via 84.96.133.254 dev eth0:1 proto static table 101
        up ip rule add from 84.96.133.38 table 101
        down ip addr del dev eth0 local 84.96.133.38/24 broadcast 84.96.133.255 label eth0:1
        down ip route del 84.96.133.38 proto static scope host dev eth0:1
        down ip route del default via 84.96.133.254 dev eth0:1 proto static table 101
        down ip rule del from 84.96.133.38 table 101
Booting with this seems to work

Code: Select all

root@xxx:~# ip rule show
0:      from all lookup local
32765:  from 84.96.133.38 lookup 101
32766:  from all lookup main
32767:  from all lookup default

root@xxx:~# ip route show table 101
default via 84.96.133.254 dev eth0  proto static

root@xxx:~# ip route show
default via x.x.x.x dev eth0
x.x.x.0/24 dev eth0  proto kernel  scope link  src x.x.x.x
84.96.133.0/24 dev eth0  proto kernel  scope link  src 84.96.133.38
84.96.133.38 dev eth0  proto static  scope host

root@xxx:~# ifconfig -a
eth0      Link encap:Ethernet  HWaddr...

eth0:1    Link encap:Ethernet  HWaddr ...
          inet addr:84.968.133.38  Bcast:84.96.133.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:20 Memory:fe500000-fe520000

lo        Link encap:Local Loopback

This seems good

But then, when I try to route http port to one of my virtual machines :

iptables -t nat -A PREROUTING -p tcp -i eth0:1 --dport 80 -j DNAT --to 192.168.2.10
=> does not work, no server

iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 80 -j DNAT --to 192.168.2.10
=> works, so it means packets sent to 84.96.133.38 arrive, but not to the specific eth0:1 interface.

gring
Posts: 6
Joined: 2013-03-28 03:50

Re: Multiple Ip's from different networks on one eth interfa

#6 Post by gring »

Finally managed to make it work

Here's my /etc/network/interfaces file :

Code: Select all

auto eth0
iface eth0 inet static
        address x.x.x.x
        netmask 255.255.255.0
        network x.x.x.x
        broadcast x.x.x.x
        gateway x.x.x.x metric 1
        post-up ip addr add dev eth0 84.96.133.38/24
        post-up ip route add default via 84.96.133.254 dev eth0 proto static metric 2 src 84.96.133.38
        pre-down ip addr del dev eth0 84.96.133.38/24
        pre-down ip route del default via 84.96.133.254 dev eth0 proto static metric 2 src 84.96.133.38
Note that you need different metrics. (the gateway with the lower metric is used by outbound connections)

If like me you use this with xen virtual machines and NAT, you can use these iptable commands :

Code: Select all

iptables -t nat -A PREROUTING -p tcp -i eth0 -d <ip address> --dport 80 -j DNAT --to 192.168.2.10
iptables -t nat -I POSTROUTING 1 -s 192.168.2.10 -j SNAT --to-source <ip address>

sugarpuff
Posts: 1
Joined: 2014-02-07 04:29

Re: [Solved]Multiple Ip's, different networks, one interface

#7 Post by sugarpuff »

Ola Señior.

Como estas? Not too good huh?

This is how I felt earlier today (and last night):
I've been looking for this for the last 8 hours, nobody seems to have an answer anywhere, and I'm starting to lose my sanity.
As you can see I did lose it. However, I also stumbled upon your thread and solved my problem, but differently.

I too have multiple IP addresses. They are static, public IP addresses, both attached to the same VPS. I needed to redirect a port for the IP on eth0:0 to another port (on that same IP).

Turns out, you cannot do this with REDIRECT. You need to use DNAT. Thus, the solution turned out to be surprisingly simple, but as you also observed, nowhere to be found online. I had the wonderful folks on #netfilter to thank for helping me, especially Jefe who ended up figuring it out.

In the end I needed just two commands (per protocol):

Code: Select all

iptables -I INPUT -p tcp -d 23.226.227.93 --dport 4434 -j ACCEPT
iptables -t nat -A PREROUTING -p tcp -d 23.226.227.93 --dport 443 -j DNAT --to 23.226.227.93:4434
(Edit: you also need a rule to allow 443, obviously).

You can now benefit from the result! Free, encrypted, blockchain-compatible DNS!.

Hopefully that might help someone.

Post Reply