[Solved]Multiple Ip's, different networks, one interface

Kernels & Hardware, configuring network, installing services

[Solved]Multiple Ip's, different networks, one interface

Postby gring » 2013-03-28 04:43

Hi,

I've been looking for this for the last 8 hours, nobody seems to have an answer anywhere, and I'm starting to lose my sanity.
note: Network manager is not installed
Sorry if this has already been answered, I didn't find it.

Short story : What is the syntax to use in /etc/network/interfaces to bind several virtual IP's to eth0 that are not in the same network than eth0's main IP ?



Long story : I'm probably stupidly wrong, but it may also be a bug, hence the verbosity

I'm migrating an old web server on Ubuntu to a new server in a xen virtual machine running on Wheezy. (From OVH, a big hosting company, no physical access to the servers)

On the old server, I had several ip adresses FROM DIFFERENT NETWORKS on the same interface.
It worked with the following good old syntax in /etc/network/interfaces (I changed the ip adresses)

Code: Select all
auto eth0
iface eth0 inet static
        address 95.125.209.64
        netmask 255.255.255.0
        network 95.125.209.0
        broadcast 95.125.209.255
        gateway 95.125.209.254

auto eth0:0
iface eth0:0 inet static
...

auto eth0:1
iface eth0:1 inet static
        address 84.96.133.38
        netmask 255.255.255.0
        network 84.96.133.0
        broadcast 84.96.133.255
        gateway 84.96.133.254


Now, trying to use the same on Wheezy doesn't work. (by the way Network-manager is NOT installed.)

Code: Select all
root@xxx:/# /etc/init.d/networking stop && /etc/init.d/networking start
[....] Deconfiguring network interfaces...RTNETLINK answers: No such process
done.
[....] Configuring network interfaces...RTNETLINK answers: File exists
Failed to bring up eth0:1.
done.

root@xxx:/# cat /etc/network/run/ifstate
lo=lo
eth0=eth0

root@xxx:/# ifconfig -a
eth0      Link encap:Ethernet  HWaddr ...

eth0:1    Link encap:Ethernet  HWaddr 4c:72:b9:7b:b3:4d
          inet addr:84.96.133.38  Bcast:84.96.133.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:20 Memory:fe500000-fe520000

lo        Link encap:Local Loopback ...


Why does eth0:1 appear anyway ?
and when I ping eth0:1 adress (84.96.133.38), it works.

I read somewhere that ifconfig is to be replaced by ip.
So, I try with this /etc/network/interfaces :

Code: Select all
auto eth0
allow-hotplug eth0
iface eth0 inet static
        address 6.134.187.173
        netmask 255.255.255.0
        network 6.134.187.0
        broadcast 6.134.187.255
        gateway 6.134.187.254
        up   ip addr add 84.96.133.38/24 dev eth0 label eth0:1
        down ip addr del 84.96.133.38/24 dev eth0 label eth0:1

iface eth0 inet6 static ...


Code: Select all
root@xxx:~# /etc/init.d/networking stop && /etc/init.d/networking start
[....] Deconfiguring network interfaces...RTNETLINK answers: No such process
done.
[ ok ] Configuring network interfaces...done.


Hurray, it seems to work, but, wait, these ip's are on different networks, I need to specify their gateways.

I try to add this

Code: Select all
auto eth0
allow-hotplug eth0
iface eth0 inet static
        address 6.134.187.173
        netmask 255.255.255.0
        network 6.134.187.0
        broadcast 6.134.187.255
        gateway 6.134.187.254
        up   ip addr add 84.96.133.38/24 dev eth0 label eth0:1
        down ip addr del 84.96.133.38/24 dev eth0 label eth0:1
        up route add -net 87.98.133.0 netmask 255.255.255.0 gw 87.98.133.254 dev eth0:1
        down route del -net 87.98.133.0 netmask 255.255.255.0 gw 87.98.133.254 dev eth0:1

iface eth0 inet6 static ...


Code: Select all
root@xxx:~# /etc/init.d/networking stop && /etc/init.d/networking start
[....] Deconfiguring network interfaces...SIOCDELRT: No such device
done.
[....] Configuring network interfaces...RTNETLINK answers: File exists
Failed to bring up eth0.
RTNETLINK answers: File exists
Failed to bring up eth0.
done.

root@xxx:~# cat /etc/network/run/ifstate
lo=lo


Ouch. There's something that seems to keep eth0 up anyway, otherwise I would be locked out
Let's rollback

Code: Select all
auto eth0
allow-hotplug eth0
iface eth0 inet static
...
        #up   ip addr add 87.98.133.38/24 dev eth0 label eth0:1
        #down ip addr del 87.98.133.38/24 dev eth0 label eth0:1
        #up route add -net 87.98.133.0 netmask 255.255.255.0 gw 87.98.133.254 dev eth0:1
        #down route del -net 87.98.133.0 netmask 255.255.255.0 gw 87.98.133.254 dev eth0:1


Code: Select all
root@xxx:~# /etc/init.d/networking stop && /etc/init.d/networking start
[ ok ] Deconfiguring network interfaces...done.
[....] Configuring network interfaces...RTNETLINK answers: File exists
Failed to bring up eth0.
RTNETLINK answers: File exists
Failed to bring up eth0.
done.


Wut ? Now I can try to change anything (like putting back the down commands), I'll always have the same answer until reboot.

Is that a bug ? Does the network script read its configuration from elsewhere ?
Is there a cleaner way to do it ?
Last edited by gring on 2013-12-10 11:08, edited 2 times in total.
gring
 
Posts: 6
Joined: 2013-03-28 03:50

Re: Multiple Ip's from different networks on one eth interfa

Postby gring » 2013-03-28 15:20

Ok, I understand why the route command doesn't work ... It's used to add the gateway to reach a network, not to go out of that network.

Still, I have no clue how to do this, while it's quite basic and nearly mandatory on production servers.

Once we solve this, we should add an example to the wiki : http://wiki.debian.org/NetworkConfigura ... _Interface
gring
 
Posts: 6
Joined: 2013-03-28 03:50

Re: Multiple Ip's from different networks on one eth interfa

Postby gring » 2013-03-28 16:19

I progressed a bit, now my /etc/network/interfaces file is as follows :

Code: Select all
auto eth0
allow-hotplug eth0
iface eth0 inet static
        address x.x.x.x
        netmask 255.255.255.0
        network x.x.x.x
        broadcast x.x.x.x
        gateway x.x.x.x
        up ip addr add dev eth0 local 84.96.133.38/24 broadcast 84.96.133.255 label eth0:1
        up ip route add 84.96.133.38 dev eth0:1 scope link
        #up ip route add 84.96.133.0/24 proto kernel scope link dev eth0:1 src84.96.133.38
        #up ip route add 0.0.0.0/0 proto kernel scope link dev eth0:1 via 84.96.133.254 src 84.96.133.38
        down ip addr del dev eth0 local 84.96.133.38/24 broadcast 84.96.133.255 label eth0:1
        down ip route del 84.96.133.38 dev eth0:1 scope link
        #down ip route del 84.96.133.0/24 proto kernel scope link dev eth0:1 src 84.96.133.38
        #down ip route del 0.0.0.0/0 proto kernel scope link dev eth0:1 via 84.96.133.254 src 84.96.133.38



This seems to work, but if I try the commented route commands, I get the "RTNETLINK answers: File exists" problem again whilre restarting network, even if I rollback the changes.
Maybe it's a problem with the protocol, or the scope ? I don't find explicit documentation about that .

Any ideas ?
gring
 
Posts: 6
Joined: 2013-03-28 03:50

Re: Multiple Ip's from different networks on one eth interfa

Postby gring » 2013-03-28 16:40

Latest try :

Code: Select all
auto eth0
allow-hotplug eth0
iface eth0 inet static
        address x.x.x.x
        netmask 255.255.255.0
        network x.x.x.x
        broadcast x.x.x.x
        gateway x.x.x.x
        up ip addr add dev eth0 local 84.96.133.38/24 broadcast 84.96.133.255 label eth0:1
        up ip route add 84.96.133.38 proto static scope host dev eth0:1
        up ip route add 84.96.133.0/24 proto static scope host dev eth0:1 src 84.96.133.38
        up ip route add 0.0.0.0/0 proto static scope link dev eth0:1 via 84.96.133.254 src 84.96.133.38
        down ip addr del dev eth0 local 84.96.133.38/24 broadcast 84.96.133.255 label eth0:1
        down ip route del 84.96.133.38 scope link dev eth0:1
        down ip route del 84.96.133.0/24 proto kernel scope link dev eth0:1 src 84.96.133.38
        down ip route del 0.0.0.0/0 proto kernel scope link dev eth0:1 via 84.96.133.254 src 84.96.133.38


still not working, and I still have to reboot the entire machine each time I make a change.
gring
 
Posts: 6
Joined: 2013-03-28 03:50

Re: Multiple Ip's from different networks on one eth interfa

Postby gring » 2013-03-28 21:02

New progress :
Code: Select all
auto eth0
allow-hotplug eth0
iface eth0 inet static
        address x.x.x.x
        netmask 255.255.255.0
        network x.x.x.x
        broadcast x.x.x.x
        gateway x.x.x.x
        up ip addr add dev eth0 local 84.96.133.38/24 broadcast 84.96.133.255 label eth0:1
        up ip route add 84.96.133.38 proto static scope host dev eth0:1
        up ip route add default via 84.96.133.254 dev eth0:1 proto static table 101
        up ip rule add from 84.96.133.38 table 101
        down ip addr del dev eth0 local 84.96.133.38/24 broadcast 84.96.133.255 label eth0:1
        down ip route del 84.96.133.38 proto static scope host dev eth0:1
        down ip route del default via 84.96.133.254 dev eth0:1 proto static table 101
        down ip rule del from 84.96.133.38 table 101


Booting with this seems to work

Code: Select all
root@xxx:~# ip rule show
0:      from all lookup local
32765:  from 84.96.133.38 lookup 101
32766:  from all lookup main
32767:  from all lookup default

root@xxx:~# ip route show table 101
default via 84.96.133.254 dev eth0  proto static

root@xxx:~# ip route show
default via x.x.x.x dev eth0
x.x.x.0/24 dev eth0  proto kernel  scope link  src x.x.x.x
84.96.133.0/24 dev eth0  proto kernel  scope link  src 84.96.133.38
84.96.133.38 dev eth0  proto static  scope host

root@xxx:~# ifconfig -a
eth0      Link encap:Ethernet  HWaddr...

eth0:1    Link encap:Ethernet  HWaddr ...
          inet addr:84.968.133.38  Bcast:84.96.133.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:20 Memory:fe500000-fe520000

lo        Link encap:Local Loopback



This seems good

But then, when I try to route http port to one of my virtual machines :

iptables -t nat -A PREROUTING -p tcp -i eth0:1 --dport 80 -j DNAT --to 192.168.2.10
=> does not work, no server

iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 80 -j DNAT --to 192.168.2.10
=> works, so it means packets sent to 84.96.133.38 arrive, but not to the specific eth0:1 interface.
gring
 
Posts: 6
Joined: 2013-03-28 03:50

Re: Multiple Ip's from different networks on one eth interfa

Postby gring » 2013-03-29 05:25

Finally managed to make it work

Here's my /etc/network/interfaces file :
Code: Select all
auto eth0
iface eth0 inet static
        address x.x.x.x
        netmask 255.255.255.0
        network x.x.x.x
        broadcast x.x.x.x
        gateway x.x.x.x metric 1
        post-up ip addr add dev eth0 84.96.133.38/24
        post-up ip route add default via 84.96.133.254 dev eth0 proto static metric 2 src 84.96.133.38
        pre-down ip addr del dev eth0 84.96.133.38/24
        pre-down ip route del default via 84.96.133.254 dev eth0 proto static metric 2 src 84.96.133.38


Note that you need different metrics. (the gateway with the lower metric is used by outbound connections)

If like me you use this with xen virtual machines and NAT, you can use these iptable commands :
Code: Select all
iptables -t nat -A PREROUTING -p tcp -i eth0 -d <ip address> --dport 80 -j DNAT --to 192.168.2.10
iptables -t nat -I POSTROUTING 1 -s 192.168.2.10 -j SNAT --to-source <ip address>
gring
 
Posts: 6
Joined: 2013-03-28 03:50

Re: [Solved]Multiple Ip's, different networks, one interface

Postby sugarpuff » 2014-02-07 04:34

Ola Señior.

Como estas? Not too good huh?

This is how I felt earlier today (and last night):

I've been looking for this for the last 8 hours, nobody seems to have an answer anywhere, and I'm starting to lose my sanity.


As you can see I did lose it. However, I also stumbled upon your thread and solved my problem, but differently.

I too have multiple IP addresses. They are static, public IP addresses, both attached to the same VPS. I needed to redirect a port for the IP on eth0:0 to another port (on that same IP).

Turns out, you cannot do this with REDIRECT. You need to use DNAT. Thus, the solution turned out to be surprisingly simple, but as you also observed, nowhere to be found online. I had the wonderful folks on #netfilter to thank for helping me, especially Jefe who ended up figuring it out.

In the end I needed just two commands (per protocol):

Code: Select all
iptables -I INPUT -p tcp -d 23.226.227.93 --dport 4434 -j ACCEPT
iptables -t nat -A PREROUTING -p tcp -d 23.226.227.93 --dport 443 -j DNAT --to 23.226.227.93:4434


(Edit: you also need a rule to allow 443, obviously).

You can now benefit from the result! Free, encrypted, blockchain-compatible DNS!.

Hopefully that might help someone.
sugarpuff
 
Posts: 1
Joined: 2014-02-07 04:29


Return to System configuration

Who is online

Users browsing this forum: bogaardesquat and 18 guests

fashionable