Primary network interface in Linux

Question

I have a server box with two interfaces. One of the interfaces is connected to the internal network, another - to external.

I have assigned IP 192.168.1.200 to internal interface (eth0). I have some local daemons listening on this IP/interface (cups, nginx, pdns).

External IP is 192.168.0.91 (eth1). Here I have only NAT masquerading. Strange things happens whenever I'm trying to access server box from itself.

If I'm trying to open webpage, located on 192.168.1.200, server uses 192.168.0.91 address and I'm getting permission denied (webserver protected to serve content on internal network only).

If I'm bring down eth1, server work OK and uses internal IP. But, as soon, as I will bring up eth1 it will immediately pick up it's IP as primary one and I will get permission denied again.

How can I explicitly set main IP/interface to use?

I'm running 64-bit version of Gentoo Linux. Both nic's drivers are compiled as modules. I'm using systemd as my init system.

EDIT:

Thanks for responses, but most interesting part is coming here:

atomic ~ # cat /etc/resolv.conf
domain local
search local
nameserver 192.168.1.200

atomic ~ # cat /etc/hosts
...
192.168.1.200   atomic ns.atomic.local atomic.local
...

Address resolving just OK:

atomic ~ # dig atomic.local 192.168.1.200

; <<>> DiG 9.9.4 <<>> atomic.local
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 38797
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 2800
;; QUESTION SECTION:
;atomic.local.          IN  A

;; ANSWER SECTION:
atomic.local.       604800  IN  A   192.168.1.200

;; Query time: 42 msec
;; SERVER: 192.168.1.200#53(192.168.1.200)
;; WHEN: Tue May 27 13:37:04 EEST 2014
;; MSG SIZE  rcvd: 55

And Nginx logs of accessing via wget:

atomic ~ # wget atomic.fhn
--2014-05-27 13:45:58--  http://atomic.local/
Resolving atomic.local... 192.168.1.200
Connecting to atomic.local|192.168.1.200|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘index.html’

[ <=>                                              ] 0  --.-K/s   in 0s      

2014-05-27 13:45:58 (0.00 B/s) - ‘index.html’ saved [0]

atomic ~ # tail -n 1 /var/log/nginx/access_log
192.168.0.91 - - [27/May/2014:13:45:58 +0300] "GET / HTTP/1.1" 200 5 "-" "Wget/1.14 (linux-gnu)"

I got 200/OK HTTP status, because I have disabled IP/network filtering while problem is not solved.

Routing table:

192.168.0.0/30 dev wan  proto kernel  scope link  src 192.168.0.2 
192.168.1.0/24 dev lan  proto kernel  scope link  src 192.168.1.200 

EDIT2:

Routing table with metric used:

192.168.0.0/30 dev wan  scope link  metric 20 
192.168.1.0/24 dev lan  scope link  metric 10 
broadcast 127.0.0.0 dev lo  table local  proto kernel  scope link  src 127.0.0.1 
local 127.0.0.0/8 dev lo  table local  proto kernel  scope host  src 127.0.0.1 
local 127.0.0.1 dev lo  table local  proto kernel  scope host  src 127.0.0.1 
broadcast 127.255.255.255 dev lo  table local  proto kernel  scope link  src 127.0.0.1 
broadcast 192.168.0.0 dev wan  table local  proto kernel  scope link  src 192.168.0.2 
local 192.168.0.2 dev wan  table local  proto kernel  scope host  src 192.168.0.2 
broadcast 192.168.0.3 dev wan  table local  proto kernel  scope link  src 192.168.0.2 
broadcast 192.168.1.0 dev lan  table local  proto kernel  scope link  src 192.168.1.200 
local 192.168.1.200 dev lan  table local  proto kernel  scope host  src 192.168.1.200 
broadcast 192.168.1.255 dev lan  table local  proto kernel  scope link  src 192.168.1.200 

Solution 1

What you need is to make sure the LAN route takes precedence over the WAN one. The trouble is here:

192.168.0.0/30 dev wan  proto kernel  scope link  src 192.168.0.2 
192.168.1.0/24 dev lan  proto kernel  scope link  src 192.168.1.200

192.168.0.0/30 covers 192.168.1.0/24 as well. Hence you have to tell the TCP/IP stack to prefer the latter when applicable otherwise it might choose randomly (I would expect most implementations to always use the first one) - use the metric argument, e.g.:

ip route add 192.168.1.0/24 dev lan metric 10
ip route add 192.168.0.0/30 dev wan metric 20

(you'll obviously need to remove the extant routes first).

Solution 2

As jofel already said, there is no such thing as primary interface on linux. What actually happens: if your computer wants to send a packet somewhere, he knows its target ip. But a source ip is also needed to be given in a package, and this is which will be filled on from the interface list and from the routing table.

In the case of a local address, things will be a bit complexer (and not simpler as we wished).

The simplest way to use the explicit 192.168.1.200 address in your url (or a hostname which resolves exactly to this address). You didn't write, but I think you are using an url with a hostname to get the actual document, and this hostname resolves - by default - to the external address.

Comments

[YoloTats.com]

There is no primary interface in Linux. You want to set the default route correctly.

[GeekMagus]

I'm understand that. But unfortunately all of my trials to achieve desired result with routes led me only to errors, so maybe you have some hint? Thanks.

[slm]

This setup seems like a mess to me. What IP are you expecting the servers to reply on. Disregard the hostnames for a moment since they have nothing to do with what IPs the daemons will reply on.

[GeekMagus]

I would like to force it using 192.168.1.200 by default. I have added current routing table.

[Sam Liao]

First you need narrow down it's your web server reject connection or your firewall/routing; second, usually sever has setting for which interface listen on.

[peterph]

Well, you can try ip route get IP_ADDRESS to see through which interface the packets will be routed. You can also monitor the traffic with iptraf or wireshark.