Configuring 2 NICs for Public & Private Networks

Portal Home > Knowledgebase > Articles Database > Configuring 2 NICs for Public & Private Networks

Posted by falconinternet, 08-24-2011, 12:48 PM
I am setting up a few new servers and switches and I'm not sure what I am doing wrong. We have a really busy web-site and we've bought very fast new servers. One is a dedicated web server and one is a dedicated MySQL server. Each server has 2 NICs in it. I want both server to use ETH0 for the public internet and ETH1 for the private. For example: Server 1 (web server): ETH0 - 97.xxx.xxx.8 ETH1 - 10.10.10.80 Server 2 (mysql server): ETH0 - 97.xxx.xxx.9 ETH1 - 10.10.10.90 Both Servers, ETH1 ports are connected to dedicated switch that isn't connected anywhere else, where ETH0 on both servers are connected up to the public internet. When I put in the private IPs in ETH1, the servers seem to have issues communicating with public internet functions going out. Like they don't know which nic to use to get out to the real world. Any ideas? Please let me know if I am not specific enough and I will try to elaborate. I am only needing help with routing, I don't need to know how to limit traffic over various NICs. Thanks! EDIT: These servers have 10GB Ports for ETH1 and I am using a 10GB Switch for the private network so they communicate very fast internally. The switch will be isolated and not connected to the internet. Last edited by falconinternet; 08-24-2011 at 12:52 PM.
Posted by relichost, 08-24-2011, 01:07 PM
Hi check your routing table. Also do the eth1 config contain gateways? and did you configure the eth ports for eth1 first ? Thanks Andrew
Posted by Posilan, 08-24-2011, 01:09 PM
Have you set default gateways and routes on each? What OS/distro are you using? Steve
Posted by falconinternet, 08-24-2011, 01:36 PM
the eth0 has the standard gateways, I don't have a gateway for eth1, do I need one? Its not going anywhere. I am not sure how to set the routes, I am using CentOS 5.6, 64-bit.
Posted by Posilan, 08-24-2011, 01:40 PM
Can you print the output of: (you can hide the exact IP's if you want, as long as local and public IP's are detectable) Steve
Posted by falconinternet, 08-24-2011, 03:00 PM
Incoming traffic seems to be ok, but when I try to connect to something on the outside, it tries to go out the 10.10.10.0/24 network.
Posted by lynxus, 08-24-2011, 03:05 PM
The routing table begs to differ. Drop the firewalls and test again?
Posted by Posilan, 08-24-2011, 03:08 PM
Agreed - routing table seems correct so something else is causing this. Can you ping the gateway IP(97.xxx.xxx.1)? Steve
Posted by falconinternet, 08-24-2011, 03:23 PM
without any changes, now it's working. Previously, I could not do a command such as "host www.google.com", now it works. And it was not a DNS server issue.. it couldn't connect to anything, even remote IPs. Is there some way to guarantee it will stay the way it is?
Posted by lynxus, 08-24-2011, 03:29 PM
May of been some kind of arp timeout waiting to expire.
Posted by falconinternet, 08-24-2011, 03:37 PM
I'm not getting 10Gb/sec speed on ETH1 now, I copied a large file over the 1Gb network and I got around 37.5MB/sec and the exact same speed over the 10Gb network. ETH1 on both servers are wired to a 24 port 10Gb Brocade switch. Is the gateway slowing it down? Do I need a gateway for the 10Gb network? I've never really dealt with these private type networks before.
Posted by lynxus, 08-24-2011, 03:42 PM
The gateway is only used when connecting to something not in the routing table. So if your sending to something on the same subnet then it shouldn't be touched ( and only packet switched ) Now, With 10gig. ive had many problems with this before. Ive found that actually getting 10gig over one TCP stream is nearly impossible. Maybe try something like iperf to help test the link with multiple streams. Id also ensure things like Jumbo packets have been enabled on the nics and the brocade switch ( Ie: packets with 9000byte mtu's rather than 1500 )
Posted by falconinternet, 08-24-2011, 03:53 PM
thanks for your help, i'll check into the MTU and Jumbo Frames. i just found it suspicious that the speed was identical on both NICs... (and the speed of the drives isn't an issue, they are 10000RPM 6Gbit/sec in a RAID 10 configuration. Duplicating the file on the same server takes less than a second. Maybe it's the overhead of SCP during the copy? I just want to make sure when apache/php is talking to MySQL, it's going the fastest possible.
Posted by lynxus, 08-24-2011, 03:56 PM
The ifconfig's show MTU's of 1500 ( thats fine for the internet facing link.. ) but the internal ones could do with 9000 ( if the switch allowed it ) id also suggest looking at what an "miitool eth1" displays. That might give you some info.

Add to Favourites Print this Article