issue with nat and multiple wan interfaces _ vyatta

22/4/2014 Issue with NAT and multiple WAN interfaces | Vyatta.org Community

http://www.vyatta.org/node/4675 1/17

11 posts / 0 newLog in or register to post comments Last post

Fri, 12/17/2010 -06:52

#1

IssuewithNATandmultipleWANinterfaces

Singularity

Issue with NAT and multiple WAN interfaces

Hello,

I am having some difficulty implementing a simple NAT setup withmultiple WAN interfaces.

The setup consists of desktop PC's on a 10.1.100.0/16 network andVoIP servers on a 10.1.20.0/16 network. These access the internetthrough a Vyatta installation with a single LAN interface eth0(10.1.30.1/16).

The Vyatta server is configured with one LAN interface and 4x ADSLPPPOE interfaces (pppoe0-pppoe3). Currently all traffic is NAT'edthrough pppoe3, which is set up as Vyatta's default route.

What I need to do is:

1) Use Source NAT to selectively route outbound traffic from theLAN through different WAN interfaces depending on the IP addressof the internal PC. This will allow me to send the (10.1.20.0/16) VoIPtraffic out a different WAN interface to general internet(10.1.100.0/16) traffic.

2) Perform Destination NAT on incoming WAN packets based oninterface and destination port. This will allow me to send incomingVoIP packets from any WAN interface to a VoIP server on the LAN.

At the moment I'm running into problems:

Vyatta.orgCommunityVyatta.com(http://vyatta.com/)

SupportCenter(http://www2.vyatta.com/support) |

Search

GETTINGSTARTED DOWNLOADS DOCUMENTATION CONTRIBUTE

SUPPORT&SERVICES COMMUNITY FORUMS



1) When using Source NAT, any rule that does not use the currentdefault route/device seems to be ignored. For some reason Vyatta isjust skipping over the rule. I have no idea why this is happening, butif I change the outbound interface and outgoing address to that ofdefault route (pppoe3), the rule works. For example, rule 5 in myconfiguration never shows any hits in the statistics when pppoe3 is setas the default route, all traffic hits rule 10 instead, which is using thedefault route interface.

2) When using Destination NAT to redirect traffic from a WANinterface which is not the default route, I end up with anasynchronous routing scenario.For example, rule 1070 in my configuration collects data fromexternal clients coming in on pppoe0, port 4569 (IAX2). It then sendsit to internal address 10.1.20.5 (VoIP Server) - This works correctly,and data can be seen arriving at the server. However, when 10.1.20.5replies, data can be seen leaving the VoIP server and arriving atVyatta. It is then for some reason sent out the default route interface(pppoe3) instead of the originating WAN interface (pppoe0).

Any ideas why this is happening? Isn't Vyatta supposed to maintainsession state and send things back out the interface from which theycame? When I check the packets leaving pppoe3, they actually havethe correct source address of pppoe0, so its like the NAT side ofthings is working, its just getting screwed up during routing.

Here is my sanitized current configuration:

interfaces{etherneteth0{address10.1.30.1/16duplexautohwid00:0c:29:64:a1:43smp_affinityautospeedauto}etherneteth1{duplexautohwid00:0c:29:64:a1:6bpppoe0{defaultroutenonemtu1492nameserverautopasswordpassworduseridadsl_voip}smp_affinityautospeedauto}etherneteth2{duplexautohwid00:0c:29:64:a1:4dpppoe1{defaultroutenone



mtu1492nameserverautopasswordpassworduseridadsl_inet2}smp_affinityautospeedauto}etherneteth3{duplexautohwid00:0c:29:64:a1:57pppoe2{defaultroutenonemtu1492nameserverautopasswordpassworduseridadsl_vpn}smp_affinityautospeedauto}etherneteth4{duplexautohwid00:0c:29:64:a1:61pppoe3{defaultrouteautomtu1492nameserverautopasswordpasswordtrafficpolicy{outVoIPPriority}useridadsl_inet}smp_affinityautospeedauto}loopbacklo{}}protocols{static{interfaceroute58.1.1.1/32{nexthopinterfacepppoe2{}}interfaceroute58.2.2.2/32{nexthopinterfacepppoe2{}}interfaceroute124.1.1.1/32{nexthopinterfacepppoe2{}}interfaceroute202.2.2.2/32{nexthopinterfacepppoe2{}}}}service{nat{rule5{description"NATforVoIPServers"outboundinterfacepppoe0



outsideaddress{address58.3.3.3}source{address10.1.0.0/16}typesource}rule10{description"GeneralOutboundNAT"outboundinterfacepppoe3outsideaddress{address150.1.1.1}source{address10.1.0.0/16}typesource}rule15{description"NATforVoIPServers"outboundinterfacepppoe0outsideaddress{address58.3.3.3}source{address10.1.20.0/24}typesource}rule1000{description"IncommingIMAP"destination{address150.1.1.1port143}inboundinterfacepppoe3insideaddress{address10.1.10.6}protocoltcpsource{address0.0.0.0/0}typedestination}rule1010{destination{address202.1.1.1port3000}inboundinterfacepppoe2insideaddress{address10.1.11.5port3389}protocoludptypedestination}rule1020{descriptionRulesforVoIPdestination{port50605090}inboundinterfacepppoe3



insideaddress{address10.1.20.1}protocoltcptypedestination}rule1030{destination{port50605090}inboundinterfacepppoe3insideaddress{address10.1.20.1}protocoludptypedestination}rule1040{destination{port3000031000}inboundinterfacepppoe3insideaddress{address10.1.20.1}protocoludptypedestination}rule1050{destination{port2021}inboundinterfacepppoe3insideaddress{address10.1.20.1}protocoltcptypedestination}rule1060{destination{address150.1.1.1port85}inboundinterfacepppoe3insideaddress{address10.1.115.188}protocoltcpsource{address0.0.0.0/0}typedestination}rule1070{destination{address58.3.3.3port4569}inboundinterfacepppoe0insideaddress{address10.1.20.5}protocoludpsource{



address0.0.0.0/0}typedestination}}ssh{port22protocolversion2}}system{hostnamevyatta.server.comlogin{banner{prelogin"VyattaServer"}uservyatta{authentication{encryptedpasswordXXXXXXXXXXXXXXXXXXXXXXplaintextpassword""}leveladmin}}ntpserver0.vyatta.pool.ntp.orgpackage{autosync1repositorycommunity{componentsmaindistributionstablepassword""urlhttp://packages.vyatta.com/vyatta(http://packages.vyatta.com/vyatta)username""}}syslog{global{facilityall{levelnotice}facilityprotocols{leveldebug}}}timezoneAustralia/Sydney}trafficpolicy{shaperVoIPPriority{bandwidth650kbitclass10{bandwidth60%burst15kceiling100%description"VoIPRTPtraffic"matchVOIPRTP{ip{dscp46}}queuetypefairqueue}class20{bandwidth5%



burst15kceiling100%description"VoIPSIPTraffic"matchVOIPSIP{ip{dscp26}}queuetypefairqueue}default{bandwidth15%burst15kceiling100%queuetypefairqueue}description"QoSpolicytodecreaseVoIPlatency"}}vpn{ipsec{espgroupp2{compressiondisablelifetime1800modetunnelpfsenableproposal1{encryptionaes128hashsha1}}ikegroupp1{lifetime7200proposal1{dhgroup2encryptionaes128hashsha1}}ipsecinterfaces{interfacepppoe2}logging{logmodesall}sitetosite{peer58.1.1.1{authentication{modepresharedsecretpresharedsecretREALPSKGOESHERE}ikegroupp1localip202.1.1.1tunnel1{allownatnetworksdisableallowpublicnetworksdisableespgroupp2localsubnet10.1.0.0/16remotesubnet10.4.0.0/16}}peer58.2.2.2{authentication{modepresharedsecretpresharedsecretREALPSKGOESHERE



}ikegroupp1localip202.1.1.1tunnel1{allownatnetworksdisableallowpublicnetworksdisableespgroupp2localsubnet10.1.0.0/16remotesubnet10.2.0.0/16}}peer124.1.1.1{authentication{modepresharedsecretpresharedsecretREALPSKGOESHERE}ikegroupp1localip202.1.1.1tunnel1{allownatnetworksdisableallowpublicnetworksdisableespgroupp2localsubnet10.1.0.0/16remotesubnet10.3.0.0/16}}peer202.2.2.2{authentication{modersarsakeynameWollongongkey}ikegroupp1localip202.1.1.1tunnel1{allownatnetworksdisableallowpublicnetworksdisableespgroupp2localsubnet10.1.0.0/16remotesubnet10.4.0.0/16}}}}rsakeys{rsakeynameWollongongkey{rsakeyREALRSAKEYGOESHERE}}}

/*Warning:Donotremovethefollowingline.*//*===vyattaconfigversion:"cluster@1:conntracksync@1:dhcprelay@1:dhcpserver@4:firewall@3:ipsec@2:nat@3:qos@1:quagga@2:system@3:vrrp@1:wanloadbalance@2:webgui@1:webproxy@1"===*/

Result of "show interfaces":

vyatta@io:~$showinterfacesInterfaceIPAddressStateLinkDescriptioneth010.1.30.1/16upupeth1upupeth2upupeth3upup



eth4upuplo127.0.0.1/8upuplo::1/128upuppppoe058.3.3.3upuppppoe1202.3.3.3upuppppoe2202.1.1.1upuppppoe3150.1.1.1upupvyatta@io:~$

IP Tables Info:

IPTABLES

FilterChainDetailsChainINPUT(policyACCEPT0packets,0bytes)pktsbytestargetprotoptinoutsourcedestination951K337MVYATTA_POST_FW_HOOKall**0.0.0.0/00.0.0.0/0

ChainFORWARD(policyACCEPT0packets,0bytes)pktsbytestargetprotoptinoutsourcedestination300K133MVYATTA_POST_FW_HOOKall**0.0.0.0/00.0.0.0/0

ChainOUTPUT(policyACCEPT22952packets,6031Kbytes)pktsbytestargetprotoptinoutsourcedestination

ChainVYATTA_POST_FW_HOOK(2references)pktsbytestargetprotoptinoutsourcedestination1251K471MACCEPTall**0.0.0.0/00.0.0.0/0

NatChainDetailsChainPREROUTING(policyACCEPT12901packets,926Kbytes)pktsbytestargetprotoptinoutsourcedestination13084936KVYATTA_PRE_DNAT_HOOKall**0.0.0.0/00.0.0.0/01789030DNATtcppppoe3*0.0.0.0/0150.1.1.1tcpdpt:143/*NAT1000*/to:10.1.10.600DNATudppppoe2*0.0.0.0/0202.1.1.1udpdpt:3000/*NAT1010*/to:10.1.11.5:338900DNATtcppppoe3*0.0.0.0/00.0.0.0/0multiportdports5060:5090/*NAT1020*/to:10.1.20.11649DNATudppppoe3*0.0.0.0/00.0.0.0/0multiportdports5060:5090/*NAT1030*/to:10.1.20.100DNATudppppoe3*0.0.0.0/00.0.0.0/0multiportdports30000:31000/*NAT1040*/to:10.1.20.100DNATtcppppoe3*0.0.0.0/00.0.0.0/0multiportdports20:21/*NAT1050*/to:10.1.20.1



00DNATtcppppoe3*0.0.0.0/0150.1.1.1tcpdpt:85/*NAT1060*/to:10.1.115.1884215DNATudppppoe0*0.0.0.0/058.3.3.3udpdpt:4569/*NAT1070*/to:10.1.20.5

ChainPOSTROUTING(policyACCEPT7741packets,565Kbytes)pktsbytestargetprotoptinoutsourcedestination145911002KVYATTA_PRE_SNAT_HOOKall**0.0.0.0/00.0.0.0/000SNATall*pppoe010.1.0.0/160.0.0.0/0/*NAT5*/to:58.3.3.36850436KSNATall*pppoe310.1.0.0/160.0.0.0/0/*NAT10*/to:150.1.1.100SNATall*pppoe010.1.20.0/240.0.0.0/0/*NAT15*/to:58.3.3.3


ChainVYATTA_PRE_DNAT_HOOK(1references)pktsbytestargetprotoptinoutsourcedestination13084936KRETURNall**0.0.0.0/00.0.0.0/0

ChainVYATTA_PRE_SNAT_HOOK(1references)pktsbytestargetprotoptinoutsourcedestination145911002KRETURNall**0.0.0.0/00.0.0.0/0

MangleChainDetailsChainPREROUTING(policyACCEPT95443packets,30Mbytes)pktsbytestargetprotoptinoutsourcedestination

ChainINPUT(policyACCEPT80297packets,28Mbytes)pktsbytestargetprotoptinoutsourcedestination

ChainFORWARD(policyACCEPT15144packets,1994Kbytes)pktsbytestargetprotoptinoutsourcedestination


ChainPOSTROUTING(policyACCEPT18608packets,3133Kbytes)pktsbytestargetprotoptinoutsourcedestination

RawChainDetailsChainPREROUTING(policyACCEPT0packets,0bytes)pktsbytestargetprotoptinoutsourcedestination1260K474MVYATTA_PRE_CT_PREROUTING_HOOKall**0.0.0.0/00.0.0.0/01260K474MNAT_CONNTRACKall**0.0.0.0/0



Top or to post comments

Tue, 12/21/2010 -09:00

Permalink

0.0.0.0/000NOTRACKall**0.0.0.0/00.0.0.0/0

ChainOUTPUT(policyACCEPT0packets,0bytes)pktsbytestargetprotoptinoutsourcedestination232036124KVYATTA_PRE_CT_OUTPUT_HOOKall**0.0.0.0/00.0.0.0/0232036124KNAT_CONNTRACKall**0.0.0.0/00.0.0.0/000NOTRACKall**0.0.0.0/00.0.0.0/0

ChainNAT_CONNTRACK(2references)pktsbytestargetprotoptinoutsourcedestination1284K480MACCEPTall**0.0.0.0/00.0.0.0/0

ChainVYATTA_PRE_CT_OUTPUT_HOOK(1references)pktsbytestargetprotoptinoutsourcedestination232036124KRETURNall**0.0.0.0/00.0.0.0/0

ChainVYATTA_PRE_CT_PREROUTING_HOOK(1references)pktsbytestargetprotoptinoutsourcedestination1260K474MRETURNall**0.0.0.0/00.0.0.0/0

If anyone can offer suggestions as to what I'm doing wrong and howto go about fixing these issues, it would be greatly appreciated.

Thanks,Jeff

Singularity

Issue with NAT and multiple WAN interfacesAs per another post I found on this forum, I have changed my configto use load balancing rules with a single interface per rule to replacemy source NAT rules... This appears to be working... However I'mstill having issues with destination NAT.

Does anyone have any ideas about why I'd be having an asymmetricrouting issue when using multiple WAN interfaces with DNAT? Usingrule 1070 from my config (see previous post) as an example, I cansee packets come in pppoe0 from a remote address on port 4569.




Tue, 12/21/2010 -09:17

Permalink

These are correctly NAT'ed to 10.1.20.5 which sees the request fromthe remote address and sends a response back through the Vyattarouter. Vyatta then receives this request and correctly changes thesource address of the packet to that of pppoe0. It is at this pointwhere we run into problems - Vyatta now takes the packet and sendsit out any one of the WAN interfaces pppoe0, pppoe1, pppoe2 orpppoe3, the interface picked seems to be random. How can I FORCEVyatta to choose the correct interface pppoe0?

Surely I can't be the only one having this sort of problem...? I'vegotta be going wrong somewhere configuration wise, yet as far as Ican see what I've done looks correct... Is there some sort of knownVyatta limitation at work here? Does anyone have a successfulmultiple WAN setup with destination NAT config they could share?

Thanks,Jeff

Tijz

Issue with NAT and multiple WAN interfacesHi Jeff,

No you're not the only one with this problem.. I'm having the same kind of problem, sadly though, no resolution as Iwas just coming here to post it myself.

Instead of ppoe interfaces I just use two ethernet interfaces connectedto two different ADSL routers. I'm also having trouble using DNAT.As you describe, only outside traffic targeting the "main" WANinterface is working correctly, traffic targeting the other interface get'snatted ok but any return traffic is routed over the "main" WANinterface...

I also fiddled with load balancing, but as far as I can see it forces youto use masquerade nat, in which case any SNAT rule does not work(As I understand it masquerade translated ANY AND ALL traffic withthe IP of the interface it is masquerading on).

So.. you're not alone in this, hopefully someone else can point us in aright direction...




Tue, 12/21/2010 -15:22

Permalink


Tue, 12/21/2010 -23:05

Permalink


Wed, 12/22/2010 -04:32

Permalink

Tijs

bjbrock

Not the answer but the reason...Routing decisions are made prior to SNAT while DNAT operates onthe packets before routing. While this isn't the answer to yourquestion it is the logic behind what is happening. ie. You cannot useNAT rules to route.

kherona

Issue with NAT and multiple WAN interfacesHi Friend,

According to Vyatta NAT documentation, the type of NAT should bemasquerade in order to gain the benefit of using multiple interfaces.

in Vyatta you have 3 types of NAT:

source (based on source IP of incoming packets).destination (based on destination IP of incoming packets).masquerade (based on the interface the packets going to leave). (youneed to make source-based-routing in order for this to work).

Tijz

Issue with NAT and multiple WAN interfacesHi all,

thanks for the replies




Wed, 12/22/2010 -09:05

Permalink

But none of these post explain if it should be possible. In theory itshould though. Because in my case I want to replace our DraytekVigor 3300 hardware appliance with a vyatta virtual appliance. TheDraytek has no issues with SNAT using both interfaces.

Surely, if Draytek can do it, vyatta can. I hope....

Singularity

Issue with NAT and multiple WAN interfacesOk, so after going over Vyatta and Netfilter / iproute2 documentation,it has become apparent that this is actually a problem that requiressource based routing, something that Vyatta does not yet supportthrough the CLI.

That means that to get this to work, you will need to get your handsdirty and edit the routing table directly to add in the required sourcebased rules.

For example, to get around the problems I was having the followingneeds to be set up for rule 1070:

This sets up four separate routing tables, one per interface. Each tablehas its interface as the default route -

sudoiprouteadddefaultdevpppoe0table1sudoiprouteadddefaultdevpppoe1table2sudoiprouteadddefaultdevpppoe2table3sudoiprouteadddefaultdevpppoe3table4

Once this is done, we need to setup routing rules to send traffic to thecorrect routing table. The following will send packets coming frominternal device 10.1.20.5 out through table 1, which has pppoe0 as itsdefault route device. Please note that for this to work, any DNAT /SNAT rules in use MUST use the specified interface (in this casepppoe0).

sudo ip rule add from 10.1.20.5 table 1

Unfortunately since this hasn't been done through the Vyatta CLI, thechanges aren't persistent and will disappear on reboot. To fix this, youneed to add anything you do you the file /etc/rc.d/rc.local - Here's an




Fri, 12/24/2010 -04:03

Permalink


example of the one I am testing with atm...

echo"Settingupcustomroutingtables..."

#Setuptables/sbin/iprouteadddefaultdevpppoe0table1/sbin/iprouteadddefaultdevpppoe1table2/sbin/iprouteadddefaultdevpppoe2table3/sbin/iprouteadddefaultdevpppoe3table4

#LocalLANInternettraffictogooutpppoe3/sbin/ipruleaddfrom10.1.100.0/24table4

#Emailandothergeneralservertraffictogooutpppoe3/sbin/ipruleaddfrom10.1.10.0/24table4

#AllVoIPServertraffictogooutpppoe0/sbin/ipruleaddfrom10.1.20.0/24table1

YMMV, but hopefully that helps you out a bit Tijz, has been drivingme crazy for about a week now . . .

Can any Vyatta employee's shed some light as to when sourced basedrouting will be available through the CLI? Being the one of the mostrequested features, It would sure make a lot of peoples lives easier.Can we expect it in the next release?

Thanks,Jeff

Tijz

Issue with NAT and multiple WAN interfacesHi Jeff,

thanks for sharing your research on this!

Will try it, allthough it makes the configuration not very portablewhich might become a problem...

Tijs



Sat, 12/25/2010 -05:48

Permalink


Sat, 12/25/2010 -21:43

Permalink

Tijz

Issue with NAT and multiple WAN interfacesSo, I just set this up. Yes I know, it's christmas.. i had a few hours tokill before going to diner :)

I immediately ran into trouble, as this source based routing does notsolve my problem entirely.

In the good old draytek I DNAT-ed port 25 from both WAN interfaceto my Exchange server. So that in case one of the WAN connectionsbroke down, mail could be delivered through the other WANconnection.

Using source based routing however (the way I understand it at themoment) I route EVERY traffic from one particular source, say myExchange server, through a specific interface (probably different thenthe default gateway).So I am still not able to DNAT prt 25 from both WAN connections toone internal IP. Or am I missing something?

Singularity

Issue with NAT and multiple WAN interfacesHi Tijz,

That situation makes things a little more difficult, but its still possible.What you need to do instead is set up iptables to MARK packetswhen they enter the router based on address / interface, then routebased on packet marking instead. I've done this with a CentOS installbefore and once set up it does work well.

An example is explained pretty well in this blog post:http://nerdboys.com/2006/05/05/conning-the-mark-multiwan-connections-using-iptables-mark-connmark-and-iproute2/(http://nerdboys.com/2006/05/05/conning-the-mark-multiwan-connections-using-iptables-mark-connmark-and-iproute2/)

Some other sites you might find useful:http://lartc.org/howto/lartc.netfilter.html




Mon, 12/27/2010 -00:51

Permalink


(http://lartc.org/howto/lartc.netfilter.html) http://www.linuxtopia.org/Linux_Firewall_iptables/x4368.html(http://www.linuxtopia.org/Linux_Firewall_iptables/x4368.html)

Tijz

Issue with NAT and multiple WAN interfacesOk, thanks! will look into that

Vyatta.com (http://www.vyatta.com) | Support Center (http://www2.vyatta.com/support) | VyattaUniversity (http://www.vyatta.com/support/training) | Contact Us

(http://www.vyatta.com/company/contact)

2012 Vyatta Inc. - All rights reserved.

Vyatta and the Vyatta logo are registered trademarks of Vyatta, Inc. Other marks are trademarksof their respective holders.

issue with nat and multiple wan interfaces _ vyatta

Documents

destination nat

different wan interfaces

vyatta server

thedefault route interface

outbound interface

use source nat

multiple wan interfaceshello

incoming wan packets