How to prevent asymmetric routing with multiple eBGP routers?

Question

I have 2 routers announcing a /22 subnet to different providers (one providers connects to each of the 2 routers). I have split the /22 in two /23 to announce one /23 on each of the routers plus the /22 (the providers will take the more specific route). This allows me to fail over and keep traffic inside the /23 in and out the same provider.

What are other ways in which I could announce just the /22 with both routers and have packets from servers on the network behind the routers go back out the same router in which they came in from?

EDIT:

The main problem I come across, which end users and clients complain about the most, is that the least hop route is sometimes not the "optimal" route. In my case, I know that Provider B may have better latency to X nation. But when packets come in from provider B, they may go out Provider A or provider B. The reverse is also true. If I send a packet to X nation out provider A, even though it may have more hops back, the packet will likely come in from Provider B (which may have higher latency, packet loss, etc. to this nation)

Solution 1

Strictly speaking, you loose complete control of inbound routing paths when you announce your prefix to multiple providers because there are independent routing decisions made downstream for return traffic to you. Furthermore, your announcements could even be modified by downstream providers after you send them.

Example

This is one example of what can happen. Suppose you have AS 777, which owns 2.2.0.0/22. You have services that a company with Router A needs to access... Let's also assume that AS 100 doesn't have a good link to you (maybe it's intermittently corrupting traffic due to physical-layer problems you haven't been able to fix). So you think to yourself, "I'll just prepend all my announcements to AS100 with a large number of ASNs so nobody will prefer the AS100 link until I can fix this".

The problem is that you only have complete control of your outbound routing decisions. You don't get complete control inbound... so let's suppose Router A's administrator doesn't know your link to AS100 is bad. They are dual-homed to AS200 and AS100, but AS100 offers much cheaper transit, per-Mbps; therefore Router A's engineer takes full routes from AS100 and only uses AS200 as a backup (taking only a default from them).

As the admin of AS 777, you can force traffic to Router A through AS 200, but traffic from Router A to 2.2.0.0/22 would still take AS 100 (because the best route is through AS 100, at Router A).

Possible solutions

Usually asymmetric paths matter because of a load-balancer or firewall that is receiving the traffic. Some possible solutions:

• For services offered in other ASNs, you could source NAT all outbound traffic to an address that is local to a link going to a specific AS (this doesn't work well for service providers though)
• For services offered in your ASN, you could source NAT all inbound traffic to an address that is local to a subnet on your BGP peering router in question (this doesn't work well for service providers though)
• You could set up a DMZ that receives traffic from all the different ASNs and offer a single ingress / egress point for traffic from your upstreams. Under some circumstances, you can even synchronize the state table on the DMZ devices. (this doesn't work well for service providers though)
• If you are willing to use a features like BGP Conditional Advertisement, you can sometimes work around these kind of problems at the cost of not using other carriers for inbound traffic to 2.2.0.0/22.

If you provide more details about the nature of the services and problem, we might be able to offer more specific advice.

Solution 2

What are other ways in which I could announce just the /22 with both routers and have packets from servers on the network behind the routers go back out the same router in which they came in from?

Why do you want this? Or more accurately, why do you want to force this behavior (since typically this is what will happen thanks to the magic of routing)?

---

The entire point of BGP and the distributed routing hierarchy that makes up the modern internet is that your packets will take the best available route to their destination. If a packet came in over Route A but Route B is a better choice for the reply for some reason then why would you not want to send your reply over the optimal route?

The only reason I can see for this is wanting to prejudice your traffic away from a more expensive carrier (in which case there are better methods).

Comments

[Mike Pennington]

Is source NAT an option on the links to your providers, or are you a service provider?

[Andy Shinn]

I think you are on the right track and that I am not really looking at the whole network picture. NAT behind the routers would allow me some more flexibility in terms of filtering and doing source NAT to different outbound gateways while just announcing the /22 out both providers.

[Deb]

I've seen stateful devices throw packets on an asymetric route on the floor. If that leg hasn't seen the SYN, the SYN-ACK will get dropped and the connection will never establish. However, these routes sound like they're outside the security perimeter so this problem may not be an issue.

[Andy Shinn]

Ultimately, I think what I am trying to accomplish does involve more of the load balance / outbound NAT level so I am accepting this since it was the first to mention NAT (and you have a really detailed answer on the asymmetric BGP nature). Source NAT at the server / load balancing level would allow me to tag / filter the IP to send to a weighted gateway (router 1 or router2) which could then default to the respective provider for that router.