Internet multi-homing

Connecting a small company to multiple service providers is a simple task. Usually you get two provider-assigned ip addresses (either static or dynamic). Since each ISP will normally allocate a single ip address, you have to use private ip addresses on the inside. ISPs will be very unwilling to run a dynamic routing protocol with small clients, and you have to configure static default routing. Usually one provider is preferred over the other, resulting in a primary and a backup default route. Another option is static route load balancing, which is discussed later. The router also has to perform two independent NAT translations, one for packets sent to ISP 1 (where local addresses get translated to the ip address assigned by ISP 1) and another translation for packets sent to ISP 2 (where local addresses get translated to the ip address assigned by ISP 2).

Scenario

Here is our scenario. In our company we have two ISPs named A and B. The router is connected to each of them on a separate Fast Ethernet interface and public static addresses are configured. For details see the network diagram below. The challenge is to achieve redundancy and load balancing or both if possible. The problem is that there is no reliable mechanism for determining ISP failures. Static routes stay in the routing table even if the next hop gateway becomes unavailable. Unless physical failure they are removed from the routing table only if the corresponding layer two or layer three protocol is down. For ethernet, link failures are very unlikely to occur. Usually there are a number of intermediate devices like switches, cable modems and routers. That's why failover will never happen. Below I suggest a way to track the availability of a next-hop router and act accordingly.

The same results can be achieved with Linux router and a bash script using ICMP echo requests to monitor the default gateway. Depending on ping results routes can be dynamically added or removed. The functionality of static route tracking may be builtin into the Linux kernel. For more information visit http://www.ssi.bg/~ja/, an excellent site with lots of interesting stuff. This is not a commercial and everything is free.

Router initial configuration

Let's start with router initial configuration.

hostname router
!
ip cef
!
!
interface FastEthernet0/0
 description Local LAN
 ip address 192.168.0.1 255.255.255.0
 ip nat inside
!
interface FastEthernet0/1
 description to ISP A
 ip address 195.168.1.2 255.255.255.252
 ip nat outside
!
interface FastEthernet0/2
 description to ISP B
 ip address 195.168.2.2 255.255.255.252
 ip nat outside
!
ip nat inside source route-map ISP_A interface FastEthernet0/1 overload
ip nat inside source route-map ISP_B interface FastEthernet0/2 overload
!
route-map ISP_A permit 10
 match interface FastEthernet0/1
!
route-map ISP_B permit 10
 match interface FastEthernet0/2

 Note: Route maps are matching outgoing interfaces and different nat translations are performed

Static route load-balancing

By default CEF is doing per session load-balancing. If load-balancing is unnecessary different administrative distances are assigned to static routes. The route with lower distance is preferred. For example, if ISP A is our primary Internet provider the default  route may be configured with a distance of 1 (all static routes are assigned this administrative distance) and the default route through ISP B may be configured with a distance of 100. In that case the default route through ISP B will be used if only the route through ISP A becomes unavailable.

Static route tracking

With Enhanced Object Tracking a generic track object can monitor presence of an ip route, state of an SLA, etc. Here we'll bind a track object to a static route. When the track object’s state is down, the static route is removed from the routing table, when the state is up the route is inserted. This is exactly what is needed to provide default route failover.
To configure such kind of static route we have to provide two prerequisites:

• an ip sla object pinging the next hop router (default gw) on both ISPs
• track objects to monitor the state of the sla objects

More configuration

Let's start with ip sla configuration:

ip sla 1
 icmp-echo 195.168.1.1 source-interface FastEthernet0/1
 timeout 1000
 frequency 3
ip sla schedule 1 life forever start-time now

First we specify the objet type an icmp-echo request an the source interface of ISP A. Specifying the source is important. We do not want to check ISP A’s availability through ISP B’s interface. Then we specify timeout of 1s and frequency of 3s. Frequency is how often icmp requests will be sent to the remote router. At last the ip sla object is put into service with the ip sla schedule command. If we are providing failover we need only one ip sla object for the primary route. If we are going to load-balance we need a second ip sla object for the ISP B. Otherwise if ISP B experiences problems half the sessions to the Internet will be black-holed. The same configurations for ISP B:

ip sla 2
 icmp-echo 195.168.2.1 source-interface FastEthernet0/2
 timeout 1000
 frequency 3
ip sla schedule 2 life forever start-time now

Next come the track objects:

track 1 rtr 1 reachability
 delay down 9 up 10
track 2 rtr 2 reachability
 delay down 9 up 10

The track objects monitor the state of the corresponding ip sla object. We use delay option to specify how long the next-hop router should be unreachable before it is declared as down. We specify a delay of 9s which is three times the ip sla frequency. In this way a single or two icmp packets lost will not affect the state of the track object. The up delay is set to 10s. In this way we stop interface flapping from affecting our routing.
At last we bind both static routes with the corresponding track objects - ISP A with object 1 and ISP B with object 2. Because I do not wish to have load-balancing I assign the route through ISP B an administrative distance 100.

ip route 0.0.0.0 0.0.0.0 fa0/1 track 1
ip route 0.0.0.0 0.0.0.0 fa0/2 100 track 2

With this we managed to provide default static route failover and optionally load-balancing which is our goal.

DHCP scenario

What about if our router acquires its public addresses and default route by DHCP? In such case, it is very difficult to define a next-hop router addresses used by ip sla objects. What is more how do we check the default route availability? An ip sla object is not an option in this case. But there is an elegant way out. The only configuration change needed is to bind route tracking with DHCP obtained routes instead of statically configured routes. Here is interface configuration changes:

interface FastEthernet0/1
 description to ISP A
 ip address dhcp
 ip dhcp client route track 1
 ip nat outside
!
 interface FastEthernet0/2
 description to ISP B
 ip address dhcp
 ip dhcp client route track 2
 ip nat outside

And do not forget to delete the static routes with ip sla objects previously configured.

comments: angel@mreji.net