| by Chris Lilley | 1 comment

BGP Multihomed Collapsed Core

In this exercise I used GNS3 to help visualize and draw a topology where NAT is not performed at the far Edge device but at a firewall closer to the core of the network. In this scenario you have your stateful firewalls performing NAT and any IPS/IDS while having the edge routers stick to strictly routing.

This stems from a curiosity from how one would handle NAT being pushed from an edge device. Most of my studies from the CCNP and from the Internet Routing Architectures book often don’t talk about NAT. I’m glad I did this exercise because it introduced an issue I wouldn’t have thought of otherwise.

The issue stems from OSPF using a default maximum of 4 equal cost paths to load balance. Normally this would not cause issues in an environment without an appliance performing NAT. With NAT introduced the traffic would equally go out both firewalls with different source IP addresses, causing issues with the packets being reconstructed at the other end. Changing the maximum paths to 1 allowed traffic to flow correctly.


Let me describe this network topology at a high level:


This network has a collapsed core topology with the core/distribution layers being combined. This was done to simplify the network a bit. There is layer3 routing starting at the access layer, and between all devices there is a routed /30. The reason behind this was to lower the chance of broadcast storms and spanning tree bringing the network down to its knees.

All devices from the firewalls down to the access switches are in the same OSPF topology. Both of the firewalls are distributing a default route to the rest of the OSPF topology and both firewalls have static default routes to the edge routers directly above them in the diagram. Both of the firewalls are performing source NAT from the Loopback address on each device.

Now working upwards from the firewalls, Edge Router 1 is an eBGP peer with Provider 1 Edge and Edge Router 2 is an eBGP peer with Provider 2 Edge. Edge Router 1 is an iBGP peer with Edge Router 2 to distribute its known BPG routes to each other. Provider 1 and 2 are advertising its loopback address to the edge devices to simulate a couple addresses on the “Internet”.

Next up is a snippet of the device configurations and anything that needs some more elaborating.

Floor1AccessSW

interface GigabitEthernet0/0
 no switchport
 ip address 10.10.254.22 255.255.255.252
 description Uplink_to_CoreSwitch1

interface GigabitEthernet0/1
 no switchport
 ip address 10.10.254.34 255.255.255.252
 description Uplink_to_CoreSwitch2

interface GigabitEthernet2/0
 switchport access vlan 10
 switchport mode access
 description PC
 
interface Vlan10	
 ip address 10.10.10.254 255.255.255.0
 
router ospf 1
 network 10.10.0.0 0.0.255.255 area 0
 maximum-paths 1

Floor2AccessSW

interface GigabitEthernet0/0
 no switchport
 ip address 10.10.254.26 255.255.255.252
 description Uplink_to_CoreSwitch1

interface GigabitEthernet0/1
 no switchport
 ip address 10.10.254.38 255.255.255.252
 description Uplink_to_CoreSwitch2

interface GigabitEthernet3/2
 switchport access vlan 20
 switchport mode access
 description PC
 
interface Vlan20
 ip address 10.10.20.254 255.255.255.0

router ospf 1
 network 10.10.0.0 0.0.255.255 area 0
 maximum-paths 1

Floor3AccessSW

Interface GigabitEthernet0/0
 no switchport
 ip address 10.10.254.18 255.255.255.252
 description Uplink_to_CoreSwitch1

interface GigabitEthernet0/1
 no switchport
 ip address 10.10.254.30 255.255.255.252
 description Uplink_to_CoreSwitch2

interface GigabitEthernet3/0
 switchport access vlan 30
 switchport mode access
 description PC

interface Vlan30
 ip address 10.10.30.254 255.255.255.0

router ospf 1
 network 10.10.0.0 0.0.255.255 area 0
 maximum-paths 1

CoreSwitch1

interface GigabitEthernet0/0
 no switchport
 ip address 10.10.254.21 255.255.255.252
 description Downlink_to_Floor1AccessSW
 
interface GigabitEthernet0/1
 no switchport
 ip address 10.10.254.2 255.255.255.252
 description Uplink_to_Firewall1

interface GigabitEthernet0/2
 no switchport
 ip address 10.10.254.10 255.255.255.252
 description Uplink_to_Firewall2

interface GigabitEthernet1/0
 no switchport
 ip address 10.10.254.25 255.255.255.252
 description Downlink_to_Floor2AccessSW
 
interface GigabitEthernet1/1
 no switchport
 ip address 10.10.254.17 255.255.255.252
 description Downlink_to_Floor3AccessSW
 
router ospf 1
 network 10.10.0.0 0.0.255.255 area 0
 maximum-paths 1

CoreSwitch2

interface GigabitEthernet0/0
 no switchport
 ip address 10.10.254.14 255.255.255.252
 description Uplink_to_Firewall2

interface GigabitEthernet0/1
 no switchport
 ip address 10.10.254.6 255.255.255.252
 description Uplink_to_Firewall1

interface GigabitEthernet0/2
 no switchport
 ip address 10.10.254.33 255.255.255.252
 description Downlink_to_Floor1AccessSW

interface GigabitEthernet0/3
 no switchport
 ip address 10.10.254.37 255.255.255.252
 description Downlink_to_Floor2AccessSW
 
interface GigabitEthernet1/0
 no switchport
 ip address 10.10.254.29 255.255.255.252
 description Downlink_to_Floor3AccessSW

router ospf 1
 network 10.10.0.0 0.0.255.255 area 0
 maximum-paths 1

Firewall 1

There is a static route to each Edge Router, and the static route to the Edge Router 2 has an AD of 10. Something I could do to improve on this would be setup tracking and IP SLA on the two static routes to further improve resiliency.

There are two reasons I created a separate OSPF process for the public IP addresses. First, I wanted to make sure the edge devices didn’t have the private IP range in there to make sure NAT works correctly. Secondly, I wanted to change the maximum-paths attribute for the private IP range only. For those reasons I created a separate OSPF process.

interface Loopback0
 ip address 50.50.50.220 255.255.255.255

interface GigabitEthernet0/0
 ip address 50.50.50.250 255.255.255.252
 ip nat outside
 description Uplink_to_EdgeRouter1

interface GigabitEthernet0/1
 ip address 10.10.254.1 255.255.255.252
 ip nat inside
 description Downlink_to_CoreSwitch1

interface GigabitEthernet0/2
 ip address 10.10.254.5 255.255.255.252
 ip nat inside
 description Downlink_to_CoreSwitch2

interface GigabitEthernet0/3
 ip address 50.50.50.234 255.255.255.252
 ip nat outside
 description Uplink_to_EdgeRouter2

router ospf 1
 network 10.10.0.0 0.0.255.255 area 0
 maximum-paths 1
 default-information originate

router ospf 2
 network 50.50.50.0 0.0.0.255 area 0

ip nat inside source list 10 interface Loopback0 overload
ip route 0.0.0.0 0.0.0.0 50.50.50.249
ip route 0.0.0.0 0.0.0.0 50.50.50.233 10

access-list 10 permit 10.10.0.0 0.0.255.255

Firewall 2

interface Loopback0
 ip address 50.50.50.221 255.255.255.255

interface GigabitEthernet0/0
 ip address 50.50.50.246 255.255.255.252
 ip nat outside
 description Uplink_to_EdgeRouter2

interface GigabitEthernet0/1
 ip address 10.10.254.13 255.255.255.252
 ip nat inside
 description Downlink_to_CoreSwitch2

interface GigabitEthernet0/2
 ip address 10.10.254.9 255.255.255.252
 ip nat inside
 description Downlink_to_CoreSwitch1

interface GigabitEthernet0/3
 ip address 50.50.50.238 255.255.255.252
 ip nat outside
 description Uplink_to_EdgeRouter1

router ospf 2
 network 50.50.50.0 0.0.0.255 area 0

router ospf 1
 network 10.10.0.0 0.0.255.255 area 0
 maximum-paths 1
 default-information originate

ip nat inside source list 10 interface Loopback0 overload
ip route 0.0.0.0 0.0.0.0 50.50.50.245
ip route 0.0.0.0 0.0.0.0 50.50.50.237 10

access-list 10 permit 10.10.0.0 0.0.255.255

EdgeRouter 1

I created a route-map called LocalOnly and applied it to the eBGP neighborship to Provider 1 Edge. Purpose of this so that only the 50.50.50.0/24 local route is advertised to the peer. Otherwise Provider 2 Edges routes would be advertised to Provider 2 as well and AS 123 could be used as a transit AS to get to either AS 111 or 222.

In order to advertise a route to a BGP peer the same route has to be in your routing table first. To satisfy this condition I created a route for 50.50.50.0/24 pointed to Null 0.

Another note, I added the following commands:

neighbor 50.50.50.253 update-source Loopback0

 neighbor 50.50.50.253 next-hop-self

Reason for the first command was to allow for links to go down and keep the iBGP neighborship up between EdgeRouter1 and 2. The second command is used so the iBGP neighbors can get to each others respective routes they are advertising to each other since there is no OSPF network command for the network between the EdgeRouter and its respective Provider router

interface Loopback0
 ip address 50.50.50.253 255.255.255.255

interface GigabitEthernet0/0
 ip address 60.60.60.253 255.255.255.252
 description Uplink_to_Provider1Edge

interface GigabitEthernet0/1
 ip address 50.50.50.249 255.255.255.252
 description Downlink_to_Firewall1

interface GigabitEthernet0/2
 ip address 50.50.50.237 255.255.255.252
 description Downlink_to_Firewall2

interface GigabitEthernet0/3
 ip address 50.50.50.241 255.255.255.252
 description Uplink_to_EdgeRouter2

router ospf 1
 network 50.50.50.0 0.0.0.255 area 0

router bgp 123
 bgp log-neighbor-changes
 network 50.50.50.0 mask 255.255.255.0
 neighbor 50.50.50.254 remote-as 123
 neighbor 50.50.50.254 update-source Loopback0
 neighbor 50.50.50.254 next-hop-self
 neighbor 60.60.60.254 remote-as 111
 neighbor 60.60.60.254 route-map LocalOnly out

ip route 50.50.50.0 255.255.255.0 Null0

route-map LocalOnly permit 10
 match ip address 10

access-list 10 permit 50.50.50.0 0.0.0.255

EdgeRouter 2

interface Loopback0
 ip address 50.50.50.254 255.255.255.255

interface GigabitEthernet0/0
 ip address 70.70.70.253 255.255.255.252
 description Uplink_to_Provider2Edge

interface GigabitEthernet0/1
 ip address 50.50.50.245 255.255.255.252
 description Downlink_to_Firewall2

interface GigabitEthernet0/2
 ip address 50.50.50.233 255.255.255.252
 description Downlink_to_Firewall1

interface GigabitEthernet0/3
 ip address 50.50.50.242 255.255.255.252
 description Uplink_to_EdgeRouter1

router ospf 1
 network 50.50.50.0 0.0.0.255 area 0

router bgp 123
 bgp log-neighbor-changes
 network 50.50.50.0 mask 255.255.255.0
 neighbor 50.50.50.253 remote-as 123
 neighbor 50.50.50.253 update-source Loopback0
 neighbor 50.50.50.253 next-hop-self
 neighbor 70.70.70.254 remote-as 222
 neighbor 70.70.70.254 route-map LocalOnly out

ip route 50.50.50.0 255.255.255.0 Null0

route-map LocalOnly permit 10
 match ip address 10

access-list 10 permit 50.50.50.0 0.0.0.255

Provider 1 Edge

interface Loopback0
 ip address 8.8.8.8 255.255.255.255

interface GigabitEthernet0/0
 ip address 60.60.60.254 255.255.255.252
 description Uplink_to_EdgeRouter1

router bgp 111
 bgp log-neighbor-changes
 network 8.8.8.8 mask 255.255.255.255
 neighbor 60.60.60.253 remote-as 123

Provider 2 Edge

interface Loopback0
 ip address 4.2.2.2 255.255.255.255

interface GigabitEthernet0/0
 ip address 70.70.70.254 255.255.255.252
 description Uplink_toEdgeRouter2

router bgp 222
 bgp log-neighbor-changes
 network 4.2.2.2 mask 255.255.255.255
 neighbor 70.70.70.253 remote-as 123

1 Comment

Grant

Jan 1, 2019, 10:43 pm Reply

Wow, great stuff Chris!

Leave a Reply