The following example specifies Ethernet interface 0 as a simplex Ethernet interface:
interface ethernet 1
ip address 128.9.1.2
transmit-interface ethernet 0
The Cisco VSS simplifies network configuration and operation by providing a loop-free Layer 2 topology using two Catalyst 6500 switches acting as one big Virtual switch. VSS reduces number of Layer 3 routing neighbors by providing a Layer 2 connectivity for access/distribution switches.
A VSS is a pair of combined 6500 switches acting as a single network element with redundancy and load balancing over port-channels (etherchannels). One switch becomes the master or active chassis and the other one becomes the VSS standby.
To share control and data traffic between two chassis a VSL – Virtual Switch Link is required. VSL is implemented as a Port Channel. The control traffic gets higher priority over data on a VSL and never gets discarded.
before configuration make sure that you have right IOS image, proper VS-capable supervisor and line cards, you can check your line cards with the following command:
6500A#switch convert check vss-capable
This is a VSS capable switch.
VSL ports can be configured in slot: 4, 5, 6
By default, 6500 series switches are configured to operate in standalone mode. The following steps are required in order to setup virtual stack across two 6500 chassis.
Step1: Make sure SSO and NSF are configured and enabled
Step2: Assign Virtual Switch Doman and Switch numbers
Step3: Configure VSL Port Channel, PO# should be unique on each chassis.
Step4: Convert Standalone Chassis to Virtual Switch mode
6500A(config)#redundancy
6500A(config-red)#mode sso
!
6500A(config)#switch virtual domain ?
<1-255> Virtual switch domain number
6500A(config)#switch virtual domain 100
Domain ID 100 config will take effect only
after the exec command 'switch convert mode virtual' is issued
6500A(config-vs-domain)#switch 1
6500A(config-vs-domain)#exit
...
6500B(config)#switch virtual domain 100
Domain ID 100 config will take effect only
after the exec command 'switch convert mode virtual' is issued
6500B(config-vs-domain)#switch 2
6500B(config-vs-domain)#exit
6500A(config)#int port-channel 100
6500A(config-if)#switch virtual link 1
6500A(config-if)#no shutdown
6500A(config-if)#exit
6500A(config)#int te5/4
6500A(config-if)#channel-group 100 mode on
6500A(config-if)#no shutdown
6500A(config)#int te6/4
6500A(config-if)#channel-group 100 mode on
6500A(config-if)#no shutdown
...
6500B(config)#int port-channel 200
6500B(config-if)#switch virtual link 2
6500B(config-if)#no shutdown
6500B(config-if)#exit
6500B(config)#int te5/4
6500B(config-if)#channel-group 200 mode on
6500B(config-if)#no shutdown
6500B(config)#int te6/4
6500B(config-if)#channel-group 200 mode on
6500B(config-if)#no shutdown
6500A#switch convert ?
check check if this switch and its modules are VSS capable or not
mode mode keyword virtual or standalone
6500A#switch convert mode ?
stand-alone stand-alone switch
virtual virtual switch
6500A#switch convert mode virtual
This command will convert all interface names
to naming convention "interface-type switch-number/slot/port",
save the running config to startup-config and
reload the switch.
NOTE: Make sure to configure one or more dual-active detection methods
once the conversion is complete and the switches have come up in VSS mode.
Do you want to proceed? [yes/no]: yes
Converting interface names
Building configuration...
Saving converted configuration to bootflash: ...
Destination filename [startup-config.converted_vs-20110705-214318]?
*** --- SHUTDOWN NOW ---
Ensure that same PFC operating mode is being used on both chassis in order to have a proper SSO redundancy mode:
6500A#show platform hardware pfc mode
PFC operating mode : PFC3C
Now, Let’s look at configuration of VSS switch after reboot:
6500A#sh run
…
!
hostname 6500A
!
switch virtual domain 100
switch mode virtual
!
mls netflow interface
mls cef error action reset
!
spanning-tree mode pvst
spanning-tree extend system-id
diagnostic bootup level minimal
!
redundancy
main-cpu
auto-sync running-config
mode sso
!
vlan internal allocation policy ascending
vlan access-log ratelimit 2000
!
!
interface Port-channel100
no switchport
no ip address
switch virtual link 1
mls qos trust cos
no mls qos channel-consistency
!
interface Port-channel200
no switchport
no ip address
switch virtual link 2
mls qos trust cos
no mls qos channel-consistency
!
interface GigabitEthernet1/1/1
no switchport
no ip address
shutdown
!
interface GigabitEthernet1/1/2
no switchport
no ip address
shutdown
...
...
interface TenGigabitEthernet1/5/4
no switchport
no ip address
mls qos trust cos
channel-group 100 mode on
...
...
interface TenGigabitEthernet1/6/4
no switchport
no ip address
mls qos trust cos
channel-group 100 mode on
...
...
interface TenGigabitEthernet2/5/4
no switchport
no ip address
mls qos trust cos
channel-group 200 mode on
...
...
interface TenGigabitEthernet2/6/4
no switchport
no ip address
mls qos trust cos
channel-group 200 mode on
...
...
interface Vlan1
no ip address
shutdown
!
ip classless
ip forward-protocol nd
!
control-plane
!
line con 0
line vty 0 4
login
!
mac-address-table aging-time 480
no event manager policy Mandatory.go_switchbus.tcl type system
!
module provision switch 1
slot 1 slot-type 147 port-type 61 number 48 virtual-slot 17
slot 3 slot-type 152 port-type 31 number 48 virtual-slot 19
slot 4 slot-type 227 port-type 60 number 8 virtual-slot 20
slot 5 slot-type 254 port-type 31 number 2 port-type 61 number 1 port-type 60 number 2 virtual-slot 21
slot 6 slot-type 254 port-type 31 number 2 port-type 61 number 1 port-type 60 number 2 virtual-slot 22
!
module provision switch 2
slot 1 slot-type 147 port-type 61 number 48 virtual-slot 33
slot 3 slot-type 152 port-type 31 number 48 virtual-slot 35
slot 4 slot-type 227 port-type 60 number 8 virtual-slot 36
slot 5 slot-type 254 port-type 31 number 2 port-type 61 number 1 port-type 60 number 2 virtual-slot 37
slot 6 slot-type 254 port-type 31 number 2 port-type 61 number 1 port-type 60 number 2 virtual-slot 38
!
end
Verification is the most important part of configuration, you need to make sure that commands that you have entered are working as expected!
6500A#show switch virtual
Switch mode : Virtual Switch
Virtual switch domain number : 100
Local switch number : 1
Local switch operational role: Virtual Switch Active
Peer switch number : 2
Peer switch operational role : Virtual Switch Standby
6500A#show switch virtual role
Switch Switch Status Preempt Priority Role Session ID
Number Oper(Conf) Oper(Conf) Local Remote
------------------------------------------------------------------
LOCAL 1 UP FALSE(N ) 100(100) ACTIVE 0 0
REMOTE 2 UP FALSE(N ) 100(100) STANDBY 4004 1462
In dual-active recovery mode: No
6500A#show switch virtual link
VSL Status : UP
VSL Uptime : 43 minutes
VSL SCP Ping : Pass
VSL ICC Ping : Pass
VSL Control Link : Te1/5/4
6500A#show switch virtual link port-channel
Flags: D - down P - bundled in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use N - not in use, no aggregation
f - failed to allocate aggregator
M - not in use, no aggregation due to minimum links not met
m - not in use, port not aggregated due to minimum links not met
u - unsuitable for bundling
d - default port
w - waiting to be aggregated
Group Port-channel Protocol Ports
------+-------------+-----------+-------------------
100 Po100(RU) - Te1/5/4(P) Te1/6/4(P)
200 Po200(RU) - Te2/5/4(P) Te2/6/4(P)
If I connect my console to secondary chassis:
6500A-sdby>
Standby console disabled
One VSS supports a maximum of 512 - 2 port channels (deducting 2 POs for VSL) Configuration of MEC is not different than any regular etherchannel! The Different is physical connectivity, one link connects to VSS Active and the other link connects to VSS Standby chassis creating a high bandwidth active/active aggregated link-bundle (Etherchannel)
Access switch:
interface GigabitEthernet1/1/1
channel-group 10 mode on
!
interface GigabitEthernet1/1/2
channel-group 10 mode on
Core switch:
interface Port-channel10
switchport
!
interface GigabitEthernet1/3/17
switchport
channel-group 10 mode on
!
interface GigabitEthernet2/3/17
switchport
channel-group 10 mode on
Verify:
6500A#sh etherchannel summary
Number of channel-groups in use: 3
Number of aggregators: 3
Group Port-channel Protocol Ports
------+-------------+-----------+--------------------------------------
10 Po10(SU) - Gi1/3/17(P) Gi2/3/17(P)
100 Po100(RU) - Te1/5/4(P) Te1/6/4(P)
200 Po200(RU) - Te2/5/4(P) Te2/6/4(P)
Switch12#sh etherchannel summary
Number of channel-groups in use: 1
Number of aggregators: 1
Group Port-channel Protocol Ports
------+-------------+-----------+--------------------------------------
10 Po10(SU) - Gi1/1/1(P) Gi1/1/2(P)
As you may now, CCIE Service Provider has been updated to new version (V.3) as of April 18 2011. There are several changes in the exam which are very important, ATM seems to be removed and IOS XR has a major role along with more IPv6 plus extended troubleshooting section. I went through my preparation notes and found best practices note which is general rule of thumb for CCIE SP lab.
General Best Practices for exam:
configure loopback0 as router-id
configure metric parameters (e.g. auto-cost reference-bandwidth / metric-style wide)
configure network types / is-types
configure timers or protocol specific features, DR priority and so on.
configure "no bgp default ipv4-unicast"
configure neighbors remote-as
configure update-source as loopback0
configure address-family ipv4 and activate neighbors (and RR if necessary)
configure next-hop-self on border-routers (if necessary)
advertise loopback address with the network command
make sure you have advertised loopback addresses
make sure next-hop is propagated correctly
set "ip bgp-community new-format"
send bgp communities to required neighbors
configure routing policies/controls based on community lists
configure label protocol
configure loopback as ldp session source - router-id
configure label options (numbers, session protection, etc)
configure mpls interfaces
configure routers for MPLS TE
configure OSPF/ISIS for MPLS TE
configure routing area for MPLS TE (or Level1/2)
configure interfaces for TE and RSVP
configure tunnel interfaces
configure static or dynamic paths
configure VRFs and try to import RTs
configure MP-IBGP
configure MP-eBGP
configure send-label in BGP (if required)
configure next-hop-unchanged (between MP-eBGP points if necessary)
set BGP attributes to control routes. e.g:
send MED to MP-BGP CE neighbor (to make it less preferred)
set Weight for a neighbor in different AS to be preferred
For a detailed Lab exam blue print you can refer to:
https://learningnetwork.cisco.com/docs/DOC-10145
Cisco Content Switching Module adds layer 4 to layer 7 content switching capabilities to the Catalyst 6500/7600 Series providing high-performance load balancing for servers, firewalls or even NAC boxes! Cisco CSM is old and not a new product, it has been replaced by newer Cisco ACE load-balancers. Cisco ACE is being sold in two different fashions, as Blade modules just like CSM or as 4710 appliances just like CSS appliance-based load-balancers.
ACE supports virtualization (multiple-contexts and resource allocation) while CSM is monolithic. CSM uses IOS configuration file and stores its configuration in the running-config. It’s really easy and straight forward to configure if you know what you’re doing!
It’s been awhile since the last time that I touched CSM as it’s end of sales and support. EOL/EOS reference link! But you might have some customers that are still using this product and they might need some levels of technical support. Last week, a call came through asking for CSM support, the customer has four CSM blades inside 6509s in redundant fashion in main and DR site…
This post quickly reviews the following topics:
The most basic and common method of setting up a load-balancer is to have a VLAN pair; one facing servers and the other VLAN facing clients (facing firewall/routers). Then the Load-balancer sits between client and server and creates a VIP (virtual ip address). That VIP is related to several real servers but all are seen as one entity or one VIP. That VIP is reflected by a NAT statement further on a firewall to a public address and is ready for being resolved by DNS.
Client request comes through client VLAN of load-balancer and hits the VIP address then it gets distributed among available real servers.
The first step is to create CLIENT and SERVER VLAN pair:
1. Create two VLANs on 6500 catalyst but do not assign an ip address.
2. Create same VLANs on the CSM and assign IP address inside the CSM:
module ContentSwitchingModule 3
vlan 302 server
ip address 10.8.108.2 255.255.255.0
!
vlan 301 client
ip address 10.8.8.2 255.255.255.0
gateway 10.8.8.254
Gateway is required for CSM to talk to clients (it might be your DMZ firewall interface)
The next step is to create server-farm (real-servers sit there) and vserver (virtual server – VIP)
probe HTTP http
recover 3
request method head
expect status 200
interval 2
failed 6
port 80
!
serverfarm HTTP_2010
nat server
no nat client
real 10.8.108.17
inservice
real 10.8.108.18
inservice
probe HTTP
!
vserver HTTP_2010
virtual 10.8.8.201 tcp www
serverfarm HTTP_2010
replicate csrp connection
persistent rebalance
inservice
The first part of above configuration is probe config. It checks the real servers health and sees whether HTTP code 200 is returned by each HTTP server. If probe to a server fails, that server will not be used.
The Serverfarm section configures two real servers (IP address is from SERVER VLAN) and uses HTTP probe for server health check. The vserver section, defines the VIP and correlates the server farm to vserver. Easy! now let’s verify:
dc1sw01#sh module csm 3 vlan
vlan IP address IP mask type
---------------------------------------------------
301 10.8.8.2 255.255.255.0 CLIENT
302 10.8.108.2 255.255.255.0 SERVER
401 10.8.12.2 255.255.255.0 CLIENT
402 10.8.112.2 255.255.252.0 SERVER
997 0.0.0.0 0.0.0.0 FT
CSM FT configuration is pretty straight-forward. FT is configured when you have two CSM blades located in two 6500 catalysts to work as a fail-over group. You can also create a fault-tolerant configuration with two CSMs in a single Catalyst 6500 series chassis.
First step to create a VLAN on both catalysts specifically for FT and the configure priority on each CSM (master must have higher priority):
vlan 997
name CSM_Failover
!
module ContentSwitchingModule 3
ft group 1 vlan 997
priority 110
preempt
!
Then you need to configure your VLAN-pair to include alternate CSM IP address to be seen as single entity by servers to be used as the default gateway (the alias command):
vlan 302 server
ip address 10.8.108.2 255.255.255.0 alt 10.8.108.3 255.255.255.0
alias 10.8.108.1 255.255.255.0
!
vlan 301 client
ip address 10.8.8.2 255.255.255.0 alt 10.8.8.3 255.255.255.0
gateway 10.8.8.254
What if we need to have more than one CLIENT/SERVER VLAN-pair?
Usually the client side is connected to a firewall (Cisco FWSM, ASA or something else). You can have different client/server VLAN pairs while firewall serves each client VLAN as a different DMZ zone.
You can also use this approach during server migration between DMZs! or even during DMZ migration…
!
vlan 402 server
ip address 10.108.12.2 255.255.255.0 alt 10.108.12.3 255.255.255.0
alias 10.108.12.1 255.255.255.0
!
vlan 401 client
ip address 10.8.12.2 255.255.255.0 alt 10.8.12.2 255.255.255.0
gateway 10.8.12.254
real TEST_1
address 10.108.12.17
inservice
real TEST_2
address 10.108.12.18
inservice
!
serverfarm TEST_SERVERS
nat server
no nat client
real name TEST_1
inservice
real name TEST_2
inservice
probe HTTP
!
vserver TEST_HTTP
virtual 10.8.12.11 tcp www
serverfarm TEST_SERVERS
replicate csrp connection
persistent rebalance
inservice
Perhaps the secondary CLIENT VLAN requires its own default gateway. You need to consider that CSM always uses the lowest CLIENT VLAN ID to reach its configured gateway, so in case of having multiple gateways for different CLIENT VLANs you need to configure the following trick:
The first CSM VLAN pair has no problem with gateway reachability. But configuring a gateway for a secondary VLAN pair is not as easy as first one. Based on the above drawing to force CSM to use 10.8.12.254 as gateway for VLAN pair 401/402 you need to configure:
serverfarm VLAN402-out
no nat server
no nat client
real 10.8.12.254
inservice
vserver VLAN402-out
virtual 0.0.0.0 0.0.0.0 any
vlan 402
serverfarm VLAN402-out
inservice
If you need to access your servers from different VLANs without using the VIP address, this section is useful for you. You might find it impossible to connect to real servers. The following debug output demonstrate that something is wrong!
Failed LB : access-denied
For direct server access configure a separate serverfarm/vserver pair:
serverfarm ROUTE
no nat server
no nat client
predictor forward
!
vserver REAL_ACCESS
virtual 10.8.112.0 255.255.255.0 any
serverfarm ROUTE
persistent rebalance
inservice
dc1sw01#show module csm 3 reals
real server farm weight state conns/hits
---------------------------------------------------------
TEST_1 TEST_SERVERS 8 OPERATIONAL 0
TEST_2 TEST_SERVERS 8 OPERATIONAL 0
10.8.12.1 VLAN402-OUT 8 OPERATIONAL 1
dc1sw01#show module csm 3 vservers
vserver type prot virtual vlan state conns
--------------------------------------------------------------------
DIRECT_ACCESS SLB any 10.8.108.0/24:0 ALL OPERATIONAL 19
TEST_HTTP SLB TCP 10.8.12.11/32:80 ALL OPERATIONAL 0
VLAN402-OUT SLB any 0.0.0.0/0:0 402 OPERATIONAL 3
REAL_ACCESS SLB any 10.8.112.0/24:0 ALL OPERATIONAL 1
dc1sw01#show module csm 3 serverfarm
server farm type predictor nat reals redirect bind id
---------------------------------------------------------------
DIRECT_ACCESS SLB Forward S 0 0 0
TEST_SERVERS SLB RoundRobin S 2 0 0
VLAN402-OUT SLB RoundRobin none 1 0 0
ROUTE SLB Forward none 0 0 0
dc1sw01#show module csm 3 vlan
vlan IP address IP mask type
---------------------------------------------------
208 10.8.8.2 255.255.255.0 CLIENT
318 10.8.108.2 255.255.255.0 SERVER
401 10.8.12.2 255.255.255.0 CLIENT
402 10.8.112.2 255.255.252.0 SERVER
997 0.0.0.0 0.0.0.0 FT
dc1sw01#show module csm 3 arp
Internet Address Physical Interface VLAN Type Status
--------------------------------------------------------------------
10.8.108.1 00-01-64-F9-1A-01 318 -ALIAS- local
10.8.108.2 00-0F-34-2C-52-06 318 --SLB-- local
10.8.108.3 00-02-FC-E1-CE-60 318 LEARNED up(0 misses)
10.8.112.1 00-01-64-F9-1A-01 402 -ALIAS- local
10.8.112.2 00-0F-34-2C-52-06 402 --SLB-- local
10.8.112.3 00-02-FC-E1-CE-60 402 LEARNED up(0 misses)
10.8.112.17 00-50-56-AD-00-96 402 REAL up(0 misses)
10.8.112.18 00-50-56-AD-00-96 402 REAL up(0 misses)
10.8.8.254 00-C0-EA-EA-EA-EA 208 GATEWAY up(0 misses)
10.8.8.2 00-0F-34-2C-52-06 208 --SLB-- local
10.8.8.3 00-02-FC-E1-CE-60 208 LEARNED up(0 misses)
10.8.12.254 00-22-90-9D-D3-0F 401 GATEWAY up(0 misses)
10.8.12.2 00-0F-34-2C-52-06 401 --SLB-- local
10.8.12.3 00-02-FC-E1-CE-60 401 LEARNED up(0 misses)
10.8.12.11 00-01-64-F9-1A-01 0 VSERVER local
If you want to PING from a VIP address:
dc1sw01#ping module csm 3 10.8.12.254
IP address Reachable
--------------------------
10.8.12.254 Yes
dc1coresw01#ping module csm 3 gateways
IP address Reachable
--------------------------
10.8.8.254 Yes
10.8.12.254 Yes
dc1sw01#ping module csm 3 reals
IP address Reachable
--------------------------
10.8.108.10 Yes
10.8.108.11 Yes
10.8.112.17 Yes
10.8.112.18 Yes
10.8.12.254 Yes
Hope that helps!
In the pervious post, we went through the Uniform model and its configuration. In this post we are going to see the differences between the Uniform model and the Pipe model. As mentioned earlier in the Part one of MPLS Quality of Service, the Pipe model does not change customer marking (IP TOS – DSCP). So in the following example, we will see that the provider is able to change the EXP marking but will not change the packet marking in the end. Provider also performs QoS tasks (even at the egress LSR) based on EXP and MPLS QoS marking, not based on IP header properties (Short-pipe model is different in this case).
As depicted in the above picture, egress PE is configured to send an Explicit Null to the P router. In this case the egress PE will be able to see transport label EXP field as well as VPN label EXP field.
R1(config)#mpls ldp explicit-null
R0#show mpls forwarding-table
Local Outgoing Prefix Bytes tag Outgoing Next Hop
tag tag or VC or Tunnel Id switched interface
16 Pop tag 3.3.3.3/32 0 Et0/0 192.168.30.3
17 0 1.1.1.1/32 137 Et0/1 192.168.10.1
In this case, on R0 (the P router) we don’t need to copy EXP from topmost label (received from R3) to the topmost transport label sent to R1, because the Explicit Null (label 0) maintains the EXP bit. This is only one example to show QoS features. In this example, we like to perform our marking on the topmost label and not rely on the second label, to demonstrate explicit-null feature. The Uniform and the Pipe models difference is not in the ingress part but in the egress, to save or not to save the customer marking on the IP packet.
Ingress PE:
!
access-list 110 permit icmp host 5.5.5.5 host 6.6.6.6
!
class-map match-any in1
match access-group 110
class-map match-any out1
match qos-group 7
!
policy-map in1
class in1
set mpls experimental imposition 3
set qos-group 7
policy-map out1
class out1
set mpls experimental topmost 7
!
R3#sh policy-map interface e0/0
Ethernet0/0
Service-policy output: out1
Class-map: out1 (match-any)
10 packets, 1220 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: qos-group 7
10 packets, 1220 bytes
5 minute rate 0 bps
QoS Set
mpls experimental topmost 7
Packets marked 10
Class-map: class-default (match-any)
28 packets, 11442 bytes
5 minute offered rate 3000 bps, drop rate 0 bps
Match: any
P:
no configuration!
Egress PE:
!
class-map match-all in1
match mpls experimental topmost 7
class-map match-all out1
match qos-group 7
!
!
policy-map in1
class in1
set qos-group 7
policy-map out1
class out1
priority 256
class class-default
shape average 500000
!
R1#sh policy-map interface e0/1
Ethernet0/1
Service-policy output: out1
Class-map: out1 (match-all)
10 packets, 1140 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: qos-group 7
Queueing
Strict Priority
Output Queue: Conversation 264
Bandwidth 256 (kbps) Burst 6400 (Bytes)
(pkts matched/bytes matched) 0/0
(total drops/bytes drops) 0/0
Class-map: class-default (match-any)
20 packets, 2080 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: any
Traffic Shaping
Why there is no configuration on P LSR? Because IOS by default maintains your EXP and as long as PE LSR requesting Explicit Null, the topmost label is kept and contains EXP value of incoming labeled packet. (there’s no label disposition on the P LSR – R0)
The Quality of Service (QoS) has become more popular than past few years, because of high-bandwidth and delay sensitive applications. QoS is prioritizing more important data over less important traffic. There are two ways to implement QoS in an IP network: IntServ and DiffServ. Internet is operating based on best-effort model.
As the DiffServ model does not require a signaling protocol such as RSVP, it’s more popular or better to say the only applicable model in today's networks. (Although we use RSVP in MPLS TE for path signaling and reservation).
IETF in RFC 3270 has recommended three QoS models for DiffServ tunneled traffic in MPLS networks:
The Pipe and Short Pipe models are almost the same: They do not change IP ToS of customers data at all. They might change the EXP field in the path (EXP of MPLS Label) but the IP header QoS field remains unchanged. The Pipe model performs forwarding/discarding/scheduling based on EXP at the egress LSR while the Short Pipe model does that based on IP ToS. Because in the Short pipe model there might be no label at all. (remember PHP operation – Penultimate-Hop Popping)
In the Uniform model, there’s no guarantee that ToS of customers packet remains intact, but the EXP and IP ToS fields of a data packet will always show the same thing. It means if the provider changes the EXP header, that will be copied later to the IP ToS field at the egress point.
IOS Default Behavior: In short IOS does not change your QoS in the path if you don’t change it.
Imposition - Copy TOS bits to EXP (TOS Reflection) when adding one or more labels by ingress LSR.
Swap - Copy EXP from old label to new label.
Disposition - Do NOT Copy EXP from label to IP.
Label to Label (on PHP) sets EXP 5 (if top is 5) after disposing the top label:
class-map match-all in1
match mpls experimental topmost 5
class-map match-all out1
match qos-group 5
!
policy-map in1
class in1
set qos-group mpls experimental topmost
policy-map out1
class out1
set mpls experimental topmost 5
!
interface Ethernet0/0
service-policy input in1
!
interface Ethernet0/1
service-policy output out1
!
Label to IP (on PE) sets precedence 5 (if top is 5) after popping the label:
class-map match-all in1
match mpls experimental topmost 5
class-map match-all out1
match qos-group 5
!
policy-map in1
class in1
set qos-group mpls experimental topmost
policy-map out1
class out1
set ip precedence 5
!
interface Ethernet0/0
service-policy input in1
!
interface Ethernet0/1
service-policy output out1
In this sceniaro based on the service provider’s policy, the EXP value is changed in the path and is copied to inner label by P router. At the egress LSR, the EXP value is copied to the DSCP. (Uniform model)
Ingress PE:
!
policy-map out1
class class-default
set mpls experimental topmost 7
policy-map in1
class class-default
set mpls experimental imposition 3
!
P:
!
class-map match-any in1
match mpls experimental topmost 0 1 2 3 4 5 6 7
class-map match-any out1
match qos-group 0
match qos-group 1
match qos-group 2
match qos-group 3
match qos-group 4
match qos-group 5
match qos-group 6
match qos-group 7
!
policy-map in1
class in1
set qos-group mpls experimental topmost
policy-map out1
class out1
set mpls experimental topmost qos-group
!
Egress PE:
!
class-map match-any in1
match mpls experimental topmost 0 1 2 3 4 5 6 7
class-map match-any out1
match qos-group 0
match qos-group 1
match qos-group 2
match qos-group 3
match qos-group 4
match qos-group 5
match qos-group 6
match qos-group 7
!
policy-map in1
class in1
set qos-group mpls experimental topmost
policy-map out1
class out1
set precedence qos-group
!
After publishing part 1 and part 2 of my BGP Multipath Trilogy! One of our good readers sent a very intelligent question here:
cc said: Very nice article Shafagh,
Any difference in any of these examples if R0 were both a P and a Route Reflector?
http://www.shafagh.net/2010/09/bgp-multipath-part-two.html?showComment=1283780646233#c5196111639519265964
There’s a known limitation:
Route Reflector Limitation
When multiple iBGP paths installed in a routing table, a route reflector will advertise only one paths (next hop). If a router is behind a route reflector, all routers that are connected to multihomed sites will not be advertised unless a different route distinguisher is configured for each VRF.
http://www.cisco.com/en/US/docs/ios/12_2t/12_2t11/feature/guide/ft11bmpl.html#wp1027175
But like everything else in the computer world, there’s always a workaround, I found a way to make each path a “unique path” (playing with 64 bit prefix - RD) to send prefixes to ingress PE and let PE import different RDs inside the routing table. Here we go:
So in this example, R0 and R1 are using different RD for the same CE (multihomed CE – One or two routers at customer site) This makes two different vpnv4 prefixes pointing to the same ipv4 subnet.
The workaround is to advertise both or several parallel paths from different PEs with different RDs so that RRs advertise all paths. The RR will advertise all parallel paths with different RDs (as they are not same) and ingress PE can run BGP multipath and use all parallel paths.
The route reflector will not filter anything. It sends the best path to the R3. As long as these two prefixes are different they are both best path for their corresponding RDs:
R2#sh ip bgp vpn all
BGP table version is 4, local router ID is 2.2.2.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:1
*>i69.69.69.69/32 1.1.1.1 0 100 0 ?
Route Distinguisher: 1:2
*>i69.69.69.69/32 10.10.10.10 0 100 0 ?
R3 needs this command:
R3(config-router)#address-family ipv4 vrf C1
R3(config-router-af)#maximum-paths ibgp 2 import 1
R3(config-router-af)#^Z
R3#sh ip route vrf C1
Routing Table: C1
Gateway of last resort is not set
69.0.0.0/32 is subnetted, 2 subnets
B 69.69.69.69 [200/0] via 10.10.10.10, 00:01:30
[200/0] via 1.1.1.1, 00:01:22
In the previous post, we reviewed basics of iBGP and eBGP multipath. In this post we get introduced with import feature of MP-BGP multipath. When there’s a difference in VRF’s RD, we have to use import keyword to import parallel paths from one VRF into another.
For example, we have the same diagram as BGP Multipath Part One, the only difference is the RD number that is used by R1. The Customer VRF RD at R2,3 and 4 is 1:1, while on the other side RD is 1:2.
When the RD is different, we need additional “import” keyword to tell the router how many parallel routes import from one VRF into another. If RD number is same on both side (both PE devices) previous post and example is enough for you.
Let’s check the configuration without any “maximum-path” command, to see the BGP natural behavior when there are parallel paths. (We know that BGP only selects one)
R1(config-router-af)#do sh ip bgp vpnv4 all
BGP table version is 281, local router ID is 1.1.1.1
Status codes: * valid, > best, i - internal,
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:1
* i5.5.5.5/32 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i55.55.55.0/24 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i192.168.25.0 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i192.168.35.0 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i192.168.45.0 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
Route Distinguisher: 1:2 (default for vrf C1)
Network Next Hop Metric LocPrf Weight Path
* i5.5.5.5/32 3.3.3.3 10 100 0 1 ?
*> 6.6.6.6/32 192.168.16.6 0 0 2 ?
*>i55.55.55.0/24 2.2.2.2 10 100 0 1 ?
r> 192.168.16.0 192.168.16.6 0 0 2 ?
*>i192.168.25.0 2.2.2.2 10 100 0 1 ?
*>i192.168.35.0 2.2.2.2 10 100 0 1 ?
*>i192.168.45.0 2.2.2.2 10 100 0 1 ?
R1(config-router-af)#do sh ip route vrf C1
Routing Table: C1
B 192.168.45.0/24 [200/10] via 2.2.2.2, 00:00:17
B 192.168.25.0/24 [200/10] via 2.2.2.2, 00:00:17
55.0.0.0/24 is subnetted, 1 subnets
B 55.55.55.0 [200/10] via 2.2.2.2, 00:00:17
5.0.0.0/32 is subnetted, 1 subnets
B 5.5.5.5 [200/10] via 2.2.2.2, 00:00:17
6.0.0.0/32 is subnetted, 1 subnets
B 6.6.6.6 [20/0] via 192.168.16.6, 07:53:26
B 192.168.35.0/24 [200/10] via 2.2.2.2, 00:00:17
C 192.168.16.0/24 is directly connected, Ethernet0/1
As you can see, there’s only one path (per prefix) at customer VRF. So let’s add “maximum-path ibgp” command for Customer VRF:
R1(config-router)#address-family ipv4 vrf C1
R1(config-router-af)#maximum-paths ibgp 3
R1(config-router-af)#
R1(config-router-af)#do sh ip bgp vpnv4 all
BGP table version is 288, local router ID is 1.1.1.1
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:1
* i5.5.5.5/32 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i55.55.55.0/24 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i192.168.25.0 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i192.168.35.0 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i192.168.45.0 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
Route Distinguisher: 1:2 (default for vrf C1)
Network Next Hop Metric LocPrf Weight Path
*>i5.5.5.5/32 2.2.2.2 10 100 0 1 ?
*> 6.6.6.6/32 192.168.16.6 0 0 2 ?
*>i55.55.55.0/24 2.2.2.2 10 100 0 1 ?
r> 192.168.16.0 192.168.16.6 0 0 2 ?
*>i192.168.25.0 2.2.2.2 10 100 0 1 ?
*>i192.168.35.0 2.2.2.2 10 100 0 1 ?
*>i192.168.45.0 2.2.2.2 10 100 0 1 ?
So the “maximum-path ibgp” only indicates how many routes from RIB (VRF’s BGP table) can be imported to the routing table. The problem is that the RIB - RD table for 1:2 (VRF C1) has only path, so maximum-path ibgp has no parallel path to import into the routing table.
The import keyword allows you to configure the VRF table to accept multiple redundant paths in addition to the best path. By default, a VRF will import only one path (best path) per prefix from the source VRF table. If the best path goes down, the destination will not be reachable until the next import event occurs, and then a new best path will be imported into the VRF table. The import event runs every 15 seconds by default.
Note: Configuring redundant paths with the import keyword can increase CPU and memory utilization significantly, especially in a network where there are many prefixes to learn and a large number of configured VRFs. It is recommended that this feature is only configured as necessary and that the minimum number of redundant paths are configured (Typically, not more than two).
R1(config-router-af)#maximum-paths ibgp 3 import 3
R1(config-router-af)#do sh ip bgp vpnv4 rd 1:2
BGP table version is 269, local router ID is 1.1.1.1
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:2 (default for vrf C1)
* i5.5.5.5/32 3.3.3.3 10 100 0 1 ?
* i 4.4.4.4 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
*> 6.6.6.6/32 192.168.16.6 0 0 2 ?
* i55.55.55.0/24 3.3.3.3 10 100 0 1 ?
* i 4.4.4.4 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
r> 192.168.16.0 192.168.16.6 0 0 2 ?
* i192.168.25.0 3.3.3.3 10 100 0 1 ?
* i 4.4.4.4 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i192.168.35.0 3.3.3.3 10 100 0 1 ?
* i 4.4.4.4 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i192.168.45.0 3.3.3.3 10 100 0 1 ?
* i 4.4.4.4 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
R1(config-router-af)#do sh ip route vrf C1
Routing Table: C1
B 192.168.45.0/24 [200/10] via 4.4.4.4, 00:04:05
[200/10] via 3.3.3.3, 00:00:35
[200/10] via 2.2.2.2, 00:04:05
B 192.168.25.0/24 [200/10] via 4.4.4.4, 00:04:05
[200/10] via 3.3.3.3, 00:00:35
[200/10] via 2.2.2.2, 00:04:05
55.0.0.0/24 is subnetted, 1 subnets
B 55.55.55.0 [200/10] via 4.4.4.4, 00:04:05
[200/10] via 3.3.3.3, 00:00:35
[200/10] via 2.2.2.2, 00:04:05
5.0.0.0/32 is subnetted, 1 subnets
B 5.5.5.5 [200/10] via 4.4.4.4, 00:04:06
[200/10] via 3.3.3.3, 00:00:36
[200/10] via 2.2.2.2, 00:04:06
6.0.0.0/32 is subnetted, 1 subnets
B 6.6.6.6 [20/0] via 192.168.16.6, 07:49:57
B 192.168.35.0/24 [200/10] via 4.4.4.4, 00:04:06
[200/10] via 3.3.3.3, 00:00:36
[200/10] via 2.2.2.2, 00:04:06
C 192.168.16.0/24 is directly connected, Ethernet0/1
Now, let’s play with the variables and see the different results:
R1(config-router-af)#maximum-paths ibgp 2 import 3
R1(config-router-af)#do sh ip bgp vpnv4 rd 1:1
BGP table version is 222, local router ID is 1.1.1.1
R1(config-router-af)#do sh ip bgp vpnv4 rd 1:2
BGP table version is 222, local router ID is 1.1.1.1
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:2 (default for vrf C1)
* i5.5.5.5/32 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
*> 6.6.6.6/32 192.168.16.6 0 0 2 ?
* i55.55.55.0/24 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
r> 192.168.16.0 192.168.16.6 0 0 2 ?
* i192.168.25.0 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i192.168.35.0 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i192.168.45.0 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
R1(config-router-af)#do sh ip route vrf C1
Routing Table: C1
B 192.168.45.0/24 [200/10] via 4.4.4.4, 00:01:09
[200/10] via 2.2.2.2, 00:01:09
B 192.168.25.0/24 [200/10] via 4.4.4.4, 00:01:10
[200/10] via 2.2.2.2, 00:01:10
55.0.0.0/24 is subnetted, 1 subnets
B 55.55.55.0 [200/10] via 4.4.4.4, 00:01:10
[200/10] via 2.2.2.2, 00:01:10
5.0.0.0/32 is subnetted, 1 subnets
B 5.5.5.5 [200/10] via 4.4.4.4, 00:01:10
[200/10] via 2.2.2.2, 00:01:10
6.0.0.0/32 is subnetted, 1 subnets
B 6.6.6.6 [20/0] via 192.168.16.6, 07:41:14
B 192.168.35.0/24 [200/10] via 4.4.4.4, 00:01:27
[200/10] via 2.2.2.2, 00:01:27
C 192.168.16.0/24 is directly connected, Ethernet0/1
R1(config-router-af)#maximum-paths ibgp 3 import 2
R1(config-router-af)#do sh ip bgp vpnv4 rd 1:1
BGP table version is 264, local router ID is 1.1.1.1
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:1
* i5.5.5.5/32 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i55.55.55.0/24 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i192.168.25.0 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i192.168.35.0 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
* i192.168.45.0 4.4.4.4 10 100 0 1 ?
* i 3.3.3.3 10 100 0 1 ?
*>i 2.2.2.2 10 100 0 1 ?
R1(config-router-af)#do sh ip bgp vpnv4 rd 1:2
R1(config-router-af)#do sh ip route vrf C1
Routing Table: C1
B 192.168.45.0/24 [200/10] via 4.4.4.4, 00:00:22
[200/10] via 2.2.2.2, 00:00:22
B 192.168.25.0/24 [200/10] via 4.4.4.4, 00:00:22
[200/10] via 2.2.2.2, 00:00:22
55.0.0.0/24 is subnetted, 1 subnets
B 55.55.55.0 [200/10] via 4.4.4.4, 00:00:22
[200/10] via 2.2.2.2, 00:00:22
5.0.0.0/32 is subnetted, 1 subnets
B 5.5.5.5 [200/10] via 4.4.4.4, 00:00:22
[200/10] via 2.2.2.2, 00:00:22
6.0.0.0/32 is subnetted, 1 subnets
B 6.6.6.6 [20/0] via 192.168.16.6, 07:46:21
B 192.168.35.0/24 [200/10] via 4.4.4.4, 00:00:31
[200/10] via 2.2.2.2, 00:00:31
C 192.168.16.0/24 is directly connected, Ethernet0/1
!
router bgp 100
no synchronization
bgp log-neighbor-changes
neighbor 2.2.2.2 remote-as 100
neighbor 2.2.2.2 update-source Loopback0
neighbor 3.3.3.3 remote-as 100
neighbor 3.3.3.3 update-source Loopback0
neighbor 4.4.4.4 remote-as 100
neighbor 4.4.4.4 update-source Loopback0
no auto-summary
!
address-family vpnv4
neighbor 2.2.2.2 activate
neighbor 2.2.2.2 send-community extended
neighbor 3.3.3.3 activate
neighbor 3.3.3.3 send-community extended
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community extended
exit-address-family
!
address-family ipv4 vrf C1
neighbor 192.168.16.6 remote-as 2
neighbor 192.168.16.6 activate
maximum-paths ibgp 3 import 3
no auto-summary
no synchronization
exit-address-family
!
Posts
All Comments
