August 4, 2011

Transmit-Interface Command

To assign a transmit interface to a receive-only interface, use the transmit-interface command in interface configuration mode


The following example specifies Ethernet interface 0 as a simplex Ethernet interface:

interface ethernet 1 
 ip address 128.9.1.2 
 transmit-interface ethernet 0


Cisco IOS IP Command Reference, Volume 1 of 4: Addressing and Services, Release 12.3 - IP Addressing and Services Commands: T through W [Cisco IOS Software Releases 12.3 Mainline] - Cisco Systems

July 10, 2011

Virtual Switching System

 

VSS Overview

The Cisco VSS simplifies network configuration and operation by providing a loop-free Layer 2 topology using two Catalyst 6500 switches acting as one big Virtual switch. VSS reduces number of Layer 3 routing neighbors by providing a Layer 2 connectivity for access/distribution switches.

VSS Simple Drawing

A VSS is a pair of combined 6500 switches acting as a single network element with redundancy and load balancing over port-channels (etherchannels). One switch becomes the master or active chassis and the other one becomes the VSS standby.

 

VSS Switch Roles

  • VSS Active: The active chassis controls the VSS operation. It runs the control plane, L2 and L3 control protocols. It also runs the management plane functions like console interface, logs, file system and even power management.
  • VSS Standby: Listens to master, checks the status, forwards the ingress traffic but sends all control traffic to the VSS active chassis for processing.

Virtual Switch Link

To share control and data traffic between two chassis a VSL – Virtual Switch Link is required. VSL is implemented as a Port Channel. The control traffic gets higher priority over data on a VSL and never gets discarded.

 

before configuration make sure that you have right IOS image, proper VS-capable supervisor and line cards, you can check your line cards with the following command:

6500A#switch convert check vss-capable
This is a VSS capable switch.
VSL ports can be configured in slot: 4, 5, 6

 

VSS Configuration

By default, 6500 series switches are configured to operate in standalone mode. The following steps are required in order to setup virtual stack across two 6500 chassis.

Step1: Make sure SSO and NSF are configured and enabled

Step2: Assign Virtual Switch Doman and Switch numbers

Step3: Configure VSL Port Channel, PO# should be unique on each chassis.

Step4: Convert Standalone Chassis to Virtual Switch mode

VSS1

Step1:

6500A(config)#redundancy
6500A(config-red)#mode sso
!

Step2:

6500A(config)#switch virtual domain ?
  <1-255>  Virtual switch domain number

6500A(config)#switch virtual domain 100
Domain ID 100 config will take effect only
after the exec command 'switch convert mode virtual' is issued

6500A(config-vs-domain)#switch 1
6500A(config-vs-domain)#exit

...

6500B(config)#switch virtual domain 100
Domain ID 100 config will take effect only
after the exec command 'switch convert mode virtual' is issued

6500B(config-vs-domain)#switch 2
6500B(config-vs-domain)#exit

Step3:

6500A(config)#int port-channel 100
6500A(config-if)#
switch virtual link 1
6500A(config-if)#no shutdown
6500A(config-if)#exit
6500A(config)#int te5/4
6500A(config-if)#channel-group 100 mode on
6500A(config-if)#no shutdown
6500A(config)#int te6/4
6500A(config-if)#channel-group 100 mode on
6500A(config-if)#no shutdown

...

6500B(config)#int port-channel 200
6500B(config-if)#switch virtual link 2
6500B(config-if)#no shutdown
6500B(config-if)#exit
6500B(config)#int te5/4
6500B(config-if)#channel-group 200 mode on
6500B(config-if)#no shutdown
6500B(config)#int te6/4
6500B(config-if)#channel-group 200 mode on
6500B(config-if)#no shutdown

Step4:

6500A#switch convert ?
  check  check if this switch and its modules are VSS capable or not
  mode   mode keyword virtual or standalone

6500A#switch convert mode ?
  stand-alone  stand-alone switch
  virtual      virtual switch

6500A#switch convert mode virtual

This command will convert all interface names
to naming convention "interface-type switch-number/slot/port",
save the running config to startup-config and
reload the switch.

NOTE: Make sure to configure one or more dual-active detection methods
once the conversion is complete and the switches have come up in VSS mode.

Do you want to proceed? [yes/no]: yes
Converting interface names
Building configuration...
Saving converted configuration to bootflash: ...
Destination filename [startup-config.converted_vs-20110705-214318]?

*** --- SHUTDOWN NOW ---

 

Ensure that same PFC operating mode is being used on both chassis in order to have a proper SSO redundancy mode:

6500A#show platform hardware pfc mode
PFC operating mode : PFC3C

Now, Let’s look at configuration of VSS switch after reboot:

6500A#sh run

!
hostname 6500A
!        
switch virtual domain 100
switch mode virtual
!        
mls netflow interface
mls cef error action reset
!
spanning-tree mode pvst
spanning-tree extend system-id
diagnostic bootup level minimal
!        
redundancy
main-cpu
  auto-sync running-config
mode sso
!
vlan internal allocation policy ascending
vlan access-log ratelimit 2000
!
!
interface Port-channel100
no switchport
no ip address
switch virtual link 1
mls qos trust cos
no mls qos channel-consistency
!
interface Port-channel200
no switchport
no ip address
switch virtual link 2
mls qos trust cos
no mls qos channel-consistency
!        
interface GigabitEthernet1/1/1
no switchport
no ip address
shutdown
!        
interface GigabitEthernet1/1/2
no switchport
no ip address
shutdown
...
...
interface TenGigabitEthernet1/5/4
no switchport
no ip address
mls qos trust cos
channel-group 100 mode on
...
...
interface TenGigabitEthernet1/6/4
no switchport
no ip address
mls qos trust cos
channel-group 100 mode on
...
...
interface TenGigabitEthernet2/5/4
no switchport
no ip address
mls qos trust cos
channel-group 200 mode on
...
...
interface TenGigabitEthernet2/6/4
no switchport
no ip address
mls qos trust cos
channel-group 200 mode on
...
...
interface Vlan1
no ip address
shutdown
!
ip classless
ip forward-protocol nd
!
control-plane
!
line con 0
line vty 0 4
login
!
mac-address-table aging-time 480
no event manager policy Mandatory.go_switchbus.tcl type system    
!
module provision switch 1
slot 1 slot-type 147 port-type 61 number 48  virtual-slot 17
slot 3 slot-type 152 port-type 31 number 48  virtual-slot 19
slot 4 slot-type 227 port-type 60 number 8  virtual-slot 20
slot 5 slot-type 254 port-type 31 number 2 port-type 61 number 1 port-type 60 number 2  virtual-slot 21
slot 6 slot-type 254 port-type 31 number 2 port-type 61 number 1 port-type 60 number 2  virtual-slot 22
!
module provision switch 2
slot 1 slot-type 147 port-type 61 number 48  virtual-slot 33
slot 3 slot-type 152 port-type 31 number 48  virtual-slot 35
slot 4 slot-type 227 port-type 60 number 8  virtual-slot 36
slot 5 slot-type 254 port-type 31 number 2 port-type 61 number 1 port-type 60 number 2  virtual-slot 37
slot 6 slot-type 254 port-type 31 number 2 port-type 61 number 1 port-type 60 number 2  virtual-slot 38
!
end

Verification:

Verification is the most important part of configuration, you need to make sure that commands that you have entered are working as expected!

6500A#show switch virtual
Switch mode                  : Virtual Switch
Virtual switch domain number : 100
Local switch number          : 1
Local switch operational role: Virtual Switch Active
Peer switch number           : 2
Peer switch operational role : Virtual Switch Standby

6500A#show switch virtual role

Switch  Switch Status  Preempt    Priority  Role     Session ID
        Number         Oper(Conf) Oper(Conf)         Local  Remote
------------------------------------------------------------------
LOCAL    1     UP      FALSE(N )   100(100)  ACTIVE   0      0  
REMOTE   2     UP      FALSE(N )   100(100)  STANDBY  4004   1462

In dual-active recovery mode: No

6500A#show switch virtual link
VSL Status : UP
VSL Uptime : 43 minutes
VSL SCP Ping : Pass
VSL ICC Ping : Pass
VSL Control Link : Te1/5/4 

6500A#show switch virtual link port-channel
Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        H - Hot-standby (LACP only)
        R - Layer3      S - Layer2
        U - in use      N - not in use, no aggregation
        f - failed to allocate aggregator

        M - not in use, no aggregation due to minimum links not met
        m - not in use, port not aggregated due to minimum links not met
        u - unsuitable for bundling
        d - default port

        w - waiting to be aggregated
 
Group  Port-channel  Protocol    Ports
------+-------------+-----------+-------------------
100    Po100(RU)        -        Te1/5/4(P)     Te1/6/4(P)    
200    Po200(RU)        -        Te2/5/4(P)     Te2/6/4(P)

If I connect my console to secondary chassis:

6500A-sdby>
Standby console disabled

 

Multi-Chassis EtherChannel

One VSS supports a maximum of 512 - 2 port channels (deducting 2 POs for VSL) Configuration of MEC is not different than any regular etherchannel! The Different is physical connectivity, one link connects to VSS Active and the other link connects to VSS Standby chassis creating a high bandwidth active/active aggregated link-bundle (Etherchannel)

VSS2

Access switch:

interface GigabitEthernet1/1/1
channel-group 10 mode on
!
interface GigabitEthernet1/1/2
channel-group 10 mode on

Core switch:

interface Port-channel10
switchport
!
interface GigabitEthernet1/3/17
switchport
channel-group 10 mode on
!
interface GigabitEthernet2/3/17
switchport
channel-group 10 mode on

Verify:

6500A#sh etherchannel summary
Number of channel-groups in use: 3
Number of aggregators:           3

Group  Port-channel  Protocol    Ports
------+-------------+-----------+--------------------------------------
10     Po10(SU)         -        Gi1/3/17(P)    Gi2/3/17(P)   
100    Po100(RU)        -        Te1/5/4(P)     Te1/6/4(P)    
200    Po200(RU)        -        Te2/5/4(P)     Te2/6/4(P)
    

 

Switch12#sh etherchannel summary
Number of channel-groups in use: 1
Number of aggregators:           1

Group  Port-channel  Protocol    Ports
------+-------------+-----------+--------------------------------------
10     Po10(SU)         -        Gi1/1/1(P)  Gi1/1/2(P)

April 24, 2011

Service Provider General Tasks

As you may now, CCIE Service Provider has been updated to new version (V.3) as of April 18 2011. There are several changes in the exam which are very important, ATM seems to be removed and IOS XR has a major role along with more IPv6 plus extended troubleshooting section. I went through my preparation notes and found best practices note which is general rule of thumb for CCIE SP lab.

CiscoSP

General Best Practices for exam:

  • Configure ISIS or OSPF backbone in each AS (Loopback reach-ability)

configure loopback0 as router-id
configure metric parameters (e.g. auto-cost reference-bandwidth / metric-style wide)
configure network types / is-types
configure timers or protocol specific features, DR priority and so on.

  • Configure BGP in each AS (to advertise loopbacks to other AS)

configure "no bgp default ipv4-unicast"
configure neighbors remote-as
configure update-source as loopback0
configure address-family ipv4 and activate neighbors (and RR if necessary)
configure next-hop-self on border-routers (if necessary)
advertise loopback address with the network command

  • Configure eBGP neighbors

make sure you have advertised loopback addresses
make sure next-hop is propagated correctly
set "ip bgp-community new-format"
send bgp communities to required neighbors
configure routing policies/controls based on community lists

  • Configure MPLS Tagging

configure label protocol
configure loopback as ldp session source - router-id
configure label options (numbers, session protection, etc)
configure mpls interfaces

  • Configure MPLS TE

configure routers for MPLS TE
configure OSPF/ISIS for MPLS TE 
configure routing area for MPLS TE (or Level1/2)
configure interfaces for TE and RSVP
configure tunnel interfaces
configure static or dynamic paths

  • Configure MPLS VPN

configure VRFs and try to import RTs
configure MP-IBGP
configure MP-eBGP
configure send-label in BGP (if required)
configure next-hop-unchanged (between MP-eBGP points if necessary)

  • Configure Route control in MPLS VPN

set BGP attributes to control routes. e.g:
send MED to MP-BGP CE neighbor (to make it less preferred)
set Weight for a neighbor in different AS to be preferred

 

For a detailed Lab exam blue print you can refer to:
https://learningnetwork.cisco.com/docs/DOC-10145

April 12, 2011

CSM Basics and FT

Cisco Content Switching Module adds layer 4 to layer 7 content switching capabilities to the Catalyst 6500/7600 Series providing high-performance load balancing for servers, firewalls or even NAC boxes! Cisco CSM is old and not a new product, it has been replaced by newer Cisco ACE load-balancers. Cisco ACE is being sold in two different fashions, as Blade modules just like CSM or as 4710 appliances just like CSS appliance-based load-balancers.

 

ACE supports virtualization (multiple-contexts and resource allocation) while CSM is monolithic. CSM uses IOS configuration file and stores its configuration in the running-config. It’s really easy and straight forward to configure if you know what you’re doing!

ImagejMObcW8Adv_full

It’s been awhile since the last time that I touched CSM as it’s end of sales and support. EOL/EOS reference link! But you might have some customers that are still using this product and they might need some levels of technical support. Last week, a call came through asking for CSM support, the customer has four CSM blades inside 6509s in redundant fashion in main and DR site…

 

This post quickly reviews the following topics:

  • CSM Basic setup
  • Fault tolerant
  • Multiple set of Client/Server VLAN pairs.
  • Dual / Multiple Gateways
  • Direct Server Access
  • CSM Verification with show commands
  • How to ping from CSM VIP address

 

Basic CSM Setup

The most basic and common method of setting up a load-balancer is to have a VLAN pair; one facing servers and the other VLAN facing clients (facing firewall/routers). Then the Load-balancer sits between client and server and creates a VIP (virtual ip address). That VIP is related to several real servers but all are seen as one entity or one VIP. That VIP is reflected by a NAT statement further on a firewall to a public address and is ready for being resolved by DNS.

Client request comes through client VLAN of load-balancer and hits the VIP address then it gets distributed among available real servers.

 

The first step is to create CLIENT and SERVER VLAN pair:

1. Create two VLANs on 6500 catalyst but do not assign an ip address.

2. Create same VLANs on the CSM and assign IP address inside the CSM:

 

module ContentSwitchingModule 3

vlan 302 server
  ip address 10.8.108.2 255.255.255.0
!
vlan 301 client
  ip address 10.8.8.2 255.255.255.0
  gateway 10.8.8.254

 

Gateway is required for CSM to talk to clients (it might be your DMZ firewall interface)

The next step is to create server-farm (real-servers sit there) and vserver (virtual server – VIP)

probe HTTP http
  recover 3
  request method head
  expect status 200
  interval 2
  failed 6
  port 80
!
serverfarm HTTP_2010
  nat server
  no nat client
  real 10.8.108.17
   inservice
  real 10.8.108.18
   inservice
  probe HTTP
!
vserver HTTP_2010
  virtual 10.8.8.201 tcp www
  serverfarm HTTP_2010
  replicate csrp connection
  persistent rebalance
  inservice

 

The first part of above configuration is probe config. It checks the real servers health and sees whether HTTP code 200 is returned by each HTTP server. If probe to a server fails, that server will not be used.

The Serverfarm section configures two real servers (IP address is from SERVER VLAN) and uses HTTP probe for server health check. The vserver section, defines the VIP and correlates the server farm to vserver. Easy! now let’s verify:

dc1sw01#sh module csm 3 vlan

vlan   IP address       IP mask          type     
---------------------------------------------------
301    10.8.8.2         255.255.255.0    CLIENT
302    10.8.108.2       255.255.255.0    SERVER
401    10.8.12.2        255.255.255.0    CLIENT
402    10.8.112.2       255.255.252.0    SERVER
997    0.0.0.0          0.0.0.0          FT

 

CSM Fault-tolerant configuration

CSM FT configuration is pretty straight-forward. FT is configured when you have two CSM blades located in two 6500 catalysts to work as a fail-over group. You can also create a fault-tolerant configuration with two CSMs in a single Catalyst 6500 series chassis.

First step to create a VLAN on both catalysts specifically for FT and the configure priority on each CSM (master must have higher priority):

 

vlan 997
name CSM_Failover
!
module ContentSwitchingModule 3
ft group 1 vlan 997
  priority 110
  preempt
!

 

Then you need to configure your VLAN-pair to include alternate CSM IP address to be seen as single entity by servers to be used as the default gateway (the alias command):

vlan 302 server
  ip address 10.8.108.2 255.255.255.0 alt 10.8.108.3 255.255.255.0
  alias 10.8.108.1 255.255.255.0
!
vlan 301 client
  ip address 10.8.8.2 255.255.255.0 alt 10.8.8.3 255.255.255.0
  gateway 10.8.8.254

 

Secondary Set of VLAN-pair

What if we need to have more than one CLIENT/SERVER VLAN-pair?

Usually the client side is connected to a firewall (Cisco FWSM, ASA or something else). You can have different client/server VLAN pairs while firewall serves each client VLAN as a different DMZ zone.

You can also use this approach during server migration between DMZs!  or even during DMZ migration…

CSM

!        
vlan 402 server
  ip address 10.108.12.2 255.255.255.0 alt 10.108.12.3 255.255.255.0
  alias 10.108.12.1 255.255.255.0
!        
vlan 401 client
  ip address 10.8.12.2 255.255.255.0 alt 10.8.12.2 255.255.255.0
  gateway 10.8.12.254

real TEST_1
  address 10.108.12.17
  inservice
real TEST_2
  address 10.108.12.18
  inservice
!
serverfarm TEST_SERVERS
  nat server
  no nat client
  real name TEST_1
   inservice
  real name TEST_2
   inservice
  probe HTTP
!
vserver TEST_HTTP
  virtual 10.8.12.11 tcp www
  serverfarm TEST_SERVERS
  replicate csrp connection
  persistent rebalance
  inservice

 

Perhaps the secondary CLIENT VLAN requires its own default gateway. You need to consider that CSM always uses the lowest CLIENT VLAN ID to reach its configured gateway, so in case of having multiple gateways for different CLIENT VLANs you need to configure the following trick:

 

Dual Gateway

The first CSM VLAN pair has no problem with gateway reachability. But configuring a gateway for a secondary VLAN pair is not as easy as first one. Based on the above drawing to force CSM to use 10.8.12.254 as gateway for VLAN pair 401/402 you need to configure:

serverfarm VLAN402-out
no nat server
no nat client
real 10.8.12.254
inservice

vserver VLAN402-out
virtual 0.0.0.0 0.0.0.0 any
vlan 402
serverfarm VLAN402-out
inservice

 

Direct Server Access

If you need to access your servers from different VLANs without using the VIP address, this section is useful for you. You might find it impossible to connect to real servers. The following debug output demonstrate that something is wrong!

Failed LB : access-denied

For direct server access configure a separate serverfarm/vserver pair:

serverfarm ROUTE
  no nat server
  no nat client
  predictor forward
!   
vserver REAL_ACCESS
  virtual 10.8.112.0 255.255.255.0 any
  serverfarm ROUTE
  persistent rebalance
  inservice

 

Verification

dc1sw01#show module csm 3 reals

real        server farm    weight  state       conns/hits
---------------------------------------------------------   
TEST_1      TEST_SERVERS   8       OPERATIONAL 0       
TEST_2      TEST_SERVERS   8       OPERATIONAL 0       
10.8.12.1   VLAN402-OUT    8       OPERATIONAL 1  

 


dc1sw01#show module csm 3 vservers

vserver        type  prot virtual            vlan state        conns
--------------------------------------------------------------------
DIRECT_ACCESS  SLB   any  10.8.108.0/24:0    ALL  OPERATIONAL  19     
TEST_HTTP      SLB   TCP  10.8.12.11/32:80   ALL  OPERATIONAL  0      
VLAN402-OUT    SLB   any  0.0.0.0/0:0        402  OPERATIONAL  3      
REAL_ACCESS    SLB   any  10.8.112.0/24:0    ALL  OPERATIONAL  1
     

 

dc1sw01#show module csm 3 serverfarm

server farm     type  predictor   nat   reals  redirect bind id
---------------------------------------------------------------
DIRECT_ACCESS   SLB   Forward     S     0      0        0     
TEST_SERVERS    SLB   RoundRobin  S     2      0        0     
VLAN402-OUT     SLB   RoundRobin  none  1      0        0     
ROUTE           SLB   Forward     none  0      0        0   

 


dc1sw01#show module csm 3 vlan   

vlan   IP address       IP mask          type     
---------------------------------------------------
208    10.8.8.2         255.255.255.0    CLIENT
318    10.8.108.2       255.255.255.0    SERVER
401    10.8.12.2        255.255.255.0    CLIENT
402    10.8.112.2       255.255.252.0    SERVER
997    0.0.0.0          0.0.0.0          FT

dc1sw01#show module csm 3 arp

Internet Address  Physical Interface  VLAN      Type       Status
--------------------------------------------------------------------
10.8.108.1       00-01-64-F9-1A-01   318       -ALIAS-    local
10.8.108.2       00-0F-34-2C-52-06   318       --SLB--    local
10.8.108.3       00-02-FC-E1-CE-60   318       LEARNED    up(0 misses)
10.8.112.1       00-01-64-F9-1A-01   402       -ALIAS-    local
10.8.112.2       00-0F-34-2C-52-06   402       --SLB--    local
10.8.112.3       00-02-FC-E1-CE-60   402       LEARNED    up(0 misses)
10.8.112.17      00-50-56-AD-00-96   402       REAL       up(0 misses)
10.8.112.18      00-50-56-AD-00-96   402       REAL       up(0 misses)
10.8.8.254       00-C0-EA-EA-EA-EA   208       GATEWAY    up(0 misses)
10.8.8.2         00-0F-34-2C-52-06   208       --SLB--    local
10.8.8.3         00-02-FC-E1-CE-60   208       LEARNED    up(0 misses)
10.8.12.254      00-22-90-9D-D3-0F   401       GATEWAY    up(0 misses)
10.8.12.2        00-0F-34-2C-52-06   401       --SLB--    local
10.8.12.3        00-02-FC-E1-CE-60   401       LEARNED    up(0 misses)
10.8.12.11       00-01-64-F9-1A-01   0         VSERVER    local

 

 

CSM PING

If you want to PING from a VIP address:

 

dc1sw01#ping module csm 3 10.8.12.254
IP address       Reachable
--------------------------
10.8.12.254      Yes


dc1coresw01#ping module csm 3 gateways

IP address       Reachable
--------------------------
10.8.8.254       Yes
10.8.12.254      Yes


dc1sw01#ping module csm 3 reals

IP address       Reachable
--------------------------
10.8.108.10      Yes
10.8.108.11      Yes
10.8.112.17      Yes
10.8.112.18      Yes
10.8.12.254      Yes

 

Hope that helps!

September 13, 2010

MPLS Quality of Service – Part Two

In the pervious post, we went through the Uniform model and its configuration. In this post we are going to see the differences between the Uniform model and the Pipe model. As mentioned earlier in the Part one of MPLS Quality of Service, the Pipe model does not change customer marking (IP TOS – DSCP). So in the following example, we will see that the provider is able to change the EXP marking but will not change the packet marking in the end. Provider also performs QoS tasks (even at the egress LSR) based on EXP and MPLS QoS marking, not based on IP header properties (Short-pipe model is different in this case).

 MPLS QOS Pipemode

As depicted in the above picture, egress PE is configured to send an Explicit Null to the P router. In this case the egress PE will be able to see transport label EXP field as well as VPN label EXP field.

R1(config)#mpls ldp explicit-null

R0#show mpls forwarding-table
Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop   
tag    tag or VC   or Tunnel Id      switched   interface             
16     Pop tag     3.3.3.3/32        0          Et0/0      192.168.30.3
17     0           1.1.1.1/32        137        Et0/1      192.168.10.1

 

In this case, on R0 (the P router) we don’t need to copy EXP from topmost label (received from R3) to the topmost transport label sent to R1, because the Explicit Null (label 0) maintains the EXP bit. This is only one example to show QoS features. In this example, we like to perform our marking on the topmost label and not rely on the second label, to demonstrate explicit-null feature. The Uniform and the Pipe models difference is not in the ingress part but in the egress, to save or not to save the customer marking on the IP packet.

 

Configuration

Ingress PE:
!
access-list 110 permit icmp host 5.5.5.5 host 6.6.6.6
!
class-map match-any in1
  match access-group 110
class-map match-any out1
  match qos-group 7
!
policy-map in1
  class in1
   set mpls experimental imposition 3
   set qos-group 7
policy-map out1
  class out1
   set mpls experimental topmost 7
!

R3#sh policy-map interface e0/0
Ethernet0/0

  Service-policy output: out1

    Class-map: out1 (match-any)
      10 packets, 1220 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: qos-group 7
        10 packets, 1220 bytes
        5 minute rate 0 bps
      QoS Set
        mpls experimental topmost 7
          Packets marked 10

    Class-map: class-default (match-any)
      28 packets, 11442 bytes
      5 minute offered rate 3000 bps, drop rate 0 bps
      Match: any

P:
no configuration!

 

Egress PE:
!
class-map match-all in1
  match mpls experimental topmost 7
class-map match-all out1
  match qos-group 7
!
!
policy-map in1
  class in1
   set qos-group 7
policy-map out1
  class out1
   priority 256
  class class-default
   shape average 500000
!

R1#sh policy-map interface e0/1
Ethernet0/1

  Service-policy output: out1

    Class-map: out1 (match-all)
      10 packets, 1140 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: qos-group 7
      Queueing
        Strict Priority
        Output Queue: Conversation 264
        Bandwidth 256 (kbps) Burst 6400 (Bytes)
        (pkts matched/bytes matched) 0/0
        (total drops/bytes drops) 0/0

    Class-map: class-default (match-any)
      20 packets, 2080 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: any
      Traffic Shaping

 

Why there is no configuration on P LSR? Because IOS by default maintains your EXP and as long as PE LSR requesting Explicit Null, the topmost label is kept and contains EXP value of incoming labeled packet. (there’s no label disposition on the P LSR – R0)

September 12, 2010

MPLS Quality of Service

The Quality of Service (QoS) has become more popular than past few years, because of high-bandwidth and delay sensitive applications. QoS is prioritizing more important data over less important traffic. There are two ways to implement QoS in an IP network: IntServ and DiffServ. Internet is operating based on best-effort model.

As the DiffServ model does not require a signaling protocol such as RSVP, it’s more popular or better to say the only applicable model in today's networks. (Although we use RSVP in MPLS TE for path signaling and reservation).

IETF in RFC 3270 has recommended three QoS models for DiffServ tunneled traffic in MPLS networks:

  • Pipe model
  • Short Pipe model
  • Uniform model

The Pipe and Short Pipe models are almost the same: They do not change IP ToS of customers data at all. They might change the EXP field in the path (EXP of MPLS Label) but the IP header QoS field remains unchanged. The Pipe model performs forwarding/discarding/scheduling based on EXP at the egress LSR while the Short Pipe model does that based on IP ToS. Because in the Short pipe model there might be no label at all. (remember PHP operation – Penultimate-Hop Popping)

In the Uniform model, there’s no guarantee that ToS of customers packet remains intact, but the EXP and IP ToS fields of a data packet will always show the same thing. It means if the provider changes the EXP header, that will be copied later to the IP ToS field at the egress point.

IOS Default Behavior: In short IOS does not change your QoS in the path if you don’t change it. 
        Imposition - Copy TOS bits to EXP (TOS Reflection) when adding one or more labels by ingress LSR.
        Swap - Copy EXP from old label to new label.
        Disposition - Do NOT Copy EXP from label to IP.

 

Configuration

Label to Label (on PHP) sets EXP 5 (if top is 5) after disposing the top label:

     class-map match-all in1
      match mpls experimental topmost 5
     class-map match-all out1
      match qos-group 5
    !
     policy-map in1
      class in1
       set qos-group mpls experimental topmost
     policy-map out1
      class out1
       set mpls experimental topmost 5
    !
    interface Ethernet0/0
     service-policy input in1
    !
    interface Ethernet0/1
     service-policy output out1
    !

Label to IP (on PE) sets precedence 5 (if top is 5) after popping the label:

         class-map match-all in1
          match mpls experimental topmost 5
         class-map match-all out1
          match qos-group 5
        !
         policy-map in1
          class in1
           set qos-group mpls experimental topmost
         policy-map out1
          class out1
           set ip precedence 5
        !
        interface Ethernet0/0
         service-policy input in1
        !
        interface Ethernet0/1
         service-policy output out1

 

Scenario

In this sceniaro based on the service provider’s policy, the EXP value is changed in the path and is copied to inner label by P router. At the egress LSR, the EXP value is copied to the DSCP. (Uniform model)

Ingress PE:
!
policy-map out1
  class class-default
   set mpls experimental topmost 7
policy-map in1
  class class-default
   set mpls experimental imposition 3
!

P:

!
class-map match-any in1
  match mpls experimental topmost 0  1  2  3  4  5  6  7
class-map match-any out1
  match qos-group 0
  match qos-group 1
  match qos-group 2
  match qos-group 3
  match qos-group 4
  match qos-group 5
  match qos-group 6
  match qos-group 7
!
policy-map in1
  class in1
   set qos-group mpls experimental topmost
policy-map out1
  class out1
   set mpls experimental topmost qos-group
!

Egress PE:

!
class-map match-any in1
  match mpls experimental topmost 0  1  2  3  4  5  6  7
class-map match-any out1
  match qos-group 0
  match qos-group 1
  match qos-group 2
  match qos-group 3
  match qos-group 4
  match qos-group 5
  match qos-group 6
  match qos-group 7
!
policy-map in1
  class in1
   set qos-group mpls experimental topmost
policy-map out1
  class out1
   set precedence qos-group
!

September 7, 2010

BGP Multipath – Part Three

After publishing part 1 and part 2 of my BGP Multipath Trilogy! One of our good readers sent a very intelligent question here:

cc said:

Very nice article Shafagh,
Any difference in any of these examples if R0 were both a P and a Route Reflector?
http://www.shafagh.net/2010/09/bgp-multipath-part-two.html?showComment=1283780646233#c5196111639519265964

 

There’s a known limitation:

Route Reflector Limitation

When multiple iBGP paths installed in a routing table, a route reflector will advertise only one paths (next hop). If a router is behind a route reflector, all routers that are connected to multihomed sites will not be advertised unless a different route distinguisher is configured for each VRF.
http://www.cisco.com/en/US/docs/ios/12_2t/12_2t11/feature/guide/ft11bmpl.html#wp1027175

But like everything else in the computer world, there’s always a workaround, I found a way to make each path a “unique path” (playing with 64 bit prefix - RD) to send prefixes to ingress PE and let PE import different RDs inside the routing table. Here we go:

So in this example, R0 and R1 are using different RD for the same CE (multihomed CE – One or two routers at customer site) This makes two different vpnv4 prefixes pointing to the same ipv4 subnet.

The workaround is to advertise both or several parallel paths from different PEs with different RDs so that RRs advertise all paths. The RR will advertise all parallel paths with different RDs (as they are not same) and ingress PE can run BGP multipath and use all parallel paths.

The route reflector will not filter anything. It sends the best path to the R3. As long as these two prefixes are different they are both best path for their corresponding RDs:

R2#sh ip bgp vpn all
BGP table version is 4, local router ID is 2.2.2.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1
*>i69.69.69.69/32   1.1.1.1                  0    100      0 ?
Route Distinguisher: 1:2
*>i69.69.69.69/32   10.10.10.10              0    100      0 ?

 

R3 needs this command:

R3(config-router)#address-family ipv4 vrf C1
R3(config-router-af)#maximum-paths ibgp 2 import 1
R3(config-router-af)#^Z
R3#sh ip route vrf C1

Routing Table: C1
Gateway of last resort is not set

     69.0.0.0/32 is subnetted, 2 subnets
B       69.69.69.69 [200/0] via 10.10.10.10, 00:01:30
                    [200/0] via 1.1.1.1, 00:01:22

September 5, 2010

BGP Multipath – Part Two

In the previous post, we reviewed basics of iBGP and eBGP multipath. In this post we get introduced with import feature of MP-BGP multipath. When there’s a difference in VRF’s RD, we have to use import keyword to import parallel paths from one VRF into another.

For example, we have the same diagram as BGP Multipath Part One, the only difference is the RD number that is used by R1. The Customer VRF RD at R2,3 and 4 is 1:1, while on the other side RD is 1:2.

BGP-Multipath-import

When the RD is different, we need additional “import” keyword to tell the router how many parallel routes import from one VRF into another. If RD number is same on both side (both PE devices) previous post and example is enough for you.

 

Let’s check the configuration without any “maximum-path” command, to see the BGP natural behavior when there are parallel paths. (We know that BGP only selects one)

 

R1(config-router-af)#do sh ip bgp vpnv4 all
BGP table version is 281, local router ID is 1.1.1.1
Status codes: * valid, > best, i - internal,
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1
* i5.5.5.5/32       4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i55.55.55.0/24    4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.25.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.35.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.45.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
Route Distinguisher: 1:2 (default for vrf C1)
   Network          Next Hop            Metric LocPrf Weight Path
* i5.5.5.5/32       3.3.3.3                 10    100      0 1 ?
*> 6.6.6.6/32       192.168.16.6             0             0 2 ?
*>i55.55.55.0/24    2.2.2.2                 10    100      0 1 ?
r> 192.168.16.0     192.168.16.6             0             0 2 ?
*>i192.168.25.0     2.2.2.2                 10    100      0 1 ?
*>i192.168.35.0     2.2.2.2                 10    100      0 1 ?
*>i192.168.45.0     2.2.2.2                 10    100      0 1 ?


R1(config-router-af)#do sh ip route vrf C1

Routing Table: C1

B    192.168.45.0/24 [200/10] via 2.2.2.2, 00:00:17
B    192.168.25.0/24 [200/10] via 2.2.2.2, 00:00:17
     55.0.0.0/24 is subnetted, 1 subnets
B       55.55.55.0 [200/10] via 2.2.2.2, 00:00:17
     5.0.0.0/32 is subnetted, 1 subnets
B       5.5.5.5 [200/10] via 2.2.2.2, 00:00:17
     6.0.0.0/32 is subnetted, 1 subnets
B       6.6.6.6 [20/0] via 192.168.16.6, 07:53:26
B    192.168.35.0/24 [200/10] via 2.2.2.2, 00:00:17
C    192.168.16.0/24 is directly connected, Ethernet0/1

 

As you can see, there’s only one path (per prefix) at customer VRF. So let’s add “maximum-path ibgp” command for Customer VRF:

 

R1(config-router)#address-family ipv4 vrf C1
R1(config-router-af)#maximum-paths ibgp 3
R1(config-router-af)#
R1(config-router-af)#do sh ip bgp vpnv4 all
BGP table version is 288, local router ID is 1.1.1.1

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1
* i5.5.5.5/32       4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i55.55.55.0/24    4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.25.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.35.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.45.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
Route Distinguisher: 1:2 (default for vrf C1)
   Network          Next Hop            Metric LocPrf Weight Path
*>i5.5.5.5/32       2.2.2.2                 10    100      0 1 ?
*> 6.6.6.6/32       192.168.16.6             0             0 2 ?
*>i55.55.55.0/24    2.2.2.2                 10    100      0 1 ?
r> 192.168.16.0     192.168.16.6             0             0 2 ?
*>i192.168.25.0     2.2.2.2                 10    100      0 1 ?
*>i192.168.35.0     2.2.2.2                 10    100      0 1 ?
*>i192.168.45.0     2.2.2.2                 10    100      0 1 ?

 

So the “maximum-path ibgp” only indicates how many routes from RIB (VRF’s BGP table) can be imported to the routing table. The problem is that the RIB - RD table for 1:2 (VRF C1) has only path, so maximum-path ibgp has no parallel path to import into the routing table.

 

The import keyword allows you to configure the VRF table to accept multiple redundant paths in addition to the best path. By default, a VRF will import only one path (best path) per prefix from the source VRF table. If the best path goes down, the destination will not be reachable until the next import event occurs, and then a new best path will be imported into the VRF table. The import event runs every 15 seconds by default.

 

Note: Configuring redundant paths with the import keyword can increase CPU and memory utilization significantly, especially in a network where there are many prefixes to learn and a large number of configured VRFs. It is recommended that this feature is only configured as necessary and that the minimum number of redundant paths are configured (Typically, not more than two).

 

maximum-paths ibgp 3 import 3

 

R1(config-router-af)#maximum-paths ibgp  3 import 3
R1(config-router-af)#do sh ip bgp vpnv4 rd 1:2


BGP table version is 269, local router ID is 1.1.1.1
   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:2 (default for vrf C1)
* i5.5.5.5/32       3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
*> 6.6.6.6/32       192.168.16.6             0             0 2 ?
* i55.55.55.0/24    3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
r> 192.168.16.0     192.168.16.6             0             0 2 ?
* i192.168.25.0     3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.35.0     3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.45.0     3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?

R1(config-router-af)#do sh ip route vrf C1

Routing Table: C1

B    192.168.45.0/24 [200/10] via 4.4.4.4, 00:04:05
                     [200/10] via 3.3.3.3, 00:00:35
                     [200/10] via 2.2.2.2, 00:04:05
B    192.168.25.0/24 [200/10] via 4.4.4.4, 00:04:05
                     [200/10] via 3.3.3.3, 00:00:35
                     [200/10] via 2.2.2.2, 00:04:05
     55.0.0.0/24 is subnetted, 1 subnets
B       55.55.55.0 [200/10] via 4.4.4.4, 00:04:05
                   [200/10] via 3.3.3.3, 00:00:35
                   [200/10] via 2.2.2.2, 00:04:05
     5.0.0.0/32 is subnetted, 1 subnets
B       5.5.5.5 [200/10] via 4.4.4.4, 00:04:06
                [200/10] via 3.3.3.3, 00:00:36
                [200/10] via 2.2.2.2, 00:04:06
     6.0.0.0/32 is subnetted, 1 subnets
B       6.6.6.6 [20/0] via 192.168.16.6, 07:49:57
B    192.168.35.0/24 [200/10] via 4.4.4.4, 00:04:06
                     [200/10] via 3.3.3.3, 00:00:36
                     [200/10] via 2.2.2.2, 00:04:06
C    192.168.16.0/24 is directly connected, Ethernet0/1

 

Now, let’s play with the variables and see the different results:

 

maximum-paths ibgp 2 import 3


R1(config-router-af)#maximum-paths ibgp 2 import 3
R1(config-router-af)#do sh ip bgp vpnv4 rd 1:1


BGP table version is 222, local router ID is 1.1.1.1

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1
* i5.5.5.5/32       3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i55.55.55.0/24    3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.25.0     3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.35.0     3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.45.0     3.3.3.3                 10    100      0 1 ?
* i                 4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?

R1(config-router-af)#do sh ip bgp vpnv4 rd 1:2
BGP table version is 222, local router ID is 1.1.1.1

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:2 (default for vrf C1)
* i5.5.5.5/32       4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
*> 6.6.6.6/32       192.168.16.6             0             0 2 ?
* i55.55.55.0/24    4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
r> 192.168.16.0     192.168.16.6             0             0 2 ?
* i192.168.25.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.35.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.45.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?

R1(config-router-af)#do sh ip route vrf C1

Routing Table: C1

B    192.168.45.0/24 [200/10] via 4.4.4.4, 00:01:09
                     [200/10] via 2.2.2.2, 00:01:09
B    192.168.25.0/24 [200/10] via 4.4.4.4, 00:01:10
                     [200/10] via 2.2.2.2, 00:01:10
     55.0.0.0/24 is subnetted, 1 subnets
B       55.55.55.0 [200/10] via 4.4.4.4, 00:01:10
                   [200/10] via 2.2.2.2, 00:01:10
     5.0.0.0/32 is subnetted, 1 subnets
B       5.5.5.5 [200/10] via 4.4.4.4, 00:01:10
                [200/10] via 2.2.2.2, 00:01:10
     6.0.0.0/32 is subnetted, 1 subnets
B       6.6.6.6 [20/0] via 192.168.16.6, 07:41:14
B    192.168.35.0/24 [200/10] via 4.4.4.4, 00:01:27
                     [200/10] via 2.2.2.2, 00:01:27
C    192.168.16.0/24 is directly connected, Ethernet0/1

 

maximum-paths ibgp 3 import 2


 

R1(config-router-af)#maximum-paths ibgp 3 import 2
R1(config-router-af)#do sh ip bgp vpnv4 rd 1:1
BGP table version is 264, local router ID is 1.1.1.1

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1
* i5.5.5.5/32       4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i55.55.55.0/24    4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.25.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.35.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.45.0     4.4.4.4                 10    100      0 1 ?
* i                 3.3.3.3                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?

R1(config-router-af)#do sh ip bgp vpnv4 rd 1:2


   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:2 (default for vrf C1)
* i5.5.5.5/32       4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
*> 6.6.6.6/32       192.168.16.6             0             0 2 ?
* i55.55.55.0/24    4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
r> 192.168.16.0     192.168.16.6             0             0 2 ?
* i192.168.25.0     4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.35.0     4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?
* i192.168.45.0     4.4.4.4                 10    100      0 1 ?
*>i                 2.2.2.2                 10    100      0 1 ?


R1(config-router-af)#do sh ip route vrf C1

Routing Table: C1

B    192.168.45.0/24 [200/10] via 4.4.4.4, 00:00:22
                     [200/10] via 2.2.2.2, 00:00:22
B    192.168.25.0/24 [200/10] via 4.4.4.4, 00:00:22
                     [200/10] via 2.2.2.2, 00:00:22
     55.0.0.0/24 is subnetted, 1 subnets
B       55.55.55.0 [200/10] via 4.4.4.4, 00:00:22
                   [200/10] via 2.2.2.2, 00:00:22
     5.0.0.0/32 is subnetted, 1 subnets
B       5.5.5.5 [200/10] via 4.4.4.4, 00:00:22
                [200/10] via 2.2.2.2, 00:00:22
     6.0.0.0/32 is subnetted, 1 subnets
B       6.6.6.6 [20/0] via 192.168.16.6, 07:46:21
B    192.168.35.0/24 [200/10] via 4.4.4.4, 00:00:31
                     [200/10] via 2.2.2.2, 00:00:31
C    192.168.16.0/24 is directly connected, Ethernet0/1

 

Final R1 Configuration

 

!
router bgp 100
no synchronization
bgp log-neighbor-changes
neighbor 2.2.2.2 remote-as 100
neighbor 2.2.2.2 update-source Loopback0
neighbor 3.3.3.3 remote-as 100
neighbor 3.3.3.3 update-source Loopback0
neighbor 4.4.4.4 remote-as 100
neighbor 4.4.4.4 update-source Loopback0
no auto-summary
!
address-family vpnv4
neighbor 2.2.2.2 activate
neighbor 2.2.2.2 send-community extended
neighbor 3.3.3.3 activate
neighbor 3.3.3.3 send-community extended
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community extended
exit-address-family
!
address-family ipv4 vrf C1
neighbor 192.168.16.6 remote-as 2
neighbor 192.168.16.6 activate
 maximum-paths ibgp  3 import 3
no auto-summary
no synchronization
exit-address-family
!