功能需求

2台M9K采用冗余组备份组技术,在IRF2堆叠环境、多context组网下,不中断业务升级集群

组网信息及描述

配置步骤

1)基础配置

2m9k形成IRF2集群,模拟多context环境。由根contextg1/1/0/20g2/1/0/20reth1share进某用户context,并配置与根墙同网段地址。如下为用户context接口配置:

<H3C>dis ip int br

*down: administratively down

(s): spoofing  (l): loopback

Interface                Physical Protocol IP Address      Description

GE1/1/0/20               up       up       66.0.0.3        --

GE2/1/0/20               up       up       66.0.1.3        --

Reth1                    up       up       50.0.0.3        --

用户context进驻默认blade组,location blade-controller-team 1。测试从R9115P17升级到E9121P02。(实验室测试,R9115P17版本,用户context进驻非默认blade组转发存在问题)

业务规划:

75.0.0.1<---->65.0.0.1走虚context。模拟用户context业务。

200.200.200.200<---->55.0.0.1走根context。模拟根墙业务。

 

2)主备及路由规划:

防火墙4块业务板划分两个failover组,一个冗余组。冗余组成员为冗余口和两个failover组,

redundancy group 0

 member interface Reth1

 member failover group 0

 member failover group 1

 node 1

  bind chassis 1

  track 1

  track 3

  track 5 interface Route-Aggregation1

  track 7 interface GigabitEthernet1/1/0/20

  node-member interface GigabitEthernet1/1/0/20

 node 2

  bind chassis 2

  track 2

  track 4

  track 6 interface Route-Aggregation2

  track 8 interface GigabitEthernet2/1/0/20

  node-member interface GigabitEthernet2/1/0/20

M9K集群对下采用冗余口,冗余口的成员口是两个三层聚合口。M9K对上两个三层物理口互联。

·         SR66m9k根墙运行ospf,并调大右侧互联链路的来回cost值。控制去往55.0.0.1走左侧路径。

·         SR66与用户context运行静态浮动路由,调低走右侧的路由优先级。控制去往65.0.0.1主走左侧路径。

·         F5000-S与根墙、用户context都走静态,控制去往75.0.0.1走用户contextreth1口,去往200.200.200.200走根墙的reth1口。

 

1升级整体思路

1、先升级主设备、再升级备设备,保证升级前后IRF2主备状态一致。

2、取消IRF2mad检测功能,关闭主设备的所有业务端口,将主设备上的业务迁移至备机,再将堆叠口断开使IRF分裂。

3IRF分裂后单独升级主设备,业务回迁主设备;升级备设备重启形成以新版本运行的IRF2

在升级版本之前,请仔细阅读新版本的版本说明书。特别注意与新版本配套的软、硬件条件,配置是否存在变更。如配置存在变更,则在升级完主设备之后下刷配置脚本再将业务回迁至主设备。备设备同理。

 

2具体操作步骤

 

2.1准备工作

准备升级目标版本,FTPTFTP服务器,配置脚本(可选)。

2.1.1检查当前软件版本

[M9000-IRF]dis version

H3C Comware Software, Version 7.1.054, Release 9115P17

Copyright (c) 2004-2016 Hangzhou H3C Tech. Co., Ltd. All rights reserved.

H3C SecPath M9010 uptime is 0 weeks, 2 days, 13 hours, 30 minutes

Last reboot reason : User reboot

2.1.2查看设备板卡信息

当前两台M9010上的板卡信息如下:

[M9000-IRF]dis device

Chassis  Slot Type             State    Subslot  Soft Ver             Patch Ver

1        0    NONE             Absent   0        NONE                 None     

1        1    NSQ1GP24TXEA0    Normal   0        M9010-9115P17        None     

1        2    NSQ1TGS32SF0     Normal   0        M9010-9115P17        None     

1        3    NONE             Absent   0        NONE                 None     

1        4    NSQ1SUPB0        Master   0        M9010-9115P17        None     

1        5    NONE             Absent   0        NONE                 None     

1        6    NONE             Absent   0        NONE                 None     

1        7    NSQ1FWCEA0       Normal   0        M9010-9115P17        None     

              CPU 1            Normal   0        M9010-9115P17       

1        8    NSQ1FWCEA0       Normal   0        M9010-9115P17        None     

              CPU 1            Normal   0        M9010-9115P17       

1        9    NONE             Absent   0        NONE                 None     

1        10   NSQ1FAB08D0      Normal   0        M9010-9115P17        None     

1        11   NSQ1FAB08D0      Normal   0        M9010-9115P17        None     

1        12   NSQ1FAB08D0      Normal   0        M9010-9115P17        None     

1        13   NSQ1FAB08D0      Normal   0        M9010-9115P17        None     

2        0    NONE             Absent   0        NONE                 None     

2        1    NSQ1GP24TXEA0    Normal   0        M9010-9115P17        None     

2        2    NSQ1TGS32SF0     Normal   0        M9010-9115P17        None     

2        3    NONE             Absent   0        NONE                 None     

2        4    NSQ1SUPB0        Standby  0        M9010-9115P17        None     

2        5    NONE             Absent   0        NONE                 None     

2        6    NONE             Absent   0        NONE                 None     

2        7    NSQ1FWCEA0       Normal   0        M9010-9115P17        None     

              CPU 1            Normal   0        M9010-9115P17       

2        8    NSQ1FWCEA0       Normal   0        M9010-9115P17        None     

              CPU 1            Normal   0        M9010-9115P17       

2        9    NONE             Absent   0        NONE                 None     

2        10   NSQ1FAB08D0      Normal   0        M9010-9115P17        None     

2        11   NSQ1FAB08D0      Normal   0        M9010-9115P17        None     

2        12   NSQ1FAB08D0      Normal   0        M9010-9115P17        None     

2        13   NSQ1FAB08D0      Normal   0        M9010-9115P17        None

其中,设备的全局主控板为1号框4槽位下的master主控板,2号框4槽位为standby主控板,每框的78槽位为三代防火墙业务板卡。

 

2.1.3检查各板卡空间使用情况

确保mastersecondary主控板的flash空间,各防火墙板卡cf卡剩余空间大于版本文件大小,以便存放ipe解压后的bootsystem文件。除以上空间外,其中ipe文件临时存放目录需要额外预留ipe文件大小的空间。

如,主控版本ipe文件及防火墙板卡版本ipe文件临时存放在主控flash中,则主控剩余空间至少需要,主控ipe*2+防火墙板卡ipe。粗略认为boot+system文件大小等于ipe文件大小。

<M9000-IRF>dir chassis1#slot4#flash:/

<M9000-IRF>dir chassis1#slot8.1#cfa0:/

<M9000-IRF>dir chassis1#slot7.1#cfa0:/

<M9000-IRF>dir chassis2#slot4#flash:/

<M9000-IRF>dir chassis2#slot8.1#cfa0:/

<M9000-IRF>dir chassis2#slot7.1#cfa0:/

 

2.1.4查看冗余备份互联等状态信息

<M9000-IRF>dis redundancy group  0

Redundancy group 0 (ID 1):

  Node ID      Chassis       Priority   Status        Track weight

  1            Chassis1      1          Primary       255

  2            Chassis2      1          Secondary     255

 

Preempt delay time remained     : 0    min

Preempt delay timer setting     : 1    min

Remaining hold-down time        : 0    sec

Hold-down timer setting         : 1    sec

Manual switchover request      : No

 

Member interfaces:

    Reth1            

Member failover groups:

    0

    1

 

Node 1:

  Node member     Physical status

    GE1/1/0/20    UP

  Track info:

    Track    Status       Reduced weight     Interface

    1        Positive     255                N/A

    3        Positive     255                N/A

    5        Positive     255                RAGG1

    7        Positive     255                GE1/1/0/20

Node 2:

  Node member     Physical status

    GE2/1/0/20    UP

  Track info:

    Track    Status       Reduced weight     Interface

    2        Positive     255                N/A

    4        Positive     255                N/A

    6        Positive     255                RAGG2

8        Positive     255                GE2/1/0/20

<M9000-IRF>dis reth interface re1

Reth1 :

  Redundancy group  : 0

  Member           Physical status         Forwarding status   Presence status

  RAGG1            UP                      Active              Normal

  RAGG2            UP                      Inactive            Normal

·         F5000学习M9Kreth1arpmac在主线路br1

<F5000S>dis arp 50.0.0.2

                Type: S-Static    D-Dynamic

IP Address       MAC Address     VLAN ID  Interface              Aging Type

50.0.0.2         0cda-41b6-41d7  50       BAGG1                  12    D

<F5000S>dis mac-address 0cda-41b6-41d7

MAC ADDR        VLAN ID    STATE            PORT INDEX             AGING TIME(s)

0cda-41b6-41d7    50       learned          Bridge-Aggregation1     AGING

·         根墙业务,66回程路由走主线路

[SR6602]dis ip routing-table 55.0.0.1

Routing Table : Public

Summary Count : 2

 

Destination/Mask    Proto  Pre  Cost         NextHop         Interface

 

0.0.0.0/0           Static 60   0            172.31.0.1      GE0/0

55.0.0.1/32         O_ASE  150  1            66.0.0.2        GE0/1

·         检查业务是否正常

2.1.5上传目标软件版本

通过FTP使用二进制模式进行版本文件传送

<M9000-IRF>ftp 10.10.241.67 vpn-instance management

Press CTRL+C to abort.

Connected to 10.10.241.67 (10.10.241.67).

220 3Com 3CDaemon FTP 服务器版本 2.0   

User (10.10.241.67:(none)): admin

331 用户名正确, 需要口令      

Password:

230 用户已登录   

Remote system type is UNIX.

ftp> binary

200 类型设置为 I.

ftp> get SECBLADENGFW-CMW710-E9121P02.ipe

再上传SECPATH9000M-CMW710-E9121P02.ipe

上传完成后,可以用<M9000-IRF>md5sum ?来计算文件值是否与源文件一致。

2.1.6指定下次启用版本

1)使用如下命令分别指定主控板及业务板卡的下次启动文件

boot-loader file flash:/SECPATH9000M-CMW710-E9121P02.ipe all main

boot-loader file flash:/SECBLADENGFW-CMW710-E9121P02.ipe chassis 1 slot 7 cpu 1 main

boot-loader file flash:/SECBLADENGFW-CMW710-E9121P02.ipe chassis 1 slot 8 cpu 1 main

boot-loader file flash:/SECBLADENGFW-CMW710-E9121P02.ipe chassis 2 slot 7 cpu 1 main

boot-loader file flash:/SECBLADENGFW-CMW710-E9121P02.ipe chassis 2 slot 8 cpu 1 main

相关输出信息如下:

Verifying the IPE file and the images.......Done.

H3C SecPath M9010 images in IPE:

  M9000-CMW710-BOOT- E9121P02.bin

  M9000-CMW710-SYSTEM-E9121P02.bin

This command will set the main startup software images. Continue? [Y/N]:y

......

......

Loading.......................................................................................................................Done.

Decompression completed.

Do you want to delete flash:/SECPATH9000M-CMW710-E9121P02.ipe now? [Y/N]:n

 

2)使用display boot-loader确认查看启动信息

相关输出信息如下:

<M9000-IRF>dis boot-loader

Software images on chassis 1 slot 4:

Current software images:

  flash:/M9000-CMW710-BOOT-R9115P17.bin

  flash:/M9000-CMW710-SYSTEM-R9115P17.bin

Main startup software images:

  flash:/M9000-CMW710-BOOT-E9121P02.bin

  flash:/M9000-CMW710-SYSTEM-E9121P02.bin

Backup startup software images:

  flash:/M9000-CMW710-BOOT-R9115P02.bin

  flash:/M9000-CMW710-SYSTEM-R9115P02.bin

Software images on chassis 1 slot 7.1:

Current software images:

  cfa0:/BLADE3FWM9000-CMW710-BOOT-R9115P17.bin

  cfa0:/BLADE3FWM9000-CMW710-SYSTEM-R9115P17.bin

Main startup software images:

  cfa0:/BLADE3FWM9000-CMW710-BOOT-E9121P02.bin

  cfa0:/BLADE3FWM9000-CMW710-SYSTEM-E9121P02.bin

Backup startup software images:

  cfa0:/BLADE3FWM9000-CMW710-BOOT-R9115P02.bin

  cfa0:/BLADE3FWM9000-CMW710-SYSTEM-R9115P02.bin

Software images on chassis 1 slot 8.1:

Current software images:

  cfa0:/BLADE3FWM9000-CMW710-BOOT-R9115P17.bin

  cfa0:/BLADE3FWM9000-CMW710-SYSTEM-R9115P17.bin

Main startup software images:

  cfa0:/BLADE3FWM9000-CMW710-BOOT-E9121P02.bin

  cfa0:/BLADE3FWM9000-CMW710-SYSTEM-E9121P02.bin

Backup startup software images:

  None

Software images on chassis 2 slot 4:

Current software images:

  flash:/M9000-CMW710-BOOT-R9115P17.bin

  flash:/M9000-CMW710-SYSTEM-R9115P17.bin

Main startup software images:

  flash:/M9000-CMW710-BOOT-E9121P02.bin

  flash:/M9000-CMW710-SYSTEM-E9121P02.bin

Backup startup software images:

  flash:/M9000-CMW710-BOOT-R9115P02.bin

  flash:/M9000-CMW710-SYSTEM-R9115P02.bin

Software images on chassis 2 slot 7.1:

Current software images:

  cfa0:/BLADE3FWM9000-CMW710-BOOT-R9115P17.bin

  cfa0:/BLADE3FWM9000-CMW710-SYSTEM-R9115P17.bin

Main startup software images:

  cfa0:/BLADE3FWM9000-CMW710-BOOT-E9121P02.bin

  cfa0:/BLADE3FWM9000-CMW710-SYSTEM-E9121P02.bin

Backup startup software images:

  None

Software images on chassis 2 slot 8.1:

Current software images:

  cfa0:/BLADE3FWM9000-CMW710-BOOT-R9115P17.bin

  cfa0:/BLADE3FWM9000-CMW710-SYSTEM-R9115P17.bin

Main startup software images:

  cfa0:/BLADE3FWM9000-CMW710-BOOT-E9121P02.bin

  cfa0:/BLADE3FWM9000-CMW710-SYSTEM-E9121P02.bin

Backup startup software images:

  None

 

2.2升级过程

 

操作序号

操作步骤

是否影响转发

转发丢包时间

1

清除IRFMAD检测相关的所有配置

0

2

将主框上的所有业务端口(包括BFD检测端口,但不包括堆叠口)加入端口组,在端口组视图下执行shutdown命令。

测试业务是否正常(将业务迁到备设备)

<3S

3

在堆叠状态下保存配置

0

4

断开堆叠链路使堆叠分裂

0

5

升级主框,升级完毕确认主框工作正常

0

6

将备框和主框上所有业务口(包括BFD检测口,但不包括堆叠口)分别加入端口组,shutdown备框端口组,同时undo shutdown主框上的端口组

注意:此时堆叠线路保持断开状态

,切换之后需要进行详细的业务测试进行确认主框升级已完成

<25S

(视操作而定)

7

升级备框,备框开始重启,立即连接堆叠线

0

8

备框重启完毕堆叠自动恢复,测试业务是否正常

0

9

恢复IRF MAD检测配置,保存配置

0

1清除bfdmad检测配置

[M9000-IRF-Route-Aggregation100]dis thi

#

interface Route-Aggregation100

 mad bfd enable

 mad ip address 17.1.1.1 255.255.255.252 member 1

 mad ip address 17.1.1.2 255.255.255.252 member 2

[M9000-IRF]undo interface Route-Aggregation 100

2将根墙上所有主框上下行业务端口shutdown

包括bfd检测端口,不包括irf端口。用户context的接口将共享根墙接口状态。

[M9000-IRF]interface rang GigabitEthernet 1/1/0/20 Route-Aggregation 1 GigabitEthernet 1/1/0/23

[M9000-IRF-if-range]shutdown

防火墙的冗余组功能切换生效,冗余组primarychassis1切换到chassis2,此时根context和用户context会发生业务迁移。

·         此时冗余组、冗余口的状态为

[M9000-IRF-if-range]dis redundancy group  0

Redundancy group 0 (ID 1):

  Node ID      Chassis       Priority   Status        Track weight

  1            Chassis1      1          Secondary     -255

  2            Chassis2      1          Primary       255

 

Preempt delay time remained     : 0    min

Preempt delay timer setting     : 1    min

Remaining hold-down time        : 0    sec

Hold-down timer setting         : 1    sec

Manual switchover request      : No

 

Member interfaces:

    Reth1             

Member failover groups:

    0

    1

 

Node 1:

  Node member     Physical status

    GE1/1/0/20    DOWN

  Track info:

    Track    Status       Reduced weight     Interface

    1        Positive     255                N/A

    3        Positive     255                N/A

    5        Negative     255                RAGG1(Fault)

    7        Negative     255                GE1/1/0/20

Node 2:

  Node member     Physical status

    GE2/1/0/20    UP

  Track info:

    Track    Status       Reduced weight     Interface

    2        Positive     255                N/A

    4        Positive     255                N/A

    6        Positive     255                RAGG2

8        Positive     255                GE2/1/0/20

[M9000-IRF-if-range]dis reth interface Reth 1

Reth1 :

  Redundancy group  : 0

  Member           Physical status         Forwarding status   Presence status

  RAGG1            DOWN                    Inactive            Normal

  RAGG2            UP                      Active              Normal

·         Sr66去往根墙的业务路由切换到备框,由ospf动态路由重新选路

[SR6602]dis ip routing-table 55.0.0.1

Routing Table : Public

Summary Count : 2

 

Destination/Mask    Proto  Pre  Cost         NextHop         Interface

 

0.0.0.0/0           Static 60   0            172.31.0.1      GE0/0

55.0.0.1/32         O_ASE  150  12            66.0.1.2        GE0/2

·         Sr66去往虚墙的业务切换到备框,高优先级路由失效,低优先级路由生效。

[SR6602]dis ip routing-table 65.0.0.1

Routing Table : Public

Summary Count : 2

 

Destination/Mask    Proto  Pre  Cost         NextHop         Interface

 

0.0.0.0/0           Static 60   0            172.31.0.1      GE0/0

65.0.0.1/32         Static 70   0            66.0.1.3        GE0/2

·          F5000上学习的m9kreth口的arp切换到br2

<F5000S>dis arp 50.0.0.2

                Type: S-Static    D-Dynamic

IP Address       MAC Address     VLAN ID  Interface              Aging Type

50.0.0.2         0cda-41b6-41d7  50       BAGG2                  6     D

3根墙保存配置,进入用户context保存配置。

此时配置文件里主框上除irf端口以外,其余端口全部shutdown,后续主框升级重启完成后,端口仍然处于down状态,业务保留在备框上,给手工回切业务至主框争取时间。从而可以手工操作undo shutdown主框和shutdown备框。

[M9000-IRF-if-range]save

[H3C]save

4确认业务正常后,手工断开irf线缆,使堆叠分裂。

·         可以查看用户context里会话在chassis2上,

[H3C]dis session table ipv4 source-ip 65.0.0.1 verbose

CPU 1 on slot 7 in chassis 2:

Initiator:

  Source      IP/port: 65.0.0.1/35

  Destination IP/port: 75.0.0.1/2048

  DS-Lite tunnel peer: -

  VPN instance/VLAN ID/VLL ID: -/-/-

  Protocol: ICMP(1)

  Inbound interface: Reth1

  Source security zone: Trust

Responder:

  Source      IP/port: 75.0.0.1/35

  Destination IP/port: 65.0.0.1/0

  DS-Lite tunnel peer: -

  VPN instance/VLAN ID/VLL ID: -/-/-

  Protocol: ICMP(1)

  Inbound interface: GigabitEthernet2/1/0/20

  Source security zone: Trust

State: ICMP_REPLY

Application: OTHER

Start time: 2016-06-29 09:38:49  TTL: 29s

Initiator->Responder:          290 packets      24360 bytes

Responder->Initiator:          290 packets      24360 bytes

 

Total sessions found: 1

在切换过程中,

·         用户context业务丢包数为1个,静态路由发生切换

F5000-S打印日志为

Reply from 75.0.0.1: bytes=56 Sequence=617 ttl=254 time=1 ms

%Jun 29 19:17:21:312 2016 F5000S LAGG/5/LAGG_INACTIVE_PARTNER: Member port GigabitEthernet1/1 of aggregation group BAGG1 becomes INACTIVE because the port's partner is improper for being attached.

%Jun 29 19:17:22:260 2016 F5000S LAGG/5/LAGG_INACTIVE_DUPLEX: Member port GigabitEthernet0/1 of aggregation group BAGG1 becomes INACTIVE because the port's duplex mode is improper for being attached.

%Jun 29 19:17:22:760 2016 F5000S IFNET/3/LINK_UPDOWN: GigabitEthernet0/1 link status is DOWN.

    Request time out

    Reply from 75.0.0.1: bytes=56 Sequence=619 ttl=254 time=1 ms

Reply from 75.0.0.1: bytes=56 Sequence=620 ttl=254 time=1 ms

·         context会话切换到备框

[M9000-IRF]dis session table ipv4 source-ip 200.200.200.200 verbose

Slot 4 in chassis 2:

Total sessions found: 0

 

CPU 1 on slot 7 in chassis 2:

Initiator:

  Source      IP/port: 200.200.200.200/72

  Destination IP/port: 55.0.0.1/2048

  DS-Lite tunnel peer: -

  VPN instance/VLAN ID/VLL ID: -/-/-

  Protocol: ICMP(1)

  Inbound interface: GigabitEthernet2/1/0/20

  Source security zone: Trust

Responder:

  Source      IP/port: 55.0.0.1/72

  Destination IP/port: 200.200.200.200/0

  DS-Lite tunnel peer: -

  VPN instance/VLAN ID/VLL ID: -/-/-

  Protocol: ICMP(1)

  Inbound interface: Reth1

  Source security zone: Trust

State: ICMP_REPLY

Application: OTHER

Start time: 2016-06-29 09:39:32  TTL: 29s

Initiator->Responder:          725 packets      60900 bytes

Responder->Initiator:          725 packets      60900 bytes

 

Total sessions found: 1

 

CPU 1 on slot 8 in chassis 2:

Total sessions found: 0

 

·         context业务丢包数为3,主要是ospf路由切换时间。

SR6602打印日志为

Reply from 55.0.0.1: bytes=56 Sequence=472 ttl=254 time=1 ms

    Reply from 55.0.0.1: bytes=56 Sequence=473 ttl=254 time=1 ms

#Jun 29 18:41:14:190 2016 SR6602 IFNET/4/INTERFACE UPDOWN:

 Trap 1.3.6.1.6.3.1.1.5.3<linkDown>: Interface 1048577 is Down, ifAdminStatus is 1, ifOperStatus is 2 

#Jun 29 18:41:14:190 2016 SR6602 OSPF/5/IF_STATE_CHANGE: OSPF TrapID1.3.6.1.2.1.14.16.2.16<ospfIfStateChange>: Non-virtual interface 66.0.0.1 index 0 Router 200.200.200.200 state change to 1.

#Jun 29 18:41:14:191 2016 SR6602 OSPF/6/ORIGINATE_LSA: OSPF TrapID1.3.6.1.2.1.14.16.2.12<ospfOriginateLsa>: Originate new LSA AreaId 0.0.0.0 LsdbType 1 LsdbLsid 200.200.200.200 LsdbRouterId 200.200.200.200 Router 200.200.200.200.

%Jun 29 18:41:14:191 2016 SR6602 IFNET/3/LINK_UPDOWN: GigabitEthernet0/1 link status is DOWN.

%Jun 29 18:41:14:191 2016 SR6602 IFNET/5/LINEPROTO_UPDOWN: Line protocol on the interface GigabitEthernet0/1 is DOWN.

%Jun 29 18:41:14:191 2016 SR6602 OSPF/5/OSPF_NBR_CHG: OSPF 1 Neighbor 66.0.0.2(GigabitEthernet0/1) from Full to Down.

    Request time out

#Jun 29 18:41:17:230 2016 SR6602 OSPF/5/MAXAGE_LSA: OSPF TrapID1.3.6.1.2.1.14.16.2.13<ospfMaxAgeLsa>: Aged a LSA Area Id 0.0.0.0 LsdbType 2 LsdbLsid 66.0.0.1 LsdbRouterId 200.200.200.200 Router 200.200.200.200 .

    Request time out

    Request time out

    Reply from 55.0.0.1: bytes=56 Sequence=477 ttl=254 time=1 ms

    Reply from 55.0.0.1: bytes=56 Sequence=478 ttl=254 time=1 ms

 

5重启chassis1

重启前务必check当前操作框为chassis.此时提示保存配置,选择n,不要保存。

<M9000-IRF>reboo

Start to check configuration with next startup configuration file, please wait.........DONE!

Current configuration may be lost after the reboot, save current configuration? [Y/N]:n

This command will reboot the device. Continue? [Y/N]:y

Now rebooting, please wait...

·         以新版本重启完成后,

<M9000-IRF>dis version

H3C Comware Software, Version 7.1.064, Ess 9121P02

Copyright (c) 2004-2016 Hangzhou H3C Tech. Co., Ltd. All rights reserved.

H3C SecPath M9010 uptime is 0 weeks, 0 days, 0 hours, 2 minutes

Last reboot reason : User reboot

·          此时查看chassis1上冗余组状态为。chassis1认为chassis2的不在位,track项均失效,Primarychassis1

<M9000-IRF>dis redundancy group 0

Redundancy group 0 (ID 1):

  Node ID      Chassis       Priority   Status        Track weight

  1            Chassis1      1          Primary       -255

  2            Chassis2      1          Secondary     -765

 

Preempt delay time remained     : 0    min

Preempt delay timer setting     : 1    min

Remaining hold-down time        : 0    sec

Hold-down timer setting         : 1    sec

Manual switchover request       : No

 

Member interfaces:

    Reth1            

Member failover groups:

    0

    1

 

Node 1:

  Node member     Physical status

    GE1/1/0/20    DOWN

  Track info:

    Track    Status       Reduced weight     Interface

    1        Positive     255                N/A

    3        Positive     255                N/A

    5        Negative     255                RAGG1(Fault)

    7        Negative     255                GE1/1/0/20

Node 2:

  Track info:

    Track    Status       Reduced weight     Interface

    2        Negative     255                N/A

    4        Negative     255                N/A

    6        Negative     255                RAGG2

    8        Negative     255                GE2/1/0/20(Absent)

·         Chassis2上冗余组状态为,chassis2认为chassis1不在位,primarychassis2上。

[M9000-IRF-if-range]dis redundancy group 0

Redundancy group 0 (ID 1):

  Node ID      Chassis       Priority   Status        Track weight

  1            Chassis1      1          Secondary     -765

  2            Chassis2      1          Primary       255

 

Preempt delay time remained     : 0    min

Preempt delay timer setting     : 1    min

Remaining hold-down time        : 0    sec

Hold-down timer setting         : 1    sec

Manual switchover request      : No

 

Member interfaces:

    Reth1            

Member failover groups:

    0

    1

 

Node 1:

  Track info:

    Track    Status       Reduced weight     Interface

    1        Negative     255                N/A

    3        Negative     255                N/A

    5        Negative     255                RAGG1(Fault)

    7        Negative     255                GE1/1/0/20(Absent)

Node 2:

  Node member     Physical status

    GE2/1/0/20    UP

  Track info:

    Track    Status       Reduced weight     Interface

    2        Positive     255                N/A

    4        Positive     255                N/A

    6        Positive     255                RAGG2

    8        Positive     255                GE2/1/0/20

 

6 操作接口

Undo shutdown chassis1上所有业务端口,Shutdown chassis2上所有业务端口

[M9000-IRF]interface range GigabitEthernet 1/1/0/20 GigabitEthernet 1/1/0/23

[M9000-IRF-if-range]undo shutdown

[M9000-IRF]interface  range  GigabitEthernet 2/1/0/20 GigabitEthernet 2/1/0/23 Route-Aggregation 2

[M9000-IRF-if-range]shutdown

此后reth1工作在Route-Aggregation 1上。

实测,虚墙业务丢包为1个,根墙丢包为24个。(为了使根墙业务更少丢包,可以在操作主框接口后稍等数秒再操作备框接口使ospf邻居状态切换时间重叠。但用户context丢包可能会受影响,主要因为reth1口形成双主。)

F5000上打印日志为

Reply from 75.0.0.1: bytes=56 Sequence=24 ttl=254 time=1 ms

    Reply from 75.0.0.1: bytes=56 Sequence=25 ttl=254 time=1 ms

%Jun 29 21:07:36:686 2016 F5000S LAGG/5/LAGG_INACTIVE_PARTNER: Member port GigabitEthernet1/2 of aggregation group BAGG2 becomes INACTIVE because the port's partner is improper for being attached.

%Jun 29 21:07:37:393 2016 F5000S IFNET/3/LINK_UPDOWN: GigabitEthernet1/2 link status is DOWN.

%Jun 29 21:07:37:396 2016 F5000S IFNET/3/LINK_UPDOWN: GigabitEthernet0/2 link status is DOWN.

%Jun 29 21:07:37:396 2016 F5000S LAGG/5/LAGG_INACTIVE_PHYSTATE: Member port GigabitEthernet0/2 of aggregation group BAGG2 becomes INACTIVE because the port's physical state (down) is improper for being attached.

%Jun 29 21:07:37:396 2016 F5000S IFNET/3/LINK_UPDOWN: Bridge-Aggregation2 link status is DOWN.

%Jun 29 21:07:37:398 2016 F5000S IFNET/3/LINK_UPDOWN: Vlan-interface50 link status is DOWN.

%Jun 29 21:07:37:398 2016 F5000S IFNET/5/LINEPROTO_UPDOWN: Line protocol on the interface Vlan-interface50 is DOWN.

%Jun 29 21:07:37:893 2016 F5000S IFNET/3/LINK_UPDOWN: GigabitEthernet1/1 link status is UP.

%Jun 29 21:07:37:895 2016 F5000S LAGG/5/LAGG_ACTIVE: Member port GigabitEthernet1/1 of aggregation group BAGG1 becomes ACTIVE.

%Jun 29 21:07:37:895 2016 F5000S IFNET/3/LINK_UPDOWN: Bridge-Aggregation1 link status is UP.

%Jun 29 21:07:37:959 2016 F5000S IFNET/3/LINK_UPDOWN: Vlan-interface50 link status is UP.

%Jun 29 21:07:37:959 2016 F5000S IFNET/5/LINEPROTO_UPDOWN: Line protocol on the interface Vlan-interface50 is UP.

%Jun 29 21:07:37:999 2016 F5000S IFNET/3/LINK_UPDOWN: GigabitEthernet0/1 link status is UP.

    Request time out

    Reply from 75.0.0.1: bytes=56 Sequence=27 ttl=254 time=1 ms

    Reply from 75.0.0.1: bytes=56 Sequence=28 ttl=254 time=1 ms

 

Sr66上打印日志为

Reply from 55.0.0.1: bytes=56 Sequence=17 ttl=254 time=1 ms

    Reply from 55.0.0.1: bytes=56 Sequence=18 ttl=254 time=1 ms

#Jun 29 20:31:29:445 2016 SR6602 IFNET/4/INTERFACE UPDOWN:

 Trap 1.3.6.1.6.3.1.1.5.3<linkDown>: Interface 1048578 is Down, ifAdminStatus is 1, ifOperStatus is 2 

#Jun 29 20:31:29:446 2016 SR6602 OSPF/5/IF_STATE_CHANGE: OSPF TrapID1.3.6.1.2.1.14.16.2.16<ospfIfStateChange>: Non-virtual interface 66.0.1.1 index 0 Router 200.200.200.200 state change to 1.

#Jun 29 20:31:29:446 2016 SR6602 OSPF/6/ORIGINATE_LSA: OSPF TrapID1.3.6.1.2.1.14.16.2.12<ospfOriginateLsa>: Originate new LSA AreaId 0.0.0.0 LsdbType 1 LsdbLsid 200.200.200.200 LsdbRouterId 200.200.200.200 Router 200.200.200.200.

%Jun 29 20:31:29:446 2016 SR6602 IFNET/3/LINK_UPDOWN: GigabitEthernet0/2 link status is DOWN.

%Jun 29 20:31:29:446 2016 SR6602 IFNET/5/LINEPROTO_UPDOWN: Line protocol on the interface GigabitEthernet0/2 is DOWN.

%Jun 29 20:31:29:447 2016 SR6602 OSPF/5/OSPF_NBR_CHG: OSPF 1 Neighbor 66.0.1.2(GigabitEthernet0/2) from Full to Down.

#Jun 29 20:31:30:244 2016 SR6602 IFNET/4/INTERFACE UPDOWN:

 Trap 1.3.6.1.6.3.1.1.5.4<linkUp>: Interface 1048577 is Up, ifAdminStatus is 1, ifOperStatus is 1 

%Jun 29 20:31:30:244 2016 SR6602 IFNET/3/LINK_UPDOWN: GigabitEthernet0/1 link status is UP.

%Jun 29 20:31:30:244 2016 SR6602 IFNET/5/LINEPROTO_UPDOWN: Line protocol on the interface GigabitEthernet0/1 is UP.

    Request time out

#Jun 29 20:31:31:233 2016 SR6602 OSPF/5/MAXAGE_LSA: OSPF TrapID1.3.6.1.2.1.14.16.2.13<ospfMaxAgeLsa>: Aged a LSA Area Id 0.0.0.0 LsdbType 2 LsdbLsid 66.0.1.1 LsdbRouterId 200.200.200.200 Router 200.200.200.200 .

    Request time out

#Jun 29 20:31:35:233 2016 SR6602 OSPF/6/ORIGINATE_LSA: OSPF TrapID1.3.6.1.2.1.14.16.2.12<ospfOriginateLsa>: Originate new LSA AreaId 0.0.0.0 LsdbType 1 LsdbLsid 200.200.200.200 LsdbRouterId 200.200.200.200 Router 200.200.200.200.

    Request time out

    Request time out

    Request time out

    Request time out

    Request time out

    Request time out

    Request time out

    Request time out

    Request time out

    Request time out

    Request time out

    Request time out

    Request time out

    Request time out

    Request time out

    Request time out

#Jun 29 20:32:10:190 2016 SR6602 OSPF/6/ORIGINATE_LSA: OSPF TrapID1.3.6.1.2.1.14.16.2.12<ospfOriginateLsa>: Originate new LSA AreaId 0.0.0.0 LsdbType 1 LsdbLsid 200.200.200.200 LsdbRouterId 200.200.200.200 Router 200.200.200.200.

#Jun 29 20:32:10:195 2016 SR6602 OSPF/6/ORIGINATE_LSA: OSPF TrapID1.3.6.1.2.1.14.16.2.12<ospfOriginateLsa>: Originate new LSA AreaId 0.0.0.0 LsdbType 2 LsdbLsid 66.0.0.1 LsdbRouterId 200.200.200.200 Router 200.200.200.200.

#Jun 29 20:32:10:196 2016 SR6602 OSPF/6/ORIGINATE_LSA: OSPF TrapID1.3.6.1.2.1.14.16.2.12<ospfOriginateLsa>: Originate new LSA AreaId 0.0.0.0 LsdbType 2 LsdbLsid 66.0.0.1 LsdbRouterId 200.200.200.200 Router 200.200.200.200.

%Jun 29 20:32:10:197 2016 SR6602 OSPF/5/OSPF_NBR_CHG: OSPF 1 Neighbor 66.0.0.2(GigabitEthernet0/1) from Loading to Full.

    Request time out

    Request time out

    Request time out

#Jun 29 20:32:15:193 2016 SR6602 OSPF/6/ORIGINATE_LSA: OSPF TrapID1.3.6.1.2.1.14.16.2.12<ospfOriginateLsa>: Originate new LSA AreaId 0.0.0.0 LsdbType 1 LsdbLsid 200.200.200.200 LsdbRouterId 200.200.200.200 Router 200.200.200.200.

#Jun 29 20:32:15:213 2016 SR6602 OSPF/6/ORIGINATE_LSA: OSPF TrapID1.3.6.1.2.1.14.16.2.12<ospfOriginateLsa>: Originate new LSA AreaId 0.0.0.0 LsdbType 2 LsdbLsid 66.0.0.1 LsdbRouterId 200.200.200.200 Router 200.200.200.200.

    Request time out

    Request time out

    Request time out

    Reply from 55.0.0.1: bytes=56 Sequence=43 ttl=254 time=3 ms

    Reply from 55.0.0.1: bytes=56 Sequence=44 ttl=254 time=2 ms

    Reply from 55.0.0.1: bytes=56 Sequence=45 ttl=254 time=1 ms

    Reply from 55.0.0.1: bytes=56 Sequence=46 ttl=254 time=1 ms

    Reply from 55.0.0.1: bytes=56 Sequence=47 ttl=254 time=1 ms

    Reply from 55.0.0.1: bytes=56 Sequence=48 ttl=254 time=1 ms

    Reply from 55.0.0.1: bytes=56 Sequence=49 ttl=254 time=1 ms

    Reply from 55.0.0.1: bytes=56 Sequence=50 ttl=254 time=1 ms

    Reply from 55.0.0.1: bytes=56 Sequence=51 ttl=254 time=1 ms

 

  --- 55.0.0.1 ping statistics ---

    52 packet(s) transmitted

    28 packet(s) received

    46.15% packet loss

    round-trip min/avg/max = 1/1/3 ms

根墙切换时间长,主要是因为接口状态切换,ospf新建邻居,路由收敛。

·         在用户context上查看相关会话

<H3C>dis session  table ipv4 source-ip 65.0.0.1 verbose

CPU 1 on slot 7 in chassis 1:

Initiator:

  Source      IP/port: 65.0.0.1/44

  Destination IP/port: 75.0.0.1/2048

  DS-Lite tunnel peer: -

  VPN instance/VLAN ID/Inline ID: -/-/-

  Protocol: ICMP(1)

  Inbound interface: Reth1

  Source security zone: Trust

Responder:

  Source      IP/port: 75.0.0.1/44

  Destination IP/port: 65.0.0.1/0

  DS-Lite tunnel peer: -

  VPN instance/VLAN ID/Inline ID: -/-/-

  Protocol: ICMP(1)

  Inbound interface: GigabitEthernet1/1/0/20

  Source security zone: Trust

State: ICMP_REPLY

Application: ICMP

Start time: 2016-06-29 11:31:14  TTL: 29s

Initiator->Responder:           50 packets       4200 bytes

Responder->Initiator:           50 packets       4200 bytes

·         根墙上查看会话:

[M9000-IRF-if-range]dis session table ipv4 source-ip 200.200.200.200 verbose

Slot 4 in chassis 1:

Total sessions found: 0

 

CPU 1 on slot 7 in chassis 1:

Initiator:

  Source      IP/port: 200.200.200.200/78

  Destination IP/port: 55.0.0.1/2048

  DS-Lite tunnel peer: -

  VPN instance/VLAN ID/Inline ID: -/-/-

  Protocol: ICMP(1)

  Inbound interface: GigabitEthernet1/1/0/20

  Source security zone: Trust

Responder:

  Source      IP/port: 55.0.0.1/78

  Destination IP/port: 200.200.200.200/0

  DS-Lite tunnel peer: -

  VPN instance/VLAN ID/Inline ID: -/-/-

  Protocol: ICMP(1)

  Inbound interface: Reth1

  Source security zone: Trust

State: ICMP_REPLY

Application: ICMP

Start time: 2016-06-29 11:32:52  TTL: 26s

Initiator->Responder:            34 packets        2856 bytes

Responder->Initiator:            34 packets        2856 bytes

 

Total sessions found: 1

 

CPU 1 on slot 8 in chassis 1:

Total sessions found: 0

7重启备框,与此同时恢复IRF堆叠线缆

<M9000-IRF>reboot

Start to check configuration with next startup configuration file, please wait.........DONE!

Current configuration may be lost after the reboot, save current configuration? [Y/N]:n

This command will reboot the device. Continue? [Y/N]:y

Now rebooting, please wait...

·         重启chassis2时,Chassis1显示的冗余组信息保持不变

[M9000-IRF-if-range]dis redundancy group 0

Redundancy group 0 (ID 1):

  Node ID      Chassis       Priority   Status        Track weight

  1            Chassis1      1          Primary       255

  2            Chassis2      1          Secondary     -765

 

Preempt delay time remained     : 0    min

Preempt delay timer setting     : 1    min

Remaining hold-down time        : 0    sec

Hold-down timer setting         : 1    sec

Manual switchover request       : No

 

Member interfaces:

    Reth1            

Member failover groups:

    0

    1

 

Node 1:

  Node member     Physical status

    GE1/1/0/20    UP

  Track info:

    Track    Status       Reduced weight     Interface

    1        Positive     255                N/A

    3        Positive     255                N/A

    5        Positive     255                RAGG1

    7        Positive     255                GE1/1/0/20

Node 2:

  Track info:

    Track    Status       Reduced weight     Interface

    2        Negative     255                N/A

    4        Negative     255                N/A

    6        Negative     255                RAGG2

    8        Negative     255                GE2/1/0/20(Absent)

 

8备框up后即是堆叠好的新版本运行的集群

再次check相关链路、冗余备份、会话、上下行路由等信息是否与升级前一致,并确认业务是否正常。

恢复bfd mad检测的配置后,保存配置即可。

至此完成IRF集群升级。

[M9000-IRF]dis version

H3C Comware Software, Version 7.1.064, Ess 9121P02

Copyright (c) 2004-2016 Hangzhou H3C Tech. Co., Ltd. All rights reserved.

H3C SecPath M9010 uptime is 0 weeks, 0 days, 1 hour, 7 minutes

Last reboot reason : User reboot

[M9000-IRF]dis irf

MemberID  Slot  Role    Priority  CPU-Mac         Description

 *+1      4     Master  32        00e0-fc0f-8c05  ---

   2      4     Standby 1         00e0-fc0f-8c17  ---

--------------------------------------------------

配置关键点及注意事项

注意事项

1)当升级单框时,执行reboot命令后首先会被询问是否保存当前配置,之后才会询问是否重启。在堆叠分裂的情况下,切勿执行保存配置的操作,所以第一次请输入N,第二次输入Y

2)执行升级步骤第7步时,务必及时连接堆叠线。假如备框重启完毕未能加入主框的堆叠,则可能导致转发不通等异常。

回退步骤

1.3.1    若业务切换到备框,发现业务不通,在一定时间内无法定位,升级工作进入如下回退步骤:

·         恢复主框所有链路,并删除之前相关的升级配置,以防意外断电后设备启动异常。

·         check堆叠状态下配置是否合理,重点检查关于备框的配置。

1.3.2    若升级chassis1完成后,在业务切换到升级后的chassis1后进行业务测试有异常且在一定时间内无法定位,升级工作进入如下回退步骤:

·         将业务流量切回到未升级的chassis2上,并删除之前相关的升级配置,以防意外断电后设备启动异常。

·         将已经升级的chassis1进行版本回退并重新加入堆叠。

案例信息

案例类型:典型配置
案例号:201607040001
创建时间:2016年7月4日
更新时间:2017年8月2日
发布时间:2016/7/11 6:30:54
文章密级:游客可见
有效期:长期有效
发布者:张文田 [z12652]
点击次数:3388
评论平均得分:5.00
关键词:m9000 m9k 冗余 备份 IRF2 堆叠 context
产品线:安全产品
产品系列:M9000系列
产品版本:R9115p17
技术分类:安全产品技术 Firewall

常用操作
收藏