To date, I’ve not had to replace a running 6120XP Fabric Interconnect but just days ago, I had to replace a 6248XP Fabric Interconnect that had been configured as part of a FI Cluster. Not long after configuring my FI Cluster, FabricA started rebooting itself anywhere every 60-90 minutes. I called Cisco TAC and after a bit of troubleshooting, it was determined that the 6248 needed to be replaced. I was told that the field engineer would install the new fabric interconnect and re-create the FI cluster, but…when the field engineer arrived he said, “I don’t do configurations…”, thus began this post.

Replacing a Fabric Interconnect is not hard really, you just need to make sure its done right to make the changeover go smoothly. The high-level steps are as follows:

1. connect the new FI to the network (do not connect the L1 and L2 cables)

2. console into the new FI and run through the setup wizard configuring the new FI as a “standalone” fabric interconnect

3. Assuming the new fabric interconnect is not at the same code level as the current FI, open your internet browser and connect to the IP address assigned to the new FI, login as admin, and upload the UCS code that is installed on the cluster member.

4. Still connected to the new FI, update both UCS Manager and the Fabric Interconnect to the code running on the existing cluster member.

5. Once the upgrades are complete, verify the Running and Startup versions match those of the existing cluster member.

6. Console into the new fabric interconnect and run the following commands:
:connect local-mgmt
:erase configuration
:yes (to reboot)

7. Connect the L1 and L2 cables to the existing fabric interconnect

8. Console into the new fabric interconnect and run through the setup wizard. When the setup wizard detects the presence of a peer Fabric interconnect, type y to add the new FI to the existing cluster. Save the configuration and reboot the new FI.

9. Login to UCS Manager or use the command line to verify the cluster state.

10. Test failover. In this example, I use the command line on Fabric-B as Fabric-A was the “new” fabric interconnect. From the command line of Fabric-B, enter the following commands:
:connect local-mgmt
:show cluster state (verify B is still the primary)
:cluster lead a (makes Fabric-A the primary)
:show cluster state (if done quickly, you’ll see the status of SWITCHOVER IN PROGRESS)

11. Login to UCS Manager once again to verify the failover is seen in the GUI and you are done.