Fix: NetApp DataFabric Manager Certificate has expired

September 28, 2016 by derek.hennessy Posted in Netapp Tagged DFM, Expired certficate, NetApp, NetApp DataFabric Manager Leave a comment

Following the upgrade of DFM from version 5.2.0 to 5.2.1 I started to see a warning in the onCommand Management console that the NetApp DataFabric Manager had expired and to create a new one.

Surprisingly the cert had expired ages ago but neither I nor anyone else noticed. The first step in fixing the issue was to check the SSL service details to find the expiry date of the current certificate. To find this open a command prompt and run the command:

dfm ssl service detail

If the cert is not valid after the current date, or in my case after Dec 9 2015 then a new one needs to be created.

The steps to create a new certificate are:

dfm ssl server setup
KeySize: 2048
Country Name: AU (or whatever two letter country code suites your needs)
State or Province: <insert your state name>
Locality Name: <insert your city>
Organization Name: <insert company name>
Common Name: <insert FQDN of your DFM server>
Email Address: <insert your address>

Once the cert has been created you’ll be prompted to restart the http services.

Once you restart the services you can acknowledge the alert in onCommand Manager and the alert will be gone

Fix: Cannot run upgrade script on host, ESXi 5.5

September 22, 2016 by derek.hennessy Posted in vCenter, VMware Tagged VMware, vSphere, vSphere 5.5, vSphere 5.5 Update 3b Leave a comment

During a recent upgrade I found that one of the ESXi hosts just would not update using Update Manager. The error I was seeing was “Cannot run upgrade script on host”.

After a bit of searching I found this article which related to ESXi 5.1 upgrade to 5.5 but the steps worked well to fix the issue I was seeing.

In order to fix the issue I performed the following steps:

Step 1: Disable HA for the cluster

Disable Cluster HA

Step 2: Go to vCenter Networking. Select the distributed vswitch and then select the hosts tab. From here, right-click on the host you need to reboot and select Remove from vSphere Distributed Switch

Remove Distributed Switch

Click Yes to remove the host from the switch.

Confirm vDS Removal

Step 3: Remove the host from the cluster

Remove ESXi host from cluster

Step 4: Enter the host into maintenance mode and then choose to reboot.

Enter Maintenance Mode
Step 5: Connect via SSH to the ESXi host and run the following commands to uninstall the FDM agent:

>
cp /opt/vmware/uninstallers/VMware-fdm-uninstall.sh /tmp
chmod +x /tmp/VMware-fdm-uninstall.sh
/tmp/VMware-fdm-uninstall.sh
>

SSH Host FDM Uninstaller
Step 6: Reboot the host

Reboot the host
Step 7: Add the ESXi host back to the cluster

rejoin host to cluster step 1

rejoin host to cluster step 2

rejoin host to cluster step 3

rejoin host to cluster step 4
Step 8: Re-add the host to the Distributed vSwitch. Go to Networking -> select the distributed vswitch. Right-click and select Manage Hosts.

Manage vDS

Select the host

Select Host

Select vnics for Uplinks to be managed by the switch

Manage vDS uplinks

Step 9: Turn vSphere HA back on for the cluster the host resides on.

Turn on vSphere HA

Step 10: Run the upgrade again from Update Manager and this time it will work.

How To: Upgrade to ESXi 5.5 Update 3b on Cisco UCS

September 20, 2016 by derek.hennessy Posted in vCenter, VMware Tagged Upgrade vSphere, VMware, vSphere 5.5, vSphere 5.5 Update3b Leave a comment

ESXi upgrade preparation

With Cisco UCS you really need to make sure that your ESXi hosts are running the correct driver version. If you’re running NFS or FCoE storage into your ESXi hosts as either datastores or RDM disks then it’s critical that you have the right fnic and enic drivers. Even if you use the Cisco Custom image for ESXi upgrades the enic and fnic drivers may not be correct according to the compatibility matrix. I’ve had this issue in the past and I saw intermittent NFS datastores going offline for a Dev ESXi host and the resolution was to upgrade the enic driver which handles ethernet storage connectivity.

The best place to go is to VMware’s compatibility site for IO drivers which comes under the System/Servers. To find out which drivers you currently have you will need to check on the driver versions on the ESXi hosts. This can be done by following KB1027206. Using the values for the Vendor ID, Device ID, Sub-Vendor ID and Sub-Device ID it’s possible to pinpoint the interoperability with your respective hardware. In my case I have both VIC1340 and VIC1240 in the mix so I had to go through the process twice. Primarily you’ll be using the ‘ethtool -i’ command to find the driver version.

e.g. You can check the UCS VIC 1240 for FCoE CNAs on ESXi 5.5 Update 3 here

In this image you can see the version of enic drivers I’m running, 2.1.2.71 doesn’t match the firmware version that will be installed as part of the Cisco Custom ISO image. This shows that the enic driver version will need to be upgraded as part of the process.

Fix: vCenter failure to upgrade – unable to configure log browser windows service

September 19, 2016 by derek.hennessy Posted in vCenter, VMware Tagged unable to configure log browser windows service, vCenter, vCenter 5.5, VMware vCenter Leave a comment

During a recent upgrade from vCenter Server 5.5 Update 2d to vCenter Server 5.5 Update 3b it kept failing at the web client upgrade. After successfully upgrading Single-Sign On I proceeded with the upgrade of vSphere Web Client. I got the following error during the installation:

Error 29702 unable to configure log browser windows service please check vminst.log in system temporary folder for details

The update to 5.5 3b caused disk capacity to fill up and make the installation process unable to finish the upgrade. The SSO install worked but the WebClient fails with error 29702. The primary issue was that over 40GB of space on C drive was taken up with SSO upgrade. I searched for fixes and found the following link but before carrying out the task of removing the Java Components and re-installing again I wanted to check with support on the procedure.

The steps I followed to fix the issue were:

Step 1: Go to Control Panel, select VMware vCenter Server – Java Components and select uninstall

Step 2: Click ok to confirm the uninstall

Step 3: Click Yes to confirm reboot

Step 4: Following the reboot you can then begin the upgrade process once again and this time it will succeeed. Run the vCenter installer and from Custom Install select vCenter Single Sign-On. Click Next.

Step 5: Click Install

Step 6: The single sign-on components will begin to install, including components such as OpenSSL

One of the key components being installed is VMware JRE.

Step 7: If you get prompted to close some applications select “Close the applications and attempt to restart them”. Click Ok.

Click ok to the prompt to close apps automatically

Step 8: Click Finish to complete the Single Sign-On upgrade

Step 9: Click on vCenter Web Client to begin the next stage of the upgrade

Step 10: Click Yes to continue

Step 11: Click Accept License agreement and click Next

Step 12: Click Install to begin the web client installation

Step 13: Click Finish to complete the installation

Once you click Finish click Ok on the dialog to advise that the services will take a few minutes to restart

Step 14: Select vCenter Inventory Service and click Install

Step 15: Click Yes for Inventory Service install

Step 16: Click Next to continue the installation process

Step 17: Click Accept License agreement and click Next

Step 18: Click Install for inventory service

Step 19: Click Finish on completion

Step 20: Install vCenter Server

Step 21: Click Ok to continue

Step 22: Click Next to continue

Step 23: Click to accept the license and click Next

Step 24: Enter the database user login credentials, VC_User

Step 25: Click Install at the Customer Experience Improvement Program

Step 26: Click Finish to complete the installation

Fix: Cisco B200 M4 – FlexFlash – FFCH_Error_old_firmware_Running_error

September 16, 2016 by derek.hennessy Posted in Cisco, UCS Tagged Cisco B200 M4, Cisco Champion, Cisco UCS, FlexFlash, UCS firmware upgrade Leave a comment

During a recent upgrade of Cisco B200 M4 blades I got the following error:

FlexFlash FFCH_ERROR_OLD_FIRMWARE_RUNNING

I really wasn’t sure what was causing the issue but it turned out to be a known bug for M4 blades. More details can be found over on Cisco BugSearch Note: You’ll need a Cisco Login to access the site. Basically the issue affects B200 M4 blades upgraded to 2.2(4) or higher.

The workaround is actually quite easy and just needs to have the FlexFlash Controller reset. This can be done using the below steps:

Step 1: Select Equipment -> Chassis # -> Server # -> Inventory -> Storage -> Reset FlexFlash Controller

Step 2: Click Yes to reset the FlexFlash controller

Step 3: Click Ok on reset notification

Fix: Cisco UCS B200 M4 Activation Failed

September 15, 2016 by derek.hennessy Posted in Cisco, UCS Tagged Cisco, Cisco Champion, Cisco UCS, UCS Activation Failure Leave a comment

During a recent upgrade I ran into a problem with activation of B200 M4 blade. This was following the infrastructure firmware upgrade and the next step was to upgrade the server firmware. However, before upgrading the server firmware I got the error from the B200 M4 blades showing the following error:

Activation failed and Activate Status Set to Failed

This turned out to be due to the B200 M4 blades shipping with version 7.0 of the board controller firmware. On investigation with Cisco I found that it’s a known bug – CSCuu78484

You can follow the commands to change the base board. You can find more information on that from the Cisco forums but the commands you need are below:

#scope server X/Y (chassis X blade Y)

#scope boardcontroller

#show image

#activate firmware version.0 force

>Select a lower version than current one

#commit-buffer

What I found was that since I was going to be upgrading the blade firmware version anyway there was no point in dropping the server firmware back and instead proceed with the upgrade which fixed the issue.

I spoke with TAC and they advised that the error could be ignored and I could proceed with the UCS upgrade. The full details of the upgrade can be found in another post.

How To: Cisco UCS Firmware Upgrade 2.2 to 3.1 with Auto-Install

September 14, 2016 by derek.hennessy Posted in Cisco, UCS Tagged Cisco, Cisco Champion, Cisco UCS, Cisco UCS Firmware upgrade 10 Comments

Recently I had to upgrade our ESXi hosts from Update 2 to Update 3 due to security patch requirements. This requirement stretches across two separate physical environments, one running IBM blades and the other running on Cisco UCS blade chassis in a Flexpod configuration. The upgrade paths for both are slightly different, and they also run on different vCenter platforms. Both of these also have different upgrade paths as one is running VMware SRM and is in linked mode. I’m not going to discuss the IBM upgrades but I did need to upgrade the firmware of the Infrastructure and Servers for Cisco UCSM.

Before you being any upgrade process I highly recommend reading the release notes to make sure that a) an upgrade path exists from your current version, b) you become aware of any known issues in the new version and c) the features you want exist in the new version

UCS Upgrade Prep Work

Check the UCS Release Guides

Check the release notes to make sure all the components and modules are supported. The release notes for UCS Manager can be found on their site. The link is listed further below in the documents section.

Some of the things to check within the release notes are:
* Resolved Caveats

UCS Version Upgrade patch

UCS Infrastructure Hardware compatibility

Minimum software version for UCS Blade servers

Open a Pre-Emptive Support Call

I opened a call with Cisco TAC to investigate the discrepancy in the firmware versions. The advice was to downgrade the B200 M4 server firmware down to 4.0 (1). However, as I was planning on upgrading anyway I’ve now confirmed that the best option is to upgrade to the planned 3.1 version. As part of this upgrade I will also upgrade all the ESXi hosts on that site the same day. There is a second UCS domain on another site that will be upgraded on another date.

Share this:

Share this:

ESXi upgrade preparation

Share this:

Share this:

Share this:

Share this:

UCS Upgrade Prep Work

Check the UCS Release Guides

Open a Pre-Emptive Support Call

Share this: