Cisco announced their release of UCS Director 5.4 back in November. As I’m currently running 5.3 and ran into an issue with a workflow Cisco support recommended upgrading to 5.4. I had a look over the Cisco UCS Director 5.4 Release Notes and there’s a new version of Java and the CentOS operating system are newer in the latest version. Due to this the upgrade procedure for 5.4 is different from previous version. In earlier versions it was possible to upload a patch via shelladmin and it would upgrade the software and database schema in place. 5.4 however requires new appliances to be deployed and a migration of database files etc. to be done between the 5.3 and 5.4 versions.

I really think that Cisco needs to look at using a HTML 5 console in the future as this upgrade path is overly complicated. Considering a lot of companies want you to be on the latest version when opening support calls, including Cisco, it would make sense for them to make it easier to perform the required upgrades.

The primary changes that have caused the modification to the upgrade path are:

  • CentOS version 5.4 to version 6.6
  • Java version 1.6 to version 1.8

Another thing to note is that version 5.54 requires 12GB RAM.

Cisco recommend standing up  the new appliances beside your current UCS Director and Bare-Metal Appliances and performing a migration. In my case there’s a few firewall rule etc already been created for the existing environment so I wanted to keep the same IP addresses and machine names. I changed the IP addresses of the current appliances to be something else within the same subnet and gave the new appliances temporary names but the existing IP addresses. Once everything had been migrated and the changes confirmed I was able to rename the appliances to be the existing ones and removed the older appliances from the infrastructure. Before commencing the upgrade I also had a sold read over the UCS Director Upgrade 5.4 Guide and the UCS Director Bare-Metal Agent 5.4 Upgrade Guide

Software Download:

Download the software form the Cisco Download Software portal. Download the UCS Director 5.4 appliance and the BareMetal Agent files.

UCSD software download

Once it’s been downloaded extract the zip to access the OVF required for a new appliance.

 

UCS Director Upgrade Preparation Steps:

The first step that is always recommended is to take a backup of the database for UCS Director. This can be done via the console under the shelladmin login. Take a snapshot within vCenter also.

UCSD BareMetal Agent Upgrade Snapshot

UCSD BareMetal Agent Upgrade Snapshot 1

Open a console into vCenter and login.
UCSD BareMetal Agent Upgrade Step 1

Log in using the shelladmin account.

UCSD BareMetal Agent Upgrade Step 2

Select option 3 to stop all services and select Y at the prompt.

UCSD BareMetal Agent Upgrade Step 3

Select option 2 to ensure that all the services have been stopped.

UCSD BareMetal Agent Upgrade Step 4

Then exit the shelladmin login and select Configure Network and give the current UCSD appliances a new IP address so the current one can be re-used.

UCSD BareMetal Agent Upgrade Step 5

Select n for a DHCP address and enter the new IP address. I’ve had to mask the actual addresses here but you can see that the final octet has changed from .54 to .235. Enter the DNS servers etc and commit the change of IP address. Make sure you can ping the appliance on its new IP address.

UCSD BareMetal Agent Upgrade Step 5

 

UCS Director Deployment Steps:

In vCenter select File -> Deploy OVF Template

UCSD BareMetal Agent Upgrade Step 6

Select the extracted OVF file from the earlier download of the appliance.

UCSD BareMetal Agent Upgrade Step 7

Have a look over the requirements and click Next.

UCSD BareMetal Agent Upgrade Step 8

Accept the license and click Next

UCSD BareMetal Agent Upgrade Step 9

Select the Host and Cluster to deploy the new appliance to. Enter a name for the appliance and click Next.

UCSD BareMetal Agent Upgrade Step 10

Select the relevant resource pool and click Next

UCSD BareMetal Agent Upgrade Step 11

Select the datastore location for where you want to place the appliance.

UCSD BareMetal Agent Upgrade Step 12

Select which disk format you want to use. In this instance I left the disk format defaults. Click Next.

UCSD BareMetal Agent Upgrade Step 13

Select the network to assign to the appliance and click Next.

UCSD BareMetal Agent Upgrade Step 14

Enter the IP address address and put in the root and shelladmin passwords. The IP address used here is the one from the old appliance. It is the one that ends in .54. Once all the information has been entered click Next.

UCSD BareMetal Agent Upgrade Step 16

Lastly, verify all the information and click Finish

UCSD BareMetal Agent Upgrade Step 17

Once the appliance has finished installing power it on. You’ll see the database creating the required  schemas

UCSD BareMetal Agent Upgrade Step 18

Log into the new appliance using the root account created during the OVF deployment. As the IP address is the one used previously by the 5.3 version it is recommended to run the /opt/scripts/configStaticIP.sh. Fill in the details as required. In the screenshot below it shows I also ran the configStaticIP.sh on the old appliance to ensure that it also retained its static address. The same entries will be needed for the new appliance. This is a recommendation from the Cisco documentation.

UCSD BareMetal Agent Upgrade Step 19

Once the IP has been set log out of the console and you’ll see an Emerald Bay UCSD 5.4.0 title for the appliance as well as the required web console login.

UCSD BareMetal Agent Upgrade Step 20

Open a browser and log in using the admin account. It will appear as a blank system. Now that the new appliance is installed the files needed can be migrated/copied from the old appliance before the old one is decommissioned.

UCSD BareMetal Agent Upgrade Step 21

 

Migrate Data between UCS Director versions:

Log back into the new appliance using the root account. Run the script  /opt/infra/migration/performMigration.sh to migrate the database and configuration files from the 5.3 appliance to the 5.4 release. The script backs up the database on the old appliance and restores it to the new one. It also executes the scripts required to upgrade the database.

UCSD BareMetal Agent Upgrade Step 22

When prompted, provide the IP address of the old appliance.

UCSD BareMetal Agent Upgrade Step 23The services on the old appliance will be requested to be shut down during the migration process.

UCSD BareMetal Agent Upgrade Step 20

Once the migration process has been completed you’ll be prompted to start the services using shelladmin on the new appliance.

UCSD BareMetal Agent Upgrade Step 24

Select option 4 to restart all of the services.

UCSD BareMetal Agent Upgrade Step 25

Run option 2 to verify that the services have started.

UCSD BareMetal Agent Upgrade Step 26

Once the migration script has completed you may need to copy some files manually from the old appliance. You’ll need to review the files to move. Enable the SSHD service on both the old and new UCS Director appliances. Check the old version against the new version to see if any of the following files need to be copied. If not, ignore them. Open WinSCP sessions to both and copy these files:

  • /etc/ntp.conf
  • mysql tuning in /etc/my.cnf

In my case the ntp.conf was different but I just edited the new file to point to internal NTP servers

Changes in the following properties files:

  • /opt/infra/client/service.properties
  • /opt/infra/inframgr/service.properties
  • /opt/infra/idaccessmgr/service.properties
  • /opt/infra/web_cloudmgr/apache-tomcat/bin/service.properties
  • /opt/infra/controller/service.properties
  • /opt/infra/eventmgr/service.properties
  • /opt/infra/networkServices/service.properties
  • /opt/infra/usageMgr/service.properties
  • /opt/infra/inframgr/hyperv.properties

Migrate the contents from the/opt/infra/uploads folder. This folder contains any files and OVFs uploaded from the Cisco UCS Director user interface. To migrate the files I ran a WinSCP session to download the required files locally and upload them to the new appliance via another WinSCP session. When prompted to overwrite existing files click Yes.

UCSD BareMetal Agent Upgrade Step 27

UCSD BareMetal Agent Upgrade Step 28

UCSD BareMetal Agent Upgrade Step 29

At this point I restarted the entire appliance to just start with a fresh slate. While doing this I powered down the old appliance so there were no potential clashes. I recommend to rename the old appliances to _temp and modify the VM names of the deployed VMs to the existing VM names. After the reboot log into UCS Director via the web console.

UCSD BareMetal Agent Upgrade Step 30

Click About and verify the version you are running.

UCSD BareMetal Agent Upgrade Step 32

You have now successfully migrated to 5.4.0. The last thing to do is to configure a weekly backup of the database. Once the script has been configured just run the job manually and ensure that the backup reaches your FTP server.

 

BareMetal Agent Upgrade Steps:

Once the main UCSD is done log into UCSD BMA using the root account and run the /opt/infra/configureInterface.sh script for eth0 so that its a different IP address. The new appliance being deployed will take over the current IP address assigned to BMA.

UCSD BareMetal Agent Upgrade Step 33

Then run the command ./stopInfraAll.sh to shut down all services.

UCSD BareMetal Agent Upgrade Step 34

Next we can begin the deployment of the new appliance. Go to File -> Deploy OVF template

UCSD BareMetal Agent Upgrade Step 35

Select the BMA OVF that was downloaded earlier.

UCSD BareMetal Agent Upgrade Step 36Click Next to continue the deployment process

UCSD BareMetal Agent Upgrade Step 37

Review the details of the appliance and click Next

UCSD BareMetal Agent Upgrade Step 38

Accept the license and click Next

UCSD BareMetal Agent Upgrade Step 39

Enter a name for the appliance and click Next

UCSD BareMetal Agent Upgrade Step 40

Select the resource pool to deploy the appliance to.

UCSD BareMetal Agent Upgrade Step 41

Select the relevant datastore and click Next.

UCSD BareMetal Agent Upgrade Step 42

Check the disk format and select whichever format fits to your environment.

UCSD BareMetal Agent Upgrade Step 43

At this point there’s a need to select the relevant networks. One is required for management and one is required for PXE boot. Click Next.

UCSD BareMetal Agent Upgrade Step 46

Enter details for the passwords for the accounts and the IP address of the old appliance. Click Next to continue.

UCSD BareMetal Agent Upgrade Step 47

Review the details of the appliance and click Finish to complete the deployment.

UCSD BareMetal Agent Upgrade Step 48

Open the console session and verify the correct IP address is assigned.

UCSD BareMetal Agent Upgrade Step 49

Open a web browser and verify that you can at least see the 5.4.0.0 version showing up.

UCSD BareMetal Agent Upgrade Step 50

Log into the new appliance with the root account.

UCSD BareMetal Agent Upgrade Step 51Once logged in run /opt/infra/migration/performBMAMigration.sh script to migrate the images, templates, and configuration files from the 5.3 BMA appliance to the 5.4 BMA appliance.

UCSD BareMetal Agent Upgrade Step 52The migration script will then go and copy all the required files.

UCSD BareMetal Agent Upgrade Step 53Once it has been completed log in toCisco UCS Director as an admin and verify that all the images from the 5.3 BMA has been migrated successfully. You may need to refresh the view to make sure it shows up correctly.

As with the main UCS Director appliance you’ll need to copy some files with WinSCP. Copy the following files from the old appliance using WinSCP

  • /etc/resolv.conf
  • /etc/hosts
  • mysql tuning in /etc/my.cnf
  • /opt/infra/networkServices/service.properties

Reboot the new appliance and shutdown the old version

(Optional)

For any Windows images migrated from the 5.3 BMA update the WinPE file with the new IP address of the 5.4 BMA. You do not need to edit WinPE if the Baremetal Agent IP address was not changed.

If you use a separate interface for PXE Traffic you’ll need to configure the PXE interface IP using the Cisco UCS Director UI if you haven’t reused the 5.3 BMA appliance PXE IP address. Otherwise it will be fine.

Using the Configure Interface option, select the BMA account and copy the /etc/dhcpd.conf file from the 5.3 BMA appliance and reconfigure it based on the new PXE Interface IP address.

Finalise Deployment:

The last steps involve renaming any of the appliances as needed and then once the new appliance has been confirmed as working correctly delete the old appliances from the environment. And that’s it, everything has been updated. It’s a real pain to have to deploy new appliances and hopefully this will be addressed in upcoming versions.

Note: I just noticed as I was writing this post that a new 5.4.2 patch has been released. I’ll have to start looking at upgrading that patch also.

Leave a Reply

Your email address will not be published. Required fields are marked *