ODAVP: Upgrading from 12.1.2.12 to 12.2.1.4

This note is about upgrading a virtualized ODA from 12.1.2.12 to 12.2.1.4.0.

#1 Download the update patch 28216780

#2 Upload the patch to the two ODA_BASE servers in /tmp/patch

#3 From the 2 nodes, run the oakcli command below

# oakcli unpack -package /tmp/patch/p28216780_122140_Linux-x86-64_1of3.zip 
# oakcli unpack -package /tmp/patch/p28216780_122140_Linux-x86-64_2of3.zip
# oakcli unpack -package /tmp/patch/p28216780_122140_Linux-x86-64_3of3.zip

#4 Remove the bundle patches from both node, and make sure there is enough space on /tmp

#5 Validate the system from the first oda_base node:

# oakcli validate -a

#6 From both ODA_BASE nodes, verify the OS patch

# oakcli validate -c ospatch -ver 12.2.1.4.0
INFO: Validating the OS patch for the version 12.2.1.4.0
INFO: 2018-11-30 13:52:17: Performing a dry run for OS patching
INFO: 2018-11-30 13:52:36: There are no conflicts. OS upgrade could be successful

#7 From both ODA_BASE nodes verify the list of components to be patched

# oakcli update -patch 12.2.1.4.0 --verify
INFO: 2018-11-30 13:56:59: Reading the metadata file now...
                Component Name            Installed Version         Proposed Patch Version
                ---------------           ------------------        -----------------
                Controller_INT            11.05.03.00               Up-to-date
                Controller_EXT            11.05.03.00               Up-to-date
                Expander                  0018                      Up-to-date
                SSD_SHARED                944A                      Up-to-date
                HDD_LOCAL                 A7E0                      Up-to-date
                HDD_SHARED                A7E0                      Up-to-date
                ILOM                      3.2.9.22 r115963          4.0.2.27.a r123795
                BIOS                      25040100                  25060300
                IPMI                      1.8.12.4                  Up-to-date
                HMP                       2.3.5.2.8                 2.4.1.0.11
                OAK                       12.1.2.12.0               12.2.1.4.0
                OL                        6.8                       6.9
                OVM                       3.4.3                     3.4.4
                GI_HOME                   12.1.0.2.170814(2660      12.2.0.1.180417(2767
                                          9783,26609945)            4384,27464465)
                DB_HOME                   12.1.0.2.170814(2660      12.1.0.2.180417(2733
                                          9783,26609945)            8029,27338020)

#8 From the first ODA_BASE node to apply the server patch. This patch will also upgrade the clusterware.

# oakcli update -patch 12.2.1.4.0 --server

This operation should last 2 hours and reboot the servers.

This is the portion that should cause the more troubles. There is no magic bullet if things start going south, but in case of error installing the clusterware, leaving one node only being upgraded, restart the patch in -local node on the remaining node. Doublecheck that is enough disk space on /OVS and that the .bashrc_profile file does not include an outdated GRID_HOME.

To restart on local:

# oakcli update -patch 12.2.1.4.0 --clean
# oakcli update -patch 12.2.1.4.0 --server -local

Also whenever necessary, the oak files may be tweaked temporarily to help with the patching, for example /opt/oracle/oak/lib/oakutils.pm  may be updated to change the wait time for the clusterwait to start from 30 to 180, or /opt/oracle/oak/pkgrepos/System/12.1.2.12.0/bin/os_update.pm may be changed to avoid the clusterware to be stopped when the ospatch is already applied, for example:

# stop the clusterware if it is running
# check if GI is installed before trying to stop.
# Comment the lines below:
# if ( (! $dryrun) && is_gi_installed() ) {
# my $stop_status = stopCrsOnLocalNode();
# if ($stop_status == SUCCESS)
# {
# # create a flag file which says clusterware restart is required
# createClsStartFile();
# }
# elsif ($stop_status == FAILED)
# {
# $dbgOsObj.trace(W, "Stop the clusterware on the Local Node before applying the OS patch");
# $dbgOsObj.trace(W, "Please stop the clusterware and its resources [Database, ASM, e.t.c] then apply the patch");
# return FAILED;
# }

#9 From the first ODA_BASE nodes apply the storage patch:

# oakcli update -patch 12.2.1.4.0 --storage

#10 From the first ODA_BASE nodes apply the database patch:

# oakcli update -patch 12.2.1.4.0 --database

#11 From the two ODA_BASE nodes, very again the components versions:

# oakcli update -patch 12.2.1.4.0 --verify
INFO: 2018-12-03 20:37:26: Reading the metadata file now...
                Component Name            Installed Version         Proposed Patch Version
                ---------------           ------------------        -----------------
                Controller_INT            11.05.03.00               Up-to-date
                Controller_EXT            11.05.03.00               Up-to-date
                Expander                  0018                      Up-to-date
                SSD_SHARED                944A                      Up-to-date
                HDD_LOCAL                 A7E0                      Up-to-date
                HDD_SHARED                A7E0                      Up-to-date
                ILOM                      4.0.2.27.a r123795        Up-to-date
                BIOS                      25060300                  Up-to-date
                IPMI                      1.8.12.4                  Up-to-date
                HMP                       2.4.1.0.11                Up-to-date
                OAK                       12.2.1.4.0                Up-to-date
                OL                        6.9                       Up-to-date
                OVM                       3.4.4                     Up-to-date
                GI_HOME                   12.2.0.1.180417(2767      Up-to-date
                                          4384,27464465)
                DB_HOME                   12.1.0.2.180417(2733      Up-to-date
                                          8029,27338020)

# oakcli show version -detail
Reading the metadata. It takes a while...
System Version  Component Name            Installed Version         Supported Version
--------------  ---------------           ------------------        -----------------
12.2.1.4.0
                Controller_INT            11.05.03.00               Up-to-date
                Controller_EXT            11.05.03.00               Up-to-date
                Expander                  0018                      Up-to-date
                SSD_SHARED                944A                      Up-to-date
                HDD_LOCAL                 A7E0                      Up-to-date
                HDD_SHARED                A7E0                      Up-to-date
                ILOM                      4.0.2.27.a r123795        Up-to-date
                BIOS                      25060300                  Up-to-date
                IPMI                      1.8.12.4                  Up-to-date
                HMP                       2.4.1.0.11                Up-to-date
                OAK                       12.2.1.4.0                Up-to-date
                OL                        6.9                       Up-to-date
                OVM                       3.4.4                     Up-to-date
                GI_HOME                   12.2.0.1.180417(2767      Up-to-date
                                          4384,27464465)
                DB_HOME                   12.1.0.2.180417(2733      Up-to-date
                                          8029,27338020)