How to Patch Oracle RAC Database 19c Using Manual Out-Of-Place

Uncategorized 7 Minutes

Let me show you how I patch an Oracle RAC Database 19c using out-of-place patching and by doing it manually – that is without using OPatchAuto.

The advantages of this solution:

  • I get more control over the process which is handy for automation
  • I can create my Oracle homes using gold images
  • Or, I can use brand-new Oracle homes instead of cloning existing Oracle homes which OPatchAuto does

My Demo System

  • 2 node Oracle RAC Database
    • Nodes: copenhagen1 and copenhagen2
    • SID: CDB1
    • Instances: CDB11 and CDB12
  • Runs Oracle Linux
  • GI home is already on 19.26
  • Database home is on 19.25

I want to:

  • Patch my database to 19.26 in a rolling manner
  • Patch just the database – GI home is already patched
  • Patch manually without using OPatchAuto

Preparation

I need to download the following to /u01/software on copenhagen1:

  1. The base release of:
    • Oracle Database (LINUX.X64_193000_db_home.zip)
  2. Latest OPatch from My Oracle Support (6880880).
  3. Patches from My Oracle Support:
    • 19.26 Release Update for Grid Infrastructure (37257886)

You can use AutoUpgrade to easily download GI patches.

I have already established SSH equivalence between the two nodes.

How to

1. Prepare a New Database Home

I can do this in advance. It doesn’t affect my current environment and doesn’t cause any downtime.

  1. I need to create a folder for the new database home. I must do this as oracle on both nodes, copenhagen1:
    [oracle@copenhagen1]$ export NEW_ORACLE_HOME=/u01/app/oracle/product/dbhome_1926
[oracle@copenhagen1]$ mkdir -p $NEW_ORACLE_HOME
    copenhagen2:
    [oracle@copenhagen2]$ export NEW_ORACLE_HOME=/u01/app/oracle/product/dbhome_1926
[oracle@copenhagen2]$ mkdir -p $NEW_ORACLE_HOME
  2. I extract the base release of Oracle Database into the new database home on the first node only:
    [oracle@copenhagen1]$ cd $NEW_ORACLE_HOME
[oracle@copenhagen1]$ unzip -oq /u01/software/LINUX.X64_193000_db_home.zip
    Optionally, I can use a golden image. You can use a gold image from a single instance install. The Oracle home is identical.
  3. I update OPatch to the latest version:
    [oracle@copenhagen1]$ rm -rf OPatch
[oracle@copenhagen1]$ unzip -oq /u01/software/p6880880_190000_Linux-x86-64.zip
  4. I want to apply the 19.26 Release Update while I install the database home. To do that, I must extract the patch file:
     [grid@copenhagen1]$ cd /u01/software
 [grid@copenhagen1]$ unzip -oq p37257886_190000_Linux-x86-64.zip -d 37257886
    • I must use the GI Release Update, not the database Release Update. The GI Release Update is a bundle patch consisting of the database and OCW Release Updates. I must apply both to my database home.
  5. Then, I can install the new database home and apply the patches at the same time:
    • The variable CLUSTER_NODES contains a comma separated list of all nodes in my cluster.
    • The parameter -applyRU is the path to the Database Release Update.
    • The parameter -applyOneOffs is the paths to the OCW Release Update.
    [oracle@copenhagen1]$ export CV_ASSUME_DISTID=OEL7.8
[oracle@copenhagen1]$ export ORACLE_HOSTNAME=$(hostname)
[oracle@copenhagen1]$ export CLUSTER_NODES=copenhagen1,copenhagen2
[oracle@copenhagen1]$ export ORACLE_BASE=/u01/app/oracle
[oracle@copenhagen1]$ export ORA_INVENTORY=/u01/app/oraInventory
[oracle@copenhagen1]$ export OLD_ORACLE_HOME=$ORACLE_HOME
[oracle@copenhagen1]$ export ORACLE_HOME=$NEW_ORACLE_HOME
[oracle@copenhagen1]$ cd $ORACLE_HOME
[oracle@copenhagen1]$ ./runInstaller -ignorePrereqFailure -waitforcompletion -silent \
     -responseFile $ORACLE_HOME/install/response/db_install.rsp \
     -applyRU /u01/software/37257886/37260974 \
     -applyOneOffs /u01/software/37257886/37268031 \
     oracle.install.option=INSTALL_DB_SWONLY \
     ORACLE_HOSTNAME=$ORACLE_HOSTNAME \
     UNIX_GROUP_NAME=oinstall \
     INVENTORY_LOCATION=$ORA_INVENTORY \
     SELECTED_LANGUAGES=en \
     ORACLE_HOME=$ORACLE_HOME \
     ORACLE_BASE=$ORACLE_BASE \
     oracle.install.db.InstallEdition=EE \
     oracle.install.db.OSDBA_GROUP=dba \
     oracle.install.db.OSBACKUPDBA_GROUP=dba \
     oracle.install.db.OSDGDBA_GROUP=dba \
     oracle.install.db.OSKMDBA_GROUP=dba \
     oracle.install.db.OSRACDBA_GROUP=dba \
     oracle.install.db.CLUSTER_NODES=$CLUSTER_NODES \
     oracle.install.db.isRACOneInstall=false \
     oracle.install.db.rac.serverpoolCardinality=0 \
     SECURITY_UPDATES_VIA_MYORACLESUPPORT=false \
     DECLINE_SECURITY_UPDATES=true
    • I install it in silent mode, but I could use the wizard instead.
    • You need to install the new database home in a way that matches your environment.
    • For inspiration, you can check the response file used in the previous database home on setting the various parameters.
    • If I have additional one-off patches to install, I add them to the comma-separated list.
  6. I run the root script on all nodes, copenhagen1:
    [root@copenhagen1]$ export NEW_ORACLE_HOME=/u01/app/oracle/product/dbhome_1926
[root@copenhagen1]$ NEW_ORACLE_HOME/root.sh
    copenhagen2:
    [root@copenhagen2]$ export NEW_ORACLE_HOME=/u01/app/oracle/product/dbhome_1926
[root@copenhagen2]$ $NEW_ORACLE_HOME/root.sh

2. Prepare Database

I can do this in advance. It doesn’t affect my current environment and doesn’t cause any downtime.

I will move the database into a new Oracle home, so I need to ensure the database configuration files are either outside the Oracle home or move them to the new Oracle home.

  1. I verify that my SP file and password file are stored in ASM – or at least outside the Oracle home:
    [oracle@copenhagen1]$ export ORACLE_HOME=$OLD_ORACLE_HOME
[oracle@copenhagen1]$ srvctl config database -db $ORACLE_UNQNAME | grep file
    • If the files are stored in the dbs folder, I copy them to new Oracle home on both nodes.
  2. I copy tnsnames.ora and sqlnet.ora to the new Oracle home on both nodes, copenhagen1:
    [oracle@copenhagen1]$ cp $OLD_ORACLE_HOME/network/admin/sqlnet.ora $NEW_ORACLE_HOME/network/admin
[oracle@copenhagen1]$ cp $OLD_ORACLE_HOME/network/admin/tnsnames.ora $NEW_ORACLE_HOME/network/admin
    copenhagen2:
    [oracle@copenhagen2]$ cp $OLD_ORACLE_HOME/network/admin/sqlnet.ora $NEW_ORACLE_HOME/network/admin
[oracle@copenhagen2]$ cp $OLD_ORACLE_HOME/network/admin/tnsnames.ora $NEW_ORACLE_HOME/network/admin
  3. I take care of any other configuration files in the Oracle home.
  4. I modify the database so it starts in the new Oracle home on the next restart.
    [oracle@copenhagen1]$ srvctl modify database -d $ORACLE_UNQNAME -o $NEW_ORACLE_HOME

3. Restart Instances

Now, I restart each instance in a rolling manner. On restart, the database starts in the new Oracle home. There is no database outage, however, each instance needs to restart.

Maintenance window starts now!

  1. I start draining the first instance, CDB11, running on copenhagen1:
    [oracle@copenhagen1]$ export ORACLE_HOME=$OLD_ORACLE_HOME
[oracle@copenhagen1]$ export PATH=$ORACLE_HOME/bin:$PATH
$ORACLE_HOME/bin/srvctl stop service \
   -d $ORACLE_UNQNAME \
   -service SALESGOLD \
   -drain_timeout 60 \
   -stopoption IMMEDIATE \
   -node copenhagen1
    • In this example, I have just one service that I’m shutting down. You probably have other services and other drain settings. You can adjust accordingly.
  2. After draining, I can shut down the instance, CDB11, on copenhagen1:
    [oracle@copenhagen1]$ $ORACLE_HOME/bin/srvctl stop instance \
   -d $ORACLE_UNQNAME \
   -i CDB11 \
   -force
  3. I immediately restart the instance:
    [oracle@copenhagen1]$ $ORACLE_HOME/bin/srvctl start instance \
   -d $ORACLE_UNQNAME \
   -i CDB11
    • The instance now restarts in the new Oracle home.
  4. I repeat the drain and restart cycle on the second instance, CDB12, running on copenhagen2. Repeat steps 1-3 but change any reference to the node and instance:
    • Change -node to copenhagen2
    • Change -i to CDB12
  5. I have now restarted all instances and they are running in the new Oracle home.
  6. I update any profiles (e.g., .bash_profile) and other scripts referring to the database home on all nodes.
  7. If the database is added to /etc/oratab, I change the file accordingly on all nodes.

4. Complete Patching

  1. I complete patching of the database by running Datapatch (ensure the environment is set correctly):
    [oracle@copenhagen1]$ env | grep ORA
[oracle@copenhagen1]$ $ORACLE_HOME/OPatch/datapatch
    • I can do this while users are connected to the database.

Most likely, there are other changes that you need to make in your own environment:

  • Update Enterprise Manager registration
  • Upgrade RMAN catalog

That’s it! I have now patched my Oracle RAC Database.

Happy Patching!

Appendix

Deinstall

In the future, I should remove the old Oracle homes. I use the deinstall tool in the respective Oracle homes.

I would recommend waiting a week or two until I’m confident the new Release Updates are fine.

Rollback

If you need to roll back, you just reverse the process. Restart the instances in the old Oracle home and let Datapatch perform the rollback of the database changes.

Combine with SwitchGridHome

You can combine this method with SwitchGridHome for Grid Infrastructure patching.

Patch with OPatchAuto

An alternative to this method is to use OPatchAuto and out-of-place patching. Check the patch readme for instructions on how to use that method.

Disable Binary Options

If you need to disable binary options using make, you can do it immediately after installing the new Oracle home. Do it on all nodes.

Other Blog Posts in This Series

Published

9 thoughts on “How to Patch Oracle RAC Database 19c Using Manual Out-Of-Place”

  1. hi Daniel,

    is dba_registry view going to show us the full version as 19.26? Moving of database services doesn’t implicitly runs the sql files which are associated with the patches. Eventually we may end up running ./datapatch -verbose to update the registry.

    Like

    Reply

  2. Hi Daniel,

    I justed wanted to let you know, that support is pushing towards the opatchauto process. I had an issue with the OOP method for SIHA GI home (see https://dohdatabase.com/2024/11/12/how-to-patch-oracle-restart-19c-and-oracle-database-using-out-of-place-switch-home/). And support was pushing me towards the OPatchAuto method without accepting, that it might be a bug which is worth fixing even in that method.I like and prefer that manual method for the same reasons. But all of oracle should promote the same methods. It is no good to have and maintain two methods for the same goal.

    In case you are curious, I can share the SR. Just let me know.

    Regards,Robert

    Like

    Reply

    1. Hi Robert,

      I’m sad to hear that you’re having a hard time with support finding a solution to your problem.

      The SwitchGridHome is a fully supported method for patching Oracle RAC and there’s a similar method for Oracle Restart. I’m not aware of any particularities with SIHA, but support should be able to clear that out.

      To my knowledge it is a fully supported method, so unless support comes up with a statement telling you otherwise, I’d ask them to help you come up with a solution with SwitchGridHome.

      I hope that helps,
      Daniel

      Like

      Reply

  3. Hi Daniel,

    Thank you for sharing input concerning the Patching processes, got a question if the Database uses Oracle Text, does it get affected or no changes, since when i was reviewing ur Standalone process u ve shared a Command to run.

    Thank you for your Assistance & Your Knowledge sharing the Oracle Community.

    Like

    Reply

  4. Hi Daniel,

    thank you for Sharing the Process of Oracle RAC Homes Patching, got a question do we have to run @?/ctx/admin/ctx_oh_files for the oracle text (if used) and move them to the new Oracle Home also?

    Thank you for your Time.

    Like

    Reply

Leave a reply to Daniel Overby Hansen Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.