I strongly recommend that you always patch out-of-place. Here’s an example of how to do it on Oracle Grid Infrastructure (GI) and Oracle Data Guard using Standby-First Patch Apply.
Standby-First Patch Apply allows you to minimize downtime to the time it takes to perform a Data Guard switchover. Further, it allows you to test the apply mechanism on the standby database by temporarily converting it into a snapshot standby database.
The scenario:
- Oracle Grid Infrastructure 19c and Oracle Database 19c
- Patching from Release Update 19.17.0 to 19.19.0
- Vertical patching – GI and database at the same time
- Data Guard setup with two RAC databases
- Cluster 1: copenhagen1 and copenhagen2
- Cluster 2: aarhus1 and aarhus2
- DB_NAME: CDB1
- DB_UNIQUE_NAME: CDB1_COPENHAGEN and CDB1_AARHUS
- Using Data Guard broker
- Patching GI using SwitchGridHome method
Let’s get started!
Step 1: Prepare
I can make the preparations without interrupting the database.
-
I ensure my environment meets the requirements for Standby-First Patch Apply.
-
I deploy new GI homes to all four hosts.
- I use the SwitchGridHome method.
- Very important: I only perform step 1 (Prepare a New Grid Home).
- I apply the Release Update 19.19.0 as part of the deployment using
gridSetup.sh ... -applyRU
as described in the blog post.
-
I deploy new database homes to all four hosts.
- I prefer to use brand-new database homes. If you clone an existing database home, be sure to clean it up before you proceed.
- When I attach the new database home to the inventory, I also apply the Release Update 19.19.0 using
runInstaller ... -applyRU
.
-
I also recompile invalid objects. This can make it easier for Datapatch later in the process:
PRIMARY SQL> @?/rdbms/admin/utlrp
Step 2: Restart Standby in New Oracle Homes
Now, I can move the standby database to the new GI and database homes.
-
On the standby hosts, aarhus1 and aarhus2, I first move the database configuration files from the old database home to the new one.
-
I change the database configuration in GI. Next time the database restarts, it will be in the new Oracle Home:
[oracle@aarhus1]$ $OLD_ORACLE_HOME/bin/srvctl modify database \ -db $ORACLE_UNQNAME \ -oraclehome $NEW_ORACLE_HOME
-
I switch to the new GI on all standby hosts, aarhus1 and aarhus2, by executing step 2 (Switch to the new Grid Home) of the SwitchGridHome method.
- It involves running
gridSetup.sh ... -switchGridHome
androot.sh
. - You can perform the switch in a rolling manner or all at once.
- The switch restarts the standby database instance. The standby database instance restarts in the new Oracle Home.
- If the profile of grid (like .bashrc) sets the ORACLE_HOME environment variable, I ensure to update it.
- It involves running
-
If I had multiple standby databases, I would process all standby databases in this step.
Step 3: Test Standby Database
This is an optional step, but I recommend that you do it.
- I convert the standby database (CDB1_AARHUS) to a snapshot standby database:
DGMGRL> convert database CDB1_AARHUS to snapshot standby;
- I test Datapatch on the standby database. It is important that I run the command on the standby database:
[oracle@aarhus1]$ $ORACLE_HOME/OPatch/datapatch -verbose
- I can also test my application on the standby database.
- At the end of my testing, I revert the standby database to a physical standby database. The database automatically reverts all the changes made during testing:
DGMGRL> convert database CDB1_AARHUS to physical standby;
Step 4: Switchover
I can perform the previous steps without interrupting my users. This step requires a maintenance window because I am doing a Data Guard switchover.
- I check that my standby database is ready to become primary. Then, I start a Data Guard switchover:
DGMGRL> connect sys/<password> as sysdba DGMGRL> validate database CDB1_AARHUS; DGMGRL> switchover to CDB1_AARHUS;
A switchover does not have to mean downtime.
If my application is configured properly, the users will experience a brownout; a short hang, while the connections switch to the new primary database.
Step 5: Restart New Standby in New Oracle Homes
Now, the primary database runs on aarhus1 and aarhus2. Next, I can move the new standby hosts, copenhagen1 and copenhagen2, to the new GI and database homes.
- I repeat step 2 (Restart Standby In New Oracle Homes) but this time for the new standby hosts, copenhagen1 and copenhagen2.
Step 6: Complete Patching
Now, both databases in my Data Guard configuration run out of the new Oracle Homes.
Only proceed with this step once all databases run out of the new Oracle Home.
I need to run this step as fast as possible after I have completed the previous step.
-
I complete the patching by running Datapatch on the primary database (CDB1_AARHUS). I add the
recomp_threshold
parameter to ensure Datapatch recompiles all objects that the patching invalidated:[orale@aarhus1]$ $ORACLE_HOME/OPatch/datapatch \ -verbose \ -recomp_threshold 10000
- I only need to run Datapatch one time. On the primary database and only on one of the instances.
-
I can run Datapatch while users are connected to my database.
-
Optionally, I can switch back to the original primary database on copenhagen1 and copenhagen2, if I prefer to run it there.
That’s it. Happy patching!
Appendix
Further Reading
- Continuous Availability – MAA Checklist for Applications for the Oracle Database
- Oracle Patch Assurance – Data Guard Standby-First Patch Apply (Doc ID 1265700.1)
Other Blog Posts in This Series
- Introduction
- How to Patch Oracle Grid Infrastructure 19c Using In-Place OPatchAuto
- How to Patch Oracle Grid Infrastructure 19c Using Out-Of-Place OPatchAuto
- How to Patch Oracle Grid Infrastructure 19c Using Out-Of-Place SwitchGridHome
- How to Patch Oracle Grid Infrastructure 19c and Oracle Data Guard Using Standby-First
- How to Patch Oracle Grid Infrastructure 19c Using Zero Downtime Oracle Grid Infrastructure Patching
- Which Method Should I Choose When Patching Oracle Grid Infrastructure 19c
- How to Avoid Interruptions When You Patch Oracle Grid Infrastructure 19c
- Patching Oracle Grid Infrastructure And Oracle Data Guard
- How to Clone Oracle Grid Infrastructure Home Using Golden Images
- How to Roll Back Oracle Grid Infrastructure 19c Using SwitchGridHome
- How to Remove an Old Oracle Grid Infrastructure 19c Home
- Use Cluster Verification Utility (cluvfy) and Avoid Surprises
- A Word about Zero Downtime Oracle Grid Infrastructure Patching
- Why You Need to Use Oracle Fleet Patching and Provisioning
- My Best Advice on Patching Oracle Grid Infrastructure
- Pro Tips
Hi, wondering if Oracle Restart or single-node Oracle GI cluster was used in this scenario? MOS Doc ID 2853839.1 says “-switchGridHome” is not applicable to Oracle Restart.
LikeLike
Hi Edmundas,
I used single instance databases managed by GI. Oracle Restart does not support this method.
Regards,
Daniel
LikeLike
Daniel, thanks for the info.
I did a new install of Oracle19c Data Guard instead of upgrading my Oracle12c Data Guard, I am using Protection Mode=MaxPerformance in 12c, but in 19c Protection Mode=MaxPerformance I see the message “Fast-Start Failover: Enabled in Potential Data Loss Mode”, is this a BUG?
regards,
jorge
LikeLike
Hi Jorge,
I don’t know. For such questions on Data Guard, I suggest that you reach out to the Data Guard Product Manager, Ludovico Caldera. You can find him on Twitter as @ludodba.
Regards,
Daniel
LikeLike
Hej Daniel,
Thanks for your post.
But I still have a question regarding the use of SwitchGridHome method in Single Instance DB managed by GI. You mentioned that Oracle Restart is NOT supported with SwitchGridHome method . So does it mean, we need to disable Oracle Restart in case we want to use SwitchGridHome method? As of now we follow below steps for GoldImage based GI Patching ! Can we replace them with SwitchGridHome ?
After extracting GoldImage of Target Home – For Example: 19.19 GoldImage on Target Server, we follow below steps.
1. Execute gridSetup.sh (From Target Home)
2. Execute root.sh (From Target Home)
3. roothas.sh -prepatch -dstcrshome $TARGET_GRID_HOME_RU (From Target Home)
4. roothas.sh -postpatch -dstcrshome $TARGET_GRID_HOME_RU (From Target Home)
5. ./oui/bin/runInstaller -updateNodeList ORACLE_HOME=$TARGET_GRID_HOME_RU CRS=”true” (From Target Home)
6. ./oui/bin/runInstaller -updateNodeList ORACLE_HOME=$SOURCE_GRID_HOME CRS=”false” (From Source Home)
7. srvctl modify listener -l LISTENER -o $TARGET_GRID_HOME_RU
8. Stop/Start Listener
Tak på forhånd
LikeLike
Hi,
The procedure I describe work for Grid Infrastructure only – a RAC managed system. If you have a single instance database, then the procedure is not supported. You have to use another approach:
https://docs.oracle.com/en/database/oracle/oracle-database/19/ladbi/switch-gi-home-patching.html#GUID-27EBDA86-24B2-407B-A4AD-C0542B472F90
I have it on my list of blog posts to write. But that list keeps growing and growing and ….. 🙂
Regards,
Daniel
LikeLike
in step 3 of How to Apply Patches Out-of-place to Oracle Grid Infrastructure and Oracle Data Guard Using Standby-First, it mentions that the step is optional
is that for all three components of that step or just for the testing… if it is optional, then when does datapatch acutally get run (is it at the end of the full process?)
LikeLike
Hi Ray,
It is all of step 3 that is optional. Since this is for testing only, you can run Datapatch even though the primary database hasn’t been patched yet. With Datapatch you can test the SQL part of the patch apply on the standby database only. When you’re done testing all changes are reverted when you convert back to physical standby database (as if Datapatch hasn’t been executed at all).
Then at the end of step 6 – when all databases in the Data Guard configuration is on the new Oracle Home, you run Datapatch on the primary databases. The SQL patches then propagates to the standby databases via redo.
Regards,
Daniel
LikeLike
Hi Daniel, Im currently busy with the next RU patch cycle. Of course Im using OOP method (even though I prefer opatchauto up to now 😉) and Standby-first if possible.
For one topic I still did not find a corresponding note within the documentation: after applying the RU on the primary (and maybe additional One-Offs, e.g. the DP-bundle) I usually enable the optimizer fixes on the primary. For the db related thing its fine due to redo but what about the param _fix_control? It has to be set to the spfile of the standby accordingly, right?
Thanks & regards
Axel D.
LikeLike
Hi Axel,
When you enable optimizer fixes on primary database, the changes do not apply to the standby database. To use DBMS_OPTIM_BUNDLE on the standby database, you must open it (read-only – no redo apply if you don’t have Active Data Guard) or switchover to the standby database to use the package.
An alternative is to find the _fix_control setting in the SPFile on the primary and set the same parameter on the standby database using ALTER SYSTEM.
Regards,
Daniel
LikeLike