Upgrade and Data Guard

You can upgrade your Oracle Database to a new release and keep the Data Guard setup intact. There is no need to rebuild a physical standby database after the upgrade.

When you upgrade the primary database, many changes go into the data dictionary. These changes are recorded in the redo stream and sent to the standby database. When the standby database applies the redo, it is implicitly upgraded.

Prerequisites

If Grid Infrastructure (GI) manages your Oracle Databases, you must upgrade GI first. Check the Grid Infrastructure Installation and Upgrade Guide for your platform.

You can do it in the same maintenance window as the database upgrade, but I recommend that you perform the GI upgrade in an earlier maintenance window. A newer version of GI can run earlier versions of Oracle Database, so you can safely upgrade GI in advance. Doing so will give you time to adapt to the new GI release.

Also, in advance, you should install the new Oracle Home on both primary and standby hosts. The two Oracle Homes must have the same patches applied, and I recommend that you always apply the latest Release Update and have a look at 555.1 for important one-offs.

How To

When upgrading with Data Guard, there are two approaches:

  1. Standby Offline method
  2. Maximum Availability Architecture (MAA) method

Standby Offline Method

Before the upgrade starts on the primary database, you shut down the standby database. You keep it shut down until the upgrade has completed on the primary database and you have finished your tests. When you are sure you will stay on the new release, the standby database is restarted and synchronized with the primary database. It will take some time before you can go live because the standby database must apply all the redo generated during the upgrade.

If you need to fall back, you can use Flashback Database on the primary database. In addition, no matter what happens to the primary database, you still have the standby database immediately ready in the pre-upgrade state.

My team recommends this method. We prefer to sacrifice a little downtime to achieve even better protection.

MAA Method

The standby database is open and applies redo while the primary database is upgraded. This means that the standby database is closely following the primary database. You can go live very soon after the upgrade completes because there is little or very little apply lag.

The downside is when you must fall back. In that case, you have two databases to bring back in time with Flashback Database. In the very unlikely event that something happens during flashback on both databases, you may need to restore your backup.

The MAA team recommends this method as it guarantees the lowest downtime.

Which One To Choose?

If you have two or more standby databases, you can combine the two methods and get the best of both worlds. Otherwise, rest assured that both methods work fine and are supported.

Standby Offline MAA
Maximum protection Minimum downtime
Upgrade team recommendation MAA recommendation
Redo transport deferred Redo transport enabled
Redo apply stopped Redo apply active
Protected by offline standby and guaranteed restore point Protected by guaranteed restore point
AutoUpgrade default

Of course, AutoUpgrade supports both methods. You can check the other blog post in the series for detailed instructions.

Note, the following implication of using the standby offline method. AutoUpgrade will defer redo log transport to all remote archive destinations. Not only standby databases, but also GoldenGate downstream capture and Real-Time Redo Transport feature of Zero Data Loss Recovery Appliance. Most likely this is not a problem, since the database is in a maintenance window during the upgrade. But remember to enable all of them afterward.

What If

Exadata

If you are running Oracle Database on Exadata, you should read the dedicated procedure created by the Maximum Availability Architecture (MAA) team.

Multiple Standby Databases

Not much changes if you have many standby databases in your Data Guard configuration. The procedure is basically the same, except that you must execute commands on all the standby databases. The order of the standby databases does not matter (unless you have cascaded standby databases – see below).

Data Guard Broker

If you have configured your Data Guard setup using Data Guard broker, then you can leave it running during the upgrade. There used to be some problems with Data Guard broker during upgrade to previous releases, but it works fine when you upgrade to Oracle Database 19c.

However, you must disable Fast-Start Failover before the upgrade. After a successful upgrade, you can enable it again.

Cascaded Standby Databases

If you have cascaded standby databases, the following applies according to the documentation:

If there are cascaded standbys in your configuration, then those cascaded standbys must follow the same rules as any other standby, but should be shut down last, and restarted in the new home first.

You must treat cascaded standby databases like any other standby database. However, the order is now important. Imagine this scenario:

  • Primary database: BOSTON
  • Standby database: CHICAGO
  • Cascaded standby database: NEWYORK

When the procedure tells you to stop standby databases: First CHICAGO, then NEWYORK When the procedure tells you to start standby databases: First NEWYORK, then CHICAGO

Far Sync

A far sync database should be treated like any other standby database. Like cascaded standby databases the order of the shutdown is important to ensure that all redo from primary reaches the standby database connected via the far sync.

Logical Standby

When you have logical standby databases in your Data Guard configuration, things are slightly different. In that case, look in the documentation.

Database Services in OCI

You need to follow the documentation for your particular database service. If you have an Exadata Cloud Service, you might find Exadata Cloud Database 19c Rolling Upgrade With DBMS_ROLLING (Doc ID 2832235.1) interesting.

Other Blog Posts in This Series

21 thoughts on “Upgrade and Data Guard

  1. We are running Oracle 12cR2 DB on Windows, and we need to update it to 19c via DG. Can we have 19c physical Standby, then switchover to reduce the downtime? If yes, please provide the processes.
    Thank you

    Like

  2. Hi Ahmed,
    That’s a good question. When upgrading you can’t reduce downtime with physical standby databases.
    What you are asking for is possible – but only with logical standby databases. Then you can use what is named a Transient Logical Standby database and perform the upgrade with downtime only during a switchover.
    We covered that topic in one of our webinars – check it out: https://www.youtube.com/watch?v=MHQ3MhqHXbg&t=2584s
    The slides are here – pages 72 and onwards: https://dohdatabase.files.wordpress.com/2021/10/emea10_zerodowntimeoperations.pdf

    Regards,
    Daniel

    Like

  3. When upgrading the GRID infrastructure in a DG environment where both the primary and standby are 2 node RAC. My question is, does it matter which cluster you upgrade first? i.e. Can you upgrade the standby GRID first or should you upgrade the primary GRID first, or does it matter?

    Like

  4. Hi Ron,

    I am not a GI expert, but to my knowledge, it doesn’t matter. A newer version of GI can run databases of previous versions. It works fine and is of course fully supported.
    Personally, I would upgrade GI on the standby database first. If there are any errors during the upgrade, you can be sure not to disrupt the primary database.

    Regards,
    Daniel

    Like

  5. Daniel:

    I have watched your Youtube viedo – How to upgrade with Data Guard and learned a lot from there. I aslo went to your Database are fun site
    to read – Upgrade and Data Guard blog. I am working on Oracle RAC 3 nodes production database with RAC 3 nodes standby database from 12.1.0.2
    to 19c. Your approach is very helpful. But I have some specific situation here and want to get some advice from you.

    1. Our RAC 3 nodes database is 270TB on primary and standby respectively. We only do RMAN full backup on weekly basis.
    During the database upgrade, I have issue to create GRP due to storage limit. Our archivelog storage is about 44TB. For 270TB database,
    If I create GRP, what is minimum size on ASM diskgroup that I need? I upgraded all database in other environments without using GRP.
    I just took risk to use RMAN backups as recovery source in case database was crashed during upgrade. I went through all of them without
    problem. Now in production upgrade, I still do not want to using GRP and only rely on RMAN backups. Is this method working?

    2. In our 270TB database, we have configured with Active-Passive physical standby without using Data Guard Broker. We use sqlplus method of
    standby for archive log propagation for finer control. Now If I want to do rolling upgrade on this huge database and do not want to configure
    Data Guard Broker at this moment. Can I do these steps:

    1. Upgrade GI Home to 19c. This has been completed successfully.

    2. Install 19c software on primary and standby with rolling on all 3 nodes.

    3. defer archive log shipping from primary and archive log applying on standby.

    4. shutdown standby and upgrade primary with DBUA.

    5. After primary upgrade to 19c successfully. mount standby from 19c Home. Then enable archive log shipping from primary and
    applying from standby. The standby will take some time to synch with primary. After that, standby will also be upgraded to 19c.

    We do not care about downtime to much. I can do primary upgrade at weekend and make sure primary will be up running in 19c version
    in weekday and Application can connect to primary. Then taking time to synch standby from primary. is this approach working or not.
    Or I have enable Data Guard Broker from primary and standby to adpot rolling upgrade on standby. or use this stupid method.

    6. I defer archive log shipping and applying, then upgrade primary first, then do the same with DBUA on standby. After two database
    all upgrade to 19c. I enable archive log shipping and applying to synch primary and standby. Is this backward method working?

    I am sorry to write so long feedback. if you can provide some advice, it will be greatly appreciated.

    Frank

    email: tao.li1@VA.gov

    Like

  6. Hi Frank,

    Thanks for the positive feed. I am glad that my blog post helped you.

    Re. 1: You don’t need much space for Flashback Logs to protect your database with a GRP during an upgrade. During an upgrade, only the data dictionary (SYSTEM and SYSAUX) tablespaces are touched. Those tablespaces are much, much smaller and typically we don’t change things in such a way that a lot of redo/flashback logs are generated.
    If you have 44 TB on your archive storage that is way more than enough for such an upgrade. Remember you don’t have to enable FLASHBACK DATABASE. AutoUpgrade will just create the GRP without enabling FLASHBACK DATABASE. The flashback logs generated are smaller when you don’t explicitly enable FLASHBACK DATABASE.

    Re. 2: You don’t have to install the 19c Oracle Home in a rolling manner. You install the Oracle Home in advance and thus don’t perform it in a rolling manner.
    Your approach – which I call standby-offline – is fully supported and works. Many customers prefer to keep the standby database disconnected during the upgrade – as another layer of protection.
    Just remember to follow the procedure in this blog post:
    https://dohdatabase.com/2022/08/22/how-to-upgrade-data-guard-standby-offline-method/
    I advise you to sync the standby database inside your maintenance window. Then you know everything works as expected before you go live. During a database upgrade not much redo is generated, so the standby database should keep up fairly quick.
    You advise you to use AutoUpgrade for the upgrade. It is our preferred method for upgrading and DBUA is mainly kept for backward compatibility. Regardless of which tool you use for the upgrade, you DON’T upgrade the standby database yourself. Simply start the standby database in the new Oracle Home and it will be implicitly upgraded by the redo coming from the primary database.
    You can’t run any upgrade tool on a standby database. The upgrade must be done on the primary database – and only there.

    Regards,
    Daniel

    Like

  7. Hi Daniel,

    We have 11.2.0.1 binary with 2 running databases on production with a physical standby running on same version on windows platform.

    We have a requirement as below:

    Disconnect the dataguard
    Upgrade the dataguard first to 12.2
    Then upgrade the primary to 12.2
    Then re-enable the data guard.
    Also, expecting no data loss.
    Could you please advise how to achive this ?
    Really confused how to achive this. Please advise.

    Thanks & regards
    Kalpak

    Like

  8. Hi Kalpak,

    First, you should definitely upgrade to Oracle Database 19c instead. Oracle Database 12.2 is already out of support!

    It is not possible to upgrade a standby database. For an upgrade a database must be open in upgrade mode (which requires read write access to the database). You can’t open a standby database in upgrade mode. You could activate the standby database and perform the upgrade, but then there is no way to re-connect with the primary database. The relationship would be broken and you would need to rebuild the standby database.

    If you want to use a standby database for upgrade, it is called a rolling upgrade. In earlier versions this method was quite complicated and had some restrictions. I would not go down this way for your upgrade.

    Focus on minimizing downtime for the upgrade of the primary databae and tolerate the downtime it incurs. Otherwise, look at Oracle GoldenGate. For tips on how to minimize upgrade downtime take a look at our webinar “How Low Can You Go? Zero Downtime Operations” (https://dohdatabase.com/webinars/). Since you are running on an old and unsupported version, some of the things mentioned in the webinar might not apply to your database.

    Happy upgrading!
    Daniel

    Like

  9. Hi Daniel,

    Thanks for your response. Appreciate it. I have tried to search almost everything online but did not see exact scenario and solution there.

    I understood it is not possible to do this without downtime, but to minimize this I am thinking of below steps to follow :

    1. Disconnect and remove config of Dataguard and convert DG to standalone server.
    2. Perform upgrade on this standalone server.
    3. Switch application to this newly upgraded server. (Now new primary) and perform the testing.
    4. Perform upgrade to old primary server.
    5. Re-configure the dataguard and enable it for 12.2

    Challenges:
    1. How to remove configuration of Dataguard and convert it to standalone server. What parameters to be altered before starting on both the servers?

    2. What are the ways to apply recovery on newly upgraded server other than import/export tables/schema ?

    3. How can I re-enable to dataguard configuration after upgrading both the servers , what parameters to be changed ?

    4. Lil bit confused about listener , should it be online during upgrade ? Also , i have 2 upgrades 11.2.0.1 to 11.2.0.4 and then 11.2.0.4 to 12.2. In that case, I need to re-configure the listener each time ?

    I need your advise on this please. Anticipating your kind suggestions.

    Thanks & regards
    Kalpak

    Like

  10. Hi Kalpak,

    Your suggested procedure will not work.

    In #1 you will need to activate the standby database. This breaks, irreversibly, the relationship between the primary and standby database.
    For #5 you need to completely rebuild the standby database to connect it with the primary again. Meaning – doing a full clone!

    Re. 1: Check How To Remove Standby Database And Convert It to Standalone Database (Doc ID 2074686.1).
    Re. 2: I don’t understand the question.
    Re. 3: That is not possible.
    Re. 4: Listener can be restarted in the new Oracle Home before the upgrade. A newer listener can handle older databases as well.

    I think you are having an idea about the process which is not possible. With Data Guard, there are two ways:
    1. The method described in this blog post which requires downtime for the entire upgrade.
    2. Rolling upgrade using a transient logical standby. But that’s a much more complicated process and since you are coming from 11.2.0.1 the process becomes even harder. You can learn about the process here: https://www.youtube.com/watch?v=c8N9Bm3cado.

    Regards,
    Daniel

    Like

  11. Hi Daniel,
    I have a requirement to setup a cascaded dataguard environment(primary, dataguard & cascaded dataguard) .
    Can we achieve Upgrade using Transient Logical Standby in this cascaded dataguard environment ?

    If so, would you be able to provide some insight into this process?

    Thanks a ton

    Like

  12. Hi Kishore,
    Yes, that’s possible. I don’t have any specific documentation that uses that example, but it is doable. It also depends on the requirements for your protection level and so forth.
    It also depends on what you need the cascading standby database for. Normally, those are used during migrations. I rarely see them used in normal production.

    Here’s the information I have about rolling upgrades:
    https://dohdatabase.com/2022/02/02/rolling-upgrades-of-oracle-database-on-exadata-cloud-service/
    https://dohdatabase.com/2022/08/24/upgrade-data-guard-and-downtime/

    You can also check out our webinar “How Low Can You Go? Zero Downtime Operations”:
    https://dohdatabase.com/webinars/

    Regards,
    Daniel

    Like

  13. If the stanbdy database was left online (but in mounted only), can the Standby Offline Method still work? Could you just reboot the standby database and then the redo logs from the primary database (which has been upgraded) will be applied? I am using Data Guard (not Active Data Guard) in the Oracle Cloud, Gen 2.

    Like

  14. Hi,

    It should work.

    It doesn’t matter whether the database is shut down, mounted, or open. What matters is the state of redo apply. One way to stop redo apply is to shut down the database, but you can leave the database mounted or even open. Just ensure that redo apply is stopped. Then you will achieve the same level of support: having a standby database at a point in time before the upgrade started.

    Just remember that you need to restart the standby database anyway to move it to the new Oracle Home.

    Regards,
    Daniel

    Like

  15. Hello Daniel, hope all is well.

    My Active Data Guard environment is Oracle12c, 12.1.0.2 on Windows 2012 using Data Guard Broker. I have 5 primary at one datacenter and 5 standby at the other datacenter.

    What’s the recommended approach for transitioning to Oracle 19c on Windows 2022?
    I’m considering:
    1. Creating new servers.
    2. Install Windows 2022.
    3. Install GI19c.
    4. Install Oracle 19c.
    5. Configure Data Guard.
    6. Then, perform an export from the 12c and import into the 19c.

    What are your thoughts?

    Regards,
    jorge

    Like

Leave a comment