Tag: standby

Upgrade Oracle Database 19c CDB to 26ai with Data Guard – MAA Method

Here’s how to upgrade an entire CDB to Oracle AI Database 26ai, including all standby databases.

I’m using the MAA Method (see appendix). The standby database is up during the upgrade and continuously applies redo. This gives me shorter downtime compared to the alternative.

1. Preparations

I’ve already prepared my database and installed a new Oracle home on both servers. The maintenance window has started, and users have left the database.

  1. This is my AutoUpgrade config file:

    global.global_log_dir=/home/oracle/autoupgrade/logs/CDB19
upg1.source_home=/u01/app/oracle/product/19
upg1.target_home=/u01/app/oracle/product/26
upg1.sid=CDB19
    • sid contains the name or SID or my database.
    • I specify the source and target Oracle homes. I’ve already installed the target Oracle home.
    • Check the appendix for additional parameters.

  2. I disable Fast-Start Failover using Data Guard broker:

    disable fast_start failover;

2. Upgrade Standby

  1. I update listener.ora on the standby host (see appendix). I change the ORACLE_HOME parameter in the static listener entry (suffixed _DGMGRL) so it matches my target Oracle home.

  2. I reload the listener:

    lsnrctl reload

  3. I create a guaranteed restore point on the standby database.

    alter database recover managed standby database cancel;
create restore point stdby_before_upg guarantee flashback database;
alter database recover managed standby database disconnect from session;

  4. I start AutoUpgrade in deploy mode:

    java -jar autoupgrade.jar -config CDB19.cfg -mode deploy
    • AutoUpgrade restarts the standby in the new Oracle home.
    • It also takes care of all config files.
    • Plus, update /etc/oratab and Grid Infrastructure configuration.

  5. I update any profiles or scripts that use the standby.

  6. I use Data Guard broker to ensure that my standby is still applying logs:

    validate database <stdby_unique_name>;

3. Upgrade Primary

  1. I update listener.ora on the primary host (see appendix). I change the ORACLE_HOME parameter in the static listener entry (suffixed _DGMGRL) so it matches my target Oracle home.

  2. I reload the listener:

    lsnrctl reload

  3. I start AutoUpgrade in deploy mode:

    java -jar autoupgrade.jar -config CDB19.cfg -mode deploy
    • AutoUpgrade starts by setting a guaranteed restore point and then proceeds with the upgrade.

  4. While the job progresses, I monitor it:

    upg> lsj -a 30
    • The -a 30 option automatically refreshes the information every 30 seconds.
    • I can also use status -job 100 -a 30 to get detailed information about a specific job.

  5. In the end, AutoUpgrade completes the upgrade:

    Job 100 completed
------------------- Final Summary --------------------
Number of databases            [ 1 ]

Jobs finished                  [1]
Jobs failed                    [0]
Jobs restored                  [0]
Jobs pending                   [0]

---- Drop GRP at your convenience once you consider it is no longer needed ----
Drop GRP from CDB19: drop restore point AUTOUPGRADE_9212_CDB191927000

Please check the summary report at:
/home/oracle/autoupgrade/logs/CDB19/cfgtoollogs/upgrade/auto/status/status.html
/home/oracle/autoupgrade/logs/CDB19/cfgtoollogs/upgrade/auto/status/status.log
    • This includes the post-upgrade checks and fixups, incl. updating /etc/oratab and Grid Infrastructure configuration.

  6. I review the Autoupgrade Summary Report. The path is printed to the console:

    vi /home/oracle/autoupgrade/logs/CDB19/cfgtoollogs/upgrade/auto/status/status.log

  7. I take care of the post-upgrade tasks.

  8. I update any profiles or scripts that use the primary.

4. Finalizing

  1. When I’m done testing the database – including application testing – and I decide that a rollback is not needed, I drop the restore points – from primary and standby.

  2. I re-enable Fast-Start Failover using Data Guard broker:

    enable fast_start failover;

That’s It!

With AutoUpgrade, you can easily upgrade your entire CDB even when you use Data Guard.

Check the other blog posts related to upgrade to Oracle AI Database 26ai.

Happy upgrading!

Appendix

Different Methods

When upgrading with Data Guard, there are two approaches:

  • Standby Offline – Before the upgrade starts on the primary database, you shut down the standby database. You keep it shut down until the upgrade has completed on the primary database and you have finished your tests. When you are sure you will stay on the new release, the standby database is restarted and synchronized with the primary database. It will take some time before you can go live because the standby database must apply all the redo generated during the upgrade.
    • If you need to fall back, you can use Flashback Database on the primary database. In addition, no matter what happens to the primary database, you still have the standby database immediately ready in the pre-upgrade state.
    • My team recommends this method. We prefer to sacrifice a little downtime to achieve even better protection.
  • Maximum Availability Architecture (MAA) – The standby database is open and applies redo while the primary database is upgraded. This means that the standby database is closely following the primary database. You can go live very soon after the upgrade completes because there is little or very little apply lag.
    • The downside is when you must fall back. In that case, you have two databases to bring back in time with Flashback Database. In the very unlikely event that something happens during flashback on both databases, you may need to restore your backup.
    • The MAA team recommends this method as it guarantees the lowest downtime.
Standby Offline MAA
Maximum protection Minimum downtime
Upgrade team recommendation MAA recommendation
Redo transport deferred Redo transport enabled
Redo apply stopped Redo apply active
Protected by offline standby and guaranteed restore point Protected by guaranteed restore point
AutoUpgrade default

Static Listener Entry

In this blog post, I update the static listener entries required by Data Guard broker (suffixed DGMGRL). My demo environment doesn’t use Oracle Restart or Oracle Grid Infrastructure, so this entry is mandatory.

If you use Oracle Restart or Oracle Grid Infrastructure, such entry is no longer needed.

Oracle Data Guard Broker and Static Service Registration (KB142582)

What If I Have Multiple Standbys?

  • Repeat step Upgrade Standby for each standby before moving on with the next step Upgrade Primary.

  • This also applies if you have cascading standbys or use far-sync instances.

What About Rolling Upgrades?

This is a very different technique. You can learn more in Virtual Classroom #10: Zero Downtime Operations of Oracle Database.

What If My Database Is A RAC Database?

There are no changes to the procedure if you have an Oracle RAC database. AutoUpgrade detects this and sets CLUSTER_DATABASE=FALSE at the appropriate time. It also updates the Grid Infrastructure configuration.

For warp-speed upgrades, take a look at distributed upgrade.

What If I Use Oracle Restart?

No changes. AutoUpgrade detects this and automatically updates the Grid Infrastructure configuration.

What If My Database Is Encrypted

You must use an auto-login keystore. AutoUpgrade checks this during the pre-upgrade analysis.

Other Config File Parameters

The config file shown above is a basic one. Let me address some of the additional parameters you can use.

  • timezone_upg: AutoUpgrade upgrades the database time zone file after the actual upgrade. This requires an additional restart of the database and might take significant time if you have lots of TIMESTAMP WITH TIME ZONE data. If so, you can postpone the time zone file upgrade or perform it in a more time-efficient manner.

  • before_action / after_action: Extend AutoUpgrade with your own functionality by using scripts before or after the job.

  • em_target_name: Allow AutoUpgrade to create blackouts before restarting the database and to update the target configuration after the upgrade. Requires a local installation of emcli. See this blog post for details.

  • rman_catalog_connect_string: Relevant for databases that store backup metadata in a recovery catalog. Upgrades the recovery catalog schema following an upgrade. See this blog post for details.

Upgrade Oracle Database 19c PDB to 26ai with Data Guard using Refreshable Clone PDB

I’ve already shown you how to upgrade a single PDB to Oracle AI Database 26ai using refreshable clone PDB.

But what about Data Guard? Refreshable clone PDB supports only deferred recovery, which means I must restore the PDB after the upgrade.

Let’s see how it works.

Upgrade

For the upgrade, I follow the procedure for a non-Data Guard upgrade.

  • In my AutoUpgrade config file, I must either:
    • Omit manage_standbys_clause.
    • Or, set it to manage_standbys_clause=none.
  • Both methods trigger AutoUpgrade to plug in with deferred recovery, which is the only supported option when using refreshable clone PDB.

Once the upgrade completes, I can proceed and restore the PDB to my standby database.

Standby Database

  • I execute all commands on the same host – the standby system.

  • On the standby, I verify that recovery status is disabled:

    SQL> select open_mode, recovery_status
     from v$pdbs where name='NEWPDB1';

OPEN_MODE    RECOVERY_STATUS
____________ __________________
MOUNTED      DISABLED
    • This means that I plugged in with deferred recovery.
    • The standby is not protecting this PDB.

  • Next, I connect to the standby using RMAN, and I restore the PDB:

    connect target /
run{
   allocate channel disk1 device type disk;
   allocate channel disk2 device type disk;

   set newname for pluggable database NEWPDB1 to new;

   restore pluggable database NEWPDB1
      from service <primary_service>
      section size 64G;
}
    • You can add more channels depending on your hardware.
    • Replace NEWPDB1 with the name of your PDB.
    • <primary_service> is a connect string to the primary database.

  • Next, I connect to the standby database using Data Guard broker and turn off redo apply:

    edit database <stdby_unique_name> set state='apply-off';

  • Back in RMAN, still connected to the standby, I switch to the newly restored data files:

    switch pluggable database NEWPDB1 to copy;

  • Then, I connect to the standby and generate a list of commands that will online all the data files:

    alter session set container=NEWPDB1;
select 'alter database datafile '||''''||name||''''||' online;' from v$datafile;
    • Save the commands for later.
    • There should be one row for each data file.

  • If my standby is an Active Data Guard, I must restart it into MOUNT mode.

    alter session set container=CDB$ROOT;
shutdown immediate
startup mount

  • Now, I can re-enable recovery and online the data files:

    alter session set container=NEWPDB1;
alter pluggable database enable recovery;
alter database datafile <file1> online;
alter database datafile <file2> online;
...
alter database datafile <filen> online;
    • I must connect to the PDB.
    • I must execute the alter database datafile ... online command for each data file.

  • I turn on redo apply:

    edit database <stdby_unique_name> set state='apply-on';

  • At this point, the standby protects my PDB.

  • After a minute or two, I check the Data Guard config:

    validate database <stdby_unique_name>;

  • Once my standby is in sync, I can do a switchover as the ultimate test:

    switchover to <stdby_unique_name>;

  • Now, I connect to the new primary and ensure the PDB opens in read write mode and unrestricted:

    select open_mode, restricted
from v$pdbs
where name='NEWPDB1';

OPEN_MODE     RESTRICTED
_____________ _____________
READ WRITE    NO

  • You can find the full procedure in Making Use Deferred PDB Recovery and the STANDBYS=NONE Feature with Oracle Multitenant (KB90519). Check sections Steps for Preparing to enable recovery of the PDB and Steps required for enabling recovery on the PDB after the files have been copied.

That’s It!

Refreshable clone PDB is a great way of upgrading a single PDB. It’s very easy, but you must take care of your standby afterward.

Check the other blog posts related to upgrade to Oracle AI Database 26ai.

Happy upgrading!

Appendix

Multiple Standby Databases

You must restore the PDB to each standby database.

If you have multiple standby databases, it’s a lot of work for the primary to handle the restore pluggable database command from all standbys.

Imagine we have three standby databases. Here’s an alternative approach:

  • First, Standby 1 restores from primary.
  • Next, standby 2 restores from standby 1.
  • At the same time, standby 3 restores from primary.
  • This spreads the load throughout your databases, allowing you to complete faster.

What If My Database Is Encrypted

AutoUpgrade handles the keys in the primary database.

After the upgrade, but before you begin the restore, you must copy the database keystore from the primary to the standby.

What If My Database Is A RAC Database?

Works on Oracle RAC Database as well.