Author: Daniel Overby Hansen

Disstinguished Product Manager for Cloud Migrations at Oracle. I help customers make the most out of Oracle Database.

How To Downgrade a PDB from Oracle Database 23ai

When talking about upgrades, Oracle Database has a great fallback mechanism; a downgrade. Even after going live on Oracle Database 23ai, you can get back to Oracle Database 19c or 21c – with no data loss.

How to Downgrade From Oracle Database 23ai

My PDB, PDB1, has already been upgraded to Oracle Database 23ai. Now, I want to downgrade to Oracle Database 19c. In the downgrade process, I will unplug from CDB23 and into CDB19.

  1. I open the PDB in downgrade mode:
    alter pluggable database PDB1 close immediate;
alter pluggable database PDB1 open downgrade;
    • Downgrade mode is a special mode – similar to upgrade mode. It enables exclusive access to the database and disables a lot of features.
  2. I set the environment to CDB23 and start the downgrade process:
    cd $ORACLE_HOME/bin
./dbdowngrade -c 'PDB1'
    • The -c command line parameter starts a downgrade of a specific PDB; not the entire CDB.
  3. After the downgrade, I close and unplug from CDB23:
    alter pluggable database PDB1 close;
alter pluggable database PDB1 unplug into '/home/oracle/scripts/pdb1.xml';
  4. Now, I connect to CDB19 running on Oracle Database 19c. I plug in and open the PDB in upgrade mode:
    create pluggable database PDB1 using '/home/oracle/scripts/pdb1.xml';
alter pluggable database PDB1 open upgrade;
  5. I switch to PDB1 and complete the downgrade by running the catrelod.sql script:
    alter session set container=PDB1;
@$ORACLE_HOME/rdbms/admin/catrelod.sql
  6. Then, I recompile all invalid objects:
    @$ORACLE_HOME/rdbms/admin/utlrp.sql
  7. A restart of the PDB. I open in normal mode:
    alter pluggable database PDB1 close;
alter pluggable database PDB1 open;
  8. I gather new dictionary statistics:
    exec dbms_stats.gather_dictionary_stats;
    • After a while, when the database is warmed up, I also gather fixed objects statistics.
  9. I verify that all components are VALID or OPTION OFF:
    select comp_id, version, status from dba_registry;
  10. I run Datapatch to ensure all SQL patches are properly applied:
    $ORACLE_HOME/OPatch/datapatch

That’s it!

Worth Knowing About Downgrades

  • The downgrade is a two-step process.

    • The first part happens while the database is running in the new Oracle Home. Startup in a special downgrade mode and execute catdwgrd.sql to start the downgrade.
    • Next, restart the database in the old Oracle Home and start in upgrade mode. catrelod.sql will re-install any missing objects in the database and finish the downgrade.

  • Oracle recommends that you install the latest Release Update in both Oracle homes; the one that you downgrade from, and the one to which you downgrade.

  • You can only downgrade if the compatible hasn’t been changed after the upgrade.

  • If the timezone file was upgraded, the same timezone file must be present in the old Oracle Home.

  • Before you start the downgrade, there’s no need to roll off any patches with Datapatch. The downgrade mechanism takes care of that.

  • The data dictionary in a downgraded database is not identical to the pre-upgraded database. The data dictionary will be different, but compatible. Here are some examples:

    • Generally, dropping objects is avoided.
    • New tables are most likely not dropped but truncated.
    • New indexes are most likely kept.

  • Although you can downgrade a database from Oracle Database 23ai to 19c, you can’t undo the multitenant migration. To get back to a non-CDB you must use other means like Data Pump, transportable tablespaces, or GoldenGate.

Want to Try?

Hopefully, you never need to downgrade a PDB. But I bet you can resist the urge to try it. Right? RIGHT?

In our hands-on lab, Hitchhiker’s Guide for Upgrading to Oracle Database 23ai, there is a downgrade exercise. Give it a try. The lab is free to use and doesn’t require any installation – it runs completely inside a browser.

Happy downgrading!

Appendix

Further Reading

AutoUpgrade New Features: Patch OCW Component In Oracle Home

Every Oracle home contains an Oracle Clusterware (OCW) component. It’s used to interact with Grid Infrastructure when you are using Oracle Restart or Oracle RAC. But even when you don’t use those, the component is still part of your Oracle home.

The Database Release Update doesn’t update the OCW component in your Oracle home. You must use the Grid Infrastructure Release Update for that.

In AutoUpgrade, it is easy to update the OCW component. Let’s see how it works.

How To Also Patch The OCW Component

  • My database hasn’t been patched for a while:

    $ORACLE_HOME/OPatch/opatch lspatches

35648110;OJVM RELEASE UPDATE: 19.21.0.0.231017 (35648110)
35787077;DATAPUMP BUNDLE PATCH 19.21.0.0.0
35643107;Database Release Update : 19.21.0.0.231017 (35643107)
29585399;OCW RELEASE UPDATE 19.3.0.0.0 (29585399)
    • I’ve never updated the OCW component, so it’s still on the patch level of the base release, 19.3.0.0.0.

  • I use the latest version of AutoUpgrade:

    wget https://download.oracle.com/otn-pub/otn_software/autoupgrade.jar

  • I create an AutoUpgrade config file, FTEX.cfg:

    global.global_log_dir=/home/oracle/autoupgrade-patching/log
global.keystore=/home/oracle/autoupgrade-patching/keystore
patch1.source_home=/u01/app/oracle/product/19
patch1.target_home=/u01/app/oracle/product/19_27
patch1.sid=FTEX
patch1.folder=/home/oracle/patch-repo
patch1.patch=OPATCH,RU,OCW,DPBP,OJVM
    • By adding OCW to the patch parameter, AutoUpgrade also downloads the GI Release Update and updates the OCW component.

  • I patch the database:

    java -jar autoupgrade.jar -config FTEX.cfg -patch -mode deploy

  • When AutoUpgrade completes, I check the new patch level:

    $ORACLE_HOME/OPatch/opatch lspatches

37499406;OJVM RELEASE UPDATE: 19.27.0.0.250415 (37499406)
37654975;OCW RELEASE UPDATE 19.27.0.0.0 (37654975)
37777295;DATAPUMP BUNDLE PATCH 19.27.0.0.0
37642901;Database Release Update : 19.27.0.0.250415 (37642901)
    • Notice how the OCW Release Update is now 19.27.0.0.0.

Some Details

  • When AutoUpgrade downloads patches, because I specified OCW, it will also download the GI Release Update:

     --------------------------------------------
 Downloading files to /home/oracle/patch-repo
 --------------------------------------------
 DATABASE RELEASE UPDATE 19.27.0.0.0
     File: p37642901_190000_Linux-x86-64.zip - LOCATED
 
 DATAPUMP BUNDLE PATCH 19.27.0.0.0
     File: p37777295_1927000DBRU_Generic.zip - LOCATED
 
 GI RELEASE UPDATE 19.27.0.0.0
     File: p37641958_190000_Linux-x86-64.zip / 83%

  • Including OCW is a smart way of downloading the GI Release Update. You can use it to patch your Grid Infrastructure.

  • In Oracle Database 23ai, you can download fully updated gold images. Besides having the latest Release Update, they also come with fully updated OCW components.

Is It Needed?

Should you update the OCW component when you patch your Oracle Database? Is it needed if you don’t use Oracle Restart, Oracle RAC, or Oracle ASM?

It is optional, but even if no GI Stack (ASM, Clusterware or RAC) is used inside the server, it is recommended not to ignore the security patches of the installed components. And apply the most recent OCW Patch.

How to apply OCW Release Update patches on db_home non-RAC / non-ASM (Doc ID 2970542.1)

Mike Dietrich has a good point as well:

As I neither use RHP/FPP or any of the HA components nor EM in my tiny little lab environments, I’m pretty certain that I won’t need the OCW bundle. But this may be different in your environments. And it doesn’t harm to apply it of course.

Adding the Oracle 19.14.0 OCW / GI bundle patch to my database home

Further, I know many customers who never patch the OCW component and haven’t run into related problems.

My recommendation: Update the OCW component when you patch your Oracle Database. Using AutoUpgrade it is so easy, that there’s no reason not to.

Happy patching!

AutoUpgrade New Features: Custom Oracle Home Name

When you create a new Oracle home using AutoUpgrade, you can now give it a custom name.

What do I mean by a custom name? Let’s examine the Oracle Inventory:

cat /u01/app/oraInventory/ContentsXML/inventory.xml

<?xml version="1.0" standalone="yes" ?>
<!-- Copyright (c) 1999, 2025, Oracle and/or its affiliates.
All rights reserved. -->
<!-- Do not modify the contents of this file by hand. -->
<INVENTORY>
<VERSION_INFO>
   <SAVED_WITH>12.2.0.7.0</SAVED_WITH>
   <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER>
</VERSION_INFO>
<HOME_LIST>
<HOME NAME="OraDB19Home1" LOC="/u01/app/oracle/product/19" TYPE="O" IDX="1"/>
<HOME NAME="OraDB21Home1" LOC="/u01/app/oracle/product/21" TYPE="O" IDX="2"/>
<HOME NAME="OraDB23Home1" LOC="/u01/app/oracle/product/23" TYPE="O" IDX="3"/>
</HOME_LIST>
<COMPOSITEHOME_LIST>
</COMPOSITEHOME_LIST>
</INVENTORY>

The Oracle home name is listed in the NAME attribute on the HOME element.

When you install a new Oracle home, the installer automatically generate a name for you, like OraDb19Home1.

In AutoUpgrade, you can decide to use a custom name instead of an auto-generated one.

How To Specify A Custom Oracle Home Name

  • I use the latest version of AutoUpgrade
    wget https://download.oracle.com/otn-pub/otn_software/autoupgrade.jar
  • I create the following config file:
    global.global_log_dir=/home/oracle/autoupgrade-patching/log
global.keystore=/home/oracle/autoupgrade-patching/keystore
patch1.source_home=/u01/app/oracle/product/19
patch1.target_home=/u01/app/oracle/product/19_28
patch1.home_settings.home_name=DBHOME1928
patch1.sid=FTEX
patch1.folder=/home/oracle/patch-repo
patch1.patch=RU,OCW,DPBP,OJVM,OPATCH
    • Notice the home_settings.home_name parameter. This is where I specify the custom name for the new Oracle home.
  • I patch the database using AutoUpgrade:
    java -jar autoupgrade.jar -config FTEX.cfg -mode deploy
  • I check the inventory:
    grep -i "19_28" /u01/app/oraInventory/ContentsXML/inventory.xml
<HOME NAME="DBHOME1928" LOC="/u01/app/oracle/product/19_28" TYPE="O" IDX="4"/>
    • AutoUpgrade created the new Oracle home with the name DBHOME1928.

What Do I Use It For

For a regular Oracle home, it’s used seldom. I’ve seen some customers using certain Oracle home names as part of their corporate standard, but most people don’t care.

However, in a read-only Oracle home, the name is important. It now becomes part of Oracle Base Home which is the location for most of the writable files from the Oracle home.

Do you have a specific use case for the Oracle home name or do you have a corporate standard mandating a certain name? Leave a comment and let me know.

Happy patching!

Can I Use the Oracle Database 23ai Preinstall RPM for Oracle Database 19c?

A customer asked me:

My ops team wants to use the Oracle Database 23ai preinstallation RPM for all our new servers. For now, the servers will host Oracle Database 19c only. In the future, we will use Oracle Database 23ai on those servers as well.

The preinstallation RPM contains instructions on which packages to install and which system settings to set. According to the Oracle Database 19c Installation Guide for Linux, this is how you run it:

dnf install oracle-database-preinstall-19c

Let’s find out whether you can use the 23ai preinstallation RPM for 19c.

Inspecting the Package

  • I use an Oracle Linux 8 test system. First, I install both packages:

    dnf -y install oracle-database-preinstall-19c
dnf -y install oracle-database-preinstall-23ai

  • Next, I look at the files in the 19c RPM:

    rpm -ql oracle-database-preinstall-19c

/etc/rc.d/init.d/oracle-database-preinstall-19c-firstboot
/etc/security/limits.d/oracle-database-preinstall-19c.conf
/etc/sysconfig/oracle-database-preinstall-19c
/etc/sysconfig/oracle-database-preinstall-19c/oracle-database-preinstall-19c-verify
/etc/sysconfig/oracle-database-preinstall-19c/oracle-database-preinstall-19c.param
/usr/bin/oracle-database-preinstall-19c-verify
/usr/share/licenses/oracle-database-preinstall-19c
/usr/share/licenses/oracle-database-preinstall-19c/LICENSE
/var/log/oracle-database-preinstall-19c
/var/log/oracle-database-preinstall-19c/results

  • The file oracle-database-preinstall-19c.param contains the settings, groups, and users. In the 23ai RPM, there is a similar file, oracle-database-preinstall-23ai.param. I can compare the two files to see any differences:

    #Remove comments before comparing
sed '/^\(#\|kernelcomment\|usercomment\)/d' /etc/sysconfig/oracle-database-preinstall-19c/oracle-database-preinstall-19c.param > /tmp/19c.params
sed '/^\(#\|kernelcomment\|usercomment\)/d' /etc/sysconfig/oracle-database-preinstall-23ai/oracle-database-preinstall-23ai.param > /tmp/23ai.params

diff /tmp/19c.params /tmp/23ai.params

  • Then, I can analyze the differences. < is the 19c setting, > is the 23ai setting.

    28c28
< kernel:*:*:*:net.ipv4.ip_local_port_range:9000 65500
---
> kernel:*:*:*:net.ipv4.ip_local_port_range:9000 65535
    • Port range extended in 23ai.
    52,53c52
< boot:x86_64:*:*:*:numa:off
< boot:*:*:*:*:transparent_hugepage:never
---
> boot:*:*:*:*:numa:off
> boot:*:*:*:*:transparent_hugepage:madvise
    • NUMA off for all architectures. Not a big deal, as this is Oracle Linux and Oracle Database runs on 64-bit architecture only.
    • But there is a change in the setting for transparent hugepages. Knowing how much havoc transparent hugepages have caused, this is an important change. I’ll get back to that.
    63a63,64
> kernel:*:*:*:kernel.panic:10
    • Adding a kernel panic setting. Shouldn’t cause any concern.

Transparent Hugepage

  • The setting for transparent hugepages changes from never to madvise. In this setting, the kernel only assigns huge pages to application processes that explicitly request huge pages through the madvise() system call (ref. Oracle Linux 8 – Configuring Huge Pages).

  • The 23ai installation guide lists:

    For optimal performance, Oracle recommends that you set Transparent HugePages to madvise on all Oracle Database servers UEK7 and later kernels and not disable Transparent HugePages as was recommended in prior releases.

  • Let me check my kernel version:

    uname -r

5.15.0-307.178.5.el8uek.x86_64
    • el8uek means my kernel is UEK8, so I can use the new setting on this system.

  • But I will use this server for Oracle Database 19c and 23ai, so can I use the new madvise setting?

    Transparent huge page memory is enabled by default on Oracle Linux and Exadata. The ‘madvise’ parameter used in the kernel command enables transparent huge pages and assigns huge pages to memory regions explicitly requested by applications using the madvise system call. In the case that Oracle Database 19c and 23ai are running on the same server, Oracle recommends setting transparent huge pages to ‘madvise’.

    Huge Pages or Transparent Huge Pages in Context of Exadata

  • So, it should be safe to run with the new setting for Oracle Database 19c. If you are concerned, you can revert to never until you install Oracle Database 23ai.

Packages

  • I can find the dependencies of the two preinstallation RPMs:
    rpm -qR oracle-database-preinstall-19c > /tmp/19c.pkgs
rpm -qR oracle-database-preinstall-23ai > /tmp/23ai.pkgs
  • Then, I can find the difference:
    diff /tmp/19c.pkgs /tmp/23ai.pkgs
  • There are many differences because the 23ai RPM now lists minimum versions for many packages. Assuming that you automatically install the latest version of the packages, I will disregard those differences.
  • This brings me to this list. These dependencies are not part of the 23ai RPM:
    ethtool
libaio-devel
libnsl
libstdc++-devel
openssl-libs
    • I would need to install those manually if I wanted to use the 23ai RPM.
  • For completeness, these are the new packages in the 23ai RPM:
    fontconfig >= 2.13.1
policycoreutils
policycoreutils-python-utils

Conclusion

You can use the 23ai preinstallation RPM if you take care of the missing packages.

I recommend running both preinstallation RPMs instead in the listed order:

dnf install oracle-database-preinstall-19c
dnf install oracle-database-preinstall-23ai

Then, you proceed with installation of Oracle homes using AutoUpgrade.

Happy installing!

Appendix

Credits

A thank-you to Rodrigo Jorge for helping with this article.

How To Install Data Pump Bundle Patch On Running Database Without Downtime

As discussed previously, the Release Updates rarely include any Data Pump fixes. If you start a Data Pump job, the absence of the many fixes may lead to errors or severe performance problems.

Hence, I always recommend that you install the Data Pump bundle patch before starting a Data Pump job.

But the fixes are not part of the Release Update and they are not RAC rolling installable, so it sounds like a pain to install, right? But it is not!

The Data Pump bundle patch is a Non-Binary Online Installable patch, so you can apply it without any downtime.

How To Apply Data Pump Bundle Patch

  1. I start with a running Oracle Database called FTEX.
    ps -ef | grep pmon
oracle     53316       1  0 13:57 ?        00:00:00 ora_pmon_FTEX
  2. I verify that the Oracle home does not have the Data Pump bundle patch:
    $ORACLE_HOME/OPatch/opatch lspatches

37499406;OJVM RELEASE UPDATE: 19.27.0.0.250415 (37499406)
37654975;OCW RELEASE UPDATE 19.27.0.0.0 (37654975)
37642901;Database Release Update : 19.27.0.0.250415 (37642901)

OPatch succeeded.
  3. I ensure that no Data Pump jobs are running currently.
  4. I patch my database.
    $ORACLE_HOME/OPatch/opatch apply

[output truncated]

Verifying environment and performing prerequisite checks...
OPatch continues with these patches:   37777295

Do you want to proceed? [y|n]
y
User Responded with: Y
All checks passed.
Backing up files...
Applying interim patch '37777295' to OH '/u01/app/oracle/product/19_27'

Patching component oracle.rdbms.dbscripts, 19.0.0.0.0...

Patching component oracle.rdbms, 19.0.0.0.0...
Patch 37777295 successfully applied.
Log file location: /u01/app/oracle/product/19_27/cfgtoollogs/opatch/opatch2025-06-23_14-26-49PM_1.log

OPatch succeeded.
    • Although the FTEX database is still running, OPatch doesn’t complain about files in use.
    • This is because the Data Pump bundle patch is marked as a non-binary online installable patch.
    • I can safely apply the patch to a running Oracle home – as long as no Data Pump jobs are running.
    • For cases like this, it’s perfectly fine to use in-place patching.
  5. I complete patching by running Datapatch:
    $ORACLE_HOME/OPatch/datapatch

[output truncated]

Adding patches to installation queue and performing prereq checks...done
Installation queue:
  No interim patches need to be rolled back
  No release update patches need to be installed
  The following interim patches will be applied:
    37777295 (DATAPUMP BUNDLE PATCH 19.27.0.0.0)

Installing patches...
Patch installation complete.  Total patches installed: 1

Validating logfiles...done
Patch 37777295 apply: SUCCESS
  logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/37777295/27238855/37777295_apply_FTEX_2025Jun24_09_10_15.log (no errors)
SQL Patching tool complete on Tue Jun 24 09:12:19 2025

That’s it! I can now enjoy the many benefits of the Data Pump bundle patch – without any downtime for a single instance database.

Happy patching!

Appendix

What about Oracle RAC Database

Although the Data Pump Bundle Patch is not a RAC Rolling Installable patch, you can still apply it to an Oracle RAC Database following the same approach above.

Simply apply the patch in turn on all nodes in your cluster. You should use OPatch with the -local option and not OPatchAuto. When all nodes are patched, you can run Datapatch.

Is It the Same as an Online or Hot Patch

No, a Non-Binary Online Installable patch is not the same as an Online or Hot patch.

A patch that only affects SQL scripts, PL/SQL, view definitions and XSL style sheets (i.e. non-binary components). This is different than an Online Patch, which can change binary files. Since it does not touch binaries, it can be installed while the database instance is running, provided the component it affects is not in use at the time. Unlike an Online Patch, it does not require later patching with an offline patch at the next maintenance period.

Source: Data Pump Recommended Proactive Patches For 19.10 and Above(Doc ID 2819284.1)

Is the Data Pump Bundle Patch Really That Important

Yes. There are more than 200 fixes for Data Pump. If you start a Data Pump job without it, you might face errors or severe performance issues.

Here’s a statement from a customer, I worked with.

Applying the bundle patch reduced the time for an import to drop from 2,5 hours to 48 minutes

The Easiest Way To Download Patches for Oracle Grid Infrastructure

Whether you patch Oracle Grid Infrastructure manually, using your own automation, or Oracle Fleet Patching and Provisioning, it all starts with downloading the patches.

Although AutoUpgrade doesn’t patch Grid Infrastructure, it can still download the latest version of OPatch and the GI Release Update.

How To Download Grid Infrastructure Release Update

I have already configured the AutoUpgrade keystore and saved my My Oracle Support credentials. You find instructions on how to do that here (search for Creating an AutoUpgrade Keystore).

  1. I download the latest version of AutoUpgrade:

    wget https://download.oracle.com/otn-pub/otn_software/autoupgrade.jar

  2. I create a config file called get-gi-patches.cfg:

    global.global_log_dir=/home/oracle/autoupgrade/logs
global.keystore=/home/oracle/autoupgrade/keystore

patch1.patch=RU,OPATCH,OCW
patch1.target_version=19
patch1.platform=LINUX.X64
patch1.folder=/home/oracle/autoupgrade/patches

  3. I create the folder for the patches (config file parameter folder).

    mkdir -p /home/oracle/autoupgrade/patches

  4. I download the patches by starting AutoUpgrade in download mode:

    java -jar autoupgrade.jar -config get-gi-patches.cfg -mode download

  5. Here’s the output from AutoUpgrade:

    AutoUpgrade Patching 25.3.250509 launched with default internal options
Processing config file ...
Loading AutoUpgrade Patching keystore
AutoUpgrade Patching keystore is loaded

Connected to MOS - Searching for specified patches

-----------------------------------------------------
Downloading files to /home/oracle/autoupgrade/patches
-----------------------------------------------------
DATABASE RELEASE UPDATE 19.27.0.0.0
    File: p37642901_190000_Linux-x86-64.zip - VALIDATED

GI RELEASE UPDATE 19.27.0.0.0
    File: p37641958_190000_Linux-x86-64.zip - VALIDATED

OPatch 12.2.0.1.46 for DB 19.0.0.0.0 (Apr 2025)
    File: p6880880_190000_Linux-x86-64.zip - VALIDATED
-----------------------------------------------------

  6. That’s it! After a few minutes, I’ve downloaded the latest GI Release Update and OPatch. GI RELEASE UPDATE 19.27.0.0.0 – p37641958_190000_Linux-x86-64.zip OPatch 12.2.0.1.46 for DB 19.0.0.0.0 (Apr 2025) – p6880880_190000_Linux-x86-64.zip

Is it really that easy? Yes, it is…

I can now patch my GI Oracle home – for Oracle RAC Database and Oracle Restart.

Happy patching!

Appendix

I Want To Download A Previous Release Update

Rather than downloading the latest Release Update, you can choose to download a specific Release Update. You can specify that in the patch parameter:

patch1.patch=RU:19.26,OPATCH,OCW
  • Notice how the RU keyword has a suffix specifying the exact Release Update.

Oracle DatabaseWorld

While you wait for the big show in Las Vegas later this year, here’s something to whet your appetite.

Oracle DatabaseWorld Multicloud AI Edition

Hit the Watch on demand button to get free access to all these sessions and get up-to-date on Oracle Database. You can watch the videos anywhere and anytime.

What’s On

Get an overview with the keynotes or dive into the details in specialized sessions.

  • Take a look into at crystal ball and hear about Oracle’s vision for data and AI. Our Executive Vice President, Juan Loaiza, shares how Oracle Database’s cutting-edge converged data architecture helps customers bring AI to data.

  • Also, there are three track keynotes giving you further details on different topics.

  • Mike Dietrich and I give you a head start on your upgrade to Oracle Database 23ai in our session Upgrade to Oracle Database 23ai: Best Practices and Customer Experience.

  • Plus, a number of other sessions that go into the details as well.

I Got Questions

If you need more information on upgrades to Oracle Database 23ai or have questions after watching our session, reach out to me and I’ll get you sorted.

You can also try upgrades in our hands-on lab. It’s free and runs in a browser – nothing to install. And – it’s better to fail in our lab, than in production.

Happy learning!

Fixing AutoUpgrade PATCH111 Error On Oracle Linux 9

I’ve seen a lot of interest in the create_home mode of AutoUpgrade. It allows you to easily provision new Oracle homes and the feedback has been very positive.

A few customers reported an error during installation of an Oracle home on Oracle Linux 9.

Errors in database [create_home_1]
Stage              [OH_PATCHING]
Operation          [STOPPED]
Status             [ERROR]
Info               [Exception: PATCH111
Err message: AutoUpgradePatchingException [OPatch failed to install ...]

Digging into the referenced log file revealed:

[Jun 25, 2025 10:48:51 AM] [WARNING]OUI-67200:Make failed to invoke "/usr/bin/make -f ins_rdbms.mk javavm_refresh ORACLE_HOME=/u01/app/oracle/product/dbhome_1927 OPATCH_SESSION=apply"....'Can't locate File/Copy.pm in @INC (you may need to install the File::Copy module) (@INC contains: /usr/local/lib64/perl5/5.32 /usr/local/share/perl5/5.32 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at /u01/app/oracle/product/dbhome_1927/javavm/install/update_javavm_binaries.pl line 64.
                                    BEGIN failed--compilation aborted at /u01/app/oracle/product/dbhome_1927/javavm/install/update_javavm_binaries.pl line 64.
                                    make: *** [ins_rdbms.mk:573: javavm_refresh] Error 2

The Workaround

You can avoid the problem by installing Perl before you start AutoUpgrade:

dnf -y install perl-core

Big shoutout to Martin Berger for providing a workaround.

Is My Server Affected

Use this command to test whether your server is affected:

perl -e 'use File::Copy;'

If it returns without any error or output, everything is fine. If you get an error, you must install Perl before proceeding:

perl -e 'use File::Copy;'
Can't locate File/Copy.pm in @INC (you may need to install the File::Copy module) (@INC contains: /usr/local/lib64/perl5/5.32 /usr/local/share/perl5/5.32 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at -e line 1.
BEGIN failed--compilation aborted at -e line 1.

Why Does It Happen

  • AutoUpgrade starts by installing the Oracle home and in some cases also applies the Release Update using the -applyRU command line parameter.

  • Next, it installs additional patches using OPatch. Some patches, like the OJVM bundle patch, requires a compilation using make. This process fails if there’s no usable Perl installation.

  • There are several MOS notes on this specific issues advising to set the PERL5LIB environment variable (Patching fails during relink , with error code 102 :: Fatal error: Command failed for target `javavm_refresh’ (Doc ID 2002334.1)).

  • To solve an unrelated issue, AutoUpgrade doesn’t always set PERL5LIB when calling OPatch and this leads to this error during the compilation.

  • We’re trying to find a smart way to handle this (without re-introducing the other issue).

To summarize, you get this error when:

  • Your operating system doesn’t have a normal Perl installation. You might have the perl executable, but not the normal modules.
  • You install the OJVM bundle patch.

The Rabbit Hole

I learned something new about Perl today. Do you dare walking down that rabbit hole with me? I’ll make it short.

  • The error from make mentions the missing Perl module, File::Copy:

    /usr/bin/make -f ins_rdbms.mk javavm_refresh ORACLE_HOME=/u01/app/oracle/product/dbhome_1927 OPATCH_SESSION=apply"....'Can't locate File/Copy.pm in @INC (you may need to install the File::Copy module)

  • A Perl module can be loaded from disk. Perl automatically attempts to find the file in the following locations:

    perl -e 'print "@INC\n"'
/usr/local/lib64/perl5/5.32
/usr/local/share/perl5/5.32
/usr/lib64/perl5/vendor_perl
/usr/share/perl5/vendor_perl
/usr/lib64/perl5
/usr/share/perl5

  • So, I can check these directories for the Perl module. To find the File::Copy module, I need to check in the File subdirectory for the file Copy.pm:

    ll /usr/local/lib64/perl5/5.32/File/Copy.pm
ll /usr/local/share/perl5/5.32/File/Copy.pm
ll /usr/lib64/perl5/vendor_perl/File/Copy.pm
ll /usr/share/perl5/vendor_perl/File/Copy.pm
ll /usr/lib64/perl5/File/Copy.pm
ll /usr/share/perl5/File/Copy.pm
ls: cannot access '/usr/local/lib64/perl5/5.32/File/Copy.pm': No such file or directory
ls: cannot access '/usr/local/share/perl5/5.32/File/Copy.pm': No such file or directory
ls: cannot access '/usr/lib64/perl5/vendor_perl/File/Copy.pm': No such file or directory
ls: cannot access '/usr/share/perl5/vendor_perl/File/Copy.pm': No such file or directory
ls: cannot access '/usr/lib64/perl5/File/Copy.pm': No such file or directory
ls: cannot access '/usr/share/perl5/File/Copy.pm': No such file or directory

  • The file is not found in any of the directories, so Perl will fail to load the module.

  • Had the file been found in just one location, all would be good.

Happy patching!

Where Is The OJVM Bundle Patch For Oracle Database 23ai?

A customer asked me:

Where do I download the OJVM bundle patch for Oracle Database 23ai? I can’t find it anywhere.

In Oracle Database 23ai, Oracle ships fully updated gold images instead of individual Release Updates. The customer installed the latest gold image and it didn’t contain the OJVM bundle patch:

$ORACLE_HOME/OPatch/opatch lspatches
37701424;OCW RELEASE UPDATE 23.8.0.25.04 (37701424) Gold Image
37701421;Database Release Update : 23.8.0.25.04 (37701421) Gold Image

In Oracle Database 19c, you would be able to see the OJVM bundle patch as a separate patch:

$ $ORACLE_HOME/OPatch/opatch lspatches
35648110;OJVM RELEASE UPDATE: 19.21.0.0.231017 (35648110)
35643107;Database Release Update : 19.21.0.0.231017 (35643107)
29585399;OCW RELEASE UPDATE 19.3.0.0.0 (29585399)

Why is it missing from Oracle Database 23ai gold image and why can’t you download it separately?

OJVM In Oracle Database 23ai

Starting in Oracle Database 21c, there is no longer a separate OJVM bundle patch. Instead, the fixes are fully included in the Release Updates.

… the OJVM component patch has been incorporated within the 21c Database Release Update (RU) patch

Oracle Recommended Patches – “Oracle JavaVM Component Database PSU and Update” (OJVM PSU and OJVM Update) Patches (Doc ID 1929745.1)

Also, there is no longer any interuption or brownout when you start Datapatch. The OJVM fixes are now fully RAC Rolling Installable and Standby-First Installable.

So, just get the newest gold image in Oracle Database 23ai and that’s it.

What About Oracle Database 19c

Let’s just recap the situation when you have OJVM in Oracle Database 19c.

  • The patch is RAC Rolling Installable
  • Standby-First Installable (as long as you don’t have Active Data Guard and extensively use OJVM on the standby database)

First, are you using OJVM or can you remove it?

Then, how do you patch OJVM?

  1. Start by downloading the OJVM bundle patch.
    1. Use AutoUpgrade and include the OJVM keyword in your patch specification, like patch1.patch=OPATCH,RU,OJVM
    2. Use the Download Assistant on My Oracle Support
  2. When you start Datapatch and it applies the OJVM fixes, there is a short brownout while the Java system is reloaded. Any session that tries to use Java at that point will get ORA-29548 but will be able to retry after a few seconds.

Here’s a little video with additional information.

What About Grid Infrastructure?

You should not apply the OJVM bundle patch to your Grid Infrastructure homes.

OJVM PSU is not needed in the Grid home, only in the database home.

Oracle Recommended Patches – “Oracle JavaVM Component Database PSU and Update” (OJVM PSU and OJVM Update) Patches (Doc ID 1929745.1)

Happy patching!

AutoUpgrade New Features: Install Oracle Home on Brand-New, Empty Server

You can use AutoUpgrade to install an Oracle home on a brand-new, empty server.

Previously, AutoUpgrade used an existing Oracle home to determine how to create a new Oracle home. Now, you can use config file parameters to instruct AutoUpgrade on installing the new Oracle home.

This allows you to use AutoUpgrade to prepare a new server for use with Oracle Database.

How To

Preparations

I start with a brand new Oracle Linux 8 system.

  1. This is a test system, so I disable SELinux:
    sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config
reboot
    • Alternatively, follow the guidelines for SELinux in your organization.
  2. Install the preinstall package and Java (required by AutoUpgrade):
    yum -y install oracle-database-preinstall-19c
yum -y install java-1.8.0-openjdk
  3. Additional configurations like mount points and swapfile.

Install Oracle Home

  1. Download AutoUpgrade:

    wget https://download.oracle.com/otn-pub/otn_software/autoupgrade.jar

  2. Create an AutoUpgrade config file called install-home.cfg:

    global.global_log_dir=/home/oracle/autoupgrade/logs
install1.patch=RU:19.27,OPATCH,OJVM,DPBP
install1.folder=/u01/app/oracle/software
install1.target_home=/u01/app/oracle/product/dbhome_1927
install1.home_settings.oracle_base=/u01/app/oracle
install1.home_settings.edition=EE
install1.home_settings.inventory_location=/u01/app/oraInventory
install1.download=no
    • I have already downloaded the base release and required patches using AutoUpgrade download mode. The files are in an NFS file share which I specify with the folder parameter.
    • I use the home_settings parameters to specify how I want to install the Oracle home. There are many other settings that I can use.

  3. I start AutoUpgrade:

    java -jar autoupgrade.jar -config install-home.cfg -patch -mode create_home

  4. On this system, the oracle user is not allowed to sudo, so I must manually execute the root scripts:

    /u01/app/oracle/product/dbhome_1927/root.sh
/u01/app/oraInventory/orainstRoot.sh
    • AutoUpgrade prints the scripts to the console.
    • It also writes the scripts into <global_log_dir>/create_home_1/<job-no>/rootsh/rootsh.log.

  5. That’s it! I’ve now installed a new Oracle home and can move on with the creation of a listener, database, and so forth

Appendix

Installing on Microsoft Windows

You can use the same functionality on Windows.

  • You must use home_settings.account_type to specify the user that runs the Windows service. If you specify a user, then it must exist already. Although runInstaller can create the user for you, AutoUpgrade doesn’t support it.
  • You should not use home_settings.inventory_location on Windows. On Windows, the Oracle Inventory is always placed in %SYSTEM_DRIVE%\Program Files\Oracle\Inventory.

Installing Java

AutoUpgrade requires Java 8 or Java 11. It doesn’t support newer versions because the tool must be backward compatible. The latest version of AutoUpgrade still supports upgrades from Oracle Database 11g.

I recommend using Oracle Java. This is what we test and develop with, however, other Java packs should be fine as well. I tested the commands in this blog post with OpenJDK.

If you have a license for Oracle Database, you can read about using Oracle JDK/JRE in the License Guide or Oracle Java SE licensing FAQ.