Patching Eats Your SYSTEM Tablespace

Everyone says: Patch your software!

I keep saying: Patch your Oracle Database!

Let me tell you a little secret. All that patching is eating space in your SYSTEM tablespace

This blog post is a continuation of Mike’s many blog posts on the topic. It adds an example with some real numbers.

Apply And Rollback Scripts

Normally, the apply and rollback scripts are stored in the Oracle home. Here’s an example:

$ ll $ORACLE_HOME/sqlpatch
drwxr-xr-x. 4 oracle oinstall     38 Apr 18  2019 29517242
drwxr-xr-x. 3 oracle oinstall     22 Jul 28 05:28 36878697
drwxr-xr-x. 3 oracle oinstall     22 Jul 28 05:27 36912597
drwxr-xr-x. 3 oracle oinstall     22 Jul 28 05:30 37056207
drwxr-xr-x. 3 oracle oinstall     22 Aug 11 08:34 37102264
drwxr-xr-x. 3 oracle oinstall     22 Aug 11 08:32 37260974
drwxr-xr-x. 3 oracle oinstall     22 Aug 11 08:36 37470729
drwxr-xr-x. 3 oracle oinstall     22 Aug 11 08:47 37499406
drwxr-xr-x. 3 oracle oinstall     22 Aug 11 08:44 37642901
drwxr-xr-x. 3 oracle oinstall     22 Aug 11 08:49 37777295

When Datapatch applies patches to your Oracle Database, it uses the apply and rollback scripts from the Oracle home.

But during apply, Datapatch also takes the rollback scripts and stores them inside the database – in the SYSTEM tablespace. This ensures that Datapatch can always roll back patches, regardless of whether the rollback scripts are in the Oracle home.

When you use the OPatch cleanup or uses out-of-place patching, there is a risk that Datapatch might need a rollback script which is no longer found in the Oracle home. But then Datapatch simply finds it in the SYSTEM tablespace instead.

This avoild a lot of chaotic situations with missing rollback scripts.

Storing The Scripts

Datapatch uses two tables that both have a column named PATCH_DIRECTORY:

select * from dba_registry_sqlpatch;
select * dba_registry_sqlpatch_ru_info;

This query shows each patch action and corresponding usage for apply/rollback script:

select * from (
   select description, round(dbms_lob.getlength(PATCH_DIRECTORY)/1024/1024, 2) as size_mb
   from dba_registry_sqlpatch
   where action='APPLY' and description not like 'Database Release Update%'
   union
   select 'Release Update ' || RU_version as description, round(dbms_lob.getlength(PATCH_DIRECTORY)/1024/1024) as size_mb
   from dba_registry_sqlpatch_ru_info)
order by description;

Here’s an example of an Oracle Database that I have patched a few times.

DESCRIPTION                                       SIZE_MB
------------------------------------------------- -------
DATAPUMP BUNDLE PATCH 19.25.0.0.0                    1.03
DATAPUMP BUNDLE PATCH 19.26.0.0.0                    1.03
DATAPUMP BUNDLE PATCH 19.27.0.0.0                    1.03
DATAPUMP BUNDLE PATCH 19.28.0.0.0                    1.04
OJVM RELEASE UPDATE: 19.25.0.0.241015 (36878697)      .01
OJVM RELEASE UPDATE: 19.26.0.0.250121 (37102264)      .02
OJVM RELEASE UPDATE: 19.27.0.0.250415 (37499406)      .02
OJVM RELEASE UPDATE: 19.28.0.0.250715 (37847857)      .02
Release Update 19.25.0.0.0                            175
Release Update 19.26.0.0.0                            184
Release Update 19.27.0.0.0                            194
Release Update 19.28.0.0.0                            203
Release Update 19.3.0.0.0                               4

10 rows selected.

That’s around 750 MB.

Cleaning Up

Datapatch only needs the rollback scripts for the patches that are currently applied. You can remove all other scripts:

$ORACLE_HOME/OPatch/datapatch -purge_old_metadata

Using the above environment, this is the result of the cleanup:

DESCRIPTION                                       SIZE_MB
------------------------------------------------- -------
DATAPUMP BUNDLE PATCH 19.25.0.0.0                    
DATAPUMP BUNDLE PATCH 19.26.0.0.0                    
DATAPUMP BUNDLE PATCH 19.27.0.0.0                    
DATAPUMP BUNDLE PATCH 19.28.0.0.0                    1.04
OJVM RELEASE UPDATE: 19.25.0.0.241015 (36878697)     
OJVM RELEASE UPDATE: 19.26.0.0.250121 (37102264)     
OJVM RELEASE UPDATE: 19.27.0.0.250415 (37499406)     
OJVM RELEASE UPDATE: 19.28.0.0.250715 (37847857)      .02
Release Update 19.25.0.0.0                           
Release Update 19.26.0.0.0                           
Release Update 19.27.0.0.0                           
Release Update 19.28.0.0.0                            203
Release Update 19.3.0.0.0                               

10 rows selected.

Datapatch now only uses little more than 200 MB.

A few comments about the cleanup functionality:

  • It is available via patch 37738908. Hopefully, it will be part of the 19.29 Release Update.
  • You can safely execute the cleanup. Datapatch doesn’t remove scripts that it might need at a later point.
  • Oracle recommends that you run the cleanup in an off-peak period.
  • The cleanup happens via a TRUNCATE TABLE command which effectively reclaims space so other segments may use it. However, it doesn’t shrink the tablespace, so the physical size of the data files remain the same.
  • To facilitate the TRUNCATE TABLE command, those records that must remain is copied to a new table. After truncating the original table, those records are moved back and the temporary table is dropped. This might lead to a little increase in space usage while Datapatch cleans up.
  • When you upgrade, the upgrade engine truncates those tables. They are of no use following an upgrade.

Final Words

Check your Oracle Database. How much space does Datapatch use? What was the largest amount of space you could reclaim? Let me know in the comments below.

Update

  • 03-SEP-2025: I correctly wrote that the tables use the SYSAUX tablespace. They use the SYSTEM tablespace. Also, I added a detail about the cleanup might take up a little more space temporarily. Thanks to Pete for letting me know.

How To Roll Back After Patching

Here’s a question I received from a customer:

I’ve patched my database to 19.25 using AutoUpgrade and out-of-place patching. How do I roll back, if needed?

Basically, a rollback is the same as patching the database. You just do it the other way around – from the higher to the lower Oracle home. But let’s look at the details.

AutoUpgrade

I’m glad to hear that the customer uses AutoUpgrade for patching. It’s my recommended method, and it has many benefits.

If you use AutoUpgrade to patch your Oracle Database, you can also use it to roll back, but only before going live:

java -jar autoupgrade.jar -restore -jobs <n>
  • n is the job ID of the patching job.
  • AutoUpgrade undoes everything it did.

AutoUpgrade relies on Flashback Database as its rollback mechanism. So, it’s no good if users have already connected to the database and added/changed data.

Allow me to repeat: Only use AutoUpgrade to roll back before go-live!

After go-live, you must roll back manually.

Manually

You can manually roll back at any time – even after go-live.

Imagine you want to roll back from 19.25 (the new Oracle home) to 19.24 (the old Oracle home). Here’s how to do it.

  • You start by setting the environment.
    export OLD_ORACLE_HOME=/u01/app/oracle/product/dbhome_19_24
    export NEW_ORACLE_HOME=/u01/app/oracle/product/dbhome_19_25
    export ORACLE_HOME=$NEW_ORACLE_HOME
    export PATH=$ORACLE_HOME/bin:$PATH
    
  • Optionally, you run Datapatch sanity check in the new Oracle home (thanks Erik for pointing that out).
    $ORACLE_HOME/OPatch/datapatch -sanity_checks
    
  • You shut down the database running in the new Oracle home.
    sqlplus / as sysdba<<EOF
       shutdown immediate
    EOF
    
  • You move the following files back to the old Oracle home:
    • PFile
    • SPFile
    • Password file
    • Network files (like tnsnames.ora and sqlnet.ora)
    • Some of the files might not be present at all or be placed outside the Oracle home
    • Check this blog post for other files that might be stored in the Oracle home
    mv $NEW_ORACLE_HOME/dbs/init$ORACLE_SID.ora $OLD_ORACLE_HOME/dbs
    mv $NEW_ORACLE_HOME/dbs/spfile$ORACLE_SID.ora $OLD_ORACLE_HOME/dbs
    mv $NEW_ORACLE_HOME/dbs/orapw$ORACLE_SID $OLD_ORACLE_HOME/dbs
    mv $NEW_ORACLE_HOME/network/admin/sqlnet.ora $OLD_ORACLE_HOME/network/admin
    mv $NEW_ORACLE_HOME/network/admin/tnsnames.ora $OLD_ORACLE_HOME/network/admin
    
  • Update /etc/oratab and set the Oracle home to the old one.
  • Update your profile scripts to reflect the old Oracle home. It could be .bashrc.
  • Start the database in the old Oracle home.
    export ORACLE_HOME=$OLD_ORACLE_HOME
    export PATH=$ORACLE_HOME/bin:$PATH
    sqlplus / as sysdba<<EOF
       startup
    EOF
    
  • Run Datapatch.
    $ORACLE_HOME/OPatch/datapatch
    
  • Finally, you fix internal directories that point to paths inside the Oracle home:
    @?/rdbms/admin/utlfixdirs.sql
    

Datapatch

When you roll back, you must execute Datapatch. It will automatically detect that you are rolling back and perform the necessary actions.

For each patch there is an apply script that brings changes into the database. Datapatch executes the apply script during patching.

For each apply script, there is always a rollback script. It will reverse the actions of the apply script. Datapatch executes the rollback script when you roll back.

You can learn much more about Datapatch in this video.

Normally, you would roll back to the Oracle home from where you came, but that’s not a requirement. This scenario is fully supported:

  • Patch from 19.23 to 19.25
  • Roll back to 19.24

How To Practice?

We have a hands-on lab in which you can try rollbacks – using AutoUpgrade and manually.

Patch Me If You Can

The lab runs in Oracle Live Labs.

  • It’s completely free
  • It runs in just a browser

Happy patching!

Further Reading