[The other day I was helping a customer perform an unplug-plug upgrade of an encrypted PDB using AutoUpgrade and a refreshable clone PDB.

But it kept failing during the initial copy of the PDB:

upg> Copying remote database 'PDB2' as 'NEWPDB2' for job 101

-------------------------------------------------
Errors in database [CDB19]
Stage     [CLONEPDB]
Operation [STOPPED]
Status    [ERROR]
Info    [
Error: UPG-4016
[Unexpected exception error]
Cause: There was an error during the database clone operation
For further details, see the log file located at /u01/app/oracle/cfgtoollogs/autoupgrade/CDB19/101/autoupgrade_20400101_user.log]

-------------------------------------------------
Logs: [/u01/app/oracle/cfgtoollogs/autoupgrade/CDB19/101/autoupgrade_20400101_user.log]
-------------------------------------------------

Luckily, there’s extensive logging in AutoUpgrade, so I went into the directory holding the logs from the CLONEPDB stage and found the following:

create pluggable database "NEWPDB2"  FROM PDB2@CLONEPDB   file_name_convert=none  tempfile reuse keystore identified by "*" REFRESH MODE MANUAL
*
ERROR at line 1:
ORA-17628: Oracle error 46659 returned by remote Oracle server
ORA-46659: master encryption keys for the given PDB not found
Help: https://docs.oracle.com/error-help/db/ora-17628/

Let’s dig into the error.

A Possible Solution

A search on MOS revealed a note with a possible solution:

• Manually export/import the encryption keys, or
• Use the undocumented INCLUDING SHARED KEYS clause on the CREATE PLUGGABLE DATABASE statement.

I didn’t like these solutions because:

• The target CDB should be able to import the keys automatically over the database link.
• It worked fine on other encrypted databases without the workaround.

So what was the problem?

The Root Cause

• In the source CDB, I could see from V$ENCRYPTION_KEYS that the source PDB, PDB2, didn’t have any encryption keys.

• There should be an encryption key activated by PDB2. But the query returned no rows.

  SELECT * FROM v$encryption_keys WHERE activating_pdbname='PDB2';
  

• So which encryption key did PDB2 use?

• It turns out that PDB2 was recently created by cloning another local PDB, PDB1.

• After cloning an encrypted PDB, the new PDB keeps using the same encryption keys as the source PDB. This is called a shared key.

• So, PDB2 was using the same encryption key as PDB1.

• For security reasons, if you try to clone a PDB using a shared key, the database errors out.

The Solution

• I advise you to rekey your database after cloning. This ensures that the clone gets its own encryption keys and, thus, strengthens security.

• After connecting to PDB2, I performed a rekey using the set key command:

  alter session set container=PDB2;
  administer key management set key
     force keystore identified by "<keystore_pwd>"
     with backup;
  
  

• Then, I could clone PDB2 using refreshable clone PDB and upgrade it without problems.

Lesson Learned

• Oracle recommends that you rotate your encryption keys by doing a rekey.
• After cloning an encrypted database, you should perform a rekey, so the clone has its own encryption keys and does not share encryption keys with another database.
• The database can operate with a shared key, but it might cause problems later.

Happy upgrading!