During our recent webinar on cross-platform migration, an attendee asked a good question:
Does Data Pump in a Full Transportable Export/Import move ADO policies and heat map information?
Let’s find out.
What Is It?
ADO means Advanced Data Optimization and is a feature that:
… automate the compression and movement of data between different tiers of storage within the database.
ADO uses heat maps that :
… provides data access tracking at the segment-level and data modification tracking at the segment and row level.
In other words,
- You might have slower and faster storage.
- You define ADO policies on tables and indexes (including partitions and subpartitions).
- The policies define in which storage the database stores the rows.
- The heat maps tell which objects you access.
- Based on heat maps, a job automatically moves rows between storage depending on the use and/or compress it.
What Happens in Data Pump
There are three pieces of essential information:
- ADO policies. Defined on objects and governs how the database should store data.
- Heat map information. The database records your usage of the data and stores the information in heat maps. Later on, the database uses the heat maps to determine what to do with your data.
- ILM settings. Tells the database how to perform the ILM (Information Lifecycle Management) maintenance. For example, you can define the parallel degree that the database uses during maintenance or the number of concurrent maintenance jobs.
You can find a test case at the end of the blog post. I’m using test data and queries from oracle-base.com.
ADO Policies
Data Pump transfers the ADO policies in all modes:
- Full transportable
- Full
- Schema
- Tablespace
- Table
You can verify it:
SELECT policy_name,
object_owner,
object_name,
object_type,
inherited_from,
enabled,
deleted
FROM dba_ilmobjects
ORDER BY 1;
Heat Map Information
Data Pump does not transfer any heat map information. I didn’t find any way to move the heat map information to the new database.
ILM Settings
Data Pump does not transfer ILM settings – not even in full modes. You must manually move the settings.
-
You can find the current ILM settings:
SELECT * FROM DBA_ILMPARAMETERS;
-
And change the settings in the target database:
EXEC DBMS_ILM_ADMIN.CUSTOMIZE_ILM(DBMS_ILM_ADMIN.DEG_PARALLEL , 4);
- You must translate the name of the setting to the corresponding PL/SQL constant. You change the setting DEGREE OF PARALLELISM by using
DBMS_ILM_ADMIN.DEG_PARALLEL.
-
The documentation holds a complete list of ILM settings.
Appendix
Further Reading
Test Data
I am using our Hands-On lab that you can use on Oracle LiveLabs. Again, thanks to oracle-base.com for test data and queries.
CONN / AS SYSDBA
--Turn on heat map tracking
ALTER SYSTEM SET HEAT_MAP=ON SCOPE=BOTH;
--Customize ILM settings
BEGIN
DBMS_ILM_ADMIN.CUSTOMIZE_ILM(DBMS_ILM_ADMIN.RETENTION_TIME, 42);
DBMS_ILM_ADMIN.CUSTOMIZE_ILM(DBMS_ILM_ADMIN.JOBLIMIT, 42);
DBMS_ILM_ADMIN.CUSTOMIZE_ILM(DBMS_ILM_ADMIN.ABS_JOBLIMIT, 42);
DBMS_ILM_ADMIN.CUSTOMIZE_ILM(DBMS_ILM_ADMIN.DEG_PARALLEL , 4);
END;
/
SELECT * FROM DBA_ILMPARAMETERS;
--Create tablespaces for ILM policy
CREATE TABLESPACE FAST_STORAGE_TS DATAFILE SIZE 1M AUTOEXTEND ON NEXT 1M;
CREATE TABLESPACE MEDIUM_STORAGE_TS DATAFILE SIZE 1M AUTOEXTEND ON NEXT 1M;
CREATE TABLESPACE SLOW_STORAGE_TS DATAFILE SIZE 1M AUTOEXTEND ON NEXT 1M;
CREATE USER TEST IDENTIFIED BY TEST QUOTA UNLIMITED ON USERS;
GRANT CREATE SESSION, CREATE TABLE TO TEST;
ALTER USER TEST QUOTA UNLIMITED ON FAST_STORAGE_TS;
ALTER USER TEST QUOTA UNLIMITED ON MEDIUM_STORAGE_TS;
ALTER USER TEST QUOTA UNLIMITED ON SLOW_STORAGE_TS;
CONN TEST/TEST
--Create table where we can track usage
CREATE TABLE t1 (
id NUMBER,
description VARCHAR2(50),
CONSTRAINT t1_pk PRIMARY KEY (id)
);
INSERT INTO t1
SELECT level,
'Description for ' || level
FROM dual
CONNECT BY level <= 10;
COMMIT;
--Generate "usage"
SELECT *
FROM t1;
SELECT *
FROM t1
WHERE id = 1;
--Create table for ILM policy
CREATE TABLE invoices (
invoice_no NUMBER NOT NULL,
invoice_date DATE NOT NULL,
comments VARCHAR2(500)
)
PARTITION BY RANGE (invoice_date)
(
PARTITION invoices_2016_q1 VALUES LESS THAN (TO_DATE('01/04/2016', 'DD/MM/YYYY')) TABLESPACE slow_storage_ts,
PARTITION invoices_2016_q2 VALUES LESS THAN (TO_DATE('01/07/2016', 'DD/MM/YYYY')) TABLESPACE slow_storage_ts,
PARTITION invoices_2016_q3 VALUES LESS THAN (TO_DATE('01/09/2016', 'DD/MM/YYYY')) TABLESPACE medium_storage_ts
ILM ADD POLICY TIER TO slow_storage_ts READ ONLY SEGMENT AFTER 6 MONTHS OF NO ACCESS,
PARTITION invoices_2016_q4 VALUES LESS THAN (TO_DATE('01/01/2017', 'DD/MM/YYYY')) TABLESPACE medium_storage_ts
ILM ADD POLICY TIER TO slow_storage_ts READ ONLY SEGMENT AFTER 6 MONTHS OF NO ACCESS,
PARTITION invoices_2017_q1 VALUES LESS THAN (TO_DATE('01/04/2017', 'DD/MM/YYYY')) TABLESPACE fast_storage_ts
ILM ADD POLICY TIER TO medium_storage_ts READ ONLY SEGMENT AFTER 3 MONTHS OF NO ACCESS,
PARTITION invoices_2017_q2 VALUES LESS THAN (TO_DATE('01/07/2017', 'DD/MM/YYYY')) TABLESPACE fast_storage_ts
ILM ADD POLICY TIER TO medium_storage_ts READ ONLY SEGMENT AFTER 3 MONTHS OF NO ACCESS
)
ILM ADD POLICY ROW STORE COMPRESS BASIC SEGMENT AFTER 3 MONTHS OF NO ACCESS;
Test Case
###############
# FULL EXPORT #
###############
export ORAENV_ASK=NO
export ORACLE_SID=UP19
. oraenv
export ORAENV_ASK=YES
rm /tmp/expdat.dmp
sql / as sysdba<<EOF
create or replace directory expdir as '/tmp';
EOF
expdp system/oracle full=y directory=expdir job_name=fullexp
export ORAENV_ASK=NO
export ORACLE_SID=CDB2
. oraenv
export ORAENV_ASK=YES
sql / as sysdba<<EOF
startup
alter pluggable database pdb1 close immediate;
drop pluggable database pdb1 including datafiles;
create pluggable database pdb1 admin user admin identified by admin;
alter pluggable database all open;
alter session set container=pdb1;
create tablespace users DATAFILE SIZE 1M AUTOEXTEND ON NEXT 1M;
CREATE TABLESPACE fast_storage_ts DATAFILE SIZE 1M AUTOEXTEND ON NEXT 1M;
CREATE TABLESPACE medium_storage_ts DATAFILE SIZE 1M AUTOEXTEND ON NEXT 1M;
CREATE TABLESPACE slow_storage_ts DATAFILE SIZE 1M AUTOEXTEND ON NEXT 1M;
create or replace directory expdir as '/tmp';
EOF
impdp system/oracle@pdb1 directory=expdir
sql / as sysdba<<EOF
alter session set container=pdb1;
select * from DBA_ILMPARAMETERS;
SELECT policy_name,
object_owner,
object_name,
object_type,
inherited_from,
enabled,
deleted
FROM dba_ilmobjects
ORDER BY 1;
SELECT track_time,
owner,
object_name,
segment_write,
full_scan,
lookup_scan
FROM dba_heat_map_seg_histogram
ORDER BY 1, 2, 3;
EOF
#################
# SCHEMA EXPORT #
#################
export ORAENV_ASK=NO
export ORACLE_SID=UP19
. oraenv
export ORAENV_ASK=YES
rm /tmp/expdat.dmp
sql / as sysdba<<EOF
create or replace directory expdir as '/tmp';
EOF
expdp system/oracle schemas=test directory=expdir job_name=schemaexp
export ORAENV_ASK=NO
export ORACLE_SID=CDB2
. oraenv
export ORAENV_ASK=YES
sql / as sysdba<<EOF
startup
alter pluggable database pdb1 close immediate;
drop pluggable database pdb1 including datafiles;
create pluggable database pdb1 admin user admin identified by admin;
alter pluggable database all open;
alter session set container=pdb1;
create tablespace users DATAFILE SIZE 1M AUTOEXTEND ON NEXT 1M;
CREATE TABLESPACE fast_storage_ts DATAFILE SIZE 1M AUTOEXTEND ON NEXT 1M;
CREATE TABLESPACE medium_storage_ts DATAFILE SIZE 1M AUTOEXTEND ON NEXT 1M;
CREATE TABLESPACE slow_storage_ts DATAFILE SIZE 1M AUTOEXTEND ON NEXT 1M;
create or replace directory expdir as '/tmp';
EOF
impdp system/oracle@pdb1 directory=expdir
sql / as sysdba<<EOF
alter session set container=pdb1;
select * from DBA_ILMPARAMETERS;
SELECT policy_name,
object_owner,
object_name,
object_type,
inherited_from,
enabled,
deleted
FROM dba_ilmobjects
ORDER BY 1;
SELECT track_time,
owner,
object_name,
segment_write,
full_scan,
lookup_scan
FROM dba_heat_map_seg_histogram
ORDER BY 1, 2, 3;
EOF
################
# TABLE EXPORT #
################
export ORAENV_ASK=NO
export ORACLE_SID=UP19
. oraenv
export ORAENV_ASK=YES
rm /tmp/expdat.dmp
sql / as sysdba<<EOF
create or replace directory expdir as '/tmp';
EOF
expdp system/oracle tables=test.t1,test.invoices directory=expdir job_name=tabexp
export ORAENV_ASK=NO
export ORACLE_SID=CDB2
. oraenv
export ORAENV_ASK=YES
sql / as sysdba<<EOF
startup
alter pluggable database pdb1 close immediate;
drop pluggable database pdb1 including datafiles;
create pluggable database pdb1 admin user admin identified by admin;
alter pluggable database all open;
alter session set container=pdb1;
create tablespace users DATAFILE SIZE 1M AUTOEXTEND ON NEXT 1M;
CREATE TABLESPACE fast_storage_ts DATAFILE SIZE 1M AUTOEXTEND ON NEXT 1M;
CREATE TABLESPACE medium_storage_ts DATAFILE SIZE 1M AUTOEXTEND ON NEXT 1M;
CREATE TABLESPACE slow_storage_ts DATAFILE SIZE 1M AUTOEXTEND ON NEXT 1M;
create or replace directory expdir as '/tmp';
CREATE USER TEST IDENTIFIED BY TEST QUOTA UNLIMITED ON USERS;
GRANT CREATE SESSION, CREATE TABLE TO TEST;
ALTER USER TEST QUOTA UNLIMITED ON FAST_STORAGE_TS;
ALTER USER TEST QUOTA UNLIMITED ON MEDIUM_STORAGE_TS;
ALTER USER TEST QUOTA UNLIMITED ON SLOW_STORAGE_TS;
EOF
impdp system/oracle@pdb1 directory=expdir
sql / as sysdba<<EOF
alter session set container=pdb1;
select * from DBA_ILMPARAMETERS;
SELECT policy_name,
object_owner,
object_name,
object_type,
inherited_from,
enabled,
deleted
FROM dba_ilmobjects
ORDER BY 1;
SELECT track_time,
owner,
object_name,
segment_write,
full_scan,
lookup_scan
FROM dba_heat_map_seg_histogram
ORDER BY 1, 2, 3;
EOF
Databases Are Fun 