Toad World

Block Corruption: RMAN with High or Normal Diskgroup

Avatar photo
Post author:Written By

While I was playing with "Scrub repair" in 12c I found the following:

"RMAN doesn't report a corrupt block in a diskgroup with NORMAL or HIGH Redundancy if there is at least one good mirror block in another failure group."

"RMAN reports a corrupt block when there is no any good copy of the block remaining"

Does RMAN know that it can recover the block from the mirror copy so that it doesn't report it?  if so, why RMAN doesn't recover the corrupt block from the mirror when it gets aware of the corruption? In ASM when you read data from a corrupt block automatically ASM recover the block from a mirror one. at least RMAN should report the corrupt block if it won't recover it. 

Note: If you know how to find in which block your data is located, read my previous article  Data block recovering process using Normal Redundancy

 

Current Environment:

SQL> select group_number, name, type from v$asm_diskgroup where name='DATA2';

GROUP_NUMBER NAME TYPE
------------ ------------------------------ ------
1 DATA2 NORMAL

SQL> select group_number, disk_number, name, path from v$asm_disk

GROUP_NUMBER DISK_NUMBER NAME PATH
------------ ----------- ---------- ------------------------------
1 0 DATA2_0000 /dev/oracleasm/disks/DISK3
1 3 DATA2_0003 /dev/oracleasm/disks/DISK2

SQL> select * from dgomez;

ID VALUE 
---------- --------------------
1 Deiby

 

Corruption of Primary Block:

Primary Block Copy:

[grid@orcl12c ~]$ dd if=/dev/oracleasm/disks/DISK3 bs=8k count=1 skip=171270 | od -a|tail -5
1+0 records in
1+0 records out
8192 bytes (8.2 kB) copied, 3.1289e-05 s, 262 MB/s
0017700 can cr can ff bel x q enq can cr can ff stx A stx del
0017720 del enq D ack us U dc1 del soh nul eot C bel 8 $ stx
0017740 A stx del del del dle em p r & # * syn ) c H
0017760 , soh stx stx A stx enq D e i b y soh ack q \
0020000
[grid@orcl12c ~]$

 

Secondary Block Copy (Mirror):

[grid@orcl12c ~]$ dd if=/dev/oracleasm/disks/DISK2 bs=8k count=1 skip=5894 | od -a|tail -5
1+0 records in
1+0 records out
8192 bytes (8.2 kB) copied, 2.2349e-05 s, 367 MB/s
0017700 can cr can ff bel x q enq can cr can ff stx A stx del
0017720 del enq D ack us U dc1 del soh nul eot C bel 8 $ stx
0017740 A stx del del del dle em p r & # * syn ) c H
0017760 , soh stx stx A stx enq D e i b y soh ack q \
0020000

 

Corruption of Primary Block Copy:

dd if=/dev/zero bs=8k count=1 seek=171270 of=/dev/oracleasm/disks/DISK3

 

Primary Block Copy:

[grid@orcl12c ~]$ dd if=/dev/oracleasm/disks/DISK3 bs=8k count=1 skip=171270 | od -a|tail -5
1+0 records in
1+0 records out
8192 bytes (8.2 kB) copied, 9.6381e-05 s, 85.0 MB/s
0000000 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
*
0020000
[grid@orcl12c ~]$

 

Secondary Block Copy (Mirror):

[grid@orcl12c ~]$ dd if=/dev/oracleasm/disks/DISK2 bs=8k count=1 skip=5894 | od -a|tail -5
1+0 records in
1+0 records out
8192 bytes (8.2 kB) copied, 9.0514e-05 s, 90.5 MB/s
0017700 can cr can ff bel x q enq can cr can ff stx A stx del
0017720 del enq D ack us U dc1 del soh nul eot C bel 8 $ stx
0017740 A stx del del del dle em p r & # * syn ) c H
0017760 , soh stx stx A stx enq D e i b y soh ack q \
0020000
[grid@orcl12c ~]$

 

Does RMAN report the corrupt block?

RMAN> validate check logical database;

File Status Marked Corrupt Empty Blocks Blocks Examined High SCN
---- ------ -------------- ------------ --------------- ----------
2 OK 0 6265 6400 1793137 
File Name: +DATA2/ORCL2/DATAFILE/dgomeztbs.270.867323161
Block Type Blocks Failing Blocks Processed
---------- -------------- ----------------
Data         0             5 
Index        0             0 
Other        0             130

 

Since RMAN didn't report any corrupted block, maybe it recovered the block?

[grid@orcl12c ~]$ dd if=/dev/oracleasm/disks/DISK3 bs=8k count=1 skip=171270 | od -a|tail -5
1+0 records in
1+0 records out
8192 bytes (8.2 kB) copied, 9.6381e-05 s, 85.0 MB/s
0000000 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
*
0020000
[grid@orcl12c ~]$

No. RMAN didn't recover the block and RMAN didn't report the block.

 

Restoring the Primary Copy from Mirror Copy:

SQL> select * from dgomez;

ID VALUE 
---------- --------------------
1 Deiby

Why this SELECT recovered the corrupt block? Read  Data block recovering process using Normal Redundancy

 

Corruption of Secondary Block (Mirror):

Primary Block Copy:

[grid@orcl12c ~]$ dd if=/dev/oracleasm/disks/DISK3 bs=8k count=1 skip=171270 | od -a|tail -5
1+0 records in
1+0 records out
8192 bytes (8.2 kB) copied, 0.000368203 s, 22.2 MB/s
0017700 can cr can ff bel x q enq can cr can ff stx A stx del
0017720 del enq D ack us U dc1 del soh nul eot C bel 8 $ stx
0017740 A stx del del del dle em p r & # * syn ) c H
0017760 , soh stx stx A stx enq D e i b y soh ack q \
0020000
[grid@orcl12c ~]$

 

Secondary Block Copy (Mirror):

[grid@orcl12c ~]$ dd if=/dev/oracleasm/disks/DISK2 bs=8k count=1 skip=5894 | od -a|tail -5
1+0 records in
1+0 records out
8192 bytes (8.2 kB) copied, 0.000274336 s, 29.9 MB/s
0017700 can cr can ff bel x q enq can cr can ff stx A stx del
0017720 del enq D ack us U dc1 del soh nul eot C bel 8 $ stx
0017740 A stx del del del dle em p r & # * syn ) c H
0017760 , soh stx stx A stx enq D e i b y soh ack q \
0020000
[grid@orcl12c ~]$

 

Corruption of Secondary Block Copy (Mirror):

dd if=/dev/zero bs=8k count=1 seek=5894 of=/dev/oracleasm/disks/DISK2

 

Primary Block Copy:

[grid@orcl12c ~]$ dd if=/dev/oracleasm/disks/DISK3 bs=8k count=1 skip=171270 | od -a|tail -5
1+0 records in
1+0 records out
8192 bytes (8.2 kB) copied, 0.000157562 s, 52.0 MB/s
0017700 can cr can ff bel x q enq can cr can ff stx A stx del
0017720 del enq D ack us U dc1 del soh nul eot C bel 8 $ stx
0017740 A stx del del del dle em p r & # * syn ) c H
0017760 , soh stx stx A stx enq D e i b y soh ack q \
0020000
[grid@orcl12c ~]$

 

Secondary Block Copy (Mirror):

[grid@orcl12c ~]$ dd if=/dev/oracleasm/disks/DISK2 bs=8k count=1 skip=5894 | od -a|tail -5
1+0 records in
1+0 records out
8192 bytes (8.2 kB) copied, 2.3467e-05 s, 349 MB/s
0000000 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
*
0020000
[grid@orcl12c ~]$

 

Does RMAN report the corrupt block?

RMAN> validate check logical database;

File Status Marked Corrupt Empty Blocks Blocks Examined High SCN
---- ------ -------------- ------------ --------------- ----------
2 OK 0 6265 6400 1793137 
File Name: +DATA2/ORCL2/DATAFILE/dgomeztbs.270.867323161
Block Type Blocks Failing Blocks Processed
---------- -------------- ----------------
Data           0          5 
Index          0          0 
Other          0          130

 

Since RMAN didn't report any corrupted block, maybe it recovered the block?

[grid@orcl12c ~]$ dd if=/dev/oracleasm/disks/DISK2 bs=8k count=1 skip=5894 | od -a|tail -5
1+0 records in
1+0 records out
8192 bytes (8.2 kB) copied, 2.3467e-05 s, 349 MB/s
0000000 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
*
0020000

No. RMAN didn't report any corrupt block and didn't recover any block.

 

Corruption of Primary and Secondary Block (There is no any good copy of the block remaining):

Corruption of Primary Block Copy:

dd if=/dev/zero bs=8k count=1 seek=171270 of=/dev/oracleasm/disks/DISK3

 

Corruption of Secondary Block Copy (Mirror):

dd if=/dev/zero bs=8k count=1 seek=5894 of=/dev/oracleasm/disks/DISK2

 

Primary Block Copy:

[grid@orcl12c ~]$ dd if=/dev/oracleasm/disks/DISK3 bs=8k count=1 skip=171270 | od -a|tail -5
1+0 records in
1+0 records out
8192 bytes (8.2 kB) copied, 2.2628e-05 s, 362 MB/s
0000000 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
*
0020000
[grid@orcl12c ~]$

 

Secondary Block Copy (Mirror):

[grid@orcl12c ~]$ dd if=/dev/oracleasm/disks/DISK2 bs=8k count=1 skip=5894 | od -a|tail -5
1+0 records in
1+0 records out
8192 bytes (8.2 kB) copied, 7.4312e-05 s, 110 MB/s
0000000 nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul nul
*
0020000
[grid@orcl12c ~]$

 

Since there is no any remaining good block, will RMAN report a corrupt block?

RMAN> validate check logical database;

File Status Marked Corrupt Empty Blocks Blocks Examined High SCN
---- ------ -------------- ------------ --------------- ----------
2 FAILED 0 6265 6400 1792670 
File Name: +DATA2/ORCL2/DATAFILE/dgomeztbs.270.867323161
Block Type Blocks Failing Blocks Processed
---------- -------------- ----------------
Data        0             4 
Index       0             0 
Other       1             131
Tags:

About the Author

Deiby Gomez

Deiby Gómez is the first Oracle ACE Director of Guatemala. He has the highest technical certification in the world: "Oracle Certified Master 11g", "Oracle Certified Master 12c" and "Maximum Availability Architecture Oracle Certified Master 12c", he is the first person ever in Central America with all these certifications. Deiby likes to work with complex scenarios, huge and highly available critical databases where a deep knowledge of Oracle is needed. Deiby also has strong knowledge on Oracle Fusion Middleware and Oracle Cloud (PaaS & IaaS). Deiby was the winner of "IOUG SELECT Journal Editor’s Choice Award 2016" in Las Vegas, USA. He is a frequent speaker in Oracle Events around the World like OTN LAD Tour '13, '14, '15, '16, '17 (Colombia, Guatemala, El Salvador, Ecuador, Uruguay, Argentina, Brazil, Perú, Mexico, Costa Rica); Collaborate in Las Vegas, USA and Oracle Open World '15, '16, '17 (Brazil and USA). He was the first Guatemalan accepted as Beta Tester (12cR2) in San Francisco in 2015. Several articles have been published by him in English, Spanish and Portuguese in Oracle’s website, Toad World, and his own blog. Deiby appeared in the Official "Oracle Magazine" in Nov/Dec 2014 Edition as an outstanding expert. Deiby is the Technical Reviewer of the book “Oracle Database 12cR2 Multitenant - Oracle Press” and he is co-author of the book “Oracle Database 12cR2 Testing Tools and Techniques for Performance and Scalability - Oracle Press”. He loves to share his knowledge, to help people, to solve problems, to make friends and to play Chess.

Start the discussion at forums.toadworld.com