Yet another exciting journey with Oracle bugs and challenges. Here is the story for you.
One of our recent successful migrations was a single instance Oracle EBS 12cR1 database to Oracle Super Cluster M7 as a RAC database with 2 instances on the same DB version (12.1.0.2). Subsequently, the customer wants to run through EBS cloning and set up an Oracle active data guard configuration.
The target systems are not Super Cluster. The requirement to clone and set up an Oracle data guard is to configure as a single-instance database onto a filesystem (non-ASM). After initiating the cloning procedure using the DUPLICATE TARGET DATABASE method trough RMAN, we noticed that RMAN is taking significant time to restore(ship) the data files to the remote server. Also, the following warning messages were appeared in the alert.log:
ASMB started with pid=63, OS id=18085
WARNING: failed to start ASMB (connection failed) state=0x1 sid=''
WARNING: ASMB exiting with error
Starting background process ASMB
Sat Mar 11 13:53:24 2017
ASMB started with pid=63, OS id=18087
WARNING: failed to start ASMB (connection failed) state=0x1 sid=''
WARNING: ASMB exiting with error
Starting background process ASMB
Sat Mar 11 13:53:27 2017
ASMB started with pid=63, OS id=18089
WARNING: failed to start ASMB (connection failed) state=0x1 sid=''
WARNING: ASMB exiting with error
Starting background process ASMB
The situation raised couple of concerns in our minds:
- Why is the restore is too slow from RMAN? (while there is no Network latency and DB files are not so big sized)
- Why Oracle is looking for an ASM instance on a non-Cluster home? (not even a standard Grid home)
After some initial investigation, we come across following MOS Docs:
- '12c RMAN Operations from ASM To Non-ASM Slow (Doc ID 2081537.1)'.
- WARNING: failed to start ASMB after RAC Database on ASM converted to Single Instance Non-ASM Database (Doc ID 2138520.1)
According to the above MOS Docs, this is an expected behavior due to an Unpublished BUG 19503821: RMAN CATALOG EXTREMELY SLOW WHEN MIGRATING DATABASE FROM ASM TO FILE SYSTEM
You need to apply a patch 19503821 to overcome from the bug.
If you similar demand, make sure you apply the patch in your environmet before you proceed with the restore/duplicate procedure.
-- Excerpt from the above notes:
APPLIES TO:
Oracle Database - Enterprise Edition - Version 12.1.0.1 to 12.1.0.2 [Release 12.1]
Information in this document applies to any platform.
SYMPTOMS:
1*. RAC Database with ASM has been converted or restored to Standalone Single Instance Non-ASM Database.
2*. From the RDBMS alert.log, it is showing continuous following messages.
3*.RMAN Restore/Duplicate from ASM to Non ASM in 12.1 take a longer time waiting for the ASMB process.
4*.Any RMAN command at the mount state which involves Non ASM location can take more time.
SOLUTION:
Apply the patch 19503821, if not available for your version/OS then please log a SR with the support to get the patch for your version.
An Oracle Database Expert for over 15 years from his 20 years of Information Technology (IT) career. Over the past 15 years of Oracle journey, he involved with several local and large scaled international banks where he implemented and managed highly complex cluster and non-cluster environments with over 100’s of business critical databases. Recognizing his efforts and contribution towards the
Oracle community, Oracle awarded him the prestigious ‘Best DBA of the year, 2011’ and Oracle ACE Director status. He also acquired industry best Oracle credentials, Oracle Certified Master (OCM), Oracle RAC Expert, OCP DBA 8i,9i,10g & 11g in addition to ITIL Expertise.
Syed is an active Oracle speaker, regularly presents technical sessions and webinars on various Oracle database technologies at many Oracle events. You can visit his technical blog, http://jaffardba.blogspot.com where he discuss and writes the workaround/solution about the issues confronted from his day-to-day activities. Apart from being the part of the core Technical Review committee member for a few Oracle technology oriented books, he also co-authored an Oracle 11g R1/R2 Real Application Cluster Essentials and Oracle Expert RAC books. His blog can be found at http://jaffardba.blogspot.com/
