Oct 19, 2017 4:27:00 AM by Syed Jaffar Hussain
Exadata Storage expansion
Most of us knew the capabilities that Exadata Database Machine delivers. Its known fact that Exadata comes in different fixed rack size capacity: 1/8 rack (2 db nodes, 3 cells), quarter rack (2 db nodes, 3 cells), half rack (4 db nodes, 7 cells) and full rack (8 db nodes, 14 cells). When you want to expand the capacity, it must be in fixed size as well, like, 1/8 to quarter, quarter to half and half to full.
With Exadata X5 Elastic configuration, one can also have customized sizing by extending capacity of the rack by adding any number of DB servers or storage servers or combination of both, up to the maximum allowed capacity in the rack.
In this blog post, I will summarize and walk through a procedure about extending Exadata storage capacity, i.e, adding a new cell to an existing Exadata Database Machine.
If the image version on the new cell doesn’t match with the existing image version, either you download the exact image version and re-image the new cell or upgrade the image on the existing servers.
Review "Reimaging Exadata Cell Node Guidance (Doc ID 2151671.1)" if you want to reimage the new cell.
Add the IP addresses acquired for the new cell to the /etc/oracle/cell/network-config/cellip.ora file on each DB node. To do this, perform the steps below from the first 1 db serer in the cluster:
cp cellip.ora cellip.ora.orig
cp cellip.ora cellip.ora-bak
/usr/local/bin/dcli -g database_nodes -l root -f cellip.ora-bak -d /etc/oracle/cell/network-config/cellip.ora
CellCLI> list cell attributes snmpsubscriber
CellCLI> alter cell snmpSubscriber=((host='10.20.14.21',port=162,community=public))
CellCLI> list cell attributes
CellCLI> alter cell notificationmethod='mail,snmp',notificationpolicy='critical,warning,clear',smtptoaddr= 'firstname.lastname@example.org',smtpfrom=
CellCLI> create celldisk all
CellCLI> list flashlog
You should see the name of the flash log. It should look like cellnodename_FLASHLOG, and its status should be "normal".If the flash log does not exist, create it using :CellCLI> create flashlog all
CellCLI> list cell attributes flashcachemode
To change the flash cache mode to match the flash cache mode of existing cells, do the following:
CellCLI> alter flashcache all flush
Wait for the command to return.
CellCLI> "drop flashcache all"
CellCLI> "alter cell flashCacheMode=writeback_or_writethrough"
The value of the flashCacheMode attribute is either writeback or writethrough. The value must match the flash cache mode of the other storage cells in the cluster.
cellcli -e create flashcache all
CellCLI> list griddisk attributes name,asmDiskGroupName,cachingpolicy,size,offset
DATAC1 default 5.6953125T 32M
DBFS_DG default 33.796875G 7.1192474365234375T
RECOC1 none 1.42388916015625T 5.6953582763671875T
When creating grid disks, begin with DATAC1, then RECOC1, and finally DBFS_DG using the following command:
CellCLI> create griddisk ALL HARDDISK PREFIX=DATAC1, size=5.6953125T, cachingpolicy='default',
comment="Cluster cluster-clux6 DR diskgroup DATAC1"
CellCLI> create griddisk ALL HARDDISK PREFIX=RECOC1,size=1.42388916015625T, cachingpolicy='none',
comment="Cluster cluster-clux6 DR diskgroup RECOC1"
CellCLI> create griddisk ALL HARDDISK PREFIX=DBFS_DG,size=33.796875G, cachingpolicy='default',
comment="Cluster cluster-clux6 DR diskgroup DBFS_DG"
CAUTION: Be sure to specify the EXACT size shown along with the unit (either T or G).
$GI_HOME/bin/kfod op=disks disks=all | grep cellName_being_added
This should list all the grid disks created in step 7 above.
alter diskgroup disk_group_nameadd disk 'comma_separated_disk_names';
The command above kicks off an ASM rebalance at the default power level. Monitor the progress of the rebalance by querying gv$asm_operation :
SQL> select * from gv$asm_operation;
Once the rebalance completes, the addition of the cell to the Oracle RAC is complete.
Reimaging Exadata Cell Node Guidance (Doc ID 2151671.1)
Written by Syed Jaffar Hussain
An Oracle Database Expert for over 15 years from his 20 years of Information Technology (IT) career. Over the past 15 years of Oracle journey, he involved with several local and large scaled international banks where he implemented and managed highly complex cluster and non-cluster environments with over 100’s of business critical databases. Recognizing his efforts and contribution towards the Oracle community, Oracle awarded him the prestigious ‘Best DBA of the year, 2011’ and Oracle ACE Director status. He also acquired industry best Oracle credentials, Oracle Certified Master (OCM), Oracle RAC Expert, OCP DBA 8i,9i,10g & 11g in addition to ITIL Expertise.
Syed is an active Oracle speaker, regularly presents technical sessions and webinars on various Oracle database technologies at many Oracle events. You can visit his technical blog, http://jaffardba.blogspot.com where he discuss and writes the workaround/solution about the issues confronted from his day-to-day activities. Apart from being the part of the core Technical Review committee member for a few Oracle technology oriented books, he also co-authored an Oracle 11g R1/R2 Real Application Cluster Essentials and Oracle Expert RAC books. His blog can be found at http://jaffardba.blogspot.com/