4-Node RAC with Service guard and CFS(HP-UX)

4-Node RAC installation is just like a 2-Node RAC in the fact that we need to prepare 4 nodes in stead of 2 nodes. If the hardware is in place max time required is 3 to 4 hours for a new DB installation.

Steps

1. Check SSH Connectivity among the cluster nodes

2. Check /etc/hosts and add VIP,IP and Interconnect details
3. Ping all the IP's, VIP will not ping at this stage
4. /app/oracle_source/clusterware/cluvfy/runcluvfy.sh stage -post hwos -n node1,node2,node3,node4 -verbose (from root)

ERROR:
Could not find a suitable set of interfaces for VIPs. Can be ignored at this stage
5. /app/oracle_source/clusterware/cluvfy/runcluvfy.sh stage -pre crsinst -n node1,node2,node3,node4 -verbose (from root)
6. Service guard health check
cmruncl -v
cmviewcl
cfscluster status
7. edit /app/oracle_source/clusterware/install/oraparam.ini to avoid version error =
vi /app/oracle_source/clusterware/install/oraparam.ini
HPUX=B.11.23 to HPUX=B.11.31
8. ./runInstaller (For clusterware installation)
9. Create OCR disks (2) and vote disk (3) before proceeding with oracle permission
10. after successful completion run /home/oracle/oraInventory/orainstRoot.sh and
/app/oracle/product/crs/root.sh as appear on screen on all 4 nodes
11. Oracle Cluster Verification Utility will fail.. Leave the screen as it is and open a new session with root and run vipca. After vipca run successfully, retry the previous screen and this time it will be success
12. At this moment, your vip should ping
13. Check on each node for crs and init.d
ps -ef|grep crs and ps -ef|grep init.d
14. Check crs-stat for vip inclusion
[root@xxxm71]:/app/oracle/product/crs/bin>./crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....m71.gsd application ONLINE ONLINE xxxm71
ora....m71.ons application ONLINE ONLINE xxxm71
ora....m71.vip application ONLINE ONLINE xxxm71
ora....m72.gsd application ONLINE ONLINE xxxm72
ora....m72.ons application ONLINE ONLINE xxxm72
ora....m72.vip application ONLINE ONLINE xxxm72
ora....m81.gsd application ONLINE ONLINE xxxm81
ora....m81.ons application ONLINE ONLINE xxxm81
ora....m81.vip application ONLINE ONLINE xxxm81
ora....m82.gsd application ONLINE ONLINE xxxm82
ora....m82.ons application ONLINE ONLINE xxxm82
ora....m82.vip application ONLINE ONLINE xxxm82

15. Applying patch 10.2.0.4
cd disk1
./runInstaller
16. run /app/oracle/product/crs/bin/crsctl stop crs and /app/oracle/product/crs/install/root102.sh as instructed by the screen on all nodes as root
17. Check on all nodes for
$ ./crsctl query crs activeversion
CRS active version on the cluster is [10.2.0.4.0]
$ ./crsctl query crs softwareversion
CRS software version on node [xxxm71] is [10.2.0.4.0]
17. Oracle Software Installation
edit oraparam.ini if required as mentioned in case of clusterware
18. run /app/oracle/product/10.2.0/db_1/root.sh as appeared on screen on all nodes from root
19. Applying 10.2.0.4 patch set
20. run dbca
21. database type > Real application cluster
option > create database
nodes> select all nodes
template>general purpose
db name>
Uncheck configure db with Enterprise manager..can do later
password for sys account
storage> cluster file system
location > use db file location from template
recovery option > enable archiving
edit archive mode parameter> %t_%s_%r.arc and give the location
Uncheck sample schemas
add a service > srv
TAF policy > Basic
memory> 40 % can be changed later
process> 600 for safe, can be changed later
SPfile name>
control file>
db file> change location as you planned
redo log> add 1 more member in each and make at least 512 M size of each member for performance aspect
Create database> FINISH
It will run netca automatically for listener

Database creation completer message !!! Congrats

Cluster database and HA services will run automaticall when click " EXIT"

22. check crs_stat for these additional services

ora.xxxm.db application ONLINE ONLINE xxxm71
ora....m1.inst application ONLINE ONLINE xxxm71
ora....m2.inst application ONLINE ONLINE xxxm72
ora....m3.inst application ONLINE ONLINE xxxm81
ora....m4.inst application ONLINE ONLINE xxxm82
ora....msrv.cs application ONLINE ONLINE xxxm82
ora....sm1.srv application ONLINE ONLINE xxxm71
ora....sm2.srv application ONLINE ONLINE xxxm72
ora....sm3.srv application ONLINE ONLINE xxxm81
ora....sm4.srv application ONLINE ONLINE xxxm82

Congratulations !!! RAC with 4 nodes done without any complexities :)

Next blog is 2-Node RAC with ASM on RHEL-5 (with lot of bugs) till then bye

Sunil

Life.. As a DBA

Life..As a DBA is cool,smooth,entertaining in fact it stands for DBA: Din Bhar Aram (Relaxing throught the day) till the application guy rushes to you with a not-so-happy mood and utters the magic words " My application running Slow". Then Starts the real life as a DBA

Once the day starts as a DBA, it starts with googling ORA- Errors and serching in a hurry in metalink for probable bug for the probable cause and hopefully for a probable solution. Alert logs, documentation become the hot objects to look at rather than Facebook.

As a DBA, it becomes a regular routine to clash with Your skills Vs Required skills. No doubt, the required skill wins. To conquer the later, there is one successful mantra i.e Read, read and read. You are already in the way to conquer the required skills as you are reading this blog :)

Everyone expects everything from a DBA and DBA has to satisfy them otherwise no doubt ball will be in your court. Everyone in the cabin will find out the RCA (Root Cause Analysis) for the issue is due to bottleneck of database. The more you try to defend, the more fingers will be raised. So, the required skills for a DBA is infinity. He requires SA(system Admin) skills, Storage admin skills, backup admin skills, developing skills, Application skills and most importantly skill to defend that Your DB is running smooth which requires Close observation and close monitoring. So, the moral of the story is even if one DBA does not have mood to work, everyone make him to work.

But in my view, DBA should work as minimal as possible. As per my experience goes, whenever one DBA works, there must be some issue happening some where in DB or someone facing some issue in his application.So, If the DBA simply surfing or chatting without monitoring the Black and white screen, one can be pretty much sure that everything is running smooth and no intervene required.

Thus Let the DBA do his task i.e Din Bhar Aram and let no one else disturb him throughout his Shift for a good prospect of the project and the company. With this appeal, i m starting my blogging carrer of DBA as there is not much of work to do rather than writing blogs in the shift. If anyone complains, I can strongly justify myself that i am getting time to blog as my database is perfectly tuned.

As a DBA,
Sunil