Setup Oracle RAC 19c 2-Node Cluster on OEL 8 with Grid Infrastructure

Setup Oracle RAC 19c two-node cluster on OEL 8 with Grid Infrastructure 19c. A comprehensive, step-by-step guide for successful installation & config.

Overview

In the realm of enterprise-grade applications, high availability and scalability are not mere luxuries; they are fundamental requirements. For critical systems like Oracle PeopleSoft, ensuring uninterrupted service and the ability to scale resources on demand is paramount. Oracle Real Application Clusters (RAC) stands as Oracle's flagship solution for achieving these objectives, providing a single database spread across multiple physical nodes, offering resilience against node failures and enabling seamless workload distribution.

This article delves into the meticulous process of setting up a two-node Oracle RAC 19c cluster on Oracle Linux 8 (OEL 8), leveraging Grid Infrastructure 19c. Oracle 19c, designated as a Long Term Support (LTS) release, offers enhanced stability, performance, and security features, making it an ideal choice for mission-critical deployments. OEL 8, with its robust kernel and enterprise-grade features, provides a solid, secure, and optimized foundation for Oracle databases. By combining these technologies, we construct a highly available, scalable, and performant database infrastructure capable of supporting the most demanding PeopleSoft environments, minimizing downtime and maximizing operational efficiency.

The journey through this setup will cover everything from initial operating system configuration and shared storage preparation to the intricate steps of installing Oracle Grid Infrastructure and creating a RAC database. We will emphasize practical, command-line driven approaches, ensuring clarity and reproducibility for DBAs and system architects aiming to implement a resilient Oracle RAC solution.

Prerequisites

A successful Oracle RAC deployment hinges on meticulous preparation. This section outlines the essential hardware, software, and network prerequisites for our two-node RAC 19c cluster on OEL 8.

Hardware Requirements (per node)

CPU: Minimum 2 cores (Intel/AMD x86-64 architecture). For production, 4+ cores are recommended.
RAM: Minimum 8 GB. For production, 16 GB or more is highly recommended, especially for PeopleSoft workloads.
Disk Space:
- OS & Oracle Binaries: 50 GB for OS, 20 GB for Grid Infrastructure Home, 20 GB for Oracle Database Home.
- Shared Storage: This is critical for ASM disk groups. We will need:
  - 1 LUN for OCR/Voting Disks (e.g., 5 GB).
  - 1 LUN for DATA Disk Group (e.g., 100 GB or more, depending on database size).
  - 1 LUN for FRA (Fast Recovery Area) Disk Group (e.g., 50 GB or more).
  These LUNs must be presented identically to both cluster nodes. iSCSI, Fibre Channel, or NFS (for certain components) are common choices. For this guide, we'll assume iSCSI LUNs.
Network Interfaces: Each node requires at least three network interfaces:
- Public Network: For client connections and VIPs. (e.g., 192.168.1.0/24)
- Private Interconnect: For inter-node communication (Cache Fusion). Must be high-speed and dedicated. (e.g., 10.0.0.0/24)
- Optional: Management Network: For administrative access, separate from public.

Software Requirements

Oracle Linux 8 (OEL 8) installed on both nodes.
Oracle Grid Infrastructure 19c software (zip file).
Oracle Database 19c software (zip file).
oracle-database-preinstall-19c RPM for OEL 8.

Network Configuration

Plan your IP addresses carefully. For this setup, we'll use the following convention:

Component	Node 1 (racnode1)	Node 2 (racnode2)	Description
Public IP	192.168.1.101	192.168.1.102	Client access
Private IP	10.0.0.101	10.0.0.102	Interconnect (high-speed)
Virtual IP (VIP)	192.168.1.103	192.168.1.104	Managed by GI, floats on node failure
SCAN IP (3)	192.168.1.105, 192.168.1.106, 192.168.1.107		Single Client Access Name

User and Group Accounts

Standard Oracle installation requires specific users and groups:

oinstall: Oracle Inventory group.
dba: OSDBA group (for database administration).
oper: OSOPER group (for database operator privileges).
asmadmin: OSASM group (for ASM administration).
asmdba: OSDBA group for ASM (for database access to ASM).
asmoper: OSOPER group for ASM.
grid: OS user for Grid Infrastructure owner.
oracle: OS user for Oracle Database owner.

The oracle-database-preinstall-19c RPM typically creates these users and groups, but it's good to be aware.

Step-by-step Implementation

1. Operating System Configuration (Node 1 & Node 2)

Perform these steps on both racnode1 and racnode2.

1.1. Install Oracle Linux 8

Ensure a minimal installation of OEL 8. Set hostnames:


# On racnode1
hostnamectl set-hostname racnode1.techventure.com

# On racnode2
hostnamectl set-hostname racnode2.techventure.com

1.2. Configure Network Interfaces

Edit network configuration files (e.g., /etc/sysconfig/network-scripts/ifcfg-eth0 for public, ifcfg-eth1 for private). Assign static IPs as per the plan. Example for racnode1:


# /etc/sysconfig/network-scripts/ifcfg-eth0 (Public)
TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"
BOOTPROTO="static"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="no"
NAME="eth0"
UUID="<your-uuid>"
DEVICE="eth0"
ONBOOT="yes"
IPADDR="192.168.1.101"
PREFIX="24"
GATEWAY="192.168.1.1"
DNS1="192.168.1.1"

# /etc/sysconfig/network-scripts/ifcfg-eth1 (Private Interconnect)
TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"
BOOTPROTO="static"
DEFROUTE="no"
IPV4_FAILURE_FATAL="no"
IPV6INIT="no"
NAME="eth1"
UUID="<your-uuid>"
DEVICE="eth1"
ONBOOT="yes"
IPADDR="10.0.0.101"
PREFIX="24"

Restart network services:


sudo systemctl restart NetworkManager
sudo systemctl restart network

1.3. Configure /etc/hosts

Add all public, private, and SCAN IPs to /etc/hosts on both nodes. This is crucial for name resolution.


# /etc/hosts on both racnode1 and racnode2
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

# Public IPs
192.168.1.101   racnode1.techventure.com   racnode1
192.168.1.102   racnode2.techventure.com   racnode2

# Private IPs
10.0.0.101      racnode1-priv.techventure.com  racnode1-priv
10.0.0.102      racnode2-priv.techventure.com  racnode2-priv

# SCAN IPs (Ensure these are NOT in DNS for a simple lab setup, or correctly configured in DNS)
192.168.1.105   racscan.techventure.com   racscan
192.168.1.106   racscan.techventure.com   racscan
192.168.1.107   racscan.techventure.com   racscan

1.4. Disable SELinux and Firewall

For a lab environment, temporarily disabling SELinux and the firewall simplifies the setup. For production, configure them securely.


# Disable SELinux (requires reboot)
sudo sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
sudo reboot

# Stop and disable firewalld
sudo systemctl stop firewalld
sudo systemctl disable firewalld

1.5. Install Oracle Preinstallation RPM

This RPM configures kernel parameters, creates necessary OS users/groups, and installs required packages.


sudo dnf -y install oracle-database-preinstall-19c

1.6. Configure NTP (Network Time Protocol)

Time synchronization is vital for RAC. Configure Chrony or NTP on both nodes and ensure they sync.


# Install chrony if not present
sudo dnf -y install chrony

# Edit /etc/chrony.conf (example)
# Add/modify server lines to point to your NTP servers
# server 0.pool.ntp.org iburst
# server 1.pool.ntp.org iburst
# server 2.pool.ntp.org iburst
# server 3.pool.ntp.org iburst

# Restart and enable chronyd
sudo systemctl enable chronyd --now
sudo chronyc sources -v

1.7. Create Oracle Base and Inventory Directories

These directories will house Oracle software. Ensure correct ownership.


sudo mkdir -p /u01/app/oraInventory
sudo mkdir -p /u01/app/grid
sudo mkdir -p /u01/app/oracle

sudo chown -R grid:oinstall /u01/app/oraInventory
sudo chown -R grid:oinstall /u01/app/grid
sudo chown -R oracle:oinstall /u01/app/oracle

sudo chmod -R 775 /u01/app

1.8. Configure SSH Equivalency

The Grid Infrastructure installer requires passwordless SSH connectivity between all cluster nodes for the grid user. This is a critical step.


# As grid user on racnode1
su - grid
ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa
ssh-copy-id racnode1
ssh-copy-id racnode2
exit

# As grid user on racnode2 (repeat for racnode2 to racnode1)
su - grid
ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa
ssh-copy-id racnode1
ssh-copy-id racnode2
exit

Verify SSH equivalency by attempting to SSH from each node to itself and the other node as the grid user without a password.

2. Shared Storage Configuration

For this guide, we'll assume iSCSI LUNs are presented to both nodes. We'll use udev rules for persistent device naming and permissions, which is a modern and robust approach compared to oracleasm.

2.1. Configure iSCSI Initiator (on both nodes)

Install iSCSI utilities, discover, and log in to the iSCSI target (your storage server).


sudo dnf -y install iscsi-initiator-utils

# Replace <ISCSI_TARGET_IP> with your iSCSI storage server's IP
sudo iscsiadm -m discovery -t st -p <ISCSI_TARGET_IP>
sudo iscsiadm -m node -L all -o update
sudo iscsiadm -m node -T iqn.<target-name> -p <ISCSI_TARGET_IP> --login
sudo systemctl enable iscsid --now

Verify the new disks (e.g., /dev/sdb, /dev/sdc, /dev/sdd) using lsblk or fdisk -l.

2.2. Create UDEV Rules for ASM Disks (on both nodes)

We need to create consistent, persistent device names and set permissions for ASM. Identify the disks using their WWN (World Wide Name) to ensure persistence across reboots and consistent naming across nodes.


# Identify WWN for each disk
sudo /usr/lib/udev/scsi_id --whitelisted --replace-whitespace /dev/sdb
sudo /usr/lib/udev/scsi_id --whitelisted --replace-whitespace /dev/sdc
sudo /usr/lib/udev/scsi_id --whitelisted --replace-whitespace /dev/sdd

# Example output:
# 3600144f0b2f1e0000000000000010001 (for /dev/sdb - OCRVOTE)
# 3600144f0b2f1e0000000000000020002 (for /dev/sdc - DATA)
# 3600144f0b2f1e0000000000000030003 (for /dev/sdd - FRA)

Create /etc/udev/rules.d/99-oracle-asmdisks.rules on both nodes:


# /etc/udev/rules.d/99-oracle-asmdisks.rules
# OCR/VOTE Disk (5GB)
KERNEL=="sd*", ENV{ID_WWN}=="3600144f0b2f1e0000000000000010001", SYMLINK+="oracleasm/ocrcvote", OWNER="grid", GROUP="asmadmin", MODE="0660"
# DATA Disk (100GB)
KERNEL=="sd*", ENV{ID_WWN}=="3600144f0b2f1e0000000000000020002", SYMLINK+="oracleasm/data", OWNER="grid", GROUP="asmadmin", MODE="0660"
# FRA Disk (50GB)
KERNEL=="sd*", ENV{ID_WWN}=="3600144f0b2f1e0000000000000030003", SYMLINK+="oracleasm/fra", OWNER="grid", GROUP="asmadmin", MODE="0660"

Apply the rules:


sudo udevadm control --reload-rules
sudo udevadm trigger
ls -l /dev/oracleasm/

You should see the symlinks created with correct ownership and permissions.

3. Grid Infrastructure Installation (Node 1)

These steps are performed primarily on racnode1 as the installation orchestrator.

3.1. Unzip Grid Infrastructure Software

Download LINUX.X64_193000_grid_home.zip to /u01/app/grid. Unzip as the grid user.


su - grid
cd /u01/app/grid
unzip /path/to/LINUX.X64_193000_grid_home.zip

3.2. Run GridSetup.sh

Navigate to the unzipped Grid Home and start the installer.


cd /u01/app/grid
./gridSetup.sh

Follow the Oracle Universal Installer (OUI) prompts:

Select Configuration Option: "Configure and Install Grid Infrastructure for a Cluster".
Select Cluster Type: "Typical Cluster".
Grid Infrastructure Management Repository: "Do not configure GIMR". (For simple two-node cluster, not typically needed).
Cluster Node Information: Add racnode1 and racnode2. OUI should detect them via SSH equivalency.
Specify Network Interface Usage:
- eth0: Public
- eth1: Private
Specify ASM Disk Group:
- Disk Group Name: OCR_VOTE
- Redundancy: External (if using hardware RAID) or Normal (for 3+ disks). For our 1 disk setup, choose External.
- Add Disks: Select /dev/oracleasm/ocrcvote.
Specify ASM Password: Enter a strong password.
Specify Installation Location:
- Oracle Base: /u01/app/grid
- Software Location: /u01/app/grid/product/19.0.0/grid (This will be automatically populated)
Privileged Operating System Groups: These should be pre-populated by the preinstall RPM.
- ASM Administrator OS Group: asmadmin
- ASM DBA OS Group: asmdba
- ASM Operator OS Group: asmoper
- Database Administrator OS Group: dba
- Database Operator OS Group: oper
Root script execution configuration: "Automatically run configuration scripts". Provide root user password.
Prerequisite Checks: Review and fix any warnings or failures. The preinstall RPM should handle most.
Summary: Review all details.
Install: Begin installation.

During the installation, OUI will prompt to run root.sh on both nodes. If you chose automatic execution, it will proceed. Otherwise, manually run /u01/app/grid/product/19.0.0/grid/root.sh as root on both nodes when prompted.


# Example of running root.sh on each node (if not automated)
# On racnode1
sudo /u01/app/grid/product/19.0.0/grid/root.sh

# On racnode2
sudo /u01/app/grid/product/19.0.0/grid/root.sh

Once complete, the Grid Infrastructure installation should succeed.

4. Database Software Installation (Node 1)

This installs the Oracle Database binaries, not the database itself. Performed on racnode1.

4.1. Unzip Database Software

Download LINUX.X64_193000_db_home.zip to /u01/app/oracle. Unzip as the oracle user.


su - oracle
cd /u01/app/oracle
unzip /path/to/LINUX.X64_193000_db_home.zip

4.2. Run runInstaller


cd /u01/app/oracle/product/19.0.0/dbhome_1
./runInstaller

Follow the OUI prompts:

Select Configuration Option: "Set Up Software Only".
Database Installation Option: "Oracle Real Application Clusters database installation".
Database Edition: "Enterprise Edition" (or your desired edition).
Oracle Home User: "Use existing Windows User" (select "Use existing Oracle Home User" for Linux). Specify oracle.
Installation Location:
- Oracle Base: /u01/app/oracle
- Software Location: /u01/app/oracle/product/19.0.0/dbhome_1
Privileged Operating System Groups: These should be pre-populated.
- Database Administrator OS Group: dba
- Database Operator OS Group: oper
Prerequisite Checks: Review and fix any warnings or failures.
Summary: Review all details.
Install: Begin installation.

You will be prompted to run root.sh on each node. Manually run it as root on both nodes.


# On racnode1
sudo /u01/app/oracle/product/19.0.0/dbhome_1/root.sh

# On racnode2
sudo /u01/app/oracle/product/19.00/dbhome_1/root.sh

5. Database Creation (Node 1)

Now, we create the actual RAC database using DBCA (Database Configuration Assistant).

5.1. Prepare Environment Variables

As the oracle user, set environment variables.


su - oracle
export ORACLE_HOME=/u01/app/oracle/product/19.0.0/dbhome_1
export ORACLE_BASE=/u01/app/oracle
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib

5.2. Run DBCA


dbca

Follow the DBCA prompts:

Database Operation: "Create Database".
Creation Mode: "Advanced Configuration".
Database Deployment Type: "Oracle Real Application Clusters database". Select all nodes (racnode1, racnode2).
Database Type: "General Purpose or Transaction Processing".
Database Identification:
- Global Database Name: PEOPLEDB.techventure.com
- Oracle SID Prefix: PEOPLEDB
- Container Database: "Create as Container database" (optional, depending on your architecture). For PeopleSoft, typically not a CDB, so uncheck.
Configuration Options:
- Memory: Set appropriate SGA/PGA sizes (e.g., 4096MB SGA, 2048MB PGA for PeopleSoft).
- Character sets: AL32UTF8 (recommended for PeopleSoft).
Storage Locations:
- Storage Type: "Automatic Storage Management (ASM)".
- Database Files Location: +DATA (This is where our /dev/oracleasm/data disk will be used).
- Fast Recovery Area (FRA) Location: +FRA (This is where our /dev/oracleasm/fra disk will be used).
- Multiplex Redo Logs and Control Files: Let DBCA handle this or specify.
ASM Disk Groups: DBCA will prompt to create DATA and FRA disk groups.
- DATA: Select /dev/oracleasm/data, Redundancy: External.
- FRA: Select /dev/oracleasm/fra, Redundancy: External.
Management Options: "Configure Enterprise Manager (EM) Database Express".
Database Credentials: Set passwords for SYS, SYSTEM, etc.
Creation Options: "Create Database".

DBCA will now proceed to create the database instances on both nodes. This process can take a significant amount of time.

6. Post-Installation Verification

Once DBCA completes, verify the cluster and database status.


# As grid user
crsctl stat res -t

This command should show all cluster resources (CSS, Diskgroup, Listener, SCAN, VIPs, Database instances) online on their respective nodes.


# Example output snippet:
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.DATA.dg
      1        ONLINE

Setup Oracle RAC 19c 2-Node Cluster on OEL 8 with Grid Infrastructure 19c