Have any questions?
info@datainfinity.net
2nd Floor, Suite 47, Vision Plaza, Msa Road
FOLLOW US
17
Aug

Oracle Database 11g: RAC Administration 

0

Introduction

In modern organizations, databases are expected to be available almost all the time. Banks, hospitals, telecommunications companies, universities, and e-commerce platforms cannot afford long database outages. This is where Oracle Real Application Clusters (RAC) becomes important.

Oracle RAC allows multiple servers, called nodes, to work together and access the same Oracle database. Instead of relying on one server, a RAC environment distributes database processing across multiple instances. If one node fails, the remaining nodes can continue providing database services. Oracle describes RAC as an architecture designed for high availability and scalability.

Oracle Database 11g RAC administration involves managing the database instances, clusterware, storage, services, networking, performance, backup and recovery, and the overall health of the cluster.

What Is Oracle RAC?

Oracle Real Application Clusters (RAC) is a technology that allows multiple Oracle database instances running on different servers to access a single shared database.

For example, imagine a two-node RAC environment:

              Applications
                   |
          -------------------
          |                 |
       Node 1             Node 2
       RAC Instance       RAC Instance
          |                 |
          -----------+-------
                     |
              Shared Storage
                     |
              Oracle Database

Both nodes work with the same database, but each node has its own database instance and memory structures.

The major objectives of RAC are:

  1. High availability
  2. Scalability
  3. Load distribution
  4. Fault tolerance
  5. Easier maintenance
  6. Continuous database services

Oracle’s 11g RAC documentation specifically covers administration of RAC databases, Clusterware, ASM, storage, services, monitoring, backup and recovery, and node management.

1. Main Components of Oracle RAC

A RAC environment consists of several important components.

1.1 RAC Nodes

A node is a physical or virtual server participating in the cluster.

For example:

Node 1
Oracle Instance: RAC1

Node 2
Oracle Instance: RAC2

Each node has its own:

  1. CPU
  2. RAM
  3. Operating system
  4. Oracle instance
  5. Background processes

However, the nodes work together as one database service.

1.2 Oracle Database Instance

Each RAC node normally runs its own Oracle database instance.

An instance contains:

  1. System Global Area (SGA)
  2. Background processes
  3. Database connections
  4. Redo processes
  5. Memory structures

For example:

Database: SALES

Instance 1 → Node 1
Instance 2 → Node 2

Both instances access the same database files.

1.3 Oracle Clusterware

Oracle Clusterware is responsible for managing the cluster.

It helps Oracle determine:

  1. Which nodes are available
  2. Which resources are running
  3. Whether a node has failed
  4. Where database services should run
  5. How cluster resources should be started and stopped

Oracle’s RAC architecture uses Clusterware as an important component for coordinating the servers in the cluster.

1.4 Oracle ASM

Automatic Storage Management (ASM) provides storage management for RAC environments.

ASM can manage:

  • Data files
  • Control files
  • Redo logs
  • Archived logs
  • Backup files

A simplified storage architecture is:

Node 1 ──┐
         ├── Oracle ASM ── Shared Storage
Node 2 ──┘

ASM is particularly important because multiple RAC instances need access to the same database files. Oracle’s 11g RAC documentation includes dedicated administration procedures for ASM and RAC storage.

2. RAC Networking

Networking is extremely important in a RAC environment.

A typical RAC installation uses several types of network communication.

Public Network

Used by:

  • Applications
  • Database clients
  • Administrators

Private Interconnect

Used for communication between RAC nodes.

For example:

Node 1  <==========>  Node 2
          Private
        Interconnect

The private interconnect is important for RAC’s internal coordination and Cache Fusion.

Virtual IP

RAC also uses Virtual IP addresses to improve client connectivity and failover behavior.

Oracle’s RAC documentation describes connectivity using services and VIP addresses as part of RAC architecture.

3. Cache Fusion

One of the most important concepts in RAC administration is Cache Fusion.

Each RAC instance has its own memory cache. However, users connected to different instances may need to access the same data blocks.

Cache Fusion allows blocks to be transferred between instances through the private interconnect rather than requiring every request to read the block from disk.

For example:

Instance 1 Cache
       |
       | Cache Fusion
       ↓
Private Interconnect
       |
       ↓
Instance 2 Cache

This allows RAC instances to coordinate access to shared data.

The Global Cache Service (GCS) and Global Enqueue Service (GES) are important components involved in RAC’s global coordination and Cache Fusion architecture. Oracle provides dedicated guidance for monitoring GCS/GES statistics and Cache Fusion performance.

4. RAC Administration Tools

A database administrator has several tools available for managing Oracle RAC.

SQL*Plus

SQL*Plus can be used for database administration tasks such as:

SELECT INSTANCE_NAME, STATUS
FROM V$INSTANCE;

This allows an administrator to check the current instance.

SRVCTL

Server Control Utility (SRVCTL) is one of the most important RAC administration tools.

It can be used to manage:

  1. RAC databases
  2. Instances
  3. Services
  4. ASM
  5. Network resources
  6. Cluster resources

For example:

srvctl status database -d ORCL

To start a database:

srvctl start database -d ORCL

To stop a database:

srvctl stop database -d ORCL

Oracle identifies SRVCTL as a central command-line utility for managing RAC databases, instances, services, and related resources.

Enterprise Manager

Oracle Enterprise Manager provides a graphical interface for administering and monitoring RAC.

It can be used to:

  • Monitor instances
  • Monitor cluster performance
  • Manage services
  • View alerts
  • Monitor storage
  • Analyze database performance

Oracle’s RAC documentation lists Enterprise Manager alongside SRVCTL and SQL*Plus as administration tools.

5. Starting and Stopping RAC

An administrator may need to start or stop individual instances or the entire database.

Check database status

srvctl status database -d ORCL

Example output might indicate that the database is running on multiple nodes.

Start database

srvctl start database -d ORCL

Stop database

srvctl stop database -d ORCL

Check services

srvctl status service -d ORCL

These commands are useful because RAC resources should generally be managed through Clusterware-aware tools rather than treating each node as an isolated database server. Oracle’s RAC administration guide provides specific procedures for starting and stopping RAC databases and instances using SRVCTL.

6. Managing RAC Services

A service represents a workload or application connection group.

For example, a company might have:

SALES_SERVICE
HR_SERVICE
PAYROLL_SERVICE

Different applications can connect through different services.

Services can help administrators:

  1. Distribute workloads
  2. Control where applications connect
  3. Manage application availability
  4. Perform planned maintenance
  5. Support workload management

Oracle RAC 11g provides service administration through SRVCTL and Enterprise Manager.

Example:

srvctl status service -d ORCL

7. Managing Storage

Storage administration is another major responsibility of a RAC administrator.

The administrator must monitor:

  1. ASM disk groups
  2. Free space
  3. Data files
  4. Redo logs
  5. Control files
  6. Archived logs

For example, ASM disk groups might be:

+DATA
+FRA

Where:

  • +DATA stores database files
  • +FRA stores recovery-related files

The administrator should continuously monitor available storage to prevent database failures caused by full disk groups.

8. Monitoring Oracle RAC

Monitoring helps administrators detect problems before they become major failures.

Important areas to monitor include:

CPU

High CPU utilization may indicate:

  1. Heavy workloads
  2. Poor SQL
  3. Too many sessions
  4. Insufficient server resources

Memory

Monitor:

  1. SGA
  2. PGA
  3. OS memory
  4. Memory pressure

Network

The RAC interconnect should be monitored carefully because excessive interconnect traffic can affect performance.

Database Sessions

Administrators should monitor:

  • Active sessions
  • Blocked sessions
  • Long-running queries
  • Failed connections

Cluster Health

Administrators should verify that:

  • All nodes are available
  • Instances are running
  • Services are online
  • ASM is healthy
  • Cluster resources are online

Oracle provides RAC-specific monitoring capabilities for the cluster database, interconnects, performance, AWR, ASH, and Cache Fusion statistics.

9. RAC Performance Tuning

RAC performance tuning is different from tuning a single-instance database because administrators must consider communication between nodes.

Some important areas include:

SQL Performance

Poor SQL statements can consume excessive resources.

Administrators should examine:

  • Execution plans
  • Index usage
  • Full table scans
  • CPU consumption
  • I/O

Cache Fusion

High amounts of block transfer between nodes may indicate application or workload problems.

Interconnect Performance

The private interconnect should have sufficient bandwidth and low latency.

Workload Distribution

Applications should be distributed appropriately across available RAC instances.

AWR and ASH

Oracle’s Automatic Workload Repository (AWR) and Active Session History (ASH) can help identify performance bottlenecks in RAC environments.

10. Backup and Recovery

RAC databases still require proper backup and recovery strategies.

RMAN — Recovery Manager is commonly used for Oracle database backup and recovery.

A RAC administrator should consider:

  • Database backups
  • Archived redo logs
  • Control file backups
  • SPFILE backups
  • Recovery procedures
  • Backup validation

Oracle’s RAC administration documentation includes specific sections covering RMAN configuration, archived redo logs, instance recovery, media recovery, and Fast Recovery Area management in RAC.

A simple RMAN connection could look like:

rman target /

A basic backup example:

BACKUP DATABASE;

In production environments, backup strategies should be carefully designed and regularly tested.

11. Node Failure

One of the major benefits of RAC is improved availability.

Consider:

Node 1 → FAILED 

Node 2 → RUNNING

Applications connected appropriately through RAC services can continue using the surviving node.

This does not mean RAC eliminates every type of outage. Administrators must still design networking, storage, applications, backups, and services correctly.

Oracle’s 11g RAC material describes RAC as providing high availability and allowing workloads to continue on remaining servers when a server fails.

12. Adding and Removing Nodes

As an organization’s workload grows, administrators may need to add another server to the cluster.

For example:

Before:

Node 1
Node 2


After:

Node 1
Node 2
Node 3

Adding nodes can increase processing capacity and provide additional availability.

Similarly, administrators may remove nodes when hardware is retired or the infrastructure is redesigned.

Oracle’s 11g RAC administration guide includes procedures for adding and deleting RAC nodes and database instances.

13. Security in RAC

RAC administrators must also protect the database environment.

Important security practices include:

  • Strong database passwords
  • Proper user privileges
  • Secure network communication
  • Regular patching
  • Auditing
  • Limiting administrative access
  • Protecting backup files
  • Monitoring suspicious activity

Security should be applied consistently across all RAC nodes.

14. Common RAC Administration Challenges

RAC provides powerful capabilities, but it also introduces additional complexity.

Common challenges include:

Network Problems

A problem with the private interconnect can affect RAC communication.

Storage Problems

ASM or shared-storage failures can affect multiple instances.

Cache Fusion Contention

Excessive block transfers can create performance problems.

Configuration Errors

Differences between nodes can cause unexpected behavior.

Poor Workload Distribution

One node may become overloaded while another remains underutilized.

Complex Troubleshooting

A problem may originate from:

Application
     ↓
Network
     ↓
Clusterware
     ↓
RAC Instance
     ↓
ASM
     ↓
Storage

Therefore, RAC administrators need knowledge of databases, operating systems, networking, storage, and clustering.

About the author