Understanding Quorum and Why It’s Important
With a cluster, multiple nodes share a common cluster database in which services are
defined that can run on any node in the cluster. The goal of the cluster is to provide
high availability so that if something bad happens on a node, the services move to
another node. It’s important to note that in some scenarios, a network problem may
stop different parts of a cluster from being able to communicate, rather than actual
node problems. In the case of a communication problem between different parts
(partitions) of the cluster, only one part of the cluster should run services, to avoid the
same service starting on different parts of the cluster, which could then cause
corruption.
The detection of “something bad” happening within a cluster is facilitated by cluster
heartbeat communications. The nodes in the cluster communicate constantly via a
heartbeat to ensure that they are available. In the event of a change of cluster status,
such as a node becoming unavailable or network problems stopping the cluster nodes
from communicating, the cluster goes into arbitration: The remaining nodes basically
fight it out to decide who should be hosting which services and applications, to avoid
split-brain. In a split-brain situation, multiple nodes in a cluster try to bring online the
same service or application, which causes the nodes to try to bring online the same
resources.