Enabling Leader Election

Enabling Leader Election for High Availability

The operator supports a high-availability (HA) mode, in which there can be more than one replicas of the operator, with only one of the replicas (the leader replica) actively operating. If the leader replica fails, the leader election process is engaged again to determine a new leader from the replicas available. The HA mode can be enabled through an optional leader election process. Leader election is disabled by default but can be enabled via a command-line flag. The following table summarizes the command-line flags relevant to leader election:

FlagDefault ValueDescription
leader-electionfalseWhether to enable leader election (or the HA mode) or not.
leader-election-lock-namespacespark-operatorKubernetes namespace of the lock resource used for leader election.
leader-election-lock-namespark-operator-lockName of the lock resource used for leader election.
leader-election-lease-duration15 secondsLeader election lease duration.
leader-election-renew-deadline14 secondsLeader election renew deadline.
leader-election-retry-period4 secondsLeader election retry period.


Was this page helpful?