public class

DefaultIdealStateCalculator

extends Object
java.lang.Object
   ↳ org.apache.helix.tools.DefaultIdealStateCalculator

Class Overview

DefaultIdealStateCalculator tries to optimally allocate master/slave partitions among espresso storage nodes. Given a batch of storage nodes, the partition and replication factor, the algorithm first given a initial state When new batches of storage nodes are added, the algorithm will calculate the new ideal state such that the total partition movements are minimized.

Summary

Public Constructors
DefaultIdealStateCalculator()
Public Methods
static ZNRecord calculateIdealState(List<String> instanceNames, int partitions, int replicas, String resourceName, String masterStateValue, String slaveStateValue)
Calculate the initial ideal state given a batch of storage instances, the replication factor and number of partitions 1.
static ZNRecord calculateIdealStateBatch(List<List<String>> instanceBatches, int partitions, int replicas, String resourceName, String masterStateValue, String slaveStateValue)
static Map<StringObject> calculateInitialIdealState(List<String> instanceNames, int partitions, int replicas)
Calculate the initial ideal state given a batch of storage instances, the replication factor and number of partitions 1.
static Map<StringObject> calculateNextIdealState(List<String> newInstances, Map<StringObject> previousIdealState)
Given the current idealState, and the list of new Instances needed to be added, calculate the new Ideal state.
ZNRecord calculateNextIdealState(List<String> newInstances, Map<StringObject> previousIdealState, String resourceName, String masterStateValue, String slaveStateValue)
static ZNRecord convertToZNRecord(Map<StringObject> result, String resourceName, String masterStateValue, String slaveStateValue)
Convert the internal result (stored as a Map) into ZNRecord.
static void main(String[] args)
[Expand]
Inherited Methods
From class java.lang.Object

Public Constructors

public DefaultIdealStateCalculator ()

Public Methods

public static ZNRecord calculateIdealState (List<String> instanceNames, int partitions, int replicas, String resourceName, String masterStateValue, String slaveStateValue)

Calculate the initial ideal state given a batch of storage instances, the replication factor and number of partitions 1. Calculate the master assignment by random shuffling 2. for each storage instance, calculate the 1st slave assignment map, by another random shuffling 3. for each storage instance, calculate the i-th slave assignment map 4. Combine the i-th slave assignment maps together

Parameters
instanceNames list of storage node instances
partitions number of partitions
replicas The number of replicas (slave partitions) per master partition
masterStateValue master state value: e.g. "MASTER" or "LEADER"
slaveStateValue slave state value: e.g. "SLAVE" or "STANDBY"
Returns
  • a ZNRecord that contain the idealstate info

public static ZNRecord calculateIdealStateBatch (List<List<String>> instanceBatches, int partitions, int replicas, String resourceName, String masterStateValue, String slaveStateValue)

public static Map<StringObject> calculateInitialIdealState (List<String> instanceNames, int partitions, int replicas)

Calculate the initial ideal state given a batch of storage instances, the replication factor and number of partitions 1. Calculate the master assignment by random shuffling 2. for each storage instance, calculate the 1st slave assignment map, by another random shuffling 3. for each storage instance, calculate the i-th slave assignment map 4. Combine the i-th slave assignment maps together

Parameters
instanceNames list of storage node instances
partitions number of partitions
replicas The number of replicas (slave partitions) per master partition
Returns
  • a map that contain the idealstate info

public static Map<StringObject> calculateNextIdealState (List<String> newInstances, Map<StringObject> previousIdealState)

Given the current idealState, and the list of new Instances needed to be added, calculate the new Ideal state. 1. Calculate how many master partitions should be moved to the new cluster of instances 2. assign the number of master partitions px to be moved to each previous node 3. for each previous node, 3.1 randomly choose px nodes, move them to temp list 3.2 for each px nodes, remove them from the slave assignment map; record the map position of the partition; 3.3 calculate # of new nodes that should be put in the slave assignment map 3.4 even-fy the slave assignment map; 3.5 randomly place # of new nodes that should be placed in 4. from all the temp master node list get from 3.1, 4.1 randomly assign them to nodes in the new cluster 5. for each node in the new cluster, 5.1 assemble the slave assignment map 5.2 even-fy the slave assignment map

Parameters
newInstances list of new added storage node instances
previousIdealState The previous ideal state
Returns
  • a map that contain the updated idealstate info

public ZNRecord calculateNextIdealState (List<String> newInstances, Map<StringObject> previousIdealState, String resourceName, String masterStateValue, String slaveStateValue)

public static ZNRecord convertToZNRecord (Map<StringObject> result, String resourceName, String masterStateValue, String slaveStateValue)

Convert the internal result (stored as a Map) into ZNRecord.

public static void main (String[] args)