Helix Tutorial: YAML Cluster Setup
As an alternative to using Helix Admin to set up the cluster, its resources, constraints, and the state model, Helix supports bootstrapping a cluster configuration based on a YAML file. Below is an annotated example of such a file for a simple distributed lock manager where a lock can only be LOCKED or RELEASED, and each lock only allows a single participant to hold it in the LOCKED state.
clusterName: lock-manager-custom-rebalancer # unique name for the cluster (required) resources: - name: lock-group # unique resource name (required) rebalancer: # required mode: USER_DEFINED # required - USER_DEFINED means we will provide our own rebalancer class: org.apache.helix.userdefinedrebalancer.LockManagerRebalancer # required for USER_DEFINED partitions: count: 12 # number of partitions for the resource (default is 1) replicas: 1 # number of replicas per partition (default is 1) stateModel: name: lock-unlock # model name (required) states: [LOCKED, RELEASED, DROPPED] # the list of possible states (required if model not built-in) transitions: # the list of possible transitions (required if model not built-in) - name: Unlock from: LOCKED to: RELEASED - name: Lock from: RELEASED to: LOCKED - name: DropLock from: LOCKED to: DROPPED - name: DropUnlock from: RELEASED to: DROPPED - name: Undrop from: DROPPED to: RELEASED initialState: RELEASED # (required if model not built-in) constraints: state: counts: # maximum number of replicas of a partition that can be in each state (required if model not built-in) - name: LOCKED count: "1" - name: RELEASED count: "-1" - name: DROPPED count: "-1" priorityList: [LOCKED, RELEASED, DROPPED] # states in order of priority (all priorities equal if not specified) transition: # transitions priority to enforce order that transitions occur priorityList: [Unlock, Lock, Undrop, DropUnlock, DropLock] # all priorities equal if not specified participants: # list of nodes that can serve replicas (optional if dynamic joining is active, required otherwise) - name: localhost_12001 host: localhost port: 12001 - name: localhost_12002 host: localhost port: 12002 - name: localhost_12003 host: localhost port: 12003
Using a file like the one above, the cluster can be set up either with the command line:
helix/helix-core/target/helix-core/pkg/bin/YAMLClusterSetup.sh localhost:2199 lock-manager-config.yaml
or with code:
YAMLClusterSetup setup = new YAMLClusterSetup(zkAddress); InputStream input = Thread.currentThread().getContextClassLoader() .getResourceAsStream("lock-manager-config.yaml"); YAMLClusterSetup.YAMLClusterConfig config = setup.setupCluster(input);
A rebalancer class is only required for the USER_DEFINED mode. It is ignored otherwise.
Built-in state models, like OnlineOffline, LeaderStandby, and MasterSlave, or state models that have already been added only require a name for stateModel. If partition and/or replica counts are not provided, a value of 1 is assumed.