Helix Tutorial: Logical Accessors
Helix constructs follow a logical hierarchy. A cluster contains participants, and serve logical resources. Each resource can be divided into partitions, which themselves can be replicated. Helix now supports configuring and modifying clusters programmatically in a hierarchical way using logical accessors.
Click here for the Javadocs of the accessors.
Configure a Participant
A participant is a combination of a host, port, and a UserConfig. A UserConfig is an arbitrary set of properties a Helix user can attach to any participant.
ParticipantId participantId = ParticipantId.from("localhost_12345"); ParticipantConfig participantConfig = new ParticipantConfig.Builder(participantId) .hostName("localhost").port(12345).build();
Configure a Resource
A Resource is essentially a combination of a RebalancerContext and a UserConfig. A RebalancerContext consists of all the key properties required to rebalance a resource, including how it is partitioned and replicated, and what state model it follows. Most Helix resources will make use of a PartitionedRebalancerContext, which is a RebalancerContext for resources that are partitioned.
Recall that there are four rebalancing modes that Helix provides, and so Helix also provides the following subclasses for PartitionedRebalancerContext:
- FullAutoRebalancerContext for FULL_AUTO mode.
- SemiAutoRebalancerContext for SEMI_AUTO mode. This class allows a user to specify “preference lists” to indicate where each partition should ideally be served
- CustomRebalancerContext for CUSTOMIZED mode. This class allows a user tp specify “preference maps” to indicate the location and state for each partition replica.
Helix also supports arbitrary subclasses of PartitionedRebalancerContext and even arbitrary implementations of RebalancerContext for applications that need a user-defined approach for rebalancing. For more, see User-Defined Rebalancing
Here is an example of a configured resource with a rebalancer context for FULL_AUTO mode and two partitions:
ResourceId resourceId = ResourceId.from("sampleResource"); StateModelDefinition stateModelDef = getStateModelDef(); Partition partition1 = new Partition(PartitionId.from(resourceId, "1")); Partition partition2 = new Partition(PartitionId.from(resourceId, "2")); FullAutoRebalancerContext rebalanceContext = new FullAutoRebalancerContext.Builder(resourceId).replicaCount(1).addPartition(partition1) .addPartition(partition2).stateModelDefId(stateModelDef.getStateModelDefId()).build(); ResourceConfig resourceConfig = new ResourceConfig.Builder(resourceId).rebalancerContext(rebalanceContext).build();
Add the Cluster
Now we can take the participant and resource configured above, add them to a cluster configuration, and then persist the entire cluster at once using a ClusterAccessor:
// configure the cluster ClusterId clusterId = ClusterId.from("sampleCluster"); ClusterConfig clusterConfig = new ClusterConfig.Builder(clusterId).addParticipant(participantConfig) .addResource(resourceConfig).addStateModelDefinition(stateModelDef).build(); // create the cluster using a ClusterAccessor HelixConnection connection = new ZkHelixConnection(zkAddr); connection.connect(); ClusterAccessor clusterAccessor = connection.createClusterAccessor(clusterId); clusterAccessor.createCluster(clusterConfig);
Create, Read, Update, and Delete
Note that you don’t have to specify the entire cluster beforehand! Helix provides a ClusterAccessor, ParticipantAccessor, and ResourceAccessor to allow changing as much or as little of the cluster as needed on the fly. You can add a resource or participant to a cluster, reconfigure a resource, participant, or cluster, remove components from the cluster, and more. See the Javadocs to see all that the accessor classes can do.
Updating a cluster, participant, or resource should involve selecting the element to change, and then letting Helix change only that component. To do this, Helix has included Delta classes for ClusterConfig, ParticipantConfig, and ResourceConfig.
Example: Updating a Participant
Tags are used for Helix depolyments where only certain participants can be allowed to serve certain resources. To do this, Helix only assigns resource replicas to participants who have a tag that the resource specifies. In this example, we will use ParticipantConfig.Delta to remove a participant tag and add another as part of a reconfiguration.
// specify the change to the participant ParticipantConfig.Delta delta = new ParticipantConfig.Delta(participantId).addTag("newTag").removeTag("oldTag"); // update the participant configuration ParticipantAccessor participantAccessor = connection.createParticipantAccessor(clusterId); participantAccessor.updateParticipant(participantId, delta);
Example: Dropping a Resource
Removing a resource from the cluster is quite simple:
Example: Reading the Cluster
Reading a full snapshot of the cluster is also a one-liner:
Cluster cluster = clusterAccessor.readCluster();
Helix also includes versions of ClusterAccessor, ParticipantAccessor, and ResourceAccessor that can complete operations atomically relative to one another. The specific semantics of the atomic operations are included in the Javadocs. These atomic classes should be used sparingly and only in cases where contention can adversely affect the correctness of a Helix-based cluster. For most deployments, this is not the case, and using these classes will cause a degradation in performance. However, the interface for all atomic accessors mirrors that of the non-atomic accessors.