Package io.github.jchejarla.springbatch.clustering.api

Class ClusterAwarePartitioner

java.lang.Object
io.github.jchejarla.springbatch.clustering.api.ClusterAwarePartitioner
All Implemented Interfaces:
org.springframework.batch.core.partition.Partitioner
public abstract class ClusterAwarePartitioner extends Object implements org.springframework.batch.core.partition.Partitioner
An abstract Spring Batch Partitioner that enables distributed partitioning in a clustered environment. It dynamically assigns partitions to available nodes in the cluster, using a shared database for coordination. Subclasses must implement methods to define how data is split into chunks and how partitioning strategies are built.

This class provides the framework for:

  • Fetching active cluster nodes and their load.
  • Dividing the workload into partitions.
  • Assigning partitions to nodes based on a chosen strategy.
  • Ensuring partition transferability in case of node failure.

Key Features:

  • Cluster-aware: Partitions are distributed across a cluster of Spring Batch instances.
  • Dynamic: Handles nodes joining and leaving the cluster.
  • Database-driven: Uses a shared database for cluster coordination and state management.
  • Extensible: Abstract class requiring subclasses to define specific partitioning logic.

Usage:

  1. Create a subclass of ClusterAwarePartitioner.
  2. Implement the abstract methods:
  3. Configure the Spring Batch job to use your custom partitioner.
Since:
1.0
Author:
Janardhan Chejarla

  • Field Details

  • Constructor Details

    • ClusterAwarePartitioner

      public ClusterAwarePartitioner()

  • Method Details

    • partition

      public Map<String,org.springframework.batch.infrastructure.item.ExecutionContext> partition(int gridSize)
      Partitions the input data into chunks and assigns them to available nodes in the cluster.

      This implementation overrides the standard Spring Batch Partitioner.partition(int) method to delegate the partitioning logic to a PartitionAssignmentStrategy obtained from the PartitionStrategyFactory. The number of partitions is determined by the number of active nodes in the cluster, not by the gridSize parameter.

      Specified by:
      partition in interface org.springframework.batch.core.partition.Partitioner
      Parameters:
      gridSize - Ignored in this implementation. The number of partitions is determined by the number of active nodes.
      Returns:
      A map of partition names to ExecutionContext instances, where each ExecutionContext contains information about the assigned node and whether partitions are transferable on node failure.
      Throws:
      org.springframework.batch.core.configuration.BatchConfigurationException - If no active nodes are found in the database, indicating a configuration issue.

    • createDistributedPartitions

      public abstract List<org.springframework.batch.infrastructure.item.ExecutionContext> createDistributedPartitions(int availableNodeCount)
      Abstract method to be implemented by subclasses to define how the input data should be split into chunks for distribution across the cluster nodes.
      Parameters:
      availableNodeCount - The number of nodes available in the cluster.
      Returns:
      A list of ExecutionContext instances, each representing a chunk of data.

    • arePartitionsTransferableWhenNodeFailed

      public abstract PartitionTransferableProp arePartitionsTransferableWhenNodeFailed()
      Abstract method to be implemented by subclasses to specify whether partitions should be transferred to other nodes when a node fails.
      Returns:
      An enum value indicating whether partitions are transferable.

    • buildPartitionStrategy

      public abstract PartitionStrategy buildPartitionStrategy()
      Abstract method to be implemented by subclasses to define the partitioning strategy to be used.
      Returns:
      A PartitionStrategy object configuring the partitioning strategy.