Skip to content

Examples

The examples/ module is a runnable Spring Boot application that stands up a real multi-node cluster and exposes jobs over REST. Use it to see coordination, distribution, and failover end-to-end.

Example jobs

Job Endpoint What it shows
Range-sum GET /api/v1/clusteredjob/addition/taskSize/{n}/from/{from}/to/{to} Splits a numeric range into n partitions, sums each on a different worker, aggregates the total.
ETL GET /api/v1/etljob/rows/{rows} CSV → XML with chunk-oriented reader/processor/writer distributed across workers.
Single-node baseline GET /api/v1/singlenodejob/addition/... The same work without clustering, to compare timings.

Run it in 60 seconds (zero-setup H2)

The bundled h2 profile uses a shared file-mode H2 database, so multiple JVMs on one machine form a cluster with no external infrastructure:

# 1. Build
mvn clean package -pl examples -am

# 2. Start two nodes (separate terminals), sharing the H2 file
java -jar examples/target/examples-3.0.0-SNAPSHOT.jar --spring.profiles.active=h2 --server.port=8081
java -jar examples/target/examples-3.0.0-SNAPSHOT.jar --spring.profiles.active=h2 --server.port=8082

# 3. Trigger a clustered job on either node
curl http://localhost:8081/api/v1/clusteredjob/addition/taskSize/4/from/1/to/100

Each node prints a >>> lifecycle trace so you can watch partitions being claimed and completed across both terminals, and the REST response returns the aggregated result.

The worker tasklet

The work each partition runs is an ordinary Spring Batch Tasklet — it just reads its slice from the ExecutionContext. Nothing in it is cluster-aware; distribution and failover happen underneath. This is the actual example source:

public class LargeSumExecutionTask implements Tasklet {

    private final String nodeId;

    public LargeSumExecutionTask(String nodeId) {
        this.nodeId = nodeId;
    }

    @Override
    public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) {
        StepExecution stepExecution = chunkContext.getStepContext().getStepExecution();
        ExecutionContext executionContext = stepExecution.getExecutionContext();
        long start = executionContext.getLong("start");
        long end = executionContext.getLong("end");
        String stepName = stepExecution.getStepName();
        System.out.println(">>> [" + nodeId + "] (worker) picked up partition "
                + stepName + " (range " + start + ".." + end + ") on "
                + (Thread.currentThread().isVirtual() ? "virtual" : "platform") + " thread "
                + Thread.currentThread());
        // Optional demo knob: -Ddemo.partition.sleepMs=NNN makes each partition run for a fixed
        // duration so failover (kill a node mid-partition -> reassignment) can be demonstrated
        // without CPU-burning huge ranges. Zero (the default) keeps the pure compute behavior.
        long sleepMs = Long.getLong("demo.partition.sleepMs", 0L);
        if (sleepMs > 0) {
            try {
                Thread.sleep(sleepMs);
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
                throw new RuntimeException("partition " + stepName + " interrupted", e);
            }
        }
        long result = 0;
        for(long i=start; i<=end; i++) {
            result += i;
        }
        executionContext.putLong("result", result);
        System.out.println(">>> [" + nodeId + "] (worker) completed partition "
                + stepName + " result=" + result);
        return RepeatStatus.FINISHED;
    }

}

The demo.partition.sleepMs knob above is purely for demos — it stretches each partition so you can kill a node mid-flight and watch failover (see below).

A chunk-oriented ETL worker

The range-sum job uses a Tasklet. A more typical ETL uses a chunk-oriented worker step (reader → processor → writer). The only cluster-specific part is that each partition reads its own slice of the input: the reader is @StepScope and pulls its window from the stepExecutionContext keys the partitioner put there (startRow/endRow). This is the real reader from the CSV→XML example:

    @Bean("customerReader")
    @StepScope   // one reader per partition, so it can read that partition's row window
    public FlatFileItemReader<Customer> customerReader(
            @Value("#{stepExecutionContext['startRow']}") long startIndex,
            @Value("#{stepExecutionContext['endRow']}") long endIndex,
            @Value("#{jobParameters['inputFile']}") String inputFile,
            LineMapper<Customer> lineMapper) {
        FlatFileItemReader<Customer> reader = new FlatFileItemReader<Customer>(lineMapper) {
            private int currentLine = 0;

            @Override
            public Customer read() throws Exception {
                Customer customer;
                while ((customer = super.read()) != null) {
                    currentLine++;
                    if (currentLine < startIndex) {
                        continue;
                    }
                    if (currentLine > endIndex) {
                        return null;
                    }
                    return customer;
                }
                return null;
            }
        };

        reader.setResource(new FileSystemResource(inputFile));
        reader.setLinesToSkip(1); // Skip header
        reader.setLineMapper(lineMapper);

        return reader;
    }

The worker step wires that reader (and a processor/writer) as an ordinary Spring Batch chunk step. Note the naming rule from the Usage guide: the worker step's bean name = StepBuilder name = the name passed to .partitioner(...) — workers look the step up by that name.

    // The chunk-oriented worker step. Its @StepScope reader/processor/writer are injected as beans
    // (the @Autowired fields above); each worker node runs this step for its assigned partitions.
    // The bean name, the StepBuilder name, and the name given to .partitioner(...) must all match.
    @Bean
    public Step etlReaderWriterStep(JobRepository jobRepository, PlatformTransactionManager txnManager) {
        return new StepBuilder("etlReaderWriterStep", jobRepository).<Customer, Customer>chunk(100, txnManager)
                .reader(customerItemReader)
                .processor(customerProcessor)
                .writer(customerXmlWriter)
                .build();
    }

The @StepScope reader/processor/writer are declared as beans and injected into the worker step (here via @Autowired fields), so Spring hands each partition its own instances. The partitioner that seeds startRow/endRow/partitionId per partition, and the writer that emits one file per partition, are in the same advancedjob/ package.

Trying failover

Start three nodes, launch a job with partitions long enough to interrupt (-Ddemo.partition.sleepMs), kill one worker, and watch its transferable partitions get reassigned to the survivors — the job still completes with every partition run exactly once. The step-by-step walkthrough, along with the PostgreSQL / MySQL / Oracle profiles and SQL for inspecting cluster state, is in the full example README.

Demo timings

The example profiles use deliberately fast failover timings so a dead node is removed in ~8s. Keep the library defaults for real deployments — see the note on that page.