Distributed task synchronization: leveraging ShedLock in Spring

In today’s distributed computing environments, coordinating tasks across multiple nodes and ensuring they run without conflicts or duplication is a critical challenge. Whether you’re managing periodic tasks, batch processes, or critical system tasks, maintaining synchronization and consistency is critical to smooth operations.

problem

Let’s say you need to run some tasks on a schedule, whether it’s a database cleanup task or some data generation tasks. If you approach the problem yourself, you can solve this problem using: @Schedules This is an annotation included in Spring Framework. This annotation allows you to run code at fixed intervals or on a cron schedule. But what happens if you have more than one service instance? In this case, the task runs on all instances of the service.

warehouse lock

ShedLock ensures that scheduled tasks run at most once concurrently. The library implements locking via external storage. When a task runs on one instance, a lock is set and all other instances skip running the task without waiting. This implements “run at most once”. External storage can be relational databases (PostgreSQL, MySQL, Oracle, etc.) operating via JDBC, NoSQL (Mongo, Redis, DynamoDB) and many others (a full list can be found on the project page).

Let’s consider an example of working with PostgreSQL. First, let’s start a database using Docker.

docker run -d -p 5432:5432 --name db \
    -e POSTGRES_USER=admin \
    -e POSTGRES_PASSWORD=password \
    -e POSTGRES_DB=demo \
    postgres:alpine

Now we need to create a lock table. You should find the SQL script for PostgreSQL on the project page.

CREATE TABLE shedlock(
    name VARCHAR(64) NOT NULL,
    lock_until TIMESTAMP NOT NULL,
    locked_at TIMESTAMP NOT NULL, 
    locked_by VARCHAR(255) NOT NULL, 
    PRIMARY KEY (name));

here:

• name – A unique identifier for the lock, typically representing the locked task or resource.
• lock_until – Timestamp indicating until the lock will be held.
• locked_at – A timestamp indicating when the lock was acquired.
• locked_by – Identifier of the entity (e.g. application instance) that acquired the lock

Next, create a Spring Boot project and add the required dependencies. build.gradle:

implementation 'net.javacrumbs.shedlock:shedlock-spring:5.10.2'
implementation 'net.javacrumbs.shedlock:shedlock-provider-jdbc-template:5.10.2'

Now let’s explain the configuration.

@Configuration
@EnableScheduling
@EnableSchedulerLock(defaultLockAtMostFor = "10m")
public class ShedLockConfig {
    @Bean
    public LockProvider lockProvider(DataSource dataSource) {
        return new JdbcTemplateLockProvider(
                JdbcTemplateLockProvider.Configuration.builder()
                        .withJdbcTemplate(new JdbcTemplate(dataSource))
                        .usingDbTime()
                        .build()
        );
    }
}

Let’s create the following: ExampleTask It starts once a minute and performs time-consuming tasks. For this we @Scheduled annotation:

@Service
public class ExampleTask {

    @Scheduled(cron = "0 * * ? * *")
    @SchedulerLock(name = "exampleTask", lockAtMostFor = "50s", lockAtLeastFor = "20s")
    public void scheduledTask() throws InterruptedException {
        System.out.println("task scheduled!");
        Thread.sleep(15000);
        System.out.println("task executed!");
    }
}

Here we Thread.sleep Simulate job execution time for 15 seconds. When the application starts and the job starts executing, records are inserted into the database.

docker exec -ti <CONTAINER ID> bash

psql -U admin demo
psql (12.16)
Type "help" for help.

demo=# SELECT * FROM shedlock;
    name     |         lock_until         |         locked_at          |   locked_by
-------------+----------------------------+----------------------------+---------------
 exampleTask | 2024-02-18 08:08:50.055274 | 2024-02-18 08:08:00.055274 | MacBook.local

At the same time, if another application tries to execute the task, it will not be able to obtain the lock and will skip executing the task.

2024-02-18 08:08:50.057 DEBUG 45988 --- [   scheduling-1] n.j.s.core.DefaultLockingTaskExecutor    
: Not executing 'exampleTask'. It's locked.

The moment the first application acquires the lock, a record is created in the database with the following lock time: lockAtMostFor In lock settings. This time is necessary to ensure that if your application crashes or dies for any reason (e.g. removing a Pod from one node to another in Kubernetes), the lock will not be set forever. After the operation executes successfully, the application updates the database entry and reduces the lock time to the current time. However, if the job execution time is very short, this value cannot be less than: lockAtLeastFor In configuration. This value is required to minimize clock desynchronization between instances. Ensures that scheduled tasks run only once at a time.

conclusion

ShedLock is a useful tool for coordinating tasks in complex Spring applications. Ensures that tasks run seamlessly and only once across multiple instances. It’s easy to set up and provides reliable task processing for Spring applications, making it a useful tool for anyone working with distributed systems.

The project code is available on GitHub.