How to query records processed between last batch job if I'm running batch class five times a day

Question

I want to process only those records in my batch job which are updated since the last time my batch job ran.

Basically, I'm running my batch job five times a day and want to process only those records which are updated since my last batch job.

For example, if my first batch job runs at 9 AM in the morning and the next one is scheduled at 1 PM, then in the 1 PM schedule I want to process only those records which are updated after 9 AM.

The logic I'm thinking of is creating a date-time field and update that field in each batch. In the next scheduled time it will filter out the records based on that.

But the problem I'm facing is where to store the last job time so that I can get it in my query and use it for the next run.

Answer 1

I can think of a couple of approaches

1 - Store the datetime in Custom Metadata. You can do this easily by exploiting this andyinthecloud GitHub repo . I have done this in one of my applications.

You could also use a custom object.

2 - If possible, make the batch job idempotent so it can run with a filter like TODAY. If it encounters records it has already processed (determined by a timestamp in the target of the batch's operation), it does nothing.