I need some architectural advice for our managed application.
Our managed app runs a job every 5 minutes and as part of this job we need to run multiple queries. The number of queries could be in the range 100-200. Now, salesforce has a limitation on the number of queries that can be issued per transaction. So we figured that we would need to break these queries into multiple batch jobs and have around 50 or so queries running per batch.
However, we also have requirements in terms of the execution order of these queries. We need to run these queries in a specific order and only process the left over queries in follow up batch jobs. So, if we have around 170 queries, lets call them Q1->Q170
we would process Batch 1 -> Q1-Q50 queries
Batch 2 -> Q51 - Q100 queries
Batch 3 -> Q101- Q150 queries
Batch 4 -> Q151 - Q170 queries
and then start over, so Batch 5 -> Q1-Q50 queries in the same order
I presume we would have to maintain some state per batch job and execute based on that. Any advice on how we can achieve this? Is there any design pattern to achieve this where we can run async jobs in a synchronous manner?