We wanted to create an outbound message queue system that would process requests individually and in order to any external web service. The frequency that these requests should be processed is variable but as an example, it might be that we would like the process to run once every 30 seconds.
It is not possible or practical to schedule the same job 120 times for every 30 seconds within the hour.
The approach I finally settled on was to chain schedule apex (wait until scheduled time, call future method and delete (abort) itself) -> future call (process request if there is one and reschedule class to rerun in 30 seconds) -> schedule apex (call future method in 30 seconds, abort itself) -> future call (process request and reschedule) and so on indefinitely. The end result has been running on production for 2 weeks now but has just failed to reschedule itself and I do not understand why.
I believe what is happening is that I am breaching some kind of governor limit but I don't understand how. I have been having similar issues more frequently on my sandbox and have received a limit exception stating that the limit for future requests has been exceeded.
From the point that the service failed, I queried for all asynchronous jobs (not just future) from the previous 24 hours and it returned just 2200. I believe this figure should be 4400 because I think the schedule jobs are not showing up (The scheduled jobs abort themselves after they call the future method).
Even so, Salesforce states that we are allowed 250,000 asynchronous apex executions every 24 hours so I don't understand what is going on. Does anyone have any experience trying to chain scheduled jobs in this way? Is there a better way to achieve this? Is there a limit that I am not aware of or am I measuring asynchronous executions in the wrong way?
Any help or advice would be greatly appreciated.