Sporadic Bulk API error with PKChunking - Failed to create batch, since the Job is not Open

Question

Sometime when retrieving the batch results for a Bulk API query job submitted with PK chunking enabled, I get the following error message:

PKChunking failed. InvalidJobState : Failed to create batch, since the Job is not Open. Current job state is 'Closed'

I have used Bulk queries without PK chunking many times and have not seen that error. Even with PK chunking enabled, most of the time it works correctly i.e. all batches are submitted and completed.

This is what I'm doing to submit the job:

• create a job with PK Chunking header using the instructions in the Bulk API documentation
• submit a batch containing the query
• after getting server response from the batch submit, close the job immediately

Am I missing any step e.g. do I need to do anything like check batch status before closing the job? Is it even necessary to close Bulk jobs if they are query jobs?

Answer 1

Very simple solution to this - don't close the job until the query has completed. When I was running jobs without PK Chunking enabled I was always closing the job immediately after submitting jobs, and that works fine, but when PK Chunking is enabled it seems that more time is needed for the queries to submit.

So now I'm

• creating a job with PK Chunking header
• submit a batch containing the query
• polling for job to complete
• downloading results
• closing the job