How to upload ContentVersion records using BulkAPI?

Question

So I've been trying to follow the sample java app provided in the documentation to write some Java code to upload ContentVersion records (files) into Salesforce using the Bulk API. It should be so maddeningly simple: So I've been trying, but regardless of the approach I try, I end up getting this:

Failed with error: INVALID_FIELD:Failed to deserialize field at col 2. Due to, 
InvalidBatch : Invalid file reference: /Users/myuser/repos/BulkFileLoader/temp/Batch_1/samplefile.txt:VersionData --

I am creating a temp folder, moving my files into the folder and creating a request.txt file in the same folder, then using

batchInfo = bulkConnection.createBatchFromDir(myJob, null, tempFolder);

I have also tried to provide the whole thing as a stream using this method:

private BatchInfo createBatchFromFiles(JobInfo job, Path fileDir, Path newCsv)
            throws AsyncApiException, IOException
    {

        Map<String, InputStream> attachments = new HashMap<>();
        for (File f : fileDir.toFile().listFiles())
        {
            Path filePath = Paths.get(f.toURI());
            attachments.put(f.getAbsolutePath(), Files.newInputStream(filePath));
        }

        return bulkConnection.createBatchWithInputStreamAttachments(job, Files.newInputStream(newCsv), attachments);
    }

Both approaches give me exactly the same error.

Here's the request.txt file generated:

Title,Description,VersionData,PathOnClient
samplefile.txt,samplefile.txt,/Users/myuser/repos/BulkFileLoader/temp/Batch_1/samplefile.txt,/Users/myuser/repos/BulkFileLoader/temp/Batch_1/samplefile.txt

When I convert my request.txt file to request.csv and try uploading with the Data Loader, there are no issues - so apparently the Data Loader is doing some magic on the files, but looking through the Data Loader source code, I have not been able to figure out what.

Any help to get this simple example working would be greatly appreciated.

Answer 1

So apparently, writing out this question helped... a lot. After a bit more experimentation, the only thing that needed changing was the output in the request.txt file: the VersionData field needs just the filename prefixed with a # like this:

Title,Description,VersionData,PathOnClient
samplefile.txt,samplefile.txt,#samplefile.txt,/Users/myuser/repos/BulkFileLoader/temp/Batch_1/samplefile.txt

Either of the two approaches (using createBatchWithInputStreamAttachments or createBatchFromDir) works equally well.