2

I am able to upload the Blob data to the local web server. However, the I have run into the error

"Exceeded max size limit of 12000000 with request size xxxxx"

This means that the file size is huge and that I will not be able to perform a full file upload. I have to get a way to configure my server does chunk uploading feature. That part is done and I'd like to convert it to a chunk based upload.

This is the full-upload code:

public with sharing class QueueableUploader implements Queueable, Database.AllowsCallouts{

    // Properties and Constructor setting the required properties for the document. Get the body and store it here.

    public void execute(QueueableContext context)
    {
        HttpRequest req = new HttpRequest();
        HttpResponse res = new HttpResponse();
        Http http = new Http();

        req.setMethod('POST');
        // Set properties here

        req.setHeader('FileEOF', '0');
        req.setEndpoint(url);

        req.setBodyAsBlob(body);
        req.setTimeout(120000);

        res = http.send(req);
        Integer statusCode = res.getStatusCode();
        String statusMessage = res.getStatus();

        if(statusCode == 200)
        {
            DOM.Document doc = new DOM.Document();      
            doc.load(res.getBody());    
            DOM.XMLNode rootNode = doc.getRootElement();
            String errorCode = rootNode.getAttribute('errorCode', null);
            if(errorCode == '200')
            {

                String fileId = rootNode.getAttribute('fileId', null);
                System.debug('Upload success for ' + fileName + '. FileId: ' + fileId);
                //fileIdMap.put(fileName, rootNode.getAttribute('errorString', null));
            }
            else
            {
                String errorMessage = rootNode.getAttribute('errorString', null);
                System.debug('Error under upload for ' + fileName +'. Error code: ' + errorCode + '. Error Message: ' + errorMessage);
            }
        }
        else
            System.debug('Error under upload for ' + fileName + '. Status code: ' + statusCode + '. Reason: ' + statusMessage);
    }

}

This is the intended modification for chunk upload.

public with sharing class QueueableUploader implements Queueable, Database.AllowsCallouts{

    // Properties and Constructor setting the required properties for the document. Get the body and store it here.

    **// HOW TO? Constructor should chunk the blob into smaller blob units to upload.**


    public void execute(QueueableContext context)
    {
        HttpRequest req = new HttpRequest();
        HttpResponse res = new HttpResponse();
        Http http = new Http();

       // Set properties here

        // FileEOF = 1 when last chunk during chunked upload
        req.setHeader('FileEOF', '0');
        req.setEndpoint(url);

        req.setBodyAsBlob(body);
        req.setTimeout(120000);

        res = http.send(req);
        Integer statusCode = res.getStatusCode();
        String statusMessage = res.getStatus();

        if(statusCode == 200)
        {
            DOM.Document doc = new DOM.Document();      
            doc.load(res.getBody());    
            DOM.XMLNode rootNode = doc.getRootElement();
            String errorCode = rootNode.getAttribute('errorCode', null);
            if(errorCode == '200')
            {
                // Call the method chunkUpload(params)
            }
            else
            {
                String errorMessage = rootNode.getAttribute('errorString', null);
                System.debug('Error under upload for ' + fileName +'. Error code: ' + errorCode + '. Error Message: ' + errorMessage);
            }
        }
        else
            System.debug('Error under upload for ' + fileName + '. Status code: ' + statusCode + '. Reason: ' + statusMessage);
    }


    public void chunkUpload(Params params)
    {

        // Send HttpRequest using params
        // Wait for response and get errorCode
        if(errorCode == '200')
        {
            // Okay, continue to the next chunk by calling chunkUpload modifying the params
        }
        else
        {
            throw exception;
        }

    }

}

From the above modification, I have a few questions:

  1. I'm missing on how to divide a Blob into smaller Blob units. How can I do that? The docs doesn't have it but in java, I can divide into smaller byte array units.

  2. From a design perspective, this is a queueable instance and I am not sure whether this can be chained (may require rewording; continue this queueable job instance until all chunk upload is complete). Is this approach correct or will I run into heap limits? I am running 10 parallel jobs at a time.

  • APEX doesn't provide tools to break a Blob apart into miniBlobs. I'm assuming that the Blob contains binary data. There are Apex heap size limits as well that will get in your way. What is body ? and where does it come from? – cropredy Jul 10 '15 at 16:37
  • body is the Blob object to be sent. – dmachop Jul 10 '15 at 17:22
  • right, I get that - but body must originate from somewhere - is it in Documents or ContentVersion? – cropredy Jul 10 '15 at 17:48
  • body is queried from the Document object and then stored as a property in Queueable uploader. – dmachop Jul 10 '15 at 20:16
  • did you look at salesforce.stackexchange.com/a/7958/2602 ? This is a hard problem because even if you convert the blob to base64 you'll run out of heap space and the receiving system has to reconstruct the file being aware of base64 encoding rules. can the remote system call you? salesforce.stackexchange.com/questions/1620/… – cropredy Jul 11 '15 at 0:10
2

A few issues here:

  1. No APEX support to split a blob
  2. If you converted the Blob to base64 with Apex, given the size of your Blob you would likely run out of heap. Even if that didn't happen, you'd have to send each base64 chunk to remote system that would have to know how to reconstruct

One alternative to consider is to send the ID of the Document (or ContentVersion) to the remote system and have the remote system use the SOAP API to fetch the document/contentVersion body.

See also

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.