Multipart form data uploads only the first file when using HttpFormBuilder ( try with given RestEndpoint and Anonymous Apex)

Question

I am using the same solution as Daniel provided :

When I use the below method to upload file, one by one, it works fine, jpg, pdf, docx anything.

But when I try to upload multiple files using the same method it only uploads the first file.

public static String writeBlobBodyParameter(String strKey, String strFile64, String strFileName) {

    String strInternalFile64 = strFile64;
    String contentDisposition = 'Content-Disposition: form-data; name="' + strKey + '"; filename="' + strFileName + '"';
    String contentDispositionCrLf = contentDisposition + STRING_SINGLE_LINE_BREAK;
    Blob contentDispositionCrLfBlob = Blob.valueOf(contentDispositionCrLf);
    String contentDispositionCrLf64 = EncodingUtil.base64Encode(contentDispositionCrLfBlob);
    String content = safelyPad(contentDisposition, contentDispositionCrLf64, STRING_SINGLE_LINE_BREAK);

    String contentTypeHeader = 'Content-Type: ' + 'application/octet-stream';
    String contentTypeCrLf = contentTypeHeader + STRING_DOUBLE_LINE_BREAK;
    Blob contentTypeCrLfBlob = Blob.valueOf(contentTypeCrLf);
    String contentTypeCrLf64 = EncodingUtil.base64Encode(contentTypeCrLfBlob);
    content += safelyPad(contentTypeHeader, contentTypeCrLf64, STRING_DOUBLE_LINE_BREAK);

    String last4Bytes = strInternalFile64.substring(strInternalFile64.length() - 4, strInternalFile64.length());

    // Avoid padding the file data with spaces, which safelyPad does
    // http://salesforce.stackexchange.com/a/33326/102
    EndingType_ACE ending = EndingType_ACE.None;
    if (last4Bytes.endsWith('==')) {

        // The '==' sequence indicates that the last group contained only one 8 bit byte
        // 8 digit binary representation of CR is 00001101
        // 8 digit binary representation of LF is 00001010
        // Stitch them together and then from the right split them into 6 bit chunks
        // 0000110100001010 becomes 0000 110100 001010
        // Note the first 4 bits 0000 are identical to the padding used to encode the
        // second original 6 bit chunk, this is handy it means we can hard code the response in
        // The decimal values of 110100 001010 are 52 10
        // The base64 mapping values of 52 10 are 0 K
        // See http://en.wikipedia.org/wiki/Base64 for base64 mapping table
        // Therefore, we replace == with 0K
        // Note: if using \n\n instead of \r\n replace == with 'oK'
        last4Bytes = last4Bytes.substring(0, 2) + '0K';
        strInternalFile64 = strInternalFile64.substring(0, strInternalFile64.length() - 4) + last4Bytes;
        // We have appended the \r\n to the Blob, so leave footer as it is.
        ending = EndingType_ACE.CrLf;
    } else if (last4Bytes.endsWith('=')) {

        // '=' indicates that encoded data already contained two out of 3x 8 bit bytes
        // We replace final 8 bit byte with a CR e.g. \r
        // 8 digit binary representation of CR is 00001101
        // Ignore the first 2 bits of 00 001101 they have already been used up as padding
        // for the existing data.
        // The Decimal value of 001101 is 13
        // The base64 value of 13 is N
        // Therefore, we replace = with N
        last4Bytes = last4Bytes.substring(0, 3) + 'N';
        strInternalFile64 = strInternalFile64.substring(0, strInternalFile64.length() - 4) + last4Bytes;

        // We have appended the CR e.g. \r, still need to prepend the line feed to the footer
        ending = EndingType_ACE.Cr;
    }

    content += strInternalFile64;

    content += writeBoundary(ending);
    return content;
}

I have tried basically everything, tried to remove the writeBoundary(ending), here it worked for file types such as pdf, docx etc, but again fails to upload any jpg format files.

or if you try to post multi format files at once, like docx + jpg or pdf + jpg,

but also docx + png works fine, So I believe its an issue with only jpg in combination with other file format or multiple jpgs.

UPDATE

I have removed content += writeBoundary(ending); this line, as only after removing this line, it started uploading multiple files at once, but still combination of files or image files does not work. Here is the last 4 bytes of few combination of files I am trying to attach.

passed
21:31:41.0 (393904784)|USER_DEBUG|[232]|DEBUG|last4Bytes -> for -> No PCP MG Required.pdf is ->Cg==
21:51:18.0 (359053937)|USER_DEBUG|[232]|DEBUG|last4Bytes -> for -> EOB8.pdf is ->Cg==

failed:
21:31:41.0 (393904784)|USER_DEBUG|[232]|DEBUG|last4Bytes -> for -> 32294.docx is ->AA==
21:31:41.0 (495545442)|USER_DEBUG|[232]|DEBUG|last4Bytes -> for -> Koala.png is ->v//Z


passed
21:30:56.0 (356503954)|USER_DEBUG|[232]|DEBUG|last4Bytes -> for -> 32294.docx is ->AA==
21:30:56.0 (473397283)|USER_DEBUG|[232]|DEBUG|last4Bytes -> for -> Jellyfish.png is ->2Q==

One file delivered (Koala.png)
21:30:19.0 (628929391)|USER_DEBUG|[232]|DEBUG|last4Bytes -> for -> Koala.png is ->v//Z
21:30:19.0 (851347947)|USER_DEBUG|[232]|DEBUG|last4Bytes -> for -> Jellyfish.png is ->2Q==

Failed
21:29:15.0 (461532001)|USER_DEBUG|[232]|DEBUG|last4Bytes -> for -> Koala.png is ->v//Z
21:29:15.0 (613812847)|USER_DEBUG|[232]|DEBUG|last4Bytes -> for -> Lighthouse.jpg is ->Ov/Z

one file delivered (Koala.png)
21:56:51.279 (5473116467)|USER_DEBUG|[232]|DEBUG|last4Bytes -> for -> Koala.png is ->v//Z
21:56:51.279 (5642287863)|USER_DEBUG|[232]|DEBUG|last4Bytes -> for -> 32294.docx is ->AA==

Have anyone tried to use this method inside a loop to create form?

For those who can help, I have created a dummy rest endpoint, you can create a multipart form data using above example and hit that. If you receive file names back that mean the server has received the request.

Endpoint to hit: https://localrestapi.herokuapp.com/multiple

Below is an example from a rest client.

or here is an example from anonymous apex:

public static String uploadFile(Blob file_body, String file_name){

    //  Compose the form
    string form64 = '';
    //******This was the boundary I was missing****
    form64 += HttpFormBuilder.writeBoundary();
    form64 += HttpFormBuilder.writeBlobBodyParameter('files', EncodingUtil.base64Encode(file_body), file_name);
    form64 += HttpFormBuilder.writeBoundary(null);
    return form64;

}

public static void fireAPI(String reqEndPoint, Blob formBlob){
    //callout ePOR service
    string contentType = HttpFormBuilder.fetchContentType();
    HttpRequest req = new HttpRequest();
    req.setMethod('POST');
    req.setEndpoint(reqEndPoint);
    req.setHeader('Content-Type', contentType);
    req.setBodyAsBlob(formBlob);
    req.setTimeout(120000);

    Http http = new Http();
    HTTPResponse res = http.send(req);
    System.debug('res -> '+res.getBody());
}

try running this in loop and you will receive only one file back.

List<Attachment> attachments = [SELECT Id, Body,Name from Attachment WHERE Id IN ('***', '***')];

String formData = '';

for (Attachment ac : attachments){
    formData += CaseAttachmentLightningController_ACE.uploadFile(ac.Body, ac.Name);
}

CaseAttachmentLightningController_ACE.fireAPI('https://localrestapi.herokuapp.com/multiple',EncodingUtil.base64Decode(formData));

Answer 1

I have gone through multiple file combinations and this seems to work. I went through multiple rest clients and saw that pdfs and docs file types does not generate two line breaks, only image file generates two line breaks (If anyone knows the exact answer to this, do let me know). I have no idea why this is so. But this seems to work for multiple files too:

public static String writeBlobBodyParameter(String strKey, String strFile64, String strFileName) {

    String mimeType = resolveMimeType(strFileName);

    String strInternalFile64 = strFile64;
    String contentDisposition = 'Content-Disposition: form-data; name="' + strKey + '"; filename="' + strFileName + '"';
    String contentDispositionCrLf = contentDisposition + STRING_SINGLE_LINE_BREAK;
    Blob contentDispositionCrLfBlob = Blob.valueOf(contentDispositionCrLf);
    String contentDispositionCrLf64 = EncodingUtil.base64Encode(contentDispositionCrLfBlob);
    String content = safelyPad(contentDisposition, contentDispositionCrLf64, STRING_SINGLE_LINE_BREAK);

    String contentTypeHeader = 'Content-Type: ' + 'application/octet-stream';
    String contentTypeCrLf = contentTypeHeader + STRING_DOUBLE_LINE_BREAK;
    Blob contentTypeCrLfBlob = Blob.valueOf(contentTypeCrLf);
    String contentTypeCrLf64 = EncodingUtil.base64Encode(contentTypeCrLfBlob);
    content += safelyPad(contentTypeHeader, contentTypeCrLf64, STRING_DOUBLE_LINE_BREAK);

    String last4Bytes = strInternalFile64.substring(strInternalFile64.length() - 4, strInternalFile64.length());

    // Avoid padding the file data with spaces, which safelyPad does
    // http://salesforce.stackexchange.com/a/33326/102
    EndingType_ACE ending = EndingType_ACE.None;
    if (last4Bytes.endsWith('==')) {

        // The '==' sequence indicates that the last group contained only one 8 bit byte
        // 8 digit binary representation of CR is 00001101
        // 8 digit binary representation of LF is 00001010
        // Stitch them together and then from the right split them into 6 bit chunks
        // 0000110100001010 becomes 0000 110100 001010
        // Note the first 4 bits 0000 are identical to the padding used to encode the
        // second original 6 bit chunk, this is handy it means we can hard code the response in
        // The decimal values of 110100 001010 are 52 10
        // The base64 mapping values of 52 10 are 0 K
        // See http://en.wikipedia.org/wiki/Base64 for base64 mapping table
        // Therefore, we replace == with 0K
        // Note: if using \n\n instead of \r\n replace == with 'oK'
        last4Bytes = last4Bytes.substring(0, 2) + '0K';
        strInternalFile64 = strInternalFile64.substring(0, strInternalFile64.length() - 4) + last4Bytes;
        // We have appended the \r\n to the Blob, so leave footer as it is.
        ending = EndingType_ACE.CrLf;
    } else if (last4Bytes.endsWith('=')) {
        last4Bytes = last4Bytes.substring(0, 3) + 'N';
        strInternalFile64 = strInternalFile64.substring(0, strInternalFile64.length() - 4) + last4Bytes;

        // We have appended the CR e.g. \r, still need to prepend the line feed to the footer
        ending = EndingType_ACE.Cr;
    }

    content += strInternalFile64;

    if(!mimeType.endsWithIgnoreCase('application/octet-stream')) {
        String strValueCrLf = content + STRING_SINGLE_LINE_BREAK;
        Blob blobValueCrLf = Blob.valueOf(strValueCrLf);
        String strValueCrLf64 = EncodingUtil.base64Encode(blobValueCrLf);
        content += safelyPad(content, strValueCrLf64, STRING_SINGLE_LINE_BREAK);
    }
    return content;
}

private static String resolveMimeType(String fileName) {
    String fileType = fileName.substringAfterLast('.');
    String mimeType = 'application/octet-stream'; // fallback value
    if (fileType.equalsIgnoreCase('png')) {
        mimeType = 'image/png';
    } else if (fileType.equalsIgnoreCase('jpeg') || fileType.equalsIgnoreCase('jpg')) {
        mimeType = 'image/jpg';
    } else if (fileType.equalsIgnoreCase('pgm')) {
        mimeType = 'image/x-portable-graymap';
    } else if (fileType.equalsIgnoreCase('ppm')) {
        mimeType = 'image/x-portable-pixmap';
    }
    return mimeType;
}

The only issue here is if you have two image files of 1 MB each, then the string size will grow exponentially, as there is an extra padding to be added and that too in base64 format, and give you heap issues (even in async mode). I was not able to find a workaround to this, but hopefully someday someone or salesforce would do that, or the best solution is to upload files one by one.