9

I'm trying to integrate Kraken.io into our org so I can have Salesforce queue up image optimization jobs that our employees upload. Kraken.io requires that the requests post a multipart/form-data body which contains a JSON parameter and the file. I'm porting code from C# where I could just write to the request stream, but in APEX with the HttpRequest class this doesn't seem to be possible. How can I go about creating a request that can do this?

17

Not very easily is the unfortunate answer. The HttpRequest class doesn't expose the underlying stream so you can't just write a bunch of binary chunks to it the way C# does. Instead, you have to base64 encode all the data you want to send, concatenate it, convert it to a blob and write the entire blob to the request's body. The problem there is that you'll probably end up with base64 padding characters somewhere other than the end of the entire concatenated string which breaks the decoding. You have to safely pad the contents of base64 string to avoid that.

I found some sample code here which got me started. I'll be honest that I didn't like the way it was written in one giant chunk of "stuff". Instead, I created my own FormBuilder class that did all the work for me. I'm posting the code below in the hopes that it will help someone else looking for this functionality in the future. Before we get to the code, I do want to stress that this code is not the best thing made since sliced bread. It is limited based on my needs, but should be easily expandable. Also, if anyone wants to improve it for performance/security/whatever, please be my guest.

Update 7/20/2016

After I posted the original code I decided to "enhance" it a bit. Originally this class was an inner class of another class. Because it was an inner class I couldn't use static methods. I decided to rip it out into its own standalone class and make it static since it doesn't really need to be instanced to be used. I also renamed it to vNHttpFormBuilder to match my naming convention for top level classes.

public class vNHttpFormBuilder {
    //  The boundary is alligned so it doesn't produce padding characters when base64 encoded.
    private final static string Boundary = '1ff13444ed8140c7a32fc4e6451aa76d';

    /**
     *  Returns the request's content type for multipart/form-data requests.
     */
    public static string GetContentType() {
        return 'multipart/form-data; charset="UTF-8"; boundary="' + Boundary + '"';
    }

    /**
     *  Pad the value with spaces until the base64 encoding is no longer padded.
     */
    private static string SafelyPad(
        string value,
        string valueCrLf64,
        string lineBreaks) {
        string valueCrLf = '';
        blob valueCrLfBlob = null;

        while (valueCrLf64.endsWith('=')) {
            value += ' ';
            valueCrLf = value + lineBreaks;
            valueCrLfBlob = blob.valueOf(valueCrLf);
            valueCrLf64 = EncodingUtil.base64Encode(valueCrLfBlob);
        }

        return valueCrLf64;
    }

    /**
     *  Write a boundary between parameters to the form's body.
     */
    public static string WriteBoundary() {
        string value = '--' + Boundary + '\r\n';
        blob valueBlob = blob.valueOf(value);

        return EncodingUtil.base64Encode(valueBlob);
    }

    /**
     *  Write a boundary at the end of the form's body.
     */
    public static string WriteBoundary(
        EndingType ending) {
        string value = '';

        if (ending == EndingType.Cr) {
            //  The file's base64 was padded with a single '=',
            //  so it was replaced with '\r'. Now we have to
            //  prepend the boundary with '\n' to complete
            //  the line break.
            value += '\n';
        } else if (ending == EndingType.None) {
            //  The file's base64 was not padded at all,
            //  so we have to prepend the boundary with
            //  '\r\n' to create the line break.
            value += '\r\n';
        }
        //  Else:
        //  The file's base64 was padded with a double '=',
        //  so they were replaced with '\r\n'. We don't have to
        //  do anything to the boundary because there's a complete
        //  line break before it.

        value += '--' + Boundary + '--';

        blob valueBlob = blob.valueOf(value);

        return EncodingUtil.base64Encode(valueBlob);
    }

    /**
     *  Wirte a file to the form's body.
     */
    public static WriteFileResult WriteFile(
        string key,
        string value,
        string mimeType,
        blob fileBlob) {
        EndingType ending = EndingType.None;
        string contentDisposition = 'Content-Disposition: form-data; name="' + key + '"; filename="' + value + '"';
        string contentDispositionCrLf = contentDisposition + '\r\n';
        blob contentDispositionCrLfBlob = blob.valueOf(contentDispositionCrLf);
        string contentDispositionCrLf64 = EncodingUtil.base64Encode(contentDispositionCrlfBlob);
        string content = SafelyPad(contentDisposition, contentDispositionCrLf64, '\r\n');
        string contentType = 'Content-Type: ' + mimeType;
        string contentTypeCrLf = contentType + '\r\n\r\n';
        blob contentTypeCrLfBlob = blob.valueOf(contentTypeCrLf);
        string contentTypeCrLf64 = EncodingUtil.base64Encode(contentTypeCrLfBlob);

        content += SafelyPad(contentType, contentTypeCrLf64, '\r\n\r\n');

        string file64 = EncodingUtil.base64Encode(fileBlob);
        integer file64Length = file64.length();
        string file64Ending = file64.substring(file64Length - 3, file64Length);

        if (file64Ending.endsWith('==')) {
            file64Ending = file64Ending.substring(0, 1) + '0K';//   0K = \r\n
            file64 = file64.substring(0, file64Length - 3) + file64Ending;
            ending = EndingType.CrLf;
        } else if (file64Ending.endsWith('=')) {
            file64Ending = file64Ending.substring(0, 2) + 'N';//    N = \r
            file64 = file64.substring(0, file64Length - 3) + file64Ending;
            ending = EndingType.Cr;
        }

        content += file64;

        return new WriteFileResult(content, ending);
    }

    /**
     *  Write a key-value pair to the form's body.
     */
    public static string WriteBodyParameter(
        string key,
        string value) {
        string contentDisposition = 'Content-Disposition: form-data; name="' + key + '"';
        string contentDispositionCrLf = contentDisposition + '\r\n\r\n';
        blob contentDispositionCrLfBlob = blob.valueOf(contentDispositionCrLf);
        string contentDispositionCrLf64 = EncodingUtil.base64Encode(contentDispositionCrLfBlob);
        string content = SafelyPad(contentDisposition, contentDispositionCrLf64, '\r\n\r\n');
        string valueCrLf = value + '\r\n';
        blob valueCrLfBlob = blob.valueOf(valueCrLf);
        string valueCrLf64 = EncodingUtil.base64Encode(valueCrLfBlob);

        content += SafelyPad(value, valueCrLf64, '\r\n');

        return content;
    }

    /**
     *  Helper class containing the result of writing a file's blob to the form's body.
     */
    public class WriteFileResult {
        public final string Content { get; private set; }
        public final EndingType EndingType { get; private set; }

        public WriteFileResult(
            string content,
            EndingType ending) {
            this.Content = content;
            this.EndingType = ending;
        }
    }

    /**
     *  Helper enum indicating how a file's base64 padding was replaced.
     */
    public enum EndingType {
        Cr,
        CrLf,
        None
    }
}

And here's how I use it when submitting files to Kreaken.io. So far all submissions have been successful so I'm assuming that the form is being composed correctly. The boundary string is a GUID and I just took out the dash separators. It also happens to have enough characters that when the full boundary is composed it doesn't produce base64 padding characters. If replacing it, I recommend following the same pattern to save yourself some headaches.

Update 7/20/2016

Updating the code below to match the re-worked "static" class further up.

private KrakenResponse Submit(
    KrakenRequest request,
    string url,
    string fileName,
    string fileMimeType,
    blob fileBlob) {
    try {
        string contentType = vNHttpFormBuilder.GetContentType();
        string json = GetJson(request, Credentials);

        //  Compose the form
        string form64 = '';

        form64 += vNHttpFormBuilder.WriteBoundary();
        form64 += vNHttpFormBuilder.WriteBodyParameter('json', json);
        form64 += vNHttpFormBuilder.WriteBoundary();

        vNHttpFormBuilder.WriteFileResult result = vNHttpFormBuilder.WriteFile('file', fileName, fileMimeType, fileBlob);

        form64 += result.Content;
        form64 += vNHttpFormBuilder.WriteBoundary(result.EndingType);

        blob formBlob = EncodingUtil.base64Decode(form64);
        string contentLength = string.valueOf(formBlob.size());

        //  Compose the http request
        HttpRequest httpRequest = new HttpRequest();

        httpRequest.setBodyAsBlob(formBlob);
        httpRequest.setEndpoint(url);
        httpRequest.setHeader('Connection', 'keep-alive');
        httpRequest.setHeader('Content-Length', contentLength);
        httpRequest.setHeader('Content-Type', contentType);
        httpRequest.setMethod('POST');
        httpRequest.setTimeout(120000);

        KrakenResponse response = GetHttpResponse(httpRequest);

        if (!response.Success) {
            System.debug(form64);
        }

        return response;
    } catch (Exception e) {
        return null;
    }
}

Lastly, I figured I might as well add the unit tests for this class, which currently have 100% coverage.

@IsTest
private static void test_vNHttpFormBuilder() {
    blob fileABlob = EncodingUtil.base64Decode('/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQECAgICAgICAgICAgMDAwMDAwMDAwP/2wBDAQEBAQEBAQIBAQICAgECAgMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwP/wAARCAAKAAoDAREAAhEBAxEB/8QAFQABAQAAAAAAAAAAAAAAAAAAAAP/xAAmEAAAAwMNAAAAAAAAAAAAAAAAFhfI6PAYKCk2R2ZnaoaJmKn4/8QAFAEBAAAAAAAAAAAAAAAAAAAAAP/EABQRAQAAAAAAAAAAAAAAAAAAAAD/2gAMAwEAAhEDEQA/AJ1euwWNClEi8WE6TqSxg6S0dsdR2joB5Hcmdd6ywCJo7EzLu2WARGQDAf/Z');
    blob fileBBlob = EncodingUtil.base64Decode('/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQECAgICAgICAgICAgMDAwMDAwMDAwP/2wBDAQEBAQEBAQIBAQICAgECAgMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwP/wAARCAAKAAoDAREAAhEBAxEB/8QAFQABAQAAAAAAAAAAAAAAAAAAAAX/xAAkEAABAwIFBQEAAAAAAAAAAAAFAwQHAgYACMjo8Akoaomp+P/EABYBAQEBAAAAAAAAAAAAAAAAAAADBf/EABwRAAMBAQEBAQEAAAAAAAAAAAECAwQABREhEv/aAAwDAQACEQMRAD8AigArOOQQW3RyxYAPjkSNCsXB8+dHnbYZxgzRYtljV0XQSy43NbxaP0MqNNTl+RdxO/thaHqlnC0RKxBXX06ZQimaCZ5lzOaBQWZnYhR8H9O5Z3b4P1nZmY/rEkk9oet6en2vV0+xsXOmvXopZ1hCGWCvVy7COXLOObNIMxE4Z5ShFPk5TSaqorfkfZNtd+ZeK9n852j6JtLvrLw5znPADw53/9k=');
    blob fileCBlob = EncodingUtil.base64Decode('/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQECAgICAgICAgICAgMDAwMDAwMDAwP/2wBDAQEBAQEBAQIBAQICAgECAgMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwP/wAARCAAMAAwDAREAAhEBAxEB/8QAFQABAQAAAAAAAAAAAAAAAAAAAAP/xAAtEAAAAgQLCQAAAAAAAAAAAAAWFwA3OGcYGSk2OUdJZoen8Gl3hoiJmKi4x//EABQBAQAAAAAAAAAAAAAAAAAAAAD/xAAUEQEAAAAAAAAAAAAAAAAAAAAA/9oADAMBAAIRAxEAPwCc3rsBjgUIgXtYLouoLDnQWTtTpOydAHc5WeKqg8m2TnN7OJAbucLPgbKvKcpvBuziQHrVrpLUS2ot1A//2Q==');
    string formA64 = '';
    string formB64 = '';
    string formC64 = '';

    Test.startTest();

    string contentType = vNHttpFormBuilder.GetContentType();

    formA64 += vNHttpFormBuilder.WriteBoundary();
    formA64 += vNHttpFormBuilder.WriteBodyParameter('key', 'value');
    formA64 += vNHttpFormBuilder.WriteBoundary();

    vNHttpFormBuilder.WriteFileResult resultA = vNHttpFormBuilder.WriteFile('key', 'value', 'image/jpeg', fileABlob);

    formA64 += resultA.Content;
    formA64 += vNHttpFormBuilder.WriteBoundary(resultA.EndingType);

    formB64 += vNHttpFormBuilder.WriteBoundary();
    formB64 += vNHttpFormBuilder.WriteBodyParameter('key', 'value');
    formB64 += vNHttpFormBuilder.WriteBoundary();

    vNHttpFormBuilder.WriteFileResult resultB = vNHttpFormBuilder.WriteFile('key', 'value', 'image/jpeg', fileBBlob);

    formB64 += resultB.Content;
    formB64 += vNHttpFormBuilder.WriteBoundary(resultB.EndingType);

    formC64 += vNHttpFormBuilder.WriteBoundary();
    formC64 += vNHttpFormBuilder.WriteBodyParameter('key', 'value');
    formC64 += vNHttpFormBuilder.WriteBoundary();

    vNHttpFormBuilder.WriteFileResult resultC = vNHttpFormBuilder.WriteFile('key', 'value', 'image/jpeg', fileCBlob);

    formC64 += resultC.Content;
    formC64 += vNHttpFormBuilder.WriteBoundary(resultC.EndingType);

    Test.stopTest();

    System.assert(contentType == 'multipart/form-data; charset="UTF-8"; boundary="1ff13444ed8140c7a32fc4e6451aa76d"');
    System.assert(formA64 == 'LS0xZmYxMzQ0NGVkODE0MGM3YTMyZmM0ZTY0NTFhYTc2ZA0KQ29udGVudC1EaXNwb3NpdGlvbjogZm9ybS1kYXRhOyBuYW1lPSJrZXkiICANCg0KdmFsdWUgIA0KLS0xZmYxMzQ0NGVkODE0MGM3YTMyZmM0ZTY0NTFhYTc2ZA0KQ29udGVudC1EaXNwb3NpdGlvbjogZm9ybS1kYXRhOyBuYW1lPSJrZXkiOyBmaWxlbmFtZT0idmFsdWUiIA0KQ29udGVudC1UeXBlOiBpbWFnZS9qcGVnICANCg0K/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQECAgICAgICAgICAgMDAwMDAwMDAwP/2wBDAQEBAQEBAQIBAQICAgECAgMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwP/wAARCAAKAAoDAREAAhEBAxEB/8QAFQABAQAAAAAAAAAAAAAAAAAAAAP/xAAmEAAAAwMNAAAAAAAAAAAAAAAAFhfI6PAYKCk2R2ZnaoaJmKn4/8QAFAEBAAAAAAAAAAAAAAAAAAAAAP/EABQRAQAAAAAAAAAAAAAAAAAAAAD/2gAMAwEAAhEDEQA/AJ1euwWNClEi8WE6TqSxg6S0dsdR2joB5Hcmdd6ywCJo7EzLu2WARGQDAf/ZDQotLTFmZjEzNDQ0ZWQ4MTQwYzdhMzJmYzRlNjQ1MWFhNzZkLS0=');
    System.assert(formB64 == 'LS0xZmYxMzQ0NGVkODE0MGM3YTMyZmM0ZTY0NTFhYTc2ZA0KQ29udGVudC1EaXNwb3NpdGlvbjogZm9ybS1kYXRhOyBuYW1lPSJrZXkiICANCg0KdmFsdWUgIA0KLS0xZmYxMzQ0NGVkODE0MGM3YTMyZmM0ZTY0NTFhYTc2ZA0KQ29udGVudC1EaXNwb3NpdGlvbjogZm9ybS1kYXRhOyBuYW1lPSJrZXkiOyBmaWxlbmFtZT0idmFsdWUiIA0KQ29udGVudC1UeXBlOiBpbWFnZS9qcGVnICANCg0K/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQECAgICAgICAgICAgMDAwMDAwMDAwP/2wBDAQEBAQEBAQIBAQICAgECAgMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwP/wAARCAAKAAoDAREAAhEBAxEB/8QAFQABAQAAAAAAAAAAAAAAAAAAAAX/xAAkEAABAwIFBQEAAAAAAAAAAAAFAwQHAgYACMjo8Akoaomp+P/EABYBAQEBAAAAAAAAAAAAAAAAAAADBf/EABwRAAMBAQEBAQEAAAAAAAAAAAECAwQABREhEv/aAAwDAQACEQMRAD8AigArOOQQW3RyxYAPjkSNCsXB8+dHnbYZxgzRYtljV0XQSy43NbxaP0MqNNTl+RdxO/thaHqlnC0RKxBXX06ZQimaCZ5lzOaBQWZnYhR8H9O5Z3b4P1nZmY/rEkk9oet6en2vV0+xsXOmvXopZ1hCGWCvVy7COXLOObNIMxE4Z5ShFPk5TSaqorfkfZNtd+ZeK9n852j6JtLvrLw5znPADw53/9kNCi0tMWZmMTM0NDRlZDgxNDBjN2EzMmZjNGU2NDUxYWE3NmQtLQ==');
    System.assert(formC64 == 'LS0xZmYxMzQ0NGVkODE0MGM3YTMyZmM0ZTY0NTFhYTc2ZA0KQ29udGVudC1EaXNwb3NpdGlvbjogZm9ybS1kYXRhOyBuYW1lPSJrZXkiICANCg0KdmFsdWUgIA0KLS0xZmYxMzQ0NGVkODE0MGM3YTMyZmM0ZTY0NTFhYTc2ZA0KQ29udGVudC1EaXNwb3NpdGlvbjogZm9ybS1kYXRhOyBuYW1lPSJrZXkiOyBmaWxlbmFtZT0idmFsdWUiIA0KQ29udGVudC1UeXBlOiBpbWFnZS9qcGVnICANCg0K/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQECAgICAgICAgICAgMDAwMDAwMDAwP/2wBDAQEBAQEBAQIBAQICAgECAgMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwP/wAARCAAMAAwDAREAAhEBAxEB/8QAFQABAQAAAAAAAAAAAAAAAAAAAAP/xAAtEAAAAgQLCQAAAAAAAAAAAAAWFwA3OGcYGSk2OUdJZoen8Gl3hoiJmKi4x//EABQBAQAAAAAAAAAAAAAAAAAAAAD/xAAUEQEAAAAAAAAAAAAAAAAAAAAA/9oADAMBAAIRAxEAPwCc3rsBjgUIgXtYLouoLDnQWTtTpOydAHc5WeKqg8m2TnN7OJAbucLPgbKvKcpvBuziQHrVrpLUS2ot1A//2Q0KLS0xZmYxMzQ0NGVkODE0MGM3YTMyZmM0ZTY0NTFhYTc2ZC0t');
}
  • In the SafelyPad method you're adding spaces to the end in order to avoid encoding problems, therefore "changing" the original value so the receiving end will get a value with trailing whitespace. Is there a workaround to avoid sending values with trailing whitespaces? – jonathanwiesel Jun 30 '17 at 20:52
  • Unfortunately, as far as I know, no. That's the trade off I accepted to prevent encoding problems. I've settled for the hope that the recipient side does a trim on the values. All that being said, it's been almost an year since I implemented this code and it's going strong so far without any issues when calling into Kraken.io. I have the image processing as a scheduled task every three minutes, and it easily processes several thousand photos per day, depending on load and it just works. – Gup3rSuR4c Jul 1 '17 at 1:04
2

Here's my version of the solution...

    string   formBoundary = '----sfdc-multi-form',
         footer = '\r\n--'+formBoundary+'--',                
         headerForm = '--'+formBoundary+'\r\nContent-Disposition: form-data; name="{0}";\r\n\n',
         headerFile = '--'+formBoundary+'\r\nContent-Disposition: form-data; name="{0}"; filename="{1}";\r\nContent-Type: {2}\r\n\n';

string documentHexContent = EncodingUtil.convertToHex(blob.valueOf((string.format(headerForm, new list {'document'})+
                                                                            )));
string fileHexContent = EncodingUtil.convertToHex(blob.valueOf('\r\n'+string.format(headerFile, new list {'filename',fileName,'application/pdf'})))+
                                    EncodingUtil.convertToHex()+
                                    EncodingUtil.convertToHex(blob.valueof(footer));
HttpRequest req = new HttpRequest();            
            req.setMethod('POST');
            req.setEndpoint('#resource URL#');
            req.setHeader('Accept', '#Accept content type#');
            req.setHeader('Content-Type', 'multipart/form-data; boundary='+formBoundary);            
            req.setBodyAsBlob(body);
            req.setHeader('Content-Length',string.valueOf(req.getBodyAsBlob().size()));
HttpResponse resp = new Http().send(req);

  • EncodingUtil.convertToHex() does work since there is no padding with this method, caveate is the heap is impacted at 2X file size so only good for smaller files. ( ~2.9MB =5.95MB HEAP ) you missed defining the body variable in the code above (ie. req.setBodyAsBlob(EncodingUtil.convertFromHex(documentHexContent+fileHexContent)); – Tom Snyder Nov 30 '18 at 15:56
  • @Jeys I would like to know what does "----sfdc-multi-form" in boundary, didn't document in Salesforce? – Vivek P Jan 9 at 6:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.