2

How can I perform the following operations in Apex on AWS S3 bucket:

  1. List the contents of the bucket
  2. Add a file to the bucket
  3. Update the file
  4. Delete the file

1 Answer 1

6

AWS S3 has many interfaces that can be used for integration. One of the options is the S3 REST API. From the API doc:

Requests to Amazon S3 can be authenticated or anonymous. Authenticated access requires credentials that AWS can use to authenticate your requests. When making REST API calls directly from your code, you create a signature using valid credentials and include the signature in your request. For information about various authentication methods and signature calculations, see Authenticating Requests (AWS Signature Version 4).

Named Credentials implements AWS Signature V4 out of the box. While so, Named Credentials doesn't work for each and every S3 REST API. These S3 APIs are not supported by NC due to "extra" required headers not emitted by NC:

  • DeleteObjects (why)
  • PutObject if there's a retention policy set by S3 Object Lock (why: same reason as DeleteObjects)

It does work for basic CRUD use cases:

  1. List the contents of the S3 bucket
  2. Add a file to the bucket
  3. Update the file
  4. Delete the file

In Winter '23, SF updated the Named Credential framework by separating the identity from the authorization method. As of this writing, the advantage of current-gen NC vs pre-Winter 23 Named Credential (aka "Legacy" NC) when it comes to interfacing with AWS APIs is the ability of current-gen NC to use AWS Security Token Service (STS).

STS is a service that allows the caller to assume a limited-privilege AWS identity for the duration of the API call. While STS is a significant enhancement and a recommended best practice, this Q&A uses Legacy NC for the sake of easier illustration.

Create a Legacy Named Credential and name it aws_s3:

  1. URL: https://<bucket>.s3-<region>.amazonaws.com
  2. Identity Type: Named Principal
  3. Authentication Protocol: AWS Signature V4
  4. AWS Access Key ID: ...
  5. AWS Secret Access Key: ...
  6. AWS Region: ...
  7. AWS Service: s3
  8. Generate Authorization Header: check

If your region is ap-southeast-2 and your bucket name is foo, then the definition of Legacy NC looks like this:

  1. URL: https://foo.s3-ap-southeast-2.amazonaws.com
  2. Identity Type: Named Principal
  3. Authentication Protocol: AWS Signature V4
  4. AWS Access Key ID: ...
  5. AWS Secret Access Key: ...
  6. AWS Region: ap-southeast-2
  7. AWS Service: s3
  8. Generate Authorization Header: check

The URL does not need a trailing slash.

Important: URL, Region and Service must all be lowercase. They're used as inputs to the signature and therefore case-sensitive.

List the contents of the bucket

HttpRequest request = new HttpRequest();
request.setMethod('GET');
request.setEndpoint('callout:aws_s3');
Http http = new Http();
HttpResponse res = http.send(request);
Assert.areEqual(200,res.getStatusCode());
System.debug(res);
System.debug(res.getBody());

For an empty bucket, the response body should look similar to this:

<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Name>foo</Name>
    <Prefix/>
    <Marker/>
    <MaxKeys>1000</MaxKeys>
    <IsTruncated>false</IsTruncated>
</ListBucketResult>

Add a file to the bucket

The object key uniquely identifies the object in a S3 bucket. We'll use the name of our file as the key when we need to operate on the file via S3 REST API.

String fileName = 'readme.txt';
String text = 'Please read this';
HttpRequest request = new HttpRequest();
request.setMethod('PUT');
request.setBody(text); // for binary, use setBodyAsBlob(...)
request.setEndpoint('callout:aws_s3/' + fileName);
Http http = new Http();
HttpResponse res = http.send(request);
Assert.areEqual(200,res.getStatusCode());

If we now list the contents of the bucket by repeating the GET call above, we should see a single S3 object with a key set to the name of our file:

<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Name>foo</Name>
    <Prefix/>
    <Marker/>
    <MaxKeys>1000</MaxKeys>
    <IsTruncated>false</IsTruncated>
    <Contents>
        <Key>readme.txt</Key>
        <LastModified>2022-12-06T23:55:41.000Z</LastModified>
        <ETag>....</ETag>
        <Size>16</Size>
        <Owner>
            <ID>...</ID>
        </Owner>
    </Contents>
</ListBucketResult>

Update the file

String fileName = 'readme.txt';
String text = 'Please ignore this';
HttpRequest request = new HttpRequest();
request.setMethod('PUT');
request.setBody(text); // for binary, use setBodyAsBlob(...)
request.setEndpoint('callout:aws_s3/' + fileName);
Http http = new Http();
HttpResponse res = http.send(request);
Assert.areEqual(200,res.getStatusCode());

Instead of listing objects in the bucket, we dump the contents of the object based on its key which happens to be the name of our file. We are expecting to see the text from the last update:

String fileName = 'readme.txt';
String expected = 'Please ignore this';
HttpRequest request = new HttpRequest();
request.setMethod('GET');
request.setEndpoint('callout:aws_s3/' + fileName);
Http http = new Http();
HttpResponse res = http.send(request);
Assert.areEqual(200,res.getStatusCode());
Assert.areEqual(expected,res.getBody());

Delete the file

String fileName = 'readme.txt';
HttpRequest request = new HttpRequest();
request.setMethod('DELETE');
request.setEndpoint('callout:aws_s3/' + fileName);
Http http = new Http();
HttpResponse res = http.send(request);
Assert.areEqual(204,res.getStatusCode());

Repeating the GET call above should show an empty bucket.

5
  • 3
    tremendous! a valuable addition to the Community
    – cropredy
    Dec 7, 2022 at 0:28
  • Do you know how it can be extended to other operations like generating a pre-signed url?
    – gvgramazio
    Mar 20, 2023 at 18:11
  • @gvgramazio Via NC - possible but very tedious. You're better off handcoding it in Apex using Crypto class and AWS docs
    – identigral
    Mar 20, 2023 at 20:30
  • I was able to reproduce it in apex reproducing the signing algorithm. However, I had to store credentials without protection in a custom setting. I was wondering if it was possible to do with NC also for security reasons.
    – gvgramazio
    Mar 21, 2023 at 13:40
  • @gvgramazio A combination of Named+External cred can do what you're after. Warning: possible issues
    – identigral
    Aug 31, 2023 at 15:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .