Named Credential auto-generate aws sig4 not working for s3 'versions'

Question

Main issue:

I get a signature mismatch error (403 forbidden) when trying to hit the REST api for s3 using the versions action (ListObjectVersions). It seems to me that this may be an error with how the auto-generated auth header is being generated, but I'm looking for clues on how to debug and/or fix this.

The exact error is:

<Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>

The odd thing about this is everything was working for the ListObjectsv2 (list-type=2 in the REST request). The only thing I changed in my url was to replace list-type=2 with versions. I'm changing the action because I need to grab all versions of an object (file) instead of just all current objects.

I checked bucket policies and IAM permissions and verified the user's access locally, so that leads me to believe it's an issue with the auto-generated auth Header within Salesforce. The user has ListBucket and ListBucketVersions permissions.

Sample new url with the error:

https://mybucket.s3.amazonaws.com/?versions&prefix=myFolderPath

Sample old url that works:

https://mybucket.s3.amazonaws.com/?list-type=2&prefix=myFolderPath

where myFolderPath is UTF-8 url encoded, and ends in a slash (before encoding).

The Apex classes that use the named credential are on api version 47, but are being migrated to 50 right now. It appears that the auto-generate feature was added in version 46.

Answer 1

I'm going to chalk this up to being an issue with the AWS S3 REST API. Their documentation clearly shows the following in their example:

GET /?versions&delimiter=Delimiter&encoding-type=EncodingType&key-marker=KeyMarker&max-keys=MaxKeys&prefix=Prefix&version-id-marker=VersionIdMarker HTTP/1.1
Host: Bucket.s3.amazonaws.com
x-amz-expected-bucket-owner: ExpectedBucketOwner

But in reality, it expects it in this format (using my original example):

https://mybucket.s3.amazonaws.com/?prefix=myFolderPath&versions=1

The query parameters have to be in alphabetical order (ASCII sort order) and the versions parameter has to have =1 on the end.

I saw that the sig4 signature has to have the query parameters in order, but it didn't say anything about the actual url sent to amazon needing to have parameters in sort order.