API Help on Returning Back custom response and reverting transaction back too in catch block

Question

I have some code below where i want to return the custom error json message to user who is doing inbound call. Also i want to revert back the whole transaction when custom exception came. How i can do both as when i return custom error message, the transaction gets success which I dont want to do in catch block.

@HTTPPost
global static void createCaseFromAPI(){
    RestRequest reqObj = RestContext.request;
    RestResponse resObj = RestContext.response;

    try{
        FirstCalloutToSomeOtherAPI();
        SecondCalloutToSomeOtherAPI();

        insert contact1;
        insert contact 2;


        if(caseMetadataList.size() == 0){       //validations from data of first callout and second callout
            throw new MyCustomAPIException('Necessary attributes has not been passed to identify supported case category, case type and case subtype !');
        }
        
        
        //Create Case based on data from first callout and second callout and after doing validations
        
        insert caseObj;

        responseMap.put('ResponseStatus','SUCCESS');        //some map is there
        
        resObj.statusCode = 200;
        resObj.addHeader('Content-Type', 'application/json');
        resObj.responseBody = Blob.valueOf(JSON.serialize(responseMap));
        createAPILog(reqObj,resObj,'CaseCustomerServiceAPI.createCase',responseMap);
        
        
    }catch(MyCustomAPIException ex) {
        system.debug('ex.getMessage() > '+ex.getMessage());
        system.debug('ex.getMessage() > '+ex.getStackTraceString());
        
        responseMap.put('ResponseStatus','ERROR'); 
        
        resObj.statusCode = 200;
        resObj.addHeader('Content-Type', 'application/json');
        resObj.responseBody = Blob.valueOf(JSON.serialize(responseMap));
//here i am facing issue
    }
}

Answer 1

After making your callouts (you can't callout again after a DML operation), set a savepoint. You can undo all changes from the time the savepoint is set and the time it is rolled back.

SavePoint sp;
try{
    FirstCalloutToSomeOtherAPI();
    SecondCalloutToSomeOtherAPI();
    sp = Database.setSavePoint();
    ...
} catch(MyCustomAPIException e) {
  Database.rollback(sp); // All changes are undone
  ...
}

In your specific code, it's also worth noting that you don't need to throw an exception, as a normal branch would work fine:

FirstCalloutToSomeOtherAPI();
SecondCalloutToSomeOtherAPI();
if(caseMetadataList.size() == 0){       //validations from data of first callout and second callout
    responseMap.put('ResponseStatus','ERROR'); 
    resObj.statusCode = 400; // Indicate bad request
    resObj.addHeader('Content-Type', 'application/json');
    resObj.responseBody = Blob.valueOf(JSON.serialize(responseMap));
    return; // exit early
}
insert contact1;
insert contact2;
//Create Case based on data from first callout and second callout and after doing validations
insert caseObj;
responseMap.put('ResponseStatus','SUCCESS');        //some map is there

resObj.statusCode = 200;
resObj.addHeader('Content-Type', 'application/json');
resObj.responseBody = Blob.valueOf(JSON.serialize(responseMap));
createAPILog(reqObj,resObj,'CaseCustomerServiceAPI.createCase',responseMap);

If you need to catch errors on the DML operations, that can be a separate step, and you can use the rollback statement to help keep your data correct.