0

While working with data extension to import data from SFMC into my application, facing some of the issues mentioned below.

  1. Is there a way to get the count of records per data extension using API in C#? or workaround?

    • No visibility access from UI, hence it required to handle error handline and timeout issues.
  2. If the data is huge in DE, API's should be called recursively until all data is retrieved. During this process (after around 20 minutes) token is expiring. Is there a way to handle this issue during the request or begin a request where it got expired?

    • Looks like DE has lakhs of records.
  3. As mentioned above, every request returns 2500 records, hence do-while logic is required to fetch all data. But strangely every request is returning the same 2500 records for me from DE. Hence, it's going into infinite loop and timing out of 6 hours of time out time. Though referred from [Marketing Cloud Data extensions API Example][1] this issue still persists.

The code snippet is as below: What could be wrong with this?

SoapClient integrationFramework = new SoapClient();
integrationFramework.Endpoint.EndpointBehaviors.Add(new FuelOAuthHeaderBehavior(fuelOAuthAccessToken));
 RetrieveRequest rr = new RetrieveRequest
                {
                    ObjectType = $"DataExtensionObject[{request.File.Id}]",

                    Properties = properties // Added dynamically
                };
                List<DataExtensionObject> dataExtensionObjects = new List<DataExtensionObject>();
do
                {
                    try
                    {
                        retrieve = integrationFramework.RetrieveAsync(new RetrieveRequest1(rr)).Result;

                        if (retrieve.OverallStatus.StartsWith("Error"))
                        {
                            throw new Exception(retrieve.OverallStatus);
                        }
                        else
                        {
                            dataExtensionObjects.AddRange(retrieve.Results.ToList().Cast<DataExtensionObject>().ToList());

                        }


                    }
                    catch (Exception ex)
                    {

                        if (ex.Message.Contains("Token Expired"))
                        {
                            sfmcValidation = IsValidSalesforceMarketingCloudLogin(new SalesforceMarketingCloudAzureRequest()
                            {
                                ClientId = request.SfmcClientid,
                                ClientSecret = request.SfmcClientSecret,
                                AuthenticationUrl = request.SfmcAuthenticationUrl
                            });

                            if (sfmcValidation.IsSuccessful)
                            {
                                fuelOAuthAccessToken = sfmcValidation.Response.AccessToken;

                                integrationFramework = new SoapClient();
                                integrationFramework.Endpoint.EndpointBehaviors.Add(
                                    new FuelOAuthHeaderBehavior(fuelOAuthAccessToken));

                            }
                            else
                            {
                                throw new Exception(sfmcValidation.Message);
                            }

                        }
                        else
                        {
                            throw ex;
                        }
                    }

                   } while (retrieve.OverallStatus.Equals("MoreDataAvailable"));

````

  [1]: https://developer.salesforce.com/docs/atlas.en-us.mc-apis.meta/mc-apis/retrieving_data_from_a_data_extension.htm
2
  • You need to use the ContinueRequest property to get further pages of data. Aside from this, I'd consider dumping the API based approach and opt for using Automation Data Extracts in combination with SFTP to get data out in bulk. It's far more scalable than the approach you're taking. – Macca Jun 7 '20 at 4:21
  • @Macca, Thanks for the help. Any references will be appreciated. – Vaibhav More Jun 7 '20 at 16:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.