1

I think I have a very specific use case on my hands here and i'm trying to determine a solution.

We are receiving batch files containing leads from a third party. These files are manually extracted from a FTP service and are then manually uploaded into a SFmc DE. Key field here is email, none other relevant fields/ID's are provided.

After importing the leads there is an automation in place to check if the emailaddress in the file already exists on a contact in service cloud. If it does, the automation retrieves some contact information. If not, a new account is created within Service cloud. This is where our process is stuck.

There is a custom integration between our service cloud and SAP. Once an account is created in service cloud, There is an immediate check to determine if the contact already Exists in SAP or not. If it does, perfect, if not a new account is created in SAP. The ID that comes from SAP is leading for all our other systems. The sync between SAP and service cloud is working as intended.

We are using Journey builder to check if the emailaddress from the DE already exists on a contact in service cloud. However if we push more then 100 accounts for creation at the same time then our custom integration with SAP breaks.

As a workaround I have Limited the import ammount to a max of 980. The 980 contacts that enter the journey are then pushed through a random split with 10 paths, where each path has a wait time 3 minutes longer then the previous path. This solution is verified and works, but manually splitting files and doing multiple imports is a tedious job which is also prone to human errors.

Has anyone every experienced a situation like this, or does someone have a better idea to make sure we are never pushing more then 100 accounts for creation at the same time?

I'm looking forward to your reactions.

Kind regards,

Patrick

  • Hi Patrick, sounds like the trigger that updates the data in SAP might not be bulkified properly and that's why it fails at 100 records. It might be a coincidence, but the limit for SOQL queries and Callouts in one transaction is exactly 100. Can you share the trigger code? – RickForce Nov 26 '19 at 14:16
1

You can batch the records using the query activity in the automation studio if you're using the Data Extension as the entry source.

Once all the records have been imported into the SFMC DE from the sftp, you can then use the query activity to look at top 100 records and push them to the entry source for the journey.

For this purpose, you might need to set a default boolean field JourneyProcessed to False in the SFMC DE and upon the import all records will be set to False. Once the contact enters the journey, you'll then use the update contact activity to update the record in your SFMC DE to True. This ensures you will only pickup the unprocessed records for the journey.

Draft SQL:

SELECT TOP 100 * /*only grabs 100 records*/
FROM [SFMC DE] a
WHERE a.JourneyProcessed = 0 /*Only picks up unprocessed records*/

Reference:

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.