Insert and Update a Data Extension

Question

I have a data extension called "master-data-extension" in ET. my objective is to insert new data extension objects in this "master-data-extension" without overwriting the existing ones. So far I have tried using 'UpdateRequest'.

UpdateRequest update = new UpdateRequest();
UpdateOptions options = new UpdateOptions();
update.setOptions(options);
update.getObjects().addAll(request.getObjects()); 
UpdateResponse response = exactTargetService.update(update);

But the problem with this procedure as mentioned, it overwrites the new records with the existing ones. An alternative would be to retrieve all of the existing data extension objects from master data extension and then add them with the new objects and insert all of them together. I could go with this approach but the problem is the master data extension would be retrieved and modified by more than one instance of my application concurrently. for example

think of two reqeusts who would get two different instances of my application. one of them retrieved existing dataextension objects from master data extension and still in the process of adding them with the new objects, while the other request is already completed. So now if the first request goes and update the master data extension, I would loose the data objects inserted by the second request.

I might be overreacting may be there is a simpler solution. but I dont know whats gonna happen with the above solution.

I would appreciate any comments or suggestion or solution.

Answer 1

Here is how you would simply add data to the data extension object - http://help.exacttarget.com/en/technical_library/web_service_guide/technical_articles/adding_data_to_data_extension_object/

the UpdateRequest is going to always update, and never add anything. The primary key, means you can't add existing records.

I would definitely not recommend pulling all the data, and re-adding. This would cause all kinds of performance issues, and in general development standards, just tastes bad.

Whatever your primary key is here, you would need to create a different primary key. I'm not exactly sure your use case, but it's the master-de records that will be unique in some way, and not the actual data you are inserting.

You could potentially remove the primary key, and instead set the key to the date/time stamp of insertion, or some sort of GUID.

Hope this helps!

---

Update

Marketing Cloud developer documentation has a new home! It has moved Web Services SOAP API documentation to the Salesforce Developers portal. Here is the new link to the one used in this answer:

https://developer.salesforce.com/docs/atlas.en-us.noversion.mc-apis.meta/mc-apis/adding_data_to_data_extension_object.htm