We have a list of around 30 data extension with around 2 millions records each and 20 columns each data extension. What is the best way of pulling the data and clearing it? Using the SOAP API it takes a while.
Maybe use REST or Bulk API?
You could write a (External) Java script that will use BULK API to get the records, then do your cleaning magic, and then upload it again via BULK API in batches.