We have a NodeJS application which receives & sends data to Postgres database. The Postgres database is in sync with salesforce object through Heroku connect. The JSON that we need to send to salesforce contains more than 1,638,400 characters (1,638,400 is the standard large text field limit).
How can we store/process a JSON payload of this size in salesforce.
I build an app that does something similar - it passes in 200 records at at a time in a JSON array to be processed, and many of those records have long text descriptions as part of the data so are too big for a long text field.
So we have a staging object with a fields on it like status, type, and then the inbound request comes to a custom Apex REST endpoint that creates a staging record and then just creates a regular attachment with the inbound JSON payload for that staging record.
When the inbound process is complete, the final post to the endpoint says 'load complete', and we kick off an apex batch process that iterates through the staging records with their attachments, de-serializes the JSON into an Apex Class, and then convert the records into their target objects (Contacts, Custom Objects). Any errors are just written back to the staging object, and once the process is complete, all successful records are just deleted via a follow up batch.
