I'm going to make sure this stays specific and not flagged as too broad.

I'm putting together an approach to store data in Heroku. This data is relatively simple in nature - it's key/value pairs, where the key is a unique ID, and the value is JSON, somewhere around 50k lines long. The primary requirement is fast retrieval of the value (JSON), by the unique key.

I've heard of three data services that Heroku offers: Postgres, Redis, and Kafka. My specific question is: solely based on the criteria above (the nature of the data being key/value pairs with JSON as the value, and the quickest retrievel), is one of these data services a better choice than the others?

A few side notes - CDN is also being evaluated, but for the sake of keeping this question specific and not being closed, looking for input on Heroku here :) Also, the volume of data will be several million rows, so let's also exclude the possibility of storing this data in Salesforce as well

  • How about big objects?
    – RedDevil
    Aug 9 '18 at 17:01
  • @RedDevil this is a really interesting idea and I hadn't thought of it before - I just did some brief research. A possible issue is that we can expect the JSON to be upwards of 1M characters. Even if we broke that up across multiple long text fields, the limit on an object appears to be 1638400. Really appreciate this idea though, and we're going to run with it in a parallel analysis Aug 9 '18 at 17:37

A few of us discussed this on the Heroku Support team, including our lead Data Support Engineer. We all came up with a handful of ways to do this, but we limited it down to Postgres or Redis, with Postgres coming out on top. As long as you're not trying to query on that big piece of JSON -- just looking it up by the key -- Postgres should work just fine for you.

  • Thank you very much for taking the time to respond! Sep 20 '18 at 21:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.