2

I am unable to fetch a full report using the Salesforce REST analytics API. As was posted on another question, I am using the following solution(python):

reportFilter = {
    "reportMetadata": {"sortBy": [{"sortColumn": "Contact.Id", "sortOrder": "Asc"}]}}
it = 0
ids = []
while True:
    ids, out, reportFilter = getReportAndConcatIds(ids, reportFilter)
    if out is True:
        break

which calls the following method:

def getReportAndConcatIds(ids, reportFilter):
    headers = {'content-type': 'application/json'}
    r = s.post(link, data=json.dumps(reportFilter), headers=headers)   
    j = r.json()
    for key in j['factMap']['T!T']['rows']:
        ids.append(key['dataCells'][0]['label'])

    reportFilter = {"reportMetadata": {"reportFilters": [{"value": ids[-1], "operator": "greaterThan", "column": "Contact.Id"}],
                                       "sortBy": [{"sortColumn": "Contact.Id", "sortOrder": "Asc"}]}}

    return ids, j['allData'], reportFilter

This performs as intended, it successfully retrieves the contact ids from a report generated within Salesforce, however, it continually loops, even if the report is only 5000 rows long.

After extensive printing I am positive that the ids are being added to the ids array after being extracted from the correct column.

Interestingly, after removing all duplicates from the ids array, the arrays length continues to grow, even after it exceeds the length of the inputted report.

Could someone please help shine some light on what I am doing wrong here?

Thanks in advance

  • Can you link to the source question where the original code came from? – Daniel Ballinger Aug 7 '17 at 1:26
  • How is the reportFilter changing with each iteration? The idea will be to move it to the last Contact ID returned from the report. By sorting the record by Order ID it should be possible to do a rudimentary paging through the results. – Daniel Ballinger Aug 7 '17 at 1:30
  • Is ids[-1] returning the highest Contact ID seen so far with each iteration? – Daniel Ballinger Aug 7 '17 at 1:42
  • Daniel - Here is the solution I followed: glazeforce.wordpress.com/2016/03/29/… reportFilter is being updated on each loop. As the report is being sorted, ids[-1] will return the last highest seen Contact ID seen so far. Let me know if you have any other questions. – Zach Aug 7 '17 at 2:40
  • Here is the original question: salesforce.stackexchange.com/questions/23175/… – Zach Aug 7 '17 at 2:43
0

... after removing all duplicates from the ids array, the arrays length continues to grow, even after it exceeds the length of the inputted report.

I suspect this is indicative of the underlying problem. The moment you start encountering duplicate Contact Ids something has gone wrong. That shouldn't be happening with this approach.

Try something like the following. Maybe consider it pseudo code, as I've never coded Python before.

for key in j['factMap']['T!T']['rows']:
    id = key['dataCells'][0]['label']
    # Check that id hasn't been processed yet
    if id in ids:
        raise Exception('Duplicate id: ' + id)
    ids.append(id)

If this starts throwing errors then the problem is likely around j['allData'] and checking if it is true. Again, I'm not across Python and how it handles typing and JSON processing. It seems like it isn't correctly detecting the allData true case.

The only other likely scenario I can think of is that ids[-1] isn't correctly returning the highest encountered contact id so far. If it was I'd think the report would eventually stop returning any further rows.

  • The issue is that these reports can actually contain duplicate ids - they sometimes span across several campaigns. You can see here how ids[-1] should return the last element in a list in python: stackoverflow.com/questions/930397/…. So it looks like j['allData'] isn't returning true like it should be... I just tested it on a report of 10 rows and it still took 3 run throughs of the above function before it returned True. Any thoughts on why this could be? Thanks for all the help so far. – Zach Aug 7 '17 at 21:41
  • Any chance your report has another unique column that could be used for sorting. I suspect sorting over a non-unique column is causing problems. – Daniel Ballinger Aug 7 '17 at 23:22
  • That said, the allData thing over only 10 rows sounds really odd. – Daniel Ballinger Aug 7 '17 at 23:24
  • After further testing it actually looks like the api response is at times returning ids that don't actually exist in the report (but are still valid contact ids). How strange is this? Unfortunately there are no other unique columns within the report. – Zach Aug 7 '17 at 23:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.