force-cli randomly fails

Question

I'm trying to use force-cli (force-cli.heroku.com) to automate a weekly export from a salesforce org. It works great until it seemingly randomly doesn't work.

#/bin/bash

DATE="$(date -v-7d +%Y-%m-%d)"
force login [email protected] -p=password

force query Select Id, IsDeleted, MasterRecordId, Name, LastName, FirstName, Salutation, Type, RecordTypeId, ParentId, Phone, Fax, Website, PhotoUrl, NumberOfEmployees, Ownership, Description, Rating, OwnerId, CreatedDate, CreatedById, LastModifiedDate, LastModifiedById, SystemModstamp, LastActivityDate, MayEdit, IsLocked . . .FROM Account WHERE CreatedDate \>\= LAST_WEEK > ./export_account/Account_extract_$DATE.csv

force logout [email protected]

It can work many times and then suddenly fail with this error:

failed MSpanList_Insert 0xbcf0f0 0x742df7d9c7f2 0x0 0x0
fatal error: MSpanList_Insert

    runtime stack:
    runtime.throw(0x5f5630, 0x10)
        /home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/panic.go:530 +0x90
    runtime.(*mSpanList).insert(0x87a868, 0xbcf0f0)
        /home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/mheap.go:933 +0x293
    runtime.(*mheap).freeSpanLocked(0x87a060, 0xbcf0f0, 0x100, 0x0)
        /home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/mheap.go:809 +0x4be
    runtime.(*mheap).grow(0x87a060, 0x8, 0x0)
        /home/travis/.gimme/versions/go1.6.linux.amd64/src/runtime/mheap.go:675 +

Has anyone seen this behavior before? Is force-cli the best way to automate exports from a salesforce org or is there a better way?

Answer 1

That's a heap error (a.k.a. "out of memory"). I'm not sure if force-cli has a way to increase allocated memory. In theory, the Salesforce DX CLI might let you do queries like this, but until release, there's no way to confirm. If you know how to write Java, you could pretty easily hack it in as an add-on module to the Migration Toolkit, which already gives you everything you need (e.g. login call, query call), but you'd have to write your own Ant Task for that purpose. Systems such as Informatica (paid services) also lets you schedule regular data exports, so you might look into that as well. All-in-all, the force-cli is probably the least reliable means of doing regular reports, but it is free.