Currently, our Web Development team is trying to mop up years of scattered coding techniques (PHP based sites), and in recent weeks, something disturbing has happened. Our API calls to Salesforce have quadrupled. (yes, 4x normal, really closer to 5x) It has been a few weeks, and eventually, SF is going to contact me and start discussing several THOUSAND dollars in fees for the increased calls (already investigated that fact). :-/
Attached I'm including a graph of our API usage for the last several days. Despite the very odd fluke of the 9th having NORMAL levels of usage (otherwise for 3 weeks it's been consistently what is seen per-day in this graph), we cannot figure out what is causing this.
On the Salesforce side, after two exchanges on different Cases with Support (sigh), I only end up with the "User" login you can download from the User page (we have a designated "robot" user who does all API calls). That user shows clearly that the two IPs for our balanced web servers ARE the ones causing this. But there is nothing inherent in Salesforce to show the path to a script / the URL, or anything else helpful. Literally just IP addresses. Reports do not help as they don't even SHOW this activity, much less what URL or script is making the call to Salesforce.
On the Apache/Linux side (as I am not a guru on such things by any means), no one knows of anything preinstalled or that can be added that would show "this script is calling out to this IP/ Site/ etc" so we could log it globally on the site and see what script(s) are causing this (very likely an "include" script, possibly one of the ones used for webinar registration or whatever).
The graph shows that the highs are during our (US) business hours. We do send out quite a bit of opted-in email marketing, and the result is people visiting the website and registering for webinars, using our Moodle, accessing other items. Various things do call out to Salesforce at different times. BUT we also have cron (timed) jobs out of vBulletin that run scripts anywhere from once a day to once an hour to once every 15 minutes. We've investigated those and haven't isolated one doing this yet (plus, none of their timings seems to fit the low-high pattern in the graph attached). There are also some URLs used internally to add people to Salesforce or that involve calling OUT so Salesforce via PHP pages.
So... I'm reaching out... hoping beyond hope that someone knows of a way to figure this out.
- Something in Linux or Apache that would show calls "to Salesforce" and include exatly WHAT is calling (script path+name, app, etc - 99% certain it's a PHP script) (which would be either by domain or by IPs - IPs are numerous and unknown what the full range is anyway).
- Something in Salesforce that Support is missing (in Tier 1's infinite wisdom).
- Something on AppExchange (I've searched, but...) that could be added to Salesforce.
Something! Help? Thanks for any insights.
This is an automated notification sent to you at your request by salesforce.com.
Your organization has made 627982 API calls within the last 24 hours.
This is 462% of your organization's 24-hour API call limit of 136000
calls. You will receive this notice once every 3 hours until your
organization's API usage drops below 90% of your 24-hour call
limit (122400 calls).
The graph is from left to right labelled with Hour-Day (so from July 02 at 1900 to July 10 at 2200)