5

My end goal is to be able to send license usage as well as data/file storage usage from clients to our org. As these values don't appear to be available directly via any API calls, I had to resort to loading up those pages and scraping the values out. This works just fine, until it needs to be scheduled to run at a particular time.

Scheduled batches can't make callouts, so I also had to resort to putting the portions of the code that handle the callouts in a method with the @future(callout=true) annotation.

Now the issue was that even though the @future method could make callouts, it wasn't able to load any internal Salesforce pages, like the organization info or storage pages. It didn't appear to keep the user's session, even though it appeared to be executing under their user. So I wrote up some extra steps to grab the user's session when it could be obtained (When the schedulable batch class is constructed) and it passes that along so that it can be used when it needs to load those internal pages. I knew that the session would expire at some point, but I was unsure of how long that would be. I also wasn't sure when exactly the schedulable batch classes were being constructed before being executed.

I found out that the org was set to have a user's session expire after 2 hours, which explains why the batch worked fine an hour after it was first set up, but not any of the following times.

So now the only idea I have left to try is to set up a public site with another webservice that can grab the require information and return it to the @future method, at which point that can be sent to our org. This of course adds quite a few more setup steps than simply copying some Apex Classes and a remote site setting as was required previously, so I would like to avoid that option if possible.

Here is my current source code for reference followed by an explanation of the process in general.


ClientOrgStatusWebservice Apex Class:

public class ClientOrgStatusWebservice {
    public ClientOrgStatusWebservice() {

    }

    @future(callout = true)
    public static void notifyCRMFuture(String hostURL, String sessionID, String orgID) {
        String clientOrgStatusJSON = JSON.serialize(ClientOrgStatusWebservice.getClientInfo(hostURL, sessionID, orgID));

        final String expectedIdentificationString = 'UniqueIdentificationString';

        ClientOrgStatusWebserviceSOAP.ClientOrgStatusWebservice clientOrgStatusWebserviceSOAP = new ClientOrgStatusWebserviceSOAP.ClientOrgStatusWebservice();
        clientOrgStatusWebserviceSOAP.endpoint_x = 'https://OurOrgDomain/clientOrgStatusNotifier/services/Soap/class/ClientOrgStatusWebservice';
        clientOrgStatusWebserviceSOAP.timeout_x = 10000;

        clientOrgStatusWebserviceSOAP.clientDataHandler(clientOrgStatusJSON, expectedIdentificationString);
    }

    public static ClientOrgStatusWebservice.ClientInfo getClientInfo(String hostURL, String sessionID, String orgID) {
        Map<String, Integer[]> clientLicenseData = ClientOrgStatusWebservice.grabClientLicenseData(hostURL, sessionID, orgID);

        Map<String, String[]> clientStorageData = ClientOrgStatusWebservice.grabStorageUsageData(hostURL, sessionID, orgID);

        ClientOrgStatusWebservice.ClientInfo clientInfo = new ClientOrgStatusWebservice.ClientInfo(orgID, clientLicenseData, clientStorageData);
        return clientInfo;
    }

    public static Map<String, Integer[]> grabClientLicenseData(String hostURL, String sessionID, String orgID) {
        String pageContent;

        if (!Test.isRunningTest()) {
            pageContent = ClientOrgStatusWebservice.getPageContent(hostURL, sessionID, '/' + orgID);
        } else {
            pageContent = ClientOrgStatusWebservice.getTestLicensePage();
        }

        String[] userLicenseTypes = new String[] {'Salesforce', 'Salesforce Platform', 'Partner', 'Customer Portal Manager', 'Authenticated Website', 'High Volume Customer Portal', 'Service Cloud', 'Force.com - Free', 'Chatter Free', 'Chatter External', 'Customer Portal Manager Standard', 'Customer Portal Manager Custom', 'Gold Partner', 'Silver Partner', 'Customer Community Login', 'Customer Community', 'Partner Community', 'Partner Community Login'};

        Map<String, Integer[]> userLicenseData = new Map<String, Integer[]>();
        for (String licenseType : userLicenseTypes) {
            userLicenseData.put(licenseType, ClientOrgStatusWebservice.getDataForLicenseType(licenseType, pageContent));
        }

        return userLicenseData;
    }

    public static Integer[] getDataForLicenseType(String licenseType, String pageData) {
        Matcher salesforceLicenseMatcher = Pattern.compile('<tr[\\sA-Za-z="().{;}]*><th[\\sA-Za-z="]*>' + licenseType + '</th><td[\\sA-Za-z="]*>Active</td><td[\\sA-Za-z="]*>([0-9,]*)</td><td[\\sA-Za-z="]*>([0-9,]*)</td><td[\\sA-Za-z="]*>([0-9,]*)</td><td[\\sA-Za-z="]*>').matcher(pageData);
        Integer[] returnData = new Integer[3];
        if (salesforceLicenseMatcher.find()) {
            returnData[0] = Integer.valueOf(salesforceLicenseMatcher.group(1).replaceAll(',', ''));
            returnData[1] = Integer.valueOf(salesforceLicenseMatcher.group(2).replaceAll(',', ''));
            returnData[2] = Integer.valueOf(salesforceLicenseMatcher.group(3).replaceAll(',', ''));
        }
        return returnData;
    }

    public static Map<String, String[]> grabStorageUsageData(String hostURL, String sessionID, String orgID) {
        String pageContent;
        if (!Test.isRunningTest()) {
            pageContent = ClientOrgStatusWebservice.getPageContent(hostURL, sessionID, '/setup/org/orgstorageusage.jsp?id=' + orgID).replaceAll('&nbsp;', ' ');
        } else {
            pageContent = ClientOrgStatusWebservice.getTestStorageUsagePage();
        }
        String[] storageTypes = new String[] {'Data Storage', 'File Storage'};

        Map<String, String[]> clientStorageData = new Map<String, String[]>();
        for (String storageType : storageTypes) {
            clientStorageData.put(storageType, ClientOrgStatusWebservice.getDataForStorageType(storageType, pageContent));
        }

        return clientStorageData;
    }

    public static String[] getDataForStorageType(String storageType, String pageData) {
        Matcher storageTypeMatcher = Pattern.compile('<tr[\\sA-Za-z="(.){;}]*><th[\\sA-Za-z="]*>' + storageType + '</th><td[\\sA-Za-z="]*>([0-9\\sA-Za-z.]*)</td><td[\\sA-Za-z="]*>([\\s0-9.A-Za-z]*)</td><td[\\sA-Za-z="]*>([0-9%]*)</td></tr>').matcher(pageData);
        String[] returnData = new String[3];
        if (storageTypeMatcher.find()) {
            returnData[0] = storageTypeMatcher.group(1);
            returnData[1] = storageTypeMatcher.group(2);
            returnData[2] = storageTypeMatcher.group(3);
        }
        return returnData;
    }

    public static String getPageContent(String hostURL, String sessionID, String url) {
        String requestURL = hostURL + url;
        HttpRequest req = new HttpRequest();
        req.setEndpoint(requestURL);
        req.setMethod('GET');
        req.setHeader('Cookie', 'sid=' + sessionID);
        String resultBody;
        try {
            resultBody = new Http().send(req).getBody();
        } catch (Exception e) {
            System.debug('Exception occurred while attempting to load: ' + requestURL + ' | ' + e.getMessage() + ' | At Line: ' + e.getLineNumber());
        }
        return resultBody;
    }

    public class ClientInfo {
        public String orgID;
        public Map<String, Integer[]> licenseData;
        public Map<String, String[]> storageData;

        public ClientInfo(String orgID, Map<String, Integer[]> licenseData, Map<String, String[]> storageData) {
            this.orgID = orgID;
            this.licenseData = licenseData;
            this.storageData = storageData;
        }
    }

    public static String getTestLicensePage() {
        return '<tr class=" dataRow even first" onblur="if (window.hiOff){hiOff(this);}" onfocus="if (window.hiOn){hiOn(this);}" onmouseout="if (window.hiOff){hiOff(this);}" onmouseover="if (window.hiOn){hiOn(this);}"><th scope="row" class=" dataCell  ">Marketing User</th><td class=" dataCell  ">Active</td><td class=" dataCell  numericalColumn">2</td><td class=" dataCell  numericalColumn">1</td><td class=" dataCell  numericalColumn">1</td></tr><!-- ListRow --><tr class=" dataRow odd" onblur="if (window.hiOff){hiOff(this);}" onfocus="if (window.hiOn){hiOn(this);}" onmouseout="if (window.hiOff){hiOff(this);}" onmouseover="if (window.hiOn){hiOn(this);}"><th scope="row" class=" dataCell  ">Apex Mobile User</th><td class=" dataCell  ">Active</td><td class=" dataCell  numericalColumn">2</td><td class=" dataCell  numericalColumn">1</td><td class=" dataCell  numericalColumn">1</td></tr><!-- ListRow --><tr class=" dataRow even" onblur="if (window.hiOff){hiOff(this);}" onfocus="if (window.hiOn){hiOn(this);}" onmouseout="if (window.hiOff){hiOff(this);}" onmouseover="if (window.hiOn){hiOn(this);}"><th scope="row" class=" dataCell  ">Offline User</th><td class=" dataCell  ">Active</td><td class=" dataCell  numericalColumn">2</td><td class=" dataCell  numericalColumn">1</td><td class=" dataCell  numericalColumn">1</td></tr><!-- ListRow --><tr class=" dataRow odd" onblur="if (window.hiOff){hiOff(this);}" onfocus="if (window.hiOn){hiOn(this);}" onmouseout="if (window.hiOff){hiOff(this);}" onmouseover="if (window.hiOn){hiOn(this);}"><th scope="row" class=" dataCell  ">Knowledge User</th><td class=" dataCell  ">Active</td><td class=" dataCell  numericalColumn">2</td><td class=" dataCell  numericalColumn">0</td><td class=" dataCell  numericalColumn">2</td></tr><!-- ListRow --><tr class=" dataRow even" onblur="if (window.hiOff){hiOff(this);}" onfocus="if (window.hiOn){hiOn(this);}" onmouseout="if (window.hiOff){hiOff(this);}" onmouseover="if (window.hiOn){hiOn(this);}"><th scope="row" class=" dataCell  ">Force.com Flow User</th><td class=" dataCell  ">Active</td><td class=" dataCell  numericalColumn">3</td><td class=" dataCell  numericalColumn">0</td><td class=" dataCell  numericalColumn">3</td></tr><!-- ListRow --><tr class=" dataRow odd" onblur="if (window.hiOff){hiOff(this);}" onfocus="if (window.hiOn){hiOn(this);}" onmouseout="if (window.hiOff){hiOff(this);}" onmouseover="if (window.hiOn){hiOn(this);}"><th scope="row" class=" dataCell  ">Service Cloud User</th><td class=" dataCell  ">Active</td><td class=" dataCell  numericalColumn">2</td><td class=" dataCell  numericalColumn">1</td><td class=" dataCell  numericalColumn">1</td></tr><!-- ListRow --><tr class=" dataRow even" onblur="if (window.hiOff){hiOff(this);}" onfocus="if (window.hiOn){hiOn(this);}" onmouseout="if (window.hiOff){hiOff(this);}" onmouseover="if (window.hiOn){hiOn(this);}"><th scope="row" class=" dataCell  ">Live Agent User</th><td class=" dataCell  ">Active</td><td class=" dataCell  numericalColumn">2</td><td class=" dataCell  numericalColumn">0</td><td class=" dataCell  numericalColumn">2</td></tr><!-- ListRow --><tr class=" dataRow odd" onblur="if (window.hiOff){hiOff(this);}" onfocus="if (window.hiOn){hiOn(this);}" onmouseout="if (window.hiOff){hiOff(this);}" onmouseover="if (window.hiOn){hiOn(this);}"><th scope="row" class=" dataCell  ">Site.com Contributor User</th><td class=" dataCell  ">Active</td><td class=" dataCell  numericalColumn">1</td><td class=" dataCell  numericalColumn">0</td><td class=" dataCell  numericalColumn">1</td></tr><!-- ListRow --><tr class=" dataRow even" onblur="if (window.hiOff){hiOff(this);}" onfocus="if (window.hiOn){hiOn(this);}" onmouseout="if (window.hiOff){hiOff(this);}" onmouseover="if (window.hiOn){hiOn(this);}"><th scope="row" class=" dataCell  ">Site.com Publisher User</th><td class=" dataCell  ">Active</td><td class=" dataCell  numericalColumn">2</td><td class=" dataCell  numericalColumn">0</td><td class=" dataCell  numericalColumn">2</td></tr><!-- ListRow --><tr class=" dataRow odd" onblur="if (window.hiOff){hiOff(this);}" onfocus="if (window.hiOn){hiOn(this);}" onmouseout="if (window.hiOff){hiOff(this);}" onmouseover="if (window.hiOn){hiOn(this);}"><th scope="row" class=" dataCell  ">Chatter Answers User</th><td class=" dataCell  ">Active</td><td class=" dataCell  numericalColumn">25</td><td class=" dataCell  numericalColumn">0</td><td class=" dataCell  numericalColumn">25</td></tr><!-- ListRow --><tr class=" dataRow even last" onblur="if (window.hiOff){hiOff(this);}" onfocus="if (window.hiOn){hiOn(this);}" onmouseout="if (window.hiOff){hiOff(this);}" onmouseover="if (window.hiOn){hiOn(this);}"><th scope="row" class=" dataCell  ">Salesforce CRM Content User</th><td class=" dataCell  ">Active</td><td class=" dataCell  numericalColumn">2</td><td class=" dataCell  numericalColumn">0</td><td class=" dataCell  numericalColumn">2</td></tr>';
    }

    public static String getTestStorageUsagePage() {
        return '<tr class=" dataRow even first" onblur="if (window.hiOff){hiOff(this);}" onfocus="if (window.hiOn){hiOn(this);}" onmouseout="if (window.hiOff){hiOff(this);}" onmouseover="if (window.hiOn){hiOn(this);}"><th scope="row" class=" dataCell  ">Data Storage</th><td class=" dataCell  numericalColumn">250.0&nbsp;MB</td><td class=" dataCell  numericalColumn">1.3&nbsp;MB</td><td class=" dataCell  numericalColumn">1%</td></tr><!-- ListRow --><tr class=" dataRow odd last" onblur="if (window.hiOff){hiOff(this);}" onfocus="if (window.hiOn){hiOn(this);}" onmouseout="if (window.hiOff){hiOff(this);}" onmouseover="if (window.hiOn){hiOn(this);}"><th scope="row" class=" dataCell  ">File Storage</th><td class=" dataCell  numericalColumn">250.0&nbsp;MB</td><td class=" dataCell  numericalColumn">110&nbsp;KB</td><td class=" dataCell  numericalColumn">0%</td></tr>';
    }
}



ClientOrgStatusWebservice_Schedulable Apex Class:

global class ClientOrgStatusWebservice_Schedulable implements Schedulable {

    global String hostURL {get; set;}
    global String sessionID {get; set;}
    global String orgID {get; set;}

    global ClientOrgStatusWebservice_Schedulable() {
        hostURL = URL.getSalesforceBaseUrl().toExternalForm();
        sessionID = UserInfo.getSessionId();
        orgID = UserInfo.getOrganizationId();
    }

    global void execute(SchedulableContext sc) {
        ClientOrgStatusWebservice.notifyCRMFuture(hostURL, sessionID, orgID);
    }
}



Essentially, this just loads up those pages, scrapes the usage info for a set of license types as well as the data/file storage for the org, puts that into an instance of the ClientOrgStatusWebservice.ClientInfo internal class, serializes it as JSON and send it to our org. Our org then deserializes it and updates the records related to the org it was sent from (based on the org ID) with that usage info.

I appreciate any insight that anyone can give.

3

Instead of using PageReference.getContent can you use an HttpRequest to achieve the same?

Something like:

HttpRequest request = new HttpRequest();
request.setMethod('GET');
request.setEndpoint(Url.getSalesforceBaseUrl().toExternalForm() + '/home/home.jsp';
request.setHeader('Cookie', 'sid=' + UserInfo.getSessionId());

HttpResponse response = new Http().send(request);
response.getBody(); //...scrape your values here

Then annotate your Database.Batchable implementation with Database.AllowsCallouts also.

  • Thanks for the response. I am currently sending an HttpRequest in order to grab these pages and it works perfectly as long as the scheduled batch was scheduled less than 2 hours before it runs because I can only get at the session ID when the Scheduled batch class is first constructed, which seems to be when it is scheduled. I am not currently utilizing a regular Apex Batch, but I am using a Scheduled batch and an @future annotated method instead, so there isn't anything that can effectively implement the Database.AllowsCallouts subclass. – jsnbdly Feb 11 '14 at 20:40
  • Ran out of characters. The issue seems to be that as soon as I attempt to schedule this process, I lose the user's session and thus cannot load internal pages. It looks like scheduled batches are executed under some sort of system context. – jsnbdly Feb 11 '14 at 20:43
  • I see, UserInfo.getSessionId will return null in Batch, Future, and Schedulable contexts. Can you parameterize your @future(callout=true) static void doScrape(String sessionId) and pass it in? – bigassforce Feb 11 '14 at 20:54
  • Yes, as per the code I left, the current method signature for the notifyRKCRMFuture method is: @future(callout = true) public static void notifyRKCRMFuture(String hostURL, String sessionID, String orgID) This method may not be called for hours after the scheduled batch is actually scheduled. As the session ID expires after a certain period of time (2 hours default), it may not longer be valid by the time the notifyRKCRMFuture method fires. – jsnbdly Feb 11 '14 at 20:56
  • ah, pardon me I see now @jsnbdly – bigassforce Feb 11 '14 at 20:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.