4

I have an understanding that Salesforce itself does its own sets of internal Performance Testing. However, there are companies who have a mandate to perform Performance Testing as one of their tasks/ milestones/protocols.

Can someone who had to do Performance Testing in their implementation please share their experiences, dos and donts, suggestions/recommendations, best practices, etc.? What are the parameters to be considered, how many users are sufficient enough, during what times it should be tested, etc.?

8

The number one rule here is do not, under any circumstances, perform load testing of any salesforce.com service or feature without formal consent. They may revoke your access and/or charge you service fees for the increased usage. Their own internal testing is sufficient to prove that the system is stable and working at optimal levels. In fact, salesforce Trust contains all of the relevant stats that you'd probably want to know, including number of transactions daily and average server response time. Outages and performance degradations are also reported here.

That being said, submit a case if you want to perform load testing to test the speed of your application using an automated tool. They will negotiate a set of parameters you can use (including number of simultaneous connections, number of users, number of tests, duration, etc). This can allow an organization to get a feel for the "average load" of the system. It's important to note that during peak load usage on a given server, all users are affected equally. For this reason, if your app is somehow performing slowly from salesforce.com's hardware, an alarm will have already gone off since a large number of clients would also be affected (at least, in most cases).

A better test is to test your network's performance. Run a bandwidth simulation test on your firewalls, routers, and other corporate infrastructure to make sure it can handle the load. This is more significant for a given organization's performance than testing the salesforce.com hardware, which is monitored and tested regularly.

You should note that salesforce.com handles nearly 1,000,000,000 transactions every weekday, or about 11,000 transactions per second. This means that the system is proven capable of handling all amounts of normal server transactions. However, if they allowed performance testing without scheduling them, a large number of entities performing testing all at once could cause a DDOS-style attack. This is why coordination is paramount, and those that violate it will be sanctioned.

An IT department that insists on testing will go through the appropriate channels. A smarter IT department knows that a proven reliable system that is already constantly monitored and tested doesn't need the external testing. This is the future of the cloud; managed hardware that performs well without needing babysitting by each individual organization using the service. This is a stark contrast to traditional corporate resources, where the IT department has to monitor and repair any outages themselves. This is a feature of salesforce.com (and other systems like Azure, S3, and so on).

| improve this answer | |
1

Salesforce removed their Number of Script Statements Limit in Winter '14 (API v 29.0) in favor of a different Governor Limit: CpuTime.

This is a surprising move since the two types of governor limits have different implications on code design. Therefore, I became naturally curious to determine if any coding best practices have changed (and how to measure the changes). The questions really are:

What can break my code?

What Governor Limits may I face? How do I deal with them?

What are the limitations of the tools(/code) I create? What are there purpose? Can any methods be generalized? Sure, it gets the job done, but does my tool force a user to get the job done right? (UI look & feel -- does it guide the user to correctly use the tool?, error handling, etc.)

I do not want any user to see a Salesforce Error Message (because he probably will just sit there stuck instead of finding help). Is it possible that my code may generate any Salesforce Error Message (like by running into a Governor Limit)? What can I do to avoid this?

Let me give an example:

Example:

I created a class called Csv to parse strings or blobs (that have a csv-format) and facilitate mapping columns (via the Header's Field Names) easily. The two performance concerns I have deal with using string.replaceAll (so using a regex match) as well as how much of my "resources" am I using up to traverse and parse the Csv. Why? I want to estimate how "big" a file I can parse successfully (with the new Governor Limit in mind). (And what is too "big"? and if I vary the #rows and #columns? ...?). So, I wrote up this little script:

integer numberOfTests=20;

date d1=date.newInstance(2013,09,30);
date d2=date.newInstance(2013,10,6);
Swipeclock s=new Swipeclock('2037');   //<---connecting to a 3rd-party API
string file=s.getActivityFile(d1,d2);  //<---getting a csv to parse
list<list<string>> result=new list<list<string>>();

list<integer> timeIn=new list<integer>();   
list<integer> timeOut=new list<integer>();  

for(integer i=0;i<numberOfTests;i++){
    timeIn.add(limits.getCpuTime());
    result=Csv.parseCsv(file,Csv.LineDelimiter.LF);  //<---TESTING PERFORMANCE FOR THIS METHOD
    timeOut.add(limits.getCpuTime());
}

system.debug('------- ParseCsv Results --------');      //<--- Print the results
integer totalTime=0;
for(integer i=0;i<n;i++){
    integer t=timeOut[i]-timeIn[i];
    totalTime+=t;
system.debug('....#'+string.valueof(i+1)+' - Time Taken = '+string.valueof(t)+' ms');
}
decimal avg=decimal.valueof(totalTime)/decimal.valueof(n);
avg=avg.setScale(2);

system.debug('................................');
system.debug('....AVERAGE TIME = '+string.valueof(avg)+' ms');

And now that the above code is written, I can start to play with my parameters to see how my method performs. :)

| improve this answer | |
  • 1
    Salesforce.com states that over 99% of existing code will work under the new governor limit, and those that are pushing the limit will be contacted beforehand to help them speed up the code. In other words, unless salesforce.com noticed you have a problem, you're probably okay. – sfdcfox Oct 8 '13 at 19:05
  • I believe that too. Plus, I've witnessed that SF allows you to push the CpuLimit by 50-100% (i.e. the "actual" limit is more like 15-20 sec instead of the stated 10 sec limit). – Scott Pelak Oct 8 '13 at 19:27
  • It just seems tricky to estimate how much Cpu my code will use under different data conditions. – Scott Pelak Oct 8 '13 at 19:31
  • 2
    I ran a simple loop to test the new limits, I was able to execute 1,506,751 lines of execution before I hit the limit. Of course, it was a simple loop with minimal work, but I think it's safe to say that most types of code should probably get up to 1,000,000 lines of execution compared to the old 200,000. – sfdcfox Oct 8 '13 at 19:58
  • 1
    Sorry, I meant to say 1,506,751 loop iterations, so actually 3,013,503 total lines of executed code before it failed. – sfdcfox Oct 8 '13 at 20:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.