4

I've found that certain string comparisons in Apex consume far more CPU time that would naturally be expected - specifically when at least one of the strings is very long ('long' here means over 100000 characters, but the effect may be significant for shorter strings).

A sample script which illustrates the issue is as follows:

List<Account> acs = [select Id, Name, BillingStreet, BillingCity, BillingPostalCode, BillingState, BillingCountry, BillingLatitude, BillingLongitude
                        from Account
                        where BillingLatitude != null
                        limit 1000];

final String COMP_STR = 'Test-Test';
final Integer COMP_STR_LEN = COMP_STR.length();

String str = '';
Integer mcount = 0;
CpuTimer t1 = new CpuTimer();
for (Account ac : acs) {
    str += '\"' + ac.BillingStreet + '\",\"' +
            ac.BillingCity + '\",\"' +
            ac.BillingPostalCode + '\",\"' +
            ac.BillingState + '\",\"' +
            ac.BillingCountry + '\",' +
            ac.BillingLatitude + ',' +
            ac.BillingLongitude + '\n';
    if (str == COMP_STR) {
        ++mcount;
    }
}
t1.showCpuTime('CPU time: ');

public class CpuTimer {

    Integer cpuTime;

    public CpuTimer() {
        cpuTime = Limits.getCpuTime();
    }

    public void showCpuTime(String txt) {
        Integer newCpuTime = Limits.getCpuTime();
        System.debug(txt + (newCpuTime - cpuTime));
        cpuTime = newCpuTime;
    }

}

Running the above script in a Developer Org using 'Execute Anonymous' showed results of 9098, 8833, 9189, 7224, 9335 and 6787 milliseconds, i.e. between 6.5 and 9.5 seconds with an average of 8.4 seconds.

With a very minor change, checking the length of the string before making the comparison:

if (str.length() == COMP_STR_LEN && str == COMP_STR) {

the results are very different: 86, 63, 96, 106, 90 and 104 milliseconds, i.e. an average of 0.091 seconds.

Is the String comparison, indeed, the reason for the big difference in consumed CPU time and, if so, is the 'fix' of checking the length first the best way to mitigate it?

I have also found that the results seem to depend on how the long string value is built - using an equally long string built by a single statement (using the Apex repeat() method) results in much faster comparisons that a string built by many concatenations. Does this possibly give clues about what the String class and the comparison operation are doing internally?

Also, is there any truth in the theory I heard: that the String comparison is doing this to prevent 'timing attacks', i.e. to prevent an attacker from determining the length of a String value by sending various length strings and noting which of these gives a significantly better response time?

5
  • 1
    In running the code you've provided myself, I'm not observing the same CPU time usage that you're seeing (even with keeping the bug sfdcfox pointed out). My runs only take around 100-150 CPU time units. I do see some difference between comparing string size first and just string comparison, but the difference is about 2x rather than the 100x that you're reporting.
    – Derek F
    Apr 27, 2022 at 14:01
  • From my tests the result appears to depend on the specific content of the strings and (quite likely, although I haven't yet definitively verified this) the fact that out data contains specific German characters such as 'Ä', 'Ö', 'Ü' and 'ß' - these are German Account addresses. In your case, did the Account query return 1000 rows and were the field values pure-ASCII or more 'exotic'? I have an example which generates random strings rather than pulling from the Account object (obviously Org dependant) so I'll post that later on...
    – John Lewis
    Apr 27, 2022 at 14:13
  • 1
    Mine are all within the ASCII range. Salesforce uses UTF-8 everywhere though, and the non-ASCII characters I'd expect you to encounter should all be in the Latin-1 Supplement block (meaning they all take a single byte).
    – Derek F
    Apr 27, 2022 at 14:22
  • I have a version that generates its own 'random' data using the following method: final String CHARS = 'ABCDEFGHIJKLMNOPQRSTUVWXYZÄÖÜßabcdefghijklmnopqrestuvwxzyäöü'; String randomString(Integer len) { str = ''; for (Integer i = 0; i < len; ++i) { Integer r = Integer.valueOf(Math.random() * CHARS.length()); str += CHARS.substring(r, r+1); } return str; } I had to reduce the number of Accounts to 500 because it otherwise breaks the governor limit but is essentially gives the same result.
    – John Lewis
    Apr 27, 2022 at 15:40
  • significantly, I think, when I repeat tests with the non-ASCII characters 'ÄÖÜß' and 'äöü' removed from the 'CHARS' string, the results are different. The version that compares string without checking the length still uses more CPU time, but the difference is a factor of 4 rather than a factor of 100 as I was finding before.
    – John Lewis
    Apr 27, 2022 at 15:47

3 Answers 3

6

Note that part of your problem is that you're creating a very large string. Your code also consequently has a logic bug. You never clear out the str variable, so it grows in size, and that growth is causing the problems at hand. You only need to make one small change to your code:

str = '\"' + ac.BillingStreet + '\",\"' +

By not concatenating the very large string repeatedly, you'll increase your performance significantly.

As Phil said, however, comparison using native objects is on the order of 10x faster, at minimum. For a specific search like this, I'd consider using an Account object as the search term:

final Account searchAccount = new Account(Name='demo',BillingCity='Denver');
Integer mcount = 0;
CpuTimer t1 = new CpuTimer();
for (Account ac : acs) {
    // I'm lazy, not going to write out all the fields...
    if(new Account(Name=ac.Name, BillingCity=ac.BillingCity) == searchAccount) {
        mcount++;
    }
}

Comparison between sObjects is incredibly fast, and the intent of the code becomes clear.

Using a native object comparison, either a List<Object> or Account has another benefit: it's impossible to accidentally have two pieces of data that are different, but string concatenation won't necessarily catch. The extra quotes and commas are a nice touch, but if any of the fields also have quotes or commas, it might be possible to construct a situation where the two strings match despite having different content.

Edit: However, one benefit of the string concatenation is that the comparison will be case-insensitive. If you need that feature, concatenation will likely be faster than calling toLowerCase() on each individual field. That's usually pretty rare if you have good data quality, but a consideration to think about.

Edit #2: Doing more research on this, I now understand the problem, and have an alternative solution.

Basically, == calls String.equalsIgnoreCase, which means that the system has to convert each character to a common value for comparison. This cost goes up as the string length increases. To combat this, you should manually convert the string to a consistent case yourself, and use String.equals instead.


List<Account> acs = [select Id, Name, BillingStreet, BillingCity, BillingPostalCode, BillingState, BillingCountry, BillingLatitude, BillingLongitude
                        from Account
                        where BillingLatitude != null
                        limit 1000];

final String COMP_STR = 'Test-Test'.toLowerCase();
final Integer COMP_STR_LEN = COMP_STR.length();

String str = '';
Integer mcount = 0;
CpuTimer t1 = new CpuTimer();
for (Account ac : acs) {
    String temp = '\"' + ac.BillingStreet + '\",\"' +
            ac.BillingCity + '\",\"' +
            ac.BillingPostalCode + '\",\"' +
            ac.BillingState + '\",\"' +
            ac.BillingCountry + '\",' +
            ac.BillingLatitude + ',' +
            ac.BillingLongitude + '\n';
    temp = temp.toLowerCase();
    str += temp;
    if (str.equals(COMP_STR)) {
        ++mcount;
    }
}
t1.showCpuTime('CPU time: ');

public class CpuTimer {

    Integer cpuTime;

    public CpuTimer() {
        cpuTime = Limits.getCpuTime();
    }

    public void showCpuTime(String txt) {
        Integer newCpuTime = Limits.getCpuTime();
        System.debug(txt + (newCpuTime - cpuTime));
        cpuTime = newCpuTime;
    }

}

Once you do this, you should get a massive boost in performance, even as the string approaches hundreds of thousands of characters.


Edit #3:

Also, is there any truth in the theory I heard: that the String comparison is doing this to prevent 'timing attacks', i.e. to prevent an attacker from determining the length of a String value by sending various length strings and noting which of these gives a significantly better response time?

I would think there's too many other variables in play to use timing attacks consistently, except for large strings. Imagine a fail-fast implementation where you were guessing a long (100,000) character password. Without fail-fast, the timing would be more or less consistent in CPU time. With fail-fast, the timing would take longer the more correct characters you have.

However, without sub-millisecond accuracy, it would be difficult to ascertain if you were making progress or not, and network latency alone would kill that level of sensitivity. If someone were really worried about timing attacks, they could always provide a random delay provided by a call to the Crypto random number generator to provide a consistent delay, or purposefully wait out a 10 second CPU use time.

However, that's more speculation. I'll check with someone and see if that is intentional or not. I'll amend this answer if I find out more.

10
  • 1
    I missed that string concatenation bug. Nice spot. Yes, using an SObject is equally effective compared to the Object[], just so long as you ensure you query exactly the same set of fields as you populate in the "searchAccount" (though your code avoids this issue nicely by actually creating a temporary account from the queried one).
    – Phil W
    Apr 27, 2022 at 12:55
  • 1
    @PhilW Indeed. Though, I did find an argument for when string concatenation might be acceptable... it's just pretty rare to need in practice, at least as far as I've seen.
    – sfdcfox
    Apr 27, 2022 at 12:59
  • 1
    @JohnLewis I didn't realize the question was a hypothetical. I do see the problem, and I have a new solution to the problem. You are correct, in a sense, that the logic in String.equalsIgnoreCase isn't fail-fast. I'm going to go log a bug. In the interim, manual conversion and String.equals can drastically improve performance. It's not the concatenation that kills, it's the String.equalsIgnoreCase.
    – sfdcfox
    Apr 27, 2022 at 16:02
  • 1
    @JohnLewis I have some theories. I'm going to work with the higherups to identify the problem and try to come up with a Known Issue/bug for this. I find that this behavior is definitely unacceptable.
    – sfdcfox
    Apr 28, 2022 at 4:13
  • 1
    @JohnLewis Apex runs on Java. I forget the specific docs, but you can find this out if you're looking for it. Anyways, I wrote unit tests to prove this behavior. It's just a matter of getting internal confirmation of the behavior. Let's just say there's a reason I'm MVP 😅.
    – sfdcfox
    Apr 28, 2022 at 4:24
5

I would say that it is the string concatenation that is causing the high CPU usage rather than the comparison itself - that's certainly our experience.

I would always recommend using an object array for such comparisons (when you are either testing equality or using the value as a key in a map or set). This is generally far faster because you avoid the significant overheads of string concatenation and the related memory management activities (and, indeed, the need to convert values of a non-string type to string).

For example:

Object[] compAddress = ...;
Integer mcount = 0;

for (Account ac : acs) {
    Object[] accountAddress = new Object[] {
        ac.BillingStreet,
        ac.BillingCity,
        ac.BillingPostalCode,
        ac.BillingState,
        ac.BillingCountry,
        ac.BillingLatitude,
        ac.BillingLongitude
    };

    if (accountAddress == compAddress) {
        mcount++;
    }
}

The overhead of constructing the object array is far smaller than string concatenation.

I cannot say that the theory you mentioned makes much sense in an Apex context anyway since it runs on multitenant infrastructure so timing will vary significantly with the same input data.

4
  • 1
    This explanation (concatentation overhead) doesn't seem to jive with the 2 order of magnitude difference in cpu time reported when the size check short-circuits the if() though.
    – Derek F
    Apr 27, 2022 at 12:49
  • True, though I must say that switching from string concatenations to object arrays has massively improved our product code performance. I guess there could be another element at play here.
    – Phil W
    Apr 27, 2022 at 12:50
  • And there you have it; The Fox spotted a bug I missed ;)
    – Phil W
    Apr 27, 2022 at 12:59
  • this is essentially what I thought myself when I began investigating this issue - the actual code is far more complex and too Org-specific to post here, but by measuring specific statements, I did surprisingly find that the string comparison (== operator which, as I understand it, is actually using the equalsIgnoreCase() method) was eating most of the CPU time once the string got very large. I'll see if I can construct a more specific example which illustrates that...
    – John Lewis
    Apr 27, 2022 at 14:16
1

There's some insight here into why comparing the string lengths is probably NOT a 100% reliable way of improving these case-insensitive string comparisons. Try the following script:

String s1 = 'Strasse';
String s2 = 'Straße';
System.debug(s1.length());
System.debug(s2.length());
System.debug((s1 == s2));

Here the German 'ß' (Eszett) character is considered 'equal' to the pair of characters 'ss' so even though the lengths of the two strings are different the strings themselves are deemed 'equal' (or at least 'equalIgnoreCase').

@sfdcfox might this give a reason why the case insensitive string comparison can't simply compare the first X characters of the strings where X is the shorter of the two lengths?

Are there any languages other than German where this 'variable-length-equality' works?

Oddly, I find that strings "Straße" and "Strasse" are NOT equal according to Java's equalsIgnoreCase().

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .