7

We have an SFDX build set up that runs our 2,800 tests against 6 different scratch org configurations in parallel. But every build fails on what seems like a random one of the scratch orgs (i.e. it isn't the configuration variations that are at play here) with timings like these:

  • the tests take 2h 19min instead of 1h 24min (ballpark for the other 5) i.e. x1.65 slower
  • it's not just the tests e.g. the push takes 12min 7s instead of 7min 36s (ballpark for the other 5) i.e. x1.60 slower

The failure problem is that there are some unit tests that confirm that some complex algorithms haven't had their performance degraded by code changes by running extreme cases that are about 20% inside of the "Apex CPU time limit".

Any suggestions on how to always get given "good orgs"? Are we being throttled in some way here?

I can't think of a way to modify the performance tests to workaround this. Can you?

PS

On the workaround part...

I'm looking to ensure that an existing algorithm isn't made slower by code changes so that existing customers are not broken for existing data. My problem here is that orgs are sometimes taking x1.6 of the time. But if that slow down is fairly independent of the specific code, then I could have say a 100ms of CPU time piece of code to calibarate - that would take 160ms on the "bad org" - and use that result to determine the repeat count on the code I do care about.

Dodgy territory though.

PPS

Running the test that exceeds the 10.0s CPU limit in the "bad org" in my development scratch org, the CPU time used is 1.2s. So a lot more than x1.6 happening in CPU usage for "bad orgs".

3

Please open up a Support Case for this issue. We'd like to get your Dev Hub Org ID so that we can investigate the scratch orgs you're referring to. If you can also send me the case number on twitter @rohitforce or ping me on the Salesforce DX community https://success.salesforce.com/_ui/core/chatter/groups/GroupProfilePage?g=0F93A000000HTp1SAG, I can expedite the investigation.

PS

An update from the case:

We just have received an update from our R&D team that, we have identified the difference between instance CS50 and CS70 so we are trying further to provide a fix for this kind of issues.

  • Thanks Rohit. Our Christmas break has just started, so it will be the beginning of January before I can do this; I'll be in touch then. – Keith C Dec 22 '18 at 12:16
  • I created the case today with some example times and org ids. The case is 21170949. – Keith C Dec 27 '18 at 11:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.