For ISVs to be safe to (re)submit their app for Security Review without issues Checkmarx still is the only authoritative tool. But it is also cumbersome and limited as:
- Community Scanner is limited in code size
- Partner Scanner only works on packaged code
- Both free Scanners can not be rerun often
- Commercial Checkmarx seems to be super expensive
- Scans are not finding a lot. Few False Positives but a ton of False Negatives
- Scans are not complete and reproducible. You fix all issues, rerun, and get new issues.
- Does not seem not to pick up many Frontend code issues
I'm very aware of the alternatives Clayton, CodeScan as well as PMD, ESlint and CLI Scanner but they are also in a way not optimal:
- PMD has no Data Flow and is Single File only, so most security checks are vague
- ESlint: not sure if that is even relevant for SecReview.
- SFDX scanner is just using PMD, Eslint and slowly adding a few Data Flow rules with its new Graph Engine extension
- Clayton is focusing on Security issues only and has very few False Positives and find many more issues that Checkmarx and real Security Reviews
- Did not evaluate CodeScan during Security Reviews but I would assume it focused more on many rules than a few good ones. Most rules are not security related and the fact that they offer to run them fast in the IDE seems to prove they only do static analysis.