Using System.runAs in all tests - Best practice or Antipattern?

Question

When doing functional tests I see many people also check the permissions by using System.runAs(userWithRightPermission) in most of their tests.

I see good reasons for doing this, because then you actually test that only allowed user can use a certain feature. And with permissions being a declarative feature, developer tend to forget putting classes and pages into perm sets sometimes.

But it still feels wrong to spam every single test with User and Permission creation. My opinion until now was that on a few tests which actually proof that permissions are applied should use System.runAs() while the others use the default admin. As permissions an access are a declarative feature there actually should not be tests. Or do you write Apex tests to check Validation Rules?

Is there a documented Best Practise or some smart experts opinions on this out there?

Answer 1

For ISV/managed package development, you don't have control over the security model in the subscriber org so it's imperative that you ship Permission Sets or Permission Set Groups in your package to simplify the installation and provisioning of your package (and to provide you with a path to upgrade your app without requiring manual permission changes upon upgrade).

Profiles are a disaster and you should never rely on the "Grant access to All Users" during installation.

As such, you need to test that your packaged Perm Sets are sufficient to let the user access your application logic without barrier. IMO, the only way to do that is with a rich set of Apex test methods that model the expected user scenarios with System.runAs()

As was stated in the original question, it's not a requirement that 100% of tests use System.runAs(), just enough test coverage to ensure that your intended user stories can be addressed with your Perm Sets.

For completeness, it should be stated that packaged Perm Sets can't grant System Perms or Standard Object perms so you may still need to create Perm Sets on the fly using Apex in your Test Setup method to grant those permissions.

Lastly, with regard to 2GP: It wasn't until recently (Summer '21) that you could specify the Permission Sets that the Admin was assigned in the build org (the user running the package version creation) so it was basically a 100% requirement to use System.runAs() since the running user was very limited and any CRUD/FLS checks generally returned false. The apexTestAccess property in sfdx-project.json supposedly provides an alternative to System.runAs()

https://help.salesforce.com/s/articleView?id=release-notes.rn_sfdx_packaging_apex_test_access.htm&type=5&release=232

Additionally, for 2GP, the Unpackaged Metadata feature is also helpful in granting permissions via Perm Sets that you don't plan to package (i.e. permissions that you expect the Admin to manually grant in the subscriber org)

https://developer.salesforce.com/docs/atlas.en-us.sfdx_dev.meta/sfdx_dev/sfdx_dev_dev2gp_unpackaged_md.htm

Answer 2

Having all/most unit tests runAs users is usually overkill in most organizations. Permission requirements frequently change as roles, profiles, and permission sets are added and updated, so having lots of tests that rely on this may result in needing to deploy code changes to apply security permissions. This can quickly become unmanageable.

I've always maintained that there are some types of unit tests that should not be written. For example, you could write a unit test to validate that one plus one equals two, or you could trust that math won't somehow stop working in the foreseeable future.

Making certain assumptions about the platform allows you to target functionality that's custom, and trust that Salesforce will obey its own rules for things like validation rules and security.

SInce it's more likely that permissions will be forgotten instead of added improperly, it makes more sense to just deploy features and trust that users will report permissions errors, which you can be quickly fixed by administrators if necessary.

In a final point, if you bake security checks into your unit tests, then a forgotten permission may mean needing to update many tests and then deploy (which can take an hour or more in large orgs) to fix a critical production issue.

That's not to say that permissions shouldn't be validated, but this should be done as part of QA/admin review before deployment. If you're using source-driven deployment (Unlocked Packages, etc), then permissions should be correct if pre-validated in the QA/UAT phase of testing before reaching production.