Any resources on what might've changed in API 37 that wasn't present in 36 that could be causing the issue I'm seeing?
If I update every other apex class in my org to 46 and leave the test class at 36, the test passes. If I bump the test class to 37, the test class fails.
We're essentially testing our email services apex class that checks for certain domains of where the email is coming from against a distributor email list (which is in a custom setting) and then forwards that email to the distributor and doesn't save the email in SF and doesn't create a case.
At the end of test class we query cases, emails, and tasks to confirm there's 0 records as the passed in email and envelope to the apex class should match a distributor email we created and we don't save the email message (just forward it along).
The test class, however finds email message records (but finds no case or task) which fails the assert.
What I found to be the issue, is in how we're forwarding the email to the distributor in the main code. We're not setting .setSaveAsActivity at all (so the default is true) and it seems like it now saves as an emailMessage record in API 37 and beyond whereas it did not in 36. I'm just trying to figure out if there's any documentation noting this? This only applies when messing with the API of the test class instead of the API of the main code which is part of my confusion as well.
If I add setSaveAsActivity(false) to the code snippet that forwards the email to the distributor, the test passes and no emailMessages are created in Salesforce.
system.assertEquals(0, ...)
, none of which include a message. Given that, your assertion failure message on its own is not sufficient to determine which line causes the failure. Please edit your post to be more explicit.