I don't quite know how to explain the behavior we're seeing. I have two test methods that test exactly the same thing (same inputs, same assertions). The only difference is that one schedules the job, the other executes it directly. The former is just to cover the schedulable code.
These tests have been in our org and passing for 3 years, unchanged. With Summer 22, suddenly, the schedulable test is failing with:
System.CalloutException: You have uncommitted work pending. Please commit or rollback before calling out
...but the directly executed one is passing. I found this StackExchange post from 2019 with the accepted answer being that "Scheduling the job counts as pending work for the purpose of callouts". If that's true...then fine I guess, I'll just not execute the callout and I'll make my assertions a lot simpler without actually testing the outcome of the code beyond "did it run."
But...why would this test have been passing fine until now? I can't find anything in the Summer 22 release notes that explicitly explain it. Though I do see this item regarding resource usage?