In our system, we have recently added a validation rule, which works fine with the actual usage but most of our existing test code doesn't comply with that rule when creating test data. Since test.isRunningTest() does not work with validation rule, is it possible to auto-deactivate it when running tests?
It's impossible to just run some code to bypass validation rules. However, there are techniques for validation rules that are under your control. One such article is here (not mine). The general technique is to create a custom setting, and use that custom setting to bypass validation rules by building it into the validation rules:
Of course, this won't help you if you're in a managed package and you're trying to worry about someone else's validation rules, but this method can allow in-house code to bypass validation rules with very little extra effort.