A Test Data Factory is used to create all your records you need for your tests in one place. This makes it easy to fix your unit tests when a validation rule or new required field is added. It also arguably reduces the complexity of your code, since you're now effectively using a factory pattern to create all of your data, which reduces the amount of code you're repeating (the DRY principle).
A @testSetup method, in contrast, sets up a miniature mock database for all unit tests in the same class to use. This can significantly reduce test runtime by setting up common data for all tests to use. In addition, if done correctly, it can reduce governor limits usage in all of your unit tests, meaning you can get more done with fewer tests.