If you don't catch exceptions or handle exceptions and use DML statements without partial DML like database.update your whole transaction by default will roll back when there is failure.
If you want to roll back previous batches,keep a flag or an identifier and in catch block just reverse the operation.
Update
An example could be to update the flag.In the finish method you can uncheck the flag to unmark all.