One possible solution I can think of is a staging object as an intermediate step and chained batch executions.
Let's say you're doing something with Accounts. First create a custom object called Account_Batch_Staging__c. Make it an exact duplicate of the account object, including as much validation of the original object as possible (with some standard objects this is difficult, so you might need an Apex trigger to accomplish this completely). Include an additional fields: either a lookup to store the real Account Id, or perhaps an External Id field to do the same.
Stage 1: Execute batch logic while moving data from Account to Account Batch Staging
Stage 2: Evaluate for your defined error condition
Stage 3: If not errors, enqueue a second batch in the finish of the first moving data from Account Staging back to Account.
If there are errors, you could either abort, clean up the staging table with a different batch, or perhaps retry.
It isn't fool proof, but it does give you a safe landing at the half-way point that is much more under your control to roll-back. Oh...and if you're short on custom objects, this does, of course, burn a custom object.