How can I work with both QueryLocator and List<SObject> in a single batch class (not a single instance)?

Question

I've been working on a chainable batch class. The constructor can take in either a service & query or a service & list of records. In almost all cases, the service/query are passed in. In these cases, I'm expecting the query to return a large number of records. As such, I need to use QueryLocator. There is currently only one situation in which I need to pass in a service/list, so I'd prefer to not have to make a whole new class just to handle it.

public class ChainableBatchManager implements Database.Batchable<SObject>, Database.Stateful, Database.AllowsCallouts {
    private final Integer batchesOf20 = 20;
    private ChainableBatch[] batchServices = new ChainableBatch[]{};

    public ChainableBatchManager() {}

    private ChainableBatchManager(ChainableBatchManager previousBatchManager) {
        this.batchServices = previousBatchManager.batchServices;
    }

    public ChainableBatchManager add(IProcessRecords recordService, String query) {
        ChainableBatch batchToAdd = new ChainableBatch(recordService, query);
        this.batchServices.add(batchToAdd);
        return this;
    }

    public ChainableBatchManager add(IProcessRecords recordService, SObject[] records) {
        ChainableBatch batchToAdd = new ChainableBatch(recordService, records);
        this.batchServices.add(batchToAdd);
        return this;
    }

    public Iterable<SObject> start(Database.BatchableContext context) {
        return batchServices[0].getRecords();
    }

    public void execute(Database.BatchableContext context, SObject[] scope) {
        try {
            batchServices[0].processRecords(scope);
        } catch(Exception ex) {
            // Error handling here
        }
    }

    public void finish(Database.BatchableContext context) {
        batchServices.remove(0);
        if (this.batchServices.isEmpty()) return;
        Database.executeBatch(new ChainableBatchManager(this));
    }

    public void executeBatch() {
        if (batchServices.isEmpty()) return;

        Database.executeBatch(this, batchesOf20);
    }

    public class ChainableBatch {
        public String query;
        public Iterable<SObject> records;
        public IProcessRecords recordService;

        public ChainableBatch(IProcessRecords recordService, String query) {
            this.recordService = recordService;
            this.query = query;
        }

        public ChainableBatch(IProcessRecords recordService, SObject[] records) {
            this.recordService = recordService;
            this.records = records;
        }

        public Iterable<SObject> getRecords() {
            if (records == null) records = (Iterable<SObject>) Database.getQueryLocator(query).iterator();

            return records;
        }
    }
}

So, as you can see in the inner ChainableBatch class, I'm trying to cast the QueryLocatorIterator as an Iterable. This is because I assumed returning an Iterable would allow me to pass back either a predefined list of records, OR a QueryLocatorIterable.

Is there some way to extract a list from the QueryLocatorIterator? Or is there some other way that I can return some generic list/iterable, whether it be a QueryLocatorIterator or a List?

Edit: Forgot to state the issue I've been running into with the code as is. I've tested this class by running a test class that uses it indirectly. I'm getting this error when I run that test: System.SerializationException: Not Serializable: Database.QueryLocator And the Stack Trace just says "External entry point"