Yes Salesforce can schedule 5 batch in parallel so there are chances that you batch might over lap.
You can do something like this
//check if there are 5 active batch jobs
//In some cases, might need to add "Status='Queued' " to the criteria
if ([SELECT count() FROM AsyncApexJob WHERE JobType='BatchApex' AND (Status = 'Processing' OR Status = 'Preparing')] < 1){
Database.executeBatch(batchClassInstance);
} else {
//schedule this same schedulable class again in 30 mins
nameOfYourSchedulableClass sc = new nameOfYourSchedulableClass();
Datetime dt = Datetime.now() + (0.024305); // i.e. 30 mins
String timeForScheduler = dt.format('s m H d M \'?\' yyyy');
Id schedId = System.Schedule('MatrixRetry'+timeForScheduler,timeForScheduler,sc);
}
It will check if any batch in processing then it will schedule your class to run 30 min late. In below query you can add your class Id in filter so it will check for your particular class and if you get records then set your schedule class to run 30 minutes late.
Reference
This is the query
[SELECT Id, CreatedDate, CreatedById, JobType, ApexClassId, Status, JobItemsProcessed, TotalJobItems, NumberOfErrors, CompletedDate, MethodName, ExtendedStatus, ParentJobId, LastProcessed, LastProcessedOffset FROM AsyncApexJob WHERE ApexClassId = 'Class Id here']
You can pass your class id in filter