I am using the serial mode of bulkUpdate/Insert. I am just wondering if there any other impacts apart from the speed.
Serial mode specifically has a benefit (and the only reason you should use it): it relieves database contention. The trade-off is that it runs slower. That's the purpose of using serial mode.
On top of my head, No there should not be any impact. It would use the same number of API calls and the Same number of Bulk API job so should be fine.