This is mentioned here in Sandbox Setup Tips and Considerations, specifically:
Sandbox copy is a long-running operation that occurs in the background. You are notified of the completion of a sandbox copy via email. Sandbox refreshes might complete in minutes, days, or even more than a week.
A number of conditions factor into the duration of a sandbox copy or refresh, including the number of customizations, data size, numbers of objects and configuration choices (for full copies), and server load. Also, sandbox refreshes are queued, so your requested copy may not start immediately after your request.
So, to answer your question, it may take a week or even more for the Sandbox refresh to complete. There's also another article, that I cannot find currently, that mentions that there are two types of refreshes, namely "slow" refreshes and "fast" refreshes. Simply put, if you have a larger data volume, you may be placed into the slower queue, while if you're doing a configuration-only refresh, you might end up in the "fast" queue. The specifics for what qualifies for each queue is not publicly mentioned, and is probably tweaked periodically to provide maximum performance.