Sadly, this is a Known Issue with no scheduled resolution date:
https://success.salesforce.com/issues_view?id=a1p3A000000enp0QAA
For posterity:
Summary: Intermittent slowness observed on NA35 during deployments
Repro: The current architecture uses a queuing framework for all
deployment requests. This is a reflection of the fact that deployment
was built to support packaging, change sets, and the ant tool -
release management use cases that are typically larger deployments.
Dev Tooling (IDE) use cases have been supported by this same
framework. And, although we have run into some similar issues in the
past, they have not been as consistent as they have been in the past
two releases. These smaller deploys have started to suffer due to the
increased sophistication of customer's development processes. Simply
put, we have seen a lot of continuous integration development. This is
a great sign for our platform, except for the fact that we are
responding to the extra load we are seeing.
That said, we are working on a few things targeting the smaller
deployments. We have finished a costing engine this release and we
have added a few new queues. This will allow us to target the smaller
deployments and have them on their own queue. The fair usage system
which governs our queuing system will only degrade large deployments,
which are much fewer than the smaller tooling deployments. This is
only in pilot in our Feb release as we need to monitor this before
rolling it out everywhere.
So, there are good things coming, but you won't see the results of
this work until the spring/summer time frame. In the mean time, please
keep the dialogue open and continue to submit the cases, so that we
have a good understanding of the overall affect.
Workaround: Also, if you are building CI systems, please separate
deployment from testing. When you include tests in the deploy, the
overall deployment will take longer which will count against the
metadata service when it is evaluated for fair usage on an instance.
Execute the deployment without tests, if possible, and then kick off
the tests through the Tooling api.