The Live Agent APIs are not wide-open - they require the affinity token (based on previously established session) and a session key. It's true that these APIs do not support oAuth but you can fix that. Deploy your own, custom REST API that functions as a proxy - when receiving a call from your client application, your API can construct a new HTTP request, ...
https://test.salesforce.com/... is the right way to go for sandboxes.
From the doc
Instead of login.salesforce.com, customers can also use the My Domain,
community, or test.salesforce.com (sandbox) domains in these
(Admittedly, it's a very brief doc note)
For sandboxes two URL types can be used:
Generic URL like https://test.salesforce.com/services/oauth2/token
Specific Classic Domain URL like https://domain--full.cs100.salesforce.com/services/oauth2/token
Lightning domain URL cannot be used.
Classic domain URL changes every refresh since Sandbox might migrate to another instance, so it is better to use ...