According to the documentation here
It states the following:
When a SOQL query is executed, up to 2,000 records can be returned at
a time in a synchronous request.
In summary:
You can query up to 2000 records or less in a single request.
Edit:
According to the documentation here
You can read the documentation to understand further, but to summarise. In your response body, you will also receive these two properties:
"nextRecordsUrl" and "done"
Image credit here.

So you can keep querying until the "done" property is equals to true.
Which means, there is no more records to retrieve. If the "done" property is false, that means there are more records to retrieve.
This is the snippet from the documentation:
If there are still more records to be returned, the response contains
a new query locator and done is false. You can continue retrieving
results from the initial query until done is true, which indicates
that all results are returned.