I don't believe that Salesforce provides a REST resource to do what you're asking for.
You may be able to accomplish this by (ab)using the List Views resource /services/data/v52.0/sobjects/Account/listviews/00BD0000005WcCNMA0/results?offset=[your value here]
as there is no documented limit on that offset parameter like there is when you use OFFSET in a query, but the results of that call are not the most friendly to work with and you're limited to 15 fields. A comment on this other question suggests that the max offset here may still be 2000.
Realistically, you'll probably need to expose an Apex class as a REST Resource. You could set up a GET method that accepts a few URL parameters, and feed that to an instance of the StandardSetController class to handle returning the paginated results (which can handle things up to 10k rows). The setPageSize()
, setPageNumber()
and getRecords()
methods are the ones you'd be primarily interested in.