I'm working on an ISV app that needs to query a large number of records- more than is allowed by the native SOQL query governor limits. If the ISV application calls the Salesforce REST API to retrieve data from within native Apex code (with programmatic security checks for access to the record and its fields), will this prevent the application from passing the security review (also assuming that the end-user will set up an authorized remote site as part of the installation process)?
There shouldn't be any problem; after all, the REST API won't give users any more access than they'd have in the UI (e.g. you can't query records you couldn't view in the UI, etc). They will, of course, make sure you're not making callouts to other orgs/systems or otherwise leaking data you shouldn't, but using any of the APIs are normally fair game as long as you're respecting the org's security settings.