As a starting point, have a read of Which API Should I Use?.
When to Use REST API
REST API provides a powerful, convenient, and simple REST-based Web services interface for interacting with Salesforce. Its advantages
include ease of integration and development, and it’s an excellent
choice of technology for use with mobile applications and Web
projects. However, if you have a large number of records to process,
you may wish to useBulk API.
When to Use SOAP API
SOAP API provides a powerful, convenient, and
simple SOAP-based Web services interface for interacting with
Salesforce.You can use SOAP API to create, retrieve, update, or delete
records. You can also use SOAP API to perform searches and much more.
Use SOAP API in any language that supports Web services. For example,
you can use SOAP API to integrate Salesforce with your organization’s
ERP and finance systems, deliver real-time sales and support
information to company portals, and populate critical business systems
with customer information.
When to Use Bulk API
Bulk API is based on REST principles and is
optimized for loading or deleting large sets of data. You can use it
to query, insert, update, upsert, or delete a large number of records
asynchronously by submitting batches which are processed in the
background by Salesforce. SOAP API, in contrast, is optimized for
real-time client applications that update small numbers of records at
a time. Although SOAP API can also be used for processing large
numbers of records, when the data sets contain hundreds of thousands
of records, it becomes less practical. Bulk API is designed to make it
simple to process data from a few thousand to millions of records.
The correct API for the job will depend on where you are calling it from, how many records you are working with and how long you are prepared to wait for the result.
Your use case
From your description of your server applications I'd suggest starting with the Bulk API. It sounds like your code is running without a user waiting for immediate output, so the asynchronous nature of the calls shouldn't be an issue. Moving more records in fewer API calls will also help keep your API call count down.
The SOAP API is another option if the number of records isn't particularly large and you are more comfortable using XML SOAP Messages after importing the WSDL.
There is also a discussion style blog post Salesforce APIs – What They Are & When to Use Them.
UPDATE for comments:
I do not see much benefit using REST over SOAP. Hence the question.
With SOAP you can import a WSDL into your tooling of choice and have native methods generated for working with the API. This makes calling the API and accessing the data relatively straight forward (E.g. strongly typed classes in .NET and Java).
There is some overhead for all the SOAP marshaling code that is generated. REST strips away a lot of this and sends more minimal messages that are closer to how HTTP operates.
Consider this diagram I've borrowed from the popular StackOverflow question: REST and SOAP
- Image by Nakkeeran
Any thoughts on the (non-bulk) REST API for exporting records?
In general, the REST API (non-bulk) isn't the first choice for a server to Salesforce integration where the focus is more on moving large volumes of records rather than responding to the user quickly and integrating with software running in a browser.
You could use the REST API if you want, but the SOAP and BULK APIs are generally better suited for your use case.