@RestResource annotated apex classes are just like normal apex class and only difference is they expose the apex class as a HTTP GET,POST,PUT or PATCH web-service and hence all the governor limits applicable to the apex class context apply here .
These limits will be 50,000 queries in a context,150 DML,100 SOQL limit of context ,CPU limit ,Heap size limit etc ,all this generally apply .
Also since its a webservice its very important to consider there will be 3MB size limit for the response you return from the apex class.
Coming to the no of API calls ,each request will count against number of API calls and no of API calls allowed for your org depends on edition and as well as license .Also these API calls can be purchased from sfdc with some additional cost(Contact SFDC for same )
Before jumping into writing your custom apex rest service i would strongly recommend to use the standard REST API provided by sfdc ,if any of the API provided by sfdc meets the requirement i would use that as they are faster and have capability to automatically return data in chunks
http://www.salesforce.com/us/developer/docs/api_rest/
The above document link has reference and you will find explanation for standard REST services available out of box.If any of them dont fit then you will need to build apex REST service and an optimized code is necessary to avoid governor limits .
Update:
API limits
http://na1.salesforce.com/help/doc/en/integrate_api_rate_limiting.htm
For enterprise its 1K per licence for salesforce licence