Well anotations are quite useful in languages like JAVA to design your own language paradigm, in APEX they are more like representation of what do they do and how they behave
1. Can we detect whether/how a class has been annotated? Yes:)
You can get class body by this SOQL
SELECT Body FROM ApexClass where Name='MyClass'
Once you get class body you can fire a regex to see if it contains any of the below string annotations
@AuraEnabled
@Deprecated
@Future
@InvocableMethod
@InvocableVariable
@IsTest
@ReadOnly
@RemoteAction
@SuppressWarnings
@TestSetup
@TestVisible
@RestResource(urlMapping='/yourUrl')
@HttpDelete
@HttpGet
@HttpPatch
@HttpPost
@HttpPut
Or as Suggested by @sfdcfox you can use SOSL like
FIND {@RestResource} in ALL FIELDS RETURNING ApexClass(Name)
- Is it possible for us to create our annotations? Unfortunately at this stage its not.
3. What might be some good use cases? There is no decent Apex XML parser available, having the ability to generate auto parsed XML instance. probaly a custom anotation would do like it helps in java. Apart from that there are so many ideas present on ideaexchange wich tells how beautifully you can use new or custom annotations. Do check them out.
https://success.salesforce.com/search?type=Ideas&keywords=annotations