You wouldn't be able to use a "binding" like that, but since you have a String for you SOQL:
string contactsToDelete = 'Select ID,LastName,FirstName,Name from Contact where CreatedDate = '+myDate;
This should give you the correct SOQL for this to work.
There is a predefined function valueOf from Date class.
Date myDate = Date.valueOf(System.Label.SetConfigurableDate;);
Also CreatedDate is a datetime field so make sure you convert the date to datetime. This can be done by utilizing the method newInstance() from DateTime class.
createdDateTime = DateTime.newInstance (myDate.year(), myDate.month(), myDate....
Not sure if I fully understood your issue, but if what you pretend is to translate a Quick Action Label you can do that through Translation Workbench, selecting Action on the Setup Component. Below is an example:
Another way is writing a separate js file where write all import statement of custom label and a export function.
To share code between components, create an ES6 module and export the variables or functions that you want to share.