Contact has a read-only Name field out of the box, which concatenates FirstName and LastName. So your query would look like this:
SELECT Contact__r.Name
FROM BOG_Membership__c
WHERE BOG_Membership_Status__c = 'Active'
AND BOG_Membership_Type_New__c <> ''
In fact, you can chain selects through multiple lookup fields, e.g. to select the name of the Account the Contact belongs to:
Contact__r.Account.Name
EDIT
You say the Contact Name is returned when you run the SOQL, but when you run the Apex you get invalid field Contact__r.Name.
I guess you mean that you have stored the SOQL query results in a BOG_Membership__c variable, and you now want to read the related Contact's name from it. If not, please share the exact code which is causing the Invalid field error.
If this is the case, then bear in mind the SOQL returns BOG_Membership__c records, and BOG_Membership__c really doesn't have a field called Contact__r.Name. To get to the related record, use SObject.getSObject(Schema.SObjectField) method:
//Assuming there is at least one membership record on the DB...
BOG_Membership__c m = [SELECT Id, Contact__c, Contact__r.Name FROM BOG_Membership__c LIMIT 1];
Schema.SObjectField contactField = BOG_Membership__c.Contact__c;
Contact c = (Contact)m.getSObject(contactField);
//For my org, this logs out Name: Some Guy
System.debug('Name: ' + c.Name);