How do I mass delete records in a custom object in Professional edition?
You can use the Force.com Excel Connector for Professional Edition. You'll have to query the records into a spreadsheet, then delete them.
Do you have access to the Developer Console? If so, you could execute anonymous apex like this:
List<CustomObjectName__c> existing = [SELECT Id From CustomObjectName__c]; delete existing;
You would almost certainly want to add a "WHERE" clause to that SOQL statement to match your needs. Also, be sure to test the SOQL in the Query Editor before you actually delete the records. It's always good to be certain you're deleting the right records.
Note: SOQL will only return 50,000 records. If you need to delete more than that, you'll either need to add a "LIMIT" clause and do multiple deletes or write some more advanced code.
For deleting records, I've always found Dataloader.io to be the best option. You can export all records from a particular object or you could use filters to narrow down your record set.
Export the records you want (make sure you select the "Id" field to export, I usually select all fields) save the file and then create a new "Delete" task. Indicate which column contains the Id's of the records to delete and then process the task.
That will delete all of the records in your spreadsheet. If you need to undelete anything, those records will remain in the recycle bin for up to 15 days. You can also just import again as long as you included all fields in your initial export.