I created a new Duplicate Rule for Account that should fire when an Account has the same (Name and Site) or (Name and Billing City). A custom matching rule was made however the rule still will not fire when I test it. I have been testing by creating 2 Accounts with the same Name and Site.
Your conditions at the bottom aren't doing what I think you think they're doing. :-)
Try creating an account with the name "true" and a city value of "true" - then it should fire.
Conditions tell the duplicate rule what accounts to evaluate. Matching Criteria is actually what defines a match/duplicate. See Create or Edit Duplicate Rules for more information.
If your goal is to require that these field values are not blank, then you need to adjust the Matching Criteria in the Matching Rule so that null values are not counted. Right now it's configured to evaluate when there are null values (MatchBlank=TRUE).