It turns out that adding the namespace did indeed fix the issue. so $Permission.ns__MySetting (or whatever ones namespace and custom setting name is) is correct, while $Permission.MySetting caused the error when installed into other orgs that use a namespace.
My issue was, that a commented out $Permission.MySetting was still trying to be interpreted by SalesForce despite being commented out. And, since it didn't have a namespace, it caused the error.
I had this issue with remote method calls from my javascript to my apex controller as well. It seems any accessor of apex/salesforce that starts with '{!$' will be counted even if commented out and throw errors if any exist, which seems like a potential bug on their part that SalesForce doesn't seem to interpret comments correctly when those are in there and apparently doesn't count them as being commented out.
$Permission.Manage_Advanced_Org_Settings__c
?$Permission.ns__MySetting
.