We have remote site settings that were there in the previous version of our salesforce app. Now if for newer version if I push upgrades to all customers will those remote sites be still whitelisted?
Yes those remote sites will be there in subscriber org.
A new package won't remove those remote site settings!
Remote site settings metadata is subscriber and developer deleteable! and not upgradable!
Hence as long as you don't delete the remote site settings metadata push upgrades should have no impact on them!