No changes should be allowed directly in production without either being in the Source of Truth (typically, a Git repository) or documented rigorously. While users may protest a restriction on Public Groups, there needs to be an understanding that their changes could be lost by accident if not placed in the Source of Truth. Nothing outside the Source of Truth should be relied on, including further customizations that are in the Source of Truth.
If you've never accidentally deleted your hard drive's master partition, you may think I'm being paranoid, but I'm not. I have seen administrators accidentally wipe out an entire org's accounts (with cascading deletes to every opportunity, case, contact, contract, etc) because of a simple misclick in the Data Loader. That employee, by the way, cost the company millions of dollars in potential lost sales and ultimately lost their job over a couple of clicks.
Mistakes can and will happen. Adding an unknown Public Group could cause a unit test to fail and seize up an entire org's deployment process. Or, a developer could create an identically named Public Group and wipe out the previous member list unaware that things are now broken. This, compounded by the fact that developers won't even know anything's wrong until it's time to deploy to production, means this is a disaster waiting to happen.
Granted, customers should be given an option to ignore this advice, because it is their data at risk, but they should also be told why it's a bad idea to ignore the warnings. Creating a Public Group in an external system seems like a monumental waste of time, but the warnings are in place for a reason. It's the same reason why most cars require you to put the car in Park/Neutral and/or press on the brake before you start the car. Most of the time, maybe nothing bad happens, but it's a risk every time.
So, the list of approved changes should either be nothing or everything (and accept the risks), or perhaps select which items you deem are low-risk enough. There's no salesforce.com-recommended list of items that could be deemed safe to ignore the rules for, because there is always an inherent risk of leaking data or "bricking" your org in very unamusing and costly ways.