To get this out of the way, this question is very broad, and it's unlikely I or anyone else can fully address it in the scope of a single answer. Further, the "right" solution is most certainly product-specific, and many different choices in product architecture may influence the final design decisions.
With that said, here are my thoughts.
Permissioning at the level of an application feature requires composing the platform's native, component-level permissions architecture with development in the app and ISV tools like Feature Management. These layers of permissioning or feature enablement don't necessarily form a cohesive whole, and there are certainly customizations you can ship in a managed package that you don't have any real way to turn off and on.
Which permission artifact to use for what: Permission Set Groups, Permission Sets and Custom Permissions [and Feature Parameters]?
These artifacts are complementary to one another. The way I think about them is this:
- When you need to create a new permissions check - a gate for functionality that is not based on FLS, Apex class access, or other existing permissioning - you use a Custom Permission. For example, you can gate Processes or Flows on Custom Permissions, you can check them in Apex and make decisions about code pathways, and you can control visibility of elements in Lightning Pages based on them. Custom Permissions are suitable for when you want to turn on and off (many) UI features provided by your application. Of course, not all UI elements are permissionable at all, and whether or not you use bleeding-edge features like Dynamic Forms and Dynamic Actions makes a difference in how much you can achieve.
- I prefer to think about Permission Sets as "role facets", although I haven't really won anybody over to that terminology. "Jobs to be done" is a fairly similar framework that is widely used. A Permission Set grants the access required to do some unit of work within the application, where that unit of work might ultimately be executed by multiple roles within the customer. They should aim to be granular enough to be composed to suit the heterogeneous role definitions of the customer base, while being expansive enough to be manageable by a customer admin. That's a fine line to walk.
- A Permission Set Group, meanwhile, composes those Permission Sets into aggregations that are more likely to be directly usable by the customer admin. A Permission Set Group in my mind represents a persona or user role within your application, or combines fine-grained Permission Sets to grant access to a larger-scale job to be done. While one hopes that Permission Set Groups provide direct utility to the customer, they have the freedom to re-compose the underlying Permission Sets to align with their own role definitions.
- Feature Management is suitable for those situations where you, rather than the customer admin, have control over whether or not a feature is exposed. For example, your Apex code can check the value of a Feature Parameter and make a decision about whether to execute
changeProtection() to expose new schema and new Custom Permissions in the customer org.
Where should I package those artifacts? In Base or Extension? Based on which criteria?
What if the metadata of a feature is split between Base and Extension. Do I need two permission artifacts?
Not necessarily. You may, but it depends on the overall architecture of the product and whether there are valid use cases for the Base and this particular Extension to be used independently of one another. If Base is invalid without any Extension, for example, Permission Sets and Permission Set Groups could live only in each Extension, and permission those elements of Base that they need, while Base itself could be devoid of Permission Sets and Permission Set Groups.
Use of Protected schema or Custom Permissions may force your hand in terms of where to locate other permissioning elements (and Apex, and Feature Parameters) that align with them.
Otherwise, I don't see a particularly strong forcing cause to select one package or the other as the locus of permissions artifacts other than the expected customer use patterns.
At a later stage I might also like to use FMA Feature Parameters to turn Features on and of per Customer. Is there anything I need to think of right now related to Permissions?
Bear in mind that if you ship Protected schema (Custom Settings, Custom Metadata, Custom Objects) and use Feature Management to expose them, you can't hide them again in the same org.
You can expose and re-hide Protected Custom Permissions using Feature Parameters, which gives you all the control discussed above in managing automation and user experience.