You might want to look at the documentation, but let's go over this real quick.
Is it ok to upgrade beta version on top of another beta version? This is not allowed for 1gp and 2gp, so it is strange that it works with unlocked packages. Is this expected behaviour and we can rely on it, or is it some sort of SF issue to be patched/fixed in the future?
Unlocked packages are designed to do this. The docs even say:
A package version contains the specific metadata and features associated with the package version, at the time it was created. As you iterate on your package, and add, remove, or change the packaged metadata, you create many package versions.
Is it true that unlocked package does not support concept of 'ancestor' at all and we can technically even do a downgrade, i.e. install lower version on top of higher one?
The only reference I recall is from the old FAQ. It says:
Q. Can I downgrade my package?
In the context of packaging, a downgrade is installing a lower version of a package on top of a higher version. For E.g.: this is when you try to install ver 2.0 of your package in an org where ver 3.0 has been installed. When you downgrade a package, the following occurs:
- The metadata of the package version (ver 2.0 in this example) is deployed to the target org.
- Any metadata that is present as part of the package in the installed org (ver 3.0 metadata in this example) is modified, deleted or deprecated based on whether they are present in the version being installed. For E.g.: an apex class will be modified to the ver 2.0 copy of it as part of the downgrade. If a custom object foo__c is not present in ver 2.0 but present in ver 3.0, foo__c is marked deprecated (See here for more info about what deprecated means in this context). If a report object is not present in ver 2.0 but present in ver 3.0, the report is deleted from the org.
You should consider any downgrade of a package carefully. Package downgrades can fail due to dependency scenarios. For E.g.: if ver 3.0 has an apex class that is referenced by another apex class in a different package, downgrading to ver 2.0 would fail with a user-friendly error message if that apex class is not present in ver 2.0 of the package.
In summation, you can, but consider yourself warned.
What is the consequences of promoting unlocked package version to released state? Can we still continue removing metadata the way we want and go with weird unordered upgrades, or some restrictions start applying at this stage? If so, do we need to promote version of unlocked package at all?
By default, unlocked packages can only be installed in non-production orgs, such as sandboxes, developer orgs, and scratch orgs. Promotion performs a run all tests and validates all metadata. After passing with 75% coverage and no errors, the version will be flagged as promoted. Once promoted, it can be installed in any org.
Promotion doesn't add any restrictions, unlike a managed package. That's because removed items are deprecated instead of deleted, if they're still being used in the subscriber org. In other words, it's impossible for an upgrade to fail to install due to any configuration changes that a subscriber may have made to an org, unlike managed packages.
If you want your package to be used in a production org, you must promote that version and meet the requirements for code coverage. This is the entire reason why you promote. Note that since promotion runs all tests, a promoted package will not need to run tests during installation, which reduces the install time significantly.
Is it possible to upgrade released/promoted version on top of beta version or vice versa?
Yes, the promotion flag only enables installation into production orgs. Of course, this means that you can only upgrade to and from beta packages in non-production orgs, but for those orgs, you can freely install whatever version you want.
Can we install beta version in prod org and then promote it post factum? Any consequences of doing so?
As is hopefully obvious by this point, no, you can't. The promotion process is what enables installing to production.
The great part about unlocked packages is exactly what may be confusing you: you're allowed to make mistakes without consequences, restructure your metadata as you need to, experiment with design choices, and so on. You can learn quite a bit on this trailhead module.
