Is it possible to make such reference happen given the fact that the dependent package exist through the same namespace?
Certainly. All 2GP can share a single namespace, and dependencies are allowed in the same namespace as long as there are no circular references.
if the previous statement is true, then when it comes to package upgrades: if I update the component on main package and then attempt to upgrade the main package, will the component be automatically upgraded or the extension package should also need a new version to rely on the new version link as part of dependency package creation?
A package with a dependency can use any future version of the version they are dependent on. They cannot use any earlier version or any alternative branch. For example, let's say that you have the following hierarchy:
If your extension package uses 1.0.2 of the main package as the dependency, then you can install 1.0.3 without a problem, but you won't be able to install 1.1.0 or 1.1.1, because the dependency will no longer be met. If you don't create such branches, this wouldn't be a concern.