I am likely not understanding some parts of the potential of LWC OSS but is it possible to built an application/ component in LWC OSS and then utilize it within Salesforce. Or is this only for building applications outside of Salesforce with the LWC framework?
Some LWC Open Source components are likely to also work well on-platform. If you're using SLDS styling, or even the open source Lightning base components, in your open source components, that will certainly help! Components that are primarily focused on presentation or composition of base components are likely to be the easiest to port or simply reuse.
In other cases, particularly where data access is involved, you'll likely use different structures on- and off-platform. For example, all of the Apex-related wire adapters are specific to the Salesforce platform. You may build your own wire adapters for different data backends (I built one for Django, for example) that will not work on-platform, where you cannot build custom wire adapters. Likewise, on-platform you can access features like Custom Labels that are not available off-platform.
To sum up: you're likely to have at least some code that can be reused between LWC on- and off-platform. Some components may be usable without changes in both environments. Others, particularly those that heavily interact with your data layer, may need extensive changes.