Its possible that some of the packages you have installed are what are called "unmanaged" packages. So while they were installed via the App Exchange, they cannot be upgraded, are license free, and their code can be edited/customized by developers or admins in the org to which they've been installed. There are a large number of unmanaged packages out there that Salesforce Labs put out such as Simple/Quote/Invoice/Order and their version of Milestones-PM that work as I described.
In the case of a "managed package", it could still be "free to use", but will have its own namespace and it's primary components, particularly it's classes and triggers cannot typically be edited/customized by a customer. Some managed package components can usually be customized to suit an org's needs, but it depends on the component and the package. Those packages will be referenced in the "Referenced Packages" folder and the org's schema.
The primary purpose of that folder is that some classes, objects and pages will have dependencies on those managed packages; primarily in terms of licensing, permissions and record sharing. Those relationships need to be maintained and can affect your code along with what your code and your users have access to.