Great question, Open Source on the platform was actually my very first Dreamforce session in 2012, and effectively kickstarted my presence in the community. I co presented with a number of others including Jeff Douglas and Reid Carlberg, it was a panel discussion format session. I remember this very topic coming up and sighting it as one of the inhibitors to open source really breaking out the way it has on other platforms. We hoped things would improve to make things more manageable... (no pun intended)
Reflection When we look at what other open source projects do on other platforms, they target two types of customers, those that simply want to install the tool and those that want to contribute. Take Jenkins, they make incredibly easy to install and upgrade it as a consumer, as you point out in this mode your in the hands of others in respect to fixes and updates. Yet they also of course offer instructions on how to build and deploy manually, typically preferred by developers, that know they are needing to customise and ideally contribute back to the project. The downside is that in most cases when this route is taken its expected that the developer handles updating their installation and marshalling back any contributions to the code base.
Force.com. So in some respects i don't think Force.com open source projects need to be that different, though in reality, i find it rare that the open source project offers a managed package (i find unmanaged packages a bit pointless to offer). Even when they are they are often not kept up to date with the repository contents. This aspect draws consumer down the DIY deployment route. It also worth noting that not all open source projects are suitable for managed package deployment, due to the way global has to be used, it constrains projects that are more developer libraries than tools. Finally we all know creating managed packages is not automated, something other platforms enjoy, meaning the open source contributors have to spend less of their precious time writing code if they want to provide this facility each time they make a release.
Process. I think the approach you take really depends on where you intent on going with your use of the open source tool and how active the project is going through its release phases. If you plan on contributing back or not basically and also how quickly you need fixes or enhancements before the open source project owners collectively decide to push a new release (i for example have given packaging org access to my open project contributors to help ease the burden on me and promote more managed releases).
So i can think of these routes you might want to consider...
- Clone Repository > Unmanaged Deployment > Pull Request, Often this is a route for folks to play around with the code, debug it to contribute an issue or just know more about the code. I personally would not recommended large amounts of unmanaged code in production, open source or no open source its just hard to manage (pun intended!), but being an ISV maybe I am bias to the world of managed packages. But this is a route you want to take, then you can of course use Refresh from Server in your IDE to update your local repo and make a pull request to contribute back.
- Clone Repository > Create/Update Private Managed Package > Install Private Managed Package This route essentially branches you away from the open source repository to a degree, especially if you start to create your own private managed package (as changes to the open source version may conflict with your package release uploads). Though this does give you a better way to manage your production upgrades of the tool in production and freedom to move in a different direction. You could consider optionally contributing back via a pull request from your cloned repo.
- Community Managed Package Install > Clone Repository > Contribute > Community Release of new Package > Upgrade Managed Package, This is probably the most 'community friendly' approach, but the cycle can be slower depending on the release frequency of the project. In this case you would be deploying to a DE org to do the work of course, rather than production. From a personal point of view, if someone is contributing some good stuff that moves it along, as i said above, i'd be more than happy to grant access to the packaging org and after a quick round table with the other committers, perform a new release.
Scoping If you do take the route to combine open source into another large package, the lack of a Apex code namespace (or package in Java terms) is a real pain. The only recommendation i have for this is to follow your own prefix on every component, class, trigger, object, field (even custom labels), literally everything! You can see evidence of this approach FinancialForce.com's Apex Commons library, where we used fflib_ on the front of everything. I'm getting used to it now, but still looks a little funny. Other option, and can be used in combination, is to use an outer class as your namespace and put everything in inner classes, depending on the size of the code, not always practical, but something to consider.
Summary. So i guess, what i am saying is my personal recommendation (and again maybe a little bias) is for the best of both worlds for "tool" type projects, is to actually be both types of consumer of the repo contents. Install the community package into production enjoy the platform upgrade facility and take fixes and enhancements into a DE org work on them and contribute back into the repo for the next release. If thats not for you, and the open source licenses typically allow for this, take it with thanks and go your own route, but if you do this, of course there will be some overhead if you want to try and sync back in the future with the original project. For "library" based projects, ideally i'd want to see some kind of effort to scope the code as above before embedding it.
Hope this helps, not sure how much technical detail you want, i can for sure add more about sync'ing MavensMate with your GitHub local repo, deploying and retrieving contents of an org, if thats what you want.
Shameless Plug, The GitHubSFDeploy tool will take unmanaged code from GitHub into a Sandbox or DE org to support some of the aspects described above. It is an interesting future enhancement to allow this to work in the reverse direction as well (e.g. automatically create a Pull Request). Also the tool does not support deploying to Production, so is really currently in the deploy to a DE org with a view to contributing back to a future managed package release route. Though if this was my intent i'd personally start with the Clone Repository route, since its easier to be editing within that from the start than merge changes into it after.
