I haven't been able to find any official documentation or community discussion of the compelling reasons to upgrade the API version on Apex classes, triggers, and pages. I know there is the general answers of "it's just a good idea" and "they might deprecate support for it eventually", but for some of our clients those answers are good enough to justify the cost of having a developer invest the time to upgrade the API version and regression test all the code to make sure nothing broke after the upgrade. Are there any more specific reasons in the areas of performance, security, or reliablity that I could use as leverage to convince developers and stakeholders that it should be upgraded on a regular basis?
Contrariwise, I will provide an argument for upgrading, although the other answers state that complacency is acceptable.
There are two factors at play when you speak of API versions: features and consistency. Both of these issues introduce two distinct needs, namely stability and flexibility.
First, let's address consistency.
You should keep all your classes, pages, and triggers at the same API version. This is important to avoid bugs like this one. Note that this bug only occurs when some classes are lower than v28 and others are higher than v28. Each class appears to run in an emulated model that is compiled in a way that different versions are binary compatible, but having different features. Every once in a while, a newer feature can bleed into an older class, resulting in an error. You need consistency for stability. While the language is very good at what it does, it is not perfect, and you would do well to keep all your code within the same version.
Next, let's address features.
Inevitably, some CEO/CFO/VP/etc will want to implement a feature. You tell them that it's no big deal, it can be done in a week. Now, at this point, you're going to be at one of two places.
If you're in a habit of not upgrading, you'll write a new class with a new version, and try to tie it in to everything else, and everything will break. Not necessarily in big ways, but big enough to be noticeable. You've ignored the first aspect, consistency. So, you start upgrading all your other classes in the second week. By the third week, they've all been upgraded, but now your hunting down bugs that are really obscure, and you have no idea why. A month as gone by, and nobody's happy, but you swear you're almost there. Conversely, if you're regularly upgrading, you'll simply spend the week writing the new feature, and presto, it'll be ready.
In other words, by not upgrading, you are not saving time. You are kicking the can down the street, and when you need to upgrade, you'll be between a rock and a hard place.
I'm not advocating that you jump from v18 to v30 overnight. There are so many changes, so many small nuances you have to consider, that it would be a risky move. I tried something similar to that on a project, and we ultimately got set back months by way of troubleshooting, hunting bugs, etc. But, I'm also not saying that you should kick the can too far down the street. You need to address version upgrades in a controlled manner. There's nothing wrong with going from v18 to v20, then doing other things and going on to v22, for example.
There are a few versions that will be tricky to upgrade to, because they were significant upgrades. When they introduced "SeeAllData" for example, it required most test methods to be completely re-written or at least modified to use the new attribute. Then, they forced the split between test methods and live code. Who knows what other gamebreaking changes will happen in the future? You want to avoid the last-minute upgrade, because it will put you in a position you don't want to be in.
So, if you don't have the resources to be bleeding edge (and many organizations don't), at least avoid being left in the dust, because you will eventually have to invest the time sooner or later, and the longer you wait, the more time you will have to commit to at once.
Honestly, it's hard to find a compelling reason to upgrade it on a regular basis. Salesforce is set up so that you can leave code/versions alone and even when they upgrade the system it won't effect your code that uses old versions.
In my opinion (I could be against the general consensus here), the only reason to update versions is to allow you to take advantage of new features. If you have a section of code that has been working for years, why take the risk that upgrading the version could effect how it works and potentially break it?
That said, when I am making improvements to existing code, one of the things I try doing is bumping up the version and ensuring it works to the new version. If I am already working on it, it usually isn't a huge issue to do but why go out of my way just to make me feel better that all of the code is in the newest version when it doesn't bring anything to the table?
Unless there's a particular feature you now need to implement which existing code doesn't support, there's no compelling reason to update the code and save it under a new API.
Now if there's an update that breaks old code or exposes a security conflict with, then you have no choice. Some of last year's HTML related updates would be good examples that may have affected some orgs. That's why you're given advance notice before implementing those types of updates. You then have a chance to check code for potential issues.
So IMO, the only compelling reason to update is when an update to the API no longer supports existing code or perhaps when you're working on a new application that interacts with existing code. In those situations, its always wise to look at that code to see if there's maintenance that needs done to it for other reasons anyway.
As the old saying goes "If it's not broken, don't fix it!"
The team I work on upgrades the API version of all components each time we cut a new release. This being a product with a pretty well-staffed team means we have rounds of QA testing, including a full set of regression tests to catch any issues that arise.
Without that kind of team supporting you I certainly wouldn't try to upgrade your entire codebase every release - you're bound to break more things than you help by doing so without solid testing. Especially as visualforce testing can't be (natively) automated.
As a very general suggestion that warrants being tailored to your environment I'd try and do a yearly or every other year run-through of API version upgrades and testing. Being on an older release is generally fine, but being on a really old release means
Here is what I've gathered so far in venues other than StackExchange: