Territory Management is more than just a sharing device, it probably makes sense when there exist clearly defined Sales Territories, with Sales Reps mapped to Territories and where Sharing Accounts and Opportunities would be tedious through conventional means - be it Account/ Sales Teams and/or Ownership/Criteria Based sharing rules.
Everyone in a territory can view all the accounts in the territory, so there is that loss of granularity vis-a-vis sharing control. It also (much like Account and Sales Teams) allows for control on the sharing of related Contacts, Opportunities and Cases.
Customizable Forecasting needs to be enabled for Territory Management and Sales Reps will have forecasts for every territory they have Opportunities in.
Also, Territory management facilitates easy movement of people in dynamic sales teams, where sales reps may need to switch between territories, and I believe there is a way to retain Opportunities even when switching across territories.
Territory Management also has a stronger justification when the User volume is quite high, as opposed to just a handful of users.
As for limits, 15 Account Based assignment rules are permitted. 50 criteria based sharing rules per sObject are permitted. Also TM cannot be turned off once enabled, hence need careful analysis before flicking the switch.
Territories have names to identify them, which CBS do not. Territories appear as a new dimension in Reorts and List-Views which make these super-powerful.
In summary, Territory Management is a scalable solution which fits the needs of a large organizations with several thousands of users, where there is a need for Account sharing organized by territories (geographical, verticals, or otherwise). It adds another layer of complexity, which interoperates with Sharing and Account / Sales Teams, and should be adopted when the envisaged benefit justifies the setup cost and complexity.