How many custom buttons does a custom object allow ? That means : how many custom buttons can I create and add to my page Layout ?
As far as I can find, the official documentation does not have one, and the object page also does not specify a limit. I would imagine, however, that you would not want a huge number of buttons attached to a single object, and the actual count of those would be rather low, instead of over a hundred for a single object. Not sure why anyone would need a hundred buttons for an object, but it seems possible.
If you do hit a size limit, you can use Enabler for Excel to convert that page layout into a VisualForce page, and code the buttons yourself.
I know this isn't much, but I couldn't find anything. Hope what I posted helps.