Event has a WhoID, an OwnerID, a WhatID along with a Record Type that can be associated with it, so you have several options on how to handle your issue.
If all the contacts/Leads have the same owner as the event, that would be one solution to your issue (unlikely, but possible).
If you have a record type for Leads or Campaigns associated with their generation, that would be another. If you don't, you could establish a Record Type for Events held for Leads or Campaigns, to associate the event with to make your connection.
If you have a specific Campaign, the WhatID would be what you'd associate with the event and that would also make the connection you're looking for.
As you've already discovered, the WhoID, if used for the lead, is not going to do the trick for you. However, if you were to use the WhoID for the manager of the Owners of ALL the Leads, something along those lines could possibly work. That solution however would depend on how your organization was set-up.
I hope this will point you to a solution that will work within your org's structure. Since each one is different, there isn't always a solution that "fits all".
BTW, make certain you have the Event field of
IsGroupEvent set to
true as that could also be part of your problem. If it's not currently set as true, try enabling it to see if the connection you're looking for is made before making any other changes.