This is not recommended to customize standard send email functionality only for Additional To option.
You can raise a case to Salesforce to create a Skinny table on Contact object which will enhance performance of contact list view.
Refer Skinny Tables
Salesforce can create skinny tables to contain frequently used fields and to avoid joins. Doing so keeps the skinny tables in sync with their source tables when the source tables are modified. If you want to use skinny tables, contact Salesforce Customer Support. When enabled, skinny tables are created and used automatically where appropriate. You can’t create, access, or modify skinny tables.
Skinny tables are most useful with tables containing millions of records to improve the performance of read-only operations. such as reports. Skinny tables aren’t a magic wand to wave at performance problems. There’s overhead in maintaining separate tables that hold copies of live data. Using them in an inappropriate context can lead to performance degradation instead of improvement.
Skinny tables can be created on custom objects, and on Account, Contact, Opportunity, Lead, and Case objects. In addition, they can enhance performance for reports, list views, and SOQL.