You won't be able to get the size directly in visualforce page, if you want to render some components based on the size of the OpportunityLineItems, then you will have to declare a variable in controller and assign the size of OpportunityLineItems using below code

public Integer sizeOfOpportunityLineItems {get;set;}

and set this variable

sizeOfOpportunityLineItems = relatedTo.OpportunityLineItems.size();

then you can use this variable in outputPanel to render components

<apex:outputPanel rendered="{!sizeOfOpportunityLineItems != 0}">

You won't be able to get the size directly in visualforce email template, for that you will have to use the repeat tag to get the count and then you can build further logic based on that variable.

<apex:variable var="sizeOfOpportunityLineItem" value="{!0}"/>
<apex:repeat value="{!relatedTo.OpportunityLineItems}">
    <apex:variable var="sizeOfOpportunityLineItem" value="{!sizeOfOpportunityLineItem + 1}"/>
</apex:repeat>
Number of Opportunity Line Items : {!sizeOfOpportunityLineItem}