It simply means:
- When you have a label attribute, the component level error message would show error in context label passed in attribute.
- When your have a empty label attribute or does not have a label attribute at all, it would not show any message with the default field's label.
Difficult to understand; maybe it would be clear when you try the example.
Output on click of save when email is empty (code at the end)-

Note: Error message for second and third sections/input are same (as apex:pageMessage doesn’t show duplicate message), default message with field's label; only difference is in #3 we have empty label hence it doesn't show a field label because its expecting us to add.
Code for illustration:
<apex:page standardController="Contact">
<apex:form id="form">
<apex:pageMessages></apex:pageMessages>
<apex:pageBlock title="Quick Edit: {!Contact.Name}">
<apex:pageBlockSection title="1. Dynamic Label" columns="1">
<apex:inputText value="{!Contact.Email}"
label="{!Contact.FirstName + '’s Email'}" required="true"/>
</apex:pageBlockSection>
<apex:pageBlockSection title="2. No label" columns="1">
<apex:inputText value="{!Contact.Email}" required="true"/>
</apex:pageBlockSection>
<apex:pageBlockSection title="3. Empty label" columns="1">
<apex:inputText value="{!Contact.Email}" label="" required="true"/>
</apex:pageBlockSection>
<apex:pageBlockSection title="4. From custom Label" columns="1">
<apex:inputText value="{!Contact.Email}"
label="{!$Label.Label_Contact_Email}" required="true"/>
</apex:pageBlockSection>
<apex:pageBlockButtons >
<apex:commandButton action="{!save}" value="Save" rerender="form"/>
</apex:pageBlockButtons>
</apex:pageBlock>
</apex:form>
</apex:page>