It's a bit unclear what you are trying to do. It looks like you are building up a List of SelectOptions and then also trying to track the Body size of all those documents.

One likely cause of heap size issues here is retrieving the actual Body of the Document records in Apex when you only need to know the BodyLength.

Try something like:

List<SelectOption> docNames = new List<SelectOption>();
long totalBodyLength = 0;
Id exFolderId = '00lG0000001bBXb';

for (Document d : [SELECT Id, Name, BodyLength
                   FROM Document
                   WHERE (FolderId = :UserInfo.getUserId() OR FolderId = :exFolderId)
                       AND ContentType = 'application/pdf'] ){

    if(d.BodyLength >= 5 * 1048576) {
        // Exclude large documents that would exceed the Heap size
        // Could probably handle via the WHERE clause
        continue;
        // Note that in theory it should even be possible to create a Document > 5MB
    }

    docNames.add(
        new SelectOption(d.Id, d.Name + ' - ' + String.valueof(d.BodyLength) + ' bytes')
    );
    totalBodyLength += d.BodyLength;
}

It's a bit unclear what you are trying to do. It looks like you are building up a List of SelectOptions and then also trying to track the Body size of all those documents.

One likely cause of heap size issues here is retrieving the actual Body of the Document records in Apex when you only need to know the BodyLength.

Try something like:

List<SelectOption> docNames = new List<SelectOption>();
long totalBodyLength = 0;
Id exFolderId = '00lG0000001bBXb';

for (Document d : [SELECT Id, Name, BodyLength
                   FROM Document
                   WHERE (FolderId = :UserInfo.getUserId() OR FolderId = :exFolderId)
                       AND ContentType = 'application/pdf'] ){

    if(d.BodyLength >= 5 * 1048576) {
        // Exclude large documents that would exceed the Heap size
        // Could probably handle via the WHERE clause
        continue;
        // Note that in theory it should even be possible to create a Document > 5MB
    }

    docNames.add(
        new SelectOption(d.Id, d.Name + ' - ' + String.valueof(d.BodyLength) + ' bytes')
    );
    totalBodyLength += d.BodyLength;
}

Consider using Limits.getHeapSize() and Limits.getLimitHeapSize() to check for current and available heap. As Adrian comments, you won't be able to use the entire heap for the document body.