Why doesn't ContentVersion Query doesn't respect OFFSET when you include VersionData?

Question

I have some code where I have to be careful about how much ContentVersion data I load into memory at any particular time to avoid blowing the heap limit.

I was planning to do an initial query on ContentVersion.ContentSize to see how much I could load, and then use OFFSET and LIMIT to get the right amount each time.

But then I found that the query with LIMIT and OFFSET got surprising results. I boiled it down to this test, where I would expect all assertions to pass:

@IsTest
private class ContentVersionIsWeirdTest {

    @IsTest
    static void testBehavior() {
        List<ContentVersion> contentVersions = new List<ContentVersion> {
                new ContentVersion(VersionData = Blob.valueOf('One'), Title = 'One', PathOnClient = 'one'),
                new ContentVersion(VersionData = Blob.valueOf('Two'), Title = 'Two', PathOnClient = 'two')
        };
        
        insert contentVersions;

        Assert.areEqual(2, [SELECT Id FROM ContentVersion].size()); // PASS 
        Assert.areEqual(2, [SELECT Id FROM ContentVersion LIMIT 2].size()); // PASS
        Assert.areEqual(2, [SELECT Id FROM ContentVersion LIMIT 2 OFFSET 0].size()); // PASS
        Assert.areEqual(2, [SELECT Id, VersionData FROM ContentVersion].size()); // PASS
        Assert.areEqual(2, [SELECT Id, VersionData FROM ContentVersion LIMIT 2].size()); // PASS
        Assert.areEqual(2, [SELECT Id, VersionData FROM ContentVersion LIMIT 2 OFFSET 0].size()); // FAIL
    }
}

It seems like including VersionData makes the OFFSET 0 cause a problem. I can't see anything about this in the considerations for ContentVersion.

Is this a known thing that I've missed?

I do have a viable workaround - in the real code, I was always ordering by PathOnClient, and I know the files I want have unique paths, so I can use that to paginate through. But I'm curious about what's going on here.