When I query for VersionData from a ContentVersion record, I get "/services/data/v44.0/sobjects/ContentVersion/0681F000000QY0eQAG/VersionData" in the response. I'm expecting a base64 encoded file. This happens whether I'm querying with the developer console or the Rest API. What gives?
That's not how the REST API works. To get the Blob (file contents), you have to call that rest resource. This allows the response to be reasonably sized without overwhelming the client, which may be a mobile device. See Get Attachment Content from a Record.