According to Salesforce Documentation we are able to remove a file from a single record that the file’s attached to. Unlike deleting a file, removing doesn’t affect the file on other records or posts, and the file remains in Files home.
To remove files from records, you must have at least collaborator access on the file. Below is a screenshot that shows I am the owner of the file.
Then on the record’s Files related list, we are support to open the row-level action menu and choose the option "Remove from Record". Below is a screen shot that confirms that I do not have access.
I am also System Admin of the org.