Currently, it seems that there is no way to remove some of the options for a picklist while using the lightning-input-field (in my case on the lightning-record-edit-form).
The only solution I came up with is to use the lightning-combobox instead. However, with this implementation, a custom creation of the record has to be developed which defeats the whole purpose of using the lighting form.