As far as i know anything you can do w permission sets you can do w profiles, but there are a few things that can as of now only be done w profiles.
The key UI difference in my experience is that only profiles can be used to control page layout assignment.
I took a look and at a top level, it looks like the following items are profile only. There may be others, I have not been able to find a clean list
Permission sets and profiles have come very close to parity. Now that record type assignment can be done in either, it's just a few key things that remain, and these likely would remain on profile, as a user can only be assigned one page layout at a time, so it wouldn't really make sense on a permission set.
FWIW, besides for the page layouts, which I knew about from general experience, I came up w this list by looking at a perm set and a profile in the enhanced editor and noted which top-level sections were missing from perm set.
Permission Set
Profile