You are facing several challenges here, so let's go briefly over them first.
First and foremost - XLSX is a binary format. More specifically it is a zip file containing the actual data in XML form. Second - the format itself is incredibly complex and complicated and you would need to do a lot of parsing there. Third - all this handling will need a lot of resources and you're very likely to hit the governor limits sooner or later.
So - how can you solve this?
While zip handling is possible natively it basically means coding an entire zip unpack algorithm and due to the nature of Apex, this will have several restrictions. Then there's the XML handling and depending on the size of your XLSX/XML this is probably the easiest part if you can make sure the files are mostly consistent. You can find out more here: https://docs.fileformat.com/spreadsheet/xlsx/
The problem with this solution is: It's a lot of work. I've built zip file handling in Apex myself and it's no joke because of the tons of restrictions you're facing - where you really need to be very creative. And on top of that, you have to build some XLSX handling from scratch - none of this can be found on the web right now, as far as I'm aware.
But fear not! There is an alternative! If you do not need to do this in a batch and more on a per request basis by a user you can do all of this directly in the browser.
There's a library for zip files:
https://stuk.github.io/jszip/
And a library for handling XLSX files:
https://github.com/SheetJS/sheetjs
Hope this will help. Cheers!