It sounds like this is a one-time data update. If that is not the case, you may want to spend more time on this and consider an enterprise ETL solution and/or changes to your data model.
For a one-off update, I would use the Data Loader over Bulk API to pull down the entire tables for ObjectA and ObjectB.
Because 8 million rows is too much for either Excel or LibreOffice to handle, I would write a simple Python script using the
csv module to populate the required field on ObjectA from ObjectB and generate a new CSV file with the appropriate field populated. You could achieve the same workflow with your preferred scripting language.
If the volume of ObjectB is significantly less than ObjectA (< 1 million records), it would be feasible to use a programmer's editor to chunk the 8 million CSV records into smaller files and then populate the field in each file with an INDEX/MATCH construct using Excel or LibreOffice, avoiding the need to write any code at all.
I'd then uploaded that generated file back to Salesforce via Data Loader/Bulk API.