What happens if we have people install those packages while they are in review? =Do customers need to reinstall our package?
No, the package will work without being reviewed, although the package will count towards org setup limits (e.g. number of custom fields per object, etc) until the review completes. There is no need to reinstall the package after review.
can they upgrade from non-reviewed to reviewed version?
Yes. A managed package is a managed package, regardless of it is reviewed or not. There is no functional difference between these two, except as before, "Aloha Apps" do not count towards org setup limits.
What happens if we make changes and publish new versions while the initial submission is in review? Do we need to resubmit again (i know ONCE reviewed you do not have to, but we're a bit unclear on this 'updates while in review' case)
You'll need to ask about current policy. Last I checked (~2014), uploading a new version resulted in an automatic failure of the review and you had to start over (not including any fees already paid). In other words, you should probably wait until your review is complete.
Unless you have a critical must-fix problem, you can probably afford to wait until your review is complete. We had a mishap where we did upload a new version, and we had to do our review twice. Salesforce will only review the latest version, so you can't provide a "moving target" for them to review. It would be better to wait until the review concludes (even if it is a fail or conditional-pass) than to have to start over from the beginning.