OK, so we didn't find a direct way to do it. However, we believe that the following approach should work for normal and community usage. In essence we implemented a component with navigation mixin then used something like:

    this[NavigationMixin.GenerateUrl]({
			type: 'standard__navItemPage',
			attributes: {
				apiName: "customTabName",
			},
			// query string parameters
			state: {
				c__showPanel: 'true' // Value must be a string
			}
		}).then(url => {
			window.open(url)
		});

Importantly, this doesn't actually navigate directly to a Visualforce page, but rather to a custom tab into which the page is placed (clearly the tab needs to be accessible by the user). Note the use of window.open allows the URL to be opened in a separate tab or window within the browser.

**EDIT**: The "query string parameters" included do NOT seem to be correctly available within the Visualforce page - this because of the use of iframes across domains. Still working on a solution for this at the moment.