I now have this working for the specific case of connecting back to Salesforce from Salesforce. So essentially it is a port of the Java sample code to Apex. So it appears that Apex's RSA-SHA256 is indeed equivalent to Java's SHA256withRSA.
Steps to get this to work were also:
- Create a certificate via Setup -> Security Controls -> Certificate and Key Management -> Create Self-Signed Certificate with a key size of 2048 and use that name in the Apex Crypto.signWithCertificate call.
- Create a connected app via Setup -> Create -> Apps -> Connected Apps -> New. Upload a copy of the cert created in the previous step to that and as a start select all the oauth scopes (you can cut them down later). Also use the "Manage" to add the profile of the User to the connected app.
In terms of using this protocol to connect to other identity providers such as Google, the (draft) spec says that:
Of the JWS signing algorithms, only HMAC SHA-256 and "none" MUST be
implemented by conforming JWT implementations. It is RECOMMENDED that
implementations also support the RSA SHA-256 and ECDSA P-256 SHA-256
algorithms. Support for other algorithms and key sizes is OPTIONAL.
The Crypto.generateMac method appears to support hmacSHA256 (alg "HS256" in JWT) so that might be the right choice in some cases.
PS
The Apex code is posted here An Apex implementation of the OAuth 2.0 JWT Bearer Token Flow.