There is a difference between hashing (which is what the SHA512 algroithm does) and encrypting.
A hash is a one-way function (or it's supposed to be). You put data in, you get a "hash" out, and that process cannot be reversed. Another property of a good hash function is that a small change in input leads to a large change in the output.
This is useful for things like passwords, where it's an extremely bad idea to store the actual data. Instead of storing the password, you store the hash of the password in your database. Then when the user enters their password, you generate the hash for their input (client-side or server-side) and compare that to the stored hash.
Encryption is a reversible process. You put text in, and provide a key, and you get a seemingly random string of text (the ciphertext) out. You can then run the ciphertext through the algorithm again (using the same key if it's "symmetrical" encryption, or a different (but related) key if it's "asymmetrical" encryption) to get the original text.
Encryption is useful when you want to transmit data between two parties (and both parties need to be able to see the original data). Using asymmetric encryption (aka Public-Key Cryptography) can also help validate the source of a piece of data (i.e this data can be decrypted by the public key, so it must have been generated by someone with the corresponding private key. If only 1 person is supposed to have the private key, then you can be reasonably sure that the data came from that person).
Crypto on Salesforce
Salesforce limits the algorithms we can use, and the ones we can use are documented in the Crypto class documentation
For hashing, we can use the MD5 (but don't use it if you can avoid doing so), SHA-1 (also don't), SHA256 and SHA512 functions.
For encrypting, we can use AES-128, AES-192, or AES-256 (more specifically, using the CBC, Cipher-Block Chaining, mode and PKCS#5 padding. We don't have a way to use other modes like GCM (Galois-Counter mode))
AES is a symmetric encryption algorithm (where the challenge is making sure both parties end up with the same key while trying to make sure nobody else knows/has the key). RSA is a family of asymmetric encryption algorithms, and our only real access to that in Salesforce is through
Crypto.sign(). The nuances of encryption vs digital signature are beyond our scope here.
For your use case
Hashing and comparing the result sounds like the way to go. I think that current best practice is something like:
- ...but add a unique-ish "salt" to the password before hashing it (so that if two people have the same password, they don't end up with the same hash. Also helps prevent leaked password hashes from other breaches from being re-used)
- Send the hash to SFDC (using SSL/TLS, basically just make sure you're using HTTPS, so that nobody else can intercept the hash)
- Compare the received hash to the one you have stored
That said, if there's a way you can integrate OAuth into this process, that'd likely be preferrable to implementing your own authentication mechanism.