When encrypting and decrypting with RSA in Java, which key do you init the Cipher with, and what is the role of ENCRYPT_MODE vs DECRYPT_MODE?
answer
- public key encrypts, private key decrypts (for confidentiality)
- init(mode, key) before doFinal
- ENCRYPT_MODE + public; DECRYPT_MODE + private
- signing is the opposite direction - use Signature, not Cipher
- Cipher is stateful and not thread-safe
basics
~10 sTo encrypt, init the Cipher in ENCRYPT_MODE with the recipient's public key. To decrypt, init in DECRYPT_MODE with the matching private key. Public key locks, private key unlocks.
solid answer
~40 sRSA uses a key pair. For confidentiality, the sender encrypts with the recipient's public key (cipher.init(Cipher.ENCRYPT_MODE, publicKey)) and only the holder of the private key can decrypt (cipher.init(Cipher.DECRYPT_MODE, privateKey)). The first argument to init is the operation mode constant, the second is the Key. A Cipher is stateful: you must call init before doFinal, and re-init to reuse it (it is also not thread-safe). The public-key-encrypts / private-key-decrypts direction is for encryption; the opposite direction (private signs, public verifies) is signing, which is done via the separate Signature class, not Cipher. A common mistake is encrypting with the private key to 'hide' data - that does not provide confidentiality because the public key is, by definition, public.
go deeper
Knows public key encrypts and private key decrypts, and that you call init with a mode constant and a key before doFinal.
Distinguishes encryption (public encrypts / private decrypts) from signing (private signs / public verifies via the Signature class), and knows Cipher is stateful.
Explains why private-key 'encryption' gives no confidentiality, the WRAP/UNWRAP modes for key wrapping, and thread-safety constraints.
Defines key-handling policy (where private keys live - HSM/KMS), key rotation, and which operations belong to Cipher vs Signature in the system design.
## The two keys and what each does RSA gives you a **key pair**: - a **public key** - safe to publish to anyone; - a **private key** - must stay secret with its owner. The math is set up so the two operations are inverses: data transformed with one key can only be reversed with the *other*. This gives two distinct use cases, and beginners often confuse them: 1. **Encryption (confidentiality):** encrypt with the **recipient's public key**, decrypt with the **recipient's private key**. Anyone can encrypt a message *to* you; only you can read it. 2. **Signing (authenticity):** the *opposite* direction - the owner transforms with their **private key**, anyone verifies with the **public key**. In Java this is the separate `java.security.Signature` class, **not** `Cipher`. ## Initializing the Cipher After `Cipher.getInstance(...)`, the Cipher is inert until you `init` it: ```java cipher.init(Cipher.ENCRYPT_MODE, recipientPublicKey); byte[] ct = cipher.doFinal(message); // ... on the recipient side ... cipher.init(Cipher.DECRYPT_MODE, recipientPrivateKey); byte[] pt = cipher.doFinal(ct); ``` The **first argument** is an `int` operation-mode constant: - `Cipher.ENCRYPT_MODE` - `Cipher.DECRYPT_MODE` - (also `WRAP_MODE` / `UNWRAP_MODE`, used to encrypt/decrypt *keys* - relevant for hybrid encryption.) The **second argument** is a `Key`. For RSA encryption it is typically a `PublicKey`; for decryption a `PrivateKey`. The JCE will reject an obviously wrong key type, but it cannot stop you from picking the *logically* wrong key (e.g., using your own private key when you meant the recipient's public key). ## Statefulness and threading A `Cipher` instance is **stateful** and **not thread-safe**. You must `init` before each independent operation, and you cannot share one instance across threads. Treat it as a short-lived, per-operation object. ## The classic confidentiality mistake Encrypting with your **private** key does **not** hide anything: because the public key is published, anyone can decrypt it. That operation is only meaningful as the basis of a *signature* (proving you, the private-key holder, produced it). So: to keep a secret, always encrypt with the **public** key of whoever should be able to read it.
- If you want to prove you authored a message, which key and which class do you use?Sign with your private key using java.security.Signature; others verify with your public key. Cipher is for confidentiality, not signing.
saying these in an interview costs you the question
- Encrypting with the private key to 'protect' data (public key can decrypt it)
- Confusing encryption (public encrypts) with signing (private signs)
- Calling doFinal without init
- Sharing a single Cipher instance across threads