encryption - AES - simple encrypt in Java, decrypt with openssl

Question

I am trying to do a simple AES encryption in Java, using Java Cryto, that can then be decrypted in ObjectiveC, using OpenSSL.

as I am not doing the ObjectiveC side, I want to make sure it works, using the openSSL command line, but I always get "bad magic number"

Here is my Java code

public class EncryptionUtils {

private static final String AES_CIPHER_METHOD = "AES";
private static final int AES_KEY_SIZE = 128;

public static byte[] generateAesKey() throws NoSuchAlgorithmException {
    KeyGenerator keyGenerator = KeyGenerator.getInstance(AES_CIPHER_METHOD);
    keyGenerator.init(AES_KEY_SIZE);
    SecretKey key = keyGenerator.generateKey();
    return key.getEncoded();
}

public static SecretKeySpec createAesKeySpec(byte[] aesKey) {
    return new SecretKeySpec(aesKey, AES_CIPHER_METHOD);
}

public static void aesEncryptFile(File in, File out, SecretKeySpec aesKeySpec) throws InvalidKeyException, NoSuchAlgorithmException, NoSuchPaddingException, IOException {
    Cipher aesCipher = Cipher.getInstance(AES_CIPHER_METHOD);
    aesCipher.init(Cipher.ENCRYPT_MODE, aesKeySpec);
    InputStream inputStream = new FileInputStream(in);
    try {
        OutputStream outputStream = new CipherOutputStream(new FileOutputStream(out), aesCipher);
        try {
            IOUtils.copy(inputStream , outputStream);
        } finally {
            outputStream.close();
        }
    } finally {
        inputStream.close();
    }
}
}


//testcode
@Test
public void testAesEncryptFile() throws IOException, InvalidKeyException, IllegalBlockSizeException, BadPaddingException, NoSuchAlgorithmException, NoSuchPaddingException {
    byte[] aesKey = EncryptionUtils.generateAesKey();
    SecretKeySpec aesKeySpec = EncryptionUtils.createAesKeySpec(aesKey);
    EncryptionUtils.aesEncryptFile(new File("C:\test\test.txt"), new File("C:\test\test-encrypted.txt"), aesKeySpec);

    FileOutputStream outputStream = new FileOutputStream("C:\test\aes.key");
    outputStream.write(aesKey);
    outputStream.close();
}

@Test
public void testAesDecryptFile() throws IOException, InvalidKeyException, IllegalBlockSizeException, BadPaddingException, NoSuchAlgorithmException, NoSuchPaddingException {
    FileInputStream keyFis = new FileInputStream("C:\test\aes.key");
    ByteArrayOutputStream keyBaos = new ByteArrayOutputStream();
    IOUtils.copy(keyFis, keyBaos);

    SecretKeySpec keySpec = new SecretKeySpec(keyBaos.toByteArray(), "AES");
    Cipher cipher = Cipher.getInstance("AES");
    cipher.init(Cipher.DECRYPT_MODE, keySpec);

    FileInputStream fileInputStream = new FileInputStream("C:\test\test-encrypted.txt");
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    IOUtils.copy(fileInputStream, baos);

    byte[] decrypted = cipher.doFinal(baos.toByteArray());
    FileOutputStream outputStream = new FileOutputStream("C:\test\test-decrypted.txt");
    outputStream.write(decrypted);
    outputStream.close();

}

Now that runs as expected, file "test-encrypted.txt" is indeed encrypted, and "test-decrypted.txt" == "test.txt"

I then tried to run a decryption on the command line using OpenSSL

openssl enc -d -aes-128-ecb -in test-encrypted.txt -k aes.key

however, this always give me

bad magic number

From what I can see, the using algorithm "AES" in Java uses "ECB" mode by default, so the above should work. What am I doing wrong.

Answer 1

The problem is indeed due to the key that is computed from the password by OpenSSL.

Most likely the reason is that OpenSSL has its own algorithm to derive a key, EVP_BytesToKey, from the password, and that is not the same as Java's.

The only solution I found was to use a Java reimplementation of that algorithm:

private static final int KEY_LENGTH = 32;    

private byte[] deriveKey(String encryptionPassword, byte[] salt) throws NoSuchAlgorithmException {
    final byte[] passAndSalt = ArrayUtils.addAll(encryptionPassword.getBytes(), salt);
    byte[] hash = new byte[0];
    byte[] keyAndIv = new byte[0];
    for (int i = 0; i < 3 && keyAndIv.length < KEY_LENGTH; i++) {
        final byte[] dataToHash = ArrayUtils.addAll(hash, passAndSalt);
        final MessageDigest md = MessageDigest.getInstance("SHA-256");
        hash = md.digest(dataToHash);
        keyAndIv = ArrayUtils.addAll(keyAndIv, hash);
    }
    return Arrays.copyOfRange(keyAndIv, 0, KEY_LENGTH);
}

ArrayUtils is part of Apache Commons library.

Full usage:

IvParameterSpec initializationVectorSpec = new IvParameterSpec(
                Hex.decodeHex(encryptionInitializationVector.toCharArray()));

cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
byte[] salt = new SecureRandom().generateSeed(8);
byte[] key = deriveKey(encryptionPassword, salt);
Key keySpec = new SecretKeySpec(key, "AES");
cipher.init(Cipher.ENCRYPT_MODE, keySpec, initializationVectorSpec);

byte[] rawEncryptedInput = cipher.doFinal(input.getBytes());
byte[] encryptedInputWithPrependedSalt = ArrayUtils.addAll(ArrayUtils.addAll(
                "Salted__".getBytes(), salt), rawEncryptedInput);
return Base64.getEncoder()
                .encodeToString(encryptedInputWithPrependedSalt);

Credit to this answer for showing me the way.