encryption - Decrypting salted AES file generated on command line with Ruby

Question

I would like to decrypt a text file within a ruby 2.1 script which was previously encrypted using OpenSSL's commandline tools: openssl enc -aes-256-cbc -a -salt -in my_file

As seen in the command, the file is AES-256-CBC encrypted, salted and base64 encoded.

The password is known, but not the IV nor the key, which are required to follow this code snippet, taken from the ruby documentation:

decipher = OpenSSL::Cipher::AES.new 256, :CBC
decipher.decrypt
decipher.key = key
decipher.iv = iv

plain = decipher.update(encrypted_text) + decipher.final

While trying to find an answer, I found the gem AESCrypt gem which supposedly simplifies en- and decrypting, yet the currently released version is not compatible with ruby 2.1. Looking at it's source code, I found that the key was retrieved by digesting the password, and the IV is just left as nil.

So I tried to get the following running:

encoded_and_encrypted_text = File.read my_file_path
encrypted_text = Base64.decode64 encoded_and_encrypted_text.to_s.strip

decipher = OpenSSL::Cipher::AES.new 256, :CBC
decipher.decrypt

decipher.key = OpenSSL::Digest::SHA256.new(my_password).digest

plain_text = decipher.update(encrypted_text) + decipher.final

But this results in OpenSSL::Cipher::CipherError: bad decrypt.

Do I need to somehow specifically handle that the file is salted? I have read in the OpenSSL documentation for the enc function that the IV, if not specified while encrypting the file, is generated from the password. Do I need to manually reconstruct the IV somehow?

Any advice would be highly appreciated :)

Answer 1

OpenSSL uses a custom header and key derivation routine. Security.SE has a good description of the header and the docs for EVP_BytesToKey describe the key derivation.

We can modify your code to use this weird and somewhat broken key derivation as follows:

encoded_and_encrypted_text = File.read my_file_path
encrypted_text = Base64.decode64 encoded_and_encrypted_text.to_s.strip

header = encrypted_text[0,8]
salt = encrypted_text[8,8]
payload = encrypted_text[16..-1]

decipher = OpenSSL::Cipher::AES.new 256, :CBC
decipher.decrypt

D_1 = OpenSSL::Digest::MD5.new(my_password + salt).digest
D_2 = OpenSSL::Digest::MD5.new(D_1 + my_password + salt).digest
D_3 = OpenSSL::Digest::MD5.new(D_2 + my_password + salt).digest

decipher.key = (D_1 + D_2)
decipher.iv = D_3

plain_text = decipher.update(payload) + decipher.final