utf 8 - Java 8 change in UTF-8 decoding

Question

We recently migrated our application to JDK 8 from JDK 7. After the change, we ran into a problem with the following snippet of code.

String output = new String(byteArray, "UTF-8");

The byte array may contain invalid UTF-8 byte sequences. The same byte array upon UTF-8 decoding, results in two difference strings on Java 7 and Java 8.

According to the answer to this SO post, Java 8 "fixes" an error in Java 7 and replaces invalid UTF-8 byte sequences with a replacement string, which is in accordance with the UTF-8 specification.

But we would like to stick with Java 7's version of the decoded string.

We have tried to use CharsetDecoder with CodingErrorAction as REPLACE, REPORT and IGNORE on Java 8. Still, we were not able to generate the same string as Java 7.

Can we do this with a technique of reasonable complexity?

Answer 1

From the pointers provided by @Holger, It was clear that we had to write a custom CharsetDecoder.

I copied over OpenJDK's version of sun.nio.cs.UTF_8 class, renamed it to CustomUTF_8 and used it to construct a string like so

String output = new String(bytes, new CustomUTF_8());

I plan to run extensive tests cross verifying the outputs generated on Java 7 and Java 8. This is an interim solution while I am trying to fix the actual problem of passing output from hmac directly to String without Base64 encoding it first to.

 String output = new String(Base64.Encoder.encode(bytes), Charset.forname("UTF-8"));