android - How to get stride and Y plane alignment values for MediaCodec encoder

Question

There's a few related questions and discussions on this subject:

• Mediacodec and camera, color space incorrect
• Getting QualComm encoders to work via MediaCodec API
• https://groups.google.com/d/msg/android-platform/awaNwgb6EbY/a-YiIOwaL0QJ
• https://code.google.com/p/android/issues/detail?id=37769

I am feeding camera preview frames (NV21 converted to NV12) to MediaCodec encoder (NV12 aka COLOR_FormatYUV420SemiPlanar). It looks like that on some devices with QualComm encoder which are running Android versions less than 4.3 I have to do some input frames processing in order to receive back frame with correct color.

On Sony Xperia ZR running Android 4.2.2 I have to add Y plane alignment in order to make it working on almost all resolutions. Code above adds 1024 bytes alignment for widths which can not be divided by 32 and 2048 bytes alignment for other resolutions. It makes MediaCodec to encode frames properly for all resolutions which can be divided by 16 (except 176x144 for which UV plane looks misaligned).

int getYPadding() {
    if (mediaCodecInfo.getName().contains("OMX.qcom") && android.os.Build.VERSION.SDK_INT < 18) {
        if ((getWidth() % 32) != 0) {
            return (getWidth()*getHeight()) % 1024;
        } else {
            return (getWidth()*getHeight()) % 2048;
        }
    }
    return 0;
}

I've tried to test this alignment on LG G2 which is running same Android 4.2.2 and has QualComm encoder, and it looks like it does not working on it correctly. UV plane is misaligned (a green stripe at the bottom of the frame). I was not able to calculate the padding which will work for both phones.

I also have access to Sony Xperia Z1 running Android 4.3 with QualComm chipset and it looks like it does not have such problems. Video on every resolution looks fine and Y plane does not needs to be aligned anyhow.

I understand that it's hardware-related and might be complicated, but since I have to support users running Android prior to 4.3 I have a question. Is it possible to programmatically determine Y plane alignment and vertical / horizontal stride values which encoder is expecting for the given color format?

Answer 1

The problem in a nutshell: there were no CTS tests for video encoding until Android 4.3 (API 18).

As a result, the behavior of MediaCodec across different devices was inconsistent, and a few bugs went unnoticed. The EncodeDecodeTest tests exercise the functions you're asking about, and as a result you can reliably feed YUV data to a 4.3+ device (though you still have to runtime-detect whether it wants planar or semi-planar).

For your specific question, the Y plane on older Qualcomm devices needs to be aligned at a 2K boundary, which isn't quite what your code is doing. For 720p video this happens naturally (720*1280 == 450 * 2048), for 176x144 you'd adjust by 1280 to start the UV plane at 26624 instead of 25344. You need to set the absolute alignment within the buffer, not a fixed amount of padding -- use uvoffset = (width*height + 2047) & ~2047.

You will need to detect the codec vendor and Android software version, and if it's Qualcomm on pre-4.3 you need to make this adjustment. If your requirements change, and you can target API 18+, these issues go away. (And you can use Surface input to MediaCodec, which avoids the U/V swap issue, though depending on your needs that may not be useful.)