android - Retrieve album art using FFmpeg

Question

I'm developing an Android application that relies on FFmpeg to retrieve audio metadata. I know it's possible to retrieve album art programmatically using FFMpeg. However, once you have decoded the art (a video frame within an MP3) how do generate an image file (a PNG) for use within an application? I've search all over but can't seem to find a working example.

Edit, here is the solution:

#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>

void retrieve_album_art(const char *path, const char *album_art_file) {
    int i, ret = 0;

    if (!path) {
        printf("Path is NULL
");
        return;
    }

    AVFormatContext *pFormatCtx = avformat_alloc_context();

    printf("Opening %s
", path);

    // open the specified path
    if (avformat_open_input(&pFormatCtx, path, NULL, NULL) != 0) {
        printf("avformat_open_input() failed");
        goto fail;
    }

    // read the format headers
    if (pFormatCtx->iformat->read_header(pFormatCtx) < 0) {
        printf("could not read the format header
");
        goto fail;
    }

    // find the first attached picture, if available
    for (i = 0; i < pFormatCtx->nb_streams; i++)
        if (pFormatCtx->streams[i]->disposition & AV_DISPOSITION_ATTACHED_PIC) {
            AVPacket pkt = pFormatCtx->streams[i]->attached_pic;
            FILE* album_art = fopen(album_art_file, "wb");
            ret = fwrite(pkt.data, pkt.size, 1, album_art);
            fclose(album_art);
            av_free_packet(&pkt);
            break;
        }

    if (ret) {
        printf("Wrote album art to %s
", album_art_file);
    }

    fail:
        av_free(pFormatCtx);
        // this line crashes for some reason...
        //avformat_free_context(pFormatCtx);
}

int main() {
    avformat_network_init();
    av_register_all();

    const char *path = "some url";
    const char *album_art_file = "some path";

    retrieve_album_art(path, album_art_file);

    return 0;
}

Answer 1

To use ffmpeg programmatically, I think you would have to call read_apic() in libavformat (which is part of ffmpeg).

From the commandline, you can apparently do this:

ffmpeg -i input.mp3 -an -vcodec copy cover.jpg

The commandline behaviour implies that the cover art image is seen as just another video stream (containing just one frame), so using libavformat in the usual way you would to demux the video part of a stream should produce that image.

Sample code for demuxing: ffmpeg/docs/examples/demuxing.c The first (and only) AVPacket that would be obtained from demuxing the video stream in an mp3 would contain the JPEG file (still encoded as JPEG, not decoded).

AVFormatContext* fmt_ctx;
// set up fmt_ctx to read first video stream
AVPacket pkt;
av_read_frame(fmt_ctx, &pkt);
FILE* image_file = fopen("image.jpg", "wb");
int result = fwrite(pkt.data, pkt.size, 1, image_file);
fclose(image_file);

If there are multiple images, I think they would be seen as separate video streams, rather than as separate packets in the same stream. The first stream would be the one with the largest resolution.

All this is probably implemented internally in terms of read_apic().

The ID3v2 spec allows for any image format, but recommends JPEG or PNG. In practice all images in ID3 are JPEG.

EDIT: Moved some of the less useful bits to postscript:

P.S. ffmpeg -i input.mp3 -f ffmetadata metadata.txt will produce an ini-like file containing the metadata, but the image is not even referred to in there, so that is not a useful approach.

P.S. There may be multiple images in an ID3v2 tag. You may have to handle the case when there is more than one image or more than one type of image present.

P.S. ffmpeg is probably not the best software for this. Use id3lib, TagLib, or one of the other implementations of ID3. These can be used either as libraries (callable from the language of your choice) or as commandline utilities. There is sample C++ code for TagLib here: How do I use TagLib to read/write coverart in different audio formats? and for id3lib here: How to get album art from audio files using id3lib.