Multicast playback in Android

1 Reply

This article is to briefly introduce how to add multicast playback support in Android platform based on AOSP 5.0.0_r2 tag.

Background

  • NuPlayer is the default media player in Android L. It’s part of libmediaplayerservice located in frameworks/av/media/libmediaplayerservice
  • libstagefright is a library used by NuPlayer to provide a lot of media related functions, e.g. media download, parse and extract. Located in frameworks/av/media/libstagefright
  • chromium_org is a library used by Browser to provide all webview related functions. <video> tag in Webview also uses this library to render its content. Located in external/chromium_org
  • Gallery2 is a default system app that handles media playback intents. Located in packages/apps/Gallery

The above components will be changed to add multicast support.

Implementations

Precondition

Multicast as a kernel option, shall be enabled in kernel build.
Note: this option is enabled by default on Nexus Tablets, but disabled on Nexus Phones.
So you probably need to compile your custom kernel if you plan to test multicast on your phone.

  CONFIG_IP_MULTICAST=y
NuPlayer
  • MediaPlayerFactory::IFactory::scoreFactory() checks the URI scheme to decide which player shall be used to play a certain media.
    In NuPlayerFactory::scoreFactory(), multicast’s score shall be added to be chosen for playing multicast stream.
    if (!strncasecmp("http://", url, 7)
        || !strncasecmp("https://", url, 8)
        || !strncasecmp("file://", url, 7)) {
      … // return kOurScore
    }
    if (!strncasecmp("rtsp://", url, 7)) {
      return kOurScore;
    }
    if (!strncasecmp("udp://", url, 6)
        || !strncasecmp("igmp://", url, 7)) {
      return kOurScore;
    }
  • NuPlayer checks the URI scheme and creates different DataSource to receive media data. MulticastSource is added to handle multicast stream.
    In NuPlayer::setDataSourceAsync(), multicast shall be supported as below
	if (IsHTTPLiveURL(url)) {
	  … // new HTTPLiveSource
	} else if (!strncasecmp(url, "rtsp://", 7)) {
	  … // new RTSPSource
	}
	…
	else if (!strncasecmp(url, "udp://", 6)
	         || !strncasecmp("igmp://", url, 7)) {
	  source = new MulticastSource(notify, httpService, url, headers);
	}
	else {
	  … // Generic source
	}
  • MulticastSource inherits NuPlayer::Source and provide related functions. Basically it just handles events and pass all functions to MulticastSession, which is implemented in libstagefright.
libstagefright
  • Multicast socket related code shall be implemented to provide the socket interface.
  • MulticastSession uses the socket to receive media data, uses existing ATSParser to parse the TS stream, and extract so-called accessUnit to feed the decoder. It could be put at frameworks/av/media/libstagefright/multicast
    This class handles the main part of multicast playback.
  • MulticastDataSource is created by DataSource and used by MediaExtractor to extract the meta data of the media, e.g. whether it has video or audio, what the video resolution is, etc.
    This class is used in <video> tag playback, that webview will retrieve the media element’s metadata.
    Note: DataSource expects the media as static content, so it uses readAt(offset) to read the data from the source. But multicast is a stream, so MulticastDataSource shall be implemented carefully to provide the correct data with offset to DataSource.
  • DataSource shall be updated to create MulticastDataSource for multicast scheme.
	if (!strncasecmp("file://", uri, 7)) {
	  … // new FileSource
	} else if (!strncasecmp("http://", uri, 7)
	           || !strncasecmp("https://", uri, 8)
	           || isWidevine) {
	  … // new Http or Cache source
	} else if (!strncasecmp("data:", uri, 5)) {
	  … // new DataURISource
	} else if (!strncasecmp("igmp://", uri, 7)
	           || !strncasecmp("udp://", uri, 6)) {
	  source = new MulticastDataSource(uri);
	} else {
	  … // new FileSource
	}
chromium_org

By default, WebMediaPlayerAndroid does not support multicast scheme, and thus it treats the URI as a file and fails to load the media. WebMediaPlayerAndroid::load() shall be updated to support multicast scheme.

  if (url_.scheme() == "udp" || url_.scheme() == "igmp") {
    DVLOG(1) << "Do not load multicast";
    UpdateReadyState(WebMediaPlayer::ReadyStateHaveNothing);
    DidLoadMediaInfo(MediaInfoLoader::kOk, GURL(url), GURL(""), false);
    return;
  }
Gallery2

By default, Gallery2 does not handle multicast, and thus if an application sends a VIEW intent with multicast scheme, no application handles the intent and exception throws. AndroidManifest.xml shall be updated to support multicast scheme, so that it handles such VIEW intent.

            <intent-filter>
                <action android:name="android.intent.action.VIEW" />
                <category android:name="android.intent.category.DEFAULT" />
                <category android:name="android.intent.category.BROWSABLE" />
                <data android:scheme="udp" />
                <data android:scheme="igmp" />
                <data android:mimeType="video/*" />
                <data android:mimeType="audio/*" />
             </intent-filter>

Conclusion

The above is a simplified introduction of the changes needed to support multicast playback in AOSP.
Google will probably not officially add such support, but some fork may be interested in multicast. This could be a start of the work.

Share