Monthly Archives: June 2015

Multicast playback in Android

This article is to briefly introduce how to add multicast playback support in Android platform based on AOSP 5.0.0_r2 tag.

Background

  • NuPlayer is the default media player in Android L. It’s part of libmediaplayerservice located in frameworks/av/media/libmediaplayerservice
  • libstagefright is a library used by NuPlayer to provide a lot of media related functions, e.g. media download, parse and extract. Located in frameworks/av/media/libstagefright
  • chromium_org is a library used by Browser to provide all webview related functions. <video> tag in Webview also uses this library to render its content. Located in external/chromium_org
  • Gallery2 is a default system app that handles media playback intents. Located in packages/apps/Gallery

The above components will be changed to add multicast support.

Implementations

Precondition

Multicast as a kernel option, shall be enabled in kernel build.
Note: this option is enabled by default on Nexus Tablets, but disabled on Nexus Phones.
So you probably need to compile your custom kernel if you plan to test multicast on your phone.

  CONFIG_IP_MULTICAST=y
NuPlayer
  • MediaPlayerFactory::IFactory::scoreFactory() checks the URI scheme to decide which player shall be used to play a certain media.
    In NuPlayerFactory::scoreFactory(), multicast’s score shall be added to be chosen for playing multicast stream.
    if (!strncasecmp("http://", url, 7)
        || !strncasecmp("https://", url, 8)
        || !strncasecmp("file://", url, 7)) {
      … // return kOurScore
    }
    if (!strncasecmp("rtsp://", url, 7)) {
      return kOurScore;
    }
    if (!strncasecmp("udp://", url, 6)
        || !strncasecmp("igmp://", url, 7)) {
      return kOurScore;
    }
  • NuPlayer checks the URI scheme and creates different DataSource to receive media data. MulticastSource is added to handle multicast stream.
    In NuPlayer::setDataSourceAsync(), multicast shall be supported as below
	if (IsHTTPLiveURL(url)) {
	  … // new HTTPLiveSource
	} else if (!strncasecmp(url, "rtsp://", 7)) {
	  … // new RTSPSource
	}
	…
	else if (!strncasecmp(url, "udp://", 6)
	         || !strncasecmp("igmp://", url, 7)) {
	  source = new MulticastSource(notify, httpService, url, headers);
	}
	else {
	  … // Generic source
	}
  • MulticastSource inherits NuPlayer::Source and provide related functions. Basically it just handles events and pass all functions to MulticastSession, which is implemented in libstagefright.
libstagefright
  • Multicast socket related code shall be implemented to provide the socket interface.
  • MulticastSession uses the socket to receive media data, uses existing ATSParser to parse the TS stream, and extract so-called accessUnit to feed the decoder. It could be put at frameworks/av/media/libstagefright/multicast
    This class handles the main part of multicast playback.
  • MulticastDataSource is created by DataSource and used by MediaExtractor to extract the meta data of the media, e.g. whether it has video or audio, what the video resolution is, etc.
    This class is used in <video> tag playback, that webview will retrieve the media element’s metadata.
    Note: DataSource expects the media as static content, so it uses readAt(offset) to read the data from the source. But multicast is a stream, so MulticastDataSource shall be implemented carefully to provide the correct data with offset to DataSource.
  • DataSource shall be updated to create MulticastDataSource for multicast scheme.
	if (!strncasecmp("file://", uri, 7)) {
	  … // new FileSource
	} else if (!strncasecmp("http://", uri, 7)
	           || !strncasecmp("https://", uri, 8)
	           || isWidevine) {
	  … // new Http or Cache source
	} else if (!strncasecmp("data:", uri, 5)) {
	  … // new DataURISource
	} else if (!strncasecmp("igmp://", uri, 7)
	           || !strncasecmp("udp://", uri, 6)) {
	  source = new MulticastDataSource(uri);
	} else {
	  … // new FileSource
	}
chromium_org

By default, WebMediaPlayerAndroid does not support multicast scheme, and thus it treats the URI as a file and fails to load the media. WebMediaPlayerAndroid::load() shall be updated to support multicast scheme.

  if (url_.scheme() == "udp" || url_.scheme() == "igmp") {
    DVLOG(1) << "Do not load multicast";
    UpdateReadyState(WebMediaPlayer::ReadyStateHaveNothing);
    DidLoadMediaInfo(MediaInfoLoader::kOk, GURL(url), GURL(""), false);
    return;
  }
Gallery2

By default, Gallery2 does not handle multicast, and thus if an application sends a VIEW intent with multicast scheme, no application handles the intent and exception throws. AndroidManifest.xml shall be updated to support multicast scheme, so that it handles such VIEW intent.

            <intent-filter>
                <action android:name="android.intent.action.VIEW" />
                <category android:name="android.intent.category.DEFAULT" />
                <category android:name="android.intent.category.BROWSABLE" />
                <data android:scheme="udp" />
                <data android:scheme="igmp" />
                <data android:mimeType="video/*" />
                <data android:mimeType="audio/*" />
             </intent-filter>

Conclusion

The above is a simplified introduction of the changes needed to support multicast playback in AOSP.
Google will probably not officially add such support, but some fork may be interested in multicast. This could be a start of the work.

Share

读书笔记(2015.6)

继续记录一下前段时间读的书

  • QED:光和物质的奇妙理论
    后悔现在才看到的,费曼的超级牛B的书。
    光是波?不对。光是粒子?不完全是。波粒二象性?不重要——
    这本书的第一讲就“轻描淡写”地把光子这么一个神奇的东西的*性质*讲得清清楚楚,计算概率就行了。
    第二讲,顺便把直觉上光是直线传播这一宏观上似乎是对的概念,重新解释了一下——光可以沿任何路径传播。不信?看它如何描述光栅的,就信了。
    第三讲开始加入了电子,内力大增,理解起来有点难度了——但是如果只看结论,却是清晰无比,得到结论的“推理”(或者说计算)过程也是一目了然。
    最后一讲看完,真心觉得费曼太牛了,能把量子电动力学这么一个高深的学科,讲得如此清晰易懂。
    用他的话说,“一个笨人能理解的东西,其他人也能理解”,虽然,我们只是在数石子而已。
    QED
  • 神们自己
    阿西莫夫的科幻小说,创意十足,无论是平行世界,还是平行世界里的智慧生物,都有基本的物理理论来支撑,所以看上去完全不觉得是“幻想”,反而很真实,很过瘾。
    The_Gods_Themselves
  • 未来边缘
    大刘的短篇小说集之一,收录了《天使时代》《2018年4月1日》《微纪元》《赡养上帝》《命运》,以前都在《科幻世界》里看过了,不过亚马逊上特价的时候还是收了。多年之后重看高中、大学时期看过的科幻短篇,调剂一下也不错。
    未来边缘
  • Doom启世录
    抱歉这本下了盗版。。。因为没有中文版卖。。。
    之前是听鸟哥说起了这本书,找了个电子版看了。印象中小学的时候在486电脑上玩过doom(不确定了),对于这类美式的游戏也不太感冒,就这么瞎看看翻完了,没有太多感觉。也许只有真正的粉丝才能从这本书里看到那个时代的激情吧。
    当然,必须得说卡马克这样的人还是非常佩服的。
    Doom启世录
  • 一网打尽:贝佐斯与亚马逊时代
    也是亚马逊特价的时候买的。之前其实不太关心亚马逊是怎么一步步从卖书开始变成最大的网上零售商的,买这本书更多的是想看看亚马逊是怎么从卖东西的,变成云计算、云服务的提供商。可惜这本书里没有答案。
    不过倒是看到一家电商如何地压榨供应商,压榨员工,为客户提供更低的价格和更好的服务。这在以前简直是不可想像的。幸而亚马逊发展起来了,国内的亚马逊,京东们也在往这个方向走着。而淘宝走在了另外一条路上。。。
    The_Everything_Store
Share