Tag Archives: Android

Multicast playback in Android

1 Reply

This article is to briefly introduce how to add multicast playback support in Android platform based on AOSP 5.0.0_r2 tag.

Background

  • NuPlayer is the default media player in Android L. It’s part of libmediaplayerservice located in frameworks/av/media/libmediaplayerservice
  • libstagefright is a library used by NuPlayer to provide a lot of media related functions, e.g. media download, parse and extract. Located in frameworks/av/media/libstagefright
  • chromium_org is a library used by Browser to provide all webview related functions. <video> tag in Webview also uses this library to render its content. Located in external/chromium_org
  • Gallery2 is a default system app that handles media playback intents. Located in packages/apps/Gallery

The above components will be changed to add multicast support.

Implementations

Precondition

Multicast as a kernel option, shall be enabled in kernel build.
Note: this option is enabled by default on Nexus Tablets, but disabled on Nexus Phones.
So you probably need to compile your custom kernel if you plan to test multicast on your phone.

  CONFIG_IP_MULTICAST=y
NuPlayer
  • MediaPlayerFactory::IFactory::scoreFactory() checks the URI scheme to decide which player shall be used to play a certain media.
    In NuPlayerFactory::scoreFactory(), multicast’s score shall be added to be chosen for playing multicast stream.
    if (!strncasecmp("http://", url, 7)
        || !strncasecmp("https://", url, 8)
        || !strncasecmp("file://", url, 7)) {
      … // return kOurScore
    }
    if (!strncasecmp("rtsp://", url, 7)) {
      return kOurScore;
    }
    if (!strncasecmp("udp://", url, 6)
        || !strncasecmp("igmp://", url, 7)) {
      return kOurScore;
    }
  • NuPlayer checks the URI scheme and creates different DataSource to receive media data. MulticastSource is added to handle multicast stream.
    In NuPlayer::setDataSourceAsync(), multicast shall be supported as below
	if (IsHTTPLiveURL(url)) {
	  … // new HTTPLiveSource
	} else if (!strncasecmp(url, "rtsp://", 7)) {
	  … // new RTSPSource
	}
	…
	else if (!strncasecmp(url, "udp://", 6)
	         || !strncasecmp("igmp://", url, 7)) {
	  source = new MulticastSource(notify, httpService, url, headers);
	}
	else {
	  … // Generic source
	}
  • MulticastSource inherits NuPlayer::Source and provide related functions. Basically it just handles events and pass all functions to MulticastSession, which is implemented in libstagefright.
libstagefright
  • Multicast socket related code shall be implemented to provide the socket interface.
  • MulticastSession uses the socket to receive media data, uses existing ATSParser to parse the TS stream, and extract so-called accessUnit to feed the decoder. It could be put at frameworks/av/media/libstagefright/multicast
    This class handles the main part of multicast playback.
  • MulticastDataSource is created by DataSource and used by MediaExtractor to extract the meta data of the media, e.g. whether it has video or audio, what the video resolution is, etc.
    This class is used in <video> tag playback, that webview will retrieve the media element’s metadata.
    Note: DataSource expects the media as static content, so it uses readAt(offset) to read the data from the source. But multicast is a stream, so MulticastDataSource shall be implemented carefully to provide the correct data with offset to DataSource.
  • DataSource shall be updated to create MulticastDataSource for multicast scheme.
	if (!strncasecmp("file://", uri, 7)) {
	  … // new FileSource
	} else if (!strncasecmp("http://", uri, 7)
	           || !strncasecmp("https://", uri, 8)
	           || isWidevine) {
	  … // new Http or Cache source
	} else if (!strncasecmp("data:", uri, 5)) {
	  … // new DataURISource
	} else if (!strncasecmp("igmp://", uri, 7)
	           || !strncasecmp("udp://", uri, 6)) {
	  source = new MulticastDataSource(uri);
	} else {
	  … // new FileSource
	}
chromium_org

By default, WebMediaPlayerAndroid does not support multicast scheme, and thus it treats the URI as a file and fails to load the media. WebMediaPlayerAndroid::load() shall be updated to support multicast scheme.

  if (url_.scheme() == "udp" || url_.scheme() == "igmp") {
    DVLOG(1) << "Do not load multicast";
    UpdateReadyState(WebMediaPlayer::ReadyStateHaveNothing);
    DidLoadMediaInfo(MediaInfoLoader::kOk, GURL(url), GURL(""), false);
    return;
  }
Gallery2

By default, Gallery2 does not handle multicast, and thus if an application sends a VIEW intent with multicast scheme, no application handles the intent and exception throws. AndroidManifest.xml shall be updated to support multicast scheme, so that it handles such VIEW intent.

            <intent-filter>
                <action android:name="android.intent.action.VIEW" />
                <category android:name="android.intent.category.DEFAULT" />
                <category android:name="android.intent.category.BROWSABLE" />
                <data android:scheme="udp" />
                <data android:scheme="igmp" />
                <data android:mimeType="video/*" />
                <data android:mimeType="audio/*" />
             </intent-filter>

Conclusion

The above is a simplified introduction of the changes needed to support multicast playback in AOSP.
Google will probably not officially add such support, but some fork may be interested in multicast. This could be a start of the work.

Share

Android上的psf播放器 MinePsfPlayer

Leave a reply

很久以前就想在Android上做一个psf的播放器，耽搁了许久，现在总算搞出了第一个版本。

稍微介绍一下历史：
1) 最早是基于sexypsf library (http://projects.raphnet.net/#sexypsf)，在PSP上写了一个能播放psf音乐的demo；不过后来不知道什么原因，没有继续做下去；
2) 后来发现了Audacious，一个从XMMS派生出来的开源播放器，里面有psf plugin，从code里看，psf的解码也是基于sexypsf的，但里面还包含了psf2的lib，在linux上工作良好；
3) 于是用了这个lib，在SDL库的基础上写了一个demo，可以在Windows/Linux上可以播放psf的demo；
4) Android出来之后，就想在上面写个psf播放器，不过刚开始没有NDK，感觉搞不定；
5) NDK出来之后，终于可以在android上写一个播放psf的demo了，在模拟器上跑，效果惨不忍睹，解码得太慢了；当时还没有Android手机，以为性能会有问题，作罢；
6) 买了Milestone之后，在实际的手机上跑了一下，发现性能上基本上没问题，但是有各种问题（实际上是code里的bug），当时在搞短信震动的app，psf的播放又往后拖了很久；
7) 现在终于搞定了psf的播放器！（在手机上听各PS经典游戏的原声，内牛满面啊）

TODO: 最高优先级的当然是PSF2的播放，这样就可以在手机上听FFX的音乐了，但愿不会拖太久。。。

总之，欢迎下载~
https://play.google.com/store/apps/details?id=com.mine.psfplayer
MinePsfPlayer_GooglePlay

Share

杂记(2012.02)

3 Replies

过完年后就懒了，很久没写博了，还是随便写点吧。

· Google的Android Market可能因为app实在太多了，我的app很难被搜到、或者说搜到了排名也不靠前，因此下载量和安装量就基本上停滞了。
  不过偶尔还是能收到一些feedback邮件，其中最令我觉得宽慰的是，我的app帮到了一些聋人——短信、来电、邮件的震动以及定时的提醒对于他们来说就是最需要的功能。看到这样的feedback “This app is almost perfect – especially for a deaf person like me!”，想必每个开发者都会很开心的吧？管它多少下载量呢…

· 前段时间看了电影《那些年，我们一起追的女孩》，很有感触，因为它讲的故事太真实了，真实到电影里的台词，一模一样的话，曾经也有那么一个人对我说过。的确，每个男生心中都有一个沈佳宜，只不过，只是在心中而已。

· 预定的《最终幻想13-2》的中文版发售的第二天就入手了，这个月在好好地玩这款游戏。许久没有玩到好玩的RPG的我，显然对这款Fami通满分游戏也赞不绝口——不说别的，沿用FF13唯一的亮点，战斗系统，再估一个新游戏出来，只要各方面别太差，就是个好游戏了。而FFB-2的各个细节都很赞，尤其是每次读取进度时的“前情提要”，让人有种看美剧的感觉，也能回忆起之前的剧情；而Noel’s theme也是听一次就会喜欢上的音乐；剧情是弱了点，但是Live Trigger和多结局的设置，通关后可以随意体验每一段故事，和“穿越时空”的主题也很契合。
  目前已经看了各个悖论结局，人物也已经升到顶级，我的评价就是，不愧为满分游戏！

· 最近项目很忙，很想吐槽，但是又不知道从何吐起。总觉得各个项目之间的管理很混乱，或者说就没有这样的管理，因此时不时就会发现，两个独立的项目，居然之间有dependency，让人很无语。不过，这种情况应该怎么处理，也是件难事，也许SVN就是不适合有太多分支同时多个项目共存的情况吧。

Share

[Android Dev] Gmail OAuth (2)

4 Replies

这一部分主要介绍如何使用已获得的Access Token去获取Gmail以及计算未读邮件的数量。

1. 从上一篇博客里获得的AccessToken以及Secret，生成OAuthConsumer，非常简单。

CommonsHttpOAuthConsumer consumer =
  new CommonsHttpOAuthConsumer("anonymous", "anonymous");
consumer.setTokenWithSecret(token, secret);

2. Gmail提供了RSS Feed，只需要使用token取得RSS Feed，就拿到了Gmail的内容。其中：
https://mail.google.com/mail/feed/atom/ 对应Inbox里的未读邮件
https://mail.google.com/mail/feed/atom/unread/ 对应所有的未读邮件
https://mail.google.com/mail/feed/atom/labelname/ 对应某个label的未读邮件
我只需要取得用户的Inbox的未读邮件，因此code是这样的:

HttpGet request = new HttpGet("https://mail.google.com/mail/feed/atom/");
// sign the request
try {
  consumer.sign(request);
} catch (Exception e) {
  e.printStackTrace();
}
// send the request
HttpClient httpClient = new DefaultHttpClient();
org.apache.http.HttpResponse response;
try {
  response = httpClient.execute(request);
  feedString = read(response.getEntity().getContent());
} catch (Exception e) {
  e.printStackTrace();
}

feedString里就保存了Google返回的feed的内容。而函数read是为了把response里的内容读出来生成String. 这个是从网上抄来的…

private static String read(InputStream in) throws IOException {
  StringBuilder sb = new StringBuilder();
  BufferedReader r = new BufferedReader(new InputStreamReader(in), 1000);
  for (String line = r.readLine(); line != null; line = r.readLine()) {
    sb.append(line);
  }
  in.close();
  return sb.toString();
}

3. 有了feed，就剩下计算unread count的工作了。这分成两部分。
i) 从String生成XML Document，其实是一个DOM tree;
ii) 在XML Document里parse数据，取得unread count.
关于i) 也是网上有相应的code:

private static Document XMLfromString(String xml){
  Document doc = null;
  DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
  try {
    DocumentBuilder db = dbf.newDocumentBuilder();
    InputSource is = new InputSource();
    is.setCharacterStream(new StringReader(xml));
    doc = db.parse(is);
  } catch (ParserConfigurationException e) {
  // XML parse error
  return null;
  } catch (SAXException e) {
  // Wrong XML file structure
  return null;
  } catch (IOException e) {
  //I/O exeption
  return null;
  }
  return doc;
}

关于ii) Gmail的一个个邮件是放在一个个entry里的，所以只需要parse entry就行了。

NodeList nodes = feedDoc.getElementsByTagName("entry");
unread_count = nodes.getLength();

这就搞定了未读邮件数量的计算。

4. 虽然正常情况下都没问题，但是假如用户改了密码，token就无效了，应该检测这种情况，然后让用户重新进行authenticate。虽然这不常见，但也得处理。
假如Token变得无效，Gmail会返回一个显示Unauthorized的网页。因此最简单的方法是通过字符串判断. 注意如果是正常的feed，string是以<?xml为开头的，而如果是unauthorized的网页，就以<HTML>开头。

private static boolean verifyValid(String doc) {
  return !(doc.startsWith("<HTML>") && doc.contains("<TITLE>Unauthorized"));
}

这样就解决了。

Q.E.D.

Share