MKV Support on Xbox One

This blog post captures information regarding MKV support on Xbox One that should be published soon on support.xbox.com.  Please refer to the Xbox support site for the latest updates.

Edit: The official post is now located at:
http://forums.xbox.com/xbox_early_access/update_preview/update_preview_forums/wave_8_1410/f/5117/t/1833639.aspx
Note that only those with access to Preview Xbox builds will be able to see it there, for others, see below.

Note that I will try to answer those questions I can here:

http://redd.it/2ghejo

 

MKV support on Xbox One

We are proud to announce that some Xbox One users on the Preview program have begun to see support for the MKV file format in the MediaPlayer and Video apps.  This was released with the 1410 preview builds, and will be updated periodically in the days to come.  The feature is targeted for general availability in the October console update.

In this article, we would like to set expectations for this feature’s capabilities today.

Background:

When one uses the term MKV, they are referring to the MKV container format, also known as Matroska container format which is defined at http://matroska.org.  The MKV container can support multiple video and audio codecs, such as H.264 or AAC audio.  In general, containers deal with how video and audio data is laid out, and what supplementary information is used to describe those A/V streams.  Containers can also contain data which complements A/V streams, such as the title, the languages of the audio streams, the subtitle or caption tracks, the fonts for those subtitles, images, chapter information, menus, etc.  MKV is a highly flexible format which supports many of these container features.

Our hope is to provide those MKV features which are most important to the vast majority of our users.  As such, it should be understood that not all features will be implemented; and that all features will be delivered in the order in which they provide the most value.  We listen to and appreciate both feedback and requests for features.

MKV Container Feature support:

As of this writing, mid September 2014, the following MKV container features are supported:

  • One or more Video tracks, the first track will be played
  • One or more Audio tracks, the first track will be played
  • One or more Captions tracks, the captions will not render, but the file will load and play
  • One or more attached fonts or images, captions and images will not render, but the file will load and play
  • Menu information is not supported, but the file will load and play
  • Chapter information is not supported, but the file will load and play
  • Files with Chapters that refer to supplemental files will fail to play additional files
  • Thumbnail images are available when browsing for files on USB drives using the file browser

This set of features should allow playback of most MKV files if they contain supported codecs.

MKV Codec support:

MKV files that contain video and audio tracks encoded with the following codecs are expected to play:

 

Matroska Id MSFT Media Foundation MF_MT_SUBTYPE Description FourCC or WAV identifiers
V_MPEG4/ISO/AVC MFVideoFormat_H264 H.264 video. H264
V_MPEG2 MFVideoFormat_MPEG2 MPEG-2 video.
V_MPEG1 MFVideoFormat_MPG1 MPEG-1 video. MPG1
V_MPEG4/MS/V3 MFVideoFormat_MP43 Microsoft MPEG 4 codec version 3. MP43
V_MPEG4/ISO/ASP MFVideoFormat_MP4V MPEG-4 part 2 video. MP4V
V_MS/VFW/FOURCC Maps to several codecs usually supported in the AVI format which are available on the console.
A_AAC MFAudioFormat_AAC Advanced Audio Coding (AAC). WAVE_FORMAT_MPEG_HEAAC
A_AC3 MFAudioFormat_Dolby_AC3 Dolby Digital (AC-3).
A_MPEG/L3 MFAudioFormat_MP3 MPEG Audio Layer-3 (MP3). WAVE_FORMAT_MPEGLAYER3
A_MPEG/L1 MFAudioFormat_MPEG MPEG-1 audio payload. WAVE_FORMAT_MPEG
A_PCM/INT/BIG MFAudioFormat_PCM Uncompressed PCM audio. WAVE_FORMAT_PCM
A_PCM/INT/LIT MFAudioFormat_PCM Uncompressed PCM audio. WAVE_FORMAT_PCM
A_PCM/FLOAT/IEEE MFAudioFormat_Float Uncompressed IEEE floating-point audio. WAVE_FORMAT_IEEE_FLOAT

Technical details regarding codecs:
http://www.matroska.org/technical/specs/codecid/index.html
http://msdn.microsoft.com/en-us/library/windows/desktop/aa372553(v=vs.85).aspx
http://msdn.microsoft.com/en-us/library/windows/desktop/aa370819(v=vs.85).aspx

 

Posted in MKV | Comments Off

Xbox announces upcoming MKV support

Xbox announced MKV support at gamescom this week.  This was picked up and blogged by MajorNelson and is seeing rave reviews on Reddit.

I’m glad to see the excitement for MKV, as it’s what I’ve been working on these days!  I’m interested in feature requests and to make sure we have support for the most common scenarios.

Posted in MKV | Comments Off

HLS v3 – The new old thing

Azure Media Services now supports a down-level version of HLS, HLS v3.  
Use: format=m3u8-aapl-v3

A little history:

As of version 4, September 2011, Apple/HLS added support for multiple-languages: Alternative Audio Renditions these were called. Multi-language support was the motivation behind choosing HLS v4 two years ago for our service. At that time, Android was seeing a lot of active development and with its largely international customer base, it was assumed that Android would adopt HLS v4 for these multi-lingual markets. Not so, as history would prove. Here we are two and a half years later and HLS v4 support is only available on the very latest Android drops. So, instead, we are reaching back and dusting off the HLS v3 draft spec from November 2010, and building backward compatibility. We are also motivated by the connected TV space, which has a lower adoption rate for device updates.

To use HLS v3, simply use:

*.ism/manifest(format=m3u8-aapl-v3)

This will mux the default audio language into the video.

But what about multi-language scenarios, since HLS v3 doesn’t support those?

HLS v4 allows video and audio to be delivered in a decoupled fashion (like Smooth, DASH and HDS), whereas HLS v3 must have the audio muxed into the video data, and both are delivered as a single stream. Since HLS v3 does not have native multi-language support, but our server does, we were able to allow content-owners to set the audio track that they want muxed in the URL using the ‘audioTrack’ attribute. If you don’t set anything, we just mux in the default audio.

You can force the language that you want muxed into the video by using the audioTrack attribute.

It looks like this:

*.ism/manifest(format=m3u8-aapl-v3,audioTrack=audio_eng)

*.ism/manifest(format=m3u8-aapl-v3,audioTrack=audio_spa)

Or

*.ism/manifest(format=m3u8-aapl-v3,audioTrack=audio)

*.ism/manifest(format=m3u8-aapl-v3,audioTrack=audio_1)

Depending on if your audio languages were detected/defined in the source media.

You can get the audioTrack name from the Smooth manifest Language, or HLS v4 manifest.

You can set your own track names by using the title=”audio_spa” attribute in the .ism file used for your multi-bitrate MP4 assets.  Similarly, you can use systemLanguage=”spa” to set the Language attribute in the manifests. Add the attributes to the line in the .ism which has the src=”yourAudio.mp4”. You can also do some renaming in the .ism/.ismc of Smooth assets if you’re feeling brave. Sorry there’s no support for this type of manipulation in the platform, perhaps some of you would like to contribute to the Azure Media Services .NET SDK Extensions project on GitHub?  For doing these types of touch-ups, there is a manifest parser/generator from Mariano, one of our friends at SouthWorks.

So now HLS works on all Android devices, right?

Nope, not a chance.

Android is highly fractured and has varying levels of implementation for streaming media. The media stacks are different from device to device, and even within a single device the media stack can be duplicated/different. If you’ve been trying to make this work for a while, you’ve seen that the same Android version may work differently on different handsets; that’s because each manufacturer has the option of tweaking the OS. All to say, it’s not an easy problem to solve, and as a stream provider, the best I can do is make sure my streams are conformant – device support/upgrades is up to Android.

We also did some deeper testing of 4.1.2 debug builds which demonstrated that the quality of the underlying native implementation was low for that release. Some manufacturers had forked the code and done some fixes, but generally, I would not expect devices 4.1.2 and below to work. This is consistent with the findings of other streaming server and player framework providers.

For the Android devices with version 4.2.2, it would seem that there are varying degrees of implementation. Some adaptive bitrate logic implementation is particularly poor (dropped frames are not triggering bitrate switching, latching to a particular bitrate or bitrate switching causes a re-start). Other devices with 4.2.2 do reasonable well, and the latest Android drops are much better.

The MSDN doc will have a table on the device testing we were able to accomplish as part of building this feature, it is in no way extensive, but does cover a good number of popular devices that can be purchased today. Just to be clear, the goal of HLS v3 is to provide spec-compliant HLS streams which pass Apple validator and other Transport Stream compliance suites; not to ‘work on all Android devices’, so your millage may vary, take it up with the the other guys.

On the connected TV side, we’ve gotten good feedback from our partners thus far.  We also got a thumbs up from the popular JW player folks (note, set type:”HLS” in the embed code for JW Player).

Posted in Uncategorized | Tagged , , , | Comments Off