Mobile Video Consumption – Making the Files

While I’ll easily admit that a lot of the things I do here at MobilityDojo HQ isn’t what most people find entertaining, (I’ve heard rumors that there are non-technical people who thinks there’s something wrong with activities like sitting up late at night processing Wireshark dumps), I do take some time off the techie stuff and just lie on the bed reading or watching a movie or something. This coupled with the fact that I just got a Samsung Galaxy Tab inspired me to document how I’ve been trying to build an infrastructure for consuming videos at home and on the road. It will be a multi-part article, but don’t worry – I’ll try to make it as simple as it gets Smile

For most media you download off the net you can just download the file directly to the device you want to use, or transfer it from your desktop. If the media file is in some obscure format you either need to download a codec, or try throwing VLC at it. Nothing wrong with this, but thing is – sometimes I have content that I’d like to re-encode into a new file. (Or transcode as it’s also known as.) A common reason is that the file is in a format that takes up a lot of storage space, but it could also be that I want it with a more cross-platform friendly codec. (VLC isn’t available on all devices.)

These files could for instance be:
– DVDs.
– Recorded TV shows from my Windows Media Center (either in wtv or dvr-ms format).
– HD video from my Sony camcorder in AVCHD format.

All of the above produce files that are far bigger than they need to be, and if I were to load up an entire season of some TV series it would quickly fill up my mobile devices (it’s not like the desktop computers suffers from this restriction). I want them in a more convenient format, and a format that will actually play back on all devices as well.

image

So first things first – do I need to change the amount of pixels, or is it sufficient to just produce a smaller file?
Let’s see – I’ve got a couple of different resolutions to account for and I want it to look decent on all of these:
MacBook Air – 1440×900 (16:10)
Samsung Galaxy Tab – 1024×600 (17:10) 
Apple iPhone – 3GS: 480×320 (15:10) / iPhone 4:960×640 (15:10) 
Windows Phone 7 – 800×480 (15:9)

Aspect ratios in parenthesis for each device. (I have other things like a HDTV and a high resolution LCD monitor, but the list above gives an idea of what’s a “minimum” to support.)

Older iPhone models have the same resolution as the 3GS as far as I know, but they probably don’t have enough CPU to pump through pixels at the same rate. The iPhone 4 provides a big boost in resolution, and even though the physical dimensions are pretty much the same as the 3GS it should provide a crisper experience. iPhone 4 is also able to handle higher resolutions than the 3GS.

Of course you can’t custom encode videos at all the different resolutions. Well, technically you can, but it wouldn’t make sense as long as the source usually comes in one specific resolution with a corresponding aspect ratio and might not look good with a different ratio. Not to mention the amount of CPU time it would take to produce four output files for each input file. The best option is of course if you are able to maintain the original resolution, and be able to just reduce the bitrate to reduce the size. The device should be able to handle the up or downscaling of the content when going into full screen mode. I assume for the sake of simplicity that my devices will be able to handle 720p content as long as I keep the bitrate at a sane level. (Testing will reveal if that is a sane assumption.)

A more important consideration is possibly what kind of file format to use. It made sense for me to attempt to use mp4/H.264 as it should be available on most platforms being a more or less open standard. I want to be able to use the default video player supplied by the OS as far as possible. These are more likely to provide hardware-acceleration if available, and when I get a new device I can be fairly confident media files will still work without further configuration and download of third-party apps. Of course I might prefer to use a different player in some instances simply because I like the interface better or something, but it should still work directly in the software supplied by the OS.

At this point I faced what turned out to be a difficult choice – how to rip/encode to my chosen output; or more specifically what software to use. Basing myself on Windows as the operating system was almost a choice by default. I have a Mac, and OSX is probably not a bad choice for video editing, but I know the Air is grossly underpowered for a task like this so I didn’t pursue this path any further. Linux is probably not a bad choice either, but while I could load it into a virtual machine I don’t think I’ll be getting the most out of my CPU cores that way either. (I have no spare computers available that aren’t ever so slightly outdated, and as such aren’t what I consider suitable for a Linux powerhouse.) So, it became Windows on the OS side.

Even though Windows has some basic encoding software freely available from Microsoft these didn’t provide me the feature set I needed so I had to evaluate a couple of other applications. I’ll spare you the details, but I tested out Haali, FFDShow, Handbrake, and a couple more. I either had some codec issue, where I had to run it through multiple programs to get the output I wanted, or I wasn’t entirely happy with the output. Handbrake was promising, but I had problems tuning the settings to give my desired output. It’s also sort of picky on the input side (with some minor bugs handling DVDs). Handbrake has come out with a new version while I’ve been working on this article though after a long release break (the dev team themselves admit it was long overdue), so I will do some re-testing, and it might still be included as a backup utility in my toolbox.

I ended up using Microsoft Expression Encoder 4 Pro since it supports the outputs I want, and most of the inputs I have. Thing is – the version you can download from MSDN does not contain all the codecs. It is fully functional, but the important H.264 codec is not present. You have to get the retail edition, as in “pay money”, to get these codecs. (It’s a royalty issue where Microsoft has to pay license fees so they are unable to provide it for free unless they take the cost themselves. Which I understand that they don’t want to do.) If you live in the US you can have it for 199 dollars (possibly 49 if the upgrade option from the MSDN version is still valid). For the rest of the world, including yours truly, multiply by a factor of 1.5 – 2.0 depending on where you live to get your price! (For mainland Europe it seems to be 255 Euros, and in the UK it doesn’t seem you can buy just the Encoder part of the Expression Suite…)

image

Encoder seems to be a decent enough program though, and while I understand it’s not a hard-core video editing application it suits my basic needs for transcoding and simple edits. The way it can be used in combination with the rest of the Expression Suite is nice (I might be needing this later on in the process). While not a requirement per se it’s also nice that there’s an SDK that will let you create your own apps on top of the encoding engine.

Most of you probably know this already, but I thought I’d repeat it at this point in time nonetheless; encoding video to H.264 is very CPU intensive. If you have a low-spec CPU you better learn to be patient. The amount of RAM is less important as Encoder is a 32-bit program, which means you’ll max out the program as long as you have 4GB installed. (Being able to hold most of the source file in RAM helps of course, but it’s not a deal breaker.) Apparently Service Pack 1 for Encoder will let you take use of your GPU cores as well, provided you have a CUDA-compatible NVidia card. No news on the release date of that, but I’m looking forward to it. (I’m hearing January, but no specifics on which day of the month.)

Ok, let’s get to it then. I need something to encode, and put on my devices. I downloaded Big Buck Bunny, a HD animation that you are free to mess around with as you like. I downloaded the MSMP4 format file with a 1920×1080 resolution. The source file is about 700 MB, and times in around 10 minutes.
Everything can be found at http://www.bigbuckbunny.org.

big_buck_bunny_1080p_stereo_720p_Thumb

I had a look through the presets in Encoder since it is simpler using these if I can, rather than creating my own from scratch. I found four interesting presets that were sensible to try out:
– H.264 HD 1080p VBR: this keeps the resolution, and outputs decent stereo audio, but lowers the bitrate slightly so the file is somewhat smaller. Uses H.264 High codec.
Resultant file size: 230 MB.
– H.264 HD 720p VBR: same as above, but 1280×720 resolution.
Resultant file size: 192 MB.
– H.264 Zune HD (AV Dock Playback): 1280×720 resolution, high bitrate. Uses H.264 Baseline codec.
Resultant file size: 418 MB.
– H.264 iPhone / iPod Touch: 480×360, decent quality, but optimized for “low” spec devices. Uses H.264 Baseline codec. (For 16:9 ratio resolution is corrected to 480×272.)
Resultant file size: 112 MB.

You’ll notice that I have two 720p presets, and one produces a file twice the size. What gives? Well, they’re using different profiles of H.264, and you could say it would be an easy choice were it not for the fact that not all devices necessarily support the “High” profile so I included both in my test. (More on which to choose later.)

All the details of the codec/profiles here:
http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC

This basically gave me four files to test out on my devices. I did not do extensive testing on my Windows 7 or OSX installation – that just worked much as I expected it to do. (On Windows 7 I used Zune for playback since I need that anyways for syncing my Windows Phone 7 device.) I’ve broken the testing down in paragraphs for each OS.

Windows Phone 7
I used an HTC Trophy for all my testing on this OS. The emulator lacks support for a lot of the codecs, so you can’t really do any testing with it. And even if everything was supported the performance results would not equal that of an actual device.
The official details of what is supported is here:
http://msdn.microsoft.com/en-us/library/ff462087(v=vs.92).aspx
You must transfer all files to and from the device via Zune if you transfer from a computer. Videos must be supported by Zune and added to the collection. I discovered that the 720p files were automatically transcoded by Zune to a trimmed down 800×450 resolution (would be the same for 1080p sources likely). So you will not be able to sync a higher resolution than what the device supports physically. The lower resolution file transferred without incurring a transcode. Both files looked good though, and I didn’t notice a big difference in the visual experience. Yes, you can notice a difference in quality, and it would probably be more noticeable in something non-animation, but it’s not bad in the lower resolution.

iPhone
I used an iPhone 3GS upgraded to iOS 4.2.1 for this test. The specs from Apple gives a couple of clues what is supported on the iPhone 4 so we’ll just go by that:
http://www.apple.com/iphone/specs.html
While 720p is listed as supported this probably does not apply to the 3GS as iTunes simply refused to transfer any such files. (And for the test scenario iTunes is the only officially approved method for transferring content to the device.) It was able to transfer the lower resolution file, and it looked decent enough here as well. A little bit less enjoyable than the HTC device, but I think that’s due to the better screen of the Windows Phone device. The playback itself was smooth. I’m guessing the the iPhone 4 with its retina display would be on par with the Trophy.

Android
I used a Samsung Galaxy Tab running Android 2.2 for these tests, and with it’s 7” screen it’s considerably larger physically than the two previous devices. I used only the video player supplied, (I do not know if Samsung has tweaked this in any way from the reference player provided by Google), but with Android there are of course plenty of different third-party players available. The official list from the SDK says that H.264 is supported but is pretty sparse on the details (since the provided link is generic the contents may change when a new version of Android is released, so it may be different by the time you read this):
http://developer.android.com/guide/appendix/media-formats.html
I didn’t have to deal with any sync software here. Sure, Samsung provides something called Kies, but I ignored this and just copied it straight to the internal flash via USB. It also supports Micro-SD cards so you can transfer it this way as well. This baby played back everything totally smooth. 1080p – no problem. (Yeah, I know it doesn’t actually display a full 1080p, but it still has to process a certain amount of input pixels.) It all looked great too. So, I guess you don’t really have to optimize all that much for this device. There is one minor issue with this device; it runs a Samsung file system (not the default Android file system), and it will not support single files over 4 GB. If you have an entire movie in HD you might have to transcode to reduce the file size. (Or root your device and reformat the file system to allow for larger files.)

I would have expected to see some more issues, but apart from the resolution restrictions all the devices behaved in what could be called a user-friendly way. To get a better feel of the quality, (let’s call it reference viewing if you will), I ran through the material a couple of times on my desktop computer with a resolution above Full HD, and I sort of decided to use 1080p VBR/720p VBR as the default profiles depending on what the source material requires. (DVD-ROM isn’t in full HD so why encode it as such…) The iPhone / iPod profile looks like it might be unavoidable as the “low-end” alternative since the Trophy and 3GS refuse to go higher. (Zune takes a lot of time to sync when it has to transcode your files so you don’t really want to incur that penalty. It will also take up space on your hard drive that you have to clear out after the sync.) I might decide to produce two output files going forward, but this will be re-evaluated along the way as we progress to other parts. So, while we’ve covered the basics and done some very important choices I’d hold on a little bit longer before you set your computer to work 24/7 encoding video Smile

Leave a Reply

Your email address will not be published. Required fields are marked *

*