I wrote a few days ago on my personal blog about my mobile video editingI project aims. Soon after I posted that, I had a long talk with @jkkmobile who, like me, is always looking for ways to improve speed and quality while keeping the weight down. We both deeply understand the tech involved, the requirements and challenges and have come up with a set of initial thoughts that we hopebare worth sharing.
To recap, the three areas of interest are cloud-based processing, arm-based smartphone and tablet processing and traditional x86 laptops. The target for this project is a sub €600 solution that is able to post 16:9 HQ quality (480p) edited content with watermarks, titles, crossfade and other cpu-bound processes. The computer solution should weigh less than 1.5 kg. Trust me, this is quite a challenge as you’ll see below.
We quickly discussed the idea of cloud-based editing but while that might be possible over good cable networks, over 3g networks it is too unreliable and too slow. We’re both interested in this as a future possibility and Clesh is a service we’re watching closely.
As for ARM based editing on smartphones and tablets, again, there are issues. While the technology is maturing quickly and there are some interesting software solutions out there (Reel Director on IOS, Movie Studio on Honeycomb Android) these solutions need tight integration of hardware and software. We’re thinking of future cameras that include camera hardware you just don’t get in ‘general purpose’ smartphones and Tablets. For a smooth and fast editing experience we also need to wait for at least the next generation of ARM platforms. There’s definitely an opportunity for someone to make a niche ARM/ANDROID camera for mobile reporters although we’re not sure that the carriers would be too pleased about the upload usage! Software needs to mature too. Of course, it doesn’t mean you can’t post the occasional 30 second clip from a phone without editing. I plan to do some of that using the Galaxy Tab which, although not a 16:9 solution, records in 720×480 and has some very simple and easy sharing tools.
Todays video editing solutions are very much about traditional computing. X86 processors, desktop operating system, rich software, common file formats and separate devices for the camera and editing parts of the process. Many will actually tell you that you shouldn’t even think about a low cost laptop. As for netbook, people that do video editing for a living often laugh.
Having used a netbook for editing and posting videos at expos’s I know its possible. Don’t let anyone tell you you can’t do it because JKK and I have over 20 million combined YouTube views and most of these were done on-the-go with a netbook but as I mentioned in the last post, the requirements have changed over the last 2 years and 4:3 VGA videos aren’t acceptable to many. Its a trend, it works against the mobile user but I (JKK already produces videos in 16:9) have to play along now if I’m to be taken seriously.
JKK and I agree that there are a number of approaches that can be made in the x86 world.
Firstly we’ve discounted the idea of using Apple Mac products with iMovie for mobile video editing due to the import process which converts video into the AIC format usable by the video editing software. The process simply takes too long. There are other software solutions though which could be interesting on the MacBook Air product, as long as there is no import processing. This needs further research but even if the import problem could be solved, the price of the Apple MBA products is outside our range. I’m focusing this project on low-cost and lightweight solutions.
In our discussion we repeatedly came back to Nvidias CUDA technology which allows a certain amount of general purpose computing to be done on the graphics module. It is truly a game-changing technology but it does require software to be re-written to take advantage of it.
You see, graphics modules (gpu) are very specific processing engines for 2d, 3D and video decoding. In some cases the GPU can also handle encoding but these basic processes are often not what you need for video editing. Consider a fade, an overlay, a watermark or a transition. These processes require general purpose processing on a frame by frame basis. This is why CUDA is so interesting for mobile and low power video editing; it breaks the requirement for pure CPU processing.
CUDA doesn’t just appear in high end graphics solutions because it also appears in the Nvidia ion2 platform that is offered with the netbook-class Pinetrail CPU. Beware though, this ion2 variant doesn’t include the CUDA you need for video processing. The lowest power processing platform that we have found that includes full CUDA capability is the Ion2 12″ Netbook platform. it couples the D525 dual-core, 1.8Ghz Atom with the full 16-core CUDA engine. They are not the best mobile cpus (speedstep is missing) but they are in a processing class that easily outpaced traditional netbook platforms. The ion2 solution also allows the platform to fall back to the embedded graphics thus saving power when the GPU is not needed. The platform also idles down to a very low power drain state. For our purposes, its a very interesting platform.
Examples of devices that use this platform are the Acer EeePC 1215n and the Acer Lamborghini VX6 which even offers a useful 3GB of memory. Both are around 1.5kg in weight.
But what about dropping CUDA and going for a general purpose CPU with a bit more power than Atom? It’s possible. The Lenovo U160 offers CPU options up to core i5. Could a boost in cpu equal the CUDA solution? It would certainly be more useful for general purpose activities and opens up the choice of software to that which isn’t optimized for CUDA. Using a higher-end architecture with faster bus speeds and one well matched to a GPU for more efficiency might bring benefits.
Two choices in the low-cost area that I’m looking at are the Lenovo U160 and the Acer Aspire 1830T. Both are available with a low power Core i5 and weigh about 1.4kg. The Acer has the better performance and battery life of the two according to reports I’ve read. Cost is relatively high though and it is going to be tough to find a solution under 600 Euro.
The other interesting thoughts we discussed was that of the keyboard requirement. Could we used Tablet PCs and save weight and space?
Editing movies is largely a mouse operation which means it could be suited to tablet operation. In general though, battery sizes are smaller (and spares more expensive.) I haven’t seen any Tablets with the CUDA 12″ netbook platform and there are only a few low cost options with laptop cpus. The Hanvon B10 is one of them. We see the advantage of the ‘modular’ tablet solution but we’re both wanting to keep or lapping scenario, the keyboard and the protection it brings when folded together.
JKK and I discussed a bunch of other important items too. Fast SSD helps a lot. Using fast SD cards means you can edit from the SD card without having to copy it to the hard drive first.
We also discussed file formats, bitrate and sizes. We’re currently in agreement that h.264 is the format that provides most flexibility but there’s a huge CPU load associated with h.264 that is used to compress files down much further than standard mpeg4 part 2 codecs. The important thing to note is that our initial and most important file transfer is from an SD card in a PC. The bitrate and file size can be large without affecting the transfer time significantly. Final compression into h.264 at 2 or even 3 mbps doesn’t save that much in final file size and its not important to us how YouTube sends the file out so why even bother with h.264?.
It seems to me that a recording format of Mpeg-2 at a bitrate of 10mbps would be acceptable for our sub 10 minute clips. They would be relatively easy to work with. One thing to note on this is that CUDA might not work with mpeg2 which brings us back to using a general purpose CPU. Testing is needed here. If we can find video editing software that uses CUDA for mpeg-2 editing (note that we also need to choose our output file format carefully too) then mpeg2 could be exactly the right choice of source codec.
There is other slight problem here in that there are very very few cameras that record in mpeg2 now.
Resolution, bitrate and aspect ratio.
16:9 is the ratio we need to aim for with YouTube. The lowest acceptable resolution is 854×480 with a bitrate of around 2mbps. This triggers HQ encoding in YouTube although I’ve had no problems with my 640×480 resolutions showing as HQ. Another option would be 720×480 which isn’t quite 16:9 but doesn’t look as bad 640×480.
Note: Recording in the resolution you want to output in will save processing.
Recording in 720p (1280×720) is another option but could require re-encoding to 480p before using in an editor because it’s a huge jump in pixels per frame. Ideally the camera will allow 480p and 780p at various bitrates. If you consider the requirement for viewfinder and Mic input you will only really find solutions in the video camera world. Combining a digital camera with these video requirements results in very little choice.
As for bitrates for the final upload file, it will depend on final codec and resolution. To ensure a reasonable chance of using 3g services to upload the file, a bitrate of around 2mbps needs to be used. For a 480p resolution it means you really need an advanced codec like h.264 to preserve the quality.
So in summary, mpeg2 at 854×480 at a relatively high bitrate seems like a good source and editing choice. Output files should be the same resolution but at around 2mbps bitrate in the h.264 codec.
What does jkkmobile use? He currently records in mpeg2 at a 16:9 ratio. Resolution is 960×540, bitrate either 6 or 9mbps. He converts that down to 854×480 which is 480p resolution. I’m not sure what format and bitrate he outputs to send to YouTube. If he has enough cpu power he will be using h.264 but he may be using wmv or something else that it is a little easier on the CPU. He certainly has the optimal settings for source files. His camera is a Canon FS100 which you can’t buy any more. There are other SD cameras from Canon that also offer 16:9 capture though.
A quick step back to the world of camera products shows that 1080p rules and it’s difficult to know exactly what alternative resolutions a device will offer. In addition, h.264 is the popular codec which at anything bigger than 480p resolution, will be a problem.
Can CUDA do all that we require or are we going to have to rely 100% on a general purpose CPU? From what I have read and been told, most video editing software that is CUDA-enabled is doing so on output to h.264 only. I’ve seen test result with mpeg2 source formats too so mpeg2 again looks like the best source format.
There is only one way to find out. I’m arranging an Asus 1215n loan device for CeBIT next week and I plan to do as much testing as I can. I will use JKKs cam to create some 16:9 mpeg2 source and test it with some CUDA-enabled software. Power Director from Cyberlink looks like a good starting point.
Many thanks to Think4IT Solutions for offering to help us with this project.
Stay tuned for part 2 which should come after CEBIT.
Posted from WordPress for Android with the Galaxy Tab
Tegra 3 is the world’s first quad-core mobile platform, Nvidia assures us, and along with those 4 CPU cores come the Tegra 3 GPU which has 12 additional cores of its own. Nvidia is happy to show off the graphical capabilities of the Tegra 3 GPU, which they claim to be up to 3x faster than the Tegra 2 GPU. A few months back (when Tegra 3 was still being called Kal-El) Nvidia demoed a game called Glowball on pre-production hardware. Now, Nvidia is showing off a new level of the game, running on a production version of the Transformer Prime:
