TehNrd

There was a time long ago when home made videos where smooth and stable. The main reason for this was the size of the camera. Look at this bad boy…

While the quality of the video was horrible it was usually pretty stable and not bouncing all over the place. For starters it would literally rest on your shoulder which is a pretty solid platform. It is also physically large and therefore requires a lot more movement to cause a shaky recording. Over the years this has become more and more of a problem as cameras have become smaller and smaller.

In small dedicated camcorders manufactures have been able to add physical image stabilization and by physical I mean there are moving parts in the camera dedicated to stabilizing the image. In phones there is simply not enough room for this type of stabilization and the smaller the devices become the worse the the shakiness becomes. This is because every little movement has an even greater effect on the position of the lens which then induces camera shake. Take a look and the very scientific diagram below. It compares the amount of lens movement when you move each camera about “1 inch”. On the iPhone this causes the lens to move quite a lot where as this amount of movement on a larger camera hardly cause the lens to move at all.

Take a look at this video supposedly shot with the new iPhone4. Image quality is great but in this simple panning movement there is some notable camera shake, especially if you view the HD version. It is not horrible but it would become quite annoying after anything longer than a minute.

So how do we solve this problem? One way is with post processing such as the image stabilization feature in iMovie. Let’s take a look at how this type of stabilization process works. We will use this animated image as an example of a shaky video. (Click image to view animation)

What the software will do is inspect each frame of the movie and find the areas of the image that are a very similar match. The software will then move the frames around so the similar areas line up on top of each other. Then on playback instead of everything jumping around the images are aligned which causes the video to be stabilized. See the image below. Each frame has a different border color and you can see how the frames are shifted to stabilize the image.

This method has one major drawback. It decreases video quality. Once all the images are line up the stabilized area needs to be cropped and then stretched back out (or zoomed) to the original dimensions of the video.

So how can this be fixed? Instead of recording at 720p the device would need to record at something around 780p and then apply the stabilization process in real time to the video. This way once everything is cropped the video is stabilized and saved at a native resolution of 720px.

There are no doubt some issues with the proposed idea with the main one being processing power. There is no way (maybe someday) this could be done in real time with software and tradition CPU cycles on a mobile device. Where this may have a chance of working is with a dedicated chip with this functionality built in to the hardware. Since Apple is now cooking their own silicon maybe this is a possibility. Even with dedicated hardware this still may be impossible at 720p but maybe it could work at lower resolutions.

Perhaps the new gyroscope in the iPhone 4 could assist with stabilization. Let me first say that, no, I don’t mean a spinning gyroscope in the camera will provide stabilization. The whole purpose of this gyroscope is for more accurate movement detection. I have no idea how accurate this thing is in detecting movement but perhaps the gyroscope could provide movement data back to the stabilization processor so it wasn’t as CPU intensive. For example, between frame 18 and frame 19 the camera moved “x” amount in direction x-y-z therefore it knows how to adjust frame 19 to 18 without massive amounts of processing.

This may be a crazy idea but something needs to be done about shaky videos especially as devices continue to shrink. I’m hoping Apple has the resources to come up with some ingenious solution.