Using AI to autofocus
Autofocus algorithms powered by artificial intelligence have the potential to completely outperform traditional autofocusing techniques in both speed and reliability.
Autofocus is important
There is nothing worse than reviewing footage or stills after a day’s shooting and discovering that a key moment has been ruined because your camera was out of focus. This can be doubly frustrating when the problem was caused by your camera’s autofocus algorithm getting inexplicably confused, and was therefore out of your control.
Autofocus algorithms are like bassists; when they’re good you barely notice them, and when they’re bad they ruin everything.
If you have to carefully monitor your camera’s autofocusing while shooting because you’re worried it might lose track it can be distracting and frustrating. A good autofocusing algorithm allows you to forget about what’s happening inside your camera and concentrate on creating content.
Why do modern cameras often struggle with autofocus?
With a little practice and experience it is not too difficult for even a novice photographer to learn to manually focus a camera correctly and quickly. So why do the autofocusing algorithms in modern cameras often seem to struggle in situations which could be handled relatively easily by a moderately competent photographer?
When you break it down, autofocusing is actually a complicated and complex series of tasks which can be very difficult to achieve with traditional processing techniques.
The first task of autofocusing is to identify the subject of the scene and the part of the image which should be most sharply in focus. This is something which is so straightforward for a human to do that it is easy to overlook — as a content creator you know instinctively and from experience exactly which part of the frame should be in focus.
But the sort of computer you would find in a typical camera does not have any concept of instinct or experience, it relies on a set of simple rules prescribed by the camera manufacturer and hidden from view. And those rules can sometimes be wrong, in ways that are surprising and unpredictable.
AI is better
Techniques based on Artificial Intelligence, or more specifically machine learning and deep neural networks, offer a different approach.
Rather than following strict rules written by engineers, these algorithms learn their own rules from large numbers of training images, and draw on that training as a proxy for experience, enabling them to perform tasks which would otherwise require human supervision.
In much the same way as a skilled darkroom technician with a loupe, a neural net can use the context of a scene to predict precisely which object within an image should be in focus, and then examine that object in great detail to determine whether it is perfectly sharp or not. It can then even predict exactly how the camera’s lens should be adjusted to bring the object perfectly into focus, all in the blink of an eye.
Autofocus algorithms powered by artificial intelligence have the potential to completely outperform traditional autofocusing techniques in both speed and reliability. That niggling worry at the back of your mind about whether your shot is in focus or not will soon become a thing of the past.
Introducing Alice
Our team is determined to use the revolutionary new capabilities of AI and computational photography to empower photographers and filmmakers by improving their tools and expanding their creative possibilities.
That’s why we are building Alice, a brand new camera optimised from the ground-up for using artificial intelligence to directly improve the workflow and creative experience of content creators.
Alice is powered by cutting-edge neural network accelerator hardware allowing computationally intensive AI algorithms such as those required for autofocusing to be run on-camera in real-time at high frame rates.
The camera and our AI features are still in development but if you would like to get closer to the action we launched a public beta program to get future customers more involved in the product development.
You’ll be able to test pre-production hardware and software features before anyone else, and we’ll be delivering a unique lecture series in computational photography. This is where the team will dive deeper into some of the theory behind using other AI-based and computational methods for colour science, auto exposure and stabilisation.
The deadline for the beta does close on Sunday 15th November and there are only a few spaces left so I’d encourage you to apply soon!
I'm curious about this. Panasonic has spent the last ten years or so working on AI autofocus, in their Depth from Defocus technology. It's interesting, but most of the time, it fails in speed against phase-detect AF simply because PDAF needs a single capture of the PDAF sensor points, while DPAF needs at least two image captures. Still better than CDAF needing many.
The things Sony's managed with eye AF, the things Olympus has managed with AI-object recognition, are another layer on this. I guess the OM-D E-M1X users are getting birds added on top of trains, planes, and automobiles this month, but it seems those deep leaning models, even when effective, have been entirely at the whim of the hardware provider.
So I'm curious where you're applying AI in AF. And it's ultimately speed that drives it -- doesn't matter what you can compute, it matters what you can computer in 1/100 or better second to get the shot. Incidentally, I am a backer... as a photographer and computer engineer, IO couldn't exactly let this slide by.