Regarding the technology (achieving shallow depth of field through an algorithm), not Google's specific implementation ...
Up until now, a decently shallow depth of field was pretty much only achievable in DSLR cameras (and compacts with sufficiently large sensor sizes, which typically cost as much as a DSLR). You can simulate it in Photoshop, but generally it takes a lot of work and the results aren't great. The "shallow depth of field" effect was one of the primary reasons why I bought a DSLR. (Yeah, yeah, yeah, quality of the lens and sensor are important too.) Being able to achieve a passable blur effect, even if it's imperfect, on a cellphone camera is really pretty awesome, considering the convenience factor. And if you wanted to be able to change the focus after you take the picture, you had to get a Lytro light field camera -- again, as expensive as a DSLR, but with a more limited feature set.
Regarding Google's specific implementation ...
I've got a Samsung Galaxy S4 Zoom, which hasn't yet gotten the Android 4.4 update, so I can't use the app itself to evaluate the Lens Blur feature, but based on the examples in the blog post, it's pretty good. It's clearly not indistinguishable from optical shallow depth of field, but it's not so bad that it's glaring. That you can adjust the focus after you shoot is icing on the cake, but tremendously delicious icing. The S4 Zoom is a really terrific point-and-shoot that happens to have a phone, so I'm excited to try it out. Even if I can use it in just 50% of the cases where I now lean on my DLSR, it'll save me from having to lug a bulky camera around AND be easier to share over wifi/data.
It does similar things, and in fact I could believe it if the Google app was just a dumbed down version of the functionality intended to be usable by a wider audience.
They did hire Marc Levoy. I have been hoping for this for awhile. Can't wait for the next iteration. There are certain things you can't replicate with just a single tiny camera, but they are doing a great job!
https://news.ycombinator.com/item?id=6483182
> There are certain things you can't replicate with just a single tiny camera
MOST DoF and bokeh effects in photography can't be replicated with a single small camera.
As a photographer, bokeh is surprisingly difficult to fake, and looks glaringly bad when you notice. The blur effect is due to focal distance ratio differences, and it's very difficult to determine the distance in software. Hell, it's hard to determine it with 2 lenses, as the HTC implementation does.
Look at this picture, for example: http://static.trustedreviews.com/94/00002b836/8517/blue-htc-... For most software, it's extremely difficult, even with distance data, to separate the bush in the back from the blue toy. As a result... messy looking blur.
Unfortunately, as a non-professional photographer, I think this article very disingenuous. Most of the shots aren't taken from the same distance, angle, nor lighting. "Here we are, closer to the subject and from a different angle; notice that we don't have to deal with distinguishing our focused subject from subjects that no longer exist!" "Sure these look the same, but this one is done with optics and is definitely better!"
While it is unfortunate that they did not do the exact angle in all shots, it is still possible to see the difference in the simulated and real effects.
I can say that, as a non-professional photographer, picking up a prime lens and using that for the family shots has been an extremely eye opening experience. To the point that I actually dislike most photos from point and shoots.
There is definitely a bit of "quit caring about aperture." And I can't argue against progress in making the phone cameras better. I'm just not seeing compelling evidence to ditch my DSLR.
They're clearly taken with the same lightning (outdoors and around the same time), but with different iso/aperture/exposure -- as they would have to be. The dedicated camera will let in much more light than the phone camera.
I do agree the zoom/distancing on the "foliage" photos are unfortunate for comparison -- but the fluffy animals more than make up for it IMNHO.
Very true. We do this kind of work in VFX all the time to reduce CG rendering costs for blurs that are expensive in 3d but cheap in 2d. Even with access to sub pixel (4x, 8x etc.) depth maps there are always lots of issues to deal with around edges.
But given that jpeg is good enough for most people I'm sure these types of tools are too.
According to the article, and my own experience with google's version, it's good enough for simple portraits. Since most photos, and close to 100% of photos anybody actually cares about, are portraits that makes it a pretty useful feature.
There have been multiple implementations of this class of algorithm (collectively "synthetic aperture") using different techniques.
Edit: below mentions the HTC "double camera" phones, and in fact it's also possible to create synthetic aperture photography with multiple cameras instead of moving a single camera to multiple positions. Then you have the added advantage of not needing to assume a static scene! But a single-camera algorithm is great for rolling out to the common devices most of us have in our pockets.
Marc Levoy's SynthCam used a circular wiping motion, and attempted to "paint" the 3d space occupied by your theoretical processed aperture.
This Google Camera only requires a single linear move, and processes the rest!
According to the text the blur itself is simulating an actual lens. Without seeing the exact algorithm it's hard to know how accurate that claim is. There are many ways to do a blur and I know what I'd use if I were trying to generate good bokeh.
Point taken. Arguably it's a case of David Pye's notion of the workmanship of certainty (in this case heavy-lifting computer vision) replacing the workmanship of risk (making adept movements of camera)
But if it works better under all conditions, fine.
http://en.m.wikipedia.org/wiki/David_Pye_(furniture)
The depth part is actually much easer than you might think and we've had techniques for doing some aspects of it since the 80's if not earlier.
One early technique was to take video form a mounted camera moving horizontally on weals looking 90 degrees to the side from the direction of travel (think looking out the side car window).
Now if you take that sequence of video frames and stack them one after the other like a deck of cards to create a 3d volume. Then you look down on that volume, what you will see are lines of color moving diagonally. Top left to bottom right, or the other way depending on your direction of travel.
These are the image features as they trace there way across the video over time. Things that are close move quickly so have a shallow diagonal. Things that are further away move slowly and have a much steeper diagonal.
Thanks for the explanation. I've been working a little bit with a guy doing some computer vision stuff for an industrial system (shingle production line), and I've noticed that a lot of (what seem like) complex problems can be effectively solved with very simple solutions.
Is there any simple literature that covers this domain? Like a book of algorithms for computer vision, or something?
Sadly I've found that most sources prefer to be academically rigorous over quickly comprehensible. Of course it's good that people have done the academic work, but it can be tragically comical how obtuse an academic paper can make a simple concept.
However, one of the most recommended books on the subject is available online, so you might want to check that out.
Computer Vision: Algorithms and Applications by Richard Szeliski
re: your first comment... I'm reminded of a lot of the wikipedia pages on mathematical concepts. Sometimes I have to laugh, because they seem so high-level that only someone who already understood the domain could understand them.
One limitation of this that nobody has mentioned yet is that if you have to pan your cameraphone as you are taking the photos to generate the depth map you will have a harder time composing your photo than you would with a traditional photo. Usually I like to spend a few seconds getting into the best position and framing my photo carefully before taking it. Getting the photo I wanted would therefore be much harder if I had to pan the camera around as I was taking it. I don't have Android so can't test it out... anyone using the app got any views on this?
You first take the picture normally. After taking the picture it asks you to slowly move the camera up to generate the depth of field, so there's no moving the camera as you're taking it
It would seem that if you have to do a depth scan anyway, it should be possible to just do a (series of) scan(s), and then post-compose and post-focus your image(s)?
Half a sec on my N5 (go slower and it actually complains). It's even shorter with good lightning and with a bit of habit it can come out quite naturally. Frame, snap, slide.
I would say that it takes about 1 second on my nexus 4. If you do it too fast the app will complain, but I find that it's actually very fast and easy to do right.
It still seems like a problem to me, if your subject (and potentially background) are moving around a lot. I shoot a lot of macro shots of bugs that are always about to jump off plants that are swaying in the wind... Not to say this tech isn't impressive and useful, but it doesn't solve DoF problems for tiny cameras in every situation.
Not endorsing this thread or anything, but getting that film to Instagram is trivial: it's called Costco. They'll develop that film, print it, scan it for you and give you digital files. Easy, cheap and high quality.
The point is, it's not difficult at all, and the optics setup to do so is actually much simpler than the computational algorithms needed to do it digitally. To think that an expensive DSLR or complex PGM algorithm is required to do so is silly. People have been shooting narrow DOF pictures for most of the past century.
Now to get smartphones to do it is perhaps harder than running out to the store and getting a camera, but I would characterize this as one of those evolutionary rather than revolutionary improvements.
In terms of digitizing, there's this archaic thing called a scanner, but a lightbox/DSLR rig can do it, but with the rise of full frame/narrow/mirrorless bodies from Fuji and Sony, you might not even need to do that anymore...
> Up until now, a decently shallow depth of field was pretty much only achievable in DSLR cameras (and compacts with sufficiently large sensor sizes, which typically cost as much as a DSLR
In 2008, I had no trouble taking shallow-depth-of-field photos with a dirt-cheap Canon A570 pocket camera. For example:
Depth of field increases as the focus point moves deeper into the frame.
Depth of field decreases as your lens length increases.
Depth of field decreases as your sensor size increases.
The A570 will have a relatively large sensor compared to a mobile phone, and your subject was very close to the lens. These things considered, the depth of field isn't impressively shallow. Reducing the sensor size to that of a phone and moving the subject further away will make shallow depth of field impossible. The portrait examples on that Google page are exactly the sort of thing that wouldn't have been possible previously.
It's not so much that a DSLR is the only way to shoot with a shallow depth of field (which can even be achieved on a phone camera if you're taking a picture of a ladybug a few inches from the lens), but that the greater the aperture and/or sensor size, the further away you can get a shallow depth of field. So on an APS-C sensor with an f/1.8 lens, you can shoot a portrait with the same level of bokeh you might only get with an extreme macro on a pocket camera, and on a full-frame sensor with an f/1.2 lens, your subject might be standing many feet away and you could still achieve the same effect.
I'm pretty sure they definitely don't count, as DOF is a ratio of focusing distance IIRC. Also the DOF would be far more shallow with a DSLR and a fast lens in the same conditions (close focus).
Up until now, a decently shallow depth of field was pretty much only achievable in DSLR cameras (and compacts with sufficiently large sensor sizes, which typically cost as much as a DSLR). You can simulate it in Photoshop, but generally it takes a lot of work and the results aren't great. The "shallow depth of field" effect was one of the primary reasons why I bought a DSLR. (Yeah, yeah, yeah, quality of the lens and sensor are important too.) Being able to achieve a passable blur effect, even if it's imperfect, on a cellphone camera is really pretty awesome, considering the convenience factor. And if you wanted to be able to change the focus after you take the picture, you had to get a Lytro light field camera -- again, as expensive as a DSLR, but with a more limited feature set.
Regarding Google's specific implementation ...
I've got a Samsung Galaxy S4 Zoom, which hasn't yet gotten the Android 4.4 update, so I can't use the app itself to evaluate the Lens Blur feature, but based on the examples in the blog post, it's pretty good. It's clearly not indistinguishable from optical shallow depth of field, but it's not so bad that it's glaring. That you can adjust the focus after you shoot is icing on the cake, but tremendously delicious icing. The S4 Zoom is a really terrific point-and-shoot that happens to have a phone, so I'm excited to try it out. Even if I can use it in just 50% of the cases where I now lean on my DLSR, it'll save me from having to lug a bulky camera around AND be easier to share over wifi/data.