I’ve been writing about focus accuracy for some time now. No lens test is any good unless it’s checked and cross-checked, and it’s one reason that more and more I use field shots in judging lens performance. Field shots or “3D” shots are also hard to do when comparing cameras/lenses, but they are the reality of making actual images, which is what the end game is. Choosing one lens over another demands the ability to consistently achieve good results; it doesn’t matter if the lens is world-class if (for example) it can’t be focused consistently.
The guys over at slrgear have lens testing down to a science, inventing their “yellow submarine” to do focus bracketing. That’s something I’ve been doing for years now, but not having the lab space I do it manually, cross-checking results with different cameras, field shots, etc. There are surprises and lenses perform differently at different distances, so field images really matter.
Another problem is that the sharpest results in the center aren’t necessarily optimal for the frame as whole; this can be seen shooting a series of focus-bracketed shots, where “giving up” a tiny amount in the center can yield a superior overall result. Few of my images rely on center sharpness; most are off-center.
Field curvature assumptions are also a problem. The Zeiss ZF 28/2 Distagon has enough field curvature to make a very noticeable difference at the 5.5 feet cited in the slrgear article. See my Dec 29, 2007 blog entry (rice pounder) for an example where the corners are sharp, in spite of being several feet behind the center.
The bottom line: field shots of a wide variety of subjects are still the best way to evaluate lens performance for real images, especially if your shooting involves certain distances or types of subject matter. That is what I do in my Zeiss ZF Lenses review. The myriad effects a lens produces (the “look”) are often what matter the most in terms of visual impact.