|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
September 5, 2008 UPDATE: Digital Photo Professional 3.2 (latest version as of September 2008) has become more efficient. However, the performance tips on this page still apply, and could be especially useful on quad and eight-core machines. On a Mac Pro quad-core 3.0GHz running Mac OS X 10.5, the following results are seen with DPP 3.2. A single batch job exploits 4 cores at least part of the time, and two jobs can exploit the almost entirely. This suggests that an eight core machine could be used to advantage if 3 or 4 jobs were run simultaneously.
Introduction continued Skip to Speeding up batch processing if you just want to know a simple technique for faster batch processing of raw (CR2) files in Digital Photo Professional. Sorry, until Canon and Nikon get their acts together and use multiple CPUs to make single-file processing go faster, our only options are to be patient, or to choose another raw-file converter. Both Nikon’s Capture and Canon’s Digital Photo Professional are among the slowest of the raw processing programs available, leading to a steady stream of complaints from photographers who have tried to use them in time-critical jobs, or even for single files. Both programs make very poor use of available CPU power. This is evident in the CPU-usage graphs shown below, using Mac OS X 10.4.6 on a PowerMac Quad during batch processing of multiple images:
PowerMac Quad 2.5GHz, 8GB RAM, striped 4-drive raid Such poor CPU utilization is particularly relevant with Mac OS, because all but the low-end PowerMacs have been dual processor and/or dual-core machines for some time now. The new Intel-based Macs are all dual-core, excepting the very cheapest MacMini. Behavior is similar on a Windows machine as seen below (though my Windows machine has two single-core CPUs, so overall utilization is higher than on the Quad, with its 2X2-cores).
AMD Opteron dual 2.4GHz, 2GB RAM, striped 2-drive raid, Window XP Service Pack 2 Why does ample CPU capacity go unused while photographers continue to bemoan the sluggish performance? The answer is simple: mediocre products are rife in every field, not the least of which is software engineering. Alternately, it could simply be willful indifference to customer needs. Neither explanation satisfies, and although my last 20 years as a professional software engineer have been blessed with working with many more competent engineers than not, I still have learned a “bit” more than I’d like to know about how “sausage” is made. Hire more workers! The logic used for batch-processing with either Digital Photo Professional or Nikon Capture is simple (!): allFiles = getListOfFilesToProcess(); Digital Photo Professional already is capable of launching multiple workers via user action eg select some files start a batch, select more files, start a batch, etc. The following logic shows how DPP could instantly improve performance for batch-processing on multiple CPU/core machines with a very simple logic change: allFiles = getListOfFilesToProcess(); Nikon Capture might have a bit more work to do, especially if Capture’s processing engine is badly written (global variables and related programming practices that make it hard to use more than one CPU). But at the least, Nikon Capture could allow more than one copy of itself to be launched, so that 2 or 3 copies could be simultaneously converting raw files. The program is written in a way that prevents this (I’ve tested it). There are plenty of other improvements that could be made; when either program is started up, it could create a background “thread” or threads to prepare images ahead of time even while the user was working on another image, caching could be added, etc. That’s in addition to the more complicated technique of using more than one thread (cpu) to process the same image in parallel, so that viewing and working with a single image would be speedier. None of this is rocket science; Apple’s Aperture, a newcomer to the raw-file processing arena, makes superb use of all available CPUs. This section applies if you are using Digital Photo Professional (DPP). It works on both Mac OS X, and Windows XP. It might also help even for computers with only a single CPU/core, depending on disk speed and other factors. But generally speaking, you'll need a dual-CPU or dual-core machine to benefit. Alternately and/or in addition, if your files reside on a shared server, you might be able to apply more than one computer to the task of batch processing. Similar principles apply to any program that offers a “batch” mode or the equivalent. For example, if you can start two or more copies of a program (whether on the same computer or not), and set each to work, you effectively have the ability to process files in parallel, even though any one copy of the program has no such capability. Regrettably, some programs are written in a way the precludes running more than one copy of the program at a time on the same machine—and that includes Nikon Capture. DPP’s batch worker facility can be exploited by starting multiple batches, each of which processes a portion of the CR2 raw files with its own “worker”. For example, on a Powermac Quad, CPU usage goes from about 160% with a single worker to nearly 400% (the max) with 3 or 4 workers. To get the “Quad” value claimed in Apple’s marketing hype, look for CPU usage in Activity Monitor showing the graph on the right. The graph on the left shows severe underutilization of the CPUs; you’ve wasted your money.
On a dual 2.4GHz Opteron, running Windows XP Service Pack 2, CPU usage goes from about 160% with a single worker to 200% (the max) with 2 workers. Look for CPU usage in the Windows Task Manager. The graph on the right, with its flatlined CPU-usage graph, indicates 100% usage for each CPU.
I tested conversion of 150 EOS 1Ds Mark II CR2 files on my PowerMac Quad using Digital Photo Professional 2.1.1.3. I chose JPEG quality 9 for this test. Results below.
To save time in making further comparisons, I cut the test down to 30 files for the WindowsXP tests. A different mix of files can be expected to change the timings a bit, as the 30-file results on the Quad show:
The CPU usage with one worker is remarkably similar to the Quad on the Opteron box (both about 160%):
Now here’s the fun part: I installed WindowsXP-SP2 onto my MacBook Pro 2.16 GHz [experience report] using Boot Camp, and it outperformed the dual Opteron box, despite being handicapped with a disk partion occupying the slowest portion of the already-slow (relatively speaking) internal hard disk. Quite an impressive performance for a laptop! Even more impressive, it outperformed the PowerMac Quad with a single worker, though not with two workers—see table below. With two workers, the Quad still enjoys the benefits of four (4) cores, as the 300% CPU usage shows.
Please note that two CPUs/cores are not sufficient to cut the time in half; there is some amount of “threading” going on, and so at least 3 CPUs/cores are needed to satisfy DPP’s gluttonous appetite for CPU time; there was not enough “oomph” on either the Opteron box or the MacBook Pro to fully exploit two workers. Thus, a dual-CPU/dual-core machine (a quad-core) would be advisable for Windows users too, though perhaps some of the newer 3.6+ GHz Windows machines would be satisfied with just two CPUs/cores. As on MacOS X, there is no way to get Nikon Capture to run more than one worker at a time. I observed file-conversion times of 5.4 seconds per D2X NEF with Nikon Capture 4.4.0, notably faster than the conversion time with DPP, even considering the smaller megapixel count. This bodes well for Nikon Capture users when a Universal Binary emerges for Mac OS X. Incidentally, the PowerMac Quad is the smoothest-running computer I’ve ever used, always responsive even under full load, and that includes quite a few dual-processor Windows machines, which all seem to get herky-jerky when under full load. Determining how many workers to use To utilize all the CPU power your computer has to offer, create batch jobs until all CPUs are 100% utilized:
You’ll quickly get the knack of it. I find I can start three batches in about 15 seconds; that extra effort is repaid with as few as 3 files. It’s also possible, if the files are accessible over a network, to utilize other computers in processing the files. If you have hundreds of images to process, that might be worth your while. Excepting the PowerMac Quad, which needs 3 workers to saturate its CPUs, all other PowerMacs will need just 2 workers, making it easy to figure—just start two workers, each one taking half the files. One caveat: on my PowerMac Quad, each batch worker takes about 230MB of resident memory, and 512MB of virtual memory. You'll need at least 512 free megabytes of memory (not total memory) to make this technique work. In other words, a minimum configuration is likely to be at least 1GB of system RAM, preferably more if you want to run Photoshop, DPP and other things at the same time. Final tip: quit DPP after you’ve started the workers; that will free up memory for everything else. Conclusions While Digital Photo Professional ought to simply automate efficient use of CPU power during batch conversion, the same effect can be gained with minimal effort, drastically cutting down the time needed for big raw-file conversion jobs—and making full use of the power of modern computers sporting multiple CPUs and/or cores. Regrettably, Nikon Capture not only cannot start more than one worker for raw processing, but actively defeats attempts to launch more than one copy of the program. On Windows XP however, its processing times as compared with Digital Photo Professional more than make up for this. What a pity those fast times can’t be further reduced on multiple CPU machines. Contact: email diglloyd a comment on this article.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
