Latest or all posts or last 15, 30, 90 or 180 days.
Welcome to diglloyd.com
In-depth review coverage is by subscription.
Also by Lloyd: MacPerformanceGuide.com and WindInMyFace.com
First-time visitor
Up to 1527MB/s sustained performance

Understanding the Sony A7R II Uncompressed ARW Raw Format

Sony uncompressed raw format support is available as a firmware update for the Sony A7R II.

See also:

14 bits stored in 16

The Sony A7R II captures 14 bits per photosite. The Sony uncompressed raw format then stores these 14 bits in 16 bits, bloating the file by 14%. In other words, Sony wastes about 9MB per image for no value or purpose at all. Even bit-packing the files (squeezing out the two unused bits) would have meant a file size reduction of about 9MB per file

It’s lame that Sony could not have cut file size down to ~75MB instead of ~86MB just by bit packing; this requires trivial CPU power to do so and would also speed up write and read times by about 14%, since the images would be ~14% smaller. So Sony users lose triply: wasted storage, degraded camera responsiveness, fewer shots before the camera buffer fills.

I really like the speed of Lexar 2000X SDXC cards. But this new super-sized Sony format means that Lexar 128GB SDXC or Lexar 256GB SDXC cards may become necessary for capacity reasons. And the storage bloat on the computer when working with Sony files....ugghhh.

Lossless, but no compression

The Sony uncompressed raw format uses no compression and is thus lossless.

Nikon and Canon use a lossless-compressed format based on Huffman encoding (also lossless). A lossless-compressed format can save far more than even the bit packing mentioned above; those formats implicitly bit-pack, but do even better for images with areas of uniform tone and little detail.

Nikon lossless-compressed and uncompressed sizes (same image)

Compression efficiency

How much an image file compresses is a function of its entropy (degree of randomness). Images with very high detail are difficult to compress and little savings can be had. Images with uniform areas such as blue sky are much more compressible (losslessly) with large space savings.

Exposure greatly affects how compressible an image is. Think of an excellent ETTR exposure: it utilizes the full 14-bit dynamic range, at least in the bright areas. An underexposed image and/or the darker areas of any image have high-order bits of each photosite in those darker areas as zeroes, which such areas highly compressible. Similar to the discussion above re 14/16 bits.

Many if not most eal-world images have dark or very dark areas (possibly over large areas of the frame) in order to not blow-out the bright areas. Real-world images also have blue sky, clouds, etc—areas that are relatively uniform, that is, highly compressible. So in practice, many images can be compressed with savings of 20% to 50%, particularly if the compression approach is tuned to images by such things as preprocessing delta encoding.

Example of compressibility

To test compressibility of Sony A7R II images, I used “gzip -9 -r”. Compressibility for this bracketed series ranged from 19% space savings to 38% space savings. Those figures could be less or more with other images, but it makes the point: Sony’s uncompressed format is hugely wasteful of storage space. For no benefit vs a lossless-compressed format as with Nikon/Canon.

As shown below, an exposure bracketing sequence was made from slightly overexposed to a perfect ETTR exposure to about two stops underexposed.

Space savings compressing Sony A7R II uncompressed raw files vs exposure bracketing
(greater underexposure means greater compression savings)

Per K writes:

Well - maybe we should give Sony some time for the next step to bring lossless compressed RAW? And how big is the size problem really? My three year old PC is Core i7 and 16GB RAM and yes, it takes some extra time processing and working in LR. But not that much. Data storage is cheap these days. TIFF files for printing are still the same size. Conversion to TIFF is up from 4 to 6 sec.

I use sidecar files. Used DNG a couple of years ago but I could not see a reason to continue with it. Will it be possible to view 2003 RAW's in 50 years? As DNG is backwards compatible - yes. If DNG or something similar exists then.....

DIGLLOYD: Fair enough to “give Sony some time”. If indeed this is an interim bittersweet solution to the lossy 11+7 bit format, let Sony say so. But it might be that the CPU in the Sony cameras is not fast enough for Huffman encoding—I don’t know. But at the least, bit packing is hyper fast, so why release a format with 14% waste in it?

Data storage is relatively cheap. But this uncompressed format halves the time before upgrades are needed for storage. So if a system could work for 3 years before more capacity is needed, that becomes less than 18 months.

It's hardly a matter of just buying another drive: performance, backups, etc all need to match, and that cycle is now accelerated by more than 2X. . A system that was good in capacity for 3 years may require a change in 14 months or so. That’s a huge jump, and storage has not grown anywhere near that fast. 8TB drives are insanely expensive, and I myself am already pushing the limits of 5TB drives. RAID-5 or similar is an option with solutions like the OWC Thunderbay 4, but that does not help for multiple redundant backups (single drives, RAID is not a good idea for backup).

Moreover it's an in-camera problem, performance issues as noted, backup capacity in the field, etc. Bottom line is that it's a needless waste with no benefit over lossless-compressed. I came back with 60GB or so of Sony files from one trip. I don't want 125GB instead! In fact my 512GB MacBook Pro has been stretched to the limit on space (half of it is occupied even before I shoot anything)—the new huge Sony files may be the breaking point and I don’t want to buy a new $3K laptop just for a 1TB SSD. So then I have to carry external storage... get the idea? Pain in the ass and/or costly. And if I did buy a new laptop, I’d want a 2TB internal SSD.

The idea that DNG will work and CRW/ARW/NEF won't work is not credible. I’m not drinking that Adobe marketing KoolAid. :)

Durable and fast, up to 1800MB/s

diglloyd Inc. | FTC Disclosure | PRIVACY POLICY | Trademarks | Terms of Use
Contact | About Lloyd Chambers | Consulting | Photo Tours
RSS Feeds | Twitter
Copyright © 2008-2017 diglloyd Inc, all rights reserved.