RAW image format - Should you shoot in RAW or JPEG?

In the last few days I came across many websites that teach photographers how to edit their RAW files. Personally, I didn't shoot many RAW images and usually took my images is JPEG format. Even so, I really enjoy editing my images and making some fine adjustments to them using photo editing software. As a hobbyist web designer I am quite familiar with both Adobe Lightroom and Photoshop CS5. After reading some articles about RAW editing, I thought that it's time to expend my knowledge about RAW processing. In this article I have decided to write about the RAW file format, Demosaicing algorithms and what are the advantages of shooting RAW. This subject is kind of new to me, and I will try to bring you relevant and interesting information regarding the RAW file format.
There are some subjects which are really above my understanding. I have linked to some useful resources so you can further read about specific subjects that I cover in this guide.

RAW File Format Explained

RAW image file refers to the file that contains an unprocessed data that was recorded directly from the camera's sensor. Raw files are also referred to as "Digital Negative", because they play the same role as "negative"s in film photography. The RAW files are an uncompressed format, which contains all the available information that the camera captures when you press the shutter button. The RAW file, as in film, needs another processing step before it can be seen as a full-color picture.

When shooting JPEG, it means that the RAW has already been processed in-camera and no other steps are necessary to view the image. JPEG is a compressed format, that means that some data is being lost during the compression process. Furthermore, once you capture a JPEG image, all the camera's settings (ex. sharpness, saturation, etc.) are applied to the image and cannot be undone. Some digital cameras allow photographers to save the image as a TIFF (lossless) image without scarifying image quality. TIFF files size is larger than JPEG files and many photographers prefer using this format for archiving their images. In addition, TIFF files can be saved in 8 or 16 bit data, while JPEG files are 8 bit. TIFF files can also be compressed (LZW compressions) but this compression does not degrade the image quality.

Here is how a RAW file looks like prior for processing it:

Bayerized version of the image (pre-processed)*

De-bayrized version of the image (after demosaicing algorithm)*

* for demonstration purpose only

The process that converts RAW data to a full-color image is known as "Demosaicing". What the bayer algorithm does is reconstructing back the colors for each pixel, according to its neighbor's pixel data. Because each pixel has only one color channel data (Red, Green or Blue), the bayer algorithm does it best to "guess" the original color for that specific pixel.

Digital camera's image sensor can only record shades of gray and not the color. To come up with a color image, the camera's CCD ( Charges-coupled device) use color filters for each photodiode that match the pattern of the Bayer sensor. The luminance and chrominance data are transferred to the image processing engine and being converted to a full-color  image using sophisticated math algorithms.

The Bayer sensor utilized color filters for each pixel in its array. Each pixel gets to have only one color light intensity information. Because the Bayer sensor only records thirds of the color information to a raw file, the algorithm needs to calculate and figure out the missing color information. Bayer algorithm has no real way to know 100% what was the original color in that specific pixel, so it uses interpolation to recover the missing color. Because the interpolation is not fully accurate, it leads to artifacts in the final images. The artifacts are the results of inaccurate "guessing" of the color in a group of pixels.

Here is a helpful video that will help you understand How the Bayer sensor works:

The Bayer algorithm exists in the camera's image processing engine. Mathematicians continue to improve the CFA (Color Filter Array) demosaicing algorithm, and come up with more advanced algorithms to improve the image quality.
The most popular color interpolation methids are bilinear and bicubic. There are popular because they are the fastest methods for reconstructing the color information from a raw file. Even so, bilinear and bicubic interpolation methods provide the lowest image quality.

Improving Bayer Interpolation Algorithm and Image Compression Methods

Alexey Lukin and Denis Kubasov from the State University of Moscow have came up with an improved interpolation algorithm. Their algorithm is based on gradient interpolation and NEDI algorithms. NEDI algorithm is used for better edging interpolation that produces less jagged edges artifacts. Even so, the problem with high quality Bayer interpolation that it consumes lots of computing resource power.  The most optimized algorithm should provide higher image quality and be efficient at the same time. Alexey and Lukin Bayer pattern interpolation has lead to better color interpolation and minimizes artifacts. More information can be found on "An Improved Demosaicing Algorithm" (PDF file).

The above study is just one of many BAYER PATTERN interpolation algorithms. As algorithms are being improved, so does the ability to achieve more accurate colors  that will lead to better image quality. Another approach is to improve the lossless compression's algorithms. The main target of those studies is to provide a way to create a reversible Bayer image compression algorithm. Yuk-Hee Chan from the Hong-Kong Polytechnic University has come up with its own improved method. The main idea behind Yuk-Hee Chan's work is to show that it compressing then demosaicing the data is better than demosaicing and then compressing it. Of course it has to be done by minimizing as possible the computing resource power needed to perform this algorithm.  More information can be found on "A Fast Reversible Compression Algorithm for Bayer Color Filter Array Images" study by King-Hong Chung and Yuk-Hee Chan, PDF file here.

I am not a mathematician. The whole section was meant to show you that Bayer interpolation and image compressions algorithms keep improving.

Imaging Processor & Computing Power

When I went through many Bayer interpolation case studies, I could understand the importance of the algorithm complexity and efficiency. As computer power gets cheaper, more sophisticated algorithms can be used.
RAW processing takes time, whether it is in your camera or on your computer. Because current digital cameras don't have the resources as a home computer, efficient interpolation algorithms have to be used for in-camera RAW conversion.

Camera manufacturers keep improving the image processing engine, incorporating improved demosaicing, noise reduction and image processing algorithms in it.

For example: Olympus imaging processor is called TruePic, Nikon has its EXPEED processor and Canon imaging processor is called DIGIC. Camera manufacturers employ their own advanced hardware architecture and image processing algorithms to improve the image quality.

Few of the imaging processor responsibility:

• High-speed processing of camera RAW data to full-color image
• Minimizing image artifacts (moiré patterns, jaggy edges, color fringing, etc.)
• Reduce false color and enhance the dynamic range
• Improve signal to noise ratio
• Improve continuous shooting speed and the overall camera performance
• Reduce noise in high-ISO levels and during long exposures
• Employs the processing power for Face detection and other related technologies
• Provide faster memory cards access
• Better tonal gradiations

Many imaging processing engines are individually customized and optimized for different cameras.

You may ask yourself why not edit RAW files in-camera rather than on your computer? - there are several reasons for that but first let's look at the advantages of shooting RAW images over JPEG.

RAW vs. JPEG - Cons and Pros

I am pretty sure that it's not the first time you read a RAW vs. JPEG section. Both Raw and Jpeg file formats exist and for a good reason. Every one of them has its own usage for different types of photographers. For one photographer shooting Raw can be a disadvantage and for the other one it will be an advantage. I want to bring up some key advantages and disadvantages as some photographers look at it, and add my personal opinion about it.

As I mentioned earlier in this guide, Jpeg images are compressed 8 bit image files, whether RAW files are an exact unprocessed data that was captured by the sensor. The differences are quite obvious, and some photographers prefer to shoot only in RAW mode, some in JPEG and many photographers shoot in RAW+JPEG mode.

The main differences between Jpeg and Raw are related to FILE SIZE, DATA and IMAGE PROCESSING. Let me explain a bit more on every one of them.

File Size

JPEG files are stored in 8-bit per color, whether RAW files usually contains 12-bit color.  So Raw files actually contains more data. The problem with Jpeg files is loss of data during the compression process. It means that some of the image data is lost forever. In contrast, with Raw files you get to keep all the data that was originally stored in the RAW file.

More data means larger file size. The RAW file size is approx. two or three times larger compared to a JPEG file. Some photographers see it as a disadvantage, after all, who want to clog its memory cards or hard-drive with (approx. 13MB) Raw files. Jpeg (8MP) filesize is around. 3MP, so the difference is quite obvious.

Even so, lets be realistic and not complain too much. After all, the price of storage is getting cheaper and you can buy a 1TB External Hard Drive for around $75! (internal HD are even cheaper).

A 1 TB hard drive can store:

1024GB (1 TB) x 1024MB = 1,048,576 Megabyte / 13.312 Megabyte (estimated 13MB RAW file size) =
78,769 RAW images on 1 TB Hard drive.

That's a lot of Raw storage space don't you think? - And  don't forget that the prices are getting LOWER and the capacity storage is getting LARGER! (I hope I've done the calculation right :) ).

When it comes to memory cards capacity and prices, it can be an issue for some photographers. Memory card capacities keep increasing and the prices keep getting lower. Even so, there are some photographers who shoot hundreds or even thousands pictures in one day, but don't have the money for high capacity cards. For them, the best option is to shoot in Jpeg mode and transfer their images to an external storage device. Camera memory cards are being offered in GB not TB (for now). For example, Transcend 400X 64GB CF costs around $170, more than two times than 1TB external HD.

What you should know is that RAW file size is larger than Jpeg. You will decide whether it is an advantage of disadvantage for you.

Data
RAW files contain the full image data as it was captured by the camera's sensor.  That means that you have more editing flexibility when it comes to editing your images on your computer. Because RAW files contains more luminance data, photographers can reconstruct clipped highlights and shadow details that otherwise would have been lost.

When shooting RAW, the image is saved without the sharpness, brightness, white balance and other in-camera setting adjustments. When the RAW file is opened in RAW editing software, you will see a flat looking image this is because it hasn't been processed yet. When you bring the Raw file into Adobe Lightroom or example, you can apply contrast, sharpness, exposure and other fine-tuning adjustments to the image. By doing so, you have better control over how the image will look at the end.

You can look at it as a compromise between Flexibility and Accessibility.  If you want to shoot images for large prints, fix color casting and exposure, Raw is probably the way to go. I personally think that using RAW is a natural procedure in an image production pipeline that should not be easily ignored.

Before I continue, take a look at this RAW vs. Jpeg video (images shot in RAW+JPEG)

You should know that there are professional photographers who prefer shooting in RAW and others that prefer shooting in JPEG. It is all depends on many aspects and their is nothing that is "Best" for everybody. I think that the pros of the RAW format outweigh the cons.  You can surely expect to achieve higher image quality when shooting RAW, and it is your decision whether it fits your needs.

Image Processing
Some will see it has a must and some will eliminate this possibility across the board. Image processing or Image editing is a crucial step for producing better looking images. It is not a "Must" procedure, but certainly one that can help you become more creative as an artist. Yes, PHOTOGRAPHY IS AN ART!

RAW files have to be processed before you can view them on your computer. Even so, I have came across FastPictureViewer codec pack that automatically decodes Raw files for fast viewing. FastPictureViewer codec pack 2.4 decoded many Raw image formats like: .dng, .cr2, .rwl, .nef, x3f (Sigma raw image), .arw, .srw and many others. There are many free RAW viewers that can help you view RAW files without processing them first in RAW converters.

RAW Converter Software
In order to produce the final image, RAW files need to be processed using Raw converters. The advantages of using a Raw file software are related to the variety of image editing functions. Furthermore, RAW software takes advantage of the high processing power of your home or work computer. It means that the process will be much quicker and allow more sophisticated Bayer interpolation algorithms to be used.

Raw editing software is also known as "Raw converters".Here is a list of some popular RAW editing software:

• Adobe - Adobe Photoshop Lightroom 3 | Adobe Photoshop CS5 (Mac and Windows)
• Bibble Labs - Bibble 5.0 Lite & Pro (Mac, PC and Linux)
• PhaseOne - Capture One 5 Pro (Mac and PC)
• Silypix - Developer Studio 4.0 (Mac and PC)
• Raw Magick - RAW Magick Lite (PC)
• HeliconSoft - Multi threaded DCRAW (PC only)
• Corel - PaintShop Photo Pro X3 (PC and Mac)
• Stepol - Raw Importer (Windows)
• DxO Labs - DxO Optics Pro (PC and Mac)
• LaserSoft Imaging - SilverFast DCPro (PC and Mac)

This is a short list of free and paid Raw converter software. There are other Raw converters that were developed by the camera manufacturer, so you probably also want to take a look at that. I have read a very nice comparison article on ShutterBug.com that compares Adobe, LaserSoft, Bibble Labs and DxO labs converters, click here for the article.

Photographers will need to learn how use Raw editing software in order to properly take advantage of their Raw images. Even so, many of those software have a low learning curve and very useful tutorials that will teach you how to use them.

Another advantage in shooting Raw are HDR images. When you shoot RAW you can create beautiful High Dynamic Range images. I won't get deeper into it but if you are interested, you can read this HDR guide on vanilladays.com and on hdrsoft.com which I have personally found it very useful.

In conclusion:
If you want to get the best image quality, flexibility and become more creative with your camera, RAW is probably the best way to go. It has its advantages and disadvantages, and it will probably won't suite for everybody. I honestly think that the pros outweigh the cons, and if you are passionate about photography, you should at least consider this option.

To RAW or not to RAW!

I have surely learned more about RAW as writing this article. I can see why many photographers love to shoot RAW and that explains why there are so many Raw editing software out there. After reading the pros and cons of shooting RAW, I can not ignore the fact that it is not for everybody. In order to view RAW files, you will need a dedicated software and RAW file size is bigger than JPEG. Even so, I think that editing RAW files with Raw converters is FUN. Even if you are not a professional photographer, you can enjoy the flexibility of the RAW file format and have more creative freedom. Yes, it is related to the creative side of people. I get really excited when I sit in front of my computer to create new graphics designs, I really enjoy it. It is not different with RAW editing, and for me it is a pleasure to use.

When I browse the Internet I have seen many debated whether photographers should shoot RAW or not. If you understand and advantages in shooting RAW and you know how to take advantage of it, you can really get more creative with it. Of course some people will tell you that they rather shoot more JPEG images than spend time working on their RAW files. You are the only one who can really understand your needs! - Read, learn and experiment with RAW conversion software. I am sure that once you try it, you will come to a conclusion if it is right for you or not.