The Complexity of Image Analysis, Part 2 - Colors
by Eric Berry
Human Perception
"True" color derives from the intrinsic properties of light. The representations of various wavelengths define color. White light, when passed through a prism, will split into its constituent components. Similarly, a wavelength of a particular frequency is defined to be a particular color. But humans only understand color to the extent that it is perceived. If the human eye can't see a particular frequency, then it is not perceived at all.
Humans do not have a sensor for each frequency of light - nor would that make much sense (you'd need an infinite number of sensors). Instead, rods and cones are employed. Cones focus on recognizing brighter light (photopic vision) with more sensitivity about individual colors, rods on dim lighting scenarios (scotopic vision) with less color sensitivity, and a combination of the two for in-between lighting (mesopic vision). But there is not a linear relationship. Instead, cones detect light along something resembling a gaussian curve, centered at different wavelengths - with non-linearly decreasing sensitivity as the amount of light decreases. There are three wavelengths at which the cones are centered, as shown below.
Rods, meanwhile, have roughly a single focal wavelength and decay nonlinearly. They are much more sensitive than cones in dim light scenarios. Perception of color is thus 1) the sum of the cone sensitivities for that particular frequency, for that particular level of brightness in conjunction with 2) the sum of the rod sensitivity for that particular frequency, for that particular level of brightness. Yellow light, thus is a response both from the red and green sensors at fairly high levels of sensitivity - so here a single wavelength of light is perceived as a combination of two different wavelength sensitivities. If that exact spot of light were instead a combination of the two wavelengths that mapped to that color, we would not be able to discern the difference.
Translating Human Perception to Computers
In the "real world", none of the above particularly matters. Human rods and cones work as they do, and everyone sees just fine. The challenge comes when you try to display colors for humans to see a particular way. In the simplest form, an image can be decomposed into color channels. So a full color image can be 3 channels, representing red, blue and green, each effectively a grayscale of the amount of that particular color for that particular pixel.
When the values for each pixel are combined, the colors are "added" together. Thus the output for that pixel is the amount of red, plus the amount of green, plus the amount of blue. The absence of any value for a pixel would be black, whereas full representation of each yields white. Note that this result is because the monitor is emitting light. When printing to paper, the result would be the opposite because the question is of reflecting light (or not). This introduces a whole new set of challenges that led to the CMYK color space. Specifically, how can you represent a color through the absorption vs the reflection of the ambient light.
As we discussed above, human color perception is not linear, and it's hard to embody accurately. For example, the sRGB color space (as opposed to the RGB color space above) has, been developed to compensate for the difference between perceived darkness and the actual value. For example, if light intensity goes from 0 to 1, humans do not perceive 0.5 as an even gray, but rather a significantly darker color. The gradient below is, in fact, a pure gradient from 0 to 1 on intensity - but humans perceive it as significantly darker. The gradient appears to be "dark" for at least 2/3 of the image.
Correcting for Colors
sRGB compensates for the differential between perceived and actual darkness by adding "gamma." This effectively changes the saved color value so that it is perceived by humans the way it is intended to look. This is, however, a particularly loose way to compensate for the differences between human vision and true light - more complex corrections are generally made in practice. But there is then the question of monitors. Not every monitor is created equally, and some will display colors with different gammas, leading the same image to appear differently across different monitors, different lighting conditions, and even different regions of the same monitor.
As discussed above, perceptual sensitivities are non-linear - and are, in fact, a function of how the eye's components respond to the wavelengths in question. So, if one were operating in the RGB world, where each pixel has a value from 0-255 for each color space, adding 10 to each of the R, G and B channels would not be perceived as exactly that difference in colors. So RGB is not effective if one wants to understand how different two colors look, as opposed to how different the colors "are."
To this end, the color space LAB (among others) was developed. The LAB color space has 3 channels that represent lightness (L), and two color-opponent dimensions (a and b). The L channel is meant to closely approximate the human perception of brightness (see above regarding non-linear brightness perception). LAB resulted from a laborious effort to ensure that euclidian distance between pixel values represents a euclidian difference in terms of perception.
Converting RGB to LAB
Simply converting RGB colors - which at its core is how humans understand colors, to the "true" representation LAB space, is a cumbersome process. The steps in the transformation are below.
First, convert RGB to XYZ (another color space).
Then solve for Lab.
Where
In each case, Xn, Yn and Yn are the tristimulus values of a specified white object. This is often a hypothetical material with unit reflectance and which follows various defined physical properties. This is thus normalizing for the appearing of a pure or known perceptual value in the image.
