The loss in contrast-discrimination ability of the human visual system under high ambient illumination level can cause image quality degradation in mobile phones. In this paper, we propose a perceived contrast compensation method by processing the original displayed image. With consideration that the perceived contrast significantly varies across the image, this method extracts the local band contrast from the original image; it then compensates these contrast components to counteract the perceived contrast degradation. Experimental results demonstrate that this method can maintain most contrast details even in high ambient illumination levels.
Mobile phones are viewed under various illumination levels. The displayed images on a mobile display can be perceived with a significant loss in contrast in daylight conditions. Image quality degradation in ambient lighting is mainly caused by screen reflections and light adaptation of the human visual system (HVS). Screen reflections decrease the contrast and color gamut of a mobile display by increasing the luminance of dark areas. Light adaptation of the HVS is another important factor. Images displayed in daylight are perceived as relatively dark due to light adaptation of the HVS because the luminance of a mobile display is considerably lower than that of the outdoor environment
. Moreover, the perceived image contrast changes according to different surround luminance levels
. CIECAM 97s
color appearance models, developed by the Commission Internationale de L'Eclairage (CIE), determined the surround compensation ratios for average, dim, and dark levels. Various algorithms, such as illuminant adaptive color reproduction
and veiling glare correction
for mobile displays, have been proposed to solve this image quality degradation problem. Display adaptive tone mapping operators can also be used to minimize visible contrast distortions for mobile displays
. All these works considered little about the spatial contrast discrimination ability of HVS in ambient lighting.
In recent work, Kim
examined the effects of surround luminance on the shape of the spatial luminance contrast sensitivity function (CSF), which represents the amount of minimum contrast at each spatial frequency that is necessary for a visual system to distinguish a sinusoidal grating or Gabor patterns over a range of spatial frequencies from a uniform field. In bright surrounding conditions, a large amount of reduction in contrast sensitivity at middle and high spatial frequencies can be observed; however, only a small amount is observed at low frequencies. As a result, a greater number of image details cannot be detected because the contrast perception threshold rises in ambient lighting, especially at middle and high frequencies.
shows the degradation of contrast perception for a middle frequency sine wave image.
(a) Sine wave image. The sine amplitude gradually increases from 0 to 1, representing how the contrast becomes greater from top to bottom. (b) The same image perceived in ambient light. It is perceived as dark, and the contrast perception threshold becomes larger, representing how additional details could not be detected.
In later research
by Kim, a global image enhancement method is proposed to compensate the perception degradation. This method enhances whole spatial frequencies of the display image based on the degraded contrast sensitivity function with a frequency filter. Kim’s method implies that the contrast sensitivity function measured at various spatial frequencies can be implemented as a modulation transfer function of the system in the Fourier domain for image filtration.
Employing an image processing method to obtain the same perceived image for viewers is an attractive idea. However, the perceived contrast may vary greatly across the image because human contrast sensitivity varies with local average luminance
. Thus, the perceived contrast in complex image cannot be represented by a single value or different values in the spatial frequency domain. A number of “local band contrast” measure methods have been proposed for other image processing approaches. These definitions of contrast
in these methods assign a contrast value to every point in the image for each spatial frequency band. Numerous applications have been identified for these definitions, especially in HVS-related image processing problems
The objective of this paper is to enhance the local band contrast in images to compensate for perceived contrast degradation in ambient light. We proposed a perceived contrast compensation method based on local band contrast, which is extracted from the displayed image considering display factors. As our aim is not to mitigate the degradation in image quality caused by screen reflections in ambient lighting, our proposed method avoids changing other image properties, such as lightness and hue. Thus, it can be combined with other illuminant adaptive color reproduction and tone mapping methods.
II. HVS CONTRAST SENSITIVITY REDUCTION
The HVS is a nonlinear system with a very large dynamic range that behaves as a band-pass spatial filter
. The spatial filtering property is characterized by the CSF, which describes how contrast sensitivities vary as a function of spatial frequency. CSF exhibits a peak in contrast sensitivity at 4.0 ~ 5.0 cycles per degree (cpd)
and drops at both lower and higher frequencies. By measuring the minimum contrast at each spatial frequency in experiments, it has been deemed necessary for visual systems to distinguish a sinusoidal grating or a Gabor pattern over a range of spatial frequencies from a uniform field.
The CSF model used was originally proposed by Barten
as a function of spatial frequency and as dependent on field size (or viewing angle in degrees) and mean luminance of the sinusoidal grating stimulus. As the mean luminance of the sinusoidal grating stimulus decreases, the contrast sensitivity at each spatial frequency and maximum resolvable spatial frequency decrease. In addition, the shape of luminance CSF likewise changes: the peaks in the functions shift toward lower spatial frequencies, broaden, and eventually disappear
Kim identified characteristics of the HVS in the spatial frequency domain by considering surround luminance
as shown in
. In general, the effect of surround luminance on the luminance CSF appears similar to that of mean luminance. A variable for compensating for surround luminance effect φ is multiplied with adaptive luminance L, as shown in Equation 1
Comparison of CSFs under varying ambient luminance. As the surrounding luminance increases from 0 nit (cd/m2) to 10,000 nit, contrast sensitivity at each spatial frequency decreases. A large reduction in contrast sensitivity at medium and high spatial frequencies can be observed. However, only a small reduction is observed at low frequencies.
is the mean luminance of white and black of the display under the given surround luminance
, and φ is the surround luminance effect function as given in Equation 2.
The surround luminance effect function φ forms a nonlinear function of luminance for approximating the perceived brightness reduction effect caused by increases in ambient illumination level. The surround luminance
can be measured by a luminance meter or be estimated by Equation 3.
is ambient illumination given in
is the reflectivity for surround surface. The surround adaptive CSF works as a weighting function to determine which parts of the image, regardless of their spatial frequency, will appear under a given ambient illumination level.
According to the HVS model, an image detail can be detected if its contrast is greater than the threshold. Consequently, the amount of perceived contrast information changes with the contrast threshold level. Because ambient lighting affects the contrast threshold level of human vision, the same information for the image displayed at different ambient lighting levels cannot be perceived.
III. PERCEIVED CONTRAST COMPENSATION METHOD
An image stored as a matrix of pixel values is the most common representation for image processing; however, it does not reflect the way humans perceive images
. The response of the HVS depends much less on absolute luminance than on the relation of local variations to surrounding luminance. Meaningful visual information in images is mainly conveyed by contrast because the HVS has specialized cells that process signal contrast (rather than absolute signals)
. The proposed method serves to compensate local contrast in images to improve image quality in ambient light.
Perceived contrast may vary greatly across the image. To simulate nonlinear, threshold characteristics of spatial vision, several local band contrast definitions have been proposed
. These definitions provide a method for transforming a pixel value image into several local band contrast images. Inspired by this research, we propose a perceptual contrast processing framework as depicted in
Data flow of perceptual contrast processing in the proposed framework.
In this method, the image is not directly enhanced; instead, the image is transformed into local band contrast data. Because these contrasts degrade under ambient illumination as described by the Kim CSF, our method enhances them to compensate the details and reconstruct the output image with these enhanced contrasts.
- 3.1. Local Band Contrast Computation
The contrast of simple patterns, such as the sinusoidal pattern or a square on a uniform background, is well defined as a single value, such as the Weber contrast and Michelson contrast; however, this is not true for real-world images. As mentioned earlier, perceived contrast may greatly vary across the image because human contrast sensitivity varies with local average luminance. Peli
proposed a complex contrast model in which the contrast of an image can be locally defined. Because human contrast sensitivity is highly dependent on spatial frequency, local contrast for each spatial frequency band is separately calculated. Lubin
modified Peli’s contrast for image quality metric, and Winkler
defined an isotropic contrast measure computed from an oriented filter. Since these definitions do not consider display factors, such as display black level and reflection light, thus resulting in contrast deviations in low luminance areas of images, we modify the local band contrast.
A mobile display is limited by maximum luminance and black luminance. The displayed luminance in ambient lighting can be modeled as Equation 4
) is displayed luminance,
) is the original pixel value (0-1),
is a display gamma (usually close to 2.2),
is the peak display luminance,
is the display black level, which is the luminance of the black pixel displayed in a perfectly dark room,
is ambient light reflected from a display surface.
)instead of the original pixel value, an image is decomposed into different frequency bands using a band-pass filter with a center frequency of 2
cpd as Peli’s method.
is the low-pass image and
are different band-pass images,
is the high-pass image. In our work, the different bands
are limited in range from 3 to 32 cpd, for instance, four different bands of 3, 6, 12 and 24 cpd, as the main differences between surround adaptive CSF and the original one are in the frequencies larger than 3 cpd and the sensitivity of human vision drops to nearly zero in the frequencies larger than 32 cpd.
For every band-pass filtered image
), we define the corresponding local luminance mean image
), which is a low-pass filtered version of the original image that contains the frequency components below the band of
The local band contrast image is calculated as in Equation 7:
- 3.2. Perceived Contrast Compensation in Different Band Contrast Images
The amount of perceived contrast information changes with the threshold contrast level. In
, fewer contrast details can be perceived in the high frequency contrast images (
) compared to the low frequency contrast images (
) because a larger number of contrast values in the high-frequency contrast images are lower than the threshold.
(a) Original Lena image and (b-d) Local band contrast images at 3, 12, 24 cycles/degree. Fewer contrast details can be perceived in the high frequency contrast images (c) and (d) compared with the low frequency contrast image (b).
Further, because ambient lighting affects the contrast threshold level of human vision, the same information for the image displayed at different ambient lighting levels cannot be perceived. For example, assuming that the surround luminance is 3000 cd/m
, a greater number of details is below the threshold and fewer are perceived, as shown in
. Most contrast information is lost in the middle frequency, while only a small amount of information is lost in the low and high frequencies.
(a) Contrast threshold functions under different surround luminance. The contrast threshold, which is the reciprocal of contrast sensitivity value, represents the just noticeable contrast in each frequency band. A higher contrast threshold value implies that a greater contrast cannot be perceived in an image. (b) Ratios of local band contrast values above contrast threshold under different ambient illuminations. These data comes from local band contrast images of Lena at 3, 6, 12, 24 cycles/degree. The decreased ratios reflect that fewer contrast details can be perceived in a high ambient illumination.
The ratios of different band contrast details are calculated as in Equation 8.
is the number of pixels for which the contrast value is larger than the contrast sensitivity threshold, and
is the total number of pixels of the band contrast image.
To compensate for the loss in image contrast caused by an increase in surround luminance, the compensation ratio in different bands should be confirmed. These compensation ratios of different bands are determined by the contrast threshold difference between the reference (dark level) and given target surround luminance. In each contrast image, the contrast value above the threshold should be amplified by the matching ratio; those below the threshold are not processed to prevent noise amplification because human vision cannot detect them even in an environment without ambient lighting.
is the reference luminance and
is the given surround luminance. The reference luminance can be chosen as an ideal environment luminance value or a low-level luminance value in a darkroom. In our work, we set the “reference luminance” as 0 cd/m
Note that contrast masking is not involved when counting compensation rate for different frequency band here. Contrast masking
typically involves an increase in the HVS contrast threshold for a signal in the presence of another one. Some contrast information above the contrast threshold may be undetectable for the contrast masking phenomenon. In contrast perception evaluation, ignoring the contrast masking effect will result in an excessive contrast estimate. However, in the compensation method, the contrast masking phenomenon exists in both the original and the output image, because of which their effects are counteracted to some extent. Contrast masking is hence not involved in our proposed method.
Finally, the output image is reconstructed in incremental steps by these enhanced contrast images. The reconstruction must be a low frequency to high frequency progression because
) are generated by
). By taking all
) into Equation 5, the final displayed luminance image can be obtained as shown in Equation 10.
IV. EXPERIMENTAL SETUP
The aim of our proposed approach is to compensate for all contrast details to match those in an ideal environment without changing other image characteristics. We therefore designed an experiment to test contrast detail visibility in different environments.
A darkroom with a large LED display was selected as the test environment (
). This display was designed with surround lighting for which the brightness could be automatically adjusted, and the color temperature was approximate to D65. A mobile phone was placed in front of the LED display and the same orientation was maintained. This ensured that the background brightness could be accurately controlled when viewing the mobile phone. The LED display was secured as the only light source in the darkroom; therefore, light reflected on the mobile phone screen would be controlled at a low level.
Experimental Setup. (a) Schematic top view of experimental setup. (b) Experiment environment photograph.
Five test images were selected from the LIVE Image Quality Assessment Database
. The images were of a lighthouse, a sailing boat, a statue, a standing woman and a woman in a hat as shown in
. Because our method only enhanced the contrast extracted from the gray image, these RGB images should be converted into the HSV. Only brightness was applied through the enhancement procedure; chrominance properties were preserved.
Five test images selected from the LIVE image quality assessment database.
The test comprised two stages, one in the darkroom environment and the other in a high ambient light environment, controlled by LED display adjusted in 3000 cd/m
as the surround luminance. In the darkroom, the five images were tested. In the high ambient light environment, the five images were enhanced by Kim’s method and our proposed method. As a result, a total of 15 test images, including five original ones and 10 enhanced ones, were randomly displayed for testing in the high ambient light environment.
In both stages, observers sat in front of the display and gradually adapted to the ambient light. The 4.7-in mobile phone—placed at a fixed distance of 25 cm—randomly displayed each image. All images were randomly display for two rounds. In the first round, observers just browsed all images and were told to notice “detail visibility” of each image. In the second round, they were asked to score each image by assessing the level of “detail visibility”. The score was a 5-rating scale, from “hardly visible” (1) to “the most visible” (5).
V. RESULTS AND DISCUSSION
- 5.1. Experimental Results
Nine observers (five male and four female) with normal color vision were selected as subjects for our experiment. The experimental results, average scores for each test image, are provided in
. The detail visibility of images processed by our proposed method are higher than that of the original images in the ambient light and enhanced by Kim’s method, and nearly equal to that in a dark room.
Five levels of detail visibility
*Origin 1 refers to the five original images tested in the dark room. Origin 2 refers to the five original images tested in the ambient light environment.
, two test images enhanced by Kim’s method and the proposed method are presented. In both methods, the contrast of details has significantly improved, as shown by the cloud and grass in
, the rock and leaves in
. Kim’s method globally enhanced all frequencies in the Fourier domain, including the direct current component, with consideration of the contrast masking. As a result, we can observe higher brightness and the loss of contrast details in the highlighted region (as shown by the lighthouse in
Two test images (a) and (d), corresponding to enhanced images (b) and (e) obtained by Kim’s method, and (c) and (f) corresponding to the proposed method. Under surrounding luminance of 3,000 nit, since ambient lighting affects the contrast threshold level of human vision, some contrast information of test images (a) and (d) cannot be perceived. In enhanced images, the contrast in the details is significantly improved, as shown by the cloud and grass in (b) and (c), and the rock and the leaves in (e) and (f).
In the proposed approach, only local band contrast is extracted from original image and adjusted, so other image characteristics, such as lightness and hue, are maintained. On the other hand, as the compensation ratios for each of the local band contrasts are strictly deduced by the CSF degradation value as Equation 9, an over-enhancement problem can be avoided. So this approach can enhance the contrast of images without changing other image characteristics.
- 5.2. Contrast Ratio Metric
Limited research has been conducted on the influence of viewing conditions in objective visual quality metrics; moreover, the issues relating to ambient illumination have been largely unexplored
. Here, we use a ratio of perceived contrast amounts in ambient lighting to those in an ideal environment to evaluate the compensation effect. We sum all pixels in different contrast bands to determine the perceived contrast ratio (CR) as Equation 11:
is the pixel number of which the contrast value is larger than threshold
in band i enhanced contrast
is that in
. The sum of
reflects the perceived contrast amount in ambient lighting and that of
reflects the amount in a reference environment.
, perceived contrast ratios of the origin image in ambient light are quite small, for many contrast details are lost. In both Kim’s method and the proposed method, the contrast ratios become larger as contrasts increase; however, they nevertheless do not reach 100% of Origin1 because the contrast gains are limited by the dynamic range of the image.
Perceived contrast ratios in five images
Perceived contrast ratios in five images
- 5.3. Dynamic Range
The dynamic range is a limitation of this contrast compensation method. Because all band contrast images are enhanced by a factor greater than 1.0, the range of final compensation images—the sum of all band images—is typically greater than the original range of gray levels. Therefore, the final image does not provide sufficient enhancement at different frequencies, especially at the high frequency. Fortunately, as the output image is reconstructed from low frequency band contrast to a high frequency one, in most cases only the highest frequency band contrast, which contains less detail than other bands, is limited by dynamic range. In Kim’s method, all frequencies are enhanced with consideration of the contrast masking, which results in more gray values exceeding the dynamic range of the image and smaller contrast perceived ratios as in
In this paper, we proposed a perceived contrast compensation method that processes the extracted local band contrast from the original image. Experimental results demonstrated that this method can effectively enhance contrast details without changing other image characteristics; moreover, most contrast details can be compensated even in high ambient illumination. Although the image dynamic range may be a limitation of our method, only a small number of contrast details are limited by it.
We thank Shengchun Yuan for providing experimental facilities; Wenjin Chang and Xiaoqian Fan for critically reading the manuscript. This work was supported by “the Fundamental Research Funds for the Central Universities” (K5051205011).
“Illuminant-adaptive color reproduction for mobile display,”
Baek Y. S.
“Determination of the perceived contrast compensation ratio for a wide range of surround luminance,”
J. Opt. Soc. Korea
DOI : 10.3807/JOSK.2014.18.1.089
Fairchild M. D.
“A revision of CIECAM97s for practical applications,”
Color Research & Application
DOI : 10.1002/col.1061
Fairchild M. D.
Hunt R. W.
Luo M. R.
“The CIECAM02 color appearance model,”
Proc. Color and Imaging Conference
“Illuminant adaptive color reproduction based on lightness adaptation and flare for mobile phone,”
Proc. Image Processing, 2006 IEEE International Conference
Luo M. R.
Li C. J.
Park D. S.
“Correcting veiling glare of refined CIECAM02 for mobile display,”
Color Research & Application
DOI : 10.1002/col.20739
“Display adaptive tone mapping,”
ACM Transactions on Graphics (TOG)
New York, USA
Kim Y. J.
“An automatic image enhancement method adaptive to the surround luminance variation for small sized mobile transmissive LCD,”
Consumer Electronics, IEEE Transactions on
DOI : 10.1109/TCE.2010.5606241
“Computing isotropic local contrast from oriented pyramid decompositions,”
“Multiscale contrast enhancement with applications to image fusion,”
DOI : 10.1117/12.56155
“Adaptive local contrast enhancement method for medical images displayed on a video monitor,”
Medical Engineering & Physics
DOI : 10.1016/S1350-4533(00)00014-X
“Model of luminance contrast‐sensitivity function for application to image assessment,”
Color Research & Application
DOI : 10.1002/col.20230
Wandell B. A.
Foundations of Vision
Barten P. G.
Contrast Sensitivity of the Human Eye and its Effects on Image Quality
SPIE Optical Engineering Press
Rohaly A. M.
“Global spatiochromatic mechanism accounting for luminance variations in contrast sensitivity functions,”
DOI : 10.1364/JOSAA.6.000312
“Spatial resolution by the human visual system. The effect of mean retinal illuminance,”
DOI : 10.1364/JOSA.56.000689
“A perceptual framework for contrast processing of high dynamic range images,”
ACM Transactions on Applied Perception (TAP)
DOI : 10.1145/1166087.1166095
Hubel D. H.
Eye, Brain, and Vision
Scientific American Library/Scientific American Books
“A visual discrimination model for imaging system design and evaluation,”
Vision Models for Target Detection and Recognition
Sheikh H. R.
Bovik A. C.
“LIVE image quality assessment database release 2,”
Jay Kuo C.-C.
“Perceptual visual quality metrics: A survey,”
Journal of Visual Communication and Image Representation
DOI : 10.1016/j.jvcir.2011.01.005