Color
Seamlessness in Multi Projector Displays
Large are high
resolution displays are essential for scientific visualization, entertainment
and defense applications. A popular way to realize such displays is to tile
multiple projectors together to create one large display. As opposed to a 19”
diagonal monitor with 60 pixels/inch resolution, tiled multi projector displays
are often 10ft x 8ft is size has 100-300 pixels/inch resolution. Thus we have
in the order of 100 million pixels.
One of the most important issues is to
make these displays seamless.
|
Display Before Correction |
Display After Correction |
Background on Color
Color
is a three dimensional quantity and can be represented in many ways. We will be
using two different representations. In the perceptual representation, color is
defined by its luminance (or radiance) and chrominance. The one-dimensional
luminance defines the brightness of a color while the two-dimensional
chrominance defines the hue and saturation of a color.
The second representation
is the one that we usually use in computer graphics where a color is defined by
three primaries of red, green and blue (RGB). This is the representation in
which the color is defined in the projector hardware. The range of colors that
can be produced by a device can be expressed by a 3D volume which is called the
color gamut of a device. Here is an example of the power point palette where we
often work with both these representation.
Here
the color palette consists of a two dimensional chrominance palette and a one
dimensional luminance slider. We humans usually find it comfortable to work
with. Once we choose a color using this representation, in the bottom we also
get the corresponding RGB representation of the same color.
Main Contributions
My thesis has
two main contributions.
1)
I
design the emineoptic function that models the color
variation in multi projector displays. The emineoptic
function helps us to provide a formal definition of color seamlessness as an
optimization problem.
2)
From
several empirical analysis of projectors, we find that
the chrominance variation is almost negligible when compared to the luminance
variation across a multi-projector display, especially when the projectors are
all of same model. Hence, we design a practical system that solves for the
restricted problem of luminance or radiance variation and thus achieves photometrically seamless displays.
We call this system PRISM for
Perceptual Radiance
Seamlessness in Multi Projector Displays. The term ‘perceptual’ here will become clear subsequently.
Properties of Color
Variation
I
categorize the color variation in multi projector displays in three different
categories.
|
Spatial
variation in the chromaticity coordinates across a single projector. |
The green
surface shows the luminance variation for input (1,1,1)
across a single projector. The red surface shows the black offset. |
Intra Projector Variation: This is defined as the color variation within
a single projector. The spatial luminance variation across a single projector
shows significant fall off from center to the fringes. However, the chrominance
remains almost constant spatially. These are illustrated on the left. Further,
usually in any display device, the amount of light projected for input (0,0,0) should be zero. However, this is not true for
projectors, and some light is always projected. This is called the black offset
as is illustrated on the left.
Inter Projector Variation: This is defined as the
color variation across different projectors. From extensive empirical analysis,
we found that the variation in chrominance across different projectors of same
model is negligible. However, the variation is luminance is significant. Even
for projectors of same model, the variation in chrominance is much smaller when
compared to that of luminance.
Overlap Variation: This is defined as the
variation caused by the overlap between the projectors. If the chrominance of
the overlapping projectors are similar, the chrominance of the overlap region
varies negligibly than the non overlapping regions. However, even if exactly
identical projectors are overlapped, the luminance in the overlap region is
multiplied by the number of overlapping projectors.
Emineoptic Function
|
|
|
|
Predicted
Response |
Actual
Response |
To
correct for the color variation, one needs to first capture the color variation
in a multi projector displays. To this effect, we design the emineoptic function that models both the luminance and the
chrominance variation in multi projector displays in a compact manner using a
few parameters. The emineoptic function defines the
color of the light reaching a viewer from a display coordinate for a particular
input projected from all contributing projectors at that display coordinate.
Now, to capture the color variation, instead of taking 224 images
for 224 color which is both compute and
storage intensive, one can reconstruct the few parameters of the emineoptic function and predict the color variation using
the emineoptic function.
Thus,
the emineoptic function identifies the different
parameters that causes the color variation and provides a comprehensive
framework to describe the intra, inter and overlap color variation. Thus, it
provides an unifying framework to study all different
algorithms for correcting the color variation problem across multi projector
displays. It can be shown that any algorithm to correct the color variation should
reconstruct the emineoptic
function, modify it and then reproject the modified emineoptic
function on the display. The accuracy of the method depends on the parameters
of the emineoptic function reconstructed. The success
of the color correction achieved depends on the method used to modify the emineoptic function. The interactiveness
of the method depends on the parameters chosen for the reprojection.
Further, this model is
general and can also be used to model the color of cameras or any other device.
To
verify the model, I reconstructed the different model parameters and predicted
the response of the display to an test image displayed
on it using the emineoptic function. This is called
the predicted response. Then, I compared this with
actual picture taken of the test image being projected on the display. This is
called the actual response, as shown above.
Color Seamlessness
There
has been a general mindset that to achieve truly seamless displays, one needs
to match the color response at every pixel of the display. However, due to the
acute spatial variation, this often means that we match the color response of
all pixels to the worst possible pixel ignoring all the good pixels which are
very much in majority. This has been the main assumptions behind many of the
previous work in this direction like blending, gamut matching and using a
common bulb for different projectors. Now, we are going to show that achieving
seamlessness does not mean a strict
uniformity of color responses across all display pixels.
|
Luminance
variation across a single projector for input (0,0,1)
at all display coordinates. |
Luminance
variation across a 2x2 array of four projectors for input (0,1,0) at all
display coordinates. |
Why do we see color seams?
As
mentioned before, the goal is to make a multi projector display look like a
single display. With that end, let us compare the luminance variation of a
single projector with that of a multiple projector as seen in the left. Note
that when a flat green field is displayed on a single projector, it looks flat
to the human eye. But the luminance response is anything but flat. In fact, it
can show as large as 80% fall off from center to fringes. However, the
difference of this from the luminance response of a multi projector display is
that it does not have sharp discontinuities, but has a slow gradual fall off. These
sharp discontinuities in the luminance variation of multi projector displays
are the cause of the color seams. This shows that perceptual uniformity does not necessarily mean strict photometric
uniformity. This is also supported by many perceptual studies that show that
humans cannot detect smooth color variations.
|
Luminance
response of the display before correction |
Luminance
response of the display after photometric uniformity. |
The
next question is how to remove these discontinuities from the luminance
function. A most simple way to do this is to match the luminance at every
display coordinate to the minimum luminance as shown in the left. This is
called photometric uniformity. However, there are two
disadvantages of this. First, this leads to a severe compression in the dynamic
range of the display. Second, it does not make use to available high system
resources. Thus, though we achieve seamlessness, the quality of display reduces
dramatically as shown below.
|
A 5x3 array of fifteen projectors before correction |
A 5x3 array of fifteen projectors after photometric uniformity |
This
idea can also be generalized to 3D color from just luminance. Thus, strict color uniformity which is the goal of all color matching algorithms would yield
poor display quality. This shows that strict color uniformity may not be the
most desirable option. We provide formal definitions of both photometric
uniformity and color uniformity using the emineoptic
function.
Optimization Problem
We
pose the problem of achieving color seamlessness as an optimization problem where the goal is to minimize the perceptual color variations while maximizing the
display quality. When this optimization is
done only on the luminance, we achieve photometric seamlessness as opposed to color seamlessness.
These two can also be defined formally using the emineoptic
function. Based on this idea we develop a system called PRISM that achieves
such a photometric seamlessness.
|
A 5x3 array of fifteen projectors before
correction |
A 5x3 array of fifteen projectors after
photometric seamlessness. |
PRISM:
Perceptual Radiance Seamlessness in Multi Projector Displays
PRISM consists of two steps.
1)
Off line Calibration
Procedure: This is repeated periodically and generates
what we call the smoothing maps for
each projectors.
2)
Online Image Correction: These smoothing maps can then be used to
correct any imagery on the display at interactive rates.
|
The maximum luminance function of the
green channel for a 5x3 array for fifteen projectors. |
Calibration
The calibration has three steps.
Reconstruction:
In this
step a digital camera is used to reconstruct the luminance parameters of the emineoptic function of the display. This includes the
maximum spatial luminance variation for each channel and the black offset, as
shown in the left. Also, the non- linear luminance response of each channel of
the projector is reconstructed.
Modification:
Next,
these luminance functions are modified using a gradient based optimization
process. Thus the modified luminance function has an overall high dynamic range
while the variation is controlled to be imperceptible. This is achieved by a
linear constraints and objective functions and is implemented using linear
programming. The amount of variation that can be tolerated also depends on the
content. For example, for a high frequency movie, a large amount of variation
can be tolerated, but for desktop environment that involves many flat colors, not
much variation can be tolerated. Thus, smoothing parameter is used to decide
the smoothness the luminance variation. Thus, the smoothest luminance function
is one that is flat. This is nothing but the luminance function corresponding to
the photometric uniformity. Thus, photometric uniformity is a special case of
photometric seamlessness. As we make the
luminance functions smoother and smoother, the display dynamic range becomes
smaller and smaller, reaching the minimum for photometric uniformity. These are
illustrated in the figure below.
Reprojection (Part I): In this step, smoothing maps (an attenuation
map and an offset map) is generated from the modified and the reconstructed
luminance functions. Further, a channel linearization function for each channel
of a projector is generated.
Image
Correction
Reprojection (Part II): In the image correction step, the attenuation
and offset map and the channel linearization function are used to correct any
image to be projected on the display. These can be achieved in real time using
the pixel shaders of the commodity graphics hardware.
|
|
|
|
|
|
|
|
|
|
The modified luminance function achieved
using different smoothing parameters. The smoothest is the flat luminance
function. |
The corresponding displays. As the luminance
function becomes smoother, the dynamic range of the display reduces. |
Results
Here are a few results.
|
A 5x3 array of fifteen projectors before
color correction |
A 5x3 array of fifteen projectors after
photometric seamlessness |
|
A 3x2 array of six projectors before color
correction |
A 3x2 array of six projectors after
photometric seamlessness |
Related Publications
- Properties of color variation in
multi projector display, Aditi
Majumder, Proceedings of SID Eurodisplay,
2002.
- Color Non Uniformity in Multi
Projector Displays: Analysis and Solutions, Aditi Majumder and
- LAM: Luminance Attenuation Map for
Photometric Uniformity in Projection Based Displays, Aditi Majumder and
- Identifying and Optimizing the Emineoptic Function for Color Seamlessness in Multi
Projector Displays, Aditi Majumder
and
Related Courses
- Large Area Displays for the
Masses, Aditi Majumder and Michael
Brown, ACM Siggraph, 2003.
- Building Large Area Displays, Aditi Majumder and Michael Brown, Eurographics 2003.
Keywords
Large
Area Displays, Tiled Displays, Multi Projector Displays, Color Matching, Color