Seamless Multi-Projector Displays Using Nonlinear Edge Blending

Show more

1. Introduction

Very large-area multi-projector display systems have wide applications since they provide immersive environments, high resolution, and a large field of view. Thus, they are extremely useful for visualizing large scientific models and in immersive virtual environments, In order to achieve large-area, high-resolution seamless display, more and more of research institutions and researchers began to study seamless technology based on multi-projector display [1] .

In the seamless tiled system, in order to eliminate image mosaic caused by cracks in optical and ultimately to achieve seamless tiled, the previous solution is to buy expensive projector or image fusion machine, which usually have an embedded projector lens and geometric optics chip calibration. But the integration technology of image fusion machine must manually adjust and do not have relevance. In recent years, in addition to considering screen, projectors, processors and other hardware, a large number of edge blending algorithms appear in the field of the edge blending [2] - [9] . Edge blending, that is, the two images into a region of overlap, which overlaps the region of the signal, according to a linear relationship between the fusion fade, makes a direct overlap between the “light zone” effects as much as possible to eliminate in order to achieve real seamless effect. Figure 1 shows a schematic diagram of seamless tiled algorithm.

However, these studies in the selection of the edge blending are a simple linear function, and therefore it does not substantially improve the output signal. In this paper, we first analyze the causes of the optical cracks. From this analysis, we introduce a non-linear and adjustable parameters function by modifying the original function of the edge blending, next, set the original image samples splicing region and then deal with the bright areas of overlap region by improved function of splicing. To the best of our knowledge, finally, we eliminate the effect of color distortion through the Gamma correction. And experimental results show that the proposed algorithm effectively eliminates the light generated by the edge of the integration region and the image has good transitions, the proposed algorithm reduces the output of the projector color differences by the value of SNR.

This paper is organized as follows: The first part (Section 2) explains the existing method of edge blending; we study the elimination of “light zone” and the match of color and brightness. In the second part of the paper (Section 3), we propose a new method of edge blending using a nonlinear function. Section 4 compares the test result of existing method with those of the proposed method to illustrate the feasibility of this algorithm. We conclude in Section 5 and present directions for further research.

2. Background and Previous Research Background and Previous Research

In this part of the paper, we present the previous research of edge blending in detail, and we present the experiments performed to study the faultiness of the previous algorithm. At the end of this section, from the results of these

(a) (b) (c)

Figure 1. The schematic diagram of seamless tiled algorithm. (a) Hard titled; (b) Soft titled; (c) Seamless title.

experiments, we present the feasibility of our proposed method.

2.1. Background

The technology of the edge blending is put a set of projector screen projection to implement edge overlap, and show a more bright, large, and high-resolution but have no-gap images which like a projector projecting a screen. When two or more combination of projection a projector screen, there will be a part of the image light overlap, the main functions of the edge blending is to make more than one projector overlap gradually reduce the brightness of the light, so that the entire increase the brightness of the screen. The current ways of edge blending have two methods: hard tiled and soft tiled (Figure 2(a)). Hard blending is a method which joints the images directly, the method easy to have a clearly crack (Figure 1(a)). soft blending is a method set a overlap zone firstly, and then try to eliminate overlap crack through the overlap function, but this method will be produce a visible light region that we referred to as optical cracks or “light zone”.

2.2. Algorithm of the Elimination of Optical Crackalgorithm of the Elimination of Optical Cracks

While the multi-projector display systems using algorithm of edge blending, the optical cracks will be eliminated, so that the images will show exactly the same to ensure the integrity of the displayed image and aesthetics. And it is more important to show map, drawing and other image information through edge blending.

The blending process of tiled image signals is the core of seamless tiled. From the preceding analysis, we can see the simple image tiled will create clearly optical cracks. To reach the seamless tiled state of Figure 1(c) which has eliminated the optical cracks, we can make each pixel value of the overlap edge of sub-images by the blending function. So that when it is superimposed in an image to another and its corresponding pixel, they will receive the expected value and brightness value of pixels. In accordance with the concept of edge blending,

Figure 2. The process of pixel of edge blending.

we use a range between 0 and 1 (normalized) function to describe the integration function (x is referred to as the position of row).

First of all, we assume the resolution of the two sample images for tiled is
$1024\times 768$ and the overlap region is
$128\times 768$ , from calculate, we can get the size of the image which after overlap is
$2\times 1024-128=1920$ , and the height is still 768. As shown in Figure 2. We make the pixel values of left overlap region of sub-image multiplied by
$f\left(x\right)$ , and the corresponds to the pixel values of left overlap region of sub-image multiplied by
$1-f\left(x\right)$ . In order to facilitate a simple calculation, we assume
$g\left(x\right)$ refer to the pixel signals which follow the changes with input values. We put the two regions overlap and put all of the pixel value in the region add others, such as Equation (1) as shown
$g\left(x\right)+\left(1-f\left(x\right)\right)g\left(x\right)=g\left(x\right)$ , the sum of the values is still equal
$g\left(x\right)$ ^{ }which figure the invariability of input images, so that we can obtain continuous and large-area but no optical cracks and no tiled mark image.

It is likely simple to result in the color distortion and the image transition effects are poor if we use such a simply linear function for the situation when the two projector driven equipments is not the same. in order to rectify this linear relationship, in this paper, first of all to improve in the equation $f\left(x\right)=kx$ ,we present an additional parameter p which use to control the exact curvature, the blending is linear for p = 1, the transition around 0.5 becomes steeper as p increases. p = 1 tends to result in a visible step at the edges of the blending region, p = 2 is used for all experiments in our paper. The following Equation (2) is the newer $f\left(x\right)$ and graphically the function looks as Figure 3.

$f\left(x\right)=\{\begin{array}{l}0.5{\left(2x\right)}^{p}\u3000\u3000\u3000\u3000\u30000\le x\le 0.5\\ 1-0.5{\left(2\left(1-x\right)\right)}^{p}\u3000\u3000\u30000.5\le x\le 1\end{array}$ (2)

Figure 3. Blend function with different values.

2.3. Algorithm of the Color/Brightness Matching

Through the above part of the handling of overlapping pixels, the edge blend is accomplished by placing a polygon with the edge blend function aboverepresented as a grey scale 1 dimensional texture. This textured polygon is blended (OpenGL speak) appropriately so it scales the pixels in the image buffer. The nice thing about this method is that it can be trivially applied to any OpenGL application since it is a post-processing stage applied after the normal geometry drawing is performed. The following figure (Figure 4) shows the mask region texture across the blend region.

From the laws of television transmission, the image brightness and image date is a square relationship, which is often said that is a Gamma relationship. so when in the processing of seamless tiled, in addition to process pixel value we also do Gamma correction related to brightness. The value of r between 1.8 and 2.2. and the Gamma correction is described the relationship between the output image’s brightness and the pixel value. If we use equation ${Y}_{\left(r,g,b\right)}=f\left(R,G,B\right)$ express Gamma function then the out brightness ${Y}_{\left(r,g,b\right)}$ is the pixel value $\left(R,G,B\right)$ (normalized to 0 and 1) raised to the power of r. and the equation is ${Y}_{\left(r,g,b\right)}={\left(R,G,B\right)}^{r}$ . Gamma correction can be seen a transfer function that in order to overcome this visual and non-linear between pixel values. Graphically the function looks as follows (Figure 5). After Gamma correction, the gray bands in overlapping region have obvious improvement.

3. Proposed Method: An Algorithm of Edge Blending Based on the Nonlinear Function

In this part of the paper, we present the algorithm of edge blending based on the non-linear function. Based on the previous research, we present use the non-linear function to enhance transition effects and we present use gamma correction to improve color/brightness matching, at the end of this section, we list the algorithm-flow in detail.

In order to facilitate the calculation, before we implement the algorithm, this paper assumes some conditions as follows [10] .

Channel Independence: This assumes that the light projected from one channel is independent of the other two. Thus, no leakage light from other channels interferes with the light projected from a channel.

Figure 4. Mask region texture across the blend region.

Figure 5. Gamma Function with different values.

Channel Constancy: This assumes that only luminance changes with changing channel inputs.

Chroma Consistency: The three colors (say, R, G, B) have the same chroma. Because the differences in brightness much more than the difference in chroma.

According the algorithm flow, we can draw flow chart as follow (Figure 6).

3.1. Algorithm of Image Title

In this paper, we use the standard image (1024 * 768) to the experimental samples (Figure 7). Taking into account the algorithm’s complexity and the effect of image blending and the simple computation of normalization, we set the size of the overlap region is 256 * 768 (account for 25% of the standard image). We can computer the size of tiled image is 1792 * 768. The actual results image as shown in Figure 8 which the overlap region has obvious optical cracks.

3.2. Algorithm of Edge Blending

The main purpose of edge blending algorithm is to eliminate the optical region which result from simply overlap. The key of the elimination of optical crack is that the selection of edge blending function. In order to take into account theimpact of brightness of the light projection to the color correction, this paper made a further modify to the Equation (2). We present the newer Equation (3) as follows where append parameter $a$ which control the luminance in the center of the blend region. $a$ ranges from 0 to 1, if it is greater than 0.5 then the

Figure 6. Flow chart of proposed method.

(a) (b) (c)

Figure 7. Image sample. (a) Left image; (b) Overlap region; (c) Right image sample.

Figure 8. Tiled image after overlap.

center of blend region will brighten, if it is less than 0.5 then it will become darker. In this paper, we assume p = 2 but different value of $a$ in Equation (3) (Figure 9), The actual results image as shown in Figure 10 which after edge blending which always have a grey band that result in luminance distortions.

$f\left(x\right)=\{\begin{array}{l}a{\left(2x\right)}^{p}\u3000\u3000\u3000\u3000\u3000\text{\hspace{0.17em}}0\le x\le 0.5\\ 1-\left(1-a\right){\left(2\left(1-x\right)\right)}^{p}\u3000\u3000\u30000.5\le x\le 1\end{array}$ (3)

3.3. Algorithm of Gamma Correction

From the analysis of the grey band in Figure 11 image, we know the reason of this is that the blending function above isn’t enough and why there is a grey band in the previous image is that the technique described so far is adding pixel values. What really needs to be done is to add brightness levels. The main function that controls how pixels are mapped to brightness is the so called gamma function of the displays, typically between 1.8 and 2.2. To facilitate of the calculation, we assume r = 2, we can see from the previous discussion ( ${Y}_{c}={Y}_{r}+{Y}_{g}+{Y}_{b}$ ) which do Gamma Correction for RGB separately. We can obtain image after the operation of Gamma correction (Figure 11).

3.4. Algorithm of Brightness/Color Matching

As we know from Figure 11, we can find the image after Gamma correction al ways exist the problem that color distortions as a result of changes in color characteristics of the projection screen and impact of the projection environment. In order to eliminate the effect of color distortion of the image, at the final step we need to adjust the color distortion, i.e. the correction method of match the output color [8] . This method can make use of output color differences between

Figure 9. Edge blending function with different value of a.

a = 0.25; a = 0.5; a = 0.75

Figure 10. Image after blending with different value of a.

multi-projectors reduced to 1.47%. We can obtain image after the brightness/ color matching as follows (Figure 12).

4. Experimental Results

In this part of the paper, we first introduce our experimental platform. And then using our proposed method apply to many filed, for example, the different image with the same value and the same image with the different value; at the end of this section, from the results of these experiments, we identify some import conclusions of edge blending:

4.1. Experimental Platform: Edge-Blending Experimental Platform: Edge-Blending

Based on the previous and proposed algorithm, we develop a software (Edge Blending v1.0) using C#. NET to monitor the algorithm of edge blending. From Figure 13, we can see the software include:

Import image samples manual.

Set Blending Region manual or automatic.

Process of Edge Blending using different method: tiled directly. Image blending using p. image correct using a. gamma correction. Brightness/color matching.

Export image after edge blending.

Figure 11. Image after Gamma correction (r = 2).

Figure 12. Image after color/brightness matching.

4.2. Experiment of Different Image with Same Value of p, a, r

In this part, we mostly discuss the different effect result from different image type but with same value of parameter ( $p=2$ , $a=0.5$ , $r=2$ ). The types of experimental image sample are pure-color image, natural image, human-culture’s image.

4.3. Experiment of Different Value of p, a, r in the Same Image

In this part, we main discuss the effect of the different value of parameter, the method of compare is that select a same image but using different value of p, a, r.

4.4. Experiment of Up-Down Edge Blending

The conventional edge blending method based on left-right edge blending, in this part, we present the edge blending method based on up-down edge blending.

4.5. Analysis of the Proposed Method

In this part, we mainly analyze the advantage of proposed method based on the value of SNR and time complexity (Tables 1-3).

Base on the three tables, we can learn that comparison of the SNR of R, G, and B as shown at Figures 17-19.

From the experimental results on the image and the value of SNR as show in Table 1, we can draw conclusions as following:

Figure 13. Edge-Blending v1.0.

Table 1. Comparison of the value of SNR and time complexity (P = 3).

Table 2. Comparison of the value of SNR and time complexity (P = 2).

Table 3. Comparison of the value of SNR and time complexity (P = 2).

The selection of tiled region considering the transitional effect as well as the time of blending, our method selects 10% of original image to be the tiled region.

The introduction of the p value can be reasonably certain extent, with the elimination of the blending region optical cracks. From Figures 14-16, we can get the max value of SNR value of RGB, so our methods select p = 2 and we also consider the effect of elimination of the optical cracks as well as the time complexity. The higher selection of p, the more complexity of time.

The introduction of the a value can adjust the gray effect caused by light coincide. Our method select a = 0.5 because when a = 0.5 we can get the max value of SNR. In practice, the brightness value can be adjusted continuously through flexible adjust the value of a.

The introduction of the r value can reduce the effect caused by Gamma, In order to facilitate computation, our method select r = 2 between 1.8 and 2.

5. Conclusions

This paper presents a algorithm of edge blending based on the non-linear function, this leads to the seamless tiled in multi-projector display system, we present a non-linear function that can be adjusted continuously, resulting in the effect of image. To the best of our knowledge, we develop software using C# to monitor the edge blending. However, we believe that this is just the first step toward achieving seamless displays. Following are some areas that need further research.

This method is applied to regular image overlap region; therefore, we are designing scalable algorithms that can apply for the anomalistic overlap region.

(a)a = 0.75a = 0.5a = 0.25(b)(c)

Figure 14. The same image-process with different value. (a) The image after simple tiled; (b) Different image with different value of a; (c) Image after Color/Brightness Matching.

(a) (b) (c)(e)

Figure 15. up-down edge blending. (a) Image sample; (b) Image after simple tiled; (c) Adjust the value of parameter; (d) Image after color matching.

Figure 16. Comparison of SNR value of B.

Figure 17. Comparison of SNR value of R.

Figure 18. Comparison of SNR value of G.

Figure 19. Comparison of SNR value of B.

This method is only quantified by the value of SNR, therefore, how to better quantify the experimental results is our currently working to research.

Acknowledgements

This work was in part supported by Guangdong provincial leading edge and key technology innovation special funds for science and technology SMEs in 2016 (No.2016A010119050).

References

[1] Majumder, A. and Stevens, R. (2002) LAM: Luminance Atenuation Map for Photometric Uniformity across a Projection Based Display. Proceedings of the ACM Symposium on Virtual Reality Software and Technology, Hong Kong, 11-13 November 2002, 147-154.

https://doi.org/10.1145/585740.585765

[2] Ramesh, R., Brown, M.S., Yang, R.G., et al. (1999) Multi-Projector Displays Using Camera-Based Registration. Proceedings of IEEE Conference on Visualization, San Francisco, CA, USA, 24-29 October 1999, 161-168.

[3] Li, K., Chen, H., Chen, Y., et al. (2000) Early Experiences and Challenges in Building and Using a Scalable Display Wall System. IEEE Computer Graphics and Applications, 20, 671-680.

https://doi.org/10.1109/38.851747

[4] Yang, R.G., Majumder, A. and Brown, M. (2005) Camera Based Calibration Techniques for Seamless Multi-Projector Displays. IEEE Transactions on Visualization and Computer Graphics, 11, 193-206.

https://doi.org/10.1109/TVCG.2005.27

[5] Weissig, D., Feldmann, I., Schussler, J., et al. (2005) A Modular High-Resolution Multi-Projection System. Proceedings of International Conference on Optical Display 2005, Berlin, Germany, 2005, 185-193.

[6] Pailthorpe, B., Bordes, N., Bleha, W.P., et al. (2001) High-Resolution Display with Uniform Illumination. Proceeding Asia Display Conference, Nagoya, Japan, 2001, 1295-1298.

[7] Moreland, J.L. and Reinsch, S. (2003) HDD: A Single Light-Source Uniform Tiled Display. San Diego Supercomputer Center (SDSC) White Paper, San Diego, CA, USA, 1-6.

[8] Wallace, G., Chen, H. and Li, K. (2003) Color Gamut Matching for Tiled Display Walls. Proceedings of the Workshop on Virtual Environments, Zurich, Switzerland, 22-23 May 2003, 293-302.

https://doi.org/10.1145/769953.769988

[9] Lv, P., Zhang, J.-F. andWang, Y.-X. (2008) A Comparative Research of Photometric Calibration Techniques in Seamless Multi-Projector Display. Journal of Image and Graphics, 13, 1272-1280.

[10] Majumder, A. and Stevens, R. (2004) Color Nonuniformity in Projection-Based Displays: Analysis and Solutions. IEEE Transactions on Visualization and Computer graphics, 10, 177-188.

https://doi.org/10.1109/TVCG.2004.1260769