Transmission of digital images has been widely studied, since the early years of the Internet . It deals with compression and transmission of images data in such a way that the receiver can start decoding and displaying the received images even without receiving the whole file. Because of the large amounts of data needed in image technology, applications are highly constrained by the available resources, and the quality of service during the transmission. In video streaming, for instance, the latency of the transmission of individual image frames plays a fundamental role   because of the isochronal character of the video. The images must be displayed at a given frequency, with a fault threshold above which the visual quality of the video is not acceptable. Many techniques have been proposed in the literature to tackle these problems, among which image compression and Progressive Image Transmission (PIT).
The primary objective of PIT is to transmit a significant and interpretable core of the image and subsequently transmit complements layers in order to gradually improve the quality. This method requires a preparation of the image to be transmitted. PIT techniques can be grouped into three main areas: the spatial domain , the methods based on transform domain, and the pyramid-structured domain  . As new areas of interest emerge, like live streaming over narrow networks, wireless sensor networks, digital image transmissions are still of a significant challenge. As reviewed in , most of the recent improvements in image coding are based on wavelet transformation. The challenge is then to organize the transmission of the bitstream to adapt to the fluctuations of the network and the receiving device capabilities.
In this work, we are interested in the progressive transmission and refinement of still images, as a process that adapts to low quality network service. A special focus is made on JPEG2000 format since it is the most used standard nowadays  . However, the method presented here is general enough and can be applied to any image format that encodes the image file as a two-dimension data container (the resolution and the color depth), and where the resolution and the color depth can be picked independently to adapt to the end user device. The display of an image is considered as a progressive process in order to adapt to network conditions. The sender selects the image data, layer by layer, from most significant to the least one, depending on the quality of the desired image at the receiver. Upon reception of these data, the receiver decodes a blurred version of the image and smooths it by statistically inferring the missing information. As more refinement data come from the network, this process is recursively repeated. Kalman filtering (KF) algorithm will be used to infer the refinement data   .
The rest of the paper is organized as follows. In Section 2, we review the literature works about progressive image transmission. Section 3 deals with the theoretical foundations on the discrete Kalman Filtering, while Section 4 presents the proposed method modeling image transmission as a filtering procedure. In Section 5, we apply the FRM-KF on a standard image gallery and discuss its performance.
2. Short Background on PIT
PIT techniques can be grouped into three main areas namely the spatial domain, the methods based on transformed domain and the pyramid structured domain.
2.1. Spatial Domain
Spatial domain methods are based on the bit-plane decomposition (BPDM)  and the vector quantization method (VQM).
The bit plane decomposition method is the most intuitive one when tackling the problem of progressive transmission. Indeed, the level of gray of each pixel in an image is coded over 8 bits having different significances. The collection of the ith significant bits of all pixels constitutes the ith bit plane to be transmitted at the ith step. On the receiver side, the binary image will be rebuilt after receiving a certain number of bit planes, and gradually refined with the arrival of the other planes. BPDM does not introduce any distortions but it suffers from a lack of flexibility and limited performance in terms of adaptation to variations in network conditions. Improvements of BPDM in terms of reduction of storage space and therefore transmission time are available in the literature: quantification of pixels and selection of areas of interest with higher priorities .
During the vector quantization the pixels are grouped in blocks (code-blocks) which are transformed each into a vector. The obtained vectors are grouped into a lighter structure called code-book where they are codewords. Codewords are progressively transmitted and used to produce an approximative image on the receiver side. The main available improvement of the VQM is the Tree-sourced Vector Quantization method (TSVQM) which consists to transmit first the vector quantizations contributing more quickly to obtain a better image quality . The VQM has some disadvantages: block effects during the display, transmission by codeword and overout of the transmitter side complexity, calculation overhead for the creation, organization and codewords selection.
2.2. Transform Domain
The main goal of transform based methods (e.g. Discrete Cosines Transform (DCT)) is to achieve the concentration of energy in low-frequency areas which are grouped into a small number of coefficients. The low frequency coefficients have a strong and decisive impact on the final and overall quality of the image. Before their transmission in the decreasing order of importance, those coefficients are hierarchized following a technical scanning pattern (e.g. the zigzag scan used in JPEG  ) or the multistage quantization based on variances of coefficients.
2.3. Pyramid Structured Domain
The pyramidal shape is ideal for progressive transmission. To form a pyramid, an image is reduced in terms of resolution according to a predefined method such as Discrete Wavelet Decomposition (DWT)   and the Quadrature Mirror Filter (QMF) . The reduced image has few coefficients and the transmission process therefore consists of transmitting first the top of the pyramid followed by the differences between the current layer and the next layer.
2.4. Fast Progressive Image Transmission
All the above techniques do not integrate a prediction on the data not yet received. Such an inference allows a faster access to a transmitted image. A method trying to achieve that goal is the pixel interpolation permitting to estimate not yet received data using a model constructed based on available data. The SIDE-MATCH algorithm is an implementation of interpolation method . Although the expected rapidity to converge to a relatively good image quality, inference methods suffer of a certain number of drawbacks, namely the difficulty of producing good quality images at the beginning of the process and their complexity inducing large calculation times.
3. The Discrete Kalman Filter
Filtering is a procedure which aims at estimating the state of a given dynamic system with noisy observations. Usually, the outputs are given as a sequence: . denotes a set of time values. It can be discrete or continuous, depending on data availability and observation rate. Each output is related to an unknown or partially known state through a stochastic model of the form
where is the noise occurring in the measurement procedure. represents and averaged relationship between and . In other terms it is a trend of evolution of Y as a function of X The observation noise is usually assumed to be a normal or Gaussian random variable . The additional hypothesis of independence of system is very common and useful for computations. In Equation (1), the observation is available while neither nor is known. Filtering aims at giving the conditional law of the hidden signal conditionally upon the subsequence despite the presence of noise . One can focus on , the conditional expectation of the signal upon subsequence noted as for simplicity   . As mentioned in   , applications of filtering cover areas such as sensorless control, prognostics and health management (PHM), fault-tolerant control of ac drives, management of storage systems, signal processing, robotics, computer vision, real-time industrial control systems, localization, navigation, mobile trajectory tracking and other applications combining knowledge of a priori dynamics with sensors measurements.
A large class of these applications is covered by the discrete filtering that can be described by the general linear problem
where , , and are matrices expressing the dynamics of the signal and the observation. The filtering problem (2) has an explicit solution in the Gaussian linear case known as the “discrete Kalman Filter” which is presented as follows. Let
and , given by the law of and . The filtering equations are given as
Since , one has . On the other hand, and if is chosen as being constant (0 for example), then its variance will be null and . The techniques developed in the linear filtering can sometimes be extended to the nonlinear case by the mean of linearization methods . However, there are more general results that can be applied in nonlinear cases such as particle filtering.
4. Presentation of the FRM-KF
4.1. Model Statement
We consider the progressive transmission of a JPEG2000 image, encoded in bitplane. We assume the transmission is done bitplane by bitplane, over a narrow network channel. Because of the poor network quality, the receiver cannot wait until all the data are transmitted before decoding and displaying the image. Moreover, the transmission can unpredictably stop at any time. Thus, the receiver has to use the data received so far to estimate as better as possible the whole image. A first approach consists in simply refreshing the estimated image with newly received layers and in displaying the result when its quality reaches a given threshold. Instead, we learn from successive bitplanes or layers, considered as partial observations of the image, to infer the missing parts. Hence, the bitplane transmission can be viewed as a dynamic system with partial observations. Since image structures are variables, we can use a representative sample of coefficients for statistical inference.
Let S be such an image, and the sequence of bitplanes extracted from S. Transmitting S consists in transmitting the layers . We call the part of S yet to be transmitted, after the sequence , has been transmitted: that is the “residual” . For convenience, we also say , and .
Deterministically, we can then write
However, following our purpose of inference, a stochastic description is needed here. Hence, from the receiver’s viewpoint, the following model that recalls the problem (2) can be considered:
with , , , where , , and .
Equation (7) describes the dynamics of the remaining information to be received while Equation (8) gives the next layer to be received. Indeed, we make the hypothesis of an arithmitico-geometric progression of the part of the image that remains to be sent ( ). On the same manner, we assume an affine relationship between the current layer to be sent ( ) and the current part of the image that remains to be sent ( ). The choice of an affine model is as simple as natural for a first modeling that will prove otherwise reasonable. Notice that and that by formulation of the problem, follows the uniform law . Indeed, except that they belong to , we do not have any prior information on coefficients, and completed information is given by
Equation (7) underlines an exponential variation of estimation errors both with their variances. The filtering procedure will consist in determining the mathematical expectation of conditionally upon , at the step . The choice of the upper bound of n = 7 is motivated by the fact that we process images by channels. And for a real color image, we have red, green and blue channels each coded on 8 bits (numbered from 0 to 7). The estimation of S is given by
Proposition 1. If then
Moreover, if and , then
Proof. Let . One has
Hence, and if then and . On the other hand, and
For the second part of Proposition 1,
Hence, if additionally et , then and .
Proposition 1 illustrates the fact that in the long run ( ), the remaining information about the image is predictable and tends to . On the other hand, the layers to be received tend to zero in the long run. That is realistic since only a finite number of layers are sufficient. Following the same principle, the remaining information shall be null in the long run. Thus, we should have . We adopt it later in the work.
The use of the Kalman filter also gives us the benefit of its memoryless characteristic: it only retains the previous state to infer the current one. So it is not necessary to keep track of all the previously computed states in memory for the prediction method.
4.2. Calibration and Validation of the Model
The dynamics of the conditional distribution law (characterized by its mean vector and its variance-covariance matrix) is stirred by the filtering equations. In order to determine coefficients , , a, b and c, we proceed by statistic regressions on the sample corresponding to all possible values of a block of pixels. Regression aims at identifying the best set of parameters which minimize the sum square error (SSE) of the best fitting model. Precisely, we shall determine and which minimize the quantity
Since we adopted , it remains to find in such a way that (18) is minimal. After and have been identified, one can obtain consecutively a, b and c by minimizing the following SSEs:
In (18), (19) and (20), we have for ,
Following the aforementioned regressions, we obtained Table 1.
Note that all the parameters satisfy the hypotheses of Proposition 1 and therefore guarantee the exponential convergence of the filter.
5. Experimental Evaluation of FRM-KF
This section aims at applying the filtering procedure we described above to a sample of 210 images coming from the University of Southern California-Signal
Table 1. Estimates of parameters.
and Image Processing Institute USC-SIPI1 database. We used a computer workstation with the following characteristics: RAM: 4 GB, Processor: 4xIntel (c) CoreTM i3-3227U CPU @1.90 GHz of 32 bits on a Ubuntu operating system 18.04 (Linux kernel: 4.15.0-74-generic).
According to Section 3 we have , , , , , , . Let us recall here that for a Uniform law the variance is given by . Figure 1 illustrates the
evolution of the visual rendering of images following the quality layers reception and the filtering procedure.
Compared to the results in , the visual rendering they obtained at their fifth step is obtained here at the 3rd step (Figure 1(d)), corresponding to a good visual quality for human perception. The Peak Signal to Noise Ratio (PSNR) was measured for the successive estimated images based on received layers. We compared our PSNRs to those of reference methods: the Set Partitioning In Hierarchical Tree (SPIHT) method, the method of Tzu-Chuen and the method of Tung .
A regression analysis showed for all considered methods that there is an affine relation between the number of received layers and the measured PSNR (at least 93% for the adjusted R-squared) with high significant2 slope and intercept.
Figure 1. Visual rendering of the model on Lenna (256 × 256). (from (b)-(h)). In (a), there is the original Lenna (256 × 256) compressed images using the OpenJPEG library (version 2.3.0-October 2017), with 6 levels of resolution and 8 levels of quality (80%, 70%, 60%, 50%, 40%, 30%, 20% and 1%, corresponding to the original image being of high quality).
Table 2 gives the regression coefficients of each method for 256 * 256 and 512 * 512 resolutions of Lenna image studied in .
Table 3 gives the difference of regression coefficients of each considered method with respect to FRM-KF. The intercept shows that Tzu-Chueng Lu method has the better initial PSNR while the FRM-KF has the worst probably because its initial estimation is drawn uniformly randomly. Fortunately, the FRM-KF has the best slope which is about 1.63 times better than the second higher slope displayed by the SPIHT method. Hence, with a few number of images (from 3) FRM-KF presents the best performance compared to other methods.
The database coming from the USC-SIPI contains 73 images having a 256 * 256 resolution, 83 images having a 512 * 512 resolution, 53 images having a 1024 * 1024 resolution and only 1 image having a 2050 * 2050 resolution. For our statistical analyses we then focused on 256 * 256, 512 * 512 and 1024 * 1024 resolutions. Again we found an affine relation between the PSNR and the number of received layers. The P-value was less than 2 × 10−16 and the adjusted R2 (model fitting factor) was between 88.77% and 96.96%. In order to give a general behavior of the FRM-KF method, the computed values of the slope and the intercept are given in Table 4.
Table 2. PSNR as function of the number of received layers.
Table 3. Difference of regression coefficients of each method with respect to FRM-KF.
Table 4. Regression coefficients of the PSNR as linear function of received layers using FRM-KF method.
We evaluated the time needed to decode the images. The first phase consisting to generate white noise, to decode the first quality layer of the original image, and to combine the both took about 2.24 × 10−1 ± 2.868 × 10−2, 8.87 × 10−1 ± 5.612 × 10−2 and 3.505 ± 1.587 × 10−1 (in terms of average ± standard deviation) seconds respectively for 256 * 256, 512 * 512 and 1024 * 1024 resolution images. The necessary times to decode each other quality layer and to combine it with previous result, were given by 3.149 × 10−2 ± 4.012 × 10−3, 1.223 × 10−1 ± 1.024 × 10−2, 4.95 × 10−1 ± 4.009 × 10−2 seconds respectively for 256 * 256, 512 * 512 and 1024 * 1024 resolution images. The images used on current mobile devices have a resolution of at least 512 * 512. With regard to the time corresponding to the processing of the 1024 * 1024 resolution image, we recommend the FRM-KF method to resolutions less or equal to 512 * 512.
Focusing on the amount of data transmitted during a streaming of images for each quality layer, we notice that less than 10% of image data is needed to obtain up to the sixth-quality image. So, the process is suitable in terms of processing and memory resources for small devices with low computing capabilities.
This work addressed the problem of image transmission in limited environment. We were interested in the progressive transmission and refinement of still images, as a process that adapts to low quality network service. In order to achieve our objectives, we proposed a stochastic model which presents the missing parts of the image as noise effects. In a stochastic context, the problem of estimating dynamically a signal conditionally upon available observations is known as filtering. Thus, we tried successfully to calibrate a Kalman filter model using statistical regression and some general considerations. The output model we got was precisely a discrete Kalman filter.
Applying the filtering procedure on a dataset of 209 images we got satisfactory results. Indeed, we evaluated the evolution of Peak Signal to Noise Ratio (PSNR) with respect to the number of received layers. An affine relation was found independently on the PIT method we considered (Set Partitioning In Hierarchical Tree, Tzu-Chuen, Tung and FRM-KF methods). The FRM-KF approach we proposed appeared to be one which improves the PSNR faster.
The performance of FRM-KF method has been further evaluated in terms of the ratios in the quality of data image/size sent and in the quality of image/time required for treatment. A high quality was reached faster with relatively small data (less than 10% of image data is needed to obtain up to the sixth-quality image). The time for treatment also decreases faster with number of received layers. However, we found that the time of image treatment might be large starting from a image resolution of 1024 * 1024. Hence, we recommend FRM-KF method for resolutions less or equal to 512 * 512.
In future works, we are expected to extend our method in multimedia communication environments, subject to disturbances, in order to ensure robustness to breakdowns and interference. We should also consider adapting our approach to video streaming in order to ensure a greater continuity of video streaming service content.
2Significance codes under R software in terms of P-value: 0 “***” 0.001 “**” 0.01 “*” 0.05 “•” 0.1 “ ” 1.
 Kiely, A.B. (1996) Progressive Transmission and Compression Images. The Telecommunications and Data Acquisition Progress Report 42-124, Jet Propulsion Laboratory, Pasadena, California, October-December 1995, 88-103.
 Chen, C., Zhu, X., de Veciana, G., Bovik, A.C. and Heath, R.W. (2015) Rate Adaptation and Admission Control for Video Transmission with Subjective Quality Constraints. IEEE Journal of Selected Topics in Signal Processing, 9, 22-36.
 Servetto, S.D. and Vetterli, M. (2000) High-Bandwidth Internet Video Telephony. The 10th International Packet Video Workshop, Forte Village Resort, Cagliari, 1-2 May 2000.
 Boujelbene, R., Jemaa, Y.B. and Zribi, M. (2019) A Comparative Study of Recent Improvements in Wavelet-Based Image Coding Schemes. Multimedia Tools and Applications, 78, 1649-1683.
 Lu, T.-C. and Chang, C.-C. (2007) A Progressive Image Transmission Technique Using Haar Wavelet Transformation. International Journal of Innovative Computing, Information and Control, 3, 1449-1461.
 Christopoulos, C., Skodras, A. and Ebrahimi, T. (2000) The JPEG2000 Still Image Coding System: An Overview. IEEE Transactions on Consumer Electronics, 46, 1103-1127.
 Jiang, J.-H., Chang, C.-C. and Chen, T.-S. (1997) Selective Progressive Image Transmission Using Diagonal Sampling Technique. Proceedings of International Symposium on Digital Media Information Base, Nara, 26-28 November 1997, 59-67.
 Ambadekar, S.P., Jain, J. and Khanapuri, J. (2019) Digital Image Watermarking through Encryption and DWT for Copyright Protection. In: Bhattacharyya, S., Mukherjee, A., Bhaumik, H., Das, S. and Yoshida, K., Recent Trends in Signal and Image Processing, Springer, Singapore, 187-195.
 Chen, T.-S. and Chang, C.-C. (1997) Progressive Image Transmission Using Side Match Method. IPSJ International Symposium on Information Systems and Technologies for Network Society, Japan, 24-26 September 1997, 191-198.
 Auger, F., Hilairet, M., Guerrero, J.M., Monmasson, E., Orlowska-Kowalska, T. and Katsura, S. (2013) Industrial Applications of the Kalman Filter: A Review. IEEE Transactions on Industrial Electronics, 60, 5458-5471.
 Carraro, C. (1989) A Few problems with Application of the Kalman Filter. In: Decarli, A., Francis, B.J., Gilchrist, R., Seeber, G.U.H., Eds., Statistical Modelling, Vol. 57, Springer, New York, 75-83.