Let denote a sequence of independent and identically distributed random vectors. In traditional non-parametric regression model analysis, one is in- terested in the following model
where is assumed to be a smooth, continuous but unknown function; the random errors are assumed to be normally and independently distributed with mean 0 and constant variance; and. Here, the predictor X is usually assumed to be directly observable without errors. Both the direct observation and error-free assumptions are however seldom true in most epidemiologic studies. For the violation of the error-free assumption,  considered an environmental study which studied the relation of mean exposure to lead up to age 10 (denoted as X) with intelligence quotient (IQ) among 10-year-old children (denoted as Y) living in the neighborhood of a lead smelter. Each child had one measurement made of blood lead (denoted as W), at a random time during their life. The blood lead measurement (i.e., W) became an approximate measure of mean blood lead over life (X). However, if we were able to make many replicate measurements (at different random time points), the mean would be a good indicator of lifetime exposure. In other words, the measure- ments of X are subject to errors and W is a perturbation of X. In the measurement error literature, this is known as the classical error model and Model (1) becomes
where, are mutually independent and represents the classical measurement error variable. Various methods and approaches for analyzing Model (2) such as deconvolution kernel approaches (e.g.,    ), design-adaptive local poly- nomial estimation method (e.g.,  ), methods based on simulation and extrapolation (SIMEX) arguments (e.g.,     ), and Bayesian approach (e.g.,  ) have been extensively studied in the literature.
In many studies, it is however too costly or impossible to measure the predictor X exactly or directly. Instead, a proxy W of X is measured. For the violation of the direct observation assumption,  modified the aforementioned environmental study in which the children’s place of residence at age 10 (assumed known exactly) were classified into three groups by proximity to the smelter―close, medium, far. Random blood lead samples, collected as describe in the aforementioned design, were averaged for each group (denoted as W), and this group mean used as a proxy for lifetime exposure for each child in the group. Here, the same approximate exposure (proxy) is used for all subjects in the same group, and true exposures, although unknown, may be assumed to vary randomly about the proxy. This is the well-known Berkson error model. In other words, the predictor X are not directly observable and measurements on its surrogates W are available instead. The true predictor X is then a perturbation of W. The model of interest now becomes
where, are mutually independent. Model (3) was first con- sidered by  and the estimation of the linear Berkson measurement error models was discussed in  . Methods based on least squares estimation (  ), minimum distance estimation (   ), regression calibration (  ) and trigonometric functions (  ) have been studied.
The stochastic structure of Model (3) is fundamentally different from Model (2). Here, the measurement error of Model (2) is independent of X, but dependent on W. This distinctive feature leads to completely different procedures in estimation and inference for the models. In particular, nonparametric estimators that are consistent in Model (2) are no longer valid in Model (3), and vice versa. In most of the existing literature, the measurement error is supposed to be only one of the two types. In the Berkson model (3), it is usually assumed that the observable variable W is measured with perfect accuracy. However, this may not be true in some situations. In such cases, W is observed through, where is a classical measurement error.  presented a good discussion of the origins of mixed Berkson and classical errors in the context of radiation dosimetry. Under this mixture of measurement errors, we observe a random sample of independent pairs, for, generated by
where, , and are mutually independent, and the re- spective error densities and are assumed to be known. Due to its potentially wide applications, statistical procedures for analyzing Model (4) has received more attention recently. For instance, a regression calibration approach was proposed by  and  in a parametric context of random exposure.  considered a bayesian approach for a semi-parametric regression function.  developed a nonparametric density estimation approach for contaminated data with a mixture of Berkson and classical errors but without further extending to estimate the regression function.  proposed a two-step nonparametric kernel method for estimating the regression function but its calculation is complicated. In this paper, we propose two non- parametric estimators for the regression function curve with the predictor being measured with either classical error, Berkson error, or a combination of both. The difficulty primarily depends on the relative smoothness of the error densities and. When is smooth enough (relative to), we are able to construct a nonparametric estimator that converges to the target curve at the parametric rate. For less smooth density, we propose a kernel estimator that converges at rates ranging from to rates that are close to the deconvolution rates.
This paper is organised as follows. In Section 2, we propose estimators for the regression function curve. We then derive the asymptotic normality of our estimators under some regularity conditions and give the rates of convergence in Section 3. Section 4 presents some numerical results from simulation studies. A brief discussion will be given in Section 5. All technical results and proofs are deferred to the Appendix.
2. Proposed Estimators
Let be a random sample from Models (4), and, , , and be the characteristic functions of, , , and, respectively. We have the following relationships:
Hence, if does not vanish,
Since and are assumed to be known, an estimate of can be computed as
Noticing that, if is absolutely integrable, the characteristic function and its density function have the following relation
under the condition that, the density estimator of is then given by
As a result, we propose the following estimator for
Example 1 Let the error densities and in Model (4) be normal densities with mean zero and variances and, respectively. It follows that
with. If we assume, then the
ratio is the characteristic function of another normal random variable. By (6), the estimator of can be written as
where is the density of the variable. If, the ratio is not integrable, and the estimators (5) and (6) can not be calculated. To overcome this issue, we propose an alternative approach for estimating.
Using a kernel function with a bandwidth h, we consider the following kernel estimator for
and an estimator for is then given by
where is the characteristic function of the kernel function.
Proceeding as above, we get an alternative estimator of by
Therefore, when (6) is no longer valid, we propose the following estimator for
Remark 1 To ensure that the proposed estimator (9) is well-behaved, we need to make the following assumption.
1. for all t; and
Example 2 We use the same model as in Example 1 with. In this case, to ensure (A2) to be valid, it is rather common to choose kernels that have a compactly supported characteristic function. For example, we choose the sinc kernel, which has characteristic function, the indicator function of the interval. From (8), we have
1. The above two nonparametric estimators of were given by  ;
2. When the variance of in Models (4) is equal to 0, which is the Berkson error model, the estimator (6) becomes
where is the density function of; and;
3. When the variance of in Models (4) is equal to 0, which is the classical error model, given in (9) reduces to the estimator of  .
3. Theoretical Properties
In this section, we study asymptotic properties of the estimators proposed in Section 2. In particular, the properties of the estimator at (6) are clear. It is easy to check that the numerator and the denominator are both unbiased estimators of and, respectively and that, converges at the fast parametric rate. Properties of the estimator at (9) need to further explore and, in what follows, we derive them.
3.1. Asymptotic Results for
In this section, we investigate the large-sample properties of the estimator at (9). For this purpose, we present the following regular conditions which are mild and can be found in  .
1. have zero means and uniformly bounded variances;
2., and are bounded, and and g have bounded kth derivatives;
3. is a real and symmetric kernel and has finite moment of order k. Namely, for and; and
4. The conditional moment is bounded for all u and some.
Let. The mean squared error (MSE) of the estimator is described in the next Theorem.
Theorem 1 ((MSCE)) Suppose that Conditions A and B hold. Then, for each x such that,
Explicit rates of convergence of the estimator can be found by examination of the asymptotic behaviour of the MSE. For the bias, using the Taylor expansion of the first term on the right-hand side of Equation (11), we have
The second term on the right-hand side of Equation (11) describes the variance of. The asymptotic behaviour of this term is more difficult to evaluate since it depends on the tail behaviour of the ratio, as  discussed, which can be classified into the following:
1. An exponential ratio of order is
with, , , and,.
2. A polynomial ratio of order is
3.1.1. Asymptotic Mean Squared Error (AMSE)
In this section, we study the asymptotic behaviour of the MSE where behaves like an exponential or a polynomial.
Theorem 2 Suppose that Conditions A and B hold and that the first half inequality of (12) is satisfied. Assume that is supported on. Then, for each x such that, we have
with being some positive constant and.
When is exponentially smoother than, we obtain a slower logarith- mic rate which is similar to the deconvolution rate for supersmooth error given in  . More precisely, the optimal bandwidth is of order with, and the estimator then converges at the rate of.
Theorem 3 Suppose Conditions A and B hold, and that. Then,
under the polynomial ratio (13), for each x such that, we have
with being some positive constant, and.
We obtain that, when behaves like a polynomial ratio of order in the tail, the convergence rates range from to deconvolution rate of ordinary smooth error of  . More precisely, the optimal bandwidth is of order when, and the estimator then converges at the rate of. When, the optimal bandwidth is of order and the estimator converges at the rate of.
3.1.2. Asymptotic Normality
The theorem below establishes asymptotic normality in the exponential ratio case.
Theorem 4 Under the conditions of Theorem (2), and for bandwidth with,
The next theorem establishes asymptotic normality in the polynomial ratio case.
Theorem 5 Suppose that Conditions A and B hold and that the inequality of (13) is satisfied. Assume that and. Then, under
and as, for each x such that, we have
where is the same as given in Theorem (4) and is equal to the second term on the right-hand side of Equation (11).
The proofs of all theorems are postponed to the Appendix.
3.2. Unknown Measurement Error Distribution
When the error densities are unknown, they can be readily estimated from additional observations (e.g., a sample from the error densities, replicated data or external data) and these estimates can be substituted into (6) and (9) to produce the estimate of. For sufficiently large sample size, the rates of convergence of the estimates remain unchanged when and are replaced by their consistent estimators (e.g.,    ).
4. Simulation Studies
We study numerical properties of the estimators proposed in Section 2. Note that we have defined two estimators, at (6) and (9). The first exists when is inte- grable, and the estimator (9) otherwise. We use the notations and for the esti- mators (6) and (9) respectively. We use the notation for the estimator that ignores the errors, that is, the estimator is the classical Nadaraya-Watson estimator of based on direct data from,. Note that is exactly equal to when. In addition, we use for the estimator of  .
We apply the various estimators introduced above to some simulated examples (see,  ):
2. (sharp unimodal), and
where is the density of an variable. For each of the above regression functions, we generate 200 data sets of randomly sampled vectors, as follows. We generate a random sample from, a random sample from and a random sample from, and put and, , where is the density of an variable, and we take and to be either normal or Laplace with zero mean. Then we generate a random sample as, where the errors are normally distributed with zero mean and variance, where with denoting the mean-squared deviation of g from its average value. We simply denoted and by, and other similar.
In our simulations we consider sample sizes, and in each case we generate 200 samples from the distribution of the random vector. Except if stated otherwise, we adopt the second order kernel K corresponding to , which is necessary to calculate and. For the band- width h, it is necessary to calculate, and, we select the value h that mini- mises the cross-validation (CV) criterion, , where the sub- script meant that the estimator was constructed without using the jth observation. We report the Integrated Squared Error, , where is the estimator considered. In all graphs, to illustrate the performance of an estimator, we show the estimated curves corresponding to the first (Q1), second (Q2) and third (Q3) quartiles of the ordered ISEs. The target curve is always represented by a solid curve. In the tables we provide the average values, denoted by MISE, of the 200 cal- culated ISEs.
Figure 1 and Table 1 illustrate the way in which the estimator improves as sample size increases. We compare, for various sample sizes, the results obtained for estimating curve (a) when, and with the pair of variance ratios equals (0.1, 0.4), and for estimating curve (b) when and ~ (N, L), (N, N), (L, L) or (L, N) with . We see clearly that, as the sample size increases, the quality of the estimators improves significantly in all cases.
For any nonparametric method for regression problem, the quality of the estimator also depends on the discrepancy of the observed sample. That is, for any given family of densities, and, and any given the noise-to-signal ratios, the performance of the estimator depends on the variances of, and. Here, we compare the results obtained from estimating curve (c) for different values of
Figure 1. Estimation of curve (a) for samples of size n = 50 (left panel), n = 100 (middle panel) or n = 250 (right panel), when, and with The solid curve is the target curve.
Table 1. MISE for estimation of curve (b) when, and ~ (N, L), (N, N), (L, L) or (L, N) with.
. As expected, Figure 2 shows that the best performance usually occur for smaller error variance (e.g.,). It is noteworthy that the effect of the variances on the estimator performance is obvious in model (4).
Finally, we compare (or), and. Figure 3 shows the boxplots of the quantities of and for estimating curve (a) when and, where is the ISE of our proposed estimator, is the ISE of the estimator that ignores the errors, and is the ISE of the estimator of  . Here, each boxplot is constructed from 200 samples. Here, in panel (a)-(L-L) (or (a)-(N-N)), the mixed errors are both Laplace (or both normal). Here, and also in panel (a)-(N-L) (or (a)-(L-N)), the errors are and (or and). In each panel, for X-axis = 1 to 7, = (0.1, 0.4), (0.1, 0.3), (0.2, 0.3), (0.2, 0.2), (0.3, 0.2), (0.3, 0.1) or (0.4, 0.1). The more a boxplot is located below the zero horizontal line, the better our method compared with the other two estimators. In the same situation, Table 2 and Table 3 report the average integrated square error (MISE) for estimating curves (b) and (c) respectively. As expected, our proposed estimator substantially outperformed the estimator that completely ignores any measurement errors. Our results show that our proposed estimator usually works better than the estimator proposed by  for estimating curves (a) and (b). It is noteworthy that the estimator proposed by  may perform better than our proposed estimator when curve (c) with is esti- mated.
In this paper, we propose a new method for estimating non-parametric regression models with the predictors being measured with a mixture of Berkson and classical errors. The method is based on the relative smoothness of and. When is
Figure 2. Estimation of function (c) for samples of size, when, and with being (0.5,0.05,0.15), (1,0.1,0.3), and (2,0.15,0.45) (from left to right). The solid curve is the target curve.
Figure 3. Boxplots of the quantities of log(ISEO/ISEI) (row 1) and log(ISEO/ISEC) (row 2) for estimating regression curve (a) when and, for various error densities and and various values of.
Table 2. MISE for estimation of curve (b) when and n = 250, for various error densities and and various values of.
Table 3. MISE for estimation of curve (c) when and n = 250, for various error densities and and various values of.
smooth enough (relative to), we propose a nonparametric estimator (6) that converges to the target curve at the parametric rate. For less smooth function, we propose a kernel estimator (9) that converges at rates ranging from to rates that are close to the deconvolution rates. Numerical results show that the new esti- mators are promising in terms of correcting the bias arising from the errors-in- variables. It generally preforms better than the approach proposed by  . The metho- dology can be readily extended to the prediction problem of nonparametric errors-in- variables regression (see, e.g.,  ). Extension of our method to the problems con- sidered in  is of future research interest.
This work was supported by Natural Science Foundation of Jiangxi Province of China under grant number 20142BAB211018.
Proof of Theorem 1
, we have
where. The result follows immediately from 14 and 15.
Proofs of the Results of Section 3.1.1.
Lemma 1 Suppose that is supported on, and for all t. Then, for
, we have
where, here, and below, C denotes a generic positive and finite constant.
Proof. It follows from (A2) of Condition A that for some large enough constant. Since, we have
The conclusion follows from
The proof for the other result is similar and requires Parseval's Theorem.
From (14) and Lemma 1, we have
The proof of Theorem 2 follows from the expressions of and.
The proof of Theorem 3 is the same as the proof of Theorem 2, but in this case we need the following lemma.
Lemma 2 Suppose that for all t, and
. Then, we have
The proof of Lemma 2 is similar to the proof of Lemma 1 and is omitted.
Proofs of the Results of Section 3.1.2.
A standard decomposition gives, goes in pro- bability to and thus we only need to prove the asymptotic normality for. As given in  , a sufficient condition for the following asymptotical normality
is that the Lyapounov's condition holds, i.e., for some,
Letting, we have
Under the conditions given in the theorem 4, we can prove that
Under the conditions given in the theorem 5, we can prove that
The rest is standard and is omitted.
 Armstrong, B.G. (1998) Effect of Measurement Error on Epidemiological Studies of Environmental and Occupational Exposures. Occupational and Environmental Medicine, 55, 651-656.
 Delaigle, A. and Meister, A. (2007) Nonparametric Regression Estimation in the Heteroscedastic Errors-in-Variables Problem. Journal of the American Statistical Association, 102, 1416-1426.
 Delaigle, A., Fan, J. and Carroll, R.J. (2009) A Design-Adaptive Local Polynomial Estimator for the Errors-in-Variables Problem. Journal of the American Statistical Association 104, 348-359.
 Cook, J.R. and Stefanski, L.A. (1994) Simulation-Extrapolation Estimation in Parametric Measurement Error Models. Journal of the American Statistical Association, 89, 1314-1328.
 Stefanski, L.A. and Cook, J.R. (1995) Simulation-Extrapolation: The Measurement Error Jackknife. Journal of the American Statistical Association, 90, 1247-1256.
 Staudenmayer, J. and Ruppert, D. (2004) Local Polynomial Regression and Simulation-Extrapolation. Journal of the Royal Statistical Society: Series B, 66, 17-30.
 Berry, S.M., Carroll, R.J. and Ruppert, D. (2002) Bayesian Smoothing and Regression Splines for Measurement Error Problems. Journal of the American Statistical Association, 97, 160-169.
 Carroll, R.J., Delaigle, A. and Hall, P. (2009) Nonparametric Prediction in Measurement Error Models. ournal of the American Statistical Association, 104, 993-1003.
 Delaigle, A., Hall, P. and Qiu, P. (2006) Nonparametric Methods for Solving the Berkson Errors-in-Variables Problem. Journal of the Royal Statistical Society: Series B, 68, 201-220.
 Reeves, G.K., Cox, D.R., Darby, S.C. and Whitley, E. (1998) Some Aspects of Measurement Error in Explanatory Variables for Continuous and Binary Regression Models. Statistics in Medicine, 17, 2157-2177.
 Schafer, M., Mullhaupt, B. and Clavien, P.A. (2002) Evidence-Based Pancreatic Head Resection for Pancreatic Cancer and Chronic Pancreatitis. Annals of Surgery, 236, 137-148.
 Mallick, B., Hoffman, F.O. and Carroll, R.J. (2002) Semiparametric Regression Modeling with Mixtures of Berkson and Classical Error, with Application to Fallout from the Nevada Test Site. Biometrics, 58, 13-20.
 Carroll, R.J., Delaigle, A. and Hall, P. (2007) Nonparametric Regression Estimation from Data Contaminated by a Mixture of Berkson and Classical Errors. Journal of the Royal Statistical Society: Series B, 69, 859-878.
 Hu, Y. and Schennach, S.M. (2008) Identification and Estimation of Nonclassical Nonlinear Errors-in-Variables Models with Continuous Distributions. Econometrica, 76, 195-216.