Estimation of Parameters
The IMT is a test for general misspecification, produced by  who pointed out that the properties of the Maximum likelihood estimator and the information matrix can be exploited to yield a family of useful tests for model mis-specification. The idea of the IMT is to compare two different estimators of the information matrix to assess model fit. The IMT is based on the information matrix equality that obtains when the model specification is correct. This equality implies the asymptotic equivalence of the Hessian and the score forms of Fisher’s information matrix . As , points out, the IMT is designed to detect the failure of this equality and the failure implies the model misspecification.  discussed the information matrix test and showed that it is useful with binary data models. Many researchers,   and  pointed out the behaviour of the asymptotic distribution of IMT statistic and dispersion matrix. The idea of the
information matrix test is to compare and , as these
differ when the model is mis-specified but not when the model is correct. , pointed out, can be estimated the covariance matrix of IMT, dependent upon the IMT of , which can be estimated without the computation of analytic third derivatives of the density function. , discussed that, the IMT is sensitive to non-normality. Moreover, he proposed a simple computation procedure which employs the Outer Product of the Gradient (OPG) covariance matrix estimator of IMT statistic. However,  argue that, such a procedure maybe give unreliable inferences, related to the stochastic nature of the covariance matrix estimator which uses high sample moments to estimate high population moments.  purposed a simple calculation procedure for the test statistic, for general binary data models, which employs the ML covariance matrix estimator instead the OPG estimator. Moreover, , computed and examined IMT and found it had good power for logistic model.
Basic Idea of the IMT
Let us consider the density function for individual observation and the data are independent, identically distribution so we have
and we consider to be the logarithm of a density function of x dependent upon p parameters , so the log-likelihood function in this case is
Now, as we defined the idea of the IMT to compare two different matrix of expected the first and second partial derivatives of the , we have
So, according to the ML method, we have
Differentiating (1) again we get
When the model is mis-specified, the above quantity will be not necessarily equal zero.
Asymptotic Distribution of
The asymptotic distribution of estimated parameters and the behaviour of the MLE under the wrong model discussed by  and more investigated considered by . , pointed out the estimation the parameters of a given regression model. In the limit for each value of the parameter vector ,
where denoted to the true model and is the fitted model. Also, consider the Kullback-Leibler divergence (KL) from the true to the approximating model conditional on X, under the wrong model. In this case , where is the least false value (LF). Note that the least false value minimizes the KL divergence, because the derivative of the KL is
Also, if we need define
these matrixes are identical when for all Y. As explained in , the distribution of the , in this case from the central limit theorem there is convergence in distribution
where, , which is leads to
So, we can say, the asymptotic MLE distribution under the null hypotheses H0, in this case
where, is the true value. And the asymptotic distribution of under alternative hypotheses H1 is
So, that is meaning ( ) if and only if when fitted the correct model (i.e. under H0).
2. The IMT under Missing Covariates for Logistic Regression Model
In this part, we apply the procedure of the IMT statistic under missing covariates for a logistic regression model. If is a p-dimensional vector of covariates draw from normal distribution and is binary with
In the following we treat the simple case where the fitted model is
for a scalar and that the true model has
where is also a scalar. We have the log-likelihood function contribution for the ith element is
and note that we only consider fitting the model with , even if the true model also includes (i.e. ). From this we get:
as is , and so we get that
3. An Alternative Formulae of Variance
In this part we are interested to find a formulae of the variance of d statistic, even when the model is mis-specified. To perform the IMT we need to find the mean and variance of
Under H0 , and so the IMT could be written as
which will have a -distribution on d.f. as T is asymptotically Normal. However, the test statistic has to be evaluated at the MLE and this introduces a complication. The MLE is the solution to
The expression for T is
and this is clearly going to be highly correlation with S. Therefore, the appropriate variance for the IMT is . As T and S are sums of independent elements, the Central limit Theorem implies that is asymptotically Normal and so we can use
To work out , so, in this case we can write
3.1. The Variance of IMT under Missing Covariates for Logistic Regression Model
We now need to find expressions for , and
We already have that
so, the variance is
and we have
taking expectation we obtain
Now we need to compute . In fact , not only if the model is correct but also when evaluated at the least false value (under wrong model), so in this case
and we have
Now we will work out , as before, since , so
and note that
where, is under the true model. So,
Hence, the required variance (9)
and we have expressions for each component from (12), (13), (14) and (15) We need to evaluate these components by simulation.
3.2. The Dispersion Matrix under Wrong Model
In fact, may be some elements of the covariance matrix of the IMT are linear combinations of others leading to singularity of the estimated covariance matrix, this point discussed by  and . We are interested to compute the , even when the wrong model has been fitted. We will compute each of the components of this variance separately. We see from Section 3.1 that we need to evaluate, e.g.
This cannot be done analytically so we simulate 5000 values of X and replace the by the mean of these 5000 values. In evaluating we use the values of the parameters , and . What do we use for ? We need to evaluate at the least false values and for and . So, e.g, the first element of is found by simulation from
and X draw from bivariate normal distribution with , and . The formulae of the least false values and has been discussed and calculated by .
4. Empirical Variance of IMT
The expression in (16) is the variance V of d at but we need an estimate, . If we have a sample how can we estimate V consistently? One candidate would be to compute
where, is the fitted value from the model with just . Now compute
as an estimate of V, we will assess this by simulation.
5. Simulation Study
This simulation examines the correctness of the form of the dispersion matrix V in (16) and (19). To achieve the aim of this simulation, we will consider a logistic regression model which has two covariates draw from bivariate normal distribution with mean zero and covariance matrix as:
and the fitted model is
・ Apply in two cases of logistic model,
・ The fitted is the true logistic model (i.e. )
・ The fitted model is mis-specified (i.e. ).
・ Use variance ( ) and correlation .
・ We choose some different components of parameters , and to calculate .
・ We compute the least false values and by formulae to calculate .
・ We compute the true variance by simulating and take the variance to be .
・ We compute the theoretical variance at the least false value and calculate and as described in section 3.2.
・ Finally, for each simulation we compute the empirical variance and take the mean over the simulations.
・ We make comparison between the diagonal elements of dispersion matrix vs. respectively.
・ Apply on different sample size and number of simulations.
6. Results and Discussion
The results were reported in tables, which show the diagonal elements of the variance matrix: denotes the empirical variance, denotes the theoretical variance and denotes the true variance. The true parameters appear as , , and ; and denote to the rank of the covariance matrix
empirical and theoretical respectively. The Ratio and are , respectively. denotes the standard deviation over a sample
where is the true model. In our simulation we consider two covariates, so in this case the dispersion matrix of d is a 3 × 3 dimensional matrix.
Firstly, we consider the results under true logistic model, Table 1, shows the results of simulation, which appeared the diagonal elements of matrix V, the empirical version and theoretical form comparing with true variance, which use in case of by sample size . Table 2, reported the results by sample size , with equal variance . We can see clearly, that all diagonal elements appeared small in value in two different cases of sample size. The first element was much closer to zero than of the rest. In almost cases the results appeared reasonable ratio which is meaning the theoretical variance and empirical variance are close to the true value. There are some slightly strange ratio almost in case of sample size , the reason may be affected by small value of standard deviation of , otherwise the ratio is close to one. In case of sample size , the behaviour of results shows almost the same pattern, with the ratio close to one and that is meaning the formulae of the variance works well. In a few cases with small values of which affected on the ratio where the first two elements were more sensitive. Overall, we have reasonable results to say that, the alternative formulae of variance works well and the two first elements still more sensitive which appeared tend to zero.
Secondly, we consider the results when the missing covariate logistic model has been fitted. That is meaning when the variance of IMT computed under H1 and uses the least false values. Table 3, shows the results of sample size . Table 4, shows the results of sample size 1000. In general, the behaviour of ratio
Table 1. Simulation results of the variance ( ), ( ) and ( ) in case of fitted true model, with sample size and .
Table 2. Simulation results of the variance ( ), ( ) and ( ) in case of fitted true model, with sample size and .
Table 3. Simulation results of the variance ( ), ( ) and ( ) in case of fitted missing covariates model, with sample size and .
Table 4. Simulation results of the variance ( ), ( ) and ( ) in case of fitted missing covariates model, with sample size and .
appeared the same behaviour which found in case of , the two cases of different sample size appeared reasonable ratio which is close to one. A few cases shows low ratio, the reason is as discussed before concerning to the small value of .
This paper carried out to investigate the behaviour of IMT and compute the covariance matrix under the wrong logistic regression model. As result, we can see that the alternative formula of the variance appeared reasonable results under the true and missing covariate model. As we computed the final form of the variance of IMT, we can see clearly it is dependent on . As we know, we made some notes on the first two elements of , which may be quite close to zero under true model and use the least false value, the related to the log likelihood functions. So, these elements leading to singularity of the estimated covariance matrix, and have effect on the behaviour of the dispersion matrix of the IMT.
I am very grateful to Professor J. N. S. Matthews, School of Mathematics and Statistics, Newcastle University for academic supporting and Dr. Hamza M. A. Boauod, (FCOPHTH) (SA), Consultant Ophthalmologist, Eye Department, Klerksdorp Hospital. South Africa for his financial support. Also thank the referees, associate editor and joint editor for their helpful comments and additional references.
 Matthews, J.N.S. and Badi, N.H. (2015) Inconsistent Treatment Estimates from Mis-Specified Logistic Regression Analyses of Randomized Trials. Statistics in Medicine, 34, 2681-2694.