Because published official statistics of causes of death are based only on the most fundamental cause of death, the social impact of influenza cannot be estimated from them. Influenza is classified sometimes as a cause leading directly to death and other causes are classified the most fundamental cause of death. Actually, influenza outbreaks are well known to induce death by stroke or heart attack      . Given these circumstances, to recognize the social impact of an influenza outbreak, the World Health Organization (WHO) recommends a concept for excess mortality caused by an influenza outbreak  . Hereinafter, “excess mortality” is defined as that caused by influenza.
Actually, it should be impossible to laboratory test for all death. Alternatively, estimation of influenza associated death from their last symptoms leads to drastic underestimation. Instead of counting influenza associated death, “excess mortality” was estimated using a statistical model with consideration of the pattern of fluctuation of non influenza associated death. In fact, WHO used Serfling model  . Furthermore, it had been adopted by the Center for Disease Control and Prevention (CDC) in the USA after modification. However, because the Serfling model excludes past epidemic seasons, the possibility exists that the number of excess mortality might be underestimated when the influenza epidemic was relatively small  . Moreover, the current CDC model uses only the latest five years; it might not use sufficient information. On the other hand, Choi and Thacker  adopted the seasonal autoregressive integrated moving average (SARIMA) model to estimate influenza and pneumonia death using data in the “non-epidemic seasons” when no influenza outbreak occurred, especially in summer. They defined “non-epidemic seasons” from widespread reporting to CDC from the states and information of influenza virus isolation. However, it sounds artificial. Moreover, they defined the number of deaths caused by influenza in epidemic season if influenza did not outbreak as the reverse pattern in “non-epidemic” seasons. Such a definition is apparently too rough and probably leads to underestimate excess mortality. The Serfling model also shared those shortcomings in SARIMA model. In Japan, some studies proposed to estimate excess mortality had been as a simple average so far    . Therefore, they were not precise estimations.
We developed the National Institution of Infectious Diseases (NIID) model, which overcomes some of the shortcomings described above such as necessity of “non-epidemic season.” It does not presume a baseline as reverse pattern in “non-epidemic season.” It is therefore a more appropriate model. Moreover, we have applied the NIID model to the timely situation awareness of influenza associated death project by Ministry of Health, Labour and Welfare (MHLW) and have published the results in Japanese at https://www.niid.go.jp/niid/ja/flu-m/2112-idsc/jinsoku/131-flu-jinsoku.html since 2000.
Actually, the NIID model is a kind of Stochastic Frontier Estimation. We explained its theoretical framework and compared it to other models including the CDC model  , SARIMA Model  , and the Takahashi model    . The objective of this study is to compare the NIID model with other models based on the same data. Even though the NIID model had been used as a procedure for official excess mortality in Japan for almost twenty years, it has not been compared with other models. In fact, among models other than the NIID model, comparison using the same data and under the same condition has never been reported.
2. Materials and Methods
2.1. Data Source
The mortality data used for this study was all caused death in Vital Statistics from January 1987 to March 2017, although CDC model, SARIMA model, and Takahashi model were applied for other types of data in the original research      so as to compare the model performance. It was monthly data because weekly data of mortality is not available in Japan. Also, it did not include virological data depending on circumstances in Japan.
Aside from the NIID model, a non-epidemic period and an epidemic period must be defined ex ante for the three models. The definitions of epidemic period mutually differed in their original research      . Therefore, we presume it as December through March for all years.
2.2. Concept Framework
Excess mortality is defined as the difference between the actual number of deaths and the epidemiological threshold. The epidemiological threshold is defined as the upper bound of 95% confidence interval (CI) of the baseline. The baseline is defined as the number of deaths that are likely to have occurred if an influenza outbreak had not happened. Therefore, if the actual deaths were fewer than the epidemiological threshold, then excess mortality was not observed.
Sometimes the actual number of deaths might be less than the baseline. The baseline was defined as the number of deaths occurring without an influenza outbreak. Therefore, it is inconsistent with the definition of the baseline. Estimation should include measurement error. Therefore, occurrence of fewer death than in the baseline can happen. However, if the actual number of deaths is less than the lower bound of 95% CI, then it must be designated it as negative excess mortality, which reflects an unusually good health condition of the population. Such a shock was not defined in the conceptual framework of excess mortality. Actually, such situations are undesirable because they imply inappropriate use of the model: overestimating the baseline and underestimating excess mortality. Therefore, the frequency and size of negative excess mortality is expected to be an indicator of the estimative capability of the model for excess mortality. We adopt it as the evaluation axis for inter-model comparison.
2.3. Statistical Procedures
2.3.1. NIID Model
Our NIID model, the Stochastic Frontier Estimation  -  , is presented as
where Dt represents all causes of death in month/year t, Tt denotes the linear time trend, and Mit is the dummy variable for a month, which is one if t is the i-th month and otherwise zero. Moreover, νt and ωt are stochastic variables as νt ~N(0, μ2) and ωt ~N(0, ξ2) and are mutually independent. Although νt is stochastic disturbances, ωt represents non-negative death caused by influenza. These disturbance terms in this model are parameterized by two parameters: ξ/μ and (μ2 + ξ2)0.5. If the null hypothesis ξ/μ = 0 is not rejected, then the Stochastic Frontier Estimation model is inappropriate.
2.3.2. CDC Model
The CDC model was defined originally for weekly data in the non-epidemic season as
However, we can arrange it for monthly data for comparison as
2.3.3. SARIMA Model
The SARIMA (k, l, m, n) model is represented in non-epidemic season as shown below.
where L is a lag operator. Also, denotes AR(k) part, 1-L signifies integration, expresses MA(m) part, stands for seasonality AR(l) part and represents the seasonality MA(n) part. The number of lags in each part (k, l, m, n) should be selected using the Akaike Information Criterion which minimize −2log likehood + 2 (number of parameters).
2.3.4. Takahashi Model
The Takahashi model assumes a baseline as geometric average for each month in the non-epidemic season.
2.4. Comparison among Models
To model performance, we apply the NIID, CDC, SARIMA, and Takahashi models to all causality data in Japan from January 1987 through May 2017. For the CDC, SARIMA and Takahashi model, we assumed the non-epidemic season as April-November in each year. As described above, we adopted the number of negative excess mortality, which is defined as the number of months during which the actual number of deaths is lower than the lower bound of 95% CI of the baseline in epidemic season.
2.5. Ethical Consideration
We used only published data. No area of ethical concern is applicable to this study.
The Takahashi model required a non-epidemic season in the same month, for example, in January. Therefore, we declined this model to compare other models under the current assumption of the non-epidemic season.
3.1. Estimation Results Obtained Using the NIID Model, CDC Model, and SARIMA Model
Table 1 presents the estimation result obtained using the NIID model. The null hypothesis of ξ/μ = 0 was rejected. Therefore, the Stochastic Frontier model was appropriate. The p-values of all coefficients except for dummy variable for March were less than 0.004 and this were significant.
Table 2 shows the CDC model. Table 3 shows AIC in combination of and the number of parameter of AR, MA, SAR and SMA parts, (k, l, m, n) in Equation (4) for SARIMA model. Based on AIC, we selected the (2, 1, 1, 2) model. Table 4 presents the estimation result obtained using the SARIMA (2, 1, 1, 2) model. The p-values of all coefficients of CDC model and SARIMA model were less than 0.004 and this were significant.
Table 1. Estimation result of NIID model.
Note: The number of observations was 365. Log likelihood was 822.690. ξ2 denotes the variance of the non-negative disturbance term. μ2 is the variance of the disturbance term.
Table 2. Estimation results obtained using the CDC model.
Note: The number of observations was 273. Log likelihood was −515.693. The coefficient of determination was 0.959138.
Table 3. Model selection in SARIMA model.
Note: The AR column shows the number of parameters in the autoregressive part, k, in Equation (3). The MA column shows the moving average part, m. The SAR column shows the seasonal autoregressive part, l. The SMA column shows the moving average part, n. AIC is defined as −2 × log of likelihood +2 Number of parameters. Finally, we selected (2, 1, 1, 2) model.
Table 4. Estimation result of SARIMA model.
Note: The number of observations was 273. Log likelihood was −3086.05. The coefficient of determination was 0.586015. Also, φs are parameters of autoregressive (AR) part, θ is the moving average (MA) part, γ is the seasonal autoregressive (SAR) part, and δs is the seasonal moving average (SMA) part.
3.2. Comparison among Three Models
Table 5 presents the number of months during which excess mortality was observed and when the negative excess mortality was observed during epidemic season in the current assumptions in the three models. Among 123 months, the NIID model found excess mortality in 56 months, the CDC model in 31 months, and the SARIMA model in 35 months. Conversely, the NIID model found negative excess mortality in only 2 months, but the CDC model and SARIMA model found it respectively in 10 months and 33 months.
Figure 1 depicts excess mortality and negative excess mortality during the epidemic season under the current assumption. Negative excess mortality in the NIID model was observed only in the earlier two years. The highest negative excess mortality in the SARIMA model was higher than CDC model. At some period, when the NIID model found excess mortality, the SARIMA model and CDC model found negative excess mortality. The correlation among the estimated excess mortality based on three models was higher than 0.8 and significant.
This paper presented NIID model which applies the situation in Japan, with results published officially for almost twenty years. Results demonstrate that little negative excess mortality existed in NIID model, but the other two models were adversely affected by several instances of negative excess mortality. Negative excess mortality demonstrates the logical inconsistency of estimation model. Therefore, the NIID model might be the best model among the three models considered. In other words, the baseline is apparently overestimated, i.e. excess mortality seems to underestimate in the other two models.
One benefit of the NIID model was that the arbitrary setting of non-epidemic season is unnecessary, unlike other models. Therefore, we can use all data. In the other models, during the ex post epidemic period, the estimation result is obtained when excess mortality is observed does not necessarily match the ex ante epidemic period as arbitrarily set before estimation. That is apparently an inconsistency in
Table 5. Number of months during which excess mortality observed and during which negative excess mortality was observed in the three models among December through March in each season.
Note: The number of months under consideration was 123.
Figure 1. Excess mortality and the negative excess mortality during epidemic season. Note: The line shows excess mortality and negative excess mortality by the NIID model. The black bar shows the CDC model. The gray bar represents the SARIMA model. Epidemic season is defined as December through March of the following year. Bars or lines higher than zero signify excess mortality. Values lower than zero represent negative excess mortality. If the actual number of deaths is less than the upper threshold and higher than the lower threshold, then both excess mortality and negative excess mortality are expected to be zero.
estimation model. Conversely, the NIID model can avoid such internal inconsistency.
Finally, one must recall that comparison among four models was conducted using the same data: all causes of death. However, the original research of the CDC and SARIMA model used other data for mortality. Moreover, those models including the Takahashi model adopted their unique definition of the non-epidemic period. Therefore, precise comparison including the type of data and definition of non-epidemic period was impossible. To compare those three models, we used common monthly data of mortality and common definition of non-epidemic season for CDC model and SARIMA model in the present study. Therefore, it is noteworthy that we did not compare the benefits and shortcomings of the four original models.
The present study was constrained by some limitations. First, we compare three models using Japanese monthly data and under the same simple assumption of non-epidemic season, even though the original CDC model and SARIMA model used weekly data and a flexible definition of non-epidemic season. If we compare the four models using weekly data and/or with original flexible definition of non-epidemic season might be different from the present study. However, in Japan, weekly mortality data is not available in general. It remains as challenge for future research.
Second, we do not compare the Takahashi model to the other three models because of simple assumption of non-epidemic season. The Takahashi model might be compared to other three models if one were to adopt the original definition of non-epidemic season in the Takahashi model. However, in fact in the last ten years, influenza activity in January or February were quite high and thus those period should be included to epidemic season, the Takahashi model might not work in this period.
Thirdly, although we used only calendar information, recent studies tend to examine virological data specifically  -  . Unfortunately, the positive rate by semi-types of influenza over all the tested sample were not collected and published in Japan, although the proportion of semi-types over all influenza-positive sample were published. Therefore, we cannot use virological data to predict excess mortality model as previous studies. For that purpose, we need information of the number of all tested sample including negative sample. That improvement of data remains as a challenge. However reform for testing policy will be necessary.
We demonstrated that the NIID model might be more appropriate than the other two models in the respect of logical adequacy concerning about “negative excess mortality.” The other two models apparently underestimate of excess mortality under the same data and definition of non-epidemic season. It has been used as a procedure for official excess mortality in Japan for almost twenty years and has been applied to large cities, with results of estimated excess mortality published timely as an MHLW project.
We acknowledge Dr. Nobuhiko Okabe, Kawasaki City Institute for Public Health C, Dr. Kiyosu Taniguchi, National Hospital Organization Mie National Hospital, and Dr. Nahoko Shindo, WHO for their helpful support.
 Lin, H.C., Chiu, H.F., Ho, S.C. and Yang, C.Y. (2014) Association of Influenza Vaccination and Reduced Risk of Stroke Hospitalization among the Elderly: A Population-Based Case-Control Study. International Journal of Environmental Research and Public Health, 11, 3639-3649.
 Asghar, Z., Coupland, C. and Siriwardena, N. (2015) Influenza Vaccination and Risk of Stroke: Self-Controlled Case-Series Study. Vaccine, 33, 5458-5463.
 Kwok, C.S., Aslam, S., Kontopantelis, E., Myint, P.K., Zaman, M.J., Buchan, I., Loke, Y.K. and Mamas, M.A. (2015) Influenza, Influenza-Like Symptoms and Their Association with Cardiovascular Risks: A Systematic Review and Meta-Analysis of Observational Studies. International Journal of Clinical Practice, 69, 928-937.
 Muhammad, S., Haasbach, E., Kotchourko, M., Strigli, A., Krenz, A., Ridder, D.A., Vogel, A.B., Marti, H.H., Al-Abed, Y., Planz, O. and Schwaninger, M. (2011) Influenza Virus Infection Aggravates Stroke Outcome. Stroke, 42, 783-791.
 Choi, K. and Thacker, S.B. (1981) An Evaluation of Influenza Mortality Surveillance, 1962-1979. I. American Journal of Epidemiology, 113, 215-226.
 Aiger, A.D., Lovell, K. and Schmitidt, P. (1977) Formulation and Estimation of Stochastic Frontier Production Function Models. Journal of Econometrics, 6, 21-37.
 Jondrow, J., Lovell, K., Materov, S. and Schmidt, P. (1982) On the Estimation of Technical Inefficiency in the Stochastic Frontier Production Function Model. Journal of Econometrics, 19, 233-239.
 Li, T. and Rosenman, R. (2001) Cost Inefficiency in Washington Hospitals: A Stochastic Frontier Approach Using Panel Data. Health Care Management Science, 4, 73-81.
 Jacobs, R. (2001) Alternative Methods to Examine Hospital Efficiency: Data Envelopment Analysis and Stochastic Frontier Analysis. Health Care Management Science, 4, 103-115.
 Jackson, M.L., Peterson, D., Nelson, J.C., Greene, S.K., Jacobsen, S.J., Belongia, E.A., Baxter, R. and Jackson, L.A. (2015) Using Winter 2009-2010 to Assess the Accuracy of Methods Which Estimate Influenza-Related Morbidity and Mortality. Epidemiology & Infection, 143, 2399-2407.
 Goldstein, E., Viboud, C., Charu, V. and Lipsitch, M. (2012) Improving the Estimation of Influenza-Related Mortality over a Seasonal Baseline. Epidemiology, 23, 829-838.
 Wu, A., Peng, Y., Du, X., Shu, Y. and Jiang, T. (2010) Correlation of Influenza Virus Excess Mortality with Antigenic Variation: Application to Rapid Estimation of Influenza Mortality Burden. PLOS Computational Biology, 6, e1000882.
 Nielsen, J., Mazick, A., Glismann, S. and Molbak, K. (2011) Excess Mortality Related to Seasonal Influenza and Extreme Temperatures in Denmark, 1994-2010. BMC Infectious Diseases, 11, 350.
 Muscatello, D.J., Newall, A.T., Dwyer, D.E. and Macintyre, C.R. (2013) Mortality Attributable to Seasonal and Pandemic Influenza, Australia, 2003 to 2009, Using a Novel Time Series Smoothing Approach. PLoS ONE, 8, e64734.
 Park, M., Wu, P., Goldstein, E., Joo, K.W. and Cowling, B.J. (2016) Influenza-Associated Excess Mortality in South Korea. American Journal of Preventive Medicine, 50, e111-e119.