JBM  Vol.7 No.12 , December 2019
Retrospective Evaluation for ORCA Surveillance Comparison with Prescription Surveillance
ABSTRACT
Background: “Online Receipt Computer Advantage” (ORCA) surveillance, an online medical claim service produced and provided freely to medical facilities by the Japan Medical Association, has been available for public use since 2010. However, its preciseness has not been evaluated. The object of the present study is comparison with ORCA surveillance and prescription surveillance (PS), which is the most reliable daily surveillance for influenza nationwide in Japan, and evaluating preciseness of ORCA surveillance. Method: We specifically examined influenza as a targeted disease. We regarded PS as the gold standard daily real-time monitoring system for influenza nationwide and for each prefecture. This study assesses correlation and discrepancies between ORCA and PS results for influenza nationwide and by prefecture. Results: Nationwide, the correlation coefficient of PS and ORCA was 0.9653; the discrepancy rate was 27%. Among prefectures, results show that Mie and Fukui prefectures had two quite similar epidemic curves. Conclusion: Results demonstrate that ORCA surveillance is comparable to PS nationwide. However, some prefectures exhibited comparable results whereas others did not. ORCA surveillance cannot break down to the municipal level.

1. Introduction

Prescription Surveillance (PS), as operated in Japan by the Japan Medical Association, Japan Pharmaceutical Association, School of Pharmacy, and EM Systems Co. Ltd., has been functioning as a nationwide real-time syndromic surveillance system [1] - [7] as along with (N) SASSy which has been serving 65% of Japan [6] - [12]. Since 2009, PS has been reporting the estimated number of influenza and chicken pox patients, in addition to the numbers of patients who are prescribed drugs of certain types, based on prescriptions at pharmacies. In fact, PS estimates the numbers of patients every day from the number of prescriptions by prefecture by age group. As of the end of April 2015, approximately ten thousand pharmacies were participating, collectively accounting for about 20% of all pharmacies nationwide. The estimated numbers of patients are presented on the PS web page the following morning (http://prescription.orca.med.or.jp/kanjyasuikei/).

That estimate by PS represents multiplication of the reciprocal of the pharmacy PS participation rate and the reciprocal of the proportion of prescriptions at external pharmacies in the prefecture by the total number of prescriptions issued in the prefecture. The results are estimates of the numbers of patients prescribed neuraminidase inhibitors (NI), anti-herpes virus drugs (AHV), antibiotic drugs (AB), antipyretic analgesics (AP), multi-ingredient cold medications (MIC), and antidiarrheal and intestinal drugs (AD). Antibiotics are classified into five types: penicillin (ABP), cephem (ABC), macrolide (ABM), new quinolone (ABQ), and others (ABO) [1] [2].

Since 2010, Online Receipt Computer Advantage (ORCA) surveillance has presented results for public use. The results include the estimated daily number of patients by prefecture or municipalities in Japan for influenza, hand-foot-and-mouth disease, Erythema infectiosum, Pharyngo conjunctival fever, RS virus infection, measles, rubella, and insolation from ORCA, which is an electronic medical claim system (http://infect.orca.med.or.jp/). The ORCA online medical claim service is produced and provided to medical facilities freely by the Japan Medical Association. As of the end of September, 2019, ORCA data were used at 17,174 medical facilities.

This information was collected from about 4218 medical facilities as of the end of September, 2019. Given Japan’s total number of medical facilities as about 100 thousand, its participation rate was then about 4%. Nevertheless, its precision has not been examined to date. This report describes examination of its precision compared to that of PS: the gold standard of daily nationwide surveillance.

2. Method

We specifically examined influenza as a targeted disease. We regarded PS as the gold standard of a daily real-time monitoring system for influenza nationwide and for each prefecture. The study period was September 1, 2010 to December 30, 2018. The study areas were regarded as nationwide and in 47 prefectures. We identified the number of influenza patients in ORCA surveillance as the number of patients diagnosed and recorded in ORCA multiplied by the ORCA surveillance participation rate for all medical facilities in the considered area. Influenza season was defined as September 1 to the end of August of the subsequent year.

We examine correlation between ORCA and PS results, and discrepancies between ORCA and PS results for influenza nationwide and by prefecture. The discrepancy was calculated as the weighted average of the absolute value of difference between ORCA surveillance results and PS results weighted by the number of influenza patients in PS.

Ethics

Data from ORCA surveillance and PS were aggregated and de-linked from personal information related to patients, medical facilities, and pharmacies: these are anonymous data. Moreover, these were published to the general public. Therefore, no ethical issues are posed by the use of these data for this study.

3. Results

About 2000 facilities, at maximum, reported influenza patients. The numbers of influenza patients estimated by ORCA surveillance and PS nationwide by season are shown in Figure 1. Figure 2 depicts the epidemic curves of ORCA surveillance and PS nationwide. The correlation coefficient of PS and ORCA nationwide results was 0.96. The discrepancy rate was 27.45%.

Figure 3 presents correlation coefficients and discrepancy rates between ORCA surveillance and PS results by prefecture. Large correlation coefficients represent higher similarity. A lower discrepancy rate implies higher similarity. Figure 3 depicts the best five prefectures for high correlation as Wakayama, Tottori, Mie, Fukui, and Nagano. Especially, correlation in the top four prefectures was higher than 0.8. The best five prefectures in terms of low discrepancy

Figure 1. Estimated number of influenza patients from ORCA surveillance and PS by Season. Note: Black bars represent the estimated number of influenza patients from ORCA surveillance. Gray bars represent data from PS.

Figure 2. Epidemic curves for Influenza from ORCA and PS. Notes: Black line is the epidemic curve in PS. Gray line is the epidemic curve in ORCA.

Figure 3. Correlation coefficient and discrepancy rate of the number of influenza patients from ORCA and PS by prefecture. Note: Gray bars show correlation coefficients for PS and ORCA. Lines show discrepancy rates among PS and ORCA. Therefore, the higher bar and lower line means that PS and ORCA are similar in the prefecture. Discrepancy rates was defined sum of absolute value of difference among the number of patients in PS and ORCA divided by total number of patients in PS in the study period.

were Ehime, Mie, Fukui, Yamagata, and Akita. Discrepancies in the top two prefectures were especially low: less than 20%. Nagasaki, Hiroshima, Okayama, Gifu, and Kagoshima were the worst five prefectures in terms of correlation: around 0.55. Discrepancy rates in these five prefectures were higher than 100%.

4. Discussion

Figure 1 depicts similarity in earlier seasons. However, discrepancies among the two surveillance modes tend to be larger more recently. For instance, discrepancies in 2011/2012 and 2012/2013 seasons were less than 0.5 million, but grew to about 2 million in the 2016/2017 season. At least nationwide, two epidemic curves showed high correlation and low discrepancy. We conclude that ORCA surveillance has sufficient precision.

At the prefecture level, results show that Mie and Fukui had similar epidemic curves. Nevertheless, correlation and discrepancy even in these best two prefectures were worse than the results obtained nationwide. In Saga prefecture, PS covers about 60% of all pharmacies, constituting the highest participation rate in all of Japan. Therefore, PS in Saga was used as the most reliable monitor of influenza by the Saga prefectural office [13]. Nevertheless, the correlation and discrepancy figures there were, respectively, about 0.6% and 100%. Therefore, ORCA surveillance showed no high similarity with PS, at least in Saga. This result suggests that ORCA surveillance provides insufficient precision at the prefecture level.

Moreover, even though ORCA surveillance provides information about municipalities, the ORCA surveillance precision for each municipality was not confirmed because PS does not provide information for municipalities. For the more than 1700 municipalities in Japan, medical facilities reporting to ORCA surveillance were about one per municipality, on average. Therefore, the information might represent information at only one medical facility, not community data. For PS, the smallest areas were defined as those where more than10 pharmacies had joined PS. Such areas are sometimes smaller than a municipality, sometimes defined as joint neighborhood municipalities. For example, one area in Saga prefecture was defined as a junior high school area. Therefore, utilization of information from ORCA surveillance at a municipality level, especially for such areas that include fewer than 10 facilities, demands some caution.

In this sense, because there were about 40 facilities in a prefecture in average, we might consider virtual districts as joint municipalities. To contain more than 10 facilities, one prefecture will have four such vertical districts, in average. That is a limitation of fineness when monitoring community outbreaks using ORCA surveillance. In decades later, because participating facilities has been increasing by about 1% annually, the preciseness of ORCA surveillance at prefecture level might improve and breaking down to municipality level might be possible.

The study described herein has some limitations. We first considered ORCA surveillance retrospectively. However, results might be different in a prospective setting. Because ORCA itself is the system for medical claims and because medical facilities claim public health insurance at the end of month, entry of diagnosis to ORCA is sometimes delayed until the end of the month. In other words, the number of patients in ORCA surveillance is probably under-reported in real-time. Totals might increase up to the end of month. In other words, the precision of real-time ORCA surveillance is expected to be less than that of retrospective evaluation, which we considered in the present study. When we trace the number of patients from visiting day to the end of month, the number of patients increased by 15%. It may be substantial uner-reporting. Prospective evaluation for ORCA surveillance is anticipated as our next challenge.

Secondly, we regarded PS as the gold standard. That might be true if one considers daily real-time surveillance for influenza. However, calculation of the total number of influenza patients during a season requires no real-time capability. In Japan, the National Official Surveillance for Infectious Diseases (NOSSID) is operated based on a related law. However, it is the sentinel surveillance. It cannot report the estimated number of infectious diseases [2]. Moreover, information from all medical claims is disclosed in the “National Database of Health Insurance Claims and Specific Health Checkups of Japan” [14] by the Ministry of Health, Labour and Welfare (MHLW) of Japan [15] [16] [17] [18]. We designate this as the National Database of Electronic Medical Claims (NDBEMC). These data were associated with 98.4% of all medical claims made in 2015 [19]. They areal most complete data, which include data from PS and ORCA as components. However, MHLW requires issuance of a permit to use NDBEMC. Although many researchers have sought permission to use it, its approval process alone requires more than one year. Therefore, evaluation of precision during the season using NDBEMC has been postponed.

Thirdly, we evaluated ORCA surveillance only for influenza patients. However, ORCA surveillance include other infection diseases and insolation as described above. Although PS includes the numbers of influenza and varicella or herpes zoster patients, it does not include some information associated with other infectious diseases. Therefore, we limited analyses in this study to the number of influenza patients. Future studies must evaluate the precision of ORCA surveillance for other infectious diseases or insolation.

Fourthly, we showed that ORCA surveillance did not provide reliable information at municipalities level due to limitation of participated medical facilities per municipalities. In average, there was only one participated medical facility to ORCA surveillance per municipalities. Instead of municipalities, we suppose virtual districts as joint municipalities which contain more than 10 facilities. Examination for these vertical districts in ORC surveillance should be the next challenge.

5. Conclusion

We confirmed results of ORCA surveillance as comparable to PS results nationwide. Regarded at the prefecture level, these were comparable in some prefectures, but not in other prefectures. The precision of ORCA surveillance was not confirmed at the municipal level. More evaluation must be done of ORCA surveillance prospectively, and using NDBEMC, and for diseases other than influenza.

Cite this paper
Ueno, T. , Sugawara, T. , Ohkusa, Y. , Kawanohara, H. and Kamei, M. (2019) Retrospective Evaluation for ORCA Surveillance Comparison with Prescription Surveillance. Journal of Biosciences and Medicines, 7, 1-8. doi: 10.4236/jbm.2019.712001.
References
[1]   Sugawara, T., Ohkusa, Y., Ibuka, Y., Kawanohara, H., Taniguchi, K. and Okabe, N. (2012) Real-Time Prescription Surveillance and Its Application to Monitoring Seasonal Influenza Activity in Japan. Journal of Medical Internet Research, 14, e14.
https://doi.org/10.2196/jmir.1881

[2]   Nakamura, Y., Sugawara, T., Kawanohara, H., Ohkusa, Y., Kamei, M. and Oishi, K. (2015) Evaluation of the Estimated Number of Influenza Patients from National Sentinel Surveillance Using National Database of Electronic Medical Claims. Japanese Journal of Infectious Diseases, 68, 27-29.
https://doi.org/10.7883/yoken.JJID.2014.092

[3]   Sugawara, T., Ohkusa, Y., Kawanohara, H. and Kamei, M. (2018) Short Term Prediction of Infectious Diseases Patients from Prescription Surveillance. Journal of Biosciences and Medicines, 6, 61-68.
https://doi.org/10.4236/jbm.2018.69007

[4]   Ohkusa, Y., Sugawara, T., Kawanohara, H. and Kamei, M. (2018) Evaluation of the Global Action Plan on Antimicrobial Resistance in Japan during Its First Eighteen Months. Drug Discoveries & Therapeutics, 12, 182-184.
https://doi.org/10.5582/ddt.2018.01011

[5]   Sugawara, T., Ohkusa, Y., Kawanohara, H. and Kamei, M. (2018) Prescription Surveillance for Early Detection System of Emerging and Reemerging Infectious Disease Outbreaks. Bio-Science Trends, 12, 523-525.
https://doi.org/10.5582/bst.2018.01201

[6]   Yamakawa, M., Yamauchi, M., Nidaira, M., Azuma, T., Nakasone, T., Ando, F., Hayamizu, T., Kinjo, S. and Nakamura, Y. (2017) Enhanced Public Health Surveillance for the Sixth Worldwide Uchinanchu Festival Conducted by the Okinawa Prefectural Government. Japan Journal of Biosciences and Medicines, 5, 106-115.
https://doi.org/10.4236/jbm.2017.59010

[7]   Kambe, C., Fujii, H., Niu, T., Matsuura, H., Nagai, N., Nakamura, Y. and Nojiri, T. (2018) Enhanced Surveillance for National (Handicapped) Sports Games in Wakayama, Japan 2015. Journal of Biosciences and Medicines, 6, 35-47.
https://doi.org/10.4236/jbm.2018.67004

[8]   Kurita, J., Nagasu, N., Nagata, N., Okuno, H. and Sugawara, O.Y. (2018) Outbreak of Human Metapneumovirus in Ibaraki, Japan and Its Descriptive Epidemiology. Health, 10, Article ID: 85348.
https://doi.org/10.4236/health.2018.106057

[9]   Kurita, J., Sugawara, T., Matsumoto, K., Nakamura, Y. and Ohkusa, Y. (2018) Association among (Nursery) School Absenteeism Surveillance System and Incidence of Infectious Diseases. School Health, 14, 21-27.
https://doi.org/10.1111/ped.14023

[10]   Kurita, J., Nagasu, N., Nagata, N., Sakurai, N., Ohkusa, Y. and Sugawara, T. (2018) Descriptive Epidemiology for Mycoplasma pneumoniae Infection Using (Nursery) School Absenteeism Surveillance System, and Proposal for Countermeasures. Journal of Biosciences and Medicines, 6, 33-42.
https://doi.org/10.4236/jbm.2018.610005

[11]   Sugishita, Y., Sugawara, T. and Ohkusa, Y. (2019) Association of Influenza Outbreak in Each Nursery School and Community in a Ward in Tokyo, Japan. Journal of Infection and Chemotherapy, 25, 695-701.
https://doi.org/10.1016/j.jiac.2019.03.010

[12]   Tanabe, Y., Kurita, J., Nagasu, N., Sugawara, T. and Ohkusa, Y. (2019) Infection Control in Nursery Schools and Schools Using a School Absenteeism Surveillance System. The Tohoku Journal of Experimental Medicine, 247, 173-178.
https://doi.org/10.1620/tjem.247.173

[13]   Ohkusa, Y., Sugawara, T., Taiguchi, K., Okabe, N., Moriya, K., Yoshimura, A., Yamaguchi, K., Nagao, K., Suetsugu, M., Furukawa, T. and Hirako, T. (2013) Analysis about Age Distribution of Influenza Patients in Saga Prefecture. Kosei No Shihyo, 60, 25-29. (In Japanese)

[14]   Ministry of Health, Labour and Welfare, Japan (2018) National Database of Health Insurance Claims and Specific Health Checkups of Japan. (In Japanese)
https://www.mhlw.go.jp/file/06-Seisakujouhou-12400000-Hokenkyoku/0000193322.pdf

[15]   Health Insurance Bureau, Ministry of Health, Labour and Welfare, Japan (2013) Expert Meeting of Providing the Information of Medical Claims and Others. The Report of Providing the Information of Medical Claims and Health Check-Up for Metabolic Syndrome to the Third Party. (In Japanese)

[16]   Health Insurance Bureau, Ministry of Health, Labour and Welfare (2011) Expert Meeting of Providing the Information of Medical Claims and Others. The Guideline for Providing the Information of Medical Claims and Health Check-Up for Metabolic Syndrome. (In Japanese)

[17]   Nakamura, Y., Sugawara, T., Ohkusa, Y., Taniguchi, K., Miyazaki, C., Momoi, M. and Okabe, N. (2018) Severe Abnormal Behavior Incidence after Administration of Neuraminidase Inhibitors Using the National Database of Medical Claims. Journal of Infection and Chemotherapy, 24, 177-181.
https://doi.org/10.1016/j.jiac.2017.10.001

[18]   Miura, K., Miyagawa, N., Murakami, Y. and Okayama, A. (2013) Outlines of the National Database on Medical Expenditure and Health Check-Up. Hoken Iryou Kagaku, 62, 31-35. (In Japanese)

[19]   Health Insurance Claims Review & Reimbursement Services (2015) Status of Medical Claims by Claim Form (Diagnosed in April, 2015). (In Japanese)
http://www.ssk.or.jp/tokeijoho/tokeijoho_rezept/tokeijoho_04_h27.files/seikyu_2704.pdf

 
 
Top