A substantial part of the general population in the Netherlands and worldwide is affected by musculoskeletal complaints (MSC)  . About 5% of the worldwide population suffers from chronic inflammatory arthritis (IA) . The most frequent types of IA are rheumatoid arthritis (RA), axial spondyloarthritis (axSpA) and peripheral spondyloarthritis such as psoriatic arthritis (PSA). Early diagnosis is vital for the response of IA treatment in order to achieve a state of remission sooner, which consequently prevents joint damage on the longer term and increases quality of life . Early diagnosis requires early recognition of patients at risk for IA.
In most countries, general practitioners (GPs) have a pivotal role in early recognition of patients with inflammatory joint complaints, because they act as a gatekeeper for referring patients to secondary care. However, because the GP sees such a large heterogeneity of patients with MSC, it has proven to be difficult to recognize those patients at risk of IA  . This is reflected by the low percentage of IA in patients referred to the rheumatology outpatient clinic. The majority, over 70 percent, is diagnosed with a non-inflammatory disease, and no chronic care by a rheumatologist is needed  . Moreover, Western countries experience an increasing demand for care, especially for musculoskeletal disorders . It is predicted that the total amount of referrals to the rheumatologist will increase enormously in the future , leading to more costs over the entire care cycle.
Integrated care, in which primary and secondary care bundle their expertise to improve the accessibility, quality and efficiency of care, may be a solution to this problem. This is in line with the principles of value based health care (VBHC), e.g. maximizing value for patients. The value is defined as the health outcomes that are most important to the patient, divided by the costs of healthcare over the entire care cycle . The Dutch government also recognizes this need for change in healthcare organization . Healthcare could be organized around customized care, by providing the patient with the right care at the right place. In the early phase of IA this might be accomplished by using cost-effective referral strategies that will decrease the time to referral for IA and the amount of referrals of patients with a non-inflammatory disease.
Previous studies have investigated the effectiveness of either structured referral sheets or shifted outpatient clinics as referral strategies    . A Cochrane review confirmed that structured referral sheets can improve appropriateness of referrals significantly, and have good potential to improve cost-effectiveness . Within consultant clinics in the community, the expertise of hospital specialists is provided in a primary care setting. Studies have revealed that the use of these shifted outpatient clinics leads to improved accessibility , equal quality of care , improved patient satisfaction, reduced waiting times and considerable cost savings to patients .
Until now, the impact of implementation of these innovative referral strategies has not yet been investigated within rheumatology and evidence of its value expressed in cost-effectiveness is lacking . Nevertheless, it is suggested to have great potential to improve appropriateness of patient referral to rheumatology centers.
The primary aim of this study is improvement in the number of appropriate patient referral to secondary care rheumatology centers.
Secondarily, the aim is to estimate the value expressed in cost-effectiveness of innovative referral strategies. Another secondary aim is to validate the electronic referral sheet including algorithms for axial spondyloarthritis, psoriatic arthritis and early rheumatoid arthritis in primary care patients.
2.1. Study Design
In this cluster randomized controlled trial, carried out in a primary care setting in the province of South-Holland in the Netherlands, primary care centers are regarded as clusters. Each cluster contains the GPs from one practice and their included patients to overcome referral bias by GPs. General practitioners in and around Rotterdam in the province of South-Holland in the Netherlands are invited to participate by a regional newsletter or by personal approach. If they are willing to participate, the researcher will train both the GP and the GP’s assistants in enrolling patients for study participation. The primary care centers will be randomly assigned to either one of the study groups.
This trial consists of three study groups; one in which an electronic referral sheet is used, one consisting of triage by a rheumatologist in a primary care setting and a control group in which patients are referred to the secondary care rheumatology center in accordance with usual care. The study was initiated in April 2017 and inclusion is expected to be finalized in December 2019.
2.2. Study Population
Patients aged eighteen years or older visiting primary care due to physical complaints, who are in the opinion of their own GP suspected for IA, will be invited to participate. In order to be eligible to participate in this study, participants should be able to understand and communicate in Dutch.
All eligible patients receive a patient information form and if agreed to participate, an informed consent form is signed and an inclusion visit is assessed in one of the three study groups. For those who do not want to participate, the reason for not participating will be registered.
Intervention group 1: Electronic structured referral sheet for IA.
This intervention group usesthe electronic structured referral algorithm for patients at risk for IA (Figure 1). This algorithm is based on a combination of recently developed and validated referral questionnaires for axial spondyloarthritis (CaFaSpA ), psoriatic arthritis (PEST ) and for early rheumatoid arthritis (CARE ). Which exact items are applicable to a specific patient is dependent on the main complaints of that patient (Figure 1).
After completing one of the three referral questionnaires, a total score will be calculated. If this score is above the threshold value of the corresponding referral algorithm, the patient is considered to be at risk for IA, and the GP is given an advice to refer the patient to a rheumatologist. For the CaFaSpA this threshold value is a score of equal to or above two out of four points , for the PEST this is a score of equal to or above three out of five points  and for the CARE this is a score of equal to or above four points out of seven and a half . To ensure quality of care, the GP can still refer the patient if the score is below the referral threshold.
Figure 1. Algorithms combined in the electronic structured referral sheet.
The GP is then requested to declare the reason why the patient is still referred. Those that are suspected of IA according to the structured referral algorithm will be referred to a rheumatology outpatient clinic and receive regular diagnostic workup, treatment and follow-up as needed based on the international guidelines for IA.
Intervention group 2: Triage by rheumatologist in a primary care setting.
The second intervention is triage by an experienced rheumatologist in a primary care setting. Patients suspected of an inflammatory rheumatic disease according to their GP, will be examined by the rheumatologist in their own primary care practice. The rheumatologist will assess the medical history, perform physical examination and advise the GP to refer the patient to secondary care or not. Patients that are not suspected of IA by the rheumatologist will remain in primary care. Those that are suspected of IA according to the rheumatologist will be referred to a rheumatology outpatient clinicand receive regular diagnostic workup, treatment and follow-up as needed based on the international guidelines for IA.
2.4. Control Group
The control group consists of randomly selected patients who are newly referred to the rheumatology outpatient clinic of the Maasstad Hospital. At the inclusion visit, the researcher will fill out the electronic structured algorithm, as used in intervention group 1, based on information from this newly referred patient. This data is used as a validation of the referral algorithm.
All patients in the control group receive regular diagnostic workup, treatment and follow-up by a rheumatologist as needed based on the international guidelines for IA.
2.5. Data collection
Data will be collected concerning the patient as well as the referring GP. After written informed consent is obtained, demographic data is retrieved from the patient administration registry provided by the hospital and GPs. Patient data include age, sex and area code. GP data include characteristics of the GP like age and sex, characteristics of the general practice like practice size, and distance to the Maasstad Hospital Rotterdam, deducted from the area code.
The follow-up of patients will cover twelve months. To collect data, questionnaires are sent to participants by email or by post directly following the inclusion visit at baseline, and again after three months, six months and twelve months (appendix 1). In order to promote participant retention and complete follow-up, reminders will be sent using e-mail, letters and telephone contact.
All data is collected and saved in data management system Castor in compliance with the Dutch Data Protection Act following current Dutch legislation. Research data that can be traced to individual patients can only be viewed by members of the research team, health care inspection and medical ethics committee.
No adverse events will be reported in the study, since no medicinal product will be tested or used as part of this study. All patients receive standard care during the study follow-up and after study closure, and any health-related events or adverse events will receive follow up as part of this standard care.
2.6. Outcome Measures
The primary outcome is the number of patients diagnosed with IA by the rheumatologist as a proportion of all referred patients suspected of IA by the GP.
Secondary outcomes are health-related quality of life, as a patient reported outcome, and costs, including both direct costs from healthcare resources utilization and indirect costs from work productivity. Health-related quality of life will be determined with the EuroQol Health Questionnaire (EQ-5D) . Healthcare resources utilization will be determined with the iMTA Medical Consumption Questionnaire (iMCQ)  as direct costs, and work productivity with the iMTA Productivity Cost Questionnaire (iPCQ)  as indirect costs. These outcomes will be used to perform a cost-effectiveness analysis.
Another secondary objective is to validate the referral models combined into the electronic structured referral sheet. The answers on the included questions will be compared with the diagnosis made by the rheumatologist obtained from the patient administration registry, as a gold standard.
2.7. Sample Size
The sample size calculation is based on a size needed for a mixed effects logistic regression model for the comparison of proportions between three groups. Primary endpoint target of percentage of IA in all referred patients is set at 35% in the intervention groups compared to 20% in usual care. These numbers are based on the finding that in usual care, of all patients newly referred to the rheumatologist, approximately 20% is diagnosed with IA    , and the expectation of increasing this by 15% with the help of an innovative referral strategy. Alpha is set at 0.05/3 = 0.0167 for each comparison (type I error, two-sided, with Bonferroni correction), and the power level is set at 0.8. To correct for a clustering effect by GPs, an inter correlation coefficient of 1.27 was used. Taking into account a patient dropout rate of 25%, the final sample size is set at 296 patients per study group, with a total of 888 patients for this study.
2.8. Data Analysis
For descriptive statistics the number and percentage will be presented for categorical variables. The number, mean, standard deviation, median, interquartile range, minimum and maximum will be presented for continuous variables.
The primary outcome, defined as patient with IA yes or no, will be analyzed using a mixed effects logistic regression model. In case of imbalance of baseline variables, these will be adjusted for in the model. Proportional odds ratios will be presented. The estimated difference between the three study groups will be compared and 95% confidence intervals will be given. Three comparisons will be made, therefore a p-value of 0.05/3 = 0.0167 will be considered significant.
For the cost-effectiveness analysis standard calculation methods will be used for EQ-5D, iPCQ and iMCQ.
For the cost-effectiveness analysis, health-related quality of life (EQ-5D), direct costs (iMCQ) and indirect costs (iPCQ) will be used. The Incremental Cost Effectiveness Ratio (ICER) will be calculated based on differences in mean costs and effectiveness, and 95% confidence intervals will be constructed . For prospectively collected data the volumes will be multiplied with standard reference prices and mean with standard error will be presented using cost-effectiveness acceptability curves.
To verify the accuracy of the algorithms used in the structured referral sheet, answers on the included questions will be compared with the rheumatologist diagnosis as a gold standard. The percentage of rheumatologist diagnosis IA correctly predicted by the referral algorithm will be determined, as well as the sensitivity and specificity for gold standard diagnoses.
For missing data, multiple imputation methods will be used. Any protocol amendment will be notified to the medical ethics committee that reviewed the protocol.
This article describes the rationale and design of a cluster randomized controlled trial to evaluate the cost-effectiveness for two innovative referral strategies for primary care patients with MSC. The study started in April 2017 and the first interim results are expected in August 2020. Results of the study will be presented at (inter)national congresses and published in peer-reviewed medical journals.
It has proven to be difficult for GPs to recognize which patients with musculoskeletal complaints are at risk of an IA  . On the one hand, patients at risk of IA require early recognition and early diagnosis for an optimal response of IA treatment to eventually increase quality of life. On the other hand, for patients not at risk of IA there is no need for care by a rheumatologist. In case patients with IA are missed or patients without IA are referred towards the rheumatologist, this leads to more costs over the entire care cycle.
Integrated care may be a solution to this problem. Previous studies have suggested that the triage by a rheumatologist or the algorithm referral strategy can improve appropriateness of referrals, improved patient satisfaction and considerable cost reductions    . However, the impact of implementation of a referral algorithm has not yet been investigated within rheumatology and for both strategies evidence of cost-effectiveness is lacking so far.
Schulpen et al. did investigate the implementation of triage by a rheumatologist in a primary care setting in the Netherlands . They offered joint consultation, in which a consulting rheumatologist examined the patients in primary care and formulated a diagnostic and therapeutic policy through close collaboration with the GP. By the end of the study period, the number of patients referred by each participating GP differed substantially from the number of patients referred by matched non-participating GPs (3.7 vs 9.7 patients/GP/year respectively). Joint consultation rheumatology led to a decrease of 62% in the number of referrals. On top of that, there was also a large decrease in the number of follow up consultations after joint consultation. Together they account for a more “time effective” approach compared to usual care. Nevertheless, Schulpen et al. emphasize the need to determine cost-effectiveness for triage as a referral strategy .
For electronic structured sheets as a referral strategy, many sets of criteria have been developed. Unfortunately, a lot of them lack validation in primary care and information on implementation and impact has not been obtained. For example in the study of Moens and van der Korst , in which an electronic structured sheet was developed for rheumatologic conditions in the Netherlands. However, this strategy has only been validated in secondary care, and information on implementation is lacking.
Akbari et al. performed a review on articles discussing interventions to improve outpatient referrals from primary care to secondary care, including structured referral sheets . Several studies evaluated the use of structured referral sheets and observed improved management of patients . For example in the study of Thomas et al. , the structured referral sheets were part of a more complex intervention which included re-organization to streamline the referral process towards secondary care. The results of the study suggest that this was successful; patients had a management decision, including diagnosis, more rapidly. However, Akbari et al. emphasize the need for further research, especially to obtain data on the process of integrated care, e.g. care across the primary-secondary care interface .
The main strength of the current study is that it provides information on the value of several referral strategies by capturing both process level and patient relevant outcomes in relation to direct and indirect costs. The combination of those outcomes allows for an overall interpretation of findings on the process of care across the primary-secondary care interface. Furthermore, this study measures the impact of referral strategies that have already shown great potential .
The current study follows a high quality design that supports strong clinical evidence. Although a randomized study design is preferred to create comparable study groups and control for all factors with equal distribution of potential confounders, the current study design was accepted after careful consideration. It is believed that the indication for referral of a patient suspected of IA should not be influenced by the location. Therefore, location has no impact on the primary outcome of percentage of IA.
A weakness of this study is that the strategic interventions, especially the triage by a rheumatologist in a primary care setting, may lead to a change in referral behavior of the GPs . Since GPs are aware of the fact that a rheumatologist will visit their general practice once every three weeks, the threshold to refer patients to the consultant clinic in a community setting might be lowered. A result of this may be an induction of care. On the other hand, GPs may choose for convenience and avoid the extra effort of referring a patient to the consultant clinic, and instead referring them directly towards secondary care. This may lead to an underrepresentation of patients in the study .
Not only the triage strategy raises some concerns, the structured referral sheets do as well. What is known, is that GPs have very crowded consultation hours and relatively little time to see each patient. Although the use of the referral sheets only costs two minutes per patient, this might be too much for GPs to fit it in their daily practice. Another concern of the use of electronic structured referral sheets is that the screening effect can cause that a few true IA cases might be missed or referred later due to false negative results. To ensure quality and equality of care, GPs will always be able to refer a patient to the rheumatology outpatient clinic at the GPs discretion even if the referral algorithm gives a negative referral advice.
An effect in process or in costs does not necessarily result in improved patient outcomes. Vice versa, possible absence of effect in patient outcomes, may be the result of insufficient improvement in the process or a follow-up that is too short in duration. If this study demonstrates improvements in health outcomes and cost-efficiency, there is sufficient supporting evidence to implement one of the referral strategies as a standard of care. Finally, with these optimization strategies a higher quality of care can be achieved, which might be of value for all patients with arthralgia.
Ethics Approval and Consent to Participate
A certification that this study will not be subject to the full extent of the Medical Research Involving Human Subjects Act (WMO) was obtained from the local medical ethics committee (Toetsingscommissie Wetenschappelijk Onderzoek Rotterdam) of the Maasstad Hospital in Rotterdam the Netherlands in October 2016. All patients will provide written informed consent after receiving full information on participating in this study and given the opportunity to ask questions. A copy of the translated consent form can be found in the appendix.
Availability of Data and Material
The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.
This study is sponsored by the department of rheumatology from the Maasstad Hospital, Rotterdam. They have had no other role in this study. This study has not received external funding and the study protocol has been sent for peer-review with a member of our Editorial Board. A certification that this study complies with the Dutch law on Medical Research in Humans was obtained from the national recognized medical ethics committee (TWOR) in October 2016. This study has not received funding or assistance from a commercial organization.
JR, AW and DLB conceptualised and designed the study, including writing the protocol with advice from all other authors. AH made substantial contributions to the design of intervention group 1, and KH and IT made substantial contributions to the design of intervention group 2. ED wrote the first draft of this manuscript, whereas AW, DLB and JH substantively revised the manuscript. All authors have read, edited and approved the final manuscript.
Table A1. Schedule of enrolment, interventions and assessments of the JOINT referral study.
Model consent form (translated from Dutch)
Strategies to improve adequate referral towards the rheumatologist: JOINT referral study.
- I have read the information. I was also able to ask questions. My questions have been answered sufficiently. I have had sufficient time to decide on participating.
- I know that participation is voluntary. I also know that I can decide at any moment to no longer participate in this study. I do not need to give a reason for that.
- I give my consent to inform my general practitioner that I participate in this study.
- I know that some people can look into my data. Those people are mentioned in the information.
- I give my consent to collect and use my data in the way and for the purposes mentioned in this information.
- I give my consent to store my data on the research site for 15 years after termination of this study.
- I want to participate in this study.
Signature: Date: __ / __ / __
I declare that I have fully informed this participant about the mentioned study.
If during the study information gets out that may influence the consent of the participant, I will timely inform him or her.
Name researcher (or representative):
Signature: Date: __ / __ / __
The participant receives the complete information, together with a copy of this signed consent form.
List of Abbreviations
Musculoskeletal complaints (MSC)
Inflammatory arthritis (IA)
Rheumatoid arthritis (RA)
Axial spondyloarthritis (axSpA)
Psoriatic arthritis (PsA)
Value based health care (VBHC)
General practitioner (GP)
EuroQol health questionnaire with 5 dimensions (EQ-5D)
Institute for Medical Technology Assessment (iMTA)
iMTA Medical Consumption Questionnaire (iMCQ)
iMTA Productivity Cost Questionnaire (iPCQ)
Incremental Cost Effectiveness Ratio (ICER)
 Murray, C.J., et al. (2015) Global, Regional, and National Disability-Adjusted Life Years (DALYs) for 306 Diseases and Injuries and Healthy Life Expectancy (HALE) for 188 Countries, 1990-2013: Quantifying the Epidemiological Transition. The Lancet, 386, 2145-2191.
 Gamez-Nava, J.I., et al. (1998) Referral and Diagnosis of Common Rheumatic Diseases by Primary Care Physicians. British Journal of Rheumatology, 37, 1215-1219.
 Kallikorm, R. and Tender, M. (2004) Rheumatic Patients in Primary and Secondary Care: Differences in Structure of Diagnoses and Working Load within a Five-Year Period. Croatian Medical Journal, 45, 588-591.
 Akbari, A., et al. (2008) Interventions to Improve Outpatient Referrals from Primary Care to Secondary Care. Cochrane Database of Systematic Reviews, CD005471.
 Gruen, R.L., et al. (2004) Specialist Outreach Clinics in Primary Care and Rural Hospital Settings. Cochrane Database of Systematic Reviews, CD003798.
 van Hoeven, L., et al. (2015) External Validation of a Referral Rule for Axial Spondyloarthritis in Primary Care Patients with Chronic Low Back Pain. PLoS ONE, 10, e0131963.
 Ibrahim, G.H., Buch, M.H., Lwson, C., Waxman, R. and Helliwel, P.S. (2009) Evaluation of an Existing Screening Tool for Psoriatic Arthritis in People with Psoriasis and the Development of a New Instrument: The Psoriasis Epidemiology Screening Tool (PEST) Questionnaire. Clinical and Experimental Rheumatology, 27, 469-474.
 ten Brinck, R.B., van Dijk, B.T., van Steenbergen, H.W., le Cessie, S., Numans, M.E., Hider, S.L., Mallen, C. and van der Helm-van Mil, A. (2018) Development and Validation of a Clinical Rule for Recognition of Early Inflammatory Arthritis. BMJ Open, 8, e023552.
 Herdman, M., Gudex, C., Lloyd, A., Janssen, M., Kind, P., Parkin, D., Bonsel, G. and Badia, X. (2011) Development and Preliminary Testing of the New Five-Level Version of EQ-5D (EQ-5D-5L). Quality of Life Research, 20, 1727-1736.
 Bouwmans, C., Koopmanschap, M., Krol, M., Severens, H. and Brouwer, W. (2013) HandleidingiMTA Medical Cost Questionnaire (iMCQ). iMTA, Erasmus University, Rotterdam.
 Bouwmans, C., Koopmanschap, M., Krol, M., Severens, H. and Brouwer, W. (2013) HandleidingiMTA Productivity Cost Questionnaire (iPCQ). iMTA, Erasmus University, Rotterdam.
 Schulpen, G.J.C., Vierhout, W.P.M., van der Heijde, D.M., Landewé, R.B., Winkens, R.A.G. and van der Linden, S. (2003) Joint Consultation of General Practitioner and Rheumatologist: Does It Matter? Annals of the Rheumatic Diseases, 62, 159-161.
 Moens, H.J.B. and van der Korst, J.K. (1992) Development and Validation of a Computer Program Using Bayes’s Theorem to Support Diagnosis of Rheumatic Disorders. Annals of the Rheumatic Diseases, 51, 266-271.
 Thomas, R., Grimshaw, J. and McClinton, S. (2003) Cluster Randomized Trial of a Guideline-Based Open Access Urological Investigation Service. Family Practice, 20, 646-654.