Patient-Reported Outcomes (Pros) in Surgical Humanitarian Missions to Low- And Middle-Income Countries: A Scoping Review

Research Article

Patient-Reported Outcomes (Pros) in Surgical Humanitarian Missions to Low- And Middle-Income Countries: A Scoping Review

Cheng H¹, Webster F^2,3, Platt J¹,Causarano N⁴, Baxter NN^3,5,6and Zhong T^1,4*
¹Department of Surgery, University of Toronto, Canada
²Departments of Family and Community Medicine, University of Toronto, Canada
³Institute of Health Policy, Management, and Evaluation, University of Toronto, Canada
⁴Departments of Surgery and Surgical Oncology, University Health Network, Canada
⁵Department of Surgery and Keenan Research Centre, Li Ka Shing Knowledge Institute, Canada
⁶Institute for Clinical Evaluative Studies (ICES), Canada

^*Corresponding author: Toni Zhong, University Health Network, 8NU – 871, 200 Elizabeth Street, Toronto, Ontario, M5G 2C4, Canada

Published: 16 Jun 2016
Cite this article as: Cheng H, Webster F, Platt J, Causarano N, Baxter NN, Zhong T. PatientReported Outcomes (Pros) in Surgical Humanitarian Missions to Low- And Middle-Income Countries: A Scoping Review. Clin Surg. 2016; 1: 1032.

Abstract

Background:Every year, health care personnel from developed nations travel to low- and middleincome countries (LMICs) to provide free surgical care to the local population. While the number of procedures performed by surgical humanitarian missions continues to increase, their impact on patients and their communities is not fully known. In particular, patient-reported outcome (PRO) reporting is lacking in surgical humanitarian missions.

Methods:A scoping review was conducted on PROs in surgical humanitarian mission to summarize the existing body of knowledge and identify gaps. The findings were presented at a plenary meeting of the Surgical Missions Outcomes Working Group.

Results:There were 43 articles in the literature that reported PROs from surgical humanitarian missions. The majority of studies (81%) were retrospective, and 12% were prospective. Sample sizes ranged from 15 to 18,653 patients. Short-term outcomes were reported in 62% of studies, intermediate outcomes in 41%, and long-term outcomes in 18%. Complication rates (81%) and performance indicators (72%), known as process indicators, were the most common outcome measures reported. Only four studies (9%) collected PROs and in none of the studies were the patient-reported outcome measures (PROMs) validated or pre-tested. At the plenary meeting, working group members reached a consensus that systematic reporting of all relevant mission outcomes and PROs in particular, is currently lacking and requires further research.

Conclusions:There is a paucity of literature on PROs for surgical humanitarian missions, with studies reporting mostly process indicators and lacking in long-term follow-up. More research is critically needed to validate existing PROMs or create new ones. Stakeholders involved in surgical humanitarian missions are urged to become aware of the multi-dimensional impact of their activities on patients and communities.

Background

Global Impact of Surgical Humanitarian Missions
Governments, non-governmental organizations (NGOs), and donors have worked for years to improve essential surgical care to low- and middle-income countries (LMICs) by deploying individual and teams of volunteers on short-term missions to provide surgical care in communities where health infrastructure is lacking [1]. The extent and impact of the health services provided by the uncounted surgical humanitarian missions around the world is unknown. In 2011, the International Committee of the Red Cross and Médecins Sans Frontières together performed 211,300 surgeries [2,3]. A survey conducted by the American College of Surgeons, Operation Giving Back, and the Harvard Humanitarian Initiative found that NGOs play a substantial role in the delivery of surgical services in LMICs, with some providing over 50,000 procedures annually [1]. While the number of procedures performed by surgical humanitarian missions continues to rise, their impact on patients and local communities has not been adequately assessed [4]. Standardized data reporting by surgical
missions is essential to determine their impact on the health of local patients and communities.

Table 1

Table 1
Results of treatment and follow-up.

Measuring the success of surgical humanitarian missions

The disaster response following the 2004 South Asian tsunami and 2010 Haitian earthquake raised international concerns about the practices of some foreign medical teams. In response, the United Nations’ Inter-Agency Standing Committee called for “greater accountability, coordination, and more stringent oversight and documentation of their work” [5]. As early as 2004, Roberts et al. [6] reviewed 15 surgical mission publications and found that only four reported health outcomes. In 2012, a systematic review concluded that surgical missions commonly failed to meet data collection and reporting standards required by national and international health authorities [7]. Missions commonly aimed to measure health outcomes but instead collected performance indicators, which measure the process of implementation (i.e. geographical area reached) and mission outputs (i.e. number of procedures performed). Performance indicators are not proxies for actual health benefits. This
deficiency was similarly identified by a study from the United States Department of Defense, which found only seven reports of health outcomes of the total 178 mission impact reports [8].

PROMs in surgical humanitarian missions

Across health care, stakeholders (including the general public, insurance companies and government) increasingly expects health care providers and institutions to report on health-related outcomes and make improvements where shortcomings are identified. The growing attention on quality improvement has helped drive the development of patient-reported outcome measures (PROMs) across health disciplines. For instance, the Center for Medicaid and Medicare Services and other agencies have historically used process measures or adverse events for quality assessment. However, the use of patient reported outcomes (PROs) has become a priority based on stated goals by the National Quality Strategy and Institute of Medicine for a more patient-centered approach [9]. PROs are increasingly required as part of health care standards and reporting in the United States. For example, the Food and Drug Administration now mandates
the use PROs to support medical product labeling claims, the US government’s Affordable Care Act has created a Patient-Centered Outcomes Research Institute, and the NIH has created the Patient Reported Outcomes Measurement Information System (PROMIS) initiative [10].

There remains considerable discussion regarding how to measure the quality of surgical care delivered by numerous humanitarian missions in communities around the world. To date, the majority of quality improvement initiatives in surgical missions have focused on measuring the frequency of adverse events such as peri-operative morbidity and mortality [11]. Adverse evente reporting is important when evaluating the outcome of life- or limb-saving surgeries; but for low-risk elective surgeries where the goal is to improve quality of life, the adverse event rate is less relevant and does not provide information about the patient's quality of life. For example, reporting low perioperative mortality following burn contracture release is expected but does not indicate if patients actually benefited from surgery. More meaningful outcomes in this case might include the ability of patients to independently perform daily activities and satisfaction with appearance. As this example demonstrates, PROs are better indicators of quality when the goal of surgery is to improve patient health-related quality of life. Their implementation would provide much-needed transparency and accountability, highly valuable data for policy making and fundraising, and stimulate improvements in patient care.

Methods

As described by Arksey and O’Mally, a scoping review is a ‘technique to map relevant literature in a field of interest’ [12]. In contrast with a systematic review, which more narrowly focuses on a particular question or study design, the scoping review tackles ‘broader topics where many different study designs might be applicable’ [12]. For this reason, the scoping review methodology is well-suited for surveying surgical humanitarian missions, which vary widely in content, study methodology, and outcome measures. We conducted a scoping review to identify existing literature reporting outcomes of surgical humanitarian mission. The keywords “surgical mission”, “medical mission”, “surgery”, and “developing countries” were used to search the Medline, Cochrane Library, and PsycINFO databases. After removing duplicates, the remaining 1645 abstracts were hand searched by three researchers (HC, JP, TZ) to select English-language articles from civilian surgical missions that reported outcome measures. We broadly defined ‘outcome measure’ as any data generated from mission activities, including performance indicators (e.g. number of surgeries performed), quality indicators (e.g. operative-site infections), and PROs (e.g. patient surveys). Only humanitarian missions providing elective procedures were included in this review. Disagreements in study inclusion were resolved through discussion and consensus amongst all members of the research team. Full-text articles were reviewed and reference lists of all articles meeting study inclusion criteria were searched for additional relevant articles.

The final step of our scoping review was to present and discuss our findings with a committee of experts. Stakeholders from the surgical humanitarian field, including surgeons, researchers, public health experts, NGO leaders, and qualitative researchers, were invited to form the Surgical Missions Outcomes Working Group working group (Table 1). The working group met at a Canadian Institutes of Health Research funded plenary meeting that was facilitated by an
independent moderator. The goals of the meeting were to: 1) review and contextualize the findings of the modified scoping review, and 2) identify emerging issues, research questions, and study methodologies that will be the basis for future research protocols.

Figure 1

Figure 1
Results of the modified scoping review process.

Results

Our modified scoping review yielded 43 unique articles published in 31 journals from 1990 to 2012 (Figure 1). The most common elective surgeries were cleft lip and palate reconstruction (20%), cataract correction (17%), and cardiac surgery (12%). The majority (81%) of studies were retrospective and 12% were prospective. Sample sizes ranged from 15 patients treated on two surgical missions to 18,653 patients treated by one humanitarian organization in 13 LMICs over an 8-year period [13,14]. Short-term outcomes (collected during the mission) were reported in 62% of studies, intermediate outcomes (collected less than one year from surgery) in 41% and long-term outcomes (collected one year or longer after surgery) in only 18%. Of the outcome measures reported, complication rates (81%) and performance indicators (72%) were the most common. Objective outcome measures (e.g. audiometry, visual acuity, facial measurements) were used in 40% of studies. Only four of the 43 articles (9%) collected PROMs (summarized in Table 2), with none using any validated PROM measures.

Table 2

Long-term studies

Article	Study design	Sample size	Response rate	Surgical treatment	PRO	Follow-up duration
Egbert et al. [15]	Case series	77	49 (64%)	Extracapsular cataract excision and posterior chamber intraocular lens implantation	Structured interview – closed-ended questions about patient ability to perform specific activities without assistance (i.e. read, walk)	12-29 months
Short-term studies
Lehnerdt et al. [16]	Case series	38	31 (82%)	Ear surgery	Questionnaire – 2 questions about improvement in hearing and ear pain	Minimum 6 months
Roux et al. [17]	Case series	52	50 (96%)	Fronto-ethmoidal MEC* reconstruction	Social impact questionnaire – closed-ended questions about aspects of social function	Minimum 6 months
Sharp et al. [18]	Retrospective cohort study	N/A	54	Cleft lip and/or palate repair	Structured interview – open-ended questions about outcomes of oral cleft repair	Minimum 6 months

Table 2: Summary of surgical humanitarian articles utilizing patient-reported outcomes (PROs)

Table 2
Summary of surgical humanitarian articles utilizing patient-reported outcomes (PROs)

1) Egbert et al. [15] reported a case series of 77 patients in Ghana who received outpatient cataract surgery. Forty-nine patients (64%) returned for a post-operative eye examination and structured interview between 12 and 29 months after surgery. The interview consisted of closed-ended questions about activities of daily living (e.g. reading, walking without assistance) that were created ad hoc. Patient responses were tabulated and no further data analysis was performed.

2) Lehnerdt et al. [16] presented a case series of 38 pediatric ear surgeries performed in northern Namibia. The surgical team returned six months after surgery and 31 patients (82%) received follow-up examinations and an ad hoc questionnaire. Performance indicators (e.g. types of tympanoplasties performed) and objective outcome measures (e.g. pure tone audiometry) were reported. In addition, patients were asked if their hearing had improved and if they still had ear pain.

3) Roux et al. [17] published a case series of 52 patients in Cambodia who underwent fronto-ethmoidal meningoencephalocele (MEC) repair. Four criteria were used to evaluate the surgical outcomes: 1) Patient and parent ratings of the cosmetic outcomes; 2) Surgeon rating of cosmetic outcomes; 3) Pre- and post-operative facial measurements; 4) Social impact of surgery as measured using a questionnaire administered pre- and post-operatively. Fifty patients (96%) were followed-up at least six months after surgery. The primary outcome measures used in this study (e.g. rating scale and social impact questionnaire) were developed ad hoc. Patient and surgeon ratings were tabulated and there was no further data analysis.

4) Sharp et al. [18] conducted a retrospective cohort studyin Cebu, Philippines to test the feasibility of conducting outcomes research during a surgical mission and to identify the salient outcome measures for cleft lip and/or palate patients and caregivers. Structured interviews (n=75) were conducted with patients and caregivers in the local dialect at least six months after surgery. Researchers asked three open-ended questions, and responses were analyzed using qualitative coding techniques.

Discussion

Although the profiled studies utilized PROMs, each has critical methodological limitations. For example, none of the studies identified specific measurable program goals, so despite using PROMs, it was difficult to assess whether the mission met its objectives. In the study by Lehnerdt et al. [16] the relationship between mission goals and
outcomes was not clear. Children were selected to receive surgery because they had ear pathologies that prevented the use of hearing aids, but the ability to use hearing aids after surgery was not one of the outcomes measured. Follow-up was limited to a single visit six months following surgery in three studies, and one study had a single follow-up visit 12 to 29 months after surgery. Although we recognize the immense challenge of coordinating follow-up in mission settings, patients should ideally be reassessed at multiple time points over longer periods of time. All four studies conducted interviews or questionnaires that were developed ad hoc by the research team, with no mention of having sought input from stakeholders, or pretesting or validating the tool. With the exception of the questionnaire described by Sharp et al. [18] researchers asked closed-ended questions about specific symptoms and activities that cannot be assumed to correlate with functional domains or HR-QOL. Roux et al. [17] asked patients “Do you go to school?” and “Are you happy at school”, but school attendance in Cambodia can be influenced by other factors not related to the outcome of MEC reconstruction. Questionnaire responses were also simply tabulated, and without further data analysis, it is difficult to interpret the study findings and draw conclusions.

Working Group Consensus / Recommendations

All working group participants agreed that more systematic reporting of relevant PROs was needed in order to understand the impact of surgical humanitarian missions, and that current knowledge was insufficient to develop the needed outcome measures. Key recommendations from the plenary meeting are summarized below.

Establishing appropriate outcome measures: Surgical missions must identify the salient health outcomes for the communities they wish to serve and the appropriate tools to evaluate them. A number of validated HR-QOL measures already exist, and their ability to measure surgical mission outcomes must be systematically evaluated. Alternately, researchers may develop and validate novel tools specifically for humanitarian missions. Researchers must also determine whether a single tool can be used for different countries and cultures and for specific mission objectives.

Study methodology: The working group proposed a mixed methods approach to identifying key PROs and developing a pilot measurement tool. Qualitative techniques would be used initially to explore patient experiences, establish definitions, and generate hypothesises for quantitative testing. Quantitative methods would then ben needed to validate existing or newly-developed patient-
reported outcomes measures. Researchers must collaborate with local health workers and other stakeholders to overcome language barriers and ensure their work has contextual relevance. Health research conducted in this setting must be sensitive to patient culture, social values, and safety (both physical and psychological).

Role of telecommunication technologies: Patient recruitment and follow-up remains a significant challenge in conducting research in LMICs. However, even patients living in remote areas can be reached through local partners and existing telecommunication technologies. Cellular service is nearly ubiquitous in some LMICs and can be used to facilitate patient follow-up. New mobile technologies allow followup to be conducted using smart phone applications. Additional precautions must also be taken to safeguard the confidentiality of patient health information when using these technologies.

Conclusions

This is the first report to address the knowledge gap in PROs in surgical humanitarian missions. Research is needed to create valid and reliable PROMs, and also to implement them in a systematic, sustainable, and cost-efficient manner. We propose a rational research program developed through methodical review of current literature and consultation with key stakeholders and experts in the field. Our goal is to stimulate research interest and collaboration, and to urge all healthcare personnel involved in humanitarian missions to become more aware of the multi-dimensional impact of their activities on patients and their communities. Lessons learned from pre-testing, measuring and validating PROMs during surgical humanitarian missions to LMICs promises to advance the quality of care in nations across all incomes.

Acid Survivors Foundation, Bangladesh	AJM Salek, Consultant plastic surgeon and Chair of the Medical Advisory Board	Arizona State University, United States	History of constipation	Treatment	Relief time (days)	Times of obstruction
1	F	22	10 years	conservative	16	1
2	F	33	15 years	conservative	7	1
3	F	16	2 years	conservative	15	1
4	M	62	no	conservative	10	1
5	F	39	2 years	total colectomy	11	5
6	F	29	no	conservative	12	1
7	M	23	2 months	conservative	12	2
8	M	75	no	conservative	10	1
9	F	47	8 years	conservative	30	2
10	M	67	2 years	conservative	10	2
11	F	66	no	conservative	9	2
12	M	51	no	conservative	13	1
13	F	27	no	conservative	4	1
14	F	29	1 years	conservative	11	3
15	F	43	20 years	conservative	18	1
16	F	56	no	conservative	8	1
17	F	53	2 years	conservative	8	1
18	F	70	no	conservative	7	1
19	F	16	1 month	conservative	9	2
20	M	66	no	conservative	10	1
21	F	35	8 years	conservative	12	2
22	F	20	3 years	conservative	13	2
23	M	66	no	ileostomy	28	3

Research Article