Patient reported outcomes measures (PROMs) can be applied in diverse settings to guide the choice of healthcare interventions. It does so not only by providing data on the clinical outcomes with a health technology or the disease course, but because it can also be combined with cost data in economic evaluations to determine the cost-effectiveness thereof. Measuring provider performance and quality of services using PROMs, the use of health technology assessment (HTA) and economic evaluations will play an increasingly important role in South African private healthcare provision and national public health policy decisions.^1,2,3,4 Specifically, under National Health Insurance (NHI), economic evaluations as part of a more formal HTA programme could be used to determine the cost-effectiveness of treatments provided as part of the NHI fund. On this point the 2019 NHI Bill stated that:

Such cost-effective evaluations frequently use health-related quality of life (HRQoL) data in cost-utility analyses (CUAs) because it allows for comparisons across health situations and programmes. These analyses can subsequently be employed for making resource allocation decisions. Indeed, this is the intention of the 2021 draft HTA guideline issued for comment by the National Department of Health as part of the Essential Drugs Programme’s appraisal of medicine informing their inclusion on to the National Essential Medicines List (NEML)⁵: the guideline proposes that a CUA be conducted if a comprehensive cost-effectiveness evaluation is required.

However, not all HRQoL data are suitable for CUAs. Data from HRQoL instruments should be generated by a PROM that is a generic instrument because it assesses the treatment outcomes across a range of populations and interventions. Such instruments should also measure attributes across multiple health domains, have a health state classification system based on the combination of responses and the health states should have associated numeric values that represent the patient or general public preference for each health state generated by the instrument.⁶ For this reason, they are also called multi-attribute utility instruments (MAUIs) or preference-based measures. It is possible to convert the results from some non-MAUIs/preference-based measures into generic HRQoL data that can be used in CUAs, but this should be avoided if possible because of the limitations thereof.^6,7

Whether for clinical decisions, economic evaluation or health policy decisions, it is important that PROMs are reliable and valid in the target population. But this can be influenced by instrument language (amongst others): if English language measurements are used in non-English speaking populations, their understanding of the questions and responses given may not correspond to the intended concepts.⁸ The importance of adequate translations into South African languages is emphasised by the findings of studies that evaluated the validity, reliability and cultural adaptation of translated HRQoL instruments. Researchers found that reliability and validity were influenced by socioeconomic factors such as education, literacy and rural or urban living, which were often associated with populations’ cultural background and historical racial inequalities.^9,10,11,12 In addition, understanding could be impacted when no equivalent word existed in the South African language,¹³ or because of difficulty in transferring English concepts into African culture.^11,14 Consequently, the issues identified required the researchers to make semantic and conceptual changes to the instruments. Changes in delivery of some instruments such as an oral explanation of the nature of the instrument and questions or requesting permission to ask questions considered sensitive in a particular cultural group were also necessary to alleviate participants’ discomfort in completing the questionnaires. Therefore, by using an inadequate translation, the interpretation of the results and conclusions generated by it may be limited. Moreover, inadequately translated instruments impact a study’s generalisability to the wider population. This in turn may cause uncertainty when such results are needed to inform the priority setting process, thereby potentially delaying patients’ access to new technologies.¹⁵ Therefore, to optimise patient outcomes and support patients’ access to technologies, HRQoL instruments should be administered in the target population’s home language(s). If such a language version is not available, they should be translated according to good translation principles as this will give greater certainty in the results obtained. Unsurprisingly, the South African Guidelines for Pharmacoeconomic Submissions (SAGPS), point out that all HRQoL instruments should be validated using South African data and request detailed information supporting the validity of the tool in the South African context.¹⁶ Similarly, the draft HTA guideline for the NEML proposes that HRQoL for a CUA should be measured from a representative sample of the South African population using a validated instrument, and the effects should be valued with a South African-based value set⁵ (a value set is a single score based on the South African general population’s preference for different health states possible with a MAUI). However, whilst a systematic review found that a body of HRQoL data exists in South Africa, details on the instrument language versions used or the translation methods employed were reported infrequently.¹⁷

This study therefore sought to (1) provide a list of HRQoL instruments suitable for CUAs that have been used in South Africa using one or more South African languages (other than English), (2) critique the methods used to produce the translated versions, and (3) make recommendations on which instruments may be suitable for future HRQoL measurement within the context of conducting CUAs in South Africa as part of national HTA.

To the best of the authors’ knowledge, this is the first review and assessment of the translation quality of HRQoL instruments used in South Africa. It identified several HRQoL instruments suitable for CUAs that have been used in South Africa (Table 2). However, few have been translated for use in the South African population according to good practice guidelines (Table 3). Encouragingly, where reported, most studies used a language version in one of the three most spoken home languages, namely isiXhosa, Afrikaans and isiZulu, which together represents just over 50% of the population.⁵⁶ Of the studies included in this review, only the instrument developers’ language versions of the EQ-5D-3L and -5L, SF-36 version 2 and SF-12 fully met the translation criteria used in the analysis. Of these, the EQ-5D-3L is currently the best placed of the existing HRQoL instruments suitable for CUAs for use in South Africa. This is based on the following factors: (1) It allows use and comparisons of results across multiple diseases (and is therefore preferred by many HTA agencies supporting healthcare priority setting and funding decisions⁶), (2) It was shown in this study to meet the standards for translation of PROMS, (3) It was found to have the most translations available of those studies included in the current analysis and (4) It has been used in a wide range of settings, populations and diseases in South Africa.¹⁷ The SF-36 and the EORTC QLQ-C30 were also frequently used but the EORTC QLQ-C30 did not meet the review’s translation criteria.

Within the context of using the studies for clinical and priority setting decisions in a multi-cultural country, it was concerning that only 39 out of the initial 144 studies included (27.1%) reported the use of translated instrument versions. However, this is higher than that reported by Bello et al.⁵⁷ Only 14.0% of the studies in their systematic review on the properties of stroke quality of life outcomes measures in Africa used translated instruments. The remainder of the findings is consistent with the reported literature. A 2003 systematic review of the translation and adaptation of generic HRQoL measures in Africa, Asia, Eastern Europe, the Middle East and South America highlighted that in the 1990s several publications raised concern about the quality of translated versions of HRQoL instruments, focusing on the quality of the methodology.⁵⁸ They found very few studies that reported that the instruments were assessed for equivalence during translation and none considered it in any detail. The authors concluded that more work needs to be performed to improve translations. Yet despite several decades of experience and numerous translation and adaptation frameworks, guidelines and good practice principles^59,60,61 problems remain with instrument translation into non-English languages. For example, more recently Al Sayah et al.⁶² also observed that reporting details were lacking in Arabic translations and cross-cultural adaptations of HRQoL instruments. Consequently, the authors could not evaluate the translation quality of most of the instruments identified. Reporting details were also lacking for Brazilian Portuguese generic and cancer specific PROMs, and most problems were with the forward translation, back translation and expert review steps.⁶³ This study’s finding that there was a lack of a pilot and cognitive debriefing step by half of investigators who created their own translations is also not unique: a systematic review of childhood HRQoL measures in sub-Saharan Africa found only two instruments (out of 10 identified) that attempted to establish cross-cultural adaptation through linguistic and conceptual equivalence testing.⁶⁴ Although not explored in these analyses, it is worth reflecting on the possible reasons for the lack of improvement in translation quality. For instance, the importance of high-quality translations and impact of poor translations and adaptations of instruments outside of these countries may not receive the necessary attention in the literature read by researchers and clinicians interested in HRQoL. Certainly, HRQoL research between 2000 and 2019 were predominantly conducted in North America and Europe, and the studies were published in English language journals⁶⁵ (which are owned by publishing companies based in these regions). Furthermore, researchers in low- and middle-income countries may also not have the resources available to conduct translations according to the guidelines and the guidelines themselves may not account for the complexities. These last two points were identified and discussed in detail by De Wet et al. in a case study of their study’s research documents that were translated into isiXhosa,⁶⁶ and were also highlighted in some of the publications included in this review.^11,13

Finally, the studies included in this review were mainly in one disease area (HIV). The 2019 review also found a strong focus on HIV and showed that HRQoL data in chronic conditions that contributed the most to the country’s burden of disease, were lacking.¹⁷ Given the existing requirement in the SAGPS and the draft NEML HTA guidelines to use a South African validated HRQoL instrument for economic evaluations and the gaps identified in this and 2019 review, further work may be needed if CUA has to form the basis of HTA under NHI. For example, if the EQ-5D were to be used for HTA under NHI, a South African value set will be needed. Until such a value set is created, one from another country must be used, which increases uncertainty in CUA results and limits health policy decisions.

Measuring patient outcomes is the basis for decisions about the most appropriate treatment for a patient, tracking healthcare quality, evaluating service performance, and priority setting such as funding of cost-effective treatments. Consequently, this work has several implications for clinical practice, health policy and HRQoL research. Without using HRQoL measures in the target patient and population’s native language or instruments translated using sound methodologies, it cannot be certain that the instruments accurately evaluate the patients’ views about their health and subsequent health outcomes. This brings into question the validity of the results obtained and compromises the instrument’s use in clinical practice and for health policy decisions. Using poorly translated instruments also contribute to wasteful and inefficient resources use when time and money are spent on administering an instrument that may produce unreliable results. This review therefore makes several recommendations based on the study’s findings and existing literature (Table 4) specific to HRQoL measurement in any multi-cultural setting.

For South Africa, where national HTA and economic evaluations are going to play a more important role in sustainable patient access to health technologies, both patients and funders would benefit from incorporating the recommendations presented in Table 4. This is because providing high quality HRQoL data for new technologies could get patients faster access to new technologies, whilst both private and public funders will have more certainty that the introduction of technologies is proven value for money.

The analysis was based on a comprehensive literature review, the foundation of which was a 2019 systematic review. However, only one researcher was involved in extracting the translation methods from the publications, which may have resulted in errors. To mitigate this, the extraction form used in Excel^® served as a checklist, and the articles were re-evaluated on more than three separate occasions. It is acknowledged that various translation frameworks exist and that the choice of Guillemin et al. and Wild et al. may be considered arbitrary. However, evidence is lacking on the best methods for translation and cross-cultural adaptation and most translation methods recommended by guidelines would achieve comparable results.⁵⁹ It is therefore unlikely that the results of this review would have been much different if other translation criteria were used. In addition, the use of publications to measure the quality of research is limited by the information provided in the article and it is therefore possible that the methodology in the studies did follow recommended translation methods but was simply poorly reported. Lastly, the analysis focussed on HRQoL instruments suitable for CUAs, thus it is not a comprehensive review of all PROMs used in South Africa.

The EQ-5D-3L may be best suited for use in South Africa where data are needed for a CUA. However, further work and detailed guidance from the National Department of Health on the most suitable HRQoL instrument for the South African context will be needed if the CUAs will be required as part of the HTA process under NHI. Acting upon the recommendations from this study could result in more robust measurement of HRQoL in South Africa and more informed decisions on the introduction and use of health technologies from both a patient and national healthcare policy standpoint. The recommendations on translation methodology quality are also relevant to clinicians wanting to obtain reliable health outcomes data from the patients’ perspective.