Viral load testing in a resource-limited setting: quality control is critical
© Greig et al; licensee BioMed Central Ltd. 2011
Received: 16 August 2010
Accepted: 12 May 2011
Published: 12 May 2011
World Health Organization guidelines now recommend routine use of viral load testing, where available, for patients receiving antiretroviral treatment (ART). However, its use has not been routinely implemented in many resource-limited settings due to cost, availability and accessibility. Viral load testing is complex, making its application in resource-limited settings challenging. We describe the issues encountered by Médecins Sans Frontières (MSF) when using routine viral load testing in a large HIV programme in sub-Saharan Africa.
Between October 2005 and August 2006, more than 1200 patients on ART had viral load tests at baseline and at three-month intervals performed by a local reference laboratory that was quality assured by an experienced international institution. Concerns with reliability of results halted testing. The quality control measures instituted with a second laboratory and outcomes of these were documented.
In 2005 and 2006, only 178 of 334 (53%) previously ART-naïve patients tested after six to 12 months of treatment had viral loads of less than 1000 copies/mL. Similar MSF programmes elsewhere demonstrated virological suppression rates of more than 85%, and duplicate testing showed unacceptable discordance. Laboratory problems encountered included: disregarded quality control; time delays; requirement for retesting; and duplicate sample variations. Potentially harmful clinical outcomes of inaccurate viral load results include: unnecessary ART regimen changes; unnecessary enhanced adherence counselling after "false failures"; and undetected virological failure.
Viral load testing performed without rigorous quality control carries the risk of erroneous and potentially damaging results. Viral load testing should be utilized only if robust quality assurance has been implemented. Our experience in this and other settings led to the development of a guide for assessing the suitability of a laboratory for viral load testing that can be used to help achieve reliable results.
Viral load (VL) testing is the only definitive method for early detection of antiretroviral treatment (ART) failure [1, 2]. HIV treatment can be managed without routine laboratory assessment , but CD4 monitoring allows for evaluation of disease progression , and VL testing can increase adherence and facilitate timely switching of failing regimens, minimizing the development of resistance . VL testing has not been routinely implemented in resource-limited settings due to cost, complexity, availability and accessibility. Calls for widespread VL test use are increasing, and 2009 World Health Organization (WHO) guidelines recommend the routine use of VL testing where available [6, 7]. However, obtaining quality controlled and reproducible results for even simple laboratory tests in resource-limited settings is challenging [8, 9]. Inaccurate or delayed results can have serious consequences for patients and programmes.
Between October 2005 and August 2006, more than 1200 patients on ART had VL tests at baseline, and every three months thereafter. Samples were collected in EDTA tubes, centrifuged for 10 minutes at 800 × g and the plasma stored in sterile tubes at -20°C for less than eight weeks. Samples were measured at a local reference laboratory (Lab 1) using Roche Cobas Amplicor HIV-1 Monitor Test v1.5 and Cobas Amplicor Analyser (Roche Diagnostics, Basel, Switzerland), with preparation of frozen plasma in accordance with the Amplicor v1.5 manual guideline.
Prior to use, Lab 1 was assessed by an MSF laboratory scientist for: training of personnel; appropriate laboratory facilities; workflow; separation of areas for sample preparation, reagent preparation and sample analysis; backup power for laboratory freezers and refrigerators; temperature monitoring guidelines; cleaning guidelines for different laboratory work areas; and access to water of appropriate quality. The laboratory was found to be of a high standard, and was supported and quality assured by an international institution with experience in VL testing, with tests performed by trained staff already experienced with the procedures. No external quality control was organized by MSF.
Due to concerns regarding the validity of results, sample collection and preparation procedures were reviewed and reinforced, and additional samples were tested as blinded duplicates: 21 duplicates were sent to Lab 1 and another laboratory in-country (Lab 2, which was internationally supported, experienced in performing VL tests, and had good facilities found during assessment by an MSF laboratory scientist; VL testing was performed using Bayer Versant HIV-1 RNA 3.0 bDNA assay with Bayer System 340 bDNA Analyzer [Bayer Diagnostics, Tarrytown, NY, USA] with samples prepared according to the Versant guidelines). A further 29 duplicates were sent to Lab 1 and a reference laboratory in another sub-Saharan African country (Lab 3) that used Roche Amplicor. The differences between paired log VL results and the mean absolute differences for all samples were calculated, and the limits of agreement between laboratories compared using the method of Bland and Altman . The results of this quality control ended the first period of VL testing.
In mid 2007, the internationally supported Lab 2 was re-assessed by an MSF laboratory scientist, and external quality control of VL performance was undertaken with the lab's consent. Ten triplicate specimens, obtained with patient consent and prepared as previously, were sent to laboratories in Europe (Lab 4 and Lab 5) that used COBAS AmpliPrep/COBAS TaqMan HIV-1 Tests (Roche Diagnostics, Basel, Switzerland). This process involved approximately one week of staff time, cost around US$1500 for testing and sending frozen samples with dry ice and triple-packaging by courier, and incurred considerable difficulties in ensuring unbroken cold chain conditions.
Results were compared as for the earlier duplicate testing. After an acceptable outcome of this quality review, VL tests were performed on all patients after at least six months on ART. Specimens were labelled with patient name, ID code and date of collection. Samples were sent in large batches with both electronic and printed lists of these details, all of which were cross checked at the time of specimen storage and again when samples were shipped frozen on ice. Results were provided on these lists, both electronically and printed.
After consideration of the MSF Ethics Review Board framework criteria, this paper was assessed as not needing ethics approval. The work described was undertaken as part of routine programme work and analyzed retrospectively. Neither the programme nor individuals have been identified.
In 2005 and 2006, 178 of 334 (53%) previously ART-naïve patients tested by Lab 1 after six to 12 months of treatment had VLs of less than 1000 copies/mL. However, only 4% showed evidence of immunological failure according to 2006 WHO guidelines . Virological suppression rates of less than 1000 copies/mL in similar MSF programmes in other countries using standardized treatment protocols were higher than 85% [12, 13], and in a review of programmes in sub-Saharan Africa, were above 76% .
Results of duplicate sample tests in testing and external quality control laboratories
Lab 1 vs. Lab 2
Lab 1 vs. Lab 3
Lab 2 vs. Lab 4 vs. Lab 5*
Number of sample sets
Range of log VL results reported
N (%) of duplicate samples with difference in log VL result >1.0
Mean (range) absolute difference in log VL result
Limits of agreement of tests (log VL result)†
-4.777 to 0.801
-3.089 to 3.246
-0.796 to 0.848
False negatives: N (%) VL undetectable in testing lab but >1000 copies/mL in QC laboratory
[same comparison but >5000 copies/mL]
False positives: N (%) >1000 copies/mL in testing lab but undetectable in QC laboratory
[same comparison but >5000 copies/mL]
In 2007, VL testing recommenced in order to review patient status and reassess programme virological suppression rates. Prior to commencement of testing, acceptable external quality control results were obtained for Lab 2 (Table 1) and agreement reached that internal quality control results for each test run would be provided (calibration curve and positive controls).
Results from the first eight test runs (614 samples) performed by Lab 2 in 2007 were released to the programme as valid, but quality control data to verify this were not released concurrently. MSF received VL results from the test runs on average 27 days (range 5-69) after sample receipt by Lab 2, but the internal quality control data were received a further 34 days (range 25-42) later, despite repeated requests. Reasons for VL result delays included laboratory stock shortages caused by delays in importing test kits and a lack of in-country technical support for instrument maintenance and repairs. As a result, more than 300 of the 614 patients tested in the first batch did not receive their VL results at their scheduled follow-up appointment, and clinicians could not use the information for optimum care.
When eventually released, quality control data showed 363 (59%) of the initial 614 samples were invalid because calibration curves or positive controls were outside acceptable limits. These faults were only recognized when data were examined by MSF staff and external international advisers; the expert interpretation was communicated to Lab 2 and the service agreement conditions were further clarified. The official release of results by Lab 2 combined with the delay in receiving quality control data meant that some patients received inaccurate results. Seventy-eight of the 363 patients from these invalid test runs were told that their VLs were greater than 1000 copies/mL, which was the project criteria for enhanced adherence counselling, whereas other results might have been inaccurately low. Lab 2 blamed calibration errors on power failures and an inverter too weak to prevent interruptions when switching from national power to generator. The initial sample volume requested by Lab 2 was inadequate for retesting, and the 363 patients had to be recalled for repeat venepuncture. Of the first 124 of these samples retested, 52 (42%) were in a second invalid run and required further repeat testing.
Sending samples internationally for ongoing quality control was not feasible because of the time, logistics and expense involved. Instead, around 10% of samples after the initial eight test runs were sent in blinded duplicate to Lab 2. Of 46 initial sample pairs in test runs with acceptable quality control, the result for seven (15%) duplicate pairs was greater than 1.0 absolute log difference (maximum 2.7 log).
Although we support the WHO recommendations on routine VL testing in principle, our results question the feasibility of safe implementation in resource-limited settings with current technology in the absence of real-time review of quality control data by an experienced third party. Inaccurate results and quality control delays created the potential for HIV patients to be attributed with "false virological failures". This caused confusion and distress for patients, since ART appeared to be failing despite potentially optimal adherence.
Extra appointments were also needed, which increased time and transport costs. ART might also have been switched to second-line unnecessarily. No regimens were changed on the basis of invalid 2007 results; however, on file review, three changes in 2006 were possibly influenced by VL results later deemed invalid. Second-line treatment carries an increased pill burden and cost, and if true virological failure occurred later, patients might be assumed to have no effective regimens in the absence of third-line ART. Importantly, poor VL performance may have resulted in false negative results, where patients with virological failure were not detected, putting them at risk of severe clinical illness and antiviral resistance.
Retesting patients because of quality control problems was a burden for them and for programme staff, especially as some patients had to be recalled to discuss released, then invalidated, results. This led to apparent frustration, loss of confidence, and inconvenience for patients. Perceived poor VL outcomes led to inaccurate assessments of programme quality, with consequent unnecessary programmatic changes.
There are potential limitations with the quality control analyses with regard to the number of externally tested samples, time of storage, and comparison of VL on duplicate samples using different tests in different laboratories and where subtype was not tested. However, these differing test methods would be expected to give comparable results [15–17]. In view of the large discrepancy between VL results from Lab 1 and immunological outcomes, outcomes achieved with standard MSF protocols in other similar settings, the results of quality control tests, and the subsequent VL results, most of the observed discordance is probably due to poor test performance and lack of appropriate quality control. Additionally, quality issues other than discordant results remain relevant.
We cannot assume that a laboratory will provide accurate results simply because it is supported by a reputable international laboratory and participates in an external quality assurance programme. Sophisticated equipment does not guarantee accurate results . Contractual agreements with laboratories should include remuneration based upon release of valid results with accompanying quality control data. Calibration data should be able to be correctly reviewed and interpreted by staff. Reluctance to release these data (either at all or with delay) should be taken as a "red flag". Lab 2 reported that it had fixed its inconsistent power problem, had the equipment serviced (one reason for test delays), and paid more attention to quality control, yet returned some additional results with invalid quality control. WHO provides guidance on training to improve quality control and quality assurance in the context of complex laboratory testing .
Barriers to obtaining reliable laboratory results for complex tests in resource-limited settings include: unpredictable power supplies; shortages of service engineers; expense of reagents in performing repeat test runs; lack of motivation and accountability; and shortages of highly qualified scientific staff [9, 18, 20]. In addition, although complex equipment may be supplied by funding agencies, quality assurance is not routinely included in their budgets, ongoing access to training and technical support is often not available, and laboratory testing is not seen as a high priority expense .
The resources and expertise needed to ensure accuracy make it unlikely that all programmes will be able to achieve sufficient confidence in VL results. Worryingly, many programmes are already using VL technology routinely in settings where it is difficult to ensure the quality of laboratory services . Our results indicate a crucial need for these programmes to review the accuracy of VL results. More adapted, reliable, cheap and point-of-care methods of VL testing are urgently required for resource-limited settings .
Our experience of the difficulties encountered in this programme and other programmes in eastern Europe, Asia and other countries in sub-Saharan Africa led to development of a guide for assessing a laboratory's suitability for VL (or other) testing (See additional file 1: Checklist). It aims to minimize the likelihood of obtaining inaccurate results, focusing on the analytic phase of testing, and could help ensure quality results.
Although the quality of VL testing may be reliable in many resource-limited settings, the need to ensure that this is true applies to all laboratories in these settings. Despite cost and logistical issues, blinded duplicate samples should be sent to the testing laboratory, with investigation triggered by a simple rule violation of any identical samples being outside the expected range of difference.
We thank Leslie Shanks for comments on the manuscript and consideration of the paper with respect to the MSF Ethics Review Board framework criteria. We thank Sunday Ashaolu for sample and laboratory management. We thank Margrethe Lüneborg-Nielsen and Jan Gerstoft for assistance with external quality control testing. We thank Carol Swantee for advice on viral load quality control. We thank Sarah Venis and Stephanie Bartlett for editing assistance.
- Kantor R, Diero L, Delong A, Kamle L, Muyonga S, Mambo F, Walumbe E, Emonyi W, Chan P, Carter EJ, Hogan J, Buziba N: Misclassification of first-line antiretroviral treatment failure based on immunological monitoring of HIV infection in resource-limited settings. Clin Infect Dis 2009, 49:454–462.PubMedView ArticleGoogle Scholar
- van Oosterhout JJ, Brown L, Weigel R, Kumwenda JJ, Mzinganjira D, Saukila N, Mhango B, Hartung T, Phiri S, Hosseinipour MC: Diagnosis of antiretroviral therapy failure in Malawi: poor performance of clinical and immunological WHO criteria. Trop Med Int Health 2009, 14:856–861.PubMedView ArticleGoogle Scholar
- DART Trial Team: Routine versus clinically driven laboratory monitoring of HIV antiretroviral therapy in Africa (DART): a randomised non-inferiority trial. Lancet 2010, 375:123–131.View ArticleGoogle Scholar
- Coutinho A, Mermin J, Ekwaru J, Were W, Bunnell R, Kaharuza F, Alexander L, Solberg P, Tappero J, Moore D: Utility of Routine Viral Load, CD4 Cell Count, and Clinical Monitoring among HIV-Infected Adults in Uganda: A Randomized Trial [abstract]. [http://www.retroconference.org/2008/Abstracts/30881.htm] 15th Conference on Retroviruses and Opportunistic Infections; February 2008; Boston, US
- The ART-LINC of IeDEA Study Group: Switching to second-line antiretroviral therapy in resource-limited settings: comparison of programmes with and without viral load monitoring. AIDS 2009, 23:1867–1874.View ArticleGoogle Scholar
- Calmy A, Ford N, Hirschel B, Reynolds SJ, Lynen L, Goemaere E, Garcia de la Vega F, Perrin L, Rodriguez W: HIV viral load monitoring in resource-limited regions: optional or necessary? Clin Infect Dis 2007, 44:128–134.PubMedView ArticleGoogle Scholar
- WHO: Rapid advice: antiretroviral therapy for HIV infection in adults and adolescents. Geneva: World Health Organization; 2009.Google Scholar
- Petti CA, Polage CR, Quinn TC, Ronald AR, Sande MA: Laboratory medicine in Africa: a barrier to effective health care. Clin Infect Dis 2006, 42:377–382.PubMedView ArticleGoogle Scholar
- Fiscus SA, Cheng B, Crowe SM, Demeter L, Jennings C, Miller V, Respess R, Stevens W, the Forum for Collaborative HIV Research Alternative Viral Load Assay Working Group: HIV-1 viral load assays for resource-limited settings. PLoS Medicine 2006, 3:1743–1750.Google Scholar
- Bland JM, Altman DG: Statistical methods for assessing agreement between two methods of clinical measurement. Int J Nurs Stud 2010, 47:931–936.View ArticleGoogle Scholar
- WHO: Antiretroviral therapy for adults and adolescents in resource-limited settings: towards universal access. Geneva: World Health Organization; 2006.Google Scholar
- Elema R, Mills C, Yun O, Lokuge K, Ssonko C, Nyirongo N, Mtonga V, Zulu H, Tu D, Verputten M, O'Brien DP: Outcomes of a remote, decentralized health center-based HIV/AIDS antiretroviral program in Zambia, 2003 to 2007. J Int Assoc Physicians AIDS Care (Chic Ill) 2009, 8:60–67.View ArticleGoogle Scholar
- Ferradini L, Laureillard D, Prak N, Ngeth C, Fernandez M, Pinoges L, Puertas G, Taburet AM, Ly N, Rouzioux C, Balkan S, Quillet C, Delfraissy J-F: Positive outcomes of HAART at 24 months in HIV-infected patients in Cambodia. AIDS 2007, 21:2293–2301.PubMedView ArticleGoogle Scholar
- Barth RE, Schim van der Loeff MF, Schuurman R, Hoepelman AIM, Wensing AMJ: Virological follow-up of adult patients in antiretroviral treatment programmes in sub-Saharan Africa: a systematic review. Lancet Infect Dis 2010, 10:155–166.PubMedView ArticleGoogle Scholar
- Galli R, Merrick L, Friesenhahn M, Ziermann R: Comprehensive comparison of the VERSANT ® HIV-1 RNA 3.0 (bDNA) and COBAS AMPLICOR HIV-1 MONITOR ® 1.5 assays on 1000 clinical specimens. J Clin Virol 2005, 34:245–252.PubMedView ArticleGoogle Scholar
- Berger A, Scherzed L, Stürmer M, Preiser W, Doerr HW, Rabenau HF: Comparative evaluation of the Cobas Amplicor HIV-1 Monitor™ Ultrasensitive Test, the new Cobas AmpliPrep/Cobas Amplicor HIV-1 Monitor™ Ultrasensitive Test and the Versant HIV RNA 3.0 assays for quantitation of HIV-1 RNA in plasma samples. J Clin Virol 2005, 33:43–51.PubMedView ArticleGoogle Scholar
- Elbeik T, Alvord WG, Trichavaroj R, de Souza M, Dewar R, Brown A, Chernoff D, Michael NL, Nassos P, Hadley K, Ng VL: Comparative analysis of HIV-1 viral load assays on subtype quantification: Bayer Versant HIV-1 RNA 3.0 versus Roche Amplicor HIV-1 Monitor version 1.5. JAIDS 2002, 29:330–339.PubMedGoogle Scholar
- Bates I, Maitland K: Are laboratory services coming of age in sub-Saharan Africa? Clin Infect Dis 2006, 42:383–384.PubMedView ArticleGoogle Scholar
- WHO: [http://www.who.int/entity/hiv/pub/drugresistance/HIVDR_Mod_11_QAQC.ppt] HIV Drug Resistance Training. Module 11: Quality Control and Quality Assurance.
- Batz HG, Guillerm M, Gonsalves G: Scaling up antiretroviral treatment in resource-poor settings [Correspondence]. Lancet 2006, 368:445.PubMedView ArticleGoogle Scholar
- Phillips A, van Oosterhout J: DART points the way for HIV treatment programmes. Lancet 2010, 375:96–98.PubMedView ArticleGoogle Scholar
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.