==== Front Ann Saudi MedAnn Saudi MedAnnals of Saudi Medicine0256-49470975-4466King Faisal Specialist Hospital and Research Centre 2747890910.5144/0256-4947.2016.245asm-4-245Original ArticlePredictive value of p16/Ki-67 immunocytochemistry for triage of women with abnormal Papanicolaou test in cervical cancer screening: a systematic review and meta-analysis Chen Cheng-Chieh aHuang Lee-Wen bBai Chyi-Huey cLee Chin-Cheng a a Department of Pathology and Laboratory Medicine b Department of Obstetrics and Gynecology, Shin Kong Wu Ho-Su Memorial Hospital c Department of Public Health, College of Medicine, Taipei Medical University, Taipei, TaiwanCorrespondence: Dr. Chin-Cheng Lee, Department of Pathology and Laboratory Medicine, Shin Kong Wu Ho-Su Memorial Hospital, No. 95, Wen Change Road, Shih Lin District, Taipei City, Taipei, 111 Taiwan, T: 886-2-28332211 ext. 2121, F: 886-2-28389360, M002058@ms.skh.org.tw, ORCID: http://orcid.org/0000-0002-3064-5016Jul-Aug 2016 36 4 245 251 Copyright © 2016, Annals of Saudi Medicine2016This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.BACKGROUND The Papanicolaou (Pap) test is one screening strategy used to prevent cervical cancer in developed countries. The p16/Ki-67 immunocytochemistry is a triage test performed on Pap smears in women with atypical squamous cells of undetermined significance (ASCUS) or low grade squamous intraepithelial lesion. OBJECTIVE Our objective was to review studies investigating the diagnostic performance of p16/Ki-67 dual stain for triage of women with abnormal Pap tests. DESIGN We conducted a systematic review and meta-analysis of diagnostic test accuracy studies. SETTINGS We followed the protocol of systematic review of diagnostic accuracy studies. PATIENTS AND METHODS We searched PubMed, The Cochrane Library, BioMed Central, and ClinicalTrials.gov for relevant studies. We included research that assessed the accuracy of p16/Ki-67 dual stain and high risk human papillomavirus testing for triage of abnormal Pap smears. Review articles and studies that provided insufficient data to construct 2×2 tables were excluded. Data synthesis was conducted using a random-effects model. MAIN OUTCOME MEASURES Sensitivity and specificity. RESULTS In seven studies encompassing 2628 patients, the pooled sensitivity and specificity of p16/Ki-67 for triage of abnormal Pap smear results were 0.91 (95% CI, 0.89 to 0.93) and 0.64 (95% CI, 0.62 to 0.66), respectively. No study used a case-control design. A subgroup analysis involving liquid-based cytology showed a sensitivity of 0.91 (95%CI, 0.89 to 0.93) and specificity of 0.64 (95%CI, 0.61 to 0.66). CONCLUSIONS Our meta-analysis of p16/Ki-67 dual stain studies showed that the test achieved high sensitivity and moderate specificity for p16/Ki-67 immunocytochemistry for high-grade squamous intraepithelial lesion and cervical cancer. We suggest that p16/Ki-67 dual stain might be a reliable ancillary method identifying high-grade squamous intraepithelial lesions in women with abnormal Pap tests. LIMITATIONS No study in the meta-analysis examined the accuracy of the p16/Ki-67 dual stain for interpretation of glandular neoplasms. ==== Body The Pap test, introduced by George Papanicolaou, is a cytological screening examination that detects abnormal epithelial cells scraped from transformation zone of uterine cervix.1 Use of the technique in cervical cancer screening reduces the incidence and mortality of cervical cancer in developed countries.2,3 However, women with equivocal cytological results usually undergo repeat Pap smears to obtain a definite diagnosis, which may cause anxiety. Immunocytochemistry for p16 protein has been shown as an effective approach for triage in women with cytological abnormalities, such as atypical squamous cells of undetermined significance (ASCUS).4 The p16 protein, a cyclin dependent kinase inhibitor, decelerates the cell cycle by facilitating the re-binding of retinoblastoma protein (Rb) and E2F transcription factor. The E7 oncoprotein, which is the product of E7 oncogene of hr-HPV interrupts the linkage between Rb and E2F transcription factor, resulting in disruption of Rb/E2F pathway.4 Therefore, overexpression of p16 protein in a dysplastic cervical epithelial cell is indicative of hr-HPV induced transformation.5 The Ki-67 antigen is a nuclear protein that expresses during all phases of the cell cycle except G0. Under normal physiological conditions, its expression is limited in basal layer squamous epithelium of uterine cervix. A staining protocol that simultaneously detects p16 protein and Ki-67 antigen in the same cervical epithelial cell has been established.6 Co-expression of p16 protein (tumor suppressor marker) and Ki-67 antigen (proliferative marker) in the same cervical epithelial cell indicates deregulation of the cell cycle and is supposed to be a positive test result.6 The hr-HPV testing has also been proposed as a tool for triage of equivocal or low-grade cytological abnormalities as its role in cervical cancer screening evolves.7,8 A systematic review and meta-analysis showed that p16 immunocytochemistry has higher specificity than hr-HPV testing to detect underlying high-grade squamous intraepithelial lesions (HSIL) in triage of ASUCS or low-grade squamous intraepithelial lesions (LSIL).9 Estimates of p16/Ki-67 dual stain in triage of abnormal Pap smear differ from study to study. The aim of the review was to evaluate the accuracy of p16/Ki-67 dual stain for triage of women with abnormal Pap smear results. PATIENTS AND METHODS We followed the review protocol based on the guideline for systematic review of diagnostic accuracy studies and used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement as framework for reporting the review.10,11 Literature search We searched PubMed, The Cochrane Library, BioMed Central, and ClinicalTrials.gov using Boolean logic of the following search terms: patients (cervical intraepithelial neoplasia OR CIN OR cervical dysplasia OR HSIL OR LSIL OR squamous intraepithelial lesion OR precancer), intervention (cytology OR pap smear OR pap test) (p16 or p16 protein or Ki-67 or Ki-67 protein or immunocytochemistry or p16/Ki-67 dual stain), and outcome (sensitivity OR diagnostic accuracy). The search strategy in PubMed is in the Appendix. No publication date restriction was applied. English language studies were reviewed. The last search was performed on October 4, 2014. Study selection and quality assessment We included studies that addressed the test performance of p16/Ki-67 immunocytochemistry and hr-HPV testing, as a comparator, for triage of women with cytological abnormalities. We also included primary studies focused on p16/Ki-67 dual stain performed on conventional smear or liquid-based cytology. Either prospective trials or retrospective studies were included. We excluded review articles and research that reported insufficient data for calculating sensitivity and specificity. One reviewer initially screened titles and abstracts for potentially relevant studies. Two reviewers independently examined full-text articles to determine eligibility. Disagreements were resolved by discussion. We assessed the methodological quality of the included studies using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2).12 The QUADAS-2 consists of four domains: patient selection, index test, reference standard, and flow and timing. Each domain has “signaling questions” that help in judging the bias and applicability of studies. A study is considered to be high quality if it used consecutive recruitment of participants, all participants used the same reference standard, index test interpretation was done without knowledge of the reference standard, and all recruited participants were included in analyses. Data extraction and synthesis We extracted data from each study to construct a 2×2 table of true positives, true negatives, false positives, and false negatives to calculate sensitivity and specificity of both the p16/Ki-67 dual stain and hr-HPV testing. Sensitivity is the proportion of true positive results out of all who have the target condition (illness) whereas specificity is the proportion of true negative results out of all who do not have the target condition (illness). Diagnostic odds ratio (DOR) was calculated as sensitivity/(1- sensitivity) over (1- specificity)/specificity. 13 A diagnostic test is thought to be discriminative if it has a DOR greater than one.14 We defined p16/Ki-67 immunocytochemistry performed on abnormal Pap smear as the index test. We considered molecular testing for hr-HPV as the comparator. A biopsy sample taken from uterine cervix was regard as the reference standard. We conducted a meta-analysis of p16/Ki-67 results extracted from included studies to generate pooled sensitivity, specificity, and DOR using a random-effects model. We plotted the summary receiver operating characteristic (SROC) curve to demonstrate overall test performance of p16/Ki-67 dual stain. The closer the curve approaches the upper left-handed corner, the better overall diagnostic performance of the test.15 We also carried out a meta-analysis for hr-HPV assays using a random-effects model to produce summary sensitivity and specificity. Pooled estimates including sensitivity, specificity, and DOR were generated along with associated 95% confidence interval (95% CI). Owing to inconsistent study design and cytological preparation, we investigated heterogeneity across studies. I2 is one of the indicators of quantitative evaluation of heterogeneity. The I2 expresses the percentage of the total variation across studies due to true differences rather than chance. Values of 25%, 50%, and 75% represent low, moderate, and high level of heterogeneity, respectively. 16 All analyses were performed by using MetaDiSc 1.4 software package.17 RESULTS Overall accuracy of p16/Ki-67 dual stain and hr- HPV assay Seven eligible studies18–24 comprising 2628 participants were identified (Figure 1). Five studies were carried out with prospective study design (Table 1).18–22 Table 2 summarizes the test results for each study. In the two retrospective studies, one retrieved Pap smears from their departmental files and the other used the residual specimens collected from the authors’ previous research.23,24 One study used conventional smears and the other used liquid-based cytology. Schmidt et al reported two sets of estimates based on triages of ASCUS and LSIL.24 To avoid double counting of results from the same participants, we extracted data for the LSIL triage. The summary sensitivity and specificity of p16/Ki-67 dual stain for triage of abnormal Pap smear results were 0.91 (95% CI, 0.89 to 0.93) and 0.64 (95% CI, 0.62 to 0.66), respectively (Figure 2, Figure 3). The meta-analysis for p16/Ki-67 dual stain yielded a pooled DOR of 15.35 (95% CI, 8.2 to 28.74) (Figure 4), showing the discriminatory power of the dual stain. We obtained pooled hr-HPV test estimates that had sensitivity of 0.94 (95%CI, 0.92 to 0.96) and specificity of 0.35 (95% CI, 0.32 to 0.37). The area under the curve (AUC) of an SROC plot presents the overall accuracy of a diagnostic test. A perfect test shows an AUC equal to 1. The AUC of an excellent test should be in the region of 0.97 or above. An AUC ranging from 0.93 to 0.96 is very good; an AUC of 0.75 to 0.92 is good.25 The SROC curve (Figure 5) of p16/Ki-67 showed an AUC of 0.87, indicating that the overall test performance of p16/Ki-67 was good. The Youden index is defined as (sensitivity + specificity – 1).26 A good diagnostic test has a Youden index over 0.5. The Youden index of p16/Ki-67 dual stain was 0.55. The meta-analysis, which was conducted using a random-effects model, demonstrated that p16/Ki-67 dual stain had comparable sensitivity and higher specificity, compared with hr-HPV testing. Investigation of heterogeneity The I2 index (45.3%) of the meta-analysis in sensitivity is lower than that (90.4%) in specificity. The Cochran’s Q test for the pooled DOR yielded a result of 20.34 (P=.0024), indicating that there was heterogeneity across studies. We performed a subgroup analysis to investigate heterogeneity. Study design might have an impact on accuracy of p16/Ki-67 dual stain. The prospectively conducted studies had a pooled sensitivity and specificity of 0.90 (95%CI, 0.87 to 0.93) and 0.69 (95% CI, 0.67 to 0.72), respectively. The pooled sensitivity and specificity of the index tests in studies using liquid-based cytology were 0.91 (95%CI, 0.89 to 0.93) and 0.64 (95%CI, 0.61 to 0.66). The results were nearly identical to these of all included studies. Two studies which addressed the triage of atypical squamous cells could not exclude high-grade squamous intraepithelial lesion (ASC-H) for cervical intraepithelial neoplasia 2 or worse (CIN2+) and had slightly decreased sensitivity of 0.88 (95%CI, 0.76 to 0.95) and specificity of 0.60 (95%CI, 0.54 to 0.67).19,21 DISCUSSION Our study showed high sensitivity and moderate specificity for the p16/Ki-67 dual stain for triage of abnormal Pap smears. The presence of illness could be ruled out if a diagnostic test that has high sensitivity for the condition yields a negative result. The meta-analysis generated the pooled estimates, indicating that the p16/Ki- 67 dual stain was superior to hr-HPV testing. Arbyn et al performed a meta-analysis that showed that two types of hr-HPV assays were less specific in detecting CIN 2+ in women with ASCUS or LSIL, compared with our study.27 According to consensus guidelines developed by the American Society for Colposcopy and Cervical Pathology, hr-HPV testing is the management of choice in women with either ASCUS or LSIL.28 We suggested that the p16/Ki-67 dual stain might be an alternative procedure in the management of patients with abnormal cervical cytology. Laboratory professionals could conduct a p16/Ki-67 dual stain by using the residual specimen after liquid-based cytology preparation. The potential benefit of the p16/Ki-67 dual stain for women with an abnormal Pap test is the reduction in repeat cytology which is a worrisome problem. Colposcopy is one way of managing women with LSIL cytology.28 Negative p16/Ki-67 immunocytochemistry performed on an abnormal or equivocal Pap smear, which could be free of HSIL, might lead to decreased colposcopic referral, easing the gynecologists’ workload. Fujii et al stated that p16/Ki-67 immunocytochemistry was a more accurate triage test for identifying CIN2+ in LSIL or ASCUS specimen, compared with hr-HPV testing.29 Their finding was compatible with our study outcome. The majority of HPV infections in young women are transient.30 About 90% of women with new HPV infections resolve within two years.30 A retrospective study of 680 biopsy-proven LSIL patients demonstrated that 49 percent regressed to negative at six months.31 Such phenomenon might lead to lower specificity of hr-HPV testing in triage of LSIL. Therefore, we prefer p16/Ki-67 dual stain to hr-HPV testing. No study had a case-control study design and four studies reported consecutive enrollment of participants. 18,19,22,24 Three studies reported that p16/Ki-67 dual stain was interpreted without knowledge of the result of pathological examination.18,22,23 One study recorded that the biopsy result was interpreted without knowledge of p16/Ki-67 dual stain.18 Three studies reported that they analyzed all participants.21–23 There were no concerns about patient selection, index test, or reference standards. Regarding applicability, the studies had a low risk of bias. Four studies had a high risk of bias owing to incomplete patient analyses (Table 3).18–20,24 The specificity estimates of the included studies ranged from 0.51 to 0.72. The meta-analysis of the p16/Ki-67 dual stain in specificity generated an I2 of 90.4%, indicating that study design, patient spectrum, triage, and specimen type across studies appeared to have greater impact on pooled specificity than on pooled sensitivity. The strength of evidence of our study relied on a rigorous methodological quality assessment based on QUADAS-2. Regarding applicability, patient selection, index test, and the reference standards of studies had a low risk of bias. Two reviewers independently reviewed titles, abstracts, and full-texts. A meta-epidemiologic analysis has indicated that MEDLINE alone is sufficient for systematic reviews of diagnostic accuracy.32 A limitation of our study is that no studies in the meta-analysis examined the accuracy of the p16/Ki-67 dual stain for interpretation of glandular neoplasms. Six studies did not report whether the reference standard results were interpreted without knowledge of the index tests.19–24 Four studies stated that not all patients were included in the analyses.18–20,24 Lijmer et al reported that studies using case-control study design might produce three-fold overestimation of diagnostic accuracy effect, compared with studies with clinical cohort.33 No study in this review used a case-control study design. In summary, our investigation yielded robust summary estimates of the p16/Ki-67 dual stain, indicating that it could be an accurate triage test to identify HSIL in abnormal Pap smear cytology. Further high quality studies using consecutive patient enrollment and complete participant analysis are required to improve understanding of the applicability of p16/Ki-67 dual stain in triage of women with abnormal Pap smear results. Future systematic reviews on the diagnostic accuracy of the p16/Ki-67 dual stain for triage of endocervical glandular abnormalities detected on Pap smears are also needed to explore the overall performance of the technique. Conflicts of Interest The authors have no conflicts of interest relevant to this article. Appendix Detailed search strategy in PubMed Key words Search details cervical intraepithelial neoplasia “cervical intraepithelial neoplasia”[MeSH Terms] OR (“cervical”[All Fields] AND “intraepithelial”[All Fields] AND “neoplasia”[All Fields]) OR “cervical intraepithelial neoplasia”[All Fields] CIN “Comput Nurs”[Journal] OR “cin”[All Fields] OR “Comput Intell Neurosci”[Journal] OR “cin”[All Fields] OR “Comput Inform Nurs”[Journal] OR “cin”[All Fields] cervical dysplasia “uterine cervical dysplasia”[MeSH Terms] OR (“uterine”[All Fields] AND “cervical”[All Fields] AND “dysplasia”[All Fields]) OR “uterine cervical dysplasia”[All Fields] OR (“cervical”[All Fields] AND “dysplasia”[All Fields]) OR “cervical dysplasia”[All Fields] OR “cervical intraepithelial neoplasia”[MeSH Terms] OR (“cervical”[All Fields] AND “intraepithelial”[All Fields] AND “neoplasia”[All Fields]) OR “cervical intraepithelial neoplasia”[All Fields] OR (“cervical”[All Fields] AND “dysplasia”[All Fields]) cytology “cytology”[Subheading] OR “cytology”[All Fields] OR “cytological techniques”[MeSH Terms] OR (“cytological”[All Fields] AND “techniques”[All Fields]) OR “cytological techniques”[All Fields] OR “cytology”[All Fields] OR “cell biology”[MeSH Terms] OR (“cell”[All Fields] AND “biology”[All Fields]) OR “cell biology”[All Fields] pap smear “papanicolaou test”[MeSH Terms] OR (“papanicolaou”[All Fields] AND “test”[All Fields]) OR “papanicolaou test”[All Fields] OR (“pap”[All Fields] AND “smear”[All Fields]) OR “pap smear”[All Fields] pap test “papanicolaou test”[MeSH Terms] OR (“papanicolaou”[All Fields] AND “test”[All Fields]) OR “papanicolaou test”[All Fields] OR (“pap”[All Fields] AND “test”[All Fields]) OR “pap test”[All Fields] sensitivity “sensitivity and specificity”[MeSH Terms] OR (“sensitivity”[All Fields] AND “specificity”[All Fields]) OR “sensitivity and specificity”[All Fields] OR “sensitivity”[All Fields] diagnostic “diagnosis”[MeSH Terms] OR “diagnosis”[All Fields] OR “diagnostic”[All Fields] p16 protein “cyclin-dependent kinase inhibitor p16”[MeSH Terms] OR (“cyclin-dependent”[All Fields] AND “kinase”[All Fields] AND “inhibitor”[All Fields] AND “p16”[All Fields]) OR “cyclin-dependent kinase inhibitor p16”[All Fields] OR (“p16”[All Fields] AND “protein”[All Fields]) OR “p16 protein”[All Fields] protein “proteins”[MeSH Terms] OR “proteins”[All Fields] OR “protein”[All Fields] immunocytochemistry “immunohistochemistry”[MeSH Terms] OR “immunohistochemistry”[All Fields] OR “immunocytochemistry”[All Fields] stain “staining and labeling”[MeSH Terms] OR (“staining”[All Fields] AND “labeling”[All Fields]) OR “staining and labeling”[All Fields] OR “stain”[All Fields] Figure 1 Flow diagram of literature search. Figure 2 Forest plot showing the pooled sensitivity estimate. Figure 3 Forest plot showing the pooled specificity estimate. Figure 4 Forest plot showing the pooled diagnostic odds ratio. Figure 5 The SROC curve for p16/Ki-67 dual stain. Individual studies are shown as circles whose area is proportionate to the sample size of the study. Table 1 Characteristics of the seven included studies. Study Study design No. of participant (total/data extraction) Mean age (range) Triage Specimen type Index test Comparator Detection Reference standard Ordi et al18 Prospective, consecutive 1169/1123 35.8 (18–91) abnormal Pap test ThinPrep CINtec PLUS Hybrid Capture 2 CIN2+ Biopsy, endocervical curettage Killeen et al19 Prospective, consecutive 515/232 35.8 (21–86) ASCUS, LSIL, ASC-H, HSIL ThinPrep CINtec PLUS Cervista CIN2/3 Biopsy Vrdoljak-Mozetic et al20 Prospective 155/145 34.9 (19–69) CIN1 Conventional smear CINtec PLUS Hybrid Capture 2 CIN2+ Biopsy Byun et al21 Prospective 56/56 46 (25–83) ASC-H, LSIL-H SurePath CINtec PLUS Cytoactive CIN2+ Histology Waldstroem et al22 Consecutive 469/469 32.3 (16–25) LSIL ThinPrep CINtec PLUS APTIMA CIN2+/CIN3+ Histology Loghavi et al23 Retrospective 188/188 median: 30 (15–77) ASCUS, LSIL SurePath CINtec PLUS Cervista CIN2+ Histology Schmidt et al24 Retrospective, consecutive 810/415 >18 LSIL ThinPrep CINtec PLUS Hybrid Capture 2 CIN2+/CIN3 Biopsy ASCUS= atypical squamous cells of undetermined significance; ASC-H= atypical squamous cells cannot exclude high grade squamous intraepithelial lesion; CIN= cervical intraepithelial neoplasia; CIN 2+ = cervical intraepithelial neoplasia 2 or worse; CIN 3+ = cervical intraepithelial neoplasia 3 or worse; HSIL= high-grade squamous intraepithelial lesion; LSIL= low-grade squamous intraepithelial lesion; LSIL-H= low-grade squamous intraepithelial lesion cannot exclude high grade squamous intraepithelial lesion. Manufacturers are as follows (listed in alphabetical order by product): APTIMA, Gen-Probe company; Cervista, Hologic; CINtec PLUS, Roche; Cytoactive, Cytoimmun Diagnostics GmbH; Hybrid Capture 2, QIAGEN; SurePath, Becton, Dickinson and Company; ThinPrep, Hologic Table 2 Summary of test results for each study in the meta-analysis. Study True positive False positive False negative True negative Total Ordi et al18 360 203 36 524 1123 Killeen et al19 33 75 2 122 232 Vrdoljak-Mozetic et al20 16 36 4 89 145 Byun et al21 16 17 5 18 56 Waldstroem et al22 77 186 10 196 469 Loghavi et al23 30 74 1 83 188 Schmidt et al24 129 89 8 189 415 Table 3 Quality of the seven studies included in the meta-analysis. Study Risk of bias Applicability concerns Patient selection Index test Reference Standard Flow and timing Patient selection Index test Reference standard Ordi et al18 LR LR LR HR LR LR LR Killeen et al19 LR UR UR HR LR LR LR Vrdoljak-Mozetič et al20 UR UR UR HR LR LR LR Byun et al21 UR UR UR UR LR LR LR Waldstroem et al22 LR LR UR LR LR LR LR Loghavi et al23 UR LR UR LR LR LR LR Schmidt et al24 LR UR UR HR LR LR LR HR=high risk; LR=low risk; UR=unknown risk ==== Refs REFERENCES 1 Burd EM Human papillomavirus and cervical cancer Clin Microbiol Rev 2003 16 1 1 17 12525422 2 Aklimunnessa K Mori M Khan MM Sakauchi F Kubo T Fujino Y Effectiveness of cervical cancer screening over cervical cancer mortality among Japanese women Jpn J Clin Oncol 2006 36 8 511 8 16844732 3 Nygård JF Skare GB Thoresen SØ The cervical cancer screening programme in Norway, 1992–2000: changes in Pap smear coverage and incidence of cervical cancer J Med Screen 2002 9 2 86 91 12133929 4 Nieh S Chen SF Chu TY Lai HC Lin YS Fu E Is p16(INK4A) expression more useful than human papillomavirus test to determine the outcome of atypical squamous cells of undetermined significance-categorized Pap smear? A comparative analysis using abnormal cervical smears with follow-up biopsies Gynecol Oncol 2005 97 1 35 40 15790434 5 Klaes R Friedrich T Spitkovsky D Ridder R Rudy W Petry U Overexpression of p16(INK4A) as a specific marker for dysplastic and neoplastic epithelial cells of the cervix uteri Int J Cancer 2001 92 2 276 84 11291057 6 Wentzensen N Schwartz L Zuna RE Smith K Mathews C Gold MA Performance of p16/Ki-67 immunostaining to detect cervical cancer precursors in a colposcopy referral population Clin Cancer Res 2012 18 15 4154 62 22675168 7 Cotton S Sharp L Little J Cruickshank M Seth R Smart L The role of human papillomavirus testing in the management of women with low-grade abnormalities: multicentre randomised controlled trial BJOG 2010 117 6 645 59 20374607 8 Whitlock EP Vesco KK Eder M Lin JS Senger CA Burda BU Liquid-based cytology and human papillomavirus testing to screen for cervical cancer: a systematic review for the U.S. Preventive Services Task Force Ann Intern Med 2011 155 10 687 97 W214 5 22006930 9 Roelens J Reuschenbach M von Knebel Doeberitz M Wentzensen N Bergeron C Arbyn M p16INK4a immunocytochemistry versus human papillomavirus testing for triage of women with minor cytologic abnormalities: a systematic review and meta-analysis Cancer Cytopathol 2012 120 5 294 307 22700382 10 Leeflang MM Deeks JJ Gatsonis C Bossuyt PM Cochrane Diagnostic Test Accuracy Working Group Systematic reviews of diagnostic test accuracy Ann Intern Med 2008 149 12 889 97 19075208 11 Moher D Liberati A Tetzlaff J Altman DG The PRISMA Group Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement BMJ 2009 339 332 6 12 Whiting PF Rutjes AW Westwood ME Mallett S Deeks JJ Reitsma JB QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies Ann Intern Med 2011 155 8 529 36 22007046 13 Glas AS Lijmer JG Prins MH Bonsel GJ Bossuyt PM The diagnostic odds ratio: a single indicator of test performance J Clin Epidemiol 2003 56 11 1129 35 14615004 14 Birim O Kappetein AP Stijnen T Bogers AJ Meta-analysis of positron emission tomographic and computed tomographic imaging in detecting mediastinal lymph node metastases in nonsmall cell lung cancer Ann Thorac Surg 2005 79 1 375 82 15620991 15 Chartrand C Leeflang MM Minion J Brewer T Pai M Accuracy of rapid influenza diagnostic tests: a meta-analysis Ann Intern Med 2012 156 7 500 11 22371850 16 Higgins JP Thompson SG Deeks JJ Altman DG Measuring inconsistency in meta-analyses BMJ 2003 327 7414 557 60 12958120 17 Zamora J Abraira V Muriel A Khan K Coomarasamy A Meta-DiSc: a software for meta-analysis of test accuracy data BMC Med Res Methodol 2006 6 31 16836745 18 Ordi J Sagasta A Munmany M Rodríguez-Carunchio L Torné A del Pino M Usefulness of p16/Ki-67 immunostaining in the triage of women referred to colposcopy Cancer Cytopathol 2014 122 3 227 35 24757722 19 Killeen JL Dye T Grace C Hiraoka M Improved abnormal Pap smear triage using cervical cancer biomarkers J Low Genit Tract Dis 2014 18 1 1 7 23760145 20 Vrdoljak-Mozetič D Krašević M Verša Ostojić D Stemberger-Papić S Rubeša-Mihaljević R Bubonja-Šonje M HPV16 genotype, p16/Ki-67 dual staining and koilocytic morphology as potential predictors of the clinical outcome for cervical low-grade squamous intraepithelial lesions Cytopathology 2015 26 1 10 8 24261348 21 Byun SW Lee A Kim S Choi YJ Lee YS Park JS Immunostaining of p16(INK4a)/Ki-67 and L1 capsid protein on liquidbased cytology specimens obtained from ASC-H and LSIL-H cases Int J Med Sci 2013 10 12 1602 7 24151431 22 Waldstrøm M Christensen RK Ørnskov D Evaluation of p16(INK4a)/Ki-67 dual stain in comparison with an mRNA human papillomavirus test on liquid-based cytology samples with low-grade squamous intraepithelial lesion Cancer Cytopathol 2013 121 3 136 45 22987560 23 Loghavi S Walts AE Bose S CINtec® PLUS dual immunostain: a triage tool for cervical pap smears with atypical squamous cells of undetermined significance and low grade squamous intraepithelial lesion Diagn Cytopathol 2013 41 7 582 7 22833355 24 Schmidt D Bergeron C Denton KJ Ridder R European CINtec Cytology Study Group p16/ki-67 dual-stain cytology in the triage of ASCUS and LSIL papanicolaou cytology: results from the European equivocal or mildly abnormal Papanicolaou cytology study Cancer Cytopathol 2011 119 3 158 66 21442767 25 Jones CM Athanasiou T Summary receiver operating characteristic curve analysis techniques in the evaluation of diagnostic tests Ann Thorac Surg 2005 79 1 16 20 15620907 26 Ruopp MD Perkins NJ Whitcomb BW Schisterman EF Youden Index and optimal cut-point estimated from observations affected by a lower limit of detection Biom J 2008 50 3 419 30 18435502 27 Arbyn M Roelens J Cuschieri K Cuzick J Szarewski A Ratnam S The APTIMA HPV assay versus the Hybrid Capture 2 test in triage of women with ASC-US or LSIL cervical cytology: a meta-analysis of the diagnostic accuracy Int J Cancer 2013 132 1 101 8 22610699 28 Massad LS Einstein MH Huh WK Katki HA Kinney WK Schiffman M 2012 updated consensus guidelines for the management of abnormal cervical cancer screening tests and cancer precursors Obstet Gynecol 2013 121 4 829 46 23635684 29 Fujii T Saito M Hasegawa T Iwata T Kuramoto H Kubushiro K Performance of p16INK4a/Ki-67 immunocytochemistry for identifying CIN2+ in atypical squamous cells of undetermined significance and low-grade squamous intraepithelial lesion specimens: a Japanese Gynecologic Oncology Group study Int J Clin Oncol 2015 20 1 134 42 24744261 30 Steben M Duarte-Franco E Human papillomavirus infection: epidemiology and pathophysiology Gynecol Oncol 2007 107 2 Suppl 1 S2 5 31 Bansal N Wright JD Cohen CJ Herzog TJ Natural history of established low grade cervical intraepithelial (CIN 1) lesions Anticancer Res 2008 28 3B 1763 6 18630456 32 van Enst WA Scholten RJ Whiting P Zwinderman AH Hooft L Meta-epidemiologic analysis indicates that MEDLINE searches are sufficient for diagnostic test accuracy systematic reviews J Clin Epidemiol 2014 67 11 1192 9 24996667 33 Lijmer JG Mol BW Heisterkamp S Bonsel GJ Prins MH van der Meulen JH Empirical evidence of design-related bias in studies of diagnostic tests JAMA 1999 282 11 1061 6 10493205