Selected Readings

Selected readings on topics in the statistics of vaccines research for those new to vaccines statistics

Vaccine Safety

Vaccine Lot Consistency

Immunological Correlates of Protection

Selections on vaccine efficacy and vaccine safety are in preparation

Selected Readings: Vaccine Safety

The following selection of readings and links on the statistics of vaccine safety has been prepared by four volunteers and two members of the committee with the intention that it be useful to those new to the field of statistics of vaccines research.  There are three sections:

·    Methods

·    Applications

·    Initiatives and Projects

Methods

Shih MC, Lai TL, Heyse JF, Chen J. Sequential generalized likelihood ratio tests for vaccine safety evaluation. Statistics in Medicine 2010; 29(26):2698-2708.

Proposes a class of sequential generalized likelihood ratio tests for evaluating adverse event rates in two-armed pre-licensure clinical trials and single-armed post-licensure studies. The approach is illustrated using data from the Rotavirus Efficacy and Safety Trial.

DuMounchel W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. The American Statistician 1999; 53:177-190.

Describes a data mining method for searching for associations in large data bases.

Kulldorff M, Davis RL, Kolczak M, et al. A maximized sequential probability ratio test for drug and vaccine safety surveillance. Sequential Analysis 2011; 30:58-78.

Proposes a maximized sequential probability ratio test to continuously monitor pre-specified safety hypotheses over time. Uses a likelihood ratio test and a flat stopping boundary that holds the Type 1 error rate across all tests performed.  The method is illustrated using data from the US Vaccine Safety Datalink and compared with Wald’s classical sequential probability ratio test. A reference table of critical stopping values is provided.

Whitaker HJ, Hocine M, Farrington CP. The methodology of self-controlled case series studies. Statistical Methods in Medical Research 2009; 18(1):7-26.

Describes the self-controlled case series method, going through the assumptions in detail and how violations may be addressed, such as when an event can influence exposure. Also mentions sequential self-controlled case series methods.

Farrington CP. Control without separate controls: Evaluation of vaccine safety using case-only methods. Vaccine 2004; 22:2064-2070.

Considers a number of case only designs along with examples – ecological methods, case-coverage, case-crossover and self-controlled case series.

Glanz JM, McClure DL, Xu S, Hambridge SJ, Lee M, Kolczak MS, Kleinman K, Mullooly JP, France EK. Four different study designs to evaluate vaccine safety were equally validated with contrasting limitations. Journal of Clinical Epidemiology 2006; 59:808-818.

Compares self-controlled case series, cohort, case-control and risk-interval designs using simulation of risks based on the US Vaccine Safety Datalink population. Power, bias and MSE are assessed.

Fine P, Chen R. Confounding in studies of adverse reactions to vaccines. American Journal of Epidemiology 1992; 136:121-135.

Looks at the size of bias that unmeasured confounding could introduce to an estimate of the risk of an event after vaccination. Uses as an example diphtheria-pertussis-tetanus vaccination and sudden infant death syndrome.

Applications

Madsen KM, Hviid A, Vestergaard M, Schendel D, Wohlfahrt J, Thorsen P, Olsen J, Melbye M. A population-based study of measles, mumps, and rubella vaccination and autism. New England Journal of Medicine 2002; 347(19):1477-82.

One of the landmark studies evaluating the claimed association between MMR vaccine and autism. Eight Danish birth cohorts were investigated under a retrospective cohort study design. Outcome and exposure variables were retrieved from six different population-based registries.

Smeeth L, Cook C, Fombonne E, Heavy L, Rodrigues L, Smith P, Hall A. MMR vaccination and pervasive developmental disorders: a case-control study. Lancet 2004; 364:963-69.

Uses the General Practice Research Database in the UK to explore the relationship between MMR vaccination and autism with a case-control design. Controls were matched on age, sex and general practice, enabling assessment of the association at any time after vaccination.

Niu MT, Erwin DE, Braun MM. Data mining in the US Vaccine Adverse Event Reporting System (VAERS): early detection of intussusception and other events after rotavirus vaccination. Vaccine 2001; 19:4627-4634.

A study in the US Vaccine Adverse Event Reporting System to test the utility of an empirical Bayesian data mining approach to detect retrospectively a known side effect of vaccination – intussusception following rotavirus vaccination. Results demonstrate the utility of the method for early detection of significant vaccine-associated events.

Haber P, Patel M, Izurieta HS, et al. Postlicensure monitoring of intussusception after Rotateq vaccination in the United States. Pediatrics 2008; 121:1206-1212

Considered rates of intussusception in children following vaccination with RotaTeq™, using data from the Vaccine Adverse Event Reporting System and the Vaccine Safety Datalink. Calculated "observed" vs. "expected" rates of intussusception, evaluated whether the rates of intussusception were higher than expected in the two databases, and concluded that the current evidence does not suggest a link between RotaTeq™ and intussusception.

Villumsen M, Sørup S, Jessa T, Ravn H, Relander T, Baker JL, Benn CS, Sørensen TIA, Aaby P, Roth A. Risk of lymphoma and leukaemia after bacille Calmette-Guérin and smallpox vaccination: A Danish case-cohort study. Vaccine 2009; 27(49):6950-6958.

A case-cohort study on non-specific effects of bacille Calmette-Guérin and smallpox vaccinations. An example of how the same set of controls (i.e. subcohort) can be used for studying more than one exposure and more than one outcome.

Miller E, Waight P, Farrington P, Stowe J, Taylor B. Idiopathic thrombocytopenic purpura and MMR vaccine. Archives of Disease in Childhood 2001; 84:227-229.

An example using the self-controlled case series method for an event where there is a true vaccine risk. It is often used in examples of the method.

Klein NP, Fireman B, Yih WK, Lewis E, Kulldorff M, Ray P, Baxter R, Hambidge S, Nordin J, Naleway A, Belongia EA, Lieu T, Baggs J, Weintraub E; Vaccine Safety Datalink. Measles-mumps-rubella-varicella combination vaccine and the risk of febrile seizures. Pediatrics 2010; 126(1):e1-8.

A study which identified an increased risk of seizure in infants following receipt of the combination measles-mumps-rubella-varicella vaccine vs. separate injections of measles-mumps-rubella and varicella vaccines. Led to national Advisory Committee on Immunization Practices policy changes around recommended use of measles-mumps-rubella-varicella vaccine.

Initiatives and Projects

The following websites provide information on some initiatives and projects related to post-licensure vaccine safety.

CDC Vaccine Safety Datalink (VSD) Project

Collaborative effort between US Center for Disease Control and Prevention Immunization Safety Office and eight managed care organizations, established in 1990 to monitor immunization safety and address gaps in scientific knowledge about rare and serious adverse events following immunization. Includes a large linked database that uses administrative data sources at each managed care organization. http://www.cdc.gov/vaccinesafety/Activities/VSD.html

FDA Sentinel Initiative

Launched in May 2008 by US FDA, the Sentinel Initiative aims to develop and implement a national electronic proactive system that will complement existing systems the Agency has in place to track reports of adverse events linked to the use of its regulated products. http://www.fda.gov/Safety/FDAsSentinelInitiative/default.htm

FDA Vaccine Adverse Event Reporting System (VAERS)

The Vaccine Adverse Event Reporting System is a national vaccine safety surveillance program co-sponsored by the FDA and the Center for Disease Control and Prevention that collects and analyzes information from reports of adverse events that occur after the administration of US licensed vaccines. http://www.fda.gov/BiologicsBloodVaccines/SafetyAvailability/

ReportaProblem/VaccineAdverseEvents/default.htm

Vaccine Adverse Event Surveillance & Communication (VAESCO)

A European research network with the primary aim of developing guidelines and a sustainable infrastructure for post licensure vaccine safety assessment in the European region. http://vaesco.net/vaesco.html

The WHO Programme for International Drug Monitoring

Forum for WHO member states to collaborate in the monitoring of drug safety. Within the Programme, individual case reports of suspected adverse drug reactions are collected and stored in a common database, presently containing over 6 million case reports. http://www.who-umc.org/DynPage.aspx

Selection by Nick Andrews, Xuan Chen, Jennifer Clark-Nelson, Clara Dominguez, Andrew Dunning and Jukka Jokinen for the Vaccines Sub-committee of the ISCB.

Selected Readings: Vaccine Lot Consistency

A volunteer and two members of the committee, together with an external reviewer, have prepared the following selection of readings on vaccine lot-to-lot consistency with the intention that it be useful to those new to the field of statistics of vaccines research.

Schuirmann DJ. A comparison of the two-one sided tests procedure and the power approach for assessing equivalence of average bioavailability. Journal of Pharmacokinetics and Biopharmaceutics 1987; 15:657–680.

Classical approach for demonstrating equivalence of two formulations/treatments. For an 0.05 alpha level, the two one-sided tests (TOST) approach is equivalent to constructing a two-sided 90% confidence interval for the treatment difference. In vaccine clinical research the procedure is often applied to log transformed antibody titers, to demonstrate that the 95% confidence interval on the ratio of geometric means for each pair of lots lies within a pre-specified interval of –D to D.

Wiens B, Iglewicz B. On testing equivalence of three populations. Journal of Biopharmaceutical Statistics 1999; 9: 465–483

Extension of the test statistic based on the minimum of pairwise test statistics between 3 treatments groups by comparing it to adjusted critical values (less conservative than the standard normal values). Application to unequal population variances and to equivalence of 3 binomial proportions. Authors assume equality of proportions for power calculation. Provide useful references for lot consistency trials for which power was based on assumption of no between lot variability.

Wiens B, Iglewicz B. Design and analysis of three treatment equivalence trials. Controlled Clinical Trials 2000; 21: 127-137

Highlights of special considerations to be taken when testing equivalence of three treatments rather than two and demonstration of the efficiency of testing 3 treatments at a time. Discussions on stepwise and pairwise comparisons when pairs of treatments are equivalent but not all the three treatments (not interesting in the case of lot-to-lot consistency study when an all-or-nothing conclusion is necessary).

Lachenbruch PA, Rida W, Kou J. Lot consistency as an equivalence problem. Journal of Biopharmaceutical Statistics 2004; 14:275–290.

Development of a nonparametric method based on measuring similarity between distributions. Comparison with the standard methods. Gives a lucid overview of lot consistency trials.

Ganju J, Izu A, Anemona A. Sample size for equivalence trials: a case study from a vaccine lot consistency trial. Statistics in Medicine 2008; 27:3743-3754.

Reanalysis of published data to demonstrate the presence of between lot variation and to show adverse impact on power under conventional assumptions. Development of an appropriate sample size formula as a function of within and between lot components of variation.

J. Nauta. Statistical analysis of influenza vaccine lot consistency studies. Journal of Biopharmaceutical Statistics, 1520-5711, Volume 16, Issue 4, 2006, Pages 443-452.

Presentation of statistical approaches for testing lot-to-lot consistency with application to an influenza vaccine trial.

Selection by Edith Langevin, Allen Izu and Jos Nauta for the Vaccines Sub-committee of the ISCB.

Other papers considered may be viewed here. 

Independent review by Robert Kohberger.

Selected Readings: Immunological Correlates of Protection

Members of the committee, together with an external reviewer, have prepared the following selection of readings on immunological correlates of protection with the intention it be useful to those new to the field of statistics of vaccines research.

Siber GR. Methods for estimating serological correlates of protection. Developments in Biological Standardization 1997; 89: 283-296.

An early but frequently cited paper which clarified a number of concepts related to correlates of protection that were later developed into statistically-based methods.

Chan IS, Li S, Matthews H, Chan C, Vessey R, Sadoff J, Heyse J. Use of statistical models for evaluating antibody response as a correlate of protection against varicella. Statistics in Medicine 2002; 21: 3411-3430.

Models the relationship between assay values and rates of disease over the longer term (up to 5 years) using a number of different statistical models.

Jodar L, Butler J, Carlone G, Dagan R, Goldblatt D, Kayhty H, Klugman K, Plikaytis B, Siber G, Kohberger R, Chang I, Cherian T. Serological criteria for evaluation and licensure of new pneumococcal conjugate vaccine formulations for use in infants. Vaccine 2003; 21: 3265-3272.

First instance of associating immunogenicity with vaccine efficacy to find a correlate of protection; led to acceptance by WHO of pneumococcal correlate following conjugate vaccination (sometimes referred to as the method of Chang and Kohberger).

Dunning AJ. A model for immunological correlates of protection. Statistics in Medicine 2006; 25: 1485-1497.

Separately parameterizes 'exposure', thus enables a 'protection curve' to be derived.

Gilbert PB, Qin L, Self SG. Evaluating a surrogate endpoint at three levels, with application to vaccine development. Statistics in Medicine 2007; 27: 4758-4778.

Methods for testing the association between assay values and disease by a Prentice-criterion method and by a causal inference method; proposes a hierarchy for correlates of protection.

Kohberger RC, Jemiolo D, Noriega F. Prediction of pertussis vaccine efficacy using a correlates of protection model. Vaccine 2008; 26: 3516-3521.

Combines results of four different assays to predict protection (an important question since one assay likely cannot capture the full effect of the immune system).

Breslow NE, Lumley T, Ballantyne CM, Chambless LE, Kulich M. Using the whole cohort in the analysis of case-cohort data. American Journal of Epidemiology 2009; 169(11):1398-405.

Describes an efficient method for evaluating the correlation between immunological measurements and a clinical endpoint of interest in two-phase (case-cohort or retrospective case-control) designs by leveraging all of the information in baseline covariates; provides background on previous methods for similar designs.

Selection by Andrew Dunning, Jukka Jokinen and Jos Nauta for the Vaccines Sub-committee of the ISCB

Reviewed by Peter Gilbert, Statistical Center for HIV/AIDS Research & Prevention