Biomarker Test Development from the Inside Out

---

By Mirella G.-Zulueta, M.D., M.Sc., Ph.D.

A biomarker is a measurable indicator of a specific biological state, particularly one relevant to the risk of contracting, or determining the presence or stage of disease. Though historically often referring to a physical trait or physiological metric (heart rate, blood pressure), the term is now typically shorthand for a molecular biomarker. Molecular biomarkers can take many forms, and as a consequence a variety of strategies have been adopted for their discovery.

The sequencing of the human genome, in conjunction with advanced analytical technologies, has made possible a new generation of molecular markers, including single-nucleotide polymorphism (SNP) analysis, genetic and proteomic profiling, epigenetic profiling, and gene expression profiling, which carry the promise of increased disease-related sensitivity and specificity coupled with higher dimensional complexity to provide greater individualized disease management.

Preventive biomarkers prospectively identify individuals at increased risk for developing disease. Diagnostic biomarkers identify the presence of disease at the earliest stage before clinical manifestation. Prognostic biomarkers stratify risk of disease progression in patients undergoing therapy. Predictive biomarkers identify patients most likely to respond to specific interventions. Therapeutic biomarkers provide a quantifiable measure of response to therapy in patients undergoing treatment. Biomarkers may also be used to identify patients at risk for developing adverse reactions to individual therapeutics.

General Reflections

Over 1,000 biomarkers are currently available as genetic tests, almost universally marketed as home-brew tests without FDA approval. At present, the FDA does not regulate or certify these tests, their analytical validity, or their clinical qualification. In 1988, the U.S. Congress enacted the Clinical Laboratory Improvement Amendments (CLIAs) to certify laboratories testing human specimens and reporting patient-specific results. Under CLIA provisions, certification requires laboratories to adhere to standards for quality control, personnel qualifications, and documentation. In September 2006, the FDA took a positive step forward and issued a draft guidance extending regulatory enforcement authority to a subset of home-brew molecular tests termed in vitro diagnostic multivariate index assays (IVDMIAs),1 which measure multiple analytes analyzed with algorithms or software programs. It is anticipated that most IVDMIAs will require some level of FDA review, and some will require full regulatory approval, before they enter the marketplace.

Although the molecular revolution has stimulated a new generation of entrepreneurs capitalizing on the inherent importance of biomarkers to individualized medicine, their potential has yet to be fully realized clinically or commercially, reflecting asynchronous development of discovery technologies and paradigms for their analytical validation, clinical qualification, and application.

Moreover, validated highly complex biomarkers have exposed previously unrecognized issues surrounding their approval by regulatory agencies.

Biomarkers can influence critical clinical decision making, substantially influencing health-care economics. Their emergence as high-cost, high-profit products has seduced entrepreneurs and venture capitalists to launch new companies focused on developing molecular biomarkers.

Success of these start-up companies depends on whether their products address substantial markets and provide direction in critical clinical decision making regarding expensive, complex, or dangerous therapeutic interventions.2 At stake is a $14 billion worldwide market growing at 10% annually, which will reach $23 billion by 2010. Historically, the paradigm in the diagnostics business was to obtain approval for marketing test kits by the FDA, which would then be sold to local clinical laboratories. In the new paradigm, molecular tests forgo FDA approval and distribution to local laboratories and, rather, are run in central laboratories.2 This strategy abrogates the need for FDA approval, permitting shorter and more economical commercialization timelines. Unfortunately, these savings in money and time can reflect the absence of definitive analytical validation and clinical qualification typically mandated by the FDA. It is this failure to produce definitive analytical validation and clinical qualification that contributes to the relatively slow integration of molecular biomarkers into patient management paradigms.3

Collaboration and cooperation between stakeholders involved in biomarker development, application, and regulation is the most expeditious path forward to translate laboratory discovery into patient management.

Robust strategies must be used to manage the three major challenges of biomarker discovery: first, the complexity and dynamic range of plasma and other biofluids; second, the anticipated low relative abundance of many disease-specific biomarkers; and third, the extent of human and disease variation. No current single technology or experimental approach has the capacity to simultaneously address these challenges. Efficient biomarker development can, however, be achieved by a phased approach that progressively shifts from an unbiased experimental paradigm emphasizing comprehensive biomarker characterization to a candidate-driven paradigm emphasizing high-throughput antibody-based assays, with concordant shifts in sample source and amount, as well as methods and specific analysis platform used.

The Biomarker Development Pipeline

Our vision for a coherent and comprehensive process for biomarker development is represented in Figure 1. The ABCs of Biomarker development. Biomarker development requires a multi-step pipeline including the key components of discovery, qualification, verification, assay optimization, clinical validation and commercialization.

For the purposes of this short article, I will focus on protein biomarker development.

Discovery is the unbiased, semi-quantitative process by which the differential expression of specific molecules between states is first defined. Discovery can employ a variety of human biological materials. The 'products' of the discovery phase are lists of proteins, genes, SNPs, etc. found to be differentially present or expressed between the normal and diseased states based on semi-quantitative assessment of relative peptide/gene abundance.4 Such lists typically contain anywhere from a few to several hundred candidates. Some of these candidates will be false positives; that is, molecules that upon further testing are not differentially expressed along the distinction of interest. For example,, the false discovery rate of differentially expressed proteins is high at this stage, particularly for the lower-abundance proteins. This is due both to the low frequency (<10%) with which low-abundance peptides in complex proteomes are selected for the peptide sequence analysis that is required for their identification, and to the fact that signals at the lower limits of instrument dynamic range and sensitivity randomly exceed detection limits, yielding artificial 'differences.'

Because of their high false-discovery rate and minimal credentialing, these putative differentially expressed proteins are referred to as 'candidate biomarkers' (not biomarkers).

Unbiased protein biomarker development from human plasma has been largely unsuccessful to date. Biomarker discovery by current methods has an implicit reliance on redundancy of signal. Unless specifically assayed, proteins that do not vary in abundance but vary as cleavage products are likely to be missed. Although it is well established that false negatives will be common in protein biomarker discovery, the full extent of the problem is not known. The detection of proteins in plasma at the desirable low nanogram per milliliter level is often reported, but the coverage at that abundance is probably sparse. It is hoped that the use of proximal biofluids rather than plasma and more sophisticated detection methods will improve candidate quality. Of at least equal importance, small sample sizes and the artifacts of discovery strategies make it highly probable that many candidates will be false positives, in the sense that they will ultimately be shown not to have the sensitivity and specificity required of clinically meaningful biomarkers.

Distillation of true positives is the single greatest challenge in biomarker development and is the emphasis of most phases in the biomarker development pipeline. Ultimately, however, the extreme nature of both human variation and disease variability means that biomarker performance must be assayed across thousands of samples. To resolve this issue, it is useful to decouple biomarker discovery from qualification, verification and validation, using sample material and technologies in the discovery phase that will optimize identification of diagnostic candidates, while substituting in each successive phase methods and samples that more closely approximate a final clinical test.

Qualification is the process by which candidates identified using methods and biological materials optimized for discovery are translated to methods and materials suitable for verification. It thus encompasses two rather distinct components. First, assuming a blood test is ultimately intended, qualification requires that the differential protein expression detected in a proximal fluid or model system be demonstrated to hold for a parallel comparison in human plasma. Second, qualification requires demonstration that the differential expression remains detectable by the assay to be employed for verification studies, which will typically differ from the discovery assay.

To advance in development, all biomarker candidates require verification. Depending on the stringency of the discovery process and the number of candidates identified, verification may be undertaken on only a subset of qualified candidates, in which case biological knowledge, marker performance in qualification studies and reagent availability are often used for prioritization. In verification, methods are employed that provide better quantification of candidates than is typical for discovery. In addition, verification, like validation, is performed on samples that closely represent the population in which a final clinical test would be deployed. Verification reintroduces the variation that was carefully minimized in discovery and qualification, and allows marker specificity to begin to be assessed.

The historical lack of candidate verification can be ascribed in part to reagent limitations. For example, if the development of high-quality protein antibody assays were fast, straightforward and inexpensive, the barriers to verification would be greatly reduced. In fortuitous cases where high-quality antibodies already exist for newly described biomarker candidates, they are appropriately used for verification by immunoassay in patient plasma. For many or most novel candidates, such antibodies will not be available, however. For select protein candidates, immediate investment in antibodies may be justified.

The product of the 'verification' phase of the pipeline is a substantially shorter list of candidate protein biomarkers each of which has been consistently detected in population-derived human plasma or other relevant clinical sample (e.g. urine) and shown to have measurable differential expression between the states of interest (e.g., breast cancer versus normal).

The next phase of biomarker development is validation and clinical assay optimization, which typically requires measurement of thousands of patient samples with single-digit measurement coefficient-of-variation values (CVs). A platform change is again required and this mandates development of suitable antibodies for each biomarker candidate to enable its quantification.

Assuming that the protein has a sample concentration in the range of picogram to nanogram per milliliter, sensitive immunotechniques, such as radioimmunoassay (RIA) or ELISA can be used. These techniques offer a higher level of sensitivity compared with more sophisticated non-immuno-based technologies, such as LC-MS/MS, and are readily available in research and development settings of clinical laboratories.5 Considerable care should be taken in the development and optimization of an ELISA. A wide variety of variables are known to affect the performance characteristics of an ELISA, including the avidity and concentration of the capture and detection antibodies, incubation time and temperature, sample volume and dilution, pH, composition and concentration of the diluent, enzyme and substrate type, and the quality of the detector.6 If the protein is not present in healthy subjects, the standard curve can be constructed by adding the protein at different concentrations into a normal plasma pool. Only after the assay is completely optimized can its analytical performance be assessed.

Multiplexing technology, which is designed to simultaneously evaluate several novel proteins, may be a viable option. Although this technique is currently available (e.g., from Axela (Toronto, Canada)), the simultaneous optimization of several protein assays is seldom achieved; for example, an ideal buffer for one protein most likely is not ideal for another, thus resulting in an analytically suboptimal assay that could jeopardize the clinical findings.

Before evaluating the clinical utility of the newly developed assay, its analytical performance must be carefully examined, including its accuracy, precision, measurement range, and reference intervals.

Clinical evaluation of a novel biomarker test involves the assessment of its diagnostic accuracy and predictability. Diagnostic accuracy establishes how accurately the test discriminates between those with and without the disease and is determined by calculating the test's sensitivity, specificity, likelihood ratio and receiver operating characteristic (ROC) curve.7 Sensitivity and specificity are the proportion of the diseased and nondiseased subjects (as defined by the gold standard), respectively, that are correctly classified as such by the examined test. The likelihood ratio and ROC curve are derived from the calculated sensitivity and specificity values. The likelihood ratio denotes the value of the test for increasing certainty about ruling a diagnosis in or out, in the context of disease prevalance, and is useful in calculating post-test odds of having a disease as the prevalance changes. The ROC curve enables the visual comparison of the diagnostic accuracy of two or more tests and the determination of appropriate cut-points, depending on the intended clinical utility of the test.

Diagnostic predictability establishes the ability of the test to predict the presence or absence of disease for a given test result and is determined by calculating the positive and negative predictive values. The former denotes the proportion of patients with positive test results who have the disease, and the latter defines the proportion of patients with a negative test who do not have the disease. The predictive values of a test vary with the prevalence of the disease in the population examined.

To establish its clinical utility, a novel marker needs to be evaluated in a series of human studies (phase 1–4 trials) that emphasize different performance characteristics and require different study populations.

The exploratory phase (phase 1) examines whether test results are different for patients with confirmed disease from those known not to have the disease. If the area under the ROC curve is 0.5, then the test is deemed not useful and efforts toward its development should be terminated.

However, satisfactory discrimination between the two groups opens the possibility of phase 2 studies to determine diagnostic accuracy. The challenge phase (phase 2) often involves similar patient groups as phase 1 and determines whether test results predict the presence or absence of disease using different cutoff values for sensitivity and specificity. After establishing the diagnostic accuracy in this well-defined study population, phase 3 is conducted to assess the performance of the test in the target population.

The advanced clinical phase (phase 3) establishes the diagnostic accuracy and predictive values in the target population involving patients across different health care and geographical settings. It is recommended that diagnostic tests should be validated in independent studies before they are adopted in clinical practice. The outcome phase (phase 4) evaluates whether the test influences positively the ultimate health outcome of tested patients by following both those who underwent testing and those who did not with regard to post-test diagnostic and therapeutic interventions and the subsequent effect on health outcome. It is important to note that this phase can be done after the biomarker has been accepted clinically and has been made commercially available—a process that is complicated by many technical and commercial factors.

Taking it to the Market

As well as the above evidence of the clinical utility of a novel marker, companies producing in vitro diagnostics (IVDs) need to take many technical, medical, financial and legal factors into consideration. Intellectual property issues relating to the biomarker, the antibodies used for detection and the intended clinical utility could present freedom to operate challenges – a subject I won’t explore here that has been addressed in a variety of other articles. I will instead focus on two other key business issues.

Regulatory Requirements

In many countries, for an in vitro diagnostic (IVD) device to enter the market, it must comply with a set of rules and regulations, such as 510(k) premarketing clearance or Premarket Approval (PMA) oversight by the FDA in the United States8, the Pharmaceutical Affairs Law (PAL) and Market Authorization Holder (MAH) oversight by the Pharmaceutical and Medical Devices Agency (PMDA) in Japan9, and IVD Directive 98/97/EC oversight by regulatory authorities of the member states of the European Union10. These regulations are somewhat similar.

For an IVD device to be commercialized in the US market, it must undergo one of two primary regulatory processes by the FDA: 510(k) clearance or PMA. The 510(k) process is used when the new test measures an existing FDA classified analyte (class I or II) where there exists a commercially available predicate test method that has been cleared by the FDA. The submitted information for the new test includes its intended use and classification, data comparing its performance to the predicate device and characterization of its analytical capability (e.g., precision, linearity, specificity and accuracy via patient correlation studies against the predicate device). This process requires 100 days for FDA's review time and user fees are $3,800 (http://www.fda.gov/oc/mdufma/coversheet.html).

The PMA process is used when the test is classified as class III; that is, either it is associated with high risk (e.g., when the outcome determines cancer treatment or diagnosis) or the clinical utility of the marker or the technology of the measurement are novel and no predicate device can be identified. The submitted information includes the same data required for 510(k) as well as clinical outcomes data, where the level of the marker is related to disease status established according to independent clinical criteria. This process requires 180–360 days to review, and often requires an FDA Advisory Panel Meeting. User fees are in the order of $240,000
(http://www.fda.gov/oc/mdufma/coversheet.html).

In the 1990s, the FDA recognized that some newly discovered analytes have no obvious predicate devices and do not have safety concerns that automatically trigger a class III designation. Therefore, in 1997, it created a new hybrid 'de novo' or 'risk-based' classification termed to address these concerns. This process allows a new analyte to be regulated as in a 510(k), but requires the demonstration of clinical effectiveness. Although the de novo process provides a viable third option for novel marker registration, the regulatory route is not always obvious and is determined ultimately by the FDA. Brain natriuretic peptide (BNP) is an example of a novel marker that was cleared by the FDA using the de novo route. This process requires 150–180 days for FDA's review and clearance, and user fees are $3,800.

Partnerships for Biomarker Development

As indicated before, the requisite of a coevolution in analytical validation, clinical qualification, and regulation, and the patient- and capital-intensive resources required in the complex process of biomarker development have generated new partnerships among federal agencies, academia, and the pharmaceutical and biotechnology industries.

An example of such partnership is the recently established collaboration between Mount Sinai Hospital of Toronto and the diagnostic company Miraculins Inc. to develop diagnostic biomarkers for preeclampsia.

Preeclampsia is a devastating pathology of pregnancy that manifests abruptly and is characterized by high blood pressure and the presence of significant amounts of protein in the urine. In the most severe cases it can progress into eclampsia which often results in seizure, stroke, coma and death of the pregnant woman. It is estimated that preeclampsia costs the global health care system US$3 billion per year. Presently, the disease is diagnosed by its symptoms, which are non-specific to preeclampsia, and as a result there is currently no reliable test that can accurately predict its onset. Preeclampsia affects 3 million mothers worldwide every year and is associated with premature births and infant illness including cerebral palsy, blindness, epilepsy, deafness and lung conditions.

There is no effective detection method for the risk of preeclampsia and the cause only recently begun to be understood.

Biomarkers of preeclampsia were discovered through pioneering research conducted by Dr. Isabella Caniggia and Dr. Stephen Lye of the Samuel Lunenfeld Research Institute of Mount Sinai Hospital (www.mshri.on.ca), and Dr. Martin Post of the Hospital for Sick Children. Dr Caniggia’s research has provided ground-breaking insights into the molecular pathways leading to the development of preeclampsia as part of Mount Sinai Hospital’s internationally recognized leadership in research and clinical care in the area of women’s and infant’s health and high-risk pregnancy. The potential and significance of Dr. Caniggia’s findings was recognized by Miraculins Inc. (TSX VENTURE: MOM.V), a medical diagnostic company focused on developing and commercializing diagnostic tests for unmet clinical needs.


References

1.     Food and Drug Administration. Draft Guidance for Industry, Clinical laboratories, and FDA Staff: In Vitro Diagnostics Multivariate Index Assays http://www.fda.gov/cdrh/oivd/guidance1610.pdf (7 September 2006)
2.     Licking, E.F. & Longman, R. The new diagnostics companies. Star-Up 12-19 (March 2006)
3.     Williams, S.A., Slavin, D.e., Wagner, J.A. & Webster,S.J. A cost-effectiveness approach to the qualification and acceptance of biomarkers. Nat. Rev.Drug Discov. 5, 897-902 (2006)
4.     Ong, S.E. & Mann, M. Mass spectrometry-based proteomics turns quantitative. Nat. Chem. Biol. 1, 252–262 (2005)
5.     Vitzthum, F. , Behrens, F. , Anderson, N.L. & Shaw, J.H. Proteomics: from basic research to diagnostic application. A review of requirements & needs. J. Proteome Res. 4, 1086–1097 (2005)
6.     Wild, D. The Immunoassay Handbook edn. 3 (Elsevier, Amsterdam; 2005
7.     Bossuyt, P.M. et al. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Clin. Chem. 49, 7–18 (2003)
8.     Code Federal Regulations, vol. 21 CFR807 http://frwebgate.access.gpo.gov/cgi-bin/get-cfr.cgi?YEAR=current&TITLE=21&PART=807&SECTION=81&SUBPART=&TYPE=TEXT
9.     Ministerial ordinance on standards for manufacturing control and quality control for medical devices and in-vitro diagnostic reagents. (Pharmaceuticals and Medical Devices Agency, Tokyo) http://www.pmda.go.jp/pal-e.html
10.     Dati, F. The new European directive on in vitro diagnostics. Clin. Chem. Lab. Med. 41, 1289–1298 (2003)

About the Author

Mirella G.-Zulueta is a Senior Business Development Officer, jointly appointed to the Office of Technology Transfer & Industrial Liaison at the Mount Sinai Hospital & the Samuel Lunenfeld Research Institute, and to the Corporate Ventures Office at the Hospital for Sick Children in Toronto, by the cross-over program of OnSETT (The Ontario Society for Excellence in Technology Transfer). Mirella practiced clinical medicine as an M.D. in Spain, obtained a Ph.D. in Molecular Biology at The University of Southern California and did her postdoctoral training in Neuroscience at The Johns Hopkins University School of Medicine. Since then, she has worked in research and development, and in business development in the field of genomics-based drug discovery and proteomics-based biomarker discovery for biotech companies in the Bay Area, California. In her current position, her main focus is the development and commercialization aspects of biomarkers of human disease.


Figure 1. The ABCs of Biomarker Development. The multiple factors that need to be taken into consideration throughout the complex process of successful biomarker development.