A Practical Guide To Post-Marketing Observational Studies (PMOS)

---

By John S. Sampalis, PhD

In the spectrum of designs available for the evaluation of medical and, specifically, pharmaceutical interventions, the Post-Marketing Phase IV or Observational Study has been considered as the black sheep of the family.

Often the results of such studies are given the lowest possible weight for evidence-based decisions regarding efficacy and safety, while Phase II and III randomized controlled trials are assigned the highest weight.

Pharmaceutical companies, health care providers and policy decision makers, however, are equally often perplexed when the results of the Phase II and III controlled trials are not observed in a real-life setting.

The reasons for the discordance between the results of Phase II/III controlled clinical trials and real-life clinical practices have been recently deciphered and well-documented and include the selection of patients and controlled setting of the Phase II/III studies that are not replicated in real life. In fact, it is now accepted that the patients participating in Phase II/III trials are very different from those actually receiving the treatment in real-life.

Issues related to compliance, adherence and accessibility of care are also very important in causing this discordance. As a result, it is now well-accepted that while Phase II/III trials provide evidence of efficacy and safety in an ideal setting, Post-Marketing Phase IV and Observational studies provide evidence of effectiveness, tolerability and safety in real-life routine clinical practice.
Although Phase IV and Post-Marketing Observational Studies could be grouped together as Post-Marketing Studies, they have important differences.

Phase IV studies are used to provide ongoing evidence of effectiveness and safety for the target population or important subgroups of patients and usually follow a protocol for treatment with well-defined patient selection criteria. Study designs employed in Phase IV trials include single prospective cohort, treatment cross-over or titration within the same cohort and parallel group randomized blinded or open-label.

While these studies are conducted according to a protocol that is compatible with the best treatment regimen, patient selection and follow-up better emulate real-life settings when compare to Phase II/III trials. In addition, outcome measures used in Phase IV studies may include assessments such as quality of life, functional capacity, satisfaction with treatment and health economics that are not collected during routine clinical practice. Finally, in Phase IV studies the drug is provided free of charge by the sponsor.

Post-Marketing Observational Studies on the other hand are true epidemiological studies in which patients are 'observed' while they are treated with the drug in a routine real-life setting. In these studies there are no patient selection criteria other than the requirement of being treated with the drug under study.

Patient follow-up and outcome assessment is not defined by a study protocol but is according to the actual routine practice of the treating physician and is affected by the patient's compliance with the follow-up schedule and time or scheduling limitations of the patient and physician. Given that in the PMOS the drug is not supplied free of charge by the sponsor, but is acquired through insurance coverage or paid by the patient, economic factors affecting accessibility to the drug may affect compliance with treatment and consequently effectiveness.

The observation of the drug performance in the real-life setting would also provide data on the actual safety and tolerance. These are factors that would affect the effectiveness of the drug in a real-life setting from the population perspective. These are the pivotal points that should determine the overall risk/benefit of the drug from the population perspective. And in the final analysis, this is the only perspective that matters.

In summary, the role of each phase of drug development and testing from Phase I to Phase IV and PMOS are distinct and they each serve to fill in a well-defined part of the knowledge gap. None is more useful or valuable than others and none should be ignored. In fact, all of these studies should be conducted with the same, highest possible scientific vigor. Phase I studies ensure that the drug is safe, Phase II studies demonstrate efficacy when compared to no treatment, Phase III studies demonstrate efficacy in comparison to the current standard or competitors, Phase IV studies demonstrate effectiveness in less controlled settings and specific subgroups while providing data on additional outcome measures, and finally, PMOS demonstrate effectiveness in a non-controlled real-life setting.

Data on safety and tolerance are collected throughout the entire program; however, the small number of highly selected patients used in early phases may underestimate the incidence and severity of adverse events. Late Phase III and Phase IV studies provide safety data when the drug is used according to the approved conditions or under indication, while PMOS provide safety and tolerance data when the drug is used in real-life under completely non-controlled situations. Safety signals generated in later stages and PMOS are more relevant to patients, governments, and third party payers since they represent the population based risk of the drug.

The PMOS, therefore, has an important place in the comprehensive development and testing of a drug and, when conducted properly, provides valuable information that completes the cycle. However, despite its simplicity, the PMOS is the most difficult of all types of studies to conduct. The lack of strict patient selection criteria, uncontrolled treatment protocol and influence of real-life conditions affecting compliance could result in serious challenges for study management and statistical analysis.

As a result, despite the accepted need for PMOS, these studies are often frowned upon by decision makers and avoided by the pharmaceutical industry. The tainted reputation of PMOS as seeding trials elicited by the inappropriate practices of marketing divisions and marketing companies has made matters even worse and has raised serious concerns about these studies whose value has been accepted by health organizations, including the FDA. At this point in time, the health care stakeholders including the pharmaceutical industry and regulators are faced with the serious challenge of designing and executing post-marketing studies that could address the need for knowledge and complete the cycle of drug testing by providing real-life evidence for safety and effectiveness.

The following sections outline some basic guidelines for the development and implementation of PMOS.

The PMOS is not a marketing trial or exercise. Therefore, it should be developed and implemented by the clinical research divisions of the pharmaceutical companies and not by the marketing divisions. However, PMOS could be beneficial to marketing by demonstrating effectiveness and safety.

The purpose of the PMOS is to generate evidence regarding effectiveness and safety and not to increase market shares.


The following is a list of essential ingredients of a PMOS:

1) Valid scientific basis:
There should be a clear Rationale and Research Questions with well-defined Research Hypotheses that address a gap in the current knowledge. The primary research hypothesis should be driving the sample size of the study.

2) Rigorous epidemiological methods:
Study design should be based on random selection of physicians and patients. Sample selection should be representative of the target population and not marketing driven.

3) Physician reimbursement for data collection not patient enrollment.
Physician/investigators should not be compensated if the patient data are not complete. Any patient treated with the study drug is eligible for inclusion in the study.

4) Selection of outcome measures:
Clinically relevant outcome measures that are used in routine care and influence decision making of health care providers.

5) Statistical consideration:
The statistical analysis must be well-defined and described. Selection of statistical methods must be such that the study objectives are addressed. There should be control for confounding and consideration for subgroup analysis.

Sample size must be justified for power of between 80 per cent and 90 per cent and 5 per cent significance for the entire sample and all subgroup analyses.

6) Study protocol:
Developed and written according to the accepted standards for epidemiological studies.

7) Informed consent:
All patients must sign an informed consent prior to enrollment in the study. The informed consent should comply will all regulatory requirements.

8) Case report forms:
All data must be recorded on case report forms and signed by the treating physicians.

9) Data management:
Data management must adhere to established guidelines with quality assurance implemented especially in the validation of safety data.

10) Independent ethics review board:
The study must be submitted and approved by a local or central ethics review board that will ensure protection of patients' interests.

11) Scientific value:
Research that is not disseminated by publication in a high quality, peer review journal is conducted in a vacuum and has minimal impact. There should be a clear and precise plan for publication and the research must have publication potential. The latter will be determined by the value of the research questions addressed.

12) Independence from the sponsor:
Participating physicians cannot be accessed or have access to the sponsor. There is no payment to physicians for treating the patients with the drug under study. Instead physicians are paid for completing the Case Report Forms and should receive payment only when data are received.

Participating physicians are randomly selected to represent the target population. Physician selection must be random and independent of the sponsor. The participating physician list is never disclosed to the sponsor prior to completion of the study and is only disclosed for auditing purposes.

The sponsor is not involved in any way in the implementation of the study. Study execution should be conducted by a third party, preferably affiliated with an academic institution. Funding should be in the form of a grant.

Since the sponsor will fund the study, some sponsor involvement, at arm’s length, ensuring proper execution should be implemented. Establishment of an independent steering or advisory committee with sponsor representation could be the best approach.

The red flags: The following are indicators of a poorly designed PMOS that should be avoided.

1) Poorly written protocol:
There is no scientific rationale, well-defined research questions, and/or study hypotheses.

2) Vague or inappropriate study design:
Physician and patient selection is non-random. The study is aimed at converting or switching patients from competitor using market access and prescribing profile to determine sample selection procedures.

The outcome assessments are inappropriate, clinically not relevant or cannot be ascertained with validity (example: recall bias).

Safety assessment is based on passive surveillance without physician involvement and review for causality, intensity, action taken and resolution of the adverse event.

3) Erroneous or no statistical methods:
The statistical methods are vague or are not appropriate for addressing the study objectives. This indicates that the development of the study was not undertaken by trained researchers and without the consultation of biostatisticians.

4) Patient incentives:
Patients should not be enticed to participate in the study. The following should be avoided:
-Payment to patients
-Support programs
-Call in numbers
-Access to services that would not be available if the patient did not participate in the study
-Lotteries or other financial incentives

5) Poor study execution:
There is poor adherence to the study protocol on behalf of physicians and patients with inadequate quality assurance.

6) Lack of patient consent and ethics board review and approval:
If the study protocol and consent form have not received full ethics approval, the patient’s rights may be violated.

7) The CRO is not a research organization:
Observational studies require epidemiological background for the development, execution and data analysis. These have been the tools of epidemiologists for centuries.

However, market access consultants and drug distribution companies have emerged as providers, conducting clinical research with Phase IV and PMOS being their main targets. The lack of trained researchers and of credibility gained through training and academic affiliations will raise concerns regarding the validity and minimize the potential impact of the study and of the results. Sponsors should allow the experts to conduct these studies if they are to be respected by the other health care stake holders.

In summary, PMO studies will provide valuable evidence in support of real-life effectiveness and safety, thus completing the comprehensive cycle of drug development and testing. However, unless these studies are conducted correctly by highly trained and credible scientists, their potential impact will be minimized and the overall effect may be harmful to patients and pharmaceutical companies.

John S. Sampalis, PhD, president of JSS Medical Research (Westmount, QC), is a clinical epidemiologist with undergraduate training in microbiology, immunology, and neuroscience and graduate training in clinical epidemiology and biostatistics.
Sampalis is also a tenured associate professor of surgery, and epidemiology and biostatistics at the faculty of medicine at McGill University (Montreal, QC). He is recognized as one of Canada's leading epidemiologists, as well as the top trauma
researcher in Canada.