Residency Research Manual: Study Design

This section contains an overview of some of the most frequently used research designs and their advantages and disadvantages. Research however can take on many forms and often times a combination of these designs is used.

Study Design and Study Implementation

Having decided on the study question, you must now conduct a more in depth literature review. Initially, you will focus on articles written on a similar topic and articles that support your hypothesis. As you concomitantly work on your study design, you will also be searching for articles that support any tools being utilized (e.g. surveys, equipment). Review articles that discuss study design and methods. It is better to design a good study early in the process, then to read about how it should have been done later. Contact the support staff in the library on either campus for help with your literature search if needed (email: librarian@midwestern.edu. / phone: (630) 515-6200 for Il or (623) 572-3308 for AZ)

This is also the time to start thinking ahead about how you will collect the data, which data collection forms you will use, what software you will be using, how to enter this data and what analysis would be needed. For all these steps support is available to you in the form of workshops and weblinks. Consulting with research staff “early on” during the design and planning stage of your project will help minimize errors or the need for time-consuming recoding or re-entry of data later. Contact a statistician to discuss your research question and the measures you are using so you have a good idea about what kind of data you want to have in your database. Data collection and data entry is a step you only want to do once!!

Study Design

Study design is one of the most challenging parts of this process. It is also the backbone of your project. You will be attending a series of workshops which will provide more detail about the components of this process. In this section you will find some information to get you started.

Designing a research project may seem like a daunting task, especially if you have limited or no research experience. Seeking out help and guidance from your faculty research mentor and others with research experience early on in this process is paramount.

Where to begin? How do you get to the final product? What process falls in the middle? Approaching the design of your project in a systematic fashion will help clarify the process.

There are many different types of study designs and just as many different ways of describing/classifying these. Below are examples of frequently used descriptors of study designs. Many research studies will use variations of these or combine several types of designs. The main message is that the design you pick must be appropriate for the goals of the study and is in part determined by your research question.

Study Design: Is the blueprint of your study and includes:
What data to collect; Whom to collect data from; How to collect data; When to collect data; How to analyze data

One major distinction between designs focuses on the nature of the study.

1. Descriptive/Observational: seeks to explore associations and is often a first step before conducting an experimental study and helpful in defining hypotheses. Examples include: cohort studies (aka incidence or longitudinal studies), case-control studies, case reports and case series, and cross-sectional studies (prevalence).
2. Analytic/Experimental: seeks to test hypotheses and in clinical research often involve testing of interventions, new treatments and drugs. Examples include: randomized controlled trial, cross-over design. Peat et al. (2002) summarized some of the terms to describe research studies in the following table:

General terms to describe research studies

Another way of describing a study’s design is to consider the type of data used in answering the research question. Do you collect new information (primary analysis), use already existing information (secondary analysis) or complete a review article (tertiary analysis)? Each of these approaches has advantages and disadvantages. Collecting new data may take much more time and time is an important consideration in the completion of your residency research requirement. However, it also gives you more flexibility in determining who your study population is, what intervention you are studying, what type of data you want to collect, etc. Secondary and tertiary analyses might get you quicker from developing your research question to the stage of data analysis but you may soon come to find out that the information collected is not really a good measure of the variable(s) you are interested in nor is the data collected on the patient population of interest. Tertiary analysis may not satisfy the requirements of your residency research as each specialty has different expectations. Please consult with your faculty research mentor and/or Program Director if you have questions regarding this.

Study designs are also often classified according to their ability to assess association or causation.

Ability of studies in terms of relative strength for assessing causation or association

In this table, systematic review or meta-analysis is placed on the same level as randomized controlled trials which many consider to be the gold standard in clinical research. Meta-analysis refers to a process of systematically obtaining results from published articles and to a complex set of statistical procedures.

Published research studies use different methodologies. A meta-analysis conducted on a small number of published articles or on study results from small randomized controlled trials cannot be considered to be superior to evidence from a multi-center randomized control trial.

Research studies can also be categorized based on what they intend to accomplish.

Several designs deserve a more elaborate discussion as it is likely you will come across them more often.

Case Report or Case Series: is a description of an individual patient or group of patients with a specific disease or condition with a focus on disease development, process and/or exposure.

Advantages:

• allows for a detailed description of patient exposure
• allows for easier collection of comprehensive data on an individual or small group of patients.

Disadvantages:

• is based on the experience of few
• impossible to generalize to a population
• no control group, does not include a non-diseased group

A case report will in many instances not fulfill the requirements of residency research.

Correlational Study Design: attempts to establish the strength and direction of the association (relationship) between variables. This does not allow statements about cause and effect (causation). This is simply a snapshot in time.

Cross-Sectional Studies:

Advantages:

• relatively quick and inexpensive
• can provide good information on indicators of potential risk factors

Disadvantages:

• only provides you with an idea of the strength of a relationship between variables and the direction of the relationship but does not allow for statements of causal relationships.

Having brought in the variable of time provides the opportunity to introduce another method of describing studies. Research studies are often described as being either a retrospective and prospective study.

Cohort Studies: Follows participants over a period of time to calculate the rate at which new disease occurs and to identify risk factors. Cohort studies can be either retrospective or prospective depending on when study started and exposure occurred.

Below you will find a schematic representation of a prospective and retrospective cohort design.

Prospective cohort design

Advantages:

• maintain a temporal sequence
• are a good choice for rare diseases or rapidly progressing diseases
• allow for the study of multiple outcomes
• provides possibility of description of experience after exposure (rate of progression, natural history of disease)

Disadvantages:

• inefficient and expensive
• often requires long follow-up period
• requires need to measure confounders but those are often not known at start of study so there is a tendency to collect too much information

Several study designs can be described as experimental. Among them are the randomized and non-randomized clinical trials, cross-over designs and pre-and post-test study designs.

Randomized Clinical Trial: In a randomized clinical trial, subjects are randomly assigned to treatment groups. Randomization is a method of dividing subjects into groups with each subject having an equal chance of being assigned to each treatment or control groups. Randomization is based on probability and attempts Start of Study to eliminate “confounders”. If subjects are randomly assigned to different groups you expect these groups to be equal on average in terms of age, weight, ethnic composition, etc. You also expect and assume that those groups then are equal on average on certain characteristics that you are not measuring and may not even know ahead of time could influence your study results. You expect your groups to be equal with the exception of the exposure, the treatment related to the group they were assigned to.

A randomized clinical trial is a prospective study and includes a treatment and control group with subjects randomly assigned. Schematically it can be represented as follows:

Advantages:

• considered the gold standard
• highest level of evidence for intervention
• allows for conclusions of causality

Disadvantages:

• expensive
• involves lots of personnel
• often provides challenges in terms of recruiting patients and raises questions about generalizability of results

Non-Randomized Clinical Trial: Subjects are part of different treatment groups but are not randomly assigned. Design can be used to compare individuals who receive an intervention to those not receiving the intervention (i.e. receiving standard of care).

Advantages:

• more appealing in instances where one wants to remain in control over who gets a specific intervention
• easier to recruit when it is difficult to convince subjects of the need for randomization

Disadvantages:

• raises questions regarding the equality and comparability between groups

Pretest-Posttest Design: In this type of design, subjects serve as their own controls. A subject is measured at baseline, before an intervention and then again after the intervention.

Advantages:

• easier, more feasible than randomized control trial
• tracks same subjects over time
• subjects serve as their own controls
• good way of starting to establish the effectiveness of a treatment

Disadvantages:

• unable to determine if differences are due to treatment or other factors such as spontaneous recovery, history, etc.

Cross-Over Study Design: Every subject receives both the treatment (intervention) and control (placebo). Works best if assignment to treatment and control are done randomly and for interventions which do not have carry-over effects.

Advantages:

• great appeal to clinicians; provides direct evidence for a response to a treatment
• perceived as more ethical
• appeal to statisticians for the ability to conduct within-subject comparison of treatments
• efficient design, need half the number of subjects for adequate statistical power
• might improve recruitment as subjects may be more willing to participate because they are guaranteed to receive intervention and not only placebo or control

Disadvantages:

• carry-over effects can make ascertaining the effects of a particular treatment difficult
• can suffer from high drop-out rates
• cannot assess long term effects
• statistical analysis is more complex
• FDA still has preference for the Randomized Control Trials.