BetaGov Blog

Learning Corner: Don't Be Scared!

Posted on October 15, 2019     By Maureen Hillhouse & Angela Hawken

People screaming in fear

Any new venture can be scary, but some experiences get scarier over time because we become more knowledgeable about possible pitfalls and complications. It sometimes becomes clear that a trial might be more challenging than we initially thought: the "easy little" aromatherapy trial requires a nonexistent distribution system, a good randomization plan that accounts for variables that might affect outcomes, and encouraging survey responses of all participants. Each of these becomes a lesson in frustration. In short, to develop and conduct research requires an understanding of all the nuances of the environment and culture. While research often seems daunting, there is no cause for alarm—we (and your peer pracademics) can often help you address potential barriers to your trial!

From the BetaGov team, in the spirit of the spooky month, we share two of the “scariest” experiences we’ve had over the last few years:

  1. Not having someone at an agency take ownership of a trial is our worst nightmare. We need someone on site not only to have enthusiasm for the trial idea, but also to take leadership to ensure that all procedures are completed as designed. This means finding someone who is willing, able, and interested in seeing the trial through to the end. Our experience has shown that this is the most important aspect of a trial—without a champion who has the authority to encourage adherence to the protocol, the trial is destined for the trash pile! Very scary!
  2. Understanding randomization and what it means. We work with our partners to make sure that they understand the need for random assignment (in a randomized controlled trial design) and launch the trial only after developing a randomization plan that is practical but also ensures that the participants in each condition are similar at baseline. A rigorous comparison between those who get the intervention and those who don’t is predicated on truly random assignment to condition. On several occasions this has gone awry. When randomization is not followed, it becomes difficult, if not impossible, to interpret the trial results. One example is when we learned at the end of a trial that the randomization plan was “adjusted.” Staff thought that some of the clients would benefit more from the intervention and switched them into the intervention group. We have since adjusted our training to make sure that we reinforce the need for fidelity to condition assignment and a mechanism to monitor fidelity. Without it, a trial is useless. Not adhering to the randomization scheme is like trick-or-treating all night and having an empty bag of goodies when you get home! All work and no reward!

We are grateful that when we explain to most pracademics why these issues are important, you get it. You are as interested in generating evidence that is useful as we are. And you are a TREAT to work with.