Full Citation: Pearce-Smith N. A randomized controlled trial comparing the effect of e-learning, with a taught workshop, on the knowledge and search skills of health. EBLIP 2006; 1(3):44-56.
The randomized controlled-trial design is a method that can yield compelling evidence of a project’s success.Pearce-Smith’s article provides a clear example of how to use this design as well as the stumbling blocks encountered when using it in an applied setting. There are several problems in using the controlled trial approach as an evaluation method. Often (though not in the case of this article) random assignment to a “control group” and the “experimental group” raises ethical concerns if you think one approach is superior. Controlled-trial studies in applied settings also can result in limited sample sizes and drop-out rates that significantly affect interpretation of results. Not surprisingly, this study yielded no conclusive evidence because of its final sample size of 15. Finally, controlled-trial approaches give minimal information that can be used for program improvement or decision-making, making them inadequate to use alone for evaluation purposes. However, as part of a mixed-method approach, the controlled trial is unbeatable for assessing some outcomes if it can be conducted correctly.
This article has two contributions of interest to those planning an evaluation. First, it provides an example of the controlled-trial approach that could be replicated if the conditions are right. Second, it provides a nice example of how to design a knowledge-and-skills assessment that could be used for evaluation even if you are not conducting a controlled trial. You can read the article at http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/54/155