Elegantly Simple Evaluation: The Engage for Health Pre/Post Assessment

Posted by nnlmneo on July 14th, 2017 Posted in: Blog

---

I wrote a post in May 2016 about a participatory evaluation conducted with a group of public librarians from the Engage for Health program, a statewide collaboration led by the NNLM Middle Atlantic Region (MAR) and the Pennsylvania Library Association (PaLA).  This was a community education program, offered by 17 libraries throughout Pennsylvania, which taught community members how to take an active role in their own health care. The training sessions introduced them to the National Library of Medicine’s MedlinePlus and presented techniques for  managing their health issues.  The NNLM MAR recently published a web page providing access to all Engage for Health materials, which include a Word version of the pre/post assessment form used to assess the program outcomes. The Engage for Health evaluation form is the topic for today’s blog post.

The form itself is short and simple, but was carefully crafted with significant input from the  Engage for Health librarians. It is a pre/post assessment form, completed by 88% of the training participants. Below are some highlights of the Engage for Health assessment method  that I think are particularly noteworthy.

It was developed directly from the program logic model.  The Engage for Health librarians developed a logic model to identify the positive changes they hoped to see in participants who enrolled in the sessions.  Specifically, the librarians said their participants ideally would report, at the end of training, that they knew how to find trustworthy online health information and that they intended to implement health management strategies introduced in the session. (If you aren’t familiar with logic models, here’s a list of NEO Shop Talk logic model posts about the topic.) As you can see,  the questions on page 2 of the evaluation form directly relate to these outcomes.

The pre-assessment information allowed a more valid interpretation of the post-session findings. The questions on the first page of the form (the pre-assessment page) allowed us to get a baseline measure of participants’ engagement in their own health care. We learned two very important things about the participants from our preliminary assessment.  First, they were highly motivated to manage their health. Yes, one would assume that a class like ours would attract participants already engaged in their own health care. However, we documented that assumption and the information affected how we interpreted the post-assessment findings.  Furthermore, the majority said our class improved their health management skills, so the program did offer this health-savvy group some valuable new techniques.   Second, we discovered that almost none of the participants had never used MedlinePlus before attending the class. We knew, therefore, that most of the 70% who said they intended to use MedlinePlus were new adopters of the resource.

The assessment form doubled as a good teaching tool.  The pre-assessment form primed the participants for the information to be presented in the class. They reflected on how they managed their health appointments and the form’s mention of MedlinePlus peaked their interest in a resource that most of them had never used.   The post-assessment engaged them in reviewing and reflecting on the main points of the training.  One open-ended question asked them to  “List one or two things you will do differently as a result of today’s class,” reinforcing in their memories  a couple of the strategies that resonated with them.

The pre/post assessment allowed anonymity. Pre/post assessments have the advantage of showing changes in individuals over time, but they often require participant names or identifiers to match data across time periods.  By printing the pre-assessment and post-assessment forms on two sides of a single sheet of paper, we did not need participant identifiers and, therefore, could protect anonymity. This increased the likelihood that our participants would  provide candid answers to our questions. (If you want to use this anonymous method, be sure to read the Engage for Health Pre/Post Evaluation Instructions.

There is always room for improvement in any evaluation method, and the  Engage for Health librarians identified changes they would make to the first version of the assessment form. For example, a few eyebrows were raised over the 45% of participants who said they could easily find online health information. The Engage for Health librarians suspected that participants were using search engines like Google and probably turning up health information from sketchy sources. They suggested that future assessment forms ask participants where they got their health information.

Overall, this pre/post assessment method was a hit with the Engage for Health public librarians.  In our data party discussions,  many said they planned to adapt the method to evaluate other programs offered in their libraries. That was an Engage for Health evaluation finding I particularly liked!

About Elegantly Simple Evaluation posts: The NEO Shop Talk uses this series to feature simple evaluations that produce highly useful information.  We are particularly interested in projects that promote health information literacy, library programs, or outreach related to National Library of Medicine resources.  If you have a project to recommend for a future blog post, please send an email to nnlmneo@uw.edu.