How I Learned to Stop Worrying and Love Logic Models (The Chili Lesson)

Posted by nnlmneo on October 28th, 2016 Posted in: Blog

---

By Michelle Malizia, Director of Library Services for the Health Sciences, University of Houston

I’ll start with a full disclosure: I am a late convert to logic models. Many years ago, I worked in a department that, for a period of time, became governed by logic models. This experience made me fear… no, hate… logic models.  Several years later, through external workshops and assistance from the NN/LM Evaluation Office, I was introduced to the tremendous value of logic models.

My closest personal analogy relates to my feelings about chili. I grew up eating my mother’s chili, which basically consisted of cans of many different types of beans floating in a type of broth. I hated it. When I was 22 years old, I had no choice but to eat someone else’s chili. This chili had lots of ground beef and spices. It was delicious. Then it occurred to me, my mother’s chili was my only frame of reference for chili. I didn’t dislike chili – I disliked my mother’s chili.

And so it goes with logic models. Once I learned a different way to make and apply them, I became a dedicated user. I now design logic models whenever I plan a new service, activity or initiative.

In 2014, I was hired as the Director of Library Services for the Health Sciences at the University of Houston (UH). In 2017, UH will have its first medical library and my task is to plan the services for the new facility. Of course, I turned to logic models because they provided the framework of not only what I am planning but why I am planning each service and ultimately, how I will evaluate if I achieved the goal.

When I started with my logic models, I was tempted to begin with the activity. I had to remind myself that it is more important to document what I hope to accomplish by that activity (i.e. outcome).  Think about it: Why do librarians teach PubMed classes? Why do librarians want to be embedded in a nursing class? Why do so many libraries provide liaison services? Many of you are probably thinking: “That’s easy, Michelle. We do those things to better serve our customers.” My response is:  How do you know those activities better serve your customers?  How can you prove it to your stakeholders?  That is the reason you should start with your outcomes rather than activities.

For example, my new library will be providing assistance with NIH Public Access Policy compliance. When I developed my logic model, I called upon my inner 3-year old to ask the question best asked by toddlers:  Why.  Because I have a creative side, I use the software Visio (a Microsoft Office software) to create my logic models. It allows me to visually see connections between activities. The chart below shows  a portion of my logic model.

 

As you can see, my long-term outcome for this activity was to ensure that UH retains and receives NIH Grants. If UH researchers don’t comply with the NIH Public Access Policy mandate, their current and future funding is in jeopardy. The intermediate outcome leading to the long-term outcome is increased compliance with the policy. In order to increase compliance, I need to make researchers aware of the policy and how to comply.  That’s when I was able to determine the best methods for me to accomplish those outcomes. For my university environment, the best way to achieve these outcomes is through workshops and consultations.

Now that I knew the “what and the why,” I needed to determine the how.  How would I know if I accomplished my goals? Again, I turned to Visio to visualize how I could assess if I achieved my outcomes.

My final step was to determine my measurable indicators. For example, in the case of workshops, my indicator was “% of workshop attendees who reported being more knowledgeable about how to comply with the policy.”  My target was 85% of attendees. To evaluate this outcome, I would use a pre- and post-test.

 

My overall work with logic models led to a pleasant surprise. Mid-way through my process, UH Libraries adopted a new strategic plan. Strategic plans are usually written in terms of goals. Some of my colleagues tried to feverishly determine where their activities fit into the library’s overall goals. Because I had already determined my outcomes, it was easy to slot my activities into the library’s overall plan.

If you have had a previous bad experience creating logic models, try it again. Ask the NEO for assistance and look at their extremely helpful guides. Like me, you may finally realize that logic models are worth the time and energy. Remember, there are many different types of chili.  Find the one that you like best.

NEO note: The evaluation field has come a long way in discovering new, less painful approaches to creating and using logic models.  If, like Michelle, you had bad experiences years ago with logic models, you might want to give them another chance.  You can learn one approach through the NEO booklet Michelle mentioned, which is  Planning Outcomes-Based Projects (Booklet 3 in our Planning and Evaluating Health Information Outreach Programs series). For alternative approaches, check out our NEO Shop Talk blog entries  Logic Model for a Birthday Party and An Easier Way to Plan: Tearless Logic Models.