DIY Build Your Own Culture of Evaluation

Posted by nnlmneo on October 6th, 2017 Posted in: Blog

---

Our organization has a culture of evaluation.

Oooh, doesn’t that sound impressive? In fact, I confess to using that term, culture of evaluation, in describing the NNLM Evaluation Office’s mission. However, if someone ever asked me to explain concretely what a culture of evaluation actually looks like, it would have taken some fast Googling, er, thinking on my part to come up with a response.

Then I discovered the Community Literacy of Ontario’s eight-module series, Developing A Culture of Evaluation. In module 1, Introduction to Evaluation, they ground the concept in seven observable indicators seen in organizations dedicated to using evaluation for learning and change. (You can read their list on page 11 of module 1).

That led me on a hunt for more online resources with suggestions on how to build a culture of evaluation. I located some good ones.  Here’s an infographic from Community Solutions Planning and Evaluation with 30 ideas for evaluation culture-building that most nonprofits could adopt. John Mayne’s brief Building an Evaluation Culture for Effective Evaluation and Results Management describes what senior management can do to make (or break) an organization’s culture of evaluation. My investigation inspired me to think of ways we can all foster a culture of evaluation in our own teams and organizations.

Put Evaluation Information on Meeting Agendas

Embrace organizational learning and adopt evaluation information as your primary educational resource. Find ways to integrate performance and outcome measures into daily planning and decision making.  A good place to start is in staff or team meetings. Usage statistics, social media metrics, attendance or membership rates are examples of data that many organizations collect routinely that might generate good discussion about your programs. If you don’t have any formally collected data related to agenda topics, consider asking your team to collect some data informally.  Check out module 3, Collecting Data, for examples of both informal and formal data collection guidance. Module 5, Taking Action, has some practical examples of how you can share evaluation data and structure discussions. (I particularly like the template on page 9 of this module.)

Take Calculated Risks Using Evaluation Data

When planning programs, collect and synthesize evaluation data to get an overview of factors that support and challenge your likelihood of success. One of the best tools for doing this is a SWOT analysis (SWOT stands for Strengths, Weaknesses, Opportunities, Threats). This NEO Shop Talk post describes how to extend the traditional SWOT discussion to identify unknowns regarding your program success.  The SWOT analysis can both help you synthesize existing information about your customers and environment, as well as identify areas where you need more information. You might want to revisit module 3’s discussion on informal data collection to help when you lack existing evaluation information.

Report Findings Early and Often

Like cockroaches, exhaustive final reports will likely survive until the end of time. if you are truly committed to a culture of evaluation, however, you need to break with this end-of-project tradition and find opportunities to share findings on an ongoing basis. Data dashboards are one example of how to engage a broad audience in your organization’s evaluation data. However, they require time and expertise that may be out of reach for many organizations. One nice tip from the Community Solutions 30-ideas infographic is to make friends with your organization’s communication team.  They can help you find opportunities in publications, websites, and social media channels to share evaluation findings. Your job will be to add substance to the numbers. While quick facts can be interesting, it is better to talk about numbers as evidence of success.  You also should not be shy about publishing less stellar findings and explaining how your organization is using them to improve programs and services.

Engage Stakeholders in the Evaluation Process

A stakeholder is anyone who has a stake in the success of your program.  They should, and usually do, influence program decisions. It’s up to you to make sure they are engaging with evaluation information so they develop informed opinions and advice.  Rather than giving them well-synthesized findings in annual reports or presentations, engage them in the actual practice of analyzing data and evaluating programs.  NEO Shop Talk has a number of posts that can help you structure meetings and discussion with stakeholders about evaluation findings.  Check out these posts on data parties, audience engagement, and Liberating Structures.

Of course, a culture of evaluation requires foundational evaluation activities. I highly recommend all of the modules in Community Literacy of Ontario’s Developing A Culture of Evaluation modules. The content is succinct and easy to read, and relatively jargon free.The NEO’s booklet series Planning and Evaluating Health Information Outreach Projects is another how-to resource on the basics of evaluation.

Acknowledgements

The full citation for Mayne’s article is Mayne, J. (2008). Building an evaluative culture for effective evaluation and results management. ILAC Brief No. 20. Rome: Institutional Learning and Change (ILAC) Initiative. This paper was retrieved from the Better Evaluation blog.

The Building a Culture of Evaluation: 30 Ideas to Apply to Your Organization infographic created by Community Solutions Planning and Evaluation was found at the Point K Learning Center’s resources web page.