NEO Shop Talk November 19th, 2017
CategoriesCategories Contact Us Archives NEO Main Site Search

Jul

07

Date prong graphic

What in Tarnation is Evaluability Assessment and Why Should We Care?

Posted by on July 7th, 2017 Posted in: Blog, Bloggers' Bookshelf, Practical Evaluation


Recently on Twitter I found a publication on evaluability assessment of impact evaluation. The tweet stated that an evaluability assessment would save time and money in the long run.

That’s good right? Evaluability was a new concept for me, so I read the small publication: Evaluability Assessment for Impact Evaluation: Guidance, Checklists and Decision Support, by Greet Peersman, Irene Guijt & Tiina Pasanen.  Obviously, you can read it for yourself, but here are some of the things that I was happy to learn.

What is evaluability???  Essentially it is ‘the extent to which an activity or project can be evaluated in a reliable and credible fashion’ (from the Organisation for Economic Co-operation and Development – Development Assistance Committee).

Why do we want to assess evaluability?

  • To catch any holes or errors in the logic of our evaluation plan – to ensure that our evaluation will effectively show the connection between the project and the impact or outcome we want to see.
  • To design an evaluation that is valid, efficient and useful, thus preventing wasted time and resources.
  • To ensure that the evaluation will do what the stakeholders need it to do in the long run.

When would an evaluability assessment take place? It would be most helpful to do an evaluability assessment as a part of your evaluation planning, before the project and data collection activities take place.

How does evaluability assessment affect outcomes-based evaluation?

The authors focus on three basic questions:

  1. Is it plausible to expect impact? Is it logical that what you are doing will lead to the outcomes you expect?
  2. Would an impact evaluation be useful and used? Have you clearly identified how the evaluation will be used by the stakeholders? Do your stakeholders find your evaluation methods credible?
  3. Is it feasible to assess or measure impact? Some outcomes data is really difficult and expensive to collect – can you reasonably expect people to collect the data that you want in the time frame that you need it? Is there good data that is already being collected that can be used? Is your evaluation plan within your budget?

I think we do many of these steps when planning our evaluations.  But using a checklist to make sure that all of them are included is really helpful- and the authors provide a checklist and decision support tool at the end of the publication.  It’s broken down so nicely that I added it to the NEO’s Tools and Resources page (under the Evaluation Planning tab).

Image of the author ABOUT Karen Vargas
As Evaluation Specialist for the NNLM Evaluation Office (NEO), Karen Vargas participates in evaluating the programs of the NNLM, assists people with their program and evaluation planning, and teaches classes on planning and evaluating library programs.

Email author Visit author's website View all posts by

Commenting (2)

Thoughts on "What in Tarnation is Evaluability Assessment and Why Should We Care?"

Cindy Olney Cindy Olney says:

These questions could be incorporated into developing logic models. Once it’s laid out, the group should consider these questions. This is a great tool!

Douglas Lord says:

I think this is an area that has been overlooked for too long; it’s all well and good to show gains in discrete areas of knowledge/skill etc., but if those data are neither reliable nor credible in a larger sense, the effort is tarnished.

Subscribe to our blog!

NEO’s Latest Tweet

Annual Archive

This project is funded by the National Library of Medicine (NLM), National Institutes of Health (NIH) under cooperative agreement number UG4LM012343 with the University of Washington.

NNLM and NATIONAL NETWORK OF LIBRARIES OF MEDICINE are service marks of the US Department of Health and Human Services | Copyright | Download PDF Reader