5 Things I Found Out as an Evaluation Newbie

Posted by nnlmneo on March 10th, 2017 Posted in: Blog

---

Since joining the NEO in October, I have learned a lot about the world of evaluation. Here are 5 things that have made me rethink how I approach evaluation, program planning, and overall life.

#1: Anyone can do evaluation
Think about a project that you are working on at work. Now take out your favorite pen and pad of paper, or open a new blank document, and write What? at the top of the page. Give yourself a few minutes to write or type out a general outline of the project. Do the same for the questions So What? and Now What? Reflect on why the project is important to your organization’s mission, and what will you do with any new found information from the project. Finished? Congratulations, you’ve just taken your first step as a budding evaluator by engaging in some Reflective Practice.

This first step does not mean you are an evaluation guru. It takes more than just a reflection piece to create a whole evaluation plan (actually, just 4 steps). What I hope you take away from this exercise is that every project could use some form of evaluation, and that there is no hocus pocus involved in evaluation. All you need is a team willing to put in the effort to create “an evaluation culture of valuing evidence, valuing questioning, and valuing evaluative thinking” (Better Evaluation). I am sure you even have one of the most basic tools you can use for evaluation, which leads me to #2.

#2: Excel is your best friend
I will not deny that Tableau, Power BI, and other really cool Data Visualization and Business Intelligence software is out there. There’s also R, for those who are looking for another programming language to conquer. If you are working at a small library, or a non profit, it might be hard to get the training or the funds for such software. Enter Excel. You can do a lot of neat things with Excel. A quick search for Excel on Stephanie Evergreen’s blog will result in many free tutorials on how to make interesting (and useful) charts in Excel. You can even make pivot tables, which can help you easily summarize complicated sets of data. Excel might not be the best tool for data visualization, but it’s a tool that many of us already have.

#3: This isn’t your grade school report card
I still remember the terror I would feel the day that report cards would come out. I should not have been afraid, because I usually received great grades. What always terrified me was the uncertainty of how my teachers reached the resulting grade. If the teacher was nice, he or she would explain how the grade was calculated, but most of the time I was left with a report card with no comments. If I wanted to strive for a better grade, I would have to arrange a meeting with the teacher. That never happened because I was very shy, and the prospect seemed more terrifying than getting the report card!

It can be scary to think about an evaluation program. What if it doesn’t come out well, and you get a “bad grade”? I’m here to tell you that the terror won’t be there, because you are not a student waiting for an ambiguous letter grade. You are in the teacher’s seat, and you get to decide what it means to succeed and what it means to fail. You have full control over the parameters of your evaluation! This does not guarantee success, but it does give you a fair shot at succeeding.

#4: Really, it’s ok to fail
Ever since I have started working with the NEO, I’ve been confronted with failure. We start most of our meetings by retelling our most recent failures, like how we forgot to put something on our Outlook calendar or could not get something to work. We’ve even written a blog post about it! I call this a failure-positive work environment. Instead of beating ourselves up about these little failures, we learn from them and carry on.

I’ve found myself reflecting on my recent work at a nonprofit and how I’ve approached failures in the past. To put it bluntly, I haven’t done well with failure. In fact, my approach to failure has usually been embarrassment, guilt, and eventual burnout. I see now that these feelings, though hard to ignore, are completely unproductive. They are also easy to prevent. If you have an evaluation plan in place, you can turn a failure into just another data point on a path towards success. As Karen wrote in the blog post about failure, “Reporting [a failure] is kind of like that commercial about how to come in late to a meeting. If you bring the food, you’re not the person who came late, you’re the person who brought breakfast.”

#5: Do not ignore outcome evaluation!
It took me a while for this information to sink in, but there are multiple ways to evaluate a program. Process evaluation assesses how you did something, “Are you doing what you said you’d do?” Outcome evaluation is a bit different, as it tries to answer the question of whether the program achieved its goal, or “Are you accomplishing the WHY of what you wanted to do?” When I think about these two types of evaluation, it’s tempting to focus on the process evaluation because I have more control over the process than the outcomes. I can plan a fantastic program, and “pass” a process evaluation. The same plan can “fail” an outcomes evaluation if people were not receptive to the program. Before you forgo your outcomes evaluation plan, remember pointers #3 and #4: you are in charge of the parameters, and failure isn’t the end of the world. Prepare an outcomes evaluation plan knowing that whatever happens, you’ll be able to use the information in the future. Also, remember that we have worksheets to help you write out any evaluation plan.

I hope you’ve found my reflections helpful in your evaluation planning. Let me know your favorite takeaways in the comments!

Photo credit: Kerry Kirk.