{"id":5164,"date":"2017-01-27T08:29:56","date_gmt":"2017-01-27T16:29:56","guid":{"rendered":"https:\/\/nnlm.gov\/evaluation\/blog\/?p=5164"},"modified":"2019-09-24T10:10:44","modified_gmt":"2019-09-24T17:10:44","slug":"failure-is-an-option","status":"publish","type":"post","link":"https:\/\/news.nnlm.gov\/nec\/2017\/01\/27\/failure-is-an-option\/","title":{"rendered":"Failure IS an Option: Measuring and Reporting It"},"content":{"rendered":"<p style=\"text-align: left\"><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-medium wp-image-5166\" src=\"\/neo\/wp-content\/uploads\/sites\/3\/2017\/01\/back-to-square-one-signpost-300x200.jpg\" alt=\"Back to Square One signpost\" width=\"300\" height=\"200\" align=\"left\" hspace=\"10\" \/><\/p>\n<p>Failure.\u00a0 We all know it\u2019s good for us.\u00a0 We learn from failure, right? In <em>Batman Begins<\/em>, Bruce Wayne\u2019s dad says \u201cWhy do we fall, Bruce? So we can learn to pick ourselves up.\u201d\u00a0 But sometimes failure, like falling, isn\u2019t much fun (although, just like falling, sometimes it is fun for the <em>other <\/em>people around you). Sometimes in our jobs we have to report our failures to someone. And sometimes the politics of our jobs makes reporting failure a definite problem.<\/p>\n<p>In the NEO office we like to start our meetings by reporting a recent failure. I think it\u2019s a fun thing to do because I think my failures are usually pretty funny.\u00a0 But Cindy has us do it from a higher motivation than getting people to laugh.\u00a0 Taking risks is about being willing to fail.<\/p>\n<p><strong>Why failure matters in evaluation<\/strong><\/p>\n<p>In general we are all really good at measuring our process (the activities we do) and not so good at measuring outcomes (the things we want to see happen because of our activities).\u00a0 This is because we have a lot of control over whether our activities are done correctly, and very little control over the outcomes.\u00a0 We want to measure something that shows that we did a great job, and we don\u2019t want to measure something that might make us look bad. That&#8217;s why we find it preferable to measure something we have control over. It can look like we failed if we didn\u2019t get the results we wanted, even if the work at our end was brilliant.<\/p>\n<p style=\"text-align: right\"><img loading=\"lazy\" decoding=\"async\" class=\"alignright size-medium wp-image-5167\" src=\"\/neo\/wp-content\/uploads\/sites\/3\/2017\/01\/sad-businesswoman-300x225.jpg\" alt=\"sad businesswoman\" width=\"300\" height=\"225\" align=\"right\" hspace=\"10\" \/><\/p>\n<p>But of course outcomes are what we really care about (<a href=\"https:\/\/nnlm.gov\/evaluation\/blog\/2016\/05\/20\/steering-by-outcomes-begin-with-the-end-in-mind\/\" target=\"_blank\" rel=\"noopener noreferrer\">Steering by Outcomes: Begin with the End in Mind<\/a>).\u00a0 They are the \u201cwhat for?\u201d of what we do.\u00a0 What if you evaluated the outcomes of some training sessions that you taught and you found out that no one used the skill that you taught them.\u00a0 That would be sad and it might look like you wasted time and resources.\u00a0 But on the other hand, what if you <em>don\u2019t<\/em> measure whether or not anyone ever uses what you taught them, and you just keep teaching the classes and reporting successful classes, never finding out that people aren\u2019t using what you taught them.\u00a0 Wouldn\u2019t that be the <em>real<\/em> waste of resources?<\/p>\n<p><strong>So how do you report failure?<\/strong><\/p>\n<p>I think getting over our fear of failure has to do with learning how to report failure so it doesn\u2019t look like, well, failure.\u00a0 The key is to stay focused on the end goal: we all really want to know the answer to the question \u201care we making a difference?\u201d\u00a0 If we stay focused on that question, then we need to figure out what indicators we can measure to find the answer. If the answer is \u201cno, we didn\u2019t make a difference\u201d then how can we report that in a way that shows we\u2019ve learned how to make the answer \u201cyes?\u201d How can we think about failure so it\u2019s about \u201clearning to pick ourselves up?\u201d or better yet, contributing to your organization\u2019s mission?<\/p>\n<p>One way is to measure outcomes early and often. If you wait until the end of your project to measure your outcomes, you can\u2019t adjust your project to enhance the possibilities of success.\u00a0 If you know early on that your short-term outcomes are not coming out the way you hope, you can change what you\u2019re doing.\u00a0 So when you do your final report, you aren\u2019t reporting failure, you\u2019re reporting lessons learned, flexibility and ultimately success.<\/p>\n<p><strong>Here\u2019s an example<\/strong><\/p>\n<p>Let\u2019s say you\u2019re teaching a series of classes to physical therapists on using PubMed Health so they can identify the most effective therapy for their patients.\u00a0 At the end of the class you have the students complete a course evaluation, in which they give high scores to the class and the teachers.\u00a0 If you are evaluating outcomes early, you might add a question like: \u201cDo you think you will use PubMed Health in the next month?\u201d\u00a0 This is an early outcome question.\u00a0\u00a0 If most of them say \u201cno\u201d to this question, you will know quickly that if you don\u2019t change something about what you\u2019re doing in future classes, it is unlikely that a follow-up survey two months later will show that they had used PubMed Health.\u00a0 Maybe you aren\u2019t giving examples that apply to these particular students. Maybe these students aren\u2019t in the position to make decisions about effective therapies. You have an opportunity to talk to some of the students and find out what you can change so your project is successful.<\/p>\n<p><strong>Complete Failure<\/strong><\/p>\n<p>You&#8217;ve tried everything, but you still don&#8217;t have the results you wanted to see.\u00a0 The good news is, if you&#8217;ve been collecting your process and outcomes data, you have a lot of information about why things didn&#8217;t turn out as hoped and what can be done differently. Reporting that information is kind of like that commercial about how to come in late to a meeting. If you bring the food, you\u2019re not the person who came late, you\u2019re the person who brought breakfast.\u00a0 If you report that you did a big project that didn\u2019t work, you\u2019re reporting failure.\u00a0 If you report that things didn\u2019t work out the way you hoped, but you have data-based suggestions for a better use of organizational resources that meet the same goal&#8211;then you\u2019re the person who is working for positive change that supports the organization, and have metaphorically brought the breakfast. Who doesn\u2019t love that person?<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Failure.\u00a0 We all know it\u2019s good for us.\u00a0 We learn from failure, right? In Batman Begins, Bruce Wayne\u2019s dad says \u201cWhy do we fall, Bruce? So we can learn to pick ourselves up.\u201d\u00a0 But sometimes failure, like falling, isn\u2019t much fun (although, just like falling, sometimes it is fun for the other people around you)&#8230;. <a href=\"https:\/\/news.nnlm.gov\/nec\/2017\/01\/27\/failure-is-an-option\/\">Read More &raquo;<\/a><\/p>\n","protected":false},"author":2959,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[27],"tags":[],"class_list":["post-5164","post","type-post","status-publish","format-standard","hentry","category-blog"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p8ICUo-1li","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts\/5164","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/users\/2959"}],"replies":[{"embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/comments?post=5164"}],"version-history":[{"count":3,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts\/5164\/revisions"}],"predecessor-version":[{"id":6509,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts\/5164\/revisions\/6509"}],"wp:attachment":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/media?parent=5164"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/categories?post=5164"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/tags?post=5164"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}