{"id":5978,"date":"2017-09-08T10:11:08","date_gmt":"2017-09-08T17:11:08","guid":{"rendered":"https:\/\/news.nnlm.gov\/neo\/?p=5978"},"modified":"2019-09-24T10:13:27","modified_gmt":"2019-09-24T17:13:27","slug":"evaluation-questions-gps-for-your-data-analysis","status":"publish","type":"post","link":"https:\/\/news.nnlm.gov\/nec\/2017\/09\/08\/evaluation-questions-gps-for-your-data-analysis\/","title":{"rendered":"Evaluation Questions: GPS for Your Data Analysis"},"content":{"rendered":"<p style=\"text-align: left\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-6005\" src=\"https:\/\/news.nnlm.gov\/neo\/wp-content\/uploads\/sites\/3\/2017\/09\/stockfresh_2519513_gps-navigator-vector-illustration_sizeXS-300x192.jpg\" alt=\"GPS\" width=\"166\" height=\"106\" align=\"left\" hspace=\"10\" srcset=\"https:\/\/news.nnlm.gov\/nec\/wp-content\/uploads\/sites\/3\/2017\/09\/stockfresh_2519513_gps-navigator-vector-illustration_sizeXS-300x192.jpg 300w, https:\/\/news.nnlm.gov\/nec\/wp-content\/uploads\/sites\/3\/2017\/09\/stockfresh_2519513_gps-navigator-vector-illustration_sizeXS.jpg 433w\" sizes=\"auto, (max-width: 166px) 100vw, 166px\" \/><\/p>\n<p>Have you ever found yourself starring at a flip drive or folder full of evaluation data, wondering how the heck to make sense of it? \u00a0Maybe you collected a bunch of information as a requirement for a funded project. \u00a0Maybe a VIP handed you data files and tasked you with &#8220;producing a report on the findings.&#8221; Now, there you sit, trying to figure out how to move forward.<\/p>\n<p>You know what&#8217;s missing in this scenario? Evaluation questions. Data cannot provide you with answers if no one ever articulated questions.<\/p>\n<p>It is ideal, of course, to\u00a0write your evaluation questions before you design your data collection processes. Otherwise, to misquote the great Yogi Berra, &#8220;if you don&#8217;t know what your evaluation questions are, you might not answer them.&#8221; As with most evaluation planning, it&#8217;s best to get your stakeholders involved in question development, to be sure you answer <em>their<\/em> questions as well as your own.<\/p>\n<p>However, we don&#8217;t live in an ideal world most of the time. The good news is, you can introduce evaluation questions during the analysis process. \u00a0Whenever you suffer from data-based confusion, stop and think &#8220;Wait, what exactly do I need to know here?&#8221; \u00a0List your questions, then figure out how to analyze your data to answer them.<\/p>\n<p>For clarification, I&#8217;m not talking about the questions used to collect a single bit of information, like those used in surveys and interview guides.\u00a0 Evaluation questions are related to your program, broadly defining all the information you need in order to implement a program and assess its value.\u00a0\u00a0Specifically, your questions identify information to (a) plan well (b) conduct and adapt your activities as needed or (c) make informed decisions about the value of a program. These questions also set boundaries for your data collection methods, helping you collect only information that is useful to you. Ultimately, your evaluation questions\u00a0guide your data collection methods, analysis, and reporting.<\/p>\n<p><strong>Revisiting Sunnydale<\/strong><\/p>\n<p>To demonstrate sample evaluation questions, let&#8217;s revisit our <a href=\"https:\/\/news.nnlm.gov\/neo\/2016\/08\/26\/from-logic-model-to-proposal-evaluation-part-1\/\">From Dusk to Dawn Project<\/a>, the fictional health information outreach project to vampires featured in other NEO blog posts. \u00a0In case you are unfamiliar with our un-outreach to the un-dead, the image to the left provides a project summary. \u00a0Evaluation questions should be written for three key decision-making points in a program: planning; implementation; and outcomes\/value assessment. Here are examples of some evaluation questions I would use if I were evaluating the <em>From Dusk to Dawn<\/em>\u00a0program.<\/p>\n<p><em>Planning: What do we need to know to plan this program?<\/em><\/p>\n<p style=\"text-align: right\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-5979\" src=\"https:\/\/news.nnlm.gov\/neo\/wp-content\/uploads\/sites\/3\/2017\/08\/Dawn-to-Dusk-write-up-300x235.jpg\" alt=\"Dawn to Dusk write up with icon of a vampire and a brief description of the program that reads:The goal of the From Dusk to Dawn project is to improve the health and well being of vampires in the Sunnydale community. In order to reach this goal, The program offers Hands-on evening classes on the use of MedlinePlus and PubMed find health information and up-to-date research about health concerns. An overnight \u201cDusk-to-Dawn\u201d health reference hotline to help the vampires with their reference questions With these activities, we hope to see Increased ability of the Internet-using Sunnydale vampires to research needed health information Vampires using their increased skills to research health information for themselves and their broods Improved health of Sunnyvale vampires and Improved relationships between the vampire and human community of Sunnydale.\" width=\"541\" height=\"424\" align=\"right\" hspace=\"10\" srcset=\"https:\/\/news.nnlm.gov\/nec\/wp-content\/uploads\/sites\/3\/2017\/08\/Dawn-to-Dusk-write-up-300x235.jpg 300w, https:\/\/news.nnlm.gov\/nec\/wp-content\/uploads\/sites\/3\/2017\/08\/Dawn-to-Dusk-write-up-768x601.jpg 768w, https:\/\/news.nnlm.gov\/nec\/wp-content\/uploads\/sites\/3\/2017\/08\/Dawn-to-Dusk-write-up.jpg 814w\" sizes=\"auto, (max-width: 541px) 100vw, 541px\" \/><\/p>\n<ul>\n<li>What qualities does our program need to attract vampires?<\/li>\n<li>What community organizations currently serve vampires and might want to partner on this program?<\/li>\n<li>What challenges do we face in implementing our plans?<\/li>\n<\/ul>\n<p><em>Implementation: What are we doing and how well are we doing it?<\/em><\/p>\n<ul>\n<li>How many calls does the\u00a0<em>From Dusk to Dawn<\/em>\u00a0Hotline get per night from vampires seeking health information?<\/li>\n<li>How satisfied are the vampires with the training sessions?<\/li>\n<li>How do the reference librarians and instructors suggest we improve the program activities?<\/li>\n<\/ul>\n<p>O<em>utcomes What did we accomplish with this program?<\/em><\/p>\n<ul>\n<li>Did we meet our objective targets?<\/li>\n<li>What unanticipated positive or negative outcomes came from our efforts?<\/li>\n<li>What reactions do different stakeholder groups have to the outcomes of this program? (Stakeholders include vampires and their families,, Sunnydale city leaders, vampire-serving CBOs, and human residents)<\/li>\n<\/ul>\n<p>If the project team starts with these questions, we can develop questionnaires, interview guides, and other evaluation methods with precision, collecting the most necessary and useful data. \u00a0These questions also will frame our data analysis and structure our reports to stakeholders.<\/p>\n<p><strong>Other Examples<\/strong><\/p>\n<p>If you want more examples of evaluation questions, check out these resources:<\/p>\n<ul>\n<li><a href=\"http:\/\/ctb.ku.edu\/en\/table-of-contents\/evaluate\/evaluation\/evaluation-plan\/main\"><em>Developing an Evaluation Plan <\/em><\/a>from the Community Toolbox<\/li>\n<li><a href=\"http:\/\/www.betterevaluation.org\/en\/plan\/engage_frame\/decide_evaluation_questions\"><em>Specify The Key Evaluation Questions <\/em><\/a>from\u00a0Better Evaluation<\/li>\n<li><a href=\"https:\/\/nnlm.gov\/neo\/guides\/booklet1figures#fig8\">Figure 8: Community Assessment Questions<\/a>\u00a0\u00a0from <em>Getting Started with Community-Based Outreach<\/em>, which is Booklet 1 of the NEO&#8217;s <a href=\"https:\/\/nnlm.gov\/neo\/professional-development\/guides\"><em>Planning and Evaluating Outreach Evaluation Projects<\/em> booklet series.<\/a><\/li>\n<li><a href=\"https:\/\/nnlm.gov\/neo\/guides\/bookletTwo508#wk2\">Worksheet 2: Process Assessment Questions and Methods <\/a>from <em>Planning Outcomes-Based Outreach Programs<\/em>, Booklet 2 in the NEO series.<\/li>\n<\/ul>\n<p>It takes resolve to begin planning an evaluation by articulating evaluation questions, but it pays off BIG in the end. \u00a0If you are on teams or work groups initiating evaluation projects, press for the group to do their prep work. \u00a0Persuade them to start with evaluation questions.<\/p>\n<p>Repeat after me:\u00a0Friends don\u2019t let friends do question-free evaluation.<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;Note: link updated 9\/21\/2017.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Have you ever found yourself starring at a flip drive or folder full of evaluation data, wondering how the heck to make sense of it? \u00a0Maybe you collected a bunch of information as a requirement for a funded project. \u00a0Maybe a VIP handed you data files and tasked you with &#8220;producing a report on the&#8230; <a href=\"https:\/\/news.nnlm.gov\/nec\/2017\/09\/08\/evaluation-questions-gps-for-your-data-analysis\/\">Read More &raquo;<\/a><\/p>\n","protected":false},"author":2959,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[27],"tags":[],"class_list":["post-5978","post","type-post","status-publish","format-standard","hentry","category-blog"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p8ICUo-1yq","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts\/5978","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/users\/2959"}],"replies":[{"embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/comments?post=5978"}],"version-history":[{"count":35,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts\/5978\/revisions"}],"predecessor-version":[{"id":6085,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts\/5978\/revisions\/6085"}],"wp:attachment":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/media?parent=5978"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/categories?post=5978"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/tags?post=5978"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}