{"id":5795,"date":"2017-06-09T12:43:38","date_gmt":"2017-06-09T19:43:38","guid":{"rendered":"https:\/\/news.nnlm.gov\/neo\/?p=5795"},"modified":"2019-09-24T10:10:44","modified_gmt":"2019-09-24T17:10:44","slug":"the-cookie-exercise-setting-criteria","status":"publish","type":"post","link":"https:\/\/news.nnlm.gov\/nec\/2017\/06\/09\/the-cookie-exercise-setting-criteria\/","title":{"rendered":"The Cookie Exercise: Setting Criteria"},"content":{"rendered":"

\"Chocolate<\/a>What do Oreo cookies have to do with evaluation?<\/p>\n

In program evaluation, we want to know the extent to which a program is successful.\u00a0 If we\u2019re evaluating an activity\u00a0we\u2019re doing at our job, we might be trying to figure out if more money should be put towards future iterations of that activity or if it should be discontinued. Or maybe there are ways the activity\u00a0can be improved to achieve our desired outcomes. (\u201cBut what about the cookies?\u201d)<\/em><\/p>\n

Evaluating your data is dependent on having something to compare it to. If we don\u2019t have something to compare our data to, we are just describing it– we aren\u2019t able to know if it matters or not. If we don\u2019t know that our project isn\u2019t meeting our criteria, we won\u2019t know where to improve. (\u201cUm… the cookies?\u201d)<\/em><\/p>\n

In order to get people to understand how to create a set of criteria and standards, we use this yummy exercise* with Oreo-type cookies that we use in the NEO’s workshop on analyzing evaluation data. (\u201cFinally, the cookies!\u201d).\u00a0<\/em>Here\u2019s how it works.\u00a0 We tell people that they are on a council to select the next Girl Scout Cookie. The decision has already been made that the next cookie will be a chocolate vanilla cream cookie, like an Oreo.<\/p>\n

Participants need to create a set of criteria on which the cookies will be judged, like: crunchiness of the chocolate cookie, smooth texture of the cream filling, flavor of chocolate cookie, etc. Then for each criteria we ask everyone to establish a standard that the cookie must perform at or above for it to be qualified to be selected, on a scale of 1-10.\u00a0 So for example, if you say that the cream filling texture must be smooth, and you give that a 6, and you say that the chocolate cookie part must be extremely chocolaty, and you give that an 8, then, in order for this cookie to be in the running, it must have at least a 6 for creaminess and at least an 8 for chocolaty-ness.<\/p>\n

\"\"<\/a><\/p>\n

In the class we actually give everyone two or three different brands of chocolate vanilla cream cookies and ask them to go through the exercise.\u00a0 As a note: if you\u2019re planning to do this, it\u2019s harder than you might think to find cookies that don\u2019t have a recognizable mark in the design that tells you exactly who makes them. If the cookie says \u201cOreo\u201d right in the middle of the cookie, it might bias the cookie tasters.<\/p>\n

After this experiment, we ask people in the class to discuss their results. Which cookie did best overall? Did any of your cookies meet your minimum requirements? In addition, think about what kinds of external factors might be important. \u00a0Maybe your stakeholders are looking for a gluten free cookie, or a cookie that doesn\u2019t have palm oil in it.<\/p>\n

I think this is a fun project that you could try on real-life things other than cookies to get used to it.\u00a0 Maybe you want to do a chart like this when deciding what school you want your child to attend, or whether or not you want to finish reading a book (speaking as a person who feels guilty putting a book down even when I dislike it).<\/p>\n

If you want to use our class materials to do this exercise, they are located here https:\/\/nnlm.gov\/classes\/dataanalysis<\/a>\u00a0and the cookie assignment is called Data Analysis Exercise 1: Logic of Eval<\/em> (it is a Word document).<\/p>\n

*This cookie exercise is adapted from Hallie Preskill and Darlene Russ-eft’s Building Evaluation Capacity: 72 Activities for Teaching and Training<\/em><\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

What do Oreo cookies have to do with evaluation? In program evaluation, we want to know the extent to which a program is successful.\u00a0 If we\u2019re evaluating an activity\u00a0we\u2019re doing at our job, we might be trying to figure out if more money should be put towards future iterations of that activity or if it… Read More »<\/a><\/p>\n","protected":false},"author":2959,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[27,5],"tags":[],"class_list":["post-5795","post","type-post","status-publish","format-standard","hentry","category-blog","category-data-analysis"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p8ICUo-1vt","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts\/5795","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/users\/2959"}],"replies":[{"embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/comments?post=5795"}],"version-history":[{"count":14,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts\/5795\/revisions"}],"predecessor-version":[{"id":6506,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts\/5795\/revisions\/6506"}],"wp:attachment":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/media?parent=5795"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/categories?post=5795"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/tags?post=5795"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}