{"id":4591,"date":"2016-07-22T12:48:45","date_gmt":"2016-07-22T19:48:45","guid":{"rendered":"https:\/\/nnlm.gov\/evaluation\/blog\/?p=4591"},"modified":"2019-09-24T10:13:29","modified_gmt":"2019-09-24T17:13:29","slug":"from-qwerty-to-quality-responses-how-to-make-survey-comment-boxes-more-inviting","status":"publish","type":"post","link":"https:\/\/news.nnlm.gov\/nec\/2016\/07\/22\/from-qwerty-to-quality-responses-how-to-make-survey-comment-boxes-more-inviting\/","title":{"rendered":"From QWERTY to Quality Responses: How To Make Survey Comment Boxes More Inviting"},"content":{"rendered":"<p style=\"text-align: left\"><a href=\"\/neo\/wp-content\/uploads\/sites\/3\/2016\/07\/stockfresh_5986502_feedback-on-target_sizeS_56605c.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-4595\" src=\"\/neo\/wp-content\/uploads\/sites\/3\/2016\/07\/stockfresh_5986502_feedback-on-target_sizeS_56605c-178x300.jpg\" alt=\"Target that says feedback and has two darts stuck in it\" width=\"187\" height=\"315\" align=\"left\" hspace=\"10 width=\" \/><\/a><\/p>\n<p>The ubiquitous comment box. \u00a0It\u2019s usually stuck at the end of a survey with a simple label such as \u201cSuggestions,\u201d \u201cComments:\u201d or \u201cPlease add additional comments here.\u201d<\/p>\n<p>Those of us who write surveys over-idealistic faith in the potential of comment boxes, also known as open-ended survey items or questions.\u00a0 These items will unleash our respondents\u2019 desire to provide creative, useful suggestions! Their comments will shed light on the difficult-to-interpret quantitative findings from closed-ended questions!<\/p>\n<p>In reality, responses in comment boxes tend to be sparse and incoherent. You get a smattering of \u201chigh-five\u201d comments from your fans. A few longer responses may come from those with an ax to grind, although their feedback may be completely off topic.\u00a0 More often, comment boxes are left blank, unless you make the mistake of requiring an answers before the respondent can move on to the next item. Then you\u2019ll probably get a lot of QWERTYs in your blank space.<\/p>\n<p>Let\u2019s face it.\u00a0 Comment boxes are the vacant lots of Survey City. \u00a0Survey writers don\u2019t put such effort into cultivating them. Survey respondents don\u2019t even notice them.<\/p>\n<p>Can we do better than that?\u00a0 Yes, we can, say the survey methods experts.<\/p>\n<p>First, you have to appreciate this fact: open-ended questions ask <em>a lot<\/em> of respondents.\u00a0 They have to <em>create<\/em> a response. That\u2019s much harder than registering their level of agreement to a statement you wrote for them. So you need strategies that make open-ended questions easier and more motivating for the survey taker.<\/p>\n<p>In his online class <a href=\"https:\/\/www.udemy.com\/secrets-that-make-surveys-work\/learn\/v4\/overview\"><em>Don\u2019t (Survey)Monkey Around: Learn to Make Your Surveys Work<\/em><\/a>, \u00a0Matthew Champagne provides the following tips for making comment boxes more inviting to respondents:<\/p>\n<ul>\n<li><em>Focus your question. <\/em>Get specific and give guidance on how you want respondents to answer. For example, \u201cPlease tell us what you think about our new web site. Tell us both what you like and what you think we can do better.\u201d I try to make the question even easier by putting boundaries on how much I expect from them.\u00a0 So, when requesting feedback on a training session, I might ask my respondents to \u201cPlease describe one action step you will take based on what you learned in this class.\u201d<\/li>\n<\/ul>\n<ul>\n<li><em>Place the open-ended question near related closed-ended questions. <\/em>For example, if you are asking users to rate the programming at your library, ask for suggestions for <em>future <\/em>programs right after they rate the current program. The closed-ended questions have primed them to write their response.<\/li>\n<\/ul>\n<ul>\n<li><em>Give them a good reason to respond. <\/em>A motivational statement tells respondents how their answers will be used. Champagne says that this technique is particularly effective if you can explain how their responses will be used for their <em>personal<\/em> For example, \u201cPlease give us one or two suggestions for improving our references services.\u00a0 Your feedback will help our reference librarians know how to provide better service to users like you.\u201d<\/li>\n<\/ul>\n<ul>\n<li><em>Give them room to write.<\/em> You need a sizable blank space that encourages your respondents to be generous with their comments. Personally, when I\u2019m responding to an open-ended comment on a survey, I want my entire response to be in view while I\u2019m writing.\u00a0 As a survey developer, I tend to uses boxes that are about three lines deep and half the width of the survey page<\/li>\n<\/ul>\n<p>Do we know that Champagne\u2019s techniques work?\u00a0 In the Dillman et al.\u2019s classic book on survey methods, the authors present research findings to support Champagne\u2019s advice. Adding motivational words to the open-ended survey questions showed a 5-15 word increase in response length and a 12-20% increase in how many respondents\u2019 submitted answers.\u00a0 The authors caution, though, that you need to use open-ended questions sparingly for the motivational statements to work well. When four open-ended questions were added to a survey, the motivational statements worked better for questions placed earlier in the survey.<\/p>\n<p>I should add, however, to never make your first survey question an open-ended one.\u00a0 The format itself seems to make people close their browsers and run for the hills.\u00a0 I always warm up the respondents with some easy closed-ended questions before they see an open-ended item.<\/p>\n<p>Dillman et al. gave an additional technique for getting better responses to open-ended items: Asking follow-up questions.\u00a0 Many online software packages now allow you to take a respondent\u2019s verbatim answer and repeat it in a follow-up question.\u00a0 For example, a follow-up question about a respondent\u2019s suggestions for improving the library facility might look like this:<\/p>\n<p>\u201cYou made this suggestion about how to improve the library facility: <strong>\u2018The library should add more group study rooms.\u2019 <\/strong>Do you have any other suggestions for improving the library facility?\u201d [<em>Bolded statement is the respondents\u2019 verbatim written comment.]<\/em><\/p>\n<p>Follow-up questions like this have been shown to increase the detail of respondents\u2019 answers to open-ended questions.\u00a0 If you are interested in testing out this format, search your survey software system for instructions on \u201cpiping.\u201d<\/p>\n<p>When possible, I like to use an Appreciative Inquiry approach for open-ended questions. The typical Appreciative Inquiry approach requires two boxes, for example:<\/p>\n<ul>\n<li>Please tell us what you liked most about the proposal-writing workshop.<\/li>\n<li>What could the instructors do to make this the best workshop possible on proposal writing?<\/li>\n<\/ul>\n<p>People find it easier to give you an example rooted in experience.\u00a0 We are story tellers at heart and you are asking for a mini-story. Once they tell their story, they are better prepared to give you advice on how to improve that experience. The Appreciative Inquiry structure also gives specific guidance on how you want them to structure their responses. \u00a0The format used for the second question is more likely to gather actionable suggestions.<\/p>\n<p>So if you really want to hear from your respondents, put some thought into your comment box questions.\u00a0 It lets them know that you want their thoughtful answers in return.<\/p>\n<p>Source:\u00a0 The research findings reported in this post are from <em>Internet, Phone, Mail and Mixed-Mode Surveys: The Tailored Design Method<\/em> (4<sup>th<\/sup> ed.), by Dillman, Smyth, and Christian (Hoboken, NJ: John Wiley and Sons, Inc, 2014, pp 128-134.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The ubiquitous comment box. \u00a0It\u2019s usually stuck at the end of a survey with a simple label such as \u201cSuggestions,\u201d \u201cComments:\u201d or \u201cPlease add additional comments here.\u201d Those of us who write surveys over-idealistic faith in the potential of comment boxes, also known as open-ended survey items or questions.\u00a0 These items will unleash our respondents\u2019&#8230; <a href=\"https:\/\/news.nnlm.gov\/nec\/2016\/07\/22\/from-qwerty-to-quality-responses-how-to-make-survey-comment-boxes-more-inviting\/\">Read More &raquo;<\/a><\/p>\n","protected":false},"author":2959,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[13],"tags":[],"class_list":["post-4591","post","type-post","status-publish","format-standard","hentry","category-questionnaires-and-surveys"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p8ICUo-1c3","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts\/4591","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/users\/2959"}],"replies":[{"embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/comments?post=4591"}],"version-history":[{"count":1,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts\/4591\/revisions"}],"predecessor-version":[{"id":6465,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts\/4591\/revisions\/6465"}],"wp:attachment":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/media?parent=4591"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/categories?post=4591"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/tags?post=4591"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}