{"id":5372,"date":"2017-04-07T12:42:51","date_gmt":"2017-04-07T19:42:51","guid":{"rendered":"https:\/\/nnlm.gov\/evaluation\/blog\/?p=5372"},"modified":"2019-09-24T10:13:28","modified_gmt":"2019-09-24T17:13:28","slug":"nonrandom-ideas-for-using-random-samples","status":"publish","type":"post","link":"https:\/\/news.nnlm.gov\/nec\/2017\/04\/07\/nonrandom-ideas-for-using-random-samples\/","title":{"rendered":"Nonrandom Ideas for Using Random Samples"},"content":{"rendered":"

I bet random sampling is something most of you learned about in your Intro to Statistics <\/em>class, t<\/em>hen never thought about again.\u00a0 Truthfully, many program evaluators seldom use random sampling.\u00a0 We often are working with smallish groups of program participants and can seek feedback from everyone (also known as conducting a census).<\/p>\n

Besides, even if you have access to contact information for a huge group of people (e.g., your membership, city, user base, fan club), why not invite everyone? Survey software packages usually charge a flat annual fee, so it costs the same to invite 80 or 8,000 people to fill out your questionnaire. In fact, with social media, you may even be able to post surveys at no cost.<\/p>\n

Today, I am here to give you examples of how you can use random sampling for pesky evaluation challenges. Then, I end with simple resources for creating your own random samples.<\/p>\n

Challenge #1: A convenience sample just isn\u2019t good enough<\/em><\/strong>. <\/em>If you invite all 10,000 people on a mailing list (aka your population) to complete a questionnaire and 3% respond, that is not<\/em> a census. It\u2019s a convenience sample. Those few respondents likely were energized by extreme positive or negative opinions about the topic. You have no idea the degree to which their opinions represent the majority. Convenience samples aren\u2019t necessarily useless. Often, you can corroborate findings and assess bias if you have other evaluation data for comparison.\u00a0 A better response rate, however, automatically increases faith in the validity of your findings.\u00a0A personalized communication process<\/a>\u00a0\u00a0is a key tactic for boosting response rate. It does not come without time cost. You must carefully review your email list. Are names spelled correctly? Are email addresses properly formatted? When email messages bounce, you should correct addresses and resend invitations. Nonrespondents should be contacted through personally written email messages or phone calls. Really late responders, if budget permits, should be mailed print copies of the questionnaires. A random sample, therefore, allows you to create a statistically valid shortlist of respondents and treat them with TLC.<\/p>\n

Challenge #2: You have too many questions you want to ask:<\/em><\/strong> A long questionnaire is another potential threat to a good response rate.\u00a0 If you have a significant number of people in your population, you can create several mini-questionnaires with a portion of your questions on each.\u00a0 You can then divide up your population randomly and send different questions to each group.<\/p>\n

Challenge #3: You are conducting a qualitative interviewing project on a controversial topic: <\/em>\u00a0<\/strong>Usually, random sampling is not recommended for qualitative interviewing projects. Purposeful sampling is more likely to provide a small sample of interviewees with the appropriate mix of experiences and viewpoints for your project. However, when subject matter is controversial, a random sampling method protects you from appearing to \u201ccherry pick\u201d participants who side with your perceived position. For example, your organization may want to implement changes that are getting mixed reactions from various influential stakeholders.\u00a0 You would use random sampling to demonstrate a good faith effort to represent a broad range of opinions.<\/p>\n

\"Spreadsheet<\/a><\/p>\n

Challenge #4: You want to collect evaluation information in a less-than-convenient manner.<\/em><\/strong>\u00a0 <\/em>It\u2019s true, some evaluation methods can be annoying.\u00a0 However, random sampling can limit the burden to you and those who help you gather data. \u00a0Suppose you want point-of-contact feedback from help desk customers. \u00a0You can randomly select a manageable number of weeks out of the 52 in a given year to collect user feedback. Maybe you want to systematically confirm the most popular usage times for your library computers. You could randomly choose different hours each day for a month or two to conduct and record a head count.<\/p>\n

Challenge #5: You want to analyze existing data and there\u2019s a lot of it.<\/em><\/strong> \u00a0<\/em>In this information age, we are surrounded by organizational and even public information that might be useful in our evaluation projects. \u00a0Social media posts, \u201ccomment box\u201d feedback, and training participants\u2019 suggestions can be sources of valuable information. These sources also may provide daunting amounts of data. Rather than wade through 50,000 public ideas of how to remodel your library, for example, it is acceptable to randomly select a more manageable portion of responses and analyze those.<\/p>\n

So, how do you pull a random sample?<\/p>\n

You first have to determine your sample size.\u00a0 For the qualitative examples described here, your time and resources will be your guide. For surveys conducted with larger populations, you can use an online calculator, such as this one from RAOSOFT<\/a>.\u00a0Then, to get your actual sample, you can put your list in Excel and use the RAND function to assigned a random number to individuals in the list and pull a random sample. Check out this video on how to pull a random sample using Excel<\/a>.<\/p>\n

You never know when your evaluation project may call out for a nice random sample. I have described just a few examples here. Happy sampling!<\/p>\n","protected":false},"excerpt":{"rendered":"

I bet random sampling is something most of you learned about in your Intro to Statistics class, then never thought about again.\u00a0 Truthfully, many program evaluators seldom use random sampling.\u00a0 We often are working with smallish groups of program participants and can seek feedback from everyone (also known as conducting a census). Besides, even if… Read More »<\/a><\/p>\n","protected":false},"author":2959,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":true,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[13],"tags":[],"class_list":["post-5372","post","type-post","status-publish","format-standard","hentry","category-questionnaires-and-surveys"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p8ICUo-1oE","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts\/5372","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/users\/2959"}],"replies":[{"embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/comments?post=5372"}],"version-history":[{"count":4,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts\/5372\/revisions"}],"predecessor-version":[{"id":6445,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/posts\/5372\/revisions\/6445"}],"wp:attachment":[{"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/media?parent=5372"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/categories?post=5372"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/news.nnlm.gov\/nec\/wp-json\/wp\/v2\/tags?post=5372"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}