Diversity in Evaluation – It’s About All of Us

Posted by nnlmneo on March 18th, 2016 Posted in: Blog

---

Unless you’ve been living under a rock, you know that culture permeates everything we do, and that we live in a diverse society with lots of different cultures. Odds are good that no matter what your job is, you take into consideration issues of culture, diversity and inclusion. This applies to evaluation as it does everywhere else.

At the 2015 Summer Evaluation Institute, each attendee was given a copy of The American Evaluation Association’s Statement on Cultural Competence in Evaluation.  I was impressed that the document was frequently mentioned, because it was clear that the AEA felt that cultural competence was central to quality evaluation. As it says in the Statement, “evaluations cannot be culture free… culture shapes the way the evaluation questions are conceptualized which in turn influences what data are collected, how the data will be collected and analyzed, and how the data are interpreted.”

The Statement describes the importance of cultural competence in terms of ethics, validity of results, and theory.

• Ethics – quality evaluation has an ethical responsibility to ensure fair, just and equitable treatment of all persons.
• Validity – evaluation results that are considered valid require trust from the diverse perspectives of the people providing the data and trust that the data will be honestly and fairly represented.
• Theory – theories underlie all of evaluation, but theories are not created in a cultural vacuum. Assumptions behind theories must be carefully examined to ensure that they apply in the cultural context of the evaluation.

The Statement makes some recommendations for essential practices for cultural competence. I highly recommend reading all of the essential practices, but here are a few examples:

• Acknowledge the complexity of cultural identity. Cultural groups are not static, and people belong to multiple cultural groups. Attempts to categorize people often collapse them into cultural groupings that may not accurately represent the true diversity that exists.
• Recognize the dynamics of power. Cultural privilege can create and perpetuate inequities in power. Work to avoid reinforcing cultural stereotypes and prejudice in evaluation. Evaluators often work with data organized by cultural categories. The choices you make in working with these data can affect prejudice and discrimination attached to such categories.
• Recognize and eliminate bias in language: Language is often used as the code for a certain treatment of groups. Thoughtful use of language can reduce bias when conducting evaluations.

This may sound good, but how can it apply to the evaluation of your outreach project?

Recently, the EvergreenData Blog had two entries on data visualizations and how they can show cultural bias. In the first one, How Dataviz Can Unintentionally Perpetuate Inequality: The Bleeding Infestation Example, she shows how using red to represent individual participants on a map made the actual participants feel like they were perceived as a threat. The more recent blog post, How Dataviz Can Unintentionally Perpetuate Inequality Part 2, shows how the categories used in a chart on median household income contribute to stereotyping certain cultures and skew the data to show something that does not accurately represent income levels of the different groups.

Does it sometimes feel like cultural competence is too much to add to your already full plate of required competencies? This quote from the AEA Statement on Cultural Competence in Evaluation may be reassuring: “Cultural competence is not a state at which one arrives; rather, it is a process of learning, unlearning, and relearning.”

Save