Measuring Allyship
Updated: Aug 3, 2021
When people think psychology, they may picture an old, white man with a pad and pencil talking to a person lounging on a Freudian type chair. When I tell people I am in graduate school for Industrial Organizational Psychology, I often get asked if 1) I either work with organizing construction materials (perhaps the industry part?) or 2) if I can read minds (I wish). Nope - psychology is all about statistics - the measurement of human thoughts, attitudes, and behaviors. At Mattingly Solutions, we take measurement seriously. In fact, it’s at the core of everything we do.
It might seem like measuring attitudes and behaviors is hard, and well, it is! It’s not quite the same as just pulling out a ruler to measure something concrete like how long the distance is between my house and my favorite brewery. In psychology, we have to take into account individual biological, psychological, and social factors that influence behavior - we have to tackle the abstract.
Good news: Our team are experts in the area of measurement; Dr. Victoria Mattingly, Chief Executive Officer, Sertrice Grice, Chief Consulting Officer, Kelsie Colley, Data Scientist, and I are all trained in collecting and analyzing data in the workplace. Notably, measurement is listed as a core competency area in the graduate training programs we attended (SIOP Education and Training Guidelines).
Measurement gives us the confidence to make data-driven changes that generate DEI ROI
Today I want to talk about measuring allyship. An ally is someone who uses their power and status to advocate and support for someone who is different from them in some meaningful way. On the surface, measuring allyship may appear easy - all you have to do is measure allyship behaviors, right? Well...who defines what an allyship behavior is? What if allyship behaviors aren’t perceived as allyship by all groups? Does having an ally identity matter? How do we measure things we can’t always see?
Read for a deeper dive? Let’s talk briefly about what it means to create and validate a scale. It’s not as simple as just creating a list of survey questions and sending it out for participants to take. Instead, measurement-sound scientists read a TON of literature and begin to draw connections between theory and concepts that define allyship. At this stage, they will decide how they define allyship and how they will measure it. Then, researchers create a list of questions that they think would capture the attitudes and behaviors of allies (or whatever they are trying to measure). They may ask subject matter experts to review their list and make adjustments. Then, they may have a select group of participants take the scale. Once they have data, they will run statistical analyses to look and see if the study measures what they think it will measure.This could include a factor analysis, which tests whether there is one or more dimension to the scale, item difficulty and discrimination, which looks at the likeliness that someone high in allyship would agree with that statement, conve