Measuring Allyship

When people think psychology, they may picture an old, white man with a pad and pencil talking to a person lounging on a Freudian type chair. When I tell people I am in graduate school for Industrial Organizational Psychology, I often get asked if 1) I either work with organizing construction materials (perhaps the industry part?) or 2) if I can read minds (I wish). Nope - psychology is all about statistics - the measurement of human thoughts, attitudes, and behaviors. At Mattingly Solutions, we take measurement seriously. In fact, it’s at the core of everything we do.

It might seem like measuring attitudes and behaviors is hard, and well, it is! It’s not quite the same as just pulling out a ruler to measure something concrete like how long the distance is between my house and my favorite brewery. In psychology, we have to take into account individual biological, psychological, and social factors that influence behavior - we have to tackle the abstract.

Good news: Our team are experts in the area of measurement; Dr. Victoria Mattingly, Chief Executive Officer, Sertrice Grice, Chief Consulting Officer, Kelsie Colley, Data Scientist, and I are all trained in collecting and analyzing data in the workplace. Notably, measurement is listed as a core competency area in the graduate training programs we attended (SIOP Education and Training Guidelines).

Measurement gives us the confidence to make data-driven changes that generate DEI ROI

Today I want to talk about measuring allyship. An ally is someone who uses their power and status to advocate and support for someone who is different from them in some meaningful way. On the surface, measuring allyship may appear easy - all you have to do is measure allyship behaviors, right? Well...who defines what an allyship behavior is? What if allyship behaviors aren’t perceived as allyship by all groups? Does having an ally identity matter? How do we measure things we can’t always see?

Check out our DEI Diagnostics!

Read for a deeper dive? Let’s talk briefly about what it means to create and validate a scale. It’s not as simple as just creating a list of survey questions and sending it out for participants to take. Instead, measurement-sound scientists read a TON of literature and begin to draw connections between theory and concepts that define allyship. At this stage, they will decide how they define allyship and how they will measure it. Then, researchers create a list of questions that they think would capture the attitudes and behaviors of allies (or whatever they are trying to measure). They may ask subject matter experts to review their list and make adjustments. Then, they may have a select group of participants take the scale. Once they have data, they will run statistical analyses to look and see if the study measures what they think it will measure.This could include a factor analysis, which tests whether there is one or more dimension to the scale, item difficulty and discrimination, which looks at the likeliness that someone high in allyship would agree with that statement, convergent and discriminant validity, which looks at the correlation between your measure on allyship and measures that you would expect to be or not be related. Reliability tests, such as test-retest or internal reliability would also be run. If the statistical tests show that your measure is actually measuring allyship, you’re on to something. And that's just the first part of the process!

TLDR: When you partner with Mattingly Solutions, you get a team of IO Psychologists equipped to evaluate and improve your DEI practices

We’re not just taking guesses as to what your DEI culture looks like and what you should do. We’re using measurement best practices to get data-driven, evidence based solutions to make change.

I did a little digging into the literature to see what measurement scales have been used and validated for allyship. In my search, I found two big topics of allyship measurement - race and sexual orientation. This struck me as strange; the literature did not often use the term ally or allyship when it came to supporting other areas of diversity, such as ability, parenthood, veteran status, age, or religion. While many allyship measures address only one topic (e.g., ally for sexual orientation; ally for racial justice) - I am proud to say Mattingly Solutions looks at the whole person and the complexities of our identities. Our measurement of allyship takes an intersectional approach, and dives deeper into our unique experiences at work.

I hope this gets you thinking:

• How should I be measuring DEI efforts and change?

• How can I grow my confidence in our current data-driven practices?

• What can we do better to ensure that we are accurately measuring what we want to measure?

• How can I grow my allyship skills? [Click here]

Leave your thoughts in the comments below, or feel free to shoot us an email at info@mattinglysolutions.com