Critique my "mom test" questions
I'm in the process of running an alpha for Fantasy Congress and want to learn more about the participants. I have two audiences:
- civics educators who want to use this in the classroom
- individuals who want to play with friends and family (like fantasy sports)
I've created a survey, but I want to make sure I'll get meaningful data from their responses. I've tried to incorporate principles from "the mom test", but it's harder than it looks 😅. Here's what I have so far (below). Most of them will be multiple choice. I've omitted the answers to keep the post concise. Any way these could be improved?
For educators:
- Do your students use computers to complete in class activities? (In this case, a computer is any device that accesses the web. Such as a laptop, desktop, or tablet.)
- Where did students access the computer the last time you incorporated them into class time? (This should not include computers used outside of school hours, such as a student's home computer used to complete homework.)
- What was the average ratio of students to computers the last time you incorporated them into class time?
- When was the last time you purchased a digital product (meaning a product that can only be accessed with a computer) for a class?
- Describe the last digital product you purchased for a class (if any).
- Does your school provide financial assistance to educators for class activities?
- How much do you spend personally on classroom activities annually?
And for individuals:
- What publications (if any) have you visited for news about US politics in the past week? (ex: New York Times, Economist, Vice, Huffington Post, etc.)
- When was the last time you watched a politically focused satirical news show? (ex: The Daily Show, Last Week Tonight with John Oliver, Full Frontal with Samantha Bee, etc.)
- What social media platforms have you used in the past 24 hours? (ex: Twitter, Facebook, Reddit, TikTok, etc.)
- What is your political affiliation?
- Some basic demographic questions about age, salary, and education
Your questions are great, but you missed something big.
The purpose of The Mom Test questions are for qualitative (not quantitative) research -- or plain language, to open conversations that let you explore and understand the life of the customer. (There's a section about Learn/Confirm in the book, which is relevant here.)
A big problem you'll hit is that you're using them in a survey.
In conversations, these questions will reveal things that really help you understand the situation (and your assumptions.) In a survey, especially a multiple choice one, you'll just get a bunch of averages and aggregates, which won't expose the true story.
Ah, fair enough. I plan to do interviews with people after the alpha, so I will try again then.
I agree with another comment that with a survey you will miss an opportunity to follow up based on what they answer.
But if you still want to do a survey I think some questions are too complicated and people need to work too hard to answer them.
For example this one
"What was the average ratio of students to computers the last time you incorporated them into class time?"
It requires one to:
Maybe you can split these questions. For example, something like
Yeah that's much simpler, thank you! Sometimes the words just don't come out right.
You are welcome, Allison