Designing a survey might seem straightforward—just ask a bunch of questions, right? Well, not quite. If you want to gather meaningful, actionable insights, your survey needs to be rooted in a clear, well-thought-out hypothesis. Here’s how I approach survey design, with a focus on how hypotheses drive every decision in the process.

Don’t Copy Someone Else’s Wording

One of the biggest pitfalls in survey design is using questions from other surveys without considering whether they truly fit your specific goals. Your questions should be crafted to test your unique hypotheses, not just to gather general data.

A professor and a student once approached me with an exciting idea: they wanted to introduce a new product on campus, gather feedback, and iterate based on our university students' unique needs. They asked for my advice on crafting the perfect survey questions. Instead of handing them a set of "golden questions," I emphasized the importance of framing their questions and response options around their specific hypotheses. After all, there's no one-size-fits-all in survey design. However, if they were benchmarking against competitors or conducting a longitudinal study, keeping the wording consistent would be crucial. Fortunately, the professor was an English expert, so I had no doubt he could nail the wording with this approach in mind.

Always tailor your questions to your hypotheses. If you need to borrow wording, make sure it aligns perfectly with what you’re trying to learn.

Frame Questions and Response Options Around Your Hypotheses

A hypothesis-driven approach means that every question and every response option should serve a specific purpose in testing your assumptions. This ensures that the data you collect is directly relevant to the decisions you need to make.

Imagine you’re working with a startup focused on mental wellness. You hypothesize that users aged 18-24 would prefer receiving mindfulness tips through short, daily text messages rather than long-form emails. To test this hypothesis, you need to be strategic about your survey questions. Instead of asking a broad question like, “How do you prefer to receive wellness tips?” you craft questions directly linked to your hypothesis: “Would you find it helpful to receive a daily text message with a mindfulness tip?” and “How often do you read emails that contain wellness advice?” This targeted approach would allow you to gather data that confirms whether your hypothesis holds true. If the data supports your assumptions, you could confidently implement a text-based tip service, potentially making it one of the app’s most popular features. By aligning your questions with your hypotheses, you ensure that the data you collect directly informs your product decisions.

Design your survey to test specific hypotheses, ensuring that your questions and response options are aligned with what you need to discover.

Tailor the Survey to the Product Phase

Your hypotheses will naturally evolve as your product moves through different stages of development. The questions you ask in your survey should reflect these evolving hypotheses, shifting focus as you move from discovery to post-launch phases.

Imagine you're working on a new mobile app aimed at helping users track their daily habits. In the early discovery phase, your hypothesis is that users struggle to maintain new habits because they find the tracking process tedious. To explore this, you design a survey with open-ended questions like, “What challenges do you face when trying to track your daily habits?” and “What features would make a habit-tracking app more useful for you?” These broad questions help you gather a wide range of insights that confirm your hypothesis and highlight specific pain points. Fast forward to the post-launch phase, and your focus shifts to refining the app based on user feedback. Now, your hypothesis is that users might not be engaging with the app’s reminder feature because it’s too rigid. To test this, you craft more specific survey questions like, “How often do you use the reminder feature?” and “How would you rate the flexibility of setting reminders?” By aligning your questions with the product development phase, you gather the right data at the right time, leading to targeted improvements that enhance the user experience.

Tailor your survey to the product development stage you’re in, and let your hypotheses guide the shift in focus from broad exploration to targeted validation.

Do You Really Need a Survey?

Your hypotheses will naturally evolve as your product moves through different stages of development. The questions you ask in your survey should reflect these evolving hypotheses, shifting focus as you move from discovery to post-launch phases.

While working at Mercari, we were exploring ways to improve the seller experience on the platform. The team initially suggested launching a survey to gather feedback from sellers about their listing experience. However, given our goal to deeply understand the challenges sellers faced, I encouraged the team to consider qualitative interviews instead of a survey. By conducting one-on-one interviews with a diverse group of sellers, we were able to gain richer insights into their pain points—like the frustration of uploading multiple photos on mobile devices or the complexity of setting prices. These insights were far more detailed and actionable than what a survey could have provided, and they directly informed several key product enhancements that ultimately made the listing process smoother and more user-friendly.

Tailor your survey to the product development stage you’re in, and let your hypotheses guide the shift in focus from broad exploration to targeted validation.

Survey Best Practices

Even when following best practices—like keeping your survey short, piloting with a test group, and ensuring clarity—your hypotheses should remain the driving force behind your design decisions. This ensures that the data you collect is directly relevant to your research goals.

During my time at Mercari, we were preparing to launch a new feature aimed at improving the buyer experience. To ensure we gathered meaningful feedback, we designed a post-launch survey to capture users' impressions. Drawing on best practices, I emphasized the importance of keeping the survey concise to avoid fatigue, which can lead to lower response rates. We limited the survey to just five key questions, each carefully crafted to test our hypotheses.

Best practices are essential, but they should always be implemented in a way that serves your hypotheses. Keep your research goals in mind to ensure your survey is as effective as possible.

Always Keep Your Hypotheses Front and Center

Designing a survey is about more than just gathering data—it’s about gathering the right data to test your hypotheses. By keeping your hypotheses front and center throughout the design process, you can ensure that your survey delivers insights that truly matter.