Identifying Good Research

Data-driven decisions are key to effective strategic direction, but decisions are only as good as the research that grounds them. It is important to ensure that you’re using reliable research that meets your needs and addresses the questions you need to understand. Key points from The Informed Association: A Practical Guide to Using Research for Results (ASAE Association Management Press, 2013) provide guidance for understanding what makes good research and how to know it when you see it.

Identify What You Need

Every inquiry should start with the question, what information do I really need? This may seem simplistic, but it’s important. Having a firm grasp on the question you need answered or the problem you have to solve will simplify the process. Let’s say you’re facing declining member attendance at conferences. You may want to look into overall conference patterns and trends—for example, whether there has been an overall shift in priorities and preferences for attendees in programming, size, location, or innovation.

One way to figure out the right questions: Ask the appropriate stakeholders. A group of staff or volunteers or a focus group of members can identify and shape the questions to make sure you find the answers that move you forward.

Once you know what you’re looking for, you can identify the best source of information. The information you need might already exist. Another organization may have conducted a study or survey on the topic. Or your association may already collect the data needed to analyze the question. If not, you may have to create your own survey, either in-house or with a trusted partner.

Determine If the Source Is Reputable

You found research that seems to address your questions, but where did it come from? Who is funding and conducting the research? Are there potential conflicts of interest that could influence the results? For example, is it conducted by a heavily partisan outfit, or funded by a corporation with a vested interest in a particular outcome? While these scenarios wouldn’t necessarily prevent the production of reliable data, they certainly warrant stronger inquiry.

If you don’t know anything about the researcher or organization that produced the research, look at the body of knowledge that the researcher or the organization has produced. Look at who cites their work and in what context. If you are going to base important decisions about the direction of your organization on a particular study, you have to be able to trust the data as much as you want your members and your industry to trust you.

Consult Multiple Sources

It is not common for results to vary from survey to survey based on the sample size used, the questions asked, the year the data were collected, and the methodology. Because of this, it is strongly recommended that you consult multiple sources for the information you need to get the most balanced view of the data you need.

Choosing to use research in your strategic, organizational, financial, and operational planning is a smart move. Just as you would heavily vet a financial planner or industry consultant, so should you thoroughly investigate any data sources that you want to use. It requires some extra work, but having the peace of mind that you’re using the best available information when making important decisions makes the initial effort worthwhile.