A Primer on Performing Meaningful Research

Editor’s note: A version of this post originally appeared on Perfect Daily Grind on April 19, 2017.

Curiosity is among my favorite human instincts. It drives innovation, encourages thinking outside the box, and is spurred by such a simple question a two-year-old can ask it: “Why?”

Of course, answering the why – whether in life in general or coffee in particular – is never as easy as asking. But we need curiosity in the coffee industry, whether we are producers, roasters, baristas, or consumers. It can spur improvements in processing methods, motivate new brewing and roasting equipment, or even reshape the supply chain.

So, let’s look at some strategies for more effective question asking and answering.

1. Be Specific

Simply put, asking “why” isn’t enough. Start asking more “how,” “which part,” and “to what extent” types of questions.

For example, as a followup to a question about why espresso flavors are affected by roast degree, ask “how do changes in heat application during a roast affect extraction percentage?” or “what part of the roast is most critical for perceived sweetness as espresso?”

By refining your query, you’ve taken an esoteric question and given it something specific, measurable, and real-world applicable. That’s the difference between idle wondering and active investigation – giving yourself the right question is key to action.

2. Take Stock & Minimize Variables

Before jumping into the question, you’ll need to pick apart the details. What do you already know? What do you think the answer is, based on your experience? What exactly are the gaps in your knowledge?

In our example above, we know that the first few minutes of a roast are mostly related to drying green coffee out, so we can focus on the later segments of a roast. Assuming you’re not a MIT grad with a firm grasp on the thermodynamic principles behind the Maillard Reaction, you’ll have to limit your research to whatever tools, observation skills, and reporting abilities are at your disposal.

Taking stock of your knowledge base also involves an important step in the Scientific Method, which is establishing a Control Group. Basically, you’ll need a standard against which to measure your experiments.

If this experiment involves experimental roast styles, for example, you’ll want to have a well-annotated standard roast against which to compare your experimental styles. Or perhaps your experiment is on the effect of brew ratio on perceived acidity. Starting with a standard ratio (maybe your default brew method) and comparing new methods against the original will enable effective differentiation of results.

Lastly, focus your research on as few variables as possible. Changing one or two factors (roast time or a single well-recorded heat adjustment, e.g.) you will help to eliminate confusion. The more variables, the more opportunities to make mistakes or misattributions.

For example, if you’re interested in figuring out the effect of fermentation time on a washed coffee, start with just two batches and don’t use different cultivars, batch weights, or yeast strains. Make everything in the experiment exactly the same (or as close as possible), except for the variable you’re examining. You can always repeat the experiment later with different variables.

3. Measure, Observe, Repeat

The scientific community is obsessed with repeatability of experiments, and for good reason. Without the ability to replicate results, research benefits no one. Repeatability makes theoretical ideas practical for everyone’s benefit, and serves as proof the conclusions are valid.

With that in mind, you’ll need to be a stickler for details. What is the exact amount of coffee you’re going to roast? What is your dose/yield ratio for espresso? What is the temperature of the brew water or the roaster or the drying tables? Measure everything… within reasonable limitations.

Then do it all again. Were you able to maintain consistent conditions? Were the results the same? If not you’ve found another “why” question to start investigating.

4. Analyze, Then Progress

What worked, what didn’t, and why do you think that those results returned the way they did? Are there data trends you can trace? Was there a definitive answer, or just some suggestions of a probability?

In a lot of cases, you should be prepared for foggy results that tend to raise more questions than they answer. That’s good. It means you’re digging in the right direction. Easy answers are rarely adequate, and can lead to overconfidence. New questions will guide your research further, slowly refining your results and giving more robust answers.

5. Guard Against Bias

There are few greater threats to meaningful research than personal bias. Predisposition towards a certain answer can take many forms – it’s normal to want to confirm your suspicions instead of upending presuppositions. There are even ways that our own minds will try to uphold pre-existing constructs without our conscious involvement. Not all bias is intentional.

One easy way to avoid confirmation bias is to involve a research partner. Two heads are almost always better than one. Keep your control and experimental samples blind, and keep an open mind, being reasonable with your expectations.

Blinding samples can be tricky – using codes can help. Sometimes creating a double-blind experiment where samples are coded and recoded to eliminate anyone’s knowledge of the answers is useful.

Managing expectations is crucial, because what works for you in your particular environment and circumstances may not work for everyone everywhere. Don’t be frustrated when an analysis doesn’t give you perfect results… dig deeper and be ready to admit the complicated nature of both experimentation in general, and coffee in particular.

Let’s say you settle on a conclusion, for example, after many rounds of testing, that roasting a dry-process Brazilian coffee more slowly through first crack produces sweeter espresso when you pull shots at a 2:1 ratio.

Even though you’ve isolated many variables (process, country of origin, a short timeframe during the roast, ratio of coffee to water, a single sensory element) there are still other factors that could be important. Implicit assumptions in your conclusion include the make/model of your roaster and espresso machine, the ambient temperature and humidity of your environment, the particular coffees tested, your personal preference in terms of flavor, the mineral content of the water, and many, many other factors.

Open-minded discussion and analysis, just like starting with good questions, will spur progress and encourage others to join in the investigation.