The ProofPilot Blog - Design, Launch & Participant in Research Studies

Healing America’s Political Divide with Randomized Controlled Trials

If the United States itself is the Great Experiment, then the solutions to our society’s problems lie not in unpleasant political discourse — but in hundreds, or thousands of controlled experiments.

Like a lot of people, I find myself stunned by the anger and unpleasantness in this election cycle. It’s more combative, emotional and dramatic than a pivotal episode of the Real Housewives.

That distrust existed before this election season and isn’t going to go away on November 9th. It will fester and make constructive conversations impossible. In that environment, how are we going to collaborate to find solutions?

Maybe we don’t have to agree on solutions, but instead, agree on hundreds — maybe thousands of mini-experiments, run at a local level, by communities themselves.

Each experiment on various programs, services, and approaches could build a renaissance in programs to solve everything from poverty to gun control to education. The US, itself, after all is the Great Democratic Experiment. Thomas Jefferson, one of the founding fathers was a renaissance man. His home and his writing show a deep respect for the scientific method. We can move past disagreements and create a culture of scientifically valid experiments to determine what works to solve our societal problems.

When it comes to experimenting on what works in humans, the randomized controlled trial is the gold standard. In the tech sector, we know it as the split A/B test.

Businesses use this strategy to build success. Business leaders don’t argue. They run controlled experiments to figure out what is most effective. E-commerce heavyweight Amazon and pharma giant Pfizer have a culture of experimentation that makes them investor darlings. Amazon runs millions of experiments every day to determine what will get you to buy a product. Pfizer has a portfolio approach to its drug pipeline. Trials will show nine out of ten of those drugs won’t work (or are unsafe) and will never make it to market. The one where clinical trials show efficacy could become a blockbuster.

The RCT isn’t only for big pharma. The research technique is useful across many services, programs and tools meant to improve the human condition. Online marketers use it to determine what marketing message is most effective. Medical researchers use it to determine the safety and efficacy of health treatments. Policy researchers use it to determine what social programs have the greatest impact.

The concept is simple. Get a group of people together, randomize them into two groups. Introduce something to one group and not the other. Watch the results.

Obviously, they can get a lot more complex than that. The basic model can be extended to support very complex questions. Personalized Medicine, for example, stems from the reality that every patient is not the same. The genetics, environment, and behaviors of each person have a significant impact on what treatments work and what do not. Researchers are increasingly leveraging those differences in study designs. This means we realize what works isn’t always the same based on environment and personal background. Effective techniques to reduce opioid dependence in rural Maine, for example, may be very different than what works in urban South Central Los Angeles. We continually learn and build experiments that take various past successes into consideration and tailor them for unique environments and situations.

By creating a culture of experimentation, all we need to do is agree to give a policy, program or approach a try in a small pilot setting and to run an experiment. The results will speak for themselves. We end up making decisions on what works. Not based on political discourse.

For this approach to work, organizations who typically run studies need to cede control to the overall community. Academics at policy firms and universities usually run these complex studies. Not only do these organizations have unsustainable cost structures, they are precisely the institutions that many have come to distrust. Data from the Edelman Trust Barometer shows the trust gap between “informed elites” (including academics and policy researchers) and the “mass audience” has grown steadily since the Great Recession of 2008. There now appears to be a conviction within the mass population that these information elites are self-serving. So the findings that come from carefully crafted studies at Harvard or Stanford fall on deaf ears. Communities themselves need to run these studies. Resources and tools need to be available, so the results are credible.

Likewise, political populists and frustrated communities need to realize that the approach does not provide quick answers. It will take months and years for these experiments to produce results. Ceding to the temptation to release opinions early could bias participants, and make the whole effort invalid. Also note, when results are available, they certainly won’t fit into a 144 character tweet.

Not every divisive issue affecting America will be solved with a culture of controlled experiments. But agreeing to thousands, maybe tens of thousands of mini-experiment conducted at the local level by communities themselves will identify techniques that have scientifically valid evidence that they work to solve a good portion of what ails America.

By ProofPilot CEO, Matthew Amsden. Originally published at on October 20, 2016.

You'll Also Be Interested In

Other Stories in the ProofPilot Blog