This article is written with organizations in mind that are running programs (rather than testing specific treatments). Determining an evaluation budget for community-based interventions, innovation initiativess, and operational processes are not as well documented from a cost perspective as the traditional medical clinical trial space is. We hope this article provides some guidance.
A lot of evaluation consulting firms and government agencies recommend 10%-15% (or more) of program budgets should be set aside for research and evaluation. That’s nuts. Few organizations can afford to set aside that much. It’s just unrealistic. That money should go towards programming, treatments and interventions. Proof Pilot solves this issue.
Reviewing Actual Spending
So instead of looking at best practices, we looked at real benchmarks. What are other organizations actually spending?
Philanthropic foundations tend to be very open, forthright and introspective about their work. This gives us the opportunity to see how they allocate evaluation resources. The diversity of what they fund (healthcare, education, workforce development etc), means what we learn has wide application across sectors.
In 2014, the Hewlett Foundation published an excellent report as it reviewed their own evaluation spending. Instead of going with the “best practice conventional wisdom,” they surveyed a group of peer organizations to see what percentage of program money they set aside for evaluation.
They found that most organizations spend between 1.5% and 7.5% of their program budget every year on research and evaluation. Smaller organizations spent a larger proportion given their relatively smaller grant amounts.
In 2009, the Evaluation RoundTable, a network of evaluation leaders at foundations across the US, Canada, and the UK conducted an assessment of actual spending. Like the Hewlett foundation, they found smaller organizations and initiatives spending a larger portion of their budget on research and evaluation (around 7.5%). Overall, across all organizations and programs reporting, 3.5% of budget is set aside for evaluation with a low of .3% and a high of 18%.
Today, a quick internet search of some household name foundations and programs show an increasingly strong commitment to evaluation and a committed budget set aside. For example, Oxfam International explicitly commits to invest “a minimum five percent (5%) of its program budgets to monitoring, evaluation, and learning by the end of the Oxfam Strategic Plan 2013–2019.” The benchmark Hewlett report suggested doubling the organization’s evaluation investment to 2% over three years.
Other institutions are less explicit about setting aside certain budgetary amounts, but their policies are clear. They show evaluations as an integral part of operations and resources and should be allocated accordingly. The Kellogg Foundation states, “we believe strongly that evaluation should be conducted not only to demonstrate that a project worked, but also to improve the way it works.” The Gates Foundation says, “our evaluation policy is rooted in our business model.”
ProofPilot Recommendation: 2% or less
ProofPilot’s primary mission is to democratize research and evaluation tools for small organizations. For smaller organizations 7.5% of program initiatives is a lot of money.
Thankfully, technology provides many ways to make this work. We’ve leveraged new ideas and techniques to bring down costs and set a pricing strategy that allows even very small operations to benchmark evaluation costs at 2% or less of program operations.
(preprinted from the ProofPilot blog with modifications)