This article originally appeared in the Chronicle of Philanthropy. Posted with permission of the Chronicle of Philanthropy.
Big data has stolen the hearts of many industries, and philanthropy is no exception.
Foundations rely on measurable results for good reason: They have limited resources and want to make sure their grants are being used as effectively as possible. But could it be that philanthropy’s emphasis on measurable results has detracted from more important goals that are not as easily quantified?
In the area where I focus most of my studies — figuring out ways philanthropic and other entities can invest in training workers and spurring economic growth — I frequently see the downside of focusing on results that are easy to measure. For example, because we want to improve economic opportunities for low- and moderate-income workers, nonprofits and community colleges train workers so they will qualify for better jobs. Easy to find measurable results here: We can count the number of working people who have advanced from, say, a job that pays $8 an hour to one that pays $10 an hour.
But these numbers can be misleading. First, the $8-an-hour jobs still exist; they’re just filled by other working people. Second, while $10 an hour is better than $8, is this enough?
The truth is that there are not enough good-quality jobs for all the people who want them. The data may tell us that we have helped some individuals, but it also shows that wages overall are not rising as fast as educational attainment and that there are far too many jobs that don’t pay a living wage.
Focusing on achieving specific performance measures can work against improving the quality of available job opportunities. Consider this real-world example: A nonprofit organization that seeks to help poor people get good jobs identified a local company that pays well and does not have high education requirements. The nonprofit offered to train new workers. The firm initially demurred, saying it does its own training. But the nonprofit brought in the local community college to do the training. Eventually, the company agreed and, in the end, was very happy with the outcome.
The performance data captured the following results: Twenty-five people took a three-month course for credit at the community college, and 24 completed it successfully. The nonprofit placed 15 people in jobs that paid, for many participants, nearly twice what their previous, minimum-wage jobs paid.
But what happened to the quality of the job opportunity itself? Are the people who fill these positions, whose economic advancement is the goal, really better off under this new arrangement?
Previously, when the company led its own training, it paid participants a training wage and offered an increase for those who successfully completed the program. Now participants in the college-run program are unpaid during the three-month training period and face increased financial challenges.
Additionally, the company no longer needs to give participants incentives, in the form of higher wages, to complete the program. It can hire the participants at the training wage. In effect, the wages attached to the company’s position have decreased, even if the new workers are paid more than in their previous jobs.
It should also be noted that there is no evidence the workers were substantially more disadvantaged than those the company would have hired under its own training program. Thus, taxpayers are now shouldering a training cost that had been borne by a private company, with no obvious public benefit.
How do we avoid these situations, in which the outcomes look good based on the data but the overall result is lower-quality jobs and public money displacing private investments?
To expand economic opportunities for low- and moderate-income workers, we need to involve everyone in the system — businesses, nonprofits, educational institutions, organized labor, government agencies, and, of course, philanthropy. We need to assess progress not only by data relating to individual workers; we must also keep an eye on larger overall changes.
Here, then, is the challenge: Systemic change is hard to measure. How do you quantify the strength and potential of relationships among leaders in various sectors? How do you capture the impact of conversations, training, and guidance that would eventually lead a CEO of a major company to invest in frontline workers, or open government leaders’ ears to the voices of workers? What data shows the value of efforts to change the way our society values work?
At a certain point, the pressure to measure can leave nonprofit leaders frustrated and even jaded when their work to meet performance goals seems insufficient in light of larger problems.
The good news for philanthropy is that it can take a longer view and invest in fundamental change. It can build a new approach to performance management that holds grantees accountable while recognizing that the most meaningful outcomes may not be fully within grantees’ control to deliver. Philanthropy has opportunities to learn about systemic change and to adopt a shared learning strategy.
Individual outcomes are important. At the end of the day, we want to help individuals. But doing so on a large scale will take more than just adding up individual outcomes — it will require looking at the larger picture and building strategies to address it.
Philanthropy needs to get comfortable with ambiguity in measurement, or else it will miss critical opportunities to make lasting change.
About the Author
Maureen Conway is vice president of the Aspen Institute and executive director of the Economic Opportunities Program.
Learn More
The Economic Opportunities Program advances strategies, policies, and ideas to help low- and moderate-income people thrive in a changing economy. Follow us on social media and join our mailing list to stay up-to-date on publications, blog posts, events, and other announcements.
