Foundations Fail at Failing January 18, 2011

Michael Remaley is the director of Public Policy Communicators NYC and president of HAMILL REMALEY breakthrough communications.

"If you hit the bull's eye every time,
you've set the target too close."

I thought of this, one of my favorite aphorisms, at the Communications Network's annual conference last September when the Hewlett Foundation's Communications Director Eric Brown talked about his organization's "failed grantmaking" contest.  Hewlett's smart internal exercise forces each department to name one grant from its portfolio that did not meet expectations, think through and explain what went wrong and help the entire organization learn from its failure. 

This is a learning exercise that more foundations should consider adopting. But more than that, it is an important example of how Hewlett's leadership has set the tone for candor about the unavoidable truth of philanthropic experimentation: failure is part of the equation. 

It is no coincidence that Hewlett is also one of the few foundations that has talked publicly about initiatives that didn't live up to expectations. It is also no coincidence that Hewlett's profile on Glasspockets gives a good indication of its commitment to transparency.  I would assert that Hewlett's reputation for being one of the most innovative, thoughtful, and effective foundations is directly related to its transparency, willingness to publicly question its strategies, and forthrightness in discussing the limitations of its successes. And that reputation further enhances its ability to exert influence and make change.

The hard sciences learned the importance of sharing candid assessments of "failed" experiments centuries ago. In fact, scientists seem to treasure results that do not meet expected outcomes even more highly than those that confirm what is already believed to be true. 

I am hardly the first person to call upon foundations to talk more openly about failure, experimentation, and unexpected outcomes. (See list below.) Hewlett's Paul Brest seems to have really kickstarted the conversation in 2007 by writing and talking about his foundation's experiences. That was followed by Robert Giloth and Susan Gewirtz's seminal 2008 piece in Foundation Review, "Philanthropy and Mistakes: An Untapped Resource." Many others, including Bob Hughes, Larry Blumenthal, Edward Pauly, Grant Oliphant, and Sean Stannard-Stockton, have added important insights about the need for foundations to be more open about their lessons learned.  The conversation about failure and experimentation seemed to grow and deepen over the past three years.  So you might think that foundations would be making major changes in how they communicate about failure.  You would be wrong.

Foundations give a lot of lip service to supporting "experimentation" in social sciences. But you almost never hear them talking about outcomes that failed to meet expectations, and even more rarely, those that call their basic strategies into question. If foundations want to be real leaders in advancing social change, they must move past the endless happy-talk that makes every grant sound like a success. Instead, they should use their web sites to detail how they are evaluating their work and what they've learned from unexpected outcomes. 

A foundation sharing its experiences with grants gone wrong is still very much the exception.  Anyone who is on the receiving end of foundation annual reports and newsletters knows this is true.  But to substantiate my assertion, I decided to do a little systematic poking around.

I figured the 21 largest supporters of the Center for Effective Philanthropy (most of which are also supporters of Grantmakers for Effective Organizations) would be the foundations most attuned to the value of self-reflection, evaluation, and sharing results that defy expectations, and also those that would have budgets big enough to support substantial evaluation efforts. I spent many hours exploring the nooks of crannies of these foundations' web sites.  I looked at numerous publications and evaluation sections of the sites, and I searched each site on the terms failure, failed, unmet expectations, unmet objective, unmet goal, experimentation, mistake, lessons learned, and assessment.

What I found was that few foundations make it easy to learn from projects that didn't go as spectacularly as planned, let alone talk frankly about what has been learned from the shortcomings of foundation strategy or execution.   Many of the 21 foundations I examined made no mention at all of evaluation criteria and organizational outcomes, even though their association with CEP and GEO implies that they demand that kind of forthrightness from grantees. The majority of the foundation sites I examined had a few project evaluation reports scattered among other foundation supported research – and many of those evaluation reports were laudatory with pablum like "real collaboration is a challenge" tacked on at the end. 

Some of the best exceptions were Robert Wood Johnson Foundation, the William and Flora Hewlett Foundation, and the Wallace Foundation. Each of those foundations not only makes it easy to find many project evaluations that are balanced in presenting positive and negative outcomes along with what was learned through the process, but also present self-critical examinations of foundation strategy and progress as whole. It is also not a coincidence that each of those foundations' profiles on Glasspockets indicates a commitment to transparency demonstrated by making public an assessment of overall foundation performance.

But perhaps the best example – the foundation that gets the Gold Star for Succeeding in Failing – is the James Irvine Foundation. The evaluation section of its site describes their approach to evaluating grantee success and links to all of its individual evaluations of initiatives. It also links to a Foundation Assessment section that has foundation annual progress reports for the last four years.  These progress reports are exceptionally detailed and well-documented, as well as frank about successes and failures.  Irvine has also produced "Insights: Lessons Learned" publications with candid assessments of their experiences with collaborations and other grantmaking practices. A search of the Irvine site on "lessons learned" produces lots of useful and interesting evaluative information and insightful critical analysis.

We are all members of the social science community and contributors to the social experiment that is American philanthropy. We now have enough examples of foundations talking humbly about their shortcomings to know that such candor only accelerates social progress and enhances the reputations of those philanthropic leaders. We've seen no evidence that talking forthrightly about the real-world circumstances leading to failure damages nonprofits or the foundations involved, so I wonder why foundations seem so reluctant to take on this leadership role.

What has your organization learned from experiments that didn't meet expectations?

Selected Readings:
A Chronology of the Dialogue on Failure
and Experimentation in Philanthropy

• Center for Effective Philanthropy (unattributed). "Indicators of Effectiveness: A Call for Foundations: Understanding and Improving Foundation Performance." 2002.
• Paul Brest. "Evaluating Our Work. Hard Lessons about Philanthropy & Community Change," Commentary on The William and Flora Hewlett Foundation website. March 2007.
• Paul Brest.  "Hard Lessons about Philanthropy & Community Change: Reflections on The Neighborhood Improvement Initiative." March 2007.
• Stephanie Strom. "Foundations Find Benefits in Facing Up to Failures." The New York Times. July 26, 2007.
• Paul Brest and James E. Canales. "Let's Stop Reinventing Potholes." The Chronicle of Philanthropy. August 9, 2007
• Sean Stannard-Stockton. "The Poster Child for Failure in Philanthropy." Stanford Social Innovation Review blog. May 14, 2008.
• Robert Giloth, Ph.D., and Susan Gewirtz, Annie E. Casey Foundation. "Philanthropy and Mistakes: An Untapped Resource." Foundation Review, September 2008.
• Maisie O'Flanagan, McKinsey & Company; Jacob Harold and Paul Brest, William and Flora Hewlett Foundation. "The Nonprofit Marketplace: Bridging the Information Gap in Philanthropy." 2008.
• Sean Stannard-Stockton. "Anatomy of a Failed Grant." Tactical Philanthropy Advisors blog. March 25, 2009.
• Grant Oliphant with Susan Herr (Online Video Interview). "What's the Upside of Philanthropic Failure?" The Communications Network. December 2009.
• Grantmakers for Effective Organizations and Council on Foundations (unattributed). "Evaluation in Philanthropy: Perspectives from the Field." December 15, 2009
• Larry Blumenthal. "A Helpful Guide to Failure in Philanthropy. Use Carefully." Philanthropy News Digest, Commentary & Opinion. January 7, 2010
• Bob Hughes. "Can Failure Be the Key to Foundation Effectiveness?" Center for Effective Philanthropy blog. January 11, 2010.
• Grantmakers in Health (unattributed) "Taking Risks at a Critical Time." Essays written specifically for the 2010 GIH annual meeting. March 2010.
• Edward Pauly. "Philanthropy with Impact: A Guide to Evaluative Thinking for Foundations and Donors." A guide published by Forum for Active Philanthropy. 2010.
• Robert G. Hughes. "The Role of Failure in Philanthropic Learning." Book Chapter In: To Improve Health and Health Care XIII, pp.93-106. Publisher: Jossey-Bass. 2010
• Patricia A. Patrizi. "Death Is Certain, Strategy Isn't: Assessing RWJF's End-of-Life Grantmaking." Published in New Directions for Evaluation, by Wiley Online Library. Volume 2010, Issue 128, pages 47–68, Winter 2010.
• Ellie Buteau. "A higher bar for transparency, accountability." Philanthropy Journal. September 14, 2010.

— Michael Remaley

Comments

[Sheila Marsh]

Bravo to Michael Remaley and the truly transparent organizations he champions! We need much more accountability from those who invest other people's money in nonprofit efforts, and we need to be willing to use the truth we find to guide future work rather than just as a marketing tool to seek more funding. For further thoughts on how accountability is a good thing for nonprofits - including foundations - who are willing to engage in it, see:

http://www.peppergrasspartnerships.com/resources/partnerships-accountability-and-consequences/

Sheila Marsh

[Sheila Marsh]

Bravo to Michael Remaley and the truly transparent organizations he champions! We need much more accountability from those who invest other people's money in nonprofit efforts, and we need to be willing to use the truth we find to guide future work rather than just as a marketing tool to seek more funding. For further thoughts on how accountability is a good thing for nonprofits - including foundations - who are willing to engage in it, see:

www.peppergrasspartnerships.com/resources/partnerships-accountability-and-consequences/

Sheila Marsh

[Lori Bartczak]

This topic has long been an area of interest among the membership of Grantmakers for Effective Organizations. Some of the questions our members grapple with include, how do we apply what we've learned in a useful way? How do we encourage candor from our grantees? How do we speak publicly about failures without putting our grantees in compromising positions? In 2011, GEO plans to engage a cohort of our members in peer learning around the topic of leveraging failures. We will share the results of what we learn from this cohort. Funders interested in learning more about this work can contact Leonor Alfonso at alfonso@geofunders.org.

[ngogerty]

One simple way to do this is to pose a simple ratio for each foundation of successes. this eliminates the stigma of publicly highlighting failures. A 100% success ratio, would naturally be suspect, similar to winning a democratic election with 99.9% of the vote.

A simple success ratio would at least allow for the acknowledgement of failure and begin frank discussions. It is a simple question all donors should ask. What is your success ratio? the follow on questions from there would lead to interesting opportunities for learning and understanding. Failure isn't all bad if it is learned from or acknowledged as research into the untested.

sharing those results can help everyone understand what succeeds better.

[Gayle L. Gifford, ACFRE]

I'm a strong supporter of "open source" grant reporting. I think the whole NGO community would benefit greatly if final reports were much more broadly disseminated online so that researchers and others could readily find similar projects, talk to the practitioners about what worked and what didn't and explore the impact of projects and people over time. Imagine being able to track particular leadership across different organizations within a community over time. Or to look for cumulative impacts of different projects. A treasure trove for researchers which now is hidden away in file cabinets.

[Michael Remaley]

Sylvia,

Great to hear from you. Thanks so much for sharing the Gaining Perspective document. I am hoping to be doing some follow up research on this topic and your piece will be a very nice addition to the information I'm gathering.

best, -michael

[Bruce Trachtenberg]

For another view about talking about "foundation failure," check out this video interview with Grant Oliphant, president of the Pittsburgh Foundation: http://bit.ly/5bYsMA

[Sylvia Burgos Toftness]

The reports mentioned in this post were among models we examined when the Northwest Area Foundation initiated an effort to reflect on and experiences from a decade of work. We hope our just-released report "Gaining Perspective: Lessons Learned from One Foundation's Exploratory Decade" encourages conversation and the sharing of both stumbles and achievements among fellow funders. The full report can be downloaded at http://www.nwaf.org/content/Lessons.